Framing a Learning Problem
Prologue & Ch. 1
||Sign up for Piazza||Optional reading (but strongly suggested)
The Discipline of Machine Learning
||Labor day (no class)
Learning & Decision Trees
||Handout: Decision Trees|
||Assignment 1 out|
||Linear Regression and Gradient Descent||LfD 1.1, 3.1-3.2.1;
Optional linear algebra review: Barber 29.1.1-29.1.9
|Alternative reading if you don't
have LfD: Flach 7.1-7.2
(see above slides),
Perceptron Linear Classifier,
||Alternative reading if you
have LfD: Flach 7.4
VC Dimension & Generalization Bounds
|Learning Theory notes, LfD 3.2.2||
||For extra help on learning theory,
read LfD Ch. 2
||Support Vector Machines||Flach 7.3; LfD 3.4
[HW1 Submission Instructions]
Alternative reading if you don't
have LfD: Flach 7.5
||Flach 11.2, LfD 4.3
||Probability Review||Handout: Probability; Generative
Model notes (Section 2 only)
extra help with naive Bayes,
read Flach 9.2
||Fall break starts tomorrow - enjoy!|
||Bayesian Methods (continued), Review||skim Flach
Generative Model notes
|Assignment 2 due
[Assignment 2 Skeleton]
[HW2 Submission Instructions]
Classification & Evaluation
||Neural Nets & Representation Discovery||Handout: Deep
||For more detail on deep learning, see: Bengio article (optional reading)|
||Deep Learning||Flach 8.4-8.6,
Sutton & Barto Ch.
||Assignment 3 due
[Assignment 3 Skeleton]
[HW3 Submission Instructions]
||Reinforcement Learning (continued)
Sutton & Barto Ch.
||No Class (Dr. Eaton out
|Assignment 4 due
[HW 4 README]
[HW4 Submission Instructions]
||No Class (Friday class
Machine Learning for Big Data
Report and Summary Slides Due
||Final Exam and Solution Key|
Eric Eaton, Ph.D.E-mail: -- Make certain you put "[CIS 419]" or "[CIS 519]" at the start of the subject line to ensure a quicker response.
Chris Clingerman (5th year PhD student, CIS)
Xiaoxiang Hu (Masters student, CIS)
Vishnu Purushothaman Sreenivasan (Masters student, CIS)
Machine learning has been essential to
the success of many recent technologies, including autonomous
vehicles, search engines, genomics, automated medical diagnosis,
image recognition, and social network analysis, among many others.
This course will introduce the fundamental concepts and algorithms
that enable computers to learn from experience, with an emphasis
on their practical application to real problems. This course
will introduce supervised learning (decision trees, logistic
regression, support vector machines, Bayesian methods, neural
networks and deep learning), unsupervised learning (clustering,
dimensionality reduction), and reinforcement
learning. Additionally, the course will discuss evaluation methodology and recent applications of machine learning, including large scale learning for big data and network analysis.
Course Website: http://www.seas.upenn.edu/~cis519/
noon to 1:30 pm
Location: Skirkanich Hall Auditorium (Berger Auditorium)
Due to overwhelming demand, Penn is
offering two different machine learning courses this
semester: CIS 419/519 (Introduction to Machine Learning) and
CIS 520 (Machine Learning). This section briefly describes
the differences between these courses.
CIS 419/519 Introduction to Machine
Learning (this course!) is a new introductory-level course
in machine learning (ML) with an emphasis on applying ML
techniques. The course is cross-listed between undergraduate (419)
and graduate (519) versions; the graduate course 519 has somewhat
different requirements as described below. CIS 419/519 is
intended for students who are interested in the practical
application of existing machine learning methods to real
problems, rather than in the statistical foundations and
theory of ML covered in CIS 520.
CIS 520 Machine Learning is a more mathematically rigorous course in statistical machine learning that provides the background necessary to design and use new ML algorithms. Consequently, CIS 520 requires students to have basic knowledge of linear algebra (matrices, eigenvectors, etc.) It uses Matlab and is said to require a lot of work, but prepares students to conduct ML research.
CIS 519 is NOT a prerequisite for CIS 520. However, it makes little sense to take CIS 519 after having already taken CIS 520. You certainly may take CIS 419/519 first and then later take CIS 520.
Essentially, you should take CIS
And, you should take CIS 520 if you're confident in your
mathematical background and:
Students registered for the graduate version of this course (CIS
519) will be required to complete additional work throughout the
semester. This work will include additional components to
the homework, additional requirements on the course project, and
(possibly) different or additional questions on the exams.
Since the two versions have different requirements, you cannot
complete the course as CIS 419 and later petition to have it
changed to CIS 519 for graduate credit; if you're considering
changing this course to CIS 519 for graduate credit, you should
register for the graduate version now.
CIS 419/519 Course Reading Packet
This is a collection of readings that will be used throughout the course. Instructions for obtaining the course readings will be discussed at the first class.
Learning From Data by
Y. S. Abu-Mostafa, M. Magdon-Ismail, and H.T. Lin.
Learning: The Art and Science of Algorithms That Make
Sense of Data by Peter Flach
An electronic copy is available
via the Penn library.
For a more advanced treatment of machine learning topics, I would
recommend one of the following books:
Attendance and active participation are
expected in every class. Participation includes asking questions,
contributing answers, proposing ideas, and providing constructive
As you will discover, I am a proponent of two-way communication and I welcome feedback during the semester about the course. I am available to answer student questions, listen to concerns, and talk about any course-related topic (or otherwise!). Come to office hours! This helps me get to know you. You are welcome to stop by and chat. There are many more exciting topics to talk about that we won't have time to cover in-class.
Whenever you e-mail me, be sure to use a meaningful subject line and include the phrase "[CIS 419]" or "[CIS 519] at the beginning of the subject line. Your e-mail will catch my attention and I will respond quicker if you do this. I make an effort to respond to e-mails within 24 hours on weekdays and 48 hours on weekends. However, unless it is a private matter, you should be posting your questions/issues to Piazza.
Although computer science work can be intense and solitary, please stay in touch with me and the TAs, particularly if you feel stuck on a topic or project and can't figure out how to proceed. Often a quick e-mail, face-to-face conference, or Piazza post can reveal solutions to problems and generate renewed creative and scholarly energy. It is essential that you begin assignments and projects early, since we will be covering a variety of challenging topics in this course.
Your grade will be based upon four
homework assignments, two exams, and a course project.
Assignments must be submitted according to the assignment
At the end of the semester, final grades will be calculated as a weighted average of all grades according to the following weights:
||40% (10% each)|
The project grade will be broken down further in the Project Description.
Incomplete grades will be given only for verifiable medical
illness or other such dire circumstances.
All graded work will receive a percentage grade between 0% and
100%. Here is how the percentage grades will map to final
letter grades; percentages are not rounded:
||77% <=||C+ (2.3)|
|93% <=||A (4.0)||73% <=||C (2.0)|
|90% <=||A- (3.7)||70% <=||C- (1.7)|
|87% <=||B+ (3.3)||67% <=||D+ (1.3)|
|83% <=||B (3.0)||60% <=||D (1.0)|
|80% <=||B- (2.7)||< 60%
There will be two exams in this course. The exams will be
closed-book and closed-notes. They will cover material from
lectures, homeworks, and assigned readings (including topics not
discussed in class). So, keep up with those readings!
I want to encourage you to discuss the material and work together
to understand it. Here are my thoughts on collaborating with other
The readings and lecture topics are group work. Please discuss the readings and associated topics with each other. Work together to understand the material. I highly recommend forming a reading group to discuss the material -- we will explore many challenging ideas and it helps to have multiple people working together to understand them.
It is fine to discuss the topics covered in the homeworks, to discuss approaches to problems, and to sketch out general solutions. However, you MUST write up the homework answers, solutions, and programs individually. You are not permitted to share specific solutions, mathematical results, program code, knowledge representations, experimental results, etc. If you made any notes or worked out something on a white board with another person while you were discussing the homework, you shouldn't use those notes while writing up your answer, however tempted you may be to do so.
You are fully permitted to (and should!) discuss projects
with members of your team. I also encourage you to work
outside of your team to understand the other topics in the
Exams and papers, of course, must be your own individual work.
If you have any questions as to what types of collaborations are
allowed and which are dishonest, please ask me before you
make a mistake.
I have no problem with you using computers or tablets to take
notes or consult reference materials during class. Tempting
though it may be, please do not check e-mail or visit websites
that are not relevant to the course during class. It is a
distraction, both for you and (more importantly) for your fellow
classmates. Please silence your phones and computers when
you enter class.