Lectures

Date	Class	Due	Notes
Aug 29	Introduction		Introduction to Class, Motivation, Logistics
Aug 31	ML Review I		Supervised Learning, Binary Classifiers
Aug 31	ML Review II		Loss Minimization, SVM and Logistic Regression
Sep 05	Multiclass Classification		Multiclass Classification - Global and Local Views
Sep 07	Self Study (No Class)		Required Reading S. Har-Peled, D. Roth and D. Zimak, Constraint Classification for Multiclass Classification and Ranking, NIPS 2003 Y. Crammer and T. Singer, Ultraconservative Online Algorithms for Multiclass Problems, JMLR 2003 L. R. Rabiner, A Tutorial on Hidden Markov Models and Selected Applications in Speech Recognition IEEE 1989. Try running Illinois-SL and its examples for next lecture. HW0 Solution Released.
Sep 12	Structured Prediction Basics		Required Reading Ming-Wei Chang, Structured Prediction with Indirect Supervision (Thesis Chapter 2) Hal Daume III, Practical Structured Learning Techniques for Natural Language Processing (Thesis Chapter 2) M. Collins, Discriminative Training for Hidden Markov Models Theory and Experiments with Perceptron Algorithms EMNLP 2002 Introduction to Structured Prediction, Structured Perceptron
Sep 14	Self Study (No Class)		Required Reading V. Punyakanok and D. Roth, The Use of Classifiers in Sequential Inference, NIPS 2001 A. McCallum, D. Freitag, and F. Pereira, Maximum entropy Markov models for information extraction and segmentation ICML 2000 C. Sutton and A. McCallum, Introduction to Conditional Random Fields for Relational Learning, Statistical Relational Learning, 2007
Sep 19	Discussion led by Dan.		Refer Introduction to Structures HMM, MEMM Sequences for today's lecture. Paper Assignments for Presentation will be released by tomorrow evening.
Sep 21	Self Study (No Class)		Required Reading I. Tsochantaridis, T. Hofman, T. Joachims, Y. Altun, Large Margin Methods for Structured and Interdependent Output Variables ICML 2004 D. Roth and W. Yih, Global Inference for Entity and Relation Identification via a Linear Programming Formulation. Introduction to Statistical Relational Learning, 2007 The longer (JMLR) version of the Tsochantaridis paper is in the list of papers. You should read it after reading the ICML version for better understanding.
Sep 26	Structured Learning	1st Critical Survey Due	Presentations Structured Perceptron with Inexact Search slides Conditional Random Fields Probabilistic Models for Segmenting and Labeling Sequence Data slides
Sep 28	Structured Learning (Contd.)	Project Proposals Due	Presentations Cutting-Plane Training of Structural SVMs slides Pranking with Ranking slides
Oct 03	Learning and Inference with Constrained Output		Presentations Global Inference for Entity and Relation Identification via a Linear Programming Formulation slides Learning and Inference over Constrained Output slides Discussion Learning with Structured Inference
Oct 05	Fall Break
Oct 10	Introduction to ILP Inference		Presentations Integer Linear Programming Inference for Conditional Random Fields slides Discussion Structured Learning paradigm and ILP Inference.
Oct 12	Latent Representations		Presentations Guiding Semi-Supervision with Constraint-Driven Learning slides Discussion Constrained EM, Background on EM and Posterior Reg.
Oct 17	Project Progress Discussion	2nd Critical Survey Due	Technical Discussion
Oct 19	Latent Representations (Contd.)		Presentations Learning Structural SVMs with Latent Variables slides Structured Output Learning with Indirect Supervision slides
Oct 24	Latent Representation Technical Discussion		Discussion Latent Representations and Implementing Latent-SSVM in Illinois-SL
Oct 26	No Class		Please come to project meeting on 27th October
Oct 31	ILP Applications		Presentations On Amortizing Inference Cost for Structured Prediction slides Relational Inference for Wikification slides Discussion ILP Applications
Nov 02	Mid-Semester Project Presentation
Nov 07	Search-based Structured Prediction		Presentations Search Based Structured Prediction slides
Nov 09	Search-based Structured Prediction (contd.)		Presentations S-MART Novel Tree-based Structured Learning Algorithms Applied to Tweet Entity Linking Learning to Search Better than Your Teacher
Nov 14	Structured Sparsity and Optimization	3rd Critical Survey Due	Presentations Structured Sparsity in Structured Prediction Discussion Sparsity as Structure
Nov 16	Deep Learning		Presentations A Unified Architecture for Natural Language Processing Deep Neural Networks with Multitask Learning Sequence-to-sequence learning as beam-search optimization
Nov 21	Thanksgiving Break
Nov 23	Thanksgiving Break
Nov 28	Deep Learning (contd.)		Presentations Segmental Recurrent Neural Networks Structured Attention Networks
Nov 30	Project Progress Discussion		Technical Discussion
Dec 05	Project Presentation
Dec 07	Project Presentation
Dec 12	No Class	4rd Critical Survey Due