CIS 620 Spring 2005

Machine-Learning Models and Algorithms for Structured Data

Fernando Pereira

Many important problems in machine learning involve implicitly or explicitly structured data:

• Classify hyperlinked Web pages
• Find genes in genetic sequence data
• Parse natural language text
• Model the probability of speech transcriptions
• Segment images
• Find the correferences in natural-language text

Important: presentation schedule

This edition of CIS 620 will be a lecture course on learning techniques for structured data. We will start with a review of fundamental prerequisites

• Introduction 1/11 PDF
• Linear binary classification and large margin 1/13-1/18 lecture slides; homework 1; homework data
  • A variety of additional tutorial material on support vector machines and kernel methods is listed here. The following are recommended
    • The Burges tutorial article
    • Schölkopf's NIPS tutorial slides
    • Another tutorial article
    They are all a bit more dense mathematically than the lectures, and they cover additional material, but you should at least skim them to reinforce the lecture material.
• Logistic regression and maximum-entropy models 1/18-1/20 lecture slides
• Undirected graphical models
  • Definitions and Markov-Gibbs equivalence 1/ 20-1/25 lecture slides
• Simple presentation of the Markov-Gibbs equivalence
• Inference in factor graphs 1/27 lecture slides
  • Introduction to factor graphs
• Homework 2
• A teeny bit of convex optimization

We'll then proceed to the main topics:

• Sequence models: HMMs, conditional random fields 2/1-2/8 lecture slides homework 3
• Probabilistic and weighted grammars 2/15-2/17 lecture slides
• Case-factor diagrams 2/22 lecture slides
• Max-margin networks Ben Taskar's thesis
• Structured object kernels (strings, trees, DAGs) Vishy's string/graph kernels Cortes+Mohri's rational kernelslectures/rat.pdf
• Methods and applications of Markov random fields
• Collective classification in graphs
• Semi-supervised learning in graphs

Coursework will include:

• A detailed topic presentation in class, including preparation of slides/handouts with main concepts, equations, algorithms, and theorems.
• Several homeworks that will test concepts or attempt to reproduce standard algorithms or results in the literature on modestly-sized datasets.