Skip to main content

Introduction (Lecture 1)

General: ML in NLP

The following two papers are here for historical reasons. These are survey papers that describe the state of the art in Machine Learning for NLP in 1999 and 2005.

Generative and Discriminative Models

Multiclass

Basic Structured Models: Sequential Models

Background

Inference with Classifiers

CRF

Structured Perceptron

BONUS: To learn how to efficiently implement averaged perceptron (without storing weight vectors), refer Fig 2.3 on page 19 in Hal Daume’s thesis.

SVM

Constrained Conditional Models

Constraint-based Models

BONUS: To learn how to convert boolean constraints to ILP constraints, refer,

Training Paradigms

Training Paradigms: Constraint-based Models

Distributed Output Representations

Applications

Unsupervised Learning and Indirect Supervision

Constraint-Driven Learning

Latent Variables

Indirect Supervision

Inference

Inference

Search Based Inference

Deep Learning

Applications