# ECE544: Pattern Recognition (Fall 2020)

## Course Information

The goal of Pattern Recognition is to find structure in data. In this course we will cover three main areas, (1) discriminative models, (2) generative models, and (3) reinforcement learning models. In particular we will cover the following: linear regression, logistic regression, support vector machines, deep nets, structured methods, learning theory, kMeans, Gaussian mixtures, expectation maximization, VAEs, GANs, Markov decision processes, Q-learning and Reinforce.**Pre-requisites:**Probability, linear algebra, and proficiency in Python.

**Recommended Text:**(1) Machine Learning: A Probabilistic Perspective by Kevin Murphy, (2) Deep Learning by Ian Goodfellow and Yoshua Bengio and Aaron Courville, (3) Pattern Recognition and Machine Learning by Christopher Bishop, (4) Graphical Models by Nir Friedman and Daphne Koller, and (5) Reinforcement Learning by Richard Sutton and Andrew Barto.

**Course Deliverables:**

(1) One course presentation (see Courseplan

**Lecture Group ID**column for your team members)

(2) One project (see Courseplan

**Project Team ID**column for your team members)

**Grading:**

1/3 class presentation, 1/3 project, 1/3 final; + 1/3 bonus for really impressive projects

Grading policy is subject to change.

**TA Hours:**

Time: Friday at 5pm.

**Final Exam:**December 8, 11am-12:30pm.

## Instructor & TAs

### Alexander Schwing

**Instructor**

Email: aschwing[at]illinois.edu

Office Hour: Friday at 8am

Website: [link]

### Class Time & Location

Class Time: Tuesday, Thursday 11:00AM-12:15PM## Lectures

The syllabus is subject to change.

Event | Date | Description | Materials | Pre-Recording | Recording |
---|---|---|---|---|---|

Lecture 1 | Aug. 25 | Introduction (Nearest Neighbor) | [Slides] [Slides Split] | [PreRec] | [Rec] |

Lecture 2 | Aug. 27 | Linear Regression | [Slides] [Slides Split] | [PreRec] | [Rec] |

Lecture 3 | Sep. 1 | Logistic Regression | [Slides] [Slides Split] | [PreRec] | [Rec] |

Lecture 4 | Sep. 3 | Optimization Primal | [Slides] [Slides Split] | [PreRec] | [Rec] |

Lecture 5 | Sep. 8 | Optimization Dual | [Slides] [Slides Split] [Team Slides] | [PreRec] | [Rec] |

Lecture 6 | Sep. 10 | Support Vector Machine | [Slides] [Slides Split] [Team Slides] | [PreRec] | [Rec] |

Lecture 7 | Sep. 15 | Multiclass Classification and Kernel Methods | [Slides] [Slides Split] [Team Slides] | [PreRec] | [Rec] |

Lecture 8 | Sep. 17 | Deep Nets 1 (Layers) | [Slides] [Slides Split] [Team Slides] | [PreRec] | [Rec] |

Lecture 9 | Sep. 22 | Deep Nets 2 (Backpropagation + PyTorch) | [Slides] [Slides Split] [Team Slides] | [PreRec] | [Rec] |

Lecture 10 | Sep. 24 | Ensemble Methods (Boosting/Random Forest/Deep Nets) & Regularization/Cross-Val | [Slides] [Slides Split] [Team Slides] | [PreRec] | [Rec] |

Lecture 11 | Sep. 29 | Structured Prediction (exhaustive search, dynamic programming) | [Slides] [Slides Split] [Team Slides] | [PreRec] | [Rec] |

Lecture 12 | Oct. 1 | Learning Theory | [Slides] [Slides Split] [Team Slides] | [PreRec] | [Rec] |

Lecture 13 | Oct. 6 | Review | [Slides] [Slides Split] | [Rec] | |

Lecture 14 | Oct. 8 | PCA, SVD | [Slides] [Slides Split] [Team Slides] | [PreRec] | [Rec] |

Lecture 15 | Oct. 13 | k-Means | [Slides] [Slides Split] [Team Slides] | [PreRec] | [Rec] |

Lecture 16 | Oct. 15 | Gaussian Mixture Models | [Slides] [Slides Split] [Team Slides] | [PreRec] | [Rec] |

Lecture 17 | Oct. 20 | Expectation Maximization | [Slides] [Slides Split] [Team Slides] | [PreRec] | [Rec] |

Lecture 18 | Oct. 22 | Hidden Markov Models | [Slides] [Slides Split] [Team Slides] | [PreRec] | [Rec] |

Lecture 19 | Oct. 27 | Variational Auto-Encoders | [Slides] [Slides Split] [Team Slides] | [PreRec] | [Rec] |

Lecture 20 | Oct. 29 | Generative Adversarial Nets | [Slides] [Slides Split] [Team Slides] | [PreRec] | [Rec] |

Election | Nov. 3 | Election Day | |||

Lecture 21 | Nov. 5 | Review | [Slides] [Slides Split] | [Rec] | |

Lecture 22 | Nov. 10 | Autoregressive Methods | [Slides] [Slides Split] [Team Slides] | [PreRec] | [Rec] |

Lecture 23 | Nov. 12 | MDP | [Slides] [Slides Split] [Team Slides] | [PreRec] | [Rec] |

Lecture 24 | Nov. 17 | Q-Learning | [Slides] [Slides Split] [Team Slides] | [PreRec] | [Rec] |

Lecture 25 | Nov. 19 | Review/Policy Gradient, Actor-Critic | [Slides] [Slides Split] | [PreRec] | [Rec] |

Fall Break | Nov. 24 | Fall Break | |||

Fall Break | Nov. 26 | Fall Break | |||

Project | Dec. 1 | Project Presentations | [Rec] | ||

Project | Dec. 3 | Project Presentations | [Rec] | ||

Exam | Dec. 8 | Final Exam |