Deep learning plays a key role in modern AI. This course introduces the theory, algorithms, and applications of deep learning. Prerequisite: an
introductory course in machine learning or statistical learning.
| Week | Date | Topics and Materials | References | Assignments | Notes and Further Reading |
| 1 | 2/18 | Lecture 1: Feedforward Networks
Video 1, Video 2 by Zhihua Zhang | Chaps. 1, 6 | | LeCun et al. (2015) |
| 2 | 2/25
2/27 | Lecture 2: Regularization for DL
Video 1, Video 2 by Zhihua Zhang | Chap. 7 | | Bickel and Li (2006) |
| 3 | 3/3 | Lecture 3: Optimization for DL
Video 1, Video 2, Video 3, Video 4 by Zhanxing Zhu | Chap. 8 | Homework 1 | Lai (2003), Toulis and Airoldi (2015) |
| 4 | 3/10 | Lecture 4: Convolutional Networks
Video 1, Video 2 by Cheng Tai | Chap. 9 | | |
| 3/12 | Lecture 5: Recurrent Networks
Video 1, Video 2 by Yadong Mu | Chap. 10 | | |
| 5 | 3/17 | Review of Lectures 1 & 2 | | | Belkin et al. (2019) |
| 6 | 3/24 | Review of Lecture 3 | | | |
| 3/26 | Review of Lectures 4 & 5 | | | |
| 7 | 3/31 | Lecture 6: Autoencoders and Generative Models I | Chap. 14, Secs. 20.1–20.8 | | |
| 8 | 4/7 | Lecture 7: Autoencoders and Generative Models II | Chap. 19, Secs. 20.9–20.15 | Homework 2 | Blei et al. (2017) |
| 9 | 4/14 | Lecture 8: DL with PyTorch | DLwPT Essential Excerpts Chaps. 1–5 | | |
| 4/16 | Lecture 9: Practical Methodology
Video 1, Video 2 by Shuchang Zhou | Chap. 11 | | |
| 10 | 4/21 | Lecture 10: Large-Scale DL
Video 1, Video 2 by Kai Jia | Sec. 12.1 | | |
| 4/23 | Lecture 11: Applications | Secs. 12.2–12.5 | Homework 3 | |
| 11 | 5/2 | Midterm exam | | Final project | Mean = 82, median = 86, Q1 = 79, Q3 = 92, high score = 100 |
| 12 | 5/5 | No class | | | |
| 5/7 | No class | | | |
| 13 | 5/12 | Lecture A: Statistical Theory for Deep Networks | | | Schmidt-Hieber (2020) |
| 14 | 5/19 | Lecture B: Analysis of Stochastic Gradient Descent | | | Toulis and Airoldi (2017), Liang and Su (2019), Chen et al. (2020) |
| 5/21 | Lecture C: Optimal Transport | | | Peyré and Cuturi (2019) |
| 15 | 5/26 | Lecture D: Approximate Bayesian Computation | | | Marin et al. (2012), Fearnhead and Prangle (2012), Bernton et al. (2019) |
| 16 | 6/2 | Oral presentations I | | | |
| 6/4 | Oral presentations II | | |
|