Deep Learning

Boston University - Fall 2022

Class is held in CGS 505 on Tuesday and Thursday 3:30-4:45pm

Course staff and office hours

Wednesday 4-5pm, MCS 200C: Prof. Iddo Drori

Thursday 11am-12pm, MCS 103: Teaching Fellow, Vitali Petsiuk

Grader, Yida Xin

Grader, Wenda Qin

Textbook

The Science of Deep Learning, Iddo Drori, Cambridge University Press, 2022

Enrolled students receive a free online version

First Day of Classes (Tuesday, September 6)

Part I: Foundations

Lecture 1 (Tuesday, September 6): Introduction

Lecture 2 (Thursday, September 8): Forward and Backpropagation

Lecture 3 (Tuesday, September 13): Optimization

Lecture 4 (Thursday, September 15): Regularization

Part II: Architectures

Lecture 5 (Tuesday, September 20): Convolutional neural networks (CNNs)

Lecture 6 (Thursday, September 22): Sequence models (RNNs, LSTM, GRU)

Lecture 7 (Tuesday, September 27): Graph neural networks (GNNs)

Lecture 8 (Thursday, September 29): Transformers

Part III: Reinforcement Learning

Lecture 9 (Tuesday, October 4): Markov decision processes

Lecture 10 (Thursday, October 6): Reinforcement learning

Lecture 11 (Tuesday, October 11): Deep reinforcement learning

Lecture 12 (Thursday, October 13): Deep reinforcement learning

Part IV: Generative Models

Lecture 13 (Tuesday, October 18): Generative adversarial networks (GANs)

Lecture 14 (Thursday, October 20): Variational autoencoders (VAEs)

Lecture 15 (Tuesday, October 25): Diffusion models

Lecture 16 (Thursday, October 27): Language and vision

Lecture 17 (Tuesday, November 1): Language, vision and graphics

Part V: Meta Learning

Lecture 18 (Thursday, November 3): Automated machine learning

Lecture 19 (Tuesday, November 8): Multi-task learning

Lecture 20 (Thursday, November 10): Meta and transfer learning

Lecture 21 (Tuesday, November 15): Online and continual learning

Part VI: Applications

Lecture 22 (Thursday, November 17): Deep learning for education

Lecture 23 (Tuesday, November 22): Deep learning for climate science

Thanksgiving recess, academic holiday, no classes (Wednesday, November 23 - Sunday, November 27)

Lecture 24 (Tuesday, November 29): Quantum computing

Lecture 25 (Thursday, December 1): Deep learning for quantum computing

Projects

Lecture 26 (Tuesday, December 6): Presentations

Lecture 27 (Thursday, December 8): Presentations

Last Day of Classes (Monday, December 12)

Exercises: quiz and programming homework

Exercise 1: Forward and Backpropagation

Exercise 2: Optimization

Exercise 3: CNN's

Exercise 4: RNN's

Exercise 5: GNN's

Exercise 6: Transformers

Exercise 7: GAN's

Exercise 8: VAE's

Exercise 9: Meta learning

Exercise 10: RL

Tutorials

Tutorial 1: PyTorch

Tutorial 2: Neural MMO Challenge

Tutorial 3: Keras

Tutorial 4: CNN's with TensorFlow

Tutorial 5: RNN's

Tutorial 6: dgl.ai, GNN library

Tutorial 7: huggingface.co, Transformers library, stability.ai

Tutorial 8: pyro.ai, probabilistic programming library

Tutorial 9: learn2learn.net, meta learning library

Tutorial 10: RLlib, reinforcement learning library