Deep Learning

Boston University - Spring 2025

Class is held in EPC 209 on Monday and Wednesday 6:30-7:45pm

Instructor: Prof. Iddo Drori, CDS 839

TF: Muhammed Yusuf Kocyigit

Grading: Presentation 20%, homeworks 25%, competition 50%, participation 5%

Textbooks

The Science of Deep Learning, Iddo Drori, Cambridge University Press, 2022

Understanding Deep Learning, Simon Prince, MIT Press, 2023

Deep Learning Foundations and Concepts, Bishop, Springer, 2023

Dive into Deep Learning, Zhang, Lipton, Li, Smola, 2023

Schedule

Martin Luther King Jr. Day (Monday, January 20): Holiday

Lecture 1 (Wednesday, January 22): Introduction

Lecture 2 (Monday, January 27): Forward and backpropagation

Lecture 3 (Wednesday, January 29): Optimization and regularization

Lecture 4 (Monday, February 3): Convolutional neural networks

Lecture 5 (Wednesday, February 5): Sequence models

Lecture 6 (Monday, February 10): Graph neural networks

Lecture 7 (Wednesday, February 12): Transformers

Presidents' Day Holiday (Monday, February 17) No classes

Lecture 8 (Monday, February 24): Transformers, mixture of experts, state-space models, singular-value fine-tuning
Reading:
Transformer circuits
Mixtral of experts, Jiang et al, 2024
Jamba-1.5: Hybrid Transformer-Mamba models at scale, Lenz et al, 2024
Transformer^2: Self-adaptive LLMs, Sun et al, 2025

Lecture 9 (Wednesday, February 26): Autoencoders

Lecture 10 (Monday, March 3): Diffusion models
Reading:
Tutorial on diffusion models for imaging and vision, Chan, 2025
Inference-time scaling for diffusion models beyond scaling denoising steps, Ma et al, 2025

Lecture 11 (Wednesday, March 5): Video and audio synthesis
Reading:
Photorealistic video generation with diffusion models, Gupta et al, 2024

Spring Recess (Saturday, March 8 - Sunday, March 16)

Lecture 12 (Monday, March 17): Building a foundation model from scratch: Data

Lecture 13 (Wednesday, March 19): Building a foundation model from scratch: Pre-training

Lecture 14 (Monday, March 24): Building a foundation from scratch: Post-training

Lecture 15 (Wednesday, March 26): Reasoning LLMs

Lecture 16 (Monday, March 31): Agents

Lecture 17 (Wednesday, April 2): Reinforcement learning

Lecture 18 (Monday, April 7): Deep reinforcement learning

Lecture 19 (Wednesday, April 9): Multi-agent deep reinforcement learning
Reading:
Multi-agent reinforcement learning, Albrecht et al, 2024

Lecture 20 (Monday, April 14): Evaluation

Lecture 21 (Wednesday, April 16): Social responsibility

Patriots' Day Holiday (Monday, April 21): No classes

Lecture 22 (Wednesday, April 23): Safety and security
Reading:
Responsible scaling policy, Anthropic, 2024
OpenAI model spec, 2024
Scaling automatic neuron description, Choi et al, 2024

Lecture 23 (Monday, April 28): AI governance
Reading:
Hardware-enabled governance mechanisms, Kulp et al, 2024
Open problems in technical AI governance, Reuel et al, 2024

Lecture 24 (Wednesday, April 30): TBD

Last day of classes (Thursday, May 1)