Learning path
Next moduleModule 14: Transformers
Module 13
Deep-learning architectures
CNNs for grids, RNNs and LSTMs for sequences, autoencoders, GANs, latent diffusion, and flow matching for generation, and transfer learning.
- 1. Convolutional networksSlide a kernel over an image to compute a feature map by hand, track output sizes with stride and padding, pool, and see why CNNs suit images.35 min
- 2. Recurrent networks, LSTMs, and GRUsUnroll an RNN through time by hand, explain why long dependencies are hard, see how LSTM and GRU gates help, and understand why transformers replaced them for language.35 min
- 3. Autoencoders, diffusion, and transfer learningLearn compressed representations with autoencoders, understand how diffusion models generate by reversing noise, and adapt pretrained models to new tasks with transfer learning.35 min
- 4. GANs, generator versus discriminatorTrain a generator and a discriminator against each other, derive the optimal discriminator and the non-saturating loss, and diagnose mode collapse and vanishing gradients.40 min
- 5. Latent diffusion and Stable DiffusionExplain why Stable Diffusion runs diffusion in an autoencoder's latent space, how text enters through cross-attention, and how classifier-free guidance trades diversity for prompt adherence; build a toy latent diffusion model with guidance.40 min
- 6. Flow matching and rectified flowsTrain a generator by regressing a velocity field onto straight lines between noise and data, sample by integrating an ODE, see why one Euler step fails and how reflow straightens paths, and relate it all to diffusion.40 min
Architectures encode assumptions (inductive biases) about the data. Convolutions assume local patterns that repeat across an image. Recurrence assumes information flows in order. Autoencoders assume data lies near a lower-dimensional space. Choosing an architecture is choosing an assumption.
| Architecture | Inductive bias | Common uses |
|---|---|---|
| MLP | generic feature interactions | tabular data, prediction heads |
| CNN | local patterns, translation structure | images, audio spectrograms |
| RNN / LSTM / GRU | sequential hidden state | time series, streaming |
| Transformer | content-based global attention | language, vision, multimodal |
| Autoencoder | compress then reconstruct | representation learning, denoising |
| GAN | a learned critic judges realism | narrow-domain images, adversarial losses |
| Diffusion / flow matching | iterative denoising along a path from noise | image, video, and audio generation |
| # | Lesson | You will be able to |
|---|---|---|
| 1 | Convolutional networks | compute a convolution and pooling by hand and track feature-map sizes |
| 2 | Recurrent networks, LSTMs, and GRUs | unroll an RNN, explain gates, and say why transformers replaced them |
| 3 | Autoencoders, diffusion, and transfer learning | explain bottlenecks, latent spaces, denoising diffusion, and fine-tuning a pretrained model |
| 4 | GANs, generator versus discriminator | train a GAN with the non-saturating loss and diagnose mode collapse |
| 5 | Latent diffusion and Stable Diffusion | explain latent compression, cross-attention conditioning, and classifier-free guidance |
| 6 | Flow matching and rectified flows | train a velocity field on straight noise-to-data paths and straighten it with reflow |