Learning path
Next moduleModule 12: Training and regularization
Module 11
Gradient descent and backpropagation
How models learn: follow the gradient downhill, and compute that gradient efficiently with the chain rule.
- 1. Gradient descentMinimize a loss by repeatedly stepping against the gradient, compute steps by hand, and understand learning rates, mini-batches, momentum, and Adam.40 min
- 2. BackpropagationCompute every gradient of a small network by applying the chain rule backward through a computational graph, verify it numerically, and understand vanishing and exploding gradients.40 min
Training is a loop: forward pass, loss, backward pass, optimizer step. Gradient descent says which way to move the parameters; backpropagation computes the gradient for every parameter in one backward sweep.
| # | Lesson | You will be able to |
|---|---|---|
| 1 | Gradient descent | compute updates by hand, find the stable learning rate, and compare SGD, momentum, and Adam |
| 2 | Backpropagation | backpropagate through a one-neuron classifier by hand and explain vanishing and exploding gradients |
Interactive labs: Gradient descent · Backpropagation