Learning path
Next moduleModule 13: Deep-learning architectures
Module 12
Training and regularization
Make networks generalize: weight decay, dropout, early stopping, normalization, augmentation, schedules, and tuning.
- 1. Training and regularizationRead training curves, stop at the right time, regularize with weight decay, dropout, normalization, and augmentation, schedule the learning rate, and tune hyperparameters without fooling yourself.40 min
- 2. Debugging neural networksFind silent training bugs in a fixed order (initial loss, gradient check, overfit one batch, per-layer statistics, learning-rate sweep) and map each symptom to its likely cause and first fix.45 min
A model that fits the training set is easy; a model that generalizes takes discipline. This module covers the tools that close the gap between training and validation loss, how to tune hyperparameters without fooling yourself, and how to find the bugs that make a network fail without an error message.
| # | Lesson | You will be able to |
|---|---|---|
| 1 | Training and regularization | read training curves, apply weight decay, dropout, early stopping, and normalization, and run a sound hyperparameter search |
| 2 | Debugging neural networks | find silent training bugs with step-0 loss, gradient checks, overfit-one-batch, per-layer statistics, and an LR range test |