Skip to content
ML to LLMs
Labs / Neural Network Lab

Gradient descent

Step across a loss surface with SGD, momentum, or Adam, and push the learning rate until it diverges.

Theory and worked examples: Gradient descent

Gradient descent lab

Narrow bowl. Click the surface to choose a starting point.

Loss surface with optimizer paths-202-2-1012θ₁θ₂
Gradient descent−η∇L (next step)darker = higher loss; ring = minimum
f = ½(x² + 10y²). Curvature is 10× steeper in y than x, so one learning rate is too big for y and too small for x. Plain gradient descent diverges once η > 2/10 = 0.2.