Skip to content
ML to LLMs
Learning path

Module 0

Mathematical and programming foundations

Vectors, matrices, derivatives, probability, and the Python stack, with the shape habit that prevents most bugs.

  1. 1. Vectors and matricesRepresent data and models as vectors, matrices, and tensors; compute dot products and matrix products by hand; and never lose track of a shape.30 min
  2. 2. Calculus for MLDerivatives as slopes, partial derivatives as sensitivities, gradients as the uphill direction, and the chain rule that powers backpropagation.30 min
  3. 3. Probability and statisticsConditional probability, Bayes' rule, expectation, variance, common distributions, sampling, and why correlation is not causation.35 min
  4. 4. The Python toolkitVectorized NumPy with broadcasting, pandas for tabular data, matplotlib for plots, and reproducible scripts.25 min
  5. 5. Information theoryCompute surprise, entropy, cross-entropy, KL divergence, and perplexity by hand, and explain why the classification loss you minimize is exactly cross-entropy.40 min
  6. 6. Numerical stabilitySpot overflow, underflow, and cancellation before they turn a training run into NaN, and fix them with log-sum-exp, log-softmax, the right float format, loss scaling, and gradient checks.40 min

Core mental model. A model is a parameterized function. Training searches for parameter values that make its predictions useful on examples it has not seen.

You need only a small, precise toolkit: linear algebra to represent data and models, calculus to know which way to adjust parameters, probability to reason about uncertainty, and NumPy to do it all fast. Each lesson below is built around the exact operations used later in the course.

#LessonYou will be able to
1Vectors and matricescompute dot products and matrix products, and track tensor shapes
2Calculus for MLtake derivatives, partial derivatives, and gradients, and apply the chain rule
3Probability and statisticsuse conditional probability, Bayes' rule, expectation, and variance; separate correlation from causation
4The Python toolkitwrite vectorized NumPy, load data with pandas, and plot with matplotlib
5Information theorycompute entropy, cross-entropy, KL divergence, and perplexity, and explain why cross-entropy is the classification loss
6Numerical stabilityuse log-sum-exp and log-softmax, choose fp32/fp16/bf16, and check gradients with finite differences

The shape habit. Write tensor shapes next to every operation. If XX is [batch,features][\text{batch}, \text{features}] and WW is [features,outputs][\text{features}, \text{outputs}], then XWXW is [batch,outputs][\text{batch}, \text{outputs}].

Code: code/00-foundations/

Next moduleModule 1: ML vocabulary and learning paradigms