Glossary and formula reference

The nonlinear function a neuron applies to its weighted sum, so the network can bend its decision boundaries.

Formal: $a = f(z)$ with $z = w\cdot x + b$; common $f$ are ReLU, GELU, sigmoid, tanh.

Example: ReLU turns $z = -1.5$ into 0 and $z = 2$ into 2.

Related: Logit, Softmax

Lesson: The forward pass

How much better an action turned out than what was expected from that state.

Formal: $A(s, a) = Q(s, a) - V(s)$; estimated as $G_t - V(s_t)$ (REINFORCE with a baseline), with GAE (PPO), or as a group-normalized reward (GRPO).

Example: An episode returns $-2.71$ from a cell whose baseline is $-3.5$, so the advantage is $+0.79$, and the actions taken become more likely.

Related: Policy gradient, Generalized advantage estimation

Lesson: Policy gradients and REINFORCE

A system that uses a model in a loop to plan, call tools, observe results, and decide what to do next.

Formal: A controller that alternates model calls and tool executions until a stop condition, under step, cost, and permission limits.

Example: A support agent looks up an order, checks the refund policy, and drafts a reply for human approval.

Related: Tool calling, Prompt injection

Lesson: Production architecture

How an agent keeps information beyond one model call: a small working set in the prompt, plus an external store it writes to and searches with tools.

Formal: Two tiers: a bounded main context (system prompt, editable core memory, recent messages) and an external store (archival notes, evicted messages), connected by memory tools whose results are appended to the next prompt.

Example: The agent saves "dog is named Miso" with archival_memory_insert; 30 turns later, after the message was evicted, archival_memory_search("dog") pages it back in.

Related: Context window, RAG, Tool calling

Lesson: Agent memory

Every GPU shares its slice so all GPUs end with the full concatenated tensor.

Formal: Each rank contributes one chunk and receives the other $N - 1$; it sends $\frac{N-1}{N}T$ bytes per rank. The second half of a ring all-reduce.

Example: FSDP all-gathers a layer's sharded weights just before computing that layer.

Related: Reduce-scatter, All-reduce, FSDP

Lesson: Collective operations from scratch

A group operation that leaves every GPU holding the sum (or average) of everyone's tensors.

Formal: A ring all-reduce is a reduce-scatter followed by an all-gather, $2(N-1)$ steps in total, and sends $2\frac{N-1}{N}T$ bytes per rank for a $T$-byte tensor.

Example: Data-parallel training all-reduces gradients every step: 13.48 GB of bf16 gradients over 8 GPUs means 23.58 GB sent per GPU.

Related: Reduce-scatter, All-gather, Data parallelism

Lesson: Collective operations from scratch

Finding data points that don't fit the normal pattern, usually without labeled examples of the anomalies.

Formal: Learn a score $a(x)$ from mostly normal data so that unusual $x$ get high scores; flag $a(x) > \tau$, with $\tau$ set from an alert budget.

Example: Flagging the top 200 of 1 million daily transactions by isolation forest score for analyst review.

Related: Isolation forest, Robust z-score

Lesson: Anomaly detection

Finding vectors that are probably the closest to a query, checking only a small part of the collection instead of every vector.

Formal: An index returns a set $S$ of $k$ vectors that approximates the exact top-$k$ under a distance; quality is measured as recall@$k = |S \cap \text{exact top-}k|/k$ against brute force.

Example: An IVF index with 1,024 cells that probes 8 scans about 0.8% of a million vectors.

Related: HNSW, Embedding, Cosine similarity

Lesson: Vector search

A learned, soft lookup: each token gathers information from other tokens, weighted by how relevant they are.

Formal: $\text{softmax}(QK^\top/\sqrt{d_k})V$, where queries, keys, and values are linear projections of the token vectors.

Example: With $q = [1, 0]$, keys $[1,0]$ and $[0,1]$, the weights are $[0.67, 0.33]$.

Related: Transformer, Context window

Lesson: Self-attention

How strongly a series is correlated with a shifted copy of itself.

Formal: $r_k = \sum_t (y_t - \bar y)(y_{t-k} - \bar y) / \sum_t (y_t - \bar y)^2$; plotting $r_k$ against $k$ gives the ACF.

Example: Daily data with a weekly pattern has ACF spikes at lags 7, 14, and 21.

Related: Stationarity

Lesson: Time series

A framework feature that records the operations applied to tensors and computes gradients by running the chain rule backwards over that record.

Formal: Reverse-mode automatic differentiation: one backward pass gives $\partial L / \partial \theta$ for every parameter $\theta$ that has requires_grad=True. Gradients add into .grad, so they must be zeroed each step.

Example: For $L = (wx + b - y)^2$ with $x = 2, w = 3, b = 1, y = 10$, L.backward() sets w.grad = -12 and b.grad = -6.

Related: Backpropagation, Gradient

Lesson: Intro to PyTorch

Computing exact gradients by recording each operation in the forward pass and applying the chain rule to that record in reverse.

Formal: Reverse-mode autodiff visits the computation graph in reverse topological order; each node adds (local derivative × upstream gradient) to its inputs' .grad.

Example: For a * a + a at a = 3, the two paths into a contribute 6 and 1, so a.grad = 7.

Related: Backpropagation

Lesson: Backpropagation

A single-number summary of the precision-recall curve: the average precision at the rank of each positive example.

Formal: $\text{AP} = \frac{1}{P}\sum_k \text{precision@}k \cdot \mathbb{1}[\text{item } k \text{ is positive}]$, where P is the number of positives.

Example: Positives at ranks 1, 2, 4, 6, 9 give AP = (1 + 1 + 0.75 + 0.667 + 0.556)/5 = 0.794.

Lesson: Classification metrics

The method for computing how much each weight contributed to the loss, by applying the chain rule backward through the network.

Formal: Reverse-mode automatic differentiation over the computational graph, multiplying local derivatives from the loss to each parameter.

Example: For a sigmoid neuron with cross-entropy, $\partial L/\partial w_j = (\hat{y} - y)x_j$.

Related: Gradient, Gradient descent

Lesson: Backpropagation

The simplest reasonable approach, used to judge whether a model is actually useful.

Formal: A reference score from a trivial predictor (mean, majority class, last value) or a simple model on the same evaluation data.

Example: With 5% positives, "always negative" scores 95% accuracy, so a 95% model has learned nothing.

Related: Metric

Lesson: Workflow, splits, and leakage

A small group of training examples processed together in one optimization step.

Formal: A mini-batch $B$ used to estimate the gradient $\frac{1}{|B|}\sum_{i\in B}\nabla\ell_i$.

Example: 1,024 examples with batch size 32 give 32 steps per epoch.

Related: Epoch, Gradient descent

Lesson: Gradient descent

A search that fits a cheap model of score versus settings, with uncertainty, and uses it to choose the next setting to try.

Formal: A surrogate (Gaussian process or TPE) gives $\mu(\lambda), \sigma(\lambda)$; an acquisition such as expected improvement $(\mu - f^* - \xi)\Phi(z) + \sigma\phi(z)$ picks the next $\lambda$.

Example: Optuna's default TPE sampler is a Bayesian optimization method.

Related: Random search, Hyperparameter

Lesson: Hyperparameter tuning

The value of where you are equals the reward for one step plus the discounted value of where you land.

Formal: $V^\pi(s) = \sum_a \pi(a \mid s) \sum_{s'} P(s' \mid s, a)[r + \gamma V^\pi(s')]$; the optimality version replaces the average over actions with a max.

Example: Value iteration repeats the optimality version until values stop changing.

Related: Markov decision process, Q-learning, Temporal-difference learning

Lesson: MDPs, states, actions, and rewards

A correction for testing many metrics at once that controls the expected fraction of false "wins" rather than the chance of any false win.

Formal: Sort p-values p_(1) ≤ … ≤ p_(m); find the largest r with p_(r) ≤ q·r/m; reject hypotheses 1..r. Controls false discovery rate at q (for independent or positively dependent tests).

Example: For p = 0.001, 0.008, 0.012, 0.041, … with m = 10 and q = 0.05, the cut falls at rank 3 (0.012 ≤ 0.015, 0.041 > 0.02).

Related: p-value

Lesson: Experimentation and A/B testing

Simple models are consistently wrong (bias); flexible models change a lot with the training data (variance); the best model balances both.

Formal: Expected squared error $= \text{bias}^2 + \text{variance} + \sigma^2$.

Example: A degree-1 polynomial underfits a sine curve; degree 15 overfits it.

Related: Overfitting, Regularization

Lesson: Overfitting and bias-variance

A way to label each token so a classifier can mark entity spans: B starts an entity, I continues it, O means outside any entity.

Formal: For a span $(s, e, X)$, token $s$ gets B-X, tokens $s+1..e-1$ get I-X, all others O; I-X may only follow B-X or I-X.

Example: "New York City mayor" is tagged B-LOC I-LOC I-LOC O.

Related: Named entity recognition, Viterbi algorithm

Lesson: Named entity recognition

A keyword-ranking formula that scores documents by how often they contain the query's words, giving more weight to rare words and less to very long documents.

Formal: $\text{BM25}(q, D) = \sum_{t \in q} \text{IDF}(t)\,\frac{f(t,D)(k_1+1)}{f(t,D) + k_1(1 - b + b\,|D|/\text{avgdl})}$, typically $k_1 \approx 1.2$, $b \approx 0.75$.

Example: A query for an error code like "E1234" is matched exactly by BM25 even when a dense embedding treats it as noise.

Related: TF-IDF, Reciprocal rank fusion

Lesson: Vector search

Estimating the uncertainty of a statistic by resampling your data with replacement many times and recomputing it each time.

Formal: Draw B resamples of size n with replacement, compute the statistic on each, and take the 2.5th and 97.5th percentiles as a 95% interval.

Example: 870 of 1,000 correct gives a bootstrap 95% interval of about [0.850, 0.890] for accuracy.

Lesson: Probability and statistics

A resample of the training set, the same size as the original, drawn with replacement, so some rows repeat and others are left out.

Formal: Draw $n$ indices uniformly from $\{1,\dots,n\}$ with replacement; each row is left out with probability $(1 - 1/n)^n \approx e^{-1} \approx 0.368$.

Example: From 1,000 rows, a bootstrap sample contains about 632 distinct rows; the roughly 368 left out are that tree's out-of-bag rows.

Related: Out-of-bag error

Lesson: Random forests and gradient boosting

NumPy's rule for combining arrays of different shapes by stretching size-1 dimensions without copying data.

Formal: Align shapes from the right; each pair of dimensions must be equal or one must be 1 (missing dimensions count as 1). The result takes the larger size in each position.

Example: [1000, 3] - [3] gives [1000, 3]; [1000] - [1000, 1] gives [1000, 1000], usually a bug.

Related: Data leakage

Lesson: Python toolkit

Whether a model's predicted probabilities match how often things actually happen.

Formal: A model is calibrated if $P(y = 1 \mid \hat{p} = p) = p$ for all $p$; measured with reliability diagrams or the Brier score.

Example: Of all emails scored 0.8 spam, about 80% should really be spam.

Related: Logit, Hallucination

Lesson: Classification metrics

Shift a growing share of traffic to a new version and roll back automatically if guard metrics get worse.

Formal: Stages s₁ < … < 1; promote while metric_canary ≤ k · metric_baseline for every gate.

Lesson: Production architecture

When training a model on a new, narrow task makes it lose skills it had before.

Formal: Performance on the original distribution drops after fine-tuning because gradient updates for the new objective overwrite weights the old behavior depended on.

Example: A model fine-tuned for many epochs on support tickets gets better at tickets but worse at general reasoning and safety refusals.

Related: Fine-tuning, LoRA

Lesson: Fine-tuning in practice

Prompting a model to write out intermediate reasoning steps before its final answer, so each step is a small prediction.

Formal: If each step fails independently with probability $\varepsilon$, an $s$-step answer is right with probability $(1-\varepsilon)^s$; writing steps as tokens lowers $\varepsilon$ because later tokens can condition on earlier results.

Example: At $\varepsilon = 0.15$ a 4-step answer is right with probability $0.85^4 = 0.522$; at $\varepsilon = 0.05$ it is $0.95^4 = 0.815$.

Related: Few-shot prompting, Self-consistency, ReAct

Lesson: Few-shot and chain-of-thought

A saved snapshot of a model's weights (and often optimizer state) during or after training.

Formal: A serialized copy of $\theta$ at step $t$, used to resume training, evaluate, or deploy.

Example: Early stopping deploys the checkpoint with the lowest validation loss, not the last one.

Related: Epoch

Lesson: Training and regularization

The split between model size and training tokens that gives the lowest loss for a fixed training compute budget, roughly 20 tokens per parameter.

Formal: $N^*, D^* = \arg\min_{6ND = C} L(N, D)$. With Approach 3 constants, $N^* \propto C^{0.45}$ and $D^* \propto C^{0.55}$, so the ratio drifts with budget.

Example: For $C = 6 \times 10^{22}$ FLOPs at 20 tokens per parameter: $N = 22.4$B, $D = 447$B tokens. Models served heavily are trained far past this ratio on purpose.

Related: Scaling law

Lesson: Scaling laws

Splitting documents into smaller pieces that can be retrieved individually.

Formal: Partitioning text into segments of size $S$ with overlap $O$, giving about $\lceil (N - O)/(S - O)\rceil$ chunks.

Example: A 1,000-word document with 200-word chunks and 50-word overlap gives 7 chunks.

Related: RAG, Embedding

Lesson: The RAG pipeline

Multipliers on the loss that make mistakes on a rare class cost more, so the model pays attention to it.

Formal: Balanced weights are $w_c = n / (C \cdot n_c)$, so each class carries total weight $n / C$. For logistic regression they inflate the predicted odds by about $n_0 / n_1$, which mainly moves the decision threshold.

Example: With 2,279 negatives and 121 positives, $w_0 = 0.5265$ and $w_1 = 9.917$: one positive counts as 18.83 negatives.

Related: SMOTE, Calibration, Loss

Lesson: Imbalanced data

A sampling trick that strengthens how much a diffusion model follows its prompt by extrapolating from an unconditional prediction toward the conditional one.

Formal: $\tilde\epsilon = \epsilon_\theta(z_t, t, \varnothing) + w[\epsilon_\theta(z_t, t, c) - \epsilon_\theta(z_t, t, \varnothing)]$; requires dropping the condition (about 10%) during training. $w = 1$ is plain conditional sampling.

Example: $\epsilon_\varnothing = 0.2$, $\epsilon_c = 0.5$, $w = 3$ gives $0.2 + 3 \times 0.3 = 1.1$.

Related: Latent diffusion

Lesson: Latent diffusion and Stable Diffusion

The problem of making good predictions for a new user or item that has no interaction history yet.

Formal: For an entity with no logged interactions, collaborative signals are undefined, so the model must rely on content and context features, priors such as popularity, or deliberate exploration.

Example: A video uploaded a minute ago has no views; the item tower embeds its title and frames so it can still be retrieved, and re-ranking gives it a small freshness boost.

Related: Two-tower model, Position bias

Lesson: ML system design

The world's answer to the same input changes, so a model that was right becomes wrong even though its inputs look normal.

Formal: A change in $P(y \mid x)$. Contrast covariate drift (change in $P(x)$) and prior drift (change in $P(y)$). Only outcome monitoring with labels can detect it.

Example: Fraudsters change tactics, so the same basket size now carries a different fraud risk.

Related: Drift, Population Stability Index

Lesson: Model monitoring and drift

Forcing a model's output to follow a format, such as a JSON schema, by blocking any next token that would break it.

Formal: At each step, tokens outside the allowed set $A_t$ (from a grammar compiled from the schema) get probability 0 and the rest are renormalized: $\tilde p(y_t) \propto p(y_t)\,\mathbb{1}[y_t \in A_t]$.

Example: A tool call generated under the schema always parses, though flight_id may still be the wrong flight.

Related: Tool calling, Temperature

Lesson: Tool use and agents

Deciding everything that goes into a model call's context window (instructions, tools, retrieved documents, history, memory) and in what order, under a token budget.

Formal: Choose and order components so that input tokens plus the output reserve fit the window $W$, while maximizing task quality; with full history, total billed input over $T$ calls is $TF + h\,T(T-1)/2$.

Example: Selecting 10 of 30 tools, 3 of 8 documents, and compacting history cut a synthetic 20-call conversation from 231,500 to 93,750 input tokens.

Related: Context window, Lost in the middle, Prefix caching, RAG

Lesson: Context engineering

The maximum number of tokens a language model can consider at once, prompt and output combined.

Formal: The maximum sequence length $n_{\max}$ over which attention is computed.

Example: Anything beyond the window is invisible to the model unless retrieved and inserted.

Related: Token, Attention

Lesson: Tokenization and pretraining

A serving scheduler that adds and removes requests from the running batch at every decode step, instead of waiting for a whole batch to finish.

Formal: Admission is decided per iteration: when a sequence emits its end token, its slot (and KV memory) is given to a waiting request on the next step. Also called in-flight or iteration-level batching.

Example: Two slots, requests of 2, 8, 3, and 3 tokens: static batching finishes at step 11 (1.45 tokens/step), continuous batching at step 8 (2.0 tokens/step).

Related: Batch, Paged attention, KV cache

Lesson: Serving LLMs

An optimizer that updates one parameter at a time, solving for it exactly while holding the others fixed, and sweeps until nothing moves.

Formal: For $j = 1, \dots, d$: $w_j \leftarrow \arg\min_{w_j} L(w_1, \dots, w_d)$; repeat. Lasso solvers use it because each one-weight problem has a closed form (soft-thresholding).

Example: scikit-learn's Lasso and ElasticNet are fitted by coordinate descent.

Related: Soft-thresholding

Lesson: Regularization and regression metrics

How closely two vectors point in the same direction, ignoring their lengths.

Formal: $\cos(a, b) = \frac{a\cdot b}{\lVert a\rVert\lVert b\rVert}$, between −1 and 1.

Example: $[1, 2, 0]$ and $[2, 1, 1]$ have cosine 0.730.

Related: Embedding, TF-IDF

Lesson: From text to vectors

Runtime budget caps (per call, steps, per task, spend rate, monthly) that halt an agent before it crosses a limit.

Lesson: Building reliable agents

A loss that punishes a model for giving low probability to the correct answer.

Formal: $-\ln p_{\text{true}}$ for one example; binary form $-[y\ln p + (1-y)\ln(1-p)]$.

Example: Probability 0.9 on the true class costs 0.105; probability 0.01 costs 4.6.

Related: Loss, Perplexity

Lesson: Logistic regression

Connectionist temporal classification: a way to train a model to output text from many audio frames without knowing which frame produced which letter.

Formal: The model emits a distribution over tokens plus a blank at each frame; a path collapses to text by merging repeats and removing blanks, and $P(y \mid x)$ sums over all paths that collapse to $y$, computed with a forward algorithm in $O(T \cdot (2U+1))$.

Example: With frame probabilities $(0.4, 0.6), (0.5, 0.5), (0.7, 0.3)$ for (blank, a), $P(\text{"a"}) = 0.77$ and the loss is $-\ln 0.77 = 0.261$.

Related: Word error rate, Spectrogram

Lesson: Speech recognition: CTC, transducers, and attention

A variance-reduction trick for A/B tests that subtracts the part of each user's outcome that was already predictable from their behavior before the experiment.

Formal: $Y^{\text{cuped}} = Y - \theta(X - \bar{X})$ with $\theta = \operatorname{Cov}(X, Y)/\operatorname{Var}(X)$, where $X$ is a pre-experiment covariate; the variance shrinks by a factor $1 - \rho^2$ and the estimate stays unbiased.

Example: With $\rho = 0.5$ between last month's and this month's sessions, variance drops 25%, so a test needing 14,749 users per arm needs about 11,062.

Related: Statistical power, Minimum detectable effect

Lesson: Experimentation and A/B testing

In many dimensions, random points are all about equally far apart, so "nearest neighbor" carries little information and data needed to cover the space grows exponentially.

Example: For 1,000 uniform random points and a random query, nearest/farthest distance ratio was 0.007 in 2 dimensions and 0.898 in 1,000 dimensions.

Lesson: K-nearest neighbors

When training uses information that won't be available at prediction time, making results look better than they are.

Formal: Any dependence of training features or preprocessing on test data, future data, or the target itself.

Example: Fitting a scaler on all data before splitting, or including "refund issued" when predicting fraud.

Related: Baseline, Generalization

Lesson: Workflow, splits, and leakage

Every GPU holds a full copy of the model and trains on a different slice of the batch; gradients are averaged so all copies stay identical.

Formal: With equal shards, the mean of shard gradients equals the full-batch gradient; one gradient all-reduce per step. Throughput scales with GPUs, memory per GPU does not drop. PyTorch's DistributedDataParallel (DDP).

Example: A 7B-style model still needs about 108.6 GB per GPU under DDP, however many GPUs you add.

Related: All-reduce, ZeRO, Tensor parallelism

Lesson: Scaling and distributed training

A ReLU unit whose input is negative for every example, so it always outputs 0 and gets zero gradient, and stops learning.

Formal: Unit $j$ is dead on a dataset if $z_j(x) \le 0$ for all $x$; then $\partial L / \partial z_j = 0$.

Example: With hidden biases initialized to −3, 97% of a second layer's units were dead and its gradients were about 1,000 times smaller than in a healthy network.

Related: Activation, Gradient

Lesson: Debugging neural networks

Function approximation, bootstrapping, and off-policy learning together can make value learning diverge.

Formal: Term from Sutton and Barto: any two of the three are stable in general, but combining all three removes convergence guarantees for TD methods.

Example: DQN has all three; replay and a target network make it work in practice without restoring the guarantee.

Related: Deep Q-network (DQN), Target network

Lesson: Deep Q-networks (DQN)

Q-learning where a neural network, instead of a table, predicts the value of each action in a state.

Formal: Minimizes $\mathbb{E}\big[\ell\big(Q(s,a;\theta) - (r + \gamma(1-d)\max_{a'} Q(s',a';\theta^-))\big)\big]$ over minibatches from a replay buffer, with a target network $\theta^-$ and usually the Huber loss $\ell$.

Example: A 16 → 32 → 4 network learns the 6-step shortest path on a 4×4 GridWorld; the 2015 version learned dozens of Atari games from pixels.

Related: Experience replay, Target network, Double DQN

Lesson: Deep Q-networks (DQN)

A fairness criterion that requires every group to be selected at the same rate.

Formal: $P(\hat Y = 1 \mid A = a)$ is equal for all groups $a$. With unequal base rates it usually conflicts with equal error rates and equal precision.

Example: Approving 31% of group A and 52% of group B violates demographic parity even if both groups have the same TPR and FPR.

Related: Equalized odds

Lesson: Fairness

An overlap score for masks, equal to F1 computed on pixels; used as a loss for imbalanced segmentation.

Formal: $\text{Dice} = 2|P \cap Y| / (|P| + |Y|) = 2\,\text{IoU}/(1 + \text{IoU})$; soft Dice loss replaces the sets with predicted probabilities.

Example: 40 shared pixels, 60 predicted, 50 true: Dice = 80/110 = 0.727, IoU = 0.571.

Related: Intersection over union, Loss

Lesson: Semantic segmentation and U-Net

A guarantee that a computation's output would look almost the same whether or not any one person's record was in the data.

Formal: $M$ is $(\varepsilon, \delta)$-DP if $P[M(D) \in S] \le e^\varepsilon P[M(D') \in S] + \delta$ for all neighbouring datasets $D, D'$ and output sets $S$. Costs add up over queries; post-processing is free.

Example: Releasing a count with Laplace noise of scale $1/\varepsilon$ is $\varepsilon$-DP; at $\varepsilon = 0.5$ the typical error is 2.

Related: DP-SGD

Lesson: Differential privacy

How much a reward later counts compared with the same reward now.

Formal: $\gamma \in [0, 1]$; the return is $G_t = \sum_k \gamma^k r_{t+k}$, and the effective horizon is about $1/(1-\gamma)$ steps.

Example: With $\gamma = 0.9$, a reward 10 steps away keeps $0.9^{10} \approx 0.35$ of its weight.

Related: Markov decision process, Bellman equation

Lesson: MDPs, states, actions, and rewards

Training a small model to imitate a large one, to get similar quality more cheaply.

Formal: Minimizing the divergence between the student's outputs and the teacher's (soft) outputs, often alongside the true labels.

Example: A small classifier trained on a large LLM's labels serves high-volume traffic at a fraction of the cost.

Related: Quantization, Inference

Lesson: Reliability, cost, and observability

A DQN variant that reduces overoptimistic Q-values by letting one network choose the next action and another score it.

Formal: $y = r + \gamma(1-d)\,Q\big(s', \arg\max_{a'} Q(s', a'; \theta); \theta^-\big)$.

Example: Online values $[-5.5, -5.1, -5.7, -4.95]$ pick action 4; the target network scores it $-5.0$, so $y = -1 + 0.99 \times (-5.0) = -5.95$ instead of the vanilla $-5.851$.

Related: Deep Q-network (DQN)

Lesson: Deep Q-networks (DQN)

Training with differential privacy: limit how much each example can push the model, then add noise to every update.

Formal: Clip each per-example gradient to norm $C$, add $\mathcal{N}(0, \sigma^2 C^2 I)$ to their sum, divide by the batch size, and step; an accountant tracks the total $\varepsilon$ across steps (Abadi et al. 2016).

Example: 100 full-batch steps at $\sigma = 4$, $\delta = 10^{-5}$ give $\varepsilon \approx 15$; $\sigma = 16$ gives about 3.2.

Related: Differential privacy, Gradient descent

Lesson: Differential privacy

Direct preference optimization: training a model on pairs of better and worse answers so it prefers the better kind, without a separate reward model or reinforcement learning loop.

Formal: $\mathcal{L} = -\log\sigma\big(\beta[\log\frac{\pi_\theta(y_w|x)}{\pi_{\text{ref}}(y_w|x)} - \log\frac{\pi_\theta(y_l|x)}{\pi_{\text{ref}}(y_l|x)}]\big)$ with a frozen reference model $\pi_{\text{ref}}$.

Example: At the start of training the policy equals the reference, so the loss is $\ln 2 = 0.693$.

Related: Fine-tuning, LoRA

Lesson: Fine-tuning in practice

When the data or the world changes after deployment, so a model's performance degrades.

Formal: Data drift is a change in $P(x)$; concept drift is a change in $P(y \mid x)$; prompt drift is a change in what users ask.

Example: A demand model trained before a price change starts under-predicting.

Related: Generalization, Calibration

Lesson: Reliability, cost, and observability

A dense vector that represents an item (word, token, document, user) so that similar items are close together.

Formal: A learned mapping $e: \text{item} \to \mathbb{R}^d$, often a row of an embedding matrix of shape $[V, d]$.

Example: "cat" and "kitten" have high cosine similarity even though the words differ.

Related: Cosine similarity, Token

Lesson: Word embeddings

The average surprise of a probability distribution: how uncertain you are before seeing the outcome.

Formal: $H(P) = -\sum_x p(x)\log p(x)$. Bits with $\log_2$, nats with $\ln$.

Example: A fair coin has 1 bit of entropy; a coin that lands heads 99% of the time has 0.08 bits.

Related: Cross-entropy, KL divergence, Perplexity

Lesson: Information theory

One full pass over the training data.

Formal: $\lceil n/|B|\rceil$ optimization steps for $n$ examples and batch size $|B|$.

Example: Validation loss is usually checked once per epoch for early stopping.

Related: Batch, Checkpoint

Lesson: Gradient descent

Usually take the best-looking action, but with a small probability take a random one, so the agent keeps discovering.

Formal: With probability $\varepsilon$ choose uniformly among actions; otherwise choose $\arg\max_a Q(s, a)$.

Example: With $\varepsilon = 0.1$ and 4 actions, the greedy action is taken with probability $0.9 + 0.1/4 = 0.925$.

Related: Q-learning, SARSA

Lesson: Q-learning and SARSA

A fairness criterion that requires the model to make the same kinds of mistakes in every group: equal true positive rates and equal false positive rates.

Formal: $P(\hat Y = 1 \mid Y = y, A = a)$ is equal across groups for $y \in \{0, 1\}$. Equal opportunity is the $y = 1$ half. When base rates differ, it cannot hold together with equal precision (unless the classifier is perfect).

Example: TPR 0.8 and FPR 0.1 in both groups satisfies equalized odds.

Related: Demographic parity, Calibration

Lesson: Fairness

A workflow where one call drafts an answer and another critiques it against criteria, looping until it passes or a cap is hit.

Lesson: Building reliable agents

A way to fit models with hidden labels, such as which Gaussian generated each point: guess soft labels, refit the parameters to them, and repeat. The likelihood never decreases.

Formal: E-step: $\gamma_{ik} = \pi_k \mathcal{N}(x_i \mid \mu_k, \Sigma_k) / \sum_j \pi_j \mathcal{N}(x_i \mid \mu_j, \Sigma_j)$. M-step: $N_k = \sum_i \gamma_{ik}$, $\pi_k = N_k/n$, $\mu_k = \sum_i \gamma_{ik} x_i / N_k$, $\Sigma_k$ the $\gamma$-weighted covariance.

Example: a point with responsibilities $(0.24, 0.76)$ contributes 0.24 of itself to component 1's mean and 0.76 to component 2's.

Lesson: DBSCAN, hierarchical clustering, and Gaussian mixtures

A buffer of recent transitions that an RL agent trains on in random minibatches instead of in the order they happened.

Formal: Store $(s, a, r, s', d)$ tuples in a fixed-capacity FIFO buffer $\mathcal{D}$ and sample minibatches uniformly (or by priority) for each gradient step. Valid for off-policy methods such as Q-learning.

Example: With capacity 2,000 and batch 32, each transition is reused in many updates, and a batch mixes transitions from many episodes.

Related: Deep Q-network (DQN), Target network

Lesson: Deep Q-networks (DQN)

The fixed number of token slots each expert gets per batch; extra tokens sent to a full expert are dropped.

Formal: $C = \lceil c \cdot Tk/E \rceil$ for $T$ tokens, top-$k$ routing, $E$ experts, and capacity factor $c$.

Example: 4,096 tokens, 64 experts, top-2, $c = 1.25$: $C = \lceil 1.25 \times 128 \rceil = 160$.

Related: Mixture of experts

Lesson: Mixture of experts

How much of an answer is actually supported by the sources it was given.

Formal: The fraction of the answer's claims that an entailment model or judge marks as supported by the retrieved context.

Example: Two of three claims supported gives faithfulness 0.67, even if the third claim happens to be true.

Related: Calibration

Lesson: Evaluating LLM systems

An input variable the model uses to make a prediction.

Formal: A component $x_j$ of the input vector $x \in \mathbb{R}^p$.

Example: Square footage when predicting house prices.

Related: Label, Parameter

Lesson: The language of ML

A system that defines each feature once and serves the same values both to training pipelines and to the live model.

Formal: An offline store of historical feature values supporting point-in-time (as-of) joins for training, plus a low-latency online store for serving, both fed by the same feature definitions.

Example: "Clicks on this item in the last hour" is computed by one streaming job, written to the online store for serving and to the offline store so training examples get the value as of each impression.

Related: Training-serving skew, Data leakage

Lesson: ML system design

Putting a few input-output examples in the prompt so the model copies the task's format and style.

Formal: In-context learning: the examples are part of the conditioning context $p(y \mid \text{examples}, x)$; no weights change.

Example: One worked example ending in Answer: 20 made a model end every answer with Answer: N, so a strict parser found the answer in 100% of outputs instead of 24%.

Related: Chain-of-thought, Fine-tuning

Lesson: Few-shot and chain-of-thought

Continuing to train a pretrained model on task-specific data so it changes its behavior.

Formal: Further optimization of some or all pretrained weights (or added adapters such as LoRA) on a new dataset.

Example: Fine-tuning a model on support transcripts to adopt a consistent tone and format.

Related: RAG, LoRA

Lesson: Adapting LLMs

An exact way to compute attention that keeps small tiles of the work in fast on-chip memory, so the full attention matrix is never written to slow GPU memory.

Formal: Tiles $Q$, $K$, $V$ into SRAM blocks and uses an online softmax (running max and sum) to produce the same output as standard attention, with $O(T)$ extra memory instead of $O(T^2)$ and far less memory traffic.

Example: At 8,192 tokens, one head's 16-bit score matrix would be 134 MB; FlashAttention never materializes it.

Related: Attention, KV cache

Lesson: LLM inference

Training a generator by teaching a network the velocity that moves points along straight paths between noise and data, then following those velocities from noise.

Formal: $x_t = (1-t)x_0 + t x_1$, loss $\mathbb{E}\lVert v_\theta(x_t, t) - (x_1 - x_0)\rVert^2$; sample by integrating $dx/dt = v_\theta$ from $t = 1$ to $0$. Rectified flow adds reflow: retraining on the model's own (noise, sample) pairs to straighten paths.

Example: $x_0 = 2$, $x_1 = -0.5$, $t = 0.25$ gives $x_t = 1.375$ and target velocity $-2.5$.

Related: Latent diffusion

Lesson: Flow matching and rectified flows

Cross-entropy that down-weights examples the model already gets right, so rare hard examples dominate the gradient.

Formal: $\text{FL} = -\alpha(1 - p_t)^\gamma \ln p_t$, where $p_t$ is the probability of the true class; $\gamma = 0$ gives cross-entropy.

Example: With $\gamma = 2$, an easy example at $p_t = 0.9$ keeps 1% of its cross-entropy; a hard one at $p_t = 0.1$ keeps 81%.

Related: Cross-entropy, Loss

Lesson: Loss functions

PyTorch's built-in version of fully sharded data parallelism: each GPU keeps only a slice of the model and gathers each layer's weights when it needs them.

Formal: Fully Sharded Data Parallel, equivalent to ZeRO stage 3. Weights are all-gathered before each wrapped module's forward and backward, and gradients are reduce-scattered. fully_shard in FSDP2; SHARD_GRAD_OP and HYBRID_SHARD give ZeRO-2-like and per-node variants.

Example: A 70B-style model first fits on 80 GB GPUs at 16 GPUs (about 72 GB each).

Related: ZeRO, All-gather

Lesson: ZeRO and FSDP

A generative adversarial network: a generator makes fake samples and a discriminator learns to tell them from real ones; the generator improves by fooling it.

Formal: $\min_G \max_D \mathbb{E}_x[\log D(x)] + \mathbb{E}_z[\log(1 - D(G(z)))]$; the generator usually minimizes the non-saturating loss $-\log D(G(z))$.

Example: For fixed $G$, the best discriminator is $D^*(x) = p_{\text{data}}/(p_{\text{data}} + p_g)$, which is 0.5 everywhere at equilibrium.

Related: Mode collapse, Loss

Lesson: GANs, generator versus discriminator

How well a model performs on new data it never saw during training.

Formal: Expected loss on the data distribution, estimated by held-out data that played no part in building the model.

Example: A model with 99% training accuracy and 70% test accuracy generalizes poorly.

Related: Overfitting, Data leakage

Lesson: The language of ML

An advantage estimate that blends one-step and multi-step lookaheads, trading bias from the critic against variance from the real returns.

Formal: $\delta_t = r_t + \gamma V(s_{t+1}) - V(s_t)$, $\hat A_t = \sum_{k \ge 0} (\gamma\lambda)^k \delta_{t+k}$. $\lambda = 0$ gives the one-step TD error; $\lambda = 1$ gives the Monte Carlo return minus $V$.

Example: With $\gamma = 0.99$, $\lambda = 0.95$, TD errors $-0.091$ and then $-0.1$ at the end give $\hat A = -0.091 + 0.9405 \times (-0.1) = -0.185$.

Related: Advantage, PPO

Lesson: PPO

The share of requests that met every latency target at once.

Formal: Fraction of requests with TTFT ≤ a, TPOT ≤ b, and end-to-end latency ≤ c.

Lesson: Serving LLMs

The direction and steepness of the loss surface: which way each parameter should move to increase the loss fastest.

Formal: $\nabla_\theta L = [\partial L/\partial\theta_1, \dots, \partial L/\partial\theta_P]$.

Example: For $L = x^2 + 2y^2$ at $(1, 1)$, $\nabla L = (2, 4)$.

Related: Gradient descent, Backpropagation

Lesson: Calculus for ML

Running several small batches and adding up their gradients before taking one optimizer step, to get the effect of a batch too big for memory.

Formal: Effective batch size = micro-batch size × accumulation steps. With a masked token loss, sum the per-token gradients and divide by the total token count across micro-batches, not the average of per-micro-batch means.

Example: Micro-batch 4 with 8 accumulation steps trains like a batch of 32.

Related: Calibration

Lesson: Fine-tuning in practice

Testing a hand-written backward pass by comparing its gradients with finite-difference estimates.

Formal: $g_{\text{num}} = [L(w + h) - L(w - h)]/2h$; relative error below about $10^{-7}$ (float64) passes, around $10^{-3}$ or more indicates a bug.

Example: A backward pass missing the ReLU mask gives relative error about 1; the correct one gives $2 \times 10^{-7}$.

Related: Backpropagation, Autograd

Lesson: Numerical stability

Saving memory in training by keeping only some layer inputs and recomputing the rest of the forward pass during backward.

Formal: Store the input of every $k$-th layer; during backward re-run each segment's forward to rebuild its activations. Gradients are identical; compute rises by about 33% (one extra forward per step). Also called activation recomputation.

Example: A 7B-style model at 2,048 tokens keeps about 9.13 GB of activations, or 0.822 GB checkpointing every layer.

Related: Activation, Backpropagation

Lesson: Gradient checkpointing

Capping how big one update can be, so a single bad batch cannot throw the weights far off.

Formal: Global-norm clipping rescales all gradients by $\min(1, c / \lVert g \rVert)$, where $\lVert g \rVert$ is the L2 norm over every parameter; the update keeps its direction.

Example: Gradients $[3, 4]$ and $[12]$ have norm 13; clipping at 1 multiplies all of them by $1/13$.

Related: Mixed precision, Gradient

Lesson: Gradient clipping and mixed precision

Repeatedly nudging parameters a small step downhill on the loss surface.

Formal: $\theta \leftarrow \theta - \eta\nabla_\theta L$; stable along curvature $\lambda$ only if $\eta < 2/\lambda$.

Example: With $\eta = 0.1$, $w = 1$ and gradient $-7.33$, the new weight is $1.733$.

Related: Learning rate, Gradient

Lesson: Gradient descent

Tying a model's answer to supplied evidence, so every claim can be traced to a source.

Formal: The fraction of an answer's claims supported by the provided context, often with citations.

Example: "Refunds are allowed within 30 days [2]," where source 2 contains that sentence.

Related: RAG, Hallucination

Lesson: Evaluating LLM systems

Attention where several query heads share one key/value head, which shrinks the KV cache with little quality loss.

Formal: With $n_h$ query heads and $n_{kv}$ KV heads ($1 < n_{kv} < n_h$), each group of $n_h / n_{kv}$ query heads attends using the same $K$ and $V$. $n_{kv} = n_h$ is multi-head attention (MHA); $n_{kv} = 1$ is multi-query attention (MQA).

Example: 32 query heads and 8 KV heads cut the KV cache to a quarter of MHA's: 131,072 instead of 524,288 bytes per token in a synthetic 8B-style model.

Related: KV cache, Attention

Lesson: LLM inference

Group relative policy optimization: RL for LLMs that scores several sampled answers per prompt and pushes up the ones better than the group average, without a value model.

Formal: For $G$ responses with rewards $r_i$, every token of response $i$ gets advantage $(r_i - \text{mean}(r)) / \text{std}(r)$; the loss is PPO's clipped surrogate plus a KL penalty to a reference model.

Example: Rewards $[1, 0, 0, 1]$ have mean 0.5 and std 0.5, giving advantages $[1, -1, -1, 1]$.

Related: PPO, Policy gradient, DPO

Lesson: Fine-tuning in practice

A fluent, confident statement that isn't supported by facts or the provided sources.

Formal: Generated content inconsistent with the source context or the world, arising because the model optimizes likely text rather than truth.

Example: Inventing a refund policy detail that appears in no document.

Related: Grounding, Calibration

Lesson: Adapting LLMs

The SVM's per-point penalty: zero for points safely past the margin, growing linearly for points inside it or on the wrong side.

Formal: $\max(0,\, 1 - y\,f(x))$ with labels $y \in \{-1, +1\}$.

Example: $y = +1$, $f(x) = -0.2$ gives loss $1.2$; $f(x) = 3$ gives 0.

Lesson: Support vector machines

Hierarchical navigable small world: a vector index that links each vector to its near neighbors in a layered graph and searches by hopping toward the query.

Formal: Each node keeps up to $M$ links per upper layer and $2M$ at layer 0; search descends greedily from the top layer, then runs best-first search with a candidate list of size efSearch.

Example: With $M = 32$ and 4-byte IDs, the bottom-layer links cost about 256 bytes per vector, 256 MB per million vectors.

Related: Approximate nearest neighbor search, Embedding

Lesson: Vector search

A regression loss that is squared for small errors and linear for large ones, so outliers can't dominate.

Formal: $\tfrac12 r^2$ if $|r| \le \delta$, else $\delta(|r| - \tfrac12\delta)$, with residual $r$.

Example: Residuals $[-0.5, 0.5, 0, 10]$ give mean Huber 2.44 ($\delta = 1$) versus MSE 25.1.

Related: Loss, Overfitting

Lesson: Loss functions

A setting you choose before training that controls how learning happens.

Formal: A value fixed outside the optimization of $\theta$, usually selected on a validation set.

Example: Learning rate, tree depth, $K$ in K-nearest neighbors.

Related: Parameter

Lesson: The language of ML

A unique token sent with a write so that retrying it does not repeat the side effect.

Formal: key = hash(task, action, payload); the server stores one result per key and returns it on repeats.

Lesson: Building reliable agents

Running a trained model to make predictions on new inputs.

Formal: Computing $f_{\theta^*}(x_{\text{new}})$ with fixed parameters; for LLMs, generating tokens from a prompt.

Example: Scoring a new transaction for fraud in 20 milliseconds.

Related: Training, Quantization

Lesson: The language of ML

The contrastive loss: each item must pick out its true partner from a batch of distractors.

Formal: $\ell_i = -\log \dfrac{\exp(\text{sim}(z_i, z_{i^+})/\tau)}{\sum_{k \ne i} \exp(\text{sim}(z_i, z_k)/\tau)}$, cross-entropy over candidates with temperature $\tau$; chance level is the log of the number of candidates.

Example: With 2 images (4 views) and $\tau = 0.5$, the lesson's example gives a loss of 0.871 versus a chance level of ln 3 = 1.099.

Related: Cross-entropy, Temperature, Cosine similarity

Lesson: Self-supervised vision

How much two regions overlap, as a fraction of the area they cover together; 1 is identical, 0 is disjoint.

Formal: $\text{IoU}(A, B) = |A \cap B| / |A \cup B|$, for boxes or pixel masks.

Example: Boxes (50, 50, 150, 150) and (60, 60, 160, 160) overlap in 8100 pixels out of 11,900, so IoU = 0.68.

Related: Non-maximum suppression, Metric

Lesson: Object detection and YOLO

An anomaly detector that counts how many random cuts it takes to separate a point from the rest; fewer cuts means more anomalous.

Formal: $s(x) = 2^{-E[h(x)]/c(\psi)}$, where $h$ is path length in random trees built on subsamples of size $\psi$ and $c(n) = 2(\ln(n-1) + \gamma) - 2(n-1)/n$.

Example: With $\psi = 256$, $c = 10.245$; a point isolated in 4 cuts on average scores 0.763.

Related: Anomaly detection

Lesson: Anomaly detection

The extra surprise you pay for believing distribution Q when reality follows P. Zero only when they match; not symmetric.

Formal: $D_{\text{KL}}(P \Vert Q) = \sum_x p(x)\log\frac{p(x)}{q(x)} = H(P, Q) - H(P) \ge 0$.

Example: For $P = [0.7, 0.2, 0.1]$ and $Q = [0.6, 0.25, 0.15]$, KL is 0.023 nats; reversed it is 0.024.

Related: Cross-entropy, Entropy, Distillation

Lesson: Information theory

Saved attention keys and values of earlier tokens, so generation doesn't recompute the whole prefix for every new token.

Formal: Cached $K, V$ tensors per layer; each new token computes only its own query against them, $O(n)$ per step.

Example: Long contexts and large batches make the KV cache the main GPU-memory cost of serving.

Related: Attention, Context window

Lesson: Self-attention

The correct answer attached to a training example.

Formal: The target value $y_i$ paired with features $x_i$ in supervised data.

Example: "Spam" or "not spam" for an email.

Related: Feature, Loss

Lesson: The language of ML

Training against slightly softened targets (a little probability on every wrong class) so the model can't push its logits to infinity.

Formal: target $= (1 - \varepsilon)\,\text{one-hot} + \varepsilon / K$.

Example: With $\varepsilon = 0.1$ and 10 classes, the true class's target is 0.91 and every other class gets 0.01.

Related: Cross-entropy, Calibration, Regularization

Lesson: Loss functions

Running a diffusion model on a compressed representation from a frozen autoencoder instead of on raw pixels, then decoding once at the end.

Formal: Train $E, \mathcal{D}$; set $z_0 = s_f E(x)$; train $\epsilon_\theta(z_t, t, c)$ on $z_t = \sqrt{\bar\alpha_t} z_0 + \sqrt{1 - \bar\alpha_t}\epsilon$; generate $x = \mathcal{D}(z_0 / s_f)$.

Example: Stable Diffusion 1.5 maps 512×512×3 images to 64×64×4 latents, 48 times fewer values.

Related: Classifier-free guidance, Flow matching

Lesson: Latent diffusion and Stable Diffusion

How big a step the optimizer takes each update.

Formal: The scalar $\eta$ in $\theta \leftarrow \theta - \eta\nabla L$; often scheduled (warmup, decay).

Example: Too high diverges (loss explodes); too low trains very slowly.

Related: Gradient descent, Hyperparameter

Lesson: Gradient descent

Starting training with a small learning rate and raising it linearly to its peak over the first steps, before decaying it.

Formal: $\eta_t = \eta_{\max}\, t / T_w$ for $t < T_w$, usually followed by cosine decay.

Example: max 3e-4 with 100 warmup steps gives 1.5e-4 at step 49.

Lesson: Training and regularization

Keeping a mixture-of-experts router from sending most tokens to the same few experts.

Formal: Either an auxiliary loss $\alpha E \sum_e F_e P_e$ (1 when uniform), or a per-expert selection bias nudged down for overloaded experts and up for underloaded ones, which leaves the gates unchanged.

Example: With 8 experts and top-2 on 12 tokens, balancing cut the busiest expert from 8 requests to 4 and dropped assignments from 6 to 2.

Related: Mixture of experts

Lesson: Mixture of experts

An anomaly score comparing a point's local density with the density around its neighbors.

Formal: The ratio of the neighbors' average local reachability density to the point's own; values well above 1 mean the point is in a sparser region than its neighbors.

Example: A point just outside a tight cluster gets a high LOF even if it is closer to the cluster than normal points in a wide cluster are to theirs.

Related: Isolation forest, Anomaly detection

Lesson: Anomaly detection

Turning log scores into probabilities without overflow or underflow by subtracting the largest score before exponentiating.

Formal: $\log\sum_i e^{s_i} = m + \log\sum_i e^{s_i - m}$ with $m = \max_i s_i$.

Example: scores $(-1000, -1001)$ underflow to $0/0$ directly; shifted to $(0, -1)$ they give probabilities $(0.731, 0.269)$.

Lesson: Naive Bayes

The raw score a model outputs before it is turned into a probability.

Formal: For logistic regression, $z = w\cdot x + b$; the probability is $\sigma(z)$. For multiclass models and LLMs, a vector of logits feeds softmax.

Example: A logit of 0 corresponds to probability 0.5.

Related: Softmax, Calibration

Lesson: Logistic regression

A cheap way to fine-tune a large model by training small add-on matrices instead of all the weights.

Formal: $W = W_0 + \frac{\alpha}{r}BA$ with frozen $W_0$ and trainable low-rank $B \in \mathbb{R}^{d_{\text{out}}\times r}$, $A \in \mathbb{R}^{r \times d_{\text{in}}}$.

Example: Rank 8 on a 4096 × 4096 matrix trains 65,536 parameters instead of 16.8 million.

Related: Fine-tuning

Lesson: Adapting LLMs

A single number measuring how wrong the model's predictions are, which training tries to minimize.

Formal: $L(\theta) = \frac{1}{n}\sum_i \ell(f_\theta(x_i), y_i)$, plus any regularization terms.

Example: Mean squared error for regression; cross-entropy for classification.

Related: Metric, Gradient descent

Lesson: The language of ML

Multiplying the loss by a big number before backward so tiny fp16 gradients do not round to zero, then dividing it back out.

Formal: $\nabla(S\mathcal{L}) = S\nabla\mathcal{L}$. A dynamic scaler skips the step and halves $S$ when any gradient overflows, and doubles $S$ after a run of clean steps (2,000 by default in PyTorch). Not needed with bf16.

Example: A gradient of $10^{-8}$ is 0 in fp16; times $2^{16}$ it is $6.55 \times 10^{-4}$, which fp16 can hold.

Related: Mixed precision

Lesson: Gradient clipping and mixed precision

The tendency of language models to use information at the start or end of a long context more reliably than information in the middle.

Formal: Answer accuracy as a function of the relevant passage's position in the input is often U-shaped (Liu et al., 2023); the size of the effect varies by model and task.

Example: Ordering retrieved chunks so the most relevant sit first and last, with the weakest in the middle, is a common mitigation.

Related: Context engineering, Context window

Lesson: Context engineering

The standard way to write down a sequential decision problem: what the agent sees, what it can do, what happens next, and what score it gets.

Formal: A tuple $(\mathcal{S}, \mathcal{A}, P, r, \gamma)$ of states, actions, transition probabilities $P(s' \mid s, a)$, rewards, and a discount, where the next state depends only on the current state and action (the Markov property).

Example: In a grid world the state is the cell, actions are the four moves, and each move costs −1 until the goal.

Related: Bellman equation, Discount factor

Lesson: MDPs, states, actions, and rewards

Mean absolute scaled error: forecast error divided by the error of the seasonal-naive forecast, so it compares across series of any scale.

Formal: $\text{MASE} = \text{MAE} / \big(\frac{1}{T-m}\sum_{t>m} |y_t - y_{t-m}|\big)$; below 1 beats seasonal naive.

Example: MAE 3.6 against a seasonal-naive MAE of 4.5 gives MASE 0.8.

Related: Baseline, Metric

Lesson: Time series

A pretraining task where some tokens are hidden and the model guesses them using the words on both sides.

Formal: Select about 15% of positions $\mathcal{M}$; replace 80% with [MASK], 10% with a random token, keep 10%; minimize $-\frac{1}{\lvert\mathcal{M}\rvert}\sum_{i \in \mathcal{M}} \log P(x_i \mid \tilde{x})$ with a bidirectional encoder.

Example: In "the cat [MASK] on the mat", the words on both sides make "sat" the likely answer.

Related: Attention, Cross-entropy, Embedding

Lesson: BERT and masked language modeling

A score for a yes/no classifier that uses all four cells of the confusion matrix and is 0 for any constant or random predictor, so it can't be gamed by predicting the majority class.

Formal: $\text{MCC} = \dfrac{TP \cdot TN - FP \cdot FN}{\sqrt{(TP+FP)(TP+FN)(TN+FP)(TN+FN)}}$, ranging from $-1$ to $+1$.

Example: $TP = 27$, $FP = 128$, $FN = 7$, $TN = 638$ gives MCC $= 16330 / 51026.9 = 0.320$, while accuracy is 0.831.

Related: Average precision, Metric

Lesson: Imbalanced data

The standard detection score: how well confidence ranks correct boxes above wrong ones, averaged over classes (and, for COCO, over IoU thresholds).

Formal: AP is the area under the interpolated precision-recall curve at one IoU threshold; mAP averages AP over classes; mAP@0.5:0.95 also averages over IoU thresholds 0.50, 0.55, ..., 0.95.

Example: Detections ranked TP, TP, FP, FP, TP against 3 objects give AP@0.5 = 0.867.

Related: Intersection over union, Metric

Lesson: Object detection and YOLO

A spectrogram whose frequency axis is pooled into bands spaced the way human pitch perception is: narrow at low frequencies, wide at high ones.

Formal: $S = \log(\max(P B^\top, \epsilon))$ where $P$ is the power spectrogram and $B$ is a matrix of triangular filters equally spaced on the mel scale $m = 2595 \log_{10}(1 + f/700)$.

Example: 80 mel filters turn a $98 \times 257$ power spectrogram into a $98 \times 80$ log-mel spectrogram, the input format of models like Whisper.

Related: Spectrogram, MFCC

Lesson: Spectrograms and mel features

How people judge a model's performance, which can differ from the loss used for training.

Formal: Any evaluation function of predictions and labels, possibly non-differentiable and threshold-dependent.

Example: RMSE in dollars, recall at a review budget, p95 latency, cost per request.

Related: Loss, Baseline

Lesson: The language of ML

Mel-frequency cepstral coefficients: a compact summary of each audio frame's spectral shape, made by compressing the log-mel spectrum.

Formal: Apply a type-II discrete cosine transform to each log-mel frame and keep the first (usually 13) coefficients; coefficient 0 tracks overall energy.

Example: A $98 \times 80$ log-mel spectrogram becomes a $98 \times 13$ MFCC matrix.

Related: Mel spectrogram

Lesson: Spectrograms and mel features

The smallest true improvement an experiment is designed to detect reliably.

Formal: The effect size $\delta$ at which the test has the chosen power $1 - \beta$ at significance $\alpha$; the sample size per arm scales as $1/\delta^2$.

Example: Detecting 10% → 11% conversion (a 10% relative MDE) at $\alpha = 0.05$ and power 0.8 takes 14,749 users per arm; halving the MDE takes about four times as many.

Related: Statistical power, P-value

Lesson: Experimentation and A/B testing

Training with most math in a 16-bit format (bf16 or fp16) for speed and memory, while keeping master weights and sensitive reductions in float32.

Formal: bf16 has float32's 8 exponent bits and 7 fraction bits; fp16 has 5 and 10 (max 65,504), so fp16 needs dynamic loss scaling to keep small gradients from flushing to zero.

Example: torch.autocast(device_type="cuda", dtype=torch.bfloat16) runs matmuls in bf16 without a loss scaler.

Related: Quantization, Log-sum-exp trick

Lesson: Numerical stability

A model whose feed-forward layers are split into many "experts", with a router sending each token to only a few of them, so it has many parameters but uses few per token.

Formal: Each MoE layer has $E$ expert networks and a router that picks the top $k$ per token. Total parameters set memory; active parameters (shared plus $k$ experts) set compute per token.

Example: 4B shared + 16 experts × 2B with top-2 routing: 36B total (72 GB in BF16), 8B active per token.

Related: Parameter, Inference

Lesson: LLM inference

A generator failure where it produces only a few kinds of output and ignores parts of the data distribution.

Formal: $p_g$ concentrates on a subset of the modes of $p_{\text{data}}$; shows up as low recall or coverage even when sample quality and losses look fine.

Example: A GAN trained on two clusters at -2 and +2 puts every sample near +2.

Related: GAN

Lesson: GANs, generator versus discriminator

An open standard for connecting AI applications to tools and data, so one integration works with many apps.

Formal: MCP clients inside a host application talk JSON-RPC (over stdio or HTTP) to MCP servers, which expose tools (callable functions), resources (readable data), and prompts (templates).

Example: An issue-tracker MCP server lets any compliant IDE or agent search and create tickets without custom glue code.

Related: Tool calling, Agent, Prompt injection

Lesson: Tool use and agents

Several LLM agents, each with its own prompt and context, that split a task and pass messages, usually coordinated by an orchestrator.

Formal: Designs range from fixed pipelines and fan-out/fan-in to orchestrator-worker (a model plans subtasks and delegates them) and peer swarms. Parallel workers cut the critical path from $ks$ to about $s + 2$ calls; total tokens depend on how much each worker re-explores.

Example: An orchestrator sends four research briefs to parallel workers and synthesizes their 400-token summaries.

Related: Agent, ReAct

Lesson: Multi-agent systems

DeepSeek's attention variant that caches one small compressed vector per token instead of full keys and values for every head.

Formal: Hidden states are projected down to a latent of size $r$ (kv_lora_rank), cached together with a shared RoPE key, and projected back up to per-head keys and values at attention time.

Example: DeepSeek-V3 caches $(512 + 64) \cdot 2 \cdot 61 = 70{,}272$ bytes per token, 71× less than full keys and values for 128 heads.

Related: Grouped-query attention, KV cache

Lesson: Open models: architecture walkthroughs

How much knowing one variable reduces uncertainty about another; catches non-linear relationships that correlation misses.

Formal: $I(X;Y) = \sum_{x,y} p(x,y) \ln \frac{p(x,y)}{p(x)p(y)}$, which is 0 exactly when $X$ and $Y$ are independent.

Example: For $y = x^2$ with $x$ symmetric around 0, correlation is about 0 but mutual information is large.

Related: Feature

Lesson: Feature engineering

Finding names of people, organizations, places, and other typed things in text, as spans.

Formal: A sequence-labeling task mapping tokens $x_1..x_n$ to tags $y_1..y_n$ (usually BIO), evaluated with entity-level precision, recall, and F1 on exact span and type matches.

Example: "Ada Lovelace joined Acme Corp" yields (Ada Lovelace, PER) and (Acme Corp, ORG).

Related: BIO tagging, Viterbi algorithm

Lesson: Named entity recognition

Cross-validation wrapped around a whole tuning procedure, so the reported score isn't inflated by picking the best of many trials.

Formal: Outer folds hold out test data; inside each, an inner CV selects hyperparameters; the outer scores are averaged.

Example: A search's best inner score of 0.88 next to a nested score of 0.85 shows the selection optimism.

Related: Hyperparameter, Overfitting

Lesson: Hyperparameter tuning

The cleanup step after a detector: keep the highest-scoring box, delete boxes that overlap it too much, repeat.

Formal: Greedy: sort boxes by score; repeatedly keep the top box and remove every remaining box with IoU above a threshold (often 0.45 to 0.7), usually per class.

Example: Three boxes on one object with IoUs above 0.6 to the best one collapse to that single box at threshold 0.5.

Related: Intersection over union

Lesson: Object detection and YOLO

The closed-form solution to least-squares linear regression, found by setting the gradient of the squared error to zero.

Formal: $X^\top X\,\theta = X^\top y$, solved with a linear solver rather than an explicit inverse.

Example: For $x = [1, 2, 3]$, $y = [2, 4, 5]$ it gives $w = 1.5$, $b = 0.667$.

Lesson: Linear regression

Turning a category into one 0/1 column per possible value, so no fake ordering is implied.

Formal: A category $c$ among $K$ values maps to the unit vector $e_c \in \{0,1\}^K$.

Example: color = green among {red, green, blue} becomes $[0, 1, 0]$.

Related: Target encoding, Feature

Lesson: Feature engineering

A free validation estimate for a random forest: score each row using only the trees that never saw it during training.

Formal: For row $i$, average the predictions of trees whose bootstrap sample excluded $i$; compute the metric over all rows.

Example: In scikit-learn, RandomForestClassifier(oob_score=True) exposes it as oob_score_.

Related: Bootstrap sample

Lesson: Random forests and gradient boosting

When a model learns the training data's noise and does worse on new data.

Formal: A large gap between training and validation performance, typically from high variance.

Example: A degree-15 polynomial with training MSE 0.056 and validation MSE 0.52.

Related: Regularization, Bias-variance trade-off

Lesson: Overfitting and bias-variance

How surprising the observed result would be if the change actually did nothing.

Formal: $P(\text{test statistic at least as extreme as observed} \mid H_0)$. It is not $P(H_0 \mid \text{data})$.