Important formulas cheat sheet

Concept	Formula	Lesson
Dot product	$x\cdot w = \sum_i x_i w_i$	Vectors
Linear model	$\hat{y} = w\cdot x + b$	Linear regression
MSE	$\frac{1}{n}\sum(y - \hat{y})^2$	Linear regression
MSE gradients	$\frac{2}{n}\sum(\hat{y}-y)x$ , $\frac{2}{n}\sum(\hat{y}-y)$	Linear regression
Ridge / lasso penalty	$\lambda\sum w^2$ / $\lambda\sum\lvert w\rvert$	Regularization
Sigmoid	$\sigma(z) = 1/(1 + e^{-z})$	Logistic regression
Binary cross-entropy	$-[y\ln p + (1-y)\ln(1-p)]$	Logistic regression
Softmax with temperature	$p_i = e^{z_i/T}/\sum_j e^{z_j/T}$	Decoding
Precision, recall, F1	$\frac{TP}{TP+FP}$ , $\frac{TP}{TP+FN}$ , $\frac{2PR}{P+R}$	Metrics
Bayes' rule	$P(A\mid B) = \frac{P(B\mid A)P(A)}{P(B)}$	Probability
Euclidean distance	$\sqrt{\sum(q_j - x_j)^2}$	KNN
Gini impurity / entropy	$1 - \sum p_k^2$ / $-\sum p_k\log_2 p_k$	Decision trees
Boosting update	$F_m = F_{m-1} + \eta h_m$	Ensembles
K-means objective	$\sum_i\lVert x_i - \mu_{c(i)}\rVert^2$	K-means
PCA	$Cv = \lambda v$ ; ratio $\lambda_k/\sum\lambda$	PCA
TF-IDF	$\text{TF}(t,d)\times\ln(N/\text{DF}(t))$	Text to vectors
Cosine similarity	$\frac{a\cdot b}{\lVert a\rVert\lVert b\rVert}$	Text to vectors
Layer	$a = f(Wx + b)$	Forward pass
Gradient descent	$\theta \leftarrow \theta - \eta\nabla_\theta L$	Gradient descent
Sigmoid + BCE error signal	$\partial L/\partial z = \hat{y} - y$	Backpropagation
Conv output size	$\lfloor (H - k + 2p)/s\rfloor + 1$	CNNs
Self-attention	$\text{softmax}(QK^\top/\sqrt{d_k})V$	Self-attention
Perplexity	$\exp(\text{mean token loss})$	Tokenization
LoRA	$W = W_0 + \frac{\alpha}{r}BA$	Adapting LLMs
nDCG	$\frac{\sum \text{rel}_i/\log_2(i+1)}{\text{IDCG}}$	LLM evaluation
Little's law	$L = \lambda W$	Production architecture