Skip to content
ML to LLMs
Quizzes

Scaling and distributed training

Covers the lesson Scaling and distributed training.

Quiz: Scaling and distributed training

Question 1 of 10
Calculationeasy

With mixed-precision AdamW (bf16 weights and gradients, fp32 master copy and two fp32 Adam moments), how many bytes per parameter does training state take, before activations?