Skip to content
ŷ
ML to LLMs
Learn
Labs
Models
Math Lab
Interview prep
Cheat sheets
Glossary
Progress
Quizzes
ZeRO and FSDP
Covers the lesson
ZeRO and FSDP
.
Quiz: ZeRO and FSDP
Question 1 of 10
Multiple choice
easy
Why does ZeRO shard the optimizer state first?
It is the largest per-parameter term (12 of 16 bytes) and is only used in the optimizer step, not in forward or backward.
DeepSpeed requires it.
The optimizer state is the most important for accuracy.
Weights cannot be sharded.
Check