Scaling laws

Covers the lesson Scaling laws.

Question 1 of 9

Calculationeasy

Using $C \approx 6ND$ , how many FLOPs does it take to train a 7B-parameter model on 2 trillion tokens? Give the answer in units of $10^{22}$ FLOPs.