Skip to content
ML to LLMs
Quizzes

Mixture of experts

Covers the lesson Mixture of experts.

Quiz: Mixture of experts

Question 1 of 9
Calculationmedium

A token's router logits for four experts are [2.0,1.0,0.5,−1.0][2.0, 1.0, 0.5, -1.0]. With top-2 routing and gates renormalized over the chosen experts, what is the gate on expert 0? (4 decimals)