Quizzes
Serving LLMs
Covers the lesson Serving LLMs.
Quiz: Serving LLMs
Question 1 of 11
Calculationmedium
Two batch slots. Four requests arrive at step 0 with output lengths . Each busy slot emits one token per step. Under static batching (a new batch starts only when every slot is free, taking requests in order), what is the throughput in tokens per step?