Open models: architecture walkthroughs

Covers the lesson Open models: architecture walkthroughs.

Quiz: Open models: architecture walkthroughs

Question 1 of 9

Calculationmedium

A config has num_hidden_layers: 28, num_attention_heads: 28, num_key_value_heads: 4, hidden_size: 3584, no head_dim field, and every layer global. How many bytes of KV cache does one token need in BF16?