Skip to content
ML to LLMs
Quizzes

Fine-tuning in practice

Covers the lesson Fine-tuning in practice.

Quiz: Fine-tuning in practice

Question 1 of 10
Calculationmedium

LoRA with rank r=16r = 16 is applied to the gate, up, and down projections of every layer in a synthetic model with d=4096d = 4096, dff=11008d_{ff} = 11008, and 32 layers. Attention is not targeted. How many trainable parameters?