Skip to content
ML to LLMs
Labs / AI Engineering

Recommendation funnel

Size candidate generation, ranking, and re-ranking stages against a latency budget and watch recall and cost move.

Theory and worked examples: ML system design

Recommendation funnel

Synthetic 10M-item catalog. Choose how many items each stage passes on and how expensive each stage is per item; watch latency, compute, and recall.

Items kept per stageCatalog: 10,000,000Candidate generation keeps: 1,000Ranking keeps: 100Re-ranking keeps: 20

Latency per request (bar) against the budget (dashed line)

Latency breakdown versus budget20 ms17 ms75 ms13 msbudget 150 ms
OverheadCandidate generationRankingRe-ranking
Fits the budget with 25.0 ms to spare and 73.4% synthetic recall. Each stage scores far fewer items than the one before, so the expensive models only ever see a few hundred or thousand items, never the 10M catalog (ranking all of it at 0.06 ms per item would take 600 s).