Back-of-Envelope Sizing for a Ranking Service
You are asked to size the ranking tier of a news-feed recommender. Assumptions: 20 M daily active users, 8 feed loads per user per day, peak traffic is 3× the daily average, and each feed load ranks 400 candidates using a model with 150 float32 features per (user, item) pair.
- Estimate average and peak QPS, and peak item-scorings per second.
- Estimate the feature bytes that must be assembled per request and the resulting peak feature-read bandwidth. What design decision does that force?
- If the item catalogue is 5 M items with 128-dimensional float32 embeddings, how big is the item embedding table, and how would you serve it?
1. QPS and scorings
Requests/day = 20 \times 10^6 \times 8 = 1.6 \times 10^8. Average QPS \approx 1.6 \times 10^8 / 10^5 \approx 1{,}600 (using \approx 10^5 seconds/day; the exact 86,400 gives ~1,850). Peak \approx 3 \times 1{,}600 \approx 5{,}000 QPS. Peak item-scorings = 5{,}000 \times 400 = 2 \times 10^6 per second — two million model evaluations per second at peak.
2. Feature bytes and bandwidth
Per (user, item) pair: 150 \times 4 = 600 B. If user features are, say, 100 of the 150 and fetched once, and 50 item features are fetched per candidate: 100 \times 4 + 400 \times 50 \times 4 = 400 + 80{,}000 \approx 80 KB per request. At 5,000 QPS that is \approx 400 MB/s of feature reads (roughly 240 MB/s if you count the full 600 B × 400 without sharing). Design consequence: item features cannot be fetched from a remote store per candidate on the request path — they must be cached in-process or served from an in-memory online store colocated with the ranker, and requests must be batched. Precomputing per-item feature vectors and refreshing them on a schedule is the standard answer.
3. Embedding table
5 \times 10^6 \times 128 \times 4 = 2.56 \times 10^9 bytes ≈ 2.6 GB. That fits comfortably in RAM on each ranking host, so replicate it to every replica (or memory-map it) rather than shard; refresh it when the item tower is retrained. If it were 500 M items (256 GB) you would shard by item id, quantise to int8 (4× smaller), or reduce dimensions. Always finish the estimate with the decision it forces.
Share this question