Sizing and Designing a Product-Page Cache
You are designing the caching layer for an e-commerce product page. Requirements: 10 million products, rendered product JSON averages 5 KB, peak traffic is 50,000 requests/s, inventory must be at most 5 seconds stale, and a single relational primary comfortably handles ~10,000 simple reads/s.
- Decide whether to cache the whole catalogue or only the hot subset, with the storage arithmetic.
- Estimate database load at a 99% cache hit ratio and at a 95% hit ratio, and say whether either is safe.
- A single product goes viral and receives 20% of all traffic. What do you do, and why can't the distributed cache alone handle it?
1. Whole catalogue vs hot subset
Whole catalogue: 10,000,000 × 5 KB = 50 GB (with per-entry overhead, call it ~60 GB). Hot-subset (80/20): 20% × 10M × 5 KB = 10 GB. Since 50-60 GB fits comfortably on a small Redis cluster (e.g. 4 shards × 16 GB + replicas), caching the whole catalogue is preferable here — it avoids the "long tail" of cold products causing a database read on every access, and it removes the need for an eviction policy to do any real work in steady state (misses become mostly new products and invalidations, not capacity evictions).
2. Database load
At 99% hit ratio: (1 − 0.99) × 50,000 = 500 QPS reach the DB — 20× under its 10k/s budget, safe with large headroom (e.g. a cold-cache event at 5× that is still under budget).
At 95% hit ratio: (1 − 0.95) × 50,000 = 2,500 QPS — still under 10k/s, but with much less headroom; a coincident cold-cache event or traffic spike could push it close to the limit. 99% is the target to design for; 95% is the floor before adding mitigations (see below).
3. Viral product / hot key
20% of 50,000 = 10,000 reads/s on a single key. A Redis node tops out around ~10^5 ops/s in aggregate across all its keys, so 10k/s on one key is survivable on one node in isolation, but it concentrates load on whichever single shard owns that key (consistent hashing spreads keys across nodes, not the load of one very hot key), and it leaves no headroom if the product gets hotter or the shard has other hot keys too.
Fix: add a short-TTL (1-5 s) in-process (L1) cache on each app server for the top-N hottest products. With ~50 app servers, the 10k/s becomes roughly 10,000 / (TTL_seconds × servers) requests to Redis per second — at a 2 s TTL, on the order of ~100/s reaching the distributed cache, three orders of magnitude less. The distributed cache alone can't solve this because it is still one logical destination per key; the fix has to add a layer closer to the reader that fans the read out across many independent processes.
Share this question