Fitting a Deep pCTR Model into 10 ms

Your ad server handles 400 k requests/s at peak; retrieval returns 1,500 candidates per request. The pCTR model must return calibrated scores within 10 ms p99, and the whole ad request must complete within 100 ms. Your current model is a DLRM-style network whose cost is dominated by embedding lookups: 40 sparse features per candidate, embedding tables totalling 300 GB.

1. How many candidate scores per second is that at peak, and why does it rule out scoring every candidate with the full model?
2. Design the scoring pipeline (stages, what runs where) so it fits the budget.
3. Where would you place caches, and what falls back to what if the embedding service degrades?