Paths Subjects Questions Quizzes Pricing Search
Advanced Open Pro

Latency Budget With Fan-Out

A search-ranking request fans out to 10 shards in parallel to fetch candidate items; each shard has p99 latency of 15 ms and the request only returns once all 10 shards have replied. The rest of the request path (auth, feature fetch, ranking, network) totals a fixed 40 ms at p99. The overall SLO is p99 < 70 ms.

  1. Explain why the naive assumption "fan-out p99 = 15 ms" is wrong, and estimate the actual fan-out latency behavior.
  2. Given the 70 ms SLO and the fixed 40 ms elsewhere, is this design likely to meet the SLO? What is the mechanism of the failure if not?
  3. Propose two concrete mitigations and state the trade-off of each.

Share this question

← Back to Model Serving & Deployment practice

We use cookies for product analytics to improve OmniAtlas. See our Privacy Policy.