Paths Subjects Questions Quizzes Pricing Search
Intermediate Open Pro

Capacity Planning for a Ranking Service

A ranking service must handle a peak of 15,000 QPS. Each request costs 12 ms of CPU time on the ranking service (excluding the separate model server). Target CPU utilisation is 55% to leave headroom for GC pauses and traffic spikes. Hosts have 16 cores each.

  1. Compute the number of cores, then hosts, needed at peak.
  2. Add 25% headroom for burst traffic and ensure at least one extra host's worth of capacity is free in each of 3 availability zones so a single AZ failure doesn't cause an overload. How many hosts total would you provision, and how are they distributed?
  3. The team wants to cut this cost. Name two changes to the serving design (not "buy fewer/cheaper hosts") that would reduce required capacity, and the risk of each.

Share this question

← Back to Model Serving & Deployment practice

We use cookies for product analytics to improve OmniAtlas. See our Privacy Policy.