Intermediate
Open
Pro
Capacity Planning for a Ranking Service
A ranking service must handle a peak of 15,000 QPS. Each request costs 12 ms of CPU time on the ranking service (excluding the separate model server). Target CPU utilisation is 55% to leave headroom for GC pauses and traffic spikes. Hosts have 16 cores each.
- Compute the number of cores, then hosts, needed at peak.
- Add 25% headroom for burst traffic and ensure at least one extra host's worth of capacity is free in each of 3 availability zones so a single AZ failure doesn't cause an overload. How many hosts total would you provision, and how are they distributed?
- The team wants to cut this cost. Name two changes to the serving design (not "buy fewer/cheaper hosts") that would reduce required capacity, and the risk of each.
Share this question