From QPS to Server Counts and an SLA
Your estimate for an API says 40,000 requests/s average and 3× at peak. A single application server comfortably serves ~2,000 requests/s; the relational primary can absorb ~5,000 writes/s; 10% of requests are writes.
- How many application servers do you deploy, and why more than the raw division suggests?
- Is a single database primary enough for the writes? What would you say about reads?
- The product team wants a 99.99% availability SLA. Translate that into downtime per year and name three concrete design consequences.
1. Application servers
Peak = 40k × 3 = 120k req/s. Raw division: 120,000 / 2,000 = 60 servers. Deploy more than 60 because (a) you want headroom to lose a server or an availability zone without exceeding capacity — with three AZs, losing one takes out a third of the fleet, so size each AZ to carry the load with one AZ down (≈ 90 servers total); (b) rolling deploys take capacity out of rotation temporarily; (c) running at 100% utilisation destroys tail latency. Something like ~90–100 servers across three AZs, targeting ~60–70% utilisation at peak, is a defensible answer. Say the reasoning, not just the number.
2. Database
Writes: 10% × 120k = 12,000 writes/s at peak, above the ~5k/s a single primary handles comfortably. Options: shard the primary by a key (e.g. user_id) across ~3–4 shards, or reduce write volume by batching / moving non-critical writes to an asynchronous queue. Reads: 108k/s at peak — far too many for the primary. Add a cache in front (assume 90%+ hit rate → ~10k/s reaching the DB) and read replicas for the rest. State the trade-off: replicas and caches introduce replication lag / stale reads, so identify which reads must go to the primary (e.g. read-your- own-writes right after a mutation).
3. 99.99% availability
Downtime budget ≈ 53 minutes per year (~4.4 min/month). Consequences:
- No single points of failure: multi-AZ for every tier, including the database primary with automated failover — a human paged at 3 a.m. already burns most of a month's budget.
- Deploy safety: canary or blue-green deploys with automatic rollback; a bad deploy that takes 15 minutes to notice consumes a quarter of the annual budget.
- Graceful degradation and dependency isolation: the API must not fail closed when a non-critical dependency fails; timeouts, circuit breakers and fallbacks keep the core path available. Also note that every serial dependency must itself be ≥ 99.99%, or the composite promise is arithmetically impossible.
Share this question