Estimating the Connection Server Fleet
A chat product has 300 million DAU. At peak, 25% of DAU are simultaneously connected via WebSocket. Each connection server can hold 600,000 concurrent idle connections.
- How many concurrent connections must the system support at peak?
- How many connection servers are needed (add a 30% safety margin)?
- What resource typically becomes the bottleneck per server before CPU does, and what is the practical fix?
1. Peak concurrent connections
0.25 x 300,000,000 = 75,000,000 concurrent connections.
2. Fleet size
75,000,000 / 600,000 = 125 servers for the raw connection count. Adding a 30% safety margin: 125 x 1.3 ≈ 163 servers (round up to ~165 to have even, easily-load-balanced capacity, and so a single server loss doesn't immediately threaten headroom).
3. Bottleneck and fix
Long-lived idle WebSocket connections are typically bounded by file
descriptor limits and per-connection memory/buffer overhead, not
CPU — an idle socket does no computational work but still consumes a
file descriptor and kernel/user-space buffer space. The fix is an
event-driven, non-blocking I/O model (epoll/kqueue-based servers
rather than one-thread-or-process-per-connection), which keeps memory
overhead low and avoids exhausting the OS thread/scheduler limits,
combined with raising OS-level file descriptor limits (ulimit -n)
appropriately for the box.
Share this question