Message Queues & Event Streaming
Almost every system-design interview past the first ten minutes reaches a moment where a synchronous call is the wrong tool: checkout should not block on the warehouse, a "like" should not wait for every follower's feed to update, an image upload should not wait for the thumbnailer. The interviewer wants you to reach for asynchronous messaging — and then, crucially, to reason about what you just gave up. Once work happens "later, somewhere else", you have to answer: in what order? how many times? what if the consumer is down for an hour? what if the database write succeeded but the publish did not?
Weak candidates draw a box labelled "Kafka" and move on. Strong candidates can say why they chose a log over a queue, which key they partition on and what that does to ordering, whether their consumer is idempotent, and how a poison message is quarantined. This subject builds exactly that vocabulary and the mental models behind it.
The scope here is the messaging layer itself. Sagas and coordinating multi-service writes are covered in the Distributed Transactions & Consensus subject; retries, circuit breakers and dashboards in the Reliability & Observability Patterns subject; the sync-vs-async API decision itself in the API Design & Communication subject. Fan-out for social feeds is worked in Design a News Feed.
Why Asynchronous Messaging
A queue between two services buys four distinct things. Name the one you actually need in an interview — they are not the same.
| Benefit | What it means | Example |
|---|---|---|
| Decoupling | Producer does not know who consumes, or whether they are up right now | Checkout emits OrderPlaced; email, fulfilment and analytics each subscribe independently |
| Buffering / load levelling | Bursts are absorbed by the broker; consumers drain at their own steady rate | Flash sale: 20k orders/s for two minutes, fulfilment processes 2k/s |
| Fan-out | One event, many independent consumers, each with its own progress | One UserSignedUp event drives welcome email, CRM sync, fraud scoring |
| Retries & durability | Broker persists the message; a failed consumer retries without the producer knowing | Payment webhook processed later after the DB blip clears |
The cost is equally concrete: eventual consistency (the caller gets a 202 Accepted, not a result), harder debugging (a request is now a trail across services), and a new class of failures around ordering and duplicates. If the caller genuinely needs the answer to proceed — "is this card valid?" — a queue is the wrong tool.
Synchronous chain Async with a queue
client → checkout → fulfilment client → checkout ──► [queue] ──► fulfilment
(blocks, p99 = sum, 202 Accepted drains at its own rate
one failure = whole producer & consumer scale
request fails) independently