Intermediate
Open
Pro
Consumer Lag Keeps Growing Even After Adding More Consumers
A clickstream topic has 8 partitions and is consumed by a group
enrichment-service currently running 8 consumer instances (one
partition each). Traffic has grown, and consumer lag (the gap between
the latest offset in each partition and the offset each consumer has
processed) is steadily increasing across all 8 partitions. An engineer
proposes scaling the consumer group from 8 to 16 instances to "add
more processing power."
- Explain why this specific scaling move will not reduce lag, and what it will actually accomplish.
- Diagnose at least two other plausible root causes of the lag growth and how you'd distinguish between them.
- Describe the actual fix(es) available, including any tradeoffs.
Share this question