Advanced
Open
Pro
Recovering From Feed Cache Loss
A Redis node holding feed caches for a shard of 2 million users fails unrecoverably and its replica had fallen behind, so the feeds for those users are effectively gone. On-call wants to know: is this a data-loss incident?
- Is it a data-loss incident? Explain what is actually lost versus what can be recovered, and from where.
- Design the recovery path for an affected user's next feed read. What does it do differently from the normal (warm-cache) path, and what is the user-facing effect?
- If all 2 million users request their feed within the same minute after the incident is noticed (e.g. a mobile client retry storm), what could go wrong, and what would you do to prevent it?
Share this question