Advanced
Open
Pro
Revenue Numbers Doubled After a Flink Job Restart
After a Flink task-manager crash and automatic restart, the revenue-per-minute dashboard shows roughly double the expected revenue for a 12-minute window around the crash, then recovers to normal. On investigation, you find that the crash happened mid-checkpoint, and the job resumed from the last successfully completed checkpoint, not the in-progress one.
- Given that Flink resumed from the last successfully completed checkpoint (which is the documented, correct behavior), explain precisely where in the pipeline the double-counting must have actually happened, and why "the checkpoint mechanism failed" is probably not the right diagnosis.
- What specific implementation detail, if missing or wrong, would produce exactly this symptom?
- Describe the fix, and one test you'd add to your deployment process to make sure this class of bug can't reach production silently again.
Share this question