Intermediate
Open
Pro
Diagnosing a 40-Minute Straggler Task in an Otherwise 2-Minute Job
You're paged because a nightly Spark job that normally finishes in
about 12 minutes has been taking over 45 minutes for the past three
nights, with no code changes and no meaningful growth in total input
data volume. Looking at the Spark UI's stage detail for the groupBy
aggregation stage, you see 199 tasks complete within 30-90 seconds of
each other, and one single task still running after 40 minutes.
- What is the most likely root cause, and what in the data would you check to confirm it?
- Adaptive Query Execution is enabled in this cluster's configuration (the Spark version is 3.3). Why might it not have automatically fixed this, or what would you check to see if it did try to?
- Propose a concrete fix, and explain why simply adding more executors to the cluster would not resolve this specific symptom.
Share this question