Intermediate
Open
Pro
Triage a Sudden Prediction-Distribution Shift
At 09:40 the on-call is paged: the fraud model's decline rate jumped from 1.1% to 3.4% in the last 30 minutes. Latency and error rates are normal. You have a dashboard with per-feature PSI, data-quality counters, and deploy markers.
- List, in order, the first checks you would run and what each one would tell you.
- You find that PSI on
merchant_category_codeis 2.9 and its "unknown category" rate went from 0.5% to 41% starting at 09:12. What most likely happened, and what is the right mitigation? - An engineer proposes kicking off the drift-triggered retraining job to "adapt the model to the new data." Explain why this is wrong here.
Share this question