Paths Subjects Questions Quizzes Pricing Search
Intermediate Open Pro

Triage a Sudden Prediction-Distribution Shift

At 09:40 the on-call is paged: the fraud model's decline rate jumped from 1.1% to 3.4% in the last 30 minutes. Latency and error rates are normal. You have a dashboard with per-feature PSI, data-quality counters, and deploy markers.

  1. List, in order, the first checks you would run and what each one would tell you.
  2. You find that PSI on merchant_category_code is 2.9 and its "unknown category" rate went from 0.5% to 41% starting at 09:12. What most likely happened, and what is the right mitigation?
  3. An engineer proposes kicking off the drift-triggered retraining job to "adapt the model to the new data." Explain why this is wrong here.

Share this question

← Back to ML Monitoring, Drift & Retraining practice

We use cookies for product analytics to improve OmniAtlas. See our Privacy Policy.