Advanced
Open
Pro
Diagnosing a Rising False-Reject Rate: Drift or Real Defects?
Part of the ML System Design Interview path →
Part of the MLOps & Production ML path →
Part of the Generative Vision & Image AI System Design path →
Over three weeks, the false-reject rate on Line 4 (a single SKU) has climbed from 0.4% to 2.1%, well past the scrap-budget ceiling. Operator overrides on this line have also risen, and when operators override, the parts they pass back onto the line are later confirmed good at final QA essentially every time. No maintenance or supplier changes are logged for this line in that window. A plant engineer suggests immediately retraining the anomaly detector on a fresh batch of recent images, including the ones that were rejected and overridden.
- Does the evidence point toward physical drift or a genuine rise in defect rate? Justify your answer using the details given.
- What would you check before accepting the "no maintenance or supplier changes logged" claim at face value?
- Evaluate the engineer's proposed fix. What would you do instead, and why is it cheaper and more correct?
Share this question