Intermediate
Open
Pro
Design the Monitoring Stack for a Churn Model
You are deploying a subscription-churn model that scores every active user nightly and hands the top 5% to a retention team. The label ("cancelled within 30 days of scoring") only becomes known 30 days after each prediction.
- Lay out the monitoring layers you would build, from cheapest to most meaningful, with 2–3 concrete metrics per layer.
- What would you use as a proxy for model quality during the 30-day window before labels mature?
- How would you compute the "true" performance metric once labels arrive so that it lines up correctly with the drift signals?
Share this question