Intermediate
Open
Pro
Canary, Shadow and A/B for a New Ranker
Your team built a new heavy ranker (v2) to replace the current production ranker (v1) for a recommendation feed. Leadership wants to know "is it better", and your on-call team wants to know "is it safe to run in production" before it ever sees real users.
- Design the rollout sequence from first deployment to full traffic, naming each stage and what question it answers.
- Explain concretely why a canary that shows healthy latency and error rates is not sufficient evidence that v2 improves engagement.
- A colleague suggests skipping straight to a 50/50 A/B test to "get a faster read." What operational risk does this introduce that the staged approach avoids?
Share this question