Paths Subjects Questions Quizzes Pricing Search
Intermediate Open Pro

Canary, Shadow and A/B for a New Ranker

Your team built a new heavy ranker (v2) to replace the current production ranker (v1) for a recommendation feed. Leadership wants to know "is it better", and your on-call team wants to know "is it safe to run in production" before it ever sees real users.

  1. Design the rollout sequence from first deployment to full traffic, naming each stage and what question it answers.
  2. Explain concretely why a canary that shows healthy latency and error rates is not sufficient evidence that v2 improves engagement.
  3. A colleague suggests skipping straight to a 50/50 A/B test to "get a faster read." What operational risk does this introduce that the staged approach avoids?

Share this question

← Back to Model Serving & Deployment practice

We use cookies for product analytics to improve OmniAtlas. See our Privacy Policy.