Case Study: Migrate a Production Prompt Suite Across a Model Deprecation
A full AI-engineering interview answer for a 60-day forced migration of 40 production prompts: baselining evals that never existed, reading a regression report, rewriting for the new model, re-laying-out for the cache, staged rollout, long-tail drift, and the registry you build so the next deprecation is boring
Model interview answer for the operational case study behind prompt versioning: your provider announces a 60-day deprecation of the model behind 40 production prompts. Walk the full migration — inventorying blast radius before touching anything, mining golden sets retroactively from production traces, categorizing the regressions a first eval run on the new model surfaces, rewriting prompts to drop old-model workarounds and adopt new-model features, re-laying-out prompts for the prefix cache, a staged shadow/canary/interleave rollout with concrete rollback triggers, the long-tail failure modes (judge drift, silent behavior change) that surface weeks after the migration looks done, and turning the fire drill into a standing prompt registry.
Practice questions (5)
-
View →
Triaging 40 Prompts Before Touching Any of Them
Intermediate · Free -
View →
Building Golden Sets Retroactively Under a Deadline
Advanced -
View →
Reading a First-Run Regression Report
Advanced -
View →
Choosing a Rollout Strategy and Rollback Triggers
Advanced -
View →
The Migration Looks Done. What Could Still Go Wrong?
Advanced