Prompt Evaluation and Versioning

prompt-engineering ends with a one-paragraph promise: prompts that ship to production are versioned, evaluated, and rolled back like code. This subject is the paragraph made real. "Prompts are code" is not a slogan — it is a load-bearing engineering claim, and the discipline it implies (a test suite, a CI gate, a release process, a rollback path) is exactly the discipline every other piece of code in your system already has. The reason this subject exists separately from prompt engineering itself is that the workflow is large enough, and skipped often enough, to deserve its own treatment: golden sets, a scoring methodology that doesn't fool itself, a CI pipeline that blocks bad diffs, an A/B layer that catches what offline evals can't, a registry that makes rollback a one-line change instead of an archaeology project, and tracing that tells you within the hour — not within the week — that something broke.

What breaks when teams skip this discipline is depressingly uniform across companies: someone edits a system prompt to fix one complaint ("be more concise," "stop over-explaining"), it ships straight to production because there's no gate to stop it, and three days later a different metric — refusal rate, tool-call frequency, completion length on a task that genuinely needs length — has quietly moved in the wrong direction. Nobody notices until support tickets pile up, and by the time someone traces it back to the prompt change, the team has burned days debugging a regression that a five-minute CI run would have caught before merge. The failure is never "the model got worse" — it's that nobody was measuring, so a real regression looked, for days, exactly like normal noise.

The other failure mode this subject addresses head-on is more insidious: silent model-version drift. Providers update models behind the same API endpoint and version alias, and a prompt that scored 94% against your golden set in March can quietly drift to 89% in June with zero code changes on your side, because the model interpreting your instructions is no longer exactly the model you tuned against. Without prompt and model version IDs logged on every trace, this kind of regression is nearly undiagnosable — it looks identical to "the prompt just isn't as good as we thought," and teams waste weeks second-guessing a prompt that was never the problem. Everything below — golden sets, scoring, CI, A/B, versioning, tracing — exists to make regressions like these visible, attributable, and reversible within minutes instead of weeks.