A/B Test Looks Like a Win — Until You Check the Guardrail Metrics

You're A/B testing a prompt change for a coding-assistant chat feature. The new prompt (v9) is meant to make the assistant propose code changes more proactively. After a week at 10% traffic, the primary metric — "user accepted the proposed code change" — is up 12% relative to the control (v8). Someone on the team wants to ramp to 100% immediately based on this result.

1. Before ramping, what guardrail metrics would you insist on checking first, and what specific failure mode is each one meant to catch?
2. Suppose the "tool-call error rate" guardrail metric is up 35% relative to control. Does this change your recommendation, and how would you investigate whether it's related to the prompt change?
3. What would make you confident enough to ramp to 100% despite an imperfect guardrail picture, versus what would make you hold or roll back?