Explaining an Offline Win That Fails Online

You retrain the ranking model with a new user-history feature. Offline evaluation on a held-out day shows AUC improving from 0.83 to 0.86 and NDCG@10 improving as well. You ship a 50/50 A/B test. After two weeks, session length and D7 retention are flat, and CTR is up only marginally (not statistically significant).

1. List at least three distinct mechanisms that could explain offline gains not showing up online, and be specific to this scenario.
2. What would you check first to figure out which mechanism is at play here?
3. The PM asks whether to ship anyway "since it can't hurt." How do you respond?