Practice — Case Study: Ad Click-Through Rate Prediction (Meta / Google Ads-style) (5 questions)

Your team ships a new pCTR model. Offline AUC goes from 0.812 to 0.826 and log loss improves slightly. After launch, the finance team reports that video-ad revenue rose 18 % while image-ad revenue fell 11 %, and several large image advertisers complain their cost per click doubled.

A per-segment check shows the new model's sum(pCTR) / sum(clicks) is 1.24 for video ads and 0.85 for image ads (the old model was ~1.0 for both).

1. Explain, using the eCPM formula, why the auction outcome changed even though ranking quality improved.
2. Two candidates cost the same bid ($2.00 CPC). The model predicts pCTR = 0.020 for a video ad and 0.017 for an image ad. What are their eCPMs? Using the calibration ratios above, what are the true expected CTRs, and which ad should have won?
3. What would you add to the system so this cannot happen again, and how often would you refresh it?