Practice — Case Study: Marketing Uplift Targeting (5 questions)
The Coupon Budget Is Cut in Half Mid-Quarter Permalink →
Your uplift targeting policy contacts 15 percent of a 200,000-customer base each cycle (30,000 contacts). The true economics, known from a synthetic simulation, are: margin per order before discount EUR 30, discount EUR 6, contact cost EUR 0.05. Finance now says the promo budget is halved to 7.5 percent of the base.
Offline results for the uplift policy at different budgets:
| Budget | 5% | 7.5% | 10% | 15% | 30% |
|---|---|---|---|---|---|
| Incremental orders | 2,799 | 3,660 | 4,356 | 5,456 | 7,504 |
| Profit (EUR) | 61,492 | 78,629 | 91,493 | 109,430 | 128,257 |
- By what percentage do incremental orders and profit fall when the budget goes from 15 percent to 7.5 percent? Why is it far less than 50 percent?
- Do you retrain the model? What exactly changes in the policy?
- A stakeholder suggests keeping the 15 percent audience and just sending a smaller coupon. Why is that a different (and riskier) decision than the one you were asked about?
- What would you check in the evaluation before committing to the new threshold?
Share this question
Only One Percent of Last Year's Customers Were Randomized Permalink →
The only randomized data you have is a past campaign where 1 percent of 200,000 customers (about 2,000) received a coupon at random and 99 percent did not. A colleague wants to build a T-learner with two gradient-boosted models, one per arm, "because it is the standard".
In a controlled simulation (same population, 140,000 users for training, 60,000 for testing, average of three seeds), the true EUR value per user of each learner's top-15-percent audience was:
| Treated share | S | T | X | DR | Response model |
|---|---|---|---|---|---|
| 1% | 2.54 | 2.52 | 3.23 | 3.01 | -3.88 |
| 5% | 3.40 | 3.50 | 3.69 | 3.73 | -3.86 |
| 50% | 3.86 | 3.92 | 3.94 | 3.95 | -3.57 |
The oracle (true effect known) earns EUR 4.21 per user.
- Which learner would you pick for the 1 percent log, and why does the T-learner struggle there?
- How would you evaluate and compare the candidates when there are only a few hundred treated users in any holdout?
- What would you do differently in the next cycle so you are not stuck at 1 percent forever?
- Why does the response model not improve as the treated share grows?
Share this question