The Coupon Budget Is Cut in Half Mid-Quarter
Your uplift targeting policy contacts 15 percent of a 200,000-customer base each cycle (30,000 contacts). The true economics, known from a synthetic simulation, are: margin per order before discount EUR 30, discount EUR 6, contact cost EUR 0.05. Finance now says the promo budget is halved to 7.5 percent of the base.
Offline results for the uplift policy at different budgets:
| Budget | 5% | 7.5% | 10% | 15% | 30% |
|---|---|---|---|---|---|
| Incremental orders | 2,799 | 3,660 | 4,356 | 5,456 | 7,504 |
| Profit (EUR) | 61,492 | 78,629 | 91,493 | 109,430 | 128,257 |
- By what percentage do incremental orders and profit fall when the budget goes from 15 percent to 7.5 percent? Why is it far less than 50 percent?
- Do you retrain the model? What exactly changes in the policy?
- A stakeholder suggests keeping the 15 percent audience and just sending a smaller coupon. Why is that a different (and riskier) decision than the one you were asked about?
- What would you check in the evaluation before committing to the new threshold?
1. The arithmetic. Incremental orders: 3,660 / 5,456 = 0.67, so a 33 percent fall. Profit: 78,629 / 109,430 = 0.72, a 28 percent fall. Half the budget gives about two thirds of the orders and 72 percent of the profit, because the contacts you drop are the marginal ones: the policy fills its budget in descending order of expected value, so the last half of the list is also the least productive half. Orders per EUR 1,000 of promo spend actually rise (about 101 to 117). The curve is concave, which is the whole point of ranking by incremental effect. (A response-model policy would not show this: its profit is negative at every budget, so halving the budget merely loses less.)
2. Retrain or not. No retraining is needed for a budget change. The model ranks; the policy is a rule on top of the ranking. What changes is the production threshold: the score at the 92.5th percentile instead of the 85th. Better still, set the cutoff by marginal value per euro: contact while the estimated value \hat v(x)=(m-c)\hat\tau(x)-c\hat\mu_0(x)-s per euro of expected spend exceeds the cutoff that exhausts the budget. A lesson of the case is that the cost-aware rule (\hat v\gt0 without a budget) would have contacted 37.6 percent of the base and earned EUR 152,240, so the shadow price of the budget constraint is high and finance's cut is costing real profit (EUR 109,430 down to about EUR 78,600 per send in this simulation). That is the number to bring to the finance conversation.
3. A smaller coupon is a different treatment. The model estimates the effect of this coupon (a EUR 6 discount). A smaller discount changes both the redemption cost and the persuadable share: some persuadables need the full incentive, and the value function v(x) has different parameters. You would need a new randomized discount- depth arm, or at least a dose-response experiment (see the pricing lesson), before assuming the ranking holds. Keeping the audience and changing the offer also removes the guarantee that the offline Qini and calibration apply.
4. Checks before committing. (a) Calibration of the predicted effect in the top 7.5 percent, where the scores are most extreme and the sample (about 750 treated users in the randomized log at that depth) is small: the 5-percent-depth estimate was off by 377 conversions in the simulation. (b) Bootstrap intervals on the Qini value at 7.5 percent. (c) A holdout inside the new audience so you can measure the realized lift after the change. (d) Guardrails (unsubscribes per contact) are not expected to improve much, because sleeping dogs are only 2.6 percent of this population.
Share this question