Diagnose Immortal-Time Bias in a Loyalty-Program Retention Claim

A retailer's analyst compares customers who joined the loyalty program with those who did not. Customers are offered membership at month 3 of their relationship; 60 percent of those still active accept. The analyst reports that members stay on average 23 months while non-members stay 16.8, and concludes that "joining the program adds six months of lifetime, worth about USD 40 each at USD 6.50 margin per month". The CFO plans to fund a USD 4M promotion on this basis.

Assume, for your analysis, that the program has no effect and every customer has a constant 5 percent chance of leaving each month.

1. Why does the comparison show members living longer even though there is no effect? Compute the expected lifetimes of the two groups under this assumption.
2. Show how a landmark analysis at month 3 changes the conclusion.
3. What alternatives would you recommend (time-varying covariate in a Cox model, matched design, experiment), and what would you tell the CFO about the USD 40?