Explain Shrinkage to a Product Manager Who Distrusts Adjusted Numbers

A PM looks at a regional dashboard of conversion for three regions and objects: "Region S shows 2 conversions from 10 visitors, 20 percent, but your model reports 8.2 percent. Why are you changing the data?" The three regions have 10, 400 and 3,000 visitors with 2, 30 and 120 conversions. Overall the company converts about 7 percent, and the model treats this as worth about 100 visitors of information.

1. Compute the shrunken estimate for each region and the weight each estimate puts on its own data.
2. Explain in plain language why Region S moves a lot and the large region barely moves, and why this is not "changing the data".
3. Give evidence (a simulation design and its result) that shrinkage improves predictions on average.
4. State two situations where partial pooling is the wrong tool or needs care.