Your Coaching Program 'Worked' — Or Did It?
Monthly sales performance for your reps has a month-to-month correlation of about \rho = 0.5 (roughly half of any rep's deviation from the team average persists next month; the rest is noise). This month, you pull the 10 lowest performers — averaging 2 standard deviations below the team mean — into a mandatory coaching program.
Assume the coaching has zero true effect. On average, what z-score should you expect from those same 10 reps next month?
About -1 — the group "improves" by a full standard deviation doing absolutely nothing.
For jointly normal (or just roughly linear) variables, the best prediction of next month's standardized score given this month's is E[Z_{t+1} \mid Z_t] = \rho \cdot Z_t. Plug in \rho = 0.5 and Z_t = -2: 0.5 \times (-2) = -1. The bottom group is expected to close half its gap to the mean — from 2 SD below to 1 SD below — purely from statistics, with the coaching program contributing nothing.
The mechanism: "the 10 lowest performers this month" is not the same group as "the 10 truly worst reps." It's a mix of genuinely below-average reps and average-or-better reps who just had an unlucky month. Next month, the unlucky noise resets (in expectation) while each rep's true underlying skill persists — so the group's average drifts back toward the population mean by exactly the correlation coefficient, whether or not anyone intervened.
This is why every "we fixed the worst performers/stores/patients" anecdote is suspect until it's checked against a control group that got no intervention: the untouched bottom decile would show the same rebound. It's the identical mechanism behind the rookie-of-the-year sophomore slump and the "Sports Illustrated cover jinx" — extremes regress toward the mean because extremity is partly luck, and luck doesn't repeat.
Share this question