Intermediate
Open
Pro
A Stakeholder Wants to A/B Test a Problem That Isn't Diagnosed Yet
A product manager comes to you: "Our free-trial-to-paid conversion rate is 8%, well below the 15% benchmark for similar products. Let's A/B test three different pricing page layouts to see which one converts best — we have plenty of trial-signup traffic to make it statistically valid quickly."
- What's the risk in going straight to a 3-variant A/B test here, even though there's enough traffic to run one validly?
- What would you propose doing first, and what specific output would you want from it before designing any A/B test variants?
- Once that first step is done, how would you decide whether A/B testing is even still the right next move, versus another round of qualitative work?
Share this question