Advanced
Open
Pro
Design and Defend Priors for a New-Checkout Conversion Test
Your site's checkout conversion has hovered around 3.2 percent for a year. You will test a redesign with 4,000 visitors per arm, expecting 3.75 percent (150 conversions) for control and 4.375 percent (175) for the redesign. A colleague says "just use a flat prior, it is objective". Another says "use a Beta(32, 968) prior, we know the rate".
- Propose a prior for both arms. Express its strength in pseudo-observations and say how you chose it.
- Use a prior predictive check to compare the flat prior, your prior and the Beta(32, 968) prior.
- Show the effect of the three priors on P(B \gt A) and describe the sensitivity analysis you would report.
- Why should the prior be the same for both arms?
Share this question