Match a job Paths Subjects Questions Quizzes Pricing
Intermediate Open Pro

Analyse the Same A/B Data with a Bayesian and a Frequentist Test and Reconcile Them

A small test has A at 41 conversions out of 800 visitors and B at 56 out of 800. The frequentist analyst reports a two-sample z-test; the Bayesian analyst reports P(B \gt A) with flat priors.

  1. Compute the key outputs of both analyses (p-value, interval, P(B \gt A), expected loss).
  2. Explain precisely why one report says "not significant" and the other says "94 percent chance B is better". Is either wrong?
  3. How sensitive is the Bayesian answer to the prior?
  4. Write the two-sentence summary you would give the PM, and say what additional data would settle the question.

Share this question

← Back to Bayesian Statistics for Data Scientists practice

We use cookies for product analytics to improve OmniAtlas. See our Privacy Policy.