Bayesian Statistics for Data Scientists
The boundary: multi-armed-bandits-and-exploration owns the mechanics of Thompson sampling (sample from each arm's posterior, play the argmax), and hypothesis-testing and ab-testing own the frequentist machinery (p-values, power, MDE, sequential tests). This subject is the Bayesian inference layer underneath: how a posterior is built, summarised, checked and computed, how partial pooling rescues small segments, and how to turn a posterior into a ship/no-ship decision. It assumes the Bayes' theorem section of probability-fundamentals, and reuses the prior-as-regularizer table from regularization-l1-l2 without repeating it.
Interviews ask this as "explain Bayesian versus frequentist", "what is a conjugate prior?", "your Bayesian A/B test says 97 percent probability B is better, do you ship?", and "how would you estimate conversion for a country with 25 visitors?". The senior answer gives the posterior in numbers, says what the prior contributed, names the decision rule, and checks the model. Everything executable below was run (Python 3.12, NumPy 2.5, SciPy 1.18, PyMC 6.3, ArviZ 1.3, fixed seeds) and is marked "(verified)". Facts recalled from the literature without re-running are marked "(from memory)". Stan and NumPyro snippets are not installed or run here; they are labelled sketches.
Bayes' Rule as Inference
For a parameter \theta and data y:
| Piece | Name | Role |
|---|---|---|
| p(\theta) | prior | what the model believes before this data |
| p(y \mid \theta) | likelihood | how probable the data are for each \theta (a function of \theta, not a distribution over it) |
| p(\theta \mid y) | posterior | the updated belief; everything you report comes from it |
| p(y) | evidence (marginal likelihood) | normalising constant; the quantity model comparison uses and MCMC avoids computing |
Since p(y) does not depend on \theta, posterior \propto likelihood \times prior. That one proportionality is what every sampler, grid and conjugate formula below exploits.
Priors are a spectrum, not a switch.
- Flat (\text{Beta}(1,1), \mathcal{N}(0, 100^2)) is not "no assumptions": it is a strong claim that every value is equally plausible, including absurd ones. It is fine for a proportion with lots of data and dangerous for unbounded scale parameters (the prior predictive check below shows why).
- Weakly informative rules out impossible regions and sets the scale (a conversion rate between 0.5 and 30 percent; a log-rate near log 20, give or take 50 percent) without trying to decide the answer. This is the default in practice.
- Informative encodes real knowledge (last year's rate, a catalogue-wide distribution, an expert). Express it in pseudo-observations: \text{Beta}(4, 76) is "80 earlier visitors, 4 converted".
Sequential updating. Today's posterior is tomorrow's prior, and the order of batches does not matter. Starting at \text{Beta}(1,1) and seeing 3 of 40, then 5 of 60, then 2 of 50 conversions gives \text{Beta}(4,38), \text{Beta}(9,93) and finally \text{Beta}(11,141), identical to a single batch of 10 of 150 (verified). This is why Bayesian estimates are natural for streaming counters and bandits.
graph LR
P["Prior: pseudo-observations"] --> U["Update with likelihood"]
D["Data batch"] --> U
U --> Q["Posterior"]
Q --> S["Summaries: mean, interval, P(effect gt 0)"]
Q --> X["Posterior predictive: next batch"]
Q -->|"becomes tomorrow's prior"| P
S --> DEC["Decision with a loss function"]