Hypothesis Testing & Statistical Inference

Hypothesis testing is the machinery a data scientist uses to say "this difference is probably not noise" — and, just as importantly, to refuse to say it when the evidence is not there. Every A/B readout, every "did the new model beat the old one", every "is churn higher in this segment" question is a hypothesis test underneath. Interviewers probe it relentlessly because the vocabulary is easy to parrot and hard to use correctly: most candidates can say "p-value" and far fewer can define it without saying something false.

This subject builds the logic once, cleanly — null and alternative, test statistic, sampling distribution, p-value, errors and power — and then walks through the standard toolkit: z and t tests (including Welch and paired variants), chi-square and proportion tests, non-parametric alternatives, confidence intervals and their duality with tests, effect sizes, multiple-comparison corrections, bootstrap intervals and sample-size planning. There is a fully worked two-sample example with real numbers you can reproduce by hand.

Two boundaries: the probability foundations (distributions, the Central Limit Theorem, expectation and variance) are covered in the Probability Fundamentals subject and assumed here. The design specifics of online experiments — randomisation units, guardrail metrics, peeking and sequential testing, variance reduction with CUPED — are covered in the A/B Testing subject; this subject provides the statistical engine those designs run on.