Hypothesis Testing

A rigorous guide to null hypotheses, test statistics, p-values, errors, power, assumptions, and responsible conclusions.

Open the full lesson →

Statistical inference compares an estimate’s uncertainty with a stated claim. Four interval estimates appear against a vertical reference claim. Intervals that cross the line and intervals that miss it make compatibility with the reference visible without reducing uncertainty to a point estimate.

Precise definition

A hypothesis test compares data with a null model $H_0$ using a pre-specified statistic. The p-value is the probability, assuming $H_0$ and all test-model assumptions, of obtaining a statistic at least as incompatible with $H_0$ as the observed value. It is not the probability that $H_0$ is true.

Notation and mathematical language

A significance level $\alpha$ controls the long-run Type I error rate for the defined procedure under $H_0$. Type II error is failing to reject when a specified alternative is true; power is one minus that probability. One- and two-sided alternatives define different extremeness regions and must be chosen before seeing results.

Conceptual picture

Testing calibrates surprise under a model. A small p-value indicates tension between data and the null package, which includes sampling, independence, distribution, and analysis choices. Effect size and confidence interval answer magnitude questions that a threshold decision cannot.

Fully worked example

Interpretation and application

Tests support experiments, quality control, and observational studies. Random assignment can justify causal comparisons under compliance and design conditions; a low p-value from observational association does not remove confounding.