Bias in Sampling

A rigorous guide to coverage, selection, nonresponse, response, and convenience bias in samples.

Question → sample → inference. A population question leads to a sampling design, a data summary, and a conclusion with uncertainty. The return arrow requires revisiting earlier choices whenever verification exposes a design or interpretation problem.

Precise definition

Sampling bias is systematic error caused by a sampling or response process that tends to make the sample statistic differ from the population parameter in a particular direction. Unlike random sampling variability, increasing the size of a biased sample does not reliably remove the error.

Notation and mathematical language

Coverage bias arises when the sampling frame omits groups; voluntary-response and convenience samples select participants nonrandomly; nonresponse bias occurs when responders differ relevantly from nonresponders; response bias arises from wording, interviewer effects, or inaccurate answers. Selection probability is each unit's chance of inclusion.

Conceptual picture

A sample can be very large and very precise about the wrong subset. Random selection aims to balance unmeasured characteristics in expectation, while random assignment addresses causal comparison after sampling; the two forms of randomization solve different problems.

Fully worked example

Interpretation and application

Sampling bias affects polls, health studies, customer analytics, and school surveys. A result should be generalized only to a population supported by the sampling design. Association within a biased sample can also differ from the target population and does not establish causation.