Bias in Data

Statistical bias is a systematic tendency for a collection or analysis method to miss the truth in a particular direction.

Open the full lesson →

Question → sample → inference. A population question leads to a sampling design, a data summary, and a conclusion with uncertainty. The return arrow requires revisiting earlier choices whenever verification exposes a design or interpretation problem.

Random error creates scatter; bias systematically shifts what the study tends to observe or conclude.

Bias versus variability

Sampling variability changes results unpredictably from sample to sample. Bias pushes results systematically because of design, measurement, processing, or analysis choices.

A very large sample reduces random error but can estimate the wrong quantity extremely precisely when bias remains.

Selection bias

Selection bias occurs when inclusion relates to the variable of interest. Convenience and voluntary-response samples often overrepresent people who are available or strongly motivated.

To reduce it, use a probability sampling method, a suitable frame, and transparent eligibility rules.

Worked example: online poll

A larger click count would not solve these design problems.

Measurement bias

An instrument or procedure can systematically over- or under-measure. Examples include a miscalibrated scale, inconsistent coding, different testing conditions, or a proxy that does not validly represent the intended concept.

Calibration, standardized protocols, blinding, and validated measures help reduce measurement bias.

Common mistakes

Using “bias” to mean personal disagreement. Statistical bias is systematic error in a process or estimator.

Claiming a large sample is representative. Selection mechanism matters.

Assuming neutral-looking wording is neutral to all respondents. Pilot and test questions.

Removing outliers without documented reasons. This can introduce analysis bias.

Listing limitations without changing the strength of the conclusion. Claims should match evidence quality.

Quick self-check

Who is in the target population, frame, invited sample, and responding sample?
Could selection, undercoverage, or nonresponse shift results?
Are wording and measurements neutral, reliable, and valid?
Could confounding explain the association?
Were cleaning, exclusions, outcomes, and analyses pre-specified and transparent?
Does the conclusion explicitly respect the likely biases and study design?