Margin of error

Before you rely on it

What to check

Ask whether the figure covers the whole sample or just a subgroup.

The common error

Where people go wrong with margin of error

Reporting a subgroup finding with the headline margin. Subgroup margins are always wider, often much wider, and the caveat is usually left off the chart.

Worked through

Margin of error in plain numbers

A poll of 1,000 people finds 50% support for an idea; the margin of error at 95% confidence is about 1.96 × the square root of 0.5 × 0.5 ÷ 1,000, which is roughly 3.1 points. So the true figure is likely between 46.9% and 53.1%. Survey 400 people and the margin grows to 4.9 points. Survey 2,500 and it falls to 2.0. To halve the margin you need four times the sample. Two candidates polling at 51% and 49% in a 3-point margin poll are statistically tied; the headline gap is smaller than the noise.

Easily confused

Margin of error and its neighbours

Margin of error is not the total error of a poll. It covers only random sampling variation. Bias from who answers, wording of questions, weighting choices and people who do not respond are separate and often larger; it is also not a confidence level; the margin is tied to a confidence level, usually 95%. And it applies to each result separately: the margin for the difference between two candidates is roughly double that for one. Standard error is the underlying quantity, and the margin is a multiple of it.

In real decisions

Where margin of error turns up

Poll reports quote it. Quality control uses it to size samples. Researchers use it to decide how many participants to recruit. Business surveys, customer satisfaction studies and A/B tests all carry margins. Reading a poll well means checking the sample size, the margin and the date, and looking at the trend across polls. When a change in a number is smaller than the margin, treat it as possibly no change.

Know the edges

Where margin of error stops being reliable

The standard formula assumes a simple random sample, and real surveys are more complex. Subgroups have larger margins because they are smaller. It breaks down for extreme percentages and very small samples. It says nothing about bias. Treat it as the minimum uncertainty, and look for information about how the sample was drawn and weighted.

Quick answers

What people ask about margin of error

Does a bigger sample always mean a smaller margin? Yes, but with diminishing returns. The margin shrinks with the square root of the sample size, so going from 1,000 to 4,000 halves it, and going from 4,000 to 16,000 halves it again. Cost rises much faster than accuracy.

Why is the margin larger for subgroups? Because a subgroup is a smaller sample. A poll of 1,000 people may have only 150 in a given age group, so the margin for that group is well above three points.

Does the margin cover question wording and bias? No. It reflects random sampling only. Poor wording, coverage gaps and non-response can add error that the margin does not show.

What confidence level is standard? Ninety-five percent is the common default in polls. Ninety percent gives a narrower margin, and ninety-nine percent a wider one.