Sampling error

A margin of error is the half-width added to and subtracted from an estimate to form a confidence interval at a stated level. An estimate of 44 percent with a margin of plus or minus four percentage points is commonly displayed as an interval from 40 to 48 percent. The confidence level describes the long-run success of intervals produced by the method, not certainty about each respondent or a promise that this one estimate is correct.

Sampling error is only one part of total survey error. A margin does not measure omitted groups, self-selection, systematic nonresponse, leading language, inaccurate answers, interviewer pressure, processing mistakes, or opinion change after the interviews. If a biased frame excludes a distinctive group, increasing the number selected from that same frame can narrow the interval around a systematically mistaken estimate.

Weighting and complex designs can increase variance, producing an effective sample smaller than the raw interview count. Subgroups also contain fewer cases than the full sample. A margin reported for 1,500 adults should not be attached unchanged to an estimate based on 180 adults in one age group.

Comparisons require uncertainty for the difference. If two candidate estimates are close, the observed ordering may reflect chance sampling variation. Analysts often say the race is too close to call, which means the data cannot reliably distinguish the population ordering at the chosen confidence level; it does not mean support is exactly equal. The same caution applies to change. A two-point movement between independent polls can be noise because both estimates carry uncertainty.

Imagine that support for a school levy is reported separately for two small neighborhoods. A six-point gap looks substantial on a bar chart, but each neighborhood estimate comes from only 120 respondents. Without subgroup uncertainty, the graph invites more confidence than the data deserve.

Precision and accuracy answer different questions. Precision concerns how much estimates would vary across repeated samples. Accuracy concerns whether the estimate is near the intended population value after coverage, response, measurement, and adjustment are considered. A narrow interval can be precisely wrong; a wider interval from a sound design may be more trustworthy.

Read every margin conditionally: it describes random variation if the sampling assumptions hold. Then inspect the rest of the chain. Sample size can improve precision, but no arithmetic converts an uncovered population, a distorted response process, or an unclear question into a representative measure.

Video lesson: Science Behind the News: Opinion Polls & Random Sampling