Repeated samples produce intervals. Ninety-five percent of them cover the unknown population mean.
From the chapter: Chapter 5: Statistics
Glossary: confidence interval
Transcript
A population has some true mean, a fixed unknown number. We draw a sample of size thirty and compute the sample mean.
Around that mean we draw an interval, plus or minus 1.96 standard errors. This is a 95 percent confidence interval.
Now repeat. Another sample, another interval. And again. And again. Twenty intervals, each from a different draw.
Most of the intervals straddle the true mean. A few miss, sometimes too high, sometimes too low.
The phrase 95 percent confidence does not mean the true mean is in any one interval with probability 0.95. The true mean is a fixed number. Either an interval contains it or not.
The 95 percent refers to the procedure. If we repeat sampling and intervalling many times, 95 percent of the resulting intervals will contain the truth in the long run.
Watch the count: out of these twenty, this many catch and this many miss. As the number of samples grows, the fraction of misses approaches one in twenty exactly.
This is the frequentist guarantee. Calibration in repeated play, not certainty about any single answer.