Textbook of AI

The Central Limit Theorem (CLT) is one of the most remarkable results in probability theory. It states that for a random sample of size $n$ from any distribution with finite mean $\mu$ and variance $\sigma^2$, the standardised sample mean $(\bar{X} - \mu) / (\sigma / \sqrt{n})$ converges in distribution to a standard normal as $n \to \infty$. The shape of the underlying distribution does not matter, provided the variance is finite.

The CLT explains why the Gaussian distribution is so ubiquitous in nature and in statistics. Many real-world quantities—measurement errors, biological traits, financial returns over short horizons—arise as sums or averages of many small, roughly independent influences, and the CLT guarantees they will be approximately Gaussian regardless of the details. In practice, the CLT "kicks in" surprisingly quickly: sample sizes of 30 or more often suffice for a reasonable Gaussian approximation, though heavy-tailed distributions require larger samples.

The CLT is the engine that drives classical statistical inference. It justifies the construction of approximate confidence intervals and hypothesis tests based on the normal distribution, even when the underlying data are far from Gaussian. In machine learning, it underlies the statistical properties of mini-batch gradient estimates, the analysis of stochastic approximation algorithms, and the construction of confidence intervals around performance metrics.