Law of Large Numbers

Keep flipping a coin. Early on the running fraction of heads jumps around — 6 out of 10, 27 out of 50, 53 out of 100. But as you pile up flips, that fraction gets pinned to the true probability and stops wiggling.

\overset{ˉ}{X}_{n} = \frac{1}{n} i = 1 \sum n X_{i} ⟶ E [X]

p (coin bias) 0.50 n (flips per run) 1000 runs 20

Dashed line: the expectation p. Light blue lines: 20 independent runs. Dark line: one focus run so you can track a single trajectory.

What to try

With default settings (p = 0.5, n = 1000, 20 runs), notice every line lands near 0.5 by the end, but some take much longer to get there than others.
Drop n to 50 and resample a few times. The curves end up all over the place — one run might finish at 0.7, another at 0.3.
Crank p to 0.2. All lines still converge, but to 0.2 instead of 0.5. The destination is the parameter; the journey is noise.

How fast?

The distance from the true mean shrinks like $σ / n$ — see the Central Limit Theorem. To halve the typical error you need 4× the samples. That’s why polling precision costs so much: quadrupling the sample size for one extra digit of accuracy.

Why it matters

Every time you average noisy data — polling, experiments, A/B tests, machine learning batches — you’re leaning on the Law of Large Numbers. It’s also why casinos win: the house edge is small, but over millions of bets the average outcome is indistinguishable from the expectation. The players see variance; the casino sees convergence.