Published
In this lesson
Wide tables, equations, and code scroll sideways. Swipe, or Tab to focus them and use the left and right arrow keys.
Expectation and variance
An expectation is a probability-weighted average. It describes a distribution’s center, when that average exists. It need not be an outcome you can actually observe.
Suppose a game pays 10 with probability 1/4 and zero otherwise. Its expected payout is 0 × 3/4 + 10 × 1/4 = 2.5. No play pays 2.5. The number describes a long-run average under repeated independent plays, not a prediction for your next payout.
| Outcome | Probability | Contribution to mean |
|---|---|---|
| 0 | 0.750 | 0.000 |
| 10 | 0.250 | 2.500 |
Mean: 2.500 · variance: 18.750 · standard deviation: 4.330
Measure deviations from the mean
Variance averages the squared distance from the mean. In the initial example it is (3/4)(0 − 2.5)² + (1/4)(10 − 2.5)² = 18.75. The standard deviation is its square root, about 4.33.
Squaring keeps positive and negative deviations from canceling. It also makes variance sensitive to extreme values. If payouts are measured in coins, variance is in coins squared; standard deviation returns to coins.
Make a prediction
Add five to both payouts. What happens to the mean and variance?
Explore the answer
The mean increases by five, to 7.5. Every deviation from the new mean is unchanged, so variance stays 18.75. Multiplying both payouts by two instead doubles the mean and quadruples variance.
Add expectations; check dependence for variance
The expectation of a sum is the sum of expectations whenever the expectations are finite. Independence is not required. This makes indicator variables useful: to count successes, give each trial a one for success and a zero for failure, then add their expectations.
Variances add for independent variables with finite variance. More generally there are covariance terms. Ten perfectly linked copies of the same payout are much riskier than ten independent plays, despite having the same expected total.
Sample means have their own spread
For n independent, identically distributed draws with variance σ², their mean has variance σ²/n and standard deviation σ/√n. Quadrupling n halves this standard error. It does not halve the variability of each underlying observation.
The law of large numbers explains convergence of averages. The central limit theorem describes their standardized shape under additional conditions. Neither says that every additional observation moves your running average closer to its target.
Some distributions have no mean
These calculations require the relevant moments to exist. A Cauchy distribution has a center of symmetry but no defined expectation or variance. Its sample mean does not settle around a population mean as sample size grows. Check the model’s tails before using an average as a summary.