Learning from samples
Move from a noisy sample to an estimate and an honest statement of uncertainty.
- Step 1
Expectation and variance
Define the center and spread before discussing convergence.
Checkpoint: What changes if every payout increases by five?
Expectation increases by five. Variance stays unchanged because distances from the new mean are unchanged.
- Step 2
Averages settle
Understand convergence and why errors need not shrink at every step.
Checkpoint: Must every new observation improve a running average?
No. Convergence is not monotone. A later observation can increase the current error while the long-run convergence statement remains valid.
- Step 3
The shape of sample means
Separate the original data from the distribution of an average.
Checkpoint: Does increasing repetitions make the normal approximation better at fixed sample size?
It makes the simulated distribution of sample means less noisy. The underlying sampling distribution and its approximation error at that sample size do not change.
- Step 4
Intervals and coverage
Distinguish nominal confidence from achieved coverage.
Checkpoint: Does a nominal 95% procedure always achieve 95% coverage?
No. Coverage depends on the data model and procedure. The usual Student interval is exact for iid normal observations, but may undercover for small skewed samples.
- Step 5
Resampling
Use the empirical distribution, while recognizing what it cannot recover.
Checkpoint: Can a million resamples recover a rare event missing from the original data?
No. Resampling the empirical distribution only repeats values present in the original sample. More resamples reduce simulation noise, not missing-data bias.
- Step 6
Testing a hypothesis
Interpret null tail areas, effect sizes, and power.
Checkpoint: Is a p-value the probability that the null hypothesis is true?
No. It is a tail probability under the specified null model, not a posterior probability for that model.
After each lesson, try to explain its main result without looking at the formula. Then change one assumption: which part of the answer would change?