Published
In this lesson
Wide tables, equations, and code scroll sideways. Swipe, or Tab to focus them and use the left and right arrow keys.
Will Rogers phenomenon
You will learn: Identify when moving an unchanged observation raises both group means.
Start with: Expectation and variance
Can average scores rise in both groups even though nobody improves? Yes: changing group membership can do it. The Will Rogers phenomenon warns against interpreting comparisons whose group definitions or membership change between measurements.
Group A contains synthetic scores 25, 25, and 10, with mean 20. Group B contains 2, 2, and 2, with mean 2. Move the score 10 from A to B. A’s mean becomes 25; B’s mean becomes 4. Both increased, but the same six scores still sum to 66 and have overall mean 11.
Make the transfer
Filled: before. Open: after. Both group means strictly increase: Yes.
| Group | Before count | After count | Before mean | After mean | Change |
|---|---|---|---|---|---|
| A | 3 | 2 | 20.000000 | 25.000000 | 5.000000 |
| B | 3 | 4 | 2.000000 | 4.000000 | 2.000000 |
| All | 6 | 6 | 11.000000 | 11.000000 | 0.000000 |
One unchanged score moves between groups. The total score stays 66.000 and the population count stays 6. The overall mean is 11.000000 before and after. Group A and B are names; the controls do not force A to have the higher mean.
Filled bars show means before the transfer; outlined bars show means afterward. The overall pair coincides because no score or total population changed. The table keeps counts beside means: changing denominators are essential to the result.
“Remaining A mean” refers to observations that stay in A. The moved score is an additional observation in A before transfer. This parameterization lets you change its value without changing the observations left behind. Names A and B do not guarantee that one mean is higher.
When do both means rise?
Suppose r observations remain in A with mean a, and the moved observation has score x. The original A mean is (ra + x)/(r + 1); afterward it is a. The increase is (a − x)/(r + 1), positive exactly when x is below a.
If B initially has n observations with mean b, adding x gives (nb + x)/(n + 1). Its increase is (x − b)/(n + 1), positive exactly when x exceeds b. Both means therefore rise precisely when b < x < a. At equality one mean stays unchanged.
In the starting example, 2 < 10 < 25. The observation is low relative to the group it leaves and high relative to the group it joins. There is no contradiction in its different effects on the two averages.
What stays invariant?
Total score ra + nb + x and total count r + n + 1 stay fixed, so the grand mean stays fixed. Reconstructing that mean requires weighting group means by their current counts.
The unweighted mean of group means need not be preserved. Here it rises from (20 + 2)/2 = 11 to (25 + 4)/2 = 14.5. The new number gives a two-person and a four-person group equal weight; it is not the six-person mean. A simple average of category averages can obscure membership changes.
Make a prediction
Move a score of 30 instead, keeping the other starting scores. Can both means rise?
Explore the answer
No. Removing 30 lowers A’s mean because it exceeds the remaining mean 25, although adding it raises B’s mean. Both increases require the moved score to lie strictly between B’s original mean and A’s remaining mean.
Reading a before-and-after comparison
Membership may change when a threshold, definition, or measurement method changes. Before interpreting improved group averages as individual improvement, check whether the same observations remain in each group, whether their values changed, and whether the weighted overall mean changed.
This experiment isolates one transfer with unchanged values. It does not establish how much of a real change comes from reclassification. New entrants, missing observations, and actual score changes need separate accounting. Controls retain at least one observation in each group so neither mean is undefined, and bound synthetic scores between zero and one hundred.
Historical reference
Feinstein, Sosin, and Wells, The Will Rogers phenomenon describes the effect in stage migration. These scores are a separate arithmetic example, not a clinical estimate. Compare Simpson’s paradox for subgroup-weight reversals and expectation for the weighted-average identity.