Updated 12 September 2026

Markov chains

A Markov chain is a sequence of states where each transition depends only on the current state — not on how you got there. This “memoryless” property makes them tractable, and they appear throughout physics, biology, economics, and computer science.

What to notice

• All runs start in state A; the first plotted point is after one transition. Solid sample paths show occupation averages, while the dotted line shows the exact probability of being in A at each step.
• When at least one state can leave itself, this two-state model has the unique stationary distribution given below. With both states absorbing it is not unique. A stationary distribution and the limit from your starting state are different questions.
• |λ₂| near 1 (sticky transitions, like P(A→A) = 0.99): slow convergence — the chain stays near its starting state for a long time. |λ₂| near 0: fast mixing — just a few steps to forget the starting state.

The stationary distribution

For a 2-state chain, the stationary distribution satisfies the balance equations: the probability flux from A to B equals the flux from B to A. This gives:

For larger chains, the stationary distribution π is the left eigenvector of the transition matrix corresponding to eigenvalue 1. For a finite chain, a stationary distribution always exists and is unique if the chain is irreducible (every state is reachable from every other). Aperiodicity is additionally needed for convergence to it from every starting state. Countably infinite chains require further recurrence conditions.

Mixing time

The speed of convergence is controlled by the second-largest eigenvalue $|\lambda_2|$. The spectral gap 1 − |λ₂| determines how quickly the chain forgets its starting state. Small spectral gap = slow mixing = many steps needed to reach equilibrium. This is why PageRank (a Markov chain on the web graph) converges slowly on tightly clustered link structures.

Connection to the Law of Large Numbers

For a finite irreducible chain, the fraction of time spent in state A converges to π(A) with probability 1 — an ergodic theorem analogous to the Law of Large Numbers for i.i.d. samples. Unlike i.i.d. averaging, the chain’s samples are correlated, but the time average still converges to the space average.

Stationary does not mean convergent

Use the Alternating preset. Starting at A, the state is B at odd steps and A at even steps. The marginal distribution never settles, yet the fraction of time in A approaches 1/2. The stationary distribution (1/2,1/2) exists; the chain simply does not converge to it from a fixed starting state.

Now use Both absorbing. Each state stays where it started forever. There is no unique stationary distribution: every mixture of A and B is stationary. The formula for π(A) becomes 0/0, so the demo reports nonuniqueness instead of inventing a number.

For the default matrix, balance gives π(A) × 0.2 = π(B) × 0.3. Together with π(A)+π(B)=1, this yields π(A)=0.6. Multiply (0.6,0.4) by the displayed matrix to verify it stays unchanged.

Read a transition as a conditional probability

Each row of the matrix answers a different question: given that the current state is A, where will the next step go; or given that it is B, where will it go? The outgoing arrows from each state sum to one, including its loop. An arrow with probability zero is unavailable even though it remains dashed in the diagram so the layout stays readable.

At the default settings, the first step from A reaches A with probability 0.8. The second reaches A with probability 0.8×0.8+0.2×0.3=0.70. These are marginal probabilities across repetitions. On one realized path, the state is always either A or B; an occupation average summarizes the states visited so far.