Published 12 September 2026

Conjugate priors

How does an uncertain parameter become a prediction for new data? Conjugacy makes the update algebra simple: a chosen prior family remains the posterior family under a particular likelihood. The posterior prediction still requires averaging over that uncertainty.

Start with Bayes’ rule, the beta distribution, and the gamma distribution. The two models below use independent observations conditional on one shared parameter. They do not assume the unknown parameter is redrawn separately for every observed trial.

Gamma prior, Poisson observations

Suppose a count c comes from a constant-rate Poisson process observed for exposure t. Its likelihood as a function of λ is proportional to λᶜe⁻ᵗλ. Choose a Gamma(α,β) prior in the shape–rate convention, with mean α/β and variance α/β². Multiplying prior and likelihood gives:

The posterior is Gamma(α+c,β+t). Count adds to shape; exposure adds to rate. Observing zero events for ten hours differs from observing zero events for one hour. The zero counts add no shape, but the longer absence adds more exposure evidence.

In the default example, Gamma(2,1) followed by four events in two hours gives Gamma(6,3). The rate mean stays 2 per hour while its variance falls from 2 to 2/3 in squared rate units. Matching the prior mean does not make the data uninformative.

The posterior mean can be written as a weighted average of the prior mean α/β and observed rate c/t, with weights β/(β+t) and t/(β+t). This is an algebraic interpretation of the parameters, not a claim that the prior came from actual historical events.

Predict counts by averaging rates

For future exposure h, first condition on λ: K|λ is Poisson(hλ). Integrating its probability against the gamma posterior produces a negative-binomial count distribution. Writing a,b for posterior shape and rate:

This parameterization counts future events. It does not require a to be an integer or an actual sequence stopped after a successes. With Gamma(6,3) and h=1, the predictive mean is 2 and variance is 8/3. Its zero-count probability is (3/4)⁶=0.1779785. Plugging the posterior mean 2 into a Poisson distribution instead gives e⁻²=0.1353353.

Total variance explains the extra spread: the predictive variance is hE[λ|data]+h²Var(λ|data). Both models have the same predictive mean, but the plug-in model omits uncertainty about λ.

Beta prior, Bernoulli trials

For s successes and f failures under one fixed success probability p, a Beta(α,β) prior becomes Beta(α+s,β+f). Here β is a second shape, not an exposure rate. Multiplying pˢ(1−p)ᶠ into the prior adds the two exponents directly.

Predicting m new trials gives a beta-binomial distribution: a binomial probability averaged over the beta posterior. For a Beta(2,2) prior and 4 successes/2 failures, the posterior is Beta(6,4). In three future trials, the mean number of successes is 1.8. The probability of none is:

Plugging in p=0.6 gives 0.4³=0.064 instead. Future trials are independent conditional on p, but are positively associated after averaging over a shared uncertain p. Seeing one future success would also teach you something about the probability of the next.

Sequential updates and their limits

With a common fixed parameter and the stated conditional independence, updating after each batch or pooling their sufficient totals gives the same posterior. For the gamma model, one event in half an hour followed by three in 1.5 hours adds the same count/exposure as four in two hours. Do not use the first batch again when forming the second likelihood.

Conjugacy guarantees a family, not a steadily decreasing numerical variance after every possible observation. Try Gamma(1,10), fifty events, and exposure 0.1. The surprising high count moves the posterior to a much larger rate, and its absolute variance increases. Nor does conjugacy prove that the prior, constant-rate assumption, or independence model is appropriate.

Reproduce the calculations in Python

The downloadable script pins Python 3.13, SciPy 1.16.2, and NumPy 2.3.3. Save it and run uv run conjugate-priors.py. SciPy’s gamma uses scale=1/rate. Its negative-binomial arguments here are n=posterior shape and p=rate/(rate+future exposure); the output count is the future event count.

# /// script
# requires-python = ">=3.13,<3.14"
# dependencies = ["scipy==1.16.2", "numpy==2.3.3"]
# ///

import math

from scipy.stats import betabinom, gamma, nbinom, poisson

shape, rate = 2 + 4, 1 + 2
posterior = gamma(a=shape, scale=1 / rate)
future_exposure = 1
predictive = nbinom(n=shape, p=rate / (rate + future_exposure))

assert math.isclose(posterior.mean(), 2)
assert math.isclose(posterior.var(), 2 / 3)
assert math.isclose(predictive.pmf(0), (3 / 4) ** 6)
assert math.isclose(predictive.var(), 8 / 3)
print(f"Gamma posterior rate mean: {posterior.mean():.7f}")
print(f"Predictive P(zero): {predictive.pmf(0):.7f}")
print(f"Plug-in P(zero): {poisson(mu=2).pmf(0):.7f}")

coin_prediction = betabinom(n=3, a=2 + 4, b=2 + 2)
assert math.isclose(coin_prediction.pmf(0), 1 / 11)
assert math.isclose(coin_prediction.mean(), 1.8)
print(f"Beta-binomial P(zero): {coin_prediction.pmf(0):.7f}")

assert math.isclose(poisson.sf(2, mu=2), gamma.cdf(1, a=3, scale=1 / 2))
print(f"Third arrival within one hour: {poisson.sf(2, mu=2):.7f}")

The assertions check the worked probabilities and the count/wait identity. See SciPy’s gamma, negative-binomial, and beta-binomial parameter documentation.

Sources and next steps

The Book of Statistical Proofs shows gamma conjugacy for a Poisson likelihood. Charles Geyer’s Poisson inference notes discuss gamma priors and parameterization. The exposure and predictive expressions above follow by multiplying or integrating the displayed densities. The beta lesson adds credible intervals; the Poisson process lesson shows what the constant-rate arrival model assumes.

Return to learning paths and foundations.