Effective sample size

The effective sample size is defined as the size of an independent and identically distributed (iid) sample that would yield the same precision as the actual correlated or weighted sample. For a sample of size n with autocorrelation coefficients ρ_k at lag k, the ESS is given by n / (1 + 2∑ρ_k).¹ In survey sampling, Leslie Kish introduced a simpler formula for unequal weighting: ESS = (∑w_i)² / ∑w_i², which reduces to n / (1 + CV²), where CV is the coefficient of variation of the weights.²

In Markov chain Monte Carlo (MCMC) methods, the ESS is a critical diagnostic for convergence and precision of posterior estimates. A low ESS indicates high autocorrelation in the chain, meaning that the samples are not independent and the effective information is less than the raw sample size. Standard practice recommends an ESS of at least 400 for reliable estimation of quantiles, as suggested by Vehtari et al. in their widely used Rank-normalized folding diagnostic.³ The ESS is computed from the autocorrelation of the chain, often using spectral density estimation or the batch means method.

In survey sampling, the effective sample size is used to adjust for design effects, such as clustering and weighting, to determine the actual precision of survey estimates. For example, the National Health and Nutrition Examination Survey (NHANES) reports effective sample sizes to account for its complex multistage sampling design.⁴ In clinical trials, the ESS is used in group sequential designs to monitor accumulating data, where correlated observations arise from repeated measurements on the same patient. The concept also appears in meta-analysis, where the effective sample size of a pooled estimate is derived from the inverse-variance weights.

Beyond the standard formulas, the effective sample size has subtle variations. For example, in the context of importance sampling, the ESS is often approximated by the reciprocal of the sum of squared normalized weights, but this approximation can be biased; a more accurate estimate uses the exponential of the Shannon entropy of the weights.⁵ In particle physics, the ESS is used in the analysis of high-energy collision data to account for event weighting, as seen in the ATLAS experiment's publications. Additionally, the concept of 'effective sample size' appears in ecology for estimating population sizes from mark-recapture data, where it is related to the number of recaptures. A less-known fact is that the ESS can be greater than the actual sample size if the weights are negatively correlated, though this is rare in practice.

Autocorrelation

Correlation of a signal with a delayed copy of itself, used to measure dependence in time series.

Markov chain Monte Carlo (MCMC)

A class of algorithms for sampling from probability distributions based on constructing a Markov chain.

Design effect

The ratio of the variance of an estimate under a complex sampling design to the variance under simple random sampling.

Importance sampling

A variance reduction technique that uses a proposal distribution to estimate properties of a target distribution.