Base rate

The base rate is the prior probability P(H) of a hypothesis before observing evidence. In Bayes' theorem, the posterior probability P(H|E) is proportional to the product of the likelihood P(E|H) and the prior. When base rates are low, even highly accurate tests can yield low positive predictive values. For instance, a test with 99% sensitivity and 99% specificity applied to a population with a 1% base rate yields a positive predictive value of only 50%.² This mathematical relationship underscores why base rates are indispensable in medical screening, forensic science, and machine learning.

Base rate neglect is a cognitive bias in which people ignore the prior probability and focus on individuating information. In a classic experiment by Daniel Kahneman and Amos Tversky, participants judged the likelihood that a described individual was an engineer or lawyer while ignoring the actual proportion of each group.³ This bias has real-world consequences: in legal settings, jurors may overestimate the probative value of DNA evidence if they disregard the rarity of a match in the general population. In medicine, both patients and clinicians can misinterpret positive test results when the disease is rare. The bias is robust across cultures and persists even among trained statisticians, though its magnitude can be reduced by presenting base rates in natural frequencies rather than percentages.⁴

Base rates inform decision-making in fields as varied as epidemiology, finance, and artificial intelligence. In epidemiology, they are used to compute attributable risk and to design screening programs; for example, the low base rate of certain cancers justifies age-based screening thresholds. In finance, base rates of default guide credit scoring and risk pricing. In machine learning, class imbalance—a form of skewed base rates—requires techniques such as oversampling or cost-sensitive learning to avoid biased classifiers.⁵ In psychology, base rates of behaviors help in diagnosing rare conditions, and in economics they underpin actuarial models. Even in everyday life, base rates affect judgments about safety and risk, such as the likelihood of a flight delay or a product defect.

Beyond the classic examples, base rates have subtle and surprising dimensions. In forensic science, the base rate of a DNA profile in a population is used to compute the likelihood ratio, but the "island problem" illustrates how ignoring the number of possible suspects can lead to erroneous convictions. In ecological studies, base rates of species occurrence are used in occupancy modeling to correct for imperfect detection. In psychometrics, base rates of rare behaviors affect the reliability of diagnostic tests; for instance, the base rate of child abuse is so low that even validated screening tools produce many false positives. Historically, the concept predates Bayes: the 18th-century mathematician Abraham de Moivre used base rates in annuity calculations, and the term "base rate" itself was popularized in the 20th century by psychologists such as Paul Meehl, who argued for the use of actuarial prediction over clinical judgment.

Prior probability

The probability of a hypothesis before new evidence is considered; synonymous with base rate.

Posterior probability

The updated probability after incorporating evidence via Bayes' theorem.

Positive predictive value

The probability that a positive test result is a true positive, dependent on the base rate.

Base rate neglect

A cognitive bias where prior probabilities are ignored in favor of specific information.