Inverse probability

The term "inverse probability" was coined to describe the reversal of the usual probability question: instead of finding the probability of an outcome given a known cause, one seeks the probability of a cause given an observed outcome. Thomas Bayes's posthumous essay, An Essay towards solving a Problem in the Doctrine of Chances (1763), provided the first solution to this problem, using a uniform prior for an unknown binomial parameter. Pierre-Simon Laplace independently derived a more general rule in 1774, often called the rule of inverse probability, which states that the probability of a cause is proportional to the prior probability times the likelihood of the evidence. Laplace applied this to diverse problems, including the probability that the sun will rise tomorrow, based on past observations, and to the analysis of astronomical data.

Inverse probability was widely used in the 19th century for statistical inference in astronomy, geodesy, and social science. Laplace used it to estimate the mass of Jupiter and to assess the probability of errors in measurements. In 1837, Siméon Denis Poisson applied inverse probability to legal decision-making, and later, in 1886, John Venn criticized the method for its reliance on subjective priors. The controversy intensified with the work of Ronald Fisher, who argued that inverse probability was not a valid basis for scientific inference because it required prior distributions that could not be objectively justified. Fisher's frequentist approach, based on likelihood and significance testing, gradually replaced inverse probability in mainstream statistics.

By the 1920s, inverse probability had largely been abandoned by statisticians, who favored frequentist methods that avoided prior distributions. However, the term persisted in some textbooks and philosophical discussions. In the mid-20th century, the work of Harold Jeffreys and later Leonard J. Savage revived Bayesian inference, but they used the term "Bayesian" rather than "inverse probability" to avoid the historical baggage. The revival was further accelerated by the development of computational methods such as Markov chain Monte Carlo in the 1990s, which made Bayesian analysis practical for complex models. Today, the term "inverse probability" is mainly of historical interest, but it is recognized as the precursor to modern Bayesian statistics.

One lesser-known fact is that the term "inverse probability" was used by Laplace to justify the method of least squares, which he derived from inverse probability principles. Another is that the concept was applied to legal evidence by Poisson, who used it to calculate the probability of wrongful conviction. In the 19th century, the term was also used in actuarial science to estimate mortality rates from limited data. A notable figure is Augustus De Morgan, who defended inverse probability in his 1838 treatise on probability, arguing that it was the only logical way to reason about uncertain causes. The term also appears in the work of George Boole, who criticized it for its reliance on uniform priors. These historical debates foreshadowed modern discussions about the role of priors in Bayesian analysis.

Prior distribution

The probability distribution representing uncertainty about a parameter before observing data.

Likelihood

The probability of the observed data given a particular parameter value.

Posterior distribution

The updated probability distribution of a parameter after incorporating observed data.

Frequentist inference

A framework of statistical inference that defines probability as the long-run frequency of events, avoiding prior distributions.