Probability density function

A probability density function assigns nonnegative density across the possible values of a continuous random variable. If X has density f, then P(a ≤ X ≤ b) = ∫ₐᵇ f(x) dx, and the normalization rule is ∫_−∞^∞ f(x) dx = 1.¹ Density can be greater than one when the measurement scale or interval is sufficiently narrow; only areas, not heights alone, represent probabilities.

For an ordinary continuous variable, the probability of any single exact value is zero, so P(X = x) = 0. The cumulative distribution function instead records accumulated probability, with F(x) = ∫_−∞^x f(t) dt; where differentiation is valid, f(x) = F′(x).²

A density describes local relative likelihood through the area it contributes over a small interval. Near a point x, the approximation P(x ≤ X ≤ x + Δx) ≈ f(x)Δx becomes accurate when Δx is small, although f(x) itself is not a probability.

Expectations are weighted integrals: for a suitable function g, E[g(X)] = ∫ g(x)f(x)dx. Thus the mean uses g(x) = x, while variance measures the expected squared distance from the mean.¹ A change of measurement scale requires a Jacobian factor: if Y = h(X) and h is one-to-one and differentiable, then f_Y(y) = f_X(h⁻¹(y))|d h⁻¹(y)/dy|.³

Standard density families provide compact models for measurements, errors, waiting times, and physical quantities. The normal distribution is symmetric and determined by its mean and variance; the exponential distribution models nonnegative waiting times under a constant hazard; and the beta distribution is confined to the interval from zero to one. The gamma, log-normal, Weibull, and Student's t distributions extend this library for skewness, positive data, or heavy tails.²

In statistical inference, a density may describe data conditional on parameters, or parameters conditional on data in a Bayesian model. A likelihood is not generally a probability density for the parameter: it is a function of the parameter with the observed data held fixed. Densities also underlie maximum likelihood, simulation, confidence procedures, and numerical integration.⁴

Density functions can be nonunique at isolated points without changing any probability. Altering f at one point, or even on a set of total length zero, leaves its integrals unchanged; densities are therefore conventionally treated as equivalent when they differ only on such sets.

Not every random variable has an ordinary density. A discrete variable uses probability masses, while a mixed variable can contain both point masses and a continuous component. More generally, probability distributions may be studied through a measure rather than a density with respect to length; singular distributions, such as the Cantor distribution, provide a less familiar case with no ordinary density despite being continuous in the cumulative-distribution sense.⁵

In several dimensions, a joint density integrates over regions, and marginal densities arise by integrating out coordinates. Dependence is represented by the joint density and cannot generally be recovered from the separate marginal curves alone.

Continuous random variable

A random variable whose distribution is described by probabilities over intervals rather than positive probability at individual ordinary points.

Cumulative distribution function

The function F(x) = P(X ≤ x), which accumulates probability up to x.

Joint density

A density for two or more random variables, whose integration over a region gives the probability of that region.

Likelihood

A function of unknown parameters formed from a statistical model and observed data; it is not automatically a probability distribution over those parameters.