Logistic function

The logistic function is defined as f(x) = 1 / (1 + e^−x), where e is Euler's number. It is a sigmoid function, characterized by an S-shaped curve that is symmetric about the point (0, 0.5). The function approaches 0 as x tends to negative infinity and 1 as x tends to positive infinity, with a maximum slope of 0.25 at x = 0. Its derivative satisfies the differential equation f' = f(1 − f), which links it to logistic growth models.

The function is infinitely differentiable and has a simple closed-form inverse, the logit function, which is used in logistic regression to map probabilities back to the real line. Its smoothness and boundedness make it attractive for optimization and neural network activation functions.

In statistics, the logistic function is the canonical link in logistic regression, modeling the probability of a binary outcome as a function of predictors. The logit transformation, log(p/(1−p)), linearizes the relationship, enabling maximum likelihood estimation. In machine learning, it serves as an activation function in neural networks, particularly in the output layer for binary classification, and as a squashing function in hidden layers, though it has been largely replaced by ReLU in deep networks due to vanishing gradient issues.

The function also appears in the softmax generalization for multiclass classification, where it is applied to each class score. Its probabilistic interpretation and differentiability underpin many probabilistic graphical models and Bayesian inference techniques.

Verhulst introduced the logistic function in 1838 to model population growth with limited resources, leading to the logistic differential equation dP/dt = rP(1 − P/K), where r is the intrinsic growth rate and K is the carrying capacity. The solution is a logistic curve that starts with exponential growth and levels off at K. This model was later popularized by Raymond Pearl and Lowell Reed in the 1920s for human population projections.

In ecology, the logistic growth model is a cornerstone for understanding density-dependent regulation, though it has been criticized for its simplicity. The discrete-time version, the logistic map, exhibits chaotic behavior for certain parameter values, illustrating how simple nonlinear equations can produce complex dynamics.

The logistic function has deep connections to the hyperbolic tangent: tanh(x/2) = 2f(x) − 1, a relationship used in neural network implementations. It also appears in the Fermi–Dirac distribution in quantum statistics, where it models the probability of an energy state being occupied by a fermion. In chemistry, it describes the kinetics of autocatalytic reactions, and in epidemiology, it models the cumulative number of infections in an outbreak, though more flexible curves are often preferred.

The function's name derives from the Greek 'logistikos', meaning 'computational' or 'skilled in calculation', and it was not widely adopted until the 20th century. A notable edge case is the generalized logistic function, which adds parameters for asymmetry and growth rate, used in growth curve analysis. The logistic function also appears in the definition of the logistic distribution, a continuous probability distribution used in survival analysis.

Sigmoid function

A function with an S-shaped curve, typically bounded and monotonic.

Logit function

The inverse of the logistic function, mapping probabilities to real numbers.

Carrying capacity

The maximum population size an environment can sustain indefinitely.

Logistic map

A discrete-time analog of the logistic equation that exhibits chaotic behavior.