Subnormal number

Subnormal numbers are defined in the IEEE 754 floating-point standard as numbers with a biased exponent of zero and a nonzero significand. In binary formats, the significand has an implicit leading bit of 1 for normal numbers, but for subnormals this bit is 0, so the value is computed as 0.f × 2^(emin). This allows representation of values down to 2^(emin - precision + 1), which for double precision is about 4.9×10^-324.

The term "denormal" was used in earlier drafts, but "subnormal" is the official term in the standard. Subnormals are also called "denormalized numbers" in some literature. They are implemented in hardware on most modern CPUs, but software emulation may be required on older or specialized processors.

The primary purpose of subnormals is to provide gradual underflow, avoiding a sudden jump from the smallest normal number to zero. This is crucial for numerical algorithms that rely on small values, such as iterative solvers or probability computations. Without subnormals, operations like x - y could underflow to zero, causing division by zero or loss of significance.

The trade-off is reduced precision: subnormals have fewer significant bits, so the relative error can be larger. Additionally, operations involving subnormals are often slower than normal operations, sometimes by an order of magnitude, because they require special handling in hardware or software. Some applications disable subnormals (e.g., using the FTZ/DAZ flags) to improve performance, at the cost of accuracy.

The concept of gradual underflow was introduced in the 1970s by William Kahan and others during the design of the IEEE 754 standard. Earlier floating-point systems, such as the IBM System/360, used abrupt underflow, which caused many numerical problems. Kahan's proposal for subnormals was controversial but eventually adopted.

The first widely used implementation was in the Intel 8087 coprocessor (1980), which supported subnormals in hardware. Later, the standard was revised in 2008 and 2019, but the subnormal representation remained unchanged. Some early computers, like the Cray-1, did not support subnormals, leading to portability issues.

One obscure fact: subnormals are not symmetric around zero; the negative subnormals mirror the positive ones, but the smallest positive subnormal is 2^(-1074) in double, while the largest negative subnormal is -2^(-1074). Another edge case: the IEEE 754 standard requires that operations on subnormals be correctly rounded, but some implementations (e.g., older GPUs) flush them to zero, violating the standard.

In decimal floating-point formats, subnormals also exist but are rarely used. The term "denormal" is sometimes used in the context of decimal arithmetic, but the official term is "subnormal". Also, some languages like Java and C# expose subnormals, but others like JavaScript do not distinguish them from normal numbers in practice.

IEEE 754

The standard for floating-point arithmetic, defining formats, operations, and rounding rules.

Underflow

A condition when a result is too small to be represented as a normal number.

Gradual underflow

The use of subnormals to smoothly transition to zero.

Significand

The fractional part of a floating-point number, also called mantissa.