Floating-point arithmetic

Floating-point numbers are stored as a sign bit, a significand (mantissa), and an exponent, analogous to scientific notation. The IEEE 754 standard, adopted in 1985, defines formats like single (32-bit) and double (64-bit) precision, with specific bit allocations: 1 sign, 8 exponent, 23 significand for single; 1 sign, 11 exponent, 52 significand for double. It also specifies special values: NaN (Not a Number), positive and negative infinity, and signed zero. The exponent is biased to allow representation of very small and very large numbers, and the significand includes an implicit leading 1 for normalized numbers. This design enables a dynamic range from about 10^-38 to 10^38 for single precision and 10^-308 to 10^308 for double precision.

Because floating-point numbers have finite precision, most real numbers cannot be represented exactly, leading to rounding errors. The IEEE 754 standard defines four rounding modes: round to nearest (ties to even), round toward zero, round toward positive infinity, and round toward negative infinity. Round-to-nearest is the default and minimizes error. These errors accumulate in arithmetic operations, causing phenomena like the famous 0.1 + 0.2 ≠ 0.3 in binary floating-point. Catastrophic cancellation can occur when subtracting nearly equal numbers, amplifying relative error. Techniques like Kahan summation and compensated algorithms reduce error in long computations.

Floating-point arithmetic includes basic operations (addition, subtraction, multiplication, division, square root) and fused multiply-add (FMA) that computes a*b+c with a single rounding. The standard also defines five exceptions: invalid operation, division by zero, overflow, underflow, and inexact. These trigger flags that can be handled by software. Subnormal numbers fill the gap between zero and the smallest normal number, providing gradual underflow. The IEEE 754 standard also specifies a recommended way to implement transcendental functions like sin, cos, and exp, though their accuracy is not mandated.

Beyond the basics, floating-point arithmetic has surprising edge cases. The sign of zero can affect calculations, such as 1/+0 = +∞ but 1/-0 = -∞. NaN is not equal to itself, which is used in some algorithms to detect uninitialized data. The 'fast inverse square root' from Quake III uses a bit-level hack to approximate 1/√x. In 2019, the 'Floating-Point Rounding Error' in the Mars Climate Orbiter caused its loss due to a unit mismatch. Also, the decimal floating-point format (IEEE 754-2008) is used in financial calculations to avoid binary rounding errors. Some processors have extended precision formats, like x86's 80-bit, which can lead to double-rounding issues.

Significand

The part of a floating-point number that holds the significant digits.

Exponent

The power to which the base is raised, scaling the significand.

NaN

A special value representing an undefined or unrepresentable result.

Subnormal

A number smaller than the smallest normal number, with reduced precision.

FMA

Fused multiply-add, a single operation computing a*b+c with one rounding.