# Why 0.1 + 0.2 ≠ 0.3, and Why It's Worse Than You Think

> Everyone knows floating point is 'imprecise.' Fewer know that the same expression can give different results on different machines, that summation order changes your answer, and why associativity — the thing algebra promised you — is a lie in float.

URL: https://harshith.in/blog/floating-point-determinism/
Author: Harshith Sunku (Staff Engineer) — https://harshith.in/about/
Tags: fundamentals, performance, systems

`0.1 + 0.2 == 0.30000000000000004` is the famous one, and it's the
*least* interesting floating-point surprise. It has a boring cause
(0.1 isn't representable in binary, same as 1/3 isn't in decimal) and
a boring fix (don't `==` floats). The genuinely hazardous facts are
the ones that make *the same program* give *different answers* on
different machines, or make a sum depend on the order you added things
— because those break reproducibility, tests, and distributed
consensus in ways that look like heisenbugs. (I've written elsewhere
on the bit-level format; this is about the traps that follow from it.)

## The Core Sin: Addition Isn't Associative

In real numbers, `(a + b) + c == a + (b + c)`. In IEEE 754, it often
doesn't, because each operation *rounds* to the nearest representable
value, and where the rounding lands depends on the intermediate
magnitudes:

```text
  (1e20 + -1e20) + 1.0  =  0.0 + 1.0     =  1.0
   1e20 + (-1e20 + 1.0) =  1e20 + -1e20  =  0.0   ← the 1.0 vanished
```

The `1.0` was **absorbed**: added to `1e20`, it falls below the
precision of that magnitude and rounds away entirely. Reorder, and it
survives. This single fact — non-associativity — is the root of most
serious float trouble, and it has three teeth:

- **Summation order changes results.** Sum a million floats
  left-to-right vs sorted vs pairwise and you get different answers.
  For a large sum of mixed magnitudes the *naive* order can lose most
  of your significant digits, because each addition rounds against a
  running total that's grown large relative to the addends.
- **Parallelism reorders implicitly.** Split that sum across 8 threads
  or SIMD lanes and the reduction order changes with the thread
  count — so your result depends on core count and scheduling.
  "Same code, same input, different answer per run" is usually this.
- **The compiler reorders too.** `-ffast-math` (and pieces of it like
  `-fassociative-math`) explicitly permits the optimizer to treat
  float math as associative for speed/vectorization — trading
  reproducibility and some correctness for throughput. Great for a
  game's shading, potentially disastrous for a numerical solver or
  anything that must match across builds. Know whether it's on.

## Catastrophic Cancellation: Losing Everything at Once

The other classic destroyer: subtracting two nearly-equal numbers.
The equal leading digits cancel, leaving only the noisy trailing bits
— you can go from 15 significant digits to 2 in one subtraction:

```text
  1.2345678901234 - 1.2345678900000 = 0.0000000001234
  15 good digits in each input → ~4 in the result
```

This is why the textbook quadratic formula is numerically unstable for
certain coefficients, why computing variance as `E[x²] - E[x]²` is a
known footgun (Welford's algorithm exists to dodge exactly this), and
why the fix is usually *algebraic reformulation* to avoid the
subtraction, not more precision. Doubling to 64-bit or 128-bit float
delays the problem; it doesn't change that the *algorithm* is throwing
away information.

## Where This Bites, and What To Do

The failure modes are concrete: flaky tests that assert float equality
(compare with a tolerance appropriate to the magnitude — absolute
epsilon fails for large values; use relative or ULP-based comparison);
distributed systems that hash or compare computed floats across
heterogeneous nodes and *disagree* (never put raw float results in a
consensus/equality path — quantize to integers or fixed-point first);
financial code using float for money (use integer cents or a decimal
type — float dollars will eventually be off by a penny in an
auditable place); and GPU/CPU results that don't match because
fused-multiply-add (one rounding instead of two) changed the last
bits.

The defensive toolkit: **never `==`** — compare with tolerance;
**don't use float for money or exact counts** — integers/decimals;
**mind the algorithm, not just the type** — reformulate to avoid
cancellation, use Kahan/Neumaier compensated summation or pairwise
reduction for large sums (Kahan carries the lost low-order bits in a
correction term and recovers most of them); and **control the knobs**
— know your `-ffast-math` status and pin FMA/rounding modes if
cross-platform determinism is a requirement, because by default it
isn't one.

## Takeaways

- The real hazard isn't representation error, it's non-associativity:
  reordering a sum changes the result, so order, parallelism, and
  fast-math all alter answers.
- Naive summation of mixed magnitudes loses precision by absorption;
  Kahan/pairwise summation recovers it.
- Catastrophic cancellation destroys significant digits in one
  near-equal subtraction — fix the algorithm (Welford, reformulated
  quadratic), not the precision.
- Never == floats; never use them for money or cross-node equality/
  consensus — quantize to integers/decimals at those boundaries.
- Know your -ffast-math and FMA settings; determinism across machines
  is opt-in, not the default.