on floating-point numbers

[Chris Angelico]

On Fri, Sep 3, 2021 at 4:29 AM Hope Rouselle <hrouselle at jevedi.com> wrote:
>
> Just sharing a case of floating-point numbers.  Nothing needed to be
> solved or to be figured out.  Just bringing up conversation.
>
> (*) An introduction to me
>
> I don't understand floating-point numbers from the inside out, but I do
> know how to work with base 2 and scientific notation.  So the idea of
> expressing a number as
>
>   mantissa * base^{power}
>
> is not foreign to me. (If that helps you to perhaps instruct me on
> what's going on here.)
>
> (*) A presentation of the behavior
>
> >>> import sys
> >>> sys.version
> '3.8.10 (tags/v3.8.10:3d8993a, May  3 2021, 11:48:03) [MSC v.1928 64 bit (AMD64)]'
>
> >>> ls = [7.23, 8.41, 6.15, 2.31, 7.73, 7.77]
> >>> sum(ls)
> 39.599999999999994
>
> >>> ls = [8.41, 6.15, 2.31, 7.73, 7.77, 7.23]
> >>> sum(ls)
> 39.60000000000001
>
> All I did was to take the first number, 7.23, and move it to the last
> position in the list.  (So we have a violation of the commutativity of
> addition.)
>

It's not about the commutativity of any particular pair of operands -
that's always guaranteed. What you're seeing here is the results of
intermediate rounding. Try this:

>>> def sum(stuff):
...     total = 0
...     for thing in stuff:
...             total += thing
...             print(thing, "-->", total)
...     return total
...
>>> ls = [7.23, 8.41, 6.15, 2.31, 7.73, 7.77]
>>> sum(ls)
7.23 --> 7.23
8.41 --> 15.64
6.15 --> 21.79
2.31 --> 24.099999999999998
7.73 --> 31.83
7.77 --> 39.599999999999994
39.599999999999994
>>> ls = [8.41, 6.15, 2.31, 7.73, 7.77, 7.23]
>>> sum(ls)
8.41 --> 8.41
6.15 --> 14.56
2.31 --> 16.87
7.73 --> 24.6
7.77 --> 32.370000000000005
7.23 --> 39.60000000000001
39.60000000000001
>>>

Nearly all floating-point confusion stems from an assumption that the
input values are exact. They usually aren't. Consider:

>>> from fractions import Fraction
>>> for n in ls: print(n, Fraction(*n.as_integer_ratio()))
...
8.41 2367204554136617/281474976710656
6.15 3462142213541069/562949953421312
2.31 5201657569612923/2251799813685248
7.73 2175801569973371/281474976710656
7.77 2187060569041797/281474976710656
7.23 2035064081618043/281474976710656

Those are the ACTUAL values you're adding. Do the same exercise with
the partial sums, and see where the rounding happens. It's probably
happening several times, in fact.

The naive summation algorithm used by sum() is compatible with a
variety of different data types - even lists, although it's documented
as being intended for numbers - but if you know for sure that you're
working with floats, there's a more accurate algorithm available to
you.

>>> math.fsum([7.23, 8.41, 6.15, 2.31, 7.73, 7.77])
39.6
>>> math.fsum([8.41, 6.15, 2.31, 7.73, 7.77, 7.23])
39.6

It seeks to minimize loss to repeated rounding and is, I believe,
independent of data order.

ChrisA