on floating-point numbers

[Hope Rouselle]

Christian Gollwitzer <auriocus at gmx.de> writes:

> Am 02.09.21 um 15:51 schrieb Hope Rouselle:
>> Just sharing a case of floating-point numbers.  Nothing needed to be
>> solved or to be figured out.  Just bringing up conversation.
>> (*) An introduction to me
>> I don't understand floating-point numbers from the inside out, but I
>> do
>> know how to work with base 2 and scientific notation.  So the idea of
>> expressing a number as
>>    mantissa * base^{power}
>> is not foreign to me. (If that helps you to perhaps instruct me on
>> what's going on here.)
>> (*) A presentation of the behavior
>> 
>>>>> import sys
>>>>> sys.version
>> '3.8.10 (tags/v3.8.10:3d8993a, May 3 2021, 11:48:03) [MSC v.1928 64
>> bit (AMD64)]'
>> 
>>>>> ls = [7.23, 8.41, 6.15, 2.31, 7.73, 7.77]
>>>>> sum(ls)
>> 39.599999999999994
>> 
>>>>> ls = [8.41, 6.15, 2.31, 7.73, 7.77, 7.23]
>>>>> sum(ls)
>> 39.60000000000001
>> All I did was to take the first number, 7.23, and move it to the
>> last
>> position in the list.  (So we have a violation of the commutativity of
>> addition.)
>
> I believe it is not commutativity, but associativity, that is
> violated. 

Shall we take this seriously?  (I will disagree, but that doesn't mean I
am not grateful for your post.  Quite the contary.)  It in general
violates associativity too, but the example above couldn't be referring
to associativity because the second sum above could not be obtained from
associativity alone.  Commutativity is required, applied to five pairs
of numbers.  How can I go from

  7.23 + 8.41 + 6.15 + 2.31 + 7.73 + 7.77

to 

  8.41 + 6.15 + 2.31 + 7.73 + 7.77 + 7.23?

Perhaps only through various application of commutativity, namely the
ones below. (I omit the parentheses for less typing.  I suppose that
does not create much trouble.  There is no use of associativity below,
except for the intented omission of parentheses.)

     7.23 + 8.41 + 6.15 + 2.31 + 7.73 + 7.77
   = 8.41 + 7.23 + 6.15 + 2.31 + 7.73 + 7.77
   = 8.41 + 6.15 + 7.23 + 2.31 + 7.73 + 7.77
   = 8.41 + 6.15 + 2.31 + 7.23 + 7.73 + 7.77
   = 8.41 + 6.15 + 2.31 + 7.73 + 7.23 + 7.77
   = 8.41 + 6.15 + 2.31 + 7.73 + 7.77 + 7.23.
   
> Even for floating point, a+b=b+a except for maybe some extreme cases
> like denormliazed numbers etc.
>
> But in general (a+b)+c != a+ (b+c)
>
> Consider decimal floating point with 2 digits.
> a=1
> b=c=0.04
>
> Then you get LHS;
>  (1 + 0.04) + 0.04 = 1 + 0.04 = 1
>
> RHS:
>
> 1 + (0.04 + 0.04) = 1 + 0.08 = 1.1
>
> Your sum is evaluated like (((a + b) + c) + ....) and hence, if you
> permute the numbers, it can be unequal. If you need better accuracy, 
> there is the Kahan summation algorithm and other alternatives:
> https://en.wikipedia.org/wiki/Kahan_summation_algorithm

Thanks!