[Tutor] precision handling

[Alan Gauld]

On 01/10/2021 09:30, Msd De wrote:
> Thank you for your email.
>
> Part of my code
> M = [[1,0],[0,1]]
> for k in range(Nslice,1,-1):
>   WFjAFjB = F_jA[k]*DF_jB[k] -  F_jB[k]*DF_jA[k]
>   Mj11 = (DF_jB[k]*F_jm1A[k]- F_jB[k]*DF_jm1A[k])/WFjAFjB
>   Mj12 = (DF_jB[k]*F_jm1B[k] - F_jB[k]*DF_jm1B[k])/WFjAFjB
>   Mj21 = (F_jA[k]*DF_jm1A[k] - DF_jA[k]*F_jm1A[k])/WFjAFjB
>   Mj22 = (F_jA[k]*DF_jm1B[k] - DF_jA[k]*F_jm1B[k])/WFjAFjB
>   Mj = [[Mj11,Mj12],[Mj21,Mj22]]
>   test1 = DF_jB[k]*F_jm1B[k]
>   test2= - F_jB[k]*DF_jm1B[k]
>   print round(Decimal(test1),10),round(Decimal(test2),10)
>   M = np.dot(M,Mj)
>
OK, So you are not actually comparing 2 constants but rather the results
of calculations.

Those calculations will almost inevitably result in slight differences
in the results even
if they should result in the same answer.

OUTPUT:
> -36027335809.8 36027335809.8
> -3.77786532033e+11 3.77786532033e+11
> -4.23206814691e+12 4.23206814691e+12
> -5.05577734264e+13 5.05577734264e+13
> -6.43053645277e+14 6.43053645277e+14
> -8.69512959787e+15 8.69512959787e+15
> -1.24814507298e+17 1.24814507298e+17
> -1.89952420024e+18 1.89952420024e+18
> -3.06113023909e+19 3.06113023909e+19
>
And looking at these outputs they are not actually the same number.
One is negative, the other positive. That could make a difference too.

BTW. Converting to Decimal for the print stage is not going to
help the calculation precision

Sometimes modifying the order of the operations can make a difference too.

For example:

a/b + c/b

can sometimes give a better result than

(a+c)/b

because the two numbers may be closer in magnitudes.
But at other times it may be worse!

Working with floats is a precarious pastime.

-- 
Alan G
Author of the Learn to Program web site
http://www.alan-g.me.uk/
http://www.amazon.com/author/alan_gauld
Follow my photo-blog on Flickr at:
http://www.flickr.com/photos/alangauldphotos