Question about floating point

[Steven D'Aprano]

On Tue, 28 Aug 2018 16:47:25 +0200, Frank Millman wrote:

> The reason for my query is this. I am assisting someone with an
> application involving monetary values. I have been trying to explain why
> they should use the decimal module. They have had a counter-argument
> from someone else who says they should just use the rounding technique
> in my third example above.

*head-desk*

And this is why we can't have nice things.

Presumably this money application doesn't just work on hard-coded literal 
values, right? So this "programmer" your friend is listening to prefers 
to write this:

    money = (a + b)*10/10

instead of:

    money = a + b

presumably because programming isn't hard enough without superstitious 
ritual that doesn't actually solve the problem.

In the second case, you have (potentially) *one* rounding error, due to 
the addition.

In the first case, you get the *exact same rounding error* when you do 
(a+b). Then you get a second rounding error by multiplying by ten, and a 
third rounding error when you divide by ten.

Now its true that sometimes those rounding errors will cancel. You found 
an example:

py> (1.1 + 2.2)*10/10 == 3.3
True

but it took me four attempts to find a counter-example, where the errors 
don't cancel:

py> (49675.23 + 10492.95)*10/10 == 60168.18
False

To prove it isn't a fluke:

py> (731984.84 + 173.32)*10/10 == 732158.16
False

py> (170734.84 - 173.39)*10/10 == 170561.45
False


Given that it has three possible three rounding errors instead of one, it 
is even possible that this "clever trick" could end up being *worse* than 
just doing a single addition. But my care factor isn't high enough to 
track down an example (if one exists).

For nearly all applications involving money, one correct solution is to 
use either integer numbers of cents (or whatever the smallest currency 
you ever care about happens to be). Then all additions, subtractions and 
multiplications will be exact, without fail, and you only need to worry 
about rounding divisions. You can minimize (but not eliminate) that by 
calculating in tenths of a cent, which effectively gives you a guard 
digit.

Or, just use decimal, which is *designed* for monetary applications 
(among other things). You decide on how many decimal places to keep (say, 
two, or three if you want a guard digit), a rounding mode (Banker's 
Rounding is recommended for financial applications), and just do your 
calculations with no "clever tricks".

Add two numbers, then add tax:

    money = (a+b)*(1+t/100)

compared to the "clever trick":

    money = (a+b)*10/10 * (1 + t)*10/10


Which would you rather do?



-- 
Steven D'Aprano
"Ever since I learned about confirmation bias, I've been seeing
it everywhere." -- Jon Ronson