[Python-ideas] Python Float Update

[Andrew Barnert]

On Jun 1, 2015, at 15:53, Nicholas Chammas <nicholas.chammas at gmail.com> wrote:
> 
>> On Mon, Jun 1, 2015 at 6:15 PM Andrew Barnert abarnert at yahoo.com wrote:
>> 
>> Obviously if you know the maximum precision needed before you start and explicitly set it to something big enough (or 7 places bigger than needed) you won't have any problem. Steven chose a low precision just to make the problems easy to see and understand; he could just as easily have constructed examples for a precision of 18.
>> 
>> Unfortunately, even in cases where it is both possible and sufficiently efficient to work out and set the precision high enough to make all of your calculations exact, that's not something most people know how to do reliably. In the fully general case, it's as hard as calculating error propagation.
>> 
>> As for the error: Python's decimal flags that too; the difference is that the flag is ignored by default. You can change it to warn or error instead. Maybe the solution is to make that easier--possibly just changing the docs. If you read the whole thing you will eventually learn that the default context ignores most such errors, but a one-liner gets you a different context that acts like SQL Server, but who reads the whole module docs (especially when they already believe they understand how decimal arithmetic works)? Maybe moving that up near the top would be useful?
> 
> This angle of discussion is what I was getting at when I wrote:
> 
> Perhaps Python needs better rules for how precision and scale are affected by calculations (here are SQL Server’s, for example), or better defaults when they are not specified?
> 
I definitely agree that some edits to the decimal module docs, plus maybe a new HOWTO, and maybe some links to outside resources that explain things to people who are used to decimals in MSSQLServer or REXX or whatever, would be helpful. The question is, who has the sufficient knowledge, skill, and time/inclination to do it?

> It sounds like there are perhaps several improvements that can be made to how decimals are handled, documented, and configured by default, that could possibly address the majority of gotchas for the majority of people in a more user friendly way than can be accomplished with floats.
> 
> For all the problems presented with decimals by Steven and others, I’m not seeing how overall they are supposed to be worse than the problems with floats.
> 
They're not worse than the problems with floats, they're the same problems... But the _effect_ of those problems can be worse, because:

 * The magnitude of the rounding errors is larger.

 * People mistakenly think they understand everything relevant about decimals, and the naive tests they try work out, so the problems may blindside them.

 * Being much more detailed and configurable means the best solution may be harder to find.

 * There's a lot of correct but potentially-misleading information out there. For example, any StackOverflow answer that says "you can solve this particular problem by using Decimal instead of float" can be very easily misinterpreted as applying to a much wider range of problems than it actually does.

 * Sometimes performance matters.

On the other hand, the effect can also be less bad, because:

 * Once people do finally understand a given problem, at least for many people and many problems, working out a solution is easier in decimal. For some uses (in particular, many financial uses, and some kinds of engineering problems), it's even trivial.

 * Being more detailed and more configurable means the best solution may be better than any solution involving float.

I don't think there's any obvious answer to the tradeoff, short of making it easier for people to choose appropriately: a good HOWTO, decimal literals or Swift-style float-convertibles, making it easier to find/construct decimal64 or DECIMAL(18) or Money types, speeding up decimal (already done, but maybe more could be done), etc.

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20150601/df2c1c1d/attachment-0001.html>