[Tutor] the binary math "wall"

[Steven D'Aprano]

On Wed, 21 Apr 2010 02:58:06 am Lowell Tackett wrote:
> I'm running headlong into the dilemma of binary math representation, 
with game-ending consequences, e.g.:
> >>> 0.15
>
> 0.14999999999999999
>
> Obviously, any attempts to manipulate this value, under the misguided
> assumption that it is truly "0.15" are ill-advised, with inevitable
> bad results.

That really depends on what sort of manipulation you are doing.

>>> x = 0.15
>>> x
0.14999999999999999
>>> x*100 == 15
True

Seems pretty accurate to me.

However:

>>> 18.15*100 == 1815
False


The simplest, roughest way to fix these sorts of problems (at the risk 
of creating *other* problems!) is to hit them with a hammer:

>>> round(18.15*100) == 1815
True

[...]
> What I'm shooting for, by the way, is an algorithm that converts a
> deg/min/sec formatted number to decimal degrees.  It [mostly] worked,
> until I stumbled upon the peculiar cases of 15 minutes and/or 45
> minutes, which exposed the flaw.

I'm afraid that due to the nature of floating point, this is a hard 
problem. Even the professionals at Hewlett-Packard's scientific 
calculator division don't always get it right, and they are *extremely* 
careful:

http://www.hpmuseum.org/cgi-sys/cgiwrap/hpmuseum/archv018.cgi?read=132690

The best result I can suggest is, change the problem! Don't pass 
degrees-minutes-seconds around using a floating point value, but as a 
tuple with distinct (DEG, MIN, SEC) integer values. Or create a custom 
class.

But if you really need D.MMSS floats, then something like this should be 
a good start.:

def dms2deg(f):
    """Convert a floating point number formatted as D.MMSS 
    into degrees.
    """
    mmss, d = math.modf(f)
    assert d == int(f)
    if mmss >= 0.60:
        raise ValueError(
        'bad fractional part, expected < .60 but got %f' % mmss)
    mmss *= 100
    m = round(mmss)
    if m >= 60:
        raise ValueError('bad minutes, expected < 60 but got %d' % m)
    s = round((mmss - m)*100, 8)
    if not 0 <= s < 60.0:
        raise ValueError('bad seconds, expected < 60.0 but got %f' % s)
    return d + m/60.0 + s/3600.0


>>> dms2deg(18.15)
18.25
>>> dms2deg(18.1515)
18.254166666666666


which compares well to my HP-48GX:

18.15 HMS->

gives 18.25, and:

18.1515 HMS->

gives 18.2541666667.


Note though that this still fails with some valid input. I will leave 
fixing it as an exercise (or I might work on it later, time 
permitting).


-- 
Steven D'Aprano