on floating-point numbers

[Peter J. Holzer]

On 2021-09-12 01:40:12 +1000, Chris Angelico wrote:
> On Sun, Sep 12, 2021 at 1:07 AM Peter J. Holzer <hjp-python at hjp.at> wrote:
> > If you have any "decimals" (i.e decimal digits to the right of your
> > decimal point) then the input values won't be exactly representable and
> > the nearest representation will use all available bits, thus losing some
> > precision with most additions.
> 
> That's an oversimplification, though - numbers like 12345.03125 can be
> perfectly accurately represented, since the fractional part is a
> (negative) power of two.

Yes. I had explained that earlier in this thread.

> The perceived inaccuracy of floating point numbers comes from an
> assumption that a string of decimal digits is exact, and the
> computer's representation of it is not. If I put this in my code:
> 
> ONE_THIRD = 0.33333
> 
> then you know full well that it's not accurate, and that's nothing to
> do with IEEE floating-point! The confusion comes from the fact that
> one fifth (0.2) can be represented precisely in decimal, and not in
> binary.

Exactly.


> Once you accept that "perfectly representable numbers" aren't
> necessarily the ones you expect them to be, 64-bit floats become
> adequate for a huge number of tasks.

Yep. That's what I was trying to convey. 


> Even 32-bit floats are pretty reliable for most tasks, although I
> suspect that there's little reason to use them now - would be curious
> to see if there's any performance benefit from restricting to the
> smaller format, given that most FPUs probably have 80-bit or wider
> internal registers.

AFAIK C compilers on 64-bit AMD/Intel architecture don't use the x87 ABI
any more, they use the various vector extensions (SSE, etc.) instead.
Those have hardware support for 64 and 32 bit FP values, so 32 bit are
probably faster, if only because you can cram more of them into a
register. Modern GPUs now have 16 bit FP numbers - those are perfectly
adequate for neural networks and also some graphics tasks and you can
transfer twice as many per memory cycle ...

        hp

-- 
   _  | Peter J. Holzer    | Story must make more sense than reality.
|_|_) |                    |
| |   | hjp at hjp.at         |    -- Charles Stross, "Creative writing
__/   | http://www.hjp.at/ |       challenge!"
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 833 bytes
Desc: not available
URL: <https://mail.python.org/pipermail/python-list/attachments/20210911/d9c3fe4d/attachment.sig>