[Numpy-discussion] Floating point warnings/errors for comparisons, etc.?

[Sebastian Berg]

Hi all,

I am trying to clean up our floating point warning handling:

    https://github.com/numpy/numpy/pull/19316

And an upcoming PR to remove most floating point error clearing.  There
are some things I am unsure about, though.

Part of why it got so confusing, is that GCC seemed to have
fixed/changed their behaviour for comparisons with NaN.  In GCC 7 it
did not give the warning, but GCC 8 does (correctly).


Comparison with NaN
-------------------

IEEE says that the default comparisons should give warnings for
comparison with NaN (except == and !=).  And notes that an alternative
should be provided (C99 does this with `isless`, etc.).

We currently break this by suppressing invalid value warnings for all
comparisons (e.g. also `NaN > 0.`).

We can easily do either version (aside possibly compiler issues).
Making it give warnings had one test case fail for `quantile`, which
uses the pattern:

    if not (np.all(q >= 0) and np.all(q <= 1)):
        raise ValueError("bad q")

This would additionally (and first) give an "invalid value" warning and
require `np.errstate(invalid="ignore") to suppress it.

I dislike diverging from IEEE, but Python also does not warn for [1]:

    float("nan") >= 0

and presumably the user either explicitly created the NaN or has seen a
warning earlier during computation when the NaN was first created.

(IEEE does not distinguish creating a new NaN with `0./0.` from a
comparison with `NaN > 0.` [2].  So we can't easily make this settable
via `np.errstate` or so.)

So, should we ignore the warning here?


Compiler Issues
---------------

Some compilers may get flags wrong.  How much effort do we want to
spend on details few users will notice?

My current problem is `1 % 0` and `divmod(1, 0)`.  The MacOS/clang CI
does not set the correct "invalid value" warning flag.  (The remainder
is NaN, so a new NaN is created and that should be indicated but the
C99 `fmod` does not set it.)


Signalling NaNs
---------------

I propose dropping any special concern for signalling NaNs.  Which
means they raise almost always.  Although, rarely we might suppress the
warning if we do it manually for normal NaNs [0].

We have two tests which check for behaviour on signalling NaNs. I could
not find having any logic to them besides someone being surprised at
signalling NaN behaviour at the time – not based on use-cases.
Even functions like `isnan` give a warning for signalling NaNs!

The "fix" for anyone having sNaN's is to convert them to qNaNs as early
as possible.  Which e.g. `np.positive(arr, out=arr)` should probably
do.  If this becomes an issue, maybe we could have an explicit ufunc.


Cheers,

Sebastian



[0] Mainly it seems SSE2 does not provide some non-error comparisons. 
So trying to avoid manually clearing errors might make some SSE code
considerable slower (e.g. `isfinite`, `np.min`).

[1] Probably Python just does not check the CPU warning flags

[2] https://www.gnu.org/software/libc/manual/html_node/FP-Exceptions.html