[Numpy-discussion] log of negative real numbers -> RuntimeWarning: invalid value encountered in log

[Sebastian Berg]

On Mon, 2020-05-25 at 10:09 -0400, Brian Racey wrote:
> Thanks. That is for performance and memory which of course is valid
> for
> most use cases. Would it really be much different than doing
> type/size
> checking of all the np.array arguments to a function to ensure the
> appropriate final np.array sizes are allocated? I'm not trying to
> question
> current practice, just trying to understand as performance will
> eventually
> be important to me.
> 
> Would a "complex default" mode ever make it into numpy, to behave
> more like
> Matlab and other packages with respect to complex number handling?
> Sure it
> would make it marginally slower if enabled, but it might open the
> door to
> better compatibility when porting code to Python.
> 

I think the SciPy versions may have such a default, or there is such a
functionality hidden somewhere (maybe even the switching behaviour).
I am not sure anyone actually uses those, so it may not be a good idea
to use them to be honest.

The type safety aspects are that you do can get float or complex
results randomly, which of course is fine for much code...

>From my side main argument in favor of the current behaviour is that
you, as someone working with complex numbers, likely immediately knows
that what you want is to use `np.log(..., dtype=...)` or cast the input
to complex.  But someone not used to complex numbers may be very
surprised if they suddenly have a complex result after a long
calculation.

That said, there is a problem here, since it is a bit clumsy to force a
complex result. `np.log(..., dtype=np.complex128)` works, but hard-
codes double precision. `np.log(arr + 0j)` works, but may look strange
(I will assume that `log` is so slow that the overhead is not very
relevant).  I am not sure that if the use case is actually big enough
to worry about it, i.e. if you work with complex numbers, you may be
fine with just converting to complex once early on...

- Sebastian



> On Mon, May 25, 2020 at 9:49 AM Eric Wieser <
> wieser.eric+numpy at gmail.com>
> wrote:
> 
> > One explanation for this behavior is that doing otherwise would be
> > slow.
> > 
> > Consider an array like
> > 
> > arr = np.array([1]*10**6 + [-1])
> > ret = np.log(arr)
> > 
> > Today, what happens is:
> > 
> >    - The output array is allocated as np.double
> >    - The input array is iterated over, and log evaluated on each
> > element
> >    in turn
> > 
> > For what you describe to happen, the behavior would have to be
> > either:
> > 
> >    - The output array is allocated as np.double
> >    -
> > 
> >    The input array is iterated over, and log evaluated on each
> > element in
> >    turn
> >    -
> > 
> >    If any negative element is encountered, allocate a new array as
> >    np.cdouble, copy all the data over, then continue. This results
> > in the
> >    whole array being promoted.
> > 
> > or:
> > 
> >    - The input array is iterated over, and checked to see if all
> > the
> >    values are positive
> >    -
> > 
> >    The output array is allocated as np.double or np.cdouble based
> > on this
> >    result
> >    -
> > 
> >    The input array is iterated over, and log evaluated on each
> > element in
> >    turn
> > 
> > In either case, you’ve converted a 1-pass iteration to a 2-pass
> > one.
> > 
> > There are static-typing-based explanations for this behavior too,
> > but I’ll
> > let someone else present one of those.
> > 
> > Eric
> > 
> > On Mon, 25 May 2020 at 14:33, Brian Racey <raceybe at gmail.com>
> > wrote:
> > 
> > > Why does numpy produce a runtime warning (invalid value
> > > encountered in
> > > log) when taking the log of a negative number? I noticed that if
> > > you coerce
> > > the argument to complex by adding 0j to the negative number, the
> > > expected
> > > result is produced (i.e. ln(-1) = pi*i).
> > > 
> > > I was surprised I couldn't find a discussion on this, as I would
> > > have
> > > expected others to have come across this before. Packages like
> > > Matlab
> > > handle negative numbers automatically by doing the complex
> > > conversion.
> > > _______________________________________________
> > > NumPy-Discussion mailing list
> > > NumPy-Discussion at python.org
> > > https://mail.python.org/mailman/listinfo/numpy-discussion
> > > 
> > _______________________________________________
> > NumPy-Discussion mailing list
> > NumPy-Discussion at python.org
> > https://mail.python.org/mailman/listinfo/numpy-discussion
> > 
> 
> _______________________________________________
> NumPy-Discussion mailing list
> NumPy-Discussion at python.org
> https://mail.python.org/mailman/listinfo/numpy-discussion