[Numpy-discussion] Deprecate numpy.dual?

[Warren Weckesser]

On 1/3/20, Sebastian Berg <sebastian at sipsolutions.net> wrote:
> On Fri, 2020-01-03 at 07:11 -0500, Warren Weckesser wrote:
>> In response to some work on improving the documentation of
>> `numpy.linalg` and how it compares to `scipy.linalg`, Kevin Sheppard
>> suggested that the documentation of the module `numpy.dual` should
>> also be improved.  When I mentioned this suggestion in the community
>> meeting on December 11, it was suggested that we should probably
>> deprecate `numpy.dual`.
>>
>> I think some current NumPy developers (myself included at the time
>> the topic came up) are unfamiliar with the history and purpose of
>> this module, so I spent some time reading code and github issues and
>> wrote up some notes.  These notes are available at
>>
>>
>> https://github.com/WarrenWeckesser/numpy-notes/blob/master/numpy-dual.md
>>
>> If you are not familiar with `numpy.dual`, you might find those notes
>> useful.
>>
>> Now that I know a bit more about `numpy.dual`, I'm not sure it should
>> be deprecated.  It provides a hook for other libraries to selectively
>> replace the use of the exposed functions in internal NumPy code, so
>> if a library has a better version of, say, `linalg.eigh`, it can
>> configure `numpy.dual` to use its version. Then, for example, NumPy
>> multivariate normal distribution code could benefit from the use of
>> that library's version of `eigh`.
>
> That is in principle true, but I do not think we use `dual` at all
> internally right now in numpy, and I doubt there is more than a hand
> full uses out there.

In the notes, I listed the internal uses of `numpy.dual` within numpy
that I found:

1. In the code that generates random variates from the multivariate normal
   distribution, one of `svd`, `eigh` or `cholesky` are used from `numpy.dual`.
2. In `matrixlib/defmatrix.py`, the `.I` property of the `matrix` class
   uses either `inv` or `pinv` from `numpy.dual` to compute its value.
3. The window function `numpy.kaiser` uses `numpy.dual.i0`.


>
> Dual is an override mechanism for functionality on ndarrays implemented
> also by numpy.
>
> In either case, I still tend towards deprecation. It seems to have
> issues and the main use case probably was to improve the situation when
> NumPy was compiled without an optimized BLAS/LAPACK. That probably was
> a common problem at some point, but I am not sure it is still an issue.
>
> Overriding functionality with faster implementations is of course a
> valid use-case and maybe `dual` is not a bad solution to the problem
> [0]. But I think we should discuss this more generally with other
> options. IMO deprecating this practically unused thing now does not
> mean we cannot do something similar in the future.

It probably makes sense to have the general discussion before
deprecating `numpy.dual`--there is a (slim?) chance that `numpy.dual`
will turn out to be the best option.

Warren


>
> - Sebastian
>
>
> [0] It has its own namespace, so is opt-in for the end user. You can
> only support a single backend at a time, although I am not sure that
> matters too much. If overrides provide a function to override, it is
> explicit to the end user as to what gets executed as well.
>
>
>> The NumPy documentation of `numpy.dual` refers specifically to SciPy,
>> but it could be used by any library.  Does anyone know if any other
>> libraries use `register_func` to put their functions into the
>> `numpy.dual` namespace?
>>
>> SciPy currently registers some functions, but there is an open issue
>> in which it is proposed that SciPy no longer register its functions
>> with `numpy.dual`:
>>
>>     https://github.com/scipy/scipy/issues/10441
>>
>> This email is to start the discussion of the future of `numpy.dual`.
>> Some of the options:
>>
>> 1. Quietly continue the status quo.
>> 2. Deprecate `numpy.dual`.
>> 3. Spend time improving the documentation of this feature, and
>>    perhaps even expand the functions that are supported.
>>
>> What do you think?  For those who were involved in the creation of
>> `numpy.dual`: is it working out like you expected?  If not, is it
>> worthwhile maintaining it?
>>
>> Warren
>>
>> _______________________________________________
>> NumPy-Discussion mailing list
>> NumPy-Discussion at python.org
>> https://mail.python.org/mailman/listinfo/numpy-discussion
>