[SciPy-Dev] Backwards Compatibility for low level LAPACK routines

[Ralf Gommers]

On Sat, Aug 4, 2018 at 12:49 AM, Ilhan Polat <ilhanpolat at gmail.com> wrote:

> It will gain a dgeqrf_lwork function as usual to return the necessary
> workspace size as
>
> lwork, info = dgeqrf_lwork(m,n)
>
> Then the "work" variable will be removed from dgeqrf signature and will
> be made hidden.
>
> For example for the previous example I gave before, the optimal size is
> 12800 and work array is returning an 12800-long array for a 400-400 array
> computation.
>

Ah okay. Then the alternative is to just leave the work parameter, ignore
it in the code if it's passed in (or give a warning/error) and document it
as not being used. Right?

If you're removing "work" from both the signature and the return value,
that's a bigger change indeed, that can't be handled well that way. I'm not
100% sure, but I think I agree that a backwards incompatible change here
will be better than introducing a bunch of new functions with worse names.

We could introduce a Python wrapper for these to give a proper
FutureWarning first.

Cheers,
Ralf



>
>
>
>
>
>
> On Sat, Aug 4, 2018 at 3:25 AM, Ralf Gommers <ralf.gommers at gmail.com>
> wrote:
>
>>
>>
>> On Thu, Aug 2, 2018 at 11:37 AM, Ilhan Polat <ilhanpolat at gmail.com>
>> wrote:
>>
>>> Due their historical evolution, there are certain LAPACK wrappers that
>>> are not standardized. Some work with minimum lwork variables instead of
>>> their optimal values. Also these routines often return quite big arrays
>>> during the lwork queries, to demonstrate :
>>>
>>> import scipy.linalg as la
>>> la.lapack.dgeqrf(a=np.random.rand(400,400), lwork=-1)
>>>
>>> is a workspace size query (via lwork=-1). The current default size is
>>> "3*a.shape[0] + 1" hence 1201. However the optimal workspace size is 12800
>>> on my machine. Therefore the mismatch is sometimes quite dramatic
>>> especially in some other routines. Notice also that to obtain this number
>>> the routine actually returns a 400-long array tau and requires the input
>>> matrix to be transferred back and forth. Moreover, they can't be handled
>>> via scipy.linalg.lapack._compute_lwork function.
>>>
>>> There are a few routines like this and I feel like they should be fixed
>>> and I'm willing to. However this means that their output signature is going
>>> to change which imply backwards compatibility breaks.
>>>
>>
>> What would the output change to? Currently it returns:
>>
>>     qr : rank-2 array('d') with bounds (m,n) and a storage
>>     tau : rank-1 array('d') with bounds (MIN(m,n))
>>     work : rank-1 array('d') with bounds (MAX(lwork,1))
>>     info : int
>>
>> Ralf
>>
>>
>>> I tried to see whether we could deprecate them with new wrappers with
>>> modified names, but to be honest, that would create too many duplicates. On
>>> the other hand I don't have a feeling of how much break this would mean out
>>> there in the wild.
>>>
>>> Is this break an acceptable one or not? (well, none is acceptable
>>> preferably, but in despair...)
>>>
>>> Any other alternatives, thoughts are most welcome.
>>>
>>> best,
>>> ilhan
>>>
>>> _______________________________________________
>>> SciPy-Dev mailing list
>>> SciPy-Dev at python.org
>>> https://mail.python.org/mailman/listinfo/scipy-dev
>>>
>>>
>>
>> _______________________________________________
>> SciPy-Dev mailing list
>> SciPy-Dev at python.org
>> https://mail.python.org/mailman/listinfo/scipy-dev
>>
>>
>
> _______________________________________________
> SciPy-Dev mailing list
> SciPy-Dev at python.org
> https://mail.python.org/mailman/listinfo/scipy-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scipy-dev/attachments/20180804/d1e8615c/attachment.html>