[SciPy-Dev] Backwards Compatibility for low level LAPACK routines

[Ilhan Polat]

Oh also the ones who use get_lapack_funcs machinery

On Fri, Aug 3, 2018, 00:04 Ilhan Polat <ilhanpolat at gmail.com> wrote:

> Cython wrappers won't be affected. This would  only affect the people
> accessing LAPACK routines manually via scipy.linalg.lapack.<funcname> API
> directly.
>
> On Thu, Aug 2, 2018, 23:17 <josef.pktd at gmail.com> wrote:
>
>> On Thu, Aug 2, 2018 at 2:37 PM, Ilhan Polat <ilhanpolat at gmail.com> wrote:
>> > Due their historical evolution, there are certain LAPACK wrappers that
>> are
>> > not standardized. Some work with minimum lwork variables instead of
>> their
>> > optimal values. Also these routines often return quite big arrays
>> during the
>> > lwork queries, to demonstrate :
>> >
>> > import scipy.linalg as la
>> > la.lapack.dgeqrf(a=np.random.rand(400,400), lwork=-1)
>> >
>> > is a workspace size query (via lwork=-1). The current default size is
>> > "3*a.shape[0] + 1" hence 1201. However the optimal workspace size is
>> 12800
>> > on my machine. Therefore the mismatch is sometimes quite dramatic
>> especially
>> > in some other routines. Notice also that to obtain this number the
>> routine
>> > actually returns a 400-long array tau and requires the input matrix to
>> be
>> > transferred back and forth. Moreover, they can't be handled via
>> > scipy.linalg.lapack._compute_lwork function.
>> >
>> > There are a few routines like this and I feel like they should be fixed
>> and
>> > I'm willing to. However this means that their output signature is going
>> to
>> > change which imply backwards compatibility breaks. I tried to see
>> whether we
>> > could deprecate them with new wrappers with modified names, but to be
>> > honest, that would create too many duplicates. On the other hand I don't
>> > have a feeling of how much break this would mean out there in the wild.
>> >
>> > Is this break an acceptable one or not? (well, none is acceptable
>> > preferably, but in despair...)
>> >
>> > Any other alternatives, thoughts are most welcome.
>>
>> Is this only for the python functions or also for the cython wrappers to
>> LAPACK?
>>
>> Binary incompatibilities are pretty painful. We just got rid of the
>> compatibility with scipy's old cython wrapper code in statsmodels.
>> Both distributing binaries and conditional compilation depending on
>> the installed scipy versions is fragile and there doesn't seem to be
>> good packaging support for it.
>>
>> Josef
>>
>>
>>
>> >
>> > best,
>> > ilhan
>> >
>> > _______________________________________________
>> > SciPy-Dev mailing list
>> > SciPy-Dev at python.org
>> > https://mail.python.org/mailman/listinfo/scipy-dev
>> >
>> _______________________________________________
>> SciPy-Dev mailing list
>> SciPy-Dev at python.org
>> https://mail.python.org/mailman/listinfo/scipy-dev
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scipy-dev/attachments/20180803/6d272e74/attachment-0001.html>