[Numpy-discussion] svd error checking vs. speed

[josef.pktd at gmail.com]

On Sat, Feb 15, 2014 at 5:18 PM, alex <argriffi at ncsu.edu> wrote:
> On Sat, Feb 15, 2014 at 5:08 PM,  <josef.pktd at gmail.com> wrote:
>> On Sat, Feb 15, 2014 at 4:56 PM, Sebastian Berg
>> <sebastian at sipsolutions.net> wrote:
>>> On Sa, 2014-02-15 at 16:37 -0500, alex wrote:
>>>> Hello list,
>>>>
>>>> Here's another idea resurrection from numpy github comments that I've
>>>> been advised could be posted here for re-discussion.
>>>>
>>>> The proposal would be to make np.linalg.svd more like scipy.linalg.svd
>>>> with respect to input checking.  The argument against the change is
>>>> raw speed; if you know that you will never feed non-finite input to
>>>> svd, then np.linalg.svd is a bit faster than scipy.linalg.svd.  An
>>>> argument for the change could be to avoid issues reported on github
>>>> like crashes, hangs, spurious non-convergence exceptions, etc. from
>>>> the undefined behavior of svd of non-finite input.
>>>>
>>>
>>> +1, unless this is a huge speed penalty, correctness (and decent error
>>> messages) should come first in my opinion, this is python after all. If
>>> this is a noticable speed difference, a kwarg may be an option (but
>>> would think about that some more).
>>
>> maybe -1
>>
>> statsmodels is using np.linalg.pinv which uses svd
>> I never ran heard of any crash (*), and the only time I compared with
>> scipy I didn't like the slowdown.
>> I didn't do any serious timings just a few examples.
>
> According to https://github.com/statsmodels/statsmodels/blob/master/statsmodels/tools/linalg.py
> statsmodels pinv(A) checks isfinite(A) at least twice and also checks
> for finiteness of the identity matrix.  Or maybe this is not the pinv
> that you meant.

that's dead code

copy of np.pinv used in linear regression
https://github.com/statsmodels/statsmodels/blob/master/statsmodels/tools/tools.py#L348
(it's a recent change to streamline some of the linalg in regression,
and master only)

outside of linear regression we still use almost only np.linalg.pinv directly

Josef

>
> Alex
> _______________________________________________
> NumPy-Discussion mailing list
> NumPy-Discussion at scipy.org
> http://mail.scipy.org/mailman/listinfo/numpy-discussion