[SciPy-Dev] Review of PR 8293 - sampling of random variates

Sun Aug 5 02:59:33 EDT 2018

A good source on the ratio of uniforms method is the book by Devroye:
"non-uniform random variate generation" or the original paper "Computer
Computer Generation of Random Variables using the ratio of uniform
deviates" by kinderman / monahan.

<scipy-dev-request at python.org> schrieb am Sa., 4. Aug. 2018, 18:03:

> Send SciPy-Dev mailing list submissions to
>         scipy-dev at python.org
>
> To subscribe or unsubscribe via the World Wide Web, visit
>         https://mail.python.org/mailman/listinfo/scipy-dev
> or, via email, send a message with subject or body 'help' to
>         scipy-dev-request at python.org
>
> You can reach the person managing the list at
>         scipy-dev-owner at python.org
>
> When replying, please edit your Subject line so it is more specific
> than "Re: Contents of SciPy-Dev digest..."
>
>
> Today's Topics:
>
>    1. Re: Review of PR 8293 - sampling of random variates (Neal Becker)
>    2. Re: Backwards Compatibility for low level LAPACK routines
>       (Ralf Gommers)
>    3. Re: Backwards Compatibility for low level LAPACK routines
>       (Pauli Virtanen)
>
>
> ----------------------------------------------------------------------
>
> Message: 1
> Date: Sat, 4 Aug 2018 07:44:06 -0400
> From: Neal Becker <ndbecker2 at gmail.com>
> To: SciPy Developers List <scipy-dev at python.org>
> Subject: Re: [SciPy-Dev] Review of PR 8293 - sampling of random
>         variates
> Message-ID:
>         <
> CAG3t+pF_5GMXGDJUJF-Z8n2iRHvTSpwG6_b4y1dMWSqwKS_29w at mail.gmail.com>
> Content-Type: text/plain; charset="utf-8"
>
> Don't know about this particular code, but unuran is pretty comprehensive,
> and there's a good book from the author.
>
> On Fri, Aug 3, 2018, 5:37 PM Phillip Feldman <phillip.m.feldman at gmail.com>
> wrote:
>
> > The ability to sample random variates from a distribution where only the
> > density is available seems quite useful.  Is there a reference that
> > describes the method?
> >
> > Phillip
> >
> > On Thu, Aug 2, 2018 at 11:43 PM, Christoph Baumgarten <
> > christoph.baumgarten at gmail.com> wrote:
> >
> >> PR 8293 introduces a method to sample random variates for more complex
> >> distributions to scipy.stats (such as hyperbolic distributions and the
> >> generalized inverse gaussian (see PR 8681)). I think the PR is in good
> >> shape, it would be great if it could be merged soon since i could then
> >> continue to work on adding new distributions. The PR has been open for 7
> >> months now. If someone could continue the review, that would bei great.
> >> Thanks!
> >>
> >> _______________________________________________
> >> SciPy-Dev mailing list
> >> SciPy-Dev at python.org
> >> https://mail.python.org/mailman/listinfo/scipy-dev
> >>
> >>
> > _______________________________________________
> > SciPy-Dev mailing list
> > SciPy-Dev at python.org
> > https://mail.python.org/mailman/listinfo/scipy-dev
> >
> -------------- next part --------------
> An HTML attachment was scrubbed...
> URL: <
> http://mail.python.org/pipermail/scipy-dev/attachments/20180804/73dd5df9/attachment-0001.html
> >
>
> ------------------------------
>
> Message: 2
> Date: Sat, 4 Aug 2018 06:06:22 -0700
> From: Ralf Gommers <ralf.gommers at gmail.com>
> To: SciPy Developers List <scipy-dev at python.org>
> Subject: Re: [SciPy-Dev] Backwards Compatibility for low level LAPACK
>         routines
> Message-ID:
>         <
> CABL7CQgmuAF2V0Pyrtt5qEBPE9M1ostJL54fx8YVVTZbgRF7KQ at mail.gmail.com>
> Content-Type: text/plain; charset="utf-8"
>
> On Sat, Aug 4, 2018 at 12:49 AM, Ilhan Polat <ilhanpolat at gmail.com> wrote:
>
> > It will gain a dgeqrf_lwork function as usual to return the necessary
> > workspace size as
> >
> > lwork, info = dgeqrf_lwork(m,n)
> >
> > Then the "work" variable will be removed from dgeqrf signature and will
> > be made hidden.
> >
> > For example for the previous example I gave before, the optimal size is
> > 12800 and work array is returning an 12800-long array for a 400-400 array
> > computation.
> >
>
> Ah okay. Then the alternative is to just leave the work parameter, ignore
> it in the code if it's passed in (or give a warning/error) and document it
> as not being used. Right?
>
> If you're removing "work" from both the signature and the return value,
> that's a bigger change indeed, that can't be handled well that way. I'm not
> 100% sure, but I think I agree that a backwards incompatible change here
> will be better than introducing a bunch of new functions with worse names.
>
> We could introduce a Python wrapper for these to give a proper
> FutureWarning first.
>
> Cheers,
> Ralf
>
>
>
> >
> >
> >
> >
> >
> >
> > On Sat, Aug 4, 2018 at 3:25 AM, Ralf Gommers <ralf.gommers at gmail.com>
> > wrote:
> >
> >>
> >>
> >> On Thu, Aug 2, 2018 at 11:37 AM, Ilhan Polat <ilhanpolat at gmail.com>
> >> wrote:
> >>
> >>> Due their historical evolution, there are certain LAPACK wrappers that
> >>> are not standardized. Some work with minimum lwork variables instead of
> >>> their optimal values. Also these routines often return quite big arrays
> >>> during the lwork queries, to demonstrate :
> >>>
> >>> import scipy.linalg as la
> >>> la.lapack.dgeqrf(a=np.random.rand(400,400), lwork=-1)
> >>>
> >>> is a workspace size query (via lwork=-1). The current default size is
> >>> "3*a.shape[0] + 1" hence 1201. However the optimal workspace size is
> 12800
> >>> on my machine. Therefore the mismatch is sometimes quite dramatic
> >>> especially in some other routines. Notice also that to obtain this
> number
> >>> the routine actually returns a 400-long array tau and requires the
> input
> >>> matrix to be transferred back and forth. Moreover, they can't be
> handled
> >>> via scipy.linalg.lapack._compute_lwork function.
> >>>
> >>> There are a few routines like this and I feel like they should be fixed
> >>> and I'm willing to. However this means that their output signature is
> going
> >>> to change which imply backwards compatibility breaks.
> >>>
> >>
> >> What would the output change to? Currently it returns:
> >>
> >>     qr : rank-2 array('d') with bounds (m,n) and a storage
> >>     tau : rank-1 array('d') with bounds (MIN(m,n))
> >>     work : rank-1 array('d') with bounds (MAX(lwork,1))
> >>     info : int
> >>
> >> Ralf
> >>
> >>
> >>> I tried to see whether we could deprecate them with new wrappers with
> >>> modified names, but to be honest, that would create too many
> duplicates. On
> >>> the other hand I don't have a feeling of how much break this would
> mean out
> >>> there in the wild.
> >>>
> >>> Is this break an acceptable one or not? (well, none is acceptable
> >>> preferably, but in despair...)
> >>>
> >>> Any other alternatives, thoughts are most welcome.
> >>>
> >>> best,
> >>> ilhan
> >>>
> >>> _______________________________________________
> >>> SciPy-Dev mailing list
> >>> SciPy-Dev at python.org
> >>> https://mail.python.org/mailman/listinfo/scipy-dev
> >>>
> >>>
> >>
> >> _______________________________________________
> >> SciPy-Dev mailing list
> >> SciPy-Dev at python.org
> >> https://mail.python.org/mailman/listinfo/scipy-dev
> >>
> >>
> >
> > _______________________________________________
> > SciPy-Dev mailing list
> > SciPy-Dev at python.org
> > https://mail.python.org/mailman/listinfo/scipy-dev
> >
> >
> -------------- next part --------------
> An HTML attachment was scrubbed...
> URL: <
> http://mail.python.org/pipermail/scipy-dev/attachments/20180804/d1e8615c/attachment-0001.html
> >
>
> ------------------------------
>
> Message: 3
> Date: Sat, 04 Aug 2018 16:00:36 +0200
> From: Pauli Virtanen <pav at iki.fi>
> To: scipy-dev at python.org
> Subject: Re: [SciPy-Dev] Backwards Compatibility for low level LAPACK
>         routines
> Message-ID: <5b464122e6ce8e5a7959f325e534f8d2543801d5.camel at iki.fi>
> Content-Type: text/plain; charset="UTF-8"
>
> la, 2018-08-04 kello 06:06 -0700, Ralf Gommers kirjoitti:
> [clip]
> > Ah okay. Then the alternative is to just leave the work parameter,
> > ignore
> > it in the code if it's passed in (or give a warning/error) and
> > document it
> > as not being used. Right?
> >
> > If you're removing "work" from both the signature and the return
> > value,
> > that's a bigger change indeed, that can't be handled well that way.
> > I'm not
> > 100% sure, but I think I agree that a backwards incompatible change
> > here
> > will be better than introducing a bunch of new functions with worse
> > names.
> >
> > We could introduce a Python wrapper for these to give a proper
> > FutureWarning first.
>
> Or, perhaps you can leave the `work` return variable in, but make it an
> 1-element array? Its value can be filled in from the `callstatement`,
> cf eg
>
> https://github.com/scipy/scipy/blob/master/scipy/linalg/flapack_gen.pyf.src#L89
> The actual work array is then made an intent(hide,cache) variable.
>
>         Pauli
>
>
>
>
> ------------------------------
>
> Subject: Digest Footer
>
> _______________________________________________
> SciPy-Dev mailing list
> SciPy-Dev at python.org
> https://mail.python.org/mailman/listinfo/scipy-dev
>
>
> ------------------------------
>
> End of SciPy-Dev Digest, Vol 178, Issue 6
> *****************************************
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scipy-dev/attachments/20180805/4e670e8a/attachment-0001.html>