From ralf.gommers at gmail.com  Thu Mar  1 02:21:01 2018
From: ralf.gommers at gmail.com (Ralf Gommers)
Date: Wed, 28 Feb 2018 23:21:01 -0800
Subject: [SciPy-Dev] GSoC 2018: Blendenpik (A least-square solver)
In-Reply-To: <CAJfd0CPu=h93pVZAZzgdz521ngkZi9uaAWLsmPKGSGU4_mkGQg@mail.gmail.com>
References: <CAJfd0CPu=h93pVZAZzgdz521ngkZi9uaAWLsmPKGSGU4_mkGQg@mail.gmail.com>
Message-ID: <CABL7CQi+jrA0AfuguVgaJotxN=-PLuOL-i4jeGP3V_nB99xkuw@mail.gmail.com>

Hi Jordi,

On Wed, Feb 28, 2018 at 7:51 AM, Jordi Montes <jomsdev at gmail.com> wrote:

> Hello,
>
> I am Jordi Montes, an enthusiastic student of discrete mathematics wanting
> to participate in this GSoC.
>
> My idea is to take advantage of the current method for dimensinality
> reduction of scipy (clarkson_woodruff_transformation) and build a
> least-square solver on top of it. The principal motivation is that it
> outperforms the other solvers in many real world applications, even those
> in LAPACK.
>
> I already discussed this in the mailing list before the GSoC admission
> term started, but I have also written a formal proposal (which is attached
> in this email).
>

That looks like a good start! Note that the proposal you have submit to
Google needs to have some sections that you are still missing, like a
timeline (broken down by week, with the work for that week) and links to
previous PRs. Here is the PSF template info to use:
http://python-gsoc.org/studenttemplate.html

It would be useful to reference Blendenpik and add a few more details about
it. Think also about answering a few obvious questions about it, like what
are the main challenges when implementing it, and how do you benchmark its
performance.

It's great to see that you've found a co-mentor who is intimately familiar
with the domain and algorithms.

Cheers,
Ralf


> As always, feel free to ask about any doubt that comes to your mind or
> point to any improvement that you think I could make.
>
>
> Thanks,
>
> Jordi.
>
> _______________________________________________
> SciPy-Dev mailing list
> SciPy-Dev at python.org
> https://mail.python.org/mailman/listinfo/scipy-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scipy-dev/attachments/20180228/ff5ebd7d/attachment-0001.html>

From pierre.debuyl at kuleuven.be  Thu Mar  1 04:18:21 2018
From: pierre.debuyl at kuleuven.be (Pierre de Buyl)
Date: Thu, 1 Mar 2018 10:18:21 +0100
Subject: [SciPy-Dev] GSoC 2018 : Rotation formalism
In-Reply-To: <CAH6Pt5obrwEO=uR1o3PQRZGyJwCijHAmgrb+Pzm7BcyQ7DgNzA@mail.gmail.com>
References: <372936706.2127.1519741937865@wamui-jasmine.atl.sa.earthlink.net>
 <CAACPFu5x8W+gXiJV-gdjMKQnxL51OJ1uBEx8MqmW8_Rg6A29OA@mail.gmail.com>
 <56F42377-BF68-4F8D-AAB0-7603C72A1D37@inria.fr>
 <CAH6Pt5obrwEO=uR1o3PQRZGyJwCijHAmgrb+Pzm7BcyQ7DgNzA@mail.gmail.com>
Message-ID: <20180301091821.GI11663@pi-x230>

On Tue, Feb 27, 2018 at 10:06:56PM +0000, Matthew Brett wrote:
> Also : http://matthew-brett.github.io/transforms3d/
> 
> This is mostly Christoph Gohlke's code, but I've been the maintainer
> for a while:
> 
> * rotation matrices
> * quaternions

In the case of quaternions, there are two typical storages: [w, x, y, z] (scalar
first) or [x, y, z, w] (scalar last). Is there a "standard" here?

Regards,

Pierre


From lagru at mailbox.org  Thu Mar  1 06:20:10 2018
From: lagru at mailbox.org (Lars G.)
Date: Thu, 1 Mar 2018 12:20:10 +0100
Subject: [SciPy-Dev] GSoC 2018: Cythonizing
In-Reply-To: <CAGu2niVLwZpw-NimvLb7=i36wQp9h9-tkxg72hW2bfQON6Zq6A@mail.gmail.com>
References: <3648ed21-1b43-975c-4ca0-05926d683736@mailbox.org>
 <CAGu2niVLwZpw-NimvLb7=i36wQp9h9-tkxg72hW2bfQON6Zq6A@mail.gmail.com>
Message-ID: <e404c01f-9ace-58a8-1f65-4878ce4ad916@mailbox.org>

On 28.02.2018 16:04, Eric Larson wrote:
> For GSoC we need to ensure (at least) that the project fits 1) the needs
> of SciPy, 2) the GSoC program scope / timeline, 3) possible mentors, and
> 4) your goals. My sense is that a proposal based on code Cythonizing
> (with proper benchmark testing and regression protection) would be good
> for SciPy maintainability and could be crafted to have a reasonable
> scope. In terms of mentors, I feel comfortable mentoring changes to the
> `signal` module but not `ndimage`, so we'd need to find a qualified
> primary volunteer mentor if that ends up being the primary proposal
> direction.
Actually, considering that my background lies in electrical engineering
I'd be more than happy to focus on the `signal` module. And from the
other response it seems like cythonizing `ndimage` wouldn't be a good idea.

> Another thing to keep in mind is that the list of GSoC ideas is not
> meant to be exhaustive. So if you have some other ideas for SciPy
> functionality, feel free to throw those out for discussion as well. In
> my experience, genuine intrinsic enthusiasm for a project -- finding
> something you'd enjoy working on in your free time even if you weren't
> getting paid to do so -- can help make for successful GSoC applications
> and experiences.

So there would be enough candidates for Cythonization in `scipy.signal`
to fit the scope of GSoC? I myself can only guess where this would be
wanted and useful.

It doesn't have to be Cythonizing either. I'd be happy to add missing
functionality to the `signal` module or rework stuff that needs it.
The content in
https://docs.scipy.org/doc/scipy-1.0.0/reference/roadmap.html#signal
doesn't seem to be a good fit for a GSoC project. The only thing I can
think of right now is to extend the API for and add more adaptive filters:
https://en.wikipedia.org/wiki/Adaptive_filter
Again, I'm not sure this is wanted or if I'm judging the need correctly.

If you guys have any ideas or wishes in that direction I'd be happy to
hear them.

Best regards,
Lars

From nikolay.mayorov at zoho.com  Thu Mar  1 08:13:08 2018
From: nikolay.mayorov at zoho.com (Nikolay Mayorov)
Date: Thu, 01 Mar 2018 18:13:08 +0500
Subject: [SciPy-Dev] GSoC 2018: Cythonizing
In-Reply-To: <e404c01f-9ace-58a8-1f65-4878ce4ad916@mailbox.org>
References: <3648ed21-1b43-975c-4ca0-05926d683736@mailbox.org>
 <CAGu2niVLwZpw-NimvLb7=i36wQp9h9-tkxg72hW2bfQON6Zq6A@mail.gmail.com>
 <e404c01f-9ace-58a8-1f65-4878ce4ad916@mailbox.org>
Message-ID: <161e1b1e91a.f3b04efd27571.76008067328158801@zoho.com>

Hey, Lars!



I don't want to rob other potential ideas or mentors, but you mentioned that you think that rotation formalism idea is already for someone else. It is absolutely not the case, nothing is settled, and if you are interested in this subject --- I'm interested to see your ideas or a proposal (and Eric Larson likely as well).



Best,

Nikolay




---- On Thu, 01 Mar 2018 16:20:10 +0500 Lars G. &lt;lagru at mailbox.org&gt; wrote ----




On 28.02.2018 16:04, Eric Larson wrote: 

&gt; For GSoC we need to ensure (at least) that the project fits 1) the needs 

&gt; of SciPy, 2) the GSoC program scope / timeline, 3) possible mentors, and 

&gt; 4) your goals. My sense is that a proposal based on code Cythonizing 

&gt; (with proper benchmark testing and regression protection) would be good 

&gt; for SciPy maintainability and could be crafted to have a reasonable 

&gt; scope. In terms of mentors, I feel comfortable mentoring changes to the 

&gt; `signal` module but not `ndimage`, so we'd need to find a qualified 

&gt; primary volunteer mentor if that ends up being the primary proposal 

&gt; direction. 

Actually, considering that my background lies in electrical engineering 

I'd be more than happy to focus on the `signal` module. And from the 

other response it seems like cythonizing `ndimage` wouldn't be a good idea. 

 

&gt; Another thing to keep in mind is that the list of GSoC ideas is not 

&gt; meant to be exhaustive. So if you have some other ideas for SciPy 

&gt; functionality, feel free to throw those out for discussion as well. In 

&gt; my experience, genuine intrinsic enthusiasm for a project -- finding 

&gt; something you'd enjoy working on in your free time even if you weren't 

&gt; getting paid to do so -- can help make for successful GSoC applications 

&gt; and experiences. 

 

So there would be enough candidates for Cythonization in `scipy.signal` 

to fit the scope of GSoC? I myself can only guess where this would be 

wanted and useful. 

 

It doesn't have to be Cythonizing either. I'd be happy to add missing 

functionality to the `signal` module or rework stuff that needs it. 

The content in 

https://docs.scipy.org/doc/scipy-1.0.0/reference/roadmap.html#signal 

doesn't seem to be a good fit for a GSoC project. The only thing I can 

think of right now is to extend the API for and add more adaptive filters: 

https://en.wikipedia.org/wiki/Adaptive_filter 

Again, I'm not sure this is wanted or if I'm judging the need correctly. 

 

If you guys have any ideas or wishes in that direction I'd be happy to 

hear them. 

 

Best regards, 

Lars 

_______________________________________________ 

SciPy-Dev mailing list 

SciPy-Dev at python.org 

https://mail.python.org/mailman/listinfo/scipy-dev 






-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scipy-dev/attachments/20180301/7254ed9a/attachment.html>

From nikolay.mayorov at zoho.com  Thu Mar  1 08:21:51 2018
From: nikolay.mayorov at zoho.com (Nikolay Mayorov)
Date: Thu, 01 Mar 2018 18:21:51 +0500
Subject: [SciPy-Dev] GSoC 2018 : Rotation formalism
In-Reply-To: <20180301091821.GI11663@pi-x230>
References: <372936706.2127.1519741937865@wamui-jasmine.atl.sa.earthlink.net>
 <CAACPFu5x8W+gXiJV-gdjMKQnxL51OJ1uBEx8MqmW8_Rg6A29OA@mail.gmail.com>
 <56F42377-BF68-4F8D-AAB0-7603C72A1D37@inria.fr>
 <CAH6Pt5obrwEO=uR1o3PQRZGyJwCijHAmgrb+Pzm7BcyQ7DgNzA@mail.gmail.com>
 <20180301091821.GI11663@pi-x230>
Message-ID: <161e1b9e4d8.fe99cfce27647.3667928799815692430@zoho.com>

I believe there is no standard for this. And there are several other things similar to this, like "passive" or "active" view on rotations, multiplication order for composition, etc. In my opinion this is sort of a "soft challenge" for this idea --- document everything precisely.



Best regards,

Nikolay




---- On Thu, 01 Mar 2018 14:18:21 +0500  Pierre de Buyl &lt;pierre.debuyl at kuleuven.be&gt; wrote ---- 




On Tue, Feb 27, 2018 at 10:06:56PM +0000, Matthew Brett wrote: 

&gt; Also : http://matthew-brett.github.io/transforms3d/ 

&gt; 

&gt; This is mostly Christoph Gohlke's code, but I've been the maintainer 

&gt; for a while: 

&gt; 

&gt; * rotation matrices 

&gt; * quaternions 



In the case of quaternions, there are two typical storages: [w, x, y, z] (scalar 

first) or [x, y, z, w] (scalar last). Is there a "standard" here? 



Regards, 



Pierre 



_______________________________________________ 

SciPy-Dev mailing list 

SciPy-Dev at python.org 

https://mail.python.org/mailman/listinfo/scipy-dev 










-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scipy-dev/attachments/20180301/e1f0f36b/attachment-0001.html>

From lagru at mailbox.org  Thu Mar  1 09:32:33 2018
From: lagru at mailbox.org (Lars G.)
Date: Thu, 1 Mar 2018 15:32:33 +0100
Subject: [SciPy-Dev] GSoC 2018: Cythonizing
In-Reply-To: <161e1b1e91a.f3b04efd27571.76008067328158801@zoho.com>
References: <3648ed21-1b43-975c-4ca0-05926d683736@mailbox.org>
 <CAGu2niVLwZpw-NimvLb7=i36wQp9h9-tkxg72hW2bfQON6Zq6A@mail.gmail.com>
 <e404c01f-9ace-58a8-1f65-4878ce4ad916@mailbox.org>
 <161e1b1e91a.f3b04efd27571.76008067328158801@zoho.com>
Message-ID: <fc3b843f-9b92-1a71-c63a-a1f126f04f2a@mailbox.org>

On 01.03.2018 14:13, Nikolay Mayorov wrote:
> Hey, Lars!
> 
> I don't want to rob other potential ideas or mentors, but you mentioned
> that you think that rotation formalism idea is already for someone else.
> It is absolutely not the case, nothing is settled, and if you are
> interested in this subject --- I'm interested to see your ideas or a
> proposal (and Eric Larson likely as well).
> 
> Best,
> Nikolay

The topic does indeed sound interesting and from what it looks like you
already have a pretty clear description of the scope, structure and
goals. However I have never done any relevant programming in that area
so I currently don't feel very confident that I'll be able to come up
with a sensible API for that or make informed decisions. First, I'll see
what comes of my first suggestions.

In the meantime I will look through the linked references and see if I
feel more confident afterwards.

Best regards,
Lars

From toddrjen at gmail.com  Thu Mar  1 10:40:16 2018
From: toddrjen at gmail.com (Todd)
Date: Thu, 1 Mar 2018 10:40:16 -0500
Subject: [SciPy-Dev] GSoC 2018: Cythonizing
In-Reply-To: <e404c01f-9ace-58a8-1f65-4878ce4ad916@mailbox.org>
References: <3648ed21-1b43-975c-4ca0-05926d683736@mailbox.org>
 <CAGu2niVLwZpw-NimvLb7=i36wQp9h9-tkxg72hW2bfQON6Zq6A@mail.gmail.com>
 <e404c01f-9ace-58a8-1f65-4878ce4ad916@mailbox.org>
Message-ID: <CAFpSVp+xP5_eGEMci7o7bijY7_HFcK2TE5FJ4TcREg5OJZRpSQ@mail.gmail.com>

On Mar 1, 2018 06:20, "Lars G." <lagru at mailbox.org> wrote:

> On 28.02.2018 16:04, Eric Larson wrote:
> > For GSoC we need to ensure (at least) that the project fits 1) the needs
> > of SciPy, 2) the GSoC program scope / timeline, 3) possible mentors, and
> > 4) your goals. My sense is that a proposal based on code Cythonizing
> > (with proper benchmark testing and regression protection) would be good
> > for SciPy maintainability and could be crafted to have a reasonable
> > scope. In terms of mentors, I feel comfortable mentoring changes to the
> > `signal` module but not `ndimage`, so we'd need to find a qualified
> > primary volunteer mentor if that ends up being the primary proposal
> > direction.
> Actually, considering that my background lies in electrical engineering
> I'd be more than happy to focus on the `signal` module. And from the
> other response it seems like cythonizing `ndimage` wouldn't be a good idea.
>
> > Another thing to keep in mind is that the list of GSoC ideas is not
> > meant to be exhaustive. So if you have some other ideas for SciPy
> > functionality, feel free to throw those out for discussion as well. In
> > my experience, genuine intrinsic enthusiasm for a project -- finding
> > something you'd enjoy working on in your free time even if you weren't
> > getting paid to do so -- can help make for successful GSoC applications
> > and experiences.
>
> So there would be enough candidates for Cythonization in `scipy.signal`
> to fit the scope of GSoC? I myself can only guess where this would be
> wanted and useful.
>
> It doesn't have to be Cythonizing either. I'd be happy to add missing
> functionality to the `signal` module or rework stuff that needs it.
> The content in
> https://docs.scipy.org/doc/scipy-1.0.0/reference/roadmap.html#signal
> doesn't seem to be a good fit for a GSoC project. The only thing I can
> think of right now is to extend the API for and add more adaptive filters:
> https://en.wikipedia.org/wiki/Adaptive_filter
> Again, I'm not sure this is wanted or if I'm judging the need correctly.
>
> If you guys have any ideas or wishes in that direction I'd be happy to
> hear them.
>
> Best regards,
> Lars
>

The first issue listed in the roadmap, convolution, is a much more
complicated issue than that description makes out.  There are a few issues,
some with some overlap behind-the-scenes:

   1. As discussed, there are a bunch of different implementations that
that use different algorithm that work better in different scenarios.
Ideally there would be one "master" function that would pick the best
algorithm for a given set of parameters.  This will depend on the number of
dimensions to be convolved over, the size of the the first signal to be
convolved, and the size of the second signal to be convolved.  Changing any
one of these can change which implementation is optimal, or even useful.
So for with vectors, it is better to use a different algorithm if the one
vector is short, if both vectors are long but one is much longer, and if
both vectors are long and of similar length.
   2. We don't have the best algorithms implemented for all  of these
scenarios.  For example the "both vectors are long but one is much longer"
scenario is best with the overlap-add algorithm, which scipy doesn't have.
Similarly, there is an fft-based version of correlation equivalent to
fftconvolve that isn't implemented, 2D and n-d versions of fft convolution
and correlation that aren't implemented, etc.
   3. The implementations only work over the number of dimensions they
apply to.  So the 1D implementations can only take vectors, the 2D
implementations can only take 2D arrays, etc.  There is no way to, say,
apply a filter along the second dimension of a 3D signal.  In order to
implement the "master" function, at least one implementation (and ideally
all implementations) should be able to be applied across additional
dimensions.

And there is overlap between these.  For example I mention the overlap-add
method in point 2, but that would most likely be implemented in part by
applying across dimensions as mentioned in point 3.

A lot of these issues apply elsewhere in scipy.signal.  For example the
stft/spectrogram uses a slow, naive implementation.  A lot of the functions
don't support applying across multidimensional arrays (for example to
create a filter bank).
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scipy-dev/attachments/20180301/918447f0/attachment.html>

From charlesr.harris at gmail.com  Thu Mar  1 11:15:53 2018
From: charlesr.harris at gmail.com (Charles R Harris)
Date: Thu, 1 Mar 2018 09:15:53 -0700
Subject: [SciPy-Dev] Spline interpolation in ndimage
In-Reply-To: <CAPOWHWkhQvKuia8NYBT4zTmpSRXmnH_MZExcBAUTvNgURaopgw@mail.gmail.com>
References: <CAPOWHWkhQvKuia8NYBT4zTmpSRXmnH_MZExcBAUTvNgURaopgw@mail.gmail.com>
Message-ID: <CAB6mnxL2kz-JjYY0ORumj44=7XC_gs10Ba3OqR0aqp++zcY_Zg@mail.gmail.com>

On Tue, Feb 27, 2018 at 4:59 AM, Jaime Fern?ndez del R?o <
jaime.frio at gmail.com> wrote:

> Hi all,
>
> We have been discussing in issue #8465
> <https://github.com/scipy/scipy/issues/8465> about the extension modes
> for the interpolation module of ndimage. As some other issues linked in
> that one show, this is an old problem, that has been recurringly discussed,
> and there mostly is agreement on what the correct behavior should be. But
> as all things ndimage, implementing any change is hard, in this case not
> only because the code is complicated, but also because of the mathematical
> subtleties of the interpolation method used.
>
> In trying to come up with a way forward for this I think I can handle the
> code complexity, but am having trouble being sure that I come up with a
> sound math approach. I think I have more or less figured it out, but I
> don't have a good academic background on signal processing, so was hoping
> that someone who does (I'm thinking of you, Chuck!) can read my longish
> description of things below and validate or correct it.
>
> My main source of knowledge has been:
>
> M. Unser, "Splines: a perfect fit for signal and image processing," in
> IEEE Signal Processing Magazine, vol. 16, no. 6, pp. 22-38, Nov 1999.
>
> Recommendations for bigger, better readings on the subject are also
> welcome!
>
> -----
>
> In what follows I'll assume the input image is 2-D and NxN for simplicity,
> but I think everything generalizes easily
>
> If I'm understanding it right, all of our interpolation functions use
> B-splines for interpolation. So instead of having NxN discrete values on a
> grid, we have NxN B-splines centered at the grid points, and we keep the
> NxN coefficients multiplying each spline to reproduce the original image at
> the grid points exactly. If the spline order is < 2, the spline
> coefficients are the same as the image values. But if order >= 2 (and we
> use 3 by default), these have to be calculated in a non-trivial fashion.
> This can be done efficiently by applying a separable filter, which is
> implemented in ndimage.spline_filter.
>
> Because B-splines have compact support, when using splines of order n we
> only need to consider the B-splines on an (n + 1)x(n + 1) grid neighborhood
> of the point being interpolated. This is more or less straightforward,
> until you move close to the edges and all of a sudden some of the points in
> your grid neighborhood fall outside the original image grid. We have our
> extend mode, which controls how this points outside should be mapped to
> points inside. But here is where things start getting tricky...
>
> When the spline coefficients are computed (i.e. when ndimage.spline_filter
> is called), assumptions have to be made about boundary conditions. But
> ndimage.spline_filter does not take a mode parameter to control this! So
> what I think ndimage does is compute the spline coefficients assuming
> "mirror symmetric" boundary conditions, i.e.:
>
> a b c d c b | a b c d | c b a b c d
>
> So if our interpolated point is within the image boundaries, but some of
> the grid points in its (n + 1)x(n + 1) neighborhood fall outside the
> boundary, the code uses a mirror symmetric extension mode to fill in those
> values. This approach has a funny smell to it, but even if it's formally
> incorrect I think it would only be marginally so, as the values are
> probably going to be very close to the correct ones.
>
> The problem comes when the point being interpolated is outside the
> boundaries of the image. We cannot use mirror-symmetric spline coefficients
> to extend if e.g. we have been asked to extend using wrap mode. So what
> ndimage does is first map the value outside the boundaries to a value
> within the boundaries, using the given extension mode, then interpolate it
> as before, using mirror-symmetric coefficients if needed because its (n +
> 1)x(n + 1) neighborhood extends outside. Again, this smells funny, but it
> is either correct or very close to correct.
>

The problem with the factorization is that it assumes infinite data points
in all dimensions, i.e, no explicit boundary conditions. With finite data
there are edge effects when the data is deconvolved to get the spline
coefficients.  The way that ndimage deals with that is to extend the data
using reflection and start far enough away that the edge effects have died
away by the time the "real" data has been reached. How far away that is, is
heuristic. I think it should not be too difficult to extend the data in the
other ways, but note that since uniform splines are being used, the
relevant coefficients lie outside the boundary and need to be picked up
from the correct spots in the interior using the symmetries.

IIRC, the b-splines are always centered at zero so that they are
symmetrical. For odd order splines that will be at the center of a pixel
(pixel points), for even order splines at an edge, pixel centers at half
integer points. The data can always be considered to be at pixel centers,
but the splines are displaced. I don't remember if ndimage treats that
correctly.



> This is mostly all good and well, except for the "wrap" and "reflect"
> extension modes: in these cases the area within one pixel of the image
> boundaries is different from anything inside the image, so we cannot use
> that approach. So what ndimage does is basically make shit up and use
> something similar, but not quite right. "reflect" is mostly correct, except
> for within that pixel of the boundary, but "wrap" is a surprising and
> confusing mess.
>
> So how do we fix this? I see two ways forward:
>
>    1. What seems the more correct approach would be to compute the spline
>    coefficients taking into account the extension mode to be used, then use
>    the same extension mode to fill in the neighborhood values when
>    interpolating for a point outside the boundaries.
>    1. First question is whether this is doable? I need to work out the
>       math, but for "wrap" it looks like it should be, not 100% sure if also is
>       for "reflect".
>       2. Assuming it is it has the main advantage of doing things in a
>       more general and understandable way once you have enough background
>       knowledge.
>       3. It does go a little bit against our API design: you can control
>       whether the input is spline-filtered automatically with a parameter, the
>       idea being that you may want to do the filtering yourself if you are going
>       to apply several different transformations to the same image. If the mode
>       of the filtering has to be synced with the mode of the transformation,
>       letting the user do it themselves is a recipe for disaster, because it's
>       going to lead to very hard to track bugs.
>       4. As elsewhere in ndimage, the current implementation does a lot
>       of caching, which works because it always interpolates for a point within
>       the image boundaries. If we started interpolating for points outside the
>       boundaries without first mapping to within there may be a performance hit
>       which has to be evaluated.
>    2. The other approach is,  for "wrap" and "reflect" modes, pad the
>    input image with an extra pixel in each direction, then compute our
>    current "mirror symmetric" spline coefficients, and leave things as they
>    are right now, aside from some changes to the mapping of values to take the
>    extra pixels into account.
>       1. This looks like a nightmare of special cases everywhere and
>       potential off-by-one errors while putting it together, but it would just go
>       along with the ugliness of the existing code.
>       2. It's unclear what we would do if we are given an input with
>       prefilter=False, so this also breaks the current API, probably even more so.
>
> Any thoughts or recommendations are very welcome!
>

Chuck
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scipy-dev/attachments/20180301/b0bbb821/attachment-0001.html>

From charlesr.harris at gmail.com  Fri Mar  2 15:24:33 2018
From: charlesr.harris at gmail.com (Charles R Harris)
Date: Fri, 2 Mar 2018 13:24:33 -0700
Subject: [SciPy-Dev] Spline interpolation in ndimage
In-Reply-To: <CAB6mnxL2kz-JjYY0ORumj44=7XC_gs10Ba3OqR0aqp++zcY_Zg@mail.gmail.com>
References: <CAPOWHWkhQvKuia8NYBT4zTmpSRXmnH_MZExcBAUTvNgURaopgw@mail.gmail.com>
 <CAB6mnxL2kz-JjYY0ORumj44=7XC_gs10Ba3OqR0aqp++zcY_Zg@mail.gmail.com>
Message-ID: <CAB6mnxJB-N-MYgpvzqNb=xdrW22WmmAwzkJUTsnrxcm-Gt1tpg@mail.gmail.com>

On Thu, Mar 1, 2018 at 9:15 AM, Charles R Harris <charlesr.harris at gmail.com>
wrote:

>
>
> On Tue, Feb 27, 2018 at 4:59 AM, Jaime Fern?ndez del R?o <
> jaime.frio at gmail.com> wrote:
>
>> Hi all,
>>
>> We have been discussing in issue #8465
>> <https://github.com/scipy/scipy/issues/8465> about the extension modes
>> for the interpolation module of ndimage. As some other issues linked in
>> that one show, this is an old problem, that has been recurringly discussed,
>> and there mostly is agreement on what the correct behavior should be. But
>> as all things ndimage, implementing any change is hard, in this case not
>> only because the code is complicated, but also because of the mathematical
>> subtleties of the interpolation method used.
>>
>> In trying to come up with a way forward for this I think I can handle the
>> code complexity, but am having trouble being sure that I come up with a
>> sound math approach. I think I have more or less figured it out, but I
>> don't have a good academic background on signal processing, so was hoping
>> that someone who does (I'm thinking of you, Chuck!) can read my longish
>> description of things below and validate or correct it.
>>
>> My main source of knowledge has been:
>>
>> M. Unser, "Splines: a perfect fit for signal and image processing," in
>> IEEE Signal Processing Magazine, vol. 16, no. 6, pp. 22-38, Nov 1999.
>>
>> Recommendations for bigger, better readings on the subject are also
>> welcome!
>>
>> -----
>>
>> In what follows I'll assume the input image is 2-D and NxN for
>> simplicity, but I think everything generalizes easily
>>
>> If I'm understanding it right, all of our interpolation functions use
>> B-splines for interpolation. So instead of having NxN discrete values on a
>> grid, we have NxN B-splines centered at the grid points, and we keep the
>> NxN coefficients multiplying each spline to reproduce the original image at
>> the grid points exactly. If the spline order is < 2, the spline
>> coefficients are the same as the image values. But if order >= 2 (and we
>> use 3 by default), these have to be calculated in a non-trivial fashion.
>> This can be done efficiently by applying a separable filter, which is
>> implemented in ndimage.spline_filter.
>>
>> Because B-splines have compact support, when using splines of order n we
>> only need to consider the B-splines on an (n + 1)x(n + 1) grid neighborhood
>> of the point being interpolated. This is more or less straightforward,
>> until you move close to the edges and all of a sudden some of the points in
>> your grid neighborhood fall outside the original image grid. We have our
>> extend mode, which controls how this points outside should be mapped to
>> points inside. But here is where things start getting tricky...
>>
>> When the spline coefficients are computed (i.e. when
>> ndimage.spline_filter is called), assumptions have to be made about
>> boundary conditions. But ndimage.spline_filter does not take a mode
>> parameter to control this! So what I think ndimage does is compute the
>> spline coefficients assuming "mirror symmetric" boundary conditions, i.e.:
>>
>> a b c d c b | a b c d | c b a b c d
>>
>> So if our interpolated point is within the image boundaries, but some of
>> the grid points in its (n + 1)x(n + 1) neighborhood fall outside the
>> boundary, the code uses a mirror symmetric extension mode to fill in those
>> values. This approach has a funny smell to it, but even if it's formally
>> incorrect I think it would only be marginally so, as the values are
>> probably going to be very close to the correct ones.
>>
>> The problem comes when the point being interpolated is outside the
>> boundaries of the image. We cannot use mirror-symmetric spline coefficients
>> to extend if e.g. we have been asked to extend using wrap mode. So what
>> ndimage does is first map the value outside the boundaries to a value
>> within the boundaries, using the given extension mode, then interpolate it
>> as before, using mirror-symmetric coefficients if needed because its (n +
>> 1)x(n + 1) neighborhood extends outside. Again, this smells funny, but it
>> is either correct or very close to correct.
>>
>
> The problem with the factorization is that it assumes infinite data points
> in all dimensions, i.e, no explicit boundary conditions. With finite data
> there are edge effects when the data is deconvolved to get the spline
> coefficients.  The way that ndimage deals with that is to extend the data
> using reflection and start far enough away that the edge effects have died
> away by the time the "real" data has been reached. How far away that is, is
> heuristic. I think it should not be too difficult to extend the data in the
> other ways, but note that since uniform splines are being used, the
> relevant coefficients lie outside the boundary and need to be picked up
> from the correct spots in the interior using the symmetries.
>
> IIRC, the b-splines are always centered at zero so that they are
> symmetrical. For odd order splines that will be at the center of a pixel
> (pixel points), for even order splines at an edge, pixel centers at half
> integer points. The data can always be considered to be at pixel centers,
> but the splines are displaced. I don't remember if ndimage treats that
> correctly.
>
>
>
>> This is mostly all good and well, except for the "wrap" and "reflect"
>> extension modes: in these cases the area within one pixel of the image
>> boundaries is different from anything inside the image, so we cannot use
>> that approach. So what ndimage does is basically make shit up and use
>> something similar, but not quite right. "reflect" is mostly correct, except
>> for within that pixel of the boundary, but "wrap" is a surprising and
>> confusing mess.
>>
>> So how do we fix this? I see two ways forward:
>>
>>    1. What seems the more correct approach would be to compute the
>>    spline coefficients taking into account the extension mode to be used, then
>>    use the same extension mode to fill in the neighborhood values when
>>    interpolating for a point outside the boundaries.
>>    1. First question is whether this is doable? I need to work out the
>>       math, but for "wrap" it looks like it should be, not 100% sure if also is
>>       for "reflect".
>>       2. Assuming it is it has the main advantage of doing things in a
>>       more general and understandable way once you have enough background
>>       knowledge.
>>       3. It does go a little bit against our API design: you can control
>>       whether the input is spline-filtered automatically with a parameter, the
>>       idea being that you may want to do the filtering yourself if you are going
>>       to apply several different transformations to the same image. If the mode
>>       of the filtering has to be synced with the mode of the transformation,
>>       letting the user do it themselves is a recipe for disaster, because it's
>>       going to lead to very hard to track bugs.
>>       4. As elsewhere in ndimage, the current implementation does a lot
>>       of caching, which works because it always interpolates for a point within
>>       the image boundaries. If we started interpolating for points outside the
>>       boundaries without first mapping to within there may be a performance hit
>>       which has to be evaluated.
>>    2. The other approach is,  for "wrap" and "reflect" modes, pad the
>>    input image with an extra pixel in each direction, then compute our
>>    current "mirror symmetric" spline coefficients, and leave things as they
>>    are right now, aside from some changes to the mapping of values to take the
>>    extra pixels into account.
>>       1. This looks like a nightmare of special cases everywhere and
>>       potential off-by-one errors while putting it together, but it would just go
>>       along with the ugliness of the existing code.
>>       2. It's unclear what we would do if we are given an input with
>>       prefilter=False, so this also breaks the current API, probably even more so.
>>
>> Any thoughts or recommendations are very welcome!
>>
>
>
To explicate a bit more as to why the b-splines are centered, consider
quadratic and cubic splines when the data points are considered to occur at
the pixel centers.

*Quadratic (even order)*


1. If the data points are taken to be halfway between  knot points, we need
to deconvolve the sequence `array([1, 6, 1])/8`. The Fourier transform of
that has no zeros, indeed, is rather smooth, and two extra coefficients,
one at each end, are required to interpolate out to the pixel edges. It is
all nicely symmetrical.

2. If the data points are taken to correspond to the knot points, we need
to deconvolve the sequence `array([1, 1])/2`. The Fourier transform of that
sequence has a zero at the Nyquist, not good, and one extra coefficient is
needed at some arbitrary end in order to get interpolation out to the pixel
edges.

*Cubic (odd order)*

1. Taking the data points to correspond to the knot points, we need to
deconvolve the sequence `array([1, 4, 1])/6`, whose Fourier transform is
nicely behaved. However *four* extra extra coefficients, two at each end,
are required to interpolate out to the pixel edges. The "extra"
coefficients are needed in order to cover the outer half of the edge
pixels, otherwise we are extrapolating. I haven't looked, but I wonder if
ndimage gets that right?


In both cases, the corner pixels may be a bit of a problem. I haven't
thought through that bit.

Chuck
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scipy-dev/attachments/20180302/1b3e61f6/attachment.html>

From charlesr.harris at gmail.com  Fri Mar  2 16:57:18 2018
From: charlesr.harris at gmail.com (Charles R Harris)
Date: Fri, 2 Mar 2018 14:57:18 -0700
Subject: [SciPy-Dev] GSoC 2018: Cythonizing
In-Reply-To: <CAPOWHW=KW0Eny760_A2GN41cyYRZPA3griF+G=1yRejfPO4pwg@mail.gmail.com>
References: <3648ed21-1b43-975c-4ca0-05926d683736@mailbox.org>
 <CAPOWHW=KW0Eny760_A2GN41cyYRZPA3griF+G=1yRejfPO4pwg@mail.gmail.com>
Message-ID: <CAB6mnxLs9X21pyGhzaF0-Vyec54d-Xt7v2MNgW2CGJ-bDPnJ=A@mail.gmail.com>

On Wed, Feb 28, 2018 at 9:11 AM, Jaime Fern?ndez del R?o <
jaime.frio at gmail.com> wrote:

> On Wed, Feb 28, 2018 at 3:40 PM Lars G. <lagru at mailbox.org> wrote:
>
>> Dear SciPy devs,
>>
>> I'm currently thinking about an application for this year's GSoC as
>> well. As there already seems to be a large interest in the rotation
>> formalism I'm trying to find another area that matches my interest and
>> skill.
>>
>> I've dug up this proposal in scikit-image from GSoC 2015
>> https://github.com/scikit-image/scikit-image/wiki/GSoC-
>> 2015#rewriting-scipyndimage-in-cython
>> and judging by the state of scipy/ndimage/src/ nobody has worked on this
>> proposal yet (feel free to correct me).
>>
>
> I mentored the not very successful, to put it mildly, GSoC 2015 project
> about cythonizing ndimage. From that experience, and further work
> afterwards, I no longer think Cython is the answer to ndimage's problems.
> The underlying C has a lot of very complicated code making lots of clever
> (often too clever for everyone's good) uses of pointer magic, that I
> honestly think are better kept in C. Or would at least need someone with a
> very deep understanding of both C and Cython, would much exceed the scope
> of a GSoC project, and I don't think I can commit to properly mentor such a
> project this summer.
>
> What would be nice is to replace the current nd_image.c file that
> implements the Python interface to the underlying C by a Cython
> implementation. That is not enough for a GSoC project, and it's not the
> most exciting thing to work on either. But if you want to put a full
> project together out of smaller, Cython related, subprojects, this could
> certainly be a part of it, and I wouldn't mind mentoring that subproject.
>
>
I think the spline bits could be vectorized and rewritten in Python without
too much loss of speed.

<snip>

Chuck
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scipy-dev/attachments/20180302/c1827b9c/attachment-0001.html>

From lagru at mailbox.org  Sat Mar  3 02:38:11 2018
From: lagru at mailbox.org (Lars G.)
Date: Sat, 3 Mar 2018 08:38:11 +0100
Subject: [SciPy-Dev] GSoC 2018: Cythonizing
In-Reply-To: <CAFpSVp+xP5_eGEMci7o7bijY7_HFcK2TE5FJ4TcREg5OJZRpSQ@mail.gmail.com>
References: <3648ed21-1b43-975c-4ca0-05926d683736@mailbox.org>
 <CAGu2niVLwZpw-NimvLb7=i36wQp9h9-tkxg72hW2bfQON6Zq6A@mail.gmail.com>
 <e404c01f-9ace-58a8-1f65-4878ce4ad916@mailbox.org>
 <CAFpSVp+xP5_eGEMci7o7bijY7_HFcK2TE5FJ4TcREg5OJZRpSQ@mail.gmail.com>
Message-ID: <7800f9b4-51b9-39cf-9ac8-9eae88e2eab9@mailbox.org>

On 01.03.2018 16:40, Todd wrote:
> The first issue listed in the roadmap, convolution, is a much more
> complicated issue than that description makes out.? There are a few
> issues, some with some overlap behind-the-scenes:
> 
> ?? 1. As discussed, there are a bunch of different implementations that
> that use different?algorithm that work better in different scenarios.?
> Ideally there would be one "master" function that would pick the best
> algorithm for a given set of parameters.? This will depend on the number
> of dimensions to be convolved over, the size of the the first signal to
> be convolved, and the size of the second signal to be convolved.?
> Changing any one of these can change which implementation is optimal, or
> even useful.? So for with vectors, it is better to use a different
> algorithm if the one vector is short, if both vectors are long but one
> is much longer, and if both vectors are long and of similar length.
> ?? 2. We don't have the best algorithms implemented for all? of these
> scenarios.? For example the "both vectors are long but one is much
> longer" scenario is best with the overlap-add algorithm, which scipy
> doesn't have.? Similarly, there is an fft-based version of correlation
> equivalent to fftconvolve that isn't implemented, 2D and n-d versions of
> fft convolution and correlation that aren't implemented, etc.
> ?? 3. The implementations only work over the number of dimensions they
> apply to.? So the 1D implementations can only take vectors, the 2D
> implementations can only take 2D arrays, etc.? There is no way to, say,
> apply a filter along the second dimension of a 3D signal.? In order to
> implement the "master" function, at least one implementation (and
> ideally all implementations) should be able to be applied across
> additional dimensions.
> 
> And there is overlap between these.? For example I mention the
> overlap-add method in point 2, but that would most likely be implemented
> in part by applying across dimensions as mentioned in point 3.
> 
> A lot of these issues apply elsewhere in scipy.signal.? For example the
> stft/spectrogram uses a slow, naive implementation.? A lot of the
> functions don't support applying across multidimensional arrays (for
> example to create a filter bank).?

So you're saying this could be a possible GSoC project? Because this
does sound the most interesting to me so far.

To make sure I understand this correctly:

- I would work with the two modules `signal` and `ndimage` as well as
  NumPy (`numpy.convolve`)?
- I would unify, redesign and extend the parts / API that deal with
  convolution with the goal to cover the most common use cases and
  minimize overlap.
- Is somebody willing to mentor this?
- Required knowledge would involve understanding different algorithms to
  implement convolution as well as optimization, Python, Cython, C, ...?
- How would you judge the size and difficulty of this task?

Thank you all for the feedback so far. :)

Best regards,
Lars


From anubhavp28 at gmail.com  Sat Mar  3 06:26:28 2018
From: anubhavp28 at gmail.com (Anubhav Patel)
Date: Sat, 3 Mar 2018 16:56:28 +0530
Subject: [SciPy-Dev] Contributing to SciPy through GSoC
In-Reply-To: <CABL7CQj2D9wAGqYA7Ho51Tin8+AuBNLzRD99Wmsjnd6s-8DeEg@mail.gmail.com>
References: <CADtkFLHAp88jMDGwyWBQ612x58BTrQfw1H1gu-E4Ur9MfV3ksg@mail.gmail.com>
 <CABL7CQj2D9wAGqYA7Ho51Tin8+AuBNLzRD99Wmsjnd6s-8DeEg@mail.gmail.com>
Message-ID: <CADtkFLFgAqM9tGYt9CVDGde-fon+z64hpP6Rgo2hZQtrS56jng@mail.gmail.com>

Hi,
I wanted feedback regarding whether a combination of rotation class and
implementation of quaternion SLERP algorithm and Davenport's Q-method
solving Wahba's Problem, will be enough for GSoC? Should I include more
rotation related algorithm for implementation? Any suggestions what more I
could do?

On Mon, Feb 26, 2018 at 9:30 PM, Ralf Gommers <ralf.gommers at gmail.com>
wrote:

> Hi Anubhev,
>
> On Mon, Feb 26, 2018 at 2:12 AM, Anubhav Patel <anubhavp28 at gmail.com>
> wrote:
>
>> Hi everyone,
>> I want to work on SciPy as part of GSoC and I have few queries.
>>
>> 1. On the Ideas Page, there was a mention of scipy.spatial.transform
>> module. I want to know what will be the exact purpose of this module?
>>
>
> Did you read the whole idea? There's a lot of detail. It says for example
> "The aim of this project is to create a module which will allow to
> conveniently describe, apply and compose rotations. ". That answer your
> question I think.
>
>
>>
>> 2. Whether the idea for a module for numerical differentiation was
>> dropped completely?
>>
>
> Yes, for now that's off the table - at least not feasible for a GSoC we've
> concluded after several attempts.
>
>
>>
>> 3. Apart from those ideas listed on ideas page, are there any other area
>> where you guys would like to see contribution on?
>>
>
> Ideas for new features on http://scipy.github.io/devdocs/roadmap.html are
> of interest, or ones you may have yourself. But given that they're not on
> the ideas page, it's not guaranteed we can find mentors for those.
>
> Cheers,
> Ralf
>
>
>
> _______________________________________________
> SciPy-Dev mailing list
> SciPy-Dev at python.org
> https://mail.python.org/mailman/listinfo/scipy-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scipy-dev/attachments/20180303/3554c9de/attachment.html>

From charlesr.harris at gmail.com  Sat Mar  3 10:05:14 2018
From: charlesr.harris at gmail.com (Charles R Harris)
Date: Sat, 3 Mar 2018 08:05:14 -0700
Subject: [SciPy-Dev] GSoC 2018: Cythonizing
In-Reply-To: <7800f9b4-51b9-39cf-9ac8-9eae88e2eab9@mailbox.org>
References: <3648ed21-1b43-975c-4ca0-05926d683736@mailbox.org>
 <CAGu2niVLwZpw-NimvLb7=i36wQp9h9-tkxg72hW2bfQON6Zq6A@mail.gmail.com>
 <e404c01f-9ace-58a8-1f65-4878ce4ad916@mailbox.org>
 <CAFpSVp+xP5_eGEMci7o7bijY7_HFcK2TE5FJ4TcREg5OJZRpSQ@mail.gmail.com>
 <7800f9b4-51b9-39cf-9ac8-9eae88e2eab9@mailbox.org>
Message-ID: <CAB6mnxLLouWCToC38+vSpVVe5vcC17sCLZsHOJN-if-j7daWZw@mail.gmail.com>

On Sat, Mar 3, 2018 at 12:38 AM, Lars G. <lagru at mailbox.org> wrote:

> On 01.03.2018 16:40, Todd wrote:
> > The first issue listed in the roadmap, convolution, is a much more
> > complicated issue than that description makes out.  There are a few
> > issues, some with some overlap behind-the-scenes:
> >
> >    1. As discussed, there are a bunch of different implementations that
> > that use different algorithm that work better in different scenarios.
> > Ideally there would be one "master" function that would pick the best
> > algorithm for a given set of parameters.  This will depend on the number
> > of dimensions to be convolved over, the size of the the first signal to
> > be convolved, and the size of the second signal to be convolved.
> > Changing any one of these can change which implementation is optimal, or
> > even useful.  So for with vectors, it is better to use a different
> > algorithm if the one vector is short, if both vectors are long but one
> > is much longer, and if both vectors are long and of similar length.
> >    2. We don't have the best algorithms implemented for all  of these
> > scenarios.  For example the "both vectors are long but one is much
> > longer" scenario is best with the overlap-add algorithm, which scipy
> > doesn't have.  Similarly, there is an fft-based version of correlation
> > equivalent to fftconvolve that isn't implemented, 2D and n-d versions of
> > fft convolution and correlation that aren't implemented, etc.
> >    3. The implementations only work over the number of dimensions they
> > apply to.  So the 1D implementations can only take vectors, the 2D
> > implementations can only take 2D arrays, etc.  There is no way to, say,
> > apply a filter along the second dimension of a 3D signal.  In order to
> > implement the "master" function, at least one implementation (and
> > ideally all implementations) should be able to be applied across
> > additional dimensions.
> >
> > And there is overlap between these.  For example I mention the
> > overlap-add method in point 2, but that would most likely be implemented
> > in part by applying across dimensions as mentioned in point 3.
> >
> > A lot of these issues apply elsewhere in scipy.signal.  For example the
> > stft/spectrogram uses a slow, naive implementation.  A lot of the
> > functions don't support applying across multidimensional arrays (for
> > example to create a filter bank).
>
> So you're saying this could be a possible GSoC project? Because this
> does sound the most interesting to me so far.
>
> To make sure I understand this correctly:
>
> - I would work with the two modules `signal` and `ndimage` as well as
>   NumPy (`numpy.convolve`)?
> - I would unify, redesign and extend the parts / API that deal with
>   convolution with the goal to cover the most common use cases and
>   minimize overlap.
> - Is somebody willing to mentor this?
> - Required knowledge would involve understanding different algorithms to
>   implement convolution as well as optimization, Python, Cython, C, ...?
> - How would you judge the size and difficulty of this task?
>

It would be a difficult project for GSOC, as a lot would depend on
identifying and designing the underlying common algorithms, including APIs
That makes it a step beyond just implementing something known in advance,
it requires strong background knowledge and familiarity with the relevant
SciPy modules.  I would be hesitant to propose it unless it could be
trimmed down to just one or two functions that are well defined before the
project starts. Just as an example of how the complexity grows, NumPy
convolution assumes finite sequences extended to +/- inf with zeros,
whereas convolution used for interpolation and filtering will have a number
of choices for edge conditions, some of which would be best handled with
one of the discrete cos transforms. I don't think anyone has sat down and
figured out how to organize all that, much less proposed a roadmap to
implement it.

Chuck
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scipy-dev/attachments/20180303/ddd8d022/attachment-0001.html>

From 3ukip0s02 at sneakemail.com  Sat Mar  3 12:47:57 2018
From: 3ukip0s02 at sneakemail.com (3ukip0s02 at sneakemail.com)
Date: Sat, 3 Mar 2018 12:47:57 -0500
Subject: [SciPy-Dev] SciPy-Dev Digest, Vol 173, Issue 3
In-Reply-To: <mailman.2066.1519920957.2834.scipy-dev@python.org>
References: <mailman.2066.1519920957.2834.scipy-dev@python.org>
Message-ID: <26548-1520099298-414934@sneakemail.com>

On Thu Mar 1 10:40:16 EST 2018, Todd wrote:

> Similarly, there is an fft-based version of correlation equivalent to
> fftconvolve that isn't implemented,
>


> 2D and n-d versions of fft convolution
> and correlation that aren't implemented, etc.
>

correlate and convolve are both N-D, and implemented using fftpack when
possible, which is compiled fortran, so I'm not sure those would benefit
much.

convolve2d and correlate2d could benefit from FFT.  Only reason they don't
is because of the boundary conditions, but I think they could be adapted to
FFT if the inputs were extended in the relevant ways first?

A lot of these issues apply elsewhere in scipy.signal.  For example the
> stft/spectrogram uses a slow, naive implementation.
>

All of the _spectral_helper functions like stft() and spectrogram() are
fftpack-based, as well.  Is there a lot of room for improvement here?
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scipy-dev/attachments/20180303/c5707ac7/attachment.html>

From mikegraham at gmail.com  Sat Mar  3 18:29:50 2018
From: mikegraham at gmail.com (Mike Graham)
Date: Sat, 3 Mar 2018 18:29:50 -0500
Subject: [SciPy-Dev] SciPy IRC channel lacks moderation
In-Reply-To: <CABL7CQj62BZdnZ=i_+2L9g0pQyD7z+wWzhFy-MOQyYJPMe2F9g@mail.gmail.com>
References: <CAEBZo3MdZU5GVQOPJrso1DKEH0fo-BPsq2zSs7P+JNVtLfmWiA@mail.gmail.com>
 <CABL7CQj-5tcJfzgx3s6bjzsG+DP2KTY1dnmL3jULFheBako3mw@mail.gmail.com>
 <CAEBZo3MOMyaTp1bmUugbb1axadjQ0RvUZSZP7ytP=6ihhNcn9g@mail.gmail.com>
 <CABL7CQj62BZdnZ=i_+2L9g0pQyD7z+wWzhFy-MOQyYJPMe2F9g@mail.gmail.com>
Message-ID: <CAEBZo3OTPZqkVcsGEokLrLbnYHOOBad4S7zFhdQ6rSHoKaJ6Mw@mail.gmail.com>

On Wed, Feb 28, 2018 at 11:49 PM, Ralf Gommers <ralf.gommers at gmail.com>
wrote:
>
>
> On Mon, Feb 26, 2018 at 9:53 AM, Mike Graham <mikegraham at gmail.com> wrote:
>
>> On Fri, Feb 23, 2018 at 1:25 AM, Ralf Gommers <ralf.gommers at gmail.com>
>> wrote:
>>
>>>
>>> I'm fine with either option, discontinuing or moderation. Either way we
>>> should make it clear that scipy devs don't hang out there.
>>>
>>
>> For what it's worth, people still get help with numpy, scipy, and other
>> scientific programming issues in the channel every day. Nathan remarked
>> that closing the channel was "a really dumb idea". ;)
>>
>> If you want to appoint Nathan (ngoldbaum on Freenode) and/or me (papna on
>> freenode) as group contact to have moderator tools, the freenode people
>> will probably do that if we can point them to this mailing list thread.
>> They will just want to see that it is what the project wants.
>>
>
> Okay, that seems fine - if Nathan and you want to do that, and it's
> helpful for a part of the community, then that seems like a good idea.
> Thanks for stepping up. Could you please request access to those moderator
> tools for Nathan and yourself, and point them to this thread?
>

Many thanks!

Best,
Mike
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scipy-dev/attachments/20180303/688ee8d5/attachment.html>

From hritiknarayan at gmail.com  Sun Mar  4 03:38:02 2018
From: hritiknarayan at gmail.com (Hritik Narayan)
Date: Sun, 4 Mar 2018 14:08:02 +0530
Subject: [SciPy-Dev] GSoC
Message-ID: <CALO3sJ4yxqA8jfT2-K_rVHUHLW7h7NboSbg+Pox0-Qsznt4bSA@mail.gmail.com>

Hey everyone, I want to contribute to SciPy via GSoC, want to get into
touch with possible mentors for clarification. Mainly, are the final
proposals to be mailed directly to the mentor, or are they to be posted on
a mailing list?

-- 
Hritik, UTC +5:30
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scipy-dev/attachments/20180304/0931ee9c/attachment.html>

From tyler.je.reddy at gmail.com  Sun Mar  4 13:39:49 2018
From: tyler.je.reddy at gmail.com (Tyler Reddy)
Date: Sun, 4 Mar 2018 11:39:49 -0700
Subject: [SciPy-Dev] Contributing to SciPy through GSoC
In-Reply-To: <CADtkFLFgAqM9tGYt9CVDGde-fon+z64hpP6Rgo2hZQtrS56jng@mail.gmail.com>
References: <CADtkFLHAp88jMDGwyWBQ612x58BTrQfw1H1gu-E4Ur9MfV3ksg@mail.gmail.com>
 <CABL7CQj2D9wAGqYA7Ho51Tin8+AuBNLzRD99Wmsjnd6s-8DeEg@mail.gmail.com>
 <CADtkFLFgAqM9tGYt9CVDGde-fon+z64hpP6Rgo2hZQtrS56jng@mail.gmail.com>
Message-ID: <CAHPuU_YX+DUVqzvx5PMNTm8dy+dKpTh-Hthr7r4BaTTPma3qJg@mail.gmail.com>

It is perhaps worth noting that I have written a low-level (Cython) Slerp
function in https://github.com/scipy/scipy/pull/8069. There was no real
intention to make that a standalone user-facing function, but if things do
move forward with rotation-related development, worth keeping in mind that
we have some pretty well-working source code for that particular routine.
Even if the referenced PR doesn't get merged some day, could always
cannibalize _slerp from there as a starting point.

On 3 March 2018 at 04:26, Anubhav Patel <anubhavp28 at gmail.com> wrote:

> Hi,
> I wanted feedback regarding whether a combination of rotation class and
> implementation of quaternion SLERP algorithm and Davenport's Q-method
> solving Wahba's Problem, will be enough for GSoC? Should I include more
> rotation related algorithm for implementation? Any suggestions what more I
> could do?
>
> On Mon, Feb 26, 2018 at 9:30 PM, Ralf Gommers <ralf.gommers at gmail.com>
> wrote:
>
>> Hi Anubhev,
>>
>> On Mon, Feb 26, 2018 at 2:12 AM, Anubhav Patel <anubhavp28 at gmail.com>
>> wrote:
>>
>>> Hi everyone,
>>> I want to work on SciPy as part of GSoC and I have few queries.
>>>
>>> 1. On the Ideas Page, there was a mention of scipy.spatial.transform
>>> module. I want to know what will be the exact purpose of this module?
>>>
>>
>> Did you read the whole idea? There's a lot of detail. It says for example
>> "The aim of this project is to create a module which will allow to
>> conveniently describe, apply and compose rotations. ". That answer your
>> question I think.
>>
>>
>>>
>>> 2. Whether the idea for a module for numerical differentiation was
>>> dropped completely?
>>>
>>
>> Yes, for now that's off the table - at least not feasible for a GSoC
>> we've concluded after several attempts.
>>
>>
>>>
>>> 3. Apart from those ideas listed on ideas page, are there any other area
>>> where you guys would like to see contribution on?
>>>
>>
>> Ideas for new features on http://scipy.github.io/devdocs/roadmap.html
>> are of interest, or ones you may have yourself. But given that they're not
>> on the ideas page, it's not guaranteed we can find mentors for those.
>>
>> Cheers,
>> Ralf
>>
>>
>>
>> _______________________________________________
>> SciPy-Dev mailing list
>> SciPy-Dev at python.org
>> https://mail.python.org/mailman/listinfo/scipy-dev
>>
>>
>
> _______________________________________________
> SciPy-Dev mailing list
> SciPy-Dev at python.org
> https://mail.python.org/mailman/listinfo/scipy-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scipy-dev/attachments/20180304/e9097bbd/attachment.html>

From ralf.gommers at gmail.com  Sun Mar  4 13:47:59 2018
From: ralf.gommers at gmail.com (Ralf Gommers)
Date: Sun, 4 Mar 2018 10:47:59 -0800
Subject: [SciPy-Dev] GSoC
In-Reply-To: <CALO3sJ4yxqA8jfT2-K_rVHUHLW7h7NboSbg+Pox0-Qsznt4bSA@mail.gmail.com>
References: <CALO3sJ4yxqA8jfT2-K_rVHUHLW7h7NboSbg+Pox0-Qsznt4bSA@mail.gmail.com>
Message-ID: <CABL7CQiwD2fY4d55SkDsHWEXE5G1+CEO-1DJ2jXrnrU_cET7QA@mail.gmail.com>

On Sun, Mar 4, 2018 at 12:38 AM, Hritik Narayan <hritiknarayan at gmail.com>
wrote:

> Hey everyone, I want to contribute to SciPy via GSoC, want to get into
> touch with possible mentors for clarification. Mainly, are the final
> proposals to be mailed directly to the mentor, or are they to be posted on
> a mailing list?
>

Hi Hritik, the proposals are to be sent to this mailing list. It's best to
post a draft early, so you still have time to incorporate feedback from us.
All mentors read this list.

Cheers,
Ralf
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scipy-dev/attachments/20180304/3b4573c3/attachment.html>

From nikolay.mayorov at zoho.com  Sun Mar  4 16:41:58 2018
From: nikolay.mayorov at zoho.com (Nikolay Mayorov)
Date: Mon, 05 Mar 2018 02:41:58 +0500
Subject: [SciPy-Dev] Contributing to SciPy through GSoC
Message-ID: <161f2f6d6b6.fb3ea05a10736.6245209566181213756@zoho.com>

Hi, Anybhev!



I think getting Rotation right is a top priority and so far unfortunately nobody dig into technical details of it. I would like to see that from students.



As for the algorithms. I believe that Wahba's problem can be generalized by adding a translation vector, but the interpretation will be different (search for "absolute orientation problem"). As for methods to solve Wahba's problem --- probably SVD based is the easiest to understand and implement, but if we decide to use quaternions as the base representation, then we can go with "Q-method".



"Cubic spline" for orientation is also a very cool algorithm which wasn't promoted anywhere, but this is the best idea I found on the subject (i.e. interpolation with continuous angular rates and acceleration). 



Other algorithms are quite small, they can be added quickly. I would say it is more of a question of what we should include. If you have some ideas outside (mostly) "aerospace" field --- they are welcome.



For all parts I would like to see more concrete and technical details. For example, if we want SLERP interpolation --- what will it be (class or function), what it will accept, what will be the most difficult part to implement it correctly. The same for all other things.



Best,

Nikolay




---- On Sat, 03 Mar 2018 16:26:28 +0500 anubhavp28 at gmail.com  wrote ----



Hi,

I wanted feedback regarding whether a combination of rotation class and implementation of quaternion SLERP algorithm and Davenport's Q-method solving Wahba's Problem, will be enough for GSoC? Should I include more rotation related algorithm for implementation? Any suggestions what more I could do?




On Mon, Feb 26, 2018 at 9:30 PM, Ralf Gommers &lt;ralf.gommers at gmail.com&gt; wrote:

Hi Anubhev,



On Mon, Feb 26, 2018 at 2:12 AM, Anubhav Patel &lt;anubhavp28 at gmail.com&gt; wrote:

Hi everyone,

I want to work on SciPy as part of GSoC and I have few queries.



1. On the Ideas Page, there was a mention of scipy.spatial.transform module. I want to know what will be the exact purpose of this module?




Did you read the whole idea? There's a lot of detail. It says for example "The aim of this project is to create a module which will allow to conveniently describe, apply and compose rotations. ". That answer your question I think.

 




2. Whether the idea for a module for numerical differentiation was dropped completely?




Yes, for now that's off the table - at least not feasible for a GSoC we've concluded after several attempts.

 




3. Apart from those ideas listed on ideas page, are there any other area where you guys would like to see contribution on?




Ideas for new features on http://scipy.github.io/devdocs/roadmap.html are of interest, or ones you may have yourself. But given that they're not on the ideas page, it's not guaranteed we can find mentors for those.


Cheers,

Ralf










_______________________________________________

 SciPy-Dev mailing list

 SciPy-Dev at python.org

 https://mail.python.org/mailman/listinfo/scipy-dev

 





_______________________________________________

SciPy-Dev mailing list 

SciPy-Dev at python.org 

https://mail.python.org/mailman/listinfo/scipy-dev 






-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scipy-dev/attachments/20180305/70b79910/attachment-0001.html>

From andyfaff at gmail.com  Sun Mar  4 22:05:36 2018
From: andyfaff at gmail.com (Andrew Nelson)
Date: Mon, 5 Mar 2018 14:05:36 +1100
Subject: [SciPy-Dev] WIP: Class based Optimizers
In-Reply-To: <CAHCaCkLHRWiS8ViNDeMFPTz1YWNSyqoX5x3exVxuG8jLsyXCww@mail.gmail.com>
References: <CAAbtOZee-V5+g=pHN+bp6YsA_Z-fa5UUSc5Zd5izM6jYS_A-aQ@mail.gmail.com>
 <1518642682.8666.18.camel@iki.fi> <5a84a711.05cfca0a.4a242.4608@mx.google.com>
 <CAPZ=bb2EeZXT=c5utHeKkfU2KH4BVEU6YDOYh4RWELDaseJUzA@mail.gmail.com>
 <CAHCaCkLHRWiS8ViNDeMFPTz1YWNSyqoX5x3exVxuG8jLsyXCww@mail.gmail.com>
Message-ID: <CAAbtOZd-ueDuaGO7LsgivoS3TM-9MpRAe9iaY_84Hq+rTEK2pg@mail.gmail.com>

Scott Sievert and I have put a lot of work into preparing a draft of the
PEP for class based scalar minimizers:

https://github.com/andyfaff/scipy/blob/a52bb4f9029389da3ab072c92c609d71ed6943c6/PEP/1-Optimizer.rst

where we've tried to address comments already made in this thread, and from
the WIP github PR. Scott and I look forward to hearing any
comments/concerns/feedback about the proposal. We can field any questions
and address them in an updated PEP, as well as on here.

Andrew.


p.s. Ralf/Pauli, could we add the PEP to a scipy/PEP, or scipy/scipep,
repo? How should we discuss such this, and any further PEP? Should we have
a scipep process, or shall we keep things simple?
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scipy-dev/attachments/20180305/84cf413c/attachment.html>

From phillip.m.feldman at gmail.com  Mon Mar  5 02:03:34 2018
From: phillip.m.feldman at gmail.com (Phillip Feldman)
Date: Sun, 4 Mar 2018 23:03:34 -0800
Subject: [SciPy-Dev] WIP: Class based Optimizers
In-Reply-To: <CAAbtOZd-ueDuaGO7LsgivoS3TM-9MpRAe9iaY_84Hq+rTEK2pg@mail.gmail.com>
References: <CAAbtOZee-V5+g=pHN+bp6YsA_Z-fa5UUSc5Zd5izM6jYS_A-aQ@mail.gmail.com>
 <1518642682.8666.18.camel@iki.fi> <5a84a711.05cfca0a.4a242.4608@mx.google.com>
 <CAPZ=bb2EeZXT=c5utHeKkfU2KH4BVEU6YDOYh4RWELDaseJUzA@mail.gmail.com>
 <CAHCaCkLHRWiS8ViNDeMFPTz1YWNSyqoX5x3exVxuG8jLsyXCww@mail.gmail.com>
 <CAAbtOZd-ueDuaGO7LsgivoS3TM-9MpRAe9iaY_84Hq+rTEK2pg@mail.gmail.com>
Message-ID: <CAB2ViTZZog8LbYGrwxTE29Tz0oXq_sDtOtiNKip_tXz5=4TD4A@mail.gmail.com>

>From the (beautifully written) draft PEP:

"Different optimization algorithms can inherit from Optimizer, with each of
the subclasses overriding the __next__ method ..."

I'm unclear re. whether this approach would allow something like a parallel
implementation of Nelder-Mead.

Phillip

On Sun, Mar 4, 2018 at 7:05 PM, Andrew Nelson <andyfaff at gmail.com> wrote:

> Scott Sievert and I have put a lot of work into preparing a draft of the
> PEP for class based scalar minimizers:
>
> https://github.com/andyfaff/scipy/blob/a52bb4f9029389da3ab072c92c609d
> 71ed6943c6/PEP/1-Optimizer.rst
>
> where we've tried to address comments already made in this thread, and
> from the WIP github PR. Scott and I look forward to hearing any
> comments/concerns/feedback about the proposal. We can field any questions
> and address them in an updated PEP, as well as on here.
>
> Andrew.
>
>
> p.s. Ralf/Pauli, could we add the PEP to a scipy/PEP, or scipy/scipep,
> repo? How should we discuss such this, and any further PEP? Should we have
> a scipep process, or shall we keep things simple?
>
> _______________________________________________
> SciPy-Dev mailing list
> SciPy-Dev at python.org
> https://mail.python.org/mailman/listinfo/scipy-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scipy-dev/attachments/20180304/b92e0bd8/attachment.html>

From andyfaff at gmail.com  Mon Mar  5 02:13:28 2018
From: andyfaff at gmail.com (Andrew Nelson)
Date: Mon, 5 Mar 2018 18:13:28 +1100
Subject: [SciPy-Dev] WIP: Class based Optimizers
In-Reply-To: <CAB2ViTZZog8LbYGrwxTE29Tz0oXq_sDtOtiNKip_tXz5=4TD4A@mail.gmail.com>
References: <CAAbtOZee-V5+g=pHN+bp6YsA_Z-fa5UUSc5Zd5izM6jYS_A-aQ@mail.gmail.com>
 <1518642682.8666.18.camel@iki.fi> <5a84a711.05cfca0a.4a242.4608@mx.google.com>
 <CAPZ=bb2EeZXT=c5utHeKkfU2KH4BVEU6YDOYh4RWELDaseJUzA@mail.gmail.com>
 <CAHCaCkLHRWiS8ViNDeMFPTz1YWNSyqoX5x3exVxuG8jLsyXCww@mail.gmail.com>
 <CAAbtOZd-ueDuaGO7LsgivoS3TM-9MpRAe9iaY_84Hq+rTEK2pg@mail.gmail.com>
 <CAB2ViTZZog8LbYGrwxTE29Tz0oXq_sDtOtiNKip_tXz5=4TD4A@mail.gmail.com>
Message-ID: <CAAbtOZdGPHmuE79W9GUkFkc3wWbsPQWGorreN82YSFzYQBktxg@mail.gmail.com>

> would allow something like a parallel implementation of Nelder-Mead.

At the moment the __next__ method would consist of the logic of the
existing loop inside _minimize_neldermead, which is done serially. I was
not aware of a parallel version of NM, but a quick search reveals there is
something along those lines. That's not in scope here, but could be added
later.

On 5 March 2018 at 18:03, Phillip Feldman <phillip.m.feldman at gmail.com>
wrote:

> From the (beautifully written) draft PEP:
>
> "Different optimization algorithms can inherit from Optimizer, with each
> of the subclasses overriding the __next__ method ..."
>
> I'm unclear re. whether this approach would allow something like a
> parallel implementation of Nelder-Mead.
>
> Phillip
>
> On Sun, Mar 4, 2018 at 7:05 PM, Andrew Nelson <andyfaff at gmail.com> wrote:
>
>> Scott Sievert and I have put a lot of work into preparing a draft of the
>> PEP for class based scalar minimizers:
>>
>> https://github.com/andyfaff/scipy/blob/a52bb4f9029389da3ab07
>> 2c92c609d71ed6943c6/PEP/1-Optimizer.rst
>>
>> where we've tried to address comments already made in this thread, and
>> from the WIP github PR. Scott and I look forward to hearing any
>> comments/concerns/feedback about the proposal. We can field any questions
>> and address them in an updated PEP, as well as on here.
>>
>> Andrew.
>>
>>
>> p.s. Ralf/Pauli, could we add the PEP to a scipy/PEP, or scipy/scipep,
>> repo? How should we discuss such this, and any further PEP? Should we have
>> a scipep process, or shall we keep things simple?
>>
>> _______________________________________________
>> SciPy-Dev mailing list
>> SciPy-Dev at python.org
>> https://mail.python.org/mailman/listinfo/scipy-dev
>>
>>
>
> _______________________________________________
> SciPy-Dev mailing list
> SciPy-Dev at python.org
> https://mail.python.org/mailman/listinfo/scipy-dev
>
>


-- 
_____________________________________
Dr. Andrew Nelson


_____________________________________
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scipy-dev/attachments/20180305/8ac96d93/attachment-0001.html>

From andyfaff at gmail.com  Mon Mar  5 02:31:43 2018
From: andyfaff at gmail.com (Andrew Nelson)
Date: Mon, 5 Mar 2018 18:31:43 +1100
Subject: [SciPy-Dev] WIP: Class based Optimizers
In-Reply-To: <CAAbtOZdGPHmuE79W9GUkFkc3wWbsPQWGorreN82YSFzYQBktxg@mail.gmail.com>
References: <CAAbtOZee-V5+g=pHN+bp6YsA_Z-fa5UUSc5Zd5izM6jYS_A-aQ@mail.gmail.com>
 <1518642682.8666.18.camel@iki.fi> <5a84a711.05cfca0a.4a242.4608@mx.google.com>
 <CAPZ=bb2EeZXT=c5utHeKkfU2KH4BVEU6YDOYh4RWELDaseJUzA@mail.gmail.com>
 <CAHCaCkLHRWiS8ViNDeMFPTz1YWNSyqoX5x3exVxuG8jLsyXCww@mail.gmail.com>
 <CAAbtOZd-ueDuaGO7LsgivoS3TM-9MpRAe9iaY_84Hq+rTEK2pg@mail.gmail.com>
 <CAB2ViTZZog8LbYGrwxTE29Tz0oXq_sDtOtiNKip_tXz5=4TD4A@mail.gmail.com>
 <CAAbtOZdGPHmuE79W9GUkFkc3wWbsPQWGorreN82YSFzYQBktxg@mail.gmail.com>
Message-ID: <CAAbtOZcGC-rj=DSvD2GP2ntVrCHA8GFThc5+RekgkibXMrN1Cw@mail.gmail.com>

For future reference parallel Nelder Mead is described
at 10.1007/s10614-007-9094-2, and some performance at
https://scwu.io/f/Parallelization_Nelder_Mead_Simplex_Algorithm_Abstract.pdf
.

On 5 March 2018 at 18:13, Andrew Nelson <andyfaff at gmail.com> wrote:

> > would allow something like a parallel implementation of Nelder-Mead.
>
> At the moment the __next__ method would consist of the logic of the
> existing loop inside _minimize_neldermead, which is done serially. I was
> not aware of a parallel version of NM, but a quick search reveals there is
> something along those lines. That's not in scope here, but could be added
> later.
>
> On 5 March 2018 at 18:03, Phillip Feldman <phillip.m.feldman at gmail.com>
> wrote:
>
>> From the (beautifully written) draft PEP:
>>
>> "Different optimization algorithms can inherit from Optimizer, with each
>> of the subclasses overriding the __next__ method ..."
>>
>> I'm unclear re. whether this approach would allow something like a
>> parallel implementation of Nelder-Mead.
>>
>> Phillip
>>
>> On Sun, Mar 4, 2018 at 7:05 PM, Andrew Nelson <andyfaff at gmail.com> wrote:
>>
>>> Scott Sievert and I have put a lot of work into preparing a draft of the
>>> PEP for class based scalar minimizers:
>>>
>>> https://github.com/andyfaff/scipy/blob/a52bb4f9029389da3ab07
>>> 2c92c609d71ed6943c6/PEP/1-Optimizer.rst
>>>
>>> where we've tried to address comments already made in this thread, and
>>> from the WIP github PR. Scott and I look forward to hearing any
>>> comments/concerns/feedback about the proposal. We can field any questions
>>> and address them in an updated PEP, as well as on here.
>>>
>>> Andrew.
>>>
>>>
>>> p.s. Ralf/Pauli, could we add the PEP to a scipy/PEP, or scipy/scipep,
>>> repo? How should we discuss such this, and any further PEP? Should we have
>>> a scipep process, or shall we keep things simple?
>>>
>>> _______________________________________________
>>> SciPy-Dev mailing list
>>> SciPy-Dev at python.org
>>> https://mail.python.org/mailman/listinfo/scipy-dev
>>>
>>>
>>
>> _______________________________________________
>> SciPy-Dev mailing list
>> SciPy-Dev at python.org
>> https://mail.python.org/mailman/listinfo/scipy-dev
>>
>>
>
>
> --
> _____________________________________
> Dr. Andrew Nelson
>
>
> _____________________________________
>



-- 
_____________________________________
Dr. Andrew Nelson


_____________________________________
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scipy-dev/attachments/20180305/9fc02e8b/attachment.html>

From ralf.gommers at gmail.com  Mon Mar  5 02:41:36 2018
From: ralf.gommers at gmail.com (Ralf Gommers)
Date: Sun, 4 Mar 2018 23:41:36 -0800
Subject: [SciPy-Dev] WIP: Class based Optimizers
In-Reply-To: <CAAbtOZd-ueDuaGO7LsgivoS3TM-9MpRAe9iaY_84Hq+rTEK2pg@mail.gmail.com>
References: <CAAbtOZee-V5+g=pHN+bp6YsA_Z-fa5UUSc5Zd5izM6jYS_A-aQ@mail.gmail.com>
 <1518642682.8666.18.camel@iki.fi> <5a84a711.05cfca0a.4a242.4608@mx.google.com>
 <CAPZ=bb2EeZXT=c5utHeKkfU2KH4BVEU6YDOYh4RWELDaseJUzA@mail.gmail.com>
 <CAHCaCkLHRWiS8ViNDeMFPTz1YWNSyqoX5x3exVxuG8jLsyXCww@mail.gmail.com>
 <CAAbtOZd-ueDuaGO7LsgivoS3TM-9MpRAe9iaY_84Hq+rTEK2pg@mail.gmail.com>
Message-ID: <CABL7CQgSNjityHbmUoPagGUinEnkxMsDsjw6sf=zxUpqCaBv0Q@mail.gmail.com>

On Sun, Mar 4, 2018 at 7:05 PM, Andrew Nelson <andyfaff at gmail.com> wrote:

> Scott Sievert and I have put a lot of work into preparing a draft of the
> PEP for class based scalar minimizers:
>
> https://github.com/andyfaff/scipy/blob/a52bb4f9029389da3ab072c92c609d
> 71ed6943c6/PEP/1-Optimizer.rst
>
> where we've tried to address comments already made in this thread, and
> from the WIP github PR. Scott and I look forward to hearing any
> comments/concerns/feedback about the proposal. We can field any questions
> and address them in an updated PEP, as well as on here.
>
> Andrew.
>
>
> p.s. Ralf/Pauli, could we add the PEP to a scipy/PEP, or scipy/scipep,
> repo? How should we discuss such this, and any further PEP? Should we have
> a scipep process, or shall we keep things simple?
>

I'd suggest a separate repo, and no custom process but rather just follow
what is done for Python Enhancement Proposals - discussion of major things
on this list, more detailed things on a PR on that new repo.

Unless there's other opinions, I can create the repo. There's also
https://github.com/numpy/neps for which build infrastructure is in progress
(I expect/hope), so we should be able to steal that soon. So for now just
open a PR on the new empty repo I'd say.

Your proposal looks quite comprehensive and well written. I would suggest
following the structure of PEPs a little more closely (
https://www.python.org/dev/peps/pep-0001/#what-belongs-in-a-successful-pep),
e.g. add the metadata and copyright bits, and use "motivation" and
"rationale" as section headers for some of the content that you have.

Ralf
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scipy-dev/attachments/20180304/e7a10f59/attachment.html>

From lagru at mailbox.org  Mon Mar  5 05:26:29 2018
From: lagru at mailbox.org (Lars G.)
Date: Mon, 5 Mar 2018 11:26:29 +0100
Subject: [SciPy-Dev] GSoC 2018: Cythonizing
In-Reply-To: <CAB6mnxLLouWCToC38+vSpVVe5vcC17sCLZsHOJN-if-j7daWZw@mail.gmail.com>
References: <3648ed21-1b43-975c-4ca0-05926d683736@mailbox.org>
 <CAGu2niVLwZpw-NimvLb7=i36wQp9h9-tkxg72hW2bfQON6Zq6A@mail.gmail.com>
 <e404c01f-9ace-58a8-1f65-4878ce4ad916@mailbox.org>
 <CAFpSVp+xP5_eGEMci7o7bijY7_HFcK2TE5FJ4TcREg5OJZRpSQ@mail.gmail.com>
 <7800f9b4-51b9-39cf-9ac8-9eae88e2eab9@mailbox.org>
 <CAB6mnxLLouWCToC38+vSpVVe5vcC17sCLZsHOJN-if-j7daWZw@mail.gmail.com>
Message-ID: <f2ab9c97-de60-7d28-080e-2b00f232152c@mailbox.org>

On 03.03.2018 16:05, Charles R Harris wrote:
> It would be a difficult project for GSOC, as a lot would depend on
> identifying and designing the underlying common algorithms, including
> APIs That makes it a step beyond just implementing something known in
> advance, it requires strong background knowledge and familiarity with
> the relevant SciPy modules.? I would be hesitant to propose it unless it
> could be trimmed down to just one or two functions that are well defined
> before the project starts. Just as an example of how the complexity
> grows, NumPy convolution assumes finite sequences extended to +/- inf
> with zeros, whereas convolution used for interpolation and filtering
> will have a number of choices for edge conditions, some of which would
> be best handled with one of the discrete cos transforms. I don't think
> anyone has sat down and figured out how to organize all that, much less
> proposed a roadmap to implement it.
> 
> Chuck

Okay, thanks for the warning. That doesn't sound promising. I think I'll
try my luck with the other options. This makes me think as well, that
the idea about cythonizing would suffer from similar problems.

Best regards,
Lars

From lagru at mailbox.org  Mon Mar  5 06:02:24 2018
From: lagru at mailbox.org (Lars G.)
Date: Mon, 5 Mar 2018 12:02:24 +0100
Subject: [SciPy-Dev] GSoC 2018: Cythonizing
In-Reply-To: <3648ed21-1b43-975c-4ca0-05926d683736@mailbox.org>
References: <3648ed21-1b43-975c-4ca0-05926d683736@mailbox.org>
Message-ID: <d63a2aa2-4271-0b27-a67c-19fa8ef75ecf@mailbox.org>

On 28.02.2018 15:32, Lars G. wrote:
> Dear SciPy devs,
> 
> I'm currently thinking about an application for this year's GSoC as
> well. As there already seems to be a large interest in the rotation
> formalism I'm trying to find another area that matches my interest and
> skill.
> 
> I've dug up this proposal in scikit-image from GSoC 2015
> https://github.com/scikit-image/scikit-image/wiki/GSoC-2015#rewriting-scipyndimage-in-cython
> and judging by the state of scipy/ndimage/src/ nobody has worked on this
> proposal yet (feel free to correct me).
> Alternatively I could imagine something similar for other sub-packages,
> e.g. scipy/signal which features many source files in C as well.
> 
> So basically if there is an interest I could try to port C / Python code
> to Cython. What I would like to know:
> 
> - Is there an interest? ;)
> - Is the original proposal in scikit-image still unfinished and are the
>   potential mentors still interested in mentoring?
> - If there is a general interest to cythonize C or Python code during a
>   GSoC project, which parts / sub-packages of SciPy would you priorize?
> 
> As for my current involvement with SciPy:
> 
> - I've already added a small function written in Cython
>   https://github.com/scipy/scipy/pull/8350
> - as part of a larger PR extending the signal module
>   https://github.com/scipy/scipy/pull/8264
>   which will possibly merged this week.
> - I already cythonized slow parts of the above PR and plan
>   to add these with new PRs after #8264 is merged.
> 
> If this receives positive feedback I'd be happy to draft a more complete
> proposal / application based on the discussion around this.
> 
> Best regards,
> Lars

Actually, considering that GSoC should be treated as a full-time job
during time of coding I must sadly pass on this. However I want to thank
you all for the feedback already given. I hope its still useful for
other potential applicants.

Best regards,
Lars

From hritiknarayan at gmail.com  Mon Mar  5 11:08:55 2018
From: hritiknarayan at gmail.com (Hritik Narayan)
Date: Mon, 5 Mar 2018 21:38:55 +0530
Subject: [SciPy-Dev] GSoC
Message-ID: <CALO3sJ67UrCtqJwVvpxi_G=CR34dirbz5hC8rmsJWJqtKoeaXA@mail.gmail.com>

I'm having trouble identifying issues/problems that I could contribute to.
Could someone help out? (Apart from the ideas listed on the Github page) I
definitely want to pick SciPy as my GSoC organisation because frankly, I'd
love contributing to something that is so powerful.

-- 
Hritik
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scipy-dev/attachments/20180305/a250904d/attachment.html>

From ralf.gommers at gmail.com  Mon Mar  5 11:11:28 2018
From: ralf.gommers at gmail.com (Ralf Gommers)
Date: Mon, 5 Mar 2018 08:11:28 -0800
Subject: [SciPy-Dev] GSoC
In-Reply-To: <CALO3sJ67UrCtqJwVvpxi_G=CR34dirbz5hC8rmsJWJqtKoeaXA@mail.gmail.com>
References: <CALO3sJ67UrCtqJwVvpxi_G=CR34dirbz5hC8rmsJWJqtKoeaXA@mail.gmail.com>
Message-ID: <CABL7CQhkqvoU49yObM_nkVCj0ztKhN-p+-BZqwGdifWLnf726Q@mail.gmail.com>

Hi Hritik,

On Mon, Mar 5, 2018 at 8:08 AM, Hritik Narayan <hritiknarayan at gmail.com>
wrote:

> I'm having trouble identifying issues/problems that I could contribute to.
> Could someone help out? (Apart from the ideas listed on the Github page) I
> definitely want to pick SciPy as my GSoC organisation because frankly, I'd
> love contributing to something that is so powerful.
>

Please have a look at the issues labelled "good first issue":
https://github.com/scipy/scipy/issues?q=is%3Aopen+is%3Aissue+label%3A%22good+first+issue%22

If none of those are of interest, then it would help if you told us which
part of SciPy you're interested in exactly.

Cheers,
Ralf



> --
> Hritik
>
> _______________________________________________
> SciPy-Dev mailing list
> SciPy-Dev at python.org
> https://mail.python.org/mailman/listinfo/scipy-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scipy-dev/attachments/20180305/a4f87f6f/attachment.html>

From stefanv at berkeley.edu  Mon Mar  5 16:34:38 2018
From: stefanv at berkeley.edu (Stefan van der Walt)
Date: Mon, 5 Mar 2018 13:34:38 -0800
Subject: [SciPy-Dev] WIP: Class based Optimizers
In-Reply-To: <CABL7CQgSNjityHbmUoPagGUinEnkxMsDsjw6sf=zxUpqCaBv0Q@mail.gmail.com>
References: <CAAbtOZee-V5+g=pHN+bp6YsA_Z-fa5UUSc5Zd5izM6jYS_A-aQ@mail.gmail.com>
 <1518642682.8666.18.camel@iki.fi>
 <5a84a711.05cfca0a.4a242.4608@mx.google.com>
 <CAPZ=bb2EeZXT=c5utHeKkfU2KH4BVEU6YDOYh4RWELDaseJUzA@mail.gmail.com>
 <CAHCaCkLHRWiS8ViNDeMFPTz1YWNSyqoX5x3exVxuG8jLsyXCww@mail.gmail.com>
 <CAAbtOZd-ueDuaGO7LsgivoS3TM-9MpRAe9iaY_84Hq+rTEK2pg@mail.gmail.com>
 <CABL7CQgSNjityHbmUoPagGUinEnkxMsDsjw6sf=zxUpqCaBv0Q@mail.gmail.com>
Message-ID: <20180305213438.sct3ilj4t7kzb7ci@carbo>

On Sun, 04 Mar 2018 23:41:36 -0800, Ralf Gommers wrote:
> Your proposal looks quite comprehensive and well written. I would suggest
> following the structure of PEPs a little more closely (
> https://www.python.org/dev/peps/pep-0001/#what-belongs-in-a-successful-pep),
> e.g. add the metadata and copyright bits, and use "motivation" and
> "rationale" as section headers for some of the content that you have.

We've tuned the Python PEP specification a bit for NumPy.  See

https://github.com/numpy/numpy/blob/master/doc/neps/nep-0000.rst

We're now finalizing the build machinery (using CircleCI, very much the
same as what is being used by SciPy).

Best regards
St?fan

From charlesr.harris at gmail.com  Mon Mar  5 21:30:57 2018
From: charlesr.harris at gmail.com (Charles R Harris)
Date: Mon, 5 Mar 2018 19:30:57 -0700
Subject: [SciPy-Dev] Spline interpolation in ndimage
In-Reply-To: <CAPOWHWkhQvKuia8NYBT4zTmpSRXmnH_MZExcBAUTvNgURaopgw@mail.gmail.com>
References: <CAPOWHWkhQvKuia8NYBT4zTmpSRXmnH_MZExcBAUTvNgURaopgw@mail.gmail.com>
Message-ID: <CAB6mnx+ibRAPZj0EY2CJ7ZXizAA=0KAst+-K8R5+PMYd7KcwtA@mail.gmail.com>

On Tue, Feb 27, 2018 at 4:59 AM, Jaime Fern?ndez del R?o <
jaime.frio at gmail.com> wrote:

> Hi all,
>
> We have been discussing in issue #8465
> <https://github.com/scipy/scipy/issues/8465> about the extension modes
> for the interpolation module of ndimage. As some other issues linked in
> that one show, this is an old problem, that has been recurringly discussed,
> and there mostly is agreement on what the correct behavior should be. But
> as all things ndimage, implementing any change is hard, in this case not
> only because the code is complicated, but also because of the mathematical
> subtleties of the interpolation method used.
>
> In trying to come up with a way forward for this I think I can handle the
> code complexity, but am having trouble being sure that I come up with a
> sound math approach. I think I have more or less figured it out, but I
> don't have a good academic background on signal processing, so was hoping
> that someone who does (I'm thinking of you, Chuck!) can read my longish
> description of things below and validate or correct it.
>
> My main source of knowledge has been:
>
> M. Unser, "Splines: a perfect fit for signal and image processing," in
> IEEE Signal Processing Magazine, vol. 16, no. 6, pp. 22-38, Nov 1999.
>
> Recommendations for bigger, better readings on the subject are also
> welcome!
>
> -----
>
> In what follows I'll assume the input image is 2-D and NxN for simplicity,
> but I think everything generalizes easily
>
> If I'm understanding it right, all of our interpolation functions use
> B-splines for interpolation. So instead of having NxN discrete values on a
> grid, we have NxN B-splines centered at the grid points, and we keep the
> NxN coefficients multiplying each spline to reproduce the original image at
> the grid points exactly. If the spline order is < 2, the spline
> coefficients are the same as the image values. But if order >= 2 (and we
> use 3 by default), these have to be calculated in a non-trivial fashion.
> This can be done efficiently by applying a separable filter, which is
> implemented in ndimage.spline_filter.
>
> Because B-splines have compact support, when using splines of order n we
> only need to consider the B-splines on an (n + 1)x(n + 1) grid neighborhood
> of the point being interpolated. This is more or less straightforward,
> until you move close to the edges and all of a sudden some of the points in
> your grid neighborhood fall outside the original image grid. We have our
> extend mode, which controls how this points outside should be mapped to
> points inside. But here is where things start getting tricky...
>
> When the spline coefficients are computed (i.e. when ndimage.spline_filter
> is called), assumptions have to be made about boundary conditions. But
> ndimage.spline_filter does not take a mode parameter to control this! So
> what I think ndimage does is compute the spline coefficients assuming
> "mirror symmetric" boundary conditions, i.e.:
>
> a b c d c b | a b c d | c b a b c d
>

For "a, b, c", I would write that as "..., a, b, c, b, a, b, c, ..."
extended to infinity in both directions because we are applying IIR
filters. Fortunately, the IIR filters fall off rapidly, so one need not
take too many extended points to start them up. Counter intuitively, I
think the interpolation "mode" only applies to the spline coefficients, not
to the original image, resulting in unexpected behavior at the edges.

If we want to have "mode" apply to the image rather than the filtered
results, we can do that, but the result of the filtering would best be a
named tuple containing the mode, the spline order, and,  in the case of a
constant extension, extra edge coefficients. The  coefficients for the
"wrap" and "constant" modes can then be obtained by appropriately extending
the data for starting up the filters. For the "wrap" and "reflect" modes,
the coefficents have the same symmetries, so that can be used to get the
needed coefficients outside the boundaries.

The spline_filter using IIR is just a clever way to solve A*coef = data.
For the cubic case and three data points, A depends on the mode as follows:

|4 2 0|
|1 4 1| x 1/6 (reflect about center of edge pixels -- we do this)
|0 2 4|

|5 1 0|
|1 4 1| x 1/6 (reflect about edge -- we don't do this)
|0 1 5|

|4 1 1|
|1 4 1| x 1/6 (wrap -- we don't do this)
|1 1 4|

Explicitly inverting such matrices also provides a good test to check that
things are done right if we go this way. As you have pointed out, the
proper coefficients will vary depending on the mode.

The spline coefficients have the same symmetry as the image pixels.
Unfortunately, `map_coordinates` reflects about the edges, which doesn't
match any of the coefficients that we compute.


> So if our interpolated point is within the image boundaries, but some of
> the grid points in its (n + 1)x(n + 1) neighborhood fall outside the
> boundary, the code uses a mirror symmetric extension mode to fill in those
> values. This approach has a funny smell to it, but even if it's formally
> incorrect I think it would only be marginally so, as the values are
> probably going to be very close to the correct ones.
>
> The problem comes when the point being interpolated is outside the
> boundaries of the image. We cannot use mirror-symmetric spline coefficients
> to extend if e.g. we have been asked to extend using wrap mode. So what
> ndimage does is first map the value outside the boundaries to a value
> within the boundaries, using the given extension mode, then interpolate it
> as before, using mirror-symmetric coefficients if needed because its (n +
> 1)x(n + 1) neighborhood extends outside. Again, this smells funny, but it
> is either correct or very close to correct.
>
> This is mostly all good and well, except for the "wrap" and "reflect"
> extension modes: in these cases the area within one pixel of the image
> boundaries is different from anything inside the image, so we cannot use
> that approach. So what ndimage does is basically make shit up and use
> something similar, but not quite right. "reflect" is mostly correct, except
> for within that pixel of the boundary, but "wrap" is a surprising and
> confusing mess.
>
> So how do we fix this? I see two ways forward:
>
>    1. What seems the more correct approach would be to compute the spline
>    coefficients taking into account the extension mode to be used, then use
>    the same extension mode to fill in the neighborhood values when
>    interpolating for a point outside the boundaries.
>    1. First question is whether this is doable? I need to work out the
>       math, but for "wrap" it looks like it should be, not 100% sure if also is
>       for "reflect".
>
>
Yes. The constant mode is actually the tricky one. I'd be tempted to make
it orthogonal to "reflect", "wrap", and "nearest", that way we don't need
to compute extra coefficients. Note that if the image isn't prefiltered, we
are running a smoothing filter over it.

>
>    1. Assuming it is it has the main advantage of doing things in a more
>       general and understandable way once you have enough background knowledge.
>       2. It does go a little bit against our API design: you can control
>       whether the input is spline-filtered automatically with a parameter, the
>       idea being that you may want to do the filtering yourself if you are going
>       to apply several different transformations to the same image. If the mode
>       of the filtering has to be synced with the mode of the transformation,
>       letting the user do it themselves is a recipe for disaster, because it's
>       going to lead to very hard to track bugs.
>       3. As elsewhere in ndimage, the current implementation does a lot
>       of caching, which works because it always interpolates for a point within
>       the image boundaries. If we started interpolating for points outside the
>       boundaries without first mapping to within there may be a performance hit
>       which has to be evaluated.
>    1. The other approach is,  for "wrap" and "reflect" modes, pad the
>    input image with an extra pixel in each direction, then compute our
>    current "mirror symmetric" spline coefficients, and leave things as they
>    are right now, aside from some changes to the mapping of values to take the
>    extra pixels into account.
>
>
Are "mirror" and "reflect" actually different? The function documentation
only mentions "reflect".


>
>    1. This looks like a nightmare of special cases everywhere and
>       potential off-by-one errors while putting it together, but it would just go
>       along with the ugliness of the existing code.
>       2. It's unclear what we would do if we are given an input with
>       prefilter=False, so this also breaks the current API, probably even more so.
>
> Any thoughts or recommendations are very welcome!
>

The docstrings need fixing, they are almost useless.

Chuck
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scipy-dev/attachments/20180305/47672b3e/attachment-0001.html>

From ralf.gommers at gmail.com  Mon Mar  5 23:06:11 2018
From: ralf.gommers at gmail.com (Ralf Gommers)
Date: Mon, 5 Mar 2018 20:06:11 -0800
Subject: [SciPy-Dev] Numba as a dependency for SciPy?
Message-ID: <CABL7CQienTTb9t1ZURWL1P_OQFK45Sb3P+0cW_LF1ufCaQpjQg@mail.gmail.com>

Hi all,

Goal of this email: start a discussion to decide whether we'd be okay with
relying on Numba as a dependency, now or in 1-2 years' time.

Context: in https://github.com/pydata/sparse/issues/126 a discussion is
ongoing about whether to adopt Cython or Numba, with Numba being preferred
by the majority. That `sparse` package is meant to provide sparse *arrays*
that down the line should either be replacing our current sparse *matrices*
or at least be integrated in scipy.sparse in addition to them. See
https://github.com/scipy/scipy/issues/8162 and
https://github.com/hameerabbasi/sparse-ndarray-protocols for more details
on that.

Also related is the question from Serge Guelton some weeks ago about
whether we'd want to rely on Pythran:
https://mail.python.org/pipermail/scipy-dev/2018-January/022325.html

On that Pythran thread I commented that we'd want to take these aspects
into account:
- portability
- performance
- maturity
- maintenance status (active devs, how quick do bugs get fixed after a
release with an issue)
- ease of use (@jit vs. Pythran comments vs. translate to .pyx syntax)
- size of generated binaries
- templating support for multiple dtypes
- debugging and optimization experience/tool

Debugging is one of the ones where I'd say Numba is still worse than
Cython, however that's being resolved as we speak:
https://github.com/numba/numba/issues/2788

One thing I missed in the above list is dependencies: while our use of
Cython only adds a build-time dependency, Numba would add a run-time
dependency. Given that binary wheels and conda packages for all major
platforms are available that's not a showstopper, but it matters.

Overall I'd say that:
- Numba is better than Cython at: performance, ease of use, size of
generated binaries, and templating support for multiple dtypes. Possibly
also maintenance status right now.
- Numba and Cython are about equally good at portability (I think, not much
data about exotic platforms for Numba).
- Cython is better than Numba at: maturity, debugging (but not for long
anymore probably), dependencies.

I'm usually pretty conservative in these things, but considering the above
I'm leaning towards saying use of Numba should be allowed in the future.
The added run-time dependency is the one major downside that's going to
stay, however compared to our Fortran headaches that's a relatively small
issue.

Thoughts?

Cheers,
Ralf
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scipy-dev/attachments/20180305/c2d81dce/attachment.html>

From ralf.gommers at gmail.com  Mon Mar  5 23:35:05 2018
From: ralf.gommers at gmail.com (Ralf Gommers)
Date: Mon, 5 Mar 2018 20:35:05 -0800
Subject: [SciPy-Dev] WIP: Class based Optimizers
In-Reply-To: <20180305213438.sct3ilj4t7kzb7ci@carbo>
References: <CAAbtOZee-V5+g=pHN+bp6YsA_Z-fa5UUSc5Zd5izM6jYS_A-aQ@mail.gmail.com>
 <1518642682.8666.18.camel@iki.fi> <5a84a711.05cfca0a.4a242.4608@mx.google.com>
 <CAPZ=bb2EeZXT=c5utHeKkfU2KH4BVEU6YDOYh4RWELDaseJUzA@mail.gmail.com>
 <CAHCaCkLHRWiS8ViNDeMFPTz1YWNSyqoX5x3exVxuG8jLsyXCww@mail.gmail.com>
 <CAAbtOZd-ueDuaGO7LsgivoS3TM-9MpRAe9iaY_84Hq+rTEK2pg@mail.gmail.com>
 <CABL7CQgSNjityHbmUoPagGUinEnkxMsDsjw6sf=zxUpqCaBv0Q@mail.gmail.com>
 <20180305213438.sct3ilj4t7kzb7ci@carbo>
Message-ID: <CABL7CQhokhgsKVH2z87mzPh-v8P1ddwVP2T9Gy4J+04R0+7TxQ@mail.gmail.com>

On Mon, Mar 5, 2018 at 1:34 PM, Stefan van der Walt <stefanv at berkeley.edu>
wrote:

> On Sun, 04 Mar 2018 23:41:36 -0800, Ralf Gommers wrote:
> > Your proposal looks quite comprehensive and well written. I would suggest
> > following the structure of PEPs a little more closely (
> > https://www.python.org/dev/peps/pep-0001/#what-belongs-
> in-a-successful-pep),
> > e.g. add the metadata and copyright bits, and use "motivation" and
> > "rationale" as section headers for some of the content that you have.
>
> We've tuned the Python PEP specification a bit for NumPy.  See
>
> https://github.com/numpy/numpy/blob/master/doc/neps/nep-0000.rst
>

Thanks, that seems fine to copy. The link to the template in nep-0000 is
broken, here it is:
https://github.com/numpy/numpy/blob/master/doc/neps/nep-template.rst

Ralf
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scipy-dev/attachments/20180305/07d7ad8c/attachment.html>

From alan.isaac at gmail.com  Mon Mar  5 23:58:17 2018
From: alan.isaac at gmail.com (Alan Isaac)
Date: Mon, 5 Mar 2018 23:58:17 -0500
Subject: [SciPy-Dev] WIP: Class based Optimizers
In-Reply-To: <CAB2ViTZZog8LbYGrwxTE29Tz0oXq_sDtOtiNKip_tXz5=4TD4A@mail.gmail.com>
References: <CAAbtOZee-V5+g=pHN+bp6YsA_Z-fa5UUSc5Zd5izM6jYS_A-aQ@mail.gmail.com>
 <1518642682.8666.18.camel@iki.fi>
 <5a84a711.05cfca0a.4a242.4608@mx.google.com>
 <CAPZ=bb2EeZXT=c5utHeKkfU2KH4BVEU6YDOYh4RWELDaseJUzA@mail.gmail.com>
 <CAHCaCkLHRWiS8ViNDeMFPTz1YWNSyqoX5x3exVxuG8jLsyXCww@mail.gmail.com>
 <CAAbtOZd-ueDuaGO7LsgivoS3TM-9MpRAe9iaY_84Hq+rTEK2pg@mail.gmail.com>
 <CAB2ViTZZog8LbYGrwxTE29Tz0oXq_sDtOtiNKip_tXz5=4TD4A@mail.gmail.com>
Message-ID: <d257219d-b34c-8129-82ed-83510f7350f0@gmail.com>

Didn't OpenOpt take some steps in this direction?
http://courses.csail.mit.edu/6.867/wiki/images/6/6e/Qp-openopt.pdf
Just making sure anything useful doesn't get overlooked, since
I did not see this mentioned.

Alan Isaac

On 3/5/2018 2:03 AM, Phillip Feldman wrote:
>  From the (beautifully written) draft PEP:
> 
> "Different optimization algorithms can inherit from Optimizer, with each of the subclasses overriding the __next__ method ..."
> 
> I'm unclear re. whether this approach would allow something like a parallel implementation of Nelder-Mead.
> 
> Phillip

> On Sun, Mar 4, 2018 at 7:05 PM, Andrew Nelson <andyfaff at gmail.com <mailto:andyfaff at gmail.com>> wrote:
> 
>     Scott Sievert and I have put a lot of work into preparing a draft of the PEP for class based scalar minimizers:
> 
>     https://github.com/andyfaff/scipy/blob/a52bb4f9029389da3ab072c92c609d71ed6943c6/PEP/1-Optimizer.rst
>     <https://github.com/andyfaff/scipy/blob/a52bb4f9029389da3ab072c92c609d71ed6943c6/PEP/1-Optimizer.rst>
> 
>     where we've tried to address comments already made in this thread, and from the WIP github PR. Scott and I look forward to hearing any comments/concerns/feedback about the
>     proposal. We can field any questions and address them in an updated PEP, as well as on here.
> 
>     Andrew.
> 
> 
>     p.s. Ralf/Pauli, could we add the PEP to a scipy/PEP, or scipy/scipep, repo? How should we discuss such this, and any further PEP? Should we have a scipep process, or shall we
>     keep things simple?

From ralf.gommers at gmail.com  Tue Mar  6 00:35:48 2018
From: ralf.gommers at gmail.com (Ralf Gommers)
Date: Mon, 5 Mar 2018 21:35:48 -0800
Subject: [SciPy-Dev] WIP: Class based Optimizers
In-Reply-To: <d257219d-b34c-8129-82ed-83510f7350f0@gmail.com>
References: <CAAbtOZee-V5+g=pHN+bp6YsA_Z-fa5UUSc5Zd5izM6jYS_A-aQ@mail.gmail.com>
 <1518642682.8666.18.camel@iki.fi> <5a84a711.05cfca0a.4a242.4608@mx.google.com>
 <CAPZ=bb2EeZXT=c5utHeKkfU2KH4BVEU6YDOYh4RWELDaseJUzA@mail.gmail.com>
 <CAHCaCkLHRWiS8ViNDeMFPTz1YWNSyqoX5x3exVxuG8jLsyXCww@mail.gmail.com>
 <CAAbtOZd-ueDuaGO7LsgivoS3TM-9MpRAe9iaY_84Hq+rTEK2pg@mail.gmail.com>
 <CAB2ViTZZog8LbYGrwxTE29Tz0oXq_sDtOtiNKip_tXz5=4TD4A@mail.gmail.com>
 <d257219d-b34c-8129-82ed-83510f7350f0@gmail.com>
Message-ID: <CABL7CQiS5fP0xdA8Ao_WMs7Tu-jKC7j3V6w30jGad4aXGocfyA@mail.gmail.com>

On Mon, Mar 5, 2018 at 8:58 PM, Alan Isaac <alan.isaac at gmail.com> wrote:

> Didn't OpenOpt take some steps in this direction?
> http://courses.csail.mit.edu/6.867/wiki/images/6/6e/Qp-openopt.pdf
> Just making sure anything useful doesn't get overlooked, since
> I did not see this mentioned.
>

Hard to say, since OpenOpt has disappeared AFAIK. There's no repo to be
found and openopt.org has been offline for a while.

Ralf



>
> Alan Isaac
>
> On 3/5/2018 2:03 AM, Phillip Feldman wrote:
>
>>  From the (beautifully written) draft PEP:
>>
>> "Different optimization algorithms can inherit from Optimizer, with each
>> of the subclasses overriding the __next__ method ..."
>>
>> I'm unclear re. whether this approach would allow something like a
>> parallel implementation of Nelder-Mead.
>>
>> Phillip
>>
>
> On Sun, Mar 4, 2018 at 7:05 PM, Andrew Nelson <andyfaff at gmail.com <mailto:
>> andyfaff at gmail.com>> wrote:
>>
>>     Scott Sievert and I have put a lot of work into preparing a draft of
>> the PEP for class based scalar minimizers:
>>
>>     https://github.com/andyfaff/scipy/blob/a52bb4f9029389da3ab07
>> 2c92c609d71ed6943c6/PEP/1-Optimizer.rst
>>     <https://github.com/andyfaff/scipy/blob/a52bb4f9029389da3ab0
>> 72c92c609d71ed6943c6/PEP/1-Optimizer.rst>
>>
>>     where we've tried to address comments already made in this thread,
>> and from the WIP github PR. Scott and I look forward to hearing any
>> comments/concerns/feedback about the
>>     proposal. We can field any questions and address them in an updated
>> PEP, as well as on here.
>>
>>     Andrew.
>>
>>
>>     p.s. Ralf/Pauli, could we add the PEP to a scipy/PEP, or
>> scipy/scipep, repo? How should we discuss such this, and any further PEP?
>> Should we have a scipep process, or shall we
>>     keep things simple?
>>
> _______________________________________________
> SciPy-Dev mailing list
> SciPy-Dev at python.org
> https://mail.python.org/mailman/listinfo/scipy-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scipy-dev/attachments/20180305/a031db44/attachment-0001.html>

From cimrman3 at ntc.zcu.cz  Tue Mar  6 06:06:20 2018
From: cimrman3 at ntc.zcu.cz (Robert Cimrman)
Date: Tue, 6 Mar 2018 12:06:20 +0100
Subject: [SciPy-Dev] ANN: SfePy 2018.1
Message-ID: <07f7c4f6-abae-a7f4-5e87-e29dbab2e296@ntc.zcu.cz>

I am pleased to announce release 2018.1 of SfePy.

Description
-----------

SfePy (simple finite elements in Python) is a software for solving systems of
coupled partial differential equations by the finite element method or by the
isogeometric analysis (limited support). It is distributed under the new BSD
license.

Home page: http://sfepy.org
Mailing list: https://mail.python.org/mm3/mailman3/lists/sfepy.python.org/
Git (source) repository, issue tracker: https://github.com/sfepy/sfepy

Highlights of this release
--------------------------

- major update of time-stepping solvers and solver handling
- Newmark and Bathe elastodynamics solvers
- interface to MUMPS linear solver
- new examples:
   - iron plate impact problem (elastodynamics)
   - incompressible Mooney-Rivlin material model (hyperelasticity) as a script

For full release notes see http://docs.sfepy.org/doc/release_notes.html#id1
(rather long and technical).

Cheers,
Robert Cimrman

---

Contributors to this release in alphabetical order:

Robert Cimrman
Jan Heczko
Jan Kopacka
Vladimir Lukes


From tyler.je.reddy at gmail.com  Tue Mar  6 15:02:35 2018
From: tyler.je.reddy at gmail.com (Tyler Reddy)
Date: Tue, 6 Mar 2018 13:02:35 -0700
Subject: [SciPy-Dev] Numba as a dependency for SciPy?
In-Reply-To: <CABL7CQienTTb9t1ZURWL1P_OQFK45Sb3P+0cW_LF1ufCaQpjQg@mail.gmail.com>
References: <CABL7CQienTTb9t1ZURWL1P_OQFK45Sb3P+0cW_LF1ufCaQpjQg@mail.gmail.com>
Message-ID: <CAHPuU_YNu+3pWLD0=8ONQsVT+h6zAJzRVTru97NHZLJg7_EHTQ@mail.gmail.com>

Interesting discussion. Would our plan be to support both side-by-side for
a while & just see what happens with the evolution of the ecosystem? If
there's no clear winner in the short-term would we discourage PRs that
simply migrate from Cython to numba for say 1.5 x performance increase?
What about an algorithm that mixes the two approaches -- some numba and
some Cython components for whatever reason -- is that discouraged?

It looks like numba plays ok with airspeed velocity -- presumably mixing
Cython / numba in our suite will be ok?



On 5 March 2018 at 21:06, Ralf Gommers <ralf.gommers at gmail.com> wrote:

> Hi all,
>
> Goal of this email: start a discussion to decide whether we'd be okay with
> relying on Numba as a dependency, now or in 1-2 years' time.
>
> Context: in https://github.com/pydata/sparse/issues/126 a discussion is
> ongoing about whether to adopt Cython or Numba, with Numba being preferred
> by the majority. That `sparse` package is meant to provide sparse *arrays*
> that down the line should either be replacing our current sparse *matrices*
> or at least be integrated in scipy.sparse in addition to them. See
> https://github.com/scipy/scipy/issues/8162 and https://github.com/
> hameerabbasi/sparse-ndarray-protocols for more details on that.
>
> Also related is the question from Serge Guelton some weeks ago about
> whether we'd want to rely on Pythran: https://mail.python.org/
> pipermail/scipy-dev/2018-January/022325.html
>
> On that Pythran thread I commented that we'd want to take these aspects
> into account:
> - portability
> - performance
> - maturity
> - maintenance status (active devs, how quick do bugs get fixed after a
> release with an issue)
> - ease of use (@jit vs. Pythran comments vs. translate to .pyx syntax)
> - size of generated binaries
> - templating support for multiple dtypes
> - debugging and optimization experience/tool
>
> Debugging is one of the ones where I'd say Numba is still worse than
> Cython, however that's being resolved as we speak:
> https://github.com/numba/numba/issues/2788
>
> One thing I missed in the above list is dependencies: while our use of
> Cython only adds a build-time dependency, Numba would add a run-time
> dependency. Given that binary wheels and conda packages for all major
> platforms are available that's not a showstopper, but it matters.
>
> Overall I'd say that:
> - Numba is better than Cython at: performance, ease of use, size of
> generated binaries, and templating support for multiple dtypes. Possibly
> also maintenance status right now.
> - Numba and Cython are about equally good at portability (I think, not
> much data about exotic platforms for Numba).
> - Cython is better than Numba at: maturity, debugging (but not for long
> anymore probably), dependencies.
>
> I'm usually pretty conservative in these things, but considering the above
> I'm leaning towards saying use of Numba should be allowed in the future.
> The added run-time dependency is the one major downside that's going to
> stay, however compared to our Fortran headaches that's a relatively small
> issue.
>
> Thoughts?
>
> Cheers,
> Ralf
>
>
>
> _______________________________________________
> SciPy-Dev mailing list
> SciPy-Dev at python.org
> https://mail.python.org/mailman/listinfo/scipy-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scipy-dev/attachments/20180306/81d01b60/attachment.html>

From serge.guelton at telecom-bretagne.eu  Tue Mar  6 16:13:33 2018
From: serge.guelton at telecom-bretagne.eu (Serge Guelton)
Date: Tue, 6 Mar 2018 22:13:33 +0100
Subject: [SciPy-Dev] Numba as a dependency for SciPy?
In-Reply-To: <CABL7CQienTTb9t1ZURWL1P_OQFK45Sb3P+0cW_LF1ufCaQpjQg@mail.gmail.com>
References: <CABL7CQienTTb9t1ZURWL1P_OQFK45Sb3P+0cW_LF1ufCaQpjQg@mail.gmail.com>
Message-ID: <20180306211333.GA29117@lakota>

> Overall I'd say that:
> - Numba is better than Cython at: performance, ease of use, size of generated
> binaries, and templating support for multiple dtypes. Possibly also maintenance
> status right now.

Hi all,

To assert that statement, I've quickly setup a bunch of notebooks with
cython kernels extracted from the scipy codebase here:

    https://github.com/serge-sans-paille/scipy-kernels

For the sake of the comparison, I've contributed Pythran implementation
of these kernels, and I hope someone in the ML will issue a PR with
numba version.

Please note that the goal is *not* to claim that a given compiler is
better than another. The benchmarks provided only target a subset of the
functions input space, I have not tuned the backend compiler for a given
architecture nor tried to enable parallelism.

It's just there to give a rough idea of the ease-of-use / performance
tradeoffs.

Best,

Serge

From perimosocordiae at gmail.com  Tue Mar  6 16:21:06 2018
From: perimosocordiae at gmail.com (CJ Carey)
Date: Tue, 06 Mar 2018 21:21:06 +0000
Subject: [SciPy-Dev] Numba as a dependency for SciPy?
In-Reply-To: <CAHPuU_YNu+3pWLD0=8ONQsVT+h6zAJzRVTru97NHZLJg7_EHTQ@mail.gmail.com>
References: <CABL7CQienTTb9t1ZURWL1P_OQFK45Sb3P+0cW_LF1ufCaQpjQg@mail.gmail.com>
 <CAHPuU_YNu+3pWLD0=8ONQsVT+h6zAJzRVTru97NHZLJg7_EHTQ@mail.gmail.com>
Message-ID: <CAEfGn+w=3SkYWGoeXoOzCsoCxcoGnW3RJmEnGDUM4cqa4gR1NQ@mail.gmail.com>

I think adding a required runtime dependency may be overly restrictive,
given scipy's position near(-ish) the base of the scientific computing
pyramid.

Would it be possible to run numba-optimized code on systems with numba
installed without impacting "vanilla" users?


On Tue, Mar 6, 2018 at 3:03 PM Tyler Reddy <tyler.je.reddy at gmail.com> wrote:

> Interesting discussion. Would our plan be to support both side-by-side for
> a while & just see what happens with the evolution of the ecosystem? If
> there's no clear winner in the short-term would we discourage PRs that
> simply migrate from Cython to numba for say 1.5 x performance increase?
> What about an algorithm that mixes the two approaches -- some numba and
> some Cython components for whatever reason -- is that discouraged?
>
> It looks like numba plays ok with airspeed velocity -- presumably mixing
> Cython / numba in our suite will be ok?
>
>
>
> On 5 March 2018 at 21:06, Ralf Gommers <ralf.gommers at gmail.com> wrote:
>
>> Hi all,
>>
>> Goal of this email: start a discussion to decide whether we'd be okay
>> with relying on Numba as a dependency, now or in 1-2 years' time.
>>
>> Context: in https://github.com/pydata/sparse/issues/126 a discussion is
>> ongoing about whether to adopt Cython or Numba, with Numba being preferred
>> by the majority. That `sparse` package is meant to provide sparse *arrays*
>> that down the line should either be replacing our current sparse *matrices*
>> or at least be integrated in scipy.sparse in addition to them. See
>> https://github.com/scipy/scipy/issues/8162 and
>> https://github.com/hameerabbasi/sparse-ndarray-protocols for more
>> details on that.
>>
>> Also related is the question from Serge Guelton some weeks ago about
>> whether we'd want to rely on Pythran:
>> https://mail.python.org/pipermail/scipy-dev/2018-January/022325.html
>>
>> On that Pythran thread I commented that we'd want to take these aspects
>> into account:
>> - portability
>> - performance
>> - maturity
>> - maintenance status (active devs, how quick do bugs get fixed after a
>> release with an issue)
>> - ease of use (@jit vs. Pythran comments vs. translate to .pyx syntax)
>> - size of generated binaries
>> - templating support for multiple dtypes
>> - debugging and optimization experience/tool
>>
>> Debugging is one of the ones where I'd say Numba is still worse than
>> Cython, however that's being resolved as we speak:
>> https://github.com/numba/numba/issues/2788
>>
>> One thing I missed in the above list is dependencies: while our use of
>> Cython only adds a build-time dependency, Numba would add a run-time
>> dependency. Given that binary wheels and conda packages for all major
>> platforms are available that's not a showstopper, but it matters.
>>
>> Overall I'd say that:
>> - Numba is better than Cython at: performance, ease of use, size of
>> generated binaries, and templating support for multiple dtypes. Possibly
>> also maintenance status right now.
>> - Numba and Cython are about equally good at portability (I think, not
>> much data about exotic platforms for Numba).
>> - Cython is better than Numba at: maturity, debugging (but not for long
>> anymore probably), dependencies.
>>
>> I'm usually pretty conservative in these things, but considering the
>> above I'm leaning towards saying use of Numba should be allowed in the
>> future. The added run-time dependency is the one major downside that's
>> going to stay, however compared to our Fortran headaches that's a
>> relatively small issue.
>>
>> Thoughts?
>>
>> Cheers,
>> Ralf
>>
>>
>>
>> _______________________________________________
>> SciPy-Dev mailing list
>> SciPy-Dev at python.org
>> https://mail.python.org/mailman/listinfo/scipy-dev
>>
>>
> _______________________________________________
> SciPy-Dev mailing list
> SciPy-Dev at python.org
> https://mail.python.org/mailman/listinfo/scipy-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scipy-dev/attachments/20180306/7226824a/attachment-0001.html>

From shoyer at gmail.com  Tue Mar  6 16:55:41 2018
From: shoyer at gmail.com (Stephan Hoyer)
Date: Tue, 06 Mar 2018 21:55:41 +0000
Subject: [SciPy-Dev] Numba as a dependency for SciPy?
In-Reply-To: <CAEfGn+w=3SkYWGoeXoOzCsoCxcoGnW3RJmEnGDUM4cqa4gR1NQ@mail.gmail.com>
References: <CABL7CQienTTb9t1ZURWL1P_OQFK45Sb3P+0cW_LF1ufCaQpjQg@mail.gmail.com>
 <CAHPuU_YNu+3pWLD0=8ONQsVT+h6zAJzRVTru97NHZLJg7_EHTQ@mail.gmail.com>
 <CAEfGn+w=3SkYWGoeXoOzCsoCxcoGnW3RJmEnGDUM4cqa4gR1NQ@mail.gmail.com>
Message-ID: <CAEQ_TvdVe0n+EZyHGyWaBKTs7PQ_=cojqEFhkEgS5njizJCDcw@mail.gmail.com>

Numba does have a mechanism for exporting pre-compiled code:
http://numba.pydata.org/numba-doc/dev/user/pycc.html

It would be interesting to see if those limitations are flexible enough for
SciPy.

On Tue, Mar 6, 2018 at 1:21 PM CJ Carey <perimosocordiae at gmail.com> wrote:

> I think adding a required runtime dependency may be overly restrictive,
> given scipy's position near(-ish) the base of the scientific computing
> pyramid.
>
> Would it be possible to run numba-optimized code on systems with numba
> installed without impacting "vanilla" users?
>
>
> On Tue, Mar 6, 2018 at 3:03 PM Tyler Reddy <tyler.je.reddy at gmail.com>
> wrote:
>
>> Interesting discussion. Would our plan be to support both side-by-side
>> for a while & just see what happens with the evolution of the ecosystem? If
>> there's no clear winner in the short-term would we discourage PRs that
>> simply migrate from Cython to numba for say 1.5 x performance increase?
>> What about an algorithm that mixes the two approaches -- some numba and
>> some Cython components for whatever reason -- is that discouraged?
>>
>> It looks like numba plays ok with airspeed velocity -- presumably mixing
>> Cython / numba in our suite will be ok?
>>
>>
>>
>> On 5 March 2018 at 21:06, Ralf Gommers <ralf.gommers at gmail.com> wrote:
>>
>>> Hi all,
>>>
>>> Goal of this email: start a discussion to decide whether we'd be okay
>>> with relying on Numba as a dependency, now or in 1-2 years' time.
>>>
>>> Context: in https://github.com/pydata/sparse/issues/126 a discussion is
>>> ongoing about whether to adopt Cython or Numba, with Numba being preferred
>>> by the majority. That `sparse` package is meant to provide sparse *arrays*
>>> that down the line should either be replacing our current sparse *matrices*
>>> or at least be integrated in scipy.sparse in addition to them. See
>>> https://github.com/scipy/scipy/issues/8162 and
>>> https://github.com/hameerabbasi/sparse-ndarray-protocols for more
>>> details on that.
>>>
>>> Also related is the question from Serge Guelton some weeks ago about
>>> whether we'd want to rely on Pythran:
>>> https://mail.python.org/pipermail/scipy-dev/2018-January/022325.html
>>>
>>> On that Pythran thread I commented that we'd want to take these aspects
>>> into account:
>>> - portability
>>> - performance
>>> - maturity
>>> - maintenance status (active devs, how quick do bugs get fixed after a
>>> release with an issue)
>>> - ease of use (@jit vs. Pythran comments vs. translate to .pyx syntax)
>>> - size of generated binaries
>>> - templating support for multiple dtypes
>>> - debugging and optimization experience/tool
>>>
>>> Debugging is one of the ones where I'd say Numba is still worse than
>>> Cython, however that's being resolved as we speak:
>>> https://github.com/numba/numba/issues/2788
>>>
>>> One thing I missed in the above list is dependencies: while our use of
>>> Cython only adds a build-time dependency, Numba would add a run-time
>>> dependency. Given that binary wheels and conda packages for all major
>>> platforms are available that's not a showstopper, but it matters.
>>>
>>> Overall I'd say that:
>>> - Numba is better than Cython at: performance, ease of use, size of
>>> generated binaries, and templating support for multiple dtypes. Possibly
>>> also maintenance status right now.
>>> - Numba and Cython are about equally good at portability (I think, not
>>> much data about exotic platforms for Numba).
>>> - Cython is better than Numba at: maturity, debugging (but not for long
>>> anymore probably), dependencies.
>>>
>>> I'm usually pretty conservative in these things, but considering the
>>> above I'm leaning towards saying use of Numba should be allowed in the
>>> future. The added run-time dependency is the one major downside that's
>>> going to stay, however compared to our Fortran headaches that's a
>>> relatively small issue.
>>>
>>> Thoughts?
>>>
>>> Cheers,
>>> Ralf
>>>
>>>
>>>
>>> _______________________________________________
>>> SciPy-Dev mailing list
>>> SciPy-Dev at python.org
>>> https://mail.python.org/mailman/listinfo/scipy-dev
>>>
>>>
>> _______________________________________________
>> SciPy-Dev mailing list
>> SciPy-Dev at python.org
>> https://mail.python.org/mailman/listinfo/scipy-dev
>>
> _______________________________________________
> SciPy-Dev mailing list
> SciPy-Dev at python.org
> https://mail.python.org/mailman/listinfo/scipy-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scipy-dev/attachments/20180306/f0afdeb3/attachment.html>

From matthew.brett at gmail.com  Tue Mar  6 17:54:15 2018
From: matthew.brett at gmail.com (Matthew Brett)
Date: Tue, 6 Mar 2018 22:54:15 +0000
Subject: [SciPy-Dev] Numba as a dependency for SciPy?
In-Reply-To: <CAEfGn+w=3SkYWGoeXoOzCsoCxcoGnW3RJmEnGDUM4cqa4gR1NQ@mail.gmail.com>
References: <CABL7CQienTTb9t1ZURWL1P_OQFK45Sb3P+0cW_LF1ufCaQpjQg@mail.gmail.com>
 <CAHPuU_YNu+3pWLD0=8ONQsVT+h6zAJzRVTru97NHZLJg7_EHTQ@mail.gmail.com>
 <CAEfGn+w=3SkYWGoeXoOzCsoCxcoGnW3RJmEnGDUM4cqa4gR1NQ@mail.gmail.com>
Message-ID: <CAH6Pt5rB2srgYbgQE6qYBOr8PxaLqeD1hb6jnDcV1MP9baewYQ@mail.gmail.com>

On Tue, Mar 6, 2018 at 9:21 PM, CJ Carey <perimosocordiae at gmail.com> wrote:
> I think adding a required runtime dependency may be overly restrictive,
> given scipy's position near(-ish) the base of the scientific computing
> pyramid.

Yes, that's a worry I have too.  You may remember Samuel Maybury on
the mailing list recently sighing somewhat when he found he had to get
numba installed on the Raspberry Pi.  My guess is numba will be fine
on the standard platforms and a significant problem on non-standard
ones.

Cheers,

Matthew

From stefanv at berkeley.edu  Tue Mar  6 18:22:53 2018
From: stefanv at berkeley.edu (Stefan van der Walt)
Date: Tue, 6 Mar 2018 15:22:53 -0800
Subject: [SciPy-Dev] SciPy Central Rescue
Message-ID: <20180306232253.fz6rixtqedai27m6@carbo>

Hi, everyone

Jiayue Li, a student working with me here at BIDS, recently converted
the database of the now-defunc SciPy Central into Sphinx-rendered pages:

https://machine-shop.github.io/scipy-central-rescue/

The source materials are at:

https://github.com/machine-shop/scipy-central-rescue

We are happy to move this over to the SciPy repo, and to integrate these
materials wherever is most applicable.  Let us know!

Thanks also to Surya Kasturi who helped us get access to the original
database.

Best regards,
St?fan

From sseibert at anaconda.com  Tue Mar  6 18:32:47 2018
From: sseibert at anaconda.com (Stanley Seibert)
Date: Tue, 6 Mar 2018 17:32:47 -0600
Subject: [SciPy-Dev] Numba as a dependency for SciPy?
In-Reply-To: <CAH6Pt5rB2srgYbgQE6qYBOr8PxaLqeD1hb6jnDcV1MP9baewYQ@mail.gmail.com>
References: <CABL7CQienTTb9t1ZURWL1P_OQFK45Sb3P+0cW_LF1ufCaQpjQg@mail.gmail.com>
 <CAHPuU_YNu+3pWLD0=8ONQsVT+h6zAJzRVTru97NHZLJg7_EHTQ@mail.gmail.com>
 <CAEfGn+w=3SkYWGoeXoOzCsoCxcoGnW3RJmEnGDUM4cqa4gR1NQ@mail.gmail.com>
 <CAH6Pt5rB2srgYbgQE6qYBOr8PxaLqeD1hb6jnDcV1MP9baewYQ@mail.gmail.com>
Message-ID: <CADv3RKSsrvnQOTtmrhXFdcEqcxG0iMXNXuhcu-4=Cg41u6s3CA@mail.gmail.com>

(Hi, as someone from the Numba project, my apologies for wading into this
discussion late.)

Last time we checked, Numba on ARM was pretty close to working already.  We
have an open PR for ARMv7 that we just ran out of time to finish QA'ing (
https://github.com/numba/numba/pull/1968) a while ago, and never have
gotten back to because there was not much demand at the time.  It sounds
like this is tripping up more people now, so we can take a look again.

One thing that really is unpleasant on ARM is compiling LLVM, so we would
probably want to make sure we had conda packages and wheels available for
llvmlite on ARM.  Is there any precedent for posting Linux ARMv7 wheels to
PyPI?  (For conda, we would just make sure they appear in Jonathan's
berryconda channel.)

A more difficult platform is POWER8, where we tried to do a port and got
stuck last year.  Recently, someone has figured out what the issues were,
and it sounds like several of them stemmed from LLVM bugs in the POWER8
backend that may or may not be fixed in the next release.  IBM is
interested in improving the situation, so hopefully that will be sorted out
soon.

On Tue, Mar 6, 2018 at 4:54 PM, Matthew Brett <matthew.brett at gmail.com>
wrote:

> On Tue, Mar 6, 2018 at 9:21 PM, CJ Carey <perimosocordiae at gmail.com>
> wrote:
> > I think adding a required runtime dependency may be overly restrictive,
> > given scipy's position near(-ish) the base of the scientific computing
> > pyramid.
>
> Yes, that's a worry I have too.  You may remember Samuel Maybury on
> the mailing list recently sighing somewhat when he found he had to get
> numba installed on the Raspberry Pi.  My guess is numba will be fine
> on the standard platforms and a significant problem on non-standard
> ones.
>
> Cheers,
>
> Matthew
> _______________________________________________
> SciPy-Dev mailing list
> SciPy-Dev at python.org
> https://mail.python.org/mailman/listinfo/scipy-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scipy-dev/attachments/20180306/f8bfaa2a/attachment-0001.html>

From charlesr.harris at gmail.com  Tue Mar  6 20:00:25 2018
From: charlesr.harris at gmail.com (Charles R Harris)
Date: Tue, 6 Mar 2018 18:00:25 -0700
Subject: [SciPy-Dev] Numba as a dependency for SciPy?
In-Reply-To: <CABL7CQienTTb9t1ZURWL1P_OQFK45Sb3P+0cW_LF1ufCaQpjQg@mail.gmail.com>
References: <CABL7CQienTTb9t1ZURWL1P_OQFK45Sb3P+0cW_LF1ufCaQpjQg@mail.gmail.com>
Message-ID: <CAB6mnx+BjKP4r2JyS9+1KK9rgZQgJ7uzJHhTw50Lvg88sBhmfw@mail.gmail.com>

On Mon, Mar 5, 2018 at 9:06 PM, Ralf Gommers <ralf.gommers at gmail.com> wrote:

> Hi all,
>
> Goal of this email: start a discussion to decide whether we'd be okay with
> relying on Numba as a dependency, now or in 1-2 years' time.
>
> Context: in https://github.com/pydata/sparse/issues/126 a discussion is
> ongoing about whether to adopt Cython or Numba, with Numba being preferred
> by the majority. That `sparse` package is meant to provide sparse *arrays*
> that down the line should either be replacing our current sparse *matrices*
> or at least be integrated in scipy.sparse in addition to them. See
> https://github.com/scipy/scipy/issues/8162 and https://github.com/
> hameerabbasi/sparse-ndarray-protocols for more details on that.
>
> Also related is the question from Serge Guelton some weeks ago about
> whether we'd want to rely on Pythran: https://mail.python.org/
> pipermail/scipy-dev/2018-January/022325.html
>
> On that Pythran thread I commented that we'd want to take these aspects
> into account:
> - portability
> - performance
> - maturity
> - maintenance status (active devs, how quick do bugs get fixed after a
> release with an issue)
> - ease of use (@jit vs. Pythran comments vs. translate to .pyx syntax)
> - size of generated binaries
> - templating support for multiple dtypes
> - debugging and optimization experience/tool
>
> Debugging is one of the ones where I'd say Numba is still worse than
> Cython, however that's being resolved as we speak:
> https://github.com/numba/numba/issues/2788
>
> One thing I missed in the above list is dependencies: while our use of
> Cython only adds a build-time dependency, Numba would add a run-time
> dependency. Given that binary wheels and conda packages for all major
> platforms are available that's not a showstopper, but it matters.
>
> Overall I'd say that:
> - Numba is better than Cython at: performance, ease of use, size of
> generated binaries, and templating support for multiple dtypes. Possibly
> also maintenance status right now.
> - Numba and Cython are about equally good at portability (I think, not
> much data about exotic platforms for Numba).
> - Cython is better than Numba at: maturity, debugging (but not for long
> anymore probably), dependencies.
>
> I'm usually pretty conservative in these things, but considering the above
> I'm leaning towards saying use of Numba should be allowed in the future.
> The added run-time dependency is the one major downside that's going to
> stay, however compared to our Fortran headaches that's a relatively small
> issue.
>

I like the idea of using Numba, but remain a bit skeptical about the
dependencies and long term maintenance. I suppose the same could have been
said about NumPy and SciPy ten years ago, the continued maintenance and
availability of both was not a foregone conclusion. It is probably best to
wait a bit to see how things shake out, but I'm not opposed to the use of
either Pythran or Numba on technical grounds. There have been other such
attempts, weave and that other tensor code -- I forget the name -- were
both present in early releases and have since disappeared.

Chuck
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scipy-dev/attachments/20180306/80e2a4a8/attachment.html>

From njs at pobox.com  Tue Mar  6 20:02:14 2018
From: njs at pobox.com (Nathaniel Smith)
Date: Tue, 6 Mar 2018 17:02:14 -0800
Subject: [SciPy-Dev] Numba as a dependency for SciPy?
In-Reply-To: <CADv3RKSsrvnQOTtmrhXFdcEqcxG0iMXNXuhcu-4=Cg41u6s3CA@mail.gmail.com>
References: <CABL7CQienTTb9t1ZURWL1P_OQFK45Sb3P+0cW_LF1ufCaQpjQg@mail.gmail.com>
 <CAHPuU_YNu+3pWLD0=8ONQsVT+h6zAJzRVTru97NHZLJg7_EHTQ@mail.gmail.com>
 <CAEfGn+w=3SkYWGoeXoOzCsoCxcoGnW3RJmEnGDUM4cqa4gR1NQ@mail.gmail.com>
 <CAH6Pt5rB2srgYbgQE6qYBOr8PxaLqeD1hb6jnDcV1MP9baewYQ@mail.gmail.com>
 <CADv3RKSsrvnQOTtmrhXFdcEqcxG0iMXNXuhcu-4=Cg41u6s3CA@mail.gmail.com>
Message-ID: <CAPJVwBnBva+LHHO+mFum=3VfNQ+CUjdHLEB0iO8xKzmjJ-f2nA@mail.gmail.com>

On Tue, Mar 6, 2018 at 3:32 PM, Stanley Seibert <sseibert at anaconda.com> wrote:
> One thing that really is unpleasant on ARM is compiling LLVM, so we would
> probably want to make sure we had conda packages and wheels available for
> llvmlite on ARM.  Is there any precedent for posting Linux ARMv7 wheels to
> PyPI?  (For conda, we would just make sure they appear in Jonathan's
> berryconda channel.)

No, there currently isn't any way to post Linux ARM wheels on PyPI.
There could be -- the main issue is defining what "Linux ARMv7" means
in enough detail for pip to figure out which wheels it should be
trying to download on a given system.

-n

-- 
Nathaniel J. Smith -- https://vorpus.org

From ralf.gommers at gmail.com  Wed Mar  7 00:24:07 2018
From: ralf.gommers at gmail.com (Ralf Gommers)
Date: Tue, 6 Mar 2018 21:24:07 -0800
Subject: [SciPy-Dev] SciPy Central Rescue
In-Reply-To: <20180306232253.fz6rixtqedai27m6@carbo>
References: <20180306232253.fz6rixtqedai27m6@carbo>
Message-ID: <CABL7CQjaAOkrbJnzRMfWnD4Rv4hsWkttWJiTOvkpdkgGP+ZYxQ@mail.gmail.com>

On Tue, Mar 6, 2018 at 3:22 PM, Stefan van der Walt <stefanv at berkeley.edu>
wrote:

> Hi, everyone
>
> Jiayue Li, a student working with me here at BIDS, recently converted
> the database of the now-defunc SciPy Central into Sphinx-rendered pages:
>
> https://machine-shop.github.io/scipy-central-rescue/


Thanks Jiayue and Stefan for doing this!


> The source materials are at:
>
> https://github.com/machine-shop/scipy-central-rescue
>
> We are happy to move this over to the SciPy repo, and to integrate these
> materials wherever is most applicable.  Let us know!
>

I would suggest to move the whole thing under the scipy Github org as a
historical snapshot, and select the most useful content and put it into
https://github.com/scipy/scipy-cookbook

Cheers,
Ralf



>
> Thanks also to Surya Kasturi who helped us get access to the original
> database.
>
> Best regards,
> St?fan
> _______________________________________________
> SciPy-Dev mailing list
> SciPy-Dev at python.org
> https://mail.python.org/mailman/listinfo/scipy-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scipy-dev/attachments/20180306/fd9f85ae/attachment.html>

From ralf.gommers at gmail.com  Wed Mar  7 02:47:18 2018
From: ralf.gommers at gmail.com (Ralf Gommers)
Date: Tue, 6 Mar 2018 23:47:18 -0800
Subject: [SciPy-Dev] Numba as a dependency for SciPy?
In-Reply-To: <CAHPuU_YNu+3pWLD0=8ONQsVT+h6zAJzRVTru97NHZLJg7_EHTQ@mail.gmail.com>
References: <CABL7CQienTTb9t1ZURWL1P_OQFK45Sb3P+0cW_LF1ufCaQpjQg@mail.gmail.com>
 <CAHPuU_YNu+3pWLD0=8ONQsVT+h6zAJzRVTru97NHZLJg7_EHTQ@mail.gmail.com>
Message-ID: <CABL7CQhxD5b=fKyqshZwS1=t0tJfCY6YVtSLR8NMhZTxdNx1ug@mail.gmail.com>

On Tue, Mar 6, 2018 at 12:02 PM, Tyler Reddy <tyler.je.reddy at gmail.com>
wrote:

> Interesting discussion. Would our plan be to support both side-by-side for
> a while & just see what happens with the evolution of the ecosystem?
>

Indeed. I expect we would get lots of easy wins - stuff that's slow now and
we can speed up by simply adding @jit and that no one wants to port to
Cython because that is a lot of work.


> If there's no clear winner in the short-term would we discourage PRs that
> simply migrate from Cython to numba for say 1.5 x performance increase?
>

Yes that's not the best idea. I'd say start carefully, just adding some jit
calls. Then it's also easy to reverse if something goes wrong.


> What about an algorithm that mixes the two approaches -- some numba and
> some Cython components for whatever reason -- is that discouraged?
>

That seems like a recipe for disaster.


>
> It looks like numba plays ok with airspeed velocity -- presumably mixing
> Cython / numba in our suite will be ok?
>

They're both okay, asv will be completely agnostic to implementation
language.

Ralf


>
>
>
> On 5 March 2018 at 21:06, Ralf Gommers <ralf.gommers at gmail.com> wrote:
>
>> Hi all,
>>
>> Goal of this email: start a discussion to decide whether we'd be okay
>> with relying on Numba as a dependency, now or in 1-2 years' time.
>>
>> Context: in https://github.com/pydata/sparse/issues/126 a discussion is
>> ongoing about whether to adopt Cython or Numba, with Numba being preferred
>> by the majority. That `sparse` package is meant to provide sparse *arrays*
>> that down the line should either be replacing our current sparse *matrices*
>> or at least be integrated in scipy.sparse in addition to them. See
>> https://github.com/scipy/scipy/issues/8162 and
>> https://github.com/hameerabbasi/sparse-ndarray-protocols for more
>> details on that.
>>
>> Also related is the question from Serge Guelton some weeks ago about
>> whether we'd want to rely on Pythran: https://mail.python.org/piperm
>> ail/scipy-dev/2018-January/022325.html
>>
>> On that Pythran thread I commented that we'd want to take these aspects
>> into account:
>> - portability
>> - performance
>> - maturity
>> - maintenance status (active devs, how quick do bugs get fixed after a
>> release with an issue)
>> - ease of use (@jit vs. Pythran comments vs. translate to .pyx syntax)
>> - size of generated binaries
>> - templating support for multiple dtypes
>> - debugging and optimization experience/tool
>>
>> Debugging is one of the ones where I'd say Numba is still worse than
>> Cython, however that's being resolved as we speak:
>> https://github.com/numba/numba/issues/2788
>>
>> One thing I missed in the above list is dependencies: while our use of
>> Cython only adds a build-time dependency, Numba would add a run-time
>> dependency. Given that binary wheels and conda packages for all major
>> platforms are available that's not a showstopper, but it matters.
>>
>> Overall I'd say that:
>> - Numba is better than Cython at: performance, ease of use, size of
>> generated binaries, and templating support for multiple dtypes. Possibly
>> also maintenance status right now.
>> - Numba and Cython are about equally good at portability (I think, not
>> much data about exotic platforms for Numba).
>> - Cython is better than Numba at: maturity, debugging (but not for long
>> anymore probably), dependencies.
>>
>> I'm usually pretty conservative in these things, but considering the
>> above I'm leaning towards saying use of Numba should be allowed in the
>> future. The added run-time dependency is the one major downside that's
>> going to stay, however compared to our Fortran headaches that's a
>> relatively small issue.
>>
>> Thoughts?
>>
>> Cheers,
>> Ralf
>>
>>
>>
>> _______________________________________________
>> SciPy-Dev mailing list
>> SciPy-Dev at python.org
>> https://mail.python.org/mailman/listinfo/scipy-dev
>>
>>
>
> _______________________________________________
> SciPy-Dev mailing list
> SciPy-Dev at python.org
> https://mail.python.org/mailman/listinfo/scipy-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scipy-dev/attachments/20180306/545e4e7c/attachment-0001.html>

From ralf.gommers at gmail.com  Wed Mar  7 03:00:57 2018
From: ralf.gommers at gmail.com (Ralf Gommers)
Date: Wed, 7 Mar 2018 00:00:57 -0800
Subject: [SciPy-Dev] Numba as a dependency for SciPy?
In-Reply-To: <CADv3RKSsrvnQOTtmrhXFdcEqcxG0iMXNXuhcu-4=Cg41u6s3CA@mail.gmail.com>
References: <CABL7CQienTTb9t1ZURWL1P_OQFK45Sb3P+0cW_LF1ufCaQpjQg@mail.gmail.com>
 <CAHPuU_YNu+3pWLD0=8ONQsVT+h6zAJzRVTru97NHZLJg7_EHTQ@mail.gmail.com>
 <CAEfGn+w=3SkYWGoeXoOzCsoCxcoGnW3RJmEnGDUM4cqa4gR1NQ@mail.gmail.com>
 <CAH6Pt5rB2srgYbgQE6qYBOr8PxaLqeD1hb6jnDcV1MP9baewYQ@mail.gmail.com>
 <CADv3RKSsrvnQOTtmrhXFdcEqcxG0iMXNXuhcu-4=Cg41u6s3CA@mail.gmail.com>
Message-ID: <CABL7CQgUng3jgLH8qQO=Wbh=_rVri=fHQ0oKrgbKwMpRsBtB4w@mail.gmail.com>

On Tue, Mar 6, 2018 at 3:32 PM, Stanley Seibert <sseibert at anaconda.com>
wrote:

> (Hi, as someone from the Numba project, my apologies for wading into this
> discussion late.)
>

No apologies needed, the inside perspective is very welcome!


>
> Last time we checked, Numba on ARM was pretty close to working already.
> We have an open PR for ARMv7 that we just ran out of time to finish QA'ing (
> https://github.com/numba/numba/pull/1968) a while ago, and never have
> gotten back to because there was not much demand at the time.  It sounds
> like this is tripping up more people now, so we can take a look again.
>
> One thing that really is unpleasant on ARM is compiling LLVM, so we would
> probably want to make sure we had conda packages and wheels available for
> llvmlite on ARM.  Is there any precedent for posting Linux ARMv7 wheels to
> PyPI?  (For conda, we would just make sure they appear in Jonathan's
> berryconda channel.)
>
> A more difficult platform is POWER8, where we tried to do a port and got
> stuck last year.  Recently, someone has figured out what the issues were,
> and it sounds like several of them stemmed from LLVM bugs in the POWER8
> backend that may or may not be fixed in the next release.  IBM is
> interested in improving the situation, so hopefully that will be sorted out
> soon.
>

Hmm, that does sound like portability is still a significant issue today.
We do sometimes break things for users on POWER8, ARM and similar platforms
because of our lack of CI there, but we do try to keep things working and
accept patches for those platforms.

Ralf



>
> On Tue, Mar 6, 2018 at 4:54 PM, Matthew Brett <matthew.brett at gmail.com>
> wrote:
>
>> On Tue, Mar 6, 2018 at 9:21 PM, CJ Carey <perimosocordiae at gmail.com>
>> wrote:
>> > I think adding a required runtime dependency may be overly restrictive,
>> > given scipy's position near(-ish) the base of the scientific computing
>> > pyramid.
>>
>> Yes, that's a worry I have too.  You may remember Samuel Maybury on
>> the mailing list recently sighing somewhat when he found he had to get
>> numba installed on the Raspberry Pi.  My guess is numba will be fine
>> on the standard platforms and a significant problem on non-standard
>> ones.
>>
>> Cheers,
>>
>> Matthew
>> _______________________________________________
>> SciPy-Dev mailing list
>> SciPy-Dev at python.org
>> https://mail.python.org/mailman/listinfo/scipy-dev
>>
>
>
> _______________________________________________
> SciPy-Dev mailing list
> SciPy-Dev at python.org
> https://mail.python.org/mailman/listinfo/scipy-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scipy-dev/attachments/20180307/ec4dc4a1/attachment.html>

From ralf.gommers at gmail.com  Wed Mar  7 02:56:49 2018
From: ralf.gommers at gmail.com (Ralf Gommers)
Date: Tue, 6 Mar 2018 23:56:49 -0800
Subject: [SciPy-Dev] Numba as a dependency for SciPy?
In-Reply-To: <CAEQ_TvdVe0n+EZyHGyWaBKTs7PQ_=cojqEFhkEgS5njizJCDcw@mail.gmail.com>
References: <CABL7CQienTTb9t1ZURWL1P_OQFK45Sb3P+0cW_LF1ufCaQpjQg@mail.gmail.com>
 <CAHPuU_YNu+3pWLD0=8ONQsVT+h6zAJzRVTru97NHZLJg7_EHTQ@mail.gmail.com>
 <CAEfGn+w=3SkYWGoeXoOzCsoCxcoGnW3RJmEnGDUM4cqa4gR1NQ@mail.gmail.com>
 <CAEQ_TvdVe0n+EZyHGyWaBKTs7PQ_=cojqEFhkEgS5njizJCDcw@mail.gmail.com>
Message-ID: <CABL7CQiXR3aB2-=z6viCuoJpndH9vhYJiGC9OeimnOHmkDYRdw@mail.gmail.com>

On Tue, Mar 6, 2018 at 1:55 PM, Stephan Hoyer <shoyer at gmail.com> wrote:

> Numba does have a mechanism for exporting pre-compiled code:
> http://numba.pydata.org/numba-doc/dev/user/pycc.html
>
> It would be interesting to see if those limitations are flexible enough
> for SciPy.
>

I suspect that (a) we're then going to run into more Numba bugs because
pre-compilation is not well-tested, and (b) we throw away some of the
advantages of Numba, e.g. we then get back the binary size explosion for
multiple dtype templating.


> On Tue, Mar 6, 2018 at 1:21 PM CJ Carey <perimosocordiae at gmail.com> wrote:
>
>> I think adding a required runtime dependency may be overly restrictive,
>> given scipy's position near(-ish) the base of the scientific computing
>> pyramid.
>>
>> Would it be possible to run numba-optimized code on systems with numba
>> installed without impacting "vanilla" users?
>>
>
It's worth thinking about. We could put a jit decorator in scipy._lib that
becomes numba @jit if numba is installed and is do-nothing otherwise.

Ralf
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scipy-dev/attachments/20180306/7cc79461/attachment.html>

From ralf.gommers at gmail.com  Wed Mar  7 03:02:45 2018
From: ralf.gommers at gmail.com (Ralf Gommers)
Date: Wed, 7 Mar 2018 00:02:45 -0800
Subject: [SciPy-Dev] GSoC 2018: Cythonizing
In-Reply-To: <d63a2aa2-4271-0b27-a67c-19fa8ef75ecf@mailbox.org>
References: <3648ed21-1b43-975c-4ca0-05926d683736@mailbox.org>
 <d63a2aa2-4271-0b27-a67c-19fa8ef75ecf@mailbox.org>
Message-ID: <CABL7CQibk4THceGdzcZQpppTp-pWeUXums-g1rdn978Jyrgp0Q@mail.gmail.com>

On Mon, Mar 5, 2018 at 3:02 AM, Lars G. <lagru at mailbox.org> wrote:

> On 28.02.2018 15:32, Lars G. wrote:
> > Dear SciPy devs,
> >
> > I'm currently thinking about an application for this year's GSoC as
> > well. As there already seems to be a large interest in the rotation
> > formalism I'm trying to find another area that matches my interest and
> > skill.
> >
> > I've dug up this proposal in scikit-image from GSoC 2015
> > https://github.com/scikit-image/scikit-image/wiki/GSoC-
> 2015#rewriting-scipyndimage-in-cython
> > and judging by the state of scipy/ndimage/src/ nobody has worked on this
> > proposal yet (feel free to correct me).
> > Alternatively I could imagine something similar for other sub-packages,
> > e.g. scipy/signal which features many source files in C as well.
> >
> > So basically if there is an interest I could try to port C / Python code
> > to Cython. What I would like to know:
> >
> > - Is there an interest? ;)
> > - Is the original proposal in scikit-image still unfinished and are the
> >   potential mentors still interested in mentoring?
> > - If there is a general interest to cythonize C or Python code during a
> >   GSoC project, which parts / sub-packages of SciPy would you priorize?
> >
> > As for my current involvement with SciPy:
> >
> > - I've already added a small function written in Cython
> >   https://github.com/scipy/scipy/pull/8350
> > - as part of a larger PR extending the signal module
> >   https://github.com/scipy/scipy/pull/8264
> >   which will possibly merged this week.
> > - I already cythonized slow parts of the above PR and plan
> >   to add these with new PRs after #8264 is merged.
> >
> > If this receives positive feedback I'd be happy to draft a more complete
> > proposal / application based on the discussion around this.
> >
> > Best regards,
> > Lars
>
> Actually, considering that GSoC should be treated as a full-time job
> during time of coding I must sadly pass on this. However I want to thank
> you all for the feedback already given. I hope its still useful for
> other potential applicants.
>

Thanks Lars. I hope you do stick around part-time!

Cheers,
Ralf
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scipy-dev/attachments/20180307/4aec7888/attachment-0001.html>

From jaime.frio at gmail.com  Wed Mar  7 03:40:19 2018
From: jaime.frio at gmail.com (=?UTF-8?Q?Jaime_Fern=C3=A1ndez_del_R=C3=ADo?=)
Date: Wed, 07 Mar 2018 08:40:19 +0000
Subject: [SciPy-Dev] Numba as a dependency for SciPy?
In-Reply-To: <CABL7CQiXR3aB2-=z6viCuoJpndH9vhYJiGC9OeimnOHmkDYRdw@mail.gmail.com>
References: <CABL7CQienTTb9t1ZURWL1P_OQFK45Sb3P+0cW_LF1ufCaQpjQg@mail.gmail.com>
 <CAHPuU_YNu+3pWLD0=8ONQsVT+h6zAJzRVTru97NHZLJg7_EHTQ@mail.gmail.com>
 <CAEfGn+w=3SkYWGoeXoOzCsoCxcoGnW3RJmEnGDUM4cqa4gR1NQ@mail.gmail.com>
 <CAEQ_TvdVe0n+EZyHGyWaBKTs7PQ_=cojqEFhkEgS5njizJCDcw@mail.gmail.com>
 <CABL7CQiXR3aB2-=z6viCuoJpndH9vhYJiGC9OeimnOHmkDYRdw@mail.gmail.com>
Message-ID: <CAPOWHWnyoNtj+r+nwBnYzGkJxP6nrLWnBwnDnJcnm=XetT-qQg@mail.gmail.com>

On Wed, Mar 7, 2018 at 9:02 AM Ralf Gommers <ralf.gommers at gmail.com> wrote:

>
>
> On Tue, Mar 6, 2018 at 1:55 PM, Stephan Hoyer <shoyer at gmail.com> wrote:
>
>> Numba does have a mechanism for exporting pre-compiled code:
>> http://numba.pydata.org/numba-doc/dev/user/pycc.html
>>
>> It would be interesting to see if those limitations are flexible enough
>> for SciPy.
>>
>
> I suspect that (a) we're then going to run into more Numba bugs because
> pre-compilation is not well-tested, and (b) we throw away some of the
> advantages of Numba, e.g. we then get back the binary size explosion for
> multiple dtype templating.
>
>
>> On Tue, Mar 6, 2018 at 1:21 PM CJ Carey <perimosocordiae at gmail.com>
>> wrote:
>>
>>> I think adding a required runtime dependency may be overly restrictive,
>>> given scipy's position near(-ish) the base of the scientific computing
>>> pyramid.
>>>
>>> Would it be possible to run numba-optimized code on systems with numba
>>> installed without impacting "vanilla" users?
>>>
>>
> It's worth thinking about. We could put a jit decorator in scipy._lib that
> becomes numba @jit if numba is installed and is do-nothing otherwise.
>

I'll admit I have a "fear of the unknown" mistrust of numba, but reading
through this thread I was thinking of something like this as being
something even I would have no problem with.

Juan Luis Cano, who probably reads this, but who I have just in case CCed
in this e-mail, is the author of https://github.com/poliastro/poliastro, a
numerical library that gave up on Fortran/Cython/C by design and embraced
Numba from the start, would be nice to hear his take on making it a
dependency.

Jaime


>
> Ralf
>
> _______________________________________________
> SciPy-Dev mailing list
> SciPy-Dev at python.org
> https://mail.python.org/mailman/listinfo/scipy-dev
>


-- 
(\__/)
( O.o)
( > <) Este es Conejo. Copia a Conejo en tu firma y ay?dale en sus planes
de dominaci?n mundial.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scipy-dev/attachments/20180307/2b0786e5/attachment.html>

From juanlu001 at gmail.com  Wed Mar  7 05:30:47 2018
From: juanlu001 at gmail.com (Juan Luis Cano)
Date: Wed, 7 Mar 2018 11:30:47 +0100
Subject: [SciPy-Dev] Numba as a dependency for SciPy?
In-Reply-To: <CAPOWHWnyoNtj+r+nwBnYzGkJxP6nrLWnBwnDnJcnm=XetT-qQg@mail.gmail.com>
References: <CABL7CQienTTb9t1ZURWL1P_OQFK45Sb3P+0cW_LF1ufCaQpjQg@mail.gmail.com>
 <CAHPuU_YNu+3pWLD0=8ONQsVT+h6zAJzRVTru97NHZLJg7_EHTQ@mail.gmail.com>
 <CAEfGn+w=3SkYWGoeXoOzCsoCxcoGnW3RJmEnGDUM4cqa4gR1NQ@mail.gmail.com>
 <CAEQ_TvdVe0n+EZyHGyWaBKTs7PQ_=cojqEFhkEgS5njizJCDcw@mail.gmail.com>
 <CABL7CQiXR3aB2-=z6viCuoJpndH9vhYJiGC9OeimnOHmkDYRdw@mail.gmail.com>
 <CAPOWHWnyoNtj+r+nwBnYzGkJxP6nrLWnBwnDnJcnm=XetT-qQg@mail.gmail.com>
Message-ID: <CAFmivUXExAdqJ-2vzg3g2gXeVXhbgNkYs2+8mF_LZLd5YGYJ8w@mail.gmail.com>

Hi all,

Thanks Jaime for pinging me, I don't follow scipy-dev regularly.

My frustration with the first release of poliastro, a Python library for
astrodynamics that I created some years ago, was that it mixed MATLAB
(Octave), FORTRAN (all caps) and Python code and that it was a mess to
distribute (this was 2013). I had no experience with C so Cython looked
intimidating to me (I even tried to do some debugging recently and simply
couldn't cope with cython-gdb, see https://groups.google.com/d/
msg/cython-users/SCk2IDG9M5g/muhUhmw9AwAJ). When I saw that I could achieve
a decent performance with numba, I threw away all the FORTRAN and now the
library is pure Python.

For some releases, my approach was exactly what Ralf said: have a `@jit`
decorator that selected what to do depending on whether numba was
installed. This was nice because there was no easy way to install numba
with pip when I started, and in doing so I guaranteed performance with
`conda install poliastro` and a working system with `pip install poliastro`.

https://github.com/poliastro/poliastro/blob/0.6.x/src/poliastro/jit.py

However, now there are wheels on PyPI for both numba and llvmlite, so I
removed the conditional `@jit` and included numba as a required dependency
(except on PyPy):

https://github.com/poliastro/poliastro/blob/master/setup.py#L40

As a last note, numba works beautifully with C code through CFFI:

https://www.anaconda.com/blog/developer-blog/calling-c-
libraries-numba-using-cffi/
http://old.pybonacci.org/2016/02/07/como-crear-extensiones-
en-c-para-python-usando-cffi-y-numba/ [my take, in Spanish]

I understand the concerns about numba because it used to work only with
conda and "it comes from a company" (which some people consider a bad
thing). numba is no doubt a complex project, but the devs are committed to
it and the installation and packaging are now few (specially compared to
the situation we had in 2013). Also, the Julia folks will drive the LLVM
ecosystem to more platforms I would say.

Hope this helps!

On Wed, Mar 7, 2018 at 9:40 AM, Jaime Fern?ndez del R?o <
jaime.frio at gmail.com> wrote:

> On Wed, Mar 7, 2018 at 9:02 AM Ralf Gommers <ralf.gommers at gmail.com>
> wrote:
>
>>
>>
>> On Tue, Mar 6, 2018 at 1:55 PM, Stephan Hoyer <shoyer at gmail.com> wrote:
>>
>>> Numba does have a mechanism for exporting pre-compiled code:
>>> http://numba.pydata.org/numba-doc/dev/user/pycc.html
>>>
>>> It would be interesting to see if those limitations are flexible enough
>>> for SciPy.
>>>
>>
>> I suspect that (a) we're then going to run into more Numba bugs because
>> pre-compilation is not well-tested, and (b) we throw away some of the
>> advantages of Numba, e.g. we then get back the binary size explosion for
>> multiple dtype templating.
>>
>>
>>> On Tue, Mar 6, 2018 at 1:21 PM CJ Carey <perimosocordiae at gmail.com>
>>> wrote:
>>>
>>>> I think adding a required runtime dependency may be overly restrictive,
>>>> given scipy's position near(-ish) the base of the scientific computing
>>>> pyramid.
>>>>
>>>> Would it be possible to run numba-optimized code on systems with numba
>>>> installed without impacting "vanilla" users?
>>>>
>>>
>> It's worth thinking about. We could put a jit decorator in scipy._lib
>> that becomes numba @jit if numba is installed and is do-nothing otherwise.
>>
>
> I'll admit I have a "fear of the unknown" mistrust of numba, but reading
> through this thread I was thinking of something like this as being
> something even I would have no problem with.
>
> Juan Luis Cano, who probably reads this, but who I have just in case CCed
> in this e-mail, is the author of https://github.com/poliastro/poliastro,
> a numerical library that gave up on Fortran/Cython/C by design and embraced
> Numba from the start, would be nice to hear his take on making it a
> dependency.
>
> Jaime
>
>
>>
>> Ralf
>>
>> _______________________________________________
>> SciPy-Dev mailing list
>> SciPy-Dev at python.org
>> https://mail.python.org/mailman/listinfo/scipy-dev
>>
>
>
> --
> (\__/)
> ( O.o)
> ( > <) Este es Conejo. Copia a Conejo en tu firma y ay?dale en sus planes
> de dominaci?n mundial.
>



-- 
Juan Luis Cano
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scipy-dev/attachments/20180307/c480e1bb/attachment-0001.html>

From pav at iki.fi  Wed Mar  7 06:09:48 2018
From: pav at iki.fi (Pauli Virtanen)
Date: Wed, 7 Mar 2018 12:09:48 +0100
Subject: [SciPy-Dev] Numba as a dependency for SciPy?
In-Reply-To: <CABL7CQienTTb9t1ZURWL1P_OQFK45Sb3P+0cW_LF1ufCaQpjQg@mail.gmail.com>
References: <CABL7CQienTTb9t1ZURWL1P_OQFK45Sb3P+0cW_LF1ufCaQpjQg@mail.gmail.com>
Message-ID: <67c2004c-2234-c3fc-9b2a-3704dfd5a450@iki.fi>

Hi,

Ralf Gommers kirjoitti 06.03.2018 klo 05:06:
> Goal of this email: start a discussion to decide whether we'd be okay with
> relying on Numba as a dependency, now or in 1-2 years' time.

I think the main concerns indeed are portability and maturity.

The advantages of Numba on the other hand are clear, as it mostly avoids 
the multiple-language mess.

We might also want to consider using cffi at some point. Currently, we 
basically just have Cython, f2py, and hand-written C code for ffi.

I guess LLVM supports most of the architectures we would be interested 
in, but e.g. the fact that ARM does not work yet is not so nice. 
Moreover, libLLVM is 50+ megabyte blob, but maybe today when people run 
text editors on web browsers instead of vice versa that's not a big deal.

The idea that we could use `scipy._lib.jit` that's either a noop or 
Numba jit does not sound good in practice: for code where the JIT is 
wanted, the performance without it is likely unacceptable. (For PyPy, 
the no-op decorator in principle could be acceptable, but only with 
numpypy which IIUC is not production ready currently, cpyext+numpy 
likely won't be faster than CPython.) Moreover, we presumably would like 
to use features such as `numba.cfunc`. So as I see it, either Numba is a 
hard dependency, or we don't use it.

On maturity: I don't know what is the API stability status for Numba, 
presumably the basic API is stable.

Numba debugging also in my experience has several paper cuts, e.g., the 
compilation and runtime errors are cryptic --- they assume you know how 
numba works, and don't include such niceties as line numbers or useful 
tracebacks, etc. Maybe this will improve in coming years.

	Pauli

From sseibert at anaconda.com  Wed Mar  7 10:12:57 2018
From: sseibert at anaconda.com (Stanley Seibert)
Date: Wed, 7 Mar 2018 09:12:57 -0600
Subject: [SciPy-Dev] Numba as a dependency for SciPy?
In-Reply-To: <67c2004c-2234-c3fc-9b2a-3704dfd5a450@iki.fi>
References: <CABL7CQienTTb9t1ZURWL1P_OQFK45Sb3P+0cW_LF1ufCaQpjQg@mail.gmail.com>
 <67c2004c-2234-c3fc-9b2a-3704dfd5a450@iki.fi>
Message-ID: <CADv3RKSuP762x7vfojkKp4zEv1ouN-=5hDCLJuJ8au-LSOjvPQ@mail.gmail.com>

A couple notes of clarification:

The way llvmlite (Numba's Python wrapper around LLVM) is designed, we
statically link to the LLVM library for several reasons:

   - LLVM changes its C++ API and its IR syntax frequently enough that it
   is never safe to assume Linux distributions ship the version Numba/llvmlite
   requires, or that it was compiled with the options we need.

   - This allows llvmlite to contain only the subset of LLVM that is needed
   for the JIT.  Your 50MB estimate is about right for the install size of
   llvmlite (54 MB on Linux x86_64, 30 MB on macOS 64-bit), but a full install
   of LLVM has more like 165MB of library code.

I just wanted to call all this out in case someone is assuming that Numba
requires an installation of LLVM.  It does not, and in fact deliberately
ignores any system installation of LLVM on purpose.

Regarding the API stability: Numba's API is stable in practice because the
most common interface (the compiler decorators, like @jit) tend to be very
simple.  We are planning to release 1.0 this year, at which point we will
clearly identify in the documentation which APIs and options are stable and
which we consider experimental.  That will also identify a clear point
where we think that Numba is safe for core projects to take as a dependency.

Debugging support continues to be an area we work on, so your criticism is
fair.

On Wed, Mar 7, 2018 at 5:09 AM, Pauli Virtanen <pav at iki.fi> wrote:

> Hi,
>
> Ralf Gommers kirjoitti 06.03.2018 klo 05:06:
>
>> Goal of this email: start a discussion to decide whether we'd be okay with
>> relying on Numba as a dependency, now or in 1-2 years' time.
>>
>
> I think the main concerns indeed are portability and maturity.
>
> The advantages of Numba on the other hand are clear, as it mostly avoids
> the multiple-language mess.
>
> We might also want to consider using cffi at some point. Currently, we
> basically just have Cython, f2py, and hand-written C code for ffi.
>
> I guess LLVM supports most of the architectures we would be interested in,
> but e.g. the fact that ARM does not work yet is not so nice. Moreover,
> libLLVM is 50+ megabyte blob, but maybe today when people run text editors
> on web browsers instead of vice versa that's not a big deal.
>
> The idea that we could use `scipy._lib.jit` that's either a noop or Numba
> jit does not sound good in practice: for code where the JIT is wanted, the
> performance without it is likely unacceptable. (For PyPy, the no-op
> decorator in principle could be acceptable, but only with numpypy which
> IIUC is not production ready currently, cpyext+numpy likely won't be faster
> than CPython.) Moreover, we presumably would like to use features such as
> `numba.cfunc`. So as I see it, either Numba is a hard dependency, or we
> don't use it.
>
> On maturity: I don't know what is the API stability status for Numba,
> presumably the basic API is stable.
>
> Numba debugging also in my experience has several paper cuts, e.g., the
> compilation and runtime errors are cryptic --- they assume you know how
> numba works, and don't include such niceties as line numbers or useful
> tracebacks, etc. Maybe this will improve in coming years.
>
>         Pauli
>
> _______________________________________________
> SciPy-Dev mailing list
> SciPy-Dev at python.org
> https://mail.python.org/mailman/listinfo/scipy-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scipy-dev/attachments/20180307/4620fba8/attachment.html>

From sseibert at anaconda.com  Wed Mar  7 10:34:12 2018
From: sseibert at anaconda.com (Stanley Seibert)
Date: Wed, 7 Mar 2018 09:34:12 -0600
Subject: [SciPy-Dev] Numba as a dependency for SciPy?
In-Reply-To: <CAFmivUXExAdqJ-2vzg3g2gXeVXhbgNkYs2+8mF_LZLd5YGYJ8w@mail.gmail.com>
References: <CABL7CQienTTb9t1ZURWL1P_OQFK45Sb3P+0cW_LF1ufCaQpjQg@mail.gmail.com>
 <CAHPuU_YNu+3pWLD0=8ONQsVT+h6zAJzRVTru97NHZLJg7_EHTQ@mail.gmail.com>
 <CAEfGn+w=3SkYWGoeXoOzCsoCxcoGnW3RJmEnGDUM4cqa4gR1NQ@mail.gmail.com>
 <CAEQ_TvdVe0n+EZyHGyWaBKTs7PQ_=cojqEFhkEgS5njizJCDcw@mail.gmail.com>
 <CABL7CQiXR3aB2-=z6viCuoJpndH9vhYJiGC9OeimnOHmkDYRdw@mail.gmail.com>
 <CAPOWHWnyoNtj+r+nwBnYzGkJxP6nrLWnBwnDnJcnm=XetT-qQg@mail.gmail.com>
 <CAFmivUXExAdqJ-2vzg3g2gXeVXhbgNkYs2+8mF_LZLd5YGYJ8w@mail.gmail.com>
Message-ID: <CADv3RKSWPEs8mXD=HJpZi4Tx+KsLZza1sYRzSquhw1mFVeX-SA@mail.gmail.com>

On Wed, Mar 7, 2018 at 4:30 AM, Juan Luis Cano <juanlu001 at gmail.com> wrote:


> I understand the concerns about numba because it used to work only with
> conda and "it comes from a company" (which some people consider a bad
> thing). numba is no doubt a complex project, but the devs are committed to
> it and the installation and packaging are now few (specially compared to
> the situation we had in 2013). Also, the Julia folks will drive the LLVM
> ecosystem to more platforms I would say.
>

I don't want to strawman the "it comes from a company" argument (which I,
not surprisingly, don't agree with), but I think buried in there is a
concern I do agree with:  It is a risk for a core project to be sponsored
by a single stakeholder.  Companies change strategy, professors lose grant
funding, postdocs move on, and hobbyists burn out and have life-changing
events.  The longevity of the project depends on being able to handle the
disappearance of any core developer (and their sponsor).

So on this front, it is fair to be concerned about Numba, although the
situation is improving.  Right now, the core Numba team consists of 3
Anaconda employees and 2 Intel employees who were recently added for their
automatic multithreading contributions in 2017.  We've been taking notes on
the challenges of onboarding other developers to the code base, and
definitely see the barriers to entry in the code base.  There are things we
will work to improve this year, which will hopefully continue to make it
easier for new developers to get involved.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scipy-dev/attachments/20180307/ee6e6f5c/attachment.html>

From andyfaff at gmail.com  Wed Mar  7 16:22:34 2018
From: andyfaff at gmail.com (Andrew Nelson)
Date: Thu, 8 Mar 2018 08:22:34 +1100
Subject: [SciPy-Dev] Numba as a dependency for SciPy?
In-Reply-To: <CADv3RKSWPEs8mXD=HJpZi4Tx+KsLZza1sYRzSquhw1mFVeX-SA@mail.gmail.com>
References: <CABL7CQienTTb9t1ZURWL1P_OQFK45Sb3P+0cW_LF1ufCaQpjQg@mail.gmail.com>
 <CAHPuU_YNu+3pWLD0=8ONQsVT+h6zAJzRVTru97NHZLJg7_EHTQ@mail.gmail.com>
 <CAEfGn+w=3SkYWGoeXoOzCsoCxcoGnW3RJmEnGDUM4cqa4gR1NQ@mail.gmail.com>
 <CAEQ_TvdVe0n+EZyHGyWaBKTs7PQ_=cojqEFhkEgS5njizJCDcw@mail.gmail.com>
 <CABL7CQiXR3aB2-=z6viCuoJpndH9vhYJiGC9OeimnOHmkDYRdw@mail.gmail.com>
 <CAPOWHWnyoNtj+r+nwBnYzGkJxP6nrLWnBwnDnJcnm=XetT-qQg@mail.gmail.com>
 <CAFmivUXExAdqJ-2vzg3g2gXeVXhbgNkYs2+8mF_LZLd5YGYJ8w@mail.gmail.com>
 <CADv3RKSWPEs8mXD=HJpZi4Tx+KsLZza1sYRzSquhw1mFVeX-SA@mail.gmail.com>
Message-ID: <CAAbtOZcNiq-LJxfneuvQrvENGqsZM_zR=8LFDyTp7=WTihrobA@mail.gmail.com>

I have a few points to chuck in from the sidelines:

1) are there any licence considerations have scipy depend on numba? For
example, I'm thinking along the lines where someone would have to bundle
(or link) numba with scipy/numpy, and be forced to use a specific licence.

2) The size could be an issue for those distributing apps. In a
conda/virtualenv/python install numba is only required to be installed
once. However, I have a few standalone python apps frozen using
PyInstaller. Every time I make one of those it has to freeze numpy, scipy,
etc, into the app structure. PyInstaller strips out a lot of stuff that
isn't necessary, but the size of scipy is still large. Having numbda in
there would increase the size of an app by another 50 Mb if the entirety of
the package was required. If one could statically link in the required
bits, then I suppose it's not so bad.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scipy-dev/attachments/20180308/cf3a7fd8/attachment.html>

From sseibert at anaconda.com  Wed Mar  7 16:57:42 2018
From: sseibert at anaconda.com (Stanley Seibert)
Date: Wed, 7 Mar 2018 15:57:42 -0600
Subject: [SciPy-Dev] Numba as a dependency for SciPy?
In-Reply-To: <CAAbtOZcNiq-LJxfneuvQrvENGqsZM_zR=8LFDyTp7=WTihrobA@mail.gmail.com>
References: <CABL7CQienTTb9t1ZURWL1P_OQFK45Sb3P+0cW_LF1ufCaQpjQg@mail.gmail.com>
 <CAHPuU_YNu+3pWLD0=8ONQsVT+h6zAJzRVTru97NHZLJg7_EHTQ@mail.gmail.com>
 <CAEfGn+w=3SkYWGoeXoOzCsoCxcoGnW3RJmEnGDUM4cqa4gR1NQ@mail.gmail.com>
 <CAEQ_TvdVe0n+EZyHGyWaBKTs7PQ_=cojqEFhkEgS5njizJCDcw@mail.gmail.com>
 <CABL7CQiXR3aB2-=z6viCuoJpndH9vhYJiGC9OeimnOHmkDYRdw@mail.gmail.com>
 <CAPOWHWnyoNtj+r+nwBnYzGkJxP6nrLWnBwnDnJcnm=XetT-qQg@mail.gmail.com>
 <CAFmivUXExAdqJ-2vzg3g2gXeVXhbgNkYs2+8mF_LZLd5YGYJ8w@mail.gmail.com>
 <CADv3RKSWPEs8mXD=HJpZi4Tx+KsLZza1sYRzSquhw1mFVeX-SA@mail.gmail.com>
 <CAAbtOZcNiq-LJxfneuvQrvENGqsZM_zR=8LFDyTp7=WTihrobA@mail.gmail.com>
Message-ID: <CADv3RKQVriBCA-GgPW8CKDfu=PndYFcqCzOibkG1A96BMhSuiA@mail.gmail.com>

On Wed, Mar 7, 2018 at 3:22 PM, Andrew Nelson <andyfaff at gmail.com> wrote:

> 1) are there any licence considerations have scipy depend on numba? For
> example, I'm thinking along the lines where someone would have to bundle
> (or link) numba with scipy/numpy, and be forced to use a specific licence.
>

Numba and llvmlite are BSD licensed, and LLVM uses the University of
Illinois/NCSA Open Source license:

https://opensource.org/licenses/UoI-NCSA.php

...which basically looks like the 3-clause BSD license.  This very similar
to SciPy's own license.

2) The size could be an issue for those distributing apps. In a
> conda/virtualenv/python install numba is only required to be installed
> once. However, I have a few standalone python apps frozen using
> PyInstaller. Every time I make one of those it has to freeze numpy, scipy,
> etc, into the app structure. PyInstaller strips out a lot of stuff that
> isn't necessary, but the size of scipy is still large. Having numbda in
> there would increase the size of an app by another 50 Mb if the entirety of
> the package was required. If one could statically link in the required
> bits, then I suppose it's not so bad.
>

llvmlite is already statically linked to the required bits of LLVM, so
there isn't much chance of reducing the size further.  In the situations
where ahead of time compilation is possible, Numba produces shared
libraries which do not require Numba or llvmlite, but the use cases where
that can be done are limited (at the moment).
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scipy-dev/attachments/20180307/cf2b773e/attachment.html>

From jheemskerk at urthecast.com  Wed Mar  7 17:24:15 2018
From: jheemskerk at urthecast.com (Jordan Heemskerk)
Date: Wed, 7 Mar 2018 22:24:15 +0000
Subject: [SciPy-Dev] Exposing additional window functions in scipy.signal
In-Reply-To: <5D91CFE5-1DEB-49CD-A66E-752EED7F5F5D@urthecast.com>
References: <5D91CFE5-1DEB-49CD-A66E-752EED7F5F5D@urthecast.com>
Message-ID: <5B769115-F750-41FC-89DF-01DE35544EC1@urthecast.com>

Greetings,

I?m hoping to make a small contribution in the scipy.signal.windows module which exposes an existing private function _cos_win as general_cosine and implements a new general_hamming window.

Lately I?ve been coming across these windows in my line of work and having an idiomatic way to produce them through SciPy would be beneficial to me. Although they are not the most common windows, they are used (see PR) and hopefully other members of the community would be interested in having a standard Python implementation.

I?ve drafted the potential changes in PR #8534 (https://github.com/scipy/scipy/pull/8534) and am hoping to recruit some reviewers.

Thanks,

Jordan Heemskerk
Urthecast Inc.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scipy-dev/attachments/20180307/63d16069/attachment-0001.html>

From ralf.gommers at gmail.com  Thu Mar  8 02:04:37 2018
From: ralf.gommers at gmail.com (Ralf Gommers)
Date: Wed, 7 Mar 2018 23:04:37 -0800
Subject: [SciPy-Dev] Numba as a dependency for SciPy?
In-Reply-To: <67c2004c-2234-c3fc-9b2a-3704dfd5a450@iki.fi>
References: <CABL7CQienTTb9t1ZURWL1P_OQFK45Sb3P+0cW_LF1ufCaQpjQg@mail.gmail.com>
 <67c2004c-2234-c3fc-9b2a-3704dfd5a450@iki.fi>
Message-ID: <CABL7CQgsq0MYwy5g=hi4ksJccBOfaXOM9YdvYc9XFAgy8MTZ9Q@mail.gmail.com>

On Wed, Mar 7, 2018 at 3:09 AM, Pauli Virtanen <pav at iki.fi> wrote:

> Hi,
>
> Ralf Gommers kirjoitti 06.03.2018 klo 05:06:
>
>> Goal of this email: start a discussion to decide whether we'd be okay with
>> relying on Numba as a dependency, now or in 1-2 years' time.
>>
>
> I think the main concerns indeed are portability and maturity.
>
> The advantages of Numba on the other hand are clear, as it mostly avoids
> the multiple-language mess.
>
> We might also want to consider using cffi at some point. Currently, we
> basically just have Cython, f2py, and hand-written C code for ffi.
>
> I guess LLVM supports most of the architectures we would be interested in,
> but e.g. the fact that ARM does not work yet is not so nice. Moreover,
> libLLVM is 50+ megabyte blob, but maybe today when people run text editors
> on web browsers instead of vice versa that's not a big deal.
>
> The idea that we could use `scipy._lib.jit` that's either a noop or Numba
> jit does not sound good in practice: for code where the JIT is wanted, the
> performance without it is likely unacceptable.


I don't think it's performance here that matters. For >99.x% of our users
Numba will install just fine, the exceptions being the exotic architectures
like POWER8. On those, having SciPy import and work as expected is enough;
those functions that are implemented with Numba then run slower - that's at
the start <1% of functions we offer for <1% of our userbase.

Also, I don't think performance will necessarily be unacceptable. There are
a bunch of places in the existing code base where we can throw in @jit and
get speedups basically for free. Performance in the noop case will then be
what it is today - not great, but apparently also not enough of a problem
that someone has attempted to go to Cython.

Ralf



> (For PyPy, the no-op decorator in principle could be acceptable, but only
> with numpypy which IIUC is not production ready currently, cpyext+numpy
> likely won't be faster than CPython.) Moreover, we presumably would like to
> use features such as `numba.cfunc`. So as I see it, either Numba is a hard
> dependency, or we don't use it.
>
> On maturity: I don't know what is the API stability status for Numba,
> presumably the basic API is stable.
>
> Numba debugging also in my experience has several paper cuts, e.g., the
> compilation and runtime errors are cryptic --- they assume you know how
> numba works, and don't include such niceties as line numbers or useful
> tracebacks, etc. Maybe this will improve in coming years.
>
>         Pauli
>
> _______________________________________________
> SciPy-Dev mailing list
> SciPy-Dev at python.org
> https://mail.python.org/mailman/listinfo/scipy-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scipy-dev/attachments/20180307/8ffddab5/attachment.html>

From pav at iki.fi  Thu Mar  8 04:38:51 2018
From: pav at iki.fi (Pauli Virtanen)
Date: Thu, 8 Mar 2018 10:38:51 +0100
Subject: [SciPy-Dev] Numba as a dependency for SciPy?
In-Reply-To: <CABL7CQgsq0MYwy5g=hi4ksJccBOfaXOM9YdvYc9XFAgy8MTZ9Q@mail.gmail.com>
References: <CABL7CQienTTb9t1ZURWL1P_OQFK45Sb3P+0cW_LF1ufCaQpjQg@mail.gmail.com>
 <67c2004c-2234-c3fc-9b2a-3704dfd5a450@iki.fi>
 <CABL7CQgsq0MYwy5g=hi4ksJccBOfaXOM9YdvYc9XFAgy8MTZ9Q@mail.gmail.com>
Message-ID: <4cc5f89c-201e-91b1-a849-5990bb05af18@iki.fi>

Ralf Gommers kirjoitti 08.03.2018 klo 08:04:
[clip]
> Also, I don't think performance will necessarily be unacceptable. There are
> a bunch of places in the existing code base where we can throw in @jit and
> get speedups basically for free. Performance in the noop case will then be
> what it is today - not great, but apparently also not enough of a problem
> that someone has attempted to go to Cython.

I guess you agree that Numba would regardless be declared a dependency 
in setup.py? People on unsupported arches can edit it away manually.

For computational tight loops operating on arrays when Numba is used as 
an alternative to Cython/C/Fortran, there probably will be a performance 
hit in the ballpark of 100x.

If we are planning to use numba features more fully, e.g. numba.cfunc 
e.g. to write callback functions, that would also require Numba as a 
hard dependency.

	Pauli

From matthew.brett at gmail.com  Thu Mar  8 05:00:29 2018
From: matthew.brett at gmail.com (Matthew Brett)
Date: Thu, 8 Mar 2018 10:00:29 +0000
Subject: [SciPy-Dev] Numba as a dependency for SciPy?
In-Reply-To: <4cc5f89c-201e-91b1-a849-5990bb05af18@iki.fi>
References: <CABL7CQienTTb9t1ZURWL1P_OQFK45Sb3P+0cW_LF1ufCaQpjQg@mail.gmail.com>
 <67c2004c-2234-c3fc-9b2a-3704dfd5a450@iki.fi>
 <CABL7CQgsq0MYwy5g=hi4ksJccBOfaXOM9YdvYc9XFAgy8MTZ9Q@mail.gmail.com>
 <4cc5f89c-201e-91b1-a849-5990bb05af18@iki.fi>
Message-ID: <CAH6Pt5ryoL7MDSLxWBCxzba9+gvYQpe+bN32-WoQAQ5cJmBc0w@mail.gmail.com>

Hi,

On Thu, Mar 8, 2018 at 9:38 AM, Pauli Virtanen <pav at iki.fi> wrote:
> Ralf Gommers kirjoitti 08.03.2018 klo 08:04:
> [clip]
>>
>> Also, I don't think performance will necessarily be unacceptable. There
>> are
>> a bunch of places in the existing code base where we can throw in @jit and
>> get speedups basically for free. Performance in the noop case will then be
>> what it is today - not great, but apparently also not enough of a problem
>> that someone has attempted to go to Cython.
>
>
> I guess you agree that Numba would regardless be declared a dependency in
> setup.py? People on unsupported arches can edit it away manually.
>
> For computational tight loops operating on arrays when Numba is used as an
> alternative to Cython/C/Fortran, there probably will be a performance hit in
> the ballpark of 100x.
>
> If we are planning to use numba features more fully, e.g. numba.cfunc e.g.
> to write callback functions, that would also require Numba as a hard
> dependency.

If we were at the top of the stack, like pystatsmodels, then this
would be reasonable, but, if we make numba a dependency, that makes
numba a dependency for almost anyone doing scientific computing.  I
think we do have to care about people not running on Intel.   If we
make numba an optional dependency, it gives us an additional
maintenance burden, because we'd have to check for each numba segment,
whether it is going to be disabling for a user without numba.

Is there anything we have at the moment where Cython won't get us into
the ballpark?  If not, my preference would be to wait for a year or
so, to see how things turn out.

Cheers,

Matthew

From gael.varoquaux at normalesup.org  Thu Mar  8 08:39:02 2018
From: gael.varoquaux at normalesup.org (Gael Varoquaux)
Date: Thu, 8 Mar 2018 14:39:02 +0100
Subject: [SciPy-Dev] Numba as a dependency for SciPy?
In-Reply-To: <CABL7CQienTTb9t1ZURWL1P_OQFK45Sb3P+0cW_LF1ufCaQpjQg@mail.gmail.com>
References: <CABL7CQienTTb9t1ZURWL1P_OQFK45Sb3P+0cW_LF1ufCaQpjQg@mail.gmail.com>
Message-ID: <20180308133902.GE1855417@phare.normalesup.org>

Hi everybody,

My take on this issue (I choose to reply to the first mail of the thread,
because I've read the thread, but wasn't sure where to insert a reply).

First, I was excited when I read the beginning of the thread. Numba is a
big promise: it can make our code faster while staying high level.

But following the thread, it seems that it is not prime time yet.

I am not worried at all that it is a "company-driven project". The code
license is right, and I think that there is a real will to grow a
community.

What I am worried about is that it has not reached a sufficient level of
portability. It is important that the base of the pyramid works easily on
less mainstream platforms like ARM. Embedded systems are important for
science, whether it be academic, industrial, or citizen science. From
what I read on this thread, there is still work to be done there,
included at the level of the Python packaging system. Things have
improved hugely in the last few years, so I am extremely hopeful. I worry
about the solution that @jit can fall back to NoOp when numba is not
available, because it means that those in such situations will have
silent slowdowns. I remember a few years ago, it was common for cluster
administrators to install numpy without linking it to an optimized blas
(using the embedded lapack-lite), and on many clusters numpy was
unusable. Computing clusters can also be quite adverse situations for
installation, as some don't have access to Internet, and the libraries
must be installed in an existing Python distribution, to play well with
domain-specific libraries.

Debugging is also a problem, but it seems slightly less of a showstopper
to me.

I would say that we should postpone this decision. Ideally, as a
community, we should help numba and the packaging ecosystem get to a
point where portability is no longer a problem.

Cheers,

Ga?l

On Mon, Mar 05, 2018 at 08:06:11PM -0800, Ralf Gommers wrote:
> Hi all,

> Goal of this email: start a discussion to decide whether we'd be okay with
> relying on Numba as a dependency, now or in 1-2 years' time.

> Context: in https://github.com/pydata/sparse/issues/126 a discussion is ongoing
> about whether to adopt Cython or Numba, with Numba being preferred by the
> majority. That `sparse` package is meant to provide sparse *arrays* that down
> the line should either be replacing our current sparse *matrices* or at least
> be integrated in scipy.sparse in addition to them. See https://github.com/scipy
> /scipy/issues/8162 and https://github.com/hameerabbasi/sparse-ndarray-protocols
> for more details on that.

> Also related is the question from Serge Guelton some weeks ago about whether
> we'd want to rely on Pythran: https://mail.python.org/pipermail/scipy-dev/
> 2018-January/022325.html

> On that Pythran thread I commented that we'd want to take these aspects into
> account:
> - portability
> - performance
> - maturity
> - maintenance status (active devs, how quick do bugs get fixed after a
> release with an issue)
> - ease of use (@jit vs. Pythran comments vs. translate to .pyx syntax)
> - size of generated binaries
> - templating support for multiple dtypes
> - debugging and optimization experience/tool

> Debugging is one of the ones where I'd say Numba is still worse than Cython,
> however that's being resolved as we speak: https://github.com/numba/numba/
> issues/2788

> One thing I missed in the above list is dependencies: while our use of Cython
> only adds a build-time dependency, Numba would add a run-time dependency. Given
> that binary wheels and conda packages for all major platforms are available
> that's not a showstopper, but it matters.

> Overall I'd say that:
> - Numba is better than Cython at: performance, ease of use, size of generated
> binaries, and templating support for multiple dtypes. Possibly also maintenance
> status right now.
> - Numba and Cython are about equally good at portability (I think, not much
> data about exotic platforms for Numba).
> - Cython is better than Numba at: maturity, debugging (but not for long anymore
> probably), dependencies.

> I'm usually pretty conservative in these things, but considering the above I'm
> leaning towards saying use of Numba should be allowed in the future. The added
> run-time dependency is the one major downside that's going to stay, however
> compared to our Fortran headaches that's a relatively small issue.

> Thoughts?

> Cheers,
> Ralf



> _______________________________________________
> SciPy-Dev mailing list
> SciPy-Dev at python.org
> https://mail.python.org/mailman/listinfo/scipy-dev


-- 
    Gael Varoquaux
    Senior Researcher, INRIA Parietal
    NeuroSpin/CEA Saclay , Bat 145, 91191 Gif-sur-Yvette France
    Phone:  ++ 33-1-69-08-79-68
    http://gael-varoquaux.info            http://twitter.com/GaelVaroquaux

From sseibert at anaconda.com  Thu Mar  8 09:44:04 2018
From: sseibert at anaconda.com (Stanley Seibert)
Date: Thu, 8 Mar 2018 08:44:04 -0600
Subject: [SciPy-Dev] Numba as a dependency for SciPy?
In-Reply-To: <CAH6Pt5ryoL7MDSLxWBCxzba9+gvYQpe+bN32-WoQAQ5cJmBc0w@mail.gmail.com>
References: <CABL7CQienTTb9t1ZURWL1P_OQFK45Sb3P+0cW_LF1ufCaQpjQg@mail.gmail.com>
 <67c2004c-2234-c3fc-9b2a-3704dfd5a450@iki.fi>
 <CABL7CQgsq0MYwy5g=hi4ksJccBOfaXOM9YdvYc9XFAgy8MTZ9Q@mail.gmail.com>
 <4cc5f89c-201e-91b1-a849-5990bb05af18@iki.fi>
 <CAH6Pt5ryoL7MDSLxWBCxzba9+gvYQpe+bN32-WoQAQ5cJmBc0w@mail.gmail.com>
Message-ID: <CADv3RKRW=O9QnP9rDhVun694OPzaAq9-Nb1TbPti4FcqCsiZGw@mail.gmail.com>

TBH, I agree with this general sentiment.  This thread has been very
valuable to clarify the Numba team's understanding of what the SciPy
community needs from a compilation solution.  Our trajectory is good, but
we're not quite there yet for a project that needs to be as conservative
about dependencies as SciPy.  We will keep working to get there, though.

However, if someone is interested in trying to implement some SciPy
functions with Numba implementations, there's nothing blocking that work in
a separate repository as an experiment.  (One of the Numba developers has
already named this hypothetical project "Scumba," which I quite like.)  If
anyone does decide to try this, please make sure to ping the Numba
developers on Gitter.  We will learn a great deal from the effort, I think.

On Thu, Mar 8, 2018 at 4:00 AM, Matthew Brett <matthew.brett at gmail.com>
wrote:

> Hi,
>
> On Thu, Mar 8, 2018 at 9:38 AM, Pauli Virtanen <pav at iki.fi> wrote:
> > Ralf Gommers kirjoitti 08.03.2018 klo 08:04:
> > [clip]
> >>
> >> Also, I don't think performance will necessarily be unacceptable. There
> >> are
> >> a bunch of places in the existing code base where we can throw in @jit
> and
> >> get speedups basically for free. Performance in the noop case will then
> be
> >> what it is today - not great, but apparently also not enough of a
> problem
> >> that someone has attempted to go to Cython.
> >
> >
> > I guess you agree that Numba would regardless be declared a dependency in
> > setup.py? People on unsupported arches can edit it away manually.
> >
> > For computational tight loops operating on arrays when Numba is used as
> an
> > alternative to Cython/C/Fortran, there probably will be a performance
> hit in
> > the ballpark of 100x.
> >
> > If we are planning to use numba features more fully, e.g. numba.cfunc
> e.g.
> > to write callback functions, that would also require Numba as a hard
> > dependency.
>
> If we were at the top of the stack, like pystatsmodels, then this
> would be reasonable, but, if we make numba a dependency, that makes
> numba a dependency for almost anyone doing scientific computing.  I
> think we do have to care about people not running on Intel.   If we
> make numba an optional dependency, it gives us an additional
> maintenance burden, because we'd have to check for each numba segment,
> whether it is going to be disabling for a user without numba.
>
> Is there anything we have at the moment where Cython won't get us into
> the ballpark?  If not, my preference would be to wait for a year or
> so, to see how things turn out.
>
> Cheers,
>
> Matthew
> _______________________________________________
> SciPy-Dev mailing list
> SciPy-Dev at python.org
> https://mail.python.org/mailman/listinfo/scipy-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scipy-dev/attachments/20180308/b739c602/attachment.html>

From roberto.bucher at supsi.ch  Thu Mar  8 09:26:49 2018
From: roberto.bucher at supsi.ch (Roberto Bucher)
Date: Thu, 8 Mar 2018 15:26:49 +0100
Subject: [SciPy-Dev] Strange problem with Kanes Method and Lagrange Method
Message-ID: <51413fa9-e66b-d269-9167-70b88eba2e98@supsi.ch>

I have two scripts which implement method to model a wheeled inverted 
pendulum, the first one implementing a Kanes method, the second one with 
Lagrange.

Using Kane, I obtain the right result into the "fr+frstar" variable, but 
after linearization the matrices A and B are completely wrong.

The same behavior seems to append with the Lagrange script, where after 
the Lagrange Method the

"form_lagranges_equations()" output is correct but the generated 
matrices A and B not...

In my opinion the right matrices should be:

A= np.matrix([ [0, 0, 1, 0], [0, 0, 0, 1], 
[((L_w*M_w+L_p*M_p)*g)/(L_w^2*M_w-J_w+J_p), 0, 0, 0], 
[-((L_w*M_w+L_p*M_p)*g)/(L_w^2*M_w-J_w+J_p), 0, 0, 0]] )

B = np.matrix( [0], [0], [-K_t/(L_w^2*M_w-J_w+J_p)], 
[(K_t*L_w^2*M_w+J_p*K_t)/(J_w*L_w^2*M_w-J_w^2+J_p*J_w)] )

Both scripts are attached

Thanks in advance

Best regards

Roberto

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scipy-dev/attachments/20180308/9f8b524f/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: pendolo_kane.py
Type: text/x-python
Size: 2731 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/scipy-dev/attachments/20180308/9f8b524f/attachment.py>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: pendolo_lagrange.py
Type: text/x-python
Size: 2604 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/scipy-dev/attachments/20180308/9f8b524f/attachment-0001.py>

From josh.craig.wilson at gmail.com  Thu Mar  8 11:16:20 2018
From: josh.craig.wilson at gmail.com (Joshua Wilson)
Date: Thu, 8 Mar 2018 10:16:20 -0600
Subject: [SciPy-Dev] Numba as a dependency for SciPy?
In-Reply-To: <CADv3RKRW=O9QnP9rDhVun694OPzaAq9-Nb1TbPti4FcqCsiZGw@mail.gmail.com>
References: <CABL7CQienTTb9t1ZURWL1P_OQFK45Sb3P+0cW_LF1ufCaQpjQg@mail.gmail.com>
 <67c2004c-2234-c3fc-9b2a-3704dfd5a450@iki.fi>
 <CABL7CQgsq0MYwy5g=hi4ksJccBOfaXOM9YdvYc9XFAgy8MTZ9Q@mail.gmail.com>
 <4cc5f89c-201e-91b1-a849-5990bb05af18@iki.fi>
 <CAH6Pt5ryoL7MDSLxWBCxzba9+gvYQpe+bN32-WoQAQ5cJmBc0w@mail.gmail.com>
 <CADv3RKRW=O9QnP9rDhVun694OPzaAq9-Nb1TbPti4FcqCsiZGw@mail.gmail.com>
Message-ID: <CAKFGQGw8rATM5KeDhXhbuQdHFm5FhRpFLf3dEoO+geFotvzPJA@mail.gmail.com>

> if someone is interested in trying to implement some SciPy functions with Numba implementations

When this thread started I actually did just that for a small part of
scipy.special. Code is here:

https://github.com/person142/special

It currently implements `special.loggamma` and some private functions
in special (sinpi, cospi, polynomial evaluation) that are needed to
support it. I'm currently seeing a factor of 2 slowdown and trying to
figure out why, but I'm very interested in figuring out how to close
the speed gap/porting more functions.

- Josh

On Thu, Mar 8, 2018 at 8:44 AM, Stanley Seibert <sseibert at anaconda.com> wrote:
> TBH, I agree with this general sentiment.  This thread has been very
> valuable to clarify the Numba team's understanding of what the SciPy
> community needs from a compilation solution.  Our trajectory is good, but
> we're not quite there yet for a project that needs to be as conservative
> about dependencies as SciPy.  We will keep working to get there, though.
>
> However, if someone is interested in trying to implement some SciPy
> functions with Numba implementations, there's nothing blocking that work in
> a separate repository as an experiment.  (One of the Numba developers has
> already named this hypothetical project "Scumba," which I quite like.)  If
> anyone does decide to try this, please make sure to ping the Numba
> developers on Gitter.  We will learn a great deal from the effort, I think.
>
> On Thu, Mar 8, 2018 at 4:00 AM, Matthew Brett <matthew.brett at gmail.com>
> wrote:
>>
>> Hi,
>>
>> On Thu, Mar 8, 2018 at 9:38 AM, Pauli Virtanen <pav at iki.fi> wrote:
>> > Ralf Gommers kirjoitti 08.03.2018 klo 08:04:
>> > [clip]
>> >>
>> >> Also, I don't think performance will necessarily be unacceptable. There
>> >> are
>> >> a bunch of places in the existing code base where we can throw in @jit
>> >> and
>> >> get speedups basically for free. Performance in the noop case will then
>> >> be
>> >> what it is today - not great, but apparently also not enough of a
>> >> problem
>> >> that someone has attempted to go to Cython.
>> >
>> >
>> > I guess you agree that Numba would regardless be declared a dependency
>> > in
>> > setup.py? People on unsupported arches can edit it away manually.
>> >
>> > For computational tight loops operating on arrays when Numba is used as
>> > an
>> > alternative to Cython/C/Fortran, there probably will be a performance
>> > hit in
>> > the ballpark of 100x.
>> >
>> > If we are planning to use numba features more fully, e.g. numba.cfunc
>> > e.g.
>> > to write callback functions, that would also require Numba as a hard
>> > dependency.
>>
>> If we were at the top of the stack, like pystatsmodels, then this
>> would be reasonable, but, if we make numba a dependency, that makes
>> numba a dependency for almost anyone doing scientific computing.  I
>> think we do have to care about people not running on Intel.   If we
>> make numba an optional dependency, it gives us an additional
>> maintenance burden, because we'd have to check for each numba segment,
>> whether it is going to be disabling for a user without numba.
>>
>> Is there anything we have at the moment where Cython won't get us into
>> the ballpark?  If not, my preference would be to wait for a year or
>> so, to see how things turn out.
>>
>> Cheers,
>>
>> Matthew
>> _______________________________________________
>> SciPy-Dev mailing list
>> SciPy-Dev at python.org
>> https://mail.python.org/mailman/listinfo/scipy-dev
>
>
>
> _______________________________________________
> SciPy-Dev mailing list
> SciPy-Dev at python.org
> https://mail.python.org/mailman/listinfo/scipy-dev
>

From juanlu001 at gmail.com  Thu Mar  8 12:25:48 2018
From: juanlu001 at gmail.com (Juan Luis Cano)
Date: Thu, 8 Mar 2018 18:25:48 +0100
Subject: [SciPy-Dev] Numba as a dependency for SciPy?
In-Reply-To: <CAKFGQGw8rATM5KeDhXhbuQdHFm5FhRpFLf3dEoO+geFotvzPJA@mail.gmail.com>
References: <CABL7CQienTTb9t1ZURWL1P_OQFK45Sb3P+0cW_LF1ufCaQpjQg@mail.gmail.com>
 <67c2004c-2234-c3fc-9b2a-3704dfd5a450@iki.fi>
 <CABL7CQgsq0MYwy5g=hi4ksJccBOfaXOM9YdvYc9XFAgy8MTZ9Q@mail.gmail.com>
 <4cc5f89c-201e-91b1-a849-5990bb05af18@iki.fi>
 <CAH6Pt5ryoL7MDSLxWBCxzba9+gvYQpe+bN32-WoQAQ5cJmBc0w@mail.gmail.com>
 <CADv3RKRW=O9QnP9rDhVun694OPzaAq9-Nb1TbPti4FcqCsiZGw@mail.gmail.com>
 <CAKFGQGw8rATM5KeDhXhbuQdHFm5FhRpFLf3dEoO+geFotvzPJA@mail.gmail.com>
Message-ID: <CAFmivUV6VWVJNU=Uv4Aa21izUuoxARNbdEAC7DfpL+M5CH6=qQ@mail.gmail.com>

On Thu, Mar 8, 2018 at 5:16 PM, Joshua Wilson <josh.craig.wilson at gmail.com>
wrote:

> > if someone is interested in trying to implement some SciPy functions
> with Numba implementations
>
> When this thread started I actually did just that for a small part of
> scipy.special. Code is here:
>

Now that you mention scipy.special, here is another data point with some
promising benchmarks, wrapping CEPHES with CFFI:

https://github.com/poliastro/pycephes#performance


>
> https://github.com/person142/special
>
> It currently implements `special.loggamma` and some private functions
> in special (sinpi, cospi, polynomial evaluation) that are needed to
> support it. I'm currently seeing a factor of 2 slowdown and trying to
> figure out why, but I'm very interested in figuring out how to close
> the speed gap/porting more functions.
>
> - Josh
>
> On Thu, Mar 8, 2018 at 8:44 AM, Stanley Seibert <sseibert at anaconda.com>
> wrote:
> > TBH, I agree with this general sentiment.  This thread has been very
> > valuable to clarify the Numba team's understanding of what the SciPy
> > community needs from a compilation solution.  Our trajectory is good, but
> > we're not quite there yet for a project that needs to be as conservative
> > about dependencies as SciPy.  We will keep working to get there, though.
> >
> > However, if someone is interested in trying to implement some SciPy
> > functions with Numba implementations, there's nothing blocking that work
> in
> > a separate repository as an experiment.  (One of the Numba developers has
> > already named this hypothetical project "Scumba," which I quite like.)
> If
> > anyone does decide to try this, please make sure to ping the Numba
> > developers on Gitter.  We will learn a great deal from the effort, I
> think.
> >
> > On Thu, Mar 8, 2018 at 4:00 AM, Matthew Brett <matthew.brett at gmail.com>
> > wrote:
> >>
> >> Hi,
> >>
> >> On Thu, Mar 8, 2018 at 9:38 AM, Pauli Virtanen <pav at iki.fi> wrote:
> >> > Ralf Gommers kirjoitti 08.03.2018 klo 08:04:
> >> > [clip]
> >> >>
> >> >> Also, I don't think performance will necessarily be unacceptable.
> There
> >> >> are
> >> >> a bunch of places in the existing code base where we can throw in
> @jit
> >> >> and
> >> >> get speedups basically for free. Performance in the noop case will
> then
> >> >> be
> >> >> what it is today - not great, but apparently also not enough of a
> >> >> problem
> >> >> that someone has attempted to go to Cython.
> >> >
> >> >
> >> > I guess you agree that Numba would regardless be declared a dependency
> >> > in
> >> > setup.py? People on unsupported arches can edit it away manually.
> >> >
> >> > For computational tight loops operating on arrays when Numba is used
> as
> >> > an
> >> > alternative to Cython/C/Fortran, there probably will be a performance
> >> > hit in
> >> > the ballpark of 100x.
> >> >
> >> > If we are planning to use numba features more fully, e.g. numba.cfunc
> >> > e.g.
> >> > to write callback functions, that would also require Numba as a hard
> >> > dependency.
> >>
> >> If we were at the top of the stack, like pystatsmodels, then this
> >> would be reasonable, but, if we make numba a dependency, that makes
> >> numba a dependency for almost anyone doing scientific computing.  I
> >> think we do have to care about people not running on Intel.   If we
> >> make numba an optional dependency, it gives us an additional
> >> maintenance burden, because we'd have to check for each numba segment,
> >> whether it is going to be disabling for a user without numba.
> >>
> >> Is there anything we have at the moment where Cython won't get us into
> >> the ballpark?  If not, my preference would be to wait for a year or
> >> so, to see how things turn out.
> >>
> >> Cheers,
> >>
> >> Matthew
> >> _______________________________________________
> >> SciPy-Dev mailing list
> >> SciPy-Dev at python.org
> >> https://mail.python.org/mailman/listinfo/scipy-dev
> >
> >
> >
> > _______________________________________________
> > SciPy-Dev mailing list
> > SciPy-Dev at python.org
> > https://mail.python.org/mailman/listinfo/scipy-dev
> >
> _______________________________________________
> SciPy-Dev mailing list
> SciPy-Dev at python.org
> https://mail.python.org/mailman/listinfo/scipy-dev
>



-- 
Juan Luis Cano
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scipy-dev/attachments/20180308/33497095/attachment.html>

From bwana.marko at yahoo.com  Thu Mar  8 12:43:07 2018
From: bwana.marko at yahoo.com (Mark Mikofski)
Date: Thu, 8 Mar 2018 17:43:07 +0000 (UTC)
Subject: [SciPy-Dev] Numba as a dependency for SciPy?lplupel
In-Reply-To: <CAFmivUV6VWVJNU=Uv4Aa21izUuoxARNbdEAC7DfpL+M5CH6=qQ@mail.gmail.com>
References: <CABL7CQienTTb9t1ZURWL1P_OQFK45Sb3P+0cW_LF1ufCaQpjQg@mail.gmail.com>
 <67c2004c-2234-c3fc-9b2a-3704dfd5a450@iki.fi>
 <CABL7CQgsq0MYwy5g=hi4ksJccBOfaXOM9YdvYc9XFAgy8MTZ9Q@mail.gmail.com>
 <4cc5f89c-201e-91b1-a849-5990bb05af18@iki.fi>
 <CAH6Pt5ryoL7MDSLxWBCxzba9+gvYQpe+bN32-WoQAQ5cJmBc0w@mail.gmail.com>
 <CADv3RKRW=O9QnP9rDhVun694OPzaAq9-Nb1TbPti4FcqCsiZGw@mail.gmail.com>
 <CAKFGQGw8rATM5KeDhXhbuQdHFm5FhRpFLf3dEoO+geFotvzPJA@mail.gmail.com>
 <CAFmivUV6VWVJNU=Uv4Aa21izUuoxARNbdEAC7DfpL+M5CH6=qQ@mail.gmail.com>
Message-ID: <346596039.13511618.1520530987803@mail.yahoo.com>

Here are some benchmarks by Will Holmgren comparing brentq with numba jit variations.?https://github.com/wholmgren/ivnumba and here is a start at a cython API for scipy.optimize zeros?https://github.com/scipy/scipy/pull/8431 although it's not sleeping to apples since I've only done bisection, I'll try to rush to do Brent Q for comparison, but this thread raises a lot of questions and I wonder if I should continue working on cythonizing SciPy.optimize.zeros? Also,?

Sent from Yahoo Mail on Android 
 
  On Thu, Mar 8, 2018 at 9:26 AM, Juan Luis Cano<juanlu001 at gmail.com> wrote:   _______________________________________________
SciPy-Dev mailing list
SciPy-Dev at python.org
https://mail.python.org/mailman/listinfo/scipy-dev
  
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scipy-dev/attachments/20180308/e477684c/attachment.html>
-------------- next part --------------
An embedded and charset-unspecified text was scrubbed...
Name: Untitled
URL: <http://mail.python.org/pipermail/scipy-dev/attachments/20180308/e477684c/attachment.ksh>

From adibhar97 at gmail.com  Thu Mar  8 14:00:10 2018
From: adibhar97 at gmail.com (Aditya Bharti)
Date: Fri, 9 Mar 2018 00:30:10 +0530
Subject: [SciPy-Dev] GSOC 2018 [Starting advice, not proposal]
Message-ID: <CANRzPHXt7Zb+Ev-ySW-g62Y=Jiv6Xj4pezZGwwC5a5C=e6QV9Q@mail.gmail.com>

Hi all,

Goal of this email: To ask for general advice on starting contributions to
scipy for GSOC 2018

To summarise: I am interested in a project which aligns well with my
research and open source experience, but I'm having a little trouble
finding issues to start working on.


I am an undergraduate researcher in Computer Vision, interested in
contributing to scipy. I was specifically looking forward to the second
project on the github project ideas page
<https://github.com/scipy/scipy/wiki/GSoC-2018-project-ideas#rotation-formalism-in-3-dimensions>:
Rotation Formalism in 3 dimensions.

I think I would be a good fit for the project because I have experience
with contributing to open source in general, and specifically to C++ and
python codebases [1] <https://mzl.la/2oV3PtL>. Due to my academic
background, I am already comfortable with 3D rotations being represented as
matrices, Euler angles and vectors as these are used extensively in
Computer Vision. Being a researcher, I will have no problem understanding
academic literature on the subject to extend that knowledge.

The initial setup completed, in order to get started with contributions to
scipy, I have asked around on multiple issues on the main scipy repo with
the 'good first issue' tags, however, after discussion it was realised that
those bugs either require a very deep understanding of the codebase [2
<https://github.com/scipy/scipy/issues/8385#issuecomment-366196873>]
<https://github.com/scipy/scipy/issues/8385#issuecomment-366196873> , or
need to be fixed by the external library author and require no work on
scipy's side [3]
<https://github.com/scipy/scipy/issues/4819#issuecomment-366814744>.

I would greatly appreciate it if you could give me advice getting started
with scipy or just generally making a strong application to scipy.

Thank you,
Aditya Bharti
(Undergraduate Researcher, International Institute of Information
Technology Hyderabad, India)

[1]: https://mzl.la/2oV3PtL
[2]: https://github.com/scipy/scipy/issues/8385#issuecomment-366196873
[3]: https://github.com/scipy/scipy/issues/4819#issuecomment-36681474
<https://github.com/scipy/scipy/issues/4819#issuecomment-366814744>

github: https://github.com/adbugger
bugzilla: https://mzl.la/2FyzaMp

P.S. I apologise in advance if the mailing list is not the correct place to
ask for such advice. I'd be happy to ask on more appropriate channels.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scipy-dev/attachments/20180309/314736eb/attachment-0001.html>

From bwana.marko at yahoo.com  Thu Mar  8 13:43:14 2018
From: bwana.marko at yahoo.com (Mark Mikofski)
Date: Thu, 8 Mar 2018 18:43:14 +0000 (UTC)
Subject: [SciPy-Dev] Numba as a dependency for SciPy?
In-Reply-To: <CAFmivUV6VWVJNU=Uv4Aa21izUuoxARNbdEAC7DfpL+M5CH6=qQ@mail.gmail.com>
References: <CABL7CQienTTb9t1ZURWL1P_OQFK45Sb3P+0cW_LF1ufCaQpjQg@mail.gmail.com>
 <67c2004c-2234-c3fc-9b2a-3704dfd5a450@iki.fi>
 <CABL7CQgsq0MYwy5g=hi4ksJccBOfaXOM9YdvYc9XFAgy8MTZ9Q@mail.gmail.com>
 <4cc5f89c-201e-91b1-a849-5990bb05af18@iki.fi>
 <CAH6Pt5ryoL7MDSLxWBCxzba9+gvYQpe+bN32-WoQAQ5cJmBc0w@mail.gmail.com>
 <CADv3RKRW=O9QnP9rDhVun694OPzaAq9-Nb1TbPti4FcqCsiZGw@mail.gmail.com>
 <CAKFGQGw8rATM5KeDhXhbuQdHFm5FhRpFLf3dEoO+geFotvzPJA@mail.gmail.com>
 <CAFmivUV6VWVJNU=Uv4Aa21izUuoxARNbdEAC7DfpL+M5CH6=qQ@mail.gmail.com>
Message-ID: <857552922.13562871.1520534594681@mail.yahoo.com>

Hi all, there's another benchmark by Will Holmgren of jitting brentq here?https://github.com/wholmgren/ivnumba and cythonizing scipy.optimizing.zeros here?https://github.com/scipy/scipy/pull/8431 but I wonder if I should keep working on this? I believe for looping a lot of speed up can quickly come from numpy. Also?I think cython and numba are both optimized to work with numpy ufuncs.
Sent from Yahoo Mail on Android 
 
  On Thu, Mar 8, 2018 at 9:26 AM, Juan Luis Cano<juanlu001 at gmail.com> wrote:   _______________________________________________
SciPy-Dev mailing list
SciPy-Dev at python.org
https://mail.python.org/mailman/listinfo/scipy-dev
  
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scipy-dev/attachments/20180308/6ab48259/attachment.html>
-------------- next part --------------
An embedded and charset-unspecified text was scrubbed...
Name: Untitled
URL: <http://mail.python.org/pipermail/scipy-dev/attachments/20180308/6ab48259/attachment.ksh>

From pav at iki.fi  Thu Mar  8 12:38:01 2018
From: pav at iki.fi (Pauli Virtanen)
Date: Thu, 8 Mar 2018 18:38:01 +0100
Subject: [SciPy-Dev] Numba as a dependency for SciPy?
In-Reply-To: <CAFmivUV6VWVJNU=Uv4Aa21izUuoxARNbdEAC7DfpL+M5CH6=qQ@mail.gmail.com>
References: <CABL7CQienTTb9t1ZURWL1P_OQFK45Sb3P+0cW_LF1ufCaQpjQg@mail.gmail.com>
 <67c2004c-2234-c3fc-9b2a-3704dfd5a450@iki.fi>
 <CABL7CQgsq0MYwy5g=hi4ksJccBOfaXOM9YdvYc9XFAgy8MTZ9Q@mail.gmail.com>
 <4cc5f89c-201e-91b1-a849-5990bb05af18@iki.fi>
 <CAH6Pt5ryoL7MDSLxWBCxzba9+gvYQpe+bN32-WoQAQ5cJmBc0w@mail.gmail.com>
 <CADv3RKRW=O9QnP9rDhVun694OPzaAq9-Nb1TbPti4FcqCsiZGw@mail.gmail.com>
 <CAKFGQGw8rATM5KeDhXhbuQdHFm5FhRpFLf3dEoO+geFotvzPJA@mail.gmail.com>
 <CAFmivUV6VWVJNU=Uv4Aa21izUuoxARNbdEAC7DfpL+M5CH6=qQ@mail.gmail.com>
Message-ID: <b36755f6-ab20-ca73-8f02-517529f20f3b@iki.fi>

Juan Luis Cano kirjoitti 08.03.2018 klo 18:25:
> Now that you mention scipy.special, here is another data point with some
> promising benchmarks, wrapping CEPHES with CFFI:

Note that this is not apples to apples --- it is comparing scalar 
functions to ufuncs.

Those scipy.special functions are ufuncs, and the only overhead is numpy 
ufunc machinery.

From ralf.gommers at gmail.com  Fri Mar  9 02:02:44 2018
From: ralf.gommers at gmail.com (Ralf Gommers)
Date: Thu, 8 Mar 2018 23:02:44 -0800
Subject: [SciPy-Dev] scipep repo
In-Reply-To: <CABL7CQjW8cAvz3Tqx+kJtOC+5T-JoTvO1fuqLjz985NxJgNsWA@mail.gmail.com>
References: <CAAbtOZcwg4xPzQr1H52XDoaW_rnhArRGMSa6rD3hii0aydsf+Q@mail.gmail.com>
 <CABL7CQiP5Tq+1DeDNSpt9POU1=scB7MLCU2OFYnnQMYWOyHPDA@mail.gmail.com>
 <CAPZ=bb1F8p41Nt1G02_Xe_foPP+sVzF11Bg7AApeLuOvvPB7BA@mail.gmail.com>
 <CABL7CQjW8cAvz3Tqx+kJtOC+5T-JoTvO1fuqLjz985NxJgNsWA@mail.gmail.com>
Message-ID: <CABL7CQhQVxNgKqp774RKtxOm8KTi_ch9iOK26N=Lng0+bUxksQ@mail.gmail.com>

<now reply-all>

On Thu, Mar 8, 2018 at 11:02 PM, Ralf Gommers <ralf.gommers at gmail.com>
wrote:

>
>
> On Fri, Feb 16, 2018 at 9:12 AM, Scott Sievert <sievert.scott at gmail.com>
> wrote:
>
>> What should it be called? "seps" is a bit ambiguous, maybe just go for a
>> long name "scipy-enhancement-proposals"?
>>
>> I?ve been calling it ?scipeps? for ?Scientific Python Enhancement
>> Proposals? because I think it plays well with the acronym ?SciPy?.
>>
>
> Sounds good to me, more concise than mine. Unless there's last minute
> objections, let's go with the `scipeps` repo name and SciPEP acronym'
>
> But the NumPy and Matplotlib have NEPs and MEPs for {NumPy, Matplotlib}
>> Enhancement Proposals respectively. On GitHub, they?re folders (not repos)
>> of names ?neps?[1]
>>
> Things are in flux right now: https://github.com/numpy/neps. I'd like to
> just copy that setup, but it looks like it will take some days to
> stabilize. @Stefan/Jarrod/Nathaniel: are we now keeping neps sources in [1]
> and only have autogenerated commits in https://github.com/numpy/neps ?
>
> Ralf
>
> and ?MEP?[2].
>> Scott
>>
>> [1]: https://github.com/numpy/numpy/tree/master/doc/neps
>> [2]: https://github.com/matplotlib/matplotlib/tree/master/doc/devel/MEP
>>
>> On February 16, 2018 at 9:00:22 AM, Ralf Gommers (ralf.gommers at gmail.com)
>> wrote:
>>
>>
>>
>> On Fri, Feb 16, 2018 at 2:18 AM, Andrew Nelson <andyfaff at gmail.com>
>> wrote:
>>
>>> Scott Sievert and I are writing an enhancement proposal, think of it as
>>> a scipep, to make an Optimizer class. Is it worth keeping such proposals in
>>> it's own repo, e.g. scipy/scipep?
>>>
>>
>> That seems like a good idea. It follows Python, NumPy, etc. - no need to
>> reinvent this wheel.
>>
>>
>>> I'm not sure I can create that myself.
>>>
>>
>> No, only owners can create new repos. What should it be called? "seps" is
>> a bit ambiguous, maybe just go for a long name
>> "scipy-enhancement-proposals"?
>>
>> Ralf
>>
>>
>>>
>>> _______________________________________________
>>> SciPy-Dev mailing list
>>> SciPy-Dev at python.org
>>> https://mail.python.org/mailman/listinfo/scipy-dev
>>>
>>>
>> _______________________________________________
>> SciPy-Dev mailing list
>> SciPy-Dev at python.org
>> https://mail.python.org/mailman/listinfo/scipy-dev
>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scipy-dev/attachments/20180308/09d9e8f3/attachment.html>

From stefanv at berkeley.edu  Fri Mar  9 03:24:27 2018
From: stefanv at berkeley.edu (Stefan van der Walt)
Date: Fri, 9 Mar 2018 00:24:27 -0800
Subject: [SciPy-Dev] scipep repo
In-Reply-To: <CABL7CQhQVxNgKqp774RKtxOm8KTi_ch9iOK26N=Lng0+bUxksQ@mail.gmail.com>
References: <CAAbtOZcwg4xPzQr1H52XDoaW_rnhArRGMSa6rD3hii0aydsf+Q@mail.gmail.com>
 <CABL7CQiP5Tq+1DeDNSpt9POU1=scB7MLCU2OFYnnQMYWOyHPDA@mail.gmail.com>
 <CAPZ=bb1F8p41Nt1G02_Xe_foPP+sVzF11Bg7AApeLuOvvPB7BA@mail.gmail.com>
 <CABL7CQjW8cAvz3Tqx+kJtOC+5T-JoTvO1fuqLjz985NxJgNsWA@mail.gmail.com>
 <CABL7CQhQVxNgKqp774RKtxOm8KTi_ch9iOK26N=Lng0+bUxksQ@mail.gmail.com>
Message-ID: <20180309082427.ulq6xamaypnjzdrh@carbo>

On Thu, 08 Mar 2018 23:02:44 -0800, Ralf Gommers wrote:
> > Things are in flux right now: https://github.com/numpy/neps. I'd like to
> > just copy that setup, but it looks like it will take some days to
> > stabilize. @Stefan/Jarrod/Nathaniel: are we now keeping neps sources in [1]
> > and only have autogenerated commits in https://github.com/numpy/neps
> > ?

The NEP sources are stored in the NumPy source repo:

https://github.com/numpy/numpy/tree/master/doc/neps

These Sphinx docs are rendered to https://github.com/numpy/neps, which
is online at:

http://www.numpy.org/neps/

The CircleCI setup for building & deploying the docs is here:

https://github.com/numpy/numpy/pull/10702

(It's very similar to the one used for SciPy except that, because we
have to push to multiple repos, there's some footwork around handling
SSH keys, and a script for pushing and replacing the built NEP webpage.)

Best regards
St?fan

From pav at iki.fi  Fri Mar  9 04:13:25 2018
From: pav at iki.fi (Pauli Virtanen)
Date: Fri, 9 Mar 2018 10:13:25 +0100
Subject: [SciPy-Dev] scipep repo
In-Reply-To: <20180309082427.ulq6xamaypnjzdrh@carbo>
References: <CAAbtOZcwg4xPzQr1H52XDoaW_rnhArRGMSa6rD3hii0aydsf+Q@mail.gmail.com>
 <CABL7CQiP5Tq+1DeDNSpt9POU1=scB7MLCU2OFYnnQMYWOyHPDA@mail.gmail.com>
 <CAPZ=bb1F8p41Nt1G02_Xe_foPP+sVzF11Bg7AApeLuOvvPB7BA@mail.gmail.com>
 <CABL7CQjW8cAvz3Tqx+kJtOC+5T-JoTvO1fuqLjz985NxJgNsWA@mail.gmail.com>
 <CABL7CQhQVxNgKqp774RKtxOm8KTi_ch9iOK26N=Lng0+bUxksQ@mail.gmail.com>
 <20180309082427.ulq6xamaypnjzdrh@carbo>
Message-ID: <b6933601-0c36-5bbb-35dd-a4f99157e6db@iki.fi>

Stefan van der Walt kirjoitti 09.03.2018 klo 09:24:
[clip]
> (It's very similar to the one used for SciPy except that, because we
> have to push to multiple repos, there's some footwork around handling
> SSH keys, and a script for pushing and replacing the built NEP webpage.)

Readthedocs could be an alternative for a less involved setup --- no 
circleci, no ssh keys, etc.

	Pauli

From ralf.gommers at gmail.com  Sat Mar 10 00:31:58 2018
From: ralf.gommers at gmail.com (Ralf Gommers)
Date: Fri, 9 Mar 2018 21:31:58 -0800
Subject: [SciPy-Dev] Numba as a dependency for SciPy?
In-Reply-To: <4cc5f89c-201e-91b1-a849-5990bb05af18@iki.fi>
References: <CABL7CQienTTb9t1ZURWL1P_OQFK45Sb3P+0cW_LF1ufCaQpjQg@mail.gmail.com>
 <67c2004c-2234-c3fc-9b2a-3704dfd5a450@iki.fi>
 <CABL7CQgsq0MYwy5g=hi4ksJccBOfaXOM9YdvYc9XFAgy8MTZ9Q@mail.gmail.com>
 <4cc5f89c-201e-91b1-a849-5990bb05af18@iki.fi>
Message-ID: <CABL7CQjvxee_8muHKrRUzhk_xN8Ah=7q8pOFHJboOLp=UsBsjQ@mail.gmail.com>

On Thu, Mar 8, 2018 at 1:38 AM, Pauli Virtanen <pav at iki.fi> wrote:

> Ralf Gommers kirjoitti 08.03.2018 klo 08:04:
> [clip]
>
>> Also, I don't think performance will necessarily be unacceptable. There
>> are
>> a bunch of places in the existing code base where we can throw in @jit and
>> get speedups basically for free. Performance in the noop case will then be
>> what it is today - not great, but apparently also not enough of a problem
>> that someone has attempted to go to Cython.
>>
>
> I guess you agree that Numba would regardless be declared a dependency in
> setup.py? People on unsupported arches can edit it away manually.
>

Yes, that's what I was thinking.


>
> For computational tight loops operating on arrays when Numba is used as an
> alternative to Cython/C/Fortran, there probably will be a performance hit
> in the ballpark of 100x.
>
> If we are planning to use numba features more fully, e.g. numba.cfunc e.g.
> to write callback functions, that would also require Numba as a hard
> dependency.


Yeah, looks like it's too early for that due to portability concerns.

Ralf
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scipy-dev/attachments/20180309/3b70403b/attachment.html>

From ralf.gommers at gmail.com  Sat Mar 10 00:34:27 2018
From: ralf.gommers at gmail.com (Ralf Gommers)
Date: Fri, 9 Mar 2018 21:34:27 -0800
Subject: [SciPy-Dev] Numba as a dependency for SciPy?
In-Reply-To: <CAH6Pt5ryoL7MDSLxWBCxzba9+gvYQpe+bN32-WoQAQ5cJmBc0w@mail.gmail.com>
References: <CABL7CQienTTb9t1ZURWL1P_OQFK45Sb3P+0cW_LF1ufCaQpjQg@mail.gmail.com>
 <67c2004c-2234-c3fc-9b2a-3704dfd5a450@iki.fi>
 <CABL7CQgsq0MYwy5g=hi4ksJccBOfaXOM9YdvYc9XFAgy8MTZ9Q@mail.gmail.com>
 <4cc5f89c-201e-91b1-a849-5990bb05af18@iki.fi>
 <CAH6Pt5ryoL7MDSLxWBCxzba9+gvYQpe+bN32-WoQAQ5cJmBc0w@mail.gmail.com>
Message-ID: <CABL7CQhKcBV60vGX2DMfwm66iAS_p_TKSK2DB3==ZxgBF3x5XQ@mail.gmail.com>

On Thu, Mar 8, 2018 at 2:00 AM, Matthew Brett <matthew.brett at gmail.com>
wrote:

> Hi,
>
> On Thu, Mar 8, 2018 at 9:38 AM, Pauli Virtanen <pav at iki.fi> wrote:
> > Ralf Gommers kirjoitti 08.03.2018 klo 08:04:
> > [clip]
> >>
> >> Also, I don't think performance will necessarily be unacceptable. There
> >> are
> >> a bunch of places in the existing code base where we can throw in @jit
> and
> >> get speedups basically for free. Performance in the noop case will then
> be
> >> what it is today - not great, but apparently also not enough of a
> problem
> >> that someone has attempted to go to Cython.
> >
> >
> > I guess you agree that Numba would regardless be declared a dependency in
> > setup.py? People on unsupported arches can edit it away manually.
> >
> > For computational tight loops operating on arrays when Numba is used as
> an
> > alternative to Cython/C/Fortran, there probably will be a performance
> hit in
> > the ballpark of 100x.
> >
> > If we are planning to use numba features more fully, e.g. numba.cfunc
> e.g.
> > to write callback functions, that would also require Numba as a hard
> > dependency.
>
> If we were at the top of the stack, like pystatsmodels, then this
> would be reasonable, but, if we make numba a dependency, that makes
> numba a dependency for almost anyone doing scientific computing.  I
> think we do have to care about people not running on Intel.   If we
> make numba an optional dependency, it gives us an additional
> maintenance burden, because we'd have to check for each numba segment,
> whether it is going to be disabling for a user without numba.
>
> Is there anything we have at the moment where Cython won't get us into
> the ballpark?


That's pretty much an irrelevant question - Cython is typically 2x to 4x
slower so that's in the ballpark, however the problem is ease of use with
Cython is so much worse that a lot of code simply won't get ported and just
stays pure Python.


>   If not, my preference would be to wait for a year or
> so, to see how things turn out.
>

Yes, agreed. The consensus seems to be "no" for now - we can revisit once
ARM & co are better supported.

Ralf
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scipy-dev/attachments/20180309/9a4bd4b0/attachment.html>

From ralf.gommers at gmail.com  Sat Mar 10 00:41:18 2018
From: ralf.gommers at gmail.com (Ralf Gommers)
Date: Fri, 9 Mar 2018 21:41:18 -0800
Subject: [SciPy-Dev] GSOC 2018 [Starting advice, not proposal]
In-Reply-To: <CANRzPHXt7Zb+Ev-ySW-g62Y=Jiv6Xj4pezZGwwC5a5C=e6QV9Q@mail.gmail.com>
References: <CANRzPHXt7Zb+Ev-ySW-g62Y=Jiv6Xj4pezZGwwC5a5C=e6QV9Q@mail.gmail.com>
Message-ID: <CABL7CQiAbOyjut0ENmKuK5FnqZhes8+_hV=O4r6Mmx0bwp-USA@mail.gmail.com>

On Thu, Mar 8, 2018 at 11:00 AM, Aditya Bharti <adibhar97 at gmail.com> wrote:

> Hi all,
>
> Goal of this email: To ask for general advice on starting contributions to
> scipy for GSOC 2018
>

Hi Aditya, welcome!


>
> To summarise: I am interested in a project which aligns well with my
> research and open source experience, but I'm having a little trouble
> finding issues to start working on.
>
>
> I am an undergraduate researcher in Computer Vision, interested in
> contributing to scipy. I was specifically looking forward to the second
> project on the github project ideas page
> <https://github.com/scipy/scipy/wiki/GSoC-2018-project-ideas#rotation-formalism-in-3-dimensions>:
> Rotation Formalism in 3 dimensions.
>
> I think I would be a good fit for the project because I have experience
> with contributing to open source in general, and specifically to C++ and
> python codebases [1] <https://mzl.la/2oV3PtL>. Due to my academic
> background, I am already comfortable with 3D rotations being represented as
> matrices, Euler angles and vectors as these are used extensively in
> Computer Vision. Being a researcher, I will have no problem understanding
> academic literature on the subject to extend that knowledge.
>
> The initial setup completed, in order to get started with contributions to
> scipy, I have asked around on multiple issues on the main scipy repo with
> the 'good first issue' tags, however, after discussion it was realised that
> those bugs either require a very deep understanding of the codebase [2
> <https://github.com/scipy/scipy/issues/8385#issuecomment-366196873>]
> <https://github.com/scipy/scipy/issues/8385#issuecomment-366196873> , or
> need to be fixed by the external library author and require no work on
> scipy's side [3]
> <https://github.com/scipy/scipy/issues/4819#issuecomment-366814744>.
>
> I would greatly appreciate it if you could give me advice getting started
> with scipy or just generally making a strong application to scipy.
>

This email is a good start, clear introduction of yourself and questions.
To get started, I'm thinking scipy.ndimage may be a good module to look at
issues for. Both because it should be relatively familiar given your
background, and because at the moment it's one of the more actively
developed modules. A good first issue could be
https://github.com/scipy/scipy/issues/7453, adding new keywords to make the
API more consistent. https://github.com/scipy/scipy/issues/7334 is a quite
straightforward one, maybe even better to start with that.

Cheers,
Ralf




>
> Thank you,
> Aditya Bharti
> (Undergraduate Researcher, International Institute of Information
> Technology Hyderabad, India)
>
> [1]: https://mzl.la/2oV3PtL
> [2]: https://github.com/scipy/scipy/issues/8385#issuecomment-366196873
> [3]: https://github.com/scipy/scipy/issues/4819#issuecomment-36681474
> <https://github.com/scipy/scipy/issues/4819#issuecomment-366814744>
>
> github: https://github.com/adbugger
> bugzilla: https://mzl.la/2FyzaMp
>
> P.S. I apologise in advance if the mailing list is not the correct place
> to ask for such advice. I'd be happy to ask on more appropriate channels.
>
> _______________________________________________
> SciPy-Dev mailing list
> SciPy-Dev at python.org
> https://mail.python.org/mailman/listinfo/scipy-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scipy-dev/attachments/20180309/413d71ed/attachment-0001.html>

From bwana.marko at yahoo.com  Sat Mar 10 03:12:56 2018
From: bwana.marko at yahoo.com (Mark Mikofski)
Date: Sat, 10 Mar 2018 08:12:56 +0000 (UTC)
Subject: [SciPy-Dev] discussion of options for vectorized
 scipy.optimize.zeros.newton
References: <673991956.14648464.1520669576952.ref@mail.yahoo.com>
Message-ID: <673991956.14648464.1520669576952@mail.yahoo.com>

PR #8357 proposes vectorizing `scipy.optimize.zeros.newton`. Several options have been discussed in the PR, and I would like to get more feedback from the mailing list.??

(1) The vectorized version causes a speed reduction for the scalar case. Benchmarks using tests from `test_zeros.py` show from 3X to 9X slowdown. For example `test_newton` with a scalar runs in 46[us] but as a numpy 1-item array takes 408[us].??

A suggested option was to move the vectorized approach to a private function, to check the inputs in the `newton()`, and call the private vectorized version if the input is not scalar. In this case should we just check the initial guess or all of the extra args? IMO we can't know that all of the args are actually used for calculating the objective, although they probably are, so the only value we know will be used is the initial guess. So if implementing this option, I propose only checking the initial guess.??

(2) Halting conditions are also a concern. The current implementation in PR #8357 will raise a RuntimeWarning and return the current guess, if any derivative becomes zero, even though the other items in the array may not have converged.??

A suggested option is to continue to iterate until either all of the derivatives are zero, or the max newton step of the items with a non-zero derivative is less than the specified tolerance. A RuntimeWarning would still be raised if any of the items had a zero derivative, and the message would provide the indices of those items.??

An additional option I considered was to add a flag to allow the user to specify either early or delayed termination in the situation where zero-derivatives exist. Early termination corresponds to the current implementation in PR #8357 (as of 2f2b3df) and delayed corresponds to the option suggested above.??

(3) There may be other issues and options I haven't considered.??

Thanks for your feedback

?As I breath in, I calm my body, as I breath out, I smile? - Thich_Nhat_Hanh
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scipy-dev/attachments/20180310/589b6cf0/attachment.html>

From matthew.brett at gmail.com  Sat Mar 10 18:06:10 2018
From: matthew.brett at gmail.com (Matthew Brett)
Date: Sat, 10 Mar 2018 23:06:10 +0000
Subject: [SciPy-Dev] Numba as a dependency for SciPy?
In-Reply-To: <CABL7CQhKcBV60vGX2DMfwm66iAS_p_TKSK2DB3==ZxgBF3x5XQ@mail.gmail.com>
References: <CABL7CQienTTb9t1ZURWL1P_OQFK45Sb3P+0cW_LF1ufCaQpjQg@mail.gmail.com>
 <67c2004c-2234-c3fc-9b2a-3704dfd5a450@iki.fi>
 <CABL7CQgsq0MYwy5g=hi4ksJccBOfaXOM9YdvYc9XFAgy8MTZ9Q@mail.gmail.com>
 <4cc5f89c-201e-91b1-a849-5990bb05af18@iki.fi>
 <CAH6Pt5ryoL7MDSLxWBCxzba9+gvYQpe+bN32-WoQAQ5cJmBc0w@mail.gmail.com>
 <CABL7CQhKcBV60vGX2DMfwm66iAS_p_TKSK2DB3==ZxgBF3x5XQ@mail.gmail.com>
Message-ID: <CAH6Pt5pHMeXbhgbbVbeUsey0rrpjh7MGRwm-ou41Q-LUUSmbEg@mail.gmail.com>

On Sat, Mar 10, 2018 at 5:34 AM, Ralf Gommers <ralf.gommers at gmail.com> wrote:
>
>
> On Thu, Mar 8, 2018 at 2:00 AM, Matthew Brett <matthew.brett at gmail.com>
> wrote:
>>
>> Hi,
>>
>> On Thu, Mar 8, 2018 at 9:38 AM, Pauli Virtanen <pav at iki.fi> wrote:
>> > Ralf Gommers kirjoitti 08.03.2018 klo 08:04:
>> > [clip]
>> >>
>> >> Also, I don't think performance will necessarily be unacceptable. There
>> >> are
>> >> a bunch of places in the existing code base where we can throw in @jit
>> >> and
>> >> get speedups basically for free. Performance in the noop case will then
>> >> be
>> >> what it is today - not great, but apparently also not enough of a
>> >> problem
>> >> that someone has attempted to go to Cython.
>> >
>> >
>> > I guess you agree that Numba would regardless be declared a dependency
>> > in
>> > setup.py? People on unsupported arches can edit it away manually.
>> >
>> > For computational tight loops operating on arrays when Numba is used as
>> > an
>> > alternative to Cython/C/Fortran, there probably will be a performance
>> > hit in
>> > the ballpark of 100x.
>> >
>> > If we are planning to use numba features more fully, e.g. numba.cfunc
>> > e.g.
>> > to write callback functions, that would also require Numba as a hard
>> > dependency.
>>
>> If we were at the top of the stack, like pystatsmodels, then this
>> would be reasonable, but, if we make numba a dependency, that makes
>> numba a dependency for almost anyone doing scientific computing.  I
>> think we do have to care about people not running on Intel.   If we
>> make numba an optional dependency, it gives us an additional
>> maintenance burden, because we'd have to check for each numba segment,
>> whether it is going to be disabling for a user without numba.
>>
>> Is there anything we have at the moment where Cython won't get us into
>> the ballpark?
>
>
> That's pretty much an irrelevant question - Cython is typically 2x to 4x
> slower so that's in the ballpark, however the problem is ease of use with
> Cython is so much worse that a lot of code simply won't get ported and just
> stays pure Python.

I must say, I haven't had that experience myself.  Yes, the barrier to
entry is relatively high, but once you're over that, I have found
Cython fairly agreeable to work with.

Cheers,

Matthew

From garyfallidis at gmail.com  Sat Mar 10 20:50:50 2018
From: garyfallidis at gmail.com (Eleftherios Garyfallidis)
Date: Sun, 11 Mar 2018 01:50:50 +0000
Subject: [SciPy-Dev] Numba as a dependency for SciPy?
In-Reply-To: <CAH6Pt5pHMeXbhgbbVbeUsey0rrpjh7MGRwm-ou41Q-LUUSmbEg@mail.gmail.com>
References: <CABL7CQienTTb9t1ZURWL1P_OQFK45Sb3P+0cW_LF1ufCaQpjQg@mail.gmail.com>
 <67c2004c-2234-c3fc-9b2a-3704dfd5a450@iki.fi>
 <CABL7CQgsq0MYwy5g=hi4ksJccBOfaXOM9YdvYc9XFAgy8MTZ9Q@mail.gmail.com>
 <4cc5f89c-201e-91b1-a849-5990bb05af18@iki.fi>
 <CAH6Pt5ryoL7MDSLxWBCxzba9+gvYQpe+bN32-WoQAQ5cJmBc0w@mail.gmail.com>
 <CABL7CQhKcBV60vGX2DMfwm66iAS_p_TKSK2DB3==ZxgBF3x5XQ@mail.gmail.com>
 <CAH6Pt5pHMeXbhgbbVbeUsey0rrpjh7MGRwm-ou41Q-LUUSmbEg@mail.gmail.com>
Message-ID: <CAN6JmWM_RmaJ854SxRb-895Tba-7TN9aVaMWLjbCJJu1Yfx9aQ@mail.gmail.com>

I looked recently quite deeply into Numba as a possible dependency for
DIPY. My opinion is that Numba is not ready to be a dependency for Dipy and
neither numpy/scipy too. Actually, you can often easily beat Numba's speed
with a bit of Cython code. That is because often Numba is not smart enough
in porting some code from Python to C. I think the benchmarks out there are
a bit misleading by comparing only very specific things or very simple
functions. I understand the excitement but we should be realistic too. The
way you write fast code in C can be very different from how you write in
Python. Definitely Numba has incredible potential but we need to give it a
bit more time to mature also we need more generic benchmarks. I would love
to hear what the creators of Numba think too.

Best,
Eleftherios


On Sat, Mar 10, 2018 at 6:07 PM Matthew Brett <matthew.brett at gmail.com>
wrote:

> On Sat, Mar 10, 2018 at 5:34 AM, Ralf Gommers <ralf.gommers at gmail.com>
> wrote:
> >
> >
> > On Thu, Mar 8, 2018 at 2:00 AM, Matthew Brett <matthew.brett at gmail.com>
> > wrote:
> >>
> >> Hi,
> >>
> >> On Thu, Mar 8, 2018 at 9:38 AM, Pauli Virtanen <pav at iki.fi> wrote:
> >> > Ralf Gommers kirjoitti 08.03.2018 klo 08:04:
> >> > [clip]
> >> >>
> >> >> Also, I don't think performance will necessarily be unacceptable.
> There
> >> >> are
> >> >> a bunch of places in the existing code base where we can throw in
> @jit
> >> >> and
> >> >> get speedups basically for free. Performance in the noop case will
> then
> >> >> be
> >> >> what it is today - not great, but apparently also not enough of a
> >> >> problem
> >> >> that someone has attempted to go to Cython.
> >> >
> >> >
> >> > I guess you agree that Numba would regardless be declared a dependency
> >> > in
> >> > setup.py? People on unsupported arches can edit it away manually.
> >> >
> >> > For computational tight loops operating on arrays when Numba is used
> as
> >> > an
> >> > alternative to Cython/C/Fortran, there probably will be a performance
> >> > hit in
> >> > the ballpark of 100x.
> >> >
> >> > If we are planning to use numba features more fully, e.g. numba.cfunc
> >> > e.g.
> >> > to write callback functions, that would also require Numba as a hard
> >> > dependency.
> >>
> >> If we were at the top of the stack, like pystatsmodels, then this
> >> would be reasonable, but, if we make numba a dependency, that makes
> >> numba a dependency for almost anyone doing scientific computing.  I
> >> think we do have to care about people not running on Intel.   If we
> >> make numba an optional dependency, it gives us an additional
> >> maintenance burden, because we'd have to check for each numba segment,
> >> whether it is going to be disabling for a user without numba.
> >>
> >> Is there anything we have at the moment where Cython won't get us into
> >> the ballpark?
> >
> >
> > That's pretty much an irrelevant question - Cython is typically 2x to 4x
> > slower so that's in the ballpark, however the problem is ease of use with
> > Cython is so much worse that a lot of code simply won't get ported and
> just
> > stays pure Python.
>
> I must say, I haven't had that experience myself.  Yes, the barrier to
> entry is relatively high, but once you're over that, I have found
> Cython fairly agreeable to work with.
>
> Cheers,
>
> Matthew
> _______________________________________________
> SciPy-Dev mailing list
> SciPy-Dev at python.org
> https://mail.python.org/mailman/listinfo/scipy-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scipy-dev/attachments/20180311/3b880be2/attachment.html>

From matthew.brett at gmail.com  Sun Mar 11 07:50:57 2018
From: matthew.brett at gmail.com (Matthew Brett)
Date: Sun, 11 Mar 2018 11:50:57 +0000
Subject: [SciPy-Dev] Numba as a dependency for SciPy?
In-Reply-To: <CAN6JmWM_RmaJ854SxRb-895Tba-7TN9aVaMWLjbCJJu1Yfx9aQ@mail.gmail.com>
References: <CABL7CQienTTb9t1ZURWL1P_OQFK45Sb3P+0cW_LF1ufCaQpjQg@mail.gmail.com>
 <67c2004c-2234-c3fc-9b2a-3704dfd5a450@iki.fi>
 <CABL7CQgsq0MYwy5g=hi4ksJccBOfaXOM9YdvYc9XFAgy8MTZ9Q@mail.gmail.com>
 <4cc5f89c-201e-91b1-a849-5990bb05af18@iki.fi>
 <CAH6Pt5ryoL7MDSLxWBCxzba9+gvYQpe+bN32-WoQAQ5cJmBc0w@mail.gmail.com>
 <CABL7CQhKcBV60vGX2DMfwm66iAS_p_TKSK2DB3==ZxgBF3x5XQ@mail.gmail.com>
 <CAH6Pt5pHMeXbhgbbVbeUsey0rrpjh7MGRwm-ou41Q-LUUSmbEg@mail.gmail.com>
 <CAN6JmWM_RmaJ854SxRb-895Tba-7TN9aVaMWLjbCJJu1Yfx9aQ@mail.gmail.com>
Message-ID: <CAH6Pt5oJm2DkseyTc8_nsZFQdVuCm5EFC4MN5dZndANYVyxKBQ@mail.gmail.com>

Hi Eleftherios,

On Sun, Mar 11, 2018 at 1:50 AM, Eleftherios Garyfallidis
<garyfallidis at gmail.com> wrote:
> I looked recently quite deeply into Numba as a possible dependency for DIPY.
> My opinion is that Numba is not ready to be a dependency for Dipy and
> neither numpy/scipy too. Actually, you can often easily beat Numba's speed
> with a bit of Cython code. That is because often Numba is not smart enough
> in porting some code from Python to C. I think the benchmarks out there are
> a bit misleading by comparing only very specific things or very simple
> functions. I understand the excitement but we should be realistic too. The
> way you write fast code in C can be very different from how you write in
> Python. Definitely Numba has incredible potential but we need to give it a
> bit more time to mature also we need more generic benchmarks. I would love
> to hear what the creators of Numba think too.

Did you see Matthew Rocklin's blog post at
https://matthewrocklin.com/blog/work/2018/01/30/the-case-for-numba ?

Have a look at the section near the end "Update from the original
blogpost authors" - your impression reminded me of those comments.

Cheers,

Matthew

From Former at physicist.net  Sun Mar 11 23:57:57 2018
From: Former at physicist.net (Adam)
Date: Sun, 11 Mar 2018 20:57:57 -0700
Subject: [SciPy-Dev] New pull request, fixing ellipj
Message-ID: <1520827077.13359.21.camel@physicist.net>

Hello everybody,

I've submitted a pull request (8548)?to fix?issue 8480. The problem
here was that whenever m was near 1.0 and u>K, ellipj was producing
wildly inaccurate values. (Note that K=ellipk(m), the quarter-period).
?

This was caused by the underlying formula used by the implementation
(16.15.4 from Abramowitz and Stegun) that was only valid when phi<pi/2
or u<K. ?As the original poster (vasilevgmaslov) noted, this method can
still be easily applied if we use the periodic properties of the
elliptic functions to 'unwind' the phi and only apply 16.15.4 to the
principal part or remainder part of phi that is between 0 and pi/2.

It was relatively easy for me implement this method and the results
agree well with what mpmath returns.?

I am pinging the dev list just to see if anyone thinks there is any
major problem with this fix and to request that this fix gets merged
into the main repo.

Thank you!

Adam
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scipy-dev/attachments/20180311/8fac2ff9/attachment.html>

From mikofski at berkeley.edu  Mon Mar 12 12:40:37 2018
From: mikofski at berkeley.edu (Mark Alexander Mikofski)
Date: Mon, 12 Mar 2018 09:40:37 -0700
Subject: [SciPy-Dev] New email address to avoid SPAM filters
Message-ID: <CAEqRcW00Kshm88+AHp8nG8Y7_BMvLtB9CDO1qYfY3=tnezFf5Q@mail.gmail.com>

 Hi All,

It's come to my attention that my mailing list emails *may* have been going
into your SPAM folders. Perhaps that's by design, but just in case, I've
changed my email address to mikofski at berkeley.edu (from
bwana.marko at yahoo.com).

If you want or need to find my emails, you can search the February and
March, 2018, archives by Author for "Mark Mikofski". Other than my
introductions, my other threads were about a proposal to vectorize some of
the scipy.optimize.zeros methods using either NumPy or Cython.

* intro:
https://mail.python.org/pipermail/scipy-dev/2018-February/022448.html
* vectorize newton (gh8357):
https://mail.python.org/pipermail/scipy-dev/2018-March/022621.html
* gh8357: https://github.com/scipy/scipy/pull/8357
* Cythonize optimize.zeros (gh8431):
https://github.com/scipy/scipy/pull/8431

I apologize if *this* is the SPAM message. That would be ironic.

-- 
Mark Mikofski, PhD (2005)
(510) 862-6891
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scipy-dev/attachments/20180312/52bd23f9/attachment.html>

From ilhanpolat at gmail.com  Mon Mar 12 12:55:37 2018
From: ilhanpolat at gmail.com (Ilhan Polat)
Date: Mon, 12 Mar 2018 17:55:37 +0100
Subject: [SciPy-Dev] New email address to avoid SPAM filters
In-Reply-To: <CAEqRcW00Kshm88+AHp8nG8Y7_BMvLtB9CDO1qYfY3=tnezFf5Q@mail.gmail.com>
References: <CAEqRcW00Kshm88+AHp8nG8Y7_BMvLtB9CDO1qYfY3=tnezFf5Q@mail.gmail.com>
Message-ID: <CAEBuzr_Z18O6LtuLEhyUHZ5we6y_gzgrwtkieFMur6TLaw8gxQ@mail.gmail.com>

Indeed I've found them in my spam folder with the notice

"This message has a from address in yahoo.com but has failed yahoo.com's
required tests for authentication."

On Mon, Mar 12, 2018 at 5:40 PM, Mark Alexander Mikofski <
mikofski at berkeley.edu> wrote:

> Hi All,
>
> It's come to my attention that my mailing list emails *may* have been
> going into your SPAM folders. Perhaps that's by design, but just in case,
> I've changed my email address to mikofski at berkeley.edu (from
> bwana.marko at yahoo.com).
>
> If you want or need to find my emails, you can search the February and
> March, 2018, archives by Author for "Mark Mikofski". Other than my
> introductions, my other threads were about a proposal to vectorize some of
> the scipy.optimize.zeros methods using either NumPy or Cython.
>
> * intro: https://mail.python.org/pipermail/scipy-dev/2018-
> February/022448.html
> * vectorize newton (gh8357): https://mail.python.org/
> pipermail/scipy-dev/2018-March/022621.html
> * gh8357: https://github.com/scipy/scipy/pull/8357
> * Cythonize optimize.zeros (gh8431): https://github.com/
> scipy/scipy/pull/8431
>
> I apologize if *this* is the SPAM message. That would be ironic.
>
> --
> Mark Mikofski, PhD (2005)
> (510) 862-6891
>
> _______________________________________________
> SciPy-Dev mailing list
> SciPy-Dev at python.org
> https://mail.python.org/mailman/listinfo/scipy-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scipy-dev/attachments/20180312/06294659/attachment.html>

From charlesr.harris at gmail.com  Mon Mar 12 14:25:42 2018
From: charlesr.harris at gmail.com (Charles R Harris)
Date: Mon, 12 Mar 2018 12:25:42 -0600
Subject: [SciPy-Dev] NumPy 1.14.2 released
Message-ID: <CAB6mnxJxPKPSQK5rJDtJr9eoDj15wquAjG558BCp+JjSBXAz9g@mail.gmail.com>

Hi All,

I am pleased to announce the release of NumPy 1.14.2. This is a bugfix
release for some bugs reported following the 1.14.1 release. The major
problems dealt with are as follows.

   - Residual bugs in the new array printing functionality.
   - Regression resulting in a relocation problem with shared library.
   - Improved PyPy compatibility.

This release supports Python 2.7 and 3.4 - 3.6. Wheels for the release are
available on PyPI. Source tarballs, zipfiles, release notes, and the
changelog are available on github
<https://github.com/numpy/numpy/releases/tag/v1.14.2>. The Python 3.6
wheels available from PIP are built with Python 3.6.2 and should be
compatible with all previous versions of Python 3.6. The source releases
were cythonized with Cython 0.26.1, which is known to *not* support the
upcoming Python 3.7 release. People who wish to run Python 3.7 should check
out the NumPy repo and try building with the, as yet, unreleased master
branch of Cython.

Contributors
============

A total of 4 people contributed to this release.  People with a "+" by their
names contributed a patch for the first time.

* Allan Haldane
* Charles Harris
* Eric Wieser
* Pauli Virtanen

Pull requests merged
====================

A total of 5 pull requests were merged for this release.

* `#10674 <https://github.com/numpy/numpy/pull/10674>`__: BUG: Further
back-compat fix for subclassed array repr
* `#10725 <https://github.com/numpy/numpy/pull/10725>`__: BUG: dragon4
fractional output mode adds too many trailing zeros
* `#10726 <https://github.com/numpy/numpy/pull/10726>`__: BUG: Fix f2py
generated code to work on PyPy
* `#10727 <https://github.com/numpy/numpy/pull/10727>`__: BUG: Fix missing
NPY_VISIBILITY_HIDDEN on npy_longdouble_to_PyLong
* `#10729 <https://github.com/numpy/numpy/pull/10729>`__: DOC: Create
1.14.2 notes and changelog.

Cheers,

Charles Harris
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scipy-dev/attachments/20180312/edb13859/attachment-0001.html>

From nino.krvavica at gmail.com  Wed Mar 14 16:53:43 2018
From: nino.krvavica at gmail.com (Nino Krvavica)
Date: Wed, 14 Mar 2018 21:53:43 +0100
Subject: [SciPy-Dev] Pull request: new function cdf2rdf
Message-ID: <CAHQHC97Ptg87Mryr+VwV=AUvGK2AMY30NzSnkkk2e34GpYbWpg@mail.gmail.com>

Hello everybody,

I've submitted a new pull request #8550 (
https://github.com/scipy/scipy/pull/8550) in which I propose adding a
cdf2rdf function to scipy.linalg. A function of the same name exists in
matlab (https://www.mathworks.com/help/matlab/ref/cdf2rdf.html), and it is
used to convert a complex eigenstructure to a real diagonal block form of
eigenvalues and associated real eigenvectors. The function was tested for
both single arrays and 1e6 stacked arrays, and it works accurately. Any
further comments and/or suggestions are welcome.

Thank you!

Nino

?
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scipy-dev/attachments/20180314/ba2f81c2/attachment.html>

From ndbecker2 at gmail.com  Thu Mar 15 08:31:37 2018
From: ndbecker2 at gmail.com (Neal Becker)
Date: Thu, 15 Mar 2018 12:31:37 +0000
Subject: [SciPy-Dev] Page trying to load scripts from unathenticated sources
Message-ID: <CAG3t+pFe_E_Z6oOMUTbRWC_f44ZEe8-YTf_1M-JNp=sJLRHK+A@mail.gmail.com>

When I visit e.g.,
https://docs.scipy.org/doc/scipy-0.19.0/reference/generated/scipy.sparse.csr_matrix.html

with latest chrome-beta, I see the above warning.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scipy-dev/attachments/20180315/93a95871/attachment.html>

From jkkulick at amazon.de  Thu Mar 15 08:40:02 2018
From: jkkulick at amazon.de (Kulick, Johannes)
Date: Thu, 15 Mar 2018 12:40:02 +0000
Subject: [SciPy-Dev] ENH: softmax
Message-ID: <9d63c4ba33e94ee3b3f421125160d08f@EX13D21EUA003.ant.amazon.com>

I just made a PR to include the softmax function into scipy.special. Feel free to review.

https://github.com/scipy/scipy/pull/8556

Best
Johannes?

Amazon Development Center Germany GmbH
Berlin - Dresden - Aachen
main office: Krausenstr. 38, 10117 Berlin
Geschaeftsfuehrer: Dr. Ralf Herbrich, Christian Schlaeger
Ust-ID: DE289237879
Eingetragen am Amtsgericht Charlottenburg HRB 149173 B

From ralf.gommers at gmail.com  Thu Mar 15 23:13:29 2018
From: ralf.gommers at gmail.com (Ralf Gommers)
Date: Thu, 15 Mar 2018 20:13:29 -0700
Subject: [SciPy-Dev] Page trying to load scripts from unathenticated
 sources
In-Reply-To: <CAG3t+pFe_E_Z6oOMUTbRWC_f44ZEe8-YTf_1M-JNp=sJLRHK+A@mail.gmail.com>
References: <CAG3t+pFe_E_Z6oOMUTbRWC_f44ZEe8-YTf_1M-JNp=sJLRHK+A@mail.gmail.com>
Message-ID: <CABL7CQhAsJf7Sufda9LT2PajPMbOvnXGPed400aVT23vhrYeAg@mail.gmail.com>

On Thu, Mar 15, 2018 at 5:31 AM, Neal Becker <ndbecker2 at gmail.com> wrote:

> When I visit e.g.,
> https://docs.scipy.org/doc/scipy-0.19.0/reference/
> generated/scipy.sparse.csr_matrix.html
>
> with latest chrome-beta, I see the above warning.
>

 Hmm, latest Chrome release doesn't show that. Possibly a bug in Chrome?

Ralf
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scipy-dev/attachments/20180315/c34eca5f/attachment.html>

From jaime.frio at gmail.com  Fri Mar 16 03:06:43 2018
From: jaime.frio at gmail.com (=?UTF-8?Q?Jaime_Fern=C3=A1ndez_del_R=C3=ADo?=)
Date: Fri, 16 Mar 2018 07:06:43 +0000
Subject: [SciPy-Dev] Page trying to load scripts from unathenticated
 sources
In-Reply-To: <CABL7CQhAsJf7Sufda9LT2PajPMbOvnXGPed400aVT23vhrYeAg@mail.gmail.com>
References: <CAG3t+pFe_E_Z6oOMUTbRWC_f44ZEe8-YTf_1M-JNp=sJLRHK+A@mail.gmail.com>
 <CABL7CQhAsJf7Sufda9LT2PajPMbOvnXGPed400aVT23vhrYeAg@mail.gmail.com>
Message-ID: <CAPOWHWnT-FX2oq1m9MoeKy_g3xVw2NUR0d5hFY7ku3VstqLK0w@mail.gmail.com>

I'm seeing it with:

Version 65.0.3325.162 (Official Build) (64-bit)

It's just a tiny icon, so it's easy to oversee.


On Fri, Mar 16, 2018 at 4:13 AM Ralf Gommers <ralf.gommers at gmail.com> wrote:

>
>
> On Thu, Mar 15, 2018 at 5:31 AM, Neal Becker <ndbecker2 at gmail.com> wrote:
>
>> When I visit e.g.,
>>
>> https://docs.scipy.org/doc/scipy-0.19.0/reference/generated/scipy.sparse.csr_matrix.html
>>
>> with latest chrome-beta, I see the above warning.
>>
>
>  Hmm, latest Chrome release doesn't show that. Possibly a bug in Chrome?
>
> Ralf
>
> _______________________________________________
> SciPy-Dev mailing list
> SciPy-Dev at python.org
> https://mail.python.org/mailman/listinfo/scipy-dev
>


-- 
(\__/)
( O.o)
( > <) Este es Conejo. Copia a Conejo en tu firma y ay?dale en sus planes
de dominaci?n mundial.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scipy-dev/attachments/20180316/74eea281/attachment.html>

From jenshnielsen at gmail.com  Fri Mar 16 04:29:15 2018
From: jenshnielsen at gmail.com (Jens Nielsen)
Date: Fri, 16 Mar 2018 08:29:15 +0000
Subject: [SciPy-Dev] Page trying to load scripts from unathenticated
 sources
In-Reply-To: <CAPOWHWnT-FX2oq1m9MoeKy_g3xVw2NUR0d5hFY7ku3VstqLK0w@mail.gmail.com>
References: <CAG3t+pFe_E_Z6oOMUTbRWC_f44ZEe8-YTf_1M-JNp=sJLRHK+A@mail.gmail.com>
 <CABL7CQhAsJf7Sufda9LT2PajPMbOvnXGPed400aVT23vhrYeAg@mail.gmail.com>
 <CAPOWHWnT-FX2oq1m9MoeKy_g3xVw2NUR0d5hFY7ku3VstqLK0w@mail.gmail.com>
Message-ID: <CAM-Pw03Ws4w6N4gh74wwWe5qK_KZH8DjTTj9YetwBs0ddtnMFw@mail.gmail.com>

Looking at the developer console it looks like the issue is likely a
missing https connection for fonts or CSS loaded from Google

"
scipy.sparse.csr_matrix.html:1 Mixed Content: The page at '
https://docs.scipy.org/doc/scipy-0.19.0/reference/generated/scipy.sparse.csr_matrix.html'
was loaded over HTTPS, but requested an insecure stylesheet '
http://fonts.googleapis.com/css?family=Open+Sans'. This request has been
blocked; the content must be served over HTTPS.

MathJax.js?config=TeX-AMS-MML_HTMLorMML:32 WARNING: cdn.mathjax.org has
been retired. Check https://www.mathjax.org/cdn-shutting-down/ for
migration tips.
replaceScript @ MathJax.js?config=TeX-AMS-MML_HTMLorMML:32
"

There is also a mathjax cdn url that looks like it needs to be updated

best
Jens

On Fri, 16 Mar 2018 at 08:13 Jaime Fern?ndez del R?o <jaime.frio at gmail.com>
wrote:

> I'm seeing it with:
>
> Version 65.0.3325.162 (Official Build) (64-bit)
>
> It's just a tiny icon, so it's easy to oversee.
>
>
> On Fri, Mar 16, 2018 at 4:13 AM Ralf Gommers <ralf.gommers at gmail.com>
> wrote:
>
>>
>>
>> On Thu, Mar 15, 2018 at 5:31 AM, Neal Becker <ndbecker2 at gmail.com> wrote:
>>
>>> When I visit e.g.,
>>>
>>> https://docs.scipy.org/doc/scipy-0.19.0/reference/generated/scipy.sparse.csr_matrix.html
>>>
>>> with latest chrome-beta, I see the above warning.
>>>
>>
>>  Hmm, latest Chrome release doesn't show that. Possibly a bug in Chrome?
>>
>> Ralf
>>
>> _______________________________________________
>> SciPy-Dev mailing list
>> SciPy-Dev at python.org
>> https://mail.python.org/mailman/listinfo/scipy-dev
>>
>
>
> --
> (\__/)
> ( O.o)
> ( > <) Este es Conejo. Copia a Conejo en tu firma y ay?dale en sus planes
> de dominaci?n mundial.
> _______________________________________________
> SciPy-Dev mailing list
> SciPy-Dev at python.org
> https://mail.python.org/mailman/listinfo/scipy-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scipy-dev/attachments/20180316/77dc1dc1/attachment.html>

From pav at iki.fi  Fri Mar 16 05:59:36 2018
From: pav at iki.fi (Pauli Virtanen)
Date: Fri, 16 Mar 2018 10:59:36 +0100
Subject: [SciPy-Dev] Page trying to load scripts from unathenticated
 sources
In-Reply-To: <CAM-Pw03Ws4w6N4gh74wwWe5qK_KZH8DjTTj9YetwBs0ddtnMFw@mail.gmail.com>
References: <CAG3t+pFe_E_Z6oOMUTbRWC_f44ZEe8-YTf_1M-JNp=sJLRHK+A@mail.gmail.com>
 <CABL7CQhAsJf7Sufda9LT2PajPMbOvnXGPed400aVT23vhrYeAg@mail.gmail.com>
 <CAPOWHWnT-FX2oq1m9MoeKy_g3xVw2NUR0d5hFY7ku3VstqLK0w@mail.gmail.com>
 <CAM-Pw03Ws4w6N4gh74wwWe5qK_KZH8DjTTj9YetwBs0ddtnMFw@mail.gmail.com>
Message-ID: <1521194376.13406.16.camel@iki.fi>

pe, 2018-03-16 kello 08:29 +0000, Jens Nielsen kirjoitti:
> Looking at the developer console it looks like the issue is likely a
> missing https connection for fonts or CSS loaded from Google
> 
> "
> scipy.sparse.csr_matrix.html:1 Mixed Content: The page at '
> https://docs.scipy.org/doc/scipy-0.19.0/reference/generated/scipy.spa
> rse.csr_matrix.html'
> was loaded over HTTPS, but requested an insecure stylesheet '
> http://fonts.googleapis.com/css?family=Open+Sans'. This request has
> been
> blocked; the content must be served over HTTPS.
> 
> MathJax.js?config=TeX-AMS-MML_HTMLorMML:32 WARNING: cdn.mathjax.org
> has
> been retired. Check https://www.mathjax.org/cdn-shutting-down/ for
> migration tips.
> replaceScript @ MathJax.js?config=TeX-AMS-MML_HTMLorMML:32
> "
> 
> There is also a mathjax cdn url that looks like it needs to be
> updated

These issues have already been fixed.

They are only present in documentation of old releases (e.g. 0.19.0
that you were looking at). We will not rebuild the documentation for
old releases.

-- 
Pauli Virtanen

From pav at iki.fi  Fri Mar 16 05:56:14 2018
From: pav at iki.fi (Pauli Virtanen)
Date: Fri, 16 Mar 2018 10:56:14 +0100
Subject: [SciPy-Dev] Page trying to load scripts from unathenticated
 sources
In-Reply-To: <CAM-Pw03Ws4w6N4gh74wwWe5qK_KZH8DjTTj9YetwBs0ddtnMFw@mail.gmail.com>
References: <CAG3t+pFe_E_Z6oOMUTbRWC_f44ZEe8-YTf_1M-JNp=sJLRHK+A@mail.gmail.com>
 <CABL7CQhAsJf7Sufda9LT2PajPMbOvnXGPed400aVT23vhrYeAg@mail.gmail.com>
 <CAPOWHWnT-FX2oq1m9MoeKy_g3xVw2NUR0d5hFY7ku3VstqLK0w@mail.gmail.com>
 <CAM-Pw03Ws4w6N4gh74wwWe5qK_KZH8DjTTj9YetwBs0ddtnMFw@mail.gmail.com>
Message-ID: <1521194174.13406.15.camel@iki.fi>

pe, 2018-03-16 kello 08:29 +0000, Jens Nielsen kirjoitti:
> was loaded over HTTPS, but requested an insecure stylesheet '
> http://fonts.googleapis.com/css?family=Open+Sans'. This request has
> been
> blocked; the content must be served over HTTPS.
> 
> MathJax.js?config=TeX-AMS-MML_HTMLorMML:32 WARNING: cdn.mathjax.org
> has
> been retired. Check https://www.mathjax.org/cdn-shutting-down/ for
> migration tips.
> replaceScript @ MathJax.js?config=TeX-AMS-MML_HTMLorMML:32
> "
> 
> There is also a mathjax cdn url that looks like it needs to be
> updated

These issues have already been fixed.

They are only present in documentation of old releases (e.g. 0.19.0
that you were looking at). We will not rebuild the documentation for
old releases.

-- 
Pauli Virtanen


From lagru at mailbox.org  Fri Mar 16 06:33:34 2018
From: lagru at mailbox.org (Lars G.)
Date: Fri, 16 Mar 2018 11:33:34 +0100
Subject: [SciPy-Dev] Documentation / tutorial for peak finding in
 scipy.signal
Message-ID: <48693513-3b3f-d6b7-9116-85b0eaf834f2@mailbox.org>

Hello,

recently the 3 user-facing functions `find_peaks`, `peak_prominences`
and `peak_widths` were added to extend SciPy's peak finding capabilities.
https://scipy.github.io/devdocs/signal.html#peak-finding
During that process the idea came up to add a section on peak finding to
the "Signal Processing" tutorial.
https://scipy.github.io/devdocs/tutorial/signal.html

I have tried a few times to come up with a first draft but each time
come away with new questions. Therefore I think it might be reasonable
to have a small discussion beforehand. Some issues / question I'd like
addressed are:

- My first impression of the existing tutorial(s) is that only the basic
concepts and math are covered with maybe one example or visualizing
plot. I think that's a good scope for a tutorial. However I feel that
this concept doesn't translate well to the topic at hand. The concepts
and math behind the 3 new functions are very simple and are already
covered in the relevant docstrings. So I'm not really sure what
additional non-duplicate content would be useful here.

- The new functions are more like building blocks that would be part of
a larger processing chain (filtering, peak finding, peak measurement &
classification). My impression is that examples would be the most useful
here to demonstrate the workflow but would be to code and plot heavy to
fit a tutorial.

- To extend on this: one thing that is very well done in Matlab is that
it provides examples for the usage of each parameter. A good example is
Matlab's version of `find_peaks` itself:
https://de.mathworks.com/help/signal/ref/findpeaks.html?requestedDomain=true
I think this would be really useful for some of the options of the new
functions. E.g. the interplay between `width` and `rel_height` comes to
mind. That would require multiple examples which is to much for a
docstring. At least that is my impression.

- To demonstrate peak finding I would need a signal to analyze. I'm not
sure how to generate complex demo-signals in only a few lines of code
that don't shift the focus of the reader. The best I could come up with
can be seen in `find_peaks` docstring. How is this usually handled?
Would it be okay to import this stuff from `misc`?

- Should this tutorial cover peak finding with wavelet transformation as
well? Reading the roadmap it seems like it's not decided whether wavelet
transformation is within the scope of SciPy. So I'd think this topic
should be left out until this is decided.

Sorry, for the wordy message and I'm in no way an expert in this topic
so please take this with a grain of salt. ;)

Best regards,
Lars

From larson.eric.d at gmail.com  Fri Mar 16 15:11:08 2018
From: larson.eric.d at gmail.com (Eric Larson)
Date: Fri, 16 Mar 2018 15:11:08 -0400
Subject: [SciPy-Dev] Documentation / tutorial for peak finding in
 scipy.signal
In-Reply-To: <48693513-3b3f-d6b7-9116-85b0eaf834f2@mailbox.org>
References: <48693513-3b3f-d6b7-9116-85b0eaf834f2@mailbox.org>
Message-ID: <CAGu2niWWSKqqfKEADetwB2OObWLtyYh_F3+-nxjnkauFHfxHmw@mail.gmail.com>

>
> - The new functions are more like building blocks that would be part of
>
a larger processing chain (filtering, peak finding, peak measurement &
> classification). My impression is that examples would be the most useful
> here to demonstrate the workflow but would be to code and plot heavy to
> fit a tutorial.
>

How about an entry in the scipy cookbook "Signal Processing" or "Other
examples" sections?

http://scipy-cookbook.readthedocs.io/

In the SciPy tutorials area you could do some basic things if you want,
even if they overlap with what is in the docstrings a bit. Then you could
put a link there to the cookbook for a more in-depth example.

- To extend on this: one thing that is very well done in Matlab is that
> it provides examples for the usage of each parameter. A good example is
> Matlab's version of `find_peaks` itself:
> https://de.mathworks.com/help/signal/ref/findpeaks.html?
> requestedDomain=true
> I think this would be really useful for some of the options of the new
> functions. E.g. the interplay between `width` and `rel_height` comes to
> mind. That would require multiple examples which is to much for a
> docstring. At least that is my impression.
>

I see MATLAB's documentation as being roughly equivalent to our docstrings.
So if it seems like the MATLAB docs are well organized and not too long,
the same could go for our docstrings.

I don't see too much of an issue if the `find_peaks` docstring becomes a
bit longer than usual. Most of the "need it now" information is
concentrated at the top (params). The examples and notes come later, and
can be browsed (or not) as necessary by users.

- To demonstrate peak finding I would need a signal to analyze. I'm not
> sure how to generate complex demo-signals in only a few lines of code
> that don't shift the focus of the reader. The best I could come up with
> can be seen in `find_peaks` docstring. How is this usually handled?
> Would it be okay to import this stuff from `misc`?
>

We have `face` and `ascent` in `misc` already for 2D signals. I think it
would be nice to have something like this for 1D signals, assuming we can
find one that won't take up a ton of space (< 500kB maybe, since that's
what ascent takes, and `face` takes up 1.5MB?). We don't want to bloat the
repo, but a reasonable sound choice could be used in many examples.

The most popular "freesound.org" clip is a creative-commons licensed
thunderstorm
recording <https://freesound.org/people/RHumphries/sounds/2523/>, maybe
this could be made mono, truncated, and/or resampled to be sufficiently
small? Other ideas welcome.

- Should this tutorial cover peak finding with wavelet transformation as
> well? Reading the roadmap it seems like it's not decided whether wavelet
> transformation is within the scope of SciPy. So I'd think this topic
> should be left out until this is decided.
>

We can always add this later, so I'd say only include it if you're
motivated to do it.

Eric
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scipy-dev/attachments/20180316/b2b47e73/attachment.html>

From mikofski at berkeley.edu  Sat Mar 17 07:06:58 2018
From: mikofski at berkeley.edu (Mark Alexander Mikofski)
Date: Sat, 17 Mar 2018 04:06:58 -0700
Subject: [SciPy-Dev] vectorized newton proposal
Message-ID: <CAEqRcW0d4GafWZTCJoSaxBdMBGbJhGpOfYw9O3_2byt-G=AdKg@mail.gmail.com>

Hi all,

I've been working on a branch of scipy.optimize.zeros in PR #8357 that uses
NumPy to speed up the newton method if the initial guess is not scalar.

In benchmarks with arrays of 100,000 elements, the NumPy vectorized code is
about 40x faster that looping over the existing code.

If you have time, please consider reviewing the PR. Here's some background

* If the initial guess is scalar, then the old scalar newton code is used

* If the initial guess is *not* scalar then it calls a private method
called "_array_newton"

* there is a new flag called "failure_idx_flag" that in "_array_newton"
appends the indices of failures and indices of zero-derivative items to the
return as a tuple of 3 arrays (results, failures, zero-derivatives),
otherwise just the results are returned

* it only raises a "RuntimeError" if all items fail, otherwise it will
always return an array of results, but it warns a "RuntimeWarning" if there
are any possibly inaccurate guesses that didn't converge because the
derivative was zero (infinite newton step)

Thanks,
Mark

-- 
Mark Mikofski, PhD (2005)
*Fiat Lux*
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scipy-dev/attachments/20180317/712a0465/attachment.html>

From lagru at mailbox.org  Sat Mar 17 09:33:09 2018
From: lagru at mailbox.org (Lars G.)
Date: Sat, 17 Mar 2018 14:33:09 +0100
Subject: [SciPy-Dev] Documentation / tutorial for peak finding in
 scipy.signal
In-Reply-To: <CAGu2niWWSKqqfKEADetwB2OObWLtyYh_F3+-nxjnkauFHfxHmw@mail.gmail.com>
References: <48693513-3b3f-d6b7-9116-85b0eaf834f2@mailbox.org>
 <CAGu2niWWSKqqfKEADetwB2OObWLtyYh_F3+-nxjnkauFHfxHmw@mail.gmail.com>
Message-ID: <13093fec-766d-ba01-26c5-cf57fd5aa2ce@mailbox.org>

On 16.03.2018 20:11, Eric Larson wrote:> How about an entry in the scipy
cookbook "Signal Processing" or "Other
> examples" sections?
> 
> http://scipy-cookbook.readthedocs.io/
> 
> In the SciPy tutorials area you could do some basic things if you want,
> even if they overlap with what is in the docstrings a bit.?Then you
> could put a link there to the cookbook for a more in-depth example.

Sounds reasonable. I'll create a notebook covering a peak finding
example and see how it goes. Maybe something based on beat detection in
an ECG signal.
> We have `face` and `ascent` in `misc` already for 2D signals. I think it
> would be nice to have something like this for 1D signals, assuming we
> can find one that won't take up a ton of space (< 500kB maybe, since
> that's what ascent takes, and `face` takes up 1.5MB?). We don't want to
> bloat the repo, but a reasonable sound choice could be used in many
> examples.
> 
> The most popular "freesound.org <http://freesound.org>" clip is a
> creative-commons licensed thunderstorm recording
> <https://freesound.org/people/RHumphries/sounds/2523/>, maybe this could
> be made mono, truncated, and/or resampled to be sufficiently small?
> Other ideas welcome.

Another source might be the Physionet database has an extensive
collection of medical signals and is licensed with the "OPC Public
Domain Dedication and License" which should be compatible with SciPy's
license.

http://www.physionet.org/physiobank/database/
https://opendatacommons.org/licenses/pddl/1.0/

A biosignal like an ECG could be used to demonstrate
- frequency based filtering (artifact elimination),
- spectral analysis and
- beat detection & classification (peak finding & measurement).
E.g. a simple task could be to extract the heart rate or systolic /
diastolic blood pressure values depending on the signal. This could
provide nice and intuitive "real world" examples.

If you like this idea I'd be happy to create a PR for this.

Best regards,
Lars

From larson.eric.d at gmail.com  Sat Mar 17 14:26:20 2018
From: larson.eric.d at gmail.com (Eric Larson)
Date: Sat, 17 Mar 2018 14:26:20 -0400
Subject: [SciPy-Dev] Documentation / tutorial for peak finding in
 scipy.signal
In-Reply-To: <13093fec-766d-ba01-26c5-cf57fd5aa2ce@mailbox.org>
References: <48693513-3b3f-d6b7-9116-85b0eaf834f2@mailbox.org>
 <CAGu2niWWSKqqfKEADetwB2OObWLtyYh_F3+-nxjnkauFHfxHmw@mail.gmail.com>
 <13093fec-766d-ba01-26c5-cf57fd5aa2ce@mailbox.org>
Message-ID: <CAGu2niWJZ=wHZsWSPB-vNOpqbVwwXTOT=U+4i9YHfyvK-+vJwQ@mail.gmail.com>

I agree an ECG signal is a good choice, and better than my suggestion.
It'll be a lower sample rate than audio, too, so you can get a longer
signal in my proposed 500k limit.

Let's wait a few days to see if others have thoughts on the signal or its
size limit, and if there are no complaints, go for it!

Eric


On Sat, Mar 17, 2018 at 9:33 AM, Lars G. <lagru at mailbox.org> wrote:

> On 16.03.2018 20:11, Eric Larson wrote:> How about an entry in the scipy
> cookbook "Signal Processing" or "Other
> > examples" sections?
> >
> > http://scipy-cookbook.readthedocs.io/
> >
> > In the SciPy tutorials area you could do some basic things if you want,
> > even if they overlap with what is in the docstrings a bit. Then you
> > could put a link there to the cookbook for a more in-depth example.
>
> Sounds reasonable. I'll create a notebook covering a peak finding
> example and see how it goes. Maybe something based on beat detection in
> an ECG signal.
> > We have `face` and `ascent` in `misc` already for 2D signals. I think it
> > would be nice to have something like this for 1D signals, assuming we
> > can find one that won't take up a ton of space (< 500kB maybe, since
> > that's what ascent takes, and `face` takes up 1.5MB?). We don't want to
> > bloat the repo, but a reasonable sound choice could be used in many
> > examples.
> >
> > The most popular "freesound.org <http://freesound.org>" clip is a
> > creative-commons licensed thunderstorm recording
> > <https://freesound.org/people/RHumphries/sounds/2523/>, maybe this could
> > be made mono, truncated, and/or resampled to be sufficiently small?
> > Other ideas welcome.
>
> Another source might be the Physionet database has an extensive
> collection of medical signals and is licensed with the "OPC Public
> Domain Dedication and License" which should be compatible with SciPy's
> license.
>
> http://www.physionet.org/physiobank/database/
> https://opendatacommons.org/licenses/pddl/1.0/
>
> A biosignal like an ECG could be used to demonstrate
> - frequency based filtering (artifact elimination),
> - spectral analysis and
> - beat detection & classification (peak finding & measurement).
> E.g. a simple task could be to extract the heart rate or systolic /
> diastolic blood pressure values depending on the signal. This could
> provide nice and intuitive "real world" examples.
>
> If you like this idea I'd be happy to create a PR for this.
>
> Best regards,
> Lars
> _______________________________________________
> SciPy-Dev mailing list
> SciPy-Dev at python.org
> https://mail.python.org/mailman/listinfo/scipy-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scipy-dev/attachments/20180317/38e37a39/attachment.html>

From bennet at umich.edu  Sat Mar 17 14:39:12 2018
From: bennet at umich.edu (Bennet Fauber)
Date: Sat, 17 Mar 2018 14:39:12 -0400
Subject: [SciPy-Dev] Documentation / tutorial for peak finding in
 scipy.signal
In-Reply-To: <CAGu2niWJZ=wHZsWSPB-vNOpqbVwwXTOT=U+4i9YHfyvK-+vJwQ@mail.gmail.com>
References: <48693513-3b3f-d6b7-9116-85b0eaf834f2@mailbox.org>
 <CAGu2niWWSKqqfKEADetwB2OObWLtyYh_F3+-nxjnkauFHfxHmw@mail.gmail.com>
 <13093fec-766d-ba01-26c5-cf57fd5aa2ce@mailbox.org>
 <CAGu2niWJZ=wHZsWSPB-vNOpqbVwwXTOT=U+4i9YHfyvK-+vJwQ@mail.gmail.com>
Message-ID: <CAB2ovou2nUbUtV58KXGWk8E5uBv2-3upBJo8qviXtPvybNmPCQ@mail.gmail.com>

This sounds like it would be a very good example.


On Sat, Mar 17, 2018 at 2:26 PM, Eric Larson <larson.eric.d at gmail.com> wrote:
> I agree an ECG signal is a good choice, and better than my suggestion. It'll
> be a lower sample rate than audio, too, so you can get a longer signal in my
> proposed 500k limit.
>
> Let's wait a few days to see if others have thoughts on the signal or its
> size limit, and if there are no complaints, go for it!
>
> Eric
>
>
> On Sat, Mar 17, 2018 at 9:33 AM, Lars G. <lagru at mailbox.org> wrote:
>>
>> On 16.03.2018 20:11, Eric Larson wrote:> How about an entry in the scipy
>> cookbook "Signal Processing" or "Other
>> > examples" sections?
>> >
>> > http://scipy-cookbook.readthedocs.io/
>> >
>> > In the SciPy tutorials area you could do some basic things if you want,
>> > even if they overlap with what is in the docstrings a bit. Then you
>> > could put a link there to the cookbook for a more in-depth example.
>>
>> Sounds reasonable. I'll create a notebook covering a peak finding
>> example and see how it goes. Maybe something based on beat detection in
>> an ECG signal.
>> > We have `face` and `ascent` in `misc` already for 2D signals. I think it
>> > would be nice to have something like this for 1D signals, assuming we
>> > can find one that won't take up a ton of space (< 500kB maybe, since
>> > that's what ascent takes, and `face` takes up 1.5MB?). We don't want to
>> > bloat the repo, but a reasonable sound choice could be used in many
>> > examples.
>> >
>> > The most popular "freesound.org <http://freesound.org>" clip is a
>> > creative-commons licensed thunderstorm recording
>> > <https://freesound.org/people/RHumphries/sounds/2523/>, maybe this could
>> > be made mono, truncated, and/or resampled to be sufficiently small?
>> > Other ideas welcome.
>>
>> Another source might be the Physionet database has an extensive
>> collection of medical signals and is licensed with the "OPC Public
>> Domain Dedication and License" which should be compatible with SciPy's
>> license.
>>
>> http://www.physionet.org/physiobank/database/
>> https://opendatacommons.org/licenses/pddl/1.0/
>>
>> A biosignal like an ECG could be used to demonstrate
>> - frequency based filtering (artifact elimination),
>> - spectral analysis and
>> - beat detection & classification (peak finding & measurement).
>> E.g. a simple task could be to extract the heart rate or systolic /
>> diastolic blood pressure values depending on the signal. This could
>> provide nice and intuitive "real world" examples.
>>
>> If you like this idea I'd be happy to create a PR for this.
>>
>> Best regards,
>> Lars
>> _______________________________________________
>> SciPy-Dev mailing list
>> SciPy-Dev at python.org
>> https://mail.python.org/mailman/listinfo/scipy-dev
>
>
>
> _______________________________________________
> SciPy-Dev mailing list
> SciPy-Dev at python.org
> https://mail.python.org/mailman/listinfo/scipy-dev
>

From ralf.gommers at gmail.com  Sat Mar 17 15:59:44 2018
From: ralf.gommers at gmail.com (Ralf Gommers)
Date: Sat, 17 Mar 2018 12:59:44 -0700
Subject: [SciPy-Dev] Documentation / tutorial for peak finding in
 scipy.signal
In-Reply-To: <CAB2ovou2nUbUtV58KXGWk8E5uBv2-3upBJo8qviXtPvybNmPCQ@mail.gmail.com>
References: <48693513-3b3f-d6b7-9116-85b0eaf834f2@mailbox.org>
 <CAGu2niWWSKqqfKEADetwB2OObWLtyYh_F3+-nxjnkauFHfxHmw@mail.gmail.com>
 <13093fec-766d-ba01-26c5-cf57fd5aa2ce@mailbox.org>
 <CAGu2niWJZ=wHZsWSPB-vNOpqbVwwXTOT=U+4i9YHfyvK-+vJwQ@mail.gmail.com>
 <CAB2ovou2nUbUtV58KXGWk8E5uBv2-3upBJo8qviXtPvybNmPCQ@mail.gmail.com>
Message-ID: <CABL7CQgM0Taey5p9+H2Rx4epgsUBLBBPHi7zDkTtZq-2ECXT1w@mail.gmail.com>

On Sat, Mar 17, 2018 at 11:39 AM, Bennet Fauber <bennet at umich.edu> wrote:

> This sounds like it would be a very good example.
>


agreed

>
>
> On Sat, Mar 17, 2018 at 2:26 PM, Eric Larson <larson.eric.d at gmail.com>
> wrote:
> > I agree an ECG signal is a good choice, and better than my suggestion.
> It'll
> > be a lower sample rate than audio, too, so you can get a longer signal
> in my
> > proposed 500k limit.
> >
> > Let's wait a few days to see if others have thoughts on the signal or its
> > size limit, and if there are no complaints, go for it!
>

If you can get away with a smaller size that would be useful, but I'm okay
with up to 500 kb.

Ralf



> >
> > Eric
> >
> >
> > On Sat, Mar 17, 2018 at 9:33 AM, Lars G. <lagru at mailbox.org> wrote:
> >>
> >> On 16.03.2018 20:11, Eric Larson wrote:> How about an entry in the scipy
> >> cookbook "Signal Processing" or "Other
> >> > examples" sections?
> >> >
> >> > http://scipy-cookbook.readthedocs.io/
> >> >
> >> > In the SciPy tutorials area you could do some basic things if you
> want,
> >> > even if they overlap with what is in the docstrings a bit. Then you
> >> > could put a link there to the cookbook for a more in-depth example.
> >>
> >> Sounds reasonable. I'll create a notebook covering a peak finding
> >> example and see how it goes. Maybe something based on beat detection in
> >> an ECG signal.
> >> > We have `face` and `ascent` in `misc` already for 2D signals. I think
> it
> >> > would be nice to have something like this for 1D signals, assuming we
> >> > can find one that won't take up a ton of space (< 500kB maybe, since
> >> > that's what ascent takes, and `face` takes up 1.5MB?). We don't want
> to
> >> > bloat the repo, but a reasonable sound choice could be used in many
> >> > examples.
> >> >
> >> > The most popular "freesound.org <http://freesound.org>" clip is a
> >> > creative-commons licensed thunderstorm recording
> >> > <https://freesound.org/people/RHumphries/sounds/2523/>, maybe this
> could
> >> > be made mono, truncated, and/or resampled to be sufficiently small?
> >> > Other ideas welcome.
> >>
> >> Another source might be the Physionet database has an extensive
> >> collection of medical signals and is licensed with the "OPC Public
> >> Domain Dedication and License" which should be compatible with SciPy's
> >> license.
> >>
> >> http://www.physionet.org/physiobank/database/
> >> https://opendatacommons.org/licenses/pddl/1.0/
> >>
> >> A biosignal like an ECG could be used to demonstrate
> >> - frequency based filtering (artifact elimination),
> >> - spectral analysis and
> >> - beat detection & classification (peak finding & measurement).
> >> E.g. a simple task could be to extract the heart rate or systolic /
> >> diastolic blood pressure values depending on the signal. This could
> >> provide nice and intuitive "real world" examples.
> >>
> >> If you like this idea I'd be happy to create a PR for this.
> >>
> >> Best regards,
> >> Lars
> >> _______________________________________________
> >> SciPy-Dev mailing list
> >> SciPy-Dev at python.org
> >> https://mail.python.org/mailman/listinfo/scipy-dev
> >
> >
> >
> > _______________________________________________
> > SciPy-Dev mailing list
> > SciPy-Dev at python.org
> > https://mail.python.org/mailman/listinfo/scipy-dev
> >
> _______________________________________________
> SciPy-Dev mailing list
> SciPy-Dev at python.org
> https://mail.python.org/mailman/listinfo/scipy-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scipy-dev/attachments/20180317/e08d5901/attachment-0001.html>

From ralf.gommers at gmail.com  Sat Mar 17 17:13:07 2018
From: ralf.gommers at gmail.com (Ralf Gommers)
Date: Sat, 17 Mar 2018 14:13:07 -0700
Subject: [SciPy-Dev] releasing SciPy 1.0.1
Message-ID: <CABL7CQi6zwzC9ekex+ZgkcADt7+V49DZeGP8OJsF8z0Bdq0Dyw@mail.gmail.com>

Hi all,

There are currently 11 PRs tagged for 1.0.1, and especially the recent f2py
issue that causes 1.0.0 to not build with NumPy master [1] needs a bugfix
release very soon. There are currently no open issues tagged with 1.0.1 -
if you know of anything critical, please speak up.

I propose to prepare things this weekend, and make the release somewhere in
the coming week.

Cheers,
Ralf

[1] https://github.com/scipy/scipy/pull/8530
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scipy-dev/attachments/20180317/746fbccc/attachment.html>

From dieter at werthmuller.org  Sat Mar 17 17:39:25 2018
From: dieter at werthmuller.org (=?UTF-8?Q?Dieter_Werthm=c3=bcller?=)
Date: Sat, 17 Mar 2018 15:39:25 -0600
Subject: [SciPy-Dev] previous/next for interp1d
In-Reply-To: <CABL7CQgM0Taey5p9+H2Rx4epgsUBLBBPHi7zDkTtZq-2ECXT1w@mail.gmail.com>
References: <48693513-3b3f-d6b7-9116-85b0eaf834f2@mailbox.org>
 <CAGu2niWWSKqqfKEADetwB2OObWLtyYh_F3+-nxjnkauFHfxHmw@mail.gmail.com>
 <13093fec-766d-ba01-26c5-cf57fd5aa2ce@mailbox.org>
 <CAGu2niWJZ=wHZsWSPB-vNOpqbVwwXTOT=U+4i9YHfyvK-+vJwQ@mail.gmail.com>
 <CAB2ovou2nUbUtV58KXGWk8E5uBv2-3upBJo8qviXtPvybNmPCQ@mail.gmail.com>
 <CABL7CQgM0Taey5p9+H2Rx4epgsUBLBBPHi7zDkTtZq-2ECXT1w@mail.gmail.com>
Message-ID: <563953c2-bac4-b32a-2b98-529585a3e90e@werthmuller.org>

Dear Devs,

I wanted to put your attention to 
https://github.com/scipy/scipy/pull/8572, which includes a 
`previous`/`next` feature to `interp1d`.

Only by making the pull request I realized that the same feature got 
already a pull request some 1.5 years ago, 
https://github.com/scipy/scipy/pull/6718. However, it did not proceed 
any further.

The two pull requests are slightly different, but it is beyond me to say 
which one is better or more adequate. However, I hope that you will 
include one or the other, as there are now already two pull requests for 
the same feature.

I included a few basic tests, copied and adjusted from the `nearest` 
interpolation. Let me know if more are required.

Regards,
Dieter

From lagru at mailbox.org  Sun Mar 18 07:23:11 2018
From: lagru at mailbox.org (Lars G.)
Date: Sun, 18 Mar 2018 12:23:11 +0100
Subject: [SciPy-Dev] Documentation / tutorial for peak finding in
 scipy.signal
In-Reply-To: <CABL7CQgM0Taey5p9+H2Rx4epgsUBLBBPHi7zDkTtZq-2ECXT1w@mail.gmail.com>
References: <48693513-3b3f-d6b7-9116-85b0eaf834f2@mailbox.org>
 <CAGu2niWWSKqqfKEADetwB2OObWLtyYh_F3+-nxjnkauFHfxHmw@mail.gmail.com>
 <13093fec-766d-ba01-26c5-cf57fd5aa2ce@mailbox.org>
 <CAGu2niWJZ=wHZsWSPB-vNOpqbVwwXTOT=U+4i9YHfyvK-+vJwQ@mail.gmail.com>
 <CAB2ovou2nUbUtV58KXGWk8E5uBv2-3upBJo8qviXtPvybNmPCQ@mail.gmail.com>
 <CABL7CQgM0Taey5p9+H2Rx4epgsUBLBBPHi7zDkTtZq-2ECXT1w@mail.gmail.com>
Message-ID: <14d2db01-5bf7-88f7-ab2c-4bcc29072c41@mailbox.org>

On 17.03.2018 20:59, Ralf Gommers wrote:
> If you can get away with a smaller size that would be useful, but I'm
> okay with up to 500 kb.

That should be possible. I just played around with ECG signals from the
MIT Arrhythmia Database
https://www.physionet.org/physiobank/database/mitdb/

The signals are sampled with 360 Hz and and an ADC resolution of 11-bit.
If `numpy.savez_compressed` is used to store the array we can get away
with `np.uint16` which translates to ~200 KiB for a 10 min window.

If we select the right window we could cover areas with different signal
properties (noise level, artifacts, baseline, amplitude, spectrum) which
would make it useful for more varied examples.

Lars

From sievert.scott at gmail.com  Sun Mar 18 16:07:07 2018
From: sievert.scott at gmail.com (Scott Sievert)
Date: Sun, 18 Mar 2018 16:07:07 -0400
Subject: [SciPy-Dev] SciPy 1.0 paper writing proposal
In-Reply-To: <CABL7CQg+wuisgcwXuTo9uPFqki2SBVOZC_iUFtrJ+A0S0cWx2Q@mail.gmail.com>
References: <CABL7CQi9Wa-JLyvqx4g15Kkj3pV0EuRijkNWV=wuKbzKpos3WQ@mail.gmail.com>
 <20180126074429.3csbkpsg7hbme22g@fastmail.com>
 <CABL7CQi5g===ntdphLV9BybWSACv_T2d4Ot0WOmKBpdCRL_+XA@mail.gmail.com>
 <CAGu2niWyHjU-v2QxnfeHOk56JOsXuhX1mAne_9hMXo5KiuSdWg@mail.gmail.com>
 <20180130195725.GC20306@lakota>
 <CAEBuzr8Mya4d0CTcNQKMKupde2vbuZbCF9mMHziggqGakw6U2g@mail.gmail.com>
 <CABL7CQg+wuisgcwXuTo9uPFqki2SBVOZC_iUFtrJ+A0S0cWx2Q@mail.gmail.com>
Message-ID: <CAPZ=bb0LrWXKECkfkBdE1_3jNE-PY5vJox2PsOgE2MvvfdOo+A@mail.gmail.com>

What updates are there on this? Ralf said mid-April was a submission
target, but I haven?t seen anything yet. I?m more than happy to write some
on my contribution.

Scott

On February 4, 2018 at 3:43:17 AM, Ralf Gommers (ralf.gommers at gmail.com)
wrote:



On Tue, Jan 30, 2018 at 8:10 PM, Ilhan Polat <ilhanpolat at gmail.com> wrote:

> Same as Eric also if you need some TeX stuff, I have a very broad and
> useless TeX experience.
>

Thanks Eric & Ilhan. I just sent the follow-up email to the coordinating
volunteers.

Cheers,
Ralf

_______________________________________________
SciPy-Dev mailing list
SciPy-Dev at python.org
https://mail.python.org/mailman/listinfo/scipy-dev
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scipy-dev/attachments/20180318/d2831e5b/attachment.html>

From adibhar97 at gmail.com  Mon Mar 19 16:05:12 2018
From: adibhar97 at gmail.com (Aditya Bharti)
Date: Tue, 20 Mar 2018 01:35:12 +0530
Subject: [SciPy-Dev] GSOC 2018 [Proposal Feedback]
Message-ID: <CANRzPHXupMpDwj3-GxWkBn05SPQp9Fkxkb2VPiEfGqobk45uWQ@mail.gmail.com>

Hi all,

Goal of this email: To ask for feedback on the draft proposal for GSOC 2018

I am an undergraduate researcher in Computer Vision interested in
contributing to scipy's Rotation Formalism in 3 Dimensions project idea as
part of GSOC 2018.

I've made some contributions to scipy and have drafted a proposal here
<https://docs.google.com/document/d/1UriyLABwgjUcYfBofSr4hqAnLzblJ71pkzm7i26O8ws/edit?usp=sharing>.
[1]

It includes my contact information, code samples, project timeline, and an
introduction about myself. I went through the proposal guidelines on the
website and think I covered all my bases.

I would greatly appreciate your feedback on this as it will help me in
making a strong application.

Thank you,
Aditya Bharti

[1]:
https://docs.google.com/document/d/1UriyLABwgjUcYfBofSr4hqAnLzblJ71pkzm7i26O8ws/edit?usp=sharing
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scipy-dev/attachments/20180320/dec97ffa/attachment.html>

From smkjain8 at gmail.com  Tue Mar 20 10:11:27 2018
From: smkjain8 at gmail.com (Samyak Jain)
Date: Tue, 20 Mar 2018 19:41:27 +0530
Subject: [SciPy-Dev] Code Sample In proposal
Message-ID: <CABbcdpBdoL_x2S81O_Q=iinAApVYpZf0YA25pFiWe7DQ=V2+jQ@mail.gmail.com>

This is regarding the code sample one must submit in the proposal. What all
needs to be there in the code sample? I am working on the Rotation
Formalism project.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scipy-dev/attachments/20180320/dcd03f92/attachment.html>

From ralf.gommers at gmail.com  Tue Mar 20 11:24:21 2018
From: ralf.gommers at gmail.com (Ralf Gommers)
Date: Tue, 20 Mar 2018 08:24:21 -0700
Subject: [SciPy-Dev] SciPy 1.0 paper writing proposal
In-Reply-To: <CAPZ=bb0LrWXKECkfkBdE1_3jNE-PY5vJox2PsOgE2MvvfdOo+A@mail.gmail.com>
References: <CABL7CQi9Wa-JLyvqx4g15Kkj3pV0EuRijkNWV=wuKbzKpos3WQ@mail.gmail.com>
 <20180126074429.3csbkpsg7hbme22g@fastmail.com>
 <CABL7CQi5g===ntdphLV9BybWSACv_T2d4Ot0WOmKBpdCRL_+XA@mail.gmail.com>
 <CAGu2niWyHjU-v2QxnfeHOk56JOsXuhX1mAne_9hMXo5KiuSdWg@mail.gmail.com>
 <20180130195725.GC20306@lakota>
 <CAEBuzr8Mya4d0CTcNQKMKupde2vbuZbCF9mMHziggqGakw6U2g@mail.gmail.com>
 <CABL7CQg+wuisgcwXuTo9uPFqki2SBVOZC_iUFtrJ+A0S0cWx2Q@mail.gmail.com>
 <CAPZ=bb0LrWXKECkfkBdE1_3jNE-PY5vJox2PsOgE2MvvfdOo+A@mail.gmail.com>
Message-ID: <CABL7CQgOqH1M9N64F-LNuRWcRd3bB3S+G4Db=APCJm3xT0TYNA@mail.gmail.com>

On Sun, Mar 18, 2018 at 1:07 PM, Scott Sievert <sievert.scott at gmail.com>
wrote:

> What updates are there on this? Ralf said mid-April was a submission
> target, but I haven?t seen anything yet. I?m more than happy to write some
> on my contribution.
>

We've decided on the document outline (
https://github.com/scipy/scipy-articles/pull/14) and the first section
drafts have started to appear (thanks Tyler and Matt!):
https://github.com/scipy/scipy-articles/issues/13
https://github.com/scipy/scipy-articles/pull/15

However you're right that we're a bit slow, mid-April finishing a first
draft could still be feasibly if people start writing asap, but submission
by then we won't make.

https://github.com/scipy/scipy-articles/issues/9 contains the sections and
people who volunteered to author them. There's still some sections that
need a volunteer; especially the technical sections could be written by
people other than the main developer/maintainer - if you're confident you
could (co-)write a section, feel free to comment there and jump in.

Cheers,
Ralf



>
> Scott
>
> On February 4, 2018 at 3:43:17 AM, Ralf Gommers (ralf.gommers at gmail.com)
> wrote:
>
>
>
> On Tue, Jan 30, 2018 at 8:10 PM, Ilhan Polat <ilhanpolat at gmail.com> wrote:
>
>> Same as Eric also if you need some TeX stuff, I have a very broad and
>> useless TeX experience.
>>
>
> Thanks Eric & Ilhan. I just sent the follow-up email to the coordinating
> volunteers.
>
> Cheers,
> Ralf
>
> _______________________________________________
> SciPy-Dev mailing list
> SciPy-Dev at python.org
> https://mail.python.org/mailman/listinfo/scipy-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scipy-dev/attachments/20180320/fd3a649b/attachment.html>

From ralf.gommers at gmail.com  Tue Mar 20 11:46:47 2018
From: ralf.gommers at gmail.com (Ralf Gommers)
Date: Tue, 20 Mar 2018 08:46:47 -0700
Subject: [SciPy-Dev] Code Sample In proposal
In-Reply-To: <CABbcdpBdoL_x2S81O_Q=iinAApVYpZf0YA25pFiWe7DQ=V2+jQ@mail.gmail.com>
References: <CABbcdpBdoL_x2S81O_Q=iinAApVYpZf0YA25pFiWe7DQ=V2+jQ@mail.gmail.com>
Message-ID: <CABL7CQhW8N150osFJDWqkyFG6G+J6q7isQthXLuNUB2Sm=+emQ@mail.gmail.com>

On Tue, Mar 20, 2018 at 7:11 AM, Samyak Jain <smkjain8 at gmail.com> wrote:

> This is regarding the code sample one must submit in the proposal. What
> all needs to be there in the code sample? I am working on the Rotation
> Formalism project.
>

It means to put in your proposal a link to the pull requests you have made.
At a minimum you need one contribution, and preferably it needs to be more
significant than a simple one-liner fix.

Ralf
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scipy-dev/attachments/20180320/714313c1/attachment.html>

From ralf.gommers at gmail.com  Tue Mar 20 11:56:11 2018
From: ralf.gommers at gmail.com (Ralf Gommers)
Date: Tue, 20 Mar 2018 08:56:11 -0700
Subject: [SciPy-Dev] GSOC 2018 [Proposal Feedback]
In-Reply-To: <CANRzPHXupMpDwj3-GxWkBn05SPQp9Fkxkb2VPiEfGqobk45uWQ@mail.gmail.com>
References: <CANRzPHXupMpDwj3-GxWkBn05SPQp9Fkxkb2VPiEfGqobk45uWQ@mail.gmail.com>
Message-ID: <CABL7CQiL5iyO9CwNSrejvkAx005mNYpaKEN-GOCNSqdL+ApZTA@mail.gmail.com>

On Mon, Mar 19, 2018 at 1:05 PM, Aditya Bharti <adibhar97 at gmail.com> wrote:

> Hi all,
>
> Goal of this email: To ask for feedback on the draft proposal for GSOC 2018
>
> I am an undergraduate researcher in Computer Vision interested in
> contributing to scipy's Rotation Formalism in 3 Dimensions project idea as
> part of GSOC 2018.
>
> I've made some contributions to scipy and have drafted a proposal here
> <https://docs.google.com/document/d/1UriyLABwgjUcYfBofSr4hqAnLzblJ71pkzm7i26O8ws/edit?usp=sharing>.
> [1]
>
> It includes my contact information, code samples, project timeline, and an
> introduction about myself. I went through the proposal guidelines on the
> website and think I covered all my bases.
>
> I would greatly appreciate your feedback on this as it will help me in
> making a strong application.
>

Hi Aditya, that looks like a solid start. I added a few comments; will
leave it to the proposed mentors to comment on some of the algorithmic
ideas. One other thing to consider is existing implementations to possibly
build upon - a couple of people suggested other packages on this list not
too long ago.

Cheers,
Ralf




>
> Thank you,
> Aditya Bharti
>
> [1]: https://docs.google.com/document/d/1UriyLABwgjUcYfBofSr4hqAnLzblJ
> 71pkzm7i26O8ws/edit?usp=sharing
>
> _______________________________________________
> SciPy-Dev mailing list
> SciPy-Dev at python.org
> https://mail.python.org/mailman/listinfo/scipy-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scipy-dev/attachments/20180320/82afbdfc/attachment-0001.html>

From larson.eric.d at gmail.com  Tue Mar 20 15:31:53 2018
From: larson.eric.d at gmail.com (Eric Larson)
Date: Tue, 20 Mar 2018 15:31:53 -0400
Subject: [SciPy-Dev] GSOC 2018 [Proposal Feedback]
In-Reply-To: <CABL7CQiL5iyO9CwNSrejvkAx005mNYpaKEN-GOCNSqdL+ApZTA@mail.gmail.com>
References: <CANRzPHXupMpDwj3-GxWkBn05SPQp9Fkxkb2VPiEfGqobk45uWQ@mail.gmail.com>
 <CABL7CQiL5iyO9CwNSrejvkAx005mNYpaKEN-GOCNSqdL+ApZTA@mail.gmail.com>
Message-ID: <CAGu2niUSGK=ROENgmZ0J4F261Hjs5_+Xem31PcXi3w9m8YXbSw@mail.gmail.com>

I second Ralf's comments, and have added a few, too.

Eric


On Tue, Mar 20, 2018 at 11:56 AM, Ralf Gommers <ralf.gommers at gmail.com>
wrote:

>
>
> On Mon, Mar 19, 2018 at 1:05 PM, Aditya Bharti <adibhar97 at gmail.com>
> wrote:
>
>> Hi all,
>>
>> Goal of this email: To ask for feedback on the draft proposal for GSOC
>> 2018
>>
>> I am an undergraduate researcher in Computer Vision interested in
>> contributing to scipy's Rotation Formalism in 3 Dimensions project idea as
>> part of GSOC 2018.
>>
>> I've made some contributions to scipy and have drafted a proposal here
>> <https://docs.google.com/document/d/1UriyLABwgjUcYfBofSr4hqAnLzblJ71pkzm7i26O8ws/edit?usp=sharing>.
>> [1]
>>
>> It includes my contact information, code samples, project timeline, and
>> an introduction about myself. I went through the proposal guidelines on the
>> website and think I covered all my bases.
>>
>> I would greatly appreciate your feedback on this as it will help me in
>> making a strong application.
>>
>
> Hi Aditya, that looks like a solid start. I added a few comments; will
> leave it to the proposed mentors to comment on some of the algorithmic
> ideas. One other thing to consider is existing implementations to possibly
> build upon - a couple of people suggested other packages on this list not
> too long ago.
>
> Cheers,
> Ralf
>
>
>
>
>>
>> Thank you,
>> Aditya Bharti
>>
>> [1]: https://docs.google.com/document/d/1UriyLABwgjUcYfBofSr4hqAn
>> LzblJ71pkzm7i26O8ws/edit?usp=sharing
>>
>> _______________________________________________
>> SciPy-Dev mailing list
>> SciPy-Dev at python.org
>> https://mail.python.org/mailman/listinfo/scipy-dev
>>
>>
>
> _______________________________________________
> SciPy-Dev mailing list
> SciPy-Dev at python.org
> https://mail.python.org/mailman/listinfo/scipy-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scipy-dev/attachments/20180320/aed5bf0e/attachment.html>

From warren.weckesser at gmail.com  Tue Mar 20 15:50:56 2018
From: warren.weckesser at gmail.com (Warren Weckesser)
Date: Tue, 20 Mar 2018 15:50:56 -0400
Subject: [SciPy-Dev] SciPy 1.0 paper writing proposal
In-Reply-To: <CABL7CQgOqH1M9N64F-LNuRWcRd3bB3S+G4Db=APCJm3xT0TYNA@mail.gmail.com>
References: <CABL7CQi9Wa-JLyvqx4g15Kkj3pV0EuRijkNWV=wuKbzKpos3WQ@mail.gmail.com>
 <20180126074429.3csbkpsg7hbme22g@fastmail.com>
 <CABL7CQi5g===ntdphLV9BybWSACv_T2d4Ot0WOmKBpdCRL_+XA@mail.gmail.com>
 <CAGu2niWyHjU-v2QxnfeHOk56JOsXuhX1mAne_9hMXo5KiuSdWg@mail.gmail.com>
 <20180130195725.GC20306@lakota>
 <CAEBuzr8Mya4d0CTcNQKMKupde2vbuZbCF9mMHziggqGakw6U2g@mail.gmail.com>
 <CABL7CQg+wuisgcwXuTo9uPFqki2SBVOZC_iUFtrJ+A0S0cWx2Q@mail.gmail.com>
 <CAPZ=bb0LrWXKECkfkBdE1_3jNE-PY5vJox2PsOgE2MvvfdOo+A@mail.gmail.com>
 <CABL7CQgOqH1M9N64F-LNuRWcRd3bB3S+G4Db=APCJm3xT0TYNA@mail.gmail.com>
Message-ID: <CAGzF1ufiEiot0OjvH=rpiugtr4hS-PxXjK-QtAk35BCoHbnrog@mail.gmail.com>

On Tue, Mar 20, 2018 at 11:24 AM, Ralf Gommers <ralf.gommers at gmail.com>
wrote:

>
>
> On Sun, Mar 18, 2018 at 1:07 PM, Scott Sievert <sievert.scott at gmail.com>
> wrote:
>
>> What updates are there on this? Ralf said mid-April was a submission
>> target, but I haven?t seen anything yet. I?m more than happy to write some
>> on my contribution.
>>
>
> We've decided on the document outline (https://github.com/scipy/
> scipy-articles/pull/14) and the first section drafts have started to
> appear (thanks Tyler and Matt!):
> https://github.com/scipy/scipy-articles/issues/13
> https://github.com/scipy/scipy-articles/pull/15
>
> However you're right that we're a bit slow, mid-April finishing a first
> draft could still be feasibly if people start writing asap, but submission
> by then we won't make.
>
> https://github.com/scipy/scipy-articles/issues/9 contains the sections
> and people who volunteered to author them. There's still some sections that
> need a volunteer; especially the technical sections could be written by
> people other than the main developer/maintainer - if you're confident you
> could (co-)write a section, feel free to comment there and jump in.
>
> Cheers,
> Ralf
>
>

I'll be able to help with writing on `stats` and `signal` (and possibly
other areas, if needed).

Warren


Scott
>
> On February 4, 2018 at 3:43:17 AM, Ralf Gommers (ralf.gommers at gmail.com)
> wrote:
>
>
>
> On Tue, Jan 30, 2018 at 8:10 PM, Ilhan Polat <ilhanpolat at gmail.com> wrote:
>
>> Same as Eric also if you need some TeX stuff, I have a very broad and
>> useless TeX experience.
>>
>
> Thanks Eric & Ilhan. I just sent the follow-up email to the coordinating
> volunteers.
>
> Cheers,
> Ralf
>
> _______________________________________________
> SciPy-Dev mailing list
> SciPy-Dev at python.org
> https://mail.python.org/mailman/listinfo/scipy-dev
>
>
>
> _______________________________________________
> SciPy-Dev mailing list
> SciPy-Dev at python.org
> https://mail.python.org/mailman/listinfo/scipy-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scipy-dev/attachments/20180320/c2324a9f/attachment.html>

From vishstar88 at gmail.com  Wed Mar 21 02:46:53 2018
From: vishstar88 at gmail.com (Vishal Gupta)
Date: Wed, 21 Mar 2018 12:16:53 +0530
Subject: [SciPy-Dev] GSoC Rotation Formalism Proposal
Message-ID: <CAEkj7mJrzVTSfFhX=LFLKu5DBfNNEk1Y5mgtMUndnNXW_ngbzw@mail.gmail.com>

Hey,

I am a 3rd CSE undergrad at SSNCE, Chennai, India. I'd like to work on
Rotation Formalism in 3 Dimensions as a part of this year's GSoC .

https://docs.google.com/document/d/1y5OalGAvYkk8UvLtFjQ2-xhRj_NTwq0fV3GvlCBUp0I/edit?usp=sharing

Thanks,
Vishal Gupta

?
<https://mailtrack.io/> Sent with Mailtrack
<https://mailtrack.io?utm_source=gmail&utm_medium=signature&utm_campaign=signaturevirality&>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scipy-dev/attachments/20180321/2c6fb3c1/attachment.html>

From balazovic.peter at gmail.com  Wed Mar 21 03:04:24 2018
From: balazovic.peter at gmail.com (Peter Balazovic)
Date: Wed, 21 Mar 2018 08:04:24 +0100
Subject: [SciPy-Dev] SciPy dependencies
Message-ID: <CAFq4t4gh50XVoGLcZo0J8g0J8jVXJ4XEHfBhZ7FeLaB44MoVMg@mail.gmail.com>

Dears,

I am trying to get installed SciPy under Yocto. I encounter compile error:

ERROR: scipy-1.0.0-r0 do_compile: Function failed: do_compile (log
file is located at
../tmp/work/cortexa9hf-neon-poky-linux-gnueabi/scipy/1.0.0-r0/temp/log.do_compile.5023)
    Log data follows:| DEBUG: Executing shell function do_compile
| ERROR: python setup.py build execution failed.|
../tmp/sysroots/x86_64-linux/usr/lib/python2.7/distutils/dist.py:267:
UserWarning: Unknown distribution option: 'python_requires'|
warnings.warn(msg)| lapack_opt_info:| openblas_lapack_info:|
libraries openblas not found in
['../tmp/sysroots/x86_64-linux/usr/lib']|   NOT AVAILABLE

The SciPy install setup.py looks for openBLAS in different directory
*x86_64-linux* instead of *target-arm* directory.
I have already installed openBLAS and it is located at *target-arm* sysroot.

./tmp/sysroots/target-arm/usr/lib/libopenblas.so
./tmp/sysroots/target-arm/usr/lib/libopenblas.a
./tmp/sysroots/target-arm/usr/include/openblas_config.h

With building Yocto there are three "build" directories *target-arm* (target
sysroot), *target-arm-tcbootstrap* (intermediate directory for the compiler
bootstrap), and *x86_64-linux* (host tools and libraries). The openBLAS is
installed within *target-arm* sysroot.

My question is how to tell and redirect SciPy installer to look for
openBLAS (and other dependencies) at *target-arm* directory (
*./tmp/sysroots/target-arm/*)?

Thanks you.

Sent from Mailspring <https://getmailspring.com/>, the best free email app
for work
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scipy-dev/attachments/20180321/789cfded/attachment-0001.html>

From adibhar97 at gmail.com  Wed Mar 21 04:40:56 2018
From: adibhar97 at gmail.com (Aditya Bharti)
Date: Wed, 21 Mar 2018 14:10:56 +0530
Subject: [SciPy-Dev] GSOC 2018 [Proposal Feedback]
In-Reply-To: <CAGu2niUSGK=ROENgmZ0J4F261Hjs5_+Xem31PcXi3w9m8YXbSw@mail.gmail.com>
References: <CANRzPHXupMpDwj3-GxWkBn05SPQp9Fkxkb2VPiEfGqobk45uWQ@mail.gmail.com>
 <CABL7CQiL5iyO9CwNSrejvkAx005mNYpaKEN-GOCNSqdL+ApZTA@mail.gmail.com>
 <CAGu2niUSGK=ROENgmZ0J4F261Hjs5_+Xem31PcXi3w9m8YXbSw@mail.gmail.com>
Message-ID: <CANRzPHVz3sbHvfHTMRsz3ZdWg-rDxkw_FZkwUkEi+T73V3Lt2Q@mail.gmail.com>

Hi all,
Thank you for your helpful comments. I've updated my proposal with the
following major changes:

   - Fleshed out the API in more detail
   - Included a github link to a sample implementation of the initializer
   functions
   - Proposed a quaternion class for faster compositions of rotations (use
   of this class will be completely optional and not required for Rotation()
   functionality)

Thank you for all your help so far. I look forward to hearing from you.

Regards,
Aditya

https://docs.google.com/document/d/1UriyLABwgjUcYfBofSr4hqAnLzblJ71pkzm7i26O8ws/edit?usp=sharing

On 21 March 2018 at 01:01, Eric Larson <larson.eric.d at gmail.com> wrote:

> I second Ralf's comments, and have added a few, too.
>
> Eric
>
>
> On Tue, Mar 20, 2018 at 11:56 AM, Ralf Gommers <ralf.gommers at gmail.com>
> wrote:
>
>>
>>
>> On Mon, Mar 19, 2018 at 1:05 PM, Aditya Bharti <adibhar97 at gmail.com>
>> wrote:
>>
>>> Hi all,
>>>
>>> Goal of this email: To ask for feedback on the draft proposal for GSOC
>>> 2018
>>>
>>> I am an undergraduate researcher in Computer Vision interested in
>>> contributing to scipy's Rotation Formalism in 3 Dimensions project idea as
>>> part of GSOC 2018.
>>>
>>> I've made some contributions to scipy and have drafted a proposal here
>>> <https://docs.google.com/document/d/1UriyLABwgjUcYfBofSr4hqAnLzblJ71pkzm7i26O8ws/edit?usp=sharing>.
>>> [1]
>>>
>>> It includes my contact information, code samples, project timeline, and
>>> an introduction about myself. I went through the proposal guidelines on the
>>> website and think I covered all my bases.
>>>
>>> I would greatly appreciate your feedback on this as it will help me in
>>> making a strong application.
>>>
>>
>> Hi Aditya, that looks like a solid start. I added a few comments; will
>> leave it to the proposed mentors to comment on some of the algorithmic
>> ideas. One other thing to consider is existing implementations to possibly
>> build upon - a couple of people suggested other packages on this list not
>> too long ago.
>>
>> Cheers,
>> Ralf
>>
>>
>>
>>
>>>
>>> Thank you,
>>> Aditya Bharti
>>>
>>> [1]: https://docs.google.com/document/d/1UriyLABwgjUcYfBofSr4hqAn
>>> LzblJ71pkzm7i26O8ws/edit?usp=sharing
>>>
>>> _______________________________________________
>>> SciPy-Dev mailing list
>>> SciPy-Dev at python.org
>>> https://mail.python.org/mailman/listinfo/scipy-dev
>>>
>>>
>>
>> _______________________________________________
>> SciPy-Dev mailing list
>> SciPy-Dev at python.org
>> https://mail.python.org/mailman/listinfo/scipy-dev
>>
>>
>
> _______________________________________________
> SciPy-Dev mailing list
> SciPy-Dev at python.org
> https://mail.python.org/mailman/listinfo/scipy-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scipy-dev/attachments/20180321/2c999423/attachment.html>

From einstein.edison at gmail.com  Wed Mar 21 06:22:08 2018
From: einstein.edison at gmail.com (Hameer Abbasi)
Date: Wed, 21 Mar 2018 11:22:08 +0100
Subject: [SciPy-Dev] GSOC 2018 [Proposal Feedback]
In-Reply-To: <CANRzPHVz3sbHvfHTMRsz3ZdWg-rDxkw_FZkwUkEi+T73V3Lt2Q@mail.gmail.com>
References: <CANRzPHXupMpDwj3-GxWkBn05SPQp9Fkxkb2VPiEfGqobk45uWQ@mail.gmail.com>
 <CABL7CQiL5iyO9CwNSrejvkAx005mNYpaKEN-GOCNSqdL+ApZTA@mail.gmail.com>
 <CAGu2niUSGK=ROENgmZ0J4F261Hjs5_+Xem31PcXi3w9m8YXbSw@mail.gmail.com>
 <CANRzPHVz3sbHvfHTMRsz3ZdWg-rDxkw_FZkwUkEi+T73V3Lt2Q@mail.gmail.com>
Message-ID: <CADViA5DPQA-bq3ZRJz5YB7TiSbCagcX+c1niubvcrbbTxsetCA@mail.gmail.com>

Hi Aditya,

I've left a few comments regarding composing and np.matrix. Using __mul__
for anything other than real multiplication is ill-advised and caused
problems before with np.matrix. I suggest you use __matmul__ instead.

Hameer

On Wed, Mar 21, 2018 at 9:40 AM, Aditya Bharti <adibhar97 at gmail.com> wrote:

> Hi all,
> Thank you for your helpful comments. I've updated my proposal with the
> following major changes:
>
>    - Fleshed out the API in more detail
>    - Included a github link to a sample implementation of the initializer
>    functions
>    - Proposed a quaternion class for faster compositions of rotations
>    (use of this class will be completely optional and not required for
>    Rotation() functionality)
>
> Thank you for all your help so far. I look forward to hearing from you.
>
> Regards,
> Aditya
>
> https://docs.google.com/document/d/1UriyLABwgjUcYfBofSr4hqAnLzblJ
> 71pkzm7i26O8ws/edit?usp=sharing
>
> On 21 March 2018 at 01:01, Eric Larson <larson.eric.d at gmail.com> wrote:
>
>> I second Ralf's comments, and have added a few, too.
>>
>> Eric
>>
>>
>> On Tue, Mar 20, 2018 at 11:56 AM, Ralf Gommers <ralf.gommers at gmail.com>
>> wrote:
>>
>>>
>>>
>>> On Mon, Mar 19, 2018 at 1:05 PM, Aditya Bharti <adibhar97 at gmail.com>
>>> wrote:
>>>
>>>> Hi all,
>>>>
>>>> Goal of this email: To ask for feedback on the draft proposal for GSOC
>>>> 2018
>>>>
>>>> I am an undergraduate researcher in Computer Vision interested in
>>>> contributing to scipy's Rotation Formalism in 3 Dimensions project idea as
>>>> part of GSOC 2018.
>>>>
>>>> I've made some contributions to scipy and have drafted a proposal here
>>>> <https://docs.google.com/document/d/1UriyLABwgjUcYfBofSr4hqAnLzblJ71pkzm7i26O8ws/edit?usp=sharing>.
>>>> [1]
>>>>
>>>> It includes my contact information, code samples, project timeline, and
>>>> an introduction about myself. I went through the proposal guidelines on the
>>>> website and think I covered all my bases.
>>>>
>>>> I would greatly appreciate your feedback on this as it will help me in
>>>> making a strong application.
>>>>
>>>
>>> Hi Aditya, that looks like a solid start. I added a few comments; will
>>> leave it to the proposed mentors to comment on some of the algorithmic
>>> ideas. One other thing to consider is existing implementations to possibly
>>> build upon - a couple of people suggested other packages on this list not
>>> too long ago.
>>>
>>> Cheers,
>>> Ralf
>>>
>>>
>>>
>>>
>>>>
>>>> Thank you,
>>>> Aditya Bharti
>>>>
>>>> [1]: https://docs.google.com/document/d/1UriyLABwgjUcYfBofSr4hqAn
>>>> LzblJ71pkzm7i26O8ws/edit?usp=sharing
>>>>
>>>> _______________________________________________
>>>> SciPy-Dev mailing list
>>>> SciPy-Dev at python.org
>>>> https://mail.python.org/mailman/listinfo/scipy-dev
>>>>
>>>>
>>>
>>> _______________________________________________
>>> SciPy-Dev mailing list
>>> SciPy-Dev at python.org
>>> https://mail.python.org/mailman/listinfo/scipy-dev
>>>
>>>
>>
>> _______________________________________________
>> SciPy-Dev mailing list
>> SciPy-Dev at python.org
>> https://mail.python.org/mailman/listinfo/scipy-dev
>>
>>
>
> _______________________________________________
> SciPy-Dev mailing list
> SciPy-Dev at python.org
> https://mail.python.org/mailman/listinfo/scipy-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scipy-dev/attachments/20180321/e5482fcb/attachment.html>

From adibhar97 at gmail.com  Wed Mar 21 06:35:40 2018
From: adibhar97 at gmail.com (Aditya Bharti)
Date: Wed, 21 Mar 2018 16:05:40 +0530
Subject: [SciPy-Dev] GSOC 2018 [Proposal Feedback]
In-Reply-To: <CADViA5DPQA-bq3ZRJz5YB7TiSbCagcX+c1niubvcrbbTxsetCA@mail.gmail.com>
References: <CANRzPHXupMpDwj3-GxWkBn05SPQp9Fkxkb2VPiEfGqobk45uWQ@mail.gmail.com>
 <CABL7CQiL5iyO9CwNSrejvkAx005mNYpaKEN-GOCNSqdL+ApZTA@mail.gmail.com>
 <CAGu2niUSGK=ROENgmZ0J4F261Hjs5_+Xem31PcXi3w9m8YXbSw@mail.gmail.com>
 <CANRzPHVz3sbHvfHTMRsz3ZdWg-rDxkw_FZkwUkEi+T73V3Lt2Q@mail.gmail.com>
 <CADViA5DPQA-bq3ZRJz5YB7TiSbCagcX+c1niubvcrbbTxsetCA@mail.gmail.com>
Message-ID: <CANRzPHX4=fLorxyMwxDxKDSeHd43PCeMPhrTMSkdr9Vs8jtS-w@mail.gmail.com>

Hi Hameer,
Thank you for your comments. They were quite informative. Just a couple of
questions.

The __matmul__ function translates to the @ operator correct? Would that
not be somewhat awkward for new users?
Also, I couldn't find the suggested __rmatmul__ anywhere on the operator
list: https://docs.python.org/3/library/operator.html. Am I looking in the
wrong place?

In addition, are there any other specific things I need to stay away from
to avoid np.matrix like problems? I just know that the matrix class is not
used much, I don't really know the reasons for its deprecation.

Thanks,
Aditya

On 21 March 2018 at 15:52, Hameer Abbasi <einstein.edison at gmail.com> wrote:

> Hi Aditya,
>
> I've left a few comments regarding composing and np.matrix. Using __mul__
> for anything other than real multiplication is ill-advised and caused
> problems before with np.matrix. I suggest you use __matmul__ instead.
>
> Hameer
>
>
> On Wed, Mar 21, 2018 at 9:40 AM, Aditya Bharti <adibhar97 at gmail.com>
> wrote:
>
>> Hi all,
>> Thank you for your helpful comments. I've updated my proposal with the
>> following major changes:
>>
>>    - Fleshed out the API in more detail
>>    - Included a github link to a sample implementation of the
>>    initializer functions
>>    - Proposed a quaternion class for faster compositions of rotations
>>    (use of this class will be completely optional and not required for
>>    Rotation() functionality)
>>
>> Thank you for all your help so far. I look forward to hearing from you.
>>
>> Regards,
>> Aditya
>>
>> https://docs.google.com/document/d/1UriyLABwgjUcYfBofSr4hqAn
>> LzblJ71pkzm7i26O8ws/edit?usp=sharing
>>
>> On 21 March 2018 at 01:01, Eric Larson <larson.eric.d at gmail.com> wrote:
>>
>>> I second Ralf's comments, and have added a few, too.
>>>
>>> Eric
>>>
>>>
>>> On Tue, Mar 20, 2018 at 11:56 AM, Ralf Gommers <ralf.gommers at gmail.com>
>>> wrote:
>>>
>>>>
>>>>
>>>> On Mon, Mar 19, 2018 at 1:05 PM, Aditya Bharti <adibhar97 at gmail.com>
>>>> wrote:
>>>>
>>>>> Hi all,
>>>>>
>>>>> Goal of this email: To ask for feedback on the draft proposal for GSOC
>>>>> 2018
>>>>>
>>>>> I am an undergraduate researcher in Computer Vision interested in
>>>>> contributing to scipy's Rotation Formalism in 3 Dimensions project idea as
>>>>> part of GSOC 2018.
>>>>>
>>>>> I've made some contributions to scipy and have drafted a proposal here
>>>>> <https://docs.google.com/document/d/1UriyLABwgjUcYfBofSr4hqAnLzblJ71pkzm7i26O8ws/edit?usp=sharing>.
>>>>> [1]
>>>>>
>>>>> It includes my contact information, code samples, project timeline,
>>>>> and an introduction about myself. I went through the proposal guidelines on
>>>>> the website and think I covered all my bases.
>>>>>
>>>>> I would greatly appreciate your feedback on this as it will help me in
>>>>> making a strong application.
>>>>>
>>>>
>>>> Hi Aditya, that looks like a solid start. I added a few comments; will
>>>> leave it to the proposed mentors to comment on some of the algorithmic
>>>> ideas. One other thing to consider is existing implementations to possibly
>>>> build upon - a couple of people suggested other packages on this list not
>>>> too long ago.
>>>>
>>>> Cheers,
>>>> Ralf
>>>>
>>>>
>>>>
>>>>
>>>>>
>>>>> Thank you,
>>>>> Aditya Bharti
>>>>>
>>>>> [1]: https://docs.google.com/document/d/1UriyLABwgjUcYfBofSr4hqAn
>>>>> LzblJ71pkzm7i26O8ws/edit?usp=sharing
>>>>>
>>>>> _______________________________________________
>>>>> SciPy-Dev mailing list
>>>>> SciPy-Dev at python.org
>>>>> https://mail.python.org/mailman/listinfo/scipy-dev
>>>>>
>>>>>
>>>>
>>>> _______________________________________________
>>>> SciPy-Dev mailing list
>>>> SciPy-Dev at python.org
>>>> https://mail.python.org/mailman/listinfo/scipy-dev
>>>>
>>>>
>>>
>>> _______________________________________________
>>> SciPy-Dev mailing list
>>> SciPy-Dev at python.org
>>> https://mail.python.org/mailman/listinfo/scipy-dev
>>>
>>>
>>
>> _______________________________________________
>> SciPy-Dev mailing list
>> SciPy-Dev at python.org
>> https://mail.python.org/mailman/listinfo/scipy-dev
>>
>>
>
> _______________________________________________
> SciPy-Dev mailing list
> SciPy-Dev at python.org
> https://mail.python.org/mailman/listinfo/scipy-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scipy-dev/attachments/20180321/7060483d/attachment-0001.html>

From einstein.edison at gmail.com  Wed Mar 21 06:48:51 2018
From: einstein.edison at gmail.com (Hameer Abbasi)
Date: Wed, 21 Mar 2018 11:48:51 +0100
Subject: [SciPy-Dev] GSOC 2018 [Proposal Feedback]
In-Reply-To: <CANRzPHX4=fLorxyMwxDxKDSeHd43PCeMPhrTMSkdr9Vs8jtS-w@mail.gmail.com>
References: <CANRzPHXupMpDwj3-GxWkBn05SPQp9Fkxkb2VPiEfGqobk45uWQ@mail.gmail.com>
 <CABL7CQiL5iyO9CwNSrejvkAx005mNYpaKEN-GOCNSqdL+ApZTA@mail.gmail.com>
 <CAGu2niUSGK=ROENgmZ0J4F261Hjs5_+Xem31PcXi3w9m8YXbSw@mail.gmail.com>
 <CANRzPHVz3sbHvfHTMRsz3ZdWg-rDxkw_FZkwUkEi+T73V3Lt2Q@mail.gmail.com>
 <CADViA5DPQA-bq3ZRJz5YB7TiSbCagcX+c1niubvcrbbTxsetCA@mail.gmail.com>
 <CANRzPHX4=fLorxyMwxDxKDSeHd43PCeMPhrTMSkdr9Vs8jtS-w@mail.gmail.com>
Message-ID: <CADViA5Ag2=Pn0AO51V7itfoBFN+mBRy8=0n-DmrRJ99CZeOv5w@mail.gmail.com>

Overloaded operators can be found here:
https://docs.python.org/3/reference/datamodel.html#emulating-numeric-types

It was deprecated because things taking it as input expected * to work like
in np.ndarray and there were other weird incompatibilities when using it in
a lot of other functions (e.g. always 2-D)

Yes, __matmul__ translates to the @ operator. New users can find it
complicated but most, if not all, Numpy/Scipy users know by now to use @ or
np.dot (unfortunately, there's no way to override np.dot yet AFAIK).

__rmatmul__ is for when you do (for example) np.ndarray @ yourclass. In
this case, np.ndarray.__matmul__ returns NotImplemented, and then the
operator falls back to your __rmatmul__ method.


On Wed, Mar 21, 2018 at 11:35 AM, Aditya Bharti <adibhar97 at gmail.com> wrote:

> Hi Hameer,
> Thank you for your comments. They were quite informative. Just a couple of
> questions.
>
> The __matmul__ function translates to the @ operator correct? Would that
> not be somewhat awkward for new users?
> Also, I couldn't find the suggested __rmatmul__ anywhere on the operator
> list: https://docs.python.org/3/library/operator.html. Am I looking in
> the wrong place?
>
> In addition, are there any other specific things I need to stay away from
> to avoid np.matrix like problems? I just know that the matrix class is not
> used much, I don't really know the reasons for its deprecation.
>
> Thanks,
> Aditya
>
> On 21 March 2018 at 15:52, Hameer Abbasi <einstein.edison at gmail.com>
> wrote:
>
>> Hi Aditya,
>>
>> I've left a few comments regarding composing and np.matrix. Using __mul__
>> for anything other than real multiplication is ill-advised and caused
>> problems before with np.matrix. I suggest you use __matmul__ instead.
>>
>> Hameer
>>
>>
>> On Wed, Mar 21, 2018 at 9:40 AM, Aditya Bharti <adibhar97 at gmail.com>
>> wrote:
>>
>>> Hi all,
>>> Thank you for your helpful comments. I've updated my proposal with the
>>> following major changes:
>>>
>>>    - Fleshed out the API in more detail
>>>    - Included a github link to a sample implementation of the
>>>    initializer functions
>>>    - Proposed a quaternion class for faster compositions of rotations
>>>    (use of this class will be completely optional and not required for
>>>    Rotation() functionality)
>>>
>>> Thank you for all your help so far. I look forward to hearing from you.
>>>
>>> Regards,
>>> Aditya
>>>
>>> https://docs.google.com/document/d/1UriyLABwgjUcYfBofSr4hqAn
>>> LzblJ71pkzm7i26O8ws/edit?usp=sharing
>>>
>>> On 21 March 2018 at 01:01, Eric Larson <larson.eric.d at gmail.com> wrote:
>>>
>>>> I second Ralf's comments, and have added a few, too.
>>>>
>>>> Eric
>>>>
>>>>
>>>> On Tue, Mar 20, 2018 at 11:56 AM, Ralf Gommers <ralf.gommers at gmail.com>
>>>> wrote:
>>>>
>>>>>
>>>>>
>>>>> On Mon, Mar 19, 2018 at 1:05 PM, Aditya Bharti <adibhar97 at gmail.com>
>>>>> wrote:
>>>>>
>>>>>> Hi all,
>>>>>>
>>>>>> Goal of this email: To ask for feedback on the draft proposal for
>>>>>> GSOC 2018
>>>>>>
>>>>>> I am an undergraduate researcher in Computer Vision interested in
>>>>>> contributing to scipy's Rotation Formalism in 3 Dimensions project idea as
>>>>>> part of GSOC 2018.
>>>>>>
>>>>>> I've made some contributions to scipy and have drafted a proposal
>>>>>> here
>>>>>> <https://docs.google.com/document/d/1UriyLABwgjUcYfBofSr4hqAnLzblJ71pkzm7i26O8ws/edit?usp=sharing>.
>>>>>> [1]
>>>>>>
>>>>>> It includes my contact information, code samples, project timeline,
>>>>>> and an introduction about myself. I went through the proposal guidelines on
>>>>>> the website and think I covered all my bases.
>>>>>>
>>>>>> I would greatly appreciate your feedback on this as it will help me
>>>>>> in making a strong application.
>>>>>>
>>>>>
>>>>> Hi Aditya, that looks like a solid start. I added a few comments; will
>>>>> leave it to the proposed mentors to comment on some of the algorithmic
>>>>> ideas. One other thing to consider is existing implementations to possibly
>>>>> build upon - a couple of people suggested other packages on this list not
>>>>> too long ago.
>>>>>
>>>>> Cheers,
>>>>> Ralf
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>>
>>>>>> Thank you,
>>>>>> Aditya Bharti
>>>>>>
>>>>>> [1]: https://docs.google.com/document/d/1UriyLABwgjUcYfBofSr4hqAn
>>>>>> LzblJ71pkzm7i26O8ws/edit?usp=sharing
>>>>>>
>>>>>> _______________________________________________
>>>>>> SciPy-Dev mailing list
>>>>>> SciPy-Dev at python.org
>>>>>> https://mail.python.org/mailman/listinfo/scipy-dev
>>>>>>
>>>>>>
>>>>>
>>>>> _______________________________________________
>>>>> SciPy-Dev mailing list
>>>>> SciPy-Dev at python.org
>>>>> https://mail.python.org/mailman/listinfo/scipy-dev
>>>>>
>>>>>
>>>>
>>>> _______________________________________________
>>>> SciPy-Dev mailing list
>>>> SciPy-Dev at python.org
>>>> https://mail.python.org/mailman/listinfo/scipy-dev
>>>>
>>>>
>>>
>>> _______________________________________________
>>> SciPy-Dev mailing list
>>> SciPy-Dev at python.org
>>> https://mail.python.org/mailman/listinfo/scipy-dev
>>>
>>>
>>
>> _______________________________________________
>> SciPy-Dev mailing list
>> SciPy-Dev at python.org
>> https://mail.python.org/mailman/listinfo/scipy-dev
>>
>>
>
> _______________________________________________
> SciPy-Dev mailing list
> SciPy-Dev at python.org
> https://mail.python.org/mailman/listinfo/scipy-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scipy-dev/attachments/20180321/9571b2b5/attachment.html>

From einstein.edison at gmail.com  Wed Mar 21 07:58:42 2018
From: einstein.edison at gmail.com (Hameer Abbasi)
Date: Wed, 21 Mar 2018 12:58:42 +0100
Subject: [SciPy-Dev] Job prospects and a paying SciPy Dev in Germany?
Message-ID: <CADViA5C+6NxyzDtjeti1aGOyUM=4_RfEb4QCM9LYk66OOshA5g@mail.gmail.com>

Hello everyone,

Sorry to put a personal question into this list (if you feel bothered,
please feel free to ignore it or reprimand me), but I couldn't find a
better place to ask.

I was looking to do development for Numpy/Scipy full-time in Germany,
preferably as a PhD candidate but a position at a company works as well.

Is anyone aware of any such opportunities in Germany (or the EU at large)?

If so, let me know.

Hameer Abbasi
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scipy-dev/attachments/20180321/93b8519c/attachment.html>

From ralf.gommers at gmail.com  Wed Mar 21 11:32:42 2018
From: ralf.gommers at gmail.com (Ralf Gommers)
Date: Wed, 21 Mar 2018 08:32:42 -0700
Subject: [SciPy-Dev] SciPy dependencies
In-Reply-To: <CAFq4t4gh50XVoGLcZo0J8g0J8jVXJ4XEHfBhZ7FeLaB44MoVMg@mail.gmail.com>
References: <CAFq4t4gh50XVoGLcZo0J8g0J8jVXJ4XEHfBhZ7FeLaB44MoVMg@mail.gmail.com>
Message-ID: <CABL7CQgeD3qvpro79hbUDUzPshcyOqN7t1Dk_QN8_vWp=qHWJg@mail.gmail.com>

On Wed, Mar 21, 2018 at 12:04 AM, Peter Balazovic <balazovic.peter at gmail.com
> wrote:

> Dears,
>
> I am trying to get installed SciPy under Yocto. I encounter compile error:
>
> ERROR: scipy-1.0.0-r0 do_compile: Function failed: do_compile (log file is located at ../tmp/work/cortexa9hf-neon-poky-linux-gnueabi/scipy/1.0.0-r0/temp/log.do_compile.5023)
>     Log data follows:| DEBUG: Executing shell function do_compile
> | ERROR: python setup.py build execution failed.| ../tmp/sysroots/x86_64-linux/usr/lib/python2.7/distutils/dist.py:267: UserWarning: Unknown distribution option: 'python_requires'|   warnings.warn(msg)| lapack_opt_info:| openblas_lapack_info:|   libraries openblas not found in ['../tmp/sysroots/x86_64-linux/usr/lib']|   NOT AVAILABLE
>
> The SciPy install setup.py looks for openBLAS in different directory
> *x86_64-linux* instead of *target-arm* directory.
> I have already installed openBLAS and it is located at *target-arm*
>  sysroot.
>
> ./tmp/sysroots/target-arm/usr/lib/libopenblas.so
> ./tmp/sysroots/target-arm/usr/lib/libopenblas.a
> ./tmp/sysroots/target-arm/usr/include/openblas_config.h
>
> With building Yocto there are three "build" directories *target-arm* (target
> sysroot), *target-arm-tcbootstrap* (intermediate directory for the
> compiler bootstrap), and *x86_64-linux* (host tools and libraries). The
> openBLAS is installed within *target-arm* sysroot.
>
> My question is how to tell and redirect SciPy installer to look for
> openBLAS (and other dependencies) at *target-arm* directory (
> *./tmp/sysroots/target-arm/*)?
>

Look at the example in site.cfg.example in the root dir of the repo, there
you can specify paths.

Cheers,
Ralf
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scipy-dev/attachments/20180321/be3c2fbf/attachment-0001.html>

From smkjain8 at gmail.com  Wed Mar 21 14:30:18 2018
From: smkjain8 at gmail.com (Samyak Jain)
Date: Thu, 22 Mar 2018 00:00:18 +0530
Subject: [SciPy-Dev] Feedback for the proposal
Message-ID: <CABbcdpDdwwwdqpVRwvw6qXzop7=cH891DKK658LB_JsbxgPpmA@mail.gmail.com>

Hi all,

I am a 2nd-year student contributing towards SciPy's Rotation Formalism in
3 dimensions project as a part of GSOC'18.

I have drafted a proposal here.
<https://docs.google.com/document/d/12cP0LTFa0H3aQqJfb1WAmr5A_KU6bN3qZ17LbKd5YzM/edit?usp=sharing>

I would appreciate any feedback on this so that I can improve my proposal.

Thanks,
Samyak Jain

Link :
https://docs.google.com/document/d/12cP0LTFa0H3aQqJfb1WAmr5A_KU6bN3qZ17LbKd5YzM/edit?usp=sharing
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scipy-dev/attachments/20180322/6c4cdea8/attachment.html>

From mikofski at berkeley.edu  Wed Mar 21 15:31:07 2018
From: mikofski at berkeley.edu (Mark Alexander Mikofski)
Date: Wed, 21 Mar 2018 12:31:07 -0700
Subject: [SciPy-Dev] Benchmark for scalar Newton
In-Reply-To: <CAEqRcW2AscAJE-AAqokUxusvErJanX3iMEsDaq9smgmaMnMU=w@mail.gmail.com>
References: <CAEqRcW2hXg-2iX17iTnQKwx0M9FMacf0F-F3+De0C9vRUNBQuA@mail.gmail.com>
 <CAEqRcW2c7j97vozAOW2eV8yviRr+E3mth26uQ9g+JYSr-aBbNw@mail.gmail.com>
 <CAEqRcW2yTdA48+O5THJifoAfhKa24thEppr74gzFKs8UxFykMQ@mail.gmail.com>
 <CAEqRcW3H7Eq5OtSK8FFdqChG2HdTTKY6-VaKmFj7sm_tEAUkvg@mail.gmail.com>
 <CAEqRcW1vi6qGrdiiLVp_42AimOxE27Z-tjVY3AFdYsHLkhSrXQ@mail.gmail.com>
 <CAEqRcW2Ok5D6wu6S+cJ4YuaJmJeS_tdqBZHvvckJE=kKOFppLg@mail.gmail.com>
 <CAEqRcW2AscAJE-AAqokUxusvErJanX3iMEsDaq9smgmaMnMU=w@mail.gmail.com>
Message-ID: <CAEqRcW0vYc-hMoUMapAHAZ1qg61cv1t92Y5Lr-U2068FvSK0Lg@mail.gmail.com>

Hi all,

I've added to the airspeed velocity benchmarks. There were only benchmarks
for the pure C methods, so I added one for the scalar Newton method in
scipy.optimize.zeros. if you want to compare it with the other zeros use
"f1", a parabola, which is equivalent to "f2" for the other zeros.

If it's convenient for you please consider reviewing scipy PR #8587 . I
look forward to your comments.

Best regards,

Mark Mikofski
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scipy-dev/attachments/20180321/c4c43b19/attachment.html>

From anubhavp28 at gmail.com  Thu Mar 22 10:43:13 2018
From: anubhavp28 at gmail.com (Anubhav Patel)
Date: Thu, 22 Mar 2018 20:13:13 +0530
Subject: [SciPy-Dev] Contributing to SciPy through GSoC
In-Reply-To: <161f2f6d6b6.fb3ea05a10736.6245209566181213756@zoho.com>
References: <161f2f6d6b6.fb3ea05a10736.6245209566181213756@zoho.com>
Message-ID: <CADtkFLGYCeD8Atj5CkRN6mR_ZUre71HhQrS4UNstH3-MMuEs8Q@mail.gmail.com>

Hi,
I am seeking feedback for my GSoC Proposal for Rotation Formalism in 3
Dimensions -
https://docs.google.com/document/d/1ylzugkvVYI7m3IXsWVLD4EkE6zQd8B9YDPKtTno9f74/edit?usp=sharing


On Mon, Mar 5, 2018 at 3:11 AM, Nikolay Mayorov <nikolay.mayorov at zoho.com>
wrote:

> Hi, Anybhev!
>
> I think getting Rotation right is a top priority and so far unfortunately
> nobody dig into technical details of it. I would like to see that from
> students.
>
> As for the algorithms. I believe that Wahba's problem can be generalized
> by adding a translation vector, but the interpretation will be different
> (search for "absolute orientation problem"). As for methods to solve
> Wahba's problem --- probably SVD based is the easiest to understand and
> implement, but if we decide to use quaternions as the base representation,
> then we can go with "Q-method".
>
> "Cubic spline" for orientation is also a very cool algorithm which wasn't
> promoted anywhere, but this is the best idea I found on the subject (i.e.
> interpolation with continuous angular rates and acceleration).
>
> Other algorithms are quite small, they can be added quickly. I would say
> it is more of a question of what we should include. If you have some ideas
> outside (mostly) "aerospace" field --- they are welcome.
>
> For all parts I would like to see more concrete and technical details. For
> example, if we want SLERP interpolation --- what will it be (class or
> function), what it will accept, what will be the most difficult part to
> implement it correctly. The same for all other things.
>
> Best,
> Nikolay
>
>
> ---- On Sat, 03 Mar 2018 16:26:28 +0500* anubhavp28 at gmail.com
> <anubhavp28 at gmail.com> * wrote ----
>
> Hi,
> I wanted feedback regarding whether a combination of rotation class and
> implementation of quaternion SLERP algorithm and Davenport's Q-method
> solving Wahba's Problem, will be enough for GSoC? Should I include more
> rotation related algorithm for implementation? Any suggestions what more I
> could do?
>
> On Mon, Feb 26, 2018 at 9:30 PM, Ralf Gommers <ralf.gommers at gmail.com>
> wrote:
>
> Hi Anubhev,
>
> On Mon, Feb 26, 2018 at 2:12 AM, Anubhav Patel <anubhavp28 at gmail.com>
> wrote:
>
> Hi everyone,
> I want to work on SciPy as part of GSoC and I have few queries.
>
> 1. On the Ideas Page, there was a mention of scipy.spatial.transform
> module. I want to know what will be the exact purpose of this module?
>
>
> Did you read the whole idea? There's a lot of detail. It says for example
> "The aim of this project is to create a module which will allow to
> conveniently describe, apply and compose rotations. ". That answer your
> question I think.
>
>
>
> 2. Whether the idea for a module for numerical differentiation was dropped
> completely?
>
>
> Yes, for now that's off the table - at least not feasible for a GSoC we've
> concluded after several attempts.
>
>
>
> 3. Apart from those ideas listed on ideas page, are there any other area
> where you guys would like to see contribution on?
>
>
> Ideas for new features on http://scipy.github.io/devdocs/roadmap.html are
> of interest, or ones you may have yourself. But given that they're not on
> the ideas page, it's not guaranteed we can find mentors for those.
> Cheers,
> Ralf
>
>
> _______________________________________________
> SciPy-Dev mailing list
> SciPy-Dev at python.org
> https://mail.python.org/mailman/listinfo/scipy-dev
>
>
> _______________________________________________
> SciPy-Dev mailing list
> SciPy-Dev at python.org
> https://mail.python.org/mailman/listinfo/scipy-dev
>
>
>
>
> _______________________________________________
> SciPy-Dev mailing list
> SciPy-Dev at python.org
> https://mail.python.org/mailman/listinfo/scipy-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scipy-dev/attachments/20180322/3da43738/attachment.html>

From chaman.ag at gmail.com  Thu Mar 22 14:51:59 2018
From: chaman.ag at gmail.com (Chaman Agrawal)
Date: Fri, 23 Mar 2018 00:21:59 +0530
Subject: [SciPy-Dev] =?utf-8?q?Fast_Walsh=E2=80=93Hadamard_transform_for_?=
 =?utf-8?q?SciPy_=3F?=
Message-ID: <CAHLSkQSHCT-5_FMq4g3nMT_0yavzLh4-88nC35kHaeV53XfVGg@mail.gmail.com>

Issue #8590 https://github.com/scipy/scipy/issues/8590

Currently there is no implementation of Fast Walsh?Hadamard transform in
SciPy. Although it seems that FWHT is not as general as FFT but it is
pretty ubiquitous ,it is there is other maths and science softwares like
MATLAB etc. .I would like to contribute towards it. Following are the
details about it.
Fast Walsh?Hadamard transform

Hadamard transform is an example of a generalized class of Fourier
transforms. It performs an orthogonal, symmetric, involutive, linear
operation on 2^(m) numbers. It is equivalent to a multidimensional DFT.
Time Complexity:

O(nlogn) with Fast Walsh-Hadamard transform (FWHT)
Comparison with FFT:

FWHT is very useful for reducing bandwidth storage requirements and
spread-spectrum analysis. Compared to the FFT, the FWHT requires less
storage space and is faster to calculate because it uses only real
additions and subtractions, while the FFT requires complex values. The FWHT
is able to represent signals with sharp discontinuities more accurately
using fewer coefficients than the FFT.
Some Usage examples:

The Hadamard transform is used in data encryption, as well as many signal
processing and data compression algorithms, such as JPEG XR and MPEG-4 AVC.
It is also a crucial part of Grover's algorithm and Shor's algorithm in
quantum computing. The Hadamard transform is also applied in scientific
methods such as NMR, mass spectroscopy and crystallography.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scipy-dev/attachments/20180323/64d0f47d/attachment.html>

From ndbecker2 at gmail.com  Thu Mar 22 14:57:05 2018
From: ndbecker2 at gmail.com (Neal Becker)
Date: Thu, 22 Mar 2018 18:57:05 +0000
Subject: [SciPy-Dev] 
	=?utf-8?q?Fast_Walsh=E2=80=93Hadamard_transform_for_?=
	=?utf-8?q?SciPy_=3F?=
In-Reply-To: <CAHLSkQSHCT-5_FMq4g3nMT_0yavzLh4-88nC35kHaeV53XfVGg@mail.gmail.com>
References: <CAHLSkQSHCT-5_FMq4g3nMT_0yavzLh4-88nC35kHaeV53XfVGg@mail.gmail.com>
Message-ID: <CAG3t+pGsuuvLqqb7BN+nrw7evbL5wzKC=0Otikst2Omdm6wdSA@mail.gmail.com>

I have my own code for FHT if you are interested

On Thu, Mar 22, 2018 at 2:52 PM Chaman Agrawal <chaman.ag at gmail.com> wrote:

> Issue #8590 https://github.com/scipy/scipy/issues/8590
>
> Currently there is no implementation of Fast Walsh?Hadamard transform in
> SciPy. Although it seems that FWHT is not as general as FFT but it is
> pretty ubiquitous ,it is there is other maths and science softwares like
> MATLAB etc. .I would like to contribute towards it. Following are the
> details about it.
> Fast Walsh?Hadamard transform
>
> Hadamard transform is an example of a generalized class of Fourier
> transforms. It performs an orthogonal, symmetric, involutive, linear
> operation on 2^(m) numbers. It is equivalent to a multidimensional DFT.
> Time Complexity:
>
> O(nlogn) with Fast Walsh-Hadamard transform (FWHT)
> Comparison with FFT:
>
> FWHT is very useful for reducing bandwidth storage requirements and
> spread-spectrum analysis. Compared to the FFT, the FWHT requires less
> storage space and is faster to calculate because it uses only real
> additions and subtractions, while the FFT requires complex values. The FWHT
> is able to represent signals with sharp discontinuities more accurately
> using fewer coefficients than the FFT.
> Some Usage examples:
>
> The Hadamard transform is used in data encryption, as well as many signal
> processing and data compression algorithms, such as JPEG XR and MPEG-4 AVC.
> It is also a crucial part of Grover's algorithm and Shor's algorithm in
> quantum computing. The Hadamard transform is also applied in scientific
> methods such as NMR, mass spectroscopy and crystallography.
>
>
> _______________________________________________
> SciPy-Dev mailing list
> SciPy-Dev at python.org
> https://mail.python.org/mailman/listinfo/scipy-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scipy-dev/attachments/20180322/e4250089/attachment.html>

From chaman.ag at gmail.com  Thu Mar 22 15:39:13 2018
From: chaman.ag at gmail.com (Chaman Agrawal)
Date: Fri, 23 Mar 2018 01:09:13 +0530
Subject: [SciPy-Dev] =?utf-8?q?Fast_Walsh=E2=80=93Hadamard_transform_for_?=
 =?utf-8?q?SciPy_=3F?=
Message-ID: <CAHLSkQQjFfecJSA-6T8CT3jyrPcm=nVSVGn23sV_ZiEBkF16ew@mail.gmail.com>

Hello,

Thanks, please send the link ,it would be good to have multiple references.

However the important part is where to implement this ,in fftpack or signal
as Ralf Gommers <https://github.com/rgommers> mentioned in the issue page.

Cheers,
Chaman
I have my own code for FHT if you are interested

On Thu, Mar 22, 2018 at 2:52 PM Chaman Agrawal <chaman.ag at gmail.com
<https://mail.python.org/mailman/listinfo/scipy-dev>> wrote:

>* Issue #8590 https://github.com/scipy/scipy/issues/8590 <https://github.com/scipy/scipy/issues/8590>
*>>* Currently there is no implementation of Fast Walsh?Hadamard transform in
*>* SciPy. Although it seems that FWHT is not as general as FFT but it is
*>* pretty ubiquitous ,it is there is other maths and science softwares like
*>* MATLAB etc. .I would like to contribute towards it. Following are the
*>* details about it.
*>* Fast Walsh?Hadamard transform
*>>* Hadamard transform is an example of a generalized class of Fourier
*>* transforms. It performs an orthogonal, symmetric, involutive, linear
*>* operation on 2^(m) numbers. It is equivalent to a multidimensional DFT.
*>* Time Complexity:
*>>* O(nlogn) with Fast Walsh-Hadamard transform (FWHT)
*>* Comparison with FFT:
*>>* FWHT is very useful for reducing bandwidth storage requirements and
*>* spread-spectrum analysis. Compared to the FFT, the FWHT requires less
*>* storage space and is faster to calculate because it uses only real
*>* additions and subtractions, while the FFT requires complex values. The FWHT
*>* is able to represent signals with sharp discontinuities more accurately
*>* using fewer coefficients than the FFT.
*>* Some Usage examples:
*>>* The Hadamard transform is used in data encryption, as well as many signal
*>* processing and data compression algorithms, such as JPEG XR and MPEG-4 AVC.
*>* It is also a crucial part of Grover's algorithm and Shor's algorithm in
*>* quantum computing. The Hadamard transform is also applied in scientific
*>* methods such as NMR, mass spectroscopy and crystallography.
*>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scipy-dev/attachments/20180323/39ca4a04/attachment-0001.html>

From sudheerachary115 at gmail.com  Thu Mar 22 23:44:11 2018
From: sudheerachary115 at gmail.com (sudheer achary)
Date: Fri, 23 Mar 2018 03:44:11 +0000
Subject: [SciPy-Dev] regarding issue #8589
Message-ID: <CAD4rTgng6cEs5=YXTbgZ3gwJkhgkLaJ7kou3s_XV8LCXOXw2jw@mail.gmail.com>

I would like to refactor the code as mentioned in
https://github.com/scipy/scipy/issues/8589 are there any issues with the
given changes made ?
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scipy-dev/attachments/20180323/9e261bed/attachment.html>

From ralf.gommers at gmail.com  Fri Mar 23 00:30:49 2018
From: ralf.gommers at gmail.com (Ralf Gommers)
Date: Thu, 22 Mar 2018 21:30:49 -0700
Subject: [SciPy-Dev] Feedback for the proposal
In-Reply-To: <CABbcdpDdwwwdqpVRwvw6qXzop7=cH891DKK658LB_JsbxgPpmA@mail.gmail.com>
References: <CABbcdpDdwwwdqpVRwvw6qXzop7=cH891DKK658LB_JsbxgPpmA@mail.gmail.com>
Message-ID: <CABL7CQjodr2xiPMR9pJyReSqST2zKc7fY9ttuRi+jd-NmkU6mw@mail.gmail.com>

Hi Samyak,

On Wed, Mar 21, 2018 at 11:30 AM, Samyak Jain <smkjain8 at gmail.com> wrote:

> Hi all,
>
> I am a 2nd-year student contributing towards SciPy's Rotation Formalism in
> 3 dimensions project as a part of GSOC'18.
>
> I have drafted a proposal here.
> <https://docs.google.com/document/d/12cP0LTFa0H3aQqJfb1WAmr5A_KU6bN3qZ17LbKd5YzM/edit?usp=sharing>
>
> I would appreciate any feedback on this so that I can improve my proposal.
>

That's a decent start, although I feel it's still a little light context.
One of the things I'm missing in your proposal (as well as in others'
proposals) is references to and discussion on existing implementations and
what can be reused / why they're insufficient. See
https://mail.python.org/pipermail/scipy-dev/2018-February/022534.html for a
recent discussion on this.

Cheers,
Ralf




> Thanks,
> Samyak Jain
>
> Link : https://docs.google.com/document/d/12cP0LTFa0H3aQqJfb1WAmr5A_
> KU6bN3qZ17LbKd5YzM/edit?usp=sharing
>
> _______________________________________________
> SciPy-Dev mailing list
> SciPy-Dev at python.org
> https://mail.python.org/mailman/listinfo/scipy-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scipy-dev/attachments/20180322/9c90631a/attachment.html>

From ralf.gommers at gmail.com  Fri Mar 23 00:39:36 2018
From: ralf.gommers at gmail.com (Ralf Gommers)
Date: Thu, 22 Mar 2018 21:39:36 -0700
Subject: [SciPy-Dev] GSoC Rotation Formalism Proposal
In-Reply-To: <CAEkj7mJrzVTSfFhX=LFLKu5DBfNNEk1Y5mgtMUndnNXW_ngbzw@mail.gmail.com>
References: <CAEkj7mJrzVTSfFhX=LFLKu5DBfNNEk1Y5mgtMUndnNXW_ngbzw@mail.gmail.com>
Message-ID: <CABL7CQgT7xYZ6hV+mwgtHtdDaUvBXCr=vEd-A7ySXyiEEZ3LQQ@mail.gmail.com>

On Tue, Mar 20, 2018 at 11:46 PM, Vishal Gupta <vishstar88 at gmail.com> wrote:

> Hey,
>
> I am a 3rd CSE undergrad at SSNCE, Chennai, India. I'd like to work on
> Rotation Formalism in 3 Dimensions as a part of this year's GSoC .
>
> https://docs.google.com/document/d/1y5OalGAvYkk8UvLtFjQ2-xhRj_
> NTwq0fV3GvlCBUp0I/edit?usp=sharing
>
>
Thanks for sharing Vishal. I put in a few initial comments that should help
fulfill the formal submission requirements.

Ralf



> Thanks,
> Vishal Gupta
>
> ?
> <https://mailtrack.io/> Sent with Mailtrack
> <https://mailtrack.io?utm_source=gmail&utm_medium=signature&utm_campaign=signaturevirality&>
>
> _______________________________________________
> SciPy-Dev mailing list
> SciPy-Dev at python.org
> https://mail.python.org/mailman/listinfo/scipy-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scipy-dev/attachments/20180322/47536806/attachment.html>

From ralf.gommers at gmail.com  Fri Mar 23 00:42:18 2018
From: ralf.gommers at gmail.com (Ralf Gommers)
Date: Thu, 22 Mar 2018 21:42:18 -0700
Subject: [SciPy-Dev] 
	=?utf-8?q?Fast_Walsh=E2=80=93Hadamard_transform_for_?=
	=?utf-8?q?SciPy_=3F?=
In-Reply-To: <CAHLSkQQjFfecJSA-6T8CT3jyrPcm=nVSVGn23sV_ZiEBkF16ew@mail.gmail.com>
References: <CAHLSkQQjFfecJSA-6T8CT3jyrPcm=nVSVGn23sV_ZiEBkF16ew@mail.gmail.com>
Message-ID: <CABL7CQgNhB45dgBZe3-GGJVxXK5oa32Mu4OZ2eSd25gDV2r98Q@mail.gmail.com>

On Thu, Mar 22, 2018 at 12:39 PM, Chaman Agrawal <chaman.ag at gmail.com>
wrote:

> Hello,
>

Hi Chaman, your email client seems to be misconfigured, you're starting new
threads. Can you please have a look at changing that?

Cheers,
Ralf


> Thanks, please send the link ,it would be good to have multiple references.
>
> However the important part is where to implement this ,in fftpack or
> signal as Ralf Gommers <https://github.com/rgommers> mentioned in the
> issue page.
>
> Cheers,
> Chaman
> I have my own code for FHT if you are interested
>
> On Thu, Mar 22, 2018 at 2:52 PM Chaman Agrawal <chaman.ag at gmail.com <https://mail.python.org/mailman/listinfo/scipy-dev>> wrote:
>
> >* Issue #8590 https://github.com/scipy/scipy/issues/8590 <https://github.com/scipy/scipy/issues/8590>
> *>>* Currently there is no implementation of Fast Walsh?Hadamard transform in
> *>* SciPy. Although it seems that FWHT is not as general as FFT but it is
> *>* pretty ubiquitous ,it is there is other maths and science softwares like
> *>* MATLAB etc. .I would like to contribute towards it. Following are the
> *>* details about it.
> *>* Fast Walsh?Hadamard transform
> *>>* Hadamard transform is an example of a generalized class of Fourier
> *>* transforms. It performs an orthogonal, symmetric, involutive, linear
> *>* operation on 2^(m) numbers. It is equivalent to a multidimensional DFT.
> *>* Time Complexity:
> *>>* O(nlogn) with Fast Walsh-Hadamard transform (FWHT)
> *>* Comparison with FFT:
> *>>* FWHT is very useful for reducing bandwidth storage requirements and
> *>* spread-spectrum analysis. Compared to the FFT, the FWHT requires less
> *>* storage space and is faster to calculate because it uses only real
> *>* additions and subtractions, while the FFT requires complex values. The FWHT
> *>* is able to represent signals with sharp discontinuities more accurately
> *>* using fewer coefficients than the FFT.
> *>* Some Usage examples:
> *>>* The Hadamard transform is used in data encryption, as well as many signal
> *>* processing and data compression algorithms, such as JPEG XR and MPEG-4 AVC.
> *>* It is also a crucial part of Grover's algorithm and Shor's algorithm in
> *>* quantum computing. The Hadamard transform is also applied in scientific
> *>* methods such as NMR, mass spectroscopy and crystallography.
> *>
>
>
>
> _______________________________________________
> SciPy-Dev mailing list
> SciPy-Dev at python.org
> https://mail.python.org/mailman/listinfo/scipy-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scipy-dev/attachments/20180322/e41d6666/attachment-0001.html>

From chaman.ag at gmail.com  Fri Mar 23 08:41:13 2018
From: chaman.ag at gmail.com (Chaman Agrawal)
Date: Fri, 23 Mar 2018 18:11:13 +0530
Subject: [SciPy-Dev] 
	=?utf-8?q?Fast_Walsh=E2=80=93Hadamard_transform_for_?=
	=?utf-8?q?SciPy_=3F?=
In-Reply-To: <CABL7CQgNhB45dgBZe3-GGJVxXK5oa32Mu4OZ2eSd25gDV2r98Q@mail.gmail.com>
References: <CAHLSkQQjFfecJSA-6T8CT3jyrPcm=nVSVGn23sV_ZiEBkF16ew@mail.gmail.com>
 <CABL7CQgNhB45dgBZe3-GGJVxXK5oa32Mu4OZ2eSd25gDV2r98Q@mail.gmail.com>
Message-ID: <CAHLSkQS6bp=YZSm8Nni_ThRGhRjj1Ja6zs_HHeaHs8BR4sySkg@mail.gmail.com>

Hello everyone,

Reference : Issue #8590 <https://github.com/scipy/scipy/issues/8590>

I wanted to get started with implementing FWHT so I wanted to ask few
things please help.

1) Will it be good to implement it in FFT pack or it should be in the
signal pack.

2) Can I implement it in C or Cython as I am not familiar with Fortran so I
can not implement it like FFT is implemented.

3) If I can implement it in C or Cython please give me an overview of the
structure for the implementation to help me getting started.

If there is any other suggestion/guidance that I might have forgot to ask
for,please provide it. It would be of great help.

Thankyou,
Chaman

On Fri, Mar 23, 2018 at 10:12 AM, Ralf Gommers <ralf.gommers at gmail.com>
wrote:

>
>
> On Thu, Mar 22, 2018 at 12:39 PM, Chaman Agrawal <chaman.ag at gmail.com>
> wrote:
>
>> Hello,
>>
>
> Hi Chaman, your email client seems to be misconfigured, you're starting
> new threads. Can you please have a look at changing that?
>
> Cheers,
> Ralf
>
>
>> Thanks, please send the link ,it would be good to have multiple
>> references.
>>
>> However the important part is where to implement this ,in fftpack or
>> signal as Ralf Gommers <https://github.com/rgommers> mentioned in the
>> issue page.
>>
>> Cheers,
>> Chaman
>> I have my own code for FHT if you are interested
>>
>> On Thu, Mar 22, 2018 at 2:52 PM Chaman Agrawal <chaman.ag at gmail.com <https://mail.python.org/mailman/listinfo/scipy-dev>> wrote:
>>
>> >* Issue #8590 https://github.com/scipy/scipy/issues/8590 <https://github.com/scipy/scipy/issues/8590>
>> *>>* Currently there is no implementation of Fast Walsh?Hadamard transform in
>> *>* SciPy. Although it seems that FWHT is not as general as FFT but it is
>> *>* pretty ubiquitous ,it is there is other maths and science softwares like
>> *>* MATLAB etc. .I would like to contribute towards it. Following are the
>> *>* details about it.
>> *>* Fast Walsh?Hadamard transform
>> *>>* Hadamard transform is an example of a generalized class of Fourier
>> *>* transforms. It performs an orthogonal, symmetric, involutive, linear
>> *>* operation on 2^(m) numbers. It is equivalent to a multidimensional DFT.
>> *>* Time Complexity:
>> *>>* O(nlogn) with Fast Walsh-Hadamard transform (FWHT)
>> *>* Comparison with FFT:
>> *>>* FWHT is very useful for reducing bandwidth storage requirements and
>> *>* spread-spectrum analysis. Compared to the FFT, the FWHT requires less
>> *>* storage space and is faster to calculate because it uses only real
>> *>* additions and subtractions, while the FFT requires complex values. The FWHT
>> *>* is able to represent signals with sharp discontinuities more accurately
>> *>* using fewer coefficients than the FFT.
>> *>* Some Usage examples:
>> *>>* The Hadamard transform is used in data encryption, as well as many signal
>> *>* processing and data compression algorithms, such as JPEG XR and MPEG-4 AVC.
>> *>* It is also a crucial part of Grover's algorithm and Shor's algorithm in
>> *>* quantum computing. The Hadamard transform is also applied in scientific
>> *>* methods such as NMR, mass spectroscopy and crystallography.
>> *>
>>
>>
>>
>> _______________________________________________
>> SciPy-Dev mailing list
>> SciPy-Dev at python.org
>> https://mail.python.org/mailman/listinfo/scipy-dev
>>
>>
>
> _______________________________________________
> SciPy-Dev mailing list
> SciPy-Dev at python.org
> https://mail.python.org/mailman/listinfo/scipy-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scipy-dev/attachments/20180323/1fb5707e/attachment.html>

From ndbecker2 at gmail.com  Fri Mar 23 08:59:27 2018
From: ndbecker2 at gmail.com (Neal Becker)
Date: Fri, 23 Mar 2018 12:59:27 +0000
Subject: [SciPy-Dev] 
	=?utf-8?q?Fast_Walsh=E2=80=93Hadamard_transform_for_?=
	=?utf-8?q?SciPy_=3F?=
In-Reply-To: <CAHLSkQS6bp=YZSm8Nni_ThRGhRjj1Ja6zs_HHeaHs8BR4sySkg@mail.gmail.com>
References: <CAHLSkQQjFfecJSA-6T8CT3jyrPcm=nVSVGn23sV_ZiEBkF16ew@mail.gmail.com>
 <CABL7CQgNhB45dgBZe3-GGJVxXK5oa32Mu4OZ2eSd25gDV2r98Q@mail.gmail.com>
 <CAHLSkQS6bp=YZSm8Nni_ThRGhRjj1Ja6zs_HHeaHs8BR4sySkg@mail.gmail.com>
Message-ID: <CAG3t+pERkXvELyRFv64x4HotY_H2krjV8vPvBjig0gYAFPCBNQ@mail.gmail.com>

I've placed my implementation here:
https://gist.github.com/nbecker/aa4d2297f36db842a704ea44dadbf8fe

This is written to use the ndarray C++ python interface library.  I don't
expect you to reuse the code directly (since that introduces dependencies
that aren't in numpy/scipy), but you can use it as a reference.

On Fri, Mar 23, 2018 at 8:41 AM Chaman Agrawal <chaman.ag at gmail.com> wrote:

> Hello everyone,
>
> Reference : Issue #8590 <https://github.com/scipy/scipy/issues/8590>
>
> I wanted to get started with implementing FWHT so I wanted to ask few
> things please help.
>
> 1) Will it be good to implement it in FFT pack or it should be in the
> signal pack.
>
> 2) Can I implement it in C or Cython as I am not familiar with Fortran so
> I can not implement it like FFT is implemented.
>
> 3) If I can implement it in C or Cython please give me an overview of the
> structure for the implementation to help me getting started.
>
> If there is any other suggestion/guidance that I might have forgot to ask
> for,please provide it. It would be of great help.
>
> Thankyou,
> Chaman
>
> On Fri, Mar 23, 2018 at 10:12 AM, Ralf Gommers <ralf.gommers at gmail.com>
> wrote:
>
>>
>>
>> On Thu, Mar 22, 2018 at 12:39 PM, Chaman Agrawal <chaman.ag at gmail.com>
>> wrote:
>>
>>> Hello,
>>>
>>
>> Hi Chaman, your email client seems to be misconfigured, you're starting
>> new threads. Can you please have a look at changing that?
>>
>> Cheers,
>> Ralf
>>
>>
>>> Thanks, please send the link ,it would be good to have multiple
>>> references.
>>>
>>> However the important part is where to implement this ,in fftpack or
>>> signal as Ralf Gommers <https://github.com/rgommers> mentioned in the
>>> issue page.
>>>
>>> Cheers,
>>> Chaman
>>> I have my own code for FHT if you are interested
>>>
>>> On Thu, Mar 22, 2018 at 2:52 PM Chaman Agrawal <chaman.ag at gmail.com <https://mail.python.org/mailman/listinfo/scipy-dev>> wrote:
>>>
>>> >* Issue #8590 https://github.com/scipy/scipy/issues/8590 <https://github.com/scipy/scipy/issues/8590>
>>> *>>* Currently there is no implementation of Fast Walsh?Hadamard transform in
>>> *>* SciPy. Although it seems that FWHT is not as general as FFT but it is
>>> *>* pretty ubiquitous ,it is there is other maths and science softwares like
>>> *>* MATLAB etc. .I would like to contribute towards it. Following are the
>>> *>* details about it.
>>> *>* Fast Walsh?Hadamard transform
>>> *>>* Hadamard transform is an example of a generalized class of Fourier
>>> *>* transforms. It performs an orthogonal, symmetric, involutive, linear
>>> *>* operation on 2^(m) numbers. It is equivalent to a multidimensional DFT.
>>> *>* Time Complexity:
>>> *>>* O(nlogn) with Fast Walsh-Hadamard transform (FWHT)
>>> *>* Comparison with FFT:
>>> *>>* FWHT is very useful for reducing bandwidth storage requirements and
>>> *>* spread-spectrum analysis. Compared to the FFT, the FWHT requires less
>>> *>* storage space and is faster to calculate because it uses only real
>>> *>* additions and subtractions, while the FFT requires complex values. The FWHT
>>> *>* is able to represent signals with sharp discontinuities more accurately
>>> *>* using fewer coefficients than the FFT.
>>> *>* Some Usage examples:
>>> *>>* The Hadamard transform is used in data encryption, as well as many signal
>>> *>* processing and data compression algorithms, such as JPEG XR and MPEG-4 AVC.
>>> *>* It is also a crucial part of Grover's algorithm and Shor's algorithm in
>>> *>* quantum computing. The Hadamard transform is also applied in scientific
>>> *>* methods such as NMR, mass spectroscopy and crystallography.
>>> *>
>>>
>>>
>>>
>>> _______________________________________________
>>> SciPy-Dev mailing list
>>> SciPy-Dev at python.org
>>> https://mail.python.org/mailman/listinfo/scipy-dev
>>>
>>>
>>
>> _______________________________________________
>> SciPy-Dev mailing list
>> SciPy-Dev at python.org
>> https://mail.python.org/mailman/listinfo/scipy-dev
>>
>>
> _______________________________________________
> SciPy-Dev mailing list
> SciPy-Dev at python.org
> https://mail.python.org/mailman/listinfo/scipy-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scipy-dev/attachments/20180323/08772f1c/attachment-0001.html>

From emailforsayantan at gmail.com  Fri Mar 23 11:24:28 2018
From: emailforsayantan at gmail.com (Sayantan Majumdar)
Date: Fri, 23 Mar 2018 20:54:28 +0530
Subject: [SciPy-Dev] Thoughts on creating a toolbox for analysis of
 nonlinear dynamical system analysis
Message-ID: <CAJvfhXBfYkQFrgsioB9A37c6pQd7GUqzzdsmFn8najQFykaYzg@mail.gmail.com>

 I work with Nonlinear Dynamical Systems and there are no good toolboxes
that provide fast and stable tools for the analysis of such systems.

there were a lot of good toolboxes in the 80's( I heard ) and there are
some packages where those algorithms have been re-implemented and sometimes
used, like the PyDSTool package.But its too slow and everything is not
stable in it plus some important tools are not there.

This can be a good thing to start in this gsoc right?

-S
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scipy-dev/attachments/20180323/396478de/attachment.html>

From ralf.gommers at gmail.com  Fri Mar 23 11:28:33 2018
From: ralf.gommers at gmail.com (Ralf Gommers)
Date: Fri, 23 Mar 2018 08:28:33 -0700
Subject: [SciPy-Dev] Thoughts on creating a toolbox for analysis of
 nonlinear dynamical system analysis
In-Reply-To: <CAJvfhXBfYkQFrgsioB9A37c6pQd7GUqzzdsmFn8najQFykaYzg@mail.gmail.com>
References: <CAJvfhXBfYkQFrgsioB9A37c6pQd7GUqzzdsmFn8najQFykaYzg@mail.gmail.com>
Message-ID: <CABL7CQhydSWi2rGVx3QiZEYAiLgGriFBw0zCLpAcVfmaoWH+Lw@mail.gmail.com>

On Fri, Mar 23, 2018 at 8:24 AM, Sayantan Majumdar <
emailforsayantan at gmail.com> wrote:

> I work with Nonlinear Dynamical Systems and there are no good toolboxes
> that provide fast and stable tools for the analysis of such systems.
>
> there were a lot of good toolboxes in the 80's( I heard ) and there are
> some packages where those algorithms have been re-implemented and sometimes
> used, like the PyDSTool package.But its too slow and everything is not
> stable in it plus some important tools are not there.
>
> This can be a good thing to start in this gsoc right?
>

No, that's not a good topic for GSoC, because it's too large in scope and
because there's no mentoring project for it if you try to start something
from scratch.

Ralf


>
> -S
>
>
>
> _______________________________________________
> SciPy-Dev mailing list
> SciPy-Dev at python.org
> https://mail.python.org/mailman/listinfo/scipy-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scipy-dev/attachments/20180323/998e4714/attachment.html>

From joseph.c.slater at gmail.com  Fri Mar 23 11:39:53 2018
From: joseph.c.slater at gmail.com (Joseph Slater)
Date: Fri, 23 Mar 2018 11:39:53 -0400
Subject: [SciPy-Dev] Thoughts on creating a toolbox for analysis of
 nonlinear dynamical system analysis
In-Reply-To: <CABL7CQhydSWi2rGVx3QiZEYAiLgGriFBw0zCLpAcVfmaoWH+Lw@mail.gmail.com>
References: <CAJvfhXBfYkQFrgsioB9A37c6pQd7GUqzzdsmFn8najQFykaYzg@mail.gmail.com>
 <CABL7CQhydSWi2rGVx3QiZEYAiLgGriFBw0zCLpAcVfmaoWH+Lw@mail.gmail.com>
Message-ID: <ACA5E874-C4B0-42D7-9F20-277E115D7AE0@gmail.com>



> On Mar 23, 2018, at 11:28 AM, Ralf Gommers <ralf.gommers at gmail.com> wrote:
> 
> 
> 
> On Fri, Mar 23, 2018 at 8:24 AM, Sayantan Majumdar <emailforsayantan at gmail.com> wrote:
> I work with Nonlinear Dynamical Systems and there are no good toolboxes that provide fast and stable tools for the analysis of such systems.
> 
> there were a lot of good toolboxes in the 80's( I heard ) and there are some packages where those algorithms have been re-implemented and sometimes used, like the PyDSTool package.But its too slow and everything is not stable in it plus some important tools are not there.
> 
> This can be a good thing to start in this gsoc right?
> 
> No, that's not a good topic for GSoC, because it's too large in scope and because there's no mentoring project for it if you try to start something from scratch.
> 
> Ralf
>  

I concur with Ralf, while also agreeing with the need. I have taken a small step with my package mousai which at this time is a rather limited harmonic balance solver that certainly needs improvements for greater robustness (on github, and pypi- https://josephcslater.github.io/mousai/). I have spent more time adding issues than anything else lately. I would certainly welcome anyone who wants to join forces. At some time, merging into scipy would be a goal, but I don't believe that it's worthy right now. 

Ralf: There was a recent discussion on the frustrations of lack of SciPy contributors- mousai is an example, along with my other projects, some more relevant to potential future inclusion in SciPy, some less. 

I'm concerned with PyDSTool: I don't know that it is being maintained, and my observation was that the overhead for being able to use it is very high. My interest would be in connecting aerodynamic and structural codes (likely commercial) together. These could include millions of states. PyDSTool seemed to me to expect hand coded equations a bit much. Maybe it's just me, but it looks great but I thought the overhead to get into using it was off-putting for most engineers/students. 

Joe





From lagru at mailbox.org  Fri Mar 23 12:39:30 2018
From: lagru at mailbox.org (Lars G.)
Date: Fri, 23 Mar 2018 17:39:30 +0100
Subject: [SciPy-Dev] Moving slow parts of find_peaks to Cython
Message-ID: <138da1f9-e4df-411a-9e05-aa1ecaabb2f6@mailbox.org>

Hello,

I'm currently working on 3 PRs that move slow parts in `find_peaks`,
`peak_prominences` and `peak_widths` to Cython.

* ENH: Speed-up peak distance filtering with Cython
  https://github.com/scipy/scipy/pull/8523
* ENH: Cythonize peak_prominences
  https://github.com/scipy/scipy/pull/8541
* ENH: Cythonize peak_widths
  https://github.com/scipy/scipy/pull/8594

I quite happy with the first two PRs and think they have left the "work
in progress" state and are ready for some final polishing. I might still
tweak the third one here or there but nothing too drastic.

Because these 3 modify the same files I'd propose to merge in the given
order (if desired) so that I can rebase and resolve the arising merge
conflicts.

If any of you guys with more experience in Cython have tips to improve
the performance even further I'd be especially happy to learn. :D
As always, thank you for your time and feedback.

Best regards,
Lars


From ilhanpolat at gmail.com  Fri Mar 23 15:09:38 2018
From: ilhanpolat at gmail.com (Ilhan Polat)
Date: Fri, 23 Mar 2018 19:09:38 +0000
Subject: [SciPy-Dev] Thoughts on creating a toolbox for analysis of
 nonlinear dynamical system analysis
In-Reply-To: <ACA5E874-C4B0-42D7-9F20-277E115D7AE0@gmail.com>
References: <CAJvfhXBfYkQFrgsioB9A37c6pQd7GUqzzdsmFn8najQFykaYzg@mail.gmail.com>
 <CABL7CQhydSWi2rGVx3QiZEYAiLgGriFBw0zCLpAcVfmaoWH+Lw@mail.gmail.com>
 <ACA5E874-C4B0-42D7-9F20-277E115D7AE0@gmail.com>
Message-ID: <CAEBuzr9s9ULpP9MRDNS97MbjO-XV-kg9qtUH=rqNeE_SrUNyYA@mail.gmail.com>

My problem with nonlinear tools is that they are exceedingly generic; even
the name itself "anything not linear". This applies particularly to
dynamics and control. Many attempts get overwhelmed quickly over the
generalness though I wholeheartedly agree with the sentiment. A non-GUI,
simulink-like tool would be great indeed.

If the scope gets narrower and it does one thing or two sufficiently well
and quick then it might(!) find a place in, say, sp.integrate or sp.signal.
but otherwise it would be indeed out of scope.

Pydstool or BMSpy require really dedicated contributors because of the
specialization needed to improve the code. Hence my jerk knee response
would be first to get their reaction about the roadmap they have in mind.


On Fri, Mar 23, 2018, 16:40 Joseph Slater <joseph.c.slater at gmail.com> wrote:

>
>
> > On Mar 23, 2018, at 11:28 AM, Ralf Gommers <ralf.gommers at gmail.com>
> wrote:
> >
> >
> >
> > On Fri, Mar 23, 2018 at 8:24 AM, Sayantan Majumdar <
> emailforsayantan at gmail.com> wrote:
> > I work with Nonlinear Dynamical Systems and there are no good toolboxes
> that provide fast and stable tools for the analysis of such systems.
> >
> > there were a lot of good toolboxes in the 80's( I heard ) and there are
> some packages where those algorithms have been re-implemented and sometimes
> used, like the PyDSTool package.But its too slow and everything is not
> stable in it plus some important tools are not there.
> >
> > This can be a good thing to start in this gsoc right?
> >
> > No, that's not a good topic for GSoC, because it's too large in scope
> and because there's no mentoring project for it if you try to start
> something from scratch.
> >
> > Ralf
> >
>
> I concur with Ralf, while also agreeing with the need. I have taken a
> small step with my package mousai which at this time is a rather limited
> harmonic balance solver that certainly needs improvements for greater
> robustness (on github, and pypi- https://josephcslater.github.io/mousai/).
> I have spent more time adding issues than anything else lately. I would
> certainly welcome anyone who wants to join forces. At some time, merging
> into scipy would be a goal, but I don't believe that it's worthy right now.
>
> Ralf: There was a recent discussion on the frustrations of lack of SciPy
> contributors- mousai is an example, along with my other projects, some more
> relevant to potential future inclusion in SciPy, some less.
>
> I'm concerned with PyDSTool: I don't know that it is being maintained, and
> my observation was that the overhead for being able to use it is very high.
> My interest would be in connecting aerodynamic and structural codes (likely
> commercial) together. These could include millions of states. PyDSTool
> seemed to me to expect hand coded equations a bit much. Maybe it's just me,
> but it looks great but I thought the overhead to get into using it was
> off-putting for most engineers/students.
>
> Joe
>
>
>
>
> _______________________________________________
> SciPy-Dev mailing list
> SciPy-Dev at python.org
> https://mail.python.org/mailman/listinfo/scipy-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scipy-dev/attachments/20180323/f4081a36/attachment.html>

From sudheerachary115 at gmail.com  Fri Mar 23 19:20:41 2018
From: sudheerachary115 at gmail.com (sudheer achary)
Date: Fri, 23 Mar 2018 23:20:41 +0000
Subject: [SciPy-Dev] Review GSoC-2018 Proposal
Message-ID: <CAD4rTgkhw+D55Zwz9U-DK28yBcjP8gk2TnvzhuoAxfFzv2wwLg@mail.gmail.com>

I am Sudheer Achary, 3rd year B.Tech computer science student from
International Institute of Information technology, Hyderabad. Iam
interested in contributing to SciPy through GSoC-2k18. i kindly request to
review my proposal and suggest me appropriate changes need to me made, I
have little more to add and here i share my google doc.

thanks.
[image: Google Document]
Scipy-Proposal GSoC 2018 Sudheer Achary
<https://docs.google.com/document/d/13U4feVYTJJfGFgSHA37p_JxpexBKZJ2-VSjs4z7owcQ/edit?usp=drivesdk>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scipy-dev/attachments/20180323/270c5b93/attachment.html>

From phillip.m.feldman at gmail.com  Fri Mar 23 20:49:10 2018
From: phillip.m.feldman at gmail.com (Phillip Feldman)
Date: Fri, 23 Mar 2018 17:49:10 -0700
Subject: [SciPy-Dev] Review GSoC-2018 Proposal
In-Reply-To: <CAD4rTgkhw+D55Zwz9U-DK28yBcjP8gk2TnvzhuoAxfFzv2wwLg@mail.gmail.com>
References: <CAD4rTgkhw+D55Zwz9U-DK28yBcjP8gk2TnvzhuoAxfFzv2wwLg@mail.gmail.com>
Message-ID: <CAB2ViTaU1St7tXSeN6rKDe-U8QPCLR=ybQZ2MovLa1XYDhMxhg@mail.gmail.com>

Dear Sudheer,

This is an extraordinarily well thought-out and well-written proposal.  I
believe that this would be a very useful addition to SciPy.

I have a suggested change of wording.  Current:

"Implement the mechanism of switching the rotational forms (eg: Euler
angles to directional cosine matrices (DCM) etc)."

Suggested:

"Implement conversions among the various rotation representations, e.g.,
from Euler angles to directional cosine matrices (DCM) etc."

Yours,

Phillip

On Fri, Mar 23, 2018 at 4:20 PM, sudheer achary <sudheerachary115 at gmail.com>
wrote:

> I am Sudheer Achary, 3rd year B.Tech computer science student from
> International Institute of Information technology, Hyderabad. Iam
> interested in contributing to SciPy through GSoC-2k18. i kindly request to
> review my proposal and suggest me appropriate changes need to me made, I
> have little more to add and here i share my google doc.
>
> thanks.
> [image: Google Document]
> Scipy-Proposal GSoC 2018 Sudheer Achary
>
> <https://docs.google.com/document/d/13U4feVYTJJfGFgSHA37p_JxpexBKZJ2-VSjs4z7owcQ/edit?usp=drivesdk>
>
>
> _______________________________________________
> SciPy-Dev mailing list
> SciPy-Dev at python.org
> https://mail.python.org/mailman/listinfo/scipy-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scipy-dev/attachments/20180323/8ac6ad1e/attachment-0001.html>

From ralf.gommers at gmail.com  Sat Mar 24 02:03:36 2018
From: ralf.gommers at gmail.com (Ralf Gommers)
Date: Fri, 23 Mar 2018 23:03:36 -0700
Subject: [SciPy-Dev] Thoughts on creating a toolbox for analysis of
 nonlinear dynamical system analysis
In-Reply-To: <ACA5E874-C4B0-42D7-9F20-277E115D7AE0@gmail.com>
References: <CAJvfhXBfYkQFrgsioB9A37c6pQd7GUqzzdsmFn8najQFykaYzg@mail.gmail.com>
 <CABL7CQhydSWi2rGVx3QiZEYAiLgGriFBw0zCLpAcVfmaoWH+Lw@mail.gmail.com>
 <ACA5E874-C4B0-42D7-9F20-277E115D7AE0@gmail.com>
Message-ID: <CABL7CQgOVVa1h+8gYeQMLM8ULJ1=QwCZEDySrMZP_jSyrWT19w@mail.gmail.com>

On Fri, Mar 23, 2018 at 8:39 AM, Joseph Slater <joseph.c.slater at gmail.com>
wrote:

>
>
> Ralf: There was a recent discussion on the frustrations of lack of SciPy
> contributors- mousai is an example, along with my other projects, some more
> relevant to potential future inclusion in SciPy, some less.
>

I wouldn't call them frustrations - compared to some years ago we're in
pretty good shape. It's also not lack of contributors (200 open PRs
currently and ~100 contributors per release), it's lack of maintainers.

Mousai does look like a good start. I'm not a domain expert, but I'm well
aware of the limited options in this area. Adding Mousai to
https://scipy.org/topical-software.html may be useful to make it more
discoverable - I'd be happy to merge a PR at
https://github.com/scipy/scipy.org

Cheers,
Ralf
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scipy-dev/attachments/20180323/a39a4157/attachment.html>

From ralf.gommers at gmail.com  Sat Mar 24 02:06:27 2018
From: ralf.gommers at gmail.com (Ralf Gommers)
Date: Fri, 23 Mar 2018 23:06:27 -0700
Subject: [SciPy-Dev] 
	=?utf-8?q?Fast_Walsh=E2=80=93Hadamard_transform_for_?=
	=?utf-8?q?SciPy_=3F?=
In-Reply-To: <CAHLSkQSHCT-5_FMq4g3nMT_0yavzLh4-88nC35kHaeV53XfVGg@mail.gmail.com>
References: <CAHLSkQSHCT-5_FMq4g3nMT_0yavzLh4-88nC35kHaeV53XfVGg@mail.gmail.com>
Message-ID: <CABL7CQg8whm3jkqYze=LB1gYw6kueRgnM6d4iST_sAm9PKi6Fg@mail.gmail.com>

On Thu, Mar 22, 2018 at 11:51 AM, Chaman Agrawal <chaman.ag at gmail.com>
wrote:

> Issue #8590 https://github.com/scipy/scipy/issues/8590
>
> Currently there is no implementation of Fast Walsh?Hadamard transform in
> SciPy. Although it seems that FWHT is not as general as FFT but it is
> pretty ubiquitous ,it is there is other maths and science softwares like
> MATLAB etc. .I would like to contribute towards it. Following are the
> details about it.
>
There seems to be enough interest and wide enough applicability, I'd be in
favour of merging a good implementation in scipy.fftpack.

Ralf
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scipy-dev/attachments/20180323/a4870447/attachment.html>

From chaman.ag at gmail.com  Sat Mar 24 03:54:53 2018
From: chaman.ag at gmail.com (Chaman Agrawal)
Date: Sat, 24 Mar 2018 13:24:53 +0530
Subject: [SciPy-Dev] GSOC'18 : Enhance the Randomized Numerical Linear
 Algebra functionality
Message-ID: <CAHLSkQTcz5xFgwfYZwnkCyW4oO6RUrmyQvO4b4bg6VLpRZ5zfg@mail.gmail.com>

Hello everyone,

Goal of the mail : To understand better about the current status about the
project and suggestions/resources to speed up my research for it.

I am a second year undergraduate at Indian Institute of Technology
Kanpur,India highly interested in Linear Algebra ,Probability Theory and
Statistics and algorithms in general.

Before sending this I went through the emails archive to avoid asking any
redundant question but I was not able to get discussion about this project.
Is this project not open or there are already some strong candidates whose
discussions I was not able to find? I wanted to ask for a more detailed
description about the project compared to what is there in the Ideas page.

I had a few doubts like by both  Clarkson Woodruff transformation and
Johnson?Lindenstrauss lemma we get benefit of dimension reduction so will
this project be more focused on dimension reduction techniques or other
randomized techniques like subsampled Hadamard Transform Matrix
<https://arxiv.org/pdf/1204.0062.pdf> etc.

Also if there is a list of desired algorithms to be implemented it would be
a great speed booster.

Thank you,
Chaman Agrawal
Indian Institute of Technology Kanpur,India
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scipy-dev/attachments/20180324/4a7e8980/attachment.html>

From rlucente at pipeline.com  Sat Mar 24 07:58:22 2018
From: rlucente at pipeline.com (Robert Lucente)
Date: Sat, 24 Mar 2018 07:58:22 -0400 (GMT-04:00)
Subject: [SciPy-Dev] Thoughts on creating a toolbox for analysis of
 nonlinear dynamical system analysis
Message-ID: <212966663.611.1521892702900@wamui-moana.atl.sa.earthlink.net>

An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scipy-dev/attachments/20180324/62490b99/attachment.html>

From anubhavp28 at gmail.com  Sat Mar 24 08:30:28 2018
From: anubhavp28 at gmail.com (Anubhav Patel)
Date: Sat, 24 Mar 2018 18:00:28 +0530
Subject: [SciPy-Dev] SciPy : GSoC Proposal [Seeking Feedback] on Rotation
 Formalism in 3 Dimension
Message-ID: <CADtkFLEnuFFTsX3osQaU6cPp5wu63Q_-6pM72Q2Rk0_h+B_JqQ@mail.gmail.com>

Hi everyone,
I am want to contribute to SciPy through GSoC 2018. I am currently seeking
feedback on my proposal (
https://docs.google.com/document/d/1ylzugkvVYI7m3IXsWVLD4EkE6zQd8B9YDPKtTno9f74/edit?usp=sharing
). I would appreciate any advice regarding the proposal.

Considering Ralf advice to look for existing implementation, I am currently
looking into few implementation for conversions between rotation
representations, for reuse and inspiration. One of the issues I felt with
almost all of them is that they are designed to work on single rotation at
a time. The API they use sort of suggest towards it. Consider the example
of transforms3d ( http://matthew-brett.github.io/transforms3d/ ), due to
it's API accepting a single rotation at a time, not a array of rotations,
they tend not to utilise numpy vector operations and miss out on the speed
improvement that comes with it.

I decided to reach out to you regarding how useful would the support for
multiple conversions among rotation formalism at once be. It is even worth
considering? How frequent does such a scenario occurs where you have to
convert a sequence of rotation from one formalism to another.

I create a test (code at
https://gist.github.com/anubhavp28/e79645a544c6e7b16b408a172e522134 ) to
see possible improvement with vector operations. I implemented rotation
matrix to quaternion conversion using vector operations and timed it
against what a user would have to do to achieve the same result with
transforms3d. My code took about 0.009185953000269365 while the one with
transform3d took about 0.022295135000604205, for 100 iterations with sample
matrices provided in the code.

I would appreciate any suggestions.

Anubhav Patel
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scipy-dev/attachments/20180324/76a5c21a/attachment-0001.html>

From nikolay.mayorov at zoho.com  Sat Mar 24 09:41:30 2018
From: nikolay.mayorov at zoho.com (Nikolay Mayorov)
Date: Sat, 24 Mar 2018 18:41:30 +0500
Subject: [SciPy-Dev] SciPy : GSoC Proposal [Seeking Feedback] on
 Rotation Formalism in 3 Dimension
In-Reply-To: <CADtkFLEnuFFTsX3osQaU6cPp5wu63Q_-6pM72Q2Rk0_h+B_JqQ@mail.gmail.com>
References: <CADtkFLEnuFFTsX3osQaU6cPp5wu63Q_-6pM72Q2Rk0_h+B_JqQ@mail.gmail.com>
Message-ID: <162583e26b0.10fa46a5996033.2062527831634623735@zoho.com>

Hi, Anubhav!

I will give the feedback to your proposal today. I already looked through it and it is on a decent side.

As for your question regarding conversions: it is absolutely necessary to support bulk from/to operations, because Rotation (as perhaps agreed) can represent multiple rotations. It can be implemented as a single conversion applied to each rotation or using vectorized numpy code. In fact I believe both implementations will be necessary depending on the number of rotations contained. I hope you got my idea. And yes, bulk conversions arise frequently in my opinion.

Best,
Nikolay
Sent using Zoho Mail


---- On Sat, 24 Mar 2018 17:30:28 +0500 anubhavp28 at gmail.com wrote ----

Hi everyone,
I am want to contribute to SciPy through GSoC 2018. I am currently seeking feedback?on my proposal (?https://docs.google.com/document/d/1ylzugkvVYI7m3IXsWVLD4EkE6zQd8B9YDPKtTno9f74/edit?usp=sharing ).?I would appreciate any advice regarding the proposal.

Considering Ralf advice to look for existing implementation, I am currently looking into few implementation for conversions between rotation representations, for reuse and inspiration. One of the issues I felt with almost all of them is that they are designed to work on single rotation at a time. The API they use sort of suggest towards it. Consider the example of transforms3d (?http://matthew-brett.github.io/transforms3d/ ), due to it's API accepting a single rotation at a time, not a array of rotations, they tend not to utilise numpy vector operations and miss out on the speed improvement that comes with it. 

I decided to reach out to you regarding how useful would the support for multiple conversions among rotation formalism at once be. It is even worth considering? How frequent does such a scenario occurs where you have to convert a sequence of rotation from one formalism to another.

I create a test (code at https://gist.github.com/anubhavp28/e79645a544c6e7b16b408a172e522134 ) to see possible improvement with vector operations. I implemented rotation matrix to quaternion conversion using vector operations and timed it against what a user would have to do to achieve the same result with transforms3d.  My code took about 0.009185953000269365 while the one with transform3d took about 0.022295135000604205, for 100 iterations with sample matrices provided in the code. 

I would appreciate any suggestions.

Anubhav Patel


_______________________________________________ 
SciPy-Dev mailing list 
SciPy-Dev at python.org 
https://mail.python.org/mailman/listinfo/scipy-dev 
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scipy-dev/attachments/20180324/951884c5/attachment.html>

From ndbecker2 at gmail.com  Sat Mar 24 10:19:13 2018
From: ndbecker2 at gmail.com (Neal Becker)
Date: Sat, 24 Mar 2018 14:19:13 +0000
Subject: [SciPy-Dev] 
	=?utf-8?q?Fast_Walsh=E2=80=93Hadamard_transform_for_?=
	=?utf-8?q?SciPy_=3F?=
In-Reply-To: <CABL7CQg8whm3jkqYze=LB1gYw6kueRgnM6d4iST_sAm9PKi6Fg@mail.gmail.com>
References: <CAHLSkQSHCT-5_FMq4g3nMT_0yavzLh4-88nC35kHaeV53XfVGg@mail.gmail.com>
 <CABL7CQg8whm3jkqYze=LB1gYw6kueRgnM6d4iST_sAm9PKi6Fg@mail.gmail.com>
Message-ID: <CAG3t+pEPG4WC-N8dfGQwfwNWap5vJWMDJHsr_Xsee53EJC8qfg@mail.gmail.com>

I see I have made use of complex walsh-hadamard in the past, here's another
code snippet that shows construction of real and complex matrixes:

def walsh (n, dtype=int):
    w2 = np.array (((1, 1), (1, -1)), dtype=dtype)
    if n == 0:
        return np.array (((1,),))
    elif (n == 1):
        return w2
    else:
        m2 = walsh (n-1);
        return np.kron (w2, m2)

def cwalsh (n):
    if n == 0:
        return np.array (((1,),))
    c2 = np.array (((1, 1j), (1j, 1)))
    if (n == 1):
        return c2
    else:
        m2 = walsh (n-1);
        return np.ascontiguousarray (np.kron (c2, m2))

The code I posted earlier has implementation of fast complex transform (as
well as fast real transform)


On Sat, Mar 24, 2018 at 2:06 AM Ralf Gommers <ralf.gommers at gmail.com> wrote:

> On Thu, Mar 22, 2018 at 11:51 AM, Chaman Agrawal <chaman.ag at gmail.com>
> wrote:
>
>> Issue #8590 https://github.com/scipy/scipy/issues/8590
>>
>> Currently there is no implementation of Fast Walsh?Hadamard transform in
>> SciPy. Although it seems that FWHT is not as general as FFT but it is
>> pretty ubiquitous ,it is there is other maths and science softwares like
>> MATLAB etc. .I would like to contribute towards it. Following are the
>> details about it.
>>
> There seems to be enough interest and wide enough applicability, I'd be in
> favour of merging a good implementation in scipy.fftpack.
>
> Ralf
>
>
> _______________________________________________
> SciPy-Dev mailing list
> SciPy-Dev at python.org
> https://mail.python.org/mailman/listinfo/scipy-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scipy-dev/attachments/20180324/75add847/attachment.html>

From joseph.c.slater at gmail.com  Sat Mar 24 12:36:17 2018
From: joseph.c.slater at gmail.com (Joseph Slater)
Date: Sat, 24 Mar 2018 12:36:17 -0400
Subject: [SciPy-Dev] Thoughts on creating a toolbox for analysis of
 nonlinear dynamical system analysis
In-Reply-To: <CABL7CQgOVVa1h+8gYeQMLM8ULJ1=QwCZEDySrMZP_jSyrWT19w@mail.gmail.com>
References: <CAJvfhXBfYkQFrgsioB9A37c6pQd7GUqzzdsmFn8najQFykaYzg@mail.gmail.com>
 <CABL7CQhydSWi2rGVx3QiZEYAiLgGriFBw0zCLpAcVfmaoWH+Lw@mail.gmail.com>
 <ACA5E874-C4B0-42D7-9F20-277E115D7AE0@gmail.com>
 <CABL7CQgOVVa1h+8gYeQMLM8ULJ1=QwCZEDySrMZP_jSyrWT19w@mail.gmail.com>
Message-ID: <D3EF43C2-24C1-4ED4-A13F-830F5B225924@gmail.com>



> On Mar 24, 2018, at 2:03 AM, Ralf Gommers <ralf.gommers at gmail.com> wrote:
> 
> 
> 
> On Fri, Mar 23, 2018 at 8:39 AM, Joseph Slater <joseph.c.slater at gmail.com> wrote:
> 
> 
> Ralf: There was a recent discussion on the frustrations of lack of SciPy contributors- mousai is an example, along with my other projects, some more relevant to potential future inclusion in SciPy, some less.
> 
> I wouldn't call them frustrations - compared to some years ago we're in pretty good shape. It's also not lack of contributors (200 open PRs currently and ~100 contributors per release), it's lack of maintainers. 
> 
> Mousai does look like a good start. I'm not a domain expert, but I'm well aware of the limited options in this area. Adding Mousai to https://scipy.org/topical-software.html may be useful to make it more discoverable - I'd be happy to merge a PR at https://github.com/scipy/scipy.org

I apologize for my laziness in description. I was being overly expedient. 

I would love to have it more discoverable. I would certainly love users, contributors, etc. Do I need to do anything? 

I have to admit being a bit cautious or timid- my personal sense of awe a the quality for the code in SciPy, Numpy, etc., highlights what is a 20+ year loss in the Matlab wilderness- with commensurate brain damage. 

I'm not sure when something is worth inclusion in scipy.org, but perhaps I should have been asking earlier. 

I do have 3 other packages, one just a little, but useful, hack:

vibrationtesting : parallel to a manuscript I'm working on. The intention is to maybe roll this into scipy, at least in part. However, the signal processing needs are definitely a bit different, so perhaps it's best as a stand alone. Pieces could be brought over eventually. 
vibration_toolbox : educational demonstration code (demonstration of the physics, not of python) with modest problem-solving usefulness
array_to_latex (much faster to make a properly formatted latex array from a numpy 2D array)

The last is tiny, a bit of a pragmatic hack, but oh so useful when trying to get an array into a document. I would love to figure out an appropriate way to make it a numpy array method- alas, my skills in object oriented programming. 

If any of this is appropriate, I appreciate the attention and guidance. 

Best Regards- Joe





From joseph.c.slater at gmail.com  Sat Mar 24 13:33:40 2018
From: joseph.c.slater at gmail.com (Joseph Slater)
Date: Sat, 24 Mar 2018 13:33:40 -0400
Subject: [SciPy-Dev] Thoughts on creating a toolbox for analysis of
 nonlinear dynamical system analysis
In-Reply-To: <CABL7CQgOVVa1h+8gYeQMLM8ULJ1=QwCZEDySrMZP_jSyrWT19w@mail.gmail.com>
References: <CAJvfhXBfYkQFrgsioB9A37c6pQd7GUqzzdsmFn8najQFykaYzg@mail.gmail.com>
 <CABL7CQhydSWi2rGVx3QiZEYAiLgGriFBw0zCLpAcVfmaoWH+Lw@mail.gmail.com>
 <ACA5E874-C4B0-42D7-9F20-277E115D7AE0@gmail.com>
 <CABL7CQgOVVa1h+8gYeQMLM8ULJ1=QwCZEDySrMZP_jSyrWT19w@mail.gmail.com>
Message-ID: <609D7EEF-1659-46B3-8CA1-86760B28BFB9@gmail.com>



> On Mar 24, 2018, at 2:03 AM, Ralf Gommers <ralf.gommers at gmail.com> wrote:
> 
> 
> 
> On Fri, Mar 23, 2018 at 8:39 AM, Joseph Slater <joseph.c.slater at gmail.com> wrote:
> 
> 
> Ralf: There was a recent discussion on the frustrations of lack of SciPy contributors- mousai is an example, along with my other projects, some more relevant to potential future inclusion in SciPy, some less.
> 
> I wouldn't call them frustrations - compared to some years ago we're in pretty good shape. It's also not lack of contributors (200 open PRs currently and ~100 contributors per release), it's lack of maintainers. 
> 
> Mousai does look like a good start. I'm not a domain expert, but I'm well aware of the limited options in this area. Adding Mousai to https://scipy.org/topical-software.html may be useful to make it more discoverable - I'd be happy to merge a PR at https://github.com/scipy/scipy.org


I've submitted a pull request. 

There is no category for Mechanical/Civil, Aerospace Engineering. May not yet be time, but it might be worthwhile to have a separate category "Engineering: other", or maybe a separate page for engineering focused tools in general. 

I notices xarray and pandas in what I thought were odd locations. I would have put them in a category closer to data frames or database management or such (pandas is listed in economics, xarray in other). Categorizing is almost a futile effort, though, given my past experience maintaining the mactex wiki. 

Best Regards- Joe


From phillip.m.feldman at gmail.com  Sat Mar 24 17:02:50 2018
From: phillip.m.feldman at gmail.com (Phillip Feldman)
Date: Sat, 24 Mar 2018 14:02:50 -0700
Subject: [SciPy-Dev] how to multiply an instance of ndarray and an object?
Message-ID: <CAB2ViTaVm95O8mgGvr1ioZgyHEPS+RJ4hB=urx=m=sRX75y5uw@mail.gmail.com>

I have a class called `Signal` that handles multiplication by an array on
the right by overloading `__mul__`.  I tried to implement multiplication by
an array on the left by overloading `__rmul__`, but wasn't able to make
this work because NumPy thinks that it knows how to handle this, so my
`__rmul__` method (below) is never called.  The result is an array having
dtype `object`.  Is there a practical way to modify NumPy to solve this
problem

Phillip

    def __rmul__(self, c):
        S= self.copy()
        S.phasors*= c
        return S
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scipy-dev/attachments/20180324/87aeeda1/attachment.html>

From nikolay.mayorov at zoho.com  Sat Mar 24 18:36:13 2018
From: nikolay.mayorov at zoho.com (Nikolay Mayorov)
Date: Sun, 25 Mar 2018 03:36:13 +0500
Subject: [SciPy-Dev] Feedback to students applying to Rotation Formalism in
 3D GSoC project
Message-ID: <1625a27b273.1009a8b2c32460.2257944450853551064@zoho.com>

I decided to give feedback to all the students in one letter. My feedback is proportional to the level of thoroughness of a proposal and I focus on the weakest points. There are strong points as well and we value them, however I don't mention them here (sorry).



To Vishal Gupta https://docs.google.com/document/d/1y5OalGAvYkk8UvLtFjQ2-xhRj_NTwq0fV3GvlCBUp0I/edit



1. Quaternion section description is weird. What's this about "Method would accept a 3x1 Axis unit vector and theta which will be used to generate a 4x1 Quaternion (real, i, j ,k). Also add a method to return a 3x3 Rotation Matrix."?

2. I think it's better to dedicate a separate API section rather than putting it into milestones.

3. When you put API section, try to make it more detailed. Discuss fine points and conventions.

4. Add references.





To Aditya Bharti https://docs.google.com/document/d/1UriyLABwgjUcYfBofSr4hqAnLzblJ71pkzm7i26O8ws/edit



1. I think that returning Euler angles corresponding to the fixed sequence is not satisfactory. So researching on how to implement that is valuable. I personally don't know much about working with extrinsic rotations, from my experience they are not used that often. Implementing conversion to arbitrary intrinsic rotations (A-B-A or A-B-C) is certainly doable, but challenging. 

2. What other important operation similar to composition (in some sense) you can think of.

3. apply method is not discussed enough. Leave the details to figure out to you.

4. slerp, random_sample and Wahba are all inconsiderate. Give a real thought about these algorithms and their APIs.

5. Quaternion class is also inconsiderate. Give a real thought or explanation why you believe is necessary. For example, if you think that quaternion is a superior representation why not to use it in Rotation.

6. I think that you can at least mention other possible algorithms (like if "I have time"). I still believe that "Quaternion spline" is a great algorithm, which will be good to introduce.





To Samyak Jain https://docs.google.com/document/d/12cP0LTFa0H3aQqJfb1WAmr5A_KU6bN3qZ17LbKd5YzM/edit



1. It's good that you honestly mentioned your planned vacation, however it certainly doesn't add you any points. I don't know what to advice you here.

2. It's good that you added theoretical descriptions, but they are not quite right at all points. Probably not the most important thing at the moment.

3. API design seems inconsistent and strange. You first mention Rotation class, but then introduce standalone function (looks like) which return 4-tuple "quaternions". It doesn't look very well thought-out.

4. References to MATLAB are not evil by itself, but you should explain what you want from them. It is not an available implementation you can use as a code source.





To Anubhav Patel https://docs.google.com/document/d/1ylzugkvVYI7m3IXsWVLD4EkE6zQd8B9YDPKtTno9f74/edit



1. Fixing the sequence of Euler angles is TOO restrictive, we can't do that. It is applicable to both "from" and "to" conversions.

2. I don't necessary see the benefits of having two possible ways to initialize the class: from __init__ and from "factory methods". Can we do it better? It's not a critical point for sure.

3. In setfrommatrix --- there is a reference listed in your proposal, where a robust and insensitive to non-orthogonality algorithm is proposed, can you spot it and suggest for this?

4. See hint 2 for Aditya Bharti.

5. I feel that rotate discussion is not detailed enough. What exactly Rotation.rotate(v) does? Is there only one way to apply "rotation"?

6. APIs for slerp, qspline and davenportq are not great. Think about how to improve them.

7. In your Timeline: you goes too much in internals of implementing qspline. It is good in one sense, however you should use understandable descriptions, not "_rates", "_slew", etc.. You are not rewriting the code, you implementing the algorithm from scratch.





To Sudheer Achary https://docs.google.com/document/d/13U4feVYTJJfGFgSHA37p_JxpexBKZJ2-VSjs4z7owcQ/edit



1. Why __init__ accepts Euler angles?

2. What is the reasoning behind introducing rotate_... and other similar methods? These are some static methods. Wouldn't it be more logical that Rotation itself rotates vectors?

3. I don't see a way to compose rotations.

4. I don't how to put it politely, but overall I don't get your API design. It makes some sense, but it just seems really inconvenient and verbose and hard to grasp. What didn't you like about Rotation being some abstraction that can rotate vectors, be composed with other Rotation and converted from/to common representations?

5. Maybe it would be better if you write method signatures appropriately and not just ().



--------------------------



OK, that's it. All proposals and my comments are opened for everybody, so you can learn from them freely. Apologies if I wasn't very attentive to some details, there are many proposals.



If any of the students want to ask or say something, I suggest to use this mail thread.


Best regards,

Nikolay Mayorov




-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scipy-dev/attachments/20180325/84659d88/attachment-0001.html>

From ilhanpolat at gmail.com  Sat Mar 24 18:59:44 2018
From: ilhanpolat at gmail.com (Ilhan Polat)
Date: Sat, 24 Mar 2018 23:59:44 +0100
Subject: [SciPy-Dev] how to multiply an instance of ndarray and an
 object?
In-Reply-To: <CAB2ViTaVm95O8mgGvr1ioZgyHEPS+RJ4hB=urx=m=sRX75y5uw@mail.gmail.com>
References: <CAB2ViTaVm95O8mgGvr1ioZgyHEPS+RJ4hB=urx=m=sRX75y5uw@mail.gmail.com>
Message-ID: <CAEBuzr-Zkh7=WdKa97gR3hY720DJF=y4W9PwzCoNNneqJssCdQ@mail.gmail.com>

Yes, you can write

__array_priority__ = 1000 # some number (bigger wins. But against what you
can never be sure)

or

__array_ufunc__ = None # NumPy 1.13+ only

somewhere in your class and NumPy will leave you alone because this
overrides its eagerness to handle the vectorization and ending up on object
data types.



On Sat, Mar 24, 2018 at 10:02 PM, Phillip Feldman <
phillip.m.feldman at gmail.com> wrote:

> I have a class called `Signal` that handles multiplication by an array on
> the right by overloading `__mul__`.  I tried to implement multiplication by
> an array on the left by overloading `__rmul__`, but wasn't able to make
> this work because NumPy thinks that it knows how to handle this, so my
> `__rmul__` method (below) is never called.  The result is an array having
> dtype `object`.  Is there a practical way to modify NumPy to solve this
> problem
>
> Phillip
>
>     def __rmul__(self, c):
>         S= self.copy()
>         S.phasors*= c
>         return S
>
> _______________________________________________
> SciPy-Dev mailing list
> SciPy-Dev at python.org
> https://mail.python.org/mailman/listinfo/scipy-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scipy-dev/attachments/20180324/f8935414/attachment.html>

From ralf.gommers at gmail.com  Sat Mar 24 23:33:21 2018
From: ralf.gommers at gmail.com (Ralf Gommers)
Date: Sat, 24 Mar 2018 20:33:21 -0700
Subject: [SciPy-Dev] ANN: SciPy 1.0.1 released
Message-ID: <CABL7CQgcWQKq_jUc=mRLVp3e7spLDb-XX1o3W7CWuGcCLntTNg@mail.gmail.com>

On behalf of the SciPy development team I am pleased to announce the
availability of Scipy 1.0.1. This is a maintenance release, no new features
with respect to 1.0.0. See the release notes below for details.

Wheels and sources can be found on PyPI (https://pypi.python.org/pypi/scipy)
and on Github (https://github.com/scipy/scipy/releases/tag/v1.0.1). The
conda-forge channel will be up to date within a couple of hours.

Thanks to everyone who contributed to this release!

Cheers,
Ralf



SciPy 1.0.1 Release Notes
====================

SciPy 1.0.1 is a bug-fix release with no new features compared to 1.0.0.
Probably the most important change is a fix for an incompatibility between
SciPy 1.0.0 and ``numpy.f2py`` in the NumPy master branch.

Authors
=======

* Saurabh Agarwal +
* Alessandro Pietro Bardelli
* Philip DeBoer
* Ralf Gommers
* Matt Haberland
* Eric Larson
* Denis Laxalde
* Mihai Capot? +
* Andrew Nelson
* Oleksandr Pavlyk
* Ilhan Polat
* Anant Prakash +
* Pauli Virtanen
* Warren Weckesser
* @xoviat
* Ted Ying +

A total of 16 people contributed to this release.
People with a "+" by their names contributed a patch for the first time.
This list of names is automatically generated, and may not be fully
complete.


Issues closed for 1.0.1
-----------------------

- `#7493 <https://github.com/scipy/scipy/issues/7493>`__:
`ndimage.morphology` functions are broken with numpy 1.13.0
- `#8118 <https://github.com/scipy/scipy/issues/8118>`__: minimize_cobyla
broken if `disp=True` passed
- `#8142 <https://github.com/scipy/scipy/issues/8142>`__: scipy-v1.0.0
pdist with metric=`minkowski` raises `ValueError:...
- `#8173 <https://github.com/scipy/scipy/issues/8173>`__:
`scipy.stats.ortho_group` produces all negative determinants...
- `#8207 <https://github.com/scipy/scipy/issues/8207>`__: gaussian_filter
seg faults on float16 numpy arrays
- `#8234 <https://github.com/scipy/scipy/issues/8234>`__:
`scipy.optimize.linprog` `interior-point` presolve bug with trivial...
- `#8243 <https://github.com/scipy/scipy/issues/8243>`__: Make csgraph
importable again via `from scipy.sparse import*`
- `#8320 <https://github.com/scipy/scipy/issues/8320>`__: scipy.root
segfaults with optimizer 'lm'


Pull requests for 1.0.1
-----------------------

- `#8068 <https://github.com/scipy/scipy/pull/8068>`__: BUG: fix numpy
deprecation test failures
- `#8082 <https://github.com/scipy/scipy/pull/8082>`__: BUG: fix
solve_lyapunov import
- `#8144 <https://github.com/scipy/scipy/pull/8144>`__: MRG: Fix for cobyla
- `#8150 <https://github.com/scipy/scipy/pull/8150>`__: MAINT: resolve
UPDATEIFCOPY deprecation errors
- `#8156 <https://github.com/scipy/scipy/pull/8156>`__: BUG: missing check
on minkowski w kwarg
- `#8187 <https://github.com/scipy/scipy/pull/8187>`__: BUG: Sign of
elements in random orthogonal 2D matrices in "ortho_group_gen"...
- `#8197 <https://github.com/scipy/scipy/pull/8197>`__: CI: uninstall oclint
- `#8215 <https://github.com/scipy/scipy/pull/8215>`__: Fixes Numpy
datatype compatibility issues
- `#8237 <https://github.com/scipy/scipy/pull/8237>`__: BUG: optimize: fix
bug when variables fixed by bounds are inconsistent...
- `#8248 <https://github.com/scipy/scipy/pull/8248>`__: BUG: declare "gfk"
variable before call of terminate() in newton-cg
- `#8280 <https://github.com/scipy/scipy/pull/8280>`__: REV: reintroduce
csgraph import in scipy.sparse
- `#8322 <https://github.com/scipy/scipy/pull/8322>`__: MAINT: prevent
scipy.optimize.root segfault closes #8320
- `#8334 <https://github.com/scipy/scipy/pull/8334>`__: TST: stats: don't
use exact equality check for hdmedian test
- `#8477 <https://github.com/scipy/scipy/pull/8477>`__: BUG:
signal/signaltools: fix wrong refcounting in PyArray_OrderFilterND
- `#8530 <https://github.com/scipy/scipy/pull/8530>`__: BUG: linalg: Fixed
typo in flapack.pyf.src.
- `#8566 <https://github.com/scipy/scipy/pull/8566>`__: CI: Temporarily pin
Cython version to 0.27.3
- `#8573 <https://github.com/scipy/scipy/pull/8573>`__: Backports for 1.0.1
- `#8581 <https://github.com/scipy/scipy/pull/8581>`__: Fix Cython 0.28
build break of qhull.pyx
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scipy-dev/attachments/20180324/86064003/attachment.html>

From ralf.gommers at gmail.com  Sun Mar 25 03:11:58 2018
From: ralf.gommers at gmail.com (Ralf Gommers)
Date: Sun, 25 Mar 2018 00:11:58 -0700
Subject: [SciPy-Dev] GSOC'18 : Enhance the Randomized Numerical Linear
 Algebra functionality
In-Reply-To: <CAHLSkQTcz5xFgwfYZwnkCyW4oO6RUrmyQvO4b4bg6VLpRZ5zfg@mail.gmail.com>
References: <CAHLSkQTcz5xFgwfYZwnkCyW4oO6RUrmyQvO4b4bg6VLpRZ5zfg@mail.gmail.com>
Message-ID: <CABL7CQgRdy6m7x=6RYSz_L3CggH3okEbma1HY5JC7VBCvuiU+w@mail.gmail.com>

On Sat, Mar 24, 2018 at 12:54 AM, Chaman Agrawal <chaman.ag at gmail.com>
wrote:

> Hello everyone,
>
> Goal of the mail : To understand better about the current status about the
> project and suggestions/resources to speed up my research for it.
>
> I am a second year undergraduate at Indian Institute of Technology
> Kanpur,India highly interested in Linear Algebra ,Probability Theory and
> Statistics and algorithms in general.
>

Hi Chaman. Yes there are a couple of strong candidates already:
https://github.com/scipy/scipy/issues/8498#issuecomment-370731126. And
you're quite late to get started on this project - I would suggest your
time is better spent looking for other opportunities.

Ralf



>
> Before sending this I went through the emails archive to avoid asking any
> redundant question but I was not able to get discussion about this project.
> Is this project not open or there are already some strong candidates whose
> discussions I was not able to find? I wanted to ask for a more detailed
> description about the project compared to what is there in the Ideas page.
>
> I had a few doubts like by both  Clarkson Woodruff transformation and
> Johnson?Lindenstrauss lemma we get benefit of dimension reduction so will
> this project be more focused on dimension reduction techniques or other
> randomized techniques like subsampled Hadamard Transform Matrix
> <https://arxiv.org/pdf/1204.0062.pdf> etc.
>
> Also if there is a list of desired algorithms to be implemented it would
> be a great speed booster.
>
> Thank you,
> Chaman Agrawal
> Indian Institute of Technology Kanpur,India
>
>
> _______________________________________________
> SciPy-Dev mailing list
> SciPy-Dev at python.org
> https://mail.python.org/mailman/listinfo/scipy-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scipy-dev/attachments/20180325/33162819/attachment-0001.html>

From chaman.ag at gmail.com  Sun Mar 25 08:31:57 2018
From: chaman.ag at gmail.com (Chaman Agrawal)
Date: Sun, 25 Mar 2018 18:01:57 +0530
Subject: [SciPy-Dev] GSOC'18 : Enhance the Randomized Numerical Linear
 Algebra functionality
In-Reply-To: <CABL7CQgRdy6m7x=6RYSz_L3CggH3okEbma1HY5JC7VBCvuiU+w@mail.gmail.com>
References: <CAHLSkQTcz5xFgwfYZwnkCyW4oO6RUrmyQvO4b4bg6VLpRZ5zfg@mail.gmail.com>
 <CABL7CQgRdy6m7x=6RYSz_L3CggH3okEbma1HY5JC7VBCvuiU+w@mail.gmail.com>
Message-ID: <CAHLSkQQ1K_sCZsmiu3jPK2kCHXwJLBqHDMprjv+2C30m6TDJUA@mail.gmail.com>

Hello Ralf,

It's unfortunate to hear this but I was expecting this as I am really late
.Then will Scipy will have only two projects for GSoC this year ? I don't
think I should ask this but still, do you have any idea other than those
mentioned in the ideas page for GSoC?

Thankyou
Chaman Agrawal

On Sun, Mar 25, 2018 at 12:41 PM, Ralf Gommers <ralf.gommers at gmail.com>
wrote:

>
>
> On Sat, Mar 24, 2018 at 12:54 AM, Chaman Agrawal <chaman.ag at gmail.com>
> wrote:
>
>> Hello everyone,
>>
>> Goal of the mail : To understand better about the current status about
>> the project and suggestions/resources to speed up my research for it.
>>
>> I am a second year undergraduate at Indian Institute of Technology
>> Kanpur,India highly interested in Linear Algebra ,Probability Theory and
>> Statistics and algorithms in general.
>>
>
> Hi Chaman. Yes there are a couple of strong candidates already:
> https://github.com/scipy/scipy/issues/8498#issuecomment-370731126. And
> you're quite late to get started on this project - I would suggest your
> time is better spent looking for other opportunities.
>
> Ralf
>
>
>
>>
>> Before sending this I went through the emails archive to avoid asking any
>> redundant question but I was not able to get discussion about this project.
>> Is this project not open or there are already some strong candidates whose
>> discussions I was not able to find? I wanted to ask for a more detailed
>> description about the project compared to what is there in the Ideas page.
>>
>> I had a few doubts like by both  Clarkson Woodruff transformation and
>> Johnson?Lindenstrauss lemma we get benefit of dimension reduction so will
>> this project be more focused on dimension reduction techniques or other
>> randomized techniques like subsampled Hadamard Transform Matrix
>> <https://arxiv.org/pdf/1204.0062.pdf> etc.
>>
>> Also if there is a list of desired algorithms to be implemented it would
>> be a great speed booster.
>>
>> Thank you,
>> Chaman Agrawal
>> Indian Institute of Technology Kanpur,India
>>
>>
>> _______________________________________________
>> SciPy-Dev mailing list
>> SciPy-Dev at python.org
>> https://mail.python.org/mailman/listinfo/scipy-dev
>>
>>
>
> _______________________________________________
> SciPy-Dev mailing list
> SciPy-Dev at python.org
> https://mail.python.org/mailman/listinfo/scipy-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scipy-dev/attachments/20180325/0601ec01/attachment.html>

From ralf.gommers at gmail.com  Sun Mar 25 15:56:11 2018
From: ralf.gommers at gmail.com (Ralf Gommers)
Date: Sun, 25 Mar 2018 12:56:11 -0700
Subject: [SciPy-Dev] GSOC'18 : Enhance the Randomized Numerical Linear
 Algebra functionality
In-Reply-To: <CAHLSkQQ1K_sCZsmiu3jPK2kCHXwJLBqHDMprjv+2C30m6TDJUA@mail.gmail.com>
References: <CAHLSkQTcz5xFgwfYZwnkCyW4oO6RUrmyQvO4b4bg6VLpRZ5zfg@mail.gmail.com>
 <CABL7CQgRdy6m7x=6RYSz_L3CggH3okEbma1HY5JC7VBCvuiU+w@mail.gmail.com>
 <CAHLSkQQ1K_sCZsmiu3jPK2kCHXwJLBqHDMprjv+2C30m6TDJUA@mail.gmail.com>
Message-ID: <CABL7CQj4fOOO2K_xNjhtY6KkdSYNCCAaSrUvB0SnuQ41CtQ+Kw@mail.gmail.com>

On Sun, Mar 25, 2018 at 5:31 AM, Chaman Agrawal <chaman.ag at gmail.com> wrote:

> Hello Ralf,
>
> It's unfortunate to hear this but I was expecting this as I am really late
> .Then will Scipy will have only two projects for GSoC this year ? I don't
> think I should ask this but still, do you have any idea other than those
> mentioned in the ideas page for GSoC?
>

At the moment there are no other ideas for which we have mentors already
available, sorry.

Ralf



>
> Thankyou
> Chaman Agrawal
>
> On Sun, Mar 25, 2018 at 12:41 PM, Ralf Gommers <ralf.gommers at gmail.com>
> wrote:
>
>>
>>
>> On Sat, Mar 24, 2018 at 12:54 AM, Chaman Agrawal <chaman.ag at gmail.com>
>> wrote:
>>
>>> Hello everyone,
>>>
>>> Goal of the mail : To understand better about the current status about
>>> the project and suggestions/resources to speed up my research for it.
>>>
>>> I am a second year undergraduate at Indian Institute of Technology
>>> Kanpur,India highly interested in Linear Algebra ,Probability Theory and
>>> Statistics and algorithms in general.
>>>
>>
>> Hi Chaman. Yes there are a couple of strong candidates already:
>> https://github.com/scipy/scipy/issues/8498#issuecomment-370731126. And
>> you're quite late to get started on this project - I would suggest your
>> time is better spent looking for other opportunities.
>>
>> Ralf
>>
>>
>>
>>>
>>> Before sending this I went through the emails archive to avoid asking
>>> any redundant question but I was not able to get discussion about this
>>> project. Is this project not open or there are already some strong
>>> candidates whose discussions I was not able to find? I wanted to ask for a
>>> more detailed description about the project compared to what is there in
>>> the Ideas page.
>>>
>>> I had a few doubts like by both  Clarkson Woodruff transformation and
>>> Johnson?Lindenstrauss lemma we get benefit of dimension reduction so will
>>> this project be more focused on dimension reduction techniques or other
>>> randomized techniques like subsampled Hadamard Transform Matrix
>>> <https://arxiv.org/pdf/1204.0062.pdf> etc.
>>>
>>> Also if there is a list of desired algorithms to be implemented it would
>>> be a great speed booster.
>>>
>>> Thank you,
>>> Chaman Agrawal
>>> Indian Institute of Technology Kanpur,India
>>>
>>>
>>> _______________________________________________
>>> SciPy-Dev mailing list
>>> SciPy-Dev at python.org
>>> https://mail.python.org/mailman/listinfo/scipy-dev
>>>
>>>
>>
>> _______________________________________________
>> SciPy-Dev mailing list
>> SciPy-Dev at python.org
>> https://mail.python.org/mailman/listinfo/scipy-dev
>>
>>
>
> _______________________________________________
> SciPy-Dev mailing list
> SciPy-Dev at python.org
> https://mail.python.org/mailman/listinfo/scipy-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scipy-dev/attachments/20180325/0e1d0c15/attachment.html>

From phillip.m.feldman at gmail.com  Sun Mar 25 19:26:24 2018
From: phillip.m.feldman at gmail.com (Phillip Feldman)
Date: Sun, 25 Mar 2018 16:26:24 -0700
Subject: [SciPy-Dev] how to multiply an instance of ndarray and an
 object?
In-Reply-To: <CAEBuzr-Zkh7=WdKa97gR3hY720DJF=y4W9PwzCoNNneqJssCdQ@mail.gmail.com>
References: <CAB2ViTaVm95O8mgGvr1ioZgyHEPS+RJ4hB=urx=m=sRX75y5uw@mail.gmail.com>
 <CAEBuzr-Zkh7=WdKa97gR3hY720DJF=y4W9PwzCoNNneqJssCdQ@mail.gmail.com>
Message-ID: <CAB2ViTY3=CnfZYN6o-MLXkUUNv=ZmVRZTmNPKeH75uQFmNhnaQ@mail.gmail.com>

Yes!  It works.  Thanks so much!

On Sat, Mar 24, 2018 at 3:59 PM, Ilhan Polat <ilhanpolat at gmail.com> wrote:

> Yes, you can write
>
> __array_priority__ = 1000 # some number (bigger wins. But against what
> you can never be sure)
>
> or
>
> __array_ufunc__ = None # NumPy 1.13+ only
>
> somewhere in your class and NumPy will leave you alone because this
> overrides its eagerness to handle the vectorization and ending up on object
> data types.
>
>
>
> On Sat, Mar 24, 2018 at 10:02 PM, Phillip Feldman <
> phillip.m.feldman at gmail.com> wrote:
>
>> I have a class called `Signal` that handles multiplication by an array on
>> the right by overloading `__mul__`.  I tried to implement multiplication by
>> an array on the left by overloading `__rmul__`, but wasn't able to make
>> this work because NumPy thinks that it knows how to handle this, so my
>> `__rmul__` method (below) is never called.  The result is an array having
>> dtype `object`.  Is there a practical way to modify NumPy to solve this
>> problem
>>
>> Phillip
>>
>>     def __rmul__(self, c):
>>         S= self.copy()
>>         S.phasors*= c
>>         return S
>>
>> _______________________________________________
>> SciPy-Dev mailing list
>> SciPy-Dev at python.org
>> https://mail.python.org/mailman/listinfo/scipy-dev
>>
>>
>
> _______________________________________________
> SciPy-Dev mailing list
> SciPy-Dev at python.org
> https://mail.python.org/mailman/listinfo/scipy-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scipy-dev/attachments/20180325/6f27385c/attachment.html>

From ralf.gommers at gmail.com  Sun Mar 25 20:45:40 2018
From: ralf.gommers at gmail.com (Ralf Gommers)
Date: Sun, 25 Mar 2018 17:45:40 -0700
Subject: [SciPy-Dev] SciPy 1.1.0 release schedule
Message-ID: <CABL7CQgKXdHgnugVxTcZj0SuYTbjfrtKZ97qFKx3R1i0_2Af+w@mail.gmail.com>

Hi all,

It's 5 months after the 1.0 release, so it's time to start planning for
1.1. There's still quite a few open issues and PRs marked for 1.1.0 (see
https://github.com/scipy/scipy/milestone/34), but not many that seem
blocking or really difficult to resolve. So I'd like to propose the
following schedule:

April 11: branch 1.0.x
April 13: rc1
April 27: rc2 (if needed)
May 4: final release

It would be useful if everyone could add PRs/issues that they think are
critical to the 1.1 milestone. Adding yourself as an "assignee" on
PRs/issues you plan to tackle would also be helpful.

Thoughts?

Cheers,
Ralf
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scipy-dev/attachments/20180325/ce5c8f9e/attachment.html>

From andyfaff at gmail.com  Sun Mar 25 20:49:43 2018
From: andyfaff at gmail.com (Andrew Nelson)
Date: Mon, 26 Mar 2018 00:49:43 +0000
Subject: [SciPy-Dev] SciPy 1.1.0 release schedule
In-Reply-To: <CABL7CQgKXdHgnugVxTcZj0SuYTbjfrtKZ97qFKx3R1i0_2Af+w@mail.gmail.com>
References: <CABL7CQgKXdHgnugVxTcZj0SuYTbjfrtKZ97qFKx3R1i0_2Af+w@mail.gmail.com>
Message-ID: <CAAbtOZd8DToJ18mNKQR40p17hbDzE85rWvbdWrcX13YhDrOwUw@mail.gmail.com>

There are several PRs that I started reviewing, but just don't have time at
the moment to finish the process. Such as the simulated dual annealing,
ratio of uniforms...
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scipy-dev/attachments/20180326/6021cca1/attachment-0001.html>

From ralf.gommers at gmail.com  Sun Mar 25 20:59:26 2018
From: ralf.gommers at gmail.com (Ralf Gommers)
Date: Sun, 25 Mar 2018 17:59:26 -0700
Subject: [SciPy-Dev] Thoughts on creating a toolbox for analysis of
 nonlinear dynamical system analysis
In-Reply-To: <212966663.611.1521892702900@wamui-moana.atl.sa.earthlink.net>
References: <212966663.611.1521892702900@wamui-moana.atl.sa.earthlink.net>
Message-ID: <CABL7CQhMtWhf9M-FwsqyS324O=rADcv_oS1hSf8Gw8pfVEF79A@mail.gmail.com>

On Sat, Mar 24, 2018 at 4:58 AM, Robert Lucente <rlucente at pipeline.com>
wrote:

> >lack of maintainers
> What are the pre-reqs for someone to become a maintainer?
>

We don't have a fixed checklist or anything, but I'd say:
- interest in becoming one
- sustained contributions over some period (of good quality). no fixed
period or number of PRs, but typical is on the order of 6 months or 10 PRs
- communication skills, on Github and/or email (code review, design
decisions, etc.)
- already doing or interest in doing PR reviews

It could well be that you read this and think "I'm ticking all of those
boxes". Or conversely thinking "sounds cool, but am I good enough?". In
either case, I'm happy to chat about it.

Cheers,
Ralf



>
>
> -----Original Message-----
> From: Ralf Gommers
> Sent: Mar 24, 2018 2:03 AM
> To: SciPy Developers List
> Subject: Re: [SciPy-Dev] Thoughts on creating a toolbox for analysis of
> nonlinear dynamical system analysis
>
>
>
> On Fri, Mar 23, 2018 at 8:39 AM, Joseph Slater <joseph.c.slater at gmail.com>
> wrote:
>
>>
>>
>> Ralf: There was a recent discussion on the frustrations of lack of SciPy
>> contributors- mousai is an example, along with my other projects, some more
>> relevant to potential future inclusion in SciPy, some less.
>>
>
> I wouldn't call them frustrations - compared to some years ago we're in
> pretty good shape. It's also not lack of contributors (200 open PRs
> currently and ~100 contributors per release), it's lack of maintainers.
>
> Mousai does look like a good start. I'm not a domain expert, but I'm well
> aware of the limited options in this area. Adding Mousai to
> https://scipy.org/topical-software.html may be useful to make it more
> discoverable - I'd be happy to merge a PR at https://github.com/scipy/
> scipy.org
>
> Cheers,
> Ralf
>
>
> _______________________________________________
> SciPy-Dev mailing list
> SciPy-Dev at python.org
> https://mail.python.org/mailman/listinfo/scipy-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scipy-dev/attachments/20180325/b316466e/attachment.html>

From ralf.gommers at gmail.com  Sun Mar 25 21:02:12 2018
From: ralf.gommers at gmail.com (Ralf Gommers)
Date: Sun, 25 Mar 2018 18:02:12 -0700
Subject: [SciPy-Dev] Thoughts on creating a toolbox for analysis of
 nonlinear dynamical system analysis
In-Reply-To: <609D7EEF-1659-46B3-8CA1-86760B28BFB9@gmail.com>
References: <CAJvfhXBfYkQFrgsioB9A37c6pQd7GUqzzdsmFn8najQFykaYzg@mail.gmail.com>
 <CABL7CQhydSWi2rGVx3QiZEYAiLgGriFBw0zCLpAcVfmaoWH+Lw@mail.gmail.com>
 <ACA5E874-C4B0-42D7-9F20-277E115D7AE0@gmail.com>
 <CABL7CQgOVVa1h+8gYeQMLM8ULJ1=QwCZEDySrMZP_jSyrWT19w@mail.gmail.com>
 <609D7EEF-1659-46B3-8CA1-86760B28BFB9@gmail.com>
Message-ID: <CABL7CQhcYhiVSJKezG__kD_gsyk998+MWRAg1kK7cmhP86tYxg@mail.gmail.com>

On Sat, Mar 24, 2018 at 10:33 AM, Joseph Slater <joseph.c.slater at gmail.com>
wrote:

>
>
> > On Mar 24, 2018, at 2:03 AM, Ralf Gommers <ralf.gommers at gmail.com>
> wrote:
> >
> >
> >
> > On Fri, Mar 23, 2018 at 8:39 AM, Joseph Slater <
> joseph.c.slater at gmail.com> wrote:
> >
> >
> > Ralf: There was a recent discussion on the frustrations of lack of SciPy
> contributors- mousai is an example, along with my other projects, some more
> relevant to potential future inclusion in SciPy, some less.
> >
> > I wouldn't call them frustrations - compared to some years ago we're in
> pretty good shape. It's also not lack of contributors (200 open PRs
> currently and ~100 contributors per release), it's lack of maintainers.
> >
> > Mousai does look like a good start. I'm not a domain expert, but I'm
> well aware of the limited options in this area. Adding Mousai to
> https://scipy.org/topical-software.html may be useful to make it more
> discoverable - I'd be happy to merge a PR at https://github.com/scipy/
> scipy.org
>
>
> I've submitted a pull request.
>

Thanks!


>
> There is no category for Mechanical/Civil, Aerospace Engineering. May not
> yet be time, but it might be worthwhile to have a separate category
> "Engineering: other", or maybe a separate page for engineering focused
> tools in general.
>
> I notices xarray and pandas in what I thought were odd locations. I would
> have put them in a category closer to data frames or database management or
> such (pandas is listed in economics, xarray in other). Categorizing is
> almost a futile effort, though, given my past experience maintaining the
> mactex wiki.
>

Yeah, it's pretty hard. If something is really weird we can move it, but
indeed no point in spending lots of effort on it.

Cheers,
Ralf
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scipy-dev/attachments/20180325/f0ddc13d/attachment.html>

From ralf.gommers at gmail.com  Sun Mar 25 21:31:01 2018
From: ralf.gommers at gmail.com (Ralf Gommers)
Date: Sun, 25 Mar 2018 18:31:01 -0700
Subject: [SciPy-Dev] SciPy 1.1.0 release schedule
In-Reply-To: <CAAbtOZd8DToJ18mNKQR40p17hbDzE85rWvbdWrcX13YhDrOwUw@mail.gmail.com>
References: <CABL7CQgKXdHgnugVxTcZj0SuYTbjfrtKZ97qFKx3R1i0_2Af+w@mail.gmail.com>
 <CAAbtOZd8DToJ18mNKQR40p17hbDzE85rWvbdWrcX13YhDrOwUw@mail.gmail.com>
Message-ID: <CABL7CQjTH3er0Ce6aOridQk5M7a2jreKq+ek5nd9Kj18Kr_Cig@mail.gmail.com>

On Sun, Mar 25, 2018 at 5:49 PM, Andrew Nelson <andyfaff at gmail.com> wrote:

> There are several PRs that I started reviewing, but just don't have time
> at the moment to finish the process. Such as the simulated dual annealing
>

https://github.com/scipy/scipy/pull/8203 is close to ready I'd say, Antonio
already finished reviewing and Jacob did a quick review as well. Would be
good to get that PR in. Maybe submit any comments you already had?



> , ratio of uniforms...
>

https://github.com/scipy/scipy/pull/8293 seems to have a bit more review
work left, but code looks close to finished. Would be nice to get it in,
but less critical I'd say - let's just see how far we get on that one.

Ralf
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scipy-dev/attachments/20180325/ff0ff6c8/attachment.html>

From rlucente at pipeline.com  Mon Mar 26 11:41:34 2018
From: rlucente at pipeline.com (Robert Lucente)
Date: Mon, 26 Mar 2018 11:41:34 -0400 (GMT-04:00)
Subject: [SciPy-Dev] Thoughts on creating a toolbox for analysis of
 nonlinear dynamical system analysis
Message-ID: <237320890.4090.1522078894360@wamui-kristoff.atl.sa.earthlink.net>

An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scipy-dev/attachments/20180326/7c55dce4/attachment-0001.html>

From lagru at mailbox.org  Mon Mar 26 14:11:08 2018
From: lagru at mailbox.org (Lars G.)
Date: Mon, 26 Mar 2018 20:11:08 +0200
Subject: [SciPy-Dev] Measuring test coverage of a Cython module
Message-ID: <28dc611e-7598-e88b-2534-e2a5252b9ca0@mailbox.org>

Hi,
I'm trying to measure the test coverage of a Cython module. However the
module in question (`scipy/signal/peak_finding_utils.pyx`) doesn't show
up in the final report.

What I have tried so far:

1. Added `plugins = Cython.coverage` to the `.coveragerc` file.

2. Added
# cython: linetrace=True
# distutils: define_macros=CYTHON_TRACE_NOGIL=1
to the header of `peak_finding_utils.pyx`

3. Build binaries inplace with
$ python setup.py build_ext --inplace

4. Measured test coverage
a) with `runtests.py`
$ python runtests.py -t scipy/signal/tests/test_peak_finding --no-build
--coverage

b) and using the `coverage` package
$ coverage run -m pytest scipy/signal/tests/test_peak_finding.py
$ coverage html

This approach is based on this blog post:
http://blog.behnel.de/posts/coverage-analysis-for-cython-modules.html

In both cases a) and b) the PYX file sadly isn't included in the report
and I don't know what to try next. Has anyone figured out a way to do
this? If I've overlooked a related issue or conversation on the mailing
list I apologize and would be grateful if you could point me to it.

Best regards,
Lars

From ralf.gommers at gmail.com  Tue Mar 27 23:00:10 2018
From: ralf.gommers at gmail.com (Ralf Gommers)
Date: Tue, 27 Mar 2018 20:00:10 -0700
Subject: [SciPy-Dev] welcome Matt and Paul to the core team
Message-ID: <CABL7CQgm0LtkTyYx-xKV-r8jYBkV+zkdO5BrS23aCZKd800W7Q@mail.gmail.com>

Hi all,

On behalf of the SciPy developers I'd like to welcome Matt Haberland and
Paul van Mulbregt as members of the core dev team.

Matt co-mentored during last years' GSoC, and is the author of the
interior-point method that improved optimize.linprog -
https://github.com/scipy/scipy/pulls/mdhaber.

Paul has been contributing to (mainly) scipy.special for over 6 months -
https://github.com/scipy/scipy/pulls/pvanmulbregt.

I'm looking forward to their continued contributions!

Cheers,
Ralf
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scipy-dev/attachments/20180327/4a96ee26/attachment.html>

From andyfaff at gmail.com  Tue Mar 27 23:03:48 2018
From: andyfaff at gmail.com (Andrew Nelson)
Date: Wed, 28 Mar 2018 03:03:48 +0000
Subject: [SciPy-Dev] welcome Matt and Paul to the core team
In-Reply-To: <CABL7CQgm0LtkTyYx-xKV-r8jYBkV+zkdO5BrS23aCZKd800W7Q@mail.gmail.com>
References: <CABL7CQgm0LtkTyYx-xKV-r8jYBkV+zkdO5BrS23aCZKd800W7Q@mail.gmail.com>
Message-ID: <CAAbtOZfuuK3b5CgrjQ0EoeCgBmvwBq=8C3w6sfHDD4EEmEFo2w@mail.gmail.com>

Welcome Matt and Paul.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scipy-dev/attachments/20180328/305e5980/attachment.html>

From warren.weckesser at gmail.com  Tue Mar 27 23:21:12 2018
From: warren.weckesser at gmail.com (Warren Weckesser)
Date: Tue, 27 Mar 2018 23:21:12 -0400
Subject: [SciPy-Dev] welcome Matt and Paul to the core team
In-Reply-To: <CABL7CQgm0LtkTyYx-xKV-r8jYBkV+zkdO5BrS23aCZKd800W7Q@mail.gmail.com>
References: <CABL7CQgm0LtkTyYx-xKV-r8jYBkV+zkdO5BrS23aCZKd800W7Q@mail.gmail.com>
Message-ID: <CAGzF1ud67KkO4xMDgYbnkTo=SqzjPrdJ6e=W=x4tmoukivRb=A@mail.gmail.com>

On Tue, Mar 27, 2018 at 11:00 PM, Ralf Gommers <ralf.gommers at gmail.com>
wrote:

> Hi all,
>
> On behalf of the SciPy developers I'd like to welcome Matt Haberland and
> Paul van Mulbregt as members of the core dev team.
>
> Matt co-mentored during last years' GSoC, and is the author of the
> interior-point method that improved optimize.linprog -
> https://github.com/scipy/scipy/pulls/mdhaber.
>
> Paul has been contributing to (mainly) scipy.special for over 6 months -
> https://github.com/scipy/scipy/pulls/pvanmulbregt.
>
>

Welcome Matt and Paul!  Keep up the great work.

Warren



> I'm looking forward to their continued contributions!
>
> Cheers,
> Ralf
>
> _______________________________________________
> SciPy-Dev mailing list
> SciPy-Dev at python.org
> https://mail.python.org/mailman/listinfo/scipy-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scipy-dev/attachments/20180327/eb4b1aff/attachment.html>

From ralf.gommers at gmail.com  Wed Mar 28 00:06:15 2018
From: ralf.gommers at gmail.com (Ralf Gommers)
Date: Tue, 27 Mar 2018 21:06:15 -0700
Subject: [SciPy-Dev] Measuring test coverage of a Cython module
In-Reply-To: <28dc611e-7598-e88b-2534-e2a5252b9ca0@mailbox.org>
References: <28dc611e-7598-e88b-2534-e2a5252b9ca0@mailbox.org>
Message-ID: <CABL7CQhv=eu=rA9WM0CFa5L760qMkHUmuJvPziT+FkhCmUZe_w@mail.gmail.com>

On Mon, Mar 26, 2018 at 11:11 AM, Lars G. <lagru at mailbox.org> wrote:

> Hi,
> I'm trying to measure the test coverage of a Cython module. However the
> module in question (`scipy/signal/peak_finding_utils.pyx`) doesn't show
> up in the final report.
>
> What I have tried so far:
>
> 1. Added `plugins = Cython.coverage` to the `.coveragerc` file.
>
> 2. Added
> # cython: linetrace=True
> # distutils: define_macros=CYTHON_TRACE_NOGIL=1
> to the header of `peak_finding_utils.pyx`
>
> 3. Build binaries inplace with
> $ python setup.py build_ext --inplace
>
> 4. Measured test coverage
> a) with `runtests.py`
> $ python runtests.py -t scipy/signal/tests/test_peak_finding --no-build
> --coverage
>
> b) and using the `coverage` package
> $ coverage run -m pytest scipy/signal/tests/test_peak_finding.py
> $ coverage html
>

The one step missing here seems to be editing .coveragerc:
http://cython.readthedocs.io/en/latest/src/tutorial/profiling_tutorial.html?highlight=coverage#enabling-coverage-analysis


>
> This approach is based on this blog post:
> http://blog.behnel.de/posts/coverage-analysis-for-cython-modules.html
>
> In both cases a) and b) the PYX file sadly isn't included in the report
> and I don't know what to try next. Has anyone figured out a way to do
> this? If I've overlooked a related issue or conversation on the mailing
> list I apologize and would be grateful if you could point me to it.
>

The only discussion I remember is one on the scikit-image list started by
Matthew Brett to get this working; not sure if that came to anything. In
general it looks like it should work, but it isn't always reliable (e.g.
https://github.com/cython/cython/issues/1985)

Ralf
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scipy-dev/attachments/20180327/951130ee/attachment.html>

From stefanv at berkeley.edu  Wed Mar 28 01:53:35 2018
From: stefanv at berkeley.edu (Stefan van der Walt)
Date: Tue, 27 Mar 2018 22:53:35 -0700
Subject: [SciPy-Dev] welcome Matt and Paul to the core team
In-Reply-To: <CABL7CQgm0LtkTyYx-xKV-r8jYBkV+zkdO5BrS23aCZKd800W7Q@mail.gmail.com>
References: <CABL7CQgm0LtkTyYx-xKV-r8jYBkV+zkdO5BrS23aCZKd800W7Q@mail.gmail.com>
Message-ID: <1626b2b2f18.27ae.acf34a9c767d7bb498a799333be0433e@fastmail.com>

Congratulations Matt & Paul, and a great big  welcome to the team!
On March 27, 2018 20:00:47 Ralf Gommers <ralf.gommers at gmail.com> wrote:
Hi all,

On behalf of the SciPy developers I'd like to welcome Matt Haberland and 
Paul van Mulbregtas members of the core dev team.

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scipy-dev/attachments/20180327/ff4c33a1/attachment-0001.html>

From charlesr.harris at gmail.com  Wed Mar 28 08:57:44 2018
From: charlesr.harris at gmail.com (Charles R Harris)
Date: Wed, 28 Mar 2018 06:57:44 -0600
Subject: [SciPy-Dev] welcome Matt and Paul to the core team
In-Reply-To: <CABL7CQgm0LtkTyYx-xKV-r8jYBkV+zkdO5BrS23aCZKd800W7Q@mail.gmail.com>
References: <CABL7CQgm0LtkTyYx-xKV-r8jYBkV+zkdO5BrS23aCZKd800W7Q@mail.gmail.com>
Message-ID: <CAB6mnx+S+08ESTF8Sb4hAB8CuyxNvQPgMjMMC6d3bu8N9JY2Sw@mail.gmail.com>

On Tue, Mar 27, 2018 at 9:00 PM, Ralf Gommers <ralf.gommers at gmail.com>
wrote:

> Hi all,
>
> On behalf of the SciPy developers I'd like to welcome Matt Haberland and
> Paul van Mulbregt as members of the core dev team.
>
> Matt co-mentored during last years' GSoC, and is the author of the
> interior-point method that improved optimize.linprog -
> https://github.com/scipy/scipy/pulls/mdhaber.
>
> Paul has been contributing to (mainly) scipy.special for over 6 months -
> https://github.com/scipy/scipy/pulls/pvanmulbregt.
>
> I'm looking forward to their continued contributions!
>
> Cheers,
> Ralf
>
>
Welcome to both.

Chuck
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scipy-dev/attachments/20180328/b0ac0698/attachment.html>

From mikofski at berkeley.edu  Wed Mar 28 10:37:54 2018
From: mikofski at berkeley.edu (Mark Alexander Mikofski)
Date: Wed, 28 Mar 2018 14:37:54 +0000
Subject: [SciPy-Dev] welcome Matt and Paul to the core team
In-Reply-To: <CAB6mnx+S+08ESTF8Sb4hAB8CuyxNvQPgMjMMC6d3bu8N9JY2Sw@mail.gmail.com>
References: <CABL7CQgm0LtkTyYx-xKV-r8jYBkV+zkdO5BrS23aCZKd800W7Q@mail.gmail.com>
 <CAB6mnx+S+08ESTF8Sb4hAB8CuyxNvQPgMjMMC6d3bu8N9JY2Sw@mail.gmail.com>
Message-ID: <CAEqRcW0kEmRs6dHbEXmiXoPUFfJ6Ke-gpauuryOwXRdwSntusQ@mail.gmail.com>

Welcome Matt and Paul. Thanks for your service!

On Wed, Mar 28, 2018, 5:58 AM Charles R Harris <charlesr.harris at gmail.com>
wrote:

>
>
> On Tue, Mar 27, 2018 at 9:00 PM, Ralf Gommers <ralf.gommers at gmail.com>
> wrote:
>
>> Hi all,
>>
>> On behalf of the SciPy developers I'd like to welcome Matt Haberland and
>> Paul van Mulbregt as members of the core dev team.
>>
>> Matt co-mentored during last years' GSoC, and is the author of the
>> interior-point method that improved optimize.linprog -
>> https://github.com/scipy/scipy/pulls/mdhaber.
>>
>> Paul has been contributing to (mainly) scipy.special for over 6 months -
>> https://github.com/scipy/scipy/pulls/pvanmulbregt.
>>
>> I'm looking forward to their continued contributions!
>>
>> Cheers,
>> Ralf
>>
>>
> Welcome to both.
>
> Chuck
> _______________________________________________
> SciPy-Dev mailing list
> SciPy-Dev at python.org
> https://mail.python.org/mailman/listinfo/scipy-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scipy-dev/attachments/20180328/ed49f730/attachment.html>

From tyler.je.reddy at gmail.com  Wed Mar 28 15:31:58 2018
From: tyler.je.reddy at gmail.com (Tyler Reddy)
Date: Wed, 28 Mar 2018 13:31:58 -0600
Subject: [SciPy-Dev] welcome Matt and Paul to the core team
In-Reply-To: <CAEqRcW0kEmRs6dHbEXmiXoPUFfJ6Ke-gpauuryOwXRdwSntusQ@mail.gmail.com>
References: <CABL7CQgm0LtkTyYx-xKV-r8jYBkV+zkdO5BrS23aCZKd800W7Q@mail.gmail.com>
 <CAB6mnx+S+08ESTF8Sb4hAB8CuyxNvQPgMjMMC6d3bu8N9JY2Sw@mail.gmail.com>
 <CAEqRcW0kEmRs6dHbEXmiXoPUFfJ6Ke-gpauuryOwXRdwSntusQ@mail.gmail.com>
Message-ID: <CAHPuU_YY-CgRLrWiGOLEjNwU1pKawWHdgjBFmNsRQ_+qJNTEcg@mail.gmail.com>

Congrats Matt & Paul

On 28 March 2018 at 08:37, Mark Alexander Mikofski <mikofski at berkeley.edu>
wrote:

> Welcome Matt and Paul. Thanks for your service!
>
> On Wed, Mar 28, 2018, 5:58 AM Charles R Harris <charlesr.harris at gmail.com>
> wrote:
>
>>
>>
>> On Tue, Mar 27, 2018 at 9:00 PM, Ralf Gommers <ralf.gommers at gmail.com>
>> wrote:
>>
>>> Hi all,
>>>
>>> On behalf of the SciPy developers I'd like to welcome Matt Haberland
>>> and Paul van Mulbregt as members of the core dev team.
>>>
>>> Matt co-mentored during last years' GSoC, and is the author of the
>>> interior-point method that improved optimize.linprog -
>>> https://github.com/scipy/scipy/pulls/mdhaber.
>>>
>>> Paul has been contributing to (mainly) scipy.special for over 6 months -
>>> https://github.com/scipy/scipy/pulls/pvanmulbregt.
>>>
>>> I'm looking forward to their continued contributions!
>>>
>>> Cheers,
>>> Ralf
>>>
>>>
>> Welcome to both.
>>
>> Chuck
>> _______________________________________________
>> SciPy-Dev mailing list
>> SciPy-Dev at python.org
>> https://mail.python.org/mailman/listinfo/scipy-dev
>>
>
> _______________________________________________
> SciPy-Dev mailing list
> SciPy-Dev at python.org
> https://mail.python.org/mailman/listinfo/scipy-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scipy-dev/attachments/20180328/2844df72/attachment.html>

From lagru at mailbox.org  Thu Mar 29 06:02:08 2018
From: lagru at mailbox.org (Lars G.)
Date: Thu, 29 Mar 2018 12:02:08 +0200
Subject: [SciPy-Dev] Documentation / tutorial for peak finding in
 scipy.signal
In-Reply-To: <14d2db01-5bf7-88f7-ab2c-4bcc29072c41@mailbox.org>
References: <48693513-3b3f-d6b7-9116-85b0eaf834f2@mailbox.org>
 <CAGu2niWWSKqqfKEADetwB2OObWLtyYh_F3+-nxjnkauFHfxHmw@mail.gmail.com>
 <13093fec-766d-ba01-26c5-cf57fd5aa2ce@mailbox.org>
 <CAGu2niWJZ=wHZsWSPB-vNOpqbVwwXTOT=U+4i9YHfyvK-+vJwQ@mail.gmail.com>
 <CAB2ovou2nUbUtV58KXGWk8E5uBv2-3upBJo8qviXtPvybNmPCQ@mail.gmail.com>
 <CABL7CQgM0Taey5p9+H2Rx4epgsUBLBBPHi7zDkTtZq-2ECXT1w@mail.gmail.com>
 <14d2db01-5bf7-88f7-ab2c-4bcc29072c41@mailbox.org>
Message-ID: <9766d1fb-8992-548b-ebb7-120667bad3b6@mailbox.org>

On 18.03.2018 12:23, Lars G. wrote:
> On 17.03.2018 20:59, Ralf Gommers wrote:
>> If you can get away with a smaller size that would be useful, but I'm
>> okay with up to 500 kb.
> 
> That should be possible. I just played around with ECG signals from the
> MIT Arrhythmia Database
> https://www.physionet.org/physiobank/database/mitdb/
> 
> The signals are sampled with 360 Hz and and an ADC resolution of 11-bit.
> If `numpy.savez_compressed` is used to store the array we can get away
> with `np.uint16` which translates to ~200 KiB for a 10 min window.
> 
> If we select the right window we could cover areas with different signal
> properties (noise level, artifacts, baseline, amplitude, spectrum) which
> would make it useful for more varied examples.
> 
> Lars

For anyone interested, I proposed an example signal here
https://github.com/scipy/scipy/pull/8627

Best regards,
Lars

From ilhanpolat at gmail.com  Thu Mar 29 06:03:09 2018
From: ilhanpolat at gmail.com (Ilhan Polat)
Date: Thu, 29 Mar 2018 12:03:09 +0200
Subject: [SciPy-Dev] welcome Matt and Paul to the core team
In-Reply-To: <CAB6mnx+S+08ESTF8Sb4hAB8CuyxNvQPgMjMMC6d3bu8N9JY2Sw@mail.gmail.com>
References: <CABL7CQgm0LtkTyYx-xKV-r8jYBkV+zkdO5BrS23aCZKd800W7Q@mail.gmail.com>
 <CAB6mnx+S+08ESTF8Sb4hAB8CuyxNvQPgMjMMC6d3bu8N9JY2Sw@mail.gmail.com>
Message-ID: <CAEBuzr_C-hJ5V-mYFFUmiDp28=+GkX_AYYV=iJ2SvtZEiHVYrQ@mail.gmail.com>

 Welcome to the team and thank you both for your contributions so far!

On Wed, Mar 28, 2018 at 2:57 PM, Charles R Harris <charlesr.harris at gmail.com
> wrote:

>
>
> On Tue, Mar 27, 2018 at 9:00 PM, Ralf Gommers <ralf.gommers at gmail.com>
> wrote:
>
>> Hi all,
>>
>> On behalf of the SciPy developers I'd like to welcome Matt Haberland and
>> Paul van Mulbregt as members of the core dev team.
>>
>> Matt co-mentored during last years' GSoC, and is the author of the
>> interior-point method that improved optimize.linprog -
>> https://github.com/scipy/scipy/pulls/mdhaber.
>>
>> Paul has been contributing to (mainly) scipy.special for over 6 months -
>> https://github.com/scipy/scipy/pulls/pvanmulbregt.
>>
>> I'm looking forward to their continued contributions!
>>
>> Cheers,
>> Ralf
>>
>>
> Welcome to both.
>
> Chuck
>
> _______________________________________________
> SciPy-Dev mailing list
> SciPy-Dev at python.org
> https://mail.python.org/mailman/listinfo/scipy-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scipy-dev/attachments/20180329/3a6a6bf3/attachment.html>

From joseph.c.slater at gmail.com  Thu Mar 29 14:20:33 2018
From: joseph.c.slater at gmail.com (Joseph Slater)
Date: Thu, 29 Mar 2018 14:20:33 -0400
Subject: [SciPy-Dev] Mousai 0.3- general purpose Harmonic Balance solver.
Message-ID: <2FEBA1D2-7390-490E-88C8-A3451B1DCE8D@gmail.com>

Dear colleagues,
I've just pushed 0.3 of Mousai, a general purpose numerical Harmonic Balance solver to pypi. 

There are now two different solution methodologies, one minimizing frequency domain errors the other time domain. Test problems and contributors welcome!

Best Regards- Joe

https://josephcslater.github.io/mousai/



From warren.weckesser at gmail.com  Thu Mar 29 15:43:50 2018
From: warren.weckesser at gmail.com (Warren Weckesser)
Date: Thu, 29 Mar 2018 15:43:50 -0400
Subject: [SciPy-Dev] New subpackage: scipy.data
Message-ID: <CAGzF1udgrXmWyXh93fpN9TOVvbmvSEt1UHXpdw6QtNJYBrSi0A@mail.gmail.com>

According to the SciPy roadmap (
https://github.com/scipy/scipy/blob/master/doc/ROADMAP.rst.txt#misc),
`scipy.misc` will eventually be removed. Currently, the combinatorial
functions and the image-related operations are all deprecated.  The only
non-deprecated functions in `misc` are `central_diff_weights()`,
`derivative()` and the two functions that return image data: `ascent()` and
`face()`.

As a steps towards the deprecation of `misc`, I propose that we create a
new package, `scipy.data`, for holding data sets.  `ascent()` and `face()`
would move there, and the new ECG data set proposed in a current pull
request (https://github.com/scipy/scipy/pull/8627) would be put there.

An early version of the roadmap suggested moving the images to
`scipy.ndimage`, but that is no longer in the text.  I think a separate
subpackage for data sets makes sense.

What do you think?

P.S. If there is already a similar proposal in the mailing list or on
github, or any other older mailing list discussions related to this, let me
know.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scipy-dev/attachments/20180329/3ee9606f/attachment.html>

From larson.eric.d at gmail.com  Thu Mar 29 15:48:01 2018
From: larson.eric.d at gmail.com (Eric Larson)
Date: Thu, 29 Mar 2018 15:48:01 -0400
Subject: [SciPy-Dev] New subpackage: scipy.data
In-Reply-To: <CAGzF1udgrXmWyXh93fpN9TOVvbmvSEt1UHXpdw6QtNJYBrSi0A@mail.gmail.com>
References: <CAGzF1udgrXmWyXh93fpN9TOVvbmvSEt1UHXpdw6QtNJYBrSi0A@mail.gmail.com>
Message-ID: <CAGu2niVd2RrSt1sqr8KjgiWfBr7iDBU-r0OuwNGK_6DDyXYNWQ@mail.gmail.com>

Sounds like a good plan to me.

If others agree, I propose that we make existing (and new) data available
in scipy.data for 1.1.0. Maybe a 2-release deprecation warning cycle before
removing them from `scipy.misc` (possibly for removing everything if we can
move the other remaining things, too)?

Eric


On Thu, Mar 29, 2018 at 3:43 PM, Warren Weckesser <
warren.weckesser at gmail.com> wrote:

> According to the SciPy roadmap (https://github.com/scipy/scip
> y/blob/master/doc/ROADMAP.rst.txt#misc),
> `scipy.misc` will eventually be removed. Currently, the combinatorial
> functions and the image-related operations are all deprecated.  The only
> non-deprecated functions in `misc` are `central_diff_weights()`,
> `derivative()` and the two functions that return image data: `ascent()` and
> `face()`.
>
> As a steps towards the deprecation of `misc`, I propose that we create a
> new package, `scipy.data`, for holding data sets.  `ascent()` and `face()`
> would move there, and the new ECG data set proposed in a current pull
> request (https://github.com/scipy/scipy/pull/8627) would be put there.
>
> An early version of the roadmap suggested moving the images to
> `scipy.ndimage`, but that is no longer in the text.  I think a separate
> subpackage for data sets makes sense.
>
> What do you think?
>
> P.S. If there is already a similar proposal in the mailing list or on
> github, or any other older mailing list discussions related to this, let me
> know.
>
>
>
> _______________________________________________
> SciPy-Dev mailing list
> SciPy-Dev at python.org
> https://mail.python.org/mailman/listinfo/scipy-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scipy-dev/attachments/20180329/26456307/attachment.html>

From josef.pktd at gmail.com  Thu Mar 29 16:06:02 2018
From: josef.pktd at gmail.com (josef.pktd at gmail.com)
Date: Thu, 29 Mar 2018 16:06:02 -0400
Subject: [SciPy-Dev] New subpackage: scipy.data
In-Reply-To: <CAGu2niVd2RrSt1sqr8KjgiWfBr7iDBU-r0OuwNGK_6DDyXYNWQ@mail.gmail.com>
References: <CAGzF1udgrXmWyXh93fpN9TOVvbmvSEt1UHXpdw6QtNJYBrSi0A@mail.gmail.com>
 <CAGu2niVd2RrSt1sqr8KjgiWfBr7iDBU-r0OuwNGK_6DDyXYNWQ@mail.gmail.com>
Message-ID: <CAMMTP+C25gsCQR6cZePaHS7MM2gkkLq4M5+L0tf6JGc09g-OFg@mail.gmail.com>

I don't like the name scipy.data and would prefer something more
explicit like scipy.datasets.

My first reaction to the proposal was that we get now a subpackage for
functions to work with data.

Like
ndimage are functions for images
signal are functions for signals
spatial are functions for neighborhood

and
data are functions for data   (e.g. like pandas or similar)

Josef

On Thu, Mar 29, 2018 at 3:48 PM, Eric Larson <larson.eric.d at gmail.com> wrote:
> Sounds like a good plan to me.
>
> If others agree, I propose that we make existing (and new) data available in
> scipy.data for 1.1.0. Maybe a 2-release deprecation warning cycle before
> removing them from `scipy.misc` (possibly for removing everything if we can
> move the other remaining things, too)?
>
> Eric
>
>
> On Thu, Mar 29, 2018 at 3:43 PM, Warren Weckesser
> <warren.weckesser at gmail.com> wrote:
>>
>> According to the SciPy roadmap
>> (https://github.com/scipy/scipy/blob/master/doc/ROADMAP.rst.txt#misc),
>> `scipy.misc` will eventually be removed. Currently, the combinatorial
>> functions and the image-related operations are all deprecated.  The only
>> non-deprecated functions in `misc` are `central_diff_weights()`,
>> `derivative()` and the two functions that return image data: `ascent()` and
>> `face()`.
>>
>> As a steps towards the deprecation of `misc`, I propose that we create a
>> new package, `scipy.data`, for holding data sets.  `ascent()` and `face()`
>> would move there, and the new ECG data set proposed in a current pull
>> request (https://github.com/scipy/scipy/pull/8627) would be put there.
>>
>> An early version of the roadmap suggested moving the images to
>> `scipy.ndimage`, but that is no longer in the text.  I think a separate
>> subpackage for data sets makes sense.
>>
>> What do you think?
>>
>> P.S. If there is already a similar proposal in the mailing list or on
>> github, or any other older mailing list discussions related to this, let me
>> know.
>>
>>
>>
>> _______________________________________________
>> SciPy-Dev mailing list
>> SciPy-Dev at python.org
>> https://mail.python.org/mailman/listinfo/scipy-dev
>>
>
>
> _______________________________________________
> SciPy-Dev mailing list
> SciPy-Dev at python.org
> https://mail.python.org/mailman/listinfo/scipy-dev
>

From gael.varoquaux at normalesup.org  Thu Mar 29 16:38:48 2018
From: gael.varoquaux at normalesup.org (Gael Varoquaux)
Date: Thu, 29 Mar 2018 22:38:48 +0200
Subject: [SciPy-Dev] New subpackage: scipy.data
In-Reply-To: <CAMMTP+C25gsCQR6cZePaHS7MM2gkkLq4M5+L0tf6JGc09g-OFg@mail.gmail.com>
References: <CAGzF1udgrXmWyXh93fpN9TOVvbmvSEt1UHXpdw6QtNJYBrSi0A@mail.gmail.com>
 <CAGu2niVd2RrSt1sqr8KjgiWfBr7iDBU-r0OuwNGK_6DDyXYNWQ@mail.gmail.com>
 <CAMMTP+C25gsCQR6cZePaHS7MM2gkkLq4M5+L0tf6JGc09g-OFg@mail.gmail.com>
Message-ID: <20180329203848.GB1453853@phare.normalesup.org>

+1

On Thu, Mar 29, 2018 at 04:06:02PM -0400, josef.pktd at gmail.com wrote:
> I don't like the name scipy.data and would prefer something more
> explicit like scipy.datasets.

> My first reaction to the proposal was that we get now a subpackage for
> functions to work with data.

> Like
> ndimage are functions for images
> signal are functions for signals
> spatial are functions for neighborhood

> and
> data are functions for data   (e.g. like pandas or similar)

> Josef

> On Thu, Mar 29, 2018 at 3:48 PM, Eric Larson <larson.eric.d at gmail.com> wrote:
> > Sounds like a good plan to me.

> > If others agree, I propose that we make existing (and new) data available in
> > scipy.data for 1.1.0. Maybe a 2-release deprecation warning cycle before
> > removing them from `scipy.misc` (possibly for removing everything if we can
> > move the other remaining things, too)?

> > Eric


> > On Thu, Mar 29, 2018 at 3:43 PM, Warren Weckesser
> > <warren.weckesser at gmail.com> wrote:

> >> According to the SciPy roadmap
> >> (https://github.com/scipy/scipy/blob/master/doc/ROADMAP.rst.txt#misc),
> >> `scipy.misc` will eventually be removed. Currently, the combinatorial
> >> functions and the image-related operations are all deprecated.  The only
> >> non-deprecated functions in `misc` are `central_diff_weights()`,
> >> `derivative()` and the two functions that return image data: `ascent()` and
> >> `face()`.

> >> As a steps towards the deprecation of `misc`, I propose that we create a
> >> new package, `scipy.data`, for holding data sets.  `ascent()` and `face()`
> >> would move there, and the new ECG data set proposed in a current pull
> >> request (https://github.com/scipy/scipy/pull/8627) would be put there.

> >> An early version of the roadmap suggested moving the images to
> >> `scipy.ndimage`, but that is no longer in the text.  I think a separate
> >> subpackage for data sets makes sense.

> >> What do you think?

> >> P.S. If there is already a similar proposal in the mailing list or on
> >> github, or any other older mailing list discussions related to this, let me
> >> know.



> >> _______________________________________________
> >> SciPy-Dev mailing list
> >> SciPy-Dev at python.org
> >> https://mail.python.org/mailman/listinfo/scipy-dev



> > _______________________________________________
> > SciPy-Dev mailing list
> > SciPy-Dev at python.org
> > https://mail.python.org/mailman/listinfo/scipy-dev

> _______________________________________________
> SciPy-Dev mailing list
> SciPy-Dev at python.org
> https://mail.python.org/mailman/listinfo/scipy-dev

-- 
    Gael Varoquaux
    Senior Researcher, INRIA Parietal
    NeuroSpin/CEA Saclay , Bat 145, 91191 Gif-sur-Yvette France
    Phone:  ++ 33-1-69-08-79-68
    http://gael-varoquaux.info            http://twitter.com/GaelVaroquaux

From bennet at umich.edu  Thu Mar 29 17:08:46 2018
From: bennet at umich.edu (Bennet Fauber)
Date: Thu, 29 Mar 2018 17:08:46 -0400
Subject: [SciPy-Dev] New subpackage: scipy.data
In-Reply-To: <20180329203848.GB1453853@phare.normalesup.org>
References: <CAGzF1udgrXmWyXh93fpN9TOVvbmvSEt1UHXpdw6QtNJYBrSi0A@mail.gmail.com>
 <CAGu2niVd2RrSt1sqr8KjgiWfBr7iDBU-r0OuwNGK_6DDyXYNWQ@mail.gmail.com>
 <CAMMTP+C25gsCQR6cZePaHS7MM2gkkLq4M5+L0tf6JGc09g-OFg@mail.gmail.com>
 <20180329203848.GB1453853@phare.normalesup.org>
Message-ID: <CAB2ovovN9sKsaGP9ziHzLj1-upHkPdFzR9GthoFH2=+prGiE=A@mail.gmail.com>

I also agree -- if this is where datasets are to be kept and made
available, then 'datasets' is probably a better name.  That also
agrees with the R package called 'datasets', and so might be
considered a more 'customary' name.



On Thu, Mar 29, 2018 at 4:38 PM, Gael Varoquaux
<gael.varoquaux at normalesup.org> wrote:
> +1
>
> On Thu, Mar 29, 2018 at 04:06:02PM -0400, josef.pktd at gmail.com wrote:
>> I don't like the name scipy.data and would prefer something more
>> explicit like scipy.datasets.
>
>> My first reaction to the proposal was that we get now a subpackage for
>> functions to work with data.
>
>> Like
>> ndimage are functions for images
>> signal are functions for signals
>> spatial are functions for neighborhood
>
>> and
>> data are functions for data   (e.g. like pandas or similar)
>
>> Josef
>
>> On Thu, Mar 29, 2018 at 3:48 PM, Eric Larson <larson.eric.d at gmail.com> wrote:
>> > Sounds like a good plan to me.
>
>> > If others agree, I propose that we make existing (and new) data available in
>> > scipy.data for 1.1.0. Maybe a 2-release deprecation warning cycle before
>> > removing them from `scipy.misc` (possibly for removing everything if we can
>> > move the other remaining things, too)?
>
>> > Eric
>
>
>> > On Thu, Mar 29, 2018 at 3:43 PM, Warren Weckesser
>> > <warren.weckesser at gmail.com> wrote:
>
>> >> According to the SciPy roadmap
>> >> (https://github.com/scipy/scipy/blob/master/doc/ROADMAP.rst.txt#misc),
>> >> `scipy.misc` will eventually be removed. Currently, the combinatorial
>> >> functions and the image-related operations are all deprecated.  The only
>> >> non-deprecated functions in `misc` are `central_diff_weights()`,
>> >> `derivative()` and the two functions that return image data: `ascent()` and
>> >> `face()`.
>
>> >> As a steps towards the deprecation of `misc`, I propose that we create a
>> >> new package, `scipy.data`, for holding data sets.  `ascent()` and `face()`
>> >> would move there, and the new ECG data set proposed in a current pull
>> >> request (https://github.com/scipy/scipy/pull/8627) would be put there.
>
>> >> An early version of the roadmap suggested moving the images to
>> >> `scipy.ndimage`, but that is no longer in the text.  I think a separate
>> >> subpackage for data sets makes sense.
>
>> >> What do you think?
>
>> >> P.S. If there is already a similar proposal in the mailing list or on
>> >> github, or any other older mailing list discussions related to this, let me
>> >> know.
>
>
>
>> >> _______________________________________________
>> >> SciPy-Dev mailing list
>> >> SciPy-Dev at python.org
>> >> https://mail.python.org/mailman/listinfo/scipy-dev
>
>
>
>> > _______________________________________________
>> > SciPy-Dev mailing list
>> > SciPy-Dev at python.org
>> > https://mail.python.org/mailman/listinfo/scipy-dev
>
>> _______________________________________________
>> SciPy-Dev mailing list
>> SciPy-Dev at python.org
>> https://mail.python.org/mailman/listinfo/scipy-dev
>
> --
>     Gael Varoquaux
>     Senior Researcher, INRIA Parietal
>     NeuroSpin/CEA Saclay , Bat 145, 91191 Gif-sur-Yvette France
>     Phone:  ++ 33-1-69-08-79-68
>     http://gael-varoquaux.info            http://twitter.com/GaelVaroquaux
> _______________________________________________
> SciPy-Dev mailing list
> SciPy-Dev at python.org
> https://mail.python.org/mailman/listinfo/scipy-dev

From lagru at mailbox.org  Thu Mar 29 17:27:47 2018
From: lagru at mailbox.org (Lars G.)
Date: Thu, 29 Mar 2018 23:27:47 +0200
Subject: [SciPy-Dev] New subpackage: scipy.data
In-Reply-To: <CAB2ovovN9sKsaGP9ziHzLj1-upHkPdFzR9GthoFH2=+prGiE=A@mail.gmail.com>
References: <CAGzF1udgrXmWyXh93fpN9TOVvbmvSEt1UHXpdw6QtNJYBrSi0A@mail.gmail.com>
 <CAGu2niVd2RrSt1sqr8KjgiWfBr7iDBU-r0OuwNGK_6DDyXYNWQ@mail.gmail.com>
 <CAMMTP+C25gsCQR6cZePaHS7MM2gkkLq4M5+L0tf6JGc09g-OFg@mail.gmail.com>
 <20180329203848.GB1453853@phare.normalesup.org>
 <CAB2ovovN9sKsaGP9ziHzLj1-upHkPdFzR9GthoFH2=+prGiE=A@mail.gmail.com>
Message-ID: <869adbc5-b25f-c5c2-1e66-f59983b88bc9@mailbox.org>

Is there a reason not to include those function where they'll most
likely be used? Meaning the images `ascent` and `face` could move to
`scipy.ndimage` and the signal `electrocardiogram` to `scipy.signal`?

Lars

On 29.03.2018 23:08, Bennet Fauber wrote:
> I also agree -- if this is where datasets are to be kept and made
> available, then 'datasets' is probably a better name.  That also
> agrees with the R package called 'datasets', and so might be
> considered a more 'customary' name.
>
>
> On Thu, Mar 29, 2018 at 4:38 PM, Gael Varoquaux
> <gael.varoquaux at normalesup.org> wrote:
>> +1
>>
>> On Thu, Mar 29, 2018 at 04:06:02PM -0400, josef.pktd at gmail.com wrote:
>>> I don't like the name scipy.data and would prefer something more
>>> explicit like scipy.datasets.
>>
>>> My first reaction to the proposal was that we get now a subpackage for
>>> functions to work with data.
>>
>>> Like
>>> ndimage are functions for images
>>> signal are functions for signals
>>> spatial are functions for neighborhood
>>
>>> and
>>> data are functions for data   (e.g. like pandas or similar)
>>
>>> Josef

From robert.kern at gmail.com  Thu Mar 29 17:34:14 2018
From: robert.kern at gmail.com (Robert Kern)
Date: Thu, 29 Mar 2018 14:34:14 -0700
Subject: [SciPy-Dev] New subpackage: scipy.data
In-Reply-To: <869adbc5-b25f-c5c2-1e66-f59983b88bc9@mailbox.org>
References: <CAGzF1udgrXmWyXh93fpN9TOVvbmvSEt1UHXpdw6QtNJYBrSi0A@mail.gmail.com>
 <CAGu2niVd2RrSt1sqr8KjgiWfBr7iDBU-r0OuwNGK_6DDyXYNWQ@mail.gmail.com>
 <CAMMTP+C25gsCQR6cZePaHS7MM2gkkLq4M5+L0tf6JGc09g-OFg@mail.gmail.com>
 <20180329203848.GB1453853@phare.normalesup.org>
 <CAB2ovovN9sKsaGP9ziHzLj1-upHkPdFzR9GthoFH2=+prGiE=A@mail.gmail.com>
 <869adbc5-b25f-c5c2-1e66-f59983b88bc9@mailbox.org>
Message-ID: <CAF6FJiu4mQYA7p9yEb8gEUWObZJhAvA3oeCx8ws1tnzg33uNHw@mail.gmail.com>

On Thu, Mar 29, 2018 at 2:27 PM, Lars G. <lagru at mailbox.org> wrote:
>
> Is there a reason not to include those function where they'll most
> likely be used? Meaning the images `ascent` and `face` could move to
> `scipy.ndimage` and the signal `electrocardiogram` to `scipy.signal`?

It would make them harder to discover, at least for me.

On the developer side, if everything is in one subpackage, it would be
easier to keep track how many bytes are being consumed by data. The
scipy.datasets namespace would be a good place to put any common
data-loading code (for instance, if we start adding large datasets that
will be downloaded upon first use rather than being distributed in the
scipy wheel).

--
Robert Kern
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scipy-dev/attachments/20180329/926c0659/attachment.html>

From bennet at umich.edu  Thu Mar 29 17:47:13 2018
From: bennet at umich.edu (Bennet Fauber)
Date: Thu, 29 Mar 2018 17:47:13 -0400
Subject: [SciPy-Dev] New subpackage: scipy.data
In-Reply-To: <CAF6FJiu4mQYA7p9yEb8gEUWObZJhAvA3oeCx8ws1tnzg33uNHw@mail.gmail.com>
References: <CAGzF1udgrXmWyXh93fpN9TOVvbmvSEt1UHXpdw6QtNJYBrSi0A@mail.gmail.com>
 <CAGu2niVd2RrSt1sqr8KjgiWfBr7iDBU-r0OuwNGK_6DDyXYNWQ@mail.gmail.com>
 <CAMMTP+C25gsCQR6cZePaHS7MM2gkkLq4M5+L0tf6JGc09g-OFg@mail.gmail.com>
 <20180329203848.GB1453853@phare.normalesup.org>
 <CAB2ovovN9sKsaGP9ziHzLj1-upHkPdFzR9GthoFH2=+prGiE=A@mail.gmail.com>
 <869adbc5-b25f-c5c2-1e66-f59983b88bc9@mailbox.org>
 <CAF6FJiu4mQYA7p9yEb8gEUWObZJhAvA3oeCx8ws1tnzg33uNHw@mail.gmail.com>
Message-ID: <CAB2ovovQu3mzXS-5NaTsVLiKREJUTiuyUsbHqr3QwZxQG+8YLg@mail.gmail.com>

Would having a single datasets library increase visibility and
potentially encourage the use of one dataset for multiple purposes?
If they are roughly indexed, as the ones at the R datasets package
site are, that could also be helpful for people who are finding their
way to analytic capability via the catalog of examples.  Someone
looking for electorcardiogram might get led to signal that way, if
that matters.




On Thu, Mar 29, 2018 at 5:34 PM, Robert Kern <robert.kern at gmail.com> wrote:
> On Thu, Mar 29, 2018 at 2:27 PM, Lars G. <lagru at mailbox.org> wrote:
>>
>> Is there a reason not to include those function where they'll most
>> likely be used? Meaning the images `ascent` and `face` could move to
>> `scipy.ndimage` and the signal `electrocardiogram` to `scipy.signal`?
>
> It would make them harder to discover, at least for me.
>
> On the developer side, if everything is in one subpackage, it would be
> easier to keep track how many bytes are being consumed by data. The
> scipy.datasets namespace would be a good place to put any common
> data-loading code (for instance, if we start adding large datasets that will
> be downloaded upon first use rather than being distributed in the scipy
> wheel).
>
> --
> Robert Kern
>
> _______________________________________________
> SciPy-Dev mailing list
> SciPy-Dev at python.org
> https://mail.python.org/mailman/listinfo/scipy-dev
>

From ilhanpolat at gmail.com  Thu Mar 29 18:08:02 2018
From: ilhanpolat at gmail.com (Ilhan Polat)
Date: Fri, 30 Mar 2018 00:08:02 +0200
Subject: [SciPy-Dev] New subpackage: scipy.data
In-Reply-To: <CAB2ovovQu3mzXS-5NaTsVLiKREJUTiuyUsbHqr3QwZxQG+8YLg@mail.gmail.com>
References: <CAGzF1udgrXmWyXh93fpN9TOVvbmvSEt1UHXpdw6QtNJYBrSi0A@mail.gmail.com>
 <CAGu2niVd2RrSt1sqr8KjgiWfBr7iDBU-r0OuwNGK_6DDyXYNWQ@mail.gmail.com>
 <CAMMTP+C25gsCQR6cZePaHS7MM2gkkLq4M5+L0tf6JGc09g-OFg@mail.gmail.com>
 <20180329203848.GB1453853@phare.normalesup.org>
 <CAB2ovovN9sKsaGP9ziHzLj1-upHkPdFzR9GthoFH2=+prGiE=A@mail.gmail.com>
 <869adbc5-b25f-c5c2-1e66-f59983b88bc9@mailbox.org>
 <CAF6FJiu4mQYA7p9yEb8gEUWObZJhAvA3oeCx8ws1tnzg33uNHw@mail.gmail.com>
 <CAB2ovovQu3mzXS-5NaTsVLiKREJUTiuyUsbHqr3QwZxQG+8YLg@mail.gmail.com>
Message-ID: <CAEBuzr_LyzGh9W-9iF_USrbjp7S6Fn9QqieUk2Md=83WYsoc=w@mail.gmail.com>

I agree with Eric: duplicating the existing ones in sp.datasets right away
and placing the appropriate deprecation warnings seems like a good way to
get rid of it.

On Thu, Mar 29, 2018 at 11:47 PM, Bennet Fauber <bennet at umich.edu> wrote:

> Would having a single datasets library increase visibility and
> potentially encourage the use of one dataset for multiple purposes?
> If they are roughly indexed, as the ones at the R datasets package
> site are, that could also be helpful for people who are finding their
> way to analytic capability via the catalog of examples.  Someone
> looking for electorcardiogram might get led to signal that way, if
> that matters.
>
>
>
>
> On Thu, Mar 29, 2018 at 5:34 PM, Robert Kern <robert.kern at gmail.com>
> wrote:
> > On Thu, Mar 29, 2018 at 2:27 PM, Lars G. <lagru at mailbox.org> wrote:
> >>
> >> Is there a reason not to include those function where they'll most
> >> likely be used? Meaning the images `ascent` and `face` could move to
> >> `scipy.ndimage` and the signal `electrocardiogram` to `scipy.signal`?
> >
> > It would make them harder to discover, at least for me.
> >
> > On the developer side, if everything is in one subpackage, it would be
> > easier to keep track how many bytes are being consumed by data. The
> > scipy.datasets namespace would be a good place to put any common
> > data-loading code (for instance, if we start adding large datasets that
> will
> > be downloaded upon first use rather than being distributed in the scipy
> > wheel).
> >
> > --
> > Robert Kern
> >
> > _______________________________________________
> > SciPy-Dev mailing list
> > SciPy-Dev at python.org
> > https://mail.python.org/mailman/listinfo/scipy-dev
> >
> _______________________________________________
> SciPy-Dev mailing list
> SciPy-Dev at python.org
> https://mail.python.org/mailman/listinfo/scipy-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scipy-dev/attachments/20180330/8729ec26/attachment-0001.html>

From warren.weckesser at gmail.com  Thu Mar 29 18:17:06 2018
From: warren.weckesser at gmail.com (Warren Weckesser)
Date: Thu, 29 Mar 2018 18:17:06 -0400
Subject: [SciPy-Dev] New subpackage: scipy.data
In-Reply-To: <CAMMTP+C25gsCQR6cZePaHS7MM2gkkLq4M5+L0tf6JGc09g-OFg@mail.gmail.com>
References: <CAGzF1udgrXmWyXh93fpN9TOVvbmvSEt1UHXpdw6QtNJYBrSi0A@mail.gmail.com>
 <CAGu2niVd2RrSt1sqr8KjgiWfBr7iDBU-r0OuwNGK_6DDyXYNWQ@mail.gmail.com>
 <CAMMTP+C25gsCQR6cZePaHS7MM2gkkLq4M5+L0tf6JGc09g-OFg@mail.gmail.com>
Message-ID: <CAGzF1udZob1uPgthNdGePr-9mvw4G9acXNNZvRvETfBKVJfUew@mail.gmail.com>

On Thu, Mar 29, 2018 at 4:06 PM, <josef.pktd at gmail.com> wrote:

> I don't like the name scipy.data and would prefer something more
> explicit like scipy.datasets.
>
>

`datasets` is fine with me, and based on other responses so far, other
folks have either implicitly or explicitly endorsed it, so lets go with it.

Warren



> My first reaction to the proposal was that we get now a subpackage for
> functions to work with data.
>
> Like
> ndimage are functions for images
> signal are functions for signals
> spatial are functions for neighborhood
>
> and
> data are functions for data   (e.g. like pandas or similar)
>
> Josef
>
> On Thu, Mar 29, 2018 at 3:48 PM, Eric Larson <larson.eric.d at gmail.com>
> wrote:
> > Sounds like a good plan to me.
> >
> > If others agree, I propose that we make existing (and new) data
> available in
> > scipy.data for 1.1.0. Maybe a 2-release deprecation warning cycle before
> > removing them from `scipy.misc` (possibly for removing everything if we
> can
> > move the other remaining things, too)?
> >
> > Eric
> >
> >
> > On Thu, Mar 29, 2018 at 3:43 PM, Warren Weckesser
> > <warren.weckesser at gmail.com> wrote:
> >>
> >> According to the SciPy roadmap
> >> (https://github.com/scipy/scipy/blob/master/doc/ROADMAP.rst.txt#misc),
> >> `scipy.misc` will eventually be removed. Currently, the combinatorial
> >> functions and the image-related operations are all deprecated.  The only
> >> non-deprecated functions in `misc` are `central_diff_weights()`,
> >> `derivative()` and the two functions that return image data: `ascent()`
> and
> >> `face()`.
> >>
> >> As a steps towards the deprecation of `misc`, I propose that we create a
> >> new package, `scipy.data`, for holding data sets.  `ascent()` and
> `face()`
> >> would move there, and the new ECG data set proposed in a current pull
> >> request (https://github.com/scipy/scipy/pull/8627) would be put there.
> >>
> >> An early version of the roadmap suggested moving the images to
> >> `scipy.ndimage`, but that is no longer in the text.  I think a separate
> >> subpackage for data sets makes sense.
> >>
> >> What do you think?
> >>
> >> P.S. If there is already a similar proposal in the mailing list or on
> >> github, or any other older mailing list discussions related to this,
> let me
> >> know.
> >>
> >>
> >>
> >> _______________________________________________
> >> SciPy-Dev mailing list
> >> SciPy-Dev at python.org
> >> https://mail.python.org/mailman/listinfo/scipy-dev
> >>
> >
> >
> > _______________________________________________
> > SciPy-Dev mailing list
> > SciPy-Dev at python.org
> > https://mail.python.org/mailman/listinfo/scipy-dev
> >
> _______________________________________________
> SciPy-Dev mailing list
> SciPy-Dev at python.org
> https://mail.python.org/mailman/listinfo/scipy-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scipy-dev/attachments/20180329/97273c0c/attachment.html>

From stefanv at berkeley.edu  Thu Mar 29 18:45:35 2018
From: stefanv at berkeley.edu (Stefan van der Walt)
Date: Thu, 29 Mar 2018 15:45:35 -0700
Subject: [SciPy-Dev] New subpackage: scipy.data
In-Reply-To: <CAGzF1udgrXmWyXh93fpN9TOVvbmvSEt1UHXpdw6QtNJYBrSi0A@mail.gmail.com>
References: <CAGzF1udgrXmWyXh93fpN9TOVvbmvSEt1UHXpdw6QtNJYBrSi0A@mail.gmail.com>
Message-ID: <20180329224535.nyn575wvqfcpg7qu@carbo>

On Thu, 29 Mar 2018 15:43:50 -0400, Warren Weckesser wrote:
> As a steps towards the deprecation of `misc`, I propose that we create a
> new package, `scipy.data`, for holding data sets.  `ascent()` and `face()`
> would move there, and the new ECG data set proposed in a current pull
> request (https://github.com/scipy/scipy/pull/8627) would be put there.

We've been doing this in scikit-image for a long time, and now regret
having any binary data in the repository; we are working on a way of
hosting it outside instead.

Can we standardize on downloader tools?  There are examples in
scikit-learn, dipy, and many other packages.  We were thinking of a very
lightweight spec + tools for solving this problem a while ago, but never
got very far:

https://github.com/data-pack/data-pack/pull/1/files

Best regards
St?fan

From einstein.edison at gmail.com  Thu Mar 29 18:50:31 2018
From: einstein.edison at gmail.com (Hameer Abbasi)
Date: Thu, 29 Mar 2018 15:50:31 -0700
Subject: [SciPy-Dev] New subpackage: scipy.data
In-Reply-To: <20180329224535.nyn575wvqfcpg7qu@carbo>
References: <CAGzF1udgrXmWyXh93fpN9TOVvbmvSEt1UHXpdw6QtNJYBrSi0A@mail.gmail.com>
 <20180329224535.nyn575wvqfcpg7qu@carbo>
Message-ID: <CADViA5AUS+8ifMwLFEY6HN2cxrFhLsOzO8CnrkRgLkD4FERwBA@mail.gmail.com>

I concur with Stefan. Not having datasets in a package seems like the best
way to go. There should be a separate go-to place for datasets (other than
minimal ones for test cases).

I would recommend branching off all datasets... Otherwise we add to Scipy's
already significant size.

On 30/03/2018 at 00:45, Stefan wrote:

On Thu, 29 Mar 2018 15:43:50 -0400, Warren Weckesser wrote:

As a steps towards the deprecation of `misc`, I propose that we create a
new package, `scipy.data`, for holding data sets. `ascent()` and `face()`
would move there, and the new ECG data set proposed in a current pull
request (https://github.com/scipy/scipy/pull/8627) would be put there.


We've been doing this in scikit-image for a long time, and now regret
having any binary data in the repository; we are working on a way of
hosting it outside instead.

Can we standardize on downloader tools? There are examples in
scikit-learn, dipy, and many other packages. We were thinking of a very
lightweight spec + tools for solving this problem a while ago, but never
got very far:

https://github.com/data-pack/data-pack/pull/1/files

Best regards
St?fan
_______________________________________________
SciPy-Dev mailing list
SciPy-Dev at python.org
https://mail.python.org/mailman/listinfo/scipy-dev
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scipy-dev/attachments/20180329/b191dbb5/attachment.html>

From warren.weckesser at gmail.com  Thu Mar 29 18:54:52 2018
From: warren.weckesser at gmail.com (Warren Weckesser)
Date: Thu, 29 Mar 2018 18:54:52 -0400
Subject: [SciPy-Dev] New subpackage: scipy.data
In-Reply-To: <20180329224535.nyn575wvqfcpg7qu@carbo>
References: <CAGzF1udgrXmWyXh93fpN9TOVvbmvSEt1UHXpdw6QtNJYBrSi0A@mail.gmail.com>
 <20180329224535.nyn575wvqfcpg7qu@carbo>
Message-ID: <CAGzF1uevvstriBxtRudTYFkNxDh_4sEYh627++WP08it-WR8ag@mail.gmail.com>

On Thu, Mar 29, 2018 at 6:45 PM, Stefan van der Walt <stefanv at berkeley.edu>
wrote:

> On Thu, 29 Mar 2018 15:43:50 -0400, Warren Weckesser wrote:
> > As a steps towards the deprecation of `misc`, I propose that we create a
> > new package, `scipy.data`, for holding data sets.  `ascent()` and
> `face()`
> > would move there, and the new ECG data set proposed in a current pull
> > request (https://github.com/scipy/scipy/pull/8627) would be put there.
>
> We've been doing this in scikit-image for a long time, and now regret
> having any binary data in the repository;



Can you summarize the problems that make you regret including the data?

Warren



> we are working on a way of
> hosting it outside instead.
>
> Can we standardize on downloader tools?  There are examples in
> scikit-learn, dipy, and many other packages.  We were thinking of a very
> lightweight spec + tools for solving this problem a while ago, but never
> got very far:
>
> https://github.com/data-pack/data-pack/pull/1/files
>
> Best regards
> St?fan
> _______________________________________________
> SciPy-Dev mailing list
> SciPy-Dev at python.org
> https://mail.python.org/mailman/listinfo/scipy-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scipy-dev/attachments/20180329/c091aef7/attachment-0001.html>

From stefanv at berkeley.edu  Thu Mar 29 19:16:17 2018
From: stefanv at berkeley.edu (Stefan van der Walt)
Date: Thu, 29 Mar 2018 16:16:17 -0700
Subject: [SciPy-Dev] New subpackage: scipy.data
In-Reply-To: <CAGzF1uevvstriBxtRudTYFkNxDh_4sEYh627++WP08it-WR8ag@mail.gmail.com>
References: <CAGzF1udgrXmWyXh93fpN9TOVvbmvSEt1UHXpdw6QtNJYBrSi0A@mail.gmail.com>
 <20180329224535.nyn575wvqfcpg7qu@carbo>
 <CAGzF1uevvstriBxtRudTYFkNxDh_4sEYh627++WP08it-WR8ag@mail.gmail.com>
Message-ID: <20180329231617.cdyxvf6ysip4fth3@carbo>

On Thu, 29 Mar 2018 18:54:52 -0400, Warren Weckesser wrote:
> Can you summarize the problems that make you regret including the
> data?

- The size of the repository (extra time on each clone, and that for
  data that isn't necessary in most use cases)
  
- Artificial limit on data sizes: we now have a default place to store
  data, but we still need an additional mechanism for larger datasets.
  How do you choose the threshold for what goes in, what is too big?
  
- Because these tiny embedded datasets are easily available, they become
  the default for demos.  If data is stored externally, realistic
  examples become more feasible and likely.

Best regards
St?fan

From joseph.c.slater at gmail.com  Thu Mar 29 19:17:56 2018
From: joseph.c.slater at gmail.com (Joseph Slater)
Date: Thu, 29 Mar 2018 19:17:56 -0400
Subject: [SciPy-Dev] Thoughts on creating a toolbox for analysis of
 nonlinear dynamical system analysis
In-Reply-To: <237320890.4090.1522078894360@wamui-kristoff.atl.sa.earthlink.net>
References: <237320890.4090.1522078894360@wamui-kristoff.atl.sa.earthlink.net>
Message-ID: <CFCEDB2D-913D-48E9-A5C5-D4D0DA48C3CB@gmail.com>



> On Mar 26, 2018, at 11:41 AM, Robert Lucente <rlucente at pipeline.com> wrote:
> 
> >In either case, I'm happy to chat about it
> Thanks for the offer but I would be wasting your time.
> 
> I am a newbie to Python and Git.
> 


There are plenty of ways to help. Just testing documents, working to fix them, for my stuff: I need examples run in jupyter notebooks for the manuals. Being new brings valuable insight as to how new users can get lost. 

Best Regards- Joe


From ilhanpolat at gmail.com  Thu Mar 29 19:54:10 2018
From: ilhanpolat at gmail.com (Ilhan Polat)
Date: Fri, 30 Mar 2018 01:54:10 +0200
Subject: [SciPy-Dev] New subpackage: scipy.data
In-Reply-To: <20180329231617.cdyxvf6ysip4fth3@carbo>
References: <CAGzF1udgrXmWyXh93fpN9TOVvbmvSEt1UHXpdw6QtNJYBrSi0A@mail.gmail.com>
 <20180329224535.nyn575wvqfcpg7qu@carbo>
 <CAGzF1uevvstriBxtRudTYFkNxDh_4sEYh627++WP08it-WR8ag@mail.gmail.com>
 <20180329231617.cdyxvf6ysip4fth3@carbo>
Message-ID: <CAEBuzr-qY0V4+C-A80c=5O0+wZwjE-CdJ4VXx02844JMEfA6hQ@mail.gmail.com>

Would a separate repo scipy-datasets help ? Then something like

try:
     importing
except:
    warn('I'm off to interwebz')
    download from the repo

might be feasible. The download part can either be that particular dataset
or the whole scipy-datasets clone.




On Fri, Mar 30, 2018 at 1:16 AM, Stefan van der Walt <stefanv at berkeley.edu>
wrote:

> On Thu, 29 Mar 2018 18:54:52 -0400, Warren Weckesser wrote:
> > Can you summarize the problems that make you regret including the
> > data?
>
> - The size of the repository (extra time on each clone, and that for
>   data that isn't necessary in most use cases)
>
> - Artificial limit on data sizes: we now have a default place to store
>   data, but we still need an additional mechanism for larger datasets.
>   How do you choose the threshold for what goes in, what is too big?
>
> - Because these tiny embedded datasets are easily available, they become
>   the default for demos.  If data is stored externally, realistic
>   examples become more feasible and likely.
>
> Best regards
> St?fan
> _______________________________________________
> SciPy-Dev mailing list
> SciPy-Dev at python.org
> https://mail.python.org/mailman/listinfo/scipy-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scipy-dev/attachments/20180330/a6b82d44/attachment.html>

From josef.pktd at gmail.com  Thu Mar 29 20:03:20 2018
From: josef.pktd at gmail.com (josef.pktd at gmail.com)
Date: Thu, 29 Mar 2018 20:03:20 -0400
Subject: [SciPy-Dev] New subpackage: scipy.data
In-Reply-To: <CAEBuzr-qY0V4+C-A80c=5O0+wZwjE-CdJ4VXx02844JMEfA6hQ@mail.gmail.com>
References: <CAGzF1udgrXmWyXh93fpN9TOVvbmvSEt1UHXpdw6QtNJYBrSi0A@mail.gmail.com>
 <20180329224535.nyn575wvqfcpg7qu@carbo>
 <CAGzF1uevvstriBxtRudTYFkNxDh_4sEYh627++WP08it-WR8ag@mail.gmail.com>
 <20180329231617.cdyxvf6ysip4fth3@carbo>
 <CAEBuzr-qY0V4+C-A80c=5O0+wZwjE-CdJ4VXx02844JMEfA6hQ@mail.gmail.com>
Message-ID: <CAMMTP+C7K3XxMw=PRyivYdKs7-7WBzLUam2jWr6OGpbX666d1g@mail.gmail.com>

On Thu, Mar 29, 2018 at 7:54 PM, Ilhan Polat <ilhanpolat at gmail.com> wrote:
> Would a separate repo scipy-datasets help ? Then something like
>
> try:
>      importing
> except:
>     warn('I'm off to interwebz')
>     download from the repo
>
> might be feasible. The download part can either be that particular dataset
> or the whole scipy-datasets clone.
>

IMO:

It depends on the scale where this should go.
I don't think it's worth it (maintaining and installing another
package or repo) for scipy
given that scipy is mostly a basic numerical library and not driven by specific
applications.

For most areas there should be already some online repos or packages and
it would be enough to have the accessing functions in scipy.datasets.
The only area that I can think of where there might not be some readily
available online source for datasets is signal.

Josef


>
>
>
> On Fri, Mar 30, 2018 at 1:16 AM, Stefan van der Walt <stefanv at berkeley.edu>
> wrote:
>>
>> On Thu, 29 Mar 2018 18:54:52 -0400, Warren Weckesser wrote:
>> > Can you summarize the problems that make you regret including the
>> > data?
>>
>> - The size of the repository (extra time on each clone, and that for
>>   data that isn't necessary in most use cases)
>>
>> - Artificial limit on data sizes: we now have a default place to store
>>   data, but we still need an additional mechanism for larger datasets.
>>   How do you choose the threshold for what goes in, what is too big?
>>
>> - Because these tiny embedded datasets are easily available, they become
>>   the default for demos.  If data is stored externally, realistic
>>   examples become more feasible and likely.
>>
>> Best regards
>> St?fan
>> _______________________________________________
>> SciPy-Dev mailing list
>> SciPy-Dev at python.org
>> https://mail.python.org/mailman/listinfo/scipy-dev
>
>
>
> _______________________________________________
> SciPy-Dev mailing list
> SciPy-Dev at python.org
> https://mail.python.org/mailman/listinfo/scipy-dev
>

From josef.pktd at gmail.com  Thu Mar 29 19:44:18 2018
From: josef.pktd at gmail.com (josef.pktd at gmail.com)
Date: Thu, 29 Mar 2018 19:44:18 -0400
Subject: [SciPy-Dev] New subpackage: scipy.data
In-Reply-To: <20180329231617.cdyxvf6ysip4fth3@carbo>
References: <CAGzF1udgrXmWyXh93fpN9TOVvbmvSEt1UHXpdw6QtNJYBrSi0A@mail.gmail.com>
 <20180329224535.nyn575wvqfcpg7qu@carbo>
 <CAGzF1uevvstriBxtRudTYFkNxDh_4sEYh627++WP08it-WR8ag@mail.gmail.com>
 <20180329231617.cdyxvf6ysip4fth3@carbo>
Message-ID: <CAMMTP+DPPXUfNQyTTUuAgKksxNHDH18BGCaNBcLgc3qmOpHxQw@mail.gmail.com>

On Thu, Mar 29, 2018 at 7:16 PM, Stefan van der Walt
<stefanv at berkeley.edu> wrote:
> On Thu, 29 Mar 2018 18:54:52 -0400, Warren Weckesser wrote:
>> Can you summarize the problems that make you regret including the
>> data?
>
> - The size of the repository (extra time on each clone, and that for
>   data that isn't necessary in most use cases)
>
> - Artificial limit on data sizes: we now have a default place to store
>   data, but we still need an additional mechanism for larger datasets.
>   How do you choose the threshold for what goes in, what is too big?
>
> - Because these tiny embedded datasets are easily available, they become
>   the default for demos.  If data is stored externally, realistic
>   examples become more feasible and likely.

In statsmodels we included datasets from the beginning both for unit tests
and for examples. By today's standard these are almost all tiny datasets.
The advantage is that many of them are old textbook dataset that often
illustrate
a problem that we can run into, while clean random generated
data is often boring.

Unit test don't have access to the internet on Debian, so there is still
the restriction of either using internal data or random data.

For notebook we rely now often on downloading from `rdatasets`, or
even having the user download a zip file if the license situation is not
clear, e.g. downloading from the supplementary material to books.

About tools for downloading datasets:
We have a helper function to download from rdatasets and a helper
function to download Stata files from the internet. Essentially all
other datasets are handled by pandas.

It's a simpler case for statsmodels because all datasets essentially
correspond  to a csv file that might be stored in another format.

Josef


>
> Best regards
> St?fan
> _______________________________________________
> SciPy-Dev mailing list
> SciPy-Dev at python.org
> https://mail.python.org/mailman/listinfo/scipy-dev

From ilhanpolat at gmail.com  Thu Mar 29 20:10:10 2018
From: ilhanpolat at gmail.com (Ilhan Polat)
Date: Fri, 30 Mar 2018 02:10:10 +0200
Subject: [SciPy-Dev] New subpackage: scipy.data
In-Reply-To: <CAMMTP+C7K3XxMw=PRyivYdKs7-7WBzLUam2jWr6OGpbX666d1g@mail.gmail.com>
References: <CAGzF1udgrXmWyXh93fpN9TOVvbmvSEt1UHXpdw6QtNJYBrSi0A@mail.gmail.com>
 <20180329224535.nyn575wvqfcpg7qu@carbo>
 <CAGzF1uevvstriBxtRudTYFkNxDh_4sEYh627++WP08it-WR8ag@mail.gmail.com>
 <20180329231617.cdyxvf6ysip4fth3@carbo>
 <CAEBuzr-qY0V4+C-A80c=5O0+wZwjE-CdJ4VXx02844JMEfA6hQ@mail.gmail.com>
 <CAMMTP+C7K3XxMw=PRyivYdKs7-7WBzLUam2jWr6OGpbX666d1g@mail.gmail.com>
Message-ID: <CAEBuzr-qtBryrHZ=sCW1JN0FMZqByBucK4G7f7FJbmk3F-=86Q@mail.gmail.com>

Yes, that's true but GitHub seems like a robust place to live. Otherwise we
can just point to any hardcoded URL. But if the size gets bigger in terms
of wheels and cloning I think within SciPy doesn't seem to be a viable
option. These all depend on what the future of datasets would be.

On Fri, Mar 30, 2018 at 2:03 AM, <josef.pktd at gmail.com> wrote:

> On Thu, Mar 29, 2018 at 7:54 PM, Ilhan Polat <ilhanpolat at gmail.com> wrote:
> > Would a separate repo scipy-datasets help ? Then something like
> >
> > try:
> >      importing
> > except:
> >     warn('I'm off to interwebz')
> >     download from the repo
> >
> > might be feasible. The download part can either be that particular
> dataset
> > or the whole scipy-datasets clone.
> >
>
> IMO:
>
> It depends on the scale where this should go.
> I don't think it's worth it (maintaining and installing another
> package or repo) for scipy
> given that scipy is mostly a basic numerical library and not driven by
> specific
> applications.
>
> For most areas there should be already some online repos or packages and
> it would be enough to have the accessing functions in scipy.datasets.
> The only area that I can think of where there might not be some readily
> available online source for datasets is signal.
>
> Josef
>
>
> >
> >
> >
> > On Fri, Mar 30, 2018 at 1:16 AM, Stefan van der Walt <
> stefanv at berkeley.edu>
> > wrote:
> >>
> >> On Thu, 29 Mar 2018 18:54:52 -0400, Warren Weckesser wrote:
> >> > Can you summarize the problems that make you regret including the
> >> > data?
> >>
> >> - The size of the repository (extra time on each clone, and that for
> >>   data that isn't necessary in most use cases)
> >>
> >> - Artificial limit on data sizes: we now have a default place to store
> >>   data, but we still need an additional mechanism for larger datasets.
> >>   How do you choose the threshold for what goes in, what is too big?
> >>
> >> - Because these tiny embedded datasets are easily available, they become
> >>   the default for demos.  If data is stored externally, realistic
> >>   examples become more feasible and likely.
> >>
> >> Best regards
> >> St?fan
> >> _______________________________________________
> >> SciPy-Dev mailing list
> >> SciPy-Dev at python.org
> >> https://mail.python.org/mailman/listinfo/scipy-dev
> >
> >
> >
> > _______________________________________________
> > SciPy-Dev mailing list
> > SciPy-Dev at python.org
> > https://mail.python.org/mailman/listinfo/scipy-dev
> >
> _______________________________________________
> SciPy-Dev mailing list
> SciPy-Dev at python.org
> https://mail.python.org/mailman/listinfo/scipy-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scipy-dev/attachments/20180330/37111d1c/attachment.html>

From josef.pktd at gmail.com  Thu Mar 29 20:52:34 2018
From: josef.pktd at gmail.com (josef.pktd at gmail.com)
Date: Thu, 29 Mar 2018 20:52:34 -0400
Subject: [SciPy-Dev] New subpackage: scipy.data
In-Reply-To: <CAEBuzr-qtBryrHZ=sCW1JN0FMZqByBucK4G7f7FJbmk3F-=86Q@mail.gmail.com>
References: <CAGzF1udgrXmWyXh93fpN9TOVvbmvSEt1UHXpdw6QtNJYBrSi0A@mail.gmail.com>
 <20180329224535.nyn575wvqfcpg7qu@carbo>
 <CAGzF1uevvstriBxtRudTYFkNxDh_4sEYh627++WP08it-WR8ag@mail.gmail.com>
 <20180329231617.cdyxvf6ysip4fth3@carbo>
 <CAEBuzr-qY0V4+C-A80c=5O0+wZwjE-CdJ4VXx02844JMEfA6hQ@mail.gmail.com>
 <CAMMTP+C7K3XxMw=PRyivYdKs7-7WBzLUam2jWr6OGpbX666d1g@mail.gmail.com>
 <CAEBuzr-qtBryrHZ=sCW1JN0FMZqByBucK4G7f7FJbmk3F-=86Q@mail.gmail.com>
Message-ID: <CAMMTP+DnMLKJhO-DipUMSWs4QE52MeEcWCFNhVjrPUhCeg88+w@mail.gmail.com>

I also think that at most small datasets should be included in scipy directly.
But I think that for online storage scipy would be better off following
some other packages.
Stefan mentions some attempts to get to a common format.
AFAIK without being up to date, both scikit-learn and scikit-image use access
to larger datasets.

For example, a dataset package also runs into the problem how much to
include. I wouldn't install a dataset package with a few gigabyte of data
if I'm only interested in a tiny fraction for the examples that are
relevant to me.
(I'm not into analyzing images, movies or BIG DATA.)

Josef


On Thu, Mar 29, 2018 at 8:10 PM, Ilhan Polat <ilhanpolat at gmail.com> wrote:
> Yes, that's true but GitHub seems like a robust place to live. Otherwise we
> can just point to any hardcoded URL. But if the size gets bigger in terms of
> wheels and cloning I think within SciPy doesn't seem to be a viable option.
> These all depend on what the future of datasets would be.
>
> On Fri, Mar 30, 2018 at 2:03 AM, <josef.pktd at gmail.com> wrote:
>>
>> On Thu, Mar 29, 2018 at 7:54 PM, Ilhan Polat <ilhanpolat at gmail.com> wrote:
>> > Would a separate repo scipy-datasets help ? Then something like
>> >
>> > try:
>> >      importing
>> > except:
>> >     warn('I'm off to interwebz')
>> >     download from the repo
>> >
>> > might be feasible. The download part can either be that particular
>> > dataset
>> > or the whole scipy-datasets clone.
>> >
>>
>> IMO:
>>
>> It depends on the scale where this should go.
>> I don't think it's worth it (maintaining and installing another
>> package or repo) for scipy
>> given that scipy is mostly a basic numerical library and not driven by
>> specific
>> applications.
>>
>> For most areas there should be already some online repos or packages and
>> it would be enough to have the accessing functions in scipy.datasets.
>> The only area that I can think of where there might not be some readily
>> available online source for datasets is signal.
>>
>> Josef
>>
>>
>> >
>> >
>> >
>> > On Fri, Mar 30, 2018 at 1:16 AM, Stefan van der Walt
>> > <stefanv at berkeley.edu>
>> > wrote:
>> >>
>> >> On Thu, 29 Mar 2018 18:54:52 -0400, Warren Weckesser wrote:
>> >> > Can you summarize the problems that make you regret including the
>> >> > data?
>> >>
>> >> - The size of the repository (extra time on each clone, and that for
>> >>   data that isn't necessary in most use cases)
>> >>
>> >> - Artificial limit on data sizes: we now have a default place to store
>> >>   data, but we still need an additional mechanism for larger datasets.
>> >>   How do you choose the threshold for what goes in, what is too big?
>> >>
>> >> - Because these tiny embedded datasets are easily available, they
>> >> become
>> >>   the default for demos.  If data is stored externally, realistic
>> >>   examples become more feasible and likely.
>> >>
>> >> Best regards
>> >> St?fan
>> >> _______________________________________________
>> >> SciPy-Dev mailing list
>> >> SciPy-Dev at python.org
>> >> https://mail.python.org/mailman/listinfo/scipy-dev
>> >
>> >
>> >
>> > _______________________________________________
>> > SciPy-Dev mailing list
>> > SciPy-Dev at python.org
>> > https://mail.python.org/mailman/listinfo/scipy-dev
>> >
>> _______________________________________________
>> SciPy-Dev mailing list
>> SciPy-Dev at python.org
>> https://mail.python.org/mailman/listinfo/scipy-dev
>
>
>
> _______________________________________________
> SciPy-Dev mailing list
> SciPy-Dev at python.org
> https://mail.python.org/mailman/listinfo/scipy-dev
>

From sievert.scott at gmail.com  Thu Mar 29 21:00:54 2018
From: sievert.scott at gmail.com (Scott Sievert)
Date: Thu, 29 Mar 2018 18:00:54 -0700
Subject: [SciPy-Dev] New subpackage: scipy.data
In-Reply-To: <CAEBuzr-qtBryrHZ=sCW1JN0FMZqByBucK4G7f7FJbmk3F-=86Q@mail.gmail.com>
References: <CAGzF1udgrXmWyXh93fpN9TOVvbmvSEt1UHXpdw6QtNJYBrSi0A@mail.gmail.com>
 <20180329224535.nyn575wvqfcpg7qu@carbo>
 <CAGzF1uevvstriBxtRudTYFkNxDh_4sEYh627++WP08it-WR8ag@mail.gmail.com>
 <20180329231617.cdyxvf6ysip4fth3@carbo>
 <CAEBuzr-qY0V4+C-A80c=5O0+wZwjE-CdJ4VXx02844JMEfA6hQ@mail.gmail.com>
 <CAMMTP+C7K3XxMw=PRyivYdKs7-7WBzLUam2jWr6OGpbX666d1g@mail.gmail.com>
 <CAEBuzr-qtBryrHZ=sCW1JN0FMZqByBucK4G7f7FJbmk3F-=86Q@mail.gmail.com>
Message-ID: <CAPZ=bb3LH2H7JkO8ZxQThO+2CNDUG_SsXeouzgjayfaFwhoUVw@mail.gmail.com>

Including some datasets would also help the scipy benchmarks be a more
realistic. Right now the benchmarks use synthetic data (at least the signal
benchmarks do).

Scott

On March 29, 2018 at 7:17:28 PM, Ilhan Polat (ilhanpolat at gmail.com) wrote:

Yes, that's true but GitHub seems like a robust place to live. Otherwise we
can just point to any hardcoded URL. But if the size gets bigger in terms
of wheels and cloning I think within SciPy doesn't seem to be a viable
option. These all depend on what the future of datasets would be.

On Fri, Mar 30, 2018 at 2:03 AM, <josef.pktd at gmail.com> wrote:

> On Thu, Mar 29, 2018 at 7:54 PM, Ilhan Polat <ilhanpolat at gmail.com> wrote:
> > Would a separate repo scipy-datasets help ? Then something like
> >
> > try:
> >      importing
> > except:
> >     warn('I'm off to interwebz')
> >     download from the repo
> >
> > might be feasible. The download part can either be that particular
> dataset
> > or the whole scipy-datasets clone.
> >
>
> IMO:
>
> It depends on the scale where this should go.
> I don't think it's worth it (maintaining and installing another
> package or repo) for scipy
> given that scipy is mostly a basic numerical library and not driven by
> specific
> applications.
>
> For most areas there should be already some online repos or packages and
> it would be enough to have the accessing functions in scipy.datasets.
> The only area that I can think of where there might not be some readily
> available online source for datasets is signal.
>
> Josef
>
>
> >
> >
> >
> > On Fri, Mar 30, 2018 at 1:16 AM, Stefan van der Walt <
> stefanv at berkeley.edu>
> > wrote:
> >>
> >> On Thu, 29 Mar 2018 18:54:52 -0400, Warren Weckesser wrote:
> >> > Can you summarize the problems that make you regret including the
> >> > data?
> >>
> >> - The size of the repository (extra time on each clone, and that for
> >>   data that isn't necessary in most use cases)
> >>
> >> - Artificial limit on data sizes: we now have a default place to store
> >>   data, but we still need an additional mechanism for larger datasets.
> >>   How do you choose the threshold for what goes in, what is too big?
> >>
> >> - Because these tiny embedded datasets are easily available, they become
> >>   the default for demos.  If data is stored externally, realistic
> >>   examples become more feasible and likely.
> >>
> >> Best regards
> >> St?fan
> >> _______________________________________________
> >> SciPy-Dev mailing list
> >> SciPy-Dev at python.org
> >> https://mail.python.org/mailman/listinfo/scipy-dev
> >
> >
> >
> > _______________________________________________
> > SciPy-Dev mailing list
> > SciPy-Dev at python.org
> > https://mail.python.org/mailman/listinfo/scipy-dev
> >
> _______________________________________________
> SciPy-Dev mailing list
> SciPy-Dev at python.org
> https://mail.python.org/mailman/listinfo/scipy-dev
>

_______________________________________________
SciPy-Dev mailing list
SciPy-Dev at python.org
https://mail.python.org/mailman/listinfo/scipy-dev
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scipy-dev/attachments/20180329/81811864/attachment-0001.html>

From larson.eric.d at gmail.com  Fri Mar 30 09:54:29 2018
From: larson.eric.d at gmail.com (Eric Larson)
Date: Fri, 30 Mar 2018 09:54:29 -0400
Subject: [SciPy-Dev] New subpackage: scipy.data
In-Reply-To: <CAPZ=bb3LH2H7JkO8ZxQThO+2CNDUG_SsXeouzgjayfaFwhoUVw@mail.gmail.com>
References: <CAGzF1udgrXmWyXh93fpN9TOVvbmvSEt1UHXpdw6QtNJYBrSi0A@mail.gmail.com>
 <20180329224535.nyn575wvqfcpg7qu@carbo>
 <CAGzF1uevvstriBxtRudTYFkNxDh_4sEYh627++WP08it-WR8ag@mail.gmail.com>
 <20180329231617.cdyxvf6ysip4fth3@carbo>
 <CAEBuzr-qY0V4+C-A80c=5O0+wZwjE-CdJ4VXx02844JMEfA6hQ@mail.gmail.com>
 <CAMMTP+C7K3XxMw=PRyivYdKs7-7WBzLUam2jWr6OGpbX666d1g@mail.gmail.com>
 <CAEBuzr-qtBryrHZ=sCW1JN0FMZqByBucK4G7f7FJbmk3F-=86Q@mail.gmail.com>
 <CAPZ=bb3LH2H7JkO8ZxQThO+2CNDUG_SsXeouzgjayfaFwhoUVw@mail.gmail.com>
Message-ID: <CAGu2niUg=jtBJuLHgGKN0tVR0k2cueXGcAzj+t7HKfQ0m2QDAA@mail.gmail.com>

>
> It depends on the scale where this should go.
>>
> In this particular case ("scipy.signal currently has no useful realistic
signals"), if we add the proposed ~100 kB data file, I suspect that we can
greatly enhance a large number of our scipy.signal examples. An ECG signal
won't be perfect for all of them, but in many cases it will be a lot better
and more instructive for users than what we can currently synthesize
ourselves (while keeping synthesis sufficiently simple at least).

Compared to a general dataset-fetching utility, the in-repo approach has
clear disadvantages in terms of being incomplete and adding to repo size.
Its advantages are in terms of simplifying doc building, access,
maintenance, uniformity of functionality (benchmarks, Debian unit tests,
doc building, etc.). On the balance, this makes it worth having IMO.

For example, a dataset package also runs into the problem how much to
> include.


A proposed rule of thumb: SciPy can have (up to) a couple of small-sized
files per module shipped with the repo in cases where such files greatly
improve our ability to showcase/test/document functionality
(benchmarks/unit tests/docstrings). This forces us to make subjective
judgments about what will be sufficiently useful, sufficiently small, and
sufficiently impactful for the module, but I think this will be a rare
enough phenomenon that it's okay.

In other words, I propose that scipy.datasets not provide an *exhaustive* or
even *extensive *resource of data for users, but rather a *minimal* one for
showcasing functionality. This seems consistent with what we already do
with ascent/face, in that they improve the image-processing examples.

We've been doing this in scikit-image for a long time, and now regret
> having any binary data in the repository


I have had a similar problem while maintaining MNE-Python, which has some
files in the repo and others in a GitHub repo (downloaded separately for
testing). I have a similar feeling about the files that live in the repo
today. However, for SciPy the problem seems a bit different in scope and
scale -- a handful of small files can go a long way for SciPy, which isn't
the case for MNE (and I would assume also many functions in scikit-image).

both scikit-learn and scikit-image use access to larger datasets.


There are other projects that also do this (MNE has huge ones hosted on
osf.io, VisPy hosts data on GitHub). It would be awesome if someone unified
all this stuff for cases where you want to deal with getting large
datasets, or many different datasets.

My 2c,
Eric
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scipy-dev/attachments/20180330/d3adf71e/attachment.html>

From josef.pktd at gmail.com  Fri Mar 30 10:29:48 2018
From: josef.pktd at gmail.com (josef.pktd at gmail.com)
Date: Fri, 30 Mar 2018 10:29:48 -0400
Subject: [SciPy-Dev] New subpackage: scipy.data
In-Reply-To: <CAGu2niUg=jtBJuLHgGKN0tVR0k2cueXGcAzj+t7HKfQ0m2QDAA@mail.gmail.com>
References: <CAGzF1udgrXmWyXh93fpN9TOVvbmvSEt1UHXpdw6QtNJYBrSi0A@mail.gmail.com>
 <20180329224535.nyn575wvqfcpg7qu@carbo>
 <CAGzF1uevvstriBxtRudTYFkNxDh_4sEYh627++WP08it-WR8ag@mail.gmail.com>
 <20180329231617.cdyxvf6ysip4fth3@carbo>
 <CAEBuzr-qY0V4+C-A80c=5O0+wZwjE-CdJ4VXx02844JMEfA6hQ@mail.gmail.com>
 <CAMMTP+C7K3XxMw=PRyivYdKs7-7WBzLUam2jWr6OGpbX666d1g@mail.gmail.com>
 <CAEBuzr-qtBryrHZ=sCW1JN0FMZqByBucK4G7f7FJbmk3F-=86Q@mail.gmail.com>
 <CAPZ=bb3LH2H7JkO8ZxQThO+2CNDUG_SsXeouzgjayfaFwhoUVw@mail.gmail.com>
 <CAGu2niUg=jtBJuLHgGKN0tVR0k2cueXGcAzj+t7HKfQ0m2QDAA@mail.gmail.com>
Message-ID: <CAMMTP+C2qUip209yv=Chj106tgtkQe42ewBo13PaPgdki5U76w@mail.gmail.com>

On Fri, Mar 30, 2018 at 9:54 AM, Eric Larson <larson.eric.d at gmail.com> wrote:
>>> It depends on the scale where this should go.
>
> In this particular case ("scipy.signal currently has no useful realistic
> signals"), if we add the proposed ~100 kB data file, I suspect that we can
> greatly enhance a large number of our scipy.signal examples. An ECG signal
> won't be perfect for all of them, but in many cases it will be a lot better
> and more instructive for users than what we can currently synthesize
> ourselves (while keeping synthesis sufficiently simple at least).
>
> Compared to a general dataset-fetching utility, the in-repo approach has
> clear disadvantages in terms of being incomplete and adding to repo size.
> Its advantages are in terms of simplifying doc building, access,
> maintenance, uniformity of functionality (benchmarks, Debian unit tests, doc
> building, etc.). On the balance, this makes it worth having IMO.
>
>> For example, a dataset package also runs into the problem how much to
>> include.
>
>
> A proposed rule of thumb: SciPy can have (up to) a couple of small-sized
> files per module shipped with the repo in cases where such files greatly
> improve our ability to showcase/test/document functionality (benchmarks/unit
> tests/docstrings). This forces us to make subjective judgments about what
> will be sufficiently useful, sufficiently small, and sufficiently impactful
> for the module, but I think this will be a rare enough phenomenon that it's
> okay.
>
> In other words, I propose that scipy.datasets not provide an exhaustive or
> even extensive resource of data for users, but rather a minimal one for
> showcasing functionality. This seems consistent with what we already do with
> ascent/face, in that they improve the image-processing examples.
>
>> We've been doing this in scikit-image for a long time, and now regret
>> having any binary data in the repository
>
>
> I have had a similar problem while maintaining MNE-Python, which has some
> files in the repo and others in a GitHub repo (downloaded separately for
> testing). I have a similar feeling about the files that live in the repo
> today. However, for SciPy the problem seems a bit different in scope and
> scale -- a handful of small files can go a long way for SciPy, which isn't
> the case for MNE (and I would assume also many functions in scikit-image).
>
>> both scikit-learn and scikit-image use access to larger datasets.
>
>
> There are other projects that also do this (MNE has huge ones hosted on
> osf.io, VisPy hosts data on GitHub). It would be awesome if someone unified
> all this stuff for cases where you want to deal with getting large datasets,
> or many different datasets.


just to say:
I agree with all of this,and think it is a very good summary of the issues

Josef

>
> My 2c,
> Eric
>
>
> _______________________________________________
> SciPy-Dev mailing list
> SciPy-Dev at python.org
> https://mail.python.org/mailman/listinfo/scipy-dev
>

From lagru at mailbox.org  Fri Mar 30 11:50:15 2018
From: lagru at mailbox.org (Lars G.)
Date: Fri, 30 Mar 2018 17:50:15 +0200
Subject: [SciPy-Dev] Measuring test coverage of a Cython module
In-Reply-To: <CABL7CQhv=eu=rA9WM0CFa5L760qMkHUmuJvPziT+FkhCmUZe_w@mail.gmail.com>
References: <28dc611e-7598-e88b-2534-e2a5252b9ca0@mailbox.org>
 <CABL7CQhv=eu=rA9WM0CFa5L760qMkHUmuJvPziT+FkhCmUZe_w@mail.gmail.com>
Message-ID: <b8baeb3a-d2b8-4985-1c97-db6ca9f07da1@mailbox.org>

On 28.03.2018 06:06, Ralf Gommers wrote:
> On Mon, Mar 26, 2018 at 11:11 AM, Lars G. <lagru at mailbox.org
> <mailto:lagru at mailbox.org>> wrote:
> 
>     Hi,
>     I'm trying to measure the test coverage of a Cython module. However the
>     module in question (`scipy/signal/peak_finding_utils.pyx`) doesn't show
>     up in the final report.
> 
>     What I have tried so far:
> 
>     1. Added `plugins = Cython.coverage` to the `.coveragerc` file.
> 
>     2. Added
>     # cython: linetrace=True
>     # distutils: define_macros=CYTHON_TRACE_NOGIL=1
>     to the header of `peak_finding_utils.pyx`
> 
>     3. Build binaries inplace with
>     $ python setup.py build_ext --inplace
> 
>     4. Measured test coverage
>     a) with `runtests.py`
>     $ python runtests.py -t scipy/signal/tests/test_peak_finding --no-build
>     --coverage
> 
>     b) and using the `coverage` package
>     $ coverage run -m pytest scipy/signal/tests/test_peak_finding.py
>     $ coverage html
> 
> 
> The one step missing here seems to be editing .coveragerc:
> http://cython.readthedocs.io/en/latest/src/tutorial/profiling_tutorial.html?highlight=coverage#enabling-coverage-analysis

Actually I did that with step 1.

> The only discussion I remember is one on the scikit-image list started
> by Matthew Brett to get this working; not sure if that came to anything.
> In general it looks like it should work, but it isn't always reliable
> (e.g. https://github.com/cython/cython/issues/1985)

Thanks for the link and your time. I'll look into it and report back if
I figure it out.

Best regards, Lars


From pav at iki.fi  Fri Mar 30 12:06:18 2018
From: pav at iki.fi (Pauli Virtanen)
Date: Fri, 30 Mar 2018 18:06:18 +0200
Subject: [SciPy-Dev] New subpackage: scipy.data
In-Reply-To: <CAGzF1udgrXmWyXh93fpN9TOVvbmvSEt1UHXpdw6QtNJYBrSi0A@mail.gmail.com>
References: <CAGzF1udgrXmWyXh93fpN9TOVvbmvSEt1UHXpdw6QtNJYBrSi0A@mail.gmail.com>
Message-ID: <1522425978.9176.106.camel@iki.fi>

Hi,

to, 2018-03-29 kello 15:43 -0400, Warren Weckesser kirjoitti:
> According to the SciPy roadmap (
> https://github.com/scipy/scipy/blob/master/doc/ROADMAP.rst.txt#misc),
> `scipy.misc` will eventually be removed. Currently, the combinatorial
> functions and the image-related operations are all deprecated.  The
> only
> non-deprecated functions in `misc` are `central_diff_weights()`,
> `derivative()` and the two functions that return image data:
> `ascent()` and
> `face()`.
> 
> As a steps towards the deprecation of `misc`, I propose that we
> create a
> new package, `scipy.data`, for holding data sets.  `ascent()` and
> `face()`
> would move there, and the new ECG data set proposed in a current pull
> request (https://github.com/scipy/scipy/pull/8627) would be put
> there.

At first sight I think that these two functions (and the third one with
ECG signal sample) alone would sound more suitable to be placed in
`scipy.ndimage` and `scipy.signal`. Top-level module for them alone
sounds overkill, and I'm not sure if discoverability alone is enough.

>From the rest of the thread, it appears that there is not very clear
picture of what else we would like to put in there.

For the downloadable datasets idea, I would not recommend doing
anything that requires maintenance. Sorting out hosting issues is
boring, and when done on a volunteer basis it always tends to fall on
the same chumps. It's not really in the core mission of the project.

Note also the (probably mostly forgotten) numpy.DataSource 
https://docs.scipy.org/doc/numpy/reference/generated/numpy.DataSource.html

	Pauli


From larson.eric.d at gmail.com  Fri Mar 30 15:03:43 2018
From: larson.eric.d at gmail.com (Eric Larson)
Date: Fri, 30 Mar 2018 15:03:43 -0400
Subject: [SciPy-Dev] New subpackage: scipy.data
In-Reply-To: <1522425978.9176.106.camel@iki.fi>
References: <CAGzF1udgrXmWyXh93fpN9TOVvbmvSEt1UHXpdw6QtNJYBrSi0A@mail.gmail.com>
 <1522425978.9176.106.camel@iki.fi>
Message-ID: <CAGu2niUt3Zz+xBPrXjF2BgYJKkcBJwNeA-r8D8u_L--=izo7xw@mail.gmail.com>

>
> Top-level module for them alone sounds overkill, and I'm not sure if
> discoverability alone is enough.
>

Fine by me. And if we follow the idea that these should be added sparingly,
we can maintain discoverability without it growing out of hand by
populating the See Also sections of each function.

Eric
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scipy-dev/attachments/20180330/193bca2f/attachment.html>

From lagru at mailbox.org  Fri Mar 30 15:04:18 2018
From: lagru at mailbox.org (Lars G.)
Date: Fri, 30 Mar 2018 21:04:18 +0200
Subject: [SciPy-Dev] New subpackage: scipy.data
In-Reply-To: <1522425978.9176.106.camel@iki.fi>
References: <CAGzF1udgrXmWyXh93fpN9TOVvbmvSEt1UHXpdw6QtNJYBrSi0A@mail.gmail.com>
 <1522425978.9176.106.camel@iki.fi>
Message-ID: <1c850aed-3c22-cb6f-9496-4ea26d8169ab@mailbox.org>

On 30.03.2018 18:06, Pauli Virtanen wrote:
> At first sight I think that these two functions (and the third one with
> ECG signal sample) alone would sound more suitable to be placed in
> `scipy.ndimage` and `scipy.signal`. Top-level module for them alone
> sounds overkill, and I'm not sure if discoverability alone is enough.

At the risk of stating the obvious, wouldn't the discoverability of
those functions be pretty high considering these datasets are or could
be used in many documentation examples and tutorials?

Lars

From phillip.m.feldman at gmail.com  Fri Mar 30 15:20:16 2018
From: phillip.m.feldman at gmail.com (Phillip Feldman)
Date: Fri, 30 Mar 2018 12:20:16 -0700
Subject: [SciPy-Dev] New subpackage: scipy.data
In-Reply-To: <CAGzF1udgrXmWyXh93fpN9TOVvbmvSEt1UHXpdw6QtNJYBrSi0A@mail.gmail.com>
References: <CAGzF1udgrXmWyXh93fpN9TOVvbmvSEt1UHXpdw6QtNJYBrSi0A@mail.gmail.com>
Message-ID: <CAB2ViTZ2fTaDuY7GXfaDDsYK7P1YbJTGeYR23oo9K4k_6OkEWQ@mail.gmail.com>

Is there a substitute for the deprecated combinatorics functions?

FYI, a few years ago, I wrote a Python combinatorics package to fill in
some of the gaps in `itertools`.  My package can be found here:

https://pypi.python.org/pypi/Combinatorics/1.4.5

Phillip

On Thu, Mar 29, 2018 at 12:43 PM, Warren Weckesser <
warren.weckesser at gmail.com> wrote:

> According to the SciPy roadmap (https://github.com/scipy/
> scipy/blob/master/doc/ROADMAP.rst.txt#misc),
> `scipy.misc` will eventually be removed. Currently, the combinatorial
> functions and the image-related operations are all deprecated.  The only
> non-deprecated functions in `misc` are `central_diff_weights()`,
> `derivative()` and the two functions that return image data: `ascent()` and
> `face()`.
>
> As a steps towards the deprecation of `misc`, I propose that we create a
> new package, `scipy.data`, for holding data sets.  `ascent()` and `face()`
> would move there, and the new ECG data set proposed in a current pull
> request (https://github.com/scipy/scipy/pull/8627) would be put there.
>
> An early version of the roadmap suggested moving the images to
> `scipy.ndimage`, but that is no longer in the text.  I think a separate
> subpackage for data sets makes sense.
>
> What do you think?
>
> P.S. If there is already a similar proposal in the mailing list or on
> github, or any other older mailing list discussions related to this, let me
> know.
>
>
>
> _______________________________________________
> SciPy-Dev mailing list
> SciPy-Dev at python.org
> https://mail.python.org/mailman/listinfo/scipy-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scipy-dev/attachments/20180330/7cdc6cba/attachment.html>

From pav at iki.fi  Fri Mar 30 15:23:14 2018
From: pav at iki.fi (Pauli Virtanen)
Date: Fri, 30 Mar 2018 21:23:14 +0200
Subject: [SciPy-Dev] New subpackage: scipy.data
In-Reply-To: <CAB2ViTZ2fTaDuY7GXfaDDsYK7P1YbJTGeYR23oo9K4k_6OkEWQ@mail.gmail.com>
References: <CAGzF1udgrXmWyXh93fpN9TOVvbmvSEt1UHXpdw6QtNJYBrSi0A@mail.gmail.com>
 <CAB2ViTZ2fTaDuY7GXfaDDsYK7P1YbJTGeYR23oo9K4k_6OkEWQ@mail.gmail.com>
Message-ID: <1522437794.2700.1.camel@iki.fi>

pe, 2018-03-30 kello 12:20 -0700, Phillip Feldman kirjoitti:
> Is there a substitute for the deprecated combinatorics functions?

I think they were moved to scipy.special. The documentation iirc says
where to find it now.

	Pauli


> FYI, a few years ago, I wrote a Python combinatorics package to fill
> in
> some of the gaps in `itertools`.  My package can be found here:
> 
> https://pypi.python.org/pypi/Combinatorics/1.4.5
> 
> Phillip
> 
> On Thu, Mar 29, 2018 at 12:43 PM, Warren Weckesser <
> warren.weckesser at gmail.com> wrote:
> 
> > According to the SciPy roadmap (https://github.com/scipy/
> > scipy/blob/master/doc/ROADMAP.rst.txt#misc),
> > `scipy.misc` will eventually be removed. Currently, the
> > combinatorial
> > functions and the image-related operations are all deprecated.  The
> > only
> > non-deprecated functions in `misc` are `central_diff_weights()`,
> > `derivative()` and the two functions that return image data:
> > `ascent()` and
> > `face()`.
> > 
> > As a steps towards the deprecation of `misc`, I propose that we
> > create a
> > new package, `scipy.data`, for holding data sets.  `ascent()` and
> > `face()`
> > would move there, and the new ECG data set proposed in a current
> > pull
> > request (https://github.com/scipy/scipy/pull/8627) would be put
> > there.
> > 
> > An early version of the roadmap suggested moving the images to
> > `scipy.ndimage`, but that is no longer in the text.  I think a
> > separate
> > subpackage for data sets makes sense.
> > 
> > What do you think?
> > 
> > P.S. If there is already a similar proposal in the mailing list or
> > on
> > github, or any other older mailing list discussions related to
> > this, let me
> > know.
> > 
> > 
> > 
> > _______________________________________________
> > SciPy-Dev mailing list
> > SciPy-Dev at python.org
> > https://mail.python.org/mailman/listinfo/scipy-dev
> > 
> > 
> 
> _______________________________________________
> SciPy-Dev mailing list
> SciPy-Dev at python.org
> https://mail.python.org/mailman/listinfo/scipy-dev


From mikofski at berkeley.edu  Fri Mar 30 15:23:22 2018
From: mikofski at berkeley.edu (Mark Alexander Mikofski)
Date: Fri, 30 Mar 2018 19:23:22 +0000
Subject: [SciPy-Dev] New subpackage: scipy.data
In-Reply-To: <CAMMTP+C2qUip209yv=Chj106tgtkQe42ewBo13PaPgdki5U76w@mail.gmail.com>
References: <CAGzF1udgrXmWyXh93fpN9TOVvbmvSEt1UHXpdw6QtNJYBrSi0A@mail.gmail.com>
 <20180329224535.nyn575wvqfcpg7qu@carbo>
 <CAGzF1uevvstriBxtRudTYFkNxDh_4sEYh627++WP08it-WR8ag@mail.gmail.com>
 <20180329231617.cdyxvf6ysip4fth3@carbo>
 <CAEBuzr-qY0V4+C-A80c=5O0+wZwjE-CdJ4VXx02844JMEfA6hQ@mail.gmail.com>
 <CAMMTP+C7K3XxMw=PRyivYdKs7-7WBzLUam2jWr6OGpbX666d1g@mail.gmail.com>
 <CAEBuzr-qtBryrHZ=sCW1JN0FMZqByBucK4G7f7FJbmk3F-=86Q@mail.gmail.com>
 <CAPZ=bb3LH2H7JkO8ZxQThO+2CNDUG_SsXeouzgjayfaFwhoUVw@mail.gmail.com>
 <CAGu2niUg=jtBJuLHgGKN0tVR0k2cueXGcAzj+t7HKfQ0m2QDAA@mail.gmail.com>
 <CAMMTP+C2qUip209yv=Chj106tgtkQe42ewBo13PaPgdki5U76w@mail.gmail.com>
Message-ID: <CAEqRcW3btHi=SqZSovnL2XcMTkBbxnOSdZPwsNwvJ+k6AMVvrg@mail.gmail.com>

I agree with what's above. Basically (1) move small datasets to centralized
scipy.datasets for testing, demos, docs, and short examples, and (2) move
large, realistic datasets to shared repo or common site like rdatasets and
explain in docs how to retrieve them. These longer tutorials could be in
Jupyter notebooks, for example.

On Mar 30, 2018 7:30 AM, <josef.pktd at gmail.com> wrote:

On Fri, Mar 30, 2018 at 9:54 AM, Eric Larson <larson.eric.d at gmail.com>
wrote:
>>> It depends on the scale where this should go.
>
> In this particular case ("scipy.signal currently has no useful realistic
> signals"), if we add the proposed ~100 kB data file, I suspect that we can
> greatly enhance a large number of our scipy.signal examples. An ECG signal
> won't be perfect for all of them, but in many cases it will be a lot
better
> and more instructive for users than what we can currently synthesize
> ourselves (while keeping synthesis sufficiently simple at least).
>
> Compared to a general dataset-fetching utility, the in-repo approach has
> clear disadvantages in terms of being incomplete and adding to repo size.
> Its advantages are in terms of simplifying doc building, access,
> maintenance, uniformity of functionality (benchmarks, Debian unit tests,
doc
> building, etc.). On the balance, this makes it worth having IMO.
>
>> For example, a dataset package also runs into the problem how much to
>> include.
>
>
> A proposed rule of thumb: SciPy can have (up to) a couple of small-sized
> files per module shipped with the repo in cases where such files greatly
> improve our ability to showcase/test/document functionality
(benchmarks/unit
> tests/docstrings). This forces us to make subjective judgments about what
> will be sufficiently useful, sufficiently small, and sufficiently
impactful
> for the module, but I think this will be a rare enough phenomenon that
it's
> okay.
>
> In other words, I propose that scipy.datasets not provide an exhaustive or
> even extensive resource of data for users, but rather a minimal one for
> showcasing functionality. This seems consistent with what we already do
with
> ascent/face, in that they improve the image-processing examples.
>
>> We've been doing this in scikit-image for a long time, and now regret
>> having any binary data in the repository
>
>
> I have had a similar problem while maintaining MNE-Python, which has some
> files in the repo and others in a GitHub repo (downloaded separately for
> testing). I have a similar feeling about the files that live in the repo
> today. However, for SciPy the problem seems a bit different in scope and
> scale -- a handful of small files can go a long way for SciPy, which isn't
> the case for MNE (and I would assume also many functions in scikit-image).
>
>> both scikit-learn and scikit-image use access to larger datasets.
>
>
> There are other projects that also do this (MNE has huge ones hosted on
> osf.io, VisPy hosts data on GitHub). It would be awesome if someone
unified
> all this stuff for cases where you want to deal with getting large
datasets,
> or many different datasets.


just to say:
I agree with all of this,and think it is a very good summary of the issues

Josef


>
> My 2c,
> Eric
>
>
> _______________________________________________
> SciPy-Dev mailing list
> SciPy-Dev at python.org
> https://mail.python.org/mailman/listinfo/scipy-dev
>
_______________________________________________
SciPy-Dev mailing list
SciPy-Dev at python.org
https://mail.python.org/mailman/listinfo/scipy-dev
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scipy-dev/attachments/20180330/1459e6e5/attachment.html>

From ralf.gommers at gmail.com  Fri Mar 30 20:17:13 2018
From: ralf.gommers at gmail.com (Ralf Gommers)
Date: Fri, 30 Mar 2018 17:17:13 -0700
Subject: [SciPy-Dev] New subpackage: scipy.data
In-Reply-To: <CAGu2niUt3Zz+xBPrXjF2BgYJKkcBJwNeA-r8D8u_L--=izo7xw@mail.gmail.com>
References: <CAGzF1udgrXmWyXh93fpN9TOVvbmvSEt1UHXpdw6QtNJYBrSi0A@mail.gmail.com>
 <1522425978.9176.106.camel@iki.fi>
 <CAGu2niUt3Zz+xBPrXjF2BgYJKkcBJwNeA-r8D8u_L--=izo7xw@mail.gmail.com>
Message-ID: <CABL7CQjJs0p4p=NkLy=pEfvB8_gZY5-f=eqCKA7VF42ZfVocJw@mail.gmail.com>

On Fri, Mar 30, 2018 at 12:03 PM, Eric Larson <larson.eric.d at gmail.com>
wrote:

> Top-level module for them alone sounds overkill, and I'm not sure if
>> discoverability alone is enough.
>>
>
> Fine by me. And if we follow the idea that these should be added
> sparingly, we can maintain discoverability without it growing out of
> hand by populating the See Also sections of each function.
>

I agree with this, the 2 images and 1 ECG signal (to be added) that we have
doesn't justify a top-level module. We don't want to grow more than the
absolute minimum of datasets. The package is already very large, which is
problematic in certain cases. E.g. numpy + scipy still fits in the AWS
Lambda limit of 50 MB, but there's not much margin.

Ralf
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scipy-dev/attachments/20180330/16c50a66/attachment.html>

From ralf.gommers at gmail.com  Fri Mar 30 20:48:19 2018
From: ralf.gommers at gmail.com (Ralf Gommers)
Date: Fri, 30 Mar 2018 17:48:19 -0700
Subject: [SciPy-Dev] Attention GSoC student applicants
Message-ID: <CABL7CQjL7XUcSDm2jbP6M0SK_NxUJi5SEem0dXMTr2m4N3tQgg@mail.gmail.com>

Hi,

First of all a big thank you to all of you who submitted an application for
Google Summer of Code 2018 to SciPy! The coming week we will asses all
proposals, and we will interview all students who submitted a complete and
good quality proposal.

One thing I'd like to point out now: the Python Software Foundation
requires you to have submitted a pull request to SciPy. It does not have to
be merged, but if you have not submitted at least one pull request yet,
then I'd suggest to work on one as soon as possible!

Cheers,
Ralf
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scipy-dev/attachments/20180330/9df6ccf4/attachment.html>

From pav at iki.fi  Sat Mar 31 11:13:19 2018
From: pav at iki.fi (Pauli Virtanen)
Date: Sat, 31 Mar 2018 17:13:19 +0200
Subject: [SciPy-Dev] Measuring test coverage of a Cython module
In-Reply-To: <b8baeb3a-d2b8-4985-1c97-db6ca9f07da1@mailbox.org>
References: <28dc611e-7598-e88b-2534-e2a5252b9ca0@mailbox.org>
 <CABL7CQhv=eu=rA9WM0CFa5L760qMkHUmuJvPziT+FkhCmUZe_w@mail.gmail.com>
 <b8baeb3a-d2b8-4985-1c97-db6ca9f07da1@mailbox.org>
Message-ID: <1522509199.2449.1.camel@iki.fi>

Hi,

pe, 2018-03-30 kello 17:50 +0200, Lars G. kirjoitti:
> The only discussion I remember is one on the scikit-image list
> > started
> > by Matthew Brett to get this working; not sure if that came to
> > anything.
> > In general it looks like it should work, but it isn't always
> > reliable
> > (e.g. https://github.com/cython/cython/issues/1985)
> 
> Thanks for the link and your time. I'll look into it and report back
> if I figure it out.

I tried to do this for https://github.com/scipy/scipy/pull/8379 but
without success (although don't remember any more what the issue was),
so if you manage to figure it out that would be useful.

	Pauli