From andyfaff at gmail.com  Wed Apr  1 19:11:09 2015
From: andyfaff at gmail.com (Andrew Nelson)
Date: Thu, 2 Apr 2015 10:11:09 +1100
Subject: [SciPy-Dev] Github code for Differential Evolution reevaluations
In-Reply-To: <CAH7vTe0Bju-HrCuucvetWUE6R4pUB3=+NHmvcfV0RJRWx=F+xg@mail.gmail.com>
References: <CAH7vTe0Bju-HrCuucvetWUE6R4pUB3=+NHmvcfV0RJRWx=F+xg@mail.gmail.com>
Message-ID: <CAAbtOZd7fewyvXUqUK0zoNimBvsQNsgibbkzty5riXW6av9F6g@mail.gmail.com>

On 1 April 2015 at 03:29, James Phillips <zunzun at zunzun.com> wrote:

> I have just reviewed the Differential Evolution code in github.  It is is
> particularly well coded and most algorithmically thorough, my professional
> compliments.
>

Thank you, the community (me at least) appreciates that.


> I noticed in the older DE code I use there is a possible performance
> improvement, and this seems to be the case in the github code as well.  Not
> every population vector changes within a generation, yet my older code
> reevaluates the unchanged vectors.  Of course this does not change the
> final results, but is a point of potential savings in total computation
> time.
>

I'm not convinced that re-evaluating unchanged members of the population
(within a generation) improves performance.  There are two performance
factors to be considered here:
1) Does the change result in a lower energy at the end?
2) Does the change lead to faster termination/convergence?
By re-evaluating members that haven't changed until they do change I think
one is reducing genetic diversity.  That unchanged vector might in fact
create a better solution in the next generation if it were allowed to
remain.  So I don't think it produces a lower energy at the end.
However, sometimes 'outliers' do prevent the solving process from
terminating.  The scipy implentation stops when the variance of the
population energies falls below a certain level.  If there is one outlier
that persists this can cause the solving process to continue for a little
while longer, but not too much.

In the end the proof is in the pudding.  There is a PR in progress at
https://github.com/scipy/scipy/pull/4191.  This PR adds many multivariate
functions to benchmark global optimizers.  If your suggestions improve
performance, as measured against those benchmarks, then perhaps we could
consider it as an option.

A.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scipy-dev/attachments/20150402/64b45238/attachment.html>

From zunzun at zunzun.com  Thu Apr  2 04:34:51 2015
From: zunzun at zunzun.com (James Phillips)
Date: Thu, 2 Apr 2015 03:34:51 -0500
Subject: [SciPy-Dev] Github code for Differential Evolution reevaluations
In-Reply-To: <CAAbtOZd7fewyvXUqUK0zoNimBvsQNsgibbkzty5riXW6av9F6g@mail.gmail.com>
References: <CAH7vTe0Bju-HrCuucvetWUE6R4pUB3=+NHmvcfV0RJRWx=F+xg@mail.gmail.com>
	<CAAbtOZd7fewyvXUqUK0zoNimBvsQNsgibbkzty5riXW6av9F6g@mail.gmail.com>
Message-ID: <CAH7vTe1ji87mK5pY73VSsXDJ2A7SuZuyKRH5wLJR1BndH9_oww@mail.gmail.com>

I meant that if a population member's parameters have not changed since the
last time they were calculated, recalculating the function value for that
member will give the same result as before.  In such a case, recalculation
of the function value is unnecessary.

1) Accounting for this will clutter up the code a bit.  Unless there is
good reason, this should be avoided.

2) If [mutation and crossover] rates are high - and they usually are -
performance gains would likely be minimal as most parameter vectors in each
new generation would change, requiring function calculation.

If the total computation time is not *significantly* reduced, then by
definition any time savings would be insignificant.  It might not have any
real-world impact, I do not know.  It certainly cannot cut total time in
half, but I'm guessing a few percent would be possible.

James

On Wed, Apr 1, 2015 at 6:11 PM, Andrew Nelson <andyfaff at gmail.com> wrote:

>
> I'm not convinced that re-evaluating unchanged members of the population
> (within a generation) improves performance.
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scipy-dev/attachments/20150402/eb61dde1/attachment.html>

From cgodshall at enthought.com  Thu Apr  2 20:01:48 2015
From: cgodshall at enthought.com (Courtenay Godshall (Enthought))
Date: Thu, 2 Apr 2015 19:01:48 -0500
Subject: [SciPy-Dev] SciPy 2015 Conference Updates - LAST CALL FOR TALKS -
	extended to 4/10, registration open, keynotes announced,
	John Hunter Plotting Contest
Message-ID: <053b01d06da1$62b69700$2823c500$@enthought.com>

----------------------------------------------------------------------------
-----------------------------------------------------------

**LAST CALL FOR SCIPY 2015 TALK AND POSTER SUBMISSIONS - EXTENSION TO 4/10*

----------------------------------------------------------------------------
-----------------------------------------------------------

SciPy 2015 will include 3 major topic tracks and 7 mini-symposia tracks.
Submit a proposal on the SciPy 2015 website: <http://scipy2015.scipy.org>
http://scipy2015.scipy.org. If you have any questions or comments, feel free
to contact us at:  <mailto:scipy-organizers at scipy.org>
scipy-organizers at scipy.org <http://scipy2015.scipy.org> . You can also
follow @scipyconf on Twitter or sign up for the mailing list on the website
for the latest updates!

 

Major topic tracks include:

- Scientific Computing in Python (General track)

- Python in Data Science

- Quantitative Finance and Computational Social Sciences

 

Mini-symposia will include the applications of Python in:

- Astronomy and astrophysics

- Computational life and medical sciences

- Engineering

- Geographic information systems (GIS)

- Geophysics

- Oceanography and meteorology

- Visualization, vision and imaging

 

--------------------------------------------------------------------------

**SCIPY 2015 REGISTRATION IS OPEN**

Please register ASAP to help us get a good headcount and open the conference
to as many people as we can. PLUS, everyone who registers before May 15 will
not only get early bird discounts, but will also be entered in a drawing for
a free registration (via refund or extra)! Register on the website at
http://scipy2015.scipy.org 

 

--------------------------------------------------------------------------

**SCIPY 2015 KEYNOTE SPEAKERS ANNOUNCED**

Keynote speakers were just announced and include Wes McKinney, author of
Pandas; Chris Wiggins, Chief Data Scientist for The New York Times; and Jake
VanderPlas, director of research at the University of Washington's eScience
Institute and core contributor to a number of scientific Python libraries
including sci-kit learn and AstroML.

 

--------------------------------------------------------------------------

**ENTER THE SCIPY JOHN HUNTER EXCELLENCE IN PLOTTING CONTEST - DUE 4/13**

In memory of John Hunter, creator of matplotlib, we are pleased to announce
the Third Annual SciPy John Hunter Excellence in Plotting Competition. This
open competition aims to highlight the importance of quality plotting to
scientific progress and showcase the capabilities of the current generation
of plotting software. Participants are invited to submit scientific plots to
be judged by a panel. The winning entries will be announced and displayed at
the conference. John Hunter's family is graciously sponsoring cash prizes up
to $1,000 for the winners. We look forward to exciting submissions that push
the boundaries of plotting! See details here:
http://scipy2015.scipy.org/ehome/115969/276538/ Entries must be submitted by
April 13, 2015 via e-mail to plotting-contest at scipy.org 

 

--------------------------------------------------------------------------

**CALENDAR AND IMPORTANT DATES**

--Sprint, Birds of a Feather, Financial Aid and Talk submissions are open
NOW

--Apr 10, 2015: Talk and Poster submission deadline

--Apr 13, 2015: Plotting contest submissions due

--Apr 15, 2015: Financial aid application deadline

--Apr 17, 2015: Tutorial schedule announced

--May 1, 2015: General conference speakers & schedule announced

--May 15, 2015 (or 150 registrants): Early-bird registration ends

--Jun 1, 2015: BoF submission deadline

--Jul 6-7, 2015: SciPy 2015 Tutorials

--Jul 8-10, 2015: SciPy 2015 General Conference

--Jul 11-12, 2015: SciPy 2015 Sprints

 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scipy-dev/attachments/20150402/ddbdbd7b/attachment.html>

From zunzun at zunzun.com  Fri Apr  3 09:51:26 2015
From: zunzun at zunzun.com (James Phillips)
Date: Fri, 3 Apr 2015 08:51:26 -0500
Subject: [SciPy-Dev] Github code for Differential Evolution reevaluations
In-Reply-To: <CAH7vTe1ji87mK5pY73VSsXDJ2A7SuZuyKRH5wLJR1BndH9_oww@mail.gmail.com>
References: <CAH7vTe0Bju-HrCuucvetWUE6R4pUB3=+NHmvcfV0RJRWx=F+xg@mail.gmail.com>
	<CAAbtOZd7fewyvXUqUK0zoNimBvsQNsgibbkzty5riXW6av9F6g@mail.gmail.com>
	<CAH7vTe1ji87mK5pY73VSsXDJ2A7SuZuyKRH5wLJR1BndH9_oww@mail.gmail.com>
Message-ID: <CAH7vTe2zyQ3_-uFXoxbH831iOdy-bfrApL_KCCg62-13+QQmLg@mail.gmail.com>

I have made some preliminary tests.  In my tests I used
numpy.array_equal(old, new) to determine if a population element needed to
be recalculated of not.

Most of the time the population element changed, so that in addition to the
required function recalculation there was the additional computational
overhead of array_equal().  While I may have had some small performance
increase due to the slightly reduced number of function evaluations, this
was less than the increased computational overhead of calling array_equal()
for every population element in every generation.

James


On Thu, Apr 2, 2015 at 3:34 AM, James Phillips <zunzun at zunzun.com> wrote:

> I meant that if a population member's parameters have not changed since
> the last time they were calculated, recalculating the function value for
> that member will give the same result as before.  In such a case,
> recalculation of the function value is unnecessary.
>
> 1) Accounting for this will clutter up the code a bit.  Unless there is
> good reason, this should be avoided.
>
> 2) If [mutation and crossover] rates are high - and they usually are -
> performance gains would likely be minimal as most parameter vectors in each
> new generation would change, requiring function calculation.
>
> If the total computation time is not *significantly* reduced, then by
> definition any time savings would be insignificant.  It might not have any
> real-world impact, I do not know.  It certainly cannot cut total time in
> half, but I'm guessing a few percent would be possible.
>
> James
>
> On Wed, Apr 1, 2015 at 6:11 PM, Andrew Nelson <andyfaff at gmail.com> wrote:
>
>>
>> I'm not convinced that re-evaluating unchanged members of the population
>> (within a generation) improves performance.
>>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scipy-dev/attachments/20150403/051807f1/attachment.html>

From ralf.gommers at gmail.com  Sat Apr  4 04:11:05 2015
From: ralf.gommers at gmail.com (Ralf Gommers)
Date: Sat, 4 Apr 2015 10:11:05 +0200
Subject: [SciPy-Dev] GSoC'15 Idea: Approximation with Parametric Splines
In-Reply-To: <CAJ_+_pXjDBKf570Y7S5BhAm2=z5NFY-cZ_YJx9ejTV8vjJ7oPw@mail.gmail.com>
References: <CAJ_+_pU6Qp8sBdOXYFk3m1yz6de9SiX1sp4R6ZpJvJ6=51bZMg@mail.gmail.com>
	<CABL7CQji01rB0FDVd4XLZQ9mwoCpx8-5TTt9MwOjttPnZ-nidQ@mail.gmail.com>
	<CAJ_+_pUz3PgkLA2iUfeG1cJ=NpYzFv4dwaFHZBFRb=Q=65m_YA@mail.gmail.com>
	<CAMRo0isfQtmA+q6j3CGaAxMTW8xbuWOUh=7k8aazVhn71bV8NQ@mail.gmail.com>
	<CAJ_+_pX4Wk5TLJvfLbMZqW6PJASUphGmRCZBL1oFXEqVk0NNSQ@mail.gmail.com>
	<CAJ_+_pUr7vNRSD0qAzuzfsYuVVF8=s8gseWudQGtsCN0vnmimA@mail.gmail.com>
	<55130A5E.6070108@hilboll.de>
	<CAJ_+_pXjDBKf570Y7S5BhAm2=z5NFY-cZ_YJx9ejTV8vjJ7oPw@mail.gmail.com>
Message-ID: <CABL7CQgOJzvC=1FrbQPRAqCeYX8LYm9x_WxXd2ifBN=5=52DAQ@mail.gmail.com>

On Sat, Mar 28, 2015 at 12:23 PM, Anastasiia Tsyplia <
anastasiyatsyplya at gmail.com> wrote:

> Hi all!
>
>
>  Thanks everyone for support during the registration period! It has been
> a brainstorming couple of weeks but I hope this is only the beginning :)
>
> I submitted two proposals to GSoC'15: ?FITPACK alternative... ? and
> ?Tensor product splines...?. Recently I was asked to edit their titles so
> they start with SciPy. But it seems I'm no longer able to make any changes.
> Melange says that I can send a comment to the organization so they allow me
> to make changes. If this tuning is crucial, please, let me do it.
>

Hi Anastasiia, it's not critical to us (Scipy mentors) but apparently
important for some of the admins. You should be able to change the title on
your proposals now. Please let me know once you've done that, then I'll
disable editing again.

Cheers,
Ralf


>  Andreas, thanks for the comment. I'll take into account your suggestions.
>
>
>  Best regards,
>
> Anastasiia
>
> 2015-03-25 21:19 GMT+02:00 Andreas Hilboll <lists at hilboll.de>:
>
>> Hi Anastasiia, Eveni, ...
>>
>> I'd very much appreciate if tensor product splines would make it into
>> scipy =)  One wish I have would be to enable periodicity, possibly via a
>> periodic kwarg, where the use would be able to provide, for any
>> dimensions, an interval in which the coordinates are considered
>> periodic.  This is probably too specific to make it into the
>> implementation, but I'd like to raise the issue early on ...
>>
>> Cheers,
>>   Andreas.
>>
>>
>> On 21.03.2015 22:31, Anastasiia Tsyplia wrote:
>> > Hi Evgeni!
>> >
>> > Just in addition to the previous letter: here
>> > <
>> https://drive.google.com/file/d/0BzveGSDwNVtBaW02VktrZndJdnc/view?usp=sharing
>> >
>> > is my GSoC proposal on tensor product splines. I would be grateful if
>> > you take a look at it!
>> >
>> > Thanks!
>> > Best regards,
>> >
>> > Anastasiia
>> >
>> > 2015-03-19 12:18 GMT+02:00 Anastasiia Tsyplia
>> > <anastasiyatsyplya at gmail.com <mailto:anastasiyatsyplya at gmail.com>>:
>> >
>> >     Hi!
>> >
>> >
>> >
>> >     Great thanks for useful tips to everyone!
>> >
>> >
>> >
>> >     Benny, thanks for the advice, I hope it will be useful to me during
>> >     the spring/summer J.
>> >
>> >
>> >
>> >     Evgeni, once again thanks for the detailed answer!
>> >
>> >
>> >
>> >     As far as I can judge, all current issues with the scipy.interpolate
>> >     are somehow related with the usage of FITPACK library. Such
>> >     conclusion can also be made by counting FITPACK word in our mailing
>> >     history J. And of course it is mentioned on the SciPy?s ideas page.
>> >
>> >
>> >
>> >     So now it becomes clear to me that reimplementig FITPACK routines is
>> >     one of the fundamental issues for scipy.interpolate module, at least
>> >     in the area of splines.
>> >
>> >
>> >
>> >     That?s why I've made my mid to revise my original proposal totally.
>> >
>> >
>> >
>> >     Here
>> >     <
>> https://drive.google.com/file/d/0BzveGSDwNVtBVDZiUlgybGNpcFk/view?usp=sharing
>> >
>> >     is my new GSoC?15 draft proposal on making the alternative to
>> >     Dierckx?s FITPACK library. I understand the difficulties and the
>> >     huge scope of the work to do. I think my proposal can be thought of
>> >     not as a proposal to reimplement FITPACK totally, but to make a
>> >     basic alternative so it can be complemented by new features in
>> future.
>> >
>> >
>> >
>> >     Currently I?m thinking of making a draft for the second proposal on
>> >     tensor product splines.
>> >
>> >
>> >
>> >     The docstring fix I wanted to make appeared to be already fixed
>> >     before me? LSo I think I?ll do fix something else on the weekend.
>> >
>> >
>> >
>> >     Please let me know what you are thinking of my new proposal so I can
>> >     revise it before the registration deadline.
>> >
>> >
>> >
>> >     Best regards,
>> >
>> >
>> >
>> >     Anastasiia
>> >
>> >
>> >     2015-03-16 14:41 GMT+02:00 Evgeni Burovski
>> >     <evgeny.burovskiy at gmail.com <mailto:evgeny.burovskiy at gmail.com>>:
>> >
>> >         Anastasiia,
>> >
>> >         For interpolation with derivatives you can use
>> >         BPoly.from_derivatives.
>> >         This constructs an interpolating polynomial in the Bernstein
>> basis
>> >         though, so you get a Bezier curve. Converting it to b-spline
>> >         basis is
>> >         possible, you just need to be a bit careful with continuity at
>> >         breakpoints. This latter part is not implemented in scipy, but
>> the
>> >         construction of the interpolating polynomial is.
>> >         BPoly.from_derivatives should also work for specifying the end
>> >         derivatives.
>> >
>> >         It is certainly possible to implement this sort of functionality
>> >         directly in the b-spline basis, but I'm not sure it's in scope
>> >         --- an
>> >         add-on for CAD could be a better fit maybe.  Unless there is a
>> >         set of
>> >         applications where using the existing functionality + conversion
>> >         from
>> >         a Bernstein basis to B-spline basis is not sufficient [which
>> might
>> >         very well be, an input from a domain expert would be welcome
>> here.]
>> >
>> >         Regarding fitpack2: yes, BivariateSplines are tensor products.
>> The
>> >         main issue with these, as well as UnivariateSpline are that
>> they are
>> >         black boxes which tightly couple manipulation of the b-spline
>> >         objects
>> >         themselves with fitting. Notice that in your blog post you had
>> >         to use
>> >         a `_from_tck` method, which is, strictly speaking, private (as
>> >         indicated by the leading underscore).  With either functional or
>> >         object-oriented wrappers around FITPACK there is no easy way of
>> >         * constructing the spline object from knots and coefficients
>> >         (you have
>> >         to use semi-private methods)
>> >         * influencing the way the fitting works. (for instance, here is
>> one
>> >         enhancement request: https://github.com/scipy/scipy/issues/2579
>> )
>> >
>> >         Regarding expo-rational splines I have no opinion :-). My gut
>> >         feeling
>> >         from quickly glancing over the link you provided is that it
>> >         falls into
>> >         a fancy side of things, while scipy.interpolate I think needs
>> more
>> >         basic functionality at present.  Again, an input from a domain
>> >         expert
>> >         would be welcome.
>> >
>> >         Regarding an issue with LSQBivariateSpline --- please open an
>> >         issue on
>> >         github for this. Best open a pull request with a fix :-). For
>> >         the GSoC
>> >         requirements I think you need a PR anyway :-).
>> >
>> >         Regarding the automatic fitting/interpolation with non-uniform
>> >         knots.
>> >         The main issue here is how to construct a good knot vector (and
>> what
>> >         is "good"). One problem of FITPACK is that it does what it does
>> and
>> >         it's quite hard to extend/improve on what it does when it
>> performs
>> >         sub-optimally. There is quite a literature on this topic, de
>> Boor's
>> >         book is one option. [Quoting Chuck Harris, "de Boor is not an
>> >         easiest
>> >         read" though.] An alternative way can, in principle, be inferred
>> >         from
>> >         FITPACK source code, from the Dierckx's book and/or other
>> references
>> >         in the FITPACK source code. Looking at MARS algorithms might be
>> >         useful
>> >         as well (py-earth is one implementation), maybe one can try
>> >         implementing generalized cross validation.
>> >
>> >         As far as specific GSoC-sized tasks are concerned: it depends
>> on you
>> >         really. Coming up with a specific proposal for spline fitting
>> would
>> >         require quite a bit of work with the literature and
>> >         experimenting: any
>> >         new algorithm should be competitive with what is already there
>> in
>> >         FITPACK.
>> >         Implementing basic tensor product splines is a definitely a
>> smaller
>> >         project, also definitely less research-y.
>> >         Implementing cardinal b-splines would involve studing what's in
>> >         ndimage and signal. The latter are likely best deprecated, but
>> the
>> >         former contain a lot of fine-tuning and offer very good
>> performance.
>> >         One reasonably well-defined task could be to implement periodic
>> >         splines in the framework of gh-3174. A challenge is to have a
>> >         numerically stable algorithm while still keeping linear algebra
>> >         banded.
>> >
>> >         All I say above is just my perspective on things :-).
>> >
>> >
>> >         Evgeni
>> >
>> >
>> >
>> >
>> >
>> >         On Thu, Mar 12, 2015 at 6:47 PM, Anastasiia Tsyplia
>> >         <anastasiyatsyplya at gmail.com
>> >         <mailto:anastasiyatsyplya at gmail.com>> wrote:
>> >         > Hello!
>> >         >
>> >         > Thanks for expanded and kind reply!
>> >         >
>> >         > Especially thanks for the link to bezierbuilder! It opened my
>> >         eyes on what
>> >         > can be done with the matplotlib. I guess now I?ll abandon my
>> >         efforts to make
>> >         > the implementation with Qt and will start again with only the
>> >         matplotlib.
>> >         > Anyway, this can wait for some time, and when it's done I'll
>> >         definitely
>> >         > share the link to repo with you.
>> >         >
>> >         > Regarding to the optimization I wrote about before:
>> >         >
>> >         > Initially I was thinking about the precise positioning of
>> >         control points
>> >         > while dragging them on the screen in order to get best fit. It
>> >         is obvious,
>> >         > that manual positioning of control points can give a good
>> >         visual result.
>> >         > Following automatic variation in some boundaries can provide
>> >         strict control
>> >         > points positions and numerically best fitting result.
>> >         >
>> >         > By now I?m thinking about the possibility to implement the
>> >         request for some
>> >         > additional parameters from the user for approximating spline
>> >         functions.
>> >         > Actually, this can be user-provided n-order derivatives in
>> >         some points (for
>> >         > example endpoints to get good extrapolation results). Maybe
>> >         this will
>> >         > require implementation of a new class like
>> >         DerivativeControlledSpline or
>> >         > something familiar.
>> >         >
>> >         > Another issue of optimization is the construction of
>> >         non-uniform knot
>> >         > vectors. Just as an example, I think in some cases non-uniform
>> >         knot vector
>> >         > can be constructed using information about the data points?
>> >         density along x
>> >         > and y axes. If these thoughts make any sense please, let me
>> >         know and I?ll
>> >         > try to expand them to some proposal-like state.
>> >         >
>> >         > Regarding to alternative tasks:
>> >         >
>> >         > The list of your alternative tasks pushed me to reread the 7th
>> >         chapter of
>> >         > the book on spline methods, what made me feel excited about
>> >         tensor product
>> >         > spline surfaces. Current module fitpack2 has a big set of
>> classes
>> >         > representing bivariate splines. Aren?t they tensor product
>> >         splines? Or the
>> >         > idea is to move away from FITPACK wrapping? Anyway I feel some
>> >         interest to
>> >         > the issue and I would be grateful if you describe the problem
>> >         more specific
>> >         > so I can estimate the effort and the milestones.
>> >         >
>> >         > Implementation of Cardinal B-splines seems to be of the less
>> >         effort, but not
>> >         > less interest :)
>> >         >
>> >         > In addition, I would like to know what you are thinking about
>> >         expo-rational
>> >         > B-splines. If their implementation in SciPy is welcome, I can
>> >         think about
>> >         > the appropriate proposal.
>> >         >
>> >         > So by now I have 4 ways to go:
>> >         >
>> >         > Tensor product spline surfaces;
>> >         >
>> >         > Cardinal B-splines;
>> >         >
>> >         > Expo-rational B-splines;
>> >         >
>> >         > Optimization methods for spline functions.
>> >         >
>> >         > If it is possible, please provide the information on their
>> >         importance to the
>> >         > SciPy project so I can choose 1 or 2 of them to make the GSoC
>> >         proposal(s).
>> >         >
>> >         > Thanks a lot and best regards,
>> >         >
>> >         > Anastasiia
>> >         >
>> >         >
>> >         > PS
>> >         >
>> >         > While discovering fitpack2 module I guess I found some
>> >         copy-paste bug in
>> >         > docstring on LSQBivariateSpline. It seems that the class
>> >         doesn?t require
>> >         > smoothing parameter on initialization but the docstring about
>> >         it somehow
>> >         > migrated from another class. Should I write about it on IRC
>> >         channel or
>> >         > somewhere else, or maybe do it by myself?
>> >         >
>> >         >
>> >         >
>> >         >
>> >         > 2015-03-09 23:48 GMT+02:00 Ralf Gommers
>> >         <ralf.gommers at gmail.com <mailto:ralf.gommers at gmail.com>>:
>> >         >>
>> >         >> Hi Anastasiia, welcome!
>> >         >>
>> >         >>
>> >         >> On Sun, Mar 8, 2015 at 10:25 AM, Anastasiia Tsyplia
>> >         >> <anastasiyatsyplya at gmail.com
>> >         <mailto:anastasiyatsyplya at gmail.com>> wrote:
>> >         >>>
>> >         >>> Hello,
>> >         >>>
>> >         >>> My name is Anastasiia Tsyplia. I am a 5th-yaer student of
>> >         National Mining
>> >         >>> University of Ukraine.
>> >         >>>
>> >         >>> I am keen on interpolation/approximation with splines and it
>> >         was a nice
>> >         >>> surprise to find out that there is a demand in interpolation
>> >         improvements
>> >         >>> amongst the Scipy's ideas for GSoC'15. However, I've spend
>> >         some time on
>> >         >>> working out the idea of my own.
>> >         >>>
>> >         >>> Recently I've made a post dedicated to description of the
>> >         parametric
>> >         >>> spline curves construction process and approaches to
>> >         approximate engineering
>> >         >>> data by spline functions and parametric spline curves with
>> >         SciPy.
>> >         >>
>> >         >>
>> >         >> Nice blog post!
>> >         >> I'll leave the commenting on technical details you have in
>> >         your draft
>> >         >> proposal to Evgeni and others, just want to say you've made a
>> >         pretty good
>> >         >> start so far.
>> >         >>>
>> >         >>> It seems that using parametric spline curves in
>> >         approximation can be
>> >         >>> extremely useful and time-saving approach. That's why I
>> >         would like to share
>> >         >>> my project idea and hope to hear some feedback as I am about
>> >         to make a
>> >         >>> proposal for the Google Summer of Code.
>> >         >>>
>> >         >>> I have a 2-year experience in programming with Python,
>> >         PyOpengl, PyQt,
>> >         >>> Matplotlib, Numpy & SciPy. Some time I spent to dive into
>> >         ctypes and
>> >         >>> scratched the surface of C. Now my priority is Cython. I've
>> >         read the book on
>> >         >>> the spline methods recommended on SciPy's idea page, so I
>> >         feel myself
>> >         >>> competent in spline methods. I feel free with recursions:
>> >         the last challenge
>> >         >>> I faced was implementation of binary space partitioning
>> >         algorithm in python
>> >         >>> as I was writing my own ray-tracer.
>> >         >>>
>> >         >>> I would like to contribute to SciPy by any means, so I'm
>> >         ready to receive
>> >         >>> instructions on my next move. And, certainly I'm looking
>> >         forward to start
>> >         >>> dealing with B-Splines in Cython as it is also a part of my
>> >         project idea.
>> >         >>
>> >         >>
>> >         >> What I recommend to all newcomers is to start by reading
>> >         >> https://github.com/scipy/scipy/blob/master/HACKING.rst.txt
>> >         and then first
>> >         >> tackly an issue labeled "easy-fix", just to get a feel for
>> the
>> >         >> development/PR process.
>> >         >>
>> >         >> I've checked open issues for Cyhon code, there aren't that
>> >         many at the
>> >         >> moment. Maybe something fun could be to take some code now
>> >         using np.ndarray
>> >         >> and change it to use memoryviews (suggestion by @jakevdp
>> that in
>> >         >> scipy.sparse.csgraph this could help). And include a
>> >         benchmark to show that
>> >         >> it does speed things up
>> >         >> (seehttps://github.com/scipy/scipy/tree/master/benchmarks
>> >         <http://github.com/scipy/scipy/tree/master/benchmarks> for
>> details).
>> >         >>
>> >         >> Regarding B-splines there's
>> >         https://github.com/scipy/scipy/issues/3423,
>> >         >> but I don't recommend tackling that now - that'll be a
>> >         significant amount of
>> >         >> work + discussion.
>> >         >>
>> >         >> Cheers,
>> >         >> Ralf
>> >         >>
>> >         >>
>> >         >> _______________________________________________
>> >         >> SciPy-Dev mailing list
>> >         >> SciPy-Dev at scipy.org <mailto:SciPy-Dev at scipy.org>
>> >         >> http://mail.scipy.org/mailman/listinfo/scipy-dev
>> >         >>
>> >         >
>> >         >
>> >         > _______________________________________________
>> >         > SciPy-Dev mailing list
>> >         > SciPy-Dev at scipy.org <mailto:SciPy-Dev at scipy.org>
>> >         > http://mail.scipy.org/mailman/listinfo/scipy-dev
>> >         >
>> >         _______________________________________________
>> >         SciPy-Dev mailing list
>> >         SciPy-Dev at scipy.org <mailto:SciPy-Dev at scipy.org>
>> >         http://mail.scipy.org/mailman/listinfo/scipy-dev
>> >
>> >
>> >
>> >
>> >
>> > _______________________________________________
>> > SciPy-Dev mailing list
>> > SciPy-Dev at scipy.org
>> > http://mail.scipy.org/mailman/listinfo/scipy-dev
>> >
>>
>>
>> --
>> -- Andreas.
>> _______________________________________________
>> SciPy-Dev mailing list
>> SciPy-Dev at scipy.org
>> http://mail.scipy.org/mailman/listinfo/scipy-dev
>>
>
>
> _______________________________________________
> SciPy-Dev mailing list
> SciPy-Dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/scipy-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scipy-dev/attachments/20150404/48f7c775/attachment.html>

From ralf.gommers at gmail.com  Sat Apr  4 04:54:05 2015
From: ralf.gommers at gmail.com (Ralf Gommers)
Date: Sat, 4 Apr 2015 10:54:05 +0200
Subject: [SciPy-Dev] 0.16.0 release
Message-ID: <CABL7CQh4E4ESabrJt6J91nAYwd9QN++p5kn3dq1xv+2bxK-gVA@mail.gmail.com>

Hi all,

It's about time to start preparing for the 0.16.0 release. Unusually, there
are zero blocking issues it looks like. A lot of PRs to merge of course,
but nothing very large that still needs a lot of work.

Most important question: does anyone want to take the release manager role
for this release? It would be good to start rotating this role for every
minor release. This will help spread the knowledge on how to do releases
and also spread the workload (it's not that much for a single release, but
it adds up).

Cheers,
Ralf
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scipy-dev/attachments/20150404/be2a8423/attachment.html>

From anastasiyatsyplya at gmail.com  Sat Apr  4 07:17:00 2015
From: anastasiyatsyplya at gmail.com (Anastasiia Tsyplia)
Date: Sat, 4 Apr 2015 14:17:00 +0300
Subject: [SciPy-Dev] GSoC'15 Idea: Approximation with Parametric Splines
In-Reply-To: <CABL7CQgOJzvC=1FrbQPRAqCeYX8LYm9x_WxXd2ifBN=5=52DAQ@mail.gmail.com>
References: <CAJ_+_pU6Qp8sBdOXYFk3m1yz6de9SiX1sp4R6ZpJvJ6=51bZMg@mail.gmail.com>
	<CABL7CQji01rB0FDVd4XLZQ9mwoCpx8-5TTt9MwOjttPnZ-nidQ@mail.gmail.com>
	<CAJ_+_pUz3PgkLA2iUfeG1cJ=NpYzFv4dwaFHZBFRb=Q=65m_YA@mail.gmail.com>
	<CAMRo0isfQtmA+q6j3CGaAxMTW8xbuWOUh=7k8aazVhn71bV8NQ@mail.gmail.com>
	<CAJ_+_pX4Wk5TLJvfLbMZqW6PJASUphGmRCZBL1oFXEqVk0NNSQ@mail.gmail.com>
	<CAJ_+_pUr7vNRSD0qAzuzfsYuVVF8=s8gseWudQGtsCN0vnmimA@mail.gmail.com>
	<55130A5E.6070108@hilboll.de>
	<CAJ_+_pXjDBKf570Y7S5BhAm2=z5NFY-cZ_YJx9ejTV8vjJ7oPw@mail.gmail.com>
	<CABL7CQgOJzvC=1FrbQPRAqCeYX8LYm9x_WxXd2ifBN=5=52DAQ@mail.gmail.com>
Message-ID: <CAJ_+_pX0O4XjgqFy_YL0tG9Mg_=YiYFVD5wn3kavNSeok-ApxQ@mail.gmail.com>

Hi Ralf!

I have already finished the edition.
Thanks!

Best regards,
Anastasiia

2015-04-04 11:11 GMT+03:00 Ralf Gommers <ralf.gommers at gmail.com>:

>
>
> On Sat, Mar 28, 2015 at 12:23 PM, Anastasiia Tsyplia <
> anastasiyatsyplya at gmail.com> wrote:
>
>> Hi all!
>>
>>
>>  Thanks everyone for support during the registration period! It has been
>> a brainstorming couple of weeks but I hope this is only the beginning :)
>>
>> I submitted two proposals to GSoC'15: ?FITPACK alternative... ? and
>> ?Tensor product splines...?. Recently I was asked to edit their titles so
>> they start with SciPy. But it seems I'm no longer able to make any changes.
>> Melange says that I can send a comment to the organization so they allow me
>> to make changes. If this tuning is crucial, please, let me do it.
>>
>
> Hi Anastasiia, it's not critical to us (Scipy mentors) but apparently
> important for some of the admins. You should be able to change the title on
> your proposals now. Please let me know once you've done that, then I'll
> disable editing again.
>
> Cheers,
> Ralf
>
>
>>  Andreas, thanks for the comment. I'll take into account your
>> suggestions.
>>
>>
>>  Best regards,
>>
>> Anastasiia
>>
>> 2015-03-25 21:19 GMT+02:00 Andreas Hilboll <lists at hilboll.de>:
>>
>>> Hi Anastasiia, Eveni, ...
>>>
>>> I'd very much appreciate if tensor product splines would make it into
>>> scipy =)  One wish I have would be to enable periodicity, possibly via a
>>> periodic kwarg, where the use would be able to provide, for any
>>> dimensions, an interval in which the coordinates are considered
>>> periodic.  This is probably too specific to make it into the
>>> implementation, but I'd like to raise the issue early on ...
>>>
>>> Cheers,
>>>   Andreas.
>>>
>>>
>>> On 21.03.2015 22:31, Anastasiia Tsyplia wrote:
>>> > Hi Evgeni!
>>> >
>>> > Just in addition to the previous letter: here
>>> > <
>>> https://drive.google.com/file/d/0BzveGSDwNVtBaW02VktrZndJdnc/view?usp=sharing
>>> >
>>> > is my GSoC proposal on tensor product splines. I would be grateful if
>>> > you take a look at it!
>>> >
>>> > Thanks!
>>> > Best regards,
>>> >
>>> > Anastasiia
>>> >
>>> > 2015-03-19 12:18 GMT+02:00 Anastasiia Tsyplia
>>> > <anastasiyatsyplya at gmail.com <mailto:anastasiyatsyplya at gmail.com>>:
>>> >
>>> >     Hi!
>>> >
>>> >
>>> >
>>> >     Great thanks for useful tips to everyone!
>>> >
>>> >
>>> >
>>> >     Benny, thanks for the advice, I hope it will be useful to me during
>>> >     the spring/summer J.
>>> >
>>> >
>>> >
>>> >     Evgeni, once again thanks for the detailed answer!
>>> >
>>> >
>>> >
>>> >     As far as I can judge, all current issues with the
>>> scipy.interpolate
>>> >     are somehow related with the usage of FITPACK library. Such
>>> >     conclusion can also be made by counting FITPACK word in our mailing
>>> >     history J. And of course it is mentioned on the SciPy?s ideas page.
>>> >
>>> >
>>> >
>>> >     So now it becomes clear to me that reimplementig FITPACK routines
>>> is
>>> >     one of the fundamental issues for scipy.interpolate module, at
>>> least
>>> >     in the area of splines.
>>> >
>>> >
>>> >
>>> >     That?s why I've made my mid to revise my original proposal totally.
>>> >
>>> >
>>> >
>>> >     Here
>>> >     <
>>> https://drive.google.com/file/d/0BzveGSDwNVtBVDZiUlgybGNpcFk/view?usp=sharing
>>> >
>>> >     is my new GSoC?15 draft proposal on making the alternative to
>>> >     Dierckx?s FITPACK library. I understand the difficulties and the
>>> >     huge scope of the work to do. I think my proposal can be thought of
>>> >     not as a proposal to reimplement FITPACK totally, but to make a
>>> >     basic alternative so it can be complemented by new features in
>>> future.
>>> >
>>> >
>>> >
>>> >     Currently I?m thinking of making a draft for the second proposal on
>>> >     tensor product splines.
>>> >
>>> >
>>> >
>>> >     The docstring fix I wanted to make appeared to be already fixed
>>> >     before me? LSo I think I?ll do fix something else on the weekend.
>>> >
>>> >
>>> >
>>> >     Please let me know what you are thinking of my new proposal so I
>>> can
>>> >     revise it before the registration deadline.
>>> >
>>> >
>>> >
>>> >     Best regards,
>>> >
>>> >
>>> >
>>> >     Anastasiia
>>> >
>>> >
>>> >     2015-03-16 14:41 GMT+02:00 Evgeni Burovski
>>> >     <evgeny.burovskiy at gmail.com <mailto:evgeny.burovskiy at gmail.com>>:
>>> >
>>> >         Anastasiia,
>>> >
>>> >         For interpolation with derivatives you can use
>>> >         BPoly.from_derivatives.
>>> >         This constructs an interpolating polynomial in the Bernstein
>>> basis
>>> >         though, so you get a Bezier curve. Converting it to b-spline
>>> >         basis is
>>> >         possible, you just need to be a bit careful with continuity at
>>> >         breakpoints. This latter part is not implemented in scipy, but
>>> the
>>> >         construction of the interpolating polynomial is.
>>> >         BPoly.from_derivatives should also work for specifying the end
>>> >         derivatives.
>>> >
>>> >         It is certainly possible to implement this sort of
>>> functionality
>>> >         directly in the b-spline basis, but I'm not sure it's in scope
>>> >         --- an
>>> >         add-on for CAD could be a better fit maybe.  Unless there is a
>>> >         set of
>>> >         applications where using the existing functionality +
>>> conversion
>>> >         from
>>> >         a Bernstein basis to B-spline basis is not sufficient [which
>>> might
>>> >         very well be, an input from a domain expert would be welcome
>>> here.]
>>> >
>>> >         Regarding fitpack2: yes, BivariateSplines are tensor products.
>>> The
>>> >         main issue with these, as well as UnivariateSpline are that
>>> they are
>>> >         black boxes which tightly couple manipulation of the b-spline
>>> >         objects
>>> >         themselves with fitting. Notice that in your blog post you had
>>> >         to use
>>> >         a `_from_tck` method, which is, strictly speaking, private (as
>>> >         indicated by the leading underscore).  With either functional
>>> or
>>> >         object-oriented wrappers around FITPACK there is no easy way of
>>> >         * constructing the spline object from knots and coefficients
>>> >         (you have
>>> >         to use semi-private methods)
>>> >         * influencing the way the fitting works. (for instance, here
>>> is one
>>> >         enhancement request:
>>> https://github.com/scipy/scipy/issues/2579)
>>> >
>>> >         Regarding expo-rational splines I have no opinion :-). My gut
>>> >         feeling
>>> >         from quickly glancing over the link you provided is that it
>>> >         falls into
>>> >         a fancy side of things, while scipy.interpolate I think needs
>>> more
>>> >         basic functionality at present.  Again, an input from a domain
>>> >         expert
>>> >         would be welcome.
>>> >
>>> >         Regarding an issue with LSQBivariateSpline --- please open an
>>> >         issue on
>>> >         github for this. Best open a pull request with a fix :-). For
>>> >         the GSoC
>>> >         requirements I think you need a PR anyway :-).
>>> >
>>> >         Regarding the automatic fitting/interpolation with non-uniform
>>> >         knots.
>>> >         The main issue here is how to construct a good knot vector
>>> (and what
>>> >         is "good"). One problem of FITPACK is that it does what it
>>> does and
>>> >         it's quite hard to extend/improve on what it does when it
>>> performs
>>> >         sub-optimally. There is quite a literature on this topic, de
>>> Boor's
>>> >         book is one option. [Quoting Chuck Harris, "de Boor is not an
>>> >         easiest
>>> >         read" though.] An alternative way can, in principle, be
>>> inferred
>>> >         from
>>> >         FITPACK source code, from the Dierckx's book and/or other
>>> references
>>> >         in the FITPACK source code. Looking at MARS algorithms might be
>>> >         useful
>>> >         as well (py-earth is one implementation), maybe one can try
>>> >         implementing generalized cross validation.
>>> >
>>> >         As far as specific GSoC-sized tasks are concerned: it depends
>>> on you
>>> >         really. Coming up with a specific proposal for spline fitting
>>> would
>>> >         require quite a bit of work with the literature and
>>> >         experimenting: any
>>> >         new algorithm should be competitive with what is already there
>>> in
>>> >         FITPACK.
>>> >         Implementing basic tensor product splines is a definitely a
>>> smaller
>>> >         project, also definitely less research-y.
>>> >         Implementing cardinal b-splines would involve studing what's in
>>> >         ndimage and signal. The latter are likely best deprecated, but
>>> the
>>> >         former contain a lot of fine-tuning and offer very good
>>> performance.
>>> >         One reasonably well-defined task could be to implement periodic
>>> >         splines in the framework of gh-3174. A challenge is to have a
>>> >         numerically stable algorithm while still keeping linear algebra
>>> >         banded.
>>> >
>>> >         All I say above is just my perspective on things :-).
>>> >
>>> >
>>> >         Evgeni
>>> >
>>> >
>>> >
>>> >
>>> >
>>> >         On Thu, Mar 12, 2015 at 6:47 PM, Anastasiia Tsyplia
>>> >         <anastasiyatsyplya at gmail.com
>>> >         <mailto:anastasiyatsyplya at gmail.com>> wrote:
>>> >         > Hello!
>>> >         >
>>> >         > Thanks for expanded and kind reply!
>>> >         >
>>> >         > Especially thanks for the link to bezierbuilder! It opened my
>>> >         eyes on what
>>> >         > can be done with the matplotlib. I guess now I?ll abandon my
>>> >         efforts to make
>>> >         > the implementation with Qt and will start again with only the
>>> >         matplotlib.
>>> >         > Anyway, this can wait for some time, and when it's done I'll
>>> >         definitely
>>> >         > share the link to repo with you.
>>> >         >
>>> >         > Regarding to the optimization I wrote about before:
>>> >         >
>>> >         > Initially I was thinking about the precise positioning of
>>> >         control points
>>> >         > while dragging them on the screen in order to get best fit.
>>> It
>>> >         is obvious,
>>> >         > that manual positioning of control points can give a good
>>> >         visual result.
>>> >         > Following automatic variation in some boundaries can provide
>>> >         strict control
>>> >         > points positions and numerically best fitting result.
>>> >         >
>>> >         > By now I?m thinking about the possibility to implement the
>>> >         request for some
>>> >         > additional parameters from the user for approximating spline
>>> >         functions.
>>> >         > Actually, this can be user-provided n-order derivatives in
>>> >         some points (for
>>> >         > example endpoints to get good extrapolation results). Maybe
>>> >         this will
>>> >         > require implementation of a new class like
>>> >         DerivativeControlledSpline or
>>> >         > something familiar.
>>> >         >
>>> >         > Another issue of optimization is the construction of
>>> >         non-uniform knot
>>> >         > vectors. Just as an example, I think in some cases
>>> non-uniform
>>> >         knot vector
>>> >         > can be constructed using information about the data points?
>>> >         density along x
>>> >         > and y axes. If these thoughts make any sense please, let me
>>> >         know and I?ll
>>> >         > try to expand them to some proposal-like state.
>>> >         >
>>> >         > Regarding to alternative tasks:
>>> >         >
>>> >         > The list of your alternative tasks pushed me to reread the
>>> 7th
>>> >         chapter of
>>> >         > the book on spline methods, what made me feel excited about
>>> >         tensor product
>>> >         > spline surfaces. Current module fitpack2 has a big set of
>>> classes
>>> >         > representing bivariate splines. Aren?t they tensor product
>>> >         splines? Or the
>>> >         > idea is to move away from FITPACK wrapping? Anyway I feel
>>> some
>>> >         interest to
>>> >         > the issue and I would be grateful if you describe the problem
>>> >         more specific
>>> >         > so I can estimate the effort and the milestones.
>>> >         >
>>> >         > Implementation of Cardinal B-splines seems to be of the less
>>> >         effort, but not
>>> >         > less interest :)
>>> >         >
>>> >         > In addition, I would like to know what you are thinking about
>>> >         expo-rational
>>> >         > B-splines. If their implementation in SciPy is welcome, I can
>>> >         think about
>>> >         > the appropriate proposal.
>>> >         >
>>> >         > So by now I have 4 ways to go:
>>> >         >
>>> >         > Tensor product spline surfaces;
>>> >         >
>>> >         > Cardinal B-splines;
>>> >         >
>>> >         > Expo-rational B-splines;
>>> >         >
>>> >         > Optimization methods for spline functions.
>>> >         >
>>> >         > If it is possible, please provide the information on their
>>> >         importance to the
>>> >         > SciPy project so I can choose 1 or 2 of them to make the GSoC
>>> >         proposal(s).
>>> >         >
>>> >         > Thanks a lot and best regards,
>>> >         >
>>> >         > Anastasiia
>>> >         >
>>> >         >
>>> >         > PS
>>> >         >
>>> >         > While discovering fitpack2 module I guess I found some
>>> >         copy-paste bug in
>>> >         > docstring on LSQBivariateSpline. It seems that the class
>>> >         doesn?t require
>>> >         > smoothing parameter on initialization but the docstring about
>>> >         it somehow
>>> >         > migrated from another class. Should I write about it on IRC
>>> >         channel or
>>> >         > somewhere else, or maybe do it by myself?
>>> >         >
>>> >         >
>>> >         >
>>> >         >
>>> >         > 2015-03-09 23:48 GMT+02:00 Ralf Gommers
>>> >         <ralf.gommers at gmail.com <mailto:ralf.gommers at gmail.com>>:
>>> >         >>
>>> >         >> Hi Anastasiia, welcome!
>>> >         >>
>>> >         >>
>>> >         >> On Sun, Mar 8, 2015 at 10:25 AM, Anastasiia Tsyplia
>>> >         >> <anastasiyatsyplya at gmail.com
>>> >         <mailto:anastasiyatsyplya at gmail.com>> wrote:
>>> >         >>>
>>> >         >>> Hello,
>>> >         >>>
>>> >         >>> My name is Anastasiia Tsyplia. I am a 5th-yaer student of
>>> >         National Mining
>>> >         >>> University of Ukraine.
>>> >         >>>
>>> >         >>> I am keen on interpolation/approximation with splines and
>>> it
>>> >         was a nice
>>> >         >>> surprise to find out that there is a demand in
>>> interpolation
>>> >         improvements
>>> >         >>> amongst the Scipy's ideas for GSoC'15. However, I've spend
>>> >         some time on
>>> >         >>> working out the idea of my own.
>>> >         >>>
>>> >         >>> Recently I've made a post dedicated to description of the
>>> >         parametric
>>> >         >>> spline curves construction process and approaches to
>>> >         approximate engineering
>>> >         >>> data by spline functions and parametric spline curves with
>>> >         SciPy.
>>> >         >>
>>> >         >>
>>> >         >> Nice blog post!
>>> >         >> I'll leave the commenting on technical details you have in
>>> >         your draft
>>> >         >> proposal to Evgeni and others, just want to say you've made
>>> a
>>> >         pretty good
>>> >         >> start so far.
>>> >         >>>
>>> >         >>> It seems that using parametric spline curves in
>>> >         approximation can be
>>> >         >>> extremely useful and time-saving approach. That's why I
>>> >         would like to share
>>> >         >>> my project idea and hope to hear some feedback as I am
>>> about
>>> >         to make a
>>> >         >>> proposal for the Google Summer of Code.
>>> >         >>>
>>> >         >>> I have a 2-year experience in programming with Python,
>>> >         PyOpengl, PyQt,
>>> >         >>> Matplotlib, Numpy & SciPy. Some time I spent to dive into
>>> >         ctypes and
>>> >         >>> scratched the surface of C. Now my priority is Cython. I've
>>> >         read the book on
>>> >         >>> the spline methods recommended on SciPy's idea page, so I
>>> >         feel myself
>>> >         >>> competent in spline methods. I feel free with recursions:
>>> >         the last challenge
>>> >         >>> I faced was implementation of binary space partitioning
>>> >         algorithm in python
>>> >         >>> as I was writing my own ray-tracer.
>>> >         >>>
>>> >         >>> I would like to contribute to SciPy by any means, so I'm
>>> >         ready to receive
>>> >         >>> instructions on my next move. And, certainly I'm looking
>>> >         forward to start
>>> >         >>> dealing with B-Splines in Cython as it is also a part of my
>>> >         project idea.
>>> >         >>
>>> >         >>
>>> >         >> What I recommend to all newcomers is to start by reading
>>> >         >> https://github.com/scipy/scipy/blob/master/HACKING.rst.txt
>>> >         and then first
>>> >         >> tackly an issue labeled "easy-fix", just to get a feel for
>>> the
>>> >         >> development/PR process.
>>> >         >>
>>> >         >> I've checked open issues for Cyhon code, there aren't that
>>> >         many at the
>>> >         >> moment. Maybe something fun could be to take some code now
>>> >         using np.ndarray
>>> >         >> and change it to use memoryviews (suggestion by @jakevdp
>>> that in
>>> >         >> scipy.sparse.csgraph this could help). And include a
>>> >         benchmark to show that
>>> >         >> it does speed things up
>>> >         >> (seehttps://github.com/scipy/scipy/tree/master/benchmarks
>>> >         <http://github.com/scipy/scipy/tree/master/benchmarks> for
>>> details).
>>> >         >>
>>> >         >> Regarding B-splines there's
>>> >         https://github.com/scipy/scipy/issues/3423,
>>> >         >> but I don't recommend tackling that now - that'll be a
>>> >         significant amount of
>>> >         >> work + discussion.
>>> >         >>
>>> >         >> Cheers,
>>> >         >> Ralf
>>> >         >>
>>> >         >>
>>> >         >> _______________________________________________
>>> >         >> SciPy-Dev mailing list
>>> >         >> SciPy-Dev at scipy.org <mailto:SciPy-Dev at scipy.org>
>>> >         >> http://mail.scipy.org/mailman/listinfo/scipy-dev
>>> >         >>
>>> >         >
>>> >         >
>>> >         > _______________________________________________
>>> >         > SciPy-Dev mailing list
>>> >         > SciPy-Dev at scipy.org <mailto:SciPy-Dev at scipy.org>
>>> >         > http://mail.scipy.org/mailman/listinfo/scipy-dev
>>> >         >
>>> >         _______________________________________________
>>> >         SciPy-Dev mailing list
>>> >         SciPy-Dev at scipy.org <mailto:SciPy-Dev at scipy.org>
>>> >         http://mail.scipy.org/mailman/listinfo/scipy-dev
>>> >
>>> >
>>> >
>>> >
>>> >
>>> > _______________________________________________
>>> > SciPy-Dev mailing list
>>> > SciPy-Dev at scipy.org
>>> > http://mail.scipy.org/mailman/listinfo/scipy-dev
>>> >
>>>
>>>
>>> --
>>> -- Andreas.
>>> _______________________________________________
>>> SciPy-Dev mailing list
>>> SciPy-Dev at scipy.org
>>> http://mail.scipy.org/mailman/listinfo/scipy-dev
>>>
>>
>>
>> _______________________________________________
>> SciPy-Dev mailing list
>> SciPy-Dev at scipy.org
>> http://mail.scipy.org/mailman/listinfo/scipy-dev
>>
>>
>
> _______________________________________________
> SciPy-Dev mailing list
> SciPy-Dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/scipy-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scipy-dev/attachments/20150404/5a5d1df6/attachment.html>

From ralf.gommers at gmail.com  Sat Apr  4 09:55:41 2015
From: ralf.gommers at gmail.com (Ralf Gommers)
Date: Sat, 4 Apr 2015 15:55:41 +0200
Subject: [SciPy-Dev] Pull request #4648
In-Reply-To: <551A07A4.2020302@gmail.com>
References: <551A07A4.2020302@gmail.com>
Message-ID: <CABL7CQhqPM6XZKH=pf40WwzHKzEhfeRAaKsr9VfLSNqF0BuM7g@mail.gmail.com>

On Tue, Mar 31, 2015 at 4:34 AM, Andreea Georgescu <
andreea.i.georgescu at gmail.com> wrote:

>  Hi,
>
> I was wondering if there's anything else I could improve on for the pull
> request #4688 <https://github.com/scipy/scipy/pull/4648> (Fixes #4408:
> Vector-valued constraints in minimize() et al). It's still marked as
> "needs-work", although I followed all the things that Pauli Virtanen
> suggested. I'm a new contributor to SciPy so I'm not sure what the process
> is at this stage :). Should I just wait for it to be reviewed and merged,
> or is there anything else I should do?
>

You did everything right (including pinging this list if review takes too
long). Denis just merged your PR. Thanks for this contribution, and keep
them coming!

Cheers,
Ralf
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scipy-dev/attachments/20150404/d7e68a9a/attachment.html>

From aeklant at gmail.com  Sat Apr  4 14:28:31 2015
From: aeklant at gmail.com (Abraham Escalante)
Date: Sat, 4 Apr 2015 12:28:31 -0600
Subject: [SciPy-Dev] `histogram2` and `signaltonoise` deprecation from
	`scipy.stats`
In-Reply-To: <CABL7CQg9t-smPd7XAssGQFnoFH5PBETv=d8W0sdmNm_xLDrCSg@mail.gmail.com>
References: <CAJJ-iFngKbCZNXh6AncSJ+kCbmH9Oh0yzTd8Rzuv+NQjmMgtVQ@mail.gmail.com>
	<CABL7CQg9t-smPd7XAssGQFnoFH5PBETv=d8W0sdmNm_xLDrCSg@mail.gmail.com>
Message-ID: <CAJJ-iFn84+DhwYHpUXyB=gSzoYtg9NVaZmHkWdDh+FA+W-mGxw@mail.gmail.com>

Update: Since there were no objections, the PRs were merged; issues #602
and #609 have been closed.

Thanks Ralf.

2015-03-23 15:47 GMT-06:00 Ralf Gommers <ralf.gommers at gmail.com>:

>
>
> On Mon, Mar 23, 2015 at 4:14 AM, Abraham Escalante <aeklant at gmail.com>
> wrote:
>
>> Hello all,
>>
>> As part of the StatistisCleanup milestone (which I aim to complete by
>> late August) `scipy.stats.histogram2` and `scipy.stats.signaltonoise` are
>> to be deprecated but of course, we would like to get opinions from the
>> community.
>>
>> In short:
>>
>>    - `histogram2` is not well tested and is unnecessary since
>>    `np.histogram2d` can be used instead.
>>    - `signaltonoise` doesn't really belong in `scipy.stats` and it is
>>    rarely used.
>>
>> For more details, please refer to issues #602 and #609.
>>
>> If you have an objection or any opinion regarding this please let me know
>> to take it into account.
>>
>
> Here are the PRs from Abraham that deprecate these functions:
>   https://github.com/scipy/scipy/pull/4656
>   https://github.com/scipy/scipy/pull/4655
>
> Barring any objections, those PRs will be merged soonish.
>
> Ralf
>
>
>
> _______________________________________________
> SciPy-Dev mailing list
> SciPy-Dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/scipy-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scipy-dev/attachments/20150404/a48722b8/attachment.html>

From aeklant at gmail.com  Sat Apr  4 15:30:35 2015
From: aeklant at gmail.com (Abraham Escalante)
Date: Sat, 4 Apr 2015 13:30:35 -0600
Subject: [SciPy-Dev] 0.16.0 release
In-Reply-To: <CABL7CQh4E4ESabrJt6J91nAYwd9QN++p5kn3dq1xv+2bxK-gVA@mail.gmail.com>
References: <CABL7CQh4E4ESabrJt6J91nAYwd9QN++p5kn3dq1xv+2bxK-gVA@mail.gmail.com>
Message-ID: <CAJJ-iFmMWz_Oy+e+2rxxruaafd6c1D-q5JaB=g=dFW_p4SD4=A@mail.gmail.com>

Hi Ralf,

This sounds interesting to me and I would like to contribute but I fear I
may be still too new to the community to be of much help but maybe I could
provide some assistance to learn.

Is this a task for admins only? Is there a way I can contribute?

Cheers,
Abraham.

2015-04-04 2:54 GMT-06:00 Ralf Gommers <ralf.gommers at gmail.com>:

> Hi all,
>
> It's about time to start preparing for the 0.16.0 release. Unusually,
> there are zero blocking issues it looks like. A lot of PRs to merge of
> course, but nothing very large that still needs a lot of work.
>
> Most important question: does anyone want to take the release manager role
> for this release? It would be good to start rotating this role for every
> minor release. This will help spread the knowledge on how to do releases
> and also spread the workload (it's not that much for a single release, but
> it adds up).
>
> Cheers,
> Ralf
>
>
> _______________________________________________
> SciPy-Dev mailing list
> SciPy-Dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/scipy-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scipy-dev/attachments/20150404/5def004a/attachment.html>

From lists at hilboll.de  Sat Apr  4 16:55:06 2015
From: lists at hilboll.de (Andreas Hilboll)
Date: Sat, 04 Apr 2015 22:55:06 +0200
Subject: [SciPy-Dev] 0.16.0 release
In-Reply-To: <CABL7CQh4E4ESabrJt6J91nAYwd9QN++p5kn3dq1xv+2bxK-gVA@mail.gmail.com>
References: <CABL7CQh4E4ESabrJt6J91nAYwd9QN++p5kn3dq1xv+2bxK-gVA@mail.gmail.com>
Message-ID: <55204FAA.2040606@hilboll.de>

Hi Ralf,

depending on the time frame and tasks I might be able to contribute.
Is there a write-up of the work involved?  If not, can you outline what
needs to be done and what skills are required?

Cheers,
  Andreas.


From andreea.i.georgescu at gmail.com  Sat Apr  4 18:06:25 2015
From: andreea.i.georgescu at gmail.com (Andreea Georgescu)
Date: Sat, 4 Apr 2015 15:06:25 -0700
Subject: [SciPy-Dev] Pull request #4648
In-Reply-To: <CABL7CQhqPM6XZKH=pf40WwzHKzEhfeRAaKsr9VfLSNqF0BuM7g@mail.gmail.com>
References: <551A07A4.2020302@gmail.com>
	<CABL7CQhqPM6XZKH=pf40WwzHKzEhfeRAaKsr9VfLSNqF0BuM7g@mail.gmail.com>
Message-ID: <5BA77563-5CC8-4716-90D5-AFC31CFFE632@gmail.com>

Awesome, thank you! :) 


> On Apr 4, 2015, at 6:55 AM, Ralf Gommers <ralf.gommers at gmail.com> wrote:
> 
> 
> 
>> On Tue, Mar 31, 2015 at 4:34 AM, Andreea Georgescu <andreea.i.georgescu at gmail.com> wrote:
>> Hi, 
>> 
>> I was wondering if there's anything else I could improve on for the pull request #4688 (Fixes #4408: Vector-valued constraints in minimize() et al). It's still marked as "needs-work", although I followed all the things that Pauli Virtanen suggested. I'm a new contributor to SciPy     so I'm not sure what the process is at this stage :). Should I just wait for it to be reviewed and merged, or is there anything else I should do?
> 
> You did everything right (including pinging this list if review takes too long). Denis just merged your PR. Thanks for this contribution, and keep them coming!
> 
> Cheers,
> Ralf
> 
> 
> _______________________________________________
> SciPy-Dev mailing list
> SciPy-Dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/scipy-dev
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scipy-dev/attachments/20150404/ec0db6a3/attachment.html>

From ralf.gommers at gmail.com  Sat Apr  4 19:01:43 2015
From: ralf.gommers at gmail.com (Ralf Gommers)
Date: Sun, 5 Apr 2015 01:01:43 +0200
Subject: [SciPy-Dev] [Numpy-discussion] GSoC students: please read
In-Reply-To: <CABL7CQiAj8pm+jQ0k5sRJE9oj40fgrJCiAgXWY01=_vwXeQKZQ@mail.gmail.com>
References: <CABL7CQi2m2OEviLkxi1TLmnyVdP3Zfas-c3mff0f+Y=zZTa_Kw@mail.gmail.com>
	<CAEQ_Tvf1JevCk--WPNLbSqkJrmKh-kJJE17F-CnFRKOCiUas0Q@mail.gmail.com>
	<CABL7CQiAj8pm+jQ0k5sRJE9oj40fgrJCiAgXWY01=_vwXeQKZQ@mail.gmail.com>
Message-ID: <CABL7CQi2631iBHOBP1bt2AneWY-ZjPzLkMSmJMS5zBveKZGD=g@mail.gmail.com>

On Mon, Mar 23, 2015 at 10:42 PM, Ralf Gommers <ralf.gommers at gmail.com>
wrote:

Hi Stephan, all,

On Mon, Mar 23, 2015 at 10:29 PM, Stephan Hoyer <shoyer at gmail.com> wrote:
>
>> On Mon, Mar 23, 2015 at 2:21 PM, Ralf Gommers <ralf.gommers at gmail.com>
>> wrote:
>>
>>> It's great to see that this year there are a lot of students interested
>>> in doing a GSoC project with Numpy or Scipy. So far five proposals have
>>> been submitted, and it looks like several more are being prepared now.
>>>
>>
>> Hi Ralf,
>>
>> Is there a centralized place for non-mentors to view proposals and give
>> feedback?
>>
>
> Hi Stephan, there isn't really. All students post their drafts to the
> mailing list, where they can get feedback. They're free to keep that draft
> wherever they want - blogs, Github, StackEdit, ftp sites and more are all
> being used. The central overview is in Melange (the official GSoC tool),
> but that's not publicly accessible.
>

This was actually a very good idea, for next year we should require
proposals on Github and added to an overview page. For this year it was a
bit late to require all students to make this change, but I've compiled an
overview of all proposals that have been submitted including links to
Melange and the public drafts that students posted to the mailing lists:
https://github.com/scipy/scipy/wiki/GSoC-project-ideas#student-applications-for-2015-to-scipy-and-numpy

I hope that this helps. Everyone who is signed up as a mentor can comment
(privately or publicly) in Melange, and everyone who's interested can now
more easily find back the mailing list threads on this and comment there.

Cheers,
Ralf




> Note that an overview of project ideas can be found at
> https://github.com/scipy/scipy/wiki/GSoC-project-ideas. If you're
> particularly interested in one or more of those, it should be easy to find
> back in the mailing list archive what students sent draft proposals for
> feedback. Your comments on individual proposals will be much appreciated.
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scipy-dev/attachments/20150405/6fe955f2/attachment.html>

From ralf.gommers at gmail.com  Sun Apr  5 15:40:17 2015
From: ralf.gommers at gmail.com (Ralf Gommers)
Date: Sun, 5 Apr 2015 21:40:17 +0200
Subject: [SciPy-Dev] 0.16.0 release
In-Reply-To: <CAJJ-iFmMWz_Oy+e+2rxxruaafd6c1D-q5JaB=g=dFW_p4SD4=A@mail.gmail.com>
References: <CABL7CQh4E4ESabrJt6J91nAYwd9QN++p5kn3dq1xv+2bxK-gVA@mail.gmail.com>
	<CAJJ-iFmMWz_Oy+e+2rxxruaafd6c1D-q5JaB=g=dFW_p4SD4=A@mail.gmail.com>
Message-ID: <CABL7CQizf4e7xo04dQdvng+12ocDR+c7LGHEe9KvJ5CmFrCKyA@mail.gmail.com>

Hi Abraham, Andreas,

Thanks for your interest to both of you!

The release manager is the person that is responsible for a number of
actions to get from "now we start planning for the next release" to "this
release is done". This description should give a reasonable overview (some
details may be a bit outdated):
https://github.com/numpy/numpy/blob/master/doc/HOWTO_RELEASE.rst.txt

The release manager is the person that has the final responsibility for
what does and does not go into a release, what are considered blocking
issues, etc. This part really has to be done by a core developer. However,
there's a lot of other things that the release manager also usually takes
care of which could equally well be done by one or more contributors who
are not core developers. Here's a list:
- build/test Windows and OS X binaries (.exe, .dmg, .whl)
- automate those binary builds/tests further
- tackle some of the issues that need to be resolved before the alpha /
beta / final release
- PRs to complete the release notes and contributor name mapping
- a PR to the scipy.org repo to put the release in the News section
- improve documentation of the release process itself (the less you know to
start with, the better you can document it)

Things that the release manager should do himself:
- propose the release schedule (dates for alpha/beta/rc/final)
- create the maintenance/0.16.x branch
- tag the (pre-)releases
- send the release announcement emails
- upload binaries to PyPi and SourceForge
- propose/make decisions on blockers and content

And what's always a team effort: merge as many PRs as possible before the
release split.

I hope this gives a clear picture. A "release team" that jointly tackles
the above tasks would be great. Still would like to see a core dev
volunteer though:)

Cheers,
Ralf

P.S. I shouldn't forget to mention that for a long time Matthew Brett has
been building the OS X wheels and done extensive Scipy Stack testing. And
that for an even longer time Christoph Gohlke has been testing pre-releases
with MSVC on Windows and resolving issues that are invariable found on that
platform. I definitely consider them part of the "release team" (not that
we have ever used that name before).




On Sat, Apr 4, 2015 at 9:30 PM, Abraham Escalante <aeklant at gmail.com> wrote:

> Hi Ralf,
>
> This sounds interesting to me and I would like to contribute but I fear I
> may be still too new to the community to be of much help but maybe I could
> provide some assistance to learn.
>
> Is this a task for admins only? Is there a way I can contribute?
>
> Cheers,
> Abraham.
>
> 2015-04-04 2:54 GMT-06:00 Ralf Gommers <ralf.gommers at gmail.com>:
>
>> Hi all,
>>
>> It's about time to start preparing for the 0.16.0 release. Unusually,
>> there are zero blocking issues it looks like. A lot of PRs to merge of
>> course, but nothing very large that still needs a lot of work.
>>
>> Most important question: does anyone want to take the release manager
>> role for this release? It would be good to start rotating this role for
>> every minor release. This will help spread the knowledge on how to do
>> releases and also spread the workload (it's not that much for a single
>> release, but it adds up).
>>
>> Cheers,
>> Ralf
>>
>>
>> _______________________________________________
>> SciPy-Dev mailing list
>> SciPy-Dev at scipy.org
>> http://mail.scipy.org/mailman/listinfo/scipy-dev
>>
>>
>
> _______________________________________________
> SciPy-Dev mailing list
> SciPy-Dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/scipy-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scipy-dev/attachments/20150405/eeec132c/attachment.html>

From ralf.gommers at gmail.com  Sun Apr  5 18:31:22 2015
From: ralf.gommers at gmail.com (Ralf Gommers)
Date: Mon, 6 Apr 2015 00:31:22 +0200
Subject: [SciPy-Dev] Growth and sustainability of Scipy development
Message-ID: <CABL7CQj1a+M5U_hh5OB_tvucPPbOurKNPz6KZA_S=c5o3LATHg@mail.gmail.com>

Hi,

Here's a long story about something I've been thinking about for a while.
Some individual parts have been discussed with a few people, but the
complete story is only how I look at things at the moment. I make some
concrete proposals that could be agreed on in this thread if they turn out
to be uncontroversial; we can start separate threads as needed though.

Summary
--------------

*Problem Statement*
The growth of core developer and project admin capacity, while pretty
strong over the past few years, is not quite keeping up with the growth in
contributions we're seeing.

*Solution*
Identify this as an important issue and start addressing it more actively.

*Main Proposals*
- recruitment: a "become a core dev" mentoring program
- docs:
    - write "how to become a core developer" documentation
    - improve the developer guide further
    - finally merge the Scipy 1.0 roadmap
- communication: set up a private core dev mailing list
- funding:
    - we got some donations (excellent, very useful)
    - have an agreed upon strategy (written down) of what to do with funds
    - then aim at getting more donations and/or getting grants
- governance: we have very little formalized, add something lightweight
that covers most bases


Long Version
-------------------

*Problem Statement*
The number of open PRs is very slowly increasing, and stands now at ~110.
Some of those PRs are stalled, but many simply need reviewing. The number
of active core developers who review and merge PRs is also increasing, but
evidently not at the rate that is necessary for PRs to be reviewed
quickly.  The sometimes long delays in getting a PR merged have likely also
discouraged further contributions from some new contributors.

Note that we did have a lot of growth in active core developers over the
past
few years - I remember Fernando Perez pointing out to me at EuroSciPy'11
that the
Scipy bus factor was exactly 2, now it's more like 8.

*Recruitment*
Therefore I'd like to look for new core developers in a more pro-active way.
That starts by very clearly saying:

  We need your help!

To be more precise, we need both domain specialists that help review PRs for
a single module and core developers that have a broad understanding of how
things are done in Scipy and that take responsibility for signing off on the
quality of PRs and are merging them independently.

A "core dev mentoring program".  This can go quite quickly for people that
are experienced Python programmers and are familiar with the Scipy
community; it may be as little as a few email conversations or Skype calls
on how to get started.  On the other end of the scale, I'd also like to
help people who are new to the Scipy community or even open source
development get started - as long as they have the right level of
coding/communication skills of course. Then the road to commit rights will
of course be longer, but as long as the ambition to get there is present
then it's worth the effort (I know I certainly wasn't very experienced in
contributing to open source projects at the time I became Scipy release
manager, and could definitely have used some help).

*Docs*
What developer docs are missing or need significant improvements? Let's add
those (I volunteer for writing duty). We have a reasonable set (links at
end of email), but it's quite scattered. And what we don't have is "how to
become a core developer" documentation, let's add that. Plus finally merge
the 1.0 roadmap: https://github.com/scipy/scipy/pull/2908.

*Communication*
I'd like to set up a new core dev closed mailing list. Not with the purpose
to do less in public, but simply for organizing the communication that we
already have off-list in a very ad-hoc manner now. Typical topics: deciding
on commit rights, discussions on ranking GSoC students, potential funding
sources and what to do with such funding. I'm not yet sure what to propose
for people with access to that list. We currently have 20 people with
commit rights, of which about half are actually active. Currently decisions
are mostly made between the active devs.

*Funding*
I'm starting to see more opportunities to use funding in effective ways to
sustain or accelerate Scipy development. Main ones are organizing developer
meetups / sprints, and targeted funding to talented individuals (students
mainly) and for work on particularly hairy bottlenecks (example: Windows
test&build). We have a donation page on scipy.org and have received a
number of donations, but have not yet actively solicited donations because
of (a) no fiscal sponsorship agreement with NumFOCUS signed yet, and (b)
while we can use the money now, I'm uncomfortable to spend large amounts
without some written down strategy on how and why we spend funds.

*Governance*
Scipy activity is currently very much centered around code, and other
project aspects get relatively little attention. Big email debates about
governance are often not very productive (more high-bandwidth communication
is necessary here), but it would be helpful to keep making steps in this
area.  Documenting processes, how decisions are
made, who decides on things like commit rights, resolving API discussions,
etc. is important.  That goes hand in hand with the developer docs mentioned
above.  The HACKING and MAINTAINERS documents together with the Numpy
developer guide (which also applies to Scipy) do answer quite a few of those
questions, but they're not exactly comprehensive.

I think we can liberally borrow from other projects on this front, and
draft something reasonable that covers at least basic aspects of how
decisions are made and about things like licensing, (non-)ownership of
trademarks and domain names. This will likely be much easier to reach
consensus on than starting from scratch.
Request: I'd like to keep detailed discussion on the content of such a
governance doc draft for later, and here just discuss/decide whether or not
we want to take this on now and who wants to be involved in putting the
first draft together.

Ralf


Developer docs:
https://github.com/scipy/scipy/blob/master/HACKING.rst.txt
https://github.com/rgommers/scipy/blob/master/doc/MAINTAINERS.rst.txt
http://docs.scipy.org/doc/numpy-dev/dev/
https://github.com/numpy/numpy/blob/master/doc/source/dev/development_environment.rst
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scipy-dev/attachments/20150406/82d1816b/attachment.html>

From lists at hilboll.de  Mon Apr  6 05:27:39 2015
From: lists at hilboll.de (Andreas Hilboll)
Date: Mon, 06 Apr 2015 11:27:39 +0200
Subject: [SciPy-Dev] 0.16.0 release
In-Reply-To: <CABL7CQizf4e7xo04dQdvng+12ocDR+c7LGHEe9KvJ5CmFrCKyA@mail.gmail.com>
References: <CABL7CQh4E4ESabrJt6J91nAYwd9QN++p5kn3dq1xv+2bxK-gVA@mail.gmail.com>	<CAJJ-iFmMWz_Oy+e+2rxxruaafd6c1D-q5JaB=g=dFW_p4SD4=A@mail.gmail.com>
	<CABL7CQizf4e7xo04dQdvng+12ocDR+c7LGHEe9KvJ5CmFrCKyA@mail.gmail.com>
Message-ID: <5522518B.8000401@hilboll.de>

Hi Ralf,

thanks for the clarification!

> However, there's a lot of other things that the release manager also
> usually takes care of which could equally well be done by one or more
> contributors who are not core developers. Here's a list:
> - build/test Windows and OS X binaries (.exe, .dmg, .whl)
> - automate those binary builds/tests further

I'm not working on Windows, so cannot help there.

> - tackle some of the issues that need to be resolved before the alpha /
> beta / final release

How is the process to decide which issues are these?  Currently, there's
11 "prio-high" issues, and 284 "defect" issues.

> - PRs to complete the release notes and contributor name mapping
> - a PR to the scipy.org <http://scipy.org> repo to put the release in
> the News section

Sure, that's easy.

> - improve documentation of the release process itself (the less you know
> to start with, the better you can document it)

I think this email and your other mail from today are already good
starters.  What's needed is a central location where volunteers can be
pointed to.  I can keep this in mind and come up with some ideas.

> I hope this gives a clear picture. A "release team" that jointly tackles
> the above tasks would be great. Still would like to see a core dev
> volunteer though:)

I'd be happy being part of the team.

-- Andreas.


From ewm at redtetrahedron.org  Mon Apr  6 11:29:59 2015
From: ewm at redtetrahedron.org (Eric Moore)
Date: Mon, 6 Apr 2015 11:29:59 -0400
Subject: [SciPy-Dev] Adding routines for updating QR decompositions
In-Reply-To: <CADeotZq4a-_2i34A-6ZunvLHOcastyy8dC5x+_L21fckksM2tQ@mail.gmail.com>
References: <CAGeA38=L5AoBB9X90repNF_W8sdQRuZVERL3wXxhC4ngsgLVBg@mail.gmail.com>
	<54870617.4020500@ensta-bretagne.fr>
	<CAE_rN=0-2KZ2gHR_afLjKefmetWwY9EpU9gyyfPZeenrSeW4fA@mail.gmail.com>
	<CAMMTP+C67nmb9uK8+d9W=g15qVC7cm5Qb4Tzv3iyzYWuyZoZNg@mail.gmail.com>
	<CAMMTP+Bq8rDTXnRmeEGj8Ke_VN7_7q3KSaOeijf_q5KrAizJAQ@mail.gmail.com>
	<CADeotZq4a-_2i34A-6ZunvLHOcastyy8dC5x+_L21fckksM2tQ@mail.gmail.com>
Message-ID: <CAGeA38nhHfGMJF7A10EwvyaGBG6twzw=0JPHCW5Gf10S-wqmRg@mail.gmail.com>

As a last follow up here, the pull request adding QR updating is complete
and soon to be merged. However, it would be good to have some more eyes on
it before then.  If these features interest you, please go take a look.
The pr is https://github.com/scipy/scipy/pull/4249.

This supports both full and economic mode QRs.

Thanks,

Eric

On Tue, Dec 9, 2014 at 3:30 PM, Alexandre Gramfort <
alexandre.gramfort at inria.fr> wrote:

> hi,
>
> I think it would be indeed a nice contrib. On the same line you have
> in scikit-learn [1]
> an implementation of cholesky update and downdate which might be
> useful to a broader audience.
>
> Best,
> Alex
>
> [1]
> https://github.com/scikit-learn/scikit-learn/blob/master/sklearn/linear_model/least_angle.py
>
> On Tue, Dec 9, 2014 at 6:06 PM,  <josef.pktd at gmail.com> wrote:
> >
> >
> > On Tue, Dec 9, 2014 at 10:28 AM, <josef.pktd at gmail.com> wrote:
> >>
> >>
> >>
> >> On Tue, Dec 9, 2014 at 10:15 AM, Clark Fitzgerald <clarkfitzg at gmail.com
> >
> >> wrote:
> >>>
> >>> I was looking for this functionality recently- this would be useful.
> >>>
> >>> Thanks!
> >>>
> >>> On Tue, Dec 9, 2014 at 6:24 AM, Irvin Probst
> >>> <irvin.probst at ensta-bretagne.fr> wrote:
> >>>>
> >>>> On 09/12/2014 15:22, Eric Moore wrote:
> >>>> > I've just submitted PR #4249 adding qr_update, qr_insert and
> qr_delete
> >>>> > for performing rank-k updates, adding rows or columns and removing
> >>>> > rows or colums.
> >>>>
> >>>> That's exactly what I was looking for in order to speed up my pole
> >>>> placement routines ! Actually I was about to start coding this myself
> >>>> tomorrow. I would really like to have this into scipy.
> >>>> Have a nice day.
> >>
> >>
> >>
> >> Same here, good to see this arriving in scipy
> >>
> >> I as looking for this some months ago.
> >>
> >> (although, IIRC, I ended up with Q-less R updating for the special use
> >> case of out of memory least squares.)
> >
> >
> > That was just the first application that I remembered.
> >
> > In other cases like best or all subset regression we need something like
> > this. In my functions for subset regression I had to work around these
> > missing features.
> >
> > Thanks,
> >
> > Josef
> >
> >>
> >>
> >> Josef
> >>
> >>
> >>
> >>>>
> >>>>
> >>>> _______________________________________________
> >>>> SciPy-Dev mailing list
> >>>> SciPy-Dev at scipy.org
> >>>> http://mail.scipy.org/mailman/listinfo/scipy-dev
> >>>
> >>>
> >>>
> >>> _______________________________________________
> >>> SciPy-Dev mailing list
> >>> SciPy-Dev at scipy.org
> >>> http://mail.scipy.org/mailman/listinfo/scipy-dev
> >>>
> >>
> >
> >
> > _______________________________________________
> > SciPy-Dev mailing list
> > SciPy-Dev at scipy.org
> > http://mail.scipy.org/mailman/listinfo/scipy-dev
> >
> _______________________________________________
> SciPy-Dev mailing list
> SciPy-Dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/scipy-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scipy-dev/attachments/20150406/8e85fb81/attachment.html>

From matthew.brett at gmail.com  Thu Apr  9 14:46:14 2015
From: matthew.brett at gmail.com (Matthew Brett)
Date: Thu, 9 Apr 2015 11:46:14 -0700
Subject: [SciPy-Dev] Growth and sustainability of Scipy development
In-Reply-To: <CABL7CQj1a+M5U_hh5OB_tvucPPbOurKNPz6KZA_S=c5o3LATHg@mail.gmail.com>
References: <CABL7CQj1a+M5U_hh5OB_tvucPPbOurKNPz6KZA_S=c5o3LATHg@mail.gmail.com>
Message-ID: <CAH6Pt5rxxOdjJwfZLFjK_Qe3reCP9yhsTH1mUPqKD=q0Kka=7g@mail.gmail.com>

Hi,

On Sun, Apr 5, 2015 at 3:31 PM, Ralf Gommers <ralf.gommers at gmail.com> wrote:
> Hi,
>
> Here's a long story about something I've been thinking about for a while.
> Some individual parts have been discussed with a few people, but the
> complete story is only how I look at things at the moment. I make some
> concrete proposals that could be agreed on in this thread if they turn out
> to be uncontroversial; we can start separate threads as needed though.
>
> Summary
> --------------
>
> *Problem Statement*
> The growth of core developer and project admin capacity, while pretty strong
> over the past few years, is not quite keeping up with the growth in
> contributions we're seeing.
>
> *Solution*
> Identify this as an important issue and start addressing it more actively.
>
> *Main Proposals*
> - recruitment: a "become a core dev" mentoring program
> - docs:
>     - write "how to become a core developer" documentation
>     - improve the developer guide further
>     - finally merge the Scipy 1.0 roadmap
> - communication: set up a private core dev mailing list
> - funding:
>     - we got some donations (excellent, very useful)
>     - have an agreed upon strategy (written down) of what to do with funds
>     - then aim at getting more donations and/or getting grants
> - governance: we have very little formalized, add something lightweight that
> covers most bases
>
>
> Long Version
> -------------------
>
> *Problem Statement*
> The number of open PRs is very slowly increasing, and stands now at ~110.
> Some of those PRs are stalled, but many simply need reviewing. The number of
> active core developers who review and merge PRs is also increasing, but
> evidently not at the rate that is necessary for PRs to be reviewed quickly.
> The sometimes long delays in getting a PR merged have likely also
> discouraged further contributions from some new contributors.
>
> Note that we did have a lot of growth in active core developers over the
> past
> few years - I remember Fernando Perez pointing out to me at EuroSciPy'11
> that the
> Scipy bus factor was exactly 2, now it's more like 8.
>
> *Recruitment*
> Therefore I'd like to look for new core developers in a more pro-active way.
> That starts by very clearly saying:
>
>   We need your help!
>
> To be more precise, we need both domain specialists that help review PRs for
> a single module and core developers that have a broad understanding of how
> things are done in Scipy and that take responsibility for signing off on the
> quality of PRs and are merging them independently.
>
> A "core dev mentoring program".  This can go quite quickly for people that
> are experienced Python programmers and are familiar with the Scipy
> community; it may be as little as a few email conversations or Skype calls
> on how to get started.  On the other end of the scale, I'd also like to help
> people who are new to the Scipy community or even open source development
> get started - as long as they have the right level of coding/communication
> skills of course. Then the road to commit rights will of course be longer,
> but as long as the ambition to get there is present then it's worth the
> effort (I know I certainly wasn't very experienced in contributing to open
> source projects at the time I became Scipy release manager, and could
> definitely have used some help).
>
> *Docs*
> What developer docs are missing or need significant improvements? Let's add
> those (I volunteer for writing duty). We have a reasonable set (links at end
> of email), but it's quite scattered. And what we don't have is "how to
> become a core developer" documentation, let's add that. Plus finally merge
> the 1.0 roadmap: https://github.com/scipy/scipy/pull/2908.
>
> *Communication*
> I'd like to set up a new core dev closed mailing list. Not with the purpose
> to do less in public, but simply for organizing the communication that we
> already have off-list in a very ad-hoc manner now. Typical topics: deciding
> on commit rights, discussions on ranking GSoC students, potential funding
> sources and what to do with such funding. I'm not yet sure what to propose
> for people with access to that list. We currently have 20 people with commit
> rights, of which about half are actually active. Currently decisions are
> mostly made between the active devs.
>
> *Funding*
> I'm starting to see more opportunities to use funding in effective ways to
> sustain or accelerate Scipy development. Main ones are organizing developer
> meetups / sprints, and targeted funding to talented individuals (students
> mainly) and for work on particularly hairy bottlenecks (example: Windows
> test&build). We have a donation page on scipy.org and have received a number
> of donations, but have not yet actively solicited donations because of (a)
> no fiscal sponsorship agreement with NumFOCUS signed yet, and (b) while we
> can use the money now, I'm uncomfortable to spend large amounts without some
> written down strategy on how and why we spend funds.
>
> *Governance*
> Scipy activity is currently very much centered around code, and other
> project aspects get relatively little attention. Big email debates about
> governance are often not very productive (more high-bandwidth communication
> is necessary here), but it would be helpful to keep making steps in this
> area.  Documenting processes, how decisions are
> made, who decides on things like commit rights, resolving API discussions,
> etc. is important.  That goes hand in hand with the developer docs mentioned
> above.  The HACKING and MAINTAINERS documents together with the Numpy
> developer guide (which also applies to Scipy) do answer quite a few of those
> questions, but they're not exactly comprehensive.
>
> I think we can liberally borrow from other projects on this front, and draft
> something reasonable that covers at least basic aspects of how decisions are
> made and about things like licensing, (non-)ownership of trademarks and
> domain names. This will likely be much easier to reach consensus on than
> starting from scratch.
> Request: I'd like to keep detailed discussion on the content of such a
> governance doc draft for later, and here just discuss/decide whether or not
> we want to take this on now and who wants to be involved in putting the
> first draft together.

Maybe a developer hangout would be the best way forward for all of these?

The discussions on governance in the past have been so miserable and
tendentious that it might be better to separate that into a separate
track.

Cheers,

Matthew


From ralf.gommers at gmail.com  Sat Apr 11 08:09:01 2015
From: ralf.gommers at gmail.com (Ralf Gommers)
Date: Sat, 11 Apr 2015 14:09:01 +0200
Subject: [SciPy-Dev] Growth and sustainability of Scipy development
In-Reply-To: <CAH6Pt5rxxOdjJwfZLFjK_Qe3reCP9yhsTH1mUPqKD=q0Kka=7g@mail.gmail.com>
References: <CABL7CQj1a+M5U_hh5OB_tvucPPbOurKNPz6KZA_S=c5o3LATHg@mail.gmail.com>
	<CAH6Pt5rxxOdjJwfZLFjK_Qe3reCP9yhsTH1mUPqKD=q0Kka=7g@mail.gmail.com>
Message-ID: <CABL7CQgAY8Re31H48F4xy6rBh8L6Brf1kY_KCP529dvqvwzCHA@mail.gmail.com>

On Thu, Apr 9, 2015 at 8:46 PM, Matthew Brett <matthew.brett at gmail.com>
wrote:

> Hi,
>
> On Sun, Apr 5, 2015 at 3:31 PM, Ralf Gommers <ralf.gommers at gmail.com>
> wrote:
> > Hi,
> >
> > Here's a long story about something I've been thinking about for a while.
> > Some individual parts have been discussed with a few people, but the
> > complete story is only how I look at things at the moment. I make some
> > concrete proposals that could be agreed on in this thread if they turn
> out
> > to be uncontroversial; we can start separate threads as needed though.
> >
> > Summary
> > --------------
> >
> > *Problem Statement*
> > The growth of core developer and project admin capacity, while pretty
> strong
> > over the past few years, is not quite keeping up with the growth in
> > contributions we're seeing.
> >
> > *Solution*
> > Identify this as an important issue and start addressing it more
> actively.
> >
> > *Main Proposals*
> > - recruitment: a "become a core dev" mentoring program
> > - docs:
> >     - write "how to become a core developer" documentation
> >     - improve the developer guide further
> >     - finally merge the Scipy 1.0 roadmap
> > - communication: set up a private core dev mailing list
> > - funding:
> >     - we got some donations (excellent, very useful)
> >     - have an agreed upon strategy (written down) of what to do with
> funds
> >     - then aim at getting more donations and/or getting grants
> > - governance: we have very little formalized, add something lightweight
> that
> > covers most bases
> >
> >
> > Long Version
> > -------------------
> >
> > *Problem Statement*
> > The number of open PRs is very slowly increasing, and stands now at ~110.
> > Some of those PRs are stalled, but many simply need reviewing. The
> number of
> > active core developers who review and merge PRs is also increasing, but
> > evidently not at the rate that is necessary for PRs to be reviewed
> quickly.
> > The sometimes long delays in getting a PR merged have likely also
> > discouraged further contributions from some new contributors.
> >
> > Note that we did have a lot of growth in active core developers over the
> > past
> > few years - I remember Fernando Perez pointing out to me at EuroSciPy'11
> > that the
> > Scipy bus factor was exactly 2, now it's more like 8.
> >
> > *Recruitment*
> > Therefore I'd like to look for new core developers in a more pro-active
> way.
> > That starts by very clearly saying:
> >
> >   We need your help!
> >
> > To be more precise, we need both domain specialists that help review PRs
> for
> > a single module and core developers that have a broad understanding of
> how
> > things are done in Scipy and that take responsibility for signing off on
> the
> > quality of PRs and are merging them independently.
> >
> > A "core dev mentoring program".  This can go quite quickly for people
> that
> > are experienced Python programmers and are familiar with the Scipy
> > community; it may be as little as a few email conversations or Skype
> calls
> > on how to get started.  On the other end of the scale, I'd also like to
> help
> > people who are new to the Scipy community or even open source development
> > get started - as long as they have the right level of
> coding/communication
> > skills of course. Then the road to commit rights will of course be
> longer,
> > but as long as the ambition to get there is present then it's worth the
> > effort (I know I certainly wasn't very experienced in contributing to
> open
> > source projects at the time I became Scipy release manager, and could
> > definitely have used some help).
> >
> > *Docs*
> > What developer docs are missing or need significant improvements? Let's
> add
> > those (I volunteer for writing duty). We have a reasonable set (links at
> end
> > of email), but it's quite scattered. And what we don't have is "how to
> > become a core developer" documentation, let's add that. Plus finally
> merge
> > the 1.0 roadmap: https://github.com/scipy/scipy/pull/2908.
> >
> > *Communication*
> > I'd like to set up a new core dev closed mailing list. Not with the
> purpose
> > to do less in public, but simply for organizing the communication that we
> > already have off-list in a very ad-hoc manner now. Typical topics:
> deciding
> > on commit rights, discussions on ranking GSoC students, potential funding
> > sources and what to do with such funding. I'm not yet sure what to
> propose
> > for people with access to that list. We currently have 20 people with
> commit
> > rights, of which about half are actually active. Currently decisions are
> > mostly made between the active devs.
> >
> > *Funding*
> > I'm starting to see more opportunities to use funding in effective ways
> to
> > sustain or accelerate Scipy development. Main ones are organizing
> developer
> > meetups / sprints, and targeted funding to talented individuals (students
> > mainly) and for work on particularly hairy bottlenecks (example: Windows
> > test&build). We have a donation page on scipy.org and have received a
> number
> > of donations, but have not yet actively solicited donations because of
> (a)
> > no fiscal sponsorship agreement with NumFOCUS signed yet, and (b) while
> we
> > can use the money now, I'm uncomfortable to spend large amounts without
> some
> > written down strategy on how and why we spend funds.
> >
> > *Governance*
> > Scipy activity is currently very much centered around code, and other
> > project aspects get relatively little attention. Big email debates about
> > governance are often not very productive (more high-bandwidth
> communication
> > is necessary here), but it would be helpful to keep making steps in this
> > area.  Documenting processes, how decisions are
> > made, who decides on things like commit rights, resolving API
> discussions,
> > etc. is important.  That goes hand in hand with the developer docs
> mentioned
> > above.  The HACKING and MAINTAINERS documents together with the Numpy
> > developer guide (which also applies to Scipy) do answer quite a few of
> those
> > questions, but they're not exactly comprehensive.
> >
> > I think we can liberally borrow from other projects on this front, and
> draft
> > something reasonable that covers at least basic aspects of how decisions
> are
> > made and about things like licensing, (non-)ownership of trademarks and
> > domain names. This will likely be much easier to reach consensus on than
> > starting from scratch.
> > Request: I'd like to keep detailed discussion on the content of such a
> > governance doc draft for later, and here just discuss/decide whether or
> not
> > we want to take this on now and who wants to be involved in putting the
> > first draft together.
>
> Maybe a developer hangout would be the best way forward for all of these?
>

Thanks Matthew, that's a good idea.

Probably a Google hangout at some time that's evening in Europe and daytime
in the US would work well. Here's a poll, please fill it out if you want to
participate: http://doodle.com/ibqnemrvuhkvexm2

Would be good to make an effort to keep the discussion as public as
possible. So let's make sure we write a summary of what gets discussed and
post it to this list. And I propose that people who are regular
contributors but happen to not have a commit bit are also very welcome to
join the hangout.



> The discussions on governance in the past have been so miserable and
> tendentious that it might be better to separate that into a separate
> track.
>

I think for Scipy previous discussions (there weren't many) were fine, but
yes I agree that for Numpy they weren't always pleasant. Let's discuss on
the hangout what the best approach is.

Ralf
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scipy-dev/attachments/20150411/ed39e1a0/attachment.html>

From ralf.gommers at gmail.com  Sat Apr 11 08:11:41 2015
From: ralf.gommers at gmail.com (Ralf Gommers)
Date: Sat, 11 Apr 2015 14:11:41 +0200
Subject: [SciPy-Dev] Growth and sustainability of Scipy development
In-Reply-To: <CABL7CQgAY8Re31H48F4xy6rBh8L6Brf1kY_KCP529dvqvwzCHA@mail.gmail.com>
References: <CABL7CQj1a+M5U_hh5OB_tvucPPbOurKNPz6KZA_S=c5o3LATHg@mail.gmail.com>
	<CAH6Pt5rxxOdjJwfZLFjK_Qe3reCP9yhsTH1mUPqKD=q0Kka=7g@mail.gmail.com>
	<CABL7CQgAY8Re31H48F4xy6rBh8L6Brf1kY_KCP529dvqvwzCHA@mail.gmail.com>
Message-ID: <CABL7CQju6hzANvfp8oVhRDBhzVj2jX2jMSJ0H+5pdFbkPGOcZw@mail.gmail.com>

On Sat, Apr 11, 2015 at 2:09 PM, Ralf Gommers <ralf.gommers at gmail.com>
wrote:

>
>
> On Thu, Apr 9, 2015 at 8:46 PM, Matthew Brett <matthew.brett at gmail.com>
> wrote:
>
>> Hi,
>>
>> On Sun, Apr 5, 2015 at 3:31 PM, Ralf Gommers <ralf.gommers at gmail.com>
>> wrote:
>> > Hi,
>> >
>> > Here's a long story about something I've been thinking about for a
>> while.
>> > Some individual parts have been discussed with a few people, but the
>> > complete story is only how I look at things at the moment. I make some
>> > concrete proposals that could be agreed on in this thread if they turn
>> out
>> > to be uncontroversial; we can start separate threads as needed though.
>> >
>> > Summary
>> > --------------
>> >
>> > *Problem Statement*
>> > The growth of core developer and project admin capacity, while pretty
>> strong
>> > over the past few years, is not quite keeping up with the growth in
>> > contributions we're seeing.
>> >
>> > *Solution*
>> > Identify this as an important issue and start addressing it more
>> actively.
>> >
>> > *Main Proposals*
>> > - recruitment: a "become a core dev" mentoring program
>> > - docs:
>> >     - write "how to become a core developer" documentation
>> >     - improve the developer guide further
>> >     - finally merge the Scipy 1.0 roadmap
>> > - communication: set up a private core dev mailing list
>> > - funding:
>> >     - we got some donations (excellent, very useful)
>> >     - have an agreed upon strategy (written down) of what to do with
>> funds
>> >     - then aim at getting more donations and/or getting grants
>> > - governance: we have very little formalized, add something lightweight
>> that
>> > covers most bases
>> >
>> >
>> > Long Version
>> > -------------------
>> >
>> > *Problem Statement*
>> > The number of open PRs is very slowly increasing, and stands now at
>> ~110.
>> > Some of those PRs are stalled, but many simply need reviewing. The
>> number of
>> > active core developers who review and merge PRs is also increasing, but
>> > evidently not at the rate that is necessary for PRs to be reviewed
>> quickly.
>> > The sometimes long delays in getting a PR merged have likely also
>> > discouraged further contributions from some new contributors.
>> >
>> > Note that we did have a lot of growth in active core developers over the
>> > past
>> > few years - I remember Fernando Perez pointing out to me at EuroSciPy'11
>> > that the
>> > Scipy bus factor was exactly 2, now it's more like 8.
>> >
>> > *Recruitment*
>> > Therefore I'd like to look for new core developers in a more pro-active
>> way.
>> > That starts by very clearly saying:
>> >
>> >   We need your help!
>> >
>> > To be more precise, we need both domain specialists that help review
>> PRs for
>> > a single module and core developers that have a broad understanding of
>> how
>> > things are done in Scipy and that take responsibility for signing off
>> on the
>> > quality of PRs and are merging them independently.
>> >
>> > A "core dev mentoring program".  This can go quite quickly for people
>> that
>> > are experienced Python programmers and are familiar with the Scipy
>> > community; it may be as little as a few email conversations or Skype
>> calls
>> > on how to get started.  On the other end of the scale, I'd also like to
>> help
>> > people who are new to the Scipy community or even open source
>> development
>> > get started - as long as they have the right level of
>> coding/communication
>> > skills of course. Then the road to commit rights will of course be
>> longer,
>> > but as long as the ambition to get there is present then it's worth the
>> > effort (I know I certainly wasn't very experienced in contributing to
>> open
>> > source projects at the time I became Scipy release manager, and could
>> > definitely have used some help).
>> >
>> > *Docs*
>> > What developer docs are missing or need significant improvements? Let's
>> add
>> > those (I volunteer for writing duty). We have a reasonable set (links
>> at end
>> > of email), but it's quite scattered. And what we don't have is "how to
>> > become a core developer" documentation, let's add that. Plus finally
>> merge
>> > the 1.0 roadmap: https://github.com/scipy/scipy/pull/2908.
>> >
>> > *Communication*
>> > I'd like to set up a new core dev closed mailing list. Not with the
>> purpose
>> > to do less in public, but simply for organizing the communication that
>> we
>> > already have off-list in a very ad-hoc manner now. Typical topics:
>> deciding
>> > on commit rights, discussions on ranking GSoC students, potential
>> funding
>> > sources and what to do with such funding. I'm not yet sure what to
>> propose
>> > for people with access to that list. We currently have 20 people with
>> commit
>> > rights, of which about half are actually active. Currently decisions are
>> > mostly made between the active devs.
>> >
>> > *Funding*
>> > I'm starting to see more opportunities to use funding in effective ways
>> to
>> > sustain or accelerate Scipy development. Main ones are organizing
>> developer
>> > meetups / sprints, and targeted funding to talented individuals
>> (students
>> > mainly) and for work on particularly hairy bottlenecks (example: Windows
>> > test&build). We have a donation page on scipy.org and have received a
>> number
>> > of donations, but have not yet actively solicited donations because of
>> (a)
>> > no fiscal sponsorship agreement with NumFOCUS signed yet, and (b) while
>> we
>> > can use the money now, I'm uncomfortable to spend large amounts without
>> some
>> > written down strategy on how and why we spend funds.
>> >
>> > *Governance*
>> > Scipy activity is currently very much centered around code, and other
>> > project aspects get relatively little attention. Big email debates about
>> > governance are often not very productive (more high-bandwidth
>> communication
>> > is necessary here), but it would be helpful to keep making steps in this
>> > area.  Documenting processes, how decisions are
>> > made, who decides on things like commit rights, resolving API
>> discussions,
>> > etc. is important.  That goes hand in hand with the developer docs
>> mentioned
>> > above.  The HACKING and MAINTAINERS documents together with the Numpy
>> > developer guide (which also applies to Scipy) do answer quite a few of
>> those
>> > questions, but they're not exactly comprehensive.
>> >
>> > I think we can liberally borrow from other projects on this front, and
>> draft
>> > something reasonable that covers at least basic aspects of how
>> decisions are
>> > made and about things like licensing, (non-)ownership of trademarks and
>> > domain names. This will likely be much easier to reach consensus on than
>> > starting from scratch.
>> > Request: I'd like to keep detailed discussion on the content of such a
>> > governance doc draft for later, and here just discuss/decide whether or
>> not
>> > we want to take this on now and who wants to be involved in putting the
>> > first draft together.
>>
>> Maybe a developer hangout would be the best way forward for all of these?
>>
>
> Thanks Matthew, that's a good idea.
>

In addition, we could briefly discuss the next release (schedule, manager).


> Probably a Google hangout at some time that's evening in Europe and
> daytime in the US would work well. Here's a poll, please fill it out if you
> want to participate: http://doodle.com/ibqnemrvuhkvexm2
>
> Would be good to make an effort to keep the discussion as public as
> possible. So let's make sure we write a summary of what gets discussed and
> post it to this list. And I propose that people who are regular
> contributors but happen to not have a commit bit are also very welcome to
> join the hangout.
>
>
>
>> The discussions on governance in the past have been so miserable and
>> tendentious that it might be better to separate that into a separate
>> track.
>>
>
> I think for Scipy previous discussions (there weren't many) were fine, but
> yes I agree that for Numpy they weren't always pleasant. Let's discuss on
> the hangout what the best approach is.
>
> Ralf
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scipy-dev/attachments/20150411/b906a331/attachment.html>

From matthew.brett at gmail.com  Sat Apr 11 14:58:21 2015
From: matthew.brett at gmail.com (Matthew Brett)
Date: Sat, 11 Apr 2015 11:58:21 -0700
Subject: [SciPy-Dev] Growth and sustainability of Scipy development
In-Reply-To: <CABL7CQgAY8Re31H48F4xy6rBh8L6Brf1kY_KCP529dvqvwzCHA@mail.gmail.com>
References: <CABL7CQj1a+M5U_hh5OB_tvucPPbOurKNPz6KZA_S=c5o3LATHg@mail.gmail.com>
	<CAH6Pt5rxxOdjJwfZLFjK_Qe3reCP9yhsTH1mUPqKD=q0Kka=7g@mail.gmail.com>
	<CABL7CQgAY8Re31H48F4xy6rBh8L6Brf1kY_KCP529dvqvwzCHA@mail.gmail.com>
Message-ID: <CAH6Pt5o3mKr0mAb8NizNGVu-qNPNwWHJRfW64kr0_6V-xPrs3A@mail.gmail.com>

Hi,

On Sat, Apr 11, 2015 at 5:09 AM, Ralf Gommers <ralf.gommers at gmail.com> wrote:
>
>
> On Thu, Apr 9, 2015 at 8:46 PM, Matthew Brett <matthew.brett at gmail.com>
> wrote:
>>
>> Hi,
>>
>> On Sun, Apr 5, 2015 at 3:31 PM, Ralf Gommers <ralf.gommers at gmail.com>
>> wrote:
>> > Hi,
>> >
>> > Here's a long story about something I've been thinking about for a
>> > while.
>> > Some individual parts have been discussed with a few people, but the
>> > complete story is only how I look at things at the moment. I make some
>> > concrete proposals that could be agreed on in this thread if they turn
>> > out
>> > to be uncontroversial; we can start separate threads as needed though.
>> >
>> > Summary
>> > --------------
>> >
>> > *Problem Statement*
>> > The growth of core developer and project admin capacity, while pretty
>> > strong
>> > over the past few years, is not quite keeping up with the growth in
>> > contributions we're seeing.
>> >
>> > *Solution*
>> > Identify this as an important issue and start addressing it more
>> > actively.
>> >
>> > *Main Proposals*
>> > - recruitment: a "become a core dev" mentoring program
>> > - docs:
>> >     - write "how to become a core developer" documentation
>> >     - improve the developer guide further
>> >     - finally merge the Scipy 1.0 roadmap
>> > - communication: set up a private core dev mailing list
>> > - funding:
>> >     - we got some donations (excellent, very useful)
>> >     - have an agreed upon strategy (written down) of what to do with
>> > funds
>> >     - then aim at getting more donations and/or getting grants
>> > - governance: we have very little formalized, add something lightweight
>> > that
>> > covers most bases
>> >
>> >
>> > Long Version
>> > -------------------
>> >
>> > *Problem Statement*
>> > The number of open PRs is very slowly increasing, and stands now at
>> > ~110.
>> > Some of those PRs are stalled, but many simply need reviewing. The
>> > number of
>> > active core developers who review and merge PRs is also increasing, but
>> > evidently not at the rate that is necessary for PRs to be reviewed
>> > quickly.
>> > The sometimes long delays in getting a PR merged have likely also
>> > discouraged further contributions from some new contributors.
>> >
>> > Note that we did have a lot of growth in active core developers over the
>> > past
>> > few years - I remember Fernando Perez pointing out to me at EuroSciPy'11
>> > that the
>> > Scipy bus factor was exactly 2, now it's more like 8.
>> >
>> > *Recruitment*
>> > Therefore I'd like to look for new core developers in a more pro-active
>> > way.
>> > That starts by very clearly saying:
>> >
>> >   We need your help!
>> >
>> > To be more precise, we need both domain specialists that help review PRs
>> > for
>> > a single module and core developers that have a broad understanding of
>> > how
>> > things are done in Scipy and that take responsibility for signing off on
>> > the
>> > quality of PRs and are merging them independently.
>> >
>> > A "core dev mentoring program".  This can go quite quickly for people
>> > that
>> > are experienced Python programmers and are familiar with the Scipy
>> > community; it may be as little as a few email conversations or Skype
>> > calls
>> > on how to get started.  On the other end of the scale, I'd also like to
>> > help
>> > people who are new to the Scipy community or even open source
>> > development
>> > get started - as long as they have the right level of
>> > coding/communication
>> > skills of course. Then the road to commit rights will of course be
>> > longer,
>> > but as long as the ambition to get there is present then it's worth the
>> > effort (I know I certainly wasn't very experienced in contributing to
>> > open
>> > source projects at the time I became Scipy release manager, and could
>> > definitely have used some help).
>> >
>> > *Docs*
>> > What developer docs are missing or need significant improvements? Let's
>> > add
>> > those (I volunteer for writing duty). We have a reasonable set (links at
>> > end
>> > of email), but it's quite scattered. And what we don't have is "how to
>> > become a core developer" documentation, let's add that. Plus finally
>> > merge
>> > the 1.0 roadmap: https://github.com/scipy/scipy/pull/2908.
>> >
>> > *Communication*
>> > I'd like to set up a new core dev closed mailing list. Not with the
>> > purpose
>> > to do less in public, but simply for organizing the communication that
>> > we
>> > already have off-list in a very ad-hoc manner now. Typical topics:
>> > deciding
>> > on commit rights, discussions on ranking GSoC students, potential
>> > funding
>> > sources and what to do with such funding. I'm not yet sure what to
>> > propose
>> > for people with access to that list. We currently have 20 people with
>> > commit
>> > rights, of which about half are actually active. Currently decisions are
>> > mostly made between the active devs.
>> >
>> > *Funding*
>> > I'm starting to see more opportunities to use funding in effective ways
>> > to
>> > sustain or accelerate Scipy development. Main ones are organizing
>> > developer
>> > meetups / sprints, and targeted funding to talented individuals
>> > (students
>> > mainly) and for work on particularly hairy bottlenecks (example: Windows
>> > test&build). We have a donation page on scipy.org and have received a
>> > number
>> > of donations, but have not yet actively solicited donations because of
>> > (a)
>> > no fiscal sponsorship agreement with NumFOCUS signed yet, and (b) while
>> > we
>> > can use the money now, I'm uncomfortable to spend large amounts without
>> > some
>> > written down strategy on how and why we spend funds.
>> >
>> > *Governance*
>> > Scipy activity is currently very much centered around code, and other
>> > project aspects get relatively little attention. Big email debates about
>> > governance are often not very productive (more high-bandwidth
>> > communication
>> > is necessary here), but it would be helpful to keep making steps in this
>> > area.  Documenting processes, how decisions are
>> > made, who decides on things like commit rights, resolving API
>> > discussions,
>> > etc. is important.  That goes hand in hand with the developer docs
>> > mentioned
>> > above.  The HACKING and MAINTAINERS documents together with the Numpy
>> > developer guide (which also applies to Scipy) do answer quite a few of
>> > those
>> > questions, but they're not exactly comprehensive.
>> >
>> > I think we can liberally borrow from other projects on this front, and
>> > draft
>> > something reasonable that covers at least basic aspects of how decisions
>> > are
>> > made and about things like licensing, (non-)ownership of trademarks and
>> > domain names. This will likely be much easier to reach consensus on than
>> > starting from scratch.
>> > Request: I'd like to keep detailed discussion on the content of such a
>> > governance doc draft for later, and here just discuss/decide whether or
>> > not
>> > we want to take this on now and who wants to be involved in putting the
>> > first draft together.
>>
>> Maybe a developer hangout would be the best way forward for all of these?
>
>
> Thanks Matthew, that's a good idea.
>
> Probably a Google hangout at some time that's evening in Europe and daytime
> in the US would work well. Here's a poll, please fill it out if you want to
> participate: http://doodle.com/ibqnemrvuhkvexm2

Done - thanks.

> Would be good to make an effort to keep the discussion as public as
> possible. So let's make sure we write a summary of what gets discussed and
> post it to this list. And I propose that people who are regular contributors
> but happen to not have a commit bit are also very welcome to join the
> hangout.

Thanks - that is a good idea.

>> The discussions on governance in the past have been so miserable and
>> tendentious that it might be better to separate that into a separate
>> track.
>
>
> I think for Scipy previous discussions (there weren't many) were fine, but
> yes I agree that for Numpy they weren't always pleasant. Let's discuss on
> the hangout what the best approach is.

Yes, right, it was the numpy discussions, but there is so much overlap
between the lists, I am sure the atmosphere over there has had an
effect.

Cheers,

Matthew


From befelix at ethz.ch  Sun Apr 12 05:52:58 2015
From: befelix at ethz.ch (Felix Berkenkamp)
Date: Sun, 12 Apr 2015 11:52:58 +0200
Subject: [SciPy-Dev] Split signal.lti class into subclasses
In-Reply-To: <54F33492.3050304@ethz.ch>
References: <54F33492.3050304@ethz.ch>
Message-ID: <552A407A.2090807@ethz.ch>

Hi everyone,

it would be great if someone could comment on my pull request from a 
month ago (see below for details)
https://github.com/scipy/scipy/pull/4576

This is my first contribution to scipy, so any feedback would be highly 
appreciated.

Thanks,
Felix

On 01/03/15 16:47, Felix Berkenkamp wrote:
> Hi everyone,
>
> I started looking into improving the signal.lti class following the
> issue discussed at
> https://github.com/scipy/scipy/issues/2912
>
> The pull request can be found here:
> https://github.com/scipy/scipy/pull/4576
>
> The main idea is to split the lti class into ss, tf, and zpk subclasses.
> Calling the lti class itself returns instances of these three subclasses.
>
> Advantages
> * No redundant information (lti class currently holds the information of
> all 3 classes)
> * Reduce overhead (creating 3 system representations)
> * Switching between the different subclasses is more explicit: obj.ss(),
> obj.tf(), obj.zpk()
> * Avoids one huge class for everything
> * Is fully backwards compatible (as far as I can tell)
> * Similar to what Octave / Matlab does (easier to convert code from
> there to scipy)
>
> Disadvantages:
> * Accessing properties that are not part of the subclass is more
> expensive (e.g.: sys = ss(1,1,1,1); sys.num  --> this now returns
> sys.tf().num).
>
> Any suggestions / comments / things I've broken?
>
> Best,
> Felix
>



From irvin.probst at ensta-bretagne.fr  Sun Apr 12 06:13:30 2015
From: irvin.probst at ensta-bretagne.fr (Irvin Probst)
Date: Sun, 12 Apr 2015 12:13:30 +0200
Subject: [SciPy-Dev] Split signal.lti class into subclasses
In-Reply-To: <552A407A.2090807@ethz.ch>
References: <54F33492.3050304@ethz.ch> <552A407A.2090807@ethz.ch>
Message-ID: <b893f0bce1ec8b27a515d189e9f58f37@webmail.ensta-bretagne.fr>

Hi,
I hope I won't upset anyone saying this, but IMHO the SS/tf stuff in 
scipy right now is close to unusable so any improvement is a good thing.



On Sun, 12 Apr 2015 11:52:58 +0200, Felix Berkenkamp wrote:
> Hi everyone,
>
> it would be great if someone could comment on my pull request from a
> month ago (see below for details)
> https://github.com/scipy/scipy/pull/4576
>
> This is my first contribution to scipy, so any feedback would be 
> highly
> appreciated.
>
> Thanks,
> Felix
>
> On 01/03/15 16:47, Felix Berkenkamp wrote:
>> Hi everyone,
>>
>> I started looking into improving the signal.lti class following the
>> issue discussed at
>> https://github.com/scipy/scipy/issues/2912
>>
>> The pull request can be found here:
>> https://github.com/scipy/scipy/pull/4576
>>
>> The main idea is to split the lti class into ss, tf, and zpk 
>> subclasses.
>> Calling the lti class itself returns instances of these three 
>> subclasses.
>>
>> Advantages
>> * No redundant information (lti class currently holds the 
>> information of
>> all 3 classes)
>> * Reduce overhead (creating 3 system representations)
>> * Switching between the different subclasses is more explicit: 
>> obj.ss(),
>> obj.tf(), obj.zpk()
>> * Avoids one huge class for everything
>> * Is fully backwards compatible (as far as I can tell)
>> * Similar to what Octave / Matlab does (easier to convert code from
>> there to scipy)
>>
>> Disadvantages:
>> * Accessing properties that are not part of the subclass is more
>> expensive (e.g.: sys = ss(1,1,1,1); sys.num  --> this now returns
>> sys.tf().num).
>>
>> Any suggestions / comments / things I've broken?
>>
>> Best,
>> Felix
>>
>
> _______________________________________________
> SciPy-Dev mailing list
> SciPy-Dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/scipy-dev



From ralf.gommers at gmail.com  Sun Apr 12 13:30:48 2015
From: ralf.gommers at gmail.com (Ralf Gommers)
Date: Sun, 12 Apr 2015 19:30:48 +0200
Subject: [SciPy-Dev] Split signal.lti class into subclasses
In-Reply-To: <b893f0bce1ec8b27a515d189e9f58f37@webmail.ensta-bretagne.fr>
References: <54F33492.3050304@ethz.ch> <552A407A.2090807@ethz.ch>
	<b893f0bce1ec8b27a515d189e9f58f37@webmail.ensta-bretagne.fr>
Message-ID: <CABL7CQgjyUf-h+gqX=QLoThKFRZiXq_ApK=-xrPSMeNxzXacwA@mail.gmail.com>

On Sun, Apr 12, 2015 at 12:13 PM, Irvin Probst <
irvin.probst at ensta-bretagne.fr> wrote:

> Hi,
> I hope I won't upset anyone saying this, but IMHO the SS/tf stuff in
> scipy right now is close to unusable so any improvement is a good thing.
>

I don't think anyone will be offended that easily:) It's still not in great
shape, but there have been significant improvements over the last few
releases with the addition of more functions that use zpk internally and
very recently the SOS format.



>  On Sun, 12 Apr 2015 11:52:58 +0200, Felix Berkenkamp wrote:
> > Hi everyone,
> >
> > it would be great if someone could comment on my pull request from a
> > month ago (see below for details)
> > https://github.com/scipy/scipy/pull/4576
> >
> > This is my first contribution to scipy, so any feedback would be
> > highly appreciated.
>

Sorry about the delay in reviewing. Your attempt at tackling this is much
appreciated - it's one of the more important issues in scipy.signal to
address. I'll have a look now.

>
> > Thanks,
> > Felix
> >
> > On 01/03/15 16:47, Felix Berkenkamp wrote:
> >> Hi everyone,
> >>
> >> I started looking into improving the signal.lti class following the
> >> issue discussed at
> >> https://github.com/scipy/scipy/issues/2912
>

Note that there is more discussion on this topic in several places. Here
are two relevant issues:
https://github.com/scipy/scipy/issues/2973 (operations on LTI instances)
https://github.com/scipy/scipy/issues/3259 (letting filter functions accept
zpk/ss input)


> >>
> >> The pull request can be found here:
> >> https://github.com/scipy/scipy/pull/4576
>
>
>>
> >> The main idea is to split the lti class into ss, tf, and zpk
> >> subclasses.
> >> Calling the lti class itself returns instances of these three
> >> subclasses.
>

After a first browse of your code I still think this is a good idea. We
need to think carefully about the API though. First thoughts:
- I'd like the classes to have more Pythonic names, instead of too short
names that are taken from Matlan
- The conversion functions should probably be named "to_zpk, to_ss, "to_tf"
rather than "zpk, tf, ss".
- There's no SOS class, we do have that format now (
https://github.com/scipy/scipy/pull/3717)
- Warren suggested "lti.from_tf/zpk/ss/sos" constructors on issue 3259.
Looks like that should fit with your PR, but good to think about now.
- I think that this fits with where we want to go with input to filter
functions (gh-3259), but ideally we'd work that out together with this PR
(not the implementation, but at least a sketch of the new API).

Ralf


>>
> >> Advantages
> >> * No redundant information (lti class currently holds the
> >> information of
> >> all 3 classes)
> >> * Reduce overhead (creating 3 system representations)
> >> * Switching between the different subclasses is more explicit:
> >> obj.ss(),
> >> obj.tf(), obj.zpk()
> >> * Avoids one huge class for everything
> >> * Is fully backwards compatible (as far as I can tell)
> >> * Similar to what Octave / Matlab does (easier to convert code from
> >> there to scipy)
> >>
> >> Disadvantages:
> >> * Accessing properties that are not part of the subclass is more
> >> expensive (e.g.: sys = ss(1,1,1,1); sys.num  --> this now returns
> >> sys.tf().num).
> >>
> >> Any suggestions / comments / things I've broken?
> >>
> >> Best,
> >> Felix
> >>
> >
> > _______________________________________________
> > SciPy-Dev mailing list
> > SciPy-Dev at scipy.org
> > http://mail.scipy.org/mailman/listinfo/scipy-dev
>
> _______________________________________________
> SciPy-Dev mailing list
> SciPy-Dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/scipy-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scipy-dev/attachments/20150412/ce643131/attachment.html>

From ralf.gommers at gmail.com  Mon Apr 13 16:06:27 2015
From: ralf.gommers at gmail.com (Ralf Gommers)
Date: Mon, 13 Apr 2015 22:06:27 +0200
Subject: [SciPy-Dev] Growth and sustainability of Scipy development
In-Reply-To: <CAH6Pt5o3mKr0mAb8NizNGVu-qNPNwWHJRfW64kr0_6V-xPrs3A@mail.gmail.com>
References: <CABL7CQj1a+M5U_hh5OB_tvucPPbOurKNPz6KZA_S=c5o3LATHg@mail.gmail.com>
	<CAH6Pt5rxxOdjJwfZLFjK_Qe3reCP9yhsTH1mUPqKD=q0Kka=7g@mail.gmail.com>
	<CABL7CQgAY8Re31H48F4xy6rBh8L6Brf1kY_KCP529dvqvwzCHA@mail.gmail.com>
	<CAH6Pt5o3mKr0mAb8NizNGVu-qNPNwWHJRfW64kr0_6V-xPrs3A@mail.gmail.com>
Message-ID: <CABL7CQh1ppnVisE4Y_qXjO1QUYKO=Wkvka_80RHrTr7jpASA8w@mail.gmail.com>

On Sat, Apr 11, 2015 at 8:58 PM, Matthew Brett <matthew.brett at gmail.com>
wrote:

> Hi,
>
> On Sat, Apr 11, 2015 at 5:09 AM, Ralf Gommers <ralf.gommers at gmail.com>
> wrote:
> >
>
> >
> > Probably a Google hangout at some time that's evening in Europe and
> daytime
> > in the US would work well. Here's a poll, please fill it out if you want
> to
> > participate: http://doodle.com/ibqnemrvuhkvexm2
>
> Done - thanks.
>

Poll dates are Fri 17 - Wed 22  April, so I'll close the poll on Wednesday
and pick a time. Please fill it out by Wed morning if you want to
participate.

Ralf
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scipy-dev/attachments/20150413/682133fd/attachment.html>

From matthew.brett at gmail.com  Tue Apr 14 18:12:37 2015
From: matthew.brett at gmail.com (Matthew Brett)
Date: Tue, 14 Apr 2015 15:12:37 -0700
Subject: [SciPy-Dev] Mass 1D interpolation, advice
Message-ID: <CAH6Pt5q1-egmtz-VByaUi0gCgCU-1YYN1jXx7-ZbOy6Pp5Xc0A@mail.gmail.com>

Hi,

In brain imaging (my world) - we need to do 1D interpolation of large
datasets, specifically, we have data in the order of (X, Y) = (200,
4000), where we are interpolating at the same ~200 X locations for
each of the 4000 rows.

We need to do this with extrapolation at the boundaries, so I think we
can't use scipy.interpolate.interp1d which is otherwise well suited
for the task.

scipy.interpolate.InterpolatedUnivariateSpline does deal with
extrapolation, but only works with vectors.

I think, in order to make speed reasonable, I will need to loop over
the rows in compiled code.

My questions are:

a) how easy would it be to try and link some external Cython code
against the fitpack code in scipy?
b) if that is not easy, what can I do that will also be of most use to
other scipy users?

Cheers,

Matthew


From evgeny.burovskiy at gmail.com  Tue Apr 14 18:24:37 2015
From: evgeny.burovskiy at gmail.com (Evgeni Burovski)
Date: Tue, 14 Apr 2015 23:24:37 +0100
Subject: [SciPy-Dev] Mass 1D interpolation, advice
In-Reply-To: <CAH6Pt5q1-egmtz-VByaUi0gCgCU-1YYN1jXx7-ZbOy6Pp5Xc0A@mail.gmail.com>
References: <CAH6Pt5q1-egmtz-VByaUi0gCgCU-1YYN1jXx7-ZbOy6Pp5Xc0A@mail.gmail.com>
Message-ID: <CAMRo0ivN0Phav8ZG+OuWqXpb=BjjCdKvmuOK41cr0G4-4dDOYw@mail.gmail.com>

What's the interpolation order?

* Warren recently added nd functionality to splev:
https://github.com/scipy/scipy/pull/4059

* Have you tried polynomial interpolators? PPoly,
BPoly.from_derivatives might be what you are after




On Tue, Apr 14, 2015 at 11:12 PM, Matthew Brett <matthew.brett at gmail.com> wrote:
> Hi,
>
> In brain imaging (my world) - we need to do 1D interpolation of large
> datasets, specifically, we have data in the order of (X, Y) = (200,
> 4000), where we are interpolating at the same ~200 X locations for
> each of the 4000 rows.
>
> We need to do this with extrapolation at the boundaries, so I think we
> can't use scipy.interpolate.interp1d which is otherwise well suited
> for the task.
>
> scipy.interpolate.InterpolatedUnivariateSpline does deal with
> extrapolation, but only works with vectors.
>
> I think, in order to make speed reasonable, I will need to loop over
> the rows in compiled code.
>
> My questions are:
>
> a) how easy would it be to try and link some external Cython code
> against the fitpack code in scipy?
> b) if that is not easy, what can I do that will also be of most use to
> other scipy users?
>
> Cheers,
>
> Matthew
> _______________________________________________
> SciPy-Dev mailing list
> SciPy-Dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/scipy-dev


From matthew.brett at gmail.com  Tue Apr 14 18:39:06 2015
From: matthew.brett at gmail.com (Matthew Brett)
Date: Tue, 14 Apr 2015 15:39:06 -0700
Subject: [SciPy-Dev] Mass 1D interpolation, advice
In-Reply-To: <CAMRo0ivN0Phav8ZG+OuWqXpb=BjjCdKvmuOK41cr0G4-4dDOYw@mail.gmail.com>
References: <CAH6Pt5q1-egmtz-VByaUi0gCgCU-1YYN1jXx7-ZbOy6Pp5Xc0A@mail.gmail.com>
	<CAMRo0ivN0Phav8ZG+OuWqXpb=BjjCdKvmuOK41cr0G4-4dDOYw@mail.gmail.com>
Message-ID: <CAH6Pt5pxgw3eJ8gYgN=QEvto=XAbGzmr8wW_xF1So2d75dhZzA@mail.gmail.com>

Hi,

On Tue, Apr 14, 2015 at 3:24 PM, Evgeni Burovski
<evgeny.burovskiy at gmail.com> wrote:
> What's the interpolation order?

Cubic spline, linear, nearest neighbor in that order of frequency.

> * Warren recently added nd functionality to splev:
> https://github.com/scipy/scipy/pull/4059

A quick scan suggested this allows an N-D array of x input values,
whereas I want to evaluate a 1D array of x input values for 2- or N-D
array of y values - is that right?

> * Have you tried polynomial interpolators? PPoly,
> BPoly.from_derivatives might be what you are after

Forgive my terrible ignorance, but is it easy to do cubic spline
interpolation using these classes?

Thanks for the reply,

Matthew


From josef.pktd at gmail.com  Tue Apr 14 19:05:11 2015
From: josef.pktd at gmail.com (josef.pktd at gmail.com)
Date: Tue, 14 Apr 2015 19:05:11 -0400
Subject: [SciPy-Dev] Mass 1D interpolation, advice
In-Reply-To: <CAH6Pt5pxgw3eJ8gYgN=QEvto=XAbGzmr8wW_xF1So2d75dhZzA@mail.gmail.com>
References: <CAH6Pt5q1-egmtz-VByaUi0gCgCU-1YYN1jXx7-ZbOy6Pp5Xc0A@mail.gmail.com>
	<CAMRo0ivN0Phav8ZG+OuWqXpb=BjjCdKvmuOK41cr0G4-4dDOYw@mail.gmail.com>
	<CAH6Pt5pxgw3eJ8gYgN=QEvto=XAbGzmr8wW_xF1So2d75dhZzA@mail.gmail.com>
Message-ID: <CAMMTP+DMRvZ82UxuJbmhqmDBN652rC3bo+3bAhZpeJcK1FixtA@mail.gmail.com>

On Tue, Apr 14, 2015 at 6:39 PM, Matthew Brett <matthew.brett at gmail.com> wrote:
> Hi,
>
> On Tue, Apr 14, 2015 at 3:24 PM, Evgeni Burovski
> <evgeny.burovskiy at gmail.com> wrote:
>> What's the interpolation order?
>
> Cubic spline, linear, nearest neighbor in that order of frequency.
>
>> * Warren recently added nd functionality to splev:
>> https://github.com/scipy/scipy/pull/4059
>
> A quick scan suggested this allows an N-D array of x input values,
> whereas I want to evaluate a 1D array of x input values for 2- or N-D
> array of y values - is that right?

If you have the same 200 x values for all the y interpolation, then
you can reuse the projection matrix which is only 200x200. Similar
idea as in Savitsky-Golay.

It would be just one big dot product 200x200 dot 200x4000, after
creating the projection matrix

(not a how to use scipy answer)

Josef

>
>> * Have you tried polynomial interpolators? PPoly,
>> BPoly.from_derivatives might be what you are after
>
> Forgive my terrible ignorance, but is it easy to do cubic spline
> interpolation using these classes?
>
> Thanks for the reply,
>
> Matthew
> _______________________________________________
> SciPy-Dev mailing list
> SciPy-Dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/scipy-dev


From evgeny.burovskiy at gmail.com  Tue Apr 14 19:11:43 2015
From: evgeny.burovskiy at gmail.com (Evgeni Burovski)
Date: Wed, 15 Apr 2015 00:11:43 +0100
Subject: [SciPy-Dev] Mass 1D interpolation, advice
In-Reply-To: <CAH6Pt5pxgw3eJ8gYgN=QEvto=XAbGzmr8wW_xF1So2d75dhZzA@mail.gmail.com>
References: <CAH6Pt5q1-egmtz-VByaUi0gCgCU-1YYN1jXx7-ZbOy6Pp5Xc0A@mail.gmail.com>
	<CAMRo0ivN0Phav8ZG+OuWqXpb=BjjCdKvmuOK41cr0G4-4dDOYw@mail.gmail.com>
	<CAH6Pt5pxgw3eJ8gYgN=QEvto=XAbGzmr8wW_xF1So2d75dhZzA@mail.gmail.com>
Message-ID: <CAMRo0it1440xv--9=TQZu9TawYHzS2m53JOQSR4Wvi3o_SYdBQ@mail.gmail.com>

>
>> * Have you tried polynomial interpolators? PPoly,
>> BPoly.from_derivatives might be what you are after
>
> Forgive my terrible ignorance, but is it easy to do cubic spline
> interpolation using these classes?

If you are prepared to estimate the derivatives somehow, then yes.
Akima1DInterpolator and PchipInterpolator do it (pchip is a mess
because of the axis argument).

Otherwise, it's probably easier to have the b-splines PR merged than
to add a cython shim to fitpack fortran code or add the functionality
directly in fortran (which is also doable).


> Thanks for the reply,
>
> Matthew
> _______________________________________________
> SciPy-Dev mailing list
> SciPy-Dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/scipy-dev


From jjstickel at gmail.com  Wed Apr 15 10:33:17 2015
From: jjstickel at gmail.com (Jonathan Stickel)
Date: Wed, 15 Apr 2015 08:33:17 -0600
Subject: [SciPy-Dev] Mass 1D interpolation, advice
In-Reply-To: <CAH6Pt5q1-egmtz-VByaUi0gCgCU-1YYN1jXx7-ZbOy6Pp5Xc0A@mail.gmail.com>
References: <CAH6Pt5q1-egmtz-VByaUi0gCgCU-1YYN1jXx7-ZbOy6Pp5Xc0A@mail.gmail.com>
Message-ID: <552E76AD.6070600@gmail.com>

On 4/14/15 16:12 , Matthew Brett wrote:
> Hi,
>
> In brain imaging (my world) - we need to do 1D interpolation of large
> datasets, specifically, we have data in the order of (X, Y) = (200,
> 4000), where we are interpolating at the same ~200 X locations for
> each of the 4000 rows.
>
> We need to do this with extrapolation at the boundaries, so I think we
> can't use scipy.interpolate.interp1d which is otherwise well suited
> for the task.
>
> scipy.interpolate.InterpolatedUnivariateSpline does deal with
> extrapolation, but only works with vectors.
>
> I think, in order to make speed reasonable, I will need to loop over
> the rows in compiled code.
>
> My questions are:
>
> a) how easy would it be to try and link some external Cython code
> against the fitpack code in scipy?
> b) if that is not easy, what can I do that will also be of most use to
> other scipy users?
>
> Cheers,
>
> Matthew

I am not sure if I understand your problem statement. Most interpolation 
problems are posed as:  given data (x,y), determine yp at xp, where xp 
are different from x. This is fine for cases where you have a single y 
and possibly multiple sets xp. I had case awhile ago where I had a 
single xp and multiple y (at the same x). In this case, the problem 
statement is:  given known locations x and interpolant locations xp, 
what are the values for yp (corresponding to xp) given y (corresponding 
to xp). To solve this problem efficiently, I wrote the code I posted here:

http://scipy-central.org/item/21/1/simple-piecewise-polynomial-interpolation

Although it is not true spline interpolation (rather a simpler local 
polynomial interpolation), it is reasonably fast and accurate. 
Extrapolation is allowed. However, I later found that 
scipy.interpolate.splrep/splev are so fast that they could be used 
instead with equivalent speed for my problem. By comparison, 
scipy.interpolate.interp1d is quite slow.

HTH,
Jonathan


From josef.pktd at gmail.com  Wed Apr 15 11:04:40 2015
From: josef.pktd at gmail.com (josef.pktd at gmail.com)
Date: Wed, 15 Apr 2015 11:04:40 -0400
Subject: [SciPy-Dev] Mass 1D interpolation, advice
In-Reply-To: <552E76AD.6070600@gmail.com>
References: <CAH6Pt5q1-egmtz-VByaUi0gCgCU-1YYN1jXx7-ZbOy6Pp5Xc0A@mail.gmail.com>
	<552E76AD.6070600@gmail.com>
Message-ID: <CAMMTP+Cz8umOHOcahDz=EouiPMxSQJLNJUsMFY1doD69yFZyZQ@mail.gmail.com>

On Wed, Apr 15, 2015 at 10:33 AM, Jonathan Stickel <jjstickel at gmail.com> wrote:
> On 4/14/15 16:12 , Matthew Brett wrote:
>> Hi,
>>
>> In brain imaging (my world) - we need to do 1D interpolation of large
>> datasets, specifically, we have data in the order of (X, Y) = (200,
>> 4000), where we are interpolating at the same ~200 X locations for
>> each of the 4000 rows.
>>
>> We need to do this with extrapolation at the boundaries, so I think we
>> can't use scipy.interpolate.interp1d which is otherwise well suited
>> for the task.
>>
>> scipy.interpolate.InterpolatedUnivariateSpline does deal with
>> extrapolation, but only works with vectors.
>>
>> I think, in order to make speed reasonable, I will need to loop over
>> the rows in compiled code.
>>
>> My questions are:
>>
>> a) how easy would it be to try and link some external Cython code
>> against the fitpack code in scipy?
>> b) if that is not easy, what can I do that will also be of most use to
>> other scipy users?
>>
>> Cheers,
>>
>> Matthew
>
> I am not sure if I understand your problem statement. Most interpolation
> problems are posed as:  given data (x,y), determine yp at xp, where xp
> are different from x. This is fine for cases where you have a single y
> and possibly multiple sets xp. I had case awhile ago where I had a
> single xp and multiple y (at the same x). In this case, the problem
> statement is:  given known locations x and interpolant locations xp,
> what are the values for yp (corresponding to xp) given y (corresponding
> to xp). To solve this problem efficiently, I wrote the code I posted here:
>
> http://scipy-central.org/item/21/1/simple-piecewise-polynomial-interpolation

nice,

3 questions

Can you construct P = Xp*inv(X)*H  directly from the block diagonal
parts instead of going through sparse block-diagonal matrices?

Why do you need monotonically increasing x? I would have thought it
works for any x

(I'm not clear about this)

My guess is that replacing inv by pinv would be more stable. But,
would pinv also be able to handle the case when there are some
identical x (by implicitly taking an average)?

Josef

>
> Although it is not true spline interpolation (rather a simpler local
> polynomial interpolation), it is reasonably fast and accurate.
> Extrapolation is allowed. However, I later found that
> scipy.interpolate.splrep/splev are so fast that they could be used
> instead with equivalent speed for my problem. By comparison,
> scipy.interpolate.interp1d is quite slow.
>
> HTH,
> Jonathan
> _______________________________________________
> SciPy-Dev mailing list
> SciPy-Dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/scipy-dev


From josef.pktd at gmail.com  Wed Apr 15 11:24:19 2015
From: josef.pktd at gmail.com (josef.pktd at gmail.com)
Date: Wed, 15 Apr 2015 11:24:19 -0400
Subject: [SciPy-Dev] Mass 1D interpolation, advice
In-Reply-To: <CAMMTP+Cz8umOHOcahDz=EouiPMxSQJLNJUsMFY1doD69yFZyZQ@mail.gmail.com>
References: <CAH6Pt5q1-egmtz-VByaUi0gCgCU-1YYN1jXx7-ZbOy6Pp5Xc0A@mail.gmail.com>
	<552E76AD.6070600@gmail.com>
	<CAMMTP+Cz8umOHOcahDz=EouiPMxSQJLNJUsMFY1doD69yFZyZQ@mail.gmail.com>
Message-ID: <CAMMTP+BrOY+J55FAeg1Sb0Qk7DuCEGKK1D1rs520yj2=81W5dA@mail.gmail.com>

On Wed, Apr 15, 2015 at 11:04 AM,  <josef.pktd at gmail.com> wrote:
> On Wed, Apr 15, 2015 at 10:33 AM, Jonathan Stickel <jjstickel at gmail.com> wrote:
>> On 4/14/15 16:12 , Matthew Brett wrote:
>>> Hi,
>>>
>>> In brain imaging (my world) - we need to do 1D interpolation of large
>>> datasets, specifically, we have data in the order of (X, Y) = (200,
>>> 4000), where we are interpolating at the same ~200 X locations for
>>> each of the 4000 rows.
>>>
>>> We need to do this with extrapolation at the boundaries, so I think we
>>> can't use scipy.interpolate.interp1d which is otherwise well suited
>>> for the task.
>>>
>>> scipy.interpolate.InterpolatedUnivariateSpline does deal with
>>> extrapolation, but only works with vectors.
>>>
>>> I think, in order to make speed reasonable, I will need to loop over
>>> the rows in compiled code.
>>>
>>> My questions are:
>>>
>>> a) how easy would it be to try and link some external Cython code
>>> against the fitpack code in scipy?
>>> b) if that is not easy, what can I do that will also be of most use to
>>> other scipy users?
>>>
>>> Cheers,
>>>
>>> Matthew
>>
>> I am not sure if I understand your problem statement. Most interpolation
>> problems are posed as:  given data (x,y), determine yp at xp, where xp
>> are different from x. This is fine for cases where you have a single y
>> and possibly multiple sets xp. I had case awhile ago where I had a
>> single xp and multiple y (at the same x). In this case, the problem
>> statement is:  given known locations x and interpolant locations xp,
>> what are the values for yp (corresponding to xp) given y (corresponding
>> to xp). To solve this problem efficiently, I wrote the code I posted here:
>>
>> http://scipy-central.org/item/21/1/simple-piecewise-polynomial-interpolation
>
> nice,
>
> 3 questions
>
> Can you construct P = Xp*inv(X)*H  directly from the block diagonal
> parts instead of going through sparse block-diagonal matrices?
>
> Why do you need monotonically increasing x? I would have thought it
> works for any x
>
> (I'm not clear about this)
>
> My guess is that replacing inv by pinv would be more stable. But,
> would pinv also be able to handle the case when there are some
> identical x (by implicitly taking an average)?

Something is wrong in the way I think about this.
(I might be thinking about a different use case where I was recently
using Kernel Ridge Regression.)

Sorry for any noise

Josef

>
> Josef
>
>>
>> Although it is not true spline interpolation (rather a simpler local
>> polynomial interpolation), it is reasonably fast and accurate.
>> Extrapolation is allowed. However, I later found that
>> scipy.interpolate.splrep/splev are so fast that they could be used
>> instead with equivalent speed for my problem. By comparison,
>> scipy.interpolate.interp1d is quite slow.
>>
>> HTH,
>> Jonathan
>> _______________________________________________
>> SciPy-Dev mailing list
>> SciPy-Dev at scipy.org
>> http://mail.scipy.org/mailman/listinfo/scipy-dev


From josef.pktd at gmail.com  Wed Apr 15 11:28:20 2015
From: josef.pktd at gmail.com (josef.pktd at gmail.com)
Date: Wed, 15 Apr 2015 11:28:20 -0400
Subject: [SciPy-Dev] Mass 1D interpolation, advice
In-Reply-To: <CAMMTP+BrOY+J55FAeg1Sb0Qk7DuCEGKK1D1rs520yj2=81W5dA@mail.gmail.com>
References: <CAH6Pt5q1-egmtz-VByaUi0gCgCU-1YYN1jXx7-ZbOy6Pp5Xc0A@mail.gmail.com>
	<552E76AD.6070600@gmail.com>
	<CAMMTP+Cz8umOHOcahDz=EouiPMxSQJLNJUsMFY1doD69yFZyZQ@mail.gmail.com>
	<CAMMTP+BrOY+J55FAeg1Sb0Qk7DuCEGKK1D1rs520yj2=81W5dA@mail.gmail.com>
Message-ID: <CAMMTP+Ct5yJuvZ6FcoC1F56TU=JGjhmU_H73qm++k7OuJmTfUA@mail.gmail.com>

On Wed, Apr 15, 2015 at 11:24 AM,  <josef.pktd at gmail.com> wrote:
> On Wed, Apr 15, 2015 at 11:04 AM,  <josef.pktd at gmail.com> wrote:
>> On Wed, Apr 15, 2015 at 10:33 AM, Jonathan Stickel <jjstickel at gmail.com> wrote:
>>> On 4/14/15 16:12 , Matthew Brett wrote:
>>>> Hi,
>>>>
>>>> In brain imaging (my world) - we need to do 1D interpolation of large
>>>> datasets, specifically, we have data in the order of (X, Y) = (200,
>>>> 4000), where we are interpolating at the same ~200 X locations for
>>>> each of the 4000 rows.
>>>>
>>>> We need to do this with extrapolation at the boundaries, so I think we
>>>> can't use scipy.interpolate.interp1d which is otherwise well suited
>>>> for the task.
>>>>
>>>> scipy.interpolate.InterpolatedUnivariateSpline does deal with
>>>> extrapolation, but only works with vectors.
>>>>
>>>> I think, in order to make speed reasonable, I will need to loop over
>>>> the rows in compiled code.
>>>>
>>>> My questions are:
>>>>
>>>> a) how easy would it be to try and link some external Cython code
>>>> against the fitpack code in scipy?
>>>> b) if that is not easy, what can I do that will also be of most use to
>>>> other scipy users?
>>>>
>>>> Cheers,
>>>>
>>>> Matthew
>>>
>>> I am not sure if I understand your problem statement. Most interpolation
>>> problems are posed as:  given data (x,y), determine yp at xp, where xp
>>> are different from x. This is fine for cases where you have a single y
>>> and possibly multiple sets xp. I had case awhile ago where I had a
>>> single xp and multiple y (at the same x). In this case, the problem
>>> statement is:  given known locations x and interpolant locations xp,
>>> what are the values for yp (corresponding to xp) given y (corresponding
>>> to xp). To solve this problem efficiently, I wrote the code I posted here:
>>>
>>> http://scipy-central.org/item/21/1/simple-piecewise-polynomial-interpolation
>>
>> nice,
>>
>> 3 questions
>>
>> Can you construct P = Xp*inv(X)*H  directly from the block diagonal
>> parts instead of going through sparse block-diagonal matrices?
>>
>> Why do you need monotonically increasing x? I would have thought it
>> works for any x
>>
>> (I'm not clear about this)
>>
>> My guess is that replacing inv by pinv would be more stable. But,
>> would pinv also be able to handle the case when there are some
>> identical x (by implicitly taking an average)?
>
> Something is wrong in the way I think about this.
> (I might be thinking about a different use case where I was recently
> using Kernel Ridge Regression.)
>
> Sorry for any noise


The projection matrix for interpolation would need to have shape
(number of new points to be interpolated, number of sample points)

Josef

>
> Josef
>
>>
>> Josef
>>
>>>
>>> Although it is not true spline interpolation (rather a simpler local
>>> polynomial interpolation), it is reasonably fast and accurate.
>>> Extrapolation is allowed. However, I later found that
>>> scipy.interpolate.splrep/splev are so fast that they could be used
>>> instead with equivalent speed for my problem. By comparison,
>>> scipy.interpolate.interp1d is quite slow.
>>>
>>> HTH,
>>> Jonathan
>>> _______________________________________________
>>> SciPy-Dev mailing list
>>> SciPy-Dev at scipy.org
>>> http://mail.scipy.org/mailman/listinfo/scipy-dev


From jjstickel at gmail.com  Wed Apr 15 11:28:25 2015
From: jjstickel at gmail.com (Jonathan Stickel)
Date: Wed, 15 Apr 2015 09:28:25 -0600
Subject: [SciPy-Dev] Mass 1D interpolation, advice
In-Reply-To: <CAMMTP+BrOY+J55FAeg1Sb0Qk7DuCEGKK1D1rs520yj2=81W5dA@mail.gmail.com>
References: <CAH6Pt5q1-egmtz-VByaUi0gCgCU-1YYN1jXx7-ZbOy6Pp5Xc0A@mail.gmail.com>	<552E76AD.6070600@gmail.com>	<CAMMTP+Cz8umOHOcahDz=EouiPMxSQJLNJUsMFY1doD69yFZyZQ@mail.gmail.com>
	<CAMMTP+BrOY+J55FAeg1Sb0Qk7DuCEGKK1D1rs520yj2=81W5dA@mail.gmail.com>
Message-ID: <552E8399.1050901@gmail.com>

On 4/15/15 09:24 , josef.pktd at gmail.com wrote:
> On Wed, Apr 15, 2015 at 11:04 AM,  <josef.pktd at gmail.com> wrote:
>> On Wed, Apr 15, 2015 at 10:33 AM, Jonathan Stickel <jjstickel at gmail.com> wrote:
>>> On 4/14/15 16:12 , Matthew Brett wrote:
>>>> Hi,
>>>>
>>>> In brain imaging (my world) - we need to do 1D interpolation of large
>>>> datasets, specifically, we have data in the order of (X, Y) = (200,
>>>> 4000), where we are interpolating at the same ~200 X locations for
>>>> each of the 4000 rows.
>>>>
>>>> We need to do this with extrapolation at the boundaries, so I think we
>>>> can't use scipy.interpolate.interp1d which is otherwise well suited
>>>> for the task.
>>>>
>>>> scipy.interpolate.InterpolatedUnivariateSpline does deal with
>>>> extrapolation, but only works with vectors.
>>>>
>>>> I think, in order to make speed reasonable, I will need to loop over
>>>> the rows in compiled code.
>>>>
>>>> My questions are:
>>>>
>>>> a) how easy would it be to try and link some external Cython code
>>>> against the fitpack code in scipy?
>>>> b) if that is not easy, what can I do that will also be of most use to
>>>> other scipy users?
>>>>
>>>> Cheers,
>>>>
>>>> Matthew
>>>
>>> I am not sure if I understand your problem statement. Most interpolation
>>> problems are posed as:  given data (x,y), determine yp at xp, where xp
>>> are different from x. This is fine for cases where you have a single y
>>> and possibly multiple sets xp. I had case awhile ago where I had a
>>> single xp and multiple y (at the same x). In this case, the problem
>>> statement is:  given known locations x and interpolant locations xp,
>>> what are the values for yp (corresponding to xp) given y (corresponding
>>> to xp). To solve this problem efficiently, I wrote the code I posted here:
>>>
>>> http://scipy-central.org/item/21/1/simple-piecewise-polynomial-interpolation
>>
>> nice,
>>
>> 3 questions
>>
>> Can you construct P = Xp*inv(X)*H  directly from the block diagonal
>> parts instead of going through sparse block-diagonal matrices?
>>

Perhaps. It has been 4 years since I worked on this, and so I don't 
remember all the details. What I did was use a loop (via list 
comprehension for speed) to calculate the inverse of all the small block 
matrices, then constructed the sparse Xinv. There may very well be a 
more elegant approach.

>> Why do you need monotonically increasing x? I would have thought it
>> works for any x
>>
>> (I'm not clear about this)
>>

I don't remember why x needs to be monotonic. I am usually careful to 
consider these things, so I am sure that there were at least nontrivial 
implementation problems if x is scattered (or repeated). Or maybe I just 
assumed that it would be easy for users (me!) to sort the data fist and 
didn't want to take the time to worry about it.

>> My guess is that replacing inv by pinv would be more stable. But,
>> would pinv also be able to handle the case when there are some
>> identical x (by implicitly taking an average)?
>

I am not so knowledgeable here. By taking the direct inverse of the 
individual small block matrices, I think the inverse method I used is 
quite stable.

> Something is wrong in the way I think about this.
> (I might be thinking about a different use case where I was recently
> using Kernel Ridge Regression.)
>
> Sorry for any noise

No worries. Nice to see some interest.

Jonathan

>
> Josef
>
>>
>> Josef
>>
>>>
>>> Although it is not true spline interpolation (rather a simpler local
>>> polynomial interpolation), it is reasonably fast and accurate.
>>> Extrapolation is allowed. However, I later found that
>>> scipy.interpolate.splrep/splev are so fast that they could be used
>>> instead with equivalent speed for my problem. By comparison,
>>> scipy.interpolate.interp1d is quite slow.
>>>
>>> HTH,
>>> Jonathan
>>> _______________________________________________
>>> SciPy-Dev mailing list
>>> SciPy-Dev at scipy.org
>>> http://mail.scipy.org/mailman/listinfo/scipy-dev
> _______________________________________________
> SciPy-Dev mailing list
> SciPy-Dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/scipy-dev
>


From matthew.brett at gmail.com  Wed Apr 15 13:57:28 2015
From: matthew.brett at gmail.com (Matthew Brett)
Date: Wed, 15 Apr 2015 10:57:28 -0700
Subject: [SciPy-Dev] Mass 1D interpolation, advice
In-Reply-To: <552E76AD.6070600@gmail.com>
References: <CAH6Pt5q1-egmtz-VByaUi0gCgCU-1YYN1jXx7-ZbOy6Pp5Xc0A@mail.gmail.com>
	<552E76AD.6070600@gmail.com>
Message-ID: <CAH6Pt5rCUs4wjdZ8kwjFup_KLfJjF8eBDqZFUXL6Tt062u3tvw@mail.gmail.com>

Hi,

On Wed, Apr 15, 2015 at 7:33 AM, Jonathan Stickel <jjstickel at gmail.com> wrote:
> On 4/14/15 16:12 , Matthew Brett wrote:
>>
>> Hi,
>>
>> In brain imaging (my world) - we need to do 1D interpolation of large
>> datasets, specifically, we have data in the order of (X, Y) = (200,
>> 4000), where we are interpolating at the same ~200 X locations for
>> each of the 4000 rows.
>>
>> We need to do this with extrapolation at the boundaries, so I think we
>> can't use scipy.interpolate.interp1d which is otherwise well suited
>> for the task.
>>
>> scipy.interpolate.InterpolatedUnivariateSpline does deal with
>> extrapolation, but only works with vectors.
>>
>> I think, in order to make speed reasonable, I will need to loop over
>> the rows in compiled code.
>>
>> My questions are:
>>
>> a) how easy would it be to try and link some external Cython code
>> against the fitpack code in scipy?
>> b) if that is not easy, what can I do that will also be of most use to
>> other scipy users?
>>
>> Cheers,
>>
>> Matthew
>
>
> I am not sure if I understand your problem statement. Most interpolation
> problems are posed as:  given data (x,y), determine yp at xp, where xp are
> different from x. This is fine for cases where you have a single y and
> possibly multiple sets xp. I had case awhile ago where I had a single xp and
> multiple y (at the same x). In this case, the problem statement is:  given
> known locations x and interpolant locations xp, what are the values for yp
> (corresponding to xp) given y (corresponding to xp).

Sorry not to be clear; that is my case too.  (1D x, xp, ND y, yp).

> To solve this problem
> efficiently, I wrote the code I posted here:
>
> http://scipy-central.org/item/21/1/simple-piecewise-polynomial-interpolation

Thanks for the pointer.

> Although it is not true spline interpolation (rather a simpler local
> polynomial interpolation), it is reasonably fast and accurate. Extrapolation
> is allowed. However, I later found that scipy.interpolate.splrep/splev are
> so fast that they could be used instead with equivalent speed for my
> problem. By comparison, scipy.interpolate.interp1d is quite slow.

Just to check - you looping over vectors in y, and using splrep/splev
on each vector?

Cheers,

Matthew


From jjstickel at gmail.com  Wed Apr 15 14:41:52 2015
From: jjstickel at gmail.com (Jonathan Stickel)
Date: Wed, 15 Apr 2015 12:41:52 -0600
Subject: [SciPy-Dev] Mass 1D interpolation, advice
In-Reply-To: <CAH6Pt5rCUs4wjdZ8kwjFup_KLfJjF8eBDqZFUXL6Tt062u3tvw@mail.gmail.com>
References: <CAH6Pt5q1-egmtz-VByaUi0gCgCU-1YYN1jXx7-ZbOy6Pp5Xc0A@mail.gmail.com>
	<552E76AD.6070600@gmail.com>
	<CAH6Pt5rCUs4wjdZ8kwjFup_KLfJjF8eBDqZFUXL6Tt062u3tvw@mail.gmail.com>
Message-ID: <552EB0F0.3060100@gmail.com>



On 4/15/15 11:57 , Matthew Brett wrote:
> Hi,
>
> On Wed, Apr 15, 2015 at 7:33 AM, Jonathan Stickel <jjstickel at gmail.com> wrote:
>> On 4/14/15 16:12 , Matthew Brett wrote:
>>>
>>> Hi,
>>>
>>> In brain imaging (my world) - we need to do 1D interpolation of large
>>> datasets, specifically, we have data in the order of (X, Y) = (200,
>>> 4000), where we are interpolating at the same ~200 X locations for
>>> each of the 4000 rows.
>>>
>>> We need to do this with extrapolation at the boundaries, so I think we
>>> can't use scipy.interpolate.interp1d which is otherwise well suited
>>> for the task.
>>>
>>> scipy.interpolate.InterpolatedUnivariateSpline does deal with
>>> extrapolation, but only works with vectors.
>>>
>>> I think, in order to make speed reasonable, I will need to loop over
>>> the rows in compiled code.
>>>
>>> My questions are:
>>>
>>> a) how easy would it be to try and link some external Cython code
>>> against the fitpack code in scipy?
>>> b) if that is not easy, what can I do that will also be of most use to
>>> other scipy users?
>>>
>>> Cheers,
>>>
>>> Matthew
>>
>>
>> I am not sure if I understand your problem statement. Most interpolation
>> problems are posed as:  given data (x,y), determine yp at xp, where xp are
>> different from x. This is fine for cases where you have a single y and
>> possibly multiple sets xp. I had case awhile ago where I had a single xp and
>> multiple y (at the same x). In this case, the problem statement is:  given
>> known locations x and interpolant locations xp, what are the values for yp
>> (corresponding to xp) given y (corresponding to xp).
>
> Sorry not to be clear; that is my case too.  (1D x, xp, ND y, yp).
>
>> To solve this problem
>> efficiently, I wrote the code I posted here:
>>
>> http://scipy-central.org/item/21/1/simple-piecewise-polynomial-interpolation
>
> Thanks for the pointer.
>
>> Although it is not true spline interpolation (rather a simpler local
>> polynomial interpolation), it is reasonably fast and accurate. Extrapolation
>> is allowed. However, I later found that scipy.interpolate.splrep/splev are
>> so fast that they could be used instead with equivalent speed for my
>> problem. By comparison, scipy.interpolate.interp1d is quite slow.
>
> Just to check - you looping over vectors in y, and using splrep/splev
> on each vector?

Essentially, yes. My application was inside an system of ODEs, which was 
called many times by an ODE solver.

Jonathan



From evgeny.burovskiy at gmail.com  Wed Apr 15 16:45:19 2015
From: evgeny.burovskiy at gmail.com (Evgeni Burovski)
Date: Wed, 15 Apr 2015 21:45:19 +0100
Subject: [SciPy-Dev] Mass 1D interpolation, advice
In-Reply-To: <CAH6Pt5rCUs4wjdZ8kwjFup_KLfJjF8eBDqZFUXL6Tt062u3tvw@mail.gmail.com>
References: <CAH6Pt5q1-egmtz-VByaUi0gCgCU-1YYN1jXx7-ZbOy6Pp5Xc0A@mail.gmail.com>
	<552E76AD.6070600@gmail.com>
	<CAH6Pt5rCUs4wjdZ8kwjFup_KLfJjF8eBDqZFUXL6Tt062u3tvw@mail.gmail.com>
Message-ID: <CAMRo0iv5-MzaoRUka0Bvj8zdjURnMW3Vkiyjcx5BCW9uPhh6pA@mail.gmail.com>

On Wed, Apr 15, 2015 at 6:57 PM, Matthew Brett <matthew.brett at gmail.com> wrote:
> Hi,
>
> On Wed, Apr 15, 2015 at 7:33 AM, Jonathan Stickel <jjstickel at gmail.com> wrote:
>> On 4/14/15 16:12 , Matthew Brett wrote:
>>>
>>> Hi,
>>>
>>> In brain imaging (my world) - we need to do 1D interpolation of large
>>> datasets, specifically, we have data in the order of (X, Y) = (200,
>>> 4000), where we are interpolating at the same ~200 X locations for
>>> each of the 4000 rows.
>>>
>>> We need to do this with extrapolation at the boundaries, so I think we
>>> can't use scipy.interpolate.interp1d which is otherwise well suited
>>> for the task.
>>>
>>> scipy.interpolate.InterpolatedUnivariateSpline does deal with
>>> extrapolation, but only works with vectors.
>>>
>>> I think, in order to make speed reasonable, I will need to loop over
>>> the rows in compiled code.
>>>
>>> My questions are:
>>>
>>> a) how easy would it be to try and link some external Cython code
>>> against the fitpack code in scipy?
>>> b) if that is not easy, what can I do that will also be of most use to
>>> other scipy users?
>>>
>>> Cheers,
>>>
>>> Matthew
>>
>>
>> I am not sure if I understand your problem statement. Most interpolation
>> problems are posed as:  given data (x,y), determine yp at xp, where xp are
>> different from x. This is fine for cases where you have a single y and
>> possibly multiple sets xp. I had case awhile ago where I had a single xp and
>> multiple y (at the same x). In this case, the problem statement is:  given
>> known locations x and interpolant locations xp, what are the values for yp
>> (corresponding to xp) given y (corresponding to xp).
>
> Sorry not to be clear; that is my case too.  (1D x, xp, ND y, yp).

With https://github.com/scipy/scipy/pull/3174:

>>> from scipy.interpolate import BSpline, make_interp_spline
>>>
>>> n, k = 22, 3
>>> x = np.sort(np.random.random(n))
>>> y = np.random.random((n, 4, 5, 6))
>>>
>>> tck = make_interp_spline(x, y)
>>> tck[-1]
3
>>> spl = BSpline(*tck)
>>> spl.extrapolate
True
>>> xp = np.random.random((51, 3))
>>> spl(xp).shape
(51, 3, 4, 5, 6)

tck format is consistent with splev:

>>> cc = spl.c.reshape((22, -1))
>>>
>>> for j in range(cc.shape[1]):
...     tck_ = (spl.t, cc[:, j], spl.k)
...     y0 = splev(0.5, tck_)
...     y1 = BSpline(*tck_)(0.5)
...     assert_allclose(y0, y1)


From ralf.gommers at gmail.com  Thu Apr 16 02:05:14 2015
From: ralf.gommers at gmail.com (Ralf Gommers)
Date: Thu, 16 Apr 2015 08:05:14 +0200
Subject: [SciPy-Dev] Growth and sustainability of Scipy development
In-Reply-To: <CABL7CQh1ppnVisE4Y_qXjO1QUYKO=Wkvka_80RHrTr7jpASA8w@mail.gmail.com>
References: <CABL7CQj1a+M5U_hh5OB_tvucPPbOurKNPz6KZA_S=c5o3LATHg@mail.gmail.com>
	<CAH6Pt5rxxOdjJwfZLFjK_Qe3reCP9yhsTH1mUPqKD=q0Kka=7g@mail.gmail.com>
	<CABL7CQgAY8Re31H48F4xy6rBh8L6Brf1kY_KCP529dvqvwzCHA@mail.gmail.com>
	<CAH6Pt5o3mKr0mAb8NizNGVu-qNPNwWHJRfW64kr0_6V-xPrs3A@mail.gmail.com>
	<CABL7CQh1ppnVisE4Y_qXjO1QUYKO=Wkvka_80RHrTr7jpASA8w@mail.gmail.com>
Message-ID: <CABL7CQgqZfQt9Dv76XLOSUVX+jjJZhY_8A265XvEgemBDDdtew@mail.gmail.com>

On Mon, Apr 13, 2015 at 10:06 PM, Ralf Gommers <ralf.gommers at gmail.com>
wrote:

>
>
> On Sat, Apr 11, 2015 at 8:58 PM, Matthew Brett <matthew.brett at gmail.com>
> wrote:
>
>> Hi,
>>
>> On Sat, Apr 11, 2015 at 5:09 AM, Ralf Gommers <ralf.gommers at gmail.com>
>> wrote:
>> >
>>
>> >
>> > Probably a Google hangout at some time that's evening in Europe and
>> daytime
>> > in the US would work well. Here's a poll, please fill it out if you
>> want to
>> > participate: http://doodle.com/ibqnemrvuhkvexm2
>>
>> Done - thanks.
>>
>
> Poll dates are Fri 17 - Wed 22  April, so I'll close the poll on Wednesday
> and pick a time. Please fill it out by Wed morning if you want to
> participate.
>

Hi all, the chosen time slot is Wednesday 22nd, 7-8 pm UTC. I'll send out a
link to join the hangout earlier that day. If you haven't filled out the
poll but would like to join please let me know - necessary, because if >10
people I need to arrange something else than Google Hangout.

Cheers,
Ralf



> Ralf
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scipy-dev/attachments/20150416/908a0683/attachment.html>

From charlesr.harris at gmail.com  Thu Apr 16 17:02:00 2015
From: charlesr.harris at gmail.com (Charles R Harris)
Date: Thu, 16 Apr 2015 15:02:00 -0600
Subject: [SciPy-Dev] Growth and sustainability of Scipy development
In-Reply-To: <CABL7CQgqZfQt9Dv76XLOSUVX+jjJZhY_8A265XvEgemBDDdtew@mail.gmail.com>
References: <CABL7CQj1a+M5U_hh5OB_tvucPPbOurKNPz6KZA_S=c5o3LATHg@mail.gmail.com>
	<CAH6Pt5rxxOdjJwfZLFjK_Qe3reCP9yhsTH1mUPqKD=q0Kka=7g@mail.gmail.com>
	<CABL7CQgAY8Re31H48F4xy6rBh8L6Brf1kY_KCP529dvqvwzCHA@mail.gmail.com>
	<CAH6Pt5o3mKr0mAb8NizNGVu-qNPNwWHJRfW64kr0_6V-xPrs3A@mail.gmail.com>
	<CABL7CQh1ppnVisE4Y_qXjO1QUYKO=Wkvka_80RHrTr7jpASA8w@mail.gmail.com>
	<CABL7CQgqZfQt9Dv76XLOSUVX+jjJZhY_8A265XvEgemBDDdtew@mail.gmail.com>
Message-ID: <CAB6mnxJFrjNdz+FUcQfGOYriZBpg7ku0coytERzV9CevvnVPwg@mail.gmail.com>

On Thu, Apr 16, 2015 at 12:05 AM, Ralf Gommers <ralf.gommers at gmail.com>
wrote:

>
>
> On Mon, Apr 13, 2015 at 10:06 PM, Ralf Gommers <ralf.gommers at gmail.com>
> wrote:
>
>>
>>
>> On Sat, Apr 11, 2015 at 8:58 PM, Matthew Brett <matthew.brett at gmail.com>
>> wrote:
>>
>>> Hi,
>>>
>>> On Sat, Apr 11, 2015 at 5:09 AM, Ralf Gommers <ralf.gommers at gmail.com>
>>> wrote:
>>> >
>>>
>>> >
>>> > Probably a Google hangout at some time that's evening in Europe and
>>> daytime
>>> > in the US would work well. Here's a poll, please fill it out if you
>>> want to
>>> > participate: http://doodle.com/ibqnemrvuhkvexm2
>>>
>>> Done - thanks.
>>>
>>
>> Poll dates are Fri 17 - Wed 22  April, so I'll close the poll on
>> Wednesday and pick a time. Please fill it out by Wed morning if you want to
>> participate.
>>
>
> Hi all, the chosen time slot is Wednesday 22nd, 7-8 pm UTC. I'll send out
> a link to join the hangout earlier that day. If you haven't filled out the
> poll but would like to join please let me know - necessary, because if >10
> people I need to arrange something else than Google Hangout.
>

I'd like to at least listen in.

Chuck
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scipy-dev/attachments/20150416/5a5dc403/attachment.html>

From jamietmorton at gmail.com  Thu Apr 16 17:43:26 2015
From: jamietmorton at gmail.com (Jamie Morton)
Date: Thu, 16 Apr 2015 15:43:26 -0600
Subject: [SciPy-Dev] Growth and sustainability of Scipy development
In-Reply-To: <CABL7CQgqZfQt9Dv76XLOSUVX+jjJZhY_8A265XvEgemBDDdtew@mail.gmail.com>
References: <CABL7CQj1a+M5U_hh5OB_tvucPPbOurKNPz6KZA_S=c5o3LATHg@mail.gmail.com>
	<CAH6Pt5rxxOdjJwfZLFjK_Qe3reCP9yhsTH1mUPqKD=q0Kka=7g@mail.gmail.com>
	<CABL7CQgAY8Re31H48F4xy6rBh8L6Brf1kY_KCP529dvqvwzCHA@mail.gmail.com>
	<CAH6Pt5o3mKr0mAb8NizNGVu-qNPNwWHJRfW64kr0_6V-xPrs3A@mail.gmail.com>
	<CABL7CQh1ppnVisE4Y_qXjO1QUYKO=Wkvka_80RHrTr7jpASA8w@mail.gmail.com>
	<CABL7CQgqZfQt9Dv76XLOSUVX+jjJZhY_8A265XvEgemBDDdtew@mail.gmail.com>
Message-ID: <CAOF_XyK=6SopuVAoVS=i=7+YBBedcBkmo+4ZbVF704p3O4dsUw@mail.gmail.com>

Mind if I could also listen in?

Thanks!
Jamie

On Thu, Apr 16, 2015 at 12:05 AM, Ralf Gommers <ralf.gommers at gmail.com>
wrote:

>
>
> On Mon, Apr 13, 2015 at 10:06 PM, Ralf Gommers <ralf.gommers at gmail.com>
> wrote:
>
>>
>>
>> On Sat, Apr 11, 2015 at 8:58 PM, Matthew Brett <matthew.brett at gmail.com>
>> wrote:
>>
>>> Hi,
>>>
>>> On Sat, Apr 11, 2015 at 5:09 AM, Ralf Gommers <ralf.gommers at gmail.com>
>>> wrote:
>>> >
>>>
>>> >
>>> > Probably a Google hangout at some time that's evening in Europe and
>>> daytime
>>> > in the US would work well. Here's a poll, please fill it out if you
>>> want to
>>> > participate: http://doodle.com/ibqnemrvuhkvexm2
>>>
>>> Done - thanks.
>>>
>>
>> Poll dates are Fri 17 - Wed 22  April, so I'll close the poll on
>> Wednesday and pick a time. Please fill it out by Wed morning if you want to
>> participate.
>>
>
> Hi all, the chosen time slot is Wednesday 22nd, 7-8 pm UTC. I'll send out
> a link to join the hangout earlier that day. If you haven't filled out the
> poll but would like to join please let me know - necessary, because if >10
> people I need to arrange something else than Google Hangout.
>
> Cheers,
> Ralf
>
>
>
>> Ralf
>>
>
>
> _______________________________________________
> SciPy-Dev mailing list
> SciPy-Dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/scipy-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scipy-dev/attachments/20150416/3f091869/attachment.html>

From yoshiki89 at gmail.com  Thu Apr 16 17:45:04 2015
From: yoshiki89 at gmail.com (Yoshiki Vazquez-Baeza)
Date: Thu, 16 Apr 2015 14:45:04 -0700
Subject: [SciPy-Dev] Growth and sustainability of Scipy development
In-Reply-To: <CAOF_XyK=6SopuVAoVS=i=7+YBBedcBkmo+4ZbVF704p3O4dsUw@mail.gmail.com>
References: <CABL7CQj1a+M5U_hh5OB_tvucPPbOurKNPz6KZA_S=c5o3LATHg@mail.gmail.com>
	<CAH6Pt5rxxOdjJwfZLFjK_Qe3reCP9yhsTH1mUPqKD=q0Kka=7g@mail.gmail.com>
	<CABL7CQgAY8Re31H48F4xy6rBh8L6Brf1kY_KCP529dvqvwzCHA@mail.gmail.com>
	<CAH6Pt5o3mKr0mAb8NizNGVu-qNPNwWHJRfW64kr0_6V-xPrs3A@mail.gmail.com>
	<CABL7CQh1ppnVisE4Y_qXjO1QUYKO=Wkvka_80RHrTr7jpASA8w@mail.gmail.com>
	<CABL7CQgqZfQt9Dv76XLOSUVX+jjJZhY_8A265XvEgemBDDdtew@mail.gmail.com>
	<CAOF_XyK=6SopuVAoVS=i=7+YBBedcBkmo+4ZbVF704p3O4dsUw@mail.gmail.com>
Message-ID: <20150416214504.GD6847@debugatron-2000.local>

I too would like to listen in.

Thanks!

Yoshiki.

On (Apr-16-15|15:43), Jamie Morton wrote:
> Mind if I could also listen in?
> 
> Thanks!
> Jamie
> 
> On Thu, Apr 16, 2015 at 12:05 AM, Ralf Gommers <ralf.gommers at gmail.com>
> wrote:
> 
> >
> >
> > On Mon, Apr 13, 2015 at 10:06 PM, Ralf Gommers <ralf.gommers at gmail.com>
> > wrote:
> >
> >>
> >>
> >> On Sat, Apr 11, 2015 at 8:58 PM, Matthew Brett <matthew.brett at gmail.com>
> >> wrote:
> >>
> >>> Hi,
> >>>
> >>> On Sat, Apr 11, 2015 at 5:09 AM, Ralf Gommers <ralf.gommers at gmail.com>
> >>> wrote:
> >>> >
> >>>
> >>> >
> >>> > Probably a Google hangout at some time that's evening in Europe and
> >>> daytime
> >>> > in the US would work well. Here's a poll, please fill it out if you
> >>> want to
> >>> > participate: http://doodle.com/ibqnemrvuhkvexm2
> >>>
> >>> Done - thanks.
> >>>
> >>
> >> Poll dates are Fri 17 - Wed 22  April, so I'll close the poll on
> >> Wednesday and pick a time. Please fill it out by Wed morning if you want to
> >> participate.
> >>
> >
> > Hi all, the chosen time slot is Wednesday 22nd, 7-8 pm UTC. I'll send out
> > a link to join the hangout earlier that day. If you haven't filled out the
> > poll but would like to join please let me know - necessary, because if >10
> > people I need to arrange something else than Google Hangout.
> >
> > Cheers,
> > Ralf
> >
> >
> >
> >> Ralf
> >>
> >
> >
> > _______________________________________________
> > SciPy-Dev mailing list
> > SciPy-Dev at scipy.org
> > http://mail.scipy.org/mailman/listinfo/scipy-dev
> >
> >

> _______________________________________________
> SciPy-Dev mailing list
> SciPy-Dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/scipy-dev



From jf4martin at gmail.com  Sat Apr 18 13:43:09 2015
From: jf4martin at gmail.com (Jim Martin)
Date: Sat, 18 Apr 2015 13:43:09 -0400
Subject: [SciPy-Dev] Improvements (scipy.integrate._ode) -- accessing the
 dense output of DOPRI5 and DOP853.
Message-ID: <CAE8oJo5GsX2Z5SPqhFOOMcT-2Hjf_supMc6Yk1fa5QfLcOnstg@mail.gmail.com>

** Summary **

The wrappers around the differential equation solvers DOPRI5 and DOP853 in
"scipy/integrate/_ode.py" do not currently allow access to their "dense
output" option (iout=2).  This option provides the coefficients of an
interpolating polynomial for the solution between the last and current
integration step.  Dense output interpolation is commonly used for event
location and improved plotting.  The "_ode.py" code can be modified to
optionally allow access to the dense output of DOPRI5/DOP853.  The proposed
changes are here:

https://github.com/scipy/scipy/compare/master...jddmartin:dense_output_from_dopri5_and_dop853
with example usage here:
  https://github.com/jddmartin/dense_output_example_usage
Existing code using "_ode.py" would not be impacted by the changes.

** Background **

The efficient numerical solution of differential equations requires
adaptive stepsizes.  Modern high order Runge-Kutta code --- such as DOPRI5
and DOP853 --- can take large variable stepsizes, so that plots made simply
by connecting lines between steps are of poor visual quality.  As discussed
by [Shampine, 1997, see Section 5 and Figure 1, and Shampine et al., 2003,
pages 54, 55] this issue is addressed in the Matlab solvers by
interpolation between integration steps, with a default of 4 output
solutions provided per step (but can be modified using the "Refine"
option). Interpolation of the solution output between steps is also
desirable for event location (see discussion in Shampine et al., 2003,
Section 2.3.1).

The Fortran codes DOPRI5/DOP853 have a dense output option (iout=2), as
discussed in Hairer et al.'s book [Hairer, 1993].  This dense output option
is illustrated in the example usage code for these routines:  dr_dopri5.f
and dr_dop853.f (see http://www.unige.ch/~hairer/software.html ). However,
without implementation of the iout=2 option, these examples cannot be
replicated using scipy.  Some popular sources such as Press et al.'s
"Numerical Recipes" and Boost's odeint have solvers based on DOPRI5/DOP853
and provide access to dense output.

** Details **

The original DOPRI5/DOP853 codes provide the option for a user-specified
callback function to be called after each internal integration step.
Currently this callback function is only passed the solution at the current
step.  With the proposed change, the callback function will be provided
(optionally) with the coefficients of an interpolating polynomial for the
solution between the last and current step by using the "dense output"
option (iout=2).

As far as the user of scipy.integrate is concerned, the main change is the
addition of an optional "dense_components" keyword argument to the
".set_solout" method of classes "ode" and "complex_ode".  Behavior should
be unchanged in the absence of this keyword.  In addition, I have put in
some additional error handling code for the bug reported here:
  https://github.com/scipy/scipy/issues/4118
and made some minor changes (see git commit messages) that should not
impact existing code.

The new "dense_components" keyword argument is used to specify the
components for which dense output is desired.  If this keyword option is
present, the callback function will now be passed the additional
information required for interpolation.  A convenience function "dense_dop"
has been provided in _ode.py for interpolation using this information.

The introduction of the "dense_dop" function seems a bit awkward ---
suggestions for alternative approaches are welcome.  Most users will simply
want to evaluate the interpolation function to determine the solution at
points between the last and current step position.   My motivation for
development of access to the dense output is an application in which I will
use the coefficients of the interpolating polynomial directly, rather than
the interpolated solution.  Access to the interpolating polynomial may also
be useful for event location.

I've updated the relevant docstrings and added test code for this new
option (I have only tested this code on my Linux system with Python 2.7).

Comments are welcome.  Thanks -- Jim.

References:
[Shampine, 1997]: Lawrence F. Shampine and Mark W. Reichelt, The MATLAB ODE
Suite, SIAM J. Sci. Comput., 18(1), 1?22. 1997 (22 pages)
http://dx.doi.org/10.1137/S1064827594276424

[Shampine, 2003], L. F. Shampine, I. Gladwell, S. Thompson, "Solving ODEs
with Matlab", 2003, Cambridge University Press.

[Hairer, 1993] Hairer et al., "Solving Ordinary Differential Equations,
Nonstiff Problems",  Second Revised Edition, Springer, 1993.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scipy-dev/attachments/20150418/43f49294/attachment.html>

From ralf.gommers at gmail.com  Sun Apr 19 06:39:18 2015
From: ralf.gommers at gmail.com (Ralf Gommers)
Date: Sun, 19 Apr 2015 12:39:18 +0200
Subject: [SciPy-Dev] Growth and sustainability of Scipy development
In-Reply-To: <20150416214504.GD6847@debugatron-2000.local>
References: <CABL7CQj1a+M5U_hh5OB_tvucPPbOurKNPz6KZA_S=c5o3LATHg@mail.gmail.com>
	<CAH6Pt5rxxOdjJwfZLFjK_Qe3reCP9yhsTH1mUPqKD=q0Kka=7g@mail.gmail.com>
	<CABL7CQgAY8Re31H48F4xy6rBh8L6Brf1kY_KCP529dvqvwzCHA@mail.gmail.com>
	<CAH6Pt5o3mKr0mAb8NizNGVu-qNPNwWHJRfW64kr0_6V-xPrs3A@mail.gmail.com>
	<CABL7CQh1ppnVisE4Y_qXjO1QUYKO=Wkvka_80RHrTr7jpASA8w@mail.gmail.com>
	<CABL7CQgqZfQt9Dv76XLOSUVX+jjJZhY_8A265XvEgemBDDdtew@mail.gmail.com>
	<CAOF_XyK=6SopuVAoVS=i=7+YBBedcBkmo+4ZbVF704p3O4dsUw@mail.gmail.com>
	<20150416214504.GD6847@debugatron-2000.local>
Message-ID: <CABL7CQjmaspnT0mfbiQHkphR_GePZ0wrmo=bAW_1mWdyTCCPYg@mail.gmail.com>

On Thu, Apr 16, 2015 at 11:45 PM, Yoshiki Vazquez-Baeza <yoshiki89 at gmail.com
> wrote:

> I too would like to listen in.
>
> Thanks!
>
> Yoshiki.
>
> On (Apr-16-15|15:43), Jamie Morton wrote:
> > Mind if I could also listen in?
>

Sure. With both of you, Chuck and everyone who signed up we're at 12 people
now. Looks like we'll be using bluejeans.com, which should be accessible
for people on Linux as well.

Ralf


> >
> > Thanks!
> > Jamie
> >
> > On Thu, Apr 16, 2015 at 12:05 AM, Ralf Gommers <ralf.gommers at gmail.com>
> > wrote:
> >
> > >
> > >
> > > On Mon, Apr 13, 2015 at 10:06 PM, Ralf Gommers <ralf.gommers at gmail.com
> >
> > > wrote:
> > >
> > >>
> > >>
> > >> On Sat, Apr 11, 2015 at 8:58 PM, Matthew Brett <
> matthew.brett at gmail.com>
> > >> wrote:
> > >>
> > >>> Hi,
> > >>>
> > >>> On Sat, Apr 11, 2015 at 5:09 AM, Ralf Gommers <
> ralf.gommers at gmail.com>
> > >>> wrote:
> > >>> >
> > >>>
> > >>> >
> > >>> > Probably a Google hangout at some time that's evening in Europe and
> > >>> daytime
> > >>> > in the US would work well. Here's a poll, please fill it out if you
> > >>> want to
> > >>> > participate: http://doodle.com/ibqnemrvuhkvexm2
> > >>>
> > >>> Done - thanks.
> > >>>
> > >>
> > >> Poll dates are Fri 17 - Wed 22  April, so I'll close the poll on
> > >> Wednesday and pick a time. Please fill it out by Wed morning if you
> want to
> > >> participate.
> > >>
> > >
> > > Hi all, the chosen time slot is Wednesday 22nd, 7-8 pm UTC. I'll send
> out
> > > a link to join the hangout earlier that day. If you haven't filled out
> the
> > > poll but would like to join please let me know - necessary, because if
> >10
> > > people I need to arrange something else than Google Hangout.
> > >
> > > Cheers,
> > > Ralf
> > >
> > >
> > >
> > >> Ralf
> > >>
> > >
> > >
> > > _______________________________________________
> > > SciPy-Dev mailing list
> > > SciPy-Dev at scipy.org
> > > http://mail.scipy.org/mailman/listinfo/scipy-dev
> > >
> > >
>
> > _______________________________________________
> > SciPy-Dev mailing list
> > SciPy-Dev at scipy.org
> > http://mail.scipy.org/mailman/listinfo/scipy-dev
>
> _______________________________________________
> SciPy-Dev mailing list
> SciPy-Dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/scipy-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scipy-dev/attachments/20150419/51bd5a32/attachment.html>

From insertinterestingnamehere at gmail.com  Wed Apr 22 01:03:19 2015
From: insertinterestingnamehere at gmail.com (Ian Henriksen)
Date: Wed, 22 Apr 2015 05:03:19 +0000
Subject: [SciPy-Dev] Growth and sustainability of Scipy development
In-Reply-To: <CABL7CQjmaspnT0mfbiQHkphR_GePZ0wrmo=bAW_1mWdyTCCPYg@mail.gmail.com>
References: <CABL7CQj1a+M5U_hh5OB_tvucPPbOurKNPz6KZA_S=c5o3LATHg@mail.gmail.com>
	<CAH6Pt5rxxOdjJwfZLFjK_Qe3reCP9yhsTH1mUPqKD=q0Kka=7g@mail.gmail.com>
	<CABL7CQgAY8Re31H48F4xy6rBh8L6Brf1kY_KCP529dvqvwzCHA@mail.gmail.com>
	<CAH6Pt5o3mKr0mAb8NizNGVu-qNPNwWHJRfW64kr0_6V-xPrs3A@mail.gmail.com>
	<CABL7CQh1ppnVisE4Y_qXjO1QUYKO=Wkvka_80RHrTr7jpASA8w@mail.gmail.com>
	<CABL7CQgqZfQt9Dv76XLOSUVX+jjJZhY_8A265XvEgemBDDdtew@mail.gmail.com>
	<CAOF_XyK=6SopuVAoVS=i=7+YBBedcBkmo+4ZbVF704p3O4dsUw@mail.gmail.com>
	<20150416214504.GD6847@debugatron-2000.local>
	<CABL7CQjmaspnT0mfbiQHkphR_GePZ0wrmo=bAW_1mWdyTCCPYg@mail.gmail.com>
Message-ID: <CAPWbfpQpwxSMKTXAaPqAFc90Cno9SwhG-2cwwBxZZd8B=FM2tA@mail.gmail.com>

On Sun, Apr 19, 2015 at 4:39 AM Ralf Gommers <ralf.gommers at gmail.com> wrote:

> On Thu, Apr 16, 2015 at 11:45 PM, Yoshiki Vazquez-Baeza <
> yoshiki89 at gmail.com> wrote:
>
>> I too would like to listen in.
>>
>> Thanks!
>>
>> Yoshiki.
>>
>> On (Apr-16-15|15:43), Jamie Morton wrote:
>> > Mind if I could also listen in?
>>
>
> Sure. With both of you, Chuck and everyone who signed up we're at 12
> people now. Looks like we'll be using bluejeans.com, which should be
> accessible for people on Linux as well.
>
> Ralf
>
>
>> >
>> > Thanks!
>> > Jamie
>> >
>> > On Thu, Apr 16, 2015 at 12:05 AM, Ralf Gommers <ralf.gommers at gmail.com>
>> > wrote:
>> >
>> > >
>> > >
>> > > On Mon, Apr 13, 2015 at 10:06 PM, Ralf Gommers <
>> ralf.gommers at gmail.com>
>> > > wrote:
>> > >
>> > >>
>> > >>
>> > >> On Sat, Apr 11, 2015 at 8:58 PM, Matthew Brett <
>> matthew.brett at gmail.com>
>> > >> wrote:
>> > >>
>> > >>> Hi,
>> > >>>
>> > >>> On Sat, Apr 11, 2015 at 5:09 AM, Ralf Gommers <
>> ralf.gommers at gmail.com>
>> > >>> wrote:
>> > >>> >
>> > >>>
>> > >>> >
>> > >>> > Probably a Google hangout at some time that's evening in Europe
>> and
>> > >>> daytime
>> > >>> > in the US would work well. Here's a poll, please fill it out if
>> you
>> > >>> want to
>> > >>> > participate: http://doodle.com/ibqnemrvuhkvexm2
>> > >>>
>> > >>> Done - thanks.
>> > >>>
>> > >>
>> > >> Poll dates are Fri 17 - Wed 22  April, so I'll close the poll on
>> > >> Wednesday and pick a time. Please fill it out by Wed morning if you
>> want to
>> > >> participate.
>> > >>
>> > >
>> > > Hi all, the chosen time slot is Wednesday 22nd, 7-8 pm UTC. I'll send
>> out
>> > > a link to join the hangout earlier that day. If you haven't filled
>> out the
>> > > poll but would like to join please let me know - necessary, because
>> if >10
>> > > people I need to arrange something else than Google Hangout.
>> > >
>> > > Cheers,
>> > > Ralf
>> > >
>> > >
>> > >
>> > >> Ralf
>> > >>
>> > >
>> > >
>> > > _______________________________________________
>> > > SciPy-Dev mailing list
>> > > SciPy-Dev at scipy.org
>> > > http://mail.scipy.org/mailman/listinfo/scipy-dev
>> > >
>> > >
>>
>> > _______________________________________________
>> > SciPy-Dev mailing list
>> > SciPy-Dev at scipy.org
>> > http://mail.scipy.org/mailman/listinfo/scipy-dev
>>
>> _______________________________________________
>> SciPy-Dev mailing list
>> SciPy-Dev at scipy.org
>> http://mail.scipy.org/mailman/listinfo/scipy-dev
>>
> _______________________________________________
> SciPy-Dev mailing list
> SciPy-Dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/scipy-dev


Mind if I join in too?
Thanks!
-Ian
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scipy-dev/attachments/20150422/0dffd061/attachment.html>

From cournape at gmail.com  Wed Apr 22 06:50:50 2015
From: cournape at gmail.com (David Cournapeau)
Date: Wed, 22 Apr 2015 11:50:50 +0100
Subject: [SciPy-Dev] Growth and sustainability of Scipy development
In-Reply-To: <CABL7CQjmaspnT0mfbiQHkphR_GePZ0wrmo=bAW_1mWdyTCCPYg@mail.gmail.com>
References: <CABL7CQj1a+M5U_hh5OB_tvucPPbOurKNPz6KZA_S=c5o3LATHg@mail.gmail.com>
	<CAH6Pt5rxxOdjJwfZLFjK_Qe3reCP9yhsTH1mUPqKD=q0Kka=7g@mail.gmail.com>
	<CABL7CQgAY8Re31H48F4xy6rBh8L6Brf1kY_KCP529dvqvwzCHA@mail.gmail.com>
	<CAH6Pt5o3mKr0mAb8NizNGVu-qNPNwWHJRfW64kr0_6V-xPrs3A@mail.gmail.com>
	<CABL7CQh1ppnVisE4Y_qXjO1QUYKO=Wkvka_80RHrTr7jpASA8w@mail.gmail.com>
	<CABL7CQgqZfQt9Dv76XLOSUVX+jjJZhY_8A265XvEgemBDDdtew@mail.gmail.com>
	<CAOF_XyK=6SopuVAoVS=i=7+YBBedcBkmo+4ZbVF704p3O4dsUw@mail.gmail.com>
	<20150416214504.GD6847@debugatron-2000.local>
	<CABL7CQjmaspnT0mfbiQHkphR_GePZ0wrmo=bAW_1mWdyTCCPYg@mail.gmail.com>
Message-ID: <CAGY4rcWfMYmfMX3iqNoKSNBaT_zP1wDTFJ3T3NmSdvUQ4U9otA@mail.gmail.com>

Is there a link we should be aware of for bluejeans ? Just making sure I
have not missed the connection info.

Thanks for setting this up Ralf,

David

On Sun, Apr 19, 2015 at 11:39 AM, Ralf Gommers <ralf.gommers at gmail.com>
wrote:

>
>
> On Thu, Apr 16, 2015 at 11:45 PM, Yoshiki Vazquez-Baeza <
> yoshiki89 at gmail.com> wrote:
>
>> I too would like to listen in.
>>
>> Thanks!
>>
>> Yoshiki.
>>
>> On (Apr-16-15|15:43), Jamie Morton wrote:
>> > Mind if I could also listen in?
>>
>
> Sure. With both of you, Chuck and everyone who signed up we're at 12
> people now. Looks like we'll be using bluejeans.com, which should be
> accessible for people on Linux as well.
>
> Ralf
>
>
>> >
>> > Thanks!
>> > Jamie
>> >
>> > On Thu, Apr 16, 2015 at 12:05 AM, Ralf Gommers <ralf.gommers at gmail.com>
>> > wrote:
>> >
>> > >
>> > >
>> > > On Mon, Apr 13, 2015 at 10:06 PM, Ralf Gommers <
>> ralf.gommers at gmail.com>
>> > > wrote:
>> > >
>> > >>
>> > >>
>> > >> On Sat, Apr 11, 2015 at 8:58 PM, Matthew Brett <
>> matthew.brett at gmail.com>
>> > >> wrote:
>> > >>
>> > >>> Hi,
>> > >>>
>> > >>> On Sat, Apr 11, 2015 at 5:09 AM, Ralf Gommers <
>> ralf.gommers at gmail.com>
>> > >>> wrote:
>> > >>> >
>> > >>>
>> > >>> >
>> > >>> > Probably a Google hangout at some time that's evening in Europe
>> and
>> > >>> daytime
>> > >>> > in the US would work well. Here's a poll, please fill it out if
>> you
>> > >>> want to
>> > >>> > participate: http://doodle.com/ibqnemrvuhkvexm2
>> > >>>
>> > >>> Done - thanks.
>> > >>>
>> > >>
>> > >> Poll dates are Fri 17 - Wed 22  April, so I'll close the poll on
>> > >> Wednesday and pick a time. Please fill it out by Wed morning if you
>> want to
>> > >> participate.
>> > >>
>> > >
>> > > Hi all, the chosen time slot is Wednesday 22nd, 7-8 pm UTC. I'll send
>> out
>> > > a link to join the hangout earlier that day. If you haven't filled
>> out the
>> > > poll but would like to join please let me know - necessary, because
>> if >10
>> > > people I need to arrange something else than Google Hangout.
>> > >
>> > > Cheers,
>> > > Ralf
>> > >
>> > >
>> > >
>> > >> Ralf
>> > >>
>> > >
>> > >
>> > > _______________________________________________
>> > > SciPy-Dev mailing list
>> > > SciPy-Dev at scipy.org
>> > > http://mail.scipy.org/mailman/listinfo/scipy-dev
>> > >
>> > >
>>
>> > _______________________________________________
>> > SciPy-Dev mailing list
>> > SciPy-Dev at scipy.org
>> > http://mail.scipy.org/mailman/listinfo/scipy-dev
>>
>> _______________________________________________
>> SciPy-Dev mailing list
>> SciPy-Dev at scipy.org
>> http://mail.scipy.org/mailman/listinfo/scipy-dev
>>
>
>
> _______________________________________________
> SciPy-Dev mailing list
> SciPy-Dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/scipy-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scipy-dev/attachments/20150422/fdbcdc77/attachment.html>

From ralf.gommers at gmail.com  Wed Apr 22 12:10:39 2015
From: ralf.gommers at gmail.com (Ralf Gommers)
Date: Wed, 22 Apr 2015 18:10:39 +0200
Subject: [SciPy-Dev] Growth and sustainability of Scipy development
In-Reply-To: <CAPWbfpQpwxSMKTXAaPqAFc90Cno9SwhG-2cwwBxZZd8B=FM2tA@mail.gmail.com>
References: <CABL7CQj1a+M5U_hh5OB_tvucPPbOurKNPz6KZA_S=c5o3LATHg@mail.gmail.com>
	<CAH6Pt5rxxOdjJwfZLFjK_Qe3reCP9yhsTH1mUPqKD=q0Kka=7g@mail.gmail.com>
	<CABL7CQgAY8Re31H48F4xy6rBh8L6Brf1kY_KCP529dvqvwzCHA@mail.gmail.com>
	<CAH6Pt5o3mKr0mAb8NizNGVu-qNPNwWHJRfW64kr0_6V-xPrs3A@mail.gmail.com>
	<CABL7CQh1ppnVisE4Y_qXjO1QUYKO=Wkvka_80RHrTr7jpASA8w@mail.gmail.com>
	<CABL7CQgqZfQt9Dv76XLOSUVX+jjJZhY_8A265XvEgemBDDdtew@mail.gmail.com>
	<CAOF_XyK=6SopuVAoVS=i=7+YBBedcBkmo+4ZbVF704p3O4dsUw@mail.gmail.com>
	<20150416214504.GD6847@debugatron-2000.local>
	<CABL7CQjmaspnT0mfbiQHkphR_GePZ0wrmo=bAW_1mWdyTCCPYg@mail.gmail.com>
	<CAPWbfpQpwxSMKTXAaPqAFc90Cno9SwhG-2cwwBxZZd8B=FM2tA@mail.gmail.com>
Message-ID: <CABL7CQgi8nBfWPmsCicMGhyWWcosVYPqKuPad2bVajeX1b3ciQ@mail.gmail.com>

On Wed, Apr 22, 2015 at 7:03 AM, Ian Henriksen <
insertinterestingnamehere at gmail.com> wrote:

> On Sun, Apr 19, 2015 at 4:39 AM Ralf Gommers <ralf.gommers at gmail.com>
> wrote:
>
>> On Thu, Apr 16, 2015 at 11:45 PM, Yoshiki Vazquez-Baeza <
>> yoshiki89 at gmail.com> wrote:
>>
>>> I too would like to listen in.
>>>
>>> Thanks!
>>>
>>> Yoshiki.
>>>
>>> On (Apr-16-15|15:43), Jamie Morton wrote:
>>> > Mind if I could also listen in?
>>>
>>
>> Sure. With both of you, Chuck and everyone who signed up we're at 12
>> people now. Looks like we'll be using bluejeans.com, which should be
>> accessible for people on Linux as well.
>>
>> Ralf
>>
>>
>>> >
>>> > Thanks!
>>> > Jamie
>>> >
>>> > On Thu, Apr 16, 2015 at 12:05 AM, Ralf Gommers <ralf.gommers at gmail.com
>>> >
>>> > wrote:
>>> >
>>> > >
>>> > >
>>> > > On Mon, Apr 13, 2015 at 10:06 PM, Ralf Gommers <
>>> ralf.gommers at gmail.com>
>>> > > wrote:
>>> > >
>>> > >>
>>> > >>
>>> > >> On Sat, Apr 11, 2015 at 8:58 PM, Matthew Brett <
>>> matthew.brett at gmail.com>
>>> > >> wrote:
>>> > >>
>>> > >>> Hi,
>>> > >>>
>>> > >>> On Sat, Apr 11, 2015 at 5:09 AM, Ralf Gommers <
>>> ralf.gommers at gmail.com>
>>> > >>> wrote:
>>> > >>> >
>>> > >>>
>>> > >>> >
>>> > >>> > Probably a Google hangout at some time that's evening in Europe
>>> and
>>> > >>> daytime
>>> > >>> > in the US would work well. Here's a poll, please fill it out if
>>> you
>>> > >>> want to
>>> > >>> > participate: http://doodle.com/ibqnemrvuhkvexm2
>>> > >>>
>>> > >>> Done - thanks.
>>> > >>>
>>> > >>
>>> > >> Poll dates are Fri 17 - Wed 22  April, so I'll close the poll on
>>> > >> Wednesday and pick a time. Please fill it out by Wed morning if you
>>> want to
>>> > >> participate.
>>> > >>
>>> > >
>>> > > Hi all, the chosen time slot is Wednesday 22nd, 7-8 pm UTC. I'll
>>> send out
>>> > > a link to join the hangout earlier that day. If you haven't filled
>>> out the
>>> > > poll but would like to join please let me know - necessary, because
>>> if >10
>>> > > people I need to arrange something else than Google Hangout.
>>> > >
>>> > > Cheers,
>>> > > Ralf
>>> > >
>>> > >
>>> > >
>>> > >> Ralf
>>> > >>
>>> > >
>>> > >
>>> > > _______________________________________________
>>> > > SciPy-Dev mailing list
>>> > > SciPy-Dev at scipy.org
>>> > > http://mail.scipy.org/mailman/listinfo/scipy-dev
>>> > >
>>> > >
>>>
>>> > _______________________________________________
>>> > SciPy-Dev mailing list
>>> > SciPy-Dev at scipy.org
>>> > http://mail.scipy.org/mailman/listinfo/scipy-dev
>>>
>>> _______________________________________________
>>> SciPy-Dev mailing list
>>> SciPy-Dev at scipy.org
>>> http://mail.scipy.org/mailman/listinfo/scipy-dev
>>>
>> _______________________________________________
>> SciPy-Dev mailing list
>> SciPy-Dev at scipy.org
>> http://mail.scipy.org/mailman/listinfo/scipy-dev
>
>
> Mind if I join in too?
> Thanks!
> -Ian
>

Don't mind at all of course.

We're at 14 people now if I counted right. Will send the links to the
meeting itself and the meeting minutes in a bit.

Ralf
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scipy-dev/attachments/20150422/42ed6395/attachment.html>

From ralf.gommers at gmail.com  Wed Apr 22 12:33:26 2015
From: ralf.gommers at gmail.com (Ralf Gommers)
Date: Wed, 22 Apr 2015 18:33:26 +0200
Subject: [SciPy-Dev] Growth and sustainability of Scipy development
In-Reply-To: <CAGY4rcWfMYmfMX3iqNoKSNBaT_zP1wDTFJ3T3NmSdvUQ4U9otA@mail.gmail.com>
References: <CABL7CQj1a+M5U_hh5OB_tvucPPbOurKNPz6KZA_S=c5o3LATHg@mail.gmail.com>
	<CAH6Pt5rxxOdjJwfZLFjK_Qe3reCP9yhsTH1mUPqKD=q0Kka=7g@mail.gmail.com>
	<CABL7CQgAY8Re31H48F4xy6rBh8L6Brf1kY_KCP529dvqvwzCHA@mail.gmail.com>
	<CAH6Pt5o3mKr0mAb8NizNGVu-qNPNwWHJRfW64kr0_6V-xPrs3A@mail.gmail.com>
	<CABL7CQh1ppnVisE4Y_qXjO1QUYKO=Wkvka_80RHrTr7jpASA8w@mail.gmail.com>
	<CABL7CQgqZfQt9Dv76XLOSUVX+jjJZhY_8A265XvEgemBDDdtew@mail.gmail.com>
	<CAOF_XyK=6SopuVAoVS=i=7+YBBedcBkmo+4ZbVF704p3O4dsUw@mail.gmail.com>
	<20150416214504.GD6847@debugatron-2000.local>
	<CABL7CQjmaspnT0mfbiQHkphR_GePZ0wrmo=bAW_1mWdyTCCPYg@mail.gmail.com>
	<CAGY4rcWfMYmfMX3iqNoKSNBaT_zP1wDTFJ3T3NmSdvUQ4U9otA@mail.gmail.com>
Message-ID: <CABL7CQgYB-_L_Kmp+hCDE_mr6z+f0maO3r6ijKZ2oVQaK-01cA@mail.gmail.com>

On Wed, Apr 22, 2015 at 12:50 PM, David Cournapeau <cournape at gmail.com>
wrote:

> Is there a link we should be aware of for bluejeans ? Just making sure I
> have not missed the connection info.
>

Good question. I didn't send it out yet, wanted to do a bit of testing
first (conclusion: should work on WIndows, OS X and 64-bit Linux but not on
32-bit Linux).

Here is the link to the meeting: https://bluejeans.com/865079980. That
should be all you need (no password), but in case it asks you for a meeting
ID that is 865079980.

The meeting minutes are at
https://drive.google.com/open?id=0Bw--OwzvZKRhREVaU1RXNXB6blU&authuser=0,
everyone should be able to edit them. I tried to get the synchronization
between StackEdit and Google Drive to work, but that seems to be a bit of a
mess. So editing live in Google Docs it is - we'll clean up to publish on
the mailing list and as rendered Markdown afterwards.

Reminder about time zones (this always goes wrong):
UTC 7-8 pm
Local time Western Europe: 9-10 pm
CDT: 2-3 pm
time from now: 2.5 hours

Cheers,
Ralf
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scipy-dev/attachments/20150422/6b19514f/attachment.html>

From ralf.gommers at gmail.com  Wed Apr 22 12:43:08 2015
From: ralf.gommers at gmail.com (Ralf Gommers)
Date: Wed, 22 Apr 2015 18:43:08 +0200
Subject: [SciPy-Dev] Growth and sustainability of Scipy development
In-Reply-To: <CABL7CQgYB-_L_Kmp+hCDE_mr6z+f0maO3r6ijKZ2oVQaK-01cA@mail.gmail.com>
References: <CABL7CQj1a+M5U_hh5OB_tvucPPbOurKNPz6KZA_S=c5o3LATHg@mail.gmail.com>
	<CAH6Pt5rxxOdjJwfZLFjK_Qe3reCP9yhsTH1mUPqKD=q0Kka=7g@mail.gmail.com>
	<CABL7CQgAY8Re31H48F4xy6rBh8L6Brf1kY_KCP529dvqvwzCHA@mail.gmail.com>
	<CAH6Pt5o3mKr0mAb8NizNGVu-qNPNwWHJRfW64kr0_6V-xPrs3A@mail.gmail.com>
	<CABL7CQh1ppnVisE4Y_qXjO1QUYKO=Wkvka_80RHrTr7jpASA8w@mail.gmail.com>
	<CABL7CQgqZfQt9Dv76XLOSUVX+jjJZhY_8A265XvEgemBDDdtew@mail.gmail.com>
	<CAOF_XyK=6SopuVAoVS=i=7+YBBedcBkmo+4ZbVF704p3O4dsUw@mail.gmail.com>
	<20150416214504.GD6847@debugatron-2000.local>
	<CABL7CQjmaspnT0mfbiQHkphR_GePZ0wrmo=bAW_1mWdyTCCPYg@mail.gmail.com>
	<CAGY4rcWfMYmfMX3iqNoKSNBaT_zP1wDTFJ3T3NmSdvUQ4U9otA@mail.gmail.com>
	<CABL7CQgYB-_L_Kmp+hCDE_mr6z+f0maO3r6ijKZ2oVQaK-01cA@mail.gmail.com>
Message-ID: <CABL7CQhXYbc0JxHTLPP2jBNNd2N8wVReCMzGE672-=fDubGJFA@mail.gmail.com>

On Wed, Apr 22, 2015 at 6:33 PM, Ralf Gommers <ralf.gommers at gmail.com>
wrote:

>
>
> On Wed, Apr 22, 2015 at 12:50 PM, David Cournapeau <cournape at gmail.com>
> wrote:
>
>> Is there a link we should be aware of for bluejeans ? Just making sure I
>> have not missed the connection info.
>>
>
> Good question. I didn't send it out yet, wanted to do a bit of testing
> first (conclusion: should work on WIndows, OS X and 64-bit Linux but not on
> 32-bit Linux).
>
> Here is the link to the meeting: https://bluejeans.com/865079980. That
> should be all you need (no password), but in case it asks you for a meeting
> ID that is 865079980.
>
> The meeting minutes are at, everyone should be able to edit them. I tried
> to get the synchronization between StackEdit and Google Drive to work, but
> that seems to be a bit of a mess. So editing live in Google Docs it is -
> we'll clean up to publish on the mailing list and as rendered Markdown
> afterwards.
>

EDIT: this is the link that should work:
https://drive.google.com/open?id=1vsLrXIAVGwkHb2hdscuWu9ZXik-3NH80X8Oh0eTy0cM&authuser=0


>
> Reminder about time zones (this always goes wrong):
> UTC 7-8 pm
> Local time Western Europe: 9-10 pm
> CDT: 2-3 pm
> time from now: 2.5 hours
>
> Cheers,
> Ralf
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scipy-dev/attachments/20150422/dc3719f9/attachment.html>

From ralf.gommers at gmail.com  Thu Apr 23 02:12:04 2015
From: ralf.gommers at gmail.com (Ralf Gommers)
Date: Thu, 23 Apr 2015 08:12:04 +0200
Subject: [SciPy-Dev] Growth and sustainability of Scipy development
In-Reply-To: <CABL7CQhXYbc0JxHTLPP2jBNNd2N8wVReCMzGE672-=fDubGJFA@mail.gmail.com>
References: <CABL7CQj1a+M5U_hh5OB_tvucPPbOurKNPz6KZA_S=c5o3LATHg@mail.gmail.com>
	<CAH6Pt5rxxOdjJwfZLFjK_Qe3reCP9yhsTH1mUPqKD=q0Kka=7g@mail.gmail.com>
	<CABL7CQgAY8Re31H48F4xy6rBh8L6Brf1kY_KCP529dvqvwzCHA@mail.gmail.com>
	<CAH6Pt5o3mKr0mAb8NizNGVu-qNPNwWHJRfW64kr0_6V-xPrs3A@mail.gmail.com>
	<CABL7CQh1ppnVisE4Y_qXjO1QUYKO=Wkvka_80RHrTr7jpASA8w@mail.gmail.com>
	<CABL7CQgqZfQt9Dv76XLOSUVX+jjJZhY_8A265XvEgemBDDdtew@mail.gmail.com>
	<CAOF_XyK=6SopuVAoVS=i=7+YBBedcBkmo+4ZbVF704p3O4dsUw@mail.gmail.com>
	<20150416214504.GD6847@debugatron-2000.local>
	<CABL7CQjmaspnT0mfbiQHkphR_GePZ0wrmo=bAW_1mWdyTCCPYg@mail.gmail.com>
	<CAGY4rcWfMYmfMX3iqNoKSNBaT_zP1wDTFJ3T3NmSdvUQ4U9otA@mail.gmail.com>
	<CABL7CQgYB-_L_Kmp+hCDE_mr6z+f0maO3r6ijKZ2oVQaK-01cA@mail.gmail.com>
	<CABL7CQhXYbc0JxHTLPP2jBNNd2N8wVReCMzGE672-=fDubGJFA@mail.gmail.com>
Message-ID: <CABL7CQhPM63D-4-mtcgcxXzvHLOX6FDNrR7+ORRMD9f_OOm=Xg@mail.gmail.com>

On Wed, Apr 22, 2015 at 6:43 PM, Ralf Gommers <ralf.gommers at gmail.com>
wrote:

>
>
> On Wed, Apr 22, 2015 at 6:33 PM, Ralf Gommers <ralf.gommers at gmail.com>
> wrote:
>
>>
>>
>> On Wed, Apr 22, 2015 at 12:50 PM, David Cournapeau <cournape at gmail.com>
>> wrote:
>>
>>> Is there a link we should be aware of for bluejeans ? Just making sure I
>>> have not missed the connection info.
>>>
>>
>> Good question. I didn't send it out yet, wanted to do a bit of testing
>> first (conclusion: should work on WIndows, OS X and 64-bit Linux but not on
>> 32-bit Linux).
>>
>> Here is the link to the meeting: https://bluejeans.com/865079980. That
>> should be all you need (no password), but in case it asks you for a meeting
>> ID that is 865079980.
>>
>> The meeting minutes are at, everyone should be able to edit them. I tried
>> to get the synchronization between StackEdit and Google Drive to work, but
>> that seems to be a bit of a mess. So editing live in Google Docs it is -
>> we'll clean up to publish on the mailing list and as rendered Markdown
>> afterwards.
>>
>
> EDIT: this is the link that should work:
> https://drive.google.com/open?id=1vsLrXIAVGwkHb2hdscuWu9ZXik-3NH80X8Oh0eTy0cM&authuser=0
>
>
>>
>> Reminder about time zones (this always goes wrong):
>> UTC 7-8 pm
>> Local time Western Europe: 9-10 pm
>> CDT: 2-3 pm
>> time from now: 2.5 hours
>>
>> Cheers,
>> Ralf
>>
>>
Hi all, we had an interesting discussion yesterday with about 16 people.
And I personally liked connecting faces to names of people I've been
interacting with recently or for years. Below the minutes we took during
and edited after the meeting.

Cheers,
Ralf


Scipy developer hangout 22 April 2015

================================

**Attendance**: Ralf Gommers, David Cournapeau, Matthew Brett, Evgeni
Burovski, Eric Moore, Josef Perktold, Charles Harris, Nathaniel Smith,
Abraham Escalante, Eric Larson, Jamie Morton, Yoshiki Vazquez-Baeza, Neal
Becker, Ian Henriksen, Ariel Rokem, Julian Taylor

Mailing list thread that triggered this hangout:
http://article.gmane.org/gmane.comp.python.scientific.devel/19607

First thing to do is agree on the agenda for this hangout. Proposed agenda:


   1.

   Broadening developer base
   2.

   Developer docs & roadmap
   3.

   Communication
   4.

   Funding
   5.

   Governance
   6.

   Next release


Agenda OK as proposed, no one has additions.


Broadening developer base

-------------------------------------

> **Proposal**: (made on mailing list) start a "core dev mentoring program".

Discussion started with this proposal, which triggered a broader discussion
of the problem to be solved. Other things to consider:

   -

   What can we learn from other projects like Scikit-learn and Sympy that
   have been more successful in attracting new developers?
   -

   David C: At least for scikit learn, they have at least one paid, full
   time person to work on the project (Olivier Grisel), and my understanding
   is that he spends most of his time managing the project (reviewing PR, etc?)
   -

   Recently Matplotlib development changed a lot. Matthew: mostly caused by
   one new lead developer spending a lot of time and energy.
   -

   Josef: Scipy needs also domain specialists; PRs tend to stall if the one
   area that that expert takes care of takes a break.
   -

   David: related to Josef?s point, do we want to have ?sub-maintainers?
   for a particular subpackage ? Having a default go-to for advice may be
   helpful to newcomers, not sure how it would work in practice
   -

   Ralf: Scipy covers a very diverse set of topics, it?s almost impossible
   to be an expert on all of them. Pauli is pretty much the only one that has
   deep knowledge on most submodules.


Discussion then turned to funding and how we could use it (see section
?Funding? below), the actual proposal wasn?t discussed in much detail.


Developer docs & roadmap

-------------------------------------

developer docs not discussed, roadmap was discussed (see under funding)


Communication

---------------------

> **Proposal**: (made on mailing list) set up a closed mailing list for
developers.

Opinions are mixed on this proposal.

Matthew: you don?t want to give the impression that there are significant
discussions going on that should be public.

Ralf: agreed. However there are some topics (GSoC student ranking,
decisions on commit rights, potential funding sources) that anyway have to
happen and cannot be public. So we can make an explicit list of topics that
that mailing list is for.

Matthew: it can still give the wrong impression, ad hoc offline discussions
like there are now is probably preferable.

Josef: if closed list gets archived, people that later get access may not
like what they see then.

Matthew: decisions should always be reasonable and based on evidence.
Ethical question to ask yourself: would you be comfortable with seeing the
discussion on a decision published in the Wall Street Journal?

Abraham: personally I would like to see the reasons of why I was or wasn?t
accepted for GSoC later on, in order to learn from it.

Ralf: there?s enough discussion/resistance here to conclude that we
shouldn?t move forward with this proposal now.

Nathaniel: suggest we tackle the specific needs one at a time: e.g., make a
little mailing list for each GSoC, come up with some less-informal
mechanism for deciding who gets commit rights, etc.

Nathaniel?s suggestion is accepted as the way forward.


Funding

-----------

Matthew: Do we have anything to spend money on?

Ralf: yes, but the first issue is that I wouldn?t be comfortable spending
significant amounts of money without some governance doc/structure that
says something about how decisions about money are made.

Matthew: say we short-circuit that now by saying you are the dictator (not
an actual proposal) and can make those decisions?

Ralf: then, yes.

Chuck: We can?t afford senior developer types.

Ralf: Yeah, but we could potentially hire masters or PhD or postdocs for
any length of time between 3 months and a year or two at a time.

Bigger issue: what do we spend money on? And how do we present that to
possible money sources? Ralf suggests the roadmap (
https://github.com/scipy/scipy/pull/2908/files) as a todo list.

Nathaniel is concerned that this is a big list of little cleanups and stuff
that may be important but don?t look very urgent or make a good story for
funders. (And also secondarily he?s somewhat concerned about the idea of
making ?1.0? a conglomerate of everyone?s favorite wishlist item; it seems
like lots of those cleanups on the list could happen after 1.0 without any
real harm? But that?s orthogonal to the governance/funding/etc. issues.)

[Ralf, adding after the meeting] looking at that roadmap again, Nathaniel
has a point. Significant features, like unifying spline representations,
are on there but not identified clearly as major new features. Other
potentially important new features are missing from the roadmap because we
didn?t consider them ?necessary for 1.0?. This needs improving.

David C. is wondering about the longer term ?vision? of the project: if we
had somebody to work say 6 months, do we want something that goes beyond
bug fixing and ?minor? features ? What makes a good ?subproject? for being
included in scipy ? How do we position ourselves compared to the popular
scikits ?

Ralf: there are things like B-splines in interpolate, introducing Propack
in scipy.sparse, etc. that would require a talented developer to spend say
3-6 months full time on in order to push that topic forward. Also, projects
like Scikit-learn (sparse) and scikit-image (ndimage) lean heavily on Scipy.

Ian: the Cython API for BLAS/LAPACK is another good example.

Matthew: improving the Windows build situation (MinGW64, allowing use of
Fortran 90) would be valuable.

Nathaniel, adding after the meeting: the biggest downstream impact change I
can think of for scipy is getting a scipy.sparse API that doesn?t use
np.matrix -- this will directly change how a lot of people work on a daily
basis, and probably cause a large increase in the number of projects that
support sparse matrices. (This is blocked on __numpy_ufunc__ support in
numpy, but that will be released soon.)


Governance

----------------

Question: do we need people to look into this in more detail and come up
with some draft proposal as a basis for discussion? Volunteers: Matthew
after June 6th, Ralf.

Chuck: would like time to read the links and book by Karl Fogel. Can we
give everyone time to do this before following up? Decision: agree with
Chuck?s suggestion, follow-up call in one month. Ralf will organize (and
find a volunteer to host the call on bluejeans.com).

Concrete parts of governance:

   -

   Fiscal responsibility team for formalizing organization, NumFOCUS
   relationship
   -

   Deciding on a mechanism for who gets commit rights and what those mean.
   -

      Chuck is concerned about commit right list growing to be very long
      and full of inactive people, which could eventually become a security
      concern. Nathaniel argued that this is not really a big deal in practice
      (sabotage against OSS projects is incredibly easy and yet almost
unheard of
      in practice). On the other hand, just kicking people out for inactivity
      could be a big source of bad feelings and drama. One way to handle both
      concerns in a minimal-drama way would be to have a
      commit-rights-policy-document that lays out how commit rights
are bestowed,
      and also says that as a matter of policy to reduce security concerns, we
      will remove commit rights from the github interface from anyone who is
      inactive for X months, but this is purely procedural -- the person will
      still be considered to be a committer and their permissions will be
      restored automatically on request.
      -

      Some other projects? rules for committers:
      -

         Karl Fogel?s (excellent and free) Producing OSS book has a section
         discussing exactly this: http://producingoss.com/en/committers.html
         -

         The Rubinius project uses a very interesting model when your first
         pull request is merged, then you automatically get commit
rights at that
         time. (Of course you still have to follow other rules,
doesn?t mean you can
         just scribble nonsense in master. Or if you do of course you will
         immediately get kicked out and your rights reverted.) Like wikis, this
         apparently works better than you would expect. Interesting
blog posts that
         are not about this exactly but do touch on it [better sources welcome]:
         -


            http://rubini.us/2014/11/10/rubinius-3-0-part-1-the-rubinius-team/
            -

            http://rubini.us/2014/11/11/rubinius-3-0-part-2-the-process/
            -

         IPython?s governance document:
         https://github.com/ipython/ipython/wiki/IPEP-29:-Project-Governance
         -

      Regarding ?what commit rights mean?, this is also very important.
      Probably something like, ?you can, after due diligence/review,
merge other
      people?s pull requests?. Does *not* mean that you can commit directly to
      master without oversight, and this should be documented somewhere.
      -

      Do we want to formalize a bit more PR review (e.g. at least N people
      agreeing on it) ?
      -

         We already do this informally based on reviewer judgement (?I
         think this is good but I?m not sure, what do the rest of you
think?? versus
         ?obviously good docstring fix, merged thanks?), and that does
seem to work
         ok I think? -njs
         -

      It might also help to think about what ?makes a good committer?.
      -

         Nathaniel suggests: the defining feature of a good committer is
         NOT AT ALL that they are an awesome brilliant coder. The
defining feature
         of a good committer is that you can trust them to know their
limits, follow
         the project rules, and ask for help or abstain when they?re
unsure. So a
         brilliant coder who spits out perfectly-implemented new
features every day
         but can?t be bothered to write unit tests should not get
commit rights; a
         newbie who knows they?re a newbie is totally fine. I.e. this is not a
         position of power, it is a position of responsibility.



Next release

-----------------

not discussed
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scipy-dev/attachments/20150423/2ca58b3a/attachment.html>

From ralf.gommers at gmail.com  Sun Apr 26 10:22:17 2015
From: ralf.gommers at gmail.com (Ralf Gommers)
Date: Sun, 26 Apr 2015 16:22:17 +0200
Subject: [SciPy-Dev] 0.16.0 release
In-Reply-To: <CABL7CQizf4e7xo04dQdvng+12ocDR+c7LGHEe9KvJ5CmFrCKyA@mail.gmail.com>
References: <CABL7CQh4E4ESabrJt6J91nAYwd9QN++p5kn3dq1xv+2bxK-gVA@mail.gmail.com>
	<CAJJ-iFmMWz_Oy+e+2rxxruaafd6c1D-q5JaB=g=dFW_p4SD4=A@mail.gmail.com>
	<CABL7CQizf4e7xo04dQdvng+12ocDR+c7LGHEe9KvJ5CmFrCKyA@mail.gmail.com>
Message-ID: <CABL7CQi_ufgMLrFA-U4T8Mpr9TY+z99pKKCiHqFZC2pBrMCw=w@mail.gmail.com>

On Sun, Apr 5, 2015 at 9:40 PM, Ralf Gommers <ralf.gommers at gmail.com> wrote:

> Hi Abraham, Andreas,
>
> Thanks for your interest to both of you!
>
> The release manager is the person that is responsible for a number of
> actions to get from "now we start planning for the next release" to "this
> release is done". This description should give a reasonable overview (some
> details may be a bit outdated):
> https://github.com/numpy/numpy/blob/master/doc/HOWTO_RELEASE.rst.txt
>
> The release manager is the person that has the final responsibility for
> what does and does not go into a release, what are considered blocking
> issues, etc. This part really has to be done by a core developer. However,
> there's a lot of other things that the release manager also usually takes
> care of which could equally well be done by one or more contributors who
> are not core developers. Here's a list:
> - build/test Windows and OS X binaries (.exe, .dmg, .whl)
> - automate those binary builds/tests further
> - tackle some of the issues that need to be resolved before the alpha /
> beta / final release
> - PRs to complete the release notes and contributor name mapping
> - a PR to the scipy.org repo to put the release in the News section
> - improve documentation of the release process itself (the less you know
> to start with, the better you can document it)
>
> Things that the release manager should do himself:
> - propose the release schedule (dates for alpha/beta/rc/final)
> - create the maintenance/0.16.x branch
> - tag the (pre-)releases
> - send the release announcement emails
> - upload binaries to PyPi and SourceForge
> - propose/make decisions on blockers and content
>
> And what's always a team effort: merge as many PRs as possible before the
> release split.
>
> I hope this gives a clear picture. A "release team" that jointly tackles
> the above tasks would be great. Still would like to see a core dev
> volunteer though:)
>

Okay, I'll volunteer to be the release manager for this release. Here's a
proposal for the release schedule:

- branching 0.16.x: 10 May
- beta 1: 12 May
- release candidate 1: 24 May
- release candidate 2: 31 May (if needed)
- final release: 7 June

There aren't many serious issues at the moment, the only nontrivial one is
the QR updating issue. Here's what's marked to go in still:
https://github.com/scipy/scipy/milestones/0.16.0. The above schedule gives
us two weeks to merge PRs. Of the larger ones, I'd still like to see these
go in:
- 4682: signal.csd/coherence/welch
- 4374: cKDTree rewrite

If you see an issue that you consider a blocker or a PR that really should
go in, please label it with the 0.16.0 milestone or mention it here.

Cheers,
Ralf




>
> Cheers,
> Ralf
>
> P.S. I shouldn't forget to mention that for a long time Matthew Brett has
> been building the OS X wheels and done extensive Scipy Stack testing. And
> that for an even longer time Christoph Gohlke has been testing pre-releases
> with MSVC on Windows and resolving issues that are invariable found on that
> platform. I definitely consider them part of the "release team" (not that
> we have ever used that name before).
>
>
>
>
> On Sat, Apr 4, 2015 at 9:30 PM, Abraham Escalante <aeklant at gmail.com>
> wrote:
>
>> Hi Ralf,
>>
>> This sounds interesting to me and I would like to contribute but I fear I
>> may be still too new to the community to be of much help but maybe I could
>> provide some assistance to learn.
>>
>> Is this a task for admins only? Is there a way I can contribute?
>>
>> Cheers,
>> Abraham.
>>
>> 2015-04-04 2:54 GMT-06:00 Ralf Gommers <ralf.gommers at gmail.com>:
>>
>>> Hi all,
>>>
>>> It's about time to start preparing for the 0.16.0 release. Unusually,
>>> there are zero blocking issues it looks like. A lot of PRs to merge of
>>> course, but nothing very large that still needs a lot of work.
>>>
>>> Most important question: does anyone want to take the release manager
>>> role for this release? It would be good to start rotating this role for
>>> every minor release. This will help spread the knowledge on how to do
>>> releases and also spread the workload (it's not that much for a single
>>> release, but it adds up).
>>>
>>> Cheers,
>>> Ralf
>>>
>>>
>>> _______________________________________________
>>> SciPy-Dev mailing list
>>> SciPy-Dev at scipy.org
>>> http://mail.scipy.org/mailman/listinfo/scipy-dev
>>>
>>>
>>
>> _______________________________________________
>> SciPy-Dev mailing list
>> SciPy-Dev at scipy.org
>> http://mail.scipy.org/mailman/listinfo/scipy-dev
>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scipy-dev/attachments/20150426/30edc5e9/attachment.html>

From ralf.gommers at gmail.com  Sun Apr 26 10:31:43 2015
From: ralf.gommers at gmail.com (Ralf Gommers)
Date: Sun, 26 Apr 2015 16:31:43 +0200
Subject: [SciPy-Dev] 0.16.0 release
In-Reply-To: <5522518B.8000401@hilboll.de>
References: <CABL7CQh4E4ESabrJt6J91nAYwd9QN++p5kn3dq1xv+2bxK-gVA@mail.gmail.com>
	<CAJJ-iFmMWz_Oy+e+2rxxruaafd6c1D-q5JaB=g=dFW_p4SD4=A@mail.gmail.com>
	<CABL7CQizf4e7xo04dQdvng+12ocDR+c7LGHEe9KvJ5CmFrCKyA@mail.gmail.com>
	<5522518B.8000401@hilboll.de>
Message-ID: <CABL7CQheb4zjKr=cRumftBg3JNS=YBNns1i86yiYX6mH==Thgg@mail.gmail.com>

On Mon, Apr 6, 2015 at 11:27 AM, Andreas Hilboll <lists at hilboll.de> wrote:

> Hi Ralf,
>
> thanks for the clarification!
>
> > However, there's a lot of other things that the release manager also
> > usually takes care of which could equally well be done by one or more
> > contributors who are not core developers. Here's a list:
> > - build/test Windows and OS X binaries (.exe, .dmg, .whl)
> > - automate those binary builds/tests further
>
> I'm not working on Windows, so cannot help there.
>
> > - tackle some of the issues that need to be resolved before the alpha /
> > beta / final release
>
> How is the process to decide which issues are these?  Currently, there's
> 11 "prio-high" issues, and 284 "defect" issues.
>

We use the Github Milestones mechanism:
https://github.com/scipy/scipy/milestones/0.16.0

Everything that's a blocker or that someone wants to do something about
before branching (for enhancements) or the first beta/rc (issues) should
simply be added to the 0.16.0 milestone.


>
> > - PRs to complete the release notes and contributor name mapping
> > - a PR to the scipy.org <http://scipy.org> repo to put the release in
> > the News section
>
> Sure, that's easy.
>

I sent a PR to update release notes just now. Checking and updating the
author name mapping (``python tools/authors.py v0.15.0..master``) would be
useful to do sometime in the next two weeks though, if you have time.

 > - improve documentation of the release process itself (the less you know
> > to start with, the better you can document it)
>
> I think this email and your other mail from today are already good
> starters.  What's needed is a central location where volunteers can be
> pointed to.  I can keep this in mind and come up with some ideas.
>

Indeed. Our developer docs are a bit scattered, I plan to reorganize them a
bit in the near future. I'll ping you and Abraham for feedback and
suggestions once I have the initial structure.


>  > I hope this gives a clear picture. A "release team" that jointly tackles
> > the above tasks would be great. Still would like to see a core dev
> > volunteer though:)
>
> I'd be happy being part of the team.
>

Great!

Ralf
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scipy-dev/attachments/20150426/4e1bd202/attachment.html>

From clancyr at gmail.com  Sun Apr 26 11:03:21 2015
From: clancyr at gmail.com (Clancy Rowley)
Date: Sun, 26 Apr 2015 11:03:21 -0400
Subject: [SciPy-Dev] 0.16.0 release
In-Reply-To: <CABL7CQi_ufgMLrFA-U4T8Mpr9TY+z99pKKCiHqFZC2pBrMCw=w@mail.gmail.com>
References: <CABL7CQh4E4ESabrJt6J91nAYwd9QN++p5kn3dq1xv+2bxK-gVA@mail.gmail.com>
	<CAJJ-iFmMWz_Oy+e+2rxxruaafd6c1D-q5JaB=g=dFW_p4SD4=A@mail.gmail.com>
	<CABL7CQizf4e7xo04dQdvng+12ocDR+c7LGHEe9KvJ5CmFrCKyA@mail.gmail.com>
	<CABL7CQi_ufgMLrFA-U4T8Mpr9TY+z99pKKCiHqFZC2pBrMCw=w@mail.gmail.com>
Message-ID: <9438D218-DDCB-491C-AC91-DD51ACFDB722@gmail.com>

On Apr 26, 2015, at 10:22 AM, Ralf Gommers <ralf.gommers at gmail.com> wrote:
> 
> There aren't many serious issues at the moment, the only nontrivial one is the QR updating issue. Here's what's marked to go in still: https://github.com/scipy/scipy/milestones/0.16.0. The above schedule gives us two weeks to merge PRs. Of the larger ones, I'd still like to see these go in:
> - 4682: signal.csd/coherence/welch
> - 4374: cKDTree rewrite
> 
> If you see an issue that you consider a blocker or a PR that really should go in, please label it with the 0.16.0 milestone or mention it here.
> 
One PR I'd really like to see go in is:

- 4675: Fix lsim (in scipy.signal)

Cheers,
Clancy



From ralf.gommers at gmail.com  Sun Apr 26 11:07:09 2015
From: ralf.gommers at gmail.com (Ralf Gommers)
Date: Sun, 26 Apr 2015 17:07:09 +0200
Subject: [SciPy-Dev] 0.16.0 release
In-Reply-To: <9438D218-DDCB-491C-AC91-DD51ACFDB722@gmail.com>
References: <CABL7CQh4E4ESabrJt6J91nAYwd9QN++p5kn3dq1xv+2bxK-gVA@mail.gmail.com>
	<CAJJ-iFmMWz_Oy+e+2rxxruaafd6c1D-q5JaB=g=dFW_p4SD4=A@mail.gmail.com>
	<CABL7CQizf4e7xo04dQdvng+12ocDR+c7LGHEe9KvJ5CmFrCKyA@mail.gmail.com>
	<CABL7CQi_ufgMLrFA-U4T8Mpr9TY+z99pKKCiHqFZC2pBrMCw=w@mail.gmail.com>
	<9438D218-DDCB-491C-AC91-DD51ACFDB722@gmail.com>
Message-ID: <CABL7CQhoMwt9tnZ2doswkEq2dKobS221JXL1u5F_66FCUhyKQw@mail.gmail.com>

On Sun, Apr 26, 2015 at 5:03 PM, Clancy Rowley <clancyr at gmail.com> wrote:

> On Apr 26, 2015, at 10:22 AM, Ralf Gommers <ralf.gommers at gmail.com> wrote:
> >
> > There aren't many serious issues at the moment, the only nontrivial one
> is the QR updating issue. Here's what's marked to go in still:
> https://github.com/scipy/scipy/milestones/0.16.0. The above schedule
> gives us two weeks to merge PRs. Of the larger ones, I'd still like to see
> these go in:
> > - 4682: signal.csd/coherence/welch
> > - 4374: cKDTree rewrite
> >
> > If you see an issue that you consider a blocker or a PR that really
> should go in, please label it with the 0.16.0 milestone or mention it here.
> >
> One PR I'd really like to see go in is:
>
> - 4675: Fix lsim (in scipy.signal)
>

Agreed. I added it to the milestone. It's pretty much done, so shouldn't be
an issue to get that merged soon.

Ralf
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scipy-dev/attachments/20150426/92eb7828/attachment.html>

From lists at hilboll.de  Sun Apr 26 12:43:54 2015
From: lists at hilboll.de (Andreas Hilboll)
Date: Sun, 26 Apr 2015 18:43:54 +0200
Subject: [SciPy-Dev] 0.16.0 release
In-Reply-To: <CABL7CQheb4zjKr=cRumftBg3JNS=YBNns1i86yiYX6mH==Thgg@mail.gmail.com>
References: <CABL7CQh4E4ESabrJt6J91nAYwd9QN++p5kn3dq1xv+2bxK-gVA@mail.gmail.com>	<CAJJ-iFmMWz_Oy+e+2rxxruaafd6c1D-q5JaB=g=dFW_p4SD4=A@mail.gmail.com>	<CABL7CQizf4e7xo04dQdvng+12ocDR+c7LGHEe9KvJ5CmFrCKyA@mail.gmail.com>	<5522518B.8000401@hilboll.de>
	<CABL7CQheb4zjKr=cRumftBg3JNS=YBNns1i86yiYX6mH==Thgg@mail.gmail.com>
Message-ID: <553D15CA.3030909@hilboll.de>

> I sent a PR to update release notes just now. Checking and updating the
> author name mapping (``python tools/authors.py v0.15.0..master``) would
> be useful to do sometime in the next two weeks though, if you have time.

If I understand you correctly, the name mappings in the file
tools/authors.py must be updated.

What's the policy on name inclusion?  Possible solutions:

- approach all new contributors individually with which name they want
to be mentioned

- take the full name as on Github

- take the Github username

- ...?

-- Andreas.


From ralf.gommers at gmail.com  Sun Apr 26 13:05:44 2015
From: ralf.gommers at gmail.com (Ralf Gommers)
Date: Sun, 26 Apr 2015 19:05:44 +0200
Subject: [SciPy-Dev] 0.16.0 release
In-Reply-To: <553D15CA.3030909@hilboll.de>
References: <CABL7CQh4E4ESabrJt6J91nAYwd9QN++p5kn3dq1xv+2bxK-gVA@mail.gmail.com>
	<CAJJ-iFmMWz_Oy+e+2rxxruaafd6c1D-q5JaB=g=dFW_p4SD4=A@mail.gmail.com>
	<CABL7CQizf4e7xo04dQdvng+12ocDR+c7LGHEe9KvJ5CmFrCKyA@mail.gmail.com>
	<5522518B.8000401@hilboll.de>
	<CABL7CQheb4zjKr=cRumftBg3JNS=YBNns1i86yiYX6mH==Thgg@mail.gmail.com>
	<553D15CA.3030909@hilboll.de>
Message-ID: <CABL7CQiR9eCETB=0UsmvRrP7_jvrb1VBdv2WwgHm7XU+ep9LKA@mail.gmail.com>

On Sun, Apr 26, 2015 at 6:43 PM, Andreas Hilboll <lists at hilboll.de> wrote:

> > I sent a PR to update release notes just now. Checking and updating the
> > author name mapping (``python tools/authors.py v0.15.0..master``) would
> > be useful to do sometime in the next two weeks though, if you have time.
>
> If I understand you correctly, the name mappings in the file
> tools/authors.py must be updated.
>
> What's the policy on name inclusion?  Possible solutions:
>
> - approach all new contributors individually with which name they want
> to be mentioned
>
> - take the full name as on Github
>
> - take the Github username
>
> - ...?
>

Good question. We want the full name in the release notes. Many people have
their full name up on Github, but the name wasn't set correctly in the
commit. In those cases I normally copy the name from Github and be done.
Same if there's a full name in the email. If there's only a Github handle
and/or an email address with a handle, then I ask the contributor
explicitly if it's OK to add the name to the release notes.

Ralf
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scipy-dev/attachments/20150426/6d2e6903/attachment.html>

From ralf.gommers at gmail.com  Mon Apr 27 16:32:36 2015
From: ralf.gommers at gmail.com (Ralf Gommers)
Date: Mon, 27 Apr 2015 22:32:36 +0200
Subject: [SciPy-Dev] GSoC'15 accepted students for Scipy/Numpy
Message-ID: <CABL7CQhEOQ3zyXHgx+1DPBZFTM5oB2Fy3n1QTvJ_JMDeROOiJw@mail.gmail.com>

Hi all,

Google has just announced which students got accepted for this year's GSoC.
For Scipy these are:
- Nikolay Mayorov, "Improve nonlinear least squares minimization
functionality in SciPy"
  mentors: Chuck & Evgeni
- Abraham Escalante, "SciPy: scipy.stats improvements"
  mentor: Ralf (Evgeni is backup mentor)

Furthermore, this proposal was accepted for Scikit-image:
- Aman Singh, "Scikit-Image: rewriting scipy.ndimage to cython"
  mentors: Jaime, Ralf & the scikit-image devs

Congratulations to all of you!

We had a lot of interest this year, which is great to see. GSoC
applications are competitive, and unfortunately there are students who
didn't make it. To those students I would say: please stay involved, and
you're very welcome to apply again next year!

Today is also the start of the "community bonding period", where the
students aren't yet expected to start working on their project but do get
time to further figure out how things work, interact with the community and
ensure that they can hit the ground running on day 1 of the coding period:
http://googlesummerofcode.blogspot.nl/2007/04/so-what-is-this-community-bonding-all.html.


It looks like it'll be an interesting and productive summer!

Cheers,
Ralf

P.S. all proposals are linked on
https://github.com/scipy/scipy/wiki/GSoC-project-ideas#student-applications-for-2015-to-scipy-and-numpy
for who's interested in the details.

P.P.S. some students have asked to get some feedback about why they
were/weren't accepted, in order to learn from it for a next time. Until
today we weren't allowed to say much, but now that Google has announced the
results I'd be happy to give some feedback - please contact me in private
if you want.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scipy-dev/attachments/20150427/4139a77b/attachment.html>

From evgeny.burovskiy at gmail.com  Mon Apr 27 18:01:08 2015
From: evgeny.burovskiy at gmail.com (Evgeni Burovski)
Date: Mon, 27 Apr 2015 23:01:08 +0100
Subject: [SciPy-Dev] 0.16.0 release
In-Reply-To: <CABL7CQi_ufgMLrFA-U4T8Mpr9TY+z99pKKCiHqFZC2pBrMCw=w@mail.gmail.com>
References: <CABL7CQh4E4ESabrJt6J91nAYwd9QN++p5kn3dq1xv+2bxK-gVA@mail.gmail.com>
	<CAJJ-iFmMWz_Oy+e+2rxxruaafd6c1D-q5JaB=g=dFW_p4SD4=A@mail.gmail.com>
	<CABL7CQizf4e7xo04dQdvng+12ocDR+c7LGHEe9KvJ5CmFrCKyA@mail.gmail.com>
	<CABL7CQi_ufgMLrFA-U4T8Mpr9TY+z99pKKCiHqFZC2pBrMCw=w@mail.gmail.com>
Message-ID: <CAMRo0itoNOJ4o0JkwWVH9q=4JF6gnSQerHZ8THAUvbfPxCCj-A@mail.gmail.com>

On Sun, Apr 26, 2015 at 3:22 PM, Ralf Gommers <ralf.gommers at gmail.com> wrote:
>
>
> On Sun, Apr 5, 2015 at 9:40 PM, Ralf Gommers <ralf.gommers at gmail.com> wrote:
>>
>> Hi Abraham, Andreas,
>>
>> Thanks for your interest to both of you!
>>
>> The release manager is the person that is responsible for a number of
>> actions to get from "now we start planning for the next release" to "this
>> release is done". This description should give a reasonable overview (some
>> details may be a bit outdated):
>> https://github.com/numpy/numpy/blob/master/doc/HOWTO_RELEASE.rst.txt
>>
>> The release manager is the person that has the final responsibility for
>> what does and does not go into a release, what are considered blocking
>> issues, etc. This part really has to be done by a core developer. However,
>> there's a lot of other things that the release manager also usually takes
>> care of which could equally well be done by one or more contributors who are
>> not core developers. Here's a list:
>> - build/test Windows and OS X binaries (.exe, .dmg, .whl)
>> - automate those binary builds/tests further
>> - tackle some of the issues that need to be resolved before the alpha /
>> beta / final release
>> - PRs to complete the release notes and contributor name mapping
>> - a PR to the scipy.org repo to put the release in the News section
>> - improve documentation of the release process itself (the less you know
>> to start with, the better you can document it)
>>
>> Things that the release manager should do himself:
>> - propose the release schedule (dates for alpha/beta/rc/final)
>> - create the maintenance/0.16.x branch
>> - tag the (pre-)releases
>> - send the release announcement emails
>> - upload binaries to PyPi and SourceForge
>> - propose/make decisions on blockers and content
>>
>> And what's always a team effort: merge as many PRs as possible before the
>> release split.
>>
>> I hope this gives a clear picture. A "release team" that jointly tackles
>> the above tasks would be great. Still would like to see a core dev volunteer
>> though:)
>
>
> Okay, I'll volunteer to be the release manager for this release. Here's a
> proposal for the release schedule:
>
> - branching 0.16.x: 10 May
> - beta 1: 12 May
> - release candidate 1: 24 May
> - release candidate 2: 31 May (if needed)
> - final release: 7 June
>
> There aren't many serious issues at the moment, the only nontrivial one is
> the QR updating issue. Here's what's marked to go in still:
> https://github.com/scipy/scipy/milestones/0.16.0. The above schedule gives
> us two weeks to merge PRs. Of the larger ones, I'd still like to see these
> go in:
> - 4682: signal.csd/coherence/welch
> - 4374: cKDTree rewrite
>
> If you see an issue that you consider a blocker or a PR that really should
> go in, please label it with the 0.16.0 milestone or mention it here.


permutation t-tests, https://github.com/scipy/scipy/pull/4440, look
nearly ready (from a distance at least)
A nice-to-have, not a release blocker for sure.

Evgeni


> Cheers,
> Ralf
>
>
>
>>
>>
>> Cheers,
>> Ralf
>>
>> P.S. I shouldn't forget to mention that for a long time Matthew Brett has
>> been building the OS X wheels and done extensive Scipy Stack testing. And
>> that for an even longer time Christoph Gohlke has been testing pre-releases
>> with MSVC on Windows and resolving issues that are invariable found on that
>> platform. I definitely consider them part of the "release team" (not that we
>> have ever used that name before).
>>
>>
>>
>>
>> On Sat, Apr 4, 2015 at 9:30 PM, Abraham Escalante <aeklant at gmail.com>
>> wrote:
>>>
>>> Hi Ralf,
>>>
>>> This sounds interesting to me and I would like to contribute but I fear I
>>> may be still too new to the community to be of much help but maybe I could
>>> provide some assistance to learn.
>>>
>>> Is this a task for admins only? Is there a way I can contribute?
>>>
>>> Cheers,
>>> Abraham.
>>>
>>> 2015-04-04 2:54 GMT-06:00 Ralf Gommers <ralf.gommers at gmail.com>:
>>>>
>>>> Hi all,
>>>>
>>>> It's about time to start preparing for the 0.16.0 release. Unusually,
>>>> there are zero blocking issues it looks like. A lot of PRs to merge of
>>>> course, but nothing very large that still needs a lot of work.
>>>>
>>>> Most important question: does anyone want to take the release manager
>>>> role for this release? It would be good to start rotating this role for
>>>> every minor release. This will help spread the knowledge on how to do
>>>> releases and also spread the workload (it's not that much for a single
>>>> release, but it adds up).
>>>>
>>>> Cheers,
>>>> Ralf
>>>>
>>>>
>>>> _______________________________________________
>>>> SciPy-Dev mailing list
>>>> SciPy-Dev at scipy.org
>>>> http://mail.scipy.org/mailman/listinfo/scipy-dev
>>>>
>>>
>>>
>>> _______________________________________________
>>> SciPy-Dev mailing list
>>> SciPy-Dev at scipy.org
>>> http://mail.scipy.org/mailman/listinfo/scipy-dev
>>>
>>
>
>
> _______________________________________________
> SciPy-Dev mailing list
> SciPy-Dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/scipy-dev
>


From ralf.gommers at gmail.com  Tue Apr 28 17:15:18 2015
From: ralf.gommers at gmail.com (Ralf Gommers)
Date: Tue, 28 Apr 2015 23:15:18 +0200
Subject: [SciPy-Dev] 0.16.0 release
In-Reply-To: <CAMRo0itoNOJ4o0JkwWVH9q=4JF6gnSQerHZ8THAUvbfPxCCj-A@mail.gmail.com>
References: <CABL7CQh4E4ESabrJt6J91nAYwd9QN++p5kn3dq1xv+2bxK-gVA@mail.gmail.com>
	<CAJJ-iFmMWz_Oy+e+2rxxruaafd6c1D-q5JaB=g=dFW_p4SD4=A@mail.gmail.com>
	<CABL7CQizf4e7xo04dQdvng+12ocDR+c7LGHEe9KvJ5CmFrCKyA@mail.gmail.com>
	<CABL7CQi_ufgMLrFA-U4T8Mpr9TY+z99pKKCiHqFZC2pBrMCw=w@mail.gmail.com>
	<CAMRo0itoNOJ4o0JkwWVH9q=4JF6gnSQerHZ8THAUvbfPxCCj-A@mail.gmail.com>
Message-ID: <CABL7CQi+aW0gM995dDKQzYmQ5C5uCR=cK74K2h4nOZPzoTn+DQ@mail.gmail.com>

On Tue, Apr 28, 2015 at 12:01 AM, Evgeni Burovski <
evgeny.burovskiy at gmail.com> wrote:

> On Sun, Apr 26, 2015 at 3:22 PM, Ralf Gommers <ralf.gommers at gmail.com>
> wrote:
> >
> >
> > On Sun, Apr 5, 2015 at 9:40 PM, Ralf Gommers <ralf.gommers at gmail.com>
> wrote:
> >>
> >> Hi Abraham, Andreas,
> >>
> >> Thanks for your interest to both of you!
> >>
> >> The release manager is the person that is responsible for a number of
> >> actions to get from "now we start planning for the next release" to
> "this
> >> release is done". This description should give a reasonable overview
> (some
> >> details may be a bit outdated):
> >> https://github.com/numpy/numpy/blob/master/doc/HOWTO_RELEASE.rst.txt
> >>
> >> The release manager is the person that has the final responsibility for
> >> what does and does not go into a release, what are considered blocking
> >> issues, etc. This part really has to be done by a core developer.
> However,
> >> there's a lot of other things that the release manager also usually
> takes
> >> care of which could equally well be done by one or more contributors
> who are
> >> not core developers. Here's a list:
> >> - build/test Windows and OS X binaries (.exe, .dmg, .whl)
> >> - automate those binary builds/tests further
> >> - tackle some of the issues that need to be resolved before the alpha /
> >> beta / final release
> >> - PRs to complete the release notes and contributor name mapping
> >> - a PR to the scipy.org repo to put the release in the News section
> >> - improve documentation of the release process itself (the less you know
> >> to start with, the better you can document it)
> >>
> >> Things that the release manager should do himself:
> >> - propose the release schedule (dates for alpha/beta/rc/final)
> >> - create the maintenance/0.16.x branch
> >> - tag the (pre-)releases
> >> - send the release announcement emails
> >> - upload binaries to PyPi and SourceForge
> >> - propose/make decisions on blockers and content
> >>
> >> And what's always a team effort: merge as many PRs as possible before
> the
> >> release split.
> >>
> >> I hope this gives a clear picture. A "release team" that jointly tackles
> >> the above tasks would be great. Still would like to see a core dev
> volunteer
> >> though:)
> >
> >
> > Okay, I'll volunteer to be the release manager for this release. Here's a
> > proposal for the release schedule:
> >
> > - branching 0.16.x: 10 May
> > - beta 1: 12 May
> > - release candidate 1: 24 May
> > - release candidate 2: 31 May (if needed)
> > - final release: 7 June
> >
> > There aren't many serious issues at the moment, the only nontrivial one
> is
> > the QR updating issue. Here's what's marked to go in still:
> > https://github.com/scipy/scipy/milestones/0.16.0. The above schedule
> gives
> > us two weeks to merge PRs. Of the larger ones, I'd still like to see
> these
> > go in:
> > - 4682: signal.csd/coherence/welch
> > - 4374: cKDTree rewrite
> >
> > If you see an issue that you consider a blocker or a PR that really
> should
> > go in, please label it with the 0.16.0 milestone or mention it here.
>
>
> permutation t-tests, https://github.com/scipy/scipy/pull/4440, look
> nearly ready (from a distance at least)
> A nice-to-have, not a release blocker for sure.
>

Indeed, would be nice to get that in. Added to the milestone for now.

Ralf



>
> Evgeni
>
>
> > Cheers,
> > Ralf
> >
> >
> >
> >>
> >>
> >> Cheers,
> >> Ralf
> >>
> >> P.S. I shouldn't forget to mention that for a long time Matthew Brett
> has
> >> been building the OS X wheels and done extensive Scipy Stack testing.
> And
> >> that for an even longer time Christoph Gohlke has been testing
> pre-releases
> >> with MSVC on Windows and resolving issues that are invariable found on
> that
> >> platform. I definitely consider them part of the "release team" (not
> that we
> >> have ever used that name before).
> >>
> >>
> >>
> >>
> >> On Sat, Apr 4, 2015 at 9:30 PM, Abraham Escalante <aeklant at gmail.com>
> >> wrote:
> >>>
> >>> Hi Ralf,
> >>>
> >>> This sounds interesting to me and I would like to contribute but I
> fear I
> >>> may be still too new to the community to be of much help but maybe I
> could
> >>> provide some assistance to learn.
> >>>
> >>> Is this a task for admins only? Is there a way I can contribute?
> >>>
> >>> Cheers,
> >>> Abraham.
> >>>
> >>> 2015-04-04 2:54 GMT-06:00 Ralf Gommers <ralf.gommers at gmail.com>:
> >>>>
> >>>> Hi all,
> >>>>
> >>>> It's about time to start preparing for the 0.16.0 release. Unusually,
> >>>> there are zero blocking issues it looks like. A lot of PRs to merge of
> >>>> course, but nothing very large that still needs a lot of work.
> >>>>
> >>>> Most important question: does anyone want to take the release manager
> >>>> role for this release? It would be good to start rotating this role
> for
> >>>> every minor release. This will help spread the knowledge on how to do
> >>>> releases and also spread the workload (it's not that much for a single
> >>>> release, but it adds up).
> >>>>
> >>>> Cheers,
> >>>> Ralf
> >>>>
> >>>>
> >>>> _______________________________________________
> >>>> SciPy-Dev mailing list
> >>>> SciPy-Dev at scipy.org
> >>>> http://mail.scipy.org/mailman/listinfo/scipy-dev
> >>>>
> >>>
> >>>
> >>> _______________________________________________
> >>> SciPy-Dev mailing list
> >>> SciPy-Dev at scipy.org
> >>> http://mail.scipy.org/mailman/listinfo/scipy-dev
> >>>
> >>
> >
> >
> > _______________________________________________
> > SciPy-Dev mailing list
> > SciPy-Dev at scipy.org
> > http://mail.scipy.org/mailman/listinfo/scipy-dev
> >
> _______________________________________________
> SciPy-Dev mailing list
> SciPy-Dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/scipy-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scipy-dev/attachments/20150428/f3fadbf2/attachment.html>