From njs at pobox.com  Mon Nov  2 21:06:51 2015
From: njs at pobox.com (Nathaniel Smith)
Date: Mon, 2 Nov 2015 18:06:51 -0800
Subject: [SciPy-Dev] Retiring wiki.scipy.org?
In-Reply-To: <CABL7CQjpdiwaMnZky+nQaOfeGHcob0ApZ=RCRMrrVq=Ctun3Rw@mail.gmail.com>
References: <loom.20151025T180550-902@post.gmane.org>
 <CABL7CQjpdiwaMnZky+nQaOfeGHcob0ApZ=RCRMrrVq=Ctun3Rw@mail.gmail.com>
Message-ID: <CAPJVwB=34hfr7X0-Tp_48kPXUCmHd8QHMNjPf3gPQ0SFzwZkvg@mail.gmail.com>

On Sat, Oct 31, 2015 at 5:02 AM, Ralf Gommers <ralf.gommers at gmail.com> wrote:
>
>
> On Sun, Oct 25, 2015 at 6:54 PM, Pauli Virtanen <pav at iki.fi> wrote:
>>
>> Hi,
>>
>> I've been looking through the stuff that was on wiki.scipy.org,
>> and it appears it would be the time to retire it.
>>
>> cf. http://pav.iki.fi/tmp/to-be-removed/localhost_8080/AllPages.html
>> for the actual material that is there.
>>
>>
>> I have the impression that:
>>
>> (1) Most of the material that is useful has been moved elsewhere.
>>
>> The download instructions, topical software, etc. are moved to
>> scipy.org and/or numpy/scipy documentation.
>>
>> The Numpy/Scipy tutorials that were there have mostly been subsumed
>> in the Numpy/Scipy documentation, or superceded by better resources
>> elsewhere eg. scipy-lectures.org. "Numpy for Matlab users" was
>> just added there too.
>>
>> The Cookbook section was converted to IPython notebooks
>> some years ago:  https://github.com/pv/SciPy-CookBook
>> However, several of these are not so relevant today.
>>
>> (2) The relevance of what's left is not so clear.
>>
>> The rest of the content seems fairly miscellaneous. Recipes
>> from converting code from Numeric/numarray are probably
>> not highly useful any more.
>>
>> If you know something valuable there not listed above, it would
>> be useful to point it out.
>>
>> Excluding edits to front/download/etc pages that are nowadays
>> on github/scipy/scipy.org, the wiki was edited 95 times in the
>> last 5 years; and 2 times within the last 2 years.
>
>
> Agreed, even in the converted material there's a lot that's not relevant
> anymore. What wasn't converted is even less relevant.
>
> There were some comments on the issue tracker about people looking for
> specific pages, so making a dump of everything available somewhere like
> Nathaniel suggested makes sense I think.
>
>> (3) Would having a wiki be useful?
>>
>> Based on the content there it's not clear if there is a need
>> for a wiki nowadays --- people can just put stuff up on
>> github/bitbucket/pypi and blog about it.
>>
>> It might be useful to have a "cookbook" collection,
>> but a github repository with ipython notebooks,
>> or Scipy Central, sound like better solutions for today
>> than a wiki.
>>
>>
>> Thoughts?
>
>
> I also had a look at visitor stats (see
> http://www.easycounter.com/report/scipy.org). Conclusion: scipy.org is a
> very active domain (global rank around 25000), but wiki.scipy.org only
> accounts for 1.65% of the total. That's still 3450 page views a day though.
> So putting redirects in for the most popular pages is important.
>
> +1 for no wiki anymore

Concrete proposal:

- take @rkern's dump of the underlying wiki data, and dump it into a
git repo at github.com/scipy/old-wiki for archival purposes
- take @pv's scrape of the rendered html pages, and dump them into the
same repo, but on the gh-pages branch

This will make the rendered HTML pages available at
http://scipy.github.io/old-wiki/, and we can point people there when
necessary. And puts the both the raw data and the readable versions
onto infrastructure that someone else will worry about maintaining
:-).

-n

-- 
Nathaniel J. Smith -- http://vorpus.org


From lzkelley at gmail.com  Tue Nov  3 10:03:49 2015
From: lzkelley at gmail.com (Luke Zoltan Kelley)
Date: Tue, 3 Nov 2015 10:03:49 -0500
Subject: [SciPy-Dev] binned_statistic: binnumber and array shapes
Message-ID: <6772E98A-93AE-4B17-86D0-16B141342902@gmail.com>

The docs for `scipy.stats.binned_statistic` explain the `binnumber` returned arrays as:

binnumber : 1-D ndarray of ints
    This assigns to each observation an integer that represents the bin
    in which this observation falls. Array has the same length as `values`.

However, it's very difficult to understand how the returned values match this description.  For example:

>>> a1 = [0.1, 0.1, 0.1, 0.6]
>>> a2 = [2.1, 2.6, 2.1, 2.1]
>>> b1 = [0.0, 0.5, 1.0]
>>> b2 = [2.0, 2.5, 3.0]
>>> stats = scipy.stats.binned_statistic_2d(a1, a2, None, 'count', bins=[b1,b2])
BinnedStatistic2dResult(statistic=array([[ 2.,  1.],
       [ 1.,  0.]]), x_edge=array([ 0. ,  0.5,  1. ]), y_edge=array([ 2. ,  2.5,  3. ]), binnumber=array([5, 6, 5, 9]))

The resulting 'statistic' array makes sense; but the 'binnumber' array is... cryptic...
Before being returned, [`statistic` is reshaped and cleaned-up](https://github.com/scipy/scipy/blob/master/scipy/stats/_binned_statistic.py#L452-L461 <https://github.com/scipy/scipy/blob/master/scipy/stats/_binned_statistic.py#L452-L461>)

Should the same thing be happening to `binnumber`?

(Unfortunately) I created an [issue for this](https://github.com/scipy/scipy/issues/5449 <https://github.com/scipy/scipy/issues/5449>), but it seemed like this (the mailing list) was probably far more appropriate; woops.  One other minor point is that the docstring for `binned_statistic_2d` says that `x` and `y` can have different lengths.  I think that's a mistake; they have to be the same shape right?

Luke
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scipy-dev/attachments/20151103/373c9888/attachment.html>

From josef.pktd at gmail.com  Tue Nov  3 10:29:08 2015
From: josef.pktd at gmail.com (josef.pktd at gmail.com)
Date: Tue, 3 Nov 2015 10:29:08 -0500
Subject: [SciPy-Dev] binned_statistic: binnumber and array shapes
In-Reply-To: <6772E98A-93AE-4B17-86D0-16B141342902@gmail.com>
References: <6772E98A-93AE-4B17-86D0-16B141342902@gmail.com>
Message-ID: <CAMMTP+A4fjScKG6oU9wmC-NQyXAPfnk=8E=rSoi7OBxK4zXDnA@mail.gmail.com>

On Tue, Nov 3, 2015 at 10:03 AM, Luke Zoltan Kelley <lzkelley at gmail.com>
wrote:

> The docs for `scipy.stats.binned_statistic` explain the `binnumber`
> returned arrays as:
>
> binnumber : 1-D ndarray of ints
>     This assigns to each observation an integer that represents the bin
>     in which this observation falls. Array has the same length as `values`.
>
>
> However, it's very difficult to understand how the returned values match
> this description.  For example:
>
> >>> a1 = [0.1, 0.1, 0.1, 0.6]
> >>> a2 = [2.1, 2.6, 2.1, 2.1]
> >>> b1 = [0.0, 0.5, 1.0]
> >>> b2 = [2.0, 2.5, 3.0]
> >>> stats = scipy.stats.binned_statistic_2d(a1, a2, None, 'count',
> bins=[b1,b2])
> BinnedStatistic2dResult(statistic=array([[ 2.,  1.],
>        [ 1.,  0.]]), x_edge=array([ 0. ,  0.5,  1. ]), y_edge=array([ 2.
> ,  2.5,  3. ]), binnumber=array([5, 6, 5, 9]))
>
> The resulting 'statistic' array makes sense; but the 'binnumber' array
> is... cryptic...
>


My guess is that there are outlier bins, one row and one column in front of
the actual bins, so counting for binnumber is based on a a larger array.


0, 1, 2, 3
4, 5*, 6*, 7
8, 9*, 10*, 11
12, 13, 14, 15

You only get the start entries in the center without outliers.

I didn't check the details.

Josef



> Before being returned, [`statistic` is reshaped and cleaned-up](
> https://github.com/scipy/scipy/blob/master/scipy/stats/_binned_statistic.py#L452-L461
> )
>
> Should the same thing be happening to `binnumber`?
>
> (Unfortunately) I created an [issue for this](
> https://github.com/scipy/scipy/issues/5449), but it seemed like this (the
> mailing list) was probably far more appropriate; woops.  One other minor
> point is that the docstring for `binned_statistic_2d` says that `x` and `y`
> can have different lengths.  I think that's a mistake; they have to be the
> same shape right?
>
> Luke
>
> _______________________________________________
> SciPy-Dev mailing list
> SciPy-Dev at scipy.org
> https://mail.scipy.org/mailman/listinfo/scipy-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scipy-dev/attachments/20151103/56ed254d/attachment.html>

From lzkelley at gmail.com  Tue Nov  3 10:31:50 2015
From: lzkelley at gmail.com (Luke Zoltan Kelley)
Date: Tue, 3 Nov 2015 10:31:50 -0500
Subject: [SciPy-Dev] binned_statistic: binnumber and array shapes
In-Reply-To: <CAMMTP+A4fjScKG6oU9wmC-NQyXAPfnk=8E=rSoi7OBxK4zXDnA@mail.gmail.com>
References: <6772E98A-93AE-4B17-86D0-16B141342902@gmail.com>
 <CAMMTP+A4fjScKG6oU9wmC-NQyXAPfnk=8E=rSoi7OBxK4zXDnA@mail.gmail.com>
Message-ID: <59A2E77B-02DE-4D63-B856-BFB4E105EB1D@gmail.com>


> On Nov 3, 2015, at 10:29 AM, josef.pktd at gmail.com wrote:
> 
> 
> 
> On Tue, Nov 3, 2015 at 10:03 AM, Luke Zoltan Kelley <lzkelley at gmail.com <mailto:lzkelley at gmail.com>> wrote:
> The docs for `scipy.stats.binned_statistic` explain the `binnumber` returned arrays as:
> 
> binnumber : 1-D ndarray of ints
>     This assigns to each observation an integer that represents the bin
>     in which this observation falls. Array has the same length as `values`.
> 
> However, it's very difficult to understand how the returned values match this description.  For example:
> 
> >>> a1 = [0.1, 0.1, 0.1, 0.6]
> >>> a2 = [2.1, 2.6, 2.1, 2.1]
> >>> b1 = [0.0, 0.5, 1.0]
> >>> b2 = [2.0, 2.5, 3.0]
> >>> stats = scipy.stats.binned_statistic_2d(a1, a2, None, 'count', bins=[b1,b2])
> BinnedStatistic2dResult(statistic=array([[ 2.,  1.],
>        [ 1.,  0.]]), x_edge=array([ 0. ,  0.5,  1. ]), y_edge=array([ 2. ,  2.5,  3. ]), binnumber=array([5, 6, 5, 9]))
> 
> The resulting 'statistic' array makes sense; but the 'binnumber' array is... cryptic...
> 
> 
> My guess is that there are outlier bins, one row and one column in front of the actual bins, so counting for binnumber is based on a a larger array.
> 
> 
> 0, 1, 2, 3
> 4, 5*, 6*, 7
> 8, 9*, 10*, 11
> 12, 13, 14, 15

That's exactly what `statistic` looks like before having the 'outlier' bins cleaned up.  It doesn't seem like there's any benefit to preserving this format.

> You only get the start entries in the center without outliers.
> 
> I didn't check the details.
> 
> Josef
> 
>  
> Before being returned, [`statistic` is reshaped and cleaned-up](https://github.com/scipy/scipy/blob/master/scipy/stats/_binned_statistic.py#L452-L461 <https://github.com/scipy/scipy/blob/master/scipy/stats/_binned_statistic.py#L452-L461>)
> 
> Should the same thing be happening to `binnumber`?
> 
> (Unfortunately) I created an [issue for this](https://github.com/scipy/scipy/issues/5449 <https://github.com/scipy/scipy/issues/5449>), but it seemed like this (the mailing list) was probably far more appropriate; woops.  One other minor point is that the docstring for `binned_statistic_2d` says that `x` and `y` can have different lengths.  I think that's a mistake; they have to be the same shape right?
> 
> Luke
> 
> _______________________________________________
> SciPy-Dev mailing list
> SciPy-Dev at scipy.org <mailto:SciPy-Dev at scipy.org>
> https://mail.scipy.org/mailman/listinfo/scipy-dev <https://mail.scipy.org/mailman/listinfo/scipy-dev>
> 
> 
> _______________________________________________
> SciPy-Dev mailing list
> SciPy-Dev at scipy.org
> https://mail.scipy.org/mailman/listinfo/scipy-dev

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scipy-dev/attachments/20151103/43e318ce/attachment.html>

From pav at iki.fi  Tue Nov  3 17:48:22 2015
From: pav at iki.fi (Pauli Virtanen)
Date: Tue, 3 Nov 2015 22:48:22 +0000 (UTC)
Subject: [SciPy-Dev] Retiring wiki.scipy.org?
References: <loom.20151025T180550-902@post.gmane.org>
 <CABL7CQjpdiwaMnZky+nQaOfeGHcob0ApZ=RCRMrrVq=Ctun3Rw@mail.gmail.com>
 <CAPJVwB=34hfr7X0-Tp_48kPXUCmHd8QHMNjPf3gPQ0SFzwZkvg@mail.gmail.com>
Message-ID: <n1bdjm$rkk$1@ger.gmane.org>

Mon, 02 Nov 2015 18:06:51 -0800, Nathaniel Smith kirjoitti:
[clip]
> Concrete proposal:
> 
> - take @rkern's dump of the underlying wiki data, and dump it into a git
> repo at github.com/scipy/old-wiki for archival purposes 
> - take @pv's scrape of the rendered html pages, and dump them 
> into the same repo, but on the gh-pages branch

Tuumasta toimeen: https://github.com/scipy/old-wiki/

The dump file itself was too large for Github (100M limit), so it's there 
as a "release" binary file.

-- 
Pauli Virtanen



From alan.c.wu at gmail.com  Tue Nov  3 18:11:08 2015
From: alan.c.wu at gmail.com (Alan Wu)
Date: Tue, 03 Nov 2015 23:11:08 +0000
Subject: [SciPy-Dev] Imsave and imwrite
References: <CABkLPOKiP-uR57BK3pOTLBEJ9amma758R4=LabsYf6EWKJ4HvQ@mail.gmail.com>
Message-ID: <CABkLPOLVOQHhKUX_2i+wq1MzHfzDaY-2JAWVAQJVHh9VdKue2Q@mail.gmail.com>

I moved imsave into imwrite and wrapped imsave to call imwrite, in order to
be consistent in naming read/write and with other packages

like matlab and opencv.



I

would like to

submit it as a pull request

.

Thanks,
Alan
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scipy-dev/attachments/20151103/92441d3c/attachment.html>

From alan.c.wu at gmail.com  Tue Nov  3 18:23:22 2015
From: alan.c.wu at gmail.com (Alan Wu)
Date: Tue, 03 Nov 2015 23:23:22 +0000
Subject: [SciPy-Dev] Imsave and imwrite
References: <CABkLPOKiP-uR57BK3pOTLBEJ9amma758R4=LabsYf6EWKJ4HvQ@mail.gmail.com>
 <CABkLPOLVOQHhKUX_2i+wq1MzHfzDaY-2JAWVAQJVHh9VdKue2Q@mail.gmail.com>
Message-ID: <CABkLPOKJwsjTkJioXkuOkoqMugENrxtMCVF2On1M=+a4zX8kUQ@mail.gmail.com>

I moved imsave into imwrite and wrapped imsave to call imwrite

for backwards compatibility

, in order to be consistent in naming read/write and with other packages

like matlab and opencv.


I

would like to

submit it

and related test

as a pull request

.

My last email messed up on my phone without me realizing.  I'm sending this
message again hopefully with better formatting.  Is there a way to remove
the previous posted message?

Thanks,
Alan
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scipy-dev/attachments/20151103/c01919fc/attachment.html>

From njs at pobox.com  Tue Nov  3 18:25:37 2015
From: njs at pobox.com (Nathaniel Smith)
Date: Tue, 3 Nov 2015 15:25:37 -0800
Subject: [SciPy-Dev] Retiring wiki.scipy.org?
In-Reply-To: <n1bdjm$rkk$1@ger.gmane.org>
References: <loom.20151025T180550-902@post.gmane.org>
 <CABL7CQjpdiwaMnZky+nQaOfeGHcob0ApZ=RCRMrrVq=Ctun3Rw@mail.gmail.com>
 <CAPJVwB=34hfr7X0-Tp_48kPXUCmHd8QHMNjPf3gPQ0SFzwZkvg@mail.gmail.com>
 <n1bdjm$rkk$1@ger.gmane.org>
Message-ID: <CAPJVwBmSDLxwT8TdnbMBU3RLhnNAsC1KmOJde=gCmkXqPuDZ4A@mail.gmail.com>

On Tue, Nov 3, 2015 at 2:48 PM, Pauli Virtanen <pav at iki.fi> wrote:
> Mon, 02 Nov 2015 18:06:51 -0800, Nathaniel Smith kirjoitti:
> [clip]
>> Concrete proposal:
>>
>> - take @rkern's dump of the underlying wiki data, and dump it into a git
>> repo at github.com/scipy/old-wiki for archival purposes
>> - take @pv's scrape of the rendered html pages, and dump them
>> into the same repo, but on the gh-pages branch
>
> Tuumasta toimeen: https://github.com/scipy/old-wiki/
>
> The dump file itself was too large for Github (100M limit), so it's there
> as a "release" binary file.

Awesome!

-n

-- 
Nathaniel J. Smith -- http://vorpus.org


From alan.c.wu at gmail.com  Tue Nov  3 18:31:31 2015
From: alan.c.wu at gmail.com (Alan Wu)
Date: Tue, 3 Nov 2015 15:31:31 -0800
Subject: [SciPy-Dev] Fwd: Imsave and imwrite
In-Reply-To: <CABkLPOKJwsjTkJioXkuOkoqMugENrxtMCVF2On1M=+a4zX8kUQ@mail.gmail.com>
References: <CABkLPOKiP-uR57BK3pOTLBEJ9amma758R4=LabsYf6EWKJ4HvQ@mail.gmail.com>
 <CABkLPOLVOQHhKUX_2i+wq1MzHfzDaY-2JAWVAQJVHh9VdKue2Q@mail.gmail.com>
 <CABkLPOKJwsjTkJioXkuOkoqMugENrxtMCVF2On1M=+a4zX8kUQ@mail.gmail.com>
Message-ID: <CABkLPOKmS3LzJafNEE5C7vUsjk+BkeeRCCkjvBX=XLtDn9mtOA@mail.gmail.com>

I moved imsave into imwrite and wrapped imsave to call imwrite for
backwards compatibility,
in order to be consistent in naming read/write and with other packages
like matlab and opencv.

I would like to submit it and related test as a pull request.

My previous emails messed up on my phone after sending.  I'm sending
this message again
on my desktop as plain text, hopefully with better formatting.  Is
there a way to remove the previous
posted messages?

Thanks,
Alan


From hherbol at gmail.com  Wed Nov  4 16:47:59 2015
From: hherbol at gmail.com (Henry Herbol)
Date: Wed, 4 Nov 2015 16:47:59 -0500
Subject: [SciPy-Dev] Pull Request 5318 - BFGS(Hess)
Message-ID: <CADzNQgmHxo1wfZgLc5Av4SGXEGa9ROvmXwcf8iiLkVi8zc99rA@mail.gmail.com>

Hey Everyone,

    Back in early October I had submitted a Pull Request that's still Open.
I was wondering if anyone had anymore feedback in regards to the request.
It was for the addition of a backtracking BFGS algorithm to work around a
case in which the optimizer only has access to the gradient of the function.

Henry Herbol
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scipy-dev/attachments/20151104/da974b99/attachment.html>

From has2k1 at gmail.com  Wed Nov  4 17:30:37 2015
From: has2k1 at gmail.com (Hassan Kibirige)
Date: Wed, 04 Nov 2015 16:30:37 -0600
Subject: [SciPy-Dev] Loess license
Message-ID: <3069208.zjzZCzNJWk@system>

I am resurrecting the loess code that was part of the deleted `scipy.sandbox` 
sub-package. It turns out that R uses the same core fortran code as it was 
published under a permissive licence (BSD-3 spirit) [1].  

The issue is, there are patches [2] that have been submitted to R and not to 
the upstream source [3]. The code in R still carries the permissive upstream 
licence [4]. Is it okay to adopt those changes?

[1] http://git.io/vlpfV
[2] http://git.io/vlpvF
[3] http://netlib.org/a/loess
[4] http://git.io/vlxj7


Hassan


From shoyer at gmail.com  Wed Nov  4 19:37:31 2015
From: shoyer at gmail.com (Stephan Hoyer)
Date: Wed, 4 Nov 2015 16:37:31 -0800
Subject: [SciPy-Dev] Loess license
In-Reply-To: <3069208.zjzZCzNJWk@system>
References: <3069208.zjzZCzNJWk@system>
Message-ID: <CAEQ_Tvfd4oTVrCtg+OOqyPWAzVhaZoOVnyrNu_JeHTSzhUfCGQ@mail.gmail.com>

So unfortunately, R is GPL licensed. These new contributions don't have a
specific license attached, but arguably as new contributions they are
actually licensed under GPL, not the original license. I'm not a lawyer,
but I think the safest path forward would be to contact the original
authors of the R patch and ask if they are willing to relicense it as BSD.

Best,
Stephan

On Wed, Nov 4, 2015 at 2:30 PM, Hassan Kibirige <has2k1 at gmail.com> wrote:

> I am resurrecting the loess code that was part of the deleted
> `scipy.sandbox`
> sub-package. It turns out that R uses the same core fortran code as it was
> published under a permissive licence (BSD-3 spirit) [1].
>
> The issue is, there are patches [2] that have been submitted to R and not
> to
> the upstream source [3]. The code in R still carries the permissive
> upstream
> licence [4]. Is it okay to adopt those changes?
>
> [1] http://git.io/vlpfV
> [2] http://git.io/vlpvF
> [3] http://netlib.org/a/loess
> [4] http://git.io/vlxj7
>
>
> Hassan
> _______________________________________________
> SciPy-Dev mailing list
> SciPy-Dev at scipy.org
> https://mail.scipy.org/mailman/listinfo/scipy-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scipy-dev/attachments/20151104/eda0b9ec/attachment.html>

From andy.terrel at gmail.com  Wed Nov  4 21:53:54 2015
From: andy.terrel at gmail.com (Andy Ray Terrel)
Date: Wed, 4 Nov 2015 20:53:54 -0600
Subject: [SciPy-Dev] Loess license
In-Reply-To: <CAEQ_Tvfd4oTVrCtg+OOqyPWAzVhaZoOVnyrNu_JeHTSzhUfCGQ@mail.gmail.com>
References: <3069208.zjzZCzNJWk@system>
 <CAEQ_Tvfd4oTVrCtg+OOqyPWAzVhaZoOVnyrNu_JeHTSzhUfCGQ@mail.gmail.com>
Message-ID: <CA+WonSR-PxkafQTeeDLjcSOnoZLUkxTg86et2+9s0D=4O3nxcQ@mail.gmail.com>

This one ( http://git.io/vlh72 ) is still very much GPL with marking on
every file. All the rest seem pretty minor (whitespace and edits from
compiler warnings).

-- Andy

On Wed, Nov 4, 2015 at 6:37 PM, Stephan Hoyer <shoyer at gmail.com> wrote:

> So unfortunately, R is GPL licensed. These new contributions don't have a
> specific license attached, but arguably as new contributions they are
> actually licensed under GPL, not the original license. I'm not a lawyer,
> but I think the safest path forward would be to contact the original
> authors of the R patch and ask if they are willing to relicense it as BSD.
>
> Best,
> Stephan
>
> On Wed, Nov 4, 2015 at 2:30 PM, Hassan Kibirige <has2k1 at gmail.com> wrote:
>
>> I am resurrecting the loess code that was part of the deleted
>> `scipy.sandbox`
>> sub-package. It turns out that R uses the same core fortran code as it was
>> published under a permissive licence (BSD-3 spirit) [1].
>>
>> The issue is, there are patches [2] that have been submitted to R and not
>> to
>> the upstream source [3]. The code in R still carries the permissive
>> upstream
>> licence [4]. Is it okay to adopt those changes?
>>
>> [1] http://git.io/vlpfV
>> [2] http://git.io/vlpvF
>> [3] http://netlib.org/a/loess
>> [4] http://git.io/vlxj7
>>
>>
>> Hassan
>> _______________________________________________
>> SciPy-Dev mailing list
>> SciPy-Dev at scipy.org
>> https://mail.scipy.org/mailman/listinfo/scipy-dev
>>
>
>
> _______________________________________________
> SciPy-Dev mailing list
> SciPy-Dev at scipy.org
> https://mail.scipy.org/mailman/listinfo/scipy-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scipy-dev/attachments/20151104/a74a9133/attachment.html>

From has2k1 at gmail.com  Wed Nov  4 22:27:14 2015
From: has2k1 at gmail.com (Hassan Kibirige)
Date: Wed, 04 Nov 2015 21:27:14 -0600
Subject: [SciPy-Dev] Loess license
In-Reply-To: <CA+WonSR-PxkafQTeeDLjcSOnoZLUkxTg86et2+9s0D=4O3nxcQ@mail.gmail.com>
References: <3069208.zjzZCzNJWk@system>
 <CAEQ_Tvfd4oTVrCtg+OOqyPWAzVhaZoOVnyrNu_JeHTSzhUfCGQ@mail.gmail.com>
 <CA+WonSR-PxkafQTeeDLjcSOnoZLUkxTg86et2+9s0D=4O3nxcQ@mail.gmail.com>
Message-ID: <180840970.jFacdyE3W7@system>

I am not worried about the compiler warnings, (there are 4 according
to gcc-4.9.3). There are two key patches:

1. http://git.io/vlhNP
2. http://git.io/vlhNQ

I have not looked into the details of the problems they address however,
I think the 2nd stems from this issue.

https://stat.ethz.ch/pipermail/r-devel/2009-March/052392.html

Hassan


On Wednesday 04 November 2015 8:53:54 PM Andy Ray Terrel wrote:
> This one ( http://git.io/vlh72 ) is still very much GPL with marking on
> every file. All the rest seem pretty minor (whitespace and edits from
> compiler warnings).
> 
> -- Andy
> 
> On Wed, Nov 4, 2015 at 6:37 PM, Stephan Hoyer <shoyer at gmail.com> wrote:
> > So unfortunately, R is GPL licensed. These new contributions don't have a
> > specific license attached, but arguably as new contributions they are
> > actually licensed under GPL, not the original license. I'm not a lawyer,
> > but I think the safest path forward would be to contact the original
> > authors of the R patch and ask if they are willing to relicense it as BSD.
> > 
> > Best,
> > Stephan
> > 
> > On Wed, Nov 4, 2015 at 2:30 PM, Hassan Kibirige <has2k1 at gmail.com> wrote:
> >> I am resurrecting the loess code that was part of the deleted
> >> `scipy.sandbox`
> >> sub-package. It turns out that R uses the same core fortran code as it
> >> was
> >> published under a permissive licence (BSD-3 spirit) [1].
> >> 
> >> The issue is, there are patches [2] that have been submitted to R and not
> >> to
> >> the upstream source [3]. The code in R still carries the permissive
> >> upstream
> >> licence [4]. Is it okay to adopt those changes?
> >> 
> >> [1] http://git.io/vlpfV
> >> [2] http://git.io/vlpvF
> >> [3] http://netlib.org/a/loess
> >> [4] http://git.io/vlxj7
> >> 
> >> 
> >> Hassan
> >> _______________________________________________
> >> SciPy-Dev mailing list
> >> SciPy-Dev at scipy.org
> >> https://mail.scipy.org/mailman/listinfo/scipy-dev
> > 
> > _______________________________________________
> > SciPy-Dev mailing list
> > SciPy-Dev at scipy.org
> > https://mail.scipy.org/mailman/listinfo/scipy-dev


From andy.terrel at gmail.com  Wed Nov  4 22:58:27 2015
From: andy.terrel at gmail.com (Andy Ray Terrel)
Date: Wed, 4 Nov 2015 21:58:27 -0600
Subject: [SciPy-Dev] Loess license
In-Reply-To: <180840970.jFacdyE3W7@system>
References: <3069208.zjzZCzNJWk@system>
 <CAEQ_Tvfd4oTVrCtg+OOqyPWAzVhaZoOVnyrNu_JeHTSzhUfCGQ@mail.gmail.com>
 <CA+WonSR-PxkafQTeeDLjcSOnoZLUkxTg86et2+9s0D=4O3nxcQ@mail.gmail.com>
 <180840970.jFacdyE3W7@system>
Message-ID: <CA+WonSSb75VUV7sdM=vduAz7bLX-bD1u26Zmb07qJAO+g3gs7A@mail.gmail.com>

On Wed, Nov 4, 2015 at 9:27 PM, Hassan Kibirige <has2k1 at gmail.com> wrote:

> I am not worried about the compiler warnings, (there are 4 according
> to gcc-4.9.3). There are two key patches:
>
> 1. http://git.io/vlhNP



This patch was pushed up stream.
https://stat.ethz.ch/pipermail/r-devel/2009-March/052439.html


>
> 2. http://git.io/vlhNQ


This one probably requires permission from Prof. Ripley (
https://stat.ethz.ch/pipermail/r-devel/2009-March/052542.html )
alternatively the bug only exists for order 0 (as the original code
required).


>
>
> I have not looked into the details of the problems they address however,
> I think the 2nd stems from this issue.
>
> https://stat.ethz.ch/pipermail/r-devel/2009-March/052392.html
>
> Hassan
>
>
> On Wednesday 04 November 2015 8:53:54 PM Andy Ray Terrel wrote:
> > This one ( http://git.io/vlh72 ) is still very much GPL with marking on
> > every file. All the rest seem pretty minor (whitespace and edits from
> > compiler warnings).
> >
> > -- Andy
> >
> > On Wed, Nov 4, 2015 at 6:37 PM, Stephan Hoyer <shoyer at gmail.com> wrote:
> > > So unfortunately, R is GPL licensed. These new contributions don't
> have a
> > > specific license attached, but arguably as new contributions they are
> > > actually licensed under GPL, not the original license. I'm not a
> lawyer,
> > > but I think the safest path forward would be to contact the original
> > > authors of the R patch and ask if they are willing to relicense it as
> BSD.
> > >
> > > Best,
> > > Stephan
> > >
> > > On Wed, Nov 4, 2015 at 2:30 PM, Hassan Kibirige <has2k1 at gmail.com>
> wrote:
> > >> I am resurrecting the loess code that was part of the deleted
> > >> `scipy.sandbox`
> > >> sub-package. It turns out that R uses the same core fortran code as it
> > >> was
> > >> published under a permissive licence (BSD-3 spirit) [1].
> > >>
> > >> The issue is, there are patches [2] that have been submitted to R and
> not
> > >> to
> > >> the upstream source [3]. The code in R still carries the permissive
> > >> upstream
> > >> licence [4]. Is it okay to adopt those changes?
> > >>
> > >> [1] http://git.io/vlpfV
> > >> [2] http://git.io/vlpvF
> > >> [3] http://netlib.org/a/loess
> > >> [4] http://git.io/vlxj7
> > >>
> > >>
> > >> Hassan
> > >> _______________________________________________
> > >> SciPy-Dev mailing list
> > >> SciPy-Dev at scipy.org
> > >> https://mail.scipy.org/mailman/listinfo/scipy-dev
> > >
> > > _______________________________________________
> > > SciPy-Dev mailing list
> > > SciPy-Dev at scipy.org
> > > https://mail.scipy.org/mailman/listinfo/scipy-dev
> _______________________________________________
> SciPy-Dev mailing list
> SciPy-Dev at scipy.org
> https://mail.scipy.org/mailman/listinfo/scipy-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scipy-dev/attachments/20151104/6f88ff18/attachment.html>

From has2k1 at gmail.com  Wed Nov  4 23:21:04 2015
From: has2k1 at gmail.com (Hassan Kibirige)
Date: Wed, 04 Nov 2015 22:21:04 -0600
Subject: [SciPy-Dev] Loess license
In-Reply-To: <CA+WonSSb75VUV7sdM=vduAz7bLX-bD1u26Zmb07qJAO+g3gs7A@mail.gmail.com>
References: <3069208.zjzZCzNJWk@system> <180840970.jFacdyE3W7@system>
 <CA+WonSSb75VUV7sdM=vduAz7bLX-bD1u26Zmb07qJAO+g3gs7A@mail.gmail.com>
Message-ID: <7310767.iLf9WYGFUt@system>

On Wednesday 04 November 2015 9:58:27 PM Andy Ray Terrel wrote:
> On Wed, Nov 4, 2015 at 9:27 PM, Hassan Kibirige <has2k1 at gmail.com> wrote:
> > I am not worried about the compiler warnings, (there are 4 according
> > to gcc-4.9.3). There are two key patches:
> > 
> > 1. http://git.io/vlhNP
> 
> This patch was pushed up stream.
> https://stat.ethz.ch/pipermail/r-devel/2009-March/052439.html
> 
The dates are misleading, indeed the changes are reflected upstream.

> > 2. http://git.io/vlhNQ
> 
> This one probably requires permission from Prof. Ripley (
> https://stat.ethz.ch/pipermail/r-devel/2009-March/052542.html )
> alternatively the bug only exists for order 0 (as the original code
> required).
> 
OK that is not a show stopper.

> > I have not looked into the details of the problems they address however,
> > I think the 2nd stems from this issue.
> > 
> > https://stat.ethz.ch/pipermail/r-devel/2009-March/052392.html
> > 
> > Hassan
> > 
> > On Wednesday 04 November 2015 8:53:54 PM Andy Ray Terrel wrote:
> > > This one ( http://git.io/vlh72 ) is still very much GPL with marking on
> > > every file. All the rest seem pretty minor (whitespace and edits from
> > > compiler warnings).
> > > 
> > > -- Andy
> > > 
> > > On Wed, Nov 4, 2015 at 6:37 PM, Stephan Hoyer <shoyer at gmail.com> wrote:
> > > > So unfortunately, R is GPL licensed. These new contributions don't
> > 
> > have a
> > 
> > > > specific license attached, but arguably as new contributions they are
> > > > actually licensed under GPL, not the original license. I'm not a
> > 
> > lawyer,
> > 
> > > > but I think the safest path forward would be to contact the original
> > > > authors of the R patch and ask if they are willing to relicense it as
> > 
> > BSD.
> > 
> > > > Best,
> > > > Stephan
> > > > 
> > > > On Wed, Nov 4, 2015 at 2:30 PM, Hassan Kibirige <has2k1 at gmail.com>
> > 
> > wrote:
> > > >> I am resurrecting the loess code that was part of the deleted
> > > >> `scipy.sandbox`
> > > >> sub-package. It turns out that R uses the same core fortran code as
> > > >> it
> > > >> was
> > > >> published under a permissive licence (BSD-3 spirit) [1].
> > > >> 
> > > >> The issue is, there are patches [2] that have been submitted to R and
> > 
> > not
> > 
> > > >> to
> > > >> the upstream source [3]. The code in R still carries the permissive
> > > >> upstream
> > > >> licence [4]. Is it okay to adopt those changes?
> > > >> 
> > > >> [1] http://git.io/vlpfV
> > > >> [2] http://git.io/vlpvF
> > > >> [3] http://netlib.org/a/loess
> > > >> [4] http://git.io/vlxj7
> > > >> 
> > > >> 
> > > >> Hassan
> > > >> _______________________________________________
> > > >> SciPy-Dev mailing list
> > > >> SciPy-Dev at scipy.org
> > > >> https://mail.scipy.org/mailman/listinfo/scipy-dev
> > > > 
> > > > _______________________________________________
> > > > SciPy-Dev mailing list
> > > > SciPy-Dev at scipy.org
> > > > https://mail.scipy.org/mailman/listinfo/scipy-dev
> > 
> > _______________________________________________
> > SciPy-Dev mailing list
> > SciPy-Dev at scipy.org
> > https://mail.scipy.org/mailman/listinfo/scipy-dev


From stefanv at berkeley.edu  Fri Nov  6 12:32:53 2015
From: stefanv at berkeley.edu (Stefan van der Walt)
Date: Fri, 06 Nov 2015 09:32:53 -0800
Subject: [SciPy-Dev] Help wanted: implementation of 3D medial axis
	skeletonization
References: <87vb9khwbd.fsf@berkeley.edu>
Message-ID: <87oaf7f2yi.fsf@berkeley.edu>

Hi all,

I have been approached by a group that is interested in sponsoring the
development of 3D skeletonization in scikit-image.  One potential
starting place would be:

http://www.insight-journal.org/browse/publication/181

Is anyone interested in working on this?  Please get in touch either on
the scikit-image mailing list or by mailing me directly.

Thanks!
St?fan


From ralf.gommers at gmail.com  Sat Nov  7 06:09:36 2015
From: ralf.gommers at gmail.com (Ralf Gommers)
Date: Sat, 7 Nov 2015 12:09:36 +0100
Subject: [SciPy-Dev] deleting scipy/speed repo
In-Reply-To: <CABL7CQg+06quqXG4xyWtFt8+FhX+Qwh1xNqeqsqvZGrJ2FDgfA@mail.gmail.com>
References: <CABL7CQg+06quqXG4xyWtFt8+FhX+Qwh1xNqeqsqvZGrJ2FDgfA@mail.gmail.com>
Message-ID: <CABL7CQgYqdiWPBbbZbHWj-AN=70NfBmEPPWwgx_AcHbDrhXrEQ@mail.gmail.com>

On Sat, Oct 31, 2015 at 12:49 PM, Ralf Gommers <ralf.gommers at gmail.com>
wrote:

> Hi,
>
> Does anyone object to deleting https://github.com/scipy/speed? There's
> not more than 200 lines of code in there and no one uses that repo anymore.
>
> The code that's in there is Python/Cython/Fortran code to solve the
> Laplace problem. For people interested in that, this blog post from Travis
> is much nicer than the repo:
> http://technicaldiscovery.blogspot.nl/2011/06/speeding-up-python-numpy-cython-and.html
>
> The only thing the blog post doesn't have is the Fortran code, but it's
> linked in the comments and originally came from
> https://github.com/certik/laplace_test
>
> Conclusion: nothing to move, just delete.
>

Done. I've made a backup at https://github.com/rgommers/speed just in case,
but I don't think anyone will miss this repo.

Ralf
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scipy-dev/attachments/20151107/dcbfb8ea/attachment.html>

From warren.weckesser at gmail.com  Sat Nov  7 15:29:33 2015
From: warren.weckesser at gmail.com (Warren Weckesser)
Date: Sat, 7 Nov 2015 15:29:33 -0500
Subject: [SciPy-Dev] deleting scipy/speed repo
In-Reply-To: <CABL7CQgYqdiWPBbbZbHWj-AN=70NfBmEPPWwgx_AcHbDrhXrEQ@mail.gmail.com>
References: <CABL7CQg+06quqXG4xyWtFt8+FhX+Qwh1xNqeqsqvZGrJ2FDgfA@mail.gmail.com>
 <CABL7CQgYqdiWPBbbZbHWj-AN=70NfBmEPPWwgx_AcHbDrhXrEQ@mail.gmail.com>
Message-ID: <CAGzF1uc1JTcuCm1hPio5M1kwok_9g8VcZpdZ84+ksZYc0Zde1Q@mail.gmail.com>

On Sat, Nov 7, 2015 at 6:09 AM, Ralf Gommers <ralf.gommers at gmail.com> wrote:

>
>
> On Sat, Oct 31, 2015 at 12:49 PM, Ralf Gommers <ralf.gommers at gmail.com>
> wrote:
>
>> Hi,
>>
>> Does anyone object to deleting https://github.com/scipy/speed? There's
>> not more than 200 lines of code in there and no one uses that repo anymore.
>>
>> The code that's in there is Python/Cython/Fortran code to solve the
>> Laplace problem. For people interested in that, this blog post from Travis
>> is much nicer than the repo:
>> http://technicaldiscovery.blogspot.nl/2011/06/speeding-up-python-numpy-cython-and.html
>>
>> The only thing the blog post doesn't have is the Fortran code, but it's
>> linked in the comments and originally came from
>> https://github.com/certik/laplace_test
>>
>> Conclusion: nothing to move, just delete.
>>
>
> Done. I've made a backup at https://github.com/rgommers/speed just in
> case, but I don't think anyone will miss this repo.
>
>

Maintenance tasks like this can be tedious and thankless, so to help
alleviate the latter: thanks Ralf!

Warren



> Ralf
>
>
>
> _______________________________________________
> SciPy-Dev mailing list
> SciPy-Dev at scipy.org
> https://mail.scipy.org/mailman/listinfo/scipy-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scipy-dev/attachments/20151107/8d5cd0de/attachment.html>

From ralf.gommers at gmail.com  Sat Nov  7 16:32:27 2015
From: ralf.gommers at gmail.com (Ralf Gommers)
Date: Sat, 7 Nov 2015 22:32:27 +0100
Subject: [SciPy-Dev] Fwd: Imsave and imwrite
In-Reply-To: <CABkLPOKmS3LzJafNEE5C7vUsjk+BkeeRCCkjvBX=XLtDn9mtOA@mail.gmail.com>
References: <CABkLPOKiP-uR57BK3pOTLBEJ9amma758R4=LabsYf6EWKJ4HvQ@mail.gmail.com>
 <CABkLPOLVOQHhKUX_2i+wq1MzHfzDaY-2JAWVAQJVHh9VdKue2Q@mail.gmail.com>
 <CABkLPOKJwsjTkJioXkuOkoqMugENrxtMCVF2On1M=+a4zX8kUQ@mail.gmail.com>
 <CABkLPOKmS3LzJafNEE5C7vUsjk+BkeeRCCkjvBX=XLtDn9mtOA@mail.gmail.com>
Message-ID: <CABL7CQiPM6aqw4vWmcPYj+4zvbVbvVHRfeAt0rHVoiD56oZUrg@mail.gmail.com>

On Wed, Nov 4, 2015 at 12:31 AM, Alan Wu <alan.c.wu at gmail.com> wrote:

> I moved imsave into imwrite and wrapped imsave to call imwrite for
> backwards compatibility,
> in order to be consistent in naming read/write and with other packages
> like matlab and opencv.
>

> I would like to submit it and related test as a pull request.
>

Alan submitted this PR at https://github.com/scipy/scipy/pull/5458

On the PR I commented already that the name change doesn't really warrant a
deprecation imho. Both "imsave" and "imwrite" are pretty clear. The latter
may be slightly more precise and consistent with OpenCV and Matlab, but on
the other hand Matplotlib also uses "imsave".

However: we do want to get rid of scipy.misc. At least I do. So the useful
stuff in there (not that much) needs to move. We recently discussed
"imread" on a PR, and that has two versions - one in misc and one in
ndimage. Warren suggested that the most logical place for it would actually
be scipy.io. Which makes sense.

If we add an image read and save function to scipy.io, then we're free to
choose the names of those functions. So:
- long-term, do we want to keep an imread and an imsave/write function?
- if so, what's the preferred name?


My previous emails messed up on my phone after sending.  I'm sending
> this message again
> on my desktop as plain text, hopefully with better formatting.  Is
> there a way to remove the previous
> posted messages?
>

There's no way to remove posted messages.

Cheers,
Ralf
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scipy-dev/attachments/20151107/b35bf009/attachment.html>

From evgeny.burovskiy at gmail.com  Sat Nov  7 17:14:46 2015
From: evgeny.burovskiy at gmail.com (Evgeni Burovski)
Date: Sat, 7 Nov 2015 22:14:46 +0000
Subject: [SciPy-Dev] plans for 0.17.0 release
In-Reply-To: <CABL7CQhJPnwAz+WGanP_HbASDGOVx=_Kh4zztb2hyZU-acuD3w@mail.gmail.com>
References: <CABL7CQhJPnwAz+WGanP_HbASDGOVx=_Kh4zztb2hyZU-acuD3w@mail.gmail.com>
Message-ID: <CAMRo0iuDJ27sNybHFD2JLBRZtQvtYjpYaPO-YxWba8eM6zuOFg@mail.gmail.com>

On Sun, Oct 25, 2015 at 5:14 PM, Ralf Gommers <ralf.gommers at gmail.com> wrote:
> Hi all,
>
> We branched 0.16.x in early May, so it's time to start thinking about
> branching 0.17.x sometime soon. If anyone would be interested in being the
> release manager for 0.17.x, or co-managing it with me, that would be very
> welcome.
>
> In order to make releasing look less like black magic I've worked on
> automating stuff a bit more (has landed in master) and on writing fairly
> detailed documentation: https://github.com/scipy/scipy/pull/5424. Please
> have a look if you're interested.
>
> Cheers,
> Ralf
>
>
> _______________________________________________
> SciPy-Dev mailing list
> SciPy-Dev at scipy.org
> https://mail.scipy.org/mailman/listinfo/scipy-dev
>


I volunteer to co-manage it with Ralf.

Here's aproposal for the release schedule:

branching 0.17.x: 22 Nov
rc1: 24 Nov
rc2: 30 Nov (if needed)
final release: 14 Dec

There do not seem to be many serious open issues --- here's today's
contents of the milestone 0.17:
https://github.com/scipy/scipy/milestones/0.17.0

If you have an issue or a PR which you would like to go in to 0.17.0,
please attach the milestone or comment on Github, or mention it in
this email thread.

At the moment we have about 140 open PRs, many of which are lingering
unreviewed. If you have bandwidth to spare, the schedule above gives
us two weeks  to review and merge PRs, and any help would very much
appreciated.

Cheers,

Evgeni


From kitchi.srikrishna at gmail.com  Sun Nov  8 05:20:06 2015
From: kitchi.srikrishna at gmail.com (Sri Krishna)
Date: Sun, 8 Nov 2015 15:50:06 +0530
Subject: [SciPy-Dev] Fwd: Imsave and imwrite
In-Reply-To: <CABL7CQiPM6aqw4vWmcPYj+4zvbVbvVHRfeAt0rHVoiD56oZUrg@mail.gmail.com>
References: <CABkLPOKiP-uR57BK3pOTLBEJ9amma758R4=LabsYf6EWKJ4HvQ@mail.gmail.com>
 <CABkLPOLVOQHhKUX_2i+wq1MzHfzDaY-2JAWVAQJVHh9VdKue2Q@mail.gmail.com>
 <CABkLPOKJwsjTkJioXkuOkoqMugENrxtMCVF2On1M=+a4zX8kUQ@mail.gmail.com>
 <CABkLPOKmS3LzJafNEE5C7vUsjk+BkeeRCCkjvBX=XLtDn9mtOA@mail.gmail.com>
 <CABL7CQiPM6aqw4vWmcPYj+4zvbVbvVHRfeAt0rHVoiD56oZUrg@mail.gmail.com>
Message-ID: <CA+qdgMdThrC+AFy7BXOeFwKN_dPAX9NHHMVN5u0kDzQaKMbnrg@mail.gmail.com>

I am partial to `imwrite` rather than `imsave`, because of the
correspondence between read/write rather than read/save (which it would be
if the functions were `imread` and `imsave`).

Since PIL is an explicit dependency, would it be better to offload these
functions into other projects like scikit-image for example? We could
always wrap those functions as "convenience functions" in scipy if required.

Thanks,
Krishna




On 8 November 2015 at 03:02, Ralf Gommers <ralf.gommers at gmail.com> wrote:

>
>
> On Wed, Nov 4, 2015 at 12:31 AM, Alan Wu <alan.c.wu at gmail.com> wrote:
>
>> I moved imsave into imwrite and wrapped imsave to call imwrite for
>> backwards compatibility,
>> in order to be consistent in naming read/write and with other packages
>> like matlab and opencv.
>>
>
>> I would like to submit it and related test as a pull request.
>>
>
> Alan submitted this PR at https://github.com/scipy/scipy/pull/5458
>
> On the PR I commented already that the name change doesn't really warrant
> a deprecation imho. Both "imsave" and "imwrite" are pretty clear. The
> latter may be slightly more precise and consistent with OpenCV and Matlab,
> but on the other hand Matplotlib also uses "imsave".
>
> However: we do want to get rid of scipy.misc. At least I do. So the useful
> stuff in there (not that much) needs to move. We recently discussed
> "imread" on a PR, and that has two versions - one in misc and one in
> ndimage. Warren suggested that the most logical place for it would actually
> be scipy.io. Which makes sense.
>
> If we add an image read and save function to scipy.io, then we're free to
> choose the names of those functions. So:
> - long-term, do we want to keep an imread and an imsave/write function?
> - if so, what's the preferred name?
>
>
> My previous emails messed up on my phone after sending.  I'm sending
>> this message again
>> on my desktop as plain text, hopefully with better formatting.  Is
>> there a way to remove the previous
>> posted messages?
>>
>
> There's no way to remove posted messages.
>
> Cheers,
> Ralf
>
>
> _______________________________________________
> SciPy-Dev mailing list
> SciPy-Dev at scipy.org
> https://mail.scipy.org/mailman/listinfo/scipy-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scipy-dev/attachments/20151108/60226018/attachment.html>

From sturla.molden at gmail.com  Sun Nov  8 09:04:24 2015
From: sturla.molden at gmail.com (Sturla Molden)
Date: Sun, 8 Nov 2015 14:04:24 +0000 (UTC)
Subject: [SciPy-Dev] Imsave and imwrite
References: <CABkLPOLVOQHhKUX_2i+wq1MzHfzDaY-2JAWVAQJVHh9VdKue2Q@mail.gmail.com>
 <CABkLPOKJwsjTkJioXkuOkoqMugENrxtMCVF2On1M=+a4zX8kUQ@mail.gmail.com>
 <CABkLPOKmS3LzJafNEE5C7vUsjk+BkeeRCCkjvBX=XLtDn9mtOA@mail.gmail.com>
 <CABL7CQiPM6aqw4vWmcPYj+4zvbVbvVHRfeAt0rHVoiD56oZUrg@mail.gmail.com>
 <CA+qdgMdThrC+AFy7BXOeFwKN_dPAX9NHHMVN5u0kDzQaKMbnrg@mail.gmail.com>
Message-ID: <2107496194468684195.311093sturla.molden-gmail.com@news.gmane.org>

Sri Krishna <kitchi.srikrishna at gmail.com> wrote:

> Since PIL is an explicit dependency, would it be better to offload these
> functions into other projects like scikit-image for example? We could
> always wrap those functions as "convenience functions" in scipy if required.

PIL is abandonware. We should use pillow or just get rid of them (they
belong in scikit-image anyway).



From ralf.gommers at gmail.com  Sun Nov  8 09:14:12 2015
From: ralf.gommers at gmail.com (Ralf Gommers)
Date: Sun, 8 Nov 2015 15:14:12 +0100
Subject: [SciPy-Dev] Imsave and imwrite
In-Reply-To: <2107496194468684195.311093sturla.molden-gmail.com@news.gmane.org>
References: <CABkLPOLVOQHhKUX_2i+wq1MzHfzDaY-2JAWVAQJVHh9VdKue2Q@mail.gmail.com>
 <CABkLPOKJwsjTkJioXkuOkoqMugENrxtMCVF2On1M=+a4zX8kUQ@mail.gmail.com>
 <CABkLPOKmS3LzJafNEE5C7vUsjk+BkeeRCCkjvBX=XLtDn9mtOA@mail.gmail.com>
 <CABL7CQiPM6aqw4vWmcPYj+4zvbVbvVHRfeAt0rHVoiD56oZUrg@mail.gmail.com>
 <CA+qdgMdThrC+AFy7BXOeFwKN_dPAX9NHHMVN5u0kDzQaKMbnrg@mail.gmail.com>
 <2107496194468684195.311093sturla.molden-gmail.com@news.gmane.org>
Message-ID: <CABL7CQgHD0j7H7bhN0J73zPKo8rmoax0+yVSUZpQxkfJd023Mg@mail.gmail.com>

On Sun, Nov 8, 2015 at 3:04 PM, Sturla Molden <sturla.molden at gmail.com>
wrote:

> Sri Krishna <kitchi.srikrishna at gmail.com> wrote:
>
> > Since PIL is an explicit dependency, would it be better to offload these
> > functions into other projects like scikit-image for example? We could
> > always wrap those functions as "convenience functions" in scipy if
> required.
>
> PIL is abandonware. We should use pillow or just get rid of them (they
> belong in scikit-image anyway).
>

When people say PIL, they mean Pillow. We're also depending on Pillow; if
there are bugs filed against scipy.misc.pilutil that occur with PIL but not
Pillow they'll simply be closed as wontfix.

Ralf



> _______________________________________________
> SciPy-Dev mailing list
> SciPy-Dev at scipy.org
> https://mail.scipy.org/mailman/listinfo/scipy-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scipy-dev/attachments/20151108/e68b7882/attachment.html>

From kitchi.srikrishna at gmail.com  Sun Nov  8 12:44:18 2015
From: kitchi.srikrishna at gmail.com (Sri Krishna)
Date: Sun, 8 Nov 2015 23:14:18 +0530
Subject: [SciPy-Dev] Imsave and imwrite
In-Reply-To: <CABL7CQgHD0j7H7bhN0J73zPKo8rmoax0+yVSUZpQxkfJd023Mg@mail.gmail.com>
References: <CABkLPOLVOQHhKUX_2i+wq1MzHfzDaY-2JAWVAQJVHh9VdKue2Q@mail.gmail.com>
 <CABkLPOKJwsjTkJioXkuOkoqMugENrxtMCVF2On1M=+a4zX8kUQ@mail.gmail.com>
 <CABkLPOKmS3LzJafNEE5C7vUsjk+BkeeRCCkjvBX=XLtDn9mtOA@mail.gmail.com>
 <CABL7CQiPM6aqw4vWmcPYj+4zvbVbvVHRfeAt0rHVoiD56oZUrg@mail.gmail.com>
 <CA+qdgMdThrC+AFy7BXOeFwKN_dPAX9NHHMVN5u0kDzQaKMbnrg@mail.gmail.com>
 <2107496194468684195.311093sturla.molden-gmail.com@news.gmane.org>
 <CABL7CQgHD0j7H7bhN0J73zPKo8rmoax0+yVSUZpQxkfJd023Mg@mail.gmail.com>
Message-ID: <CA+qdgMfoWrxHQ3jwDgUb0rkwGAHsz0CNftYgwdtezHb2Xkfw9A@mail.gmail.com>

On 8 November 2015 at 19:44, Ralf Gommers <ralf.gommers at gmail.com> wrote:

>
>
> On Sun, Nov 8, 2015 at 3:04 PM, Sturla Molden <sturla.molden at gmail.com>
> wrote:
>
>> Sri Krishna <kitchi.srikrishna at gmail.com> wrote:
>>
>> > Since PIL is an explicit dependency, would it be better to offload these
>> > functions into other projects like scikit-image for example? We could
>> > always wrap those functions as "convenience functions" in scipy if
>> required.
>>
>> PIL is abandonware. We should use pillow or just get rid of them (they
>> belong in scikit-image anyway).
>>
>
> When people say PIL, they mean Pillow. We're also depending on Pillow; if
> there are bugs filed against scipy.misc.pilutil that occur with PIL but not
> Pillow they'll simply be closed as wontfix.
>

What I mean is since we are depending on an external library anyway
(Pillow) why not just migrate the function into scikit-image if it doesn't
exist already? Or use pillow's inbuilt file open and close functions?

I don't see a strong reason to keep these functions in scipy since there
are other packages that have similar functionality.

>
> Ralf
>
>
>
>> _______________________________________________
>> SciPy-Dev mailing list
>> SciPy-Dev at scipy.org
>> https://mail.scipy.org/mailman/listinfo/scipy-dev
>>
>
>
> _______________________________________________
> SciPy-Dev mailing list
> SciPy-Dev at scipy.org
> https://mail.scipy.org/mailman/listinfo/scipy-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scipy-dev/attachments/20151108/00c65662/attachment.html>

From warren.weckesser at gmail.com  Mon Nov  9 15:41:39 2015
From: warren.weckesser at gmail.com (Warren Weckesser)
Date: Mon, 9 Nov 2015 15:41:39 -0500
Subject: [SciPy-Dev] Fwd: Imsave and imwrite
In-Reply-To: <CABL7CQiPM6aqw4vWmcPYj+4zvbVbvVHRfeAt0rHVoiD56oZUrg@mail.gmail.com>
References: <CABkLPOKiP-uR57BK3pOTLBEJ9amma758R4=LabsYf6EWKJ4HvQ@mail.gmail.com>
 <CABkLPOLVOQHhKUX_2i+wq1MzHfzDaY-2JAWVAQJVHh9VdKue2Q@mail.gmail.com>
 <CABkLPOKJwsjTkJioXkuOkoqMugENrxtMCVF2On1M=+a4zX8kUQ@mail.gmail.com>
 <CABkLPOKmS3LzJafNEE5C7vUsjk+BkeeRCCkjvBX=XLtDn9mtOA@mail.gmail.com>
 <CABL7CQiPM6aqw4vWmcPYj+4zvbVbvVHRfeAt0rHVoiD56oZUrg@mail.gmail.com>
Message-ID: <CAGzF1udzVqSo73eRLgLjnHc+6N+PK_1UiucPMk5Ft1v-XGYgyg@mail.gmail.com>

On Sat, Nov 7, 2015 at 4:32 PM, Ralf Gommers <ralf.gommers at gmail.com> wrote:

>
>
> On Wed, Nov 4, 2015 at 12:31 AM, Alan Wu <alan.c.wu at gmail.com> wrote:
>
>> I moved imsave into imwrite and wrapped imsave to call imwrite for
>> backwards compatibility,
>> in order to be consistent in naming read/write and with other packages
>> like matlab and opencv.
>>
>
>> I would like to submit it and related test as a pull request.
>>
>
> Alan submitted this PR at https://github.com/scipy/scipy/pull/5458
>
> On the PR I commented already that the name change doesn't really warrant
> a deprecation imho. Both "imsave" and "imwrite" are pretty clear. The
> latter may be slightly more precise and consistent with OpenCV and Matlab,
> but on the other hand Matplotlib also uses "imsave".
>
> However: we do want to get rid of scipy.misc. At least I do. So the useful
> stuff in there (not that much) needs to move. We recently discussed
> "imread" on a PR, and that has two versions - one in misc and one in
> ndimage. Warren suggested that the most logical place for it would actually
> be scipy.io. Which makes sense.
>
> If we add an image read and save function to scipy.io, then we're free to
> choose the names of those functions. So:
> - long-term, do we want to keep an imread and an imsave/write function?
> - if so, what's the preferred name?
>
>

I prefer the read/write verb pairing over read/save, so I'd be fine with
moving `imread` and `imsave` from `misc` to `io` and renaming `imsave` to
`imwrite`.  (The `misc` versions would have the usual deprecation cycle, of
course.)

For comparison, the other I/O functions in `scipy.io` that use either
read/write or load/save are (sorry if the following doesn't come out
properly formatted in your email viewer):

Format         Input          Output
WAV            wavfile.read   wavfile.write
Matrix market  mmread         mmwrite
IDL            readsav
Matlab files   loadmat        savemat
ARFF           arff.loadarff


Warren

P.S. As long as we're on the subject of writing image files, I'll put in a
plug for the package "numpngw" for writing numpy arrays to PNG files,
including animated PNG: https://github.com/WarrenWeckesser/numpngw



>
> My previous emails messed up on my phone after sending.  I'm sending
>> this message again
>> on my desktop as plain text, hopefully with better formatting.  Is
>> there a way to remove the previous
>> posted messages?
>>
>
> There's no way to remove posted messages.
>
> Cheers,
> Ralf
>
>
> _______________________________________________
> SciPy-Dev mailing list
> SciPy-Dev at scipy.org
> https://mail.scipy.org/mailman/listinfo/scipy-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scipy-dev/attachments/20151109/2016a3a0/attachment.html>

From david.mikolas1 at gmail.com  Tue Nov 10 03:37:42 2015
From: david.mikolas1 at gmail.com (David Mikolas)
Date: Tue, 10 Nov 2015 16:37:42 +0800
Subject: [SciPy-Dev] plans for 0.17.0 release
In-Reply-To: <CAMRo0iuDJ27sNybHFD2JLBRZtQvtYjpYaPO-YxWba8eM6zuOFg@mail.gmail.com>
References: <CABL7CQhJPnwAz+WGanP_HbASDGOVx=_Kh4zztb2hyZU-acuD3w@mail.gmail.com>
 <CAMRo0iuDJ27sNybHFD2JLBRZtQvtYjpYaPO-YxWba8eM6zuOFg@mail.gmail.com>
Message-ID: <CAFYhK+7rX4s=vzOWrqF+SD-p3G_8pT2jdD+P-2hGhfd5uaRE2Q@mail.gmail.com>

I am new to SciPy-dev. Will the dense output option in scipy.integrate.ode
become available in 0.17.0? This is a feature already available in the
original FORTAN, but wasn't implemented in the wrapper.

I've just started a battery of tests (all those in Hull et. al. 1972,
below) for personal "fun", but not sure how to report.

https://github.com/jddmartin/scipy/tree/dense_output_from_dopri5_and_dop853
https://github.com/jddmartin/dense_output_example_usage

Comparing Numerical Methods for Ordinary Differential Equations

T.E. Hull, W.H. Enright, B.M. Fellen and A.E. Sedgwick,

SIAM J. Numer. Anal. 9, (4), Dec. 1972,  pp 603-637

On Sun, Nov 8, 2015 at 6:14 AM, Evgeni Burovski <evgeny.burovskiy at gmail.com>
wrote:

> On Sun, Oct 25, 2015 at 5:14 PM, Ralf Gommers <ralf.gommers at gmail.com>
> wrote:
> > Hi all,
> >
> > We branched 0.16.x in early May, so it's time to start thinking about
> > branching 0.17.x sometime soon. If anyone would be interested in being
> the
> > release manager for 0.17.x, or co-managing it with me, that would be very
> > welcome.
> >
> > In order to make releasing look less like black magic I've worked on
> > automating stuff a bit more (has landed in master) and on writing fairly
> > detailed documentation: https://github.com/scipy/scipy/pull/5424. Please
> > have a look if you're interested.
> >
> > Cheers,
> > Ralf
> >
> >
> > _______________________________________________
> > SciPy-Dev mailing list
> > SciPy-Dev at scipy.org
> > https://mail.scipy.org/mailman/listinfo/scipy-dev
> >
>
>
> I volunteer to co-manage it with Ralf.
>
> Here's aproposal for the release schedule:
>
> branching 0.17.x: 22 Nov
> rc1: 24 Nov
> rc2: 30 Nov (if needed)
> final release: 14 Dec
>
> There do not seem to be many serious open issues --- here's today's
> contents of the milestone 0.17:
> https://github.com/scipy/scipy/milestones/0.17.0
>
> If you have an issue or a PR which you would like to go in to 0.17.0,
> please attach the milestone or comment on Github, or mention it in
> this email thread.
>
> At the moment we have about 140 open PRs, many of which are lingering
> unreviewed. If you have bandwidth to spare, the schedule above gives
> us two weeks  to review and merge PRs, and any help would very much
> appreciated.
>
> Cheers,
>
> Evgeni
> _______________________________________________
> SciPy-Dev mailing list
> SciPy-Dev at scipy.org
> https://mail.scipy.org/mailman/listinfo/scipy-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scipy-dev/attachments/20151110/11587920/attachment.html>

From jf4martin at gmail.com  Tue Nov 10 08:12:03 2015
From: jf4martin at gmail.com (Jim Martin)
Date: Tue, 10 Nov 2015 08:12:03 -0500
Subject: [SciPy-Dev] plans for 0.17.0 release
In-Reply-To: <CAFYhK+7rX4s=vzOWrqF+SD-p3G_8pT2jdD+P-2hGhfd5uaRE2Q@mail.gmail.com>
References: <CABL7CQhJPnwAz+WGanP_HbASDGOVx=_Kh4zztb2hyZU-acuD3w@mail.gmail.com>
 <CAMRo0iuDJ27sNybHFD2JLBRZtQvtYjpYaPO-YxWba8eM6zuOFg@mail.gmail.com>
 <CAFYhK+7rX4s=vzOWrqF+SD-p3G_8pT2jdD+P-2hGhfd5uaRE2Q@mail.gmail.com>
Message-ID: <CAE8oJo4VCnEWgHc0a6igf-f7116f71ddx=Nx_fQtyU4FkfvFfQ@mail.gmail.com>

Hi David and Scipy developers

I wrote these dense output extensions that you listed:

https://github.com/jddmartin/scipy/tree/dense_output_from_dopri5_and_dop853
https://github.com/jddmartin/dense_output_example_usage

but I didn't issue any pull request to scipy.  I sent this message to the
scipy developers list:
  http://article.gmane.org/gmane.comp.python.scientific.devel/19635/
explaining the changes.  I was hoping for some feedback before issuing a
pull request.

Perhaps a scipy developer could comment on these proposed changes?  I tried
to be as complete as possible in the mailing list message.   These changes
come with 100% test coverage, documentation and are backwards compatible.
They do seem to satisfy a need ...

Any feedback is much appreciated.

Thanks! -- Jim.




On Tue, Nov 10, 2015 at 3:37 AM, David Mikolas <david.mikolas1 at gmail.com>
wrote:

> I am new to SciPy-dev. Will the dense output option in scipy.integrate.ode
> become available in 0.17.0? This is a feature already available in the
> original FORTAN, but wasn't implemented in the wrapper.
>
> I've just started a battery of tests (all those in Hull et. al. 1972,
> below) for personal "fun", but not sure how to report.
>
> https://github.com/jddmartin/scipy/tree/dense_output_from_dopri5_and_dop853
> https://github.com/jddmartin/dense_output_example_usage
>
> Comparing Numerical Methods for Ordinary Differential Equations
>
> T.E. Hull, W.H. Enright, B.M. Fellen and A.E. Sedgwick,
>
> SIAM J. Numer. Anal. 9, (4), Dec. 1972,  pp 603-637
>
> On Sun, Nov 8, 2015 at 6:14 AM, Evgeni Burovski <
> evgeny.burovskiy at gmail.com> wrote:
>
>> On Sun, Oct 25, 2015 at 5:14 PM, Ralf Gommers <ralf.gommers at gmail.com>
>> wrote:
>> > Hi all,
>> >
>> > We branched 0.16.x in early May, so it's time to start thinking about
>> > branching 0.17.x sometime soon. If anyone would be interested in being
>> the
>> > release manager for 0.17.x, or co-managing it with me, that would be
>> very
>> > welcome.
>> >
>> > In order to make releasing look less like black magic I've worked on
>> > automating stuff a bit more (has landed in master) and on writing fairly
>> > detailed documentation: https://github.com/scipy/scipy/pull/5424.
>> Please
>> > have a look if you're interested.
>> >
>> > Cheers,
>> > Ralf
>> >
>> >
>> > _______________________________________________
>> > SciPy-Dev mailing list
>> > SciPy-Dev at scipy.org
>> > https://mail.scipy.org/mailman/listinfo/scipy-dev
>> >
>>
>>
>> I volunteer to co-manage it with Ralf.
>>
>> Here's aproposal for the release schedule:
>>
>> branching 0.17.x: 22 Nov
>> rc1: 24 Nov
>> rc2: 30 Nov (if needed)
>> final release: 14 Dec
>>
>> There do not seem to be many serious open issues --- here's today's
>> contents of the milestone 0.17:
>> https://github.com/scipy/scipy/milestones/0.17.0
>>
>> If you have an issue or a PR which you would like to go in to 0.17.0,
>> please attach the milestone or comment on Github, or mention it in
>> this email thread.
>>
>> At the moment we have about 140 open PRs, many of which are lingering
>> unreviewed. If you have bandwidth to spare, the schedule above gives
>> us two weeks  to review and merge PRs, and any help would very much
>> appreciated.
>>
>> Cheers,
>>
>> Evgeni
>> _______________________________________________
>> SciPy-Dev mailing list
>> SciPy-Dev at scipy.org
>> https://mail.scipy.org/mailman/listinfo/scipy-dev
>>
>
>
> _______________________________________________
> SciPy-Dev mailing list
> SciPy-Dev at scipy.org
> https://mail.scipy.org/mailman/listinfo/scipy-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scipy-dev/attachments/20151110/dd3071eb/attachment.html>

From Sylvain.Gubian at pmi.com  Tue Nov 10 09:05:36 2015
From: Sylvain.Gubian at pmi.com (Gubian, Sylvain)
Date: Tue, 10 Nov 2015 14:05:36 +0000
Subject: [SciPy-Dev] optimize - add algorithm for global optimization:
	GenSA
Message-ID: <A3786E9825AE3D4F8CC0EC80FBFAD32E02349A58CC@PMICHLAUEXM23.PMINTL.NET>

Dear All,

Thanks a lot for Jake, Andrew, Chuck and Sturla comments.
I would like to address some comments so that I would know how to go forward, like doing a pull request whenever you think it makes sense.

@Jake: As you mentioned, there are situations where basinhopping may not work as well as PyGenSA, especially in cases of high number of parameters. This implementation has actually been focused on solving high dimension problems. However, it also works for very simple problems, but in those cases it might show non-optimal performance (in terms of avg number of function calls). The battery test shows that in general, the number of successes is almost always better than basinhopping, PyGenSA is a robust implementation.
As attached, a PDF version of the poster I presented to PyCon UK Coventry in September 2015 showing a performance benchmark.

@Chuck: About license, I am the author and maintainer of the R version. The python version is a pure python implementation and there is no code taken from R. Therefore, I don't think there is any license issue.

Best regards,

Sylvain.

------------------------------------------------------------------------------------------------------------------------------------

Date: Sat, 24 Oct 2015 08:39:44 -0600

From: Charles R Harris <charlesr.harris at gmail.com<mailto:charlesr.harris at gmail.com>>

To: SciPy Developers List <scipy-dev at scipy.org<mailto:scipy-dev at scipy.org>>

Subject: Re: [SciPy-Dev] optimize - add algorithm for global

                optimization:     GenSA

Message-ID:

                <CAB6mnxJy8n+yfJ9TMJAZWNGLJ4J4_hs58T-W3snrgdsmRs0b0w at mail.gmail.com<mailto:CAB6mnxJy8n+yfJ9TMJAZWNGLJ4J4_hs58T-W3snrgdsmRs0b0w at mail.gmail.com>>

Content-Type: text/plain; charset="utf-8"

...

I'm a bit concerned about license issues If any of the code comes from R. R has an incompatible license.



Chuck
------------------------------------------------------------------------------------------------------------------------------------

Date: Sat, 24 Oct 2015 10:00:22 +1100

From: Andrew Nelson <andyfaff at gmail.com<mailto:andyfaff at gmail.com>>

To: SciPy Developers List <scipy-dev at scipy.org<mailto:scipy-dev at scipy.org>>

Subject: Re: [SciPy-Dev] optimize - add algorithm for global

                optimization:     GenSA

Message-ID:

                <CAAbtOZctBQeqjc5+QucA4XADC=F1MqKjPTHJDMb6XQjB-dkM7w at mail.gmail.com<mailto:CAAbtOZctBQeqjc5+QucA4XADC=F1MqKjPTHJDMb6XQjB-dkM7w at mail.gmail.com>>

Content-Type: text/plain; charset="utf-8"

...

I have a scipy PR for testing global optimisers. It has approx 120 functions, which is 2.5 times more than in the linked paper. I think for any global optimisers to be added it should give a good performance against those benchmarks. The main criterion is the number of successes and the avg number of function evaluations. Time is of secondary importance.

Any optimisers added to scipy.optimize should conform to the generally standardised syntax (naming conventions, etc) used by the module.

------------------------------------------------------------------------------------------------------------------------------------

Date: Fri, 23 Oct 2015 14:06:17 +0000

From: Jacob Stevenson <jstevenson131 at gmail.com<mailto:jstevenson131 at gmail.com>>

To: SciPy Developers List <scipy-dev at scipy.org<mailto:scipy-dev at scipy.org>>

Subject: Re: [SciPy-Dev] optimize - add algorithm for global

                optimization:     GenSA

Message-ID:

                <CAEEL3ux+0Wxa+v6Azv6kdRZKBN8qts_RdU1pEG3S+hjeYNry-w at mail.gmail.com<mailto:CAEEL3ux+0Wxa+v6Azv6kdRZKBN8qts_RdU1pEG3S+hjeYNry-w at mail.gmail.com>>

Content-Type: text/plain; charset="utf-8"

...

In my opinion a robust implementation of a simulated annealing based optimizer would be welcome.  There are cases when this would be preferable to basinhopping, e.g. when non-smooth or non-continuous functions make the local optimization step in basinhopping less effective.



I think the first step is to make a pull request (or send a link if you already did) where we can review the code and have discussions.



Best,

Jake
------------------------------------------------------------------------------------------------------------------------------------

Date: Fri, 23 Oct 2015 13:10:38 +0000

From: "Gubian, Sylvain" <Sylvain.Gubian at pmi.com<mailto:Sylvain.Gubian at pmi.com>>

To: "scipy-dev at scipy.org<mailto:scipy-dev at scipy.org>" <scipy-dev at scipy.org<mailto:scipy-dev at scipy.org>>

Subject: [SciPy-Dev] optimize - add algorithm for global optimization:

                GenSA

Message-ID:

                <A3786E9825AE3D4F8CC0EC80FBFAD32E02349A0696 at PMICHLAUEXM23.PMINTL.NET<mailto:A3786E9825AE3D4F8CC0EC80FBFAD32E02349A0696 at PMICHLAUEXM23.PMINTL.NET>>

Content-Type: text/plain; charset="Windows-1252"



Hi everyone,



We would like to propose a new method, GenSA,  for global optimization to be included in the optimize module.



GenSA is an implementation of the General Simulated Annealing algorithm (GSA, http://www.sciencedirect.com/science/article/pii/S0378437196002713). This approach generalizes CSA (Classical Simulated Annealing) and FSA (Fast Simulated Annealing) to search for the global minimum more efficiently. The algorithm is explained in more detail in this reference: http://journal.r-project.org/archive/2013-1/xiang-gubian-suomela-etal.pdf.



SciPy has already in the past included a method based on simulated annealing, called anneal, which has been deprecated in 0.14 (with an advice to use basinhopping) and eventually removed in 0.16.



A previously published comparison of 18 optimization methods in the R language (http://www.jstatsoft.org/v60/i06/paper) shows that GenSA is, among the methods tested, one of the ?most capable of consistently returning a solution near the global minimum of each test function?. This paper however did not consider basinhopping, so we have performed some tests which tend to show that GenSA is more efficient than basinhopping  for high dimension problems. The results have been presented in a poster in PyCon UK 2015 (Coventry).



The code is ready and passes unit tests and PEP8. We hope it would be a useful addition to SciPy and would be happy to have your opinion.



Thanks,



Sylvain.

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scipy-dev/attachments/20151110/b1e70e4c/attachment.html>

From robert.kern at gmail.com  Tue Nov 10 09:29:02 2015
From: robert.kern at gmail.com (Robert Kern)
Date: Tue, 10 Nov 2015 14:29:02 +0000
Subject: [SciPy-Dev] optimize - add algorithm for global optimization:
	GenSA
In-Reply-To: <A3786E9825AE3D4F8CC0EC80FBFAD32E02349A58CC@PMICHLAUEXM23.PMINTL.NET>
References: <A3786E9825AE3D4F8CC0EC80FBFAD32E02349A58CC@PMICHLAUEXM23.PMINTL.NET>
Message-ID: <CAF6FJiuR-DpCBb8kVso_tpHy5D_TeHGtXyY0tY=XJy8K=cG5vA@mail.gmail.com>

On Tue, Nov 10, 2015 at 2:05 PM, Gubian, Sylvain <Sylvain.Gubian at pmi.com>
wrote:
>
> Dear All,
>
> Thanks a lot for Jake, Andrew, Chuck and Sturla comments.
>
> I would like to address some comments so that I would know how to go
forward, like doing a pull request whenever you think it makes sense.

I think it's time to see the code, whether in the form of a pull request
against scipy or just a release of the current code as a standalone package.

> @Jake: As you mentioned, there are situations where basinhopping may not
work as well as PyGenSA, especially in cases of high number of parameters.
This implementation has actually been focused on solving high dimension
problems. However, it also works for very simple problems, but in those
cases it might show non-optimal performance (in terms of avg number of
function calls). The battery test shows that in general, the number of
successes is almost always better than basinhopping, PyGenSA is a robust
implementation.
>
> As attached, a PDF version of the poster I presented to PyCon UK Coventry
in September 2015 showing a performance benchmark.

This was not attached.

--
Robert Kern
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scipy-dev/attachments/20151110/0ed927b7/attachment.html>

From jstevenson131 at gmail.com  Tue Nov 10 11:56:14 2015
From: jstevenson131 at gmail.com (Jacob Stevenson)
Date: Tue, 10 Nov 2015 16:56:14 +0000
Subject: [SciPy-Dev] optimize - add algorithm for global optimization:
	GenSA
In-Reply-To: <CAF6FJiuR-DpCBb8kVso_tpHy5D_TeHGtXyY0tY=XJy8K=cG5vA@mail.gmail.com>
References: <A3786E9825AE3D4F8CC0EC80FBFAD32E02349A58CC@PMICHLAUEXM23.PMINTL.NET>
 <CAF6FJiuR-DpCBb8kVso_tpHy5D_TeHGtXyY0tY=XJy8K=cG5vA@mail.gmail.com>
Message-ID: <CAEEL3uxmvjwDZgfL=Ft0BHtuctpKsVzuvMZrXFFU_NKLS4waJg@mail.gmail.com>

Just to be clear, I don't think high dimensionality is one of
basinhopping's weak points.  Researchers use it regularly in cases with
hundreds or thousands of dimensions.  I saw in one of the GenSa papers
<https://journal.r-project.org/archive/2013-1/xiang-gubian-suomela-etal.pdf>
they mention the Thompson problem, so as an example here's
<http://journals.aps.org/prl/abstract/10.1103/PhysRevLett.110.165502> a
paper using basinhopping on a Thompson-like problem with with 600 atoms (free
version
<http://www.researchgate.net/profile/Halim_Kusumaatmaja/publication/236914161_Defect_Motifs_for_Constant_Mean_Curvature_Surfaces/links/00b7d5398a0f06bf22000000.pdf>
).

I wasn't trying to say that one method was better than the other, I was
just pointing out that basinhopping does have some weak points and it would
be nice to fill those in.  More specifically, basinhopping builds on top of
the abilities of local optimizers, and local optimizers often work best
when the function is reasonably continuous and smooth.  So if you try to
use basinhopping with lbfgs on a function with a lot of discontinuities,
you're going to have a bad day.

On Tue, 10 Nov 2015 at 15:29 Robert Kern <robert.kern at gmail.com> wrote:

> On Tue, Nov 10, 2015 at 2:05 PM, Gubian, Sylvain <Sylvain.Gubian at pmi.com>
> wrote:
> >
> > Dear All,
> >
> > Thanks a lot for Jake, Andrew, Chuck and Sturla comments.
> >
> > I would like to address some comments so that I would know how to go
> forward, like doing a pull request whenever you think it makes sense.
>
> I think it's time to see the code, whether in the form of a pull request
> against scipy or just a release of the current code as a standalone package.
>
> > @Jake: As you mentioned, there are situations where basinhopping may not
> work as well as PyGenSA, especially in cases of high number of parameters.
> This implementation has actually been focused on solving high dimension
> problems. However, it also works for very simple problems, but in those
> cases it might show non-optimal performance (in terms of avg number of
> function calls). The battery test shows that in general, the number of
> successes is almost always better than basinhopping, PyGenSA is a robust
> implementation.
> >
> > As attached, a PDF version of the poster I presented to PyCon UK
> Coventry in September 2015 showing a performance benchmark.
>
> This was not attached.
>
> --
> Robert Kern
> _______________________________________________
> SciPy-Dev mailing list
> SciPy-Dev at scipy.org
> https://mail.scipy.org/mailman/listinfo/scipy-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scipy-dev/attachments/20151110/9c0574bd/attachment.html>

From robert.kern at gmail.com  Wed Nov 11 05:36:24 2015
From: robert.kern at gmail.com (Robert Kern)
Date: Wed, 11 Nov 2015 10:36:24 +0000
Subject: [SciPy-Dev] Random matrices
In-Reply-To: <630461273-1446155740-cardhu_decombobulator_blackberry.rim.net-980370625-@b28.c1.bise6.blackberry>
References: <CAK0Mvaozbo46a=j9oNOrb-WFBdqfFA1tYFH6uhBFzNy_4jrxAg@mail.gmail.com>
 <CAF6FJisHqnFCVUmp4z4ReAavSE39=U_394Ls-XmL37_weRS_Ng@mail.gmail.com>
 <630461273-1446155740-cardhu_decombobulator_blackberry.rim.net-980370625-@b28.c1.bise6.blackberry>
Message-ID: <CAF6FJiuDwCXnDUTMH8rkLCq7ybJnUD00f1Hhcf_J9O5J2ucGOg@mail.gmail.com>

On Thu, Oct 29, 2015 at 9:55 PM, <philip.deboer at gmail.com> wrote:
>
> Right; since these are compact groups so they should have a const pdf, I
suppose dependent on dimension.
>
> So I should declare a new class, inherit from multi_rv_generic, and put
my code into rvs, with dimension as an input?
>
> Do I need to define additional public functions (pdf, entropy), or are
they all optional?

I think they are all optional.

--
Robert Kern
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scipy-dev/attachments/20151111/eb0c3f14/attachment.html>

From charlesr.harris at gmail.com  Thu Nov 12 16:11:14 2015
From: charlesr.harris at gmail.com (Charles R Harris)
Date: Thu, 12 Nov 2015 14:11:14 -0700
Subject: [SciPy-Dev] Numpy 1.10.2rc1
Message-ID: <CAB6mnx+airBuoGoLwbQ46ZK8sQ38tiNmwFkAkb536DjSYyZRAA@mail.gmail.com>

Hi All,

I am pleased to announce the release of Numpy 1.10.2rc1. This release
should fix the problems exposed in 1.10.1, which is not to say there are no
remaining problems. Please test this thoroughly, exspecially if you
experienced problems with 1.10.1. Julian Taylor has opened an issue
relating to cblas detection on Debian (and probably Debian derived
distributions) that is not dealt with in this release. Hopefully a solution
will be available before the final.

To all who reported issues with 1.10.1 and to those who helped close them,
a big thank you.

Source and binary files may be found on Sourceforge
<https://sourceforge.net/projects/numpy/files/NumPy/1.10.1/>.

Chuck
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scipy-dev/attachments/20151112/fc3e7f7d/attachment.html>

From evgeny.burovskiy at gmail.com  Fri Nov 13 11:19:10 2015
From: evgeny.burovskiy at gmail.com (Evgeni Burovski)
Date: Fri, 13 Nov 2015 16:19:10 +0000
Subject: [SciPy-Dev] dense output option in integrate.ode
Message-ID: <CAMRo0itRn-P_d_Pp2ocymJUHgYy=fChBy8YFEC77-8=TU9hM4A@mail.gmail.com>

Hi David, Hi Jim,

> I am new to SciPy-dev. Will the dense output option in scipy.integrate.ode
> become available in 0.17.0? This is a feature already available in the
> original FORTAN, but wasn't implemented in the wrapper.

If the feature is sent as a pull request against the scipy master
branch, the PR is reviewed by the maintainers of the integrate package
and merged into master before the release split, then yes, it would be
available in 0.17.0.

So far I do not see any progress towards it.


> I wrote these dense output extensions that you listed:
>
> https://github.com/jddmartin/scipy/tree/dense_output_from_dopri5_and_dop853
> https://github.com/jddmartin/dense_output_example_usage
>
> but I didn't issue any pull request to scipy.  I sent this message to the
> scipy developers list:
>   http://article.gmane.org/gmane.comp.python.scientific.devel/19635/
> explaining the changes.  I was hoping for some feedback before issuing a
> pull request.

Ah, I see that the email likely fell through the cracks back in April.
Sorry about that.
You might want to ping that email thread once more or send a pull
request on github (or both).

<snip>

Cheers,

Evgeni


From orion at cora.nwra.com  Fri Nov 13 13:56:47 2015
From: orion at cora.nwra.com (Orion Poplawski)
Date: Fri, 13 Nov 2015 11:56:47 -0700
Subject: [SciPy-Dev] scipy Fedora test failure
Message-ID: <5646326F.7060001@cora.nwra.com>

I'm trying to get scipy updating in Fedora.  Running into test failures.  Full
log is here:
https://kojipkgs.fedoraproject.org//work/tasks/8123/11808123/build.log

NumPy version 1.10.1
NumPy relaxed strides checking option: True
NumPy is installed in /usr/lib64/python2.7/site-packages/numpy
SciPy version 0.16.1
SciPy is installed in
/builddir/build/BUILDROOT/scipy-0.16.1-3.fc24.x86_64/usr/lib64/python2.7/site-packages/scipy
Python version 2.7.10 (default, Oct 15 2015, 20:40:13) [GCC 5.1.1 20150618
(Red Hat 5.1.1-4)]
nose version 1.3.7
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/usr/lib64/python2.7/site-packages/numpy/testing/nosetester.py", line
438, in test
    t = NumpyTestProgram(argv=argv, exit=False, plugins=plugins)
  File "/usr/lib/python2.7/site-packages/nose/core.py", line 121, in __init__
    **extra_args)
  File "/usr/lib64/python2.7/unittest/main.py", line 95, in __init__
    self.runTests()
  File "/usr/lib64/python2.7/site-packages/numpy/testing/noseclasses.py", line
351, in runTests
    self.result = self.testRunner.run(self.test)
  File "/usr/lib/python2.7/site-packages/nose/core.py", line 66, in run
    result.printErrors()
  File "/usr/lib/python2.7/site-packages/nose/result.py", line 103, in printErrors
    _TextTestResult.printErrors(self)
  File "/usr/lib64/python2.7/unittest/runner.py", line 108, in printErrors
    self.printErrorList('ERROR', self.errors)
  File "/usr/lib64/python2.7/unittest/runner.py", line 114, in printErrorList
    self.stream.writeln("%s: %s" % (flavour,self.getDescription(test)))
  File "/usr/lib/python2.7/site-packages/nose/result.py", line 82, in
getDescription
    return test.shortDescription() or str(test)
  File "/usr/lib/python2.7/site-packages/nose/case.py", line 51, in __str__
    return str(self.test)
  File "/usr/lib/python2.7/site-packages/nose/case.py", line 286, in __str__
    name = "%s%s" % (name, arg)
  File "/usr/lib64/python2.7/site-packages/numpy/core/numeric.py", line 1709,
in array_repr
    ', ', "array(")
  File "/usr/lib64/python2.7/site-packages/numpy/core/arrayprint.py", line
447, in array2string
    separator, prefix, formatter=formatter)
  File "/usr/lib64/python2.7/site-packages/numpy/core/arrayprint.py", line
260, in _array2string
    'int': IntegerFormat(data),
  File "/usr/lib64/python2.7/site-packages/numpy/core/arrayprint.py", line
637, in __init__
    max_str_len = max(len(str(maximum.reduce(data))),
DeprecationWarning: unorderable dtypes; returning scalar but in the future
this will be an error
RPM build errors:
error: Bad exit status from /var/tmp/rpm-tmp.6SOdNx (%check)

This *appears* to be what triggers the failure - but why does a
DeprecationWarning do this?  Or is there something further up in the test?

-- 
Orion Poplawski
Technical Manager                     303-415-9701 x222
NWRA, Boulder/CoRA Office             FAX: 303-415-9702
3380 Mitchell Lane                       orion at nwra.com
Boulder, CO 80301                   http://www.nwra.com


From charlesr.harris at gmail.com  Fri Nov 13 14:29:17 2015
From: charlesr.harris at gmail.com (Charles R Harris)
Date: Fri, 13 Nov 2015 12:29:17 -0700
Subject: [SciPy-Dev] scipy Fedora test failure
In-Reply-To: <5646326F.7060001@cora.nwra.com>
References: <5646326F.7060001@cora.nwra.com>
Message-ID: <CAB6mnx+avKDGdoqaYnTtXkTuO-ZRSEkTu=6O=cUuv2bsZ_c1oQ@mail.gmail.com>

On Fri, Nov 13, 2015 at 11:56 AM, Orion Poplawski <orion at cora.nwra.com>
wrote:

> I'm trying to get scipy updating in Fedora.  Running into test failures.
> Full
> log is here:
> https://kojipkgs.fedoraproject.org//work/tasks/8123/11808123/build.log
>
> NumPy version 1.10.1
> NumPy relaxed strides checking option: True
> NumPy is installed in /usr/lib64/python2.7/site-packages/numpy
> SciPy version 0.16.1
> SciPy is installed in
>
> /builddir/build/BUILDROOT/scipy-0.16.1-3.fc24.x86_64/usr/lib64/python2.7/site-packages/scipy
> Python version 2.7.10 (default, Oct 15 2015, 20:40:13) [GCC 5.1.1 20150618
> (Red Hat 5.1.1-4)]
> nose version 1.3.7
> Traceback (most recent call last):
>   File "<string>", line 1, in <module>
>   File "/usr/lib64/python2.7/site-packages/numpy/testing/nosetester.py",
> line
> 438, in test
>     t = NumpyTestProgram(argv=argv, exit=False, plugins=plugins)
>   File "/usr/lib/python2.7/site-packages/nose/core.py", line 121, in
> __init__
>     **extra_args)
>   File "/usr/lib64/python2.7/unittest/main.py", line 95, in __init__
>     self.runTests()
>   File "/usr/lib64/python2.7/site-packages/numpy/testing/noseclasses.py",
> line
> 351, in runTests
>     self.result = self.testRunner.run(self.test)
>   File "/usr/lib/python2.7/site-packages/nose/core.py", line 66, in run
>     result.printErrors()
>   File "/usr/lib/python2.7/site-packages/nose/result.py", line 103, in
> printErrors
>     _TextTestResult.printErrors(self)
>   File "/usr/lib64/python2.7/unittest/runner.py", line 108, in printErrors
>     self.printErrorList('ERROR', self.errors)
>   File "/usr/lib64/python2.7/unittest/runner.py", line 114, in
> printErrorList
>     self.stream.writeln("%s: %s" % (flavour,self.getDescription(test)))
>   File "/usr/lib/python2.7/site-packages/nose/result.py", line 82, in
> getDescription
>     return test.shortDescription() or str(test)
>   File "/usr/lib/python2.7/site-packages/nose/case.py", line 51, in __str__
>     return str(self.test)
>   File "/usr/lib/python2.7/site-packages/nose/case.py", line 286, in
> __str__
>     name = "%s%s" % (name, arg)
>   File "/usr/lib64/python2.7/site-packages/numpy/core/numeric.py", line
> 1709,
> in array_repr
>     ', ', "array(")
>   File "/usr/lib64/python2.7/site-packages/numpy/core/arrayprint.py", line
> 447, in array2string
>     separator, prefix, formatter=formatter)
>   File "/usr/lib64/python2.7/site-packages/numpy/core/arrayprint.py", line
> 260, in _array2string
>     'int': IntegerFormat(data),
>   File "/usr/lib64/python2.7/site-packages/numpy/core/arrayprint.py", line
> 637, in __init__
>     max_str_len = max(len(str(maximum.reduce(data))),
> DeprecationWarning: unorderable dtypes; returning scalar but in the future
> this will be an error
> RPM build errors:
> error: Bad exit status from /var/tmp/rpm-tmp.6SOdNx (%check)
>
> This *appears* to be what triggers the failure - but why does a
> DeprecationWarning do this?  Or is there something further up in the test?
>
>
Bug in the Numpy 1.10.1 release. Could you try 1.10.2rc1?

Chuck
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scipy-dev/attachments/20151113/9abab960/attachment.html>

From lzkelley at gmail.com  Sat Nov 14 19:11:30 2015
From: lzkelley at gmail.com (Luke Zoltan Kelley)
Date: Sat, 14 Nov 2015 19:11:30 -0500
Subject: [SciPy-Dev] Scipy developent environment using python3.4
Message-ID: <D08A5B52-0B84-48C7-8A77-AD8A014D310B@gmail.com>

I've been playing around with development of scipy using an anaconda python2.7 environment, which has been working well.  I created a new anaconda environment, cloned the scipy repo, and installed using `python setup.py build_ext --inplace`.

In a recent PR I introduced an error with a python3 incompatibility, so I wanted to create a python3 environment in which to test as-well.  I followed the same procedure, and it seemed to build fine, but when I try to import the `stats` module I get the error:

$ python -c "import scipy.stats; scipy.stats.test()"
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/Users/lzkelley/Programs/public/scipy/scipy/stats/__init__.py", line 338, in <module>
    from .stats import *
  File "/Users/lzkelley/Programs/public/scipy/scipy/stats/stats.py", line 180, in <module>
    import scipy.special as special
  File "/Users/lzkelley/Programs/public/scipy/scipy/special/__init__.py", line 627, in <module>
    from ._ufuncs import *
ImportError: dlopen(/Users/lzkelley/Programs/public/scipy/scipy/special/_ufuncs.so, 2): Symbol not found: _PyCObject_Type
  Referenced from: /Users/lzkelley/Programs/public/scipy/scipy/special/_ufuncs.so
  Expected in: dynamic lookup

Has anyone seen anything like this before?


I think the problem has something to do with conflicts between the different versions of python.  After trying to build the py34 version, I went back to my py27 environment and when running one of the test modules - the output errors were changed...  For example, after running,

python -c "import scipy.stats; scipy.stats.test()"

I get a new error (from a module that I haven't touched):

======================================================================
ERROR: test_large_pseudo_determinant (test_multivariate.TestMultivariateNormal)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/lzkelley/Programs/public/scipy/scipy/stats/tests/test_multivariate.py", line 131, in test_large_pseudo_determinant
    assert_equal(scipy.linalg.det(cov), 0)
  File "/Users/lzkelley/Programs/public/scipy/scipy/linalg/basic.py", line 746, in det
    a_det, info = fdet(a1, overwrite_a=overwrite_a)
TypeError: 'NoneType' object is not callable


Regarding use of the inplace build instead of using 'runtests.py':
The inplace build is great, because I can do tests with a running ipython session and simply do (e.g.) `reload(scipy.stats._binned_statistic); scipy.stats._binned_statistic.binned_statistic(...)` to see my changes immediately.  Using something like `runtests.py` requires a re-build each time, which is extremely time-consuming.  As a rookie, it's very difficult to make effective changes without running dozens of tests, quickly.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scipy-dev/attachments/20151114/fe4d7880/attachment.html>

From matthew.brett at gmail.com  Sat Nov 14 19:16:12 2015
From: matthew.brett at gmail.com (Matthew Brett)
Date: Sat, 14 Nov 2015 16:16:12 -0800
Subject: [SciPy-Dev] Scipy developent environment using python3.4
In-Reply-To: <D08A5B52-0B84-48C7-8A77-AD8A014D310B@gmail.com>
References: <D08A5B52-0B84-48C7-8A77-AD8A014D310B@gmail.com>
Message-ID: <CAH6Pt5pd+7ZgwXw1gbS1yGt4mJhpEeibJ6KN6VBkk2Dwh_0GYQ@mail.gmail.com>

I'm afraid I don't use Anaconda myself - does it have the concept of
an inplace install?

If you are stuck you could switch to Hombrew / virtualenvs / "pip
install -e ." instead.


From robert.kern at gmail.com  Sat Nov 14 19:21:45 2015
From: robert.kern at gmail.com (Robert Kern)
Date: Sun, 15 Nov 2015 00:21:45 +0000
Subject: [SciPy-Dev] Scipy developent environment using python3.4
In-Reply-To: <CAH6Pt5pd+7ZgwXw1gbS1yGt4mJhpEeibJ6KN6VBkk2Dwh_0GYQ@mail.gmail.com>
References: <D08A5B52-0B84-48C7-8A77-AD8A014D310B@gmail.com>
 <CAH6Pt5pd+7ZgwXw1gbS1yGt4mJhpEeibJ6KN6VBkk2Dwh_0GYQ@mail.gmail.com>
Message-ID: <CAF6FJisj4fiG4HLWT7WryKwniV+q=PF+viZ+YyH3RAupOfcPQQ@mail.gmail.com>

On Nov 15, 2015 00:16, "Matthew Brett" <matthew.brett at gmail.com> wrote:
>
> I'm afraid I don't use Anaconda myself - does it have the concept of
> an inplace install?

Of course it does.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scipy-dev/attachments/20151115/0781dc68/attachment.html>

From lzkelley at gmail.com  Sat Nov 14 19:23:27 2015
From: lzkelley at gmail.com (Luke Zoltan Kelley)
Date: Sat, 14 Nov 2015 19:23:27 -0500
Subject: [SciPy-Dev] Scipy developent environment using python3.4
In-Reply-To: <CAF6FJisj4fiG4HLWT7WryKwniV+q=PF+viZ+YyH3RAupOfcPQQ@mail.gmail.com>
References: <D08A5B52-0B84-48C7-8A77-AD8A014D310B@gmail.com>
 <CAH6Pt5pd+7ZgwXw1gbS1yGt4mJhpEeibJ6KN6VBkk2Dwh_0GYQ@mail.gmail.com>
 <CAF6FJisj4fiG4HLWT7WryKwniV+q=PF+viZ+YyH3RAupOfcPQQ@mail.gmail.com>
Message-ID: <9FE83CFE-B62E-450A-8E72-3F35D0B23F4E@gmail.com>

Do you mean there is a special way to perform an inplace build with anaconda?

> On Nov 14, 2015, at 7:21 PM, Robert Kern <robert.kern at gmail.com> wrote:
> 
> On Nov 15, 2015 00:16, "Matthew Brett" <matthew.brett at gmail.com <mailto:matthew.brett at gmail.com>> wrote:
> >
> > I'm afraid I don't use Anaconda myself - does it have the concept of
> > an inplace install?
> 
> Of course it does.
> _______________________________________________
> SciPy-Dev mailing list
> SciPy-Dev at scipy.org
> https://mail.scipy.org/mailman/listinfo/scipy-dev

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scipy-dev/attachments/20151114/ed359ea2/attachment.html>

From robert.kern at gmail.com  Sat Nov 14 19:25:28 2015
From: robert.kern at gmail.com (Robert Kern)
Date: Sun, 15 Nov 2015 00:25:28 +0000
Subject: [SciPy-Dev] Scipy developent environment using python3.4
In-Reply-To: <9FE83CFE-B62E-450A-8E72-3F35D0B23F4E@gmail.com>
References: <D08A5B52-0B84-48C7-8A77-AD8A014D310B@gmail.com>
 <CAH6Pt5pd+7ZgwXw1gbS1yGt4mJhpEeibJ6KN6VBkk2Dwh_0GYQ@mail.gmail.com>
 <CAF6FJisj4fiG4HLWT7WryKwniV+q=PF+viZ+YyH3RAupOfcPQQ@mail.gmail.com>
 <9FE83CFE-B62E-450A-8E72-3F35D0B23F4E@gmail.com>
Message-ID: <CAF6FJiuETyAupDvKTCtZPCz8yXyE42uuot5z3=DiVxGen2vXEQ@mail.gmail.com>

On Nov 15, 2015 00:23, "Luke Zoltan Kelley" <lzkelley at gmail.com> wrote:
>
> Do you mean there is a special way to perform an inplace build with
anaconda?

No, I mean of course Anaconda has the concept of an inplace install because
Python does, and Anaconda is a Python distribution.

>> On Nov 14, 2015, at 7:21 PM, Robert Kern <robert.kern at gmail.com> wrote:
>>
>> On Nov 15, 2015 00:16, "Matthew Brett" <matthew.brett at gmail.com> wrote:
>> >
>> > I'm afraid I don't use Anaconda myself - does it have the concept of
>> > an inplace install?
>>
>> Of course it does.
>>
>> _______________________________________________
>>
>> SciPy-Dev mailing list
>> SciPy-Dev at scipy.org
>> https://mail.scipy.org/mailman/listinfo/scipy-dev
>
>
>
> _______________________________________________
> SciPy-Dev mailing list
> SciPy-Dev at scipy.org
> https://mail.scipy.org/mailman/listinfo/scipy-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scipy-dev/attachments/20151115/6c3351fb/attachment.html>

From charlesr.harris at gmail.com  Sat Nov 14 22:51:43 2015
From: charlesr.harris at gmail.com (Charles R Harris)
Date: Sat, 14 Nov 2015 20:51:43 -0700
Subject: [SciPy-Dev] optimize - add algorithm for global optimization:
	GenSA
In-Reply-To: <CAEEL3uxmvjwDZgfL=Ft0BHtuctpKsVzuvMZrXFFU_NKLS4waJg@mail.gmail.com>
References: <A3786E9825AE3D4F8CC0EC80FBFAD32E02349A58CC@PMICHLAUEXM23.PMINTL.NET>
 <CAF6FJiuR-DpCBb8kVso_tpHy5D_TeHGtXyY0tY=XJy8K=cG5vA@mail.gmail.com>
 <CAEEL3uxmvjwDZgfL=Ft0BHtuctpKsVzuvMZrXFFU_NKLS4waJg@mail.gmail.com>
Message-ID: <CAB6mnxJCD+Mr98GdGKQdhKLUe8nTmxCorKLWBTVq6EM_j1AN3A@mail.gmail.com>

On Tue, Nov 10, 2015 at 9:56 AM, Jacob Stevenson <jstevenson131 at gmail.com>
wrote:

> Just to be clear, I don't think high dimensionality is one of
> basinhopping's weak points.  Researchers use it regularly in cases with
> hundreds or thousands of dimensions.  I saw in one of the GenSa papers
> <https://journal.r-project.org/archive/2013-1/xiang-gubian-suomela-etal.pdf>
> they mention the Thompson problem, so as an example here's
> <http://journals.aps.org/prl/abstract/10.1103/PhysRevLett.110.165502> a
> paper using basinhopping on a Thompson-like problem with with 600 atoms (free
> version
> <http://www.researchgate.net/profile/Halim_Kusumaatmaja/publication/236914161_Defect_Motifs_for_Constant_Mean_Curvature_Surfaces/links/00b7d5398a0f06bf22000000.pdf>
> ).
>
> I wasn't trying to say that one method was better than the other, I was
> just pointing out that basinhopping does have some weak points and it would
> be nice to fill those in.  More specifically, basinhopping builds on top of
> the abilities of local optimizers, and local optimizers often work best
> when the function is reasonably continuous and smooth.  So if you try to
> use basinhopping with lbfgs on a function with a lot of discontinuities,
> you're going to have a bad day.
>

Out of curiosity, do any of the tests benchmark functions with discrete
variables? Many years ago I converted a filter design from double precision
coefficients to 16 bit fixed point coefficients and sought to optimize the
rounding. The fastest algorithm, when it worked, was SA, but most of the
time it just went off on an extended tour of parameter space.  The most
reliable, if slow, method was a GA.

Chuck
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scipy-dev/attachments/20151114/38e20de8/attachment.html>

From giselle1701 at gmail.com  Sun Nov 15 09:09:07 2015
From: giselle1701 at gmail.com (Giselle Sosa Jones)
Date: Sun, 15 Nov 2015 09:39:07 -0430
Subject: [SciPy-Dev] SLSQP question
Message-ID: <CA+_N=LOJZYuG55VGCVGEGgs0uzzi-BMoxsKwb0-429CWbf+41Q@mail.gmail.com>

Hello.

I would like to know if anyone knows the difference between SQP and SLSQP?
Where do the Least Squares come to use in the method?

Thank you.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scipy-dev/attachments/20151115/213948f0/attachment.html>

From orion at cora.nwra.com  Mon Nov 16 14:19:02 2015
From: orion at cora.nwra.com (Orion Poplawski)
Date: Mon, 16 Nov 2015 12:19:02 -0700
Subject: [SciPy-Dev] scipy Fedora test failure
In-Reply-To: <CAB6mnx+avKDGdoqaYnTtXkTuO-ZRSEkTu=6O=cUuv2bsZ_c1oQ@mail.gmail.com>
References: <5646326F.7060001@cora.nwra.com>
 <CAB6mnx+avKDGdoqaYnTtXkTuO-ZRSEkTu=6O=cUuv2bsZ_c1oQ@mail.gmail.com>
Message-ID: <564A2C26.9030802@cora.nwra.com>

On 11/13/2015 12:29 PM, Charles R Harris wrote:
> 
> 
> On Fri, Nov 13, 2015 at 11:56 AM, Orion Poplawski <orion at cora.nwra.com
> <mailto:orion at cora.nwra.com>> wrote:
> 
>     I'm trying to get scipy updating in Fedora.  Running into test failures.  Full
>     log is here:
>     https://kojipkgs.fedoraproject.org//work/tasks/8123/11808123/build.log
> 
>     DeprecationWarning: unorderable dtypes; returning scalar but in the future
>     this will be an error
>     RPM build errors:
>     error: Bad exit status from /var/tmp/rpm-tmp.6SOdNx (%check)
> 
>     This *appears* to be what triggers the failure - but why does a
>     DeprecationWarning do this?  Or is there something further up in the test?
> 
> 
> Bug in the Numpy 1.10.1 release. Could you try 1.10.2rc1?
> 
> Chuck


Thanks, that did the trick.


-- 
Orion Poplawski
Technical Manager                     303-415-9701 x222
NWRA, Boulder/CoRA Office             FAX: 303-415-9702
3380 Mitchell Lane                       orion at nwra.com
Boulder, CO 80301                   http://www.nwra.com


From warren.weckesser at gmail.com  Mon Nov 16 15:46:38 2015
From: warren.weckesser at gmail.com (Warren Weckesser)
Date: Mon, 16 Nov 2015 15:46:38 -0500
Subject: [SciPy-Dev] Extend odeint to handle matrices and complex numbers
Message-ID: <CAGzF1ucKDnyAggSz3ouQgDEjhuATpq-XEi6WUOLNOSVhWArTag@mail.gmail.com>

I wrote a wrapper of odeint called odeintw that is currently available on
github: https://github.com/WarrenWeckesser/odeintw

To quote from README.md in the repository:

    odeintw provides a wrapper of scipy.integrate.odeint that allows
    it to handle complex and matrix differential equations.
    That is, it can solve equations of the form

        dZ/dt = F(Z, t, param1, param2, ...)

    where t is real and Z is a real or complex array.


What do folks think of adding this functionality directly to odeint in
scipy?  It would be completely backwards compatible, and would not
introduce any overhead in the callbacks if the inputs are 1-d real arrays.

Warren
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scipy-dev/attachments/20151116/fe89980e/attachment.html>

From m.oliver at jacobs-university.de  Mon Nov 16 16:46:53 2015
From: m.oliver at jacobs-university.de (Marcel Oliver)
Date: Mon, 16 Nov 2015 22:46:53 +0100
Subject: [SciPy-Dev]  Extend odeint to handle matrices and complex numbers
In-Reply-To: <CAGzF1ucKDnyAggSz3ouQgDEjhuATpq-XEi6WUOLNOSVhWArTag@mail.gmail.com>
References: <CAGzF1ucKDnyAggSz3ouQgDEjhuATpq-XEi6WUOLNOSVhWArTag@mail.gmail.com>
Message-ID: <22090.20173.949807.933577@localhost.localdomain>

Warren Weckesser writes:
 > I wrote a wrapper of odeint called odeintw that is currently available on
 > github: https://github.com/WarrenWeckesser/odeintw
 > 
 > To quote from README.md in the repository:
 > 
 >     odeintw provides a wrapper of scipy.integrate.odeint that allows
 >     it to handle complex and matrix differential equations.
 >     That is, it can solve equations of the form
 > 
 >         dZ/dt = F(Z, t, param1, param2, ...)
 > 
 >     where t is real and Z is a real or complex array.
 > 
 > What do folks think of adding this functionality directly to odeint in scipy? 
 > It would be completely backwards compatible, and would not introduce any
 > overhead in the callbacks if the inputs are 1-d real arrays.
 > 
 > Warren

I cannot really comment on the code internals, but to have transparent
nd-array-valued vector fields would be a major improvement (so don't
stop at "matrix"!).

I have encountered the need over and over again, and one can always
flatten out the structures, but a lot of code clarity, in particular
the closeness between mathematical representation and the code, is
getting lost.

And why stop at odeint?  There is integrate.ode which currently has
the same limitation.

--Marcel


From warren.weckesser at gmail.com  Mon Nov 16 17:12:32 2015
From: warren.weckesser at gmail.com (Warren Weckesser)
Date: Mon, 16 Nov 2015 17:12:32 -0500
Subject: [SciPy-Dev] Extend odeint to handle matrices and complex numbers
In-Reply-To: <22090.20173.949807.933577@localhost.localdomain>
References: <CAGzF1ucKDnyAggSz3ouQgDEjhuATpq-XEi6WUOLNOSVhWArTag@mail.gmail.com>
 <22090.20173.949807.933577@localhost.localdomain>
Message-ID: <CAGzF1ufV0P_wxWLDFoXFjnnJe5i2uLRYHSs=PF0P_dJvg8sYkQ@mail.gmail.com>

On Mon, Nov 16, 2015 at 4:46 PM, Marcel Oliver <
m.oliver at jacobs-university.de> wrote:

> Warren Weckesser writes:
>  > I wrote a wrapper of odeint called odeintw that is currently available
> on
>  > github: https://github.com/WarrenWeckesser/odeintw
>  >
>  > To quote from README.md in the repository:
>  >
>  >     odeintw provides a wrapper of scipy.integrate.odeint that allows
>  >     it to handle complex and matrix differential equations.
>  >     That is, it can solve equations of the form
>  >
>  >         dZ/dt = F(Z, t, param1, param2, ...)
>  >
>  >     where t is real and Z is a real or complex array.
>  >
>  > What do folks think of adding this functionality directly to odeint in
> scipy?
>  > It would be completely backwards compatible, and would not introduce any
>  > overhead in the callbacks if the inputs are 1-d real arrays.
>  >
>  > Warren
>
> I cannot really comment on the code internals, but to have transparent
> nd-array-valued vector fields would be a major improvement (so don't
> stop at "matrix"!).
>


I shouldn't have said "matrix", because in fact it already handles
n-dimensional arrays.
The case of a 2-d matrix is just the most common request that I've seen.

For example, in the following a system in the shape of a 2x2x2 array is
solved:


In [21]: from odeintw import odeintw

In [22]: def func(x, t):
   ....:     return -x
   ....:

In [23]: x0 = np.arange(8.0).reshape(2,2,2)

In [24]: t = np.arange(5.0)

In [25]: sol = odeintw(func, x0, t)

In [26]: sol.shape
Out[26]: (5, 2, 2, 2)




> I have encountered the need over and over again, and one can always
> flatten out the structures, but a lot of code clarity, in particular
> the closeness between mathematical representation and the code, is
> getting lost.
>
> And why stop at odeint?  There is integrate.ode which currently has
> the same limitation.
>
>

That's certainly possible.  I referred to odeintw as a wrapper of odeint
(and it is), but most of the code is actually about wrapping the user's
callback functions (i.e. the differential equations and the Jacobian
function, if given).

Warren




> --Marcel
> _______________________________________________
> SciPy-Dev mailing list
> SciPy-Dev at scipy.org
> https://mail.scipy.org/mailman/listinfo/scipy-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scipy-dev/attachments/20151116/0d137e39/attachment.html>

From benny.malengier at gmail.com  Mon Nov 16 18:00:20 2015
From: benny.malengier at gmail.com (Benny Malengier)
Date: Tue, 17 Nov 2015 00:00:20 +0100
Subject: [SciPy-Dev] Extend odeint to handle matrices and complex numbers
In-Reply-To: <CAGzF1ufV0P_wxWLDFoXFjnnJe5i2uLRYHSs=PF0P_dJvg8sYkQ@mail.gmail.com>
References: <CAGzF1ucKDnyAggSz3ouQgDEjhuATpq-XEi6WUOLNOSVhWArTag@mail.gmail.com>
 <22090.20173.949807.933577@localhost.localdomain>
 <CAGzF1ufV0P_wxWLDFoXFjnnJe5i2uLRYHSs=PF0P_dJvg8sYkQ@mail.gmail.com>
Message-ID: <CAOyNq8ocmj_sEzT4=UwNCKd4LGOTLynFROYAVy77LLwvLhaSBg@mail.gmail.com>

Warren,

It would be most useful to add that to odes, https://github.com/bmcage/odes
which interfaces the modern (Krylov precond, ...) cvode code instead of the
old codebases included in scipy.
We have been working on a new api for odes which returns named tuples, see
the examples here: https://github.com/bmcage/odes/tree/master/docs/ipython
(chrome might be needed). This as a way to move forward the ode solvers in
scipy which don't have a nice/good/liked API (many suggestions for rewrites
have been suggested in the past). The solution as named tuple can be
accessed as sol.values.t and roots eg sol.roots.t, ...

One drawback to deprecate the old solvers in scipy has been the complex
solver zvode. Allen Hindmarsh (
http://history.siam.org/oralhistories/hindmarsh.htm) did not include a
complex solver in the new codes in sundials. Several people have indicated
their need for (transparent) complex ODE solvers. Your approach might solve
this.

Personally, I advise against using old solvers as lsode or vode as
available in scipy for actual academic (or industrial) use.

Benny


2015-11-16 23:12 GMT+01:00 Warren Weckesser <warren.weckesser at gmail.com>:

>
>
> On Mon, Nov 16, 2015 at 4:46 PM, Marcel Oliver <
> m.oliver at jacobs-university.de> wrote:
>
>> Warren Weckesser writes:
>>  > I wrote a wrapper of odeint called odeintw that is currently available
>> on
>>  > github: https://github.com/WarrenWeckesser/odeintw
>>  >
>>  > To quote from README.md in the repository:
>>  >
>>  >     odeintw provides a wrapper of scipy.integrate.odeint that allows
>>  >     it to handle complex and matrix differential equations.
>>  >     That is, it can solve equations of the form
>>  >
>>  >         dZ/dt = F(Z, t, param1, param2, ...)
>>  >
>>  >     where t is real and Z is a real or complex array.
>>  >
>>  > What do folks think of adding this functionality directly to odeint in
>> scipy?
>>  > It would be completely backwards compatible, and would not introduce
>> any
>>  > overhead in the callbacks if the inputs are 1-d real arrays.
>>  >
>>  > Warren
>>
>> I cannot really comment on the code internals, but to have transparent
>> nd-array-valued vector fields would be a major improvement (so don't
>> stop at "matrix"!).
>>
>
>
> I shouldn't have said "matrix", because in fact it already handles
> n-dimensional arrays.
> The case of a 2-d matrix is just the most common request that I've seen.
>
> For example, in the following a system in the shape of a 2x2x2 array is
> solved:
>
>
> In [21]: from odeintw import odeintw
>
> In [22]: def func(x, t):
>    ....:     return -x
>    ....:
>
> In [23]: x0 = np.arange(8.0).reshape(2,2,2)
>
> In [24]: t = np.arange(5.0)
>
> In [25]: sol = odeintw(func, x0, t)
>
> In [26]: sol.shape
> Out[26]: (5, 2, 2, 2)
>
>
>
>
>> I have encountered the need over and over again, and one can always
>> flatten out the structures, but a lot of code clarity, in particular
>> the closeness between mathematical representation and the code, is
>> getting lost.
>>
>> And why stop at odeint?  There is integrate.ode which currently has
>> the same limitation.
>>
>>
>
> That's certainly possible.  I referred to odeintw as a wrapper of odeint
> (and it is), but most of the code is actually about wrapping the user's
> callback functions (i.e. the differential equations and the Jacobian
> function, if given).
>
> Warren
>
>
>
>
>> --Marcel
>> _______________________________________________
>> SciPy-Dev mailing list
>> SciPy-Dev at scipy.org
>> https://mail.scipy.org/mailman/listinfo/scipy-dev
>>
>
>
> _______________________________________________
> SciPy-Dev mailing list
> SciPy-Dev at scipy.org
> https://mail.scipy.org/mailman/listinfo/scipy-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scipy-dev/attachments/20151117/f3c6b718/attachment.html>

From warren.weckesser at gmail.com  Mon Nov 16 18:19:27 2015
From: warren.weckesser at gmail.com (Warren Weckesser)
Date: Mon, 16 Nov 2015 18:19:27 -0500
Subject: [SciPy-Dev] Extend odeint to handle matrices and complex numbers
In-Reply-To: <CAOyNq8ocmj_sEzT4=UwNCKd4LGOTLynFROYAVy77LLwvLhaSBg@mail.gmail.com>
References: <CAGzF1ucKDnyAggSz3ouQgDEjhuATpq-XEi6WUOLNOSVhWArTag@mail.gmail.com>
 <22090.20173.949807.933577@localhost.localdomain>
 <CAGzF1ufV0P_wxWLDFoXFjnnJe5i2uLRYHSs=PF0P_dJvg8sYkQ@mail.gmail.com>
 <CAOyNq8ocmj_sEzT4=UwNCKd4LGOTLynFROYAVy77LLwvLhaSBg@mail.gmail.com>
Message-ID: <CAGzF1ucAH8GN7PJBBHix87DfpXsRk-huHbQh=q1cRcHFWhmBqA@mail.gmail.com>

On Mon, Nov 16, 2015 at 6:00 PM, Benny Malengier <benny.malengier at gmail.com>
wrote:

> Warren,
>
> It would be most useful to add that to odes,
> https://github.com/bmcage/odes which interfaces the modern (Krylov
> precond, ...) cvode code instead of the old codebases included in scipy.
> We have been working on a new api for odes which returns named tuples, see
> the examples here: https://github.com/bmcage/odes/tree/master/docs/ipython
> (chrome might be needed). This as a way to move forward the ode solvers in
> scipy which don't have a nice/good/liked API (many suggestions for rewrites
> have been suggested in the past). The solution as named tuple can be
> accessed as sol.values.t and roots eg sol.roots.t, ...
>
> One drawback to deprecate the old solvers in scipy has been the complex
> solver zvode. Allen Hindmarsh (
> http://history.siam.org/oralhistories/hindmarsh.htm) did not include a
> complex solver in the new codes in sundials. Several people have indicated
> their need for (transparent) complex ODE solvers. Your approach might solve
> this.
>
> Personally, I advise against using old solvers as lsode or vode as
> available in scipy for actual academic (or industrial) use.
>
> Benny
>


Hi Benny,

I'm familiar with `odes`, but I haven't looked at it in a while.  I just
cloned the repo, and I'll start poking around.  I don't think it will be
difficult to do a wrapper for `odes` similar to what `odeintw` does for
scipy's `odeint`.

Warren



>
>
> 2015-11-16 23:12 GMT+01:00 Warren Weckesser <warren.weckesser at gmail.com>:
>
>>
>>
>> On Mon, Nov 16, 2015 at 4:46 PM, Marcel Oliver <
>> m.oliver at jacobs-university.de> wrote:
>>
>>> Warren Weckesser writes:
>>>  > I wrote a wrapper of odeint called odeintw that is currently
>>> available on
>>>  > github: https://github.com/WarrenWeckesser/odeintw
>>>  >
>>>  > To quote from README.md in the repository:
>>>  >
>>>  >     odeintw provides a wrapper of scipy.integrate.odeint that allows
>>>  >     it to handle complex and matrix differential equations.
>>>  >     That is, it can solve equations of the form
>>>  >
>>>  >         dZ/dt = F(Z, t, param1, param2, ...)
>>>  >
>>>  >     where t is real and Z is a real or complex array.
>>>  >
>>>  > What do folks think of adding this functionality directly to odeint
>>> in scipy?
>>>  > It would be completely backwards compatible, and would not introduce
>>> any
>>>  > overhead in the callbacks if the inputs are 1-d real arrays.
>>>  >
>>>  > Warren
>>>
>>> I cannot really comment on the code internals, but to have transparent
>>> nd-array-valued vector fields would be a major improvement (so don't
>>> stop at "matrix"!).
>>>
>>
>>
>> I shouldn't have said "matrix", because in fact it already handles
>> n-dimensional arrays.
>> The case of a 2-d matrix is just the most common request that I've seen.
>>
>> For example, in the following a system in the shape of a 2x2x2 array is
>> solved:
>>
>>
>> In [21]: from odeintw import odeintw
>>
>> In [22]: def func(x, t):
>>    ....:     return -x
>>    ....:
>>
>> In [23]: x0 = np.arange(8.0).reshape(2,2,2)
>>
>> In [24]: t = np.arange(5.0)
>>
>> In [25]: sol = odeintw(func, x0, t)
>>
>> In [26]: sol.shape
>> Out[26]: (5, 2, 2, 2)
>>
>>
>>
>>
>>> I have encountered the need over and over again, and one can always
>>> flatten out the structures, but a lot of code clarity, in particular
>>> the closeness between mathematical representation and the code, is
>>> getting lost.
>>>
>>> And why stop at odeint?  There is integrate.ode which currently has
>>> the same limitation.
>>>
>>>
>>
>> That's certainly possible.  I referred to odeintw as a wrapper of odeint
>> (and it is), but most of the code is actually about wrapping the user's
>> callback functions (i.e. the differential equations and the Jacobian
>> function, if given).
>>
>> Warren
>>
>>
>>
>>
>>> --Marcel
>>> _______________________________________________
>>> SciPy-Dev mailing list
>>> SciPy-Dev at scipy.org
>>> https://mail.scipy.org/mailman/listinfo/scipy-dev
>>>
>>
>>
>> _______________________________________________
>> SciPy-Dev mailing list
>> SciPy-Dev at scipy.org
>> https://mail.scipy.org/mailman/listinfo/scipy-dev
>>
>>
>
> _______________________________________________
> SciPy-Dev mailing list
> SciPy-Dev at scipy.org
> https://mail.scipy.org/mailman/listinfo/scipy-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scipy-dev/attachments/20151116/403c0b01/attachment.html>

From benny.malengier at gmail.com  Tue Nov 17 03:06:10 2015
From: benny.malengier at gmail.com (Benny Malengier)
Date: Tue, 17 Nov 2015 09:06:10 +0100
Subject: [SciPy-Dev] Extend odeint to handle matrices and complex numbers
In-Reply-To: <CAGzF1ucAH8GN7PJBBHix87DfpXsRk-huHbQh=q1cRcHFWhmBqA@mail.gmail.com>
References: <CAGzF1ucKDnyAggSz3ouQgDEjhuATpq-XEi6WUOLNOSVhWArTag@mail.gmail.com>
 <22090.20173.949807.933577@localhost.localdomain>
 <CAGzF1ufV0P_wxWLDFoXFjnnJe5i2uLRYHSs=PF0P_dJvg8sYkQ@mail.gmail.com>
 <CAOyNq8ocmj_sEzT4=UwNCKd4LGOTLynFROYAVy77LLwvLhaSBg@mail.gmail.com>
 <CAGzF1ucAH8GN7PJBBHix87DfpXsRk-huHbQh=q1cRcHFWhmBqA@mail.gmail.com>
Message-ID: <CAOyNq8qJM4qvvU8J4e5Cij65V+r=Oo2aSD9MkvLgsroDKr6bwg@mail.gmail.com>

2015-11-17 0:19 GMT+01:00 Warren Weckesser <warren.weckesser at gmail.com>:

>
>
> On Mon, Nov 16, 2015 at 6:00 PM, Benny Malengier <
> benny.malengier at gmail.com> wrote:
>
>> Warren,
>>
>> It would be most useful to add that to odes,
>> https://github.com/bmcage/odes which interfaces the modern (Krylov
>> precond, ...) cvode code instead of the old codebases included in scipy.
>> We have been working on a new api for odes which returns named tuples,
>> see the examples here:
>> https://github.com/bmcage/odes/tree/master/docs/ipython (chrome might be
>> needed). This as a way to move forward the ode solvers in scipy which don't
>> have a nice/good/liked API (many suggestions for rewrites have been
>> suggested in the past). The solution as named tuple can be accessed as
>> sol.values.t and roots eg sol.roots.t, ...
>>
>> One drawback to deprecate the old solvers in scipy has been the complex
>> solver zvode. Allen Hindmarsh (
>> http://history.siam.org/oralhistories/hindmarsh.htm) did not include a
>> complex solver in the new codes in sundials. Several people have indicated
>> their need for (transparent) complex ODE solvers. Your approach might solve
>> this.
>>
>> Personally, I advise against using old solvers as lsode or vode as
>> available in scipy for actual academic (or industrial) use.
>>
>> Benny
>>
>
>
> Hi Benny,
>
> I'm familiar with `odes`, but I haven't looked at it in a while.  I just
> cloned the repo, and I'll start poking around.  I don't think it will be
> difficult to do a wrapper for `odes` similar to what `odeintw` does for
> scipy's `odeint`.
>
> Warren
>

Great.

Instead of a wrapper an option that activates complex/matrix is also a
possible way. Possibly somewhat more transparent in actual use (eg set
option to true if any of the given parameters are complex, instead of an if
structure to select another integrator obejct (eg odew instead of ode).

There are 3 pull requests being worked on,
https://github.com/bmcage/odes/pulls, which when ready should finalize the
new API. After that a release is planned. All inputs on improvements to the
new API are welcome.

Idea is to later use cvodes instead of cvode as soon as options indicate
user wants also sensitivity output, after which also sensitivities would be
present in the output named tuple. Handling complex/matrix with an option
would fit with this approach also.

Benny
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scipy-dev/attachments/20151117/60d5e9a7/attachment.html>

From sturla.molden at gmail.com  Thu Nov 19 01:23:15 2015
From: sturla.molden at gmail.com (Sturla Molden)
Date: Thu, 19 Nov 2015 07:23:15 +0100
Subject: [SciPy-Dev] optimize - add algorithm for global optimization:
	GenSA
In-Reply-To: <CAB6mnxJCD+Mr98GdGKQdhKLUe8nTmxCorKLWBTVq6EM_j1AN3A@mail.gmail.com>
References: <A3786E9825AE3D4F8CC0EC80FBFAD32E02349A58CC@PMICHLAUEXM23.PMINTL.NET>
 <CAF6FJiuR-DpCBb8kVso_tpHy5D_TeHGtXyY0tY=XJy8K=cG5vA@mail.gmail.com>
 <CAEEL3uxmvjwDZgfL=Ft0BHtuctpKsVzuvMZrXFFU_NKLS4waJg@mail.gmail.com>
 <CAB6mnxJCD+Mr98GdGKQdhKLUe8nTmxCorKLWBTVq6EM_j1AN3A@mail.gmail.com>
Message-ID: <n2jprl$u4t$1@ger.gmane.org>

On 15/11/15 04:51, Charles R Harris wrote:

> The most reliable, if slow, method was a GA.

GAs are great, but it hard to wrap them in a general routine we could 
put in SciPy. The parameters have to be codable in such a way that we 
can meaningfully simulate recombinations and mutations.

Tabu search is also a method that can compete with SA, but does not have 
the same tendency to go astray.


Sturla





From charlesr.harris at gmail.com  Thu Nov 19 13:04:20 2015
From: charlesr.harris at gmail.com (Charles R Harris)
Date: Thu, 19 Nov 2015 11:04:20 -0700
Subject: [SciPy-Dev] optimize - add algorithm for global optimization:
	GenSA
In-Reply-To: <n2jprl$u4t$1@ger.gmane.org>
References: <A3786E9825AE3D4F8CC0EC80FBFAD32E02349A58CC@PMICHLAUEXM23.PMINTL.NET>
 <CAF6FJiuR-DpCBb8kVso_tpHy5D_TeHGtXyY0tY=XJy8K=cG5vA@mail.gmail.com>
 <CAEEL3uxmvjwDZgfL=Ft0BHtuctpKsVzuvMZrXFFU_NKLS4waJg@mail.gmail.com>
 <CAB6mnxJCD+Mr98GdGKQdhKLUe8nTmxCorKLWBTVq6EM_j1AN3A@mail.gmail.com>
 <n2jprl$u4t$1@ger.gmane.org>
Message-ID: <CAB6mnxL40nCHM6wq7PsMt6zodWLjgzs2zYzCKZbTn-TPBuHppA@mail.gmail.com>

On Wed, Nov 18, 2015 at 11:23 PM, Sturla Molden <sturla.molden at gmail.com>
wrote:

> On 15/11/15 04:51, Charles R Harris wrote:
>
> The most reliable, if slow, method was a GA.
>>
>
> GAs are great, but it hard to wrap them in a general routine we could put
> in SciPy. The parameters have to be codable in such a way that we can
> meaningfully simulate recombinations and mutations.
>
> Tabu search is also a method that can compete with SA, but does not have
> the same tendency to go astray.
>

Might be interesting to discretize some of the test functions by rounding
either the function or its arguments. I expect step size would be critical
for algorithms heavily dependent on gradients.

Chuck
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scipy-dev/attachments/20151119/7bd6deaf/attachment.html>

From fperez.net at gmail.com  Thu Nov 19 16:42:21 2015
From: fperez.net at gmail.com (Fernando Perez)
Date: Thu, 19 Nov 2015 13:42:21 -0800
Subject: [SciPy-Dev] [JOB] Project Jupyter is hiring two postdoctoral
	fellows @ UC Berkeley.
Message-ID: <CAHAreOqnDRigyBqbTHzg_pTyVhNQ14Yz2EacZaiMDnt-7JA-zQ@mail.gmail.com>

Hi all,

We are delighted to announce today that Project Jupyter/IPython has two
postdoctoral fellowships open at UC Berkeley, open immediately.  Interested
candidates can apply here:

https://aprecruit.berkeley.edu/apply/JPF00899

We hope to find candidates who will work on a number of challenging
questions over the next few years, as described in our grant proposal here:

http://blog.jupyter.org/2015/07/07/project-jupyter-computational-narratives-as-the-engine-of-collaborative-data-science/

Interested candidates should carefully read that proposal before applying
to familiarize themselves with the full scope of the questions we intend to
tackle.

We'd like to thank the support of the Helmsley Trust, the Gordon and Betty
Moore Foundation and the Alfred P. Sloan Foundation.

Cheers,

Brian Granger and Fernando Perez.

-- 
Fernando Perez (@fperez_org; http://fperez.org)
fperez.net-at-gmail: mailing lists only (I ignore this when swamped!)
fernando.perez-at-berkeley: contact me here for any direct mail
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scipy-dev/attachments/20151119/aae1fc0a/attachment.html>

From ralf.gommers at gmail.com  Sun Nov 22 15:41:03 2015
From: ralf.gommers at gmail.com (Ralf Gommers)
Date: Sun, 22 Nov 2015 21:41:03 +0100
Subject: [SciPy-Dev] plans for 0.17.0 release
In-Reply-To: <CAMRo0iuDJ27sNybHFD2JLBRZtQvtYjpYaPO-YxWba8eM6zuOFg@mail.gmail.com>
References: <CABL7CQhJPnwAz+WGanP_HbASDGOVx=_Kh4zztb2hyZU-acuD3w@mail.gmail.com>
 <CAMRo0iuDJ27sNybHFD2JLBRZtQvtYjpYaPO-YxWba8eM6zuOFg@mail.gmail.com>
Message-ID: <CABL7CQhFsai2E3b-5+D9ha1j+U+g9t49XmnkC651AZbM_VCwSA@mail.gmail.com>

On Sat, Nov 7, 2015 at 11:14 PM, Evgeni Burovski <evgeny.burovskiy at gmail.com
> wrote:

> On Sun, Oct 25, 2015 at 5:14 PM, Ralf Gommers <ralf.gommers at gmail.com>
> wrote:
> > Hi all,
> >
> > We branched 0.16.x in early May, so it's time to start thinking about
> > branching 0.17.x sometime soon. If anyone would be interested in being
> the
> > release manager for 0.17.x, or co-managing it with me, that would be very
> > welcome.
> >
> > In order to make releasing look less like black magic I've worked on
> > automating stuff a bit more (has landed in master) and on writing fairly
> > detailed documentation: https://github.com/scipy/scipy/pull/5424. Please
> > have a look if you're interested.
> >
> > Cheers,
> > Ralf
> >
> >
> > _______________________________________________
> > SciPy-Dev mailing list
> > SciPy-Dev at scipy.org
> > https://mail.scipy.org/mailman/listinfo/scipy-dev
> >
>
>
> I volunteer to co-manage it with Ralf.
>
> Here's aproposal for the release schedule:
>
> branching 0.17.x: 22 Nov
>

Looks like we won't be making it by today, there's a few too many things
that need to go in still open. Doesn't look too bad though, maybe in 3-4
days we'll be good to go.

Ralf
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scipy-dev/attachments/20151122/99dd0b54/attachment.html>

From david.mikolas1 at gmail.com  Mon Nov 23 02:47:10 2015
From: david.mikolas1 at gmail.com (David Mikolas)
Date: Mon, 23 Nov 2015 15:47:10 +0800
Subject: [SciPy-Dev] dense output option in integrate.ode
In-Reply-To: <CAMRo0itRn-P_d_Pp2ocymJUHgYy=fChBy8YFEC77-8=TU9hM4A@mail.gmail.com>
References: <CAMRo0itRn-P_d_Pp2ocymJUHgYy=fChBy8YFEC77-8=TU9hM4A@mail.gmail.com>
Message-ID: <CAFYhK+7h73ZOFH55yA1_fy_s6KYt4ByJDe2+OEsYbJ2PFc_wtg@mail.gmail.com>

One way that dense output for dopri5 and dop853 can be very useful is if
the integration is expensive/long and you don't want to repeat it, but you
want to obtain results at new time points at a later date, or even iterate
on it - for example, find the time of closest approac. This is done by
saving the interpolation coefficients. I put a simple example here, though
there is no saving to disk yet.

    http://pastebin.com/e6qNjbL9   dendop_test_v00.py

I wonder if this could be developed into an option to return an
interpolator object or function. If I can help let me know.

On Sat, Nov 14, 2015 at 12:19 AM, Evgeni Burovski <
evgeny.burovskiy at gmail.com> wrote:

> Hi David, Hi Jim,
>
> > I am new to SciPy-dev. Will the dense output option in
> scipy.integrate.ode
> > become available in 0.17.0? This is a feature already available in the
> > original FORTAN, but wasn't implemented in the wrapper.
>
> If the feature is sent as a pull request against the scipy master
> branch, the PR is reviewed by the maintainers of the integrate package
> and merged into master before the release split, then yes, it would be
> available in 0.17.0.
>
> So far I do not see any progress towards it.
>
>
> > I wrote these dense output extensions that you listed:
> >
> >
> https://github.com/jddmartin/scipy/tree/dense_output_from_dopri5_and_dop853
> > https://github.com/jddmartin/dense_output_example_usage
> >
> > but I didn't issue any pull request to scipy.  I sent this message to the
> > scipy developers list:
> >   http://article.gmane.org/gmane.comp.python.scientific.devel/19635/
> > explaining the changes.  I was hoping for some feedback before issuing a
> > pull request.
>
> Ah, I see that the email likely fell through the cracks back in April.
> Sorry about that.
> You might want to ping that email thread once more or send a pull
> request on github (or both).
>
> <snip>
>
> Cheers,
>
> Evgeni
> _______________________________________________
> SciPy-Dev mailing list
> SciPy-Dev at scipy.org
> https://mail.scipy.org/mailman/listinfo/scipy-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scipy-dev/attachments/20151123/4a717522/attachment.html>

From evgeny.burovskiy at gmail.com  Mon Nov 23 03:28:45 2015
From: evgeny.burovskiy at gmail.com (Evgeni Burovski)
Date: Mon, 23 Nov 2015 08:28:45 +0000
Subject: [SciPy-Dev] dense output option in integrate.ode
In-Reply-To: <CAFYhK+7h73ZOFH55yA1_fy_s6KYt4ByJDe2+OEsYbJ2PFc_wtg@mail.gmail.com>
References: <CAMRo0itRn-P_d_Pp2ocymJUHgYy=fChBy8YFEC77-8=TU9hM4A@mail.gmail.com>
 <CAFYhK+7h73ZOFH55yA1_fy_s6KYt4ByJDe2+OEsYbJ2PFc_wtg@mail.gmail.com>
Message-ID: <CAMRo0iuX=GnFSVc3CN_Xse2yXSmLu6O_bPEE3p=HRXCJBCOo9A@mail.gmail.com>

Hi David,

This sounds like a useful feature.
I guess a pull request would be welcome. (even though I'm not a user, and I
won't be able to review it).

Evgeni
23.11.2015 10:47 ???????????? "David Mikolas" <david.mikolas1 at gmail.com>
???????:

> One way that dense output for dopri5 and dop853 can be very useful is if
> the integration is expensive/long and you don't want to repeat it, but you
> want to obtain results at new time points at a later date, or even iterate
> on it - for example, find the time of closest approac. This is done by
> saving the interpolation coefficients. I put a simple example here, though
> there is no saving to disk yet.
>
>     http://pastebin.com/e6qNjbL9   dendop_test_v00.py
>
> I wonder if this could be developed into an option to return an
> interpolator object or function. If I can help let me know.
>
> On Sat, Nov 14, 2015 at 12:19 AM, Evgeni Burovski <
> evgeny.burovskiy at gmail.com> wrote:
>
>> Hi David, Hi Jim,
>>
>> > I am new to SciPy-dev. Will the dense output option in
>> scipy.integrate.ode
>> > become available in 0.17.0? This is a feature already available in the
>> > original FORTAN, but wasn't implemented in the wrapper.
>>
>> If the feature is sent as a pull request against the scipy master
>> branch, the PR is reviewed by the maintainers of the integrate package
>> and merged into master before the release split, then yes, it would be
>> available in 0.17.0.
>>
>> So far I do not see any progress towards it.
>>
>>
>> > I wrote these dense output extensions that you listed:
>> >
>> >
>> https://github.com/jddmartin/scipy/tree/dense_output_from_dopri5_and_dop853
>> > https://github.com/jddmartin/dense_output_example_usage
>> >
>> > but I didn't issue any pull request to scipy.  I sent this message to
>> the
>> > scipy developers list:
>> >   http://article.gmane.org/gmane.comp.python.scientific.devel/19635/
>> > explaining the changes.  I was hoping for some feedback before issuing a
>> > pull request.
>>
>> Ah, I see that the email likely fell through the cracks back in April.
>> Sorry about that.
>> You might want to ping that email thread once more or send a pull
>> request on github (or both).
>>
>> <snip>
>>
>> Cheers,
>>
>> Evgeni
>> _______________________________________________
>> SciPy-Dev mailing list
>> SciPy-Dev at scipy.org
>> https://mail.scipy.org/mailman/listinfo/scipy-dev
>>
>
>
> _______________________________________________
> SciPy-Dev mailing list
> SciPy-Dev at scipy.org
> https://mail.scipy.org/mailman/listinfo/scipy-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scipy-dev/attachments/20151123/b00a68b0/attachment.html>

From david.mikolas1 at gmail.com  Mon Nov 23 06:43:28 2015
From: david.mikolas1 at gmail.com (David Mikolas)
Date: Mon, 23 Nov 2015 19:43:28 +0800
Subject: [SciPy-Dev] dense output option in integrate.ode
In-Reply-To: <CAMRo0iuX=GnFSVc3CN_Xse2yXSmLu6O_bPEE3p=HRXCJBCOo9A@mail.gmail.com>
References: <CAMRo0itRn-P_d_Pp2ocymJUHgYy=fChBy8YFEC77-8=TU9hM4A@mail.gmail.com>
 <CAFYhK+7h73ZOFH55yA1_fy_s6KYt4ByJDe2+OEsYbJ2PFc_wtg@mail.gmail.com>
 <CAMRo0iuX=GnFSVc3CN_Xse2yXSmLu6O_bPEE3p=HRXCJBCOo9A@mail.gmail.com>
Message-ID: <CAFYhK+76G99SyUSLiCVCM=AQ2ALyfqOMXV2NmOYpDdeUQmU5Uw@mail.gmail.com>

OK, Jim Martin has started this and I'm trying to add momentum and any help
that I can. I'm already using this feature (access to interpolation
coefficients) by installing an environment with his changes, so I can run
the integration flat out, yet get results at arbitrary density with nearly
the same precision.

I'm new to SciPy-dev and mail lists in general, appreciate any/all
suggestions. Thanks!

David

On Mon, Nov 23, 2015 at 4:28 PM, Evgeni Burovski <evgeny.burovskiy at gmail.com
> wrote:

> Hi David,
>
> This sounds like a useful feature.
> I guess a pull request would be welcome. (even though I'm not a user, and
> I won't be able to review it).
>
> Evgeni
> 23.11.2015 10:47 ???????????? "David Mikolas" <david.mikolas1 at gmail.com>
> ???????:
>
> One way that dense output for dopri5 and dop853 can be very useful is if
>> the integration is expensive/long and you don't want to repeat it, but you
>> want to obtain results at new time points at a later date, or even iterate
>> on it - for example, find the time of closest approac. This is done by
>> saving the interpolation coefficients. I put a simple example here, though
>> there is no saving to disk yet.
>>
>>     http://pastebin.com/e6qNjbL9   dendop_test_v00.py
>>
>> I wonder if this could be developed into an option to return an
>> interpolator object or function. If I can help let me know.
>>
>> On Sat, Nov 14, 2015 at 12:19 AM, Evgeni Burovski <
>> evgeny.burovskiy at gmail.com> wrote:
>>
>>> Hi David, Hi Jim,
>>>
>>> > I am new to SciPy-dev. Will the dense output option in
>>> scipy.integrate.ode
>>> > become available in 0.17.0? This is a feature already available in the
>>> > original FORTAN, but wasn't implemented in the wrapper.
>>>
>>> If the feature is sent as a pull request against the scipy master
>>> branch, the PR is reviewed by the maintainers of the integrate package
>>> and merged into master before the release split, then yes, it would be
>>> available in 0.17.0.
>>>
>>> So far I do not see any progress towards it.
>>>
>>>
>>> > I wrote these dense output extensions that you listed:
>>> >
>>> >
>>> https://github.com/jddmartin/scipy/tree/dense_output_from_dopri5_and_dop853
>>> > https://github.com/jddmartin/dense_output_example_usage
>>> >
>>> > but I didn't issue any pull request to scipy.  I sent this message to
>>> the
>>> > scipy developers list:
>>> >   http://article.gmane.org/gmane.comp.python.scientific.devel/19635/
>>> > explaining the changes.  I was hoping for some feedback before issuing
>>> a
>>> > pull request.
>>>
>>> Ah, I see that the email likely fell through the cracks back in April.
>>> Sorry about that.
>>> You might want to ping that email thread once more or send a pull
>>> request on github (or both).
>>>
>>> <snip>
>>>
>>> Cheers,
>>>
>>> Evgeni
>>> _______________________________________________
>>> SciPy-Dev mailing list
>>> SciPy-Dev at scipy.org
>>> https://mail.scipy.org/mailman/listinfo/scipy-dev
>>>
>>
>>
>> _______________________________________________
>> SciPy-Dev mailing list
>> SciPy-Dev at scipy.org
>> https://mail.scipy.org/mailman/listinfo/scipy-dev
>>
>>
> _______________________________________________
> SciPy-Dev mailing list
> SciPy-Dev at scipy.org
> https://mail.scipy.org/mailman/listinfo/scipy-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scipy-dev/attachments/20151123/378e0085/attachment.html>

From kitchi.srikrishna at gmail.com  Sun Nov 29 04:59:37 2015
From: kitchi.srikrishna at gmail.com (Sri Krishna)
Date: Sun, 29 Nov 2015 15:29:37 +0530
Subject: [SciPy-Dev] Limit number of iterations in sigma clipping
In-Reply-To: <CAMMTP+DTEHxa=01=9=QBUCykD6H-4HP9bJCLAGWdu8ZgvFcsSA@mail.gmail.com>
References: <CA+qdgMdkHEQQGZ3Cn=bhVftoF+1hXnw98WR4c3T3Rmm=1QvHbA@mail.gmail.com>
 <CABL7CQi+o2qKn1RhUYUzSQaDA0ABH9n9knq+ku5xwKEfsD07Gw@mail.gmail.com>
 <CA+qdgMc0P+ZBh7526JcFibqc1kM2Jm=eZUXyjr1WPXcgWytxvg@mail.gmail.com>
 <CABL7CQgS=0buBi2pGUeTzi2ThA2EnxRPvLiAS4g88UB4iMgSKg@mail.gmail.com>
 <CAMMTP+DTEHxa=01=9=QBUCykD6H-4HP9bJCLAGWdu8ZgvFcsSA@mail.gmail.com>
Message-ID: <CA+qdgMfc_BpOTqyLD7E-4g2r+fATEZmSyaS46qSmEjrMrOwPqQ@mail.gmail.com>

On 18 October 2015 at 21:12, <josef.pktd at gmail.com> wrote:

>
>
>
> On Sun, Oct 18, 2015 at 11:23 AM, Ralf Gommers <ralf.gommers at gmail.com>
> wrote:
>
>>
>>
>> On Sun, Oct 18, 2015 at 1:18 PM, Sri Krishna <kitchi.srikrishna at gmail.com
>> > wrote:
>>
>>> On 18 October 2015 at 15:05, Ralf Gommers <ralf.gommers at gmail.com>
>>> wrote:
>>>
>>>>
>>>>
>>>>
>>>> On Fri, Oct 16, 2015 at 6:21 AM, Sri Krishna <
>>>> kitchi.srikrishna at gmail.com> wrote:
>>>>
>>>>> Hi,
>>>>>
>>>>> The current sigma clipping function has no iters keyword.
>>>>>
>>>>
>>>> For clarity: this is stats.sigmaclip
>>>>
>>>>
>>>>> I feel it would be useful to include it in the function, with a
>>>>> default like `iters=None` so that if people do want to use the keyword they
>>>>> will be able to do so.
>>>>>
>>>>> With the defaults of `iters = None` it shouldn't break anyone's
>>>>> workflow and will correspond to the current default behaviour.
>>>>>
>>>>> If there is any interest in this, I can file a pull request soon.
>>>>>
>>>>
>>>> Makes sense to me to add this.
>>>>
>>>> I just saw that AstroPy has a much more extensive sigma clipping
>>>> function. Replacing stats.sigmaclip with
>>>> http://docs.astropy.org/en/latest/api/astropy.stats.sigma_clip.html
>>>> would be the way to go I think. In a backwards-compatible fashion of
>>>> course, so the `sigma` keyword from that function should be left out.
>>>>
>>>
>>> Well, I am working on a pull request on the astropy sigma clipping
>>> routine to incorporate the scipy function - The scipy function can be upto
>>> 100x faster than the astropy function in benchmarks that I've run, and I'm
>>> not convinced that it is worthwhile loosing the speed for features. So once
>>> the PR is merged, the astropy function will have the option to use the
>>> scipy function as well as the existing astropy function,
>>>
>>
>> Ouch, 100x is a lot.
>>
>>
>>> As far as I can tell, the speed improvements in the
>>> scipy.stats.sigmaclip function come because we don't use masked arrays, and
>>> the sigma clipped data is discarded every iteration. We could potentially
>>> use another function instead of the mean (i.e., median or a user supplied
>>> function analogous to astropy) without too much of a speed hit.
>>>
>>> The only three features we need to port from the astropy function are
>>> the `cenfunc` and `varfunc` keywords to define a custom function to
>>> calculate mean/standard deviation and the `axis` keyword. I'm not sure we
>>> need to use masked arrays because of the previously mentioned performance
>>> implications.
>>>
>>> So should I work on a pull request to adapt the astropy function into
>>> scipy in a backwards compatible manner?
>>>
>>
>> All the keyword you mentioned make sense to add I think, so yes please.
>>
>> And a bit of bikeshedding: the names used by AstroPy aren't optimal imho.
>> Maybe:
>> - iters --> numiter
>>
>
> statsmodels uses maxiter in cases like this because it can stop early
>
>
>> - cenfunc --> centerfunc
>>
>
> I think this should correspond to the robust ANOVA function, IIRC
>
>
> names have power and are not bikesheds
>
>
> (default color choices ain't no bikesheds either)
>
>
>
>
I've taken my time about it, but finally got around to finishing the PR here
<https://github.com/scipy/scipy/pull/5559>.

Do let me know if I need to add/remove/throw away/fix anything! I hope I've
done everything correctly - This is the first time I'm submitting a PR to
scipy.

Thanks,
Krishna
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scipy-dev/attachments/20151129/ec8a9b96/attachment.html>

From charlesr.harris at gmail.com  Sun Nov 29 15:28:47 2015
From: charlesr.harris at gmail.com (Charles R Harris)
Date: Sun, 29 Nov 2015 13:28:47 -0700
Subject: [SciPy-Dev] Python development on fedora 23.
Message-ID: <CAB6mnxLzF8y=zraENAJ2xuL2XZLUWnm6N6+hyOH_ddZdci4Y3A@mail.gmail.com>

Hi Fedora users,

Python distutils on fedora 23 is configured for hardening, hence
`redhat-rpm-config` is a dependency if you want to build numpy, scipy, etc.
The symptom is a "broken toolchain" error. A bug is open for this and it
might get fixed on the Python end, but I don't expect anything soon.

Chuck
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scipy-dev/attachments/20151129/24b66c47/attachment.html>