From rlucas7 at vt.edu  Sat Feb  1 11:56:56 2020
From: rlucas7 at vt.edu (Lucas Roberts)
Date: Sat, 1 Feb 2020 11:56:56 -0500
Subject: [SciPy-Dev] language labels in github issues/PRs?
Message-ID: <CADCr_xOm9iv4xKKBBAtnTCbt57bYmaG6c4QS9x5Y5oGaQwcNEQ@mail.gmail.com>

Hi scipy-dev,

I was looking at my stack of emails for scipy and noticed some comments on:
https://github.com/scipy/scipy/issues/3203#
and that the c code is involved. In the past I've marked these types of
emails for myself to flag for follow up (e.g. I'd like to work on them if
still open in the future). Problem is that the stack of these flags gets
too large in my inbox. I noticed there aren't tags for language in the
scipy github. Scipy has C, C++, Cython, Fortran, and Python (I think that
is all) used in various places.

My question is would it be helpful to have language tags?

We probably wouldnt' want a python tag but if an issue or a PR touches
C/C++/Cython/fortran adding a label would help discoverability and
exclusion.

This might help searching of new contributors and for core team, e.g. you
could search for issues touching c code (in my case) or exclude issues
touching fortran (also my case, I don't know fortran).

I didn't know how to exclude it Github and learned that it has some
searching functionality that allows to exclude issues/PRs with certain tags:

https://help.github.com/en/github/searching-for-information-on-github/searching-issues-and-pull-requests#search-by-label

-- 
Sincerely,
-Lucas
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scipy-dev/attachments/20200201/f1481871/attachment.html>

From ralf.gommers at gmail.com  Sat Feb  1 13:21:07 2020
From: ralf.gommers at gmail.com (Ralf Gommers)
Date: Sat, 1 Feb 2020 19:21:07 +0100
Subject: [SciPy-Dev] language labels in github issues/PRs?
In-Reply-To: <CADCr_xOm9iv4xKKBBAtnTCbt57bYmaG6c4QS9x5Y5oGaQwcNEQ@mail.gmail.com>
References: <CADCr_xOm9iv4xKKBBAtnTCbt57bYmaG6c4QS9x5Y5oGaQwcNEQ@mail.gmail.com>
Message-ID: <CABL7CQg2-ETsQSFKGmS9eC02oY47PN8F+pEUjYy9VWB0=yFjwA@mail.gmail.com>

On Sat, Feb 1, 2020 at 6:59 PM Lucas Roberts <rlucas7 at vt.edu> wrote:

> Hi scipy-dev,
>
> I was looking at my stack of emails for scipy and noticed some comments on:
> https://github.com/scipy/scipy/issues/3203#
> and that the c code is involved. In the past I've marked these types of
> emails for myself to flag for follow up (e.g. I'd like to work on them if
> still open in the future). Problem is that the stack of these flags gets
> too large in my inbox. I noticed there aren't tags for language in the
> scipy github. Scipy has C, C++, Cython, Fortran, and Python (I think that
> is all) used in various places.
>
> My question is would it be helpful to have language tags?
>

Sounds like a good idea to me, +1 for "language: Fortran" and "language
Cython" tags. Not sure if C and C++ should be separate or together.

Cheers,
Ralf


> We probably wouldnt' want a python tag but if an issue or a PR touches
> C/C++/Cython/fortran adding a label would help discoverability and
> exclusion.
>
> This might help searching of new contributors and for core team, e.g. you
> could search for issues touching c code (in my case) or exclude issues
> touching fortran (also my case, I don't know fortran).
>
> I didn't know how to exclude it Github and learned that it has some
> searching functionality that allows to exclude issues/PRs with certain tags:
>
>
> https://help.github.com/en/github/searching-for-information-on-github/searching-issues-and-pull-requests#search-by-label
>
> --
> Sincerely,
> -Lucas
> _______________________________________________
> SciPy-Dev mailing list
> SciPy-Dev at python.org
> https://mail.python.org/mailman/listinfo/scipy-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scipy-dev/attachments/20200201/3b1b28c1/attachment.html>

From kaistriega at gmail.com  Sun Feb  2 02:37:25 2020
From: kaistriega at gmail.com (Kai Striega)
Date: Sun, 2 Feb 2020 15:37:25 +0800
Subject: [SciPy-Dev] language labels in github issues/PRs?
In-Reply-To: <CABL7CQg2-ETsQSFKGmS9eC02oY47PN8F+pEUjYy9VWB0=yFjwA@mail.gmail.com>
References: <CADCr_xOm9iv4xKKBBAtnTCbt57bYmaG6c4QS9x5Y5oGaQwcNEQ@mail.gmail.com>
 <CABL7CQg2-ETsQSFKGmS9eC02oY47PN8F+pEUjYy9VWB0=yFjwA@mail.gmail.com>
Message-ID: <CAPDXd17F4HGpJeS9S--+UZrnYiqW5rHbjzL_Pme3JnNKxLEdWg@mail.gmail.com>

Sounds like a good idea to me. I would also like to second not having a
"python" specific label.

On Sun., 2 Feb. 2020, 02:21 Ralf Gommers, <ralf.gommers at gmail.com> wrote:

>
>
> On Sat, Feb 1, 2020 at 6:59 PM Lucas Roberts <rlucas7 at vt.edu> wrote:
>
>> Hi scipy-dev,
>>
>> I was looking at my stack of emails for scipy and noticed some comments
>> on:
>> https://github.com/scipy/scipy/issues/3203#
>> and that the c code is involved. In the past I've marked these types of
>> emails for myself to flag for follow up (e.g. I'd like to work on them if
>> still open in the future). Problem is that the stack of these flags gets
>> too large in my inbox. I noticed there aren't tags for language in the
>> scipy github. Scipy has C, C++, Cython, Fortran, and Python (I think that
>> is all) used in various places.
>>
>> My question is would it be helpful to have language tags?
>>
>
> Sounds like a good idea to me, +1 for "language: Fortran" and "language
> Cython" tags. Not sure if C and C++ should be separate or together.
>
> Cheers,
> Ralf
>
>
>> We probably wouldnt' want a python tag but if an issue or a PR touches
>> C/C++/Cython/fortran adding a label would help discoverability and
>> exclusion.
>>
>> This might help searching of new contributors and for core team, e.g. you
>> could search for issues touching c code (in my case) or exclude issues
>> touching fortran (also my case, I don't know fortran).
>>
>> I didn't know how to exclude it Github and learned that it has some
>> searching functionality that allows to exclude issues/PRs with certain tags:
>>
>>
>> https://help.github.com/en/github/searching-for-information-on-github/searching-issues-and-pull-requests#search-by-label
>>
>> --
>> Sincerely,
>> -Lucas
>> _______________________________________________
>> SciPy-Dev mailing list
>> SciPy-Dev at python.org
>> https://mail.python.org/mailman/listinfo/scipy-dev
>>
> _______________________________________________
> SciPy-Dev mailing list
> SciPy-Dev at python.org
> https://mail.python.org/mailman/listinfo/scipy-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scipy-dev/attachments/20200202/40914fe5/attachment.html>

From ralf.gommers at gmail.com  Sun Feb  2 16:14:24 2020
From: ralf.gommers at gmail.com (Ralf Gommers)
Date: Sun, 2 Feb 2020 22:14:24 +0100
Subject: [SciPy-Dev] Proposed addition to
 scipy.sparse.csgraph.maximum_flow
In-Reply-To: <kQAmzvZjs17K_020-WNkxJtR2Lwd0qoX1-y01kElSufAptXNem4qcBb4Q1k0_iMzLIRwf-Ar9g92XATRu9eZN7D4pX58MIu2b9busLLkm_M=@protonmail.com>
References: <kQAmzvZjs17K_020-WNkxJtR2Lwd0qoX1-y01kElSufAptXNem4qcBb4Q1k0_iMzLIRwf-Ar9g92XATRu9eZN7D4pX58MIu2b9busLLkm_M=@protonmail.com>
Message-ID: <CABL7CQjd-nQTPY0LMozk9gTa80MKkdo3SjLeXzwiqJ7mGj_vvQ@mail.gmail.com>

Hi,

This reply is a bit late, sorry about that. Your changes do seem useful and
there has recently been activity on improving maximum_flow. If you could
open either an issue or a PR on GitHub, that would make it easier to
respond/review and ping an expert.

Thanks,
Ralf


On Thu, Jan 2, 2020 at 3:50 PM mhwende <mhwende at protonmail.com> wrote:

> Hi,
>
> recently I have made use of the maximum flow algorithm for
> CSR matrices, i.e., the scipy.sparse.csgraph.maximum_flow function.
>
> My test case was a non-sparse, bipartite network with lots of connections
> between the two node sets. The value of the maximum flow was small in
> comparison to the total number of nodes and edges in the graph.
> I wanted to use the scipy.sparse.csgraph module, however with a depth first
> search (DFS) instead of the breadth-first search that you have implemented.
> Therefore, I have implemented a new option to use DFS instead of BFS with
> your implementation. When measuring the execution time for a random graph,
> there was a signnificant speedup of the DFS procedure compared to the
> BFS variant.
>
> During these tests, I also refactored the two other functions that you call
> within the main csgraph.maximum_flow function, i.e., _add_reverse_edges and
> _make_edge_pointers. Depending on the input graph, I found that the time
> spent in this functions can be larger than the execution time for the
> actual
> Edmond-Karps algorithm.
>
> Below, you can find a table showing the measured execution times in
> seconds.
> Within the columns of this table, the execution time is broken down into
> parts
> as follows:
>
> a) Adding reverse edges (function _add_reverse_edges).
> b) Making edge pointers (function _make_edge_pointers).
> c) Actual max flow computation (function _edmond_karps)
> d) Total execution time (function maximum_flow).
>
> Rows of the table refer to the changes which were made to the
> implementation,
> where each measurement was performed with either DFS or BFS for the
> construction of augmenting paths:
>
> 1) Original implementation.
> 2) After the reimplementation of _make_edge_pointers.
> 3) After the reimplementation of _add_reverse_edges.
>
> Here are the execution times:
>
>         |     a)    |     b)    |     c)    |     d)
> --------+-----------+-----------+-----------+-----------
> 1. BFS  |   1.29 s  |   3.81 s  |   8.93 s  |  14.08 s
> 1. DFS  |   1.29 s  |   3.82 s  |   0.50 s  |   5.66 s
> --------+-----------+-----------+-----------+-----------
> 2. BFS  |   1.54 s  |   0.22 s  |   8.94 s  |  10.76 s
> 2. DFS  |   1.33 s  |   0.22 s  |   0.50 s  |   2.10 s
> --------+-----------+-----------+-----------+-----------
> 3. BFS  |   0.16 s  |   0.27 s  |   9.01 s  |   9.50 s
> 3. DFS  |   0.16 s  |   0.23 s  |   0.52 s  |   0.96 s
>
> By comparing the total execution times for BFS in 1d) and for
> DFS in 3d), my code changes have led to a speedup of about 14,
> for the network example which I have mentioned.
>
> Please let me know if would like to incorporate some or all of my
> changes into your code base.
> My changes (a single commit) can be found here:
>
> https://github.com/mhwende/scipy/commit/e7be2dc29364fe95f6fae20a4c0775716cdba5e5
>
> Finally, here is the script which I have used for testing.
> The above timings have been produced with the command:
>
> python scipymaxflowtest 2000 2000 --dfs=0
>
> This will create a graph with 4000 nodes and 4,000,000 edges,
> with randomly chosen zero or unit capacities.
>
> # File scipymaxflowtest.py
>
> import time
> import argparse
> import numpy as np
> import scipy.sparse as sp
>
>
> def scipymaxflowtest(layer_sizes, dfs):
>   layer_sizes = np.array(layer_sizes, dtype = np.int)
>   num_layers = len(layer_sizes)
>   print("Layer sizes:", *layer_sizes)
>   print("Number of layers:", num_layers)
>   num_nodes = 2 + np.sum(layer_sizes)
>   print("Number of nodes:", num_nodes)
>   num_edges = layer_sizes[0] + layer_sizes[-1]
>   for i in range(layer_sizes.size - 1):
>     num_edges += layer_sizes[i] * layer_sizes[i + 1]
>   print("Number of edges:", num_edges)
>   #data = np.ones(num_edges, dtype = np.int)
>   data = np.random.randint(0, 2, num_edges)
>   indices = np.zeros(num_edges, dtype = np.int)
>   indptr = np.zeros(num_nodes + 1, dtype = np.int)
>   ptr = 0
>   for j in range(layer_sizes[0]):
>     indices[ptr] = 2 + j
>     ptr += 1
>   indptr[1] = ptr
>   indptr[2] = ptr
>   layer_sum = 0
>   for k in range(num_layers - 1):
>     next_layer_sum = layer_sum + layer_sizes[k]
>     for i in range(layer_sizes[k]):
>       for j in range(layer_sizes[k + 1]):
>         indices[ptr] = 2 + next_layer_sum + j
>         ptr += 1
>       indptr[2 + layer_sum + i + 1] = ptr
>     layer_sum = next_layer_sum
>   for i in range(layer_sizes[-1]):
>     indices[ptr] = 1
>     ptr += 1
>     indptr[2 + layer_sum + i + 1] = ptr
>   indptr[num_nodes] = ptr
>
>   # Check that we set as many entries in the index array as there are
> edges.
>   assert ptr == num_edges
>
>   # Create CSR matrix from data and index arrays.
>   a = sp.csr_matrix((data, indices, indptr), shape = (num_nodes,
> num_nodes))
>   if num_nodes <= 12:
>     print("A =\n{}".format(a.toarray()))
>
>   t0 = time.time()
>   flow_result = sp.csgraph.maximum_flow(a, 0, 1, dfs)
>   t1 = time.time()
>   flow_value = flow_result.flow_value
>   print("Maximum flow: {} (took {:.6f} s)".format(flow_value, t1 - t0))
>   return flow_value
>
>
> if __name__ == "__main__":
>   parser = argparse.ArgumentParser()
>   parser.add_argument(
>       "layer_sizes", type = int, nargs = "+",
>       help = "Number of nodes on individual graph layers.")
>   parser.add_argument(
>       "--dfs", type = int, default = 0,
>       help = "Whether to user depth first search.")
>   args = parser.parse_args()
>   np.random.seed(4891243)
>   scipymaxflowtest(args.layer_sizes, args.dfs)
>
> _______________________________________________
> SciPy-Dev mailing list
> SciPy-Dev at python.org
> https://mail.python.org/mailman/listinfo/scipy-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scipy-dev/attachments/20200202/0db2d6e2/attachment.html>

From ralf.gommers at gmail.com  Sun Feb  2 16:45:00 2020
From: ralf.gommers at gmail.com (Ralf Gommers)
Date: Sun, 2 Feb 2020 22:45:00 +0100
Subject: [SciPy-Dev] Implementation choices in
 scipy.sparse.linalg.LinearOperator
In-Reply-To: <382E3555-B904-494D-9E7C-9A3339B54987@gmail.com>
References: <0464AE5D-EF1D-4B14-A9FA-DCD3497156FA@gmail.com>
 <382E3555-B904-494D-9E7C-9A3339B54987@gmail.com>
Message-ID: <CABL7CQhoWzFSRiUR+ATCJmj+GqpAqSawtmVC_gS-PPuWuYirCA@mail.gmail.com>

On Fri, Jan 24, 2020 at 6:54 PM matteo ravasi <matteoravasi at gmail.com>
wrote:

> Dear All,
> just trying to revive this as I didn?t get any reply.
>

Hi Matteo, thanks for being persistent! The main reason for the lack of
reply is probably that no one remembers the answer - the code is quite old
(pre-2008), and the git history says Pauli is the only currently active dev
who has done significant work on it in the last years.


> I can see that there is more interest in the community with respect to
> ad-hoc matrix-vector multiplications (aka Linear operators) - see also the
> discussion in Re: [SciPy-Dev] Efficient Toeplitz Matrix-Vector
> Multiplication and related PR.
>
> Hopefully some of the core developers or anyone involved in this area can
> comment here ;)
>
> Thank you!
> MR
>
>
>
> On 8 Jan 2020, at 20:31, matteo ravasi <matteoravasi at gmail.com> wrote:
>
> Dear Scipy Developers,
> some of you may be aware of the PyLops project (
> https://github.com/equinor/pylops), which heavily builds on the
> scipy.sparse.linalg.LinearOperator class of scipy and provides an extensive
> library of general purpose and domain specific linear operators.
>
> Recently a users asked for a functionality that would create the
> equivalent dense matrix representation of an operator (to be mostly used
> for teaching purposes). While implementing it, we came across two internal
> behaviours in the interface.py file that we would like to mention and
> discuss about:
>
> - The method _matmat and _rmatmat as implemented per today requires
> operators to work on (N, 1)/(M, 1) arrays. Whilst the first sentence in the
> Notes of LinearOperator states *"The user-defined matvec() function must
> properly handle the case where v has shape (N,) as well as the (N,1) case*",
> I tend to believe that the first option should be the encouraged one, as I
> do not really see any advantage with carrying over trailing dimensions in
> this case. Moreover, some of the linear solvers in scipy that can be used
> together with linear operator such as lsqr or lsmr, require the RHS b
> vector to have a single dimension (b : (m,) ndarray). Again, although a
> quick test showed that they work also when providing a b vector of shape
> (m, 1), this shows to me that the extra trailing dimension is just a
> nuisance and should be discouraged. Assuming that you agree with this
> argument, an alternative solution that works on (N, ) input is actually
> possible and currently implemented in PyLops for _matmat (see here
> https://github.com/equinor/pylops/blob/master/pylops/LinearOperator.py).
> This avoids having implementations that work for both (N, ) and (N, 1),
> which sometimes requires adding additional checks and squeezes within the
> implementation of _matvec and _rmatvec in situations where trailing
> dimensions in the input vector are not really bringing any benefit.
>
> It seems to be, only based on this description, that we can't just break
backwards compatibility here, however it is likely not hard to add a
squeeze call in the right places (or even one central validation function)
and then make everything else assume (N,).

If it would make your life in PyLops easier (not entirely clear to me that
that's why you are asking), then we could look at the impact of deprecating
(N, 1). If no deprecation is needed, a refactoring seems welcome.

See also https://github.com/scipy/scipy/issues/2473, I remembered that we
had an open issue of merging back PyOperators changes. Now there's PyLops,
which may be in a similar position as PyOperators was.

If you would want to start doing some refactoring/maintenance/improvements
in ways that help PyLops (or just make LinearOperator better), that would
be great - we'd love to have someone take a bit of ownership of it.


>
> - The above mentioned new feature has been currently added as a method to
> the PyLops LinearOperator class (and named todense in a similar fashion to
> scipy.sparse.xxx_matrix.todense methods). Doing so, we realized that every
> time two operators are summed/subtracted/multiplied? (pretty much when one
> of the overloading of scipy.sparse.linalg.LinearOperator is invoked), the
> returned object is not of LinearOperator type anymore but of the type of
> the private class in scipy.sparse.linalg.interface used to perform the
> requested operator (e.g, _SumLinearOperator). I wonder if returning an
> object of a ?private? type is actually something recommended to do in scipy
> (or even if this happens anywhere else in scipy).
>
> When we did this in scipy.stats, we also exposed the new classes. It's
normally not good practice to return instances of private classes I'd say.


> Again, in PyLops we need to overload the __add__ method (and similar
> methods) and make it into a thin wrappers of the scipy equivalent,
> returning an object of our parent class LinearOperator. This is to ensure
> that the returned object would maintain the additional properties and
> methods of our Linearoperator class, but also to avoid ?leaking? any of the
> private classes to the end users. I wonder if also the scipy API should
> perhaps avoid returning to end users objects of some private classes.
>
> That's really hard to avoid when users or other libraries create
subclasses.

>
> This email is not intended to criticise any of current behaviours in
> scipy.sparse.linalg.LinearOperator, rather to gain an understanding of some
> of the decisions made by the core developers of the
> scipy.sparse.linalg.interface submodule, which may all have their own right
> to be the way they are today :)
>
>
I hope the above helps!

Cheers,
Ralf


> Looking forward to hearing from you.
>
> Best wishes,
> MR
>
>
> _______________________________________________
> SciPy-Dev mailing list
> SciPy-Dev at python.org
> https://mail.python.org/mailman/listinfo/scipy-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scipy-dev/attachments/20200202/bdc2e2a3/attachment-0001.html>

From pav at iki.fi  Sun Feb  2 17:07:22 2020
From: pav at iki.fi (Pauli Virtanen)
Date: Mon, 03 Feb 2020 00:07:22 +0200
Subject: [SciPy-Dev] Implementation choices in
 scipy.sparse.linalg.LinearOperator
In-Reply-To: <CABL7CQhoWzFSRiUR+ATCJmj+GqpAqSawtmVC_gS-PPuWuYirCA@mail.gmail.com>
References: <0464AE5D-EF1D-4B14-A9FA-DCD3497156FA@gmail.com>
 <382E3555-B904-494D-9E7C-9A3339B54987@gmail.com>
 <CABL7CQhoWzFSRiUR+ATCJmj+GqpAqSawtmVC_gS-PPuWuYirCA@mail.gmail.com>
Message-ID: <49a465e3b4c7654589cc7f8d29a2b7514721f052.camel@iki.fi>

su, 2020-02-02 kello 22:45 +0100, Ralf Gommers kirjoitti:
> It seems to be, only based on this description, that we can't just
> break backwards compatibility here, however it is likely not hard to
> add a squeeze call in the right places (or even one central
> validation function) and then make everything else assume (N,). 
> 
> If it would make your life in PyLops easier (not entirely clear to me
> that that's why you are asking), then we could look at the impact of
> deprecating (N, 1). If no deprecation is needed, a refactoring seems
> welcome.

This issue was discussed recently in one of the related tickets. The
conclusion was that interface changes to coercing internally to (N,) or
(N,1) are not fully backward-compatible, because the distinction of the
two "vector types" is visible to the user-provided matmat/vec
implementations which may be written to handle only one of the cases
(even though they are supposed to support both).

It's trivial to change, but the question of what amount of user code we
want to break is less trivial.

-- 




From tyler.je.reddy at gmail.com  Mon Feb  3 12:59:03 2020
From: tyler.je.reddy at gmail.com (Tyler Reddy)
Date: Mon, 3 Feb 2020 10:59:03 -0700
Subject: [SciPy-Dev] SciPy 1.0 Paper is online in Nature Methods
Message-ID: <CAHPuU_azqSDXYg8ZjKuwsytxDr9NjcF3n0+0_Koz7Qz2yC1McQ@mail.gmail.com>

Hi all,

The SciPy 1.0 journal article is now published online in Nature Methods:
https://www.nature.com/articles/s41592-019-0686-2

Well done team!

Best wishes,
Tyler
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scipy-dev/attachments/20200203/2b652091/attachment.html>

From mhaberla at calpoly.edu  Mon Feb  3 13:13:33 2020
From: mhaberla at calpoly.edu (Matt Haberland)
Date: Mon, 3 Feb 2020 10:13:33 -0800
Subject: [SciPy-Dev] SciPy 1.0 Paper is online in Nature Methods
In-Reply-To: <CAHPuU_azqSDXYg8ZjKuwsytxDr9NjcF3n0+0_Koz7Qz2yC1McQ@mail.gmail.com>
References: <CAHPuU_azqSDXYg8ZjKuwsytxDr9NjcF3n0+0_Koz7Qz2yC1McQ@mail.gmail.com>
Message-ID: <CADuxUizZHTzMrgURCsuz+Q4OgccGG25u=to_bqE78G-H42xHiw@mail.gmail.com>

Thanks to everyone who contributed and especially Tyler for the final push!


On Mon, Feb 3, 2020 at 9:59 AM Tyler Reddy <tyler.je.reddy at gmail.com> wrote:

> Hi all,
>
> The SciPy 1.0 journal article is now published online in Nature Methods:
> https://www.nature.com/articles/s41592-019-0686-2
>
> Well done team!
>
> Best wishes,
> Tyler
> _______________________________________________
> SciPy-Dev mailing list
> SciPy-Dev at python.org
> https://mail.python.org/mailman/listinfo/scipy-dev
>


-- 
Matt Haberland
Assistant Professor
BioResource and Agricultural Engineering
08A-3K, Cal Poly
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scipy-dev/attachments/20200203/a1e6e428/attachment.html>

From ralf.gommers at gmail.com  Mon Feb  3 14:58:24 2020
From: ralf.gommers at gmail.com (Ralf Gommers)
Date: Mon, 3 Feb 2020 20:58:24 +0100
Subject: [SciPy-Dev] SciPy 1.0 Paper is online in Nature Methods
In-Reply-To: <CADuxUizZHTzMrgURCsuz+Q4OgccGG25u=to_bqE78G-H42xHiw@mail.gmail.com>
References: <CAHPuU_azqSDXYg8ZjKuwsytxDr9NjcF3n0+0_Koz7Qz2yC1McQ@mail.gmail.com>
 <CADuxUizZHTzMrgURCsuz+Q4OgccGG25u=to_bqE78G-H42xHiw@mail.gmail.com>
Message-ID: <CABL7CQiP2e1H00trU8gEDxp1wTT3GFKmM3YfDj4S83bpP_PO0g@mail.gmail.com>

On Mon, Feb 3, 2020 at 7:14 PM Matt Haberland <mhaberla at calpoly.edu> wrote:

> Thanks to everyone who contributed and especially Tyler for the final push!
>
>
> On Mon, Feb 3, 2020 at 9:59 AM Tyler Reddy <tyler.je.reddy at gmail.com>
> wrote:
>
>> Hi all,
>>
>> The SciPy 1.0 journal article is now published online in Nature Methods:
>> https://www.nature.com/articles/s41592-019-0686-2
>>
>
Awesome!


>> Well done team!
>>
>
Thanks indeed to everyone who contributed to the paper and to SciPy in the
past. Most importantly I want to thank Tyler and Matt, they together did a
lot of writing and reviewing as well as dealing with author signups,
ticking a million of Nature Methods' boxes, and whatever else was needed to
get this over the line.

Cheers,
Ralf



>
>> Best wishes,
>> Tyler
>> _______________________________________________
>> SciPy-Dev mailing list
>> SciPy-Dev at python.org
>> https://mail.python.org/mailman/listinfo/scipy-dev
>>
>
>
> --
> Matt Haberland
> Assistant Professor
> BioResource and Agricultural Engineering
> 08A-3K, Cal Poly
> _______________________________________________
> SciPy-Dev mailing list
> SciPy-Dev at python.org
> https://mail.python.org/mailman/listinfo/scipy-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scipy-dev/attachments/20200203/d301ea24/attachment.html>

From andyfaff at gmail.com  Mon Feb  3 15:29:14 2020
From: andyfaff at gmail.com (Andrew Nelson)
Date: Tue, 4 Feb 2020 07:29:14 +1100
Subject: [SciPy-Dev] SciPy 1.0 Paper is online in Nature Methods
In-Reply-To: <CABL7CQiP2e1H00trU8gEDxp1wTT3GFKmM3YfDj4S83bpP_PO0g@mail.gmail.com>
References: <CAHPuU_azqSDXYg8ZjKuwsytxDr9NjcF3n0+0_Koz7Qz2yC1McQ@mail.gmail.com>
 <CADuxUizZHTzMrgURCsuz+Q4OgccGG25u=to_bqE78G-H42xHiw@mail.gmail.com>
 <CABL7CQiP2e1H00trU8gEDxp1wTT3GFKmM3YfDj4S83bpP_PO0g@mail.gmail.com>
Message-ID: <CAAbtOZc0MJ0A+fYt0++3VQEn=Gatmm2sSNd9X-VqHk71sVrb3A@mail.gmail.com>

>
> Thanks indeed to everyone who contributed to the paper and to SciPy in the
> past. Most importantly I want to thank Tyler and Matt, they together did a
> lot of writing and reviewing as well as dealing with author signups,
> ticking a million of Nature Methods' boxes, and whatever else was needed to
> get this over the line.
>

Hear, hear.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scipy-dev/attachments/20200204/46804f43/attachment-0001.html>

From mikofski at berkeley.edu  Mon Feb  3 15:33:11 2020
From: mikofski at berkeley.edu (Dr. Mark Alexander Mikofski PhD)
Date: Mon, 3 Feb 2020 12:33:11 -0800
Subject: [SciPy-Dev] SciPy 1.0 Paper is online in Nature Methods
In-Reply-To: <CAAbtOZc0MJ0A+fYt0++3VQEn=Gatmm2sSNd9X-VqHk71sVrb3A@mail.gmail.com>
References: <CAHPuU_azqSDXYg8ZjKuwsytxDr9NjcF3n0+0_Koz7Qz2yC1McQ@mail.gmail.com>
 <CADuxUizZHTzMrgURCsuz+Q4OgccGG25u=to_bqE78G-H42xHiw@mail.gmail.com>
 <CABL7CQiP2e1H00trU8gEDxp1wTT3GFKmM3YfDj4S83bpP_PO0g@mail.gmail.com>
 <CAAbtOZc0MJ0A+fYt0++3VQEn=Gatmm2sSNd9X-VqHk71sVrb3A@mail.gmail.com>
Message-ID: <CAEqRcW0Ns+4S-nJFEdCoWQ3-zKvpjnUC3419NuvGjwgJ561tyA@mail.gmail.com>

Congratulations SciPy! Thanks to all those who worked hard for this!

On Mon, Feb 3, 2020, 12:30 PM Andrew Nelson <andyfaff at gmail.com> wrote:

> Thanks indeed to everyone who contributed to the paper and to SciPy in the
>> past. Most importantly I want to thank Tyler and Matt, they together did a
>> lot of writing and reviewing as well as dealing with author signups,
>> ticking a million of Nature Methods' boxes, and whatever else was needed to
>> get this over the line.
>>
>
> Hear, hear.
> _______________________________________________
> SciPy-Dev mailing list
> SciPy-Dev at python.org
> https://mail.python.org/mailman/listinfo/scipy-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scipy-dev/attachments/20200203/3e63da70/attachment.html>

From charlesr.harris at gmail.com  Mon Feb  3 16:32:30 2020
From: charlesr.harris at gmail.com (Charles R Harris)
Date: Mon, 3 Feb 2020 14:32:30 -0700
Subject: [SciPy-Dev] SciPy 1.0 Paper is online in Nature Methods
In-Reply-To: <CAHPuU_azqSDXYg8ZjKuwsytxDr9NjcF3n0+0_Koz7Qz2yC1McQ@mail.gmail.com>
References: <CAHPuU_azqSDXYg8ZjKuwsytxDr9NjcF3n0+0_Koz7Qz2yC1McQ@mail.gmail.com>
Message-ID: <CAB6mnxL-32O41FMqovjC0Diy2tWPPKbyv16qFkrWdbqrJgnt6Q@mail.gmail.com>

On Mon, Feb 3, 2020 at 10:59 AM Tyler Reddy <tyler.je.reddy at gmail.com>
wrote:

> Hi all,
>
> The SciPy 1.0 journal article is now published online in Nature Methods:
> https://www.nature.com/articles/s41592-019-0686-2
>
> Well done team!
>
>
Congratulations to all. It wasn't easy.

Chuck
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scipy-dev/attachments/20200203/34b41972/attachment.html>

From matteoravasi at gmail.com  Tue Feb  4 15:02:07 2020
From: matteoravasi at gmail.com (Matteo Ravasi)
Date: Tue, 4 Feb 2020 21:02:07 +0100
Subject: [SciPy-Dev] Implementation choices in
 scipy.sparse.linalg.LinearOperator
In-Reply-To: <49a465e3b4c7654589cc7f8d29a2b7514721f052.camel@iki.fi>
References: <0464AE5D-EF1D-4B14-A9FA-DCD3497156FA@gmail.com>
 <382E3555-B904-494D-9E7C-9A3339B54987@gmail.com>
 <CABL7CQhoWzFSRiUR+ATCJmj+GqpAqSawtmVC_gS-PPuWuYirCA@mail.gmail.com>
 <49a465e3b4c7654589cc7f8d29a2b7514721f052.camel@iki.fi>
Message-ID: <0E6FFEFF-3E53-4B19-92B1-8C9751CF8B79@gmail.com>

Dear Ralf, Pauli,
thanks a lot for your replies and for pointing to some interesting discussion around the LinearOperator class. 

I was unfortunately not aware of PyOperators and it?s a shame it seems like it is unmaintained... anyhow, your answers were very useful and I agree that some changes while nice may have back-ward implications for current codes that we don?t want to break. 

As the requirement for supporting both (N,) and (N,1) is there we shall keep it, but perhaps we could add a flag squeeze=False (or something along those lines) in matmat and rmatmat so to give users the chance to use their (N,) instead of (N,1) implementation of a linear operator... but I can give it a deeper thought and see if I come up with any ideas how to make some improvements in this direction and see if I can help making any improvement with the interface.py file :)

Thanks again!
iMR

> On 2 Feb 2020, at 23:07, Pauli Virtanen <pav at iki.fi> wrote:
> 
> su, 2020-02-02 kello 22:45 +0100, Ralf Gommers kirjoitti:
>> It seems to be, only based on this description, that we can't just
>> break backwards compatibility here, however it is likely not hard to
>> add a squeeze call in the right places (or even one central
>> validation function) and then make everything else assume (N,). 
>> 
>> If it would make your life in PyLops easier (not entirely clear to me
>> that that's why you are asking), then we could look at the impact of
>> deprecating (N, 1). If no deprecation is needed, a refactoring seems
>> welcome.
> 
> This issue was discussed recently in one of the related tickets. The
> conclusion was that interface changes to coercing internally to (N,) or
> (N,1) are not fully backward-compatible, because the distinction of the
> two "vector types" is visible to the user-provided matmat/vec
> implementations which may be written to handle only one of the cases
> (even though they are supposed to support both).
> 
> It's trivial to change, but the question of what amount of user code we
> want to break is less trivial.
> 
> -- 
> 
> 
> 
> _______________________________________________
> SciPy-Dev mailing list
> SciPy-Dev at python.org
> https://mail.python.org/mailman/listinfo/scipy-dev

From rlucas7 at vt.edu  Fri Feb  7 11:15:24 2020
From: rlucas7 at vt.edu (Lucas Roberts)
Date: Fri, 7 Feb 2020 11:15:24 -0500
Subject: [SciPy-Dev] SciPy-Dev Digest, Vol 196, Issue 1
In-Reply-To: <mailman.7.1580662803.25193.scipy-dev@python.org>
References: <mailman.7.1580662803.25193.scipy-dev@python.org>
Message-ID: <CADCr_xMPcKrxGYSXCJh48yB2YP3iyw+z4FZO3gzyY-CSmDN+ug@mail.gmail.com>

On Sun, Feb 2, 2020 at 12:00 PM <scipy-dev-request at python.org> wrote:

> Send SciPy-Dev mailing list submissions to
>         scipy-dev at python.org
>
> To subscribe or unsubscribe via the World Wide Web, visit
>         https://mail.python.org/mailman/listinfo/scipy-dev
> or, via email, send a message with subject or body 'help' to
>         scipy-dev-request at python.org
>
> You can reach the person managing the list at
>         scipy-dev-owner at python.org
>
> When replying, please edit your Subject line so it is more specific
> than "Re: Contents of SciPy-Dev digest..."
>
>
> Today's Topics:
>
>    1. language labels in github issues/PRs? (Lucas Roberts)
>    2. Re: language labels in github issues/PRs? (Ralf Gommers)
>    3. Re: language labels in github issues/PRs? (Kai Striega)
>
>
> ----------------------------------------------------------------------
>
> Message: 1
> Date: Sat, 1 Feb 2020 11:56:56 -0500
> From: Lucas Roberts <rlucas7 at vt.edu>
> To: scipy-dev at python.org
> Subject: [SciPy-Dev] language labels in github issues/PRs?
> Message-ID:
>         <
> CADCr_xOm9iv4xKKBBAtnTCbt57bYmaG6c4QS9x5Y5oGaQwcNEQ at mail.gmail.com>
> Content-Type: text/plain; charset="utf-8"
>
> Hi scipy-dev,
>
> I was looking at my stack of emails for scipy and noticed some comments on:
> https://github.com/scipy/scipy/issues/3203#
> and that the c code is involved. In the past I've marked these types of
> emails for myself to flag for follow up (e.g. I'd like to work on them if
> still open in the future). Problem is that the stack of these flags gets
> too large in my inbox. I noticed there aren't tags for language in the
> scipy github. Scipy has C, C++, Cython, Fortran, and Python (I think that
> is all) used in various places.
>
> My question is would it be helpful to have language tags?
>
> We probably wouldnt' want a python tag but if an issue or a PR touches
> C/C++/Cython/fortran adding a label would help discoverability and
> exclusion.
>
> This might help searching of new contributors and for core team, e.g. you
> could search for issues touching c code (in my case) or exclude issues
> touching fortran (also my case, I don't know fortran).
>
> I didn't know how to exclude it Github and learned that it has some
> searching functionality that allows to exclude issues/PRs with certain
> tags:
>
>
> https://help.github.com/en/github/searching-for-information-on-github/searching-issues-and-pull-requests#search-by-label
>
> --
> Sincerely,
> -Lucas
> -------------- next part --------------
> An HTML attachment was scrubbed...
> URL: <
> http://mail.python.org/pipermail/scipy-dev/attachments/20200201/f1481871/attachment-0001.html
> >
>
> ------------------------------
>
> Message: 2
> Date: Sat, 1 Feb 2020 19:21:07 +0100
> From: Ralf Gommers <ralf.gommers at gmail.com>
> To: SciPy Developers List <scipy-dev at python.org>
> Subject: Re: [SciPy-Dev] language labels in github issues/PRs?
> Message-ID:
>         <CABL7CQg2-ETsQSFKGmS9eC02oY47PN8F+pEUjYy9VWB0=
> yFjwA at mail.gmail.com>
> Content-Type: text/plain; charset="utf-8"
>
> On Sat, Feb 1, 2020 at 6:59 PM Lucas Roberts <rlucas7 at vt.edu> wrote:
>
> > Hi scipy-dev,
> >
> > I was looking at my stack of emails for scipy and noticed some comments
> on:
> > https://github.com/scipy/scipy/issues/3203#
> > and that the c code is involved. In the past I've marked these types of
> > emails for myself to flag for follow up (e.g. I'd like to work on them if
> > still open in the future). Problem is that the stack of these flags gets
> > too large in my inbox. I noticed there aren't tags for language in the
> > scipy github. Scipy has C, C++, Cython, Fortran, and Python (I think that
> > is all) used in various places.
> >
> > My question is would it be helpful to have language tags?
> >
>
> Sounds like a good idea to me, +1 for "language: Fortran" and "language
> Cython" tags. Not sure if C and C++ should be separate or together.
>
> Hmm, I haven't looked at much of the C++ code so no firm opinion on that
one from me.
They are different languages but if we aren't using a bulk of the
differences (e.g. virtual functions, templated types, namespaces, etc)
then there really isn't much of a need to distinguish between the two with
tags. Plus, I'd guess that most folks who know C are also comfortable w/C++
to some degree.
If we find a C/C++ tag isn't specific enough then we could always break
them out at a later time, so let's say 3 tags:
Fortran, C/C++, and a Cython tag.


> Cheers,
> Ralf
>
>
> > We probably wouldnt' want a python tag but if an issue or a PR touches
> > C/C++/Cython/fortran adding a label would help discoverability and
> > exclusion.
> >
> > This might help searching of new contributors and for core team, e.g. you
> > could search for issues touching c code (in my case) or exclude issues
> > touching fortran (also my case, I don't know fortran).
> >
> > I didn't know how to exclude it Github and learned that it has some
> > searching functionality that allows to exclude issues/PRs with certain
> tags:
> >
> >
> >
> https://help.github.com/en/github/searching-for-information-on-github/searching-issues-and-pull-requests#search-by-label
> >
> > --
> > Sincerely,
> > -Lucas
> > _______________________________________________
> > SciPy-Dev mailing list
> > SciPy-Dev at python.org
> > https://mail.python.org/mailman/listinfo/scipy-dev
> >
> -------------- next part --------------
> An HTML attachment was scrubbed...
> URL: <
> http://mail.python.org/pipermail/scipy-dev/attachments/20200201/3b1b28c1/attachment-0001.html
> >
>
> ------------------------------
>
> Message: 3
> Date: Sun, 2 Feb 2020 15:37:25 +0800
> From: Kai Striega <kaistriega at gmail.com>
> To: SciPy Developers List <scipy-dev at python.org>
> Subject: Re: [SciPy-Dev] language labels in github issues/PRs?
> Message-ID:
>         <
> CAPDXd17F4HGpJeS9S--+UZrnYiqW5rHbjzL_Pme3JnNKxLEdWg at mail.gmail.com>
> Content-Type: text/plain; charset="utf-8"
>
> Sounds like a good idea to me. I would also like to second not having a
> "python" specific label.
>
> Agreed.

On Sun., 2 Feb. 2020, 02:21 Ralf Gommers, <ralf.gommers at gmail.com> wrote:
>
> >
> >
> > On Sat, Feb 1, 2020 at 6:59 PM Lucas Roberts <rlucas7 at vt.edu> wrote:
> >
> >> Hi scipy-dev,
> >>
> >> I was looking at my stack of emails for scipy and noticed some comments
> >> on:
> >> https://github.com/scipy/scipy/issues/3203#
> >> and that the c code is involved. In the past I've marked these types of
> >> emails for myself to flag for follow up (e.g. I'd like to work on them
> if
> >> still open in the future). Problem is that the stack of these flags gets
> >> too large in my inbox. I noticed there aren't tags for language in the
> >> scipy github. Scipy has C, C++, Cython, Fortran, and Python (I think
> that
> >> is all) used in various places.
> >>
> >> My question is would it be helpful to have language tags?
> >>
> >
> > Sounds like a good idea to me, +1 for "language: Fortran" and "language
> > Cython" tags. Not sure if C and C++ should be separate or together.
> >
> > Cheers,
> > Ralf
> >
> >
> >> We probably wouldnt' want a python tag but if an issue or a PR touches
> >> C/C++/Cython/fortran adding a label would help discoverability and
> >> exclusion.
> >>
> >> This might help searching of new contributors and for core team, e.g.
> you
> >> could search for issues touching c code (in my case) or exclude issues
> >> touching fortran (also my case, I don't know fortran).
> >>
> >> I didn't know how to exclude it Github and learned that it has some
> >> searching functionality that allows to exclude issues/PRs with certain
> tags:
> >>
> >>
> >>
> https://help.github.com/en/github/searching-for-information-on-github/searching-issues-and-pull-requests#search-by-label
> >>
> >> --
> >> Sincerely,
> >> -Lucas
> >> _______________________________________________
> >> SciPy-Dev mailing list
> >> SciPy-Dev at python.org
> >> https://mail.python.org/mailman/listinfo/scipy-dev
> >>
> > _______________________________________________
> > SciPy-Dev mailing list
> > SciPy-Dev at python.org
> > https://mail.python.org/mailman/listinfo/scipy-dev
> >
> -------------- next part --------------
> An HTML attachment was scrubbed...
> URL: <
> http://mail.python.org/pipermail/scipy-dev/attachments/20200202/40914fe5/attachment-0001.html
> >
>
> ------------------------------
>
> Subject: Digest Footer
>
> _______________________________________________
> SciPy-Dev mailing list
> SciPy-Dev at python.org
> https://mail.python.org/mailman/listinfo/scipy-dev
>
>
> ------------------------------
>
> End of SciPy-Dev Digest, Vol 196, Issue 1
> *****************************************
>


-- 
Sincerely,
-Lucas
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scipy-dev/attachments/20200207/0525489e/attachment.html>

From kevinsmith at ucsb.edu  Fri Feb  7 15:04:19 2020
From: kevinsmith at ucsb.edu (Kevin Smith)
Date: Fri, 7 Feb 2020 12:04:19 -0800
Subject: [SciPy-Dev] Extending scipy.stats.kstat
Message-ID: <CAF5JWrNNbwTrYNtE9utK_CUAi+A4La82FzCaeKqU8VM2ai2UuQ@mail.gmail.com>

Hello,

I am writing a function for multivariate k-statistics (minimum-variance
unbiased estimators of multivariate cumulants), and am wondering if there
would be any interest in adding this to code to scipy.stats. The current
function, scipy.stats.kstat, is limited to univariate k-stats up to order
4, whereas my implementation extends to arbitrary order.

Thanks,
Kevin
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scipy-dev/attachments/20200207/eb1b268e/attachment-0001.html>

From josef.pktd at gmail.com  Fri Feb  7 15:13:54 2020
From: josef.pktd at gmail.com (josef.pktd at gmail.com)
Date: Fri, 7 Feb 2020 15:13:54 -0500
Subject: [SciPy-Dev] Extending scipy.stats.kstat
In-Reply-To: <CAF5JWrNNbwTrYNtE9utK_CUAi+A4La82FzCaeKqU8VM2ai2UuQ@mail.gmail.com>
References: <CAF5JWrNNbwTrYNtE9utK_CUAi+A4La82FzCaeKqU8VM2ai2UuQ@mail.gmail.com>
Message-ID: <CAMMTP+C2PYxdQgoF-QdKsjxqx7JoYxAvc-jceke+4S+kKG6RCg@mail.gmail.com>

On Fri, Feb 7, 2020 at 3:04 PM Kevin Smith <kevinsmith at ucsb.edu> wrote:

> Hello,
>
> I am writing a function for multivariate k-statistics (minimum-variance
> unbiased estimators of multivariate cumulants), and am wondering if there
> would be any interest in adding this to code to scipy.stats. The current
> function, scipy.stats.kstat, is limited to univariate k-stats up to order
> 4, whereas my implementation extends to arbitrary order.
>

What are they used for?

AFAIR, I never found a use for kstat/cumulants.
But maybe I just have gaps in memory.
Once upon a time, I wrote functions to convert between moments and cumulants
https://www.statsmodels.org/stable/stats.html#moment-helpers

If there is a demand for it, then it would fit well in scipy.stats.

Josef



>
> Thanks,
> Kevin
> _______________________________________________
> SciPy-Dev mailing list
> SciPy-Dev at python.org
> https://mail.python.org/mailman/listinfo/scipy-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scipy-dev/attachments/20200207/bf8bf38c/attachment.html>

From kevinsmith at ucsb.edu  Fri Feb  7 15:41:06 2020
From: kevinsmith at ucsb.edu (Kevin Smith)
Date: Fri, 7 Feb 2020 12:41:06 -0800
Subject: [SciPy-Dev] Extending scipy.stats.kstat
In-Reply-To: <CAMMTP+C2PYxdQgoF-QdKsjxqx7JoYxAvc-jceke+4S+kKG6RCg@mail.gmail.com>
References: <CAF5JWrNNbwTrYNtE9utK_CUAi+A4La82FzCaeKqU8VM2ai2UuQ@mail.gmail.com>
 <CAMMTP+C2PYxdQgoF-QdKsjxqx7JoYxAvc-jceke+4S+kKG6RCg@mail.gmail.com>
Message-ID: <CAF5JWrPFjDXBdikrDvFuq3crd+kHxCKzCK20aj1rhym3+wm7bg@mail.gmail.com>

Cumulants in general have been useful in signal processing (the first
paragraph of https://epubs.siam.org/doi/pdf/10.1137/17M1149365 has some
references). I am currently using them for a system identification
problem---though statistics is not my field, so perhaps their use is more
niche than I realize. Thanks for linking your module; I may find that
useful.

Kevin

On Fri, Feb 7, 2020 at 12:13 PM <josef.pktd at gmail.com> wrote:

>
>
> On Fri, Feb 7, 2020 at 3:04 PM Kevin Smith <kevinsmith at ucsb.edu> wrote:
>
>> Hello,
>>
>> I am writing a function for multivariate k-statistics (minimum-variance
>> unbiased estimators of multivariate cumulants), and am wondering if there
>> would be any interest in adding this to code to scipy.stats. The current
>> function, scipy.stats.kstat, is limited to univariate k-stats up to order
>> 4, whereas my implementation extends to arbitrary order.
>>
>
> What are they used for?
>
> AFAIR, I never found a use for kstat/cumulants.
> But maybe I just have gaps in memory.
> Once upon a time, I wrote functions to convert between moments and
> cumulants
> https://www.statsmodels.org/stable/stats.html#moment-helpers
>
> If there is a demand for it, then it would fit well in scipy.stats.
>
> Josef
>
>
>
>>
>> Thanks,
>> Kevin
>> _______________________________________________
>> SciPy-Dev mailing list
>> SciPy-Dev at python.org
>> https://mail.python.org/mailman/listinfo/scipy-dev
>>
> _______________________________________________
> SciPy-Dev mailing list
> SciPy-Dev at python.org
> https://mail.python.org/mailman/listinfo/scipy-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scipy-dev/attachments/20200207/04773afd/attachment.html>

From olivier.grisel at ensta.org  Sat Feb  8 11:30:59 2020
From: olivier.grisel at ensta.org (Olivier Grisel)
Date: Sat, 8 Feb 2020 17:30:59 +0100
Subject: [SciPy-Dev] Discontinued Rackspace Open Compute Discount
Message-ID: <CAFvE7K7KHd=2yPyadJLHL7N5QMNkGECfskinmSLFuVeZFCXqsg@mail.gmail.com>

Hi all,

I received an email end of January that I just discovered today
telling me that the Rackspace Open Compute Discount grant we use to
host http://wheels.scipy.org for nightly builds and release management
of our community has been discontinued on December 31st 2019.

I would like to take down the Rackspace hosted cloudfiles as soon as
possible but we need to find a replacement as this will break all the
CIs that rely on nightly wheels and also all the wheel release
infrastructure for several project in the ecosystem.

Apparently the fees for January and the beginning of February already
sum to $936.51. This is a large cost because there were a bunch of
unused left-over block volumes that I already deleted.

To host wheels for nightly builds and release automation, a possible
alternatives include using the github release pages but they are not
really designed for this kind of workflow.

Alternatively there is a PyPI repo hosting feature in Azure Pipelines
in public preview but I am not sure how it works, if it's free for
open source projects and if we could use it as a replacement for the
Rackspace cloud storage we currently use to host
http://wheels.scipy.org .

https://docs.microsoft.com/en-us/azure/devops/pipelines/artifacts/pypi?view=azure-devops&tabs=yaml

Important: if you have uploaded any important files to one of the
cloud file storage / blob containers on Rackspace, please download a
copy as soon as possible.

But please do not download things you do not really want to keep to
avoid adding any unnecessary bandwidth costs. If you have file write
permission on one of those containers, please delete the files you own
and do not need anymore.

Feel free to join this room to discuss the matter interactively:
https://gitter.im/scikit-learn/nightly-build-hosting

Feel free to reply to this email on the scipy-dev mailing list if you
have suggestions for setting up a free replacement.

-- 
Olivier
http://twitter.com/ogrisel - http://github.com/ogrisel

From rlucente at pipeline.com  Sat Feb  8 12:29:12 2020
From: rlucente at pipeline.com (Robert Lucente)
Date: Sat, 8 Feb 2020 12:29:12 -0500 (GMT-05:00)
Subject: [SciPy-Dev] Discontinued Rackspace Open Compute Discount
Message-ID: <470871448.2376.1581182952898@wamui-napoleon.atl.sa.earthlink.net>

Several CI services are free for open source projects

although there are throughput limits

https://about.gitlab.com 

https://github.com/features/actions 

https://travis-ci.org 

https://www.appveyor.com 

-----Original Message-----
>From: Olivier Grisel <olivier.grisel at ensta.org>
>Sent: Feb 8, 2020 11:30 AM
>To: SciPy Developers List <scipy-dev at python.org>
>Subject: [SciPy-Dev] Discontinued Rackspace Open Compute Discount
>
>Hi all,
>
>I received an email end of January that I just discovered today
>telling me that the Rackspace Open Compute Discount grant we use to
>host http://wheels.scipy.org for nightly builds and release management
>of our community has been discontinued on December 31st 2019.
>
>I would like to take down the Rackspace hosted cloudfiles as soon as
>possible but we need to find a replacement as this will break all the
>CIs that rely on nightly wheels and also all the wheel release
>infrastructure for several project in the ecosystem.
>
>Apparently the fees for January and the beginning of February already
>sum to $936.51. This is a large cost because there were a bunch of
>unused left-over block volumes that I already deleted.
>
>To host wheels for nightly builds and release automation, a possible
>alternatives include using the github release pages but they are not
>really designed for this kind of workflow.
>
>Alternatively there is a PyPI repo hosting feature in Azure Pipelines
>in public preview but I am not sure how it works, if it's free for
>open source projects and if we could use it as a replacement for the
>Rackspace cloud storage we currently use to host
>http://wheels.scipy.org .
>
>https://docs.microsoft.com/en-us/azure/devops/pipelines/artifacts/pypi?view=azure-devops&tabs=yaml
>
>Important: if you have uploaded any important files to one of the
>cloud file storage / blob containers on Rackspace, please download a
>copy as soon as possible.
>
>But please do not download things you do not really want to keep to
>avoid adding any unnecessary bandwidth costs. If you have file write
>permission on one of those containers, please delete the files you own
>and do not need anymore.
>
>Feel free to join this room to discuss the matter interactively:
>https://gitter.im/scikit-learn/nightly-build-hosting
>
>Feel free to reply to this email on the scipy-dev mailing list if you
>have suggestions for setting up a free replacement.
>
>-- 
>Olivier
>http://twitter.com/ogrisel - http://github.com/ogrisel
>_______________________________________________
>SciPy-Dev mailing list
>SciPy-Dev at python.org
>https://mail.python.org/mailman/listinfo/scipy-dev

From olivier.grisel at ensta.org  Sat Feb  8 12:40:18 2020
From: olivier.grisel at ensta.org (Olivier Grisel)
Date: Sat, 8 Feb 2020 18:40:18 +0100
Subject: [SciPy-Dev] Discontinued Rackspace Open Compute Discount
In-Reply-To: <470871448.2376.1581182952898@wamui-napoleon.atl.sa.earthlink.net>
References: <470871448.2376.1581182952898@wamui-napoleon.atl.sa.earthlink.net>
Message-ID: <CAFvE7K43Uz2deJy81r_jxXT-8=qYYcpcjssU3TFrB4AUSuuvnw@mail.gmail.com>

The problem is to find a common PyPI-style host for nightly builds
that we would agree on.

The scikit-learn CI publishes a wheels for the master branch on a
nightly basis and uses the numpy and scipy wheels to tests itself
against their master branch. For this we all use the
http://wheels.scipy.org via the
https://github.com/ogrisel/wheelhouse-uploader tool that uploads
wheels to a shared rackspace file container, delete older dev wheels
and update the index page if necessary.

We do not want to publish nightly wheels to the main PyPI to avoid
bloating it with files that are only useful for continuous integration
and testing. In particular the main PyPI host would not allow for the
automatic deletion of older nightly builds as far as I know.

-- 
Olivier

From olivier.grisel at ensta.org  Sat Feb  8 12:56:28 2020
From: olivier.grisel at ensta.org (Olivier Grisel)
Date: Sat, 8 Feb 2020 18:56:28 +0100
Subject: [SciPy-Dev] Discontinued Rackspace Open Compute Discount
In-Reply-To: <CAFvE7K43Uz2deJy81r_jxXT-8=qYYcpcjssU3TFrB4AUSuuvnw@mail.gmail.com>
References: <470871448.2376.1581182952898@wamui-napoleon.atl.sa.earthlink.net>
 <CAFvE7K43Uz2deJy81r_jxXT-8=qYYcpcjssU3TFrB4AUSuuvnw@mail.gmail.com>
Message-ID: <CAFvE7K4vCb72dE2=-3Wy1bfKCCax7g8JxEP8qWSGi-exm_mn=A@mail.gmail.com>

Here is a tweet by Van Lindberg from the PSF who used to work at Rackspace:

https://twitter.com/VanL/status/1225928953350774785

-- 
Olivier

From andy.terrel at gmail.com  Sat Feb  8 14:51:48 2020
From: andy.terrel at gmail.com (Andy Ray Terrel)
Date: Sat, 8 Feb 2020 13:51:48 -0600
Subject: [SciPy-Dev] Discontinued Rackspace Open Compute Discount
In-Reply-To: <CAFvE7K4vCb72dE2=-3Wy1bfKCCax7g8JxEP8qWSGi-exm_mn=A@mail.gmail.com>
References: <470871448.2376.1581182952898@wamui-napoleon.atl.sa.earthlink.net>
 <CAFvE7K43Uz2deJy81r_jxXT-8=qYYcpcjssU3TFrB4AUSuuvnw@mail.gmail.com>
 <CAFvE7K4vCb72dE2=-3Wy1bfKCCax7g8JxEP8qWSGi-exm_mn=A@mail.gmail.com>
Message-ID: <CA+WonSRKO+XWtffy3YC80wpCZS7Z6cnbSPp=gy4NmuF6ucDn8Q@mail.gmail.com>

FWIW, they gave NumFOCUS 2 more years of discount.

On Sat, Feb 8, 2020 at 11:58 AM Olivier Grisel <olivier.grisel at ensta.org>
wrote:

> Here is a tweet by Van Lindberg from the PSF who used to work at Rackspace:
>
> https://twitter.com/VanL/status/1225928953350774785
>
> --
> Olivier
> _______________________________________________
> SciPy-Dev mailing list
> SciPy-Dev at python.org
> https://mail.python.org/mailman/listinfo/scipy-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scipy-dev/attachments/20200208/4a1d5ee5/attachment.html>

From matthew.brett at gmail.com  Sat Feb  8 15:02:45 2020
From: matthew.brett at gmail.com (Matthew Brett)
Date: Sat, 8 Feb 2020 20:02:45 +0000
Subject: [SciPy-Dev] Discontinued Rackspace Open Compute Discount
In-Reply-To: <CA+WonSRKO+XWtffy3YC80wpCZS7Z6cnbSPp=gy4NmuF6ucDn8Q@mail.gmail.com>
References: <470871448.2376.1581182952898@wamui-napoleon.atl.sa.earthlink.net>
 <CAFvE7K43Uz2deJy81r_jxXT-8=qYYcpcjssU3TFrB4AUSuuvnw@mail.gmail.com>
 <CAFvE7K4vCb72dE2=-3Wy1bfKCCax7g8JxEP8qWSGi-exm_mn=A@mail.gmail.com>
 <CA+WonSRKO+XWtffy3YC80wpCZS7Z6cnbSPp=gy4NmuF6ucDn8Q@mail.gmail.com>
Message-ID: <CAH6Pt5ooyWb+jwqM2OjVVnf8noox0PO7DxY74x_47h9nJowq8g@mail.gmail.com>

Hi,

Can we use the NumFOCUS account for scientific Python wheels?

Cheers,

Matthew

On Sat, Feb 8, 2020 at 7:52 PM Andy Ray Terrel <andy.terrel at gmail.com> wrote:
>
> FWIW, they gave NumFOCUS 2 more years of discount.
>
> On Sat, Feb 8, 2020 at 11:58 AM Olivier Grisel <olivier.grisel at ensta.org> wrote:
>>
>> Here is a tweet by Van Lindberg from the PSF who used to work at Rackspace:
>>
>> https://twitter.com/VanL/status/1225928953350774785
>>
>> --
>> Olivier
>> _______________________________________________
>> SciPy-Dev mailing list
>> SciPy-Dev at python.org
>> https://mail.python.org/mailman/listinfo/scipy-dev
>
> _______________________________________________
> SciPy-Dev mailing list
> SciPy-Dev at python.org
> https://mail.python.org/mailman/listinfo/scipy-dev

From tyler.je.reddy at gmail.com  Sat Feb  8 23:06:23 2020
From: tyler.je.reddy at gmail.com (Tyler Reddy)
Date: Sat, 8 Feb 2020 21:06:23 -0700
Subject: [SciPy-Dev] Discontinued Rackspace Open Compute Discount
In-Reply-To: <CAH6Pt5ooyWb+jwqM2OjVVnf8noox0PO7DxY74x_47h9nJowq8g@mail.gmail.com>
References: <470871448.2376.1581182952898@wamui-napoleon.atl.sa.earthlink.net>
 <CAFvE7K43Uz2deJy81r_jxXT-8=qYYcpcjssU3TFrB4AUSuuvnw@mail.gmail.com>
 <CAFvE7K4vCb72dE2=-3Wy1bfKCCax7g8JxEP8qWSGi-exm_mn=A@mail.gmail.com>
 <CA+WonSRKO+XWtffy3YC80wpCZS7Z6cnbSPp=gy4NmuF6ucDn8Q@mail.gmail.com>
 <CAH6Pt5ooyWb+jwqM2OjVVnf8noox0PO7DxY74x_47h9nJowq8g@mail.gmail.com>
Message-ID: <CAHPuU_YANWETiCWZYuz-uTNFzK4DNquT1N7sLUn9qcN2A5imKA@mail.gmail.com>

Well, this sounds like a mess! I'll try to follow here/gitter convo to see
what exactly you want us to do from SciPy release side.

On Sat, 8 Feb 2020 at 13:03, Matthew Brett <matthew.brett at gmail.com> wrote:

> Hi,
>
> Can we use the NumFOCUS account for scientific Python wheels?
>
> Cheers,
>
> Matthew
>
> On Sat, Feb 8, 2020 at 7:52 PM Andy Ray Terrel <andy.terrel at gmail.com>
> wrote:
> >
> > FWIW, they gave NumFOCUS 2 more years of discount.
> >
> > On Sat, Feb 8, 2020 at 11:58 AM Olivier Grisel <olivier.grisel at ensta.org>
> wrote:
> >>
> >> Here is a tweet by Van Lindberg from the PSF who used to work at
> Rackspace:
> >>
> >> https://twitter.com/VanL/status/1225928953350774785
> >>
> >> --
> >> Olivier
> >> _______________________________________________
> >> SciPy-Dev mailing list
> >> SciPy-Dev at python.org
> >> https://mail.python.org/mailman/listinfo/scipy-dev
> >
> > _______________________________________________
> > SciPy-Dev mailing list
> > SciPy-Dev at python.org
> > https://mail.python.org/mailman/listinfo/scipy-dev
> _______________________________________________
> SciPy-Dev mailing list
> SciPy-Dev at python.org
> https://mail.python.org/mailman/listinfo/scipy-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scipy-dev/attachments/20200208/7eedf94b/attachment-0001.html>

From ilhanpolat at gmail.com  Sun Feb  9 04:40:59 2020
From: ilhanpolat at gmail.com (Ilhan Polat)
Date: Sun, 9 Feb 2020 10:40:59 +0100
Subject: [SciPy-Dev] Discontinued Rackspace Open Compute Discount
In-Reply-To: <CAHPuU_YANWETiCWZYuz-uTNFzK4DNquT1N7sLUn9qcN2A5imKA@mail.gmail.com>
References: <470871448.2376.1581182952898@wamui-napoleon.atl.sa.earthlink.net>
 <CAFvE7K43Uz2deJy81r_jxXT-8=qYYcpcjssU3TFrB4AUSuuvnw@mail.gmail.com>
 <CAFvE7K4vCb72dE2=-3Wy1bfKCCax7g8JxEP8qWSGi-exm_mn=A@mail.gmail.com>
 <CA+WonSRKO+XWtffy3YC80wpCZS7Z6cnbSPp=gy4NmuF6ucDn8Q@mail.gmail.com>
 <CAH6Pt5ooyWb+jwqM2OjVVnf8noox0PO7DxY74x_47h9nJowq8g@mail.gmail.com>
 <CAHPuU_YANWETiCWZYuz-uTNFzK4DNquT1N7sLUn9qcN2A5imKA@mail.gmail.com>
Message-ID: <CAEBuzr8cK23+M2zAYbsZ_5GKpXVm0T3pfyCqR5Mny+5NfnSNmw@mail.gmail.com>

I was also looking for possibilities via GitHub actions. Here are a few
pages helped me to cover some initial distance.

https://help.github.com/en/actions/automating-your-workflow-with-github-actions/using-python-with-github-actions
https://github.com/marketplace/actions/python-wheels-manylinux-build

https://github.com/polm/fugashi/pull/5/files



On Sun, Feb 9, 2020 at 5:07 AM Tyler Reddy <tyler.je.reddy at gmail.com> wrote:

> Well, this sounds like a mess! I'll try to follow here/gitter convo to see
> what exactly you want us to do from SciPy release side.
>
> On Sat, 8 Feb 2020 at 13:03, Matthew Brett <matthew.brett at gmail.com>
> wrote:
>
>> Hi,
>>
>> Can we use the NumFOCUS account for scientific Python wheels?
>>
>> Cheers,
>>
>> Matthew
>>
>> On Sat, Feb 8, 2020 at 7:52 PM Andy Ray Terrel <andy.terrel at gmail.com>
>> wrote:
>> >
>> > FWIW, they gave NumFOCUS 2 more years of discount.
>> >
>> > On Sat, Feb 8, 2020 at 11:58 AM Olivier Grisel <
>> olivier.grisel at ensta.org> wrote:
>> >>
>> >> Here is a tweet by Van Lindberg from the PSF who used to work at
>> Rackspace:
>> >>
>> >> https://twitter.com/VanL/status/1225928953350774785
>> >>
>> >> --
>> >> Olivier
>> >> _______________________________________________
>> >> SciPy-Dev mailing list
>> >> SciPy-Dev at python.org
>> >> https://mail.python.org/mailman/listinfo/scipy-dev
>> >
>> > _______________________________________________
>> > SciPy-Dev mailing list
>> > SciPy-Dev at python.org
>> > https://mail.python.org/mailman/listinfo/scipy-dev
>> _______________________________________________
>> SciPy-Dev mailing list
>> SciPy-Dev at python.org
>> https://mail.python.org/mailman/listinfo/scipy-dev
>>
> _______________________________________________
> SciPy-Dev mailing list
> SciPy-Dev at python.org
> https://mail.python.org/mailman/listinfo/scipy-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scipy-dev/attachments/20200209/056bfd03/attachment.html>

From andyfaff at gmail.com  Sun Feb  9 05:02:15 2020
From: andyfaff at gmail.com (Andrew Nelson)
Date: Sun, 9 Feb 2020 21:02:15 +1100
Subject: [SciPy-Dev] Discontinued Rackspace Open Compute Discount
In-Reply-To: <CAEBuzr8cK23+M2zAYbsZ_5GKpXVm0T3pfyCqR5Mny+5NfnSNmw@mail.gmail.com>
References: <470871448.2376.1581182952898@wamui-napoleon.atl.sa.earthlink.net>
 <CAFvE7K43Uz2deJy81r_jxXT-8=qYYcpcjssU3TFrB4AUSuuvnw@mail.gmail.com>
 <CAFvE7K4vCb72dE2=-3Wy1bfKCCax7g8JxEP8qWSGi-exm_mn=A@mail.gmail.com>
 <CA+WonSRKO+XWtffy3YC80wpCZS7Z6cnbSPp=gy4NmuF6ucDn8Q@mail.gmail.com>
 <CAH6Pt5ooyWb+jwqM2OjVVnf8noox0PO7DxY74x_47h9nJowq8g@mail.gmail.com>
 <CAHPuU_YANWETiCWZYuz-uTNFzK4DNquT1N7sLUn9qcN2A5imKA@mail.gmail.com>
 <CAEBuzr8cK23+M2zAYbsZ_5GKpXVm0T3pfyCqR5Mny+5NfnSNmw@mail.gmail.com>
Message-ID: <CAAbtOZdOvZP_HMi12R-=u1nuXLbX2YghRjDzaYHbrW=cRr+HdQ@mail.gmail.com>

Making wheel artefacts from a CI run is reasonably straightforward. The
bigger issue is finding a wheelhouse to store the builds.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scipy-dev/attachments/20200209/3715a804/attachment.html>

From njs at pobox.com  Sun Feb  9 05:13:59 2020
From: njs at pobox.com (Nathaniel Smith)
Date: Sun, 9 Feb 2020 02:13:59 -0800
Subject: [SciPy-Dev] Discontinued Rackspace Open Compute Discount
In-Reply-To: <CAAbtOZdOvZP_HMi12R-=u1nuXLbX2YghRjDzaYHbrW=cRr+HdQ@mail.gmail.com>
References: <470871448.2376.1581182952898@wamui-napoleon.atl.sa.earthlink.net>
 <CAFvE7K43Uz2deJy81r_jxXT-8=qYYcpcjssU3TFrB4AUSuuvnw@mail.gmail.com>
 <CAFvE7K4vCb72dE2=-3Wy1bfKCCax7g8JxEP8qWSGi-exm_mn=A@mail.gmail.com>
 <CA+WonSRKO+XWtffy3YC80wpCZS7Z6cnbSPp=gy4NmuF6ucDn8Q@mail.gmail.com>
 <CAH6Pt5ooyWb+jwqM2OjVVnf8noox0PO7DxY74x_47h9nJowq8g@mail.gmail.com>
 <CAHPuU_YANWETiCWZYuz-uTNFzK4DNquT1N7sLUn9qcN2A5imKA@mail.gmail.com>
 <CAEBuzr8cK23+M2zAYbsZ_5GKpXVm0T3pfyCqR5Mny+5NfnSNmw@mail.gmail.com>
 <CAAbtOZdOvZP_HMi12R-=u1nuXLbX2YghRjDzaYHbrW=cRr+HdQ@mail.gmail.com>
Message-ID: <CAPJVwB=xntyoB3unk1=o8QCTBO7MFuMdE_BRWhqiqeyuc9Gp5g@mail.gmail.com>

On Sun, Feb 9, 2020 at 2:02 AM Andrew Nelson <andyfaff at gmail.com> wrote:
>
> Making wheel artefacts from a CI run is reasonably straightforward. The bigger issue is finding a wheelhouse to store the builds.

Yeah. And unfortunately Github Actions don't have any usable way to
store artifacts from builds. (There is an artifact system that lets
you pass files between steps within a single run, but there's no
programmatic way to fetch those files afterwards.) And Github Releases
do let you attach arbitrary files, but I don't know of any way to
point pip at a Github Release.

You might ping Ernest Durbin @ the PSF to see if they can help out.
The PSF already has substantial cloud infrastructure and relationships
with the big vendors (e.g. I think PyPI gets free storage in S3), so
they might be able to set up another storage bucket pretty easily.

-n

-- 
Nathaniel J. Smith -- https://vorpus.org

From olivier.grisel at ensta.org  Sun Feb  9 06:19:57 2020
From: olivier.grisel at ensta.org (Olivier Grisel)
Date: Sun, 9 Feb 2020 12:19:57 +0100
Subject: [SciPy-Dev] Discontinued Rackspace Open Compute Discount
In-Reply-To: <CAPJVwB=xntyoB3unk1=o8QCTBO7MFuMdE_BRWhqiqeyuc9Gp5g@mail.gmail.com>
References: <470871448.2376.1581182952898@wamui-napoleon.atl.sa.earthlink.net>
 <CAFvE7K43Uz2deJy81r_jxXT-8=qYYcpcjssU3TFrB4AUSuuvnw@mail.gmail.com>
 <CAFvE7K4vCb72dE2=-3Wy1bfKCCax7g8JxEP8qWSGi-exm_mn=A@mail.gmail.com>
 <CA+WonSRKO+XWtffy3YC80wpCZS7Z6cnbSPp=gy4NmuF6ucDn8Q@mail.gmail.com>
 <CAH6Pt5ooyWb+jwqM2OjVVnf8noox0PO7DxY74x_47h9nJowq8g@mail.gmail.com>
 <CAHPuU_YANWETiCWZYuz-uTNFzK4DNquT1N7sLUn9qcN2A5imKA@mail.gmail.com>
 <CAEBuzr8cK23+M2zAYbsZ_5GKpXVm0T3pfyCqR5Mny+5NfnSNmw@mail.gmail.com>
 <CAAbtOZdOvZP_HMi12R-=u1nuXLbX2YghRjDzaYHbrW=cRr+HdQ@mail.gmail.com>
 <CAPJVwB=xntyoB3unk1=o8QCTBO7MFuMdE_BRWhqiqeyuc9Gp5g@mail.gmail.com>
Message-ID: <CAFvE7K7_UgDnQZR37BKFXOnjRFJZT_p47BDpuTmkqDwBXt1ZTw@mail.gmail.com>

Using Azure Artifacts feed seems to work:

https://github.com/MacPython/scikit-learn-wheels/pull/23#issuecomment-583804796

But apparently it will require project maintainers to create a new
Azure account (it's possible to do so using your existing github
credentials), create public project there and a feed whose content
that can be publicly accessed via pip by anyone. The CI script. It's
possible to configure a retention policy to automatically clean-up old
nightly builds and release artifacts to stay below the limit of 2GB
(which would be more than enough for release automation and nightly
builds publishing).

For scikit-learn we already use Azure Pipelines for most automated
testing (pytest in PR) and we wanted to migrated our
https://github.com/MacPython/scikit-learn-wheels release automation
(based on https://github.com/matthew-brett/multibuild/ and backed by
on travis and appveyor) to use it Azure Pipelines instead. Appveyor is
too slow and Azure Pipelines is much faster for us. So migrating to
Azure Artifacts to also host the nightly build wheelhouse would be
quite natural in our case. The upload credentials is automatically
handled via the TwineAuthenticate at 1 task setup in the Azure Pipelines
yaml configuration.

However for projects that do not want to use Azure Pipelines for
building, I cannot find how to generate a secret token to "twine
upload" their wheels from a different CI to an Azure Artifact feed.

Another alternative would be to use anaconda.org that can also host
wheelhouse. The clean-up is not automatic but can be scripted with a
cron task on the CI. anaconda uploads might be more flexible w.r.t.
credentials for uploading from various CIs.

-- 
Olivier

From olivier.grisel at ensta.org  Sun Feb  9 06:21:30 2020
From: olivier.grisel at ensta.org (Olivier Grisel)
Date: Sun, 9 Feb 2020 12:21:30 +0100
Subject: [SciPy-Dev] Discontinued Rackspace Open Compute Discount
In-Reply-To: <CAFvE7K7_UgDnQZR37BKFXOnjRFJZT_p47BDpuTmkqDwBXt1ZTw@mail.gmail.com>
References: <470871448.2376.1581182952898@wamui-napoleon.atl.sa.earthlink.net>
 <CAFvE7K43Uz2deJy81r_jxXT-8=qYYcpcjssU3TFrB4AUSuuvnw@mail.gmail.com>
 <CAFvE7K4vCb72dE2=-3Wy1bfKCCax7g8JxEP8qWSGi-exm_mn=A@mail.gmail.com>
 <CA+WonSRKO+XWtffy3YC80wpCZS7Z6cnbSPp=gy4NmuF6ucDn8Q@mail.gmail.com>
 <CAH6Pt5ooyWb+jwqM2OjVVnf8noox0PO7DxY74x_47h9nJowq8g@mail.gmail.com>
 <CAHPuU_YANWETiCWZYuz-uTNFzK4DNquT1N7sLUn9qcN2A5imKA@mail.gmail.com>
 <CAEBuzr8cK23+M2zAYbsZ_5GKpXVm0T3pfyCqR5Mny+5NfnSNmw@mail.gmail.com>
 <CAAbtOZdOvZP_HMi12R-=u1nuXLbX2YghRjDzaYHbrW=cRr+HdQ@mail.gmail.com>
 <CAPJVwB=xntyoB3unk1=o8QCTBO7MFuMdE_BRWhqiqeyuc9Gp5g@mail.gmail.com>
 <CAFvE7K7_UgDnQZR37BKFXOnjRFJZT_p47BDpuTmkqDwBXt1ZTw@mail.gmail.com>
Message-ID: <CAFvE7K6gE4RyJDQueHrrbisWJKPL1HYLShbMT5TpncY1LfGSWA@mail.gmail.com>

> And unfortunately Github Actions don't have any usable way to store artifacts from builds.

This is a planned feature but it does not support PyPI-style, pip
compatible wheelhouse just yet:

https://github.com/features/packages

-- 
Olivier

From olivier.grisel at ensta.org  Sun Feb  9 06:29:05 2020
From: olivier.grisel at ensta.org (Olivier Grisel)
Date: Sun, 9 Feb 2020 12:29:05 +0100
Subject: [SciPy-Dev] Discontinued Rackspace Open Compute Discount
In-Reply-To: <CAFvE7K6gE4RyJDQueHrrbisWJKPL1HYLShbMT5TpncY1LfGSWA@mail.gmail.com>
References: <470871448.2376.1581182952898@wamui-napoleon.atl.sa.earthlink.net>
 <CAFvE7K43Uz2deJy81r_jxXT-8=qYYcpcjssU3TFrB4AUSuuvnw@mail.gmail.com>
 <CAFvE7K4vCb72dE2=-3Wy1bfKCCax7g8JxEP8qWSGi-exm_mn=A@mail.gmail.com>
 <CA+WonSRKO+XWtffy3YC80wpCZS7Z6cnbSPp=gy4NmuF6ucDn8Q@mail.gmail.com>
 <CAH6Pt5ooyWb+jwqM2OjVVnf8noox0PO7DxY74x_47h9nJowq8g@mail.gmail.com>
 <CAHPuU_YANWETiCWZYuz-uTNFzK4DNquT1N7sLUn9qcN2A5imKA@mail.gmail.com>
 <CAEBuzr8cK23+M2zAYbsZ_5GKpXVm0T3pfyCqR5Mny+5NfnSNmw@mail.gmail.com>
 <CAAbtOZdOvZP_HMi12R-=u1nuXLbX2YghRjDzaYHbrW=cRr+HdQ@mail.gmail.com>
 <CAPJVwB=xntyoB3unk1=o8QCTBO7MFuMdE_BRWhqiqeyuc9Gp5g@mail.gmail.com>
 <CAFvE7K7_UgDnQZR37BKFXOnjRFJZT_p47BDpuTmkqDwBXt1ZTw@mail.gmail.com>
 <CAFvE7K6gE4RyJDQueHrrbisWJKPL1HYLShbMT5TpncY1LfGSWA@mail.gmail.com>
Message-ID: <CAFvE7K6Z0B-LD05qKwoGQmb+wWo4HEF9zHFy-5cQzbpR5sYPcw@mail.gmail.com>

> You might ping Ernest Durbin @ the PSF to see if they can help out.
The PSF already has substantial cloud infrastructure and relationships
with the big vendors (e.g. I think PyPI gets free storage in S3), so
they might be able to set up another storage bucket pretty easily.

The problem is that we would need to automate the clean-up of old
build artifacts. On Rackspace, the
https://github.com/ogrisel/wheelhouse-uploader tool I developed would
do that automatically (for nightly-builds) but honestly. I used Apache
Libcloud so it might work almost out of the box with AWS S3 blob
storage but I have never tested and honestly  I am not interested in
maintaining that tool if more standard alternatives based on twine can
do handle retention logic automatically without custom development.

Also, managing AWS upload credentials for a bunch of loosely coupled
open source projects will be a mess (it was for the rackspace based
solution, Matthew Brett was a SPOF with only myself as a high latency
backup spare). I would rather have each project be autonomous in their
ability to share and revoke upload credentials among maintainers.

-- 
Olivier

From andyfaff at gmail.com  Sun Feb  9 06:33:37 2020
From: andyfaff at gmail.com (Andrew Nelson)
Date: Sun, 9 Feb 2020 22:33:37 +1100
Subject: [SciPy-Dev] Discontinued Rackspace Open Compute Discount
In-Reply-To: <CAFvE7K6Z0B-LD05qKwoGQmb+wWo4HEF9zHFy-5cQzbpR5sYPcw@mail.gmail.com>
References: <470871448.2376.1581182952898@wamui-napoleon.atl.sa.earthlink.net>
 <CAFvE7K43Uz2deJy81r_jxXT-8=qYYcpcjssU3TFrB4AUSuuvnw@mail.gmail.com>
 <CAFvE7K4vCb72dE2=-3Wy1bfKCCax7g8JxEP8qWSGi-exm_mn=A@mail.gmail.com>
 <CA+WonSRKO+XWtffy3YC80wpCZS7Z6cnbSPp=gy4NmuF6ucDn8Q@mail.gmail.com>
 <CAH6Pt5ooyWb+jwqM2OjVVnf8noox0PO7DxY74x_47h9nJowq8g@mail.gmail.com>
 <CAHPuU_YANWETiCWZYuz-uTNFzK4DNquT1N7sLUn9qcN2A5imKA@mail.gmail.com>
 <CAEBuzr8cK23+M2zAYbsZ_5GKpXVm0T3pfyCqR5Mny+5NfnSNmw@mail.gmail.com>
 <CAAbtOZdOvZP_HMi12R-=u1nuXLbX2YghRjDzaYHbrW=cRr+HdQ@mail.gmail.com>
 <CAPJVwB=xntyoB3unk1=o8QCTBO7MFuMdE_BRWhqiqeyuc9Gp5g@mail.gmail.com>
 <CAFvE7K7_UgDnQZR37BKFXOnjRFJZT_p47BDpuTmkqDwBXt1ZTw@mail.gmail.com>
 <CAFvE7K6gE4RyJDQueHrrbisWJKPL1HYLShbMT5TpncY1LfGSWA@mail.gmail.com>
 <CAFvE7K6Z0B-LD05qKwoGQmb+wWo4HEF9zHFy-5cQzbpR5sYPcw@mail.gmail.com>
Message-ID: <CAAbtOZe-jucfwSFkQ7WTYqOYZy8Qe7+ZEtRbBVptaweCGzi_4Q@mail.gmail.com>

Left field, I wonder if we could use something like test.pypi.org?
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scipy-dev/attachments/20200209/1887017d/attachment.html>

From olivier.grisel at ensta.org  Sun Feb  9 07:11:10 2020
From: olivier.grisel at ensta.org (Olivier Grisel)
Date: Sun, 9 Feb 2020 13:11:10 +0100
Subject: [SciPy-Dev] Discontinued Rackspace Open Compute Discount
In-Reply-To: <CAAbtOZe-jucfwSFkQ7WTYqOYZy8Qe7+ZEtRbBVptaweCGzi_4Q@mail.gmail.com>
References: <470871448.2376.1581182952898@wamui-napoleon.atl.sa.earthlink.net>
 <CAFvE7K43Uz2deJy81r_jxXT-8=qYYcpcjssU3TFrB4AUSuuvnw@mail.gmail.com>
 <CAFvE7K4vCb72dE2=-3Wy1bfKCCax7g8JxEP8qWSGi-exm_mn=A@mail.gmail.com>
 <CA+WonSRKO+XWtffy3YC80wpCZS7Z6cnbSPp=gy4NmuF6ucDn8Q@mail.gmail.com>
 <CAH6Pt5ooyWb+jwqM2OjVVnf8noox0PO7DxY74x_47h9nJowq8g@mail.gmail.com>
 <CAHPuU_YANWETiCWZYuz-uTNFzK4DNquT1N7sLUn9qcN2A5imKA@mail.gmail.com>
 <CAEBuzr8cK23+M2zAYbsZ_5GKpXVm0T3pfyCqR5Mny+5NfnSNmw@mail.gmail.com>
 <CAAbtOZdOvZP_HMi12R-=u1nuXLbX2YghRjDzaYHbrW=cRr+HdQ@mail.gmail.com>
 <CAPJVwB=xntyoB3unk1=o8QCTBO7MFuMdE_BRWhqiqeyuc9Gp5g@mail.gmail.com>
 <CAFvE7K7_UgDnQZR37BKFXOnjRFJZT_p47BDpuTmkqDwBXt1ZTw@mail.gmail.com>
 <CAFvE7K6gE4RyJDQueHrrbisWJKPL1HYLShbMT5TpncY1LfGSWA@mail.gmail.com>
 <CAFvE7K6Z0B-LD05qKwoGQmb+wWo4HEF9zHFy-5cQzbpR5sYPcw@mail.gmail.com>
 <CAAbtOZe-jucfwSFkQ7WTYqOYZy8Qe7+ZEtRbBVptaweCGzi_4Q@mail.gmail.com>
Message-ID: <CAFvE7K423GfMmLBdHwg--VR-NP=Zit1eJ+hX4L_nTbbLM8endA@mail.gmail.com>

> Left field, I wonder if we could use something like test.pypi.org?

Good point. Last time I checked it wasn't possible to programmatically
delete old packages to avoid wasting resources for nightly builds. I
cannot find a deletion action in the API docs:

https://warehouse.readthedocs.io/api-reference/

But maybe test.pypi.org has some specific built-in retention policy
that would work for us? I tried to google a bit and it's not how long
it keeps artifacts around.

Also I do not know if the test instance is meant to be reliable enough
to share nightly-build artifacts between the continuous integration
servers for numpy, scipy and their dependencies. If it's not available
enough it might trigger annoying random CI failures.

-- 
Olivier

From andy.terrel at gmail.com  Sun Feb  9 07:38:23 2020
From: andy.terrel at gmail.com (Andy Ray Terrel)
Date: Sun, 9 Feb 2020 06:38:23 -0600
Subject: [SciPy-Dev] Discontinued Rackspace Open Compute Discount
In-Reply-To: <CAH6Pt5ooyWb+jwqM2OjVVnf8noox0PO7DxY74x_47h9nJowq8g@mail.gmail.com>
References: <470871448.2376.1581182952898@wamui-napoleon.atl.sa.earthlink.net>
 <CAFvE7K43Uz2deJy81r_jxXT-8=qYYcpcjssU3TFrB4AUSuuvnw@mail.gmail.com>
 <CAFvE7K4vCb72dE2=-3Wy1bfKCCax7g8JxEP8qWSGi-exm_mn=A@mail.gmail.com>
 <CA+WonSRKO+XWtffy3YC80wpCZS7Z6cnbSPp=gy4NmuF6ucDn8Q@mail.gmail.com>
 <CAH6Pt5ooyWb+jwqM2OjVVnf8noox0PO7DxY74x_47h9nJowq8g@mail.gmail.com>
Message-ID: <CA+WonSRZ1prkf6MbSoHYBFkOR_5J9Fv4jcW34G456MYJKterxw@mail.gmail.com>

Yes, but I recommend ya?ll only using it for the short term, as Rackspace
sprung this so suddenly. I asked them to extend it so we didn?t owe them
money, but my intention is to set it up so we can move servers off
Rackspace.

Additionally I have a bunch of AWS credits that we are still figuring out.
So if you want to use AWS, we can set up an org and y?all will have full
control.

Finally if you do want to use Azure pipeline, Microsoft offered us some
discounts but most teams I worked with had enough on the free their.

? Andy

On Sat, Feb 8, 2020 at 2:03 PM Matthew Brett <matthew.brett at gmail.com>
wrote:

> Hi,
>
> Can we use the NumFOCUS account for scientific Python wheels?
>
> Cheers,
>
> Matthew
>
> On Sat, Feb 8, 2020 at 7:52 PM Andy Ray Terrel <andy.terrel at gmail.com>
> wrote:
> >
> > FWIW, they gave NumFOCUS 2 more years of discount.
> >
> > On Sat, Feb 8, 2020 at 11:58 AM Olivier Grisel <olivier.grisel at ensta.org>
> wrote:
> >>
> >> Here is a tweet by Van Lindberg from the PSF who used to work at
> Rackspace:
> >>
> >> https://twitter.com/VanL/status/1225928953350774785
> >>
> >> --
> >> Olivier
> >> _______________________________________________
> >> SciPy-Dev mailing list
> >> SciPy-Dev at python.org
> >> https://mail.python.org/mailman/listinfo/scipy-dev
> >
> > _______________________________________________
> > SciPy-Dev mailing list
> > SciPy-Dev at python.org
> > https://mail.python.org/mailman/listinfo/scipy-dev
> _______________________________________________
> SciPy-Dev mailing list
> SciPy-Dev at python.org
> https://mail.python.org/mailman/listinfo/scipy-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scipy-dev/attachments/20200209/39444146/attachment.html>

From olivier.grisel at ensta.org  Sun Feb  9 10:17:48 2020
From: olivier.grisel at ensta.org (Olivier Grisel)
Date: Sun, 9 Feb 2020 16:17:48 +0100
Subject: [SciPy-Dev] Discontinued Rackspace Open Compute Discount
In-Reply-To: <CA+WonSRZ1prkf6MbSoHYBFkOR_5J9Fv4jcW34G456MYJKterxw@mail.gmail.com>
References: <470871448.2376.1581182952898@wamui-napoleon.atl.sa.earthlink.net>
 <CAFvE7K43Uz2deJy81r_jxXT-8=qYYcpcjssU3TFrB4AUSuuvnw@mail.gmail.com>
 <CAFvE7K4vCb72dE2=-3Wy1bfKCCax7g8JxEP8qWSGi-exm_mn=A@mail.gmail.com>
 <CA+WonSRKO+XWtffy3YC80wpCZS7Z6cnbSPp=gy4NmuF6ucDn8Q@mail.gmail.com>
 <CAH6Pt5ooyWb+jwqM2OjVVnf8noox0PO7DxY74x_47h9nJowq8g@mail.gmail.com>
 <CA+WonSRZ1prkf6MbSoHYBFkOR_5J9Fv4jcW34G456MYJKterxw@mail.gmail.com>
Message-ID: <CAFvE7K58Hhc68MUdF90DF+w1ACDw2MCxESmDfCDs3JvdRiB41Q@mail.gmail.com>

I asked the question about using test.pypi.org for nightly builds here:

https://discuss.python.org/t/publishing-nightly-builds-on-https-test-pypi-org-with-a-time-based-retention-policy/3152

Note that the tensorflow project is abusing much more than we would:
they upload 2.2GB worth of binaries every day just for the CPU
version, on the main pypi.org instance (under the tf-nightly project
name).

If there was an automated way to setup a retention policy for dev
wheels on the test.pypi.org warehouse that would be ideal from an
operational standpoint point of view.

-- 
Olivier

From olivier.grisel at ensta.org  Sun Feb  9 11:36:47 2020
From: olivier.grisel at ensta.org (Olivier Grisel)
Date: Sun, 9 Feb 2020 17:36:47 +0100
Subject: [SciPy-Dev] Discontinued Rackspace Open Compute Discount
In-Reply-To: <CAFvE7K58Hhc68MUdF90DF+w1ACDw2MCxESmDfCDs3JvdRiB41Q@mail.gmail.com>
References: <470871448.2376.1581182952898@wamui-napoleon.atl.sa.earthlink.net>
 <CAFvE7K43Uz2deJy81r_jxXT-8=qYYcpcjssU3TFrB4AUSuuvnw@mail.gmail.com>
 <CAFvE7K4vCb72dE2=-3Wy1bfKCCax7g8JxEP8qWSGi-exm_mn=A@mail.gmail.com>
 <CA+WonSRKO+XWtffy3YC80wpCZS7Z6cnbSPp=gy4NmuF6ucDn8Q@mail.gmail.com>
 <CAH6Pt5ooyWb+jwqM2OjVVnf8noox0PO7DxY74x_47h9nJowq8g@mail.gmail.com>
 <CA+WonSRZ1prkf6MbSoHYBFkOR_5J9Fv4jcW34G456MYJKterxw@mail.gmail.com>
 <CAFvE7K58Hhc68MUdF90DF+w1ACDw2MCxESmDfCDs3JvdRiB41Q@mail.gmail.com>
Message-ID: <CAFvE7K5+M5COuuiSHHajNiVJL_omAX71VcUFfk88A+A-KmdikQ@mail.gmail.com>

Paul Moore answered on discord to recommend us to use our own PEP 503
index (instead of test.pypi.org) to publish nightly builds.

I think setting up a new shared organization on https://anaconda.org/
to shared the wheels of nightly builds for cython, numpy, scipy,
pandas, scikit-learn, gensim,... would be the best way to go forward.
It makes it possible to preserve the convenience of the shared nightly
builds currently uploaded to http://wheels.scipy.org .

anaconda.org does not have a built-in retention policy but the
anaconda-client package provides a command line "anaconda remove" to
delete individual files.

So it should be possible to write a CRON job somewhere to clean-up old
nightly builds (which is currently not possible to do on
test.pypi.org).

I have also investigated further if we could do the  same on Azure
Artifacts but it is more cumbersome: to share upload credentials to a
shared feed to several projects' maintainer teams requires generating
"Personal Access Tokens". Each token has a maximum validity limit of 1
year. We would have to reconfigure all the CIs at least once a year
which is a bummer.

For the name of the shared anaconda cloud organization, would
"scipy-nightly" be fine? Or maybe "scipy-wheels-nightly" to be extra
explicit? The intent would be to share this space with the
aforementioned projects of the scipy ecosystem.

For the second use case of a staging wheelhouse for (tagged) release
automation, one could use test.scipy.org as it matches the intended
use case of that service.

-- 
Olivier

From olivier.grisel at ensta.org  Wed Feb 12 12:35:21 2020
From: olivier.grisel at ensta.org (Olivier Grisel)
Date: Wed, 12 Feb 2020 18:35:21 +0100
Subject: [SciPy-Dev] Discontinued Rackspace Open Compute Discount
In-Reply-To: <CAFvE7K5+M5COuuiSHHajNiVJL_omAX71VcUFfk88A+A-KmdikQ@mail.gmail.com>
References: <470871448.2376.1581182952898@wamui-napoleon.atl.sa.earthlink.net>
 <CAFvE7K43Uz2deJy81r_jxXT-8=qYYcpcjssU3TFrB4AUSuuvnw@mail.gmail.com>
 <CAFvE7K4vCb72dE2=-3Wy1bfKCCax7g8JxEP8qWSGi-exm_mn=A@mail.gmail.com>
 <CA+WonSRKO+XWtffy3YC80wpCZS7Z6cnbSPp=gy4NmuF6ucDn8Q@mail.gmail.com>
 <CAH6Pt5ooyWb+jwqM2OjVVnf8noox0PO7DxY74x_47h9nJowq8g@mail.gmail.com>
 <CA+WonSRZ1prkf6MbSoHYBFkOR_5J9Fv4jcW34G456MYJKterxw@mail.gmail.com>
 <CAFvE7K58Hhc68MUdF90DF+w1ACDw2MCxESmDfCDs3JvdRiB41Q@mail.gmail.com>
 <CAFvE7K5+M5COuuiSHHajNiVJL_omAX71VcUFfk88A+A-KmdikQ@mail.gmail.com>
Message-ID: <CAFvE7K7bmoPZcRQrEqt11mE=X0FgqAUYmhQbj0gvcM5QL=i4GQ@mail.gmail.com>

Status update:

- I got a reply from Rackspace and and they agree to continue the discount
through April, 2020. This should give us enough time to move
operations. I also deleted many useless costly resources on that
account so that the running cost for the coming month should not
exceed $100/month even if we go beyond end of April.

- I already upgraded the
https://github.com/MacPython/scikit-learn-wheels infrastructure to
upload nightly builds and temporary release artifacts to anaconda.org,
details below.

- OpenBLAS builds used for numpy and scipy will be pushed to github
releases: https://github.com/MacPython/openblas-libs/issues/14

- We still need to update the MacPython/*-wheels for the rest of the
projects but I can help prepare some pull requests.

Based on past observed CDN traffic on the Rackspace account (around
800 GB/month), it should be fine to use the free anaconda.org package
hosting service to host the nightly builds for all the community
projects that currently rely on http://wheels.scipy.org.

For scikit-learn, I decided to keep the release staging artifacts
separated from the nightly build artifacts:

- https://anaconda.org/scipy-wheels-nightly/ as a shared host for all
nightly builds
- https://anaconda.org/scikit-learn-wheels-staging/ as a temporary
store for scikit-learn wheels prior to final upload to pypi.org when
making a release.

Here is the configuration in scikit-learn CI that does the wheel
upload to anaconda.org:

https://github.com/MacPython/scikit-learn-wheels/blob/c0f96681684657dc94a7c0db1d7a93b7435408f4/azure/posix.yml#L108-L141

## Shared nightly build hosting

The scipy-wheels-nightly organization is meant to make it easier for
numpy, scipy, pandas, scikit-learn and other projects with long build
time to publish binaries for their development branch on a daily basis
so that continuous integration can efficiently run their tests against
the dev branch of their dependencies, so as to spot regressions, ASAP
before making a release.

For instance, once numpy and scipy have been configured to upload
nightly builds there it will be possible to reconfigure the scheduled
tests of a project that depends on scikit-learn to run against the dev
branch of the full stack using:

pip install -i https://pypi.anaconda.org/scipy-wheels-nightly/simple
scikit-learn
pip install --no-build-isolation -e .
pytest

If you are a maintainer of one of the projects that publishes
nightly-builds on http://wheels.scipy.org, **please create an account
on https://anaconda.org and reply to this email with your identifier**
so that I can grant you permissions on this shared organization.

>From there you will be able to generate tokens to use as secret
variable in your CI servers. To use those access tokens in a CI setup
will need to enable permissions for the token:

- Write access to API
- Upload pypi packages

## Staging hosting for release artifacts

Each project is free to use its own staging area. Staging areas are
just useful as a synchronization step in a multi-CI release automation
setup. There is really no need to publish those wheels to end-users.
The end users will grab the wheels of the released versions from the
main https://pypi.org index once the release is done.

That being said, I plan to also create a default staging area for
convenience, e.g.:

- https://anaconda.org/multibuild-wheels-staging/

for projects who do not really care and want to use the
https://github.com/matthew-brett/multibuild scripts with minimal admin
efforts.

Thanks for your attention, let me know if you have any comment on this plan.

-- 
Olivier

From tom.w.augspurger at gmail.com  Wed Feb 12 12:45:22 2020
From: tom.w.augspurger at gmail.com (Tom Augspurger)
Date: Wed, 12 Feb 2020 11:45:22 -0600
Subject: [SciPy-Dev] Discontinued Rackspace Open Compute Discount
In-Reply-To: <CAFvE7K7bmoPZcRQrEqt11mE=X0FgqAUYmhQbj0gvcM5QL=i4GQ@mail.gmail.com>
References: <470871448.2376.1581182952898@wamui-napoleon.atl.sa.earthlink.net>
 <CAFvE7K43Uz2deJy81r_jxXT-8=qYYcpcjssU3TFrB4AUSuuvnw@mail.gmail.com>
 <CAFvE7K4vCb72dE2=-3Wy1bfKCCax7g8JxEP8qWSGi-exm_mn=A@mail.gmail.com>
 <CA+WonSRKO+XWtffy3YC80wpCZS7Z6cnbSPp=gy4NmuF6ucDn8Q@mail.gmail.com>
 <CAH6Pt5ooyWb+jwqM2OjVVnf8noox0PO7DxY74x_47h9nJowq8g@mail.gmail.com>
 <CA+WonSRZ1prkf6MbSoHYBFkOR_5J9Fv4jcW34G456MYJKterxw@mail.gmail.com>
 <CAFvE7K58Hhc68MUdF90DF+w1ACDw2MCxESmDfCDs3JvdRiB41Q@mail.gmail.com>
 <CAFvE7K5+M5COuuiSHHajNiVJL_omAX71VcUFfk88A+A-KmdikQ@mail.gmail.com>
 <CAFvE7K7bmoPZcRQrEqt11mE=X0FgqAUYmhQbj0gvcM5QL=i4GQ@mail.gmail.com>
Message-ID: <CAE1aY-kjnNtsUych1kDMSt2hT5qk7K6e4gOuQnUY-LSy0otbjg@mail.gmail.com>

Thanks for working on this Olivier.

My Anaconda.org account is tomaugspurger.

I'll take care of updating https://github.com/MacPython/pandas-wheels over
the next couple days.

On Wed, Feb 12, 2020 at 11:36 AM Olivier Grisel <olivier.grisel at ensta.org>
wrote:

> Status update:
>
> - I got a reply from Rackspace and and they agree to continue the discount
> through April, 2020. This should give us enough time to move
> operations. I also deleted many useless costly resources on that
> account so that the running cost for the coming month should not
> exceed $100/month even if we go beyond end of April.
>
> - I already upgraded the
> https://github.com/MacPython/scikit-learn-wheels infrastructure to
> upload nightly builds and temporary release artifacts to anaconda.org,
> details below.
>
> - OpenBLAS builds used for numpy and scipy will be pushed to github
> releases: https://github.com/MacPython/openblas-libs/issues/14
>
> - We still need to update the MacPython/*-wheels for the rest of the
> projects but I can help prepare some pull requests.
>
> Based on past observed CDN traffic on the Rackspace account (around
> 800 GB/month), it should be fine to use the free anaconda.org package
> hosting service to host the nightly builds for all the community
> projects that currently rely on http://wheels.scipy.org.
>
> For scikit-learn, I decided to keep the release staging artifacts
> separated from the nightly build artifacts:
>
> - https://anaconda.org/scipy-wheels-nightly/ as a shared host for all
> nightly builds
> - https://anaconda.org/scikit-learn-wheels-staging/ as a temporary
> store for scikit-learn wheels prior to final upload to pypi.org when
> making a release.
>
> Here is the configuration in scikit-learn CI that does the wheel
> upload to anaconda.org:
>
>
> https://github.com/MacPython/scikit-learn-wheels/blob/c0f96681684657dc94a7c0db1d7a93b7435408f4/azure/posix.yml#L108-L141
>
> ## Shared nightly build hosting
>
> The scipy-wheels-nightly organization is meant to make it easier for
> numpy, scipy, pandas, scikit-learn and other projects with long build
> time to publish binaries for their development branch on a daily basis
> so that continuous integration can efficiently run their tests against
> the dev branch of their dependencies, so as to spot regressions, ASAP
> before making a release.
>
> For instance, once numpy and scipy have been configured to upload
> nightly builds there it will be possible to reconfigure the scheduled
> tests of a project that depends on scikit-learn to run against the dev
> branch of the full stack using:
>
> pip install -i https://pypi.anaconda.org/scipy-wheels-nightly/simple
> scikit-learn
> <https://pypi.anaconda.org/scipy-wheels-nightly/simplescikit-learn>
> pip install --no-build-isolation -e .
> pytest
>
> If you are a maintainer of one of the projects that publishes
> nightly-builds on http://wheels.scipy.org, **please create an account
> on https://anaconda.org and reply to this email with your identifier**
> so that I can grant you permissions on this shared organization.
>
> From there you will be able to generate tokens to use as secret
> variable in your CI servers. To use those access tokens in a CI setup
> will need to enable permissions for the token:
>
> - Write access to API
> - Upload pypi packages
>
> ## Staging hosting for release artifacts
>
> Each project is free to use its own staging area. Staging areas are
> just useful as a synchronization step in a multi-CI release automation
> setup. There is really no need to publish those wheels to end-users.
> The end users will grab the wheels of the released versions from the
> main https://pypi.org index once the release is done.
>
> That being said, I plan to also create a default staging area for
> convenience, e.g.:
>
> - https://anaconda.org/multibuild-wheels-staging/
>
> for projects who do not really care and want to use the
> https://github.com/matthew-brett/multibuild scripts with minimal admin
> efforts.
>
> Thanks for your attention, let me know if you have any comment on this
> plan.
>
> --
> Olivier
> _______________________________________________
> SciPy-Dev mailing list
> SciPy-Dev at python.org
> https://mail.python.org/mailman/listinfo/scipy-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scipy-dev/attachments/20200212/0c4f0ffe/attachment.html>

From tyler.je.reddy at gmail.com  Wed Feb 12 13:12:56 2020
From: tyler.je.reddy at gmail.com (Tyler Reddy)
Date: Wed, 12 Feb 2020 11:12:56 -0700
Subject: [SciPy-Dev] Discontinued Rackspace Open Compute Discount
In-Reply-To: <CAE1aY-kjnNtsUych1kDMSt2hT5qk7K6e4gOuQnUY-LSy0otbjg@mail.gmail.com>
References: <470871448.2376.1581182952898@wamui-napoleon.atl.sa.earthlink.net>
 <CAFvE7K43Uz2deJy81r_jxXT-8=qYYcpcjssU3TFrB4AUSuuvnw@mail.gmail.com>
 <CAFvE7K4vCb72dE2=-3Wy1bfKCCax7g8JxEP8qWSGi-exm_mn=A@mail.gmail.com>
 <CA+WonSRKO+XWtffy3YC80wpCZS7Z6cnbSPp=gy4NmuF6ucDn8Q@mail.gmail.com>
 <CAH6Pt5ooyWb+jwqM2OjVVnf8noox0PO7DxY74x_47h9nJowq8g@mail.gmail.com>
 <CA+WonSRZ1prkf6MbSoHYBFkOR_5J9Fv4jcW34G456MYJKterxw@mail.gmail.com>
 <CAFvE7K58Hhc68MUdF90DF+w1ACDw2MCxESmDfCDs3JvdRiB41Q@mail.gmail.com>
 <CAFvE7K5+M5COuuiSHHajNiVJL_omAX71VcUFfk88A+A-KmdikQ@mail.gmail.com>
 <CAFvE7K7bmoPZcRQrEqt11mE=X0FgqAUYmhQbj0gvcM5QL=i4GQ@mail.gmail.com>
 <CAE1aY-kjnNtsUych1kDMSt2hT5qk7K6e4gOuQnUY-LSy0otbjg@mail.gmail.com>
Message-ID: <CAHPuU_ao0QL44pp=Z_iYZnvVPCDCx_vOOtqbeGz0mD839x2B=A@mail.gmail.com>

Hi Olivier,

My Anaconda.org account is treddy



On Wed, 12 Feb 2020 at 10:46, Tom Augspurger <tom.w.augspurger at gmail.com>
wrote:

> Thanks for working on this Olivier.
>
> My Anaconda.org account is tomaugspurger.
>
> I'll take care of updating https://github.com/MacPython/pandas-wheels
> over the next couple days.
>
> On Wed, Feb 12, 2020 at 11:36 AM Olivier Grisel <olivier.grisel at ensta.org>
> wrote:
>
>> Status update:
>>
>> - I got a reply from Rackspace and and they agree to continue the discount
>> through April, 2020. This should give us enough time to move
>> operations. I also deleted many useless costly resources on that
>> account so that the running cost for the coming month should not
>> exceed $100/month even if we go beyond end of April.
>>
>> - I already upgraded the
>> https://github.com/MacPython/scikit-learn-wheels infrastructure to
>> upload nightly builds and temporary release artifacts to anaconda.org,
>> details below.
>>
>> - OpenBLAS builds used for numpy and scipy will be pushed to github
>> releases: https://github.com/MacPython/openblas-libs/issues/14
>>
>> - We still need to update the MacPython/*-wheels for the rest of the
>> projects but I can help prepare some pull requests.
>>
>> Based on past observed CDN traffic on the Rackspace account (around
>> 800 GB/month), it should be fine to use the free anaconda.org package
>> hosting service to host the nightly builds for all the community
>> projects that currently rely on http://wheels.scipy.org.
>>
>> For scikit-learn, I decided to keep the release staging artifacts
>> separated from the nightly build artifacts:
>>
>> - https://anaconda.org/scipy-wheels-nightly/ as a shared host for all
>> nightly builds
>> - https://anaconda.org/scikit-learn-wheels-staging/ as a temporary
>> store for scikit-learn wheels prior to final upload to pypi.org when
>> making a release.
>>
>> Here is the configuration in scikit-learn CI that does the wheel
>> upload to anaconda.org:
>>
>>
>> https://github.com/MacPython/scikit-learn-wheels/blob/c0f96681684657dc94a7c0db1d7a93b7435408f4/azure/posix.yml#L108-L141
>>
>> ## Shared nightly build hosting
>>
>> The scipy-wheels-nightly organization is meant to make it easier for
>> numpy, scipy, pandas, scikit-learn and other projects with long build
>> time to publish binaries for their development branch on a daily basis
>> so that continuous integration can efficiently run their tests against
>> the dev branch of their dependencies, so as to spot regressions, ASAP
>> before making a release.
>>
>> For instance, once numpy and scipy have been configured to upload
>> nightly builds there it will be possible to reconfigure the scheduled
>> tests of a project that depends on scikit-learn to run against the dev
>> branch of the full stack using:
>>
>> pip install -i https://pypi.anaconda.org/scipy-wheels-nightly/simple
>> scikit-learn
>> <https://pypi.anaconda.org/scipy-wheels-nightly/simplescikit-learn>
>> pip install --no-build-isolation -e .
>> pytest
>>
>> If you are a maintainer of one of the projects that publishes
>> nightly-builds on http://wheels.scipy.org, **please create an account
>> on https://anaconda.org and reply to this email with your identifier**
>> so that I can grant you permissions on this shared organization.
>>
>> From there you will be able to generate tokens to use as secret
>> variable in your CI servers. To use those access tokens in a CI setup
>> will need to enable permissions for the token:
>>
>> - Write access to API
>> - Upload pypi packages
>>
>> ## Staging hosting for release artifacts
>>
>> Each project is free to use its own staging area. Staging areas are
>> just useful as a synchronization step in a multi-CI release automation
>> setup. There is really no need to publish those wheels to end-users.
>> The end users will grab the wheels of the released versions from the
>> main https://pypi.org index once the release is done.
>>
>> That being said, I plan to also create a default staging area for
>> convenience, e.g.:
>>
>> - https://anaconda.org/multibuild-wheels-staging/
>>
>> for projects who do not really care and want to use the
>> https://github.com/matthew-brett/multibuild scripts with minimal admin
>> efforts.
>>
>> Thanks for your attention, let me know if you have any comment on this
>> plan.
>>
>> --
>> Olivier
>> _______________________________________________
>> SciPy-Dev mailing list
>> SciPy-Dev at python.org
>> https://mail.python.org/mailman/listinfo/scipy-dev
>>
> _______________________________________________
> SciPy-Dev mailing list
> SciPy-Dev at python.org
> https://mail.python.org/mailman/listinfo/scipy-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scipy-dev/attachments/20200212/8b0bba76/attachment-0001.html>

From grlee77 at gmail.com  Wed Feb 12 13:21:34 2020
From: grlee77 at gmail.com (Gregory Lee)
Date: Wed, 12 Feb 2020 13:21:34 -0500
Subject: [SciPy-Dev] Discontinued Rackspace Open Compute Discount
In-Reply-To: <CAE1aY-kjnNtsUych1kDMSt2hT5qk7K6e4gOuQnUY-LSy0otbjg@mail.gmail.com>
References: <470871448.2376.1581182952898@wamui-napoleon.atl.sa.earthlink.net>
 <CAFvE7K43Uz2deJy81r_jxXT-8=qYYcpcjssU3TFrB4AUSuuvnw@mail.gmail.com>
 <CAFvE7K4vCb72dE2=-3Wy1bfKCCax7g8JxEP8qWSGi-exm_mn=A@mail.gmail.com>
 <CA+WonSRKO+XWtffy3YC80wpCZS7Z6cnbSPp=gy4NmuF6ucDn8Q@mail.gmail.com>
 <CAH6Pt5ooyWb+jwqM2OjVVnf8noox0PO7DxY74x_47h9nJowq8g@mail.gmail.com>
 <CA+WonSRZ1prkf6MbSoHYBFkOR_5J9Fv4jcW34G456MYJKterxw@mail.gmail.com>
 <CAFvE7K58Hhc68MUdF90DF+w1ACDw2MCxESmDfCDs3JvdRiB41Q@mail.gmail.com>
 <CAFvE7K5+M5COuuiSHHajNiVJL_omAX71VcUFfk88A+A-KmdikQ@mail.gmail.com>
 <CAFvE7K7bmoPZcRQrEqt11mE=X0FgqAUYmhQbj0gvcM5QL=i4GQ@mail.gmail.com>
 <CAE1aY-kjnNtsUych1kDMSt2hT5qk7K6e4gOuQnUY-LSy0otbjg@mail.gmail.com>
Message-ID: <CAJR3sXebe8bz+C1YuW5i7eS1bq3DjN9Bj3AByakmuM1FZcLS0w@mail.gmail.com>

Thanks for the very helpful update, Oliver!

I can take care of updating the following two repositories:
https://github.com/MacPython/pywavelets-wheels
<https://github.com/MacPython/pandas-wheels>
https://github.com/pyFFTW/pyFFTW-wheels
<https://github.com/MacPython/pandas-wheels>
Neither of those are doing nightly builds, but both have been using
wheels.scipy.org for release staging.

My anaconda.org account is under: grlee77

switching to https://anaconda.org/multibuild-wheels-staging/ should be fine
for our purposes


On Wed, Feb 12, 2020 at 12:45 PM Tom Augspurger <tom.w.augspurger at gmail.com>
wrote:

> Thanks for working on this Olivier.
>
> My Anaconda.org account is tomaugspurger.
>
> I'll take care of updating https://github.com/MacPython/pandas-wheels
> over the next couple days.
>
> On Wed, Feb 12, 2020 at 11:36 AM Olivier Grisel <olivier.grisel at ensta.org>
> wrote:
>
>> Status update:
>>
>> - I got a reply from Rackspace and and they agree to continue the discount
>> through April, 2020. This should give us enough time to move
>> operations. I also deleted many useless costly resources on that
>> account so that the running cost for the coming month should not
>> exceed $100/month even if we go beyond end of April.
>>
>> - I already upgraded the
>> https://github.com/MacPython/scikit-learn-wheels infrastructure to
>> upload nightly builds and temporary release artifacts to anaconda.org,
>> details below.
>>
>> - OpenBLAS builds used for numpy and scipy will be pushed to github
>> releases: https://github.com/MacPython/openblas-libs/issues/14
>>
>> - We still need to update the MacPython/*-wheels for the rest of the
>> projects but I can help prepare some pull requests.
>>
>> Based on past observed CDN traffic on the Rackspace account (around
>> 800 GB/month), it should be fine to use the free anaconda.org package
>> hosting service to host the nightly builds for all the community
>> projects that currently rely on http://wheels.scipy.org.
>>
>> For scikit-learn, I decided to keep the release staging artifacts
>> separated from the nightly build artifacts:
>>
>> - https://anaconda.org/scipy-wheels-nightly/ as a shared host for all
>> nightly builds
>> - https://anaconda.org/scikit-learn-wheels-staging/ as a temporary
>> store for scikit-learn wheels prior to final upload to pypi.org when
>> making a release.
>>
>> Here is the configuration in scikit-learn CI that does the wheel
>> upload to anaconda.org:
>>
>>
>> https://github.com/MacPython/scikit-learn-wheels/blob/c0f96681684657dc94a7c0db1d7a93b7435408f4/azure/posix.yml#L108-L141
>>
>> ## Shared nightly build hosting
>>
>> The scipy-wheels-nightly organization is meant to make it easier for
>> numpy, scipy, pandas, scikit-learn and other projects with long build
>> time to publish binaries for their development branch on a daily basis
>> so that continuous integration can efficiently run their tests against
>> the dev branch of their dependencies, so as to spot regressions, ASAP
>> before making a release.
>>
>> For instance, once numpy and scipy have been configured to upload
>> nightly builds there it will be possible to reconfigure the scheduled
>> tests of a project that depends on scikit-learn to run against the dev
>> branch of the full stack using:
>>
>> pip install -i https://pypi.anaconda.org/scipy-wheels-nightly/simple
>> scikit-learn
>> <https://pypi.anaconda.org/scipy-wheels-nightly/simplescikit-learn>
>> pip install --no-build-isolation -e .
>> pytest
>>
>> If you are a maintainer of one of the projects that publishes
>> nightly-builds on http://wheels.scipy.org, **please create an account
>> on https://anaconda.org and reply to this email with your identifier**
>> so that I can grant you permissions on this shared organization.
>>
>> From there you will be able to generate tokens to use as secret
>> variable in your CI servers. To use those access tokens in a CI setup
>> will need to enable permissions for the token:
>>
>> - Write access to API
>> - Upload pypi packages
>>
>> ## Staging hosting for release artifacts
>>
>> Each project is free to use its own staging area. Staging areas are
>> just useful as a synchronization step in a multi-CI release automation
>> setup. There is really no need to publish those wheels to end-users.
>> The end users will grab the wheels of the released versions from the
>> main https://pypi.org index once the release is done.
>>
>> That being said, I plan to also create a default staging area for
>> convenience, e.g.:
>>
>> - https://anaconda.org/multibuild-wheels-staging/
>>
>> for projects who do not really care and want to use the
>> https://github.com/matthew-brett/multibuild scripts with minimal admin
>> efforts.
>>
>> Thanks for your attention, let me know if you have any comment on this
>> plan.
>>
>> --
>> Olivier
>> _______________________________________________
>> SciPy-Dev mailing list
>> SciPy-Dev at python.org
>> https://mail.python.org/mailman/listinfo/scipy-dev
>>
> _______________________________________________
> SciPy-Dev mailing list
> SciPy-Dev at python.org
> https://mail.python.org/mailman/listinfo/scipy-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scipy-dev/attachments/20200212/c532110a/attachment.html>

From ralf.gommers at gmail.com  Fri Feb 14 22:04:39 2020
From: ralf.gommers at gmail.com (Ralf Gommers)
Date: Fri, 14 Feb 2020 21:04:39 -0600
Subject: [SciPy-Dev] adding Khatri-Rao product to linalg
Message-ID: <CABL7CQjDbfYoPSx5-P1cpPMxqV8rct1QnaGdmKYaxaR4=zxFnA@mail.gmail.com>

Hi all,

I think this wasn't run by this list yet:
https://github.com/scipy/scipy/pull/11456 is a PR to add the Khatri-Rao
product to scipy.linalg. It seems like a good fit to me. Please comment
here or on the PR soon if you disagree. Unless there's objections, we'll
merge it in a few days.

Cheers,
Ralf
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scipy-dev/attachments/20200214/e13d1e65/attachment.html>

From rlucas7 at vt.edu  Fri Feb 14 23:38:39 2020
From: rlucas7 at vt.edu (rlucas7 at vt.edu)
Date: Fri, 14 Feb 2020 23:38:39 -0500
Subject: [SciPy-Dev] adding Khatri-Rao product to linalg
In-Reply-To: <CABL7CQjDbfYoPSx5-P1cpPMxqV8rct1QnaGdmKYaxaR4=zxFnA@mail.gmail.com>
References: <CABL7CQjDbfYoPSx5-P1cpPMxqV8rct1QnaGdmKYaxaR4=zxFnA@mail.gmail.com>
Message-ID: <50937718-1B22-4BFA-A92A-15E75498FA02@vt.edu>

No objections on inclusion from me-I could see some niche statistical applications for this one, so +1 for the feature. Those statistical applications would likely be in other packages taking a dependency on scipy.

I took a brief look at the PR, it looks like all tests are with dense arrays. 
 should they cover sparse arrays too or is explicitly out of scope here?

Not super familiar myself with how the sparse codes work I in scipy.
It would seem that sparse Khatri-Rao with a dense or a sparse should be possible, not sure if result should be sparse or dense though so figured I?d ask the question. 

-Lucas Roberts 

> On Feb 14, 2020, at 10:05 PM, Ralf Gommers <ralf.gommers at gmail.com> wrote:
> 
> ?
> Hi all,
> 
> I think this wasn't run by this list yet: https://github.com/scipy/scipy/pull/11456 is a PR to add the Khatri-Rao product to scipy.linalg. It seems like a good fit to me. Please comment here or on the PR soon if you disagree. Unless there's objections, we'll merge it in a few days.
> 
> Cheers,
> Ralf
> 
> _______________________________________________
> SciPy-Dev mailing list
> SciPy-Dev at python.org
> https://mail.python.org/mailman/listinfo/scipy-dev
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scipy-dev/attachments/20200214/892c4c40/attachment.html>

From thomas.roderick at gmail.com  Fri Feb 14 23:51:40 2020
From: thomas.roderick at gmail.com (Tom Roderick)
Date: Fri, 14 Feb 2020 22:51:40 -0600
Subject: [SciPy-Dev] Newbie question: submitting PRs, review
Message-ID: <CAG95VH8dpXnhRc0SpfYiOfGpBy3=TeTKKbg7aPZzaosCq4S4_Q@mail.gmail.com>

Hi folks

This is my first time mailing the list, so forgive me if I come across
naive -- it is because I am!

I noticed that the metric in scipy.spatial.distance.sokalmichener is the
wrong metric, as per many sources (e.g.
https://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.352.6123&rep=rep1&type=pdf)
-- the rogerstanimoto dissimilarity is in its place. I'd like to correct
it, but am not 100% sure I understand the process.

So, wanted to reach out to the mailing list and hopefully get some
mentoring. Is the Scipy process as simple as: fix code, add unit test,
submit Github PR?

Best,
-Tom
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scipy-dev/attachments/20200214/0ea51d53/attachment.html>

From smishra99.iitkgp at gmail.com  Sat Feb 15 01:33:01 2020
From: smishra99.iitkgp at gmail.com (Shubham Mishra)
Date: Sat, 15 Feb 2020 12:03:01 +0530
Subject: [SciPy-Dev] [Newbie] Information regarding GSoC
Message-ID: <CAPhLvQMkh8B7hq7Ho4o1+my_cpDs7askueykzjNzRm8-LmFefA@mail.gmail.com>

Hello,
Newbie here. Is Scipy participating in GSoC this year?
I am curious to work under Scipy for GSoC if it is possible. Can someone
mentor me regarding this?

Thanking you,
Yours sincerely,
Shubham Mishra
(Github handle: grapheo12)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scipy-dev/attachments/20200215/63bb9fda/attachment.html>

From smishra99.iitkgp at gmail.com  Sat Feb 15 01:56:58 2020
From: smishra99.iitkgp at gmail.com (Shubham Mishra)
Date: Sat, 15 Feb 2020 12:26:58 +0530
Subject: [SciPy-Dev] Newbie question: submitting PRs, review
In-Reply-To: <CAG95VH8dpXnhRc0SpfYiOfGpBy3=TeTKKbg7aPZzaosCq4S4_Q@mail.gmail.com>
References: <CAG95VH8dpXnhRc0SpfYiOfGpBy3=TeTKKbg7aPZzaosCq4S4_Q@mail.gmail.com>
Message-ID: <CAPhLvQNr42-Xm6eSWUkD95MoV5XN+MKZObVH2HFpeqf5PDX7_g@mail.gmail.com>

Hey,
I am a newbie too. I recently merged 1 PR. Here is what I did:
1. Communicated over the issue on Github issue page (In your case, you
might have to raise one)
2. Went through the Contribution guidelines.
3. Forked the repo and created a dev environment.
4. Made the required changes.
5. Raised a PR and asked for review.

You can go through HACKING.rst.txt on the root directory of the repo for
reference.

Hope this helps.
Warm regards,
Shubham Mishra (grapheo12)

On Sat, 15 Feb 2020, 10:22 Tom Roderick, <thomas.roderick at gmail.com> wrote:

> Hi folks
>
> This is my first time mailing the list, so forgive me if I come across
> naive -- it is because I am!
>
> I noticed that the metric in scipy.spatial.distance.sokalmichener is the
> wrong metric, as per many sources (e.g.
> https://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.352.6123&rep=rep1&type=pdf)
> -- the rogerstanimoto dissimilarity is in its place. I'd like to correct
> it, but am not 100% sure I understand the process.
>
> So, wanted to reach out to the mailing list and hopefully get some
> mentoring. Is the Scipy process as simple as: fix code, add unit test,
> submit Github PR?
>
> Best,
> -Tom
> _______________________________________________
> SciPy-Dev mailing list
> SciPy-Dev at python.org
> https://mail.python.org/mailman/listinfo/scipy-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scipy-dev/attachments/20200215/5d6f39ed/attachment-0001.html>

From olivier.grisel at ensta.org  Sat Feb 15 05:10:29 2020
From: olivier.grisel at ensta.org (Olivier Grisel)
Date: Sat, 15 Feb 2020 11:10:29 +0100
Subject: [SciPy-Dev] Discontinued Rackspace Open Compute Discount
In-Reply-To: <CAJR3sXebe8bz+C1YuW5i7eS1bq3DjN9Bj3AByakmuM1FZcLS0w@mail.gmail.com>
References: <470871448.2376.1581182952898@wamui-napoleon.atl.sa.earthlink.net>
 <CAFvE7K43Uz2deJy81r_jxXT-8=qYYcpcjssU3TFrB4AUSuuvnw@mail.gmail.com>
 <CAFvE7K4vCb72dE2=-3Wy1bfKCCax7g8JxEP8qWSGi-exm_mn=A@mail.gmail.com>
 <CA+WonSRKO+XWtffy3YC80wpCZS7Z6cnbSPp=gy4NmuF6ucDn8Q@mail.gmail.com>
 <CAH6Pt5ooyWb+jwqM2OjVVnf8noox0PO7DxY74x_47h9nJowq8g@mail.gmail.com>
 <CA+WonSRZ1prkf6MbSoHYBFkOR_5J9Fv4jcW34G456MYJKterxw@mail.gmail.com>
 <CAFvE7K58Hhc68MUdF90DF+w1ACDw2MCxESmDfCDs3JvdRiB41Q@mail.gmail.com>
 <CAFvE7K5+M5COuuiSHHajNiVJL_omAX71VcUFfk88A+A-KmdikQ@mail.gmail.com>
 <CAFvE7K7bmoPZcRQrEqt11mE=X0FgqAUYmhQbj0gvcM5QL=i4GQ@mail.gmail.com>
 <CAE1aY-kjnNtsUych1kDMSt2hT5qk7K6e4gOuQnUY-LSy0otbjg@mail.gmail.com>
 <CAJR3sXebe8bz+C1YuW5i7eS1bq3DjN9Bj3AByakmuM1FZcLS0w@mail.gmail.com>
Message-ID: <CAFvE7K4FbOU0vMdejQHo7Qr9BR6FKRX17tCQXe7aRG6Hc=DQTQ@mail.gmail.com>

Tyler, Tom: I added you as co-owners of the scipy-wheels-nightly organization.
Gregory, Tyler, Tom: I added you as co-owners of the
multibuild-wheels-staging organization.

-- 
Olivier

From ralf.gommers at gmail.com  Sat Feb 15 13:11:33 2020
From: ralf.gommers at gmail.com (Ralf Gommers)
Date: Sat, 15 Feb 2020 12:11:33 -0600
Subject: [SciPy-Dev] [Newbie] Information regarding GSoC
In-Reply-To: <CAPhLvQMkh8B7hq7Ho4o1+my_cpDs7askueykzjNzRm8-LmFefA@mail.gmail.com>
References: <CAPhLvQMkh8B7hq7Ho4o1+my_cpDs7askueykzjNzRm8-LmFefA@mail.gmail.com>
Message-ID: <CABL7CQh_c1rcRVKstW2Au+jpn5nniLLwPC5Mu-sOj9EKymK_Ng@mail.gmail.com>

On Sat, Feb 15, 2020 at 12:33 AM Shubham Mishra <smishra99.iitkgp at gmail.com>
wrote:

> Hello,
> Newbie here. Is Scipy participating in GSoC this year?
> I am curious to work under Scipy for GSoC if it is possible. Can someone
> mentor me regarding this?
>

Hi Shubham, thanks for asking. We haven't discussed it yet, now would be a
good time to do that - I'll open a separate thread for it.

Cheers,
Ralf



> Thanking you,
> Yours sincerely,
> Shubham Mishra
> (Github handle: grapheo12)
> _______________________________________________
> SciPy-Dev mailing list
> SciPy-Dev at python.org
> https://mail.python.org/mailman/listinfo/scipy-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scipy-dev/attachments/20200215/2dee45bb/attachment.html>

From ralf.gommers at gmail.com  Sat Feb 15 13:15:47 2020
From: ralf.gommers at gmail.com (Ralf Gommers)
Date: Sat, 15 Feb 2020 12:15:47 -0600
Subject: [SciPy-Dev] participating in GSoC'20?
Message-ID: <CABL7CQgJNzbRnC=E=8TY89tnW1FFv4=fShaDQoZM4fN7vSvVrw@mail.gmail.com>

Hi all,

It's that time of the year again already. Do we want to participate in this
years' Google Summer of Code? Last year was quite successful (Peter's work
on scipy.fft). We do need some volunteers to mentors though, and ideally
also someone to admin (I'm pretty swamped). Any takers? Any good topic
ideas?

Cheers,
Ralf
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scipy-dev/attachments/20200215/cd47b751/attachment.html>

From eryole at gmail.com  Sat Feb 15 14:05:55 2020
From: eryole at gmail.com (Nicolas Cellier)
Date: Sat, 15 Feb 2020 20:05:55 +0100
Subject: [SciPy-Dev] participating in GSoC'20?
In-Reply-To: <CABL7CQgJNzbRnC=E=8TY89tnW1FFv4=fShaDQoZM4fN7vSvVrw@mail.gmail.com>
References: <CABL7CQgJNzbRnC=E=8TY89tnW1FFv4=fShaDQoZM4fN7vSvVrw@mail.gmail.com>
Message-ID: <24B42FCB-710D-45BB-B6ED-F80BE6EFB6A5@getmailspring.com>

Hello.

As topic ideas, it could be good to use the (not so) recent scipy.integrate.solve_ivp interface to implement more modern ODE solvers (like SSP, Rosenbrock Wanner). The goal could be to provide more efficient solvers as the one available with the Julia equivalent library (DifferentialEquation.jl if I remember well).
An other idea could be the implementation of Jacobian coloring and sparsity detection. They are really useful tools that can highly improve methods implying Jacobian computation. (see https://pdfs.semanticscholar.org/ae4d/65769b6551a51d6fc6be2f021515bffa0798.pdf (https://link.getmailspring.com/link/24B42FCB-710D-45BB-B6ED-F80BE6EFB6A5 at getmailspring.com/0?redirect=https%3A%2F%2Fpdfs.semanticscholar.org%2Fae4d%2F65769b6551a51d6fc6be2f021515bffa0798.pdf&recipient=c2NpcHktZGV2QHB5dGhvbi5vcmc%3D) for a reference on Jacobian coloring).
Cheers,
Nicolas Cellier

On f?vr. 15 2020, at 7:15 pm, Ralf Gommers <ralf.gommers at gmail.com> wrote:
> Hi all,
>
> It's that time of the year again already. Do we want to participate in this years' Google Summer of Code? Last year was quite successful (Peter's work on scipy.fft). We do need some volunteers to mentors though, and ideally also someone to admin (I'm pretty swamped). Any takers? Any good topic ideas?
>
> Cheers,
> Ralf
>
> _______________________________________________
> SciPy-Dev mailing list
> SciPy-Dev at python.org
> https://mail.python.org/mailman/listinfo/scipy-dev
>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scipy-dev/attachments/20200215/3e1546a9/attachment.html>

From ralf.gommers at gmail.com  Sat Feb 15 21:00:01 2020
From: ralf.gommers at gmail.com (Ralf Gommers)
Date: Sat, 15 Feb 2020 20:00:01 -0600
Subject: [SciPy-Dev] Newbie question: submitting PRs, review
In-Reply-To: <CAG95VH8dpXnhRc0SpfYiOfGpBy3=TeTKKbg7aPZzaosCq4S4_Q@mail.gmail.com>
References: <CAG95VH8dpXnhRc0SpfYiOfGpBy3=TeTKKbg7aPZzaosCq4S4_Q@mail.gmail.com>
Message-ID: <CABL7CQjViHLOjPbLdGi=_49hcUZ22w7tOwAMgsfyhtxR02fK4w@mail.gmail.com>

On Fri, Feb 14, 2020 at 10:52 PM Tom Roderick <thomas.roderick at gmail.com>
wrote:

> Hi folks
>
> This is my first time mailing the list, so forgive me if I come across
> naive -- it is because I am!
>

Hi Tom, welcome!


> I noticed that the metric in scipy.spatial.distance.sokalmichener is the
> wrong metric, as per many sources (e.g.
> https://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.352.6123&rep=rep1&type=pdf)
> -- the rogerstanimoto dissimilarity is in its place. I'd like to correct
> it, but am not 100% sure I understand the process.
>

In this case there's some previous discussion on this topic. This is the
relevant bug report: https://github.com/scipy/scipy/issues/2011. And this
identifies more issues and talks about how we should handle them:
https://github.com/scipy/scipy/pull/3163#issuecomment-34224341.

>
> So, wanted to reach out to the mailing list and hopefully get some
> mentoring. Is the Scipy process as simple as: fix code, add unit test,
> submit Github PR?
>

It can be, but API changes or backwards incompatible changes usually are
more involved (discussion wise mainly). I think you can submit the minimal
required fix here though, and add some more references you found to issue
2011. Regarding process there's some more guidance in the contributor
guide, http://scipy.github.io/devdocs/dev/contributor/contributor_toc.html,
in particular the "development workflow" section.

Cheeers,
Ralf



> Best,
> -Tom
> _______________________________________________
> SciPy-Dev mailing list
> SciPy-Dev at python.org
> https://mail.python.org/mailman/listinfo/scipy-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scipy-dev/attachments/20200215/6948d660/attachment-0001.html>

From ralf.gommers at gmail.com  Sun Feb 16 07:02:17 2020
From: ralf.gommers at gmail.com (Ralf Gommers)
Date: Sun, 16 Feb 2020 06:02:17 -0600
Subject: [SciPy-Dev] adding Khatri-Rao product to linalg
In-Reply-To: <50937718-1B22-4BFA-A92A-15E75498FA02@vt.edu>
References: <CABL7CQjDbfYoPSx5-P1cpPMxqV8rct1QnaGdmKYaxaR4=zxFnA@mail.gmail.com>
 <50937718-1B22-4BFA-A92A-15E75498FA02@vt.edu>
Message-ID: <CABL7CQgsoY+xbGha+Rc3GvwSooLQW2uLVDsEwNAcksiaOCu_+Q@mail.gmail.com>

On Fri, Feb 14, 2020 at 10:39 PM <rlucas7 at vt.edu> wrote:

> No objections on inclusion from me-I could see some niche statistical
> applications for this one, so +1 for the feature. Those statistical
> applications would likely be in other packages taking a dependency on scipy.
>
> I took a brief look at the PR, it looks like all tests are with dense
> arrays.
>  should they cover sparse arrays too or is explicitly out of scope here?
>
> Not super familiar myself with how the sparse codes work I in scipy.
> It would seem that sparse Khatri-Rao with a dense or a sparse should be
> possible, not sure if result should be sparse or dense though so figured
> I?d ask the question.
>

Functions in scipy.linalg don't support sparse arrays; scipy.sparse.linalg
is for that. So if there's interest, that could be added separately (not
sure that's worth it though, there's probably other functions that would be
higher on the list for sparse).

Cheers,
Ralf



>
> -Lucas Roberts
>
> On Feb 14, 2020, at 10:05 PM, Ralf Gommers <ralf.gommers at gmail.com> wrote:
>
> ?
> Hi all,
>
> I think this wasn't run by this list yet:
> https://github.com/scipy/scipy/pull/11456 is a PR to add the Khatri-Rao
> product to scipy.linalg. It seems like a good fit to me. Please comment
> here or on the PR soon if you disagree. Unless there's objections, we'll
> merge it in a few days.
>
> Cheers,
> Ralf
>
> _______________________________________________
> SciPy-Dev mailing list
> SciPy-Dev at python.org
> https://mail.python.org/mailman/listinfo/scipy-dev
>
> _______________________________________________
> SciPy-Dev mailing list
> SciPy-Dev at python.org
> https://mail.python.org/mailman/listinfo/scipy-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scipy-dev/attachments/20200216/fd8fde8d/attachment.html>

From rlucas7 at vt.edu  Sun Feb 16 11:12:25 2020
From: rlucas7 at vt.edu (rlucas7 at vt.edu)
Date: Sun, 16 Feb 2020 11:12:25 -0500
Subject: [SciPy-Dev] adding Khatri-Rao product to linalg
In-Reply-To: <CABL7CQgsoY+xbGha+Rc3GvwSooLQW2uLVDsEwNAcksiaOCu_+Q@mail.gmail.com>
References: <CABL7CQgsoY+xbGha+Rc3GvwSooLQW2uLVDsEwNAcksiaOCu_+Q@mail.gmail.com>
Message-ID: <0F94C5CB-E9C9-4776-89A4-9CD6AD023C8B@vt.edu>



Sincerely, 

-Lucas Roberts 

> On Feb 16, 2020, at 7:03 AM, Ralf Gommers <ralf.gommers at gmail.com> wrote:
> 
> ?
> 
> 
>> On Fri, Feb 14, 2020 at 10:39 PM <rlucas7 at vt.edu> wrote:
>> No objections on inclusion from me-I could see some niche statistical applications for this one, so +1 for the feature. Those statistical applications would likely be in other packages taking a dependency on scipy.
>> 
>> I took a brief look at the PR, it looks like all tests are with dense arrays. 
>>  should they cover sparse arrays too or is explicitly out of scope here?
>> 
>> Not super familiar myself with how the sparse codes work I in scipy.
>> It would seem that sparse Khatri-Rao with a dense or a sparse should be possible, not sure if result should be sparse or dense though so figured I?d ask the question. 
> 
> Functions in scipy.linalg don't support sparse arrays; scipy.sparse.linalg is for that. So if there's interest, that could be added separately (not sure that's worth it though, there's probably other functions that would be higher on the list for sparse).
> 
> Cheers,
> Ralf
> 

Ok that makes sense, thanks for clarifying. 

>> 
>> -Lucas Roberts 
>> 
>>>> On Feb 14, 2020, at 10:05 PM, Ralf Gommers <ralf.gommers at gmail.com> wrote:
>>>> 
>>> ?
>>> Hi all,
>>> 
>>> I think this wasn't run by this list yet: https://github.com/scipy/scipy/pull/11456 is a PR to add the Khatri-Rao product to scipy.linalg. It seems like a good fit to me. Please comment here or on the PR soon if you disagree. Unless there's objections, we'll merge it in a few days.
>>> 
>>> Cheers,
>>> Ralf
>>> 
>>> _______________________________________________
>>> SciPy-Dev mailing list
>>> SciPy-Dev at python.org
>>> https://mail.python.org/mailman/listinfo/scipy-dev
>> _______________________________________________
>> SciPy-Dev mailing list
>> SciPy-Dev at python.org
>> https://mail.python.org/mailman/listinfo/scipy-dev
> _______________________________________________
> SciPy-Dev mailing list
> SciPy-Dev at python.org
> https://mail.python.org/mailman/listinfo/scipy-dev
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scipy-dev/attachments/20200216/efbd361e/attachment.html>

From rlucas7 at vt.edu  Sun Feb 16 21:35:59 2020
From: rlucas7 at vt.edu (Lucas Roberts)
Date: Sun, 16 Feb 2020 21:35:59 -0500
Subject: [SciPy-Dev] changing default for median_absolute_deviation
Message-ID: <CADCr_xNO71+VpreQ3qztmGPMdJi3YPy4zO5YPfGiPZwaSmgyQw@mail.gmail.com>

Hi scipy-dev,

Looking for guidance if the change warrants ignoring deprecation policy or
if I should create a deprecation warning for existing behavior to change
defaults in median_absolute_deviation()

PR:
https://github.com/scipy/scipy/pull/11431

[CONTEXT]
In this PR:
https://github.com/scipy/scipy/pull/9637
the median_absolute_deviation() function was added to scipy.stats
the function takes in a scale argument that allows for a robust estimator
of the scale (robust to outliers). The choice of scale constant assuming
normal input data seemed reasonable to me at the time.
However, in this PR:
https://github.com/scipy/scipy/issues/11090
a few thought otherwise and we came to the conclusion that the default
scale should be 1 which would also ensure internal consistency with the
stats.iqr() function (similar signature and functionality).

I've opened a PR to change the defaults here:
https://github.com/scipy/scipy/pull/11431
and pinged those on the 11090 issue.

[ON DEPRECATION]:

[REASON FOR]
Main reason to not follow deprecation policy are:
1. Following deprecation policy would maintain existing confusing behavior
for up to 1 year from now.
2. Function released in 1.3.0 so somewhat new
3. Seems the default is confusing users and is non-obvious and inconsistent
with iqr() default

[REASON NOT]
1. Some users may depend on existing default
2. Defaults with normal scaling exist in several places (c.f. 11090 issue
comments)
3. Deprecation warning needs to be done to give fair warning of the change.

[COMMENTS]
I haven't much experience here with deprecations, when to do it vs consider
a defect so would appreciate any guidance.

If we think the deprecation cycle should follow, should I leave the PR and
open a separate deprecation PR?
https://github.com/scipy/scipy/pull/11431
Alternately could revert changes and convert to a deprecation warning PR.

Thanks in advance.
-- 
-Lucas
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scipy-dev/attachments/20200216/3f72c5fb/attachment.html>

From thomas.roderick at gmail.com  Sun Feb 16 21:46:24 2020
From: thomas.roderick at gmail.com (Tom Roderick)
Date: Sun, 16 Feb 2020 20:46:24 -0600
Subject: [SciPy-Dev] Newbie question: submitting PRs, review
In-Reply-To: <CABL7CQjViHLOjPbLdGi=_49hcUZ22w7tOwAMgsfyhtxR02fK4w@mail.gmail.com>
References: <CAG95VH8dpXnhRc0SpfYiOfGpBy3=TeTKKbg7aPZzaosCq4S4_Q@mail.gmail.com>
 <CABL7CQjViHLOjPbLdGi=_49hcUZ22w7tOwAMgsfyhtxR02fK4w@mail.gmail.com>
Message-ID: <CAG95VH_t4gU=vKkcRTntjX3pGn-Ax6sGKksb8Pey=OJJSSMyAg@mail.gmail.com>

So is the way forward to:
(1) Write a minimal PR
(2) Bring open discussion?

The issue appears to be pretty old, and now I'm concerned that people may
be expecting one metric and getting another!

Happy to roll up my sleeves, want to make sure there isn't another blocker
here outside of a working branch?

On Sat, Feb 15, 2020 at 8:00 PM Ralf Gommers <ralf.gommers at gmail.com> wrote:

>
>
> On Fri, Feb 14, 2020 at 10:52 PM Tom Roderick <thomas.roderick at gmail.com>
> wrote:
>
>> Hi folks
>>
>> This is my first time mailing the list, so forgive me if I come across
>> naive -- it is because I am!
>>
>
> Hi Tom, welcome!
>
>
>> I noticed that the metric in scipy.spatial.distance.sokalmichener is the
>> wrong metric, as per many sources (e.g.
>> https://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.352.6123&rep=rep1&type=pdf)
>> -- the rogerstanimoto dissimilarity is in its place. I'd like to correct
>> it, but am not 100% sure I understand the process.
>>
>
> In this case there's some previous discussion on this topic. This is the
> relevant bug report: https://github.com/scipy/scipy/issues/2011. And this
> identifies more issues and talks about how we should handle them:
> https://github.com/scipy/scipy/pull/3163#issuecomment-34224341.
>
>>
>> So, wanted to reach out to the mailing list and hopefully get some
>> mentoring. Is the Scipy process as simple as: fix code, add unit test,
>> submit Github PR?
>>
>
> It can be, but API changes or backwards incompatible changes usually are
> more involved (discussion wise mainly). I think you can submit the minimal
> required fix here though, and add some more references you found to issue
> 2011. Regarding process there's some more guidance in the contributor
> guide, http://scipy.github.io/devdocs/dev/contributor/contributor_toc.html,
> in particular the "development workflow" section.
>
> Cheeers,
> Ralf
>
>
>
>> Best,
>> -Tom
>> _______________________________________________
>> SciPy-Dev mailing list
>> SciPy-Dev at python.org
>> https://mail.python.org/mailman/listinfo/scipy-dev
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scipy-dev/attachments/20200216/1e17ee93/attachment.html>

From ralf.gommers at gmail.com  Sun Feb 16 21:57:26 2020
From: ralf.gommers at gmail.com (Ralf Gommers)
Date: Sun, 16 Feb 2020 18:57:26 -0800
Subject: [SciPy-Dev] Newbie question: submitting PRs, review
In-Reply-To: <CAG95VH_t4gU=vKkcRTntjX3pGn-Ax6sGKksb8Pey=OJJSSMyAg@mail.gmail.com>
References: <CAG95VH8dpXnhRc0SpfYiOfGpBy3=TeTKKbg7aPZzaosCq4S4_Q@mail.gmail.com>
 <CABL7CQjViHLOjPbLdGi=_49hcUZ22w7tOwAMgsfyhtxR02fK4w@mail.gmail.com>
 <CAG95VH_t4gU=vKkcRTntjX3pGn-Ax6sGKksb8Pey=OJJSSMyAg@mail.gmail.com>
Message-ID: <CABL7CQjQDTunDOcbEgRqzrB6FXpQ2+u40c3sBOAM_LbUB6XaTA@mail.gmail.com>

On Sun, Feb 16, 2020 at 6:46 PM Tom Roderick <thomas.roderick at gmail.com>
wrote:

> So is the way forward to:
> (1) Write a minimal PR
> (2) Bring open discussion?
>
> The issue appears to be pretty old, and now I'm concerned that people may
> be expecting one metric and getting another!
>

True. The alternatives would be to either add a FutureWarning before making
the switch, or deprecating and then introducing the right metric under a
different name.

I think if it's really a clear cut bug, we can make the switch and add an
entry in the "backwards incompatible changes" section of the release notes
to explain it.


>
> Happy to roll up my sleeves, want to make sure there isn't another blocker
> here outside of a working branch?
>

I think the PR I linked to made the same change you're proposing, but it
was really large and got abandoned. If you make a small PR with just this
change and a clear explanation, I think there won't be another blocker.

Cheers,
Ralf


>
> On Sat, Feb 15, 2020 at 8:00 PM Ralf Gommers <ralf.gommers at gmail.com>
> wrote:
>
>>
>>
>> On Fri, Feb 14, 2020 at 10:52 PM Tom Roderick <thomas.roderick at gmail.com>
>> wrote:
>>
>>> Hi folks
>>>
>>> This is my first time mailing the list, so forgive me if I come across
>>> naive -- it is because I am!
>>>
>>
>> Hi Tom, welcome!
>>
>>
>>> I noticed that the metric in scipy.spatial.distance.sokalmichener is the
>>> wrong metric, as per many sources (e.g.
>>> https://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.352.6123&rep=rep1&type=pdf)
>>> -- the rogerstanimoto dissimilarity is in its place. I'd like to correct
>>> it, but am not 100% sure I understand the process.
>>>
>>
>> In this case there's some previous discussion on this topic. This is the
>> relevant bug report: https://github.com/scipy/scipy/issues/2011. And
>> this identifies more issues and talks about how we should handle them:
>> https://github.com/scipy/scipy/pull/3163#issuecomment-34224341.
>>
>>>
>>> So, wanted to reach out to the mailing list and hopefully get some
>>> mentoring. Is the Scipy process as simple as: fix code, add unit test,
>>> submit Github PR?
>>>
>>
>> It can be, but API changes or backwards incompatible changes usually are
>> more involved (discussion wise mainly). I think you can submit the minimal
>> required fix here though, and add some more references you found to issue
>> 2011. Regarding process there's some more guidance in the contributor
>> guide,
>> http://scipy.github.io/devdocs/dev/contributor/contributor_toc.html, in
>> particular the "development workflow" section.
>>
>> Cheeers,
>> Ralf
>>
>>
>>
>>> Best,
>>> -Tom
>>> _______________________________________________
>>> SciPy-Dev mailing list
>>> SciPy-Dev at python.org
>>> https://mail.python.org/mailman/listinfo/scipy-dev
>>>
>>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scipy-dev/attachments/20200216/33619e6b/attachment-0001.html>

From thomas.roderick at gmail.com  Sun Feb 16 22:27:06 2020
From: thomas.roderick at gmail.com (Tom Roderick)
Date: Sun, 16 Feb 2020 21:27:06 -0600
Subject: [SciPy-Dev] Newbie question: submitting PRs, review
In-Reply-To: <CABL7CQjQDTunDOcbEgRqzrB6FXpQ2+u40c3sBOAM_LbUB6XaTA@mail.gmail.com>
References: <CAG95VH8dpXnhRc0SpfYiOfGpBy3=TeTKKbg7aPZzaosCq4S4_Q@mail.gmail.com>
 <CABL7CQjViHLOjPbLdGi=_49hcUZ22w7tOwAMgsfyhtxR02fK4w@mail.gmail.com>
 <CAG95VH_t4gU=vKkcRTntjX3pGn-Ax6sGKksb8Pey=OJJSSMyAg@mail.gmail.com>
 <CABL7CQjQDTunDOcbEgRqzrB6FXpQ2+u40c3sBOAM_LbUB6XaTA@mail.gmail.com>
Message-ID: <CAG95VH_8qpLXnBJ-mf0VN91WGzsX17K-_WCgSo2JGc_cBgR9Vg@mail.gmail.com>

Thanks Ralf! Will do.

On Sun, Feb 16, 2020 at 8:59 PM Ralf Gommers <ralf.gommers at gmail.com> wrote:

>
>
> On Sun, Feb 16, 2020 at 6:46 PM Tom Roderick <thomas.roderick at gmail.com>
> wrote:
>
>> So is the way forward to:
>> (1) Write a minimal PR
>> (2) Bring open discussion?
>>
>> The issue appears to be pretty old, and now I'm concerned that people may
>> be expecting one metric and getting another!
>>
>
> True. The alternatives would be to either add a FutureWarning before
> making the switch, or deprecating and then introducing the right metric
> under a different name.
>
> I think if it's really a clear cut bug, we can make the switch and add an
> entry in the "backwards incompatible changes" section of the release notes
> to explain it.
>
>
>>
>> Happy to roll up my sleeves, want to make sure there isn't another
>> blocker here outside of a working branch?
>>
>
> I think the PR I linked to made the same change you're proposing, but it
> was really large and got abandoned. If you make a small PR with just this
> change and a clear explanation, I think there won't be another blocker.
>
> Cheers,
> Ralf
>
>
>>
>> On Sat, Feb 15, 2020 at 8:00 PM Ralf Gommers <ralf.gommers at gmail.com>
>> wrote:
>>
>>>
>>>
>>> On Fri, Feb 14, 2020 at 10:52 PM Tom Roderick <thomas.roderick at gmail.com>
>>> wrote:
>>>
>>>> Hi folks
>>>>
>>>> This is my first time mailing the list, so forgive me if I come across
>>>> naive -- it is because I am!
>>>>
>>>
>>> Hi Tom, welcome!
>>>
>>>
>>>> I noticed that the metric in scipy.spatial.distance.sokalmichener is
>>>> the wrong metric, as per many sources (e.g.
>>>> https://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.352.6123&rep=rep1&type=pdf)
>>>> -- the rogerstanimoto dissimilarity is in its place. I'd like to correct
>>>> it, but am not 100% sure I understand the process.
>>>>
>>>
>>> In this case there's some previous discussion on this topic. This is the
>>> relevant bug report: https://github.com/scipy/scipy/issues/2011. And
>>> this identifies more issues and talks about how we should handle them:
>>> https://github.com/scipy/scipy/pull/3163#issuecomment-34224341.
>>>
>>>>
>>>> So, wanted to reach out to the mailing list and hopefully get some
>>>> mentoring. Is the Scipy process as simple as: fix code, add unit test,
>>>> submit Github PR?
>>>>
>>>
>>> It can be, but API changes or backwards incompatible changes usually are
>>> more involved (discussion wise mainly). I think you can submit the minimal
>>> required fix here though, and add some more references you found to issue
>>> 2011. Regarding process there's some more guidance in the contributor
>>> guide,
>>> http://scipy.github.io/devdocs/dev/contributor/contributor_toc.html, in
>>> particular the "development workflow" section.
>>>
>>> Cheeers,
>>> Ralf
>>>
>>>
>>>
>>>> Best,
>>>> -Tom
>>>> _______________________________________________
>>>> SciPy-Dev mailing list
>>>> SciPy-Dev at python.org
>>>> https://mail.python.org/mailman/listinfo/scipy-dev
>>>>
>>> _______________________________________________
> SciPy-Dev mailing list
> SciPy-Dev at python.org
> https://mail.python.org/mailman/listinfo/scipy-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scipy-dev/attachments/20200216/9ff7cdaf/attachment.html>

From tyler.je.reddy at gmail.com  Mon Feb 17 00:02:02 2020
From: tyler.je.reddy at gmail.com (Tyler Reddy)
Date: Sun, 16 Feb 2020 22:02:02 -0700
Subject: [SciPy-Dev] Discontinued Rackspace Open Compute Discount
In-Reply-To: <CAFvE7K4FbOU0vMdejQHo7Qr9BR6FKRX17tCQXe7aRG6Hc=DQTQ@mail.gmail.com>
References: <470871448.2376.1581182952898@wamui-napoleon.atl.sa.earthlink.net>
 <CAFvE7K43Uz2deJy81r_jxXT-8=qYYcpcjssU3TFrB4AUSuuvnw@mail.gmail.com>
 <CAFvE7K4vCb72dE2=-3Wy1bfKCCax7g8JxEP8qWSGi-exm_mn=A@mail.gmail.com>
 <CA+WonSRKO+XWtffy3YC80wpCZS7Z6cnbSPp=gy4NmuF6ucDn8Q@mail.gmail.com>
 <CAH6Pt5ooyWb+jwqM2OjVVnf8noox0PO7DxY74x_47h9nJowq8g@mail.gmail.com>
 <CA+WonSRZ1prkf6MbSoHYBFkOR_5J9Fv4jcW34G456MYJKterxw@mail.gmail.com>
 <CAFvE7K58Hhc68MUdF90DF+w1ACDw2MCxESmDfCDs3JvdRiB41Q@mail.gmail.com>
 <CAFvE7K5+M5COuuiSHHajNiVJL_omAX71VcUFfk88A+A-KmdikQ@mail.gmail.com>
 <CAFvE7K7bmoPZcRQrEqt11mE=X0FgqAUYmhQbj0gvcM5QL=i4GQ@mail.gmail.com>
 <CAE1aY-kjnNtsUych1kDMSt2hT5qk7K6e4gOuQnUY-LSy0otbjg@mail.gmail.com>
 <CAJR3sXebe8bz+C1YuW5i7eS1bq3DjN9Bj3AByakmuM1FZcLS0w@mail.gmail.com>
 <CAFvE7K4FbOU0vMdejQHo7Qr9BR6FKRX17tCQXe7aRG6Hc=DQTQ@mail.gmail.com>
Message-ID: <CAHPuU_a4897sotNHgSiWzh+0JeOsvWxfA-WngD6z5ozywP6Jcw@mail.gmail.com>

Following Olivier's lead I've observed our first automatic upload to
anaconda.org from scipy-wheels from t his testing PR:
https://github.com/MacPython/scipy-wheels/pull/69

I've restored the full build matrix & the cron requirement after confirming
the upload worked to the target location using a secret key:
https://anaconda.org/scipy-wheels-nightly/scipy/files

I'll probably address the staging area for SciPy release wheels in a
separate PR from this cron job for weekly uploads. Since our master branch
wheels builds typically have some failures in the matrix for unrelated
reasons it is complicated enough as it is.

I did notice a 3.0 GB "free limit" I think---hopefully that doesn't become
an issue.


On Sat, 15 Feb 2020 at 03:11, Olivier Grisel <olivier.grisel at ensta.org>
wrote:

> Tyler, Tom: I added you as co-owners of the scipy-wheels-nightly
> organization.
> Gregory, Tyler, Tom: I added you as co-owners of the
> multibuild-wheels-staging organization.
>
> --
> Olivier
> _______________________________________________
> SciPy-Dev mailing list
> SciPy-Dev at python.org
> https://mail.python.org/mailman/listinfo/scipy-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scipy-dev/attachments/20200216/ec801389/attachment.html>

From smishra99.iitkgp at gmail.com  Mon Feb 17 03:27:05 2020
From: smishra99.iitkgp at gmail.com (Shubham Mishra)
Date: Mon, 17 Feb 2020 13:57:05 +0530
Subject: [SciPy-Dev] [Newbie] Information regarding GSoC
In-Reply-To: <CABL7CQh_c1rcRVKstW2Au+jpn5nniLLwPC5Mu-sOj9EKymK_Ng@mail.gmail.com>
References: <CAPhLvQMkh8B7hq7Ho4o1+my_cpDs7askueykzjNzRm8-LmFefA@mail.gmail.com>
 <CABL7CQh_c1rcRVKstW2Au+jpn5nniLLwPC5Mu-sOj9EKymK_Ng@mail.gmail.com>
Message-ID: <CAPhLvQMgZWsaQEAH9Dc7W4sBp8N6x4muVE-2RKbLh81QnPWH7A@mail.gmail.com>

Hello all.
Thanks for hearing me out regarding GSoC. I am interested in the
differential equation solvers as put forward by Nicolas. Should I start
preparing for the same?
I have done a basic freshman level math course where ODEs were taught, but
not up to that level. Can someone suggest some reading materials (books,
papers, tutorials... anything) from where I can learn?
I am really eager to learn and contribute in a meaningful way.

Yours sincerely,
Shubham Mishra (grapheo12)

On Sat, 15 Feb 2020, 23:41 Ralf Gommers, <ralf.gommers at gmail.com> wrote:

>
>
> On Sat, Feb 15, 2020 at 12:33 AM Shubham Mishra <
> smishra99.iitkgp at gmail.com> wrote:
>
>> Hello,
>> Newbie here. Is Scipy participating in GSoC this year?
>> I am curious to work under Scipy for GSoC if it is possible. Can someone
>> mentor me regarding this?
>>
>
> Hi Shubham, thanks for asking. We haven't discussed it yet, now would be a
> good time to do that - I'll open a separate thread for it.
>
> Cheers,
> Ralf
>
>
>
>> Thanking you,
>> Yours sincerely,
>> Shubham Mishra
>> (Github handle: grapheo12)
>> _______________________________________________
>> SciPy-Dev mailing list
>> SciPy-Dev at python.org
>> https://mail.python.org/mailman/listinfo/scipy-dev
>>
> _______________________________________________
> SciPy-Dev mailing list
> SciPy-Dev at python.org
> https://mail.python.org/mailman/listinfo/scipy-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scipy-dev/attachments/20200217/307e8961/attachment-0001.html>

From kaushikaakash7539 at gmail.com  Mon Feb 17 12:51:09 2020
From: kaushikaakash7539 at gmail.com (Aakash kaushik)
Date: Mon, 17 Feb 2020 23:21:09 +0530
Subject: [SciPy-Dev] contributing to Scipy
Message-ID: <CAKum75C=agRFHv5Br6sWoyEGrenFf-Kxd9wnE=0xvhn5LhiBVA@mail.gmail.com>

hey everyone my name is Aakash Kaushik,
i am new here and also to the open source world.
i have used Scipy a lot in my projects and wanted to contribute to it i
know python with beginner level and have background in machine learning, is
there any issues i can contribute with?
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scipy-dev/attachments/20200217/03412891/attachment.html>

From matti.picus at gmail.com  Mon Feb 17 12:59:50 2020
From: matti.picus at gmail.com (Matti Picus)
Date: Mon, 17 Feb 2020 19:59:50 +0200
Subject: [SciPy-Dev] contributing to Scipy
In-Reply-To: <CAKum75C=agRFHv5Br6sWoyEGrenFf-Kxd9wnE=0xvhn5LhiBVA@mail.gmail.com>
References: <CAKum75C=agRFHv5Br6sWoyEGrenFf-Kxd9wnE=0xvhn5LhiBVA@mail.gmail.com>
Message-ID: <44d970e9-18ac-f976-2551-044801268468@gmail.com>

On 17/2/20 7:51 pm, Aakash kaushik wrote:
> hey everyone my name is Aakash Kaushik,
> i am new here and also to the open source world.
> i have used Scipy a lot in my projects and wanted to contribute to it 
> i know python with beginner level and have background in machine 
> learning, is there any issues i can contribute with?
>

Welcome. You will want to read about contributing 
https://docs.scipy.org/doc/scipy/reference/hacking.html

If you wish to contribute code,these are the issues marked "good first 
issue" 
https://github.com/scipy/scipy/issues?q=is%3Aopen+is%3Aissue+label%3A%22good+first+issue%22. 


There are many other ways to contribute: documentation, triaging issues, 
and more. Take a look around the links and let us know if something is 
not clear or if you need help getting started.

Matti


From sseibert at anaconda.com  Mon Feb 17 13:20:30 2020
From: sseibert at anaconda.com (Stanley Seibert)
Date: Mon, 17 Feb 2020 12:20:30 -0600
Subject: [SciPy-Dev] Discontinued Rackspace Open Compute Discount
In-Reply-To: <CAHPuU_a4897sotNHgSiWzh+0JeOsvWxfA-WngD6z5ozywP6Jcw@mail.gmail.com>
References: <470871448.2376.1581182952898@wamui-napoleon.atl.sa.earthlink.net>
 <CAFvE7K43Uz2deJy81r_jxXT-8=qYYcpcjssU3TFrB4AUSuuvnw@mail.gmail.com>
 <CAFvE7K4vCb72dE2=-3Wy1bfKCCax7g8JxEP8qWSGi-exm_mn=A@mail.gmail.com>
 <CA+WonSRKO+XWtffy3YC80wpCZS7Z6cnbSPp=gy4NmuF6ucDn8Q@mail.gmail.com>
 <CAH6Pt5ooyWb+jwqM2OjVVnf8noox0PO7DxY74x_47h9nJowq8g@mail.gmail.com>
 <CA+WonSRZ1prkf6MbSoHYBFkOR_5J9Fv4jcW34G456MYJKterxw@mail.gmail.com>
 <CAFvE7K58Hhc68MUdF90DF+w1ACDw2MCxESmDfCDs3JvdRiB41Q@mail.gmail.com>
 <CAFvE7K5+M5COuuiSHHajNiVJL_omAX71VcUFfk88A+A-KmdikQ@mail.gmail.com>
 <CAFvE7K7bmoPZcRQrEqt11mE=X0FgqAUYmhQbj0gvcM5QL=i4GQ@mail.gmail.com>
 <CAE1aY-kjnNtsUych1kDMSt2hT5qk7K6e4gOuQnUY-LSy0otbjg@mail.gmail.com>
 <CAJR3sXebe8bz+C1YuW5i7eS1bq3DjN9Bj3AByakmuM1FZcLS0w@mail.gmail.com>
 <CAFvE7K4FbOU0vMdejQHo7Qr9BR6FKRX17tCQXe7aRG6Hc=DQTQ@mail.gmail.com>
 <CAHPuU_a4897sotNHgSiWzh+0JeOsvWxfA-WngD6z5ozywP6Jcw@mail.gmail.com>
Message-ID: <CADv3RKRreR2X-bVt1=0_tn9gWUg9aRd6B0_0F7AVsBB3i+GACA@mail.gmail.com>

Quick followup on this: I spoke to the site manager for Anaconda.org and
they raised the storage limit for the scipy-wheels-nightly org to 50GB, so
I think you are set for a while.

On Sun, Feb 16, 2020 at 11:03 PM Tyler Reddy <tyler.je.reddy at gmail.com>
wrote:

> Following Olivier's lead I've observed our first automatic upload to
> anaconda.org from scipy-wheels from t his testing PR:
> https://github.com/MacPython/scipy-wheels/pull/69
>
> I've restored the full build matrix & the cron requirement after
> confirming the upload worked to the target location using a secret key:
> https://anaconda.org/scipy-wheels-nightly/scipy/files
>
> I'll probably address the staging area for SciPy release wheels in a
> separate PR from this cron job for weekly uploads. Since our master branch
> wheels builds typically have some failures in the matrix for unrelated
> reasons it is complicated enough as it is.
>
> I did notice a 3.0 GB "free limit" I think---hopefully that doesn't become
> an issue.
>
>
> On Sat, 15 Feb 2020 at 03:11, Olivier Grisel <olivier.grisel at ensta.org>
> wrote:
>
>> Tyler, Tom: I added you as co-owners of the scipy-wheels-nightly
>> organization.
>> Gregory, Tyler, Tom: I added you as co-owners of the
>> multibuild-wheels-staging organization.
>>
>> --
>> Olivier
>> _______________________________________________
>> SciPy-Dev mailing list
>> SciPy-Dev at python.org
>> https://mail.python.org/mailman/listinfo/scipy-dev
>>
> _______________________________________________
> SciPy-Dev mailing list
> SciPy-Dev at python.org
> https://mail.python.org/mailman/listinfo/scipy-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scipy-dev/attachments/20200217/950de858/attachment.html>

From melissawm at gmail.com  Tue Feb 18 10:37:47 2020
From: melissawm at gmail.com (=?UTF-8?Q?Melissa_Mendon=C3=A7a?=)
Date: Tue, 18 Feb 2020 12:37:47 -0300
Subject: [SciPy-Dev] scipy.linalg.svd for n-dimensional arrays in stacked
 format
In-Reply-To: <CAC7J6VYEOn227nh9eFVsGbmaJcGtA3cpYzeDuesHNAnNf2zLyw@mail.gmail.com>
References: <CAC7J6VYEOn227nh9eFVsGbmaJcGtA3cpYzeDuesHNAnNf2zLyw@mail.gmail.com>
Message-ID: <CAC7J6VYXPT437De536OE6YKb887Pru9aG=W6qkQ0aSmWmGTO_g@mail.gmail.com>

Hello all,

I just came across this today. While we generally encourage users to use
scipy.linalg instead of numpy.linalg, I realized that numpy.linalg.svd
works for general n-dimensional arrays in "stacked format" (see [1, 2])
while scipy.linalg.svd does not.

Example:
=========================
>>> import numpy as np
>>> from scipy.linalg import svd
>>> A = np.random.rand(2,3,4)
>>> U, s, Vt = np.linalg.svd(A) # works fine
>>> W, sigma, Yt = svd(A)


---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
<ipython-input-6-fc853a2e096f> in <module>
----> 1 W, sigma, Yt = svd(A)

/opt/miniconda/envs/normal/lib/python3.7/site-packages/scipy/linalg/decomp_svd.py
in svd(a, full_matrices, compute_uv, overwrite_a, check_finite,
lapack_driver)
    109     a1 = _asarray_validated(a, check_finite=check_finite)
    110     if len(a1.shape) != 2:
--> 111         raise ValueError('expected matrix')
    112     m, n = a1.shape
    113     overwrite_a = overwrite_a or (_datacopied(a1, a))

ValueError: expected matrix
=========================

My question is if this is a design decision or a missing feature. If this
is an interesting funcionality that should be added to SciPy, I can look
into it.

Thanks,

-- 
Melissa Weber Mendon?a

[1] In the docs for numpy.linalg.svd:  "If `a` has more than two
dimensions, then broadcasting rules apply, as explained in
:ref:`routines.linalg-broadcasting`. This means that SVD is working in
"stacked" mode: it iterates over all indices of the first ``a.ndim - 2``
dimensions and for each combination SVD is applied to the last two indices.
The matrix `a` can be reconstructed from the decomposition with either ``(u
* s[..., None, :]) @ vh`` or ``u @ (s[..., None] * vh)``. (The ``@``
operator can be replaced by the function ``np.matmul`` for python versions
below 3.5.)
[2] https://numpy.org/devdocs/reference/generated/numpy.linalg.svd.html
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scipy-dev/attachments/20200218/2674cd86/attachment-0001.html>

From ilhanpolat at gmail.com  Tue Feb 18 16:00:02 2020
From: ilhanpolat at gmail.com (Ilhan Polat)
Date: Tue, 18 Feb 2020 22:00:02 +0100
Subject: [SciPy-Dev] scipy.linalg.svd for n-dimensional arrays in
 stacked format
In-Reply-To: <CAC7J6VYXPT437De536OE6YKb887Pru9aG=W6qkQ0aSmWmGTO_g@mail.gmail.com>
References: <CAC7J6VYEOn227nh9eFVsGbmaJcGtA3cpYzeDuesHNAnNf2zLyw@mail.gmail.com>
 <CAC7J6VYXPT437De536OE6YKb887Pru9aG=W6qkQ0aSmWmGTO_g@mail.gmail.com>
Message-ID: <CAEBuzr9khbey8U2YCbqem5jq1vQvK17EO-fy9Kttoe-OWqH5Dg@mail.gmail.com>

Hi Melissa,

Yes this is a much-sought after feature not only just SVD but for all
solve-alikes and eig-alikes. I was actually waiting for that magical moment
that I would have some time to work on it but kept on procrastinating. So
by all means, go for it, if you feel like it.

Best
ilhan



On Tue, Feb 18, 2020 at 4:39 PM Melissa Mendon?a <melissawm at gmail.com>
wrote:

> Hello all,
>
> I just came across this today. While we generally encourage users to use
> scipy.linalg instead of numpy.linalg, I realized that numpy.linalg.svd
> works for general n-dimensional arrays in "stacked format" (see [1, 2])
> while scipy.linalg.svd does not.
>
> Example:
> =========================
> >>> import numpy as np
> >>> from scipy.linalg import svd
> >>> A = np.random.rand(2,3,4)
> >>> U, s, Vt = np.linalg.svd(A) # works fine
> >>> W, sigma, Yt = svd(A)
>
>
> ---------------------------------------------------------------------------
> ValueError                                Traceback (most recent call last)
> <ipython-input-6-fc853a2e096f> in <module>
> ----> 1 W, sigma, Yt = svd(A)
>
> /opt/miniconda/envs/normal/lib/python3.7/site-packages/scipy/linalg/decomp_svd.py
> in svd(a, full_matrices, compute_uv, overwrite_a, check_finite,
> lapack_driver)
>     109     a1 = _asarray_validated(a, check_finite=check_finite)
>     110     if len(a1.shape) != 2:
> --> 111         raise ValueError('expected matrix')
>     112     m, n = a1.shape
>     113     overwrite_a = overwrite_a or (_datacopied(a1, a))
>
> ValueError: expected matrix
> =========================
>
> My question is if this is a design decision or a missing feature. If this
> is an interesting funcionality that should be added to SciPy, I can look
> into it.
>
> Thanks,
>
> --
> Melissa Weber Mendon?a
>
> [1] In the docs for numpy.linalg.svd:  "If `a` has more than two
> dimensions, then broadcasting rules apply, as explained in
> :ref:`routines.linalg-broadcasting`. This means that SVD is working in
> "stacked" mode: it iterates over all indices of the first ``a.ndim - 2``
> dimensions and for each combination SVD is applied to the last two indices.
> The matrix `a` can be reconstructed from the decomposition with either ``(u
> * s[..., None, :]) @ vh`` or ``u @ (s[..., None] * vh)``. (The ``@``
> operator can be replaced by the function ``np.matmul`` for python versions
> below 3.5.)
> [2] https://numpy.org/devdocs/reference/generated/numpy.linalg.svd.html
>
> _______________________________________________
> SciPy-Dev mailing list
> SciPy-Dev at python.org
> https://mail.python.org/mailman/listinfo/scipy-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scipy-dev/attachments/20200218/959e7c8e/attachment.html>

From ralf.gommers at gmail.com  Wed Feb 19 01:17:52 2020
From: ralf.gommers at gmail.com (Ralf Gommers)
Date: Tue, 18 Feb 2020 22:17:52 -0800
Subject: [SciPy-Dev] Discontinued Rackspace Open Compute Discount
In-Reply-To: <CADv3RKRreR2X-bVt1=0_tn9gWUg9aRd6B0_0F7AVsBB3i+GACA@mail.gmail.com>
References: <470871448.2376.1581182952898@wamui-napoleon.atl.sa.earthlink.net>
 <CAFvE7K43Uz2deJy81r_jxXT-8=qYYcpcjssU3TFrB4AUSuuvnw@mail.gmail.com>
 <CAFvE7K4vCb72dE2=-3Wy1bfKCCax7g8JxEP8qWSGi-exm_mn=A@mail.gmail.com>
 <CA+WonSRKO+XWtffy3YC80wpCZS7Z6cnbSPp=gy4NmuF6ucDn8Q@mail.gmail.com>
 <CAH6Pt5ooyWb+jwqM2OjVVnf8noox0PO7DxY74x_47h9nJowq8g@mail.gmail.com>
 <CA+WonSRZ1prkf6MbSoHYBFkOR_5J9Fv4jcW34G456MYJKterxw@mail.gmail.com>
 <CAFvE7K58Hhc68MUdF90DF+w1ACDw2MCxESmDfCDs3JvdRiB41Q@mail.gmail.com>
 <CAFvE7K5+M5COuuiSHHajNiVJL_omAX71VcUFfk88A+A-KmdikQ@mail.gmail.com>
 <CAFvE7K7bmoPZcRQrEqt11mE=X0FgqAUYmhQbj0gvcM5QL=i4GQ@mail.gmail.com>
 <CAE1aY-kjnNtsUych1kDMSt2hT5qk7K6e4gOuQnUY-LSy0otbjg@mail.gmail.com>
 <CAJR3sXebe8bz+C1YuW5i7eS1bq3DjN9Bj3AByakmuM1FZcLS0w@mail.gmail.com>
 <CAFvE7K4FbOU0vMdejQHo7Qr9BR6FKRX17tCQXe7aRG6Hc=DQTQ@mail.gmail.com>
 <CAHPuU_a4897sotNHgSiWzh+0JeOsvWxfA-WngD6z5ozywP6Jcw@mail.gmail.com>
 <CADv3RKRreR2X-bVt1=0_tn9gWUg9aRd6B0_0F7AVsBB3i+GACA@mail.gmail.com>
Message-ID: <CABL7CQjP8mMoGP+dqMte5piJUsWcfGAfHq_ze_Z2YYnPKbN80g@mail.gmail.com>

On Mon, Feb 17, 2020 at 10:20 AM Stanley Seibert <sseibert at anaconda.com>
wrote:

> Quick followup on this: I spoke to the site manager for Anaconda.org and
> they raised the storage limit for the scipy-wheels-nightly org to 50GB, so
> I think you are set for a while.
>

Thanks Stan!

And thanks Olivier, Tyler and everyone else who is helping to resolve this!

Ralf


> On Sun, Feb 16, 2020 at 11:03 PM Tyler Reddy <tyler.je.reddy at gmail.com>
> wrote:
>
>> Following Olivier's lead I've observed our first automatic upload to
>> anaconda.org from scipy-wheels from t his testing PR:
>> https://github.com/MacPython/scipy-wheels/pull/69
>>
>> I've restored the full build matrix & the cron requirement after
>> confirming the upload worked to the target location using a secret key:
>> https://anaconda.org/scipy-wheels-nightly/scipy/files
>>
>> I'll probably address the staging area for SciPy release wheels in a
>> separate PR from this cron job for weekly uploads. Since our master branch
>> wheels builds typically have some failures in the matrix for unrelated
>> reasons it is complicated enough as it is.
>>
>> I did notice a 3.0 GB "free limit" I think---hopefully that doesn't
>> become an issue.
>>
>>
>> On Sat, 15 Feb 2020 at 03:11, Olivier Grisel <olivier.grisel at ensta.org>
>> wrote:
>>
>>> Tyler, Tom: I added you as co-owners of the scipy-wheels-nightly
>>> organization.
>>> Gregory, Tyler, Tom: I added you as co-owners of the
>>> multibuild-wheels-staging organization.
>>>
>>> --
>>> Olivier
>>> _______________________________________________
>>> SciPy-Dev mailing list
>>> SciPy-Dev at python.org
>>> https://mail.python.org/mailman/listinfo/scipy-dev
>>>
>> _______________________________________________
>> SciPy-Dev mailing list
>> SciPy-Dev at python.org
>> https://mail.python.org/mailman/listinfo/scipy-dev
>>
> _______________________________________________
> SciPy-Dev mailing list
> SciPy-Dev at python.org
> https://mail.python.org/mailman/listinfo/scipy-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scipy-dev/attachments/20200218/6976b871/attachment.html>

From smishra99.iitkgp at gmail.com  Thu Feb 20 11:52:49 2020
From: smishra99.iitkgp at gmail.com (Shubham Mishra)
Date: Thu, 20 Feb 2020 22:22:49 +0530
Subject: [SciPy-Dev] [Newbie] Information regarding GSoC
In-Reply-To: <CAPhLvQMgZWsaQEAH9Dc7W4sBp8N6x4muVE-2RKbLh81QnPWH7A@mail.gmail.com>
References: <CAPhLvQMkh8B7hq7Ho4o1+my_cpDs7askueykzjNzRm8-LmFefA@mail.gmail.com>
 <CABL7CQh_c1rcRVKstW2Au+jpn5nniLLwPC5Mu-sOj9EKymK_Ng@mail.gmail.com>
 <CAPhLvQMgZWsaQEAH9Dc7W4sBp8N6x4muVE-2RKbLh81QnPWH7A@mail.gmail.com>
Message-ID: <CAPhLvQOqzMBNUQDRNKxQPZZp9_byKRz0WZ8LRP+5wkaHXTYYoA@mail.gmail.com>

Hello,
This is a follow up on the above mail.
According to GSoC timelines, the date of release of organizations is
tomorrow.
Please let me know if Scipy is really participating.
I'll be contributing anyway, however.

Warm regards,
Shubham Mishra

On Mon, Feb 17, 2020 at 1:57 PM Shubham Mishra <smishra99.iitkgp at gmail.com>
wrote:

> Hello all.
> Thanks for hearing me out regarding GSoC. I am interested in the
> differential equation solvers as put forward by Nicolas. Should I start
> preparing for the same?
> I have done a basic freshman level math course where ODEs were taught, but
> not up to that level. Can someone suggest some reading materials (books,
> papers, tutorials... anything) from where I can learn?
> I am really eager to learn and contribute in a meaningful way.
>
> Yours sincerely,
> Shubham Mishra (grapheo12)
>
> On Sat, 15 Feb 2020, 23:41 Ralf Gommers, <ralf.gommers at gmail.com> wrote:
>
>>
>>
>> On Sat, Feb 15, 2020 at 12:33 AM Shubham Mishra <
>> smishra99.iitkgp at gmail.com> wrote:
>>
>>> Hello,
>>> Newbie here. Is Scipy participating in GSoC this year?
>>> I am curious to work under Scipy for GSoC if it is possible. Can someone
>>> mentor me regarding this?
>>>
>>
>> Hi Shubham, thanks for asking. We haven't discussed it yet, now would be
>> a good time to do that - I'll open a separate thread for it.
>>
>> Cheers,
>> Ralf
>>
>>
>>
>>> Thanking you,
>>> Yours sincerely,
>>> Shubham Mishra
>>> (Github handle: grapheo12)
>>> _______________________________________________
>>> SciPy-Dev mailing list
>>> SciPy-Dev at python.org
>>> https://mail.python.org/mailman/listinfo/scipy-dev
>>>
>> _______________________________________________
>> SciPy-Dev mailing list
>> SciPy-Dev at python.org
>> https://mail.python.org/mailman/listinfo/scipy-dev
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scipy-dev/attachments/20200220/007c1872/attachment.html>

From asaadel1 at jhu.edu  Thu Feb 20 12:37:26 2020
From: asaadel1 at jhu.edu (Ali Saad-Eldin)
Date: Thu, 20 Feb 2020 17:37:26 +0000
Subject: [SciPy-Dev] ENH: Implement a Quadratic Assignment Problem Solver
Message-ID: <BL0PR01MB52174C285DE3E56B70B539A499130@BL0PR01MB5217.prod.exchangelabs.com>

Hello!

I would like to add a Quadratic Approximation Problem (QAP)<https://en.wikipedia.org/wiki/Quadratic_assignment_problem> solver function, by implementing the Fast Approximate QAP (FAQ) algorithm<https://journals.plos.org/plosone/article/file?id=10.1371/journal.pone.0121002&type=printable> (Vogelstein, 2015). QAP is a combinatorial optimization problem, and the FAQ algorithm can be applied to solve special cases of QAP, including the Graph Matching Problem (GMP) and the Traveling Salesman Problem (TSP).

Since the QAP is a combinatorial optimization problem, I'd like to have the FAQ implementation exposed through scipy.optimize. The module would accept parameters such as permutation initialization type (single barycenter initialization, or several initializations "close" to the barycenter), maximum Frank-Wolfe iterations, and whether you would like to solve a special case of the QAP (such as GMP). The module would fit with the two cost matrices in the objective function, returning the score (minimized objection function value) and indices of the optimal permutation matrix from the objective function. The implementation will also give the option to include seeds

I have already implemented FAQ in GraSPy<https://github.com/neurodata/graspy/blob/master/graspy/match/faq.py>, and proof of effectiveness can be found here<https://graspy.neurodata.io/tutorials/matching/faq.html> .

Best,
Ali Saad-Eldin

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scipy-dev/attachments/20200220/87190f01/attachment.html>

From olivetti at fbk.eu  Fri Feb 21 05:02:34 2020
From: olivetti at fbk.eu (Emanuele Olivetti)
Date: Fri, 21 Feb 2020 11:02:34 +0100
Subject: [SciPy-Dev] ENH: Implement a Quadratic Assignment Problem Solver
In-Reply-To: <BL0PR01MB52174C285DE3E56B70B539A499130@BL0PR01MB5217.prod.exchangelabs.com>
References: <BL0PR01MB52174C285DE3E56B70B539A499130@BL0PR01MB5217.prod.exchangelabs.com>
Message-ID: <CAK9Wt6Dcr5ge1q1JtSe01JTa14O3s+S0ks7OizO4DXJsewnDNQ@mail.gmail.com>

Dear Ali and SciPy,

Some time ago, I implemented another algorithm for the approximate solution
of the QAP / Graph Matching: the Doubly Stochastic Projected Fixed-Point
(DSPFP) algorithm proposed in Lu et al. (2016) [*]. Input and output are
basically the same as the FAQ algorithm of Ali. If there is interest in
adding approximate QAP / Graph Matching algorithms to SciPy, I'd happy to
contribute with it:
  https://github.com/emanuele/DSPFP

Best,

Emanuele

[*] : http://dx.doi.org/10.1016/j.patcog.2016.07.015
Pre-print about the same manuscript and algorithm (named fastPFP at that
time, 2012) here: https://arxiv.org/abs/1207.1114

On Thu, Feb 20, 2020 at 6:38 PM Ali Saad-Eldin <asaadel1 at jhu.edu> wrote:

> Hello!
>
> I would like to add a Quadratic Approximation Problem (QAP)
> <https://en.wikipedia.org/wiki/Quadratic_assignment_problem> solver
> function, by implementing the Fast Approximate QAP (FAQ) algorithm
> <https://journals.plos.org/plosone/article/file?id=10.1371/journal.pone.0121002&type=printable> (Vogelstein,
> 2015). QAP is a combinatorial optimization problem, and the FAQ algorithm
> can be applied to solve special cases of QAP, including the Graph Matching
> Problem (GMP) and the Traveling Salesman Problem (TSP).
>
> Since the QAP is a combinatorial optimization problem, I'd like to have
> the FAQ implementation exposed through scipy.optimize. The module would
> accept parameters such as permutation initialization type (single
> barycenter initialization, or several initializations "close" to the
> barycenter), maximum Frank-Wolfe iterations, and whether you would like to
> solve a special case of the QAP (such as GMP). The module would fit with
> the two cost matrices in the objective function, returning the score
> (minimized objection function value) and indices of the optimal permutation
> matrix from the objective function. The implementation will also give the
> option to include seeds
>
> I have already implemented FAQ in GraSPy
> <https://github.com/neurodata/graspy/blob/master/graspy/match/faq.py>,
> and proof of effectiveness can be found here
> <https://graspy.neurodata.io/tutorials/matching/faq.html> .
>
> Best,
> Ali Saad-Eldin
>
> _______________________________________________
> SciPy-Dev mailing list
> SciPy-Dev at python.org
> https://mail.python.org/mailman/listinfo/scipy-dev
>

-- 
--
Le informazioni contenute nella presente comunicazione sono di natura?
privata e come tali sono da considerarsi riservate ed indirizzate?
esclusivamente ai destinatari indicati e per le finalit? strettamente?
legate al relativo contenuto. Se avete ricevuto questo messaggio per?
errore, vi preghiamo di eliminarlo e di inviare una comunicazione?
all?indirizzo e-mail del mittente.

--
The information transmitted is 
intended only for the person or entity to which it is addressed and may 
contain confidential and/or privileged material. If you received this in 
error, please contact the sender and delete the material.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scipy-dev/attachments/20200221/3e893343/attachment-0001.html>

From ralf.gommers at gmail.com  Fri Feb 21 07:13:35 2020
From: ralf.gommers at gmail.com (Ralf Gommers)
Date: Fri, 21 Feb 2020 04:13:35 -0800
Subject: [SciPy-Dev] ENH: Implement a Quadratic Assignment Problem Solver
In-Reply-To: <CAK9Wt6Dcr5ge1q1JtSe01JTa14O3s+S0ks7OizO4DXJsewnDNQ@mail.gmail.com>
References: <BL0PR01MB52174C285DE3E56B70B539A499130@BL0PR01MB5217.prod.exchangelabs.com>
 <CAK9Wt6Dcr5ge1q1JtSe01JTa14O3s+S0ks7OizO4DXJsewnDNQ@mail.gmail.com>
Message-ID: <CABL7CQi8v=W4j9wVNPRMahuiBL1xbQEmSCgG3dtGkFuer=DwXg@mail.gmail.com>

On Fri, Feb 21, 2020 at 2:03 AM Emanuele Olivetti <olivetti at fbk.eu> wrote:

> Dear Ali and SciPy,
>
> Some time ago, I implemented another algorithm for the approximate
> solution of the QAP / Graph Matching: the Doubly Stochastic Projected
> Fixed-Point (DSPFP) algorithm proposed in Lu et al. (2016) [*]. Input and
> output are basically the same as the FAQ algorithm of Ali. If there is
> interest in adding approximate QAP / Graph Matching algorithms to SciPy,
> I'd happy to contribute with it:
>   https://github.com/emanuele/DSPFP
>
> Best,
>
> Emanuele
>
> [*] : http://dx.doi.org/10.1016/j.patcog.2016.07.015
> Pre-print about the same manuscript and algorithm (named fastPFP at that
> time, 2012) here: https://arxiv.org/abs/1207.1114
>
> On Thu, Feb 20, 2020 at 6:38 PM Ali Saad-Eldin <asaadel1 at jhu.edu> wrote:
>
>> Hello!
>>
>> I would like to add a Quadratic Approximation Problem (QAP)
>> <https://en.wikipedia.org/wiki/Quadratic_assignment_problem> solver
>> function, by implementing the Fast Approximate QAP (FAQ) algorithm
>> <https://journals.plos.org/plosone/article/file?id=10.1371/journal.pone.0121002&type=printable> (Vogelstein,
>> 2015). QAP is a combinatorial optimization problem, and the FAQ algorithm
>> can be applied to solve special cases of QAP, including the Graph Matching
>> Problem (GMP) and the Traveling Salesman Problem (TSP).
>>
>> Since the QAP is a combinatorial optimization problem, I'd like to have
>> the FAQ implementation exposed through scipy.optimize. The module would
>> accept parameters such as permutation initialization type (single
>> barycenter initialization, or several initializations "close" to the
>> barycenter), maximum Frank-Wolfe iterations, and whether you would like to
>> solve a special case of the QAP (such as GMP). The module would fit with
>> the two cost matrices in the objective function, returning the score
>> (minimized objection function value) and indices of the optimal permutation
>> matrix from the objective function. The implementation will also give the
>> option to include seeds
>>
>
Since the inputs are graphs and the functionality resembles things that
live in scipy.sparse.csgraph, I'm wondering whether this either should go
into sparse.csgraph or if it's better in scipy.optimize then should it be
able to understand sparse inputs?

Cheers,
Ralf


>> I have already implemented FAQ in GraSPy
>> <https://github.com/neurodata/graspy/blob/master/graspy/match/faq.py>,
>> and proof of effectiveness can be found here
>> <https://graspy.neurodata.io/tutorials/matching/faq.html> .
>>
>> Best,
>> Ali Saad-Eldin
>>
>> _______________________________________________
>> SciPy-Dev mailing list
>> SciPy-Dev at python.org
>> https://mail.python.org/mailman/listinfo/scipy-dev
>>
>
> --
> Le informazioni contenute nella presente comunicazione sono di natura privata
> e come tali sono da considerarsi riservate ed indirizzate esclusivamente
> ai destinatari indicati e per le finalit? strettamente legate al relativo
> contenuto. Se avete ricevuto questo messaggio per errore, vi preghiamo di
> eliminarlo e di inviare una comunicazione all?indirizzo e-mail del
> mittente.
> --
> The information transmitted is intended only for the person or entity to
> which it is addressed and may contain confidential and/or privileged
> material. If you received this in error, please contact the sender and
> delete the material.
> _______________________________________________
> SciPy-Dev mailing list
> SciPy-Dev at python.org
> https://mail.python.org/mailman/listinfo/scipy-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scipy-dev/attachments/20200221/a02a2452/attachment.html>

From kaistriega at gmail.com  Fri Feb 21 08:26:44 2020
From: kaistriega at gmail.com (Kai Striega)
Date: Fri, 21 Feb 2020 21:26:44 +0800
Subject: [SciPy-Dev] ENH: Implement a Quadratic Assignment Problem Solver
In-Reply-To: <CABL7CQi8v=W4j9wVNPRMahuiBL1xbQEmSCgG3dtGkFuer=DwXg@mail.gmail.com>
References: <BL0PR01MB52174C285DE3E56B70B539A499130@BL0PR01MB5217.prod.exchangelabs.com>
 <CAK9Wt6Dcr5ge1q1JtSe01JTa14O3s+S0ks7OizO4DXJsewnDNQ@mail.gmail.com>
 <CABL7CQi8v=W4j9wVNPRMahuiBL1xbQEmSCgG3dtGkFuer=DwXg@mail.gmail.com>
Message-ID: <CAPDXd1401M4Nq5RQBc+mA9bseRuJxSn-EdVNVgwrcMHWjQ0Mog@mail.gmail.com>

Hi All,

Since the inputs are graphs and the functionality resembles things that
> live in scipy.sparse.csgraph, I'm wondering whether this either should go
> into sparse.csgraph or if it's better in scipy.optimize then should it be
> able to understand sparse inputs?
>

I'd say a QAP solver fits into scipy.optimize better than
scipy.sparse.csgraph. We already have scipy.optimize.linear_sum_assignment
<https://docs.scipy.org/doc/scipy/reference/generated/scipy.optimize.linear_sum_assignment.html#scipy.optimize.linear_sum_assignment>
and an *"assignment problems"* category, although right, now it's a
subcategory of linear programming. I also vaguely remember talk of a
general quadratic programming solver to follow the linear programming
solver, although I don't think there has been any progress there. Further,
a QAP solver can be formulated as a more general quadratic programming
problem, while the algorithms in csgraph all look like "classic" graph
theory algorithms. Saying that, I haven't worked with csgraph much, so I
may be wrong regarding its scope.

Just adding my 2c

Kai

On Fri, 21 Feb 2020 at 20:14, Ralf Gommers <ralf.gommers at gmail.com> wrote:

>
>
> On Fri, Feb 21, 2020 at 2:03 AM Emanuele Olivetti <olivetti at fbk.eu> wrote:
>
>> Dear Ali and SciPy,
>>
>> Some time ago, I implemented another algorithm for the approximate
>> solution of the QAP / Graph Matching: the Doubly Stochastic Projected
>> Fixed-Point (DSPFP) algorithm proposed in Lu et al. (2016) [*]. Input and
>> output are basically the same as the FAQ algorithm of Ali. If there is
>> interest in adding approximate QAP / Graph Matching algorithms to SciPy,
>> I'd happy to contribute with it:
>>   https://github.com/emanuele/DSPFP
>>
>> Best,
>>
>> Emanuele
>>
>> [*] : http://dx.doi.org/10.1016/j.patcog.2016.07.015
>> Pre-print about the same manuscript and algorithm (named fastPFP at that
>> time, 2012) here: https://arxiv.org/abs/1207.1114
>>
>> On Thu, Feb 20, 2020 at 6:38 PM Ali Saad-Eldin <asaadel1 at jhu.edu> wrote:
>>
>>> Hello!
>>>
>>> I would like to add a Quadratic Approximation Problem (QAP)
>>> <https://en.wikipedia.org/wiki/Quadratic_assignment_problem> solver
>>> function, by implementing the Fast Approximate QAP (FAQ) algorithm
>>> <https://journals.plos.org/plosone/article/file?id=10.1371/journal.pone.0121002&type=printable> (Vogelstein,
>>> 2015). QAP is a combinatorial optimization problem, and the FAQ algorithm
>>> can be applied to solve special cases of QAP, including the Graph Matching
>>> Problem (GMP) and the Traveling Salesman Problem (TSP).
>>>
>>> Since the QAP is a combinatorial optimization problem, I'd like to have
>>> the FAQ implementation exposed through scipy.optimize. The module would
>>> accept parameters such as permutation initialization type (single
>>> barycenter initialization, or several initializations "close" to the
>>> barycenter), maximum Frank-Wolfe iterations, and whether you would like to
>>> solve a special case of the QAP (such as GMP). The module would fit with
>>> the two cost matrices in the objective function, returning the score
>>> (minimized objection function value) and indices of the optimal permutation
>>> matrix from the objective function. The implementation will also give the
>>> option to include seeds
>>>
>>
> Since the inputs are graphs and the functionality resembles things that
> live in scipy.sparse.csgraph, I'm wondering whether this either should go
> into sparse.csgraph or if it's better in scipy.optimize then should it be
> able to understand sparse inputs?
>
> Cheers,
> Ralf
>
>
>>> I have already implemented FAQ in GraSPy
>>> <https://github.com/neurodata/graspy/blob/master/graspy/match/faq.py>,
>>> and proof of effectiveness can be found here
>>> <https://graspy.neurodata.io/tutorials/matching/faq.html> .
>>>
>>> Best,
>>> Ali Saad-Eldin
>>>
>>> _______________________________________________
>>> SciPy-Dev mailing list
>>> SciPy-Dev at python.org
>>> https://mail.python.org/mailman/listinfo/scipy-dev
>>>
>>
>> --
>> Le informazioni contenute nella presente comunicazione sono di natura privata
>> e come tali sono da considerarsi riservate ed indirizzate esclusivamente
>> ai destinatari indicati e per le finalit? strettamente legate al
>> relativo contenuto. Se avete ricevuto questo messaggio per errore, vi
>> preghiamo di eliminarlo e di inviare una comunicazione all?indirizzo
>> e-mail del mittente.
>> --
>> The information transmitted is intended only for the person or entity to
>> which it is addressed and may contain confidential and/or privileged
>> material. If you received this in error, please contact the sender and
>> delete the material.
>> _______________________________________________
>> SciPy-Dev mailing list
>> SciPy-Dev at python.org
>> https://mail.python.org/mailman/listinfo/scipy-dev
>>
> _______________________________________________
> SciPy-Dev mailing list
> SciPy-Dev at python.org
> https://mail.python.org/mailman/listinfo/scipy-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scipy-dev/attachments/20200221/04fa0a9d/attachment-0001.html>

From mhaberla at calpoly.edu  Fri Feb 21 09:21:48 2020
From: mhaberla at calpoly.edu (Matt Haberland)
Date: Fri, 21 Feb 2020 06:21:48 -0800
Subject: [SciPy-Dev] ENH: Implement a Quadratic Assignment Problem Solver
In-Reply-To: <CAPDXd1401M4Nq5RQBc+mA9bseRuJxSn-EdVNVgwrcMHWjQ0Mog@mail.gmail.com>
References: <BL0PR01MB52174C285DE3E56B70B539A499130@BL0PR01MB5217.prod.exchangelabs.com>
 <CAK9Wt6Dcr5ge1q1JtSe01JTa14O3s+S0ks7OizO4DXJsewnDNQ@mail.gmail.com>
 <CABL7CQi8v=W4j9wVNPRMahuiBL1xbQEmSCgG3dtGkFuer=DwXg@mail.gmail.com>
 <CAPDXd1401M4Nq5RQBc+mA9bseRuJxSn-EdVNVgwrcMHWjQ0Mog@mail.gmail.com>
Message-ID: <CADuxUixUitw7FinXUsaNTq7CxRjJn3C=-yctJDKhex2XQyJihg@mail.gmail.com>

I can't say where it would fit better, but it sounds like QAP would fit
well in optimize next to linear_sum_assignment now and the more general MIP
and QP solvers when they happen.

On Fri, Feb 21, 2020, 5:27 AM Kai Striega <kaistriega at gmail.com> wrote:

> Hi All,
>
> Since the inputs are graphs and the functionality resembles things that
>> live in scipy.sparse.csgraph, I'm wondering whether this either should go
>> into sparse.csgraph or if it's better in scipy.optimize then should it be
>> able to understand sparse inputs?
>>
>
> I'd say a QAP solver fits into scipy.optimize better than
> scipy.sparse.csgraph. We already have scipy.optimize.linear_sum_assignment
> <https://docs.scipy.org/doc/scipy/reference/generated/scipy.optimize.linear_sum_assignment.html#scipy.optimize.linear_sum_assignment>
> and an *"assignment problems"* category, although right, now it's a
> subcategory of linear programming. I also vaguely remember talk of a
> general quadratic programming solver to follow the linear programming
> solver, although I don't think there has been any progress there. Further,
> a QAP solver can be formulated as a more general quadratic programming
> problem, while the algorithms in csgraph all look like "classic" graph
> theory algorithms. Saying that, I haven't worked with csgraph much, so I
> may be wrong regarding its scope.
>
> Just adding my 2c
>
> Kai
>
> On Fri, 21 Feb 2020 at 20:14, Ralf Gommers <ralf.gommers at gmail.com> wrote:
>
>>
>>
>> On Fri, Feb 21, 2020 at 2:03 AM Emanuele Olivetti <olivetti at fbk.eu>
>> wrote:
>>
>>> Dear Ali and SciPy,
>>>
>>> Some time ago, I implemented another algorithm for the approximate
>>> solution of the QAP / Graph Matching: the Doubly Stochastic Projected
>>> Fixed-Point (DSPFP) algorithm proposed in Lu et al. (2016) [*]. Input and
>>> output are basically the same as the FAQ algorithm of Ali. If there is
>>> interest in adding approximate QAP / Graph Matching algorithms to SciPy,
>>> I'd happy to contribute with it:
>>>   https://github.com/emanuele/DSPFP
>>>
>>> Best,
>>>
>>> Emanuele
>>>
>>> [*] : http://dx.doi.org/10.1016/j.patcog.2016.07.015
>>> Pre-print about the same manuscript and algorithm (named fastPFP at that
>>> time, 2012) here: https://arxiv.org/abs/1207.1114
>>>
>>> On Thu, Feb 20, 2020 at 6:38 PM Ali Saad-Eldin <asaadel1 at jhu.edu> wrote:
>>>
>>>> Hello!
>>>>
>>>> I would like to add a Quadratic Approximation Problem (QAP)
>>>> <https://en.wikipedia.org/wiki/Quadratic_assignment_problem> solver
>>>> function, by implementing the Fast Approximate QAP (FAQ) algorithm
>>>> <https://journals.plos.org/plosone/article/file?id=10.1371/journal.pone.0121002&type=printable> (Vogelstein,
>>>> 2015). QAP is a combinatorial optimization problem, and the FAQ algorithm
>>>> can be applied to solve special cases of QAP, including the Graph Matching
>>>> Problem (GMP) and the Traveling Salesman Problem (TSP).
>>>>
>>>> Since the QAP is a combinatorial optimization problem, I'd like to have
>>>> the FAQ implementation exposed through scipy.optimize. The module would
>>>> accept parameters such as permutation initialization type (single
>>>> barycenter initialization, or several initializations "close" to the
>>>> barycenter), maximum Frank-Wolfe iterations, and whether you would like to
>>>> solve a special case of the QAP (such as GMP). The module would fit with
>>>> the two cost matrices in the objective function, returning the score
>>>> (minimized objection function value) and indices of the optimal permutation
>>>> matrix from the objective function. The implementation will also give the
>>>> option to include seeds
>>>>
>>>
>> Since the inputs are graphs and the functionality resembles things that
>> live in scipy.sparse.csgraph, I'm wondering whether this either should go
>> into sparse.csgraph or if it's better in scipy.optimize then should it be
>> able to understand sparse inputs?
>>
>> Cheers,
>> Ralf
>>
>>
>>>> I have already implemented FAQ in GraSPy
>>>> <https://github.com/neurodata/graspy/blob/master/graspy/match/faq.py>,
>>>> and proof of effectiveness can be found here
>>>> <https://graspy.neurodata.io/tutorials/matching/faq.html> .
>>>>
>>>> Best,
>>>> Ali Saad-Eldin
>>>>
>>>> _______________________________________________
>>>> SciPy-Dev mailing list
>>>> SciPy-Dev at python.org
>>>> https://mail.python.org/mailman/listinfo/scipy-dev
>>>>
>>>
>>> --
>>> Le informazioni contenute nella presente comunicazione sono di natura privata
>>> e come tali sono da considerarsi riservate ed indirizzate esclusivamente
>>> ai destinatari indicati e per le finalit? strettamente legate al
>>> relativo contenuto. Se avete ricevuto questo messaggio per errore, vi
>>> preghiamo di eliminarlo e di inviare una comunicazione all?indirizzo
>>> e-mail del mittente.
>>> --
>>> The information transmitted is intended only for the person or entity to
>>> which it is addressed and may contain confidential and/or privileged
>>> material. If you received this in error, please contact the sender and
>>> delete the material.
>>> _______________________________________________
>>> SciPy-Dev mailing list
>>> SciPy-Dev at python.org
>>> https://mail.python.org/mailman/listinfo/scipy-dev
>>>
>> _______________________________________________
>> SciPy-Dev mailing list
>> SciPy-Dev at python.org
>> https://mail.python.org/mailman/listinfo/scipy-dev
>>
> _______________________________________________
> SciPy-Dev mailing list
> SciPy-Dev at python.org
> https://mail.python.org/mailman/listinfo/scipy-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scipy-dev/attachments/20200221/582f98b5/attachment.html>

From scipy-dev at fuglede.dk  Fri Feb 21 13:43:35 2020
From: scipy-dev at fuglede.dk (=?iso-8859-1?Q?S=F8ren_Fuglede_J=F8rgensen?=)
Date: Fri, 21 Feb 2020 19:43:35 +0100
Subject: [SciPy-Dev] ENH: Implement a Quadratic Assignment Problem Solver
In-Reply-To: <CADuxUixUitw7FinXUsaNTq7CxRjJn3C=-yctJDKhex2XQyJihg@mail.gmail.com>
References: <BL0PR01MB52174C285DE3E56B70B539A499130@BL0PR01MB5217.prod.exchangelabs.com>
 <CAK9Wt6Dcr5ge1q1JtSe01JTa14O3s+S0ks7OizO4DXJsewnDNQ@mail.gmail.com>
 <CABL7CQi8v=W4j9wVNPRMahuiBL1xbQEmSCgG3dtGkFuer=DwXg@mail.gmail.com>
 <CAPDXd1401M4Nq5RQBc+mA9bseRuJxSn-EdVNVgwrcMHWjQ0Mog@mail.gmail.com>
 <CADuxUixUitw7FinXUsaNTq7CxRjJn3C=-yctJDKhex2XQyJihg@mail.gmail.com>
Message-ID: <20200221184335.gxo4s4lcaxse3jx2@sleipner.sleipner.fuglede.dk>

Hi all

Regarding where the implementation could live, let me also quickly note that in https://github.com/scipy/scipy/pull/10296#issuecomment-512850498, we had a bit of discussion on how a sparsity-friendly implementation of linear_sum_assignment could find a natural home in csgraph, perhaps as csgraph.minimum_weight_full_matching. (FWIW, I still haven't given up on solving the licensing issue mentioned in that thread, but let me use this discussion as an excuse to push for an update.) In that case, we would want to spell out the link between that algorithm and its scipy.optimize counterpart clearly in the documentation, to ensure discoverability. Similarly, in this case, if the QAP implementation ends up it csgraph, it would be useful to be able to find it through scipy.optimize somehow (would having a simple alias in scipy.optimize be overkill/confusing?)

S?ren

On Fri, Feb 21, 2020 at 06:21:48AM -0800, Matt Haberland wrote:
> I can't say where it would fit better, but it sounds like QAP would fit
> well in optimize next to linear_sum_assignment now and the more general MIP
> and QP solvers when they happen.
> 
> On Fri, Feb 21, 2020, 5:27 AM Kai Striega <kaistriega at gmail.com> wrote:
> 
> > Hi All,
> >
> > Since the inputs are graphs and the functionality resembles things that
> >> live in scipy.sparse.csgraph, I'm wondering whether this either should go
> >> into sparse.csgraph or if it's better in scipy.optimize then should it be
> >> able to understand sparse inputs?
> >>
> >
> > I'd say a QAP solver fits into scipy.optimize better than
> > scipy.sparse.csgraph. We already have scipy.optimize.linear_sum_assignment
> > <https://docs.scipy.org/doc/scipy/reference/generated/scipy.optimize.linear_sum_assignment.html#scipy.optimize.linear_sum_assignment>
> > and an *"assignment problems"* category, although right, now it's a
> > subcategory of linear programming. I also vaguely remember talk of a
> > general quadratic programming solver to follow the linear programming
> > solver, although I don't think there has been any progress there. Further,
> > a QAP solver can be formulated as a more general quadratic programming
> > problem, while the algorithms in csgraph all look like "classic" graph
> > theory algorithms. Saying that, I haven't worked with csgraph much, so I
> > may be wrong regarding its scope.
> >
> > Just adding my 2c
> >
> > Kai
> >
> > On Fri, 21 Feb 2020 at 20:14, Ralf Gommers <ralf.gommers at gmail.com> wrote:
> >
> >>
> >>
> >> On Fri, Feb 21, 2020 at 2:03 AM Emanuele Olivetti <olivetti at fbk.eu>
> >> wrote:
> >>
> >>> Dear Ali and SciPy,
> >>>
> >>> Some time ago, I implemented another algorithm for the approximate
> >>> solution of the QAP / Graph Matching: the Doubly Stochastic Projected
> >>> Fixed-Point (DSPFP) algorithm proposed in Lu et al. (2016) [*]. Input and
> >>> output are basically the same as the FAQ algorithm of Ali. If there is
> >>> interest in adding approximate QAP / Graph Matching algorithms to SciPy,
> >>> I'd happy to contribute with it:
> >>>   https://github.com/emanuele/DSPFP
> >>>
> >>> Best,
> >>>
> >>> Emanuele
> >>>
> >>> [*] : http://dx.doi.org/10.1016/j.patcog.2016.07.015
> >>> Pre-print about the same manuscript and algorithm (named fastPFP at that
> >>> time, 2012) here: https://arxiv.org/abs/1207.1114
> >>>
> >>> On Thu, Feb 20, 2020 at 6:38 PM Ali Saad-Eldin <asaadel1 at jhu.edu> wrote:
> >>>
> >>>> Hello!
> >>>>
> >>>> I would like to add a Quadratic Approximation Problem (QAP)
> >>>> <https://en.wikipedia.org/wiki/Quadratic_assignment_problem> solver
> >>>> function, by implementing the Fast Approximate QAP (FAQ) algorithm
> >>>> <https://journals.plos.org/plosone/article/file?id=10.1371/journal.pone.0121002&type=printable> (Vogelstein,
> >>>> 2015). QAP is a combinatorial optimization problem, and the FAQ algorithm
> >>>> can be applied to solve special cases of QAP, including the Graph Matching
> >>>> Problem (GMP) and the Traveling Salesman Problem (TSP).
> >>>>
> >>>> Since the QAP is a combinatorial optimization problem, I'd like to have
> >>>> the FAQ implementation exposed through scipy.optimize. The module would
> >>>> accept parameters such as permutation initialization type (single
> >>>> barycenter initialization, or several initializations "close" to the
> >>>> barycenter), maximum Frank-Wolfe iterations, and whether you would like to
> >>>> solve a special case of the QAP (such as GMP). The module would fit with
> >>>> the two cost matrices in the objective function, returning the score
> >>>> (minimized objection function value) and indices of the optimal permutation
> >>>> matrix from the objective function. The implementation will also give the
> >>>> option to include seeds
> >>>>
> >>>
> >> Since the inputs are graphs and the functionality resembles things that
> >> live in scipy.sparse.csgraph, I'm wondering whether this either should go
> >> into sparse.csgraph or if it's better in scipy.optimize then should it be
> >> able to understand sparse inputs?
> >>
> >> Cheers,
> >> Ralf
> >>
> >>
> >>>> I have already implemented FAQ in GraSPy
> >>>> <https://github.com/neurodata/graspy/blob/master/graspy/match/faq.py>,
> >>>> and proof of effectiveness can be found here
> >>>> <https://graspy.neurodata.io/tutorials/matching/faq.html> .
> >>>>
> >>>> Best,
> >>>> Ali Saad-Eldin
> >>>>
> >>>> _______________________________________________
> >>>> SciPy-Dev mailing list
> >>>> SciPy-Dev at python.org
> >>>> https://mail.python.org/mailman/listinfo/scipy-dev
> >>>>
> >>>
> >>> --
> >>> Le informazioni contenute nella presente comunicazione sono di natura privata
> >>> e come tali sono da considerarsi riservate ed indirizzate esclusivamente
> >>> ai destinatari indicati e per le finalit? strettamente legate al
> >>> relativo contenuto. Se avete ricevuto questo messaggio per errore, vi
> >>> preghiamo di eliminarlo e di inviare una comunicazione all?indirizzo
> >>> e-mail del mittente.
> >>> --
> >>> The information transmitted is intended only for the person or entity to
> >>> which it is addressed and may contain confidential and/or privileged
> >>> material. If you received this in error, please contact the sender and
> >>> delete the material.
> >>> _______________________________________________
> >>> SciPy-Dev mailing list
> >>> SciPy-Dev at python.org
> >>> https://mail.python.org/mailman/listinfo/scipy-dev
> >>>
> >> _______________________________________________
> >> SciPy-Dev mailing list
> >> SciPy-Dev at python.org
> >> https://mail.python.org/mailman/listinfo/scipy-dev
> >>
> > _______________________________________________
> > SciPy-Dev mailing list
> > SciPy-Dev at python.org
> > https://mail.python.org/mailman/listinfo/scipy-dev
> >

> _______________________________________________
> SciPy-Dev mailing list
> SciPy-Dev at python.org
> https://mail.python.org/mailman/listinfo/scipy-dev


From asaadel1 at jhu.edu  Fri Feb 21 15:11:22 2020
From: asaadel1 at jhu.edu (Ali Saad-Eldin)
Date: Fri, 21 Feb 2020 20:11:22 +0000
Subject: [SciPy-Dev] ENH: Implement a Quadratic Assignment Problem Solver
In-Reply-To: <CADuxUixUitw7FinXUsaNTq7CxRjJn3C=-yctJDKhex2XQyJihg@mail.gmail.com>
References: <BL0PR01MB52174C285DE3E56B70B539A499130@BL0PR01MB5217.prod.exchangelabs.com>
 <CAK9Wt6Dcr5ge1q1JtSe01JTa14O3s+S0ks7OizO4DXJsewnDNQ@mail.gmail.com>
 <CABL7CQi8v=W4j9wVNPRMahuiBL1xbQEmSCgG3dtGkFuer=DwXg@mail.gmail.com>
 <CAPDXd1401M4Nq5RQBc+mA9bseRuJxSn-EdVNVgwrcMHWjQ0Mog@mail.gmail.com>,
 <CADuxUixUitw7FinXUsaNTq7CxRjJn3C=-yctJDKhex2XQyJihg@mail.gmail.com>
Message-ID: <BL0PR01MB5217BB699A7B26C31C02354799120@BL0PR01MB5217.prod.exchangelabs.com>

Hi all,
I agree with both Kai and Matt. Since the QAP is a combinatorial optimization problem (even though it has graph theory applications), and that there is already an "assignment problem" category in scipy.optimize, I think a QAP solver would fit better there. Also, it could accept sparse inputs but the first steps of the algorithm operate on dense arrays.
Best,
Ali
________________________________
From: SciPy-Dev <scipy-dev-bounces+asaadel1=jhu.edu at python.org> on behalf of Matt Haberland <mhaberla at calpoly.edu>
Sent: Friday, February 21, 2020 9:21 AM
To: SciPy Developers List <scipy-dev at python.org>
Subject: Re: [SciPy-Dev] ENH: Implement a Quadratic Assignment Problem Solver

I can't say where it would fit better, but it sounds like QAP would fit well in optimize next to linear_sum_assignment now and the more general MIP and QP solvers when they happen.

On Fri, Feb 21, 2020, 5:27 AM Kai Striega <kaistriega at gmail.com<mailto:kaistriega at gmail.com>> wrote:
Hi All,

Since the inputs are graphs and the functionality resembles things that live in scipy.sparse.csgraph, I'm wondering whether this either should go into sparse.csgraph or if it's better in scipy.optimize then should it be able to understand sparse inputs?

I'd say a QAP solver fits into scipy.optimize better than scipy.sparse.csgraph. We already have scipy.optimize.linear_sum_assignment<https://docs.scipy.org/doc/scipy/reference/generated/scipy.optimize.linear_sum_assignment.html#scipy.optimize.linear_sum_assignment> and an "assignment problems" category, although right, now it's a subcategory of linear programming. I also vaguely remember talk of a general quadratic programming solver to follow the linear programming solver, although I don't think there has been any progress there. Further, a QAP solver can be formulated as a more general quadratic programming problem, while the algorithms in csgraph all look like "classic" graph theory algorithms. Saying that, I haven't worked with csgraph much, so I may be wrong regarding its scope.

Just adding my 2c

Kai

On Fri, 21 Feb 2020 at 20:14, Ralf Gommers <ralf.gommers at gmail.com<mailto:ralf.gommers at gmail.com>> wrote:


On Fri, Feb 21, 2020 at 2:03 AM Emanuele Olivetti <olivetti at fbk.eu<mailto:olivetti at fbk.eu>> wrote:
Dear Ali and SciPy,

Some time ago, I implemented another algorithm for the approximate solution of the QAP / Graph Matching: the Doubly Stochastic Projected Fixed-Point (DSPFP) algorithm proposed in Lu et al. (2016) [*]. Input and output are basically the same as the FAQ algorithm of Ali. If there is interest in adding approximate QAP / Graph Matching algorithms to SciPy, I'd happy to contribute with it:
  https://github.com/emanuele/DSPFP

Best,

Emanuele

[*] : http://dx.doi.org/10.1016/j.patcog.2016.07.015
Pre-print about the same manuscript and algorithm (named fastPFP at that time, 2012) here: https://arxiv.org/abs/1207.1114

On Thu, Feb 20, 2020 at 6:38 PM Ali Saad-Eldin <asaadel1 at jhu.edu<mailto:asaadel1 at jhu.edu>> wrote:
Hello!

I would like to add a Quadratic Approximation Problem (QAP)<https://en.wikipedia.org/wiki/Quadratic_assignment_problem> solver function, by implementing the Fast Approximate QAP (FAQ) algorithm<https://journals.plos.org/plosone/article/file?id=10.1371/journal.pone.0121002&type=printable> (Vogelstein, 2015). QAP is a combinatorial optimization problem, and the FAQ algorithm can be applied to solve special cases of QAP, including the Graph Matching Problem (GMP) and the Traveling Salesman Problem (TSP).

Since the QAP is a combinatorial optimization problem, I'd like to have the FAQ implementation exposed through scipy.optimize. The module would accept parameters such as permutation initialization type (single barycenter initialization, or several initializations "close" to the barycenter), maximum Frank-Wolfe iterations, and whether you would like to solve a special case of the QAP (such as GMP). The module would fit with the two cost matrices in the objective function, returning the score (minimized objection function value) and indices of the optimal permutation matrix from the objective function. The implementation will also give the option to include seeds

Since the inputs are graphs and the functionality resembles things that live in scipy.sparse.csgraph, I'm wondering whether this either should go into sparse.csgraph or if it's better in scipy.optimize then should it be able to understand sparse inputs?

Cheers,
Ralf


I have already implemented FAQ in GraSPy<https://github.com/neurodata/graspy/blob/master/graspy/match/faq.py>, and proof of effectiveness can be found here<https://graspy.neurodata.io/tutorials/matching/faq.html> .

Best,
Ali Saad-Eldin

_______________________________________________
SciPy-Dev mailing list
SciPy-Dev at python.org<mailto:SciPy-Dev at python.org>
https://mail.python.org/mailman/listinfo/scipy-dev

--
Le informazioni contenute nella presente comunicazione sono di natura privata e come tali sono da considerarsi riservate ed indirizzate esclusivamente ai destinatari indicati e per le finalit? strettamente legate al relativo contenuto. Se avete ricevuto questo messaggio per errore, vi preghiamo di eliminarlo e di inviare una comunicazione all?indirizzo e-mail del mittente.
--
The information transmitted is intended only for the person or entity to which it is addressed and may contain confidential and/or privileged material. If you received this in error, please contact the sender and delete the material.
_______________________________________________
SciPy-Dev mailing list
SciPy-Dev at python.org<mailto:SciPy-Dev at python.org>
https://mail.python.org/mailman/listinfo/scipy-dev
_______________________________________________
SciPy-Dev mailing list
SciPy-Dev at python.org<mailto:SciPy-Dev at python.org>
https://mail.python.org/mailman/listinfo/scipy-dev
_______________________________________________
SciPy-Dev mailing list
SciPy-Dev at python.org<mailto:SciPy-Dev at python.org>
https://mail.python.org/mailman/listinfo/scipy-dev
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scipy-dev/attachments/20200221/1374648c/attachment-0001.html>

From ralf.gommers at gmail.com  Sun Feb 23 19:25:36 2020
From: ralf.gommers at gmail.com (Ralf Gommers)
Date: Sun, 23 Feb 2020 16:25:36 -0800
Subject: [SciPy-Dev] participating in GSoC'20?
In-Reply-To: <24B42FCB-710D-45BB-B6ED-F80BE6EFB6A5@getmailspring.com>
References: <CABL7CQgJNzbRnC=E=8TY89tnW1FFv4=fShaDQoZM4fN7vSvVrw@mail.gmail.com>
 <24B42FCB-710D-45BB-B6ED-F80BE6EFB6A5@getmailspring.com>
Message-ID: <CABL7CQijELacD83tmwr50noG4_jv6aNvFPnyfW1Z_oBjNR+e-A@mail.gmail.com>

On Sat, Feb 15, 2020 at 11:07 AM Nicolas Cellier <eryole at gmail.com> wrote:

> Hello.
>
> As topic ideas, it could be good to use the (not so) recent
> scipy.integrate.solve_ivp  interface to implement more modern ODE solvers
> (like SSP, Rosenbrock Wanner). The goal could be to provide more efficient
> solvers as the one available with the Julia equivalent library
> (DifferentialEquation.jl if I remember well).
>
> An other idea could be the implementation of Jacobian coloring and
> sparsity detection. They are really useful tools that can highly improve
> methods implying Jacobian computation.  (see
> https://pdfs.semanticscholar.org/ae4d/65769b6551a51d6fc6be2f021515bffa0798.pdf
> <https://link.getmailspring.com/link/24B42FCB-710D-45BB-B6ED-F80BE6EFB6A5 at getmailspring.com/0?redirect=https%3A%2F%2Fpdfs.semanticscholar.org%2Fae4d%2F65769b6551a51d6fc6be2f021515bffa0798.pdf&recipient=c2NpcHktZGV2QHB5dGhvbi5vcmc%3D>
> for a reference on Jacobian coloring).
>

Thanks for the ideas Nicolas.


> Cheers,
>
> Nicolas Cellier
>
>
>
> On f?vr. 15 2020, at 7:15 pm, Ralf Gommers <ralf.gommers at gmail.com> wrote:
>
> Hi all,
>
> It's that time of the year again already. Do we want to participate in
> this years' Google Summer of Code? Last year was quite successful (Peter's
> work on scipy.fft). We do need some volunteers to mentors though, and
> ideally also someone to admin (I'm pretty swamped). Any takers? Any good
> topic ideas?
>
> It looks like the answer is "no" this year. I think that's okay - let's
see next year.

Cheers,
Ralf


> Cheers,
> Ralf
>
> _______________________________________________
> SciPy-Dev mailing list
> SciPy-Dev at python.org
> https://mail.python.org/mailman/listinfo/scipy-dev
>
> [image: Sent from Mailspring]
> _______________________________________________
> SciPy-Dev mailing list
> SciPy-Dev at python.org
> https://mail.python.org/mailman/listinfo/scipy-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scipy-dev/attachments/20200223/5d44e7ee/attachment.html>

From ralf.gommers at gmail.com  Sun Feb 23 19:29:04 2020
From: ralf.gommers at gmail.com (Ralf Gommers)
Date: Sun, 23 Feb 2020 16:29:04 -0800
Subject: [SciPy-Dev] [Newbie] Information regarding GSoC
In-Reply-To: <CAPhLvQOqzMBNUQDRNKxQPZZp9_byKRz0WZ8LRP+5wkaHXTYYoA@mail.gmail.com>
References: <CAPhLvQMkh8B7hq7Ho4o1+my_cpDs7askueykzjNzRm8-LmFefA@mail.gmail.com>
 <CABL7CQh_c1rcRVKstW2Au+jpn5nniLLwPC5Mu-sOj9EKymK_Ng@mail.gmail.com>
 <CAPhLvQMgZWsaQEAH9Dc7W4sBp8N6x4muVE-2RKbLh81QnPWH7A@mail.gmail.com>
 <CAPhLvQOqzMBNUQDRNKxQPZZp9_byKRz0WZ8LRP+5wkaHXTYYoA@mail.gmail.com>
Message-ID: <CABL7CQj=8RWGEeTAK1jDT8gH3bW5u-U_hWocLhuVSrLg_++8Cw@mail.gmail.com>

On Thu, Feb 20, 2020 at 8:53 AM Shubham Mishra <smishra99.iitkgp at gmail.com>
wrote:

> Hello,
> This is a follow up on the above mail.
> According to GSoC timelines, the date of release of organizations is
> tomorrow.
> Please let me know if Scipy is really participating.
>

Hi Shubham, it looks like we unfortunately do not have enough SciPy
maintainers available to be able to participate.

I'll be contributing anyway, however.
>

That's great to hear! Please dive in and ask for help if you need it. I
would suggest to start with some smaller issues; the topics Nicolas
suggested are really large and we don't have an issue with a possible plan
of attack.

Cheers,
Ralf


> Warm regards,
> Shubham Mishra
>
> On Mon, Feb 17, 2020 at 1:57 PM Shubham Mishra <smishra99.iitkgp at gmail.com>
> wrote:
>
>> Hello all.
>> Thanks for hearing me out regarding GSoC. I am interested in the
>> differential equation solvers as put forward by Nicolas. Should I start
>> preparing for the same?
>> I have done a basic freshman level math course where ODEs were taught,
>> but not up to that level. Can someone suggest some reading materials
>> (books, papers, tutorials... anything) from where I can learn?
>> I am really eager to learn and contribute in a meaningful way.
>>
>> Yours sincerely,
>> Shubham Mishra (grapheo12)
>>
>> On Sat, 15 Feb 2020, 23:41 Ralf Gommers, <ralf.gommers at gmail.com> wrote:
>>
>>>
>>>
>>> On Sat, Feb 15, 2020 at 12:33 AM Shubham Mishra <
>>> smishra99.iitkgp at gmail.com> wrote:
>>>
>>>> Hello,
>>>> Newbie here. Is Scipy participating in GSoC this year?
>>>> I am curious to work under Scipy for GSoC if it is possible. Can
>>>> someone mentor me regarding this?
>>>>
>>>
>>> Hi Shubham, thanks for asking. We haven't discussed it yet, now would be
>>> a good time to do that - I'll open a separate thread for it.
>>>
>>> Cheers,
>>> Ralf
>>>
>>>
>>>
>>>> Thanking you,
>>>> Yours sincerely,
>>>> Shubham Mishra
>>>> (Github handle: grapheo12)
>>>> _______________________________________________
>>>> SciPy-Dev mailing list
>>>> SciPy-Dev at python.org
>>>> https://mail.python.org/mailman/listinfo/scipy-dev
>>>>
>>> _______________________________________________
>>> SciPy-Dev mailing list
>>> SciPy-Dev at python.org
>>> https://mail.python.org/mailman/listinfo/scipy-dev
>>>
>> _______________________________________________
> SciPy-Dev mailing list
> SciPy-Dev at python.org
> https://mail.python.org/mailman/listinfo/scipy-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scipy-dev/attachments/20200223/37041f9e/attachment.html>

From olivetti at fbk.eu  Tue Feb 25 06:17:25 2020
From: olivetti at fbk.eu (Emanuele Olivetti)
Date: Tue, 25 Feb 2020 12:17:25 +0100
Subject: [SciPy-Dev] ENH: Implement a Quadratic Assignment Problem Solver
In-Reply-To: <BL0PR01MB5217BB699A7B26C31C02354799120@BL0PR01MB5217.prod.exchangelabs.com>
References: <BL0PR01MB52174C285DE3E56B70B539A499130@BL0PR01MB5217.prod.exchangelabs.com>
 <CAK9Wt6Dcr5ge1q1JtSe01JTa14O3s+S0ks7OizO4DXJsewnDNQ@mail.gmail.com>
 <CABL7CQi8v=W4j9wVNPRMahuiBL1xbQEmSCgG3dtGkFuer=DwXg@mail.gmail.com>
 <CAPDXd1401M4Nq5RQBc+mA9bseRuJxSn-EdVNVgwrcMHWjQ0Mog@mail.gmail.com>
 <CADuxUixUitw7FinXUsaNTq7CxRjJn3C=-yctJDKhex2XQyJihg@mail.gmail.com>
 <BL0PR01MB5217BB699A7B26C31C02354799120@BL0PR01MB5217.prod.exchangelabs.com>
Message-ID: <CAK9Wt6AXhqJ_mLoe=7RrQw2b7HVC-1GBabhc=Og2m-Un=UzhFw@mail.gmail.com>

On Fri, Feb 21, 2020 at 9:26 PM Ali Saad-Eldin <asaadel1 at jhu.edu> wrote:

> Hi all,
> [...]
>
Also, it could accept sparse inputs but the first steps of the algorithm
> operate on dense arrays.
>
>
Same for DSPFP.

Emanuele

-- 
--
Le informazioni contenute nella presente comunicazione sono di natura?
privata e come tali sono da considerarsi riservate ed indirizzate?
esclusivamente ai destinatari indicati e per le finalit? strettamente?
legate al relativo contenuto. Se avete ricevuto questo messaggio per?
errore, vi preghiamo di eliminarlo e di inviare una comunicazione?
all?indirizzo e-mail del mittente.

--
The information transmitted is 
intended only for the person or entity to which it is addressed and may 
contain confidential and/or privileged material. If you received this in 
error, please contact the sender and delete the material.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scipy-dev/attachments/20200225/1d746c9b/attachment.html>

From ralf.gommers at gmail.com  Wed Feb 26 00:54:18 2020
From: ralf.gommers at gmail.com (Ralf Gommers)
Date: Tue, 25 Feb 2020 21:54:18 -0800
Subject: [SciPy-Dev] ENH: Implement a Quadratic Assignment Problem Solver
In-Reply-To: <CAK9Wt6AXhqJ_mLoe=7RrQw2b7HVC-1GBabhc=Og2m-Un=UzhFw@mail.gmail.com>
References: <BL0PR01MB52174C285DE3E56B70B539A499130@BL0PR01MB5217.prod.exchangelabs.com>
 <CAK9Wt6Dcr5ge1q1JtSe01JTa14O3s+S0ks7OizO4DXJsewnDNQ@mail.gmail.com>
 <CABL7CQi8v=W4j9wVNPRMahuiBL1xbQEmSCgG3dtGkFuer=DwXg@mail.gmail.com>
 <CAPDXd1401M4Nq5RQBc+mA9bseRuJxSn-EdVNVgwrcMHWjQ0Mog@mail.gmail.com>
 <CADuxUixUitw7FinXUsaNTq7CxRjJn3C=-yctJDKhex2XQyJihg@mail.gmail.com>
 <BL0PR01MB5217BB699A7B26C31C02354799120@BL0PR01MB5217.prod.exchangelabs.com>
 <CAK9Wt6AXhqJ_mLoe=7RrQw2b7HVC-1GBabhc=Og2m-Un=UzhFw@mail.gmail.com>
Message-ID: <CABL7CQjSQSswYU1w=KQ35W0y86jLK5gLWm7C1Cq_Vtf4FfaEOg@mail.gmail.com>

On Tue, Feb 25, 2020 at 3:17 AM Emanuele Olivetti <olivetti at fbk.eu> wrote:

>
>
> On Fri, Feb 21, 2020 at 9:26 PM Ali Saad-Eldin <asaadel1 at jhu.edu> wrote:
>
>> Hi all,
>> [...]
>>
> Also, it could accept sparse inputs but the first steps of the algorithm
>> operate on dense arrays.
>>
>>
> Same for DSPFP.
>

Sounds good to me.

Cheers,
Ralf



> Emanuele
>
>
> --
> Le informazioni contenute nella presente comunicazione sono di natura privata
> e come tali sono da considerarsi riservate ed indirizzate esclusivamente
> ai destinatari indicati e per le finalit? strettamente legate al relativo
> contenuto. Se avete ricevuto questo messaggio per errore, vi preghiamo di
> eliminarlo e di inviare una comunicazione all?indirizzo e-mail del
> mittente.
> --
> The information transmitted is intended only for the person or entity to
> which it is addressed and may contain confidential and/or privileged
> material. If you received this in error, please contact the sender and
> delete the material.
> _______________________________________________
> SciPy-Dev mailing list
> SciPy-Dev at python.org
> https://mail.python.org/mailman/listinfo/scipy-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scipy-dev/attachments/20200225/506e3223/attachment.html>

From olivetti at fbk.eu  Wed Feb 26 07:01:52 2020
From: olivetti at fbk.eu (Emanuele Olivetti)
Date: Wed, 26 Feb 2020 13:01:52 +0100
Subject: [SciPy-Dev] ENH: Implement a Quadratic Assignment Problem Solver
In-Reply-To: <BL0PR01MB5217BB699A7B26C31C02354799120@BL0PR01MB5217.prod.exchangelabs.com>
References: <BL0PR01MB52174C285DE3E56B70B539A499130@BL0PR01MB5217.prod.exchangelabs.com>
 <CAK9Wt6Dcr5ge1q1JtSe01JTa14O3s+S0ks7OizO4DXJsewnDNQ@mail.gmail.com>
 <CABL7CQi8v=W4j9wVNPRMahuiBL1xbQEmSCgG3dtGkFuer=DwXg@mail.gmail.com>
 <CAPDXd1401M4Nq5RQBc+mA9bseRuJxSn-EdVNVgwrcMHWjQ0Mog@mail.gmail.com>
 <CADuxUixUitw7FinXUsaNTq7CxRjJn3C=-yctJDKhex2XQyJihg@mail.gmail.com>
 <BL0PR01MB5217BB699A7B26C31C02354799120@BL0PR01MB5217.prod.exchangelabs.com>
Message-ID: <CAK9Wt6D40HW3UFTsUY5XzC3ojdbLNXk-MeaGWXs=vnw94NpstQ@mail.gmail.com>

Hi Ali and Everyone interested in this topic,

What API would be OK for SciPy, for algorithms about QAP/GMP?

@Ali: I see that your implementation of the FAQ algorithm follows the
.fit() / .fit_predict() pattern of scikit-learn. In my case, with DSPFP, it
is something very basic and not thought for a proper API. I can change it
though. Moreover, I still don't have proper tests. Ideas? Comments?

Best,

Emanuele


On Fri, Feb 21, 2020 at 9:26 PM Ali Saad-Eldin <asaadel1 at jhu.edu> wrote:

> Hi all,
> I agree with both Kai and Matt. Since the QAP is a combinatorial
> optimization problem (even though it has graph theory applications), and
> that there is already an "assignment problem" category in scipy.optimize, I
> think a QAP solver would fit better there. Also, it could accept sparse
> inputs but the first steps of the algorithm operate on dense arrays.
> Best,
> Ali
> ------------------------------
> *From:* SciPy-Dev <scipy-dev-bounces+asaadel1=jhu.edu at python.org> on
> behalf of Matt Haberland <mhaberla at calpoly.edu>
> *Sent:* Friday, February 21, 2020 9:21 AM
> *To:* SciPy Developers List <scipy-dev at python.org>
> *Subject:* Re: [SciPy-Dev] ENH: Implement a Quadratic Assignment Problem
> Solver
>
> I can't say where it would fit better, but it sounds like QAP would fit
> well in optimize next to linear_sum_assignment now and the more general MIP
> and QP solvers when they happen.
>
> On Fri, Feb 21, 2020, 5:27 AM Kai Striega <kaistriega at gmail.com> wrote:
>
> Hi All,
>
> Since the inputs are graphs and the functionality resembles things that
> live in scipy.sparse.csgraph, I'm wondering whether this either should go
> into sparse.csgraph or if it's better in scipy.optimize then should it be
> able to understand sparse inputs?
>
>
> I'd say a QAP solver fits into scipy.optimize better than
> scipy.sparse.csgraph. We already have scipy.optimize.linear_sum_assignment
> <https://docs.scipy.org/doc/scipy/reference/generated/scipy.optimize.linear_sum_assignment.html#scipy.optimize.linear_sum_assignment>
> and an *"assignment problems"* category, although right, now it's a
> subcategory of linear programming. I also vaguely remember talk of a
> general quadratic programming solver to follow the linear programming
> solver, although I don't think there has been any progress there. Further,
> a QAP solver can be formulated as a more general quadratic programming
> problem, while the algorithms in csgraph all look like "classic" graph
> theory algorithms. Saying that, I haven't worked with csgraph much, so I
> may be wrong regarding its scope.
>
> Just adding my 2c
>
> Kai
>
> On Fri, 21 Feb 2020 at 20:14, Ralf Gommers <ralf.gommers at gmail.com> wrote:
>
>
>
> On Fri, Feb 21, 2020 at 2:03 AM Emanuele Olivetti <olivetti at fbk.eu> wrote:
>
> Dear Ali and SciPy,
>
> Some time ago, I implemented another algorithm for the approximate
> solution of the QAP / Graph Matching: the Doubly Stochastic Projected
> Fixed-Point (DSPFP) algorithm proposed in Lu et al. (2016) [*]. Input and
> output are basically the same as the FAQ algorithm of Ali. If there is
> interest in adding approximate QAP / Graph Matching algorithms to SciPy,
> I'd happy to contribute with it:
>   https://github.com/emanuele/DSPFP
>
> Best,
>
> Emanuele
>
> [*] : http://dx.doi.org/10.1016/j.patcog.2016.07.015
> Pre-print about the same manuscript and algorithm (named fastPFP at that
> time, 2012) here: https://arxiv.org/abs/1207.1114
>
> On Thu, Feb 20, 2020 at 6:38 PM Ali Saad-Eldin <asaadel1 at jhu.edu> wrote:
>
> Hello!
>
> I would like to add a Quadratic Approximation Problem (QAP)
> <https://en.wikipedia.org/wiki/Quadratic_assignment_problem> solver
> function, by implementing the Fast Approximate QAP (FAQ) algorithm
> <https://journals.plos.org/plosone/article/file?id=10.1371/journal.pone.0121002&type=printable> (Vogelstein,
> 2015). QAP is a combinatorial optimization problem, and the FAQ algorithm
> can be applied to solve special cases of QAP, including the Graph Matching
> Problem (GMP) and the Traveling Salesman Problem (TSP).
>
> Since the QAP is a combinatorial optimization problem, I'd like to have
> the FAQ implementation exposed through scipy.optimize. The module would
> accept parameters such as permutation initialization type (single
> barycenter initialization, or several initializations "close" to the
> barycenter), maximum Frank-Wolfe iterations, and whether you would like to
> solve a special case of the QAP (such as GMP). The module would fit with
> the two cost matrices in the objective function, returning the score
> (minimized objection function value) and indices of the optimal permutation
> matrix from the objective function. The implementation will also give the
> option to include seeds
>
>
> Since the inputs are graphs and the functionality resembles things that
> live in scipy.sparse.csgraph, I'm wondering whether this either should go
> into sparse.csgraph or if it's better in scipy.optimize then should it be
> able to understand sparse inputs?
>
> Cheers,
> Ralf
>
>
> I have already implemented FAQ in GraSPy
> <https://github.com/neurodata/graspy/blob/master/graspy/match/faq.py>,
> and proof of effectiveness can be found here
> <https://graspy.neurodata.io/tutorials/matching/faq.html> .
>
> Best,
> Ali Saad-Eldin
>
> _______________________________________________
> SciPy-Dev mailing list
> SciPy-Dev at python.org
> https://mail.python.org/mailman/listinfo/scipy-dev
>
>
> --
> Le informazioni contenute nella presente comunicazione sono di natura privata
> e come tali sono da considerarsi riservate ed indirizzate esclusivamente
> ai destinatari indicati e per le finalit? strettamente legate al relativo
> contenuto. Se avete ricevuto questo messaggio per errore, vi preghiamo di
> eliminarlo e di inviare una comunicazione all?indirizzo e-mail del
> mittente.
> --
> The information transmitted is intended only for the person or entity to
> which it is addressed and may contain confidential and/or privileged
> material. If you received this in error, please contact the sender and
> delete the material.
> _______________________________________________
> SciPy-Dev mailing list
> SciPy-Dev at python.org
> https://mail.python.org/mailman/listinfo/scipy-dev
>
> _______________________________________________
> SciPy-Dev mailing list
> SciPy-Dev at python.org
> https://mail.python.org/mailman/listinfo/scipy-dev
>
> _______________________________________________
> SciPy-Dev mailing list
> SciPy-Dev at python.org
> https://mail.python.org/mailman/listinfo/scipy-dev
>
> _______________________________________________
> SciPy-Dev mailing list
> SciPy-Dev at python.org
> https://mail.python.org/mailman/listinfo/scipy-dev
>

-- 
--
Le informazioni contenute nella presente comunicazione sono di natura?
privata e come tali sono da considerarsi riservate ed indirizzate?
esclusivamente ai destinatari indicati e per le finalit? strettamente?
legate al relativo contenuto. Se avete ricevuto questo messaggio per?
errore, vi preghiamo di eliminarlo e di inviare una comunicazione?
all?indirizzo e-mail del mittente.

--
The information transmitted is 
intended only for the person or entity to which it is addressed and may 
contain confidential and/or privileged material. If you received this in 
error, please contact the sender and delete the material.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scipy-dev/attachments/20200226/73738684/attachment-0001.html>

From mhaberla at calpoly.edu  Thu Feb 27 02:22:26 2020
From: mhaberla at calpoly.edu (Matt Haberland)
Date: Wed, 26 Feb 2020 23:22:26 -0800
Subject: [SciPy-Dev] ENH: Implement a Quadratic Assignment Problem Solver
In-Reply-To: <CAK9Wt6D40HW3UFTsUY5XzC3ojdbLNXk-MeaGWXs=vnw94NpstQ@mail.gmail.com>
References: <BL0PR01MB52174C285DE3E56B70B539A499130@BL0PR01MB5217.prod.exchangelabs.com>
 <CAK9Wt6Dcr5ge1q1JtSe01JTa14O3s+S0ks7OizO4DXJsewnDNQ@mail.gmail.com>
 <CABL7CQi8v=W4j9wVNPRMahuiBL1xbQEmSCgG3dtGkFuer=DwXg@mail.gmail.com>
 <CAPDXd1401M4Nq5RQBc+mA9bseRuJxSn-EdVNVgwrcMHWjQ0Mog@mail.gmail.com>
 <CADuxUixUitw7FinXUsaNTq7CxRjJn3C=-yctJDKhex2XQyJihg@mail.gmail.com>
 <BL0PR01MB5217BB699A7B26C31C02354799120@BL0PR01MB5217.prod.exchangelabs.com>
 <CAK9Wt6D40HW3UFTsUY5XzC3ojdbLNXk-MeaGWXs=vnw94NpstQ@mail.gmail.com>
Message-ID: <CADuxUix_bKsxjjiRz6zKZbA1=wm-+juDGnDS3Q0ASf=21RuJuQ@mail.gmail.com>

I would follow the example of linear_sum_assignment
<https://docs.scipy.org/doc/scipy/reference/generated/scipy.optimize.linear_sum_assignment.html>.
For QAP, it looks like the only difference would be that you'd need two
input matrices (weights and distances).

On Wed, Feb 26, 2020 at 4:02 AM Emanuele Olivetti <olivetti at fbk.eu> wrote:

> Hi Ali and Everyone interested in this topic,
>
> What API would be OK for SciPy, for algorithms about QAP/GMP?
>
> @Ali: I see that your implementation of the FAQ algorithm follows the
> .fit() / .fit_predict() pattern of scikit-learn. In my case, with DSPFP, it
> is something very basic and not thought for a proper API. I can change it
> though. Moreover, I still don't have proper tests. Ideas? Comments?
>
> Best,
>
> Emanuele
>
>
> On Fri, Feb 21, 2020 at 9:26 PM Ali Saad-Eldin <asaadel1 at jhu.edu> wrote:
>
>> Hi all,
>> I agree with both Kai and Matt. Since the QAP is a combinatorial
>> optimization problem (even though it has graph theory applications), and
>> that there is already an "assignment problem" category in scipy.optimize, I
>> think a QAP solver would fit better there. Also, it could accept sparse
>> inputs but the first steps of the algorithm operate on dense arrays.
>> Best,
>> Ali
>> ------------------------------
>> *From:* SciPy-Dev <scipy-dev-bounces+asaadel1=jhu.edu at python.org> on
>> behalf of Matt Haberland <mhaberla at calpoly.edu>
>> *Sent:* Friday, February 21, 2020 9:21 AM
>> *To:* SciPy Developers List <scipy-dev at python.org>
>> *Subject:* Re: [SciPy-Dev] ENH: Implement a Quadratic Assignment Problem
>> Solver
>>
>> I can't say where it would fit better, but it sounds like QAP would fit
>> well in optimize next to linear_sum_assignment now and the more general MIP
>> and QP solvers when they happen.
>>
>> On Fri, Feb 21, 2020, 5:27 AM Kai Striega <kaistriega at gmail.com> wrote:
>>
>> Hi All,
>>
>> Since the inputs are graphs and the functionality resembles things that
>> live in scipy.sparse.csgraph, I'm wondering whether this either should go
>> into sparse.csgraph or if it's better in scipy.optimize then should it be
>> able to understand sparse inputs?
>>
>>
>> I'd say a QAP solver fits into scipy.optimize better than
>> scipy.sparse.csgraph. We already have
>> scipy.optimize.linear_sum_assignment
>> <https://docs.scipy.org/doc/scipy/reference/generated/scipy.optimize.linear_sum_assignment.html#scipy.optimize.linear_sum_assignment>
>> and an *"assignment problems"* category, although right, now it's a
>> subcategory of linear programming. I also vaguely remember talk of a
>> general quadratic programming solver to follow the linear programming
>> solver, although I don't think there has been any progress there. Further,
>> a QAP solver can be formulated as a more general quadratic programming
>> problem, while the algorithms in csgraph all look like "classic" graph
>> theory algorithms. Saying that, I haven't worked with csgraph much, so I
>> may be wrong regarding its scope.
>>
>> Just adding my 2c
>>
>> Kai
>>
>> On Fri, 21 Feb 2020 at 20:14, Ralf Gommers <ralf.gommers at gmail.com>
>> wrote:
>>
>>
>>
>> On Fri, Feb 21, 2020 at 2:03 AM Emanuele Olivetti <olivetti at fbk.eu>
>> wrote:
>>
>> Dear Ali and SciPy,
>>
>> Some time ago, I implemented another algorithm for the approximate
>> solution of the QAP / Graph Matching: the Doubly Stochastic Projected
>> Fixed-Point (DSPFP) algorithm proposed in Lu et al. (2016) [*]. Input and
>> output are basically the same as the FAQ algorithm of Ali. If there is
>> interest in adding approximate QAP / Graph Matching algorithms to SciPy,
>> I'd happy to contribute with it:
>>   https://github.com/emanuele/DSPFP
>>
>> Best,
>>
>> Emanuele
>>
>> [*] : http://dx.doi.org/10.1016/j.patcog.2016.07.015
>> Pre-print about the same manuscript and algorithm (named fastPFP at that
>> time, 2012) here: https://arxiv.org/abs/1207.1114
>>
>> On Thu, Feb 20, 2020 at 6:38 PM Ali Saad-Eldin <asaadel1 at jhu.edu> wrote:
>>
>> Hello!
>>
>> I would like to add a Quadratic Approximation Problem (QAP)
>> <https://en.wikipedia.org/wiki/Quadratic_assignment_problem> solver
>> function, by implementing the Fast Approximate QAP (FAQ) algorithm
>> <https://journals.plos.org/plosone/article/file?id=10.1371/journal.pone.0121002&type=printable> (Vogelstein,
>> 2015). QAP is a combinatorial optimization problem, and the FAQ algorithm
>> can be applied to solve special cases of QAP, including the Graph Matching
>> Problem (GMP) and the Traveling Salesman Problem (TSP).
>>
>> Since the QAP is a combinatorial optimization problem, I'd like to have
>> the FAQ implementation exposed through scipy.optimize. The module would
>> accept parameters such as permutation initialization type (single
>> barycenter initialization, or several initializations "close" to the
>> barycenter), maximum Frank-Wolfe iterations, and whether you would like to
>> solve a special case of the QAP (such as GMP). The module would fit with
>> the two cost matrices in the objective function, returning the score
>> (minimized objection function value) and indices of the optimal permutation
>> matrix from the objective function. The implementation will also give the
>> option to include seeds
>>
>>
>> Since the inputs are graphs and the functionality resembles things that
>> live in scipy.sparse.csgraph, I'm wondering whether this either should go
>> into sparse.csgraph or if it's better in scipy.optimize then should it be
>> able to understand sparse inputs?
>>
>> Cheers,
>> Ralf
>>
>>
>> I have already implemented FAQ in GraSPy
>> <https://github.com/neurodata/graspy/blob/master/graspy/match/faq.py>,
>> and proof of effectiveness can be found here
>> <https://graspy.neurodata.io/tutorials/matching/faq.html> .
>>
>> Best,
>> Ali Saad-Eldin
>>
>> _______________________________________________
>> SciPy-Dev mailing list
>> SciPy-Dev at python.org
>> https://mail.python.org/mailman/listinfo/scipy-dev
>>
>>
>> --
>> Le informazioni contenute nella presente comunicazione sono di natura privata
>> e come tali sono da considerarsi riservate ed indirizzate esclusivamente
>> ai destinatari indicati e per le finalit? strettamente legate al
>> relativo contenuto. Se avete ricevuto questo messaggio per errore, vi
>> preghiamo di eliminarlo e di inviare una comunicazione all?indirizzo
>> e-mail del mittente.
>> --
>> The information transmitted is intended only for the person or entity to
>> which it is addressed and may contain confidential and/or privileged
>> material. If you received this in error, please contact the sender and
>> delete the material.
>> _______________________________________________
>> SciPy-Dev mailing list
>> SciPy-Dev at python.org
>> https://mail.python.org/mailman/listinfo/scipy-dev
>>
>> _______________________________________________
>> SciPy-Dev mailing list
>> SciPy-Dev at python.org
>> https://mail.python.org/mailman/listinfo/scipy-dev
>>
>> _______________________________________________
>> SciPy-Dev mailing list
>> SciPy-Dev at python.org
>> https://mail.python.org/mailman/listinfo/scipy-dev
>>
>> _______________________________________________
>> SciPy-Dev mailing list
>> SciPy-Dev at python.org
>> https://mail.python.org/mailman/listinfo/scipy-dev
>>
>
> --
> Le informazioni contenute nella presente comunicazione sono di natura privata
> e come tali sono da considerarsi riservate ed indirizzate esclusivamente
> ai destinatari indicati e per le finalit? strettamente legate al relativo
> contenuto. Se avete ricevuto questo messaggio per errore, vi preghiamo di
> eliminarlo e di inviare una comunicazione all?indirizzo e-mail del
> mittente.
> --
> The information transmitted is intended only for the person or entity to
> which it is addressed and may contain confidential and/or privileged
> material. If you received this in error, please contact the sender and
> delete the material.
>


-- 
Matt Haberland
Assistant Professor
BioResource and Agricultural Engineering
08A-3K, Cal Poly
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scipy-dev/attachments/20200226/7b2da9fc/attachment-0001.html>

From tomirendo at gmail.com  Thu Feb 27 05:44:09 2020
From: tomirendo at gmail.com (yotam vaknin)
Date: Thu, 27 Feb 2020 12:44:09 +0200
Subject: [SciPy-Dev] Choosing the result of dtype for integration
Message-ID: <CANMcLn3qZh6ZUR44XgChNTsjCEFTuk243G35Qszc1yYmnGzKJg@mail.gmail.com>

Hello list,

I hope I'm now visiting a very old topic, but I wanted to suggest a simple
change to the integration mechanism.

At this moment, the dtype of the result of an integration is the dtype of
the initial condition. For example, when using ?solve_ivp" to solve the
initial value problem
y0=np.array([1,0] )
dy/dt= -i H at y0
which is a very a standard calculation in Quantum Mechanics, the integrator
will return nonsense, since the process involves complex numbers, and the
initial conditions were (wrongly assigned to be) real. The solution is of
course y0=np.array([1,0], dtype=complex), but this is very hard to figure
out to new users coming from, for e.g., Matlab.

I wanted to suggest that instead of choosing the dtype using the initial
condition, that the resulting dtype will be the dtype of the derivative (or
better yet, derivative*dt + initial condition). This will solve the
problem, and introduce minimal confusing.

Thanks for all of your great work!

Regards,
Yotam Vaknin
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scipy-dev/attachments/20200227/cedc86af/attachment.html>

From shashaank.n at columbia.edu  Thu Feb 27 11:11:50 2020
From: shashaank.n at columbia.edu (Shashaank Narayanan)
Date: Thu, 27 Feb 2020 11:11:50 -0500
Subject: [SciPy-Dev] SciPy Contribution: Gammatone Filters in
 scipy.signal
In-Reply-To: <CAPs8s8nU0hn607Obv2vdpgHdj2PKvE6mFSXcO1z4cN+chFSH2w@mail.gmail.com>
References: <CAPs8s8nU0hn607Obv2vdpgHdj2PKvE6mFSXcO1z4cN+chFSH2w@mail.gmail.com>
Message-ID: <CAPs8s8=XsXWgJqm8tD+B1e-55CFVuBfUMRREivJoo3cVjdUUUQ@mail.gmail.com>

Hi,

Newbie here. I just wanted to follow up on the Gammatone filters for signal
processing contribution idea that I posted recently. I would also be
interested in contributing to SciPy by doing maintenance/bug fixes for the
scipy.signal module.


Thanks,
Shashaank

On Thu, Jan 30, 2020 at 5:38 PM Shashaank Narayanan <
shashaank.n at columbia.edu> wrote:

> Hello SciPy Team,
>
> I am new to this mailing list, and I am interested in contributing to
> SciPy. I would like to suggest a new feature to be added to the
> scipy.signal module: gammatone filters. Gammatone filters are becoming
> increasingly popular in the fields of digital signal processing and music
> analysis as it effectively models the auditory filters of the human
> auditory system. Currently, there are very few implementations of gammatone
> filters available for Python, and these implementations are not generalized
> to basic finite impulse response (FIR) and infinite impulse response (IIR)
> filters like SciPy has.
>
> I have written my own gammatone FIR filter using NumPy based on Malcolm
> Slaney's 1993 paper on the topic (
> https://engineering.purdue.edu/~malcolm/apple/tr35/PattersonsEar.pdf).
> This paper was used for Matlab's implementation of gammatone filters. I am
> in the process of writing a gammatone IIR filter with NumPy and SciPy.
> Please let me know if this feature will fit with the scipy.signal module.
> Appreciate your time and guidance.
>
>
> Thanks,
> Shashaank
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scipy-dev/attachments/20200227/502de917/attachment.html>

From greg at gpanders.com  Thu Feb 27 16:46:28 2020
From: greg at gpanders.com (Greg Anders)
Date: Thu, 27 Feb 2020 14:46:28 -0700
Subject: [SciPy-Dev] Disable rpaths in shared objects when cross compiling
In-Reply-To: <20200227214237.ddm7cqhq77i4j24a@Kepler>
References: <20200227214237.ddm7cqhq77i4j24a@Kepler>
Message-ID: <20200227214628.b6wmzrlvxjl7wrkd@Kepler>

Hi all,

I'm cross compiling Scipy in Yocto Linux for an embedded platform. I'm 
able to compile Scipy, but Yocto is giving me warnings because the 
shared objects are linked with the -rpath flag using absolute paths on 
the build host. I'd like to disable the -rpath flag when linking the 
shared objects, but I'm not sure how to do that.

I've dug through the numpy.distutils code and I found the following in 
the CCompiler_customize_cmd function:

     if allow('rpath'):
         self.set_runtime_library_dirs(cmd.rpath)

So it *looks* like if I can pass 'rpath' into the optional `ignore` 
paramater to the `customize_cmd` function, numpy will not use the -rpath 
flag when linking.

However, it's not clear to me how to _use_ the `ignore` parameter of 
`customize_cmd`. All usages of `customize_cmd` in build_ext.py only use 
a single parameter, so `ignore` is always set to the default value of an 
empty tuple.

Is there an easier way to disable rpaths than patching numpy.distutils 
to path `ignore=('rpath')` as a second parameter to `customize_cmd`?

Thanks!

Greg

From david at drhagen.com  Thu Feb 27 20:46:02 2020
From: david at drhagen.com (David Hagen)
Date: Thu, 27 Feb 2020 20:46:02 -0500
Subject: [SciPy-Dev] Choosing the result of dtype for integration
In-Reply-To: <CANMcLn3qZh6ZUR44XgChNTsjCEFTuk243G35Qszc1yYmnGzKJg@mail.gmail.com>
References: <CANMcLn3qZh6ZUR44XgChNTsjCEFTuk243G35Qszc1yYmnGzKJg@mail.gmail.com>
Message-ID: <CALO6Ft66LnuxtGPx8rW=RR+g5VQ2MQAKMYT+iJ5e6zkQ3EYLRA@mail.gmail.com>

I am all for improving handling of complex integration, but I think the
current behavior matches how SciPy handles real and complex domains, namely
by making the user opt into the complex domain. In Matlab log(-1.0) is
3.14i. In NumPy, log(-1.0) is NaN. NumPy and SciPy require the user to be
explicit about working in the complex domain. Matlab happily takes you from
the real domain to the complex domain without warning. I have done a lot of
ODE integration in Matlab, and I always disliked how easily it accepted
complex numbers. If I had a log somewhere in my RHS and it accidentally got
a negative number, the whole thing would keep running and just return a
giant block of complex nonsense. Now, this is a bit different because it is
a lot harder to accidentally get a complex RHS, which is why I am only
weakly in favor of maintaining the status quo.

On Thu, Feb 27, 2020 at 5:45 AM yotam vaknin <tomirendo at gmail.com> wrote:

> Hello list,
>
> I hope I'm now visiting a very old topic, but I wanted to suggest a simple
> change to the integration mechanism.
>
> At this moment, the dtype of the result of an integration is the dtype of
> the initial condition. For example, when using ?solve_ivp" to solve the
> initial value problem
> y0=np.array([1,0] )
> dy/dt= -i H at y0
> which is a very a standard calculation in Quantum Mechanics, the
> integrator will return nonsense, since the process involves complex
> numbers, and the initial conditions were (wrongly assigned to be) real. The
> solution is of course y0=np.array([1,0], dtype=complex), but this is very
> hard to figure out to new users coming from, for e.g., Matlab.
>
> I wanted to suggest that instead of choosing the dtype using the initial
> condition, that the resulting dtype will be the dtype of the derivative (or
> better yet, derivative*dt + initial condition). This will solve the
> problem, and introduce minimal confusing.
>
> Thanks for all of your great work!
>
> Regards,
> Yotam Vaknin
>
> _______________________________________________
> SciPy-Dev mailing list
> SciPy-Dev at python.org
> https://mail.python.org/mailman/listinfo/scipy-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scipy-dev/attachments/20200227/39a95b45/attachment.html>

From toddrjen at gmail.com  Fri Feb 28 10:12:16 2020
From: toddrjen at gmail.com (Todd)
Date: Fri, 28 Feb 2020 10:12:16 -0500
Subject: [SciPy-Dev] SciPy Contribution: Gammatone Filters in
 scipy.signal
In-Reply-To: <CAPs8s8=XsXWgJqm8tD+B1e-55CFVuBfUMRREivJoo3cVjdUUUQ@mail.gmail.com>
References: <CAPs8s8nU0hn607Obv2vdpgHdj2PKvE6mFSXcO1z4cN+chFSH2w@mail.gmail.com>
 <CAPs8s8=XsXWgJqm8tD+B1e-55CFVuBfUMRREivJoo3cVjdUUUQ@mail.gmail.com>
Message-ID: <CAFpSVpKh-etEW4ZSHQ5GoYwad4VC=dYw+-278t8qvue4K7aw4Q@mail.gmail.com>

I would be extremely interested in having gammatone filters implemented.
I have been wanting it for a long time, but haven't gotten around to doing
it myself yet.  But I don't speak for the scipy core developers, they would
need to weigh in on this.

On Thu, Feb 27, 2020 at 11:16 AM Shashaank Narayanan <
shashaank.n at columbia.edu> wrote:

> Hi,
>
> Newbie here. I just wanted to follow up on the Gammatone filters for
> signal processing contribution idea that I posted recently. I would also be
> interested in contributing to SciPy by doing maintenance/bug fixes for the
> scipy.signal module.
>
>
> Thanks,
> Shashaank
>
> On Thu, Jan 30, 2020 at 5:38 PM Shashaank Narayanan <
> shashaank.n at columbia.edu> wrote:
>
>> Hello SciPy Team,
>>
>> I am new to this mailing list, and I am interested in contributing to
>> SciPy. I would like to suggest a new feature to be added to the
>> scipy.signal module: gammatone filters. Gammatone filters are becoming
>> increasingly popular in the fields of digital signal processing and music
>> analysis as it effectively models the auditory filters of the human
>> auditory system. Currently, there are very few implementations of gammatone
>> filters available for Python, and these implementations are not generalized
>> to basic finite impulse response (FIR) and infinite impulse response (IIR)
>> filters like SciPy has.
>>
>> I have written my own gammatone FIR filter using NumPy based on Malcolm
>> Slaney's 1993 paper on the topic (
>> https://engineering.purdue.edu/~malcolm/apple/tr35/PattersonsEar.pdf).
>> This paper was used for Matlab's implementation of gammatone filters. I am
>> in the process of writing a gammatone IIR filter with NumPy and SciPy.
>> Please let me know if this feature will fit with the scipy.signal module.
>> Appreciate your time and guidance.
>>
>>
>> Thanks,
>> Shashaank
>>
> _______________________________________________
> SciPy-Dev mailing list
> SciPy-Dev at python.org
> https://mail.python.org/mailman/listinfo/scipy-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scipy-dev/attachments/20200228/4928f919/attachment.html>

From tomirendo at gmail.com  Fri Feb 28 14:06:45 2020
From: tomirendo at gmail.com (yotam vaknin)
Date: Fri, 28 Feb 2020 21:06:45 +0200
Subject: [SciPy-Dev] Choosing the result of dtype for integration
In-Reply-To: <CALO6Ft66LnuxtGPx8rW=RR+g5VQ2MQAKMYT+iJ5e6zkQ3EYLRA@mail.gmail.com>
References: <CALO6Ft66LnuxtGPx8rW=RR+g5VQ2MQAKMYT+iJ5e6zkQ3EYLRA@mail.gmail.com>
Message-ID: <C942D922-2621-44E4-A384-3C2F379C857A@gmail.com>

Thanks David for your response!

I understand your desire to keep the surprising parts of the complex domain (sqrt, log) on an opt-in basis. But a lot of the non problematic parts are not opt-in at the moment. Functions with real images (with respect to the reals, like exp, sin, erf) don?t have a complex opt-in functionality. I would expect integration to fall into this category, since keeping the status quo results in behavior which is very unintuitive (as oppose to log(-1)). 

If I understand everything correctly, the current implementation of the sqrt function shouldn?t result in any problem, since sqrt doesn?t return complex values when one doesn?t ask for them, keeping the complex functionality on an opt-in basis. 

Regards,
Yotam Vaknin

> On 28 Feb 2020, at 3:46, David Hagen <david at drhagen.com> wrote:
> 
> ?
> I am all for improving handling of complex integration, but I think the current behavior matches how SciPy handles real and complex domains, namely by making the user opt into the complex domain. In Matlab log(-1.0) is 3.14i. In NumPy, log(-1.0) is NaN. NumPy and SciPy require the user to be explicit about working in the complex domain. Matlab happily takes you from the real domain to the complex domain without warning. I have done a lot of ODE integration in Matlab, and I always disliked how easily it accepted complex numbers. If I had a log somewhere in my RHS and it accidentally got a negative number, the whole thing would keep running and just return a giant block of complex nonsense. Now, this is a bit different because it is a lot harder to accidentally get a complex RHS, which is why I am only weakly in favor of maintaining the status quo.
> 
>> On Thu, Feb 27, 2020 at 5:45 AM yotam vaknin <tomirendo at gmail.com> wrote:
>> Hello list,
>> 
>> I hope I'm now visiting a very old topic, but I wanted to suggest a simple change to the integration mechanism.
>> 
>> At this moment, the dtype of the result of an integration is the dtype of the initial condition. For example, when using ?solve_ivp" to solve the initial value problem 
>> y0=np.array([1,0] )
>> dy/dt= -i H at y0
>> which is a very a standard calculation in Quantum Mechanics, the integrator will return nonsense, since the process involves complex numbers, and the initial conditions were (wrongly assigned to be) real. The solution is of course y0=np.array([1,0], dtype=complex), but this is very hard to figure out to new users coming from, for e.g., Matlab.
>> 
>> I wanted to suggest that instead of choosing the dtype using the initial condition, that the resulting dtype will be the dtype of the derivative (or better yet, derivative*dt + initial condition). This will solve the problem, and introduce minimal confusing.
>> 
>> Thanks for all of your great work!
>> 
>> Regards,
>> Yotam Vaknin
>> 
>> _______________________________________________
>> SciPy-Dev mailing list
>> SciPy-Dev at python.org
>> https://mail.python.org/mailman/listinfo/scipy-dev
> _______________________________________________
> SciPy-Dev mailing list
> SciPy-Dev at python.org
> https://mail.python.org/mailman/listinfo/scipy-dev
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scipy-dev/attachments/20200228/3796bbd1/attachment.html>

From tomirendo at gmail.com  Fri Feb 28 14:15:56 2020
From: tomirendo at gmail.com (yotam vaknin)
Date: Fri, 28 Feb 2020 21:15:56 +0200
Subject: [SciPy-Dev] Choosing the result of dtype for integration
In-Reply-To: <CALO6Ft66LnuxtGPx8rW=RR+g5VQ2MQAKMYT+iJ5e6zkQ3EYLRA@mail.gmail.com>
References: <CALO6Ft66LnuxtGPx8rW=RR+g5VQ2MQAKMYT+iJ5e6zkQ3EYLRA@mail.gmail.com>
Message-ID: <2D7450B9-6835-4367-AC46-FEBC401B8DB2@gmail.com>

Thanks David for your response!

I understand your desire to keep the surprising parts of the complex domain (sqrt, log) on an opt-in basis. But a lot of the non problematic parts are not opt-in at the moment. Functions with real images (with respect to the reals, like exp, sin, erf) don?t have a complex opt-in functionality. I would expect integration to fall into this category, since keeping the status quo results in behavior which is very unintuitive (as oppose to log(-1)). 

If I understand everything correctly, the current implementation of the sqrt function shouldn?t result in any problem, since sqrt doesn?t return complex values when one doesn?t ask for them, keeping the complex functionality on an opt-in basis. 

Regards,
Yotam Vaknin

> On 28 Feb 2020, at 3:46, David Hagen <david at drhagen.com> wrote:
> 
> ?
> I am all for improving handling of complex integration, but I think the current behavior matches how SciPy handles real and complex domains, namely by making the user opt into the complex domain. In Matlab log(-1.0) is 3.14i. In NumPy, log(-1.0) is NaN. NumPy and SciPy require the user to be explicit about working in the complex domain. Matlab happily takes you from the real domain to the complex domain without warning. I have done a lot of ODE integration in Matlab, and I always disliked how easily it accepted complex numbers. If I had a log somewhere in my RHS and it accidentally got a negative number, the whole thing would keep running and just return a giant block of complex nonsense. Now, this is a bit different because it is a lot harder to accidentally get a complex RHS, which is why I am only weakly in favor of maintaining the status quo.
> 
>> On Thu, Feb 27, 2020 at 5:45 AM yotam vaknin <tomirendo at gmail.com> wrote:
>> Hello list,
>> 
>> I hope I'm now visiting a very old topic, but I wanted to suggest a simple change to the integration mechanism.
>> 
>> At this moment, the dtype of the result of an integration is the dtype of the initial condition. For example, when using ?solve_ivp" to solve the initial value problem 
>> y0=np.array([1,0] )
>> dy/dt= -i H at y0
>> which is a very a standard calculation in Quantum Mechanics, the integrator will return nonsense, since the process involves complex numbers, and the initial conditions were (wrongly assigned to be) real. The solution is of course y0=np.array([1,0], dtype=complex), but this is very hard to figure out to new users coming from, for e.g., Matlab.
>> 
>> I wanted to suggest that instead of choosing the dtype using the initial condition, that the resulting dtype will be the dtype of the derivative (or better yet, derivative*dt + initial condition). This will solve the problem, and introduce minimal confusing.
>> 
>> Thanks for all of your great work!
>> 
>> Regards,
>> Yotam Vaknin
>> 
>> _______________________________________________
>> SciPy-Dev mailing list
>> SciPy-Dev at python.org
>> https://mail.python.org/mailman/listinfo/scipy-dev
> _______________________________________________
> SciPy-Dev mailing list
> SciPy-Dev at python.org
> https://mail.python.org/mailman/listinfo/scipy-dev
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scipy-dev/attachments/20200228/b1496925/attachment.html>

From david at drhagen.com  Fri Feb 28 19:01:03 2020
From: david at drhagen.com (David Hagen)
Date: Fri, 28 Feb 2020 19:01:03 -0500
Subject: [SciPy-Dev] Choosing the result of dtype for integration
In-Reply-To: <C942D922-2621-44E4-A384-3C2F379C857A@gmail.com>
References: <CALO6Ft66LnuxtGPx8rW=RR+g5VQ2MQAKMYT+iJ5e6zkQ3EYLRA@mail.gmail.com>
 <C942D922-2621-44E4-A384-3C2F379C857A@gmail.com>
Message-ID: <CALO6Ft7iCbJoecnE6oKRX1ifSaiZHRLgyXxh5Taq=o=C8ob-5w@mail.gmail.com>

>  Functions with real images (with respect to the reals, like exp, sin,
erf) don?t have a complex opt-in functionality. I would expect integration
to fall into this category

I agree with the principle here, but differ on the conclusion. `exp`,
`sin`, etc. return a real value when given a real value; if you want a
complex output, provide a complex input. I think the same should go for
`solve_ivp`; if you want a complex `y`, provide a complex `y0`.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scipy-dev/attachments/20200228/e4d3139b/attachment-0001.html>