From harrigan.matthew at gmail.com  Thu Nov  1 06:51:21 2018
From: harrigan.matthew at gmail.com (Matthew Harrigan)
Date: Thu, 1 Nov 2018 06:51:21 -0400
Subject: [Numpy-discussion] asanyarray vs. asarray
In-Reply-To: <CABL7CQh=HmhQC=eBzc0XCQ+K41T_3+59BjWQBjya2FkEVO-TSQ@mail.gmail.com>
References: <a7e76515-95ab-e867-dc96-92c54d8b0952@gmail.com>
 <24100c7f-20fd-4eed-99b0-d37660f52223@Canary>
 <CAJNV+9vH2o_2KCiMVc6S1ww=-NXpfGK39UoJVKKJNmv7pN602A@mail.gmail.com>
 <CAEQ_TvezgD7_KNB5PD_zjKQ=j+dN-k7BkEzNVVM9+VNAw8b85w@mail.gmail.com>
 <cb1364db-9e46-4614-a1df-d5dc33ad2c64@Canary>
 <CABL7CQgLqiTAO4H+3MUZzDbF3eR_njO4kH9UD1C_t27gnkGYYw@mail.gmail.com>
 <CABL7CQgXNt6ZGrDF8OJz73PDGJmbvaKs6q-3ArFkeLF86bpSEg@mail.gmail.com>
 <CAJNV+9uXs9at6RYyymQcoGpP4cvr+waVitHWXnsy4MciE+6RnA@mail.gmail.com>
 <CAFMmPGNbhJu1vrREnugi54gWHK9MPqFenRMsnscau=tjwVn8Hw@mail.gmail.com>
 <CALGmxEJCMfm7J8BWbHiXG0LTJxvk=5vp9X=42psY7LiA3uL8fA@mail.gmail.com>
 <CABL7CQhn45aM9rUYedLAOzFMb2wRLRJMbwE8taWEuLrqSGpmCQ@mail.gmail.com>
 <CAL1kJvC4Py+ikqp6bxyx-_qm9aD_mEhsCFshCR4QXWZt1YGMNg@mail.gmail.com>
 <CAEQ_TvdLV5KEn7XvKqc_1m2zNkE7FZR_LaXR7MutPS2ULC0iZQ@mail.gmail.com>
 <CABL7CQh=HmhQC=eBzc0XCQ+K41T_3+59BjWQBjya2FkEVO-TSQ@mail.gmail.com>
Message-ID: <CAOfRF=iiTLgnOcEwsmuE9QvQRzMjebVhetUv9qvHdoDF+7rFEw@mail.gmail.com>

>
> I don't think so. Dtypes have nothing to do with a whole set of use cases
> that add extra methods or attributes. Random made-up example: user has a
> system with 1000 sensor signals, some of which should be treated with
> robust statistics for <reasons like unreliable hardware>. So user writes a
> subclass robust_ndarray, adds a bunch of methods like median/iqr/mad, and
> uses isinstance checks in functions that accept both ndarray and
> robust_ndarray to figure out how to preprocess sensor signals.
>
> Of course you can do everything you can do with subclasses also in other
> ways, but such "let's add some methods or attributes" are much more common
> (I think, hard to prove) than "let's change how indexing or multiplication
> works" in end user code.
>
> Cheers,
> Ralf
>

The build on Ralf's thought, a common subclass use case would be to add
logging to various methods and attributes.  That might actually be useful
for ndarray for understanding what is under the hood of some function in a
downstream project.  It would satisfy SOLID and not be related at all to
dtype subclasses.


On Wed, Oct 31, 2018 at 8:28 PM Ralf Gommers <ralf.gommers at gmail.com> wrote:

>
>
> On Tue, Oct 30, 2018 at 2:22 PM Stephan Hoyer <shoyer at gmail.com> wrote:
>
>> On Mon, Oct 29, 2018 at 9:49 PM Eric Wieser <wieser.eric+numpy at gmail.com>
>> wrote:
>>
>>> The latter - changing the behavior of multiplication breaks the
>>> principle.
>>>
>>> But this is not the main reason for deprecating matrix - almost all of
>>> the problems I?ve seen have been caused by the way that matrices behave
>>> when sliced. The way that m[i][j] and m[i,j] are different is just one
>>> example of this, the fact that they must be 2d is another.
>>>
>>> Matrices behaving differently on multiplication isn?t super different in
>>> my mind to how string arrays fail to multiply at all.
>>>
>>> Eric
>>>
>> It's certainly fine for arithmetic to work differently on an element-wise
>> basis or even to error. But np.matrix changes the shape of results from
>> various ndarray operations (e.g., both multiplication and indexing), which
>> is more than any dtype can do.
>>
>> The Liskov substitution principle (LSP) suggests that the set of
>> reasonable ndarray subclasses are exactly those that could also in
>> principle correspond to a new dtype.
>>
>
> I don't think so. Dtypes have nothing to do with a whole set of use cases
> that add extra methods or attributes. Random made-up example: user has a
> system with 1000 sensor signals, some of which should be treated with
> robust statistics for <reasons like unreliable hardware>. So user writes a
> subclass robust_ndarray, adds a bunch of methods like median/iqr/mad, and
> uses isinstance checks in functions that accept both ndarray and
> robust_ndarray to figure out how to preprocess sensor signals.
>
> Of course you can do everything you can do with subclasses also in other
> ways, but such "let's add some methods or attributes" are much more common
> (I think, hard to prove) than "let's change how indexing or multiplication
> works" in end user code.
>
> Cheers,
> Ralf
>
>
>
>> Of np.ndarray subclasses in wide-spread use, I think only the various
>> "array with units" types come close satisfying this criteria. They only
>> fall short insofar as they present a misleading dtype (without unit
>> information).
>>
>> The main problem with subclassing for numpy.ndarray is that it guarantees
>> too much: a large set of operations/methods along with a specific memory
>> layout exposed as part of its public API. Worse, ndarray itself is a little
>> quirky (e.g., with indexing, and its handling of scalars vs. 0d arrays). In
>> practice, it's basically impossible to layer on complex behavior with these
>> exact semantics, so only extremely minimal ndarray subclasses don't violate
>> LSP.
>>
>> Once we have more easily extended dtypes, I suspect most of the good use
>> cases for subclassing will have gone away.
>> _______________________________________________
>> NumPy-Discussion mailing list
>> NumPy-Discussion at python.org
>> https://mail.python.org/mailman/listinfo/numpy-discussion
>>
> _______________________________________________
> NumPy-Discussion mailing list
> NumPy-Discussion at python.org
> https://mail.python.org/mailman/listinfo/numpy-discussion
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20181101/2cd7845f/attachment-0001.html>

From m.h.vankerkwijk at gmail.com  Thu Nov  1 15:06:29 2018
From: m.h.vankerkwijk at gmail.com (Marten van Kerkwijk)
Date: Thu, 1 Nov 2018 15:06:29 -0400
Subject: [Numpy-discussion] asanyarray vs. asarray
In-Reply-To: <CAOfRF=iiTLgnOcEwsmuE9QvQRzMjebVhetUv9qvHdoDF+7rFEw@mail.gmail.com>
References: <a7e76515-95ab-e867-dc96-92c54d8b0952@gmail.com>
 <24100c7f-20fd-4eed-99b0-d37660f52223@Canary>
 <CAJNV+9vH2o_2KCiMVc6S1ww=-NXpfGK39UoJVKKJNmv7pN602A@mail.gmail.com>
 <CAEQ_TvezgD7_KNB5PD_zjKQ=j+dN-k7BkEzNVVM9+VNAw8b85w@mail.gmail.com>
 <cb1364db-9e46-4614-a1df-d5dc33ad2c64@Canary>
 <CABL7CQgLqiTAO4H+3MUZzDbF3eR_njO4kH9UD1C_t27gnkGYYw@mail.gmail.com>
 <CABL7CQgXNt6ZGrDF8OJz73PDGJmbvaKs6q-3ArFkeLF86bpSEg@mail.gmail.com>
 <CAJNV+9uXs9at6RYyymQcoGpP4cvr+waVitHWXnsy4MciE+6RnA@mail.gmail.com>
 <CAFMmPGNbhJu1vrREnugi54gWHK9MPqFenRMsnscau=tjwVn8Hw@mail.gmail.com>
 <CALGmxEJCMfm7J8BWbHiXG0LTJxvk=5vp9X=42psY7LiA3uL8fA@mail.gmail.com>
 <CABL7CQhn45aM9rUYedLAOzFMb2wRLRJMbwE8taWEuLrqSGpmCQ@mail.gmail.com>
 <CAL1kJvC4Py+ikqp6bxyx-_qm9aD_mEhsCFshCR4QXWZt1YGMNg@mail.gmail.com>
 <CAEQ_TvdLV5KEn7XvKqc_1m2zNkE7FZR_LaXR7MutPS2ULC0iZQ@mail.gmail.com>
 <CABL7CQh=HmhQC=eBzc0XCQ+K41T_3+59BjWQBjya2FkEVO-TSQ@mail.gmail.com>
 <CAOfRF=iiTLgnOcEwsmuE9QvQRzMjebVhetUv9qvHdoDF+7rFEw@mail.gmail.com>
Message-ID: <CAJNV+9txfeMR0VrX_GX4kVf1MSn79jww04OXHZiXbORtbVcVVA@mail.gmail.com>

The substitution principle is interesting (and, being trained as an
astronomer, not a computer scientist, I had not heard of it before). I
think matrix is indeed obviously wrong here (with indexing being more
annoying, but multiplication being a good example as well).

Perhaps more interesting as an example to consider is MaskedArray, which is
much closer to a sensible subclass, though different from Quantity in that
what is masked can itself be an ndarray subclass. In a sense, it is more of
a container class, in which the operations are done on what is inside it,
with some care taken about which elements are fixed. This becomes quite
clear when one thinks of implementing __array_ufunc__ or
__array_function__: for Quantity, calling super after dealing with the
units is very logical, for MaskedArray, it makes more sense to call the
(universal) function again on the contents [1].

For this particular class, if reimplemented, it might make most sense as a
"mixin" since its attributes depend both on the masked class (.mask, etc.)
and on what is being masked (say, .unit for a quantity). Thus, the final
class might be an auto-generated new class (e.g.,
MaskedQuantity(MaskedArray, Quantity)). We have just added a new
Distribution class to astropy which is based on this idea [2] (since this
uses casting from structured dtypes which hold the samples to real arrays
on which functions are evaluated, this probably could be done just as well
or better with more flexible dtypes, but we have to deal with what's
available in the real world, not the ideal one...).

-- Marten

[1]
http://www.numpy.org/neps/nep-0013-ufunc-overrides.html#subclass-hierarchies
[2] https://github.com/astropy/astropy/pull/6945
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20181101/3539a13e/attachment.html>

From m.h.vankerkwijk at gmail.com  Sun Nov  4 10:59:24 2018
From: m.h.vankerkwijk at gmail.com (Marten van Kerkwijk)
Date: Sun, 4 Nov 2018 10:59:24 -0500
Subject: [Numpy-discussion] Should unique types of all arguments be passed
 on in __array_function__?
Message-ID: <CAJNV+9vx9AJpJ3ZqTHMt9QbEH5kjsz5JSbJZsPB3DaY__yKgZA@mail.gmail.com>

Hi All,

While thinking about implementations using __array_function__, I wondered
whether the "types" argument passed on is not defined too narrowly.
Currently, it only contains the types of arguments that provide
__array_ufunc__, but  wouldn't it make more sense to provide the unique
types of all arguments, independently of whether those types have defined
__array_ufunc__? It would seem quite useful for any override to know, e.g.,
whether a string or an integer is passed on.

I thought of this partially as I was wondering how an implementation for
ndarray itself would look like. For that, it is definitely useful to know
all unique types, since if it is only ndarray, no casting whatsoever needs
to be done, while if there are integers, lists, etc, an attempt has to be
made to turn these into arrays (i.e., the `as[any]array` calls currently
present in the implementations, which really more logically are part of
`ndarray.__array_function__` dispatch).

Should we change this? It is quite trivially done, but perhaps I am missing
a reason for omitting the non-override types.

All the best,

Marten
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20181104/30ed894f/attachment.html>

From m.h.vankerkwijk at gmail.com  Sun Nov  4 11:44:06 2018
From: m.h.vankerkwijk at gmail.com (Marten van Kerkwijk)
Date: Sun, 4 Nov 2018 11:44:06 -0500
Subject: [Numpy-discussion] Implementations of ndarray.__array_function__
 (and ndarray.__array_ufunc__)
Message-ID: <CAJNV+9sU-Cb7qv5n2q-LR_29koV2Lqhpi_YWymUGjvg_YuZCfg@mail.gmail.com>

Hi again,

Another thought about __array_function__, this time about the
implementation for ndarray. In it, we currently check whether any of the
types define a (different) __array_function__, and, if so, give up. This
seems too strict: I think that, at least in principle, subclasses should be
allowed through even if they override __array_function__.

This thought was triggered by Travis pointing to the Liskov substitution
principle [1], that code written for a given type should just work on a
(properly written) subclass. This suggests `ndarray` should not exclude
subclasses even if they override __array_function__, since if the subclass
does not work that way, it can already ensure an error is raised since it
knows it is called first.

Indeed, this is also how python itself works: if, eg., I subclass list as
follows:
```
class MyList(list):
    def __radd__(self, other):
        return NotImplemented
```
then any `list + mylist` will just concatenate the lists, even though
`MyList.__radd__` explicitly tells it cannot do it (it returning
`NotImplemented` means that  `list.__add__` gets a change).

The reason that we do not already follow this logic may be that currently
`ndarray.__array_function__` ends by calling the public function, which
will lead to infinite recursion if there is a subclass that overrides
__array_function__ and returns NotImplemented. However, inside
ndarray.__array_function__, there is no real reason to call the public
function - one might as well just call the implementation, in which case
this is not a problem.

Does the above make sense? I realize that the same would be true for
`__array_ufunc__`, though there the situation is slightly trickier since it
is not as easy to bypass any further override checks. Nevertheless, it does
seem like it would be correct to do the same there. (And if we agree this
is the case, I'd quite happily implement it -- with the merger of
multiarray and umath it has become much easier to do.)

All the best,

Marten

[1] https://en.wikipedia.org/wiki/Liskov_substitution_principle
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20181104/b52ff32f/attachment.html>

From charlesr.harris at gmail.com  Sun Nov  4 12:31:05 2018
From: charlesr.harris at gmail.com (Charles R Harris)
Date: Sun, 4 Nov 2018 10:31:05 -0700
Subject: [Numpy-discussion] NumPy 1.15.4 release
Message-ID: <CAB6mnx+h1BAVqjgDngO1L1+dZuWLgiZOGjCf_-pMbs2ZG02Xog@mail.gmail.com>

Hi All,

On behalf of the NumPy team, I am pleased to announce the release of NumPy
1.15.4. This is a bugfix release for bugs and regressions reported
following the 1.15.3 release.  The most noticeable fix is probably having a
boolean type fill value for masked arrays after the use of `==` and `!=`.
The Python versions supported by this release are 2.7, 3.4-3.7. Wheels for
this release can be downloaded from PyPI
<https://pypi.org/project/numpy/1.15.4/>, source archives are available
from Github <https://github.com/numpy/numpy/releases/tag/v1.15.4>.

Compatibility Note
==================

The NumPy 1.15.x OS X wheels released on PyPI no longer contain 32-bit
binaries.  That will also be the case in future releases. See `#11625 <
https://github.com/numpy/numpy/issues/11625>`__ for the related
discussion.  Those needing 32-bit support should look elsewhere or build
from source.

Contributors
============

A total of 4 people contributed to this release.  People with a "+" by their
names contributed a patch for the first time.

   - Charles Harris
   - Matti Picus
   - Sebastian Berg
   - bbbbbbbbba +

Pull requests merged
====================

A total of 4 pull requests were merged for this release.

   - `#12296 <https://github.com/numpy/numpy/pull/12296>`__: BUG: Dealloc
   cached buffer info
   - `#12297 <https://github.com/numpy/numpy/pull/12297>`__: BUG: Fix fill
   value in masked array '==' and '!=' ops.
   - `#12307 <https://github.com/numpy/numpy/pull/12307>`__: DOC: Correct
   the default value of `optimize` in `numpy.einsum`
   - `#12320 <https://github.com/numpy/numpy/pull/12320>`__: REL: Prepare
   for the NumPy 1.15.4 release

Cheers,

Charles Harris
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20181104/66742fb0/attachment.html>

From charlesr.harris at gmail.com  Sun Nov  4 13:04:36 2018
From: charlesr.harris at gmail.com (Charles R Harris)
Date: Sun, 4 Nov 2018 11:04:36 -0700
Subject: [Numpy-discussion] Prep for NumPy 1.16.0 branch
Message-ID: <CAB6mnxJUty767___L5pMOrAKVE-SamvHLVCqTXz3dDvDHum8sQ@mail.gmail.com>

Hi All,

Time to begin looking forward to the NumPy 1.16.x branch. I think there are
three main topics to address:


   1. current PRs that need review and merging,
   2. critical fixes that need to be made,
   3. status of `__array_function__`.

The last probably needs some discussion. `__array_fuction__` seems to be
working at this point, but does cause noticeable slowdowns in some function
calls. I don't know if those slowdowns are significant in practice, the
only way to discover that may be to make the release, or at least thorough
testing of the release candidates, but we should at least discuss them and
possible workarounds if needed. Trying to have things in good shape is
important because 1.16.x will be the last release that supports Python 2.7,
and even though we will be maintaining it for the next year, that will be
easier if it is stable. As to the first two topics, I think we should try
to be conservative at this point and look mostly for bug fixes and
documentation updates.

Thoughts?

Chuck
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20181104/34e7aa97/attachment.html>

From m.h.vankerkwijk at gmail.com  Sun Nov  4 13:30:47 2018
From: m.h.vankerkwijk at gmail.com (Marten van Kerkwijk)
Date: Sun, 4 Nov 2018 13:30:47 -0500
Subject: [Numpy-discussion] Prep for NumPy 1.16.0 branch
In-Reply-To: <CAB6mnxJUty767___L5pMOrAKVE-SamvHLVCqTXz3dDvDHum8sQ@mail.gmail.com>
References: <CAB6mnxJUty767___L5pMOrAKVE-SamvHLVCqTXz3dDvDHum8sQ@mail.gmail.com>
Message-ID: <CAJNV+9v+F+U--3H91odm=EU7Y=C1Z2WHhD-w34bmOqhZorF7yg@mail.gmail.com>

Hi Chuck,

For `__array_function__`, there was some discussion in
https://github.com/numpy/numpy/issues/12225 that for 1.16 we might want to
follow after all Nathaniel's suggestion of using an environment variable or
so to opt in (since introspection breaks on python2 with our wrapped
implementations).  Given also the possibly significant hit in performance,
this may be the best option.
All the best,

Marten
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20181104/dee99437/attachment.html>

From matti.picus at gmail.com  Sun Nov  4 16:48:19 2018
From: matti.picus at gmail.com (Matti Picus)
Date: Sun, 4 Nov 2018 16:48:19 -0500
Subject: [Numpy-discussion] Prep for NumPy 1.16.0 branch
In-Reply-To: <CAB6mnxJUty767___L5pMOrAKVE-SamvHLVCqTXz3dDvDHum8sQ@mail.gmail.com>
References: <CAB6mnxJUty767___L5pMOrAKVE-SamvHLVCqTXz3dDvDHum8sQ@mail.gmail.com>
Message-ID: <84bd348f-d85f-c7ee-3795-eea3bcb2f734@gmail.com>


On 4/11/18 8:04 pm, Charles R Harris wrote:
> Hi All,
>
> Time to begin looking forward to the NumPy 1.16.x branch. I think 
> there are three main topics to address:
>
>  1. current PRs that need review and merging,
>  2. critical fixes that need to be made,
>  3. status of `__array_function__`.
>
> The last probably needs some discussion. `__array_fuction__` seems to 
> be working at this point, but does cause noticeable slowdowns in some 
> function calls. I don't know if those slowdowns are significant in 
> practice, the only way to discover that may be to make the release, or 
> at least thorough testing of the release candidates, but we should at 
> least discuss them and possible workarounds if needed. Trying to have 
> things in good shape is important because 1.16.x will be the last 
> release that supports Python 2.7, and even though we will be 
> maintaining it for the next year, that will be easier if it is stable. 
> As to the first two topics, I think we should try to be conservative 
> at this point and look mostly for bug fixes and documentation updates.
>
> Thoughts?
>
> Chuck
>

Beyond things with the 1.16 milestone, it would be nice to address the 
structured array cleanup 
https://gist.github.com/ahaldane/6cd44886efb449f9c8d5ea012747323b and to 
get the matmul-as-ufunc https://github.com/numpy/numpy/pull/12219 merged


Matti


From shoyer at gmail.com  Sun Nov  4 19:51:07 2018
From: shoyer at gmail.com (Stephan Hoyer)
Date: Sun, 4 Nov 2018 16:51:07 -0800
Subject: [Numpy-discussion] Should unique types of all arguments be
 passed on in __array_function__?
In-Reply-To: <CAJNV+9vx9AJpJ3ZqTHMt9QbEH5kjsz5JSbJZsPB3DaY__yKgZA@mail.gmail.com>
References: <CAJNV+9vx9AJpJ3ZqTHMt9QbEH5kjsz5JSbJZsPB3DaY__yKgZA@mail.gmail.com>
Message-ID: <CAEQ_TvfGJVrZd+wWOnDYgOSpkrwQ+7j0-esuXyMXedjwNsOGAA@mail.gmail.com>

On Sun, Nov 4, 2018 at 8:03 AM Marten van Kerkwijk <
m.h.vankerkwijk at gmail.com> wrote:

> I thought of this partially as I was wondering how an implementation for
> ndarray itself would look like. For that, it is definitely useful to know
> all unique types, since if it is only ndarray, no casting whatsoever needs
> to be done, while if there are integers, lists, etc, an attempt has to be
> made to turn these into arrays
>

OK, so hypothetically we could invoke versions of each the numpy function
that doesn't call `as[any]array`, and this would slightly speed-up
subclasses that call super().__array_function__?

The former feels pretty unlikely for now -- and would be speeding up a
somewhat niche use-case (more niche even than __array_function__ in
general) -- but perhaps I could be convinced.


> (i.e., the `as[any]array` calls currently present in the implementations,
> which really more logically are part of `ndarray.__array_function__`
> dispatch).
>

I can sort of see the reasoning for this, but I suspect the overhead of
actually calling `ndarray.__array_function__` as part of calling every
NumPy functions would be prohibitive. It would mean that __array_function__
attributes get checked twice, once for dispatching and once in
`ndarray.__array_function__`.

It would also mean that `ndarray.__array_function__` would need to grow a
general purpose coercion mechanism for converting array-like arguments into
ndarray objects. I suspect this isn't really possible given the diversity
of function signatures in NumPy, e.g., consider the handling of lists in
np.block() (recurse) vs. np.concatenate (pass through) vs ufuncs (coerce to
ndarray). The best we could do would be add another special function like
dispatchers for handling coercion for each specific NumPy functions.

Should we change this? It is quite trivially done, but perhaps I am missing
> a reason for omitting the non-override types.
>

Realistically, without these other changes in NumPy, how would this improve
code using __array_function__? From a general purpose dispatching
perspective, are there cases where you'd want to return NotImplemented
based on types that don't implement __array_function__?

I guess this might help if your alternative array class is super-explicit,
and doesn't automatically call `asmyarray()` on each argument. You could
rely on __array_function__ to return NotImplement (and thus raise
TypeError) rather than type checking in every function you write for your
alternative arrays.

One minor downside would speed: now __array_function__ implementations need
to check a longer list of types.

Another minor downside: if users follow the example of
NDArrayOperatorsMixin docstring, they would now need to explicitly list all
of the scalar types (without __array_function__) that they support,
including builtin types like int and type(None). I suppose this ties into
our recommended best practices for doing type checking in
__array_ufunc__/__array_function__ implementations, which should probably
be updated regardless:
https://github.com/numpy/numpy/issues/12258#issuecomment-432858949

Best,
Stephan
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20181104/d6cdd646/attachment-0001.html>

From shoyer at gmail.com  Sun Nov  4 20:16:12 2018
From: shoyer at gmail.com (Stephan Hoyer)
Date: Sun, 4 Nov 2018 17:16:12 -0800
Subject: [Numpy-discussion] Prep for NumPy 1.16.0 branch
In-Reply-To: <CAJNV+9v+F+U--3H91odm=EU7Y=C1Z2WHhD-w34bmOqhZorF7yg@mail.gmail.com>
References: <CAB6mnxJUty767___L5pMOrAKVE-SamvHLVCqTXz3dDvDHum8sQ@mail.gmail.com>
 <CAJNV+9v+F+U--3H91odm=EU7Y=C1Z2WHhD-w34bmOqhZorF7yg@mail.gmail.com>
Message-ID: <CAEQ_Tvf-cuKO5gcNfrXoMNPqnVra1g8dExWbCWRP2sUTKDk99Q@mail.gmail.com>

On Sun, Nov 4, 2018 at 10:32 AM Marten van Kerkwijk <
m.h.vankerkwijk at gmail.com> wrote:

> Hi Chuck,
>
> For `__array_function__`, there was some discussion in
> https://github.com/numpy/numpy/issues/12225 that for 1.16 we might want
> to follow after all Nathaniel's suggestion of using an environment variable
> or so to opt in (since introspection breaks on python2 with our wrapped
> implementations).  Given also the possibly significant hit in performance,
> this may be the best option.
> All the best,
>
> Marten
>

I am also leaning towards this right now, depending on how long we plan to
wait for releasing 1.16. It will take us at least a little while to sort
out performance issues for __array_function__, I'd guess at least a few
weeks. Then a blocker still might turn up during the release candidate
process (though I think we've found most of the major bugs / downstream
issues already through tests on NumPy's dev branch).

Overall, it does feels a little misguided to rush in a change as pervasive
as __array_function__ for a long term support release. If we exclude
__array_function__ I expect the whole release process for 1.16 would go
much smoother. We might even try to get 1.17 out faster than usual, so we
can minimize the number of additional changes besides __array_function__
and going Python 3 only -- that's already a good bit of change.

Note that if we make this change (reverting __array_function__), we'll need
to revisit where we put a few deprecation warnings -- these will need to be
restored into function bodies, not their dispatcher functions.

Also: it would be really nice if we get matmul-as-ufunc in before (or at
the same time) as __array_function__, so we have a complete story about it
being possible to override everything in NumPy. This is another argument
for delaying __array_function__, if matmul-as-ufunc can't make it in time
for 1.16.

Best,
Stephan
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20181104/92996e56/attachment.html>

From shoyer at gmail.com  Sun Nov  4 20:57:01 2018
From: shoyer at gmail.com (Stephan Hoyer)
Date: Sun, 4 Nov 2018 17:57:01 -0800
Subject: [Numpy-discussion] Implementations of
 ndarray.__array_function__ (and ndarray.__array_ufunc__)
In-Reply-To: <CAJNV+9sU-Cb7qv5n2q-LR_29koV2Lqhpi_YWymUGjvg_YuZCfg@mail.gmail.com>
References: <CAJNV+9sU-Cb7qv5n2q-LR_29koV2Lqhpi_YWymUGjvg_YuZCfg@mail.gmail.com>
Message-ID: <CAEQ_TvejSV12DKGijt=TH83ux1qcQuQ87yAt8w6Cgsnxkdo0BA@mail.gmail.com>

On Sun, Nov 4, 2018 at 8:45 AM Marten van Kerkwijk <
m.h.vankerkwijk at gmail.com> wrote:

> Does the above make sense? I realize that the same would be true for
> `__array_ufunc__`, though there the situation is slightly trickier since it
> is not as easy to bypass any further override checks. Nevertheless, it does
> seem like it would be correct to do the same there. (And if we agree this
> is the case, I'd quite happily implement it -- with the merger of
> multiarray and umath it has become much easier to do.)
>

Marten actually implemented a draft version of this already in
https://github.com/numpy/numpy/pull/12328 :). I found reading over the PR
helpful for understand this proposal.

I guess the practical import of this change is that it makes it (much?)
easier to write __array_function__ for ndarray subclasses: if there's a
function where NumPy's default function works fine, you don't need to
bother with returning anything other than NotImplemented from
__array_function__. It's sort of like NotImplementedButCoercible, but only
for ndarray subclasses.

One minor downside is that this might make it harder to eventually
deprecate and/or contemplate removing checks for 'mean' methods in
functions like np.mean(), because __array_function__ implementers might
still be relying on this.

But so far, I think this makes sense.

The PR includes additional changes to np.core.overrides, but I'm not sure
if those are actually required here (or rather only possible due to this
change). I guess they are needed if you want to be able to count on
ndarray.__array_function__ being called after subclass __array_function__
methods.

I'm not sure I like this part: it means that ndarray.__array_function__
actually gets called when other arguments implement __array_function__. For
interactions with objects that aren't ndarray subclasses this is entirely
pointless and would unnecessarily slow things down, since
ndarray._array_function__ will always return NotImplemented.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20181104/ea1ece2f/attachment.html>

From charlesr.harris at gmail.com  Sun Nov  4 22:02:24 2018
From: charlesr.harris at gmail.com (Charles R Harris)
Date: Sun, 4 Nov 2018 20:02:24 -0700
Subject: [Numpy-discussion] Prep for NumPy 1.16.0 branch
In-Reply-To: <CAEQ_Tvf-cuKO5gcNfrXoMNPqnVra1g8dExWbCWRP2sUTKDk99Q@mail.gmail.com>
References: <CAB6mnxJUty767___L5pMOrAKVE-SamvHLVCqTXz3dDvDHum8sQ@mail.gmail.com>
 <CAJNV+9v+F+U--3H91odm=EU7Y=C1Z2WHhD-w34bmOqhZorF7yg@mail.gmail.com>
 <CAEQ_Tvf-cuKO5gcNfrXoMNPqnVra1g8dExWbCWRP2sUTKDk99Q@mail.gmail.com>
Message-ID: <CAB6mnx+C0m=_fhpcKOjDF=yfsu_zUKDYh1TGnPRNuU5p9KvkZg@mail.gmail.com>

On Sun, Nov 4, 2018 at 6:16 PM Stephan Hoyer <shoyer at gmail.com> wrote:

> On Sun, Nov 4, 2018 at 10:32 AM Marten van Kerkwijk <
> m.h.vankerkwijk at gmail.com> wrote:
>
>> Hi Chuck,
>>
>> For `__array_function__`, there was some discussion in
>> https://github.com/numpy/numpy/issues/12225 that for 1.16 we might want
>> to follow after all Nathaniel's suggestion of using an environment variable
>> or so to opt in (since introspection breaks on python2 with our wrapped
>> implementations).  Given also the possibly significant hit in performance,
>> this may be the best option.
>> All the best,
>>
>> Marten
>>
>
> I am also leaning towards this right now, depending on how long we plan to
> wait for releasing 1.16. It will take us at least a little while to sort
> out performance issues for __array_function__, I'd guess at least a few
> weeks. Then a blocker still might turn up during the release candidate
> process (though I think we've found most of the major bugs / downstream
> issues already through tests on NumPy's dev branch).
>

My tentative schedule is to branch in about two weeks, then allow 2 weeks
of testing for rc1, possibly another two weeks for rc2, and then a final.
so possibly about six weeks to final release. That leaves 2 to 4 weeks of
slack before 2019.


> Overall, it does feels a little misguided to rush in a change as pervasive
> as __array_function__ for a long term support release. If we exclude
> __array_function__ I expect the whole release process for 1.16 would go
> much smoother. We might even try to get 1.17 out faster than usual, so we
> can minimize the number of additional changes besides __array_function__
> and going Python 3 only -- that's already a good bit of change.
>

I would like to get 1.17 out a bit early. I'm not sure how many backwards
incompatible changes we want to have in the first post python2 release. My
initial thoughts are to drop Python 2.7 testing, go to C99, and get the new
fft in. Beyond that, I'm hesitant to start tearing out all the Python2
special casing in the first new release, although that could certainly be
the main task for 1.17 and would clean up the code considerably. It might
also be a good time to catch up on changing deprecations to errors.
Thoughts on how to proceed are welcome.


> Note that if we make this change (reverting __array_function__), we'll
> need to revisit where we put a few deprecation warnings -- these will need
> to be restored into function bodies, not their dispatcher functions.
>
> Also: it would be really nice if we get matmul-as-ufunc in before (or at
> the same time) as __array_function__, so we have a complete story about it
> being possible to override everything in NumPy. This is another argument
> for delaying __array_function__, if matmul-as-ufunc can't make it in time
> for 1.16.
>

That's two votes for matmul-as-ufunc. How much would it cost to simply make
__array_function__ a nop?

Chuck
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20181104/da4a1c3e/attachment-0001.html>

From mark.harfouche at gmail.com  Sun Nov  4 22:34:51 2018
From: mark.harfouche at gmail.com (Mark Harfouche)
Date: Sun, 4 Nov 2018 22:34:51 -0500
Subject: [Numpy-discussion] out parameter for np.fromfile
Message-ID: <CAC=AwPy=4+tw=fzgzY4b8pwh8-EbhYDM9uVFnuCJ2Jj7PHy9KA@mail.gmail.com>

I was wondering what would your thoughts be on adding an output parameter
to np.fromfile?

The advantage would be when interfacing with executables like ffmpeg which
are arguably easier to use by calling them as a subprocess compared to a
shared library in python.

Having the output parameter in np.fromfile would enable pre-allocation of
large arrays that are reused during the computation of new image frames
when decoding large video files.

Thoughts are appreciated!

Mark
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20181104/d882dbb2/attachment.html>

From mark.harfouche at gmail.com  Sun Nov  4 22:43:04 2018
From: mark.harfouche at gmail.com (Mark Harfouche)
Date: Sun, 4 Nov 2018 22:43:04 -0500
Subject: [Numpy-discussion] Prep for NumPy 1.16.0 branch
In-Reply-To: <CAB6mnx+C0m=_fhpcKOjDF=yfsu_zUKDYh1TGnPRNuU5p9KvkZg@mail.gmail.com>
References: <CAB6mnxJUty767___L5pMOrAKVE-SamvHLVCqTXz3dDvDHum8sQ@mail.gmail.com>
 <CAJNV+9v+F+U--3H91odm=EU7Y=C1Z2WHhD-w34bmOqhZorF7yg@mail.gmail.com>
 <CAEQ_Tvf-cuKO5gcNfrXoMNPqnVra1g8dExWbCWRP2sUTKDk99Q@mail.gmail.com>
 <CAB6mnx+C0m=_fhpcKOjDF=yfsu_zUKDYh1TGnPRNuU5p9KvkZg@mail.gmail.com>
Message-ID: <CAC=AwPzG-G22pbJ_C9DQ3qNn2TjGy=bEzf6miEZmEp8awHgx_g@mail.gmail.com>

> Thoughts on how to proceed are welcome.

I've been involved in scikit-image and that project  tore out the python2
only code rather quickly after 2.7 support was dropped. I think it caused a
few hiccups when backporting bugfixes. I imagine that `1.16.1` and `1.16.2`
releases will come out quickly and as such, I think removing `if else`
statements for python2 immediately after `1.16` is released will cause
annoyances in the first few months bugs are being ironed out.

My 2cents.

On Sun, Nov 4, 2018 at 10:04 PM Charles R Harris <charlesr.harris at gmail.com>
wrote:

>
>
> On Sun, Nov 4, 2018 at 6:16 PM Stephan Hoyer <shoyer at gmail.com> wrote:
>
>> On Sun, Nov 4, 2018 at 10:32 AM Marten van Kerkwijk <
>> m.h.vankerkwijk at gmail.com> wrote:
>>
>>> Hi Chuck,
>>>
>>> For `__array_function__`, there was some discussion in
>>> https://github.com/numpy/numpy/issues/12225 that for 1.16 we might want
>>> to follow after all Nathaniel's suggestion of using an environment variable
>>> or so to opt in (since introspection breaks on python2 with our wrapped
>>> implementations).  Given also the possibly significant hit in performance,
>>> this may be the best option.
>>> All the best,
>>>
>>> Marten
>>>
>>
>> I am also leaning towards this right now, depending on how long we plan
>> to wait for releasing 1.16. It will take us at least a little while to sort
>> out performance issues for __array_function__, I'd guess at least a few
>> weeks. Then a blocker still might turn up during the release candidate
>> process (though I think we've found most of the major bugs / downstream
>> issues already through tests on NumPy's dev branch).
>>
>
> My tentative schedule is to branch in about two weeks, then allow 2 weeks
> of testing for rc1, possibly another two weeks for rc2, and then a final.
> so possibly about six weeks to final release. That leaves 2 to 4 weeks of
> slack before 2019.
>
>
>> Overall, it does feels a little misguided to rush in a change as
>> pervasive as __array_function__ for a long term support release. If we
>> exclude __array_function__ I expect the whole release process for 1.16
>> would go much smoother. We might even try to get 1.17 out faster than
>> usual, so we can minimize the number of additional changes besides
>> __array_function__ and going Python 3 only -- that's already a good bit of
>> change.
>>
>
> I would like to get 1.17 out a bit early. I'm not sure how many backwards
> incompatible changes we want to have in the first post python2 release. My
> initial thoughts are to drop Python 2.7 testing, go to C99, and get the new
> fft in. Beyond that, I'm hesitant to start tearing out all the Python2
> special casing in the first new release, although that could certainly be
> the main task for 1.17 and would clean up the code considerably. It might
> also be a good time to catch up on changing deprecations to errors.
> Thoughts on how to proceed are welcome.
>
>
>> Note that if we make this change (reverting __array_function__), we'll
>> need to revisit where we put a few deprecation warnings -- these will need
>> to be restored into function bodies, not their dispatcher functions.
>>
>> Also: it would be really nice if we get matmul-as-ufunc in before (or at
>> the same time) as __array_function__, so we have a complete story about it
>> being possible to override everything in NumPy. This is another argument
>> for delaying __array_function__, if matmul-as-ufunc can't make it in time
>> for 1.16.
>>
>
> That's two votes for matmul-as-ufunc. How much would it cost to simply
> make __array_function__ a nop?
>
> Chuck
> _______________________________________________
> NumPy-Discussion mailing list
> NumPy-Discussion at python.org
> https://mail.python.org/mailman/listinfo/numpy-discussion
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20181104/982ed3e0/attachment.html>

From m.h.vankerkwijk at gmail.com  Mon Nov  5 09:00:05 2018
From: m.h.vankerkwijk at gmail.com (Marten van Kerkwijk)
Date: Mon, 5 Nov 2018 09:00:05 -0500
Subject: [Numpy-discussion] Should unique types of all arguments be
 passed on in __array_function__?
In-Reply-To: <CAEQ_TvfGJVrZd+wWOnDYgOSpkrwQ+7j0-esuXyMXedjwNsOGAA@mail.gmail.com>
References: <CAJNV+9vx9AJpJ3ZqTHMt9QbEH5kjsz5JSbJZsPB3DaY__yKgZA@mail.gmail.com>
 <CAEQ_TvfGJVrZd+wWOnDYgOSpkrwQ+7j0-esuXyMXedjwNsOGAA@mail.gmail.com>
Message-ID: <CAJNV+9v-UqP3-ThNf2n9ZBWdEhKS=PVifP-U7cku+DvnOk5gFg@mail.gmail.com>

Hi Stephan,

I fear my example about thinking about `ndarray.__array_function__`
distracted from the gist of my question, which was whether for
`__array_function__` implementations *generally* it wouldn't be handier to
have all unique types rather than just those that override
`__array_function__`. It would seem that for any other implementation than
for numpy itself, the presence of __array_function__ is indeed almost
irrelevant. As a somewhat random example, why would it, e.g., for DASK be
useful to know that another argument is a Quantity, but not that it is a
file handle? (Presumably, it cannot handle either...)

All the best,

Marten
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20181105/898a86ff/attachment.html>

From m.h.vankerkwijk at gmail.com  Mon Nov  5 09:08:18 2018
From: m.h.vankerkwijk at gmail.com (Marten van Kerkwijk)
Date: Mon, 5 Nov 2018 09:08:18 -0500
Subject: [Numpy-discussion] Should unique types of all arguments be
 passed on in __array_function__?
In-Reply-To: <CAEQ_TvfGJVrZd+wWOnDYgOSpkrwQ+7j0-esuXyMXedjwNsOGAA@mail.gmail.com>
References: <CAJNV+9vx9AJpJ3ZqTHMt9QbEH5kjsz5JSbJZsPB3DaY__yKgZA@mail.gmail.com>
 <CAEQ_TvfGJVrZd+wWOnDYgOSpkrwQ+7j0-esuXyMXedjwNsOGAA@mail.gmail.com>
Message-ID: <CAJNV+9sCGUOoK4TXi0nKa+-XHjDtNKi2BQx+Zmh4J+9szY=vmg@mail.gmail.com>

More specifically:

Should we change this? It is quite trivially done, but perhaps I am missing
>> a reason for omitting the non-override types.
>>
>
> Realistically, without these other changes in NumPy, how would this
> improve code using __array_function__? From a general purpose dispatching
> perspective, are there cases where you'd want to return NotImplemented
> based on types that don't implement __array_function__?
>

I think, yes, that would be the closest analogy to the python operators.
Saves you from having separate cases for types that have and do not have
`__array_function__`.


> I guess this might help if your alternative array class is super-explicit,
> and doesn't automatically call `asmyarray()` on each argument. You could
> rely on __array_function__ to return NotImplement (and thus raise
> TypeError) rather than type checking in every function you write for your
> alternative arrays.
>

Indeed.


> One minor downside would speed: now __array_function__ implementations
> need to check a longer list of types.
>

That's true.

>
> Another minor downside: if users follow the example of
> NDArrayOperatorsMixin docstring, they would now need to explicitly list all
> of the scalar types (without __array_function__) that they support,
> including builtin types like int and type(None). I suppose this ties into
> our recommended best practices for doing type checking in
> __array_ufunc__/__array_function__ implementations, which should probably
> be updated regardless:
> https://github.com/numpy/numpy/issues/12258#issuecomment-432858949
>
>
Also true. It makes me wonder again whether passing on the types is useful
at all... But I end up thinking that it is not up to an implementation to
raise TypeError - it should just return NotImplemented.

If we'd wanted to give more information, we might also consider passing on
`overloaded_args` - then perhaps one has the best of both worlds.

All the best,

Marten
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20181105/ad3f92bb/attachment-0001.html>

From m.h.vankerkwijk at gmail.com  Mon Nov  5 09:23:59 2018
From: m.h.vankerkwijk at gmail.com (Marten van Kerkwijk)
Date: Mon, 5 Nov 2018 09:23:59 -0500
Subject: [Numpy-discussion] Implementations of
 ndarray.__array_function__ (and ndarray.__array_ufunc__)
In-Reply-To: <CAEQ_TvejSV12DKGijt=TH83ux1qcQuQ87yAt8w6Cgsnxkdo0BA@mail.gmail.com>
References: <CAJNV+9sU-Cb7qv5n2q-LR_29koV2Lqhpi_YWymUGjvg_YuZCfg@mail.gmail.com>
 <CAEQ_TvejSV12DKGijt=TH83ux1qcQuQ87yAt8w6Cgsnxkdo0BA@mail.gmail.com>
Message-ID: <CAJNV+9s65Y0LruGxswuDxFiZ2XST4MiwBjSkQtgdb5s9yOt-FA@mail.gmail.com>

On Sun, Nov 4, 2018 at 8:57 PM Stephan Hoyer <shoyer at gmail.com> wrote:

> On Sun, Nov 4, 2018 at 8:45 AM Marten van Kerkwijk <
> m.h.vankerkwijk at gmail.com> wrote:
>
>> Does the above make sense? I realize that the same would be true for
>> `__array_ufunc__`, though there the situation is slightly trickier since it
>> is not as easy to bypass any further override checks. Nevertheless, it does
>> seem like it would be correct to do the same there. (And if we agree this
>> is the case, I'd quite happily implement it -- with the merger of
>> multiarray and umath it has become much easier to do.)
>>
>
> Marten actually implemented a draft version of this already in
> https://github.com/numpy/numpy/pull/12328 :). I found reading over the PR
> helpful for understand this proposal.
>
> I guess the practical import of this change is that it makes it (much?)
> easier to write __array_function__ for ndarray subclasses: if there's a
> function where NumPy's default function works fine, you don't need to
> bother with returning anything other than NotImplemented from
> __array_function__. It's sort of like NotImplementedButCoercible, but only
> for ndarray subclasses.
>

Yes, return NotImplemented if there is another array, or, even simpler,
just call super.

Note that it is not quite like `NotImplementedButCoercible`, since no
actual coercion to ndarray would necessarily be needed - with adherence to
the Liskov substitution principle, the subclass might stay intact (if only
partially initialized).


> One minor downside is that this might make it harder to eventually
> deprecate and/or contemplate removing checks for 'mean' methods in
> functions like np.mean(), because __array_function__ implementers might
> still be relying on this.
>

I think this is somewhat minor indeed, since we can (and should) insist
that subclasses here properly behave as subclasses, so if an
ndarray-specific implementation breaks a subclass, that might well indicate
that the subclass is not quite good enough (and we can now point out there
is a way to override the function). It might also indicate that the code
itself could be better - that would be a win.

But so far, I think this makes sense.
>
> The PR includes additional changes to np.core.overrides, but I'm not sure
> if those are actually required here (or rather only possible due to this
> change). I guess they are needed if you want to be able to count on
> ndarray.__array_function__ being called after subclass __array_function__
> methods.
>

It is mostly a transfer of functionality from `get_override_types_and_args`
to the place where the implementation is decided upon. Perhaps more logical
even if we do not pursue this.

>
> I'm not sure I like this part: it means that ndarray.__array_function__
> actually gets called when other arguments implement __array_function__. For
> interactions with objects that aren't ndarray subclasses this is entirely
> pointless and would unnecessarily slow things down, since
> ndarray._array_function__ will always return NotImplemented.
>

Agreed here. I did in fact think about it, but wasn't sure (and didn't have
time to think how to check) that the gain in time for cases where an
ndarray comes before the relevant array mimic (and there thus a needless
call to ndarray.__array_function__ can be prevented) was worth it compared
to the cost of attempting to do the removal for cases where the array mimic
came first or where there was no regular ndarray in the first place. But I
think this is an implementation detail; for now, let me add a note to the
PR about it.

All the best,

Marten
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20181105/88ccd32c/attachment.html>

From m.h.vankerkwijk at gmail.com  Mon Nov  5 09:28:12 2018
From: m.h.vankerkwijk at gmail.com (Marten van Kerkwijk)
Date: Mon, 5 Nov 2018 09:28:12 -0500
Subject: [Numpy-discussion] Should unique types of all arguments be
 passed on in __array_function__?
In-Reply-To: <CAEQ_TvfGJVrZd+wWOnDYgOSpkrwQ+7j0-esuXyMXedjwNsOGAA@mail.gmail.com>
References: <CAJNV+9vx9AJpJ3ZqTHMt9QbEH5kjsz5JSbJZsPB3DaY__yKgZA@mail.gmail.com>
 <CAEQ_TvfGJVrZd+wWOnDYgOSpkrwQ+7j0-esuXyMXedjwNsOGAA@mail.gmail.com>
Message-ID: <CAJNV+9twm79spHEetWqznj5Snhz4tT8tmjOFiCOmm8gPun-QYg@mail.gmail.com>

Hi Stephan,

Another part of your reply worth considering, though slightly off topic for
the question here, of what to pass on in `types`:


On Sun, Nov 4, 2018 at 7:51 PM Stephan Hoyer <shoyer at gmail.com> wrote:

> On Sun, Nov 4, 2018 at 8:03 AM Marten van Kerkwijk <
> m.h.vankerkwijk at gmail.com> wrote:
>
>> I thought of this partially as I was wondering how an implementation for
>> ndarray itself would look like. For that, it is definitely useful to know
>> all unique types, since if it is only ndarray, no casting whatsoever needs
>> to be done, while if there are integers, lists, etc, an attempt has to be
>> made to turn these into arrays
>>
>
> OK, so hypothetically we could invoke versions of each the numpy function
> that doesn't call `as[any]array`, and this would slightly speed-up
> subclasses that call super().__array_function__?
>
> A longer-term goal that I had in mind here was generally for the
implementations to just be able to assume their arguments are ndarray,
i.e., be free to assume there is a shape, dtype, etc. That is not
specifically useful for subclasses; for pure python code, it might also
mean array mimics could happily use the implementation.  But perhaps more
importantly, the code would become substantially cleaner.

Anyway, really a longer-term goal...

All the best,

Marten
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20181105/daadea60/attachment.html>

From m.h.vankerkwijk at gmail.com  Mon Nov  5 09:36:39 2018
From: m.h.vankerkwijk at gmail.com (Marten van Kerkwijk)
Date: Mon, 5 Nov 2018 09:36:39 -0500
Subject: [Numpy-discussion] out parameter for np.fromfile
In-Reply-To: <CAC=AwPy=4+tw=fzgzY4b8pwh8-EbhYDM9uVFnuCJ2Jj7PHy9KA@mail.gmail.com>
References: <CAC=AwPy=4+tw=fzgzY4b8pwh8-EbhYDM9uVFnuCJ2Jj7PHy9KA@mail.gmail.com>
Message-ID: <CAJNV+9tJz-je0NEAmenT=-KiOiHFWV33C_5biaTG0-X_Ov9ifA@mail.gmail.com>

Hi Mark,

Having an `out` might make sense. With present numpy, if you are really
dealing with a file or file-like object, you might consider using
`np.memmap` to access the data more directly. If it is something that looks
more like a buffer, `np.frombuffer` may be useful (that doesn't copy data,
but points the array at the memory that holds the buffer).

All the best,

Marten


On Sun, Nov 4, 2018 at 10:35 PM Mark Harfouche <mark.harfouche at gmail.com>
wrote:

> I was wondering what would your thoughts be on adding an output parameter
> to np.fromfile?
>
> The advantage would be when interfacing with executables like ffmpeg
> which are arguably easier to use by calling them as a subprocess compared
> to a shared library in python.
>
> Having the output parameter in np.fromfile would enable pre-allocation of
> large arrays that are reused during the computation of new image frames
> when decoding large video files.
>
> Thoughts are appreciated!
>
> Mark
> _______________________________________________
> NumPy-Discussion mailing list
> NumPy-Discussion at python.org
> https://mail.python.org/mailman/listinfo/numpy-discussion
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20181105/67c42e00/attachment-0001.html>

From m.h.vankerkwijk at gmail.com  Mon Nov  5 09:42:42 2018
From: m.h.vankerkwijk at gmail.com (Marten van Kerkwijk)
Date: Mon, 5 Nov 2018 09:42:42 -0500
Subject: [Numpy-discussion] Prep for NumPy 1.16.0 branch
In-Reply-To: <CAC=AwPzG-G22pbJ_C9DQ3qNn2TjGy=bEzf6miEZmEp8awHgx_g@mail.gmail.com>
References: <CAB6mnxJUty767___L5pMOrAKVE-SamvHLVCqTXz3dDvDHum8sQ@mail.gmail.com>
 <CAJNV+9v+F+U--3H91odm=EU7Y=C1Z2WHhD-w34bmOqhZorF7yg@mail.gmail.com>
 <CAEQ_Tvf-cuKO5gcNfrXoMNPqnVra1g8dExWbCWRP2sUTKDk99Q@mail.gmail.com>
 <CAB6mnx+C0m=_fhpcKOjDF=yfsu_zUKDYh1TGnPRNuU5p9KvkZg@mail.gmail.com>
 <CAC=AwPzG-G22pbJ_C9DQ3qNn2TjGy=bEzf6miEZmEp8awHgx_g@mail.gmail.com>
Message-ID: <CAJNV+9sD6tgWk8eVX-ZgVguyYMuxLD1wWpcQTbvGMmgxfh2tog@mail.gmail.com>

For astropy, we also waiting a little before having a rip-python2-out
fiesta.

I think it is worth trying to get matmul in 1.16, independently of
__array_function__ - it really belongs to ufunc overwrites and all the
groundwork has been done.

For __array_function__, is it at all an option to go to the
disable-by-default step? I.e., by default have array_function_dispatch just
return the implementation instead of wrapping it? Though perhaps reversion
is indeed cleaner; most people who would like to play with it are quite
able to install the development version...

-- Marten

On Sun, Nov 4, 2018 at 10:43 PM Mark Harfouche <mark.harfouche at gmail.com>
wrote:

> > Thoughts on how to proceed are welcome.
>
> I've been involved in scikit-image and that project  tore out the python2
> only code rather quickly after 2.7 support was dropped. I think it caused a
> few hiccups when backporting bugfixes. I imagine that `1.16.1` and `1.16.2`
> releases will come out quickly and as such, I think removing `if else`
> statements for python2 immediately after `1.16` is released will cause
> annoyances in the first few months bugs are being ironed out.
>
> My 2cents.
>
> On Sun, Nov 4, 2018 at 10:04 PM Charles R Harris <
> charlesr.harris at gmail.com> wrote:
>
>>
>>
>> On Sun, Nov 4, 2018 at 6:16 PM Stephan Hoyer <shoyer at gmail.com> wrote:
>>
>>> On Sun, Nov 4, 2018 at 10:32 AM Marten van Kerkwijk <
>>> m.h.vankerkwijk at gmail.com> wrote:
>>>
>>>> Hi Chuck,
>>>>
>>>> For `__array_function__`, there was some discussion in
>>>> https://github.com/numpy/numpy/issues/12225 that for 1.16 we might
>>>> want to follow after all Nathaniel's suggestion of using an environment
>>>> variable or so to opt in (since introspection breaks on python2 with our
>>>> wrapped implementations).  Given also the possibly significant hit in
>>>> performance, this may be the best option.
>>>> All the best,
>>>>
>>>> Marten
>>>>
>>>
>>> I am also leaning towards this right now, depending on how long we plan
>>> to wait for releasing 1.16. It will take us at least a little while to sort
>>> out performance issues for __array_function__, I'd guess at least a few
>>> weeks. Then a blocker still might turn up during the release candidate
>>> process (though I think we've found most of the major bugs / downstream
>>> issues already through tests on NumPy's dev branch).
>>>
>>
>> My tentative schedule is to branch in about two weeks, then allow 2 weeks
>> of testing for rc1, possibly another two weeks for rc2, and then a final.
>> so possibly about six weeks to final release. That leaves 2 to 4 weeks of
>> slack before 2019.
>>
>>
>>> Overall, it does feels a little misguided to rush in a change as
>>> pervasive as __array_function__ for a long term support release. If we
>>> exclude __array_function__ I expect the whole release process for 1.16
>>> would go much smoother. We might even try to get 1.17 out faster than
>>> usual, so we can minimize the number of additional changes besides
>>> __array_function__ and going Python 3 only -- that's already a good bit of
>>> change.
>>>
>>
>> I would like to get 1.17 out a bit early. I'm not sure how many backwards
>> incompatible changes we want to have in the first post python2 release. My
>> initial thoughts are to drop Python 2.7 testing, go to C99, and get the new
>> fft in. Beyond that, I'm hesitant to start tearing out all the Python2
>> special casing in the first new release, although that could certainly be
>> the main task for 1.17 and would clean up the code considerably. It might
>> also be a good time to catch up on changing deprecations to errors.
>> Thoughts on how to proceed are welcome.
>>
>>
>>> Note that if we make this change (reverting __array_function__), we'll
>>> need to revisit where we put a few deprecation warnings -- these will need
>>> to be restored into function bodies, not their dispatcher functions.
>>>
>>> Also: it would be really nice if we get matmul-as-ufunc in before (or at
>>> the same time) as __array_function__, so we have a complete story about it
>>> being possible to override everything in NumPy. This is another argument
>>> for delaying __array_function__, if matmul-as-ufunc can't make it in time
>>> for 1.16.
>>>
>>
>> That's two votes for matmul-as-ufunc. How much would it cost to simply
>> make __array_function__ a nop?
>>
>> Chuck
>> _______________________________________________
>> NumPy-Discussion mailing list
>> NumPy-Discussion at python.org
>> https://mail.python.org/mailman/listinfo/numpy-discussion
>>
> _______________________________________________
> NumPy-Discussion mailing list
> NumPy-Discussion at python.org
> https://mail.python.org/mailman/listinfo/numpy-discussion
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20181105/805e2c73/attachment.html>

From mark.harfouche at gmail.com  Mon Nov  5 11:24:57 2018
From: mark.harfouche at gmail.com (Mark Harfouche)
Date: Mon, 5 Nov 2018 11:24:57 -0500
Subject: [Numpy-discussion] out parameter for np.fromfile
In-Reply-To: <CAJNV+9tJz-je0NEAmenT=-KiOiHFWV33C_5biaTG0-X_Ov9ifA@mail.gmail.com>
References: <CAC=AwPy=4+tw=fzgzY4b8pwh8-EbhYDM9uVFnuCJ2Jj7PHy9KA@mail.gmail.com>
 <CAJNV+9tJz-je0NEAmenT=-KiOiHFWV33C_5biaTG0-X_Ov9ifA@mail.gmail.com>
Message-ID: <CAC=AwPyJD_atu3G1FKrGWY8nbFsvGT6zdQOc6dbb_qeptb-ecQ@mail.gmail.com>

Thanks Marten.

I tried memap for a few things but it seemed to create an other OS level
buffer in specific situations. I think the `seek` operation in the `memmap`
also caused some performance bottlenecks. Maybe I'll have time to summarize
my findings an other day.

The particular usecase of `ffmpeg` is tricky since it is grabbing a lot of
data from `stdout` which isn't a typical file buffer. Specifically, it is
often `buffered` but `unseekable`.

On Mon, Nov 5, 2018 at 9:38 AM Marten van Kerkwijk <
m.h.vankerkwijk at gmail.com> wrote:

> Hi Mark,
>
> Having an `out` might make sense. With present numpy, if you are really
> dealing with a file or file-like object, you might consider using
> `np.memmap` to access the data more directly. If it is something that looks
> more like a buffer, `np.frombuffer` may be useful (that doesn't copy data,
> but points the array at the memory that holds the buffer).
>
> All the best,
>
> Marten
>
>
> On Sun, Nov 4, 2018 at 10:35 PM Mark Harfouche <mark.harfouche at gmail.com>
> wrote:
>
>> I was wondering what would your thoughts be on adding an output parameter
>> to np.fromfile?
>>
>> The advantage would be when interfacing with executables like ffmpeg
>> which are arguably easier to use by calling them as a subprocess compared
>> to a shared library in python.
>>
>> Having the output parameter in np.fromfile would enable pre-allocation
>> of large arrays that are reused during the computation of new image frames
>> when decoding large video files.
>>
>> Thoughts are appreciated!
>>
>> Mark
>> _______________________________________________
>> NumPy-Discussion mailing list
>> NumPy-Discussion at python.org
>> https://mail.python.org/mailman/listinfo/numpy-discussion
>>
> _______________________________________________
> NumPy-Discussion mailing list
> NumPy-Discussion at python.org
> https://mail.python.org/mailman/listinfo/numpy-discussion
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20181105/d8d39ba5/attachment-0001.html>

From f.s.farimani at gmail.com  Mon Nov  5 16:44:52 2018
From: f.s.farimani at gmail.com (Foad Sojoodi Farimani)
Date: Mon, 5 Nov 2018 22:44:52 +0100
Subject: [Numpy-discussion] numpy pprint?
Message-ID: <CADjqfdp7VO_3KfwJTC-_mVLVnPyxbzZuY6+=hAzmvooe8NE+sg@mail.gmail.com>

Hello everyone,

Following this question <https://stackoverflow.com/q/53126305/4999991>, I'm
convinced that numpy ndarrays are not MATLAB/mathematical multidimentional
matrices and I should stop expecting them to be. However I still think it
would have a lot of benefit to have a function like sympy's pprint to
pretty print. something like pandas .head and .tail method plus  .left
.right .UpLeft .UpRight .DownLeft .DownRight methods. when nothing
mentioned it would show 4 corners and put dots in the middle if the array
is to big for the terminal.

Best,
Foad
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20181105/d6f30961/attachment.html>

From mark.harfouche at gmail.com  Tue Nov  6 00:11:20 2018
From: mark.harfouche at gmail.com (Mark Harfouche)
Date: Tue, 6 Nov 2018 00:11:20 -0500
Subject: [Numpy-discussion] numpy pprint?
In-Reply-To: <CADjqfdp7VO_3KfwJTC-_mVLVnPyxbzZuY6+=hAzmvooe8NE+sg@mail.gmail.com>
References: <CADjqfdp7VO_3KfwJTC-_mVLVnPyxbzZuY6+=hAzmvooe8NE+sg@mail.gmail.com>
Message-ID: <CAC=AwPy8qnC6NNH0NTmkzp7zGr+KaZ6Mcp=0sC-FhaEUX2hgXw@mail.gmail.com>

Foad,

Visualizing data is definitely a complex field. I definitely feel your pain.
Printing your data is but one way of visualizing it, and probably only
useful for very small and constrained datasets.
Have you looked into set_printoptions
<https://docs.scipy.org/doc/numpy-1.15.1/reference/generated/numpy.set_printoptions.html>
to see how numpy?s existing capabilities might help you with your
visualization?

The code you showed seems quite good. I wouldn?t worry about performance
when it comes to functions that will seldom be called in tight loops.
As you?ll learn more about python and numpy, you?ll keep expanding it to
include more use cases.
For many of my projects, I create small submodules for visualization
tailored to the specific needs of the particular project.
I?ll try to incorporate your functions and see how I use them.

Your original post seems to have some confusion about C Style vs F Style
ordering. I hope that has been resolved.
There is also a lot of good documentation
https://docs.scipy.org/doc/numpy/user/numpy-for-matlab-users.html#numpy-for-matlab-users-notes
about transitioning from matlab.

Mark

On Mon, Nov 5, 2018 at 4:46 PM Foad Sojoodi Farimani <f.s.farimani at gmail.com>
wrote:

> Hello everyone,
>
> Following this question <https://stackoverflow.com/q/53126305/4999991>,
> I'm convinced that numpy ndarrays are not MATLAB/mathematical
> multidimentional matrices and I should stop expecting them to be. However I
> still think it would have a lot of benefit to have a function like sympy's
> pprint to pretty print. something like pandas .head and .tail method plus
> .left .right .UpLeft .UpRight .DownLeft .DownRight methods. when nothing
> mentioned it would show 4 corners and put dots in the middle if the array
> is to big for the terminal.
>
> Best,
> Foad
> _______________________________________________
> NumPy-Discussion mailing list
> NumPy-Discussion at python.org
> https://mail.python.org/mailman/listinfo/numpy-discussion
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20181106/00259953/attachment.html>

From wieser.eric+numpy at gmail.com  Tue Nov  6 00:50:31 2018
From: wieser.eric+numpy at gmail.com (Eric Wieser)
Date: Mon, 5 Nov 2018 21:50:31 -0800
Subject: [Numpy-discussion] numpy pprint?
In-Reply-To: <CAC=AwPy8qnC6NNH0NTmkzp7zGr+KaZ6Mcp=0sC-FhaEUX2hgXw@mail.gmail.com>
References: <CADjqfdp7VO_3KfwJTC-_mVLVnPyxbzZuY6+=hAzmvooe8NE+sg@mail.gmail.com>
 <CAC=AwPy8qnC6NNH0NTmkzp7zGr+KaZ6Mcp=0sC-FhaEUX2hgXw@mail.gmail.com>
Message-ID: <CAL1kJvDbUZzu1Ky6d2wK=b-mDh2FLkLjoK5hYp+H3Jko9ScCYw@mail.gmail.com>

Hijacking this thread while on the topic of pprint - we might want to look
into a table-based `_html_repr_` or `_latex_repr_` for use in ipython -
where we can print the full array and let scrollbars replace ellipses.

Eric

On Mon, 5 Nov 2018 at 21:11 Mark Harfouche <mark.harfouche at gmail.com> wrote:

> Foad,
>
> Visualizing data is definitely a complex field. I definitely feel your
> pain.
> Printing your data is but one way of visualizing it, and probably only
> useful for very small and constrained datasets.
> Have you looked into set_printoptions
> <https://docs.scipy.org/doc/numpy-1.15.1/reference/generated/numpy.set_printoptions.html>
> to see how numpy?s existing capabilities might help you with your
> visualization?
>
> The code you showed seems quite good. I wouldn?t worry about performance
> when it comes to functions that will seldom be called in tight loops.
> As you?ll learn more about python and numpy, you?ll keep expanding it to
> include more use cases.
> For many of my projects, I create small submodules for visualization
> tailored to the specific needs of the particular project.
> I?ll try to incorporate your functions and see how I use them.
>
> Your original post seems to have some confusion about C Style vs F Style
> ordering. I hope that has been resolved.
> There is also a lot of good documentation
>
> https://docs.scipy.org/doc/numpy/user/numpy-for-matlab-users.html#numpy-for-matlab-users-notes
> about transitioning from matlab.
>
> Mark
>
> On Mon, Nov 5, 2018 at 4:46 PM Foad Sojoodi Farimani <
> f.s.farimani at gmail.com> wrote:
>
>> Hello everyone,
>>
>> Following this question <https://stackoverflow.com/q/53126305/4999991>,
>> I'm convinced that numpy ndarrays are not MATLAB/mathematical
>> multidimentional matrices and I should stop expecting them to be. However I
>> still think it would have a lot of benefit to have a function like sympy's
>> pprint to pretty print. something like pandas .head and .tail method plus
>> .left .right .UpLeft .UpRight .DownLeft .DownRight methods. when nothing
>> mentioned it would show 4 corners and put dots in the middle if the array
>> is to big for the terminal.
>>
>> Best,
>> Foad
>> _______________________________________________
>> NumPy-Discussion mailing list
>> NumPy-Discussion at python.org
>> https://mail.python.org/mailman/listinfo/numpy-discussion
>>
> _______________________________________________
> NumPy-Discussion mailing list
> NumPy-Discussion at python.org
> https://mail.python.org/mailman/listinfo/numpy-discussion
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20181105/188eb27e/attachment-0001.html>

From f.s.farimani at gmail.com  Tue Nov  6 02:26:02 2018
From: f.s.farimani at gmail.com (Foad Sojoodi Farimani)
Date: Tue, 6 Nov 2018 08:26:02 +0100
Subject: [Numpy-discussion] numpy pprint?
In-Reply-To: <CAC=AwPy8qnC6NNH0NTmkzp7zGr+KaZ6Mcp=0sC-FhaEUX2hgXw@mail.gmail.com>
References: <CADjqfdp7VO_3KfwJTC-_mVLVnPyxbzZuY6+=hAzmvooe8NE+sg@mail.gmail.com>
 <CAC=AwPy8qnC6NNH0NTmkzp7zGr+KaZ6Mcp=0sC-FhaEUX2hgXw@mail.gmail.com>
Message-ID: <CADjqfdopCXA1BOvZF3TiPRkKWZB0NC1VF0pQ83zkA8UTT=Y4Zw@mail.gmail.com>

Dear Mark,

Thanks for the reply. I will write in between your lines:

On Tue, Nov 6, 2018 at 6:11 AM Mark Harfouche <mark.harfouche at gmail.com>
wrote:

> Foad,
>
> Visualizing data is definitely a complex field. I definitely feel your
> pain.
>
I have actually been using numpy for a couple of years without noticing
these issues. recently I have been trying to encourage my collogues to move
from MATLAB to Python and also prepare some workshops for PhD network of my
university.

> Printing your data is but one way of visualizing it, and probably only
> useful for very small and constrained datasets.
>
well actually it can be very useful. Consider Pandas .head() and .tail()
methods or Sympy's pretty printing functionalities. for bigger datasets the
function can get the terminals width and height and then based on the input
(U(n),D(n),L(n),R(n),UR(n,m),UL(n,m),DR(n,m),DL(n,m)) display what can be
shown and put horizontal 3-dots \u2026 ? or vertical/inclined ones. Or id
it is Jupyter then one can use Markdown/LaTeX for pretty printing or even
HTML to add sliders as suggested by Eric.

> Have you looked into set_printoptions
> <https://docs.scipy.org/doc/numpy-1.15.1/reference/generated/numpy.set_printoptions.html>
> to see how numpy?s existing capabilities might help you with your
> visualization?
>
This is indeed very useful. specially the threshold option can help a lot
with adjusting the width. but only for specific cases.

> The code you showed seems quite good. I wouldn?t worry about performance
> when it comes to functions that will seldom be called in tight loops.
>
Thanks but I know it is very bad:

   - it does not work properly for floats
   - it only works for 1D and 2D
   - there can be some recursive function I believe.

As you?ll learn more about python and numpy, you?ll keep expanding it to
> include more use cases.
> For many of my projects, I create small submodules for visualization
> tailored to the specific needs of the particular project.
> I?ll try to incorporate your functions and see how I use them.
>
Thanks a lot. looking forwards to your feedback

> Your original post seems to have some confusion about C Style vs F Style
> ordering. I hope that has been resolved.
>
I actually came to the conclusion that calling it C-Style or F-Style or
maybe row-major column-major are bad practices. Numpy's ndarrays are not
mathematical multidimensional arrays but Pythons nested, homogenous and
uniform lists.  it means for example 1, [1], [[1]] and [[[1]]] are all
different, while in all other mathematical languages out there (including
Sympy's matrices) they are the same.

> There is also a lot of good documentation
>
> https://docs.scipy.org/doc/numpy/user/numpy-for-matlab-users.html#numpy-for-matlab-users-notes
> about transitioning from matlab.
>
I have seen this one and many others, which I'm trying to comprehend and
then put in some slides made in Jupyter notebooks. Maybe when they are
ready I will create a GitHub repo and upload them alongside the possible
video recordings of the workshops.

Foad

> Mark
>
> On Mon, Nov 5, 2018 at 4:46 PM Foad Sojoodi Farimani <
> f.s.farimani at gmail.com> wrote:
>
>> Hello everyone,
>>
>> Following this question <https://stackoverflow.com/q/53126305/4999991>,
>> I'm convinced that numpy ndarrays are not MATLAB/mathematical
>> multidimentional matrices and I should stop expecting them to be. However I
>> still think it would have a lot of benefit to have a function like sympy's
>> pprint to pretty print. something like pandas .head and .tail method plus
>> .left .right .UpLeft .UpRight .DownLeft .DownRight methods. when nothing
>> mentioned it would show 4 corners and put dots in the middle if the array
>> is to big for the terminal.
>>
>> Best,
>> Foad
>> _______________________________________________
>> NumPy-Discussion mailing list
>> NumPy-Discussion at python.org
>> https://mail.python.org/mailman/listinfo/numpy-discussion
>>
> _______________________________________________
> NumPy-Discussion mailing list
> NumPy-Discussion at python.org
> https://mail.python.org/mailman/listinfo/numpy-discussion
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20181106/7c3966b7/attachment.html>

From f.s.farimani at gmail.com  Tue Nov  6 02:28:19 2018
From: f.s.farimani at gmail.com (Foad Sojoodi Farimani)
Date: Tue, 6 Nov 2018 08:28:19 +0100
Subject: [Numpy-discussion] numpy pprint?
In-Reply-To: <CAL1kJvDbUZzu1Ky6d2wK=b-mDh2FLkLjoK5hYp+H3Jko9ScCYw@mail.gmail.com>
References: <CADjqfdp7VO_3KfwJTC-_mVLVnPyxbzZuY6+=hAzmvooe8NE+sg@mail.gmail.com>
 <CAC=AwPy8qnC6NNH0NTmkzp7zGr+KaZ6Mcp=0sC-FhaEUX2hgXw@mail.gmail.com>
 <CAL1kJvDbUZzu1Ky6d2wK=b-mDh2FLkLjoK5hYp+H3Jko9ScCYw@mail.gmail.com>
Message-ID: <CADjqfdoY9Bjy-7-hvj4mDUOm5PnY_P3BCz4YaYNL0Qw_o75c7w@mail.gmail.com>

It is not highking if I asked for it :))
for IPython/Jupyter using Markdown/LaTeX would be awesome
or even better using HTML to add sliders just like Pandas...

F.

On Tue, Nov 6, 2018 at 6:51 AM Eric Wieser <wieser.eric+numpy at gmail.com>
wrote:

> Hijacking this thread while on the topic of pprint - we might want to look
> into a table-based `_html_repr_` or `_latex_repr_` for use in ipython -
> where we can print the full array and let scrollbars replace ellipses.
>
> Eric
>
> On Mon, 5 Nov 2018 at 21:11 Mark Harfouche <mark.harfouche at gmail.com>
> wrote:
>
>> Foad,
>>
>> Visualizing data is definitely a complex field. I definitely feel your
>> pain.
>> Printing your data is but one way of visualizing it, and probably only
>> useful for very small and constrained datasets.
>> Have you looked into set_printoptions
>> <https://docs.scipy.org/doc/numpy-1.15.1/reference/generated/numpy.set_printoptions.html>
>> to see how numpy?s existing capabilities might help you with your
>> visualization?
>>
>> The code you showed seems quite good. I wouldn?t worry about performance
>> when it comes to functions that will seldom be called in tight loops.
>> As you?ll learn more about python and numpy, you?ll keep expanding it to
>> include more use cases.
>> For many of my projects, I create small submodules for visualization
>> tailored to the specific needs of the particular project.
>> I?ll try to incorporate your functions and see how I use them.
>>
>> Your original post seems to have some confusion about C Style vs F Style
>> ordering. I hope that has been resolved.
>> There is also a lot of good documentation
>>
>> https://docs.scipy.org/doc/numpy/user/numpy-for-matlab-users.html#numpy-for-matlab-users-notes
>> about transitioning from matlab.
>>
>> Mark
>>
>> On Mon, Nov 5, 2018 at 4:46 PM Foad Sojoodi Farimani <
>> f.s.farimani at gmail.com> wrote:
>>
>>> Hello everyone,
>>>
>>> Following this question <https://stackoverflow.com/q/53126305/4999991>,
>>> I'm convinced that numpy ndarrays are not MATLAB/mathematical
>>> multidimentional matrices and I should stop expecting them to be. However I
>>> still think it would have a lot of benefit to have a function like sympy's
>>> pprint to pretty print. something like pandas .head and .tail method plus
>>> .left .right .UpLeft .UpRight .DownLeft .DownRight methods. when nothing
>>> mentioned it would show 4 corners and put dots in the middle if the array
>>> is to big for the terminal.
>>>
>>> Best,
>>> Foad
>>> _______________________________________________
>>> NumPy-Discussion mailing list
>>> NumPy-Discussion at python.org
>>> https://mail.python.org/mailman/listinfo/numpy-discussion
>>>
>> _______________________________________________
>> NumPy-Discussion mailing list
>> NumPy-Discussion at python.org
>> https://mail.python.org/mailman/listinfo/numpy-discussion
>>
> _______________________________________________
> NumPy-Discussion mailing list
> NumPy-Discussion at python.org
> https://mail.python.org/mailman/listinfo/numpy-discussion
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20181106/2f331640/attachment-0001.html>

From wieser.eric+numpy at gmail.com  Tue Nov  6 03:45:17 2018
From: wieser.eric+numpy at gmail.com (Eric Wieser)
Date: Tue, 6 Nov 2018 00:45:17 -0800
Subject: [Numpy-discussion] numpy pprint?
In-Reply-To: <CADjqfdoY9Bjy-7-hvj4mDUOm5PnY_P3BCz4YaYNL0Qw_o75c7w@mail.gmail.com>
References: <CADjqfdp7VO_3KfwJTC-_mVLVnPyxbzZuY6+=hAzmvooe8NE+sg@mail.gmail.com>
 <CAC=AwPy8qnC6NNH0NTmkzp7zGr+KaZ6Mcp=0sC-FhaEUX2hgXw@mail.gmail.com>
 <CAL1kJvDbUZzu1Ky6d2wK=b-mDh2FLkLjoK5hYp+H3Jko9ScCYw@mail.gmail.com>
 <CADjqfdoY9Bjy-7-hvj4mDUOm5PnY_P3BCz4YaYNL0Qw_o75c7w@mail.gmail.com>
Message-ID: <CAL1kJvBo3cZ2XY1kni3addVDG3bERv93=FH1qycM58efVX8xug@mail.gmail.com>

Here's how that could look

https://numpyintegration-ericwieser.notebooks.azure.com/j/notebooks/pprint.ipynb

Feel free to play around and see if you can produce something more useful



On Mon, 5 Nov 2018 at 23:28 Foad Sojoodi Farimani <f.s.farimani at gmail.com>
wrote:

> It is not highking if I asked for it :))
> for IPython/Jupyter using Markdown/LaTeX would be awesome
> or even better using HTML to add sliders just like Pandas...
>
> F.
>
> On Tue, Nov 6, 2018 at 6:51 AM Eric Wieser <wieser.eric+numpy at gmail.com>
> wrote:
>
>> Hijacking this thread while on the topic of pprint - we might want to
>> look into a table-based `_html_repr_` or `_latex_repr_` for use in ipython
>> - where we can print the full array and let scrollbars replace ellipses.
>>
>> Eric
>>
>> On Mon, 5 Nov 2018 at 21:11 Mark Harfouche <mark.harfouche at gmail.com>
>> wrote:
>>
>>> Foad,
>>>
>>> Visualizing data is definitely a complex field. I definitely feel your
>>> pain.
>>> Printing your data is but one way of visualizing it, and probably only
>>> useful for very small and constrained datasets.
>>> Have you looked into set_printoptions
>>> <https://docs.scipy.org/doc/numpy-1.15.1/reference/generated/numpy.set_printoptions.html>
>>> to see how numpy?s existing capabilities might help you with your
>>> visualization?
>>>
>>> The code you showed seems quite good. I wouldn?t worry about performance
>>> when it comes to functions that will seldom be called in tight loops.
>>> As you?ll learn more about python and numpy, you?ll keep expanding it to
>>> include more use cases.
>>> For many of my projects, I create small submodules for visualization
>>> tailored to the specific needs of the particular project.
>>> I?ll try to incorporate your functions and see how I use them.
>>>
>>> Your original post seems to have some confusion about C Style vs F Style
>>> ordering. I hope that has been resolved.
>>> There is also a lot of good documentation
>>>
>>> https://docs.scipy.org/doc/numpy/user/numpy-for-matlab-users.html#numpy-for-matlab-users-notes
>>> about transitioning from matlab.
>>>
>>> Mark
>>>
>>> On Mon, Nov 5, 2018 at 4:46 PM Foad Sojoodi Farimani <
>>> f.s.farimani at gmail.com> wrote:
>>>
>>>> Hello everyone,
>>>>
>>>> Following this question <https://stackoverflow.com/q/53126305/4999991>,
>>>> I'm convinced that numpy ndarrays are not MATLAB/mathematical
>>>> multidimentional matrices and I should stop expecting them to be. However I
>>>> still think it would have a lot of benefit to have a function like sympy's
>>>> pprint to pretty print. something like pandas .head and .tail method plus
>>>> .left .right .UpLeft .UpRight .DownLeft .DownRight methods. when nothing
>>>> mentioned it would show 4 corners and put dots in the middle if the array
>>>> is to big for the terminal.
>>>>
>>>> Best,
>>>> Foad
>>>> _______________________________________________
>>>> NumPy-Discussion mailing list
>>>> NumPy-Discussion at python.org
>>>> https://mail.python.org/mailman/listinfo/numpy-discussion
>>>>
>>> _______________________________________________
>>> NumPy-Discussion mailing list
>>> NumPy-Discussion at python.org
>>> https://mail.python.org/mailman/listinfo/numpy-discussion
>>>
>> _______________________________________________
>> NumPy-Discussion mailing list
>> NumPy-Discussion at python.org
>> https://mail.python.org/mailman/listinfo/numpy-discussion
>>
> _______________________________________________
> NumPy-Discussion mailing list
> NumPy-Discussion at python.org
> https://mail.python.org/mailman/listinfo/numpy-discussion
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20181106/26f5983b/attachment.html>

From f.s.farimani at gmail.com  Tue Nov  6 03:55:59 2018
From: f.s.farimani at gmail.com (Foad Sojoodi Farimani)
Date: Tue, 6 Nov 2018 09:55:59 +0100
Subject: [Numpy-discussion] numpy pprint?
In-Reply-To: <CAL1kJvBo3cZ2XY1kni3addVDG3bERv93=FH1qycM58efVX8xug@mail.gmail.com>
References: <CADjqfdp7VO_3KfwJTC-_mVLVnPyxbzZuY6+=hAzmvooe8NE+sg@mail.gmail.com>
 <CAC=AwPy8qnC6NNH0NTmkzp7zGr+KaZ6Mcp=0sC-FhaEUX2hgXw@mail.gmail.com>
 <CAL1kJvDbUZzu1Ky6d2wK=b-mDh2FLkLjoK5hYp+H3Jko9ScCYw@mail.gmail.com>
 <CADjqfdoY9Bjy-7-hvj4mDUOm5PnY_P3BCz4YaYNL0Qw_o75c7w@mail.gmail.com>
 <CAL1kJvBo3cZ2XY1kni3addVDG3bERv93=FH1qycM58efVX8xug@mail.gmail.com>
Message-ID: <CADjqfdqZP2EN=k-Ww1+E0DY462BT3av231+Hpf9Tz7iOtLvgkA@mail.gmail.com>

Wow, this is awesome.
Some points though:

   - not everybody uses IPython/Jupyter having the functionality for
   conventional consols would also help. something like
   Sypy's init_printing/init_session which smartly chooses the right
   representation considering the terminal.
   - I don't think putting everything in boxes is helping. it is confusing.
   I would rather having horizontal and vertical square brackets represent
   each nested array
   - it would be awesome if in IPython/Jupyter hovering over an element a
   popup would show the index
   - one could read the width and height of the terminal and other options
   I mentioned in reply Mark to show L R U P or combination of these plus some
   numbers (similar to Pandas .head .tail) methods and then show the rest by
   unicod 3dot

P.S. I had no idea our university Microsoft services also offers Azure
Notebooks awesome :P

F.

On Tue, Nov 6, 2018 at 9:45 AM Eric Wieser <wieser.eric+numpy at gmail.com>
wrote:

> Here's how that could look
>
>
> https://numpyintegration-ericwieser.notebooks.azure.com/j/notebooks/pprint.ipynb
>
> Feel free to play around and see if you can produce something more useful
>
>
>
> On Mon, 5 Nov 2018 at 23:28 Foad Sojoodi Farimani <f.s.farimani at gmail.com>
> wrote:
>
>> It is not highking if I asked for it :))
>> for IPython/Jupyter using Markdown/LaTeX would be awesome
>> or even better using HTML to add sliders just like Pandas...
>>
>> F.
>>
>> On Tue, Nov 6, 2018 at 6:51 AM Eric Wieser <wieser.eric+numpy at gmail.com>
>> wrote:
>>
>>> Hijacking this thread while on the topic of pprint - we might want to
>>> look into a table-based `_html_repr_` or `_latex_repr_` for use in ipython
>>> - where we can print the full array and let scrollbars replace ellipses.
>>>
>>> Eric
>>>
>>> On Mon, 5 Nov 2018 at 21:11 Mark Harfouche <mark.harfouche at gmail.com>
>>> wrote:
>>>
>>>> Foad,
>>>>
>>>> Visualizing data is definitely a complex field. I definitely feel your
>>>> pain.
>>>> Printing your data is but one way of visualizing it, and probably only
>>>> useful for very small and constrained datasets.
>>>> Have you looked into set_printoptions
>>>> <https://docs.scipy.org/doc/numpy-1.15.1/reference/generated/numpy.set_printoptions.html>
>>>> to see how numpy?s existing capabilities might help you with your
>>>> visualization?
>>>>
>>>> The code you showed seems quite good. I wouldn?t worry about
>>>> performance when it comes to functions that will seldom be called in tight
>>>> loops.
>>>> As you?ll learn more about python and numpy, you?ll keep expanding it
>>>> to include more use cases.
>>>> For many of my projects, I create small submodules for visualization
>>>> tailored to the specific needs of the particular project.
>>>> I?ll try to incorporate your functions and see how I use them.
>>>>
>>>> Your original post seems to have some confusion about C Style vs F
>>>> Style ordering. I hope that has been resolved.
>>>> There is also a lot of good documentation
>>>>
>>>> https://docs.scipy.org/doc/numpy/user/numpy-for-matlab-users.html#numpy-for-matlab-users-notes
>>>> about transitioning from matlab.
>>>>
>>>> Mark
>>>>
>>>> On Mon, Nov 5, 2018 at 4:46 PM Foad Sojoodi Farimani <
>>>> f.s.farimani at gmail.com> wrote:
>>>>
>>>>> Hello everyone,
>>>>>
>>>>> Following this question <https://stackoverflow.com/q/53126305/4999991>,
>>>>> I'm convinced that numpy ndarrays are not MATLAB/mathematical
>>>>> multidimentional matrices and I should stop expecting them to be. However I
>>>>> still think it would have a lot of benefit to have a function like sympy's
>>>>> pprint to pretty print. something like pandas .head and .tail method plus
>>>>> .left .right .UpLeft .UpRight .DownLeft .DownRight methods. when nothing
>>>>> mentioned it would show 4 corners and put dots in the middle if the array
>>>>> is to big for the terminal.
>>>>>
>>>>> Best,
>>>>> Foad
>>>>> _______________________________________________
>>>>> NumPy-Discussion mailing list
>>>>> NumPy-Discussion at python.org
>>>>> https://mail.python.org/mailman/listinfo/numpy-discussion
>>>>>
>>>> _______________________________________________
>>>> NumPy-Discussion mailing list
>>>> NumPy-Discussion at python.org
>>>> https://mail.python.org/mailman/listinfo/numpy-discussion
>>>>
>>> _______________________________________________
>>> NumPy-Discussion mailing list
>>> NumPy-Discussion at python.org
>>> https://mail.python.org/mailman/listinfo/numpy-discussion
>>>
>> _______________________________________________
>> NumPy-Discussion mailing list
>> NumPy-Discussion at python.org
>> https://mail.python.org/mailman/listinfo/numpy-discussion
>>
> _______________________________________________
> NumPy-Discussion mailing list
> NumPy-Discussion at python.org
> https://mail.python.org/mailman/listinfo/numpy-discussion
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20181106/78ad9184/attachment-0001.html>

From mark.harfouche at gmail.com  Tue Nov  6 05:07:04 2018
From: mark.harfouche at gmail.com (Mark Harfouche)
Date: Tue, 6 Nov 2018 05:07:04 -0500
Subject: [Numpy-discussion] numpy pprint?
In-Reply-To: <CADjqfdqZP2EN=k-Ww1+E0DY462BT3av231+Hpf9Tz7iOtLvgkA@mail.gmail.com>
References: <CADjqfdp7VO_3KfwJTC-_mVLVnPyxbzZuY6+=hAzmvooe8NE+sg@mail.gmail.com>
 <CAC=AwPy8qnC6NNH0NTmkzp7zGr+KaZ6Mcp=0sC-FhaEUX2hgXw@mail.gmail.com>
 <CAL1kJvDbUZzu1Ky6d2wK=b-mDh2FLkLjoK5hYp+H3Jko9ScCYw@mail.gmail.com>
 <CADjqfdoY9Bjy-7-hvj4mDUOm5PnY_P3BCz4YaYNL0Qw_o75c7w@mail.gmail.com>
 <CAL1kJvBo3cZ2XY1kni3addVDG3bERv93=FH1qycM58efVX8xug@mail.gmail.com>
 <CADjqfdqZP2EN=k-Ww1+E0DY462BT3av231+Hpf9Tz7iOtLvgkA@mail.gmail.com>
Message-ID: <CAC=AwPzafbhmY3oOYUHUa-0b9+G4FNxDy_pa6VQbPXK8D6+HKQ@mail.gmail.com>

 Foad,
In response to:

Thanks but I know it is very bad:

   - it does not work properly for floats
   - it only works for 1D and 2D
   - there can be some recursive function I believe.

I think this is the awesome part about being able to write 10 lines of code
that are specified to representing exactly 1 thing.

Other than that, yeah, encouraging people to transition from matlab is
challenging. Matlab is definitely good at doing matrix operations. Python3
somewhat helps in that regard.

I'm super glad you are bringing usability issues up and working toward
solving them.

Maybe you can describe the interface for python you find practical to
introduce to newcomers so as to motivate the discussion?

Mark


On Tue, Nov 6, 2018 at 3:57 AM Foad Sojoodi Farimani <f.s.farimani at gmail.com>
wrote:

> Wow, this is awesome.
> Some points though:
>
>    - not everybody uses IPython/Jupyter having the functionality for
>    conventional consols would also help. something like
>    Sypy's init_printing/init_session which smartly chooses the right
>    representation considering the terminal.
>    - I don't think putting everything in boxes is helping. it is
>    confusing. I would rather having horizontal and vertical square brackets
>    represent each nested array
>    - it would be awesome if in IPython/Jupyter hovering over an element a
>    popup would show the index
>    - one could read the width and height of the terminal and other
>    options I mentioned in reply Mark to show L R U P or combination of these
>    plus some numbers (similar to Pandas .head .tail) methods and then show the
>    rest by unicod 3dot
>
> P.S. I had no idea our university Microsoft services also offers Azure
> Notebooks awesome :P
>
> F.
>
> On Tue, Nov 6, 2018 at 9:45 AM Eric Wieser <wieser.eric+numpy at gmail.com>
> wrote:
>
>> Here's how that could look
>>
>>
>> https://numpyintegration-ericwieser.notebooks.azure.com/j/notebooks/pprint.ipynb
>>
>> Feel free to play around and see if you can produce something more useful
>>
>>
>>
>> On Mon, 5 Nov 2018 at 23:28 Foad Sojoodi Farimani <f.s.farimani at gmail.com>
>> wrote:
>>
>>> It is not highking if I asked for it :))
>>> for IPython/Jupyter using Markdown/LaTeX would be awesome
>>> or even better using HTML to add sliders just like Pandas...
>>>
>>> F.
>>>
>>> On Tue, Nov 6, 2018 at 6:51 AM Eric Wieser <wieser.eric+numpy at gmail.com>
>>> wrote:
>>>
>>>> Hijacking this thread while on the topic of pprint - we might want to
>>>> look into a table-based `_html_repr_` or `_latex_repr_` for use in ipython
>>>> - where we can print the full array and let scrollbars replace ellipses.
>>>>
>>>> Eric
>>>>
>>>> On Mon, 5 Nov 2018 at 21:11 Mark Harfouche <mark.harfouche at gmail.com>
>>>> wrote:
>>>>
>>>>> Foad,
>>>>>
>>>>> Visualizing data is definitely a complex field. I definitely feel your
>>>>> pain.
>>>>> Printing your data is but one way of visualizing it, and probably only
>>>>> useful for very small and constrained datasets.
>>>>> Have you looked into set_printoptions
>>>>> <https://docs.scipy.org/doc/numpy-1.15.1/reference/generated/numpy.set_printoptions.html>
>>>>> to see how numpy?s existing capabilities might help you with your
>>>>> visualization?
>>>>>
>>>>> The code you showed seems quite good. I wouldn?t worry about
>>>>> performance when it comes to functions that will seldom be called in tight
>>>>> loops.
>>>>> As you?ll learn more about python and numpy, you?ll keep expanding it
>>>>> to include more use cases.
>>>>> For many of my projects, I create small submodules for visualization
>>>>> tailored to the specific needs of the particular project.
>>>>> I?ll try to incorporate your functions and see how I use them.
>>>>>
>>>>> Your original post seems to have some confusion about C Style vs F
>>>>> Style ordering. I hope that has been resolved.
>>>>> There is also a lot of good documentation
>>>>>
>>>>> https://docs.scipy.org/doc/numpy/user/numpy-for-matlab-users.html#numpy-for-matlab-users-notes
>>>>> about transitioning from matlab.
>>>>>
>>>>> Mark
>>>>>
>>>>> On Mon, Nov 5, 2018 at 4:46 PM Foad Sojoodi Farimani <
>>>>> f.s.farimani at gmail.com> wrote:
>>>>>
>>>>>> Hello everyone,
>>>>>>
>>>>>> Following this question
>>>>>> <https://stackoverflow.com/q/53126305/4999991>, I'm convinced that
>>>>>> numpy ndarrays are not MATLAB/mathematical multidimentional matrices and I
>>>>>> should stop expecting them to be. However I still think it would have a lot
>>>>>> of benefit to have a function like sympy's pprint to pretty print.
>>>>>> something like pandas .head and .tail method plus  .left .right .UpLeft
>>>>>> .UpRight .DownLeft .DownRight methods. when nothing mentioned it would show
>>>>>> 4 corners and put dots in the middle if the array is to big for the
>>>>>> terminal.
>>>>>>
>>>>>> Best,
>>>>>> Foad
>>>>>> _______________________________________________
>>>>>> NumPy-Discussion mailing list
>>>>>> NumPy-Discussion at python.org
>>>>>> https://mail.python.org/mailman/listinfo/numpy-discussion
>>>>>>
>>>>> _______________________________________________
>>>>> NumPy-Discussion mailing list
>>>>> NumPy-Discussion at python.org
>>>>> https://mail.python.org/mailman/listinfo/numpy-discussion
>>>>>
>>>> _______________________________________________
>>>> NumPy-Discussion mailing list
>>>> NumPy-Discussion at python.org
>>>> https://mail.python.org/mailman/listinfo/numpy-discussion
>>>>
>>> _______________________________________________
>>> NumPy-Discussion mailing list
>>> NumPy-Discussion at python.org
>>> https://mail.python.org/mailman/listinfo/numpy-discussion
>>>
>> _______________________________________________
>> NumPy-Discussion mailing list
>> NumPy-Discussion at python.org
>> https://mail.python.org/mailman/listinfo/numpy-discussion
>>
> _______________________________________________
> NumPy-Discussion mailing list
> NumPy-Discussion at python.org
> https://mail.python.org/mailman/listinfo/numpy-discussion
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20181106/201943a1/attachment.html>

From f.s.farimani at gmail.com  Tue Nov  6 05:16:23 2018
From: f.s.farimani at gmail.com (Foad Sojoodi Farimani)
Date: Tue, 6 Nov 2018 11:16:23 +0100
Subject: [Numpy-discussion] numpy pprint?
In-Reply-To: <CAC=AwPzafbhmY3oOYUHUa-0b9+G4FNxDy_pa6VQbPXK8D6+HKQ@mail.gmail.com>
References: <CADjqfdp7VO_3KfwJTC-_mVLVnPyxbzZuY6+=hAzmvooe8NE+sg@mail.gmail.com>
 <CAC=AwPy8qnC6NNH0NTmkzp7zGr+KaZ6Mcp=0sC-FhaEUX2hgXw@mail.gmail.com>
 <CAL1kJvDbUZzu1Ky6d2wK=b-mDh2FLkLjoK5hYp+H3Jko9ScCYw@mail.gmail.com>
 <CADjqfdoY9Bjy-7-hvj4mDUOm5PnY_P3BCz4YaYNL0Qw_o75c7w@mail.gmail.com>
 <CAL1kJvBo3cZ2XY1kni3addVDG3bERv93=FH1qycM58efVX8xug@mail.gmail.com>
 <CADjqfdqZP2EN=k-Ww1+E0DY462BT3av231+Hpf9Tz7iOtLvgkA@mail.gmail.com>
 <CAC=AwPzafbhmY3oOYUHUa-0b9+G4FNxDy_pa6VQbPXK8D6+HKQ@mail.gmail.com>
Message-ID: <CADjqfdr=NyP8FT_wpHEgbnsg1Y0T6-pm73QBGRkBKySiPq9WTw@mail.gmail.com>

In between your lines:

On Tue, Nov 6, 2018 at 11:07 AM Mark Harfouche <mark.harfouche at gmail.com>
wrote:

> Foad,
> In response to:
>
> Thanks but I know it is very bad:
>
>    - it does not work properly for floats
>    - it only works for 1D and 2D
>    - there can be some recursive function I believe.
>
> I think this is the awesome part about being able to write 10 lines of
> code that are specified to representing exactly 1 thing.
>
> Other than that, yeah, encouraging people to transition from matlab is
> challenging. Matlab is definitely good at doing matrix operations. Python3
> somewhat helps in that regard.
>
> I'm super glad you are bringing usability issues up and working toward
> solving them.
>
> Maybe you can describe the interface for python you find practical to
> introduce to newcomers so as to motivate the discussion?
>

I have been thinking about Spyder but it has a lot of issues with the
standard python distribution and pip. Jupyterlab would be awesome except
some Jupyter Notebook extensions are missing. For example variable
inspector, RISE for slides, Hinterland, ... For the moment Jupyter Notebook
is the most reliable/complete I could find.

F.


>
> Mark
>
>
> On Tue, Nov 6, 2018 at 3:57 AM Foad Sojoodi Farimani <
> f.s.farimani at gmail.com> wrote:
>
>> Wow, this is awesome.
>> Some points though:
>>
>>    - not everybody uses IPython/Jupyter having the functionality for
>>    conventional consols would also help. something like
>>    Sypy's init_printing/init_session which smartly chooses the right
>>    representation considering the terminal.
>>    - I don't think putting everything in boxes is helping. it is
>>    confusing. I would rather having horizontal and vertical square brackets
>>    represent each nested array
>>    - it would be awesome if in IPython/Jupyter hovering over an element
>>    a popup would show the index
>>    - one could read the width and height of the terminal and other
>>    options I mentioned in reply Mark to show L R U P or combination of these
>>    plus some numbers (similar to Pandas .head .tail) methods and then show the
>>    rest by unicod 3dot
>>
>> P.S. I had no idea our university Microsoft services also offers Azure
>> Notebooks awesome :P
>>
>> F.
>>
>> On Tue, Nov 6, 2018 at 9:45 AM Eric Wieser <wieser.eric+numpy at gmail.com>
>> wrote:
>>
>>> Here's how that could look
>>>
>>>
>>> https://numpyintegration-ericwieser.notebooks.azure.com/j/notebooks/pprint.ipynb
>>>
>>> Feel free to play around and see if you can produce something more useful
>>>
>>>
>>>
>>> On Mon, 5 Nov 2018 at 23:28 Foad Sojoodi Farimani <
>>> f.s.farimani at gmail.com> wrote:
>>>
>>>> It is not highking if I asked for it :))
>>>> for IPython/Jupyter using Markdown/LaTeX would be awesome
>>>> or even better using HTML to add sliders just like Pandas...
>>>>
>>>> F.
>>>>
>>>> On Tue, Nov 6, 2018 at 6:51 AM Eric Wieser <wieser.eric+numpy at gmail.com>
>>>> wrote:
>>>>
>>>>> Hijacking this thread while on the topic of pprint - we might want to
>>>>> look into a table-based `_html_repr_` or `_latex_repr_` for use in ipython
>>>>> - where we can print the full array and let scrollbars replace ellipses.
>>>>>
>>>>> Eric
>>>>>
>>>>> On Mon, 5 Nov 2018 at 21:11 Mark Harfouche <mark.harfouche at gmail.com>
>>>>> wrote:
>>>>>
>>>>>> Foad,
>>>>>>
>>>>>> Visualizing data is definitely a complex field. I definitely feel
>>>>>> your pain.
>>>>>> Printing your data is but one way of visualizing it, and probably
>>>>>> only useful for very small and constrained datasets.
>>>>>> Have you looked into set_printoptions
>>>>>> <https://docs.scipy.org/doc/numpy-1.15.1/reference/generated/numpy.set_printoptions.html>
>>>>>> to see how numpy?s existing capabilities might help you with your
>>>>>> visualization?
>>>>>>
>>>>>> The code you showed seems quite good. I wouldn?t worry about
>>>>>> performance when it comes to functions that will seldom be called in tight
>>>>>> loops.
>>>>>> As you?ll learn more about python and numpy, you?ll keep expanding it
>>>>>> to include more use cases.
>>>>>> For many of my projects, I create small submodules for visualization
>>>>>> tailored to the specific needs of the particular project.
>>>>>> I?ll try to incorporate your functions and see how I use them.
>>>>>>
>>>>>> Your original post seems to have some confusion about C Style vs F
>>>>>> Style ordering. I hope that has been resolved.
>>>>>> There is also a lot of good documentation
>>>>>>
>>>>>> https://docs.scipy.org/doc/numpy/user/numpy-for-matlab-users.html#numpy-for-matlab-users-notes
>>>>>> about transitioning from matlab.
>>>>>>
>>>>>> Mark
>>>>>>
>>>>>> On Mon, Nov 5, 2018 at 4:46 PM Foad Sojoodi Farimani <
>>>>>> f.s.farimani at gmail.com> wrote:
>>>>>>
>>>>>>> Hello everyone,
>>>>>>>
>>>>>>> Following this question
>>>>>>> <https://stackoverflow.com/q/53126305/4999991>, I'm convinced that
>>>>>>> numpy ndarrays are not MATLAB/mathematical multidimentional matrices and I
>>>>>>> should stop expecting them to be. However I still think it would have a lot
>>>>>>> of benefit to have a function like sympy's pprint to pretty print.
>>>>>>> something like pandas .head and .tail method plus  .left .right .UpLeft
>>>>>>> .UpRight .DownLeft .DownRight methods. when nothing mentioned it would show
>>>>>>> 4 corners and put dots in the middle if the array is to big for the
>>>>>>> terminal.
>>>>>>>
>>>>>>> Best,
>>>>>>> Foad
>>>>>>> _______________________________________________
>>>>>>> NumPy-Discussion mailing list
>>>>>>> NumPy-Discussion at python.org
>>>>>>> https://mail.python.org/mailman/listinfo/numpy-discussion
>>>>>>>
>>>>>> _______________________________________________
>>>>>> NumPy-Discussion mailing list
>>>>>> NumPy-Discussion at python.org
>>>>>> https://mail.python.org/mailman/listinfo/numpy-discussion
>>>>>>
>>>>> _______________________________________________
>>>>> NumPy-Discussion mailing list
>>>>> NumPy-Discussion at python.org
>>>>> https://mail.python.org/mailman/listinfo/numpy-discussion
>>>>>
>>>> _______________________________________________
>>>> NumPy-Discussion mailing list
>>>> NumPy-Discussion at python.org
>>>> https://mail.python.org/mailman/listinfo/numpy-discussion
>>>>
>>> _______________________________________________
>>> NumPy-Discussion mailing list
>>> NumPy-Discussion at python.org
>>> https://mail.python.org/mailman/listinfo/numpy-discussion
>>>
>> _______________________________________________
>> NumPy-Discussion mailing list
>> NumPy-Discussion at python.org
>> https://mail.python.org/mailman/listinfo/numpy-discussion
>>
> _______________________________________________
> NumPy-Discussion mailing list
> NumPy-Discussion at python.org
> https://mail.python.org/mailman/listinfo/numpy-discussion
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20181106/0d6369cd/attachment-0001.html>

From mark.harfouche at gmail.com  Tue Nov  6 05:41:20 2018
From: mark.harfouche at gmail.com (Mark Harfouche)
Date: Tue, 6 Nov 2018 05:41:20 -0500
Subject: [Numpy-discussion] numpy pprint?
In-Reply-To: <CADjqfdr=NyP8FT_wpHEgbnsg1Y0T6-pm73QBGRkBKySiPq9WTw@mail.gmail.com>
References: <CADjqfdp7VO_3KfwJTC-_mVLVnPyxbzZuY6+=hAzmvooe8NE+sg@mail.gmail.com>
 <CAC=AwPy8qnC6NNH0NTmkzp7zGr+KaZ6Mcp=0sC-FhaEUX2hgXw@mail.gmail.com>
 <CAL1kJvDbUZzu1Ky6d2wK=b-mDh2FLkLjoK5hYp+H3Jko9ScCYw@mail.gmail.com>
 <CADjqfdoY9Bjy-7-hvj4mDUOm5PnY_P3BCz4YaYNL0Qw_o75c7w@mail.gmail.com>
 <CAL1kJvBo3cZ2XY1kni3addVDG3bERv93=FH1qycM58efVX8xug@mail.gmail.com>
 <CADjqfdqZP2EN=k-Ww1+E0DY462BT3av231+Hpf9Tz7iOtLvgkA@mail.gmail.com>
 <CAC=AwPzafbhmY3oOYUHUa-0b9+G4FNxDy_pa6VQbPXK8D6+HKQ@mail.gmail.com>
 <CADjqfdr=NyP8FT_wpHEgbnsg1Y0T6-pm73QBGRkBKySiPq9WTw@mail.gmail.com>
Message-ID: <CAC=AwPwsng=MFiYdRkqDXJ882vo9YeJw8vFoOz62Rk69zZNiqA@mail.gmail.com>

To install spyder I wonder if Anaconda is a possibility.

It also installs a lot of packages that your pupils/peers might be using
but that you might not anticipate.

On a semi-related note, a recent change to the repr broke a lot of
downstream tests. Htlm and latex reprs are probably easier to experiment
with on that sense.

That said. That might just be a doctest issue and not a numpy issue.

On Tue, Nov 6, 2018 at 5:18 AM Foad Sojoodi Farimani <f.s.farimani at gmail.com>
wrote:

> In between your lines:
>
> On Tue, Nov 6, 2018 at 11:07 AM Mark Harfouche <mark.harfouche at gmail.com>
> wrote:
>
>> Foad,
>> In response to:
>>
>> Thanks but I know it is very bad:
>>
>>    - it does not work properly for floats
>>    - it only works for 1D and 2D
>>    - there can be some recursive function I believe.
>>
>> I think this is the awesome part about being able to write 10 lines of
>> code that are specified to representing exactly 1 thing.
>>
>> Other than that, yeah, encouraging people to transition from matlab is
>> challenging. Matlab is definitely good at doing matrix operations. Python3
>> somewhat helps in that regard.
>>
>> I'm super glad you are bringing usability issues up and working toward
>> solving them.
>>
>> Maybe you can describe the interface for python you find practical to
>> introduce to newcomers so as to motivate the discussion?
>>
>
> I have been thinking about Spyder but it has a lot of issues with the
> standard python distribution and pip. Jupyterlab would be awesome except
> some Jupyter Notebook extensions are missing. For example variable
> inspector, RISE for slides, Hinterland, ... For the moment Jupyter Notebook
> is the most reliable/complete I could find.
>
> F.
>
>
>>
>> Mark
>>
>>
>> On Tue, Nov 6, 2018 at 3:57 AM Foad Sojoodi Farimani <
>> f.s.farimani at gmail.com> wrote:
>>
>>> Wow, this is awesome.
>>> Some points though:
>>>
>>>    - not everybody uses IPython/Jupyter having the functionality for
>>>    conventional consols would also help. something like
>>>    Sypy's init_printing/init_session which smartly chooses the right
>>>    representation considering the terminal.
>>>    - I don't think putting everything in boxes is helping. it is
>>>    confusing. I would rather having horizontal and vertical square brackets
>>>    represent each nested array
>>>    - it would be awesome if in IPython/Jupyter hovering over an element
>>>    a popup would show the index
>>>    - one could read the width and height of the terminal and other
>>>    options I mentioned in reply Mark to show L R U P or combination of these
>>>    plus some numbers (similar to Pandas .head .tail) methods and then show the
>>>    rest by unicod 3dot
>>>
>>> P.S. I had no idea our university Microsoft services also offers Azure
>>> Notebooks awesome :P
>>>
>>> F.
>>>
>>> On Tue, Nov 6, 2018 at 9:45 AM Eric Wieser <wieser.eric+numpy at gmail.com>
>>> wrote:
>>>
>>>> Here's how that could look
>>>>
>>>>
>>>> https://numpyintegration-ericwieser.notebooks.azure.com/j/notebooks/pprint.ipynb
>>>>
>>>> Feel free to play around and see if you can produce something more
>>>> useful
>>>>
>>>>
>>>>
>>>> On Mon, 5 Nov 2018 at 23:28 Foad Sojoodi Farimani <
>>>> f.s.farimani at gmail.com> wrote:
>>>>
>>>>> It is not highking if I asked for it :))
>>>>> for IPython/Jupyter using Markdown/LaTeX would be awesome
>>>>> or even better using HTML to add sliders just like Pandas...
>>>>>
>>>>> F.
>>>>>
>>>>> On Tue, Nov 6, 2018 at 6:51 AM Eric Wieser <
>>>>> wieser.eric+numpy at gmail.com> wrote:
>>>>>
>>>>>> Hijacking this thread while on the topic of pprint - we might want to
>>>>>> look into a table-based `_html_repr_` or `_latex_repr_` for use in ipython
>>>>>> - where we can print the full array and let scrollbars replace ellipses.
>>>>>>
>>>>>> Eric
>>>>>>
>>>>>> On Mon, 5 Nov 2018 at 21:11 Mark Harfouche <mark.harfouche at gmail.com>
>>>>>> wrote:
>>>>>>
>>>>>>> Foad,
>>>>>>>
>>>>>>> Visualizing data is definitely a complex field. I definitely feel
>>>>>>> your pain.
>>>>>>> Printing your data is but one way of visualizing it, and probably
>>>>>>> only useful for very small and constrained datasets.
>>>>>>> Have you looked into set_printoptions
>>>>>>> <https://docs.scipy.org/doc/numpy-1.15.1/reference/generated/numpy.set_printoptions.html>
>>>>>>> to see how numpy?s existing capabilities might help you with your
>>>>>>> visualization?
>>>>>>>
>>>>>>> The code you showed seems quite good. I wouldn?t worry about
>>>>>>> performance when it comes to functions that will seldom be called in tight
>>>>>>> loops.
>>>>>>> As you?ll learn more about python and numpy, you?ll keep expanding
>>>>>>> it to include more use cases.
>>>>>>> For many of my projects, I create small submodules for visualization
>>>>>>> tailored to the specific needs of the particular project.
>>>>>>> I?ll try to incorporate your functions and see how I use them.
>>>>>>>
>>>>>>> Your original post seems to have some confusion about C Style vs F
>>>>>>> Style ordering. I hope that has been resolved.
>>>>>>> There is also a lot of good documentation
>>>>>>>
>>>>>>> https://docs.scipy.org/doc/numpy/user/numpy-for-matlab-users.html#numpy-for-matlab-users-notes
>>>>>>> about transitioning from matlab.
>>>>>>>
>>>>>>> Mark
>>>>>>>
>>>>>>> On Mon, Nov 5, 2018 at 4:46 PM Foad Sojoodi Farimani <
>>>>>>> f.s.farimani at gmail.com> wrote:
>>>>>>>
>>>>>>>> Hello everyone,
>>>>>>>>
>>>>>>>> Following this question
>>>>>>>> <https://stackoverflow.com/q/53126305/4999991>, I'm convinced that
>>>>>>>> numpy ndarrays are not MATLAB/mathematical multidimentional matrices and I
>>>>>>>> should stop expecting them to be. However I still think it would have a lot
>>>>>>>> of benefit to have a function like sympy's pprint to pretty print.
>>>>>>>> something like pandas .head and .tail method plus  .left .right .UpLeft
>>>>>>>> .UpRight .DownLeft .DownRight methods. when nothing mentioned it would show
>>>>>>>> 4 corners and put dots in the middle if the array is to big for the
>>>>>>>> terminal.
>>>>>>>>
>>>>>>>> Best,
>>>>>>>> Foad
>>>>>>>> _______________________________________________
>>>>>>>> NumPy-Discussion mailing list
>>>>>>>> NumPy-Discussion at python.org
>>>>>>>> https://mail.python.org/mailman/listinfo/numpy-discussion
>>>>>>>>
>>>>>>> _______________________________________________
>>>>>>> NumPy-Discussion mailing list
>>>>>>> NumPy-Discussion at python.org
>>>>>>> https://mail.python.org/mailman/listinfo/numpy-discussion
>>>>>>>
>>>>>> _______________________________________________
>>>>>> NumPy-Discussion mailing list
>>>>>> NumPy-Discussion at python.org
>>>>>> https://mail.python.org/mailman/listinfo/numpy-discussion
>>>>>>
>>>>> _______________________________________________
>>>>> NumPy-Discussion mailing list
>>>>> NumPy-Discussion at python.org
>>>>> https://mail.python.org/mailman/listinfo/numpy-discussion
>>>>>
>>>> _______________________________________________
>>>> NumPy-Discussion mailing list
>>>> NumPy-Discussion at python.org
>>>> https://mail.python.org/mailman/listinfo/numpy-discussion
>>>>
>>> _______________________________________________
>>> NumPy-Discussion mailing list
>>> NumPy-Discussion at python.org
>>> https://mail.python.org/mailman/listinfo/numpy-discussion
>>>
>> _______________________________________________
>> NumPy-Discussion mailing list
>> NumPy-Discussion at python.org
>> https://mail.python.org/mailman/listinfo/numpy-discussion
>>
> _______________________________________________
> NumPy-Discussion mailing list
> NumPy-Discussion at python.org
> https://mail.python.org/mailman/listinfo/numpy-discussion
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20181106/7990efb5/attachment-0001.html>

From deak.andris at gmail.com  Tue Nov  6 05:43:01 2018
From: deak.andris at gmail.com (Andras Deak)
Date: Tue, 6 Nov 2018 11:43:01 +0100
Subject: [Numpy-discussion] numpy pprint?
In-Reply-To: <CADjqfdopCXA1BOvZF3TiPRkKWZB0NC1VF0pQ83zkA8UTT=Y4Zw@mail.gmail.com>
References: <CADjqfdp7VO_3KfwJTC-_mVLVnPyxbzZuY6+=hAzmvooe8NE+sg@mail.gmail.com>
 <CAC=AwPy8qnC6NNH0NTmkzp7zGr+KaZ6Mcp=0sC-FhaEUX2hgXw@mail.gmail.com>
 <CADjqfdopCXA1BOvZF3TiPRkKWZB0NC1VF0pQ83zkA8UTT=Y4Zw@mail.gmail.com>
Message-ID: <CAMEWA4NS5+k+PLbq8h02-FHvWCohOdLTKkF1Ms+ni3MUxL4=Cw@mail.gmail.com>

On Tue, Nov 6, 2018 at 8:26 AM Foad Sojoodi Farimani
<f.s.farimani at gmail.com> wrote:
>
> Dear Mark,
>
> Thanks for the reply. I will write in between your lines:
>
> On Tue, Nov 6, 2018 at 6:11 AM Mark Harfouche <mark.harfouche at gmail.com> wrote:
>>
>> Foad,
>>
>> Visualizing data is definitely a complex field. I definitely feel your pain.
>
> I have actually been using numpy for a couple of years without noticing these issues. recently I have been trying to encourage my collogues to move from MATLAB to Python and also prepare some workshops for PhD network of my university.
>>
>> Printing your data is but one way of visualizing it, and probably only useful for very small and constrained datasets.
>
> well actually it can be very useful. Consider Pandas .head() and .tail() methods or Sympy's pretty printing functionalities. for bigger datasets the function can get the terminals width and height and then based on the input (U(n),D(n),L(n),R(n),UR(n,m),UL(n,m),DR(n,m),DL(n,m)) display what can be shown and put horizontal 3-dots \u2026 ? or vertical/inclined ones. Or id it is Jupyter then one can use Markdown/LaTeX for pretty printing or even HTML to add sliders as suggested by Eric.
>>
>> Have you looked into set_printoptions to see how numpy?s existing capabilities might help you with your visualization?
>
> This is indeed very useful. specially the threshold option can help a lot with adjusting the width. but only for specific cases.
>>
>> The code you showed seems quite good. I wouldn?t worry about performance when it comes to functions that will seldom be called in tight loops.
>
> Thanks but I know it is very bad:
>
> it does not work properly for floats
> it only works for 1D and 2D
> there can be some recursive function I believe.
>>
>> As you?ll learn more about python and numpy, you?ll keep expanding it to include more use cases.
>> For many of my projects, I create small submodules for visualization tailored to the specific needs of the particular project.
>> I?ll try to incorporate your functions and see how I use them.
>
> Thanks a lot. looking forwards to your feedback
>>
>> Your original post seems to have some confusion about C Style vs F Style ordering. I hope that has been resolved.
>
> I actually came to the conclusion that calling it C-Style or F-Style or maybe row-major column-major are bad practices. Numpy's ndarrays are not mathematical multidimensional arrays but Pythons nested, homogenous and uniform lists.  it means for example 1, [1], [[1]] and [[[1]]] are all different, while in all other mathematical languages out there (including Sympy's matrices) they are the same.

I'm probably missing your point, because I don't understand your
claim. Mathematically speaking, 1 and [1] and [[1] and [[[1]]] are
different objects. One is a scalar, the second is an element of R^n
with n=1 which is basically a scalar too from a math perspective, the
third one is a 2-index object (an operator acting on R^1), the last
one is a three-index object. These are all mathematically distinct.
Furthermore, row-major and column-major order are a purely technical
detail describing how the underlying data that is being represented by
these multidimensional arrays is laid out in memory. So C/F-style
order and the semantics of multidimensional arrays, at least as I see
it, are independent notions.

Andr?s

From f.s.farimani at gmail.com  Tue Nov  6 05:55:47 2018
From: f.s.farimani at gmail.com (Foad Sojoodi Farimani)
Date: Tue, 6 Nov 2018 11:55:47 +0100
Subject: [Numpy-discussion] numpy pprint?
In-Reply-To: <CAMEWA4NS5+k+PLbq8h02-FHvWCohOdLTKkF1Ms+ni3MUxL4=Cw@mail.gmail.com>
References: <CADjqfdp7VO_3KfwJTC-_mVLVnPyxbzZuY6+=hAzmvooe8NE+sg@mail.gmail.com>
 <CAC=AwPy8qnC6NNH0NTmkzp7zGr+KaZ6Mcp=0sC-FhaEUX2hgXw@mail.gmail.com>
 <CADjqfdopCXA1BOvZF3TiPRkKWZB0NC1VF0pQ83zkA8UTT=Y4Zw@mail.gmail.com>
 <CAMEWA4NS5+k+PLbq8h02-FHvWCohOdLTKkF1Ms+ni3MUxL4=Cw@mail.gmail.com>
Message-ID: <CADjqfdrUbqFZ8U1pm7DToC-oPxwNJkcdQYLEoqz7iRG7_-nn9Q@mail.gmail.com>

Dear Andr?s,

Try those different option in MATLAB for example. or
Octave/Scilab/Sympy-Matrix... they are all the same. The term
"multidimensional arrays"  is a little bit vague. one might think of
multidimensional matrices ( I don't think there is such a thing in math) if
coming from MATLAB. I also think the row-major column major terminology is
confusing. there are no rows or columns for that matter. Numpy ndarrays are
homogeneous, uniform nested lists. one can represent different layers of
this list in different ways using rows or columns.

regardless of all these different point of views having graphical and
pretty printing representations would help a lot. that's my main goal at
the moment.

Best,
Foad

On Tue, Nov 6, 2018 at 11:43 AM Andras Deak <deak.andris at gmail.com> wrote:

> On Tue, Nov 6, 2018 at 8:26 AM Foad Sojoodi Farimani
> <f.s.farimani at gmail.com> wrote:
> >
> > Dear Mark,
> >
> > Thanks for the reply. I will write in between your lines:
> >
> > On Tue, Nov 6, 2018 at 6:11 AM Mark Harfouche <mark.harfouche at gmail.com>
> wrote:
> >>
> >> Foad,
> >>
> >> Visualizing data is definitely a complex field. I definitely feel your
> pain.
> >
> > I have actually been using numpy for a couple of years without noticing
> these issues. recently I have been trying to encourage my collogues to move
> from MATLAB to Python and also prepare some workshops for PhD network of my
> university.
> >>
> >> Printing your data is but one way of visualizing it, and probably only
> useful for very small and constrained datasets.
> >
> > well actually it can be very useful. Consider Pandas .head() and .tail()
> methods or Sympy's pretty printing functionalities. for bigger datasets the
> function can get the terminals width and height and then based on the input
> (U(n),D(n),L(n),R(n),UR(n,m),UL(n,m),DR(n,m),DL(n,m)) display what can be
> shown and put horizontal 3-dots \u2026 ? or vertical/inclined ones. Or id
> it is Jupyter then one can use Markdown/LaTeX for pretty printing or even
> HTML to add sliders as suggested by Eric.
> >>
> >> Have you looked into set_printoptions to see how numpy?s existing
> capabilities might help you with your visualization?
> >
> > This is indeed very useful. specially the threshold option can help a
> lot with adjusting the width. but only for specific cases.
> >>
> >> The code you showed seems quite good. I wouldn?t worry about
> performance when it comes to functions that will seldom be called in tight
> loops.
> >
> > Thanks but I know it is very bad:
> >
> > it does not work properly for floats
> > it only works for 1D and 2D
> > there can be some recursive function I believe.
> >>
> >> As you?ll learn more about python and numpy, you?ll keep expanding it
> to include more use cases.
> >> For many of my projects, I create small submodules for visualization
> tailored to the specific needs of the particular project.
> >> I?ll try to incorporate your functions and see how I use them.
> >
> > Thanks a lot. looking forwards to your feedback
> >>
> >> Your original post seems to have some confusion about C Style vs F
> Style ordering. I hope that has been resolved.
> >
> > I actually came to the conclusion that calling it C-Style or F-Style or
> maybe row-major column-major are bad practices. Numpy's ndarrays are not
> mathematical multidimensional arrays but Pythons nested, homogenous and
> uniform lists.  it means for example 1, [1], [[1]] and [[[1]]] are all
> different, while in all other mathematical languages out there (including
> Sympy's matrices) they are the same.
>
> I'm probably missing your point, because I don't understand your
> claim. Mathematically speaking, 1 and [1] and [[1] and [[[1]]] are
> different objects. One is a scalar, the second is an element of R^n
> with n=1 which is basically a scalar too from a math perspective, the
> third one is a 2-index object (an operator acting on R^1), the last
> one is a three-index object. These are all mathematically distinct.
> Furthermore, row-major and column-major order are a purely technical
> detail describing how the underlying data that is being represented by
> these multidimensional arrays is laid out in memory. So C/F-style
> order and the semantics of multidimensional arrays, at least as I see
> it, are independent notions.
>
> Andr?s
> _______________________________________________
> NumPy-Discussion mailing list
> NumPy-Discussion at python.org
> https://mail.python.org/mailman/listinfo/numpy-discussion
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20181106/810acd12/attachment.html>

From charlesr.harris at gmail.com  Tue Nov  6 09:42:05 2018
From: charlesr.harris at gmail.com (Charles R Harris)
Date: Tue, 6 Nov 2018 07:42:05 -0700
Subject: [Numpy-discussion] numpy pprint?
In-Reply-To: <CADjqfdrUbqFZ8U1pm7DToC-oPxwNJkcdQYLEoqz7iRG7_-nn9Q@mail.gmail.com>
References: <CADjqfdp7VO_3KfwJTC-_mVLVnPyxbzZuY6+=hAzmvooe8NE+sg@mail.gmail.com>
 <CAC=AwPy8qnC6NNH0NTmkzp7zGr+KaZ6Mcp=0sC-FhaEUX2hgXw@mail.gmail.com>
 <CADjqfdopCXA1BOvZF3TiPRkKWZB0NC1VF0pQ83zkA8UTT=Y4Zw@mail.gmail.com>
 <CAMEWA4NS5+k+PLbq8h02-FHvWCohOdLTKkF1Ms+ni3MUxL4=Cw@mail.gmail.com>
 <CADjqfdrUbqFZ8U1pm7DToC-oPxwNJkcdQYLEoqz7iRG7_-nn9Q@mail.gmail.com>
Message-ID: <CAB6mnxKDPjO-L9PKiA2RPcwHwVrLkg0M60r4xT_Pnev-D8atUg@mail.gmail.com>

On Tue, Nov 6, 2018 at 3:56 AM Foad Sojoodi Farimani <f.s.farimani at gmail.com>
wrote:

> Dear Andr?s,
>
> Try those different option in MATLAB for example. or
> Octave/Scilab/Sympy-Matrix... they are all the same. The term
> "multidimensional arrays"  is a little bit vague. one might think of
> multidimensional matrices ( I don't think there is such a thing in math) if
> coming from MATLAB. I also think the row-major column major terminology is
> confusing. there are no rows or columns for that matter. Numpy ndarrays are
> homogeneous, uniform nested lists. one can represent different layers of
> this list in different ways using rows or columns.
>
>
I think the current popular terminology is `tensors` for `multidimensional
arrays`. Note that matrices are a different type of object.

<snip>

Chuck
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20181106/edd6ca19/attachment-0001.html>

From robert.kern at gmail.com  Tue Nov  6 15:11:13 2018
From: robert.kern at gmail.com (Robert Kern)
Date: Tue, 6 Nov 2018 12:11:13 -0800
Subject: [Numpy-discussion] numpy pprint?
In-Reply-To: <CAB6mnxKDPjO-L9PKiA2RPcwHwVrLkg0M60r4xT_Pnev-D8atUg@mail.gmail.com>
References: <CADjqfdp7VO_3KfwJTC-_mVLVnPyxbzZuY6+=hAzmvooe8NE+sg@mail.gmail.com>
 <CAC=AwPy8qnC6NNH0NTmkzp7zGr+KaZ6Mcp=0sC-FhaEUX2hgXw@mail.gmail.com>
 <CADjqfdopCXA1BOvZF3TiPRkKWZB0NC1VF0pQ83zkA8UTT=Y4Zw@mail.gmail.com>
 <CAMEWA4NS5+k+PLbq8h02-FHvWCohOdLTKkF1Ms+ni3MUxL4=Cw@mail.gmail.com>
 <CADjqfdrUbqFZ8U1pm7DToC-oPxwNJkcdQYLEoqz7iRG7_-nn9Q@mail.gmail.com>
 <CAB6mnxKDPjO-L9PKiA2RPcwHwVrLkg0M60r4xT_Pnev-D8atUg@mail.gmail.com>
Message-ID: <CAF6FJiuTqcvM1irVmg1jyxQC9jKjeVBvckkR5z6VBNEtS=do7g@mail.gmail.com>

On Tue, Nov 6, 2018 at 6:43 AM Charles R Harris <charlesr.harris at gmail.com>
wrote:

>
> On Tue, Nov 6, 2018 at 3:56 AM Foad Sojoodi Farimani <
> f.s.farimani at gmail.com> wrote:
>
>> Dear Andr?s,
>>
>> Try those different option in MATLAB for example. or
>> Octave/Scilab/Sympy-Matrix... they are all the same.
>>
>
Of course, these are all systems with a focus on matrices per se rather
than general arrays. They take liberties with non-2-dim arrays that make
sense if the focus is on treating 2-dim arrays as matrices. One of the
motivating reason's for numpy's early development (as Numeric) was to get
away from those assumptions and limitations and be a general array
processing system. Part of the reason that we choose the terminology
"multidimensional array" is to emphasize those differences.


> The term "multidimensional arrays"  is a little bit vague. one might think
>> of multidimensional matrices ( I don't think there is such a thing in math)
>> if coming from MATLAB. I also think the row-major column major terminology
>> is confusing. there are no rows or columns for that matter.
>>
>
Granted, but it's long-established terminology, and not actually important
for a user to know unless if someone is working in C with a flat
representation of the allocated memory.


> Numpy ndarrays are homogeneous, uniform nested lists. one can represent
>> different layers of this list in different ways using rows or columns.
>>
>
You have to be careful here as well. "list" also has semantic baggage. Data
structures are generally only called "lists" in a wide variety of
programming languages if they have cheap appends and other such mutation
operations. numpy arrays don't (as well as the things that we call "arrays"
in FORTRAN and C/C++ that are distinct from what we would call "lists" in
those languages).

Please be assured that "multidimensional array" is terminology that we
didn't make up. It does derive from a tradition of mathematical programming
in FORTRAN and C and makes meaningful semantic distinctions within that
tradition. There are other traditions, and we might well have settled on
different terminology if we had derived from those. We do expect people to
come from a variety of traditions and have a period of adjustment as they
learn some new terminology. That's perfectly reasonable, which is good,
because it is entirely unavoidable. There isn't a universal set of
terminology that's going to work with everyone's experience out of the gate.

I think the current popular terminology is `tensors` for `multidimensional
> arrays`. Note that matrices are a different type of object.
>

Popular, but quite misleading, in the same way that not every 2-dim array
is a matrix. As someone who works on tensor machine learning methods once
complained to me.

-- 
Robert Kern
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20181106/c8f1b65b/attachment.html>

From f.s.farimani at gmail.com  Tue Nov  6 16:50:54 2018
From: f.s.farimani at gmail.com (Foad Sojoodi Farimani)
Date: Tue, 6 Nov 2018 22:50:54 +0100
Subject: [Numpy-discussion] numpy pprint?
In-Reply-To: <CADjqfdp7VO_3KfwJTC-_mVLVnPyxbzZuY6+=hAzmvooe8NE+sg@mail.gmail.com>
References: <CADjqfdp7VO_3KfwJTC-_mVLVnPyxbzZuY6+=hAzmvooe8NE+sg@mail.gmail.com>
Message-ID: <CADjqfdpKr3m8U+O=+o3ndiEHGzYjLRLDCEfps=EqfWt9E1igvw@mail.gmail.com>

Hello Numpyers,

I just added the pretty printing for 3D arrays too:
https://stackoverflow.com/a/53164538/4999991
I would highly appreciate if you could check the implementation and let me
know how do you think about it.

Best,
Foad

On Mon, Nov 5, 2018 at 10:44 PM Foad Sojoodi Farimani <
f.s.farimani at gmail.com> wrote:

> Hello everyone,
>
> Following this question <https://stackoverflow.com/q/53126305/4999991>,
> I'm convinced that numpy ndarrays are not MATLAB/mathematical
> multidimentional matrices and I should stop expecting them to be. However I
> still think it would have a lot of benefit to have a function like sympy's
> pprint to pretty print. something like pandas .head and .tail method plus
> .left .right .UpLeft .UpRight .DownLeft .DownRight methods. when nothing
> mentioned it would show 4 corners and put dots in the middle if the array
> is to big for the terminal.
>
> Best,
> Foad
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20181106/5f09581b/attachment.html>

From stefanv at berkeley.edu  Tue Nov  6 18:06:40 2018
From: stefanv at berkeley.edu (Stefan van der Walt)
Date: Tue, 6 Nov 2018 15:06:40 -0800
Subject: [Numpy-discussion] Prep for NumPy 1.16.0 branch
In-Reply-To: <CAEQ_Tvf-cuKO5gcNfrXoMNPqnVra1g8dExWbCWRP2sUTKDk99Q@mail.gmail.com>
References: <CAB6mnxJUty767___L5pMOrAKVE-SamvHLVCqTXz3dDvDHum8sQ@mail.gmail.com>
 <CAJNV+9v+F+U--3H91odm=EU7Y=C1Z2WHhD-w34bmOqhZorF7yg@mail.gmail.com>
 <CAEQ_Tvf-cuKO5gcNfrXoMNPqnVra1g8dExWbCWRP2sUTKDk99Q@mail.gmail.com>
Message-ID: <20181106230640.eqgvpmt2wcqglsh4@carbo>

On Sun, 04 Nov 2018 17:16:12 -0800, Stephan Hoyer wrote:
> On Sun, Nov 4, 2018 at 10:32 AM Marten van Kerkwijk <
> m.h.vankerkwijk at gmail.com> wrote:
> 
> > For `__array_function__`, there was some discussion in
> > https://github.com/numpy/numpy/issues/12225 that for 1.16 we might want
> > to follow after all Nathaniel's suggestion of using an environment variable
> > or so to opt in (since introspection breaks on python2 with our wrapped
> > implementations).  Given also the possibly significant hit in performance,
> > this may be the best option.
> > All the best,
> 
> I am also leaning towards this right now, depending on how long we plan to
> wait for releasing 1.16. It will take us at least a little while to sort
> out performance issues for __array_function__, I'd guess at least a few
> weeks. Then a blocker still might turn up during the release candidate
> process (though I think we've found most of the major bugs / downstream
> issues already through tests on NumPy's dev branch).

Just to make sure I understand correctly: the suggestion is to use an
environment variable to temporarily toggle the feature, but the plan in
the long run will be to have it enabled all the time, correct?

St?fan

From stefanv at berkeley.edu  Tue Nov  6 18:52:05 2018
From: stefanv at berkeley.edu (Stefan van der Walt)
Date: Tue, 6 Nov 2018 15:52:05 -0800
Subject: [Numpy-discussion] numpy pprint?
In-Reply-To: <CAF6FJiuTqcvM1irVmg1jyxQC9jKjeVBvckkR5z6VBNEtS=do7g@mail.gmail.com>
References: <CADjqfdp7VO_3KfwJTC-_mVLVnPyxbzZuY6+=hAzmvooe8NE+sg@mail.gmail.com>
 <CAC=AwPy8qnC6NNH0NTmkzp7zGr+KaZ6Mcp=0sC-FhaEUX2hgXw@mail.gmail.com>
 <CADjqfdopCXA1BOvZF3TiPRkKWZB0NC1VF0pQ83zkA8UTT=Y4Zw@mail.gmail.com>
 <CAMEWA4NS5+k+PLbq8h02-FHvWCohOdLTKkF1Ms+ni3MUxL4=Cw@mail.gmail.com>
 <CADjqfdrUbqFZ8U1pm7DToC-oPxwNJkcdQYLEoqz7iRG7_-nn9Q@mail.gmail.com>
 <CAB6mnxKDPjO-L9PKiA2RPcwHwVrLkg0M60r4xT_Pnev-D8atUg@mail.gmail.com>
 <CAF6FJiuTqcvM1irVmg1jyxQC9jKjeVBvckkR5z6VBNEtS=do7g@mail.gmail.com>
Message-ID: <20181106235205.6v4bo6aglwnzsoop@carbo>

On Tue, 06 Nov 2018 12:11:13 -0800, Robert Kern wrote:
> Popular, but quite misleading, in the same way that not every 2-dim array
> is a matrix. As someone who works on tensor machine learning methods once
> complained to me.

Are you referring to vectors, structured arrays, or something else?

St?fan

From robert.kern at gmail.com  Tue Nov  6 19:13:35 2018
From: robert.kern at gmail.com (Robert Kern)
Date: Tue, 6 Nov 2018 16:13:35 -0800
Subject: [Numpy-discussion] numpy pprint?
In-Reply-To: <20181106235205.6v4bo6aglwnzsoop@carbo>
References: <CADjqfdp7VO_3KfwJTC-_mVLVnPyxbzZuY6+=hAzmvooe8NE+sg@mail.gmail.com>
 <CAC=AwPy8qnC6NNH0NTmkzp7zGr+KaZ6Mcp=0sC-FhaEUX2hgXw@mail.gmail.com>
 <CADjqfdopCXA1BOvZF3TiPRkKWZB0NC1VF0pQ83zkA8UTT=Y4Zw@mail.gmail.com>
 <CAMEWA4NS5+k+PLbq8h02-FHvWCohOdLTKkF1Ms+ni3MUxL4=Cw@mail.gmail.com>
 <CADjqfdrUbqFZ8U1pm7DToC-oPxwNJkcdQYLEoqz7iRG7_-nn9Q@mail.gmail.com>
 <CAB6mnxKDPjO-L9PKiA2RPcwHwVrLkg0M60r4xT_Pnev-D8atUg@mail.gmail.com>
 <CAF6FJiuTqcvM1irVmg1jyxQC9jKjeVBvckkR5z6VBNEtS=do7g@mail.gmail.com>
 <20181106235205.6v4bo6aglwnzsoop@carbo>
Message-ID: <CAF6FJivaXzyzw9apaHKCQMxgSPeXZi9re69dcTF+aCrYUJRm+g@mail.gmail.com>

On Tue, Nov 6, 2018 at 3:55 PM Stefan van der Walt <stefanv at berkeley.edu>
wrote:

> On Tue, 06 Nov 2018 12:11:13 -0800, Robert Kern wrote:
> > Popular, but quite misleading, in the same way that not every 2-dim array
> > is a matrix. As someone who works on tensor machine learning methods once
> > complained to me.
>
> Are you referring to vectors, structured arrays, or something else?
>

I was responding to this statement by Chuck:

> I think the current popular terminology is `tensors` for
`multidimensional arrays`.

Mostly popularized by Tensorflow. But the "tensors" that flow through
Tensorflow are mostly just multidimensional arrays and have no
tensor-algebraic meaning. Similarly, a 2-dim array (say, a grayscale
intensity image) doesn't necessarily have a matrix-algebraic
interpretation, either. A 640x480 grayscale image is not a linear
transformation from RR^640 to RR^480. It's just a collection of numbers
that are convenient to organize as a 2D grid.

This seems to be a pain point with some tensor methods ML researchers who
have to explain their work to an audience that seems to think that
Tensorflow must make their lives (and theses) easy. :-)

-- 
Robert Kern
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20181106/55fa0ee0/attachment.html>

From matti.picus at gmail.com  Tue Nov  6 19:28:36 2018
From: matti.picus at gmail.com (Matti Picus)
Date: Tue, 6 Nov 2018 16:28:36 -0800
Subject: [Numpy-discussion] Weekly status meeting 8.11 at 12:00 pacific time
Message-ID: <814a37a5-2650-5a63-f692-d859bd194dd6@gmail.com>

We will be holding our weekly BIDS NumPy status meeting on Thurs Nov 8 
at noon pacific time. We moved the meeting to Thursday because of a 
scheduling conflict. Please join us.

The draft agenda, along with details of how to join, is up at 
https://hackmd.io/TTurMvviSkarcxf8vURq-Q?both

Previous sessions' notes are available at 
https://github.com/BIDS-numpy/docs/tree/master/status_meetings


Matti, Tyler and Stefan


From shoyer at gmail.com  Tue Nov  6 23:09:42 2018
From: shoyer at gmail.com (Stephan Hoyer)
Date: Tue, 6 Nov 2018 23:09:42 -0500
Subject: [Numpy-discussion] Prep for NumPy 1.16.0 branch
In-Reply-To: <20181106230640.eqgvpmt2wcqglsh4@carbo>
References: <CAB6mnxJUty767___L5pMOrAKVE-SamvHLVCqTXz3dDvDHum8sQ@mail.gmail.com>
 <CAJNV+9v+F+U--3H91odm=EU7Y=C1Z2WHhD-w34bmOqhZorF7yg@mail.gmail.com>
 <CAEQ_Tvf-cuKO5gcNfrXoMNPqnVra1g8dExWbCWRP2sUTKDk99Q@mail.gmail.com>
 <20181106230640.eqgvpmt2wcqglsh4@carbo>
Message-ID: <CAEQ_TvdmdP2ZVr1e44MQjM8TF5nTubJkc6p209A86UWQaEZvcg@mail.gmail.com>

On Tue, Nov 6, 2018 at 6:08 PM Stefan van der Walt <stefanv at berkeley.edu>
wrote:

> On Sun, 04 Nov 2018 17:16:12 -0800, Stephan Hoyer wrote:
> > On Sun, Nov 4, 2018 at 10:32 AM Marten van Kerkwijk <
> > m.h.vankerkwijk at gmail.com> wrote:
> >
> > > For `__array_function__`, there was some discussion in
> > > https://github.com/numpy/numpy/issues/12225 that for 1.16 we might
> want
> > > to follow after all Nathaniel's suggestion of using an environment
> variable
> > > or so to opt in (since introspection breaks on python2 with our wrapped
> > > implementations).  Given also the possibly significant hit in
> performance,
> > > this may be the best option.
> > > All the best,
> >
> > I am also leaning towards this right now, depending on how long we plan
> to
> > wait for releasing 1.16. It will take us at least a little while to sort
> > out performance issues for __array_function__, I'd guess at least a few
> > weeks. Then a blocker still might turn up during the release candidate
> > process (though I think we've found most of the major bugs / downstream
> > issues already through tests on NumPy's dev branch).
>
> Just to make sure I understand correctly: the suggestion is to use an
> environment variable to temporarily toggle the feature, but the plan in
> the long run will be to have it enabled all the time, correct?
>

Yes, exactly. __array_function__ would be opt-in only for 1.16 but enabled
by default for 1.17.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20181106/783c13a0/attachment.html>

From wieser.eric+numpy at gmail.com  Wed Nov  7 01:23:33 2018
From: wieser.eric+numpy at gmail.com (Eric Wieser)
Date: Tue, 6 Nov 2018 22:23:33 -0800
Subject: [Numpy-discussion] numpy pprint?
In-Reply-To: <CADjqfdqZP2EN=k-Ww1+E0DY462BT3av231+Hpf9Tz7iOtLvgkA@mail.gmail.com>
References: <CADjqfdp7VO_3KfwJTC-_mVLVnPyxbzZuY6+=hAzmvooe8NE+sg@mail.gmail.com>
 <CAC=AwPy8qnC6NNH0NTmkzp7zGr+KaZ6Mcp=0sC-FhaEUX2hgXw@mail.gmail.com>
 <CAL1kJvDbUZzu1Ky6d2wK=b-mDh2FLkLjoK5hYp+H3Jko9ScCYw@mail.gmail.com>
 <CADjqfdoY9Bjy-7-hvj4mDUOm5PnY_P3BCz4YaYNL0Qw_o75c7w@mail.gmail.com>
 <CAL1kJvBo3cZ2XY1kni3addVDG3bERv93=FH1qycM58efVX8xug@mail.gmail.com>
 <CADjqfdqZP2EN=k-Ww1+E0DY462BT3av231+Hpf9Tz7iOtLvgkA@mail.gmail.com>
Message-ID: <CAL1kJvAJ4aFVP+=AinEMEyE-pFB5EzOnV9M=q5yzwmV3Vju=ow@mail.gmail.com>

Foad:

having the functionality for conventional consols would also help

I think the most important thing in a conventional console is to output the
array in a format that allows you to reconstruct the object. That makes it
way easier for people to reproduce each others problems without having
their full dataset. If your goal is to visualize complex arrays, I think
the console is a pretty limited tool, and numpy already does as much as is
worthwhile there.

I don?t think putting everything in boxes is helping. it is confusing. I
would rather having horizontal and vertical square brackets represent each
nested array

See my update at the same link, which shows an alternative which draws
those brackets as you envi

it would be awesome if in IPython/Jupyter hovering over an element a popup
would show the index

It? already does?

to show L R U P or combination of these plus some numbers

I don?t know what you mean by this.

Eric

On Tue, 6 Nov 2018 at 00:56 Foad Sojoodi Farimani f.s.farimani at gmail.com
<http://mailto:f.s.farimani at gmail.com> wrote:

Wow, this is awesome.
> Some points though:
>
>    - not everybody uses IPython/Jupyter having the functionality for
>    conventional consols would also help. something like
>    Sypy's init_printing/init_session which smartly chooses the right
>    representation considering the terminal.
>    - I don't think putting everything in boxes is helping. it is
>    confusing. I would rather having horizontal and vertical square brackets
>    represent each nested array
>    - it would be awesome if in IPython/Jupyter hovering over an element a
>    popup would show the index
>    - one could read the width and height of the terminal and other
>    options I mentioned in reply Mark to show L R U P or combination of these
>    plus some numbers (similar to Pandas .head .tail) methods and then show the
>    rest by unicod 3dot
>
> P.S. I had no idea our university Microsoft services also offers Azure
> Notebooks awesome :P
>
> F.
>
> On Tue, Nov 6, 2018 at 9:45 AM Eric Wieser <wieser.eric+numpy at gmail.com>
> wrote:
>
>> Here's how that could look
>>
>>
>> https://numpyintegration-ericwieser.notebooks.azure.com/j/notebooks/pprint.ipynb
>>
>> Feel free to play around and see if you can produce something more useful
>>
>>
>>
>> On Mon, 5 Nov 2018 at 23:28 Foad Sojoodi Farimani <f.s.farimani at gmail.com>
>> wrote:
>>
>>> It is not highking if I asked for it :))
>>> for IPython/Jupyter using Markdown/LaTeX would be awesome
>>> or even better using HTML to add sliders just like Pandas...
>>>
>>> F.
>>>
>>> On Tue, Nov 6, 2018 at 6:51 AM Eric Wieser <wieser.eric+numpy at gmail.com>
>>> wrote:
>>>
>>>> Hijacking this thread while on the topic of pprint - we might want to
>>>> look into a table-based `_html_repr_` or `_latex_repr_` for use in ipython
>>>> - where we can print the full array and let scrollbars replace ellipses.
>>>>
>>>> Eric
>>>>
>>>> On Mon, 5 Nov 2018 at 21:11 Mark Harfouche <mark.harfouche at gmail.com>
>>>> wrote:
>>>>
>>>>> Foad,
>>>>>
>>>>> Visualizing data is definitely a complex field. I definitely feel your
>>>>> pain.
>>>>> Printing your data is but one way of visualizing it, and probably only
>>>>> useful for very small and constrained datasets.
>>>>> Have you looked into set_printoptions
>>>>> <https://docs.scipy.org/doc/numpy-1.15.1/reference/generated/numpy.set_printoptions.html>
>>>>> to see how numpy?s existing capabilities might help you with your
>>>>> visualization?
>>>>>
>>>>> The code you showed seems quite good. I wouldn?t worry about
>>>>> performance when it comes to functions that will seldom be called in tight
>>>>> loops.
>>>>> As you?ll learn more about python and numpy, you?ll keep expanding it
>>>>> to include more use cases.
>>>>> For many of my projects, I create small submodules for visualization
>>>>> tailored to the specific needs of the particular project.
>>>>> I?ll try to incorporate your functions and see how I use them.
>>>>>
>>>>> Your original post seems to have some confusion about C Style vs F
>>>>> Style ordering. I hope that has been resolved.
>>>>> There is also a lot of good documentation
>>>>>
>>>>> https://docs.scipy.org/doc/numpy/user/numpy-for-matlab-users.html#numpy-for-matlab-users-notes
>>>>> about transitioning from matlab.
>>>>>
>>>>> Mark
>>>>>
>>>>> On Mon, Nov 5, 2018 at 4:46 PM Foad Sojoodi Farimani <
>>>>> f.s.farimani at gmail.com> wrote:
>>>>>
>>>>>> Hello everyone,
>>>>>>
>>>>>> Following this question
>>>>>> <https://stackoverflow.com/q/53126305/4999991>, I'm convinced that
>>>>>> numpy ndarrays are not MATLAB/mathematical multidimentional matrices and I
>>>>>> should stop expecting them to be. However I still think it would have a lot
>>>>>> of benefit to have a function like sympy's pprint to pretty print.
>>>>>> something like pandas .head and .tail method plus  .left .right .UpLeft
>>>>>> .UpRight .DownLeft .DownRight methods. when nothing mentioned it would show
>>>>>> 4 corners and put dots in the middle if the array is to big for the
>>>>>> terminal.
>>>>>>
>>>>>> Best,
>>>>>> Foad
>>>>>> _______________________________________________
>>>>>> NumPy-Discussion mailing list
>>>>>> NumPy-Discussion at python.org
>>>>>> https://mail.python.org/mailman/listinfo/numpy-discussion
>>>>>>
>>>>> _______________________________________________
>>>>> NumPy-Discussion mailing list
>>>>> NumPy-Discussion at python.org
>>>>> https://mail.python.org/mailman/listinfo/numpy-discussion
>>>>>
>>>> _______________________________________________
>>>> NumPy-Discussion mailing list
>>>> NumPy-Discussion at python.org
>>>> https://mail.python.org/mailman/listinfo/numpy-discussion
>>>>
>>> _______________________________________________
>>> NumPy-Discussion mailing list
>>> NumPy-Discussion at python.org
>>> https://mail.python.org/mailman/listinfo/numpy-discussion
>>>
>> _______________________________________________
>> NumPy-Discussion mailing list
>> NumPy-Discussion at python.org
>> https://mail.python.org/mailman/listinfo/numpy-discussion
>>
> _______________________________________________
> NumPy-Discussion mailing list
> NumPy-Discussion at python.org
> https://mail.python.org/mailman/listinfo/numpy-discussion
>
?
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20181106/d4cf4dde/attachment-0001.html>

From f.s.farimani at gmail.com  Wed Nov  7 03:24:30 2018
From: f.s.farimani at gmail.com (Foad Sojoodi Farimani)
Date: Wed, 7 Nov 2018 09:24:30 +0100
Subject: [Numpy-discussion] numpy pprint?
In-Reply-To: <CAL1kJvAJ4aFVP+=AinEMEyE-pFB5EzOnV9M=q5yzwmV3Vju=ow@mail.gmail.com>
References: <CADjqfdp7VO_3KfwJTC-_mVLVnPyxbzZuY6+=hAzmvooe8NE+sg@mail.gmail.com>
 <CAC=AwPy8qnC6NNH0NTmkzp7zGr+KaZ6Mcp=0sC-FhaEUX2hgXw@mail.gmail.com>
 <CAL1kJvDbUZzu1Ky6d2wK=b-mDh2FLkLjoK5hYp+H3Jko9ScCYw@mail.gmail.com>
 <CADjqfdoY9Bjy-7-hvj4mDUOm5PnY_P3BCz4YaYNL0Qw_o75c7w@mail.gmail.com>
 <CAL1kJvBo3cZ2XY1kni3addVDG3bERv93=FH1qycM58efVX8xug@mail.gmail.com>
 <CADjqfdqZP2EN=k-Ww1+E0DY462BT3av231+Hpf9Tz7iOtLvgkA@mail.gmail.com>
 <CAL1kJvAJ4aFVP+=AinEMEyE-pFB5EzOnV9M=q5yzwmV3Vju=ow@mail.gmail.com>
Message-ID: <CADjqfdqrXo0BZzCjDqBj=PE3F4fJ-JrVMTKDSWt+QnpGQe-V7w@mail.gmail.com>

Dear Eric,

In between your lines:

On Wed, Nov 7, 2018 at 7:23 AM Eric Wieser <wieser.eric+numpy at gmail.com>
wrote:

> Foad:
>
> having the functionality for conventional consols would also help
>
> I think the most important thing in a conventional console is to output
> the array in a format that allows you to reconstruct the object. That makes
> it way easier for people to reproduce each others problems without having
> their full dataset. If your goal is to visualize complex arrays, I think
> the console is a pretty limited tool, and numpy already does as much as is
> worthwhile there.
>
I agree with most of what you say:

   1. the current representation of numpy array with print() is of course
   already there and it is not my goal to replace it. but rather add something
   like Sympy's pprint function, an alternative representation.
   2. the reason I'm using console is first because there are people who
   use it and secondly because I have no idea how to do what you are doing :))
   there is room for both I think

I don?t think putting everything in boxes is helping. it is confusing. I
> would rather having horizontal and vertical square brackets represent each
> nested array
>
> See my update at the same link, which shows an alternative which draws
> those brackets as you envi
>
wow this is awesome:


[image: 2018-11-07_09-07-37.gif]

I wonder if you could make this the default view without the need of those
other inputs or the lamda function? this is almost 80% of what I had in mind



> it would be awesome if in IPython/Jupyter hovering over an element a popup
> would show the index
>
> It? already does?
>
it doesn't on my browser :(

> to show L R U P or combination of these plus some numbers
>
> I don?t know what you mean by this.
>
imaging the Pandas .head() and .tail functions which accept positive
integer inputs to show specific number of rows. now our print function
could have  two inputs one string which should be L for left, R for right,
U for up and D for down. respectively UL, UR, DL and DR for corners.
another input is a tuple of integers which in the case of U,D,L,R is only
one integer showing the number of rows or columns. and in the case of UL,
UR, DL, DR two integers showing the number of rows and columns in that
specific corner to be shown.

What could be added:

   1. adding slide bars for big datasets
   2. compressing the result according to the terminals dimensions (as
   Pandas does)
   3. editing the variables like Spyders variable explorer
   4. adding dimensions or rows/columns within a dimension or elements in
   rows/columns

Again thanks a lot for your help. I appreciate your kind support.

Best,
Foad

Eric
>
> On Tue, 6 Nov 2018 at 00:56 Foad Sojoodi Farimani f.s.farimani at gmail.com
> <http://mailto:f.s.farimani at gmail.com> wrote:
>
> Wow, this is awesome.
>> Some points though:
>>
>>    - not everybody uses IPython/Jupyter having the functionality for
>>    conventional consols would also help. something like
>>    Sypy's init_printing/init_session which smartly chooses the right
>>    representation considering the terminal.
>>    - I don't think putting everything in boxes is helping. it is
>>    confusing. I would rather having horizontal and vertical square brackets
>>    represent each nested array
>>    - it would be awesome if in IPython/Jupyter hovering over an element
>>    a popup would show the index
>>    - one could read the width and height of the terminal and other
>>    options I mentioned in reply Mark to show L R U P or combination of these
>>    plus some numbers (similar to Pandas .head .tail) methods and then show the
>>    rest by unicod 3dot
>>
>> P.S. I had no idea our university Microsoft services also offers Azure
>> Notebooks awesome :P
>>
>> F.
>>
>> On Tue, Nov 6, 2018 at 9:45 AM Eric Wieser <wieser.eric+numpy at gmail.com>
>> wrote:
>>
>>> Here's how that could look
>>>
>>>
>>> https://numpyintegration-ericwieser.notebooks.azure.com/j/notebooks/pprint.ipynb
>>>
>>> Feel free to play around and see if you can produce something more useful
>>>
>>>
>>>
>>> On Mon, 5 Nov 2018 at 23:28 Foad Sojoodi Farimani <
>>> f.s.farimani at gmail.com> wrote:
>>>
>>>> It is not highking if I asked for it :))
>>>> for IPython/Jupyter using Markdown/LaTeX would be awesome
>>>> or even better using HTML to add sliders just like Pandas...
>>>>
>>>> F.
>>>>
>>>> On Tue, Nov 6, 2018 at 6:51 AM Eric Wieser <wieser.eric+numpy at gmail.com>
>>>> wrote:
>>>>
>>>>> Hijacking this thread while on the topic of pprint - we might want to
>>>>> look into a table-based `_html_repr_` or `_latex_repr_` for use in ipython
>>>>> - where we can print the full array and let scrollbars replace ellipses.
>>>>>
>>>>> Eric
>>>>>
>>>>> On Mon, 5 Nov 2018 at 21:11 Mark Harfouche <mark.harfouche at gmail.com>
>>>>> wrote:
>>>>>
>>>>>> Foad,
>>>>>>
>>>>>> Visualizing data is definitely a complex field. I definitely feel
>>>>>> your pain.
>>>>>> Printing your data is but one way of visualizing it, and probably
>>>>>> only useful for very small and constrained datasets.
>>>>>> Have you looked into set_printoptions
>>>>>> <https://docs.scipy.org/doc/numpy-1.15.1/reference/generated/numpy.set_printoptions.html>
>>>>>> to see how numpy?s existing capabilities might help you with your
>>>>>> visualization?
>>>>>>
>>>>>> The code you showed seems quite good. I wouldn?t worry about
>>>>>> performance when it comes to functions that will seldom be called in tight
>>>>>> loops.
>>>>>> As you?ll learn more about python and numpy, you?ll keep expanding it
>>>>>> to include more use cases.
>>>>>> For many of my projects, I create small submodules for visualization
>>>>>> tailored to the specific needs of the particular project.
>>>>>> I?ll try to incorporate your functions and see how I use them.
>>>>>>
>>>>>> Your original post seems to have some confusion about C Style vs F
>>>>>> Style ordering. I hope that has been resolved.
>>>>>> There is also a lot of good documentation
>>>>>>
>>>>>> https://docs.scipy.org/doc/numpy/user/numpy-for-matlab-users.html#numpy-for-matlab-users-notes
>>>>>> about transitioning from matlab.
>>>>>>
>>>>>> Mark
>>>>>>
>>>>>> On Mon, Nov 5, 2018 at 4:46 PM Foad Sojoodi Farimani <
>>>>>> f.s.farimani at gmail.com> wrote:
>>>>>>
>>>>>>> Hello everyone,
>>>>>>>
>>>>>>> Following this question
>>>>>>> <https://stackoverflow.com/q/53126305/4999991>, I'm convinced that
>>>>>>> numpy ndarrays are not MATLAB/mathematical multidimentional matrices and I
>>>>>>> should stop expecting them to be. However I still think it would have a lot
>>>>>>> of benefit to have a function like sympy's pprint to pretty print.
>>>>>>> something like pandas .head and .tail method plus  .left .right .UpLeft
>>>>>>> .UpRight .DownLeft .DownRight methods. when nothing mentioned it would show
>>>>>>> 4 corners and put dots in the middle if the array is to big for the
>>>>>>> terminal.
>>>>>>>
>>>>>>> Best,
>>>>>>> Foad
>>>>>>> _______________________________________________
>>>>>>> NumPy-Discussion mailing list
>>>>>>> NumPy-Discussion at python.org
>>>>>>> https://mail.python.org/mailman/listinfo/numpy-discussion
>>>>>>>
>>>>>> _______________________________________________
>>>>>> NumPy-Discussion mailing list
>>>>>> NumPy-Discussion at python.org
>>>>>> https://mail.python.org/mailman/listinfo/numpy-discussion
>>>>>>
>>>>> _______________________________________________
>>>>> NumPy-Discussion mailing list
>>>>> NumPy-Discussion at python.org
>>>>> https://mail.python.org/mailman/listinfo/numpy-discussion
>>>>>
>>>> _______________________________________________
>>>> NumPy-Discussion mailing list
>>>> NumPy-Discussion at python.org
>>>> https://mail.python.org/mailman/listinfo/numpy-discussion
>>>>
>>> _______________________________________________
>>> NumPy-Discussion mailing list
>>> NumPy-Discussion at python.org
>>> https://mail.python.org/mailman/listinfo/numpy-discussion
>>>
>> _______________________________________________
>> NumPy-Discussion mailing list
>> NumPy-Discussion at python.org
>> https://mail.python.org/mailman/listinfo/numpy-discussion
>>
> _______________________________________________
> NumPy-Discussion mailing list
> NumPy-Discussion at python.org
> https://mail.python.org/mailman/listinfo/numpy-discussion
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20181107/da7720f6/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: 2018-11-07_09-07-37.gif
Type: image/gif
Size: 37249 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20181107/da7720f6/attachment-0001.gif>

From f.s.farimani at gmail.com  Wed Nov  7 04:16:14 2018
From: f.s.farimani at gmail.com (Foad Sojoodi Farimani)
Date: Wed, 7 Nov 2018 10:16:14 +0100
Subject: [Numpy-discussion] numpy pprint?
In-Reply-To: <CADjqfdqrXo0BZzCjDqBj=PE3F4fJ-JrVMTKDSWt+QnpGQe-V7w@mail.gmail.com>
References: <CADjqfdp7VO_3KfwJTC-_mVLVnPyxbzZuY6+=hAzmvooe8NE+sg@mail.gmail.com>
 <CAC=AwPy8qnC6NNH0NTmkzp7zGr+KaZ6Mcp=0sC-FhaEUX2hgXw@mail.gmail.com>
 <CAL1kJvDbUZzu1Ky6d2wK=b-mDh2FLkLjoK5hYp+H3Jko9ScCYw@mail.gmail.com>
 <CADjqfdoY9Bjy-7-hvj4mDUOm5PnY_P3BCz4YaYNL0Qw_o75c7w@mail.gmail.com>
 <CAL1kJvBo3cZ2XY1kni3addVDG3bERv93=FH1qycM58efVX8xug@mail.gmail.com>
 <CADjqfdqZP2EN=k-Ww1+E0DY462BT3av231+Hpf9Tz7iOtLvgkA@mail.gmail.com>
 <CAL1kJvAJ4aFVP+=AinEMEyE-pFB5EzOnV9M=q5yzwmV3Vju=ow@mail.gmail.com>
 <CADjqfdqrXo0BZzCjDqBj=PE3F4fJ-JrVMTKDSWt+QnpGQe-V7w@mail.gmail.com>
Message-ID: <CADjqfdptRT1ni=KBE9R7b7-eDx3-b7m4H+g=sLyR+WRPHwz+CA@mail.gmail.com>

Der All,

Here in this NoteBook
<https://numpypprint-fsfarimani.notebooks.azure.com/j/notebooks/pprint.ipynb>
I tried to compare my unicode implementation with Eric's HTML version plus
links to the discussions on different forums if you like to follow

For example:

[image: chrome_2018-11-07_10-15-20.png]

Best regards,
Foad


On Wed, Nov 7, 2018 at 9:24 AM Foad Sojoodi Farimani <f.s.farimani at gmail.com>
wrote:

> Dear Eric,
>
> In between your lines:
>
> On Wed, Nov 7, 2018 at 7:23 AM Eric Wieser <wieser.eric+numpy at gmail.com>
> wrote:
>
>> Foad:
>>
>> having the functionality for conventional consols would also help
>>
>> I think the most important thing in a conventional console is to output
>> the array in a format that allows you to reconstruct the object. That makes
>> it way easier for people to reproduce each others problems without having
>> their full dataset. If your goal is to visualize complex arrays, I think
>> the console is a pretty limited tool, and numpy already does as much as is
>> worthwhile there.
>>
> I agree with most of what you say:
>
>    1. the current representation of numpy array with print() is of course
>    already there and it is not my goal to replace it. but rather add something
>    like Sympy's pprint function, an alternative representation.
>    2. the reason I'm using console is first because there are people who
>    use it and secondly because I have no idea how to do what you are doing :))
>    there is room for both I think
>
> I don?t think putting everything in boxes is helping. it is confusing. I
>> would rather having horizontal and vertical square brackets represent each
>> nested array
>>
>> See my update at the same link, which shows an alternative which draws
>> those brackets as you envi
>>
> wow this is awesome:
>
>
> [image: 2018-11-07_09-07-37.gif]
>
> I wonder if you could make this the default view without the need of those
> other inputs or the lamda function? this is almost 80% of what I had in mind
>
>
>
>> it would be awesome if in IPython/Jupyter hovering over an element a
>> popup would show the index
>>
>> It? already does?
>>
> it doesn't on my browser :(
>
>> to show L R U P or combination of these plus some numbers
>>
>> I don?t know what you mean by this.
>>
> imaging the Pandas .head() and .tail functions which accept positive
> integer inputs to show specific number of rows. now our print function
> could have  two inputs one string which should be L for left, R for right,
> U for up and D for down. respectively UL, UR, DL and DR for corners.
> another input is a tuple of integers which in the case of U,D,L,R is only
> one integer showing the number of rows or columns. and in the case of UL,
> UR, DL, DR two integers showing the number of rows and columns in that
> specific corner to be shown.
>
> What could be added:
>
>    1. adding slide bars for big datasets
>    2. compressing the result according to the terminals dimensions (as
>    Pandas does)
>    3. editing the variables like Spyders variable explorer
>    4. adding dimensions or rows/columns within a dimension or elements in
>    rows/columns
>
> Again thanks a lot for your help. I appreciate your kind support.
>
> Best,
> Foad
>
> Eric
>>
>> On Tue, 6 Nov 2018 at 00:56 Foad Sojoodi Farimani f.s.farimani at gmail.com
>> <http://mailto:f.s.farimani at gmail.com> wrote:
>>
>> Wow, this is awesome.
>>> Some points though:
>>>
>>>    - not everybody uses IPython/Jupyter having the functionality for
>>>    conventional consols would also help. something like
>>>    Sypy's init_printing/init_session which smartly chooses the right
>>>    representation considering the terminal.
>>>    - I don't think putting everything in boxes is helping. it is
>>>    confusing. I would rather having horizontal and vertical square brackets
>>>    represent each nested array
>>>    - it would be awesome if in IPython/Jupyter hovering over an element
>>>    a popup would show the index
>>>    - one could read the width and height of the terminal and other
>>>    options I mentioned in reply Mark to show L R U P or combination of these
>>>    plus some numbers (similar to Pandas .head .tail) methods and then show the
>>>    rest by unicod 3dot
>>>
>>> P.S. I had no idea our university Microsoft services also offers Azure
>>> Notebooks awesome :P
>>>
>>> F.
>>>
>>> On Tue, Nov 6, 2018 at 9:45 AM Eric Wieser <wieser.eric+numpy at gmail.com>
>>> wrote:
>>>
>>>> Here's how that could look
>>>>
>>>>
>>>> https://numpyintegration-ericwieser.notebooks.azure.com/j/notebooks/pprint.ipynb
>>>>
>>>> Feel free to play around and see if you can produce something more
>>>> useful
>>>>
>>>>
>>>>
>>>> On Mon, 5 Nov 2018 at 23:28 Foad Sojoodi Farimani <
>>>> f.s.farimani at gmail.com> wrote:
>>>>
>>>>> It is not highking if I asked for it :))
>>>>> for IPython/Jupyter using Markdown/LaTeX would be awesome
>>>>> or even better using HTML to add sliders just like Pandas...
>>>>>
>>>>> F.
>>>>>
>>>>> On Tue, Nov 6, 2018 at 6:51 AM Eric Wieser <
>>>>> wieser.eric+numpy at gmail.com> wrote:
>>>>>
>>>>>> Hijacking this thread while on the topic of pprint - we might want to
>>>>>> look into a table-based `_html_repr_` or `_latex_repr_` for use in ipython
>>>>>> - where we can print the full array and let scrollbars replace ellipses.
>>>>>>
>>>>>> Eric
>>>>>>
>>>>>> On Mon, 5 Nov 2018 at 21:11 Mark Harfouche <mark.harfouche at gmail.com>
>>>>>> wrote:
>>>>>>
>>>>>>> Foad,
>>>>>>>
>>>>>>> Visualizing data is definitely a complex field. I definitely feel
>>>>>>> your pain.
>>>>>>> Printing your data is but one way of visualizing it, and probably
>>>>>>> only useful for very small and constrained datasets.
>>>>>>> Have you looked into set_printoptions
>>>>>>> <https://docs.scipy.org/doc/numpy-1.15.1/reference/generated/numpy.set_printoptions.html>
>>>>>>> to see how numpy?s existing capabilities might help you with your
>>>>>>> visualization?
>>>>>>>
>>>>>>> The code you showed seems quite good. I wouldn?t worry about
>>>>>>> performance when it comes to functions that will seldom be called in tight
>>>>>>> loops.
>>>>>>> As you?ll learn more about python and numpy, you?ll keep expanding
>>>>>>> it to include more use cases.
>>>>>>> For many of my projects, I create small submodules for visualization
>>>>>>> tailored to the specific needs of the particular project.
>>>>>>> I?ll try to incorporate your functions and see how I use them.
>>>>>>>
>>>>>>> Your original post seems to have some confusion about C Style vs F
>>>>>>> Style ordering. I hope that has been resolved.
>>>>>>> There is also a lot of good documentation
>>>>>>>
>>>>>>> https://docs.scipy.org/doc/numpy/user/numpy-for-matlab-users.html#numpy-for-matlab-users-notes
>>>>>>> about transitioning from matlab.
>>>>>>>
>>>>>>> Mark
>>>>>>>
>>>>>>> On Mon, Nov 5, 2018 at 4:46 PM Foad Sojoodi Farimani <
>>>>>>> f.s.farimani at gmail.com> wrote:
>>>>>>>
>>>>>>>> Hello everyone,
>>>>>>>>
>>>>>>>> Following this question
>>>>>>>> <https://stackoverflow.com/q/53126305/4999991>, I'm convinced that
>>>>>>>> numpy ndarrays are not MATLAB/mathematical multidimentional matrices and I
>>>>>>>> should stop expecting them to be. However I still think it would have a lot
>>>>>>>> of benefit to have a function like sympy's pprint to pretty print.
>>>>>>>> something like pandas .head and .tail method plus  .left .right .UpLeft
>>>>>>>> .UpRight .DownLeft .DownRight methods. when nothing mentioned it would show
>>>>>>>> 4 corners and put dots in the middle if the array is to big for the
>>>>>>>> terminal.
>>>>>>>>
>>>>>>>> Best,
>>>>>>>> Foad
>>>>>>>> _______________________________________________
>>>>>>>> NumPy-Discussion mailing list
>>>>>>>> NumPy-Discussion at python.org
>>>>>>>> https://mail.python.org/mailman/listinfo/numpy-discussion
>>>>>>>>
>>>>>>> _______________________________________________
>>>>>>> NumPy-Discussion mailing list
>>>>>>> NumPy-Discussion at python.org
>>>>>>> https://mail.python.org/mailman/listinfo/numpy-discussion
>>>>>>>
>>>>>> _______________________________________________
>>>>>> NumPy-Discussion mailing list
>>>>>> NumPy-Discussion at python.org
>>>>>> https://mail.python.org/mailman/listinfo/numpy-discussion
>>>>>>
>>>>> _______________________________________________
>>>>> NumPy-Discussion mailing list
>>>>> NumPy-Discussion at python.org
>>>>> https://mail.python.org/mailman/listinfo/numpy-discussion
>>>>>
>>>> _______________________________________________
>>>> NumPy-Discussion mailing list
>>>> NumPy-Discussion at python.org
>>>> https://mail.python.org/mailman/listinfo/numpy-discussion
>>>>
>>> _______________________________________________
>>> NumPy-Discussion mailing list
>>> NumPy-Discussion at python.org
>>> https://mail.python.org/mailman/listinfo/numpy-discussion
>>>
>> _______________________________________________
>> NumPy-Discussion mailing list
>> NumPy-Discussion at python.org
>> https://mail.python.org/mailman/listinfo/numpy-discussion
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20181107/46a651d0/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: chrome_2018-11-07_10-15-20.png
Type: image/png
Size: 28101 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20181107/46a651d0/attachment-0001.png>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: 2018-11-07_09-07-37.gif
Type: image/gif
Size: 37249 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20181107/46a651d0/attachment-0001.gif>

From einstein.edison at gmail.com  Fri Nov  9 10:15:31 2018
From: einstein.edison at gmail.com (Hameer Abbasi)
Date: Fri, 9 Nov 2018 16:15:31 +0100
Subject: [Numpy-discussion] asarray/anyarray; matrix/subclass
Message-ID: <48ba8375-8499-4d57-acf0-803d02d120ce@Canary>


> Begin forwarded message:
>
> From: Stephan Hoyer
> Date: Friday, Nov 09, 2018 at 3:19 PM
> To: Hameer Abbasi
> Cc: Stefan van der Walt , Marten van Kerkwijk
> Subject: asarray/anyarray; matrix/subclass
>
> This is a great discussion, but let's try to have it in public (e.g., on the NumPy mailing list).
> On Fri, Nov 9, 2018 at 8:42 AM Hameer Abbasi <einstein.edison at gmail.com (mailto:einstein.edison at gmail.com)> wrote:
> > Hi Stephan,
> >
> > The issue I have with writing another function is that asarray/asanyarray are so widely used that it?d be a huge maintenance task to update them throughout NumPy, not to mention other codebases, not to mention other codebases having to rely on newer NumPy versions for this. In short, it would dramatically reduce adaptability of this function.
> >
> > One path we can take is to allow asarray/asanyarray to be overridable via __array_function__ (the former is debatable). This solves most of our duck-array related issues without introducing another protocol.
> >
> > Regardless of what path we choose, I would recommend changing asanyarray to not pass through np.matrix regardless, instead passing through mat.view(type=np.ndarray) instead, which has O(1) cost and memory. In the vast majority of contexts, it?s used to ensure an array-ish structure for another operation, and usually there?s no guarantee that what comes out will be a matrix anyway. I suggest we raise a FutureWarning and then change this behaviour.
> >
> > There have been a number of discussions about deprecating np.matrix (and a few about MaskedArray as well, though there are less compelling reasons for that one). I suggest we start down that path as soon as possible. The biggest (only?) user I know of blocking that is scipy.sparse, and we?re on our way to replacing that with PyData/Sparse.
> >
> > Best Regards,
> > Hameer Abbasi
> >
> >
> > > On Friday, Nov 09, 2018 at 1:26 AM, Stephan Hoyer <shoyer at gmail.com (mailto:shoyer at gmail.com)> wrote:
> > > Hi Hameer,
> > >
> > > I'd love to talk about this in more detail. I agree that something like this is needed.
> > >
> > > The challenge with reusing an existing function like asanyarray() is that there is at least one (somewhat?) widely used ndarray subclass that badly violates the Liskov Substitution Principle: np.matrix.
> > >
> > > NumPy can't really use np.asanyarray() widely for internal purposes until we don't have to worry about np matrix. We might special case np.matrix in some way, but then asanyarray() would do totally opposite things on different versions of NumPy. It's almost certainly a better idea to just write a new function with the desired semantics, and "soft deprecate" asanyarray(). The new function can explicitly black list np.matrix, as well as any other subclasses we know of that badly violate LSP.
> > >
> > > Cheers,
> > > Stephan
> > > On Thu, Nov 8, 2018 at 5:06 PM Hameer Abbasi <einstein.edison at gmail.com (mailto:einstein.edison at gmail.com)> wrote:
> > > > No, Stefan, I?ll do that now. Putting you in the cc.
> > > >
> > > > It slipped my mind among the million other things I had in mind ? Namely: My job visa. It was only done this Monday.
> > > >
> > > > Hi, Marten, Stephan:
> > > >
> > > > Stefan wants me to write up a NEP that allows a given object to specify that it is a duck array ? Namely, that it follows duck-array semantics.
> > > >
> > > > We were thinking of switching asanyarray to switch to passing through anything that implements the duck-array protocol along with ndarray subclasses. I?m sure this would help XArray and Quantity work better with existing codebases, along with PyData/Sparse arrays.
> > > >
> > > > Would you be interested?
> > > >
> > > > Best Regards,
> > > > Hameer Abbasi
> > > >
> > > >
> > > > > On Thursday, Nov 08, 2018 at 9:09 PM, Stefan van der Walt <stefanv at berkeley.edu (mailto:stefanv at berkeley.edu)> wrote:
> > > > > Hi Hameer,
> > > > >
> > > > > In last week's meeting, we had the following in the notes:
> > > > >
> > > > > > Hameer is contacting Marten & Stephan and write up a draft NEP for
> > > > > > clarifying the asarray/asanyarray and matrix/subclass path forward.
> > > > >
> > > > > Did any of that happen that you could share?
> > > > >
> > > > > Thanks and best regards,
> > > > > St?fan

Hello, everyone,

Me, Stefan van der Walt, Stephan Hoyer and Marten van Kerkwijk were having a discussion about the state of matrix, asarray and asanyarray. Our thoughts are summarised above (in the quoted text that I?m forwarding)

Basically, this grew out of a discussion relating to asanyarray/asarray inconsistencies in NumPy about which to use where. Historically, asarray was used in many libraries/places instead of asanyarray usually because np.matrix caused problems due to its special behaviour with regard to indexing (it always returns a 2-D object when eliminating one dimension, but a 0-D one when eliminating both), its behaviour regarding __mul__ (the multiplication operator represents matrix multiplication rather than element-wise multiplication) and its fixed dimensionality (matrix is 2D only). Because of these three things, as Stephan accurately pointed out, it violates the Liskov Substitution Principle.

Because of this behaviour, many libraries switched from using asanyarray to asarray, as np.matrix wouldn?t work with their code. This shut out other matrix subclasses from being used as well, such as MaskedArray and astropy.Quantity. Even if asanyarray is used, there is usually no guarantee that a matrix will be returned instead of an array.

The changes I?m proposing are twofold, but simple:
asanyarray should return mat.view(type=np.ndarray) instead of matrices, after an appropriate time with a FutureWarning. This allows us to preserve the performance (Creating a view is O(1) both in memory and time), and the mutability of the original matrix. This change should happen after a FutureWarning and the usual grace period.
In the spirit of allowing duck-arrays to work with existing NumPy code, asanyarray should be overridable via __array_function__, so that duck arrays can decide whether to pass themselves through. If subclasses are allowed, so should ducka-arrays as well.

This is a part of a larger effort to deprecate np.matrix. As far as I?m aware, it has one big customer (scipy.sparse). The effort to replace that is already underway at PyData/Sparse.

Best Regards,
Hameer Abbasi

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20181109/bd6f0ad9/attachment.html>

From dpgrote at lbl.gov  Fri Nov  9 18:00:01 2018
From: dpgrote at lbl.gov (David Grote)
Date: Fri, 9 Nov 2018 15:00:01 -0800
Subject: [Numpy-discussion] Problem with libgfortran installed with pip
 install numpy
In-Reply-To: <CAB6mnxL5xYx7B6ezcVKQt1Tt8tMNHR_H19a1FpZVTtBWYwz6ww@mail.gmail.com>
References: <CA+fvD_Ks_Gx1wqEiHi0bMzD27fJQE29dZUGpzWo5y3_=xE_OaQ@mail.gmail.com>
 <CAB6mnxL5xYx7B6ezcVKQt1Tt8tMNHR_H19a1FpZVTtBWYwz6ww@mail.gmail.com>
Message-ID: <CA+fvD_KCHMOYe3pf_WZwRyG5Z0r5q-ZuM6kEE-qT11MfQJMCkw@mail.gmail.com>

Hi Matthew - Do you have any comment in this?
   Thanks!
        Dave


On Wed, Sep 5, 2018 at 5:01 PM Charles R Harris <charlesr.harris at gmail.com>
wrote:

>
>
> On Wed, Sep 5, 2018 at 5:38 PM David Grote <dpgrote at lbl.gov> wrote:
>
>>
>> Hi - I have recently come across this problem. On my mac, I build a
>> Fortran code, producing a shared object, that I import into Python along
>> with numpy. This had been working fine until recently when I started seeing
>> sag faults deep inside the Fortran code, usually in Fortran print
>> statements. I tracked this down to a gfortran version issue.
>>
>> I use the brew installation of Python and gcc (using the most recent
>> version, 8.2.0). gcc of course installs a version of libgfortran.dylib.
>> Doing a lsof of a running Python, I see that it finds that copy of
>> libgfortran, and also a copy that was downloaded with numpy
>> (/usr/local/lib/python3.7/site-packages/numpy/.dylibs/libgfortran.3.dylib).
>> Looking at numpy's copy of libgfortran, I see that it is version 4.9.0,
>> much older. Since my code is importing numpy first, the OS seems be using
>> numpy's version of libgfortran to link when importing my code. I know from
>> other experience that older versions of libgfortran are not compatible with
>> code compiled using a new version of gfortran and so therefore segfaults
>> happen.
>>
>> If I download the numpy source and do python setup.py install, I don't
>> have this problem.
>>
>> After this long description, my question is why is such an old version of
>> gcc used to build the distribution of numpy that gets installed from pypi?
>> gcc version 4.9.0 is from 2014. Can a newer version be used?
>>
>
> The library came in with the use of OpenBLAS, I don't think there is a
> fundamental reason that a newer version of gfortran couldn't be used, but I
> have little experience with the Mac. Note that we have also given up on 32
> bit Python on Mac for library related reasons. Matthew Brett would be the
> guy to discuss this with.
>
> Chuck
> _______________________________________________
> NumPy-Discussion mailing list
> NumPy-Discussion at python.org
> https://mail.python.org/mailman/listinfo/numpy-discussion
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20181109/d36c401f/attachment.html>

From njs at pobox.com  Fri Nov  9 18:08:40 2018
From: njs at pobox.com (Nathaniel Smith)
Date: Fri, 9 Nov 2018 15:08:40 -0800
Subject: [Numpy-discussion] Problem with libgfortran installed with pip
 install numpy
In-Reply-To: <CA+fvD_Ks_Gx1wqEiHi0bMzD27fJQE29dZUGpzWo5y3_=xE_OaQ@mail.gmail.com>
References: <CA+fvD_Ks_Gx1wqEiHi0bMzD27fJQE29dZUGpzWo5y3_=xE_OaQ@mail.gmail.com>
Message-ID: <CAPJVwB=-bS+S08VUSHa+JdhNY3N7M0u5G92M4BQ-Gomb8MoTfA@mail.gmail.com>

On Wed, Sep 5, 2018 at 4:37 PM, David Grote <dpgrote at lbl.gov> wrote:
>
> Hi - I have recently come across this problem. On my mac, I build a Fortran
> code, producing a shared object, that I import into Python along with numpy.
> This had been working fine until recently when I started seeing sag faults
> deep inside the Fortran code, usually in Fortran print statements. I tracked
> this down to a gfortran version issue.
>
> I use the brew installation of Python and gcc (using the most recent
> version, 8.2.0). gcc of course installs a version of libgfortran.dylib.
> Doing a lsof of a running Python, I see that it finds that copy of
> libgfortran, and also a copy that was downloaded with numpy
> (/usr/local/lib/python3.7/site-packages/numpy/.dylibs/libgfortran.3.dylib).
> Looking at numpy's copy of libgfortran, I see that it is version 4.9.0, much
> older. Since my code is importing numpy first, the OS seems be using numpy's
> version of libgfortran to link when importing my code. I know from other
> experience that older versions of libgfortran are not compatible with code
> compiled using a new version of gfortran and so therefore segfaults happen.

Normally on MacOS, it's fine to have multiple versions of the same
library used at the same time, because the linker looks up symbols
using a (source library, symbol name) pair. (This is called the
"two-level namespace".) So it's strange that these two libgfortrans
would interfere with each other. Does gfortran not use the two-level
namespace when linking fortran code?

-n

-- 
Nathaniel J. Smith -- https://vorpus.org

From shoyer at gmail.com  Fri Nov  9 18:28:09 2018
From: shoyer at gmail.com (Stephan Hoyer)
Date: Fri, 9 Nov 2018 18:28:09 -0500
Subject: [Numpy-discussion] Should unique types of all arguments be
 passed on in __array_function__?
In-Reply-To: <CAJNV+9v-UqP3-ThNf2n9ZBWdEhKS=PVifP-U7cku+DvnOk5gFg@mail.gmail.com>
References: <CAJNV+9vx9AJpJ3ZqTHMt9QbEH5kjsz5JSbJZsPB3DaY__yKgZA@mail.gmail.com>
 <CAEQ_TvfGJVrZd+wWOnDYgOSpkrwQ+7j0-esuXyMXedjwNsOGAA@mail.gmail.com>
 <CAJNV+9v-UqP3-ThNf2n9ZBWdEhKS=PVifP-U7cku+DvnOk5gFg@mail.gmail.com>
Message-ID: <CAEQ_TveHB4hGfmuM6dF509mx70mHzZ75MrXVoQSdCdt7rPwaGA@mail.gmail.com>

On Mon, Nov 5, 2018 at 9:00 AM Marten van Kerkwijk <
m.h.vankerkwijk at gmail.com> wrote:

> Hi Stephan,
>
> I fear my example about thinking about `ndarray.__array_function__`
> distracted from the gist of my question, which was whether for
> `__array_function__` implementations *generally* it wouldn't be handier to
> have all unique types rather than just those that override
> `__array_function__`. It would seem that for any other implementation than
> for numpy itself, the presence of __array_function__ is indeed almost
> irrelevant. As a somewhat random example, why would it, e.g., for DASK be
> useful to know that another argument is a Quantity, but not that it is a
> file handle? (Presumably, it cannot handle either...)
>

In practice, it is of course easy to simply ignore arguments that don?t
define __array_function__. But I do think the distinction is important for
more than merely ndarray: the value of the types argument tells you the set
of types that might have a conflicting implementation.

For example, Dask might be happy to handle any non-arrays as scalars (like
NumPy), e.g., it should be fine to make a dask array consisting of a
decimal object. Since decimal doesn?t define __array_function__, there?s no
need to do anything special to handle it inside
dask.array.Array.__array_function__. If decimal appeared in types, then
dask would have to be careful to let arbitrary types that don?t define
__array_function__ pass through.

In contrast, dask definitely wants to know if another type defines
__array_function__, because they might have a conflicting implementation.
This is the main reason why we have the types argument in the first place ?
to make these checks easy. In my experience, it is super common for Python
arithmetic methods to be implemented improperly, i.e., never returning
NotImplemented. This will hopefully be less common with __array_function__.

More broadly, it is only necessary to reject an argument type at the
__array_function__ level if it defines __array_function__ itself, because
that?s the only case where it would make a difference to return
NotImplemented rather than trying (and failing) to call the overriden
function implementation.



>
> All the best,
>
> Marten
> _______________________________________________
> NumPy-Discussion mailing list
> NumPy-Discussion at python.org
> https://mail.python.org/mailman/listinfo/numpy-discussion
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20181109/de357598/attachment.html>

From shoyer at gmail.com  Fri Nov  9 18:39:13 2018
From: shoyer at gmail.com (Stephan Hoyer)
Date: Fri, 9 Nov 2018 18:39:13 -0500
Subject: [Numpy-discussion] asarray/anyarray; matrix/subclass
In-Reply-To: <48ba8375-8499-4d57-acf0-803d02d120ce@Canary>
References: <48ba8375-8499-4d57-acf0-803d02d120ce@Canary>
Message-ID: <CAEQ_Tvc4uK_p0Fuoknq_-QRxD+oonOAjf4D_HCoAsaDbQk8zJQ@mail.gmail.com>

I?m still not sure I agree with the advantages of reusing asanyarray(),
even if matrix did not exist. Yes, asanyarray will exist in old NumPy
versions, but you can?t use it with sparse arrays anyways because it will
have the wrong semantics. I expect this would be a bug magnet, with
inadvertent loading of sparse arrays into memory if you?re accidentally
using old NumPy.

With regards to the protocol, I would suggest a dedicated method, e.g.,
__asanyarray__ (or something similar based on the final chosen name of the
function). Coercing to arrays is special enough to have its own dedicated
protocol, and it could be useful for libraries like xarray to check for
__asanyarray__ attributes before deciding which coercion mechanism to use.
On Fri, Nov 9, 2018 at 10:17 AM Hameer Abbasi <einstein.edison at gmail.com>
wrote:

> Begin forwarded message:
>
> From: Stephan Hoyer
> Date: Friday, Nov 09, 2018 at 3:19 PM
> To: Hameer Abbasi
> Cc: Stefan van der Walt , Marten van Kerkwijk
> Subject: asarray/anyarray; matrix/subclass
>
> This is a great discussion, but let's try to have it in public (e.g., on
> the NumPy mailing list).
> On Fri, Nov 9, 2018 at 8:42 AM Hameer Abbasi <einstein.edison at gmail.com>
> wrote:
>
>> Hi Stephan,
>>
>> The issue I have with writing another function is that asarray/asanyarray
>> are so widely used that it?d be a huge maintenance task to update them
>> throughout NumPy, not to mention other codebases, not to mention other
>> codebases having to rely on newer NumPy versions for this. In short, it
>> would dramatically reduce adaptability of this function.
>>
>> One path we can take is to allow asarray/asanyarray to be overridable via
>> __array_function__ (the former is debatable). This solves most of our
>> duck-array related issues without introducing another protocol.
>>
>> Regardless of what path we choose, I would recommend changing asanyarray
>> to not pass through np.matrix regardless, instead passing through
>> mat.view(type=np.ndarray) instead, which has O(1) cost and memory. In the
>> vast majority of contexts, it?s used to ensure an array-ish structure for
>> another operation, and usually there?s no guarantee that what comes out
>> will be a matrix anyway. I suggest we raise a FutureWarning and then change
>> this behaviour.
>>
>> There have been a number of discussions about deprecating np.matrix (and
>> a few about MaskedArray as well, though there are less compelling reasons
>> for that one). I suggest we start down that path as soon as possible. The
>> biggest (only?) user I know of blocking that is scipy.sparse, and we?re on
>> our way to replacing that with PyData/Sparse.
>>
>> Best Regards,
>> Hameer Abbasi
>>
>> On Friday, Nov 09, 2018 at 1:26 AM, Stephan Hoyer <shoyer at gmail.com>
>> wrote:
>> Hi Hameer,
>>
>> I'd love to talk about this in more detail. I agree that something like
>> this is needed.
>>
>> The challenge with reusing an existing function like asanyarray() is that
>> there is at least one (somewhat?) widely used ndarray subclass that badly
>> violates the Liskov Substitution Principle: np.matrix.
>>
>> NumPy can't really use np.asanyarray() widely for internal purposes until
>> we don't have to worry about np matrix. We might special case np.matrix in
>> some way, but then asanyarray() would do totally opposite things on
>> different versions of NumPy. It's almost certainly a better idea to just
>> write a new function with the desired semantics, and "soft deprecate"
>> asanyarray(). The new function can explicitly black list np.matrix, as well
>> as any other subclasses we know of that badly violate LSP.
>>
>> Cheers,
>> Stephan
>> On Thu, Nov 8, 2018 at 5:06 PM Hameer Abbasi <einstein.edison at gmail.com>
>> wrote:
>>
>>> No, Stefan, I?ll do that now. Putting you in the cc.
>>>
>>> It slipped my mind among the million other things I had in mind ?
>>> Namely: My job visa. It was only done this Monday.
>>>
>>> Hi, Marten, Stephan:
>>>
>>> Stefan wants me to write up a NEP that allows a given object to specify
>>> that it is a duck array ? Namely, that it follows duck-array semantics.
>>>
>>> We were thinking of switching asanyarray to switch to passing through
>>> anything that implements the duck-array protocol along with ndarray
>>> subclasses. I?m sure this would help XArray and Quantity work better with
>>> existing codebases, along with PyData/Sparse arrays.
>>>
>>> Would you be interested?
>>>
>>> Best Regards,
>>> Hameer Abbasi
>>>
>>> On Thursday, Nov 08, 2018 at 9:09 PM, Stefan van der Walt <
>>> stefanv at berkeley.edu> wrote:
>>> Hi Hameer,
>>>
>>> In last week's meeting, we had the following in the notes:
>>>
>>> Hameer is contacting Marten & Stephan and write up a draft NEP for
>>> clarifying the asarray/asanyarray and matrix/subclass path forward.
>>>
>>>
>>> Did any of that happen that you could share?
>>>
>>> Thanks and best regards,
>>> St?fan
>>>
>>>
> Hello, everyone,
>
> Me, Stefan van der Walt, Stephan Hoyer and Marten van Kerkwijk were having
> a discussion about the state of matrix, asarray and asanyarray. Our
> thoughts are summarised above (in the quoted text that I?m forwarding)
>
> Basically, this grew out of a discussion relating to asanyarray/asarray
> inconsistencies in NumPy about which to use where. Historically, asarray
> was used in many libraries/places instead of asanyarray usually because
> np.matrix caused problems due to its special behaviour with regard to
> indexing (it always returns a 2-D object when eliminating one dimension,
> but a 0-D one when eliminating both), its behaviour regarding __mul__ (the
> multiplication operator represents matrix multiplication rather than
> element-wise multiplication) and its fixed dimensionality (matrix is 2D
> only). Because of these three things, as Stephan accurately pointed out, it
> violates the Liskov Substitution Principle.
>
> Because of this behaviour, many libraries switched from using asanyarray
> to asarray, as np.matrix wouldn?t work with their code. This shut out other
> matrix subclasses from being used as well, such as MaskedArray and
> astropy.Quantity. Even if asanyarray is used, there is usually no guarantee
> that a matrix will be returned instead of an array.
>
> The changes I?m proposing are twofold, but simple:
>
>    - asanyarray should return mat.view(type=np.ndarray) instead of
>    matrices, after an appropriate time with a FutureWarning. This allows us to
>    preserve the performance (Creating a view is O(1) both in memory and time),
>    and the mutability of the original matrix. This change should happen after
>    a FutureWarning and the usual grace period.
>    - In the spirit of allowing duck-arrays to work with existing NumPy
>    code, asanyarray should be overridable via __array_function__, so that duck
>    arrays can decide whether to pass themselves through. If subclasses are
>    allowed, so should ducka-arrays as well.
>
> This is a part of a larger effort to deprecate np.matrix. As far as I?m
> aware, it has one big customer (scipy.sparse). The effort to replace that
> is already underway at PyData/Sparse.
>
> Best Regards,
> Hameer Abbasi
>
> _______________________________________________
> NumPy-Discussion mailing list
> NumPy-Discussion at python.org
> https://mail.python.org/mailman/listinfo/numpy-discussion
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20181109/9accb25d/attachment-0001.html>

From njs at pobox.com  Fri Nov  9 18:45:46 2018
From: njs at pobox.com (Nathaniel Smith)
Date: Fri, 9 Nov 2018 15:45:46 -0800
Subject: [Numpy-discussion] asarray/anyarray; matrix/subclass
In-Reply-To: <48ba8375-8499-4d57-acf0-803d02d120ce@Canary>
References: <48ba8375-8499-4d57-acf0-803d02d120ce@Canary>
Message-ID: <CAPJVwBm2ERxY6MOy7GMHMg9guY1=Rik9oA5xq+RdQP2C+m-+YQ@mail.gmail.com>

But matrix isn't the only problem with asanyarray. np.ma also violates
Liskov. No doubt there are other problematic ndarray subclasses out
there too...

If we were going to try to reuse asanyarray through some deprecation
mechanism, I think we'd need to deprecate allowing asanyarray to
return *any* ndarray subclass, unless they explicitly provided an
__asanyarray__ dunder. But at that point I'm not sure what the point
would be of reusing it.

On Fri, Nov 9, 2018 at 7:15 AM, Hameer Abbasi <einstein.edison at gmail.com> wrote:
> Begin forwarded message:
>
> From: Stephan Hoyer
> Date: Friday, Nov 09, 2018 at 3:19 PM
> To: Hameer Abbasi
> Cc: Stefan van der Walt , Marten van Kerkwijk
> Subject: asarray/anyarray; matrix/subclass
>
> This is a great discussion, but let's try to have it in public (e.g., on the
> NumPy mailing list).
> On Fri, Nov 9, 2018 at 8:42 AM Hameer Abbasi <einstein.edison at gmail.com>
> wrote:
>>
>> Hi Stephan,
>>
>> The issue I have with writing another function is that asarray/asanyarray
>> are so widely used that it?d be a huge maintenance task to update them
>> throughout NumPy, not to mention other codebases, not to mention other
>> codebases having to rely on newer NumPy versions for this. In short, it
>> would dramatically reduce adaptability of this function.
>>
>> One path we can take is to allow asarray/asanyarray to be overridable via
>> __array_function__ (the former is debatable). This solves most of our
>> duck-array related issues without introducing another protocol.
>>
>> Regardless of what path we choose, I would recommend changing asanyarray
>> to not pass through np.matrix regardless, instead passing through
>> mat.view(type=np.ndarray) instead, which has O(1) cost and memory. In the
>> vast majority of contexts, it?s used to ensure an array-ish structure for
>> another operation, and usually there?s no guarantee that what comes out will
>> be a matrix anyway. I suggest we raise a FutureWarning and then change this
>> behaviour.
>>
>> There have been a number of discussions about deprecating np.matrix (and a
>> few about MaskedArray as well, though there are less compelling reasons for
>> that one). I suggest we start down that path as soon as possible. The
>> biggest (only?) user I know of blocking that is scipy.sparse, and we?re on
>> our way to replacing that with PyData/Sparse.
>>
>> Best Regards,
>> Hameer Abbasi
>>
>> On Friday, Nov 09, 2018 at 1:26 AM, Stephan Hoyer <shoyer at gmail.com>
>> wrote:
>> Hi Hameer,
>>
>> I'd love to talk about this in more detail. I agree that something like
>> this is needed.
>>
>> The challenge with reusing an existing function like asanyarray() is that
>> there is at least one (somewhat?) widely used ndarray subclass that badly
>> violates the Liskov Substitution Principle: np.matrix.
>>
>> NumPy can't really use np.asanyarray() widely for internal purposes until
>> we don't have to worry about np matrix. We might special case np.matrix in
>> some way, but then asanyarray() would do totally opposite things on
>> different versions of NumPy. It's almost certainly a better idea to just
>> write a new function with the desired semantics, and "soft deprecate"
>> asanyarray(). The new function can explicitly black list np.matrix, as well
>> as any other subclasses we know of that badly violate LSP.
>>
>> Cheers,
>> Stephan
>> On Thu, Nov 8, 2018 at 5:06 PM Hameer Abbasi <einstein.edison at gmail.com>
>> wrote:
>>>
>>> No, Stefan, I?ll do that now. Putting you in the cc.
>>>
>>> It slipped my mind among the million other things I had in mind ? Namely:
>>> My job visa. It was only done this Monday.
>>>
>>> Hi, Marten, Stephan:
>>>
>>> Stefan wants me to write up a NEP that allows a given object to specify
>>> that it is a duck array ? Namely, that it follows duck-array semantics.
>>>
>>> We were thinking of switching asanyarray to switch to passing through
>>> anything that implements the duck-array protocol along with ndarray
>>> subclasses. I?m sure this would help XArray and Quantity work better with
>>> existing codebases, along with PyData/Sparse arrays.
>>>
>>> Would you be interested?
>>>
>>> Best Regards,
>>> Hameer Abbasi
>>>
>>> On Thursday, Nov 08, 2018 at 9:09 PM, Stefan van der Walt
>>> <stefanv at berkeley.edu> wrote:
>>> Hi Hameer,
>>>
>>> In last week's meeting, we had the following in the notes:
>>>
>>> Hameer is contacting Marten & Stephan and write up a draft NEP for
>>> clarifying the asarray/asanyarray and matrix/subclass path forward.
>>>
>>>
>>> Did any of that happen that you could share?
>>>
>>> Thanks and best regards,
>>> St?fan
>
>
> Hello, everyone,
>
> Me, Stefan van der Walt, Stephan Hoyer and Marten van Kerkwijk were having a
> discussion about the state of matrix, asarray and asanyarray. Our thoughts
> are summarised above (in the quoted text that I?m forwarding)
>
> Basically, this grew out of a discussion relating to asanyarray/asarray
> inconsistencies in NumPy about which to use where. Historically, asarray was
> used in many libraries/places instead of asanyarray usually because
> np.matrix caused problems due to its special behaviour with regard to
> indexing (it always returns a 2-D object when eliminating one dimension, but
> a 0-D one when eliminating both), its behaviour regarding __mul__ (the
> multiplication operator represents matrix multiplication rather than
> element-wise multiplication) and its fixed dimensionality (matrix is 2D
> only). Because of these three things, as Stephan accurately pointed out, it
> violates the Liskov Substitution Principle.
>
> Because of this behaviour, many libraries switched from using asanyarray to
> asarray, as np.matrix wouldn?t work with their code. This shut out other
> matrix subclasses from being used as well, such as MaskedArray and
> astropy.Quantity. Even if asanyarray is used, there is usually no guarantee
> that a matrix will be returned instead of an array.
>
> The changes I?m proposing are twofold, but simple:
>
> asanyarray should return mat.view(type=np.ndarray) instead of matrices,
> after an appropriate time with a FutureWarning. This allows us to preserve
> the performance (Creating a view is O(1) both in memory and time), and the
> mutability of the original matrix. This change should happen after a
> FutureWarning and the usual grace period.
> In the spirit of allowing duck-arrays to work with existing NumPy code,
> asanyarray should be overridable via __array_function__, so that duck arrays
> can decide whether to pass themselves through. If subclasses are allowed, so
> should ducka-arrays as well.
>
> This is a part of a larger effort to deprecate np.matrix. As far as I?m
> aware, it has one big customer (scipy.sparse). The effort to replace that is
> already underway at PyData/Sparse.
>
> Best Regards,
> Hameer Abbasi
>
>
> _______________________________________________
> NumPy-Discussion mailing list
> NumPy-Discussion at python.org
> https://mail.python.org/mailman/listinfo/numpy-discussion
>



-- 
Nathaniel J. Smith -- https://vorpus.org

From shoyer at gmail.com  Fri Nov  9 19:59:32 2018
From: shoyer at gmail.com (Stephan Hoyer)
Date: Fri, 9 Nov 2018 19:59:32 -0500
Subject: [Numpy-discussion] asarray/anyarray; matrix/subclass
In-Reply-To: <CAPJVwBm2ERxY6MOy7GMHMg9guY1=Rik9oA5xq+RdQP2C+m-+YQ@mail.gmail.com>
References: <48ba8375-8499-4d57-acf0-803d02d120ce@Canary>
 <CAPJVwBm2ERxY6MOy7GMHMg9guY1=Rik9oA5xq+RdQP2C+m-+YQ@mail.gmail.com>
Message-ID: <CAEQ_TvdCNuY6tctQzPFhnwNCiGxMXuQFHJ3=w9kRsz_NPKxN7g@mail.gmail.com>

On Fri, Nov 9, 2018 at 6:46 PM Nathaniel Smith <njs at pobox.com> wrote:

> But matrix isn't the only problem with asanyarray. np.ma also violates
> Liskov. No doubt there are other problematic ndarray subclasses out
> there too...
>

Please forgive my ignorance (I don't really use mask arrays), but how
specifically do masked arrays violate Liskov? In most cases shouldn't they
work the same as base numpy arrays, except with operations keeping track of
masks?

I'm sure there are some cases where masked arrays have different semantics
than NumPy arrays, but are any of these intentional?

I would guess that the worst current violation is that there is a risk of
losing mask information in some operations, but implementing
__array_function__ would presumably make it possible to fix most of these.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20181109/b8e9b46c/attachment.html>

From njs at pobox.com  Fri Nov  9 20:09:04 2018
From: njs at pobox.com (Nathaniel Smith)
Date: Fri, 9 Nov 2018 17:09:04 -0800
Subject: [Numpy-discussion] asarray/anyarray; matrix/subclass
In-Reply-To: <CAEQ_TvdCNuY6tctQzPFhnwNCiGxMXuQFHJ3=w9kRsz_NPKxN7g@mail.gmail.com>
References: <48ba8375-8499-4d57-acf0-803d02d120ce@Canary>
 <CAPJVwBm2ERxY6MOy7GMHMg9guY1=Rik9oA5xq+RdQP2C+m-+YQ@mail.gmail.com>
 <CAEQ_TvdCNuY6tctQzPFhnwNCiGxMXuQFHJ3=w9kRsz_NPKxN7g@mail.gmail.com>
Message-ID: <CAPJVwBk1FXgKs6ZwME=x8ciO6LZPa4nxK4Rgb0A5fv2x9_47JA@mail.gmail.com>

On Fri, Nov 9, 2018 at 4:59 PM, Stephan Hoyer <shoyer at gmail.com> wrote:
> On Fri, Nov 9, 2018 at 6:46 PM Nathaniel Smith <njs at pobox.com> wrote:
>>
>> But matrix isn't the only problem with asanyarray. np.ma also violates
>> Liskov. No doubt there are other problematic ndarray subclasses out
>> there too...
>
>
> Please forgive my ignorance (I don't really use mask arrays), but how
> specifically do masked arrays violate Liskov? In most cases shouldn't they
> work the same as base numpy arrays, except with operations keeping track of
> masks?

Since many operations silently skip over masked values, the
computation semantics are different. For example, in a regular array,
sum()/size() == mean(), but with a masked array these are totally
different operations. So if you have code that was written for regular
arrays, but pass in a masked array, there's a solid chance that it
will silently return nonsensical results.

(This is why it's better for NAs to propagate by default.)

-n

-- 
Nathaniel J. Smith -- https://vorpus.org

From matti.picus at gmail.com  Sat Nov 10 11:02:23 2018
From: matti.picus at gmail.com (Matti Picus)
Date: Sat, 10 Nov 2018 08:02:23 -0800
Subject: [Numpy-discussion] asarray/anyarray; matrix/subclass
In-Reply-To: <CAPJVwBk1FXgKs6ZwME=x8ciO6LZPa4nxK4Rgb0A5fv2x9_47JA@mail.gmail.com>
References: <48ba8375-8499-4d57-acf0-803d02d120ce@Canary>
 <CAPJVwBm2ERxY6MOy7GMHMg9guY1=Rik9oA5xq+RdQP2C+m-+YQ@mail.gmail.com>
 <CAEQ_TvdCNuY6tctQzPFhnwNCiGxMXuQFHJ3=w9kRsz_NPKxN7g@mail.gmail.com>
 <CAPJVwBk1FXgKs6ZwME=x8ciO6LZPa4nxK4Rgb0A5fv2x9_47JA@mail.gmail.com>
Message-ID: <ccafec93-41b0-1472-abb7-e0d6caaf350c@gmail.com>

On 9/11/18 5:09 pm, Nathaniel Smith wrote:
> On Fri, Nov 9, 2018 at 4:59 PM, Stephan Hoyer <shoyer at gmail.com> wrote:
>> On Fri, Nov 9, 2018 at 6:46 PM Nathaniel Smith <njs at pobox.com> wrote:
>>> But matrix isn't the only problem with asanyarray. np.ma also violates
>>> Liskov. No doubt there are other problematic ndarray subclasses out
>>> there too...
>>>
>>>
>>> Please forgive my ignorance (I don't really use mask arrays), but how
>>> specifically do masked arrays violate Liskov? In most cases shouldn't they
>>> work the same as base numpy arrays, except with operations keeping track of
>>> masks?
> Since many operations silently skip over masked values, the
> computation semantics are different. For example, in a regular array,
> sum()/size() == mean(), but with a masked array these are totally
> different operations. So if you have code that was written for regular
> arrays, but pass in a masked array, there's a solid chance that it
> will silently return nonsensical results.
>
> (This is why it's better for NAs to propagate by default.)
>
> -n


Echos of the discussions in neps 12, 24, 25, 26. http://www.numpy.org/neps


Matti


From m.h.vankerkwijk at gmail.com  Sat Nov 10 12:49:34 2018
From: m.h.vankerkwijk at gmail.com (Marten van Kerkwijk)
Date: Sat, 10 Nov 2018 12:49:34 -0500
Subject: [Numpy-discussion] asarray/anyarray; matrix/subclass
In-Reply-To: <ccafec93-41b0-1472-abb7-e0d6caaf350c@gmail.com>
References: <48ba8375-8499-4d57-acf0-803d02d120ce@Canary>
 <CAPJVwBm2ERxY6MOy7GMHMg9guY1=Rik9oA5xq+RdQP2C+m-+YQ@mail.gmail.com>
 <CAEQ_TvdCNuY6tctQzPFhnwNCiGxMXuQFHJ3=w9kRsz_NPKxN7g@mail.gmail.com>
 <CAPJVwBk1FXgKs6ZwME=x8ciO6LZPa4nxK4Rgb0A5fv2x9_47JA@mail.gmail.com>
 <ccafec93-41b0-1472-abb7-e0d6caaf350c@gmail.com>
Message-ID: <CAJNV+9vr+MbaVcMTwKC=6Zei2VPQoVnqXpDn6dQDCis7sqm42w@mail.gmail.com>

Hi Hameer,

I do not think we should change `asanyarray` itself to special-case matrix;
rather, we could start converting `asarray` to `asanyarray` and solve the
problems that produces for matrices in `matrix` itself (e.g., by overriding
the relevant function with `__array_function__`).

I think the idea of providing an `__anyarray__` method (in analogy with
`__array__`) might work. Indeed, the default in `ndarray` (and thus all its
subclasses) could be to let it return `self`  and to override it for
`matrix` to return an ndarray view.

All the best,

Marten

p.s. Note that we are already giving PendingDeprecationWarning for matrix;
https://github.com/numpy/numpy/pull/10142.



On Sat, Nov 10, 2018 at 11:02 AM Matti Picus <matti.picus at gmail.com> wrote:

> On 9/11/18 5:09 pm, Nathaniel Smith wrote:
> > On Fri, Nov 9, 2018 at 4:59 PM, Stephan Hoyer <shoyer at gmail.com> wrote:
> >> On Fri, Nov 9, 2018 at 6:46 PM Nathaniel Smith <njs at pobox.com> wrote:
> >>> But matrix isn't the only problem with asanyarray. np.ma also violates
> >>> Liskov. No doubt there are other problematic ndarray subclasses out
> >>> there too...
> >>>
> >>>
> >>> Please forgive my ignorance (I don't really use mask arrays), but how
> >>> specifically do masked arrays violate Liskov? In most cases shouldn't
> they
> >>> work the same as base numpy arrays, except with operations keeping
> track of
> >>> masks?
> > Since many operations silently skip over masked values, the
> > computation semantics are different. For example, in a regular array,
> > sum()/size() == mean(), but with a masked array these are totally
> > different operations. So if you have code that was written for regular
> > arrays, but pass in a masked array, there's a solid chance that it
> > will silently return nonsensical results.
> >
> > (This is why it's better for NAs to propagate by default.)
> >
> > -n
>
>
> Echos of the discussions in neps 12, 24, 25, 26. http://www.numpy.org/neps
>
>
> Matti
>
> _______________________________________________
> NumPy-Discussion mailing list
> NumPy-Discussion at python.org
> https://mail.python.org/mailman/listinfo/numpy-discussion
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20181110/5cde9117/attachment.html>

From m.h.vankerkwijk at gmail.com  Sat Nov 10 12:59:01 2018
From: m.h.vankerkwijk at gmail.com (Marten van Kerkwijk)
Date: Sat, 10 Nov 2018 12:59:01 -0500
Subject: [Numpy-discussion] Should unique types of all arguments be
 passed on in __array_function__?
In-Reply-To: <CAEQ_TveHB4hGfmuM6dF509mx70mHzZ75MrXVoQSdCdt7rPwaGA@mail.gmail.com>
References: <CAJNV+9vx9AJpJ3ZqTHMt9QbEH5kjsz5JSbJZsPB3DaY__yKgZA@mail.gmail.com>
 <CAEQ_TvfGJVrZd+wWOnDYgOSpkrwQ+7j0-esuXyMXedjwNsOGAA@mail.gmail.com>
 <CAJNV+9v-UqP3-ThNf2n9ZBWdEhKS=PVifP-U7cku+DvnOk5gFg@mail.gmail.com>
 <CAEQ_TveHB4hGfmuM6dF509mx70mHzZ75MrXVoQSdCdt7rPwaGA@mail.gmail.com>
Message-ID: <CAJNV+9sHsn+GgHicViB7_ZO=f8ZfJs4bDtF8hdNxQAXmv7tuog@mail.gmail.com>

> More broadly, it is only necessary to reject an argument type at the
> __array_function__ level if it defines __array_function__ itself, because
> that?s the only case where it would make a difference to return
> NotImplemented rather than trying (and failing) to call the overriden
> function implementation.
>

Yes, this makes sense -- these are the only types that could possibly
change the outcome if the class now called fails to produce a result.
Indeed, that reasoning makes it logical that `ndarray` itself is not
present even though it defines `__array_ufunc__` - we know it cannot handle
anything with a `__array_ufunc__` implementation.

Hameer, is Stephan's argument convincing to you too? If so, I'll close the
PR.

-- Marten
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20181110/b890e36d/attachment.html>

From shoyer at gmail.com  Sat Nov 10 15:16:28 2018
From: shoyer at gmail.com (Stephan Hoyer)
Date: Sat, 10 Nov 2018 15:16:28 -0500
Subject: [Numpy-discussion] asarray/anyarray; matrix/subclass
In-Reply-To: <CAJNV+9vr+MbaVcMTwKC=6Zei2VPQoVnqXpDn6dQDCis7sqm42w@mail.gmail.com>
References: <48ba8375-8499-4d57-acf0-803d02d120ce@Canary>
 <CAPJVwBm2ERxY6MOy7GMHMg9guY1=Rik9oA5xq+RdQP2C+m-+YQ@mail.gmail.com>
 <CAEQ_TvdCNuY6tctQzPFhnwNCiGxMXuQFHJ3=w9kRsz_NPKxN7g@mail.gmail.com>
 <CAPJVwBk1FXgKs6ZwME=x8ciO6LZPa4nxK4Rgb0A5fv2x9_47JA@mail.gmail.com>
 <ccafec93-41b0-1472-abb7-e0d6caaf350c@gmail.com>
 <CAJNV+9vr+MbaVcMTwKC=6Zei2VPQoVnqXpDn6dQDCis7sqm42w@mail.gmail.com>
Message-ID: <CAEQ_TvdmswZdvEvVLkwKw5iibCbTZ+pn3HFF-0x9qaDJpkg+uA@mail.gmail.com>

On Sat, Nov 10, 2018 at 9:49 AM Marten van Kerkwijk <
m.h.vankerkwijk at gmail.com> wrote:

> Hi Hameer,
>
> I do not think we should change `asanyarray` itself to special-case
> matrix; rather, we could start converting `asarray` to `asanyarray` and
> solve the problems that produces for matrices in `matrix` itself (e.g., by
> overriding the relevant function with `__array_function__`).
>
> I think the idea of providing an `__anyarray__` method (in analogy with
> `__array__`) might work. Indeed, the default in `ndarray` (and thus all its
> subclasses) could be to let it return `self`  and to override it for
> `matrix` to return an ndarray view.
>

Yes, we certainly would rather implement a matrix.__anyarray__ method (if
we're already doing a new protocol) rather than special case np.matrix
explicitly.

Unfortunately, per Nathaniel's comments about NA skipping behavior, it
seems like we will also need MaskedArray.__anyarray__ to return something
other than itself. In principle, we should probably write new version of
MaskedArray that doesn't deviate from ndarray semantics, but that's a
rather large project (we'd also probably want to stop subclassing ndarray).

Changing the default aggregation behavior for the existing MaskedArray is
also an option but that would be a serious annoyance to users and backwards
compatibility break. If the only way MaskedArray violates Liskov is in
terms of NA skipping aggregations by default, then this might be viable. In
practice, this would require adding an explicit skipna argument so
FutureWarnings could be silenced. The plus side of this option is that it
would make it easier to use np.anyarray() or any new coercion function
throughout the internal NumPy code base.

To summarize, I think these are our options:
1. Change the behavior of np.anyarray() to check for an __anyarray__()
protocol. Change np.matrix.__anyarray__() to return a base numpy array
(this is a minor backwards compatibility break, but probably for the best).
Start issuing a FutureWarning for any MaskedArray operations that violate
Liskov and add a skipna argument that in the future will default to
skipna=False.
2. Introduce a new coercion function, e.g., np.duckarray(). This is the
easiest option because we don't need to cleanup NumPy's existing ndarray
subclasses.

P.S. I'm just glad pandas stopped subclassing ndarray a while ago --
there's no way pandas.Series() could be fixed up to not violate Liskov :).
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20181110/c0f17017/attachment.html>

From wieser.eric+numpy at gmail.com  Sat Nov 10 16:15:16 2018
From: wieser.eric+numpy at gmail.com (Eric Wieser)
Date: Sat, 10 Nov 2018 13:15:16 -0800
Subject: [Numpy-discussion] asarray/anyarray; matrix/subclass
In-Reply-To: <CAEQ_TvdmswZdvEvVLkwKw5iibCbTZ+pn3HFF-0x9qaDJpkg+uA@mail.gmail.com>
References: <48ba8375-8499-4d57-acf0-803d02d120ce@Canary>
 <CAPJVwBm2ERxY6MOy7GMHMg9guY1=Rik9oA5xq+RdQP2C+m-+YQ@mail.gmail.com>
 <CAEQ_TvdCNuY6tctQzPFhnwNCiGxMXuQFHJ3=w9kRsz_NPKxN7g@mail.gmail.com>
 <CAPJVwBk1FXgKs6ZwME=x8ciO6LZPa4nxK4Rgb0A5fv2x9_47JA@mail.gmail.com>
 <ccafec93-41b0-1472-abb7-e0d6caaf350c@gmail.com>
 <CAJNV+9vr+MbaVcMTwKC=6Zei2VPQoVnqXpDn6dQDCis7sqm42w@mail.gmail.com>
 <CAEQ_TvdmswZdvEvVLkwKw5iibCbTZ+pn3HFF-0x9qaDJpkg+uA@mail.gmail.com>
Message-ID: <CAL1kJvDTrO29bnGX1Yd5yuVEwE8CABOf_G_8i3+BfSeKtV4L3g@mail.gmail.com>

> If the only way MaskedArray violates Liskov is in terms of NA skipping
aggregations by default, then this might be viable

One of the ways to fix these liskov substitution problems is just to
introduce more base classes - for instance, if we had an `NDContainer` base
class with only slicing support, then masked arrays would be an exact
liskov substitution, but np.matrix would not.

Eric

On Sat, 10 Nov 2018 at 12:17 Stephan Hoyer <shoyer at gmail.com> wrote:

> On Sat, Nov 10, 2018 at 9:49 AM Marten van Kerkwijk <
> m.h.vankerkwijk at gmail.com> wrote:
>
>> Hi Hameer,
>>
>> I do not think we should change `asanyarray` itself to special-case
>> matrix; rather, we could start converting `asarray` to `asanyarray` and
>> solve the problems that produces for matrices in `matrix` itself (e.g., by
>> overriding the relevant function with `__array_function__`).
>>
>> I think the idea of providing an `__anyarray__` method (in analogy with
>> `__array__`) might work. Indeed, the default in `ndarray` (and thus all its
>> subclasses) could be to let it return `self`  and to override it for
>> `matrix` to return an ndarray view.
>>
>
> Yes, we certainly would rather implement a matrix.__anyarray__ method (if
> we're already doing a new protocol) rather than special case np.matrix
> explicitly.
>
> Unfortunately, per Nathaniel's comments about NA skipping behavior, it
> seems like we will also need MaskedArray.__anyarray__ to return something
> other than itself. In principle, we should probably write new version of
> MaskedArray that doesn't deviate from ndarray semantics, but that's a
> rather large project (we'd also probably want to stop subclassing ndarray).
>
> Changing the default aggregation behavior for the existing MaskedArray is
> also an option but that would be a serious annoyance to users and backwards
> compatibility break. If the only way MaskedArray violates Liskov is in
> terms of NA skipping aggregations by default, then this might be viable. In
> practice, this would require adding an explicit skipna argument so
> FutureWarnings could be silenced. The plus side of this option is that it
> would make it easier to use np.anyarray() or any new coercion function
> throughout the internal NumPy code base.
>
> To summarize, I think these are our options:
> 1. Change the behavior of np.anyarray() to check for an __anyarray__()
> protocol. Change np.matrix.__anyarray__() to return a base numpy array
> (this is a minor backwards compatibility break, but probably for the best).
> Start issuing a FutureWarning for any MaskedArray operations that violate
> Liskov and add a skipna argument that in the future will default to
> skipna=False.
> 2. Introduce a new coercion function, e.g., np.duckarray(). This is the
> easiest option because we don't need to cleanup NumPy's existing ndarray
> subclasses.
>
> P.S. I'm just glad pandas stopped subclassing ndarray a while ago --
> there's no way pandas.Series() could be fixed up to not violate Liskov :).
> _______________________________________________
> NumPy-Discussion mailing list
> NumPy-Discussion at python.org
> https://mail.python.org/mailman/listinfo/numpy-discussion
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20181110/f92d4007/attachment-0001.html>

From shoyer at gmail.com  Sat Nov 10 16:19:15 2018
From: shoyer at gmail.com (Stephan Hoyer)
Date: Sat, 10 Nov 2018 16:19:15 -0500
Subject: [Numpy-discussion] Prep for NumPy 1.16.0 branch
In-Reply-To: <CAEQ_TvdmdP2ZVr1e44MQjM8TF5nTubJkc6p209A86UWQaEZvcg@mail.gmail.com>
References: <CAB6mnxJUty767___L5pMOrAKVE-SamvHLVCqTXz3dDvDHum8sQ@mail.gmail.com>
 <CAJNV+9v+F+U--3H91odm=EU7Y=C1Z2WHhD-w34bmOqhZorF7yg@mail.gmail.com>
 <CAEQ_Tvf-cuKO5gcNfrXoMNPqnVra1g8dExWbCWRP2sUTKDk99Q@mail.gmail.com>
 <20181106230640.eqgvpmt2wcqglsh4@carbo>
 <CAEQ_TvdmdP2ZVr1e44MQjM8TF5nTubJkc6p209A86UWQaEZvcg@mail.gmail.com>
Message-ID: <CAEQ_TvdiuAfZjW813RAVbvZqW+q6KGrKR8ON5nHxHjaKQVmSxA@mail.gmail.com>

On Tue, Nov 6, 2018 at 8:09 PM Stephan Hoyer <shoyer at gmail.com> wrote:

> On Tue, Nov 6, 2018 at 6:08 PM Stefan van der Walt <stefanv at berkeley.edu>
> wrote:
>
>> On Sun, 04 Nov 2018 17:16:12 -0800, Stephan Hoyer wrote:
>> > On Sun, Nov 4, 2018 at 10:32 AM Marten van Kerkwijk <
>> > m.h.vankerkwijk at gmail.com> wrote:
>> >
>> > > For `__array_function__`, there was some discussion in
>> > > https://github.com/numpy/numpy/issues/12225 that for 1.16 we might
>> want
>> > > to follow after all Nathaniel's suggestion of using an environment
>> variable
>> > > or so to opt in (since introspection breaks on python2 with our
>> wrapped
>> > > implementations).  Given also the possibly significant hit in
>> performance,
>> > > this may be the best option.
>> > > All the best,
>> >
>> > I am also leaning towards this right now, depending on how long we plan
>> to
>> > wait for releasing 1.16. It will take us at least a little while to sort
>> > out performance issues for __array_function__, I'd guess at least a few
>> > weeks. Then a blocker still might turn up during the release candidate
>> > process (though I think we've found most of the major bugs / downstream
>> > issues already through tests on NumPy's dev branch).
>>
>> Just to make sure I understand correctly: the suggestion is to use an
>> environment variable to temporarily toggle the feature, but the plan in
>> the long run will be to have it enabled all the time, correct?
>>
>
> Yes, exactly. __array_function__ would be opt-in only for 1.16 but enabled
> by default for 1.17.
>

I've gone ahead and made a pull request making __array_function__ opt-in
only for now:
https://github.com/numpy/numpy/pull/12362
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20181110/cd410379/attachment.html>

From einstein.edison at gmail.com  Sat Nov 10 17:07:10 2018
From: einstein.edison at gmail.com (Hameer Abbasi)
Date: Sat, 10 Nov 2018 23:07:10 +0100
Subject: [Numpy-discussion] Should unique types of all arguments be
 passed on in __array_function__?
In-Reply-To: <CAJNV+9sHsn+GgHicViB7_ZO=f8ZfJs4bDtF8hdNxQAXmv7tuog@mail.gmail.com>
References: <CAJNV+9vx9AJpJ3ZqTHMt9QbEH5kjsz5JSbJZsPB3DaY__yKgZA@mail.gmail.com>
 <CAEQ_TvfGJVrZd+wWOnDYgOSpkrwQ+7j0-esuXyMXedjwNsOGAA@mail.gmail.com>
 <CAJNV+9v-UqP3-ThNf2n9ZBWdEhKS=PVifP-U7cku+DvnOk5gFg@mail.gmail.com>
 <CAEQ_TveHB4hGfmuM6dF509mx70mHzZ75MrXVoQSdCdt7rPwaGA@mail.gmail.com>
 <CAJNV+9sHsn+GgHicViB7_ZO=f8ZfJs4bDtF8hdNxQAXmv7tuog@mail.gmail.com>
Message-ID: <6acc78cc-3c78-4440-b692-8482c350992d@Canary>


> On Saturday, Nov 10, 2018 at 6:59 PM, Marten van Kerkwijk <m.h.vankerkwijk at gmail.com (mailto:m.h.vankerkwijk at gmail.com)> wrote:
>
> > More broadly, it is only necessary to reject an argument type at the __array_function__ level if it defines __array_function__ itself, because that?s the only case where it would make a difference to return NotImplemented rather than trying (and failing) to call the overriden function implementation.
>
> Yes, this makes sense -- these are the only types that could possibly change the outcome if the class now called fails to produce a result. Indeed, that reasoning makes it logical that `ndarray` itself is not present even though it defines `__array_ufunc__` - we know it cannot handle anything with a `__array_ufunc__` implementation.
>
> Hameer, is Stephan's argument convincing to you too? If so, I'll close the PR.
I agree with Stephan here, other than the fact that ndarray should be in the list of types. I can think of many cases in PyData/Sparse where I dont want to allow mixed inputs, but maybe that?s a tangential discussion.

Best Regards,
Hameer Abbasi

>
> -- Marten _______________________________________________
> NumPy-Discussion mailing list
> NumPy-Discussion at python.org
> https://mail.python.org/mailman/listinfo/numpy-discussion
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20181110/a193a61d/attachment.html>

From einstein.edison at gmail.com  Sat Nov 10 17:20:46 2018
From: einstein.edison at gmail.com (Hameer Abbasi)
Date: Sat, 10 Nov 2018 23:20:46 +0100
Subject: [Numpy-discussion] asarray/anyarray; matrix/subclass
In-Reply-To: <CAEQ_TvdmswZdvEvVLkwKw5iibCbTZ+pn3HFF-0x9qaDJpkg+uA@mail.gmail.com>
References: <48ba8375-8499-4d57-acf0-803d02d120ce@Canary>
 <CAPJVwBm2ERxY6MOy7GMHMg9guY1=Rik9oA5xq+RdQP2C+m-+YQ@mail.gmail.com>
 <CAEQ_TvdCNuY6tctQzPFhnwNCiGxMXuQFHJ3=w9kRsz_NPKxN7g@mail.gmail.com>
 <CAPJVwBk1FXgKs6ZwME=x8ciO6LZPa4nxK4Rgb0A5fv2x9_47JA@mail.gmail.com>
 <ccafec93-41b0-1472-abb7-e0d6caaf350c@gmail.com>
 <CAJNV+9vr+MbaVcMTwKC=6Zei2VPQoVnqXpDn6dQDCis7sqm42w@mail.gmail.com>
 <CAEQ_TvdmswZdvEvVLkwKw5iibCbTZ+pn3HFF-0x9qaDJpkg+uA@mail.gmail.com>
Message-ID: <859687cb-c1f0-4e9e-a8ed-b92b35c40969@Canary>


> On Saturday, Nov 10, 2018 at 9:16 PM, Stephan Hoyer <shoyer at gmail.com (mailto:shoyer at gmail.com)> wrote:
> On Sat, Nov 10, 2018 at 9:49 AM Marten van Kerkwijk <m.h.vankerkwijk at gmail.com (mailto:m.h.vankerkwijk at gmail.com)> wrote:
> > Hi Hameer,
> >
> > I do not think we should change `asanyarray` itself to special-case matrix; rather, we could start converting `asarray` to `asanyarray` and solve the problems that produces for matrices in `matrix` itself (e.g., by overriding the relevant function with `__array_function__`).
> >
> > I think the idea of providing an `__anyarray__` method (in analogy with `__array__`) might work. Indeed, the default in `ndarray` (and thus all its subclasses) could be to let it return `self` and to override it for `matrix` to return an ndarray view.
>
> Yes, we certainly would rather implement a matrix.__anyarray__ method (if we're already doing a new protocol) rather than special case np.matrix explicitly.
>
> Unfortunately, per Nathaniel's comments about NA skipping behavior, it seems like we will also need MaskedArray.__anyarray__ to return something other than itself. In principle, we should probably write new version of MaskedArray that doesn't deviate from ndarray semantics, but that's a rather large project (we'd also probably want to stop subclassing ndarray).
>
> Changing the default aggregation behavior for the existing MaskedArray is also an option but that would be a serious annoyance to users and backwards compatibility break. If the only way MaskedArray violates Liskov is in terms of NA skipping aggregations by default, then this might be viable. In practice, this would require adding an explicit skipna argument so FutureWarnings could be silenced. The plus side of this option is that it would make it easier to use np.anyarray() or any new coercion function throughout the internal NumPy code base.
>
> To summarize, I think these are our options:
> 1. Change the behavior of np.anyarray() to check for an __anyarray__() protocol. Change np.matrix.__anyarray__() to return a base numpy array (this is a minor backwards compatibility break, but probably for the best). Start issuing a FutureWarning for any MaskedArray operations that violate Liskov and add a skipna argument that in the future will default to skipna=False.
>
>
>
>
>

> 2. Introduce a new coercion function, e.g., np.duckarray(). This is the easiest option because we don't need to cleanup NumPy's existing ndarray subclasses.
>
>
>
>
>

My vote is still for 1. I don?t have an issue for PyData/Sparse depending on recent-ish NumPy versions ? It?ll need a lot of the recent protocols anyway, although I could be convinced otherwise if major package devs (scikits, SciPy, Dask) were to weigh in and say they?ll jump on it (which seems unlikely given SciPy?s policy to support old NumPy versions).

>
>
> P.S. I'm just glad pandas stopped subclassing ndarray a while ago -- there's no way pandas.Series() could be fixed up to not violate Liskov :). _______________________________________________
> NumPy-Discussion mailing list
> NumPy-Discussion at python.org
> https://mail.python.org/mailman/listinfo/numpy-discussion
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20181110/2e91759a/attachment-0001.html>

From shoyer at gmail.com  Sat Nov 10 17:39:15 2018
From: shoyer at gmail.com (Stephan Hoyer)
Date: Sat, 10 Nov 2018 17:39:15 -0500
Subject: [Numpy-discussion] asarray/anyarray; matrix/subclass
In-Reply-To: <859687cb-c1f0-4e9e-a8ed-b92b35c40969@Canary>
References: <48ba8375-8499-4d57-acf0-803d02d120ce@Canary>
 <CAPJVwBm2ERxY6MOy7GMHMg9guY1=Rik9oA5xq+RdQP2C+m-+YQ@mail.gmail.com>
 <CAEQ_TvdCNuY6tctQzPFhnwNCiGxMXuQFHJ3=w9kRsz_NPKxN7g@mail.gmail.com>
 <CAPJVwBk1FXgKs6ZwME=x8ciO6LZPa4nxK4Rgb0A5fv2x9_47JA@mail.gmail.com>
 <ccafec93-41b0-1472-abb7-e0d6caaf350c@gmail.com>
 <CAJNV+9vr+MbaVcMTwKC=6Zei2VPQoVnqXpDn6dQDCis7sqm42w@mail.gmail.com>
 <CAEQ_TvdmswZdvEvVLkwKw5iibCbTZ+pn3HFF-0x9qaDJpkg+uA@mail.gmail.com>
 <859687cb-c1f0-4e9e-a8ed-b92b35c40969@Canary>
Message-ID: <CAEQ_Tvc1BoPwkUbu1sOENaz0nsMf==tcqfYtnZR+yjcxPWrL7g@mail.gmail.com>

On Sat, Nov 10, 2018 at 2:22 PM Hameer Abbasi <einstein.edison at gmail.com>
wrote:

> To summarize, I think these are our options:
>
> 1. Change the behavior of np.anyarray() to check for an __anyarray__()
> protocol. Change np.matrix.__anyarray__() to return a base numpy array
> (this is a minor backwards compatibility break, but probably for the best).
> Start issuing a FutureWarning for any MaskedArray operations that violate
> Liskov and add a skipna argument that in the future will default to
> skipna=False.
>
> 2. Introduce a new coercion function, e.g., np.duckarray(). This is the
> easiest option because we don't need to cleanup NumPy's existing ndarray
> subclasses.
>
>
> My vote is still for 1. I don?t have an issue for PyData/Sparse depending
> on recent-ish NumPy versions ? It?ll need a lot of the recent protocols
> anyway, although I could be convinced otherwise if major package devs
> (scikits, SciPy, Dask) were to weigh in and say they?ll jump on it (which
> seems unlikely given SciPy?s policy to support old NumPy versions).
>

I agree that option (1) is fine for PyData/sparse. The bigger issue is that
this change should be conditional on making breaking changes (at least
raising FutureWarning for now) to np.ma.MaskedArray.

I don't know how people who currently use MaskedArray would feel about
that. I would love to hear their thoughts.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20181110/203d5ab0/attachment.html>

From m.h.vankerkwijk at gmail.com  Sat Nov 10 17:50:56 2018
From: m.h.vankerkwijk at gmail.com (Marten van Kerkwijk)
Date: Sat, 10 Nov 2018 17:50:56 -0500
Subject: [Numpy-discussion] asarray/anyarray; matrix/subclass
In-Reply-To: <CAEQ_Tvc1BoPwkUbu1sOENaz0nsMf==tcqfYtnZR+yjcxPWrL7g@mail.gmail.com>
References: <48ba8375-8499-4d57-acf0-803d02d120ce@Canary>
 <CAPJVwBm2ERxY6MOy7GMHMg9guY1=Rik9oA5xq+RdQP2C+m-+YQ@mail.gmail.com>
 <CAEQ_TvdCNuY6tctQzPFhnwNCiGxMXuQFHJ3=w9kRsz_NPKxN7g@mail.gmail.com>
 <CAPJVwBk1FXgKs6ZwME=x8ciO6LZPa4nxK4Rgb0A5fv2x9_47JA@mail.gmail.com>
 <ccafec93-41b0-1472-abb7-e0d6caaf350c@gmail.com>
 <CAJNV+9vr+MbaVcMTwKC=6Zei2VPQoVnqXpDn6dQDCis7sqm42w@mail.gmail.com>
 <CAEQ_TvdmswZdvEvVLkwKw5iibCbTZ+pn3HFF-0x9qaDJpkg+uA@mail.gmail.com>
 <859687cb-c1f0-4e9e-a8ed-b92b35c40969@Canary>
 <CAEQ_Tvc1BoPwkUbu1sOENaz0nsMf==tcqfYtnZR+yjcxPWrL7g@mail.gmail.com>
Message-ID: <CAJNV+9tTzeLBM5H+vY7ei-qDk7tEYeyVZskn5NUJq-nx6fiX-A@mail.gmail.com>

On Sat, Nov 10, 2018 at 5:39 PM Stephan Hoyer <shoyer at gmail.com> wrote:

> On Sat, Nov 10, 2018 at 2:22 PM Hameer Abbasi <einstein.edison at gmail.com>
> wrote:
>
>> To summarize, I think these are our options:
>>
>> 1. Change the behavior of np.anyarray() to check for an __anyarray__()
>> protocol. Change np.matrix.__anyarray__() to return a base numpy array
>> (this is a minor backwards compatibility break, but probably for the best).
>> Start issuing a FutureWarning for any MaskedArray operations that violate
>> Liskov and add a skipna argument that in the future will default to
>> skipna=False.
>>
>> 2. Introduce a new coercion function, e.g., np.duckarray(). This is the
>> easiest option because we don't need to cleanup NumPy's existing ndarray
>> subclasses.
>>
>>
>> My vote is still for 1. I don?t have an issue for PyData/Sparse depending
>> on recent-ish NumPy versions ? It?ll need a lot of the recent protocols
>> anyway, although I could be convinced otherwise if major package devs
>> (scikits, SciPy, Dask) were to weigh in and say they?ll jump on it (which
>> seems unlikely given SciPy?s policy to support old NumPy versions).
>>
>
> I agree that option (1) is fine for PyData/sparse. The bigger issue is
> that this change should be conditional on making breaking changes (at least
> raising FutureWarning for now) to np.ma.MaskedArray.
>

Might be good to try before worrying too much - MaskedArray already
overrides *a lot*; it is not at all obvious to me that things wouldn't
"just work" if we bulk-replaced `asarray` with `asanyarray`.  And with
`__array_function__` we now have the option to fix code paths that do not
work immediately.

-- Marten
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20181110/0c41b421/attachment.html>

From shoyer at gmail.com  Sat Nov 10 17:51:39 2018
From: shoyer at gmail.com (Stephan Hoyer)
Date: Sat, 10 Nov 2018 17:51:39 -0500
Subject: [Numpy-discussion] Should unique types of all arguments be
 passed on in __array_function__?
In-Reply-To: <6acc78cc-3c78-4440-b692-8482c350992d@Canary>
References: <CAJNV+9vx9AJpJ3ZqTHMt9QbEH5kjsz5JSbJZsPB3DaY__yKgZA@mail.gmail.com>
 <CAEQ_TvfGJVrZd+wWOnDYgOSpkrwQ+7j0-esuXyMXedjwNsOGAA@mail.gmail.com>
 <CAJNV+9v-UqP3-ThNf2n9ZBWdEhKS=PVifP-U7cku+DvnOk5gFg@mail.gmail.com>
 <CAEQ_TveHB4hGfmuM6dF509mx70mHzZ75MrXVoQSdCdt7rPwaGA@mail.gmail.com>
 <CAJNV+9sHsn+GgHicViB7_ZO=f8ZfJs4bDtF8hdNxQAXmv7tuog@mail.gmail.com>
 <6acc78cc-3c78-4440-b692-8482c350992d@Canary>
Message-ID: <CAEQ_TvdO5oDB=O7roipN8ZnTDYPuQ7DzgwPZ_gGyHu-RLeqfUw@mail.gmail.com>

On Sat, Nov 10, 2018 at 2:08 PM Hameer Abbasi <einstein.edison at gmail.com>
wrote:

> I agree with Stephan here, other than the fact that ndarray should be in
> the list of types. I can think of many cases in PyData/Sparse where I dont
> want to allow mixed inputs, but maybe that?s a tangential discussion.
>

To be clear: ndarray *is* currently preserved in the list of types passed
to __array_function__ (because ndarray.__array_function__ is defined).
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20181110/59c6d0a6/attachment.html>

From einstein.edison at gmail.com  Sat Nov 10 18:16:33 2018
From: einstein.edison at gmail.com (Hameer Abbasi)
Date: Sun, 11 Nov 2018 00:16:33 +0100
Subject: [Numpy-discussion] Should unique types of all arguments be
 passed on in __array_function__?
In-Reply-To: <CAEQ_TvdO5oDB=O7roipN8ZnTDYPuQ7DzgwPZ_gGyHu-RLeqfUw@mail.gmail.com>
References: <CAJNV+9vx9AJpJ3ZqTHMt9QbEH5kjsz5JSbJZsPB3DaY__yKgZA@mail.gmail.com>
 <CAEQ_TvfGJVrZd+wWOnDYgOSpkrwQ+7j0-esuXyMXedjwNsOGAA@mail.gmail.com>
 <CAJNV+9v-UqP3-ThNf2n9ZBWdEhKS=PVifP-U7cku+DvnOk5gFg@mail.gmail.com>
 <CAEQ_TveHB4hGfmuM6dF509mx70mHzZ75MrXVoQSdCdt7rPwaGA@mail.gmail.com>
 <CAJNV+9sHsn+GgHicViB7_ZO=f8ZfJs4bDtF8hdNxQAXmv7tuog@mail.gmail.com>
 <6acc78cc-3c78-4440-b692-8482c350992d@Canary>
 <CAEQ_TvdO5oDB=O7roipN8ZnTDYPuQ7DzgwPZ_gGyHu-RLeqfUw@mail.gmail.com>
Message-ID: <5d1ee0c0-893f-4af9-b325-fe88a0ff94c9@Canary>

In that case, ignore my comment. :)

Best Regards,
Hameer Abbasi

> On Saturday, Nov 10, 2018 at 11:52 PM, Stephan Hoyer <shoyer at gmail.com (mailto:shoyer at gmail.com)> wrote:
> On Sat, Nov 10, 2018 at 2:08 PM Hameer Abbasi <einstein.edison at gmail.com (mailto:einstein.edison at gmail.com)> wrote:
> > I agree with Stephan here, other than the fact that ndarray should be in the list of types. I can think of many cases in PyData/Sparse where I dont want to allow mixed inputs, but maybe that?s a tangential discussion.
> >
> >
>
>
> To be clear: ndarray *is* currently preserved in the list of types passed to __array_function__ (because ndarray.__array_function__ is defined). _______________________________________________
> NumPy-Discussion mailing list
> NumPy-Discussion at python.org
> https://mail.python.org/mailman/listinfo/numpy-discussion
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20181111/f5e8067c/attachment-0001.html>

From charlesr.harris at gmail.com  Sat Nov 10 19:39:10 2018
From: charlesr.harris at gmail.com (Charles R Harris)
Date: Sat, 10 Nov 2018 17:39:10 -0700
Subject: [Numpy-discussion] asarray/anyarray; matrix/subclass
In-Reply-To: <CAL1kJvDTrO29bnGX1Yd5yuVEwE8CABOf_G_8i3+BfSeKtV4L3g@mail.gmail.com>
References: <48ba8375-8499-4d57-acf0-803d02d120ce@Canary>
 <CAPJVwBm2ERxY6MOy7GMHMg9guY1=Rik9oA5xq+RdQP2C+m-+YQ@mail.gmail.com>
 <CAEQ_TvdCNuY6tctQzPFhnwNCiGxMXuQFHJ3=w9kRsz_NPKxN7g@mail.gmail.com>
 <CAPJVwBk1FXgKs6ZwME=x8ciO6LZPa4nxK4Rgb0A5fv2x9_47JA@mail.gmail.com>
 <ccafec93-41b0-1472-abb7-e0d6caaf350c@gmail.com>
 <CAJNV+9vr+MbaVcMTwKC=6Zei2VPQoVnqXpDn6dQDCis7sqm42w@mail.gmail.com>
 <CAEQ_TvdmswZdvEvVLkwKw5iibCbTZ+pn3HFF-0x9qaDJpkg+uA@mail.gmail.com>
 <CAL1kJvDTrO29bnGX1Yd5yuVEwE8CABOf_G_8i3+BfSeKtV4L3g@mail.gmail.com>
Message-ID: <CAB6mnx+HfXA45D50vSGt1U977JRqjoNc6jOQRBhuSFVBQxBp6g@mail.gmail.com>

On Sat, Nov 10, 2018 at 2:15 PM Eric Wieser <wieser.eric+numpy at gmail.com>
wrote:

> > If the only way MaskedArray violates Liskov is in terms of NA skipping
> aggregations by default, then this might be viable
>
> One of the ways to fix these liskov substitution problems is just to
> introduce more base classes - for instance, if we had an `NDContainer` base
> class with only slicing support, then masked arrays would be an exact
> liskov substitution, but np.matrix would not.
>
> Eric
>

I've had the same thought and wouldn't be surprised if others have
considered that possibility. Travis would be a good guy to ask about that.

<snip>

Chuck
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20181110/75fafc44/attachment.html>

From efiring at hawaii.edu  Sun Nov 11 01:44:01 2018
From: efiring at hawaii.edu (Eric Firing)
Date: Sat, 10 Nov 2018 20:44:01 -1000
Subject: [Numpy-discussion] asarray/anyarray; matrix/subclass
In-Reply-To: <CAEQ_Tvc1BoPwkUbu1sOENaz0nsMf==tcqfYtnZR+yjcxPWrL7g@mail.gmail.com>
References: <48ba8375-8499-4d57-acf0-803d02d120ce@Canary>
 <CAPJVwBm2ERxY6MOy7GMHMg9guY1=Rik9oA5xq+RdQP2C+m-+YQ@mail.gmail.com>
 <CAEQ_TvdCNuY6tctQzPFhnwNCiGxMXuQFHJ3=w9kRsz_NPKxN7g@mail.gmail.com>
 <CAPJVwBk1FXgKs6ZwME=x8ciO6LZPa4nxK4Rgb0A5fv2x9_47JA@mail.gmail.com>
 <ccafec93-41b0-1472-abb7-e0d6caaf350c@gmail.com>
 <CAJNV+9vr+MbaVcMTwKC=6Zei2VPQoVnqXpDn6dQDCis7sqm42w@mail.gmail.com>
 <CAEQ_TvdmswZdvEvVLkwKw5iibCbTZ+pn3HFF-0x9qaDJpkg+uA@mail.gmail.com>
 <859687cb-c1f0-4e9e-a8ed-b92b35c40969@Canary>
 <CAEQ_Tvc1BoPwkUbu1sOENaz0nsMf==tcqfYtnZR+yjcxPWrL7g@mail.gmail.com>
Message-ID: <8dff2289-e52e-d906-ee44-a165281102b3@hawaii.edu>

On 2018/11/10 12:39 PM, Stephan Hoyer wrote:
> On Sat, Nov 10, 2018 at 2:22 PM Hameer Abbasi <einstein.edison at gmail.com 
> <mailto:einstein.edison at gmail.com>> wrote:
> 
>         To summarize, I think these are our options:
> 
>         1. Change the behavior of np.anyarray() to check for an
>         __anyarray__() protocol. Change np.matrix.__anyarray__() to
>         return a base numpy array (this is a minor backwards
>         compatibility break, but probably for the best). Start issuing a
>         FutureWarning for any MaskedArray operations that violate Liskov
>         and add a skipna argument that in the future will default to
>         skipna=False.
> 
>         2. Introduce a new coercion function, e.g., np.duckarray(). This
>         is the easiest option because we don't need to cleanup NumPy's
>         existing ndarray subclasses.
> 
> 
>     My vote is still for 1. I don?t have an issue for PyData/Sparse
>     depending on recent-ish NumPy versions ? It?ll need a lot of the
>     recent protocols anyway, although I could be convinced otherwise if
>     major package devs (scikits, SciPy, Dask) were to weigh in and say
>     they?ll jump on it (which seems unlikely given SciPy?s policy to
>     support old NumPy versions).
> 
> 
> I agree that option (1) is fine for PyData/sparse. The bigger issue is 
> that this change should be conditional on making breaking changes (at 
> least raising FutureWarning for now) to np.ma.MaskedArray.
> 
> I don't know how people who currently use MaskedArray would feel about 
> that. I would love to hear their thoughts.

Thank you.  I am a user of masked arrays, and have been since pre-numpy 
days.  I introduced their extensive use in matplotlib long ago.  I have 
been a bit concerned, indeed, that all of the discussion of modifying 
masked arrays seems to be by people who don't actually use them 
explicitly (though they might be using them without knowing it via 
internal operations in matplotlib, or they might be quickly getting rid 
of them after they are yielded by netCDF4.Dataset()).

I think that those of us who do use masked arrays recognize that they 
are not perfect; they have some quirks and gotchas, and one has to be 
careful to use numpy.ma functions instead of numpy functions in most 
cases.  But we use them because they have real advantages over the 
alternatives, which are using nans and/or manually tracking independent 
masks throughout calculations.  These advantages are largely because 
masked values *don't* behave like nan, *don't* propagate.  This is 
fundamental to the design, and motivated by real-life use cases.

The proposal to add a skipna kwarg to MaskedArray looks to me like it is 
giving purity priority over practicality.  It will force ma users to 
insert skipna kwargs all over the place--because the default will be 
contrary to the primary purposes of using masked arrays, in most cases. 
How many people will it actually benefit?  How many people are being 
bitten, and how badly, by masked array behavior?

If there were a prospect of truly integrating missing/masked value 
handling into numpy, simplifying or phasing out numpy.ma, I would be 
delighted--I think it is the biggest single fundamental improvement that 
could be made, from the user's standpoint.  I was sad to see Mark 
Wiebe's work in that direction come to grief.

If there are ways of gradually improving numpy.ma and its 
interoperability with the rest of numpy and with the proliferation of 
duck arrays, I'm all in favor--so long as they don't effectively wreck 
numpy.ma for its present intended purposes.

Eric

> 
> _______________________________________________
> NumPy-Discussion mailing list
> NumPy-Discussion at python.org
> https://mail.python.org/mailman/listinfo/numpy-discussion
> 


From m.h.vankerkwijk at gmail.com  Sun Nov 11 10:47:55 2018
From: m.h.vankerkwijk at gmail.com (Marten van Kerkwijk)
Date: Sun, 11 Nov 2018 10:47:55 -0500
Subject: [Numpy-discussion] asarray/anyarray; matrix/subclass
In-Reply-To: <8dff2289-e52e-d906-ee44-a165281102b3@hawaii.edu>
References: <48ba8375-8499-4d57-acf0-803d02d120ce@Canary>
 <CAPJVwBm2ERxY6MOy7GMHMg9guY1=Rik9oA5xq+RdQP2C+m-+YQ@mail.gmail.com>
 <CAEQ_TvdCNuY6tctQzPFhnwNCiGxMXuQFHJ3=w9kRsz_NPKxN7g@mail.gmail.com>
 <CAPJVwBk1FXgKs6ZwME=x8ciO6LZPa4nxK4Rgb0A5fv2x9_47JA@mail.gmail.com>
 <ccafec93-41b0-1472-abb7-e0d6caaf350c@gmail.com>
 <CAJNV+9vr+MbaVcMTwKC=6Zei2VPQoVnqXpDn6dQDCis7sqm42w@mail.gmail.com>
 <CAEQ_TvdmswZdvEvVLkwKw5iibCbTZ+pn3HFF-0x9qaDJpkg+uA@mail.gmail.com>
 <859687cb-c1f0-4e9e-a8ed-b92b35c40969@Canary>
 <CAEQ_Tvc1BoPwkUbu1sOENaz0nsMf==tcqfYtnZR+yjcxPWrL7g@mail.gmail.com>
 <8dff2289-e52e-d906-ee44-a165281102b3@hawaii.edu>
Message-ID: <CAJNV+9tW78AiF2710UOT2pqfh3=pn+-YtKKc5tDFWrcYz4nz-Q@mail.gmail.com>

Hi Eric,

Thanks very much for the detailed response; it is good to be reminded that
`MaskedArray` is used in a package that, indeed, (nearly?) all of us use!

But I do think that those of us who have been trying to change MaskedArray,
are generally good at making sure the tests continue to pass, i.e., that
the behaviour does not change (the main exception in the last few years was
that views should be taken of masks too, not just the data).

I also think that between __array_ufunc__ and __array_function__, it has
become quite easy to ensure that one no longer has to rely on `np.ma`
functions, i.e., that the regular numpy functions will do the right thing.
But it will need work to actually implement that.

All the best,

Marten
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20181111/3ed03ec3/attachment.html>

From stefanv at berkeley.edu  Sun Nov 11 17:07:48 2018
From: stefanv at berkeley.edu (Stefan van der Walt)
Date: Sun, 11 Nov 2018 14:07:48 -0800
Subject: [Numpy-discussion] Developer Meeting, Berkeley, 30 Nov / 1 Dec
Message-ID: <20181111220748.hjhomk7wmd2mh3k6@carbo>

Hi everyone,

On Friday 30 November & Saturday 1 December we will host a NumPy
Development Meeting at the Berkeley Institute for Data Science.  We will
discuss the work being done on dtypes, review NEPs under implementation,
and solicit feedback for updating the community roadmap.

Please get in touch if you would like to attend, so that we can tally
the numbers and work out travel support.

Best regards,
St?fan

From stefanv at berkeley.edu  Sun Nov 11 17:16:36 2018
From: stefanv at berkeley.edu (Stefan van der Walt)
Date: Sun, 11 Nov 2018 14:16:36 -0800
Subject: [Numpy-discussion] Weekly status meeting 8.11 at 12:00 pacific
 time
In-Reply-To: <814a37a5-2650-5a63-f692-d859bd194dd6@gmail.com>
References: <814a37a5-2650-5a63-f692-d859bd194dd6@gmail.com>
Message-ID: <20181111221636.efqb7nl24shrpzsq@carbo>

On Tue, 06 Nov 2018 16:28:36 -0800, Matti Picus wrote:
> We will be holding our weekly BIDS NumPy status meeting on Thurs Nov 8 at
> noon pacific time.

The meeting notes are now posted at
https://github.com/BIDS-numpy/docs/blob/master/status_meetings/status-2018-11-08.md

St?fan

From shoyer at gmail.com  Sun Nov 11 21:39:47 2018
From: shoyer at gmail.com (Stephan Hoyer)
Date: Sun, 11 Nov 2018 18:39:47 -0800
Subject: [Numpy-discussion] asarray/anyarray; matrix/subclass
In-Reply-To: <8dff2289-e52e-d906-ee44-a165281102b3@hawaii.edu>
References: <48ba8375-8499-4d57-acf0-803d02d120ce@Canary>
 <CAPJVwBm2ERxY6MOy7GMHMg9guY1=Rik9oA5xq+RdQP2C+m-+YQ@mail.gmail.com>
 <CAEQ_TvdCNuY6tctQzPFhnwNCiGxMXuQFHJ3=w9kRsz_NPKxN7g@mail.gmail.com>
 <CAPJVwBk1FXgKs6ZwME=x8ciO6LZPa4nxK4Rgb0A5fv2x9_47JA@mail.gmail.com>
 <ccafec93-41b0-1472-abb7-e0d6caaf350c@gmail.com>
 <CAJNV+9vr+MbaVcMTwKC=6Zei2VPQoVnqXpDn6dQDCis7sqm42w@mail.gmail.com>
 <CAEQ_TvdmswZdvEvVLkwKw5iibCbTZ+pn3HFF-0x9qaDJpkg+uA@mail.gmail.com>
 <859687cb-c1f0-4e9e-a8ed-b92b35c40969@Canary>
 <CAEQ_Tvc1BoPwkUbu1sOENaz0nsMf==tcqfYtnZR+yjcxPWrL7g@mail.gmail.com>
 <8dff2289-e52e-d906-ee44-a165281102b3@hawaii.edu>
Message-ID: <CAEQ_TvfWYgjM45LraLNuivMw+9129TAyvL=ya+XQC=4EmDrXVg@mail.gmail.com>

On Sat, Nov 10, 2018 at 10:45 PM Eric Firing <efiring at hawaii.edu> wrote:

> On 2018/11/10 12:39 PM, Stephan Hoyer wrote:
> > On Sat, Nov 10, 2018 at 2:22 PM Hameer Abbasi <einstein.edison at gmail.com
> > <mailto:einstein.edison at gmail.com>> wrote:
> >
> >         To summarize, I think these are our options:
> >
> >         1. Change the behavior of np.anyarray() to check for an
> >         __anyarray__() protocol. Change np.matrix.__anyarray__() to
> >         return a base numpy array (this is a minor backwards
> >         compatibility break, but probably for the best). Start issuing a
> >         FutureWarning for any MaskedArray operations that violate Liskov
> >         and add a skipna argument that in the future will default to
> >         skipna=False.
> >
> >         2. Introduce a new coercion function, e.g., np.duckarray(). This
> >         is the easiest option because we don't need to cleanup NumPy's
> >         existing ndarray subclasses.
> >
> >
> >     My vote is still for 1. I don?t have an issue for PyData/Sparse
> >     depending on recent-ish NumPy versions ? It?ll need a lot of the
> >     recent protocols anyway, although I could be convinced otherwise if
> >     major package devs (scikits, SciPy, Dask) were to weigh in and say
> >     they?ll jump on it (which seems unlikely given SciPy?s policy to
> >     support old NumPy versions).
> >
> >
> > I agree that option (1) is fine for PyData/sparse. The bigger issue is
> > that this change should be conditional on making breaking changes (at
> > least raising FutureWarning for now) to np.ma.MaskedArray.
> >
> > I don't know how people who currently use MaskedArray would feel about
> > that. I would love to hear their thoughts.
>
> Thank you.  I am a user of masked arrays, and have been since pre-numpy
> days.  I introduced their extensive use in matplotlib long ago.  I have
> been a bit concerned, indeed, that all of the discussion of modifying
> masked arrays seems to be by people who don't actually use them
> explicitly (though they might be using them without knowing it via
> internal operations in matplotlib, or they might be quickly getting rid
> of them after they are yielded by netCDF4.Dataset()).
>
> I think that those of us who do use masked arrays recognize that they
> are not perfect; they have some quirks and gotchas, and one has to be
> careful to use numpy.ma functions instead of numpy functions in most
> cases.  But we use them because they have real advantages over the
> alternatives, which are using nans and/or manually tracking independent
> masks throughout calculations.  These advantages are largely because
> masked values *don't* behave like nan, *don't* propagate.  This is
> fundamental to the design, and motivated by real-life use cases.
>
> The proposal to add a skipna kwarg to MaskedArray looks to me like it is
> giving purity priority over practicality.  It will force ma users to
> insert skipna kwargs all over the place--because the default will be
> contrary to the primary purposes of using masked arrays, in most cases.
> How many people will it actually benefit?  How many people are being
> bitten, and how badly, by masked array behavior?
>
> If there were a prospect of truly integrating missing/masked value
> handling into numpy, simplifying or phasing out numpy.ma, I would be
> delighted--I think it is the biggest single fundamental improvement that
> could be made, from the user's standpoint.  I was sad to see Mark
> Wiebe's work in that direction come to grief.
>
> If there are ways of gradually improving numpy.ma and its
> interoperability with the rest of numpy and with the proliferation of
> duck arrays, I'm all in favor--so long as they don't effectively wreck
> numpy.ma for its present intended purposes.


Eric -- thank you for sharing your perspective! I guess it should not be
surprising that the semantics of MaskedArray intentionally deviate from the
semantics of base NumPy arrays.

This deviation is fortunately less severe than than deviations in the
behavior of np.matrix, but it still presents some difficulties for duck
typing. We're in a position to reduce (but still not eliminate) these
differences with new protocols like __array_function__.

I think Nathaniel actually summarized these issues pretty well in NEP 16 (
http://www.numpy.org/neps/nep-0016-abstract-array.html). If we want a
coercion function that guarantees an object is a "full duck array", then it
can't pass on either np.matrix or MaskedArray in their current state.
Anything less than full compatibility provides a shaky foundation for use
in downstream projects or inside NumPy itself.

In theory (certainly if we were starting from scratch) it would make sense
to make asabstractarray() pass on any ndarray subclass, but this would
require willingness to make breaking changes to both np.matrix and
MaskedArray.

I would suggest adopting a variation of the proposal in NEP 16, except
using a protocol rather an abstract base class per NEP 22, e.g.,

# names still to be determined
def asabstractarray(array, dtype):
    if hasattr(array, '__abstractarray__'):
        return array.__abstractarray__(array, dtype=dtype)
    return asarray(array, dtype)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20181111/c5a66bdc/attachment-0001.html>

From dpgrote at lbl.gov  Mon Nov 12 19:49:40 2018
From: dpgrote at lbl.gov (David Grote)
Date: Mon, 12 Nov 2018 16:49:40 -0800
Subject: [Numpy-discussion] Problem with libgfortran installed with pip
 install numpy
In-Reply-To: <CAPJVwB=-bS+S08VUSHa+JdhNY3N7M0u5G92M4BQ-Gomb8MoTfA@mail.gmail.com>
References: <CA+fvD_Ks_Gx1wqEiHi0bMzD27fJQE29dZUGpzWo5y3_=xE_OaQ@mail.gmail.com>
 <CAPJVwB=-bS+S08VUSHa+JdhNY3N7M0u5G92M4BQ-Gomb8MoTfA@mail.gmail.com>
Message-ID: <CA+fvD_+dVfsn3wXKEs0cUv4x464j2sbOG40cXKKsws43cxkg6w@mail.gmail.com>

Yes, thanks, that's something I hadn't looked into. For legacy reasons, my
code was being built with the option -flat_namespace. I don't remember the
reason why, but many years ago that option was needed for the code to run
on the mac. The code is made up of several shared objects that have
dependencies on each other and apparently there was a problem getting it
all linked together properly without that option. But, I tried it out now
without flat_namespace and it seemed to work fine. Whatever the problem
was, it seems to been fixed some other way. It works Ok having the pip
version of numpy (with its old libgfortran).

I'm still curious about why such an old version of gfortran is still being
used.
   Dave

On Fri, Nov 9, 2018 at 3:09 PM Nathaniel Smith <njs at pobox.com> wrote:

> On Wed, Sep 5, 2018 at 4:37 PM, David Grote <dpgrote at lbl.gov> wrote:
> >
> > Hi - I have recently come across this problem. On my mac, I build a
> Fortran
> > code, producing a shared object, that I import into Python along with
> numpy.
> > This had been working fine until recently when I started seeing sag
> faults
> > deep inside the Fortran code, usually in Fortran print statements. I
> tracked
> > this down to a gfortran version issue.
> >
> > I use the brew installation of Python and gcc (using the most recent
> > version, 8.2.0). gcc of course installs a version of libgfortran.dylib.
> > Doing a lsof of a running Python, I see that it finds that copy of
> > libgfortran, and also a copy that was downloaded with numpy
> >
> (/usr/local/lib/python3.7/site-packages/numpy/.dylibs/libgfortran.3.dylib).
> > Looking at numpy's copy of libgfortran, I see that it is version 4.9.0,
> much
> > older. Since my code is importing numpy first, the OS seems be using
> numpy's
> > version of libgfortran to link when importing my code. I know from other
> > experience that older versions of libgfortran are not compatible with
> code
> > compiled using a new version of gfortran and so therefore segfaults
> happen.
>
> Normally on MacOS, it's fine to have multiple versions of the same
> library used at the same time, because the linker looks up symbols
> using a (source library, symbol name) pair. (This is called the
> "two-level namespace".) So it's strange that these two libgfortrans
> would interfere with each other. Does gfortran not use the two-level
> namespace when linking fortran code?
>
> -n
>
> --
> Nathaniel J. Smith -- https://vorpus.org
> _______________________________________________
> NumPy-Discussion mailing list
> NumPy-Discussion at python.org
> https://mail.python.org/mailman/listinfo/numpy-discussion
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20181112/ebf48bd3/attachment.html>

From matti.picus at gmail.com  Tue Nov 13 20:19:25 2018
From: matti.picus at gmail.com (Matti Picus)
Date: Tue, 13 Nov 2018 17:19:25 -0800
Subject: [Numpy-discussion] Weekly status meeting Nov 14 at 12:00 Pacific
 time
Message-ID: <9f446058-21b1-df1d-b61c-db0289a846b7@gmail.com>

We will be holding our weekly BIDS NumPy status meeting on Wed Nov 14
at noon Pacific time. Please join us.

The draft agenda, along with details of how to join, is up at
https://hackmd.io/H0x6z5uYSgex2FA6p5nlvw?both

Previous sessions' notes are available at
https://github.com/BIDS-numpy/docs/tree/master/status_meetings


Matti, Tyler and Stefan


From shoyer at gmail.com  Wed Nov 14 11:57:21 2018
From: shoyer at gmail.com (Stephan Hoyer)
Date: Wed, 14 Nov 2018 08:57:21 -0800
Subject: [Numpy-discussion] Vectorized version of numpy.linspace
Message-ID: <CAEQ_TveU8MWX=L_7_Fz8sNbT+=aYQC3=FuSsxpGTFcR86Si-Dw@mail.gmail.com>

It recently came up on GitHub (at part of the discussion in
https://github.com/numpy/numpy/issues/12379) that numpy.linspace could, at
least in principle, be modified to support array inputs:

It looks like this has been requested on StackOverflow, too:
https://stackoverflow.com/questions/46694167/vectorized-numpy-linspace-across-multi-dimensional-arrays

My tentative proposal:
- "start" and "stop" are broadcast against each other to form start/stop
arrays. (Or we could require that start/stop have matching shape.)
- A new dimension of size "num" is inserted into the result, either along
the first or last axis.
- A new keyword argument "axis" could control where the axis is inserted in
the result.
- Vectorization support should be added in the same way to geomspace and
logspace.

Does this seem like a good idea? It's a relatively simple generalization,
and one that I, at least, would find useful (I can think of a use-case in
my own code that came up just last week).

I doubt I'll have time to implement this myself in the near future, but I
thought I would get the discussion going -- this might be a good project
for a new contributor to work on.

Cheers,
Stephan
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20181114/d69f2cd5/attachment.html>

From ralf.gommers at gmail.com  Wed Nov 14 12:28:47 2018
From: ralf.gommers at gmail.com (Ralf Gommers)
Date: Wed, 14 Nov 2018 09:28:47 -0800
Subject: [Numpy-discussion] Vectorized version of numpy.linspace
In-Reply-To: <CAEQ_TveU8MWX=L_7_Fz8sNbT+=aYQC3=FuSsxpGTFcR86Si-Dw@mail.gmail.com>
References: <CAEQ_TveU8MWX=L_7_Fz8sNbT+=aYQC3=FuSsxpGTFcR86Si-Dw@mail.gmail.com>
Message-ID: <CABL7CQgek9ib8zz7j7MxNHrop7ZCFsQ5imejDDB1zNv9Wcd3GA@mail.gmail.com>

On Wed, Nov 14, 2018 at 8:57 AM Stephan Hoyer <shoyer at gmail.com> wrote:

> It recently came up on GitHub (at part of the discussion in
> https://github.com/numpy/numpy/issues/12379) that numpy.linspace could,
> at least in principle, be modified to support array inputs:
>
> It looks like this has been requested on StackOverflow, too:
>
> https://stackoverflow.com/questions/46694167/vectorized-numpy-linspace-across-multi-dimensional-arrays
>
> My tentative proposal:
> - "start" and "stop" are broadcast against each other to form start/stop
> arrays. (Or we could require that start/stop have matching shape.)
> - A new dimension of size "num" is inserted into the result, either along
> the first or last axis.
> - A new keyword argument "axis" could control where the axis is inserted
> in the result.
> - Vectorization support should be added in the same way to geomspace and
> logspace.
>
> Does this seem like a good idea? It's a relatively simple generalization,
> and one that I, at least, would find useful (I can think of a use-case in
> my own code that came up just last week).
>

This feels a bit forced. There's not much relevance to the minor
performance gain, and for code clarity it probably also wouldn't help
(actually it hurts usability for 99.x% of use cases by making the doc more
complicated). Not sure that it really would require a new axis argument, as
Marten said on the issue. Also, the num keyword cannot be vectorized,
unless one returns a list of arrays, which would actually be more natural
here than a 2-D array.

So, at best a don't care for me - I'm -0.5.

Cheers,
Ralf



>
> I doubt I'll have time to implement this myself in the near future, but I
> thought I would get the discussion going -- this might be a good project
> for a new contributor to work on.
>
> Cheers,
> Stephan
>
> _______________________________________________
> NumPy-Discussion mailing list
> NumPy-Discussion at python.org
> https://mail.python.org/mailman/listinfo/numpy-discussion
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20181114/ec7cb2c5/attachment.html>

From gerrit.holl at gmail.com  Wed Nov 14 12:41:02 2018
From: gerrit.holl at gmail.com (Gerrit Holl)
Date: Wed, 14 Nov 2018 17:41:02 +0000
Subject: [Numpy-discussion] Vectorized version of numpy.linspace
In-Reply-To: <CABL7CQgek9ib8zz7j7MxNHrop7ZCFsQ5imejDDB1zNv9Wcd3GA@mail.gmail.com>
References: <CAEQ_TveU8MWX=L_7_Fz8sNbT+=aYQC3=FuSsxpGTFcR86Si-Dw@mail.gmail.com>
 <CABL7CQgek9ib8zz7j7MxNHrop7ZCFsQ5imejDDB1zNv9Wcd3GA@mail.gmail.com>
Message-ID: <CAC6_WpJcDaUoUxRX5VPjUvKkCfG_wvxy1yQk+mB8pO-u6MASfA@mail.gmail.com>

On Wed, 14 Nov 2018 at 17:29, Ralf Gommers <ralf.gommers at gmail.com> wrote:
> On Wed, Nov 14, 2018 at 8:57 AM Stephan Hoyer <shoyer at gmail.com> wrote:
>>
>> It recently came up on GitHub (at part of the discussion in https://github.com/numpy/numpy/issues/12379) that numpy.linspace could, at least in principle, be modified to support array inputs:
>>
>> It looks like this has been requested on StackOverflow, too:
>> https://stackoverflow.com/questions/46694167/vectorized-numpy-linspace-across-multi-dimensional-arrays
>>
>> My tentative proposal:
>> - "start" and "stop" are broadcast against each other to form start/stop arrays. (Or we could require that start/stop have matching shape.)
>> - A new dimension of size "num" is inserted into the result, either along the first or last axis.
>> - A new keyword argument "axis" could control where the axis is inserted in the result.
>> - Vectorization support should be added in the same way to geomspace and logspace.
>>
>> Does this seem like a good idea? It's a relatively simple generalization, and one that I, at least, would find useful (I can think of a use-case in my own code that came up just last week).
>
>
> This feels a bit forced. There's not much relevance to the minor performance gain, and for code clarity it probably also wouldn't help (actually it hurts usability for 99.x% of use cases by making the doc more complicated). Not sure that it really would require a new axis argument, as Marten said on the issue. Also, the num keyword cannot be vectorized, unless one returns a list of arrays, which would actually be more natural here than a 2-D array.
>
> So, at best a don't care for me - I'm -0.5.

For what it's worth, I had a use case for this in the past week, when
I needed many simple linear interpolations between two values (thus a
linspace) with only the value of boundary points varying.  However,
this was the first time I've ever needed it, and I found a recipe on
Stack Overflow within minutes
(https://stackoverflow.com/a/42617889/974555) so it wasn't a big deal
that it wasn't available in core numpy.

Gerrit.

From harrigan.matthew at gmail.com  Wed Nov 14 12:57:35 2018
From: harrigan.matthew at gmail.com (Matthew Harrigan)
Date: Wed, 14 Nov 2018 12:57:35 -0500
Subject: [Numpy-discussion] Vectorized version of numpy.linspace
In-Reply-To: <CAC6_WpJcDaUoUxRX5VPjUvKkCfG_wvxy1yQk+mB8pO-u6MASfA@mail.gmail.com>
References: <CAEQ_TveU8MWX=L_7_Fz8sNbT+=aYQC3=FuSsxpGTFcR86Si-Dw@mail.gmail.com>
 <CABL7CQgek9ib8zz7j7MxNHrop7ZCFsQ5imejDDB1zNv9Wcd3GA@mail.gmail.com>
 <CAC6_WpJcDaUoUxRX5VPjUvKkCfG_wvxy1yQk+mB8pO-u6MASfA@mail.gmail.com>
Message-ID: <CAOfRF=jQWLWAF3vREG7Jf6D0a3MWB8R_Y91BVFh1svgq1_sD0Q@mail.gmail.com>

I put in an issue a while ago, https://github.com/numpy/numpy/issues/8839

My use case was somwhat similar to meshgrid but with a nonrectangular
domain.  Not terribly hard to code, but my expectation is that numpy
functions should always allow broadcasting if that operation makes sense.

On Nov 14, 2018 12:42 PM, "Gerrit Holl" <gerrit.holl at gmail.com> wrote:

On Wed, 14 Nov 2018 at 17:29, Ralf Gommers <ralf.gommers at gmail.com> wrote:
> On Wed, Nov 14, 2018 at 8:57 AM Stephan Hoyer <shoyer at gmail.com> wrote:
>>
>> It recently came up on GitHub (at part of the discussion in
https://github.com/numpy/numpy/issues/12379) that numpy.linspace could, at
least in principle, be modified to support array inputs:
>>
>> It looks like this has been requested on StackOverflow, too:
>>
https://stackoverflow.com/questions/46694167/vectorized-numpy-linspace-across-multi-dimensional-arrays
>>
>> My tentative proposal:
>> - "start" and "stop" are broadcast against each other to form start/stop
arrays. (Or we could require that start/stop have matching shape.)
>> - A new dimension of size "num" is inserted into the result, either
along the first or last axis.
>> - A new keyword argument "axis" could control where the axis is inserted
in the result.
>> - Vectorization support should be added in the same way to geomspace and
logspace.
>>
>> Does this seem like a good idea? It's a relatively simple
generalization, and one that I, at least, would find useful (I can think of
a use-case in my own code that came up just last week).
>
>
> This feels a bit forced. There's not much relevance to the minor
performance gain, and for code clarity it probably also wouldn't help
(actually it hurts usability for 99.x% of use cases by making the doc more
complicated). Not sure that it really would require a new axis argument, as
Marten said on the issue. Also, the num keyword cannot be vectorized,
unless one returns a list of arrays, which would actually be more natural
here than a 2-D array.
>
> So, at best a don't care for me - I'm -0.5.

For what it's worth, I had a use case for this in the past week, when
I needed many simple linear interpolations between two values (thus a
linspace) with only the value of boundary points varying.  However,
this was the first time I've ever needed it, and I found a recipe on
Stack Overflow within minutes
(https://stackoverflow.com/a/42617889/974555) so it wasn't a big deal
that it wasn't available in core numpy.


Gerrit.

_______________________________________________
NumPy-Discussion mailing list
NumPy-Discussion at python.org
https://mail.python.org/mailman/listinfo/numpy-discussion
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20181114/24a0a021/attachment-0001.html>

From m.h.vankerkwijk at gmail.com  Wed Nov 14 13:06:42 2018
From: m.h.vankerkwijk at gmail.com (Marten van Kerkwijk)
Date: Wed, 14 Nov 2018 13:06:42 -0500
Subject: [Numpy-discussion] Vectorized version of numpy.linspace
In-Reply-To: <CABL7CQgek9ib8zz7j7MxNHrop7ZCFsQ5imejDDB1zNv9Wcd3GA@mail.gmail.com>
References: <CAEQ_TveU8MWX=L_7_Fz8sNbT+=aYQC3=FuSsxpGTFcR86Si-Dw@mail.gmail.com>
 <CABL7CQgek9ib8zz7j7MxNHrop7ZCFsQ5imejDDB1zNv9Wcd3GA@mail.gmail.com>
Message-ID: <CAJNV+9vROfVDcbpvxUh3pApVYLBsOFV855njYmd07i2PgGZEpA@mail.gmail.com>

Just to add: nothing conceptually is strange for start and end to be
arrays. Indeed, the code would work for arrays as is if it didn't check the
`step == 0` case to handle denormals (arguably an even less common case
than array-like inputs...), and made a trivial change to get the new axis
to be at the start.

(I guess I'm agreeing with Matthew here that if conceptually things make
sense for arrays, then it should just work; there are of course
counterexample, such as `np.arange`, for which array-valued input makes
even less sense than allowing float and complex).
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20181114/f36f39bf/attachment.html>

From lagru at mailbox.org  Wed Nov 14 13:20:33 2018
From: lagru at mailbox.org (=?UTF-8?Q?Lars_Gr=c3=bcter?=)
Date: Wed, 14 Nov 2018 19:20:33 +0100
Subject: [Numpy-discussion] Vectorized version of numpy.linspace
In-Reply-To: <CAEQ_TveU8MWX=L_7_Fz8sNbT+=aYQC3=FuSsxpGTFcR86Si-Dw@mail.gmail.com>
References: <CAEQ_TveU8MWX=L_7_Fz8sNbT+=aYQC3=FuSsxpGTFcR86Si-Dw@mail.gmail.com>
Message-ID: <3f9fa77e-421b-e71d-ae6d-0994078fc23b@mailbox.org>

On 14.11.18 17:57, Stephan Hoyer wrote:
> It recently came up on GitHub (at part of the discussion
> in?https://github.com/numpy/numpy/issues/12379)?that numpy.linspace
> could, at least in principle, be modified to support array inputs:
> 
> It looks like this has been requested on StackOverflow, too:
> https://stackoverflow.com/questions/46694167/vectorized-numpy-linspace-across-multi-dimensional-arrays
> 
> My tentative proposal:
> - "start" and "stop" are broadcast against each other to form start/stop
> arrays. (Or we could require that start/stop have matching shape.)
> - A new dimension of size "num" is inserted into the result, either
> along the first or last axis.
> - A new keyword argument "axis" could control where the axis is inserted
> in the result.
> - Vectorization support should be added in the same way to geomspace and
> logspace.

This reminds me of a function [1] I wrote which I think has a lot of
similarities to what Stephan describes here. It is currently part of a
PR to rewrite numpy.pad [2].

Maybe that's helpful to know depending on how this discussion resolves. :)

Best regards,
Lars

[1]
https://github.com/lagru/numpy/blob/605fc1d7f871b1c8cf2783f594dee88cdf96d7ec/numpy/lib/arraypad.py#L14-L67
[2] https://github.com/numpy/numpy/pull/11358

From m.h.vankerkwijk at gmail.com  Wed Nov 14 14:32:08 2018
From: m.h.vankerkwijk at gmail.com (Marten van Kerkwijk)
Date: Wed, 14 Nov 2018 14:32:08 -0500
Subject: [Numpy-discussion] Vectorized version of numpy.linspace
In-Reply-To: <3f9fa77e-421b-e71d-ae6d-0994078fc23b@mailbox.org>
References: <CAEQ_TveU8MWX=L_7_Fz8sNbT+=aYQC3=FuSsxpGTFcR86Si-Dw@mail.gmail.com>
 <3f9fa77e-421b-e71d-ae6d-0994078fc23b@mailbox.org>
Message-ID: <CAJNV+9txKwCLbRtNOYhg+GHci6+8pEtFg3NHXN9qk_t8jUs+Eg@mail.gmail.com>

Code being better than words: see https://github.com/numpy/numpy/pull/12388
for an implementation. The change in the code proper is very small, though
it is worrying that it causes two rather unrelated tests too fail (even if
arguably both tests were wrong).

Note that this does not give flexibility to put the axis where one wants;
as written, the code made putting it at the start the obvious solution, as
it avoids doing anything with the shapes of start and stop.

-- Marten
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20181114/0a169972/attachment.html>

From m.h.vankerkwijk at gmail.com  Wed Nov 14 14:34:31 2018
From: m.h.vankerkwijk at gmail.com (Marten van Kerkwijk)
Date: Wed, 14 Nov 2018 14:34:31 -0500
Subject: [Numpy-discussion] Vectorized version of numpy.linspace
In-Reply-To: <3f9fa77e-421b-e71d-ae6d-0994078fc23b@mailbox.org>
References: <CAEQ_TveU8MWX=L_7_Fz8sNbT+=aYQC3=FuSsxpGTFcR86Si-Dw@mail.gmail.com>
 <3f9fa77e-421b-e71d-ae6d-0994078fc23b@mailbox.org>
Message-ID: <CAJNV+9uqcS_M0td-LHzB-p2tyjK1OtEYHC_LJ7-TW+TEz+P-cg@mail.gmail.com>

On Wed, Nov 14, 2018 at 1:21 PM Lars Gr?ter <lagru at mailbox.org> wrote:
<snip>

>
> This reminds me of a function [1] I wrote which I think has a lot of
> similarities to what Stephan describes here. It is currently part of a
> PR to rewrite numpy.pad [2].
>
> If we start to write the equivalent internally, then perhaps we should
indeed expose it!
Would my trial indeed help, or is it insufficient? (Sorry for not checking
myself in detail; it is not immediately obvious.)
-- Marten
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20181114/ff917527/attachment.html>

From sebastian at sipsolutions.net  Wed Nov 14 17:06:10 2018
From: sebastian at sipsolutions.net (Sebastian Berg)
Date: Wed, 14 Nov 2018 23:06:10 +0100
Subject: [Numpy-discussion] Vectorized version of numpy.linspace
In-Reply-To: <CAJNV+9txKwCLbRtNOYhg+GHci6+8pEtFg3NHXN9qk_t8jUs+Eg@mail.gmail.com>
References: <CAEQ_TveU8MWX=L_7_Fz8sNbT+=aYQC3=FuSsxpGTFcR86Si-Dw@mail.gmail.com>
 <3f9fa77e-421b-e71d-ae6d-0994078fc23b@mailbox.org>
 <CAJNV+9txKwCLbRtNOYhg+GHci6+8pEtFg3NHXN9qk_t8jUs+Eg@mail.gmail.com>
Message-ID: <2aa3b023e09ef196da423ad208e60bdb19062978.camel@sipsolutions.net>

On Wed, 2018-11-14 at 14:32 -0500, Marten van Kerkwijk wrote:
> Code being better than words: see 
> https://github.com/numpy/numpy/pull/12388 for an implementation. The
> change in the code proper is very small, though it is worrying that
> it causes two rather unrelated tests too fail (even if arguably both
> tests were wrong).
> 
> Note that this does not give flexibility to put the axis where one
> wants; as written, the code made putting it at the start the obvious
> solution, as it avoids doing anything with the shapes of start and
> stop.

Hehe, my first gut feeling was the last axis to be the obvious one ;).
This has been discussed before (but what hasn't) I believe, probably
some old issue or even PR somewhere.
I am mildly in favor, just because there is probably not much reason
against an easy vectorization. Doesn't need to be advertised much in
the docs anyway.
Although it might be good to settle the "obvious" part in case I am not
alone in first thinking of -1 being the obvious default. I would
probably skip the axis argument for now, unless someone actually has a
use case.

I think this used to half work at some point, but it was probably so
broken that we can just make it well defined?

- Sebastian


> 
> -- Marten
> _______________________________________________
> NumPy-Discussion mailing list
> NumPy-Discussion at python.org
> https://mail.python.org/mailman/listinfo/numpy-discussion
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 833 bytes
Desc: This is a digitally signed message part
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20181114/493e14fa/attachment.sig>

From shoyer at gmail.com  Wed Nov 14 17:46:11 2018
From: shoyer at gmail.com (Stephan Hoyer)
Date: Wed, 14 Nov 2018 14:46:11 -0800
Subject: [Numpy-discussion] Vectorized version of numpy.linspace
In-Reply-To: <2aa3b023e09ef196da423ad208e60bdb19062978.camel@sipsolutions.net>
References: <CAEQ_TveU8MWX=L_7_Fz8sNbT+=aYQC3=FuSsxpGTFcR86Si-Dw@mail.gmail.com>
 <3f9fa77e-421b-e71d-ae6d-0994078fc23b@mailbox.org>
 <CAJNV+9txKwCLbRtNOYhg+GHci6+8pEtFg3NHXN9qk_t8jUs+Eg@mail.gmail.com>
 <2aa3b023e09ef196da423ad208e60bdb19062978.camel@sipsolutions.net>
Message-ID: <CAEQ_Tve9pmZAj5WrvC2GhKkgk-j-yq3Eo-C8oreUJTB0kAKgUQ@mail.gmail.com>

On Wed, Nov 14, 2018 at 2:35 PM Sebastian Berg <sebastian at sipsolutions.net>
wrote:

> On Wed, 2018-11-14 at 14:32 -0500, Marten van Kerkwijk wrote:
> > Code being better than words: see
> > https://github.com/numpy/numpy/pull/12388 for an implementation. The
> > change in the code proper is very small, though it is worrying that
> > it causes two rather unrelated tests too fail (even if arguably both
> > tests were wrong).
> >
> > Note that this does not give flexibility to put the axis where one
> > wants; as written, the code made putting it at the start the obvious
> > solution, as it avoids doing anything with the shapes of start and
> > stop.
>
> Hehe, my first gut feeling was the last axis to be the obvious one ;).
> This has been discussed before (but what hasn't) I believe, probably
> some old issue or even PR somewhere.
> I am mildly in favor, just because there is probably not much reason
> against an easy vectorization. Doesn't need to be advertised much in
> the docs anyway.
> Although it might be good to settle the "obvious" part in case I am not
> alone in first thinking of -1 being the obvious default. I would
> probably skip the axis argument for now, unless someone actually has a
> use case.


Indeed -- I think the best argument for adding an "axis" argument is that
it allows people to be explicit about where the axis ends up, e.g., both
np.linspace(start, stop, num=5, axis=0) and np.linspace(start, stop, num=5,
axis=-1) make their intent quite clear.

To me, axis=0 feels like the right default, matching np.concatenate and
np.stack. But NumPy already has split conventions for this sort of thing
(e.g., gufuncs add axes at the end), so I like the explicit option.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20181114/e0b65c36/attachment-0001.html>

From einstein.edison at gmail.com  Wed Nov 14 17:59:53 2018
From: einstein.edison at gmail.com (Hameer Abbasi)
Date: Wed, 14 Nov 2018 23:59:53 +0100
Subject: [Numpy-discussion] Vectorized version of numpy.linspace
In-Reply-To: <f1bb5ea8-de15-4a33-ac2d-ab044fed6486@Canary>
References: <CAEQ_TveU8MWX=L_7_Fz8sNbT+=aYQC3=FuSsxpGTFcR86Si-Dw@mail.gmail.com>
 <3f9fa77e-421b-e71d-ae6d-0994078fc23b@mailbox.org>
 <CAJNV+9txKwCLbRtNOYhg+GHci6+8pEtFg3NHXN9qk_t8jUs+Eg@mail.gmail.com>
 <2aa3b023e09ef196da423ad208e60bdb19062978.camel@sipsolutions.net>
 <CAEQ_Tve9pmZAj5WrvC2GhKkgk-j-yq3Eo-C8oreUJTB0kAKgUQ@mail.gmail.com>
 <f1bb5ea8-de15-4a33-ac2d-ab044fed6486@Canary>
Message-ID: <d4fce6d7-4872-422e-8d82-8b3f5579ab52@Canary>


> On Wednesday, Nov 14, 2018 at 11:46 PM, Stephan Hoyer <shoyer at gmail.com (mailto:shoyer at gmail.com)> wrote:
>
>
> On Wed, Nov 14, 2018 at 2:35 PM Sebastian Berg <sebastian at sipsolutions.net (mailto:sebastian at sipsolutions.net)> wrote:
> > On Wed, 2018-11-14 at 14:32 -0500, Marten van Kerkwijk wrote:
> > > Code being better than words: see
> > > https://github.com/numpy/numpy/pull/12388 for an implementation. The
> > > change in the code proper is very small, though it is worrying that
> > > it causes two rather unrelated tests too fail (even if arguably both
> > > tests were wrong).
> > >
> > > Note that this does not give flexibility to put the axis where one
> > > wants; as written, the code made putting it at the start the obvious
> > > solution, as it avoids doing anything with the shapes of start and
> > > stop.
> >
> > Hehe, my first gut feeling was the last axis to be the obvious one ;).
> > This has been discussed before (but what hasn't) I believe, probably
> > some old issue or even PR somewhere.
> > I am mildly in favor, just because there is probably not much reason
> > against an easy vectorization. Doesn't need to be advertised much in
> > the docs anyway.
> > Although it might be good to settle the "obvious" part in case I am not
> > alone in first thinking of -1 being the obvious default. I would
> > probably skip the axis argument for now, unless someone actually has a
> > use case.
>
> Indeed -- I think the best argument for adding an "axis" argument is that it allows people to be explicit about where the axis ends up, e.g., both np.linspace(start, stop, num=5, axis=0) and np.linspace(start, stop, num=5, axis=-1) make their intent quite clear.
>
> To me, axis=0 feels like the right default, matching np.concatenate and np.stack. But NumPy already has split conventions for this sort of thing (e.g., gufuncs add axes at the end), so I like the explicit option.

I?d like to have another vote for axis=-1 by default. Stack and concatenate are different because we are concatenating/stacking complete arrays, so it makes sense to ?compose? them along the first axis to maintain C-contiguous-ness.

I actually think of this as the reverse, we are ?composing/combining? lots of 1D arrays over all the other dimensions, so to preserve C-contiguous-ness, it?s better to have axis=-1.

> _______________________________________________
> NumPy-Discussion mailing list
> NumPy-Discussion at python.org
> https://mail.python.org/mailman/listinfo/numpy-discussion

Best Regards,
Hameer Abbasi

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20181114/1e83dfd5/attachment.html>

From m.h.vankerkwijk at gmail.com  Wed Nov 14 18:14:33 2018
From: m.h.vankerkwijk at gmail.com (Marten van Kerkwijk)
Date: Wed, 14 Nov 2018 18:14:33 -0500
Subject: [Numpy-discussion] Vectorized version of numpy.linspace
In-Reply-To: <d4fce6d7-4872-422e-8d82-8b3f5579ab52@Canary>
References: <CAEQ_TveU8MWX=L_7_Fz8sNbT+=aYQC3=FuSsxpGTFcR86Si-Dw@mail.gmail.com>
 <3f9fa77e-421b-e71d-ae6d-0994078fc23b@mailbox.org>
 <CAJNV+9txKwCLbRtNOYhg+GHci6+8pEtFg3NHXN9qk_t8jUs+Eg@mail.gmail.com>
 <2aa3b023e09ef196da423ad208e60bdb19062978.camel@sipsolutions.net>
 <CAEQ_Tve9pmZAj5WrvC2GhKkgk-j-yq3Eo-C8oreUJTB0kAKgUQ@mail.gmail.com>
 <f1bb5ea8-de15-4a33-ac2d-ab044fed6486@Canary>
 <d4fce6d7-4872-422e-8d82-8b3f5579ab52@Canary>
Message-ID: <CAJNV+9tjWYd1GOpCmA_PU6y5c7=-e3z+WXz8ub_kqf4cFRABpQ@mail.gmail.com>

I see the logic in having the linear space be last, but one non-negligible
advantage of the default being the first axis is that whatever is produced
broadcasts properly against start and stop.
-- Marten
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20181114/be28781c/attachment.html>

From shoyer at gmail.com  Wed Nov 14 18:20:19 2018
From: shoyer at gmail.com (Stephan Hoyer)
Date: Wed, 14 Nov 2018 15:20:19 -0800
Subject: [Numpy-discussion] Vectorized version of numpy.linspace
In-Reply-To: <CAJNV+9tjWYd1GOpCmA_PU6y5c7=-e3z+WXz8ub_kqf4cFRABpQ@mail.gmail.com>
References: <CAEQ_TveU8MWX=L_7_Fz8sNbT+=aYQC3=FuSsxpGTFcR86Si-Dw@mail.gmail.com>
 <3f9fa77e-421b-e71d-ae6d-0994078fc23b@mailbox.org>
 <CAJNV+9txKwCLbRtNOYhg+GHci6+8pEtFg3NHXN9qk_t8jUs+Eg@mail.gmail.com>
 <2aa3b023e09ef196da423ad208e60bdb19062978.camel@sipsolutions.net>
 <CAEQ_Tve9pmZAj5WrvC2GhKkgk-j-yq3Eo-C8oreUJTB0kAKgUQ@mail.gmail.com>
 <f1bb5ea8-de15-4a33-ac2d-ab044fed6486@Canary>
 <d4fce6d7-4872-422e-8d82-8b3f5579ab52@Canary>
 <CAJNV+9tjWYd1GOpCmA_PU6y5c7=-e3z+WXz8ub_kqf4cFRABpQ@mail.gmail.com>
Message-ID: <CAEQ_Tvet7QcKkjX25VA2=aRbBt5mRWpmnMyCo4W5pvtcw6=0ag@mail.gmail.com>

On Wed, Nov 14, 2018 at 3:16 PM Marten van Kerkwijk <
m.h.vankerkwijk at gmail.com> wrote:

> I see the logic in having the linear space be last, but one non-negligible
> advantage of the default being the first axis is that whatever is produced
> broadcasts properly against start and stop.
> -- Marten
>

Yes, this is exactly why I needed to insert the new axis at the start.

That said, either default axis position is fine by me as long as we have
the explicit option.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20181114/6b71a37f/attachment.html>

From wieser.eric+numpy at gmail.com  Wed Nov 14 21:27:22 2018
From: wieser.eric+numpy at gmail.com (Eric Wieser)
Date: Wed, 14 Nov 2018 18:27:22 -0800
Subject: [Numpy-discussion] Vectorized version of numpy.linspace
In-Reply-To: <CAEQ_Tvet7QcKkjX25VA2=aRbBt5mRWpmnMyCo4W5pvtcw6=0ag@mail.gmail.com>
References: <CAEQ_TveU8MWX=L_7_Fz8sNbT+=aYQC3=FuSsxpGTFcR86Si-Dw@mail.gmail.com>
 <3f9fa77e-421b-e71d-ae6d-0994078fc23b@mailbox.org>
 <CAJNV+9txKwCLbRtNOYhg+GHci6+8pEtFg3NHXN9qk_t8jUs+Eg@mail.gmail.com>
 <2aa3b023e09ef196da423ad208e60bdb19062978.camel@sipsolutions.net>
 <CAEQ_Tve9pmZAj5WrvC2GhKkgk-j-yq3Eo-C8oreUJTB0kAKgUQ@mail.gmail.com>
 <f1bb5ea8-de15-4a33-ac2d-ab044fed6486@Canary>
 <d4fce6d7-4872-422e-8d82-8b3f5579ab52@Canary>
 <CAJNV+9tjWYd1GOpCmA_PU6y5c7=-e3z+WXz8ub_kqf4cFRABpQ@mail.gmail.com>
 <CAEQ_Tvet7QcKkjX25VA2=aRbBt5mRWpmnMyCo4W5pvtcw6=0ag@mail.gmail.com>
Message-ID: <CAL1kJvDEXb2Fh8C0VVeO9whpXoUP1eeVSv0VVb=JxtQrzYYqJg@mail.gmail.com>

I too buy into axis=0 being the better default here for broadcasting
reasons. Having it possible to specify explicitly would be handy though,
for something like:

x_ramped = np.linspace(x.min(axis=2), x.max(axis=2), 100, axis=2)

?

On Wed, 14 Nov 2018 at 15:20 Stephan Hoyer <shoyer at gmail.com> wrote:

> On Wed, Nov 14, 2018 at 3:16 PM Marten van Kerkwijk <
> m.h.vankerkwijk at gmail.com> wrote:
>
>> I see the logic in having the linear space be last, but one
>> non-negligible advantage of the default being the first axis is that
>> whatever is produced broadcasts properly against start and stop.
>> -- Marten
>>
>
> Yes, this is exactly why I needed to insert the new axis at the start.
>
> That said, either default axis position is fine by me as long as we have
> the explicit option.
> _______________________________________________
> NumPy-Discussion mailing list
> NumPy-Discussion at python.org
> https://mail.python.org/mailman/listinfo/numpy-discussion
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20181114/1656efe6/attachment-0001.html>

From sebastian at sipsolutions.net  Thu Nov 15 03:44:22 2018
From: sebastian at sipsolutions.net (Sebastian Berg)
Date: Thu, 15 Nov 2018 09:44:22 +0100
Subject: [Numpy-discussion] Vectorized version of numpy.linspace
In-Reply-To: <CAEQ_Tve9pmZAj5WrvC2GhKkgk-j-yq3Eo-C8oreUJTB0kAKgUQ@mail.gmail.com>
References: <CAEQ_TveU8MWX=L_7_Fz8sNbT+=aYQC3=FuSsxpGTFcR86Si-Dw@mail.gmail.com>
 <3f9fa77e-421b-e71d-ae6d-0994078fc23b@mailbox.org>
 <CAJNV+9txKwCLbRtNOYhg+GHci6+8pEtFg3NHXN9qk_t8jUs+Eg@mail.gmail.com>
 <2aa3b023e09ef196da423ad208e60bdb19062978.camel@sipsolutions.net>
 <CAEQ_Tve9pmZAj5WrvC2GhKkgk-j-yq3Eo-C8oreUJTB0kAKgUQ@mail.gmail.com>
Message-ID: <7c2ecd57e4ca0f2e107ae0152d158a556c1d8b1e.camel@sipsolutions.net>

On Wed, 2018-11-14 at 14:46 -0800, Stephan Hoyer wrote:
> 
> 
> On Wed, Nov 14, 2018 at 2:35 PM Sebastian Berg <
> sebastian at sipsolutions.net> wrote:
> > On Wed, 2018-11-14 at 14:32 -0500, Marten van Kerkwijk wrote:
<snip>
> > some old issue or even PR somewhere.
> > I am mildly in favor, just because there is probably not much
> > reason
> > against an easy vectorization. Doesn't need to be advertised much
> > in
> > the docs anyway.
> > Although it might be good to settle the "obvious" part in case I am
> > not
> > alone in first thinking of -1 being the obvious default. I would
> > probably skip the axis argument for now, unless someone actually
> > has a
> > use case.
> 
> Indeed -- I think the best argument for adding an "axis" argument is
> that it allows people to be explicit about where the axis ends up,
> e.g., both np.linspace(start, stop, num=5, axis=0) and
> np.linspace(start, stop, num=5, axis=-1) make their intent quite
> clear.
> 
> To me, axis=0 feels like the right default, matching np.concatenate
> and np.stack. But NumPy already has split conventions for this sort
> of thing (e.g., gufuncs add axes at the end), so I like the explicit
> option.

I think that argument goes both ways. Because linspace with an array
input can be seen as stacking the linear ramps and not stacking some
interpolated intermediate values from start/stop. (Sure, it is more
then one dimension, but I would seriously argue the linear ramps are
the basic building block and not the input start/stop arrays.)

- Sebastian


> _______________________________________________
> NumPy-Discussion mailing list
> NumPy-Discussion at python.org
> https://mail.python.org/mailman/listinfo/numpy-discussion
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 833 bytes
Desc: This is a digitally signed message part
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20181115/21e19ca9/attachment.sig>

From romesh.abey at gmail.com  Sun Nov 18 23:03:44 2018
From: romesh.abey at gmail.com (Romesh Abeysuriya)
Date: Mon, 19 Nov 2018 12:03:44 +0800
Subject: [Numpy-discussion] lstsq underdetermined behaviour
Message-ID: <CAGeX49Gp4-kHZetELqxqqaPPUof5HrBedLuLj51aGfqbEztRCQ@mail.gmail.com>

Hi all,

I'm solving an underdetermined system using `numpy.linalg.lstsq` and
trying to track down its behavior for underdetermined systems. In
previous versions of numpy (e.g. 1.14) in `linalg.py` the definition
for `lstsq` calls `dgelsd` for real inputs, which I think means that
the underdetermined system is solved with the minimum-norm solution
(that is, minimizing the norm of the solution vector, in addition to
minimizing the residual). In 1.15 the call is instead to
`_umath_linalg.lstsq_m` and I'm not sure what this actually ends up
doing - does this end up being the same as `dgelsd`? If so, it would
be great if the documentation for  `numpy.linalg.lstsq` stated that it
is returning the minimum-norm solution (as it stands, it reads as
undefined, so in theory I don't think one can rely on any particular
solution being returned for an underdetermined system)

Cheers,
Romesh

From wieser.eric+numpy at gmail.com  Sun Nov 18 23:24:18 2018
From: wieser.eric+numpy at gmail.com (Eric Wieser)
Date: Sun, 18 Nov 2018 20:24:18 -0800
Subject: [Numpy-discussion] lstsq underdetermined behaviour
In-Reply-To: <CAGeX49Gp4-kHZetELqxqqaPPUof5HrBedLuLj51aGfqbEztRCQ@mail.gmail.com>
References: <CAGeX49Gp4-kHZetELqxqqaPPUof5HrBedLuLj51aGfqbEztRCQ@mail.gmail.com>
Message-ID: <CAL1kJvBMUQokteFy2jVdtHnpBicuyg4MuW6LyApxjp0JDmUFng@mail.gmail.com>

> In 1.15 the call is instead to `_umath_linalg.lstsq_m` and I'm not sure
what this actually ends up doing - does this end up being the same as
`dgelsd`?

When the arguments are real, yes. What changed is that the dispatching now
happens in C, which was done as a step towards the incomplete
https://github.com/numpy/numpy/issues/8720.

I'm not an expert - but aren't "minimum norm" and "least squares" two ways
to state the same thing?

Eric

On Sun, 18 Nov 2018 at 20:04 Romesh Abeysuriya <romesh.abey at gmail.com>
wrote:

> Hi all,
>
> I'm solving an underdetermined system using `numpy.linalg.lstsq` and
> trying to track down its behavior for underdetermined systems. In
> previous versions of numpy (e.g. 1.14) in `linalg.py` the definition
> for `lstsq` calls `dgelsd` for real inputs, which I think means that
> the underdetermined system is solved with the minimum-norm solution
> (that is, minimizing the norm of the solution vector, in addition to
> minimizing the residual). In 1.15 the call is instead to
> `_umath_linalg.lstsq_m` and I'm not sure what this actually ends up
> doing - does this end up being the same as `dgelsd`? If so, it would
> be great if the documentation for  `numpy.linalg.lstsq` stated that it
> is returning the minimum-norm solution (as it stands, it reads as
> undefined, so in theory I don't think one can rely on any particular
> solution being returned for an underdetermined system)
>
> Cheers,
> Romesh
> _______________________________________________
> NumPy-Discussion mailing list
> NumPy-Discussion at python.org
> https://mail.python.org/mailman/listinfo/numpy-discussion
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20181118/5432ad60/attachment.html>

From charlesr.harris at gmail.com  Sun Nov 18 23:29:55 2018
From: charlesr.harris at gmail.com (Charles R Harris)
Date: Sun, 18 Nov 2018 21:29:55 -0700
Subject: [Numpy-discussion] lstsq underdetermined behaviour
In-Reply-To: <CAL1kJvBMUQokteFy2jVdtHnpBicuyg4MuW6LyApxjp0JDmUFng@mail.gmail.com>
References: <CAGeX49Gp4-kHZetELqxqqaPPUof5HrBedLuLj51aGfqbEztRCQ@mail.gmail.com>
 <CAL1kJvBMUQokteFy2jVdtHnpBicuyg4MuW6LyApxjp0JDmUFng@mail.gmail.com>
Message-ID: <CAB6mnxKnBGFFv8c5nE+1uVTwT9NcTqboGWY5jvsHQRuR1BEm=Q@mail.gmail.com>

On Sun, Nov 18, 2018 at 9:24 PM Eric Wieser <wieser.eric+numpy at gmail.com>
wrote:

> > In 1.15 the call is instead to `_umath_linalg.lstsq_m` and I'm not sure
> what this actually ends up doing - does this end up being the same as
> `dgelsd`?
>
> When the arguments are real, yes. What changed is that the dispatching now
> happens in C, which was done as a step towards the incomplete
> https://github.com/numpy/numpy/issues/8720.
>
> I'm not an expert - but aren't "minimum norm" and "least squares" two ways
> to state the same thing?
>
>
If there aren't enough data points to uniquely determine the minimizing
solution, the solution vector of shortest length is returned. In practice
it is pretty useless because it depends on the column scaling and there is
generally no natural metric in the solution space.

<snip>

Chuck
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20181118/4b0876e3/attachment.html>

From romesh.abey at gmail.com  Mon Nov 19 00:15:33 2018
From: romesh.abey at gmail.com (Romesh Abeysuriya)
Date: Mon, 19 Nov 2018 13:15:33 +0800
Subject: [Numpy-discussion] lstsq underdetermined behaviour
In-Reply-To: <CAB6mnxKnBGFFv8c5nE+1uVTwT9NcTqboGWY5jvsHQRuR1BEm=Q@mail.gmail.com>
References: <CAGeX49Gp4-kHZetELqxqqaPPUof5HrBedLuLj51aGfqbEztRCQ@mail.gmail.com>
 <CAL1kJvBMUQokteFy2jVdtHnpBicuyg4MuW6LyApxjp0JDmUFng@mail.gmail.com>
 <CAB6mnxKnBGFFv8c5nE+1uVTwT9NcTqboGWY5jvsHQRuR1BEm=Q@mail.gmail.com>
Message-ID: <CAGeX49EWh4zrB=xHCkcRR7=wir_hCu56cpvYiOMs_9taX+qaTg@mail.gmail.com>

Thanks both! Yes, I guess it's typically 'least squares' referring to
the residual vector, and 'minimum norm' referring to the solution
vector. That's certainly how the documentation for `dgelsd` frames it.
In my case, the minimum norm solution can be sensibly interpreted (and
in particular, it guarantees that the solution is 0 for missing
variables), so it's great to know that I can rely on this being
returned

Cheers,
Romesh



On Mon, Nov 19, 2018 at 12:30 PM Charles R Harris
<charlesr.harris at gmail.com> wrote:
>
>
>
> On Sun, Nov 18, 2018 at 9:24 PM Eric Wieser <wieser.eric+numpy at gmail.com> wrote:
>>
>> > In 1.15 the call is instead to `_umath_linalg.lstsq_m` and I'm not sure what this actually ends up doing - does this end up being the same as `dgelsd`?
>>
>> When the arguments are real, yes. What changed is that the dispatching now happens in C, which was done as a step towards the incomplete https://github.com/numpy/numpy/issues/8720.
>>
>> I'm not an expert - but aren't "minimum norm" and "least squares" two ways to state the same thing?
>>
>
> If there aren't enough data points to uniquely determine the minimizing solution, the solution vector of shortest length is returned. In practice it is pretty useless because it depends on the column scaling and there is generally no natural metric in the solution space.
>
> <snip>
>
> Chuck
> _______________________________________________
> NumPy-Discussion mailing list
> NumPy-Discussion at python.org
> https://mail.python.org/mailman/listinfo/numpy-discussion

From charlesr.harris at gmail.com  Mon Nov 26 12:41:24 2018
From: charlesr.harris at gmail.com (Charles R Harris)
Date: Mon, 26 Nov 2018 10:41:24 -0700
Subject: [Numpy-discussion] NumPy 1.16
Message-ID: <CAB6mnxJ-rkJc4=ihYuzOAb_u5KYL4ZtjbT2pB+fdRYZsdSLyBg@mail.gmail.com>

Hi All,

Just an update of the NumPy 1.16 release schedule. The last PR milestoned
for the release is #12219 <https://github.com/numpy/numpy/pull/12219>, and
it is about done. The current release blocker is the upcoming OpenBLAS
0.3.4, which should fix the reported threading problems, but if it doesn't
come out in the next month we might want to revisit that. The OpenBLAS
issues for 0.3.4 are here
<https://github.com/xianyi/OpenBLAS/issues?q=is%3Aopen+is%3Aissue+milestone%3A0.3.4>.
Meanwhile, other PR's continue to go in.

Chuck
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20181126/8fbd85a3/attachment.html>

From tyler.je.reddy at gmail.com  Mon Nov 26 13:47:47 2018
From: tyler.je.reddy at gmail.com (Tyler Reddy)
Date: Mon, 26 Nov 2018 10:47:47 -0800
Subject: [Numpy-discussion] ANN: SciPy 1.2.0rc1 -- please test
Message-ID: <CAHPuU_ZGypUKxVAGL5YXrOu1jiOzdGdssshmQ7751nMJL+GBYA@mail.gmail.com>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

Hi all,

On behalf of the SciPy development team I'm pleased to announce
the release candidate SciPy 1.2.0rc1. Please help us test out this
release candidate -- the 1.2.x series will be an LTS release and the
last to support Python 2.7.

Sources and binary wheels can be found at:
https://pypi.python.org/pypi/scipy
and at:
https://github.com/scipy/scipy/releases/tag/v1.2.0rc1

To install the release candidate with pip:

pip install scipy==1.2.0rc1


==========================
SciPy 1.2.0 Release Notes
==========================

Note: Scipy 1.2.0 is not released yet!

SciPy 1.2.0 is the culmination of 6 months of hard work. It contains
many new features, numerous bug-fixes, improved test coverage and better
documentation. There have been a number of deprecations and API changes
in this release, which are documented below. All users are encouraged to
upgrade to this release, as there are a large number of bug-fixes and
optimizations. Before upgrading, we recommend that users check that
their own code does not use deprecated SciPy functionality (to do so,
run your code with ``python -Wd`` and check for ``DeprecationWarning`` s).
Our development attention will now shift to bug-fix releases on the
1.2.x branch, and on adding new features on the master branch.

This release requires Python 2.7 or 3.4+ and NumPy 1.8.2 or greater.

Note: This will be the last SciPy release to support Python 2.7.
          Consequently, the 1.2.x series will be a long term support (LTS)
          release; we will backport bug fixes until 1 Jan 2020.

For running on PyPy, PyPy3 6.0+ and NumPy 1.15.0 are required.

Highlights of this release
----------------------------

- - 1-D root finding improvements with a new solver, ``toms748``, and a new
  unified interface, ``root_scalar``
- - New ``dual_annealing`` optimization method that combines stochastic and
  local deterministic searching
- - A new optimization algorithm, ``shgo`` (simplicial homology
  global optimization) for derivative free optimization problems
- - A new category of quaternion-based transformations are available in
`scipy.spatial.transform`

New features
============

`scipy.ndimage` improvements
- --------------------------------

Proper spline coefficient calculations have been added for the ``mirror``,
``wrap``, and ``reflect`` modes of `scipy.ndimage.rotate`

`scipy.fftpack` improvements
- --------------------------------

DCT-IV, DST-IV, DCT-I, and DST-I orthonormalization are now supported in
`scipy.fftpack`.

`scipy.interpolate` improvements
- --------------------------------

`scipy.interpolate.pade` now accepts a new argument for the order of the
numerator

`scipy.cluster` improvements
- ----------------------------

`scipy.cluster.vq.kmeans2` gained a new initialization method, kmeans++.

`scipy.special` improvements
- ----------------------------

The function ``softmax`` was added to `scipy.special`.

`scipy.optimize` improvements
- -----------------------------

The one-dimensional nonlinear solvers have been given a unified interface
`scipy.optimize.root_scalar`, similar to the `scipy.optimize.root` interface
for multi-dimensional solvers. ``scipy.optimize.root_scalar(f, bracket=[a
,b],
method="brenth")`` is equivalent to ``scipy.optimize.brenth(f, a ,b)``.  If
no
``method`` is specified, an appropriate one will be selected based upon the
bracket and the number of derivatives available.

The so-called Algorithm 748 of Alefeld, Potra and Shi for root-finding
within
an enclosing interval has been added as `scipy.optimize.toms748`. This
provides
guaranteed convergence to a root with convergence rate per function
evaluation
of approximately 1.65 (for sufficiently well-behaved functions.)

``differential_evolution`` now has the ``updating`` and ``workers``
keywords.
The first chooses between continuous updating of the best solution vector
(the
default), or once per generation. Continuous updating can lead to faster
convergence. The ``workers`` keyword accepts an ``int`` or map-like
callable,
and parallelises the solver (having the side effect of updating once per
generation). Supplying an ``int`` evaluates the trial solutions in N
parallel
parts. Supplying a map-like callable allows other parallelisation approaches
(such as ``mpi4py``, or ``joblib``) to be used.

``dual_annealing`` (and ``shgo`` below) is a powerful new general purpose
global optizimation (GO) algorithm. ``dual_annealing`` uses two annealing
processes to accelerate the convergence towards the global minimum of an
objective mathematical function. The first annealing process controls the
stochastic Markov chain searching and the second annealing process controls
the
deterministic minimization. So, dual annealing is a hybrid method that takes
advantage of stochastic and local deterministic searching in an efficient
way.

``shgo`` (simplicial homology global optimization) is a similar algorithm
appropriate for solving black box and derivative free optimization (DFO)
problems. The algorithm generally converges to the global solution in finite
time. The convergence holds for non-linear inequality and
equality constraints. In addition to returning a global minimum, the
algorithm also returns any other global and local minima found after every
iteration. This makes it useful for exploring the solutions in a domain.

`scipy.optimize.newton` can now accept a scalar or an array

``MINPACK`` usage is now thread-safe, such that ``MINPACK`` + callbacks may
be used on multiple threads.

`scipy.signal` improvements
- ---------------------------

Digital filter design functions now include a parameter to specify the
sampling
rate. Previously, digital filters could only be specified using normalized
frequency, but different functions used different scales (e.g. 0 to 1 for
``butter`` vs 0 to ? for ``freqz``), leading to errors and confusion.  With
the ``fs`` parameter, ordinary frequencies can now be entered directly into
functions, with the normalization handled internally.

``find_peaks`` and related functions no longer raise an exception if the
properties of a peak have unexpected values (e.g. a prominence of 0). A
``PeakPropertyWarning`` is given instead.

The new keyword argument ``plateau_size`` was added to ``find_peaks``.
``plateau_size`` may be used to select peaks based on the length of the
flat top of a peak.

``welch()`` and ``csd()`` methods in `scipy.signal` now support calculation
of a median average PSD, using ``average='mean'`` keyword

`scipy.sparse` improvements
- ---------------------------

The `scipy.sparse.bsr_matrix.tocsr` method is now implemented directly
instead
of converting via COO format, and the `scipy.sparse.bsr_matrix.tocsc` method
is now also routed via CSR conversion instead of COO. The efficiency of both
conversions is now improved.

The issue where SuperLU or UMFPACK solvers crashed on matrices with
non-canonical format in `scipy.sparse.linalg` was fixed. The solver wrapper
canonicalizes the matrix if necessary before calling the SuperLU or UMFPACK
solver.

The ``largest`` option of `scipy.sparse.linalg.lobpcg()` was fixed to have
a correct (and expected) behavior. The order of the eigenvalues was made
consistent with the ARPACK solver (``eigs()``), i.e. ascending for the
smallest eigenvalues, and descending for the largest eigenvalues.

The `scipy.sparse.random` function is now faster and also supports integer
and
complex values by passing the appropriate value to the ``dtype`` argument.

`scipy.spatial` improvements
- ----------------------------

The function `scipy.spatial.distance.jaccard` was modified to return 0
instead
of ``np.nan`` when two all-zero vectors are compared.

Support for the Jensen Shannon distance, the square-root of the divergence,
has
been added under `scipy.spatial.distance.jensenshannon`

An optional keyword was added to the function
`scipy.spatial.cKDTree.query_ball_point()` to sort or not sort the returned
indices. Not sorting the indices can speed up calls.

A new category of quaternion-based transformations are available in
`scipy.spatial.transform`, including spherical linear interpolation of
rotations (``Slerp``), conversions to and from quaternions, Euler angles,
and general rotation and inversion capabilities
(`spatial.transform.Rotation`), and uniform random sampling of 3D
rotations (`spatial.transform.Rotation.random`).

`scipy.stats` improvements
- --------------------------

The Yeo-Johnson power transformation is now supported (``yeojohnson``,
``yeojohnson_llf``, ``yeojohnson_normmax``, ``yeojohnson_normplot``). Unlike
the Box-Cox transformation, the Yeo-Johnson transformation can accept
negative
values.

Added a general method to sample random variates based on the density only,
in
the new function ``rvs_ratio_uniforms``.

The Yule-Simon distribution (``yulesimon``) was added -- this is a new
discrete probability distribution.

``stats`` and ``mstats`` now have access to a new regression method,
``siegelslopes``, a robust linear regression algorithm

`scipy.stats.gaussian_kde` now has the ability to deal with weighted
samples,
and should have a modest improvement in performance

Levy Stable Parameter Estimation, PDF, and CDF calculations are now
supported
for `scipy.stats.levy_stable`.

The Brunner-Munzel test is now available as ``brunnermunzel`` in ``stats``
and ``mstats``

`scipy.linalg` improvements
- --------------------------

`scipy.linalg.lapack` now exposes the LAPACK routines using the Rectangular
Full Packed storage (RFP) for upper triangular, lower triangular, symmetric,
or Hermitian matrices; the upper trapezoidal fat matrix RZ decomposition
routines are now available as well.

Deprecated features
===================
The functions ``hyp2f0``, ``hyp1f2`` and ``hyp3f0`` in ``scipy.special``
have
been deprecated.


Backwards incompatible changes
==============================

LAPACK version 3.4.0 or later is now required. Building with
Apple Accelerate is no longer supported.

The function ``scipy.linalg.subspace_angles(A, B)`` now gives correct
results for all angles. Before this, the function only returned
correct values for those angles which were greater than pi/4.

Support for the Bento build system has been removed. Bento has not been
maintained for several years, and did not have good Python 3 or wheel
support,
hence it was time to remove it.

The required signature of `scipy.optimize.lingprog` ``method=simplex``
callback function has changed. Before iteration begins, the simplex solver
first converts the problem into a standard form that does not, in general,
have the same variables or constraints
as the problem defined by the user. Previously, the simplex solver would
pass a
user-specified callback function several separate arguments, such as the
current solution vector ``xk``, corresponding to this standard form problem.
Unfortunately, the relationship between the standard form problem and the
user-defined problem was not documented, limiting the utility of the
information passed to the callback function.

In addition to numerous bug fix changes, the simplex solver now passes a
user-specified callback function a single ``OptimizeResult`` object
containing
information that corresponds directly to the user-defined problem. In future
releases, this ``OptimizeResult`` object may be expanded to include
additional
information, such as variables corresponding to the standard-form problem
and
information concerning the relationship between the standard-form and
user-defined problems.

The implementation of `scipy.sparse.random` has changed, and this affects
the
numerical values returned for both ``sparse.random`` and ``sparse.rand`` for
some matrix shapes and a given seed.

`scipy.optimize.newton` will no longer use Halley's method in cases where it
negatively impacts convergence

Other changes
=============


Authors
=======

* @endolith
* @luzpaz
* Hameer Abbasi +
* akahard2dj +
* Anton Akhmerov
* Joseph Albert
* alexthomas93 +
* ashish +
* atpage +
* Blair Azzopardi +
* Yoshiki V?zquez Baeza
* Bence Bagi +
* Christoph Baumgarten
* Lucas Bellomo +
* BH4 +
* Aditya Bharti
* Max Bolingbroke
* Fran?ois Boulogne
* Ward Bradt +
* Matthew Brett
* Evgeni Burovski
* Rafa? Byczek +
* Alfredo Canziani +
* CJ Carey
* Luc?a Cheung +
* Poom Chiarawongse +
* Jeanne Choo +
* Robert Cimrman
* Graham Clenaghan +
* cynthia-rempel +
* Johannes Damp +
* Jaime Fernandez del Rio
* Dowon +
* emmi474 +
* Stefan Endres +
* Thomas Etherington +
* Alex Fikl +
* fo40225 +
* Joseph Fox-Rabinovitz
* Lars G
* Abhinav Gautam +
* Stiaan Gerber +
* C.A.M. Gerlach +
* Ralf Gommers
* Todd Goodall
* Lars Grueter +
* Sylvain Gubian +
* Matt Haberland
* David Hagen
* Will Handley +
* Charles Harris
* Ian Henriksen
* Thomas Hisch +
* Theodore Hu
* Michael Hudson-Doyle +
* Nicolas Hug +
* jakirkham +
* Jakob Jakobson +
* James +
* Jan Schl?ter
* jeanpauphilet +
* josephmernst +
* Kai +
* Kai-Striega +
* kalash04 +
* Toshiki Kataoka +
* Konrad0 +
* Tom Krauss +
* Johannes Kulick
* Lars Gr?ter +
* Eric Larson
* Denis Laxalde
* Will Lee +
* Katrin Leinweber +
* Yin Li +
* P. L. Lim +
* Jesse Livezey +
* Duncan Macleod +
* MatthewFlamm +
* Nikolay Mayorov
* Mike McClurg +
* Christian Meyer +
* Mark Mikofski
* Naoto Mizuno +
* mohmmadd +
* Nathan Musoke
* Anju Geetha Nair +
* Andrew Nelson
* Ayappan P +
* Nick Papior
* Haesun Park +
* Ronny Pfannschmidt +
* pijyoi +
* Ilhan Polat
* Anthony Polloreno +
* Ted Pudlik
* puenka
* Eric Quintero
* Pradeep Reddy Raamana +
* Vyas Ramasubramani +
* Ramon Vi?as +
* Tyler Reddy
* Joscha Reimer
* Antonio H Ribeiro
* richardjgowers +
* Rob +
* robbystk +
* Lucas Roberts +
* rohan +
* Joaquin Derrac Rus +
* Josua Sassen +
* Bruce Sharpe +
* Max Shinn +
* Scott Sievert
* Sourav Singh
* Strahinja Luki? +
* Kai Striega +
* Shinya SUZUKI +
* Mike Toews +
* Piotr Uchwat
* Miguel de Val-Borro +
* Nicky van Foreest
* Paul van Mulbregt
* Gael Varoquaux
* Pauli Virtanen
* Stefan van der Walt
* Warren Weckesser
* Joshua Wharton +
* Bernhard M. Wiedemann +
* Eric Wieser
* Josh Wilson
* Tony Xiang +
* Roman Yurchak +
* Roy Zywina +

A total of 137 people contributed to this release.
People with a "+" by their names contributed a patch for the first time.
This list of names is automatically generated, and may not be fully
complete.

Issues closed for 1.2.0
- -----------------------

* `#1240 <https://github.com/scipy/scipy/issues/1240>`__: Allowing
multithreaded use of minpack through scipy.optimize...
* `#1432 <https://github.com/scipy/scipy/issues/1432>`__: scipy.stats.mode
extremely slow (Trac #905)
* `#3372 <https://github.com/scipy/scipy/issues/3372>`__: Please add Sphinx
search field to online scipy html docs
* `#3678 <https://github.com/scipy/scipy/issues/3678>`__:
_clough_tocher_2d_single direction between centroids
* `#4174 <https://github.com/scipy/scipy/issues/4174>`__: lobpcg "largest"
option invalid?
* `#5493 <https://github.com/scipy/scipy/issues/5493>`__: anderson_ksamp
p-values>1
* `#5743 <https://github.com/scipy/scipy/issues/5743>`__: slsqp fails to
detect infeasible problem
* `#6139 <https://github.com/scipy/scipy/issues/6139>`__:
scipy.optimize.linprog failed to find a feasible starting point...
* `#6358 <https://github.com/scipy/scipy/issues/6358>`__: stats: docstring
for `vonmises_line` points to `vonmises_line`...
* `#6498 <https://github.com/scipy/scipy/issues/6498>`__: runtests.py is
missing in pypi distfile
* `#7426 <https://github.com/scipy/scipy/issues/7426>`__:
scipy.stats.ksone(n).pdf(x) returns nan for positive values of...
* `#7455 <https://github.com/scipy/scipy/issues/7455>`__:
scipy.stats.ksone.pdf(2,x) return incorrect values for x near...
* `#7456 <https://github.com/scipy/scipy/issues/7456>`__:
scipy.special.smirnov and scipy.special.smirnovi have accuracy...
* `#7492 <https://github.com/scipy/scipy/issues/7492>`__:
scipy.special.kolmogorov(x)/kolmogi(p) inefficient, inaccurate...
* `#7914 <https://github.com/scipy/scipy/issues/7914>`__: TravisCI not
failing when it should for -OO run
* `#8064 <https://github.com/scipy/scipy/issues/8064>`__: linalg.solve test
crashes on Windows
* `#8212 <https://github.com/scipy/scipy/issues/8212>`__: LAPACK
Rectangular Full Packed routines
* `#8256 <https://github.com/scipy/scipy/issues/8256>`__:
differential_evolution bug converges to wrong results in complex...
* `#8443 <https://github.com/scipy/scipy/issues/8443>`__: Deprecate
`hyp2f0`, `hyp1f2`, and `hyp3f0`?
* `#8452 <https://github.com/scipy/scipy/issues/8452>`__: DOC: ARPACK
tutorial has two conflicting equations
* `#8680 <https://github.com/scipy/scipy/issues/8680>`__: scipy fails
compilation when building from source
* `#8686 <https://github.com/scipy/scipy/issues/8686>`__: Division by zero
in _trustregion.py when x0 is exactly equal...
* `#8700 <https://github.com/scipy/scipy/issues/8700>`__: _MINPACK_LOCK not
held when calling into minpack from least_squares
* `#8786 <https://github.com/scipy/scipy/issues/8786>`__: erroneous moment
values for t-distribution
* `#8791 <https://github.com/scipy/scipy/issues/8791>`__: Checking COLA
condition in istft should be optional (or omitted)
* `#8843 <https://github.com/scipy/scipy/issues/8843>`__: imresize cannot
be deprecated just yet
* `#8844 <https://github.com/scipy/scipy/issues/8844>`__: Inverse Wishart
Log PDF Incorrect for Non-diagonal Scale Matrix?
* `#8878 <https://github.com/scipy/scipy/issues/8878>`__: vonmises and
vonmises_line in stats: vonmises wrong and superfluous?
* `#8895 <https://github.com/scipy/scipy/issues/8895>`__: v1.1.0
`ndi.rotate` documentation ? reused parameters not filled...
* `#8900 <https://github.com/scipy/scipy/issues/8900>`__: Missing complex
conjugation in scipy.sparse.linalg.LinearOperator
* `#8904 <https://github.com/scipy/scipy/issues/8904>`__: BUG: if zero
derivative at root, then Newton fails with RuntimeWarning
* `#8911 <https://github.com/scipy/scipy/issues/8911>`__:
make_interp_spline bc_type incorrect input interpretation
* `#8942 <https://github.com/scipy/scipy/issues/8942>`__: MAINT: Refactor
`_linprog.py` and `_linprog_ip.py` to remove...
* `#8947 <https://github.com/scipy/scipy/issues/8947>`__: np.int64 in
scipy.fftpack.next_fast_len
* `#9020 <https://github.com/scipy/scipy/issues/9020>`__: BUG:
linalg.subspace_angles gives wrong results
* `#9033 <https://github.com/scipy/scipy/issues/9033>`__:
scipy.stats.normaltest sometimes gives incorrect returns b/c...
* `#9036 <https://github.com/scipy/scipy/issues/9036>`__: Bizarre times for
`scipy.sparse.rand` function with 'low' density...
* `#9044 <https://github.com/scipy/scipy/issues/9044>`__:
optimize.minimize(method=`trust-constr`) result dict does not...
* `#9071 <https://github.com/scipy/scipy/issues/9071>`__: doc/linalg: add
cho_solve_banded to see also of cholesky_banded
* `#9082 <https://github.com/scipy/scipy/issues/9082>`__: eigenvalue
sorting in scipy.sparse.linalg.eigsh
* `#9086 <https://github.com/scipy/scipy/issues/9086>`__:
signaltools.py:491: FutureWarning: Using a non-tuple sequence...
* `#9091 <https://github.com/scipy/scipy/issues/9091>`__:
test_spline_filter failure on 32-bit
* `#9122 <https://github.com/scipy/scipy/issues/9122>`__: Typo on scipy
minimization tutorial
* `#9135 <https://github.com/scipy/scipy/issues/9135>`__: doc error at
https://docs.scipy.org/doc/scipy/reference/tutorial/stats/discrete_poisson.html
* `#9167 <https://github.com/scipy/scipy/issues/9167>`__: DOC: BUG: typo in
ndimage LowLevelCallable tutorial example
* `#9169 <https://github.com/scipy/scipy/issues/9169>`__: truncnorm does
not work if b < a in scipy.stats
* `#9250 <https://github.com/scipy/scipy/issues/9250>`__:
scipy.special.tests.test_mpmath::TestSystematic::test_pcfw fails...
* `#9259 <https://github.com/scipy/scipy/issues/9259>`__: rv.expect() ==
rv.mean() is false for rv.mean() == nan (and inf)
* `#9286 <https://github.com/scipy/scipy/issues/9286>`__: DOC: Rosenbrock
expression in optimize.minimize tutorial
* `#9316 <https://github.com/scipy/scipy/issues/9316>`__: SLSQP fails in
nested optimization
* `#9337 <https://github.com/scipy/scipy/issues/9337>`__:
scipy.signal.find_peaks key typo in documentation
* `#9345 <https://github.com/scipy/scipy/issues/9345>`__: Example from
documentation of scipy.sparse.linalg.eigs raises...
* `#9383 <https://github.com/scipy/scipy/issues/9383>`__: Default value for
"mode" in "ndimage.shift"
* `#9419 <https://github.com/scipy/scipy/issues/9419>`__: dual_annealing
off by one in the number of iterations
* `#9442 <https://github.com/scipy/scipy/issues/9442>`__: Error in
Defintion of Rosenbrock Function
* `#9453 <https://github.com/scipy/scipy/issues/9453>`__: TST:
test_eigs_consistency() doesn't have consistent results


Pull requests for 1.2.0
- -----------------------

* `#7352 <https://github.com/scipy/scipy/pull/7352>`__: ENH: add Brunner
Munzel test to scipy.stats.
* `#7373 <https://github.com/scipy/scipy/pull/7373>`__: BUG: Jaccard
distance for all-zero arrays would return np.nan
* `#7374 <https://github.com/scipy/scipy/pull/7374>`__: ENH: Add PDF, CDF
and parameter estimation for Stable Distributions
* `#8098 <https://github.com/scipy/scipy/pull/8098>`__: ENH: Add shgo for
global optimization of NLPs.
* `#8203 <https://github.com/scipy/scipy/pull/8203>`__: ENH: adding
simulated dual annealing to optimize
* `#8259 <https://github.com/scipy/scipy/pull/8259>`__: Option to follow
original Storn and Price algorithm and its parallelisation
* `#8293 <https://github.com/scipy/scipy/pull/8293>`__: ENH add
ratio-of-uniforms method for rv generation to scipy.stats
* `#8294 <https://github.com/scipy/scipy/pull/8294>`__: BUG: Fix slowness
in stats.mode
* `#8295 <https://github.com/scipy/scipy/pull/8295>`__: ENH: add Jensen
Shannon distance to `scipy.spatial.distance`
* `#8357 <https://github.com/scipy/scipy/pull/8357>`__: ENH: vectorize
scalar zero-search-functions
* `#8397 <https://github.com/scipy/scipy/pull/8397>`__: Add `fs=` parameter
to filter design functions
* `#8537 <https://github.com/scipy/scipy/pull/8537>`__: ENH: Implement mode
parameter for spline filtering.
* `#8558 <https://github.com/scipy/scipy/pull/8558>`__: ENH: small speedup
for stats.gaussian_kde
* `#8560 <https://github.com/scipy/scipy/pull/8560>`__: BUG: fix p-value
calc of anderson_ksamp in scipy.stats
* `#8614 <https://github.com/scipy/scipy/pull/8614>`__: ENH: correct
p-values for stats.kendalltau and stats.mstats.kendalltau
* `#8670 <https://github.com/scipy/scipy/pull/8670>`__: ENH: Require Lapack
3.4.0
* `#8683 <https://github.com/scipy/scipy/pull/8683>`__: Correcting kmeans
documentation
* `#8725 <https://github.com/scipy/scipy/pull/8725>`__: MAINT: Cleanup
scipy.optimize.leastsq
* `#8726 <https://github.com/scipy/scipy/pull/8726>`__: BUG: Fix
_get_output in scipy.ndimage to support string
* `#8733 <https://github.com/scipy/scipy/pull/8733>`__: MAINT: stats: A bit
of clean up.
* `#8737 <https://github.com/scipy/scipy/pull/8737>`__: BUG: Improve
numerical precision/convergence failures of smirnov/kolmogorov
* `#8738 <https://github.com/scipy/scipy/pull/8738>`__: MAINT: stats: A bit
of clean up in test_distributions.py.
* `#8740 <https://github.com/scipy/scipy/pull/8740>`__: BF/ENH: make
minpack thread safe
* `#8742 <https://github.com/scipy/scipy/pull/8742>`__: BUG: Fix division
by zero in trust-region optimization methods
* `#8746 <https://github.com/scipy/scipy/pull/8746>`__: MAINT: signal: Fix
a docstring of a private function, and fix...
* `#8750 <https://github.com/scipy/scipy/pull/8750>`__: DOC clarified
description of norminvgauss in scipy.stats
* `#8753 <https://github.com/scipy/scipy/pull/8753>`__: DOC: signal: Fix a
plot title in the chirp docstring.
* `#8755 <https://github.com/scipy/scipy/pull/8755>`__: DOC: MAINT: Fix
link to the wheel documentation in developer...
* `#8760 <https://github.com/scipy/scipy/pull/8760>`__: BUG: stats:
boltzmann wasn't setting the upper bound.
* `#8763 <https://github.com/scipy/scipy/pull/8763>`__: [DOC] Improved
scipy.cluster.hierarchy documentation
* `#8765 <https://github.com/scipy/scipy/pull/8765>`__: DOC: added example
for scipy.stat.mstats.tmin
* `#8788 <https://github.com/scipy/scipy/pull/8788>`__: DOC: fix definition
of optional `disp` parameter
* `#8802 <https://github.com/scipy/scipy/pull/8802>`__: MAINT: Suppress
dd_real unused function compiler warnings.
* `#8803 <https://github.com/scipy/scipy/pull/8803>`__: ENH: Add
full_output support to optimize.newton()
* `#8804 <https://github.com/scipy/scipy/pull/8804>`__: MAINT: stats cleanup
* `#8808 <https://github.com/scipy/scipy/pull/8808>`__: DOC: add note about
isinstance for frozen rvs
* `#8812 <https://github.com/scipy/scipy/pull/8812>`__: Updated numpydoc
submodule
* `#8813 <https://github.com/scipy/scipy/pull/8813>`__: MAINT: stats: Fix
multinomial docstrings, and do some clean up.
* `#8816 <https://github.com/scipy/scipy/pull/8816>`__: BUG: fixed _stats
of t-distribution in scipy.stats
* `#8817 <https://github.com/scipy/scipy/pull/8817>`__: BUG: ndimage: Fix
validation of the origin argument in correlate...
* `#8822 <https://github.com/scipy/scipy/pull/8822>`__: BUG: integrate: Fix
crash with repeated t values in odeint.
* `#8832 <https://github.com/scipy/scipy/pull/8832>`__: Hyperlink DOIs
against preferred resolver
* `#8837 <https://github.com/scipy/scipy/pull/8837>`__: BUG: sparse: Ensure
correct dtype for sparse comparison operations.
* `#8839 <https://github.com/scipy/scipy/pull/8839>`__: DOC: stats: A few
tweaks to the linregress docstring.
* `#8846 <https://github.com/scipy/scipy/pull/8846>`__: BUG: stats: Fix
logpdf method of invwishart.
* `#8849 <https://github.com/scipy/scipy/pull/8849>`__: DOC: signal: Fixed
mistake in the firwin docstring.
* `#8854 <https://github.com/scipy/scipy/pull/8854>`__: DOC: fix type
descriptors in ltisys documentation
* `#8865 <https://github.com/scipy/scipy/pull/8865>`__: Fix tiny typo in
docs for chi2 pdf
* `#8870 <https://github.com/scipy/scipy/pull/8870>`__: Fixes related to
invertibility of STFT
* `#8872 <https://github.com/scipy/scipy/pull/8872>`__: ENH: special: Add
the softmax function
* `#8874 <https://github.com/scipy/scipy/pull/8874>`__: DOC correct gamma
function in docstrings in scipy.stats
* `#8876 <https://github.com/scipy/scipy/pull/8876>`__: ENH: Added TOMS
Algorithm 748 as 1-d root finder; 17 test function...
* `#8882 <https://github.com/scipy/scipy/pull/8882>`__: ENH: Only use
Halley's adjustment to Newton if close enough.
* `#8883 <https://github.com/scipy/scipy/pull/8883>`__: FIX: optimize: make
jac and hess truly optional for 'trust-constr'
* `#8885 <https://github.com/scipy/scipy/pull/8885>`__: TST: Do not error
on warnings raised about non-tuple indexing.
* `#8887 <https://github.com/scipy/scipy/pull/8887>`__: MAINT: filter out
np.matrix PendingDeprecationWarning's in numpy...
* `#8889 <https://github.com/scipy/scipy/pull/8889>`__: DOC: optimize:
separate legacy interfaces from new ones
* `#8890 <https://github.com/scipy/scipy/pull/8890>`__: ENH: Add
optimize.root_scalar() as a universal dispatcher for...
* `#8899 <https://github.com/scipy/scipy/pull/8899>`__: DCT-IV, DST-IV and
DCT-I, DST-I orthonormalization support in...
* `#8901 <https://github.com/scipy/scipy/pull/8901>`__: MAINT: Reorganize
flapack.pyf.src file
* `#8907 <https://github.com/scipy/scipy/pull/8907>`__: BUG: ENH: Check if
guess for newton is already zero before checking...
* `#8908 <https://github.com/scipy/scipy/pull/8908>`__: ENH: Make sorting
optional for cKDTree.query_ball_point()
* `#8910 <https://github.com/scipy/scipy/pull/8910>`__: DOC: sparse.csgraph
simple examples.
* `#8914 <https://github.com/scipy/scipy/pull/8914>`__: DOC: interpolate:
fix equivalences of string aliases
* `#8918 <https://github.com/scipy/scipy/pull/8918>`__: add
float_control(precise, on) to _fpumode.c
* `#8919 <https://github.com/scipy/scipy/pull/8919>`__: MAINT: interpolate:
improve error messages for common `bc_type`...
* `#8920 <https://github.com/scipy/scipy/pull/8920>`__: DOC: update
Contributing to SciPy to say "prefer no PEP8 only...
* `#8924 <https://github.com/scipy/scipy/pull/8924>`__: MAINT: special:
deprecate `hyp2f0`, `hyp1f2`, and `hyp3f0`
* `#8927 <https://github.com/scipy/scipy/pull/8927>`__: MAINT: special:
remove `errprint`
* `#8932 <https://github.com/scipy/scipy/pull/8932>`__: Fix broadcasting
scale arg of entropy
* `#8936 <https://github.com/scipy/scipy/pull/8936>`__: Fix (some)
non-tuple index warnings
* `#8937 <https://github.com/scipy/scipy/pull/8937>`__: ENH: implement
sparse matrix BSR to CSR conversion directly.
* `#8938 <https://github.com/scipy/scipy/pull/8938>`__: DOC: add
@_ni_docstrings.docfiller in ndimage.rotate
* `#8940 <https://github.com/scipy/scipy/pull/8940>`__: Update
_discrete_distns.py
* `#8943 <https://github.com/scipy/scipy/pull/8943>`__: DOC: Finish
dangling sentence in `convolve` docstring
* `#8944 <https://github.com/scipy/scipy/pull/8944>`__: MAINT: Address
tuple indexing and warnings
* `#8945 <https://github.com/scipy/scipy/pull/8945>`__: ENH:
spatial.transform.Rotation [GSOC2018]
* `#8950 <https://github.com/scipy/scipy/pull/8950>`__: csgraph Dijkstra
function description rewording
* `#8953 <https://github.com/scipy/scipy/pull/8953>`__: DOC, MAINT: HTTP ->
HTTPS, and other linkrot fixes
* `#8955 <https://github.com/scipy/scipy/pull/8955>`__: BUG: np.int64 in
scipy.fftpack.next_fast_len
* `#8958 <https://github.com/scipy/scipy/pull/8958>`__: MAINT: Add more
descriptive error message for phase one simplex.
* `#8962 <https://github.com/scipy/scipy/pull/8962>`__: BUG: sparse.linalg:
add missing conjugate to _ScaledLinearOperator.adjoint
* `#8963 <https://github.com/scipy/scipy/pull/8963>`__: BUG: sparse.linalg:
downgrade LinearOperator TypeError to warning
* `#8965 <https://github.com/scipy/scipy/pull/8965>`__: ENH: Wrapped RFP
format and RZ decomposition routines
* `#8969 <https://github.com/scipy/scipy/pull/8969>`__: MAINT: doc and code
fixes for optimize.newton
* `#8970 <https://github.com/scipy/scipy/pull/8970>`__: Added 'average'
keyword for welch/csd to enable median averaging
* `#8971 <https://github.com/scipy/scipy/pull/8971>`__: Better imresize
deprecation warning
* `#8972 <https://github.com/scipy/scipy/pull/8972>`__: MAINT: Switch
np.where(c) for np.nonzero(c)
* `#8975 <https://github.com/scipy/scipy/pull/8975>`__: MAINT: Fix
warning-based failures
* `#8979 <https://github.com/scipy/scipy/pull/8979>`__: DOC: fix
description of count_sort keyword of dendrogram
* `#8982 <https://github.com/scipy/scipy/pull/8982>`__: MAINT: optimize:
Fixed minor mistakes in test_linprog.py (#8978)
* `#8984 <https://github.com/scipy/scipy/pull/8984>`__: BUG: sparse.linalg:
ensure expm casts integer inputs to float
* `#8986 <https://github.com/scipy/scipy/pull/8986>`__: BUG:
optimize/slsqp: do not exit with convergence on steps where...
* `#8989 <https://github.com/scipy/scipy/pull/8989>`__: MAINT: use
collections.abc in basinhopping
* `#8990 <https://github.com/scipy/scipy/pull/8990>`__: ENH extend p-values
of anderson_ksamp in scipy.stats
* `#8991 <https://github.com/scipy/scipy/pull/8991>`__: ENH: Weighted kde
* `#8993 <https://github.com/scipy/scipy/pull/8993>`__: ENH:
spatial.transform.Rotation.random [GSOC 2018]
* `#8994 <https://github.com/scipy/scipy/pull/8994>`__: ENH:
spatial.transform.Slerp [GSOC 2018]
* `#8995 <https://github.com/scipy/scipy/pull/8995>`__: TST: time.time in
test
* `#9007 <https://github.com/scipy/scipy/pull/9007>`__: Fix typo in
fftpack.rst
* `#9013 <https://github.com/scipy/scipy/pull/9013>`__: Added correct
plotting code for two sided output from spectrogram
* `#9014 <https://github.com/scipy/scipy/pull/9014>`__: BUG:
differential_evolution with inf objective functions
* `#9017 <https://github.com/scipy/scipy/pull/9017>`__: BUG: fixed #8446
corner case for asformat(array|dense)
* `#9018 <https://github.com/scipy/scipy/pull/9018>`__: MAINT:
_lib/ccallback: remove unused code
* `#9021 <https://github.com/scipy/scipy/pull/9021>`__: BUG: Issue with
subspace_angles
* `#9022 <https://github.com/scipy/scipy/pull/9022>`__: DOC: Added "See
Also" section to lombscargle docstring
* `#9034 <https://github.com/scipy/scipy/pull/9034>`__: BUG: Fix tolerance
printing behavior, remove meaningless tol...
* `#9035 <https://github.com/scipy/scipy/pull/9035>`__: TST: improve
signal.bsplines test coverage
* `#9037 <https://github.com/scipy/scipy/pull/9037>`__: ENH: add a new init
method for k-means
* `#9039 <https://github.com/scipy/scipy/pull/9039>`__: DOC: Add examples
to fftpack.irfft docstrings
* `#9048 <https://github.com/scipy/scipy/pull/9048>`__: ENH:
scipy.sparse.random
* `#9050 <https://github.com/scipy/scipy/pull/9050>`__: BUG:
scipy.io.hb_write: fails for matrices not in csc format
* `#9051 <https://github.com/scipy/scipy/pull/9051>`__: MAINT: Fix slow
sparse.rand for k < mn/3 (#9036).
* `#9054 <https://github.com/scipy/scipy/pull/9054>`__: MAINT: spatial:
Explicitly initialize LAPACK output parameters.
* `#9055 <https://github.com/scipy/scipy/pull/9055>`__: DOC: Add examples
to scipy.special docstrings
* `#9056 <https://github.com/scipy/scipy/pull/9056>`__: ENH: Use one thread
in OpenBLAS
* `#9059 <https://github.com/scipy/scipy/pull/9059>`__: DOC: Update README
with link to Code of Conduct
* `#9060 <https://github.com/scipy/scipy/pull/9060>`__: BLD: remove support
for the Bento build system.
* `#9062 <https://github.com/scipy/scipy/pull/9062>`__: DOC add sections to
overview in scipy.stats
* `#9066 <https://github.com/scipy/scipy/pull/9066>`__: BUG: Correct
"remez" error message
* `#9069 <https://github.com/scipy/scipy/pull/9069>`__: DOC: update linalg
section of roadmap for LAPACK versions.
* `#9079 <https://github.com/scipy/scipy/pull/9079>`__: MAINT: add
spatial.transform to refguide check; complete some...
* `#9081 <https://github.com/scipy/scipy/pull/9081>`__: MAINT: Add warnings
if pivot value is close to tolerance in linprog(method='simplex')
* `#9084 <https://github.com/scipy/scipy/pull/9084>`__: BUG fix incorrect
p-values of kurtosistest in scipy.stats
* `#9095 <https://github.com/scipy/scipy/pull/9095>`__: DOC: add sections
to mstats overview in scipy.stats
* `#9096 <https://github.com/scipy/scipy/pull/9096>`__: BUG: Add test for
Stackoverflow example from issue 8174.
* `#9101 <https://github.com/scipy/scipy/pull/9101>`__: ENH: add Siegel
slopes (robust regression) to scipy.stats
* `#9105 <https://github.com/scipy/scipy/pull/9105>`__: allow
resample_poly() to output float32 for float32 inputs.
* `#9112 <https://github.com/scipy/scipy/pull/9112>`__: MAINT: optimize:
make trust-constr accept constraint dict (#9043)
* `#9118 <https://github.com/scipy/scipy/pull/9118>`__: Add doc entry to
cholesky_banded
* `#9120 <https://github.com/scipy/scipy/pull/9120>`__: eigsh documentation
parameters
* `#9125 <https://github.com/scipy/scipy/pull/9125>`__: interpolative:
correctly reconstruct full rank matrices
* `#9126 <https://github.com/scipy/scipy/pull/9126>`__: MAINT: Use warnings
for unexpected peak properties
* `#9129 <https://github.com/scipy/scipy/pull/9129>`__: BUG: Do not catch
and silence KeyboardInterrupt
* `#9131 <https://github.com/scipy/scipy/pull/9131>`__: DOC: Correct the
typo in scipy.optimize tutorial page
* `#9133 <https://github.com/scipy/scipy/pull/9133>`__: FIX: Avoid use of
bare except
* `#9134 <https://github.com/scipy/scipy/pull/9134>`__: DOC: Update of
'return_eigenvectors' description
* `#9137 <https://github.com/scipy/scipy/pull/9137>`__: DOC: typo fixes for
discrete Poisson tutorial
* `#9139 <https://github.com/scipy/scipy/pull/9139>`__: FIX: Doctest
failure in optimize tutorial
* `#9143 <https://github.com/scipy/scipy/pull/9143>`__: DOC: missing sigma
in Pearson r formula
* `#9145 <https://github.com/scipy/scipy/pull/9145>`__: MAINT: Refactor
linear programming solvers
* `#9149 <https://github.com/scipy/scipy/pull/9149>`__: FIX: Make
scipy.odr.ODR ifixx equal to its data.fix if given
* `#9156 <https://github.com/scipy/scipy/pull/9156>`__: DOC: special:
Mention the sigmoid function in the expit docstring.
* `#9160 <https://github.com/scipy/scipy/pull/9160>`__: Fixed a latex
delimiter error in levy()
* `#9170 <https://github.com/scipy/scipy/pull/9170>`__: DOC: correction /
update of docstrings of distributions in scipy.stats
* `#9171 <https://github.com/scipy/scipy/pull/9171>`__: better description
of the hierarchical clustering parameter
* `#9174 <https://github.com/scipy/scipy/pull/9174>`__: domain check for a
< b in stats.truncnorm
* `#9175 <https://github.com/scipy/scipy/pull/9175>`__: DOC: Minor grammar
fix
* `#9176 <https://github.com/scipy/scipy/pull/9176>`__: BUG:
CloughTocher2DInterpolator: fix miscalculation at neighborless...
* `#9177 <https://github.com/scipy/scipy/pull/9177>`__: BUILD: Document the
"clean" target in the doc/Makefile.
* `#9178 <https://github.com/scipy/scipy/pull/9178>`__: MAINT: make
refguide-check more robust for printed numpy arrays
* `#9186 <https://github.com/scipy/scipy/pull/9186>`__: MAINT: Remove
np.ediff1d occurence
* `#9188 <https://github.com/scipy/scipy/pull/9188>`__: DOC: correct typo
in extending ndimage with C
* `#9190 <https://github.com/scipy/scipy/pull/9190>`__: ENH: Support
specifying axes for fftconvolve
* `#9192 <https://github.com/scipy/scipy/pull/9192>`__: MAINT: optimize:
fixed @pv style suggestions from #9112
* `#9200 <https://github.com/scipy/scipy/pull/9200>`__: Fix
make_interp_spline(..., k=0 or 1, axis<0)
* `#9201 <https://github.com/scipy/scipy/pull/9201>`__: BUG:
sparse.linalg/gmres: use machine eps in breakdown check
* `#9204 <https://github.com/scipy/scipy/pull/9204>`__: MAINT: fix up
stats.spearmanr and match mstats.spearmanr with...
* `#9206 <https://github.com/scipy/scipy/pull/9206>`__: MAINT: include
benchmarks and dev files in sdist.
* `#9208 <https://github.com/scipy/scipy/pull/9208>`__: TST: signal: bump
bsplines test tolerance for complex data
* `#9210 <https://github.com/scipy/scipy/pull/9210>`__: TST: mark tests as
slow, fix missing random seed
* `#9211 <https://github.com/scipy/scipy/pull/9211>`__: ENH: add capability
to specify orders in pade func
* `#9217 <https://github.com/scipy/scipy/pull/9217>`__: MAINT: Include
``success`` and ``nit`` in OptimizeResult returned...
* `#9222 <https://github.com/scipy/scipy/pull/9222>`__: ENH: interpolate:
Use scipy.spatial.distance to speed-up Rbf
* `#9229 <https://github.com/scipy/scipy/pull/9229>`__: MNT: Fix Fourier
filter double case
* `#9233 <https://github.com/scipy/scipy/pull/9233>`__: BUG:
spatial/distance: fix pdist/cdist performance regression...
* `#9234 <https://github.com/scipy/scipy/pull/9234>`__: FIX: Proper
suppression
* `#9235 <https://github.com/scipy/scipy/pull/9235>`__: BENCH: rationalize
slow benchmarks + miscellaneous fixes
* `#9238 <https://github.com/scipy/scipy/pull/9238>`__: BENCH: limit number
of parameter combinations in spatial.*KDTree...
* `#9239 <https://github.com/scipy/scipy/pull/9239>`__: DOC: stats: Fix
LaTeX markup of a couple distribution PDFs.
* `#9241 <https://github.com/scipy/scipy/pull/9241>`__: ENH: Evaluate
plateau size during peak finding
* `#9242 <https://github.com/scipy/scipy/pull/9242>`__: ENH: stats:
Implement _ppf and _logpdf for crystalball, and do...
* `#9246 <https://github.com/scipy/scipy/pull/9246>`__: DOC: Properly
render versionadded directive in HTML documentation
* `#9255 <https://github.com/scipy/scipy/pull/9255>`__: DOC: mention
RootResults in optimization reference guide
* `#9260 <https://github.com/scipy/scipy/pull/9260>`__: TST: relax some
tolerances so tests pass with x87 math
* `#9264 <https://github.com/scipy/scipy/pull/9264>`__: TST Use
assert_raises "match" parameter instead of the "message"...
* `#9267 <https://github.com/scipy/scipy/pull/9267>`__: DOC: clarify
expect() return val when moment is inf/nan
* `#9272 <https://github.com/scipy/scipy/pull/9272>`__: DOC: Add
description of default bounds to linprog
* `#9277 <https://github.com/scipy/scipy/pull/9277>`__: MAINT:
sparse/linalg: make test deterministic
* `#9278 <https://github.com/scipy/scipy/pull/9278>`__: MAINT: interpolate:
pep8 cleanup in test_polyint
* `#9279 <https://github.com/scipy/scipy/pull/9279>`__: Fixed docstring for
resample
* `#9280 <https://github.com/scipy/scipy/pull/9280>`__: removed first check
for float in get_sum_dtype
* `#9281 <https://github.com/scipy/scipy/pull/9281>`__: BUG: only accept 1d
input for bartlett / levene in scipy.stats
* `#9282 <https://github.com/scipy/scipy/pull/9282>`__: MAINT: dense_output
and t_eval are mutually exclusive inputs
* `#9283 <https://github.com/scipy/scipy/pull/9283>`__: MAINT: add docs and
do some cleanups in interpolate.Rbf
* `#9288 <https://github.com/scipy/scipy/pull/9288>`__: Run
distance_transform_edt tests on all types
* `#9294 <https://github.com/scipy/scipy/pull/9294>`__: DOC: fix the
formula typo
* `#9298 <https://github.com/scipy/scipy/pull/9298>`__: MAINT:
optimize/trust-constr: restore .niter attribute for backward-compat
* `#9299 <https://github.com/scipy/scipy/pull/9299>`__: DOC: clarification
of default rvs method in scipy.stats
* `#9301 <https://github.com/scipy/scipy/pull/9301>`__: MAINT: removed
unused import sys
* `#9302 <https://github.com/scipy/scipy/pull/9302>`__: MAINT: removed
unused imports
* `#9303 <https://github.com/scipy/scipy/pull/9303>`__: DOC: signal: Refer
to fs instead of nyq in the firwin docstring.
* `#9305 <https://github.com/scipy/scipy/pull/9305>`__: ENH: Added
Yeo-Johnson power transformation
* `#9306 <https://github.com/scipy/scipy/pull/9306>`__: ENH - add dual
annealing
* `#9309 <https://github.com/scipy/scipy/pull/9309>`__: ENH add the
yulesimon distribution to scipy.stats
* `#9317 <https://github.com/scipy/scipy/pull/9317>`__: Nested SLSQP bug
fix.
* `#9320 <https://github.com/scipy/scipy/pull/9320>`__: MAINT: stats: avoid
underflow in stats.geom.ppf
* `#9326 <https://github.com/scipy/scipy/pull/9326>`__: Add example for
Rosenbrock function
* `#9332 <https://github.com/scipy/scipy/pull/9332>`__: Sort file lists
* `#9340 <https://github.com/scipy/scipy/pull/9340>`__: Fix typo in
find_peaks documentation
* `#9343 <https://github.com/scipy/scipy/pull/9343>`__: MAINT Use np.full
when possible
* `#9344 <https://github.com/scipy/scipy/pull/9344>`__: DOC: added examples
to docstring of dirichlet class
* `#9346 <https://github.com/scipy/scipy/pull/9346>`__: DOC: Fix import of
scipy.sparse.linalg in example (#9345)
* `#9350 <https://github.com/scipy/scipy/pull/9350>`__: Fix interpolate
read only
* `#9351 <https://github.com/scipy/scipy/pull/9351>`__: MAINT: special.erf:
use the x->-x symmetry
* `#9356 <https://github.com/scipy/scipy/pull/9356>`__: Fix documentation
typo
* `#9358 <https://github.com/scipy/scipy/pull/9358>`__: DOC: improve doc
for ksone and kstwobign in scipy.stats
* `#9362 <https://github.com/scipy/scipy/pull/9362>`__: DOC: Change
datatypes of A matrices in linprog
* `#9364 <https://github.com/scipy/scipy/pull/9364>`__: MAINT: Adds
implicit none to fftpack fortran sources
* `#9369 <https://github.com/scipy/scipy/pull/9369>`__: DOC: minor tweak to
CoC (updated NumFOCUS contact address).
* `#9373 <https://github.com/scipy/scipy/pull/9373>`__: Fix exception if
python is called with -OO option
* `#9374 <https://github.com/scipy/scipy/pull/9374>`__: FIX: AIX
compilation issue with NAN and INFINITY
* `#9376 <https://github.com/scipy/scipy/pull/9376>`__: COBLYA -> COBYLA in
docs
* `#9377 <https://github.com/scipy/scipy/pull/9377>`__: DOC: Add examples
integrate: fixed_quad and quadrature
* `#9379 <https://github.com/scipy/scipy/pull/9379>`__: MAINT: TST: Make
tests NumPy 1.8 compatible
* `#9385 <https://github.com/scipy/scipy/pull/9385>`__: CI: On Travis
matrix "OPTIMIZE=-OO" flag ignored
* `#9387 <https://github.com/scipy/scipy/pull/9387>`__: Fix defaut value
for 'mode' in 'ndimage.shift' in the doc
* `#9392 <https://github.com/scipy/scipy/pull/9392>`__: BUG: rank has to be
integer in rank_filter: fixed issue 9388
* `#9399 <https://github.com/scipy/scipy/pull/9399>`__: DOC: Misc. typos
* `#9400 <https://github.com/scipy/scipy/pull/9400>`__: TST: stats: Fix the
expected r-value of a linregress test.
* `#9405 <https://github.com/scipy/scipy/pull/9405>`__: BUG: np.hstack does
not accept generator expressions
* `#9408 <https://github.com/scipy/scipy/pull/9408>`__: ENH: linalg:
Shorter ill-conditioned warning message
* `#9418 <https://github.com/scipy/scipy/pull/9418>`__: DOC: Fix ndimage
docstrings and reduce doc build warnings
* `#9421 <https://github.com/scipy/scipy/pull/9421>`__: DOC: Add missing
docstring examples in scipy.spatial
* `#9422 <https://github.com/scipy/scipy/pull/9422>`__: DOC: Add an example
to integrate.newton_cotes
* `#9427 <https://github.com/scipy/scipy/pull/9427>`__: BUG: Fixed defect
with maxiter #9419 in dual annealing
* `#9431 <https://github.com/scipy/scipy/pull/9431>`__: BENCH: Add dual
annealing to scipy benchmark (see #9415)
* `#9435 <https://github.com/scipy/scipy/pull/9435>`__: DOC: Add docstring
examples for stats.binom_test
* `#9443 <https://github.com/scipy/scipy/pull/9443>`__: DOC: Fix the order
of indices in optimize tutorial
* `#9444 <https://github.com/scipy/scipy/pull/9444>`__: MAINT: interpolate:
use operator.index for checking/coercing...
* `#9445 <https://github.com/scipy/scipy/pull/9445>`__: DOC: Added missing
example to stats.mstats.kruskal
* `#9446 <https://github.com/scipy/scipy/pull/9446>`__: DOC: Add note about
version changed for jaccard distance
* `#9447 <https://github.com/scipy/scipy/pull/9447>`__: BLD: version-script
handling in setup.py
* `#9448 <https://github.com/scipy/scipy/pull/9448>`__: TST: skip a
problematic linalg test
* `#9449 <https://github.com/scipy/scipy/pull/9449>`__: TST: fix missing
seed in lobpcg test.
* `#9456 <https://github.com/scipy/scipy/pull/9456>`__: TST:
test_eigs_consistency() now sorts output

Checksums
=========

MD5
~~~

47d309402d2e5574be8fa261fadfaf58
scipy-1.2.0rc1-cp27-cp27m-macosx_10_6_intel.macosx_10_9_intel.macosx_10_9_x86_64.macosx_10_10_intel.macosx_10_10_x86_64.whl
911dfde5be66403c07c60e19aa631dc2
scipy-1.2.0rc1-cp27-cp27m-manylinux1_i686.whl
a693189336365595b42b0d93f825b826
scipy-1.2.0rc1-cp27-cp27m-manylinux1_x86_64.whl
ec5abd33480761ed9701f7fd2274fc47  scipy-1.2.0rc1-cp27-cp27m-win32.whl
bc3d40311f057b12f8fea97166ef8112  scipy-1.2.0rc1-cp27-cp27m-win_amd64.whl
33848233e6438b1ff9183d8a4794daed
scipy-1.2.0rc1-cp27-cp27mu-manylinux1_i686.whl
c2cb1166ce0071e1fe42ed1c3e60b75e
scipy-1.2.0rc1-cp27-cp27mu-manylinux1_x86_64.whl
a04b5a758f555e05e6431b8a1f035888
scipy-1.2.0rc1-cp34-cp34m-macosx_10_6_intel.macosx_10_9_intel.macosx_10_9_x86_64.macosx_10_10_intel.macosx_10_10_x86_64.whl
ad511246d0742cf0669fedf292cc01bb
scipy-1.2.0rc1-cp34-cp34m-manylinux1_i686.whl
15aa08ef43a6c5cb320bc015f01087ad
scipy-1.2.0rc1-cp34-cp34m-manylinux1_x86_64.whl
86a59b81a3e6894d9054139311a2c51f  scipy-1.2.0rc1-cp34-cp34m-win32.whl
8b6e33253579916ea873c45989ee5bea  scipy-1.2.0rc1-cp34-cp34m-win_amd64.whl
db7a4de02828471bf9f800814ff68627
scipy-1.2.0rc1-cp35-cp35m-macosx_10_6_intel.macosx_10_9_intel.macosx_10_9_x86_64.macosx_10_10_intel.macosx_10_10_x86_64.whl
c296b6270d29d3ec6dafddf8ceae67fb
scipy-1.2.0rc1-cp35-cp35m-manylinux1_i686.whl
3ba7a825f61822128d960fa728010e51
scipy-1.2.0rc1-cp35-cp35m-manylinux1_x86_64.whl
c083c8287da110b707d181f6638ce122  scipy-1.2.0rc1-cp35-cp35m-win32.whl
2242eac92681085258535ed96bd040d7  scipy-1.2.0rc1-cp35-cp35m-win_amd64.whl
d5c238903c00e91a40d56023f1a27ed4
scipy-1.2.0rc1-cp36-cp36m-macosx_10_6_intel.macosx_10_9_intel.macosx_10_9_x86_64.macosx_10_10_intel.macosx_10_10_x86_64.whl
e8ab7487a9a53b86a6510772c45af787
scipy-1.2.0rc1-cp36-cp36m-manylinux1_i686.whl
9991b5958d736488bef638eea463945d
scipy-1.2.0rc1-cp36-cp36m-manylinux1_x86_64.whl
9c108a9d7e967b8c9a5e5143b1a15b40  scipy-1.2.0rc1-cp36-cp36m-win32.whl
54d63041f0315d341d9ffb028a98e767  scipy-1.2.0rc1-cp36-cp36m-win_amd64.whl
534276c864ab3139811561c022608cc3
scipy-1.2.0rc1-cp37-cp37m-macosx_10_6_intel.macosx_10_9_intel.macosx_10_9_x86_64.macosx_10_10_intel.macosx_10_10_x86_64.whl
4bd26c179a10891087bd81a658573683
scipy-1.2.0rc1-cp37-cp37m-manylinux1_i686.whl
1a3170b4f1df42f28efbe197e54eb9a3
scipy-1.2.0rc1-cp37-cp37m-manylinux1_x86_64.whl
72b89f7e2c1d13dc8dbb21600fb184da  scipy-1.2.0rc1-cp37-cp37m-win32.whl
433ef294c4a015da0a6e0f063289a658  scipy-1.2.0rc1-cp37-cp37m-win_amd64.whl
83abb1befce326916e0435d428b36e62  scipy-1.2.0rc1.tar.gz
21b7570fb577543807feb4b4c1fdad8a  scipy-1.2.0rc1.tar.xz
11221dc23e0d3b316b5564f2f435aaf1  scipy-1.2.0rc1.zip

SHA256
~~~~~~

a3282027743e89d5fcb6cd7a9e4ecdbbde61bf8126fd19683e97b69f5f6a2163
scipy-1.2.0rc1-cp27-cp27m-macosx_10_6_intel.macosx_10_9_intel.macosx_10_9_x86_64.macosx_10_10_intel.macosx_10_10_x86_64.whl
8b8841e03620d4d704f4efd3daed575e23a6272ae9c9be2c4cc751febee984f7
scipy-1.2.0rc1-cp27-cp27m-manylinux1_i686.whl
254ea2f7f3c5afef9f02c0192d2dbd8f5336c1b1c53efae7ef64a8880cccc299
scipy-1.2.0rc1-cp27-cp27m-manylinux1_x86_64.whl
d449832419df3e37a3942778b46c140fd61d1e4f38f9e34bed278a77aacdbd31
scipy-1.2.0rc1-cp27-cp27m-win32.whl
e57f3e3eaa88cd7bc93466122fca48c36863e03aeb24f9d570a2e6b2ea3cbf92
scipy-1.2.0rc1-cp27-cp27m-win_amd64.whl
00dcb606101fa10951ee235af69dbff55d999b01c9bb2bc0f64df5fc3aff4eb6
scipy-1.2.0rc1-cp27-cp27mu-manylinux1_i686.whl
38857eb49f7e38d3ec079772225a79235d0bd847e24d2fa8c9a9fa70ee69f0a2
scipy-1.2.0rc1-cp27-cp27mu-manylinux1_x86_64.whl
9b9950278f36c4e7779148cc9a719d41a305024c70005f994412845172ade346
scipy-1.2.0rc1-cp34-cp34m-macosx_10_6_intel.macosx_10_9_intel.macosx_10_9_x86_64.macosx_10_10_intel.macosx_10_10_x86_64.whl
c8a2fc15c64f1b162b02f86d28153b65f64720ca64e832bcf5bfae060a507949
scipy-1.2.0rc1-cp34-cp34m-manylinux1_i686.whl
4e205e0e5e88fe2105c594a706ac9fa558fe7a4daa2bef2c86b343ab507d8bd6
scipy-1.2.0rc1-cp34-cp34m-manylinux1_x86_64.whl
b04fb8ba796198d83614138d7f96e88791d83f5f4e31e628cce51fda1a092f66
scipy-1.2.0rc1-cp34-cp34m-win32.whl
c34e4ce9a8c62c85f322d7897a257a839a4bb7fd94e08701044e9b1cc1bb15a6
scipy-1.2.0rc1-cp34-cp34m-win_amd64.whl
cb7f4c289c06514c6e224263868a98d5e6aa6e8a90f2304b16ff276aa96030ce
scipy-1.2.0rc1-cp35-cp35m-macosx_10_6_intel.macosx_10_9_intel.macosx_10_9_x86_64.macosx_10_10_intel.macosx_10_10_x86_64.whl
dcaeed34e7e965e935ba252fd3290c04cafb7d84f6f44c9c08269504aa2e0a05
scipy-1.2.0rc1-cp35-cp35m-manylinux1_i686.whl
13b234b4fbda1368474a667fc29e8365b622c4c607ed490f00643625c8f50cca
scipy-1.2.0rc1-cp35-cp35m-manylinux1_x86_64.whl
073560d6598b97b8f214a31856a1d43c96701b46c5cc9805789f962c3b8a0e97
scipy-1.2.0rc1-cp35-cp35m-win32.whl
600cf596b83f185dca3455d03ca802de0b4de98cb4c8d041226089d78d04c2bc
scipy-1.2.0rc1-cp35-cp35m-win_amd64.whl
77b7ed8e13f03a843d2f11bd140f224abe62fb56656f35348578e3838ff4d528
scipy-1.2.0rc1-cp36-cp36m-macosx_10_6_intel.macosx_10_9_intel.macosx_10_9_x86_64.macosx_10_10_intel.macosx_10_10_x86_64.whl
98786b516d1955592f921c2b36505840ff921c9e82f5c28a6c2a580fb3e96af1
scipy-1.2.0rc1-cp36-cp36m-manylinux1_i686.whl
8f62fc6ac6b842d7c3c499269150ec81d26a0dea5ce818aef3b0e9e14e53c5c7
scipy-1.2.0rc1-cp36-cp36m-manylinux1_x86_64.whl
213cfc35ec2fbf86c5957d1ada99a1fe1eacccdb498d4790089af9cf50cadab4
scipy-1.2.0rc1-cp36-cp36m-win32.whl
287e46f87399eb9897c726c6d358c893f6c769300599bc5da95cbf3397a00aa7
scipy-1.2.0rc1-cp36-cp36m-win_amd64.whl
7116d193da4baca6f7cd1cd7810548fa03ed4a06e05f325978b2f0018b05ead9
scipy-1.2.0rc1-cp37-cp37m-macosx_10_6_intel.macosx_10_9_intel.macosx_10_9_x86_64.macosx_10_10_intel.macosx_10_10_x86_64.whl
9cd3fe767225b0dcfcc9f09ed18288b51c29b35ca16dc4e5a6f3587b4ac7d7b5
scipy-1.2.0rc1-cp37-cp37m-manylinux1_i686.whl
4024aea96a01ca05af701f93fd227a7f646258116903899a75b1f4a1f0134bf8
scipy-1.2.0rc1-cp37-cp37m-manylinux1_x86_64.whl
eccd0e6d321585b6e2ee18fa0ee1db4fe042e52fb4ae8e28a3d223df6bcd6a8e
scipy-1.2.0rc1-cp37-cp37m-win32.whl
96f6c69b2d8f63dad5612385521d0cf8b62f665a62f56b6b3d3fb7042a63c34c
scipy-1.2.0rc1-cp37-cp37m-win_amd64.whl
ca9ba36bb271dcb4273e330c0bd418a1c3566ff76248dd634efecbe0e9c1721c
scipy-1.2.0rc1.tar.gz
b8fe757f377b43c733a0ba235b990fb4b3722bd6f5930a26359d16752e94560c
scipy-1.2.0rc1.tar.xz
da1980e7e037e2275821d7611a91eadafdc157b096f36f41d05cc0ea4ae539bc
scipy-1.2.0rc1.zip
-----BEGIN PGP SIGNATURE-----

iQIzBAEBCAAdFiEEgfAqrOuERrV4fwr6XCLdIy4FKasFAlv8LB8ACgkQXCLdIy4F
KauhWw//cFUboxtuNZeEh98Pn2mfVIqRF8hc6VK7jeXU64qiAdOnbBRKXfoXWpbt
8fEckTFVVt2CQzIEZju429P0eWv8psfvn90cNnj5VHz7h46AH5gQee/cfP1kNWGO
mxcYczqZN8+ldDnww4hfk0R+3IrpfmQbVSMEqBF84uottfgUjUzPtcFHJmmmcOF2
7XWhAP5IWMctcwq3r3WAa8yLUmlEIB3Fs9mPPr1We7e0QiXnEueUrXVZF9Oz3UpJ
UFiTwX5nxnNdxZJwtLbBXhxJUbvPnooby0x1NwxFG6AwrWaPG3my0e9kIzpwqfsf
0B7Pt/xPEHWIizaqMyfUiaS7FZOUfvq6xs5aCyQeP9Wtnwherm6SQ/Qs+LFSb8E2
9cgyFmaBeslrTMd3f1YMcb6d1hmNSjbBmWccZEXCqjC9UClxbAQDHDcdmTl+4vp7
xATCxWkYVP9lOLH5TtIC1Bo0AZXiOxI3JnojQS65JH0w4unh1IJJkENurzCATh4/
vnyddy0x2e1ok5bX8fpVFrCc8lSdO8aDY5cYEU55XZhkyroet3IaOYqwPzPQPwoJ
QgYEsqxa2Urn0faV5uuDFTscnCAHK0EVXhr3yyL0YoIGOPffM8JaswEAUaFwLrq0
KEbOavdImtSwqL9v6x5tDyYXh/mHQ70GGtPS68iSc1NexRrwLAw=
=F/+Z
-----END PGP SIGNATURE-----
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20181126/3579b5e9/attachment-0001.html>

From charlesr.harris at gmail.com  Mon Nov 26 16:16:04 2018
From: charlesr.harris at gmail.com (Charles R Harris)
Date: Mon, 26 Nov 2018 14:16:04 -0700
Subject: [Numpy-discussion] ANN: SciPy 1.2.0rc1 -- please test
In-Reply-To: <CAHPuU_ZGypUKxVAGL5YXrOu1jiOzdGdssshmQ7751nMJL+GBYA@mail.gmail.com>
References: <CAHPuU_ZGypUKxVAGL5YXrOu1jiOzdGdssshmQ7751nMJL+GBYA@mail.gmail.com>
Message-ID: <CAB6mnxK96Tkp3_9yeOejOFiyj5c3PGQv-kZyt_5qAeybqbTvZQ@mail.gmail.com>

Congratulations on your first release. May there be many more :)

Chuck

On Mon, Nov 26, 2018 at 11:48 AM Tyler Reddy <tyler.je.reddy at gmail.com>
wrote:

> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA256
>
> Hi all,
>
> On behalf of the SciPy development team I'm pleased to announce
> the release candidate SciPy 1.2.0rc1. Please help us test out this
> release candidate -- the 1.2.x series will be an LTS release and the
> last to support Python 2.7.
>
> Sources and binary wheels can be found at:
> https://pypi.python.org/pypi/scipy
> and at:
> https://github.com/scipy/scipy/releases/tag/v1.2.0rc1
>
> To install the release candidate with pip:
>
> pip install scipy==1.2.0rc1
>
>
> ==========================
> SciPy 1.2.0 Release Notes
> ==========================
>
> Note: Scipy 1.2.0 is not released yet!
>
> SciPy 1.2.0 is the culmination of 6 months of hard work. It contains
> many new features, numerous bug-fixes, improved test coverage and better
> documentation. There have been a number of deprecations and API changes
> in this release, which are documented below. All users are encouraged to
> upgrade to this release, as there are a large number of bug-fixes and
> optimizations. Before upgrading, we recommend that users check that
> their own code does not use deprecated SciPy functionality (to do so,
> run your code with ``python -Wd`` and check for ``DeprecationWarning`` s).
> Our development attention will now shift to bug-fix releases on the
> 1.2.x branch, and on adding new features on the master branch.
>
> This release requires Python 2.7 or 3.4+ and NumPy 1.8.2 or greater.
>
> Note: This will be the last SciPy release to support Python 2.7.
>           Consequently, the 1.2.x series will be a long term support (LTS)
>           release; we will backport bug fixes until 1 Jan 2020.
>
> For running on PyPy, PyPy3 6.0+ and NumPy 1.15.0 are required.
>
> Highlights of this release
> ----------------------------
>
> - - 1-D root finding improvements with a new solver, ``toms748``, and a new
>   unified interface, ``root_scalar``
> - - New ``dual_annealing`` optimization method that combines stochastic and
>   local deterministic searching
> - - A new optimization algorithm, ``shgo`` (simplicial homology
>   global optimization) for derivative free optimization problems
> - - A new category of quaternion-based transformations are available in
> `scipy.spatial.transform`
>
> New features
> ============
>
> `scipy.ndimage` improvements
> - --------------------------------
>
> Proper spline coefficient calculations have been added for the ``mirror``,
> ``wrap``, and ``reflect`` modes of `scipy.ndimage.rotate`
>
> `scipy.fftpack` improvements
> - --------------------------------
>
> DCT-IV, DST-IV, DCT-I, and DST-I orthonormalization are now supported in
> `scipy.fftpack`.
>
> `scipy.interpolate` improvements
> - --------------------------------
>
> `scipy.interpolate.pade` now accepts a new argument for the order of the
> numerator
>
> `scipy.cluster` improvements
> - ----------------------------
>
> `scipy.cluster.vq.kmeans2` gained a new initialization method, kmeans++.
>
> `scipy.special` improvements
> - ----------------------------
>
> The function ``softmax`` was added to `scipy.special`.
>
> `scipy.optimize` improvements
> - -----------------------------
>
> The one-dimensional nonlinear solvers have been given a unified interface
> `scipy.optimize.root_scalar`, similar to the `scipy.optimize.root`
> interface
> for multi-dimensional solvers. ``scipy.optimize.root_scalar(f, bracket=[a
> ,b],
> method="brenth")`` is equivalent to ``scipy.optimize.brenth(f, a ,b)``.
> If no
> ``method`` is specified, an appropriate one will be selected based upon the
> bracket and the number of derivatives available.
>
> The so-called Algorithm 748 of Alefeld, Potra and Shi for root-finding
> within
> an enclosing interval has been added as `scipy.optimize.toms748`. This
> provides
> guaranteed convergence to a root with convergence rate per function
> evaluation
> of approximately 1.65 (for sufficiently well-behaved functions.)
>
> ``differential_evolution`` now has the ``updating`` and ``workers``
> keywords.
> The first chooses between continuous updating of the best solution vector
> (the
> default), or once per generation. Continuous updating can lead to faster
> convergence. The ``workers`` keyword accepts an ``int`` or map-like
> callable,
> and parallelises the solver (having the side effect of updating once per
> generation). Supplying an ``int`` evaluates the trial solutions in N
> parallel
> parts. Supplying a map-like callable allows other parallelisation
> approaches
> (such as ``mpi4py``, or ``joblib``) to be used.
>
> ``dual_annealing`` (and ``shgo`` below) is a powerful new general purpose
> global optizimation (GO) algorithm. ``dual_annealing`` uses two annealing
> processes to accelerate the convergence towards the global minimum of an
> objective mathematical function. The first annealing process controls the
> stochastic Markov chain searching and the second annealing process
> controls the
> deterministic minimization. So, dual annealing is a hybrid method that
> takes
> advantage of stochastic and local deterministic searching in an efficient
> way.
>
> ``shgo`` (simplicial homology global optimization) is a similar algorithm
> appropriate for solving black box and derivative free optimization (DFO)
> problems. The algorithm generally converges to the global solution in
> finite
> time. The convergence holds for non-linear inequality and
> equality constraints. In addition to returning a global minimum, the
> algorithm also returns any other global and local minima found after every
> iteration. This makes it useful for exploring the solutions in a domain.
>
> `scipy.optimize.newton` can now accept a scalar or an array
>
> ``MINPACK`` usage is now thread-safe, such that ``MINPACK`` + callbacks may
> be used on multiple threads.
>
> `scipy.signal` improvements
> - ---------------------------
>
> Digital filter design functions now include a parameter to specify the
> sampling
> rate. Previously, digital filters could only be specified using normalized
> frequency, but different functions used different scales (e.g. 0 to 1 for
> ``butter`` vs 0 to ? for ``freqz``), leading to errors and confusion.  With
> the ``fs`` parameter, ordinary frequencies can now be entered directly into
> functions, with the normalization handled internally.
>
> ``find_peaks`` and related functions no longer raise an exception if the
> properties of a peak have unexpected values (e.g. a prominence of 0). A
> ``PeakPropertyWarning`` is given instead.
>
> The new keyword argument ``plateau_size`` was added to ``find_peaks``.
> ``plateau_size`` may be used to select peaks based on the length of the
> flat top of a peak.
>
> ``welch()`` and ``csd()`` methods in `scipy.signal` now support calculation
> of a median average PSD, using ``average='mean'`` keyword
>
> `scipy.sparse` improvements
> - ---------------------------
>
> The `scipy.sparse.bsr_matrix.tocsr` method is now implemented directly
> instead
> of converting via COO format, and the `scipy.sparse.bsr_matrix.tocsc`
> method
> is now also routed via CSR conversion instead of COO. The efficiency of
> both
> conversions is now improved.
>
> The issue where SuperLU or UMFPACK solvers crashed on matrices with
> non-canonical format in `scipy.sparse.linalg` was fixed. The solver wrapper
> canonicalizes the matrix if necessary before calling the SuperLU or UMFPACK
> solver.
>
> The ``largest`` option of `scipy.sparse.linalg.lobpcg()` was fixed to have
> a correct (and expected) behavior. The order of the eigenvalues was made
> consistent with the ARPACK solver (``eigs()``), i.e. ascending for the
> smallest eigenvalues, and descending for the largest eigenvalues.
>
> The `scipy.sparse.random` function is now faster and also supports integer
> and
> complex values by passing the appropriate value to the ``dtype`` argument.
>
> `scipy.spatial` improvements
> - ----------------------------
>
> The function `scipy.spatial.distance.jaccard` was modified to return 0
> instead
> of ``np.nan`` when two all-zero vectors are compared.
>
> Support for the Jensen Shannon distance, the square-root of the
> divergence, has
> been added under `scipy.spatial.distance.jensenshannon`
>
> An optional keyword was added to the function
> `scipy.spatial.cKDTree.query_ball_point()` to sort or not sort the returned
> indices. Not sorting the indices can speed up calls.
>
> A new category of quaternion-based transformations are available in
> `scipy.spatial.transform`, including spherical linear interpolation of
> rotations (``Slerp``), conversions to and from quaternions, Euler angles,
> and general rotation and inversion capabilities
> (`spatial.transform.Rotation`), and uniform random sampling of 3D
> rotations (`spatial.transform.Rotation.random`).
>
> `scipy.stats` improvements
> - --------------------------
>
> The Yeo-Johnson power transformation is now supported (``yeojohnson``,
> ``yeojohnson_llf``, ``yeojohnson_normmax``, ``yeojohnson_normplot``).
> Unlike
> the Box-Cox transformation, the Yeo-Johnson transformation can accept
> negative
> values.
>
> Added a general method to sample random variates based on the density
> only, in
> the new function ``rvs_ratio_uniforms``.
>
> The Yule-Simon distribution (``yulesimon``) was added -- this is a new
> discrete probability distribution.
>
> ``stats`` and ``mstats`` now have access to a new regression method,
> ``siegelslopes``, a robust linear regression algorithm
>
> `scipy.stats.gaussian_kde` now has the ability to deal with weighted
> samples,
> and should have a modest improvement in performance
>
> Levy Stable Parameter Estimation, PDF, and CDF calculations are now
> supported
> for `scipy.stats.levy_stable`.
>
> The Brunner-Munzel test is now available as ``brunnermunzel`` in ``stats``
> and ``mstats``
>
> `scipy.linalg` improvements
> - --------------------------
>
> `scipy.linalg.lapack` now exposes the LAPACK routines using the Rectangular
> Full Packed storage (RFP) for upper triangular, lower triangular,
> symmetric,
> or Hermitian matrices; the upper trapezoidal fat matrix RZ decomposition
> routines are now available as well.
>
> Deprecated features
> ===================
> The functions ``hyp2f0``, ``hyp1f2`` and ``hyp3f0`` in ``scipy.special``
> have
> been deprecated.
>
>
> Backwards incompatible changes
> ==============================
>
> LAPACK version 3.4.0 or later is now required. Building with
> Apple Accelerate is no longer supported.
>
> The function ``scipy.linalg.subspace_angles(A, B)`` now gives correct
> results for all angles. Before this, the function only returned
> correct values for those angles which were greater than pi/4.
>
> Support for the Bento build system has been removed. Bento has not been
> maintained for several years, and did not have good Python 3 or wheel
> support,
> hence it was time to remove it.
>
> The required signature of `scipy.optimize.lingprog` ``method=simplex``
> callback function has changed. Before iteration begins, the simplex solver
> first converts the problem into a standard form that does not, in general,
> have the same variables or constraints
> as the problem defined by the user. Previously, the simplex solver would
> pass a
> user-specified callback function several separate arguments, such as the
> current solution vector ``xk``, corresponding to this standard form
> problem.
> Unfortunately, the relationship between the standard form problem and the
> user-defined problem was not documented, limiting the utility of the
> information passed to the callback function.
>
> In addition to numerous bug fix changes, the simplex solver now passes a
> user-specified callback function a single ``OptimizeResult`` object
> containing
> information that corresponds directly to the user-defined problem. In
> future
> releases, this ``OptimizeResult`` object may be expanded to include
> additional
> information, such as variables corresponding to the standard-form problem
> and
> information concerning the relationship between the standard-form and
> user-defined problems.
>
> The implementation of `scipy.sparse.random` has changed, and this affects
> the
> numerical values returned for both ``sparse.random`` and ``sparse.rand``
> for
> some matrix shapes and a given seed.
>
> `scipy.optimize.newton` will no longer use Halley's method in cases where
> it
> negatively impacts convergence
>
> Other changes
> =============
>
>
> Authors
> =======
>
> * @endolith
> * @luzpaz
> * Hameer Abbasi +
> * akahard2dj +
> * Anton Akhmerov
> * Joseph Albert
> * alexthomas93 +
> * ashish +
> * atpage +
> * Blair Azzopardi +
> * Yoshiki V?zquez Baeza
> * Bence Bagi +
> * Christoph Baumgarten
> * Lucas Bellomo +
> * BH4 +
> * Aditya Bharti
> * Max Bolingbroke
> * Fran?ois Boulogne
> * Ward Bradt +
> * Matthew Brett
> * Evgeni Burovski
> * Rafa? Byczek +
> * Alfredo Canziani +
> * CJ Carey
> * Luc?a Cheung +
> * Poom Chiarawongse +
> * Jeanne Choo +
> * Robert Cimrman
> * Graham Clenaghan +
> * cynthia-rempel +
> * Johannes Damp +
> * Jaime Fernandez del Rio
> * Dowon +
> * emmi474 +
> * Stefan Endres +
> * Thomas Etherington +
> * Alex Fikl +
> * fo40225 +
> * Joseph Fox-Rabinovitz
> * Lars G
> * Abhinav Gautam +
> * Stiaan Gerber +
> * C.A.M. Gerlach +
> * Ralf Gommers
> * Todd Goodall
> * Lars Grueter +
> * Sylvain Gubian +
> * Matt Haberland
> * David Hagen
> * Will Handley +
> * Charles Harris
> * Ian Henriksen
> * Thomas Hisch +
> * Theodore Hu
> * Michael Hudson-Doyle +
> * Nicolas Hug +
> * jakirkham +
> * Jakob Jakobson +
> * James +
> * Jan Schl?ter
> * jeanpauphilet +
> * josephmernst +
> * Kai +
> * Kai-Striega +
> * kalash04 +
> * Toshiki Kataoka +
> * Konrad0 +
> * Tom Krauss +
> * Johannes Kulick
> * Lars Gr?ter +
> * Eric Larson
> * Denis Laxalde
> * Will Lee +
> * Katrin Leinweber +
> * Yin Li +
> * P. L. Lim +
> * Jesse Livezey +
> * Duncan Macleod +
> * MatthewFlamm +
> * Nikolay Mayorov
> * Mike McClurg +
> * Christian Meyer +
> * Mark Mikofski
> * Naoto Mizuno +
> * mohmmadd +
> * Nathan Musoke
> * Anju Geetha Nair +
> * Andrew Nelson
> * Ayappan P +
> * Nick Papior
> * Haesun Park +
> * Ronny Pfannschmidt +
> * pijyoi +
> * Ilhan Polat
> * Anthony Polloreno +
> * Ted Pudlik
> * puenka
> * Eric Quintero
> * Pradeep Reddy Raamana +
> * Vyas Ramasubramani +
> * Ramon Vi?as +
> * Tyler Reddy
> * Joscha Reimer
> * Antonio H Ribeiro
> * richardjgowers +
> * Rob +
> * robbystk +
> * Lucas Roberts +
> * rohan +
> * Joaquin Derrac Rus +
> * Josua Sassen +
> * Bruce Sharpe +
> * Max Shinn +
> * Scott Sievert
> * Sourav Singh
> * Strahinja Luki? +
> * Kai Striega +
> * Shinya SUZUKI +
> * Mike Toews +
> * Piotr Uchwat
> * Miguel de Val-Borro +
> * Nicky van Foreest
> * Paul van Mulbregt
> * Gael Varoquaux
> * Pauli Virtanen
> * Stefan van der Walt
> * Warren Weckesser
> * Joshua Wharton +
> * Bernhard M. Wiedemann +
> * Eric Wieser
> * Josh Wilson
> * Tony Xiang +
> * Roman Yurchak +
> * Roy Zywina +
>
> A total of 137 people contributed to this release.
> People with a "+" by their names contributed a patch for the first time.
> This list of names is automatically generated, and may not be fully
> complete.
>
> Issues closed for 1.2.0
> - -----------------------
>
> * `#1240 <https://github.com/scipy/scipy/issues/1240>`__: Allowing
> multithreaded use of minpack through scipy.optimize...
> * `#1432 <https://github.com/scipy/scipy/issues/1432>`__:
> scipy.stats.mode extremely slow (Trac #905)
> * `#3372 <https://github.com/scipy/scipy/issues/3372>`__: Please add
> Sphinx search field to online scipy html docs
> * `#3678 <https://github.com/scipy/scipy/issues/3678>`__:
> _clough_tocher_2d_single direction between centroids
> * `#4174 <https://github.com/scipy/scipy/issues/4174>`__: lobpcg
> "largest" option invalid?
> * `#5493 <https://github.com/scipy/scipy/issues/5493>`__: anderson_ksamp
> p-values>1
> * `#5743 <https://github.com/scipy/scipy/issues/5743>`__: slsqp fails to
> detect infeasible problem
> * `#6139 <https://github.com/scipy/scipy/issues/6139>`__:
> scipy.optimize.linprog failed to find a feasible starting point...
> * `#6358 <https://github.com/scipy/scipy/issues/6358>`__: stats:
> docstring for `vonmises_line` points to `vonmises_line`...
> * `#6498 <https://github.com/scipy/scipy/issues/6498>`__: runtests.py is
> missing in pypi distfile
> * `#7426 <https://github.com/scipy/scipy/issues/7426>`__:
> scipy.stats.ksone(n).pdf(x) returns nan for positive values of...
> * `#7455 <https://github.com/scipy/scipy/issues/7455>`__:
> scipy.stats.ksone.pdf(2,x) return incorrect values for x near...
> * `#7456 <https://github.com/scipy/scipy/issues/7456>`__:
> scipy.special.smirnov and scipy.special.smirnovi have accuracy...
> * `#7492 <https://github.com/scipy/scipy/issues/7492>`__:
> scipy.special.kolmogorov(x)/kolmogi(p) inefficient, inaccurate...
> * `#7914 <https://github.com/scipy/scipy/issues/7914>`__: TravisCI not
> failing when it should for -OO run
> * `#8064 <https://github.com/scipy/scipy/issues/8064>`__: linalg.solve
> test crashes on Windows
> * `#8212 <https://github.com/scipy/scipy/issues/8212>`__: LAPACK
> Rectangular Full Packed routines
> * `#8256 <https://github.com/scipy/scipy/issues/8256>`__:
> differential_evolution bug converges to wrong results in complex...
> * `#8443 <https://github.com/scipy/scipy/issues/8443>`__: Deprecate
> `hyp2f0`, `hyp1f2`, and `hyp3f0`?
> * `#8452 <https://github.com/scipy/scipy/issues/8452>`__: DOC: ARPACK
> tutorial has two conflicting equations
> * `#8680 <https://github.com/scipy/scipy/issues/8680>`__: scipy fails
> compilation when building from source
> * `#8686 <https://github.com/scipy/scipy/issues/8686>`__: Division by
> zero in _trustregion.py when x0 is exactly equal...
> * `#8700 <https://github.com/scipy/scipy/issues/8700>`__: _MINPACK_LOCK
> not held when calling into minpack from least_squares
> * `#8786 <https://github.com/scipy/scipy/issues/8786>`__: erroneous
> moment values for t-distribution
> * `#8791 <https://github.com/scipy/scipy/issues/8791>`__: Checking COLA
> condition in istft should be optional (or omitted)
> * `#8843 <https://github.com/scipy/scipy/issues/8843>`__: imresize cannot
> be deprecated just yet
> * `#8844 <https://github.com/scipy/scipy/issues/8844>`__: Inverse Wishart
> Log PDF Incorrect for Non-diagonal Scale Matrix?
> * `#8878 <https://github.com/scipy/scipy/issues/8878>`__: vonmises and
> vonmises_line in stats: vonmises wrong and superfluous?
> * `#8895 <https://github.com/scipy/scipy/issues/8895>`__: v1.1.0
> `ndi.rotate` documentation ? reused parameters not filled...
> * `#8900 <https://github.com/scipy/scipy/issues/8900>`__: Missing complex
> conjugation in scipy.sparse.linalg.LinearOperator
> * `#8904 <https://github.com/scipy/scipy/issues/8904>`__: BUG: if zero
> derivative at root, then Newton fails with RuntimeWarning
> * `#8911 <https://github.com/scipy/scipy/issues/8911>`__:
> make_interp_spline bc_type incorrect input interpretation
> * `#8942 <https://github.com/scipy/scipy/issues/8942>`__: MAINT: Refactor
> `_linprog.py` and `_linprog_ip.py` to remove...
> * `#8947 <https://github.com/scipy/scipy/issues/8947>`__: np.int64 in
> scipy.fftpack.next_fast_len
> * `#9020 <https://github.com/scipy/scipy/issues/9020>`__: BUG:
> linalg.subspace_angles gives wrong results
> * `#9033 <https://github.com/scipy/scipy/issues/9033>`__:
> scipy.stats.normaltest sometimes gives incorrect returns b/c...
> * `#9036 <https://github.com/scipy/scipy/issues/9036>`__: Bizarre times
> for `scipy.sparse.rand` function with 'low' density...
> * `#9044 <https://github.com/scipy/scipy/issues/9044>`__:
> optimize.minimize(method=`trust-constr`) result dict does not...
> * `#9071 <https://github.com/scipy/scipy/issues/9071>`__: doc/linalg: add
> cho_solve_banded to see also of cholesky_banded
> * `#9082 <https://github.com/scipy/scipy/issues/9082>`__: eigenvalue
> sorting in scipy.sparse.linalg.eigsh
> * `#9086 <https://github.com/scipy/scipy/issues/9086>`__:
> signaltools.py:491: FutureWarning: Using a non-tuple sequence...
> * `#9091 <https://github.com/scipy/scipy/issues/9091>`__:
> test_spline_filter failure on 32-bit
> * `#9122 <https://github.com/scipy/scipy/issues/9122>`__: Typo on scipy
> minimization tutorial
> * `#9135 <https://github.com/scipy/scipy/issues/9135>`__: doc error at
> https://docs.scipy.org/doc/scipy/reference/tutorial/stats/discrete_poisson.html
> * `#9167 <https://github.com/scipy/scipy/issues/9167>`__: DOC: BUG: typo
> in ndimage LowLevelCallable tutorial example
> * `#9169 <https://github.com/scipy/scipy/issues/9169>`__: truncnorm does
> not work if b < a in scipy.stats
> * `#9250 <https://github.com/scipy/scipy/issues/9250>`__:
> scipy.special.tests.test_mpmath::TestSystematic::test_pcfw fails...
> * `#9259 <https://github.com/scipy/scipy/issues/9259>`__: rv.expect() ==
> rv.mean() is false for rv.mean() == nan (and inf)
> * `#9286 <https://github.com/scipy/scipy/issues/9286>`__: DOC: Rosenbrock
> expression in optimize.minimize tutorial
> * `#9316 <https://github.com/scipy/scipy/issues/9316>`__: SLSQP fails in
> nested optimization
> * `#9337 <https://github.com/scipy/scipy/issues/9337>`__:
> scipy.signal.find_peaks key typo in documentation
> * `#9345 <https://github.com/scipy/scipy/issues/9345>`__: Example from
> documentation of scipy.sparse.linalg.eigs raises...
> * `#9383 <https://github.com/scipy/scipy/issues/9383>`__: Default value
> for "mode" in "ndimage.shift"
> * `#9419 <https://github.com/scipy/scipy/issues/9419>`__: dual_annealing
> off by one in the number of iterations
> * `#9442 <https://github.com/scipy/scipy/issues/9442>`__: Error in
> Defintion of Rosenbrock Function
> * `#9453 <https://github.com/scipy/scipy/issues/9453>`__: TST:
> test_eigs_consistency() doesn't have consistent results
>
>
> Pull requests for 1.2.0
> - -----------------------
>
> * `#7352 <https://github.com/scipy/scipy/pull/7352>`__: ENH: add Brunner
> Munzel test to scipy.stats.
> * `#7373 <https://github.com/scipy/scipy/pull/7373>`__: BUG: Jaccard
> distance for all-zero arrays would return np.nan
> * `#7374 <https://github.com/scipy/scipy/pull/7374>`__: ENH: Add PDF, CDF
> and parameter estimation for Stable Distributions
> * `#8098 <https://github.com/scipy/scipy/pull/8098>`__: ENH: Add shgo for
> global optimization of NLPs.
> * `#8203 <https://github.com/scipy/scipy/pull/8203>`__: ENH: adding
> simulated dual annealing to optimize
> * `#8259 <https://github.com/scipy/scipy/pull/8259>`__: Option to follow
> original Storn and Price algorithm and its parallelisation
> * `#8293 <https://github.com/scipy/scipy/pull/8293>`__: ENH add
> ratio-of-uniforms method for rv generation to scipy.stats
> * `#8294 <https://github.com/scipy/scipy/pull/8294>`__: BUG: Fix slowness
> in stats.mode
> * `#8295 <https://github.com/scipy/scipy/pull/8295>`__: ENH: add Jensen
> Shannon distance to `scipy.spatial.distance`
> * `#8357 <https://github.com/scipy/scipy/pull/8357>`__: ENH: vectorize
> scalar zero-search-functions
> * `#8397 <https://github.com/scipy/scipy/pull/8397>`__: Add `fs=`
> parameter to filter design functions
> * `#8537 <https://github.com/scipy/scipy/pull/8537>`__: ENH: Implement
> mode parameter for spline filtering.
> * `#8558 <https://github.com/scipy/scipy/pull/8558>`__: ENH: small
> speedup for stats.gaussian_kde
> * `#8560 <https://github.com/scipy/scipy/pull/8560>`__: BUG: fix p-value
> calc of anderson_ksamp in scipy.stats
> * `#8614 <https://github.com/scipy/scipy/pull/8614>`__: ENH: correct
> p-values for stats.kendalltau and stats.mstats.kendalltau
> * `#8670 <https://github.com/scipy/scipy/pull/8670>`__: ENH: Require
> Lapack 3.4.0
> * `#8683 <https://github.com/scipy/scipy/pull/8683>`__: Correcting kmeans
> documentation
> * `#8725 <https://github.com/scipy/scipy/pull/8725>`__: MAINT: Cleanup
> scipy.optimize.leastsq
> * `#8726 <https://github.com/scipy/scipy/pull/8726>`__: BUG: Fix
> _get_output in scipy.ndimage to support string
> * `#8733 <https://github.com/scipy/scipy/pull/8733>`__: MAINT: stats: A
> bit of clean up.
> * `#8737 <https://github.com/scipy/scipy/pull/8737>`__: BUG: Improve
> numerical precision/convergence failures of smirnov/kolmogorov
> * `#8738 <https://github.com/scipy/scipy/pull/8738>`__: MAINT: stats: A
> bit of clean up in test_distributions.py.
> * `#8740 <https://github.com/scipy/scipy/pull/8740>`__: BF/ENH: make
> minpack thread safe
> * `#8742 <https://github.com/scipy/scipy/pull/8742>`__: BUG: Fix division
> by zero in trust-region optimization methods
> * `#8746 <https://github.com/scipy/scipy/pull/8746>`__: MAINT: signal:
> Fix a docstring of a private function, and fix...
> * `#8750 <https://github.com/scipy/scipy/pull/8750>`__: DOC clarified
> description of norminvgauss in scipy.stats
> * `#8753 <https://github.com/scipy/scipy/pull/8753>`__: DOC: signal: Fix
> a plot title in the chirp docstring.
> * `#8755 <https://github.com/scipy/scipy/pull/8755>`__: DOC: MAINT: Fix
> link to the wheel documentation in developer...
> * `#8760 <https://github.com/scipy/scipy/pull/8760>`__: BUG: stats:
> boltzmann wasn't setting the upper bound.
> * `#8763 <https://github.com/scipy/scipy/pull/8763>`__: [DOC] Improved
> scipy.cluster.hierarchy documentation
> * `#8765 <https://github.com/scipy/scipy/pull/8765>`__: DOC: added
> example for scipy.stat.mstats.tmin
> * `#8788 <https://github.com/scipy/scipy/pull/8788>`__: DOC: fix
> definition of optional `disp` parameter
> * `#8802 <https://github.com/scipy/scipy/pull/8802>`__: MAINT: Suppress
> dd_real unused function compiler warnings.
> * `#8803 <https://github.com/scipy/scipy/pull/8803>`__: ENH: Add
> full_output support to optimize.newton()
> * `#8804 <https://github.com/scipy/scipy/pull/8804>`__: MAINT: stats
> cleanup
> * `#8808 <https://github.com/scipy/scipy/pull/8808>`__: DOC: add note
> about isinstance for frozen rvs
> * `#8812 <https://github.com/scipy/scipy/pull/8812>`__: Updated numpydoc
> submodule
> * `#8813 <https://github.com/scipy/scipy/pull/8813>`__: MAINT: stats: Fix
> multinomial docstrings, and do some clean up.
> * `#8816 <https://github.com/scipy/scipy/pull/8816>`__: BUG: fixed _stats
> of t-distribution in scipy.stats
> * `#8817 <https://github.com/scipy/scipy/pull/8817>`__: BUG: ndimage: Fix
> validation of the origin argument in correlate...
> * `#8822 <https://github.com/scipy/scipy/pull/8822>`__: BUG: integrate:
> Fix crash with repeated t values in odeint.
> * `#8832 <https://github.com/scipy/scipy/pull/8832>`__: Hyperlink DOIs
> against preferred resolver
> * `#8837 <https://github.com/scipy/scipy/pull/8837>`__: BUG: sparse:
> Ensure correct dtype for sparse comparison operations.
> * `#8839 <https://github.com/scipy/scipy/pull/8839>`__: DOC: stats: A few
> tweaks to the linregress docstring.
> * `#8846 <https://github.com/scipy/scipy/pull/8846>`__: BUG: stats: Fix
> logpdf method of invwishart.
> * `#8849 <https://github.com/scipy/scipy/pull/8849>`__: DOC: signal:
> Fixed mistake in the firwin docstring.
> * `#8854 <https://github.com/scipy/scipy/pull/8854>`__: DOC: fix type
> descriptors in ltisys documentation
> * `#8865 <https://github.com/scipy/scipy/pull/8865>`__: Fix tiny typo in
> docs for chi2 pdf
> * `#8870 <https://github.com/scipy/scipy/pull/8870>`__: Fixes related to
> invertibility of STFT
> * `#8872 <https://github.com/scipy/scipy/pull/8872>`__: ENH: special: Add
> the softmax function
> * `#8874 <https://github.com/scipy/scipy/pull/8874>`__: DOC correct gamma
> function in docstrings in scipy.stats
> * `#8876 <https://github.com/scipy/scipy/pull/8876>`__: ENH: Added TOMS
> Algorithm 748 as 1-d root finder; 17 test function...
> * `#8882 <https://github.com/scipy/scipy/pull/8882>`__: ENH: Only use
> Halley's adjustment to Newton if close enough.
> * `#8883 <https://github.com/scipy/scipy/pull/8883>`__: FIX: optimize:
> make jac and hess truly optional for 'trust-constr'
> * `#8885 <https://github.com/scipy/scipy/pull/8885>`__: TST: Do not error
> on warnings raised about non-tuple indexing.
> * `#8887 <https://github.com/scipy/scipy/pull/8887>`__: MAINT: filter out
> np.matrix PendingDeprecationWarning's in numpy...
> * `#8889 <https://github.com/scipy/scipy/pull/8889>`__: DOC: optimize:
> separate legacy interfaces from new ones
> * `#8890 <https://github.com/scipy/scipy/pull/8890>`__: ENH: Add
> optimize.root_scalar() as a universal dispatcher for...
> * `#8899 <https://github.com/scipy/scipy/pull/8899>`__: DCT-IV, DST-IV
> and DCT-I, DST-I orthonormalization support in...
> * `#8901 <https://github.com/scipy/scipy/pull/8901>`__: MAINT: Reorganize
> flapack.pyf.src file
> * `#8907 <https://github.com/scipy/scipy/pull/8907>`__: BUG: ENH: Check
> if guess for newton is already zero before checking...
> * `#8908 <https://github.com/scipy/scipy/pull/8908>`__: ENH: Make sorting
> optional for cKDTree.query_ball_point()
> * `#8910 <https://github.com/scipy/scipy/pull/8910>`__: DOC:
> sparse.csgraph simple examples.
> * `#8914 <https://github.com/scipy/scipy/pull/8914>`__: DOC: interpolate:
> fix equivalences of string aliases
> * `#8918 <https://github.com/scipy/scipy/pull/8918>`__: add
> float_control(precise, on) to _fpumode.c
> * `#8919 <https://github.com/scipy/scipy/pull/8919>`__: MAINT:
> interpolate: improve error messages for common `bc_type`...
> * `#8920 <https://github.com/scipy/scipy/pull/8920>`__: DOC: update
> Contributing to SciPy to say "prefer no PEP8 only...
> * `#8924 <https://github.com/scipy/scipy/pull/8924>`__: MAINT: special:
> deprecate `hyp2f0`, `hyp1f2`, and `hyp3f0`
> * `#8927 <https://github.com/scipy/scipy/pull/8927>`__: MAINT: special:
> remove `errprint`
> * `#8932 <https://github.com/scipy/scipy/pull/8932>`__: Fix broadcasting
> scale arg of entropy
> * `#8936 <https://github.com/scipy/scipy/pull/8936>`__: Fix (some)
> non-tuple index warnings
> * `#8937 <https://github.com/scipy/scipy/pull/8937>`__: ENH: implement
> sparse matrix BSR to CSR conversion directly.
> * `#8938 <https://github.com/scipy/scipy/pull/8938>`__: DOC: add
> @_ni_docstrings.docfiller in ndimage.rotate
> * `#8940 <https://github.com/scipy/scipy/pull/8940>`__: Update
> _discrete_distns.py
> * `#8943 <https://github.com/scipy/scipy/pull/8943>`__: DOC: Finish
> dangling sentence in `convolve` docstring
> * `#8944 <https://github.com/scipy/scipy/pull/8944>`__: MAINT: Address
> tuple indexing and warnings
> * `#8945 <https://github.com/scipy/scipy/pull/8945>`__: ENH:
> spatial.transform.Rotation [GSOC2018]
> * `#8950 <https://github.com/scipy/scipy/pull/8950>`__: csgraph Dijkstra
> function description rewording
> * `#8953 <https://github.com/scipy/scipy/pull/8953>`__: DOC, MAINT: HTTP
> -> HTTPS, and other linkrot fixes
> * `#8955 <https://github.com/scipy/scipy/pull/8955>`__: BUG: np.int64 in
> scipy.fftpack.next_fast_len
> * `#8958 <https://github.com/scipy/scipy/pull/8958>`__: MAINT: Add more
> descriptive error message for phase one simplex.
> * `#8962 <https://github.com/scipy/scipy/pull/8962>`__: BUG:
> sparse.linalg: add missing conjugate to _ScaledLinearOperator.adjoint
> * `#8963 <https://github.com/scipy/scipy/pull/8963>`__: BUG:
> sparse.linalg: downgrade LinearOperator TypeError to warning
> * `#8965 <https://github.com/scipy/scipy/pull/8965>`__: ENH: Wrapped RFP
> format and RZ decomposition routines
> * `#8969 <https://github.com/scipy/scipy/pull/8969>`__: MAINT: doc and
> code fixes for optimize.newton
> * `#8970 <https://github.com/scipy/scipy/pull/8970>`__: Added 'average'
> keyword for welch/csd to enable median averaging
> * `#8971 <https://github.com/scipy/scipy/pull/8971>`__: Better imresize
> deprecation warning
> * `#8972 <https://github.com/scipy/scipy/pull/8972>`__: MAINT: Switch
> np.where(c) for np.nonzero(c)
> * `#8975 <https://github.com/scipy/scipy/pull/8975>`__: MAINT: Fix
> warning-based failures
> * `#8979 <https://github.com/scipy/scipy/pull/8979>`__: DOC: fix
> description of count_sort keyword of dendrogram
> * `#8982 <https://github.com/scipy/scipy/pull/8982>`__: MAINT: optimize:
> Fixed minor mistakes in test_linprog.py (#8978)
> * `#8984 <https://github.com/scipy/scipy/pull/8984>`__: BUG:
> sparse.linalg: ensure expm casts integer inputs to float
> * `#8986 <https://github.com/scipy/scipy/pull/8986>`__: BUG:
> optimize/slsqp: do not exit with convergence on steps where...
> * `#8989 <https://github.com/scipy/scipy/pull/8989>`__: MAINT: use
> collections.abc in basinhopping
> * `#8990 <https://github.com/scipy/scipy/pull/8990>`__: ENH extend
> p-values of anderson_ksamp in scipy.stats
> * `#8991 <https://github.com/scipy/scipy/pull/8991>`__: ENH: Weighted kde
> * `#8993 <https://github.com/scipy/scipy/pull/8993>`__: ENH:
> spatial.transform.Rotation.random [GSOC 2018]
> * `#8994 <https://github.com/scipy/scipy/pull/8994>`__: ENH:
> spatial.transform.Slerp [GSOC 2018]
> * `#8995 <https://github.com/scipy/scipy/pull/8995>`__: TST: time.time in
> test
> * `#9007 <https://github.com/scipy/scipy/pull/9007>`__: Fix typo in
> fftpack.rst
> * `#9013 <https://github.com/scipy/scipy/pull/9013>`__: Added correct
> plotting code for two sided output from spectrogram
> * `#9014 <https://github.com/scipy/scipy/pull/9014>`__: BUG:
> differential_evolution with inf objective functions
> * `#9017 <https://github.com/scipy/scipy/pull/9017>`__: BUG: fixed #8446
> corner case for asformat(array|dense)
> * `#9018 <https://github.com/scipy/scipy/pull/9018>`__: MAINT:
> _lib/ccallback: remove unused code
> * `#9021 <https://github.com/scipy/scipy/pull/9021>`__: BUG: Issue with
> subspace_angles
> * `#9022 <https://github.com/scipy/scipy/pull/9022>`__: DOC: Added "See
> Also" section to lombscargle docstring
> * `#9034 <https://github.com/scipy/scipy/pull/9034>`__: BUG: Fix
> tolerance printing behavior, remove meaningless tol...
> * `#9035 <https://github.com/scipy/scipy/pull/9035>`__: TST: improve
> signal.bsplines test coverage
> * `#9037 <https://github.com/scipy/scipy/pull/9037>`__: ENH: add a new
> init method for k-means
> * `#9039 <https://github.com/scipy/scipy/pull/9039>`__: DOC: Add examples
> to fftpack.irfft docstrings
> * `#9048 <https://github.com/scipy/scipy/pull/9048>`__: ENH:
> scipy.sparse.random
> * `#9050 <https://github.com/scipy/scipy/pull/9050>`__: BUG:
> scipy.io.hb_write: fails for matrices not in csc format
> * `#9051 <https://github.com/scipy/scipy/pull/9051>`__: MAINT: Fix slow
> sparse.rand for k < mn/3 (#9036).
> * `#9054 <https://github.com/scipy/scipy/pull/9054>`__: MAINT: spatial:
> Explicitly initialize LAPACK output parameters.
> * `#9055 <https://github.com/scipy/scipy/pull/9055>`__: DOC: Add examples
> to scipy.special docstrings
> * `#9056 <https://github.com/scipy/scipy/pull/9056>`__: ENH: Use one
> thread in OpenBLAS
> * `#9059 <https://github.com/scipy/scipy/pull/9059>`__: DOC: Update
> README with link to Code of Conduct
> * `#9060 <https://github.com/scipy/scipy/pull/9060>`__: BLD: remove
> support for the Bento build system.
> * `#9062 <https://github.com/scipy/scipy/pull/9062>`__: DOC add sections
> to overview in scipy.stats
> * `#9066 <https://github.com/scipy/scipy/pull/9066>`__: BUG: Correct
> "remez" error message
> * `#9069 <https://github.com/scipy/scipy/pull/9069>`__: DOC: update
> linalg section of roadmap for LAPACK versions.
> * `#9079 <https://github.com/scipy/scipy/pull/9079>`__: MAINT: add
> spatial.transform to refguide check; complete some...
> * `#9081 <https://github.com/scipy/scipy/pull/9081>`__: MAINT: Add
> warnings if pivot value is close to tolerance in linprog(method='simplex')
> * `#9084 <https://github.com/scipy/scipy/pull/9084>`__: BUG fix incorrect
> p-values of kurtosistest in scipy.stats
> * `#9095 <https://github.com/scipy/scipy/pull/9095>`__: DOC: add sections
> to mstats overview in scipy.stats
> * `#9096 <https://github.com/scipy/scipy/pull/9096>`__: BUG: Add test for
> Stackoverflow example from issue 8174.
> * `#9101 <https://github.com/scipy/scipy/pull/9101>`__: ENH: add Siegel
> slopes (robust regression) to scipy.stats
> * `#9105 <https://github.com/scipy/scipy/pull/9105>`__: allow
> resample_poly() to output float32 for float32 inputs.
> * `#9112 <https://github.com/scipy/scipy/pull/9112>`__: MAINT: optimize:
> make trust-constr accept constraint dict (#9043)
> * `#9118 <https://github.com/scipy/scipy/pull/9118>`__: Add doc entry to
> cholesky_banded
> * `#9120 <https://github.com/scipy/scipy/pull/9120>`__: eigsh
> documentation parameters
> * `#9125 <https://github.com/scipy/scipy/pull/9125>`__: interpolative:
> correctly reconstruct full rank matrices
> * `#9126 <https://github.com/scipy/scipy/pull/9126>`__: MAINT: Use
> warnings for unexpected peak properties
> * `#9129 <https://github.com/scipy/scipy/pull/9129>`__: BUG: Do not catch
> and silence KeyboardInterrupt
> * `#9131 <https://github.com/scipy/scipy/pull/9131>`__: DOC: Correct the
> typo in scipy.optimize tutorial page
> * `#9133 <https://github.com/scipy/scipy/pull/9133>`__: FIX: Avoid use of
> bare except
> * `#9134 <https://github.com/scipy/scipy/pull/9134>`__: DOC: Update of
> 'return_eigenvectors' description
> * `#9137 <https://github.com/scipy/scipy/pull/9137>`__: DOC: typo fixes
> for discrete Poisson tutorial
> * `#9139 <https://github.com/scipy/scipy/pull/9139>`__: FIX: Doctest
> failure in optimize tutorial
> * `#9143 <https://github.com/scipy/scipy/pull/9143>`__: DOC: missing
> sigma in Pearson r formula
> * `#9145 <https://github.com/scipy/scipy/pull/9145>`__: MAINT: Refactor
> linear programming solvers
> * `#9149 <https://github.com/scipy/scipy/pull/9149>`__: FIX: Make
> scipy.odr.ODR ifixx equal to its data.fix if given
> * `#9156 <https://github.com/scipy/scipy/pull/9156>`__: DOC: special:
> Mention the sigmoid function in the expit docstring.
> * `#9160 <https://github.com/scipy/scipy/pull/9160>`__: Fixed a latex
> delimiter error in levy()
> * `#9170 <https://github.com/scipy/scipy/pull/9170>`__: DOC: correction /
> update of docstrings of distributions in scipy.stats
> * `#9171 <https://github.com/scipy/scipy/pull/9171>`__: better
> description of the hierarchical clustering parameter
> * `#9174 <https://github.com/scipy/scipy/pull/9174>`__: domain check for
> a < b in stats.truncnorm
> * `#9175 <https://github.com/scipy/scipy/pull/9175>`__: DOC: Minor
> grammar fix
> * `#9176 <https://github.com/scipy/scipy/pull/9176>`__: BUG:
> CloughTocher2DInterpolator: fix miscalculation at neighborless...
> * `#9177 <https://github.com/scipy/scipy/pull/9177>`__: BUILD: Document
> the "clean" target in the doc/Makefile.
> * `#9178 <https://github.com/scipy/scipy/pull/9178>`__: MAINT: make
> refguide-check more robust for printed numpy arrays
> * `#9186 <https://github.com/scipy/scipy/pull/9186>`__: MAINT: Remove
> np.ediff1d occurence
> * `#9188 <https://github.com/scipy/scipy/pull/9188>`__: DOC: correct typo
> in extending ndimage with C
> * `#9190 <https://github.com/scipy/scipy/pull/9190>`__: ENH: Support
> specifying axes for fftconvolve
> * `#9192 <https://github.com/scipy/scipy/pull/9192>`__: MAINT: optimize:
> fixed @pv style suggestions from #9112
> * `#9200 <https://github.com/scipy/scipy/pull/9200>`__: Fix
> make_interp_spline(..., k=0 or 1, axis<0)
> * `#9201 <https://github.com/scipy/scipy/pull/9201>`__: BUG:
> sparse.linalg/gmres: use machine eps in breakdown check
> * `#9204 <https://github.com/scipy/scipy/pull/9204>`__: MAINT: fix up
> stats.spearmanr and match mstats.spearmanr with...
> * `#9206 <https://github.com/scipy/scipy/pull/9206>`__: MAINT: include
> benchmarks and dev files in sdist.
> * `#9208 <https://github.com/scipy/scipy/pull/9208>`__: TST: signal: bump
> bsplines test tolerance for complex data
> * `#9210 <https://github.com/scipy/scipy/pull/9210>`__: TST: mark tests
> as slow, fix missing random seed
> * `#9211 <https://github.com/scipy/scipy/pull/9211>`__: ENH: add
> capability to specify orders in pade func
> * `#9217 <https://github.com/scipy/scipy/pull/9217>`__: MAINT: Include
> ``success`` and ``nit`` in OptimizeResult returned...
> * `#9222 <https://github.com/scipy/scipy/pull/9222>`__: ENH: interpolate:
> Use scipy.spatial.distance to speed-up Rbf
> * `#9229 <https://github.com/scipy/scipy/pull/9229>`__: MNT: Fix Fourier
> filter double case
> * `#9233 <https://github.com/scipy/scipy/pull/9233>`__: BUG:
> spatial/distance: fix pdist/cdist performance regression...
> * `#9234 <https://github.com/scipy/scipy/pull/9234>`__: FIX: Proper
> suppression
> * `#9235 <https://github.com/scipy/scipy/pull/9235>`__: BENCH:
> rationalize slow benchmarks + miscellaneous fixes
> * `#9238 <https://github.com/scipy/scipy/pull/9238>`__: BENCH: limit
> number of parameter combinations in spatial.*KDTree...
> * `#9239 <https://github.com/scipy/scipy/pull/9239>`__: DOC: stats: Fix
> LaTeX markup of a couple distribution PDFs.
> * `#9241 <https://github.com/scipy/scipy/pull/9241>`__: ENH: Evaluate
> plateau size during peak finding
> * `#9242 <https://github.com/scipy/scipy/pull/9242>`__: ENH: stats:
> Implement _ppf and _logpdf for crystalball, and do...
> * `#9246 <https://github.com/scipy/scipy/pull/9246>`__: DOC: Properly
> render versionadded directive in HTML documentation
> * `#9255 <https://github.com/scipy/scipy/pull/9255>`__: DOC: mention
> RootResults in optimization reference guide
> * `#9260 <https://github.com/scipy/scipy/pull/9260>`__: TST: relax some
> tolerances so tests pass with x87 math
> * `#9264 <https://github.com/scipy/scipy/pull/9264>`__: TST Use
> assert_raises "match" parameter instead of the "message"...
> * `#9267 <https://github.com/scipy/scipy/pull/9267>`__: DOC: clarify
> expect() return val when moment is inf/nan
> * `#9272 <https://github.com/scipy/scipy/pull/9272>`__: DOC: Add
> description of default bounds to linprog
> * `#9277 <https://github.com/scipy/scipy/pull/9277>`__: MAINT:
> sparse/linalg: make test deterministic
> * `#9278 <https://github.com/scipy/scipy/pull/9278>`__: MAINT:
> interpolate: pep8 cleanup in test_polyint
> * `#9279 <https://github.com/scipy/scipy/pull/9279>`__: Fixed docstring
> for resample
> * `#9280 <https://github.com/scipy/scipy/pull/9280>`__: removed first
> check for float in get_sum_dtype
> * `#9281 <https://github.com/scipy/scipy/pull/9281>`__: BUG: only accept
> 1d input for bartlett / levene in scipy.stats
> * `#9282 <https://github.com/scipy/scipy/pull/9282>`__: MAINT:
> dense_output and t_eval are mutually exclusive inputs
> * `#9283 <https://github.com/scipy/scipy/pull/9283>`__: MAINT: add docs
> and do some cleanups in interpolate.Rbf
> * `#9288 <https://github.com/scipy/scipy/pull/9288>`__: Run
> distance_transform_edt tests on all types
> * `#9294 <https://github.com/scipy/scipy/pull/9294>`__: DOC: fix the
> formula typo
> * `#9298 <https://github.com/scipy/scipy/pull/9298>`__: MAINT:
> optimize/trust-constr: restore .niter attribute for backward-compat
> * `#9299 <https://github.com/scipy/scipy/pull/9299>`__: DOC:
> clarification of default rvs method in scipy.stats
> * `#9301 <https://github.com/scipy/scipy/pull/9301>`__: MAINT: removed
> unused import sys
> * `#9302 <https://github.com/scipy/scipy/pull/9302>`__: MAINT: removed
> unused imports
> * `#9303 <https://github.com/scipy/scipy/pull/9303>`__: DOC: signal:
> Refer to fs instead of nyq in the firwin docstring.
> * `#9305 <https://github.com/scipy/scipy/pull/9305>`__: ENH: Added
> Yeo-Johnson power transformation
> * `#9306 <https://github.com/scipy/scipy/pull/9306>`__: ENH - add dual
> annealing
> * `#9309 <https://github.com/scipy/scipy/pull/9309>`__: ENH add the
> yulesimon distribution to scipy.stats
> * `#9317 <https://github.com/scipy/scipy/pull/9317>`__: Nested SLSQP bug
> fix.
> * `#9320 <https://github.com/scipy/scipy/pull/9320>`__: MAINT: stats:
> avoid underflow in stats.geom.ppf
> * `#9326 <https://github.com/scipy/scipy/pull/9326>`__: Add example for
> Rosenbrock function
> * `#9332 <https://github.com/scipy/scipy/pull/9332>`__: Sort file lists
> * `#9340 <https://github.com/scipy/scipy/pull/9340>`__: Fix typo in
> find_peaks documentation
> * `#9343 <https://github.com/scipy/scipy/pull/9343>`__: MAINT Use np.full
> when possible
> * `#9344 <https://github.com/scipy/scipy/pull/9344>`__: DOC: added
> examples to docstring of dirichlet class
> * `#9346 <https://github.com/scipy/scipy/pull/9346>`__: DOC: Fix import
> of scipy.sparse.linalg in example (#9345)
> * `#9350 <https://github.com/scipy/scipy/pull/9350>`__: Fix interpolate
> read only
> * `#9351 <https://github.com/scipy/scipy/pull/9351>`__: MAINT:
> special.erf: use the x->-x symmetry
> * `#9356 <https://github.com/scipy/scipy/pull/9356>`__: Fix documentation
> typo
> * `#9358 <https://github.com/scipy/scipy/pull/9358>`__: DOC: improve doc
> for ksone and kstwobign in scipy.stats
> * `#9362 <https://github.com/scipy/scipy/pull/9362>`__: DOC: Change
> datatypes of A matrices in linprog
> * `#9364 <https://github.com/scipy/scipy/pull/9364>`__: MAINT: Adds
> implicit none to fftpack fortran sources
> * `#9369 <https://github.com/scipy/scipy/pull/9369>`__: DOC: minor tweak
> to CoC (updated NumFOCUS contact address).
> * `#9373 <https://github.com/scipy/scipy/pull/9373>`__: Fix exception if
> python is called with -OO option
> * `#9374 <https://github.com/scipy/scipy/pull/9374>`__: FIX: AIX
> compilation issue with NAN and INFINITY
> * `#9376 <https://github.com/scipy/scipy/pull/9376>`__: COBLYA -> COBYLA
> in docs
> * `#9377 <https://github.com/scipy/scipy/pull/9377>`__: DOC: Add examples
> integrate: fixed_quad and quadrature
> * `#9379 <https://github.com/scipy/scipy/pull/9379>`__: MAINT: TST: Make
> tests NumPy 1.8 compatible
> * `#9385 <https://github.com/scipy/scipy/pull/9385>`__: CI: On Travis
> matrix "OPTIMIZE=-OO" flag ignored
> * `#9387 <https://github.com/scipy/scipy/pull/9387>`__: Fix defaut value
> for 'mode' in 'ndimage.shift' in the doc
> * `#9392 <https://github.com/scipy/scipy/pull/9392>`__: BUG: rank has to
> be integer in rank_filter: fixed issue 9388
> * `#9399 <https://github.com/scipy/scipy/pull/9399>`__: DOC: Misc. typos
> * `#9400 <https://github.com/scipy/scipy/pull/9400>`__: TST: stats: Fix
> the expected r-value of a linregress test.
> * `#9405 <https://github.com/scipy/scipy/pull/9405>`__: BUG: np.hstack
> does not accept generator expressions
> * `#9408 <https://github.com/scipy/scipy/pull/9408>`__: ENH: linalg:
> Shorter ill-conditioned warning message
> * `#9418 <https://github.com/scipy/scipy/pull/9418>`__: DOC: Fix ndimage
> docstrings and reduce doc build warnings
> * `#9421 <https://github.com/scipy/scipy/pull/9421>`__: DOC: Add missing
> docstring examples in scipy.spatial
> * `#9422 <https://github.com/scipy/scipy/pull/9422>`__: DOC: Add an
> example to integrate.newton_cotes
> * `#9427 <https://github.com/scipy/scipy/pull/9427>`__: BUG: Fixed defect
> with maxiter #9419 in dual annealing
> * `#9431 <https://github.com/scipy/scipy/pull/9431>`__: BENCH: Add dual
> annealing to scipy benchmark (see #9415)
> * `#9435 <https://github.com/scipy/scipy/pull/9435>`__: DOC: Add
> docstring examples for stats.binom_test
> * `#9443 <https://github.com/scipy/scipy/pull/9443>`__: DOC: Fix the
> order of indices in optimize tutorial
> * `#9444 <https://github.com/scipy/scipy/pull/9444>`__: MAINT:
> interpolate: use operator.index for checking/coercing...
> * `#9445 <https://github.com/scipy/scipy/pull/9445>`__: DOC: Added
> missing example to stats.mstats.kruskal
> * `#9446 <https://github.com/scipy/scipy/pull/9446>`__: DOC: Add note
> about version changed for jaccard distance
> * `#9447 <https://github.com/scipy/scipy/pull/9447>`__: BLD:
> version-script handling in setup.py
> * `#9448 <https://github.com/scipy/scipy/pull/9448>`__: TST: skip a
> problematic linalg test
> * `#9449 <https://github.com/scipy/scipy/pull/9449>`__: TST: fix missing
> seed in lobpcg test.
> * `#9456 <https://github.com/scipy/scipy/pull/9456>`__: TST:
> test_eigs_consistency() now sorts output
>
> Checksums
> =========
>
> MD5
> ~~~
>
> 47d309402d2e5574be8fa261fadfaf58
> scipy-1.2.0rc1-cp27-cp27m-macosx_10_6_intel.macosx_10_9_intel.macosx_10_9_x86_64.macosx_10_10_intel.macosx_10_10_x86_64.whl
> 911dfde5be66403c07c60e19aa631dc2
> scipy-1.2.0rc1-cp27-cp27m-manylinux1_i686.whl
> a693189336365595b42b0d93f825b826
> scipy-1.2.0rc1-cp27-cp27m-manylinux1_x86_64.whl
> ec5abd33480761ed9701f7fd2274fc47  scipy-1.2.0rc1-cp27-cp27m-win32.whl
> bc3d40311f057b12f8fea97166ef8112  scipy-1.2.0rc1-cp27-cp27m-win_amd64.whl
> 33848233e6438b1ff9183d8a4794daed
> scipy-1.2.0rc1-cp27-cp27mu-manylinux1_i686.whl
> c2cb1166ce0071e1fe42ed1c3e60b75e
> scipy-1.2.0rc1-cp27-cp27mu-manylinux1_x86_64.whl
> a04b5a758f555e05e6431b8a1f035888
> scipy-1.2.0rc1-cp34-cp34m-macosx_10_6_intel.macosx_10_9_intel.macosx_10_9_x86_64.macosx_10_10_intel.macosx_10_10_x86_64.whl
> ad511246d0742cf0669fedf292cc01bb
> scipy-1.2.0rc1-cp34-cp34m-manylinux1_i686.whl
> 15aa08ef43a6c5cb320bc015f01087ad
> scipy-1.2.0rc1-cp34-cp34m-manylinux1_x86_64.whl
> 86a59b81a3e6894d9054139311a2c51f  scipy-1.2.0rc1-cp34-cp34m-win32.whl
> 8b6e33253579916ea873c45989ee5bea  scipy-1.2.0rc1-cp34-cp34m-win_amd64.whl
> db7a4de02828471bf9f800814ff68627
> scipy-1.2.0rc1-cp35-cp35m-macosx_10_6_intel.macosx_10_9_intel.macosx_10_9_x86_64.macosx_10_10_intel.macosx_10_10_x86_64.whl
> c296b6270d29d3ec6dafddf8ceae67fb
> scipy-1.2.0rc1-cp35-cp35m-manylinux1_i686.whl
> 3ba7a825f61822128d960fa728010e51
> scipy-1.2.0rc1-cp35-cp35m-manylinux1_x86_64.whl
> c083c8287da110b707d181f6638ce122  scipy-1.2.0rc1-cp35-cp35m-win32.whl
> 2242eac92681085258535ed96bd040d7  scipy-1.2.0rc1-cp35-cp35m-win_amd64.whl
> d5c238903c00e91a40d56023f1a27ed4
> scipy-1.2.0rc1-cp36-cp36m-macosx_10_6_intel.macosx_10_9_intel.macosx_10_9_x86_64.macosx_10_10_intel.macosx_10_10_x86_64.whl
> e8ab7487a9a53b86a6510772c45af787
> scipy-1.2.0rc1-cp36-cp36m-manylinux1_i686.whl
> 9991b5958d736488bef638eea463945d
> scipy-1.2.0rc1-cp36-cp36m-manylinux1_x86_64.whl
> 9c108a9d7e967b8c9a5e5143b1a15b40  scipy-1.2.0rc1-cp36-cp36m-win32.whl
> 54d63041f0315d341d9ffb028a98e767  scipy-1.2.0rc1-cp36-cp36m-win_amd64.whl
> 534276c864ab3139811561c022608cc3
> scipy-1.2.0rc1-cp37-cp37m-macosx_10_6_intel.macosx_10_9_intel.macosx_10_9_x86_64.macosx_10_10_intel.macosx_10_10_x86_64.whl
> 4bd26c179a10891087bd81a658573683
> scipy-1.2.0rc1-cp37-cp37m-manylinux1_i686.whl
> 1a3170b4f1df42f28efbe197e54eb9a3
> scipy-1.2.0rc1-cp37-cp37m-manylinux1_x86_64.whl
> 72b89f7e2c1d13dc8dbb21600fb184da  scipy-1.2.0rc1-cp37-cp37m-win32.whl
> 433ef294c4a015da0a6e0f063289a658  scipy-1.2.0rc1-cp37-cp37m-win_amd64.whl
> 83abb1befce326916e0435d428b36e62  scipy-1.2.0rc1.tar.gz
> 21b7570fb577543807feb4b4c1fdad8a  scipy-1.2.0rc1.tar.xz
> 11221dc23e0d3b316b5564f2f435aaf1  scipy-1.2.0rc1.zip
>
> SHA256
> ~~~~~~
>
> a3282027743e89d5fcb6cd7a9e4ecdbbde61bf8126fd19683e97b69f5f6a2163
> scipy-1.2.0rc1-cp27-cp27m-macosx_10_6_intel.macosx_10_9_intel.macosx_10_9_x86_64.macosx_10_10_intel.macosx_10_10_x86_64.whl
> 8b8841e03620d4d704f4efd3daed575e23a6272ae9c9be2c4cc751febee984f7
> scipy-1.2.0rc1-cp27-cp27m-manylinux1_i686.whl
> 254ea2f7f3c5afef9f02c0192d2dbd8f5336c1b1c53efae7ef64a8880cccc299
> scipy-1.2.0rc1-cp27-cp27m-manylinux1_x86_64.whl
> d449832419df3e37a3942778b46c140fd61d1e4f38f9e34bed278a77aacdbd31
> scipy-1.2.0rc1-cp27-cp27m-win32.whl
> e57f3e3eaa88cd7bc93466122fca48c36863e03aeb24f9d570a2e6b2ea3cbf92
> scipy-1.2.0rc1-cp27-cp27m-win_amd64.whl
> 00dcb606101fa10951ee235af69dbff55d999b01c9bb2bc0f64df5fc3aff4eb6
> scipy-1.2.0rc1-cp27-cp27mu-manylinux1_i686.whl
> 38857eb49f7e38d3ec079772225a79235d0bd847e24d2fa8c9a9fa70ee69f0a2
> scipy-1.2.0rc1-cp27-cp27mu-manylinux1_x86_64.whl
> 9b9950278f36c4e7779148cc9a719d41a305024c70005f994412845172ade346
> scipy-1.2.0rc1-cp34-cp34m-macosx_10_6_intel.macosx_10_9_intel.macosx_10_9_x86_64.macosx_10_10_intel.macosx_10_10_x86_64.whl
> c8a2fc15c64f1b162b02f86d28153b65f64720ca64e832bcf5bfae060a507949
> scipy-1.2.0rc1-cp34-cp34m-manylinux1_i686.whl
> 4e205e0e5e88fe2105c594a706ac9fa558fe7a4daa2bef2c86b343ab507d8bd6
> scipy-1.2.0rc1-cp34-cp34m-manylinux1_x86_64.whl
> b04fb8ba796198d83614138d7f96e88791d83f5f4e31e628cce51fda1a092f66
> scipy-1.2.0rc1-cp34-cp34m-win32.whl
> c34e4ce9a8c62c85f322d7897a257a839a4bb7fd94e08701044e9b1cc1bb15a6
> scipy-1.2.0rc1-cp34-cp34m-win_amd64.whl
> cb7f4c289c06514c6e224263868a98d5e6aa6e8a90f2304b16ff276aa96030ce
> scipy-1.2.0rc1-cp35-cp35m-macosx_10_6_intel.macosx_10_9_intel.macosx_10_9_x86_64.macosx_10_10_intel.macosx_10_10_x86_64.whl
> dcaeed34e7e965e935ba252fd3290c04cafb7d84f6f44c9c08269504aa2e0a05
> scipy-1.2.0rc1-cp35-cp35m-manylinux1_i686.whl
> 13b234b4fbda1368474a667fc29e8365b622c4c607ed490f00643625c8f50cca
> scipy-1.2.0rc1-cp35-cp35m-manylinux1_x86_64.whl
> 073560d6598b97b8f214a31856a1d43c96701b46c5cc9805789f962c3b8a0e97
> scipy-1.2.0rc1-cp35-cp35m-win32.whl
> 600cf596b83f185dca3455d03ca802de0b4de98cb4c8d041226089d78d04c2bc
> scipy-1.2.0rc1-cp35-cp35m-win_amd64.whl
> 77b7ed8e13f03a843d2f11bd140f224abe62fb56656f35348578e3838ff4d528
> scipy-1.2.0rc1-cp36-cp36m-macosx_10_6_intel.macosx_10_9_intel.macosx_10_9_x86_64.macosx_10_10_intel.macosx_10_10_x86_64.whl
> 98786b516d1955592f921c2b36505840ff921c9e82f5c28a6c2a580fb3e96af1
> scipy-1.2.0rc1-cp36-cp36m-manylinux1_i686.whl
> 8f62fc6ac6b842d7c3c499269150ec81d26a0dea5ce818aef3b0e9e14e53c5c7
> scipy-1.2.0rc1-cp36-cp36m-manylinux1_x86_64.whl
> 213cfc35ec2fbf86c5957d1ada99a1fe1eacccdb498d4790089af9cf50cadab4
> scipy-1.2.0rc1-cp36-cp36m-win32.whl
> 287e46f87399eb9897c726c6d358c893f6c769300599bc5da95cbf3397a00aa7
> scipy-1.2.0rc1-cp36-cp36m-win_amd64.whl
> 7116d193da4baca6f7cd1cd7810548fa03ed4a06e05f325978b2f0018b05ead9
> scipy-1.2.0rc1-cp37-cp37m-macosx_10_6_intel.macosx_10_9_intel.macosx_10_9_x86_64.macosx_10_10_intel.macosx_10_10_x86_64.whl
> 9cd3fe767225b0dcfcc9f09ed18288b51c29b35ca16dc4e5a6f3587b4ac7d7b5
> scipy-1.2.0rc1-cp37-cp37m-manylinux1_i686.whl
> 4024aea96a01ca05af701f93fd227a7f646258116903899a75b1f4a1f0134bf8
> scipy-1.2.0rc1-cp37-cp37m-manylinux1_x86_64.whl
> eccd0e6d321585b6e2ee18fa0ee1db4fe042e52fb4ae8e28a3d223df6bcd6a8e
> scipy-1.2.0rc1-cp37-cp37m-win32.whl
> 96f6c69b2d8f63dad5612385521d0cf8b62f665a62f56b6b3d3fb7042a63c34c
> scipy-1.2.0rc1-cp37-cp37m-win_amd64.whl
> ca9ba36bb271dcb4273e330c0bd418a1c3566ff76248dd634efecbe0e9c1721c
> scipy-1.2.0rc1.tar.gz
> b8fe757f377b43c733a0ba235b990fb4b3722bd6f5930a26359d16752e94560c
> scipy-1.2.0rc1.tar.xz
> da1980e7e037e2275821d7611a91eadafdc157b096f36f41d05cc0ea4ae539bc
> scipy-1.2.0rc1.zip
> -----BEGIN PGP SIGNATURE-----
>
> iQIzBAEBCAAdFiEEgfAqrOuERrV4fwr6XCLdIy4FKasFAlv8LB8ACgkQXCLdIy4F
> KauhWw//cFUboxtuNZeEh98Pn2mfVIqRF8hc6VK7jeXU64qiAdOnbBRKXfoXWpbt
> 8fEckTFVVt2CQzIEZju429P0eWv8psfvn90cNnj5VHz7h46AH5gQee/cfP1kNWGO
> mxcYczqZN8+ldDnww4hfk0R+3IrpfmQbVSMEqBF84uottfgUjUzPtcFHJmmmcOF2
> 7XWhAP5IWMctcwq3r3WAa8yLUmlEIB3Fs9mPPr1We7e0QiXnEueUrXVZF9Oz3UpJ
> UFiTwX5nxnNdxZJwtLbBXhxJUbvPnooby0x1NwxFG6AwrWaPG3my0e9kIzpwqfsf
> 0B7Pt/xPEHWIizaqMyfUiaS7FZOUfvq6xs5aCyQeP9Wtnwherm6SQ/Qs+LFSb8E2
> 9cgyFmaBeslrTMd3f1YMcb6d1hmNSjbBmWccZEXCqjC9UClxbAQDHDcdmTl+4vp7
> xATCxWkYVP9lOLH5TtIC1Bo0AZXiOxI3JnojQS65JH0w4unh1IJJkENurzCATh4/
> vnyddy0x2e1ok5bX8fpVFrCc8lSdO8aDY5cYEU55XZhkyroet3IaOYqwPzPQPwoJ
> QgYEsqxa2Urn0faV5uuDFTscnCAHK0EVXhr3yyL0YoIGOPffM8JaswEAUaFwLrq0
> KEbOavdImtSwqL9v6x5tDyYXh/mHQ70GGtPS68iSc1NexRrwLAw=
> =F/+Z
> -----END PGP SIGNATURE-----
>
>
>
> _______________________________________________
> NumPy-Discussion mailing list
> NumPy-Discussion at python.org
> https://mail.python.org/mailman/listinfo/numpy-discussion
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20181126/b0254758/attachment-0001.html>

From ralf.gommers at gmail.com  Mon Nov 26 20:09:01 2018
From: ralf.gommers at gmail.com (Ralf Gommers)
Date: Mon, 26 Nov 2018 17:09:01 -0800
Subject: [Numpy-discussion] NumPy 1.16
In-Reply-To: <CAB6mnxJ-rkJc4=ihYuzOAb_u5KYL4ZtjbT2pB+fdRYZsdSLyBg@mail.gmail.com>
References: <CAB6mnxJ-rkJc4=ihYuzOAb_u5KYL4ZtjbT2pB+fdRYZsdSLyBg@mail.gmail.com>
Message-ID: <CABL7CQjYqqBuNVMrARgqfTVHW5jBe__Pss0WQioekGE4RmDhSg@mail.gmail.com>

On Mon, Nov 26, 2018 at 9:41 AM Charles R Harris <charlesr.harris at gmail.com>
wrote:

> Hi All,
>
> Just an update of the NumPy 1.16 release schedule. The last PR milestoned
> for the release is #12219 <https://github.com/numpy/numpy/pull/12219>, and
> it is about done. The current release blocker is the upcoming OpenBLAS
> 0.3.4, which should fix the reported threading problems, but if it doesn't
> come out in the next month we might want to revisit that. The OpenBLAS
> issues for 0.3.4 are here
> <https://github.com/xianyi/OpenBLAS/issues?q=is%3Aopen+is%3Aissue+milestone%3A0.3.4>.
>
>

>From a read through those issues, they seem to be related to new threading
issues introduced in 0.3.1. IIRC we're building against 0.3.0 for 1.15.x.
So the reported issues may not all be fixed.

Ralf
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20181126/c006b513/attachment.html>

From charlesr.harris at gmail.com  Mon Nov 26 22:39:07 2018
From: charlesr.harris at gmail.com (Charles R Harris)
Date: Mon, 26 Nov 2018 20:39:07 -0700
Subject: [Numpy-discussion] NumPy 1.16
In-Reply-To: <CABL7CQjYqqBuNVMrARgqfTVHW5jBe__Pss0WQioekGE4RmDhSg@mail.gmail.com>
References: <CAB6mnxJ-rkJc4=ihYuzOAb_u5KYL4ZtjbT2pB+fdRYZsdSLyBg@mail.gmail.com>
 <CABL7CQjYqqBuNVMrARgqfTVHW5jBe__Pss0WQioekGE4RmDhSg@mail.gmail.com>
Message-ID: <CAB6mnxKs1vAAO5yPxn5SQJHF-u8AMjOMZwue6+kv_e+8i6_E9A@mail.gmail.com>

On Mon, Nov 26, 2018 at 6:09 PM Ralf Gommers <ralf.gommers at gmail.com> wrote:

>
>
> On Mon, Nov 26, 2018 at 9:41 AM Charles R Harris <
> charlesr.harris at gmail.com> wrote:
>
>> Hi All,
>>
>> Just an update of the NumPy 1.16 release schedule. The last PR milestoned
>> for the release is #12219 <https://github.com/numpy/numpy/pull/12219>, and
>> it is about done. The current release blocker is the upcoming OpenBLAS
>> 0.3.4, which should fix the reported threading problems, but if it doesn't
>> come out in the next month we might want to revisit that. The OpenBLAS
>> issues for 0.3.4 are here
>> <https://github.com/xianyi/OpenBLAS/issues?q=is%3Aopen+is%3Aissue+milestone%3A0.3.4>.
>>
>>
>
> From a read through those issues, they seem to be related to new threading
> issues introduced in 0.3.1. IIRC we're building against 0.3.0 for 1.15.x.
> So the reported issues may not all be fixed.
>

One of the threading problems, present since 0.2.15, is already fixed for
0.3.4, see https://github.com/xianyi/OpenBLAS/issues/1844. It was the easy
one :) I think the milestone was put together after the fact.

Chuck
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20181126/efb18c07/attachment.html>

From matti.picus at gmail.com  Tue Nov 27 15:57:06 2018
From: matti.picus at gmail.com (Matti Picus)
Date: Tue, 27 Nov 2018 12:57:06 -0800
Subject: [Numpy-discussion] Weekly status meeting Nov 28 at 12:00 Pacific
 time
Message-ID: <623a1c81-943c-023b-91ff-5d497eb000d4@gmail.com>

We will be holding our weekly BIDS NumPy status meeting on Wed Nov 28
at noon Pacific time. Please join us.

The draft agenda, along with details of how to join, is up at
https://hackmd.io/Gn1ymjwkRjm9WVY5Cgbwsw?both

Previous sessions' notes are available at
https://github.com/BIDS-numpy/docs/tree/master/status_meetings


Matti, Tyler and Stefan

From matti.picus at gmail.com  Wed Nov 28 16:14:59 2018
From: matti.picus at gmail.com (Matti Picus)
Date: Wed, 28 Nov 2018 13:14:59 -0800
Subject: [Numpy-discussion] Reminder: Numpy dev meeting Fri-Sat Nov 30-Dec 1
Message-ID: <ab7da719-88a7-d6a9-3401-218ab968301e@gmail.com>

We will be meeting at BIDS 9:00AM Friday for a two-day NumPy developer 
meeting. All are welcome, if you haven't already please let Stefan know 
you are coming so we can plan for space. A tentative schedule is here


https://hackmd.io/gFqjPUSvSmm-0gmBDbrTBw?both


Feel free to add content (just tag it with your name).


Tyler, Matti, Stefan.


From einstein.edison at gmail.com  Wed Nov 28 16:17:59 2018
From: einstein.edison at gmail.com (Hameer Abbasi)
Date: Wed, 28 Nov 2018 22:17:59 +0100
Subject: [Numpy-discussion] Reminder: Numpy dev meeting Fri-Sat Nov
 30-Dec 1
In-Reply-To: <ab7da719-88a7-d6a9-3401-218ab968301e@gmail.com>
References: <ab7da719-88a7-d6a9-3401-218ab968301e@gmail.com>
Message-ID: <44e4c2ca-ed73-4496-ac41-7546f7d35353@Canary>

Hi everyone!

Just want to add that I?ll be available remotely for some of the time. You can reach me via email and we can set up a call.

Best Regards,
Hameer Abbasi

> On Wednesday, Nov 28, 2018 at 10:15 PM, Matti Picus <matti.picus at gmail.com (mailto:matti.picus at gmail.com)> wrote:
> We will be meeting at BIDS 9:00AM Friday for a two-day NumPy developer
> meeting. All are welcome, if you haven't already please let Stefan know
> you are coming so we can plan for space. A tentative schedule is here
>
>
> https://hackmd.io/gFqjPUSvSmm-0gmBDbrTBw?both
>
>
> Feel free to add content (just tag it with your name).
>
>
> Tyler, Matti, Stefan.
>
> _______________________________________________
> NumPy-Discussion mailing list
> NumPy-Discussion at python.org
> https://mail.python.org/mailman/listinfo/numpy-discussion
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20181128/36c6f7a1/attachment.html>

From stefanv at berkeley.edu  Wed Nov 28 16:21:37 2018
From: stefanv at berkeley.edu (Stefan van der Walt)
Date: Wed, 28 Nov 2018 13:21:37 -0800
Subject: [Numpy-discussion] Reminder: Numpy dev meeting Fri-Sat Nov
 30-Dec 1
In-Reply-To: <44e4c2ca-ed73-4496-ac41-7546f7d35353@Canary>
References: <ab7da719-88a7-d6a9-3401-218ab968301e@gmail.com>
 <44e4c2ca-ed73-4496-ac41-7546f7d35353@Canary>
Message-ID: <20181128212137.sl5y34unxah4bbnk@carbo>

On Wed, 28 Nov 2018 22:17:59 +0100, Hameer Abbasi wrote:
> Just want to add that I?ll be available remotely for some of the
> time. You can reach me via email and we can set up a call.

Thank you, Hameer!

St?fan

From teoliphant at gmail.com  Wed Nov 28 18:34:51 2018
From: teoliphant at gmail.com (Travis Oliphant)
Date: Wed, 28 Nov 2018 17:34:51 -0600
Subject: [Numpy-discussion] Reminder: Numpy dev meeting Fri-Sat Nov
 30-Dec 1
In-Reply-To: <ab7da719-88a7-d6a9-3401-218ab968301e@gmail.com>
References: <ab7da719-88a7-d6a9-3401-218ab968301e@gmail.com>
Message-ID: <CAFMmPGOPPP_OCbgEYbYdxrj-93h8zvBTpEp58coaSg-mWdPQgg@mail.gmail.com>

I will be available remotely as well, but unable to come to BIDS this
week.

I am particularly interested in how to improve the dtype subsystem ---
potentially using libndtypes from the xnd project.

Thanks,

-Travis




On Wed, Nov 28, 2018 at 3:16 PM Matti Picus <matti.picus at gmail.com> wrote:

> We will be meeting at BIDS 9:00AM Friday for a two-day NumPy developer
> meeting. All are welcome, if you haven't already please let Stefan know
> you are coming so we can plan for space. A tentative schedule is here
>
>
> https://hackmd.io/gFqjPUSvSmm-0gmBDbrTBw?both
>
>
> Feel free to add content (just tag it with your name).
>
>
> Tyler, Matti, Stefan.
>
> _______________________________________________
> NumPy-Discussion mailing list
> NumPy-Discussion at python.org
> https://mail.python.org/mailman/listinfo/numpy-discussion
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20181128/34a31ab0/attachment.html>

From stefanv at berkeley.edu  Wed Nov 28 18:39:44 2018
From: stefanv at berkeley.edu (Stefan van der Walt)
Date: Wed, 28 Nov 2018 15:39:44 -0800
Subject: [Numpy-discussion] Reminder: Numpy dev meeting Fri-Sat Nov
 30-Dec 1
In-Reply-To: <CAFMmPGOPPP_OCbgEYbYdxrj-93h8zvBTpEp58coaSg-mWdPQgg@mail.gmail.com>
References: <ab7da719-88a7-d6a9-3401-218ab968301e@gmail.com>
 <CAFMmPGOPPP_OCbgEYbYdxrj-93h8zvBTpEp58coaSg-mWdPQgg@mail.gmail.com>
Message-ID: <20181128233944.xrx7lggpxib2glbh@carbo>

On Wed, 28 Nov 2018 17:34:51 -0600, Travis Oliphant wrote:
> I am particularly interested in how to improve the dtype subsystem ---
> potentially using libndtypes from the xnd project.

We'd love to hear your opinion on the matter, and can set up a Zoom call
for that purpose.  Please let me know off-list which between noon and
2pm California time works best.

Best regards,
St?fan

From charlesr.harris at gmail.com  Wed Nov 28 23:17:16 2018
From: charlesr.harris at gmail.com (Charles R Harris)
Date: Wed, 28 Nov 2018 21:17:16 -0700
Subject: [Numpy-discussion] Reminder: Numpy dev meeting Fri-Sat Nov
 30-Dec 1
In-Reply-To: <CAFMmPGOPPP_OCbgEYbYdxrj-93h8zvBTpEp58coaSg-mWdPQgg@mail.gmail.com>
References: <ab7da719-88a7-d6a9-3401-218ab968301e@gmail.com>
 <CAFMmPGOPPP_OCbgEYbYdxrj-93h8zvBTpEp58coaSg-mWdPQgg@mail.gmail.com>
Message-ID: <CAB6mnxJwtPOMJYqFReos9ENc7rXn+hky_vwQtWUNSSscEF3p_A@mail.gmail.com>

On Wed, Nov 28, 2018 at 4:35 PM Travis Oliphant <teoliphant at gmail.com>
wrote:

> I will be available remotely as well, but unable to come to BIDS this
> week.
>
> I am particularly interested in how to improve the dtype subsystem ---
> potentially using libndtypes from the xnd project.
>
>
That would be good. We are at a critical spot where, if we are not careful,
there will be a proliferation of incompatible solutions. I believe that
Quansight is doing some work that should possibly be part of NumPy.

Chuck
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20181128/2785cab7/attachment.html>

From oleksandr.pavlyk at intel.com  Thu Nov 29 14:50:38 2018
From: oleksandr.pavlyk at intel.com (Pavlyk, Oleksandr)
Date: Thu, 29 Nov 2018 19:50:38 +0000
Subject: [Numpy-discussion] How to get sources of the current state of 1.16?
Message-ID: <4C9EDA7282E297428F3986994EB0FBD30606D40B@ORSMSX108.amr.corp.intel.com>

Are they the same as the current master, or is there a mechanism to query issue tracker for all PRs designated to make it into 1.16?

Even then, what should the base be?

Perhaps this is already documented in either an issue or a PR.

Thanks for the pointer.

Sasha
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20181129/24fc1dc7/attachment.html>

From robert.kern at gmail.com  Thu Nov 29 14:59:25 2018
From: robert.kern at gmail.com (Robert Kern)
Date: Thu, 29 Nov 2018 11:59:25 -0800
Subject: [Numpy-discussion] How to get sources of the current state of
 1.16?
In-Reply-To: <4C9EDA7282E297428F3986994EB0FBD30606D40B@ORSMSX108.amr.corp.intel.com>
References: <4C9EDA7282E297428F3986994EB0FBD30606D40B@ORSMSX108.amr.corp.intel.com>
Message-ID: <CAF6FJisDx7d7uGvwiLttmw-YRNzQ6egqnPRsVFb5-qHfrU0HSw@mail.gmail.com>

There is a GitHub Milestone for 1.16:
https://github.com/numpy/numpy/milestone/58

On Thu, Nov 29, 2018 at 11:54 AM Pavlyk, Oleksandr <
oleksandr.pavlyk at intel.com> wrote:

> Are they the same as the current master, or is there a mechanism to query
> issue tracker for all PRs designated to make it into 1.16?
>
>
>
> Even then, what should the base be?
>
>
>
> Perhaps this is already documented in either an issue or a PR.
>
>
>
> Thanks for the pointer.
>
>
>
> Sasha
> _______________________________________________
> NumPy-Discussion mailing list
> NumPy-Discussion at python.org
> https://mail.python.org/mailman/listinfo/numpy-discussion
>


-- 
Robert Kern
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20181129/dc03fad6/attachment.html>

From oleksandr.pavlyk at intel.com  Thu Nov 29 17:35:28 2018
From: oleksandr.pavlyk at intel.com (Pavlyk, Oleksandr)
Date: Thu, 29 Nov 2018 22:35:28 +0000
Subject: [Numpy-discussion] How to get sources of the current state of
 1.16?
In-Reply-To: <CAF6FJisDx7d7uGvwiLttmw-YRNzQ6egqnPRsVFb5-qHfrU0HSw@mail.gmail.com>
References: <4C9EDA7282E297428F3986994EB0FBD30606D40B@ORSMSX108.amr.corp.intel.com>
 <CAF6FJisDx7d7uGvwiLttmw-YRNzQ6egqnPRsVFb5-qHfrU0HSw@mail.gmail.com>
Message-ID: <4C9EDA7282E297428F3986994EB0FBD30606D4E8@ORSMSX108.amr.corp.intel.com>

Sorry for not being more explicit.

Yes, the milestone gives me a set of PRs whose changes are designated for 1.16

If I were to manually build sources for 1.16 candidate I would

1.       create a branch at some SHA in the master,

2.       cherry-pick commits from PRs enumerated in milestone/58
not reachable from the chosen SHA.

What I am missing is the convention on how to choose the merge-base in step 1.

I am interesting in building sources so that I can try to generate patches on top of these sources in advance, and have myself enough time to resolve possible conflicts.

Thanks,
Sasha



From: NumPy-Discussion [mailto:numpy-discussion-bounces+oleksandr.pavlyk=intel.com at python.org] On Behalf Of Robert Kern
Sent: Thursday, November 29, 2018 1:59 PM
To: Discussion of Numerical Python <numpy-discussion at python.org>
Subject: Re: [Numpy-discussion] How to get sources of the current state of 1.16?

There is a GitHub Milestone for 1.16: https://github.com/numpy/numpy/milestone/58

On Thu, Nov 29, 2018 at 11:54 AM Pavlyk, Oleksandr <oleksandr.pavlyk at intel.com<mailto:oleksandr.pavlyk at intel.com>> wrote:
Are they the same as the current master, or is there a mechanism to query issue tracker for all PRs designated to make it into 1.16?

Even then, what should the base be?

Perhaps this is already documented in either an issue or a PR.

Thanks for the pointer.

Sasha
_______________________________________________
NumPy-Discussion mailing list
NumPy-Discussion at python.org<mailto:NumPy-Discussion at python.org>
https://mail.python.org/mailman/listinfo/numpy-discussion


--
Robert Kern
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20181129/b4a8c5df/attachment-0001.html>

From robert.kern at gmail.com  Thu Nov 29 18:00:45 2018
From: robert.kern at gmail.com (Robert Kern)
Date: Thu, 29 Nov 2018 15:00:45 -0800
Subject: [Numpy-discussion] How to get sources of the current state of
 1.16?
In-Reply-To: <4C9EDA7282E297428F3986994EB0FBD30606D4E8@ORSMSX108.amr.corp.intel.com>
References: <4C9EDA7282E297428F3986994EB0FBD30606D40B@ORSMSX108.amr.corp.intel.com>
 <CAF6FJisDx7d7uGvwiLttmw-YRNzQ6egqnPRsVFb5-qHfrU0HSw@mail.gmail.com>
 <4C9EDA7282E297428F3986994EB0FBD30606D4E8@ORSMSX108.amr.corp.intel.com>
Message-ID: <CAF6FJitUDMkafrvd31L7ihTTDUu6xm7_FJgM=vkPFvna+q1ojQ@mail.gmail.com>

On Thu, Nov 29, 2018 at 2:38 PM Pavlyk, Oleksandr <
oleksandr.pavlyk at intel.com> wrote:

> Sorry for not being more explicit.
>
>
>
> Yes, the milestone gives me a set of PRs whose changes are designated for
> 1.16
>
>
>
> If I were to manually build sources for 1.16 candidate I would
>
> 1.       create a branch at some SHA in the master,
>
> 2.       cherry-pick commits from PRs enumerated in milestone/58
> not reachable from the chosen SHA.
>
>
>
> What I am missing is the convention on how to choose the merge-base in
> step 1.
>
>
>
> I am interesting in building sources so that I can try to generate patches
> on top of these sources in advance, and have myself enough time to resolve
> possible conflicts.
>

Just master right now. At some point, there will be a release branch, but
that will be much closer to the release when most of the PRs are merged
already merged and it's just last minute fixes and release-specific changes
(e.g. bumping version numbers, etc.) to do.

-- 
Robert Kern
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20181129/8219b8a0/attachment.html>

From charlesr.harris at gmail.com  Thu Nov 29 22:09:54 2018
From: charlesr.harris at gmail.com (Charles R Harris)
Date: Thu, 29 Nov 2018 19:09:54 -0800
Subject: [Numpy-discussion] How to get sources of the current state of
 1.16?
In-Reply-To: <4C9EDA7282E297428F3986994EB0FBD30606D40B@ORSMSX108.amr.corp.intel.com>
References: <4C9EDA7282E297428F3986994EB0FBD30606D40B@ORSMSX108.amr.corp.intel.com>
Message-ID: <CAB6mnxJF-0LNzkvHpMt4kSg+tk4_o8jxLjEY0-g5EAL00trASQ@mail.gmail.com>

On Thu, Nov 29, 2018 at 11:53 AM Pavlyk, Oleksandr <
oleksandr.pavlyk at intel.com> wrote:

> Are they the same as the current master, or is there a mechanism to query
> issue tracker for all PRs designated to make it into 1.16?
>
>
>
> Even then, what should the base be?
>
>
Current master is the 1.16 development branch and everything in it will
become part of 1.16 when the 1.16.x branch is made. Master will then become
the 1.17 development branch. All PRs start in master.

Chuck
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20181129/81e6a291/attachment.html>