From thomas.robitaille at gmail.com  Tue Oct  1 03:01:40 2013
From: thomas.robitaille at gmail.com (Thomas Robitaille)
Date: Tue, 1 Oct 2013 09:01:40 +0200
Subject: [Numpy-discussion] Issue with np.median and array subclasses in
	1.8.0rc (worked with 1.7.0)
Message-ID: <CAGMHX_3hhzmH43vjwpzxwA0pZ6FAXWrTv-AVQYKUmRDV=M9fQQ@mail.gmail.com>

Hi,

The behavior for ``np.median`` and array sub-classes has changed in
1.8.0rc, which breaks unit-handling code (such as the ``quantities``
package, or ``astropy.units``):

https://github.com/numpy/numpy/issues/3846

This previously worked from Numpy 1.5 (at least) to Numpy 1.7. Is
there a new (and better) way to override the ``np.median`` behavior?

Cheers,
Tom


From jensj at fysik.dtu.dk  Tue Oct  1 06:00:45 2013
From: jensj at fysik.dtu.dk (=?ISO-8859-1?Q?Jens_J=F8rgen_Mortensen?=)
Date: Tue, 1 Oct 2013 12:00:45 +0200
Subject: [Numpy-discussion] 1.8.0rc1
In-Reply-To: <CAB6mnxJ+WAucyTEoj4w6nN67ja=jWH3kU+b7AZ+K4M1B37HH0g@mail.gmail.com>
References: <CAB6mnxJ+WAucyTEoj4w6nN67ja=jWH3kU+b7AZ+K4M1B37HH0g@mail.gmail.com>
Message-ID: <524A9D4D.4080106@fysik.dtu.dk>

Den 30-09-2013 17:17, Charles R Harris skrev:
> Hi All,
>
> NumPy 1.8.0rc1 is up now on sourceforge 
> <http://sourceforge.net/projects/numpy/files/NumPy/1.8.0rc1/> .The 
> binary builds are included except for Python 3.3 on windows, which 
> will arrive later. Many thanks to Ralf for the binaries, and to those 
> who found and fixed the bugs in the last beta. Any remaining bugs are 
> all my fault ;) I hope this will be the last release before final, so 
> please test it thoroughly.

I seems that solve() doesn't like arrays like zeros((3, 0)).  I get this 
with 1.8.0rc1 on Ubuntu 13.04 and Python 2.7.4:

Python 2.7.4 (default, Apr 19 2013, 18:32:33)
[GCC 4.7.3] on linux2
Type "help", "copyright", "credits" or "license" for more information.
 >>> import numpy as np
 >>> np
<module 'numpy' from 
'/home/jensj/numpy-1.8.0rc1/lib/python/numpy/__init__.pyc'>
 >>> np.linalg.solve(np.eye(3), np.zeros((3, 0)))
Traceback (most recent call last):
   File "<stdin>", line 1, in <module>
   File "/home/jensj/numpy-1.8.0rc1/lib/python/numpy/linalg/linalg.py", 
line 380, in solve
     r = gufunc(a, b, signature=signature, extobj=extobj)
ValueError: cannot remove a zero-sized axis from an iterator

with 1.7.1 it works OK:

 >>> np
<module 'numpy' from '/usr/lib/python2.7/dist-packages/numpy/__init__.pyc'>
 >>> np.linalg.solve(np.eye(3), np.zeros((3, 0)))
array([], shape=(3, 0), dtype=float64)

Jens J?rgen

> Chuck

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20131001/c9080035/attachment.html>

From davidmenhur at gmail.com  Tue Oct  1 06:11:12 2013
From: davidmenhur at gmail.com (=?UTF-8?B?RGHPgGlk?=)
Date: Tue, 1 Oct 2013 12:11:12 +0200
Subject: [Numpy-discussion] 1.8.0rc1
In-Reply-To: <CAB6mnxJ+WAucyTEoj4w6nN67ja=jWH3kU+b7AZ+K4M1B37HH0g@mail.gmail.com>
References: <CAB6mnxJ+WAucyTEoj4w6nN67ja=jWH3kU+b7AZ+K4M1B37HH0g@mail.gmail.com>
Message-ID: <CAJhcF=3CSuMWrJzMH4H=_7nC4h1bg=11H5R_vdcxdb+cFqCHBQ@mail.gmail.com>

On 30 September 2013 17:17, Charles R Harris <charlesr.harris at gmail.com>wrote:

> NumPy 1.8.0rc1 is up now on sourceforge<http://sourceforge.net/projects/numpy/files/NumPy/1.8.0rc1/>.The binary builds are included except for Python 3.3 on windows, which
> will arrive later. Many thanks to Ralf for the binaries, and to those who
> found and fixed the bugs in the last beta. Any remaining bugs are all my
> fault ;) I hope this will be the last release before final, so please test
> it thoroughly.
>

I installed it with

# python setup.py install

But something is wrong there:

>>> import numpy as np
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/lib64/python2.7/site-packages/numpy/__init__.py", line 137, in
<module>
    import add_newdocs
  File "/usr/lib64/python2.7/site-packages/numpy/add_newdocs.py", line 13,
in <module>
    from numpy.lib import add_newdoc
  File "/usr/lib64/python2.7/site-packages/numpy/lib/__init__.py", line 4,
in <module>
    from type_check import *
  File "/usr/lib64/python2.7/site-packages/numpy/lib/type_check.py", line
8, in <module>
    import numpy.core.numeric as _nx
  File "/usr/lib64/python2.7/site-packages/numpy/core/__init__.py", line
45, in <module>
    from numpy.testing import Tester
  File "/usr/lib64/python2.7/site-packages/numpy/testing/__init__.py", line
10, in <module>
    import decorators as dec
  File "/usr/lib64/python2.7/site-packages/numpy/testing/decorators.py",
line 19, in <module>
    from numpy.testing.utils import \
  File "/usr/lib64/python2.7/site-packages/numpy/testing/utils.py", line
12, in <module>
    from .nosetester import import_nose
  File "/usr/lib64/python2.7/site-packages/numpy/testing/nosetester.py",
line 12, in <module>
    from numpy.compat import basestring
ImportError: cannot import name basestring


I am using Python27 on Fedora 19.

$ gcc --version
gcc (GCC) 4.8.1 20130603 (Red Hat 4.8.1-1)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20131001/d188a454/attachment.html>

From davidmenhur at gmail.com  Tue Oct  1 06:18:03 2013
From: davidmenhur at gmail.com (=?UTF-8?B?RGHPgGlk?=)
Date: Tue, 1 Oct 2013 12:18:03 +0200
Subject: [Numpy-discussion] 1.8.0rc1
In-Reply-To: <CAJhcF=3CSuMWrJzMH4H=_7nC4h1bg=11H5R_vdcxdb+cFqCHBQ@mail.gmail.com>
References: <CAB6mnxJ+WAucyTEoj4w6nN67ja=jWH3kU+b7AZ+K4M1B37HH0g@mail.gmail.com>
	<CAJhcF=3CSuMWrJzMH4H=_7nC4h1bg=11H5R_vdcxdb+cFqCHBQ@mail.gmail.com>
Message-ID: <CAJhcF=0zRSEMGkgOANv2o_42eMKGYa3scqVRMxeU5Rm_C793Rg@mail.gmail.com>

Disregard that, I had not cleaned the previous installation properly.

Sorry for the noise.


On 1 October 2013 12:11, Da?id <davidmenhur at gmail.com> wrote:

>
> On 30 September 2013 17:17, Charles R Harris <charlesr.harris at gmail.com>wrote:
>
>> NumPy 1.8.0rc1 is up now on sourceforge<http://sourceforge.net/projects/numpy/files/NumPy/1.8.0rc1/>.The binary builds are included except for Python 3.3 on windows, which
>> will arrive later. Many thanks to Ralf for the binaries, and to those who
>> found and fixed the bugs in the last beta. Any remaining bugs are all my
>> fault ;) I hope this will be the last release before final, so please test
>> it thoroughly.
>>
>
> I installed it with
>
> # python setup.py install
>
> But something is wrong there:
>
> >>> import numpy as np
>
> Traceback (most recent call last):
>   File "<stdin>", line 1, in <module>
>   File "/usr/lib64/python2.7/site-packages/numpy/__init__.py", line 137,
> in <module>
>     import add_newdocs
>   File "/usr/lib64/python2.7/site-packages/numpy/add_newdocs.py", line 13,
> in <module>
>     from numpy.lib import add_newdoc
>   File "/usr/lib64/python2.7/site-packages/numpy/lib/__init__.py", line 4,
> in <module>
>     from type_check import *
>   File "/usr/lib64/python2.7/site-packages/numpy/lib/type_check.py", line
> 8, in <module>
>     import numpy.core.numeric as _nx
>   File "/usr/lib64/python2.7/site-packages/numpy/core/__init__.py", line
> 45, in <module>
>     from numpy.testing import Tester
>   File "/usr/lib64/python2.7/site-packages/numpy/testing/__init__.py",
> line 10, in <module>
>     import decorators as dec
>   File "/usr/lib64/python2.7/site-packages/numpy/testing/decorators.py",
> line 19, in <module>
>     from numpy.testing.utils import \
>   File "/usr/lib64/python2.7/site-packages/numpy/testing/utils.py", line
> 12, in <module>
>     from .nosetester import import_nose
>   File "/usr/lib64/python2.7/site-packages/numpy/testing/nosetester.py",
> line 12, in <module>
>     from numpy.compat import basestring
> ImportError: cannot import name basestring
>
>
> I am using Python27 on Fedora 19.
>
> $ gcc --version
> gcc (GCC) 4.8.1 20130603 (Red Hat 4.8.1-1)
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20131001/a6e513ac/attachment.html>

From njs at pobox.com  Tue Oct  1 06:29:01 2013
From: njs at pobox.com (Nathaniel Smith)
Date: Tue, 1 Oct 2013 11:29:01 +0100
Subject: [Numpy-discussion] 1.8.0rc1
In-Reply-To: <524A9D4D.4080106@fysik.dtu.dk>
References: <CAB6mnxJ+WAucyTEoj4w6nN67ja=jWH3kU+b7AZ+K4M1B37HH0g@mail.gmail.com>
	<524A9D4D.4080106@fysik.dtu.dk>
Message-ID: <CAPJVwBmozGuRydHD5W8QeKPHvj=LZOkn3api-9HN4pydzf1v6w@mail.gmail.com>

On Tue, Oct 1, 2013 at 11:00 AM, Jens J?rgen Mortensen
<jensj at fysik.dtu.dk> wrote:
> I seems that solve() doesn't like arrays like zeros((3, 0)).  I get this
> with 1.8.0rc1 on Ubuntu 13.04 and Python 2.7.4:

Yep, looks like a simple bug in the new gufunc stuff, thanks for
catching it. Filed here: https://github.com/numpy/numpy/issues/3847

-n


From robert.kern at gmail.com  Tue Oct  1 06:29:44 2013
From: robert.kern at gmail.com (Robert Kern)
Date: Tue, 1 Oct 2013 11:29:44 +0100
Subject: [Numpy-discussion] Masked arrays: Rationale for "False
	convention"
In-Reply-To: <CADDwiVD_ShY+3Fu7EtDJk36+t11rquS_giYAogwVJT2c5Qgd0w@mail.gmail.com>
References: <CADDwiVDxgBWnNcUPSV3k5bTx66dZ4pH5P7vMyYP5UFgTHx6w_g@mail.gmail.com>
	<CAB6mnxJifX7q3inqEQ1G9cy91LiP9p0VgKiBuq1UudWQy3CfbQ@mail.gmail.com>
	<CAMMTP+Aqh2KtBqXm5W0fWvq1ZvSqSivWX-NDCuJG-9MRH6s-QA@mail.gmail.com>
	<524A3385.1030101@hawaii.edu>
	<CADDwiVD_ShY+3Fu7EtDJk36+t11rquS_giYAogwVJT2c5Qgd0w@mail.gmail.com>
Message-ID: <CAF6FJit+kCB2e8-iPS8m3gX6PBVpr33-_xXn3hebnOLJJB2iog@mail.gmail.com>

On Tue, Oct 1, 2013 at 3:57 AM, Ond?ej ?ert?k <ondrej.certik at gmail.com>
wrote:

> I see, that makes sense. So to remember this, the rule is:
>
> "Specify elements that you want to get masked using True in 'mask'".

Yes. This convention dates back at least to the original MA package in
Numeric; I don't know if Paul Dubois stole it from any previous software.

One way to motivate the convention is to think about doing a binary
operation on masked arrays, which is really the most common kind of thing
one does with masked arrays. The mask of the result is the logical OR of
the two operand masks (barring additional masked elements from domain
violations, 0/0, etc.). I assume that the convention was decided mostly on
what was most convenient and efficient for the common internal operations
for *implementing* the masked arrays and not necessarily matching any
particular intuitions when putting data *into* the masked arrays.

--
Robert Kern
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20131001/adc0e355/attachment.html>

From sebastian at sipsolutions.net  Tue Oct  1 06:37:10 2013
From: sebastian at sipsolutions.net (Sebastian Berg)
Date: Tue, 01 Oct 2013 12:37:10 +0200
Subject: [Numpy-discussion] 1.8.0rc1
In-Reply-To: <524A9D4D.4080106@fysik.dtu.dk>
References: <CAB6mnxJ+WAucyTEoj4w6nN67ja=jWH3kU+b7AZ+K4M1B37HH0g@mail.gmail.com>
	<524A9D4D.4080106@fysik.dtu.dk>
Message-ID: <1380623830.5259.2.camel@sebastian-laptop>

On Tue, 2013-10-01 at 12:00 +0200, Jens J?rgen Mortensen wrote:
> Den 30-09-2013 17:17, Charles R Harris skrev:
> 
> > Hi All,
> > 
> > 
> > NumPy 1.8.0rc1 is up now on sourceforge .The binary builds are
> > included except for Python 3.3 on windows, which will arrive later.
> > Many thanks to Ralf for the binaries, and to those who found and
> > fixed the bugs in the last beta. Any remaining bugs are all my
> > fault ;) I hope this will be the last release before final, so
> > please test it thoroughly.
> > 
> 
> I seems that solve() doesn't like arrays like zeros((3, 0)).  I get
> this with 1.8.0rc1 on Ubuntu 13.04 and Python 2.7.4:
> 
Hey,

can you create an issue on github so I don't forget it? This needs
another special case to handle with the new gufunc based code. (Added
checks there recently to fix these, but the `K=0` one is missing).

- Sebastian

> Python 2.7.4 (default, Apr 19 2013, 18:32:33) 
> [GCC 4.7.3] on linux2
> Type "help", "copyright", "credits" or "license" for more information.
> >>> import numpy as np
> >>> np
> <module 'numpy' from
> '/home/jensj/numpy-1.8.0rc1/lib/python/numpy/__init__.pyc'>
> >>> np.linalg.solve(np.eye(3), np.zeros((3, 0)))
> Traceback (most recent call last):
>   File "<stdin>", line 1, in <module>
>   File "/home/jensj/numpy-1.8.0rc1/lib/python/numpy/linalg/linalg.py",
> line 380, in solve
>     r = gufunc(a, b, signature=signature, extobj=extobj)
> ValueError: cannot remove a zero-sized axis from an iterator
> 
> with 1.7.1 it works OK:
> 
> >>> np
> <module 'numpy' from
> '/usr/lib/python2.7/dist-packages/numpy/__init__.pyc'>
> >>> np.linalg.solve(np.eye(3), np.zeros((3, 0)))
> array([], shape=(3, 0), dtype=float64)
> 
> Jens J?rgen
> 
> > Chuck
> > 
> 
> _______________________________________________
> NumPy-Discussion mailing list
> NumPy-Discussion at scipy.org
> http://mail.scipy.org/mailman/listinfo/numpy-discussion




From njs at pobox.com  Tue Oct  1 06:41:26 2013
From: njs at pobox.com (Nathaniel Smith)
Date: Tue, 1 Oct 2013 11:41:26 +0100
Subject: [Numpy-discussion] [SciPy-Dev] 1.8.0rc1
In-Reply-To: <CAB6mnxK8sbncqieEQDfiYtCWx1QM7R1Zj4PcyYu1d85+8qQzVA@mail.gmail.com>
References: <CAB6mnxJ+WAucyTEoj4w6nN67ja=jWH3kU+b7AZ+K4M1B37HH0g@mail.gmail.com>
	<CAPJVwBnyBSZ-oYbD1XRD57Wj2a4LcPHvN6T6v2n1cFKUs7qEpw@mail.gmail.com>
	<5249F26B.2040109@uci.edu>
	<CAB6mnx+KPxNtMU7BDPALrK8Dz4rfEEphoDPcKqHA096D6Yg+VQ@mail.gmail.com>
	<524A0561.4000808@uci.edu>
	<CAB6mnxLj+St9Nrac-HzCN=HTrn_F+KebwOJ8n1zsbNPZY1xBag@mail.gmail.com>
	<524A0CE3.3080502@googlemail.com> <524A1393.1040701@uci.edu>
	<CAB6mnxK-aw6Om6Yky=HGf5Otvj20uydC03dZF35ReK63vvuVJg@mail.gmail.com>
	<CAB6mnxK8sbncqieEQDfiYtCWx1QM7R1Zj4PcyYu1d85+8qQzVA@mail.gmail.com>
Message-ID: <CAPJVwBkWCdj-wpq-QeXgMVqi4DN=HvnOAi8wehmyKTtu1P=X8A@mail.gmail.com>

On Tue, Oct 1, 2013 at 3:27 AM, Charles R Harris
<charlesr.harris at gmail.com> wrote:
>>> >> On Mon, Sep 30, 2013 at 5:12 PM, Christoph Gohlke <cgohlke at uci.edu
>>> >> <mailto:cgohlke at uci.edu>> wrote:
>>> >>      >     NumPy 1.8.0rc1 looks good. All tests pass on Windows and
>>> >> most
>>> >>      3rd party
>>> >>      >     packages test OK now. Thank you.
>>> >>      >
>>> >>      >     A few tests still fail in the following packages when run
>>> >> with
>>> >>      >     numpy-MKL-1.8.0rc1-win-amd64-py3.3 compared to
>>> >>      >     numpy-MKL-1.7.1-win-amd64-py3.3:
>>> >>      >
>>> >>      >     1) Pandas 0.12.0
>>> >>      >
>>> >>      >     ```
>>> >>      >
>>> >>
>>> >> ======================================================================
>>> >>      >     FAIL: test_nansum_buglet
>>> >> (pandas.tests.test_series.TestNanops)
>>> >>      >
>>> >>
>>> >> ----------------------------------------------------------------------
>>> >>      >     Traceback (most recent call last):
>>> >>      >         File
>>> >>      "X:\Python33\lib\site-packages\pandas\tests\test_series.py",
>>> >>      >     line 254, in test_nansum_buglet
>>> >>      >           assert_almost_equal(result, 1)
>>> >>      >         File
>>> >>      "X:\Python33\lib\site-packages\pandas\util\testing.py", line
>>> >>      >     134, in assert_almost_equal
>>> >>      >           np.testing.assert_(isiterable(b))
>>> >>      >         File
>>> >>      "D:\Dev\Compile\Test\numpy-build\numpy\testing\utils.py", line
>>> >>      >     44, in assert_
>>> >>      >           raise AssertionError(msg)
>>> >>      >     AssertionError
>>> >>      >     ```
>>> >>      >
>>> >>      >     Possibly related:
>>> >>      >
>>> >>      >     ```
>>> >>      >     >>> import numpy as np
>>> >>      >     >>> from pandas import Series
>>> >>      >     >>> s = Series([0.0])
>>> >>      >     >>> result = np.nansum(s)
>>> >>      >     >>> print(result)
>>> >>      >     Traceback (most recent call last):
>>> >>      >         File "<stdin>", line 1, in <module>
>>> >>      >         File
>>> >> "X:\Python33\lib\site-packages\pandas\core\base.py", line
>>> >>      >     19, in
>>> >>      >     __str__
>>> >>      >           return self.__unicode__()
>>> >>      >         File
>>> >>      "X:\Python33\lib\site-packages\pandas\core\series.py", line
>>> >>      >     1115, in __unicode__
>>> >>      >           length=len(self) > 50,
>>> >>      >     TypeError: len() of unsized object
>>> >>      >     ```
[...]
>
> The pandas test passes for current pandas dev, so it looks like a bug on
> their end that has been taken care of.
>
> test_nansum_buglet (__main__.TestNanops) ... ok

I'm concerned about this. 0.12.0 is currently the latest pandas
release, so even if it is a bug on their side, we're going to be
converting it from a latent bug to a real bug when we release...
CC'ing pydata, do you guys have any insight into what changed here?

The code is:
  s = pandas.Series([1.0, np.nan])
  result = np.nansum(s)
With numpy 1.7.1, 'result' comes out as a np.float64. With numpy
maintenance/1.8.x, 'result' comes out as a 0-d Series object. Series
is a subclass of ndarray, but it's supposed to always be 1-d, so all
kinds of stuff blows up as soon as you have a 0-d Series object.

I'm not sure what changed in numpy's nansum; if I try this same test
with a simple no-op ndarray subclass:
  class MyArray(np.ndarray):
      pass
  np.nansum(np.array([1.0, np.nan]).view(MyArray))
then 1.7.1 and maintenance/1.8.x both act the same, and both return a
0-d MyArray, so it's not just a question of whether we remembered to
handle subclasses at all.

-n


From njs at pobox.com  Tue Oct  1 06:43:50 2013
From: njs at pobox.com (Nathaniel Smith)
Date: Tue, 1 Oct 2013 11:43:50 +0100
Subject: [Numpy-discussion] [SciPy-Dev] 1.8.0rc1
In-Reply-To: <5249F26B.2040109@uci.edu>
References: <CAB6mnxJ+WAucyTEoj4w6nN67ja=jWH3kU+b7AZ+K4M1B37HH0g@mail.gmail.com>
	<CAPJVwBnyBSZ-oYbD1XRD57Wj2a4LcPHvN6T6v2n1cFKUs7qEpw@mail.gmail.com>
	<5249F26B.2040109@uci.edu>
Message-ID: <CAPJVwBkjT0Ygk5-P3-=HRjo2=VN8fzy21ULoavSvozMhNx51+A@mail.gmail.com>

On Mon, Sep 30, 2013 at 10:51 PM, Christoph Gohlke <cgohlke at uci.edu> wrote:
> 2) Bottleneck 0.7.0
>
> https://github.com/kwgoodman/bottleneck/issues/71#issuecomment-25331701

I can't tell if these are real bugs in numpy, or tests checking that
bottleneck is bug-for-bug compatible with old numpy and we just fixed
some bugs, or what. It's clearly something to do with the
nanarg{max,min} rewrite -- @charris, do you know what's going on here?

-n


From charlesr.harris at gmail.com  Tue Oct  1 08:56:31 2013
From: charlesr.harris at gmail.com (Charles R Harris)
Date: Tue, 1 Oct 2013 06:56:31 -0600
Subject: [Numpy-discussion] [SciPy-Dev] 1.8.0rc1
In-Reply-To: <CAPJVwBkjT0Ygk5-P3-=HRjo2=VN8fzy21ULoavSvozMhNx51+A@mail.gmail.com>
References: <CAB6mnxJ+WAucyTEoj4w6nN67ja=jWH3kU+b7AZ+K4M1B37HH0g@mail.gmail.com>
	<CAPJVwBnyBSZ-oYbD1XRD57Wj2a4LcPHvN6T6v2n1cFKUs7qEpw@mail.gmail.com>
	<5249F26B.2040109@uci.edu>
	<CAPJVwBkjT0Ygk5-P3-=HRjo2=VN8fzy21ULoavSvozMhNx51+A@mail.gmail.com>
Message-ID: <CAB6mnx+4kxVdDyBZ=DNUoiCS9MaT0qVwomoa1UOm6Xvf4ej0Rw@mail.gmail.com>

On Tue, Oct 1, 2013 at 4:43 AM, Nathaniel Smith <njs at pobox.com> wrote:

> On Mon, Sep 30, 2013 at 10:51 PM, Christoph Gohlke <cgohlke at uci.edu>
> wrote:
> > 2) Bottleneck 0.7.0
> >
> > https://github.com/kwgoodman/bottleneck/issues/71#issuecomment-25331701
>
> I can't tell if these are real bugs in numpy, or tests checking that
> bottleneck is bug-for-bug compatible with old numpy and we just fixed
> some bugs, or what. It's clearly something to do with the
> nanarg{max,min} rewrite -- @charris, do you know what's going on here?
>
>
Yes ;) The previous behaviour of nanarg for all-nan axis was to cast nan to
intp when the result was an array, and return nan when a scalar. The
current behaviour is to return the most negative value of intp as an error
marker in both cases and raise a warning. It is a change in behavior, but I
think one that needs to be made.

Chuck
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20131001/829df70a/attachment.html>

From grb at skogoglandskap.no  Tue Oct  1 09:17:14 2013
From: grb at skogoglandskap.no (Graeme B. Bell)
Date: Tue, 1 Oct 2013 13:17:14 +0000
Subject: [Numpy-discussion] review/approval request
In-Reply-To: <mailman.4572.1380631866.971.numpy-discussion@scipy.org>
References: <mailman.4572.1380631866.971.numpy-discussion@scipy.org>
Message-ID: <3942095D-1770-4B51-B8DE-BE4C560A059E@skogoglandskap.no>


Regarding 1.8:

A few months ago I submitted a patch that fixes some bugs, including a major bug in select() that has been around for years, and optimised the code quite a bit.  

https://github.com/numpy/numpy/pull/3537

It would be nice to see the patch in 1.8, but it has languished for a couple of months waiting for final approval/merging. 

The patch is pretty small and the new code is not complicated. 

Are there any core devs that would have time/interest to help me get this approved/merged into numpy master? I have already adjusted the patch as suggested by njsmith to suit inclusion in master. 

Graeme. 




>>> 331701
>> 
>> I can't tell if these are real bugs in numpy, or tests checking that
>> bottleneck is bug-for-bug compatible with old numpy and we just fixed
>> some bugs, or what. It's clearly something to do with the
>> nanarg{max,min} rewrite -- @charris, do you know what's going on here?
>> 
>> 
> Yes ;) The previous behaviour of nanarg for all-nan axis was to cast nan to
> intp when the result was an array, and return nan when a scalar. The
> current behaviour is to return the most negative value of intp as an error
> marker in both cases and raise a warning. It is a change in behavior, but I
> think one that needs to be made.
> 
> Chuck
> -------------- next part --------------
> An HTML attachment was scrubbed...
> URL: http://mail.scipy.org/pipermail/numpy-discussion/attachments/20131001/829df70a/attachment.html 
> 
> ------------------------------
> 
> _______________________________________________
> NumPy-Discussion mailing list
> NumPy-Discussion at scipy.org
> http://mail.scipy.org/mailman/listinfo/numpy-discussion
> 
> 
> End of NumPy-Discussion Digest, Vol 85, Issue 2
> ***********************************************



From njs at pobox.com  Tue Oct  1 09:25:37 2013
From: njs at pobox.com (Nathaniel Smith)
Date: Tue, 1 Oct 2013 14:25:37 +0100
Subject: [Numpy-discussion] [SciPy-Dev] 1.8.0rc1
In-Reply-To: <CAB6mnx+4kxVdDyBZ=DNUoiCS9MaT0qVwomoa1UOm6Xvf4ej0Rw@mail.gmail.com>
References: <CAB6mnxJ+WAucyTEoj4w6nN67ja=jWH3kU+b7AZ+K4M1B37HH0g@mail.gmail.com>
	<CAPJVwBnyBSZ-oYbD1XRD57Wj2a4LcPHvN6T6v2n1cFKUs7qEpw@mail.gmail.com>
	<5249F26B.2040109@uci.edu>
	<CAPJVwBkjT0Ygk5-P3-=HRjo2=VN8fzy21ULoavSvozMhNx51+A@mail.gmail.com>
	<CAB6mnx+4kxVdDyBZ=DNUoiCS9MaT0qVwomoa1UOm6Xvf4ej0Rw@mail.gmail.com>
Message-ID: <CAPJVwBkKL1-r11s59zMmQ2UbArNjnEo9=a7odhOFFajzKQC=Eg@mail.gmail.com>

On Tue, Oct 1, 2013 at 1:56 PM, Charles R Harris
<charlesr.harris at gmail.com> wrote:
> On Tue, Oct 1, 2013 at 4:43 AM, Nathaniel Smith <njs at pobox.com> wrote:
>>
>> On Mon, Sep 30, 2013 at 10:51 PM, Christoph Gohlke <cgohlke at uci.edu>
>> wrote:
>> > 2) Bottleneck 0.7.0
>> >
>> > https://github.com/kwgoodman/bottleneck/issues/71#issuecomment-25331701
>>
>> I can't tell if these are real bugs in numpy, or tests checking that
>> bottleneck is bug-for-bug compatible with old numpy and we just fixed
>> some bugs, or what. It's clearly something to do with the
>> nanarg{max,min} rewrite -- @charris, do you know what's going on here?
>>
>
> Yes ;) The previous behaviour of nanarg for all-nan axis was to cast nan to
> intp when the result was an array, and return nan when a scalar. The current
> behaviour is to return the most negative value of intp as an error marker in
> both cases and raise a warning. It is a change in behavior, but I think one
> that needs to be made.

Ah, okay! I kind of lost track of the nanfunc changes by the end there.

So for the bottleneck issue, it sounds like the problem is just that
bottleneck is still emulating the old numpy behaviour in this corner
case, which isn't really a problem. So we don't really need to worry
about that, both behaviours are correct, just maybe out of sync.

I'm a little dubious about this "make up some weird value that will
*probably* blow up if people try to use it without checking, and also
raise a warning" thing, wouldn't it make more sense to just raise an
error? That's what exceptions are for? I guess I should have said
something earlier though...

-n


From njs at pobox.com  Tue Oct  1 09:48:05 2013
From: njs at pobox.com (Nathaniel Smith)
Date: Tue, 1 Oct 2013 14:48:05 +0100
Subject: [Numpy-discussion] [pydata] Re:  [SciPy-Dev] 1.8.0rc1
In-Reply-To: <9c643170-cff1-41a8-9bb3-66aec33a38e3@googlegroups.com>
References: <CAB6mnxJ+WAucyTEoj4w6nN67ja=jWH3kU+b7AZ+K4M1B37HH0g@mail.gmail.com>
	<CAPJVwBnyBSZ-oYbD1XRD57Wj2a4LcPHvN6T6v2n1cFKUs7qEpw@mail.gmail.com>
	<5249F26B.2040109@uci.edu>
	<CAB6mnx+KPxNtMU7BDPALrK8Dz4rfEEphoDPcKqHA096D6Yg+VQ@mail.gmail.com>
	<524A0561.4000808@uci.edu>
	<CAB6mnxLj+St9Nrac-HzCN=HTrn_F+KebwOJ8n1zsbNPZY1xBag@mail.gmail.com>
	<524A0CE3.3080502@googlemail.com> <524A1393.1040701@uci.edu>
	<CAB6mnxK-aw6Om6Yky=HGf5Otvj20uydC03dZF35ReK63vvuVJg@mail.gmail.com>
	<CAB6mnxK8sbncqieEQDfiYtCWx1QM7R1Zj4PcyYu1d85+8qQzVA@mail.gmail.com>
	<CAPJVwBkWCdj-wpq-QeXgMVqi4DN=HvnOAi8wehmyKTtu1P=X8A@mail.gmail.com>
	<9c643170-cff1-41a8-9bb3-66aec33a38e3@googlegroups.com>
Message-ID: <CAPJVwBmSZKfLEV7-+hoccnHjpv8kVyORupMDdjHvOgMMayAKhA@mail.gmail.com>

On Tue, Oct 1, 2013 at 2:39 PM, Jeff <jeffreback at gmail.com> wrote:
> Just built from numpy master looks ok
>
> In [7]: pd.__version__
> Out[7]: '0.12.0'
>
> In [8]: np.__version__
> Out[8]: '1.9.0.dev-74abfa2'
>
> In [9]: s = pandas.Series([1.0, np.nan])
>
> In [10]: np.nansum(s)
> Out[10]: 1.0
>
> In [11]: type(np.nansum(s))
> Out[11]: numpy.float64

Yeah, I can confirm this...

With pandas 0.12.0, this code:

  np.nansum(pandas.Series([1.0, np.nan]))

works correctly using current master (74abfa27), but is broken with
current maintenance/1.8.x (f5fe91b9).

-n

> On Tuesday, October 1, 2013 6:41:26 AM UTC-4, Nathaniel Smith wrote:
>>
>> On Tue, Oct 1, 2013 at 3:27 AM, Charles R Harris
>> <charles... at gmail.com> wrote:
>> >>> >> On Mon, Sep 30, 2013 at 5:12 PM, Christoph Gohlke <cgo... at uci.edu
>> >>> >> <mailto:cgo... at uci.edu>> wrote:
>> >>> >>      >     NumPy 1.8.0rc1 looks good. All tests pass on Windows and
>> >>> >> most
>> >>> >>      3rd party
>> >>> >>      >     packages test OK now. Thank you.
>> >>> >>      >
>> >>> >>      >     A few tests still fail in the following packages when
>> >>> >> run
>> >>> >> with
>> >>> >>      >     numpy-MKL-1.8.0rc1-win-amd64-py3.3 compared to
>> >>> >>      >     numpy-MKL-1.7.1-win-amd64-py3.3:
>> >>> >>      >
>> >>> >>      >     1) Pandas 0.12.0
>> >>> >>      >
>> >>> >>      >     ```
>> >>> >>      >
>> >>> >>
>> >>> >>
>> >>> >> ======================================================================
>> >>> >>      >     FAIL: test_nansum_buglet
>> >>> >> (pandas.tests.test_series.TestNanops)
>> >>> >>      >
>> >>> >>
>> >>> >>
>> >>> >> ----------------------------------------------------------------------
>> >>> >>      >     Traceback (most recent call last):
>> >>> >>      >         File
>> >>> >>      "X:\Python33\lib\site-packages\pandas\tests\test_series.py",
>> >>> >>      >     line 254, in test_nansum_buglet
>> >>> >>      >           assert_almost_equal(result, 1)
>> >>> >>      >         File
>> >>> >>      "X:\Python33\lib\site-packages\pandas\util\testing.py", line
>> >>> >>      >     134, in assert_almost_equal
>> >>> >>      >           np.testing.assert_(isiterable(b))
>> >>> >>      >         File
>> >>> >>      "D:\Dev\Compile\Test\numpy-build\numpy\testing\utils.py", line
>> >>> >>      >     44, in assert_
>> >>> >>      >           raise AssertionError(msg)
>> >>> >>      >     AssertionError
>> >>> >>      >     ```
>> >>> >>      >
>> >>> >>      >     Possibly related:
>> >>> >>      >
>> >>> >>      >     ```
>> >>> >>      >     >>> import numpy as np
>> >>> >>      >     >>> from pandas import Series
>> >>> >>      >     >>> s = Series([0.0])
>> >>> >>      >     >>> result = np.nansum(s)
>> >>> >>      >     >>> print(result)
>> >>> >>      >     Traceback (most recent call last):
>> >>> >>      >         File "<stdin>", line 1, in <module>
>> >>> >>      >         File
>> >>> >> "X:\Python33\lib\site-packages\pandas\core\base.py", line
>> >>> >>      >     19, in
>> >>> >>      >     __str__
>> >>> >>      >           return self.__unicode__()
>> >>> >>      >         File
>> >>> >>      "X:\Python33\lib\site-packages\pandas\core\series.py", line
>> >>> >>      >     1115, in __unicode__
>> >>> >>      >           length=len(self) > 50,
>> >>> >>      >     TypeError: len() of unsized object
>> >>> >>      >     ```
>> [...]
>> >
>> > The pandas test passes for current pandas dev, so it looks like a bug on
>> > their end that has been taken care of.
>> >
>> > test_nansum_buglet (__main__.TestNanops) ... ok
>>
>> I'm concerned about this. 0.12.0 is currently the latest pandas
>> release, so even if it is a bug on their side, we're going to be
>> converting it from a latent bug to a real bug when we release...
>> CC'ing pydata, do you guys have any insight into what changed here?
>>
>> The code is:
>>   s = pandas.Series([1.0, np.nan])
>>   result = np.nansum(s)
>> With numpy 1.7.1, 'result' comes out as a np.float64. With numpy
>> maintenance/1.8.x, 'result' comes out as a 0-d Series object. Series
>> is a subclass of ndarray, but it's supposed to always be 1-d, so all
>> kinds of stuff blows up as soon as you have a 0-d Series object.
>>
>> I'm not sure what changed in numpy's nansum; if I try this same test
>> with a simple no-op ndarray subclass:
>>   class MyArray(np.ndarray):
>>       pass
>>   np.nansum(np.array([1.0, np.nan]).view(MyArray))
>> then 1.7.1 and maintenance/1.8.x both act the same, and both return a
>> 0-d MyArray, so it's not just a question of whether we remembered to
>> handle subclasses at all.
>>
>> -n
>
> --
> You received this message because you are subscribed to the Google Groups
> "PyData" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to pydata+unsubscribe at googlegroups.com.
> For more options, visit https://groups.google.com/groups/opt_out.


From charlesr.harris at gmail.com  Tue Oct  1 09:52:33 2013
From: charlesr.harris at gmail.com (Charles R Harris)
Date: Tue, 1 Oct 2013 07:52:33 -0600
Subject: [Numpy-discussion] [SciPy-Dev] 1.8.0rc1
In-Reply-To: <CAPJVwBkKL1-r11s59zMmQ2UbArNjnEo9=a7odhOFFajzKQC=Eg@mail.gmail.com>
References: <CAB6mnxJ+WAucyTEoj4w6nN67ja=jWH3kU+b7AZ+K4M1B37HH0g@mail.gmail.com>
	<CAPJVwBnyBSZ-oYbD1XRD57Wj2a4LcPHvN6T6v2n1cFKUs7qEpw@mail.gmail.com>
	<5249F26B.2040109@uci.edu>
	<CAPJVwBkjT0Ygk5-P3-=HRjo2=VN8fzy21ULoavSvozMhNx51+A@mail.gmail.com>
	<CAB6mnx+4kxVdDyBZ=DNUoiCS9MaT0qVwomoa1UOm6Xvf4ej0Rw@mail.gmail.com>
	<CAPJVwBkKL1-r11s59zMmQ2UbArNjnEo9=a7odhOFFajzKQC=Eg@mail.gmail.com>
Message-ID: <CAB6mnxL_2=Pc7ZF2Yy8odFTA4K=E2jH=xZJ6YbhRQHDDJs8NOg@mail.gmail.com>

On Tue, Oct 1, 2013 at 7:25 AM, Nathaniel Smith <njs at pobox.com> wrote:

> On Tue, Oct 1, 2013 at 1:56 PM, Charles R Harris
> <charlesr.harris at gmail.com> wrote:
> > On Tue, Oct 1, 2013 at 4:43 AM, Nathaniel Smith <njs at pobox.com> wrote:
> >>
> >> On Mon, Sep 30, 2013 at 10:51 PM, Christoph Gohlke <cgohlke at uci.edu>
> >> wrote:
> >> > 2) Bottleneck 0.7.0
> >> >
> >> >
> https://github.com/kwgoodman/bottleneck/issues/71#issuecomment-25331701
> >>
> >> I can't tell if these are real bugs in numpy, or tests checking that
> >> bottleneck is bug-for-bug compatible with old numpy and we just fixed
> >> some bugs, or what. It's clearly something to do with the
> >> nanarg{max,min} rewrite -- @charris, do you know what's going on here?
> >>
> >
> > Yes ;) The previous behaviour of nanarg for all-nan axis was to cast nan
> to
> > intp when the result was an array, and return nan when a scalar. The
> current
> > behaviour is to return the most negative value of intp as an error
> marker in
> > both cases and raise a warning. It is a change in behavior, but I think
> one
> > that needs to be made.
>
> Ah, okay! I kind of lost track of the nanfunc changes by the end there.
>
> So for the bottleneck issue, it sounds like the problem is just that
> bottleneck is still emulating the old numpy behaviour in this corner
> case, which isn't really a problem. So we don't really need to worry
> about that, both behaviours are correct, just maybe out of sync.
>
> I'm a little dubious about this "make up some weird value that will
> *probably* blow up if people try to use it without checking, and also
> raise a warning" thing, wouldn't it make more sense to just raise an
> error? That's what exceptions are for? I guess I should have said
> something earlier though...
>
>
I figure the blowup is safe, as we can't allocate arrays big enough that
the minimum intp value would be a valid index. I considered raising an
error, and if there is a consensus the behavior could be changed. Or we
could add a keyword to determine the behavior.

Chuck
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20131001/6eae5808/attachment.html>

From njs at pobox.com  Tue Oct  1 09:58:24 2013
From: njs at pobox.com (Nathaniel Smith)
Date: Tue, 1 Oct 2013 14:58:24 +0100
Subject: [Numpy-discussion] [pydata] Re:  [SciPy-Dev] 1.8.0rc1
In-Reply-To: <d7825e00-2c41-4835-a414-64c419b2c848@googlegroups.com>
References: <CAB6mnxJ+WAucyTEoj4w6nN67ja=jWH3kU+b7AZ+K4M1B37HH0g@mail.gmail.com>
	<CAPJVwBnyBSZ-oYbD1XRD57Wj2a4LcPHvN6T6v2n1cFKUs7qEpw@mail.gmail.com>
	<5249F26B.2040109@uci.edu>
	<CAB6mnx+KPxNtMU7BDPALrK8Dz4rfEEphoDPcKqHA096D6Yg+VQ@mail.gmail.com>
	<524A0561.4000808@uci.edu>
	<CAB6mnxLj+St9Nrac-HzCN=HTrn_F+KebwOJ8n1zsbNPZY1xBag@mail.gmail.com>
	<524A0CE3.3080502@googlemail.com> <524A1393.1040701@uci.edu>
	<CAB6mnxK-aw6Om6Yky=HGf5Otvj20uydC03dZF35ReK63vvuVJg@mail.gmail.com>
	<CAB6mnxK8sbncqieEQDfiYtCWx1QM7R1Zj4PcyYu1d85+8qQzVA@mail.gmail.com>
	<CAPJVwBkWCdj-wpq-QeXgMVqi4DN=HvnOAi8wehmyKTtu1P=X8A@mail.gmail.com>
	<d7825e00-2c41-4835-a414-64c419b2c848@googlegroups.com>
Message-ID: <CAPJVwBku21StXbjhLvFfqynHg9igPuq61guG+232uPNLygygpg@mail.gmail.com>

I bet the difference is:

In master, nansum ultimately calls arr.sum(...), which will be
intercepted by Series.sum.

In 1.8.x, nansum ultimately calls np.add.reduce(...), which can't be
intercepted and will return the wrong thing.

AFAICT the np.add.reduce(a, ...) call could just be replaced with
a.sum(...), but I might be missing something... surely there must have
been some reason it was written that way in the first place?

On Tue, Oct 1, 2013 at 2:48 PM, Jeff <jeffreback at gmail.com> wrote:
> so...looks like a backport issue?
>
>
> On Tuesday, October 1, 2013 6:41:26 AM UTC-4, Nathaniel Smith wrote:
>>
>> On Tue, Oct 1, 2013 at 3:27 AM, Charles R Harris
>> <charles... at gmail.com> wrote:
>> >>> >> On Mon, Sep 30, 2013 at 5:12 PM, Christoph Gohlke <cgo... at uci.edu
>> >>> >> <mailto:cgo... at uci.edu>> wrote:
>> >>> >>      >     NumPy 1.8.0rc1 looks good. All tests pass on Windows and
>> >>> >> most
>> >>> >>      3rd party
>> >>> >>      >     packages test OK now. Thank you.
>> >>> >>      >
>> >>> >>      >     A few tests still fail in the following packages when
>> >>> >> run
>> >>> >> with
>> >>> >>      >     numpy-MKL-1.8.0rc1-win-amd64-py3.3 compared to
>> >>> >>      >     numpy-MKL-1.7.1-win-amd64-py3.3:
>> >>> >>      >
>> >>> >>      >     1) Pandas 0.12.0
>> >>> >>      >
>> >>> >>      >     ```
>> >>> >>      >
>> >>> >>
>> >>> >>
>> >>> >> ======================================================================
>> >>> >>      >     FAIL: test_nansum_buglet
>> >>> >> (pandas.tests.test_series.TestNanops)
>> >>> >>      >
>> >>> >>
>> >>> >>
>> >>> >> ----------------------------------------------------------------------
>> >>> >>      >     Traceback (most recent call last):
>> >>> >>      >         File
>> >>> >>      "X:\Python33\lib\site-packages\pandas\tests\test_series.py",
>> >>> >>      >     line 254, in test_nansum_buglet
>> >>> >>      >           assert_almost_equal(result, 1)
>> >>> >>      >         File
>> >>> >>      "X:\Python33\lib\site-packages\pandas\util\testing.py", line
>> >>> >>      >     134, in assert_almost_equal
>> >>> >>      >           np.testing.assert_(isiterable(b))
>> >>> >>      >         File
>> >>> >>      "D:\Dev\Compile\Test\numpy-build\numpy\testing\utils.py", line
>> >>> >>      >     44, in assert_
>> >>> >>      >           raise AssertionError(msg)
>> >>> >>      >     AssertionError
>> >>> >>      >     ```
>> >>> >>      >
>> >>> >>      >     Possibly related:
>> >>> >>      >
>> >>> >>      >     ```
>> >>> >>      >     >>> import numpy as np
>> >>> >>      >     >>> from pandas import Series
>> >>> >>      >     >>> s = Series([0.0])
>> >>> >>      >     >>> result = np.nansum(s)
>> >>> >>      >     >>> print(result)
>> >>> >>      >     Traceback (most recent call last):
>> >>> >>      >         File "<stdin>", line 1, in <module>
>> >>> >>      >         File
>> >>> >> "X:\Python33\lib\site-packages\pandas\core\base.py", line
>> >>> >>      >     19, in
>> >>> >>      >     __str__
>> >>> >>      >           return self.__unicode__()
>> >>> >>      >         File
>> >>> >>      "X:\Python33\lib\site-packages\pandas\core\series.py", line
>> >>> >>      >     1115, in __unicode__
>> >>> >>      >           length=len(self) > 50,
>> >>> >>      >     TypeError: len() of unsized object
>> >>> >>      >     ```
>> [...]
>> >
>> > The pandas test passes for current pandas dev, so it looks like a bug on
>> > their end that has been taken care of.
>> >
>> > test_nansum_buglet (__main__.TestNanops) ... ok
>>
>> I'm concerned about this. 0.12.0 is currently the latest pandas
>> release, so even if it is a bug on their side, we're going to be
>> converting it from a latent bug to a real bug when we release...
>> CC'ing pydata, do you guys have any insight into what changed here?
>>
>> The code is:
>>   s = pandas.Series([1.0, np.nan])
>>   result = np.nansum(s)
>> With numpy 1.7.1, 'result' comes out as a np.float64. With numpy
>> maintenance/1.8.x, 'result' comes out as a 0-d Series object. Series
>> is a subclass of ndarray, but it's supposed to always be 1-d, so all
>> kinds of stuff blows up as soon as you have a 0-d Series object.
>>
>> I'm not sure what changed in numpy's nansum; if I try this same test
>> with a simple no-op ndarray subclass:
>>   class MyArray(np.ndarray):
>>       pass
>>   np.nansum(np.array([1.0, np.nan]).view(MyArray))
>> then 1.7.1 and maintenance/1.8.x both act the same, and both return a
>> 0-d MyArray, so it's not just a question of whether we remembered to
>> handle subclasses at all.
>>
>> -n
>
> --
> You received this message because you are subscribed to the Google Groups
> "PyData" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to pydata+unsubscribe at googlegroups.com.
> For more options, visit https://groups.google.com/groups/opt_out.


From njs at pobox.com  Tue Oct  1 10:12:26 2013
From: njs at pobox.com (Nathaniel Smith)
Date: Tue, 1 Oct 2013 15:12:26 +0100
Subject: [Numpy-discussion] nanargmax failure case (was: Re: [SciPy-Dev]
	1.8.0rc1)
Message-ID: <CAPJVwBkcp_Cy0=Q5zGXR=W4Vhio4r8cat-j0ZzYUjPmGGuwRtA@mail.gmail.com>

[switching subject to break out from the giant 1.8.0rc1 thread]

On Tue, Oct 1, 2013 at 2:52 PM, Charles R Harris
<charlesr.harris at gmail.com> wrote:
>
>
>
> On Tue, Oct 1, 2013 at 7:25 AM, Nathaniel Smith <njs at pobox.com> wrote:
>>
>> On Tue, Oct 1, 2013 at 1:56 PM, Charles R Harris
>> <charlesr.harris at gmail.com> wrote:
>> > On Tue, Oct 1, 2013 at 4:43 AM, Nathaniel Smith <njs at pobox.com> wrote:
>> >>
>> >> On Mon, Sep 30, 2013 at 10:51 PM, Christoph Gohlke <cgohlke at uci.edu>
>> >> wrote:
>> >> > 2) Bottleneck 0.7.0
>> >> >
>> >> >
>> >> > https://github.com/kwgoodman/bottleneck/issues/71#issuecomment-25331701
>> >>
>> >> I can't tell if these are real bugs in numpy, or tests checking that
>> >> bottleneck is bug-for-bug compatible with old numpy and we just fixed
>> >> some bugs, or what. It's clearly something to do with the
>> >> nanarg{max,min} rewrite -- @charris, do you know what's going on here?
>> >>
>> >
>> > Yes ;) The previous behaviour of nanarg for all-nan axis was to cast nan
>> > to
>> > intp when the result was an array, and return nan when a scalar. The
>> > current
>> > behaviour is to return the most negative value of intp as an error
>> > marker in
>> > both cases and raise a warning. It is a change in behavior, but I think
>> > one
>> > that needs to be made.
>>
>> Ah, okay! I kind of lost track of the nanfunc changes by the end there.
>>
>> So for the bottleneck issue, it sounds like the problem is just that
>> bottleneck is still emulating the old numpy behaviour in this corner
>> case, which isn't really a problem. So we don't really need to worry
>> about that, both behaviours are correct, just maybe out of sync.
>>
>> I'm a little dubious about this "make up some weird value that will
>> *probably* blow up if people try to use it without checking, and also
>> raise a warning" thing, wouldn't it make more sense to just raise an
>> error? That's what exceptions are for? I guess I should have said
>> something earlier though...
>>
>
> I figure the blowup is safe, as we can't allocate arrays big enough that the
> minimum intp value would be a valid index. I considered raising an error,
> and if there is a consensus the behavior could be changed. Or we could add a
> keyword to determine the behavior.

Yeah, the intp value can't be a valid index, so that covers 95% of
cases, but I'm worried about that other 5%. It could still pass
silently as the endpoint of a slice, or participate in some sort of
integer arithmetic calculation, etc. I assume you also share this
worry to some extent or you wouldn't have put in the warning ;-).

I guess the bigger question is, why would we *not* use the standard
method for signaling an exceptional condition here, i.e., exceptions?
That way we're 100% guaranteed that if people aren't prepared to
handle it then they'll at least know something has gone wrong, and if
they are prepared to handle it then it's very easy and standard, just
use try/except. Right now I guess you have to check for the special
value, but also do something to silence warnings, but just for that
one line? Sounds kind of complicated...

-n


From charlesr.harris at gmail.com  Tue Oct  1 10:13:37 2013
From: charlesr.harris at gmail.com (Charles R Harris)
Date: Tue, 1 Oct 2013 08:13:37 -0600
Subject: [Numpy-discussion] [pydata] Re: [SciPy-Dev] 1.8.0rc1
In-Reply-To: <CAPJVwBku21StXbjhLvFfqynHg9igPuq61guG+232uPNLygygpg@mail.gmail.com>
References: <CAB6mnxJ+WAucyTEoj4w6nN67ja=jWH3kU+b7AZ+K4M1B37HH0g@mail.gmail.com>
	<CAPJVwBnyBSZ-oYbD1XRD57Wj2a4LcPHvN6T6v2n1cFKUs7qEpw@mail.gmail.com>
	<5249F26B.2040109@uci.edu>
	<CAB6mnx+KPxNtMU7BDPALrK8Dz4rfEEphoDPcKqHA096D6Yg+VQ@mail.gmail.com>
	<524A0561.4000808@uci.edu>
	<CAB6mnxLj+St9Nrac-HzCN=HTrn_F+KebwOJ8n1zsbNPZY1xBag@mail.gmail.com>
	<524A0CE3.3080502@googlemail.com> <524A1393.1040701@uci.edu>
	<CAB6mnxK-aw6Om6Yky=HGf5Otvj20uydC03dZF35ReK63vvuVJg@mail.gmail.com>
	<CAB6mnxK8sbncqieEQDfiYtCWx1QM7R1Zj4PcyYu1d85+8qQzVA@mail.gmail.com>
	<CAPJVwBkWCdj-wpq-QeXgMVqi4DN=HvnOAi8wehmyKTtu1P=X8A@mail.gmail.com>
	<d7825e00-2c41-4835-a414-64c419b2c848@googlegroups.com>
	<CAPJVwBku21StXbjhLvFfqynHg9igPuq61guG+232uPNLygygpg@mail.gmail.com>
Message-ID: <CAB6mnxL9TC4si1-uXQ3_m-wC2mg8PrzgRqiUOArOVrYdPg_bnA@mail.gmail.com>

On Tue, Oct 1, 2013 at 7:58 AM, Nathaniel Smith <njs at pobox.com> wrote:

> I bet the difference is:
>
> In master, nansum ultimately calls arr.sum(...), which will be
> intercepted by Series.sum.
>
> In 1.8.x, nansum ultimately calls np.add.reduce(...), which can't be
> intercepted and will return the wrong thing.
>
> AFAICT the np.add.reduce(a, ...) call could just be replaced with
> a.sum(...), but I might be missing something... surely there must have
> been some reason it was written that way in the first place?
>

No good reason, just a bit more efficient. The change for current master
was because of the change in nansum behavior for empty slices. Changing the
call to a.sum is not a problem, although I confess that it seems a bit
fragile...

<snip>

Chuck
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20131001/689a2067/attachment.html>

From njs at pobox.com  Tue Oct  1 10:18:25 2013
From: njs at pobox.com (Nathaniel Smith)
Date: Tue, 1 Oct 2013 15:18:25 +0100
Subject: [Numpy-discussion] [pydata] Re: [SciPy-Dev] 1.8.0rc1
In-Reply-To: <CAB6mnxL9TC4si1-uXQ3_m-wC2mg8PrzgRqiUOArOVrYdPg_bnA@mail.gmail.com>
References: <CAB6mnxJ+WAucyTEoj4w6nN67ja=jWH3kU+b7AZ+K4M1B37HH0g@mail.gmail.com>
	<CAPJVwBnyBSZ-oYbD1XRD57Wj2a4LcPHvN6T6v2n1cFKUs7qEpw@mail.gmail.com>
	<5249F26B.2040109@uci.edu>
	<CAB6mnx+KPxNtMU7BDPALrK8Dz4rfEEphoDPcKqHA096D6Yg+VQ@mail.gmail.com>
	<524A0561.4000808@uci.edu>
	<CAB6mnxLj+St9Nrac-HzCN=HTrn_F+KebwOJ8n1zsbNPZY1xBag@mail.gmail.com>
	<524A0CE3.3080502@googlemail.com> <524A1393.1040701@uci.edu>
	<CAB6mnxK-aw6Om6Yky=HGf5Otvj20uydC03dZF35ReK63vvuVJg@mail.gmail.com>
	<CAB6mnxK8sbncqieEQDfiYtCWx1QM7R1Zj4PcyYu1d85+8qQzVA@mail.gmail.com>
	<CAPJVwBkWCdj-wpq-QeXgMVqi4DN=HvnOAi8wehmyKTtu1P=X8A@mail.gmail.com>
	<d7825e00-2c41-4835-a414-64c419b2c848@googlegroups.com>
	<CAPJVwBku21StXbjhLvFfqynHg9igPuq61guG+232uPNLygygpg@mail.gmail.com>
	<CAB6mnxL9TC4si1-uXQ3_m-wC2mg8PrzgRqiUOArOVrYdPg_bnA@mail.gmail.com>
Message-ID: <CAPJVwBnD6dX=V68FZSPRv34feG-0_bqxZjq7s0uiszw0TufgDw@mail.gmail.com>

On Tue, Oct 1, 2013 at 3:13 PM, Charles R Harris
<charlesr.harris at gmail.com> wrote:
>
>
>
> On Tue, Oct 1, 2013 at 7:58 AM, Nathaniel Smith <njs at pobox.com> wrote:
>>
>> I bet the difference is:
>>
>> In master, nansum ultimately calls arr.sum(...), which will be
>> intercepted by Series.sum.
>>
>> In 1.8.x, nansum ultimately calls np.add.reduce(...), which can't be
>> intercepted and will return the wrong thing.
>>
>> AFAICT the np.add.reduce(a, ...) call could just be replaced with
>> a.sum(...), but I might be missing something... surely there must have
>> been some reason it was written that way in the first place?
>
> No good reason, just a bit more efficient. The change for current master was
> because of the change in nansum behavior for empty slices. Changing the call
> to a.sum is not a problem, although I confess that it seems a bit fragile...

Yeah, ndarray subclassing is always fragile :-/. But hopefully
__numpy_ufunc__ will solve the problem in 1.9 and going forward...? (I
forget if it's implemented for .reduce yet.)

Filed a tracker bug here: https://github.com/numpy/numpy/issues/3849

-n


From charlesr.harris at gmail.com  Tue Oct  1 10:20:09 2013
From: charlesr.harris at gmail.com (Charles R Harris)
Date: Tue, 1 Oct 2013 08:20:09 -0600
Subject: [Numpy-discussion] nanargmax failure case (was: Re: [SciPy-Dev]
	1.8.0rc1)
In-Reply-To: <CAPJVwBkcp_Cy0=Q5zGXR=W4Vhio4r8cat-j0ZzYUjPmGGuwRtA@mail.gmail.com>
References: <CAPJVwBkcp_Cy0=Q5zGXR=W4Vhio4r8cat-j0ZzYUjPmGGuwRtA@mail.gmail.com>
Message-ID: <CAB6mnxLLgYw0Z-7t9fTN8-kSGWH16ATr=6HOpfxH8=u+d7Xt6A@mail.gmail.com>

On Tue, Oct 1, 2013 at 8:12 AM, Nathaniel Smith <njs at pobox.com> wrote:

> [switching subject to break out from the giant 1.8.0rc1 thread]
>
> On Tue, Oct 1, 2013 at 2:52 PM, Charles R Harris
> <charlesr.harris at gmail.com> wrote:
> >
> >
> >
> > On Tue, Oct 1, 2013 at 7:25 AM, Nathaniel Smith <njs at pobox.com> wrote:
> >>
> >> On Tue, Oct 1, 2013 at 1:56 PM, Charles R Harris
> >> <charlesr.harris at gmail.com> wrote:
> >> > On Tue, Oct 1, 2013 at 4:43 AM, Nathaniel Smith <njs at pobox.com>
> wrote:
> >> >>
> >> >> On Mon, Sep 30, 2013 at 10:51 PM, Christoph Gohlke <cgohlke at uci.edu>
> >> >> wrote:
> >> >> > 2) Bottleneck 0.7.0
> >> >> >
> >> >> >
> >> >> >
> https://github.com/kwgoodman/bottleneck/issues/71#issuecomment-25331701
> >> >>
> >> >> I can't tell if these are real bugs in numpy, or tests checking that
> >> >> bottleneck is bug-for-bug compatible with old numpy and we just fixed
> >> >> some bugs, or what. It's clearly something to do with the
> >> >> nanarg{max,min} rewrite -- @charris, do you know what's going on
> here?
> >> >>
> >> >
> >> > Yes ;) The previous behaviour of nanarg for all-nan axis was to cast
> nan
> >> > to
> >> > intp when the result was an array, and return nan when a scalar. The
> >> > current
> >> > behaviour is to return the most negative value of intp as an error
> >> > marker in
> >> > both cases and raise a warning. It is a change in behavior, but I
> think
> >> > one
> >> > that needs to be made.
> >>
> >> Ah, okay! I kind of lost track of the nanfunc changes by the end there.
> >>
> >> So for the bottleneck issue, it sounds like the problem is just that
> >> bottleneck is still emulating the old numpy behaviour in this corner
> >> case, which isn't really a problem. So we don't really need to worry
> >> about that, both behaviours are correct, just maybe out of sync.
> >>
> >> I'm a little dubious about this "make up some weird value that will
> >> *probably* blow up if people try to use it without checking, and also
> >> raise a warning" thing, wouldn't it make more sense to just raise an
> >> error? That's what exceptions are for? I guess I should have said
> >> something earlier though...
> >>
> >
> > I figure the blowup is safe, as we can't allocate arrays big enough that
> the
> > minimum intp value would be a valid index. I considered raising an error,
> > and if there is a consensus the behavior could be changed. Or we could
> add a
> > keyword to determine the behavior.
>
> Yeah, the intp value can't be a valid index, so that covers 95% of
> cases, but I'm worried about that other 5%. It could still pass
> silently as the endpoint of a slice, or participate in some sort of
> integer arithmetic calculation, etc. I assume you also share this
> worry to some extent or you wouldn't have put in the warning ;-).
>
> I guess the bigger question is, why would we *not* use the standard
> method for signaling an exceptional condition here, i.e., exceptions?
> That way we're 100% guaranteed that if people aren't prepared to
> handle it then they'll at least know something has gone wrong, and if
> they are prepared to handle it then it's very easy and standard, just
> use try/except. Right now I guess you have to check for the special
> value, but also do something to silence warnings, but just for that
> one line? Sounds kind of complicated...
>

The main reason was for the case of multiple axis, where some of the
results would be valid and others not. The simple thing might be to raise
an exception but keep the current return values so that users could
determine where the problem occurred.

Chuck
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20131001/87a741f4/attachment.html>

From njs at pobox.com  Tue Oct  1 10:47:14 2013
From: njs at pobox.com (Nathaniel Smith)
Date: Tue, 1 Oct 2013 15:47:14 +0100
Subject: [Numpy-discussion] nanargmax failure case (was: Re: [SciPy-Dev]
	1.8.0rc1)
In-Reply-To: <CAB6mnxLLgYw0Z-7t9fTN8-kSGWH16ATr=6HOpfxH8=u+d7Xt6A@mail.gmail.com>
References: <CAPJVwBkcp_Cy0=Q5zGXR=W4Vhio4r8cat-j0ZzYUjPmGGuwRtA@mail.gmail.com>
	<CAB6mnxLLgYw0Z-7t9fTN8-kSGWH16ATr=6HOpfxH8=u+d7Xt6A@mail.gmail.com>
Message-ID: <CAPJVwBmrD4TKW9A0fs4A9ydBJut2z2e+8m9RMdvijZ7FHYZ_zQ@mail.gmail.com>

On Tue, Oct 1, 2013 at 3:20 PM, Charles R Harris
<charlesr.harris at gmail.com> wrote:
>
>
>
> On Tue, Oct 1, 2013 at 8:12 AM, Nathaniel Smith <njs at pobox.com> wrote:
>>
>> [switching subject to break out from the giant 1.8.0rc1 thread]
>>
>> On Tue, Oct 1, 2013 at 2:52 PM, Charles R Harris
>> <charlesr.harris at gmail.com> wrote:
>> >
>> >
>> >
>> > On Tue, Oct 1, 2013 at 7:25 AM, Nathaniel Smith <njs at pobox.com> wrote:
>> >>
>> >> On Tue, Oct 1, 2013 at 1:56 PM, Charles R Harris
>> >> <charlesr.harris at gmail.com> wrote:
>> >> > On Tue, Oct 1, 2013 at 4:43 AM, Nathaniel Smith <njs at pobox.com>
>> >> > wrote:
>> >> >>
>> >> >> On Mon, Sep 30, 2013 at 10:51 PM, Christoph Gohlke <cgohlke at uci.edu>
>> >> >> wrote:
>> >> >> > 2) Bottleneck 0.7.0
>> >> >> >
>> >> >> >
>> >> >> >
>> >> >> > https://github.com/kwgoodman/bottleneck/issues/71#issuecomment-25331701
>> >> >>
>> >> >> I can't tell if these are real bugs in numpy, or tests checking that
>> >> >> bottleneck is bug-for-bug compatible with old numpy and we just
>> >> >> fixed
>> >> >> some bugs, or what. It's clearly something to do with the
>> >> >> nanarg{max,min} rewrite -- @charris, do you know what's going on
>> >> >> here?
>> >> >>
>> >> >
>> >> > Yes ;) The previous behaviour of nanarg for all-nan axis was to cast
>> >> > nan
>> >> > to
>> >> > intp when the result was an array, and return nan when a scalar. The
>> >> > current
>> >> > behaviour is to return the most negative value of intp as an error
>> >> > marker in
>> >> > both cases and raise a warning. It is a change in behavior, but I
>> >> > think
>> >> > one
>> >> > that needs to be made.
>> >>
>> >> Ah, okay! I kind of lost track of the nanfunc changes by the end there.
>> >>
>> >> So for the bottleneck issue, it sounds like the problem is just that
>> >> bottleneck is still emulating the old numpy behaviour in this corner
>> >> case, which isn't really a problem. So we don't really need to worry
>> >> about that, both behaviours are correct, just maybe out of sync.
>> >>
>> >> I'm a little dubious about this "make up some weird value that will
>> >> *probably* blow up if people try to use it without checking, and also
>> >> raise a warning" thing, wouldn't it make more sense to just raise an
>> >> error? That's what exceptions are for? I guess I should have said
>> >> something earlier though...
>> >>
>> >
>> > I figure the blowup is safe, as we can't allocate arrays big enough that
>> > the
>> > minimum intp value would be a valid index. I considered raising an
>> > error,
>> > and if there is a consensus the behavior could be changed. Or we could
>> > add a
>> > keyword to determine the behavior.
>>
>> Yeah, the intp value can't be a valid index, so that covers 95% of
>> cases, but I'm worried about that other 5%. It could still pass
>> silently as the endpoint of a slice, or participate in some sort of
>> integer arithmetic calculation, etc. I assume you also share this
>> worry to some extent or you wouldn't have put in the warning ;-).
>>
>> I guess the bigger question is, why would we *not* use the standard
>> method for signaling an exceptional condition here, i.e., exceptions?
>> That way we're 100% guaranteed that if people aren't prepared to
>> handle it then they'll at least know something has gone wrong, and if
>> they are prepared to handle it then it's very easy and standard, just
>> use try/except. Right now I guess you have to check for the special
>> value, but also do something to silence warnings, but just for that
>> one line? Sounds kind of complicated...
>
>
> The main reason was for the case of multiple axis, where some of the results
> would be valid and others not. The simple thing might be to raise an
> exception but keep the current return values so that users could determine
> where the problem occurred.

Oh, duh, yes, right, now I remember this discussion. Sorry for being slow.

In the past we've *always* raised in error in the multiple axis case,
right? Has anyone ever complained? Wanting to get all
nanargmax/nanargmin results, of which some might be errors, without
just writing a loop, seems like a pretty exotic case to me, so I'm not
sure we should optimize for it at the expense of returning
possibly-misleading results in the scalar case.

Like (I think) you say, we could get the best of both worlds by
encoding the results in the same way we do right now, but then raise
an exception and attach the results to the exception so they can be
retrieved if wanted. Kind of cumbersome, but maybe good?

This is a more general problem though of course -- we've run into it
in the gufunc linalg code too, where there's some question about you
do in e.g. chol() if some sub-matrices are positive-definite and some
are not.

Off the top of my head the general solution might be to define a
MultiError exception type that has a standard generic format for
describing such things. It'd need a mask saying which values were
valid, rather than encoding them directly into the return values --
otherwise we have the problem where nanargmax wants to use INT_MIN,
chol wants to use NaN, and maybe the next function along doesn't have
any usable flag value available at all. So probably more thought is
needed before nailing down exactly how we handle such "partial" errors
for vectorized functions.

In the short term (i.e., 1.8.0), maybe we should defer this discussion
by simply raising a regular ValueError for nanarg functions on all
errors? That's not a regression from 1.7, since 1.7 also didn't
provide any way to get at partial results in the event of an error,
and it leaves us in a good position to solve the more general problem
later.

-n


From robert.kern at gmail.com  Tue Oct  1 11:52:06 2013
From: robert.kern at gmail.com (Robert Kern)
Date: Tue, 1 Oct 2013 16:52:06 +0100
Subject: [Numpy-discussion] [SciPy-Dev] 1.8.0rc1
In-Reply-To: <l2eqeu$1l2$1@ger.gmane.org>
References: <CAB6mnxJ+WAucyTEoj4w6nN67ja=jWH3kU+b7AZ+K4M1B37HH0g@mail.gmail.com>
	<20131001132847.GA14614@bromo.med.uc.edu> <l2eqeu$1l2$1@ger.gmane.org>
Message-ID: <CAF6FJis0YBNitUmt67A9M=95FCzj=QXADgDGmz1VnfRS=0+1+Q@mail.gmail.com>

On Tue, Oct 1, 2013 at 4:41 PM, Pauli Virtanen <pav at iki.fi> wrote:
>
> Hi,
>
> 01.10.2013 16:28, Jack Howarth kirjoitti:
> [clip]
> > /sw/bin/python2.7 setup.py build
> >
> > which fails at...
> >
> > /sw/bin/gfortran -Wall -L/sw/lib
build/temp.macosx-10.6-x86_64-2.7/numpy/linalg/lapack_litemodule.o
build/temp.macosx-10.6-x86_64-2.7/numpy/linalg/lapack_lite/python_xerbla.o
-L/sw/lib -L/sw/lib/gcc4.8/lib/gcc/x86_64-apple-darwin10.8.0/4.8.1
-Lbuild/temp.macosx-10.6-x86_64-2.7 -llapack -lptf77blas -lptcblas -latlas
-lgfortran -o build/lib.macosx-10.6-x86_64-2.7/numpy/linalg/lapack_lite.so
> > Undefined symbols for architecture x86_64:
> >   "_main", referenced from:
> >       start in crt1.10.6.o
> [clip]
>
> Something is screwed up in your build environment: the `-shared` flag is
> missing from the link command.
>
> Perhaps you have set one of the the environment variables FFLAGS,
> CFLAGS, LDFLAGS?

Also the `-undefined dynamic_lookup` flag.

--
Robert Kern
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20131001/7190cc5b/attachment.html>

From robert.kern at gmail.com  Tue Oct  1 12:10:39 2013
From: robert.kern at gmail.com (Robert Kern)
Date: Tue, 1 Oct 2013 17:10:39 +0100
Subject: [Numpy-discussion] [SciPy-Dev] 1.8.0rc1
In-Reply-To: <20131001160221.GA15735@bromo.med.uc.edu>
References: <CAB6mnxJ+WAucyTEoj4w6nN67ja=jWH3kU+b7AZ+K4M1B37HH0g@mail.gmail.com>
	<20131001132847.GA14614@bromo.med.uc.edu> <l2eqeu$1l2$1@ger.gmane.org>
	<CAF6FJis0YBNitUmt67A9M=95FCzj=QXADgDGmz1VnfRS=0+1+Q@mail.gmail.com>
	<20131001160221.GA15735@bromo.med.uc.edu>
Message-ID: <CAF6FJisSxJHOTLNOKPd64Qus-8A4n_EKpWnhWAM=Vz-V0q-Hwg@mail.gmail.com>

On Tue, Oct 1, 2013 at 5:02 PM, Jack Howarth <howarth at bromo.med.uc.edu>
wrote:
>
> On Tue, Oct 01, 2013 at 04:52:06PM +0100, Robert Kern wrote:
> > On Tue, Oct 1, 2013 at 4:41 PM, Pauli Virtanen <pav at iki.fi> wrote:
> > >
> > > Hi,
> > >
> > > 01.10.2013 16:28, Jack Howarth kirjoitti:
> > > [clip]
> > > > /sw/bin/python2.7 setup.py build
> > > >
> > > > which fails at...
> > > >
> > > > /sw/bin/gfortran -Wall -L/sw/lib
> > build/temp.macosx-10.6-x86_64-2.7/numpy/linalg/lapack_litemodule.o
> >
build/temp.macosx-10.6-x86_64-2.7/numpy/linalg/lapack_lite/python_xerbla.o
> > -L/sw/lib -L/sw/lib/gcc4.8/lib/gcc/x86_64-apple-darwin10.8.0/4.8.1
> > -Lbuild/temp.macosx-10.6-x86_64-2.7 -llapack -lptf77blas -lptcblas
-latlas
> > -lgfortran -o
build/lib.macosx-10.6-x86_64-2.7/numpy/linalg/lapack_lite.so
> > > > Undefined symbols for architecture x86_64:
> > > >   "_main", referenced from:
> > > >       start in crt1.10.6.o
> > > [clip]
> > >
> > > Something is screwed up in your build environment: the `-shared` flag
is
> > > missing from the link command.
> > >
> > > Perhaps you have set one of the the environment variables FFLAGS,
> > > CFLAGS, LDFLAGS?
> >
> > Also the `-undefined dynamic_lookup` flag.
>
> The consensus of the fink developers is that you are introducing a bug in
both
> scipy and numpy. The build should be able to pass additional flags on
these
> variables and the scipy/numpy build should be able to append any
additional
> flags required. In particular, both MacPorts and fink will want to be
able to
> pass -L/opt/local/lib or -L/sw/lib via LDFLAGS. The changes added to
scipy and
> numpy have broken this and now require that these additional flags be
manually
> patched into the Makefiles of numpy and scipy rather than just passing
them
> on LDFLAGS as has always worked in the past.

Oh no it hasn't. It has been a consistent thorn in our side for a very long
time. In the case of Fortran modules built by numpy.distutils, $LDFLAGS has
replaced rather than appended flags since time immemorial. It is a
compromise solution to work around the fact that the wide variety of
Fortran compilers are very finicky about their flags, and distutils is not
very accommodating about letting users change the flags to suit their local
environments. If you think you have a better solution to this problem that
does not degrade the existing flexibility, your PR will be cheerfully
accepted. No one thinks this is desirable behavior, but it is most
certainly not *new* behavior.

--
Robert Kern
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20131001/a9797d0d/attachment.html>

From josef.pktd at gmail.com  Tue Oct  1 12:19:14 2013
From: josef.pktd at gmail.com (josef.pktd at gmail.com)
Date: Tue, 1 Oct 2013 12:19:14 -0400
Subject: [Numpy-discussion] nanargmax failure case (was: Re: [SciPy-Dev]
	1.8.0rc1)
In-Reply-To: <CAPJVwBmrD4TKW9A0fs4A9ydBJut2z2e+8m9RMdvijZ7FHYZ_zQ@mail.gmail.com>
References: <CAPJVwBkcp_Cy0=Q5zGXR=W4Vhio4r8cat-j0ZzYUjPmGGuwRtA@mail.gmail.com>
	<CAB6mnxLLgYw0Z-7t9fTN8-kSGWH16ATr=6HOpfxH8=u+d7Xt6A@mail.gmail.com>
	<CAPJVwBmrD4TKW9A0fs4A9ydBJut2z2e+8m9RMdvijZ7FHYZ_zQ@mail.gmail.com>
Message-ID: <CAMMTP+CuXfv0xm7axL7wtX-VuxSOUXDFLMtajj2UyPyUsAd+uQ@mail.gmail.com>

On Tue, Oct 1, 2013 at 10:47 AM, Nathaniel Smith <njs at pobox.com> wrote:
> On Tue, Oct 1, 2013 at 3:20 PM, Charles R Harris
> <charlesr.harris at gmail.com> wrote:
>>
>>
>>
>> On Tue, Oct 1, 2013 at 8:12 AM, Nathaniel Smith <njs at pobox.com> wrote:
>>>
>>> [switching subject to break out from the giant 1.8.0rc1 thread]
>>>
>>> On Tue, Oct 1, 2013 at 2:52 PM, Charles R Harris
>>> <charlesr.harris at gmail.com> wrote:
>>> >
>>> >
>>> >
>>> > On Tue, Oct 1, 2013 at 7:25 AM, Nathaniel Smith <njs at pobox.com> wrote:
>>> >>
>>> >> On Tue, Oct 1, 2013 at 1:56 PM, Charles R Harris
>>> >> <charlesr.harris at gmail.com> wrote:
>>> >> > On Tue, Oct 1, 2013 at 4:43 AM, Nathaniel Smith <njs at pobox.com>
>>> >> > wrote:
>>> >> >>
>>> >> >> On Mon, Sep 30, 2013 at 10:51 PM, Christoph Gohlke <cgohlke at uci.edu>
>>> >> >> wrote:
>>> >> >> > 2) Bottleneck 0.7.0
>>> >> >> >
>>> >> >> >
>>> >> >> >
>>> >> >> > https://github.com/kwgoodman/bottleneck/issues/71#issuecomment-25331701
>>> >> >>
>>> >> >> I can't tell if these are real bugs in numpy, or tests checking that
>>> >> >> bottleneck is bug-for-bug compatible with old numpy and we just
>>> >> >> fixed
>>> >> >> some bugs, or what. It's clearly something to do with the
>>> >> >> nanarg{max,min} rewrite -- @charris, do you know what's going on
>>> >> >> here?
>>> >> >>
>>> >> >
>>> >> > Yes ;) The previous behaviour of nanarg for all-nan axis was to cast
>>> >> > nan
>>> >> > to
>>> >> > intp when the result was an array, and return nan when a scalar. The
>>> >> > current
>>> >> > behaviour is to return the most negative value of intp as an error
>>> >> > marker in
>>> >> > both cases and raise a warning. It is a change in behavior, but I
>>> >> > think
>>> >> > one
>>> >> > that needs to be made.
>>> >>
>>> >> Ah, okay! I kind of lost track of the nanfunc changes by the end there.
>>> >>
>>> >> So for the bottleneck issue, it sounds like the problem is just that
>>> >> bottleneck is still emulating the old numpy behaviour in this corner
>>> >> case, which isn't really a problem. So we don't really need to worry
>>> >> about that, both behaviours are correct, just maybe out of sync.
>>> >>
>>> >> I'm a little dubious about this "make up some weird value that will
>>> >> *probably* blow up if people try to use it without checking, and also
>>> >> raise a warning" thing, wouldn't it make more sense to just raise an
>>> >> error? That's what exceptions are for? I guess I should have said
>>> >> something earlier though...
>>> >>
>>> >
>>> > I figure the blowup is safe, as we can't allocate arrays big enough that
>>> > the
>>> > minimum intp value would be a valid index. I considered raising an
>>> > error,
>>> > and if there is a consensus the behavior could be changed. Or we could
>>> > add a
>>> > keyword to determine the behavior.
>>>
>>> Yeah, the intp value can't be a valid index, so that covers 95% of
>>> cases, but I'm worried about that other 5%. It could still pass
>>> silently as the endpoint of a slice, or participate in some sort of
>>> integer arithmetic calculation, etc. I assume you also share this
>>> worry to some extent or you wouldn't have put in the warning ;-).
>>>
>>> I guess the bigger question is, why would we *not* use the standard
>>> method for signaling an exceptional condition here, i.e., exceptions?
>>> That way we're 100% guaranteed that if people aren't prepared to
>>> handle it then they'll at least know something has gone wrong, and if
>>> they are prepared to handle it then it's very easy and standard, just
>>> use try/except. Right now I guess you have to check for the special
>>> value, but also do something to silence warnings, but just for that
>>> one line? Sounds kind of complicated...
>>
>>
>> The main reason was for the case of multiple axis, where some of the results
>> would be valid and others not. The simple thing might be to raise an
>> exception but keep the current return values so that users could determine
>> where the problem occurred.
>
> Oh, duh, yes, right, now I remember this discussion. Sorry for being slow.
>
> In the past we've *always* raised in error in the multiple axis case,
> right? Has anyone ever complained? Wanting to get all
> nanargmax/nanargmin results, of which some might be errors, without
> just writing a loop, seems like a pretty exotic case to me, so I'm not
> sure we should optimize for it at the expense of returning
> possibly-misleading results in the scalar case.
>
> Like (I think) you say, we could get the best of both worlds by
> encoding the results in the same way we do right now, but then raise
> an exception and attach the results to the exception so they can be
> retrieved if wanted. Kind of cumbersome, but maybe good?
>
> This is a more general problem though of course -- we've run into it
> in the gufunc linalg code too, where there's some question about you
> do in e.g. chol() if some sub-matrices are positive-definite and some
> are not.
>
> Off the top of my head the general solution might be to define a
> MultiError exception type that has a standard generic format for
> describing such things. It'd need a mask saying which values were
> valid, rather than encoding them directly into the return values --
> otherwise we have the problem where nanargmax wants to use INT_MIN,
> chol wants to use NaN, and maybe the next function along doesn't have
> any usable flag value available at all. So probably more thought is
> needed before nailing down exactly how we handle such "partial" errors
> for vectorized functions.
>
> In the short term (i.e., 1.8.0), maybe we should defer this discussion
> by simply raising a regular ValueError for nanarg functions on all
> errors? That's not a regression from 1.7, since 1.7 also didn't
> provide any way to get at partial results in the event of an error,
> and it leaves us in a good position to solve the more general problem
> later.

Can we make the error optional in these cases?

like np.seterr for zerodivision, invalid, or floating point errors
that allows ignore and raise
np.seterr(linalg='ignore')

I don't know about nanarg, but thinking about some applications for
gufunc linalg code.

In some cases I might require for example invertibility of all
matrices and raise if one fails,
in other case I would be happy with nans, and just sum the results
with nansum for example or replace them by some fill value.

Josef

>
> -n
> _______________________________________________
> NumPy-Discussion mailing list
> NumPy-Discussion at scipy.org
> http://mail.scipy.org/mailman/listinfo/numpy-discussion


From charlesr.harris at gmail.com  Tue Oct  1 12:33:58 2013
From: charlesr.harris at gmail.com (Charles R Harris)
Date: Tue, 1 Oct 2013 10:33:58 -0600
Subject: [Numpy-discussion] nanargmax failure case (was: Re: [SciPy-Dev]
	1.8.0rc1)
In-Reply-To: <CAMMTP+CuXfv0xm7axL7wtX-VuxSOUXDFLMtajj2UyPyUsAd+uQ@mail.gmail.com>
References: <CAPJVwBkcp_Cy0=Q5zGXR=W4Vhio4r8cat-j0ZzYUjPmGGuwRtA@mail.gmail.com>
	<CAB6mnxLLgYw0Z-7t9fTN8-kSGWH16ATr=6HOpfxH8=u+d7Xt6A@mail.gmail.com>
	<CAPJVwBmrD4TKW9A0fs4A9ydBJut2z2e+8m9RMdvijZ7FHYZ_zQ@mail.gmail.com>
	<CAMMTP+CuXfv0xm7axL7wtX-VuxSOUXDFLMtajj2UyPyUsAd+uQ@mail.gmail.com>
Message-ID: <CAB6mnx+5S=E6e0cbF4VUMgVspKwBJbKQxi4V19j5GfOtMJQzDw@mail.gmail.com>

On Tue, Oct 1, 2013 at 10:19 AM, <josef.pktd at gmail.com> wrote:

> On Tue, Oct 1, 2013 at 10:47 AM, Nathaniel Smith <njs at pobox.com> wrote:
> > On Tue, Oct 1, 2013 at 3:20 PM, Charles R Harris
> > <charlesr.harris at gmail.com> wrote:
> >>
> >>
> >>
> >> On Tue, Oct 1, 2013 at 8:12 AM, Nathaniel Smith <njs at pobox.com> wrote:
> >>>
> >>> [switching subject to break out from the giant 1.8.0rc1 thread]
> >>>
> >>> On Tue, Oct 1, 2013 at 2:52 PM, Charles R Harris
> >>> <charlesr.harris at gmail.com> wrote:
> >>> >
> >>> >
> >>> >
> >>> > On Tue, Oct 1, 2013 at 7:25 AM, Nathaniel Smith <njs at pobox.com>
> wrote:
> >>> >>
> >>> >> On Tue, Oct 1, 2013 at 1:56 PM, Charles R Harris
> >>> >> <charlesr.harris at gmail.com> wrote:
> >>> >> > On Tue, Oct 1, 2013 at 4:43 AM, Nathaniel Smith <njs at pobox.com>
> >>> >> > wrote:
> >>> >> >>
> >>> >> >> On Mon, Sep 30, 2013 at 10:51 PM, Christoph Gohlke <
> cgohlke at uci.edu>
> >>> >> >> wrote:
> >>> >> >> > 2) Bottleneck 0.7.0
> >>> >> >> >
> >>> >> >> >
> >>> >> >> >
> >>> >> >> >
> https://github.com/kwgoodman/bottleneck/issues/71#issuecomment-25331701
> >>> >> >>
> >>> >> >> I can't tell if these are real bugs in numpy, or tests checking
> that
> >>> >> >> bottleneck is bug-for-bug compatible with old numpy and we just
> >>> >> >> fixed
> >>> >> >> some bugs, or what. It's clearly something to do with the
> >>> >> >> nanarg{max,min} rewrite -- @charris, do you know what's going on
> >>> >> >> here?
> >>> >> >>
> >>> >> >
> >>> >> > Yes ;) The previous behaviour of nanarg for all-nan axis was to
> cast
> >>> >> > nan
> >>> >> > to
> >>> >> > intp when the result was an array, and return nan when a scalar.
> The
> >>> >> > current
> >>> >> > behaviour is to return the most negative value of intp as an error
> >>> >> > marker in
> >>> >> > both cases and raise a warning. It is a change in behavior, but I
> >>> >> > think
> >>> >> > one
> >>> >> > that needs to be made.
> >>> >>
> >>> >> Ah, okay! I kind of lost track of the nanfunc changes by the end
> there.
> >>> >>
> >>> >> So for the bottleneck issue, it sounds like the problem is just that
> >>> >> bottleneck is still emulating the old numpy behaviour in this corner
> >>> >> case, which isn't really a problem. So we don't really need to worry
> >>> >> about that, both behaviours are correct, just maybe out of sync.
> >>> >>
> >>> >> I'm a little dubious about this "make up some weird value that will
> >>> >> *probably* blow up if people try to use it without checking, and
> also
> >>> >> raise a warning" thing, wouldn't it make more sense to just raise an
> >>> >> error? That's what exceptions are for? I guess I should have said
> >>> >> something earlier though...
> >>> >>
> >>> >
> >>> > I figure the blowup is safe, as we can't allocate arrays big enough
> that
> >>> > the
> >>> > minimum intp value would be a valid index. I considered raising an
> >>> > error,
> >>> > and if there is a consensus the behavior could be changed. Or we
> could
> >>> > add a
> >>> > keyword to determine the behavior.
> >>>
> >>> Yeah, the intp value can't be a valid index, so that covers 95% of
> >>> cases, but I'm worried about that other 5%. It could still pass
> >>> silently as the endpoint of a slice, or participate in some sort of
> >>> integer arithmetic calculation, etc. I assume you also share this
> >>> worry to some extent or you wouldn't have put in the warning ;-).
> >>>
> >>> I guess the bigger question is, why would we *not* use the standard
> >>> method for signaling an exceptional condition here, i.e., exceptions?
> >>> That way we're 100% guaranteed that if people aren't prepared to
> >>> handle it then they'll at least know something has gone wrong, and if
> >>> they are prepared to handle it then it's very easy and standard, just
> >>> use try/except. Right now I guess you have to check for the special
> >>> value, but also do something to silence warnings, but just for that
> >>> one line? Sounds kind of complicated...
> >>
> >>
> >> The main reason was for the case of multiple axis, where some of the
> results
> >> would be valid and others not. The simple thing might be to raise an
> >> exception but keep the current return values so that users could
> determine
> >> where the problem occurred.
> >
> > Oh, duh, yes, right, now I remember this discussion. Sorry for being
> slow.
> >
> > In the past we've *always* raised in error in the multiple axis case,
> > right? Has anyone ever complained? Wanting to get all
> > nanargmax/nanargmin results, of which some might be errors, without
> > just writing a loop, seems like a pretty exotic case to me, so I'm not
> > sure we should optimize for it at the expense of returning
> > possibly-misleading results in the scalar case.
> >
> > Like (I think) you say, we could get the best of both worlds by
> > encoding the results in the same way we do right now, but then raise
> > an exception and attach the results to the exception so they can be
> > retrieved if wanted. Kind of cumbersome, but maybe good?
> >
> > This is a more general problem though of course -- we've run into it
> > in the gufunc linalg code too, where there's some question about you
> > do in e.g. chol() if some sub-matrices are positive-definite and some
> > are not.
> >
> > Off the top of my head the general solution might be to define a
> > MultiError exception type that has a standard generic format for
> > describing such things. It'd need a mask saying which values were
> > valid, rather than encoding them directly into the return values --
> > otherwise we have the problem where nanargmax wants to use INT_MIN,
> > chol wants to use NaN, and maybe the next function along doesn't have
> > any usable flag value available at all. So probably more thought is
> > needed before nailing down exactly how we handle such "partial" errors
> > for vectorized functions.
> >
> > In the short term (i.e., 1.8.0), maybe we should defer this discussion
> > by simply raising a regular ValueError for nanarg functions on all
> > errors? That's not a regression from 1.7, since 1.7 also didn't
> > provide any way to get at partial results in the event of an error,
> > and it leaves us in a good position to solve the more general problem
> > later.
>
> Can we make the error optional in these cases?
>
> like np.seterr for zerodivision, invalid, or floating point errors
> that allows ignore and raise
> np.seterr(linalg='ignore')
>
> I don't know about nanarg, but thinking about some applications for
> gufunc linalg code.
>
> In some cases I might require for example invertibility of all
> matrices and raise if one fails,
> in other case I would be happy with nans, and just sum the results
> with nansum for example or replace them by some fill value.
>
> I'm thinking warnings might be more flexible than exceptions:

with warnings.catch_warnings():
    warnings.simplefilter('error')
    ...

Chuck
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20131001/ae7e7a58/attachment.html>

From ondrej.certik at gmail.com  Tue Oct  1 13:23:43 2013
From: ondrej.certik at gmail.com (=?UTF-8?B?T25kxZllaiDEjGVydMOtaw==?=)
Date: Tue, 1 Oct 2013 11:23:43 -0600
Subject: [Numpy-discussion] Masked arrays: Rationale for "False
	convention"
In-Reply-To: <524A3BDC.4070007@hawaii.edu>
References: <CADDwiVDxgBWnNcUPSV3k5bTx66dZ4pH5P7vMyYP5UFgTHx6w_g@mail.gmail.com>
	<CAB6mnxJifX7q3inqEQ1G9cy91LiP9p0VgKiBuq1UudWQy3CfbQ@mail.gmail.com>
	<CAMMTP+Aqh2KtBqXm5W0fWvq1ZvSqSivWX-NDCuJG-9MRH6s-QA@mail.gmail.com>
	<524A3385.1030101@hawaii.edu>
	<CADDwiVD_ShY+3Fu7EtDJk36+t11rquS_giYAogwVJT2c5Qgd0w@mail.gmail.com>
	<524A3BDC.4070007@hawaii.edu>
Message-ID: <CADDwiVD8E-k7yRUOopeAp=afX5VrEpnJhS+abV6YB5pXLeF+uw@mail.gmail.com>

On Tue, Oct 1, 2013 at 4:29 AM, Robert Kern <robert.kern at gmail.com> wrote:
> On Tue, Oct 1, 2013 at 3:57 AM, Ond?ej ?ert?k <ondrej.certik at gmail.com>
> wrote:
>
>> I see, that makes sense. So to remember this, the rule is:
>>
>> "Specify elements that you want to get masked using True in 'mask'".
>
> Yes. This convention dates back at least to the original MA package in
> Numeric; I don't know if Paul Dubois stole it from any previous software.

I see, thanks.

>
> One way to motivate the convention is to think about doing a binary
> operation on masked arrays, which is really the most common kind of thing
> one does with masked arrays. The mask of the result is the logical OR of the
> two operand masks (barring additional masked elements from domain
> violations, 0/0, etc.).

In the other convention, you just use logical AND, so that seams equally
simple, unless I am missing something.

> I assume that the convention was decided mostly on
> what was most convenient and efficient for the common internal operations
> for *implementing* the masked arrays and not necessarily matching any
> particular intuitions when putting data *into* the masked arrays.

That makes sense.


On Mon, Sep 30, 2013 at 9:05 PM, Eric Firing <efiring at hawaii.edu> wrote:
> On 2013/09/30 4:57 PM, Ond?ej ?ert?k wrote:
>>
>> But why do I need to invert the mask when I want to see the valid elements:
>>
>> In [1]: from numpy import ma
>>
>> In [2]: a = ma.array([1, 2, 3, 4], mask=[False, False, True, False])
>>
>> In [3]: a
>> Out[3]:
>> masked_array(data = [1 2 -- 4],
>>               mask = [False False  True False],
>>         fill_value = 999999)
>>
>>
>> In [4]: a[~a.mask]
>> Out[4]:
>> masked_array(data = [1 2 4],
>>               mask = [False False False],
>>         fill_value = 999999)
>>
>>
>> I would find natural to write [4] as a[a.mask]. This is when it gets confusing.
>
> There is no getting around it; each of the two possible conventions has
> its advantages.  But try this instead:
>
> In [2]: a = ma.array([1, 2, 3, 4], mask=[False, False, True, False])
>
> In [3]: a.compressed()
> Out[3]: array([1, 2, 4])
>
>
> I do occasionally need a "goodmask" which is the inverse of a.mask, but
> not very often; and when I do, needing to invert a.mask doesn't bother me.

a.compressed() works for getting data out --- but I also use it to
assign data in,
e.g.:

a[~a.mask] = 1




Thanks everybody for the discussion. It sheds some light onto the current
convention.

Ondrej


From njs at pobox.com  Tue Oct  1 16:13:11 2013
From: njs at pobox.com (Nathaniel Smith)
Date: Tue, 1 Oct 2013 21:13:11 +0100
Subject: [Numpy-discussion] nanargmax failure case (was: Re: [SciPy-Dev]
	1.8.0rc1)
In-Reply-To: <CAB6mnx+5S=E6e0cbF4VUMgVspKwBJbKQxi4V19j5GfOtMJQzDw@mail.gmail.com>
References: <CAPJVwBkcp_Cy0=Q5zGXR=W4Vhio4r8cat-j0ZzYUjPmGGuwRtA@mail.gmail.com>
	<CAB6mnxLLgYw0Z-7t9fTN8-kSGWH16ATr=6HOpfxH8=u+d7Xt6A@mail.gmail.com>
	<CAPJVwBmrD4TKW9A0fs4A9ydBJut2z2e+8m9RMdvijZ7FHYZ_zQ@mail.gmail.com>
	<CAMMTP+CuXfv0xm7axL7wtX-VuxSOUXDFLMtajj2UyPyUsAd+uQ@mail.gmail.com>
	<CAB6mnx+5S=E6e0cbF4VUMgVspKwBJbKQxi4V19j5GfOtMJQzDw@mail.gmail.com>
Message-ID: <CAPJVwB=13HZXkmntiuqJfHHcCbPUcF=e9QY+sANcw1q95OC6pQ@mail.gmail.com>

On 1 Oct 2013 17:34, "Charles R Harris" <charlesr.harris at gmail.com> wrote:
>
>
>
>
> On Tue, Oct 1, 2013 at 10:19 AM, <josef.pktd at gmail.com> wrote:
>>
>> On Tue, Oct 1, 2013 at 10:47 AM, Nathaniel Smith <njs at pobox.com> wrote:
>> > On Tue, Oct 1, 2013 at 3:20 PM, Charles R Harris
>> > <charlesr.harris at gmail.com> wrote:
>> >>
>> >>
>> >>
>> >> On Tue, Oct 1, 2013 at 8:12 AM, Nathaniel Smith <njs at pobox.com> wrote:
>> >>>
>> >>> [switching subject to break out from the giant 1.8.0rc1 thread]
>> >>>
>> >>> On Tue, Oct 1, 2013 at 2:52 PM, Charles R Harris
>> >>> <charlesr.harris at gmail.com> wrote:
>> >>> >
>> >>> >
>> >>> >
>> >>> > On Tue, Oct 1, 2013 at 7:25 AM, Nathaniel Smith <njs at pobox.com>
wrote:
>> >>> >>
>> >>> >> On Tue, Oct 1, 2013 at 1:56 PM, Charles R Harris
>> >>> >> <charlesr.harris at gmail.com> wrote:
>> >>> >> > On Tue, Oct 1, 2013 at 4:43 AM, Nathaniel Smith <njs at pobox.com>
>> >>> >> > wrote:
>> >>> >> >>
>> >>> >> >> On Mon, Sep 30, 2013 at 10:51 PM, Christoph Gohlke <
cgohlke at uci.edu>
>> >>> >> >> wrote:
>> >>> >> >> > 2) Bottleneck 0.7.0
>> >>> >> >> >
>> >>> >> >> >
>> >>> >> >> >
>> >>> >> >> >
https://github.com/kwgoodman/bottleneck/issues/71#issuecomment-25331701
>> >>> >> >>
>> >>> >> >> I can't tell if these are real bugs in numpy, or tests
checking that
>> >>> >> >> bottleneck is bug-for-bug compatible with old numpy and we just
>> >>> >> >> fixed
>> >>> >> >> some bugs, or what. It's clearly something to do with the
>> >>> >> >> nanarg{max,min} rewrite -- @charris, do you know what's going
on
>> >>> >> >> here?
>> >>> >> >>
>> >>> >> >
>> >>> >> > Yes ;) The previous behaviour of nanarg for all-nan axis was to
cast
>> >>> >> > nan
>> >>> >> > to
>> >>> >> > intp when the result was an array, and return nan when a
scalar. The
>> >>> >> > current
>> >>> >> > behaviour is to return the most negative value of intp as an
error
>> >>> >> > marker in
>> >>> >> > both cases and raise a warning. It is a change in behavior, but
I
>> >>> >> > think
>> >>> >> > one
>> >>> >> > that needs to be made.
>> >>> >>
>> >>> >> Ah, okay! I kind of lost track of the nanfunc changes by the end
there.
>> >>> >>
>> >>> >> So for the bottleneck issue, it sounds like the problem is just
that
>> >>> >> bottleneck is still emulating the old numpy behaviour in this
corner
>> >>> >> case, which isn't really a problem. So we don't really need to
worry
>> >>> >> about that, both behaviours are correct, just maybe out of sync.
>> >>> >>
>> >>> >> I'm a little dubious about this "make up some weird value that
will
>> >>> >> *probably* blow up if people try to use it without checking, and
also
>> >>> >> raise a warning" thing, wouldn't it make more sense to just raise
an
>> >>> >> error? That's what exceptions are for? I guess I should have said
>> >>> >> something earlier though...
>> >>> >>
>> >>> >
>> >>> > I figure the blowup is safe, as we can't allocate arrays big
enough that
>> >>> > the
>> >>> > minimum intp value would be a valid index. I considered raising an
>> >>> > error,
>> >>> > and if there is a consensus the behavior could be changed. Or we
could
>> >>> > add a
>> >>> > keyword to determine the behavior.
>> >>>
>> >>> Yeah, the intp value can't be a valid index, so that covers 95% of
>> >>> cases, but I'm worried about that other 5%. It could still pass
>> >>> silently as the endpoint of a slice, or participate in some sort of
>> >>> integer arithmetic calculation, etc. I assume you also share this
>> >>> worry to some extent or you wouldn't have put in the warning ;-).
>> >>>
>> >>> I guess the bigger question is, why would we *not* use the standard
>> >>> method for signaling an exceptional condition here, i.e., exceptions?
>> >>> That way we're 100% guaranteed that if people aren't prepared to
>> >>> handle it then they'll at least know something has gone wrong, and if
>> >>> they are prepared to handle it then it's very easy and standard, just
>> >>> use try/except. Right now I guess you have to check for the special
>> >>> value, but also do something to silence warnings, but just for that
>> >>> one line? Sounds kind of complicated...
>> >>
>> >>
>> >> The main reason was for the case of multiple axis, where some of the
results
>> >> would be valid and others not. The simple thing might be to raise an
>> >> exception but keep the current return values so that users could
determine
>> >> where the problem occurred.
>> >
>> > Oh, duh, yes, right, now I remember this discussion. Sorry for being
slow.
>> >
>> > In the past we've *always* raised in error in the multiple axis case,
>> > right? Has anyone ever complained? Wanting to get all
>> > nanargmax/nanargmin results, of which some might be errors, without
>> > just writing a loop, seems like a pretty exotic case to me, so I'm not
>> > sure we should optimize for it at the expense of returning
>> > possibly-misleading results in the scalar case.
>> >
>> > Like (I think) you say, we could get the best of both worlds by
>> > encoding the results in the same way we do right now, but then raise
>> > an exception and attach the results to the exception so they can be
>> > retrieved if wanted. Kind of cumbersome, but maybe good?
>> >
>> > This is a more general problem though of course -- we've run into it
>> > in the gufunc linalg code too, where there's some question about you
>> > do in e.g. chol() if some sub-matrices are positive-definite and some
>> > are not.
>> >
>> > Off the top of my head the general solution might be to define a
>> > MultiError exception type that has a standard generic format for
>> > describing such things. It'd need a mask saying which values were
>> > valid, rather than encoding them directly into the return values --
>> > otherwise we have the problem where nanargmax wants to use INT_MIN,
>> > chol wants to use NaN, and maybe the next function along doesn't have
>> > any usable flag value available at all. So probably more thought is
>> > needed before nailing down exactly how we handle such "partial" errors
>> > for vectorized functions.
>> >
>> > In the short term (i.e., 1.8.0), maybe we should defer this discussion
>> > by simply raising a regular ValueError for nanarg functions on all
>> > errors? That's not a regression from 1.7, since 1.7 also didn't
>> > provide any way to get at partial results in the event of an error,
>> > and it leaves us in a good position to solve the more general problem
>> > later.
>>
>> Can we make the error optional in these cases?
>>
>> like np.seterr for zerodivision, invalid, or floating point errors
>> that allows ignore and raise
>> np.seterr(linalg='ignore')
>>
>> I don't know about nanarg, but thinking about some applications for
>> gufunc linalg code.
>>
>> In some cases I might require for example invertibility of all
>> matrices and raise if one fails,
>> in other case I would be happy with nans, and just sum the results
>> with nansum for example or replace them by some fill value.
>>
> I'm thinking warnings might be more flexible than exceptions:
>
> with warnings.catch_warnings():
>     warnings.simplefilter('error')
>     ...

Sure. Passing in a callback or just leaving the function out and telling
people to implement it themselves would be even more flexible :-). But we
have to trade off complexity of usage, complexity of teaching people how to
do stuff (nobody knows how to use catch_warnings, we only know because we
started writing warning tests just in the last year or so), usefulness in
common situations, etc. The warnings api doesn't give you any way to pass
results out, you still need a separate channel to say what failed and what
succeeded (and maybe for the failures to say what the different failures
are).

Anyway this back and forth still supprts my main suggestion for *right*
now, which is that this is sufficiently nonobvious that with 1.8 breathing
down our necks we should start with the safe behaviour and then work up
from there.

-n
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20131001/4d834f37/attachment.html>

From josef.pktd at gmail.com  Tue Oct  1 16:37:34 2013
From: josef.pktd at gmail.com (josef.pktd at gmail.com)
Date: Tue, 1 Oct 2013 16:37:34 -0400
Subject: [Numpy-discussion] nanargmax failure case (was: Re: [SciPy-Dev]
	1.8.0rc1)
In-Reply-To: <CAPJVwB=13HZXkmntiuqJfHHcCbPUcF=e9QY+sANcw1q95OC6pQ@mail.gmail.com>
References: <CAPJVwBkcp_Cy0=Q5zGXR=W4Vhio4r8cat-j0ZzYUjPmGGuwRtA@mail.gmail.com>
	<CAB6mnxLLgYw0Z-7t9fTN8-kSGWH16ATr=6HOpfxH8=u+d7Xt6A@mail.gmail.com>
	<CAPJVwBmrD4TKW9A0fs4A9ydBJut2z2e+8m9RMdvijZ7FHYZ_zQ@mail.gmail.com>
	<CAMMTP+CuXfv0xm7axL7wtX-VuxSOUXDFLMtajj2UyPyUsAd+uQ@mail.gmail.com>
	<CAB6mnx+5S=E6e0cbF4VUMgVspKwBJbKQxi4V19j5GfOtMJQzDw@mail.gmail.com>
	<CAPJVwB=13HZXkmntiuqJfHHcCbPUcF=e9QY+sANcw1q95OC6pQ@mail.gmail.com>
Message-ID: <CAMMTP+DqGCe755RdAn-w3vwTyfxXtJFJeKxicuKCm2ayEe1WHQ@mail.gmail.com>

On Tue, Oct 1, 2013 at 4:13 PM, Nathaniel Smith <njs at pobox.com> wrote:
> On 1 Oct 2013 17:34, "Charles R Harris" <charlesr.harris at gmail.com> wrote:
>>
>>
>>
>>
>> On Tue, Oct 1, 2013 at 10:19 AM, <josef.pktd at gmail.com> wrote:
>>>
>>> On Tue, Oct 1, 2013 at 10:47 AM, Nathaniel Smith <njs at pobox.com> wrote:
>>> > On Tue, Oct 1, 2013 at 3:20 PM, Charles R Harris
>>> > <charlesr.harris at gmail.com> wrote:
>>> >>
>>> >>
>>> >>
>>> >> On Tue, Oct 1, 2013 at 8:12 AM, Nathaniel Smith <njs at pobox.com> wrote:
>>> >>>
>>> >>> [switching subject to break out from the giant 1.8.0rc1 thread]
>>> >>>
>>> >>> On Tue, Oct 1, 2013 at 2:52 PM, Charles R Harris
>>> >>> <charlesr.harris at gmail.com> wrote:
>>> >>> >
>>> >>> >
>>> >>> >
>>> >>> > On Tue, Oct 1, 2013 at 7:25 AM, Nathaniel Smith <njs at pobox.com>
>>> >>> > wrote:
>>> >>> >>
>>> >>> >> On Tue, Oct 1, 2013 at 1:56 PM, Charles R Harris
>>> >>> >> <charlesr.harris at gmail.com> wrote:
>>> >>> >> > On Tue, Oct 1, 2013 at 4:43 AM, Nathaniel Smith <njs at pobox.com>
>>> >>> >> > wrote:
>>> >>> >> >>
>>> >>> >> >> On Mon, Sep 30, 2013 at 10:51 PM, Christoph Gohlke
>>> >>> >> >> <cgohlke at uci.edu>
>>> >>> >> >> wrote:
>>> >>> >> >> > 2) Bottleneck 0.7.0
>>> >>> >> >> >
>>> >>> >> >> >
>>> >>> >> >> >
>>> >>> >> >> >
>>> >>> >> >> > https://github.com/kwgoodman/bottleneck/issues/71#issuecomment-25331701
>>> >>> >> >>
>>> >>> >> >> I can't tell if these are real bugs in numpy, or tests checking
>>> >>> >> >> that
>>> >>> >> >> bottleneck is bug-for-bug compatible with old numpy and we just
>>> >>> >> >> fixed
>>> >>> >> >> some bugs, or what. It's clearly something to do with the
>>> >>> >> >> nanarg{max,min} rewrite -- @charris, do you know what's going
>>> >>> >> >> on
>>> >>> >> >> here?
>>> >>> >> >>
>>> >>> >> >
>>> >>> >> > Yes ;) The previous behaviour of nanarg for all-nan axis was to
>>> >>> >> > cast
>>> >>> >> > nan
>>> >>> >> > to
>>> >>> >> > intp when the result was an array, and return nan when a scalar.
>>> >>> >> > The
>>> >>> >> > current
>>> >>> >> > behaviour is to return the most negative value of intp as an
>>> >>> >> > error
>>> >>> >> > marker in
>>> >>> >> > both cases and raise a warning. It is a change in behavior, but
>>> >>> >> > I
>>> >>> >> > think
>>> >>> >> > one
>>> >>> >> > that needs to be made.
>>> >>> >>
>>> >>> >> Ah, okay! I kind of lost track of the nanfunc changes by the end
>>> >>> >> there.
>>> >>> >>
>>> >>> >> So for the bottleneck issue, it sounds like the problem is just
>>> >>> >> that
>>> >>> >> bottleneck is still emulating the old numpy behaviour in this
>>> >>> >> corner
>>> >>> >> case, which isn't really a problem. So we don't really need to
>>> >>> >> worry
>>> >>> >> about that, both behaviours are correct, just maybe out of sync.
>>> >>> >>
>>> >>> >> I'm a little dubious about this "make up some weird value that
>>> >>> >> will
>>> >>> >> *probably* blow up if people try to use it without checking, and
>>> >>> >> also
>>> >>> >> raise a warning" thing, wouldn't it make more sense to just raise
>>> >>> >> an
>>> >>> >> error? That's what exceptions are for? I guess I should have said
>>> >>> >> something earlier though...
>>> >>> >>
>>> >>> >
>>> >>> > I figure the blowup is safe, as we can't allocate arrays big enough
>>> >>> > that
>>> >>> > the
>>> >>> > minimum intp value would be a valid index. I considered raising an
>>> >>> > error,
>>> >>> > and if there is a consensus the behavior could be changed. Or we
>>> >>> > could
>>> >>> > add a
>>> >>> > keyword to determine the behavior.
>>> >>>
>>> >>> Yeah, the intp value can't be a valid index, so that covers 95% of
>>> >>> cases, but I'm worried about that other 5%. It could still pass
>>> >>> silently as the endpoint of a slice, or participate in some sort of
>>> >>> integer arithmetic calculation, etc. I assume you also share this
>>> >>> worry to some extent or you wouldn't have put in the warning ;-).
>>> >>>
>>> >>> I guess the bigger question is, why would we *not* use the standard
>>> >>> method for signaling an exceptional condition here, i.e., exceptions?
>>> >>> That way we're 100% guaranteed that if people aren't prepared to
>>> >>> handle it then they'll at least know something has gone wrong, and if
>>> >>> they are prepared to handle it then it's very easy and standard, just
>>> >>> use try/except. Right now I guess you have to check for the special
>>> >>> value, but also do something to silence warnings, but just for that
>>> >>> one line? Sounds kind of complicated...
>>> >>
>>> >>
>>> >> The main reason was for the case of multiple axis, where some of the
>>> >> results
>>> >> would be valid and others not. The simple thing might be to raise an
>>> >> exception but keep the current return values so that users could
>>> >> determine
>>> >> where the problem occurred.
>>> >
>>> > Oh, duh, yes, right, now I remember this discussion. Sorry for being
>>> > slow.
>>> >
>>> > In the past we've *always* raised in error in the multiple axis case,
>>> > right? Has anyone ever complained? Wanting to get all
>>> > nanargmax/nanargmin results, of which some might be errors, without
>>> > just writing a loop, seems like a pretty exotic case to me, so I'm not
>>> > sure we should optimize for it at the expense of returning
>>> > possibly-misleading results in the scalar case.
>>> >
>>> > Like (I think) you say, we could get the best of both worlds by
>>> > encoding the results in the same way we do right now, but then raise
>>> > an exception and attach the results to the exception so they can be
>>> > retrieved if wanted. Kind of cumbersome, but maybe good?
>>> >
>>> > This is a more general problem though of course -- we've run into it
>>> > in the gufunc linalg code too, where there's some question about you
>>> > do in e.g. chol() if some sub-matrices are positive-definite and some
>>> > are not.
>>> >
>>> > Off the top of my head the general solution might be to define a
>>> > MultiError exception type that has a standard generic format for
>>> > describing such things. It'd need a mask saying which values were
>>> > valid, rather than encoding them directly into the return values --
>>> > otherwise we have the problem where nanargmax wants to use INT_MIN,
>>> > chol wants to use NaN, and maybe the next function along doesn't have
>>> > any usable flag value available at all. So probably more thought is
>>> > needed before nailing down exactly how we handle such "partial" errors
>>> > for vectorized functions.
>>> >
>>> > In the short term (i.e., 1.8.0), maybe we should defer this discussion
>>> > by simply raising a regular ValueError for nanarg functions on all
>>> > errors? That's not a regression from 1.7, since 1.7 also didn't
>>> > provide any way to get at partial results in the event of an error,
>>> > and it leaves us in a good position to solve the more general problem
>>> > later.
>>>
>>> Can we make the error optional in these cases?
>>>
>>> like np.seterr for zerodivision, invalid, or floating point errors
>>> that allows ignore and raise
>>> np.seterr(linalg='ignore')
>>>
>>> I don't know about nanarg, but thinking about some applications for
>>> gufunc linalg code.
>>>
>>> In some cases I might require for example invertibility of all
>>> matrices and raise if one fails,
>>> in other case I would be happy with nans, and just sum the results
>>> with nansum for example or replace them by some fill value.
>>>
>> I'm thinking warnings might be more flexible than exceptions:
>>
>> with warnings.catch_warnings():
>>     warnings.simplefilter('error')
>>     ...
>
> Sure. Passing in a callback or just leaving the function out and telling
> people to implement it themselves would be even more flexible :-). But we
> have to trade off complexity of usage, complexity of teaching people how to
> do stuff (nobody knows how to use catch_warnings, we only know because we
> started writing warning tests just in the last year or so), usefulness in
> common situations, etc. The warnings api doesn't give you any way to pass
> results out, you still need a separate channel to say what failed and what
> succeeded (and maybe for the failures to say what the different failures
> are).

Since numpy and scipy just moved to python 2.6, it's time to advertise
and support
warnings.catch_warnings().

If you want to wait for a "missing value support" in numpy to support
this, then this postpones this to .... (numpy 3.0?)
while gufuncs seem to be happening now.

Josef
"from the balcony"
3-dimensional panel data linear algebra without vec and kron ?


>
> Anyway this back and forth still supprts my main suggestion for *right* now,
> which is that this is sufficiently nonobvious that with 1.8 breathing down
> our necks we should start with the safe behaviour and then work up from
> there.
>
> -n
>
>
> _______________________________________________
> NumPy-Discussion mailing list
> NumPy-Discussion at scipy.org
> http://mail.scipy.org/mailman/listinfo/numpy-discussion
>


From charlesr.harris at gmail.com  Tue Oct  1 16:37:51 2013
From: charlesr.harris at gmail.com (Charles R Harris)
Date: Tue, 1 Oct 2013 14:37:51 -0600
Subject: [Numpy-discussion] nanargmax failure case (was: Re: [SciPy-Dev]
	1.8.0rc1)
In-Reply-To: <CAPJVwB=13HZXkmntiuqJfHHcCbPUcF=e9QY+sANcw1q95OC6pQ@mail.gmail.com>
References: <CAPJVwBkcp_Cy0=Q5zGXR=W4Vhio4r8cat-j0ZzYUjPmGGuwRtA@mail.gmail.com>
	<CAB6mnxLLgYw0Z-7t9fTN8-kSGWH16ATr=6HOpfxH8=u+d7Xt6A@mail.gmail.com>
	<CAPJVwBmrD4TKW9A0fs4A9ydBJut2z2e+8m9RMdvijZ7FHYZ_zQ@mail.gmail.com>
	<CAMMTP+CuXfv0xm7axL7wtX-VuxSOUXDFLMtajj2UyPyUsAd+uQ@mail.gmail.com>
	<CAB6mnx+5S=E6e0cbF4VUMgVspKwBJbKQxi4V19j5GfOtMJQzDw@mail.gmail.com>
	<CAPJVwB=13HZXkmntiuqJfHHcCbPUcF=e9QY+sANcw1q95OC6pQ@mail.gmail.com>
Message-ID: <CAB6mnx+hWotbUZJ1msazSZ42+7E2SMYT0J-Ekes5=p+oToxyOw@mail.gmail.com>

On Tue, Oct 1, 2013 at 2:13 PM, Nathaniel Smith <njs at pobox.com> wrote:

> On 1 Oct 2013 17:34, "Charles R Harris" <charlesr.harris at gmail.com> wrote:
> >
> >
> >
> >
> > On Tue, Oct 1, 2013 at 10:19 AM, <josef.pktd at gmail.com> wrote:
> >>
> >> On Tue, Oct 1, 2013 at 10:47 AM, Nathaniel Smith <njs at pobox.com> wrote:
> >> > On Tue, Oct 1, 2013 at 3:20 PM, Charles R Harris
> >> > <charlesr.harris at gmail.com> wrote:
> >> >>
> >> >>
> >> >>
> >> >> On Tue, Oct 1, 2013 at 8:12 AM, Nathaniel Smith <njs at pobox.com>
> wrote:
> >> >>>
> >> >>> [switching subject to break out from the giant 1.8.0rc1 thread]
> >> >>>
> >> >>> On Tue, Oct 1, 2013 at 2:52 PM, Charles R Harris
> >> >>> <charlesr.harris at gmail.com> wrote:
> >> >>> >
> >> >>> >
> >> >>> >
> >> >>> > On Tue, Oct 1, 2013 at 7:25 AM, Nathaniel Smith <njs at pobox.com>
> wrote:
> >> >>> >>
> >> >>> >> On Tue, Oct 1, 2013 at 1:56 PM, Charles R Harris
> >> >>> >> <charlesr.harris at gmail.com> wrote:
> >> >>> >> > On Tue, Oct 1, 2013 at 4:43 AM, Nathaniel Smith <njs at pobox.com
> >
> >> >>> >> > wrote:
> >> >>> >> >>
> >> >>> >> >> On Mon, Sep 30, 2013 at 10:51 PM, Christoph Gohlke <
> cgohlke at uci.edu>
> >> >>> >> >> wrote:
> >> >>> >> >> > 2) Bottleneck 0.7.0
> >> >>> >> >> >
> >> >>> >> >> >
> >> >>> >> >> >
> >> >>> >> >> >
> https://github.com/kwgoodman/bottleneck/issues/71#issuecomment-25331701
> >> >>> >> >>
> >> >>> >> >> I can't tell if these are real bugs in numpy, or tests
> checking that
> >> >>> >> >> bottleneck is bug-for-bug compatible with old numpy and we
> just
> >> >>> >> >> fixed
> >> >>> >> >> some bugs, or what. It's clearly something to do with the
> >> >>> >> >> nanarg{max,min} rewrite -- @charris, do you know what's going
> on
> >> >>> >> >> here?
> >> >>> >> >>
> >> >>> >> >
> >> >>> >> > Yes ;) The previous behaviour of nanarg for all-nan axis was
> to cast
> >> >>> >> > nan
> >> >>> >> > to
> >> >>> >> > intp when the result was an array, and return nan when a
> scalar. The
> >> >>> >> > current
> >> >>> >> > behaviour is to return the most negative value of intp as an
> error
> >> >>> >> > marker in
> >> >>> >> > both cases and raise a warning. It is a change in behavior,
> but I
> >> >>> >> > think
> >> >>> >> > one
> >> >>> >> > that needs to be made.
> >> >>> >>
> >> >>> >> Ah, okay! I kind of lost track of the nanfunc changes by the end
> there.
> >> >>> >>
> >> >>> >> So for the bottleneck issue, it sounds like the problem is just
> that
> >> >>> >> bottleneck is still emulating the old numpy behaviour in this
> corner
> >> >>> >> case, which isn't really a problem. So we don't really need to
> worry
> >> >>> >> about that, both behaviours are correct, just maybe out of sync.
> >> >>> >>
> >> >>> >> I'm a little dubious about this "make up some weird value that
> will
> >> >>> >> *probably* blow up if people try to use it without checking, and
> also
> >> >>> >> raise a warning" thing, wouldn't it make more sense to just
> raise an
> >> >>> >> error? That's what exceptions are for? I guess I should have said
> >> >>> >> something earlier though...
> >> >>> >>
> >> >>> >
> >> >>> > I figure the blowup is safe, as we can't allocate arrays big
> enough that
> >> >>> > the
> >> >>> > minimum intp value would be a valid index. I considered raising an
> >> >>> > error,
> >> >>> > and if there is a consensus the behavior could be changed. Or we
> could
> >> >>> > add a
> >> >>> > keyword to determine the behavior.
> >> >>>
> >> >>> Yeah, the intp value can't be a valid index, so that covers 95% of
> >> >>> cases, but I'm worried about that other 5%. It could still pass
> >> >>> silently as the endpoint of a slice, or participate in some sort of
> >> >>> integer arithmetic calculation, etc. I assume you also share this
> >> >>> worry to some extent or you wouldn't have put in the warning ;-).
> >> >>>
> >> >>> I guess the bigger question is, why would we *not* use the standard
> >> >>> method for signaling an exceptional condition here, i.e.,
> exceptions?
> >> >>> That way we're 100% guaranteed that if people aren't prepared to
> >> >>> handle it then they'll at least know something has gone wrong, and
> if
> >> >>> they are prepared to handle it then it's very easy and standard,
> just
> >> >>> use try/except. Right now I guess you have to check for the special
> >> >>> value, but also do something to silence warnings, but just for that
> >> >>> one line? Sounds kind of complicated...
> >> >>
> >> >>
> >> >> The main reason was for the case of multiple axis, where some of the
> results
> >> >> would be valid and others not. The simple thing might be to raise an
> >> >> exception but keep the current return values so that users could
> determine
> >> >> where the problem occurred.
> >> >
> >> > Oh, duh, yes, right, now I remember this discussion. Sorry for being
> slow.
> >> >
> >> > In the past we've *always* raised in error in the multiple axis case,
> >> > right? Has anyone ever complained? Wanting to get all
> >> > nanargmax/nanargmin results, of which some might be errors, without
> >> > just writing a loop, seems like a pretty exotic case to me, so I'm not
> >> > sure we should optimize for it at the expense of returning
> >> > possibly-misleading results in the scalar case.
> >> >
> >> > Like (I think) you say, we could get the best of both worlds by
> >> > encoding the results in the same way we do right now, but then raise
> >> > an exception and attach the results to the exception so they can be
> >> > retrieved if wanted. Kind of cumbersome, but maybe good?
> >> >
> >> > This is a more general problem though of course -- we've run into it
> >> > in the gufunc linalg code too, where there's some question about you
> >> > do in e.g. chol() if some sub-matrices are positive-definite and some
> >> > are not.
> >> >
> >> > Off the top of my head the general solution might be to define a
> >> > MultiError exception type that has a standard generic format for
> >> > describing such things. It'd need a mask saying which values were
> >> > valid, rather than encoding them directly into the return values --
> >> > otherwise we have the problem where nanargmax wants to use INT_MIN,
> >> > chol wants to use NaN, and maybe the next function along doesn't have
> >> > any usable flag value available at all. So probably more thought is
> >> > needed before nailing down exactly how we handle such "partial" errors
> >> > for vectorized functions.
> >> >
> >> > In the short term (i.e., 1.8.0), maybe we should defer this discussion
> >> > by simply raising a regular ValueError for nanarg functions on all
> >> > errors? That's not a regression from 1.7, since 1.7 also didn't
> >> > provide any way to get at partial results in the event of an error,
> >> > and it leaves us in a good position to solve the more general problem
> >> > later.
> >>
> >> Can we make the error optional in these cases?
> >>
> >> like np.seterr for zerodivision, invalid, or floating point errors
> >> that allows ignore and raise
> >> np.seterr(linalg='ignore')
> >>
> >> I don't know about nanarg, but thinking about some applications for
> >> gufunc linalg code.
> >>
> >> In some cases I might require for example invertibility of all
> >> matrices and raise if one fails,
> >> in other case I would be happy with nans, and just sum the results
> >> with nansum for example or replace them by some fill value.
> >>
> > I'm thinking warnings might be more flexible than exceptions:
> >
> > with warnings.catch_warnings():
> >     warnings.simplefilter('error')
> >     ...
>
> Sure. Passing in a callback or just leaving the function out and telling
> people to implement it themselves would be even more flexible :-). But we
> have to trade off complexity of usage, complexity of teaching people how to
> do stuff (nobody knows how to use catch_warnings, we only know because we
> started writing warning tests just in the last year or so), usefulness in
> common situations, etc. The warnings api doesn't give you any way to pass
> results out, you still need a separate channel to say what failed and what
> succeeded (and maybe for the failures to say what the different failures
> are).
>
> Anyway this back and forth still supprts my main suggestion for *right*
> now, which is that this is sufficiently nonobvious that with 1.8 breathing
> down our necks we should start with the safe behaviour and then work up
> from there.
>

I'm surely not opposed to raising an exception if there is agreement on
that. I think it would also be pretty easy to attach the result to the
exception. For the latter it would be good to have an exception type that
we could maybe reuse for other parts of Numpy.

Chuck
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20131001/504b6ca2/attachment.html>

From njs at pobox.com  Tue Oct  1 16:55:01 2013
From: njs at pobox.com (Nathaniel Smith)
Date: Tue, 1 Oct 2013 21:55:01 +0100
Subject: [Numpy-discussion] nanargmax failure case (was: Re: [SciPy-Dev]
	1.8.0rc1)
In-Reply-To: <CAMMTP+DqGCe755RdAn-w3vwTyfxXtJFJeKxicuKCm2ayEe1WHQ@mail.gmail.com>
References: <CAPJVwBkcp_Cy0=Q5zGXR=W4Vhio4r8cat-j0ZzYUjPmGGuwRtA@mail.gmail.com>
	<CAB6mnxLLgYw0Z-7t9fTN8-kSGWH16ATr=6HOpfxH8=u+d7Xt6A@mail.gmail.com>
	<CAPJVwBmrD4TKW9A0fs4A9ydBJut2z2e+8m9RMdvijZ7FHYZ_zQ@mail.gmail.com>
	<CAMMTP+CuXfv0xm7axL7wtX-VuxSOUXDFLMtajj2UyPyUsAd+uQ@mail.gmail.com>
	<CAB6mnx+5S=E6e0cbF4VUMgVspKwBJbKQxi4V19j5GfOtMJQzDw@mail.gmail.com>
	<CAPJVwB=13HZXkmntiuqJfHHcCbPUcF=e9QY+sANcw1q95OC6pQ@mail.gmail.com>
	<CAMMTP+DqGCe755RdAn-w3vwTyfxXtJFJeKxicuKCm2ayEe1WHQ@mail.gmail.com>
Message-ID: <CAPJVwBk7JgSOW+M02AH4Oe7Yk4i+0mxz0aCmkdyDNhMVUhrv_Q@mail.gmail.com>

On Tue, Oct 1, 2013 at 9:37 PM,  <josef.pktd at gmail.com> wrote:
> Since numpy and scipy just moved to python 2.6, it's time to advertise
> and support
> warnings.catch_warnings().

warnings.catch_warnings is a very useful tool and this is all fun to
talk about, but realistically we're simply not going to merge any
change which involves telling people "the way you detect failure in
this function is to use the catch_warnings() context manager".

> If you want to wait for a "missing value support" in numpy to support
> this, then this postpones this to .... (numpy 3.0?)
> while gufuncs seem to be happening now.

No-one said anything about missing value support :-). I don't see how
it would really solve the problem -- we'll probably never allow
missing values to magically appear in arbitrary function outputs (e.g.
you can't put a bitpattern NA in a regular integer dtype, it's just
not possible).

On Tue, Oct 1, 2013 at 9:37 PM, Charles R Harris
<charlesr.harris at gmail.com> wrote:
> I'm surely not opposed to raising an exception if there is agreement on
> that. I think it would also be pretty easy to attach the result to the
> exception. For the latter it would be good to have an exception type that we
> could maybe reuse for other parts of Numpy.

It is true that no-one's ever objected to the ValueError that
nanarg{max,min} have raised in the past, right? That seems like one
measure of agreement that it's at least acceptable.

Spitball of a proper solution, though some thought would need to go
into how to get this out of gufuncs. (And absolutely unsuitable for
1.8!):

class VectorizedError(object):
    pass

def vectorized_raise(result, good_mask, exceptions):
    exc_types = set([type(e) for e in exceptions])
    exc_types.add(VectorizedError)
    exc_type = type.__new__(type, "SubVectorizedError", tuple(exc_types), {})
    exc = exc_type("


From njs at pobox.com  Tue Oct  1 16:58:23 2013
From: njs at pobox.com (Nathaniel Smith)
Date: Tue, 1 Oct 2013 21:58:23 +0100
Subject: [Numpy-discussion] nanargmax failure case (was: Re: [SciPy-Dev]
	1.8.0rc1)
In-Reply-To: <CAPJVwBk7JgSOW+M02AH4Oe7Yk4i+0mxz0aCmkdyDNhMVUhrv_Q@mail.gmail.com>
References: <CAPJVwBkcp_Cy0=Q5zGXR=W4Vhio4r8cat-j0ZzYUjPmGGuwRtA@mail.gmail.com>
	<CAB6mnxLLgYw0Z-7t9fTN8-kSGWH16ATr=6HOpfxH8=u+d7Xt6A@mail.gmail.com>
	<CAPJVwBmrD4TKW9A0fs4A9ydBJut2z2e+8m9RMdvijZ7FHYZ_zQ@mail.gmail.com>
	<CAMMTP+CuXfv0xm7axL7wtX-VuxSOUXDFLMtajj2UyPyUsAd+uQ@mail.gmail.com>
	<CAB6mnx+5S=E6e0cbF4VUMgVspKwBJbKQxi4V19j5GfOtMJQzDw@mail.gmail.com>
	<CAPJVwB=13HZXkmntiuqJfHHcCbPUcF=e9QY+sANcw1q95OC6pQ@mail.gmail.com>
	<CAMMTP+DqGCe755RdAn-w3vwTyfxXtJFJeKxicuKCm2ayEe1WHQ@mail.gmail.com>
	<CAPJVwBk7JgSOW+M02AH4Oe7Yk4i+0mxz0aCmkdyDNhMVUhrv_Q@mail.gmail.com>
Message-ID: <CAPJVwBkoZ7cLcVAkATsx49zW7OHCDknU7Yc4hj0DyAL=DrjRZw@mail.gmail.com>

whoops, fat-fingered this out while in the middle of typing it

On Tue, Oct 1, 2013 at 9:55 PM, Nathaniel Smith <njs at pobox.com> wrote:
> Spitball of a proper solution, though some thought would need to go
> into how to get this out of gufuncs. (And absolutely unsuitable for
> 1.8!):
>
> class VectorizedError(object):
>     pass
>
> def vectorized_raise(result, good_mask, exceptions):
>     exc_types = set([type(e) for e in exceptions])
>     exc_types.add(VectorizedError)
>     exc_type = type.__new__(type, "SubVectorizedError", tuple(exc_types), {})
>     exc = exc_type("

    # FIXME: use some heuristics to look at the exception messages and try to
    # say something more useful here
    exc = exc_type("Multiple errors")
    exc.result = result
    exc.good_mask = good_mask
    exc.exceptions = exceptions
    raise exc

Okay, maybe that's a little ridiculous, but, you know. Discuss :-).
(The black magic at the beginning is to ensure that if you have a
ValueError and a TypeError consolidated into a single VectorizedError,
then the exception that is raised can be caught by code that's looking
for a ValueError, a TypeError, *or* a VectorizedError.)

-n


From cgohlke at uci.edu  Wed Oct  2 03:31:49 2013
From: cgohlke at uci.edu (Christoph Gohlke)
Date: Wed, 02 Oct 2013 00:31:49 -0700
Subject: [Numpy-discussion] 1.8.0rc1
In-Reply-To: <CAB6mnxJ+WAucyTEoj4w6nN67ja=jWH3kU+b7AZ+K4M1B37HH0g@mail.gmail.com>
References: <CAB6mnxJ+WAucyTEoj4w6nN67ja=jWH3kU+b7AZ+K4M1B37HH0g@mail.gmail.com>
Message-ID: <524BCBE5.8030306@uci.edu>

On 9/30/2013 8:17 AM, Charles R Harris wrote:
> Hi All,
>
> NumPy 1.8.0rc1 is up now on sourceforge
> <http://sourceforge.net/projects/numpy/files/NumPy/1.8.0rc1/> .The
> binary builds are included except for Python 3.3 on windows, which will
> arrive later. Many thanks to Ralf for the binaries, and to those who
> found and fixed the bugs in the last beta. Any remaining bugs are all my
> fault ;) I hope this will be the last release before final, so please
> test it thoroughly.
>
> Chuck
>

One more: Cython-0.19.1 fails relaxed_strides.test_one_sized with 
"ValueError: ndarray is not C-contiguous". Tested on 
numpy-MKL-1.8.0rc1-win-amd64-py2.7 and msvc9.

https://github.com/cython/cython/blob/0.19.x/tests/memoryview/relaxed_strides.pyx#L29

The relevant test output is attached.

Christoph
-------------- next part --------------
======================================================================
FAIL: test_one_sized (line 29) (relaxed_strides.__test__)
Doctest: relaxed_strides.__test__.test_one_sized (line 29)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "X:\Python27-x64\lib\doctest.py", line 2201, in runTest
    raise self.failureException(self.format_failure(new.getvalue()))
AssertionError: Failed doctest test for relaxed_strides.__test__.test_one_sized (line 29)
  File "Cython-0.19.1\BUILD\memoryview\c\relaxed_strides\relaxed_strides.pyd", line unknown line number, in test_one_sized (line 29)

----------------------------------------------------------------------
File "Cython-0.19.1\BUILD\memoryview\c\relaxed_strides\relaxed_strides.pyd", line ?, in relaxed_strides.__test__.test_one_sized (line 29)
Failed example:
    test_one_sized(a)[0]
Exception raised:
    Traceback (most recent call last):
      File "X:\Python27-x64\lib\doctest.py", line 1289, in __run
        compileflags, 1) in test.globs
      File "<doctest relaxed_strides.__test__.test_one_sized (line 29)[3]>", line 1, in <module>
        test_one_sized(a)[0]
      File "relaxed_strides.pyx", line 38, in relaxed_strides.test_one_sized (relaxed_strides.c:1379)
      File "stringsource", line 619, in View.MemoryView.memoryview_cwrapper (relaxed_strides.c:7145)
      File "stringsource", line 327, in View.MemoryView.memoryview.__cinit__ (relaxed_strides.c:3555)
    ValueError: ndarray is not C-contiguous


======================================================================
FAIL: test_one_sized (line 29) (relaxed_strides.__test__)
Doctest: relaxed_strides.__test__.test_one_sized (line 29)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "X:\Python27-x64\lib\doctest.py", line 2201, in runTest
    raise self.failureException(self.format_failure(new.getvalue()))
AssertionError: Failed doctest test for relaxed_strides.__test__.test_one_sized (line 29)
  File "Cython-0.19.1\BUILD\memoryview\cpp\relaxed_strides\relaxed_strides.pyd", line unknown line number, in test_one_sized (line 29)

----------------------------------------------------------------------
File "Cython-0.19.1\BUILD\memoryview\cpp\relaxed_strides\relaxed_strides.pyd", line ?, in relaxed_strides.__test__.test_one_sized (line 29)
Failed example:
    test_one_sized(a)[0]
Exception raised:
    Traceback (most recent call last):
      File "X:\Python27-x64\lib\doctest.py", line 1289, in __run
        compileflags, 1) in test.globs
      File "<doctest relaxed_strides.__test__.test_one_sized (line 29)[3]>", line 1, in <module>
        test_one_sized(a)[0]
      File "relaxed_strides.pyx", line 38, in relaxed_strides.test_one_sized (relaxed_strides.cpp:1379)
      File "stringsource", line 619, in View.MemoryView.memoryview_cwrapper (relaxed_strides.cpp:7145)
      File "stringsource", line 327, in View.MemoryView.memoryview.__cinit__ (relaxed_strides.cpp:3555)
    ValueError: ndarray is not C-contiguous


From njs at pobox.com  Wed Oct  2 05:04:09 2013
From: njs at pobox.com (Nathaniel Smith)
Date: Wed, 2 Oct 2013 10:04:09 +0100
Subject: [Numpy-discussion] 1.8.0rc1
In-Reply-To: <524BCBE5.8030306@uci.edu>
References: <CAB6mnxJ+WAucyTEoj4w6nN67ja=jWH3kU+b7AZ+K4M1B37HH0g@mail.gmail.com>
	<524BCBE5.8030306@uci.edu>
Message-ID: <CAPJVwBmgTMNubdiJtg9fyf3-52hhDE+ZSc11ke2kbtRWFg17iA@mail.gmail.com>

This is a complicated issue to describe but i think the bottom line is that
the test is just wonky here. the behaviour it's checking for is:
wrong in old numpy, but we do it anyway (bug)
wrong in current numpy without RELAXED_STRIDES, and we get it right (I.e.
don't do it, fixed bug)
correct in current numpy with RELAXED_STRIDES (and I think the test will
pass if you build 1.8 this way?) (fixed bug again, since the definition of
"correct" flipped)
correct in future numpys

(@seberg, does that look right?)

I guess maybe we could go back and make sure it stays broken in 1.8, but
it's probably not worth the trouble because this is such a corner case.
Nothing real is going to break because of this AFAICT.

-n
On 2 Oct 2013 08:31, "Christoph Gohlke" <cgohlke at uci.edu> wrote:

> On 9/30/2013 8:17 AM, Charles R Harris wrote:
>
>> Hi All,
>>
>> NumPy 1.8.0rc1 is up now on sourceforge
>> <http://sourceforge.net/**projects/numpy/files/NumPy/1.**8.0rc1/<http://sourceforge.net/projects/numpy/files/NumPy/1.8.0rc1/>>
>> .The
>> binary builds are included except for Python 3.3 on windows, which will
>> arrive later. Many thanks to Ralf for the binaries, and to those who
>> found and fixed the bugs in the last beta. Any remaining bugs are all my
>> fault ;) I hope this will be the last release before final, so please
>> test it thoroughly.
>>
>> Chuck
>>
>>
> One more: Cython-0.19.1 fails relaxed_strides.test_one_sized with
> "ValueError: ndarray is not C-contiguous". Tested on
> numpy-MKL-1.8.0rc1-win-amd64-**py2.7 and msvc9.
>
> https://github.com/cython/**cython/blob/0.19.x/tests/**
> memoryview/relaxed_strides.**pyx#L29<https://github.com/cython/cython/blob/0.19.x/tests/memoryview/relaxed_strides.pyx#L29>
>
> The relevant test output is attached.
>
> Christoph
>
> _______________________________________________
> NumPy-Discussion mailing list
> NumPy-Discussion at scipy.org
> http://mail.scipy.org/mailman/listinfo/numpy-discussion
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20131002/a530f2fa/attachment.html>

From sebastian at sipsolutions.net  Wed Oct  2 06:54:45 2013
From: sebastian at sipsolutions.net (Sebastian Berg)
Date: Wed, 02 Oct 2013 12:54:45 +0200
Subject: [Numpy-discussion] 1.8.0rc1
In-Reply-To: <CAPJVwBmgTMNubdiJtg9fyf3-52hhDE+ZSc11ke2kbtRWFg17iA@mail.gmail.com>
References: <CAB6mnxJ+WAucyTEoj4w6nN67ja=jWH3kU+b7AZ+K4M1B37HH0g@mail.gmail.com>
	<524BCBE5.8030306@uci.edu>
	<CAPJVwBmgTMNubdiJtg9fyf3-52hhDE+ZSc11ke2kbtRWFg17iA@mail.gmail.com>
Message-ID: <1380711285.15888.3.camel@sebastian-laptop>

On Wed, 2013-10-02 at 10:04 +0100, Nathaniel Smith wrote:
> This is a complicated issue to describe but i think the bottom line is
> that the test is just wonky here. the behaviour it's checking for is:
> wrong in old numpy, but we do it anyway (bug)
> wrong in current numpy without RELAXED_STRIDES, and we get it right
> (I.e. don't do it, fixed bug)
> correct in current numpy with RELAXED_STRIDES (and I think the test
> will pass if you build 1.8 this way?) (fixed bug again, since the
> definition of "correct" flipped)
> correct in future numpys
> 
Sounds about right, but I did not yet look at the test exactly. The old
numpy had some corner cases where the *not* relaxed strides were not
enforced. I could imagine that the test was written to accommodate the
relaxed strides, but runs through fine with the old behaviour, too.

That said, if this is the case, this must be something like a size 1
array corner case, so I would agree with Nathaniel, that probably the
test should just see whether relaxed strides is active or not and change
behaviour accordingly.

If this has to do with the Buffer protocol change we have to have a
closer look if it might be breaking code out there. (even if the buffer
protocol change is correct and this should be fixed on cythons end in
that case)

- Sebastian


> (@seberg, does that look right?)
> 
> I guess maybe we could go back and make sure it stays broken in 1.8,
> but it's probably not worth the trouble because this is such a corner
> case. Nothing real is going to break because of this AFAICT.
> 
> -n
> 
> On 2 Oct 2013 08:31, "Christoph Gohlke" <cgohlke at uci.edu> wrote:
>         On 9/30/2013 8:17 AM, Charles R Harris wrote:
>                 Hi All,
>                 
>                 NumPy 1.8.0rc1 is up now on sourceforge
>                 <http://sourceforge.net/projects/numpy/files/NumPy/1.8.0rc1/> .The
>                 binary builds are included except for Python 3.3 on
>                 windows, which will
>                 arrive later. Many thanks to Ralf for the binaries,
>                 and to those who
>                 found and fixed the bugs in the last beta. Any
>                 remaining bugs are all my
>                 fault ;) I hope this will be the last release before
>                 final, so please
>                 test it thoroughly.
>                 
>                 Chuck
>                 
>         
>         One more: Cython-0.19.1 fails relaxed_strides.test_one_sized
>         with "ValueError: ndarray is not C-contiguous". Tested on
>         numpy-MKL-1.8.0rc1-win-amd64-py2.7 and msvc9.
>         
>         https://github.com/cython/cython/blob/0.19.x/tests/memoryview/relaxed_strides.pyx#L29
>         
>         The relevant test output is attached.
>         
>         Christoph
>         
>         _______________________________________________
>         NumPy-Discussion mailing list
>         NumPy-Discussion at scipy.org
>         http://mail.scipy.org/mailman/listinfo/numpy-discussion
>         
> _______________________________________________
> NumPy-Discussion mailing list
> NumPy-Discussion at scipy.org
> http://mail.scipy.org/mailman/listinfo/numpy-discussion




From sebastian at sipsolutions.net  Wed Oct  2 07:51:04 2013
From: sebastian at sipsolutions.net (Sebastian Berg)
Date: Wed, 02 Oct 2013 13:51:04 +0200
Subject: [Numpy-discussion] 1.8.0rc1
In-Reply-To: <1380711285.15888.3.camel@sebastian-laptop>
References: <CAB6mnxJ+WAucyTEoj4w6nN67ja=jWH3kU+b7AZ+K4M1B37HH0g@mail.gmail.com>
	<524BCBE5.8030306@uci.edu>
	<CAPJVwBmgTMNubdiJtg9fyf3-52hhDE+ZSc11ke2kbtRWFg17iA@mail.gmail.com>
	<1380711285.15888.3.camel@sebastian-laptop>
Message-ID: <1380714664.17995.2.camel@sebastian-laptop>

On Wed, 2013-10-02 at 12:54 +0200, Sebastian Berg wrote:
> On Wed, 2013-10-02 at 10:04 +0100, Nathaniel Smith wrote:
> > This is a complicated issue to describe but i think the bottom line is
> > that the test is just wonky here. the behaviour it's checking for is:
> > wrong in old numpy, but we do it anyway (bug)
> > wrong in current numpy without RELAXED_STRIDES, and we get it right
> > (I.e. don't do it, fixed bug)
> > correct in current numpy with RELAXED_STRIDES (and I think the test
> > will pass if you build 1.8 this way?) (fixed bug again, since the
> > definition of "correct" flipped)
> > correct in future numpys
> > 

OK, I had a look. And it is the 1-dimensional size-1 special case that
was always "relaxed", but is not anymore. If anyone thinks this is a
problem, we could re-add the special case (rethinking it, it is almost
impossible to hurt, since the array has only one element). Otherwise, I
think the test should check if numpy was compiled with relaxed strides,
for more extensive tests this is necessary anyway.

- Sebastian


> Sounds about right, but I did not yet look at the test exactly. The old
> numpy had some corner cases where the *not* relaxed strides were not
> enforced. I could imagine that the test was written to accommodate the
> relaxed strides, but runs through fine with the old behaviour, too.
> 
> That said, if this is the case, this must be something like a size 1
> array corner case, so I would agree with Nathaniel, that probably the
> test should just see whether relaxed strides is active or not and change
> behaviour accordingly.
> 
> If this has to do with the Buffer protocol change we have to have a
> closer look if it might be breaking code out there. (even if the buffer
> protocol change is correct and this should be fixed on cythons end in
> that case)
> 
> - Sebastian
> 
> 
> > (@seberg, does that look right?)
> > 
> > I guess maybe we could go back and make sure it stays broken in 1.8,
> > but it's probably not worth the trouble because this is such a corner
> > case. Nothing real is going to break because of this AFAICT.
> > 
> > -n
> > 
> > On 2 Oct 2013 08:31, "Christoph Gohlke" <cgohlke at uci.edu> wrote:
> >         On 9/30/2013 8:17 AM, Charles R Harris wrote:
> >                 Hi All,
> >                 
> >                 NumPy 1.8.0rc1 is up now on sourceforge
> >                 <http://sourceforge.net/projects/numpy/files/NumPy/1.8.0rc1/> .The
> >                 binary builds are included except for Python 3.3 on
> >                 windows, which will
> >                 arrive later. Many thanks to Ralf for the binaries,
> >                 and to those who
> >                 found and fixed the bugs in the last beta. Any
> >                 remaining bugs are all my
> >                 fault ;) I hope this will be the last release before
> >                 final, so please
> >                 test it thoroughly.
> >                 
> >                 Chuck
> >                 
> >         
> >         One more: Cython-0.19.1 fails relaxed_strides.test_one_sized
> >         with "ValueError: ndarray is not C-contiguous". Tested on
> >         numpy-MKL-1.8.0rc1-win-amd64-py2.7 and msvc9.
> >         
> >         https://github.com/cython/cython/blob/0.19.x/tests/memoryview/relaxed_strides.pyx#L29
> >         
> >         The relevant test output is attached.
> >         
> >         Christoph
> >         
> >         _______________________________________________
> >         NumPy-Discussion mailing list
> >         NumPy-Discussion at scipy.org
> >         http://mail.scipy.org/mailman/listinfo/numpy-discussion
> >         
> > _______________________________________________
> > NumPy-Discussion mailing list
> > NumPy-Discussion at scipy.org
> > http://mail.scipy.org/mailman/listinfo/numpy-discussion
> 
> 
> _______________________________________________
> NumPy-Discussion mailing list
> NumPy-Discussion at scipy.org
> http://mail.scipy.org/mailman/listinfo/numpy-discussion
> 




From stefan at sun.ac.za  Wed Oct  2 11:29:41 2013
From: stefan at sun.ac.za (=?ISO-8859-1?Q?St=E9fan_van_der_Walt?=)
Date: Wed, 2 Oct 2013 17:29:41 +0200
Subject: [Numpy-discussion] [SciPy-Dev] 1.8.0rc1
In-Reply-To: <CAB6mnx+mXiRMOgy5eXA=JjoW7i4qtaC=WNfBXcm-2XppdVj53A@mail.gmail.com>
References: <CAB6mnxJ+WAucyTEoj4w6nN67ja=jWH3kU+b7AZ+K4M1B37HH0g@mail.gmail.com>
	<CAPJVwBnyBSZ-oYbD1XRD57Wj2a4LcPHvN6T6v2n1cFKUs7qEpw@mail.gmail.com>
	<5249F26B.2040109@uci.edu>
	<CAB6mnx+KPxNtMU7BDPALrK8Dz4rfEEphoDPcKqHA096D6Yg+VQ@mail.gmail.com>
	<CAB6mnx+mXiRMOgy5eXA=JjoW7i4qtaC=WNfBXcm-2XppdVj53A@mail.gmail.com>
Message-ID: <CABDkGQnOv7iCH0nR8yjgLUkdF=TW-X=skrLpUBkvTgZYVAbYHg@mail.gmail.com>

Hi Chuck

On Tue, Oct 1, 2013 at 1:07 AM, Charles R Harris
<charlesr.harris at gmail.com> wrote:
>> I'll bet the skimage problems come from
>> https://github.com/numpy/numpy/pull/3811. They may be doing something
>> naughty...
>>
>
> Reverting that commit fixes those skimage failures. However, there are a
> number of python2.7 failures that look pretty strange.

What is the exact change in behavior with that PR?  I'm trying to
figure out what skimage does wrong in this case.

St?fan


From charlesr.harris at gmail.com  Wed Oct  2 11:43:42 2013
From: charlesr.harris at gmail.com (Charles R Harris)
Date: Wed, 2 Oct 2013 09:43:42 -0600
Subject: [Numpy-discussion] [SciPy-Dev] 1.8.0rc1
In-Reply-To: <CABDkGQnOv7iCH0nR8yjgLUkdF=TW-X=skrLpUBkvTgZYVAbYHg@mail.gmail.com>
References: <CAB6mnxJ+WAucyTEoj4w6nN67ja=jWH3kU+b7AZ+K4M1B37HH0g@mail.gmail.com>
	<CAPJVwBnyBSZ-oYbD1XRD57Wj2a4LcPHvN6T6v2n1cFKUs7qEpw@mail.gmail.com>
	<5249F26B.2040109@uci.edu>
	<CAB6mnx+KPxNtMU7BDPALrK8Dz4rfEEphoDPcKqHA096D6Yg+VQ@mail.gmail.com>
	<CAB6mnx+mXiRMOgy5eXA=JjoW7i4qtaC=WNfBXcm-2XppdVj53A@mail.gmail.com>
	<CABDkGQnOv7iCH0nR8yjgLUkdF=TW-X=skrLpUBkvTgZYVAbYHg@mail.gmail.com>
Message-ID: <CAB6mnxKcgfpnu6mCR=Ph-QG1+aJjj3USxQzdG0kfBbC7VVG-ZQ@mail.gmail.com>

Hi Stefan,


On Wed, Oct 2, 2013 at 9:29 AM, St?fan van der Walt <stefan at sun.ac.za>wrote:

> Hi Chuck
>
> On Tue, Oct 1, 2013 at 1:07 AM, Charles R Harris
> <charlesr.harris at gmail.com> wrote:
> >> I'll bet the skimage problems come from
> >> https://github.com/numpy/numpy/pull/3811. They may be doing something
> >> naughty...
> >>
> >
> > Reverting that commit fixes those skimage failures. However, there are a
> > number of python2.7 failures that look pretty strange.
>
> What is the exact change in behavior with that PR?  I'm trying to
> figure out what skimage does wrong in this case.
>
>
The current master, and reverted for the 1.8 release only, is stricter
about np.bool only taking values 0 or 1. Apparently the convolve returns
boolean (I haven't checked) for boolean input, and consequently the check
if the return value matches the number of 1 elements in the convolution
kernel will fail when that number is greater than one. That is why the
proposed fix is to view the boolean as uint8 instead. Note that
out=(boolean) will still cause problems.

Chuck
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20131002/d01e51c4/attachment.html>

From charlesr.harris at gmail.com  Wed Oct  2 12:03:43 2013
From: charlesr.harris at gmail.com (Charles R Harris)
Date: Wed, 2 Oct 2013 10:03:43 -0600
Subject: [Numpy-discussion] Behavior of nan{max, min} and nanarg{max,
	min} for all-nan slices.
Message-ID: <CAB6mnxLO+c0CZHSUbL3NZ-quivy5NdSUtaM_iAL1ay2pUwXr0w@mail.gmail.com>

Hi All,

The question is what to do when all-nan slices are encountered in the
nan{max,min} and nanarg{max, min} functions. Currently in 1.8.0, the first
returns nan and raises a warning, the second returns intp.min and raises a
warning. It is proposed that the nanarg{max, min} functions, and possibly
the nan{max, min} also, raise an error instead. Raising errors would be
consistent with the behavior of the arg{max, min} and amax/amin functions
when they encounter empty arrays. OTOH, now that we no longer support
Python 2.4/2.5 the catch_warnings context manager can serve the same
purpose by changing the warnings into exceptions. So, what to do?

Thoughts?

Chuck
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20131002/ab2f8523/attachment.html>

From njs at pobox.com  Wed Oct  2 12:09:40 2013
From: njs at pobox.com (Nathaniel Smith)
Date: Wed, 2 Oct 2013 17:09:40 +0100
Subject: [Numpy-discussion] Behavior of nan{max, min} and nanarg{max,
 min} for all-nan slices.
In-Reply-To: <CAB6mnxLO+c0CZHSUbL3NZ-quivy5NdSUtaM_iAL1ay2pUwXr0w@mail.gmail.com>
References: <CAB6mnxLO+c0CZHSUbL3NZ-quivy5NdSUtaM_iAL1ay2pUwXr0w@mail.gmail.com>
Message-ID: <CAPJVwBk9pNN66ReF1oNKKnfYnrXQ11g4ZzXbi6+8T6gK64Xp8Q@mail.gmail.com>

+1 to making the nan functions consistent with the non-nan functions.
On 2 Oct 2013 17:03, "Charles R Harris" <charlesr.harris at gmail.com> wrote:

> Hi All,
>
> The question is what to do when all-nan slices are encountered in the
> nan{max,min} and nanarg{max, min} functions. Currently in 1.8.0, the first
> returns nan and raises a warning, the second returns intp.min and raises a
> warning. It is proposed that the nanarg{max, min} functions, and possibly
> the nan{max, min} also, raise an error instead. Raising errors would be
> consistent with the behavior of the arg{max, min} and amax/amin functions
> when they encounter empty arrays. OTOH, now that we no longer support
> Python 2.4/2.5 the catch_warnings context manager can serve the same
> purpose by changing the warnings into exceptions. So, what to do?
>
> Thoughts?
>
> Chuck
>
> _______________________________________________
> NumPy-Discussion mailing list
> NumPy-Discussion at scipy.org
> http://mail.scipy.org/mailman/listinfo/numpy-discussion
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20131002/ee0aee21/attachment.html>

From ben.root at ou.edu  Wed Oct  2 12:12:00 2013
From: ben.root at ou.edu (Benjamin Root)
Date: Wed, 2 Oct 2013 12:12:00 -0400
Subject: [Numpy-discussion] [SciPy-Dev] 1.8.0rc1
In-Reply-To: <CAB6mnxKcgfpnu6mCR=Ph-QG1+aJjj3USxQzdG0kfBbC7VVG-ZQ@mail.gmail.com>
References: <CAB6mnxJ+WAucyTEoj4w6nN67ja=jWH3kU+b7AZ+K4M1B37HH0g@mail.gmail.com>
	<CAPJVwBnyBSZ-oYbD1XRD57Wj2a4LcPHvN6T6v2n1cFKUs7qEpw@mail.gmail.com>
	<5249F26B.2040109@uci.edu>
	<CAB6mnx+KPxNtMU7BDPALrK8Dz4rfEEphoDPcKqHA096D6Yg+VQ@mail.gmail.com>
	<CAB6mnx+mXiRMOgy5eXA=JjoW7i4qtaC=WNfBXcm-2XppdVj53A@mail.gmail.com>
	<CABDkGQnOv7iCH0nR8yjgLUkdF=TW-X=skrLpUBkvTgZYVAbYHg@mail.gmail.com>
	<CAB6mnxKcgfpnu6mCR=Ph-QG1+aJjj3USxQzdG0kfBbC7VVG-ZQ@mail.gmail.com>
Message-ID: <CANNq6F=WAvB8M4VyCKRznW7vwwQeEUR_U2owk4OTFEbT_trLDw@mail.gmail.com>

On Wed, Oct 2, 2013 at 11:43 AM, Charles R Harris <charlesr.harris at gmail.com
> wrote:

> Hi Stefan,
>
>
> On Wed, Oct 2, 2013 at 9:29 AM, St?fan van der Walt <stefan at sun.ac.za>wrote:
>
>> Hi Chuck
>>
>> On Tue, Oct 1, 2013 at 1:07 AM, Charles R Harris
>> <charlesr.harris at gmail.com> wrote:
>> >> I'll bet the skimage problems come from
>> >> https://github.com/numpy/numpy/pull/3811. They may be doing something
>> >> naughty...
>> >>
>> >
>> > Reverting that commit fixes those skimage failures. However, there are a
>> > number of python2.7 failures that look pretty strange.
>>
>> What is the exact change in behavior with that PR?  I'm trying to
>> figure out what skimage does wrong in this case.
>>
>>
> The current master, and reverted for the 1.8 release only, is stricter
> about np.bool only taking values 0 or 1. Apparently the convolve returns
> boolean (I haven't checked) for boolean input, and consequently the check
> if the return value matches the number of 1 elements in the convolution
> kernel will fail when that number is greater than one. That is why the
> proposed fix is to view the boolean as uint8 instead. Note that
> out=(boolean) will still cause problems.
>
> Chuck
>
>
So, just to be clear... what would happen if I had an array of floats
between 0 and 1 inclusive and I cast that as a boolean using astype()?

Ben Root
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20131002/f099636c/attachment.html>

From charlesr.harris at gmail.com  Wed Oct  2 12:24:24 2013
From: charlesr.harris at gmail.com (Charles R Harris)
Date: Wed, 2 Oct 2013 10:24:24 -0600
Subject: [Numpy-discussion] [SciPy-Dev] 1.8.0rc1
In-Reply-To: <CANNq6F=WAvB8M4VyCKRznW7vwwQeEUR_U2owk4OTFEbT_trLDw@mail.gmail.com>
References: <CAB6mnxJ+WAucyTEoj4w6nN67ja=jWH3kU+b7AZ+K4M1B37HH0g@mail.gmail.com>
	<CAPJVwBnyBSZ-oYbD1XRD57Wj2a4LcPHvN6T6v2n1cFKUs7qEpw@mail.gmail.com>
	<5249F26B.2040109@uci.edu>
	<CAB6mnx+KPxNtMU7BDPALrK8Dz4rfEEphoDPcKqHA096D6Yg+VQ@mail.gmail.com>
	<CAB6mnx+mXiRMOgy5eXA=JjoW7i4qtaC=WNfBXcm-2XppdVj53A@mail.gmail.com>
	<CABDkGQnOv7iCH0nR8yjgLUkdF=TW-X=skrLpUBkvTgZYVAbYHg@mail.gmail.com>
	<CAB6mnxKcgfpnu6mCR=Ph-QG1+aJjj3USxQzdG0kfBbC7VVG-ZQ@mail.gmail.com>
	<CANNq6F=WAvB8M4VyCKRznW7vwwQeEUR_U2owk4OTFEbT_trLDw@mail.gmail.com>
Message-ID: <CAB6mnxJKCA_0QtFvAiHKm2nvtziXdZZq1z1pSq3u-=R0L1Bd6A@mail.gmail.com>

On Wed, Oct 2, 2013 at 10:12 AM, Benjamin Root <ben.root at ou.edu> wrote:

>
>
>
> On Wed, Oct 2, 2013 at 11:43 AM, Charles R Harris <
> charlesr.harris at gmail.com> wrote:
>
>> Hi Stefan,
>>
>>
>> On Wed, Oct 2, 2013 at 9:29 AM, St?fan van der Walt <stefan at sun.ac.za>wrote:
>>
>>> Hi Chuck
>>>
>>> On Tue, Oct 1, 2013 at 1:07 AM, Charles R Harris
>>> <charlesr.harris at gmail.com> wrote:
>>> >> I'll bet the skimage problems come from
>>> >> https://github.com/numpy/numpy/pull/3811. They may be doing something
>>> >> naughty...
>>> >>
>>> >
>>> > Reverting that commit fixes those skimage failures. However, there are
>>> a
>>> > number of python2.7 failures that look pretty strange.
>>>
>>> What is the exact change in behavior with that PR?  I'm trying to
>>> figure out what skimage does wrong in this case.
>>>
>>>
>> The current master, and reverted for the 1.8 release only, is stricter
>> about np.bool only taking values 0 or 1. Apparently the convolve returns
>> boolean (I haven't checked) for boolean input, and consequently the check
>> if the return value matches the number of 1 elements in the convolution
>> kernel will fail when that number is greater than one. That is why the
>> proposed fix is to view the boolean as uint8 instead. Note that
>> out=(boolean) will still cause problems.
>>
>> Chuck
>>
>>
> So, just to be clear... what would happen if I had an array of floats
> between 0 and 1 inclusive and I cast that as a boolean using astype()?
>
>
You get zeros and ones. I don't think that has changed. Of course, you can
still mess things up using views ;)

Chuck
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20131002/e43f83ed/attachment.html>

From stefan at sun.ac.za  Wed Oct  2 12:37:50 2013
From: stefan at sun.ac.za (=?ISO-8859-1?Q?St=E9fan_van_der_Walt?=)
Date: Wed, 2 Oct 2013 18:37:50 +0200
Subject: [Numpy-discussion] Behavior of nan{max, min} and nanarg{max,
 min} for all-nan slices.
In-Reply-To: <CAB6mnxLO+c0CZHSUbL3NZ-quivy5NdSUtaM_iAL1ay2pUwXr0w@mail.gmail.com>
References: <CAB6mnxLO+c0CZHSUbL3NZ-quivy5NdSUtaM_iAL1ay2pUwXr0w@mail.gmail.com>
Message-ID: <CABDkGQmWHrYdoEvJigFOnL4SAkOY4JeChW7woR3cZsr2ArwGMA@mail.gmail.com>

On 2 Oct 2013 18:04, "Charles R Harris" <charlesr.harris at gmail.com> wrote:
>
> The question is what to do when all-nan slices are encountered in the
nan{max,min} and nanarg{max, min} functions. Currently in 1.8.0, the first
returns nan and raises a warning, the second returns intp.min and raises a
warning. It is proposed that the nanarg{max, min} functions, and possibly
the nan{max, min} also, raise an error instead.

I agree with Nathan; this sounds like more reasonable behaviour to me.

St?fan
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20131002/6d1b01ab/attachment.html>

From josef.pktd at gmail.com  Wed Oct  2 12:56:19 2013
From: josef.pktd at gmail.com (josef.pktd at gmail.com)
Date: Wed, 2 Oct 2013 12:56:19 -0400
Subject: [Numpy-discussion] Behavior of nan{max, min} and nanarg{max,
 min} for all-nan slices.
In-Reply-To: <CABDkGQmWHrYdoEvJigFOnL4SAkOY4JeChW7woR3cZsr2ArwGMA@mail.gmail.com>
References: <CAB6mnxLO+c0CZHSUbL3NZ-quivy5NdSUtaM_iAL1ay2pUwXr0w@mail.gmail.com>
	<CABDkGQmWHrYdoEvJigFOnL4SAkOY4JeChW7woR3cZsr2ArwGMA@mail.gmail.com>
Message-ID: <CAMMTP+AvPxqE3FwQrvnz74GQeso5M4KipUuAbaLcgy2OvxLQww@mail.gmail.com>

On Wed, Oct 2, 2013 at 12:37 PM, St?fan van der Walt <stefan at sun.ac.za> wrote:
> On 2 Oct 2013 18:04, "Charles R Harris" <charlesr.harris at gmail.com> wrote:
>>
>> The question is what to do when all-nan slices are encountered in the
>> nan{max,min} and nanarg{max, min} functions. Currently in 1.8.0, the first
>> returns nan and raises a warning, the second returns intp.min and raises a
>> warning. It is proposed that the nanarg{max, min} functions, and possibly
>> the nan{max, min} also, raise an error instead.
>
> I agree with Nathan; this sounds like more reasonable behaviour to me.

If I understand what you are proposing

-1 on raising an error with nan{max, min},

an empty array is empty in all columns
an array with nans, might be empty in only some columns.

as far as I understand, nan{max, min} only make sense with arrays that
can hold a nan, so we can return nans.

If a user calls with ints or bool, then there are either no nans or
the array is empty, and I don't care.

---
aside
with nanarg{max, min} I would just return 0 in an all nan column,
since the max or min is nan, and one is at zero.
(but I'm not arguing)

Josef


>
> St?fan
>
>
> _______________________________________________
> NumPy-Discussion mailing list
> NumPy-Discussion at scipy.org
> http://mail.scipy.org/mailman/listinfo/numpy-discussion
>


From charlesr.harris at gmail.com  Wed Oct  2 13:05:28 2013
From: charlesr.harris at gmail.com (Charles R Harris)
Date: Wed, 2 Oct 2013 11:05:28 -0600
Subject: [Numpy-discussion] Behavior of nan{max, min} and nanarg{max,
 min} for all-nan slices.
In-Reply-To: <CAMMTP+AvPxqE3FwQrvnz74GQeso5M4KipUuAbaLcgy2OvxLQww@mail.gmail.com>
References: <CAB6mnxLO+c0CZHSUbL3NZ-quivy5NdSUtaM_iAL1ay2pUwXr0w@mail.gmail.com>
	<CABDkGQmWHrYdoEvJigFOnL4SAkOY4JeChW7woR3cZsr2ArwGMA@mail.gmail.com>
	<CAMMTP+AvPxqE3FwQrvnz74GQeso5M4KipUuAbaLcgy2OvxLQww@mail.gmail.com>
Message-ID: <CAB6mnxKjDBPQ=HFbbA2f+8iOT=vuQy9x-JyKw_tdP1QSeM=sFw@mail.gmail.com>

On Wed, Oct 2, 2013 at 10:56 AM, <josef.pktd at gmail.com> wrote:

> On Wed, Oct 2, 2013 at 12:37 PM, St?fan van der Walt <stefan at sun.ac.za>
> wrote:
> > On 2 Oct 2013 18:04, "Charles R Harris" <charlesr.harris at gmail.com>
> wrote:
> >>
> >> The question is what to do when all-nan slices are encountered in the
> >> nan{max,min} and nanarg{max, min} functions. Currently in 1.8.0, the
> first
> >> returns nan and raises a warning, the second returns intp.min and
> raises a
> >> warning. It is proposed that the nanarg{max, min} functions, and
> possibly
> >> the nan{max, min} also, raise an error instead.
> >
> > I agree with Nathan; this sounds like more reasonable behaviour to me.
>
> If I understand what you are proposing
>
> -1 on raising an error with nan{max, min},
>
> an empty array is empty in all columns
> an array with nans, might be empty in only some columns.
>
> as far as I understand, nan{max, min} only make sense with arrays that
> can hold a nan, so we can return nans.
>

That was my original thought.


>
> If a user calls with ints or bool, then there are either no nans or
> the array is empty, and I don't care.
>
> ---
> aside
> with nanarg{max, min} I would just return 0 in an all nan column,
> since the max or min is nan, and one is at zero.
> (but I'm not arguing)
>
>
That is an interesting proposal. I like it.

Chuck
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20131002/fbeeea81/attachment.html>

From ben.root at ou.edu  Wed Oct  2 13:13:20 2013
From: ben.root at ou.edu (Benjamin Root)
Date: Wed, 2 Oct 2013 13:13:20 -0400
Subject: [Numpy-discussion] Behavior of nan{max, min} and nanarg{max,
 min} for all-nan slices.
In-Reply-To: <CAB6mnxKjDBPQ=HFbbA2f+8iOT=vuQy9x-JyKw_tdP1QSeM=sFw@mail.gmail.com>
References: <CAB6mnxLO+c0CZHSUbL3NZ-quivy5NdSUtaM_iAL1ay2pUwXr0w@mail.gmail.com>
	<CABDkGQmWHrYdoEvJigFOnL4SAkOY4JeChW7woR3cZsr2ArwGMA@mail.gmail.com>
	<CAMMTP+AvPxqE3FwQrvnz74GQeso5M4KipUuAbaLcgy2OvxLQww@mail.gmail.com>
	<CAB6mnxKjDBPQ=HFbbA2f+8iOT=vuQy9x-JyKw_tdP1QSeM=sFw@mail.gmail.com>
Message-ID: <CANNq6Fk1hmyZ+xpJjBhNJSv1ZnC_TG+c2rLDBg+1ciQ4rLHhQA@mail.gmail.com>

On Wed, Oct 2, 2013 at 1:05 PM, Charles R Harris
<charlesr.harris at gmail.com>wrote:

>
>
>
> On Wed, Oct 2, 2013 at 10:56 AM, <josef.pktd at gmail.com> wrote:
>
>> On Wed, Oct 2, 2013 at 12:37 PM, St?fan van der Walt <stefan at sun.ac.za>
>> wrote:
>> > On 2 Oct 2013 18:04, "Charles R Harris" <charlesr.harris at gmail.com>
>> wrote:
>> >>
>> >> The question is what to do when all-nan slices are encountered in the
>> >> nan{max,min} and nanarg{max, min} functions. Currently in 1.8.0, the
>> first
>> >> returns nan and raises a warning, the second returns intp.min and
>> raises a
>> >> warning. It is proposed that the nanarg{max, min} functions, and
>> possibly
>> >> the nan{max, min} also, raise an error instead.
>> >
>> > I agree with Nathan; this sounds like more reasonable behaviour to me.
>>
>> If I understand what you are proposing
>>
>> -1 on raising an error with nan{max, min},
>>
>> an empty array is empty in all columns
>> an array with nans, might be empty in only some columns.
>>
>> as far as I understand, nan{max, min} only make sense with arrays that
>> can hold a nan, so we can return nans.
>>
>
> That was my original thought.
>
>
>>
>> If a user calls with ints or bool, then there are either no nans or
>> the array is empty, and I don't care.
>>
>> ---
>> aside
>> with nanarg{max, min} I would just return 0 in an all nan column,
>> since the max or min is nan, and one is at zero.
>> (but I'm not arguing)
>>
>>
> That is an interesting proposal. I like it.
>
> Chuck
>
>
>
And it is logically consistent, I think.  a[nanargmax(a)] == nanmax(a)
(ignoring the silly detail that you can't do an equality on nans).

Ben Root
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20131002/f61c16de/attachment.html>

From stefan at sun.ac.za  Wed Oct  2 14:05:33 2013
From: stefan at sun.ac.za (=?ISO-8859-1?Q?St=E9fan_van_der_Walt?=)
Date: Wed, 2 Oct 2013 20:05:33 +0200
Subject: [Numpy-discussion] Behavior of nan{max, min} and nanarg{max,
 min} for all-nan slices.
In-Reply-To: <CANNq6Fk1hmyZ+xpJjBhNJSv1ZnC_TG+c2rLDBg+1ciQ4rLHhQA@mail.gmail.com>
References: <CAB6mnxLO+c0CZHSUbL3NZ-quivy5NdSUtaM_iAL1ay2pUwXr0w@mail.gmail.com>
	<CABDkGQmWHrYdoEvJigFOnL4SAkOY4JeChW7woR3cZsr2ArwGMA@mail.gmail.com>
	<CAMMTP+AvPxqE3FwQrvnz74GQeso5M4KipUuAbaLcgy2OvxLQww@mail.gmail.com>
	<CAB6mnxKjDBPQ=HFbbA2f+8iOT=vuQy9x-JyKw_tdP1QSeM=sFw@mail.gmail.com>
	<CANNq6Fk1hmyZ+xpJjBhNJSv1ZnC_TG+c2rLDBg+1ciQ4rLHhQA@mail.gmail.com>
Message-ID: <CABDkGQmCTQVToKaWReZ_UHagot6QVVkUZyVMPa5iCPGzhJ1ykw@mail.gmail.com>

On 2 Oct 2013 19:14, "Benjamin Root" <ben.root at ou.edu> wrote:
>
> And it is logically consistent, I think.  a[nanargmax(a)] == nanmax(a)
(ignoring the silly detail that you can't do an equality on nans).

Why do you call this a silly detail? It seems to me a fundamental flaw to
this approach.

St?fan
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20131002/d93bc114/attachment.html>

From josef.pktd at gmail.com  Wed Oct  2 14:49:24 2013
From: josef.pktd at gmail.com (josef.pktd at gmail.com)
Date: Wed, 2 Oct 2013 14:49:24 -0400
Subject: [Numpy-discussion] Behavior of nan{max, min} and nanarg{max,
 min} for all-nan slices.
In-Reply-To: <CABDkGQmCTQVToKaWReZ_UHagot6QVVkUZyVMPa5iCPGzhJ1ykw@mail.gmail.com>
References: <CAB6mnxLO+c0CZHSUbL3NZ-quivy5NdSUtaM_iAL1ay2pUwXr0w@mail.gmail.com>
	<CABDkGQmWHrYdoEvJigFOnL4SAkOY4JeChW7woR3cZsr2ArwGMA@mail.gmail.com>
	<CAMMTP+AvPxqE3FwQrvnz74GQeso5M4KipUuAbaLcgy2OvxLQww@mail.gmail.com>
	<CAB6mnxKjDBPQ=HFbbA2f+8iOT=vuQy9x-JyKw_tdP1QSeM=sFw@mail.gmail.com>
	<CANNq6Fk1hmyZ+xpJjBhNJSv1ZnC_TG+c2rLDBg+1ciQ4rLHhQA@mail.gmail.com>
	<CABDkGQmCTQVToKaWReZ_UHagot6QVVkUZyVMPa5iCPGzhJ1ykw@mail.gmail.com>
Message-ID: <CAMMTP+Cty_8uWbYse4BXB0t2MRxhQumkgNTHsubfUw8cZ8PB1w@mail.gmail.com>

On Wed, Oct 2, 2013 at 2:05 PM, St?fan van der Walt <stefan at sun.ac.za> wrote:
> On 2 Oct 2013 19:14, "Benjamin Root" <ben.root at ou.edu> wrote:
>>
>> And it is logically consistent, I think.  a[nanargmax(a)] == nanmax(a)
>> (ignoring the silly detail that you can't do an equality on nans).
>
> Why do you call this a silly detail? It seems to me a fundamental flaw to
> this approach.

a nan is a nan is a NaN

>>> np.testing.assert_equal([0, np.nan], [0, np.nan])
>>>

Josef

>
> St?fan
>
>
> _______________________________________________
> NumPy-Discussion mailing list
> NumPy-Discussion at scipy.org
> http://mail.scipy.org/mailman/listinfo/numpy-discussion
>


From josef.pktd at gmail.com  Wed Oct  2 14:51:31 2013
From: josef.pktd at gmail.com (josef.pktd at gmail.com)
Date: Wed, 2 Oct 2013 14:51:31 -0400
Subject: [Numpy-discussion] Behavior of nan{max, min} and nanarg{max,
 min} for all-nan slices.
In-Reply-To: <CAMMTP+Cty_8uWbYse4BXB0t2MRxhQumkgNTHsubfUw8cZ8PB1w@mail.gmail.com>
References: <CAB6mnxLO+c0CZHSUbL3NZ-quivy5NdSUtaM_iAL1ay2pUwXr0w@mail.gmail.com>
	<CABDkGQmWHrYdoEvJigFOnL4SAkOY4JeChW7woR3cZsr2ArwGMA@mail.gmail.com>
	<CAMMTP+AvPxqE3FwQrvnz74GQeso5M4KipUuAbaLcgy2OvxLQww@mail.gmail.com>
	<CAB6mnxKjDBPQ=HFbbA2f+8iOT=vuQy9x-JyKw_tdP1QSeM=sFw@mail.gmail.com>
	<CANNq6Fk1hmyZ+xpJjBhNJSv1ZnC_TG+c2rLDBg+1ciQ4rLHhQA@mail.gmail.com>
	<CABDkGQmCTQVToKaWReZ_UHagot6QVVkUZyVMPa5iCPGzhJ1ykw@mail.gmail.com>
	<CAMMTP+Cty_8uWbYse4BXB0t2MRxhQumkgNTHsubfUw8cZ8PB1w@mail.gmail.com>
Message-ID: <CAMMTP+BVJWr=8MV5chvBs=nsrqu8HtB9X2avQ084SkRkVKabdw@mail.gmail.com>

On Wed, Oct 2, 2013 at 2:49 PM,  <josef.pktd at gmail.com> wrote:
> On Wed, Oct 2, 2013 at 2:05 PM, St?fan van der Walt <stefan at sun.ac.za> wrote:
>> On 2 Oct 2013 19:14, "Benjamin Root" <ben.root at ou.edu> wrote:
>>>
>>> And it is logically consistent, I think.  a[nanargmax(a)] == nanmax(a)
>>> (ignoring the silly detail that you can't do an equality on nans).
>>
>> Why do you call this a silly detail? It seems to me a fundamental flaw to
>> this approach.
>
> a nan is a nan is a NaN
>
>>>> np.testing.assert_equal([0, np.nan], [0, np.nan])
>>>>

and the functions have "nan" in their names
nan in - NaN out

what about nanmean, nansum, ...?

Josef

>
> Josef
>
>>
>> St?fan
>>
>>
>> _______________________________________________
>> NumPy-Discussion mailing list
>> NumPy-Discussion at scipy.org
>> http://mail.scipy.org/mailman/listinfo/numpy-discussion
>>


From ben.root at ou.edu  Wed Oct  2 15:04:56 2013
From: ben.root at ou.edu (Benjamin Root)
Date: Wed, 2 Oct 2013 15:04:56 -0400
Subject: [Numpy-discussion] Behavior of nan{max, min} and nanarg{max,
 min} for all-nan slices.
In-Reply-To: <CABDkGQmCTQVToKaWReZ_UHagot6QVVkUZyVMPa5iCPGzhJ1ykw@mail.gmail.com>
References: <CAB6mnxLO+c0CZHSUbL3NZ-quivy5NdSUtaM_iAL1ay2pUwXr0w@mail.gmail.com>
	<CABDkGQmWHrYdoEvJigFOnL4SAkOY4JeChW7woR3cZsr2ArwGMA@mail.gmail.com>
	<CAMMTP+AvPxqE3FwQrvnz74GQeso5M4KipUuAbaLcgy2OvxLQww@mail.gmail.com>
	<CAB6mnxKjDBPQ=HFbbA2f+8iOT=vuQy9x-JyKw_tdP1QSeM=sFw@mail.gmail.com>
	<CANNq6Fk1hmyZ+xpJjBhNJSv1ZnC_TG+c2rLDBg+1ciQ4rLHhQA@mail.gmail.com>
	<CABDkGQmCTQVToKaWReZ_UHagot6QVVkUZyVMPa5iCPGzhJ1ykw@mail.gmail.com>
Message-ID: <CANNq6Fkh9+tX6WPJ7_TO5u-T50qjfK_X7+-E1r4mvqaaqVvx0Q@mail.gmail.com>

On Wed, Oct 2, 2013 at 2:05 PM, St?fan van der Walt <stefan at sun.ac.za>wrote:

> On 2 Oct 2013 19:14, "Benjamin Root" <ben.root at ou.edu> wrote:
> >
> > And it is logically consistent, I think.  a[nanargmax(a)] == nanmax(a)
> (ignoring the silly detail that you can't do an equality on nans).
>
> Why do you call this a silly detail? It seems to me a fundamental flaw to
> this approach.
>
> Just saying that it conceptually makes sense, even if the exact code I
used wouldn't be perfectly correct. Because these are NaN functions, it
means that the users are already aware of the need to handle nans
appropriately. Just because you can't actually do equality between two NaNs
in the same way as one can do with numbers does not invalidate the concept.

Ben Root
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20131002/c0f7cd3c/attachment.html>

From njs at pobox.com  Wed Oct  2 15:19:14 2013
From: njs at pobox.com (Nathaniel Smith)
Date: Wed, 2 Oct 2013 20:19:14 +0100
Subject: [Numpy-discussion] Behavior of nan{max, min} and nanarg{max,
 min} for all-nan slices.
In-Reply-To: <CAMMTP+BVJWr=8MV5chvBs=nsrqu8HtB9X2avQ084SkRkVKabdw@mail.gmail.com>
References: <CAB6mnxLO+c0CZHSUbL3NZ-quivy5NdSUtaM_iAL1ay2pUwXr0w@mail.gmail.com>
	<CABDkGQmWHrYdoEvJigFOnL4SAkOY4JeChW7woR3cZsr2ArwGMA@mail.gmail.com>
	<CAMMTP+AvPxqE3FwQrvnz74GQeso5M4KipUuAbaLcgy2OvxLQww@mail.gmail.com>
	<CAB6mnxKjDBPQ=HFbbA2f+8iOT=vuQy9x-JyKw_tdP1QSeM=sFw@mail.gmail.com>
	<CANNq6Fk1hmyZ+xpJjBhNJSv1ZnC_TG+c2rLDBg+1ciQ4rLHhQA@mail.gmail.com>
	<CABDkGQmCTQVToKaWReZ_UHagot6QVVkUZyVMPa5iCPGzhJ1ykw@mail.gmail.com>
	<CAMMTP+Cty_8uWbYse4BXB0t2MRxhQumkgNTHsubfUw8cZ8PB1w@mail.gmail.com>
	<CAMMTP+BVJWr=8MV5chvBs=nsrqu8HtB9X2avQ084SkRkVKabdw@mail.gmail.com>
Message-ID: <CAPJVwB=sovDyYfF5SkB2FyWA4sKSQo4ARgZFb9M67SXRy3SkQw@mail.gmail.com>

On Wed, Oct 2, 2013 at 7:51 PM,  <josef.pktd at gmail.com> wrote:
> On Wed, Oct 2, 2013 at 2:49 PM,  <josef.pktd at gmail.com> wrote:
>> On Wed, Oct 2, 2013 at 2:05 PM, St?fan van der Walt <stefan at sun.ac.za> wrote:
>>> On 2 Oct 2013 19:14, "Benjamin Root" <ben.root at ou.edu> wrote:
>>>>
>>>> And it is logically consistent, I think.  a[nanargmax(a)] == nanmax(a)
>>>> (ignoring the silly detail that you can't do an equality on nans).
>>>
>>> Why do you call this a silly detail? It seems to me a fundamental flaw to
>>> this approach.
>>
>> a nan is a nan is a NaN
>>
>>>>> np.testing.assert_equal([0, np.nan], [0, np.nan])
>>>>>
>
> and the functions have "nan" in their names
> nan in - NaN out

This makes no sense :-). The nan in the names means "pretend the nans
aren't there", not "please scatter nans in the output"!

These are just vectorized operations that can fail in some cases and
not others, there's nothing special about the fact that the function's
definition also involves nan.

> what about nanmean, nansum, ...?

They do the same thing as mean([]), sum([]), etc., which are
well-defined. (nan and 0 respectively.)

-n


From charlesr.harris at gmail.com  Wed Oct  2 15:19:27 2013
From: charlesr.harris at gmail.com (Charles R Harris)
Date: Wed, 2 Oct 2013 13:19:27 -0600
Subject: [Numpy-discussion] Behavior of nan{max, min} and nanarg{max,
 min} for all-nan slices.
In-Reply-To: <CAMMTP+BVJWr=8MV5chvBs=nsrqu8HtB9X2avQ084SkRkVKabdw@mail.gmail.com>
References: <CAB6mnxLO+c0CZHSUbL3NZ-quivy5NdSUtaM_iAL1ay2pUwXr0w@mail.gmail.com>
	<CABDkGQmWHrYdoEvJigFOnL4SAkOY4JeChW7woR3cZsr2ArwGMA@mail.gmail.com>
	<CAMMTP+AvPxqE3FwQrvnz74GQeso5M4KipUuAbaLcgy2OvxLQww@mail.gmail.com>
	<CAB6mnxKjDBPQ=HFbbA2f+8iOT=vuQy9x-JyKw_tdP1QSeM=sFw@mail.gmail.com>
	<CANNq6Fk1hmyZ+xpJjBhNJSv1ZnC_TG+c2rLDBg+1ciQ4rLHhQA@mail.gmail.com>
	<CABDkGQmCTQVToKaWReZ_UHagot6QVVkUZyVMPa5iCPGzhJ1ykw@mail.gmail.com>
	<CAMMTP+Cty_8uWbYse4BXB0t2MRxhQumkgNTHsubfUw8cZ8PB1w@mail.gmail.com>
	<CAMMTP+BVJWr=8MV5chvBs=nsrqu8HtB9X2avQ084SkRkVKabdw@mail.gmail.com>
Message-ID: <CAB6mnxLDGeieOwr+6jnUtbf3BHRMcWYZcSaFtCuxkRF1_foYAA@mail.gmail.com>

On Wed, Oct 2, 2013 at 12:51 PM, <josef.pktd at gmail.com> wrote:

> On Wed, Oct 2, 2013 at 2:49 PM,  <josef.pktd at gmail.com> wrote:
> > On Wed, Oct 2, 2013 at 2:05 PM, St?fan van der Walt <stefan at sun.ac.za>
> wrote:
> >> On 2 Oct 2013 19:14, "Benjamin Root" <ben.root at ou.edu> wrote:
> >>>
> >>> And it is logically consistent, I think.  a[nanargmax(a)] == nanmax(a)
> >>> (ignoring the silly detail that you can't do an equality on nans).
> >>
> >> Why do you call this a silly detail? It seems to me a fundamental flaw
> to
> >> this approach.
> >
> > a nan is a nan is a NaN
> >
> >>>> np.testing.assert_equal([0, np.nan], [0, np.nan])
> >>>>
>
> and the functions have "nan" in their names
> nan in - NaN out
>
> what about nanmean, nansum, ...?
>

nanmean returns nan for empty slices while nansum returns nan in 1.8,
consistent with previous behavior, and will return 0 in 1.9.

The main problem I had was deciding what arg{max, min} should return as the
return value is an integer. I like your suggestion of returning 0.

One further possibility is to add a keyword 'raise' to make the behavior
selectable.

Chuck
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20131002/22f1a7f6/attachment.html>

From tmp50 at ukr.net  Wed Oct  2 15:32:41 2013
From: tmp50 at ukr.net (Dmitrey)
Date: Wed, 02 Oct 2013 22:32:41 +0300
Subject: [Numpy-discussion] MATLAB fsolve now available in Python
Message-ID: <1380742043.471698032.9rc2y0ro@frv43.ukr.net>

Hi all, 
New solver for systems of nonlinear equations ( SNLE ) has been connected to free Python framework OpenOpt: fsolve from? MATLAB ? Optimization Toolbox; uploaded into PYPI in v. 0.5112. 

As well as fmincon , currently it's available for Python 2 only. 
Unlike scipy.optimize fsolve, it can handle sparse derivatives, user-supplied or from FuncDesigner automatic differentiation. 
See the example with 15000 equations. 
To keep discussion in a single place please use the OpenOpt forum thread. Regards, D. 
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20131002/d92e3d48/attachment.html>

From njs at pobox.com  Wed Oct  2 15:35:46 2013
From: njs at pobox.com (Nathaniel Smith)
Date: Wed, 2 Oct 2013 20:35:46 +0100
Subject: [Numpy-discussion] Behavior of nan{max, min} and nanarg{max,
 min} for all-nan slices.
In-Reply-To: <CAB6mnxLDGeieOwr+6jnUtbf3BHRMcWYZcSaFtCuxkRF1_foYAA@mail.gmail.com>
References: <CAB6mnxLO+c0CZHSUbL3NZ-quivy5NdSUtaM_iAL1ay2pUwXr0w@mail.gmail.com>
	<CABDkGQmWHrYdoEvJigFOnL4SAkOY4JeChW7woR3cZsr2ArwGMA@mail.gmail.com>
	<CAMMTP+AvPxqE3FwQrvnz74GQeso5M4KipUuAbaLcgy2OvxLQww@mail.gmail.com>
	<CAB6mnxKjDBPQ=HFbbA2f+8iOT=vuQy9x-JyKw_tdP1QSeM=sFw@mail.gmail.com>
	<CANNq6Fk1hmyZ+xpJjBhNJSv1ZnC_TG+c2rLDBg+1ciQ4rLHhQA@mail.gmail.com>
	<CABDkGQmCTQVToKaWReZ_UHagot6QVVkUZyVMPa5iCPGzhJ1ykw@mail.gmail.com>
	<CAMMTP+Cty_8uWbYse4BXB0t2MRxhQumkgNTHsubfUw8cZ8PB1w@mail.gmail.com>
	<CAMMTP+BVJWr=8MV5chvBs=nsrqu8HtB9X2avQ084SkRkVKabdw@mail.gmail.com>
	<CAB6mnxLDGeieOwr+6jnUtbf3BHRMcWYZcSaFtCuxkRF1_foYAA@mail.gmail.com>
Message-ID: <CAPJVwBm2z-qDoNcosEh3_vMMFNE74unFU492Mnu31_141dt8qg@mail.gmail.com>

On Wed, Oct 2, 2013 at 8:19 PM, Charles R Harris
<charlesr.harris at gmail.com> wrote:
>
>
>
> On Wed, Oct 2, 2013 at 12:51 PM, <josef.pktd at gmail.com> wrote:
>>
>> On Wed, Oct 2, 2013 at 2:49 PM,  <josef.pktd at gmail.com> wrote:
>> > On Wed, Oct 2, 2013 at 2:05 PM, St?fan van der Walt <stefan at sun.ac.za>
>> > wrote:
>> >> On 2 Oct 2013 19:14, "Benjamin Root" <ben.root at ou.edu> wrote:
>> >>>
>> >>> And it is logically consistent, I think.  a[nanargmax(a)] == nanmax(a)
>> >>> (ignoring the silly detail that you can't do an equality on nans).
>> >>
>> >> Why do you call this a silly detail? It seems to me a fundamental flaw
>> >> to
>> >> this approach.
>> >
>> > a nan is a nan is a NaN
>> >
>> >>>> np.testing.assert_equal([0, np.nan], [0, np.nan])
>> >>>>
>>
>> and the functions have "nan" in their names
>> nan in - NaN out
>>
>> what about nanmean, nansum, ...?
>
> nanmean returns nan for empty slices while nansum returns nan in 1.8,
> consistent with previous behavior, and will return 0 in 1.9.
>
> The main problem I had was deciding what arg{max, min} should return as the
> return value is an integer. I like your suggestion of returning 0.

I don't understand the justification for returning 0 at all. "nan" is
not the max or min or the array. Even if argmax/argmin return nan[1],
it's just a special code meaning "undefined", it has no relation to
the nans inside the array. So returning 0 just feels to me like pure
"we have to return *something* and this something! (that we can kind
of justify is no-one looks too hard!)". This exactly the impulse that
"when in doubt refuse the temptation to guess" is written to
counteract...

Seriously what user calls nanargmax and *wants* to get pointed to a
random nan inside the array? Isn't the whole point of calling
nanargmax to avoid exactly this situation?

-n

[1] It seems clear that we shouldn't mess about with argmax/argmin for
1.8, but my guess is that in the long run we'll come up with a more
general convention for signalling partial errors, and eventually want
to switch nanargmax/nanargmin to using that.


From stefan at sun.ac.za  Wed Oct  2 15:43:38 2013
From: stefan at sun.ac.za (=?ISO-8859-1?Q?St=E9fan_van_der_Walt?=)
Date: Wed, 2 Oct 2013 21:43:38 +0200
Subject: [Numpy-discussion] Behavior of nan{max, min} and nanarg{max,
 min} for all-nan slices.
In-Reply-To: <CAB6mnxLDGeieOwr+6jnUtbf3BHRMcWYZcSaFtCuxkRF1_foYAA@mail.gmail.com>
References: <CAB6mnxLO+c0CZHSUbL3NZ-quivy5NdSUtaM_iAL1ay2pUwXr0w@mail.gmail.com>
	<CABDkGQmWHrYdoEvJigFOnL4SAkOY4JeChW7woR3cZsr2ArwGMA@mail.gmail.com>
	<CAMMTP+AvPxqE3FwQrvnz74GQeso5M4KipUuAbaLcgy2OvxLQww@mail.gmail.com>
	<CAB6mnxKjDBPQ=HFbbA2f+8iOT=vuQy9x-JyKw_tdP1QSeM=sFw@mail.gmail.com>
	<CANNq6Fk1hmyZ+xpJjBhNJSv1ZnC_TG+c2rLDBg+1ciQ4rLHhQA@mail.gmail.com>
	<CABDkGQmCTQVToKaWReZ_UHagot6QVVkUZyVMPa5iCPGzhJ1ykw@mail.gmail.com>
	<CAMMTP+Cty_8uWbYse4BXB0t2MRxhQumkgNTHsubfUw8cZ8PB1w@mail.gmail.com>
	<CAMMTP+BVJWr=8MV5chvBs=nsrqu8HtB9X2avQ084SkRkVKabdw@mail.gmail.com>
	<CAB6mnxLDGeieOwr+6jnUtbf3BHRMcWYZcSaFtCuxkRF1_foYAA@mail.gmail.com>
Message-ID: <CABDkGQ=2=tmUu_mVgkqpFfH8LnDg4whOVYhOi1nsRoo2vqVoZQ@mail.gmail.com>

On 2 Oct 2013 21:19, "Charles R Harris" <charlesr.harris at gmail.com> wrote:
>
>
> The main problem I had was deciding what arg{max, min} should return as
the return value is an integer. I like your suggestion of returning 0.

This doesn't allow the user to know the difference between valid and"
invalid" output, does it?

> One further possibility is to add a keyword 'raise' to make the behavior
selectable.

Preferably just pick a sensible default.

St?fan
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20131002/4e2efc66/attachment.html>

From piet at vanoostrum.org  Wed Oct  2 16:31:46 2013
From: piet at vanoostrum.org (Piet van Oostrum)
Date: Wed, 02 Oct 2013 16:31:46 -0400
Subject: [Numpy-discussion] 1.8.0rc1
References: <CAB6mnxJ+WAucyTEoj4w6nN67ja=jWH3kU+b7AZ+K4M1B37HH0g@mail.gmail.com>
Message-ID: <m2siwjjtxp.fsf@cochabamba.vanoostrum.org>

Charles R Harris <charlesr.harris at gmail.com> writes:

> Hi All,
>
> NumPy 1.8.0rc1 is up now on sourceforge .The binary builds are included except for Python 3.3 on
> windows, which will arrive later. Many thanks to Ralf for the binaries, and to those who found and
> fixed the bugs in the last beta. Any remaining bugs are all my fault ;) I hope this will be the
> last release before final, so please test it thoroughly.

I have installed 1.0.8rc1 on Python 3.3.2 on Mac OS X Snow Leopard (10.6.8) from the binary installer http://ufpr.dl.sourceforge.net/project/numpy/NumPy/1.8.0rc1/numpy-1.8.0rc1-py3.3-python.org-macosx10.6.dmg an the test fails with 20 errors. I have tried it also with installing from source but it also gives these erros (I haven't checked if the errors are the same bit for bit, but they were also 20). Here is the output of the test run.

-------------- next part --------------
An embedded and charset-unspecified text was scrubbed...
Name: numpy.test
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20131002/62957027/attachment.ksh>
-------------- next part --------------

-- 
Piet van Oostrum <piet at vanoostrum.org>
WWW: http://pietvanoostrum.com/
PGP key: [8DAE142BE17999C4]

From charlesr.harris at gmail.com  Wed Oct  2 19:29:28 2013
From: charlesr.harris at gmail.com (Charles R Harris)
Date: Wed, 2 Oct 2013 17:29:28 -0600
Subject: [Numpy-discussion] 1.8.0, nan functions, and pandas
Message-ID: <CAB6mnxLyn8BHDMTMfaLez__xv6V=kGQHMWO9w5TYXOF=mr_hqw@mail.gmail.com>

Hi All,

There was a problem with pandas 0.12 and the numpy nan functions when
applying the nansum function to a pandas Series object. We thought we had
fixed it by the slight of hand use of `a.sum` instead of `add.reduce(a)` so
that the Series object could use its own version of `sum` which overrides
the ndarray version. However, the pandas sum method has a different
signature, so success was illusory and due to not naming the keyword
arguments in the call. The new nan functions nanmean, nanvar, and nanstd
also failed, as the actually do name the keyword arguments. For pandas
development all the functions work as the pandas sum method now include a
`**kwargs` which gobbles up the missing keyword `keepdims`.

At this point I don't think there is anything we can legitimately do to fix
the pandas 0.12 problem. I propose to name all the keyword arguments
arguments so that pandas fails, as it should, and simply document that the
nan functions will not work with pandas versions < 0.13.

Thoughts?

Chuck
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20131002/0a24599b/attachment.html>

From charlesr.harris at gmail.com  Wed Oct  2 19:41:00 2013
From: charlesr.harris at gmail.com (Charles R Harris)
Date: Wed, 2 Oct 2013 17:41:00 -0600
Subject: [Numpy-discussion] 1.8.0, nan functions, and pandas
In-Reply-To: <CAB6mnxLyn8BHDMTMfaLez__xv6V=kGQHMWO9w5TYXOF=mr_hqw@mail.gmail.com>
References: <CAB6mnxLyn8BHDMTMfaLez__xv6V=kGQHMWO9w5TYXOF=mr_hqw@mail.gmail.com>
Message-ID: <CAB6mnxKJ-HRjmAejj4s_jec7qGq47asGyHrvpXpMba_jhodSRg@mail.gmail.com>

On Wed, Oct 2, 2013 at 5:29 PM, Charles R Harris
<charlesr.harris at gmail.com>wrote:

> Hi All,
>
> There was a problem with pandas 0.12 and the numpy nan functions when
> applying the nansum function to a pandas Series object. We thought we had
> fixed it by the slight of hand use of `a.sum` instead of `add.reduce(a)` so
> that the Series object could use its own version of `sum` which overrides
> the ndarray version. However, the pandas sum method has a different
> signature, so success was illusory and due to not naming the keyword
> arguments in the call. The new nan functions nanmean, nanvar, and nanstd
> also failed, as the actually do name the keyword arguments. For pandas
> development all the functions work as the pandas sum method now include a
> `**kwargs` which gobbles up the missing keyword `keepdims`.
>
> At this point I don't think there is anything we can legitimately do to
> fix the pandas 0.12 problem. I propose to name all the keyword arguments
> arguments so that pandas fails, as it should, and simply document that the
> nan functions will not work with pandas versions < 0.13.
>
> Thoughts?
>
>
One thing we maybe should do is convert to arrays and wrap the result as we
do for the linalg functions. Currently, the nanfunctions happily keep
subtypes of ndarray and work with them. But I am not sure the usual
wrapping will work for Series. l note that Matrix also fails in the same
way as Series as it's sum function doesn't have a keepdims keyword. Hmm...
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20131002/c99adb16/attachment.html>

From daverz at gmail.com  Wed Oct  2 21:00:06 2013
From: daverz at gmail.com (Dave Cook)
Date: Wed, 2 Oct 2013 18:00:06 -0700
Subject: [Numpy-discussion] Products of small float32 values.
Message-ID: <CAJL+0ek=PMcnjTt38vMu--mMpzgEJ=fAoNjUbCgbuwoBA+NjvQ@mail.gmail.com>

Can someone explain what is going on here?

In [153]:

small = ones(1, dtype='float32')

In [154]:

small

Out[154]:

array([ 1.], dtype=float32)

In [155]:

small*1e-45

Out[155]:

array([  1.40129846e-45], dtype=float32)

In [156]:

small*1e-46

Out[156]:

array([ 0.], dtype=float32)

I would expect float32 to be able to represent
numbers with exponents as small as -127.

Thanks,
Dave Cook
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20131002/1d2b3841/attachment.html>

From matthew.brett at gmail.com  Wed Oct  2 21:06:08 2013
From: matthew.brett at gmail.com (Matthew Brett)
Date: Wed, 2 Oct 2013 18:06:08 -0700
Subject: [Numpy-discussion] Products of small float32 values.
In-Reply-To: <CAJL+0ek=PMcnjTt38vMu--mMpzgEJ=fAoNjUbCgbuwoBA+NjvQ@mail.gmail.com>
References: <CAJL+0ek=PMcnjTt38vMu--mMpzgEJ=fAoNjUbCgbuwoBA+NjvQ@mail.gmail.com>
Message-ID: <CAH6Pt5q2LrFkEraqTCy8+PUq6QoppbMzQpapQeWXMTwnAYRVcw@mail.gmail.com>

Hi,

On Wed, Oct 2, 2013 at 6:00 PM, Dave Cook <daverz at gmail.com> wrote:
> Can someone explain what is going on here?
>
> In [153]:
>
> small = ones(1, dtype='float32')
>
> In [154]:
>
> small
>
> Out[154]:
>
> array([ 1.], dtype=float32)
>
> In [155]:
>
> small*1e-45
>
> Out[155]:
>
> array([  1.40129846e-45], dtype=float32)
>
> In [156]:
>
> small*1e-46
>
> Out[156]:
>
> array([ 0.], dtype=float32)
>
> I would expect float32 to be able to represent
> numbers with exponents as small as -127.

Is it possible you are thinking of 2**-126 rather than 10**-126?

In [3]: print np.finfo(np.float32)
Machine parameters for float32
---------------------------------------------------------------------
precision=  6   resolution= 1.0000000e-06
machep=   -23   eps=        1.1920929e-07
negep =   -24   epsneg=     5.9604645e-08
minexp=  -126   tiny=       1.1754944e-38
maxexp=   128   max=        3.4028235e+38
nexp  =     8   min=        -max
---------------------------------------------------------------------


In [4]: 2**-126
Out[4]: 1.1754943508222875e-38

So I think your 10**-45 was already 'subnormal' and then 10**-46 went
down to 0...

Cheers,

Matthew


From daverz at gmail.com  Wed Oct  2 21:20:56 2013
From: daverz at gmail.com (Dave Cook)
Date: Wed, 2 Oct 2013 18:20:56 -0700
Subject: [Numpy-discussion] Products of small float32 values.
In-Reply-To: <CAH6Pt5q2LrFkEraqTCy8+PUq6QoppbMzQpapQeWXMTwnAYRVcw@mail.gmail.com>
References: <CAJL+0ek=PMcnjTt38vMu--mMpzgEJ=fAoNjUbCgbuwoBA+NjvQ@mail.gmail.com>
	<CAH6Pt5q2LrFkEraqTCy8+PUq6QoppbMzQpapQeWXMTwnAYRVcw@mail.gmail.com>
Message-ID: <CAJL+0em1O5CbyQaMCX8AWZwTa68s++DKW4CnMqwNiHsVqUG8FQ@mail.gmail.com>

On Wed, Oct 2, 2013 at 6:06 PM, Matthew Brett <matthew.brett at gmail.com>wrote:

>
>
> Is it possible you are thinking of 2**-126 rather than 10**-126?
>
>
Yup, brainfart...



> In [3]: print np.finfo(np.float32)
> Machine parameters for float32
> ---------------------------------------------------------------------
> precision=  6   resolution= 1.0000000e-06
> machep=   -23   eps=        1.1920929e-07
> negep =   -24   epsneg=     5.9604645e-08
> minexp=  -126   tiny=       1.1754944e-38
> maxexp=   128   max=        3.4028235e+38
> nexp  =     8   min=        -max
> ---------------------------------------------------------------------
>

Thanks for that.

Dave Cook
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20131002/92dcec91/attachment.html>

From daniele at grinta.net  Thu Oct  3 06:06:33 2013
From: daniele at grinta.net (Daniele Nicolodi)
Date: Thu, 03 Oct 2013 12:06:33 +0200
Subject: [Numpy-discussion] Behavior of nan{max, min} and nanarg{max,
 min} for all-nan slices.
In-Reply-To: <CAB6mnxLDGeieOwr+6jnUtbf3BHRMcWYZcSaFtCuxkRF1_foYAA@mail.gmail.com>
References: <CAB6mnxLO+c0CZHSUbL3NZ-quivy5NdSUtaM_iAL1ay2pUwXr0w@mail.gmail.com>
	<CABDkGQmWHrYdoEvJigFOnL4SAkOY4JeChW7woR3cZsr2ArwGMA@mail.gmail.com>
	<CAMMTP+AvPxqE3FwQrvnz74GQeso5M4KipUuAbaLcgy2OvxLQww@mail.gmail.com>
	<CAB6mnxKjDBPQ=HFbbA2f+8iOT=vuQy9x-JyKw_tdP1QSeM=sFw@mail.gmail.com>
	<CANNq6Fk1hmyZ+xpJjBhNJSv1ZnC_TG+c2rLDBg+1ciQ4rLHhQA@mail.gmail.com>
	<CABDkGQmCTQVToKaWReZ_UHagot6QVVkUZyVMPa5iCPGzhJ1ykw@mail.gmail.com>
	<CAMMTP+Cty_8uWbYse4BXB0t2MRxhQumkgNTHsubfUw8cZ8PB1w@mail.gmail.com>
	<CAMMTP+BVJWr=8MV5chvBs=nsrqu8HtB9X2avQ084SkRkVKabdw@mail.gmail.com>
	<CAB6mnxLDGeieOwr+6jnUtbf3BHRMcWYZcSaFtCuxkRF1_foYAA@mail.gmail.com>
Message-ID: <524D41A9.6040309@grinta.net>

Hello,

sorry, I don't know where exactly jump in in the thread, it is getting
quite long and articulated...

On 02/10/2013 21:19, Charles R Harris wrote:
> The main problem I had was deciding what arg{max, min} should return as
> the return value is an integer. I like your suggestion of returning 0.

What about returning -1? It is still an integer (on my numpy version the
return value is a signed integer), it still has the property that
a[np.argmin(a)] == nan for a of only nans, but it is easily identifiable
as an anomalous return value if needed.

> One further possibility is to add a keyword 'raise' to make the behavior
> selectable.

I don't like function arguments that change the function behavior.

Cheers,
Daniele



From charlesr.harris at gmail.com  Thu Oct  3 07:56:31 2013
From: charlesr.harris at gmail.com (Charles R Harris)
Date: Thu, 3 Oct 2013 05:56:31 -0600
Subject: [Numpy-discussion] Behavior of nan{max, min} and nanarg{max,
 min} for all-nan slices.
In-Reply-To: <524D41A9.6040309@grinta.net>
References: <CAB6mnxLO+c0CZHSUbL3NZ-quivy5NdSUtaM_iAL1ay2pUwXr0w@mail.gmail.com>
	<CABDkGQmWHrYdoEvJigFOnL4SAkOY4JeChW7woR3cZsr2ArwGMA@mail.gmail.com>
	<CAMMTP+AvPxqE3FwQrvnz74GQeso5M4KipUuAbaLcgy2OvxLQww@mail.gmail.com>
	<CAB6mnxKjDBPQ=HFbbA2f+8iOT=vuQy9x-JyKw_tdP1QSeM=sFw@mail.gmail.com>
	<CANNq6Fk1hmyZ+xpJjBhNJSv1ZnC_TG+c2rLDBg+1ciQ4rLHhQA@mail.gmail.com>
	<CABDkGQmCTQVToKaWReZ_UHagot6QVVkUZyVMPa5iCPGzhJ1ykw@mail.gmail.com>
	<CAMMTP+Cty_8uWbYse4BXB0t2MRxhQumkgNTHsubfUw8cZ8PB1w@mail.gmail.com>
	<CAMMTP+BVJWr=8MV5chvBs=nsrqu8HtB9X2avQ084SkRkVKabdw@mail.gmail.com>
	<CAB6mnxLDGeieOwr+6jnUtbf3BHRMcWYZcSaFtCuxkRF1_foYAA@mail.gmail.com>
	<524D41A9.6040309@grinta.net>
Message-ID: <CAB6mnxK9aSV3tpH6Fpp2rkX8bz-Tos4YQLpRBOPAuj=CcXGB_w@mail.gmail.com>

On Thu, Oct 3, 2013 at 4:06 AM, Daniele Nicolodi <daniele at grinta.net> wrote:

> Hello,
>
> sorry, I don't know where exactly jump in in the thread, it is getting
> quite long and articulated...
>
> On 02/10/2013 21:19, Charles R Harris wrote:
> > The main problem I had was deciding what arg{max, min} should return as
> > the return value is an integer. I like your suggestion of returning 0.
>
> What about returning -1? It is still an integer (on my numpy version the
> return value is a signed integer), it still has the property that
> a[np.argmin(a)] == nan for a of only nans, but it is easily identifiable
> as an anomalous return value if needed.
>

The problem is that -1 is a valid index, whereas intp.min will always be
out of range and lead to an IndexError if used.

Chuck
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20131003/e77e0d12/attachment.html>

From daniele at grinta.net  Thu Oct  3 08:34:39 2013
From: daniele at grinta.net (Daniele Nicolodi)
Date: Thu, 03 Oct 2013 14:34:39 +0200
Subject: [Numpy-discussion] Behavior of nan{max, min} and nanarg{max,
 min} for all-nan slices.
In-Reply-To: <CAB6mnxK9aSV3tpH6Fpp2rkX8bz-Tos4YQLpRBOPAuj=CcXGB_w@mail.gmail.com>
References: <CAB6mnxLO+c0CZHSUbL3NZ-quivy5NdSUtaM_iAL1ay2pUwXr0w@mail.gmail.com>
	<CABDkGQmWHrYdoEvJigFOnL4SAkOY4JeChW7woR3cZsr2ArwGMA@mail.gmail.com>
	<CAMMTP+AvPxqE3FwQrvnz74GQeso5M4KipUuAbaLcgy2OvxLQww@mail.gmail.com>
	<CAB6mnxKjDBPQ=HFbbA2f+8iOT=vuQy9x-JyKw_tdP1QSeM=sFw@mail.gmail.com>
	<CANNq6Fk1hmyZ+xpJjBhNJSv1ZnC_TG+c2rLDBg+1ciQ4rLHhQA@mail.gmail.com>
	<CABDkGQmCTQVToKaWReZ_UHagot6QVVkUZyVMPa5iCPGzhJ1ykw@mail.gmail.com>
	<CAMMTP+Cty_8uWbYse4BXB0t2MRxhQumkgNTHsubfUw8cZ8PB1w@mail.gmail.com>
	<CAMMTP+BVJWr=8MV5chvBs=nsrqu8HtB9X2avQ084SkRkVKabdw@mail.gmail.com>
	<CAB6mnxLDGeieOwr+6jnUtbf3BHRMcWYZcSaFtCuxkRF1_foYAA@mail.gmail.com>
	<524D41A9.6040309@grinta.net>
	<CAB6mnxK9aSV3tpH6Fpp2rkX8bz-Tos4YQLpRBOPAuj=CcXGB_w@mail.gmail.com>
Message-ID: <524D645F.3020608@grinta.net>

On 03/10/2013 13:56, Charles R Harris wrote:
> On Thu, Oct 3, 2013 at 4:06 AM, Daniele Nicolodi <daniele at grinta.net
> <mailto:daniele at grinta.net>> wrote:
> 
>     Hello,
> 
>     sorry, I don't know where exactly jump in in the thread, it is getting
>     quite long and articulated...
> 
>     On 02/10/2013 21:19, Charles R Harris wrote:
>     > The main problem I had was deciding what arg{max, min} should
>     return as
>     > the return value is an integer. I like your suggestion of returning 0.
> 
>     What about returning -1? It is still an integer (on my numpy version the
>     return value is a signed integer), it still has the property that
>     a[np.argmin(a)] == nan for a of only nans, but it is easily identifiable
>     as an anomalous return value if needed.
> 
> 
> The problem is that -1 is a valid index, whereas intp.min will always be
> out of range and lead to an IndexError if used.

If the goal is to have an error raised, just do it and do not rely on
the fact that using an invalid index will soon or later result in an error.

My proposal was a compromise to the proposal of returning 0. 0 is a
valid index that cannot be distinguished from a valid return value, -1
is a valid index but can be easily distinguished as a special case.

I don't have a strong preference between

try:
    i = np.nanargmin(a)
except ValueError:
    something()

and

i = np.nanargmin(a)
if i < 0:
    something()

but definitely I don't like returning 0:

i = np.nanargmin(a)
if i == 0:
    # uhm, wait, is the minimum at index 0 or the array is all nans?!?
    if np.isnan(a[0]):
        something()

Cheers,
Daniele



From ben.root at ou.edu  Thu Oct  3 09:10:56 2013
From: ben.root at ou.edu (Benjamin Root)
Date: Thu, 3 Oct 2013 09:10:56 -0400
Subject: [Numpy-discussion] Behavior of nan{max, min} and nanarg{max,
 min} for all-nan slices.
In-Reply-To: <524D41A9.6040309@grinta.net>
References: <CAB6mnxLO+c0CZHSUbL3NZ-quivy5NdSUtaM_iAL1ay2pUwXr0w@mail.gmail.com>
	<CABDkGQmWHrYdoEvJigFOnL4SAkOY4JeChW7woR3cZsr2ArwGMA@mail.gmail.com>
	<CAMMTP+AvPxqE3FwQrvnz74GQeso5M4KipUuAbaLcgy2OvxLQww@mail.gmail.com>
	<CAB6mnxKjDBPQ=HFbbA2f+8iOT=vuQy9x-JyKw_tdP1QSeM=sFw@mail.gmail.com>
	<CANNq6Fk1hmyZ+xpJjBhNJSv1ZnC_TG+c2rLDBg+1ciQ4rLHhQA@mail.gmail.com>
	<CABDkGQmCTQVToKaWReZ_UHagot6QVVkUZyVMPa5iCPGzhJ1ykw@mail.gmail.com>
	<CAMMTP+Cty_8uWbYse4BXB0t2MRxhQumkgNTHsubfUw8cZ8PB1w@mail.gmail.com>
	<CAMMTP+BVJWr=8MV5chvBs=nsrqu8HtB9X2avQ084SkRkVKabdw@mail.gmail.com>
	<CAB6mnxLDGeieOwr+6jnUtbf3BHRMcWYZcSaFtCuxkRF1_foYAA@mail.gmail.com>
	<524D41A9.6040309@grinta.net>
Message-ID: <CANNq6F=v4XzPTvgg88qEnuSm44qo50+2OW1tP7C9Px10FJswGQ@mail.gmail.com>

On Thu, Oct 3, 2013 at 6:06 AM, Daniele Nicolodi <daniele at grinta.net> wrote:

> Hello,
>
> sorry, I don't know where exactly jump in in the thread, it is getting
> quite long and articulated...
>
> On 02/10/2013 21:19, Charles R Harris wrote:
> > The main problem I had was deciding what arg{max, min} should return as
> > the return value is an integer. I like your suggestion of returning 0.
>
> What about returning -1? It is still an integer (on my numpy version the
> return value is a signed integer), it still has the property that
> a[np.argmin(a)] == nan for a of only nans, but it is easily identifiable
> as an anomalous return value if needed.
>
>
This actually makes a lot of sense. We would never return -1 for any other
reason. And if it is used for indexing anywhere else, that's (mostly) ok. A
problem might occur if the indexes gathered from this function are then
used to define slices. But I can't really convince myself that it would be
all that terrible in that case, too. Documentation will be paramount here.

Ben Root
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20131003/a345b3a4/attachment.html>

From jsseabold at gmail.com  Thu Oct  3 09:57:21 2013
From: jsseabold at gmail.com (Skipper Seabold)
Date: Thu, 3 Oct 2013 09:57:21 -0400
Subject: [Numpy-discussion] Behavior of nan{max, min} and nanarg{max,
 min} for all-nan slices.
In-Reply-To: <CANNq6F=v4XzPTvgg88qEnuSm44qo50+2OW1tP7C9Px10FJswGQ@mail.gmail.com>
References: <CAB6mnxLO+c0CZHSUbL3NZ-quivy5NdSUtaM_iAL1ay2pUwXr0w@mail.gmail.com>
	<CABDkGQmWHrYdoEvJigFOnL4SAkOY4JeChW7woR3cZsr2ArwGMA@mail.gmail.com>
	<CAMMTP+AvPxqE3FwQrvnz74GQeso5M4KipUuAbaLcgy2OvxLQww@mail.gmail.com>
	<CAB6mnxKjDBPQ=HFbbA2f+8iOT=vuQy9x-JyKw_tdP1QSeM=sFw@mail.gmail.com>
	<CANNq6Fk1hmyZ+xpJjBhNJSv1ZnC_TG+c2rLDBg+1ciQ4rLHhQA@mail.gmail.com>
	<CABDkGQmCTQVToKaWReZ_UHagot6QVVkUZyVMPa5iCPGzhJ1ykw@mail.gmail.com>
	<CAMMTP+Cty_8uWbYse4BXB0t2MRxhQumkgNTHsubfUw8cZ8PB1w@mail.gmail.com>
	<CAMMTP+BVJWr=8MV5chvBs=nsrqu8HtB9X2avQ084SkRkVKabdw@mail.gmail.com>
	<CAB6mnxLDGeieOwr+6jnUtbf3BHRMcWYZcSaFtCuxkRF1_foYAA@mail.gmail.com>
	<524D41A9.6040309@grinta.net>
	<CANNq6F=v4XzPTvgg88qEnuSm44qo50+2OW1tP7C9Px10FJswGQ@mail.gmail.com>
Message-ID: <CAKF=Djss+A+=kVkc_xW=CToSCFFUXfeb1+OHPh_r8s_hVJNzow@mail.gmail.com>

On Thu, Oct 3, 2013 at 9:10 AM, Benjamin Root <ben.root at ou.edu> wrote:
> On Thu, Oct 3, 2013 at 6:06 AM, Daniele Nicolodi <daniele at grinta.net> wrote:
>>
>> Hello,
>>
>> sorry, I don't know where exactly jump in in the thread, it is getting
>> quite long and articulated...
>>
>> On 02/10/2013 21:19, Charles R Harris wrote:
>> > The main problem I had was deciding what arg{max, min} should return as
>> > the return value is an integer. I like your suggestion of returning 0.
>>
>> What about returning -1? It is still an integer (on my numpy version the
>> return value is a signed integer), it still has the property that
>> a[np.argmin(a)] == nan for a of only nans, but it is easily identifiable
>> as an anomalous return value if needed.
>>
>
> This actually makes a lot of sense. We would never return -1 for any other
> reason. And if it is used for indexing anywhere else, that's (mostly) ok. A
> problem might occur if the indexes gathered from this function are then used
> to define slices. But I can't really convince myself that it would be all
> that terrible in that case, too. Documentation will be paramount here.

Please, no. It's another thing to remember and another way to shoot
yourself in the foot and introduce casual bugs.

FWIW, my vote is to raise an error or return a nan, which will likely
eventually raise an error. If I have all nans, it's usually the case
that something's off, and I'd like to know sooner rather than later.

Skipper


From charlesr.harris at gmail.com  Thu Oct  3 14:59:27 2013
From: charlesr.harris at gmail.com (Charles R Harris)
Date: Thu, 3 Oct 2013 12:59:27 -0600
Subject: [Numpy-discussion] Behavior of nan{max, min} and nanarg{max,
 min} for all-nan slices.
In-Reply-To: <CAKF=Djss+A+=kVkc_xW=CToSCFFUXfeb1+OHPh_r8s_hVJNzow@mail.gmail.com>
References: <CAB6mnxLO+c0CZHSUbL3NZ-quivy5NdSUtaM_iAL1ay2pUwXr0w@mail.gmail.com>
	<CABDkGQmWHrYdoEvJigFOnL4SAkOY4JeChW7woR3cZsr2ArwGMA@mail.gmail.com>
	<CAMMTP+AvPxqE3FwQrvnz74GQeso5M4KipUuAbaLcgy2OvxLQww@mail.gmail.com>
	<CAB6mnxKjDBPQ=HFbbA2f+8iOT=vuQy9x-JyKw_tdP1QSeM=sFw@mail.gmail.com>
	<CANNq6Fk1hmyZ+xpJjBhNJSv1ZnC_TG+c2rLDBg+1ciQ4rLHhQA@mail.gmail.com>
	<CABDkGQmCTQVToKaWReZ_UHagot6QVVkUZyVMPa5iCPGzhJ1ykw@mail.gmail.com>
	<CAMMTP+Cty_8uWbYse4BXB0t2MRxhQumkgNTHsubfUw8cZ8PB1w@mail.gmail.com>
	<CAMMTP+BVJWr=8MV5chvBs=nsrqu8HtB9X2avQ084SkRkVKabdw@mail.gmail.com>
	<CAB6mnxLDGeieOwr+6jnUtbf3BHRMcWYZcSaFtCuxkRF1_foYAA@mail.gmail.com>
	<524D41A9.6040309@grinta.net>
	<CANNq6F=v4XzPTvgg88qEnuSm44qo50+2OW1tP7C9Px10FJswGQ@mail.gmail.com>
	<CAKF=Djss+A+=kVkc_xW=CToSCFFUXfeb1+OHPh_r8s_hVJNzow@mail.gmail.com>
Message-ID: <CAB6mnxLoRDZvHnFpVFK5e6VeqfzqxTQX2wLe+SKZ5SxzqXgUtg@mail.gmail.com>

<snip>


> Please, no. It's another thing to remember and another way to shoot
> yourself in the foot and introduce casual bugs.
>
> FWIW, my vote is to raise an error or return a nan, which will likely
> eventually raise an error. If I have all nans, it's usually the case
> that something's off, and I'd like to know sooner rather than later.
>
>
Here is what I have currently implemented. First, define an AllNanError

class AllNanError(ValueError):
    def __init__(self, msg, result):
        ValueError.__init__(self, msg)
        self.result = result

For nanmax/nanmin/nanargmax/nanargmin this error is raised for all-nan axis
and the result is attached. The exception can then be caught and the result
examined. A ValueError is what amax, amin return for empty arrays.

For nanmax/nanmin the result for an empty slice is nan. For
argnanmax/argnanmin the result of an empty slice is -1, which is easier to
read and remember than intp.min. A ValueError is what argmin, argmax
currently return for empty arrays. Note that both of these functions can
give wrong results if they contain some min/max values respectively. That
is an old bug and I haven't fixed it.

The nanmean/nanvar/nanstd functions currently raise a warning for all-nan
slices and the result for such is nan. These could also be made to raise an
error.

Thoughts?

Chuck
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20131003/bff303af/attachment.html>

From njs at pobox.com  Thu Oct  3 15:11:37 2013
From: njs at pobox.com (Nathaniel Smith)
Date: Thu, 3 Oct 2013 20:11:37 +0100
Subject: [Numpy-discussion] Behavior of nan{max, min} and nanarg{max,
 min} for all-nan slices.
In-Reply-To: <CAB6mnxLoRDZvHnFpVFK5e6VeqfzqxTQX2wLe+SKZ5SxzqXgUtg@mail.gmail.com>
References: <CAB6mnxLO+c0CZHSUbL3NZ-quivy5NdSUtaM_iAL1ay2pUwXr0w@mail.gmail.com>
	<CABDkGQmWHrYdoEvJigFOnL4SAkOY4JeChW7woR3cZsr2ArwGMA@mail.gmail.com>
	<CAMMTP+AvPxqE3FwQrvnz74GQeso5M4KipUuAbaLcgy2OvxLQww@mail.gmail.com>
	<CAB6mnxKjDBPQ=HFbbA2f+8iOT=vuQy9x-JyKw_tdP1QSeM=sFw@mail.gmail.com>
	<CANNq6Fk1hmyZ+xpJjBhNJSv1ZnC_TG+c2rLDBg+1ciQ4rLHhQA@mail.gmail.com>
	<CABDkGQmCTQVToKaWReZ_UHagot6QVVkUZyVMPa5iCPGzhJ1ykw@mail.gmail.com>
	<CAMMTP+Cty_8uWbYse4BXB0t2MRxhQumkgNTHsubfUw8cZ8PB1w@mail.gmail.com>
	<CAMMTP+BVJWr=8MV5chvBs=nsrqu8HtB9X2avQ084SkRkVKabdw@mail.gmail.com>
	<CAB6mnxLDGeieOwr+6jnUtbf3BHRMcWYZcSaFtCuxkRF1_foYAA@mail.gmail.com>
	<524D41A9.6040309@grinta.net>
	<CANNq6F=v4XzPTvgg88qEnuSm44qo50+2OW1tP7C9Px10FJswGQ@mail.gmail.com>
	<CAKF=Djss+A+=kVkc_xW=CToSCFFUXfeb1+OHPh_r8s_hVJNzow@mail.gmail.com>
	<CAB6mnxLoRDZvHnFpVFK5e6VeqfzqxTQX2wLe+SKZ5SxzqXgUtg@mail.gmail.com>
Message-ID: <CAPJVwBmzfLF7cgvFnmzR8FWx5HzOfZpPX-pijkqSGhNrEt+UXQ@mail.gmail.com>

On Thu, Oct 3, 2013 at 7:59 PM, Charles R Harris
<charlesr.harris at gmail.com> wrote:
>
> <snip>
>
>>
>> Please, no. It's another thing to remember and another way to shoot
>> yourself in the foot and introduce casual bugs.
>>
>> FWIW, my vote is to raise an error or return a nan, which will likely
>> eventually raise an error. If I have all nans, it's usually the case
>> that something's off, and I'd like to know sooner rather than later.
>>
>
> Here is what I have currently implemented. First, define an AllNanError
>
> class AllNanError(ValueError):
>     def __init__(self, msg, result):
>         ValueError.__init__(self, msg)
>         self.result = result
>
> For nanmax/nanmin/nanargmax/nanargmin this error is raised for all-nan axis
> and the result is attached. The exception can then be caught and the result
> examined. A ValueError is what amax, amin return for empty arrays.
>
> For nanmax/nanmin the result for an empty slice is nan. For
> argnanmax/argnanmin the result of an empty slice is -1, which is easier to
> read and remember than intp.min. A ValueError is what argmin, argmax
> currently return for empty arrays. Note that both of these functions can
> give wrong results if they contain some min/max values respectively. That is
> an old bug and I haven't fixed it.
>
> The nanmean/nanvar/nanstd functions currently raise a warning for all-nan
> slices and the result for such is nan. These could also be made to raise an
> error.
>
> Thoughts?

Is this intended for 1.8 or master?

-n


From charlesr.harris at gmail.com  Thu Oct  3 15:40:20 2013
From: charlesr.harris at gmail.com (Charles R Harris)
Date: Thu, 3 Oct 2013 13:40:20 -0600
Subject: [Numpy-discussion] Behavior of nan{max, min} and nanarg{max,
 min} for all-nan slices.
In-Reply-To: <CAPJVwBmzfLF7cgvFnmzR8FWx5HzOfZpPX-pijkqSGhNrEt+UXQ@mail.gmail.com>
References: <CAB6mnxLO+c0CZHSUbL3NZ-quivy5NdSUtaM_iAL1ay2pUwXr0w@mail.gmail.com>
	<CABDkGQmWHrYdoEvJigFOnL4SAkOY4JeChW7woR3cZsr2ArwGMA@mail.gmail.com>
	<CAMMTP+AvPxqE3FwQrvnz74GQeso5M4KipUuAbaLcgy2OvxLQww@mail.gmail.com>
	<CAB6mnxKjDBPQ=HFbbA2f+8iOT=vuQy9x-JyKw_tdP1QSeM=sFw@mail.gmail.com>
	<CANNq6Fk1hmyZ+xpJjBhNJSv1ZnC_TG+c2rLDBg+1ciQ4rLHhQA@mail.gmail.com>
	<CABDkGQmCTQVToKaWReZ_UHagot6QVVkUZyVMPa5iCPGzhJ1ykw@mail.gmail.com>
	<CAMMTP+Cty_8uWbYse4BXB0t2MRxhQumkgNTHsubfUw8cZ8PB1w@mail.gmail.com>
	<CAMMTP+BVJWr=8MV5chvBs=nsrqu8HtB9X2avQ084SkRkVKabdw@mail.gmail.com>
	<CAB6mnxLDGeieOwr+6jnUtbf3BHRMcWYZcSaFtCuxkRF1_foYAA@mail.gmail.com>
	<524D41A9.6040309@grinta.net>
	<CANNq6F=v4XzPTvgg88qEnuSm44qo50+2OW1tP7C9Px10FJswGQ@mail.gmail.com>
	<CAKF=Djss+A+=kVkc_xW=CToSCFFUXfeb1+OHPh_r8s_hVJNzow@mail.gmail.com>
	<CAB6mnxLoRDZvHnFpVFK5e6VeqfzqxTQX2wLe+SKZ5SxzqXgUtg@mail.gmail.com>
	<CAPJVwBmzfLF7cgvFnmzR8FWx5HzOfZpPX-pijkqSGhNrEt+UXQ@mail.gmail.com>
Message-ID: <CAB6mnxKYpfmJNn+jMXs-E=cPZP+si55ECsStvc8GraX-R5o3Rg@mail.gmail.com>

On Thu, Oct 3, 2013 at 1:11 PM, Nathaniel Smith <njs at pobox.com> wrote:

> On Thu, Oct 3, 2013 at 7:59 PM, Charles R Harris
> <charlesr.harris at gmail.com> wrote:
> >
> > <snip>
> >
> >>
> >> Please, no. It's another thing to remember and another way to shoot
> >> yourself in the foot and introduce casual bugs.
> >>
> >> FWIW, my vote is to raise an error or return a nan, which will likely
> >> eventually raise an error. If I have all nans, it's usually the case
> >> that something's off, and I'd like to know sooner rather than later.
> >>
> >
> > Here is what I have currently implemented. First, define an AllNanError
> >
> > class AllNanError(ValueError):
> >     def __init__(self, msg, result):
> >         ValueError.__init__(self, msg)
> >         self.result = result
> >
> > For nanmax/nanmin/nanargmax/nanargmin this error is raised for all-nan
> axis
> > and the result is attached. The exception can then be caught and the
> result
> > examined. A ValueError is what amax, amin return for empty arrays.
> >
> > For nanmax/nanmin the result for an empty slice is nan. For
> > argnanmax/argnanmin the result of an empty slice is -1, which is easier
> to
> > read and remember than intp.min. A ValueError is what argmin, argmax
> > currently return for empty arrays. Note that both of these functions can
> > give wrong results if they contain some min/max values respectively.
> That is
> > an old bug and I haven't fixed it.
> >
> > The nanmean/nanvar/nanstd functions currently raise a warning for all-nan
> > slices and the result for such is nan. These could also be made to raise
> an
> > error.
> >
> > Thoughts?
>
> Is this intended for 1.8 or master?
>

I was thinking both. The nanarg* functions are changing behavior anyway, so
might as well get it all done in 1.8. I also think there will need to be an
rc2 in anycase.

the current non-nan aware mean/var/std only raise warnings on insufficient
degrees of freedom and return nan. That's a bit of a change, they used to
return nans, negative numbers, and other things in that situation.

Chuck
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20131003/07900fc5/attachment.html>

From daniele at grinta.net  Thu Oct  3 15:44:34 2013
From: daniele at grinta.net (Daniele Nicolodi)
Date: Thu, 03 Oct 2013 21:44:34 +0200
Subject: [Numpy-discussion] Behavior of nan{max, min} and nanarg{max,
 min} for all-nan slices.
In-Reply-To: <CAB6mnxLoRDZvHnFpVFK5e6VeqfzqxTQX2wLe+SKZ5SxzqXgUtg@mail.gmail.com>
References: <CAB6mnxLO+c0CZHSUbL3NZ-quivy5NdSUtaM_iAL1ay2pUwXr0w@mail.gmail.com>
	<CABDkGQmWHrYdoEvJigFOnL4SAkOY4JeChW7woR3cZsr2ArwGMA@mail.gmail.com>
	<CAMMTP+AvPxqE3FwQrvnz74GQeso5M4KipUuAbaLcgy2OvxLQww@mail.gmail.com>
	<CAB6mnxKjDBPQ=HFbbA2f+8iOT=vuQy9x-JyKw_tdP1QSeM=sFw@mail.gmail.com>
	<CANNq6Fk1hmyZ+xpJjBhNJSv1ZnC_TG+c2rLDBg+1ciQ4rLHhQA@mail.gmail.com>
	<CABDkGQmCTQVToKaWReZ_UHagot6QVVkUZyVMPa5iCPGzhJ1ykw@mail.gmail.com>
	<CAMMTP+Cty_8uWbYse4BXB0t2MRxhQumkgNTHsubfUw8cZ8PB1w@mail.gmail.com>
	<CAMMTP+BVJWr=8MV5chvBs=nsrqu8HtB9X2avQ084SkRkVKabdw@mail.gmail.com>
	<CAB6mnxLDGeieOwr+6jnUtbf3BHRMcWYZcSaFtCuxkRF1_foYAA@mail.gmail.com>
	<524D41A9.6040309@grinta.net>
	<CANNq6F=v4XzPTvgg88qEnuSm44qo50+2OW1tP7C9Px10FJswGQ@mail.gmail.com>
	<CAKF=Djss+A+=kVkc_xW=CToSCFFUXfeb1+OHPh_r8s_hVJNzow@mail.gmail.com>
	<CAB6mnxLoRDZvHnFpVFK5e6VeqfzqxTQX2wLe+SKZ5SxzqXgUtg@mail.gmail.com>
Message-ID: <524DC922.90707@grinta.net>

On 03/10/2013 20:59, Charles R Harris wrote:
> Here is what I have currently implemented. First, define an AllNanError
> 
> class AllNanError(ValueError):
>     def __init__(self, msg, result):
>         ValueError.__init__(self, msg)
>         self.result = result
>  
> For nanmax/nanmin/nanargmax/nanargmin this error is raised for all-nan axis
> and the result is attached. The exception can then be caught and the
> result examined. A ValueError is what amax, amin return for empty arrays.
> 
> For nanmax/nanmin the result for an empty slice is nan. For
> argnanmax/argnanmin the result of an empty slice is -1, which is easier
> to read and remember than intp.min.

I think an error in this cases would be less confusing.

> A ValueError is what argmin, argmax
> currently return for empty arrays. Note that both of these functions can
> give wrong results if they contain some min/max values respectively.
> That is an old bug and I haven't fixed it.
> 
> The nanmean/nanvar/nanstd functions currently raise a warning for
> all-nan slices and the result for such is nan. These could also be made
> to raise an error.

I think an error in that case would be more consistent.

Cheers,
Daniele



From njs at pobox.com  Thu Oct  3 16:38:25 2013
From: njs at pobox.com (Nathaniel Smith)
Date: Thu, 3 Oct 2013 21:38:25 +0100
Subject: [Numpy-discussion] Behavior of nan{max, min} and nanarg{max,
 min} for all-nan slices.
In-Reply-To: <CAB6mnxKYpfmJNn+jMXs-E=cPZP+si55ECsStvc8GraX-R5o3Rg@mail.gmail.com>
References: <CAB6mnxLO+c0CZHSUbL3NZ-quivy5NdSUtaM_iAL1ay2pUwXr0w@mail.gmail.com>
	<CABDkGQmWHrYdoEvJigFOnL4SAkOY4JeChW7woR3cZsr2ArwGMA@mail.gmail.com>
	<CAMMTP+AvPxqE3FwQrvnz74GQeso5M4KipUuAbaLcgy2OvxLQww@mail.gmail.com>
	<CAB6mnxKjDBPQ=HFbbA2f+8iOT=vuQy9x-JyKw_tdP1QSeM=sFw@mail.gmail.com>
	<CANNq6Fk1hmyZ+xpJjBhNJSv1ZnC_TG+c2rLDBg+1ciQ4rLHhQA@mail.gmail.com>
	<CABDkGQmCTQVToKaWReZ_UHagot6QVVkUZyVMPa5iCPGzhJ1ykw@mail.gmail.com>
	<CAMMTP+Cty_8uWbYse4BXB0t2MRxhQumkgNTHsubfUw8cZ8PB1w@mail.gmail.com>
	<CAMMTP+BVJWr=8MV5chvBs=nsrqu8HtB9X2avQ084SkRkVKabdw@mail.gmail.com>
	<CAB6mnxLDGeieOwr+6jnUtbf3BHRMcWYZcSaFtCuxkRF1_foYAA@mail.gmail.com>
	<524D41A9.6040309@grinta.net>
	<CANNq6F=v4XzPTvgg88qEnuSm44qo50+2OW1tP7C9Px10FJswGQ@mail.gmail.com>
	<CAKF=Djss+A+=kVkc_xW=CToSCFFUXfeb1+OHPh_r8s_hVJNzow@mail.gmail.com>
	<CAB6mnxLoRDZvHnFpVFK5e6VeqfzqxTQX2wLe+SKZ5SxzqXgUtg@mail.gmail.com>
	<CAPJVwBmzfLF7cgvFnmzR8FWx5HzOfZpPX-pijkqSGhNrEt+UXQ@mail.gmail.com>
	<CAB6mnxKYpfmJNn+jMXs-E=cPZP+si55ECsStvc8GraX-R5o3Rg@mail.gmail.com>
Message-ID: <CAPJVwBkLO7yJCyad5u3HaTznye3_zDKcqR4wJ9Kb9Z=dnP3GAw@mail.gmail.com>

On Thu, Oct 3, 2013 at 8:40 PM, Charles R Harris
<charlesr.harris at gmail.com> wrote:
>
>
>
> On Thu, Oct 3, 2013 at 1:11 PM, Nathaniel Smith <njs at pobox.com> wrote:
>>
>> On Thu, Oct 3, 2013 at 7:59 PM, Charles R Harris
>> <charlesr.harris at gmail.com> wrote:
>> >
>> > <snip>
>> >
>> >>
>> >> Please, no. It's another thing to remember and another way to shoot
>> >> yourself in the foot and introduce casual bugs.
>> >>
>> >> FWIW, my vote is to raise an error or return a nan, which will likely
>> >> eventually raise an error. If I have all nans, it's usually the case
>> >> that something's off, and I'd like to know sooner rather than later.
>> >>
>> >
>> > Here is what I have currently implemented. First, define an AllNanError
>> >
>> > class AllNanError(ValueError):
>> >     def __init__(self, msg, result):
>> >         ValueError.__init__(self, msg)
>> >         self.result = result
>> >
>> > For nanmax/nanmin/nanargmax/nanargmin this error is raised for all-nan
>> > axis
>> > and the result is attached. The exception can then be caught and the
>> > result
>> > examined. A ValueError is what amax, amin return for empty arrays.
>> >
>> > For nanmax/nanmin the result for an empty slice is nan. For
>> > argnanmax/argnanmin the result of an empty slice is -1, which is easier
>> > to
>> > read and remember than intp.min. A ValueError is what argmin, argmax
>> > currently return for empty arrays. Note that both of these functions can
>> > give wrong results if they contain some min/max values respectively.
>> > That is
>> > an old bug and I haven't fixed it.
>> >
>> > The nanmean/nanvar/nanstd functions currently raise a warning for
>> > all-nan
>> > slices and the result for such is nan. These could also be made to raise
>> > an
>> > error.
>> >
>> > Thoughts?
>>
>> Is this intended for 1.8 or master?
>
> I was thinking both. The nanarg* functions are changing behavior anyway, so
> might as well get it all done in 1.8. I also think there will need to be an
> rc2 in anycase.

This is obviously a complicated and contentious enough issue that I
think for 1.8 we need to punt rather than try to force something out
under time pressure. We obviously need an rc2 anyway with all the
other stuff that's come in, but for 1.8 I'm going to suggest again we
go with:
- leave nanmax/nanmin as is
- make nanargmax/nanargmin just raise a simple ValueError on all-nans
(so that same as 1.7 for operations on multiple subarrays; the only
difference would be that for 1d operations 1.8 would raise where 1.7
returned nan).
I'm not saying we should stick with this forever, but it solves the
immediately problem that started this whole mess, and I can't see how
anyone could object to it as a temporary solution -- it's basically
the same as what 1.7 does. And it clears the deck for whatever
cleverer thing we come up with in master.

Does that make sense? I'm not going to out-right say -1 on doing
anything else for 1.8, but the longer this drags on the more tempting
it becomes, just because who can really evaluate a more interesting
proposal with the release breathing down their neck?

> the current non-nan aware mean/var/std only raise warnings on insufficient
> degrees of freedom and return nan. That's a bit of a change, they used to
> return nans, negative numbers, and other things in that situation.

Seems reasonable to me.

-n


From charlesr.harris at gmail.com  Thu Oct  3 17:07:16 2013
From: charlesr.harris at gmail.com (Charles R Harris)
Date: Thu, 3 Oct 2013 15:07:16 -0600
Subject: [Numpy-discussion] Behavior of nan{max, min} and nanarg{max,
 min} for all-nan slices.
In-Reply-To: <CAPJVwBkLO7yJCyad5u3HaTznye3_zDKcqR4wJ9Kb9Z=dnP3GAw@mail.gmail.com>
References: <CAB6mnxLO+c0CZHSUbL3NZ-quivy5NdSUtaM_iAL1ay2pUwXr0w@mail.gmail.com>
	<CABDkGQmWHrYdoEvJigFOnL4SAkOY4JeChW7woR3cZsr2ArwGMA@mail.gmail.com>
	<CAMMTP+AvPxqE3FwQrvnz74GQeso5M4KipUuAbaLcgy2OvxLQww@mail.gmail.com>
	<CAB6mnxKjDBPQ=HFbbA2f+8iOT=vuQy9x-JyKw_tdP1QSeM=sFw@mail.gmail.com>
	<CANNq6Fk1hmyZ+xpJjBhNJSv1ZnC_TG+c2rLDBg+1ciQ4rLHhQA@mail.gmail.com>
	<CABDkGQmCTQVToKaWReZ_UHagot6QVVkUZyVMPa5iCPGzhJ1ykw@mail.gmail.com>
	<CAMMTP+Cty_8uWbYse4BXB0t2MRxhQumkgNTHsubfUw8cZ8PB1w@mail.gmail.com>
	<CAMMTP+BVJWr=8MV5chvBs=nsrqu8HtB9X2avQ084SkRkVKabdw@mail.gmail.com>
	<CAB6mnxLDGeieOwr+6jnUtbf3BHRMcWYZcSaFtCuxkRF1_foYAA@mail.gmail.com>
	<524D41A9.6040309@grinta.net>
	<CANNq6F=v4XzPTvgg88qEnuSm44qo50+2OW1tP7C9Px10FJswGQ@mail.gmail.com>
	<CAKF=Djss+A+=kVkc_xW=CToSCFFUXfeb1+OHPh_r8s_hVJNzow@mail.gmail.com>
	<CAB6mnxLoRDZvHnFpVFK5e6VeqfzqxTQX2wLe+SKZ5SxzqXgUtg@mail.gmail.com>
	<CAPJVwBmzfLF7cgvFnmzR8FWx5HzOfZpPX-pijkqSGhNrEt+UXQ@mail.gmail.com>
	<CAB6mnxKYpfmJNn+jMXs-E=cPZP+si55ECsStvc8GraX-R5o3Rg@mail.gmail.com>
	<CAPJVwBkLO7yJCyad5u3HaTznye3_zDKcqR4wJ9Kb9Z=dnP3GAw@mail.gmail.com>
Message-ID: <CAB6mnx+Q3mGZTfaFtYuBGcMOW467itMxyd5DhQ2STO3FS538mA@mail.gmail.com>

On Thu, Oct 3, 2013 at 2:38 PM, Nathaniel Smith <njs at pobox.com> wrote:

> On Thu, Oct 3, 2013 at 8:40 PM, Charles R Harris
> <charlesr.harris at gmail.com> wrote:
> >
> >
> >
> > On Thu, Oct 3, 2013 at 1:11 PM, Nathaniel Smith <njs at pobox.com> wrote:
> >>
> >> On Thu, Oct 3, 2013 at 7:59 PM, Charles R Harris
> >> <charlesr.harris at gmail.com> wrote:
> >> >
> >> > <snip>
> >> >
> >> >>
> >> >> Please, no. It's another thing to remember and another way to shoot
> >> >> yourself in the foot and introduce casual bugs.
> >> >>
> >> >> FWIW, my vote is to raise an error or return a nan, which will likely
> >> >> eventually raise an error. If I have all nans, it's usually the case
> >> >> that something's off, and I'd like to know sooner rather than later.
> >> >>
> >> >
> >> > Here is what I have currently implemented. First, define an
> AllNanError
> >> >
> >> > class AllNanError(ValueError):
> >> >     def __init__(self, msg, result):
> >> >         ValueError.__init__(self, msg)
> >> >         self.result = result
> >> >
> >> > For nanmax/nanmin/nanargmax/nanargmin this error is raised for all-nan
> >> > axis
> >> > and the result is attached. The exception can then be caught and the
> >> > result
> >> > examined. A ValueError is what amax, amin return for empty arrays.
> >> >
> >> > For nanmax/nanmin the result for an empty slice is nan. For
> >> > argnanmax/argnanmin the result of an empty slice is -1, which is
> easier
> >> > to
> >> > read and remember than intp.min. A ValueError is what argmin, argmax
> >> > currently return for empty arrays. Note that both of these functions
> can
> >> > give wrong results if they contain some min/max values respectively.
> >> > That is
> >> > an old bug and I haven't fixed it.
> >> >
> >> > The nanmean/nanvar/nanstd functions currently raise a warning for
> >> > all-nan
> >> > slices and the result for such is nan. These could also be made to
> raise
> >> > an
> >> > error.
> >> >
> >> > Thoughts?
> >>
> >> Is this intended for 1.8 or master?
> >
> > I was thinking both. The nanarg* functions are changing behavior anyway,
> so
> > might as well get it all done in 1.8. I also think there will need to be
> an
> > rc2 in anycase.
>
> This is obviously a complicated and contentious enough issue that I
> think for 1.8 we need to punt rather than try to force something out
> under time pressure. We obviously need an rc2 anyway with all the
> other stuff that's come in, but for 1.8 I'm going to suggest again we
> go with:
> - leave nanmax/nanmin as is
> - make nanargmax/nanargmin just raise a simple ValueError on all-nans
> (so that same as 1.7 for operations on multiple subarrays; the only
> difference would be that for 1d operations 1.8 would raise where 1.7
> returned nan).
>

OK, sounds good to me. I'll leave the warnings in.

<snip>

Chuck
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20131003/f526eba2/attachment.html>

From Catherine.M.Moroney at jpl.nasa.gov  Thu Oct  3 19:05:53 2013
From: Catherine.M.Moroney at jpl.nasa.gov (Moroney, Catherine M (398D))
Date: Thu, 3 Oct 2013 23:05:53 +0000
Subject: [Numpy-discussion] subsampling arrays without loops
Message-ID: <2FF3F79D-A7E2-42C1-AEEB-B5E4A4B37378@jpl.nasa.gov>

I know I have a lot yet to learn about array striding tricks, so please
pardon the triviality of this question.

Here is the problem both in words and "dumb" python:

I have a large NxM array that I want to break down into smaller nxn chunks
where n divides evenly into both N and M.  Then I want to calculate the
fraction of pixels in each nxn chunk that meets a certain criteria: say (x > 1) & (x < 2).

Here is the "dumb" python code:

npix = 4
num_true = numpy.zeros((data.shape[0]/npix, data.shape[1]/npix))

for iline in xrange(0, data.shape[0]/npix):
   for ismp in xrange(0, data.shape[1]/npix):
       excerpt = data[iline*npix:(iline+1)*npix, ismp*npix:(ismp+1)*npix] 
       num_true[iline,simp] = numpy.where( (excerpt >= 1) & (excerpt <= 2), True, False).sum()

so I'm looping through the number of 4x4 subsets in both dimensions, cutting out a chunk
of the data ("excerpt"), and then counting the number of pixels in that excerpt that
meet a certain criteria and storing that result for each excerpt.

I want to avoid all the loops over iline and ismp.  What's the best way of doing this
in pure python?  I could always write a Fortran/C routine for this task, but I want to
learn how best to do it with numpy.  

Thank you for any advice,

Catherine



From jaime.frio at gmail.com  Thu Oct  3 19:17:31 2013
From: jaime.frio at gmail.com (=?ISO-8859-1?Q?Jaime_Fern=E1ndez_del_R=EDo?=)
Date: Thu, 3 Oct 2013 16:17:31 -0700
Subject: [Numpy-discussion] subsampling arrays without loops
In-Reply-To: <2FF3F79D-A7E2-42C1-AEEB-B5E4A4B37378@jpl.nasa.gov>
References: <2FF3F79D-A7E2-42C1-AEEB-B5E4A4B37378@jpl.nasa.gov>
Message-ID: <CAPOWHW=S4rSwJw0xMSpxaSpZfOqk5Tf3-Pqd0aGwCtq0qOPMyA@mail.gmail.com>

On Thu, Oct 3, 2013 at 4:05 PM, Moroney, Catherine M (398D) <
Catherine.M.Moroney at jpl.nasa.gov> wrote:

> I know I have a lot yet to learn about array striding tricks, so please
> pardon the triviality of this question.
>
> Here is the problem both in words and "dumb" python:
>
> I have a large NxM array that I want to break down into smaller nxn chunks
> where n divides evenly into both N and M.  Then I want to calculate the
> fraction of pixels in each nxn chunk that meets a certain criteria: say (x
> > 1) & (x < 2).
>

If n divides both N and M, you can simply reshape your array to 4D, then
reduce it back to 2D:

import numpy as np

N, M = 1000, 2000
n = 100

a = np.random.rand(N,M) * 5
a_view = a.reshape(a.shape[0] // n, n, a.shape[1] // n, n)
a_fractions = np.sum((a_view > 1) & (a_view < 2), axis=(1, 3)) / (n * n)

>>> a_fractions.shape
(10L, 20L)

>>> a_fractions
array([[ 0.1965,  0.1964,  0.202 ,  0.1997,  0.1976,  0.1997,  0.2026,
         0.1951,  0.2051,  0.1995,  0.1926,  0.2006,  0.1973,  0.1964,
         0.2046,  0.1977,  0.2066,  0.2009,  0.2003,  0.2013],
       ...
       [ 0.2028,  0.1943,  0.2036,  0.1992,  0.2   ,  0.2009,  0.1971,
         0.1996,  0.196 ,  0.196 ,  0.1983,  0.2021,  0.2031,  0.1955,
         0.1916,  0.1939,  0.202 ,  0.2064,  0.2021,  0.1954]])

Jaime

 --
(\__/)
( O.o)
( > <) Este es Conejo. Copia a Conejo en tu firma y ay?dale en sus planes
de dominaci?n mundial.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20131003/48ba06ec/attachment.html>

From tmp50 at ukr.net  Fri Oct  4 15:26:38 2013
From: tmp50 at ukr.net (Dmitrey)
Date: Fri, 04 Oct 2013 22:26:38 +0300
Subject: [Numpy-discussion] [ANN] MATLAB ODE solvers - now available in
	Python
Message-ID: <1380914598.651925561.b7d4jpuh@frv43.ukr.net>

Several MATLAB ODE dy/dt = f(y,t) solvers (ode15s, ode23, ode113, ode23t, ode23tb, ode45, ode23s)? have been connected to free OpenOpt Suite package (possibly with? FuncDesigner ? automatic differentiation) in addition to scipy_lsoda (scipy.integrate.odeint), see the example . 
Currently only reltol parameter is available; future plans may include abstol, Python3 and PyPy support, solver ode15i for solving f(dy/dt, y, t) = 0, possibility to use single MATLAB session for several ODE probs. Sparse matrices handling is implemented for fmincon and fsolve but not ode solvers yet. 
---------------------- 
Regards, D. http://openopt.org/Dmitrey 
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20131004/6bafe132/attachment.html>

From charlesr.harris at gmail.com  Fri Oct  4 16:37:46 2013
From: charlesr.harris at gmail.com (Charles R Harris)
Date: Fri, 4 Oct 2013 14:37:46 -0600
Subject: [Numpy-discussion] Behavior of nan{max, min} and nanarg{max,
 min} for all-nan slices.
In-Reply-To: <CAB6mnx+Q3mGZTfaFtYuBGcMOW467itMxyd5DhQ2STO3FS538mA@mail.gmail.com>
References: <CAB6mnxLO+c0CZHSUbL3NZ-quivy5NdSUtaM_iAL1ay2pUwXr0w@mail.gmail.com>
	<CABDkGQmWHrYdoEvJigFOnL4SAkOY4JeChW7woR3cZsr2ArwGMA@mail.gmail.com>
	<CAMMTP+AvPxqE3FwQrvnz74GQeso5M4KipUuAbaLcgy2OvxLQww@mail.gmail.com>
	<CAB6mnxKjDBPQ=HFbbA2f+8iOT=vuQy9x-JyKw_tdP1QSeM=sFw@mail.gmail.com>
	<CANNq6Fk1hmyZ+xpJjBhNJSv1ZnC_TG+c2rLDBg+1ciQ4rLHhQA@mail.gmail.com>
	<CABDkGQmCTQVToKaWReZ_UHagot6QVVkUZyVMPa5iCPGzhJ1ykw@mail.gmail.com>
	<CAMMTP+Cty_8uWbYse4BXB0t2MRxhQumkgNTHsubfUw8cZ8PB1w@mail.gmail.com>
	<CAMMTP+BVJWr=8MV5chvBs=nsrqu8HtB9X2avQ084SkRkVKabdw@mail.gmail.com>
	<CAB6mnxLDGeieOwr+6jnUtbf3BHRMcWYZcSaFtCuxkRF1_foYAA@mail.gmail.com>
	<524D41A9.6040309@grinta.net>
	<CANNq6F=v4XzPTvgg88qEnuSm44qo50+2OW1tP7C9Px10FJswGQ@mail.gmail.com>
	<CAKF=Djss+A+=kVkc_xW=CToSCFFUXfeb1+OHPh_r8s_hVJNzow@mail.gmail.com>
	<CAB6mnxLoRDZvHnFpVFK5e6VeqfzqxTQX2wLe+SKZ5SxzqXgUtg@mail.gmail.com>
	<CAPJVwBmzfLF7cgvFnmzR8FWx5HzOfZpPX-pijkqSGhNrEt+UXQ@mail.gmail.com>
	<CAB6mnxKYpfmJNn+jMXs-E=cPZP+si55ECsStvc8GraX-R5o3Rg@mail.gmail.com>
	<CAPJVwBkLO7yJCyad5u3HaTznye3_zDKcqR4wJ9Kb9Z=dnP3GAw@mail.gmail.com>
	<CAB6mnx+Q3mGZTfaFtYuBGcMOW467itMxyd5DhQ2STO3FS538mA@mail.gmail.com>
Message-ID: <CAB6mnx+w+HmDyd3qFzKpW77AJt-veQbJwap2ftNrw0iy59Wk1A@mail.gmail.com>

I've made a PR for this. The commit message is

    Deal with subclasses of ndarray, like pandas.Series and matrix.

        Subclasses may not define the new keyword keepdims or deal
        gracefully with ufuncs in all their forms. This is solved by
        throwing the problem onto the np.sum, np.any, etc. functions
        that have ugly hacks to deal with the problem.

    Settle handling of all-nan slices.

        nanmax, nanmin -- Raise warning, return NaN for slice.
        nanargmax, nanargmin -- Raise ValueError.
        nansum -- Return 0 for slice.
        nanmean, nanvar, nanstd -- Raise warning, return NaN for slice.

    Make NaN functions work for scalar arguments.

        This may seem silly, but it removes a check for special cases.

    Update tests

        Deal with new all-nan handling.
        Test with matrix class as example of subclass without keepdims.
        Test with scalar arguments.

Note that 1.7 already raises a ValueError for nanarg{max, min} on account
of casting rules that don't allow nans to be cast to ints. This just makes
the same hold true for scalar returns.

If there are no objections, this will go into 1.9 as a starting point and
be backported to 1.8.

Chuck
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20131004/5f2a9ef5/attachment.html>

From ecarlson at eng.ua.edu  Fri Oct  4 18:19:25 2013
From: ecarlson at eng.ua.edu (Eric Carlson)
Date: Fri, 04 Oct 2013 17:19:25 -0500
Subject: [Numpy-discussion] [ANN] MATLAB ODE solvers - now available in
	Python
In-Reply-To: <1380914598.651925561.b7d4jpuh@frv43.ukr.net>
References: <1380914598.651925561.b7d4jpuh@frv43.ukr.net>
Message-ID: <l2nesl$8ab$1@ger.gmane.org>

Hello,
Does this require a MATLAB install, or are these equivalent routines?

Thanks,
Eric



From tmp50 at ukr.net  Sat Oct  5 04:27:04 2013
From: tmp50 at ukr.net (Dmitrey)
Date: Sat, 05 Oct 2013 11:27:04 +0300
Subject: [Numpy-discussion] [ANN] MATLAB ODE solvers - now available
 in	Python
In-Reply-To: <l2nesl$8ab$1@ger.gmane.org>
References: <1380914598.651925561.b7d4jpuh@frv43.ukr.net>
	<l2nesl$8ab$1@ger.gmane.org>
Message-ID: <1380961403.991447284.5qt9wlwp@frv43.ukr.net>

It requires MATLAB or MATLAB Component Runtime? ( http://www.mathworks.com/products/compiler/mcr/ ) 
I'm not regular subscriber of the mail list thus you'd better ask openopt forum. 

---------------------- 
Regards, D. http://openopt.org/Dmitrey 
--- ???????? ????????? --- 
?? ????: "Eric Carlson" < ecarlson at eng.ua.edu > 
????: 5 ??????? 2013, 01:19:28 

Hello,
Does this require a MATLAB install, or are these equivalent routines?

Thanks,
Eric

_______________________________________________
NumPy-Discussion mailing list
 
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20131005/efa7fba5/attachment.html>

From d.l.goldsmith at gmail.com  Sat Oct  5 13:15:29 2013
From: d.l.goldsmith at gmail.com (David Goldsmith)
Date: Sat, 5 Oct 2013 10:15:29 -0700
Subject: [Numpy-discussion] [ANN] MATLAB ODE solvers - now available in
	Python
Message-ID: <CAFtPsZpmozy1FwyZz3aEW0FY=MRWWxHeDJ9b_+sjZzyu8+D15w@mail.gmail.com>

MCR stands for MATLAB Compiler Runtime and if that's all it requires,
that's great, 'cause that's free.  Look forward to giving this a try; does
the distribution come w/ examples?

DG

Date: Sat, 05 Oct 2013 11:27:04 +0300

> From: Dmitrey <tmp50 at ukr.net>
> Subject: Re: [Numpy-discussion] [ANN] MATLAB ODE solvers - now
>         available in    Python
> To: Discussion of Numerical Python <numpy-discussion at scipy.org>
> Cc: scipy-user at scipy.org, numpy-discussion at scipy.org
> Message-ID: <1380961403.991447284.5qt9wlwp at frv43.ukr.net>
> Content-Type: text/plain; charset="utf-8"
>
> It requires MATLAB or MATLAB Component Runtime? (
> http://www.mathworks.com/products/compiler/mcr/ )
> I'm not regular subscriber of the mail list thus you'd better ask openopt
> forum.
>
> ----------------------
> Regards, D. http://openopt.org/Dmitrey
> --- ???????? ????????? ---
> ?? ????: "Eric Carlson" < ecarlson at eng.ua.edu >
> ????: 5 ??????? 2013, 01:19:28
>
> Hello,
> Does this require a MATLAB install, or are these equivalent routines?
>
> Thanks,
> Eric
>
> _______________________________________________
> NumPy-Discussion mailing list
>
> -------------- next part --------------
> An HTML attachment was scrubbed...
> URL:
> http://mail.scipy.org/pipermail/numpy-discussion/attachments/20131005/efa7fba5/attachment-0001.html
>
> ------------------------------
>
> _______________________________________________
> NumPy-Discussion mailing list
> NumPy-Discussion at scipy.org
> http://mail.scipy.org/mailman/listinfo/numpy-discussion
>
>
> End of NumPy-Discussion Digest, Vol 85, Issue 16
> ************************************************
>



-- 
>From "A Letter From The Future" in "Peak Everything" by Richard Heinberg:

"By the time I was an older teenager, a certain...attitude was developing
among the young people...a feeling of utter contempt for anyone over a
certain age--maybe 30 or 40.  The adults had consumed so many resources,
and now there were none left for their own children...when those adults
were younger, they [were] just doing what everybody else was doing...they
figured it was normal to cut down ancient forests for...phone books, pump
every last gallon of oil to power their SUV's...[but] for...my generation
all that was just a dim memory...We [grew up] living in darkness, with
shortages of food and water, with riots in the streets, with people begging
on street corners...for us, the adults were the enemy."

Want to *really* understand what's *really* going on?  Read "Peak
Everything."
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20131005/23db3edd/attachment.html>

From tmp50 at ukr.net  Sat Oct  5 14:36:48 2013
From: tmp50 at ukr.net (Dmitrey)
Date: Sat, 05 Oct 2013 21:36:48 +0300
Subject: [Numpy-discussion] [ANN] MATLAB ODE solvers - now available
 in	Python
In-Reply-To: <CAFtPsZpmozy1FwyZz3aEW0FY=MRWWxHeDJ9b_+sjZzyu8+D15w@mail.gmail.com>
References: <CAFtPsZpmozy1FwyZz3aEW0FY=MRWWxHeDJ9b_+sjZzyu8+D15w@mail.gmail.com>
Message-ID: <1380997576.559804301.aoynahlj@frv43.ukr.net>

? 

--- ???????? ????????? --- 
?? ????: "David Goldsmith" < d.l.goldsmith at gmail.com > 
????: 5 ??????? 2013, 20:15:38 

MCR stands for MATLAB Compiler Runtime and if that's all it requires, that's great, 'cause that's free.? Look forward to giving this a try; does the distribution come w/ examples? 


Seems like using the MATLAB solvers with MCR requires my wrappers containing in several files to be compiled with MATLAB Compiler before. I have no license for MATLAB thus I may have problems if I'll make it done and will spread it with OpenOpt suite code, also, binary files are incompatible with BSD license. 
On the other hand, IIRC a little bit obsolete MATLAB versions (I don't think difference is essential) have more liberal licenses. 
As for MATLAB solvers examples, I have already mentioned them in the mail list, you could see them in http://openopt.org/ODE (just replace solver name from scipy_lsoda to ode23s or any other), http://openopt.org/NLP , http://openopt.org/SNLE 

---------------------- 
Regards, D. http://openopt.org/Dmitrey 
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20131005/dd6638db/attachment.html>

From Bernhard.Spinnler at gmx.net  Sun Oct  6 13:30:53 2013
From: Bernhard.Spinnler at gmx.net (Bernhard Spinnler)
Date: Sun, 6 Oct 2013 19:30:53 +0200
Subject: [Numpy-discussion] Problem with numpy's array reference assignment?
Message-ID: <76136111-2257-4E29-AAD5-EDEC423F8D31@gmx.net>

I have problems to get a piece of code to work with a new numpy/scipy version. The code essentially sets up a matrix Ryy and a vector Rya and solves the system of linear equations Ryy*c = Rya for c. Then it checks whether the resulting vector c satisfies the equation: Ryy*c must be equal to Rya.

While this code runs fine on
- python-2.7.5.p1, numpy-1.7.0, scipy-0.12.0.p0
- python-2.7.3, numpy-1.7.1, scipy-0.12.0

it fails on
- python 2.7.2, numpy 1.9.0.dev-fde3dee, scipy 0.14.0.dev-4938da3
with error:

	AssertionError: 
	Arrays are not almost equal to 6 decimals

	(mismatch 100.0%)
	 x: array([ 9.+0.j,  8.+0.j,  7.+0.j])
	 y: array([ 7.+0.j,  6.+0.j,  5.+0.j])

The piece of code is:
------------------------------------
import numpy as np
from numpy.testing import assert_array_almost_equal

lag0 = 5
N = 3
D = 2
corr_ya = np.array([0,1,2,3,4,5,6,7,8,9+0j])
corr_yy = np.array([0,1,2,3,4,5,4,3,2,1])

Rya = corr_ya[lag0+D:lag0+D-N:-1]
Ryy = np.zeros((N, N), complex)
for i in np.arange(N):
    Ryy[i,:] = corr_yy[lag0-i:lag0-i+N]
    
c = np.linalg.solve(Ryy, Rya)

# check result
Ryy_x_c = np.dot(Ryy, c)
print "Ryy*c =", repr(Ryy_x_c)
print "Rya =", repr(Rya)
assert_array_almost_equal(Ryy_x_c, Rya)
------------------------------------

I guess that it has something to do with numpy assigning arrays by reference and not copying them, since the problem goes away when vector Rya is copied before passing it to solve, i.e. doing Rya_copy = copy(Rya) and passing Rya_copy to solve.

The problem also does not occur when the initial array corr_ya is an integer array, e.g. by deleting the "+0j" from the last element of corr_ya above. (Normally, my initial arrays are complex. I just used artificially simplified arrays above to show the problem.) I could imagine that this is due to numpy copying the integers into a new complex array instead of creating a reference that is passed to solve.

Could it be that a bug has been introduced in recent numpy/scipy version? Or am I misunderstanding something?

Thanks,
	Bernhard

From charlesr.harris at gmail.com  Sun Oct  6 13:53:20 2013
From: charlesr.harris at gmail.com (Charles R Harris)
Date: Sun, 6 Oct 2013 11:53:20 -0600
Subject: [Numpy-discussion] Problem with numpy's array reference
	assignment?
In-Reply-To: <76136111-2257-4E29-AAD5-EDEC423F8D31@gmx.net>
References: <76136111-2257-4E29-AAD5-EDEC423F8D31@gmx.net>
Message-ID: <CAB6mnx+Q7pToEjFNAj3VQq6Cx2wfnZoJDVby4-A=UFbgeJSimQ@mail.gmail.com>

On Sun, Oct 6, 2013 at 11:30 AM, Bernhard Spinnler <
Bernhard.Spinnler at gmx.net> wrote:

> I have problems to get a piece of code to work with a new numpy/scipy
> version. The code essentially sets up a matrix Ryy and a vector Rya and
> solves the system of linear equations Ryy*c = Rya for c. Then it checks
> whether the resulting vector c satisfies the equation: Ryy*c must be equal
> to Rya.
>
> While this code runs fine on
> - python-2.7.5.p1, numpy-1.7.0, scipy-0.12.0.p0
> - python-2.7.3, numpy-1.7.1, scipy-0.12.0
>
> it fails on
> - python 2.7.2, numpy 1.9.0.dev-fde3dee, scipy 0.14.0.dev-4938da3
> with error:
>
>         AssertionError:
>         Arrays are not almost equal to 6 decimals
>
>         (mismatch 100.0%)
>          x: array([ 9.+0.j,  8.+0.j,  7.+0.j])
>          y: array([ 7.+0.j,  6.+0.j,  5.+0.j])
>
> The piece of code is:
> ------------------------------------
> import numpy as np
> from numpy.testing import assert_array_almost_equal
>
> lag0 = 5
> N = 3
> D = 2
> corr_ya = np.array([0,1,2,3,4,5,6,7,8,9+0j])
> corr_yy = np.array([0,1,2,3,4,5,4,3,2,1])
>
> Rya = corr_ya[lag0+D:lag0+D-N:-1]
> Ryy = np.zeros((N, N), complex)
> for i in np.arange(N):
>     Ryy[i,:] = corr_yy[lag0-i:lag0-i+N]
>
> c = np.linalg.solve(Ryy, Rya)
>
> # check result
> Ryy_x_c = np.dot(Ryy, c)
> print "Ryy*c =", repr(Ryy_x_c)
> print "Rya =", repr(Rya)
> assert_array_almost_equal(Ryy_x_c, Rya)
> ------------------------------------
>
> I guess that it has something to do with numpy assigning arrays by
> reference and not copying them, since the problem goes away when vector Rya
> is copied before passing it to solve, i.e. doing Rya_copy = copy(Rya) and
> passing Rya_copy to solve.
>
> The problem also does not occur when the initial array corr_ya is an
> integer array, e.g. by deleting the "+0j" from the last element of corr_ya
> above. (Normally, my initial arrays are complex. I just used artificially
> simplified arrays above to show the problem.) I could imagine that this is
> due to numpy copying the integers into a new complex array instead of
> creating a reference that is passed to solve.
>
> Could it be that a bug has been introduced in recent numpy/scipy version?
> Or am I misunderstanding something?
>
>
It is certainly possible, there have been changes in the linalg routines.

Chuck
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20131006/cf12ee34/attachment.html>

From charlesr.harris at gmail.com  Sun Oct  6 14:09:25 2013
From: charlesr.harris at gmail.com (Charles R Harris)
Date: Sun, 6 Oct 2013 12:09:25 -0600
Subject: [Numpy-discussion] Problem with numpy's array reference
	assignment?
In-Reply-To: <CAB6mnx+Q7pToEjFNAj3VQq6Cx2wfnZoJDVby4-A=UFbgeJSimQ@mail.gmail.com>
References: <76136111-2257-4E29-AAD5-EDEC423F8D31@gmx.net>
	<CAB6mnx+Q7pToEjFNAj3VQq6Cx2wfnZoJDVby4-A=UFbgeJSimQ@mail.gmail.com>
Message-ID: <CAB6mnxLCZH-Q_RNGd6M8Vmdd5YwF4woY9kuQMK5y71+p4EtH0w@mail.gmail.com>

I've opened an issue <https://github.com/numpy/numpy/issues/3876>.

<snip>

Chuck
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20131006/7f637ac6/attachment.html>

From d.l.goldsmith at gmail.com  Mon Oct  7 00:16:23 2013
From: d.l.goldsmith at gmail.com (David Goldsmith)
Date: Sun, 6 Oct 2013 21:16:23 -0700
Subject: [Numpy-discussion] [ANN] MATLAB ODE solvers - now available in
	Python (Dmitrey)
Message-ID: <CAFtPsZouBMUWW072PNkPP17OfJtfswi8=9KuZE0dPPsnjptd0g@mail.gmail.com>

On Sun, Oct 6, 2013 at 10:00 AM, <numpy-discussion-request at scipy.org> wrote:

> Message: 2
> Date: Sat, 05 Oct 2013 21:36:48 +0300
> From: Dmitrey <tmp50 at ukr.net>
> Subject: Re: [Numpy-discussion] [ANN] MATLAB ODE solvers - now
>         available in    Python
> To: Discussion of Numerical Python <numpy-discussion at scipy.org>
> Cc: numpy-discussion at scipy.org
> Message-ID: <1380997576.559804301.aoynahlj at frv43.ukr.net>
> Content-Type: text/plain; charset="utf-8"
>


> Seems like using the MATLAB solvers with MCR requires my wrappers
> containing in several files to be compiled with MATLAB Compiler before. I
> have no license for MATLAB thus I may have problems if I'll make it done
> and will spread it with OpenOpt suite code, also, binary files are
> incompatible with BSD license.
>

Darn, knew it was too good to be true.


> On the other hand, IIRC a little bit obsolete MATLAB versions (I don't
> think difference is essential) have more liberal licenses.
> As for MATLAB solvers examples, I have already mentioned them in the mail
> list, you could see them in http://openopt.org/ODE (just replace solver
> name from scipy_lsoda to ode23s or any other), http://openopt.org/NLP ,
> http://openopt.org/SNLE


Oooops, so sorry. :-o

DG

>
> ----------------------
> Regards, D. http://openopt.org/Dmitrey
> -------------- next part --------------
> An HTML attachment was scrubbed...
> URL:
> http://mail.scipy.org/pipermail/numpy-discussion/attachments/20131005/dd6638db/attachment-0001.html
>
> ------------------------------
>
> _______________________________________________
> NumPy-Discussion mailing list
> NumPy-Discussion at scipy.org
> http://mail.scipy.org/mailman/listinfo/numpy-discussion
>
>
> End of NumPy-Discussion Digest, Vol 85, Issue 17
> ************************************************
>



-- 
>From "A Letter From The Future" in "Peak Everything" by Richard Heinberg:

"By the time I was an older teenager, a certain...attitude was developing
among the young people...a feeling of utter contempt for anyone over a
certain age--maybe 30 or 40.  The adults had consumed so many resources,
and now there were none left for their own children...when those adults
were younger, they [were] just doing what everybody else was doing...they
figured it was normal to cut down ancient forests for...phone books, pump
every last gallon of oil to power their SUV's...[but] for...my generation
all that was just a dim memory...We [grew up] living in darkness, with
shortages of food and water, with riots in the streets, with people begging
on street corners...for us, the adults were the enemy."

Want to *really* understand what's *really* going on?  Read "Peak
Everything."
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20131006/6fe7cf72/attachment.html>

From sgonzi at staffmail.ed.ac.uk  Mon Oct  7 13:15:03 2013
From: sgonzi at staffmail.ed.ac.uk (Siegfried Gonzi)
Date: Mon, 7 Oct 2013 18:15:03 +0100
Subject: [Numpy-discussion] Equivalent to IDL's help function
Message-ID: <765BF3F2-7FE1-414E-A134-F363D841D0DC@staffmail.ed.ac.uk>

Hi all

What is the equivalent to IDL its help function, e.g.

==
IDL> a = make_array(23,23,)

IDL> help,a 

will result in:

A               FLOAT     = Array[23, 23]

or

IDL> a = create_struct('idl',23)

IDL> help,a

gives:

A               STRUCT    = -> <Anonymous> Array[1]

==

I have been looking for it ever since using numpy. It would make my life so much easier.


Thanks, Siegfried


-- 
The University of Edinburgh is a charitable body, registered in
Scotland, with registration number SC005336.



From aron at ahmadia.net  Mon Oct  7 13:18:07 2013
From: aron at ahmadia.net (Aron Ahmadia)
Date: Mon, 7 Oct 2013 13:18:07 -0400
Subject: [Numpy-discussion] Equivalent to IDL's help function
In-Reply-To: <765BF3F2-7FE1-414E-A134-F363D841D0DC@staffmail.ed.ac.uk>
References: <765BF3F2-7FE1-414E-A134-F363D841D0DC@staffmail.ed.ac.uk>
Message-ID: <CAPhiW4irmrtgmU5+=yHL96NfKO-BRP6-dydWowsESQ8eAt30wQ@mail.gmail.com>

If you're using IPython (strongly recommended)

a?
a??

If you're just using Python

help(a)

A


On Mon, Oct 7, 2013 at 1:15 PM, Siegfried Gonzi
<sgonzi at staffmail.ed.ac.uk> wrote:
> Hi all
>
> What is the equivalent to IDL its help function, e.g.
>
> ==
> IDL> a = make_array(23,23,)
>
> IDL> help,a
>
> will result in:
>
> A               FLOAT     = Array[23, 23]
>
> or
>
> IDL> a = create_struct('idl',23)
>
> IDL> help,a
>
> gives:
>
> A               STRUCT    = -> <Anonymous> Array[1]
>
> ==
>
> I have been looking for it ever since using numpy. It would make my life so much easier.
>
>
> Thanks, Siegfried
>
>
> --
> The University of Edinburgh is a charitable body, registered in
> Scotland, with registration number SC005336.
>
> _______________________________________________
> NumPy-Discussion mailing list
> NumPy-Discussion at scipy.org
> http://mail.scipy.org/mailman/listinfo/numpy-discussion


From ben.root at ou.edu  Mon Oct  7 13:18:44 2013
From: ben.root at ou.edu (Benjamin Root)
Date: Mon, 7 Oct 2013 13:18:44 -0400
Subject: [Numpy-discussion] Equivalent to IDL's help function
In-Reply-To: <765BF3F2-7FE1-414E-A134-F363D841D0DC@staffmail.ed.ac.uk>
References: <765BF3F2-7FE1-414E-A134-F363D841D0DC@staffmail.ed.ac.uk>
Message-ID: <CANNq6FkJL5Q-aLy-oqxCF5=tUVxS94OopAbb3hmwz018W_cKog@mail.gmail.com>

On Mon, Oct 7, 2013 at 1:15 PM, Siegfried Gonzi
<sgonzi at staffmail.ed.ac.uk>wrote:

> Hi all
>
> What is the equivalent to IDL its help function, e.g.
>
> ==
> IDL> a = make_array(23,23,)
>
> IDL> help,a
>
> will result in:
>
> A               FLOAT     = Array[23, 23]
>
> or
>
> IDL> a = create_struct('idl',23)
>
> IDL> help,a
>
> gives:
>
> A               STRUCT    = -> <Anonymous> Array[1]
>
> ==
>
> I have been looking for it ever since using numpy. It would make my life
> so much easier.
>
>
> Thanks, Siegfried
>
>
help(a)

Note that this is an inherent feature of python, not just numpy. So, you
can do help(range) or help(np.zeros) or any such thing.

Cheers!
Ben Root
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20131007/82f3abbe/attachment.html>

From njs at pobox.com  Mon Oct  7 13:43:25 2013
From: njs at pobox.com (Nathaniel Smith)
Date: Mon, 7 Oct 2013 18:43:25 +0100
Subject: [Numpy-discussion] Equivalent to IDL's help function
In-Reply-To: <765BF3F2-7FE1-414E-A134-F363D841D0DC@staffmail.ed.ac.uk>
References: <765BF3F2-7FE1-414E-A134-F363D841D0DC@staffmail.ed.ac.uk>
Message-ID: <CAPJVwB=xXsmXeFGgpN-jOoteXzuehYqtaujfou_ZC=w_GZGKVQ@mail.gmail.com>

The other answers tell you how to access Python's excellent quick reference
documentation, but from your examples I'm not sure if that's what you're
looking for? If you just want to know what value some variable has at the
interactive prompt, just type its name and hit enter. (This is a general
feature of the prompt: it shows you the value returned by whatever you
type, whether that be a single variable or a complex expression.)

Some other useful commands: dir(x), type(x)

-n
On 7 Oct 2013 18:17, "Siegfried Gonzi" <sgonzi at staffmail.ed.ac.uk> wrote:

> Hi all
>
> What is the equivalent to IDL its help function, e.g.
>
> ==
> IDL> a = make_array(23,23,)
>
> IDL> help,a
>
> will result in:
>
> A               FLOAT     = Array[23, 23]
>
> or
>
> IDL> a = create_struct('idl',23)
>
> IDL> help,a
>
> gives:
>
> A               STRUCT    = -> <Anonymous> Array[1]
>
> ==
>
> I have been looking for it ever since using numpy. It would make my life
> so much easier.
>
>
> Thanks, Siegfried
>
>
> --
> The University of Edinburgh is a charitable body, registered in
> Scotland, with registration number SC005336.
>
> _______________________________________________
> NumPy-Discussion mailing list
> NumPy-Discussion at scipy.org
> http://mail.scipy.org/mailman/listinfo/numpy-discussion
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20131007/459e1a3f/attachment.html>

From aronne.merrelli at gmail.com  Mon Oct  7 14:38:53 2013
From: aronne.merrelli at gmail.com (Aronne Merrelli)
Date: Mon, 7 Oct 2013 13:38:53 -0500
Subject: [Numpy-discussion] Equivalent to IDL's help function
In-Reply-To: <765BF3F2-7FE1-414E-A134-F363D841D0DC@staffmail.ed.ac.uk>
References: <765BF3F2-7FE1-414E-A134-F363D841D0DC@staffmail.ed.ac.uk>
Message-ID: <CAHNdQ4+jdw3fk18gtsJbfjuY1QJ0GzDLW6QyBE0oKu2XM1Anpg@mail.gmail.com>

There isn't anything quite the same. (I think what you are really asking
for is a way to print the essential info about one variable name, at least
that is how I would use the IDL "help" procedure). In IPython, I use the
whos magic to do something similar, although it just prints this info for
all variables or all variables of one class, rather than just one variable.
I do not think there is a way to do it for just one variable.

Here are some examples - you can see this works quite well but it will
become unwieldy if your interactive namespace becomes large:

In [1]: x = 1; y = 2; z = 3.3; d = {'one':1, 'two':2, 'three':3}

In [2]: whos
Variable   Type     Data/Info
-----------------------------
d          dict     n=3
x          int      1
y          int      2
z          float    3.3

In [3]: whos dict
Variable   Type    Data/Info
----------------------------
d          dict    n=3

In [4]: xa = np.arange(111); ya = np.ones((22,4))

In [5]: whos ndarray
Variable   Type       Data/Info
-------------------------------
xa         ndarray    111: 111 elems, type `int64`, 888 bytes
ya         ndarray    22x4: 88 elems, type `float64`, 704 bytes




On Mon, Oct 7, 2013 at 12:15 PM, Siegfried Gonzi
<sgonzi at staffmail.ed.ac.uk>wrote:

> Hi all
>
> What is the equivalent to IDL its help function, e.g.
>
> ==
> IDL> a = make_array(23,23,)
>
> IDL> help,a
>
> will result in:
>
> A               FLOAT     = Array[23, 23]
>
> or
>
> IDL> a = create_struct('idl',23)
>
> IDL> help,a
>
> gives:
>
> A               STRUCT    = -> <Anonymous> Array[1]
>
> ==
>
> I have been looking for it ever since using numpy. It would make my life
> so much easier.
>
>
> Thanks, Siegfried
>
>
> --
> The University of Edinburgh is a charitable body, registered in
> Scotland, with registration number SC005336.
>
> _______________________________________________
> NumPy-Discussion mailing list
> NumPy-Discussion at scipy.org
> http://mail.scipy.org/mailman/listinfo/numpy-discussion
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20131007/cccb9996/attachment.html>

From tmp50 at ukr.net  Mon Oct  7 16:22:17 2013
From: tmp50 at ukr.net (Dmitrey)
Date: Mon, 07 Oct 2013 23:22:17 +0300
Subject: [Numpy-discussion] [ANN] MATLAB ODE solvers - now available
 in	Python (Dmitrey)
In-Reply-To: <CAFtPsZouBMUWW072PNkPP17OfJtfswi8=9KuZE0dPPsnjptd0g@mail.gmail.com>
References: <CAFtPsZouBMUWW072PNkPP17OfJtfswi8=9KuZE0dPPsnjptd0g@mail.gmail.com>
Message-ID: <1381177204.440489225.k74gn9x8@frv43.ukr.net>

FYI scipy ODE solvers vode, dopri5, dop853 also have been connected to OpenOpt, possibly with automatic differentiation by FuncDesigner? (dopri5 and dop853 don't use derivatives although). 

---------------------- 
Regards, D. http://openopt.org/Dmitrey 

--- ???????? ????????? --- 
?? ????: "David Goldsmith" < d.l.goldsmith at gmail.com > 
????: 7 ??????? 2013, 07:16:33 

On Sun, Oct 6, 2013 at 10:00 AM, < numpy-discussion-request at scipy.org > wrote: 
Message: 2 
Date: Sat, 05 Oct 2013 21:36:48 +0300 
From: Dmitrey < tmp50 at ukr.net > 
Subject: Re: [Numpy-discussion] [ANN] MATLAB ODE solvers - now 
? ? ? ? available in ? ?Python 
To: Discussion of Numerical Python < numpy-discussion at scipy.org > 
Cc: numpy-discussion at scipy.org 
Message-ID: < 1380997576.559804301.aoynahlj at frv43.ukr.net > 
Content-Type: text/plain; charset="utf-8" 
? Seems like using the MATLAB solvers with MCR requires my wrappers containing in several files to be compiled with MATLAB Compiler before. I have no license for MATLAB thus I may have problems if I'll make it done and will spread it with OpenOpt suite code, also, binary files are incompatible with BSD license. 

Darn, knew it was too good to be true. ? On the other hand, IIRC a little bit obsolete MATLAB versions (I don't think difference is essential) have more liberal licenses. 
As for MATLAB solvers examples, I have already mentioned them in the mail list, you could see them in http://openopt.org/ODE (just replace solver name from scipy_lsoda to ode23s or any other), http://openopt.org/NLP , http://openopt.org/SNLE 
Oooops, so sorry. :-o? 
DG? 
---------------------- 
Regards, D. http://openopt.org/Dmitrey 
-------------- next part -------------- 
An HTML attachment was scrubbed... 
URL: http://mail.scipy.org/pipermail/numpy-discussion/attachments/20131005/dd6638db/attachment-0001.html 

------------------------------ 

_______________________________________________ 
NumPy-Discussion mailing list 
NumPy-Discussion at scipy.org 
http://mail.scipy.org/mailman/listinfo/numpy-discussion 


End of NumPy-Discussion Digest, Vol 85, Issue 17 
************************************************ 



-- 
>From "A Letter From The Future" in "Peak Everything" by Richard Heinberg: 

"By the time I was an older teenager, a certain...attitude was developing among the young people...a feeling of utter contempt for anyone over a certain age--maybe 30 or 40.? The adults had consumed so many resources, and now there were none left for their own children...when those adults were younger, they [were] just doing what everybody else was doing...they figured it was normal to cut down ancient forests for...phone books, pump every last gallon of oil to power their SUV's...[but] for...my generation all that was just a dim memory...We [grew up] living in darkness, with shortages of food and water, with riots in the streets, with people begging on street corners...for us, the adults were the enemy." 

Want to really understand what's really going on?? Read "Peak Everything." 


_______________________________________________
NumPy-Discussion mailing list
 NumPy-Discussion at scipy.org 
 http://mail.scipy.org/mailman/listinfo/numpy-discussion 

 
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20131007/20a4c824/attachment.html>

From sgonzi at staffmail.ed.ac.uk  Mon Oct  7 16:40:08 2013
From: sgonzi at staffmail.ed.ac.uk (Siegfried Gonzi)
Date: Mon, 7 Oct 2013 21:40:08 +0100
Subject: [Numpy-discussion] NumPy-Discussion Digest, Vol 85, Issue 19
In-Reply-To: <mailman.5464.1381177012.971.numpy-discussion@scipy.org>
References: <mailman.5464.1381177012.971.numpy-discussion@scipy.org>
Message-ID: <42BAB22A-EEE7-4A6D-A9B8-85D4A2DCB9F0@staffmail.ed.ac.uk>


On 7 Oct 2013, at 21:16, numpy-discussion-request at scipy.org wrote:
> 
> 
> Message: 5
> Date: Mon, 7 Oct 2013 13:38:53 -0500
> From: Aronne Merrelli <aronne.merrelli at gmail.com>
> Subject: Re: [Numpy-discussion] Equivalent to IDL's help function
> 
> 
> There isn't anything quite the same. (I think what you are really asking
> for is a way to print the essential info about one variable name, at least
> that is how I would use the IDL "help" procedure). In IPython, I use the
> whos magic to do something similar, although it just prints this info for
> all variables or all variables of one class, rather than just one variable.
> I do not think there is a way to do it for just one variable.
> 
> Here are some examples - you can see this works quite well but it will
> become unwieldy if your interactive namespace becomes large:
> 
> In [1]: x = 1; y = 2; z = 3.3; d = {'one':1, 'two':2, 'three':3}
> 
> In [2]: whos
> Variable   Type     Data/Info
> -----------------------------
> d          dict     n=3
> x          int      1
> y          int      2
> z          float    3.3
> 
> In [3]: whos dict
> Variable   Type    Data/Info
> ----------------------------
> d          dict    n=3
> 
> In [4]: xa = np.arange(111); ya = np.ones((22,4))
> 
> In [5]: whos ndarray
> Variable   Type       Data/Info
> -------------------------------
> xa         ndarray    111: 111 elems, type `int64`, 888 bytes
> ya         ndarray    22x4: 88 elems, type `float64`, 704 bytes
> 
> 

Hi

[I hope I am not screwing up the digest reply function here].

I am after a  "whos" which would work in a script. It is not very often that I develop code at the command line.

I am definitely not one of the best  programmers out there but I used "help" a lot in my IDL  scripts and code. Our research group is migrating away from IDL towards Python.

I think Python's help is not the same than IDL's help. I know copying things from other languages is not always a good idea but one cannot argue that IDL's help comes in rather handy while developing and testing code.

 

 

> 
> 
> On Mon, Oct 7, 2013 at 12:15 PM, Siegfried Gonzi
> <sgonzi at staffmail.ed.ac.uk>wrote:
> 
>> Hi all
>> 
>> What is the equivalent to IDL its help function, e.g.
>> 
>> ==
>> IDL> a = make_array(23,23,)
>> 
>> IDL> help,a
>> 
>> will result in:
>> 
>> A               FLOAT     = Array[23, 23]
>> 
>> or
>> 
>> IDL> a = create_struct('idl',23)
>> 
>> IDL> help,a
>> 
>> gives:
>> 
>> A               STRUCT    = -> <Anonymous> Array[1]
>> 
>> ==
>> 
>> I have been looking for it ever since using numpy. It would make my life
>> so much easier.
>> 
>> 
>> Thanks, Siegfried
>> 
>> 
>> --
>> The University of Edinburgh is a charitable body, registered in
>> Scotland, with registration number SC005336.
>> 
>> _______________________________________________
>> NumPy-Discussion mailing list
>> NumPy-Discussion at scipy.org
>> http://mail.scipy.org/mailman/listinfo/numpy-discussion
>> 
> -------------- next part --------------
> An HTML attachment was scrubbed...
> URL: http://mail.scipy.org/pipermail/numpy-discussion/attachments/20131007/cccb9996/attachment-0001.html 
> 
> ------------------------------
> 
> Message: 6
> Date: Mon, 07 Oct 2013 23:22:17 +0300
> From: Dmitrey <tmp50 at ukr.net>
> Subject: Re: [Numpy-discussion] [ANN] MATLAB ODE solvers - now
> 	available in	Python (Dmitrey)
> To: Discussion of Numerical Python <numpy-discussion at scipy.org>
> Cc: numpy-discussion at scipy.org
> Message-ID: <1381177204.440489225.k74gn9x8 at frv43.ukr.net>
> Content-Type: text/plain; charset="utf-8"
> 
> FYI scipy ODE solvers vode, dopri5, dop853 also have been connected to OpenOpt, possibly with automatic differentiation by FuncDesigner? (dopri5 and dop853 don't use derivatives although). 
> 
> ---------------------- 
> Regards, D. http://openopt.org/Dmitrey 
> 
> --- ???????? ????????? --- 
> ?? ????: "David Goldsmith" < d.l.goldsmith at gmail.com > 
> ????: 7 ??????? 2013, 07:16:33 
> 
> On Sun, Oct 6, 2013 at 10:00 AM, < numpy-discussion-request at scipy.org > wrote: 
> Message: 2 
> Date: Sat, 05 Oct 2013 21:36:48 +0300 
> From: Dmitrey < tmp50 at ukr.net > 
> Subject: Re: [Numpy-discussion] [ANN] MATLAB ODE solvers - now 
> ? ? ? ? available in ? ?Python 
> To: Discussion of Numerical Python < numpy-discussion at scipy.org > 
> Cc: numpy-discussion at scipy.org 
> Message-ID: < 1380997576.559804301.aoynahlj at frv43.ukr.net > 
> Content-Type: text/plain; charset="utf-8" 
> ? Seems like using the MATLAB solvers with MCR requires my wrappers containing in several files to be compiled with MATLAB Compiler before. I have no license for MATLAB thus I may have problems if I'll make it done and will spread it with OpenOpt suite code, also, binary files are incompatible with BSD license. 
> 
> Darn, knew it was too good to be true. ? On the other hand, IIRC a little bit obsolete MATLAB versions (I don't think difference is essential) have more liberal licenses. 
> As for MATLAB solvers examples, I have already mentioned them in the mail list, you could see them in http://openopt.org/ODE (just replace solver name from scipy_lsoda to ode23s or any other), http://openopt.org/NLP , http://openopt.org/SNLE 
> Oooops, so sorry. :-o? 
> DG? 
> ---------------------- 
> Regards, D. http://openopt.org/Dmitrey 
> -------------- next part -------------- 
> An HTML attachment was scrubbed... 
> URL: http://mail.scipy.org/pipermail/numpy-discussion/attachments/20131005/dd6638db/attachment-0001.html 
> 
> ------------------------------ 
> 
> _______________________________________________ 
> NumPy-Discussion mailing list 
> NumPy-Discussion at scipy.org 
> http://mail.scipy.org/mailman/listinfo/numpy-discussion 
> 
> 
> End of NumPy-Discussion Digest, Vol 85, Issue 17 
> ************************************************ 
> 
> 
> 
> -- 
>> From "A Letter From The Future" in "Peak Everything" by Richard Heinberg: 
> 
> "By the time I was an older teenager, a certain...attitude was developing among the young people...a feeling of utter contempt for anyone over a certain age--maybe 30 or 40.? The adults had consumed so many resources, and now there were none left for their own children...when those adults were younger, they [were] just doing what everybody else was doing...they figured it was normal to cut down ancient forests for...phone books, pump every last gallon of oil to power their SUV's...[but] for...my generation all that was just a dim memory...We [grew up] living in darkness, with shortages of food and water, with riots in the streets, with people begging on street corners...for us, the adults were the enemy." 
> 
> Want to really understand what's really going on?? Read "Peak Everything." 
> 
> 
> _______________________________________________
> NumPy-Discussion mailing list
> NumPy-Discussion at scipy.org 
> http://mail.scipy.org/mailman/listinfo/numpy-discussion 
> 
> 
> -------------- next part --------------
> An HTML attachment was scrubbed...
> URL: http://mail.scipy.org/pipermail/numpy-discussion/attachments/20131007/20a4c824/attachment.html 
> 
> ------------------------------
> 
> _______________________________________________
> NumPy-Discussion mailing list
> NumPy-Discussion at scipy.org
> http://mail.scipy.org/mailman/listinfo/numpy-discussion
> 
> 
> End of NumPy-Discussion Digest, Vol 85, Issue 19
> ************************************************
> 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20131007/d0ee7bb2/attachment.html>
-------------- next part --------------
An embedded and charset-unspecified text was scrubbed...
Name: not available
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20131007/d0ee7bb2/attachment.ksh>

From njs at pobox.com  Mon Oct  7 16:54:35 2013
From: njs at pobox.com (Nathaniel Smith)
Date: Mon, 7 Oct 2013 21:54:35 +0100
Subject: [Numpy-discussion] NumPy-Discussion Digest, Vol 85, Issue 19
In-Reply-To: <42BAB22A-EEE7-4A6D-A9B8-85D4A2DCB9F0@staffmail.ed.ac.uk>
References: <mailman.5464.1381177012.971.numpy-discussion@scipy.org>
	<42BAB22A-EEE7-4A6D-A9B8-85D4A2DCB9F0@staffmail.ed.ac.uk>
Message-ID: <CAPJVwBnkw5+Bc3i=0H2LfDRJo07GVaOQ-wwMM1fF_6XdVMkjEA@mail.gmail.com>

On Mon, Oct 7, 2013 at 9:40 PM, Siegfried Gonzi
<sgonzi at staffmail.ed.ac.uk> wrote:
> [I hope I am not screwing up the digest reply function here].

[You did, FYI. It happens...]

> I am after a  "whos" which would work in a script. It is not very often that
> I develop code at the command line.
>
> I am definitely not one of the best  programmers out there but I used "help"
> a lot in my IDL  scripts and code. Our research group is migrating away from
> IDL towards Python.
>
> I think Python's help is not the same than IDL's help. I know copying things
> from other languages is not always a good idea but one cannot argue that
> IDL's help comes in rather handy while developing and testing code.

>From a script try:

print(repr(obj))

or

print(type(obj))

or if you always want just a 1 line summary:

def idl_help(obj):
    if isinstance(obj, np.ndarray):
        print("<ndarray with shape %s, dtype=%s>" % (obj.shape, obj.dtype))
    else:
        print(type(obj)
idl_help(obj)

(Customize to suit)

-n


From Bernhard.Spinnler at gmx.net  Mon Oct  7 17:09:41 2013
From: Bernhard.Spinnler at gmx.net (Bernhard Spinnler)
Date: Mon, 7 Oct 2013 23:09:41 +0200
Subject: [Numpy-discussion] Bug in numpy.correlate documentation
Message-ID: <847A3113-2EF8-4AE6-84DC-95AB2D9EFEDB@gmx.net>

The numpy.correlate documentation says:

	correlate(a, v) = z[k] = sum_n a[n] * conj(v[n+k])

In [1]: a = [1, 2]

In [2]: v = [2, 1j]

In [3]: z = correlate(a, v, 'full')

In [4]: z
Out[4]: array([ 0.-1.j,  2.-2.j,  4.+0.j])

However, according to the documentation, z should be

	z[-1] = a[1] * conj(v[0]) = 4.+0.j
	z[0]  = a[0] * conj(v[0]) + a[1] * conj(v[1]) = 2.-2.j
	z[1] = a[0] * conj(v[1]) = 0.-1.j

which is the time reversed version of what correlate() calculates.

IMHO, the correlate() code is correct. The correct formula in the docs (which is also the correlation formula in standard text books) should be

	z[k] = sum_n a[n+k] * conj(v[n])

Cheers,
	Bernhard

From fernando.j at inbox.com  Mon Oct  7 19:27:55 2013
From: fernando.j at inbox.com (john fernando)
Date: Mon, 7 Oct 2013 15:27:55 -0800
Subject: [Numpy-discussion] switching from Numeric to numpy
Message-ID: <188EE056906.00000B14fernando.j@inbox.com>

I have the following C code which is an extension to my python code.

The python and C code use 
#include <Numeric/arrayobject.h>

what is the equivalent I can use in numpy that causes the minimum code change?

I did look through the old messages but didn't really find the answer-any help to a novice just starting out in numpy/python/C is much appreciated.

____________________________________________________________
FREE ONLINE PHOTOSHARING - Share your photos online with your friends and family!
Visit http://www.inbox.com/photosharing to find out more!




From charlesr.harris at gmail.com  Mon Oct  7 20:04:39 2013
From: charlesr.harris at gmail.com (Charles R Harris)
Date: Mon, 7 Oct 2013 18:04:39 -0600
Subject: [Numpy-discussion] switching from Numeric to numpy
In-Reply-To: <188EE056906.00000B14fernando.j@inbox.com>
References: <188EE056906.00000B14fernando.j@inbox.com>
Message-ID: <CAB6mnx+A+7ii0+1WPL_nqtFSvyVfd3t00fZnONnSjuJ3b0FLEA@mail.gmail.com>

On Mon, Oct 7, 2013 at 5:27 PM, john fernando <fernando.j at inbox.com> wrote:

> I have the following C code which is an extension to my python code.
>
> The python and C code use
> #include <Numeric/arrayobject.h>
>
> what is the equivalent I can use in numpy that causes the minimum code
> change?
>
> I did look through the old messages but didn't really find the answer-any
> help to a novice just starting out in numpy/python/C is much appreciated.
>

I haven't done any porting myself, but you might want to look at
numpy/oldnumeric to see how compatibility is done. That module is gone in
numpy master, but is included in earlier versions. As to the include, for
numpy it is `<numpy/arrayobject.h>, but a lot of things have been
added/changed/deprecated. There are also two scripts in oldnumeric that
help translating numeric to numpy: alter_code1.py and alter_code2.py. I
haven't used either.

There are probably folks on the list who can give you more specific advice.

Chuck
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20131007/e95c4ce9/attachment.html>

From sunk.cs at gmail.com  Tue Oct  8 04:06:25 2013
From: sunk.cs at gmail.com (Ke Sun)
Date: Tue, 8 Oct 2013 10:06:25 +0200
Subject: [Numpy-discussion] Efficient square distance computation
Message-ID: <20131008080625.GA26473@probe.unige.ch>

Dear all,

I have written the following function to compute the square distances of a large
matrix (each sample a row). It compute row by row and print the overall progress.
The progress output is important and I didn't use matrix multiplication.

I give as input a 70,000x800 matrix. The output should be a 70,000x70,000
matrix. The program runs really slow (16 hours for 1/3 progress). And it eats
36G memory (fortunately I have enough). 

Could you give some insights on how to modify the code to be efficient and
to eat less memory?

thanks,
Ke Sun

def dist2_large( data ):
    import time
    if data.ndim != 2: raise RuntimeError( "data should be a matrix" )
    N,D = data.shape

    print 'using the sample-wise implementation'
    print '%d samples, %d dimensions' % (N,D)

    start_t = time.time()
    d2 = np.zeros( [N,N] )
    for i in range( N ):
        print "\r%5d/%d" % (i+1, N), 
        for j in range( N ):
            d2[i,j] = ((data[i] - data[j])**2).sum()

    total_t = time.time() - start_t
    hours = (total_t / 3600)
    minutes = (total_t % 3600) / 60
    print "\nfinished in %2dh%2dm" % (hours, minutes)

    return d2



From matthew.brett at gmail.com  Tue Oct  8 04:49:14 2013
From: matthew.brett at gmail.com (Matthew Brett)
Date: Tue, 8 Oct 2013 01:49:14 -0700
Subject: [Numpy-discussion] Efficient square distance computation
In-Reply-To: <20131008080625.GA26473@probe.unige.ch>
References: <20131008080625.GA26473@probe.unige.ch>
Message-ID: <CAH6Pt5otBzcm8v2z357Tme2a4RK=qo7YetWRMosxHM+UtnpumA@mail.gmail.com>

Hi,

On Tue, Oct 8, 2013 at 1:06 AM, Ke Sun <sunk.cs at gmail.com> wrote:
> Dear all,
>
> I have written the following function to compute the square distances of a large
> matrix (each sample a row). It compute row by row and print the overall progress.
> The progress output is important and I didn't use matrix multiplication.
>
> I give as input a 70,000x800 matrix. The output should be a 70,000x70,000
> matrix. The program runs really slow (16 hours for 1/3 progress). And it eats
> 36G memory (fortunately I have enough).

That is very slow.

As a matter of interest - why didn't you use matrix multiplication?
On a machine I had access to it took about 20 minutes.

You've got a 70000 by 70000 element output matrix so I think that's
37G already (if the matrix is double precision float).

> Could you give some insights on how to modify the code to be efficient and
> to eat less memory?

You could try using Cython - but I'm guessing that the BLAS routines
in numpy will already do this the most efficient way.

Best,

Matthew


From jtaylor.debian at googlemail.com  Tue Oct  8 05:01:15 2013
From: jtaylor.debian at googlemail.com (Julian Taylor)
Date: Tue, 8 Oct 2013 11:01:15 +0200
Subject: [Numpy-discussion] Efficient square distance computation
In-Reply-To: <20131008080625.GA26473@probe.unige.ch>
References: <20131008080625.GA26473@probe.unige.ch>
Message-ID: <CAK5FAtFU+cJyTb1idNHF8BZBKZBXZ-M0Z91NEx6R4RNmUUsn_g@mail.gmail.com>

your computation is symmetric so you only need to compute the upper or
lower triangle which will save both memory and time.


On Tue, Oct 8, 2013 at 10:06 AM, Ke Sun <sunk.cs at gmail.com> wrote:

> Dear all,
>
> I have written the following function to compute the square distances of a
> large
> matrix (each sample a row). It compute row by row and print the overall
> progress.
> The progress output is important and I didn't use matrix multiplication.
>
> I give as input a 70,000x800 matrix. The output should be a 70,000x70,000
> matrix. The program runs really slow (16 hours for 1/3 progress). And it
> eats
> 36G memory (fortunately I have enough).
>
> Could you give some insights on how to modify the code to be efficient and
> to eat less memory?
>
> thanks,
> Ke Sun
>
> def dist2_large( data ):
>     import time
>     if data.ndim != 2: raise RuntimeError( "data should be a matrix" )
>     N,D = data.shape
>
>     print 'using the sample-wise implementation'
>     print '%d samples, %d dimensions' % (N,D)
>
>     start_t = time.time()
>     d2 = np.zeros( [N,N] )
>     for i in range( N ):
>         print "\r%5d/%d" % (i+1, N),
>         for j in range( N ):
>             d2[i,j] = ((data[i] - data[j])**2).sum()
>
>     total_t = time.time() - start_t
>     hours = (total_t / 3600)
>     minutes = (total_t % 3600) / 60
>     print "\nfinished in %2dh%2dm" % (hours, minutes)
>
>     return d2
>
> _______________________________________________
> NumPy-Discussion mailing list
> NumPy-Discussion at scipy.org
> http://mail.scipy.org/mailman/listinfo/numpy-discussion
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20131008/feb298c4/attachment.html>

From heng at cantab.net  Tue Oct  8 05:55:58 2013
From: heng at cantab.net (Henry Gomersall)
Date: Tue, 08 Oct 2013 10:55:58 +0100
Subject: [Numpy-discussion] Efficient square distance computation
In-Reply-To: <20131008080625.GA26473@probe.unige.ch>
References: <20131008080625.GA26473@probe.unige.ch>
Message-ID: <5253D6AE.1090206@cantab.net>

On 08/10/13 09:06, Ke Sun wrote:
> I give as input a 70,000x800 matrix. The output should be a 70,000x70,000
> matrix. The program runs really slow (16 hours for 1/3 progress). And it eats
> 36G memory (fortunately I have enough).

At this stage I'd be asking myself what I'm trying to achieve and why 
I'm trying to explicitly calculate every element of a 70,000x70,000 
matrix and how I might do it otherwise.

Most optimisations when it comes to things like this make use of the 
structure of the matrix and knowledge of the algorithm in which it fits.

Cheers,

Henry


From heng at cantab.net  Tue Oct  8 05:56:52 2013
From: heng at cantab.net (Henry Gomersall)
Date: Tue, 08 Oct 2013 10:56:52 +0100
Subject: [Numpy-discussion] Efficient square distance computation
In-Reply-To: <CAH6Pt5otBzcm8v2z357Tme2a4RK=qo7YetWRMosxHM+UtnpumA@mail.gmail.com>
References: <20131008080625.GA26473@probe.unige.ch>
	<CAH6Pt5otBzcm8v2z357Tme2a4RK=qo7YetWRMosxHM+UtnpumA@mail.gmail.com>
Message-ID: <5253D6E4.5040104@cantab.net>

On 08/10/13 09:49, Matthew Brett wrote:
> On Tue, Oct 8, 2013 at 1:06 AM, Ke Sun<sunk.cs at gmail.com>  wrote:
>> >Dear all,
>> >
>> >I have written the following function to compute the square distances of a large
>> >matrix (each sample a row). It compute row by row and print the overall progress.
>> >The progress output is important and I didn't use matrix multiplication.
>> >
>> >I give as input a 70,000x800 matrix. The output should be a 70,000x70,000
>> >matrix. The program runs really slow (16 hours for 1/3 progress). And it eats
>> >36G memory (fortunately I have enough).
> That is very slow.
>
> As a matter of interest - why didn't you use matrix multiplication?
> On a machine I had access to it took about 20 minutes.

Out of interest, how did you do this with matrix multiplication?

Henry


From jtaylor.debian at googlemail.com  Tue Oct  8 07:23:57 2013
From: jtaylor.debian at googlemail.com (Julian Taylor)
Date: Tue, 8 Oct 2013 13:23:57 +0200
Subject: [Numpy-discussion] Efficient square distance computation
In-Reply-To: <5253D6E4.5040104@cantab.net>
References: <20131008080625.GA26473@probe.unige.ch>
	<CAH6Pt5otBzcm8v2z357Tme2a4RK=qo7YetWRMosxHM+UtnpumA@mail.gmail.com>
	<5253D6E4.5040104@cantab.net>
Message-ID: <CAK5FAtHmvqt2ML1B0jGLzioYDoztFLuOL1pcwdJBTrePkj1X-w@mail.gmail.com>

> Out of interest, how did you do this with matrix multiplication?

http://stackoverflow.com/a/4856692
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20131008/39950e11/attachment.html>

From sunk.cs at gmail.com  Tue Oct  8 07:38:28 2013
From: sunk.cs at gmail.com (Ke Sun)
Date: Tue, 8 Oct 2013 13:38:28 +0200
Subject: [Numpy-discussion] Efficient square distance computation
In-Reply-To: <CAH6Pt5otBzcm8v2z357Tme2a4RK=qo7YetWRMosxHM+UtnpumA@mail.gmail.com>
References: <20131008080625.GA26473@probe.unige.ch>
	<CAH6Pt5otBzcm8v2z357Tme2a4RK=qo7YetWRMosxHM+UtnpumA@mail.gmail.com>
Message-ID: <20131008113828.GA31484@probe.unige.ch>

On Tue, Oct 08, 2013 at 01:49:14AM -0700, Matthew Brett wrote:
> Hi,
> 
> On Tue, Oct 8, 2013 at 1:06 AM, Ke Sun <sunk.cs at gmail.com> wrote:
> > Dear all,
> >
> > I have written the following function to compute the square distances of a large
> > matrix (each sample a row). It compute row by row and print the overall progress.
> > The progress output is important and I didn't use matrix multiplication.
> >
> > I give as input a 70,000x800 matrix. The output should be a 70,000x70,000
> > matrix. The program runs really slow (16 hours for 1/3 progress). And it eats
> > 36G memory (fortunately I have enough).
> 
> That is very slow.
> 
> As a matter of interest - why didn't you use matrix multiplication?
Because it will cost hours and I want to see the progress and 
know how far it goes. Another concern is to save memory and
compute one sample at a time.

> On a machine I had access to it took about 20 minutes.
How? I am using matrix multiplication (the same code as 
http://stackoverflow.com/a/4856692) and it runs for around 18 hours.

Best,
Ke Sun
> 
> You've got a 70000 by 70000 element output matrix so I think that's
> 37G already (if the matrix is double precision float).
> 
> > Could you give some insights on how to modify the code to be efficient and
> > to eat less memory?
> 
> You could try using Cython - but I'm guessing that the BLAS routines
> in numpy will already do this the most efficient way.


From valentin at haenel.co  Tue Oct  8 08:23:12 2013
From: valentin at haenel.co (Valentin Haenel)
Date: Tue, 8 Oct 2013 14:23:12 +0200
Subject: [Numpy-discussion] Question about typenum
Message-ID: <20131008122312.GA13963@kudu.in-berlin.de>

Hi,

I have a quick question about typenum.  Certain functions, like
`PyArray_SimpleNewFromData` `PyArray_SimpleNew` take a typeenum
argument. Is there any way to go from typeenum to something that can be
passed to the dtype constructor, like mapping 12 -> '<f8'?

thanks,

V-



From jtaylor.debian at googlemail.com  Tue Oct  8 08:40:37 2013
From: jtaylor.debian at googlemail.com (Julian Taylor)
Date: Tue, 8 Oct 2013 14:40:37 +0200
Subject: [Numpy-discussion] Efficient square distance computation
In-Reply-To: <20131008113828.GA31484@probe.unige.ch>
References: <20131008080625.GA26473@probe.unige.ch>
	<CAH6Pt5otBzcm8v2z357Tme2a4RK=qo7YetWRMosxHM+UtnpumA@mail.gmail.com>
	<20131008113828.GA31484@probe.unige.ch>
Message-ID: <CAK5FAtHn_xoK935uZs_jEWanvaqkR3yKXGW9Po0Mh8OkzJSxRw@mail.gmail.com>

On Tue, Oct 8, 2013 at 1:38 PM, Ke Sun <sunk.cs at gmail.com> wrote:

>
> > On a machine I had access to it took about 20 minutes.
> How? I am using matrix multiplication (the same code as
> http://stackoverflow.com/a/4856692) and it runs for around 18 hours.
>
>
make sure you are using an optimized BLAS library.
e.g. with openblas I could also do this size in about 100 seconds wall time
(20 minutes cpu time).
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20131008/6dcf4b00/attachment.html>

From bahtiyor_zohidov at mail.ru  Tue Oct  8 09:15:54 2013
From: bahtiyor_zohidov at mail.ru (=?UTF-8?B?SGFwcHltYW4=?=)
Date: Tue, 08 Oct 2013 17:15:54 +0400
Subject: [Numpy-discussion] =?utf-8?q?Python_function_for_line_intersectio?=
	=?utf-8?b?bj8/?=
Message-ID: <1381238154.58791984@f77.i.mail.ru>

 Hello,

I am having trouble with solving line intersection problem. I would solve it using standard functions which may already exist in Python library. 
Let me explain it below:
I have sets of lines A(x,y) and B(x,y) and one map as a matrix form, also with its resolution dx, dy which can be changed.
I wanted to calculate all intersected points of line with the map depending on dx, dy value. I already did it, but two problems:
1) When some coordinate value (x or y) are zero, the program returns wrong answer. For example, distance is zero, but it is not zero in fact.
2) How can I optimize my code (using Python standard library), if possible.

Or is there standard Python library for that purpose???

import numpy as np

def intrsctd_pnts(map, dx, dy, A, B, line_number):

??? height, width = np.shape(map)

??? # slope of each line
??? m = (B[:, 1] - A[:, 1]) / (B[:, 0] - A[:, 0])
??? b = A[:, 1] - m * A[:, 0]

??? lx_min = np.minimum(A[:,0], B[:,0])
??? lx_max = np.maximum(A[:,0], B[:,0])
??? ly_min = np.minimum(A[:,1], B[:,1])
??? ly_max = np.maximum(A[:,1], B[:,1])

??? lx_floor = np.ceil(lx_min /dx)*dx
??? lx_ceil = np.floor(lx_max /dx)*dx
??? ly_floor = np.ceil(ly_min /dy)*dy
??? ly_ceil = np.floor(ly_max /dy)*dy

??? distance = []
??? i_pixel = []
??? intersect_point = []
??? for i in range(line_number):
??????? AB = []
??????? # calculate all intersected x coordinates
??????? x = np.arange(lx_floor[i], lx_ceil[i]+dx, dx)
??????? y = m[i] * x + b[i]
??????? AB = zip(x, y)
??????? # calculate all intersected y coordinates
??????? y = np.arange(ly_floor[i], ly_ceil[i]+dy, dy)
??????? x = (y - b[i]) / m[i]
??????? AB.extend(zip(x, y))
??????? AB.append((A[i,0], A[i,1]))
??????? AB.append((B[i,0], B[i,1]))
??????? AB.sort()
??????? AB = np.asarray(AB)
??????? intersect_point.append(AB)
??????? # compute distance between two points
??????? dist_x = np.diff(AB[:,0])
??????? dist_y = np.diff(AB[:,1])
??????? distance.append( np.sqrt((dist_x)**2 + (dist_y)**2) )

??????? # FIND pixel value where line intersects that pixel
??????? col = (np.floor( min(AB[:,0]) / dx)).astype(int)
??????? row = (np.floor( min(AB[:,1]) / dy)).astype(int)

??????? # FIND  pixel to which this lines belong
??????? i_pixel.append(col + width*row)

??? return distance, i_pixel, intersect_point


def main():
??? # INPUTS:
? ? map = np.array( [ [ 4.5,? 4.5,? 4.5,? 3.4],
?????????????????????????????? [ 3.9,? 4.5,? 5.2,? 4.5],
?? ? ? ? ? ? ? ? ? ? ? ? ? ? ? [ 3.9,? 3.9,? 2.5,? 2.2],
? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?[ 3.4,? 3.9,? 2.9,? 2.2],
???????????????????????????????[ 2.5,? 3.4,? 2.2,? 1.4],
? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?[ 2.5,? 2.2,? 2.5,? 1.2] )
??? dx = dy = 0.5
??? A = np.array( [ [ 0. , 0. ], 
? ? ? ? ? ? ? ? ? ? ? ???? [ 1.9 , 0.] ])
? 
? ? B = np.array( [ [ 1.1 , 2.3],
? ? ? ? ? ? ? ? ? ? ? ? ?? [ 1.3 , 1.3] ])

? ? line_number = len(A)? # or len(B)
? 
? ? distance, pixels, int_points = intrsctd_pnts(map, dx, dy, A, B, line_number)
? 
?? ##? distance should not be zero

if __name__ == '__main__':
??? main()


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20131008/5bc6cac4/attachment.html>

From jaime.frio at gmail.com  Tue Oct  8 09:42:04 2013
From: jaime.frio at gmail.com (=?ISO-8859-1?Q?Jaime_Fern=E1ndez_del_R=EDo?=)
Date: Tue, 8 Oct 2013 06:42:04 -0700
Subject: [Numpy-discussion] Efficient square distance computation
In-Reply-To: <20131008113828.GA31484@probe.unige.ch>
References: <20131008080625.GA26473@probe.unige.ch>
	<CAH6Pt5otBzcm8v2z357Tme2a4RK=qo7YetWRMosxHM+UtnpumA@mail.gmail.com>
	<20131008113828.GA31484@probe.unige.ch>
Message-ID: <CAPOWHW=hqPGVjaMPiMpz-UJzwcFwPBEZ6hP4sqCa9CWgqwJCHw@mail.gmail.com>

On Tue, Oct 8, 2013 at 4:38 AM, Ke Sun <sunk.cs at gmail.com> wrote:

> On Tue, Oct 08, 2013 at 01:49:14AM -0700, Matthew Brett wrote:
> > Hi,
> >
> > On Tue, Oct 8, 2013 at 1:06 AM, Ke Sun <sunk.cs at gmail.com> wrote:
> > > Dear all,
> > >
> > > I have written the following function to compute the square distances
> of a large
> > > matrix (each sample a row). It compute row by row and print the
> overall progress.
> > > The progress output is important and I didn't use matrix
> multiplication.
> > >
> > > I give as input a 70,000x800 matrix. The output should be a
> 70,000x70,000
> > > matrix. The program runs really slow (16 hours for 1/3 progress). And
> it eats
> > > 36G memory (fortunately I have enough).
> >
> > That is very slow.
> >
> > As a matter of interest - why didn't you use matrix multiplication?
> Because it will cost hours and I want to see the progress and
> know how far it goes. Another concern is to save memory and
> compute one sample at a time.
>

Another option you may want to consider is to do your calculation in
chunks, not one item at a time, e.g.:

rows, cols = 70000, 800

data = np.random.rand(rows, cols)

chunks = 100

chunk_len = rows // chunks


out = np.empty((rows, rows))

for j in xrange(0, rows, chunk_len):

    chunk_j = data[j: j + chunk_len]

        for k in xrange(j, rows, chunk_len):

            chunk_k = data[k: k + chunk_len]

            out[j: j + chunk_len,

                k: k + chunk_len] = np.dot(chunk_j, chunk_k.T)

            if j != k:

                out[k: k + chunk_len,

                    j: j + chunk_len] = out[j: j + chunk_len,

                                            k: k + chunk_len].T

q = np.diag(out)

out *= -2

out += q

out += q[:, np.newaxis]

This way you can still gauge progress, use mostly the fast, efficient
vectorized approach and probably offset the (relatively small amount of)
Python looping by not calculating most of the symmetrical items.

Jaime
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20131008/5ff706fc/attachment.html>

From rhattersley at gmail.com  Tue Oct  8 14:51:22 2013
From: rhattersley at gmail.com (Richard Hattersley)
Date: Tue, 8 Oct 2013 19:51:22 +0100
Subject: [Numpy-discussion] Question about typenum
In-Reply-To: <20131008122312.GA13963@kudu.in-berlin.de>
References: <20131008122312.GA13963@kudu.in-berlin.de>
Message-ID: <CAP=RS9=NFDE4=H6v41GwiFiS+RaMi8ySDfnXKJYmTk+_8v_aKw@mail.gmail.com>

Hi Valentin,

On 8 October 2013 13:23, Valentin Haenel <valentin at haenel.co> wrote:

> Certain functions, like
> `PyArray_SimpleNewFromData` `PyArray_SimpleNew` take a typeenum
> Is there any way to go from typeenum to something that can be
> passed to the dtype constructor, like mapping 12 -> '<f8'?
>

If you just want the corresponding dtype instance (aka PyArray_Descr) then
`PyArray_DescrFromType` should be what you're after.

But if you really need the '<f8' string then I'd be tempted to get the
PyArray_Descr and then use the Python API (e.g. PyObject_GetAttrString) to
request the "str" attribute. Under the hood this attribute is implemented
by `arraydescr_protocol_typestr_get` but that's not part of the public API.

Regards,
Richard Hattersley
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20131008/fa0a98a8/attachment.html>

From valentin at haenel.co  Tue Oct  8 14:56:33 2013
From: valentin at haenel.co (Valentin Haenel)
Date: Tue, 8 Oct 2013 20:56:33 +0200
Subject: [Numpy-discussion] Question about typenum
In-Reply-To: <CAP=RS9=NFDE4=H6v41GwiFiS+RaMi8ySDfnXKJYmTk+_8v_aKw@mail.gmail.com>
References: <20131008122312.GA13963@kudu.in-berlin.de>
	<CAP=RS9=NFDE4=H6v41GwiFiS+RaMi8ySDfnXKJYmTk+_8v_aKw@mail.gmail.com>
Message-ID: <20131008185633.GC13963@kudu.in-berlin.de>

Hi Richard,

* Richard Hattersley <rhattersley at gmail.com> [2013-10-08]:
> On 8 October 2013 13:23, Valentin Haenel <valentin at haenel.co> wrote:
> 
> > Certain functions, like
> > `PyArray_SimpleNewFromData` `PyArray_SimpleNew` take a typeenum
> > Is there any way to go from typeenum to something that can be
> > passed to the dtype constructor, like mapping 12 -> '<f8'?
> >
> 
> If you just want the corresponding dtype instance (aka PyArray_Descr) then
> `PyArray_DescrFromType` should be what you're after.
> 
> But if you really need the '<f8' string then I'd be tempted to get the
> PyArray_Descr and then use the Python API (e.g. PyObject_GetAttrString) to
> request the "str" attribute. Under the hood this attribute is implemented
> by `arraydescr_protocol_typestr_get` but that's not part of the public API.

Thanks for your answer. I ended up using: PyArray_TypeObjectFromType
from cython so:

np.dtype(cnp.PyArray_TypeObjectFromType(self.ndtype)).str

Maybe i can avoid the np.dtype call, when using PyArray_Descr?

best,

V-


From rhattersley at gmail.com  Tue Oct  8 15:10:41 2013
From: rhattersley at gmail.com (Richard Hattersley)
Date: Tue, 8 Oct 2013 20:10:41 +0100
Subject: [Numpy-discussion] Bug in numpy.correlate documentation
In-Reply-To: <847A3113-2EF8-4AE6-84DC-95AB2D9EFEDB@gmx.net>
References: <847A3113-2EF8-4AE6-84DC-95AB2D9EFEDB@gmx.net>
Message-ID: <CAP=RS9k54vtNFHy9ppG=U09oEHwB=KLV0xvwR6BfFgB3o5S=fw@mail.gmail.com>

Hi Bernard,

Looks like you're on to something - two other people have raised this
discrepancy before: https://github.com/numpy/numpy/issues/2588.
Unfortunately, when it comes to resolving the discrepancy one of the
previous comments takes the opposite view. Namely, that the docstring is
correct and the code is wrong.

Do different domains use different conventions here? Are there some
references to back up one stance or another?

But all else being equal, I'm guessing there'll be far more appetite for
updating the documentation than the code.

Regards,
Richard Hattersley


On 7 October 2013 22:09, Bernhard Spinnler <Bernhard.Spinnler at gmx.net>wrote:

> The numpy.correlate documentation says:
>
>         correlate(a, v) = z[k] = sum_n a[n] * conj(v[n+k])
>
> In [1]: a = [1, 2]
>
> In [2]: v = [2, 1j]
>
> In [3]: z = correlate(a, v, 'full')
>
> In [4]: z
> Out[4]: array([ 0.-1.j,  2.-2.j,  4.+0.j])
>
> However, according to the documentation, z should be
>
>         z[-1] = a[1] * conj(v[0]) = 4.+0.j
>         z[0]  = a[0] * conj(v[0]) + a[1] * conj(v[1]) = 2.-2.j
>         z[1] = a[0] * conj(v[1]) = 0.-1.j
>
> which is the time reversed version of what correlate() calculates.
>
> IMHO, the correlate() code is correct. The correct formula in the docs
> (which is also the correlation formula in standard text books) should be
>
>         z[k] = sum_n a[n+k] * conj(v[n])
>
> Cheers,
>         Bernhard
> _______________________________________________
> NumPy-Discussion mailing list
> NumPy-Discussion at scipy.org
> http://mail.scipy.org/mailman/listinfo/numpy-discussion
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20131008/03529007/attachment.html>

From rhattersley at gmail.com  Tue Oct  8 15:21:25 2013
From: rhattersley at gmail.com (Richard Hattersley)
Date: Tue, 8 Oct 2013 20:21:25 +0100
Subject: [Numpy-discussion] Question about typenum
In-Reply-To: <20131008185633.GC13963@kudu.in-berlin.de>
References: <20131008122312.GA13963@kudu.in-berlin.de>
	<CAP=RS9=NFDE4=H6v41GwiFiS+RaMi8ySDfnXKJYmTk+_8v_aKw@mail.gmail.com>
	<20131008185633.GC13963@kudu.in-berlin.de>
Message-ID: <CAP=RS9m4oELZZsv7faMOtiZrZXN6y3=tF=ocXM=Km2MNUyj7Fw@mail.gmail.com>

On 8 October 2013 19:56, Valentin Haenel <valentin at haenel.co> wrote:

> I ended up using: PyArray_TypeObjectFromType
> from cython so:
>
> np.dtype(cnp.PyArray_TypeObjectFromType(self.ndtype)).str
>
> Maybe i can avoid the np.dtype call, when using PyArray_Descr?
>

In short: yes.

`PyArray_TypeObjectFromType` first uses `PyArray_DescrFromType` to figure
out the dtype from the type number, and then it returns the corresponding
array scalar type. Passing this array scalar type to `np.dtype` gets you
back to the dtype that had just been looked up inside TypeObjectFromType.

Regards,
Richard
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20131008/98988814/attachment.html>

From valentin at haenel.co  Tue Oct  8 15:36:01 2013
From: valentin at haenel.co (Valentin Haenel)
Date: Tue, 8 Oct 2013 21:36:01 +0200
Subject: [Numpy-discussion] Question about typenum
In-Reply-To: <CAP=RS9m4oELZZsv7faMOtiZrZXN6y3=tF=ocXM=Km2MNUyj7Fw@mail.gmail.com>
References: <20131008122312.GA13963@kudu.in-berlin.de>
	<CAP=RS9=NFDE4=H6v41GwiFiS+RaMi8ySDfnXKJYmTk+_8v_aKw@mail.gmail.com>
	<20131008185633.GC13963@kudu.in-berlin.de>
	<CAP=RS9m4oELZZsv7faMOtiZrZXN6y3=tF=ocXM=Km2MNUyj7Fw@mail.gmail.com>
Message-ID: <20131008193601.GD13963@kudu.in-berlin.de>

* Richard Hattersley <rhattersley at gmail.com> [2013-10-08]:
> On 8 October 2013 19:56, Valentin Haenel <valentin at haenel.co> wrote:
> 
> > I ended up using: PyArray_TypeObjectFromType
> > from cython so:
> >
> > np.dtype(cnp.PyArray_TypeObjectFromType(self.ndtype)).str
> >
> > Maybe i can avoid the np.dtype call, when using PyArray_Descr?
> >
> 
> In short: yes.
> 
> `PyArray_TypeObjectFromType` first uses `PyArray_DescrFromType` to figure
> out the dtype from the type number, and then it returns the corresponding
> array scalar type. Passing this array scalar type to `np.dtype` gets you
> back to the dtype that had just been looked up inside TypeObjectFromType.

Excellent! Thank you very much!

V-


From matthew.brett at gmail.com  Tue Oct  8 15:44:11 2013
From: matthew.brett at gmail.com (Matthew Brett)
Date: Tue, 8 Oct 2013 12:44:11 -0700
Subject: [Numpy-discussion] Efficient square distance computation
In-Reply-To: <20131008113828.GA31484@probe.unige.ch>
References: <20131008080625.GA26473@probe.unige.ch>
	<CAH6Pt5otBzcm8v2z357Tme2a4RK=qo7YetWRMosxHM+UtnpumA@mail.gmail.com>
	<20131008113828.GA31484@probe.unige.ch>
Message-ID: <CAH6Pt5o=1BLuDALezQn8x2aiJ13o-g-eDKWg=8hsRupdxiH06w@mail.gmail.com>

Hi,

On Tue, Oct 8, 2013 at 4:38 AM, Ke Sun <sunk.cs at gmail.com> wrote:
> On Tue, Oct 08, 2013 at 01:49:14AM -0700, Matthew Brett wrote:
>> Hi,
>>
>> On Tue, Oct 8, 2013 at 1:06 AM, Ke Sun <sunk.cs at gmail.com> wrote:
>> > Dear all,
>> >
>> > I have written the following function to compute the square distances of a large
>> > matrix (each sample a row). It compute row by row and print the overall progress.
>> > The progress output is important and I didn't use matrix multiplication.
>> >
>> > I give as input a 70,000x800 matrix. The output should be a 70,000x70,000
>> > matrix. The program runs really slow (16 hours for 1/3 progress). And it eats
>> > 36G memory (fortunately I have enough).
>>
>> That is very slow.
>>
>> As a matter of interest - why didn't you use matrix multiplication?
> Because it will cost hours and I want to see the progress and
> know how far it goes. Another concern is to save memory and
> compute one sample at a time.
>
>> On a machine I had access to it took about 20 minutes.
> How? I am using matrix multiplication (the same code as
> http://stackoverflow.com/a/4856692) and it runs for around 18 hours.

I wonder if you are running into disk swap - the code there does
involve a large temporary array.

I believe the appended version of the code is correct, and I think it
is also memory efficient.

On a fast machine with lots of memory, it ran in about 5 minutes.
It's using EPD, which might be using multiple cores for the matrix
multiply.

Does the code also work for you in reasonable time?

Another suggestion I saw which only calculates the unique values (say
lower diagonal) is scipy.spatial.distance

http://docs.scipy.org/doc/scipy/reference/generated/scipy.spatial.distance.pdist.html#scipy.spatial.distance.pdist

To a first pass that seems to be slower than the matrix multiply.

Cheers,

Matthew

import datetime
import numpy as np

def pdista(X):
    """Squared pairwise distances between all columns of X."""
    B = np.dot(X.T, X)
    q = np.diag(B)[:, None].copy() # copy necessary?
    B *= -2
    B += q
    B += q.T
    return B

M = 70000
N = 800
A = np.random.normal(size=(M, N))

start = datetime.datetime.now()
dists = pdista(A.T)
elapsed = datetime.datetime.now() - start
print(elapsed)


From matthew.brett at gmail.com  Tue Oct  8 17:11:36 2013
From: matthew.brett at gmail.com (Matthew Brett)
Date: Tue, 8 Oct 2013 14:11:36 -0700
Subject: [Numpy-discussion] Efficient square distance computation
In-Reply-To: <CAH6Pt5o=1BLuDALezQn8x2aiJ13o-g-eDKWg=8hsRupdxiH06w@mail.gmail.com>
References: <20131008080625.GA26473@probe.unige.ch>
	<CAH6Pt5otBzcm8v2z357Tme2a4RK=qo7YetWRMosxHM+UtnpumA@mail.gmail.com>
	<20131008113828.GA31484@probe.unige.ch>
	<CAH6Pt5o=1BLuDALezQn8x2aiJ13o-g-eDKWg=8hsRupdxiH06w@mail.gmail.com>
Message-ID: <CAH6Pt5oih6rz=YUFsSa4s_bQw=2rL-65taTAQCCmGGvWDWDDFQ@mail.gmail.com>

Hi,

On Tue, Oct 8, 2013 at 12:44 PM, Matthew Brett <matthew.brett at gmail.com> wrote:
> Hi,
>
> On Tue, Oct 8, 2013 at 4:38 AM, Ke Sun <sunk.cs at gmail.com> wrote:
>> On Tue, Oct 08, 2013 at 01:49:14AM -0700, Matthew Brett wrote:
>>> Hi,
>>>
>>> On Tue, Oct 8, 2013 at 1:06 AM, Ke Sun <sunk.cs at gmail.com> wrote:
>>> > Dear all,
>>> >
>>> > I have written the following function to compute the square distances of a large
>>> > matrix (each sample a row). It compute row by row and print the overall progress.
>>> > The progress output is important and I didn't use matrix multiplication.
>>> >
>>> > I give as input a 70,000x800 matrix. The output should be a 70,000x70,000
>>> > matrix. The program runs really slow (16 hours for 1/3 progress). And it eats
>>> > 36G memory (fortunately I have enough).
>>>
>>> That is very slow.
>>>
>>> As a matter of interest - why didn't you use matrix multiplication?
>> Because it will cost hours and I want to see the progress and
>> know how far it goes. Another concern is to save memory and
>> compute one sample at a time.
>>
>>> On a machine I had access to it took about 20 minutes.
>> How? I am using matrix multiplication (the same code as
>> http://stackoverflow.com/a/4856692) and it runs for around 18 hours.
>
> I wonder if you are running into disk swap - the code there does
> involve a large temporary array.
>
> I believe the appended version of the code is correct, and I think it
> is also memory efficient.
>
> On a fast machine with lots of memory, it ran in about 5 minutes.
> It's using EPD, which might be using multiple cores for the matrix
> multiply.
>
> Does the code also work for you in reasonable time?
>
> Another suggestion I saw which only calculates the unique values (say
> lower diagonal) is scipy.spatial.distance
>
> http://docs.scipy.org/doc/scipy/reference/generated/scipy.spatial.distance.pdist.html#scipy.spatial.distance.pdist
>
> To a first pass that seems to be slower than the matrix multiply.

On a machine that should have been about as fast:

import scipy.spatial.distance as ssd
dists = ssd.pdist(A, 'sqeuclidean')

on the same size matrix took about 40 minutes.

Cheers,

Matthew


From Roelf.Schreurs at tibra.com  Wed Oct  9 09:46:12 2013
From: Roelf.Schreurs at tibra.com (Roelf Schreurs)
Date: Wed, 9 Oct 2013 13:46:12 +0000
Subject: [Numpy-discussion] select column based on another column value
Message-ID: <39665C187B885C4296F1D80EC22699D680DD3E7B@LCINF16.tibra.com>

Hi

I have the following array and want to calculate the average per day.

Import numpy as np
from datetime import date

today =  date(today.year, today.month, 9)
yesterday = date(today.year, today.month, 8)
twodaysago = date(today.year, today.month, 7)

testdata = np.array([[today, "r", 3.2],[today, "r", 4.3],[today, "r", 5.3],[yesterday, "r", 6.3],[yesterday, "r", 9.3],[yesterday, "r", 1.3],[twodaysago, "r", 3.3],[twodaysago, "r", 1.2],[twodaysago, "r", 4.3]])


which produces
array([[datetime.date(2013, 10, 9), 'r', 3.2],
       [datetime.date(2013, 10, 9), 'r', 4.3],
       [datetime.date(2013, 10, 9), 'r', 5.3],
       [datetime.date(2013, 10, 8), 'r', 6.3],
       [datetime.date(2013, 10, 8), 'r', 9.3],
       [datetime.date(2013, 10, 8), 'r', 1.3],
       [datetime.date(2013, 10, 7), 'r', 3.3],
       [datetime.date(2013, 10, 7), 'r', 1.2],
       [datetime.date(2013, 10, 7), 'r', 4.3]], dtype=object)

And I want the output as (numbers aren't correct below):

2013-10-09, 4.54
2013-10-08, 5.43
2013-10-07, 2.76

But not sure how.
Thanks

Roelf



Tibra Trading Europe Limited is Authorised and Regulated by the FCA (Company No. 06061713)
Tibra Global Services Pty Limited London Branch (Company No. BR014370)
Tibra Equities Europe Limited (Company No. 07283912)
Registered in England & Wales - Level 12, 5 Aldermanbury Square London EC2V 7HR

Unless otherwise stated, this communication is neither an offer nor solicitation of an offer to sell or purchase any investment


The contents of this email including any attachments are confidential. If you have received this email in error, please advise the sender by return email and delete this email. Any unauthorised use of the contents of the email is prohibited and you must not disseminate, copy or distribute the message or use the information contained in the email or its attachments in any way.

The views or opinions expressed are the author's own and may not reflect the views or opinions of Tibra. Tibra does not guarantee the integrity of any emails or attached files. E-mails may be interfered with, may contain computer viruses or other defects. Under no circumstances do we accept liability for any loss or damage which may result from your receipt of this message or any attachments.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20131009/df2c50b5/attachment.html>

From charanpal at gmail.com  Wed Oct  9 10:04:09 2013
From: charanpal at gmail.com (Charanpal Dhanjal)
Date: Wed, 09 Oct 2013 16:04:09 +0200
Subject: [Numpy-discussion] Segfault with QR Decomposition
Message-ID: <52556259.7030003@gmail.com>

I get a segmentation fault upon running the following:

import numpy
A = numpy.ones((7000000, 8))
Q, R = numpy.linalg.qr(A)

on Python 2.7.3, Linux 64-bit using numpy 1.9.0.dev-ec3603f linked
against OpenBLAS. If A is a smaller matrix then the QR decomposition
works (for example A has shape (4000000, 8)). I haven't quite narrowed
down the exact threshold where the crash occurs, however I know that the
above A is 448 MB (Q and R are no bigger), and the machine in question
has 32GB of RAM. I also tested scipy.linalg.qr (version
0.14.0.dev-ced994c) with the same results.

I don't get the same problem on my laptop which is Python 2.7.3, Linux
64-bit but with numpy 1.8.0rc1 linked to OpenBLAS. Both machines have
OpenBLAS 0.2.6. Does anyone have some insight into why this problem is
occuring?

Thanks very much for any help,
Charanpal


From gexarcha1 at gmail.com  Wed Oct  9 10:22:12 2013
From: gexarcha1 at gmail.com (Georgios Exarchakis)
Date: Wed, 09 Oct 2013 07:22:12 -0700
Subject: [Numpy-discussion] select column based on another column value
In-Reply-To: <39665C187B885C4296F1D80EC22699D680DD3E7B@LCINF16.tibra.com>
References: <39665C187B885C4296F1D80EC22699D680DD3E7B@LCINF16.tibra.com>
Message-ID: <52556694.6070007@gmail.com>

I think you need this

 >>> testdata[testdata[:,0]==today][:,2].mean()
4.266666666666667

 >>> testdata[testdata[:,0]==yesterday][:,2].mean()
5.633333333333334

 >>> testdata[testdata[:,0]==twodaysago][:,2].mean()
2.9333333333333336


On 10/09/2013 06:46 AM, Roelf Schreurs wrote:
>
> Hi
>
> I have the following array and want to calculate the average per day.
>
> Import numpy as np
>
> from datetime import date
>
> today =  date(today.year, today.month, 9)
>
> yesterday = date(today.year, today.month, 8)
>
> twodaysago = date(today.year, today.month, 7)
>
> testdata = np.array([[today, "r", 3.2],[today, "r", 4.3],[today, "r", 
> 5.3],[yesterday, "r", 6.3],[yesterday, "r", 9.3],[yesterday, "r", 
> 1.3],[twodaysago, "r", 3.3],[twodaysago, "r", 1.2],[twodaysago, "r", 
> 4.3]])
>
> which produces
>
> array([[datetime.date(2013, 10, 9), 'r', 3.2],
>
>        [datetime.date(2013, 10, 9), 'r', 4.3],
>
>        [datetime.date(2013, 10, 9), 'r', 5.3],
>
>        [datetime.date(2013, 10, 8), 'r', 6.3],
>
>        [datetime.date(2013, 10, 8), 'r', 9.3],
>
>        [datetime.date(2013, 10, 8), 'r', 1.3],
>
>        [datetime.date(2013, 10, 7), 'r', 3.3],
>
>        [datetime.date(2013, 10, 7), 'r', 1.2],
>
>        [datetime.date(2013, 10, 7), 'r', 4.3]], dtype=object)
>
> And I want the output as (numbers aren't correct below):
>
> 2013-10-09, 4.54
>
> 2013-10-08, 5.43
>
> 2013-10-07, 2.76
>
> But not sure how.
>
> Thanks
>
> *Roelf *
>
>
>
> Tibra Trading Europe Limited is Authorised and Regulated by the FCA 
> (Company No. 06061713)
> Tibra Global Services Pty Limited London Branch (Company No. BR014370)
> Tibra Equities Europe Limited (Company No. 07283912)
> Registered in England & Wales - Level 12, 5 Aldermanbury Square London 
> EC2V 7HR
>
> Unless otherwise stated, this communication is neither an offer nor 
> solicitation of an offer to sell or purchase any investment
>
>
> The contents of this email including any attachments are confidential. 
> If you have received this email in error, please advise the sender by 
> return email and delete this email. Any unauthorised use of the 
> contents of the email is prohibited and you must not disseminate, copy 
> or distribute the message or use the information contained in the 
> email or its attachments in any way.
>
> The views or opinions expressed are the author's own and may not 
> reflect the views or opinions of Tibra. Tibra does not guarantee the 
> integrity of any emails or attached files. E-mails may be interfered 
> with, may contain computer viruses or other defects. Under no 
> circumstances do we accept liability for any loss or damage which may 
> result from your receipt of this message or any attachments.
>
>
> _______________________________________________
> NumPy-Discussion mailing list
> NumPy-Discussion at scipy.org
> http://mail.scipy.org/mailman/listinfo/numpy-discussion

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20131009/f361030e/attachment.html>

From Roelf.Schreurs at tibra.com  Wed Oct  9 10:31:17 2013
From: Roelf.Schreurs at tibra.com (Roelf Schreurs)
Date: Wed, 9 Oct 2013 14:31:17 +0000
Subject: [Numpy-discussion] select column based on another column value
In-Reply-To: <52556694.6070007@gmail.com>
References: <39665C187B885C4296F1D80EC22699D680DD3E7B@LCINF16.tibra.com>
	<52556694.6070007@gmail.com>
Message-ID: <39665C187B885C4296F1D80EC22699D680DD4202@LCINF16.tibra.com>

That does work thanks.
And in a for loop was my goal, so this works for me.

for i in np.unique(testdata[:,0]):
    print(i, testdata[testdata[:,0]==i][:,2].mean())

(datetime.date(2013, 10, 7), 2.9333333333333336)
(datetime.date(2013, 10, 8), 5.633333333333334)
(datetime.date(2013, 10, 9), 4.266666666666667)

Roelf

From: numpy-discussion-bounces at scipy.org [mailto:numpy-discussion-bounces at scipy.org] On Behalf Of Georgios Exarchakis
Sent: 09 October 2013 15:22
To: Discussion of Numerical Python
Subject: Re: [Numpy-discussion] select column based on another column value

I think you need this

>>> testdata[testdata[:,0]==today][:,2].mean()
4.266666666666667

>>> testdata[testdata[:,0]==yesterday][:,2].mean()
5.633333333333334

>>> testdata[testdata[:,0]==twodaysago][:,2].mean()
2.9333333333333336

On 10/09/2013 06:46 AM, Roelf Schreurs wrote:
Hi

I have the following array and want to calculate the average per day.

Import numpy as np
from datetime import date

today =  date(today.year, today.month, 9)
yesterday = date(today.year, today.month, 8)
twodaysago = date(today.year, today.month, 7)

testdata = np.array([[today, "r", 3.2],[today, "r", 4.3],[today, "r", 5.3],[yesterday, "r", 6.3],[yesterday, "r", 9.3],[yesterday, "r", 1.3],[twodaysago, "r", 3.3],[twodaysago, "r", 1.2],[twodaysago, "r", 4.3]])


which produces
array([[datetime.date(2013, 10, 9), 'r', 3.2],
       [datetime.date(2013, 10, 9), 'r', 4.3],
       [datetime.date(2013, 10, 9), 'r', 5.3],
       [datetime.date(2013, 10, 8), 'r', 6.3],
       [datetime.date(2013, 10, 8), 'r', 9.3],
       [datetime.date(2013, 10, 8), 'r', 1.3],
       [datetime.date(2013, 10, 7), 'r', 3.3],
       [datetime.date(2013, 10, 7), 'r', 1.2],
       [datetime.date(2013, 10, 7), 'r', 4.3]], dtype=object)

And I want the output as (numbers aren't correct below):

2013-10-09, 4.54
2013-10-08, 5.43
2013-10-07, 2.76

But not sure how.
Thanks

Roelf



Tibra Trading Europe Limited is Authorised and Regulated by the FCA (Company No. 06061713)
Tibra Global Services Pty Limited London Branch (Company No. BR014370)
Tibra Equities Europe Limited (Company No. 07283912)
Registered in England & Wales - Level 12, 5 Aldermanbury Square London EC2V 7HR

Unless otherwise stated, this communication is neither an offer nor solicitation of an offer to sell or purchase any investment


The contents of this email including any attachments are confidential. If you have received this email in error, please advise the sender by return email and delete this email. Any unauthorised use of the contents of the email is prohibited and you must not disseminate, copy or distribute the message or use the information contained in the email or its attachments in any way.

The views or opinions expressed are the author's own and may not reflect the views or opinions of Tibra. Tibra does not guarantee the integrity of any emails or attached files. E-mails may be interfered with, may contain computer viruses or other defects. Under no circumstances do we accept liability for any loss or damage which may result from your receipt of this message or any attachments.




_______________________________________________

NumPy-Discussion mailing list

NumPy-Discussion at scipy.org<mailto:NumPy-Discussion at scipy.org>

http://mail.scipy.org/mailman/listinfo/numpy-discussion



Tibra Trading Europe Limited is Authorised and Regulated by the FCA (Company No. 06061713)
Tibra Global Services Pty Limited London Branch (Company No. BR014370)
Tibra Equities Europe Limited (Company No. 07283912)
Registered in England & Wales - Level 12, 5 Aldermanbury Square London EC2V 7HR

Unless otherwise stated, this communication is neither an offer nor solicitation of an offer to sell or purchase any investment


The contents of this email including any attachments are confidential. If you have received this email in error, please advise the sender by return email and delete this email. Any unauthorised use of the contents of the email is prohibited and you must not disseminate, copy or distribute the message or use the information contained in the email or its attachments in any way.

The views or opinions expressed are the author's own and may not reflect the views or opinions of Tibra. Tibra does not guarantee the integrity of any emails or attached files. E-mails may be interfered with, may contain computer viruses or other defects. Under no circumstances do we accept liability for any loss or damage which may result from your receipt of this message or any attachments.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20131009/f2549685/attachment.html>

From charlesr.harris at gmail.com  Wed Oct  9 10:40:54 2013
From: charlesr.harris at gmail.com (Charles R Harris)
Date: Wed, 9 Oct 2013 08:40:54 -0600
Subject: [Numpy-discussion] Segfault with QR Decomposition
In-Reply-To: <52556259.7030003@gmail.com>
References: <52556259.7030003@gmail.com>
Message-ID: <CAB6mnx+aHWPbUfsUi0NP5ZeUoi8=A1pBg8vJP5Gr1T_oEb6n2A@mail.gmail.com>

On Wed, Oct 9, 2013 at 8:04 AM, Charanpal Dhanjal <charanpal at gmail.com>wrote:

> I get a segmentation fault upon running the following:
>
> import numpy
> A = numpy.ones((7000000, 8))
> Q, R = numpy.linalg.qr(A)
>
> on Python 2.7.3, Linux 64-bit using numpy 1.9.0.dev-ec3603f linked
> against OpenBLAS. If A is a smaller matrix then the QR decomposition
> works (for example A has shape (4000000, 8)). I haven't quite narrowed
> down the exact threshold where the crash occurs, however I know that the
> above A is 448 MB (Q and R are no bigger), and the machine in question
> has 32GB of RAM. I also tested scipy.linalg.qr (version
> 0.14.0.dev-ced994c) with the same results.
>
> I don't get the same problem on my laptop which is Python 2.7.3, Linux
> 64-bit but with numpy 1.8.0rc1 linked to OpenBLAS. Both machines have
> OpenBLAS 0.2.6. Does anyone have some insight into why this problem is
> occuring?
>
>
Works fine here with '1.9.0.dev-7457f15' linked against ATLAS. I suspect
problem is in OpenBLAS. What architecture/os do the two machines have? Is
OpenBLAS using more than one core?

Chuck
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20131009/860e7ac0/attachment.html>

From charanpal at gmail.com  Wed Oct  9 11:02:57 2013
From: charanpal at gmail.com (Charanpal Dhanjal)
Date: Wed, 09 Oct 2013 17:02:57 +0200
Subject: [Numpy-discussion]  Segfault with QR Decomposition
References: 52556259.7030003@gmail.com
Message-ID: <52557021.5060103@gmail.com>

> >/ I get a segmentation fault upon running the following:
> />/
> />/ import numpy
> />/ A = numpy.ones((7000000, 8))
> />/ Q, R = numpy.linalg.qr(A)
> />/
> />/ on Python 2.7.3, Linux 64-bit using numpy 1.9.0.dev-ec3603f linked
> />/ against OpenBLAS. If A is a smaller matrix then the QR decomposition
> />/ works (for example A has shape (4000000, 8)). I haven't quite narrowed
> />/ down the exact threshold where the crash occurs, however I know that the
> />/ above A is 448 MB (Q and R are no bigger), and the machine in question
> />/ has 32GB of RAM. I also tested scipy.linalg.qr (version
> />/ 0.14.0.dev-ced994c) with the same results.
> />/
> />/ I don't get the same problem on my laptop which is Python 2.7.3, Linux
> />/ 64-bit but with numpy 1.8.0rc1 linked to OpenBLAS. Both machines have
> />/ OpenBLAS 0.2.6. Does anyone have some insight into why this problem is
> />/ occuring?
> />/
> />/
> /Works fine here with '1.9.0.dev-7457f15' linked against ATLAS. I suspect
> problem is in OpenBLAS. What architecture/os do the two machines have? Is
> OpenBLAS using more than one core?

Thanks for testing the code. On the desktop machine I am using Debian GNU/Linux 7.2 (wheezy) compiled for x86-64 (Intel(R) Xeon(R) CPU E5-1620), and on my laptop I use Ubuntu 13.04 for x86-64 (Intel(R) Core(TM) i7-3740QM). I tried: 

export OPENBLAS_NUM_THREADS=1

and 

export OPENBLAS_NUM_THREADS=1

with the same results (a segfault). 

Charanpal 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20131009/1b0ce628/attachment.html>

From charanpal at gmail.com  Wed Oct  9 11:05:35 2013
From: charanpal at gmail.com (Charanpal Dhanjal)
Date: Wed, 09 Oct 2013 17:05:35 +0200
Subject: [Numpy-discussion]   Segfault with QR Decomposition
Message-ID: <525570BF.7020509@gmail.com>

Oops, the second line should have been

export OPENBLAS_NUM_THREADS=8



From jtaylor.debian at googlemail.com  Wed Oct  9 11:24:30 2013
From: jtaylor.debian at googlemail.com (Julian Taylor)
Date: Wed, 9 Oct 2013 17:24:30 +0200
Subject: [Numpy-discussion] Segfault with QR Decomposition
In-Reply-To: <525570BF.7020509@gmail.com>
References: <525570BF.7020509@gmail.com>
Message-ID: <CAK5FAtGMGzm7VvaO=dhKTQJxTE8s+uzisz151CdiVJp5fDrOQQ@mail.gmail.com>

yes thats probably openblas fault.
Openblas crashes all the time as soon as your matrices get bigger than a
couple of megabytes.
I'll investigate and report it upstream (as I have already far too often
for the exact same reason ...)


On Wed, Oct 9, 2013 at 5:05 PM, Charanpal Dhanjal <charanpal at gmail.com>wrote:

> Oops, the second line should have been
>
> export OPENBLAS_NUM_THREADS=8
>
> _______________________________________________
> NumPy-Discussion mailing list
> NumPy-Discussion at scipy.org
> http://mail.scipy.org/mailman/listinfo/numpy-discussion
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20131009/2b18b558/attachment.html>

From jtaylor.debian at googlemail.com  Wed Oct  9 13:21:51 2013
From: jtaylor.debian at googlemail.com (Julian Taylor)
Date: Wed, 09 Oct 2013 19:21:51 +0200
Subject: [Numpy-discussion] Segfault with QR Decomposition
In-Reply-To: <CAK5FAtGMGzm7VvaO=dhKTQJxTE8s+uzisz151CdiVJp5fDrOQQ@mail.gmail.com>
References: <525570BF.7020509@gmail.com>
	<CAK5FAtGMGzm7VvaO=dhKTQJxTE8s+uzisz151CdiVJp5fDrOQQ@mail.gmail.com>
Message-ID: <525590AF.4090505@googlemail.com>

https://github.com/xianyi/OpenBLAS/issues/304

On 09.10.2013 17:24, Julian Taylor wrote:
> yes thats probably openblas fault.
> Openblas crashes all the time as soon as your matrices get bigger than a
> couple of megabytes.
> I'll investigate and report it upstream (as I have already far too often
> for the exact same reason ...)
> 
> 
> On Wed, Oct 9, 2013 at 5:05 PM, Charanpal Dhanjal <charanpal at gmail.com
> <mailto:charanpal at gmail.com>> wrote:
> 
>     Oops, the second line should have been
> 
>     export OPENBLAS_NUM_THREADS=8
> 
>     _______________________________________________
>     NumPy-Discussion mailing list
>     NumPy-Discussion at scipy.org <mailto:NumPy-Discussion at scipy.org>
>     http://mail.scipy.org/mailman/listinfo/numpy-discussion
> 
> 
> 
> 
> _______________________________________________
> NumPy-Discussion mailing list
> NumPy-Discussion at scipy.org
> http://mail.scipy.org/mailman/listinfo/numpy-discussion
> 



From Bernhard.Spinnler at gmx.net  Wed Oct  9 14:48:41 2013
From: Bernhard.Spinnler at gmx.net (Bernhard Spinnler)
Date: Wed, 9 Oct 2013 20:48:41 +0200
Subject: [Numpy-discussion] Bug in numpy.correlate documentation
In-Reply-To: <CAP=RS9k54vtNFHy9ppG=U09oEHwB=KLV0xvwR6BfFgB3o5S=fw@mail.gmail.com>
References: <847A3113-2EF8-4AE6-84DC-95AB2D9EFEDB@gmx.net>
	<CAP=RS9k54vtNFHy9ppG=U09oEHwB=KLV0xvwR6BfFgB3o5S=fw@mail.gmail.com>
Message-ID: <94AEB34E-F6D7-4833-BE0D-707728481A90@gmx.net>

Hi Richard,

Ah, I searched the list but didn't find those posts before?

I can easily imagine that correlation is defined differently in different disciplines. Both ways are correct and it's just a convention or definition. In my field (Digital Communications, Digital Signal Processing) the vast majority uses the convention implemented by the code. Here are a few examples of prominent text books:

- Papoulis, "Probaility, Random Variables, and Stochastic Processes", McGraw-Hill, 2nd ed.
- Benvenuto, Cherubini, "Algorithms for Communications Systems and their Applications", Wiley.
- Carlson, "Communication Systems" 4th ed. 2002, McGraw-Hill.

Last not least, Matlab's xcorr() function behaves exactly like correlate() does right now, see
- http://www.mathworks.de/de/help/signal/ref/xcorr.html

But, as you say, the most important aspect might be, that most people will probably prefer changing the docs instead of changing the code.

Should I file a bug somewhere?

Cheers,
	Bernhard


On 08.10.2013, at 21:10, Richard Hattersley <rhattersley at gmail.com> wrote:

> Hi Bernard,
> 
> Looks like you're on to something - two other people have raised this discrepancy before: https://github.com/numpy/numpy/issues/2588. Unfortunately, when it comes to resolving the discrepancy one of the previous comments takes the opposite view. Namely, that the docstring is correct and the code is wrong.
> 
> Do different domains use different conventions here? Are there some references to back up one stance or another?
> 
> But all else being equal, I'm guessing there'll be far more appetite for updating the documentation than the code.
> 
> Regards,
> Richard Hattersley
> 
> 
> On 7 October 2013 22:09, Bernhard Spinnler <Bernhard.Spinnler at gmx.net> wrote:
> The numpy.correlate documentation says:
> 
>         correlate(a, v) = z[k] = sum_n a[n] * conj(v[n+k])
> 
> In [1]: a = [1, 2]
> 
> In [2]: v = [2, 1j]
> 
> In [3]: z = correlate(a, v, 'full')
> 
> In [4]: z
> Out[4]: array([ 0.-1.j,  2.-2.j,  4.+0.j])
> 
> However, according to the documentation, z should be
> 
>         z[-1] = a[1] * conj(v[0]) = 4.+0.j
>         z[0]  = a[0] * conj(v[0]) + a[1] * conj(v[1]) = 2.-2.j
>         z[1] = a[0] * conj(v[1]) = 0.-1.j
> 
> which is the time reversed version of what correlate() calculates.
> 
> IMHO, the correlate() code is correct. The correct formula in the docs (which is also the correlation formula in standard text books) should be
> 
>         z[k] = sum_n a[n+k] * conj(v[n])
> 
> Cheers,
>         Bernhard
> _______________________________________________
> NumPy-Discussion mailing list
> NumPy-Discussion at scipy.org
> http://mail.scipy.org/mailman/listinfo/numpy-discussion
> 
> _______________________________________________
> NumPy-Discussion mailing list
> NumPy-Discussion at scipy.org
> http://mail.scipy.org/mailman/listinfo/numpy-discussion

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20131009/38b19747/attachment.html>

From d.l.goldsmith at gmail.com  Wed Oct  9 16:19:05 2013
From: d.l.goldsmith at gmail.com (David Goldsmith)
Date: Wed, 9 Oct 2013 13:19:05 -0700
Subject: [Numpy-discussion] Bug in numpy.correlate documentation
Message-ID: <CAFtPsZr9itPhLK34iXrOEND3b8dCVygbCUtNYp+s_H_iw_9uEg@mail.gmail.com>

Looks like Wolfram MathWorld would favor the docstring, but the possibility
of a "use-domain" dependency seems plausible (after all, a similar dilemma
is observed, e.g., w/ the Fourier Transform)--I guess one discipline's
future is another discipline's past. :-)

http://mathworld.wolfram.com/Autocorrelation.html

DG

Date: Tue, 8 Oct 2013 20:10:41 +0100

> From: Richard Hattersley <rhattersley at gmail.com>
> Subject: Re: [Numpy-discussion] Bug in numpy.correlate documentation
> To: Discussion of Numerical Python <numpy-discussion at scipy.org>
> Message-ID:
>         <CAP=RS9k54vtNFHy9ppG=U09oEHwB=KLV0xvwR6BfFgB3o5S=
> fw at mail.gmail.com>
> Content-Type: text/plain; charset="iso-8859-1"
>
> Hi Bernard,
>
> Looks like you're on to something - two other people have raised this
> discrepancy before: https://github.com/numpy/numpy/issues/2588.
> Unfortunately, when it comes to resolving the discrepancy one of the
> previous comments takes the opposite view. Namely, that the docstring is
> correct and the code is wrong.
>
> Do different domains use different conventions here? Are there some
> references to back up one stance or another?
>
> But all else being equal, I'm guessing there'll be far more appetite for
> updating the documentation than the code.
>
> Regards,
> Richard Hattersley
>
>
> On 7 October 2013 22:09, Bernhard Spinnler <Bernhard.Spinnler at gmx.net
> >wrote:
>
> > The numpy.correlate documentation says:
> >
> >         correlate(a, v) = z[k] = sum_n a[n] * conj(v[n+k])
> >
>
<snip>

> > [so] according to the documentation, z should be
> >
> >         z[-1] = a[1] * conj(v[0]) = 4.+0.j
> >         z[0]  = a[0] * conj(v[0]) + a[1] * conj(v[1]) = 2.-2.j
> >         z[1] = a[0] * conj(v[1]) = 0.-1.j
> >
> > which is the time reversed version of what correlate() calculates.
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20131009/3604d98c/attachment.html>

From njs at pobox.com  Wed Oct  9 16:54:07 2013
From: njs at pobox.com (Nathaniel Smith)
Date: Wed, 9 Oct 2013 21:54:07 +0100
Subject: [Numpy-discussion] Bug in numpy.correlate documentation
In-Reply-To: <94AEB34E-F6D7-4833-BE0D-707728481A90@gmx.net>
References: <847A3113-2EF8-4AE6-84DC-95AB2D9EFEDB@gmx.net>
	<CAP=RS9k54vtNFHy9ppG=U09oEHwB=KLV0xvwR6BfFgB3o5S=fw@mail.gmail.com>
	<94AEB34E-F6D7-4833-BE0D-707728481A90@gmx.net>
Message-ID: <CAPJVwBmMZKN=z8V-aHUU+85LZ88xYWmAwxgzHk5GhtfuW8HN9A@mail.gmail.com>

On Wed, Oct 9, 2013 at 7:48 PM, Bernhard Spinnler
<Bernhard.Spinnler at gmx.net> wrote:
> Hi Richard,
>
> Ah, I searched the list but didn't find those posts before?
>
> I can easily imagine that correlation is defined differently in different
> disciplines. Both ways are correct and it's just a convention or definition.
> In my field (Digital Communications, Digital Signal Processing) the vast
> majority uses the convention implemented by the code. Here are a few
> examples of prominent text books:
>
> - Papoulis, "Probaility, Random Variables, and Stochastic Processes",
> McGraw-Hill, 2nd ed.
> - Benvenuto, Cherubini, "Algorithms for Communications Systems and their
> Applications", Wiley.
> - Carlson, "Communication Systems" 4th ed. 2002, McGraw-Hill.
>
> Last not least, Matlab's xcorr() function behaves exactly like correlate()
> does right now, see
> - http://www.mathworks.de/de/help/signal/ref/xcorr.html
>
> But, as you say, the most important aspect might be, that most people will
> probably prefer changing the docs instead of changing the code.

Yeah, unless the current behaviour is actually broken or redundant in
some way, we're not going to switch from one perfectly good convention
to another perfectly good convention and break everyone's code in the
process.

The most helpful thing would be if you could file a pull request that
just changes the docstring to what you think it should be. Extra bonus
points if it points out that there is another definition some people
might be expecting instead, and explains how those people can use the
existing functions to get what they want. :-)

-n


From lists at hilboll.de  Thu Oct 10 08:43:25 2013
From: lists at hilboll.de (Andreas Hilboll)
Date: Thu, 10 Oct 2013 14:43:25 +0200
Subject: [Numpy-discussion] Reading from binary file with memmap, with offset
Message-ID: <5256A0ED.8070508@hilboll.de>

Hi,

I have a problem using memmap correctly. I need to read a data file
which consists of an ASCII header and appended binary single precision
floating point values. memmap complains that the "Size of available data
is not a multiple of the data-type size." But as far as I can tell, the
size *doas* match the data-type size.

The problem is illustrated by the following code:

---8<-------

url = "http://www.iup.uni-bremen.de/~hilboll/download/20120204.XD4_N2"
localfile = "np_memmap.dat"

import os
import urllib

import numpy as np


# download data file
if not os.access(localfile, os.R_OK):
  urllib.urlretrieve(url, localfile)

with open(localfile, "r") as fd:
  # read offset from first line of file
  offset = int(fd.readlines()[0].split()[-2])
  # jump to begin of data block
  fd.seek(offset)
  # read until EOF
  blob = fd.read()
  print("Size of data blob [bytes]: {}".format(len(blob)))
  print("This can actually be divided by 4: {} / 4.0 = {}".format(
            len(blob), len(blob) / 4.0))
  # go back to begin of data block
  fd.seek(offset)
  print("But it cannot be loaded as np.memmap with dtype float32:")
  np.memmap(fd, dtype="float32", mode="r")

---8<-------

Any help is greatly appreciated :)

-- Andreas.


From jtaylor.debian at googlemail.com  Thu Oct 10 08:57:39 2013
From: jtaylor.debian at googlemail.com (Julian Taylor)
Date: Thu, 10 Oct 2013 14:57:39 +0200
Subject: [Numpy-discussion] Reading from binary file with memmap,
	with offset
In-Reply-To: <5256A0ED.8070508@hilboll.de>
References: <5256A0ED.8070508@hilboll.de>
Message-ID: <CAK5FAtGmYnktk0NHfRfzN3v0k3pj2LjXgR1oa-nYus1oDa0xog@mail.gmail.com>

you have to use the offset keyword argument of np.memmap, else it will
always start from the beginning of the file

>>> np.memmap(fd, dtype="float32", mode="r", offset=offset)


On Thu, Oct 10, 2013 at 2:43 PM, Andreas Hilboll <lists at hilboll.de> wrote:

> Hi,
>
> I have a problem using memmap correctly. I need to read a data file
> which consists of an ASCII header and appended binary single precision
> floating point values. memmap complains that the "Size of available data
> is not a multiple of the data-type size." But as far as I can tell, the
> size *doas* match the data-type size.
>
> The problem is illustrated by the following code:
>
> ---8<-------
>
> url = "http://www.iup.uni-bremen.de/~hilboll/download/20120204.XD4_N2"
> localfile = "np_memmap.dat"
>
> import os
> import urllib
>
> import numpy as np
>
>
> # download data file
> if not os.access(localfile, os.R_OK):
>   urllib.urlretrieve(url, localfile)
>
> with open(localfile, "r") as fd:
>   # read offset from first line of file
>   offset = int(fd.readlines()[0].split()[-2])
>   # jump to begin of data block
>   fd.seek(offset)
>   # read until EOF
>   blob = fd.read()
>   print("Size of data blob [bytes]: {}".format(len(blob)))
>   print("This can actually be divided by 4: {} / 4.0 = {}".format(
>             len(blob), len(blob) / 4.0))
>   # go back to begin of data block
>   fd.seek(offset)
>   print("But it cannot be loaded as np.memmap with dtype float32:")
>   np.memmap(fd, dtype="float32", mode="r")
>
> ---8<-------
>
> Any help is greatly appreciated :)
>
> -- Andreas.
> _______________________________________________
> NumPy-Discussion mailing list
> NumPy-Discussion at scipy.org
> http://mail.scipy.org/mailman/listinfo/numpy-discussion
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20131010/581b333d/attachment.html>

From joferkington at gmail.com  Thu Oct 10 09:07:58 2013
From: joferkington at gmail.com (Joe Kington)
Date: Thu, 10 Oct 2013 08:07:58 -0500
Subject: [Numpy-discussion] Reading from binary file with memmap,
	with offset
In-Reply-To: <5256A0ED.8070508@hilboll.de>
References: <5256A0ED.8070508@hilboll.de>
Message-ID: <CACe1pJdfBnyC3J82C=7kUFrwgX05=NEH=aO_CCB3tvvFidYkiA@mail.gmail.com>

You just need to supply the "offset" kwarg to "memmap".

for example:

with open(localfile, "r") as fd:
  # read offset from first line of file
  offset = int(next(fd).split()[-2])
  np.memmap(fd, dtype="float32", mode="r", offset=offset)

Also, there's no need to do things like "offset =
int(fd.readlines()[0].split()[-2])"

Just do "offset = int(next(fd).split()[-2])" instead.  Readlines reads the
entire file into memory. You only want the first line.

Hope that helps!
-Joe


On Thu, Oct 10, 2013 at 7:43 AM, Andreas Hilboll <lists at hilboll.de> wrote:

> Hi,
>
> I have a problem using memmap correctly. I need to read a data file
> which consists of an ASCII header and appended binary single precision
> floating point values. memmap complains that the "Size of available data
> is not a multiple of the data-type size." But as far as I can tell, the
> size *doas* match the data-type size.
>
> The problem is illustrated by the following code:
>
> ---8<-------
>
> url = "http://www.iup.uni-bremen.de/~hilboll/download/20120204.XD4_N2"
> localfile = "np_memmap.dat"
>
> import os
> import urllib
>
> import numpy as np
>
>
> # download data file
> if not os.access(localfile, os.R_OK):
>   urllib.urlretrieve(url, localfile)
>
> with open(localfile, "r") as fd:
>   # read offset from first line of file
>   offset = int(fd.readlines()[0].split()[-2])
>   # jump to begin of data block
>   fd.seek(offset)
>   # read until EOF
>   blob = fd.read()
>   print("Size of data blob [bytes]: {}".format(len(blob)))
>   print("This can actually be divided by 4: {} / 4.0 = {}".format(
>             len(blob), len(blob) / 4.0))
>   # go back to begin of data block
>   fd.seek(offset)
>   print("But it cannot be loaded as np.memmap with dtype float32:")
>   np.memmap(fd, dtype="float32", mode="r")
>
> ---8<-------
>
> Any help is greatly appreciated :)
>
> -- Andreas.
> _______________________________________________
> NumPy-Discussion mailing list
> NumPy-Discussion at scipy.org
> http://mail.scipy.org/mailman/listinfo/numpy-discussion
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20131010/1797d962/attachment.html>

From lists at hilboll.de  Thu Oct 10 09:16:42 2013
From: lists at hilboll.de (Andreas Hilboll)
Date: Thu, 10 Oct 2013 15:16:42 +0200
Subject: [Numpy-discussion] Reading from binary file with memmap,
 with offset
In-Reply-To: <CAK5FAtGmYnktk0NHfRfzN3v0k3pj2LjXgR1oa-nYus1oDa0xog@mail.gmail.com>
References: <5256A0ED.8070508@hilboll.de>
	<CAK5FAtGmYnktk0NHfRfzN3v0k3pj2LjXgR1oa-nYus1oDa0xog@mail.gmail.com>
Message-ID: <5256A8BA.9050203@hilboll.de>

On 10.10.2013 14:57, Julian Taylor wrote:
> you have to use the offset keyword argument of np.memmap, else it will
> always start from the beginning of the file
> 
>>>> np.memmap(fd, dtype="float32", mode="r", offset=offset)

Thanks for that! I just proposed an update to the memmap docstring to
better reflect this:

   https://github.com/numpy/numpy/pull/3890

-- Andreas.


From lists at hilboll.de  Thu Oct 10 09:17:26 2013
From: lists at hilboll.de (Andreas Hilboll)
Date: Thu, 10 Oct 2013 15:17:26 +0200
Subject: [Numpy-discussion] Reading from binary file with memmap,
 with offset
In-Reply-To: <CACe1pJdfBnyC3J82C=7kUFrwgX05=NEH=aO_CCB3tvvFidYkiA@mail.gmail.com>
References: <5256A0ED.8070508@hilboll.de>
	<CACe1pJdfBnyC3J82C=7kUFrwgX05=NEH=aO_CCB3tvvFidYkiA@mail.gmail.com>
Message-ID: <5256A8E6.10601@hilboll.de>

On 10.10.2013 15:07, Joe Kington wrote:
> You just need to supply the "offset" kwarg to "memmap".
> 
> for example:
> 
> with open(localfile, "r") as fd:
>   # read offset from first line of file
>   offset = int(next(fd).split()[-2])
>   np.memmap(fd, dtype="float32", mode="r", offset=offset)
> 
> Also, there's no need to do things like "offset =
> int(fd.readlines()[0].split()[-2])"
> 
> Just do "offset = int(next(fd).split()[-2])" instead.  Readlines reads
> the entire file into memory. You only want the first line.

Thanks, Joe, I actually didn't know `next` yet ...

-- Andreas.


From sunk.cs at gmail.com  Thu Oct 10 10:00:47 2013
From: sunk.cs at gmail.com (Ke Sun)
Date: Thu, 10 Oct 2013 16:00:47 +0200
Subject: [Numpy-discussion] Efficient square distance computation
In-Reply-To: <CAK5FAtHn_xoK935uZs_jEWanvaqkR3yKXGW9Po0Mh8OkzJSxRw@mail.gmail.com>
References: <20131008080625.GA26473@probe.unige.ch>
	<CAH6Pt5otBzcm8v2z357Tme2a4RK=qo7YetWRMosxHM+UtnpumA@mail.gmail.com>
	<20131008113828.GA31484@probe.unige.ch>
	<CAK5FAtHn_xoK935uZs_jEWanvaqkR3yKXGW9Po0Mh8OkzJSxRw@mail.gmail.com>
Message-ID: <20131010140047.GA5067@probe.unige.ch>

> > > On a machine I had access to it took about 20 minutes.
> > How? I am using matrix multiplication (the same code as
> > http://stackoverflow.com/a/4856692) and it runs for around 18 hours.
> >
> >
> make sure you are using an optimized BLAS library.
> e.g. with openblas I could also do this size in about 100 seconds wall time
> (20 minutes cpu time).

I recompiled numpy+scipy with intel compiler and MKL. Now it takes 
around 2 MINUTES on my machine. Horay! Thanks!

Best,
Ke Sun


From d.l.goldsmith at gmail.com  Thu Oct 10 13:27:41 2013
From: d.l.goldsmith at gmail.com (David Goldsmith)
Date: Thu, 10 Oct 2013 10:27:41 -0700
Subject: [Numpy-discussion] Bug in numpy.correlate documentation
Message-ID: <CAFtPsZqdvzoN1NyXTr+MTLJ+a8sG1G3KxzOLtFi2FORWf8+bCQ@mail.gmail.com>

Date: Wed, 9 Oct 2013 21:54:07 +0100

> From: Nathaniel Smith <njs at pobox.com>
> Subject: Re: [Numpy-discussion] Bug in numpy.correlate documentation
> To: Discussion of Numerical Python <numpy-discussion at scipy.org>
> Message-ID:
>         <CAPJVwBmMZKN=
> z8V-aHUU+85LZ88xYWmAwxgzHk5GhtfuW8HN9A at mail.gmail.com>
> Content-Type: text/plain; charset=UTF-8
>
> On Wed, Oct 9, 2013 at 7:48 PM, Bernhard Spinnler
> <Bernhard.Spinnler at gmx.net> wrote:
> > Hi Richard,
> >
> > Ah, I searched the list but didn't find those posts before?
> >
> > I can easily imagine that correlation is defined differently in different
> > disciplines. Both ways are correct and it's just a convention or
> definition.
> > In my field (Digital Communications, Digital Signal Processing) the vast
> > majority uses the convention implemented by the code. Here are a few
> > examples of prominent text books:
> >
> > - Papoulis, "Probaility, Random Variables, and Stochastic Processes",
> > McGraw-Hill, 2nd ed.
> > - Benvenuto, Cherubini, "Algorithms for Communications Systems and their
> > Applications", Wiley.
> > - Carlson, "Communication Systems" 4th ed. 2002, McGraw-Hill.
> >
> > Last not least, Matlab's xcorr() function behaves exactly like
> correlate()
> > does right now, see
> > - http://www.mathworks.de/de/help/signal/ref/xcorr.html
> >
> > But, as you say, the most important aspect might be, that most people
> will
> > probably prefer changing the docs instead of changing the code.
>
> Yeah, unless the current behaviour is actually broken or redundant in
> some way, we're not going to switch from one perfectly good convention
> to another perfectly good convention and break everyone's code in the
> process.
>
> The most helpful thing would be if you could file a pull request that
> just changes the docstring to what you think it should be. Extra bonus
> points if it points out that there is another definition some people
> might be expecting instead, and explains how those people can use the
> existing functions to get what they want. :-)
>
> -n
>

IMHO, "point[ing] out that there is another definition some people might be
expecting instead, and explain[ing] how those people can use the existing
functions to get what they want" should be a requirement for the docstring
("Notes" section), not merely worth "extra bonus points."  But then I'm
not, presently, in a position to edit the docstring myself, so that's just
MHO.

IAE, I found what appears to me to be another "vote" for the extant
docstring: Box & Jenkins, 1976, "Time Series Analysis: Forecasting and
Control," Holden-Day, Oakland, pg. 374.  Perhaps a "switch" (with a default
value that maintains current definition, so that extant uses would not
require a code change) c/should be added to the function signature so that
users can get easily get what they want?

DG
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20131010/57f83d6c/attachment.html>

From Bernhard.Spinnler at gmx.net  Thu Oct 10 15:12:08 2013
From: Bernhard.Spinnler at gmx.net (Bernhard Spinnler)
Date: Thu, 10 Oct 2013 21:12:08 +0200
Subject: [Numpy-discussion] Bug in numpy.correlate documentation
In-Reply-To: <CAFtPsZr9itPhLK34iXrOEND3b8dCVygbCUtNYp+s_H_iw_9uEg@mail.gmail.com>
References: <CAFtPsZr9itPhLK34iXrOEND3b8dCVygbCUtNYp+s_H_iw_9uEg@mail.gmail.com>
Message-ID: <39E54564-4A84-4887-8627-2DC93C4D2143@gmx.net>

It seems to me that Wolfram is following yet another path. From http://mathworld.wolfram.com/Autocorrelation.html and more importantly http://mathworld.wolfram.com/Cross-Correlation.html, equation (5):

z_mathworld[k] = sum_n conj(a[n]) * v[n+k] 
	= conj( sum_n a[n] * conj(v[n+k]) )
	= conj( z_numpyDocstring[k] )
	= conj( z_numpyCode[-k] )

is the conjugate of what the numpy docstring says. So, now we have at least three definitions to chose from :-)

Cheers,
	Bernhard

On 09.10.2013, at 22:19, David Goldsmith <d.l.goldsmith at gmail.com> wrote:

> Looks like Wolfram MathWorld would favor the docstring, but the possibility of a "use-domain" dependency seems plausible (after all, a similar dilemma is observed, e.g., w/ the Fourier Transform)--I guess one discipline's future is another discipline's past. :-)
> 
> http://mathworld.wolfram.com/Autocorrelation.html
> 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20131010/a02984a2/attachment.html>

From Bernhard.Spinnler at gmx.net  Thu Oct 10 15:31:12 2013
From: Bernhard.Spinnler at gmx.net (Bernhard Spinnler)
Date: Thu, 10 Oct 2013 21:31:12 +0200
Subject: [Numpy-discussion] Bug in numpy.correlate documentation
In-Reply-To: <CAFtPsZqdvzoN1NyXTr+MTLJ+a8sG1G3KxzOLtFi2FORWf8+bCQ@mail.gmail.com>
References: <CAFtPsZqdvzoN1NyXTr+MTLJ+a8sG1G3KxzOLtFi2FORWf8+bCQ@mail.gmail.com>
Message-ID: <09928B8C-1962-42A4-8566-297A2B07C1B1@gmx.net>


On 10.10.2013, at 19:27, David Goldsmith <d.l.goldsmith at gmail.com> wrote:
> On Wed, Oct 9, 2013 at 7:48 PM, Bernhard Spinnler
> <Bernhard.Spinnler at gmx.net> wrote:
> > Hi Richard,
> >
> > Ah, I searched the list but didn't find those posts before?
> >
> > I can easily imagine that correlation is defined differently in different
> > disciplines. Both ways are correct and it's just a convention or definition.
> > In my field (Digital Communications, Digital Signal Processing) the vast
> > majority uses the convention implemented by the code. Here are a few
> > examples of prominent text books:
> >
> > - Papoulis, "Probaility, Random Variables, and Stochastic Processes",
> > McGraw-Hill, 2nd ed.
> > - Benvenuto, Cherubini, "Algorithms for Communications Systems and their
> > Applications", Wiley.
> > - Carlson, "Communication Systems" 4th ed. 2002, McGraw-Hill.
> >
> > Last not least, Matlab's xcorr() function behaves exactly like correlate()
> > does right now, see
> > - http://www.mathworks.de/de/help/signal/ref/xcorr.html
> >
> > But, as you say, the most important aspect might be, that most people will
> > probably prefer changing the docs instead of changing the code.
> 
> Yeah, unless the current behaviour is actually broken or redundant in
> some way, we're not going to switch from one perfectly good convention
> to another perfectly good convention and break everyone's code in the
> process.
> 
> The most helpful thing would be if you could file a pull request that
> just changes the docstring to what you think it should be. Extra bonus
> points if it points out that there is another definition some people
> might be expecting instead, and explains how those people can use the
> existing functions to get what they want. :-)
> 
> -n
> 
> IMHO, "point[ing] out that there is another definition some people might be expecting instead, and explain[ing] how those people can use the existing functions to get what they want" should be a requirement for the docstring ("Notes" section), not merely worth "extra bonus points."  But then I'm not, presently, in a position to edit the docstring myself, so that's just MHO.  
> 
> IAE, I found what appears to me to be another "vote" for the extant docstring: Box & Jenkins, 1976, "Time Series Analysis: Forecasting and Control," Holden-Day, Oakland, pg. 374.  Perhaps a "switch" (with a default value that maintains current definition, so that extant uses would not require a code change) c/should be added to the function signature so that users can get easily get what they want?
> 

As pointed out in another post in this thread, there are now at least three different definitions of correlation which are in use in different disciplines of science and engineering:

Numpy code:

z_numpyCode[k] = sum_n a[n+k] * conj(v[n])


Numpy docs:

z_numpyDoc[k] = sum_n a[n] * conj(v[n+k])
                         = sum_n a[n-k] * conj(v[n])
                         = z_numpyCode[-k]


Wolfram Mathworld:

z_mmca[k] = sum_n conj(a[n]) * v[n+k]
                 = conj( sum_n a[n] * conj(v[n+k]) )
                 = conj( z_numpyDoc[k] )
                 = conj( z_numpyCode[-k] )

I'm sure there are even more if you search long enough. But shouldn't the primary objective be to bring the docs in line with the code (which is definitely not "broken")? It took me 2 days of debugging my code recently only to discover that numpy correlate() was calculating a different correlation than the docs said.

I can try to come up with a proposal for the docs. Could anyone point me to where I can find the docs? I can clone the numpy repo, however, I'm not a numpy developer.

Best wishes,
	Bernhard


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20131010/af03125b/attachment.html>

From jtaylor.debian at googlemail.com  Thu Oct 10 19:19:38 2013
From: jtaylor.debian at googlemail.com (Julian Taylor)
Date: Fri, 11 Oct 2013 01:19:38 +0200
Subject: [Numpy-discussion] Bug in numpy.correlate documentation
In-Reply-To: <09928B8C-1962-42A4-8566-297A2B07C1B1@gmx.net>
References: <CAFtPsZqdvzoN1NyXTr+MTLJ+a8sG1G3KxzOLtFi2FORWf8+bCQ@mail.gmail.com>
	<09928B8C-1962-42A4-8566-297A2B07C1B1@gmx.net>
Message-ID: <5257360A.6010608@googlemail.com>

On 10.10.2013 21:31, Bernhard Spinnler wrote:
> 
> On 10.10.2013, at 19:27, David Goldsmith <d.l.goldsmith at gmail.com
> <mailto:d.l.goldsmith at gmail.com>> wrote:
>>
>>     On Wed, Oct 9, 2013 at 7:48 PM, Bernhard Spinnler
>>     <Bernhard.Spinnler at gmx.net <mailto:Bernhard.Spinnler at gmx.net>> wrote:
>>     > Hi Richard,
>>     >
>>     > Ah, I searched the list but didn't find those posts before?
>>     >
>>     > I can easily imagine that correlation is defined differently in
>>     different
>>     > disciplines. Both ways are correct and it's just a convention or
>>     definition.
>>     > In my field (Digital Communications, Digital Signal Processing)
>>     the vast
>>     > majority uses the convention implemented by the code. Here are a few
>>     > examples of prominent text books:
>>     >
>>     > - Papoulis, "Probaility, Random Variables, and Stochastic
>>     Processes",
>>     > McGraw-Hill, 2nd ed.
>>     > - Benvenuto, Cherubini, "Algorithms for Communications Systems
>>     and their
>>     > Applications", Wiley.
>>     > - Carlson, "Communication Systems" 4th ed. 2002, McGraw-Hill.
>>     >
>>     > Last not least, Matlab's xcorr() function behaves exactly like
>>     correlate()
>>     > does right now, see
>>     > - http://www.mathworks.de/de/help/signal/ref/xcorr.html
>>     >
>>     > But, as you say, the most important aspect might be, that most
>>     people will
>>     > probably prefer changing the docs instead of changing the code.
>>
>>     Yeah, unless the current behaviour is actually broken or redundant in
>>     some way, we're not going to switch from one perfectly good convention
>>     to another perfectly good convention and break everyone's code in the
>>     process.
>>
>>     The most helpful thing would be if you could file a pull request that
>>     just changes the docstring to what you think it should be. Extra bonus
>>     points if it points out that there is another definition some people
>>     might be expecting instead, and explains how those people can use the
>>     existing functions to get what they want. :-)
>>
>>     -n
>>
>>
>> IMHO, "point[ing] out that there is another definition some people
>> might be expecting instead, and explain[ing] how those people can use
>> the existing functions to get what they want" should be a requirement
>> for the docstring ("Notes" section), not merely worth "extra bonus
>> points."  But then I'm not, presently, in a position to edit the
>> docstring myself, so that's just MHO. 
>>
>> IAE, I found what appears to me to be another "vote" for the extant
>> docstring: Box & Jenkins, 1976, "Time Series Analysis: Forecasting and
>> Control," Holden-Day, Oakland, pg. 374.  Perhaps a "switch" (with a
>> default value that maintains current definition, so that extant uses
>> would not require a code change) c/should be added to the function
>> signature so that users can get easily get what they want?
>>
> 
> As pointed out in another post in this thread, there are now at least
> three different definitions of correlation which are in use in different
> disciplines of science and engineering:
> 
> Numpy code:
> 
> z_numpyCode[k] = sum_n a[n+k] * conj(v[n])
> 
> 
> Numpy docs:
> 
> z_numpyDoc[k] = sum_n a[n] * conj(v[n+k])
>                          = sum_n a[n-k] * conj(v[n])
>                          = z_numpyCode[-k]
> 
> 
> Wolfram Mathworld:
> 
> z_mmca[k] = sum_n conj(a[n]) * v[n+k]
>                  = conj( sum_n a[n] * conj(v[n+k]) )
>                  = conj( z_numpyDoc[k] )
>                  = conj( z_numpyCode[-k] )
> 
> I'm sure there are even more if you search long enough. But shouldn't
> the primary objective be to bring the docs in line with the code (which
> is definitely not "broken")? It took me 2 days of debugging my
> code recently only to discover that numpy correlate() was calculating a
> different correlation than the docs said.
> 
> I can try to come up with a proposal for the docs. Could anyone point me
> to where I can find the docs? I can clone the numpy repo, however, I'm
> not a numpy developer.
> 

yes we should only change the documentation to match the (hopefully
correct) code.
the documentation is in the docstring of the correlate function in
numpy/core/numeric.py line 819


From stefan at sun.ac.za  Fri Oct 11 02:10:21 2013
From: stefan at sun.ac.za (=?ISO-8859-1?Q?St=E9fan_van_der_Walt?=)
Date: Fri, 11 Oct 2013 08:10:21 +0200
Subject: [Numpy-discussion] Python function for line intersection??
In-Reply-To: <1381238154.58791984@f77.i.mail.ru>
References: <1381238154.58791984@f77.i.mail.ru>
Message-ID: <CABDkGQk-2SQApXbYP=UuLSUfP8hjDGFdixx5PkYPv1uavBngMw@mail.gmail.com>

On Tue, Oct 8, 2013 at 3:15 PM, Happyman <bahtiyor_zohidov at mail.ru> wrote:
>     # slope of each line
>     m = (B[:, 1] - A[:, 1]) / (B[:, 0] - A[:, 0])
>     b = A[:, 1] - m * A[:, 0]

Calculating the slope of a line leads to difficulties, e.g. when
dealing with vertical lines.  Rather, parameterize lines as shown
here:

http://paulbourke.net/geometry/pointlineplane/

St?fan


From ralf.gommers at gmail.com  Sat Oct 12 01:22:46 2013
From: ralf.gommers at gmail.com (Ralf Gommers)
Date: Sat, 12 Oct 2013 07:22:46 +0200
Subject: [Numpy-discussion] ANN: Scipy 0.13.0 release candidate 1
Message-ID: <CABL7CQjm+sQhT=zNoj8fb0WxfPePQJPpU5Lw_ncj0j2yfZM54A@mail.gmail.com>

Hi all,

Hi all,

I'm happy to announce the availability of the first release candidate of
Scipy 0.13.0. Please try this RC and report any issues on the scipy-dev
mailing list.

Source tarballs, binaries and release notes can be found at
http://sourceforge.net/projects/scipy/files/scipy/0.13.0rc1/.

Thanks to everyone who helped test and fix the beta release. This is
shaping up to be a very solid release.

Cheers,
Ralf



==========================
SciPy 0.13.0 Release Notes
==========================

.. note:: Scipy 0.13.0 is not released yet!

.. contents::

SciPy 0.13.0 is the culmination of 7 months of hard work. It contains
many new features, numerous bug-fixes, improved test coverage and
better documentation.  There have been a number of deprecations and
API changes in this release, which are documented below.  All users
are encouraged to upgrade to this release, as there are a large number
of bug-fixes and optimizations.  Moreover, our development attention
will now shift to bug-fix releases on the 0.13.x branch, and on adding
new features on the master branch.

This release requires Python 2.6, 2.7 or 3.1-3.3 and NumPy 1.5.1 or greater.


New features
============

``scipy.integrate`` improvements
--------------------------------

N-dimensional numerical integration
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

A new function `scipy.integrate.nquad`, which provides N-dimensional
integration functionality with a more flexible interface than ``dblquad``
and
``tplquad``, has been added.

``dopri*`` improvements
^^^^^^^^^^^^^^^^^^^^^^^

The intermediate results from the ``dopri`` family of ODE solvers can now be
accessed by a *solout* callback function.


``scipy.linalg`` improvements
-----------------------------

Interpolative decompositions
^^^^^^^^^^^^^^^^^^^^^^^^^^^^

Scipy now includes a new module `scipy.linalg.interpolative`
containing routines for computing interpolative matrix decompositions
(ID). This feature is based on the ID software package by
P.G. Martinsson, V. Rokhlin, Y. Shkolnisky, and M. Tygert, previously
adapted for Python in the PymatrixId package by K.L. Ho.

Polar decomposition
^^^^^^^^^^^^^^^^^^^

A new function `scipy.linalg.polar`, to compute the polar decomposition
of a matrix, was added.

BLAS level 3 functions
^^^^^^^^^^^^^^^^^^^^^^

The BLAS functions ``symm``, ``syrk``, ``syr2k``, ``hemm``, ``herk`` and
``her2k`` are now wrapped in `scipy.linalg`.

Matrix functions
^^^^^^^^^^^^^^^^

Several matrix function algorithms have been implemented or updated
following
detailed descriptions in recent papers of Nick Higham and his co-authors.
These include the matrix square root (``sqrtm``), the matrix logarithm
(``logm``), the matrix exponential (``expm``) and its Frechet derivative
(``expm_frechet``), and fractional matrix powers
(``fractional_matrix_power``).


``scipy.optimize`` improvements
-------------------------------

Trust-region unconstrained minimization algorithms
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

The ``minimize`` function gained two trust-region solvers for unconstrained
minimization: ``dogleg`` and ``trust-ncg``.


``scipy.sparse`` improvements
-----------------------------

Boolean comparisons and sparse matrices
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

All sparse matrix types now support boolean data, and boolean operations.
Two
sparse matrices `A` and `B` can be compared in all the expected ways `A <
B`,
`A >= B`, `A != B`, producing similar results as dense Numpy arrays.
Comparisons with dense matrices and scalars are also supported.

CSR and CSC fancy indexing
^^^^^^^^^^^^^^^^^^^^^^^^^^

Compressed sparse row and column sparse matrix types now support fancy
indexing
with boolean matrices, slices, and lists. So where A is a (CSC or CSR)
sparse
matrix, you can do things like::

    >>> A[A > 0.5] = 1  # since Boolean sparse matrices work
    >>> A[:2, :3] = 2
    >>> A[[1,2], 2] = 3


``scipy.sparse.linalg`` improvements
------------------------------------

The new function ``onenormest`` provides a lower bound of the 1-norm of a
linear operator and has been implemented according to Higham and Tisseur
(2000).  This function is not only useful for sparse matrices, but can also
be
used to estimate the norm of products or powers of dense matrices without
explictly building the intermediate matrix.

The multiplicative action of the matrix exponential of a linear operator
(``expm_multiply``) has been implemented following the description in
Al-Mohy
and Higham (2011).

Abstract linear operators (`scipy.sparse.linalg.LinearOperator`) can now be
multiplied, added to each other, and exponentiated, producing new linear
operators. This enables easier construction of composite linear operations.


``scipy.spatial`` improvements
------------------------------

The vertices of a `ConvexHull` can now be accessed via the `vertices`
attribute,
which gives proper orientation in 2-D.


``scipy.signal`` improvements
-----------------------------

The cosine window function `scipy.signal.cosine` was added.


``scipy.special`` improvements
------------------------------

New functions `scipy.special.xlogy` and `scipy.special.xlog1py` were added.
These functions can simplify and speed up code that has to calculate
``x * log(y)`` and give 0 when ``x == 0``.


``scipy.io`` improvements
-------------------------

Unformatted Fortran file reader
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

The new class `scipy.io.FortranFile` facilitates reading unformatted
sequential files written by Fortran code.

``scipy.io.wavfile`` enhancements
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

`scipy.io.wavfile.write` now accepts a file buffer. Previously it only
accepted a filename.

`scipy.io.wavfile.read` and `scipy.io.wavfile.write` can now handle floating
point WAV files.


``scipy.interpolate`` improvements
----------------------------------

B-spline derivatives and antiderivatives
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

`scipy.interpolate.splder` and `scipy.interpolate.splantider` functions
for computing B-splines that represent derivatives and antiderivatives
of B-splines were added.  These functions are also available in the
class-based FITPACK interface as ``UnivariateSpline.derivative`` and
``UnivariateSpline.antiderivative``.


``scipy.stats`` improvements
----------------------------

Distributions now allow using keyword parameters in addition to
positional parameters in all methods.

The function `scipy.stats.power_divergence` has been added for the
Cressie-Read power divergence statistic and goodness of fit test.
Included in this family of statistics is the "G-test"
(http://en.wikipedia.org/wiki/G-test).

`scipy.stats.mood` now accepts multidimensional input.

An option was added to `scipy.stats.wilcoxon` for continuity correction.

`scipy.stats.chisquare` now has an `axis` argument.

`scipy.stats.mstats.chisquare` now has `axis` and `ddof` arguments.


Deprecated features
===================

``expm2`` and ``expm3``
-----------------------

The matrix exponential functions `scipy.linalg.expm2` and
`scipy.linalg.expm3`
are deprecated. All users should use the numerically more robust
`scipy.linalg.expm` function instead.

``scipy.stats`` functions
-------------------------

`scipy.stats.oneway` is deprecated; `scipy.stats.f_oneway` should be used
instead.

`scipy.stats.glm` is deprecated.  `scipy.stats.ttest_ind` is an equivalent
function; more full-featured general (and generalized) linear model
implementations can be found in statsmodels.

`scipy.stats.cmedian` is deprecated; ``numpy.median`` should be used
instead.


Backwards incompatible changes
==============================

LIL matrix assignment
---------------------
Assigning values to LIL matrices with two index arrays now works similarly
as
assigning into ndarrays::

    >>> x = lil_matrix((3, 3))
    >>> x[[0,1,2],[0,1,2]]=[0,1,2]
    >>> x.todense()
    matrix([[ 0.,  0.,  0.],
            [ 0.,  1.,  0.],
            [ 0.,  0.,  2.]])

rather than giving the result::

    >>> x.todense()
    matrix([[ 0.,  1.,  2.],
            [ 0.,  1.,  2.],
            [ 0.,  1.,  2.]])

Users relying on the previous behavior will need to revisit their code.
The previous behavior is obtained by ``x[numpy.ix_([0,1,2],[0,1,2])] = ...`.


Deprecated ``radon`` function removed
-------------------------------------

The ``misc.radon`` function, which was deprecated in scipy 0.11.0, has been
removed.  Users can find a more full-featured ``radon`` function in
scikit-image.


Removed deprecated keywords ``xa`` and ``xb`` from ``stats.distributions``
--------------------------------------------------------------------------

The keywords ``xa`` and ``xb``, which were deprecated since 0.11.0, have
been removed from the distributions in ``scipy.stats``.

Changes to MATLAB file readers / writers
----------------------------------------

The major change is that 1D arrays in numpy now become row vectors (shape
1, N)
when saved to a MATLAB 5 format file.  Previously 1D arrays saved as column
vectors (N, 1).  This is to harmonize the behavior of writing MATLAB 4 and 5
formats, and adapt to the defaults of numpy and MATLAB - for example
``np.atleast_2d`` returns 1D arrays as row vectors.

Trying to save arrays of greater than 2 dimensions in MATLAB 4 format now
raises
an error instead of silently reshaping the array as 2D.

``scipy.io.loadmat('afile')`` used to look for `afile` on the Python system
path
(``sys.path``); now ``loadmat`` only looks in the current directory for a
relative path filename.

Other changes
=============

Security fix: ``scipy.weave`` previously used temporary directories in an
insecure manner under certain circumstances.

Cython is now required to build *unreleased* versions of scipy.
The C files generated from Cython sources are not included in the git repo
anymore.  They are however still shipped in source releases.

The code base received a fairly large PEP8 cleanup.  A ``tox pep8``
command has been added; new code should pass this test command.

Authors
=======

This release contains work by the following people (contributed at least
one patch to this release, names in alphabetical order):

* Jorge Ca?ardo Alastuey +
* Tom Aldcroft +
* Max Bolingbroke +
* Joseph Jon Booker +
* Fran?ois Boulogne
* Matthew Brett
* Christian Brodbeck +
* Per Brodtkorb +
* Christian Brueffer +
* Lars Buitinck
* Evgeni Burovski +
* Tim Cera
* Lawrence Chan +
* David Cournapeau
* Draz?en Luc?anin +
* Alexander J. Dunlap +
* endolith
* Andr? Gaul +
* Christoph Gohlke
* Ralf Gommers
* Alex Griffing +
* Blake Griffith +
* Charles Harris
* Bob Helmbold +
* Andreas Hilboll
* Kat Huang +
* Oleksandr (Sasha) Huziy +
* Gert-Ludwig Ingold +
* Thouis (Ray) Jones
* Juan Luis Cano Rodr?guez +
* Robert Kern
* Andreas Kloeckner +
* Sytse Knypstra +
* Gustav Larsson +
* Denis Laxalde
* Christopher Lee
* Tim Leslie
* Wendy Liu +
* Clemens Novak +
* Takuya Oshima +
* Josef Perktold
* Illia Polosukhin +
* Przemek Porebski +
* Steve Richardson +
* Branden Rolston +
* Skipper Seabold
* Fazlul Shahriar
* Leo Singer +
* Rohit Sivaprasad +
* Daniel B. Smith +
* Julian Taylor
* Louis Thibault +
* Tomas Tomecek +
* John Travers
* Richard Tsai +
* Jacob Vanderplas
* Patrick Varilly
* Pauli Virtanen
* Stefan van der Walt
* Warren Weckesser
* Pedro Werneck +
* Nils Werner +
* Michael Wimmer +
* Nathan Woods +
* Tony S. Yu +

A total of 65 people contributed to this release.
People with a "+" by their names contributed a patch for the first time.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20131012/eab9fb9a/attachment.html>

From Bernhard.Spinnler at gmx.net  Mon Oct 14 14:25:39 2013
From: Bernhard.Spinnler at gmx.net (Bernhard Spinnler)
Date: Mon, 14 Oct 2013 20:25:39 +0200
Subject: [Numpy-discussion] Bug in numpy.correlate documentation
In-Reply-To: <5257360A.6010608@googlemail.com>
References: <CAFtPsZqdvzoN1NyXTr+MTLJ+a8sG1G3KxzOLtFi2FORWf8+bCQ@mail.gmail.com>
	<09928B8C-1962-42A4-8566-297A2B07C1B1@gmx.net>
	<5257360A.6010608@googlemail.com>
Message-ID: <7BD1CBF3-19A7-4152-9B84-29D42C00C2CC@gmx.net>


On 11.10.2013, at 01:19, Julian Taylor <jtaylor.debian at googlemail.com> wrote:
>>> 
>>>    Yeah, unless the current behaviour is actually broken or redundant in
>>>    some way, we're not going to switch from one perfectly good convention
>>>    to another perfectly good convention and break everyone's code in the
>>>    process.
>>> 
>>>    The most helpful thing would be if you could file a pull request that
>>>    just changes the docstring to what you think it should be. Extra bonus
>>>    points if it points out that there is another definition some people
>>>    might be expecting instead, and explains how those people can use the
>>>    existing functions to get what they want. :-)
>>> 
>>>    -n
>>> 
>>> 
>>> IMHO, "point[ing] out that there is another definition some people
>>> might be expecting instead, and explain[ing] how those people can use
>>> the existing functions to get what they want" should be a requirement
>>> for the docstring ("Notes" section), not merely worth "extra bonus
>>> points."  But then I'm not, presently, in a position to edit the
>>> docstring myself, so that's just MHO. 
>>> 
>>> IAE, I found what appears to me to be another "vote" for the extant
>>> docstring: Box & Jenkins, 1976, "Time Series Analysis: Forecasting and
>>> Control," Holden-Day, Oakland, pg. 374.  Perhaps a "switch" (with a
>>> default value that maintains current definition, so that extant uses
>>> would not require a code change) c/should be added to the function
>>> signature so that users can get easily get what they want?
>>> 
>> 
>> As pointed out in another post in this thread, there are now at least
>> three different definitions of correlation which are in use in different
>> disciplines of science and engineering:
>> 
>> Numpy code:
>> 
>> z_numpyCode[k] = sum_n a[n+k] * conj(v[n])
>> 
>> 
>> Numpy docs:
>> 
>> z_numpyDoc[k] = sum_n a[n] * conj(v[n+k])
>>                         = sum_n a[n-k] * conj(v[n])
>>                         = z_numpyCode[-k]
>> 
>> 
>> Wolfram Mathworld:
>> 
>> z_mmca[k] = sum_n conj(a[n]) * v[n+k]
>>                 = conj( sum_n a[n] * conj(v[n+k]) )
>>                 = conj( z_numpyDoc[k] )
>>                 = conj( z_numpyCode[-k] )
>> 
>> I'm sure there are even more if you search long enough. But shouldn't
>> the primary objective be to bring the docs in line with the code (which
>> is definitely not "broken")? It took me 2 days of debugging my
>> code recently only to discover that numpy correlate() was calculating a
>> different correlation than the docs said.
>> 
>> I can try to come up with a proposal for the docs. Could anyone point me
>> to where I can find the docs? I can clone the numpy repo, however, I'm
>> not a numpy developer.
>> 
> 
> yes we should only change the documentation to match the (hopefully
> correct) code.
> the documentation is in the docstring of the correlate function in
> numpy/core/numeric.py line 819
> _______________________________________________

Ok, corrected the docstring, mentioning one alternative definition of correlation. Pull request filed: https://github.com/numpy/numpy/pull/3913.

Bernhard




From charlesr.harris at gmail.com  Mon Oct 14 17:37:22 2013
From: charlesr.harris at gmail.com (Charles R Harris)
Date: Mon, 14 Oct 2013 15:37:22 -0600
Subject: [Numpy-discussion] NumPy 1.8.0rc2 release
Message-ID: <CAB6mnxJxnG5nPwbpoaMhh1QobSGj9A6=5NcrLRDRimDDgyx9rA@mail.gmail.com>

Hi All,

NumPy 1.8.0rc2 is up now on
sourceforge<http://sourceforge.net/projects/numpy/files/NumPy/1.8.0rc2/>.
Binary builds are included, except for Python 3.3 on windows. Many thanks
to Ralf for the binaries and to those who found and fixed the bugs in rc1.
Please test this thoroughly, especially if you have access to one of the
less common platforms. Testing of rc1 turned up several bugs that would
have been a embarrassment if they had made their way into the release and
we are very happy that they were discovered.

Chuck
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20131014/efea3a8e/attachment.html>

From lists at onerussian.com  Tue Oct 15 12:13:24 2013
From: lists at onerussian.com (Yaroslav Halchenko)
Date: Tue, 15 Oct 2013 12:13:24 -0400
Subject: [Numpy-discussion] RFC: is it worth giving a lightning talk at
 PyCon 2014 on numpy vbench-marking?
Message-ID: <20131015161324.GL27621@onerussian.com>

Hi Guys,

PyCon 2014 will be just around the corner from where I am, so I decided
to attend.  Being lazy (or busy) I haven't submitted any big talk but thinking
to submit few lightning talks (just 5 min and 400 characters abstract limit),
and I think it might be worth letting people know about my little project.  I
would really appreciate your sincere feedback (e.g. "not worth it" would be
valuable too).  Here is the title/abstract

    numpy-vbench -- speed benchmarks for NumPy

    http://yarikoptic.github.io/numpy-vbench provides collection of speed
    performance benchmarks for NumPy.  Benchmarking of multiple
    maintenance and current development branches allows not only to timely
    react to new performance regressions, but also to compare NumPy
    performance across releases.  Your contributions would help to
    guarantee that your code does not become slower with a new NumPy
    release.

btw -- fresh results are here http://yarikoptic.github.io/numpy-vbench/ .

I have tuned benchmarking so it now reflects the best performance across
multiple executions of the whole battery, thus eliminating spurious
variance if estimate is provided from a single point in time.  Eventually I
expect many of those curves to become even "cleaner".

-- 
Yaroslav O. Halchenko, Ph.D.
http://neuro.debian.net http://www.pymvpa.org http://www.fail2ban.org
Senior Research Associate,     Psychological and Brain Sciences Dept.
Dartmouth College, 419 Moore Hall, Hinman Box 6207, Hanover, NH 03755
Phone: +1 (603) 646-9834                       Fax: +1 (603) 646-1419
WWW:   http://www.linkedin.com/in/yarik        


From aron at ahmadia.net  Tue Oct 15 12:18:10 2013
From: aron at ahmadia.net (Aron Ahmadia)
Date: Tue, 15 Oct 2013 12:18:10 -0400
Subject: [Numpy-discussion] RFC: is it worth giving a lightning talk at
 PyCon 2014 on numpy vbench-marking?
In-Reply-To: <20131015161324.GL27621@onerussian.com>
References: <20131015161324.GL27621@onerussian.com>
Message-ID: <CAPhiW4jY4FGG+YieF2hvj5uNkO7TTAv4GYtC5=9tmOvouk2gUg@mail.gmail.com>

FWIW, I think it's a really cool project.  I'm not the target demographic
at PyCon, but I would be very surprised if this isn't a well-received talk.


On Tue, Oct 15, 2013 at 12:13 PM, Yaroslav Halchenko
<lists at onerussian.com>wrote:

> Hi Guys,
>
> PyCon 2014 will be just around the corner from where I am, so I decided
> to attend.  Being lazy (or busy) I haven't submitted any big talk but
> thinking
> to submit few lightning talks (just 5 min and 400 characters abstract
> limit),
> and I think it might be worth letting people know about my little project.
>  I
> would really appreciate your sincere feedback (e.g. "not worth it" would be
> valuable too).  Here is the title/abstract
>
>     numpy-vbench -- speed benchmarks for NumPy
>
>     http://yarikoptic.github.io/numpy-vbench provides collection of speed
>     performance benchmarks for NumPy.  Benchmarking of multiple
>     maintenance and current development branches allows not only to timely
>     react to new performance regressions, but also to compare NumPy
>     performance across releases.  Your contributions would help to
>     guarantee that your code does not become slower with a new NumPy
>     release.
>
> btw -- fresh results are here http://yarikoptic.github.io/numpy-vbench/ .
>
> I have tuned benchmarking so it now reflects the best performance across
> multiple executions of the whole battery, thus eliminating spurious
> variance if estimate is provided from a single point in time.  Eventually I
> expect many of those curves to become even "cleaner".
>
> --
> Yaroslav O. Halchenko, Ph.D.
> http://neuro.debian.net http://www.pymvpa.org http://www.fail2ban.org
> Senior Research Associate,     Psychological and Brain Sciences Dept.
> Dartmouth College, 419 Moore Hall, Hinman Box 6207, Hanover, NH 03755
> Phone: +1 (603) 646-9834                       Fax: +1 (603) 646-1419
> WWW:   http://www.linkedin.com/in/yarik
> _______________________________________________
> NumPy-Discussion mailing list
> NumPy-Discussion at scipy.org
> http://mail.scipy.org/mailman/listinfo/numpy-discussion
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20131015/ad4e1401/attachment.html>

From njs at pobox.com  Tue Oct 15 13:00:39 2013
From: njs at pobox.com (Nathaniel Smith)
Date: Tue, 15 Oct 2013 18:00:39 +0100
Subject: [Numpy-discussion] RFC: is it worth giving a lightning talk at
 PyCon 2014 on numpy vbench-marking?
In-Reply-To: <20131015161324.GL27621@onerussian.com>
References: <20131015161324.GL27621@onerussian.com>
Message-ID: <CAPJVwBnaL4gX1UZ1x+a2LeOTXO8ED81gE8m6GE+AtnDFcLAwyQ@mail.gmail.com>

On Tue, Oct 15, 2013 at 5:13 PM, Yaroslav Halchenko
<lists at onerussian.com> wrote:
> Hi Guys,
>
> PyCon 2014 will be just around the corner from where I am, so I decided
> to attend.  Being lazy (or busy) I haven't submitted any big talk but thinking
> to submit few lightning talks (just 5 min and 400 characters abstract limit),
> and I think it might be worth letting people know about my little project.  I
> would really appreciate your sincere feedback (e.g. "not worth it" would be
> valuable too).  Here is the title/abstract
>
>     numpy-vbench -- speed benchmarks for NumPy
>
>     http://yarikoptic.github.io/numpy-vbench provides collection of speed
>     performance benchmarks for NumPy.  Benchmarking of multiple
>     maintenance and current development branches allows not only to timely
>     react to new performance regressions, but also to compare NumPy
>     performance across releases.  Your contributions would help to
>     guarantee that your code does not become slower with a new NumPy
>     release.

What do you have to lose?

> btw -- fresh results are here http://yarikoptic.github.io/numpy-vbench/ .
>
> I have tuned benchmarking so it now reflects the best performance across
> multiple executions of the whole battery, thus eliminating spurious
> variance if estimate is provided from a single point in time.  Eventually I
> expect many of those curves to become even "cleaner".

On another note, what do you think of moving the vbench benchmarks
into the main numpy tree? We already require everyone who submits a
bug fix to add a test; there are a bunch of speed enhancements coming
in these days and it would be nice if we had some way to ask people to
submit a benchmark along with each one so that we know that the
enhancement stays enhanced...

-n


From lists at onerussian.com  Tue Oct 15 13:20:56 2013
From: lists at onerussian.com (Yaroslav Halchenko)
Date: Tue, 15 Oct 2013 13:20:56 -0400
Subject: [Numpy-discussion] RFC: is it worth giving a lightning talk at
 PyCon 2014 on numpy vbench-marking?
In-Reply-To: <CAPJVwBnaL4gX1UZ1x+a2LeOTXO8ED81gE8m6GE+AtnDFcLAwyQ@mail.gmail.com>
References: <20131015161324.GL27621@onerussian.com>
	<CAPJVwBnaL4gX1UZ1x+a2LeOTXO8ED81gE8m6GE+AtnDFcLAwyQ@mail.gmail.com>
Message-ID: <20131015172056.GM27621@onerussian.com>


On Tue, 15 Oct 2013, Nathaniel Smith wrote:
> > and I think it might be worth letting people know about my little project.  I
> > would really appreciate your sincere feedback (e.g. "not worth it" would be
> > valuable too).  Here is the title/abstract

> >     numpy-vbench -- speed benchmarks for NumPy

> >     http://yarikoptic.github.io/numpy-vbench provides collection of speed
> >     performance benchmarks for NumPy.  Benchmarking of multiple
> >     maintenance and current development branches allows not only to timely
> >     react to new performance regressions, but also to compare NumPy
> >     performance across releases.  Your contributions would help to
> >     guarantee that your code does not become slower with a new NumPy
> >     release.

> What do you have to lose?

time? ;)

> > btw -- fresh results are here http://yarikoptic.github.io/numpy-vbench/ .

> > I have tuned benchmarking so it now reflects the best performance across
> > multiple executions of the whole battery, thus eliminating spurious
> > variance if estimate is provided from a single point in time.  Eventually I
> > expect many of those curves to become even "cleaner".

> On another note, what do you think of moving the vbench benchmarks
> into the main numpy tree? We already require everyone who submits a
> bug fix to add a test; there are a bunch of speed enhancements coming
> in these days and it would be nice if we had some way to ask people to
> submit a benchmark along with each one so that we know that the
> enhancement stays enhanced...

I would be thrilled to do so, but

#1 I would need time I currently do not have to harmonize my setup for
   such inclusion

#2 I am still not sure if my tune ups on top of vbench should get merged
   as is https://github.com/pydata/vbench/pull/33

#3 vbench itself might go through a considerable RF if someone finds some
   spare time: https://github.com/pydata/vbench/issues/34
  
If merge for the PR #2 would happen, and test_perf.py from pandas would
get integrated within vbench -- then it might be logical to think about
#1 regardless of the ultimate resolution to #3 ;)  Whenever I have
some time for it I would try to push in that direction, and I do not
think it is unfeasible to see all of that done actually by PyCon 2014
(so the talk will have a different url ;) ), I just can't promise ATM.

-- 
Yaroslav O. Halchenko, Ph.D.
http://neuro.debian.net http://www.pymvpa.org http://www.fail2ban.org
Senior Research Associate,     Psychological and Brain Sciences Dept.
Dartmouth College, 419 Moore Hall, Hinman Box 6207, Hanover, NH 03755
Phone: +1 (603) 646-9834                       Fax: +1 (603) 646-1419
WWW:   http://www.linkedin.com/in/yarik        


From njs at pobox.com  Tue Oct 15 13:25:17 2013
From: njs at pobox.com (Nathaniel Smith)
Date: Tue, 15 Oct 2013 18:25:17 +0100
Subject: [Numpy-discussion] RFC: is it worth giving a lightning talk at
 PyCon 2014 on numpy vbench-marking?
In-Reply-To: <20131015172056.GM27621@onerussian.com>
References: <20131015161324.GL27621@onerussian.com>
	<CAPJVwBnaL4gX1UZ1x+a2LeOTXO8ED81gE8m6GE+AtnDFcLAwyQ@mail.gmail.com>
	<20131015172056.GM27621@onerussian.com>
Message-ID: <CAPJVwB=S3VpXVxb5ZN4gPgv8wU1QTmTO4TpMtwep=Exg+WmvBw@mail.gmail.com>

On Tue, Oct 15, 2013 at 6:20 PM, Yaroslav Halchenko
<lists at onerussian.com> wrote:
>
> On Tue, 15 Oct 2013, Nathaniel Smith wrote:
>> On another note, what do you think of moving the vbench benchmarks
>> into the main numpy tree? We already require everyone who submits a
>> bug fix to add a test; there are a bunch of speed enhancements coming
>> in these days and it would be nice if we had some way to ask people to
>> submit a benchmark along with each one so that we know that the
>> enhancement stays enhanced...
>
> I would be thrilled to do so, but
>
> #1 I would need time I currently do not have to harmonize my setup for
>    such inclusion
>
> #2 I am still not sure if my tune ups on top of vbench should get merged
>    as is https://github.com/pydata/vbench/pull/33
>
> #3 vbench itself might go through a considerable RF if someone finds some
>    spare time: https://github.com/pydata/vbench/issues/34
>
> If merge for the PR #2 would happen, and test_perf.py from pandas would
> get integrated within vbench -- then it might be logical to think about
> #1 regardless of the ultimate resolution to #3 ;)  Whenever I have
> some time for it I would try to push in that direction, and I do not
> think it is unfeasible to see all of that done actually by PyCon 2014
> (so the talk will have a different url ;) ), I just can't promise ATM.

Okay, well, keep it in mind :-)

-n


From lists at onerussian.com  Tue Oct 15 13:36:25 2013
From: lists at onerussian.com (Yaroslav Halchenko)
Date: Tue, 15 Oct 2013 13:36:25 -0400
Subject: [Numpy-discussion] RFC: is it worth giving a lightning talk at
 PyCon 2014 on numpy vbench-marking?
In-Reply-To: <20131015161324.GL27621@onerussian.com>
References: <20131015161324.GL27621@onerussian.com>
Message-ID: <20131015173625.GN27621@onerussian.com>

ok -- since no negative feedback received -- submitted as is.  I will
let you know when it gets rejected or accepted.

cheers,

On Tue, 15 Oct 2013, Yaroslav Halchenko wrote:

> Hi Guys,

> PyCon 2014 will be just around the corner from where I am, so I decided
> to attend.  Being lazy (or busy) I haven't submitted any big talk but thinking
> to submit few lightning talks (just 5 min and 400 characters abstract limit),
> and I think it might be worth letting people know about my little project.  I
> would really appreciate your sincere feedback (e.g. "not worth it" would be
> valuable too).  Here is the title/abstract

>     numpy-vbench -- speed benchmarks for NumPy

>     http://yarikoptic.github.io/numpy-vbench provides collection of speed
>     performance benchmarks for NumPy.  Benchmarking of multiple
>     maintenance and current development branches allows not only to timely
>     react to new performance regressions, but also to compare NumPy
>     performance across releases.  Your contributions would help to
>     guarantee that your code does not become slower with a new NumPy
>     release.

> btw -- fresh results are here http://yarikoptic.github.io/numpy-vbench/ .

> I have tuned benchmarking so it now reflects the best performance across
> multiple executions of the whole battery, thus eliminating spurious
> variance if estimate is provided from a single point in time.  Eventually I
> expect many of those curves to become even "cleaner".
-- 
Yaroslav O. Halchenko, Ph.D.
http://neuro.debian.net http://www.pymvpa.org http://www.fail2ban.org
Senior Research Associate,     Psychological and Brain Sciences Dept.
Dartmouth College, 419 Moore Hall, Hinman Box 6207, Hanover, NH 03755
Phone: +1 (603) 646-9834                       Fax: +1 (603) 646-1419
WWW:   http://www.linkedin.com/in/yarik        


From cournape at gmail.com  Tue Oct 15 14:46:43 2013
From: cournape at gmail.com (David Cournapeau)
Date: Tue, 15 Oct 2013 19:46:43 +0100
Subject: [Numpy-discussion] NumPy 1.8.0rc2 release
In-Reply-To: <CAB6mnxJxnG5nPwbpoaMhh1QobSGj9A6=5NcrLRDRimDDgyx9rA@mail.gmail.com>
References: <CAB6mnxJxnG5nPwbpoaMhh1QobSGj9A6=5NcrLRDRimDDgyx9rA@mail.gmail.com>
Message-ID: <CAGY4rcUscTBFggzwCf1vSRQj7dFHuEGFXSwq31A3x=zjLQnZ4w@mail.gmail.com>

It looks better than rc1, thanks for the great work. I have only tested on
rh5 for now, but building the following against numpy 1.7.1 and running
against 1.8.0 rc2 only give a few failures for the full list of packages
supported by Enthought. Bottleneck / larry are caused by numpy, the sklearn
may be a bug in numpy or scikit learn or scipy (eigh issue).

List of packages:

GDAL-1.10.0
MDP-3.3
Pycluster-1.50
ScientificPython-2.9.0
SimPy-2.2
astropy-0.2.4
basemap-1.0.6
biopython-1.59
chaco-4.3.0
enable-4.3.0
fastnumpy-1.0
fwrap-0.1.1
h5py-2.2.0
llvmmath-0.1.1
matplotlib-1.3.0
mayavi-4.3.0
netCDF4-1.0.5
networkx-1.8.1
nltk-2.0.1
numba-0.10.2
opencv-2.4.5
pandas-0.12.0
pyfits-3.0.6
pygarrayimage-0.0.7
pygrib-1.9.2
pyhdf-0.8.3
pysparse-1.2.dev213
pytables-2.4.0
scikits.image-0.8.2
scikits.rsformats-0.1
scikits.timeseries-0.91.3
scimath-4.1.2
scipy-0.12.0
traits-4.3.0

As for the bottleneck/larry failures (for reference):

 ======================================================================
  FAIL: Test nanargmin.
  ----------------------------------------------------------------------
  Traceback (most recent call last):
    File
"/home/vagrant/src/master-env/lib/python2.7/site-packages/nose/case.py",
line 197, in runTest
      self.test(*self.arg)
    File
"/home/vagrant/src/master-env/lib/python2.7/site-packages/bottleneck/tests/func_test.py",
line 78, in unit_maker
      assert_array_equal(actual, desired, err_msg)
    File
"/home/vagrant/src/master-env/lib/python2.7/site-packages/numpy/testing/utils.py",
line 718, in assert_array_equal
      verbose=verbose, header='Arrays are not equal')
    File
"/home/vagrant/src/master-env/lib/python2.7/site-packages/numpy/testing/utils.py",
line 644, in assert_array_compare
      raise AssertionError(msg)
  AssertionError:
  Arrays are not equal

  func nanargmin | input a44 (float32) | shape (4,) | axis -1

  Input array:
  [ nan  nan  nan  nan]

  (mismatch 100.0%)
   x: array(nan)
   y: array('Crashed',
        dtype='|S7')

  ======================================================================
  FAIL: Test nanargmax.
  ----------------------------------------------------------------------
  Traceback (most recent call last):
    File
"/home/vagrant/src/master-env/lib/python2.7/site-packages/nose/case.py",
line 197, in runTest
      self.test(*self.arg)
    File
"/home/vagrant/src/master-env/lib/python2.7/site-packages/bottleneck/tests/func_test.py",
line 78, in unit_maker
      assert_array_equal(actual, desired, err_msg)
    File
"/home/vagrant/src/master-env/lib/python2.7/site-packages/numpy/testing/utils.py",
line 718, in assert_array_equal
      verbose=verbose, header='Arrays are not equal')
    File
"/home/vagrant/src/master-env/lib/python2.7/site-packages/numpy/testing/utils.py",
line 644, in assert_array_compare
      raise AssertionError(msg)
  AssertionError:
  Arrays are not equal

  func nanargmax | input a44 (float32) | shape (4,) | axis -1

  Input array:
  [ nan  nan  nan  nan]

  (mismatch 100.0%)
   x: array(nan)
   y: array('Crashed',
        dtype='|S7')

  ----------------------------------------------------------------------
  Ran 124 tests in 85.714s

  FAILED (failures=2)
FAIL

and larry:

======================================================================
ERROR: Failure: IndexError (too many indices)
----------------------------------------------------------------------
Traceback (most recent call last):
  File
"/home/vagrant/src/master-env/lib/python2.7/site-packages/nose/loader.py",
line 253, in generate
    for test in g():
  File
"/home/vagrant/src/master-env/lib/python2.7/site-packages/la/tests/all_nan_test.py",
line 31, in test_all_nan
    actual = getattr(lar(), method)(*parameters)
  File
"/home/vagrant/src/master-env/lib/python2.7/site-packages/la/deflarry.py",
line 3066, in quantile
    x = quantile(self.x, q, axis=axis)
  File
"/home/vagrant/src/master-env/lib/python2.7/site-packages/la/farray/normalize.py",
line 289, in quantile
    y = np.apply_along_axis(_quantileraw1d, axis, x, q)
  File
"/home/vagrant/src/master-env/lib/python2.7/site-packages/numpy/lib/shape_base.py",
line 79, in apply_along_axis
    res = func1d(arr[tuple(i.tolist())],*args)
  File
"/home/vagrant/src/master-env/lib/python2.7/site-packages/la/farray/normalize.py",
line 228, in _quantileraw1d
    xi = xi[idx,:]
IndexError: too many indices

======================================================================
ERROR: larry.quantile_1
----------------------------------------------------------------------
Traceback (most recent call last):
  File
"/home/vagrant/src/master-env/lib/python2.7/site-packages/la/tests/deflarry_test.py",
line 3401, in test_quantile_1
    actual = self.l1.quantile(2)
  File
"/home/vagrant/src/master-env/lib/python2.7/site-packages/la/deflarry.py",
line 3066, in quantile
    x = quantile(self.x, q, axis=axis)
  File
"/home/vagrant/src/master-env/lib/python2.7/site-packages/la/farray/normalize.py",
line 289, in quantile
    y = np.apply_along_axis(_quantileraw1d, axis, x, q)
  File
"/home/vagrant/src/master-env/lib/python2.7/site-packages/numpy/lib/shape_base.py",
line 79, in apply_along_axis
    res = func1d(arr[tuple(i.tolist())],*args)
  File
"/home/vagrant/src/master-env/lib/python2.7/site-packages/la/farray/normalize.py",
line 228, in _quantileraw1d
    xi = xi[idx,:]
IndexError: too many indices

(more similar)


On Mon, Oct 14, 2013 at 10:37 PM, Charles R Harris <
charlesr.harris at gmail.com> wrote:

> Hi All,
>
> NumPy 1.8.0rc2 is up now on sourceforge<http://sourceforge.net/projects/numpy/files/NumPy/1.8.0rc2/>.
> Binary builds are included, except for Python 3.3 on windows. Many thanks
> to Ralf for the binaries and to those who found and fixed the bugs in rc1.
> Please test this thoroughly, especially if you have access to one of the
> less common platforms. Testing of rc1 turned up several bugs that would
> have been a embarrassment if they had made their way into the release and
> we are very happy that they were discovered.
>
> Chuck
>
> _______________________________________________
> NumPy-Discussion mailing list
> NumPy-Discussion at scipy.org
> http://mail.scipy.org/mailman/listinfo/numpy-discussion
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20131015/ed44cff4/attachment.html>

From charlesr.harris at gmail.com  Tue Oct 15 15:06:12 2013
From: charlesr.harris at gmail.com (Charles R Harris)
Date: Tue, 15 Oct 2013 13:06:12 -0600
Subject: [Numpy-discussion] NumPy 1.8.0rc2 release
In-Reply-To: <CAGY4rcUscTBFggzwCf1vSRQj7dFHuEGFXSwq31A3x=zjLQnZ4w@mail.gmail.com>
References: <CAB6mnxJxnG5nPwbpoaMhh1QobSGj9A6=5NcrLRDRimDDgyx9rA@mail.gmail.com>
	<CAGY4rcUscTBFggzwCf1vSRQj7dFHuEGFXSwq31A3x=zjLQnZ4w@mail.gmail.com>
Message-ID: <CAB6mnxKTN=OwXR7N4=JtApM9EkYy224BHGTr7O+zoJ95y2gq_Q@mail.gmail.com>

Hi David,


On Tue, Oct 15, 2013 at 12:46 PM, David Cournapeau <cournape at gmail.com>wrote:

> It looks better than rc1, thanks for the great work. I have only tested on
> rh5 for now, but building the following against numpy 1.7.1 and running
> against 1.8.0 rc2 only give a few failures for the full list of packages
> supported by Enthought. Bottleneck / larry are caused by numpy, the sklearn
> may be a bug in numpy or scikit learn or scipy (eigh issue).
>
> List of packages:
>
> GDAL-1.10.0
> MDP-3.3
> Pycluster-1.50
> ScientificPython-2.9.0
> SimPy-2.2
> astropy-0.2.4
> basemap-1.0.6
> biopython-1.59
> chaco-4.3.0
> enable-4.3.0
> fastnumpy-1.0
> fwrap-0.1.1
> h5py-2.2.0
> llvmmath-0.1.1
> matplotlib-1.3.0
> mayavi-4.3.0
> netCDF4-1.0.5
> networkx-1.8.1
> nltk-2.0.1
> numba-0.10.2
> opencv-2.4.5
> pandas-0.12.0
> pyfits-3.0.6
> pygarrayimage-0.0.7
> pygrib-1.9.2
> pyhdf-0.8.3
> pysparse-1.2.dev213
> pytables-2.4.0
> scikits.image-0.8.2
> scikits.rsformats-0.1
> scikits.timeseries-0.91.3
> scimath-4.1.2
> scipy-0.12.0
> traits-4.3.0
>
> As for the bottleneck/larry failures (for reference):
>
>  ======================================================================
>   FAIL: Test nanargmin.
>   ----------------------------------------------------------------------
>   Traceback (most recent call last):
>     File
> "/home/vagrant/src/master-env/lib/python2.7/site-packages/nose/case.py",
> line 197, in runTest
>       self.test(*self.arg)
>     File
> "/home/vagrant/src/master-env/lib/python2.7/site-packages/bottleneck/tests/func_test.py",
> line 78, in unit_maker
>       assert_array_equal(actual, desired, err_msg)
>     File
> "/home/vagrant/src/master-env/lib/python2.7/site-packages/numpy/testing/utils.py",
> line 718, in assert_array_equal
>       verbose=verbose, header='Arrays are not equal')
>     File
> "/home/vagrant/src/master-env/lib/python2.7/site-packages/numpy/testing/utils.py",
> line 644, in assert_array_compare
>       raise AssertionError(msg)
>   AssertionError:
>   Arrays are not equal
>
>   func nanargmin | input a44 (float32) | shape (4,) | axis -1
>
>   Input array:
>   [ nan  nan  nan  nan]
>
>   (mismatch 100.0%)
>    x: array(nan)
>    y: array('Crashed',
>         dtype='|S7')
>
>   ======================================================================
>   FAIL: Test nanargmax.
>   ----------------------------------------------------------------------
>   Traceback (most recent call last):
>     File
> "/home/vagrant/src/master-env/lib/python2.7/site-packages/nose/case.py",
> line 197, in runTest
>       self.test(*self.arg)
>     File
> "/home/vagrant/src/master-env/lib/python2.7/site-packages/bottleneck/tests/func_test.py",
> line 78, in unit_maker
>       assert_array_equal(actual, desired, err_msg)
>     File
> "/home/vagrant/src/master-env/lib/python2.7/site-packages/numpy/testing/utils.py",
> line 718, in assert_array_equal
>       verbose=verbose, header='Arrays are not equal')
>     File
> "/home/vagrant/src/master-env/lib/python2.7/site-packages/numpy/testing/utils.py",
> line 644, in assert_array_compare
>       raise AssertionError(msg)
>   AssertionError:
>   Arrays are not equal
>
>   func nanargmax | input a44 (float32) | shape (4,) | axis -1
>
>   Input array:
>   [ nan  nan  nan  nan]
>
>   (mismatch 100.0%)
>    x: array(nan)
>    y: array('Crashed',
>         dtype='|S7')
>
>   ----------------------------------------------------------------------
>   Ran 124 tests in 85.714s
>
>   FAILED (failures=2)
> FAIL
>
>
Not going to fix these, nanarg{max, min} now raises an exception for this
case.


> and larry:
>
> ======================================================================
> ERROR: Failure: IndexError (too many indices)
> ----------------------------------------------------------------------
> Traceback (most recent call last):
>   File
> "/home/vagrant/src/master-env/lib/python2.7/site-packages/nose/loader.py",
> line 253, in generate
>     for test in g():
>   File
> "/home/vagrant/src/master-env/lib/python2.7/site-packages/la/tests/all_nan_test.py",
> line 31, in test_all_nan
>     actual = getattr(lar(), method)(*parameters)
>   File
> "/home/vagrant/src/master-env/lib/python2.7/site-packages/la/deflarry.py",
> line 3066, in quantile
>     x = quantile(self.x, q, axis=axis)
>   File
> "/home/vagrant/src/master-env/lib/python2.7/site-packages/la/farray/normalize.py",
> line 289, in quantile
>     y = np.apply_along_axis(_quantileraw1d, axis, x, q)
>   File
> "/home/vagrant/src/master-env/lib/python2.7/site-packages/numpy/lib/shape_base.py",
> line 79, in apply_along_axis
>     res = func1d(arr[tuple(i.tolist())],*args)
>   File
> "/home/vagrant/src/master-env/lib/python2.7/site-packages/la/farray/normalize.py",
> line 228, in _quantileraw1d
>     xi = xi[idx,:]
> IndexError: too many indices
>
> ======================================================================
> ERROR: larry.quantile_1
> ----------------------------------------------------------------------
> Traceback (most recent call last):
>   File
> "/home/vagrant/src/master-env/lib/python2.7/site-packages/la/tests/deflarry_test.py",
> line 3401, in test_quantile_1
>     actual = self.l1.quantile(2)
>   File
> "/home/vagrant/src/master-env/lib/python2.7/site-packages/la/deflarry.py",
> line 3066, in quantile
>     x = quantile(self.x, q, axis=axis)
>   File
> "/home/vagrant/src/master-env/lib/python2.7/site-packages/la/farray/normalize.py",
> line 289, in quantile
>     y = np.apply_along_axis(_quantileraw1d, axis, x, q)
>   File
> "/home/vagrant/src/master-env/lib/python2.7/site-packages/numpy/lib/shape_base.py",
> line 79, in apply_along_axis
>     res = func1d(arr[tuple(i.tolist())],*args)
>   File
> "/home/vagrant/src/master-env/lib/python2.7/site-packages/la/farray/normalize.py",
> line 228, in _quantileraw1d
>     xi = xi[idx,:]
> IndexError: too many indices
>
> (more similar)
>

Iarry problem, the proper form here is xi[x,...]

Chuck
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20131015/57682ca2/attachment.html>

From cckidder at gmail.com  Wed Oct 16 11:39:43 2013
From: cckidder at gmail.com (Chad Kidder)
Date: Wed, 16 Oct 2013 09:39:43 -0600
Subject: [Numpy-discussion] Using array mask swaps array axes
Message-ID: <CANM+tMa8gHpOumZQzEedk0BsJYnu8tbx-A6T1eL7Lh=+AcJv1w@mail.gmail.com>

Just found what should be a bug in 1.7.1.  I'm running python(x,y) on
windows here:

>>> dataMatrix[ii,:,mask].shape
(201, 23)
>>> dataMatrix[ii,:,:].shape
(23, 201)
>>> dataMatrix.shape
(24, 23, 201)
>>> mask
array([ True,  True,  True,  True,  True,  True,  True,  True,  True,
        ...
        True,  True,  True], dtype=bool)

using a mask should not change the order of the dimensions.  Is there a
reason for this behavior, and if so, how do I avoid it in the future?
Thanks

     --Chad Kidder
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20131016/9ed6c0ef/attachment.html>

From ben.root at ou.edu  Wed Oct 16 11:50:03 2013
From: ben.root at ou.edu (Benjamin Root)
Date: Wed, 16 Oct 2013 11:50:03 -0400
Subject: [Numpy-discussion] Using array mask swaps array axes
In-Reply-To: <CANM+tMa8gHpOumZQzEedk0BsJYnu8tbx-A6T1eL7Lh=+AcJv1w@mail.gmail.com>
References: <CANM+tMa8gHpOumZQzEedk0BsJYnu8tbx-A6T1eL7Lh=+AcJv1w@mail.gmail.com>
Message-ID: <CANNq6FmLiUL5o2piEO7vus6MYVgYVd4WnaiDY_kiXbjOy2FyYA@mail.gmail.com>

On Wed, Oct 16, 2013 at 11:39 AM, Chad Kidder <cckidder at gmail.com> wrote:

> Just found what should be a bug in 1.7.1.  I'm running python(x,y) on
> windows here:
>
> >>> dataMatrix[ii,:,mask].shape
> (201, 23)
> >>> dataMatrix[ii,:,:].shape
> (23, 201)
> >>> dataMatrix.shape
> (24, 23, 201)
> >>> mask
> array([ True,  True,  True,  True,  True,  True,  True,  True,  True,
>         ...
>         True,  True,  True], dtype=bool)
>
> using a mask should not change the order of the dimensions.  Is there a
> reason for this behavior, and if so, how do I avoid it in the future?
> Thanks
>
>      --Chad Kidder
>
>
Chad,

The issue here is one where there is the mixing of fancy indexing (I
presume that is what "ii" is), slicing and boolean indexing. If I remember
correctly, the changing of the dimension orders was an inadvertent
byproduct of handing all this array accessing methods in one shot. I think
this was addressed in 1.8. Sorry for being very brief and vague, hopefully
someone else who understands what the resolution was can fill in.

Cheers!
Ben Root
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20131016/df0bb01c/attachment.html>

From sebastian at sipsolutions.net  Wed Oct 16 12:28:23 2013
From: sebastian at sipsolutions.net (Sebastian Berg)
Date: Wed, 16 Oct 2013 18:28:23 +0200
Subject: [Numpy-discussion] Using array mask swaps array axes
In-Reply-To: <CANNq6FmLiUL5o2piEO7vus6MYVgYVd4WnaiDY_kiXbjOy2FyYA@mail.gmail.com>
References: <CANM+tMa8gHpOumZQzEedk0BsJYnu8tbx-A6T1eL7Lh=+AcJv1w@mail.gmail.com>
	<CANNq6FmLiUL5o2piEO7vus6MYVgYVd4WnaiDY_kiXbjOy2FyYA@mail.gmail.com>
Message-ID: <1381940903.19985.15.camel@sebastian-laptop>

On Wed, 2013-10-16 at 11:50 -0400, Benjamin Root wrote:
> 
> 
> 
> On Wed, Oct 16, 2013 at 11:39 AM, Chad Kidder <cckidder at gmail.com>
> wrote:
>         Just found what should be a bug in 1.7.1.  I'm running
>         python(x,y) on windows here:
>         
>         >>> dataMatrix[ii,:,mask].shape
>         (201, 23)
>         >>> dataMatrix[ii,:,:].shape
>         (23, 201)
>         >>> dataMatrix.shape
>         (24, 23, 201)
>         >>> mask
>         array([ True,  True,  True,  True,  True,  True,  True,  True,
>         True,
>                 ...
>                 True,  True,  True], dtype=bool)
>         
>         
>         using a mask should not change the order of the dimensions.
>         Is there a reason for this behavior, and if so, how do I avoid
>         it in the future?  Thanks
>         
>         
>              --Chad Kidder
>         
>         
> 
> 
> Chad,
> 
> The issue here is one where there is the mixing of fancy indexing (I
> presume that is what "ii" is), slicing and boolean indexing. If I
> remember correctly, the changing of the dimension orders was an
> inadvertent byproduct of handing all this array accessing methods in
> one shot. I think this was addressed in 1.8. Sorry for being very
> brief and vague, hopefully someone else who understands what the
> resolution was can fill in.
> 
Yes, in fact `ii` can just be a normal integer, since an integer *is*
considered an advanced/fancy index (in the sense that it forces
transposing, not in the sense that it forces a copy by itself, so
integers are *both* advanced and view based indices!).

This is how advanced/fancy indexing works, there is `np.ix_` which helps
in some cases, but not exactly for your problem. For a more detailed
description of fancy indexing check:
http://docs.scipy.org/doc/numpy/reference/arrays.indexing.html#advanced-indexing

You have a slice between the mask and the integer `arr[1, :, mask]`,
which means the `mask` result dimension is transposed to the front. It
would not be if it was `arr[:, 1, mask]` (since numpy can actually guess
where it came from in that case).

Since you seem to always have exactly one advanced (mask) index, in your
example, the simplest solution is probably:
`dataMatrix[ii,...][:,mask]`
(first view based slicing, then the advanced boolean index. Since the
first part (if `ii` is an integer) will not copy the data, this will
also work for assignments).

- Sebastian


> Cheers!
> Ben Root
> 
> 
> 
> _______________________________________________
> NumPy-Discussion mailing list
> NumPy-Discussion at scipy.org
> http://mail.scipy.org/mailman/listinfo/numpy-discussion




From charlesr.harris at gmail.com  Wed Oct 16 12:55:23 2013
From: charlesr.harris at gmail.com (Charles R Harris)
Date: Wed, 16 Oct 2013 10:55:23 -0600
Subject: [Numpy-discussion] NumPy 1.8.0rc2 release
In-Reply-To: <CAB6mnxKTN=OwXR7N4=JtApM9EkYy224BHGTr7O+zoJ95y2gq_Q@mail.gmail.com>
References: <CAB6mnxJxnG5nPwbpoaMhh1QobSGj9A6=5NcrLRDRimDDgyx9rA@mail.gmail.com>
	<CAGY4rcUscTBFggzwCf1vSRQj7dFHuEGFXSwq31A3x=zjLQnZ4w@mail.gmail.com>
	<CAB6mnxKTN=OwXR7N4=JtApM9EkYy224BHGTr7O+zoJ95y2gq_Q@mail.gmail.com>
Message-ID: <CAB6mnx+Lf0jJUczGZyWMys_D8wcr3p1DbDeO8Pz5brakB2r71g@mail.gmail.com>

On Tue, Oct 15, 2013 at 1:06 PM, Charles R Harris <charlesr.harris at gmail.com
> wrote:

> Hi David,
>
>
> On Tue, Oct 15, 2013 at 12:46 PM, David Cournapeau <cournape at gmail.com>wrote:
>
>> It looks better than rc1, thanks for the great work. I have only tested
>> on rh5 for now, but building the following against numpy 1.7.1 and running
>> against 1.8.0 rc2 only give a few failures for the full list of packages
>> supported by Enthought. Bottleneck / larry are caused by numpy, the sklearn
>> may be a bug in numpy or scikit learn or scipy (eigh issue).
>>
>> List of packages:
>>
>> GDAL-1.10.0
>> MDP-3.3
>> Pycluster-1.50
>> ScientificPython-2.9.0
>> SimPy-2.2
>> astropy-0.2.4
>> basemap-1.0.6
>> biopython-1.59
>> chaco-4.3.0
>> enable-4.3.0
>> fastnumpy-1.0
>> fwrap-0.1.1
>> h5py-2.2.0
>> llvmmath-0.1.1
>> matplotlib-1.3.0
>> mayavi-4.3.0
>> netCDF4-1.0.5
>> networkx-1.8.1
>> nltk-2.0.1
>> numba-0.10.2
>> opencv-2.4.5
>> pandas-0.12.0
>> pyfits-3.0.6
>> pygarrayimage-0.0.7
>> pygrib-1.9.2
>> pyhdf-0.8.3
>> pysparse-1.2.dev213
>> pytables-2.4.0
>> scikits.image-0.8.2
>> scikits.rsformats-0.1
>> scikits.timeseries-0.91.3
>> scimath-4.1.2
>> scipy-0.12.0
>> traits-4.3.0
>>
>> As for the bottleneck/larry failures (for reference):
>>
>>  ======================================================================
>>   FAIL: Test nanargmin.
>>   ----------------------------------------------------------------------
>>   Traceback (most recent call last):
>>     File
>> "/home/vagrant/src/master-env/lib/python2.7/site-packages/nose/case.py",
>> line 197, in runTest
>>       self.test(*self.arg)
>>     File
>> "/home/vagrant/src/master-env/lib/python2.7/site-packages/bottleneck/tests/func_test.py",
>> line 78, in unit_maker
>>       assert_array_equal(actual, desired, err_msg)
>>     File
>> "/home/vagrant/src/master-env/lib/python2.7/site-packages/numpy/testing/utils.py",
>> line 718, in assert_array_equal
>>       verbose=verbose, header='Arrays are not equal')
>>     File
>> "/home/vagrant/src/master-env/lib/python2.7/site-packages/numpy/testing/utils.py",
>> line 644, in assert_array_compare
>>       raise AssertionError(msg)
>>   AssertionError:
>>   Arrays are not equal
>>
>>   func nanargmin | input a44 (float32) | shape (4,) | axis -1
>>
>>   Input array:
>>   [ nan  nan  nan  nan]
>>
>>   (mismatch 100.0%)
>>    x: array(nan)
>>    y: array('Crashed',
>>         dtype='|S7')
>>
>>   ======================================================================
>>   FAIL: Test nanargmax.
>>   ----------------------------------------------------------------------
>>   Traceback (most recent call last):
>>     File
>> "/home/vagrant/src/master-env/lib/python2.7/site-packages/nose/case.py",
>> line 197, in runTest
>>       self.test(*self.arg)
>>     File
>> "/home/vagrant/src/master-env/lib/python2.7/site-packages/bottleneck/tests/func_test.py",
>> line 78, in unit_maker
>>       assert_array_equal(actual, desired, err_msg)
>>     File
>> "/home/vagrant/src/master-env/lib/python2.7/site-packages/numpy/testing/utils.py",
>> line 718, in assert_array_equal
>>       verbose=verbose, header='Arrays are not equal')
>>     File
>> "/home/vagrant/src/master-env/lib/python2.7/site-packages/numpy/testing/utils.py",
>> line 644, in assert_array_compare
>>       raise AssertionError(msg)
>>   AssertionError:
>>   Arrays are not equal
>>
>>   func nanargmax | input a44 (float32) | shape (4,) | axis -1
>>
>>   Input array:
>>   [ nan  nan  nan  nan]
>>
>>   (mismatch 100.0%)
>>    x: array(nan)
>>    y: array('Crashed',
>>         dtype='|S7')
>>
>>   ----------------------------------------------------------------------
>>   Ran 124 tests in 85.714s
>>
>>   FAILED (failures=2)
>> FAIL
>>
>>
> Not going to fix these, nanarg{max, min} now raises an exception for this
> case.
>
>
>> and larry:
>>
>> ======================================================================
>> ERROR: Failure: IndexError (too many indices)
>> ----------------------------------------------------------------------
>> Traceback (most recent call last):
>>   File
>> "/home/vagrant/src/master-env/lib/python2.7/site-packages/nose/loader.py",
>> line 253, in generate
>>     for test in g():
>>   File
>> "/home/vagrant/src/master-env/lib/python2.7/site-packages/la/tests/all_nan_test.py",
>> line 31, in test_all_nan
>>     actual = getattr(lar(), method)(*parameters)
>>   File
>> "/home/vagrant/src/master-env/lib/python2.7/site-packages/la/deflarry.py",
>> line 3066, in quantile
>>     x = quantile(self.x, q, axis=axis)
>>   File
>> "/home/vagrant/src/master-env/lib/python2.7/site-packages/la/farray/normalize.py",
>> line 289, in quantile
>>     y = np.apply_along_axis(_quantileraw1d, axis, x, q)
>>   File
>> "/home/vagrant/src/master-env/lib/python2.7/site-packages/numpy/lib/shape_base.py",
>> line 79, in apply_along_axis
>>     res = func1d(arr[tuple(i.tolist())],*args)
>>   File
>> "/home/vagrant/src/master-env/lib/python2.7/site-packages/la/farray/normalize.py",
>> line 228, in _quantileraw1d
>>     xi = xi[idx,:]
>> IndexError: too many indices
>>
>> ======================================================================
>> ERROR: larry.quantile_1
>> ----------------------------------------------------------------------
>> Traceback (most recent call last):
>>   File
>> "/home/vagrant/src/master-env/lib/python2.7/site-packages/la/tests/deflarry_test.py",
>> line 3401, in test_quantile_1
>>     actual = self.l1.quantile(2)
>>   File
>> "/home/vagrant/src/master-env/lib/python2.7/site-packages/la/deflarry.py",
>> line 3066, in quantile
>>     x = quantile(self.x, q, axis=axis)
>>   File
>> "/home/vagrant/src/master-env/lib/python2.7/site-packages/la/farray/normalize.py",
>> line 289, in quantile
>>     y = np.apply_along_axis(_quantileraw1d, axis, x, q)
>>   File
>> "/home/vagrant/src/master-env/lib/python2.7/site-packages/numpy/lib/shape_base.py",
>> line 79, in apply_along_axis
>>     res = func1d(arr[tuple(i.tolist())],*args)
>>   File
>> "/home/vagrant/src/master-env/lib/python2.7/site-packages/la/farray/normalize.py",
>> line 228, in _quantileraw1d
>>     xi = xi[idx,:]
>> IndexError: too many indices
>>
>> (more similar)
>>
>
> Iarry problem, the proper form here is xi[x,...]
>

It has been fixed in larry master

Chuck

>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20131016/fb0eecce/attachment.html>

From thomas_unterthiner at web.de  Wed Oct 16 13:04:52 2013
From: thomas_unterthiner at web.de (Thomas Unthiner)
Date: Wed, 16 Oct 2013 19:04:52 +0200 (CEST)
Subject: [Numpy-discussion] numpy ACML support is kind of broken
Message-ID: <trinity-d3c2641d-ccd2-4db8-9012-d3785442a014-1381943092272@3capp-webde-bs01>

An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20131016/fc06a3eb/attachment.html>

From cckidder at gmail.com  Wed Oct 16 18:06:29 2013
From: cckidder at gmail.com (Chad Kidder)
Date: Wed, 16 Oct 2013 16:06:29 -0600
Subject: [Numpy-discussion] Using array mask swaps array axes
In-Reply-To: <1381940903.19985.15.camel@sebastian-laptop>
References: <CANM+tMa8gHpOumZQzEedk0BsJYnu8tbx-A6T1eL7Lh=+AcJv1w@mail.gmail.com>
	<CANNq6FmLiUL5o2piEO7vus6MYVgYVd4WnaiDY_kiXbjOy2FyYA@mail.gmail.com>
	<1381940903.19985.15.camel@sebastian-laptop>
Message-ID: <CANM+tMYFtkwzzyyrzkPn645OsJOVYzozBYZnZ-VXdkfmzo2FZQ@mail.gmail.com>

Thanks, that works.  It will be nice when the original way works also.
On Oct 16, 2013 10:28 AM, "Sebastian Berg" <sebastian at sipsolutions.net>
wrote:

> On Wed, 2013-10-16 at 11:50 -0400, Benjamin Root wrote:
> >
> >
> >
> > On Wed, Oct 16, 2013 at 11:39 AM, Chad Kidder <cckidder at gmail.com>
> > wrote:
> >         Just found what should be a bug in 1.7.1.  I'm running
> >         python(x,y) on windows here:
> >
> >         >>> dataMatrix[ii,:,mask].shape
> >         (201, 23)
> >         >>> dataMatrix[ii,:,:].shape
> >         (23, 201)
> >         >>> dataMatrix.shape
> >         (24, 23, 201)
> >         >>> mask
> >         array([ True,  True,  True,  True,  True,  True,  True,  True,
> >         True,
> >                 ...
> >                 True,  True,  True], dtype=bool)
> >
> >
> >         using a mask should not change the order of the dimensions.
> >         Is there a reason for this behavior, and if so, how do I avoid
> >         it in the future?  Thanks
> >
> >
> >              --Chad Kidder
> >
> >
> >
> >
> > Chad,
> >
> > The issue here is one where there is the mixing of fancy indexing (I
> > presume that is what "ii" is), slicing and boolean indexing. If I
> > remember correctly, the changing of the dimension orders was an
> > inadvertent byproduct of handing all this array accessing methods in
> > one shot. I think this was addressed in 1.8. Sorry for being very
> > brief and vague, hopefully someone else who understands what the
> > resolution was can fill in.
> >
> Yes, in fact `ii` can just be a normal integer, since an integer *is*
> considered an advanced/fancy index (in the sense that it forces
> transposing, not in the sense that it forces a copy by itself, so
> integers are *both* advanced and view based indices!).
>
> This is how advanced/fancy indexing works, there is `np.ix_` which helps
> in some cases, but not exactly for your problem. For a more detailed
> description of fancy indexing check:
>
> http://docs.scipy.org/doc/numpy/reference/arrays.indexing.html#advanced-indexing
>
> You have a slice between the mask and the integer `arr[1, :, mask]`,
> which means the `mask` result dimension is transposed to the front. It
> would not be if it was `arr[:, 1, mask]` (since numpy can actually guess
> where it came from in that case).
>
> Since you seem to always have exactly one advanced (mask) index, in your
> example, the simplest solution is probably:
> `dataMatrix[ii,...][:,mask]`
> (first view based slicing, then the advanced boolean index. Since the
> first part (if `ii` is an integer) will not copy the data, this will
> also work for assignments).
>
> - Sebastian
>
>
> > Cheers!
> > Ben Root
> >
> >
> >
> > _______________________________________________
> > NumPy-Discussion mailing list
> > NumPy-Discussion at scipy.org
> > http://mail.scipy.org/mailman/listinfo/numpy-discussion
>
>
> _______________________________________________
> NumPy-Discussion mailing list
> NumPy-Discussion at scipy.org
> http://mail.scipy.org/mailman/listinfo/numpy-discussion
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20131016/f28fc3b3/attachment.html>

From d.l.goldsmith at gmail.com  Wed Oct 16 19:06:36 2013
From: d.l.goldsmith at gmail.com (David Goldsmith)
Date: Wed, 16 Oct 2013 16:06:36 -0700
Subject: [Numpy-discussion] OT: How are SVG data converted into curves
Message-ID: <CAFtPsZqPYD=ZhfQwffvpr9hy2QHxv+R=+DTmFd2AuK2-pHBnHg@mail.gmail.com>

Does anyone on this list know how Scalable Vector Graphics C, S, etc.
command data are translated into curves (i.e., pixel maps) and might you be
willing to answer some questions off-list?  Thanks!

DG

PS: I receive numpy-discussion in digest mode, so if you "qualify," please
reply directly to my email.  Thanks again.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20131016/8fe86e57/attachment.html>

From daniele at grinta.net  Wed Oct 16 19:23:04 2013
From: daniele at grinta.net (Daniele Nicolodi)
Date: Thu, 17 Oct 2013 01:23:04 +0200
Subject: [Numpy-discussion] OT: How are SVG data converted into curves
In-Reply-To: <CAFtPsZqPYD=ZhfQwffvpr9hy2QHxv+R=+DTmFd2AuK2-pHBnHg@mail.gmail.com>
References: <CAFtPsZqPYD=ZhfQwffvpr9hy2QHxv+R=+DTmFd2AuK2-pHBnHg@mail.gmail.com>
Message-ID: <525F1FD8.6070005@grinta.net>

On 17/10/2013 01:06, David Goldsmith wrote:
> Does anyone on this list know how Scalable Vector Graphics C, S, etc.
> command data are translated into curves (i.e., pixel maps) and might you
> be willing to answer some questions off-list?  Thanks!

Hello,

the format is quite clearly documented. For the path data you can see
http://www.w3.org/TR/SVG/paths.html. There are several open source
libraries that implement SVG rendering, you may look at those to see how
the rendering can be implemented in practice.

Cheers,
Daniele





From d.l.goldsmith at gmail.com  Wed Oct 16 21:56:54 2013
From: d.l.goldsmith at gmail.com (David Goldsmith)
Date: Wed, 16 Oct 2013 18:56:54 -0700
Subject: [Numpy-discussion] On Topic: Faster way to implement Bernstein
	polys: explicit or recursion?
Message-ID: <CAFtPsZouZCqaxVYdM_T1w+3yamBioOQ8CfGZRth+Kz2A3QOL0A@mail.gmail.com>

Many thanks to Daniele Nicolodi for pointing me to the Wikipedia article
on B?zier curves.  Said article gives two formulae for the B?zier curve of
degree n: one explicit, one recursive.  Using numpy.polynomial.Polynomial
as the base class, and its evaluation method for the evaluation in each
dimension, which approach is likely to be faster for evaluation at
parameter t? Does it depend on the degree, i.e., one approach will likely
be faster for low degree while the other will likely be faster for higher
degree?  Thanks!

DG
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20131016/94b736b6/attachment.html>

From piet at vanoostrum.org  Wed Oct 16 22:37:45 2013
From: piet at vanoostrum.org (Piet van Oostrum)
Date: Wed, 16 Oct 2013 22:37:45 -0400
Subject: [Numpy-discussion] NumPy 1.8.0rc2 release
References: <CAB6mnxJxnG5nPwbpoaMhh1QobSGj9A6=5NcrLRDRimDDgyx9rA@mail.gmail.com>
Message-ID: <m261swfwqu.fsf@cochabamba.vanoostrum.org>

Charles R Harris <charlesr.harris at gmail.com> writes:

> Hi All,
>
> NumPy 1.8.0rc2 is up now on sourceforge. Binary builds are included, except for Python 3.3 on
> windows. Many thanks to Ralf for the binaries and to those who found and fixed the bugs in rc1.
> Please test this thoroughly, especially if you have access to one of the less common platforms.
> Testing of rc1 turned up several bugs that would have been a embarrassment if they had made their
> way into the release and we are very happy that they were discovered.

I had to remove the previous numpy installation from site-packages
(Python 3.3.2, Mac OS X 10.6.8). Without that removal the tests failed,
because a non functioning test file had remained behind. Or maybe some
more files. The previous was 1.0.8rc1, IIRC.
-- 
Piet van Oostrum <piet at vanoostrum.org>
WWW: http://pietvanoostrum.com/
PGP key: [8DAE142BE17999C4]



From poojagupta.135 at gmail.com  Fri Oct 18 06:33:12 2013
From: poojagupta.135 at gmail.com (Pooja Gupta)
Date: Fri, 18 Oct 2013 16:03:12 +0530
Subject: [Numpy-discussion] how to do scoring of random point to avoid
	overlapping in python
Message-ID: <CAGCW8uv4YGhwk33gxaF=hQ0A55w8N_OUS5bagKRDA1fPSQJciw@mail.gmail.com>

I have generated random point around a object and then evaluate each random
point on certain criteria. But problem is that every time I am getting new
point. How i can resolve this problem so that my result should be uniform.
Is any way to evaluate the position of random point.


for arang in range(1000):                # generate 1000 random point
around object
    arang = arang + 1
    x,y,z = 9.251, 24.410, 64.133       # coordinates of objects (i
have 500 object coordinates)

    x1,y1,z1 = (uniform(x-3.5,x+3.5),uniform(y-3.5,y+3.5),uniform(z-3.5,z+3.5))
#randompoint
    pacord = [x1,y1,z1]               #random point coordinates
    dist_pap = euDist(uacoord, pacord)      # check distance between
object and random points

    if  (dist_pap > 2.5):                   # if the random point far from obect
        dist_pap1 = dist_pap
vecpw = euvector(uacoord, pacord)      # generate vectors b/w objject
and random point

# angle between angle between object and random point
num1 = np.dot (vect1, vecpw)
denom1 = np.linalg.norm(vect1) * np.linalg.norm(vecpw)
ang1 = rad2deg(np.arccos(num1/denom1))
if 140 > ang1 >100:    # check angle
    ang2= ang1print pacord


Queries
every time i am getting new result (new positions of the random
point). How to fix it.
on above basis I want to score each random point and the two random
point should be 2.5 distance apart from each other. How I can avoid
overlapping of the random points.



-- 
???? ??????, ??????
????????? ?????? ??????? ??????

"???? ?? ??? ?? ?????????? ??, ??? ?? ???? ???? ??? ?? ?
?? ???? ?? ?? ???????? ??, ?? ???? ?????? ?? ????? ????"
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20131018/91e8ee34/attachment.html>

From klemm at phys.ethz.ch  Fri Oct 18 10:51:36 2013
From: klemm at phys.ethz.ch (Hanno Klemm)
Date: Fri, 18 Oct 2013 16:51:36 +0200
Subject: [Numpy-discussion] how to do scoring of random point to avoid
 overlapping in python
In-Reply-To: <CAGCW8uv4YGhwk33gxaF=hQ0A55w8N_OUS5bagKRDA1fPSQJciw@mail.gmail.com>
References: <CAGCW8uv4YGhwk33gxaF=hQ0A55w8N_OUS5bagKRDA1fPSQJciw@mail.gmail.com>
Message-ID: <a8c9038d260d18ab6900b5787075d976@phys.ethz.ch>

On 18.10.2013 12:33, Pooja Gupta wrote:
> I have generated random point around a object and then evaluate each
> random point on certain criteria. But problem is that every time I am
> getting new point. How i can resolve this problem so that my result
> should be uniform. Is any way to evaluate the position of random
> point.
> 
> for arang in range(1000): # generate 1000 random point around object
>  arang = arang + 1
>  x,y,z = 9.251, 24.410, 64.133 # coordinates of objects (i have 500
> object coordinates)
> 
>  x1,y1,z1 =
> (uniform(x-3.5,x+3.5),uniform(y-3.5,y+3.5),uniform(z-3.5,z+3.5))
> #randompoint
>  pacord = [x1,y1,z1] #random point coordinates
>  dist_pap = euDist(uacoord, pacord) # check distance between object
> and random points
> 
>  if (dist_pap > 2.5): # if the random point far from obect
>  dist_pap1 = dist_pap
> vecpw = euvector(uacoord, pacord) # generate vectors b/w objject and
> random point
> 
> # angle between angle between object and random point
> num1 = np.dot (vect1, vecpw)
> denom1 = np.linalg.norm(vect1) * np.linalg.norm(vecpw)
> ang1 = rad2deg(np.arccos(num1/denom1))
> 
> if 140 > ang1 >100: # check angle
>  ang2= ang1
> print pacord
> 
> Queries
> every time i am getting new result (new positions of the random
> point). How to fix it.
> on above basis I want to score each random point and the two random
> point should be 2.5 distance apart from each other. How I can avoid
> overlapping of the random points.
> 

I am not sure if i understand the question correctly but if you always 
want to get the same random number, every time you run the script, you 
can fix the random seed by using np.random.seed(). Regarding your second 
question, when I understand correctly, you somehow want to find two 
points that have a distance of 2.5. If you already have one of them, I 
would generate the second one using spherical coordinates and specifying 
the distance a priori. something like:

def pt2(pt1, distance=2.5):
     theta = np.random.rand()*np.pi
     phi = np.random.rand()*2*np.pi
     r = distance
     x = r*np.sin(theta)*np.cos(phi)
     y = r*np.sin(theta)*np.sin(phi)
     z = r*np.cos(theta)
     return pt1 + np.array((x,y,z))


In [367]: pt1 = np.array([0,0,0])

In [368]: pt2(pt1)
Out[368]: a = array([-2.29954368, -0.57223342,  0.79664785])

In [369]: np.linalg.norm(a)
Out[369]: 2.5

Hope this helps,
Hanno



From poojagupta.135 at gmail.com  Fri Oct 18 10:58:28 2013
From: poojagupta.135 at gmail.com (Pooja Gupta)
Date: Fri, 18 Oct 2013 20:28:28 +0530
Subject: [Numpy-discussion] how to do scoring of random point to avoid
 overlapping in python
In-Reply-To: <a8c9038d260d18ab6900b5787075d976@phys.ethz.ch>
References: <CAGCW8uv4YGhwk33gxaF=hQ0A55w8N_OUS5bagKRDA1fPSQJciw@mail.gmail.com>
	<a8c9038d260d18ab6900b5787075d976@phys.ethz.ch>
Message-ID: <CAGCW8uufq5vyJGMAp6AOH_VkfPUYocpiLqSBw23gN5dWPcGeUQ@mail.gmail.com>

Thanks Hanno
I got some idea.
How about the bin(grid)??????



On Fri, Oct 18, 2013 at 8:21 PM, Hanno Klemm <klemm at phys.ethz.ch> wrote:

> On 18.10.2013 12:33, Pooja Gupta wrote:
> > I have generated random point around a object and then evaluate each
> > random point on certain criteria. But problem is that every time I am
> > getting new point. How i can resolve this problem so that my result
> > should be uniform. Is any way to evaluate the position of random
> > point.
> >
> > for arang in range(1000): # generate 1000 random point around object
> >  arang = arang + 1
> >  x,y,z = 9.251, 24.410, 64.133 # coordinates of objects (i have 500
> > object coordinates)
> >
> >  x1,y1,z1 =
> > (uniform(x-3.5,x+3.5),uniform(y-3.5,y+3.5),uniform(z-3.5,z+3.5))
> > #randompoint
> >  pacord = [x1,y1,z1] #random point coordinates
> >  dist_pap = euDist(uacoord, pacord) # check distance between object
> > and random points
> >
> >  if (dist_pap > 2.5): # if the random point far from obect
> >  dist_pap1 = dist_pap
> > vecpw = euvector(uacoord, pacord) # generate vectors b/w objject and
> > random point
> >
> > # angle between angle between object and random point
> > num1 = np.dot (vect1, vecpw)
> > denom1 = np.linalg.norm(vect1) * np.linalg.norm(vecpw)
> > ang1 = rad2deg(np.arccos(num1/denom1))
> >
> > if 140 > ang1 >100: # check angle
> >  ang2= ang1
> > print pacord
> >
> > Queries
> > every time i am getting new result (new positions of the random
> > point). How to fix it.
> > on above basis I want to score each random point and the two random
> > point should be 2.5 distance apart from each other. How I can avoid
> > overlapping of the random points.
> >
>
> I am not sure if i understand the question correctly but if you always
> want to get the same random number, every time you run the script, you
> can fix the random seed by using np.random.seed(). Regarding your second
> question, when I understand correctly, you somehow want to find two
> points that have a distance of 2.5. If you already have one of them, I
> would generate the second one using spherical coordinates and specifying
> the distance a priori. something like:
>
> def pt2(pt1, distance=2.5):
>      theta = np.random.rand()*np.pi
>      phi = np.random.rand()*2*np.pi
>      r = distance
>      x = r*np.sin(theta)*np.cos(phi)
>      y = r*np.sin(theta)*np.sin(phi)
>      z = r*np.cos(theta)
>      return pt1 + np.array((x,y,z))
>
>
> In [367]: pt1 = np.array([0,0,0])
>
> In [368]: pt2(pt1)
> Out[368]: a = array([-2.29954368, -0.57223342,  0.79664785])
>
> In [369]: np.linalg.norm(a)
> Out[369]: 2.5
>
> Hope this helps,
> Hanno
>
> _______________________________________________
> NumPy-Discussion mailing list
> NumPy-Discussion at scipy.org
> http://mail.scipy.org/mailman/listinfo/numpy-discussion
>



-- 
???? ??????, ??????
????????? ?????? ??????? ??????

"???? ?? ??? ?? ?????????? ??, ??? ?? ???? ???? ??? ?? ?
?? ???? ?? ?? ???????? ??, ?? ???? ?????? ?? ????? ????"
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20131018/b609ced5/attachment.html>

From chris.barker at noaa.gov  Fri Oct 18 15:55:45 2013
From: chris.barker at noaa.gov (Chris Barker)
Date: Fri, 18 Oct 2013 12:55:45 -0700
Subject: [Numpy-discussion] Equivalent to IDL's help function
In-Reply-To: <765BF3F2-7FE1-414E-A134-F363D841D0DC@staffmail.ed.ac.uk>
References: <765BF3F2-7FE1-414E-A134-F363D841D0DC@staffmail.ed.ac.uk>
Message-ID: <CALGmxEJd3UeRSgBMimoYiqXq=j_20tWBO0PL1C=XTy7vqdcceQ@mail.gmail.com>

On Mon, Oct 7, 2013 at 10:15 AM, Siegfried Gonzi
<sgonzi at staffmail.ed.ac.uk> wrote:
> What is the equivalent to IDL its help function, e.g.
>
> ==
> IDL> a = make_array(23,23,)
>
> IDL> help,a
>
> will result in:
>
> A               FLOAT     = Array[23, 23]

am I missing something, or is this what you get when you type a name
on the command line:

In [15]: a = np.ones((2,3), dtype=np.float32)

In [16]: a
Out[16]:
array([[ 1.,  1.,  1.],
       [ 1.,  1.,  1.]], dtype=float32)

or print a

etc...


-- 

Christopher Barker, Ph.D.
Oceanographer

Emergency Response Division
NOAA/NOS/OR&R            (206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115       (206) 526-6317   main reception

Chris.Barker at noaa.gov


From chris.barker at noaa.gov  Fri Oct 18 15:59:43 2013
From: chris.barker at noaa.gov (Chris Barker)
Date: Fri, 18 Oct 2013 12:59:43 -0700
Subject: [Numpy-discussion] OT: How are SVG data converted into curves
In-Reply-To: <525F1FD8.6070005@grinta.net>
References: <CAFtPsZqPYD=ZhfQwffvpr9hy2QHxv+R=+DTmFd2AuK2-pHBnHg@mail.gmail.com>
	<525F1FD8.6070005@grinta.net>
Message-ID: <CALGmxELgu1zN5f9ih==d5kNRNfarJj+x3PZQAo6x7s6RDjEdHg@mail.gmail.com>

On Wed, Oct 16, 2013 at 4:23 PM, Daniele Nicolodi <daniele at grinta.net> wrote:

> the format is quite clearly documented. For the path data you can see
> http://www.w3.org/TR/SVG/paths.html. There are several open source
> libraries that implement SVG rendering, you may look at those to see how
> the rendering can be implemented in practice.

InkScape is particularly nice -- it's an open source desktop vector
drawing program, but can also be called on the command line and used
as a SVG renderer.

Someone wrote a minimal wxPython-based SVG renderer a while back --
not sure what happened to that.

-Chris




-- 

Christopher Barker, Ph.D.
Oceanographer

Emergency Response Division
NOAA/NOS/OR&R            (206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115       (206) 526-6317   main reception

Chris.Barker at noaa.gov


From raul at virtualmaterials.com  Fri Oct 18 16:34:44 2013
From: raul at virtualmaterials.com (Raul Cota)
Date: Fri, 18 Oct 2013 14:34:44 -0600
Subject: [Numpy-discussion] switching from Numeric to numpy
In-Reply-To: <188EE056906.00000B14fernando.j@inbox.com>
References: <188EE056906.00000B14fernando.j@inbox.com>
Message-ID: <52619B64.3020508@virtualmaterials.com>

John,


Just noticed this message,

We are already cleaning up all of our code to not be numpy based but for 
porting from Numeric to numpy:


In our C code we settled for the following,

#define NUMPY

#if !defined(NUMPY)
     #include "arrayobject.h"
#else
     #include "numpy/oldnumeric.h"
#endif


And this,

#ifdef NUMPY
     pyArray = (PyArrayObject *) PyArray_SimpleNew( 1, dims, 
PyArray_DOUBLE);
#else
     pyArray = (PyArrayObject *) PyArray_FromDims( 1, dims, PyArray_DOUBLE);
#endif


And I think that was all the changes we needed and while we were 
transitioning we could easily compile for Numeric or numpy.


Most of our headaches were on the python code because of very subtle 
things but even there, the bulk of the changes worked out rather nicely 
using the oldnumeric stuff.


Raul




On 07/10/2013 5:27 PM, john fernando wrote:
> I have the following C code which is an extension to my python code.
>
> The python and C code use
> #include <Numeric/arrayobject.h>
>
> what is the equivalent I can use in numpy that causes the minimum code change?
>
> I did look through the old messages but didn't really find the answer-any help to a novice just starting out in numpy/python/C is much appreciated.
>
> ____________________________________________________________
> FREE ONLINE PHOTOSHARING - Share your photos online with your friends and family!
> Visit http://www.inbox.com/photosharing to find out more!
>
>
> _______________________________________________
> NumPy-Discussion mailing list
> NumPy-Discussion at scipy.org
> http://mail.scipy.org/mailman/listinfo/numpy-discussion
>



From robert.kern at gmail.com  Sat Oct 19 06:37:09 2013
From: robert.kern at gmail.com (Robert Kern)
Date: Sat, 19 Oct 2013 11:37:09 +0100
Subject: [Numpy-discussion] OT: How are SVG data converted into curves
In-Reply-To: <CAFtPsZqPYD=ZhfQwffvpr9hy2QHxv+R=+DTmFd2AuK2-pHBnHg@mail.gmail.com>
References: <CAFtPsZqPYD=ZhfQwffvpr9hy2QHxv+R=+DTmFd2AuK2-pHBnHg@mail.gmail.com>
Message-ID: <CAF6FJivoHGFWny7jh4kvMkadfwwB1G6Bjc=N1rOq77jC6XMAdg@mail.gmail.com>

On Thu, Oct 17, 2013 at 12:06 AM, David Goldsmith <d.l.goldsmith at gmail.com>
wrote:
>
> Does anyone on this list know how Scalable Vector Graphics C, S, etc.
command data are translated into curves (i.e., pixel maps) and might you be
willing to answer some questions off-list?  Thanks!

It's not clear if you are asking about how the SVG path command are parsed,
normalized, and interpreted into a path composed of vector primitives or
how those paths actually get rendered into pixels. If the former, you may
be interested in the svg.path package which does exactly that (and only
that). Most vector drawing libraries (Agg, the painting API of any
reasonable GUI toolkit) will be able to implement those primitives to draw
onto a pixel image.

  https://pypi.python.org/pypi/svg.path/

--
Robert Kern
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20131019/05029771/attachment.html>

From ralf.gommers at gmail.com  Sat Oct 19 17:40:15 2013
From: ralf.gommers at gmail.com (Ralf Gommers)
Date: Sat, 19 Oct 2013 23:40:15 +0200
Subject: [Numpy-discussion] ANN: SciPy 0.13.0 release
Message-ID: <CABL7CQi5B0qdbWMQBrVwRsZ33BGsV2ATaUCEAbA5rjNjm=iyoQ@mail.gmail.com>

On behalf of the SciPy development team I'm pleased to announce the
availability of SciPy 0.13.0. This release contains some interesting new
features (see highlights below) and half a year's worth of maintenance
work. 65 people contributed to this release.

Some of the highlights are:

  - support for fancy indexing and boolean comparisons with sparse matrices
  - interpolative decompositions and matrix functions in the linalg module
  - two new trust-region solvers for unconstrained minimization

This release requires Python 2.6, 2.7 or 3.1-3.3 and NumPy 1.5.1 or
greater. Support for Python 2.4 and 2.5 has been dropped as of this release.

Sources and binaries can be found at
http://sourceforge.net/projects/scipy/files/scipy/0.13.0/, release notes
are copied below.

Enjoy,
Ralf




==========================
SciPy 0.13.0 Release Notes
==========================

.. contents::

SciPy 0.13.0 is the culmination of 7 months of hard work. It contains
many new features, numerous bug-fixes, improved test coverage and
better documentation.  There have been a number of deprecations and
API changes in this release, which are documented below.  All users
are encouraged to upgrade to this release, as there are a large number
of bug-fixes and optimizations.  Moreover, our development attention
will now shift to bug-fix releases on the 0.13.x branch, and on adding
new features on the master branch.

This release requires Python 2.6, 2.7 or 3.1-3.3 and NumPy 1.5.1 or greater.
Highlights of this release are:

  - support for fancy indexing and boolean comparisons with sparse matrices
  - interpolative decompositions and matrix functions in the linalg module
  - two new trust-region solvers for unconstrained minimization


New features
============

``scipy.integrate`` improvements
--------------------------------

N-dimensional numerical integration
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

A new function `scipy.integrate.nquad`, which provides N-dimensional
integration functionality with a more flexible interface than ``dblquad``
and
``tplquad``, has been added.

``dopri*`` improvements
^^^^^^^^^^^^^^^^^^^^^^^

The intermediate results from the ``dopri`` family of ODE solvers can now be
accessed by a *solout* callback function.


``scipy.linalg`` improvements
-----------------------------

Interpolative decompositions
^^^^^^^^^^^^^^^^^^^^^^^^^^^^

Scipy now includes a new module `scipy.linalg.interpolative`
containing routines for computing interpolative matrix decompositions
(ID). This feature is based on the ID software package by
P.G. Martinsson, V. Rokhlin, Y. Shkolnisky, and M. Tygert, previously
adapted for Python in the PymatrixId package by K.L. Ho.

Polar decomposition
^^^^^^^^^^^^^^^^^^^

A new function `scipy.linalg.polar`, to compute the polar decomposition
of a matrix, was added.

BLAS level 3 functions
^^^^^^^^^^^^^^^^^^^^^^

The BLAS functions ``symm``, ``syrk``, ``syr2k``, ``hemm``, ``herk`` and
``her2k`` are now wrapped in `scipy.linalg`.

Matrix functions
^^^^^^^^^^^^^^^^

Several matrix function algorithms have been implemented or updated
following
detailed descriptions in recent papers of Nick Higham and his co-authors.
These include the matrix square root (``sqrtm``), the matrix logarithm
(``logm``), the matrix exponential (``expm``) and its Frechet derivative
(``expm_frechet``), and fractional matrix powers
(``fractional_matrix_power``).


``scipy.optimize`` improvements
-------------------------------

Trust-region unconstrained minimization algorithms
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

The ``minimize`` function gained two trust-region solvers for unconstrained
minimization: ``dogleg`` and ``trust-ncg``.


``scipy.sparse`` improvements
-----------------------------

Boolean comparisons and sparse matrices
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

All sparse matrix types now support boolean data, and boolean operations.
Two
sparse matrices `A` and `B` can be compared in all the expected ways `A <
B`,
`A >= B`, `A != B`, producing similar results as dense Numpy arrays.
Comparisons with dense matrices and scalars are also supported.

CSR and CSC fancy indexing
^^^^^^^^^^^^^^^^^^^^^^^^^^

Compressed sparse row and column sparse matrix types now support fancy
indexing
with boolean matrices, slices, and lists. So where A is a (CSC or CSR)
sparse
matrix, you can do things like::

    >>> A[A > 0.5] = 1  # since Boolean sparse matrices work
    >>> A[:2, :3] = 2
    >>> A[[1,2], 2] = 3


``scipy.sparse.linalg`` improvements
------------------------------------

The new function ``onenormest`` provides a lower bound of the 1-norm of a
linear operator and has been implemented according to Higham and Tisseur
(2000).  This function is not only useful for sparse matrices, but can also
be
used to estimate the norm of products or powers of dense matrices without
explictly building the intermediate matrix.

The multiplicative action of the matrix exponential of a linear operator
(``expm_multiply``) has been implemented following the description in
Al-Mohy
and Higham (2011).

Abstract linear operators (`scipy.sparse.linalg.LinearOperator`) can now be
multiplied, added to each other, and exponentiated, producing new linear
operators. This enables easier construction of composite linear operations.


``scipy.spatial`` improvements
------------------------------

The vertices of a `ConvexHull` can now be accessed via the `vertices`
attribute,
which gives proper orientation in 2-D.


``scipy.signal`` improvements
-----------------------------

The cosine window function `scipy.signal.cosine` was added.


``scipy.special`` improvements
------------------------------

New functions `scipy.special.xlogy` and `scipy.special.xlog1py` were added.
These functions can simplify and speed up code that has to calculate
``x * log(y)`` and give 0 when ``x == 0``.


``scipy.io`` improvements
-------------------------

Unformatted Fortran file reader
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

The new class `scipy.io.FortranFile` facilitates reading unformatted
sequential files written by Fortran code.

``scipy.io.wavfile`` enhancements
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

`scipy.io.wavfile.write` now accepts a file buffer. Previously it only
accepted a filename.

`scipy.io.wavfile.read` and `scipy.io.wavfile.write` can now handle floating
point WAV files.


``scipy.interpolate`` improvements
----------------------------------

B-spline derivatives and antiderivatives
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

`scipy.interpolate.splder` and `scipy.interpolate.splantider` functions
for computing B-splines that represent derivatives and antiderivatives
of B-splines were added.  These functions are also available in the
class-based FITPACK interface as ``UnivariateSpline.derivative`` and
``UnivariateSpline.antiderivative``.


``scipy.stats`` improvements
----------------------------

Distributions now allow using keyword parameters in addition to
positional parameters in all methods.

The function `scipy.stats.power_divergence` has been added for the
Cressie-Read power divergence statistic and goodness of fit test.
Included in this family of statistics is the "G-test"
(http://en.wikipedia.org/wiki/G-test).

`scipy.stats.mood` now accepts multidimensional input.

An option was added to `scipy.stats.wilcoxon` for continuity correction.

`scipy.stats.chisquare` now has an `axis` argument.

`scipy.stats.mstats.chisquare` now has `axis` and `ddof` arguments.


Deprecated features
===================

``expm2`` and ``expm3``
-----------------------

The matrix exponential functions `scipy.linalg.expm2` and
`scipy.linalg.expm3`
are deprecated. All users should use the numerically more robust
`scipy.linalg.expm` function instead.

``scipy.stats`` functions
-------------------------

`scipy.stats.oneway` is deprecated; `scipy.stats.f_oneway` should be used
instead.

`scipy.stats.glm` is deprecated.  `scipy.stats.ttest_ind` is an equivalent
function; more full-featured general (and generalized) linear model
implementations can be found in statsmodels.

`scipy.stats.cmedian` is deprecated; ``numpy.median`` should be used
instead.


Backwards incompatible changes
==============================

LIL matrix assignment
---------------------
Assigning values to LIL matrices with two index arrays now works similarly
as
assigning into ndarrays::

    >>> x = lil_matrix((3, 3))
    >>> x[[0,1,2],[0,1,2]]=[0,1,2]
    >>> x.todense()
    matrix([[ 0.,  0.,  0.],
            [ 0.,  1.,  0.],
            [ 0.,  0.,  2.]])

rather than giving the result::

    >>> x.todense()
    matrix([[ 0.,  1.,  2.],
            [ 0.,  1.,  2.],
            [ 0.,  1.,  2.]])

Users relying on the previous behavior will need to revisit their code.
The previous behavior is obtained by ``x[numpy.ix_([0,1,2],[0,1,2])] = ...`.


Deprecated ``radon`` function removed
-------------------------------------

The ``misc.radon`` function, which was deprecated in scipy 0.11.0, has been
removed.  Users can find a more full-featured ``radon`` function in
scikit-image.


Removed deprecated keywords ``xa`` and ``xb`` from ``stats.distributions``
--------------------------------------------------------------------------

The keywords ``xa`` and ``xb``, which were deprecated since 0.11.0, have
been removed from the distributions in ``scipy.stats``.

Changes to MATLAB file readers / writers
----------------------------------------

The major change is that 1D arrays in numpy now become row vectors (shape
1, N)
when saved to a MATLAB 5 format file.  Previously 1D arrays saved as column
vectors (N, 1).  This is to harmonize the behavior of writing MATLAB 4 and 5
formats, and adapt to the defaults of numpy and MATLAB - for example
``np.atleast_2d`` returns 1D arrays as row vectors.

Trying to save arrays of greater than 2 dimensions in MATLAB 4 format now
raises
an error instead of silently reshaping the array as 2D.

``scipy.io.loadmat('afile')`` used to look for `afile` on the Python system
path
(``sys.path``); now ``loadmat`` only looks in the current directory for a
relative path filename.


Other changes
=============

Security fix: ``scipy.weave`` previously used temporary directories in an
insecure manner under certain circumstances.

Cython is now required to build *unreleased* versions of scipy.
The C files generated from Cython sources are not included in the git repo
anymore.  They are however still shipped in source releases.

The code base received a fairly large PEP8 cleanup.  A ``tox pep8``
command has been added; new code should pass this test command.

Scipy cannot be compiled with gfortran 4.1 anymore (at least on RH5), likely
due to that compiler version not supporting entry constructs well.


Authors
=======

This release contains work by the following people (contributed at least
one patch to this release, names in alphabetical order):

* Jorge Ca?ardo Alastuey +
* Tom Aldcroft +
* Max Bolingbroke +
* Joseph Jon Booker +
* Fran?ois Boulogne
* Matthew Brett
* Christian Brodbeck +
* Per Brodtkorb +
* Christian Brueffer +
* Lars Buitinck
* Evgeni Burovski +
* Tim Cera
* Lawrence Chan +
* David Cournapeau
* Draz?en Luc?anin +
* Alexander J. Dunlap +
* endolith
* Andr? Gaul +
* Christoph Gohlke
* Ralf Gommers
* Alex Griffing +
* Blake Griffith +
* Charles Harris
* Bob Helmbold +
* Andreas Hilboll
* Kat Huang +
* Oleksandr (Sasha) Huziy +
* Gert-Ludwig Ingold +
* Thouis (Ray) Jones
* Juan Luis Cano Rodr?guez +
* Robert Kern
* Andreas Kloeckner +
* Sytse Knypstra +
* Gustav Larsson +
* Denis Laxalde
* Christopher Lee
* Tim Leslie
* Wendy Liu +
* Clemens Novak +
* Takuya Oshima +
* Josef Perktold
* Illia Polosukhin +
* Przemek Porebski +
* Steve Richardson +
* Branden Rolston +
* Skipper Seabold
* Fazlul Shahriar
* Leo Singer +
* Rohit Sivaprasad +
* Daniel B. Smith +
* Julian Taylor
* Louis Thibault +
* Tomas Tomecek +
* John Travers
* Richard Tsai +
* Jacob Vanderplas
* Patrick Varilly
* Pauli Virtanen
* Stefan van der Walt
* Warren Weckesser
* Pedro Werneck +
* Nils Werner +
* Michael Wimmer +
* Nathan Woods +
* Tony S. Yu +

A total of 65 people contributed to this release.
People with a "+" by their names contributed a patch for the first time.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20131019/e215d455/attachment.html>

From charlesr.harris at gmail.com  Sun Oct 20 13:30:25 2013
From: charlesr.harris at gmail.com (Charles R Harris)
Date: Sun, 20 Oct 2013 11:30:25 -0600
Subject: [Numpy-discussion] Preparing for 1.7.2 release
Message-ID: <CAB6mnx+=jgv-iMaBf-tDTpEaU5Gc4REPvZ-2jR0uh4K_ich1-A@mail.gmail.com>

Hi All,

After the release of Numpy 1.8.0 there will be a point release of the long
term stable branch, Numpy 1.7.2. There have been some 86 non-merge commits
since 1.7.1 was released, see the attached list. If anyone thinks something
is missing and needs a backport, please reply on the list.

Chuck
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20131020/adce549f/attachment.html>
-------------- next part --------------
commit c5d83e206617f79e576fc3214cb8fd67d11ca85a
Author: Christoph Dann <cdann at cdann.de>
Date:   Wed Oct 16 15:17:58 2013 -0400

    added spaces to literal string concats
    
    literal string concatination without spaces not
    allowed in C++11 anymore

commit 0093c8864c43397d918e1ef64782f89bd6aec5ee
Author: cgohlke <cgohlke at uci.edu>
Date:   Tue Oct 15 14:17:40 2013 -0700

    BUG: close file-like objects returned by urlopen
    
    Fix `ResourceWarning: unclosed file` on Python 3

commit bf4c6b2ec0191b9c2ae1129edccb8904a4621d9d
Author: Marten van Kerkwijk <mhvk at astro.utoronto.ca>
Date:   Sat Oct 12 19:50:43 2013 -0400

    Small corrections to tests

commit 1b283cf0eae908138daa34b5777784cac64cb115
Author: Marten van Kerkwijk <mhvk at astro.utoronto.ca>
Date:   Sat Oct 12 16:06:14 2013 -0400

    Add test cases to ensure NotImplemented is passed on

commit 72bb6ad851f60ace43ea2470c942e015e4db3231
Author: Marten van Kerkwijk <mhvk at astro.utoronto.ca>
Date:   Sat Oct 12 12:11:37 2013 -0400

    Ensure NotImplemented is passed on in MaskedArray ufunc's

commit 4a4e97b745d2fe6412c3c045b5e0d8fa64440921
Author: Julian Taylor <jtaylor.debian at googlemail.com>
Date:   Sat Oct 12 14:06:16 2013 +0200

    BUG: Disable check for oldstyle classes in python3
    
    backport of #3885

commit d69cd4e01a41d5e8e1615723df728a054eaff24a
Author: Charles Harris <charlesr.harris at gmail.com>
Date:   Sun Oct 6 10:32:55 2013 -0600

    BUG: Include sunmath.h in npy_math.h when __SUNPRO_CC defined.
    
    The include is needed for isinf.

commit 92adb806f705fed63ba20e0bc5f67fa2f7d500a3
Author: Ralf Gommers <ralf.gommers at googlemail.com>
Date:   Sun May 19 22:15:09 2013 +0200

    BLD: fix setuptools-specific easy_install issue.  Closes gh-3160.
    
    An error is raised by setuptools when trying to write to /dev/null.  Was fixed
    in distribute, but not in setuptools.
    
    No multi-arch support with plain setuptools should be OK, because multi-arch is
    Ubuntu specific (at least for now), and they ship distribute.

commit 60f2effbe552dcb5aa021d7871cbf645f2331d56
Author: Jason Madden <jamadden at gmail.com>
Date:   Sat Sep 28 10:24:37 2013 -0500

    Do not fail with NameError if SandboxViolation is raised

commit cd3e35bf9f130908bea0bafc0746c82e78e17530
Author: Charles Harris <charlesr.harris at gmail.com>
Date:   Sat Sep 28 09:47:26 2013 -0600

    TST: Remove Python 2.4 test from travis.yml.
    
    The tests always fails due to lack of Python 2.4 pip support in current
    bots. It would be good to test with 2.4 if we could find a way to work
    around the pip failure, but without that the test is just noise.

commit 21f7a535c8d396e5c8a9f4c3125a9b5c2044251b
Author: Charles Harris <charlesr.harris at gmail.com>
Date:   Fri Sep 27 17:10:44 2013 -0600

    MAINT: Get rid of a Python 3 resource warning.

commit 5b5e01b97d083035098c8d4292ddc97f30659ac5
Author: Charles Harris <charlesr.harris at gmail.com>
Date:   Fri Sep 27 16:58:44 2013 -0600

    BUG: Do not use python with statement.
    
    The python with statement is not available in Python 2.4, 2.5.

commit 52caf48a1940772b8baeca8d90a41bc7e0a43901
Author: Nathaniel J. Smith <njs at pobox.com>
Date:   Fri Sep 27 20:29:14 2013 +0100

    [fix] If we fail to raise an error, at least dump something to the console
    
    Conflicts:
    	numpy/core/tests/test_ufunc.py

commit 25de8c5d3cff5ba990ff9b47205c74a93fcb14e0
Author: Nathaniel J. Smith <njs at pobox.com>
Date:   Fri Sep 27 18:12:52 2013 +0100

    [fix] Check for errors in PyArray_CanCastTypeto DEPRECATE
    
    If a user had set warnings to raise errors, then this DEPRECATE would
    leave us with an unpropagated exception and cause havoc downstream.
    
    Unfortunately there is no way to propagate an exception from here, so
    we just have to throw it away :-(. But this is still better than the
    alternative...
    
    Conflicts:
    	numpy/core/tests/test_ufunc.py

commit a81fd3f471834b4cda31d93515070d6c636e1502
Author: Julian Taylor <jtaylor.debian at googlemail.com>
Date:   Wed Sep 25 23:28:55 2013 +0200

    BUG: fix hashing of long integers under python3
    
    python3 long_hash is more complex than int_hash so instead of copying
    into numpy call it via the Python capi.
    Same for long long for wich the numpy hash function is not correct with
    python 2.7 on i386.
    Will be slower but doesn't need adapting each timy python changes.
    closes #3793

commit 12498e684c6375e424bc50a43ef0119832f603c8
Author: Christoph Gohlke <cgohlke at uci.edu>
Date:   Sat Sep 14 11:27:22 2013 -0700

    BUG: use correct type for hash values
    
    As of Python 3.2 hash values are of type Py_hash_t. Previous versions use C long.

commit 1e2f9baa8575e721f4d010086ebe0170487653d0
Author: Julian Taylor <jtaylor.debian at googlemail.com>
Date:   Thu Sep 12 17:42:55 2013 +0200

    BUG: f2py, fix decref on wrong object
    
    missing brackets causes decref on an wrong object.
    shows itself as abort with negative refcount in test_callback using
    python-dbg.

commit 9980d0654383abd8be3d0bedaee309061b34d3e2
Author: Christoph Gohlke <cgohlke at uci.edu>
Date:   Sun Jun 9 12:33:15 2013 -0600

    MAINT: use PyOS_snprintf instead of snprintf
    
    PyOS_snprintf is portable and more secure than snprintf.

commit 8eba2aa1cbfdf927f6332609865adb9e78fc20e3
Author: Pauli Virtanen <pav at iki.fi>
Date:   Sat Apr 13 16:35:57 2013 +0300

    BUG: linalg: fix LAPACK error handling in lapack_litemodule
    
    If an exception is pending (raised from xerbla), the routines must
    return NULL.

commit 0090a01579510d01f158bcf8ea5d3f5cbb2d2792
Author: Pauli Virtanen <pav at iki.fi>
Date:   Sat Apr 13 16:14:04 2013 +0300

    BUG: linalg: do not assume that GIL is enabled in xerbla_
    
    With the new ufunc-based linalg, GIL is released in ufuncs, and needs to
    be reacquired when raising errors in xerbla_.

commit dd266a36968a322a57e86f249956e8edd941325d
Author: Christoph Gohlke <cgohlke at uci.edu>
Date:   Sun Sep 8 17:52:03 2013 -0700

    TST: add test for Python 3.4 _ast.NameConstant

commit 5116a00b08fef7ff1cc156ec64cdfa17b6b404a8
Author: Christoph Gohlke <cgohlke at uci.edu>
Date:   Sun Sep 8 16:29:30 2013 -0700

    ENH: add support for Python 3.4 ast.NameConstant

commit 61110399b8868fa36e5d847e4f2bb086796b33fb
Author: Warren Weckesser <warren.weckesser at gmail.com>
Date:   Tue Jun 4 14:42:05 2013 -0400

    TST: linalg: loosen the requirements of some tests in test_blasdot.py.  Addresses part of gh-3399.
    
    Closes #3656.

commit c4452412f6b72ac6227bb24ed1d99da2a3e724a2
Author: Christoph Gohlke <cgohlke at uci.edu>
Date:   Mon Aug 26 21:41:17 2013 -0700

    BLD: restore compatibility with ANSI C

commit b8f6adcbac74315b0188c4554527c9a3d82becfb
Author: David Cournapeau <cournape at gmail.com>
Date:   Sun Aug 25 12:12:11 2013 +0200

    REL: fix version in maintenance branch.

commit a204d08b8cc7b8ba4e9e26b70de62326be10b700
Author: Nicolas Scheffer <scheffer at speech.sri.com>
Date:   Mon Nov 12 12:00:28 2012 -0800

    ENH: Remove the need for temporary copies in numpy.dot
    
    in numpy core, modified the blas function call to handle
    C and F order arrays without the need for copy.
    This brings a significant speed up for matrix multiplication
    Added a full test for matrix multiplication in test_blasdot
    (cherry picked from commit ea037770e03f23aca1a06274a1a8e8bf0e0e2ee4)

commit 73f34e3bd36fc37c11e6ea6776d63501c9d2c3c5
Author: Mark Wiebe <mwwiebe at gmail.com>
Date:   Mon Aug 12 17:56:43 2013 -0700

    BUG: Fix additional zero-sized index case for ufunc.reduceat

commit 571e28b21a6a0d5fe52a690d95b9270789a98295
Author: Mark Wiebe <mwwiebe at gmail.com>
Date:   Mon Aug 12 16:56:38 2013 -0700

    TST: Add test for additional ufunc.reduceat empty parameter case

commit 31dd2bbc644e8542c45fbdf737415319903fb50d
Author: Mark Wiebe <mwwiebe at gmail.com>
Date:   Thu Jan 17 15:12:26 2013 -0800

    TST: Change reduceat empty test to use i4 indices
    
    On 32-bit platforms, was failing in casting the test
    i8 index to i4. It may be desirable to allow this
    in reduceat, but that change is independent of this pull
    request.

commit ef4b8af9ffc042fe91cdf3db994ad3e44ecc0775
Author: Mark Wiebe <mwwiebe at gmail.com>
Date:   Mon Jan 7 10:57:46 2013 -0800

    BUG: Fix ufunc.reduceat regression with empty index (gh-2892)

commit 2d0487f93f6c93941b9f6a89b2d0d0c65dd1801f
Author: Mark Wiebe <mwwiebe at gmail.com>
Date:   Mon Jan 7 09:04:18 2013 -0800

    TST: Add test for reduceat ufunc regression on zero-sized array

commit 8ffdb69633a123883c0a6a5570ba853bb4f411bd
Author: Mark Wiebe <mwwiebe at gmail.com>
Date:   Mon Jan 7 16:15:57 2013 -0800

    BUG: Fix to allow x.fill(tuple) where x is a struct array

commit eced0d6c65ae3ab6363e1a32ebe6207df95ebec6
Author: Mark Wiebe <mwwiebe at gmail.com>
Date:   Mon Jan 7 15:58:44 2013 -0800

    TST: Test for x.fill(tuple) where x is a struct array

commit 271cb700cba71e7de04df3175228939fa23d1df5
Author: Sebastian Berg <sebastian at sipsolutions.net>
Date:   Mon Aug 5 19:36:58 2013 +0200

    BUG: Boolean assignment allowed writing to 0-sized array
    
    This was because of the assumption that broadcasting works
    if the dimension is not > 1, but correct is != 1.
    
    Adepted from a patch provided by prossahl, backports gh-3575

commit 6ec2a54e52097eacbb150d7e9eb6413028e27109
Author: jeromekelleher <jerome.kelleher at ed.ac.uk>
Date:   Sun Jul 21 20:22:22 2013 +0200

    Update halffloat.c
    
    Corrected preprocessor form.

commit c57c417fd744453bdf8077a04cce580042545347
Author: F?lix Hartmann <felix.hartmann at crans.org>
Date:   Wed Jul 3 15:15:55 2013 +0200

    TST: add test for negative axis values in np.insert.

commit 79b094e851b0cedb7a5731816a16be7c552a9df3
Author: Sebastian Berg <sebastian at sipsolutions.net>
Date:   Sun Aug 4 19:30:56 2013 +0200

    BUG: Fix np.insert with negative axis.
    
    In some cases a negative axis argument to np.insert would result
    in wrong behaviour due to np.rollaxis, add modulo operation to
    avoid this (an error is still raised due to arr.shape[axis]).
    
    Closes gh-3494

commit 8afdd14d612d98cd0c9360cbdf4f70292a7cf497
Author: Charles Harris <charlesr.harris at gmail.com>
Date:   Tue Jul 2 18:30:06 2013 -0600

    BUG: Fix failure to return monic polynomials from roots.
    
    This bug affected the various polynomial class methods fromroots due to
    the ability to specify both window and domain. In that circumstance the
    roots are mapped from the domain to the window by the substitution
    `x = off + scl*x`. The polynomial that was being generated was monic in
    the window before substitution, but if scl was not one it was not monic
    considered as a function of the variable x in the domain. The fix is to
    divide the generated coefficients by `scl ** deg` so that the scaling of
    the highest degree term after substitution is canceled.
    
    It might be better to make the scaling optional in the future, but this
    fix makes the result match the documentation.
    
    Closes #3467.

commit 2d5fd12eb9b145545f0731ac7c5e0d8ea8f2a787
Author: Christoph Gohlke <cgohlke at uci.edu>
Date:   Tue Jul 2 20:24:13 2013 -0700

    BUG: Restore compatibility with ISO C
    
    Fixes msvc compile errors

commit 854776e16d264a18e41c93b87ce304ff5d061295
Author: Blake Griffith <blake.a.griffith at gmail.com>
Date:   Mon Jul 1 17:13:40 2013 -0500

    BUG: Class should inherit from object.

commit f66a4063e077af33d32df782815909d950160070
Author: Frederic <nouiz at nouiz.org>
Date:   Thu May 23 11:45:40 2013 -0400

    code clean up. Remove code that was never executed.
    
    @seberg saw this.

commit b97f500364d984485200af14332c3b9653c2813b
Author: Frederic <nouiz at nouiz.org>
Date:   Fri May 17 11:47:20 2013 -0400

    updated the comment.

commit 91cb5e377a95dbd8c63eeb9c04e354fa2621ef3f
Author: Frederic <nouiz at nouiz.org>
Date:   Fri May 17 11:45:16 2013 -0400

    use assert function instead of python keyword

commit 71858ddf54e9da3f9c19a0ecf506553b757d36d6
Author: Frederic <nouiz at nouiz.org>
Date:   Thu May 16 16:51:12 2013 -0400

    Move computation inside an if to speed it up.

commit f77b43a3eb39b34918ef52ae8103dd78d07c7af5
Author: Frederic <nouiz at nouiz.org>
Date:   Thu May 16 16:38:52 2013 -0400

    remove comparison not needed as this is check above and we break in that case.

commit 473e3e31413ea80f0c186e345469930fc6dc49b0
Author: Frederic <nouiz at nouiz.org>
Date:   Thu May 16 15:52:54 2013 -0400

    fix comment style.

commit df388acd385300a03115721beee5892a703508ad
Author: Frederic <nouiz at nouiz.org>
Date:   Thu May 16 14:49:43 2013 -0400

    reorder code to remove not needed computation in some cases.

commit 8fb0dedd61eb508c43b96654b865452a6693161f
Author: Frederic <nouiz at nouiz.org>
Date:   Wed May 15 13:00:04 2013 -0400

    Add test for the array_priority for comparison function.
    
    Conflicts:
    	numpy/core/tests/test_multiarray.py

commit 18c4eafcfb979b3661821db17bf7c4e3d8ee3e94
Author: Frederic <nouiz at nouiz.org>
Date:   Wed May 15 12:59:14 2013 -0400

    Commit from @seberg to make comparison use respect __array_priority__

commit e2105b2d3c373bbf1968da1a04acb5aeab986fee
Author: Frederic <nouiz at nouiz.org>
Date:   Wed May 15 11:59:34 2013 -0400

    Revert "Make comparison function (gt, ge, ...) respect __array_priority__."
    
    This reverts commit aa7a04741146c44ab1735782a89a60b038028bf2.

commit 250c4d66dbdddf41070a2b2cfe02e27acff2dec3
Author: Frederic <nouiz at nouiz.org>
Date:   Fri May 10 16:58:55 2013 -0400

    Make comparison function (gt, ge, ...) respect __array_priority__.

commit 51ebb86b1e4b361bc8a70f5b2b7818615dc08623
Author: Charles Harris <charlesr.harris at gmail.com>
Date:   Fri Jun 21 16:20:04 2013 -0600

    BUG: Make np.load transfer file ownership to the returned NpzFile.
    
    This assures that when the loaded file is closed it also closes the
    file descriptor, avoiding a resource warning in Python3.
    
    Closes #3457.

commit 561c8743472866a3dec6ff34a4b502cc3dc9dc8d
Author: Julian Taylor <jtaylor.debian at googlemail.com>
Date:   Tue Jun 25 19:13:38 2013 +0200

    BUG: add module extensions to load_library search list
    
    on mac, windows and linux (with >= py3.2) shared libraries and loadable
    module have different extensions, so check both for all platforms.

commit ba29f6a27b7c9182348cf903dc6989b8163f190c
Author: Julian Taylor <jtaylor.debian at googlemail.com>
Date:   Fri Mar 29 16:40:31 2013 +0100

    BUG: hardcode some known shared library extensions
    
    the configuration variables are not a reliable mean to get the shared
    library extension. darwin, windows and debug linux are wrong in these
    variables.
    SHLIB_SUFFIX is also wrong as of python 3.3.1
    closes #3057

commit de7a8f21ac151bea683ebee3f7c6c2373297218c
Author: Julian Taylor <jtaylor.debian at googlemail.com>
Date:   Wed Dec 5 19:57:38 2012 +0100

    BUG: add platform specific python include directories to search paths
    
    the generic and platform specific path diverge in python3.3 in ubuntu 13.04

commit c065d6003da4584d1f831d85621f463a260a9ce6
Author: Julian Taylor <jtaylor.debian at googlemail.com>
Date:   Fri Dec 14 14:30:05 2012 +0100

    BUG: fix a spelling error in mtrand.beta docstring

commit 3a587d787b184ff94f970baf6d4b4ee1db9ba96b
Author: Julian Taylor <jtaylor.debian at googlemail.com>
Date:   Tue Mar 26 19:35:06 2013 +0100

    BUG: fix undefined function and add integer divisions in numarray

commit ef4a7461bb42164dd45345302be94f1af739e012
Author: Julian Taylor <jtaylor.debian at googlemail.com>
Date:   Sat May 25 22:37:07 2013 +0200

    BUG: set filename to None for memmaps of unnamed files
    
    Closes gh-3302

commit d2a4cc79105defc8a5e32aa90da0adca89d21290
Author: Julian Taylor <jtaylor.debian at googlemail.com>
Date:   Mon May 27 20:54:13 2013 +0200

    BUG: fix small leak in datetime metadata initialization

commit d7a98ece86e52a5f50484df44ab0181fedf8198e
Author: Julian Taylor <jtaylor.debian at googlemail.com>
Date:   Tue Jun 11 23:50:05 2013 +0200

    BUG: fix non-swapping strided copies in GetStridedCopySwap
    
    Fixed PyArray_GetStridedCopySwap{Pair,}Fn returns paths that do not swap
    the data.
    It never triggered because the alignment requirement for complex data
    was too low (4-8 bytes) and always fullfiled by the memory allocator.
    As this is now fixed it caused failures on sparc with 16 byte long
    doubles.

commit 5e71756fd25199d2db5716b11811ee6f3437ca97
Author: Julian Taylor <jtaylor.debian at googlemail.com>
Date:   Sun May 26 17:08:27 2013 +0200

    ENH: improve performance of byte_bounds for continous arrays
    
    reduces the memmap slicing performance issue in gh-3364 to only about a
    4 times slowdown compared to 10 times.

commit f003b08a8be515c14b5ac4e17f0ab8f1b3126bfc
Author: Charles Harris <charlesr.harris at gmail.com>
Date:   Thu Jun 20 16:49:49 2013 -0600

    BUG: Campanion Matrix was scalar, not matrix for degree 1.
    
    The companion matrices returned by the various polynomial types was
    a scalar in the degree one case instead of a 2-D array. Fix that and
    add a test to check for that result.
    
    Closes #3459.

commit e41e71be3a5be44da70b9a2cb0151a2f9f935153
Author: Julian Taylor <jtaylor.debian at googlemail.com>
Date:   Wed Jun 19 21:53:57 2013 +0200

    BUG: fix uninitialized-use on invalid sort kind
    
    orign is unset at that point but used in the fail goto,
    can probably only be triggered by using the C-api wrong.

commit 20cee69c17e48ec9a266f6f59a48e83188b8b163
Author: Andrew Horton <andrew.j.horton at gmail.com>
Date:   Sun Jun 16 18:48:58 2013 +0100

    Call to byte_swap_vector should use dst, not d1, because this is
    modified by the call to the strided memcpy and will end up pointing
    after the end of the destination buffer in that case.

commit 0a950e6034a0fb77bef61b448b39fb42ff798d73
Author: Eric Firing <efiring at hawaii.edu>
Date:   Sat Jun 15 15:31:52 2013 -1000

    BUG: add __len__ method to ma.mvoid; closes #576

commit 48e2039cf57d737885acfa54af3d032915d2681d
Author: Eric Firing <efiring at hawaii.edu>
Date:   Sat Jun 15 16:47:55 2013 -1000

    BUG: np.ma.compress treated inputs in wrong order; closes #2495

commit fe62f174ac89673c20ddaf27fa545c08ac70d74a
Author: Eric Firing <efiring at hawaii.edu>
Date:   Sat Jun 15 21:33:17 2013 -1000

    BUG: field assignment in masked array did not reset mask; closes #2403
    
    The previous behavior when setting a field after indexing to
    select an element was suitable for the hard mask case, but
    not for the default soft mask.  In addition, the _hardmask
    value was not being set at all in the mvoid instance.  With
    this changeset, the _hardmask is passed in and __setitem__
    takes it into account.
    
    Conflicts:
    
    	numpy/ma/core.py

commit db3e22900d4392031337f99cbdf1a0c8bd941fcc
Author: Warren Weckesser <warren.weckesser at gmail.com>
Date:   Sat Jun 15 11:22:05 2013 -0400

    BUG: ma: ma.average didn't handle complex arrays correctly (issue gh-2684)

commit e82362736eb2380faa866ee2eb80b7e73ccebbf5
Author: Warren Weckesser <warren.weckesser at gmail.com>
Date:   Sat Jun 15 01:42:36 2013 -0400

    MAINT: ma: clean up ma/test_extras.py: don't use 'import *'; PEP8 whitespace
    
    This commit was cherry-picked from 4a7f27601.
    The test for the new 'invert' option of 'in1d' was removed to resolve the conflict.
    
    Conflicts:
    	numpy/ma/tests/test_extras.py

commit ef5fa413e984264d5c78b33eae408b00aaa27ab2
Author: bebert218 <guillaume.viry at gmail.com>
Date:   Wed May 22 13:58:38 2013 +0900

    BUG: The npv function in financial.py was incorrectly implemented.
    
    Correct the implementation of the npv function, its documentation, and
    the mirr function that depends on it. The test_financial.py is also
    corrected to take into account those modifications
    
    The npv function behavior was contrary to what the documentation stated
    as it summed indexes 1 to M instead of 0 to M-1. The mirr function used
    a corrective factor to get the correct result in spite of that error so
    that factor is removed.
    
    Closes #649

commit 83e868afd3162c5f5995c3ab1f71023ad6f82de8
Author: mwtoews <mwtoews at gmail.com>
Date:   Sun May 26 23:28:50 2013 +1200

    BUG: Handle a version string from a custom-built mingw64.
    
    The custom build compiler "GNU Fortran (rubenvb-4.8.0) 4.8.0" was
    incorrectly parsed as version '-4.8.0' and the flag "-mno-cygwin" was
    added to the compilation.
    
    See http://cens.ioc.ee/pipermail/f2py-users/2010-October/002092.html.

commit d45d4232337acbe5c31589aa7338ab40d51f56f0
Author: Julian Taylor <jtaylor.debian at googlemail.com>
Date:   Tue May 28 23:00:11 2013 +0200

    BUG: include mtrand cython files in sdist
    
    the files are required to do a full source rebuild.

commit 7e588870a5412feb2ace16156c9a93bdbea051cb
Author: Edward Catmur <ecatmur at ahl.com>
Date:   Tue May 7 17:47:38 2013 +0100

    TST: add a test for issue #3312

commit 2f1113335f58ab059c68834e3929a673d1e7d57d
Author: Edward Catmur <ecatmur at ahl.com>
Date:   Tue May 7 16:49:39 2013 +0100

    BUG: fix potentially infinite recursion in VOID_nonzero. Fixes #3312.

commit 29cda37dd553993a80864146b376a7a1fffac995
Author: Charles Harris <charlesr.harris at gmail.com>
Date:   Fri Apr 26 09:59:14 2013 -0600

    MAINT: Fix tests for 2to3 and Python < 2.6.
    
    Use asbytes instead of 'b' and the 'u' string prefix instead of sixu.

commit 216ddba3c011e15009071ca49230cfcaeaffbb9d
Author: Jay Bourque <jay.bourque at continuum.io>
Date:   Thu Apr 25 17:00:06 2013 -0500

    Add tests for strings shorter than 64 characters

commit 69e9e6306a1635b41147d3743dac976dc6d4f5cf
Author: Jay Bourque <jay.bourque at continuum.io>
Date:   Thu Apr 25 13:38:16 2013 -0500

    Add support for unicode strings

commit ab3aa117a0e11074c99f3b8fb6297fd71860290c
Author: Jay Bourque <jay.bourque at continuum.io>
Date:   Wed Apr 24 21:42:15 2013 -0500

    Fix for astype('S') string truncate issue
    
    Calling astype('S') for an array of string objects results in a string
    array where dtype='S64', even if the original string objects are longer
    than 64 characters. Add call to GetParamsFromObject() to determine maximum
    string object length, and use that as string dtype size.

commit bd869d0e16c96f9cfdbf3db297bbf84737f4b9a6
Author: Han Genuit <hangenuit at gmail.com>
Date:   Sat Sep 15 17:47:22 2012 +0200

    TST: Add test for concatenate with None-axis
    
    Conflicts:
    	numpy/core/tests/test_shape_base.py

commit f2450236670389bbbf6288bd6b9515d7b6baf3ca
Author: Han Genuit <hangenuit at gmail.com>
Date:   Sat Sep 15 16:48:53 2012 +0200

    BUG: Concatenate with axis None should work regardless of matching array shapes

commit 8e7c6a77df290742071b6499ceafa9f6a9ded571
Author: Jos de Kloe <kloedej at bhw288.knmi.nl>
Date:   Fri Apr 12 12:18:26 2013 +0200

    replace exec by eval to ensure the c variable is defined for all relevant python versions

commit 971bab3d51726b95f5afe0c22cbbd7983023f626
Author: Alex Ford <fordas at uw.edu>
Date:   Sat Apr 6 11:18:45 2013 -0700

    BUG: Backport #3188, Fix numpy.void pickling.
    
    During call to PyArray_scalar a PyVoidScalarObject is created, and it's
    obval field set to a newly allocated block of memory of the correct item
    size. With a null base member, the subsequent call to copyswap can not
    determine an item size and returns without copying.  Adding direct copy
    of input data if no base is provided, as no swap is required.
    
    Adding regression test for constructor and original pickle repro case.

commit dccfdbd1e8ee44925e702d850c3663daaa54d927
Author: Sebastian Berg <sebastian at sipsolutions.net>
Date:   Mon Mar 18 15:53:10 2013 +0100

    TST: test no reuse of non-existing buffers in nditer
    
    This slightly changes the test to cover more cases including
    the one where buffers are sometimes used and sometimes not.

commit 4c2469ab6bb0bbb29e0b21b6f5f1abf53d8968d3
Author: Sebastian Berg <sebastian at sipsolutions.net>
Date:   Mon Mar 18 15:44:12 2013 +0100

    BUG: no buffer reuse in nditer, if there was no buffer previously
    
    If the buffer is used or not can possible change during iteration.
    In this case, the buffer cannot be reused if it never existed...

commit 5c154dfdde7e143a24fa6849cf7a47f4cc2ecf56
Author: Sebastian Berg <sebastian at sipsolutions.net>
Date:   Sun Mar 17 16:50:01 2013 +0100

    TST: Add (slow) nditer reduce buffer reuse test

commit 9899d23f422abb83f84e637400fc2ce23ed0a322
Author: Sebastian Berg <sebastian at sipsolutions.net>
Date:   Sun Mar 17 16:04:36 2013 +0100

    BUG: Do not reuse nditer buffers when not filled enough.
    
    This checks if the previous time around, the buffers were filled
    with as much data as they would be filled this time around. Since
    This is difficult for the initial loop before reusing is activated
    because in that case the buffer may be larger then just the
    first outer reduce dimension. In that case do not allow reuse
    unless the index along that dimension was 0.
    When the inner reduce index is not 0, then also the reusing of
    the buffer is dangerous.

From charlesr.harris at gmail.com  Sun Oct 20 13:35:28 2013
From: charlesr.harris at gmail.com (Charles R Harris)
Date: Sun, 20 Oct 2013 11:35:28 -0600
Subject: [Numpy-discussion] Numpy 1.8.0 release
Message-ID: <CAB6mnx+Y58NeHP9YmYfKNCG1Yjmb1f75kmBvrr2wEJ+y0FHs7g@mail.gmail.com>

Hi All,

I'm planning on releasing Numpy 1.8.0 next weekend. There have been a few
minor fixes since 1.8.0rc2, but nothing that I think warrants another rc
release. Please make sure to test the 1.8.0rc2 or maintenance/1.8.x branch
with your code, for after next weekend it will be too late.

Chuck
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20131020/193af3a4/attachment.html>

From jdlmail at gmail.com  Sun Oct 20 13:55:43 2013
From: jdlmail at gmail.com (Yaron de Leeuw)
Date: Sun, 20 Oct 2013 19:55:43 +0200
Subject: [Numpy-discussion] pull request: organization of the
	npyio.recfromcsv function arguments
Message-ID: <CADY+y7EMoqQ3LvqKwEmyj-k-eZeDq7Sbc1ujRV+n+n7cw_Q4Kg@mail.gmail.com>

Hi all.

I've submitted a pull request to organise the recfromcsv arguments.
It closes a known bug, and is a bit more sane in my opinion.

I would appreciate your feedback.
https://github.com/numpy/numpy/pull/3830

Thank you,
Yaron
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20131020/a43c284a/attachment.html>

From juanlu001 at gmail.com  Sun Oct 20 17:01:23 2013
From: juanlu001 at gmail.com (Juan Luis Cano)
Date: Sun, 20 Oct 2013 23:01:23 +0200
Subject: [Numpy-discussion] ANN: SciPy 0.13.0 release
In-Reply-To: <CABL7CQi5B0qdbWMQBrVwRsZ33BGsV2ATaUCEAbA5rjNjm=iyoQ@mail.gmail.com>
References: <CABL7CQi5B0qdbWMQBrVwRsZ33BGsV2ATaUCEAbA5rjNjm=iyoQ@mail.gmail.com>
Message-ID: <526444A3.9090602@gmail.com>

On 10/19/2013 11:40 PM, Ralf Gommers wrote:
> On behalf of the SciPy development team I'm pleased to announce the 
> availability of SciPy 0.13.0. This release contains some interesting 
> new features (see highlights below) and half a year's worth of 
> maintenance work. 65 people contributed to this release.

Congrats!

Only a tiny little thing: the docs still point to 0.12.0:

http://docs.scipy.org/doc/scipy/reference/

>
> Some of the highlights are:
>
>   - support for fancy indexing and boolean comparisons with sparse 
> matrices
>   - interpolative decompositions and matrix functions in the linalg module
>   - two new trust-region solvers for unconstrained minimization
>
> This release requires Python 2.6, 2.7 or 3.1-3.3 and NumPy 1.5.1 or 
> greater. Support for Python 2.4 and 2.5 has been dropped as of this 
> release.
>
> Sources and binaries can be found at 
> http://sourceforge.net/projects/scipy/files/scipy/0.13.0/, release 
> notes are copied below.
>
> Enjoy,
> Ralf
>
>
>
>
> ==========================
> SciPy 0.13.0 Release Notes
> ==========================
>
> .. contents::
>
> SciPy 0.13.0 is the culmination of 7 months of hard work. It contains
> many new features, numerous bug-fixes, improved test coverage and
> better documentation.  There have been a number of deprecations and
> API changes in this release, which are documented below.  All users
> are encouraged to upgrade to this release, as there are a large number
> of bug-fixes and optimizations.  Moreover, our development attention
> will now shift to bug-fix releases on the 0.13.x branch, and on adding
> new features on the master branch.
>
> This release requires Python 2.6, 2.7 or 3.1-3.3 and NumPy 1.5.1 or 
> greater.
> Highlights of this release are:
>
>   - support for fancy indexing and boolean comparisons with sparse 
> matrices
>   - interpolative decompositions and matrix functions in the linalg module
>   - two new trust-region solvers for unconstrained minimization
>
>
> New features
> ============
>
> ``scipy.integrate`` improvements
> --------------------------------
>
> N-dimensional numerical integration
> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
>
> A new function `scipy.integrate.nquad`, which provides N-dimensional
> integration functionality with a more flexible interface than 
> ``dblquad`` and
> ``tplquad``, has been added.
>
> ``dopri*`` improvements
> ^^^^^^^^^^^^^^^^^^^^^^^
>
> The intermediate results from the ``dopri`` family of ODE solvers can 
> now be
> accessed by a *solout* callback function.
>
>
> ``scipy.linalg`` improvements
> -----------------------------
>
> Interpolative decompositions
> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
>
> Scipy now includes a new module `scipy.linalg.interpolative`
> containing routines for computing interpolative matrix decompositions
> (ID). This feature is based on the ID software package by
> P.G. Martinsson, V. Rokhlin, Y. Shkolnisky, and M. Tygert, previously
> adapted for Python in the PymatrixId package by K.L. Ho.
>
> Polar decomposition
> ^^^^^^^^^^^^^^^^^^^
>
> A new function `scipy.linalg.polar`, to compute the polar decomposition
> of a matrix, was added.
>
> BLAS level 3 functions
> ^^^^^^^^^^^^^^^^^^^^^^
>
> The BLAS functions ``symm``, ``syrk``, ``syr2k``, ``hemm``, ``herk`` and
> ``her2k`` are now wrapped in `scipy.linalg`.
>
> Matrix functions
> ^^^^^^^^^^^^^^^^
>
> Several matrix function algorithms have been implemented or updated 
> following
> detailed descriptions in recent papers of Nick Higham and his co-authors.
> These include the matrix square root (``sqrtm``), the matrix logarithm
> (``logm``), the matrix exponential (``expm``) and its Frechet derivative
> (``expm_frechet``), and fractional matrix powers 
> (``fractional_matrix_power``).
>
>
> ``scipy.optimize`` improvements
> -------------------------------
>
> Trust-region unconstrained minimization algorithms
> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
>
> The ``minimize`` function gained two trust-region solvers for 
> unconstrained
> minimization: ``dogleg`` and ``trust-ncg``.
>
>
> ``scipy.sparse`` improvements
> -----------------------------
>
> Boolean comparisons and sparse matrices
> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
>
> All sparse matrix types now support boolean data, and boolean 
> operations.  Two
> sparse matrices `A` and `B` can be compared in all the expected ways 
> `A < B`,
> `A >= B`, `A != B`, producing similar results as dense Numpy arrays.
> Comparisons with dense matrices and scalars are also supported.
>
> CSR and CSC fancy indexing
> ^^^^^^^^^^^^^^^^^^^^^^^^^^
>
> Compressed sparse row and column sparse matrix types now support fancy 
> indexing
> with boolean matrices, slices, and lists. So where A is a (CSC or CSR) 
> sparse
> matrix, you can do things like::
>
>     >>> A[A > 0.5] = 1  # since Boolean sparse matrices work
>     >>> A[:2, :3] = 2
>     >>> A[[1,2], 2] = 3
>
>
> ``scipy.sparse.linalg`` improvements
> ------------------------------------
>
> The new function ``onenormest`` provides a lower bound of the 1-norm of a
> linear operator and has been implemented according to Higham and Tisseur
> (2000).  This function is not only useful for sparse matrices, but can 
> also be
> used to estimate the norm of products or powers of dense matrices without
> explictly building the intermediate matrix.
>
> The multiplicative action of the matrix exponential of a linear operator
> (``expm_multiply``) has been implemented following the description in 
> Al-Mohy
> and Higham (2011).
>
> Abstract linear operators (`scipy.sparse.linalg.LinearOperator`) can 
> now be
> multiplied, added to each other, and exponentiated, producing new linear
> operators. This enables easier construction of composite linear 
> operations.
>
>
> ``scipy.spatial`` improvements
> ------------------------------
>
> The vertices of a `ConvexHull` can now be accessed via the `vertices` 
> attribute,
> which gives proper orientation in 2-D.
>
>
> ``scipy.signal`` improvements
> -----------------------------
>
> The cosine window function `scipy.signal.cosine` was added.
>
>
> ``scipy.special`` improvements
> ------------------------------
>
> New functions `scipy.special.xlogy` and `scipy.special.xlog1py` were 
> added.
> These functions can simplify and speed up code that has to calculate
> ``x * log(y)`` and give 0 when ``x == 0``.
>
>
> ``scipy.io <http://scipy.io>`` improvements
> -------------------------
>
> Unformatted Fortran file reader
> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
>
> The new class `scipy.io.FortranFile` facilitates reading unformatted
> sequential files written by Fortran code.
>
> ``scipy.io.wavfile`` enhancements
> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
>
> `scipy.io.wavfile.write` now accepts a file buffer. Previously it only
> accepted a filename.
>
> `scipy.io.wavfile.read` and `scipy.io.wavfile.write` can now handle 
> floating
> point WAV files.
>
>
> ``scipy.interpolate`` improvements
> ----------------------------------
>
> B-spline derivatives and antiderivatives
> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
>
> `scipy.interpolate.splder` and `scipy.interpolate.splantider` functions
> for computing B-splines that represent derivatives and antiderivatives
> of B-splines were added.  These functions are also available in the
> class-based FITPACK interface as ``UnivariateSpline.derivative`` and
> ``UnivariateSpline.antiderivative``.
>
>
> ``scipy.stats`` improvements
> ----------------------------
>
> Distributions now allow using keyword parameters in addition to
> positional parameters in all methods.
>
> The function `scipy.stats.power_divergence` has been added for the
> Cressie-Read power divergence statistic and goodness of fit test.
> Included in this family of statistics is the "G-test"
> (http://en.wikipedia.org/wiki/G-test).
>
> `scipy.stats.mood` now accepts multidimensional input.
>
> An option was added to `scipy.stats.wilcoxon` for continuity correction.
>
> `scipy.stats.chisquare` now has an `axis` argument.
>
> `scipy.stats.mstats.chisquare` now has `axis` and `ddof` arguments.
>
>
> Deprecated features
> ===================
>
> ``expm2`` and ``expm3``
> -----------------------
>
> The matrix exponential functions `scipy.linalg.expm2` and 
> `scipy.linalg.expm3`
> are deprecated. All users should use the numerically more robust
> `scipy.linalg.expm` function instead.
>
> ``scipy.stats`` functions
> -------------------------
>
> `scipy.stats.oneway` is deprecated; `scipy.stats.f_oneway` should be used
> instead.
>
> `scipy.stats.glm` is deprecated.  `scipy.stats.ttest_ind` is an equivalent
> function; more full-featured general (and generalized) linear model
> implementations can be found in statsmodels.
>
> `scipy.stats.cmedian` is deprecated; ``numpy.median`` should be used 
> instead.
>
>
> Backwards incompatible changes
> ==============================
>
> LIL matrix assignment
> ---------------------
> Assigning values to LIL matrices with two index arrays now works 
> similarly as
> assigning into ndarrays::
>
>     >>> x = lil_matrix((3, 3))
>     >>> x[[0,1,2],[0,1,2]]=[0,1,2]
>     >>> x.todense()
>     matrix([[ 0.,  0.,  0.],
>             [ 0.,  1.,  0.],
>             [ 0.,  0.,  2.]])
>
> rather than giving the result::
>
>     >>> x.todense()
>     matrix([[ 0.,  1.,  2.],
>             [ 0.,  1.,  2.],
>             [ 0.,  1.,  2.]])
>
> Users relying on the previous behavior will need to revisit their code.
> The previous behavior is obtained by ``x[numpy.ix_([0,1,2],[0,1,2])] = 
> ...`.
>
>
> Deprecated ``radon`` function removed
> -------------------------------------
>
> The ``misc.radon`` function, which was deprecated in scipy 0.11.0, has 
> been
> removed.  Users can find a more full-featured ``radon`` function in
> scikit-image.
>
>
> Removed deprecated keywords ``xa`` and ``xb`` from ``stats.distributions``
> --------------------------------------------------------------------------
>
> The keywords ``xa`` and ``xb``, which were deprecated since 0.11.0, have
> been removed from the distributions in ``scipy.stats``.
>
> Changes to MATLAB file readers / writers
> ----------------------------------------
>
> The major change is that 1D arrays in numpy now become row vectors 
> (shape 1, N)
> when saved to a MATLAB 5 format file.  Previously 1D arrays saved as 
> column
> vectors (N, 1).  This is to harmonize the behavior of writing MATLAB 4 
> and 5
> formats, and adapt to the defaults of numpy and MATLAB - for example
> ``np.atleast_2d`` returns 1D arrays as row vectors.
>
> Trying to save arrays of greater than 2 dimensions in MATLAB 4 format 
> now raises
> an error instead of silently reshaping the array as 2D.
>
> ``scipy.io.loadmat('afile')`` used to look for `afile` on the Python 
> system path
> (``sys.path``); now ``loadmat`` only looks in the current directory for a
> relative path filename.
>
>
> Other changes
> =============
>
> Security fix: ``scipy.weave`` previously used temporary directories in an
> insecure manner under certain circumstances.
>
> Cython is now required to build *unreleased* versions of scipy.
> The C files generated from Cython sources are not included in the git repo
> anymore.  They are however still shipped in source releases.
>
> The code base received a fairly large PEP8 cleanup.  A ``tox pep8``
> command has been added; new code should pass this test command.
>
> Scipy cannot be compiled with gfortran 4.1 anymore (at least on RH5), 
> likely
> due to that compiler version not supporting entry constructs well.
>
>
> Authors
> =======
>
> This release contains work by the following people (contributed at least
> one patch to this release, names in alphabetical order):
>
> * Jorge Ca?ardo Alastuey +
> * Tom Aldcroft +
> * Max Bolingbroke +
> * Joseph Jon Booker +
> * Fran?ois Boulogne
> * Matthew Brett
> * Christian Brodbeck +
> * Per Brodtkorb +
> * Christian Brueffer +
> * Lars Buitinck
> * Evgeni Burovski +
> * Tim Cera
> * Lawrence Chan +
> * David Cournapeau
> * Draz?en Luc?anin +
> * Alexander J. Dunlap +
> * endolith
> * Andr? Gaul +
> * Christoph Gohlke
> * Ralf Gommers
> * Alex Griffing +
> * Blake Griffith +
> * Charles Harris
> * Bob Helmbold +
> * Andreas Hilboll
> * Kat Huang +
> * Oleksandr (Sasha) Huziy +
> * Gert-Ludwig Ingold +
> * Thouis (Ray) Jones
> * Juan Luis Cano Rodr?guez +
> * Robert Kern
> * Andreas Kloeckner +
> * Sytse Knypstra +
> * Gustav Larsson +
> * Denis Laxalde
> * Christopher Lee
> * Tim Leslie
> * Wendy Liu +
> * Clemens Novak +
> * Takuya Oshima +
> * Josef Perktold
> * Illia Polosukhin +
> * Przemek Porebski +
> * Steve Richardson +
> * Branden Rolston +
> * Skipper Seabold
> * Fazlul Shahriar
> * Leo Singer +
> * Rohit Sivaprasad +
> * Daniel B. Smith +
> * Julian Taylor
> * Louis Thibault +
> * Tomas Tomecek +
> * John Travers
> * Richard Tsai +
> * Jacob Vanderplas
> * Patrick Varilly
> * Pauli Virtanen
> * Stefan van der Walt
> * Warren Weckesser
> * Pedro Werneck +
> * Nils Werner +
> * Michael Wimmer +
> * Nathan Woods +
> * Tony S. Yu +
>
> A total of 65 people contributed to this release.
> People with a "+" by their names contributed a patch for the first time.
>
>
>
> _______________________________________________
> NumPy-Discussion mailing list
> NumPy-Discussion at scipy.org
> http://mail.scipy.org/mailman/listinfo/numpy-discussion

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20131020/cc9c37f1/attachment.html>

From pav at iki.fi  Sun Oct 20 18:04:52 2013
From: pav at iki.fi (Pauli Virtanen)
Date: Mon, 21 Oct 2013 01:04:52 +0300
Subject: [Numpy-discussion] ANN: SciPy 0.13.0 release
In-Reply-To: <526444A3.9090602@gmail.com>
References: <CABL7CQi5B0qdbWMQBrVwRsZ33BGsV2ATaUCEAbA5rjNjm=iyoQ@mail.gmail.com>
	<526444A3.9090602@gmail.com>
Message-ID: <l41k1q$ru3$1@ger.gmane.org>

21.10.2013 00:01, Juan Luis Cano kirjoitti:
> On 10/19/2013 11:40 PM, Ralf Gommers wrote:
>> On behalf of the SciPy development team I'm pleased to announce the
>> availability of SciPy 0.13.0. This release contains some interesting
>> new features (see highlights below) and half a year's worth of
>> maintenance work. 65 people contributed to this release.
> 
> Congrats!
> 
> Only a tiny little thing: the docs still point to 0.12.0:
> 
> http://docs.scipy.org/doc/scipy/reference/

Should be fixed now.

-- 
Pauli Virtanen




From stefan at sun.ac.za  Mon Oct 21 02:48:41 2013
From: stefan at sun.ac.za (=?ISO-8859-1?Q?St=E9fan_van_der_Walt?=)
Date: Mon, 21 Oct 2013 08:48:41 +0200
Subject: [Numpy-discussion] ANN: scikit-image 0.9 release
Message-ID: <CABDkGQnFM95eBkS+OhiMNjy7bSbcd5xEbuhqQvym1WS29J6r9Q@mail.gmail.com>

We're happy to announce the release of scikit-image v0.9.0!

scikit-image is an image processing toolbox for SciPy that includes algorithms
for segmentation, geometric transformations, color space manipulation,
analysis, filtering, morphology, feature detection, and more.

For more information, examples, and documentation, please visit our website:

    http://scikit-image.org


New Features
------------

`scikit-image` now runs without translation under both Python 2 and 3.

In addition to several bug fixes, speed improvements and examples, the 204 pull
requests merged for this release include the following new features (PR number
in brackets):

Segmentation:

- 3D support in SLIC segmentation (#546)
- SLIC voxel spacing (#719)
- Generalized anisotropic spacing support for random_walker (#775)
- Yen threshold method (#686)

Transforms and filters:

- SART algorithm for tomography reconstruction (#584)
- Gabor filters (#371)
- Hough transform for ellipses (#597)
- Fast resampling of nD arrays (#511)
- Rotation axis center for Radon transforms with inverses. (#654)
- Reconstruction circle in inverse Radon transform (#567)
- Pixelwise image adjustment curves and methods (#505)

Feature detection:

- [experimental API] BRIEF feature descriptor (#591)
- [experimental API] Censure (STAR) Feature Detector (#668)
- Octagon structural element (#669)
- Add non rotation invariant uniform LBPs (#704)

Color and noise:

- Add deltaE color comparison and lab2lch conversion (#665)
- Isotropic denoising (#653)
- Generator to add various types of random noise to images (#625)
- Color deconvolution for immunohistochemical images (#441)
- Color label visualization (#485)

Drawing and visualization:

- Wu's anti-aliased circle, line, bezier curve (#709)
- Linked image viewers and docked plugins (#575)
- Rotated ellipse + bezier curve drawing (#510)
- PySide & PyQt4 compatibility in skimage-viewer (#551)

Other:

- Python 3 support without 2to3. (#620)
- 3D Marching Cubes (#469)
- Line, Circle, Ellipse total least squares fitting and RANSAC algorithm (#440)
- N-dimensional array padding (#577)
- Add a wrapper around `scipy.ndimage.gaussian_filter` with useful
default behaviors. (#712)
- Predefined structuring elements for 3D morphology (#484)


API changes
-----------

The following backward-incompatible API changes were made between 0.8 and 0.9:

- No longer wrap ``imread`` output in an ``Image`` class
- Change default value of `sigma` parameter in ``skimage.segmentation.slic``
  to 0
- ``hough_circle`` now returns a stack of arrays that are the same size as the
  input image. Set the ``full_output`` flag to True for the old behavior.
- The following functions were deprecated over two releases:
  `skimage.filter.denoise_tv_chambolle`,
  `skimage.morphology.is_local_maximum`, `skimage.transform.hough`,
  `skimage.transform.probabilistic_hough`,`skimage.transform.hough_peaks`.
  Their functionality still exists, but under different names.


Contributors to this release
----------------------------

This release was made possible by the collaborative efforts of many
contributors, both new and old.  They are listed in alphabetical order by
surname:

- Ankit Agrawal
- K.-Michael Aye
- Chris Beaumont
- Fran?ois Boulogne
- Luis Pedro Coelho
- Marianne Corvellec
- Olivier Debeir
- Ferdinand Deger
- Kemal Eren
- Jostein B? Fl?ystad
- Christoph Gohlke
- Emmanuelle Gouillart
- Christian Horea
- Thouis (Ray) Jones
- Almar Klein
- Xavier Moles Lopez
- Alexis Mignon
- Juan Nunez-Iglesias
- Zachary Pincus
- Nicolas Pinto
- Davin Potts
- Malcolm Reynolds
- Umesh Sharma
- Johannes Sch?nberger
- Chintak Sheth
- Kirill Shklovsky
- Steven Silvester
- Matt Terry
- Riaan van den Dool
- St?fan van der Walt
- Josh Warner
- Adam Wisniewski
- Yang Zetian
- Tony S Yu


From mads.ipsen at gmail.com  Mon Oct 21 03:20:12 2013
From: mads.ipsen at gmail.com (Mads Ipsen)
Date: Mon, 21 Oct 2013 09:20:12 +0200
Subject: [Numpy-discussion] Porting to the new C-API (1.7)
Message-ID: <5264D5AC.7020408@gmail.com>

Hi,

I've been using the numpy-1.6 C-API as part of a large C++ based OpenGL
application.

The C++ classes are exposed in Python by using SWIG, and utilize numpy
arrays both as inputs to methods and method return values to the Python
caller.

To enable numpy in the SWIG generated Python module, the SWIG generated
C++ file define

  #define PY_ARRAY_UNIQUE_SYMBOL PyArray_API

whereas all other C++ files that need access to the numpy C-API contain

  #define NO_IMPORT_ARRAY
  #include "numpy/arrayobject.h"

I have now updated to numpy-1.7, and receive warnings of the form

#warning "Using deprecated NumPy API, disable it by
#defining NPY_NO_DEPRECATED_API NPY_1_7_API_VERSION"

This is basically fine with me, and I don't mind doing an update of my
code to the new C-API.

I have a few questions though:

1) Since I am apparently using the old API, where can I find a list of
   the deprecated things I use? That would make the upgrade easier.

2) Do I still have to use the PY_ARRAY_UNIQUE_SYMBOL approach when
   using the new C-API.

3) According to some websites you can do something like

  #define PY_ARRAY_UNIQUE_SYMBOL PyArrayXXX

This puzzles me a bit. Is there a doc somewhere where this whole thing
is explained in detail. I must admit, its somewhat hard to grasp
what's going on.

Best regards,

Mads

-- 
+-----------------------------------------------------+
| Mads Ipsen                                          |
+----------------------+------------------------------+
| G?seb?ksvej 7, 4. tv |                              |
| DK-2500 Valby        | phone:          +45-29716388 |
| Denmark              | email:  mads.ipsen at gmail.com |
+----------------------+------------------------------+



From Marc.Poinot at onera.fr  Mon Oct 21 07:22:33 2013
From: Marc.Poinot at onera.fr (Marc.Poinot at onera.fr)
Date: Mon, 21 Oct 2013 13:22:33 +0200
Subject: [Numpy-discussion] Contiguous memory zone with C API
Message-ID: <52650E79.9020601@onera.fr>


Hi all,

I try to change my old C API numpy code to the 'new' API. I used to hack some internal stuff in Numpy (yes it's bad...) and I wonder now how to change it.

Let's take an example: I have ocount numpy arrays with data allocated elsewhere than numpy, the data pointer of the PyArrayObject is set with the malloc'd zone.
Now I select one of these array to be the memory base for all of them, then I realloc each data pointer to make sure the ocount of them have a contiguous zone.
The code is now 'rejected' by the new API, how can I do that without hacking into the PyArray_Object?

   first=(PyArrayObject*)context->ctg_obj[0];
   psize=PyArray_NBYTES(first);
   for (j=1;j<ocount;j++)
   {
     current=(PyArrayObject*)context->ctg_obj[j];
     tsize=PyArray_NBYTES(current);
     psize+=tsize;
((PyArrayObject*)first)->data=realloc(PyArray_DATA(first),psize); /* *** how to do that with the API ? */
memcpy(PyArray_DATA(first)+psize-tsize,PyArray_DATA(current),tsize);
     free(PyArray_DATA(current));
((PyArrayObject*)current)->data=PyArray_DATA(first)+psize-tsize;
   }


I use that trick to make sure that separate numpy each representing a coordinate of a vector can be gather in a single array.
I've had a look at PyArray_resize but it requires a own_data flag which I do not have...
Any hint, remark?

-- 
-- ----------------------------------------------------------------------------
-- Marc POINOT [ONERA/DSNA] Tel:+33.1.46.73.42.84  Fax:+33.1.46.73.41.66
-- Avertissement/disclaimer http://www.onera.fr/en/emails-terms
-- ----------------------------------------------------------------------------
--



From nouiz at nouiz.org  Mon Oct 21 11:33:14 2013
From: nouiz at nouiz.org (=?ISO-8859-1?Q?Fr=E9d=E9ric_Bastien?=)
Date: Mon, 21 Oct 2013 11:33:14 -0400
Subject: [Numpy-discussion] Numpy 1.8.0 release
In-Reply-To: <CAB6mnx+Y58NeHP9YmYfKNCG1Yjmb1f75kmBvrr2wEJ+y0FHs7g@mail.gmail.com>
References: <CAB6mnx+Y58NeHP9YmYfKNCG1Yjmb1f75kmBvrr2wEJ+y0FHs7g@mail.gmail.com>
Message-ID: <CADKKbtgHu5e-+RG5jP7n38XOM=EcmO5UCv9y=yGwJ4_sriTizQ@mail.gmail.com>

Hi,

This is to tell that all Theano tests pass with the branch 1.8.x with
the commit 397fdec2a2c

thanks

Fr?d?ric

On Sun, Oct 20, 2013 at 1:35 PM, Charles R Harris
<charlesr.harris at gmail.com> wrote:
> Hi All,
>
> I'm planning on releasing Numpy 1.8.0 next weekend. There have been a few
> minor fixes since 1.8.0rc2, but nothing that I think warrants another rc
> release. Please make sure to test the 1.8.0rc2 or maintenance/1.8.x branch
> with your code, for after next weekend it will be too late.
>
> Chuck
>
> _______________________________________________
> NumPy-Discussion mailing list
> NumPy-Discussion at scipy.org
> http://mail.scipy.org/mailman/listinfo/numpy-discussion
>


From charlesr.harris at gmail.com  Mon Oct 21 13:04:18 2013
From: charlesr.harris at gmail.com (Charles R Harris)
Date: Mon, 21 Oct 2013 11:04:18 -0600
Subject: [Numpy-discussion] Numpy 1.8.0 release
In-Reply-To: <CADKKbtgHu5e-+RG5jP7n38XOM=EcmO5UCv9y=yGwJ4_sriTizQ@mail.gmail.com>
References: <CAB6mnx+Y58NeHP9YmYfKNCG1Yjmb1f75kmBvrr2wEJ+y0FHs7g@mail.gmail.com>
	<CADKKbtgHu5e-+RG5jP7n38XOM=EcmO5UCv9y=yGwJ4_sriTizQ@mail.gmail.com>
Message-ID: <CAB6mnxKrH6ax92yoiVVCUz47Q87GGOFdNNU2g1TQFnYAa6vNQg@mail.gmail.com>

On Mon, Oct 21, 2013 at 9:33 AM, Fr?d?ric Bastien <nouiz at nouiz.org> wrote:

> Hi,
>
> This is to tell that all Theano tests pass with the branch 1.8.x with
> the commit 397fdec2a2c
>
> thanks
>
> Fr?d?ric
>

Thanks for testing

<snip>

Chuck
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20131021/7bbdb7c0/attachment.html>

From mb at linux.vnet.ibm.com  Mon Oct 21 13:53:02 2013
From: mb at linux.vnet.ibm.com (Mounir E. Bsaibes)
Date: Mon, 21 Oct 2013 12:53:02 -0500
Subject: [Numpy-discussion] Is there a contributors agreement for numypy?
Message-ID: <1382377982.24732.12.camel@mblaptop>

I am  checking whether there is a Contributor's agreement that new
contributors have to sign. Or, whether there was one for the
documentation marathon. 

Does anyone knows the answer to this question or can point me to where I
can possibly find the answer? 

Thanks,
Mounir



From charlesr.harris at gmail.com  Mon Oct 21 14:00:42 2013
From: charlesr.harris at gmail.com (Charles R Harris)
Date: Mon, 21 Oct 2013 12:00:42 -0600
Subject: [Numpy-discussion] Is there a contributors agreement for numypy?
In-Reply-To: <1382377982.24732.12.camel@mblaptop>
References: <1382377982.24732.12.camel@mblaptop>
Message-ID: <CAB6mnxLJnchv0uUyyCio==b-k+n+cYGx+TVEt+6JQ6WUT-g8gQ@mail.gmail.com>

On Mon, Oct 21, 2013 at 11:53 AM, Mounir E. Bsaibes
<mb at linux.vnet.ibm.com>wrote:

> I am  checking whether there is a Contributor's agreement that new
> contributors have to sign. Or, whether there was one for the
> documentation marathon.
>
> Does anyone knows the answer to this question or can point me to where I
> can possibly find the answer?
>
> Thanks,
> Mounir
>
>
There is no agreement needed, but all numpy is released under the
simplified BSD license and any contributions need to be compatible with
that. I don't know that there is any special license for the documentation.
Anyone?

Chuck
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20131021/7e5de187/attachment.html>

From pav at iki.fi  Mon Oct 21 14:23:06 2013
From: pav at iki.fi (Pauli Virtanen)
Date: Mon, 21 Oct 2013 21:23:06 +0300
Subject: [Numpy-discussion] Is there a contributors agreement for numypy?
In-Reply-To: <CAB6mnxLJnchv0uUyyCio==b-k+n+cYGx+TVEt+6JQ6WUT-g8gQ@mail.gmail.com>
References: <1382377982.24732.12.camel@mblaptop>
	<CAB6mnxLJnchv0uUyyCio==b-k+n+cYGx+TVEt+6JQ6WUT-g8gQ@mail.gmail.com>
Message-ID: <l43re0$95u$1@ger.gmane.org>

21.10.2013 21:00, Charles R Harris kirjoitti:
[clip]
> There is no agreement needed, but all numpy is released under the 
> simplified BSD license and any contributions need to be compatible
> with that. I don't know that there is any special license for the
> documentation. Anyone?

I don't think the documentation has a separate license; also BSD.

-- 
Pauli Virtanen



From mb at linux.vnet.ibm.com  Mon Oct 21 15:36:11 2013
From: mb at linux.vnet.ibm.com (Mounir E. Bsaibes)
Date: Mon, 21 Oct 2013 14:36:11 -0500
Subject: [Numpy-discussion] Is there a contributors agreement for numypy?
In-Reply-To: <l43re0$95u$1@ger.gmane.org>
References: <1382377982.24732.12.camel@mblaptop>
	<CAB6mnxLJnchv0uUyyCio==b-k+n+cYGx+TVEt+6JQ6WUT-g8gQ@mail.gmail.com>
	<l43re0$95u$1@ger.gmane.org>
Message-ID: <1382384171.24732.58.camel@mblaptop>

On Mon, 2013-10-21 at 21:23 +0300, Pauli Virtanen wrote:
> 21.10.2013 21:00, Charles R Harris kirjoitti:
> [clip]
> > There is no agreement needed, but all numpy is released under the 
> > simplified BSD license and any contributions need to be compatible
> > with that. I don't know that there is any special license for the
> > documentation. Anyone?
> 
> I don't think the documentation has a separate license; also BSD.
>
How the contributors know that their contributions would be released
under BSD ? 




From pav at iki.fi  Mon Oct 21 15:39:53 2013
From: pav at iki.fi (Pauli Virtanen)
Date: Mon, 21 Oct 2013 22:39:53 +0300
Subject: [Numpy-discussion] Is there a contributors agreement for numypy?
In-Reply-To: <1382384171.24732.58.camel@mblaptop>
References: <1382377982.24732.12.camel@mblaptop>	<CAB6mnxLJnchv0uUyyCio==b-k+n+cYGx+TVEt+6JQ6WUT-g8gQ@mail.gmail.com>	<l43re0$95u$1@ger.gmane.org>
	<1382384171.24732.58.camel@mblaptop>
Message-ID: <l43vtv$una$1@ger.gmane.org>

21.10.2013 22:36, Mounir E. Bsaibes kirjoitti:
> On Mon, 2013-10-21 at 21:23 +0300, Pauli Virtanen wrote:
>> 21.10.2013 21:00, Charles R Harris kirjoitti:
>> [clip]
>>> There is no agreement needed, but all numpy is released under the 
>>> simplified BSD license and any contributions need to be compatible
>>> with that. I don't know that there is any special license for the
>>> documentation. Anyone?
>>
>> I don't think the documentation has a separate license; also BSD.
>
> How the contributors know that their contributions would be released
> under BSD ?

The project is BSD-licensed. Contributing implies agreement to the license.

-- 
Pauli Virtanen



From chris.barker at noaa.gov  Mon Oct 21 23:29:52 2013
From: chris.barker at noaa.gov (Chris Barker)
Date: Mon, 21 Oct 2013 20:29:52 -0700
Subject: [Numpy-discussion] "official" binaries on web page.
Message-ID: <CALGmxE+f7MczNDeOdYNsuKACQd4zzruUGzKz23DB_8esGtMn2g@mail.gmail.com>

If you go to numpy.org, and try to figure out how to install numpy,
you are most likely to end up here:

http://www.scipy.org/install.html

where there is no mention of the binaries built by the numpy project
itself, either Windows or Mac.

There probably should be.

-Chris


-- 

Christopher Barker, Ph.D.
Oceanographer

Emergency Response Division
NOAA/NOS/OR&R            (206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115       (206) 526-6317   main reception

Chris.Barker at noaa.gov


From Marc.Poinot at onera.fr  Tue Oct 22 05:31:05 2013
From: Marc.Poinot at onera.fr (Marc.Poinot at onera.fr)
Date: Tue, 22 Oct 2013 11:31:05 +0200
Subject: [Numpy-discussion] Contiguous memory zone with C API
In-Reply-To: <52650E79.9020601@onera.fr>
References: <52650E79.9020601@onera.fr>
Message-ID: <526645D9.7040404@onera.fr>

On 10/21/13 13:22, Marc.Poinot at onera.fr wrote:
> Hi all,
>
> I try to change my old C API numpy code to the 'new' API. I used to hack some internal stuff in Numpy (yes it's bad...) and I wonder now how to change it.
>
Ok, I've used PyDataMem_RENEW to reallocate through the API, but there's no way to change data pointer or ownership flag.
I think the API miss with some kind of:

PyDataMem_SET(PyArrayObject *object, void *ptr) to set actual data field (hidden now in PyArrayObject_fields)

and

PyArray_SETOWNDATA(arr) PyArray_UNSETOWNDATA(arr) to set/unset OWN DATA flag if you change memory from external allocation (at my own risks)
PyArray_ISOWNDATA(arr) to check it

This is more or less what is performed in shape.c or ctors.c for example, I use to operate on arr->data and arr->flags but if you plan to hide it and to move it 'to a private 
header in a future release' it would be nice to have some macros.

-- 
-- ----------------------------------------------------------------------------
-- Marc POINOT [ONERA/DSNA] Tel:+33.1.46.73.42.84  Fax:+33.1.46.73.41.66
-- Avertissement/disclaimer http://www.onera.fr/en/emails-terms
-- ----------------------------------------------------------------------------
--



From pav at iki.fi  Tue Oct 22 09:07:17 2013
From: pav at iki.fi (Pauli Virtanen)
Date: Tue, 22 Oct 2013 16:07:17 +0300
Subject: [Numpy-discussion] "official" binaries on web page.
In-Reply-To: <CALGmxE+f7MczNDeOdYNsuKACQd4zzruUGzKz23DB_8esGtMn2g@mail.gmail.com>
References: <CALGmxE+f7MczNDeOdYNsuKACQd4zzruUGzKz23DB_8esGtMn2g@mail.gmail.com>
Message-ID: <l45t9s$tgs$1@ger.gmane.org>

22.10.2013 06:29, Chris Barker kirjoitti:
> If you go to numpy.org, and try to figure out how to install numpy,
> you are most likely to end up here:
> 
> http://www.scipy.org/install.html
> 
> where there is no mention of the binaries built by the numpy project
> itself, either Windows or Mac.

The links are there: http://www.scipy.org/install.html#custom

-- 
Pauli Virtanen




From chris.barker at noaa.gov  Tue Oct 22 10:28:54 2013
From: chris.barker at noaa.gov (Chris Barker)
Date: Tue, 22 Oct 2013 07:28:54 -0700
Subject: [Numpy-discussion] "official" binaries on web page.
In-Reply-To: <l45t9s$tgs$1@ger.gmane.org>
References: <CALGmxE+f7MczNDeOdYNsuKACQd4zzruUGzKz23DB_8esGtMn2g@mail.gmail.com>
	<l45t9s$tgs$1@ger.gmane.org>
Message-ID: <CALGmxELEb1fxkgK=v9v4vjofpChE2c5bN5Mr=qufFxFEQ+7ruw@mail.gmail.com>

On Tue, Oct 22, 2013 at 6:07 AM, Pauli Virtanen <pav at iki.fi> wrote:
> 22.10.2013 06:29, Chris Barker kirjoitti:
>> If you go to numpy.org, and try to figure out how to install numpy,
>> you are most likely to end up here:
>>
>> http://www.scipy.org/install.html
>>
>> where there is no mention of the binaries built by the numpy project
>> itself, either Windows or Mac.
>
> The links are there: http://www.scipy.org/install.html#custom

Boy! that's subtle -- I literally looked at least 3-4 times and didn't notice.

It seems a bit odd that they are under "custom", and kind of as a side note:

"projects? sites may also offer official binary packages (e.g. numpy,
scipy library)"

and "may" -- this is the official site, and we don't _know_ if
binaries are provided?

Anyway, a lot of effort goes into those, it'd be nice for it to be
more prominent.

-Chris





-- 

Christopher Barker, Ph.D.
Oceanographer

Emergency Response Division
NOAA/NOS/OR&R            (206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115       (206) 526-6317   main reception

Chris.Barker at noaa.gov


From jjhelmus at gmail.com  Tue Oct 22 10:48:10 2013
From: jjhelmus at gmail.com (Jonathan Helmus)
Date: Tue, 22 Oct 2013 09:48:10 -0500
Subject: [Numpy-discussion] "official" binaries on web page.
In-Reply-To: <CALGmxELEb1fxkgK=v9v4vjofpChE2c5bN5Mr=qufFxFEQ+7ruw@mail.gmail.com>
References: <CALGmxE+f7MczNDeOdYNsuKACQd4zzruUGzKz23DB_8esGtMn2g@mail.gmail.com>	<l45t9s$tgs$1@ger.gmane.org>
	<CALGmxELEb1fxkgK=v9v4vjofpChE2c5bN5Mr=qufFxFEQ+7ruw@mail.gmail.com>
Message-ID: <5266902A.8040606@gmail.com>

On 10/22/2013 09:28 AM, Chris Barker wrote:
> On Tue, Oct 22, 2013 at 6:07 AM, Pauli Virtanen <pav at iki.fi> wrote:
>> 22.10.2013 06:29, Chris Barker kirjoitti:
>>> If you go to numpy.org, and try to figure out how to install numpy,
>>> you are most likely to end up here:
>>>
>>> http://www.scipy.org/install.html
>>>
>>> where there is no mention of the binaries built by the numpy project
>>> itself, either Windows or Mac.
>> The links are there: http://www.scipy.org/install.html#custom
> Boy! that's subtle -- I literally looked at least 3-4 times and didn't notice.
>
> It seems a bit odd that they are under "custom", and kind of as a side note:
>
> "projects? sites may also offer official binary packages (e.g. numpy,
> scipy library)"
>
> and "may" -- this is the official site, and we don't _know_ if
> binaries are provided?
>
> Anyway, a lot of effort goes into those, it'd be nice for it to be
> more prominent.
>
> -Chris

That page is generated from the rst file 
https://github.com/scipy/scipy.org/blob/master/www/install.rst.  I'm 
sure a Pull requests against that repository would be welcome.  You can 
even do it with an online editor, 
https://github.com/scipy/scipy.org/edit/master/www/install.rst!

     - Jonathan Helmus


From ralf.gommers at gmail.com  Tue Oct 22 12:36:07 2013
From: ralf.gommers at gmail.com (Ralf Gommers)
Date: Tue, 22 Oct 2013 18:36:07 +0200
Subject: [Numpy-discussion] "official" binaries on web page.
In-Reply-To: <CALGmxELEb1fxkgK=v9v4vjofpChE2c5bN5Mr=qufFxFEQ+7ruw@mail.gmail.com>
References: <CALGmxE+f7MczNDeOdYNsuKACQd4zzruUGzKz23DB_8esGtMn2g@mail.gmail.com>
	<l45t9s$tgs$1@ger.gmane.org>
	<CALGmxELEb1fxkgK=v9v4vjofpChE2c5bN5Mr=qufFxFEQ+7ruw@mail.gmail.com>
Message-ID: <CABL7CQhJ1S-swgy3jbBJAqYFh2gJtNjh-yEmFT+ONqkfdT6=DQ@mail.gmail.com>

On Tue, Oct 22, 2013 at 4:28 PM, Chris Barker <chris.barker at noaa.gov> wrote:

> On Tue, Oct 22, 2013 at 6:07 AM, Pauli Virtanen <pav at iki.fi> wrote:
> > 22.10.2013 06:29, Chris Barker kirjoitti:
> >> If you go to numpy.org, and try to figure out how to install numpy,
> >> you are most likely to end up here:
> >>
> >> http://www.scipy.org/install.html
> >>
> >> where there is no mention of the binaries built by the numpy project
> >> itself, either Windows or Mac.
> >
> > The links are there: http://www.scipy.org/install.html#custom
>
> Boy! that's subtle -- I literally looked at least 3-4 times and didn't
> notice.
>
> It seems a bit odd that they are under "custom", and kind of as a side
> note:
>
> "projects? sites may also offer official binary packages (e.g. numpy,
> scipy library)"
>
> and "may" -- this is the official site, and we don't _know_ if
> binaries are provided?
>
> Anyway, a lot of effort goes into those, it'd be nice for it to be
> more prominent.
>

You can argue with the exact wording, but the layout of that page is on
purpose. scipy.org is split into two parts: (a) a SciPy Stack part, and (b)
a numpy & scipy library part. You're looking at the stack part, and the
preferred method to install that stack is a Python distribution.

If you look at the library part, the binary installers are at the top of
the page: http://www.scipy.org/scipylib/download.html

Cheers,
Ralf



> -Chris
>
>
>
>
>
> --
>
> Christopher Barker, Ph.D.
> Oceanographer
>
> Emergency Response Division
> NOAA/NOS/OR&R            (206) 526-6959   voice
> 7600 Sand Point Way NE   (206) 526-6329   fax
> Seattle, WA  98115       (206) 526-6317   main reception
>
> Chris.Barker at noaa.gov
> _______________________________________________
> NumPy-Discussion mailing list
> NumPy-Discussion at scipy.org
> http://mail.scipy.org/mailman/listinfo/numpy-discussion
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20131022/e46ac0af/attachment.html>

From cgohlke at uci.edu  Tue Oct 22 16:35:27 2013
From: cgohlke at uci.edu (Christoph Gohlke)
Date: Tue, 22 Oct 2013 13:35:27 -0700
Subject: [Numpy-discussion] Numpy 1.8.0 release
In-Reply-To: <CAB6mnx+Y58NeHP9YmYfKNCG1Yjmb1f75kmBvrr2wEJ+y0FHs7g@mail.gmail.com>
References: <CAB6mnx+Y58NeHP9YmYfKNCG1Yjmb1f75kmBvrr2wEJ+y0FHs7g@mail.gmail.com>
Message-ID: <5266E18F.1090503@uci.edu>

On 10/20/2013 10:35 AM, Charles R Harris wrote:
> Hi All,
>
> I'm planning on releasing Numpy 1.8.0 next weekend. There have been a
> few minor fixes since 1.8.0rc2, but nothing that I think warrants
> another rc release. Please make sure to test the 1.8.0rc2 or
> maintenance/1.8.x branch with your code, for after next weekend it will
> be too late.
>
> Chuck
>

Hello,

how about backpoting "BUG: fix small issues found with cppcheck" 
<https://github.com/numpy/numpy/pull/3832> to 1.7.x?

Christoph


From chris.barker at noaa.gov  Tue Oct 22 16:44:51 2013
From: chris.barker at noaa.gov (Chris Barker)
Date: Tue, 22 Oct 2013 13:44:51 -0700
Subject: [Numpy-discussion] "official" binaries on web page.
In-Reply-To: <CABL7CQhJ1S-swgy3jbBJAqYFh2gJtNjh-yEmFT+ONqkfdT6=DQ@mail.gmail.com>
References: <CALGmxE+f7MczNDeOdYNsuKACQd4zzruUGzKz23DB_8esGtMn2g@mail.gmail.com>
	<l45t9s$tgs$1@ger.gmane.org>
	<CALGmxELEb1fxkgK=v9v4vjofpChE2c5bN5Mr=qufFxFEQ+7ruw@mail.gmail.com>
	<CABL7CQhJ1S-swgy3jbBJAqYFh2gJtNjh-yEmFT+ONqkfdT6=DQ@mail.gmail.com>
Message-ID: <CALGmxELbZ1SVJRr=Q52TRzHHrozix8p1UuV=Qajf5RJCyAn-7w@mail.gmail.com>

On Tue, Oct 22, 2013 at 9:36 AM, Ralf Gommers <ralf.gommers at gmail.com> wrote:
> You can argue with the exact wording,

I won't argue, I'll suggest an alternative in a pull request...

> but the layout of that page is on
> purpose. scipy.org is split into two parts: (a) a SciPy Stack part, and (b)
> a numpy & scipy library part. You're looking at the stack part, and the
> preferred method to install that stack is a Python distribution.

I'm not sure who decided that that's the "preferred" way, but If we
take that as a given, maybe that page is fine. However:

> If you look at the library part, the binary installers are at the top of the
> page: http://www.scipy.org/scipylib/download.html

But finding that page is not obvious. Simple usability:

I know what the heck I'm looking at -- I've been using numpy, and
numeric before it for something like 15 years. But I just went to grab
a new binary, and had a heck of a time finding it -- I'm pretty sure I
ended up remembering that the downloads are on sourceforge and looking
there. And I've been teaching lately, and my students are having
trouble finding them, too.


If you google "numpy" you get the numpy.org page as the first hit

If you go to numpy.org, there is a "geting numpy" link, that takes you to:

http://www.scipy.org/install.html

There is also a "Installing NumPy and SciPy" link that takes you to
that same page.

There is a link to the sourceforge page further won, but no indication
of why you would want to go there.

So I still have no idea how I would find:

http://www.scipy.org/scipylib/download.html


So I submit that we could fix this up a bit....

(off to do a pull request)

-Chris










> Cheers,
> Ralf
>
>
>>
>> -Chris
>>
>>
>>
>>
>>
>> --
>>
>> Christopher Barker, Ph.D.
>> Oceanographer
>>
>> Emergency Response Division
>> NOAA/NOS/OR&R            (206) 526-6959   voice
>> 7600 Sand Point Way NE   (206) 526-6329   fax
>> Seattle, WA  98115       (206) 526-6317   main reception
>>
>> Chris.Barker at noaa.gov
>> _______________________________________________
>> NumPy-Discussion mailing list
>> NumPy-Discussion at scipy.org
>> http://mail.scipy.org/mailman/listinfo/numpy-discussion
>
>
>
> _______________________________________________
> NumPy-Discussion mailing list
> NumPy-Discussion at scipy.org
> http://mail.scipy.org/mailman/listinfo/numpy-discussion
>



-- 

Christopher Barker, Ph.D.
Oceanographer

Emergency Response Division
NOAA/NOS/OR&R            (206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115       (206) 526-6317   main reception

Chris.Barker at noaa.gov


From chris.barker at noaa.gov  Tue Oct 22 17:14:52 2013
From: chris.barker at noaa.gov (Chris Barker)
Date: Tue, 22 Oct 2013 14:14:52 -0700
Subject: [Numpy-discussion] "official" binaries on web page.
In-Reply-To: <CALGmxELbZ1SVJRr=Q52TRzHHrozix8p1UuV=Qajf5RJCyAn-7w@mail.gmail.com>
References: <CALGmxE+f7MczNDeOdYNsuKACQd4zzruUGzKz23DB_8esGtMn2g@mail.gmail.com>
	<l45t9s$tgs$1@ger.gmane.org>
	<CALGmxELEb1fxkgK=v9v4vjofpChE2c5bN5Mr=qufFxFEQ+7ruw@mail.gmail.com>
	<CABL7CQhJ1S-swgy3jbBJAqYFh2gJtNjh-yEmFT+ONqkfdT6=DQ@mail.gmail.com>
	<CALGmxELbZ1SVJRr=Q52TRzHHrozix8p1UuV=Qajf5RJCyAn-7w@mail.gmail.com>
Message-ID: <CALGmxELuxCDxyh1JH4aTdB2_bX=bkDmtZJp8TtNkgYVsPvF9pA@mail.gmail.com>

On Tue, Oct 22, 2013 at 1:44 PM, Chris Barker <chris.barker at noaa.gov> wrote:
> (off to do a pull request)

Done.

https://github.com/scipy/scipy.org/pull/30

-Chris

-- 

Christopher Barker, Ph.D.
Oceanographer

Emergency Response Division
NOAA/NOS/OR&R            (206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115       (206) 526-6317   main reception

Chris.Barker at noaa.gov


From chris.barker at noaa.gov  Tue Oct 22 17:18:53 2013
From: chris.barker at noaa.gov (Chris Barker)
Date: Tue, 22 Oct 2013 14:18:53 -0700
Subject: [Numpy-discussion] "official" binaries on web page.
In-Reply-To: <CALGmxELbZ1SVJRr=Q52TRzHHrozix8p1UuV=Qajf5RJCyAn-7w@mail.gmail.com>
References: <CALGmxE+f7MczNDeOdYNsuKACQd4zzruUGzKz23DB_8esGtMn2g@mail.gmail.com>
	<l45t9s$tgs$1@ger.gmane.org>
	<CALGmxELEb1fxkgK=v9v4vjofpChE2c5bN5Mr=qufFxFEQ+7ruw@mail.gmail.com>
	<CABL7CQhJ1S-swgy3jbBJAqYFh2gJtNjh-yEmFT+ONqkfdT6=DQ@mail.gmail.com>
	<CALGmxELbZ1SVJRr=Q52TRzHHrozix8p1UuV=Qajf5RJCyAn-7w@mail.gmail.com>
Message-ID: <CALGmxEJ4mXr_mxjA4SbmJM58+PECdR=iE6GgZ52CLcz2nOJ9xA@mail.gmail.com>

By the way:

On Tue, Oct 22, 2013 at 1:44 PM, Chris Barker <chris.barker at noaa.gov> wrote:

> If you google "numpy" you get the numpy.org page as the first hit
>
> If you go to numpy.org, there is a "geting numpy" link, that takes you to:
>
> http://www.scipy.org/install.html

The www.numpy.org page could perhaps use some updating, but I can't
find that one on gitHub.

-Chris


-- 

Christopher Barker, Ph.D.
Oceanographer

Emergency Response Division
NOAA/NOS/OR&R            (206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115       (206) 526-6317   main reception

Chris.Barker at noaa.gov


From ralf.gommers at gmail.com  Tue Oct 22 17:23:12 2013
From: ralf.gommers at gmail.com (Ralf Gommers)
Date: Tue, 22 Oct 2013 23:23:12 +0200
Subject: [Numpy-discussion] "official" binaries on web page.
In-Reply-To: <CALGmxEJ4mXr_mxjA4SbmJM58+PECdR=iE6GgZ52CLcz2nOJ9xA@mail.gmail.com>
References: <CALGmxE+f7MczNDeOdYNsuKACQd4zzruUGzKz23DB_8esGtMn2g@mail.gmail.com>
	<l45t9s$tgs$1@ger.gmane.org>
	<CALGmxELEb1fxkgK=v9v4vjofpChE2c5bN5Mr=qufFxFEQ+7ruw@mail.gmail.com>
	<CABL7CQhJ1S-swgy3jbBJAqYFh2gJtNjh-yEmFT+ONqkfdT6=DQ@mail.gmail.com>
	<CALGmxELbZ1SVJRr=Q52TRzHHrozix8p1UuV=Qajf5RJCyAn-7w@mail.gmail.com>
	<CALGmxEJ4mXr_mxjA4SbmJM58+PECdR=iE6GgZ52CLcz2nOJ9xA@mail.gmail.com>
Message-ID: <CABL7CQj3i8uGda7Br0Sxy5tDJvDbGUCLgQWTpfjrpFwZ8neT7Q@mail.gmail.com>

On Tue, Oct 22, 2013 at 11:18 PM, Chris Barker <chris.barker at noaa.gov>wrote:

> By the way:
>
> On Tue, Oct 22, 2013 at 1:44 PM, Chris Barker <chris.barker at noaa.gov>
> wrote:
>
> > If you google "numpy" you get the numpy.org page as the first hit
> >
> > If you go to numpy.org, there is a "geting numpy" link, that takes you
> to:
> >
> > http://www.scipy.org/install.html
>
> The www.numpy.org page could perhaps use some updating, but I can't
> find that one on gitHub.
>

https://github.com/numpy/numpy.org

Agreed that numpy.org is not in great shape. Volunteers welcome I'd say.

Ralf
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20131022/51b792f6/attachment.html>

From ralf.gommers at gmail.com  Tue Oct 22 17:23:40 2013
From: ralf.gommers at gmail.com (Ralf Gommers)
Date: Tue, 22 Oct 2013 23:23:40 +0200
Subject: [Numpy-discussion] "official" binaries on web page.
In-Reply-To: <CALGmxELuxCDxyh1JH4aTdB2_bX=bkDmtZJp8TtNkgYVsPvF9pA@mail.gmail.com>
References: <CALGmxE+f7MczNDeOdYNsuKACQd4zzruUGzKz23DB_8esGtMn2g@mail.gmail.com>
	<l45t9s$tgs$1@ger.gmane.org>
	<CALGmxELEb1fxkgK=v9v4vjofpChE2c5bN5Mr=qufFxFEQ+7ruw@mail.gmail.com>
	<CABL7CQhJ1S-swgy3jbBJAqYFh2gJtNjh-yEmFT+ONqkfdT6=DQ@mail.gmail.com>
	<CALGmxELbZ1SVJRr=Q52TRzHHrozix8p1UuV=Qajf5RJCyAn-7w@mail.gmail.com>
	<CALGmxELuxCDxyh1JH4aTdB2_bX=bkDmtZJp8TtNkgYVsPvF9pA@mail.gmail.com>
Message-ID: <CABL7CQj+uUq32kWfzMz+ECF_D0Uh6nCB1sZsvsY9aUVMMZ9UhA@mail.gmail.com>

On Tue, Oct 22, 2013 at 11:14 PM, Chris Barker <chris.barker at noaa.gov>wrote:

> On Tue, Oct 22, 2013 at 1:44 PM, Chris Barker <chris.barker at noaa.gov>
> wrote:
> > (off to do a pull request)
>
> Done.
>
> https://github.com/scipy/scipy.org/pull/30
>

Thanks Chris!
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20131022/d7198845/attachment.html>

From ralf.gommers at gmail.com  Tue Oct 22 17:28:10 2013
From: ralf.gommers at gmail.com (Ralf Gommers)
Date: Tue, 22 Oct 2013 23:28:10 +0200
Subject: [Numpy-discussion] "official" binaries on web page.
In-Reply-To: <CALGmxELbZ1SVJRr=Q52TRzHHrozix8p1UuV=Qajf5RJCyAn-7w@mail.gmail.com>
References: <CALGmxE+f7MczNDeOdYNsuKACQd4zzruUGzKz23DB_8esGtMn2g@mail.gmail.com>
	<l45t9s$tgs$1@ger.gmane.org>
	<CALGmxELEb1fxkgK=v9v4vjofpChE2c5bN5Mr=qufFxFEQ+7ruw@mail.gmail.com>
	<CABL7CQhJ1S-swgy3jbBJAqYFh2gJtNjh-yEmFT+ONqkfdT6=DQ@mail.gmail.com>
	<CALGmxELbZ1SVJRr=Q52TRzHHrozix8p1UuV=Qajf5RJCyAn-7w@mail.gmail.com>
Message-ID: <CABL7CQggFwDU+1b-wn++fMuOhjpnDReCiMRq4-OCJVYeNaE-sw@mail.gmail.com>

On Tue, Oct 22, 2013 at 10:44 PM, Chris Barker <chris.barker at noaa.gov>wrote:

> On Tue, Oct 22, 2013 at 9:36 AM, Ralf Gommers <ralf.gommers at gmail.com>
> wrote:
> > You can argue with the exact wording,
>
> I won't argue, I'll suggest an alternative in a pull request...
>
> > but the layout of that page is on
> > purpose. scipy.org is split into two parts: (a) a SciPy Stack part, and
> (b)
> > a numpy & scipy library part. You're looking at the stack part, and the
> > preferred method to install that stack is a Python distribution.
>
> I'm not sure who decided that that's the "preferred" way


https://github.com/scipy/scipy.org/pull/3

I'm sure the there mentioned scipy-user and numfocus mailing list
discussions from a year ago can be found as well with a bit of searching.

Cheers,
Ralf

, but If we
> take that as a given, maybe that page is fine. However:
>
> > If you look at the library part, the binary installers are at the top of
> the
> > page: http://www.scipy.org/scipylib/download.html
>
> But finding that page is not obvious. Simple usability:
>
> I know what the heck I'm looking at -- I've been using numpy, and
> numeric before it for something like 15 years. But I just went to grab
> a new binary, and had a heck of a time finding it -- I'm pretty sure I
> ended up remembering that the downloads are on sourceforge and looking
> there. And I've been teaching lately, and my students are having
> trouble finding them, too.
>
>
> If you google "numpy" you get the numpy.org page as the first hit
>
> If you go to numpy.org, there is a "geting numpy" link, that takes you to:
>
> http://www.scipy.org/install.html
>
> There is also a "Installing NumPy and SciPy" link that takes you to
> that same page.
>
> There is a link to the sourceforge page further won, but no indication
> of why you would want to go there.
>
> So I still have no idea how I would find:
>
> http://www.scipy.org/scipylib/download.html
>
>
> So I submit that we could fix this up a bit....
>
> (off to do a pull request)
>
> -Chris
>
>
>
>
>
>
>
>
>
>
> > Cheers,
> > Ralf
> >
> >
> >>
> >> -Chris
> >>
> >>
> >>
> >>
> >>
> >> --
> >>
> >> Christopher Barker, Ph.D.
> >> Oceanographer
> >>
> >> Emergency Response Division
> >> NOAA/NOS/OR&R            (206) 526-6959   voice
> >> 7600 Sand Point Way NE   (206) 526-6329   fax
> >> Seattle, WA  98115       (206) 526-6317   main reception
> >>
> >> Chris.Barker at noaa.gov
> >> _______________________________________________
> >> NumPy-Discussion mailing list
> >> NumPy-Discussion at scipy.org
> >> http://mail.scipy.org/mailman/listinfo/numpy-discussion
> >
> >
> >
> > _______________________________________________
> > NumPy-Discussion mailing list
> > NumPy-Discussion at scipy.org
> > http://mail.scipy.org/mailman/listinfo/numpy-discussion
> >
>
>
>
> --
>
> Christopher Barker, Ph.D.
> Oceanographer
>
> Emergency Response Division
> NOAA/NOS/OR&R            (206) 526-6959   voice
> 7600 Sand Point Way NE   (206) 526-6329   fax
> Seattle, WA  98115       (206) 526-6317   main reception
>
> Chris.Barker at noaa.gov
> _______________________________________________
> NumPy-Discussion mailing list
> NumPy-Discussion at scipy.org
> http://mail.scipy.org/mailman/listinfo/numpy-discussion
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20131022/c554270c/attachment.html>

From chris.barker at noaa.gov  Tue Oct 22 17:54:57 2013
From: chris.barker at noaa.gov (Chris Barker)
Date: Tue, 22 Oct 2013 14:54:57 -0700
Subject: [Numpy-discussion] "official" binaries on web page.
In-Reply-To: <CABL7CQj3i8uGda7Br0Sxy5tDJvDbGUCLgQWTpfjrpFwZ8neT7Q@mail.gmail.com>
References: <CALGmxE+f7MczNDeOdYNsuKACQd4zzruUGzKz23DB_8esGtMn2g@mail.gmail.com>
	<l45t9s$tgs$1@ger.gmane.org>
	<CALGmxELEb1fxkgK=v9v4vjofpChE2c5bN5Mr=qufFxFEQ+7ruw@mail.gmail.com>
	<CABL7CQhJ1S-swgy3jbBJAqYFh2gJtNjh-yEmFT+ONqkfdT6=DQ@mail.gmail.com>
	<CALGmxELbZ1SVJRr=Q52TRzHHrozix8p1UuV=Qajf5RJCyAn-7w@mail.gmail.com>
	<CALGmxEJ4mXr_mxjA4SbmJM58+PECdR=iE6GgZ52CLcz2nOJ9xA@mail.gmail.com>
	<CABL7CQj3i8uGda7Br0Sxy5tDJvDbGUCLgQWTpfjrpFwZ8neT7Q@mail.gmail.com>
Message-ID: <CALGmxEKpAuVRt5KGsSrH=w8WuNHyhF5Mk81-m_w_5=5Q5i9bqA@mail.gmail.com>

On Tue, Oct 22, 2013 at 2:23 PM, Ralf Gommers <ralf.gommers at gmail.com> wrote:
> https://github.com/numpy/numpy.org
>
> Agreed that numpy.org is not in great shape. Volunteers welcome I'd say.

I just did a pull request to update the "Getting Numpy" link -- but
yes, it needs more than that...someday.

-Chris

-- 

Christopher Barker, Ph.D.
Oceanographer

Emergency Response Division
NOAA/NOS/OR&R            (206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115       (206) 526-6317   main reception

Chris.Barker at noaa.gov


From charlesr.harris at gmail.com  Tue Oct 22 19:37:26 2013
From: charlesr.harris at gmail.com (Charles R Harris)
Date: Tue, 22 Oct 2013 17:37:26 -0600
Subject: [Numpy-discussion] Numpy 1.8.0 release
In-Reply-To: <5266E18F.1090503@uci.edu>
References: <CAB6mnx+Y58NeHP9YmYfKNCG1Yjmb1f75kmBvrr2wEJ+y0FHs7g@mail.gmail.com>
	<5266E18F.1090503@uci.edu>
Message-ID: <CAB6mnx+Dmf8T2LXwEwOpS4tQKNfUnbA6kiROeCZqZB63tvt-Kg@mail.gmail.com>

On Tue, Oct 22, 2013 at 2:35 PM, Christoph Gohlke <cgohlke at uci.edu> wrote:

> On 10/20/2013 10:35 AM, Charles R Harris wrote:
> > Hi All,
> >
> > I'm planning on releasing Numpy 1.8.0 next weekend. There have been a
> > few minor fixes since 1.8.0rc2, but nothing that I think warrants
> > another rc release. Please make sure to test the 1.8.0rc2 or
> > maintenance/1.8.x branch with your code, for after next weekend it will
> > be too late.
> >
> > Chuck
> >
>
> Hello,
>
> how about backpoting "BUG: fix small issues found with cppcheck"
> <https://github.com/numpy/numpy/pull/3832> to 1.7.x?
>

Backported... Chuck
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20131022/b9b05830/attachment.html>

From matthew.brett at gmail.com  Tue Oct 22 19:44:54 2013
From: matthew.brett at gmail.com (Matthew Brett)
Date: Tue, 22 Oct 2013 16:44:54 -0700
Subject: [Numpy-discussion] "official" binaries on web page.
In-Reply-To: <CABL7CQggFwDU+1b-wn++fMuOhjpnDReCiMRq4-OCJVYeNaE-sw@mail.gmail.com>
References: <CALGmxE+f7MczNDeOdYNsuKACQd4zzruUGzKz23DB_8esGtMn2g@mail.gmail.com>
	<l45t9s$tgs$1@ger.gmane.org>
	<CALGmxELEb1fxkgK=v9v4vjofpChE2c5bN5Mr=qufFxFEQ+7ruw@mail.gmail.com>
	<CABL7CQhJ1S-swgy3jbBJAqYFh2gJtNjh-yEmFT+ONqkfdT6=DQ@mail.gmail.com>
	<CALGmxELbZ1SVJRr=Q52TRzHHrozix8p1UuV=Qajf5RJCyAn-7w@mail.gmail.com>
	<CABL7CQggFwDU+1b-wn++fMuOhjpnDReCiMRq4-OCJVYeNaE-sw@mail.gmail.com>
Message-ID: <CAH6Pt5qEVEYEyAYrwu=Xp_BkbhNM+p56E+L76BQOLA2xQ-VOvQ@mail.gmail.com>

Hi,

On Tue, Oct 22, 2013 at 2:28 PM, Ralf Gommers <ralf.gommers at gmail.com> wrote:
>
>
>
> On Tue, Oct 22, 2013 at 10:44 PM, Chris Barker <chris.barker at noaa.gov>
> wrote:
>>
>> On Tue, Oct 22, 2013 at 9:36 AM, Ralf Gommers <ralf.gommers at gmail.com>
>> wrote:
>> > You can argue with the exact wording,
>>
>> I won't argue, I'll suggest an alternative in a pull request...
>>
>> > but the layout of that page is on
>> > purpose. scipy.org is split into two parts: (a) a SciPy Stack part, and
>> > (b)
>> > a numpy & scipy library part. You're looking at the stack part, and the
>> > preferred method to install that stack is a Python distribution.
>>
>> I'm not sure who decided that that's the "preferred" way
>
>
> https://github.com/scipy/scipy.org/pull/3
>
> I'm sure the there mentioned scipy-user and numfocus mailing list
> discussions from a year ago can be found as well with a bit of searching.

Do you happen to have any searchable phrases or leads for these
discussions?   That would be very helpful for reference.

These issues come up (without resolution) at :
http://mail.scipy.org/pipermail/numpy-discussion/2013-February/065339.html

Cheers,

Matthew


From chris.barker at noaa.gov  Wed Oct 23 10:51:59 2013
From: chris.barker at noaa.gov (Chris Barker - NOAA Federal)
Date: Wed, 23 Oct 2013 07:51:59 -0700
Subject: [Numpy-discussion] "official" binaries on web page.
In-Reply-To: <CAH6Pt5qEVEYEyAYrwu=Xp_BkbhNM+p56E+L76BQOLA2xQ-VOvQ@mail.gmail.com>
References: <CALGmxE+f7MczNDeOdYNsuKACQd4zzruUGzKz23DB_8esGtMn2g@mail.gmail.com>
	<l45t9s$tgs$1@ger.gmane.org>
	<CALGmxELEb1fxkgK=v9v4vjofpChE2c5bN5Mr=qufFxFEQ+7ruw@mail.gmail.com>
	<CABL7CQhJ1S-swgy3jbBJAqYFh2gJtNjh-yEmFT+ONqkfdT6=DQ@mail.gmail.com>
	<CALGmxELbZ1SVJRr=Q52TRzHHrozix8p1UuV=Qajf5RJCyAn-7w@mail.gmail.com>
	<CABL7CQggFwDU+1b-wn++fMuOhjpnDReCiMRq4-OCJVYeNaE-sw@mail.gmail.com>
	<CAH6Pt5qEVEYEyAYrwu=Xp_BkbhNM+p56E+L76BQOLA2xQ-VOvQ@mail.gmail.com>
Message-ID: <7401257806648702858@unknownmsgid>

 Ralf Gommers <ralf.gommers at gmail.com> wrote:
>>>> but the layout of that page is on
>>>> purpose. scipy.org is split into two parts: (a) a SciPy Stack part, and
>>>> (b)
>>>> a numpy & scipy library part. You're looking at the stack part, and the
>>>> preferred method to install that stack is a Python distribution.

OK, I'm not sure that's a great idea, but if we take that as a given:

That page could use some clarification about what the heck the "stack"
is, and what its relationship to the scipy and numpy packages is.

And I still think it wouldn't hurt to more obviously point people to
how to get either numpy or scipy themselves.

So maybe my section about the "official" binaries, but lower on the
page. I don't like "custom" as a title, as that makes it sound like
advanced numpy-fu, not where a newbie looking for just numpy is going
to look.

But it sounds like the real problem is with the surrounding
pages--that's the page you find when you try to figure out how to get
numpy--if that page is about the stack, it should not be linked to
directly from the numpy.org page without explanation.

We do have a branding problem: "scipy" is a package, a "stack" and a
ecosystem/community. It should be clear which one is being referred to
when.

-Chris


From jim.vickroy at noaa.gov  Wed Oct 23 11:16:06 2013
From: jim.vickroy at noaa.gov (jim vickroy)
Date: Wed, 23 Oct 2013 09:16:06 -0600
Subject: [Numpy-discussion] "official" binaries on web page.
In-Reply-To: <7401257806648702858@unknownmsgid>
References: <CALGmxE+f7MczNDeOdYNsuKACQd4zzruUGzKz23DB_8esGtMn2g@mail.gmail.com>
	<l45t9s$tgs$1@ger.gmane.org>
	<CALGmxELEb1fxkgK=v9v4vjofpChE2c5bN5Mr=qufFxFEQ+7ruw@mail.gmail.com>
	<CABL7CQhJ1S-swgy3jbBJAqYFh2gJtNjh-yEmFT+ONqkfdT6=DQ@mail.gmail.com>
	<CALGmxELbZ1SVJRr=Q52TRzHHrozix8p1UuV=Qajf5RJCyAn-7w@mail.gmail.com>
	<CABL7CQggFwDU+1b-wn++fMuOhjpnDReCiMRq4-OCJVYeNaE-sw@mail.gmail.com>
	<CAH6Pt5qEVEYEyAYrwu=Xp_BkbhNM+p56E+L76BQOLA2xQ-VOvQ@mail.gmail.com>
	<7401257806648702858@unknownmsgid>
Message-ID: <5267E836.60708@noaa.gov>

On 10/23/2013 8:51 AM, Chris Barker - NOAA Federal wrote:
>   Ralf Gommers <ralf.gommers at gmail.com> wrote:
>>>>> but the layout of that page is on
>>>>> purpose. scipy.org is split into two parts: (a) a SciPy Stack part, and
>>>>> (b)
>>>>> a numpy & scipy library part. You're looking at the stack part, and the
>>>>> preferred method to install that stack is a Python distribution.
> OK, I'm not sure that's a great idea, but if we take that as a given:
>
> That page could use some clarification about what the heck the "stack"
> is, and what its relationship to the scipy and numpy packages is.
>
> And I still think it wouldn't hurt to more obviously point people to
> how to get either numpy or scipy themselves.
>
> So maybe my section about the "official" binaries, but lower on the
> page. I don't like "custom" as a title, as that makes it sound like
> advanced numpy-fu, not where a newbie looking for just numpy is going
> to look.
>
> But it sounds like the real problem is with the surrounding
> pages--that's the page you find when you try to figure out how to get
> numpy--if that page is about the stack, it should not be linked to
> directly from the numpy.org page without explanation.
>
> We do have a branding problem: "scipy" is a package, a "stack" and a
> ecosystem/community. It should be clear which one is being referred to
> when.
>
> -Chris
> _______________________________________________
> NumPy-Discussion mailing list
> NumPy-Discussion at scipy.org
> http://mail.scipy.org/mailman/listinfo/numpy-discussion
Chris, thanks for taking this on!  You very clearly state  all of the 
confusion I have had with Numpy and Scipy distributions and branding.  I 
also agree that relying on a Python distribution to provide Scipy and 
Numpy is not a good idea.  --jv
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20131023/9a6a9b17/attachment.html>

From matthew.brett at gmail.com  Wed Oct 23 13:10:21 2013
From: matthew.brett at gmail.com (Matthew Brett)
Date: Wed, 23 Oct 2013 10:10:21 -0700
Subject: [Numpy-discussion] "official" binaries on web page.
In-Reply-To: <5267E836.60708@noaa.gov>
References: <CALGmxE+f7MczNDeOdYNsuKACQd4zzruUGzKz23DB_8esGtMn2g@mail.gmail.com>
	<l45t9s$tgs$1@ger.gmane.org>
	<CALGmxELEb1fxkgK=v9v4vjofpChE2c5bN5Mr=qufFxFEQ+7ruw@mail.gmail.com>
	<CABL7CQhJ1S-swgy3jbBJAqYFh2gJtNjh-yEmFT+ONqkfdT6=DQ@mail.gmail.com>
	<CALGmxELbZ1SVJRr=Q52TRzHHrozix8p1UuV=Qajf5RJCyAn-7w@mail.gmail.com>
	<CABL7CQggFwDU+1b-wn++fMuOhjpnDReCiMRq4-OCJVYeNaE-sw@mail.gmail.com>
	<CAH6Pt5qEVEYEyAYrwu=Xp_BkbhNM+p56E+L76BQOLA2xQ-VOvQ@mail.gmail.com>
	<7401257806648702858@unknownmsgid> <5267E836.60708@noaa.gov>
Message-ID: <CAH6Pt5oWMBxA8qkuuSU3SzuP83f43X9DSaL2xYOuKNME15rY+w@mail.gmail.com>

Hi,

On Wed, Oct 23, 2013 at 8:16 AM, jim vickroy <jim.vickroy at noaa.gov> wrote:
> On 10/23/2013 8:51 AM, Chris Barker - NOAA Federal wrote:
>
>  Ralf Gommers <ralf.gommers at gmail.com> wrote:
>
> but the layout of that page is on
> purpose. scipy.org is split into two parts: (a) a SciPy Stack part, and
> (b)
> a numpy & scipy library part. You're looking at the stack part, and the
> preferred method to install that stack is a Python distribution.
>
> OK, I'm not sure that's a great idea, but if we take that as a given:
>
> That page could use some clarification about what the heck the "stack"
> is, and what its relationship to the scipy and numpy packages is.
>
> And I still think it wouldn't hurt to more obviously point people to
> how to get either numpy or scipy themselves.
>
> So maybe my section about the "official" binaries, but lower on the
> page. I don't like "custom" as a title, as that makes it sound like
> advanced numpy-fu, not where a newbie looking for just numpy is going
> to look.
>
> But it sounds like the real problem is with the surrounding
> pages--that's the page you find when you try to figure out how to get
> numpy--if that page is about the stack, it should not be linked to
> directly from the numpy.org page without explanation.
>
> We do have a branding problem: "scipy" is a package, a "stack" and a
> ecosystem/community. It should be clear which one is being referred to
> when.
>
> -Chris
> _______________________________________________
> NumPy-Discussion mailing list
> NumPy-Discussion at scipy.org
> http://mail.scipy.org/mailman/listinfo/numpy-discussion
>
> Chris, thanks for taking this on!  You very clearly state  all of the
> confusion I have had with Numpy and Scipy distributions and branding.  I
> also agree that relying on a Python distribution to provide Scipy and Numpy
> is not a good idea.  --jv

I know what you mean.

I think there are two groups of users out there:

1) Want software, want it now, with minimum fuss; no beef with
installer being primarily based at a single company
2) Like to know how everything got onto my computer, strong preference
for community-supported distributions

I guess that there's a fairly large group 2 in the Python community.
 For me, that was a powerful reason to switch from MATLAB.  Some of
this group is just starting, they own Macs or have Windows machines,
and they need binary installers.

I don't think there's any practical way of shifting people between
groups, and trying to do so will likely cause heated arguments.

There's no need to prefer one group over the other - we just need to
make sure that both groups have instructions and binaries they can
recognize as being for their case.  As in:

(Group 1): The easiest way to get ....
(Group 2): You can also install the stack from community-supported
binaries, this is more complicated, but possible by ...

Cheers,

Matthew


From pav at iki.fi  Wed Oct 23 13:55:19 2013
From: pav at iki.fi (Pauli Virtanen)
Date: Wed, 23 Oct 2013 20:55:19 +0300
Subject: [Numpy-discussion] "official" binaries on web page.
In-Reply-To: <CAH6Pt5oWMBxA8qkuuSU3SzuP83f43X9DSaL2xYOuKNME15rY+w@mail.gmail.com>
References: <CALGmxE+f7MczNDeOdYNsuKACQd4zzruUGzKz23DB_8esGtMn2g@mail.gmail.com>	<l45t9s$tgs$1@ger.gmane.org>	<CALGmxELEb1fxkgK=v9v4vjofpChE2c5bN5Mr=qufFxFEQ+7ruw@mail.gmail.com>	<CABL7CQhJ1S-swgy3jbBJAqYFh2gJtNjh-yEmFT+ONqkfdT6=DQ@mail.gmail.com>	<CALGmxELbZ1SVJRr=Q52TRzHHrozix8p1UuV=Qajf5RJCyAn-7w@mail.gmail.com>	<CABL7CQggFwDU+1b-wn++fMuOhjpnDReCiMRq4-OCJVYeNaE-sw@mail.gmail.com>	<CAH6Pt5qEVEYEyAYrwu=Xp_BkbhNM+p56E+L76BQOLA2xQ-VOvQ@mail.gmail.com>	<7401257806648702858@unknownmsgid>
	<5267E836.60708@noaa.gov>
	<CAH6Pt5oWMBxA8qkuuSU3SzuP83f43X9DSaL2xYOuKNME15rY+w@mail.gmail.com>
Message-ID: <l492ht$r9v$1@ger.gmane.org>

23.10.2013 20:10, Matthew Brett kirjoitti:
[clip]
> There's no need to prefer one group over the other - we just need to
> make sure that both groups have instructions and binaries they can
> recognize as being for their case.  As in:
> 
> (Group 1): The easiest way to get ....
> (Group 2): You can also install the stack from community-supported
> binaries, this is more complicated, but possible by ...

This is pretty much what scipy.org/install.html page currently says.
What can be improved is adding more noticeable links to the binaries.

I'm convinced that relying on a Python distribution on Windows and OSX
is a good idea, and needs to be emphasized over needs of advanced users,
who should have enough patience to read the bottom of the page.

-- 
Pauli Virtanen



From pav at iki.fi  Wed Oct 23 14:06:36 2013
From: pav at iki.fi (Pauli Virtanen)
Date: Wed, 23 Oct 2013 21:06:36 +0300
Subject: [Numpy-discussion] "official" binaries on web page.
In-Reply-To: <7401257806648702858@unknownmsgid>
References: <CALGmxE+f7MczNDeOdYNsuKACQd4zzruUGzKz23DB_8esGtMn2g@mail.gmail.com>	<l45t9s$tgs$1@ger.gmane.org>	<CALGmxELEb1fxkgK=v9v4vjofpChE2c5bN5Mr=qufFxFEQ+7ruw@mail.gmail.com>	<CABL7CQhJ1S-swgy3jbBJAqYFh2gJtNjh-yEmFT+ONqkfdT6=DQ@mail.gmail.com>	<CALGmxELbZ1SVJRr=Q52TRzHHrozix8p1UuV=Qajf5RJCyAn-7w@mail.gmail.com>	<CABL7CQggFwDU+1b-wn++fMuOhjpnDReCiMRq4-OCJVYeNaE-sw@mail.gmail.com>	<CAH6Pt5qEVEYEyAYrwu=Xp_BkbhNM+p56E+L76BQOLA2xQ-VOvQ@mail.gmail.com>
	<7401257806648702858@unknownmsgid>
Message-ID: <l49373$360$1@ger.gmane.org>

23.10.2013 17:51, Chris Barker - NOAA Federal kirjoitti:
[clip]
> But it sounds like the real problem is with the surrounding
> pages--that's the page you find when you try to figure out how to get
> numpy--if that page is about the stack, it should not be linked to
> directly from the numpy.org page without explanation.
> 
> We do have a branding problem: "scipy" is a package, a "stack" and a
> ecosystem/community. It should be clear which one is being referred to
> when.

Yep, the scipy.org website has a navigation structure problem, in that
the "scipy library" and "scipy stack and community" parts are not
separated clearly enough.

The navigation items for both sections are visible at the same time, the
graphical style is the same, numpy.org is on a different domain etc., so
it's a bit of a mess. Still an improvement over Moinmoin, though.

One option would be to separate the navigation tree of the "scipy
library" part from the entry page. This would likely make things much
clearer.

-- 
Pauli Virtanen



From matthew.brett at gmail.com  Wed Oct 23 14:10:52 2013
From: matthew.brett at gmail.com (Matthew Brett)
Date: Wed, 23 Oct 2013 11:10:52 -0700
Subject: [Numpy-discussion] "official" binaries on web page.
In-Reply-To: <l492ht$r9v$1@ger.gmane.org>
References: <CALGmxE+f7MczNDeOdYNsuKACQd4zzruUGzKz23DB_8esGtMn2g@mail.gmail.com>
	<l45t9s$tgs$1@ger.gmane.org>
	<CALGmxELEb1fxkgK=v9v4vjofpChE2c5bN5Mr=qufFxFEQ+7ruw@mail.gmail.com>
	<CABL7CQhJ1S-swgy3jbBJAqYFh2gJtNjh-yEmFT+ONqkfdT6=DQ@mail.gmail.com>
	<CALGmxELbZ1SVJRr=Q52TRzHHrozix8p1UuV=Qajf5RJCyAn-7w@mail.gmail.com>
	<CABL7CQggFwDU+1b-wn++fMuOhjpnDReCiMRq4-OCJVYeNaE-sw@mail.gmail.com>
	<CAH6Pt5qEVEYEyAYrwu=Xp_BkbhNM+p56E+L76BQOLA2xQ-VOvQ@mail.gmail.com>
	<7401257806648702858@unknownmsgid> <5267E836.60708@noaa.gov>
	<CAH6Pt5oWMBxA8qkuuSU3SzuP83f43X9DSaL2xYOuKNME15rY+w@mail.gmail.com>
	<l492ht$r9v$1@ger.gmane.org>
Message-ID: <CAH6Pt5pVR6RBXRGxFHAaE3yniyzk11-pV0xFXZt9hFcY60naGg@mail.gmail.com>

Hi,

On Wed, Oct 23, 2013 at 10:55 AM, Pauli Virtanen <pav at iki.fi> wrote:
> 23.10.2013 20:10, Matthew Brett kirjoitti:
> [clip]
>> There's no need to prefer one group over the other - we just need to
>> make sure that both groups have instructions and binaries they can
>> recognize as being for their case.  As in:
>>
>> (Group 1): The easiest way to get ....
>> (Group 2): You can also install the stack from community-supported
>> binaries, this is more complicated, but possible by ...
>
> This is pretty much what scipy.org/install.html page currently says.
> What can be improved is adding more noticeable links to the binaries.

Yes, sure.  Obviously it's a subtle matter of emphasis, and also
obviously, that current situation has some people confused.

> I'm convinced that relying on a Python distribution on Windows and OSX
> is a good idea, and needs to be emphasized over needs of advanced users,
> who should have enough patience to read the bottom of the page.

I am arguing that whether it's a good idea or not, according to us, is
not going affect the guys and gals in Group 2, and that, unless we
want to lose or discourage those people, we'll always need to support
(and give mind- and page-space to) community-built binary installers.

Cheers,

Matthew


From dketch at gmail.com  Wed Oct 23 14:41:59 2013
From: dketch at gmail.com (David Ketcheson)
Date: Wed, 23 Oct 2013 21:41:59 +0300
Subject: [Numpy-discussion] Redirecting f2py output to a logfile (when
	called via distutils)
Message-ID: <CAPdrMXirhRheAoNM==-G6gkVaWqqPn6mLXUW0E+pFbPfyPePgw@mail.gmail.com>

In Clawpack, we use numpy.distutils to handle compilation and wrapping of
Fortran source files via f2py.  This generates a huge amount of output
(warnings generated by f2py in intermediate steps) that (it seems) can
safely be ignored.  However, it's problematic to have pages of warnings fly
by whenever a user installs the package.  I would like to redirect this
output to a log file.

I have tried adding

    sys.stdout = open('install.log', 'w')

in the setup.py file.  This redirects some of the output, but not all of
it.  Below I've pasted an example of output that still prints to the
terminal.  How can I force all of this output to go into a log file and not
be printed to the screen?

An example setup.py file is here:
https://github.com/clawpack/pyclaw/blob/master/src/pyclaw/setup.py

Some example output that I can't redirect is here:

    In file included from
> build/src.macosx-10.5-x86_64-2.7/clawpack/riemann/euler_4wave_2Dmodule.c:18:
>
>     In file included from
> build/src.macosx-10.5-x86_64-2.7/fortranobject.h:13:
>
>     In file included from
> /Users/ketch/anaconda/lib/python2.7/site-packages/numpy/core/include/numpy/arrayobject.h:15:
>
>     In file included from
> /Users/ketch/anaconda/lib/python2.7/site-packages/numpy/core/include/numpy/ndarrayobject.h:17:
>
>     In file included from
> /Users/ketch/anaconda/lib/python2.7/site-packages/numpy/core/include/numpy/ndarraytypes.h:1728:
>
>
> /Users/ketch/anaconda/lib/python2.7/site-packages/numpy/core/include/numpy/npy_deprecated_api.h:11:2:
> warning: "Using deprecated NumPy API, disable it by #defining
> NPY_NO_DEPRECATED_API NPY_1_7_API_VERSION" [-W#warnings]
>
>     #warning "Using deprecated NumPy API, disable it by #defining
> NPY_NO_DEPRECATED_API NPY_1_7_API_VERSION"
>
>      ^
>
>
> build/src.macosx-10.5-x86_64-2.7/clawpack/riemann/euler_4wave_2Dmodule.c:111:12:
> warning: unused function 'f2py_size' [-Wunused-function]
>
>     static int f2py_size(PyArrayObject* var, ...)
>
>                ^
>
>     2 warnings generated.
>
>     clawpack/riemann/src/rpn2_euler_4wave.f90:2.59:
>
>
>
>     subroutine
> rpn2(ixy,maxm,meqn,mwaves,maux,mbc,mx,ql,qr,auxl,auxr,wave,s,amdq,ap
>
>                                                                1
>
>     Warning: Unused dummy argument 'auxl' at (1)
>
>     clawpack/riemann/src/rpn2_euler_4wave.f90:2.64:
>
>
>
>     subroutine
> rpn2(ixy,maxm,meqn,mwaves,maux,mbc,mx,ql,qr,auxl,auxr,wave,s,amdq,ap
>
>                                                                     1
>     Warning: Unused dummy argument 'auxr' at (1)
>

There is a LOT more of it, of course.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20131023/93bbf9c0/attachment.html>

From pav at iki.fi  Wed Oct 23 15:08:57 2013
From: pav at iki.fi (Pauli Virtanen)
Date: Wed, 23 Oct 2013 22:08:57 +0300
Subject: [Numpy-discussion] "official" binaries on web page.
In-Reply-To: <l49373$360$1@ger.gmane.org>
References: <CALGmxE+f7MczNDeOdYNsuKACQd4zzruUGzKz23DB_8esGtMn2g@mail.gmail.com>	<l45t9s$tgs$1@ger.gmane.org>	<CALGmxELEb1fxkgK=v9v4vjofpChE2c5bN5Mr=qufFxFEQ+7ruw@mail.gmail.com>	<CABL7CQhJ1S-swgy3jbBJAqYFh2gJtNjh-yEmFT+ONqkfdT6=DQ@mail.gmail.com>	<CALGmxELbZ1SVJRr=Q52TRzHHrozix8p1UuV=Qajf5RJCyAn-7w@mail.gmail.com>	<CABL7CQggFwDU+1b-wn++fMuOhjpnDReCiMRq4-OCJVYeNaE-sw@mail.gmail.com>	<CAH6Pt5qEVEYEyAYrwu=Xp_BkbhNM+p56E+L76BQOLA2xQ-VOvQ@mail.gmail.com>	<7401257806648702858@unknownmsgid>
	<l49373$360$1@ger.gmane.org>
Message-ID: <l496s0$gij$1@ger.gmane.org>

23.10.2013 21:06, Pauli Virtanen kirjoitti:
> 23.10.2013 17:51, Chris Barker - NOAA Federal kirjoitti:
> [clip]
>> But it sounds like the real problem is with the surrounding
>> pages--that's the page you find when you try to figure out how to get
>> numpy--if that page is about the stack, it should not be linked to
>> directly from the numpy.org page without explanation.
>>
>> We do have a branding problem: "scipy" is a package, a "stack" and a
>> ecosystem/community. It should be clear which one is being referred to
>> when.
> 
> Yep, the scipy.org website has a navigation structure problem, in that
> the "scipy library" and "scipy stack and community" parts are not
> separated clearly enough.

This may help:
https://github.com/scipy/scipy.org/pull/31




From argriffi at ncsu.edu  Wed Oct 23 15:37:01 2013
From: argriffi at ncsu.edu (alex)
Date: Wed, 23 Oct 2013 15:37:01 -0400
Subject: [Numpy-discussion] profiling numpy functions
Message-ID: <CAE5GFc+RtxfdUCyikBzB2aH+grqjrZ5Xj0sdib-2NgPvGEPbCg@mail.gmail.com>

I have a question about numpy profiling.  When I run my script with -m
cProfile I see timings for lots of functions, but not for some like
np.exp().  How do I see these timings?  I don't want to have to work around
it by writing a lot of functions like def mynumpyexp(A): return np.exp(A).
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20131023/5571a1c2/attachment.html>

From chris.barker at noaa.gov  Wed Oct 23 15:50:41 2013
From: chris.barker at noaa.gov (Chris Barker)
Date: Wed, 23 Oct 2013 12:50:41 -0700
Subject: [Numpy-discussion] "official" binaries on web page.
In-Reply-To: <CAH6Pt5pVR6RBXRGxFHAaE3yniyzk11-pV0xFXZt9hFcY60naGg@mail.gmail.com>
References: <CALGmxE+f7MczNDeOdYNsuKACQd4zzruUGzKz23DB_8esGtMn2g@mail.gmail.com>
	<l45t9s$tgs$1@ger.gmane.org>
	<CALGmxELEb1fxkgK=v9v4vjofpChE2c5bN5Mr=qufFxFEQ+7ruw@mail.gmail.com>
	<CABL7CQhJ1S-swgy3jbBJAqYFh2gJtNjh-yEmFT+ONqkfdT6=DQ@mail.gmail.com>
	<CALGmxELbZ1SVJRr=Q52TRzHHrozix8p1UuV=Qajf5RJCyAn-7w@mail.gmail.com>
	<CABL7CQggFwDU+1b-wn++fMuOhjpnDReCiMRq4-OCJVYeNaE-sw@mail.gmail.com>
	<CAH6Pt5qEVEYEyAYrwu=Xp_BkbhNM+p56E+L76BQOLA2xQ-VOvQ@mail.gmail.com>
	<7401257806648702858@unknownmsgid> <5267E836.60708@noaa.gov>
	<CAH6Pt5oWMBxA8qkuuSU3SzuP83f43X9DSaL2xYOuKNME15rY+w@mail.gmail.com>
	<l492ht$r9v$1@ger.gmane.org>
	<CAH6Pt5pVR6RBXRGxFHAaE3yniyzk11-pV0xFXZt9hFcY60naGg@mail.gmail.com>
Message-ID: <CALGmxEKwYrUopgbuAgVxrgtMq9wUf3J+Gi-ZvUApRfQJpTK3GQ@mail.gmail.com>

On Wed, Oct 23, 2013 at 11:10 AM, Matthew Brett <matthew.brett at gmail.com> wrote:

> I am arguing that whether it's a good idea or not, according to us, is
> not going affect the guys and gals in Group 2, and that, unless we
> want to lose or discourage those people, we'll always need to support
> (and give mind- and page-space to) community-built binary installers.

Indeed, and also web pages that make it easy to find them, and clear
what you are or aren't getting in each case.

When updating the scipy site, we do need to keep in mind that people
often (most often?) come into a particular page via a link from
elsewhere or a search -- i.e. they haven't read the home page, etc. So
even if a page is about how to install the entire scipy stack, folks
that are searching for just numpy, etc should find what they need --
or links to what they need.

This makes me think: apparently there is an offical "scipy stack" --
and I even found it with a quick google:

http://www.scipy.org/stackspec.html

There is not a whole lot there, and all the packages in it either
currently provide binary installers (or aren't all that hard to build,
or Chris Gohlke has built them). So maybe we should have a single
point of download for a "community" scipy stack, compatible with the
python.org python install.

All of the hard work has been done -- we just need to set it up
somewhere (and maintain it)

-Chris


Christopher Barker, Ph.D.
Oceanographer

Emergency Response Division
NOAA/NOS/OR&R            (206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115       (206) 526-6317   main reception

Chris.Barker at noaa.gov


From pav at iki.fi  Wed Oct 23 16:24:12 2013
From: pav at iki.fi (Pauli Virtanen)
Date: Wed, 23 Oct 2013 23:24:12 +0300
Subject: [Numpy-discussion] "official" binaries on web page.
In-Reply-To: <CALGmxEKwYrUopgbuAgVxrgtMq9wUf3J+Gi-ZvUApRfQJpTK3GQ@mail.gmail.com>
References: <CALGmxE+f7MczNDeOdYNsuKACQd4zzruUGzKz23DB_8esGtMn2g@mail.gmail.com>	<l45t9s$tgs$1@ger.gmane.org>	<CALGmxELEb1fxkgK=v9v4vjofpChE2c5bN5Mr=qufFxFEQ+7ruw@mail.gmail.com>	<CABL7CQhJ1S-swgy3jbBJAqYFh2gJtNjh-yEmFT+ONqkfdT6=DQ@mail.gmail.com>	<CALGmxELbZ1SVJRr=Q52TRzHHrozix8p1UuV=Qajf5RJCyAn-7w@mail.gmail.com>	<CABL7CQggFwDU+1b-wn++fMuOhjpnDReCiMRq4-OCJVYeNaE-sw@mail.gmail.com>	<CAH6Pt5qEVEYEyAYrwu=Xp_BkbhNM+p56E+L76BQOLA2xQ-VOvQ@mail.gmail.com>	<7401257806648702858@unknownmsgid>
	<5267E836.60708@noaa.gov>	<CAH6Pt5oWMBxA8qkuuSU3SzuP83f43X9DSaL2xYOuKNME15rY+w@mail.gmail.com>	<l492ht$r9v$1@ger.gmane.org>	<CAH6Pt5pVR6RBXRGxFHAaE3yniyzk11-pV0xFXZt9hFcY60naGg@mail.gmail.com>
	<CALGmxEKwYrUopgbuAgVxrgtMq9wUf3J+Gi-ZvUApRfQJpTK3GQ@mail.gmail.com>
Message-ID: <l49b92$3ga$1@ger.gmane.org>

23.10.2013 22:50, Chris Barker kirjoitti:
[clip]
> This makes me think: apparently there is an offical "scipy stack" --
> and I even found it with a quick google:
> 
> http://www.scipy.org/stackspec.html

If you click "More information..." on the front page, or "About Scipy"
in the sidebar, it takes you to an explanation that says that the scipy
exists and what it is. A newcomer may possibly read that.

-- 
Pauli Virtanen




From pav at iki.fi  Wed Oct 23 16:27:02 2013
From: pav at iki.fi (Pauli Virtanen)
Date: Wed, 23 Oct 2013 23:27:02 +0300
Subject: [Numpy-discussion] "official" binaries on web page.
In-Reply-To: <l49b92$3ga$1@ger.gmane.org>
References: <CALGmxE+f7MczNDeOdYNsuKACQd4zzruUGzKz23DB_8esGtMn2g@mail.gmail.com>	<l45t9s$tgs$1@ger.gmane.org>	<CALGmxELEb1fxkgK=v9v4vjofpChE2c5bN5Mr=qufFxFEQ+7ruw@mail.gmail.com>	<CABL7CQhJ1S-swgy3jbBJAqYFh2gJtNjh-yEmFT+ONqkfdT6=DQ@mail.gmail.com>	<CALGmxELbZ1SVJRr=Q52TRzHHrozix8p1UuV=Qajf5RJCyAn-7w@mail.gmail.com>	<CABL7CQggFwDU+1b-wn++fMuOhjpnDReCiMRq4-OCJVYeNaE-sw@mail.gmail.com>	<CAH6Pt5qEVEYEyAYrwu=Xp_BkbhNM+p56E+L76BQOLA2xQ-VOvQ@mail.gmail.com>	<7401257806648702858@unknownmsgid>	<5267E836.60708@noaa.gov>	<CAH6Pt5oWMBxA8qkuuSU3SzuP83f43X9DSaL2xYOuKNME15rY+w@mail.gmail.com>	<l492ht$r9v$1@ger.gmane.org>	<CAH6Pt5pVR6RBXRGxFHAaE3yniyzk11-pV0xFXZt9hFcY60naGg@mail.gmail.com>	<CALGmxEKwYrUopgbuAgVxrgtMq9wUf3J+Gi-ZvUApRfQJpTK3GQ@mail.gmail.com>
	<l49b92$3ga$1@ger.gmane.org>
Message-ID: <l49bec$3ga$2@ger.gmane.org>

23.10.2013 23:24, Pauli Virtanen kirjoitti:
> 23.10.2013 22:50, Chris Barker kirjoitti:
> [clip]
>> This makes me think: apparently there is an offical "scipy stack" --
>> and I even found it with a quick google:
>>
>> http://www.scipy.org/stackspec.html
> 
> If you click "More information..." on the front page, or "About Scipy"
> in the sidebar, it takes you to an explanation that says that the scipy
> exists and what it is. A newcomer may possibly read that.

The reason why it's so obscure is probably that the discussion seems to
have mostly been on the Numfocus mailing list, and not here, and I don't
remember it being announced at any point.

Oh well,

-- 
Pauli Virtanen



From chris.barker at noaa.gov  Wed Oct 23 16:32:23 2013
From: chris.barker at noaa.gov (Chris Barker)
Date: Wed, 23 Oct 2013 13:32:23 -0700
Subject: [Numpy-discussion] "official" binaries on web page.
In-Reply-To: <CALGmxEKwYrUopgbuAgVxrgtMq9wUf3J+Gi-ZvUApRfQJpTK3GQ@mail.gmail.com>
References: <CALGmxE+f7MczNDeOdYNsuKACQd4zzruUGzKz23DB_8esGtMn2g@mail.gmail.com>
	<l45t9s$tgs$1@ger.gmane.org>
	<CALGmxELEb1fxkgK=v9v4vjofpChE2c5bN5Mr=qufFxFEQ+7ruw@mail.gmail.com>
	<CABL7CQhJ1S-swgy3jbBJAqYFh2gJtNjh-yEmFT+ONqkfdT6=DQ@mail.gmail.com>
	<CALGmxELbZ1SVJRr=Q52TRzHHrozix8p1UuV=Qajf5RJCyAn-7w@mail.gmail.com>
	<CABL7CQggFwDU+1b-wn++fMuOhjpnDReCiMRq4-OCJVYeNaE-sw@mail.gmail.com>
	<CAH6Pt5qEVEYEyAYrwu=Xp_BkbhNM+p56E+L76BQOLA2xQ-VOvQ@mail.gmail.com>
	<7401257806648702858@unknownmsgid> <5267E836.60708@noaa.gov>
	<CAH6Pt5oWMBxA8qkuuSU3SzuP83f43X9DSaL2xYOuKNME15rY+w@mail.gmail.com>
	<l492ht$r9v$1@ger.gmane.org>
	<CAH6Pt5pVR6RBXRGxFHAaE3yniyzk11-pV0xFXZt9hFcY60naGg@mail.gmail.com>
	<CALGmxEKwYrUopgbuAgVxrgtMq9wUf3J+Gi-ZvUApRfQJpTK3GQ@mail.gmail.com>
Message-ID: <CALGmxEKW_nHRadO6xKvK8WPeT-OJMR=g3tDHmf=NPUViwrpaTw@mail.gmail.com>

Folks,

After some more discussion here and on the pull request I submitted
earlier, here is anew pull request -- I think conforms to what folks
suggested, but still makes it a lot easier for folks to find the
stand-alone installers if that's what they are looking for.

https://github.com/scipy/scipy.org/pull/32

-Chris


On Wed, Oct 23, 2013 at 12:50 PM, Chris Barker <chris.barker at noaa.gov> wrote:
> On Wed, Oct 23, 2013 at 11:10 AM, Matthew Brett <matthew.brett at gmail.com> wrote:
>
>> I am arguing that whether it's a good idea or not, according to us, is
>> not going affect the guys and gals in Group 2, and that, unless we
>> want to lose or discourage those people, we'll always need to support
>> (and give mind- and page-space to) community-built binary installers.
>
> Indeed, and also web pages that make it easy to find them, and clear
> what you are or aren't getting in each case.
>
> When updating the scipy site, we do need to keep in mind that people
> often (most often?) come into a particular page via a link from
> elsewhere or a search -- i.e. they haven't read the home page, etc. So
> even if a page is about how to install the entire scipy stack, folks
> that are searching for just numpy, etc should find what they need --
> or links to what they need.
>
> This makes me think: apparently there is an offical "scipy stack" --
> and I even found it with a quick google:
>
> http://www.scipy.org/stackspec.html
>
> There is not a whole lot there, and all the packages in it either
> currently provide binary installers (or aren't all that hard to build,
> or Chris Gohlke has built them). So maybe we should have a single
> point of download for a "community" scipy stack, compatible with the
> python.org python install.
>
> All of the hard work has been done -- we just need to set it up
> somewhere (and maintain it)
>
> -Chris
>
>
> Christopher Barker, Ph.D.
> Oceanographer
>
> Emergency Response Division
> NOAA/NOS/OR&R            (206) 526-6959   voice
> 7600 Sand Point Way NE   (206) 526-6329   fax
> Seattle, WA  98115       (206) 526-6317   main reception
>
> Chris.Barker at noaa.gov



-- 

Christopher Barker, Ph.D.
Oceanographer

Emergency Response Division
NOAA/NOS/OR&R            (206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115       (206) 526-6317   main reception

Chris.Barker at noaa.gov


From ralf.gommers at gmail.com  Wed Oct 23 16:34:38 2013
From: ralf.gommers at gmail.com (Ralf Gommers)
Date: Wed, 23 Oct 2013 22:34:38 +0200
Subject: [Numpy-discussion] "official" binaries on web page.
In-Reply-To: <l49bec$3ga$2@ger.gmane.org>
References: <CALGmxE+f7MczNDeOdYNsuKACQd4zzruUGzKz23DB_8esGtMn2g@mail.gmail.com>
	<l45t9s$tgs$1@ger.gmane.org>
	<CALGmxELEb1fxkgK=v9v4vjofpChE2c5bN5Mr=qufFxFEQ+7ruw@mail.gmail.com>
	<CABL7CQhJ1S-swgy3jbBJAqYFh2gJtNjh-yEmFT+ONqkfdT6=DQ@mail.gmail.com>
	<CALGmxELbZ1SVJRr=Q52TRzHHrozix8p1UuV=Qajf5RJCyAn-7w@mail.gmail.com>
	<CABL7CQggFwDU+1b-wn++fMuOhjpnDReCiMRq4-OCJVYeNaE-sw@mail.gmail.com>
	<CAH6Pt5qEVEYEyAYrwu=Xp_BkbhNM+p56E+L76BQOLA2xQ-VOvQ@mail.gmail.com>
	<7401257806648702858@unknownmsgid> <5267E836.60708@noaa.gov>
	<CAH6Pt5oWMBxA8qkuuSU3SzuP83f43X9DSaL2xYOuKNME15rY+w@mail.gmail.com>
	<l492ht$r9v$1@ger.gmane.org>
	<CAH6Pt5pVR6RBXRGxFHAaE3yniyzk11-pV0xFXZt9hFcY60naGg@mail.gmail.com>
	<CALGmxEKwYrUopgbuAgVxrgtMq9wUf3J+Gi-ZvUApRfQJpTK3GQ@mail.gmail.com>
	<l49b92$3ga$1@ger.gmane.org> <l49bec$3ga$2@ger.gmane.org>
Message-ID: <CABL7CQgwSSoUAe4aMhK2wEobWEDd_aZXHuMZv=p12CkQojvc3w@mail.gmail.com>

On Wed, Oct 23, 2013 at 10:27 PM, Pauli Virtanen <pav at iki.fi> wrote:

> 23.10.2013 23:24, Pauli Virtanen kirjoitti:
> > 23.10.2013 22:50, Chris Barker kirjoitti:
> > [clip]
> >> This makes me think: apparently there is an offical "scipy stack" --
> >> and I even found it with a quick google:
> >>
> >> http://www.scipy.org/stackspec.html
> >
> > If you click "More information..." on the front page, or "About Scipy"
> > in the sidebar, it takes you to an explanation that says that the scipy
> > exists and what it is. A newcomer may possibly read that.
>
> The reason why it's so obscure is probably that the discussion seems to
> have mostly been on the Numfocus mailing list, and not here, and I don't
> remember it being announced at any point.
>

Here are (some of?) the relevant threads:
http://thread.gmane.org/gmane.comp.python.scientific.user/32928/focus=32939
http://thread.gmane.org/gmane.comp.python.scientific.user/33270/focus=33313
https://groups.google.com/forum/#!searchin/numfocus/scipy$20stack/numfocus/MnRzBhmqXqk/mFGIIbRFWSIJ

Ralf
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20131023/ab005f27/attachment.html>

From josef.pktd at gmail.com  Wed Oct 23 16:34:55 2013
From: josef.pktd at gmail.com (josef.pktd at gmail.com)
Date: Wed, 23 Oct 2013 16:34:55 -0400
Subject: [Numpy-discussion] "official" binaries on web page.
In-Reply-To: <CALGmxEKW_nHRadO6xKvK8WPeT-OJMR=g3tDHmf=NPUViwrpaTw@mail.gmail.com>
References: <CALGmxE+f7MczNDeOdYNsuKACQd4zzruUGzKz23DB_8esGtMn2g@mail.gmail.com>
	<l45t9s$tgs$1@ger.gmane.org>
	<CALGmxELEb1fxkgK=v9v4vjofpChE2c5bN5Mr=qufFxFEQ+7ruw@mail.gmail.com>
	<CABL7CQhJ1S-swgy3jbBJAqYFh2gJtNjh-yEmFT+ONqkfdT6=DQ@mail.gmail.com>
	<CALGmxELbZ1SVJRr=Q52TRzHHrozix8p1UuV=Qajf5RJCyAn-7w@mail.gmail.com>
	<CABL7CQggFwDU+1b-wn++fMuOhjpnDReCiMRq4-OCJVYeNaE-sw@mail.gmail.com>
	<CAH6Pt5qEVEYEyAYrwu=Xp_BkbhNM+p56E+L76BQOLA2xQ-VOvQ@mail.gmail.com>
	<7401257806648702858@unknownmsgid> <5267E836.60708@noaa.gov>
	<CAH6Pt5oWMBxA8qkuuSU3SzuP83f43X9DSaL2xYOuKNME15rY+w@mail.gmail.com>
	<l492ht$r9v$1@ger.gmane.org>
	<CAH6Pt5pVR6RBXRGxFHAaE3yniyzk11-pV0xFXZt9hFcY60naGg@mail.gmail.com>
	<CALGmxEKwYrUopgbuAgVxrgtMq9wUf3J+Gi-ZvUApRfQJpTK3GQ@mail.gmail.com>
	<CALGmxEKW_nHRadO6xKvK8WPeT-OJMR=g3tDHmf=NPUViwrpaTw@mail.gmail.com>
Message-ID: <CAMMTP+CtNoq3u2nFDU0jpFFeL0RNduqUw_D0h2G1yZqW=0PU9g@mail.gmail.com>

My first stop is always pypi to find the link to binaries.

https://pypi.python.org/pypi/scipy
however the direct download link points to 0.11

Josef


From ralf.gommers at gmail.com  Wed Oct 23 16:45:47 2013
From: ralf.gommers at gmail.com (Ralf Gommers)
Date: Wed, 23 Oct 2013 22:45:47 +0200
Subject: [Numpy-discussion] "official" binaries on web page.
In-Reply-To: <CAMMTP+CtNoq3u2nFDU0jpFFeL0RNduqUw_D0h2G1yZqW=0PU9g@mail.gmail.com>
References: <CALGmxE+f7MczNDeOdYNsuKACQd4zzruUGzKz23DB_8esGtMn2g@mail.gmail.com>
	<l45t9s$tgs$1@ger.gmane.org>
	<CALGmxELEb1fxkgK=v9v4vjofpChE2c5bN5Mr=qufFxFEQ+7ruw@mail.gmail.com>
	<CABL7CQhJ1S-swgy3jbBJAqYFh2gJtNjh-yEmFT+ONqkfdT6=DQ@mail.gmail.com>
	<CALGmxELbZ1SVJRr=Q52TRzHHrozix8p1UuV=Qajf5RJCyAn-7w@mail.gmail.com>
	<CABL7CQggFwDU+1b-wn++fMuOhjpnDReCiMRq4-OCJVYeNaE-sw@mail.gmail.com>
	<CAH6Pt5qEVEYEyAYrwu=Xp_BkbhNM+p56E+L76BQOLA2xQ-VOvQ@mail.gmail.com>
	<7401257806648702858@unknownmsgid> <5267E836.60708@noaa.gov>
	<CAH6Pt5oWMBxA8qkuuSU3SzuP83f43X9DSaL2xYOuKNME15rY+w@mail.gmail.com>
	<l492ht$r9v$1@ger.gmane.org>
	<CAH6Pt5pVR6RBXRGxFHAaE3yniyzk11-pV0xFXZt9hFcY60naGg@mail.gmail.com>
	<CALGmxEKwYrUopgbuAgVxrgtMq9wUf3J+Gi-ZvUApRfQJpTK3GQ@mail.gmail.com>
	<CALGmxEKW_nHRadO6xKvK8WPeT-OJMR=g3tDHmf=NPUViwrpaTw@mail.gmail.com>
	<CAMMTP+CtNoq3u2nFDU0jpFFeL0RNduqUw_D0h2G1yZqW=0PU9g@mail.gmail.com>
Message-ID: <CABL7CQjPTpJjS8pOCQLYgTQ-4C3RpJOEhYJayHVn4cmBbrrDDA@mail.gmail.com>

On Wed, Oct 23, 2013 at 10:34 PM, <josef.pktd at gmail.com> wrote:

> My first stop is always pypi to find the link to binaries.
>
> https://pypi.python.org/pypi/scipy
> however the direct download link points to 0.11
>

Grrr. IIRC that's because you have to link a page on SF which has the
actual binaries, and that link has to be updated by hand which I apparently
forgot when making a release.

I'd be tempted to replace that unreadable link with
http://sourceforge.net/projects/scipy/files/scipy/ and not care about
pip/easy_install scanning that page. Binary installers don't work anyway
and source bundles are on pypi already, so download_url is mainly for human
consumption.

Ralf
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20131023/a505a05d/attachment.html>

From chris.barker at noaa.gov  Wed Oct 23 17:03:43 2013
From: chris.barker at noaa.gov (Chris Barker)
Date: Wed, 23 Oct 2013 14:03:43 -0700
Subject: [Numpy-discussion] "official" binaries on web page.
In-Reply-To: <CABL7CQjPTpJjS8pOCQLYgTQ-4C3RpJOEhYJayHVn4cmBbrrDDA@mail.gmail.com>
References: <CALGmxE+f7MczNDeOdYNsuKACQd4zzruUGzKz23DB_8esGtMn2g@mail.gmail.com>
	<l45t9s$tgs$1@ger.gmane.org>
	<CALGmxELEb1fxkgK=v9v4vjofpChE2c5bN5Mr=qufFxFEQ+7ruw@mail.gmail.com>
	<CABL7CQhJ1S-swgy3jbBJAqYFh2gJtNjh-yEmFT+ONqkfdT6=DQ@mail.gmail.com>
	<CALGmxELbZ1SVJRr=Q52TRzHHrozix8p1UuV=Qajf5RJCyAn-7w@mail.gmail.com>
	<CABL7CQggFwDU+1b-wn++fMuOhjpnDReCiMRq4-OCJVYeNaE-sw@mail.gmail.com>
	<CAH6Pt5qEVEYEyAYrwu=Xp_BkbhNM+p56E+L76BQOLA2xQ-VOvQ@mail.gmail.com>
	<7401257806648702858@unknownmsgid> <5267E836.60708@noaa.gov>
	<CAH6Pt5oWMBxA8qkuuSU3SzuP83f43X9DSaL2xYOuKNME15rY+w@mail.gmail.com>
	<l492ht$r9v$1@ger.gmane.org>
	<CAH6Pt5pVR6RBXRGxFHAaE3yniyzk11-pV0xFXZt9hFcY60naGg@mail.gmail.com>
	<CALGmxEKwYrUopgbuAgVxrgtMq9wUf3J+Gi-ZvUApRfQJpTK3GQ@mail.gmail.com>
	<CALGmxEKW_nHRadO6xKvK8WPeT-OJMR=g3tDHmf=NPUViwrpaTw@mail.gmail.com>
	<CAMMTP+CtNoq3u2nFDU0jpFFeL0RNduqUw_D0h2G1yZqW=0PU9g@mail.gmail.com>
	<CABL7CQjPTpJjS8pOCQLYgTQ-4C3RpJOEhYJayHVn4cmBbrrDDA@mail.gmail.com>
Message-ID: <CALGmxEL5j04UhzEr=2UoCxdVvDhaaVPqDp8buh+ffvwfV3NoVg@mail.gmail.com>

On Wed, Oct 23, 2013 at 1:45 PM, Ralf Gommers <ralf.gommers at gmail.com> wrote:

> I'd be tempted to replace that unreadable link with
> http://sourceforge.net/projects/scipy/files/scipy/ and not care about
> pip/easy_install scanning that page. Binary installers don't work anyway and
> source bundles are on pypi already, so download_url is mainly for human
> consumption.

until we have binary wheels, then that sounds like a fine idea.

-Chris


-- 

Christopher Barker, Ph.D.
Oceanographer

Emergency Response Division
NOAA/NOS/OR&R            (206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115       (206) 526-6317   main reception

Chris.Barker at noaa.gov


From pwang at continuum.io  Wed Oct 23 19:00:45 2013
From: pwang at continuum.io (Peter Wang)
Date: Wed, 23 Oct 2013 18:00:45 -0500
Subject: [Numpy-discussion] Announcing Bokeh 0.2: interactive web plotting
	for Python
Message-ID: <CABGrp+8oesOhiN71Eu5CsTxxWhuZGFQ-NqRVav3OE8PL92Mq+w@mail.gmail.com>

Hi everyone,

I'm excited to announce the v0.2 release of Bokeh, an interactive web
plotting library for Python.  The long-term vision for Bokeh is to
provide rich interactivity, using the full power of Javascript and
Canvas, to Python users who don't need to write any JS or learn
the DOM.

The full blog post announcement is here:
http://continuum.io/blog/bokeh02

The project website (with interactive gallery) is at:
http://bokeh.pydata.org

And the Git repo is:
https://github.com/ContinuumIO/bokeh


Cheers,
Peter
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20131023/891fb52a/attachment.html>

From jason-sage at creativetrax.com  Thu Oct 24 07:35:05 2013
From: jason-sage at creativetrax.com (Jason Grout)
Date: Thu, 24 Oct 2013 06:35:05 -0500
Subject: [Numpy-discussion] Announcing Bokeh 0.2: interactive web
 plotting for Python
In-Reply-To: <CABGrp+8oesOhiN71Eu5CsTxxWhuZGFQ-NqRVav3OE8PL92Mq+w@mail.gmail.com>
References: <CABGrp+8oesOhiN71Eu5CsTxxWhuZGFQ-NqRVav3OE8PL92Mq+w@mail.gmail.com>
Message-ID: <526905E9.40000@creativetrax.com>

On 10/23/13 6:00 PM, Peter Wang wrote:
> Hi everyone,
>
> I'm excited to announce the v0.2 release of Bokeh, an interactive web
> plotting library for Python.  The long-term vision for Bokeh is to
> provide rich interactivity, using the full power of Javascript and
> Canvas, to Python users who don't need to write any JS or learn
> the DOM.
>
> The full blog post announcement is here:
> http://continuum.io/blog/bokeh02
>
> The project website (with interactive gallery) is at:
> http://bokeh.pydata.org
>
> And the Git repo is:
> https://github.com/ContinuumIO/bokeh

This looks really cool.  I was checking out how easy it would be to 
embed in the Sage Cell Server [1].  I briefly looked at the code, and it 
appears that the IPython notebook mode does not use nodejs, redis, 
gevent, etc.?  Is that right?

Thanks,

Jason

[1] https://sagecell.sagemath.org




From jason-sage at creativetrax.com  Thu Oct 24 07:36:53 2013
From: jason-sage at creativetrax.com (Jason Grout)
Date: Thu, 24 Oct 2013 06:36:53 -0500
Subject: [Numpy-discussion] Announcing Bokeh 0.2: interactive web
 plotting for Python
In-Reply-To: <526905E9.40000@creativetrax.com>
References: <CABGrp+8oesOhiN71Eu5CsTxxWhuZGFQ-NqRVav3OE8PL92Mq+w@mail.gmail.com>
	<526905E9.40000@creativetrax.com>
Message-ID: <52690655.1030605@creativetrax.com>

On 10/24/13 6:35 AM, Jason Grout wrote:
> This looks really cool.  I was checking out how easy it would be to
> embed in the Sage Cell Server [1].  I briefly looked at the code, and it
> appears that the IPython notebook mode does not use nodejs, redis,
> gevent, etc.?  Is that right?

Or maybe the better way to phrase it is: what are the absolute minimum 
dependencies if all I want to do is to display in the IPython notebook?

Thanks,

Jason



From jason-sage at creativetrax.com  Thu Oct 24 08:39:44 2013
From: jason-sage at creativetrax.com (Jason Grout)
Date: Thu, 24 Oct 2013 07:39:44 -0500
Subject: [Numpy-discussion] Announcing Bokeh 0.2: interactive web
 plotting for Python
In-Reply-To: <CABGrp+8oesOhiN71Eu5CsTxxWhuZGFQ-NqRVav3OE8PL92Mq+w@mail.gmail.com>
References: <CABGrp+8oesOhiN71Eu5CsTxxWhuZGFQ-NqRVav3OE8PL92Mq+w@mail.gmail.com>
Message-ID: <52691510.80004@creativetrax.com>

On 10/23/13 6:00 PM, Peter Wang wrote:
>
> The project website (with interactive gallery) is at:
> http://bokeh.pydata.org

Just a suggestion: could you put the source below each gallery image, 
like matplotlib does in their gallery?  I see lots of pretty plots, but 
I have to go digging in github or somewhere to see how you made these 
plots.  Since Bokeh is (at least partly) about making beautiful plots 
easy, showing off the source code is half of the story.

Thanks,

Jason

--
Jason Grout



From pwang at continuum.io  Thu Oct 24 10:44:47 2013
From: pwang at continuum.io (Peter Wang)
Date: Thu, 24 Oct 2013 09:44:47 -0500
Subject: [Numpy-discussion] Announcing Bokeh 0.2: interactive web
 plotting for Python
In-Reply-To: <52691510.80004@creativetrax.com>
References: <CABGrp+8oesOhiN71Eu5CsTxxWhuZGFQ-NqRVav3OE8PL92Mq+w@mail.gmail.com>
	<52691510.80004@creativetrax.com>
Message-ID: <CABGrp+90Sgc9UR8Y9sxRLMM6CZQNdi0AXTJqRhJdG5vc-G-KTg@mail.gmail.com>

On Thu, Oct 24, 2013 at 7:39 AM, Jason Grout <jason-sage at creativetrax.com>wrote:

> On 10/23/13 6:00 PM, Peter Wang wrote:
> >
> > The project website (with interactive gallery) is at:
> > http://bokeh.pydata.org
>
> Just a suggestion: could you put the source below each gallery image,
> like matplotlib does in their gallery?  I see lots of pretty plots, but
> I have to go digging in github or somewhere to see how you made these
> plots.  Since Bokeh is (at least partly) about making beautiful plots
> easy, showing off the source code is half of the story.
>

Thanks for the suggestion - we actually did have that at one point, but
experienced some formatting issues and are working on addressing that today.

-Peter
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20131024/3ef2ed36/attachment.html>

From pwang at continuum.io  Thu Oct 24 10:47:42 2013
From: pwang at continuum.io (Peter Wang)
Date: Thu, 24 Oct 2013 09:47:42 -0500
Subject: [Numpy-discussion] Announcing Bokeh 0.2: interactive web
 plotting for Python
In-Reply-To: <52690655.1030605@creativetrax.com>
References: <CABGrp+8oesOhiN71Eu5CsTxxWhuZGFQ-NqRVav3OE8PL92Mq+w@mail.gmail.com>
	<526905E9.40000@creativetrax.com> <52690655.1030605@creativetrax.com>
Message-ID: <CABGrp+_TEkgAW0QYcV0qeAw_cAzkhoQ_AHPpCn-q1B-1SRgAXQ@mail.gmail.com>

On Thu, Oct 24, 2013 at 6:36 AM, Jason Grout <jason-sage at creativetrax.com>wrote:

> On 10/24/13 6:35 AM, Jason Grout wrote:
> > This looks really cool.  I was checking out how easy it would be to
> > embed in the Sage Cell Server [1].  I briefly looked at the code, and it
> > appears that the IPython notebook mode does not use nodejs, redis,
> > gevent, etc.?  Is that right?
>
> Or maybe the better way to phrase it is: what are the absolute minimum
> dependencies if all I want to do is to display in the IPython notebook?
>

You actually should not need nodejs, redis, and gevent if you just want to
embed the full source code of bokeh.js into the IPython notebook itself.
 Also, the data will be baked into the DOM as javascript variables.

You will still have interactivity *within* plots inside a single Notebook,
but they will not drive events back to the server side.  Also, if your data
is large, then the notebook will also get pretty big.  (We will be working
on more efficient encodings in a future release.)

-Peter
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20131024/05b851d1/attachment.html>

From jason-sage at creativetrax.com  Thu Oct 24 11:11:43 2013
From: jason-sage at creativetrax.com (Jason Grout)
Date: Thu, 24 Oct 2013 10:11:43 -0500
Subject: [Numpy-discussion] Announcing Bokeh 0.2: interactive web
 plotting for Python
In-Reply-To: <CABGrp+_TEkgAW0QYcV0qeAw_cAzkhoQ_AHPpCn-q1B-1SRgAXQ@mail.gmail.com>
References: <CABGrp+8oesOhiN71Eu5CsTxxWhuZGFQ-NqRVav3OE8PL92Mq+w@mail.gmail.com>	<526905E9.40000@creativetrax.com>
	<52690655.1030605@creativetrax.com>
	<CABGrp+_TEkgAW0QYcV0qeAw_cAzkhoQ_AHPpCn-q1B-1SRgAXQ@mail.gmail.com>
Message-ID: <526938AF.5020509@creativetrax.com>

On 10/24/13 9:47 AM, Peter Wang wrote:
> You will still have interactivity *within* plots inside a single
> Notebook, but they will not drive events back to the server side.  Also,
> if your data is large, then the notebook will also get pretty big.  (We
> will be working on more efficient encodings in a future release.)
>

It would be really cool if you could hook into the new IPython comm 
infrastructure to push events back to the server in IPython (this is not 
quite merged yet, but probably ready for experimentation like this). 
The comm infrastructure basically opens up a communication channel 
between objects on the server and the browser.  Messages get sent over 
the normal IPython channels.  The server and browser objects just use 
either send() or an on_message() handler.  See 
https://github.com/ipython/ipython/pull/4195

Here's a very simple example of the Comm implementation working with 
matplotlib images in the Sage Cell server (which is built on top of the 
IPython infrastructure):  http://sagecell.sagemath.org/?q=fyjgmk (I'd 
love to see a bokeh version of this sort of thing :).

FYI, here is the javascript code we use for the above example: 
https://github.com/sagemath/sagecell/blob/master/static/compute_server.js#L768 
and the python code is at 
https://github.com/sagemath/sagecell/blob/master/graphics.py#L399

Thanks,

Jason



From pwang at continuum.io  Thu Oct 24 14:42:23 2013
From: pwang at continuum.io (Peter Wang)
Date: Thu, 24 Oct 2013 13:42:23 -0500
Subject: [Numpy-discussion] Announcing Bokeh 0.2: interactive web
 plotting for Python
In-Reply-To: <526938AF.5020509@creativetrax.com>
References: <CABGrp+8oesOhiN71Eu5CsTxxWhuZGFQ-NqRVav3OE8PL92Mq+w@mail.gmail.com>
	<526905E9.40000@creativetrax.com> <52690655.1030605@creativetrax.com>
	<CABGrp+_TEkgAW0QYcV0qeAw_cAzkhoQ_AHPpCn-q1B-1SRgAXQ@mail.gmail.com>
	<526938AF.5020509@creativetrax.com>
Message-ID: <CABGrp+95+cgG7o7ws-qXOruaKUOj-VCV6ZV9ALEHa0SPazzJNg@mail.gmail.com>

On Thu, Oct 24, 2013 at 10:11 AM, Jason Grout
<jason-sage at creativetrax.com>wrote:

> It would be really cool if you could hook into the new IPython comm
>  infrastructure to push events back to the server in IPython (this is not
> quite merged yet, but probably ready for experimentation like this).
> The comm infrastructure basically opens up a communication channel
> between objects on the server and the browser.  Messages get sent over
> the normal IPython channels.  The server and browser objects just use
> either send() or an on_message() handler.  See
> https://github.com/ipython/ipython/pull/4195


Yeah, I think we should definitely look into integrating with this
mechanism for when we are embedded in a Notebook.  However, we always want
the underlying infrastructure to be independent of IPython Notebook,
because we want people to be able to build analytical applications on top
of these components.


> Here's a very simple example of the Comm implementation working with
> matplotlib images in the Sage Cell server (which is built on top of the
> IPython infrastructure):  http://sagecell.sagemath.org/?q=fyjgmk (I'd
> love to see a bokeh version of this sort of thing :).
>

This is interesting, and introducing widgets is already on the roadmap,
tentatively v0.4.  When running against a plot server, Bokeh plots already
push selections back to the server side.  (That's how the linked brushing
in e.g. this example works: https://www.wakari.io/sharing/bundle/pwang/cars)

Our immediate short-term priorities for 0.3 are improving the layout
mechanism, incorporating large data processing into the plot server, and
investigating basic interop with Matplotlib objects.


-Peter
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20131024/0aca583a/attachment.html>

From jason-sage at creativetrax.com  Thu Oct 24 15:55:03 2013
From: jason-sage at creativetrax.com (Jason Grout)
Date: Thu, 24 Oct 2013 14:55:03 -0500
Subject: [Numpy-discussion] Announcing Bokeh 0.2: interactive web
 plotting for Python
In-Reply-To: <CABGrp+95+cgG7o7ws-qXOruaKUOj-VCV6ZV9ALEHa0SPazzJNg@mail.gmail.com>
References: <CABGrp+8oesOhiN71Eu5CsTxxWhuZGFQ-NqRVav3OE8PL92Mq+w@mail.gmail.com>	<526905E9.40000@creativetrax.com>
	<52690655.1030605@creativetrax.com>	<CABGrp+_TEkgAW0QYcV0qeAw_cAzkhoQ_AHPpCn-q1B-1SRgAXQ@mail.gmail.com>	<526938AF.5020509@creativetrax.com>
	<CABGrp+95+cgG7o7ws-qXOruaKUOj-VCV6ZV9ALEHa0SPazzJNg@mail.gmail.com>
Message-ID: <52697B17.6060106@creativetrax.com>

On 10/24/13 1:42 PM, Peter Wang wrote:
> On Thu, Oct 24, 2013 at 10:11 AM, Jason Grout
> <jason-sage at creativetrax.com <mailto:jason-sage at creativetrax.com>> wrote:
>
>     It would be really cool if you could hook into the new IPython comm
>     infrastructure to push events back to the server in IPython (this is not
>     quite merged yet, but probably ready for experimentation like this).
>     The comm infrastructure basically opens up a communication channel
>     between objects on the server and the browser.  Messages get sent over
>     the normal IPython channels.  The server and browser objects just use
>     either send() or an on_message() handler.  See
>     https://github.com/ipython/ipython/pull/4195
>
>
> Yeah, I think we should definitely look into integrating with this
> mechanism for when we are embedded in a Notebook.  However, we always
> want the underlying infrastructure to be independent of IPython
> Notebook, because we want people to be able to build analytical
> applications on top of these components.

That makes a lot of sense.  And looking at the code, it looks like you 
are cleanly separating out the session objects controlling communication 
from the plot machinery.  That will hopefully make it much easier to 
have different transports for the communication.


>
>     Here's a very simple example of the Comm implementation working with
>     matplotlib images in the Sage Cell server (which is built on top of the
>     IPython infrastructure): http://sagecell.sagemath.org/?q=fyjgmk (I'd
>     love to see a bokeh version of this sort of thing :).
>
>
> This is interesting, and introducing widgets is already on the roadmap,
> tentatively v0.4.  When running against a plot server, Bokeh plots
> already push selections back to the server side.  (That's how the linked
> brushing in e.g. this example works:
> https://www.wakari.io/sharing/bundle/pwang/cars)
>
> Our immediate short-term priorities for 0.3 are improving the layout
> mechanism, incorporating large data processing into the plot server, and
> investigating basic interop with Matplotlib objects.
>
>

Great to hear.


Jason



From dr.andrew.straw at gmail.com  Fri Oct 25 09:07:12 2013
From: dr.andrew.straw at gmail.com (Andrew Straw)
Date: Fri, 25 Oct 2013 16:07:12 +0300
Subject: [Numpy-discussion] NumPy 1.8.0rc2 release
In-Reply-To: <CACfw6g4ztC2vJUsdOTFJJJqnO4u4TenwhPj-bHLzcS_y8WqG_g@mail.gmail.com>
References: <CACfw6g4ztC2vJUsdOTFJJJqnO4u4TenwhPj-bHLzcS_y8WqG_g@mail.gmail.com>
Message-ID: <CACfw6g79PcXFa71HxOFVonqLVAK_1ip5vH_2oXK3-ZWEUuQfOw@mail.gmail.com>

Hi,

I found an unexpected difference from numpy 1.7.1 and 1.8.0rc2 with Python
3.3.2 on Ubuntu 12.04 (amd64). Here is the test program:

import numpy as np
print(np.__version__)
K = np.array([[ 0.        ,  0.        ,  0.        ,  0.        ],
              [-0.33333333,  0.        ,  0.        ,  0.        ],
              [ 0.33333333, -0.33333333,  0.        ,  0.        ],
              [ 0.33333333, -0.33333333,  0.33333333,  0.        ]])
w, V = np.linalg.eigh(K)
print('w')
print(w)

with numpy 1.7.1:

1.7.1
w
[-0.33333333 -0.33333333 -0.33333333  0.99999999]

with numpy 1.8.0rc2:

1.8.0rc2
w
[ 0.  0.  0.  0.]

Apologies if this is my mistake!
-Andrew
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20131025/52dcbe9e/attachment.html>

From argriffi at ncsu.edu  Fri Oct 25 09:32:43 2013
From: argriffi at ncsu.edu (alex)
Date: Fri, 25 Oct 2013 09:32:43 -0400
Subject: [Numpy-discussion] NumPy 1.8.0rc2 release
In-Reply-To: <CACfw6g79PcXFa71HxOFVonqLVAK_1ip5vH_2oXK3-ZWEUuQfOw@mail.gmail.com>
References: <CACfw6g4ztC2vJUsdOTFJJJqnO4u4TenwhPj-bHLzcS_y8WqG_g@mail.gmail.com>
	<CACfw6g79PcXFa71HxOFVonqLVAK_1ip5vH_2oXK3-ZWEUuQfOw@mail.gmail.com>
Message-ID: <CAE5GFcJ3yjVtoyLYS6yO_DfaH_wOj0JLFgmp0Xx9Rer5sXcpdw@mail.gmail.com>

On Fri, Oct 25, 2013 at 9:07 AM, Andrew Straw <dr.andrew.straw at gmail.com>
wrote:
>
> Hi,
>
> I found an unexpected difference from numpy 1.7.1 and 1.8.0rc2 with
Python 3.3.2 on Ubuntu 12.04 (amd64). Here is the test program:
>
> import numpy as np
> print(np.__version__)
> K = np.array([[ 0.        ,  0.        ,  0.        ,  0.        ],
>               [-0.33333333,  0.        ,  0.        ,  0.        ],
>               [ 0.33333333, -0.33333333,  0.        ,  0.        ],
>               [ 0.33333333, -0.33333333,  0.33333333,  0.        ]])
> w, V = np.linalg.eigh(K)
> print('w')
> print(w)
>
> with numpy 1.7.1:
>
> 1.7.1
> w
> [-0.33333333 -0.33333333 -0.33333333  0.99999999]
>
> with numpy 1.8.0rc2:
>
> 1.8.0rc2
> w
> [ 0.  0.  0.  0.]
>
> Apologies if this is my mistake!
> -Andrew
>
>
> _______________________________________________
> NumPy-Discussion mailing list
> NumPy-Discussion at scipy.org
> http://mail.scipy.org/mailman/listinfo/numpy-discussion
>

My first reaction was to think that eigh would not have guarantees because
the matrix is not hermitian, but now I see that eigh deliberately uses the
upper vs. lower part according to the UPLO arg which is documented as
defaulting to 'L', so I see why this difference between numpy versions is
an unexpected problem.

Alex
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20131025/a35724b7/attachment.html>

From matthew.brett at gmail.com  Fri Oct 25 10:54:22 2013
From: matthew.brett at gmail.com (Matthew Brett)
Date: Fri, 25 Oct 2013 07:54:22 -0700
Subject: [Numpy-discussion] ANN: SciPy 0.13.0 release
In-Reply-To: <CABL7CQi5B0qdbWMQBrVwRsZ33BGsV2ATaUCEAbA5rjNjm=iyoQ@mail.gmail.com>
References: <CABL7CQi5B0qdbWMQBrVwRsZ33BGsV2ATaUCEAbA5rjNjm=iyoQ@mail.gmail.com>
Message-ID: <CAH6Pt5ogO_YDo3+ahJ_we+nD87=a6mpkhf-=ye-=21hL0Gq2yA@mail.gmail.com>

Hi,

On Sat, Oct 19, 2013 at 2:40 PM, Ralf Gommers <ralf.gommers at gmail.com> wrote:
> On behalf of the SciPy development team I'm pleased to announce the
> availability of SciPy 0.13.0. This release contains some interesting new
> features (see highlights below) and half a year's worth of maintenance work.
> 65 people contributed to this release.
>
> Some of the highlights are:
>
>   - support for fancy indexing and boolean comparisons with sparse matrices
>   - interpolative decompositions and matrix functions in the linalg module
>   - two new trust-region solvers for unconstrained minimization
>
> This release requires Python 2.6, 2.7 or 3.1-3.3 and NumPy 1.5.1 or greater.
> Support for Python 2.4 and 2.5 has been dropped as of this release.
>
> Sources and binaries can be found at
> http://sourceforge.net/projects/scipy/files/scipy/0.13.0/, release notes are
> copied below.

Sorry to be slow to the party, but I just wanted to point out:

git shortlog -ns v0.12.0..v0.13.0

   389  Pauli Virtanen
   225  Ralf Gommers
   105  alex
   104  Blake Griffith
   101  Warren Weckesser
...

So - to y'all, but in particular to Pauli and Ralf - thank you for
this all this great, patient, organized work.

A deep bow,

Matthew


From charlesr.harris at gmail.com  Fri Oct 25 12:45:34 2013
From: charlesr.harris at gmail.com (Charles R Harris)
Date: Fri, 25 Oct 2013 10:45:34 -0600
Subject: [Numpy-discussion] NumPy 1.8.0rc2 release
In-Reply-To: <CACfw6g79PcXFa71HxOFVonqLVAK_1ip5vH_2oXK3-ZWEUuQfOw@mail.gmail.com>
References: <CACfw6g4ztC2vJUsdOTFJJJqnO4u4TenwhPj-bHLzcS_y8WqG_g@mail.gmail.com>
	<CACfw6g79PcXFa71HxOFVonqLVAK_1ip5vH_2oXK3-ZWEUuQfOw@mail.gmail.com>
Message-ID: <CAB6mnxJ_7WWFPuc9y1heNR4U60DQ5ux0VLoNOMK3DUW3qr_ofA@mail.gmail.com>

On Fri, Oct 25, 2013 at 7:07 AM, Andrew Straw <dr.andrew.straw at gmail.com>wrote:

> Hi,
>
> I found an unexpected difference from numpy 1.7.1 and 1.8.0rc2 with Python
> 3.3.2 on Ubuntu 12.04 (amd64). Here is the test program:
>
> import numpy as np
> print(np.__version__)
> K = np.array([[ 0.        ,  0.        ,  0.        ,  0.        ],
>               [-0.33333333,  0.        ,  0.        ,  0.        ],
>               [ 0.33333333, -0.33333333,  0.        ,  0.        ],
>               [ 0.33333333, -0.33333333,  0.33333333,  0.        ]])
> w, V = np.linalg.eigh(K)
> print('w')
> print(w)
>
> with numpy 1.7.1:
>
> 1.7.1
> w
> [-0.33333333 -0.33333333 -0.33333333  0.99999999]
>
> with numpy 1.8.0rc2:
>
> 1.8.0rc2
> w
> [ 0.  0.  0.  0.]
>

This appears to have gotten fixed since rc2:

In [1]: K = np.array([[ 0.        ,  0.        ,  0.        ,  0.        ],
              [-0.33333333,  0.        ,  0.        ,  0.        ],
              [ 0.33333333, -0.33333333,  0.        ,  0.        ],
              [ 0.33333333, -0.33333333,  0.33333333,  0.        ]])

In [2]: eigh(K)
Out[2]:
(array([-0.33333333, -0.33333333, -0.33333333,  0.99999999]),
 array([[-0.78251031,  0.37104402,  0.00198815,  0.5       ],
       [-0.29974269,  0.03728557,  0.81164285, -0.5       ],
       [ 0.54246491,  0.46764373,  0.48686874,  0.5       ],
       [-0.05969728, -0.80140218,  0.32278596,  0.5       ]]))

In [3]: np.__version__
Out[3]: '1.8.0.dev-ced0a94'

Could you try the current 1.8.x version on github and see if the problem
persists?

Chuck
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20131025/ea052e8e/attachment.html>

From njs at pobox.com  Fri Oct 25 12:52:59 2013
From: njs at pobox.com (Nathaniel Smith)
Date: Fri, 25 Oct 2013 17:52:59 +0100
Subject: [Numpy-discussion] NumPy 1.8.0rc2 release
In-Reply-To: <CAB6mnxJ_7WWFPuc9y1heNR4U60DQ5ux0VLoNOMK3DUW3qr_ofA@mail.gmail.com>
References: <CACfw6g4ztC2vJUsdOTFJJJqnO4u4TenwhPj-bHLzcS_y8WqG_g@mail.gmail.com>
	<CACfw6g79PcXFa71HxOFVonqLVAK_1ip5vH_2oXK3-ZWEUuQfOw@mail.gmail.com>
	<CAB6mnxJ_7WWFPuc9y1heNR4U60DQ5ux0VLoNOMK3DUW3qr_ofA@mail.gmail.com>
Message-ID: <CAPJVwBnkNfAFP1=cEL02RZkODNYy+2LMKpXc+DXuG2Ftc4gHJg@mail.gmail.com>

It's py3 only, see the discussion in #3977.
On 25 Oct 2013 17:45, "Charles R Harris" <charlesr.harris at gmail.com> wrote:

>
>
>
> On Fri, Oct 25, 2013 at 7:07 AM, Andrew Straw <dr.andrew.straw at gmail.com>wrote:
>
>> Hi,
>>
>> I found an unexpected difference from numpy 1.7.1 and 1.8.0rc2 with
>> Python 3.3.2 on Ubuntu 12.04 (amd64). Here is the test program:
>>
>> import numpy as np
>> print(np.__version__)
>> K = np.array([[ 0.        ,  0.        ,  0.        ,  0.        ],
>>               [-0.33333333,  0.        ,  0.        ,  0.        ],
>>               [ 0.33333333, -0.33333333,  0.        ,  0.        ],
>>               [ 0.33333333, -0.33333333,  0.33333333,  0.        ]])
>> w, V = np.linalg.eigh(K)
>> print('w')
>> print(w)
>>
>> with numpy 1.7.1:
>>
>> 1.7.1
>> w
>> [-0.33333333 -0.33333333 -0.33333333  0.99999999]
>>
>> with numpy 1.8.0rc2:
>>
>> 1.8.0rc2
>> w
>> [ 0.  0.  0.  0.]
>>
>
> This appears to have gotten fixed since rc2:
>
> In [1]: K = np.array([[ 0.        ,  0.        ,  0.        ,  0.        ],
>               [-0.33333333,  0.        ,  0.        ,  0.        ],
>               [ 0.33333333, -0.33333333,  0.        ,  0.        ],
>               [ 0.33333333, -0.33333333,  0.33333333,  0.        ]])
>
> In [2]: eigh(K)
> Out[2]:
> (array([-0.33333333, -0.33333333, -0.33333333,  0.99999999]),
>  array([[-0.78251031,  0.37104402,  0.00198815,  0.5       ],
>        [-0.29974269,  0.03728557,  0.81164285, -0.5       ],
>        [ 0.54246491,  0.46764373,  0.48686874,  0.5       ],
>        [-0.05969728, -0.80140218,  0.32278596,  0.5       ]]))
>
> In [3]: np.__version__
> Out[3]: '1.8.0.dev-ced0a94'
>
> Could you try the current 1.8.x version on github and see if the problem
> persists?
>
> Chuck
>
>
> _______________________________________________
> NumPy-Discussion mailing list
> NumPy-Discussion at scipy.org
> http://mail.scipy.org/mailman/listinfo/numpy-discussion
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20131025/eed1615d/attachment.html>

From dr.andrew.straw at gmail.com  Fri Oct 25 14:54:40 2013
From: dr.andrew.straw at gmail.com (Andrew Straw)
Date: Fri, 25 Oct 2013 21:54:40 +0300
Subject: [Numpy-discussion] NumPy 1.8.0rc2 release
In-Reply-To: <CAPJVwBnkNfAFP1=cEL02RZkODNYy+2LMKpXc+DXuG2Ftc4gHJg@mail.gmail.com>
References: <CACfw6g4ztC2vJUsdOTFJJJqnO4u4TenwhPj-bHLzcS_y8WqG_g@mail.gmail.com>
	<CACfw6g79PcXFa71HxOFVonqLVAK_1ip5vH_2oXK3-ZWEUuQfOw@mail.gmail.com>
	<CAB6mnxJ_7WWFPuc9y1heNR4U60DQ5ux0VLoNOMK3DUW3qr_ofA@mail.gmail.com>
	<CAPJVwBnkNfAFP1=cEL02RZkODNYy+2LMKpXc+DXuG2Ftc4gHJg@mail.gmail.com>
Message-ID: <CACfw6g79VCOU1Lk1nyxW1fgCU8BSWpD74e3s1RuZBW=4PTumnw@mail.gmail.com>

Yeah, sorry for the confusion. I thought my email didn't go through. I
created a bug report on github instead. Now I find the email was indeed
sent. Sorry about that. Looking at the github activity (see
https://github.com/numpy/numpy/issues/3977 ), I think a fix is almost at
hand thanks to Julian Taylor.

Best,
Andrew


On Fri, Oct 25, 2013 at 7:52 PM, Nathaniel Smith <njs at pobox.com> wrote:

> It's py3 only, see the discussion in #3977.
> On 25 Oct 2013 17:45, "Charles R Harris" <charlesr.harris at gmail.com>
> wrote:
>
>>
>>
>>
>> On Fri, Oct 25, 2013 at 7:07 AM, Andrew Straw <dr.andrew.straw at gmail.com>wrote:
>>
>>> Hi,
>>>
>>> I found an unexpected difference from numpy 1.7.1 and 1.8.0rc2 with
>>> Python 3.3.2 on Ubuntu 12.04 (amd64). Here is the test program:
>>>
>>> import numpy as np
>>> print(np.__version__)
>>> K = np.array([[ 0.        ,  0.        ,  0.        ,  0.        ],
>>>               [-0.33333333,  0.        ,  0.        ,  0.        ],
>>>               [ 0.33333333, -0.33333333,  0.        ,  0.        ],
>>>               [ 0.33333333, -0.33333333,  0.33333333,  0.        ]])
>>> w, V = np.linalg.eigh(K)
>>> print('w')
>>> print(w)
>>>
>>> with numpy 1.7.1:
>>>
>>> 1.7.1
>>> w
>>> [-0.33333333 -0.33333333 -0.33333333  0.99999999]
>>>
>>> with numpy 1.8.0rc2:
>>>
>>> 1.8.0rc2
>>> w
>>> [ 0.  0.  0.  0.]
>>>
>>
>> This appears to have gotten fixed since rc2:
>>
>> In [1]: K = np.array([[ 0.        ,  0.        ,  0.        ,  0.
>> ],
>>               [-0.33333333,  0.        ,  0.        ,  0.        ],
>>               [ 0.33333333, -0.33333333,  0.        ,  0.        ],
>>               [ 0.33333333, -0.33333333,  0.33333333,  0.        ]])
>>
>> In [2]: eigh(K)
>> Out[2]:
>> (array([-0.33333333, -0.33333333, -0.33333333,  0.99999999]),
>>  array([[-0.78251031,  0.37104402,  0.00198815,  0.5       ],
>>        [-0.29974269,  0.03728557,  0.81164285, -0.5       ],
>>        [ 0.54246491,  0.46764373,  0.48686874,  0.5       ],
>>        [-0.05969728, -0.80140218,  0.32278596,  0.5       ]]))
>>
>> In [3]: np.__version__
>> Out[3]: '1.8.0.dev-ced0a94'
>>
>> Could you try the current 1.8.x version on github and see if the problem
>> persists?
>>
>> Chuck
>>
>>
>> _______________________________________________
>> NumPy-Discussion mailing list
>> NumPy-Discussion at scipy.org
>> http://mail.scipy.org/mailman/listinfo/numpy-discussion
>>
>>
> _______________________________________________
> NumPy-Discussion mailing list
> NumPy-Discussion at scipy.org
> http://mail.scipy.org/mailman/listinfo/numpy-discussion
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20131025/ec42077d/attachment.html>

From jaime.frio at gmail.com  Fri Oct 25 15:32:36 2013
From: jaime.frio at gmail.com (=?ISO-8859-1?Q?Jaime_Fern=E1ndez_del_R=EDo?=)
Date: Fri, 25 Oct 2013 12:32:36 -0700
Subject: [Numpy-discussion] vectorizing recursive sequences
Message-ID: <CAPOWHWkezdC0w94jzkzuO5y2A0QFxgdgk+8NJUy0LLAbSrRH+Q@mail.gmail.com>

I recently came up with a way of vectorizing some recursive sequence
calculations. While it works, I am afraid it is relying on implementation
details potentially subject to change. The basic idea is illustrated by
this function, calculating the first n items of the Fibonacci sequence:

def fibonacci(n):

    fib = np.empty(n, dtype=np.intp)

    fib[:2] = 1

    np.add(fib[:-2], fib[1:-1], out=fib[2:])

    return fib


>>> fibonacci(10)

array([ 1,  1,  2,  3,  5,  8, 13, 21, 34, 55], dtype=int64)


I believe that the biggest issue that could break this is if the ufunc
decided to buffer the arrays, as this is relying on the inputs and outputs
of np.add sharing the same memory.


You can use the same idea to do more complicated things, for instance to
calculate the items of the sequence:


f[0] = a[0]

f[n] = a[n] + x * f[n-1]


from numpy.lib.stride_tricks import as_strided

from numpy.core.umath_tests import inner1d


def f(a, x):

    out = np.array(a, copy=True, dtype=np.double)

    n = len(a)

    out_view = as_strided(out, shape=(n-1, 2), strides=out.strides*2)

    inner1d(out_view, [x, 1], out=out[1:])

    return out


>>> f(np.arange(10), 0.1)

array([ 0.        ,  1.        ,  2.1       ,  3.21      ,  4.321     ,

        5.4321    ,  6.54321   ,  7.654321  ,  8.7654321 ,  9.87654321])

Again, I think  buffering is the clearest danger of doing something like
this, as the first input and output must share the same memory for this to
work. That this is a real concern is easy to see: since `inner1d` only has
loops registered for long ints and double floats:

>>> inner1d.types
['ll->l', 'dd->d']

the above function `f` doesn't work if the `out` array is created, e.g. as
np.float32, since there will be buffering happening because of the type
casting.

So I have two questions:

1. Is there some other reason, aside from buffering, that could go wrong,
or change in a future release?
2. As far as buffering is concerned, I thought of calling `np.setbuffer(1)`
before any of these functions, but it complains and requests that the value
be a multiple of 16. Is there any other way to ensure that the data is
fetched from an updated array in every internal iteration?

Thanks!

Jaime

-- 
(\__/)
( O.o)
( > <) Este es Conejo. Copia a Conejo en tu firma y ay?dale en sus planes
de dominaci?n mundial.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20131025/8fbfb85d/attachment.html>

From pierre.haessig at crans.org  Fri Oct 25 16:43:38 2013
From: pierre.haessig at crans.org (Pierre Haessig)
Date: Fri, 25 Oct 2013 22:43:38 +0200
Subject: [Numpy-discussion] vectorizing recursive sequences
In-Reply-To: <CAPOWHWkezdC0w94jzkzuO5y2A0QFxgdgk+8NJUy0LLAbSrRH+Q@mail.gmail.com>
References: <CAPOWHWkezdC0w94jzkzuO5y2A0QFxgdgk+8NJUy0LLAbSrRH+Q@mail.gmail.com>
Message-ID: <90b66073-de8c-488e-acfb-bcde5934b062@email.android.com>


"Jaime Fern?ndez del R?o" <jaime.frio at gmail.com> a ?crit?:
>I recently came up with a way of vectorizing some recursive sequence
>calculations. While it works, I am afraid it is relying on
>implementation
>details potentially subject to change. The basic idea is illustrated by
>this function, calculating the first n items of the Fibonacci sequence:
>
>def fibonacci(n):
>
>    fib = np.empty(n, dtype=np.intp)
>
>    fib[:2] = 1
>
>    np.add(fib[:-2], fib[1:-1], out=fib[2:])
>
>    return fib
>
>
>>>> fibonacci(10)
>
>array([ 1,  1,  2,  3,  5,  8, 13, 21, 34, 55], dtype=int64)
>
>
>I believe that the biggest issue that could break this is if the ufunc
>decided to buffer the arrays, as this is relying on the inputs and
>outputs
>of np.add sharing the same memory.
>
>
>You can use the same idea to do more complicated things, for instance
>to
>calculate the items of the sequence:
>
>
>f[0] = a[0]
>
>f[n] = a[n] + x * f[n-1]
>
>
>from numpy.lib.stride_tricks import as_strided
>
>from numpy.core.umath_tests import inner1d
>
>
>def f(a, x):
>
>    out = np.array(a, copy=True, dtype=np.double)
>
>    n = len(a)
>
>    out_view = as_strided(out, shape=(n-1, 2), strides=out.strides*2)
>
>    inner1d(out_view, [x, 1], out=out[1:])
>
>    return out
>
>
>>>> f(np.arange(10), 0.1)
>
>array([ 0.        ,  1.        ,  2.1       ,  3.21      ,  4.321     ,
>
>       5.4321    ,  6.54321   ,  7.654321  ,  8.7654321 ,  9.87654321])
>
>Again, I think  buffering is the clearest danger of doing something
>like
>this, as the first input and output must share the same memory for this
>to
>work. That this is a real concern is easy to see: since `inner1d` only
>has
>loops registered for long ints and double floats:
>
>>>> inner1d.types
>['ll->l', 'dd->d']
>
>the above function `f` doesn't work if the `out` array is created, e.g.
>as
>np.float32, since there will be buffering happening because of the type
>casting.
>
>So I have two questions:
>
>1. Is there some other reason, aside from buffering, that could go
>wrong,
>or change in a future release?
>2. As far as buffering is concerned, I thought of calling
>`np.setbuffer(1)`
>before any of these functions, but it complains and requests that the
>value
>be a multiple of 16. Is there any other way to ensure that the data is
>fetched from an updated array in every internal iteration?
>
>Thanks!
>
>Jaime
>
>-- 
>(\__/)
>( O.o)
>( > <) Este es Conejo. Copia a Conejo en tu firma y ay?dale en sus
>planes
>de dominaci?n mundial.
>
>
>------------------------------------------------------------------------
>
>_______________________________________________
>NumPy-Discussion mailing list
>NumPy-Discussion at scipy.org
>http://mail.scipy.org/mailman/listinfo/numpy-discussion

Hi,

If you're recursive computation is always linear, like Fibonacci, your usecase falls in the scope of scipy.signal.lfilter which is compiled code based.

For your last example, I think parameters would be : numerator b = [1] and denominator a=[1 -x]
(can't check the code on my cell phone sorry)

best,
Pierre
-- 
Pierre Haessig
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20131025/854340ae/attachment.html>

From chris.barker at noaa.gov  Fri Oct 25 20:08:44 2013
From: chris.barker at noaa.gov (Chris Barker)
Date: Fri, 25 Oct 2013 17:08:44 -0700
Subject: [Numpy-discussion] [Pythonmac-SIG] Building extension modules
	with Xcode 5
In-Reply-To: <DBABBCB4-8936-44E8-90D3-7E099043947C@acm.org>
References: <1382658638.14083.YahooMailNeo@web184703.mail.ne1.yahoo.com>
	<5269C5E5.9020204@codebykevin.com>
	<1382665677.4566.YahooMailNeo@web184704.mail.ne1.yahoo.com>
	<nad-81273C.19295024102013@news.gmane.org>
	<CALGmxEL=nyuJ-svYPRED604Dtmehw6oR=w4UDV-sT79oyH_yVQ@mail.gmail.com>
	<DBABBCB4-8936-44E8-90D3-7E099043947C@acm.org>
Message-ID: <CALGmxE+hcLO8Dr+XsfX1EMb1ybrMzSPB8UvyU5HhHdODkwAdHQ@mail.gmail.com>

Ned,

I think this fell off the list (I really don't like when reply is not set
to the list...)

On Fri, Oct 25, 2013 at 4:33 PM, Ned Deily <nad at acm.org> wrote:

>
> On Oct 25, 2013, at 15:17 , Chris Barker <chris.barker at noaa.gov> wrote:
>
> > On Thu, Oct 24, 2013 at 7:29 PM, Ned Deily <nad at acm.org> wrote:
> > I'm really confused here.   Using the current 2.7.5 64-bit installer
> (from
> > May) on 10.8.5 with Xcode 5.0.1 works just fine for me.
> >
> > I'm not on 10.8 or XCode5, but does it make a difference whether you
> installed XCode 5 after Xcode 4? i.e. someone may have gcc4.2 around form
> XCode 4, but not if they installed XCode 5 on a raw machine.
>
> Not that I know of.  gcc-4.2 is a red herring as far as I can tell.
>  Distutils has known how to deal with a missing compiler for a couple of
> releases now.


Good to know -- I know I really need to move forward -- I wonder if there
is an XCode5 for 10.6? pProbably not -- but for some screwy reason Apple
does not provide a free XCode4 for 10.6, either (though you can hack the
installer for 10.7 ...


> However, I did just realize that I had forgotten about a problem when
> using the current 32-bit-only installers with Xcode 5.  I've just posted
> about that; it's described now in http://bugs.python.org/issue19400
>
> Good to fix that, but does anyone need it anymore?

Though I use it when I need to force 32 bit, even on a 64 bit system (I'm
working with some not-64-bit safe C++ code...)

-Chris

-- 

Christopher Barker, Ph.D.
Oceanographer

Emergency Response Division
NOAA/NOS/OR&R            (206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115       (206) 526-6317   main reception

Chris.Barker at noaa.gov
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20131025/3844de66/attachment.html>

From chris.barker at noaa.gov  Fri Oct 25 20:19:02 2013
From: chris.barker at noaa.gov (Chris Barker - NOAA Federal)
Date: Fri, 25 Oct 2013 17:19:02 -0700
Subject: [Numpy-discussion] [Pythonmac-SIG] Building extension modules
	with Xcode 5
In-Reply-To: <CALGmxE+hcLO8Dr+XsfX1EMb1ybrMzSPB8UvyU5HhHdODkwAdHQ@mail.gmail.com>
References: <1382658638.14083.YahooMailNeo@web184703.mail.ne1.yahoo.com>
	<5269C5E5.9020204@codebykevin.com>
	<1382665677.4566.YahooMailNeo@web184704.mail.ne1.yahoo.com>
	<nad-81273C.19295024102013@news.gmane.org>
	<CALGmxEL=nyuJ-svYPRED604Dtmehw6oR=w4UDV-sT79oyH_yVQ@mail.gmail.com>
	<DBABBCB4-8936-44E8-90D3-7E099043947C@acm.org>
	<CALGmxE+hcLO8Dr+XsfX1EMb1ybrMzSPB8UvyU5HhHdODkwAdHQ@mail.gmail.com>
Message-ID: <-4560032494472153313@unknownmsgid>

Oops!

Wrong list--darn auto complete!

Sorry about that,

Chris

On Oct 25, 2013, at 5:08 PM, Chris Barker <chris.barker at noaa.gov> wrote:

Ned,

I think this fell off the list (I really don't like when reply is not set
to the list...)

On Fri, Oct 25, 2013 at 4:33 PM, Ned Deily <nad at acm.org> wrote:

>
> On Oct 25, 2013, at 15:17 , Chris Barker <chris.barker at noaa.gov> wrote:
>
> > On Thu, Oct 24, 2013 at 7:29 PM, Ned Deily <nad at acm.org> wrote:
> > I'm really confused here.   Using the current 2.7.5 64-bit installer
> (from
> > May) on 10.8.5 with Xcode 5.0.1 works just fine for me.
> >
> > I'm not on 10.8 or XCode5, but does it make a difference whether you
> installed XCode 5 after Xcode 4? i.e. someone may have gcc4.2 around form
> XCode 4, but not if they installed XCode 5 on a raw machine.
>
> Not that I know of.  gcc-4.2 is a red herring as far as I can tell.
>  Distutils has known how to deal with a missing compiler for a couple of
> releases now.


Good to know -- I know I really need to move forward -- I wonder if there
is an XCode5 for 10.6? pProbably not -- but for some screwy reason Apple
does not provide a free XCode4 for 10.6, either (though you can hack the
installer for 10.7 ...


> However, I did just realize that I had forgotten about a problem when
> using the current 32-bit-only installers with Xcode 5.  I've just posted
> about that; it's described now in http://bugs.python.org/issue19400
>
> Good to fix that, but does anyone need it anymore?

Though I use it when I need to force 32 bit, even on a 64 bit system (I'm
working with some not-64-bit safe C++ code...)

-Chris

-- 

Christopher Barker, Ph.D.
Oceanographer

Emergency Response Division
NOAA/NOS/OR&R            (206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115       (206) 526-6317   main reception

Chris.Barker at noaa.gov
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20131025/174da7e3/attachment.html>

From mrbago at gmail.com  Sun Oct 27 00:33:55 2013
From: mrbago at gmail.com (Bago)
Date: Sat, 26 Oct 2013 21:33:55 -0700
Subject: [Numpy-discussion] vectorizing recursive sequences
In-Reply-To: <CAPOWHWkezdC0w94jzkzuO5y2A0QFxgdgk+8NJUy0LLAbSrRH+Q@mail.gmail.com>
References: <CAPOWHWkezdC0w94jzkzuO5y2A0QFxgdgk+8NJUy0LLAbSrRH+Q@mail.gmail.com>
Message-ID: <CAHHh6CZXde2TQupaw_CZ-f_wJbxmJBsa4-hUqRb29==LAHekNw@mail.gmail.com>

This behavior seems to depend on the order in which elements of the arrays
are processes. That seems like a dangerous thing to rely on, the main
reason I can thing of that someone would want to change the loop order is
to implement parallel ufuncs.

Bago



On Fri, Oct 25, 2013 at 12:32 PM, Jaime Fern?ndez del R?o <
jaime.frio at gmail.com> wrote:

> I recently came up with a way of vectorizing some recursive sequence
> calculations. While it works, I am afraid it is relying on implementation
> details potentially subject to change. The basic idea is illustrated by
> this function, calculating the first n items of the Fibonacci sequence:
>
> def fibonacci(n):
>
>     fib = np.empty(n, dtype=np.intp)
>
>     fib[:2] = 1
>
>     np.add(fib[:-2], fib[1:-1], out=fib[2:])
>
>     return fib
>
>
> >>> fibonacci(10)
>
> array([ 1,  1,  2,  3,  5,  8, 13, 21, 34, 55], dtype=int64)
>
>
> I believe that the biggest issue that could break this is if the ufunc
> decided to buffer the arrays, as this is relying on the inputs and outputs
> of np.add sharing the same memory.
>
>
> You can use the same idea to do more complicated things, for instance to
> calculate the items of the sequence:
>
>
> f[0] = a[0]
>
> f[n] = a[n] + x * f[n-1]
>
>
> from numpy.lib.stride_tricks import as_strided
>
> from numpy.core.umath_tests import inner1d
>
>
> def f(a, x):
>
>     out = np.array(a, copy=True, dtype=np.double)
>
>     n = len(a)
>
>     out_view = as_strided(out, shape=(n-1, 2), strides=out.strides*2)
>
>     inner1d(out_view, [x, 1], out=out[1:])
>
>     return out
>
>
> >>> f(np.arange(10), 0.1)
>
> array([ 0.        ,  1.        ,  2.1       ,  3.21      ,  4.321     ,
>
>         5.4321    ,  6.54321   ,  7.654321  ,  8.7654321 ,  9.87654321])
>
> Again, I think  buffering is the clearest danger of doing something like
> this, as the first input and output must share the same memory for this to
> work. That this is a real concern is easy to see: since `inner1d` only has
> loops registered for long ints and double floats:
>
> >>> inner1d.types
> ['ll->l', 'dd->d']
>
> the above function `f` doesn't work if the `out` array is created, e.g. as
> np.float32, since there will be buffering happening because of the type
> casting.
>
> So I have two questions:
>
> 1. Is there some other reason, aside from buffering, that could go wrong,
> or change in a future release?
> 2. As far as buffering is concerned, I thought of calling
> `np.setbuffer(1)` before any of these functions, but it complains and
> requests that the value be a multiple of 16. Is there any other way to
> ensure that the data is fetched from an updated array in every internal
> iteration?
>
> Thanks!
>
> Jaime
>
> --
> (\__/)
> ( O.o)
> ( > <) Este es Conejo. Copia a Conejo en tu firma y ay?dale en sus planes
> de dominaci?n mundial.
>
> _______________________________________________
> NumPy-Discussion mailing list
> NumPy-Discussion at scipy.org
> http://mail.scipy.org/mailman/listinfo/numpy-discussion
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20131026/89552b6f/attachment.html>

From mistersheik at gmail.com  Sun Oct 27 05:01:47 2013
From: mistersheik at gmail.com (Neil Girdhar)
Date: Sun, 27 Oct 2013 05:01:47 -0400
Subject: [Numpy-discussion] A context manager for print options
Message-ID: <CAA68w_my+343iXWROEtPxYYvE9SdbKEtj68a=FasiLxUNV+XKg@mail.gmail.com>

Why not replace get_printoptions/set_printoptions with a context manager
accessed using numpy.printoptions in the same way that numpy.errstate
exposes a context manager to seterr/geterr?  This would make the set method
redundant.

Also, the context manager returned by numpy.errstate, numpy.printoptions,
etc. could expose the dictionary directly.  This would make the get methods
redundant.

Best,

Neil
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20131027/cc309fb7/attachment.html>

From njs at pobox.com  Sun Oct 27 06:48:35 2013
From: njs at pobox.com (Nathaniel Smith)
Date: Sun, 27 Oct 2013 10:48:35 +0000
Subject: [Numpy-discussion] A context manager for print options
In-Reply-To: <CAA68w_my+343iXWROEtPxYYvE9SdbKEtj68a=FasiLxUNV+XKg@mail.gmail.com>
References: <CAA68w_my+343iXWROEtPxYYvE9SdbKEtj68a=FasiLxUNV+XKg@mail.gmail.com>
Message-ID: <CAPJVwBk1Wi9iXe2_22+qTL5mMsn-+6aa5B1=QRFtjYjjDOu+CQ@mail.gmail.com>

On 27 Oct 2013 09:02, "Neil Girdhar" <mistersheik at gmail.com> wrote:
>
> Why not replace get_printoptions/set_printoptions with a context manager
accessed using numpy.printoptions in the same way that numpy.errstate
exposes a context manager to seterr/geterr?  This would make the set method
redundant.
>
> Also, the context manager returned by numpy.errstate, numpy.printoptions,
etc. could expose the dictionary directly.  This would make the get methods
redundant.

I think the only reason is that no one's written up a patch and submitted a
PR. Maybe you'll be the one? ;-)

-n
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20131027/93b8f480/attachment.html>

From timmichelsen at gmx-topmail.de  Sun Oct 27 08:42:39 2013
From: timmichelsen at gmx-topmail.de (Tim Michelsen)
Date: Sun, 27 Oct 2013 13:42:39 +0100
Subject: [Numpy-discussion] diferences between frompyfunc & vectorize
Message-ID: <l4j1nm$khv$1@ger.gmane.org>

Hello,
what are the differences between the two transformation functions
frompyfunc & vectorize

* http://docs.scipy.org/doc/numpy/reference/generated/numpy.vectorize.html
* http://docs.scipy.org/doc/numpy/reference/generated/numpy.frompyfunc.html

I cannot see the which one I shall prefer for what purpose.

Thanks in advance,
Timmie



From freddie at witherden.org  Sun Oct 27 14:28:11 2013
From: freddie at witherden.org (Freddie Witherden)
Date: Sun, 27 Oct 2013 18:28:11 +0000
Subject: [Numpy-discussion] Robust Sorting of Points
Message-ID: <526D5B3B.7030107@witherden.org>

Hi all,

This is a question which has been bugging me for a while.  I have an (N,
3) array where N ~ 16 of points.  These points are all unique and
separated by a reasonable distance.

I wish to sort these points into a canonical order in a fashion which is
robust against small perturbations.  In other words changing any
component of any of the points by an epsilon ~ 1e-12 should not affect
the resulting sorted order.

Considering a direct application of np.lexsort:

  In [6]: my_array = np.array([[-0.5, 0, 2**0.5],
                               [0.5, 0, 2**0.5 - 1e-15]])

  In [7]: my_array[np.lexsort(my_array.T)]
  Out[7]:  array([[ 0.5       ,  0.        ,  1.41421356],
                  [-0.5       ,  0.        ,  1.41421356]])

however, if the small 1e-15 perturbation is removed the order changes to
the 'natural' ordering.  Hence, np.lexsort is out.

Rounding the array before sorting is not suitable either; just because
(a - b) < epsilon does not mean that np.around(a, decimals=x) ==
np.around(b, decimals=b).

I am therefore looking for an efficient (= within a factor of 10 of
np.lexsort) solution to the problem.  I've looked at writing my own
comparison function cmp(x, y) which looks at the next dimension if
abs(x[i] - y[i]) < epsilon however using this with sorted is thousands
of times slower.  Given that I have well over 100,000 of these arrays
this is nuisance.

My other idea is to therefore find a means of quickly replacing all
numbers within 10*epsilon of a given number in an array with that
number.  This should permit the application of np.lexsort in order to
obtain the desired ordering (which is what I'm interesting in).
However, I am yet to figure out how to do this efficiently.

Before I throw in the towel and drop down to C are there any other neat
tricks I am missing?

Regards, Freddie.

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 836 bytes
Desc: OpenPGP digital signature
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20131027/153d8aca/attachment.sig>

From njs at pobox.com  Sun Oct 27 14:35:07 2013
From: njs at pobox.com (Nathaniel Smith)
Date: Sun, 27 Oct 2013 18:35:07 +0000
Subject: [Numpy-discussion] Robust Sorting of Points
In-Reply-To: <526D5B3B.7030107@witherden.org>
References: <526D5B3B.7030107@witherden.org>
Message-ID: <CAPJVwBmiwmxPSSGY1R7Xm8kPhbr49wMDR2G=x-OOFWGyv5nAOA@mail.gmail.com>

On Sun, Oct 27, 2013 at 6:28 PM, Freddie Witherden
<freddie at witherden.org> wrote:
> Hi all,
>
> This is a question which has been bugging me for a while.  I have an (N,
> 3) array where N ~ 16 of points.  These points are all unique and
> separated by a reasonable distance.
>
> I wish to sort these points into a canonical order in a fashion which is
> robust against small perturbations.  In other words changing any
> component of any of the points by an epsilon ~ 1e-12 should not affect
> the resulting sorted order.

I don't understand how this is possible even in principle.

Say your points are

 a = [0, 0, 0]
 b = [0, 0, 1e-12]

According to your criterion, either a or b should go first -- I don't
know which. Let's say our canonical ordering decides that a goes
first. But if you perturb both of them, then you have

 a = [0, 0, 1e-12]
 b = [0, 0, 0]

And now your requirement says that a still has to go first. But if a
goes first this time, then b had to go first the last time, by
symmetry. Thus your criterion is self-contradictory...?

-n


From freddie at witherden.org  Sun Oct 27 14:42:37 2013
From: freddie at witherden.org (Freddie Witherden)
Date: Sun, 27 Oct 2013 18:42:37 +0000
Subject: [Numpy-discussion] Robust Sorting of Points
In-Reply-To: <CAPJVwBmiwmxPSSGY1R7Xm8kPhbr49wMDR2G=x-OOFWGyv5nAOA@mail.gmail.com>
References: <526D5B3B.7030107@witherden.org>
	<CAPJVwBmiwmxPSSGY1R7Xm8kPhbr49wMDR2G=x-OOFWGyv5nAOA@mail.gmail.com>
Message-ID: <526D5E9D.4000503@witherden.org>

On 27/10/13 18:35, Nathaniel Smith wrote:
> On Sun, Oct 27, 2013 at 6:28 PM, Freddie Witherden
> <freddie at witherden.org> wrote:
>> Hi all,
>>
>> This is a question which has been bugging me for a while.  I have an (N,
>> 3) array where N ~ 16 of points.  These points are all unique and
>> separated by a reasonable distance.
>>
>> I wish to sort these points into a canonical order in a fashion which is
>> robust against small perturbations.  In other words changing any
>> component of any of the points by an epsilon ~ 1e-12 should not affect
>> the resulting sorted order.
> 
> I don't understand how this is possible even in principle.
> 
> Say your points are
> 
>  a = [0, 0, 0]
>  b = [0, 0, 1e-12]
> 
> According to your criterion, either a or b should go first -- I don't
> know which. Let's say our canonical ordering decides that a goes
> first. But if you perturb both of them, then you have
> 
>  a = [0, 0, 1e-12]
>  b = [0, 0, 0]
> 
> And now your requirement says that a still has to go first. But if a
> goes first this time, then b had to go first the last time, by
> symmetry. Thus your criterion is self-contradictory...?

Not exactly; in your case the distance between a and b is of the order
epislon.  However, my points are "all unique and separated by a
reasonable distance."  This requires at least one of the components of
any two points to differ in all instances, permitting an ordering to be
defined.  (Where if epislon ~ 1e-12 the minimum instance between any two
points is of order ~ 1e-6.)

Regards, Freddie.

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 836 bytes
Desc: OpenPGP digital signature
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20131027/2da72e05/attachment.sig>

From daniele at grinta.net  Sun Oct 27 14:54:02 2013
From: daniele at grinta.net (Daniele Nicolodi)
Date: Sun, 27 Oct 2013 19:54:02 +0100
Subject: [Numpy-discussion] Robust Sorting of Points
In-Reply-To: <526D5E9D.4000503@witherden.org>
References: <526D5B3B.7030107@witherden.org>	<CAPJVwBmiwmxPSSGY1R7Xm8kPhbr49wMDR2G=x-OOFWGyv5nAOA@mail.gmail.com>
	<526D5E9D.4000503@witherden.org>
Message-ID: <526D614A.9030404@grinta.net>

On 27/10/2013 19:42, Freddie Witherden wrote:
> On 27/10/13 18:35, Nathaniel Smith wrote:
>> On Sun, Oct 27, 2013 at 6:28 PM, Freddie Witherden
>> <freddie at witherden.org> wrote:
>>> Hi all,
>>>
>>> This is a question which has been bugging me for a while.  I have an (N,
>>> 3) array where N ~ 16 of points.  These points are all unique and
>>> separated by a reasonable distance.
>>>
>>> I wish to sort these points into a canonical order in a fashion which is
>>> robust against small perturbations.  In other words changing any
>>> component of any of the points by an epsilon ~ 1e-12 should not affect
>>> the resulting sorted order.
>>
>> I don't understand how this is possible even in principle.
>>
>> Say your points are
>>
>>  a = [0, 0, 0]
>>  b = [0, 0, 1e-12]
>>
>> According to your criterion, either a or b should go first -- I don't
>> know which. Let's say our canonical ordering decides that a goes
>> first. But if you perturb both of them, then you have
>>
>>  a = [0, 0, 1e-12]
>>  b = [0, 0, 0]
>>
>> And now your requirement says that a still has to go first. But if a
>> goes first this time, then b had to go first the last time, by
>> symmetry. Thus your criterion is self-contradictory...?
> 
> Not exactly; in your case the distance between a and b is of the order
> epislon.  However, my points are "all unique and separated by a
> reasonable distance."  This requires at least one of the components of
> any two points to differ in all instances, permitting an ordering to be
> defined.  (Where if epislon ~ 1e-12 the minimum instance between any two
> points is of order ~ 1e-6.)

Do you mean that all you points are distributed around some fixed points
in your space?  In this case, it looks like what you are looking for is
categorization or clustering and not sorting.  Once you perform
clustering, you can simply define an arbitrary order in which report the
content of each cluster.  If this is not the case, the problem that
Nathaniel highlishts is still present.

Cheers,
Daniele



From matthew.brett at gmail.com  Sun Oct 27 15:17:10 2013
From: matthew.brett at gmail.com (Matthew Brett)
Date: Sun, 27 Oct 2013 12:17:10 -0700
Subject: [Numpy-discussion] A context manager for print options
In-Reply-To: <CAA68w_my+343iXWROEtPxYYvE9SdbKEtj68a=FasiLxUNV+XKg@mail.gmail.com>
References: <CAA68w_my+343iXWROEtPxYYvE9SdbKEtj68a=FasiLxUNV+XKg@mail.gmail.com>
Message-ID: <CAH6Pt5p9WQkMeetMxyH594dyoSwPuCC19q4xg8oga3ab1Ti_tw@mail.gmail.com>

Hi,

On Sun, Oct 27, 2013 at 2:01 AM, Neil Girdhar <mistersheik at gmail.com> wrote:
> Why not replace get_printoptions/set_printoptions with a context manager
> accessed using numpy.printoptions in the same way that numpy.errstate
> exposes a context manager to seterr/geterr?  This would make the set method
> redundant.
>
> Also, the context manager returned by numpy.errstate, numpy.printoptions,
> etc. could expose the dictionary directly.  This would make the get methods
> redundant.

Great idea ...

Matthew


From freddie at witherden.org  Sun Oct 27 15:22:33 2013
From: freddie at witherden.org (Freddie Witherden)
Date: Sun, 27 Oct 2013 19:22:33 +0000
Subject: [Numpy-discussion] Robust Sorting of Points
In-Reply-To: <526D614A.9030404@grinta.net>
References: <526D5B3B.7030107@witherden.org>	<CAPJVwBmiwmxPSSGY1R7Xm8kPhbr49wMDR2G=x-OOFWGyv5nAOA@mail.gmail.com>	<526D5E9D.4000503@witherden.org>
	<526D614A.9030404@grinta.net>
Message-ID: <526D67F9.5020405@witherden.org>

On 27/10/13 18:54, Daniele Nicolodi wrote:
> On 27/10/2013 19:42, Freddie Witherden wrote:
>> On 27/10/13 18:35, Nathaniel Smith wrote:
>>> On Sun, Oct 27, 2013 at 6:28 PM, Freddie Witherden
>>> <freddie at witherden.org> wrote:
>>>> Hi all,
>>>>
>>>> This is a question which has been bugging me for a while.  I have an (N,
>>>> 3) array where N ~ 16 of points.  These points are all unique and
>>>> separated by a reasonable distance.
>>>>
>>>> I wish to sort these points into a canonical order in a fashion which is
>>>> robust against small perturbations.  In other words changing any
>>>> component of any of the points by an epsilon ~ 1e-12 should not affect
>>>> the resulting sorted order.
>>>
>>> I don't understand how this is possible even in principle.
>>>
>>> Say your points are
>>>
>>>  a = [0, 0, 0]
>>>  b = [0, 0, 1e-12]
>>>
>>> According to your criterion, either a or b should go first -- I don't
>>> know which. Let's say our canonical ordering decides that a goes
>>> first. But if you perturb both of them, then you have
>>>
>>>  a = [0, 0, 1e-12]
>>>  b = [0, 0, 0]
>>>
>>> And now your requirement says that a still has to go first. But if a
>>> goes first this time, then b had to go first the last time, by
>>> symmetry. Thus your criterion is self-contradictory...?
>>
>> Not exactly; in your case the distance between a and b is of the order
>> epislon.  However, my points are "all unique and separated by a
>> reasonable distance."  This requires at least one of the components of
>> any two points to differ in all instances, permitting an ordering to be
>> defined.  (Where if epislon ~ 1e-12 the minimum instance between any two
>> points is of order ~ 1e-6.)
> 
> Do you mean that all you points are distributed around some fixed points
> in your space?  In this case, it looks like what you are looking for is
> categorization or clustering and not sorting.  Once you perform
> clustering, you can simply define an arbitrary order in which report the
> content of each cluster.  If this is not the case, the problem that
> Nathaniel highlishts is still present.

I am not entirely sure what you mean here.  If x is my array of points
of size (16, 3) then I am guarenteeing that

  np.min(scipy.spatial.distance.pdist(x)) >= 1e-6

In this instance I am unsure how the issue highlighted by Nathaniel
might arise.  Of course it is (very) possible that I am missing
something, however, I believe under the terms of this constraint that it
is always possible to define an order with which to iterate through the
points which is invarient to shuffling of the points and small
pertubations of the components.

Regards, Freddie.


-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 836 bytes
Desc: OpenPGP digital signature
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20131027/da474b1a/attachment.sig>

From josef.pktd at gmail.com  Sun Oct 27 16:22:34 2013
From: josef.pktd at gmail.com (josef.pktd at gmail.com)
Date: Sun, 27 Oct 2013 16:22:34 -0400
Subject: [Numpy-discussion] Robust Sorting of Points
In-Reply-To: <526D67F9.5020405@witherden.org>
References: <526D5B3B.7030107@witherden.org>
	<CAPJVwBmiwmxPSSGY1R7Xm8kPhbr49wMDR2G=x-OOFWGyv5nAOA@mail.gmail.com>
	<526D5E9D.4000503@witherden.org> <526D614A.9030404@grinta.net>
	<526D67F9.5020405@witherden.org>
Message-ID: <CAMMTP+DZ20BTMQ6CTYWYo+FFBurq351gHfnXxf3WbN31FVFKYA@mail.gmail.com>

On Sun, Oct 27, 2013 at 3:22 PM, Freddie Witherden
<freddie at witherden.org> wrote:
> On 27/10/13 18:54, Daniele Nicolodi wrote:
>> On 27/10/2013 19:42, Freddie Witherden wrote:
>>> On 27/10/13 18:35, Nathaniel Smith wrote:
>>>> On Sun, Oct 27, 2013 at 6:28 PM, Freddie Witherden
>>>> <freddie at witherden.org> wrote:
>>>>> Hi all,
>>>>>
>>>>> This is a question which has been bugging me for a while.  I have an (N,
>>>>> 3) array where N ~ 16 of points.  These points are all unique and
>>>>> separated by a reasonable distance.
>>>>>
>>>>> I wish to sort these points into a canonical order in a fashion which is
>>>>> robust against small perturbations.  In other words changing any
>>>>> component of any of the points by an epsilon ~ 1e-12 should not affect
>>>>> the resulting sorted order.
>>>>
>>>> I don't understand how this is possible even in principle.
>>>>
>>>> Say your points are
>>>>
>>>>  a = [0, 0, 0]
>>>>  b = [0, 0, 1e-12]
>>>>
>>>> According to your criterion, either a or b should go first -- I don't
>>>> know which. Let's say our canonical ordering decides that a goes
>>>> first. But if you perturb both of them, then you have
>>>>
>>>>  a = [0, 0, 1e-12]
>>>>  b = [0, 0, 0]
>>>>
>>>> And now your requirement says that a still has to go first. But if a
>>>> goes first this time, then b had to go first the last time, by
>>>> symmetry. Thus your criterion is self-contradictory...?
>>>
>>> Not exactly; in your case the distance between a and b is of the order
>>> epislon.  However, my points are "all unique and separated by a
>>> reasonable distance."  This requires at least one of the components of
>>> any two points to differ in all instances, permitting an ordering to be
>>> defined.  (Where if epislon ~ 1e-12 the minimum instance between any two
>>> points is of order ~ 1e-6.)
>>
>> Do you mean that all you points are distributed around some fixed points
>> in your space?  In this case, it looks like what you are looking for is
>> categorization or clustering and not sorting.  Once you perform
>> clustering, you can simply define an arbitrary order in which report the
>> content of each cluster.  If this is not the case, the problem that
>> Nathaniel highlishts is still present.
>
> I am not entirely sure what you mean here.  If x is my array of points
> of size (16, 3) then I am guarenteeing that
>
>   np.min(scipy.spatial.distance.pdist(x)) >= 1e-6
>
> In this instance I am unsure how the issue highlighted by Nathaniel
> might arise.  Of course it is (very) possible that I am missing
> something, however, I believe under the terms of this constraint that it
> is always possible to define an order with which to iterate through the
> points which is invarient to shuffling of the points and small
> pertubations of the components.


If the epsilon or scale depends on the column, then, I think, divmod
should work to cut off the noise

>>> my_array[np.lexsort(divmod(my_array, [1e-1, 1e-12, 1])[0].T)]
array([[-0.5       ,  0.        ,  1.41421356],
       [ 0.5       ,  0.        ,  1.41421356]])

Josef


>
> Regards, Freddie.
>
>
>
> _______________________________________________
> NumPy-Discussion mailing list
> NumPy-Discussion at scipy.org
> http://mail.scipy.org/mailman/listinfo/numpy-discussion
>


From josef.pktd at gmail.com  Sun Oct 27 16:26:04 2013
From: josef.pktd at gmail.com (josef.pktd at gmail.com)
Date: Sun, 27 Oct 2013 16:26:04 -0400
Subject: [Numpy-discussion] Robust Sorting of Points
In-Reply-To: <CAMMTP+DZ20BTMQ6CTYWYo+FFBurq351gHfnXxf3WbN31FVFKYA@mail.gmail.com>
References: <526D5B3B.7030107@witherden.org>
	<CAPJVwBmiwmxPSSGY1R7Xm8kPhbr49wMDR2G=x-OOFWGyv5nAOA@mail.gmail.com>
	<526D5E9D.4000503@witherden.org> <526D614A.9030404@grinta.net>
	<526D67F9.5020405@witherden.org>
	<CAMMTP+DZ20BTMQ6CTYWYo+FFBurq351gHfnXxf3WbN31FVFKYA@mail.gmail.com>
Message-ID: <CAMMTP+DRRRBEt5doOZEcz4pYGPP2APN1NODCVmGVaRYB_VqXBw@mail.gmail.com>

On Sun, Oct 27, 2013 at 4:22 PM,  <josef.pktd at gmail.com> wrote:
> On Sun, Oct 27, 2013 at 3:22 PM, Freddie Witherden
> <freddie at witherden.org> wrote:
>> On 27/10/13 18:54, Daniele Nicolodi wrote:
>>> On 27/10/2013 19:42, Freddie Witherden wrote:
>>>> On 27/10/13 18:35, Nathaniel Smith wrote:
>>>>> On Sun, Oct 27, 2013 at 6:28 PM, Freddie Witherden
>>>>> <freddie at witherden.org> wrote:
>>>>>> Hi all,
>>>>>>
>>>>>> This is a question which has been bugging me for a while.  I have an (N,
>>>>>> 3) array where N ~ 16 of points.  These points are all unique and
>>>>>> separated by a reasonable distance.
>>>>>>
>>>>>> I wish to sort these points into a canonical order in a fashion which is
>>>>>> robust against small perturbations.  In other words changing any
>>>>>> component of any of the points by an epsilon ~ 1e-12 should not affect
>>>>>> the resulting sorted order.
>>>>>
>>>>> I don't understand how this is possible even in principle.
>>>>>
>>>>> Say your points are
>>>>>
>>>>>  a = [0, 0, 0]
>>>>>  b = [0, 0, 1e-12]
>>>>>
>>>>> According to your criterion, either a or b should go first -- I don't
>>>>> know which. Let's say our canonical ordering decides that a goes
>>>>> first. But if you perturb both of them, then you have
>>>>>
>>>>>  a = [0, 0, 1e-12]
>>>>>  b = [0, 0, 0]
>>>>>
>>>>> And now your requirement says that a still has to go first. But if a
>>>>> goes first this time, then b had to go first the last time, by
>>>>> symmetry. Thus your criterion is self-contradictory...?
>>>>
>>>> Not exactly; in your case the distance between a and b is of the order
>>>> epislon.  However, my points are "all unique and separated by a
>>>> reasonable distance."  This requires at least one of the components of
>>>> any two points to differ in all instances, permitting an ordering to be
>>>> defined.  (Where if epislon ~ 1e-12 the minimum instance between any two
>>>> points is of order ~ 1e-6.)
>>>
>>> Do you mean that all you points are distributed around some fixed points
>>> in your space?  In this case, it looks like what you are looking for is
>>> categorization or clustering and not sorting.  Once you perform
>>> clustering, you can simply define an arbitrary order in which report the
>>> content of each cluster.  If this is not the case, the problem that
>>> Nathaniel highlishts is still present.
>>
>> I am not entirely sure what you mean here.  If x is my array of points
>> of size (16, 3) then I am guarenteeing that
>>
>>   np.min(scipy.spatial.distance.pdist(x)) >= 1e-6
>>
>> In this instance I am unsure how the issue highlighted by Nathaniel
>> might arise.  Of course it is (very) possible that I am missing
>> something, however, I believe under the terms of this constraint that it
>> is always possible to define an order with which to iterate through the
>> points which is invarient to shuffling of the points and small
>> pertubations of the components.
>
>
> If the epsilon or scale depends on the column, then, I think, divmod
> should work to cut off the noise
>
>>>> my_array[np.lexsort(divmod(my_array, [1e-1, 1e-12, 1])[0].T)]
> array([[-0.5       ,  0.        ,  1.41421356],
>        [ 0.5       ,  0.        ,  1.41421356]])

>>> my_array = np.array([[-0.5, 0, 2**0.5], [-0.5, 1e-16, 2**0.5], [-0.5, -1e-16, 2**0.5],[0.5, 0, 2**0.5 - 1e-15]])
>>> my_array[np.lexsort(divmod(my_array, [1e-1, 1e-12, 1])[0].T)]
array([[ -5.00000000e-01,  -1.00000000e-16,   1.41421356e+00],
       [ -5.00000000e-01,   0.00000000e+00,   1.41421356e+00],
       [ -5.00000000e-01,   1.00000000e-16,   1.41421356e+00],
       [  5.00000000e-01,   0.00000000e+00,   1.41421356e+00]])


>
> Josef
>
>
>>
>> Regards, Freddie.
>>
>>
>>
>> _______________________________________________
>> NumPy-Discussion mailing list
>> NumPy-Discussion at scipy.org
>> http://mail.scipy.org/mailman/listinfo/numpy-discussion
>>


From freddie at witherden.org  Sun Oct 27 16:51:19 2013
From: freddie at witherden.org (Freddie Witherden)
Date: Sun, 27 Oct 2013 20:51:19 +0000
Subject: [Numpy-discussion] Robust Sorting of Points
In-Reply-To: <CAMMTP+DZ20BTMQ6CTYWYo+FFBurq351gHfnXxf3WbN31FVFKYA@mail.gmail.com>
References: <526D5B3B.7030107@witherden.org>	<CAPJVwBmiwmxPSSGY1R7Xm8kPhbr49wMDR2G=x-OOFWGyv5nAOA@mail.gmail.com>	<526D5E9D.4000503@witherden.org>
	<526D614A.9030404@grinta.net>	<526D67F9.5020405@witherden.org>
	<CAMMTP+DZ20BTMQ6CTYWYo+FFBurq351gHfnXxf3WbN31FVFKYA@mail.gmail.com>
Message-ID: <526D7CC7.3080008@witherden.org>

On 27/10/13 20:22, josef.pktd at gmail.com wrote:
> On Sun, Oct 27, 2013 at 3:22 PM, Freddie Witherden
> <freddie at witherden.org> wrote:
>> On 27/10/13 18:54, Daniele Nicolodi wrote:
>>> On 27/10/2013 19:42, Freddie Witherden wrote:
>>>> On 27/10/13 18:35, Nathaniel Smith wrote:
>>>>> On Sun, Oct 27, 2013 at 6:28 PM, Freddie Witherden
>>>>> <freddie at witherden.org> wrote:
>>>>>> Hi all,
>>>>>>
>>>>>> This is a question which has been bugging me for a while.  I have an (N,
>>>>>> 3) array where N ~ 16 of points.  These points are all unique and
>>>>>> separated by a reasonable distance.
>>>>>>
>>>>>> I wish to sort these points into a canonical order in a fashion which is
>>>>>> robust against small perturbations.  In other words changing any
>>>>>> component of any of the points by an epsilon ~ 1e-12 should not affect
>>>>>> the resulting sorted order.
>>>>>
>>>>> I don't understand how this is possible even in principle.
>>>>>
>>>>> Say your points are
>>>>>
>>>>>  a = [0, 0, 0]
>>>>>  b = [0, 0, 1e-12]
>>>>>
>>>>> According to your criterion, either a or b should go first -- I don't
>>>>> know which. Let's say our canonical ordering decides that a goes
>>>>> first. But if you perturb both of them, then you have
>>>>>
>>>>>  a = [0, 0, 1e-12]
>>>>>  b = [0, 0, 0]
>>>>>
>>>>> And now your requirement says that a still has to go first. But if a
>>>>> goes first this time, then b had to go first the last time, by
>>>>> symmetry. Thus your criterion is self-contradictory...?
>>>>
>>>> Not exactly; in your case the distance between a and b is of the order
>>>> epislon.  However, my points are "all unique and separated by a
>>>> reasonable distance."  This requires at least one of the components of
>>>> any two points to differ in all instances, permitting an ordering to be
>>>> defined.  (Where if epislon ~ 1e-12 the minimum instance between any two
>>>> points is of order ~ 1e-6.)
>>>
>>> Do you mean that all you points are distributed around some fixed points
>>> in your space?  In this case, it looks like what you are looking for is
>>> categorization or clustering and not sorting.  Once you perform
>>> clustering, you can simply define an arbitrary order in which report the
>>> content of each cluster.  If this is not the case, the problem that
>>> Nathaniel highlishts is still present.
>>
>> I am not entirely sure what you mean here.  If x is my array of points
>> of size (16, 3) then I am guarenteeing that
>>
>>   np.min(scipy.spatial.distance.pdist(x)) >= 1e-6
>>
>> In this instance I am unsure how the issue highlighted by Nathaniel
>> might arise.  Of course it is (very) possible that I am missing
>> something, however, I believe under the terms of this constraint that it
>> is always possible to define an order with which to iterate through the
>> points which is invarient to shuffling of the points and small
>> pertubations of the components.
> 
> 
> If the epsilon or scale depends on the column, then, I think, divmod
> should work to cut off the noise
> 
>>>> my_array[np.lexsort(divmod(my_array, [1e-1, 1e-12, 1])[0].T)]
> array([[-0.5       ,  0.        ,  1.41421356],
>        [ 0.5       ,  0.        ,  1.41421356]])

An interesting proposal.  However, it appears to have the same issues as
the rounding approach.  Consider:

In [5]: a, b = 1.0 + 1e-13, 1.0 - 1e-13

In [6]: abs(a - b) < 1e-12
Out[6]: True

In [7]: divmod(a, 1e-6)[0] == divmod(b, 1e-6)[0]
Out[7]: False

Hence should np.lexsort encounter such a pair it will consider a and b
to be different even though they are within epsilon of one another.

Regards, Freddie.



-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 836 bytes
Desc: OpenPGP digital signature
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20131027/c25169d6/attachment.sig>

From jmarch at enthought.com  Sun Oct 27 17:05:02 2013
From: jmarch at enthought.com (Jonathan March)
Date: Sun, 27 Oct 2013 16:05:02 -0500
Subject: [Numpy-discussion] Robust Sorting of Points
In-Reply-To: <526D7CC7.3080008@witherden.org>
References: <526D5B3B.7030107@witherden.org>
	<CAPJVwBmiwmxPSSGY1R7Xm8kPhbr49wMDR2G=x-OOFWGyv5nAOA@mail.gmail.com>
	<526D5E9D.4000503@witherden.org> <526D614A.9030404@grinta.net>
	<526D67F9.5020405@witherden.org>
	<CAMMTP+DZ20BTMQ6CTYWYo+FFBurq351gHfnXxf3WbN31FVFKYA@mail.gmail.com>
	<526D7CC7.3080008@witherden.org>
Message-ID: <CACJ8dbU-mM_NFtAXHRkWdWhAMZ63YjLOMkORtnocriOJe0L+bA@mail.gmail.com>

If an "almost always works" solution is good enough, then sort on the
distance to some fixed random point that is in the vicinity of your N
points.

Jonathan


On Sun, Oct 27, 2013 at 3:51 PM, Freddie Witherden <freddie at witherden.org>wrote:

> On 27/10/13 20:22, josef.pktd at gmail.com wrote:
> > On Sun, Oct 27, 2013 at 3:22 PM, Freddie Witherden
> > <freddie at witherden.org> wrote:
> >> On 27/10/13 18:54, Daniele Nicolodi wrote:
> >>> On 27/10/2013 19:42, Freddie Witherden wrote:
> >>>> On 27/10/13 18:35, Nathaniel Smith wrote:
> >>>>> On Sun, Oct 27, 2013 at 6:28 PM, Freddie Witherden
> >>>>> <freddie at witherden.org> wrote:
> >>>>>> Hi all,
> >>>>>>
> >>>>>> This is a question which has been bugging me for a while.  I have
> an (N,
> >>>>>> 3) array where N ~ 16 of points.  These points are all unique and
> >>>>>> separated by a reasonable distance.
> >>>>>>
> >>>>>> I wish to sort these points into a canonical order in a fashion
> which is
> >>>>>> robust against small perturbations.  In other words changing any
> >>>>>> component of any of the points by an epsilon ~ 1e-12 should not
> affect
> >>>>>> the resulting sorted order.
> >>>>>
> >>>>> I don't understand how this is possible even in principle.
> >>>>>
> >>>>> Say your points are
> >>>>>
> >>>>>  a = [0, 0, 0]
> >>>>>  b = [0, 0, 1e-12]
> >>>>>
> >>>>> According to your criterion, either a or b should go first -- I don't
> >>>>> know which. Let's say our canonical ordering decides that a goes
> >>>>> first. But if you perturb both of them, then you have
> >>>>>
> >>>>>  a = [0, 0, 1e-12]
> >>>>>  b = [0, 0, 0]
> >>>>>
> >>>>> And now your requirement says that a still has to go first. But if a
> >>>>> goes first this time, then b had to go first the last time, by
> >>>>> symmetry. Thus your criterion is self-contradictory...?
> >>>>
> >>>> Not exactly; in your case the distance between a and b is of the order
> >>>> epislon.  However, my points are "all unique and separated by a
> >>>> reasonable distance."  This requires at least one of the components of
> >>>> any two points to differ in all instances, permitting an ordering to
> be
> >>>> defined.  (Where if epislon ~ 1e-12 the minimum instance between any
> two
> >>>> points is of order ~ 1e-6.)
> >>>
> >>> Do you mean that all you points are distributed around some fixed
> points
> >>> in your space?  In this case, it looks like what you are looking for is
> >>> categorization or clustering and not sorting.  Once you perform
> >>> clustering, you can simply define an arbitrary order in which report
> the
> >>> content of each cluster.  If this is not the case, the problem that
> >>> Nathaniel highlishts is still present.
> >>
> >> I am not entirely sure what you mean here.  If x is my array of points
> >> of size (16, 3) then I am guarenteeing that
> >>
> >>   np.min(scipy.spatial.distance.pdist(x)) >= 1e-6
> >>
> >> In this instance I am unsure how the issue highlighted by Nathaniel
> >> might arise.  Of course it is (very) possible that I am missing
> >> something, however, I believe under the terms of this constraint that it
> >> is always possible to define an order with which to iterate through the
> >> points which is invarient to shuffling of the points and small
> >> pertubations of the components.
> >
> >
> > If the epsilon or scale depends on the column, then, I think, divmod
> > should work to cut off the noise
> >
> >>>> my_array[np.lexsort(divmod(my_array, [1e-1, 1e-12, 1])[0].T)]
> > array([[-0.5       ,  0.        ,  1.41421356],
> >        [ 0.5       ,  0.        ,  1.41421356]])
>
> An interesting proposal.  However, it appears to have the same issues as
> the rounding approach.  Consider:
>
> In [5]: a, b = 1.0 + 1e-13, 1.0 - 1e-13
>
> In [6]: abs(a - b) < 1e-12
> Out[6]: True
>
> In [7]: divmod(a, 1e-6)[0] == divmod(b, 1e-6)[0]
> Out[7]: False
>
> Hence should np.lexsort encounter such a pair it will consider a and b
> to be different even though they are within epsilon of one another.
>
> Regards, Freddie.
>
>
>
>
> _______________________________________________
> NumPy-Discussion mailing list
> NumPy-Discussion at scipy.org
> http://mail.scipy.org/mailman/listinfo/numpy-discussion
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20131027/43554864/attachment.html>

From freddie at witherden.org  Sun Oct 27 18:41:23 2013
From: freddie at witherden.org (Freddie Witherden)
Date: Sun, 27 Oct 2013 22:41:23 +0000
Subject: [Numpy-discussion] Robust Sorting of Points
In-Reply-To: <CACJ8dbU-mM_NFtAXHRkWdWhAMZ63YjLOMkORtnocriOJe0L+bA@mail.gmail.com>
References: <526D5B3B.7030107@witherden.org>	<CAPJVwBmiwmxPSSGY1R7Xm8kPhbr49wMDR2G=x-OOFWGyv5nAOA@mail.gmail.com>	<526D5E9D.4000503@witherden.org>
	<526D614A.9030404@grinta.net>	<526D67F9.5020405@witherden.org>	<CAMMTP+DZ20BTMQ6CTYWYo+FFBurq351gHfnXxf3WbN31FVFKYA@mail.gmail.com>	<526D7CC7.3080008@witherden.org>
	<CACJ8dbU-mM_NFtAXHRkWdWhAMZ63YjLOMkORtnocriOJe0L+bA@mail.gmail.com>
Message-ID: <526D9693.8050704@witherden.org>

On 27/10/13 21:05, Jonathan March wrote:
> If an "almost always works" solution is good enough, then sort on the
> distance to some fixed random point that is in the vicinity of your N
> points.

I had considered this.  Unfortunately I need a solution which really
does always work.

The only pure-Python solution I can envision -- at the moment anyway --
is to do some cleverness with the output of np.unique to identify
similar values and replace them with an arbitrarily chosen one.  This
should permit the output to be passed to np.lexsort without issue.

Regards, Freddie.




-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 836 bytes
Desc: OpenPGP digital signature
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20131027/d3e97ee1/attachment.sig>

From njs at pobox.com  Sun Oct 27 18:53:37 2013
From: njs at pobox.com (Nathaniel Smith)
Date: Sun, 27 Oct 2013 22:53:37 +0000
Subject: [Numpy-discussion] Robust Sorting of Points
In-Reply-To: <526D9693.8050704@witherden.org>
References: <526D5B3B.7030107@witherden.org>
	<CAPJVwBmiwmxPSSGY1R7Xm8kPhbr49wMDR2G=x-OOFWGyv5nAOA@mail.gmail.com>
	<526D5E9D.4000503@witherden.org> <526D614A.9030404@grinta.net>
	<526D67F9.5020405@witherden.org>
	<CAMMTP+DZ20BTMQ6CTYWYo+FFBurq351gHfnXxf3WbN31FVFKYA@mail.gmail.com>
	<526D7CC7.3080008@witherden.org>
	<CACJ8dbU-mM_NFtAXHRkWdWhAMZ63YjLOMkORtnocriOJe0L+bA@mail.gmail.com>
	<526D9693.8050704@witherden.org>
Message-ID: <CAPJVwB=WWfU3TeWe0c0xz-v=+Kp75PKcvrAEG5Y_CLc4uXXK5w@mail.gmail.com>

On Sun, Oct 27, 2013 at 10:41 PM, Freddie Witherden
<freddie at witherden.org> wrote:
> On 27/10/13 21:05, Jonathan March wrote:
>> If an "almost always works" solution is good enough, then sort on the
>> distance to some fixed random point that is in the vicinity of your N
>> points.
>
> I had considered this.  Unfortunately I need a solution which really
> does always work.
>
> The only pure-Python solution I can envision -- at the moment anyway --
> is to do some cleverness with the output of np.unique to identify
> similar values and replace them with an arbitrarily chosen one.  This
> should permit the output to be passed to np.lexsort without issue.

# Warning 1: Untested code.
# Warning 2: I can't tell if this is clever or horrible.
r = np.random.RandomState(0)
while True:
    # This perturbation leaves lexical order unchanged, but will
eventually shift
    # all points away from rounding bin boundarys.
    shift = r.rand()
    if np.any(np.around(perturbed + (shift - 1e-11), decimals=10) !=
np.around(perturbed + (shift + 1e-11))):
        continue
    return np.lexsort(np.around(perturbed, decimals=10))

-n


From mistersheik at gmail.com  Sun Oct 27 18:59:33 2013
From: mistersheik at gmail.com (Neil Girdhar)
Date: Sun, 27 Oct 2013 18:59:33 -0400
Subject: [Numpy-discussion] Testing
Message-ID: <CAA68w_muyja7UWyVsh0qSQzfb47i7EPgOgeJAHU+-WPve4f-Lg@mail.gmail.com>

How do I test a patch that I've made locally?  I can't seem to import numpy
locally:

Error importing numpy: you should not try to import numpy from
        its source directory; please exit the numpy source tree, and
relaunch
        your python intepreter from there.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20131027/35e8cf19/attachment.html>

From njs at pobox.com  Sun Oct 27 19:12:21 2013
From: njs at pobox.com (Nathaniel Smith)
Date: Sun, 27 Oct 2013 23:12:21 +0000
Subject: [Numpy-discussion] Testing
In-Reply-To: <CAA68w_muyja7UWyVsh0qSQzfb47i7EPgOgeJAHU+-WPve4f-Lg@mail.gmail.com>
References: <CAA68w_muyja7UWyVsh0qSQzfb47i7EPgOgeJAHU+-WPve4f-Lg@mail.gmail.com>
Message-ID: <CAPJVwBmA5o7UwisORVMe6e+kE-7oSKzUBezHodLBiyaEWDSV3Q@mail.gmail.com>

On Sun, Oct 27, 2013 at 10:59 PM, Neil Girdhar <mistersheik at gmail.com> wrote:
> How do I test a patch that I've made locally?  I can't seem to import numpy
> locally:
>
> Error importing numpy: you should not try to import numpy from
>         its source directory; please exit the numpy source tree, and
> relaunch
>         your python intepreter from there.

python runtests.py --help

-n


From charlesr.harris at gmail.com  Sun Oct 27 19:12:38 2013
From: charlesr.harris at gmail.com (Charles R Harris)
Date: Sun, 27 Oct 2013 17:12:38 -0600
Subject: [Numpy-discussion] Testing
In-Reply-To: <CAA68w_muyja7UWyVsh0qSQzfb47i7EPgOgeJAHU+-WPve4f-Lg@mail.gmail.com>
References: <CAA68w_muyja7UWyVsh0qSQzfb47i7EPgOgeJAHU+-WPve4f-Lg@mail.gmail.com>
Message-ID: <CAB6mnxK8BSbOTJWDYBBUcX=R7=r2LJDMWHBLzr=21kyRkQF-yQ@mail.gmail.com>

On Sun, Oct 27, 2013 at 4:59 PM, Neil Girdhar <mistersheik at gmail.com> wrote:

> How do I test a patch that I've made locally?  I can't seem to import
> numpy locally:
>
> Error importing numpy: you should not try to import numpy from
>         its source directory; please exit the numpy source tree, and
> relaunch
>         your python intepreter from there.
>
>
>
If you are running current master do

python runtests.py --help

Chuck

>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20131027/d8c0c793/attachment.html>

From mistersheik at gmail.com  Sun Oct 27 19:13:42 2013
From: mistersheik at gmail.com (Neil Girdhar)
Date: Sun, 27 Oct 2013 19:13:42 -0400
Subject: [Numpy-discussion] Testing
In-Reply-To: <CAB6mnxK8BSbOTJWDYBBUcX=R7=r2LJDMWHBLzr=21kyRkQF-yQ@mail.gmail.com>
References: <CAA68w_muyja7UWyVsh0qSQzfb47i7EPgOgeJAHU+-WPve4f-Lg@mail.gmail.com>
	<CAB6mnxK8BSbOTJWDYBBUcX=R7=r2LJDMWHBLzr=21kyRkQF-yQ@mail.gmail.com>
Message-ID: <CAA68w_==x2r+VGfH5ufs_hrw62LcjMAEoLWXW7MvEmNTStNqYA@mail.gmail.com>

Since I am trying to add a "printoptions" context manager, I would like to
test it.  Should I add tests, or can I somehow use it from an ipython shell?


On Sun, Oct 27, 2013 at 7:12 PM, Charles R Harris <charlesr.harris at gmail.com
> wrote:

>
>
>
> On Sun, Oct 27, 2013 at 4:59 PM, Neil Girdhar <mistersheik at gmail.com>wrote:
>
>> How do I test a patch that I've made locally?  I can't seem to import
>> numpy locally:
>>
>> Error importing numpy: you should not try to import numpy from
>>         its source directory; please exit the numpy source tree, and
>> relaunch
>>         your python intepreter from there.
>>
>>
>>
> If you are running current master do
>
> python runtests.py --help
>
> Chuck
>
>>
>>
> _______________________________________________
> NumPy-Discussion mailing list
> NumPy-Discussion at scipy.org
> http://mail.scipy.org/mailman/listinfo/numpy-discussion
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20131027/b39158a8/attachment.html>

From mistersheik at gmail.com  Sun Oct 27 19:14:27 2013
From: mistersheik at gmail.com (Neil Girdhar)
Date: Sun, 27 Oct 2013 19:14:27 -0400
Subject: [Numpy-discussion] Testing
In-Reply-To: <CAA68w_==x2r+VGfH5ufs_hrw62LcjMAEoLWXW7MvEmNTStNqYA@mail.gmail.com>
References: <CAA68w_muyja7UWyVsh0qSQzfb47i7EPgOgeJAHU+-WPve4f-Lg@mail.gmail.com>
	<CAB6mnxK8BSbOTJWDYBBUcX=R7=r2LJDMWHBLzr=21kyRkQF-yQ@mail.gmail.com>
	<CAA68w_==x2r+VGfH5ufs_hrw62LcjMAEoLWXW7MvEmNTStNqYA@mail.gmail.com>
Message-ID: <CAA68w_=R1jL4LhCyMOnhfOgTAt6gjsKCa5ut7rh5x=EuH=b24Q@mail.gmail.com>

Ah, sorry, didn't see that I can do that from runtests!!  Thanks!!


On Sun, Oct 27, 2013 at 7:13 PM, Neil Girdhar <mistersheik at gmail.com> wrote:

> Since I am trying to add a "printoptions" context manager, I would like to
> test it.  Should I add tests, or can I somehow use it from an ipython shell?
>
>
> On Sun, Oct 27, 2013 at 7:12 PM, Charles R Harris <
> charlesr.harris at gmail.com> wrote:
>
>>
>>
>>
>> On Sun, Oct 27, 2013 at 4:59 PM, Neil Girdhar <mistersheik at gmail.com>wrote:
>>
>>> How do I test a patch that I've made locally?  I can't seem to import
>>> numpy locally:
>>>
>>> Error importing numpy: you should not try to import numpy from
>>>         its source directory; please exit the numpy source tree, and
>>> relaunch
>>>         your python intepreter from there.
>>>
>>>
>>>
>> If you are running current master do
>>
>> python runtests.py --help
>>
>> Chuck
>>
>>>
>>>
>> _______________________________________________
>> NumPy-Discussion mailing list
>> NumPy-Discussion at scipy.org
>> http://mail.scipy.org/mailman/listinfo/numpy-discussion
>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20131027/eba292d2/attachment.html>

From mistersheik at gmail.com  Sun Oct 27 21:23:51 2013
From: mistersheik at gmail.com (Neil Girdhar)
Date: Sun, 27 Oct 2013 21:23:51 -0400
Subject: [Numpy-discussion] Code review request: PrintOptions
Message-ID: <CAA68w_=unUBKysZQbF0obSCAM310rrnvwWGx4GEfPgD56ccbQQ@mail.gmail.com>

This is my first code review request, so I may have done some things wrong.
 I think the following URL should work?
https://github.com/MisterSheik/numpy/compare

Best,

Neil
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20131027/9532881e/attachment.html>

From charlesr.harris at gmail.com  Sun Oct 27 21:38:23 2013
From: charlesr.harris at gmail.com (Charles R Harris)
Date: Sun, 27 Oct 2013 19:38:23 -0600
Subject: [Numpy-discussion] Code review request: PrintOptions
In-Reply-To: <CAA68w_=unUBKysZQbF0obSCAM310rrnvwWGx4GEfPgD56ccbQQ@mail.gmail.com>
References: <CAA68w_=unUBKysZQbF0obSCAM310rrnvwWGx4GEfPgD56ccbQQ@mail.gmail.com>
Message-ID: <CAB6mnxKgMOP+K0jDya-ztWFXmNu47+MR0-yNXRm1vLMmVyNkpw@mail.gmail.com>

On Sun, Oct 27, 2013 at 7:23 PM, Neil Girdhar <mistersheik at gmail.com> wrote:

> This is my first code review request, so I may have done some things
> wrong.  I think the following URL should work?
> https://github.com/MisterSheik/numpy/compare
>
> The first thing to do is make a new branch for your work. Probably the
easiest way from where you are is to make the branch, which will have your
changes in it, then go back to master and git reset --hard to the last
commit before your work. Working in master is a big no-no. See
`doc/source/dev/gitwash/development_workflow.rst`. When you are ready, make
a PR for that branch. The code will get reviewed at that point.

Chuck
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20131027/d7bf836f/attachment.html>

From mistersheik at gmail.com  Sun Oct 27 21:42:02 2013
From: mistersheik at gmail.com (Neil Girdhar)
Date: Sun, 27 Oct 2013 21:42:02 -0400
Subject: [Numpy-discussion] Code review request: PrintOptions
In-Reply-To: <CAB6mnxKgMOP+K0jDya-ztWFXmNu47+MR0-yNXRm1vLMmVyNkpw@mail.gmail.com>
References: <CAA68w_=unUBKysZQbF0obSCAM310rrnvwWGx4GEfPgD56ccbQQ@mail.gmail.com>
	<CAB6mnxKgMOP+K0jDya-ztWFXmNu47+MR0-yNXRm1vLMmVyNkpw@mail.gmail.com>
Message-ID: <CAA68w_k3rN8CLeRbStuZ6D+KN-j+aFsD=rxRp+4zG3sgzD_0cQ@mail.gmail.com>

Yeah, I realized that I missed that and figured it wouldn't matter since it
was my own master and I don't plan on making other changes to numpy.  If
you don't mind, how do I move my changelist into a branch?  I'm really
worried I'm going to lose my changes.


On Sun, Oct 27, 2013 at 9:38 PM, Charles R Harris <charlesr.harris at gmail.com
> wrote:

>
>
>
> On Sun, Oct 27, 2013 at 7:23 PM, Neil Girdhar <mistersheik at gmail.com>wrote:
>
>> This is my first code review request, so I may have done some things
>> wrong.  I think the following URL should work?
>> https://github.com/MisterSheik/numpy/compare
>>
>> The first thing to do is make a new branch for your work. Probably the
> easiest way from where you are is to make the branch, which will have your
> changes in it, then go back to master and git reset --hard to the last
> commit before your work. Working in master is a big no-no. See
> `doc/source/dev/gitwash/development_workflow.rst`. When you are ready, make
> a PR for that branch. The code will get reviewed at that point.
>
> Chuck
>
> _______________________________________________
> NumPy-Discussion mailing list
> NumPy-Discussion at scipy.org
> http://mail.scipy.org/mailman/listinfo/numpy-discussion
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20131027/d2b7fd92/attachment.html>

From mistersheik at gmail.com  Sun Oct 27 21:45:36 2013
From: mistersheik at gmail.com (Neil Girdhar)
Date: Sun, 27 Oct 2013 21:45:36 -0400
Subject: [Numpy-discussion] Code review request: PrintOptions
In-Reply-To: <CAA68w_k3rN8CLeRbStuZ6D+KN-j+aFsD=rxRp+4zG3sgzD_0cQ@mail.gmail.com>
References: <CAA68w_=unUBKysZQbF0obSCAM310rrnvwWGx4GEfPgD56ccbQQ@mail.gmail.com>
	<CAB6mnxKgMOP+K0jDya-ztWFXmNu47+MR0-yNXRm1vLMmVyNkpw@mail.gmail.com>
	<CAA68w_k3rN8CLeRbStuZ6D+KN-j+aFsD=rxRp+4zG3sgzD_0cQ@mail.gmail.com>
Message-ID: <CAA68w_n3V8_b--oi=nkAQfkNBoLsarHvsULisc2pyzsEBQidOA@mail.gmail.com>

Is this what I want?  https://github.com/numpy/numpy/pull/3987


On Sun, Oct 27, 2013 at 9:42 PM, Neil Girdhar <mistersheik at gmail.com> wrote:

> Yeah, I realized that I missed that and figured it wouldn't matter since
> it was my own master and I don't plan on making other changes to numpy.  If
> you don't mind, how do I move my changelist into a branch?  I'm really
> worried I'm going to lose my changes.
>
>
> On Sun, Oct 27, 2013 at 9:38 PM, Charles R Harris <
> charlesr.harris at gmail.com> wrote:
>
>>
>>
>>
>> On Sun, Oct 27, 2013 at 7:23 PM, Neil Girdhar <mistersheik at gmail.com>wrote:
>>
>>> This is my first code review request, so I may have done some things
>>> wrong.  I think the following URL should work?
>>> https://github.com/MisterSheik/numpy/compare
>>>
>>> The first thing to do is make a new branch for your work. Probably the
>> easiest way from where you are is to make the branch, which will have your
>> changes in it, then go back to master and git reset --hard to the last
>> commit before your work. Working in master is a big no-no. See
>> `doc/source/dev/gitwash/development_workflow.rst`. When you are ready, make
>> a PR for that branch. The code will get reviewed at that point.
>>
>> Chuck
>>
>> _______________________________________________
>> NumPy-Discussion mailing list
>> NumPy-Discussion at scipy.org
>> http://mail.scipy.org/mailman/listinfo/numpy-discussion
>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20131027/1f78f948/attachment.html>

From aron at ahmadia.net  Sun Oct 27 21:46:10 2013
From: aron at ahmadia.net (Aron Ahmadia)
Date: Sun, 27 Oct 2013 21:46:10 -0400
Subject: [Numpy-discussion] Code review request: PrintOptions
In-Reply-To: <CAA68w_k3rN8CLeRbStuZ6D+KN-j+aFsD=rxRp+4zG3sgzD_0cQ@mail.gmail.com>
References: <CAA68w_=unUBKysZQbF0obSCAM310rrnvwWGx4GEfPgD56ccbQQ@mail.gmail.com>
	<CAB6mnxKgMOP+K0jDya-ztWFXmNu47+MR0-yNXRm1vLMmVyNkpw@mail.gmail.com>
	<CAA68w_k3rN8CLeRbStuZ6D+KN-j+aFsD=rxRp+4zG3sgzD_0cQ@mail.gmail.com>
Message-ID: <CAPhiW4irdBPBLTREXvJiBfvp4Ss5GRZTAdWA3qyFVfzrAWU1Jw@mail.gmail.com>

On Sun, Oct 27, 2013 at 9:42 PM, Neil Girdhar <mistersheik at gmail.com> wrote:

> Yeah, I realized that I missed that and figured it wouldn't matter since
> it was my own master and I don't plan on making other changes to numpy.  If
> you don't mind, how do I move my changelist into a branch?  I'm really
> worried I'm going to lose my changes.


'master' is just another branch.  You can do:

    git checkout -b new_branch_name

In your current repository to create a new branch pointing to where you are
now (the master branch which you want to move)

Chuck's instructions follow that:

   git checkout master
   git reset --hard origin/master

Some people advise that you just delete the 'master' branch on your local
repository/forks.  This can simplify the process of accidentally committing
to it, though it's easy to recover from.

Cheers,
Aron
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20131027/f85f41e5/attachment.html>

From gexarcha1 at gmail.com  Sun Oct 27 22:13:03 2013
From: gexarcha1 at gmail.com (Georgios Exarchakis)
Date: Sun, 27 Oct 2013 19:13:03 -0700
Subject: [Numpy-discussion] memmory management question
Message-ID: <526DC82F.3070606@gmail.com>

Hi,

I am using numpy with ipython from anaconda and I observe the following 
behavior:

    Python 2.7.5 |Anaconda 1.7.0 (64-bit)| (default, Jun 28 2013, 22:10:09)
    Type "copyright", "credits" or "license" for more information.

    IPython 1.0.0 -- An enhanced Interactive Python.
    ?         -> Introduction and overview of IPython's features.
    %quickref -> Quick reference.
    help      -> Python's own help system.
    object?   -> Details about 'object', use 'object??' for extra details.
    Using matplotlib backend: Qt4Agg

    In [1]: a= np.random.rand(500000,1000)

    In [2]: a = a[:10000]

    In [3]: c= np.random.rand(500000,1000)

    In [4]:


After In[1] I have an extra 3.7 GB of memory used, but this memory is 
not released at In[2]. I thought there might be some clever memory 
management trick so I executted In[3] but that just added an extra 3.7GB 
of memorry without releasing anything.


Is that the right behavior in this case?

If yes then how do you release memorry by slicing away parts of an 
array? Can you give me a description of the numpy internals in this case?

Thank you very much for your time,

Georgios
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20131027/fba92f36/attachment.html>

From charlesr.harris at gmail.com  Mon Oct 28 00:13:26 2013
From: charlesr.harris at gmail.com (Charles R Harris)
Date: Sun, 27 Oct 2013 22:13:26 -0600
Subject: [Numpy-discussion] Robust Sorting of Points
In-Reply-To: <526D5B3B.7030107@witherden.org>
References: <526D5B3B.7030107@witherden.org>
Message-ID: <CAB6mnx+Au2cW3Ee-VuAO9pAL9g41CzkyXm3kFK0fOffzByXo8w@mail.gmail.com>

On Sun, Oct 27, 2013 at 12:28 PM, Freddie Witherden
<freddie at witherden.org>wrote:

> Hi all,
>
> This is a question which has been bugging me for a while.  I have an (N,
> 3) array where N ~ 16 of points.  These points are all unique and
> separated by a reasonable distance.
>
> I wish to sort these points into a canonical order in a fashion which is
> robust against small perturbations.  In other words changing any
> component of any of the points by an epsilon ~ 1e-12 should not affect
> the resulting sorted order.
>
> Considering a direct application of np.lexsort:
>
>   In [6]: my_array = np.array([[-0.5, 0, 2**0.5],
>                                [0.5, 0, 2**0.5 - 1e-15]])
>
>   In [7]: my_array[np.lexsort(my_array.T)]
>   Out[7]:  array([[ 0.5       ,  0.        ,  1.41421356],
>                   [-0.5       ,  0.        ,  1.41421356]])
>
> however, if the small 1e-15 perturbation is removed the order changes to
> the 'natural' ordering.  Hence, np.lexsort is out.
>
> Rounding the array before sorting is not suitable either; just because
> (a - b) < epsilon does not mean that np.around(a, decimals=x) ==
> np.around(b, decimals=b).
>
> I am therefore looking for an efficient (= within a factor of 10 of
> np.lexsort) solution to the problem.  I've looked at writing my own
> comparison function cmp(x, y) which looks at the next dimension if
> abs(x[i] - y[i]) < epsilon however using this with sorted is thousands
> of times slower.  Given that I have well over 100,000 of these arrays
> this is nuisance.
>
> My other idea is to therefore find a means of quickly replacing all
> numbers within 10*epsilon of a given number in an array with that
> number.  This should permit the application of np.lexsort in order to
> obtain the desired ordering (which is what I'm interesting in).
> However, I am yet to figure out how to do this efficiently.
>
> Before I throw in the towel and drop down to C are there any other neat
> tricks I am missing?
>
>
Sort them as strings?

In [17]: a = np.random.random((5,3))

In [18]: a
Out[18]:
array([[ 0.64734085,  0.71582772,  0.82743219],
       [ 0.92769057,  0.46880266,  0.73836167],
       [ 0.11904745,  0.56834934,  0.16144849],
       [ 0.59186013,  0.90698496,  0.56950572],
       [ 0.7317681 ,  0.93495724,  0.19217244]])

In [19]: a.view('S24').sort(0)

In [20]: a
Out[20]:
array([[ 0.7317681 ,  0.93495724,  0.19217244],
       [ 0.64734085,  0.71582772,  0.82743219],
       [ 0.59186013,  0.90698496,  0.56950572],
       [ 0.92769057,  0.46880266,  0.73836167],
       [ 0.11904745,  0.56834934,  0.16144849]])

Chuck
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20131027/2d358f72/attachment.html>

From charlesr.harris at gmail.com  Mon Oct 28 00:16:37 2013
From: charlesr.harris at gmail.com (Charles R Harris)
Date: Sun, 27 Oct 2013 22:16:37 -0600
Subject: [Numpy-discussion] Robust Sorting of Points
In-Reply-To: <CAB6mnx+Au2cW3Ee-VuAO9pAL9g41CzkyXm3kFK0fOffzByXo8w@mail.gmail.com>
References: <526D5B3B.7030107@witherden.org>
	<CAB6mnx+Au2cW3Ee-VuAO9pAL9g41CzkyXm3kFK0fOffzByXo8w@mail.gmail.com>
Message-ID: <CAB6mnxL3e=L6uTbqCnhUi77+-XizKAvPyofyABgKnufsFhh4Xg@mail.gmail.com>

On Sun, Oct 27, 2013 at 10:13 PM, Charles R Harris <
charlesr.harris at gmail.com> wrote:

>
>
>
> On Sun, Oct 27, 2013 at 12:28 PM, Freddie Witherden <freddie at witherden.org
> > wrote:
>
>> Hi all,
>>
>> This is a question which has been bugging me for a while.  I have an (N,
>> 3) array where N ~ 16 of points.  These points are all unique and
>> separated by a reasonable distance.
>>
>> I wish to sort these points into a canonical order in a fashion which is
>> robust against small perturbations.  In other words changing any
>> component of any of the points by an epsilon ~ 1e-12 should not affect
>> the resulting sorted order.
>>
>> Considering a direct application of np.lexsort:
>>
>>   In [6]: my_array = np.array([[-0.5, 0, 2**0.5],
>>                                [0.5, 0, 2**0.5 - 1e-15]])
>>
>>   In [7]: my_array[np.lexsort(my_array.T)]
>>   Out[7]:  array([[ 0.5       ,  0.        ,  1.41421356],
>>                   [-0.5       ,  0.        ,  1.41421356]])
>>
>> however, if the small 1e-15 perturbation is removed the order changes to
>> the 'natural' ordering.  Hence, np.lexsort is out.
>>
>> Rounding the array before sorting is not suitable either; just because
>> (a - b) < epsilon does not mean that np.around(a, decimals=x) ==
>> np.around(b, decimals=b).
>>
>> I am therefore looking for an efficient (= within a factor of 10 of
>> np.lexsort) solution to the problem.  I've looked at writing my own
>> comparison function cmp(x, y) which looks at the next dimension if
>> abs(x[i] - y[i]) < epsilon however using this with sorted is thousands
>> of times slower.  Given that I have well over 100,000 of these arrays
>> this is nuisance.
>>
>> My other idea is to therefore find a means of quickly replacing all
>> numbers within 10*epsilon of a given number in an array with that
>> number.  This should permit the application of np.lexsort in order to
>> obtain the desired ordering (which is what I'm interesting in).
>> However, I am yet to figure out how to do this efficiently.
>>
>> Before I throw in the towel and drop down to C are there any other neat
>> tricks I am missing?
>>
>>
> Sort them as strings?
>
> In [17]: a = np.random.random((5,3))
>
> In [18]: a
> Out[18]:
> array([[ 0.64734085,  0.71582772,  0.82743219],
>        [ 0.92769057,  0.46880266,  0.73836167],
>        [ 0.11904745,  0.56834934,  0.16144849],
>        [ 0.59186013,  0.90698496,  0.56950572],
>        [ 0.7317681 ,  0.93495724,  0.19217244]])
>
> In [19]: a.view('S24').sort(0)
>
> In [20]: a
> Out[20]:
> array([[ 0.7317681 ,  0.93495724,  0.19217244],
>        [ 0.64734085,  0.71582772,  0.82743219],
>        [ 0.59186013,  0.90698496,  0.56950572],
>        [ 0.92769057,  0.46880266,  0.73836167],
>        [ 0.11904745,  0.56834934,  0.16144849]])
>
>
nvm, that won't work.

Chuck
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20131027/ea84e214/attachment.html>

From davidmenhur at gmail.com  Mon Oct 28 08:22:23 2013
From: davidmenhur at gmail.com (=?UTF-8?B?RGHPgGlk?=)
Date: Mon, 28 Oct 2013 13:22:23 +0100
Subject: [Numpy-discussion] memmory management question
In-Reply-To: <526DC82F.3070606@gmail.com>
References: <526DC82F.3070606@gmail.com>
Message-ID: <CAJhcF=3p_2EoO38kM7y6yCDO+X26Yq0tdZp5KyKVw=j+qT+t1A@mail.gmail.com>

On 28 October 2013 03:13, Georgios Exarchakis <gexarcha1 at gmail.com> wrote:

> If yes then how do you release memorry by slicing away parts of an array?


An array is a single Python object. In your example, there is always one
reference pointing to the array (either the whole array or only a view), so
the memory cannot be released.

In your case, a = a[:10000].copy() releases the memory by creating a new
object and pointing a to this new one, so the big array gets unreferenced
and thus, garbage collected. The price to pay is that, for some time, you
have living in memory both the original array and your new one: if they
were very big, you could run out of memory; and that you have to perform a
memcopy (again, slow if you have lots of data).


/David.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20131028/33b7701f/attachment.html>

From pierre.haessig at crans.org  Mon Oct 28 08:37:19 2013
From: pierre.haessig at crans.org (Pierre Haessig)
Date: Mon, 28 Oct 2013 13:37:19 +0100
Subject: [Numpy-discussion] memmory management question
In-Reply-To: <526DC82F.3070606@gmail.com>
References: <526DC82F.3070606@gmail.com>
Message-ID: <526E5A7F.6030703@crans.org>

Hi,

Along the line of what David said, I just looked at the flags :

a = np.arange(10)

a.flags
  [...]
  OWNDATA : True

a = a[:3]

a.flags
  [...]
  OWNDATA : False

Indeed, after a=a[:3], a is not the same Python object but still points
to the data of the first object.

What I didn't find (by quick googling) is how to access the original
array. Is it possible to access it (with Python code) ?

best,
Pierre



-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 900 bytes
Desc: OpenPGP digital signature
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20131028/6747a0b4/attachment.sig>

From robert.kern at gmail.com  Mon Oct 28 08:40:22 2013
From: robert.kern at gmail.com (Robert Kern)
Date: Mon, 28 Oct 2013 12:40:22 +0000
Subject: [Numpy-discussion] memmory management question
In-Reply-To: <526E5A7F.6030703@crans.org>
References: <526DC82F.3070606@gmail.com> <526E5A7F.6030703@crans.org>
Message-ID: <CAF6FJisDL0=W_UgZu97wxh3+ricaK68LSjdnPxPV5ge_1QYVGA@mail.gmail.com>

On Mon, Oct 28, 2013 at 12:37 PM, Pierre Haessig <pierre.haessig at crans.org>
wrote:

> What I didn't find (by quick googling) is how to access the original
> array. Is it possible to access it (with Python code) ?

a.base

--
Robert Kern
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20131028/88c80a24/attachment.html>

From pierre.haessig at crans.org  Mon Oct 28 08:44:20 2013
From: pierre.haessig at crans.org (Pierre Haessig)
Date: Mon, 28 Oct 2013 13:44:20 +0100
Subject: [Numpy-discussion] Robust Sorting of Points
In-Reply-To: <526D5B3B.7030107@witherden.org>
References: <526D5B3B.7030107@witherden.org>
Message-ID: <526E5C24.4070007@crans.org>

Hi,

Le 27/10/2013 19:28, Freddie Witherden a ?crit :
> I wish to sort these points into a canonical order in a fashion which is
> robust against small perturbations.  In other words changing any
> component of any of the points by an epsilon ~ 1e-12 should not affect
> the resulting sorted order.
Can you give more precision on what you mean by "canonical order". Since
there is no natural order in R^n for n>1, I guess your problem is more
about *defining*  what is the order you want rather than *implementing*
it in C/Python or whatever.

best,
Pierre

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 900 bytes
Desc: OpenPGP digital signature
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20131028/5b65d077/attachment.sig>

From pierre.haessig at crans.org  Mon Oct 28 09:25:45 2013
From: pierre.haessig at crans.org (Pierre Haessig)
Date: Mon, 28 Oct 2013 14:25:45 +0100
Subject: [Numpy-discussion] memmory management question
In-Reply-To: <CAF6FJisDL0=W_UgZu97wxh3+ricaK68LSjdnPxPV5ge_1QYVGA@mail.gmail.com>
References: <526DC82F.3070606@gmail.com> <526E5A7F.6030703@crans.org>
	<CAF6FJisDL0=W_UgZu97wxh3+ricaK68LSjdnPxPV5ge_1QYVGA@mail.gmail.com>
Message-ID: <526E65D9.4090505@crans.org>

Le 28/10/2013 13:40, Robert Kern a ?crit :
> > What I didn't find (by quick googling) is how to access the original
> > array. Is it possible to access it (with Python code) ?
>
> a.base
Thanks! Is there a specific paragraph I missed in the user guide ?

I had googled "numpy access original array" and  first result is
http://docs.scipy.org/doc/numpy/user/basics.indexing.html
In this document, it is mentionned several time that slicing yields
"views of the original data", but the .base attribute is not mentionned.
Should it be or is it out-of-scope of the Indexing guide ?

best,
Pierre

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 900 bytes
Desc: OpenPGP digital signature
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20131028/5420dec7/attachment.sig>

From thouis at gmail.com  Mon Oct 28 09:41:37 2013
From: thouis at gmail.com (Thouis (Ray) Jones)
Date: Mon, 28 Oct 2013 09:41:37 -0400
Subject: [Numpy-discussion] Robust Sorting of Points
In-Reply-To: <526D9693.8050704@witherden.org>
References: <526D5B3B.7030107@witherden.org>
	<CAPJVwBmiwmxPSSGY1R7Xm8kPhbr49wMDR2G=x-OOFWGyv5nAOA@mail.gmail.com>
	<526D5E9D.4000503@witherden.org> <526D614A.9030404@grinta.net>
	<526D67F9.5020405@witherden.org>
	<CAMMTP+DZ20BTMQ6CTYWYo+FFBurq351gHfnXxf3WbN31FVFKYA@mail.gmail.com>
	<526D7CC7.3080008@witherden.org>
	<CACJ8dbU-mM_NFtAXHRkWdWhAMZ63YjLOMkORtnocriOJe0L+bA@mail.gmail.com>
	<526D9693.8050704@witherden.org>
Message-ID: <CAHGWWxFBhkJ3PUNq4DXATXNMbyHHt6f5YwwwAQxJSo=Z6yt8NA@mail.gmail.com>

Always, *always*, or just with high enough probability that you don't
realistically have to worry about it failing.  If the latter, I wonder if
you could do something with random projections.  Off the top of my head, I
wonder if something like the sum of ranks when ordered under a set of
random projection directions might be stable for a large enough set of
directions.  It's been a while since my Randomized Algorithms class, though.

Ray Jones




On Sun, Oct 27, 2013 at 6:41 PM, Freddie Witherden <freddie at witherden.org>wrote:

> On 27/10/13 21:05, Jonathan March wrote:
> > If an "almost always works" solution is good enough, then sort on the
> > distance to some fixed random point that is in the vicinity of your N
> > points.
>
> I had considered this.  Unfortunately I need a solution which really
> does always work.
>
> The only pure-Python solution I can envision -- at the moment anyway --
> is to do some cleverness with the output of np.unique to identify
> similar values and replace them with an arbitrarily chosen one.  This
> should permit the output to be passed to np.lexsort without issue.
>
> Regards, Freddie.
>
>
>
>
>
> _______________________________________________
> NumPy-Discussion mailing list
> NumPy-Discussion at scipy.org
> http://mail.scipy.org/mailman/listinfo/numpy-discussion
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20131028/3a5bc361/attachment.html>

From robert.kern at gmail.com  Mon Oct 28 09:56:00 2013
From: robert.kern at gmail.com (Robert Kern)
Date: Mon, 28 Oct 2013 13:56:00 +0000
Subject: [Numpy-discussion] memmory management question
In-Reply-To: <526E65D9.4090505@crans.org>
References: <526DC82F.3070606@gmail.com> <526E5A7F.6030703@crans.org>
	<CAF6FJisDL0=W_UgZu97wxh3+ricaK68LSjdnPxPV5ge_1QYVGA@mail.gmail.com>
	<526E65D9.4090505@crans.org>
Message-ID: <CAF6FJis+z4Qgbvd_LW0RsrpSihBYqnTAzT0qgxa8vDx0KN29ag@mail.gmail.com>

On Mon, Oct 28, 2013 at 1:25 PM, Pierre Haessig <pierre.haessig at crans.org>
wrote:
>
> Le 28/10/2013 13:40, Robert Kern a ?crit :
> > > What I didn't find (by quick googling) is how to access the original
> > > array. Is it possible to access it (with Python code) ?
> >
> > a.base
> Thanks! Is there a specific paragraph I missed in the user guide ?
>
> I had googled "numpy access original array" and  first result is
> http://docs.scipy.org/doc/numpy/user/basics.indexing.html
> In this document, it is mentionned several time that slicing yields
> "views of the original data", but the .base attribute is not mentionned.
> Should it be or is it out-of-scope of the Indexing guide ?

I think it's out-of-scope for "Numpy basics", certainly. It's mostly an
implementation detail. There isn't much that you can usefully do with it as
a user.

http://docs.scipy.org/doc/numpy/reference/generated/numpy.ndarray.base.html

--
Robert Kern
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20131028/d2dbf0ee/attachment.html>

From ndbecker2 at gmail.com  Mon Oct 28 11:47:13 2013
From: ndbecker2 at gmail.com (Neal Becker)
Date: Mon, 28 Oct 2013 11:47:13 -0400
Subject: [Numpy-discussion] is np vector a sequence?
Message-ID: <l4m0tn$lv8$1@ger.gmane.org>

isinstance (np.zeros (10), collections.Sequence)
Out[36]: False

That's unfortunate.



From argriffi at ncsu.edu  Mon Oct 28 11:52:04 2013
From: argriffi at ncsu.edu (alex)
Date: Mon, 28 Oct 2013 11:52:04 -0400
Subject: [Numpy-discussion] is np vector a sequence?
In-Reply-To: <l4m0tn$lv8$1@ger.gmane.org>
References: <l4m0tn$lv8$1@ger.gmane.org>
Message-ID: <CAE5GFcJABY8CDo1wxGS-CKPC4BbWa5DJZD92=qmCE1eq8tL8ng@mail.gmail.com>

On Mon, Oct 28, 2013 at 11:47 AM, Neal Becker <ndbecker2 at gmail.com> wrote:
>
> isinstance (np.zeros (10), collections.Sequence)
> Out[36]: False
>
> That's unfortunate.

There seems to be a discussion here
https://github.com/numpy/numpy/issues/2776

Alex
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20131028/ef545f25/attachment.html>

From sgonzi at staffmail.ed.ac.uk  Mon Oct 28 12:34:18 2013
From: sgonzi at staffmail.ed.ac.uk (Siegfried Gonzi)
Date: Mon, 28 Oct 2013 16:34:18 +0000
Subject: [Numpy-discussion] Python parallel programming on Mac OS X Maverick
Message-ID: <43263CE3-B64E-49A1-811F-8D15C46D8F85@staffmail.ed.ac.uk>

Hi all

Quick question: What is the minimum RAM requirement for doing parallel programming with Python/Numpy on Mac OS X Maverick?

I am about to buy a Macbook Pro 15" and I'd like to know if 8GB RAM (with SSD flash storage) for the Haswell quad core will be enough. I have never done any parallel programming with Python/Numpy but plan to get to grips with it on my new Macbook Pro where memory from now on is being soldered on and non-replaceable.

Apple has a 14 days no quibbles refund policy but I am not sure if I can work out what I need within 14 days.


Thanks,
Siegfried
-- 
The University of Edinburgh is a charitable body, registered in
Scotland, with registration number SC005336.



From andy.terrel at gmail.com  Mon Oct 28 13:11:48 2013
From: andy.terrel at gmail.com (Andy Ray Terrel)
Date: Mon, 28 Oct 2013 12:11:48 -0500
Subject: [Numpy-discussion] Python parallel programming on Mac OS X
	Maverick
In-Reply-To: <43263CE3-B64E-49A1-811F-8D15C46D8F85@staffmail.ed.ac.uk>
References: <43263CE3-B64E-49A1-811F-8D15C46D8F85@staffmail.ed.ac.uk>
Message-ID: <CA+WonSRWs53WX_jg3ZkFT8ykJMRL_KydYTn73h9x1ud0rCx-Gw@mail.gmail.com>

Hi Siegfried,

Parallel programs, just like serial programs, will consume as much (or
as little) memory as they are required to do so.  The Python
interpreter and Numpy libraries all fit under 20 MB. So even a haswell
with its 4 cores is going to be insignificant overhead for running
Python in parallel.

-- Andy

On Mon, Oct 28, 2013 at 11:34 AM, Siegfried Gonzi
<sgonzi at staffmail.ed.ac.uk> wrote:
> Hi all
>
> Quick question: What is the minimum RAM requirement for doing parallel programming with Python/Numpy on Mac OS X Maverick?
>
> I am about to buy a Macbook Pro 15" and I'd like to know if 8GB RAM (with SSD flash storage) for the Haswell quad core will be enough. I have never done any parallel programming with Python/Numpy but plan to get to grips with it on my new Macbook Pro where memory from now on is being soldered on and non-replaceable.
>
> Apple has a 14 days no quibbles refund policy but I am not sure if I can work out what I need within 14 days.
>
>
> Thanks,
> Siegfried
> --
> The University of Edinburgh is a charitable body, registered in
> Scotland, with registration number SC005336.
>
> _______________________________________________
> NumPy-Discussion mailing list
> NumPy-Discussion at scipy.org
> http://mail.scipy.org/mailman/listinfo/numpy-discussion


From jmarch at enthought.com  Mon Oct 28 15:07:33 2013
From: jmarch at enthought.com (Jonathan March)
Date: Mon, 28 Oct 2013 14:07:33 -0500
Subject: [Numpy-discussion] Python parallel programming on Mac OS X
	Maverick
In-Reply-To: <43263CE3-B64E-49A1-811F-8D15C46D8F85@staffmail.ed.ac.uk>
References: <43263CE3-B64E-49A1-811F-8D15C46D8F85@staffmail.ed.ac.uk>
Message-ID: <CACJ8dbX4wy_KtnOmNSOYkHPG62HPmG8wN=cCD+YJpoTe7zey9w@mail.gmail.com>

Siegfried,

Re Mavericks and (I)Python:
https://github.com/ipython/ipython/issues/4431
https://mail.python.org/pipermail/python-dev/2013-October/129832.html

-- Jonathan


On Mon, Oct 28, 2013 at 11:34 AM, Siegfried Gonzi <sgonzi at staffmail.ed.ac.uk
> wrote:

> Hi all
>
> Quick question: What is the minimum RAM requirement for doing parallel
> programming with Python/Numpy on Mac OS X Maverick?
>
> I am about to buy a Macbook Pro 15" and I'd like to know if 8GB RAM (with
> SSD flash storage) for the Haswell quad core will be enough. I have never
> done any parallel programming with Python/Numpy but plan to get to grips
> with it on my new Macbook Pro where memory from now on is being soldered on
> and non-replaceable.
>
> Apple has a 14 days no quibbles refund policy but I am not sure if I can
> work out what I need within 14 days.
>
>
> Thanks,
> Siegfried
> --
> The University of Edinburgh is a charitable body, registered in
> Scotland, with registration number SC005336.
>
> _______________________________________________
> NumPy-Discussion mailing list
> NumPy-Discussion at scipy.org
> http://mail.scipy.org/mailman/listinfo/numpy-discussion
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20131028/9aef6c26/attachment.html>

From brett.olsen at gmail.com  Mon Oct 28 15:21:15 2013
From: brett.olsen at gmail.com (Brett Olsen)
Date: Mon, 28 Oct 2013 14:21:15 -0500
Subject: [Numpy-discussion] Robust Sorting of Points
In-Reply-To: <526D9693.8050704@witherden.org>
References: <526D5B3B.7030107@witherden.org>
	<CAPJVwBmiwmxPSSGY1R7Xm8kPhbr49wMDR2G=x-OOFWGyv5nAOA@mail.gmail.com>
	<526D5E9D.4000503@witherden.org> <526D614A.9030404@grinta.net>
	<526D67F9.5020405@witherden.org>
	<CAMMTP+DZ20BTMQ6CTYWYo+FFBurq351gHfnXxf3WbN31FVFKYA@mail.gmail.com>
	<526D7CC7.3080008@witherden.org>
	<CACJ8dbU-mM_NFtAXHRkWdWhAMZ63YjLOMkORtnocriOJe0L+bA@mail.gmail.com>
	<526D9693.8050704@witherden.org>
Message-ID: <CAFq1z2XWkWK=fAPgUD7_yuMpQZgvmpn9BwEXgUNcx-af9ecfAg@mail.gmail.com>

Here's some code implementing the "replace similar values with an
arbitrarily chosen one" (in this case the smallest of the similar values).
 I didn't see any way to do this cleverly with strides, so I just did a
simple loop.  It's about 100 times slower in pure Python, or a bit under 10
times slower if you're willing to use a bit of Cython.  Not sure if this is
good enough for your purposes.  I imagine you could go a bit faster if you
were willing to do the lexical integration by hand (since you've already
done the separate sorting of each subarray for value replacement purposes)
instead of passing that off to np.lexsort.

Note that this approach will only work if your points are not only
well-separated in space but also either well-separated or identical in each
dimension as well.  It's OK to have points with the same, say, x value, but
if you have points that have close x values before the noise is added, then
the noise can move intermediate points around in the sort order.  It works
well with the gridded data I used as a sample, but if you're, say,
generating random points, this could be a problem:

point 1 is (1, 0, 1e-12)
point 2 is (0, 1, 0)

These are well separated.  The algorithm will pool those z values and
report 1 as coming before 2.  Unless you get jitter like this:

point 1: (1, 0, 1.5e-12)
point 2: (0, 1, -0.5e-12)

Now they won't be pooled any more and we'll get 2 as coming before 1.

Anyway, here's the code:

In [1]:

import numpy as np
In [2]:

def gen_grid(n, d):
    #Generate a bunch of grid points, n in each dimension of spacing d
    vals = np.linspace(0, (n-1)*d, n)
    x, y, z = np.meshgrid(vals, vals, vals)
    grid = np.empty((n**3, 3))
    grid[:,0] = x.flatten()
    grid[:,1] = y.flatten()
    grid[:,2] = z.flatten()
    return grid

def jitter(array, epsilon=1e-12):
    #Add random jitter from a uniform distribution of width epsilon
    return array + np.random.random(array.shape) * epsilon - epsilon / 2
In [3]:

grid = gen_grid(4, 0.1)
print np.lexsort(grid.T)
print np.lexsort(jitter(grid.T))
[ 0  4  8 12 16 20 24 28 32 36 40 44 48 52 56 60  1  5  9 13 17 21 25 29 33
 37 41 45 49 53 57 61  2  6 10 14 18 22 26 30 34 38 42 46 50 54 58 62  3  7
 11 15 19 23 27 31 35 39 43 47 51 55 59 63]
[60  4 48 32 40 12 36 28 44 56 16  8 24  0 52 20 45 25 49  1 53 29  9 33  5
 61 41 37 17 13 21 57 22 50 18 10  2 62 58 54  6 34 26 42 38 46 14 30  3 11
 55 63 27 15 35 43 31 39  7 59 47 23 51 19]
In [4]:

def pool_values(A, epsilon=1e-12):
    idx = np.argsort(A)
    for i in range(1, len(A)):
        if A[idx[i]] - A[idx[i-1]] < epsilon:
            A[idx[i]] = A[idx[i-1]]
    return A

def stable_sort(grid):
    return np.lexsort((pool_values(grid[:,0]),
                       pool_values(grid[:,1]),
                       pool_values(grid[:,2])))
In [5]:

print stable_sort(grid)
print stable_sort(jitter(grid))
[ 0  4  8 12 16 20 24 28 32 36 40 44 48 52 56 60  1  5  9 13 17 21 25 29 33
 37 41 45 49 53 57 61  2  6 10 14 18 22 26 30 34 38 42 46 50 54 58 62  3  7
 11 15 19 23 27 31 35 39 43 47 51 55 59 63]
[ 0  4  8 12 16 20 24 28 32 36 40 44 48 52 56 60  1  5  9 13 17 21 25 29 33
 37 41 45 49 53 57 61  2  6 10 14 18 22 26 30 34 38 42 46 50 54 58 62  3  7
 11 15 19 23 27 31 35 39 43 47 51 55 59 63]
In [6]:

%timeit np.lexsort(jitter(grid.T))
100000 loops, best of 3: 10.4 ?s per loop
In [7]:

%timeit stable_sort(jitter(grid))
1000 loops, best of 3: 1.39 ms per loop
In [8]:

%load_ext cythonmagic
In [12]:

%%cython
import numpy as np
cimport numpy as np

cdef fast_pool_values(double[:] A, double epsilon=1e-12):
    cdef long[:] idx = np.argsort(A)
    cdef int i
    for i in range(1, len(A)):
        if A[idx[i]] - A[idx[i-1]] < epsilon:
            A[idx[i]] = A[idx[i-1]]
    return A

def fast_stable_sort(grid):
    return np.lexsort((fast_pool_values(grid[:,0]),
                       fast_pool_values(grid[:,1]),
                       fast_pool_values(grid[:,2])))
In [10]:

%timeit np.lexsort(jitter(grid.T))
10000 loops, best of 3: 38.5 ?s per loop
In [13]:

%timeit fast_stable_sort(jitter(grid))
1000 loops, best of 3: 309 ?s per loop


On Sun, Oct 27, 2013 at 5:41 PM, Freddie Witherden <freddie at witherden.org>wrote:

> On 27/10/13 21:05, Jonathan March wrote:
> > If an "almost always works" solution is good enough, then sort on the
> > distance to some fixed random point that is in the vicinity of your N
> > points.
>
> I had considered this.  Unfortunately I need a solution which really
> does always work.
>
> The only pure-Python solution I can envision -- at the moment anyway --
> is to do some cleverness with the output of np.unique to identify
> similar values and replace them with an arbitrarily chosen one.  This
> should permit the output to be passed to np.lexsort without issue.
>
> Regards, Freddie.
>
>
>
>
>
> _______________________________________________
> NumPy-Discussion mailing list
> NumPy-Discussion at scipy.org
> http://mail.scipy.org/mailman/listinfo/numpy-discussion
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20131028/5bb38282/attachment.html>

From freddie at witherden.org  Tue Oct 29 05:21:06 2013
From: freddie at witherden.org (Freddie Witherden)
Date: Tue, 29 Oct 2013 09:21:06 +0000
Subject: [Numpy-discussion] Robust Sorting of Points
In-Reply-To: <526E5C24.4070007@crans.org>
References: <526D5B3B.7030107@witherden.org> <526E5C24.4070007@crans.org>
Message-ID: <526F7E02.7080701@witherden.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On 28/10/2013 12:44, Pierre Haessig wrote:
> Hi,
> 
> Le 27/10/2013 19:28, Freddie Witherden a ?crit :
>> I wish to sort these points into a canonical order in a fashion
>> which is robust against small perturbations.  In other words
>> changing any component of any of the points by an epsilon ~ 1e-12
>> should not affect the resulting sorted order.
> Can you give more precision on what you mean by "canonical order".
> Since there is no natural order in R^n for n>1, I guess your
> problem is more about *defining*  what is the order you want rather
> than *implementing* it in C/Python or whatever.

The order itself does not need to satisfy any specific properties.
Any order which can be defined and implemented in a robust fashion
will do.

Regards, Freddie.

-----BEGIN PGP SIGNATURE-----
Version: GnuPG/MacGPG2 v2.0.20 (Darwin)
Comment: GPGTools - http://gpgtools.org
Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/

iEYEARECAAYFAlJvfgIACgkQ/J9EM/uoqVekVQCgkFxKxBCLUn6InBxyM995FVq3
x88AnjUSopT8YJgXUwIyKalAmLmVznvb
=nHMZ
-----END PGP SIGNATURE-----


From pierre.haessig at crans.org  Tue Oct 29 06:37:24 2013
From: pierre.haessig at crans.org (Pierre Haessig)
Date: Tue, 29 Oct 2013 11:37:24 +0100
Subject: [Numpy-discussion] Robust Sorting of Points
In-Reply-To: <526F7E02.7080701@witherden.org>
References: <526D5B3B.7030107@witherden.org> <526E5C24.4070007@crans.org>
	<526F7E02.7080701@witherden.org>
Message-ID: <526F8FE4.30309@crans.org>

Hi Freddie,

Le 29/10/2013 10:21, Freddie Witherden a ?crit :
> The order itself does not need to satisfy any specific properties.
I can't agree with you : if there is no specific property, then keeping
the list *unchanged* would be a fine solution (and very fast and very
very robust) ;-)

what about defining a comparison function (in the sense of the old cmp
keyword of list.sort) such as :

def compare(point, other):
    delta = point - other
    argmax = np.abs(delta).argmax()
    delta_max = delta[argmax]
    if delta_max > 0:
        return 1
    elif delta_max < 0:
        return -1
    else:
        return 0

This function returns a comparison of the coordinates with the  biggest
absolute difference. Of course this doesn't define an *absolute order*
(since it doesn't exist). But I think it defines a *relative order* (if
this notion exists mathematically !!!!) which is indeed robust.

To apply this comparison function, the simple solution I see (for Python
<3) is, using your first example ;

>>> my_array = np.array([[-0.5, 0, 2**0.5],
                               [0.5, 0, 2**0.5 - 1e-15]])


>>> l = list(my_array)

>>>  l.sort(cmp = compare)

>>> l
[array([-0.5       ,  0.        ,  1.41421356]),
 array([ 0.5       ,  0.        ,  1.41421356])]


Now, if this comparison function indeed answers your question, the next
step is to plug it either in numpy sorting machinery or in the new
Python sorting which uses a "key function" (cf.
http://docs.python.org/3.3/howto/sorting.html#the-old-way-using-the-cmp-parameter).

best,
Pierre

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 900 bytes
Desc: OpenPGP digital signature
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20131029/1e077b49/attachment.sig>

From pierre.haessig at crans.org  Tue Oct 29 06:58:12 2013
From: pierre.haessig at crans.org (Pierre Haessig)
Date: Tue, 29 Oct 2013 11:58:12 +0100
Subject: [Numpy-discussion] Robust Sorting of Points
In-Reply-To: <526F8FE4.30309@crans.org>
References: <526D5B3B.7030107@witherden.org> <526E5C24.4070007@crans.org>
	<526F7E02.7080701@witherden.org> <526F8FE4.30309@crans.org>
Message-ID: <526F94C4.3030901@crans.org>

Le 29/10/2013 11:37, Pierre Haessig a ?crit :
> def compare(point, other):
>     delta = point - other
>     argmax = np.abs(delta).argmax()
>     delta_max = delta[argmax]
>     if delta_max > 0:
>         return 1
>     elif delta_max < 0:
>         return -1
>     else:
>         return 0
>
> This function returns a comparison of the coordinates with the  biggest
> absolute difference. Of course this doesn't define an *absolute order*
> (since it doesn't exist). But I think it defines a *relative order* (if
> this notion exists mathematically !!!!) which is indeed robust.
In fact this comparison is not robust for points whose difference has
two coordinates of almost same absolute magnitude but of different sign :

p1 = np.array([1,-1,0])
p2 = np.array([1,-1+1e-3,0])
p3 = np.array([1,-1-1e-3,0])

compare(p1,p1)
Out[26]: 0

compare(p1,p2)
Out[27]: -1

compare(p1,p3)
Out[28]: 1

sorry,
Pierre
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20131029/e17ce2df/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 900 bytes
Desc: OpenPGP digital signature
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20131029/e17ce2df/attachment.sig>

From heng at cantab.net  Tue Oct 29 12:47:29 2013
From: heng at cantab.net (Henry Gomersall)
Date: Tue, 29 Oct 2013 16:47:29 +0000
Subject: [Numpy-discussion] getting the equivalent complex dtype from a real
	or int array
Message-ID: <526FE6A1.8050809@cantab.net>

Is there a way to extract the size of array that would be created by 
doing 1j*array?

The problem I'm having is in creating an empty array to fill with 
complex values without knowing a priori what the input data type is.

For example, I have a real or int array `a`.

I want to create an array `b` which can hold values from 1j*a in such a 
way that I don't need to compute those explicitly (because I only need 
parts of the array say), without upcasting (or indeed downcasting) the 
result.

So if `a` was dtype 'float32`, `b` would be of dtype `complex64`. If `a` 
was `int64`, `b` would be of dtype `complex128` etc.

Cheers,

Henry


From heng at cantab.net  Tue Oct 29 12:56:05 2013
From: heng at cantab.net (Henry Gomersall)
Date: Tue, 29 Oct 2013 16:56:05 +0000
Subject: [Numpy-discussion] getting the equivalent complex dtype from a
 real or int array
In-Reply-To: <526FE6A1.8050809@cantab.net>
References: <526FE6A1.8050809@cantab.net>
Message-ID: <526FE8A5.4060307@cantab.net>

On 29/10/13 16:47, Henry Gomersall wrote:
> Is there a way to extract the size of array that would be created by
> doing 1j*array?
Of course, I mean dtype of the array.

Henry


From frankliuao at gmail.com  Tue Oct 29 13:02:13 2013
From: frankliuao at gmail.com (Ao Liu)
Date: Tue, 29 Oct 2013 12:02:13 -0500
Subject: [Numpy-discussion] numpy random uniform seed?
Message-ID: <CA+uij2-rbJ=OCsza+U8U2mQ7LFWQwHtzuBBhRxRNhVGF1LCJsg@mail.gmail.com>

Hi,

I've been using np.random.uniform and mpi4py.

I found that the random number each processor (or rank) generated are the
same, so I was wondering how random.uniform chose its seeds. Theoretically,
those ranks shouldn't have anything to do with others. The only possibility
that I can think of is that they all used the computer time as the seed, so
the random numbers were the same.

Anyone has more ideas and suggestions on how to correct this? Thanks,
Best,
Ao

-- 
I am Ao.Liu, always hungry and still waiting for what I've always been
believing.

To know me more, please visit my little tiny personal website:
www.frankliuao.com
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20131029/b914d6a1/attachment.html>

From robert.kern at gmail.com  Tue Oct 29 13:02:33 2013
From: robert.kern at gmail.com (Robert Kern)
Date: Tue, 29 Oct 2013 17:02:33 +0000
Subject: [Numpy-discussion] getting the equivalent complex dtype from a
 real or int array
In-Reply-To: <526FE6A1.8050809@cantab.net>
References: <526FE6A1.8050809@cantab.net>
Message-ID: <CAF6FJiuYnDbE1Uo9J6OnL1pq+oVZX-ecqKz0Qe9MigyQT69V_g@mail.gmail.com>

On Tue, Oct 29, 2013 at 4:47 PM, Henry Gomersall <heng at cantab.net> wrote:
>
> Is there a way to extract the size of array that would be created by
> doing 1j*array?
>
> The problem I'm having is in creating an empty array to fill with
> complex values without knowing a priori what the input data type is.
>
> For example, I have a real or int array `a`.
>
> I want to create an array `b` which can hold values from 1j*a in such a
> way that I don't need to compute those explicitly (because I only need
> parts of the array say), without upcasting (or indeed downcasting) the
> result.
>
> So if `a` was dtype 'float32`, `b` would be of dtype `complex64`. If `a`
> was `int64`, `b` would be of dtype `complex128` etc.

Quick and dirty:

# Get a tiny array from `a` to test the dtype of its output when multiplied
# by a complex float. It must be an array rather than a scalar since the
# casting rules are different for array*scalar and scalar*scalar.
dt = (a.flat[:2] * 1j).dtype
b = np.empty(shape, dtype=dt)

--
Robert Kern
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20131029/8beecabd/attachment.html>

From robert.kern at gmail.com  Tue Oct 29 13:14:06 2013
From: robert.kern at gmail.com (Robert Kern)
Date: Tue, 29 Oct 2013 17:14:06 +0000
Subject: [Numpy-discussion] numpy random uniform seed?
In-Reply-To: <CA+uij2-rbJ=OCsza+U8U2mQ7LFWQwHtzuBBhRxRNhVGF1LCJsg@mail.gmail.com>
References: <CA+uij2-rbJ=OCsza+U8U2mQ7LFWQwHtzuBBhRxRNhVGF1LCJsg@mail.gmail.com>
Message-ID: <CAF6FJivcyEH-qUmq0rD4pbXLNVazZj8rpHP86S2htDEtyLu41g@mail.gmail.com>

On Tue, Oct 29, 2013 at 5:02 PM, Ao Liu <frankliuao at gmail.com> wrote:
>
> Hi,
>
> I've been using np.random.uniform and mpi4py.
>
> I found that the random number each processor (or rank) generated are the
same, so I was wondering how random.uniform chose its seeds. Theoretically,
those ranks shouldn't have anything to do with others. The only possibility
that I can think of is that they all used the computer time as the seed, so
the random numbers were the same.

numpy.random grabs random bytes from /dev/urandom on UNIX-type systems and
the equivalent Crypto API on Windows. I suspect that MPI is forking the
main process after the global RandomState instance has already been
initialized, thus copying the state into each process.

You should not use the convenience aliases in numpy.random when doing any
kind of parallel programming. Instead, you should instantiate
numpy.random.RandomState() in each subprocess and call its methods. If you
really don't care about controlling the seed, then
numpy.random.RandomState() with no arguments should be sufficient. The OS
will take care of giving each of those instantiations unique states.

--
Robert Kern
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20131029/3fd14ecb/attachment.html>

From heng at cantab.net  Tue Oct 29 13:36:11 2013
From: heng at cantab.net (Henry Gomersall)
Date: Tue, 29 Oct 2013 17:36:11 +0000
Subject: [Numpy-discussion] getting the equivalent complex dtype from a
 real or int array
In-Reply-To: <CAF6FJiuYnDbE1Uo9J6OnL1pq+oVZX-ecqKz0Qe9MigyQT69V_g@mail.gmail.com>
References: <526FE6A1.8050809@cantab.net>
	<CAF6FJiuYnDbE1Uo9J6OnL1pq+oVZX-ecqKz0Qe9MigyQT69V_g@mail.gmail.com>
Message-ID: <526FF20B.3070104@cantab.net>

On 29/10/13 17:02, Robert Kern wrote:
>
> Quick and dirty:
>
> # Get a tiny array from `a` to test the dtype of its output when 
> multiplied
> # by a complex float. It must be an array rather than a scalar since the
> # casting rules are different for array*scalar and scalar*scalar.
> dt = (a.flat[:2] * 1j).dtype
> b = np.empty(shape, dtype=dt)
Yeah, I thought about that, but it seemed a little *too* dirty.

I'm currently doing this, which is fairly elegant, but a bit wordy:

_rc_dtype_pairs = {numpy.dtype('float32'): numpy.dtype('complex64'),
         numpy.dtype('float64'): numpy.dtype('complex128'),
         numpy.dtype('longdouble'): numpy.dtype('clongdouble'),
         numpy.dtype('complex64'): numpy.dtype('float32'),
         numpy.dtype('complex128'): numpy.dtype('float64'),
         numpy.dtype('clongdouble'): numpy.dtype('longdouble')}

_default_dtype = numpy.dtype('float64')

if a.dtype not in _rc_dtype_pairs:
     complex_dtype=_rc_dtype_pairs[_default_dtype]
else:
     complex_dtype=_rc_dtype_pairs[a.dtype]

Cheers,

Henry


From sebastian at sipsolutions.net  Tue Oct 29 14:01:05 2013
From: sebastian at sipsolutions.net (Sebastian Berg)
Date: Tue, 29 Oct 2013 19:01:05 +0100
Subject: [Numpy-discussion] getting the equivalent complex dtype from a
 real or int array
In-Reply-To: <526FE6A1.8050809@cantab.net>
References: <526FE6A1.8050809@cantab.net>
Message-ID: <1383069665.11725.1.camel@sebastian-laptop>

On Tue, 2013-10-29 at 16:47 +0000, Henry Gomersall wrote:
> Is there a way to extract the size of array that would be created by 
> doing 1j*array?
> 

There is np.result_type. It does the handling of scalars as normal,
dtypes will be handled like arrays (scalars are allowed to lose
precision).

- Sebastian

> The problem I'm having is in creating an empty array to fill with 
> complex values without knowing a priori what the input data type is.
> 
> For example, I have a real or int array `a`.
> 
> I want to create an array `b` which can hold values from 1j*a in such a 
> way that I don't need to compute those explicitly (because I only need 
> parts of the array say), without upcasting (or indeed downcasting) the 
> result.
> 
> So if `a` was dtype 'float32`, `b` would be of dtype `complex64`. If `a` 
> was `int64`, `b` would be of dtype `complex128` etc.
> 
> Cheers,
> 
> Henry
> _______________________________________________
> NumPy-Discussion mailing list
> NumPy-Discussion at scipy.org
> http://mail.scipy.org/mailman/listinfo/numpy-discussion
> 




From charlesr.harris at gmail.com  Tue Oct 29 15:57:44 2013
From: charlesr.harris at gmail.com (Charles R Harris)
Date: Tue, 29 Oct 2013 13:57:44 -0600
Subject: [Numpy-discussion] 1.7.2
Message-ID: <CAB6mnxLWGPk+3ar0jgrCaWfvUTnjJVqpWqAQf1a1aDzxsyKVsw@mail.gmail.com>

Hi All,

I'm going to tag 1.7.2 soon. That is, unless someone else would like the
experience of making a release. Any volunteers?

Chuck
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20131029/461498c4/attachment.html>

From charlesr.harris at gmail.com  Tue Oct 29 16:00:00 2013
From: charlesr.harris at gmail.com (Charles R Harris)
Date: Tue, 29 Oct 2013 14:00:00 -0600
Subject: [Numpy-discussion] 1.7.2
In-Reply-To: <CAB6mnxLWGPk+3ar0jgrCaWfvUTnjJVqpWqAQf1a1aDzxsyKVsw@mail.gmail.com>
References: <CAB6mnxLWGPk+3ar0jgrCaWfvUTnjJVqpWqAQf1a1aDzxsyKVsw@mail.gmail.com>
Message-ID: <CAB6mnx+9z0ZCfdKkkf2y39V3ZoGa7r7MMWQcWY9S0C7-Vjj_hQ@mail.gmail.com>

On Tue, Oct 29, 2013 at 1:57 PM, Charles R Harris <charlesr.harris at gmail.com
> wrote:

> Hi All,
>
> I'm going to tag 1.7.2 soon. That is, unless someone else would like the
> experience of making a release. Any volunteers?
>
>
Make that 1.7.2rc1.

Chuck
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20131029/8322f355/attachment.html>

From jtaylor.debian at googlemail.com  Tue Oct 29 18:55:44 2013
From: jtaylor.debian at googlemail.com (Julian Taylor)
Date: Tue, 29 Oct 2013 23:55:44 +0100
Subject: [Numpy-discussion] 1.7.2
In-Reply-To: <CAB6mnx+9z0ZCfdKkkf2y39V3ZoGa7r7MMWQcWY9S0C7-Vjj_hQ@mail.gmail.com>
References: <CAB6mnxLWGPk+3ar0jgrCaWfvUTnjJVqpWqAQf1a1aDzxsyKVsw@mail.gmail.com>
	<CAB6mnx+9z0ZCfdKkkf2y39V3ZoGa7r7MMWQcWY9S0C7-Vjj_hQ@mail.gmail.com>
Message-ID: <52703CF0.3040605@googlemail.com>

On 29.10.2013 21:00, Charles R Harris wrote:
> 
> 
> 
> On Tue, Oct 29, 2013 at 1:57 PM, Charles R Harris
> <charlesr.harris at gmail.com <mailto:charlesr.harris at gmail.com>> wrote:
> 
>     Hi All,
> 
>     I'm going to tag 1.7.2 soon. That is, unless someone else would like
>     the experience of making a release. Any volunteers?
> 
> 
> Make that 1.7.2rc1.
> 
> Chuck
> 

I got a free day on friday so I can probably help out.


From charlesr.harris at gmail.com  Tue Oct 29 19:56:59 2013
From: charlesr.harris at gmail.com (Charles R Harris)
Date: Tue, 29 Oct 2013 17:56:59 -0600
Subject: [Numpy-discussion] 1.7.2
In-Reply-To: <52703CF0.3040605@googlemail.com>
References: <CAB6mnxLWGPk+3ar0jgrCaWfvUTnjJVqpWqAQf1a1aDzxsyKVsw@mail.gmail.com>
	<CAB6mnx+9z0ZCfdKkkf2y39V3ZoGa7r7MMWQcWY9S0C7-Vjj_hQ@mail.gmail.com>
	<52703CF0.3040605@googlemail.com>
Message-ID: <CAB6mnxJQGYbaEL5X0CRADZZMmjkAoeZ14EnaF+-kmUxkTaUX7g@mail.gmail.com>

On Tue, Oct 29, 2013 at 4:55 PM, Julian Taylor <
jtaylor.debian at googlemail.com> wrote:

> On 29.10.2013 21:00, Charles R Harris wrote:
> >
> >
> >
> > On Tue, Oct 29, 2013 at 1:57 PM, Charles R Harris
> > <charlesr.harris at gmail.com <mailto:charlesr.harris at gmail.com>> wrote:
> >
> >     Hi All,
> >
> >     I'm going to tag 1.7.2 soon. That is, unless someone else would like
> >     the experience of making a release. Any volunteers?
> >
> >
> > Make that 1.7.2rc1.
> >
> > Chuck
> >
>
> I got a free day on friday so I can probably help out.
>

Sounds good. I'll need to give you permissions on sourceforge so you can
make folders and upload files. Are you registered there? I also need to
update the HOWTO_RELEASE documentation as it is a bit sparse at the moment.
For an RC we need binaries also. Ralf can probably help with that when he
gets time, or you can try setting up to do it yourself. I haven't tried the
latter myself, but someday...

I suspect it will take longer than one day but hopefully it will be easier
than 1.8.0

Chuck
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20131029/8876299c/attachment.html>

From d.l.goldsmith at gmail.com  Tue Oct 29 22:19:34 2013
From: d.l.goldsmith at gmail.com (David Goldsmith)
Date: Tue, 29 Oct 2013 19:19:34 -0700
Subject: [Numpy-discussion] getting the equivalent complex dtype from a
 real or int array
Message-ID: <CAFtPsZpuUxczh2XvofYNMoYuBM-V=w90EV6rp5dgZJNVD+uehQ@mail.gmail.com>

We really ought to have a special page for all of Robert's little gems!

DG

On Tue, Oct 29, 2013 at 10:00 AM, <numpy-discussion-request at scipy.org>wrote:

>
> -----------------------------Message: 5
> Date: Tue, 29 Oct 2013 17:02:33 +0000
> From: Robert Kern <robert.kern at gmail.com>
> Subject: Re: [Numpy-discussion] getting the equivalent complex dtype
>         from a real or int array
> To: Discussion of Numerical Python <numpy-discussion at scipy.org>
> Message-ID:
>         <
> CAF6FJiuYnDbE1Uo9J6OnL1pq+oVZX-ecqKz0Qe9MigyQT69V_g at mail.gmail.com>
> Content-Type: text/plain; charset="utf-8"
>
> On Tue, Oct 29, 2013 at 4:47 PM, Henry Gomersall <heng at cantab.net> wrote:
> >
> > Is there a way to extract the size of array that would be created by
> > doing 1j*array?
> >
> > The problem I'm having is in creating an empty array to fill with
> > complex values without knowing a priori what the input data type is.
> >
> > For example, I have a real or int array `a`.
> >
> > I want to create an array `b` which can hold values from 1j*a in such a
> > way that I don't need to compute those explicitly (because I only need
> > parts of the array say), without upcasting (or indeed downcasting) the
> > result.
> >
> > So if `a` was dtype 'float32`, `b` would be of dtype `complex64`. If `a`
> > was `int64`, `b` would be of dtype `complex128` etc.
>
> Quick and dirty:
>
> # Get a tiny array from `a` to test the dtype of its output when multiplied
> # by a complex float. It must be an array rather than a scalar since the
> # casting rules are different for array*scalar and scalar*scalar.
> dt = (a.flat[:2] * 1j).dtype
> b = np.empty(shape, dtype=dt)
>
> --
> Robert Kern
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20131029/ff847bca/attachment.html>

From eraldo.pomponi at gmail.com  Wed Oct 30 04:45:15 2013
From: eraldo.pomponi at gmail.com (Eraldo Pomponi)
Date: Wed, 30 Oct 2013 09:45:15 +0100
Subject: [Numpy-discussion] getting the equivalent complex dtype from a
 real or int array
In-Reply-To: <CAFtPsZpuUxczh2XvofYNMoYuBM-V=w90EV6rp5dgZJNVD+uehQ@mail.gmail.com>
References: <CAFtPsZpuUxczh2XvofYNMoYuBM-V=w90EV6rp5dgZJNVD+uehQ@mail.gmail.com>
Message-ID: <CAEaCG7f4DFQVn08RxZKoAxu+YPVtN0izs86O93QytBezZwKqeg@mail.gmail.com>

> We really ought to have a special page for all of Robert's little gems!
>

I'm totally in favor or having that page. In my gmail account almost every
Robert's answer gets a star!!!
Maybe one day I'll try to put them together.

Cheers,
EP
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20131030/45984ec9/attachment.html>

From heng at cantab.net  Wed Oct 30 05:23:11 2013
From: heng at cantab.net (Henry Gomersall)
Date: Wed, 30 Oct 2013 09:23:11 +0000
Subject: [Numpy-discussion] getting the equivalent complex dtype from a
 real or int array
In-Reply-To: <1383069665.11725.1.camel@sebastian-laptop>
References: <526FE6A1.8050809@cantab.net>
	<1383069665.11725.1.camel@sebastian-laptop>
Message-ID: <5270CFFF.6050505@cantab.net>

On 29/10/13 18:01, Sebastian Berg wrote:
> On Tue, 2013-10-29 at 16:47 +0000, Henry Gomersall wrote:
>> >Is there a way to extract the size of array that would be created by
>> >doing 1j*array?
>> >
> There is np.result_type. It does the handling of scalars as normal,
> dtypes will be handled like arrays (scalars are allowed to lose
> precision).
This is clearly the correct way to do this. Many thanks.

Henry


From ralf.gommers at gmail.com  Wed Oct 30 15:00:47 2013
From: ralf.gommers at gmail.com (Ralf Gommers)
Date: Wed, 30 Oct 2013 20:00:47 +0100
Subject: [Numpy-discussion] 1.7.2
In-Reply-To: <CAB6mnxJQGYbaEL5X0CRADZZMmjkAoeZ14EnaF+-kmUxkTaUX7g@mail.gmail.com>
References: <CAB6mnxLWGPk+3ar0jgrCaWfvUTnjJVqpWqAQf1a1aDzxsyKVsw@mail.gmail.com>
	<CAB6mnx+9z0ZCfdKkkf2y39V3ZoGa7r7MMWQcWY9S0C7-Vjj_hQ@mail.gmail.com>
	<52703CF0.3040605@googlemail.com>
	<CAB6mnxJQGYbaEL5X0CRADZZMmjkAoeZ14EnaF+-kmUxkTaUX7g@mail.gmail.com>
Message-ID: <CABL7CQi_w70eOX5EhO8wmHp7h+efcKkP7qEUg5f0+w4c40dr6g@mail.gmail.com>

On Wed, Oct 30, 2013 at 12:56 AM, Charles R Harris <
charlesr.harris at gmail.com> wrote:

>
>
>
> On Tue, Oct 29, 2013 at 4:55 PM, Julian Taylor <
> jtaylor.debian at googlemail.com> wrote:
>
>> On 29.10.2013 21:00, Charles R Harris wrote:
>> >
>> >
>> >
>> > On Tue, Oct 29, 2013 at 1:57 PM, Charles R Harris
>> > <charlesr.harris at gmail.com <mailto:charlesr.harris at gmail.com>> wrote:
>> >
>> >     Hi All,
>> >
>> >     I'm going to tag 1.7.2 soon. That is, unless someone else would like
>> >     the experience of making a release. Any volunteers?
>> >
>> >
>> > Make that 1.7.2rc1.
>> >
>> > Chuck
>> >
>>
>> I got a free day on friday so I can probably help out.
>>
>
> Sounds good. I'll need to give you permissions on sourceforge so you can
> make folders and upload files. Are you registered there? I also need to
> update the HOWTO_RELEASE documentation as it is a bit sparse at the moment.
> For an RC we need binaries also. Ralf can probably help with that when he
> gets time
>

I'll be on holiday for a week starting on Friday....

Ralf


> , or you can try setting up to do it yourself. I haven't tried the latter
> myself, but someday...
>
> I suspect it will take longer than one day but hopefully it will be easier
> than 1.8.0
>
> Chuck
>
>
> _______________________________________________
> NumPy-Discussion mailing list
> NumPy-Discussion at scipy.org
> http://mail.scipy.org/mailman/listinfo/numpy-discussion
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20131030/aa33d156/attachment.html>

From tanayamitshah at gmail.com  Wed Oct 30 16:32:31 2013
From: tanayamitshah at gmail.com (Tanay Shah)
Date: Wed, 30 Oct 2013 13:32:31 -0700 (PDT)
Subject: [Numpy-discussion] timedelta64 NaT doesn't print correctly when
	printing an ndarray
Message-ID: <1383165151252-35596.post@n7.nabble.com>

When printing a timedelta64('NaT') scalar, it converts to string correctly
but when you print an ndarray where an element is timedelta64('NaT'), it
prints the element as its internal representation (-9223372036854775808)


>>> numpy.timedelta64('NaT')
numpy.timedelta64('NaT')
>>> print numpy.timedelta64('NaT')
NaT

>>> numpy.array([None], dtype=numpy.timedelta64)
array([-9223372036854775808], dtype=timedelta64)
>>> print numpy.array([None], dtype=numpy.timedelta64)
[-9223372036854775808]

Could someone suggest a method by which I can get around this to display the
NaT correctly?



--
View this message in context: http://numpy-discussion.10968.n7.nabble.com/timedelta64-NaT-doesn-t-print-correctly-when-printing-an-ndarray-tp35596.html
Sent from the Numpy-discussion mailing list archive at Nabble.com.


From charlesr.harris at gmail.com  Wed Oct 30 17:49:56 2013
From: charlesr.harris at gmail.com (Charles R Harris)
Date: Wed, 30 Oct 2013 15:49:56 -0600
Subject: [Numpy-discussion] ANN: NumPy 1.8.0 release.
Message-ID: <CAB6mnxLNKmtd2-7B3=89oi3yE_7xyw3yn7b5RpY70kQtAYL5Pw@mail.gmail.com>

I am pleased to announce the availability of NumPy 1.8.0. This release is
the culmination of over a years worth of work by the NumPy team and
contains many fixes, enhancements, and new features. Highlights are:

   - New, no 2to3, Python 2 and Python 3 are supported by a common code
   base.
   - New, gufuncs for linear algebra, enabling operations on stacked arrays.
   - New, inplace fancy indexing for ufuncs with the ``.at`` method.
   - New, ``partition`` function, partial sorting via selection for fast
   median.
   - New, ``nanmean``, ``nanvar``, and ``nanstd`` functions skipping NaNs.
   - New, ``full`` and ``full_like`` functions to create value initialized
   arrays.
   - New, ``PyUFunc_RegisterLoopForDescr``, better ufunc support for user
   dtypes.
   - Numerous performance improvements in many areas.

This release requires Python 2.6, 2.7 or 3.2-3.3, support for Python 2.4
and 2.5 has been dropped. Sources and binaries can be found at
http://sourceforge.net/projects/numpy/files/NumPy/1.8.0/.

Some 119 people contributed to this release. This is a remarkable increase
and shows that there is still life in this venerable code that had its
beginning in Numeric some 18 years ago. Many thanks to you all.

Enjoy,

Chuck


NumPy 1.8.0 Release Notes
*************************

This release supports  Python 2.6 -2.7 and 3.2 - 3.3.


Highlights
==========


* New, no 2to3, Python 2 and Python 3 are supported by a common code base.
* New, gufuncs for linear algebra, enabling operations on stacked arrays.
* New, inplace fancy indexing for ufuncs with the ``.at`` method.
* New, ``partition`` function, partial sorting via selection for fast
median.
* New, ``nanmean``, ``nanvar``, and ``nanstd`` functions skipping NaNs.
* New, ``full`` and ``full_like`` functions to create value initialized
arrays.
* New, ``PyUFunc_RegisterLoopForDescr``, better ufunc support for user
dtypes.
* Numerous performance improvements in many areas.


Dropped Support
===============


Support for Python versions 2.4 and 2.5 has been dropped,

Support for SCons has been removed.


Future Changes
==============


The Datetime64 type remains experimental in this release. In 1.9 there will
probably be some changes to make it more useable.

The diagonal method currently returns a new array and raises a
FutureWarning. In 1.9 it will return a readonly view.

Multiple field selection from a array of structured type currently
returns a new array and raises a FutureWarning. In 1.9 it will return a
readonly view.

The numpy/oldnumeric and numpy/numarray compatibility modules will be
removed in 1.9.


Compatibility notes
===================


The doc/sphinxext content has been moved into its own github repository,
and is included in numpy as a submodule. See the instructions in
doc/HOWTO_BUILD_DOCS.rst.txt for how to access the content.

.. _numpydoc: https://github.com/numpy/numpydoc

The hash function of numpy.void scalars has been changed.  Previously the
pointer to the data was hashed as an integer.  Now, the hash function uses
the tuple-hash algorithm to combine the hash functions of the elements of
the scalar, but only if the scalar is read-only.

Numpy has switched its build system to using 'separate compilation' by
default.  In previous releases this was supported, but not default. This
should produce the same results as the old system, but if you're trying to
do something complicated like link numpy statically or using an unusual
compiler, then it's possible you will encounter problems. If so, please
file a bug and as a temporary workaround you can re-enable the old build
system by exporting the shell variable NPY_SEPARATE_COMPILATION=0.

For the AdvancedNew iterator the ``oa_ndim`` flag should now be -1 to
indicate
that no ``op_axes`` and ``itershape`` are passed in. The ``oa_ndim == 0``
case, now indicates a 0-D iteration and ``op_axes`` being NULL and the old
usage is deprecated. This does not effect the ``NpyIter_New`` or
``NpyIter_MultiNew`` functions.

The functions nanargmin and nanargmax now return np.iinfo['intp'].min for
the index in all-NaN slices. Previously the functions would raise a
ValueError
for array returns and NaN for scalar returns.

NPY_RELAXED_STRIDES_CHECKING
~~~~~~~~~~~~~~~~~~~~~~~~~~~~
There is a new compile time environment variable
``NPY_RELAXED_STRIDES_CHECKING``. If this variable is set to 1, then
numpy will consider more arrays to be C- or F-contiguous -- for
example, it becomes possible to have a column vector which is
considered both C- and F-contiguous simultaneously. The new definition
is more accurate, allows for faster code that makes fewer unnecessary
copies, and simplifies numpy's code internally. However, it may also
break third-party libraries that make too-strong assumptions about the
stride values of C- and F-contiguous arrays. (It is also currently
known that this breaks Cython code using memoryviews, which will be
fixed in Cython.) THIS WILL BECOME THE DEFAULT IN A FUTURE RELEASE, SO
PLEASE TEST YOUR CODE NOW AGAINST NUMPY BUILT WITH::

  NPY_RELAXED_STRIDES_CHECKING=1 python setup.py install

You can check whether NPY_RELAXED_STRIDES_CHECKING is in effect by
running::

  np.ones((10, 1), order="C").flags.f_contiguous

This will be ``True`` if relaxed strides checking is enabled, and
``False`` otherwise. The typical problem we've seen so far is C code
that works with C-contiguous arrays, and assumes that the itemsize can
be accessed by looking at the last element in the ``PyArray_STRIDES(arr)``
array. When relaxed strides are in effect, this is not true (and in
fact, it never was true in some corner cases). Instead, use
``PyArray_ITEMSIZE(arr)``.

For more information check the "Internal memory layout of an ndarray"
section in the documentation.

Binary operations with non-arrays as second argument
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Binary operations of the form ``<array-or-subclass> * <non-array-subclass>``
where ``<non-array-subclass>`` declares an ``__array_priority__`` higher
than
that of ``<array-or-subclass>`` will now unconditionally return
*NotImplemented*, giving ``<non-array-subclass>`` a chance to handle the
operation.  Previously, `NotImplemented` would only be returned if
``<non-array-subclass>`` actually implemented the reversed operation, and
after
a (potentially expensive) array conversion of ``<non-array-subclass>`` had
been
attempted. (`bug <https://github.com/numpy/numpy/issues/3375>`_, `pull
request
<https://github.com/numpy/numpy/pull/3501>`_)

Function `median` used with `overwrite_input` only partially sorts array
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
If `median` is used with `overwrite_input` option the input array will now
only
be partially sorted instead of fully sorted.

Fix to financial.npv
~~~~~~~~~~~~~~~~~~~~
The npv function had a bug. Contrary to what the documentation stated, it
summed from indexes ``1`` to ``M`` instead of from ``0`` to ``M - 1``. The
fix changes the returned value. The mirr function called the npv function,
but worked around the problem, so that was also fixed and the return value
of the mirr function remains unchanged.

Runtime warnings when comparing NaN numbers
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Comparing ``NaN`` floating point numbers now raises the ``invalid`` runtime
warning. If a ``NaN`` is expected the warning can be ignored using
np.errstate.
E.g.::

  with np.errstate(invalid='ignore'):
      operation()


New Features
============


Support for linear algebra on stacked arrays
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
The gufunc machinery is now used for np.linalg, allowing operations on
stacked arrays and vectors. For example::

    >>> a
    array([[[ 1.,  1.],
            [ 0.,  1.]],

           [[ 1.,  1.],
            [ 0.,  1.]]])

    >>> np.linalg.inv(a)
    array([[[ 1., -1.],
            [ 0.,  1.]],

           [[ 1., -1.],
            [ 0.,  1.]]])

In place fancy indexing for ufuncs
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
The function ``at`` has been added to ufunc objects to allow in place
ufuncs with no buffering when fancy indexing is used. For example, the
following will increment the first and second items in the array, and will
increment the third item twice: ``numpy.add.at(arr, [0, 1, 2, 2], 1)``

This is what many have mistakenly thought ``arr[[0, 1, 2, 2]] += 1`` would
do,
but that does not work as the incremented value of ``arr[2]`` is simply
copied
into the third slot in ``arr`` twice, not incremented twice.

New functions `partition` and `argpartition`
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
New functions to partially sort arrays via a selection algorithm.

A ``partition`` by index ``k`` moves the ``k`` smallest element to the
front of
an array.  All elements before ``k`` are then smaller or equal than the
value
in position ``k`` and all elements following ``k`` are then greater or equal
than the value in position ``k``. The ordering of the values within these
bounds is undefined.
A sequence of indices can be provided to sort all of them into their sorted
position at once iterative partitioning.
This can be used to efficiently obtain order statistics like median or
percentiles of samples.
``partition`` has a linear time complexity of ``O(n)`` while a full sort has
``O(n log(n))``.

New functions `nanmean`, `nanvar` and `nanstd`
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
New nan aware statistical functions are added. In these functions the
results are what would be obtained if nan values were ommited from all
computations.

New functions `full` and `full_like`
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
New convenience functions to create arrays filled with a specific value;
complementary to the existing `zeros` and `zeros_like` functions.

IO compatibility with large files
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Large NPZ files >2GB can be loaded on 64-bit systems.

Building against OpenBLAS
~~~~~~~~~~~~~~~~~~~~~~~~~
It is now possible to build numpy against OpenBLAS by editing site.cfg.

New constant
~~~~~~~~~~~~
Euler's constant is now exposed in numpy as euler_gamma.

New modes for qr
~~~~~~~~~~~~~~~~
New modes 'complete', 'reduced', and 'raw' have been added to the qr
factorization and the old 'full' and 'economic' modes are deprecated.
The 'reduced' mode replaces the old 'full' mode and is the default as was
the 'full' mode, so backward compatibility can be maintained by not
specifying the mode.

The 'complete' mode returns a full dimensional factorization, which can be
useful for obtaining a basis for the orthogonal complement of the range
space. The 'raw' mode returns arrays that contain the Householder
reflectors and scaling factors that can be used in the future to apply q
without needing to convert to a matrix. The 'economic' mode is simply
deprecated, there isn't much use for it and it isn't any more efficient
than the 'raw' mode.

New `invert` argument to `in1d`
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
The function `in1d` now accepts a `invert` argument which, when `True`,
causes the returned array to be inverted.

Advanced indexing using `np.newaxis`
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
It is now possible to use `np.newaxis`/`None` together with index
arrays instead of only in simple indices. This means that
``array[np.newaxis, [0, 1]]`` will now work as expected and select the first
two rows while prepending a new axis to the array.


C-API
~~~~~
New ufuncs can now be registered with builtin input types and a custom
output type. Before this change, NumPy wouldn't be able to find the right
ufunc loop function when the ufunc was called from Python, because the ufunc
loop signature matching logic wasn't looking at the output operand type.
Now the correct ufunc loop is found, as long as the user provides an output
argument with the correct output type.

runtests.py
~~~~~~~~~~~
A simple test runner script ``runtests.py`` was added. It also builds Numpy
via
``setup.py build`` and can be used to run tests easily during development.


Improvements
============

IO performance improvements
~~~~~~~~~~~~~~~~~~~~~~~~~~~
Performance in reading large files was improved by chunking (see also IO
compatibility).

Performance improvements to `pad`
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
The `pad` function has a new implementation, greatly improving performance
for
all inputs except `mode=<function>` (retained for backwards compatibility).
Scaling with dimensionality is dramatically improved for rank >= 4.

Performance improvements to `isnan`, `isinf`, `isfinite` and `byteswap`
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
`isnan`, `isinf`, `isfinite` and `byteswap` have been improved to take
advantage of compiler builtins to avoid expensive calls to libc.
This improves performance of these operations by about a factor of two on
gnu
libc systems.

Performance improvements via SSE2 vectorization
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Several functions have been optimized to make use of SSE2 CPU SIMD
instructions.

* Float32 and float64:
    * base math (`add`, `subtract`, `divide`, `multiply`)
    * `sqrt`
    * `minimum/maximum`
    * `absolute`
* Bool:
    * `logical_or`
    * `logical_and`
    * `logical_not`

This improves performance of these operations up to 4x/2x for
float32/float64
and up to 10x for bool depending on the location of the data in the CPU
caches.
The performance gain is greatest for in-place operations.

In order to use the improved functions the SSE2 instruction set must be
enabled
at compile time. It is enabled by default on x86_64 systems. On x86_32 with
a
capable CPU it must be enabled by passing the appropriate flag to the CFLAGS
build variable (-msse2 with gcc).

Performance improvements to `median`
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
`median` is now implemented in terms of `partition` instead of `sort` which
reduces its time complexity from O(n log(n)) to O(n).
If used with the `overwrite_input` option the array will now only be
partially
sorted instead of fully sorted.


Overrideable operand flags in ufunc C-API
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
When creating a ufunc, the default ufunc operand flags can be overridden
via the new op_flags attribute of the ufunc object. For example, to set
the operand flag for the first input to read/write:

PyObject \*ufunc = PyUFunc_FromFuncAndData(...);
ufunc->op_flags[0] = NPY_ITER_READWRITE;

This allows a ufunc to perform an operation in place. Also, global nditer
flags
can be overridden via the new iter_flags attribute of the ufunc object.
For example, to set the reduce flag for a ufunc:

ufunc->iter_flags = NPY_ITER_REDUCE_OK;


Changes
=======


General
~~~~~~~
The function np.take now allows 0-d arrays as indices.

The separate compilation mode is now enabled by default.

Several changes to np.insert and np.delete:

* Previously, negative indices and indices that pointed past the end of
  the array were simply ignored. Now, this will raise a Future or
Deprecation
  Warning. In the future they will be treated like normal indexing treats
  them -- negative indices will wrap around, and out-of-bound indices will
  generate an error.
* Previously, boolean indices were treated as if they were integers (always
  referring to either the 0th or 1st item in the array). In the future, they
  will be treated as masks. In this release, they raise a FutureWarning
  warning of this coming change.
* In Numpy 1.7. np.insert already allowed the syntax
  `np.insert(arr, 3, [1,2,3])` to insert multiple items at a single
position.
  In Numpy 1.8. this is also possible for `np.insert(arr, [3], [1, 2, 3])`.

Padded regions from np.pad are now correctly rounded, not truncated.

C-API Array Additions
~~~~~~~~~~~~~~~~~~~~~
Four new functions have been added to the array C-API.

* PyArray_Partition
* PyArray_ArgPartition
* PyArray_SelectkindConverter
* PyDataMem_NEW_ZEROED

C-API Ufunc Additions
~~~~~~~~~~~~~~~~~~~~~
One new function has been added to the ufunc C-API that allows to register
an inner loop for user types using the descr.

* PyUFunc_RegisterLoopForDescr

Deprecations
============

The 'full' and 'economic' modes of qr factorization are deprecated.

General
~~~~~~~
The use of non-integer for indices and most integer arguments has been
deprecated. Previously float indices and function arguments such as axes or
shapes were truncated to integers without warning. For example
`arr.reshape(3., -1)` or `arr[0.]` will trigger a deprecation warning in
NumPy 1.8., and in some future version of NumPy they will raise an error.


Authors
=======

This release contains work by the following people who contributed at least
one patch to this release. The names are in alphabetical order by first
name:

* 87
* Adam Ginsburg +
* Adam Griffiths +
* Alexander Belopolsky +
* Alex Barth +
* Alex Ford +
* Andreas Hilboll +
* Andreas Kloeckner +
* Andreas Schwab +
* Andrew Horton +
* argriffing +
* Arink Verma +
* Bago Amirbekian +
* Bartosz Telenczuk +
* bebert218 +
* Benjamin Root +
* Bill Spotz +
* Bradley M. Froehle
* Carwyn Pelley +
* Charles Harris
* Chris
* Christian Brueffer +
* Christoph Dann +
* Christoph Gohlke
* Dan Hipschman +
* Daniel +
* Dan Miller +
* daveydave400 +
* David Cournapeau
* David Warde-Farley
* Denis Laxalde
* dmuellner +
* Edward Catmur +
* Egor Zindy +
* endolith
* Eric Firing
* Eric Fode
* Eric Moore +
* Eric Price +
* Fazlul Shahriar +
* F?lix Hartmann +
* Fernando Perez
* Frank B +
* Frank Breitling +
* Frederic
* Gabriel
* GaelVaroquaux
* Guillaume Gay +
* Han Genuit
* HaroldMills +
* hklemm +
* jamestwebber +
* Jason Madden +
* Jay Bourque
* jeromekelleher +
* Jes?s G?mez +
* jmozmoz +
* jnothman +
* Johannes Sch?nberger +
* John Benediktsson +
* John Salvatier +
* John Stechschulte +
* Jonathan Waltman +
* Joon Ro +
* Jos de Kloe +
* Joseph Martinot-Lagarde +
* Josh Warner (Mac) +
* Jostein B? Fl?ystad +
* Juan Luis Cano Rodr?guez +
* Julian Taylor +
* Julien Phalip +
* K.-Michael Aye +
* Kumar Appaiah +
* Lars Buitinck
* Leon Weber +
* Luis Pedro Coelho
* Marcin Juszkiewicz
* Mark Wiebe
* Marten van Kerkwijk +
* Martin Baeuml +
* Martin Spacek
* Martin Teichmann +
* Matt Davis +
* Matthew Brett
* Maximilian Albert +
* m-d-w +
* Michael Droettboom
* mwtoews +
* Nathaniel J. Smith
* Nicolas Scheffer +
* Nils Werner +
* ochoadavid +
* Ond?ej ?ert?k
* ovillellas +
* Paul Ivanov
* Pauli Virtanen
* peterjc
* Ralf Gommers
* Raul Cota +
* Richard Hattersley +
* Robert Costa +
* Robert Kern
* Rob Ruana +
* Ronan Lamy
* Sandro Tosi
* Sascha Peilicke +
* Sebastian Berg
* Skipper Seabold
* Stefan van der Walt
* Steve +
* Takafumi Arakaki +
* Thomas Robitaille +
* Tomas Tomecek +
* Travis E. Oliphant
* Valentin Haenel
* Vladimir Rutsky +
* Warren Weckesser
* Yaroslav Halchenko
* Yury V. Zaytsev +

A total of 119 people contributed to this release.
People with a "+" by their names contributed a patch for the first time.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20131030/39e4d860/attachment.html>

From irving at naml.us  Wed Oct 30 20:52:15 2013
From: irving at naml.us (Geoffrey Irving)
Date: Wed, 30 Oct 2013 17:52:15 -0700
Subject: [Numpy-discussion] char with native integer signedness
Message-ID: <CAJ1ofpfLBwSh-r=9n=-MbYpw_CmLEeQCd-9kavEgmXPwSsYTtw@mail.gmail.com>

Is there a standard way in numpy of getting a char with C-native
integer signedness?  I.e.,

    boost::is_signed<char>::value ? numpy.byte : numpy.ubyte

but without nonsensical mixing of languages?

Thanks,
Geoffrey


From robert.kern at gmail.com  Thu Oct 31 05:08:01 2013
From: robert.kern at gmail.com (Robert Kern)
Date: Thu, 31 Oct 2013 09:08:01 +0000
Subject: [Numpy-discussion] char with native integer signedness
In-Reply-To: <CAJ1ofpfLBwSh-r=9n=-MbYpw_CmLEeQCd-9kavEgmXPwSsYTtw@mail.gmail.com>
References: <CAJ1ofpfLBwSh-r=9n=-MbYpw_CmLEeQCd-9kavEgmXPwSsYTtw@mail.gmail.com>
Message-ID: <CAF6FJiv=wUN-kxENxLjYrWnBee4nUt9+n9-mCPQ1EjuQu2M8Yg@mail.gmail.com>

On Thu, Oct 31, 2013 at 12:52 AM, Geoffrey Irving <irving at naml.us> wrote:
>
> Is there a standard way in numpy of getting a char with C-native
> integer signedness?  I.e.,
>
>     boost::is_signed<char>::value ? numpy.byte : numpy.ubyte
>
> but without nonsensical mixing of languages?

This is for interop with a C/C++ extension, right? Do this test in that
extension's C/C++ code to expose the right dtype. As far as I know, this is
not something determined by the hardware, but the compiler used. Since the
compiler of numpy may be different from your extension, only your extension
can do that test properly.

--
Robert Kern
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20131031/a2d05350/attachment.html>

From ndbecker2 at gmail.com  Thu Oct 31 08:58:36 2013
From: ndbecker2 at gmail.com (Neal Becker)
Date: Thu, 31 Oct 2013 08:58:36 -0400
Subject: [Numpy-discussion] ANN: NumPy 1.8.0 release.
References: <CAB6mnxLNKmtd2-7B3=89oi3yE_7xyw3yn7b5RpY70kQtAYL5Pw@mail.gmail.com>
Message-ID: <l4tk5i$j44$2@ger.gmane.org>

Thanks for the release!

I am having a hard time finding the build instructions.  Could you please add 
this to the announcement?



From charlesr.harris at gmail.com  Thu Oct 31 09:10:05 2013
From: charlesr.harris at gmail.com (Charles R Harris)
Date: Thu, 31 Oct 2013 07:10:05 -0600
Subject: [Numpy-discussion] ANN: NumPy 1.8.0 release.
In-Reply-To: <l4tk5i$j44$2@ger.gmane.org>
References: <CAB6mnxLNKmtd2-7B3=89oi3yE_7xyw3yn7b5RpY70kQtAYL5Pw@mail.gmail.com>
	<l4tk5i$j44$2@ger.gmane.org>
Message-ID: <CAB6mnxJFga=bXikMf-D0bNXSgcboL5RiArcKN3tE=rVDfwB9EQ@mail.gmail.com>

On Thu, Oct 31, 2013 at 6:58 AM, Neal Becker <ndbecker2 at gmail.com> wrote:

> Thanks for the release!
>
> I am having a hard time finding the build instructions.  Could you please
> add
> this to the announcement?
>

What sort of build instructions are you looking for?

Chuck
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20131031/ff43a984/attachment.html>

From ndbecker2 at gmail.com  Thu Oct 31 09:13:41 2013
From: ndbecker2 at gmail.com (Neal Becker)
Date: Thu, 31 Oct 2013 09:13:41 -0400
Subject: [Numpy-discussion] ANN: NumPy 1.8.0 release.
References: <CAB6mnxLNKmtd2-7B3=89oi3yE_7xyw3yn7b5RpY70kQtAYL5Pw@mail.gmail.com>
	<l4tk5i$j44$2@ger.gmane.org>
	<CAB6mnxJFga=bXikMf-D0bNXSgcboL5RiArcKN3tE=rVDfwB9EQ@mail.gmail.com>
Message-ID: <l4tl1s$s1k$1@ger.gmane.org>

Charles R Harris wrote:

> On Thu, Oct 31, 2013 at 6:58 AM, Neal Becker <ndbecker2 at gmail.com> wrote:
> 
>> Thanks for the release!
>>
>> I am having a hard time finding the build instructions.  Could you please
>> add
>> this to the announcement?
>>
> 
> What sort of build instructions are you looking for?
> 
> Chuck

How to build from source, what are some settings for site.cfg.  I did get this 
figured out (wanted to try out openblas), but it could be a small barrier to 
new users.



From charlesr.harris at gmail.com  Thu Oct 31 09:26:55 2013
From: charlesr.harris at gmail.com (Charles R Harris)
Date: Thu, 31 Oct 2013 07:26:55 -0600
Subject: [Numpy-discussion] ANN: NumPy 1.8.0 release.
In-Reply-To: <l4tl1s$s1k$1@ger.gmane.org>
References: <CAB6mnxLNKmtd2-7B3=89oi3yE_7xyw3yn7b5RpY70kQtAYL5Pw@mail.gmail.com>
	<l4tk5i$j44$2@ger.gmane.org>
	<CAB6mnxJFga=bXikMf-D0bNXSgcboL5RiArcKN3tE=rVDfwB9EQ@mail.gmail.com>
	<l4tl1s$s1k$1@ger.gmane.org>
Message-ID: <CAB6mnxJ7TOqQzoL-XvyqvRa+TdK+=TU_hqX7QrEDxqkZwUHgOg@mail.gmail.com>

On Thu, Oct 31, 2013 at 7:13 AM, Neal Becker <ndbecker2 at gmail.com> wrote:

> Charles R Harris wrote:
>
> > On Thu, Oct 31, 2013 at 6:58 AM, Neal Becker <ndbecker2 at gmail.com>
> wrote:
> >
> >> Thanks for the release!
> >>
> >> I am having a hard time finding the build instructions.  Could you
> please
> >> add
> >> this to the announcement?
> >>
> >
> > What sort of build instructions are you looking for?
> >
> > Chuck
>
> How to build from source, what are some settings for site.cfg.  I did get
> this
> figured out (wanted to try out openblas), but it could be a small barrier
> to
> new users.
>
>
That should be explained in INSTALL.txt. It may be a bit outdated at this
point. If so, could you make a PR adding relevant bits from your experience.

Chuck
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20131031/d14ad915/attachment.html>

From chris.barker at noaa.gov  Thu Oct 31 11:52:46 2013
From: chris.barker at noaa.gov (Chris Barker)
Date: Thu, 31 Oct 2013 08:52:46 -0700
Subject: [Numpy-discussion] memmory management question
In-Reply-To: <526E65D9.4090505@crans.org>
References: <526DC82F.3070606@gmail.com> <526E5A7F.6030703@crans.org>
	<CAF6FJisDL0=W_UgZu97wxh3+ricaK68LSjdnPxPV5ge_1QYVGA@mail.gmail.com>
	<526E65D9.4090505@crans.org>
Message-ID: <CALGmxE+V7gHGLFDetLXyxGmMZLasFe=NdDhHtxrNvudmr9mEAA@mail.gmail.com>

On Mon, Oct 28, 2013 at 6:25 AM, Pierre Haessig <pierre.haessig at crans.org>wrote:


> > a.base
>


> In this document, it is mentionned several time that slicing yields
> "views of the original data", but the .base attribute is not mentionned.
> Should it be or is it out-of-scope of the Indexing guide ?
>

Indeed, that is not the least bit "basic" numpy usage -- the "basic way to
do what you want is to keep the original array around in the first place --
i.e. give the slice a different name.

-Chris


-- 

Christopher Barker, Ph.D.
Oceanographer

Emergency Response Division
NOAA/NOS/OR&R            (206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115       (206) 526-6317   main reception

Chris.Barker at noaa.gov
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20131031/d50506ed/attachment.html>

From irving at naml.us  Thu Oct 31 12:19:41 2013
From: irving at naml.us (Geoffrey Irving)
Date: Thu, 31 Oct 2013 09:19:41 -0700
Subject: [Numpy-discussion] char with native integer signedness
In-Reply-To: <CAF6FJiv=wUN-kxENxLjYrWnBee4nUt9+n9-mCPQ1EjuQu2M8Yg@mail.gmail.com>
References: <CAJ1ofpfLBwSh-r=9n=-MbYpw_CmLEeQCd-9kavEgmXPwSsYTtw@mail.gmail.com>
	<CAF6FJiv=wUN-kxENxLjYrWnBee4nUt9+n9-mCPQ1EjuQu2M8Yg@mail.gmail.com>
Message-ID: <CAJ1ofpfPANaXxhVP8qLx5XB4289jkEh95GdwAXKbwPv7fMsxtw@mail.gmail.com>

On Thu, Oct 31, 2013 at 2:08 AM, Robert Kern <robert.kern at gmail.com> wrote:
> On Thu, Oct 31, 2013 at 12:52 AM, Geoffrey Irving <irving at naml.us> wrote:
>>
>> Is there a standard way in numpy of getting a char with C-native
>> integer signedness?  I.e.,
>>
>>     boost::is_signed<char>::value ? numpy.byte : numpy.ubyte
>>
>> but without nonsensical mixing of languages?
>
> This is for interop with a C/C++ extension, right? Do this test in that
> extension's C/C++ code to expose the right dtype. As far as I know, this is
> not something determined by the hardware, but the compiler used. Since the
> compiler of numpy may be different from your extension, only your extension
> can do that test properly.

It's not determined by the hardware, but I believe it is standardized
by each platform's ABI even if it can be adjusted by the compiler.
>From the gcc man page:

       -funsigned-char
           Let the type "char" be unsigned, like "unsigned char".

           Each kind of machine has a default for what "char" should
be.  It is either like "unsigned char" by
           default or like "signed char" by default.

           Ideally, a portable program should always use "signed char"
or "unsigned char" when it depends on the
           signedness of an object.  But many programs have been
written to use plain "char" and expect it to be
           signed, or expect it to be unsigned, depending on the
machines they were written for.  This option, and
           its inverse, let you make such a program work with the
opposite default.

           The type "char" is always a distinct type from each of
"signed char" or "unsigned char", even though its
           behavior is always just like one of those two.

Geoffrey


From robert.kern at gmail.com  Thu Oct 31 12:31:48 2013
From: robert.kern at gmail.com (Robert Kern)
Date: Thu, 31 Oct 2013 16:31:48 +0000
Subject: [Numpy-discussion] char with native integer signedness
In-Reply-To: <CAJ1ofpfPANaXxhVP8qLx5XB4289jkEh95GdwAXKbwPv7fMsxtw@mail.gmail.com>
References: <CAJ1ofpfLBwSh-r=9n=-MbYpw_CmLEeQCd-9kavEgmXPwSsYTtw@mail.gmail.com>
	<CAF6FJiv=wUN-kxENxLjYrWnBee4nUt9+n9-mCPQ1EjuQu2M8Yg@mail.gmail.com>
	<CAJ1ofpfPANaXxhVP8qLx5XB4289jkEh95GdwAXKbwPv7fMsxtw@mail.gmail.com>
Message-ID: <CAF6FJiuUw7wjw9O3rWsv-Sx26A7Tx4--SuGs3iA8PogLAbx6Xw@mail.gmail.com>

On Thu, Oct 31, 2013 at 4:19 PM, Geoffrey Irving <irving at naml.us> wrote:
>
> On Thu, Oct 31, 2013 at 2:08 AM, Robert Kern <robert.kern at gmail.com>
wrote:
> > On Thu, Oct 31, 2013 at 12:52 AM, Geoffrey Irving <irving at naml.us>
wrote:
> >>
> >> Is there a standard way in numpy of getting a char with C-native
> >> integer signedness?  I.e.,
> >>
> >>     boost::is_signed<char>::value ? numpy.byte : numpy.ubyte
> >>
> >> but without nonsensical mixing of languages?
> >
> > This is for interop with a C/C++ extension, right? Do this test in that
> > extension's C/C++ code to expose the right dtype. As far as I know,
this is
> > not something determined by the hardware, but the compiler used. Since
the
> > compiler of numpy may be different from your extension, only your
extension
> > can do that test properly.
>
> It's not determined by the hardware, but I believe it is standardized
> by each platform's ABI even if it can be adjusted by the compiler.
> >From the gcc man page:
>
>        -funsigned-char
>            Let the type "char" be unsigned, like "unsigned char".

Fair enough. numpy doesn't distinguish between these cases as it only uses
plain 'char' for 'S' arrays, which don't really care about the numerical
value assigned the bits. It explicitly uses 'signed char' elsewhere, so
this platform setting isn't relevant to it. Consequently, numpy also
doesn't expose this platform setting. I think I stand by my recommendation.

--
Robert Kern
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20131031/5ccf31e4/attachment.html>

From dineshbvadhia at hotmail.com  Thu Oct 31 15:28:59 2013
From: dineshbvadhia at hotmail.com (Dinesh Vadhia)
Date: Thu, 31 Oct 2013 12:28:59 -0700
Subject: [Numpy-discussion] ANN: NumPy 1.8.0 release.
In-Reply-To: <CAB6mnxJ7TOqQzoL-XvyqvRa+TdK+=TU_hqX7QrEDxqkZwUHgOg@mail.gmail.com>
References: <CAB6mnxLNKmtd2-7B3=89oi3yE_7xyw3yn7b5RpY70kQtAYL5Pw@mail.gmail.com><l4tk5i$j44$2@ger.gmane.org><CAB6mnxJFga=bXikMf-D0bNXSgcboL5RiArcKN3tE=rVDfwB9EQ@mail.gmail.com><l4tl1s$s1k$1@ger.gmane.org>
	<CAB6mnxJ7TOqQzoL-XvyqvRa+TdK+=TU_hqX7QrEDxqkZwUHgOg@mail.gmail.com>
Message-ID: <BAY173-DS189109B23BF6A129E68075A30B0@phx.gbl>

Use site.cfg.example as template to create a new site.cfg.  For openblas, uncomment:

[openblas]
library_dirs = /opt/OpenBLAS/lib
include_dirs = /opt/OpenBLAS/include

Also, uncomment default section:

[DEFAULT]
library_dirs = /usr/local/lib
include_dirs = /usr/local/include

That should do it - hopefully.





 
On Thu, Oct 31, 2013 at 7:13 AM, Neal Becker <ndbecker2 at gmail.com> wrote:

  Charles R Harris wrote:

  > On Thu, Oct 31, 2013 at 6:58 AM, Neal Becker <ndbecker2 at gmail.com> wrote:
  >
  >> Thanks for the release!
  >>
  >> I am having a hard time finding the build instructions.  Could you please
  >> add
  >> this to the announcement?
  >>
  >
  > What sort of build instructions are you looking for?
  >
  > Chuck


  How to build from source, what are some settings for site.cfg.  I did get this
  figured out (wanted to try out openblas), but it could be a small barrier to
  new users.





That should be explained in INSTALL.txt. It may be a bit outdated at this point. If so, could you make a PR adding relevant bits from your experience.


Chuck


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20131031/c4d7f42e/attachment.html>

From dineshbvadhia at hotmail.com  Thu Oct 31 15:28:59 2013
From: dineshbvadhia at hotmail.com (Dinesh Vadhia)
Date: Thu, 31 Oct 2013 12:28:59 -0700
Subject: [Numpy-discussion] ANN: NumPy 1.8.0 release.
In-Reply-To: <CAB6mnxJ7TOqQzoL-XvyqvRa+TdK+=TU_hqX7QrEDxqkZwUHgOg@mail.gmail.com>
References: <CAB6mnxLNKmtd2-7B3=89oi3yE_7xyw3yn7b5RpY70kQtAYL5Pw@mail.gmail.com><l4tk5i$j44$2@ger.gmane.org><CAB6mnxJFga=bXikMf-D0bNXSgcboL5RiArcKN3tE=rVDfwB9EQ@mail.gmail.com><l4tl1s$s1k$1@ger.gmane.org>
	<CAB6mnxJ7TOqQzoL-XvyqvRa+TdK+=TU_hqX7QrEDxqkZwUHgOg@mail.gmail.com>
Message-ID: <BAY173-DS21D01A3DDA7B8022FF7F85A30B0@phx.gbl>

Use site.cfg.example as template to create a new site.cfg.  For openblas, uncomment:

[openblas]
library_dirs = /opt/OpenBLAS/lib
include_dirs = /opt/OpenBLAS/include

Also, uncomment default section:

[DEFAULT]
library_dirs = /usr/local/lib
include_dirs = /usr/local/include

That should do it - hopefully.





 
On Thu, Oct 31, 2013 at 7:13 AM, Neal Becker <ndbecker2 at gmail.com> wrote:

  Charles R Harris wrote:

  > On Thu, Oct 31, 2013 at 6:58 AM, Neal Becker <ndbecker2 at gmail.com> wrote:
  >
  >> Thanks for the release!
  >>
  >> I am having a hard time finding the build instructions.  Could you please
  >> add
  >> this to the announcement?
  >>
  >
  > What sort of build instructions are you looking for?
  >
  > Chuck


  How to build from source, what are some settings for site.cfg.  I did get this
  figured out (wanted to try out openblas), but it could be a small barrier to
  new users.





That should be explained in INSTALL.txt. It may be a bit outdated at this point. If so, could you make a PR adding relevant bits from your experience.


Chuck


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20131031/9bb1e4d9/attachment.html>

From ndbecker2 at gmail.com  Thu Oct 31 19:22:32 2013
From: ndbecker2 at gmail.com (Neal Becker)
Date: Thu, 31 Oct 2013 19:22:32 -0400
Subject: [Numpy-discussion] strange behavior of += with object array
Message-ID: <l4uonf$jeg$1@ger.gmane.org>

import numpy as np
#from accumulator import stat2nd_double                                                                                     

## Just to make this really clear, I'm making a dummy
## class here that overloads +=
class stat2nd_double (object):
    def __iadd__ (self, x):
        return self

m = np.empty ((2,3), dtype=object)
m[:,:] = stat2nd_double()

m[0,0] += 1.0  <<<< no error here

m += np.ones ((2,3)) <<< but this gives an error

Traceback (most recent call last):
  File "test_stat.py", line 13, in <module>
    m += np.ones ((2,3))
TypeError: unsupported operand type(s) for +: 'stat2nd_double' and 'float'