From matthew.brett at gmail.com  Wed May  1 00:16:38 2013
From: matthew.brett at gmail.com (Matthew Brett)
Date: Tue, 30 Apr 2013 21:16:38 -0700
Subject: [Numpy-discussion] could anyone check on a 32bit system?
In-Reply-To: <20130501030849.GO5140@onerussian.com>
References: <CANNq6F=NCdd1cRWmp7rskr6NJt3sjVvinN1XQVcXjJRgiGk=_A@mail.gmail.com>
	<20130501030849.GO5140@onerussian.com>
Message-ID: <CAH6Pt5ryQQX6jqSUQNH_416RXFD2+1dGPV1rRn9DK-ZvZkC9Zg@mail.gmail.com>

Hi,

On Tue, Apr 30, 2013 at 8:08 PM, Yaroslav Halchenko
<lists at onerussian.com> wrote:
> could anyone on 32bit system with fresh numpy (1.7.1) test following:
>
>> wget -nc http://www.onerussian.com/tmp/data.npy ; python -c 'import numpy as np; data1 = np.load("/tmp/data.npy"); print  np.sum(data1[1,:,0,1]) - np.sum(data1, axis=1)[1,0,1]'
>
> 0.0
>
> because unfortunately it seems on fresh ubuntu raring (in 32bit build only,
> seems ok in 64 bit... also never ran into it on older numpy releases):
>
>> python -c 'import numpy as np; data1 = np.load("/tmp/data.npy"); print  np.sum(data1[1,:,0,1]) - np.sum(data1, axis=1)[1,0,1]'
> -1.11022302463e-16
>
> PS detected by failed tests of pymvpa

Reduced case on numpy 1.7.1, 32-bit Ubuntu 12.04.2

In [64]: data = np.array([[ 0.49505185,  0.47212842],
       [ 0.53529587,  0.04366172],
       [-0.13461665, -0.01664215]])

In [65]: np.sum(data[:, 0]) - np.sum(data, axis=0)[0]
Out[65]: 1.1102230246251565e-16

No difference for single vector:

In [4]: data1 = data[:, 0:1]

In [5]: np.sum(data1[:, 0]) - np.sum(data1, axis=0)[0]
Out[5]: 0.0

Puzzling to me...

Cheers,

Matthew


From chris.barker at noaa.gov  Wed May  1 01:13:53 2013
From: chris.barker at noaa.gov (Chris Barker - NOAA Federal)
Date: Tue, 30 Apr 2013 22:13:53 -0700
Subject: [Numpy-discussion] nanmean(),
 nanstd() and other "missing" functions for 1.8
In-Reply-To: <CANNq6F=NCdd1cRWmp7rskr6NJt3sjVvinN1XQVcXjJRgiGk=_A@mail.gmail.com>
References: <CANNq6F=NCdd1cRWmp7rskr6NJt3sjVvinN1XQVcXjJRgiGk=_A@mail.gmail.com>
Message-ID: <1888613198476891170@unknownmsgid>

On Apr 30, 2013, at 6:37 PM, Benjamin Root <ben.root at ou.edu> wrote:
> I can not think of any reason not to include these functions in v1.8.

+1


> Of course, the documentation for discussed before: np.minmax().  My thinking is that it would return a 2xN array

How about a tuple: (min, max)?

-Chris


From matthew.brett at gmail.com  Wed May  1 01:20:33 2013
From: matthew.brett at gmail.com (Matthew Brett)
Date: Tue, 30 Apr 2013 22:20:33 -0700
Subject: [Numpy-discussion] could anyone check on a 32bit system?
In-Reply-To: <CAH6Pt5ryQQX6jqSUQNH_416RXFD2+1dGPV1rRn9DK-ZvZkC9Zg@mail.gmail.com>
References: <CANNq6F=NCdd1cRWmp7rskr6NJt3sjVvinN1XQVcXjJRgiGk=_A@mail.gmail.com>
	<20130501030849.GO5140@onerussian.com>
	<CAH6Pt5ryQQX6jqSUQNH_416RXFD2+1dGPV1rRn9DK-ZvZkC9Zg@mail.gmail.com>
Message-ID: <CAH6Pt5oAD_U6kiH4QFFp4fQuPzM9OgL=He0D2sL5Wy7NMCa3hg@mail.gmail.com>

Hi,

On Tue, Apr 30, 2013 at 9:16 PM, Matthew Brett <matthew.brett at gmail.com> wrote:
> Hi,
>
> On Tue, Apr 30, 2013 at 8:08 PM, Yaroslav Halchenko
> <lists at onerussian.com> wrote:
>> could anyone on 32bit system with fresh numpy (1.7.1) test following:
>>
>>> wget -nc http://www.onerussian.com/tmp/data.npy ; python -c 'import numpy as np; data1 = np.load("/tmp/data.npy"); print  np.sum(data1[1,:,0,1]) - np.sum(data1, axis=1)[1,0,1]'
>>
>> 0.0
>>
>> because unfortunately it seems on fresh ubuntu raring (in 32bit build only,
>> seems ok in 64 bit... also never ran into it on older numpy releases):
>>
>>> python -c 'import numpy as np; data1 = np.load("/tmp/data.npy"); print  np.sum(data1[1,:,0,1]) - np.sum(data1, axis=1)[1,0,1]'
>> -1.11022302463e-16
>>
>> PS detected by failed tests of pymvpa
>
> Reduced case on numpy 1.7.1, 32-bit Ubuntu 12.04.2
>
> In [64]: data = np.array([[ 0.49505185,  0.47212842],
>        [ 0.53529587,  0.04366172],
>        [-0.13461665, -0.01664215]])
>
> In [65]: np.sum(data[:, 0]) - np.sum(data, axis=0)[0]
> Out[65]: 1.1102230246251565e-16
>
> No difference for single vector:
>
> In [4]: data1 = data[:, 0:1]
>
> In [5]: np.sum(data1[:, 0]) - np.sum(data1, axis=0)[0]
> Out[5]: 0.0

Also true on current numpy trunk:

In [2]: import numpy as np

In [3]: np.__version__
Out[3]: '1.8.0.dev-a8805f6'

In [4]: data = np.array([[ 0.49505185,  0.47212842],
   ....:        [ 0.53529587,  0.04366172],
   ....:        [-0.13461665, -0.01664215]])

In [5]: np.sum(data[:, 0]) - np.sum(data, axis=0)[0]
Out[5]: 1.1102230246251565e-16

Not true on numpy 1.6.1:

In [2]: np.__version__
Out[2]: '1.6.1'

In [3]: data = np.array([[ 0.49505185,  0.47212842],
   ....:        [ 0.53529587,  0.04366172],
   ....:        [-0.13461665, -0.01664215]])

In [4]: np.sum(data[:, 0]) - np.sum(data, axis=0)[0]
Out[4]: 0.0

Cheers,

Matthew


From brad.froehle at gmail.com  Wed May  1 01:32:41 2013
From: brad.froehle at gmail.com (Bradley M. Froehle)
Date: Tue, 30 Apr 2013 22:32:41 -0700
Subject: [Numpy-discussion] could anyone check on a 32bit system?
In-Reply-To: <20130501030849.GO5140@onerussian.com>
References: <CANNq6F=NCdd1cRWmp7rskr6NJt3sjVvinN1XQVcXjJRgiGk=_A@mail.gmail.com>
	<20130501030849.GO5140@onerussian.com>
Message-ID: <CAHXv-MjuDr8=3n8iZvG7MmJ5WLR8OXm_Mddut=2cci0dL92h6w@mail.gmail.com>

On Tue, Apr 30, 2013 at 8:08 PM, Yaroslav Halchenko <lists at onerussian.com>wrote:

> could anyone on 32bit system with fresh numpy (1.7.1) test following:
>
> > wget -nc http://www.onerussian.com/tmp/data.npy ; python -c 'import
> numpy as np; data1 = np.load("/tmp/data.npy"); print
>  np.sum(data1[1,:,0,1]) - np.sum(data1, axis=1)[1,0,1]'
>
> 0.0
>
> because unfortunately it seems on fresh ubuntu raring (in 32bit build only,
> seems ok in 64 bit... also never ran into it on older numpy releases):
>
> > python -c 'import numpy as np; data1 = np.load("/tmp/data.npy"); print
>  np.sum(data1[1,:,0,1]) - np.sum(data1, axis=1)[1,0,1]'
> -1.11022302463e-16
>

Perhaps on the 32-bit system one call is using the 80-bit extended
precision register for the summation and the other one is not?

-Brad
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20130430/02f514c7/attachment.html>

From sebastian at sipsolutions.net  Wed May  1 05:00:13 2013
From: sebastian at sipsolutions.net (Sebastian Berg)
Date: Wed, 01 May 2013 11:00:13 +0200
Subject: [Numpy-discussion] could anyone check on a 32bit system?
In-Reply-To: <CAH6Pt5oAD_U6kiH4QFFp4fQuPzM9OgL=He0D2sL5Wy7NMCa3hg@mail.gmail.com>
References: <CANNq6F=NCdd1cRWmp7rskr6NJt3sjVvinN1XQVcXjJRgiGk=_A@mail.gmail.com>
	<20130501030849.GO5140@onerussian.com>
	<CAH6Pt5ryQQX6jqSUQNH_416RXFD2+1dGPV1rRn9DK-ZvZkC9Zg@mail.gmail.com>
	<CAH6Pt5oAD_U6kiH4QFFp4fQuPzM9OgL=He0D2sL5Wy7NMCa3hg@mail.gmail.com>
Message-ID: <1367398813.2545.11.camel@sebastian-laptop>

On Tue, 2013-04-30 at 22:20 -0700, Matthew Brett wrote:
> Hi,
> 
> On Tue, Apr 30, 2013 at 9:16 PM, Matthew Brett <matthew.brett at gmail.com> wrote:
> > Hi,
> >
> > On Tue, Apr 30, 2013 at 8:08 PM, Yaroslav Halchenko
> > <lists at onerussian.com> wrote:
> >> could anyone on 32bit system with fresh numpy (1.7.1) test following:
> >>
> >>> wget -nc http://www.onerussian.com/tmp/data.npy ; python -c 'import numpy as np; data1 = np.load("/tmp/data.npy"); print  np.sum(data1[1,:,0,1]) - np.sum(data1, axis=1)[1,0,1]'
> >>
> >> 0.0
> >>
> >> because unfortunately it seems on fresh ubuntu raring (in 32bit build only,
> >> seems ok in 64 bit... also never ran into it on older numpy releases):
> >>
> >>> python -c 'import numpy as np; data1 = np.load("/tmp/data.npy"); print  np.sum(data1[1,:,0,1]) - np.sum(data1, axis=1)[1,0,1]'
> >> -1.11022302463e-16
> >>
> >> PS detected by failed tests of pymvpa
> >
> > Reduced case on numpy 1.7.1, 32-bit Ubuntu 12.04.2
> >
> > In [64]: data = np.array([[ 0.49505185,  0.47212842],
> >        [ 0.53529587,  0.04366172],
> >        [-0.13461665, -0.01664215]])
> >
> > In [65]: np.sum(data[:, 0]) - np.sum(data, axis=0)[0]
> > Out[65]: 1.1102230246251565e-16
> >
> > No difference for single vector:
> >
> > In [4]: data1 = data[:, 0:1]
> >
> > In [5]: np.sum(data1[:, 0]) - np.sum(data1, axis=0)[0]
> > Out[5]: 0.0
> 
> Also true on current numpy trunk:
> 
> In [2]: import numpy as np
> 
> In [3]: np.__version__
> Out[3]: '1.8.0.dev-a8805f6'
> 
> In [4]: data = np.array([[ 0.49505185,  0.47212842],
>    ....:        [ 0.53529587,  0.04366172],
>    ....:        [-0.13461665, -0.01664215]])
> 
> In [5]: np.sum(data[:, 0]) - np.sum(data, axis=0)[0]
> Out[5]: 1.1102230246251565e-16
> 
> Not true on numpy 1.6.1:
> 
> In [2]: np.__version__
> Out[2]: '1.6.1'
> 
> In [3]: data = np.array([[ 0.49505185,  0.47212842],
>    ....:        [ 0.53529587,  0.04366172],
>    ....:        [-0.13461665, -0.01664215]])
> 
> In [4]: np.sum(data[:, 0]) - np.sum(data, axis=0)[0]
> Out[4]: 0.0
> 

Puzzles me, I didn't think calculation order was different in both cases
and optimized for the reduction part.  But maybe check the code, if it
is optimized, it would calculate this more like `res += data[0]; res +=
data[1]; res += data[2]` (for faster memory access), which would
probably kill the extended registers (I don't know this hardware stuff,
so might be wrong). One simple try hinting that this may be going on
would be to data fortran order.

- Sebastian

> Cheers,
> 
> Matthew
> _______________________________________________
> NumPy-Discussion mailing list
> NumPy-Discussion at scipy.org
> http://mail.scipy.org/mailman/listinfo/numpy-discussion
> 




From sebastian at sipsolutions.net  Wed May  1 05:08:15 2013
From: sebastian at sipsolutions.net (Sebastian Berg)
Date: Wed, 01 May 2013 11:08:15 +0200
Subject: [Numpy-discussion] could anyone check on a 32bit system?
In-Reply-To: <1367398813.2545.11.camel@sebastian-laptop>
References: <CANNq6F=NCdd1cRWmp7rskr6NJt3sjVvinN1XQVcXjJRgiGk=_A@mail.gmail.com>
	<20130501030849.GO5140@onerussian.com>
	<CAH6Pt5ryQQX6jqSUQNH_416RXFD2+1dGPV1rRn9DK-ZvZkC9Zg@mail.gmail.com>
	<CAH6Pt5oAD_U6kiH4QFFp4fQuPzM9OgL=He0D2sL5Wy7NMCa3hg@mail.gmail.com>
	<1367398813.2545.11.camel@sebastian-laptop>
Message-ID: <1367399295.2545.14.camel@sebastian-laptop>

On Wed, 2013-05-01 at 11:00 +0200, Sebastian Berg wrote:
> On Tue, 2013-04-30 at 22:20 -0700, Matthew Brett wrote:
> > Hi,
> > 
> > On Tue, Apr 30, 2013 at 9:16 PM, Matthew Brett <matthew.brett at gmail.com> wrote:
> > > Hi,
> > >
> > > On Tue, Apr 30, 2013 at 8:08 PM, Yaroslav Halchenko
> > > <lists at onerussian.com> wrote:
> > >> could anyone on 32bit system with fresh numpy (1.7.1) test following:
> > >>
> > >>> wget -nc http://www.onerussian.com/tmp/data.npy ; python -c 'import numpy as np; data1 = np.load("/tmp/data.npy"); print  np.sum(data1[1,:,0,1]) - np.sum(data1, axis=1)[1,0,1]'
> > >>
> > >> 0.0
> > >>
> > >> because unfortunately it seems on fresh ubuntu raring (in 32bit build only,
> > >> seems ok in 64 bit... also never ran into it on older numpy releases):
> > >>
> > >>> python -c 'import numpy as np; data1 = np.load("/tmp/data.npy"); print  np.sum(data1[1,:,0,1]) - np.sum(data1, axis=1)[1,0,1]'
> > >> -1.11022302463e-16
> > >>
> > >> PS detected by failed tests of pymvpa
> > >
> > > Reduced case on numpy 1.7.1, 32-bit Ubuntu 12.04.2
> > >
> > > In [64]: data = np.array([[ 0.49505185,  0.47212842],
> > >        [ 0.53529587,  0.04366172],
> > >        [-0.13461665, -0.01664215]])
> > >
> > > In [65]: np.sum(data[:, 0]) - np.sum(data, axis=0)[0]
> > > Out[65]: 1.1102230246251565e-16
> > >
> > > No difference for single vector:
> > >
> > > In [4]: data1 = data[:, 0:1]
> > >
> > > In [5]: np.sum(data1[:, 0]) - np.sum(data1, axis=0)[0]
> > > Out[5]: 0.0
> > 
> > Also true on current numpy trunk:
> > 
> > In [2]: import numpy as np
> > 
> > In [3]: np.__version__
> > Out[3]: '1.8.0.dev-a8805f6'
> > 
> > In [4]: data = np.array([[ 0.49505185,  0.47212842],
> >    ....:        [ 0.53529587,  0.04366172],
> >    ....:        [-0.13461665, -0.01664215]])
> > 
> > In [5]: np.sum(data[:, 0]) - np.sum(data, axis=0)[0]
> > Out[5]: 1.1102230246251565e-16
> > 
> > Not true on numpy 1.6.1:
> > 
> > In [2]: np.__version__
> > Out[2]: '1.6.1'
> > 
> > In [3]: data = np.array([[ 0.49505185,  0.47212842],
> >    ....:        [ 0.53529587,  0.04366172],
> >    ....:        [-0.13461665, -0.01664215]])
> > 
> > In [4]: np.sum(data[:, 0]) - np.sum(data, axis=0)[0]
> > Out[4]: 0.0
> > 
> 
> Puzzles me, I didn't think calculation order was different in both cases
> and optimized for the reduction part.  But maybe check the code, if it
> is optimized, it would calculate this more like `res += data[0]; res +=
> data[1]; res += data[2]` (for faster memory access), which would
> probably kill the extended registers (I don't know this hardware stuff,
> so might be wrong). One simple try hinting that this may be going on
> would be to data fortran order.
> 

Well I guess it is optimized and the reason I thought it was not, was
because it probably was not before 1.7. so I think this is probably the
reason.

> - Sebastian
> 
> > Cheers,
> > 
> > Matthew
> > _______________________________________________
> > NumPy-Discussion mailing list
> > NumPy-Discussion at scipy.org
> > http://mail.scipy.org/mailman/listinfo/numpy-discussion
> > 
> 
> 
> _______________________________________________
> NumPy-Discussion mailing list
> NumPy-Discussion at scipy.org
> http://mail.scipy.org/mailman/listinfo/numpy-discussion
> 




From lists at onerussian.com  Wed May  1 08:49:33 2013
From: lists at onerussian.com (Yaroslav Halchenko)
Date: Wed, 1 May 2013 08:49:33 -0400
Subject: [Numpy-discussion] could anyone check on a 32bit system?
In-Reply-To: <1367399295.2545.14.camel@sebastian-laptop>
References: <CANNq6F=NCdd1cRWmp7rskr6NJt3sjVvinN1XQVcXjJRgiGk=_A@mail.gmail.com>
	<20130501030849.GO5140@onerussian.com>
	<CAH6Pt5ryQQX6jqSUQNH_416RXFD2+1dGPV1rRn9DK-ZvZkC9Zg@mail.gmail.com>
	<CAH6Pt5oAD_U6kiH4QFFp4fQuPzM9OgL=He0D2sL5Wy7NMCa3hg@mail.gmail.com>
	<1367398813.2545.11.camel@sebastian-laptop>
	<1367399295.2545.14.camel@sebastian-laptop>
Message-ID: <20130501124933.GP5140@onerussian.com>

Thanks everyone for the feedback.

Is it worth me starting a bisection to catch where it was introduced?

On Wed, 01 May 2013, Sebastian Berg wrote:
> > so might be wrong). One simple try hinting that this may be going on
> > would be to data fortran order.


> Well I guess it is optimized and the reason I thought it was not, was
> because it probably was not before 1.7. so I think this is probably the
> reason.

-- 
Yaroslav O. Halchenko, Ph.D.
http://neuro.debian.net http://www.pymvpa.org http://www.fail2ban.org
Senior Research Associate,     Psychological and Brain Sciences Dept.
Dartmouth College, 419 Moore Hall, Hinman Box 6207, Hanover, NH 03755
Phone: +1 (603) 646-9834                       Fax: +1 (603) 646-1419
WWW:   http://www.linkedin.com/in/yarik        


From njs at pobox.com  Wed May  1 08:55:56 2013
From: njs at pobox.com (Nathaniel Smith)
Date: Wed, 1 May 2013 08:55:56 -0400
Subject: [Numpy-discussion] could anyone check on a 32bit system?
In-Reply-To: <20130501124933.GP5140@onerussian.com>
References: <CANNq6F=NCdd1cRWmp7rskr6NJt3sjVvinN1XQVcXjJRgiGk=_A@mail.gmail.com>
	<20130501030849.GO5140@onerussian.com>
	<CAH6Pt5ryQQX6jqSUQNH_416RXFD2+1dGPV1rRn9DK-ZvZkC9Zg@mail.gmail.com>
	<CAH6Pt5oAD_U6kiH4QFFp4fQuPzM9OgL=He0D2sL5Wy7NMCa3hg@mail.gmail.com>
	<1367398813.2545.11.camel@sebastian-laptop>
	<1367399295.2545.14.camel@sebastian-laptop>
	<20130501124933.GP5140@onerussian.com>
Message-ID: <CAPJVwBmnQy_jaqFfJO0m-tMLXpdPrAx8H66NGYkZmhO43L7tRA@mail.gmail.com>

On 1 May 2013 08:49, "Yaroslav Halchenko" <lists at onerussian.com> wrote:
>
> Thanks everyone for the feedback.
>
> Is it worth me starting a bisection to catch where it was introduced?

Is it a bug, or just typical fp rounding issues? Do we know which answer is
correct?

-n
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20130501/820ca436/attachment.html>

From lists at onerussian.com  Wed May  1 09:01:25 2013
From: lists at onerussian.com (Yaroslav Halchenko)
Date: Wed, 1 May 2013 09:01:25 -0400
Subject: [Numpy-discussion] could anyone check on a 32bit system?
In-Reply-To: <CAPJVwBmnQy_jaqFfJO0m-tMLXpdPrAx8H66NGYkZmhO43L7tRA@mail.gmail.com>
References: <CANNq6F=NCdd1cRWmp7rskr6NJt3sjVvinN1XQVcXjJRgiGk=_A@mail.gmail.com>
	<20130501030849.GO5140@onerussian.com>
	<CAH6Pt5ryQQX6jqSUQNH_416RXFD2+1dGPV1rRn9DK-ZvZkC9Zg@mail.gmail.com>
	<CAH6Pt5oAD_U6kiH4QFFp4fQuPzM9OgL=He0D2sL5Wy7NMCa3hg@mail.gmail.com>
	<1367398813.2545.11.camel@sebastian-laptop>
	<1367399295.2545.14.camel@sebastian-laptop>
	<20130501124933.GP5140@onerussian.com>
	<CAPJVwBmnQy_jaqFfJO0m-tMLXpdPrAx8H66NGYkZmhO43L7tRA@mail.gmail.com>
Message-ID: <20130501130125.GQ5140@onerussian.com>


On Wed, 01 May 2013, Nathaniel Smith wrote:
>    > Thanks everyone for the feedback.
>    > Is it worth me starting a bisection to catch where it was introduced?

>    Is it a bug, or just typical fp rounding issues? Do we know which answer
>    is correct?

to ignorant me, even without considering 'correctness', it is just
a typical regression -- results changed from one release to another (and
not to the better side).

-- 
Yaroslav O. Halchenko, Ph.D.
http://neuro.debian.net http://www.pymvpa.org http://www.fail2ban.org
Senior Research Associate,     Psychological and Brain Sciences Dept.
Dartmouth College, 419 Moore Hall, Hinman Box 6207, Hanover, NH 03755
Phone: +1 (603) 646-9834                       Fax: +1 (603) 646-1419
WWW:   http://www.linkedin.com/in/yarik        


From pav at iki.fi  Wed May  1 09:12:08 2013
From: pav at iki.fi (Pauli Virtanen)
Date: Wed, 01 May 2013 16:12:08 +0300
Subject: [Numpy-discussion] could anyone check on a 32bit system?
In-Reply-To: <20130501130125.GQ5140@onerussian.com>
References: <CANNq6F=NCdd1cRWmp7rskr6NJt3sjVvinN1XQVcXjJRgiGk=_A@mail.gmail.com>
	<20130501030849.GO5140@onerussian.com>
	<CAH6Pt5ryQQX6jqSUQNH_416RXFD2+1dGPV1rRn9DK-ZvZkC9Zg@mail.gmail.com>
	<CAH6Pt5oAD_U6kiH4QFFp4fQuPzM9OgL=He0D2sL5Wy7NMCa3hg@mail.gmail.com>
	<1367398813.2545.11.camel@sebastian-laptop>
	<1367399295.2545.14.camel@sebastian-laptop>
	<20130501124933.GP5140@onerussian.com>
	<CAPJVwBmnQy_jaqFfJO0m-tMLXpdPrAx8H66NGYkZmhO43L7tRA@mail.gmail.com>
	<20130501130125.GQ5140@onerussian.com>
Message-ID: <klr4b3$shp$1@ger.gmane.org>

01.05.2013 16:01, Yaroslav Halchenko kirjoitti:
[clip]
> to ignorant me, even without considering 'correctness', it is just
> a typical regression -- results changed from one release to another (and
> not to the better side).

To me this seems to be a consequence of performing additions in a
different order than previously. Both results are IMHO correct, so I'm
not sure there is anything to fix here. Third-party code relying on a
certain outcome of rounding error is likely incorrect anyway.

-- 
Pauli Virtanen



From ben.root at ou.edu  Wed May  1 09:52:13 2013
From: ben.root at ou.edu (Benjamin Root)
Date: Wed, 1 May 2013 09:52:13 -0400
Subject: [Numpy-discussion] nanmean(),
 nanstd() and other "missing" functions for 1.8
In-Reply-To: <1888613198476891170@unknownmsgid>
References: <CANNq6F=NCdd1cRWmp7rskr6NJt3sjVvinN1XQVcXjJRgiGk=_A@mail.gmail.com>
	<1888613198476891170@unknownmsgid>
Message-ID: <CANNq6FnZ4LOA8F1efJW+ZEeJtS3gwYag4mbvOMGSOt=rj_TwsQ@mail.gmail.com>

On Wed, May 1, 2013 at 1:13 AM, Chris Barker - NOAA Federal <
chris.barker at noaa.gov> wrote:

> > Of course, the documentation for discussed before: np.minmax().  My
> thinking is that it would return a 2xN array
>
> How about a tuple: (min, max)?
>
>
I am not familiar enough with numpy internals to know which is the better
approach to implement.  I kind of feel that the 2xN array approach would be
more flexible in case a user wants all of this information in a single
array, while still allowing for unpacking as if it was a tuple.  I would
rather enable unforeseen use-cases rather than needlessly restricting them.

Ben Root
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20130501/3da4e7f2/attachment.html>

From njs at pobox.com  Wed May  1 09:54:56 2013
From: njs at pobox.com (Nathaniel Smith)
Date: Wed, 1 May 2013 09:54:56 -0400
Subject: [Numpy-discussion] could anyone check on a 32bit system?
In-Reply-To: <klr4b3$shp$1@ger.gmane.org>
References: <CANNq6F=NCdd1cRWmp7rskr6NJt3sjVvinN1XQVcXjJRgiGk=_A@mail.gmail.com>
	<20130501030849.GO5140@onerussian.com>
	<CAH6Pt5ryQQX6jqSUQNH_416RXFD2+1dGPV1rRn9DK-ZvZkC9Zg@mail.gmail.com>
	<CAH6Pt5oAD_U6kiH4QFFp4fQuPzM9OgL=He0D2sL5Wy7NMCa3hg@mail.gmail.com>
	<1367398813.2545.11.camel@sebastian-laptop>
	<1367399295.2545.14.camel@sebastian-laptop>
	<20130501124933.GP5140@onerussian.com>
	<CAPJVwBmnQy_jaqFfJO0m-tMLXpdPrAx8H66NGYkZmhO43L7tRA@mail.gmail.com>
	<20130501130125.GQ5140@onerussian.com> <klr4b3$shp$1@ger.gmane.org>
Message-ID: <CAPJVwBkBiHFhx0jEE444zgK5bvQFQa0CkX+9M3+OfXfJz_2+6g@mail.gmail.com>

On Wed, May 1, 2013 at 9:12 AM, Pauli Virtanen <pav at iki.fi> wrote:
> 01.05.2013 16:01, Yaroslav Halchenko kirjoitti:
> [clip]
>> to ignorant me, even without considering 'correctness', it is just
>> a typical regression -- results changed from one release to another (and
>> not to the better side).
>
> To me this seems to be a consequence of performing additions in a
> different order than previously. Both results are IMHO correct, so I'm
> not sure there is anything to fix here. Third-party code relying on a
> certain outcome of rounding error is likely incorrect anyway.

Yeah, seems to just be the standard floating point indeterminism.
Using Matthew's numbers and pure Python floats:

In [9]: (0.49505185 + 0.53529587) + -0.13461665
Out[9]: 0.89573107

In [10]: 0.49505185 + (0.53529587 + -0.13461665)
Out[10]: 0.8957310700000001

In [11]: _9 - _10
Out[11]: -1.1102230246251565e-16

Looks like a bug in pymvpa or its test suite to me.

-n


From davidmenhur at gmail.com  Wed May  1 10:14:22 2013
From: davidmenhur at gmail.com (=?UTF-8?B?RGHPgGlk?=)
Date: Wed, 1 May 2013 16:14:22 +0200
Subject: [Numpy-discussion] nanmean(),
 nanstd() and other "missing" functions for 1.8
In-Reply-To: <CANNq6F=NCdd1cRWmp7rskr6NJt3sjVvinN1XQVcXjJRgiGk=_A@mail.gmail.com>
References: <CANNq6F=NCdd1cRWmp7rskr6NJt3sjVvinN1XQVcXjJRgiGk=_A@mail.gmail.com>
Message-ID: <CAJhcF=2vWv82GMYOsAVYKAUOzXu4KcpG92HGvUY5unGpgPnfiw@mail.gmail.com>

On 1 May 2013 03:36, Benjamin Root <ben.root at ou.edu> wrote:
> Are there any other functions that others feel are "missing" from numpy and
> would like to see for v1.8?  Let's discuss them here.

I would like to have sincos, to compute sin and cos of the same number
faster. According to some benchmarks, it is barely slower than just
computing one of them.



On 1 May 2013 07:13, Chris Barker - NOAA Federal <chris.barker at noaa.gov> wrote:
>> Of course, the documentation for discussed before: np.minmax().  My thinking is that it would return a 2xN array
>
> How about a tuple: (min, max)?

Consider the case of np.minmax(matrix, axis=1), you will end up with a
tuple of two arrays. In that scenario, you probably want to do
computations with both numbers, so having them in an array seems more
convenient.

If there is enough reason, we could always add a "unpack=True" flag
and then return a tuple.


From juanlu001 at gmail.com  Wed May  1 10:55:44 2013
From: juanlu001 at gmail.com (Juan Luis Cano)
Date: Wed, 01 May 2013 16:55:44 +0200
Subject: [Numpy-discussion] nanmean(),
 nanstd() and other "missing" functions for 1.8
In-Reply-To: <CAJhcF=2vWv82GMYOsAVYKAUOzXu4KcpG92HGvUY5unGpgPnfiw@mail.gmail.com>
References: <CANNq6F=NCdd1cRWmp7rskr6NJt3sjVvinN1XQVcXjJRgiGk=_A@mail.gmail.com>
	<CAJhcF=2vWv82GMYOsAVYKAUOzXu4KcpG92HGvUY5unGpgPnfiw@mail.gmail.com>
Message-ID: <51812CF0.30004@gmail.com>

On 05/01/2013 04:14 PM, Da?id wrote:
> On 1 May 2013 03:36, Benjamin Root <ben.root at ou.edu> wrote:
>> Are there any other functions that others feel are "missing" from numpy and
>> would like to see for v1.8?  Let's discuss them here.
> I would like to have sincos, to compute sin and cos of the same number
> faster. According to some benchmarks, it is barely slower than just
> computing one of them.

Where does this `sincos` function come from?


From zploskey at gmail.com  Wed May  1 11:06:17 2013
From: zploskey at gmail.com (Zachary Ploskey)
Date: Wed, 1 May 2013 08:06:17 -0700
Subject: [Numpy-discussion] nanmean(),
 nanstd() and other "missing" functions for 1.8
In-Reply-To: <51812CF0.30004@gmail.com>
References: <CANNq6F=NCdd1cRWmp7rskr6NJt3sjVvinN1XQVcXjJRgiGk=_A@mail.gmail.com>
	<CAJhcF=2vWv82GMYOsAVYKAUOzXu4KcpG92HGvUY5unGpgPnfiw@mail.gmail.com>
	<51812CF0.30004@gmail.com>
Message-ID: <CAHf4x6c=N+J3Sgipah+b-yuM1OSZa5+=poWwYqdW0BRvE_FaUQ@mail.gmail.com>

The sincos function is in the c standard library in math.h.
On May 1, 2013 7:56 AM, "Juan Luis Cano" <juanlu001 at gmail.com> wrote:

> On 05/01/2013 04:14 PM, Da?id wrote:
> > On 1 May 2013 03:36, Benjamin Root <ben.root at ou.edu> wrote:
> >> Are there any other functions that others feel are "missing" from numpy
> and
> >> would like to see for v1.8?  Let's discuss them here.
> > I would like to have sincos, to compute sin and cos of the same number
> > faster. According to some benchmarks, it is barely slower than just
> > computing one of them.
>
> Where does this `sincos` function come from?
> _______________________________________________
> NumPy-Discussion mailing list
> NumPy-Discussion at scipy.org
> http://mail.scipy.org/mailman/listinfo/numpy-discussion
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20130501/71822591/attachment.html>

From toddrjen at gmail.com  Wed May  1 11:13:49 2013
From: toddrjen at gmail.com (Todd)
Date: Wed, 1 May 2013 17:13:49 +0200
Subject: [Numpy-discussion] nanmean(),
 nanstd() and other "missing" functions for 1.8
In-Reply-To: <CANNq6F=NCdd1cRWmp7rskr6NJt3sjVvinN1XQVcXjJRgiGk=_A@mail.gmail.com>
References: <CANNq6F=NCdd1cRWmp7rskr6NJt3sjVvinN1XQVcXjJRgiGk=_A@mail.gmail.com>
Message-ID: <CAFpSVpK-Qr2qzGtKHCyW5roj2RmLRg6UkJLvC6cmRxJboYna7w@mail.gmail.com>

On Wed, May 1, 2013 at 3:36 AM, Benjamin Root <ben.root at ou.edu> wrote:

>
> Are there any other functions that others feel are "missing" from numpy
> and would like to see for v1.8?  Let's discuss them here.
>

As I mentioned before, I think numpy should have some equations for dealing
with n-dimensional vectors (but would also work with complex dtypes).  This
would include n-dimensional equivalents of np.abs and np.angle, as well as
a function to go back to an n-dimensional vector from the length and
angle.  Considering how critical vector analysis is to signal processing, I
am surprised these don't already exist.  There aren't even functions that
work with 2-dimensional vectors, you have to construct a complex array
first (which isn't that easy to do).

Speaking of which, I think there should be a function to construct a
complex array out of two identically-shaped floating-point arrays, as well
as perhaps an np.i class that converts a real array to an imaginary one
(using __mul__ and such).
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20130501/1b3af6e5/attachment.html>

From robert.kern at gmail.com  Wed May  1 11:14:33 2013
From: robert.kern at gmail.com (Robert Kern)
Date: Wed, 1 May 2013 16:14:33 +0100
Subject: [Numpy-discussion] nanmean(),
 nanstd() and other "missing" functions for 1.8
In-Reply-To: <CAHf4x6c=N+J3Sgipah+b-yuM1OSZa5+=poWwYqdW0BRvE_FaUQ@mail.gmail.com>
References: <CANNq6F=NCdd1cRWmp7rskr6NJt3sjVvinN1XQVcXjJRgiGk=_A@mail.gmail.com>
	<CAJhcF=2vWv82GMYOsAVYKAUOzXu4KcpG92HGvUY5unGpgPnfiw@mail.gmail.com>
	<51812CF0.30004@gmail.com>
	<CAHf4x6c=N+J3Sgipah+b-yuM1OSZa5+=poWwYqdW0BRvE_FaUQ@mail.gmail.com>
Message-ID: <CAF6FJiuC23NwHVSPFu7Z6ZURNO8mVKj3kXeWY4j6ggqcLR4siw@mail.gmail.com>

On Wed, May 1, 2013 at 4:06 PM, Zachary Ploskey <zploskey at gmail.com> wrote:
> The sincos function is in the c standard library in math.h.

I don't think it's part of the C99 standard. It appears to be provided
in glibc as a non-standard extension. We would have to provide our own
copy, but one is available in the Cephes library.

--
Robert Kern


From jslavin at cfa.harvard.edu  Wed May  1 11:20:21 2013
From: jslavin at cfa.harvard.edu (Jonathan Slavin)
Date: Wed, 01 May 2013 11:20:21 -0400
Subject: [Numpy-discussion] clip with None argument changes dtype
Message-ID: <1367421621.4115.16.camel@shevek>

Hi all,

I'm wondering if you think the following behavior in numpy.clip is a bug
(it certainly confused me for a while):
>>> x = np.arange(5.)
>>> xx = x.clip(None,3.)
>>> xx
array([0.0, 1.0, 2.0, 3.0, 3.0], dtype=object)
Since xx now has the dtype of object, doing things like
>>> np.exp(xx)
AttributeError                            Traceback (most recent call
last)
<ipython-input-6-30aa315cc2b1> in <module>()
----> 1 np.exp(xx)
Which, if you don't know about the change in the dtype is a very
confusing error message.  It seems to me that either clip should give an
error message when None is given for the a_min argument or, better,
should not change the dtype of the input array as it does.  This comes
up because I want to only clip the maximum.  Of course you can clip the
minimum by simply omitting the second argument.  The asymmetry of this
is not good, I think.  I suppose that using the maximum function is
better in this situation (as I recently found out), though the docs make
it seem that one needs to supply two arrays -- even though you don't.

Jon
-- 
______________________________________________________________
Jonathan D. Slavin              Harvard-Smithsonian CfA
jslavin at cfa.harvard.edu         60 Garden Street, MS 83
phone: (617) 496-7981           Cambridge, MA 02138-1516
 cell: (781) 363-0035           USA
______________________________________________________________



From davidmenhur at gmail.com  Wed May  1 11:22:28 2013
From: davidmenhur at gmail.com (=?UTF-8?B?RGHPgGlk?=)
Date: Wed, 1 May 2013 17:22:28 +0200
Subject: [Numpy-discussion] nanmean(),
 nanstd() and other "missing" functions for 1.8
In-Reply-To: <CAFpSVpK-Qr2qzGtKHCyW5roj2RmLRg6UkJLvC6cmRxJboYna7w@mail.gmail.com>
References: <CANNq6F=NCdd1cRWmp7rskr6NJt3sjVvinN1XQVcXjJRgiGk=_A@mail.gmail.com>
	<CAFpSVpK-Qr2qzGtKHCyW5roj2RmLRg6UkJLvC6cmRxJboYna7w@mail.gmail.com>
Message-ID: <CAJhcF=1CSk8882NHqz7Ru1PY-FfVFum2SumwQduJrpGLeo9+=w@mail.gmail.com>

On 1 May 2013 17:13, Todd <toddrjen at gmail.com> wrote:
> Speaking of which, I think there should be a function to construct a complex
> array out of two identically-shaped floating-point arrays, as well as
> perhaps an np.i class that converts a real array to an imaginary one (using
> __mul__ and such).

np.i would be exactly the same as array * 1j, or am I missing anything?

The same goes for constructing a complex, real + imag * 1j


From toddrjen at gmail.com  Wed May  1 11:34:34 2013
From: toddrjen at gmail.com (Todd)
Date: Wed, 1 May 2013 17:34:34 +0200
Subject: [Numpy-discussion] nanmean(),
 nanstd() and other "missing" functions for 1.8
In-Reply-To: <CAJhcF=1CSk8882NHqz7Ru1PY-FfVFum2SumwQduJrpGLeo9+=w@mail.gmail.com>
References: <CANNq6F=NCdd1cRWmp7rskr6NJt3sjVvinN1XQVcXjJRgiGk=_A@mail.gmail.com>
	<CAFpSVpK-Qr2qzGtKHCyW5roj2RmLRg6UkJLvC6cmRxJboYna7w@mail.gmail.com>
	<CAJhcF=1CSk8882NHqz7Ru1PY-FfVFum2SumwQduJrpGLeo9+=w@mail.gmail.com>
Message-ID: <CAFpSVpK_KQf-cyatEJdAnYmUiGucrAusXo3h0vxDHfXn8=OJ3w@mail.gmail.com>

On Wed, May 1, 2013 at 5:22 PM, Da?id <davidmenhur at gmail.com> wrote:

> On 1 May 2013 17:13, Todd <toddrjen at gmail.com> wrote:
> > Speaking of which, I think there should be a function to construct a
> complex
> > array out of two identically-shaped floating-point arrays, as well as
> > perhaps an np.i class that converts a real array to an imaginary one
> (using
> > __mul__ and such).
>
> np.i would be exactly the same as array * 1j, or am I missing anything?
>
> The same goes for constructing a complex, real + imag * 1j
>
>
>
it would always produce a numpy array.  So array*1j and array*np.i (or np.j
if you prefer, perhaps both) would be the same, but list*1j and list*np.i
would not.


The function version would also probably allow you to specify the dtype,
which 1j does not.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20130501/1fbd27ee/attachment.html>

From robert.kern at gmail.com  Wed May  1 11:36:13 2013
From: robert.kern at gmail.com (Robert Kern)
Date: Wed, 1 May 2013 16:36:13 +0100
Subject: [Numpy-discussion] nanmean(),
 nanstd() and other "missing" functions for 1.8
In-Reply-To: <CAJhcF=1CSk8882NHqz7Ru1PY-FfVFum2SumwQduJrpGLeo9+=w@mail.gmail.com>
References: <CANNq6F=NCdd1cRWmp7rskr6NJt3sjVvinN1XQVcXjJRgiGk=_A@mail.gmail.com>
	<CAFpSVpK-Qr2qzGtKHCyW5roj2RmLRg6UkJLvC6cmRxJboYna7w@mail.gmail.com>
	<CAJhcF=1CSk8882NHqz7Ru1PY-FfVFum2SumwQduJrpGLeo9+=w@mail.gmail.com>
Message-ID: <CAF6FJitghsVG7XoMaMB2buZ1yLOvnTU5kgS76+jscq6VnwSHbw@mail.gmail.com>

On Wed, May 1, 2013 at 4:22 PM, Da?id <davidmenhur at gmail.com> wrote:
> On 1 May 2013 17:13, Todd <toddrjen at gmail.com> wrote:
>> Speaking of which, I think there should be a function to construct a complex
>> array out of two identically-shaped floating-point arrays, as well as
>> perhaps an np.i class that converts a real array to an imaginary one (using
>> __mul__ and such).
>
> np.i would be exactly the same as array * 1j, or am I missing anything?

I don't think we have a ufunc loop for multiply() that takes a float64
and a complex128 and returns a complex128. We just have a
(complex128,complex128)->complex128. `x * 1j` first converts `x` to a
complex128 array with the value in the real component, then multiplies
that with 1j to move that value over to the imag component. A single
operation that takes a float64 array and just makes a complex128 array
with the values put in the imag component will reduce a temporary.

> The same goes for constructing a complex, real + imag * 1j

Similarly, we can eliminate two temporaries here.

Both of the cases are probably best addressed by a single function.
The syntactic sugar of an np.i object is unnecessary, IMO.

  imag = np.tocomplex(0.0, x)
  z = np.tocomplex(x, y)

--
Robert Kern


From lists at onerussian.com  Wed May  1 12:09:48 2013
From: lists at onerussian.com (Yaroslav Halchenko)
Date: Wed, 1 May 2013 12:09:48 -0400
Subject: [Numpy-discussion] could anyone check on a 32bit system?
In-Reply-To: <CAPJVwBkBiHFhx0jEE444zgK5bvQFQa0CkX+9M3+OfXfJz_2+6g@mail.gmail.com>
References: <20130501030849.GO5140@onerussian.com>
	<CAH6Pt5ryQQX6jqSUQNH_416RXFD2+1dGPV1rRn9DK-ZvZkC9Zg@mail.gmail.com>
	<CAH6Pt5oAD_U6kiH4QFFp4fQuPzM9OgL=He0D2sL5Wy7NMCa3hg@mail.gmail.com>
	<1367398813.2545.11.camel@sebastian-laptop>
	<1367399295.2545.14.camel@sebastian-laptop>
	<20130501124933.GP5140@onerussian.com>
	<CAPJVwBmnQy_jaqFfJO0m-tMLXpdPrAx8H66NGYkZmhO43L7tRA@mail.gmail.com>
	<20130501130125.GQ5140@onerussian.com> <klr4b3$shp$1@ger.gmane.org>
	<CAPJVwBkBiHFhx0jEE444zgK5bvQFQa0CkX+9M3+OfXfJz_2+6g@mail.gmail.com>
Message-ID: <20130501160948.GR5140@onerussian.com>


On Wed, 01 May 2013, Nathaniel Smith wrote:
> > not sure there is anything to fix here. Third-party code relying on a
> > certain outcome of rounding error is likely incorrect anyway.

> Yeah, seems to just be the standard floating point indeterminism.
> Using Matthew's numbers and pure Python floats:

> In [9]: (0.49505185 + 0.53529587) + -0.13461665
> Out[9]: 0.89573107

> In [10]: 0.49505185 + (0.53529587 + -0.13461665)
> Out[10]: 0.8957310700000001

> In [11]: _9 - _10
> Out[11]: -1.1102230246251565e-16

> Looks like a bug in pymvpa or its test suite to me.

well -- sure thing we will "fix" the unittest to not rely on precise
correspondence any longer since released 1.7.1 is effected.  So it is not
a matter of me avoiding "fixing" pymvpa's "bug".

I brought it to your attention because

1. from e.g.

np.sum(data[:, 0]) - np.sum(data, axis=0)[0]

which presumably should be the same order of additions for 0-th column it is
not clear that they do not have to be identical.

2. so far they were identical across many numpy releases

3. they are identical on other architectures (e.g. amd64)

-- 
Yaroslav O. Halchenko, Ph.D.
http://neuro.debian.net http://www.pymvpa.org http://www.fail2ban.org
Senior Research Associate,     Psychological and Brain Sciences Dept.
Dartmouth College, 419 Moore Hall, Hinman Box 6207, Hanover, NH 03755
Phone: +1 (603) 646-9834                       Fax: +1 (603) 646-1419
WWW:   http://www.linkedin.com/in/yarik        


From warren.weckesser at gmail.com  Wed May  1 12:19:38 2013
From: warren.weckesser at gmail.com (Warren Weckesser)
Date: Wed, 1 May 2013 12:19:38 -0400
Subject: [Numpy-discussion] nanmean(),
 nanstd() and other "missing" functions for 1.8
In-Reply-To: <CAJhcF=2vWv82GMYOsAVYKAUOzXu4KcpG92HGvUY5unGpgPnfiw@mail.gmail.com>
References: <CANNq6F=NCdd1cRWmp7rskr6NJt3sjVvinN1XQVcXjJRgiGk=_A@mail.gmail.com>
	<CAJhcF=2vWv82GMYOsAVYKAUOzXu4KcpG92HGvUY5unGpgPnfiw@mail.gmail.com>
Message-ID: <CAGzF1ueF-2L8Bbg9fNqi1+z4uCDhg3pq0zg=qi64CAoFCofb7Q@mail.gmail.com>

On Wed, May 1, 2013 at 10:14 AM, Da?id <davidmenhur at gmail.com> wrote:

> On 1 May 2013 03:36, Benjamin Root <ben.root at ou.edu> wrote:
> > Are there any other functions that others feel are "missing" from numpy
> and
> > would like to see for v1.8?  Let's discuss them here.
>
> I would like to have sincos, to compute sin and cos of the same number
> faster. According to some benchmarks, it is barely slower than just
> computing one of them.
>
>

+1

Warren



>
> On 1 May 2013 07:13, Chris Barker - NOAA Federal <chris.barker at noaa.gov>
> wrote:
> >> Of course, the documentation for discussed before: np.minmax().  My
> thinking is that it would return a 2xN array
> >
> > How about a tuple: (min, max)?
>
> Consider the case of np.minmax(matrix, axis=1), you will end up with a
> tuple of two arrays. In that scenario, you probably want to do
> computations with both numbers, so having them in an array seems more
> convenient.
>
> If there is enough reason, we could always add a "unpack=True" flag
> and then return a tuple.
> _______________________________________________
> NumPy-Discussion mailing list
> NumPy-Discussion at scipy.org
> http://mail.scipy.org/mailman/listinfo/numpy-discussion
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20130501/d9eec616/attachment.html>

From chris.barker at noaa.gov  Wed May  1 12:58:17 2013
From: chris.barker at noaa.gov (Chris Barker - NOAA Federal)
Date: Wed, 1 May 2013 09:58:17 -0700
Subject: [Numpy-discussion] nanmean(),
 nanstd() and other "missing" functions for 1.8
In-Reply-To: <CANNq6FnZ4LOA8F1efJW+ZEeJtS3gwYag4mbvOMGSOt=rj_TwsQ@mail.gmail.com>
References: <CANNq6F=NCdd1cRWmp7rskr6NJt3sjVvinN1XQVcXjJRgiGk=_A@mail.gmail.com>
	<1888613198476891170@unknownmsgid>
	<CANNq6FnZ4LOA8F1efJW+ZEeJtS3gwYag4mbvOMGSOt=rj_TwsQ@mail.gmail.com>
Message-ID: <CALGmxELuBevPwVyO+g6ZktCiifZw5XXCY5CSXFiuBvhD8a_G=w@mail.gmail.com>

On Wed, May 1, 2013 at 6:52 AM, Benjamin Root <ben.root at ou.edu> wrote:

> How about a tuple: (min, max)?
>
>>
>>
> I am not familiar enough with numpy internals to know which is the better
> approach to implement.  I kind of feel that the 2xN array approach would be
> more flexible in case a user wants all of this information in a single
> array, while still allowing for unpacking as if it was a tuple.
>

hmm, my thinking is that the min and max values really are two different
results, so getting two separate arrays makes sense to me.

however, you are right, Python's nifty generic sequence unpacking lets you
use a (2X...) array similarly to a tuple of two arrays, so why not?

Food for thought on one reason:

min, max = np.minmax(arr)

would result in two arrays, but they would be views on the same array, Not
sure if that matters, though.

-Chris



-Chris




-- 

Christopher Barker, Ph.D.
Oceanographer

Emergency Response Division
NOAA/NOS/OR&R            (206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115       (206) 526-6317   main reception

Chris.Barker at noaa.gov
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20130501/9d239f68/attachment.html>

From matthew.brett at gmail.com  Wed May  1 13:24:37 2013
From: matthew.brett at gmail.com (Matthew Brett)
Date: Wed, 1 May 2013 10:24:37 -0700
Subject: [Numpy-discussion] could anyone check on a 32bit system?
In-Reply-To: <20130501160948.GR5140@onerussian.com>
References: <20130501030849.GO5140@onerussian.com>
	<CAH6Pt5ryQQX6jqSUQNH_416RXFD2+1dGPV1rRn9DK-ZvZkC9Zg@mail.gmail.com>
	<CAH6Pt5oAD_U6kiH4QFFp4fQuPzM9OgL=He0D2sL5Wy7NMCa3hg@mail.gmail.com>
	<1367398813.2545.11.camel@sebastian-laptop>
	<1367399295.2545.14.camel@sebastian-laptop>
	<20130501124933.GP5140@onerussian.com>
	<CAPJVwBmnQy_jaqFfJO0m-tMLXpdPrAx8H66NGYkZmhO43L7tRA@mail.gmail.com>
	<20130501130125.GQ5140@onerussian.com> <klr4b3$shp$1@ger.gmane.org>
	<CAPJVwBkBiHFhx0jEE444zgK5bvQFQa0CkX+9M3+OfXfJz_2+6g@mail.gmail.com>
	<20130501160948.GR5140@onerussian.com>
Message-ID: <CAH6Pt5pF-F-xERj5pvozw890dO0=LAd62LT8NqOgpOyvz-vYaQ@mail.gmail.com>

HI,

On Wed, May 1, 2013 at 9:09 AM, Yaroslav Halchenko <lists at onerussian.com> wrote:
>
> On Wed, 01 May 2013, Nathaniel Smith wrote:
>> > not sure there is anything to fix here. Third-party code relying on a
>> > certain outcome of rounding error is likely incorrect anyway.
>
>> Yeah, seems to just be the standard floating point indeterminism.
>> Using Matthew's numbers and pure Python floats:
>
>> In [9]: (0.49505185 + 0.53529587) + -0.13461665
>> Out[9]: 0.89573107
>
>> In [10]: 0.49505185 + (0.53529587 + -0.13461665)
>> Out[10]: 0.8957310700000001
>
>> In [11]: _9 - _10
>> Out[11]: -1.1102230246251565e-16
>
>> Looks like a bug in pymvpa or its test suite to me.
>
> well -- sure thing we will "fix" the unittest to not rely on precise
> correspondence any longer since released 1.7.1 is effected.  So it is not
> a matter of me avoiding "fixing" pymvpa's "bug".
>
> I brought it to your attention because
>
> 1. from e.g.
>
> np.sum(data[:, 0]) - np.sum(data, axis=0)[0]
>
> which presumably should be the same order of additions for 0-th column it is
> not clear that they do not have to be identical.

I agree it's surprising, but I guess it's reasonable for numpy to
reserve the right to add these guys up in whatever order it chooses,
and (in this case) maybe a different order for the axis=None, axis=X
cases.

Also, y'all may have noticed that it is the presence of the second
vector in the array which causes the difference in the sums of the
first (see my first email in this thread).   If this is an order
effect I guess this means that the order of operations in an sum(a,
axis=X) operation depends on the shape of the array.   And it looks
like it depends on memory layout:

In [24]: data = np.array([[ 0.49505185,  0],
   ....:        [ 0.53529587,  0],
   ....:        [-0.13461665, 0]])

In [25]: np.sum(data[:, 0]) - np.sum(data, axis=0)[0]
Out[25]: 1.1102230246251565e-16

In [26]: data_F = np.array(data, order='F')

In [27]: np.sum(data_F[:, 0]) - np.sum(data_F, axis=0)[0]
Out[27]: 0.0

Do we allow the results to be different for different memory layout?

> 2. so far they were identical across many numpy releases
>
> 3. they are identical on other architectures (e.g. amd64)

To me that is surprising.  I would have guessed that the order is the
same on 32 and 64 bit, but something about the precision of
intermediate operations is different.   I don't know enough about
amd64 to guess what that could be.  Bradley's suggestion seems kind of
reasonable but it's strange that numpy should use intel-80 bit
intermediate values differently for 32 and 64 bit.

Cheers,

Matthew


From ben.root at ou.edu  Wed May  1 13:50:18 2013
From: ben.root at ou.edu (Benjamin Root)
Date: Wed, 1 May 2013 13:50:18 -0400
Subject: [Numpy-discussion] Proposal of new function: iteraxis()
In-Reply-To: <CAKF8KOftSdcUR7w3kOrfjP=NtPJDQ_N45DL883WaVapraUbOfQ@mail.gmail.com>
References: <CAKF8KOcsm42_VCqAXkvridSewsF=zDUkXJP6K3HtJpUiDKXK6w@mail.gmail.com>
	<CAF6FJitXxPEox7TtMic77fjhEhELwBSy+1ZyDMq4MC-8Rv2Y_g@mail.gmail.com>
	<CAH6Pt5q07bkJbYCb4cW97NYf-5vdjr-R_axXvzJ2XKSgUgccnA@mail.gmail.com>
	<CAF6FJisPwpEoheYSW+aY66+_Gf693yiE9SNgC-67+csryTLokg@mail.gmail.com>
	<CAH6Pt5rfH6qEkM+6MRRjLEGENciQ-Fhdv258Vyrp_tsj35+K_A@mail.gmail.com>
	<CAF6FJiudk2Ltj+QHTQ_UisW1SnYmHTxmWFFATQbnuh=FHz2HvA@mail.gmail.com>
	<20130426205002.GA4942@phare.normalesup.org>
	<CAKF8KOftSdcUR7w3kOrfjP=NtPJDQ_N45DL883WaVapraUbOfQ@mail.gmail.com>
Message-ID: <CANNq6F=Qxubntr4-CxwUwVR_Nk77F-zUX+-g7pRQ_7P9cvggTA@mail.gmail.com>

On Mon, Apr 29, 2013 at 2:10 PM, Andrew Giessel <
andrew_giessel at hms.harvard.edu> wrote:

> Matthew:  Thanks for the link to array order discussion.
>
> Any more thoughts on Phil's slice() function?
>
>
>
I rather like Phil's solution.  Just some caveats.  Will it always return
views or copies?  It should be one or the other (I haven't looked closely
enough to check), and it should be documented to that affect.  Plus, tests
should be added to make sure it does that.

Cheers!
Ben Root
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20130501/9c1b6a0d/attachment.html>

From robert.kern at gmail.com  Wed May  1 14:07:14 2013
From: robert.kern at gmail.com (Robert Kern)
Date: Wed, 1 May 2013 19:07:14 +0100
Subject: [Numpy-discussion] could anyone check on a 32bit system?
In-Reply-To: <CAH6Pt5pF-F-xERj5pvozw890dO0=LAd62LT8NqOgpOyvz-vYaQ@mail.gmail.com>
References: <20130501030849.GO5140@onerussian.com>
	<CAH6Pt5ryQQX6jqSUQNH_416RXFD2+1dGPV1rRn9DK-ZvZkC9Zg@mail.gmail.com>
	<CAH6Pt5oAD_U6kiH4QFFp4fQuPzM9OgL=He0D2sL5Wy7NMCa3hg@mail.gmail.com>
	<1367398813.2545.11.camel@sebastian-laptop>
	<1367399295.2545.14.camel@sebastian-laptop>
	<20130501124933.GP5140@onerussian.com>
	<CAPJVwBmnQy_jaqFfJO0m-tMLXpdPrAx8H66NGYkZmhO43L7tRA@mail.gmail.com>
	<20130501130125.GQ5140@onerussian.com> <klr4b3$shp$1@ger.gmane.org>
	<CAPJVwBkBiHFhx0jEE444zgK5bvQFQa0CkX+9M3+OfXfJz_2+6g@mail.gmail.com>
	<20130501160948.GR5140@onerussian.com>
	<CAH6Pt5pF-F-xERj5pvozw890dO0=LAd62LT8NqOgpOyvz-vYaQ@mail.gmail.com>
Message-ID: <CAF6FJivU9+7R4zk+88PaAcC=oMZTn9iW8vhVSTUw0T6f3NKrew@mail.gmail.com>

On Wed, May 1, 2013 at 6:24 PM, Matthew Brett <matthew.brett at gmail.com> wrote:
> HI,
>
> On Wed, May 1, 2013 at 9:09 AM, Yaroslav Halchenko <lists at onerussian.com> wrote:

>> 3. they are identical on other architectures (e.g. amd64)
>
> To me that is surprising.  I would have guessed that the order is the
> same on 32 and 64 bit, but something about the precision of
> intermediate operations is different.   I don't know enough about
> amd64 to guess what that could be.  Bradley's suggestion seems kind of
> reasonable but it's strange that numpy should use intel-80 bit
> intermediate values differently for 32 and 64 bit.

"numpy" isn't doing anything different between the two. numpy
generates the same C code. The C compiler may be generating different
machine instructions for that code on different architectures, even
closely related ones like i386 and amd64. Different optimization flags
and compiler versions will probably also affect this, not just the
target architecture. It's possible that those are actually the source
of this observation.

--
Robert Kern


From blake.a.griffith at gmail.com  Wed May  1 14:12:46 2013
From: blake.a.griffith at gmail.com (Blake Griffith)
Date: Wed, 1 May 2013 13:12:46 -0500
Subject: [Numpy-discussion] GSoC proposal -- Numpy SciPy
In-Reply-To: <CAPJVwBmaDbh_v3fY_1msJqm-_tCd0pF+4V=EK+=_KBbNQDuz7g@mail.gmail.com>
References: <CAOiFMpwKXRGdRU_xPo1XTTjaiLB+vxOYuV29Nhsdw1rJxh364w@mail.gmail.com>
	<CAPJVwBmE7d5tcV53mM2NDC07Gj7XEF3U7j+JGdVjXpTh58aBbA@mail.gmail.com>
	<klp80b$qbl$1@ger.gmane.org>
	<CAPJVwBmaDbh_v3fY_1msJqm-_tCd0pF+4V=EK+=_KBbNQDuz7g@mail.gmail.com>
Message-ID: <CAOiFMpxEHLLwt3e8pKGWDH9qJqcfL+hvo_hM0sG3SBRZm6NRQA@mail.gmail.com>

Oh wow, I just assumed that `dot` was a ufunc... However, it would still be
useful to have ufuncs working well with the sparse package. I don't
understand everything that is going on in
https://github.com/numpy/numpy/blob/master/numpy/core/src/umath/ufunc_object.c

But I assumed that I would be able to add the ability to check for
something like _ufunc_override_. I'm not sure where this piece of logic
should be inserted, or what the performance implications to NumPy would
be... I'm trying to figure this out. But major optimizations to ufuncs is
out of the scope of this GSoC.

I will look into what can be done about the `dot` function.


On Tue, Apr 30, 2013 at 6:53 PM, Nathaniel Smith <njs at pobox.com> wrote:

> On Tue, Apr 30, 2013 at 4:02 PM, Pauli Virtanen <pav at iki.fi> wrote:
> > 30.04.2013 22:37, Nathaniel Smith kirjoitti:
> > [clip]
> >> How do you plan to go about this? The obvious option of just calling
> >> scipy.sparse.issparse() on ufunc entry raises some problems, since
> >> numpy can't depend on or even import scipy, and we might be reluctant
> >> to add such a special case for what's a rather more general problem.
> >> OTOH it might be possible to solve the problem in general, e.g., see
> >> the prototyped _ufunc_override_ special method in:
> >>
> >>   https://github.com/njsmith/numpyNEP/blob/master/numpyNEP.py
> >>
> >> but I don't know if you want to get into such a debate within the
> >> scope of your GSoC. What were you thinking?
> >
> > To me it seems that the right thing to do here is the general solution.
> >
> > Do you see immediate problems in e.g. just enabling something like your
> > _ufunc_override_?
>
> Just that we might want to think a bit about the design space before
> implementing something. E.g., apparently doing Python attribute lookup
> is very expensive -- we recently had a patch to skip
> __array_interface__ checks whenever possible -- is adding another such
> per-operation overhead ok? I guess we could use similar checks (skip
> checking for known types like int/float/ndarray), or only check for
> _ufunc_override_ on the class (not the instance) and cache the result
> per-class?
>
> > The easy thing is that there are no backward compatibility problems
> > here, since if the magic is missing, the old logic is used. Currently,
> > the numpy dot() and ufuncs also most of the time do nothing sensible
> > with sparse matrix inputs even though they in some cases return values.
> > Which then makes writing generic sparse/dense code more painful than
> > just __mul__ being matrix multiplication.
>
> I agree, but, if the main target is 'dot' then the current
> _ufunc_override_ design alone won't do it, since 'dot' is not a
> ufunc...
>
> -n
> _______________________________________________
> NumPy-Discussion mailing list
> NumPy-Discussion at scipy.org
> http://mail.scipy.org/mailman/listinfo/numpy-discussion
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20130501/e567347c/attachment.html>

From davidmenhur at gmail.com  Wed May  1 14:32:42 2013
From: davidmenhur at gmail.com (=?UTF-8?B?RGHPgGlk?=)
Date: Wed, 1 May 2013 20:32:42 +0200
Subject: [Numpy-discussion] GSoC proposal -- Numpy SciPy
In-Reply-To: <CAOiFMpxEHLLwt3e8pKGWDH9qJqcfL+hvo_hM0sG3SBRZm6NRQA@mail.gmail.com>
References: <CAOiFMpwKXRGdRU_xPo1XTTjaiLB+vxOYuV29Nhsdw1rJxh364w@mail.gmail.com>
	<CAPJVwBmE7d5tcV53mM2NDC07Gj7XEF3U7j+JGdVjXpTh58aBbA@mail.gmail.com>
	<klp80b$qbl$1@ger.gmane.org>
	<CAPJVwBmaDbh_v3fY_1msJqm-_tCd0pF+4V=EK+=_KBbNQDuz7g@mail.gmail.com>
	<CAOiFMpxEHLLwt3e8pKGWDH9qJqcfL+hvo_hM0sG3SBRZm6NRQA@mail.gmail.com>
Message-ID: <CAJhcF=38iwwSDPLGYSNBD6HLvY3-_OkbAMEBYAdV_scGouhqoA@mail.gmail.com>

On 1 May 2013 20:12, Blake Griffith <blake.a.griffith at gmail.com> wrote:
> However, it would still be useful to have ufuncs working well with the
> sparse package.

How are you planning to deal with ufunc(0) != 0? cos(sparse) is actually dense.


From raul at virtualmaterials.com  Wed May  1 14:39:35 2013
From: raul at virtualmaterials.com (Raul Cota)
Date: Wed, 01 May 2013 12:39:35 -0600
Subject: [Numpy-discussion] GSoC : Performance parity between numpy
 arrays and Python scalars
In-Reply-To: <CAJAs4rQ4qQmo4sy5NM=CU9XWSrShOxBn0ti9f-Z4NiZhhehewQ@mail.gmail.com>
References: <CAJAs4rQ4qQmo4sy5NM=CU9XWSrShOxBn0ti9f-Z4NiZhhehewQ@mail.gmail.com>
Message-ID: <51816167.6040607@virtualmaterials.com>

An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20130501/958eb35f/attachment.html>

From blake.a.griffith at gmail.com  Wed May  1 15:02:55 2013
From: blake.a.griffith at gmail.com (Blake Griffith)
Date: Wed, 1 May 2013 14:02:55 -0500
Subject: [Numpy-discussion] GSoC proposal -- Numpy SciPy
In-Reply-To: <CAJhcF=38iwwSDPLGYSNBD6HLvY3-_OkbAMEBYAdV_scGouhqoA@mail.gmail.com>
References: <CAOiFMpwKXRGdRU_xPo1XTTjaiLB+vxOYuV29Nhsdw1rJxh364w@mail.gmail.com>
	<CAPJVwBmE7d5tcV53mM2NDC07Gj7XEF3U7j+JGdVjXpTh58aBbA@mail.gmail.com>
	<klp80b$qbl$1@ger.gmane.org>
	<CAPJVwBmaDbh_v3fY_1msJqm-_tCd0pF+4V=EK+=_KBbNQDuz7g@mail.gmail.com>
	<CAOiFMpxEHLLwt3e8pKGWDH9qJqcfL+hvo_hM0sG3SBRZm6NRQA@mail.gmail.com>
	<CAJhcF=38iwwSDPLGYSNBD6HLvY3-_OkbAMEBYAdV_scGouhqoA@mail.gmail.com>
Message-ID: <CAOiFMpz2EeMdn3R9c6D3k4mWBZ5D4dGi+goM+mtPt57MNbS=Mg@mail.gmail.com>

There are several situations where that comes up (Like comparing two sparse
matrices A == B) There is a SparseEfficiancyWarning that can be thrown, but
the way this should be implemented still needs to be discussed. I will be
writing a specification on how ufuncs and ndarrays are handled by the
sparse package, the spec can be found here
https://github.com/cowlicks/scipy-sparse-ndarray-and-ufunc-spec/blob/master/Spec.markdown.
In general, a unary ufunc operating on a sparse matrix should return a
sparse matrix.

If you really want to do cos(sparse) you will be able to. But if you are
just interested in the initially non zero elements should probably do
something like: sparse.data = np.cos(sparse.data)



On Wed, May 1, 2013 at 1:32 PM, Da?id <davidmenhur at gmail.com> wrote:

> On 1 May 2013 20:12, Blake Griffith <blake.a.griffith at gmail.com> wrote:
> > However, it would still be useful to have ufuncs working well with the
> > sparse package.
>
> How are you planning to deal with ufunc(0) != 0? cos(sparse) is actually
> dense.
> _______________________________________________
> NumPy-Discussion mailing list
> NumPy-Discussion at scipy.org
> http://mail.scipy.org/mailman/listinfo/numpy-discussion
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20130501/7fb57211/attachment.html>

From lists at onerussian.com  Wed May  1 15:29:07 2013
From: lists at onerussian.com (Yaroslav Halchenko)
Date: Wed, 1 May 2013 15:29:07 -0400
Subject: [Numpy-discussion] could anyone check on a 32bit system?
In-Reply-To: <CAF6FJivU9+7R4zk+88PaAcC=oMZTn9iW8vhVSTUw0T6f3NKrew@mail.gmail.com>
References: <1367398813.2545.11.camel@sebastian-laptop>
	<1367399295.2545.14.camel@sebastian-laptop>
	<20130501124933.GP5140@onerussian.com>
	<CAPJVwBmnQy_jaqFfJO0m-tMLXpdPrAx8H66NGYkZmhO43L7tRA@mail.gmail.com>
	<20130501130125.GQ5140@onerussian.com> <klr4b3$shp$1@ger.gmane.org>
	<CAPJVwBkBiHFhx0jEE444zgK5bvQFQa0CkX+9M3+OfXfJz_2+6g@mail.gmail.com>
	<20130501160948.GR5140@onerussian.com>
	<CAH6Pt5pF-F-xERj5pvozw890dO0=LAd62LT8NqOgpOyvz-vYaQ@mail.gmail.com>
	<CAF6FJivU9+7R4zk+88PaAcC=oMZTn9iW8vhVSTUw0T6f3NKrew@mail.gmail.com>
Message-ID: <20130501192907.GS5140@onerussian.com>

just for completeness... I haven't yet double checked if I have done it
correctly but here is the bisected commit:

aed9925a9d5fe9a407d0ca2c65cb577116c4d0f1 is the first bad commit
commit aed9925a9d5fe9a407d0ca2c65cb577116c4d0f1
Author: Mark Wiebe <mwiebe at enthought.com>
Date:   Tue Aug 2 13:34:13 2011 -0500

    ENH: ufunc: Rewrite PyUFunc_Reduce to be more general and easier to adapt to NA masks
    
    This generalizes the 'axis' parameter to accept None or a list of
    axes on which to do the reduction.

:040000 040000 2bdd71a1ea60c0dbfe370c77f69724fab28038e1 44f54a15f480ccaf519d10e9c42032de86bd0dca M      numpy
bisect run success

FWIW ( ;-) ):

# git describe --tags aed9925a9d5fe9a407d0ca2c65cb577116c4d0f1
v0.3.0-7757-gaed9925

# git describe --tags --contains aed9925a9d5fe9a407d0ca2c65cb577116c4d0f1
v1.7.0b1~377^2~126


On Wed, 01 May 2013, Robert Kern wrote:

> On Wed, May 1, 2013 at 6:24 PM, Matthew Brett <matthew.brett at gmail.com> wrote:
> > HI,

> > On Wed, May 1, 2013 at 9:09 AM, Yaroslav Halchenko <lists at onerussian.com> wrote:

> >> 3. they are identical on other architectures (e.g. amd64)

> > To me that is surprising.  I would have guessed that the order is the
> > same on 32 and 64 bit, but something about the precision of
> > intermediate operations is different.   I don't know enough about
> > amd64 to guess what that could be.  Bradley's suggestion seems kind of
> > reasonable but it's strange that numpy should use intel-80 bit
> > intermediate values differently for 32 and 64 bit.

> "numpy" isn't doing anything different between the two. numpy
> generates the same C code. The C compiler may be generating different
> machine instructions for that code on different architectures, even
> closely related ones like i386 and amd64. Different optimization flags
> and compiler versions will probably also affect this, not just the
> target architecture. It's possible that those are actually the source
> of this observation.
-- 
Yaroslav O. Halchenko, Ph.D.
http://neuro.debian.net http://www.pymvpa.org http://www.fail2ban.org
Senior Research Associate,     Psychological and Brain Sciences Dept.
Dartmouth College, 419 Moore Hall, Hinman Box 6207, Hanover, NH 03755
Phone: +1 (603) 646-9834                       Fax: +1 (603) 646-1419
WWW:   http://www.linkedin.com/in/yarik        


From josef.pktd at gmail.com  Wed May  1 15:50:15 2013
From: josef.pktd at gmail.com (josef.pktd at gmail.com)
Date: Wed, 1 May 2013 15:50:15 -0400
Subject: [Numpy-discussion] could anyone check on a 32bit system?
In-Reply-To: <CAH6Pt5pF-F-xERj5pvozw890dO0=LAd62LT8NqOgpOyvz-vYaQ@mail.gmail.com>
References: <20130501030849.GO5140@onerussian.com>
	<CAH6Pt5ryQQX6jqSUQNH_416RXFD2+1dGPV1rRn9DK-ZvZkC9Zg@mail.gmail.com>
	<CAH6Pt5oAD_U6kiH4QFFp4fQuPzM9OgL=He0D2sL5Wy7NMCa3hg@mail.gmail.com>
	<1367398813.2545.11.camel@sebastian-laptop>
	<1367399295.2545.14.camel@sebastian-laptop>
	<20130501124933.GP5140@onerussian.com>
	<CAPJVwBmnQy_jaqFfJO0m-tMLXpdPrAx8H66NGYkZmhO43L7tRA@mail.gmail.com>
	<20130501130125.GQ5140@onerussian.com> <klr4b3$shp$1@ger.gmane.org>
	<CAPJVwBkBiHFhx0jEE444zgK5bvQFQa0CkX+9M3+OfXfJz_2+6g@mail.gmail.com>
	<20130501160948.GR5140@onerussian.com>
	<CAH6Pt5pF-F-xERj5pvozw890dO0=LAd62LT8NqOgpOyvz-vYaQ@mail.gmail.com>
Message-ID: <CAMMTP+DwcdLi_AtfyjO7+rFyJ_9fOGnA=G-dVFvGxpFESV-NXw@mail.gmail.com>

On Wed, May 1, 2013 at 1:24 PM, Matthew Brett <matthew.brett at gmail.com> wrote:
> HI,
>
> On Wed, May 1, 2013 at 9:09 AM, Yaroslav Halchenko <lists at onerussian.com> wrote:
>>
>> On Wed, 01 May 2013, Nathaniel Smith wrote:
>>> > not sure there is anything to fix here. Third-party code relying on a
>>> > certain outcome of rounding error is likely incorrect anyway.
>>
>>> Yeah, seems to just be the standard floating point indeterminism.
>>> Using Matthew's numbers and pure Python floats:
>>
>>> In [9]: (0.49505185 + 0.53529587) + -0.13461665
>>> Out[9]: 0.89573107
>>
>>> In [10]: 0.49505185 + (0.53529587 + -0.13461665)
>>> Out[10]: 0.8957310700000001
>>
>>> In [11]: _9 - _10
>>> Out[11]: -1.1102230246251565e-16
>>
>>> Looks like a bug in pymvpa or its test suite to me.
>>
>> well -- sure thing we will "fix" the unittest to not rely on precise
>> correspondence any longer since released 1.7.1 is effected.  So it is not
>> a matter of me avoiding "fixing" pymvpa's "bug".
>>
>> I brought it to your attention because
>>
>> 1. from e.g.
>>
>> np.sum(data[:, 0]) - np.sum(data, axis=0)[0]
>>
>> which presumably should be the same order of additions for 0-th column it is
>> not clear that they do not have to be identical.
>
> I agree it's surprising, but I guess it's reasonable for numpy to
> reserve the right to add these guys up in whatever order it chooses,
> and (in this case) maybe a different order for the axis=None, axis=X
> cases.
>
> Also, y'all may have noticed that it is the presence of the second
> vector in the array which causes the difference in the sums of the
> first (see my first email in this thread).   If this is an order
> effect I guess this means that the order of operations in an sum(a,
> axis=X) operation depends on the shape of the array.   And it looks
> like it depends on memory layout:
>
> In [24]: data = np.array([[ 0.49505185,  0],
>    ....:        [ 0.53529587,  0],
>    ....:        [-0.13461665, 0]])
>
> In [25]: np.sum(data[:, 0]) - np.sum(data, axis=0)[0]
> Out[25]: 1.1102230246251565e-16
>
> In [26]: data_F = np.array(data, order='F')
>
> In [27]: np.sum(data_F[:, 0]) - np.sum(data_F, axis=0)[0]
> Out[27]: 0.0
>
> Do we allow the results to be different for different memory layout?

Wasn't this the point of some of Mark Wiebe's optimization?

As far as I understand he got rid of some of the C bias, and made
calculations faster for Fortran contiguous arrays.

I rather have speed for my Fortran arrays, then relying on float
precision issues, that bite anyway when running on many different
kinds
of machines. (I usually have to lower some unit test precision during
Debian testing of statsmodels.)

Josef

>
>> 2. so far they were identical across many numpy releases
>>
>> 3. they are identical on other architectures (e.g. amd64)
>
> To me that is surprising.  I would have guessed that the order is the
> same on 32 and 64 bit, but something about the precision of
> intermediate operations is different.   I don't know enough about
> amd64 to guess what that could be.  Bradley's suggestion seems kind of
> reasonable but it's strange that numpy should use intel-80 bit
> intermediate values differently for 32 and 64 bit.
>
> Cheers,
>
> Matthew
> _______________________________________________
> NumPy-Discussion mailing list
> NumPy-Discussion at scipy.org
> http://mail.scipy.org/mailman/listinfo/numpy-discussion


From sebastian at sipsolutions.net  Wed May  1 16:01:00 2013
From: sebastian at sipsolutions.net (Sebastian Berg)
Date: Wed, 01 May 2013 22:01:00 +0200
Subject: [Numpy-discussion] could anyone check on a 32bit system?
In-Reply-To: <20130501192907.GS5140@onerussian.com>
References: <1367398813.2545.11.camel@sebastian-laptop>
	<1367399295.2545.14.camel@sebastian-laptop>
	<20130501124933.GP5140@onerussian.com>
	<CAPJVwBmnQy_jaqFfJO0m-tMLXpdPrAx8H66NGYkZmhO43L7tRA@mail.gmail.com>
	<20130501130125.GQ5140@onerussian.com> <klr4b3$shp$1@ger.gmane.org>
	<CAPJVwBkBiHFhx0jEE444zgK5bvQFQa0CkX+9M3+OfXfJz_2+6g@mail.gmail.com>
	<20130501160948.GR5140@onerussian.com>
	<CAH6Pt5pF-F-xERj5pvozw890dO0=LAd62LT8NqOgpOyvz-vYaQ@mail.gmail.com>
	<CAF6FJivU9+7R4zk+88PaAcC=oMZTn9iW8vhVSTUw0T6f3NKrew@mail.gmail.com>
	<20130501192907.GS5140@onerussian.com>
Message-ID: <1367438460.2545.22.camel@sebastian-laptop>

On Wed, 2013-05-01 at 15:29 -0400, Yaroslav Halchenko wrote:
> just for completeness... I haven't yet double checked if I have done it
> correctly but here is the bisected commit:
> 
> aed9925a9d5fe9a407d0ca2c65cb577116c4d0f1 is the first bad commit
> commit aed9925a9d5fe9a407d0ca2c65cb577116c4d0f1
> Author: Mark Wiebe <mwiebe at enthought.com>
> Date:   Tue Aug 2 13:34:13 2011 -0500
> 
>     ENH: ufunc: Rewrite PyUFunc_Reduce to be more general and easier to adapt to NA masks
>     
>     This generalizes the 'axis' parameter to accept None or a list of
>     axes on which to do the reduction.
> 
> :040000 040000 2bdd71a1ea60c0dbfe370c77f69724fab28038e1 44f54a15f480ccaf519d10e9c42032de86bd0dca M      numpy
> bisect run success
> 
> FWIW ( ;-) ):
> 

There really is no point discussing here, this has to do with numpy
doing iteration order optimization, and you actually *want* this. Lets
for a second assume that the old behavior was better, then the next guy
is going to ask: "Why is np.add.reduce(array, axis=0) so much slower
then reduce(array, np.add)?". This is huge speed improvement by Marks
new iterator for reductions over the slow axes, so instead of trying to
track "regressions" down, I think the right thing is to say kudos for
doing this improvement :).

Just my opinion,

Sebastian

> # git describe --tags aed9925a9d5fe9a407d0ca2c65cb577116c4d0f1
> v0.3.0-7757-gaed9925
> 
> # git describe --tags --contains aed9925a9d5fe9a407d0ca2c65cb577116c4d0f1
> v1.7.0b1~377^2~126
> 
> 
> On Wed, 01 May 2013, Robert Kern wrote:
> 
> > On Wed, May 1, 2013 at 6:24 PM, Matthew Brett <matthew.brett at gmail.com> wrote:
> > > HI,
> 
> > > On Wed, May 1, 2013 at 9:09 AM, Yaroslav Halchenko <lists at onerussian.com> wrote:
> 
> > >> 3. they are identical on other architectures (e.g. amd64)
> 
> > > To me that is surprising.  I would have guessed that the order is the
> > > same on 32 and 64 bit, but something about the precision of
> > > intermediate operations is different.   I don't know enough about
> > > amd64 to guess what that could be.  Bradley's suggestion seems kind of
> > > reasonable but it's strange that numpy should use intel-80 bit
> > > intermediate values differently for 32 and 64 bit.
> 
> > "numpy" isn't doing anything different between the two. numpy
> > generates the same C code. The C compiler may be generating different
> > machine instructions for that code on different architectures, even
> > closely related ones like i386 and amd64. Different optimization flags
> > and compiler versions will probably also affect this, not just the
> > target architecture. It's possible that those are actually the source
> > of this observation.




From matthew.brett at gmail.com  Wed May  1 16:19:39 2013
From: matthew.brett at gmail.com (Matthew Brett)
Date: Wed, 1 May 2013 13:19:39 -0700
Subject: [Numpy-discussion] could anyone check on a 32bit system?
In-Reply-To: <1367438460.2545.22.camel@sebastian-laptop>
References: <1367398813.2545.11.camel@sebastian-laptop>
	<1367399295.2545.14.camel@sebastian-laptop>
	<20130501124933.GP5140@onerussian.com>
	<CAPJVwBmnQy_jaqFfJO0m-tMLXpdPrAx8H66NGYkZmhO43L7tRA@mail.gmail.com>
	<20130501130125.GQ5140@onerussian.com> <klr4b3$shp$1@ger.gmane.org>
	<CAPJVwBkBiHFhx0jEE444zgK5bvQFQa0CkX+9M3+OfXfJz_2+6g@mail.gmail.com>
	<20130501160948.GR5140@onerussian.com>
	<CAH6Pt5pF-F-xERj5pvozw890dO0=LAd62LT8NqOgpOyvz-vYaQ@mail.gmail.com>
	<CAF6FJivU9+7R4zk+88PaAcC=oMZTn9iW8vhVSTUw0T6f3NKrew@mail.gmail.com>
	<20130501192907.GS5140@onerussian.com>
	<1367438460.2545.22.camel@sebastian-laptop>
Message-ID: <CAH6Pt5osZpvkkca_mJ7e+iNgY=VyjLuW8D+RjQGc6WiZWoc8Eg@mail.gmail.com>

Hi,

On Wed, May 1, 2013 at 1:01 PM, Sebastian Berg
<sebastian at sipsolutions.net> wrote:
> On Wed, 2013-05-01 at 15:29 -0400, Yaroslav Halchenko wrote:
>> just for completeness... I haven't yet double checked if I have done it
>> correctly but here is the bisected commit:
>>
>> aed9925a9d5fe9a407d0ca2c65cb577116c4d0f1 is the first bad commit
>> commit aed9925a9d5fe9a407d0ca2c65cb577116c4d0f1
>> Author: Mark Wiebe <mwiebe at enthought.com>
>> Date:   Tue Aug 2 13:34:13 2011 -0500
>>
>>     ENH: ufunc: Rewrite PyUFunc_Reduce to be more general and easier to adapt to NA masks
>>
>>     This generalizes the 'axis' parameter to accept None or a list of
>>     axes on which to do the reduction.
>>
>> :040000 040000 2bdd71a1ea60c0dbfe370c77f69724fab28038e1 44f54a15f480ccaf519d10e9c42032de86bd0dca M      numpy
>> bisect run success
>>
>> FWIW ( ;-) ):
>>
>
> There really is no point discussing here, this has to do with numpy
> doing iteration order optimization, and you actually *want* this. Lets
> for a second assume that the old behavior was better, then the next guy
> is going to ask: "Why is np.add.reduce(array, axis=0) so much slower
> then reduce(array, np.add)?". This is huge speed improvement by Marks
> new iterator for reductions over the slow axes, so instead of trying to
> track "regressions" down, I think the right thing is to say kudos for
> doing this improvement :).

I don't believe Yarick meant his bisection to be a criticism, but as
an aid to full understanding.

Is it an issue that Fortran and C contiguous arrays give different
rounding error for the sums?

Cheers,

Matthew


From lists at onerussian.com  Wed May  1 16:33:27 2013
From: lists at onerussian.com (Yaroslav Halchenko)
Date: Wed, 1 May 2013 16:33:27 -0400
Subject: [Numpy-discussion] could anyone check on a 32bit system?
In-Reply-To: <CAH6Pt5osZpvkkca_mJ7e+iNgY=VyjLuW8D+RjQGc6WiZWoc8Eg@mail.gmail.com>
References: <CAPJVwBmnQy_jaqFfJO0m-tMLXpdPrAx8H66NGYkZmhO43L7tRA@mail.gmail.com>
	<20130501130125.GQ5140@onerussian.com> <klr4b3$shp$1@ger.gmane.org>
	<CAPJVwBkBiHFhx0jEE444zgK5bvQFQa0CkX+9M3+OfXfJz_2+6g@mail.gmail.com>
	<20130501160948.GR5140@onerussian.com>
	<CAH6Pt5pF-F-xERj5pvozw890dO0=LAd62LT8NqOgpOyvz-vYaQ@mail.gmail.com>
	<CAF6FJivU9+7R4zk+88PaAcC=oMZTn9iW8vhVSTUw0T6f3NKrew@mail.gmail.com>
	<20130501192907.GS5140@onerussian.com>
	<1367438460.2545.22.camel@sebastian-laptop>
	<CAH6Pt5osZpvkkca_mJ7e+iNgY=VyjLuW8D+RjQGc6WiZWoc8Eg@mail.gmail.com>
Message-ID: <20130501203327.GT5140@onerussian.com>


On Wed, 01 May 2013, Matthew Brett wrote:

> > There really is no point discussing here, this has to do with numpy
> > doing iteration order optimization, and you actually *want* this. Lets
> > for a second assume that the old behavior was better, then the next guy
> > is going to ask: "Why is np.add.reduce(array, axis=0) so much slower
> > then reduce(array, np.add)?". This is huge speed improvement by Marks
> > new iterator for reductions over the slow axes, so instead of trying to
> > track "regressions" down, I think the right thing is to say kudos for
> > doing this improvement :).

> I don't believe Yarick meant his bisection to be a criticism, but as
> an aid to full understanding.

Exactly right, Matthew -- thank you!  And kudos to Mark!

N.B. I am generally furry and kind, not fuzzy and evil

-- 
Yaroslav O. Halchenko, Ph.D.
http://neuro.debian.net http://www.pymvpa.org http://www.fail2ban.org
Senior Research Associate,     Psychological and Brain Sciences Dept.
Dartmouth College, 419 Moore Hall, Hinman Box 6207, Hanover, NH 03755
Phone: +1 (603) 646-9834                       Fax: +1 (603) 646-1419
WWW:   http://www.linkedin.com/in/yarik        


From lists at onerussian.com  Wed May  1 16:37:07 2013
From: lists at onerussian.com (Yaroslav Halchenko)
Date: Wed, 1 May 2013 16:37:07 -0400
Subject: [Numpy-discussion] could anyone check on a 32bit system?
In-Reply-To: <1367438460.2545.22.camel@sebastian-laptop>
References: <20130501124933.GP5140@onerussian.com>
	<CAPJVwBmnQy_jaqFfJO0m-tMLXpdPrAx8H66NGYkZmhO43L7tRA@mail.gmail.com>
	<20130501130125.GQ5140@onerussian.com> <klr4b3$shp$1@ger.gmane.org>
	<CAPJVwBkBiHFhx0jEE444zgK5bvQFQa0CkX+9M3+OfXfJz_2+6g@mail.gmail.com>
	<20130501160948.GR5140@onerussian.com>
	<CAH6Pt5pF-F-xERj5pvozw890dO0=LAd62LT8NqOgpOyvz-vYaQ@mail.gmail.com>
	<CAF6FJivU9+7R4zk+88PaAcC=oMZTn9iW8vhVSTUw0T6f3NKrew@mail.gmail.com>
	<20130501192907.GS5140@onerussian.com>
	<1367438460.2545.22.camel@sebastian-laptop>
Message-ID: <20130501203707.GU5140@onerussian.com>


On Wed, 01 May 2013, Sebastian Berg wrote:

> There really is no point discussing here, this has to do with numpy
> doing iteration order optimization, and you actually *want* this. Lets
> for a second assume that the old behavior was better, then the next guy
> is going to ask: "Why is np.add.reduce(array, axis=0) so much slower
> then reduce(array, np.add)?". This is huge speed improvement by Marks
> new iterator for reductions over the slow axes...

btw -- is there something like panda's vbench for numpy?  i.e. where
it would be possible to track/visualize such performance
improvements/hits?

-- 
Yaroslav O. Halchenko, Ph.D.
http://neuro.debian.net http://www.pymvpa.org http://www.fail2ban.org
Senior Research Associate,     Psychological and Brain Sciences Dept.
Dartmouth College, 419 Moore Hall, Hinman Box 6207, Hanover, NH 03755
Phone: +1 (603) 646-9834                       Fax: +1 (603) 646-1419
WWW:   http://www.linkedin.com/in/yarik        


From sebastian at sipsolutions.net  Wed May  1 16:48:24 2013
From: sebastian at sipsolutions.net (Sebastian Berg)
Date: Wed, 01 May 2013 22:48:24 +0200
Subject: [Numpy-discussion] could anyone check on a 32bit system?
In-Reply-To: <20130501203707.GU5140@onerussian.com>
References: <20130501124933.GP5140@onerussian.com>
	<CAPJVwBmnQy_jaqFfJO0m-tMLXpdPrAx8H66NGYkZmhO43L7tRA@mail.gmail.com>
	<20130501130125.GQ5140@onerussian.com> <klr4b3$shp$1@ger.gmane.org>
	<CAPJVwBkBiHFhx0jEE444zgK5bvQFQa0CkX+9M3+OfXfJz_2+6g@mail.gmail.com>
	<20130501160948.GR5140@onerussian.com>
	<CAH6Pt5pF-F-xERj5pvozw890dO0=LAd62LT8NqOgpOyvz-vYaQ@mail.gmail.com>
	<CAF6FJivU9+7R4zk+88PaAcC=oMZTn9iW8vhVSTUw0T6f3NKrew@mail.gmail.com>
	<20130501192907.GS5140@onerussian.com>
	<1367438460.2545.22.camel@sebastian-laptop>
	<20130501203707.GU5140@onerussian.com>
Message-ID: <1367441304.2545.27.camel@sebastian-laptop>

On Wed, 2013-05-01 at 16:37 -0400, Yaroslav Halchenko wrote:
> On Wed, 01 May 2013, Sebastian Berg wrote:
> 
> > There really is no point discussing here, this has to do with numpy
> > doing iteration order optimization, and you actually *want* this. Lets
> > for a second assume that the old behavior was better, then the next guy
> > is going to ask: "Why is np.add.reduce(array, axis=0) so much slower
> > then reduce(array, np.add)?". This is huge speed improvement by Marks
> > new iterator for reductions over the slow axes...
> 
> btw -- is there something like panda's vbench for numpy?  i.e. where
> it would be possible to track/visualize such performance
> improvements/hits?
> 

Sorry if it seemed harsh, but only skimmed mails and it seemed a bit
like the an obvious piece was missing... There are no benchmark tests I
am aware of. You can try:

a = np.random.random((1000, 1000))

and then time a.sum(1) and a.sum(0), on 1.7. the fast axis (1), is only
slightly faster then the sum over the slow axis. On earlier numpy
versions you will probably see something like half the speed for the
slow axis (only got ancient or 1.7 numpy right now, so reluctant to give
exact timings).

- Sebastian



From ben.root at ou.edu  Wed May  1 21:10:11 2013
From: ben.root at ou.edu (Benjamin Root)
Date: Wed, 1 May 2013 21:10:11 -0400
Subject: [Numpy-discussion] nanmean(),
 nanstd() and other "missing" functions for 1.8
In-Reply-To: <CALGmxELuBevPwVyO+g6ZktCiifZw5XXCY5CSXFiuBvhD8a_G=w@mail.gmail.com>
References: <CANNq6F=NCdd1cRWmp7rskr6NJt3sjVvinN1XQVcXjJRgiGk=_A@mail.gmail.com>
	<1888613198476891170@unknownmsgid>
	<CANNq6FnZ4LOA8F1efJW+ZEeJtS3gwYag4mbvOMGSOt=rj_TwsQ@mail.gmail.com>
	<CALGmxELuBevPwVyO+g6ZktCiifZw5XXCY5CSXFiuBvhD8a_G=w@mail.gmail.com>
Message-ID: <CANNq6FnMnRRH2kBWq_YV-goRTmEojaUBoQNMxy7_F_U5vEBcrQ@mail.gmail.com>

So, to summarize the thread so far:

Consensus:
np.nanmean()
np.nanstd()
np.minmax()
np.argminmax()

Vague Consensus:
np.sincos()

No Consensus (possibly out of scope for this topic):
Better constructors for complex types

I can probably whip up the PR for the nanmean() and nanstd(), and can
certainly help out with the minmax funcs.

Cheers!
Ben Root
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20130501/087b154d/attachment.html>

From mmick003 at ucr.edu  Wed May  1 21:42:57 2013
From: mmick003 at ucr.edu (Mark Micklich)
Date: Wed, 1 May 2013 18:42:57 -0700
Subject: [Numpy-discussion] numPy not imported into Python
Message-ID: <CAFZWFtOnGJEc2kR4VokuOERzBccqJbo4qrOabG_NmYmv0UiQgg@mail.gmail.com>

Hello -- After installing numPy, I'm getting the following error message
when attempting to import numarray:

       ImportError: No module named numarray

I do have numPy installed.  I'm running under Lubuntu 12.10 and the Spyder
2.1.10 IDE.  I'm fairly new to developing Python on Linux.  I assume there
is some path issue, but I'm not clear where to start.  If numPy is
installed, how to I point Spyder to the numPy modules so I can get numarray
to work?

Thanks,
Mark
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20130501/9d43a733/attachment.html>

From chanley at gmail.com  Wed May  1 21:47:37 2013
From: chanley at gmail.com (Christopher Hanley)
Date: Wed, 1 May 2013 21:47:37 -0400
Subject: [Numpy-discussion] numPy not imported into Python
In-Reply-To: <CAFZWFtOnGJEc2kR4VokuOERzBccqJbo4qrOabG_NmYmv0UiQgg@mail.gmail.com>
References: <CAFZWFtOnGJEc2kR4VokuOERzBccqJbo4qrOabG_NmYmv0UiQgg@mail.gmail.com>
Message-ID: <CADeni3usRQcdVRKcKVoNPxdZtMST1u=k1Qz1ZR0nuTMiZtGgPQ@mail.gmail.com>

Mark,

Numpy is not numarray. Numarray is an older package that has long since
been replaced by numpy.  You should only use numpy in any development from
now on.

Chris

On Wednesday, May 1, 2013, Mark Micklich wrote:

> Hello -- After installing numPy, I'm getting the following error message
> when attempting to import numarray:
>
>        ImportError: No module named numarray
>
> I do have numPy installed.  I'm running under Lubuntu 12.10 and the Spyder
> 2.1.10 IDE.  I'm fairly new to developing Python on Linux.  I assume there
> is some path issue, but I'm not clear where to start.  If numPy is
> installed, how to I point Spyder to the numPy modules so I can get numarray
> to work?
>
> Thanks,
> Mark
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20130501/29c66976/attachment.html>

From mmick003 at ucr.edu  Wed May  1 21:52:28 2013
From: mmick003 at ucr.edu (Mark Micklich)
Date: Wed, 1 May 2013 18:52:28 -0700
Subject: [Numpy-discussion] numPy not imported into Python
In-Reply-To: <CADeni3usRQcdVRKcKVoNPxdZtMST1u=k1Qz1ZR0nuTMiZtGgPQ@mail.gmail.com>
References: <CAFZWFtOnGJEc2kR4VokuOERzBccqJbo4qrOabG_NmYmv0UiQgg@mail.gmail.com>
	<CADeni3usRQcdVRKcKVoNPxdZtMST1u=k1Qz1ZR0nuTMiZtGgPQ@mail.gmail.com>
Message-ID: <CAFZWFtMudoUTM4H8XE9UVkQ5ZNRW2aPLhiK_CaypCnhnY3yPGA@mail.gmail.com>

Oh!  I imported numpy and that worked.  I probably should have mentioned
I'm working through the 2005 "*Numerical Methods in Engineering with Python*"
textbook from the school library.  The examples are still good, but the
Import statement used in the book is obsolete.  Thanks for the quick reply.

Mark

On Wed, May 1, 2013 at 6:47 PM, Christopher Hanley <chanley at gmail.com>wrote:

> Mark,
>
> Numpy is not numarray. Numarray is an older package that has long since
> been replaced by numpy.  You should only use numpy in any development from
> now on.
>
> Chris
>
>
> On Wednesday, May 1, 2013, Mark Micklich wrote:
>
>> Hello -- After installing numPy, I'm getting the following error message
>> when attempting to import numarray:
>>
>>        ImportError: No module named numarray
>>
>> I do have numPy installed.  I'm running under Lubuntu 12.10 and the Spyder
>> 2.1.10 IDE.  I'm fairly new to developing Python on Linux.  I assume there
>> is some path issue, but I'm not clear where to start.  If numPy is
>> installed, how to I point Spyder to the numPy modules so I can get numarray
>> to work?
>>
>> Thanks,
>> Mark
>>
>>
> _______________________________________________
> NumPy-Discussion mailing list
> NumPy-Discussion at scipy.org
> http://mail.scipy.org/mailman/listinfo/numpy-discussion
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20130501/96099145/attachment.html>

From charlesr.harris at gmail.com  Wed May  1 23:12:31 2013
From: charlesr.harris at gmail.com (Charles R Harris)
Date: Wed, 1 May 2013 21:12:31 -0600
Subject: [Numpy-discussion] nanmean(),
 nanstd() and other "missing" functions for 1.8
In-Reply-To: <CANNq6FnMnRRH2kBWq_YV-goRTmEojaUBoQNMxy7_F_U5vEBcrQ@mail.gmail.com>
References: <CANNq6F=NCdd1cRWmp7rskr6NJt3sjVvinN1XQVcXjJRgiGk=_A@mail.gmail.com>
	<1888613198476891170@unknownmsgid>
	<CANNq6FnZ4LOA8F1efJW+ZEeJtS3gwYag4mbvOMGSOt=rj_TwsQ@mail.gmail.com>
	<CALGmxELuBevPwVyO+g6ZktCiifZw5XXCY5CSXFiuBvhD8a_G=w@mail.gmail.com>
	<CANNq6FnMnRRH2kBWq_YV-goRTmEojaUBoQNMxy7_F_U5vEBcrQ@mail.gmail.com>
Message-ID: <CAB6mnx+vsuGPeQ6w79Kvwb+xjXODdcL7rPWTavVc9Vp1iR46qg@mail.gmail.com>

On Wed, May 1, 2013 at 7:10 PM, Benjamin Root <ben.root at ou.edu> wrote:

> So, to summarize the thread so far:
>
> Consensus:
> np.nanmean()
> np.nanstd()
> np.minmax()
> np.argminmax()
>
> Vague Consensus:
> np.sincos()
>
>
If the return of sincos (cossin?) is an array, then it could be reshaped to
be exp(1j*x), which together with exp(2*pi*1j*x) would cover some pretty
common cases.

No Consensus (possibly out of scope for this topic):
> Better constructors for complex types
>
> I can probably whip up the PR for the nanmean() and nanstd(), and can
> certainly help out with the minmax funcs.
>
> Cheers!
> Ben Root
>
>
Chuck
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20130501/e74e94a8/attachment.html>

From charlesr.harris at gmail.com  Wed May  1 23:28:45 2013
From: charlesr.harris at gmail.com (Charles R Harris)
Date: Wed, 1 May 2013 21:28:45 -0600
Subject: [Numpy-discussion] GSoC : Performance parity between numpy
 arrays and Python scalars
In-Reply-To: <CAJAs4rQ4qQmo4sy5NM=CU9XWSrShOxBn0ti9f-Z4NiZhhehewQ@mail.gmail.com>
References: <CAJAs4rQ4qQmo4sy5NM=CU9XWSrShOxBn0ti9f-Z4NiZhhehewQ@mail.gmail.com>
Message-ID: <CAB6mnxJ23DptRMMvngAgZpViz_dUerNKz08HW94U3ypGpMOKdg@mail.gmail.com>

On Tue, Apr 30, 2013 at 8:26 PM, Arink Verma <arinkverma at iitrpr.ac.in>wrote:

> Hi all!
> I have written my application[1] for *Performance parity between numpy
> arrays and Python scalars[2]. *It would be a great help if you view it.
> Does it look achievable and deliverable according to the project.
>
> [1]
> http://www.google-melange.com/gsoc/proposal/review/google/gsoc2013/arinkverma/40001#
> [2] http://projects.scipy.org/scipy/wiki/SummerofCodeIdeas
>
>
>
Hi Arink,

Have you already done some profiling? That could be tricky at the C level.
I'm also curious about the hash table, what gets hashed and where do you
get the improved efficiency? Admittedly, the way in which ufuncs currently
detect scalars is a bit heavy weight and a fast path for certain inputs
values could help. Is that what you are doing?

As to the schedule, I suspect that it may be a bit ambitious but I don't
see that as fatal by any means. Identifying bottlenecks and experimenting
with solutions would be useful work.

Chuck
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20130501/ab9fa688/attachment.html>

From arinkverma at iitrpr.ac.in  Thu May  2 00:25:12 2013
From: arinkverma at iitrpr.ac.in (Arink Verma)
Date: Thu, 2 May 2013 09:55:12 +0530
Subject: [Numpy-discussion] GSoC : Performance parity between numpy
 arrays and Python scalars
In-Reply-To: <51816167.6040607@virtualmaterials.com>
References: <CAJAs4rQ4qQmo4sy5NM=CU9XWSrShOxBn0ti9f-Z4NiZhhehewQ@mail.gmail.com>
	<51816167.6040607@virtualmaterials.com>
Message-ID: <CAJAs4rRTpvMCFURQy5F5yTC4PqPvyF0TStSMVQ3xG+MAFMGLEw@mail.gmail.com>

@Raul
I will pull new version, and try to include that also.
What is wrong with macros for inline function?
Yes, time for ufunc is reduced to almost half, for lookup table, I
am generating key from argument type and returning
the appropriated value.[1]

@Chuck
Yes I did some profiling with oprofiler for "python -m timeit -n 1000000 -s
'import numpy as np;x = np.asarray(1.0)' 'x+x'". see data sheet.[2]

As every time a ufunc is invoked, the code has to check every single data
type possible (bool, int, double, etc) until if finds the best match for
the data that the operation is being performed on. In scalar, we can send
best match, from pre-populated table. At present the implementation is not
well-structured and support only addition for int+int and float+float. [1]

[1]
https://github.com/arinkverma/numpy/commit/e2d8de7e7b643c7a76ff92bc1219847f9328aad0
[2]
https://docs.google.com/spreadsheet/ccc?key=0AnPqyp8kuQw0dG1hdjZiazE2dGtTY1JXVGFsWEEzbXc#gid=0



On Thu, May 2, 2013 at 12:09 AM, Raul Cota <raul at virtualmaterials.com>wrote:

>
> It is great that you are looking into this !! We are currently running on
> a fork of numpy because we really need these performance improvements .
>
>
> I noticed that, as suggested, you took from the pull request I posted a
> while ago for the
> PyObject_GetAttrString
> PyObject_GetBuffer
>
> issues.
>
> ( https://github.com/raulcota/numpy )
>
>
> A couple of comments on that,
>
> - Seems like you did not grab the latest revisions of that code that I
> posted that fixes the style of the comments and 'attempts' to fix an issue
> reported about Python 3 . I say 'attempts' because I thought it was fixed
> but I someone mentioned this was not correct.
>
> - There was also some feedback from Nathaniel about not liking the macros
> and siding for inline functions. I have not gotten around to it, but it
> would be nice if you jump on that boat.
>
> On the has lookup table, haven't looked at the implementation but the
> speed up is remarkable.
>
>
> Cheers !
>
> Raul
>
>
>
> On 30/04/2013 8:26 PM, Arink Verma wrote:
>
>  Hi all!
> I have written my application[1] for *Performance parity between numpy
> arrays and Python scalars[2]. *It would be a great help if you view it.
> Does it look achievable and deliverable according to the project.
>
>  [1]
> http://www.google-melange.com/gsoc/proposal/review/google/gsoc2013/arinkverma/40001#
>  [2] http://projects.scipy.org/scipy/wiki/SummerofCodeIdeas
>
>
> --
> Arink
> Computer Science and Engineering
> Indian Institute of Technology Ropar
> www.arinkverma.in
>
>
> _______________________________________________
> NumPy-Discussion mailing listNumPy-Discussion at scipy.orghttp://mail.scipy.org/mailman/listinfo/numpy-discussion
>
>
>
> _______________________________________________
> NumPy-Discussion mailing list
> NumPy-Discussion at scipy.org
> http://mail.scipy.org/mailman/listinfo/numpy-discussion
>
>


-- 
Arink
Computer Science and Engineering
Indian Institute of Technology Ropar
www.arinkverma.in
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20130502/c60b417a/attachment.html>

From cournape at gmail.com  Thu May  2 00:42:16 2013
From: cournape at gmail.com (David Cournapeau)
Date: Thu, 2 May 2013 05:42:16 +0100
Subject: [Numpy-discussion] GSoC : Performance parity between numpy
 arrays and Python scalars
In-Reply-To: <CAJAs4rRTpvMCFURQy5F5yTC4PqPvyF0TStSMVQ3xG+MAFMGLEw@mail.gmail.com>
References: <CAJAs4rQ4qQmo4sy5NM=CU9XWSrShOxBn0ti9f-Z4NiZhhehewQ@mail.gmail.com>
	<51816167.6040607@virtualmaterials.com>
	<CAJAs4rRTpvMCFURQy5F5yTC4PqPvyF0TStSMVQ3xG+MAFMGLEw@mail.gmail.com>
Message-ID: <CAGY4rcUMYSS+9zVXU46CXyw--_TbsUjMOA9TEzgBFbiD_UziMQ@mail.gmail.com>

On Thu, May 2, 2013 at 5:25 AM, Arink Verma <arinkverma at iitrpr.ac.in> wrote:
> @Raul
> I will pull new version, and try to include that also.
> What is wrong with macros for inline function?
> Yes, time for ufunc is reduced to almost half, for lookup table, I am
> generating key from argument type and returning the appropriated value.[1]
>
> @Chuck
> Yes I did some profiling with oprofiler for "python -m timeit -n 1000000 -s
> 'import numpy as np;x = np.asarray(1.0)' 'x+x'". see data sheet.[2]
>
> As every time a ufunc is invoked, the code has to check every single data
> type possible (bool, int, double, etc) until if finds the best match for the
> data that the operation is being performed on. In scalar, we can send best
> match, from pre-populated table. At present the implementation is not
> well-structured and support only addition for int+int and float+float. [1]

You are pointing out something that may well be the main difficulty:
the code there is messy, and we need to ensure that optimisations
don't preclude later extensions (especially with regard to new dtype
addition).

David


From ben.root at ou.edu  Thu May  2 01:01:14 2013
From: ben.root at ou.edu (Benjamin Root)
Date: Thu, 2 May 2013 01:01:14 -0400
Subject: [Numpy-discussion] nanmean(),
 nanstd() and other "missing" functions for 1.8
In-Reply-To: <CANNq6FnMnRRH2kBWq_YV-goRTmEojaUBoQNMxy7_F_U5vEBcrQ@mail.gmail.com>
References: <CANNq6F=NCdd1cRWmp7rskr6NJt3sjVvinN1XQVcXjJRgiGk=_A@mail.gmail.com>
	<1888613198476891170@unknownmsgid>
	<CANNq6FnZ4LOA8F1efJW+ZEeJtS3gwYag4mbvOMGSOt=rj_TwsQ@mail.gmail.com>
	<CALGmxELuBevPwVyO+g6ZktCiifZw5XXCY5CSXFiuBvhD8a_G=w@mail.gmail.com>
	<CANNq6FnMnRRH2kBWq_YV-goRTmEojaUBoQNMxy7_F_U5vEBcrQ@mail.gmail.com>
Message-ID: <CANNq6FmXA4jaC5t5oTd4MSyK8=+Vee-g09A=f=QyRPGODtPxgA@mail.gmail.com>

I have created a PR for the first two (and got np.nanvar() for free).

https://github.com/numpy/numpy/pull/3297

Cheers!
Ben Root
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20130502/9086fcdb/attachment.html>

From arinkverma at iitrpr.ac.in  Thu May  2 06:26:58 2013
From: arinkverma at iitrpr.ac.in (Arink Verma)
Date: Thu, 2 May 2013 15:56:58 +0530
Subject: [Numpy-discussion] GSoC : Performance parity between numpy
 arrays and Python scalars
In-Reply-To: <CAGY4rcUMYSS+9zVXU46CXyw--_TbsUjMOA9TEzgBFbiD_UziMQ@mail.gmail.com>
References: <CAJAs4rQ4qQmo4sy5NM=CU9XWSrShOxBn0ti9f-Z4NiZhhehewQ@mail.gmail.com>
	<51816167.6040607@virtualmaterials.com>
	<CAJAs4rRTpvMCFURQy5F5yTC4PqPvyF0TStSMVQ3xG+MAFMGLEw@mail.gmail.com>
	<CAGY4rcUMYSS+9zVXU46CXyw--_TbsUjMOA9TEzgBFbiD_UziMQ@mail.gmail.com>
Message-ID: <CAJAs4rRhAs4bwu+x3b-fQDi+8eMUuRnZ-C66SmZE6HcHFrY2AQ@mail.gmail.com>

Yes, we need to ensure that..
Code generator can be made, which can create code for table of registered
dtype during build time itself.

Also at present there lot of duplicate code that attempts to work around
these slow paths, simplification of that code is also required.


On Thu, May 2, 2013 at 10:12 AM, David Cournapeau <cournape at gmail.com>wrote:

> On Thu, May 2, 2013 at 5:25 AM, Arink Verma <arinkverma at iitrpr.ac.in>
> wrote:
> > @Raul
> > I will pull new version, and try to include that also.
> > What is wrong with macros for inline function?
> > Yes, time for ufunc is reduced to almost half, for lookup table, I am
> > generating key from argument type and returning the appropriated
> value.[1]
> >
> > @Chuck
> > Yes I did some profiling with oprofiler for "python -m timeit -n 1000000
> -s
> > 'import numpy as np;x = np.asarray(1.0)' 'x+x'". see data sheet.[2]
> >
> > As every time a ufunc is invoked, the code has to check every single data
> > type possible (bool, int, double, etc) until if finds the best match for
> the
> > data that the operation is being performed on. In scalar, we can send
> best
> > match, from pre-populated table. At present the implementation is not
> > well-structured and support only addition for int+int and float+float.
> [1]
>
> You are pointing out something that may well be the main difficulty:
> the code there is messy, and we need to ensure that optimisations
> don't preclude later extensions (especially with regard to new dtype
> addition).
>
> David
> _______________________________________________
> NumPy-Discussion mailing list
> NumPy-Discussion at scipy.org
> http://mail.scipy.org/mailman/listinfo/numpy-discussion
>



-- 
Arink
Computer Science and Engineering
Indian Institute of Technology Ropar
www.arinkverma.in
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20130502/ab67142e/attachment.html>

From njs at pobox.com  Thu May  2 07:03:45 2013
From: njs at pobox.com (Nathaniel Smith)
Date: Thu, 2 May 2013 07:03:45 -0400
Subject: [Numpy-discussion] nanmean(),
 nanstd() and other "missing" functions for 1.8
In-Reply-To: <CAB6mnx+vsuGPeQ6w79Kvwb+xjXODdcL7rPWTavVc9Vp1iR46qg@mail.gmail.com>
References: <CANNq6F=NCdd1cRWmp7rskr6NJt3sjVvinN1XQVcXjJRgiGk=_A@mail.gmail.com>
	<1888613198476891170@unknownmsgid>
	<CANNq6FnZ4LOA8F1efJW+ZEeJtS3gwYag4mbvOMGSOt=rj_TwsQ@mail.gmail.com>
	<CALGmxELuBevPwVyO+g6ZktCiifZw5XXCY5CSXFiuBvhD8a_G=w@mail.gmail.com>
	<CANNq6FnMnRRH2kBWq_YV-goRTmEojaUBoQNMxy7_F_U5vEBcrQ@mail.gmail.com>
	<CAB6mnx+vsuGPeQ6w79Kvwb+xjXODdcL7rPWTavVc9Vp1iR46qg@mail.gmail.com>
Message-ID: <CAPJVwBkb4SfLvc8sAgynZyjs=gYTX5qhH1Y5f768LNWzZ6bQ9A@mail.gmail.com>

On 1 May 2013 23:12, "Charles R Harris" <charlesr.harris at gmail.com> wrote:
>
>
>
> On Wed, May 1, 2013 at 7:10 PM, Benjamin Root <ben.root at ou.edu> wrote:
>>
>> So, to summarize the thread so far:
>>
>> Consensus:
>> np.nanmean()
>> np.nanstd()
>> np.minmax()
>> np.argminmax()
>>
>> Vague Consensus:
>> np.sincos()
>>
>
> If the return of sincos (cossin?) is an array, then it could be reshaped
to be exp(1j*x), which together with exp(2*pi*1j*x) would cover some pretty
common cases.

Ufuncs already have some convention for what to do with multiple output
arguments, right? Presumably whatever they do is what sincos should do.
(And minmax/argminmax likewise, for consistency, even if they aren't
ufuncs. Though they could be generalized ufuncs, or minmax could be
minimummaximum.reduce.)

I haven't checked, but I assume that what multiple output argument ufuncs
do is to return a tuple. You can't use a single array in the general case,
because the multiple output types might not be homogenous.

-n
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20130502/805d6267/attachment.html>

From sebastian at sipsolutions.net  Thu May  2 07:15:47 2013
From: sebastian at sipsolutions.net (Sebastian Berg)
Date: Thu, 02 May 2013 13:15:47 +0200
Subject: [Numpy-discussion] nanmean(),
 nanstd() and other "missing" functions for 1.8
In-Reply-To: <CAPJVwBkb4SfLvc8sAgynZyjs=gYTX5qhH1Y5f768LNWzZ6bQ9A@mail.gmail.com>
References: <CANNq6F=NCdd1cRWmp7rskr6NJt3sjVvinN1XQVcXjJRgiGk=_A@mail.gmail.com>
	<1888613198476891170@unknownmsgid>
	<CANNq6FnZ4LOA8F1efJW+ZEeJtS3gwYag4mbvOMGSOt=rj_TwsQ@mail.gmail.com>
	<CALGmxELuBevPwVyO+g6ZktCiifZw5XXCY5CSXFiuBvhD8a_G=w@mail.gmail.com>
	<CANNq6FnMnRRH2kBWq_YV-goRTmEojaUBoQNMxy7_F_U5vEBcrQ@mail.gmail.com>
	<CAB6mnx+vsuGPeQ6w79Kvwb+xjXODdcL7rPWTavVc9Vp1iR46qg@mail.gmail.com>
	<CAPJVwBkb4SfLvc8sAgynZyjs=gYTX5qhH1Y5f768LNWzZ6bQ9A@mail.gmail.com>
Message-ID: <1367493347.3046.3.camel@sebastian-laptop>

On Thu, 2013-05-02 at 07:03 -0400, Nathaniel Smith wrote:
> On 1 May 2013 23:12, "Charles R Harris" <charlesr.harris at gmail.com>
> wrote:
> >
> >
> >
> > On Wed, May 1, 2013 at 7:10 PM, Benjamin Root <ben.root at ou.edu>
> wrote:
> >>
> >> So, to summarize the thread so far:
> >>
> >> Consensus:
> >> np.nanmean()
> >> np.nanstd()
> >> np.minmax()
> >> np.argminmax()
> >>
> >> Vague Consensus:
> >> np.sincos()
> >>
> >
> > If the return of sincos (cossin?) is an array, then it could be
> reshaped to be exp(1j*x), which together with exp(2*pi*1j*x) would
> cover some pretty common cases.
> 
> Ufuncs already have some convention for what to do with multiple
> output arguments, right? Presumably whatever they do is what sincos
> should do. (And minmax/argminmax likewise, for consistency, even if
> they aren't ufuncs. Though they could be generalized ufuncs, or minmax
> could be minimummaximum.reduce.)
> 
I think for sincos it makes sense, for an example a ufunc with multiple
arguments is `np.modf`. But I doubt reductions are automatically defined
for these, so minmax probably needs to be a generalized ufunc (can you
have an axis argument with those?). 

- Sebastian

> I haven't checked, but I assume that what multiple output argument
> ufuncs do is to return a tuple. You can't use a single array in the
> general case, because the multiple output types might not be
> homogenous.
> 
> -n
> 
> _______________________________________________
> NumPy-Discussion mailing list
> NumPy-Discussion at scipy.org
> http://mail.scipy.org/mailman/listinfo/numpy-discussion




From cournape at gmail.com  Thu May  2 08:42:13 2013
From: cournape at gmail.com (David Cournapeau)
Date: Thu, 2 May 2013 13:42:13 +0100
Subject: [Numpy-discussion] GSoC : Performance parity between numpy
 arrays and Python scalars
In-Reply-To: <CAJAs4rRhAs4bwu+x3b-fQDi+8eMUuRnZ-C66SmZE6HcHFrY2AQ@mail.gmail.com>
References: <CAJAs4rQ4qQmo4sy5NM=CU9XWSrShOxBn0ti9f-Z4NiZhhehewQ@mail.gmail.com>
	<51816167.6040607@virtualmaterials.com>
	<CAJAs4rRTpvMCFURQy5F5yTC4PqPvyF0TStSMVQ3xG+MAFMGLEw@mail.gmail.com>
	<CAGY4rcUMYSS+9zVXU46CXyw--_TbsUjMOA9TEzgBFbiD_UziMQ@mail.gmail.com>
	<CAJAs4rRhAs4bwu+x3b-fQDi+8eMUuRnZ-C66SmZE6HcHFrY2AQ@mail.gmail.com>
Message-ID: <CAGY4rcXR7oiD0QM3EvVnJ3V+Eu1ypX1LWksF3hE=vM991Kn+HA@mail.gmail.com>

On Thu, May 2, 2013 at 11:26 AM, Arink Verma <arinkverma at iitrpr.ac.in> wrote:
> Yes, we need to ensure that..
> Code generator can be made, which can create code for table of registered
> dtype during build time itself.

So dtypes can be registered at runtime as well. In an ideal world,
'native' numpy types would not be special cases. This is too big for a
GSoC, but we should make sure we don't make it worse.

>
> Also at present there lot of duplicate code that attempts to work around
> these slow paths, simplification of that code is also required.

That there is room for consolidation would be an understatement :)

David
>
> On Thu, May 2, 2013 at 10:12 AM, David Cournapeau <cournape at gmail.com>
> wrote:
>>
>> On Thu, May 2, 2013 at 5:25 AM, Arink Verma <arinkverma at iitrpr.ac.in>
>> wrote:
>> > @Raul
>> > I will pull new version, and try to include that also.
>> > What is wrong with macros for inline function?
>> > Yes, time for ufunc is reduced to almost half, for lookup table, I am
>> > generating key from argument type and returning the appropriated
>> > value.[1]
>> >
>> > @Chuck
>> > Yes I did some profiling with oprofiler for "python -m timeit -n 1000000
>> > -s
>> > 'import numpy as np;x = np.asarray(1.0)' 'x+x'". see data sheet.[2]
>> >
>> > As every time a ufunc is invoked, the code has to check every single
>> > data
>> > type possible (bool, int, double, etc) until if finds the best match for
>> > the
>> > data that the operation is being performed on. In scalar, we can send
>> > best
>> > match, from pre-populated table. At present the implementation is not
>> > well-structured and support only addition for int+int and float+float.
>> > [1]
>>
>> You are pointing out something that may well be the main difficulty:
>> the code there is messy, and we need to ensure that optimisations
>> don't preclude later extensions (especially with regard to new dtype
>> addition).
>>
>> David
>> _______________________________________________
>> NumPy-Discussion mailing list
>> NumPy-Discussion at scipy.org
>> http://mail.scipy.org/mailman/listinfo/numpy-discussion
>
>
>
>
> --
> Arink
> Computer Science and Engineering
> Indian Institute of Technology Ropar
> www.arinkverma.in
>
> _______________________________________________
> NumPy-Discussion mailing list
> NumPy-Discussion at scipy.org
> http://mail.scipy.org/mailman/listinfo/numpy-discussion
>


From njs at pobox.com  Thu May  2 09:14:32 2013
From: njs at pobox.com (Nathaniel Smith)
Date: Thu, 2 May 2013 09:14:32 -0400
Subject: [Numpy-discussion] GSoC : Performance parity between numpy
 arrays and Python scalars
In-Reply-To: <CAJAs4rRhAs4bwu+x3b-fQDi+8eMUuRnZ-C66SmZE6HcHFrY2AQ@mail.gmail.com>
References: <CAJAs4rQ4qQmo4sy5NM=CU9XWSrShOxBn0ti9f-Z4NiZhhehewQ@mail.gmail.com>
	<51816167.6040607@virtualmaterials.com>
	<CAJAs4rRTpvMCFURQy5F5yTC4PqPvyF0TStSMVQ3xG+MAFMGLEw@mail.gmail.com>
	<CAGY4rcUMYSS+9zVXU46CXyw--_TbsUjMOA9TEzgBFbiD_UziMQ@mail.gmail.com>
	<CAJAs4rRhAs4bwu+x3b-fQDi+8eMUuRnZ-C66SmZE6HcHFrY2AQ@mail.gmail.com>
Message-ID: <CAPJVwB=0E6AaSmN+x7OckkfzuCVTvKuKvV=0Ts2r9Fny5inWvg@mail.gmail.com>

On Thu, May 2, 2013 at 6:26 AM, Arink Verma <arinkverma at iitrpr.ac.in> wrote:
> Yes, we need to ensure that..
> Code generator can be made, which can create code for table of registered
> dtype during build time itself.

I'd probably just generate it at run-time on an as-needed basis.
(I.e., use the full lookup logic the first time, then save the
result.) New dtypes can be registered, which will mean the tables need
to change size at runtime anyway. If someone does some strange thing
like add float16's and float64's, we can do the lookup to determine
that this should be handled by the float64/float64 loop, and then
store that information so that the next time it's fast (but we
probably don't want to be calculating all combinations at build-time,
which would require running the full type resolution machinery, esp.
since it wouldn't really bring any benefits that I can see).

* Re: the profiling, I wrote a full oprofile->callgrind format script
years ago: http://vorpus.org/~njs/op2calltree.py
Haven't used it in years either but neither oprofile nor kcachegrind
are terribly fast-moving projects so it's probably still working, or
could be made so without much work.
Or easier is to use the gperftools CPU profiler:
https://gperftools.googlecode.com/svn/trunk/doc/cpuprofile.html

Instead of linking to it at build time, you can just use ctypes:

In [7]: profiler = ctypes.CDLL("libprofiler.so.0")

In [8]: profiler.ProfilerStart("some-file-name-here")
Out[8]: 1

In [9]: # do stuff here

In [10]: profiler.ProfilerStop()
PROFILE: interrupts/evictions/bytes = 2/0/592
Out[10]: 46

Then all the pprof analysis tools are available as described on that webpage.

* Please don't trust those random suggestions for possible
improvements I threw out when writing the original description.
Probably it's true that FP flag checking and ufunc type lookup are
expensive, but one should fix what the profile says to fix, not what
someone guessed might be good to fix based on a few minutes thought.

* Instead of making a giant table of everything that needs to be done
to make stuff fast first, before writing any code, I'd suggest picking
one operation, figuring out what change would be the biggest
improvement for it, making that change, checking that it worked, and
then repeat until that operation is really fast. Then if there's still
time pick another operation. Producing a giant todo list isn't very
productive by itself if there's no time then to actually do all the
things on the list :-).

* Did you notice this line on the requirements page? "Having your
first pull request merged before the GSoC application deadline (May 3)
is required for your application to be accepted."

-n


From cournape at gmail.com  Thu May  2 09:25:28 2013
From: cournape at gmail.com (David Cournapeau)
Date: Thu, 2 May 2013 14:25:28 +0100
Subject: [Numpy-discussion] Profiling (was GSoC : Performance parity between
 numpy arrays and Python scalars)
Message-ID: <CAGY4rcVCN9v4fjWO8QtubQvvC8aTvxsYzBya+QzL3_82C5b4YQ@mail.gmail.com>

> * Re: the profiling, I wrote a full oprofile->callgrind format script
> years ago: http://vorpus.org/~njs/op2calltree.py
> Haven't used it in years either but neither oprofile nor kcachegrind
> are terribly fast-moving projects so it's probably still working, or
> could be made so without much work.
> Or easier is to use the gperftools CPU profiler:
> https://gperftools.googlecode.com/svn/trunk/doc/cpuprofile.html

I don't have experience with gperftools, but on recent linux kernels,
you can also use perf, which can't be made easier to use (no runtime
support needed), but you need a 'recent' kernel

http://indico.cern.ch/getFile.py/access?contribId=20&sessionId=4&resId=0&materialId=slides&confId=141309

I am hoping to talk a bit about those for our diving into numpy c code
tutorial in June, what's the + of gperf in your opinion ?

David


From robert.kern at gmail.com  Thu May  2 09:28:55 2013
From: robert.kern at gmail.com (Robert Kern)
Date: Thu, 2 May 2013 14:28:55 +0100
Subject: [Numpy-discussion] nanmean(),
 nanstd() and other "missing" functions for 1.8
In-Reply-To: <CAPJVwBkb4SfLvc8sAgynZyjs=gYTX5qhH1Y5f768LNWzZ6bQ9A@mail.gmail.com>
References: <CANNq6F=NCdd1cRWmp7rskr6NJt3sjVvinN1XQVcXjJRgiGk=_A@mail.gmail.com>
	<1888613198476891170@unknownmsgid>
	<CANNq6FnZ4LOA8F1efJW+ZEeJtS3gwYag4mbvOMGSOt=rj_TwsQ@mail.gmail.com>
	<CALGmxELuBevPwVyO+g6ZktCiifZw5XXCY5CSXFiuBvhD8a_G=w@mail.gmail.com>
	<CANNq6FnMnRRH2kBWq_YV-goRTmEojaUBoQNMxy7_F_U5vEBcrQ@mail.gmail.com>
	<CAB6mnx+vsuGPeQ6w79Kvwb+xjXODdcL7rPWTavVc9Vp1iR46qg@mail.gmail.com>
	<CAPJVwBkb4SfLvc8sAgynZyjs=gYTX5qhH1Y5f768LNWzZ6bQ9A@mail.gmail.com>
Message-ID: <CAF6FJiuTJgOrWsvgKaAkQEP=gPkmzKL0ETT+KaAp_Eva9N2Hug@mail.gmail.com>

On Thu, May 2, 2013 at 12:03 PM, Nathaniel Smith <njs at pobox.com> wrote:
> On 1 May 2013 23:12, "Charles R Harris" <charlesr.harris at gmail.com> wrote:
>>
>> On Wed, May 1, 2013 at 7:10 PM, Benjamin Root <ben.root at ou.edu> wrote:
>>>
>>> So, to summarize the thread so far:
>>>
>>> Consensus:
>>> np.nanmean()
>>> np.nanstd()
>>> np.minmax()
>>> np.argminmax()
>>>
>>> Vague Consensus:
>>> np.sincos()
>>>
>>
>> If the return of sincos (cossin?) is an array, then it could be reshaped
>> to be exp(1j*x), which together with exp(2*pi*1j*x) would cover some pretty
>> common cases.

It couldn't be a mere reshape, since the complex dtype requires the
real and imag components to be adjacent to each other. They wouldn't
be so if sincos's return type is an array (nor even the cossin
alternative). It always requires a memory copy (except in the "who
cares?" case of a scalar). Composition with an efficient
np.tocomplex(real, imag) implementation would cover those use cases
whether sincos returns tuples or arrays.

> Ufuncs already have some convention for what to do with multiple output
> arguments, right? Presumably whatever they do is what sincos should do. (And
> minmax/argminmax likewise, for consistency, even if they aren't ufuncs.
> Though they could be generalized ufuncs, or minmax could be
> minimummaximum.reduce.)
>
> I haven't checked, but I assume that what multiple output argument ufuncs do
> is to return a tuple. You can't use a single array in the general case,
> because the multiple output types might not be homogenous.

Correct.

[~]
|19> np.modf.nout
2

[~]
|20> np.modf(np.linspace(0, 1, 5))
(array([ 0.  ,  0.25,  0.5 ,  0.75,  0.  ]), array([ 0.,  0.,  0.,  0.,  1.]))

--
Robert Kern


From charlesr.harris at gmail.com  Thu May  2 09:38:55 2013
From: charlesr.harris at gmail.com (Charles R Harris)
Date: Thu, 2 May 2013 07:38:55 -0600
Subject: [Numpy-discussion] nanmean(),
 nanstd() and other "missing" functions for 1.8
In-Reply-To: <CAF6FJiuTJgOrWsvgKaAkQEP=gPkmzKL0ETT+KaAp_Eva9N2Hug@mail.gmail.com>
References: <CANNq6F=NCdd1cRWmp7rskr6NJt3sjVvinN1XQVcXjJRgiGk=_A@mail.gmail.com>
	<1888613198476891170@unknownmsgid>
	<CANNq6FnZ4LOA8F1efJW+ZEeJtS3gwYag4mbvOMGSOt=rj_TwsQ@mail.gmail.com>
	<CALGmxELuBevPwVyO+g6ZktCiifZw5XXCY5CSXFiuBvhD8a_G=w@mail.gmail.com>
	<CANNq6FnMnRRH2kBWq_YV-goRTmEojaUBoQNMxy7_F_U5vEBcrQ@mail.gmail.com>
	<CAB6mnx+vsuGPeQ6w79Kvwb+xjXODdcL7rPWTavVc9Vp1iR46qg@mail.gmail.com>
	<CAPJVwBkb4SfLvc8sAgynZyjs=gYTX5qhH1Y5f768LNWzZ6bQ9A@mail.gmail.com>
	<CAF6FJiuTJgOrWsvgKaAkQEP=gPkmzKL0ETT+KaAp_Eva9N2Hug@mail.gmail.com>
Message-ID: <CAB6mnx+x9HryRa6tOuqCGCQfbhC1a4J0ShRiduX8s7YsM9svmw@mail.gmail.com>

On Thu, May 2, 2013 at 7:28 AM, Robert Kern <robert.kern at gmail.com> wrote:

> On Thu, May 2, 2013 at 12:03 PM, Nathaniel Smith <njs at pobox.com> wrote:
> > On 1 May 2013 23:12, "Charles R Harris" <charlesr.harris at gmail.com>
> wrote:
> >>
> >> On Wed, May 1, 2013 at 7:10 PM, Benjamin Root <ben.root at ou.edu> wrote:
> >>>
> >>> So, to summarize the thread so far:
> >>>
> >>> Consensus:
> >>> np.nanmean()
> >>> np.nanstd()
> >>> np.minmax()
> >>> np.argminmax()
> >>>
> >>> Vague Consensus:
> >>> np.sincos()
> >>>
> >>
> >> If the return of sincos (cossin?) is an array, then it could be reshaped
> >> to be exp(1j*x), which together with exp(2*pi*1j*x) would cover some
> pretty
> >> common cases.
>
> It couldn't be a mere reshape, since the complex dtype requires the
> real and imag components to be adjacent to each other. They wouldn't
> be so if sincos's return type is an array (nor even the cossin
> alternative). It always requires a memory copy (except in the "who
> cares?" case of a scalar). Composition with an efficient
> np.tocomplex(real, imag) implementation would cover those use cases
> whether sincos returns tuples or arrays.
>

I would assume the basic return type would be complex, i.e., the cos/sin
adjacent. The cos/sin parts would then be real/imag views into the array.


>
> > Ufuncs already have some convention for what to do with multiple output
> > arguments, right? Presumably whatever they do is what sincos should do.
> (And
> > minmax/argminmax likewise, for consistency, even if they aren't ufuncs.
> > Though they could be generalized ufuncs, or minmax could be
> > minimummaximum.reduce.)
> >
> > I haven't checked, but I assume that what multiple output argument
> ufuncs do
> > is to return a tuple. You can't use a single array in the general case,
> > because the multiple output types might not be homogenous.
>
> Correct.
>
> [~]
> |19> np.modf.nout
> 2
>
> [~]
> |20> np.modf(np.linspace(0, 1, 5))
> (array([ 0.  ,  0.25,  0.5 ,  0.75,  0.  ]), array([ 0.,  0.,  0.,  0.,
>  1.]))
>
> --
> Robert Kern
> _______________________________________________
> NumPy-Discussion mailing list
> NumPy-Discussion at scipy.org
> http://mail.scipy.org/mailman/listinfo/numpy-discussion
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20130502/87f5195c/attachment.html>

From robert.kern at gmail.com  Thu May  2 09:47:05 2013
From: robert.kern at gmail.com (Robert Kern)
Date: Thu, 2 May 2013 14:47:05 +0100
Subject: [Numpy-discussion] nanmean(),
 nanstd() and other "missing" functions for 1.8
In-Reply-To: <CAB6mnx+x9HryRa6tOuqCGCQfbhC1a4J0ShRiduX8s7YsM9svmw@mail.gmail.com>
References: <CANNq6F=NCdd1cRWmp7rskr6NJt3sjVvinN1XQVcXjJRgiGk=_A@mail.gmail.com>
	<1888613198476891170@unknownmsgid>
	<CANNq6FnZ4LOA8F1efJW+ZEeJtS3gwYag4mbvOMGSOt=rj_TwsQ@mail.gmail.com>
	<CALGmxELuBevPwVyO+g6ZktCiifZw5XXCY5CSXFiuBvhD8a_G=w@mail.gmail.com>
	<CANNq6FnMnRRH2kBWq_YV-goRTmEojaUBoQNMxy7_F_U5vEBcrQ@mail.gmail.com>
	<CAB6mnx+vsuGPeQ6w79Kvwb+xjXODdcL7rPWTavVc9Vp1iR46qg@mail.gmail.com>
	<CAPJVwBkb4SfLvc8sAgynZyjs=gYTX5qhH1Y5f768LNWzZ6bQ9A@mail.gmail.com>
	<CAF6FJiuTJgOrWsvgKaAkQEP=gPkmzKL0ETT+KaAp_Eva9N2Hug@mail.gmail.com>
	<CAB6mnx+x9HryRa6tOuqCGCQfbhC1a4J0ShRiduX8s7YsM9svmw@mail.gmail.com>
Message-ID: <CAF6FJiuawKKjnh9G0FfgB2D9c8Yy8e=u6W2ejWsa_qia9x_Fng@mail.gmail.com>

On Thu, May 2, 2013 at 2:38 PM, Charles R Harris
<charlesr.harris at gmail.com> wrote:
>
> On Thu, May 2, 2013 at 7:28 AM, Robert Kern <robert.kern at gmail.com> wrote:
>>
>> On Thu, May 2, 2013 at 12:03 PM, Nathaniel Smith <njs at pobox.com> wrote:
>> > On 1 May 2013 23:12, "Charles R Harris" <charlesr.harris at gmail.com>
>> > wrote:
>> >>
>> >> On Wed, May 1, 2013 at 7:10 PM, Benjamin Root <ben.root at ou.edu> wrote:
>> >>>
>> >>> So, to summarize the thread so far:
>> >>>
>> >>> Consensus:
>> >>> np.nanmean()
>> >>> np.nanstd()
>> >>> np.minmax()
>> >>> np.argminmax()
>> >>>
>> >>> Vague Consensus:
>> >>> np.sincos()
>> >>>
>> >>
>> >> If the return of sincos (cossin?) is an array, then it could be
>> >> reshaped
>> >> to be exp(1j*x), which together with exp(2*pi*1j*x) would cover some
>> >> pretty
>> >> common cases.
>>
>> It couldn't be a mere reshape, since the complex dtype requires the
>> real and imag components to be adjacent to each other. They wouldn't
>> be so if sincos's return type is an array (nor even the cossin
>> alternative). It always requires a memory copy (except in the "who
>> cares?" case of a scalar). Composition with an efficient
>> np.tocomplex(real, imag) implementation would cover those use cases
>> whether sincos returns tuples or arrays.
>
> I would assume the basic return type would be complex, i.e., the cos/sin
> adjacent. The cos/sin parts would then be real/imag views into the array.

You mean that the implementation of cossin (to make things easier on
ourselves) would create an (N,2) contiguous array, fill it with the
cos and sin results, then reshape it to return the expected (2,N)
array (or 2-tuple)? How would the user then reconstitute the exp(1j*x)
result efficiently? If the use case is that important, I would just
make exp(1j*x) into its own ufunc and have it use the C sincos()
function internally.

--
Robert Kern


From njs at pobox.com  Thu May  2 09:58:52 2013
From: njs at pobox.com (Nathaniel Smith)
Date: Thu, 2 May 2013 09:58:52 -0400
Subject: [Numpy-discussion] Profiling (was GSoC : Performance parity
 between numpy arrays and Python scalars)
In-Reply-To: <CAGY4rcVCN9v4fjWO8QtubQvvC8aTvxsYzBya+QzL3_82C5b4YQ@mail.gmail.com>
References: <CAGY4rcVCN9v4fjWO8QtubQvvC8aTvxsYzBya+QzL3_82C5b4YQ@mail.gmail.com>
Message-ID: <CAPJVwBkh0jda763ps+PF+yHrSJNNeSYdd7=bjUTi-jwjAp-vwg@mail.gmail.com>

On Thu, May 2, 2013 at 9:25 AM, David Cournapeau <cournape at gmail.com> wrote:
>> * Re: the profiling, I wrote a full oprofile->callgrind format script
>> years ago: http://vorpus.org/~njs/op2calltree.py
>> Haven't used it in years either but neither oprofile nor kcachegrind
>> are terribly fast-moving projects so it's probably still working, or
>> could be made so without much work.
>> Or easier is to use the gperftools CPU profiler:
>> https://gperftools.googlecode.com/svn/trunk/doc/cpuprofile.html
>
> I don't have experience with gperftools, but on recent linux kernels,
> you can also use perf, which can't be made easier to use (no runtime
> support needed), but you need a 'recent' kernel
>
> http://indico.cern.ch/getFile.py/access?contribId=20&sessionId=4&resId=0&materialId=slides&confId=141309
>
> I am hoping to talk a bit about those for our diving into numpy c code
> tutorial in June, what's the + of gperf in your opinion ?

For what I've used profiling for, THE key feature is proper callgraph
support ("show me the *total* time spent in each function, including
callees"). Otherwise, silly example, let's say you have a bug where
you wrote:

func1() {
  for (i = 0; i < 10000000; i++)
     foo = add(foo, bar[0])
}

Obviously this is a waste of time, since you're actually performing
the same operation over and over. Many profilers, given this, will
tell you that all the time is spent in 'add', which is useless,
because you don't want to speed up 'add', you want to speed up 'func1'
(probably by not calling 'add' so many times!). If you have relatively
flat code like most kernel code this isn't an issue, but I generally
don't...

perf is a fabulous framework and doesn't have any way to get full
callgraph information out so IME it's been useless. They have
reporting modes that claim to (like some "fractal" thing?) but AFAI
been able to tell from docs/googling/mailing lists, there is nobody
who understands how to interpret this output except the people who
wrote it. Really a shame that it falls down in the last mile like
that, hopefully they will fix this soon.

callgrind has the *fabulous* kcachegrind front-end, but it only
measures memory access performance on a simulated machine, which is
very useful sometimes (if you're trying to optimize cache locality),
but there's no guarantee that the bottlenecks on its simulated machine
are the same as the bottlenecks on your real machine.

oprofile is getting long in the tooth (superseded by perf), and it's
built-in reporting tools are merely ok, but it does have full
callgraph information and with the script above you can get the output
into kcachegrind.

perftools don't have all the fancy features of the in-kernel options,
but they're trivial to use, and their reporting options are genuinely
useful (though not quite as awesome as kcachegrind). So while in
theory it's the least whizz-bang awesome of all of these options, in
practice I find it the most useful.

(Also, beware of terminology collision, "gperf" is something else again...)

-n


From arinkverma at iitrpr.ac.in  Thu May  2 10:18:39 2013
From: arinkverma at iitrpr.ac.in (Arink Verma)
Date: Thu, 2 May 2013 19:48:39 +0530
Subject: [Numpy-discussion] GSoC : Performance parity between numpy
 arrays and Python scalars
In-Reply-To: <CAPJVwB=0E6AaSmN+x7OckkfzuCVTvKuKvV=0Ts2r9Fny5inWvg@mail.gmail.com>
References: <CAJAs4rQ4qQmo4sy5NM=CU9XWSrShOxBn0ti9f-Z4NiZhhehewQ@mail.gmail.com>
	<51816167.6040607@virtualmaterials.com>
	<CAJAs4rRTpvMCFURQy5F5yTC4PqPvyF0TStSMVQ3xG+MAFMGLEw@mail.gmail.com>
	<CAGY4rcUMYSS+9zVXU46CXyw--_TbsUjMOA9TEzgBFbiD_UziMQ@mail.gmail.com>
	<CAJAs4rRhAs4bwu+x3b-fQDi+8eMUuRnZ-C66SmZE6HcHFrY2AQ@mail.gmail.com>
	<CAPJVwB=0E6AaSmN+x7OckkfzuCVTvKuKvV=0Ts2r9Fny5inWvg@mail.gmail.com>
Message-ID: <CAJAs4rRXahsJR=ZseCXu+j-XmNzkSkpoB+VW-bOeJWoDsuXnkA@mail.gmail.com>

Updating table at runtime, seems a good option. But then we have
maintain separate file for caching and storing.

I will see, op2calltree.py <http://vorpus.org/~njs/op2calltree.py> and
gperftools  both.

>* Instead of making a giant table of everything that needs to be done
>to make stuff fast first, before writing any code, I'd suggest picking
>one operation, figuring out what change would be the biggest
>improvement for it, making that change, checking that it worked, and
>then repeat until that operation is really fast.
Working like that only, firstly optimizing sum operation specifically for
int scalar then will move to other.


>* Did you notice this line on the requirements page? "Having your
>first pull request merged before the GSoC application deadline (May 3)
>is required for your application to be accepted."
Thanks for reminding!
I was too busy with my university exams, I forgot to do that.
Does the merge has to be related to gsoc project, or any other improvement
can be consider?



On Thu, May 2, 2013 at 6:44 PM, Nathaniel Smith <njs at pobox.com> wrote:

> On Thu, May 2, 2013 at 6:26 AM, Arink Verma <arinkverma at iitrpr.ac.in>
> wrote:
> > Yes, we need to ensure that..
> > Code generator can be made, which can create code for table of registered
> > dtype during build time itself.
>
> I'd probably just generate it at run-time on an as-needed basis.
> (I.e., use the full lookup logic the first time, then save the
> result.) New dtypes can be registered, which will mean the tables need
> to change size at runtime anyway. If someone does some strange thing
> like add float16's and float64's, we can do the lookup to determine
> that this should be handled by the float64/float64 loop, and then
> store that information so that the next time it's fast (but we
> probably don't want to be calculating all combinations at build-time,
> which would require running the full type resolution machinery, esp.
> since it wouldn't really bring any benefits that I can see).
>
> * Re: the profiling, I wrote a full oprofile->callgrind format script
> years ago: http://vorpus.org/~njs/op2calltree.py
> Haven't used it in years either but neither oprofile nor kcachegrind
> are terribly fast-moving projects so it's probably still working, or
> could be made so without much work.
> Or easier is to use the gperftools CPU profiler:
> https://gperftools.googlecode.com/svn/trunk/doc/cpuprofile.html
>
> Instead of linking to it at build time, you can just use ctypes:
>
> In [7]: profiler = ctypes.CDLL("libprofiler.so.0")
>
> In [8]: profiler.ProfilerStart("some-file-name-here")
> Out[8]: 1
>
> In [9]: # do stuff here
>
> In [10]: profiler.ProfilerStop()
> PROFILE: interrupts/evictions/bytes = 2/0/592
> Out[10]: 46
>
> Then all the pprof analysis tools are available as described on that
> webpage.
>
> * Please don't trust those random suggestions for possible
> improvements I threw out when writing the original description.
> Probably it's true that FP flag checking and ufunc type lookup are
> expensive, but one should fix what the profile says to fix, not what
> someone guessed might be good to fix based on a few minutes thought.
>
> * Instead of making a giant table of everything that needs to be done
> to make stuff fast first, before writing any code, I'd suggest picking
> one operation, figuring out what change would be the biggest
> improvement for it, making that change, checking that it worked, and
> then repeat until that operation is really fast. Then if there's still
> time pick another operation. Producing a giant todo list isn't very
> productive by itself if there's no time then to actually do all the
> things on the list :-).
>
> * Did you notice this line on the requirements page? "Having your
> first pull request merged before the GSoC application deadline (May 3)
> is required for your application to be accepted."
>
> -n
> _______________________________________________
> NumPy-Discussion mailing list
> NumPy-Discussion at scipy.org
> http://mail.scipy.org/mailman/listinfo/numpy-discussion
>



-- 
Arink
Computer Science and Engineering
Indian Institute of Technology Ropar
www.arinkverma.in
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20130502/ec39b502/attachment.html>

From charlesr.harris at gmail.com  Thu May  2 10:28:58 2013
From: charlesr.harris at gmail.com (Charles R Harris)
Date: Thu, 2 May 2013 08:28:58 -0600
Subject: [Numpy-discussion] nanmean(),
 nanstd() and other "missing" functions for 1.8
In-Reply-To: <CAF6FJiuawKKjnh9G0FfgB2D9c8Yy8e=u6W2ejWsa_qia9x_Fng@mail.gmail.com>
References: <CANNq6F=NCdd1cRWmp7rskr6NJt3sjVvinN1XQVcXjJRgiGk=_A@mail.gmail.com>
	<1888613198476891170@unknownmsgid>
	<CANNq6FnZ4LOA8F1efJW+ZEeJtS3gwYag4mbvOMGSOt=rj_TwsQ@mail.gmail.com>
	<CALGmxELuBevPwVyO+g6ZktCiifZw5XXCY5CSXFiuBvhD8a_G=w@mail.gmail.com>
	<CANNq6FnMnRRH2kBWq_YV-goRTmEojaUBoQNMxy7_F_U5vEBcrQ@mail.gmail.com>
	<CAB6mnx+vsuGPeQ6w79Kvwb+xjXODdcL7rPWTavVc9Vp1iR46qg@mail.gmail.com>
	<CAPJVwBkb4SfLvc8sAgynZyjs=gYTX5qhH1Y5f768LNWzZ6bQ9A@mail.gmail.com>
	<CAF6FJiuTJgOrWsvgKaAkQEP=gPkmzKL0ETT+KaAp_Eva9N2Hug@mail.gmail.com>
	<CAB6mnx+x9HryRa6tOuqCGCQfbhC1a4J0ShRiduX8s7YsM9svmw@mail.gmail.com>
	<CAF6FJiuawKKjnh9G0FfgB2D9c8Yy8e=u6W2ejWsa_qia9x_Fng@mail.gmail.com>
Message-ID: <CAB6mnx+zryzFSUoc=N-iza6_n3UmcDUcSO_W=STa1vnrEpzSYg@mail.gmail.com>

On Thu, May 2, 2013 at 7:47 AM, Robert Kern <robert.kern at gmail.com> wrote:

> On Thu, May 2, 2013 at 2:38 PM, Charles R Harris
> <charlesr.harris at gmail.com> wrote:
> >
> > On Thu, May 2, 2013 at 7:28 AM, Robert Kern <robert.kern at gmail.com>
> wrote:
> >>
> >> On Thu, May 2, 2013 at 12:03 PM, Nathaniel Smith <njs at pobox.com> wrote:
> >> > On 1 May 2013 23:12, "Charles R Harris" <charlesr.harris at gmail.com>
> >> > wrote:
> >> >>
> >> >> On Wed, May 1, 2013 at 7:10 PM, Benjamin Root <ben.root at ou.edu>
> wrote:
> >> >>>
> >> >>> So, to summarize the thread so far:
> >> >>>
> >> >>> Consensus:
> >> >>> np.nanmean()
> >> >>> np.nanstd()
> >> >>> np.minmax()
> >> >>> np.argminmax()
> >> >>>
> >> >>> Vague Consensus:
> >> >>> np.sincos()
> >> >>>
> >> >>
> >> >> If the return of sincos (cossin?) is an array, then it could be
> >> >> reshaped
> >> >> to be exp(1j*x), which together with exp(2*pi*1j*x) would cover some
> >> >> pretty
> >> >> common cases.
> >>
> >> It couldn't be a mere reshape, since the complex dtype requires the
> >> real and imag components to be adjacent to each other. They wouldn't
> >> be so if sincos's return type is an array (nor even the cossin
> >> alternative). It always requires a memory copy (except in the "who
> >> cares?" case of a scalar). Composition with an efficient
> >> np.tocomplex(real, imag) implementation would cover those use cases
> >> whether sincos returns tuples or arrays.
> >
> > I would assume the basic return type would be complex, i.e., the cos/sin
> > adjacent. The cos/sin parts would then be real/imag views into the array.
>
> You mean that the implementation of cossin (to make things easier on
> ourselves) would create an (N,2) contiguous array, fill it with the
> cos and sin results, then reshape it to return the expected (2,N)
>

Just return the transpose.

Chuck
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20130502/eaeb4b30/attachment.html>

From robert.kern at gmail.com  Thu May  2 10:40:43 2013
From: robert.kern at gmail.com (Robert Kern)
Date: Thu, 2 May 2013 15:40:43 +0100
Subject: [Numpy-discussion] nanmean(),
 nanstd() and other "missing" functions for 1.8
In-Reply-To: <CAB6mnx+zryzFSUoc=N-iza6_n3UmcDUcSO_W=STa1vnrEpzSYg@mail.gmail.com>
References: <CANNq6F=NCdd1cRWmp7rskr6NJt3sjVvinN1XQVcXjJRgiGk=_A@mail.gmail.com>
	<1888613198476891170@unknownmsgid>
	<CANNq6FnZ4LOA8F1efJW+ZEeJtS3gwYag4mbvOMGSOt=rj_TwsQ@mail.gmail.com>
	<CALGmxELuBevPwVyO+g6ZktCiifZw5XXCY5CSXFiuBvhD8a_G=w@mail.gmail.com>
	<CANNq6FnMnRRH2kBWq_YV-goRTmEojaUBoQNMxy7_F_U5vEBcrQ@mail.gmail.com>
	<CAB6mnx+vsuGPeQ6w79Kvwb+xjXODdcL7rPWTavVc9Vp1iR46qg@mail.gmail.com>
	<CAPJVwBkb4SfLvc8sAgynZyjs=gYTX5qhH1Y5f768LNWzZ6bQ9A@mail.gmail.com>
	<CAF6FJiuTJgOrWsvgKaAkQEP=gPkmzKL0ETT+KaAp_Eva9N2Hug@mail.gmail.com>
	<CAB6mnx+x9HryRa6tOuqCGCQfbhC1a4J0ShRiduX8s7YsM9svmw@mail.gmail.com>
	<CAF6FJiuawKKjnh9G0FfgB2D9c8Yy8e=u6W2ejWsa_qia9x_Fng@mail.gmail.com>
	<CAB6mnx+zryzFSUoc=N-iza6_n3UmcDUcSO_W=STa1vnrEpzSYg@mail.gmail.com>
Message-ID: <CAF6FJitjxpoXdeR3VH1PL3S__QFi6O4Rkww0EO1=BdQMH6YFFg@mail.gmail.com>

On Thu, May 2, 2013 at 3:28 PM, Charles R Harris
<charlesr.harris at gmail.com> wrote:
>
>
> On Thu, May 2, 2013 at 7:47 AM, Robert Kern <robert.kern at gmail.com> wrote:
>>
>> On Thu, May 2, 2013 at 2:38 PM, Charles R Harris
>> <charlesr.harris at gmail.com> wrote:
>> >
>> > On Thu, May 2, 2013 at 7:28 AM, Robert Kern <robert.kern at gmail.com>
>> > wrote:
>> >>
>> >> On Thu, May 2, 2013 at 12:03 PM, Nathaniel Smith <njs at pobox.com> wrote:
>> >> > On 1 May 2013 23:12, "Charles R Harris" <charlesr.harris at gmail.com>
>> >> > wrote:
>> >> >>
>> >> >> On Wed, May 1, 2013 at 7:10 PM, Benjamin Root <ben.root at ou.edu>
>> >> >> wrote:
>> >> >>>
>> >> >>> So, to summarize the thread so far:
>> >> >>>
>> >> >>> Consensus:
>> >> >>> np.nanmean()
>> >> >>> np.nanstd()
>> >> >>> np.minmax()
>> >> >>> np.argminmax()
>> >> >>>
>> >> >>> Vague Consensus:
>> >> >>> np.sincos()
>> >> >>>
>> >> >>
>> >> >> If the return of sincos (cossin?) is an array, then it could be
>> >> >> reshaped
>> >> >> to be exp(1j*x), which together with exp(2*pi*1j*x) would cover some
>> >> >> pretty
>> >> >> common cases.
>> >>
>> >> It couldn't be a mere reshape, since the complex dtype requires the
>> >> real and imag components to be adjacent to each other. They wouldn't
>> >> be so if sincos's return type is an array (nor even the cossin
>> >> alternative). It always requires a memory copy (except in the "who
>> >> cares?" case of a scalar). Composition with an efficient
>> >> np.tocomplex(real, imag) implementation would cover those use cases
>> >> whether sincos returns tuples or arrays.
>> >
>> > I would assume the basic return type would be complex, i.e., the cos/sin
>> > adjacent. The cos/sin parts would then be real/imag views into the
>> > array.
>>
>> You mean that the implementation of cossin (to make things easier on
>> ourselves) would create an (N,2) contiguous array, fill it with the
>> cos and sin results, then reshape it to return the expected (2,N)
>
> Just return the transpose.

Yes, that's what I was getting at with that sentence. I don't doubt
that that is possible. The problem I was pointing out was in the
following sentence, which you snipped:

  "How would the user then reconstitute the exp(1j*x) result efficiently?"

Please show me the code that the user would write to compute exp(1j*x)
using np.cossin() without memory copies. My suspicion is that it will
be non-intuitive enough that it should always be hidden away into a
well-commented utility function. In that case, I think we should just
provide an np.exp1j() ufunc that just uses the C sincos() function
internally and let np.sincos()/np.cossin() do whatever is most natural
and consistent with other nout>1 ufuncs freed from the constraints of
this use case.

--
Robert Kern


From francesc at continuum.io  Thu May  2 10:51:46 2013
From: francesc at continuum.io (Francesc Alted)
Date: Thu, 02 May 2013 16:51:46 +0200
Subject: [Numpy-discussion] Profiling (was GSoC : Performance parity
 between numpy arrays and Python scalars)
In-Reply-To: <CAPJVwBkh0jda763ps+PF+yHrSJNNeSYdd7=bjUTi-jwjAp-vwg@mail.gmail.com>
References: <CAGY4rcVCN9v4fjWO8QtubQvvC8aTvxsYzBya+QzL3_82C5b4YQ@mail.gmail.com>
	<CAPJVwBkh0jda763ps+PF+yHrSJNNeSYdd7=bjUTi-jwjAp-vwg@mail.gmail.com>
Message-ID: <51827D82.9030006@continuum.io>

On 5/2/13 3:58 PM, Nathaniel Smith wrote:
> callgrind has the *fabulous* kcachegrind front-end, but it only
> measures memory access performance on a simulated machine, which is
> very useful sometimes (if you're trying to optimize cache locality),
> but there's no guarantee that the bottlenecks on its simulated machine
> are the same as the bottlenecks on your real machine.

Agreed, there is no guarantee, but my experience is that kcachegrind 
normally gives you a pretty decent view of cache faults and hence it can 
do pretty good predictions on how this affects your computations.  I 
have used this feature extensively for optimizing parts of the Blosc 
compressor, and I cannot be more happier (to the point that, if it were 
not for Valgrind, I could not figure out many interesting memory access 
optimizations).

-- 
Francesc Alted



From charlesr.harris at gmail.com  Thu May  2 10:57:33 2013
From: charlesr.harris at gmail.com (Charles R Harris)
Date: Thu, 2 May 2013 08:57:33 -0600
Subject: [Numpy-discussion] nanmean(),
 nanstd() and other "missing" functions for 1.8
In-Reply-To: <CAF6FJitjxpoXdeR3VH1PL3S__QFi6O4Rkww0EO1=BdQMH6YFFg@mail.gmail.com>
References: <CANNq6F=NCdd1cRWmp7rskr6NJt3sjVvinN1XQVcXjJRgiGk=_A@mail.gmail.com>
	<1888613198476891170@unknownmsgid>
	<CANNq6FnZ4LOA8F1efJW+ZEeJtS3gwYag4mbvOMGSOt=rj_TwsQ@mail.gmail.com>
	<CALGmxELuBevPwVyO+g6ZktCiifZw5XXCY5CSXFiuBvhD8a_G=w@mail.gmail.com>
	<CANNq6FnMnRRH2kBWq_YV-goRTmEojaUBoQNMxy7_F_U5vEBcrQ@mail.gmail.com>
	<CAB6mnx+vsuGPeQ6w79Kvwb+xjXODdcL7rPWTavVc9Vp1iR46qg@mail.gmail.com>
	<CAPJVwBkb4SfLvc8sAgynZyjs=gYTX5qhH1Y5f768LNWzZ6bQ9A@mail.gmail.com>
	<CAF6FJiuTJgOrWsvgKaAkQEP=gPkmzKL0ETT+KaAp_Eva9N2Hug@mail.gmail.com>
	<CAB6mnx+x9HryRa6tOuqCGCQfbhC1a4J0ShRiduX8s7YsM9svmw@mail.gmail.com>
	<CAF6FJiuawKKjnh9G0FfgB2D9c8Yy8e=u6W2ejWsa_qia9x_Fng@mail.gmail.com>
	<CAB6mnx+zryzFSUoc=N-iza6_n3UmcDUcSO_W=STa1vnrEpzSYg@mail.gmail.com>
	<CAF6FJitjxpoXdeR3VH1PL3S__QFi6O4Rkww0EO1=BdQMH6YFFg@mail.gmail.com>
Message-ID: <CAB6mnxK5YjYR1p1HfpdNa+tOx067Yw79U034FCvZvCBV4q3YZg@mail.gmail.com>

On Thu, May 2, 2013 at 8:40 AM, Robert Kern <robert.kern at gmail.com> wrote:

> On Thu, May 2, 2013 at 3:28 PM, Charles R Harris
> <charlesr.harris at gmail.com> wrote:
> >
> >
> > On Thu, May 2, 2013 at 7:47 AM, Robert Kern <robert.kern at gmail.com>
> wrote:
> >>
> >> On Thu, May 2, 2013 at 2:38 PM, Charles R Harris
> >> <charlesr.harris at gmail.com> wrote:
> >> >
> >> > On Thu, May 2, 2013 at 7:28 AM, Robert Kern <robert.kern at gmail.com>
> >> > wrote:
> >> >>
> >> >> On Thu, May 2, 2013 at 12:03 PM, Nathaniel Smith <njs at pobox.com>
> wrote:
> >> >> > On 1 May 2013 23:12, "Charles R Harris" <charlesr.harris at gmail.com
> >
> >> >> > wrote:
> >> >> >>
> >> >> >> On Wed, May 1, 2013 at 7:10 PM, Benjamin Root <ben.root at ou.edu>
> >> >> >> wrote:
> >> >> >>>
> >> >> >>> So, to summarize the thread so far:
> >> >> >>>
> >> >> >>> Consensus:
> >> >> >>> np.nanmean()
> >> >> >>> np.nanstd()
> >> >> >>> np.minmax()
> >> >> >>> np.argminmax()
> >> >> >>>
> >> >> >>> Vague Consensus:
> >> >> >>> np.sincos()
> >> >> >>>
> >> >> >>
> >> >> >> If the return of sincos (cossin?) is an array, then it could be
> >> >> >> reshaped
> >> >> >> to be exp(1j*x), which together with exp(2*pi*1j*x) would cover
> some
> >> >> >> pretty
> >> >> >> common cases.
> >> >>
> >> >> It couldn't be a mere reshape, since the complex dtype requires the
> >> >> real and imag components to be adjacent to each other. They wouldn't
> >> >> be so if sincos's return type is an array (nor even the cossin
> >> >> alternative). It always requires a memory copy (except in the "who
> >> >> cares?" case of a scalar). Composition with an efficient
> >> >> np.tocomplex(real, imag) implementation would cover those use cases
> >> >> whether sincos returns tuples or arrays.
> >> >
> >> > I would assume the basic return type would be complex, i.e., the
> cos/sin
> >> > adjacent. The cos/sin parts would then be real/imag views into the
> >> > array.
> >>
> >> You mean that the implementation of cossin (to make things easier on
> >> ourselves) would create an (N,2) contiguous array, fill it with the
> >> cos and sin results, then reshape it to return the expected (2,N)
> >
> > Just return the transpose.
>
> Yes, that's what I was getting at with that sentence. I don't doubt
> that that is possible. The problem I was pointing out was in the
> following sentence, which you snipped:
>
>   "How would the user then reconstitute the exp(1j*x) result efficiently?"
>
> Please show me the code that the user would write to compute exp(1j*x)
> using np.cossin() without memory copies. My suspicion is that it will
> be non-intuitive enough that it should always be hidden away into a
> well-commented utility function. In that case, I think we should just
> provide an np.exp1j() ufunc that just uses the C sincos() function
> internally and let np.sincos()/np.cossin() do whatever is most natural
> and consistent with other nout>1 ufuncs freed from the constraints of
> this use case.
>

As you say, have two functions, one of which would use the other with a
view/transpose, whatever. For instance, given exp1j(), have another
function that returns the real/imag parts. The question is what the
underlying function should be and for that I think exp1j() would be a good
choice.

Chuck
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20130502/2b286039/attachment.html>

From charlesr.harris at gmail.com  Thu May  2 11:36:30 2013
From: charlesr.harris at gmail.com (Charles R Harris)
Date: Thu, 2 May 2013 09:36:30 -0600
Subject: [Numpy-discussion] GSoC : Performance parity between numpy
 arrays and Python scalars
In-Reply-To: <CAPJVwB=0E6AaSmN+x7OckkfzuCVTvKuKvV=0Ts2r9Fny5inWvg@mail.gmail.com>
References: <CAJAs4rQ4qQmo4sy5NM=CU9XWSrShOxBn0ti9f-Z4NiZhhehewQ@mail.gmail.com>
	<51816167.6040607@virtualmaterials.com>
	<CAJAs4rRTpvMCFURQy5F5yTC4PqPvyF0TStSMVQ3xG+MAFMGLEw@mail.gmail.com>
	<CAGY4rcUMYSS+9zVXU46CXyw--_TbsUjMOA9TEzgBFbiD_UziMQ@mail.gmail.com>
	<CAJAs4rRhAs4bwu+x3b-fQDi+8eMUuRnZ-C66SmZE6HcHFrY2AQ@mail.gmail.com>
	<CAPJVwB=0E6AaSmN+x7OckkfzuCVTvKuKvV=0Ts2r9Fny5inWvg@mail.gmail.com>
Message-ID: <CAB6mnxJAJxMzVt3r+dBXf_gAKzteuVzJm0eAkjb=7B0o9MDyRw@mail.gmail.com>

On Thu, May 2, 2013 at 7:14 AM, Nathaniel Smith <njs at pobox.com> wrote:

> On Thu, May 2, 2013 at 6:26 AM, Arink Verma <arinkverma at iitrpr.ac.in>
> wrote:
> > Yes, we need to ensure that..
> > Code generator can be made, which can create code for table of registered
> > dtype during build time itself.
>
> I'd probably just generate it at run-time on an as-needed basis.
> (I.e., use the full lookup logic the first time, then save the
> result.) New dtypes can be registered, which will mean the tables need
> to change size at runtime anyway. If someone does some strange thing
> like add float16's and float64's, we can do the lookup to determine
> that this should be handled by the float64/float64 loop, and then
> store that information so that the next time it's fast (but we
> probably don't want to be calculating all combinations at build-time,
> which would require running the full type resolution machinery, esp.
> since it wouldn't really bring any benefits that I can see).
>
> * Re: the profiling, I wrote a full oprofile->callgrind format script
> years ago: http://vorpus.org/~njs/op2calltree.py
> Haven't used it in years either but neither oprofile nor kcachegrind
> are terribly fast-moving projects so it's probably still working, or
> could be made so without much work.
> Or easier is to use the gperftools CPU profiler:
> https://gperftools.googlecode.com/svn/trunk/doc/cpuprofile.html
>
> Instead of linking to it at build time, you can just use ctypes:
>
> In [7]: profiler = ctypes.CDLL("libprofiler.so.0")
>
> In [8]: profiler.ProfilerStart("some-file-name-here")
> Out[8]: 1
>
> In [9]: # do stuff here
>
> In [10]: profiler.ProfilerStop()
> PROFILE: interrupts/evictions/bytes = 2/0/592
> Out[10]: 46
>
> Then all the pprof analysis tools are available as described on that
> webpage.
>
> * Please don't trust those random suggestions for possible
> improvements I threw out when writing the original description.
> Probably it's true that FP flag checking and ufunc type lookup are
> expensive, but one should fix what the profile says to fix, not what
> someone guessed might be good to fix based on a few minutes thought.
>
> * Instead of making a giant table of everything that needs to be done
> to make stuff fast first, before writing any code, I'd suggest picking
> one operation, figuring out what change would be the biggest
> improvement for it, making that change, checking that it worked, and
> then repeat until that operation is really fast. Then if there's still
> time pick another operation. Producing a giant todo list isn't very
> productive by itself if there's no time then to actually do all the
> things on the list :-).
>
> * Did you notice this line on the requirements page? "Having your
> first pull request merged before the GSoC application deadline (May 3)
> is required for your application to be accepted."
>

Where is that last requirement? It seems out of line to me. Arink now has a
pull request, but it looks intrusive enough and needs enough work that I
don't think we can just put it in.

Chuck
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20130502/e3e6aede/attachment.html>

From robert.kern at gmail.com  Thu May  2 11:43:41 2013
From: robert.kern at gmail.com (Robert Kern)
Date: Thu, 2 May 2013 16:43:41 +0100
Subject: [Numpy-discussion] nanmean(),
 nanstd() and other "missing" functions for 1.8
In-Reply-To: <CAB6mnxK5YjYR1p1HfpdNa+tOx067Yw79U034FCvZvCBV4q3YZg@mail.gmail.com>
References: <CANNq6F=NCdd1cRWmp7rskr6NJt3sjVvinN1XQVcXjJRgiGk=_A@mail.gmail.com>
	<1888613198476891170@unknownmsgid>
	<CANNq6FnZ4LOA8F1efJW+ZEeJtS3gwYag4mbvOMGSOt=rj_TwsQ@mail.gmail.com>
	<CALGmxELuBevPwVyO+g6ZktCiifZw5XXCY5CSXFiuBvhD8a_G=w@mail.gmail.com>
	<CANNq6FnMnRRH2kBWq_YV-goRTmEojaUBoQNMxy7_F_U5vEBcrQ@mail.gmail.com>
	<CAB6mnx+vsuGPeQ6w79Kvwb+xjXODdcL7rPWTavVc9Vp1iR46qg@mail.gmail.com>
	<CAPJVwBkb4SfLvc8sAgynZyjs=gYTX5qhH1Y5f768LNWzZ6bQ9A@mail.gmail.com>
	<CAF6FJiuTJgOrWsvgKaAkQEP=gPkmzKL0ETT+KaAp_Eva9N2Hug@mail.gmail.com>
	<CAB6mnx+x9HryRa6tOuqCGCQfbhC1a4J0ShRiduX8s7YsM9svmw@mail.gmail.com>
	<CAF6FJiuawKKjnh9G0FfgB2D9c8Yy8e=u6W2ejWsa_qia9x_Fng@mail.gmail.com>
	<CAB6mnx+zryzFSUoc=N-iza6_n3UmcDUcSO_W=STa1vnrEpzSYg@mail.gmail.com>
	<CAF6FJitjxpoXdeR3VH1PL3S__QFi6O4Rkww0EO1=BdQMH6YFFg@mail.gmail.com>
	<CAB6mnxK5YjYR1p1HfpdNa+tOx067Yw79U034FCvZvCBV4q3YZg@mail.gmail.com>
Message-ID: <CAF6FJitpm5Vx0Sp5cErKRPo038Akkezwj9TtHH8qcR7HKVjMsQ@mail.gmail.com>

On Thu, May 2, 2013 at 3:57 PM, Charles R Harris
<charlesr.harris at gmail.com> wrote:
>
> On Thu, May 2, 2013 at 8:40 AM, Robert Kern <robert.kern at gmail.com> wrote:
>>
>> On Thu, May 2, 2013 at 3:28 PM, Charles R Harris
>> <charlesr.harris at gmail.com> wrote:
>> >
>> >
>> > On Thu, May 2, 2013 at 7:47 AM, Robert Kern <robert.kern at gmail.com>
>> > wrote:
>> >>
>> >> On Thu, May 2, 2013 at 2:38 PM, Charles R Harris
>> >> <charlesr.harris at gmail.com> wrote:
>> >> >
>> >> > On Thu, May 2, 2013 at 7:28 AM, Robert Kern <robert.kern at gmail.com>
>> >> > wrote:
>> >> >>
>> >> >> On Thu, May 2, 2013 at 12:03 PM, Nathaniel Smith <njs at pobox.com>
>> >> >> wrote:
>> >> >> > On 1 May 2013 23:12, "Charles R Harris"
>> >> >> > <charlesr.harris at gmail.com>
>> >> >> > wrote:
>> >> >> >>
>> >> >> >> On Wed, May 1, 2013 at 7:10 PM, Benjamin Root <ben.root at ou.edu>
>> >> >> >> wrote:
>> >> >> >>>
>> >> >> >>> So, to summarize the thread so far:
>> >> >> >>>
>> >> >> >>> Consensus:
>> >> >> >>> np.nanmean()
>> >> >> >>> np.nanstd()
>> >> >> >>> np.minmax()
>> >> >> >>> np.argminmax()
>> >> >> >>>
>> >> >> >>> Vague Consensus:
>> >> >> >>> np.sincos()
>> >> >> >>>
>> >> >> >>
>> >> >> >> If the return of sincos (cossin?) is an array, then it could be
>> >> >> >> reshaped
>> >> >> >> to be exp(1j*x), which together with exp(2*pi*1j*x) would cover
>> >> >> >> some
>> >> >> >> pretty
>> >> >> >> common cases.
>> >> >>
>> >> >> It couldn't be a mere reshape, since the complex dtype requires the
>> >> >> real and imag components to be adjacent to each other. They wouldn't
>> >> >> be so if sincos's return type is an array (nor even the cossin
>> >> >> alternative). It always requires a memory copy (except in the "who
>> >> >> cares?" case of a scalar). Composition with an efficient
>> >> >> np.tocomplex(real, imag) implementation would cover those use cases
>> >> >> whether sincos returns tuples or arrays.
>> >> >
>> >> > I would assume the basic return type would be complex, i.e., the
>> >> > cos/sin
>> >> > adjacent. The cos/sin parts would then be real/imag views into the
>> >> > array.
>> >>
>> >> You mean that the implementation of cossin (to make things easier on
>> >> ourselves) would create an (N,2) contiguous array, fill it with the
>> >> cos and sin results, then reshape it to return the expected (2,N)
>> >
>> > Just return the transpose.
>>
>> Yes, that's what I was getting at with that sentence. I don't doubt
>> that that is possible. The problem I was pointing out was in the
>> following sentence, which you snipped:
>>
>>   "How would the user then reconstitute the exp(1j*x) result efficiently?"
>>
>> Please show me the code that the user would write to compute exp(1j*x)
>> using np.cossin() without memory copies. My suspicion is that it will
>> be non-intuitive enough that it should always be hidden away into a
>> well-commented utility function. In that case, I think we should just
>> provide an np.exp1j() ufunc that just uses the C sincos() function
>> internally and let np.sincos()/np.cossin() do whatever is most natural
>> and consistent with other nout>1 ufuncs freed from the constraints of
>> this use case.
>
> As you say, have two functions, one of which would use the other with a
> view/transpose, whatever. For instance, given exp1j(), have another function
> that returns the real/imag parts. The question is what the underlying
> function should be and for that I think exp1j() would be a good choice.

I don't see why we would bother. Just implement them both as ufuncs
that use a C sincos() function internally and be done with it.
Implementing one in terms of the other requires that the other is not
a ufunc (a minor irritation) and always returns non-contiguous arrays
(a more substantial irritation). There isn't anything to be gained by
implementing one in terms of the other.

--
Robert Kern


From njs at pobox.com  Thu May  2 12:15:36 2013
From: njs at pobox.com (Nathaniel Smith)
Date: Thu, 2 May 2013 12:15:36 -0400
Subject: [Numpy-discussion] Profiling (was GSoC : Performance parity
 between numpy arrays and Python scalars)
In-Reply-To: <51827D82.9030006@continuum.io>
References: <CAGY4rcVCN9v4fjWO8QtubQvvC8aTvxsYzBya+QzL3_82C5b4YQ@mail.gmail.com>
	<CAPJVwBkh0jda763ps+PF+yHrSJNNeSYdd7=bjUTi-jwjAp-vwg@mail.gmail.com>
	<51827D82.9030006@continuum.io>
Message-ID: <CAPJVwBmw03Xa-OKjpG_J2ZyNZp07FeTyTVHe5LbN+J4cHn+cYA@mail.gmail.com>

On Thu, May 2, 2013 at 10:51 AM, Francesc Alted <francesc at continuum.io> wrote:
> On 5/2/13 3:58 PM, Nathaniel Smith wrote:
>> callgrind has the *fabulous* kcachegrind front-end, but it only
>> measures memory access performance on a simulated machine, which is
>> very useful sometimes (if you're trying to optimize cache locality),
>> but there's no guarantee that the bottlenecks on its simulated machine
>> are the same as the bottlenecks on your real machine.
>
> Agreed, there is no guarantee, but my experience is that kcachegrind
> normally gives you a pretty decent view of cache faults and hence it can
> do pretty good predictions on how this affects your computations.  I
> have used this feature extensively for optimizing parts of the Blosc
> compressor, and I cannot be more happier (to the point that, if it were
> not for Valgrind, I could not figure out many interesting memory access
> optimizations).

Right -- if you have code where you know that memory is the bottleneck
(so esp. integer-heavy code), then callgrind is perfect. In fact it
was originally written to make it easier to optimize the bzip2
compressor :-). My point isn't that it's not useful, just, it's a
little more of a specialist tool, so I hesitate to recommend it as the
first profiler for people to reach for. An extreme example would be,
last time I played with this, I found that for numpy scalar float64 *
float64, 50% of the total time was in fiddling with floating point
control registers. But that time would be invisible to callgrind's
measurements...

-n


From raul at virtualmaterials.com  Thu May  2 12:27:09 2013
From: raul at virtualmaterials.com (Raul Cota)
Date: Thu, 02 May 2013 10:27:09 -0600
Subject: [Numpy-discussion] GSoC : Performance parity between numpy
 arrays and Python scalars
In-Reply-To: <CAPJVwB=0E6AaSmN+x7OckkfzuCVTvKuKvV=0Ts2r9Fny5inWvg@mail.gmail.com>
References: <CAJAs4rQ4qQmo4sy5NM=CU9XWSrShOxBn0ti9f-Z4NiZhhehewQ@mail.gmail.com>
	<51816167.6040607@virtualmaterials.com>
	<CAJAs4rRTpvMCFURQy5F5yTC4PqPvyF0TStSMVQ3xG+MAFMGLEw@mail.gmail.com>
	<CAGY4rcUMYSS+9zVXU46CXyw--_TbsUjMOA9TEzgBFbiD_UziMQ@mail.gmail.com>
	<CAJAs4rRhAs4bwu+x3b-fQDi+8eMUuRnZ-C66SmZE6HcHFrY2AQ@mail.gmail.com>
	<CAPJVwB=0E6AaSmN+x7OckkfzuCVTvKuKvV=0Ts2r9Fny5inWvg@mail.gmail.com>
Message-ID: <518293DD.1010004@virtualmaterials.com>


For the sake of completeness, I don't think I ever mentioned what I used 
to profile when I was working on speeding up the scalars. I used AQTime 
7. It is commercial and only for Windows (as far as I know). It works 
great and it gave me fairly accurate timings and all sorts of visual 
navigation features. I do have to mock around with the numpy code every 
time I want to compile it to get it to play nicely with Visual Studio to 
generate the proper bindings for the profiler.

Raul


On 02/05/2013 7:14 AM, Nathaniel Smith wrote:
> On Thu, May 2, 2013 at 6:26 AM, Arink Verma <arinkverma at iitrpr.ac.in> wrote:
>> Yes, we need to ensure that..
>> Code generator can be made, which can create code for table of registered
>> dtype during build time itself.
> I'd probably just generate it at run-time on an as-needed basis.
> (I.e., use the full lookup logic the first time, then save the
> result.) New dtypes can be registered, which will mean the tables need
> to change size at runtime anyway. If someone does some strange thing
> like add float16's and float64's, we can do the lookup to determine
> that this should be handled by the float64/float64 loop, and then
> store that information so that the next time it's fast (but we
> probably don't want to be calculating all combinations at build-time,
> which would require running the full type resolution machinery, esp.
> since it wouldn't really bring any benefits that I can see).
>
> * Re: the profiling, I wrote a full oprofile->callgrind format script
> years ago: http://vorpus.org/~njs/op2calltree.py
> Haven't used it in years either but neither oprofile nor kcachegrind
> are terribly fast-moving projects so it's probably still working, or
> could be made so without much work.
> Or easier is to use the gperftools CPU profiler:
> https://gperftools.googlecode.com/svn/trunk/doc/cpuprofile.html
>
> Instead of linking to it at build time, you can just use ctypes:
>
> In [7]: profiler = ctypes.CDLL("libprofiler.so.0")
>
> In [8]: profiler.ProfilerStart("some-file-name-here")
> Out[8]: 1
>
> In [9]: # do stuff here
>
> In [10]: profiler.ProfilerStop()
> PROFILE: interrupts/evictions/bytes = 2/0/592
> Out[10]: 46
>
> Then all the pprof analysis tools are available as described on that webpage.
>
> * Please don't trust those random suggestions for possible
> improvements I threw out when writing the original description.
> Probably it's true that FP flag checking and ufunc type lookup are
> expensive, but one should fix what the profile says to fix, not what
> someone guessed might be good to fix based on a few minutes thought.
>
> * Instead of making a giant table of everything that needs to be done
> to make stuff fast first, before writing any code, I'd suggest picking
> one operation, figuring out what change would be the biggest
> improvement for it, making that change, checking that it worked, and
> then repeat until that operation is really fast. Then if there's still
> time pick another operation. Producing a giant todo list isn't very
> productive by itself if there's no time then to actually do all the
> things on the list :-).
>
> * Did you notice this line on the requirements page? "Having your
> first pull request merged before the GSoC application deadline (May 3)
> is required for your application to be accepted."
>
> -n
> _______________________________________________
> NumPy-Discussion mailing list
> NumPy-Discussion at scipy.org
> http://mail.scipy.org/mailman/listinfo/numpy-discussion
>



From chris.barker at noaa.gov  Thu May  2 12:38:45 2013
From: chris.barker at noaa.gov (Chris Barker - NOAA Federal)
Date: Thu, 2 May 2013 09:38:45 -0700
Subject: [Numpy-discussion] numPy not imported into Python
In-Reply-To: <CAFZWFtMudoUTM4H8XE9UVkQ5ZNRW2aPLhiK_CaypCnhnY3yPGA@mail.gmail.com>
References: <CAFZWFtOnGJEc2kR4VokuOERzBccqJbo4qrOabG_NmYmv0UiQgg@mail.gmail.com>
	<CADeni3usRQcdVRKcKVoNPxdZtMST1u=k1Qz1ZR0nuTMiZtGgPQ@mail.gmail.com>
	<CAFZWFtMudoUTM4H8XE9UVkQ5ZNRW2aPLhiK_CaypCnhnY3yPGA@mail.gmail.com>
Message-ID: <CALGmxEJzy=nAqY9H6ZO5xOE2Gw0HJoGnnvEbUhkV6h7EZMkZJA@mail.gmail.com>

On Wed, May 1, 2013 at 6:52 PM, Mark Micklich <mmick003 at ucr.edu> wrote:

> Oh!  I imported numpy and that worked.  I probably should have mentioned
> I'm working through the 2005 "*Numerical Methods in Engineering with
> Python*" textbook from the school library.  The examples are still good,
> but the Import statement used in the book is obsolete.  Thanks for the
> quick reply.
>
> jsut so you know, there may be other subtle issues -- numpy has mostly the
same API as numarray, but not exactly, so if somethign doesn't appear to be
working as the book suggests, make sure to check out the current numpy docs.

Note that there seems to be a new addition of the book:

Numerical Methods in Engineering with Python 3

I assume it uses a recent numpy version. (too bad there isn't a py2, recent
numpy version, but there you go...)

-Chris





> Mark
>
> On Wed, May 1, 2013 at 6:47 PM, Christopher Hanley <chanley at gmail.com>wrote:
>
>> Mark,
>>
>> Numpy is not numarray. Numarray is an older package that has long since
>> been replaced by numpy.  You should only use numpy in any development from
>> now on.
>>
>> Chris
>>
>>
>> On Wednesday, May 1, 2013, Mark Micklich wrote:
>>
>>> Hello -- After installing numPy, I'm getting the following error message
>>> when attempting to import numarray:
>>>
>>>        ImportError: No module named numarray
>>>
>>> I do have numPy installed.  I'm running under Lubuntu 12.10 and the Spyder
>>> 2.1.10 IDE.  I'm fairly new to developing Python on Linux.  I assume there
>>> is some path issue, but I'm not clear where to start.  If numPy is
>>> installed, how to I point Spyder to the numPy modules so I can get numarray
>>> to work?
>>>
>>> Thanks,
>>> Mark
>>>
>>>
>> _______________________________________________
>> NumPy-Discussion mailing list
>> NumPy-Discussion at scipy.org
>> http://mail.scipy.org/mailman/listinfo/numpy-discussion
>>
>>
>
> _______________________________________________
> NumPy-Discussion mailing list
> NumPy-Discussion at scipy.org
> http://mail.scipy.org/mailman/listinfo/numpy-discussion
>
>


-- 

Christopher Barker, Ph.D.
Oceanographer

Emergency Response Division
NOAA/NOS/OR&R            (206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115       (206) 526-6317   main reception

Chris.Barker at noaa.gov
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20130502/243cbdc4/attachment.html>

From pav at iki.fi  Thu May  2 12:45:25 2013
From: pav at iki.fi (Pauli Virtanen)
Date: Thu, 2 May 2013 16:45:25 +0000 (UTC)
Subject: [Numpy-discussion] GSoC : Performance parity between numpy
	arrays and Python scalars
References: <CAJAs4rQ4qQmo4sy5NM=CU9XWSrShOxBn0ti9f-Z4NiZhhehewQ@mail.gmail.com>
	<51816167.6040607@virtualmaterials.com>
	<CAJAs4rRTpvMCFURQy5F5yTC4PqPvyF0TStSMVQ3xG+MAFMGLEw@mail.gmail.com>
	<CAGY4rcUMYSS+9zVXU46CXyw--_TbsUjMOA9TEzgBFbiD_UziMQ@mail.gmail.com>
	<CAJAs4rRhAs4bwu+x3b-fQDi+8eMUuRnZ-C66SmZE6HcHFrY2AQ@mail.gmail.com>
	<CAPJVwB=0E6AaSmN+x7OckkfzuCVTvKuKvV=0Ts2r9Fny5inWvg@mail.gmail.com>
	<CAB6mnxJAJxMzVt3r+dBXf_gAKzteuVzJm0eAkjb=7B0o9MDyRw@mail.gmail.com>
Message-ID: <loom.20130502T183915-278@post.gmane.org>

Charles R Harris <charlesr.harris <at> gmail.com> writes:
[clip]
> * Did you notice this line on the requirements page? "Having your
> first pull request merged before the GSoC application deadline (May 3)
> is required for your application to be accepted."
> 
> Where is that last requirement? It seems out of line to me.
> Arink now has a pull request, but it looks intrusive enough
> and needs enough work that I don't think we can just put it in.

Well, we wrote so here:

    http://projects.scipy.org/scipy/wiki/SummerofCodeIdeas?

but that's maybe just a mistake -- PSF states exactly the opposite:

    http://wiki.python.org/moin/SummerOfCode/ApplicationTemplate2013

-- 
Pauli Virtanen



From ralf.gommers at gmail.com  Thu May  2 13:49:26 2013
From: ralf.gommers at gmail.com (Ralf Gommers)
Date: Thu, 2 May 2013 19:49:26 +0200
Subject: [Numpy-discussion] GSoC : Performance parity between numpy
 arrays and Python scalars
In-Reply-To: <loom.20130502T183915-278@post.gmane.org>
References: <CAJAs4rQ4qQmo4sy5NM=CU9XWSrShOxBn0ti9f-Z4NiZhhehewQ@mail.gmail.com>
	<51816167.6040607@virtualmaterials.com>
	<CAJAs4rRTpvMCFURQy5F5yTC4PqPvyF0TStSMVQ3xG+MAFMGLEw@mail.gmail.com>
	<CAGY4rcUMYSS+9zVXU46CXyw--_TbsUjMOA9TEzgBFbiD_UziMQ@mail.gmail.com>
	<CAJAs4rRhAs4bwu+x3b-fQDi+8eMUuRnZ-C66SmZE6HcHFrY2AQ@mail.gmail.com>
	<CAPJVwB=0E6AaSmN+x7OckkfzuCVTvKuKvV=0Ts2r9Fny5inWvg@mail.gmail.com>
	<CAB6mnxJAJxMzVt3r+dBXf_gAKzteuVzJm0eAkjb=7B0o9MDyRw@mail.gmail.com>
	<loom.20130502T183915-278@post.gmane.org>
Message-ID: <CABL7CQhxHqueOyzWMtE_VL6J-_XEkuUD4SvxmHq6tLBWE6Ek=Q@mail.gmail.com>

On Thu, May 2, 2013 at 6:45 PM, Pauli Virtanen <pav at iki.fi> wrote:

> Charles R Harris <charlesr.harris <at> gmail.com> writes:
> [clip]
> > * Did you notice this line on the requirements page? "Having your
> > first pull request merged before the GSoC application deadline (May 3)
> > is required for your application to be accepted."
> >
> > Where is that last requirement? It seems out of line to me.
> > Arink now has a pull request, but it looks intrusive enough
> > and needs enough work that I don't think we can just put it in.
>
> Well, we wrote so here:
>
>     http://projects.scipy.org/scipy/wiki/SummerofCodeIdeas
>
> but that's maybe just a mistake -- PSF states exactly the opposite:
>
>     http://wiki.python.org/moin/SummerOfCode/ApplicationTemplate2013
>

It wasn't a mistake - the part of a PR process that is most interesting in
the context of evaluating GSoC applications is the dialogue and how the
submitter deals with feedback.

I forgot to add on that page (although I think it was in one of my emails)
that the patch shouldn't be completely trivial - fixing a typo doesn't
really tell us all that much. But in this case Chuck's suggestion on the PR
of how to get something merged looks fine.

Cheers,
Ralf
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20130502/7a151335/attachment.html>

From charlesr.harris at gmail.com  Thu May  2 15:54:15 2013
From: charlesr.harris at gmail.com (Charles R Harris)
Date: Thu, 2 May 2013 13:54:15 -0600
Subject: [Numpy-discussion] GSoC : Performance parity between numpy
 arrays and Python scalars
In-Reply-To: <CABL7CQhxHqueOyzWMtE_VL6J-_XEkuUD4SvxmHq6tLBWE6Ek=Q@mail.gmail.com>
References: <CAJAs4rQ4qQmo4sy5NM=CU9XWSrShOxBn0ti9f-Z4NiZhhehewQ@mail.gmail.com>
	<51816167.6040607@virtualmaterials.com>
	<CAJAs4rRTpvMCFURQy5F5yTC4PqPvyF0TStSMVQ3xG+MAFMGLEw@mail.gmail.com>
	<CAGY4rcUMYSS+9zVXU46CXyw--_TbsUjMOA9TEzgBFbiD_UziMQ@mail.gmail.com>
	<CAJAs4rRhAs4bwu+x3b-fQDi+8eMUuRnZ-C66SmZE6HcHFrY2AQ@mail.gmail.com>
	<CAPJVwB=0E6AaSmN+x7OckkfzuCVTvKuKvV=0Ts2r9Fny5inWvg@mail.gmail.com>
	<CAB6mnxJAJxMzVt3r+dBXf_gAKzteuVzJm0eAkjb=7B0o9MDyRw@mail.gmail.com>
	<loom.20130502T183915-278@post.gmane.org>
	<CABL7CQhxHqueOyzWMtE_VL6J-_XEkuUD4SvxmHq6tLBWE6Ek=Q@mail.gmail.com>
Message-ID: <CAB6mnx+tY2ibkQQ=ZsXYJdMnZX1B7g2zA=3QDprXyTkP_NLoKg@mail.gmail.com>

On Thu, May 2, 2013 at 11:49 AM, Ralf Gommers <ralf.gommers at gmail.com>wrote:

>
>
>
> On Thu, May 2, 2013 at 6:45 PM, Pauli Virtanen <pav at iki.fi> wrote:
>
>> Charles R Harris <charlesr.harris <at> gmail.com> writes:
>> [clip]
>> > * Did you notice this line on the requirements page? "Having your
>> > first pull request merged before the GSoC application deadline (May 3)
>> > is required for your application to be accepted."
>> >
>> > Where is that last requirement? It seems out of line to me.
>> > Arink now has a pull request, but it looks intrusive enough
>> > and needs enough work that I don't think we can just put it in.
>>
>> Well, we wrote so here:
>>
>>     http://projects.scipy.org/scipy/wiki/SummerofCodeIdeas
>>
>> but that's maybe just a mistake -- PSF states exactly the opposite:
>>
>>     http://wiki.python.org/moin/SummerOfCode/ApplicationTemplate2013
>>
>
> It wasn't a mistake - the part of a PR process that is most interesting in
> the context of evaluating GSoC applications is the dialogue and how the
> submitter deals with feedback.
>
> I forgot to add on that page (although I think it was in one of my emails)
> that the patch shouldn't be completely trivial - fixing a typo doesn't
> really tell us all that much. But in this case Chuck's suggestion on the PR
> of how to get something merged looks fine.
>
>
My feeling is that learning to work with the community is part of the
process after acceptance and one of the reasons there are mentors. You
might get some bad choices skipping the submission/acceptance bit, but you
might also close the door on people who are new to the whole thing.
Ideally, the applicants would already have involved themselves with the
community, practically that may often not be ths case.

Chuck
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20130502/cdc34fcd/attachment.html>

From ralf.gommers at gmail.com  Thu May  2 18:29:35 2013
From: ralf.gommers at gmail.com (Ralf Gommers)
Date: Fri, 3 May 2013 00:29:35 +0200
Subject: [Numpy-discussion] GSoC : Performance parity between numpy
 arrays and Python scalars
In-Reply-To: <CAB6mnx+tY2ibkQQ=ZsXYJdMnZX1B7g2zA=3QDprXyTkP_NLoKg@mail.gmail.com>
References: <CAJAs4rQ4qQmo4sy5NM=CU9XWSrShOxBn0ti9f-Z4NiZhhehewQ@mail.gmail.com>
	<51816167.6040607@virtualmaterials.com>
	<CAJAs4rRTpvMCFURQy5F5yTC4PqPvyF0TStSMVQ3xG+MAFMGLEw@mail.gmail.com>
	<CAGY4rcUMYSS+9zVXU46CXyw--_TbsUjMOA9TEzgBFbiD_UziMQ@mail.gmail.com>
	<CAJAs4rRhAs4bwu+x3b-fQDi+8eMUuRnZ-C66SmZE6HcHFrY2AQ@mail.gmail.com>
	<CAPJVwB=0E6AaSmN+x7OckkfzuCVTvKuKvV=0Ts2r9Fny5inWvg@mail.gmail.com>
	<CAB6mnxJAJxMzVt3r+dBXf_gAKzteuVzJm0eAkjb=7B0o9MDyRw@mail.gmail.com>
	<loom.20130502T183915-278@post.gmane.org>
	<CABL7CQhxHqueOyzWMtE_VL6J-_XEkuUD4SvxmHq6tLBWE6Ek=Q@mail.gmail.com>
	<CAB6mnx+tY2ibkQQ=ZsXYJdMnZX1B7g2zA=3QDprXyTkP_NLoKg@mail.gmail.com>
Message-ID: <CABL7CQhG1w8fzcNf1Pw1Dzg5ZMA9yDr4xgQL0=0RbaRfC1jEbw@mail.gmail.com>

On Thu, May 2, 2013 at 9:54 PM, Charles R Harris
<charlesr.harris at gmail.com>wrote:

>
>
> On Thu, May 2, 2013 at 11:49 AM, Ralf Gommers <ralf.gommers at gmail.com>wrote:
>
>>
>>
>>
>> On Thu, May 2, 2013 at 6:45 PM, Pauli Virtanen <pav at iki.fi> wrote:
>>
>>> Charles R Harris <charlesr.harris <at> gmail.com> writes:
>>> [clip]
>>> > * Did you notice this line on the requirements page? "Having your
>>> > first pull request merged before the GSoC application deadline (May 3)
>>> > is required for your application to be accepted."
>>> >
>>> > Where is that last requirement? It seems out of line to me.
>>> > Arink now has a pull request, but it looks intrusive enough
>>> > and needs enough work that I don't think we can just put it in.
>>>
>>> Well, we wrote so here:
>>>
>>>     http://projects.scipy.org/scipy/wiki/SummerofCodeIdeas
>>>
>>> but that's maybe just a mistake -- PSF states exactly the opposite:
>>>
>>>     http://wiki.python.org/moin/SummerOfCode/ApplicationTemplate2013
>>>
>>
>> It wasn't a mistake - the part of a PR process that is most interesting
>> in the context of evaluating GSoC applications is the dialogue and how the
>> submitter deals with feedback.
>>
>> I forgot to add on that page (although I think it was in one of my
>> emails) that the patch shouldn't be completely trivial - fixing a typo
>> doesn't really tell us all that much. But in this case Chuck's suggestion
>> on the PR of how to get something merged looks fine.
>>
>>
> My feeling is that learning to work with the community is part of the
> process after acceptance and one of the reasons there are mentors. You
> might get some bad choices skipping the submission/acceptance bit, but you
> might also close the door on people who are new to the whole thing.
> Ideally, the applicants would already have involved themselves with the
> community, practically that may often not be ths case.
>

You may be right in all of that, but since there's a good chance that there
are more applicants than slots I'd rather not make those bad choices if
they're acceptable.

Right now we have three solid proposals, from Arink, Blake and Surya. If
we're lucky we'll get three slots, but if not then we'll have a tough
choice to make.

The application deadline is tomorrow, so now is the time for final tweaks
to the proposals. After that of course the plan can still be worked out
more, but it can't be edited on Melange anymore.

Ralf
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20130503/68bca2f2/attachment.html>

From ralf.gommers at gmail.com  Thu May  2 18:30:52 2013
From: ralf.gommers at gmail.com (Ralf Gommers)
Date: Fri, 3 May 2013 00:30:52 +0200
Subject: [Numpy-discussion] GSoC : Performance parity between numpy
 arrays and Python scalars
In-Reply-To: <CABL7CQhG1w8fzcNf1Pw1Dzg5ZMA9yDr4xgQL0=0RbaRfC1jEbw@mail.gmail.com>
References: <CAJAs4rQ4qQmo4sy5NM=CU9XWSrShOxBn0ti9f-Z4NiZhhehewQ@mail.gmail.com>
	<51816167.6040607@virtualmaterials.com>
	<CAJAs4rRTpvMCFURQy5F5yTC4PqPvyF0TStSMVQ3xG+MAFMGLEw@mail.gmail.com>
	<CAGY4rcUMYSS+9zVXU46CXyw--_TbsUjMOA9TEzgBFbiD_UziMQ@mail.gmail.com>
	<CAJAs4rRhAs4bwu+x3b-fQDi+8eMUuRnZ-C66SmZE6HcHFrY2AQ@mail.gmail.com>
	<CAPJVwB=0E6AaSmN+x7OckkfzuCVTvKuKvV=0Ts2r9Fny5inWvg@mail.gmail.com>
	<CAB6mnxJAJxMzVt3r+dBXf_gAKzteuVzJm0eAkjb=7B0o9MDyRw@mail.gmail.com>
	<loom.20130502T183915-278@post.gmane.org>
	<CABL7CQhxHqueOyzWMtE_VL6J-_XEkuUD4SvxmHq6tLBWE6Ek=Q@mail.gmail.com>
	<CAB6mnx+tY2ibkQQ=ZsXYJdMnZX1B7g2zA=3QDprXyTkP_NLoKg@mail.gmail.com>
	<CABL7CQhG1w8fzcNf1Pw1Dzg5ZMA9yDr4xgQL0=0RbaRfC1jEbw@mail.gmail.com>
Message-ID: <CABL7CQi_qbZRaoyjhfhBR_=Eix=hwSJcCmqYzMAzQ7+MWB4hQA@mail.gmail.com>

On Fri, May 3, 2013 at 12:29 AM, Ralf Gommers <ralf.gommers at gmail.com>wrote:

>
>
>
> On Thu, May 2, 2013 at 9:54 PM, Charles R Harris <
> charlesr.harris at gmail.com> wrote:
>
>>
>>
>> On Thu, May 2, 2013 at 11:49 AM, Ralf Gommers <ralf.gommers at gmail.com>wrote:
>>
>>>
>>>
>>>
>>> On Thu, May 2, 2013 at 6:45 PM, Pauli Virtanen <pav at iki.fi> wrote:
>>>
>>>> Charles R Harris <charlesr.harris <at> gmail.com> writes:
>>>> [clip]
>>>> > * Did you notice this line on the requirements page? "Having your
>>>> > first pull request merged before the GSoC application deadline (May 3)
>>>> > is required for your application to be accepted."
>>>> >
>>>> > Where is that last requirement? It seems out of line to me.
>>>> > Arink now has a pull request, but it looks intrusive enough
>>>> > and needs enough work that I don't think we can just put it in.
>>>>
>>>> Well, we wrote so here:
>>>>
>>>>     http://projects.scipy.org/scipy/wiki/SummerofCodeIdeas
>>>>
>>>> but that's maybe just a mistake -- PSF states exactly the opposite:
>>>>
>>>>     http://wiki.python.org/moin/SummerOfCode/ApplicationTemplate2013
>>>>
>>>
>>> It wasn't a mistake - the part of a PR process that is most interesting
>>> in the context of evaluating GSoC applications is the dialogue and how the
>>> submitter deals with feedback.
>>>
>>> I forgot to add on that page (although I think it was in one of my
>>> emails) that the patch shouldn't be completely trivial - fixing a typo
>>> doesn't really tell us all that much. But in this case Chuck's suggestion
>>> on the PR of how to get something merged looks fine.
>>>
>>>
>> My feeling is that learning to work with the community is part of the
>> process after acceptance and one of the reasons there are mentors. You
>> might get some bad choices skipping the submission/acceptance bit, but you
>> might also close the door on people who are new to the whole thing.
>> Ideally, the applicants would already have involved themselves with the
>> community, practically that may often not be ths case.
>>
>
> You may be right in all of that, but since there's a good chance that
> there are more applicants than slots I'd rather not make those bad choices
> if they're acceptable.
>

acceptable --> avoidable


> Right now we have three solid proposals, from Arink, Blake and Surya. If
> we're lucky we'll get three slots, but if not then we'll have a tough
> choice to make.
>
> The application deadline is tomorrow, so now is the time for final tweaks
> to the proposals. After that of course the plan can still be worked out
> more, but it can't be edited on Melange anymore.
>
> Ralf
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20130503/df335df8/attachment.html>

From josef.pktd at gmail.com  Thu May  2 20:47:21 2013
From: josef.pktd at gmail.com (josef.pktd at gmail.com)
Date: Thu, 2 May 2013 20:47:21 -0400
Subject: [Numpy-discussion] GSoC : Performance parity between numpy
 arrays and Python scalars
In-Reply-To: <CABL7CQi_qbZRaoyjhfhBR_=Eix=hwSJcCmqYzMAzQ7+MWB4hQA@mail.gmail.com>
References: <CAJAs4rQ4qQmo4sy5NM=CU9XWSrShOxBn0ti9f-Z4NiZhhehewQ@mail.gmail.com>
	<51816167.6040607@virtualmaterials.com>
	<CAJAs4rRTpvMCFURQy5F5yTC4PqPvyF0TStSMVQ3xG+MAFMGLEw@mail.gmail.com>
	<CAGY4rcUMYSS+9zVXU46CXyw--_TbsUjMOA9TEzgBFbiD_UziMQ@mail.gmail.com>
	<CAJAs4rRhAs4bwu+x3b-fQDi+8eMUuRnZ-C66SmZE6HcHFrY2AQ@mail.gmail.com>
	<CAPJVwB=0E6AaSmN+x7OckkfzuCVTvKuKvV=0Ts2r9Fny5inWvg@mail.gmail.com>
	<CAB6mnxJAJxMzVt3r+dBXf_gAKzteuVzJm0eAkjb=7B0o9MDyRw@mail.gmail.com>
	<loom.20130502T183915-278@post.gmane.org>
	<CABL7CQhxHqueOyzWMtE_VL6J-_XEkuUD4SvxmHq6tLBWE6Ek=Q@mail.gmail.com>
	<CAB6mnx+tY2ibkQQ=ZsXYJdMnZX1B7g2zA=3QDprXyTkP_NLoKg@mail.gmail.com>
	<CABL7CQhG1w8fzcNf1Pw1Dzg5ZMA9yDr4xgQL0=0RbaRfC1jEbw@mail.gmail.com>
	<CABL7CQi_qbZRaoyjhfhBR_=Eix=hwSJcCmqYzMAzQ7+MWB4hQA@mail.gmail.com>
Message-ID: <CAMMTP+Du-qvpTtBZLkveTGe1BDy_is6rx2TgfhzHFgGTF6Lg-g@mail.gmail.com>

On Thu, May 2, 2013 at 6:30 PM, Ralf Gommers <ralf.gommers at gmail.com> wrote:
>
>
>
> On Fri, May 3, 2013 at 12:29 AM, Ralf Gommers <ralf.gommers at gmail.com>
> wrote:
>>
>>
>>
>>
>> On Thu, May 2, 2013 at 9:54 PM, Charles R Harris
>> <charlesr.harris at gmail.com> wrote:
>>>
>>>
>>>
>>> On Thu, May 2, 2013 at 11:49 AM, Ralf Gommers <ralf.gommers at gmail.com>
>>> wrote:
>>>>
>>>>
>>>>
>>>>
>>>> On Thu, May 2, 2013 at 6:45 PM, Pauli Virtanen <pav at iki.fi> wrote:
>>>>>
>>>>> Charles R Harris <charlesr.harris <at> gmail.com> writes:
>>>>> [clip]
>>>>> > * Did you notice this line on the requirements page? "Having your
>>>>> > first pull request merged before the GSoC application deadline (May
>>>>> > 3)
>>>>> > is required for your application to be accepted."
>>>>> >
>>>>> > Where is that last requirement? It seems out of line to me.
>>>>> > Arink now has a pull request, but it looks intrusive enough
>>>>> > and needs enough work that I don't think we can just put it in.
>>>>>
>>>>> Well, we wrote so here:
>>>>>
>>>>>     http://projects.scipy.org/scipy/wiki/SummerofCodeIdeas
>>>>>
>>>>> but that's maybe just a mistake -- PSF states exactly the opposite:
>>>>>
>>>>>     http://wiki.python.org/moin/SummerOfCode/ApplicationTemplate2013
>>>>
>>>>
>>>> It wasn't a mistake - the part of a PR process that is most interesting
>>>> in the context of evaluating GSoC applications is the dialogue and how the
>>>> submitter deals with feedback.
>>>>
>>>> I forgot to add on that page (although I think it was in one of my
>>>> emails) that the patch shouldn't be completely trivial - fixing a typo
>>>> doesn't really tell us all that much. But in this case Chuck's suggestion on
>>>> the PR of how to get something merged looks fine.
>>>>
>>>
>>> My feeling is that learning to work with the community is part of the
>>> process after acceptance and one of the reasons there are mentors. You might
>>> get some bad choices skipping the submission/acceptance bit, but you might
>>> also close the door on people who are new to the whole thing. Ideally, the
>>> applicants would already have involved themselves with the community,
>>> practically that may often not be ths case.
>>
>>
>> You may be right in all of that, but since there's a good chance that
>> there are more applicants than slots I'd rather not make those bad choices
>> if they're acceptable.
>
>
> acceptable --> avoidable
>
>>
>> Right now we have three solid proposals, from Arink, Blake and Surya. If
>> we're lucky we'll get three slots, but if not then we'll have a tough choice
>> to make.
>>
>> The application deadline is tomorrow, so now is the time for final tweaks
>> to the proposals. After that of course the plan can still be worked out
>> more, but it can't be edited on Melange anymore.

Terri can still make it editable on Melange if necessary.

Josef

>>
>> Ralf
>>
>>
>
>
> _______________________________________________
> NumPy-Discussion mailing list
> NumPy-Discussion at scipy.org
> http://mail.scipy.org/mailman/listinfo/numpy-discussion
>


From charlesr.harris at gmail.com  Fri May  3 13:22:37 2013
From: charlesr.harris at gmail.com (Charles R Harris)
Date: Fri, 3 May 2013 11:22:37 -0600
Subject: [Numpy-discussion] GSoC : Performance parity between numpy
 arrays and Python scalars
In-Reply-To: <CAMMTP+Du-qvpTtBZLkveTGe1BDy_is6rx2TgfhzHFgGTF6Lg-g@mail.gmail.com>
References: <CAJAs4rQ4qQmo4sy5NM=CU9XWSrShOxBn0ti9f-Z4NiZhhehewQ@mail.gmail.com>
	<51816167.6040607@virtualmaterials.com>
	<CAJAs4rRTpvMCFURQy5F5yTC4PqPvyF0TStSMVQ3xG+MAFMGLEw@mail.gmail.com>
	<CAGY4rcUMYSS+9zVXU46CXyw--_TbsUjMOA9TEzgBFbiD_UziMQ@mail.gmail.com>
	<CAJAs4rRhAs4bwu+x3b-fQDi+8eMUuRnZ-C66SmZE6HcHFrY2AQ@mail.gmail.com>
	<CAPJVwB=0E6AaSmN+x7OckkfzuCVTvKuKvV=0Ts2r9Fny5inWvg@mail.gmail.com>
	<CAB6mnxJAJxMzVt3r+dBXf_gAKzteuVzJm0eAkjb=7B0o9MDyRw@mail.gmail.com>
	<loom.20130502T183915-278@post.gmane.org>
	<CABL7CQhxHqueOyzWMtE_VL6J-_XEkuUD4SvxmHq6tLBWE6Ek=Q@mail.gmail.com>
	<CAB6mnx+tY2ibkQQ=ZsXYJdMnZX1B7g2zA=3QDprXyTkP_NLoKg@mail.gmail.com>
	<CABL7CQhG1w8fzcNf1Pw1Dzg5ZMA9yDr4xgQL0=0RbaRfC1jEbw@mail.gmail.com>
	<CABL7CQi_qbZRaoyjhfhBR_=Eix=hwSJcCmqYzMAzQ7+MWB4hQA@mail.gmail.com>
	<CAMMTP+Du-qvpTtBZLkveTGe1BDy_is6rx2TgfhzHFgGTF6Lg-g@mail.gmail.com>
Message-ID: <CAB6mnxLf26ZT5qFGrTgdC36eBR2N1DoXGzUXkkMca29uOxdQaw@mail.gmail.com>

On Thu, May 2, 2013 at 6:47 PM, <josef.pktd at gmail.com> wrote:

> On Thu, May 2, 2013 at 6:30 PM, Ralf Gommers <ralf.gommers at gmail.com>
> wrote:
> >
> >
> >
> > On Fri, May 3, 2013 at 12:29 AM, Ralf Gommers <ralf.gommers at gmail.com>
> > wrote:
> >>
> >>
> >>
> >>
> >> On Thu, May 2, 2013 at 9:54 PM, Charles R Harris
> >> <charlesr.harris at gmail.com> wrote:
> >>>
> >>>
> >>>
> >>> On Thu, May 2, 2013 at 11:49 AM, Ralf Gommers <ralf.gommers at gmail.com>
> >>> wrote:
> >>>>
> >>>>
> >>>>
> >>>>
> >>>> On Thu, May 2, 2013 at 6:45 PM, Pauli Virtanen <pav at iki.fi> wrote:
> >>>>>
> >>>>> Charles R Harris <charlesr.harris <at> gmail.com> writes:
> >>>>> [clip]
> >>>>> > * Did you notice this line on the requirements page? "Having your
> >>>>> > first pull request merged before the GSoC application deadline (May
> >>>>> > 3)
> >>>>> > is required for your application to be accepted."
> >>>>> >
> >>>>> > Where is that last requirement? It seems out of line to me.
> >>>>> > Arink now has a pull request, but it looks intrusive enough
> >>>>> > and needs enough work that I don't think we can just put it in.
> >>>>>
> >>>>> Well, we wrote so here:
> >>>>>
> >>>>>     http://projects.scipy.org/scipy/wiki/SummerofCodeIdeas
> >>>>>
> >>>>> but that's maybe just a mistake -- PSF states exactly the opposite:
> >>>>>
> >>>>>     http://wiki.python.org/moin/SummerOfCode/ApplicationTemplate2013
> >>>>
> >>>>
> >>>> It wasn't a mistake - the part of a PR process that is most
> interesting
> >>>> in the context of evaluating GSoC applications is the dialogue and
> how the
> >>>> submitter deals with feedback.
> >>>>
> >>>> I forgot to add on that page (although I think it was in one of my
> >>>> emails) that the patch shouldn't be completely trivial - fixing a typo
> >>>> doesn't really tell us all that much. But in this case Chuck's
> suggestion on
> >>>> the PR of how to get something merged looks fine.
> >>>>
> >>>
> >>> My feeling is that learning to work with the community is part of the
> >>> process after acceptance and one of the reasons there are mentors. You
> might
> >>> get some bad choices skipping the submission/acceptance bit, but you
> might
> >>> also close the door on people who are new to the whole thing. Ideally,
> the
> >>> applicants would already have involved themselves with the community,
> >>> practically that may often not be ths case.
> >>
> >>
> >> You may be right in all of that, but since there's a good chance that
> >> there are more applicants than slots I'd rather not make those bad
> choices
> >> if they're acceptable.
> >
> >
> > acceptable --> avoidable
> >
> >>
> >> Right now we have three solid proposals, from Arink, Blake and Surya. If
> >> we're lucky we'll get three slots, but if not then we'll have a tough
> choice
> >> to make.
> >>
> >> The application deadline is tomorrow, so now is the time for final
> tweaks
> >> to the proposals. After that of course the plan can still be worked out
> >> more, but it can't be edited on Melange anymore.
>
> Terri can still make it editable on Melange if necessary.
>
>
Arink, you still have work to do for a PR.

Chuck.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20130503/ac6a0d0e/attachment.html>

From arinkverma at iitrpr.ac.in  Fri May  3 13:59:46 2013
From: arinkverma at iitrpr.ac.in (Arink Verma)
Date: Fri, 3 May 2013 23:29:46 +0530
Subject: [Numpy-discussion] GSoC : Performance parity between numpy
 arrays and Python scalars
In-Reply-To: <CAB6mnxLf26ZT5qFGrTgdC36eBR2N1DoXGzUXkkMca29uOxdQaw@mail.gmail.com>
References: <CAJAs4rQ4qQmo4sy5NM=CU9XWSrShOxBn0ti9f-Z4NiZhhehewQ@mail.gmail.com>
	<51816167.6040607@virtualmaterials.com>
	<CAJAs4rRTpvMCFURQy5F5yTC4PqPvyF0TStSMVQ3xG+MAFMGLEw@mail.gmail.com>
	<CAGY4rcUMYSS+9zVXU46CXyw--_TbsUjMOA9TEzgBFbiD_UziMQ@mail.gmail.com>
	<CAJAs4rRhAs4bwu+x3b-fQDi+8eMUuRnZ-C66SmZE6HcHFrY2AQ@mail.gmail.com>
	<CAPJVwB=0E6AaSmN+x7OckkfzuCVTvKuKvV=0Ts2r9Fny5inWvg@mail.gmail.com>
	<CAB6mnxJAJxMzVt3r+dBXf_gAKzteuVzJm0eAkjb=7B0o9MDyRw@mail.gmail.com>
	<loom.20130502T183915-278@post.gmane.org>
	<CABL7CQhxHqueOyzWMtE_VL6J-_XEkuUD4SvxmHq6tLBWE6Ek=Q@mail.gmail.com>
	<CAB6mnx+tY2ibkQQ=ZsXYJdMnZX1B7g2zA=3QDprXyTkP_NLoKg@mail.gmail.com>
	<CABL7CQhG1w8fzcNf1Pw1Dzg5ZMA9yDr4xgQL0=0RbaRfC1jEbw@mail.gmail.com>
	<CABL7CQi_qbZRaoyjhfhBR_=Eix=hwSJcCmqYzMAzQ7+MWB4hQA@mail.gmail.com>
	<CAMMTP+Du-qvpTtBZLkveTGe1BDy_is6rx2TgfhzHFgGTF6Lg-g@mail.gmail.com>
	<CAB6mnxLf26ZT5qFGrTgdC36eBR2N1DoXGzUXkkMca29uOxdQaw@mail.gmail.com>
Message-ID: <CAJAs4rTHFWp8BEtkKKw0e4kg-Z1XoLbGSub9iEWduMsPEmCx2g@mail.gmail.com>

I hardly found, any thing to improve and correct.. not even typo in docs?
Where we need to avoid the version checks?



On Fri, May 3, 2013 at 10:52 PM, Charles R Harris <charlesr.harris at gmail.com
> wrote:

>
>
> On Thu, May 2, 2013 at 6:47 PM, <josef.pktd at gmail.com> wrote:
>
>> On Thu, May 2, 2013 at 6:30 PM, Ralf Gommers <ralf.gommers at gmail.com>
>> wrote:
>> >
>> >
>> >
>> > On Fri, May 3, 2013 at 12:29 AM, Ralf Gommers <ralf.gommers at gmail.com>
>> > wrote:
>> >>
>> >>
>> >>
>> >>
>> >> On Thu, May 2, 2013 at 9:54 PM, Charles R Harris
>> >> <charlesr.harris at gmail.com> wrote:
>> >>>
>> >>>
>> >>>
>> >>> On Thu, May 2, 2013 at 11:49 AM, Ralf Gommers <ralf.gommers at gmail.com
>> >
>> >>> wrote:
>> >>>>
>> >>>>
>> >>>>
>> >>>>
>> >>>> On Thu, May 2, 2013 at 6:45 PM, Pauli Virtanen <pav at iki.fi> wrote:
>> >>>>>
>> >>>>> Charles R Harris <charlesr.harris <at> gmail.com> writes:
>> >>>>> [clip]
>> >>>>> > * Did you notice this line on the requirements page? "Having your
>> >>>>> > first pull request merged before the GSoC application deadline
>> (May
>> >>>>> > 3)
>> >>>>> > is required for your application to be accepted."
>> >>>>> >
>> >>>>> > Where is that last requirement? It seems out of line to me.
>> >>>>> > Arink now has a pull request, but it looks intrusive enough
>> >>>>> > and needs enough work that I don't think we can just put it in.
>> >>>>>
>> >>>>> Well, we wrote so here:
>> >>>>>
>> >>>>>     http://projects.scipy.org/scipy/wiki/SummerofCodeIdeas
>> >>>>>
>> >>>>> but that's maybe just a mistake -- PSF states exactly the opposite:
>> >>>>>
>> >>>>>
>> http://wiki.python.org/moin/SummerOfCode/ApplicationTemplate2013
>> >>>>
>> >>>>
>> >>>> It wasn't a mistake - the part of a PR process that is most
>> interesting
>> >>>> in the context of evaluating GSoC applications is the dialogue and
>> how the
>> >>>> submitter deals with feedback.
>> >>>>
>> >>>> I forgot to add on that page (although I think it was in one of my
>> >>>> emails) that the patch shouldn't be completely trivial - fixing a
>> typo
>> >>>> doesn't really tell us all that much. But in this case Chuck's
>> suggestion on
>> >>>> the PR of how to get something merged looks fine.
>> >>>>
>> >>>
>> >>> My feeling is that learning to work with the community is part of the
>> >>> process after acceptance and one of the reasons there are mentors.
>> You might
>> >>> get some bad choices skipping the submission/acceptance bit, but you
>> might
>> >>> also close the door on people who are new to the whole thing.
>> Ideally, the
>> >>> applicants would already have involved themselves with the community,
>> >>> practically that may often not be ths case.
>> >>
>> >>
>> >> You may be right in all of that, but since there's a good chance that
>> >> there are more applicants than slots I'd rather not make those bad
>> choices
>> >> if they're acceptable.
>> >
>> >
>> > acceptable --> avoidable
>> >
>> >>
>> >> Right now we have three solid proposals, from Arink, Blake and Surya.
>> If
>> >> we're lucky we'll get three slots, but if not then we'll have a tough
>> choice
>> >> to make.
>> >>
>> >> The application deadline is tomorrow, so now is the time for final
>> tweaks
>> >> to the proposals. After that of course the plan can still be worked out
>> >> more, but it can't be edited on Melange anymore.
>>
>> Terri can still make it editable on Melange if necessary.
>>
>>
> Arink, you still have work to do for a PR.
>
> Chuck.
>
>
> _______________________________________________
> NumPy-Discussion mailing list
> NumPy-Discussion at scipy.org
> http://mail.scipy.org/mailman/listinfo/numpy-discussion
>
>


-- 
Arink
Computer Science and Engineering
Indian Institute of Technology Ropar
www.arinkverma.in
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20130503/c46630db/attachment.html>

From arinkverma at iitrpr.ac.in  Fri May  3 14:13:18 2013
From: arinkverma at iitrpr.ac.in (Arink Verma)
Date: Fri, 3 May 2013 23:43:18 +0530
Subject: [Numpy-discussion] GSoC : Performance parity between numpy
 arrays and Python scalars
In-Reply-To: <CAJAs4rTHFWp8BEtkKKw0e4kg-Z1XoLbGSub9iEWduMsPEmCx2g@mail.gmail.com>
References: <CAJAs4rQ4qQmo4sy5NM=CU9XWSrShOxBn0ti9f-Z4NiZhhehewQ@mail.gmail.com>
	<51816167.6040607@virtualmaterials.com>
	<CAJAs4rRTpvMCFURQy5F5yTC4PqPvyF0TStSMVQ3xG+MAFMGLEw@mail.gmail.com>
	<CAGY4rcUMYSS+9zVXU46CXyw--_TbsUjMOA9TEzgBFbiD_UziMQ@mail.gmail.com>
	<CAJAs4rRhAs4bwu+x3b-fQDi+8eMUuRnZ-C66SmZE6HcHFrY2AQ@mail.gmail.com>
	<CAPJVwB=0E6AaSmN+x7OckkfzuCVTvKuKvV=0Ts2r9Fny5inWvg@mail.gmail.com>
	<CAB6mnxJAJxMzVt3r+dBXf_gAKzteuVzJm0eAkjb=7B0o9MDyRw@mail.gmail.com>
	<loom.20130502T183915-278@post.gmane.org>
	<CABL7CQhxHqueOyzWMtE_VL6J-_XEkuUD4SvxmHq6tLBWE6Ek=Q@mail.gmail.com>
	<CAB6mnx+tY2ibkQQ=ZsXYJdMnZX1B7g2zA=3QDprXyTkP_NLoKg@mail.gmail.com>
	<CABL7CQhG1w8fzcNf1Pw1Dzg5ZMA9yDr4xgQL0=0RbaRfC1jEbw@mail.gmail.com>
	<CABL7CQi_qbZRaoyjhfhBR_=Eix=hwSJcCmqYzMAzQ7+MWB4hQA@mail.gmail.com>
	<CAMMTP+Du-qvpTtBZLkveTGe1BDy_is6rx2TgfhzHFgGTF6Lg-g@mail.gmail.com>
	<CAB6mnxLf26ZT5qFGrTgdC36eBR2N1DoXGzUXkkMca29uOxdQaw@mail.gmail.com>
	<CAJAs4rTHFWp8BEtkKKw0e4kg-Z1XoLbGSub9iEWduMsPEmCx2g@mail.gmail.com>
Message-ID: <CAJAs4rSw5Ox99GsDckd7_59PuGJ1V-g6FjDb9kqV1FBpQx9UVg@mail.gmail.com>

I have created a new PR, have removed one irrelevant version check.
https://github.com/numpy/numpy/pull/3304/files


On Fri, May 3, 2013 at 11:29 PM, Arink Verma <arinkverma at iitrpr.ac.in>wrote:

> I hardly found, any thing to improve and correct.. not even typo in docs?
> Where we need to avoid the version checks?
>
>
>
> On Fri, May 3, 2013 at 10:52 PM, Charles R Harris <
> charlesr.harris at gmail.com> wrote:
>
>>
>>
>> On Thu, May 2, 2013 at 6:47 PM, <josef.pktd at gmail.com> wrote:
>>
>>> On Thu, May 2, 2013 at 6:30 PM, Ralf Gommers <ralf.gommers at gmail.com>
>>> wrote:
>>> >
>>> >
>>> >
>>> > On Fri, May 3, 2013 at 12:29 AM, Ralf Gommers <ralf.gommers at gmail.com>
>>> > wrote:
>>> >>
>>> >>
>>> >>
>>> >>
>>> >> On Thu, May 2, 2013 at 9:54 PM, Charles R Harris
>>> >> <charlesr.harris at gmail.com> wrote:
>>> >>>
>>> >>>
>>> >>>
>>> >>> On Thu, May 2, 2013 at 11:49 AM, Ralf Gommers <
>>> ralf.gommers at gmail.com>
>>> >>> wrote:
>>> >>>>
>>> >>>>
>>> >>>>
>>> >>>>
>>> >>>> On Thu, May 2, 2013 at 6:45 PM, Pauli Virtanen <pav at iki.fi> wrote:
>>> >>>>>
>>> >>>>> Charles R Harris <charlesr.harris <at> gmail.com> writes:
>>> >>>>> [clip]
>>> >>>>> > * Did you notice this line on the requirements page? "Having your
>>> >>>>> > first pull request merged before the GSoC application deadline
>>> (May
>>> >>>>> > 3)
>>> >>>>> > is required for your application to be accepted."
>>> >>>>> >
>>> >>>>> > Where is that last requirement? It seems out of line to me.
>>> >>>>> > Arink now has a pull request, but it looks intrusive enough
>>> >>>>> > and needs enough work that I don't think we can just put it in.
>>> >>>>>
>>> >>>>> Well, we wrote so here:
>>> >>>>>
>>> >>>>>     http://projects.scipy.org/scipy/wiki/SummerofCodeIdeas
>>> >>>>>
>>> >>>>> but that's maybe just a mistake -- PSF states exactly the opposite:
>>> >>>>>
>>> >>>>>
>>> http://wiki.python.org/moin/SummerOfCode/ApplicationTemplate2013
>>> >>>>
>>> >>>>
>>> >>>> It wasn't a mistake - the part of a PR process that is most
>>> interesting
>>> >>>> in the context of evaluating GSoC applications is the dialogue and
>>> how the
>>> >>>> submitter deals with feedback.
>>> >>>>
>>> >>>> I forgot to add on that page (although I think it was in one of my
>>> >>>> emails) that the patch shouldn't be completely trivial - fixing a
>>> typo
>>> >>>> doesn't really tell us all that much. But in this case Chuck's
>>> suggestion on
>>> >>>> the PR of how to get something merged looks fine.
>>> >>>>
>>> >>>
>>> >>> My feeling is that learning to work with the community is part of the
>>> >>> process after acceptance and one of the reasons there are mentors.
>>> You might
>>> >>> get some bad choices skipping the submission/acceptance bit, but you
>>> might
>>> >>> also close the door on people who are new to the whole thing.
>>> Ideally, the
>>> >>> applicants would already have involved themselves with the community,
>>> >>> practically that may often not be ths case.
>>> >>
>>> >>
>>> >> You may be right in all of that, but since there's a good chance that
>>> >> there are more applicants than slots I'd rather not make those bad
>>> choices
>>> >> if they're acceptable.
>>> >
>>> >
>>> > acceptable --> avoidable
>>> >
>>> >>
>>> >> Right now we have three solid proposals, from Arink, Blake and Surya.
>>> If
>>> >> we're lucky we'll get three slots, but if not then we'll have a tough
>>> choice
>>> >> to make.
>>> >>
>>> >> The application deadline is tomorrow, so now is the time for final
>>> tweaks
>>> >> to the proposals. After that of course the plan can still be worked
>>> out
>>> >> more, but it can't be edited on Melange anymore.
>>>
>>> Terri can still make it editable on Melange if necessary.
>>>
>>>
>> Arink, you still have work to do for a PR.
>>
>> Chuck.
>>
>>
>> _______________________________________________
>> NumPy-Discussion mailing list
>> NumPy-Discussion at scipy.org
>> http://mail.scipy.org/mailman/listinfo/numpy-discussion
>>
>>
>
>
> --
> Arink
> Computer Science and Engineering
> Indian Institute of Technology Ropar
> www.arinkverma.in
>



-- 
Arink
Computer Science and Engineering
Indian Institute of Technology Ropar
www.arinkverma.in
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20130503/79bb655e/attachment.html>

From charlesr.harris at gmail.com  Fri May  3 14:46:06 2013
From: charlesr.harris at gmail.com (Charles R Harris)
Date: Fri, 3 May 2013 12:46:06 -0600
Subject: [Numpy-discussion] GSoC : Performance parity between numpy
 arrays and Python scalars
In-Reply-To: <CAJAs4rSw5Ox99GsDckd7_59PuGJ1V-g6FjDb9kqV1FBpQx9UVg@mail.gmail.com>
References: <CAJAs4rQ4qQmo4sy5NM=CU9XWSrShOxBn0ti9f-Z4NiZhhehewQ@mail.gmail.com>
	<51816167.6040607@virtualmaterials.com>
	<CAJAs4rRTpvMCFURQy5F5yTC4PqPvyF0TStSMVQ3xG+MAFMGLEw@mail.gmail.com>
	<CAGY4rcUMYSS+9zVXU46CXyw--_TbsUjMOA9TEzgBFbiD_UziMQ@mail.gmail.com>
	<CAJAs4rRhAs4bwu+x3b-fQDi+8eMUuRnZ-C66SmZE6HcHFrY2AQ@mail.gmail.com>
	<CAPJVwB=0E6AaSmN+x7OckkfzuCVTvKuKvV=0Ts2r9Fny5inWvg@mail.gmail.com>
	<CAB6mnxJAJxMzVt3r+dBXf_gAKzteuVzJm0eAkjb=7B0o9MDyRw@mail.gmail.com>
	<loom.20130502T183915-278@post.gmane.org>
	<CABL7CQhxHqueOyzWMtE_VL6J-_XEkuUD4SvxmHq6tLBWE6Ek=Q@mail.gmail.com>
	<CAB6mnx+tY2ibkQQ=ZsXYJdMnZX1B7g2zA=3QDprXyTkP_NLoKg@mail.gmail.com>
	<CABL7CQhG1w8fzcNf1Pw1Dzg5ZMA9yDr4xgQL0=0RbaRfC1jEbw@mail.gmail.com>
	<CABL7CQi_qbZRaoyjhfhBR_=Eix=hwSJcCmqYzMAzQ7+MWB4hQA@mail.gmail.com>
	<CAMMTP+Du-qvpTtBZLkveTGe1BDy_is6rx2TgfhzHFgGTF6Lg-g@mail.gmail.com>
	<CAB6mnxLf26ZT5qFGrTgdC36eBR2N1DoXGzUXkkMca29uOxdQaw@mail.gmail.com>
	<CAJAs4rTHFWp8BEtkKKw0e4kg-Z1XoLbGSub9iEWduMsPEmCx2g@mail.gmail.com>
	<CAJAs4rSw5Ox99GsDckd7_59PuGJ1V-g6FjDb9kqV1FBpQx9UVg@mail.gmail.com>
Message-ID: <CAB6mnxLfT7i46eKr4E=cABofeu0a2MSgEzs9Yk6vZmqu8WQfbA@mail.gmail.com>

On Fri, May 3, 2013 at 12:13 PM, Arink Verma <arinkverma at iitrpr.ac.in>wrote:

> I have created a new PR, have removed one irrelevant version check.
> https://github.com/numpy/numpy/pull/3304/files
>

I made some remarks on the PR.

The convention on numpy-discussion is bottom posting so you should do that
to avoid future complaints.

<snip>

Chuck
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20130503/ba4899e6/attachment.html>

From pierre.raybaut at gmail.com  Fri May  3 15:48:25 2013
From: pierre.raybaut at gmail.com (Pierre Raybaut)
Date: Fri, 3 May 2013 21:48:25 +0200
Subject: [Numpy-discussion] ANN: New WinPython with Python 2.7.4 and 3.3.1
	(32/64bit)
Message-ID: <CAKegKuA7JD8dwrpNDr1QL8Hhm2iozjBNqNqW1TRR7fNXd+7nQQ@mail.gmail.com>

Hi all,

I am pleased to announce that four new versions of WinPython have been
released yesterday with Python 2.7.4 and 3.3.1, 32 and 64 bits. Many
packages have been added or upgraded (see the automatically-generated
changelogs).

Special thanks to Christoph Gohlke for building most of the binary packages
bundled in WinPython.

WinPython is a free open-source portable distribution of Python
for Windows, designed for scientists.

It is a full-featured (see
http://code.google.com/p/winpython/wiki/PackageIndex)
Python-based scientific environment:
  * Designed for scientists (thanks to the integrated libraries
NumPy, SciPy, Matplotlib, guiqwt, etc.:
    * Regular *scientific users*: interactive data processing
and visualization using Python with Spyder
    * *Advanced scientific users and software developers*:
Python applications development with Spyder, version control with
Mercurial and other development tools (like gettext)
  * *Portable*: preconfigured, it should run out of the box on any machine
under Windows (without any installation requirements) and the folder
containing WinPython can be moved to any location (local, network or
removable drive)
  * *Flexible*: one can install (or should I write "use" as it's portable)
as many WinPython versions as necessary (like isolated and self-consistent
environments), even if those versions are running different versions of
Python (2.7, 3.3) or different architectures (32bit or 64bit) on the same
machine
  * *Customizable*: using the integrated package manager (wppm,
as WinPython Package Manager), it's possible to install, uninstall
or upgrade Python packages (see
http://code.google.com/p/winpython/wiki/WPPM for
more details on supported package formats).

*WinPython is not an attempt to replace Python(x,y)*, this is
just something different (see
http://code.google.com/p/winpython/wiki/Roadmap): more flexible, easier to
maintain, movable and less invasive for the OS, but certainly less
user-friendly, with less packages/contents and without any integration to
Windows explorer [*].

[*] Actually there is an optional integration into Windows
explorer, providing the same features as the official Python installer
regarding file associations and context menu entry (this option may be
activated through the WinPython Control Panel), and adding shortcuts to
Windows Start menu.

Enjoy!
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20130503/20e3fc3a/attachment.html>

From pav at iki.fi  Fri May  3 19:43:32 2013
From: pav at iki.fi (Pauli Virtanen)
Date: Sat, 04 May 2013 02:43:32 +0300
Subject: [Numpy-discussion] numpy.scipy.org page 404s
In-Reply-To: <CAF6FJiu1z+iWvtQLVV2pcc2676iQkhdf_tvtM3K8m+LgtRE=HA@mail.gmail.com>
References: <CADeni3uq_w3rnERu39NDNrsDE31CO_49YobmqKGpfrRE4nNE6Q@mail.gmail.com>
	<CAF6FJiujKNHS09EkqdaMN1guo-kB=FTFAOwk7upxGUqUXVyD2g@mail.gmail.com>
	<517A9D98.70003@hilboll.de>
	<CAF6FJiszCiTNOUUFft1zVV7pv8_CkEPv-P+KFsUfe8OCJ9x=wQ@mail.gmail.com>
	<517AA0CF.3000801@hilboll.de>
	<CAF6FJiu1z+iWvtQLVV2pcc2676iQkhdf_tvtM3K8m+LgtRE=HA@mail.gmail.com>
Message-ID: <km1i30$5v9$1@ger.gmane.org>

26.04.2013 18:51, Robert Kern kirjoitti:
[clip]
> Right now, the recurring cost is kicking the www.scipy.org wiki every
> once in a while under the deluge of spam.

It's dying a slow death again.

How about restricting editing pages to people in EditorGroup? There
should be an option in Moin config for that. I don't think there is any
other solution to the spam with the Moin wiki at present.

-- 
Pauli Virtanen



From fpacchiani at hotmail.com  Sat May  4 01:05:29 2013
From: fpacchiani at hotmail.com (Francesco Pacchiani)
Date: Sat, 4 May 2013 07:05:29 +0200
Subject: [Numpy-discussion] Error installing numpy 1.7.1
Message-ID: <DUB120-W22A64C52FC49C939F5E131DFBF0@phx.gbl>

Hello,


I want to install numpy 1.7.1 for use with python 3.2.1 and have
 been blocked by an error during the build process. My OS is Mandriva 
2001. I'm installing numpy from source because python2.7 is already 
installed and seems to be the default, the rpm's don't install 
anything in python3.2/site-packages.

The command used is "python3 setup.py build" and below I have posted
 the end of the outptut which contains the error message. I understand 
it is the compilation of _capi.c that posses problems, but have not been
 able to solve it. The fenv.h file is installed in different directories


/usr/include

/usr/local/src/numpy-1.7.1/numpy/core/include/numpy/fenv/fenv.h and 

/usr/local/src/numpy-1.7.1/build/py3k/numpy/core/include/numpy/fenv/fenv.h


Using strace, one realizes that there is a call for 
"build/lib.linux-x86_64-3.2/numpy/numarray/_capi.cpython-32mu.so" that 
generates a "No such file or directory". The end of the output of 
"strace python3 setup.py build"  command is posted below as well. Indeed
 no such library exists, should it?


Does anybody understand the problem or have a hint to give. Thanks.

Francesco



Output after executing:  python3 setup.py build


creating build/temp.linux-x86_64-3.2/numpy/lib

creating build/temp.linux-x86_64-3.2/numpy/lib/src

compile options: '-Inumpy/core/include 
-Ibuild/src.linux-x86_64-3.2/numpy/core/include/numpy 
-Inumpy/core/src/private -Inumpy/core/src -Inumpy/core 
-Inumpy/core/src/npymath -Inumpy/core/src/multiarray 
-Inumpy/core/src/umath -Inumpy/core/src/npysort -Inumpy/core/include 
-I/usr/include/python3.2mu 
-Ibuild/src.linux-x86_64-3.2/numpy/core/src/multiarray 
-Ibuild/src.linux-x86_64-3.2/numpy/core/src/umath -c'

gcc: numpy/lib/src/_compiled_base.c

gcc -pthread -shared -Wl,--as-needed -Wl,--no-undefined -Wl,-z,relro
 -Wl,-O1 -Wl,--build-id -Wl,--enable-new-dtags 
build/temp.linux-x86_64-3.2/numpy/lib/src/_compiled_base.o -L/usr/lib64 
-Lbuild/temp.linux-x86_64-3.2 -lpython3.2mu -o 
build/lib.linux-x86_64-3.2/numpy/lib/_compiled_base.cpython-32mu.so

building 'numpy.numarray._capi' extension

compiling C sources

C compiler: gcc -pthread -DDYNAMIC_ANNOTATIONS_ENABLED=1 -DNDEBUG 
-O2 -g -frecord-gcc-switches -Wstrict-aliasing=2 -pipe -Wformat 
-Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -fstack-protector 
--param=ssp-buffer-size=4 -fPIC -g -O2 -g -frecord-gcc-switches 
-Wstrict-aliasing=2 -pipe -Wformat -Werror=format-security 
-Wp,-D_FORTIFY_SOURCE=2 -fstack-protector --param=ssp-buffer-size=4 
-fPIC -fPIC


creating build/temp.linux-x86_64-3.2/numpy/numarray

compile options: '-Inumpy/core/include 
-Ibuild/src.linux-x86_64-3.2/numpy/core/include/numpy 
-Inumpy/core/src/private -Inumpy/core/src -Inumpy/core 
-Inumpy/core/src/npymath -Inumpy/core/src/multiarray 
-Inumpy/core/src/umath -Inumpy/core/src/npysort -Inumpy/core/include 
-I/usr/include/python3.2mu 
-Ibuild/src.linux-x86_64-3.2/numpy/core/src/multiarray 
-Ibuild/src.linux-x86_64-3.2/numpy/core/src/umath -c'

gcc: numpy/numarray/_capi.c

numpy/numarray/_capi.c: In function ?deferred_libnumarray_init?:

numpy/numarray/_capi.c:31:5: warning: dereferencing type-punned pointer might break strict-aliasing rules [-Wstrict-aliasing]

numpy/numarray/_capi.c: In function ?NA_callCUFuncCore?:

numpy/numarray/_capi.c:579:5: warning: dereferencing type-punned pointer might break strict-aliasing rules [-Wstrict-aliasing]

numpy/numarray/_capi.c: In function ?NA_new_cfunc?:

numpy/numarray/_capi.c:1018:5: warning: dereferencing type-punned pointer might break strict-aliasing rules [-Wstrict-aliasing]

numpy/numarray/_capi.c: In function ?NA_OutputArray?:

numpy/numarray/_capi.c:1104:5: warning: passing argument 2 of ?(int 
(*)(struct PyArrayObject *, struct PyArrayObject *))*(PyArray_API + 
2296u)? from incompatible pointer type [enabled by default]

numpy/numarray/_capi.c:1104:5: note: expected ?struct PyArrayObject *? but argument is of type ?struct PyObject *?

numpy/numarray/_capi.c: In function ?NA_callStrideConvCFuncCore?:

numpy/numarray/_capi.c:2644:5: warning: dereferencing type-punned pointer might break strict-aliasing rules [-Wstrict-aliasing]

numpy/numarray/_capi.c: In function ?NA_IeeeMask32?:

numpy/numarray/_capi.c:3073:5: warning: dereferencing type-punned pointer will break strict-aliasing rules [-Wstrict-aliasing]

numpy/numarray/_capi.c: In function ?NA_IeeeMask64?:

numpy/numarray/_capi.c:3117:5: warning: dereferencing type-punned pointer will break strict-aliasing rules [-Wstrict-aliasing]

gcc -pthread -shared -Wl,--as-needed -Wl,--no-undefined -Wl,-z,relro
 -Wl,-O1 -Wl,--build-id -Wl,--enable-new-dtags 
build/temp.linux-x86_64-3.2/numpy/numarray/_capi.o -L/usr/lib64 
-Lbuild/temp.linux-x86_64-3.2 -lpython3.2mu -o 
build/lib.linux-x86_64-3.2/numpy/numarray/_capi.cpython-32mu.so

build/temp.linux-x86_64-3.2/numpy/numarray/_capi.o: In function `NA_checkFPErrors':

/usr/local/src/numpy-1.7.1/build/py3k/numpy/numarray/_capi.c:3019: undefined reference to `fetestexcept'

/usr/local/src/numpy-1.7.1/build/py3k/numpy/numarray/_capi.c:3026: undefined reference to `feclearexcept'

build/temp.linux-x86_64-3.2/numpy/numarray/_capi.o: In function `int_overflow_error':

/usr/local/src/numpy-1.7.1/build/py3k/numpy/numarray/_capi.c:263: undefined reference to `feraiseexcept'

build/temp.linux-x86_64-3.2/numpy/numarray/_capi.o: In function `num_atanh':

/usr/local/src/numpy-1.7.1/build/py3k/numpy/numarray/_capi.c:233: undefined reference to `log'

build/temp.linux-x86_64-3.2/numpy/numarray/_capi.o: In function `num_asinh':

/usr/local/src/numpy-1.7.1/build/py3k/numpy/numarray/_capi.c:228: undefined reference to `log'

/usr/local/src/numpy-1.7.1/build/py3k/numpy/numarray/_capi.c:228: undefined reference to `sqrt'

build/temp.linux-x86_64-3.2/numpy/numarray/_capi.o: In function `num_acosh':

/usr/local/src/numpy-1.7.1/build/py3k/numpy/numarray/_capi.c:213: undefined reference to `sqrt'

build/temp.linux-x86_64-3.2/numpy/numarray/_capi.o: In function `num_log':

/usr/local/src/numpy-1.7.1/build/py3k/numpy/numarray/_capi.c:190: undefined reference to `log'

build/temp.linux-x86_64-3.2/numpy/numarray/_capi.o: In function `num_acosh':

/usr/local/src/numpy-1.7.1/build/py3k/numpy/numarray/_capi.c:213: undefined reference to `log'

build/temp.linux-x86_64-3.2/numpy/numarray/_capi.o: In function `num_pow':

/usr/local/src/numpy-1.7.1/build/py3k/numpy/numarray/_capi.c:207: undefined reference to `pow'

build/temp.linux-x86_64-3.2/numpy/numarray/_capi.o: In function `num_log10':

/usr/local/src/numpy-1.7.1/build/py3k/numpy/numarray/_capi.c:198: undefined reference to `log10'

build/temp.linux-x86_64-3.2/numpy/numarray/_capi.o: In function `num_round':

/usr/local/src/numpy-1.7.1/build/py3k/numpy/numarray/_capi.c:239: undefined reference to `ceil'

/usr/local/src/numpy-1.7.1/build/py3k/numpy/numarray/_capi.c:239: undefined reference to `floor'

collect2: ld returned 1 exit status

build/temp.linux-x86_64-3.2/numpy/numarray/_capi.o: In function `NA_checkFPErrors':

/usr/local/src/numpy-1.7.1/build/py3k/numpy/numarray/_capi.c:3019: undefined reference to `fetestexcept'

/usr/local/src/numpy-1.7.1/build/py3k/numpy/numarray/_capi.c:3026: undefined reference to `feclearexcept'

build/temp.linux-x86_64-3.2/numpy/numarray/_capi.o: In function `int_overflow_error':

/usr/local/src/numpy-1.7.1/build/py3k/numpy/numarray/_capi.c:263: undefined reference to `feraiseexcept'

build/temp.linux-x86_64-3.2/numpy/numarray/_capi.o: In function `num_atanh':

/usr/local/src/numpy-1.7.1/build/py3k/numpy/numarray/_capi.c:233: undefined reference to `log'

build/temp.linux-x86_64-3.2/numpy/numarray/_capi.o: In function `num_asinh':

/usr/local/src/numpy-1.7.1/build/py3k/numpy/numarray/_capi.c:228: undefined reference to `log'

/usr/local/src/numpy-1.7.1/build/py3k/numpy/numarray/_capi.c:228: undefined reference to `sqrt'

build/temp.linux-x86_64-3.2/numpy/numarray/_capi.o: In function `num_acosh':

/usr/local/src/numpy-1.7.1/build/py3k/numpy/numarray/_capi.c:213: undefined reference to `sqrt'

build/temp.linux-x86_64-3.2/numpy/numarray/_capi.o: In function `num_log':

/usr/local/src/numpy-1.7.1/build/py3k/numpy/numarray/_capi.c:190: undefined reference to `log'

build/temp.linux-x86_64-3.2/numpy/numarray/_capi.o: In function `num_acosh':

/usr/local/src/numpy-1.7.1/build/py3k/numpy/numarray/_capi.c:213: undefined reference to `log'

build/temp.linux-x86_64-3.2/numpy/numarray/_capi.o: In function `num_pow':

/usr/local/src/numpy-1.7.1/build/py3k/numpy/numarray/_capi.c:207: undefined reference to `pow'

build/temp.linux-x86_64-3.2/numpy/numarray/_capi.o: In function `num_log10':

/usr/local/src/numpy-1.7.1/build/py3k/numpy/numarray/_capi.c:198: undefined reference to `log10'

build/temp.linux-x86_64-3.2/numpy/numarray/_capi.o: In function `num_round':

/usr/local/src/numpy-1.7.1/build/py3k/numpy/numarray/_capi.c:239: undefined reference to `ceil'

/usr/local/src/numpy-1.7.1/build/py3k/numpy/numarray/_capi.c:239: undefined reference to `floor'

collect2: ld returned 1 exit status

error: Command "gcc -pthread -shared -Wl,--as-needed 
-Wl,--no-undefined -Wl,-z,relro -Wl,-O1 -Wl,--build-id 
-Wl,--enable-new-dtags 
build/temp.linux-x86_64-3.2/numpy/numarray/_capi.o -L/usr/lib64 
-Lbuild/temp.linux-x86_64-3.2 -lpython3.2mu -o 
build/lib.linux-x86_64-3.2/numpy/numarray/_capi.cpython-32mu.so" failed 
with exit status 1



Output of "strace python3 setup.py build"


stat("build/lib.linux-x86_64-3.2/numpy/core/umath_tests.cpython-32mu.so", {st_mode=S_IFREG|0755, st_size=54991, ...}) = 0

stat("build/lib.linux-x86_64-3.2/numpy/core/umath_tests.cpython-32mu.so", {st_mode=S_IFREG|0755, st_size=54991, ...}) = 0

stat("build/src.linux-x86_64-3.2/numpy/core/src/umath/umath_tests.c", {st_mode=S_IFREG|0644, st_size=14253, ...}) = 0

stat("build/src.linux-x86_64-3.2/numpy/core/src/umath/umath_tests.c", {st_mode=S_IFREG|0644, st_size=14253, ...}) = 0

stat("build/lib.linux-x86_64-3.2/numpy/core/multiarray_tests.cpython-32mu.so", {st_mode=S_IFREG|0755, st_size=56661, ...}) = 0

stat("build/lib.linux-x86_64-3.2/numpy/core/multiarray_tests.cpython-32mu.so", {st_mode=S_IFREG|0755, st_size=56661, ...}) = 0

stat("build/src.linux-x86_64-3.2/numpy/core/src/multiarray/multiarray_tests.c", {st_mode=S_IFREG|0644, st_size=14087, ...}) = 0

stat("build/src.linux-x86_64-3.2/numpy/core/src/multiarray/multiarray_tests.c", {st_mode=S_IFREG|0644, st_size=14087, ...}) = 0

stat("build/lib.linux-x86_64-3.2/numpy/lib/_compiled_base.cpython-32mu.so", {st_mode=S_IFREG|0755, st_size=100309, ...}) = 0

stat("build/lib.linux-x86_64-3.2/numpy/lib/_compiled_base.cpython-32mu.so", {st_mode=S_IFREG|0755, st_size=100309, ...}) = 0

stat("numpy/lib/src/_compiled_base.c", {st_mode=S_IFREG|0644, st_size=50102, ...}) = 0

stat("numpy/lib/src/_compiled_base.c", {st_mode=S_IFREG|0644, st_size=50102, ...}) = 0

stat("build/lib.linux-x86_64-3.2/numpy/numarray/_capi.cpython-32mu.so", 0x7fff78c15920) = -1 ENOENT (No such file or directory)

write(1, "\33[39mbuilding 'numpy.numarray._c"..., 51building 'numpy.numarray._capi' extension

) = 51

write(1, "\33[39mcompiling C sources\33[0m\n", 29compiling C sources

) = 29

write(1, "\33[39mC compiler: gcc -pthread -D"..., 408C compiler: gcc
 -pthread -DDYNAMIC_ANNOTATIONS_ENABLED=1 -DNDEBUG -O2 -g 
-frecord-gcc-switches -Wstrict-aliasing=2 -pipe -Wformat 
-Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -fstack-protector 
--param=ssp-buffer-size=4 -fPIC -g -O2 -g -frecord-gcc-switches 
-Wstrict-aliasing=2 -pipe -Wformat -Werror=format-security 
-Wp,-D_FORTIFY_SOURCE=2 -fstack-protector --param=ssp-buffer-size=4 
-fPIC -fPIC

) = 408

write(1, "\n", 1

)                       = 1

stat("build/temp.linux-x86_64-3.2/numpy/numarray", {st_mode=S_IFDIR|0755, st_size=4096, ...}) = 0

write(1, "\33[39mcompile options: '-Inumpy/c"..., 415compile 
options: '-Inumpy/core/include 
-Ibuild/src.linux-x86_64-3.2/numpy/core/include/numpy 
-Inumpy/core/src/private -Inumpy/core/src -Inumpy/core 
-Inumpy/core/src/npymath -Inumpy/core/src/multiarray 
-Inumpy/core/src/umath -Inumpy/core/src/npysort -Inumpy/core/include 
-I/usr/include/python3.2mu 
-Ibuild/src.linux-x86_64-3.2/numpy/core/src/multiarray 
-Ibuild/src.linux-x86_64-3.2/numpy/core/src/umath -c'

) = 415

write(1, "\33[39mgcc: numpy/numarray/_capi.c"..., 37gcc: numpy/numarray/_capi.c

) = 37

getcwd("/usr/local/src/numpy-1.7.1/build/py3k", 1026) = 38

getcwd("/usr/local/src/numpy-1.7.1/build/py3k", 1026) = 38

open("/tmp/tmpicyzqn/npoyom", O_RDWR|O_CREAT|O_EXCL|O_NOFOLLOW, 0600) = 3

fcntl(3, F_GETFD)                       = 0

fcntl(3, F_SETFD, FD_CLOEXEC)           = 0

fstat(3, {st_mode=S_IFREG|0600, st_size=0, ...}) = 0

ioctl(3, SNDCTL_TMR_TIMEBASE or SNDRV_TIMER_IOCTL_NEXT_DEVICE or 
TCGETS, 0x7fff78c13e48) = -1 ENOTTY (Inappropriate ioctl for device)

fstat(3, {st_mode=S_IFREG|0600, st_size=0, ...}) = 0

lseek(3, 0, SEEK_CUR)                   = 0

ioctl(3, SNDCTL_TMR_TIMEBASE or SNDRV_TIMER_IOCTL_NEXT_DEVICE or 
TCGETS, 0x7fff78c13bf8) = -1 ENOTTY (Inappropriate ioctl for device)

lseek(3, 0, SEEK_CUR)                   = 0

lseek(3, 0, SEEK_CUR)                   = 0

close(3)                                = 0

open("/tmp/tmpicyzqn/w8jsoc", O_RDWR|O_CREAT|O_EXCL|O_NOFOLLOW, 0600) = 3

fcntl(3, F_GETFD)                       = 0

fcntl(3, F_SETFD, FD_CLOEXEC)           = 0

fstat(3, {st_mode=S_IFREG|0600, st_size=0, ...}) = 0

ioctl(3, SNDCTL_TMR_TIMEBASE or SNDRV_TIMER_IOCTL_NEXT_DEVICE or 
TCGETS, 0x7fff78c13e48) = -1 ENOTTY (Inappropriate ioctl for device)

fstat(3, {st_mode=S_IFREG|0600, st_size=0, ...}) = 0

lseek(3, 0, SEEK_CUR)                   = 0

ioctl(3, SNDCTL_TMR_TIMEBASE or SNDRV_TIMER_IOCTL_NEXT_DEVICE or 
TCGETS, 0x7fff78c13bf8) = -1 ENOTTY (Inappropriate ioctl for device)

lseek(3, 0, SEEK_CUR)                   = 0

lseek(3, 0, SEEK_CUR)                   = 0

close(3)                                = 0

rt_sigaction(SIGINT, {SIG_IGN, [], SA_RESTORER, 0x7f6f35d55250}, {0x7f6f363dfe70, [], SA_RESTORER, 0x7f6f35d55250}, 8) = 0

rt_sigaction(SIGQUIT, {SIG_IGN, [], SA_RESTORER, 0x7f6f35d55250}, {SIG_DFL, [], SA_RESTORER, 0x7f6f35d55250}, 8) = 0

rt_sigprocmask(SIG_BLOCK, [CHLD], [], 8) = 0

clone(child_stack=0, flags=CLONE_PARENT_SETTID|SIGCHLD, parent_tidptr=0x7fff78c1475c) = 24562

wait4(24562, numpy/numarray/_capi.c: In function ?deferred_libnumarray_init?:

numpy/numarray/_capi.c:31:5: warning: dereferencing type-punned pointer might break strict-aliasing rules [-Wstrict-aliasing]

numpy/numarray/_capi.c: In function ?NA_callCUFuncCore?:

numpy/numarray/_capi.c:579:5: warning: dereferencing type-punned pointer might break strict-aliasing rules [-Wstrict-aliasing]

numpy/numarray/_capi.c: In function ?NA_new_cfunc?:

numpy/numarray/_capi.c:1018:5: warning: dereferencing type-punned pointer might break strict-aliasing rules [-Wstrict-aliasing]

numpy/numarray/_capi.c: In function ?NA_OutputArray?:

numpy/numarray/_capi.c:1104:5: warning: passing argument 2 of ?(int 
(*)(struct PyArrayObject *, struct PyArrayObject *))*(PyArray_API + 
2296u)? from incompatible pointer type [enabled by default]

numpy/numarray/_capi.c:1104:5: note: expected ?struct PyArrayObject *? but argument is of type ?struct PyObject *?

numpy/numarray/_capi.c: In function ?NA_callStrideConvCFuncCore?:

numpy/numarray/_capi.c:2644:5: warning: dereferencing type-punned pointer might break strict-aliasing rules [-Wstrict-aliasing]

numpy/numarray/_capi.c: In function ?NA_IeeeMask32?:

numpy/numarray/_capi.c:3073:5: warning: dereferencing type-punned pointer will break strict-aliasing rules [-Wstrict-aliasing]

numpy/numarray/_capi.c: In function ?NA_IeeeMask64?:

numpy/numarray/_capi.c:3117:5: warning: dereferencing type-punned pointer will break strict-aliasing rules [-Wstrict-aliasing]

[{WIFEXITED(s) && WEXITSTATUS(s) == 0}], 0, NULL) = 24562

rt_sigaction(SIGINT, {0x7f6f363dfe70, [], SA_RESTORER, 0x7f6f35d55250}, NULL, 8) = 0

rt_sigaction(SIGQUIT, {SIG_DFL, [], SA_RESTORER, 0x7f6f35d55250}, NULL, 8) = 0

rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0

--- {si_signo=SIGCHLD, si_code=CLD_EXITED, si_pid=24562, si_status=0, si_utime=0, si_stime=0} (Child exited) ---

open("/tmp/tmpicyzqn/w8jsoc", O_RDONLY) = 3

fstat(3, {st_mode=S_IFREG|0600, st_size=2, ...}) = 0

ioctl(3, SNDCTL_TMR_TIMEBASE or SNDRV_TIMER_IOCTL_NEXT_DEVICE or 
TCGETS, 0x7fff78c141c8) = -1 ENOTTY (Inappropriate ioctl for device)

fstat(3, {st_mode=S_IFREG|0600, st_size=2, ...}) = 0

lseek(3, 0, SEEK_CUR)                   = 0

lseek(3, 0, SEEK_CUR)                   = 0

fstat(3, {st_mode=S_IFREG|0600, st_size=2, ...}) = 0

lseek(3, 0, SEEK_CUR)                   = 0

read(3, "0\n", 8192)                    = 2

fstat(3, {st_mode=S_IFREG|0600, st_size=2, ...}) = 0

lseek(3, 0, SEEK_CUR)                   = 2

read(3, "", 8192)                       = 0

fstat(3, {st_mode=S_IFREG|0600, st_size=2, ...}) = 0

lseek(3, 0, SEEK_CUR)                   = 2

read(3, "", 8192)                       = 0

close(3)                                = 0

unlink("/tmp/tmpicyzqn/w8jsoc")         = 0

open("/tmp/tmpicyzqn/npoyom", O_RDONLY) = 3

fstat(3, {st_mode=S_IFREG|0600, st_size=1514, ...}) = 0

ioctl(3, SNDCTL_TMR_TIMEBASE or SNDRV_TIMER_IOCTL_NEXT_DEVICE or 
TCGETS, 0x7fff78c141c8) = -1 ENOTTY (Inappropriate ioctl for device)

fstat(3, {st_mode=S_IFREG|0600, st_size=1514, ...}) = 0

lseek(3, 0, SEEK_CUR)                   = 0

lseek(3, 0, SEEK_CUR)                   = 0

fstat(3, {st_mode=S_IFREG|0600, st_size=1514, ...}) = 0

lseek(3, 0, SEEK_CUR)                   = 0

read(3, "numpy/numarray/_capi.c: In funct"..., 8192) = 1514

fstat(3, {st_mode=S_IFREG|0600, st_size=1514, ...}) = 0

lseek(3, 0, SEEK_CUR)                   = 1514

read(3, "", 8192)                       = 0

fstat(3, {st_mode=S_IFREG|0600, st_size=1514, ...}) = 0

lseek(3, 0, SEEK_CUR)                   = 1514

read(3, "", 8192)                       = 0

close(3)                                = 0

unlink("/tmp/tmpicyzqn/npoyom")         = 0

stat("build/lib.linux-x86_64-3.2/numpy/numarray/_capi.cpython-32mu.so", 0x7fff78c15290) = -1 ENOENT (No such file or directory)

stat("build/lib.linux-x86_64-3.2/numpy/numarray", {st_mode=S_IFDIR|0755, st_size=4096, ...}) = 0

write(1, "\33[39mgcc -pthread -shared -Wl,--"..., 299gcc -pthread 
-shared -Wl,--as-needed -Wl,--no-undefined -Wl,-z,relro -Wl,-O1 
-Wl,--build-id -Wl,--enable-new-dtags 
build/temp.linux-x86_64-3.2/numpy/numarray/_capi.o -L/usr/lib64 
-Lbuild/temp.linux-x86_64-3.2 -lpython3.2mu -o 
build/lib.linux-x86_64-3.2/numpy/numarray/_capi.cpython-32mu.so

) = 299

getcwd("/usr/local/src/numpy-1.7.1/build/py3k", 1026) = 38

getcwd("/usr/local/src/numpy-1.7.1/build/py3k", 1026) = 38

open("/tmp/tmpicyzqn/pur_c5", O_RDWR|O_CREAT|O_EXCL|O_NOFOLLOW, 0600) = 3

fcntl(3, F_GETFD)                       = 0

fcntl(3, F_SETFD, FD_CLOEXEC)           = 0

fstat(3, {st_mode=S_IFREG|0600, st_size=0, ...}) = 0

ioctl(3, SNDCTL_TMR_TIMEBASE or SNDRV_TIMER_IOCTL_NEXT_DEVICE or 
TCGETS, 0x7fff78c14468) = -1 ENOTTY (Inappropriate ioctl for device)

fstat(3, {st_mode=S_IFREG|0600, st_size=0, ...}) = 0

lseek(3, 0, SEEK_CUR)                   = 0

ioctl(3, SNDCTL_TMR_TIMEBASE or SNDRV_TIMER_IOCTL_NEXT_DEVICE or 
TCGETS, 0x7fff78c14218) = -1 ENOTTY (Inappropriate ioctl for device)

lseek(3, 0, SEEK_CUR)                   = 0

lseek(3, 0, SEEK_CUR)                   = 0

close(3)                                = 0

open("/tmp/tmpicyzqn/vscch3", O_RDWR|O_CREAT|O_EXCL|O_NOFOLLOW, 0600) = 3

fcntl(3, F_GETFD)                       = 0

fcntl(3, F_SETFD, FD_CLOEXEC)           = 0

fstat(3, {st_mode=S_IFREG|0600, st_size=0, ...}) = 0

ioctl(3, SNDCTL_TMR_TIMEBASE or SNDRV_TIMER_IOCTL_NEXT_DEVICE or 
TCGETS, 0x7fff78c14468) = -1 ENOTTY (Inappropriate ioctl for device)

fstat(3, {st_mode=S_IFREG|0600, st_size=0, ...}) = 0

lseek(3, 0, SEEK_CUR)                   = 0

ioctl(3, SNDCTL_TMR_TIMEBASE or SNDRV_TIMER_IOCTL_NEXT_DEVICE or 
TCGETS, 0x7fff78c14218) = -1 ENOTTY (Inappropriate ioctl for device)

lseek(3, 0, SEEK_CUR)                   = 0

lseek(3, 0, SEEK_CUR)                   = 0

close(3)                                = 0

rt_sigaction(SIGINT, {SIG_IGN, [], SA_RESTORER, 0x7f6f35d55250}, {0x7f6f363dfe70, [], SA_RESTORER, 0x7f6f35d55250}, 8) = 0

rt_sigaction(SIGQUIT, {SIG_IGN, [], SA_RESTORER, 0x7f6f35d55250}, {SIG_DFL, [], SA_RESTORER, 0x7f6f35d55250}, 8) = 0

rt_sigprocmask(SIG_BLOCK, [CHLD], [], 8) = 0

clone(child_stack=0, flags=CLONE_PARENT_SETTID|SIGCHLD, parent_tidptr=0x7fff78c14d7c) = 24570

wait4(24570, build/temp.linux-x86_64-3.2/numpy/numarray/_capi.o: In function `NA_checkFPErrors':

/usr/local/src/numpy-1.7.1/build/py3k/numpy/numarray/_capi.c:3019: undefined reference to `fetestexcept'

/usr/local/src/numpy-1.7.1/build/py3k/numpy/numarray/_capi.c:3026: undefined reference to `feclearexcept'

build/temp.linux-x86_64-3.2/numpy/numarray/_capi.o: In function `int_overflow_error':

/usr/local/src/numpy-1.7.1/build/py3k/numpy/numarray/_capi.c:263: undefined reference to `feraiseexcept'

build/temp.linux-x86_64-3.2/numpy/numarray/_capi.o: In function `num_atanh':

/usr/local/src/numpy-1.7.1/build/py3k/numpy/numarray/_capi.c:233: undefined reference to `log'

build/temp.linux-x86_64-3.2/numpy/numarray/_capi.o: In function `num_asinh':

/usr/local/src/numpy-1.7.1/build/py3k/numpy/numarray/_capi.c:228: undefined reference to `log'

/usr/local/src/numpy-1.7.1/build/py3k/numpy/numarray/_capi.c:228: undefined reference to `sqrt'

build/temp.linux-x86_64-3.2/numpy/numarray/_capi.o: In function `num_acosh':

/usr/local/src/numpy-1.7.1/build/py3k/numpy/numarray/_capi.c:213: undefined reference to `sqrt'

build/temp.linux-x86_64-3.2/numpy/numarray/_capi.o: In function `num_log':

/usr/local/src/numpy-1.7.1/build/py3k/numpy/numarray/_capi.c:190: undefined reference to `log'

build/temp.linux-x86_64-3.2/numpy/numarray/_capi.o: In function `num_acosh':

/usr/local/src/numpy-1.7.1/build/py3k/numpy/numarray/_capi.c:213: undefined reference to `log'

build/temp.linux-x86_64-3.2/numpy/numarray/_capi.o: In function `num_pow':

/usr/local/src/numpy-1.7.1/build/py3k/numpy/numarray/_capi.c:207: undefined reference to `pow'

build/temp.linux-x86_64-3.2/numpy/numarray/_capi.o: In function `num_log10':

/usr/local/src/numpy-1.7.1/build/py3k/numpy/numarray/_capi.c:198: undefined reference to `log10'

build/temp.linux-x86_64-3.2/numpy/numarray/_capi.o: In function `num_round':

/usr/local/src/numpy-1.7.1/build/py3k/numpy/numarray/_capi.c:239: undefined reference to `ceil'

/usr/local/src/numpy-1.7.1/build/py3k/numpy/numarray/_capi.c:239: undefined reference to `floor'

collect2: ld returned 1 exit status

[{WIFEXITED(s) && WEXITSTATUS(s) == 0}], 0, NULL) = 24570

rt_sigaction(SIGINT, {0x7f6f363dfe70, [], SA_RESTORER, 0x7f6f35d55250}, NULL, 8) = 0

rt_sigaction(SIGQUIT, {SIG_DFL, [], SA_RESTORER, 0x7f6f35d55250}, NULL, 8) = 0

rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0

--- {si_signo=SIGCHLD, si_code=CLD_EXITED, si_pid=24570, si_status=0, si_utime=0, si_stime=0} (Child exited) ---

open("/tmp/tmpicyzqn/vscch3", O_RDONLY) = 3

fstat(3, {st_mode=S_IFREG|0600, st_size=2, ...}) = 0

ioctl(3, SNDCTL_TMR_TIMEBASE or SNDRV_TIMER_IOCTL_NEXT_DEVICE or 
TCGETS, 0x7fff78c147e8) = -1 ENOTTY (Inappropriate ioctl for device)

fstat(3, {st_mode=S_IFREG|0600, st_size=2, ...}) = 0

lseek(3, 0, SEEK_CUR)                   = 0

lseek(3, 0, SEEK_CUR)                   = 0

fstat(3, {st_mode=S_IFREG|0600, st_size=2, ...}) = 0

lseek(3, 0, SEEK_CUR)                   = 0

read(3, "1\n", 8192)                    = 2

fstat(3, {st_mode=S_IFREG|0600, st_size=2, ...}) = 0

lseek(3, 0, SEEK_CUR)                   = 2

read(3, "", 8192)                       = 0

fstat(3, {st_mode=S_IFREG|0600, st_size=2, ...}) = 0

lseek(3, 0, SEEK_CUR)                   = 2

read(3, "", 8192)                       = 0

close(3)                                = 0

unlink("/tmp/tmpicyzqn/vscch3")         = 0

open("/tmp/tmpicyzqn/pur_c5", O_RDONLY) = 3

fstat(3, {st_mode=S_IFREG|0600, st_size=2091, ...}) = 0

ioctl(3, SNDCTL_TMR_TIMEBASE or SNDRV_TIMER_IOCTL_NEXT_DEVICE or 
TCGETS, 0x7fff78c147e8) = -1 ENOTTY (Inappropriate ioctl for device)

fstat(3, {st_mode=S_IFREG|0600, st_size=2091, ...}) = 0

lseek(3, 0, SEEK_CUR)                   = 0

lseek(3, 0, SEEK_CUR)                   = 0

fstat(3, {st_mode=S_IFREG|0600, st_size=2091, ...}) = 0

lseek(3, 0, SEEK_CUR)                   = 0

read(3, "build/temp.linux-x86_64-3.2/nump"..., 8192) = 2091

fstat(3, {st_mode=S_IFREG|0600, st_size=2091, ...}) = 0

lseek(3, 0, SEEK_CUR)                   = 2091

read(3, "", 8192)                       = 0

fstat(3, {st_mode=S_IFREG|0600, st_size=2091, ...}) = 0

lseek(3, 0, SEEK_CUR)                   = 2091

read(3, "", 8192)                       = 0

close(3)                                = 0

unlink("/tmp/tmpicyzqn/pur_c5")         = 0

write(1, "build/temp.linux-x86_64-3.2/nump"..., 
2090build/temp.linux-x86_64-3.2/numpy/numarray/_capi.o: In function 
`NA_checkFPErrors':

/usr/local/src/numpy-1.7.1/build/py3k/numpy/numarray/_capi.c:3019: undefined reference to `fetestexcept'

/usr/local/src/numpy-1.7.1/build/py3k/numpy/numarray/_capi.c:3026: undefined reference to `feclearexcept'

build/temp.linux-x86_64-3.2/numpy/numarray/_capi.o: In function `int_overflow_error':

/usr/local/src/numpy-1.7.1/build/py3k/numpy/numarray/_capi.c:263: undefined reference to `feraiseexcept'

build/temp.linux-x86_64-3.2/numpy/numarray/_capi.o: In function `num_atanh':

/usr/local/src/numpy-1.7.1/build/py3k/numpy/numarray/_capi.c:233: undefined reference to `log'

build/temp.linux-x86_64-3.2/numpy/numarray/_capi.o: In function `num_asinh':

/usr/local/src/numpy-1.7.1/build/py3k/numpy/numarray/_capi.c:228: undefined reference to `log'

/usr/local/src/numpy-1.7.1/build/py3k/numpy/numarray/_capi.c:228: undefined reference to `sqrt'

build/temp.linux-x86_64-3.2/numpy/numarray/_capi.o: In function `num_acosh':

/usr/local/src/numpy-1.7.1/build/py3k/numpy/numarray/_capi.c:213: undefined reference to `sqrt'

build/temp.linux-x86_64-3.2/numpy/numarray/_capi.o: In function `num_log':

/usr/local/src/numpy-1.7.1/build/py3k/numpy/numarray/_capi.c:190: undefined reference to `log'

build/temp.linux-x86_64-3.2/numpy/numarray/_capi.o: In function `num_acosh':

/usr/local/src/numpy-1.7.1/build/py3k/numpy/numarray/_capi.c:213: undefined reference to `log'

build/temp.linux-x86_64-3.2/numpy/numarray/_capi.o: In function `num_pow':

/usr/local/src/numpy-1.7.1/build/py3k/numpy/numarray/_capi.c:207: undefined reference to `pow'

build/temp.linux-x86_64-3.2/numpy/numarray/_capi.o: In function `num_log10':

/usr/local/src/numpy-1.7.1/build/py3k/numpy/numarray/_capi.c:198: undefined reference to `log10'

build/temp.linux-x86_64-3.2/numpy/numarray/_capi.o: In function `num_round':

/usr/local/src/numpy-1.7.1/build/py3k/numpy/numarray/_capi.c:239: undefined reference to `ceil'

/usr/local/src/numpy-1.7.1/build/py3k/numpy/numarray/_capi.c:239: undefined reference to `floor'

collect2: ld returned 1 exit status) = 2090

write(1, "\n", 1

)                       = 1

chdir("/usr/local/src/numpy-1.7.1")     = 0

write(2, "error: Command \"gcc -pthread -sh"..., 333error: Command 
"gcc -pthread -shared -Wl,--as-needed -Wl,--no-undefined -Wl,-z,relro 
-Wl,-O1 -Wl,--build-id -Wl,--enable-new-dtags 
build/temp.linux-x86_64-3.2/numpy/numarray/_capi.o -L/usr/lib64 
-Lbuild/temp.linux-x86_64-3.2 -lpython3.2mu -o 
build/lib.linux-x86_64-3.2/numpy/numarray/_capi.cpython-32mu.so" failed 
with exit status 1

) = 333

lstat("/tmp/tmpicyzqn", {st_mode=S_IFDIR|0700, st_size=4096, ...}) = 0

open("/tmp/tmpicyzqn", O_RDONLY|O_NONBLOCK|O_DIRECTORY|O_CLOEXEC) = 3

getdents(3, /* 2 entries */, 32768)     = 48

getdents(3, /* 0 entries */, 32768)     = 0

close(3)                                = 0

rmdir("/tmp/tmpicyzqn")                 = 0

rt_sigaction(SIGINT, {SIG_DFL, [], SA_RESTORER, 0x7f6f360bc960}, {0x7f6f363dfe70, [], SA_RESTORER, 0x7f6f35d55250}, 8) = 0

exit_group(1)

	
	
	
	

 		 	   		  
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20130504/6c330d09/attachment.html>

From robert.kern at gmail.com  Sat May  4 07:12:36 2013
From: robert.kern at gmail.com (Robert Kern)
Date: Sat, 4 May 2013 12:12:36 +0100
Subject: [Numpy-discussion] numpy.scipy.org page 404s
In-Reply-To: <km1i30$5v9$1@ger.gmane.org>
References: <CADeni3uq_w3rnERu39NDNrsDE31CO_49YobmqKGpfrRE4nNE6Q@mail.gmail.com>
	<CAF6FJiujKNHS09EkqdaMN1guo-kB=FTFAOwk7upxGUqUXVyD2g@mail.gmail.com>
	<517A9D98.70003@hilboll.de>
	<CAF6FJiszCiTNOUUFft1zVV7pv8_CkEPv-P+KFsUfe8OCJ9x=wQ@mail.gmail.com>
	<517AA0CF.3000801@hilboll.de>
	<CAF6FJiu1z+iWvtQLVV2pcc2676iQkhdf_tvtM3K8m+LgtRE=HA@mail.gmail.com>
	<km1i30$5v9$1@ger.gmane.org>
Message-ID: <CAF6FJitw=U2LCXN6riONqdJ+KAJNR8vo+0m5zhKFPRZ0JXochQ@mail.gmail.com>

On Sat, May 4, 2013 at 12:43 AM, Pauli Virtanen <pav at iki.fi> wrote:
> 26.04.2013 18:51, Robert Kern kirjoitti:
> [clip]
>> Right now, the recurring cost is kicking the www.scipy.org wiki every
>> once in a while under the deluge of spam.
>
> It's dying a slow death again.
>
> How about restricting editing pages to people in EditorGroup? There
> should be an option in Moin config for that. I don't think there is any
> other solution to the spam with the Moin wiki at present.

Done.

--
Robert Kern


From bakhtiyor_zokhidov at mail.ru  Sat May  4 14:18:09 2013
From: bakhtiyor_zokhidov at mail.ru (=?UTF-8?B?QmFraHRpeW9yIFpva2hpZG92?=)
Date: Sat, 04 May 2013 22:18:09 +0400
Subject: [Numpy-discussion] =?utf-8?q?=22Lists=22_and_=22Join=22_function_?=
	=?utf-8?q?needed?=
Message-ID: <1367691489.128936894@f226.mail.ru>


Hi,
I have the following code which represents intersected point of each cell in the given two points, A(x0,y0) and B(x1,y1).

def intersected_points(x0, x1, y0, y1):
# slope 
m = (y1 - y0 )/( x1 - x0)
# Boundary of the selected points
x_ceil = ceil( min (x0, x1 ))
x_floor = floor( max(x0, x1))
y_ceil = ceil( min(y0, y1))
y_floor = floor( max(y0, y1))
# calculate all intersected x coordinate
ax = []
for x in arange(x_ceil, x_floor + 1, 1):
ax.append([ x, m * ( x - x0 ) + y0 ])
# calculate all intersected y coordinate
for y in arange(y_ceil, y_floor + 1, 1):
ax.append([ ( y - y0 ) * ( 1./m ) + x0, y ])
return ax

Sample values:?intersected_points(1.5,4.4,0.5,4.1)
Output:?[[2.0, 1.1206896551724137], [3.0, 2.3620689655172411], [4.0, 3.6034482758620685], [1.9027777777777779, 1.0], [2.7083333333333335, 2.0], [3.5138888888888893, 3.0], [4.3194444444444446, 4.0]]

The output I got is unsorted values, so, for each cell coordinates, where line crosses:
BUT, The result I want to get should be something in increased orders like:?(x0,y0), (x1,y1), (x2,y2), (x3,y3)
where x0, y0 - intial, x1,y1 - final point. Other values are intersected line coordinates!

Any answers will be appreciated,

--? Bakhtiyor Zokhidov
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20130504/50125a27/attachment.html>

From warren.weckesser at gmail.com  Sat May  4 21:24:01 2013
From: warren.weckesser at gmail.com (Warren Weckesser)
Date: Sat, 4 May 2013 21:24:01 -0400
Subject: [Numpy-discussion] "Lists" and "Join" function needed
In-Reply-To: <1367691489.128936894@f226.mail.ru>
References: <1367691489.128936894@f226.mail.ru>
Message-ID: <CAGzF1ucZasH3vbODBPTa+iP0qP2feEwKQfttbOCt_HuoAorJKw@mail.gmail.com>

On 5/4/13, Bakhtiyor Zokhidov <bakhtiyor_zokhidov at mail.ru> wrote:
>
> Hi,
> I have the following code which represents intersected point of each cell in
> the given two points, A(x0,y0) and B(x1,y1).
>
> def intersected_points(x0, x1, y0, y1):
> # slope
> m = (y1 - y0 )/( x1 - x0)
> # Boundary of the selected points
> x_ceil = ceil( min (x0, x1 ))
> x_floor = floor( max(x0, x1))
> y_ceil = ceil( min(y0, y1))
> y_floor = floor( max(y0, y1))
> # calculate all intersected x coordinate
> ax = []
> for x in arange(x_ceil, x_floor + 1, 1):
> ax.append([ x, m * ( x - x0 ) + y0 ])
> # calculate all intersected y coordinate
> for y in arange(y_ceil, y_floor + 1, 1):
> ax.append([ ( y - y0 ) * ( 1./m ) + x0, y ])
> return ax
>
> Sample values: intersected_points(1.5,4.4,0.5,4.1)
> Output: [[2.0, 1.1206896551724137], [3.0, 2.3620689655172411], [4.0,
> 3.6034482758620685], [1.9027777777777779, 1.0], [2.7083333333333335, 2.0],
> [3.5138888888888893, 3.0], [4.3194444444444446, 4.0]]
>
> The output I got is unsorted values, so, for each cell coordinates, where
> line crosses:
> BUT, The result I want to get should be something in increased orders
> like: (x0,y0), (x1,y1), (x2,y2), (x3,y3)
> where x0, y0 - intial, x1,y1 - final point. Other values are intersected
> line coordinates!
>
> Any answers will be appreciated,
>
> --  Bakhtiyor Zokhidov


You also asked this question on stackoverflow
(http://stackoverflow.com/questions/16377826/distance-for-each-intersected-points-of-a-line-in-increased-order-in-2d-coordina).
 I've posted an answer there.

Warren


From bakhtiyor_zokhidov at mail.ru  Sun May  5 07:39:28 2013
From: bakhtiyor_zokhidov at mail.ru (=?UTF-8?B?QmFraHRpeW9yIFpva2hpZG92?=)
Date: Sun, 05 May 2013 15:39:28 +0400
Subject: [Numpy-discussion]
 =?utf-8?q?=22Lists=22_and_=22Join=22_function_?= =?utf-8?q?needed?=
In-Reply-To: <CAGzF1ucZasH3vbODBPTa+iP0qP2feEwKQfttbOCt_HuoAorJKw@mail.gmail.com>
References: <1367691489.128936894@f226.mail.ru>
	<CAGzF1ucZasH3vbODBPTa+iP0qP2feEwKQfttbOCt_HuoAorJKw@mail.gmail.com>
Message-ID: <1367753968.5695169@f169.mail.ru>

 Thanks for the answer in Stackoverflow. I checked it out.



???????,  4 ??? 2013, 21:24 -04:00 ?? Warren Weckesser <warren.weckesser at gmail.com>:
>On 5/4/13, Bakhtiyor Zokhidov < bakhtiyor_zokhidov at mail.ru > wrote:
>>
>> Hi,
>> I have the following code which represents intersected point of each cell in
>> the given two points, A(x0,y0) and B(x1,y1).
>>
>> def intersected_points(x0, x1, y0, y1):
>> # slope
>> m = (y1 - y0 )/( x1 - x0)
>> # Boundary of the selected points
>> x_ceil = ceil( min (x0, x1 ))
>> x_floor = floor( max(x0, x1))
>> y_ceil = ceil( min(y0, y1))
>> y_floor = floor( max(y0, y1))
>> # calculate all intersected x coordinate
>> ax = []
>> for x in arange(x_ceil, x_floor + 1, 1):
>> ax.append([ x, m * ( x - x0 ) + y0 ])
>> # calculate all intersected y coordinate
>> for y in arange(y_ceil, y_floor + 1, 1):
>> ax.append([ ( y - y0 ) * ( 1./m ) + x0, y ])
>> return ax
>>
>> Sample values: intersected_points(1.5,4.4,0.5,4.1)
>> Output: [[2.0, 1.1206896551724137], [3.0, 2.3620689655172411], [4.0,
>> 3.6034482758620685], [1.9027777777777779, 1.0], [2.7083333333333335, 2.0],
>> [3.5138888888888893, 3.0], [4.3194444444444446, 4.0]]
>>
>> The output I got is unsorted values, so, for each cell coordinates, where
>> line crosses:
>> BUT, The result I want to get should be something in increased orders
>> like: (x0,y0), (x1,y1), (x2,y2), (x3,y3)
>> where x0, y0 - intial, x1,y1 - final point. Other values are intersected
>> line coordinates!
>>
>> Any answers will be appreciated,
>>
>> --  Bakhtiyor Zokhidov
>
>
>You also asked this question on stackoverflow
>( http://stackoverflow.com/questions/16377826/distance-for-each-intersected-points-of-a-line-in-increased-order-in-2d-coordina ).
>?I've posted an answer there.
>
>Warren
>_______________________________________________
>NumPy-Discussion mailing list
>NumPy-Discussion at scipy.org
>http://mail.scipy.org/mailman/listinfo/numpy-discussion

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20130505/a7ef9ed1/attachment.html>

From pav at iki.fi  Sun May  5 08:34:53 2013
From: pav at iki.fi (Pauli Virtanen)
Date: Sun, 05 May 2013 15:34:53 +0300
Subject: [Numpy-discussion] numpy.scipy.org page 404s
In-Reply-To: <CAF6FJitw=U2LCXN6riONqdJ+KAJNR8vo+0m5zhKFPRZ0JXochQ@mail.gmail.com>
References: <CADeni3uq_w3rnERu39NDNrsDE31CO_49YobmqKGpfrRE4nNE6Q@mail.gmail.com>
	<CAF6FJiujKNHS09EkqdaMN1guo-kB=FTFAOwk7upxGUqUXVyD2g@mail.gmail.com>
	<517A9D98.70003@hilboll.de>
	<CAF6FJiszCiTNOUUFft1zVV7pv8_CkEPv-P+KFsUfe8OCJ9x=wQ@mail.gmail.com>
	<517AA0CF.3000801@hilboll.de>
	<CAF6FJiu1z+iWvtQLVV2pcc2676iQkhdf_tvtM3K8m+LgtRE=HA@mail.gmail.com>
	<km1i30$5v9$1@ger.gmane.org>
	<CAF6FJitw=U2LCXN6riONqdJ+KAJNR8vo+0m5zhKFPRZ0JXochQ@mail.gmail.com>
Message-ID: <km5jl8$fei$1@ger.gmane.org>

04.05.2013 14:12, Robert Kern kirjoitti:
[clip]
>> How about restricting editing pages to people in EditorGroup? There
>> should be an option in Moin config for that. I don't think there is any
>> other solution to the spam with the Moin wiki at present.
> 
> Done.

How do we go about removing the spam pages?  They seem to be hurting the
site's performance quite a lot --- for instance,
http://scipy.org/Cookbook doesn't load currently.

A good assumption is probably that new pages created in the past couple
of weeks are all spam...

-- 
Pauli Virtanen



From robert.kern at gmail.com  Sun May  5 08:42:26 2013
From: robert.kern at gmail.com (Robert Kern)
Date: Sun, 5 May 2013 13:42:26 +0100
Subject: [Numpy-discussion] numpy.scipy.org page 404s
In-Reply-To: <km5jl8$fei$1@ger.gmane.org>
References: <CADeni3uq_w3rnERu39NDNrsDE31CO_49YobmqKGpfrRE4nNE6Q@mail.gmail.com>
	<CAF6FJiujKNHS09EkqdaMN1guo-kB=FTFAOwk7upxGUqUXVyD2g@mail.gmail.com>
	<517A9D98.70003@hilboll.de>
	<CAF6FJiszCiTNOUUFft1zVV7pv8_CkEPv-P+KFsUfe8OCJ9x=wQ@mail.gmail.com>
	<517AA0CF.3000801@hilboll.de>
	<CAF6FJiu1z+iWvtQLVV2pcc2676iQkhdf_tvtM3K8m+LgtRE=HA@mail.gmail.com>
	<km1i30$5v9$1@ger.gmane.org>
	<CAF6FJitw=U2LCXN6riONqdJ+KAJNR8vo+0m5zhKFPRZ0JXochQ@mail.gmail.com>
	<km5jl8$fei$1@ger.gmane.org>
Message-ID: <CAF6FJivRK8wG+WgY=_U7G+7KSjVNHchaiberpAi2t381SXaHOA@mail.gmail.com>

On Sun, May 5, 2013 at 1:34 PM, Pauli Virtanen <pav at iki.fi> wrote:
> 04.05.2013 14:12, Robert Kern kirjoitti:
> [clip]
>>> How about restricting editing pages to people in EditorGroup? There
>>> should be an option in Moin config for that. I don't think there is any
>>> other solution to the spam with the Moin wiki at present.
>>
>> Done.
>
> How do we go about removing the spam pages?  They seem to be hurting the
> site's performance quite a lot --- for instance,
> http://scipy.org/Cookbook doesn't load currently.
>
> A good assumption is probably that new pages created in the past couple
> of weeks are all spam...

Manually. I have a whitelist of known-good pages that helps narrow it
down. I'll take care of it in a few hours.

--
Robert Kern


From pav at iki.fi  Sun May  5 08:49:58 2013
From: pav at iki.fi (Pauli Virtanen)
Date: Sun, 05 May 2013 15:49:58 +0300
Subject: [Numpy-discussion] numpy.scipy.org page 404s
In-Reply-To: <CAF6FJivRK8wG+WgY=_U7G+7KSjVNHchaiberpAi2t381SXaHOA@mail.gmail.com>
References: <CADeni3uq_w3rnERu39NDNrsDE31CO_49YobmqKGpfrRE4nNE6Q@mail.gmail.com>
	<CAF6FJiujKNHS09EkqdaMN1guo-kB=FTFAOwk7upxGUqUXVyD2g@mail.gmail.com>
	<517A9D98.70003@hilboll.de>
	<CAF6FJiszCiTNOUUFft1zVV7pv8_CkEPv-P+KFsUfe8OCJ9x=wQ@mail.gmail.com>
	<517AA0CF.3000801@hilboll.de>
	<CAF6FJiu1z+iWvtQLVV2pcc2676iQkhdf_tvtM3K8m+LgtRE=HA@mail.gmail.com>
	<km1i30$5v9$1@ger.gmane.org>
	<CAF6FJitw=U2LCXN6riONqdJ+KAJNR8vo+0m5zhKFPRZ0JXochQ@mail.gmail.com>
	<km5jl8$fei$1@ger.gmane.org>
	<CAF6FJivRK8wG+WgY=_U7G+7KSjVNHchaiberpAi2t381SXaHOA@mail.gmail.com>
Message-ID: <km5khh$mm0$1@ger.gmane.org>

05.05.2013 15:42, Robert Kern kirjoitti:
[clip: spam]
> Manually. I have a whitelist of known-good pages that helps narrow it
> down. I'll take care of it in a few hours.

Thanks a lot!

Cheers,
Pauli



From robert.kern at gmail.com  Sun May  5 11:01:26 2013
From: robert.kern at gmail.com (Robert Kern)
Date: Sun, 5 May 2013 16:01:26 +0100
Subject: [Numpy-discussion] numpy.scipy.org page 404s
In-Reply-To: <km5khh$mm0$1@ger.gmane.org>
References: <CADeni3uq_w3rnERu39NDNrsDE31CO_49YobmqKGpfrRE4nNE6Q@mail.gmail.com>
	<CAF6FJiujKNHS09EkqdaMN1guo-kB=FTFAOwk7upxGUqUXVyD2g@mail.gmail.com>
	<517A9D98.70003@hilboll.de>
	<CAF6FJiszCiTNOUUFft1zVV7pv8_CkEPv-P+KFsUfe8OCJ9x=wQ@mail.gmail.com>
	<517AA0CF.3000801@hilboll.de>
	<CAF6FJiu1z+iWvtQLVV2pcc2676iQkhdf_tvtM3K8m+LgtRE=HA@mail.gmail.com>
	<km1i30$5v9$1@ger.gmane.org>
	<CAF6FJitw=U2LCXN6riONqdJ+KAJNR8vo+0m5zhKFPRZ0JXochQ@mail.gmail.com>
	<km5jl8$fei$1@ger.gmane.org>
	<CAF6FJivRK8wG+WgY=_U7G+7KSjVNHchaiberpAi2t381SXaHOA@mail.gmail.com>
	<km5khh$mm0$1@ger.gmane.org>
Message-ID: <CAF6FJisshWgMgnnt5XC_TbEXSotd7MFKjSG24LoQb0Z7m_WXeg@mail.gmail.com>

On Sun, May 5, 2013 at 1:49 PM, Pauli Virtanen <pav at iki.fi> wrote:
> 05.05.2013 15:42, Robert Kern kirjoitti:
> [clip: spam]
>> Manually. I have a whitelist of known-good pages that helps narrow it
>> down. I'll take care of it in a few hours.
>
> Thanks a lot!

Done.

--
Robert Kern


From cournape at gmail.com  Sun May  5 17:57:45 2013
From: cournape at gmail.com (David Cournapeau)
Date: Sun, 5 May 2013 22:57:45 +0100
Subject: [Numpy-discussion] Profiling (was GSoC : Performance parity
 between numpy arrays and Python scalars)
In-Reply-To: <CAPJVwBkh0jda763ps+PF+yHrSJNNeSYdd7=bjUTi-jwjAp-vwg@mail.gmail.com>
References: <CAGY4rcVCN9v4fjWO8QtubQvvC8aTvxsYzBya+QzL3_82C5b4YQ@mail.gmail.com>
	<CAPJVwBkh0jda763ps+PF+yHrSJNNeSYdd7=bjUTi-jwjAp-vwg@mail.gmail.com>
Message-ID: <CAGY4rcUGWCCqegE16bzShZnoQDJ3TtC6kEvXNPkh5nB5Z-N8Sw@mail.gmail.com>

On Thu, May 2, 2013 at 2:58 PM, Nathaniel Smith <njs at pobox.com> wrote:
> On Thu, May 2, 2013 at 9:25 AM, David Cournapeau <cournape at gmail.com> wrote:
>>> * Re: the profiling, I wrote a full oprofile->callgrind format script
>>> years ago: http://vorpus.org/~njs/op2calltree.py
>>> Haven't used it in years either but neither oprofile nor kcachegrind
>>> are terribly fast-moving projects so it's probably still working, or
>>> could be made so without much work.
>>> Or easier is to use the gperftools CPU profiler:
>>> https://gperftools.googlecode.com/svn/trunk/doc/cpuprofile.html
>>
>> I don't have experience with gperftools, but on recent linux kernels,
>> you can also use perf, which can't be made easier to use (no runtime
>> support needed), but you need a 'recent' kernel
>>
>> http://indico.cern.ch/getFile.py/access?contribId=20&sessionId=4&resId=0&materialId=slides&confId=141309
>>
>> I am hoping to talk a bit about those for our diving into numpy c code
>> tutorial in June, what's the + of gperf in your opinion ?
>
> For what I've used profiling for, THE key feature is proper callgraph
> support ("show me the *total* time spent in each function, including
> callees"). Otherwise, silly example, let's say you have a bug where
> you wrote:
>
> func1() {
>   for (i = 0; i < 10000000; i++)
>      foo = add(foo, bar[0])
> }
>
> Obviously this is a waste of time, since you're actually performing
> the same operation over and over. Many profilers, given this, will
> tell you that all the time is spent in 'add', which is useless,
> because you don't want to speed up 'add', you want to speed up 'func1'
> (probably by not calling 'add' so many times!). If you have relatively
> flat code like most kernel code this isn't an issue, but I generally
> don't...
>
> perf is a fabulous framework and doesn't have any way to get full
> callgraph information out so IME it's been useless. They have
> reporting modes that claim to (like some "fractal" thing?) but AFAI
> been able to tell from docs/googling/mailing lists, there is nobody
> who understands how to interpret this output except the people who
> wrote it. Really a shame that it falls down in the last mile like
> that, hopefully they will fix this soon.

Perf doc is written for Vulcan, but it does what I think you want, say:

void work(int n) {
  volatile int i=0; //don't optimize away
  while(i++ < n);
}
void easy() { work(1000 * 1000 * 50); }
void hard() { work(1000*1000*1000); }
int main() { easy(); hard(); }

compile with gcc -g -O0, and then:

perf record -g -a -- ./a.out
perf report -g -a --stdio

gives me

  95.22%            a.out  a.out
  [.] work
                      |
                      --- work
                         |
                         |--89.84%-- hard
                         |          main
                         |          __libc_start_main
                         |
                          --5.38%-- easy
                                    main
                                    __libc_start_main


or maybe even better with the -G option

 95.22%            a.out  a.out
 [.] work
                      |
                      --- __libc_start_main
                          main
                         |
                         |--94.35%-- hard
                         |          work
                         |
                          --5.65%-- easy
                                    work


I agree it would be better with kcachgrind-like GUI, but it looks like
it is coming. I don't really understand the ncurses frontend to the
call graph.

(example taken/adapted from the great Yosefk blog:
http://www.yosefk.com/blog/how-profilers-lie-the-cases-of-gprof-and-kcachegrind.html)

cheers,
David


From njs at pobox.com  Sun May  5 19:26:25 2013
From: njs at pobox.com (Nathaniel Smith)
Date: Sun, 5 May 2013 19:26:25 -0400
Subject: [Numpy-discussion] Profiling (was GSoC : Performance parity
 between numpy arrays and Python scalars)
In-Reply-To: <CAGY4rcUGWCCqegE16bzShZnoQDJ3TtC6kEvXNPkh5nB5Z-N8Sw@mail.gmail.com>
References: <CAGY4rcVCN9v4fjWO8QtubQvvC8aTvxsYzBya+QzL3_82C5b4YQ@mail.gmail.com>
	<CAPJVwBkh0jda763ps+PF+yHrSJNNeSYdd7=bjUTi-jwjAp-vwg@mail.gmail.com>
	<CAGY4rcUGWCCqegE16bzShZnoQDJ3TtC6kEvXNPkh5nB5Z-N8Sw@mail.gmail.com>
Message-ID: <CAPJVwB=22CD=SuHHK_BB+06sroaeVSMA+x5y62Cpg0iHgCyAQw@mail.gmail.com>

On Sun, May 5, 2013 at 5:57 PM, David Cournapeau <cournape at gmail.com> wrote:
>> perf is a fabulous framework and doesn't have any way to get full
>> callgraph information out so IME it's been useless. They have
>> reporting modes that claim to (like some "fractal" thing?) but AFAI
>> been able to tell from docs/googling/mailing lists, there is nobody
>> who understands how to interpret this output except the people who
>> wrote it. Really a shame that it falls down in the last mile like
>> that, hopefully they will fix this soon.
>
> Perf doc is written for Vulcan, but it does what I think you want, say:
>
> void work(int n) {
>   volatile int i=0; //don't optimize away
>   while(i++ < n);
> }
> void easy() { work(1000 * 1000 * 50); }
> void hard() { work(1000*1000*1000); }
> int main() { easy(); hard(); }
>
> compile with gcc -g -O0, and then:
>
> perf record -g -a -- ./a.out
> perf report -g -a --stdio
>
> gives me
>
>   95.22%            a.out  a.out
>   [.] work
>                       |
>                       --- work
>                          |
>                          |--89.84%-- hard
>                          |          main
>                          |          __libc_start_main
>                          |
>                           --5.38%-- easy
>                                     main
>                                     __libc_start_main
>
>
> or maybe even better with the -G option
>
>  95.22%            a.out  a.out
>  [.] work
>                       |
>                       --- __libc_start_main
>                           main
>                          |
>                          |--94.35%-- hard
>                          |          work
>                          |
>                           --5.65%-- easy
>                                     work
>

Yeah I've seen these displays before and I can see the information is
there, and (knowing the code you ran) that somehow the first number
has to do with the time spent under 'hard' and the second to do with
time spent under 'easy', but I have no idea how to generalize this to
arbitrary samples of these output formats. That's what I meant.

-n


From daniele at grinta.net  Mon May  6 04:51:21 2013
From: daniele at grinta.net (Daniele Nicolodi)
Date: Mon, 06 May 2013 10:51:21 +0200
Subject: [Numpy-discussion] numpy ring buffer
Message-ID: <51876F09.4090608@grinta.net>

Hello,

numpy arrays are great for interfacing python with libraries that expect
continuous memory buffers for data passing.  However, libraries
interfacing to data acquisition hardware often use those buffers as ring
buffers where, once the buffer has been filled with data, new data will
be written overwriting the data at the beginning of the buffer.  The
application is supposed to have read this data meanwhile.

Efficiently processing the data in the ring buffer requires addressing
efficiently the content of the ring buffer.  In C or Cython this is very
easy, just compute the wrap around when accessing the single elements of
the array:

  buffer = np.empty(size)
  adc.setup(buffer)
  start = 0
  for n in adc.read():
      # data processing
      for i in range(start, start + n):
          element = buffer[start % size]
	  ....
      start += n

My current approach to do the same thing in Python is to use the
np.roll() function to "linearize" the buffer access:

  buffer = np.empty(size)
  adc.setup(buffer)
  start = 0
  for n in adc.read():
      data = np.roll(buffer, -start)[:n]
      start += n
      # data processing
      process(data)

Since np.roll() returns a view on the array i suppose this is very
efficient.  Does anyone have a better idea on how to do this?

Thank you. Cheers,
Daniele


From robert.kern at gmail.com  Mon May  6 05:01:37 2013
From: robert.kern at gmail.com (Robert Kern)
Date: Mon, 6 May 2013 10:01:37 +0100
Subject: [Numpy-discussion] numpy ring buffer
In-Reply-To: <51876F09.4090608@grinta.net>
References: <51876F09.4090608@grinta.net>
Message-ID: <CAF6FJisEXQ+bmKwbjUeOAgO4xkR1U9Sa2ycW1Z-9NUPzi4FPYw@mail.gmail.com>

On Mon, May 6, 2013 at 9:51 AM, Daniele Nicolodi <daniele at grinta.net> wrote:
> Hello,
>
> numpy arrays are great for interfacing python with libraries that expect
> continuous memory buffers for data passing.  However, libraries
> interfacing to data acquisition hardware often use those buffers as ring
> buffers where, once the buffer has been filled with data, new data will
> be written overwriting the data at the beginning of the buffer.  The
> application is supposed to have read this data meanwhile.
>
> Efficiently processing the data in the ring buffer requires addressing
> efficiently the content of the ring buffer.  In C or Cython this is very
> easy, just compute the wrap around when accessing the single elements of
> the array:
>
>   buffer = np.empty(size)
>   adc.setup(buffer)
>   start = 0
>   for n in adc.read():
>       # data processing
>       for i in range(start, start + n):
>           element = buffer[start % size]
>           ....
>       start += n
>
> My current approach to do the same thing in Python is to use the
> np.roll() function to "linearize" the buffer access:
>
>   buffer = np.empty(size)
>   adc.setup(buffer)
>   start = 0
>   for n in adc.read():
>       data = np.roll(buffer, -start)[:n]
>       start += n
>       # data processing
>       process(data)
>
> Since np.roll() returns a view on the array i suppose this is very
> efficient.  Does anyone have a better idea on how to do this?

np.roll() copies all of the data every time. It does not return a
view. Try a function like this instead:

[~]
|5> def ring_window(buffer, start, n):
..>     length = len(buffer)
..>     start %= length
..>     end = start + n
..>     if end <= length:
..>         window = buffer[start:end]
..>     else:
..>         end %= length
..>         window = np.concatenate((buffer[start:length], buffer[0:end]))
..>     return end, window
..>

[~]
|6> buffer = np.arange(20)

[~]
|7> start = 0

[~]
|8> for i in range(5):
..>     start, window = ring_window(buffer, start, 6)
..>     print start, window
..>
6 [0 1 2 3 4 5]
12 [ 6  7  8  9 10 11]
18 [12 13 14 15 16 17]
4 [18 19  0  1  2  3]
10 [4 5 6 7 8 9]

--
Robert Kern


From cournape at gmail.com  Mon May  6 05:28:02 2013
From: cournape at gmail.com (David Cournapeau)
Date: Mon, 6 May 2013 10:28:02 +0100
Subject: [Numpy-discussion] Any plans for windows 64-bit installer for
	1.7?
In-Reply-To: <CADDwiVAJs5QTuB6jWRysnFQjvTR=u9h1B1qNUHjiSu0i4y3zfg@mail.gmail.com>
References: <CAH6Pt5pMGbZ1wOHeMsUGjDQoSqdZ8dZx8Qfk7R5z2sd0SHH2OQ@mail.gmail.com>
	<CAH6Pt5oXFjkJ+++HxDZho_bVqROvW8+7dPhZ57H4rExACZxqvw@mail.gmail.com>
	<CABL7CQh6k5XKzWsEM3gTkT+tTPvbCMSCKuaB0xFk12iz0JgwPg@mail.gmail.com>
	<CAH6Pt5qvqsEYWMr9NXQPkPCufqV3VMizsVypizvWLzfy6NpBaA@mail.gmail.com>
	<CAF6FJiv693xK0+enL3U9VHMVuLXNQkFTQXUpTvzcNFQfWSXPrw@mail.gmail.com>
	<CAB6mnxJEBhrxoBdO62e_CHagU+_EAqC_2wRoYw5ZNA+uy_K-rw@mail.gmail.com>
	<CAH6Pt5oZ1D-+=EzN1RrRK=9kJYSB949ZM=Kv8cc4Xzt4zQ-CHg@mail.gmail.com>
	<CAH6Pt5pqbHUbL3aPRag8sNJnJthO7tL79o6qFry96aQCADdyLg@mail.gmail.com>
	<51119007.6090806@uci.edu>
	<CAH6Pt5rvYXnFsS6D-Bashp770MDRRqs8QLS1W4pvK9u213WsDA@mail.gmail.com>
	<CABL7CQicsKKE+Y=ObhPqOKs1BzhTE=OS_5j6mt+7n-MJGdwY_A@mail.gmail.com>
	<CAH6Pt5rPn4MxttRhFDZ3evaX371zoLh8qD5NUBwH8WU-ZSJ6Mw@mail.gmail.com>
	<CABL7CQiSJ0WJjUgBLXKF-bp1GnDKWRSEAy0pqQ+HyQODSeTM6Q@mail.gmail.com>
	<CAH6Pt5pmktcmqFnRxrjsC3hcKs3Qr2r7r31BYDQtJJiga=oF6g@mail.gmail.com>
	<5113399F.3090803@astro.uio.no>
	<CADDwiVAJs5QTuB6jWRysnFQjvTR=u9h1B1qNUHjiSu0i4y3zfg@mail.gmail.com>
Message-ID: <CAGY4rcUx=bgWGQ2xNwAxn43U=2QDy0D1K+kFE=RBXoSL=kOY6g@mail.gmail.com>

On Thu, Feb 7, 2013 at 6:21 AM, Ond?ej ?ert?k <ondrej.certik at gmail.com> wrote:
> On Wed, Feb 6, 2013 at 9:20 PM, Dag Sverre Seljebotn
> <d.s.seljebotn at astro.uio.no> wrote:
>> On 02/07/2013 12:16 AM, Matthew Brett wrote:
> [...]
>>> Can you clarify the people you think will get stuck?  I think I'm
>>> right in saying that anyone with a C extension should be able to build
>>> them against numpy, by installing the free (as-in-beer) MS tools?  So
>>> do you just mean people needing a Fortran compiler?  That's a small
>>> constituency, I think.
>>
>> Off the top of my head there's SciPy and pymc...
>>
>> Anyway, I'm butting in because I wish this discussion could separate
>> between the user perspective and the developer perspective.
>>
>> FWIW,
>>
>> 1) From a user's perspective, I don't understand this either. If you are
>> already using a closed source, not-free-as-in-beer operating system, why
>> would you not use (or buy!) a closed source, not-free-as-in-beer Fortran
>> compiler?
>
> Indeed. Though I really have no clue on the Windows use cases. Maybe
> most Windows users don't want to compile anything, just
> use numpy and scipy from Python?
>
>>
>> 2) BUT, the argument I've seen that I can at least understand is that
>> the release manager should be able to do a release using only open
>> source tools (even using Wine instead of Windows) and not rely on a
>> limited number of licenses. And that the release manager must be able to
>> perform all the official builds directly.
>
> As the release manager, I really only have two requirements:
>
> * I want to ssh in there from my Ubuntu
> * I want to automate the whole process
>
> For Mac, linux and Wine I can do that. So I have just spend few hours
> browsing the net and it looks like that the combination of Windows
> PowerShell 2.0:
>
> http://en.wikipedia.org/wiki/Windows_PowerShell
>
> and some SSH server, there are quite a few, one commercial but free
> for one user one connection (perfect for me!):
>
> http://www.powershellinside.com/powershell/ssh/
>
> So if I understand the pages correctly, I can login there from linux,
> and then I use the PowerShell commands to script anything. It looks
> like I can even use my Fabric fabfiles with powershell:
>
> https://gist.github.com/diyan/2850866
>
> I can also use git with PowerShell:
>
> http://windows.github.com/
> http://haacked.com/archive/2011/12/13/better-git-with-powershell.aspx

Ondrej, you may be interested in some hack I've done to use winrm from
fabric: https://github.com/fabric/fabric/pull/872

It gives a new winrm_run function where you can put any batch command.
While the code is a hack, it works pretty well in practice. This works
from mac os x and linux, without the need for wine, or ssh on windows.

David


From daniele at grinta.net  Mon May  6 05:39:15 2013
From: daniele at grinta.net (Daniele Nicolodi)
Date: Mon, 06 May 2013 11:39:15 +0200
Subject: [Numpy-discussion] numpy ring buffer
In-Reply-To: <CAF6FJisEXQ+bmKwbjUeOAgO4xkR1U9Sa2ycW1Z-9NUPzi4FPYw@mail.gmail.com>
References: <51876F09.4090608@grinta.net>
	<CAF6FJisEXQ+bmKwbjUeOAgO4xkR1U9Sa2ycW1Z-9NUPzi4FPYw@mail.gmail.com>
Message-ID: <51877A43.5010601@grinta.net>

On 06/05/2013 11:01, Robert Kern wrote:
> np.roll() copies all of the data every time. It does not return a
> view.

Are you sure about that?  Either I'm missing something, or it returns a
view in my testing (with a fairly old numpy, though):

In [209]: np.__version__
Out[209]: '1.6.2'

In [210]: v1 = np.arange(10)

In [211]: v1.flags['OWNDATA']
Out[211]: True

In [212]: v2 = np.roll(v1, -1)

In [213]: v2.flags['OWNDATA']
Out[213]: False


Cheers,
Daniele



From sebastian at sipsolutions.net  Mon May  6 05:50:29 2013
From: sebastian at sipsolutions.net (Sebastian Berg)
Date: Mon, 06 May 2013 11:50:29 +0200
Subject: [Numpy-discussion] numpy ring buffer
In-Reply-To: <51877A43.5010601@grinta.net>
References: <51876F09.4090608@grinta.net>
	<CAF6FJisEXQ+bmKwbjUeOAgO4xkR1U9Sa2ycW1Z-9NUPzi4FPYw@mail.gmail.com>
	<51877A43.5010601@grinta.net>
Message-ID: <1367833829.2506.5.camel@sebastian-laptop>

On Mon, 2013-05-06 at 11:39 +0200, Daniele Nicolodi wrote:
> On 06/05/2013 11:01, Robert Kern wrote:
> > np.roll() copies all of the data every time. It does not return a
> > view.
> 
> Are you sure about that?  Either I'm missing something, or it returns a
> view in my testing (with a fairly old numpy, though):
> 
> In [209]: np.__version__
> Out[209]: '1.6.2'
> 
> In [210]: v1 = np.arange(10)
> 
> In [211]: v1.flags['OWNDATA']
> Out[211]: True
> 
> In [212]: v2 = np.roll(v1, -1)
> 
> In [213]: v2.flags['OWNDATA']
> Out[213]: False
> 

Don't trust owndata in that regard... since it returns a view, but a
view into a copy. For example if you have subclasses that can be very
common. Try np.may_share_memory(v1, v2) for example.

- Sebastian

> 
> Cheers,
> Daniele
> 
> _______________________________________________
> NumPy-Discussion mailing list
> NumPy-Discussion at scipy.org
> http://mail.scipy.org/mailman/listinfo/numpy-discussion
> 




From daniele at grinta.net  Mon May  6 05:52:08 2013
From: daniele at grinta.net (Daniele Nicolodi)
Date: Mon, 06 May 2013 11:52:08 +0200
Subject: [Numpy-discussion] numpy ring buffer
In-Reply-To: <51877A43.5010601@grinta.net>
References: <51876F09.4090608@grinta.net>
	<CAF6FJisEXQ+bmKwbjUeOAgO4xkR1U9Sa2ycW1Z-9NUPzi4FPYw@mail.gmail.com>
	<51877A43.5010601@grinta.net>
Message-ID: <51877D48.5000406@grinta.net>

On 06/05/2013 11:39, Daniele Nicolodi wrote:
> On 06/05/2013 11:01, Robert Kern wrote:
>> np.roll() copies all of the data every time. It does not return a
>> view.
> 
> Are you sure about that?  Either I'm missing something, or it returns a
> view in my testing (with a fairly old numpy, though):

Ops... Yes, I missed something: np.roll() returns a view, but not a view
on the original array, which is indeed copied.

Your method however also copies the data in a temporary buffer for
operating on it as a single chunk of data.  It just reduces the copy to
the interesting region of the buffer array.

It looks like there is no way to use numpy to operate on a ring buffer
without copying the data, and in my use case I would like to avoid the
copying.  I'll write it in Cython...

Cheers,
Daniele




From robert.kern at gmail.com  Mon May  6 05:52:10 2013
From: robert.kern at gmail.com (Robert Kern)
Date: Mon, 6 May 2013 10:52:10 +0100
Subject: [Numpy-discussion] numpy ring buffer
In-Reply-To: <51877A43.5010601@grinta.net>
References: <51876F09.4090608@grinta.net>
	<CAF6FJisEXQ+bmKwbjUeOAgO4xkR1U9Sa2ycW1Z-9NUPzi4FPYw@mail.gmail.com>
	<51877A43.5010601@grinta.net>
Message-ID: <CAF6FJivcfgcW172=x6Cnj-3Uw+NAVrABzZKhOFz0j_kwjaf_xg@mail.gmail.com>

On Mon, May 6, 2013 at 10:39 AM, Daniele Nicolodi <daniele at grinta.net> wrote:
> On 06/05/2013 11:01, Robert Kern wrote:
>> np.roll() copies all of the data every time. It does not return a
>> view.
>
> Are you sure about that?  Either I'm missing something, or it returns a
> view in my testing (with a fairly old numpy, though):
>
> In [209]: np.__version__
> Out[209]: '1.6.2'
>
> In [210]: v1 = np.arange(10)
>
> In [211]: v1.flags['OWNDATA']
> Out[211]: True
>
> In [212]: v2 = np.roll(v1, -1)
>
> In [213]: v2.flags['OWNDATA']
> Out[213]: False

It may return a view on something, but it isn't a view on the original
array. It can't be, because the rolled result cannot be represented as
uniformly-strided memory access on the original data. Check the
source.

https://github.com/numpy/numpy/blob/master/numpy/core/numeric.py#L1173

The reason you see a "view" is that final `reshape()` call. But it
first copies the original data using `take()`.

--
Robert Kern


From robert.kern at gmail.com  Mon May  6 05:57:20 2013
From: robert.kern at gmail.com (Robert Kern)
Date: Mon, 6 May 2013 10:57:20 +0100
Subject: [Numpy-discussion] numpy ring buffer
In-Reply-To: <51877D48.5000406@grinta.net>
References: <51876F09.4090608@grinta.net>
	<CAF6FJisEXQ+bmKwbjUeOAgO4xkR1U9Sa2ycW1Z-9NUPzi4FPYw@mail.gmail.com>
	<51877A43.5010601@grinta.net> <51877D48.5000406@grinta.net>
Message-ID: <CAF6FJiv5hnj9YcgYCd8U4DaPAhdrbAq8tuX1WBOVLgPkCgOdgA@mail.gmail.com>

On Mon, May 6, 2013 at 10:52 AM, Daniele Nicolodi <daniele at grinta.net> wrote:
> On 06/05/2013 11:39, Daniele Nicolodi wrote:
>> On 06/05/2013 11:01, Robert Kern wrote:
>>> np.roll() copies all of the data every time. It does not return a
>>> view.
>>
>> Are you sure about that?  Either I'm missing something, or it returns a
>> view in my testing (with a fairly old numpy, though):
>
> Ops... Yes, I missed something: np.roll() returns a view, but not a view
> on the original array, which is indeed copied.
>
> Your method however also copies the data in a temporary buffer for
> operating on it as a single chunk of data.  It just reduces the copy to
> the interesting region of the buffer array.

Yes, but only in the one case where the window overlaps past the end.
The tradeoff depends on the window size and the buffer size.
Alternately, you can write your processing code to work with one or
two chunks, then you don't need to do any copying.

--
Robert Kern


From bakhtiyor_zokhidov at mail.ru  Mon May  6 08:24:44 2013
From: bakhtiyor_zokhidov at mail.ru (=?UTF-8?B?QmFraHRpeW9yIFpva2hpZG92?=)
Date: Mon, 06 May 2013 16:24:44 +0400
Subject: [Numpy-discussion] =?utf-8?q?Matching_line_points_with_grid_coord?=
 =?utf-8?q?inates_numerically?=
Message-ID: <1367843084.653834182@f334.mail.ru>

 Dear Python users,

I am having difficulty with numerically scaling to match line coordinates vs grid cell size coordinates. I want to calculate the following function:?F = ?distance_of_crossed_line x intersected_cell_value
The problem here is that when I calculate crossed_line_length in line coordinates that will unmatch vs grid coordinate, which is also another x,y with step e.g., dx=dy=2.5 each grid size. I want to do numerical calculation , say, F(distance, intersected_grid_value) function where, intersected_grid_value - values in intersected grid, distance - intersected_line_length (given below or can be found? http://stackoverflow.com/questions/16377826/distance-for-each-intersected-points-of-a-line-in-increased-order-in-2d-coordina )

import numpy as np
import scipy as sp

def distance_of_crossed_line(x0, x1, y0, y1):
? ? ? ?# slope
? ? ? ?m = (y1 - y0) / (x1 - x0)
? ? ? ?# Boundary of the selected points
? ? ? ?x_ceil = np.ceil(min(x0, x1))
? ? ? ?x_floor = np.floor(max(x0, x1))
? ? ? ?y_ceil = np.ceil(min(y0, y1))
? ? ? ?y_floor = np.floor(max(y0, y1))
? ? ? ?# calculate all intersected x coordinate
? ? ? ?x = np.arange(x_ceil, x_floor + 1)
? ? ? ?y = m * (x - x0) + y0
? ? ? ?ax = zip(x, y)
? ? ? ?# calculate all intersected y coordinate
? ? ? ?y = np.arange(y_ceil, y_floor + 1)
? ? ? ?x = (y - y0) / m + x0
? ? ? ?ax.extend(zip(x, y))
? ? ? ?ax.append((x0, y0))
? ? ? ?ax.append((x1, y1))
? ? ? ?ax.sort()

? ? ? ?# Transpose
? ? ? ?ax = np.array(ax).T
? ? ? ?# Calculate difference of intersections in X
? ? ? ?dist_x = np.diff(ax[0])
? ? ? ?# Calculate difference of intersections in Y
? ? ? ?dist_y = np.diff(ax[1])
? ? ? ?return np.sqrt(dist_x**2 + dist_y**2)

# PLEASE, note that line points are different from 2D array axis. they should be matched with each other. 
# 2D array.
d_array = np.array[[4.5, 4.5, 4.5, 3.4, 2.5],[ 3.9, 4.5, 5.2, 4.5, 3.4],[3.9, 3.9, 2.5, 2.2, 1.9]]

# Two sample points as a line
x = np.array([ -80, -40 ])
y = np.array([ 60, 55 ])

# The problem: 
F = intersected_line_length * array_grid_values_where_line_crossed_area
* It is not necessary for me to overlay lines onto grid cells properly, JUST, I need to calculate numerically accurate F function Thanks for the answer and guidance in advance,

-- 
Bakhtiyor Zokhidov
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20130506/23b9cd3b/attachment.html>

From funkydevnull at gmail.com  Mon May  6 08:52:31 2013
From: funkydevnull at gmail.com (Funky Dev)
Date: Mon, 06 May 2013 14:52:31 +0200
Subject: [Numpy-discussion] mpfr integration into numpy?
Message-ID: <5187A78F.7070801@gmail.com>

Hi,

I've got a project in which it turns out we need much higher precision
than even __float128 and playing around with a few alternatives mpfr
seems to be the highest performing possibility.  So I've starting
writing a cythonized class mpfr_array which provides array-like
functionality but with mpfr_t as a "primitive" type. 

While this is doable it feels very much like re-inventing the wheel as
I'm basically rewriting some part of numpy's functionality.  So I was
wondering, as I think this is a very standard use case, is there any
interest in adding mpfr_t as a "primitive" dtype to numpy?  I know one
can define new dtype's but I don't want it to by a python object since
then there will be a large python overhead for all operations (i.e. dot
products will happen in python, not C).  mpfr_t is a very natural dtype
to add as its fast, C-based and supports general precision.

I have to admit complete ignorance to numpy's internals but as I'm
writing my own verison of such a class I would be happy to work with
anyone more versed in numpy than myself to extend numpy with built-in
mpfr_t support.

On a related note, if this was done, would it automatically work with
functionality such as numpy.linalg.inv(), etc...?  In principle such
functions could have been written with macros to be more type-flexible
(i.e. an ADD(a,b,c) mapping to e.g. a=b+c for floats or to mpfr_add(a,
b, c) for mpfr_t) but I suspect this is not the case.

thanks,
Sheer


From njs at pobox.com  Mon May  6 09:57:30 2013
From: njs at pobox.com (Nathaniel Smith)
Date: Mon, 6 May 2013 09:57:30 -0400
Subject: [Numpy-discussion] mpfr integration into numpy?
In-Reply-To: <5187A78F.7070801@gmail.com>
References: <5187A78F.7070801@gmail.com>
Message-ID: <CAPJVwB=N2cWEje9qbrx-jH3xuEt2FoLvO=Hjbtyf-AoB+vsb_g@mail.gmail.com>

On Mon, May 6, 2013 at 8:52 AM, Funky Dev <funkydevnull at gmail.com> wrote:
> Hi,
>
> I've got a project in which it turns out we need much higher precision
> than even __float128 and playing around with a few alternatives mpfr
> seems to be the highest performing possibility.  So I've starting
> writing a cythonized class mpfr_array which provides array-like
> functionality but with mpfr_t as a "primitive" type.
>
> While this is doable it feels very much like re-inventing the wheel as
> I'm basically rewriting some part of numpy's functionality.  So I was
> wondering, as I think this is a very standard use case, is there any
> interest in adding mpfr_t as a "primitive" dtype to numpy?  I know one
> can define new dtype's but I don't want it to by a python object since
> then there will be a large python overhead for all operations (i.e. dot
> products will happen in python, not C).  mpfr_t is a very natural dtype
> to add as its fast, C-based and supports general precision.

You actually can define new dtypes implemented in C code -- in fact,
right now, that's the only way to do it. There are some examples of
how to do this from third-party code here:
  https://github.com/numpy/numpy-dtypes
Since mpfr is GPLed, it couldn't be included in core numpy in any
case, but I don't see any reason you couldn't implement it this way
and distribute it as a third-party extension.

There would be some subtleties to work out regarding memory management
-- I guess mpfr_t objects have a fixed width (which is a requirement
for dtypes), but internally malloc some buffers, so you have to be
careful with copying them etc.? Should be doable with some care.

> On a related note, if this was done, would it automatically work with
> functionality such as numpy.linalg.inv(), etc...?  In principle such
> functions could have been written with macros to be more type-flexible
> (i.e. an ADD(a,b,c) mapping to e.g. a=b+c for floats or to mpfr_add(a,
> b, c) for mpfr_t) but I suspect this is not the case.

No, it wouldn't work automatically. But in numpy 1.8 you can register
new implementations of 'inv' and friends when your library is loaded,
which np.linalg.inv() will then automatically use if it sees your new
dtype.

-n


From lists at onerussian.com  Mon May  6 10:32:41 2013
From: lists at onerussian.com (Yaroslav Halchenko)
Date: Mon, 6 May 2013 10:32:41 -0400
Subject: [Numpy-discussion] Really cruel draft of vbench setup for NumPy
 (.add.reduce benchmarks since 2011)
Message-ID: <20130506143241.GV5140@onerussian.com>


On Wed, 01 May 2013, Sebastian Berg wrote:
> > btw -- is there something like panda's vbench for numpy?  i.e. where
> > it would be possible to track/visualize such performance
> > improvements/hits?


> Sorry if it seemed harsh, but only skimmed mails and it seemed a bit
> like the an obvious piece was missing... There are no benchmark tests I
> am aware of. You can try:

> a = np.random.random((1000, 1000))

> and then time a.sum(1) and a.sum(0), on 1.7. the fast axis (1), is only
> slightly faster then the sum over the slow axis. On earlier numpy
> versions you will probably see something like half the speed for the
> slow axis (only got ancient or 1.7 numpy right now, so reluctant to give
> exact timings).

FWIW -- just as a cruel first attempt look at

http://www.onerussian.com/tmp/numpy-vbench-20130506/vb_vb_reduce.html

why float16 case is so special?

I have pushed this really coarse setup (based on some elderly copy of
pandas' vbench) to
https://github.com/yarikoptic/numpy-vbench

if you care to tune it up/extend and then I could fire it up again on
that box (which doesn't do anything else ATM AFAIK).   Since majority of
time is spent actually building it (did it with ccache though) it would
be neat if you come up with more of benchmarks to run which you might
think could be interesting/important.

-- 
Yaroslav O. Halchenko, Ph.D.
http://neuro.debian.net http://www.pymvpa.org http://www.fail2ban.org
Senior Research Associate,     Psychological and Brain Sciences Dept.
Dartmouth College, 419 Moore Hall, Hinman Box 6207, Hanover, NH 03755
Phone: +1 (603) 646-9834                       Fax: +1 (603) 646-1419
WWW:   http://www.linkedin.com/in/yarik        


From josef.pktd at gmail.com  Mon May  6 11:56:54 2013
From: josef.pktd at gmail.com (josef.pktd at gmail.com)
Date: Mon, 6 May 2013 11:56:54 -0400
Subject: [Numpy-discussion] Really cruel draft of vbench setup for NumPy
 (.add.reduce benchmarks since 2011)
In-Reply-To: <20130506143241.GV5140@onerussian.com>
References: <20130506143241.GV5140@onerussian.com>
Message-ID: <CAMMTP+D2yQY9ze7sZMZ+LTdpiOAA4BUk-4awoqtSr2K0sAAD9A@mail.gmail.com>

On Mon, May 6, 2013 at 10:32 AM, Yaroslav Halchenko
<lists at onerussian.com> wrote:
>
> On Wed, 01 May 2013, Sebastian Berg wrote:
>> > btw -- is there something like panda's vbench for numpy?  i.e. where
>> > it would be possible to track/visualize such performance
>> > improvements/hits?
>
>
>> Sorry if it seemed harsh, but only skimmed mails and it seemed a bit
>> like the an obvious piece was missing... There are no benchmark tests I
>> am aware of. You can try:
>
>> a = np.random.random((1000, 1000))
>
>> and then time a.sum(1) and a.sum(0), on 1.7. the fast axis (1), is only
>> slightly faster then the sum over the slow axis. On earlier numpy
>> versions you will probably see something like half the speed for the
>> slow axis (only got ancient or 1.7 numpy right now, so reluctant to give
>> exact timings).
>
> FWIW -- just as a cruel first attempt look at
>
> http://www.onerussian.com/tmp/numpy-vbench-20130506/vb_vb_reduce.html
>
> why float16 case is so special?
>
> I have pushed this really coarse setup (based on some elderly copy of
> pandas' vbench) to
> https://github.com/yarikoptic/numpy-vbench
>
> if you care to tune it up/extend and then I could fire it up again on
> that box (which doesn't do anything else ATM AFAIK).   Since majority of
> time is spent actually building it (did it with ccache though) it would
> be neat if you come up with more of benchmarks to run which you might
> think could be interesting/important.

nice results

Thanks Yaroslav,

Josef
my default: axis=0


>
> --
> Yaroslav O. Halchenko, Ph.D.
> http://neuro.debian.net http://www.pymvpa.org http://www.fail2ban.org
> Senior Research Associate,     Psychological and Brain Sciences Dept.
> Dartmouth College, 419 Moore Hall, Hinman Box 6207, Hanover, NH 03755
> Phone: +1 (603) 646-9834                       Fax: +1 (603) 646-1419
> WWW:   http://www.linkedin.com/in/yarik
> _______________________________________________
> NumPy-Discussion mailing list
> NumPy-Discussion at scipy.org
> http://mail.scipy.org/mailman/listinfo/numpy-discussion


From sebastian at sipsolutions.net  Mon May  6 12:03:52 2013
From: sebastian at sipsolutions.net (Sebastian Berg)
Date: Mon, 06 May 2013 18:03:52 +0200
Subject: [Numpy-discussion] Really cruel draft of vbench setup for NumPy
 (.add.reduce benchmarks since 2011)
In-Reply-To: <20130506143241.GV5140@onerussian.com>
References: <20130506143241.GV5140@onerussian.com>
Message-ID: <1367856232.2506.31.camel@sebastian-laptop>

On Mon, 2013-05-06 at 10:32 -0400, Yaroslav Halchenko wrote:
> On Wed, 01 May 2013, Sebastian Berg wrote:
> > > btw -- is there something like panda's vbench for numpy?  i.e. where
> > > it would be possible to track/visualize such performance
> > > improvements/hits?
> 
> 
> > Sorry if it seemed harsh, but only skimmed mails and it seemed a bit
> > like the an obvious piece was missing... There are no benchmark tests I
> > am aware of. You can try:
> 
> > a = np.random.random((1000, 1000))
> 
> > and then time a.sum(1) and a.sum(0), on 1.7. the fast axis (1), is only
> > slightly faster then the sum over the slow axis. On earlier numpy
> > versions you will probably see something like half the speed for the
> > slow axis (only got ancient or 1.7 numpy right now, so reluctant to give
> > exact timings).
> 
> FWIW -- just as a cruel first attempt look at
> 
> http://www.onerussian.com/tmp/numpy-vbench-20130506/vb_vb_reduce.html
> 
> why float16 case is so special?

Float16 is special, it is cpu-bound -- not memory bound as most
reductions -- because it is not a native type. First thought it
was weird, but it actually makes sense, if you have a and b as float16:

a + b

is actually more like (I believe...):

float16(float32(a) + float32(b))

This means there is type casting going on *inside* the ufunc! Normally
casting is handled outside the ufunc (by the buffered iterator).  Now I
did not check, but when the iteration order is not optimized, the ufunc
*can* simplify this to something similar to this (along the reduction
axis):

result = float32(a[0])
for i in xrange(a[1:]):
    result += float32(a.next())
return float16(result)

While for "optimized" iteration order, this cannot happen because the
intermediate result is always written back.

This means for optimized iteration order a single conversion to float is
necessary (in the inner loop), while for unoptimized iteration order two
conversions to float and one back is done.  Since this conversion is
costly, the memory throughput is actually not important (no gain from
buffering). This leads to the visible slowdown.  This is of course a bit
annoying, but not sure how you would solve it (Have the dtype signal
that it doesn't even want iteration order optimization? Try to do move
those weird float16 conversations from the ufunc to the iterator
somehow?).

> 
> I have pushed this really coarse setup (based on some elderly copy of
> pandas' vbench) to
> https://github.com/yarikoptic/numpy-vbench
> 
> if you care to tune it up/extend and then I could fire it up again on
> that box (which doesn't do anything else ATM AFAIK).   Since majority of
> time is spent actually building it (did it with ccache though) it would
> be neat if you come up with more of benchmarks to run which you might
> think could be interesting/important.
> 

I think this is pretty cool! Probably would be a while until there are
many tests, but if you or someone could set such thing up it could
slowly grow when larger code changes are done?

Regards,

Sebastian




From lists at onerussian.com  Mon May  6 12:11:53 2013
From: lists at onerussian.com (Yaroslav Halchenko)
Date: Mon, 6 May 2013 12:11:53 -0400
Subject: [Numpy-discussion] Really cruel draft of vbench setup for NumPy
 (.add.reduce benchmarks since 2011)
In-Reply-To: <1367856232.2506.31.camel@sebastian-laptop>
References: <20130506143241.GV5140@onerussian.com>
	<1367856232.2506.31.camel@sebastian-laptop>
Message-ID: <20130506161153.GW5140@onerussian.com>


On Mon, 06 May 2013, Sebastian Berg wrote:

> > if you care to tune it up/extend and then I could fire it up again on
> > that box (which doesn't do anything else ATM AFAIK).   Since majority of
> > time is spent actually building it (did it with ccache though) it would
> > be neat if you come up with more of benchmarks to run which you might
> > think could be interesting/important.

> I think this is pretty cool! Probably would be a while until there are
> many tests, but if you or someone could set such thing up it could
> slowly grow when larger code changes are done?

that is the idea but it would be nice to gather such simple
benchmark-tests.  if you could hint on the numpy functionality you think
especially worth benchmarking (I know -- there is a lot of things
which could be set to be benchmarked) -- that would be a nice starting
point: just list functionality/functions  you consider of primary
interest. and either it is worth testing for different types or just a
gross estimate (e.g. for the selection of types in a loop)

As for myself -- I guess I will add fancy indexing and slicing tests.

Adding them is quite easy: have a look at
https://github.com/yarikoptic/numpy-vbench/blob/master/vb_reduce.py
which is actually a bit more cumbersome because of running them for
different types.
This one is more obvious:
https://github.com/yarikoptic/numpy-vbench/blob/master/vb_io.py

-- 
Yaroslav O. Halchenko, Ph.D.
http://neuro.debian.net http://www.pymvpa.org http://www.fail2ban.org
Senior Research Associate,     Psychological and Brain Sciences Dept.
Dartmouth College, 419 Moore Hall, Hinman Box 6207, Hanover, NH 03755
Phone: +1 (603) 646-9834                       Fax: +1 (603) 646-1419
WWW:   http://www.linkedin.com/in/yarik        


From charlesr.harris at gmail.com  Mon May  6 13:05:54 2013
From: charlesr.harris at gmail.com (Charles R Harris)
Date: Mon, 6 May 2013 11:05:54 -0600
Subject: [Numpy-discussion] mpfr integration into numpy?
In-Reply-To: <CAPJVwB=N2cWEje9qbrx-jH3xuEt2FoLvO=Hjbtyf-AoB+vsb_g@mail.gmail.com>
References: <5187A78F.7070801@gmail.com>
	<CAPJVwB=N2cWEje9qbrx-jH3xuEt2FoLvO=Hjbtyf-AoB+vsb_g@mail.gmail.com>
Message-ID: <CAB6mnxJ+O89HwVH5qHiMPy0qUxaZ3mPUOt-togOvLOwN65ro1w@mail.gmail.com>

On Mon, May 6, 2013 at 7:57 AM, Nathaniel Smith <njs at pobox.com> wrote:

> On Mon, May 6, 2013 at 8:52 AM, Funky Dev <funkydevnull at gmail.com> wrote:
> > Hi,
> >
> > I've got a project in which it turns out we need much higher precision
> > than even __float128 and playing around with a few alternatives mpfr
> > seems to be the highest performing possibility.  So I've starting
> > writing a cythonized class mpfr_array which provides array-like
> > functionality but with mpfr_t as a "primitive" type.
> >
> > While this is doable it feels very much like re-inventing the wheel as
> > I'm basically rewriting some part of numpy's functionality.  So I was
> > wondering, as I think this is a very standard use case, is there any
> > interest in adding mpfr_t as a "primitive" dtype to numpy?  I know one
> > can define new dtype's but I don't want it to by a python object since
> > then there will be a large python overhead for all operations (i.e. dot
> > products will happen in python, not C).  mpfr_t is a very natural dtype
> > to add as its fast, C-based and supports general precision.
>
> You actually can define new dtypes implemented in C code -- in fact,
> right now, that's the only way to do it. There are some examples of
> how to do this from third-party code here:
>   https://github.com/numpy/numpy-dtypes
> Since mpfr is GPLed, it couldn't be included in core numpy in any
> case, but I don't see any reason you couldn't implement it this way
> and distribute it as a third-party extension.
>

The rational test class recently merged into numpy is more complete and
fixed
up than the version in numpy-dtypes, although I can't help thinking it
could be streamlined even more.


>
> There would be some subtleties to work out regarding memory management
> -- I guess mpfr_t objects have a fixed width (which is a requirement
> for dtypes), but internally malloc some buffers, so you have to be
> careful with copying them etc.? Should be doable with some care.
>
> > On a related note, if this was done, would it automatically work with
> > functionality such as numpy.linalg.inv(), etc...?  In principle such
> > functions could have been written with macros to be more type-flexible
> > (i.e. an ADD(a,b,c) mapping to e.g. a=b+c for floats or to mpfr_add(a,
> > b, c) for mpfr_t) but I suspect this is not the case.
>
> No, it wouldn't work automatically. But in numpy 1.8 you can register
> new implementations of 'inv' and friends when your library is loaded,
> which np.linalg.inv() will then automatically use if it sees your new
> dtype.
>
>
Chuck
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20130506/0ef397c3/attachment.html>

From mrbago at gmail.com  Mon May  6 18:54:33 2013
From: mrbago at gmail.com (Bago)
Date: Mon, 6 May 2013 15:54:33 -0700
Subject: [Numpy-discussion] searchsorted descending arrays
Message-ID: <CAHHh6Ca28xAbKNzarWBu62i2U=9b=HfH0uH4c7SaNdn==4DQ9Q@mail.gmail.com>

I submitted a patch a little while ago,
https://github.com/numpy/numpy/pull/3107, which gave the searchsorted
function the ability to search arrays sorted in descending order. At the
time my  approach was to detect the sortorder of the array by comparing the
first and last elements. This works pretty well in most cases, but fails in
one notable case. After giving it some thought, I think the best way to add
searching of descending arrays to numpy would be by adding a keyword to the
searchsorted function. I wanted to know what you guys thought of this
before updating the pr.

I would like to add something like the following to numpy:

A = [10, 9, 2, 1]
np.searchsorted(A, 5, sortorder='descending')

the other option would be to auto-detect the order, but then this case
might surprise some users:

A = [0, 0, 0]
A = np.sort(A)[::-1]
print np.searchsorted(A, [1, -1])
# [3, 0]

This might surprise a user who expects to be searching a descending array


Bago
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20130506/b3bd2610/attachment.html>

From njs at pobox.com  Mon May  6 19:16:41 2013
From: njs at pobox.com (Nathaniel Smith)
Date: Mon, 6 May 2013 19:16:41 -0400
Subject: [Numpy-discussion] searchsorted descending arrays
In-Reply-To: <CAHHh6Ca28xAbKNzarWBu62i2U=9b=HfH0uH4c7SaNdn==4DQ9Q@mail.gmail.com>
References: <CAHHh6Ca28xAbKNzarWBu62i2U=9b=HfH0uH4c7SaNdn==4DQ9Q@mail.gmail.com>
Message-ID: <CAPJVwBkDdrnOk4n44pCU9vCtnD0XfB8GtyNTFCZ4djp5v9D3GQ@mail.gmail.com>

On Mon, May 6, 2013 at 6:54 PM, Bago <mrbago at gmail.com> wrote:
> I submitted a patch a little while ago,
> https://github.com/numpy/numpy/pull/3107, which gave the searchsorted
> function the ability to search arrays sorted in descending order. At the
> time my  approach was to detect the sortorder of the array by comparing the
> first and last elements. This works pretty well in most cases, but fails in
> one notable case. After giving it some thought, I think the best way to add
> searching of descending arrays to numpy would be by adding a keyword to the
> searchsorted function. I wanted to know what you guys thought of this before
> updating the pr.
>
> I would like to add something like the following to numpy:
>
> A = [10, 9, 2, 1]
> np.searchsorted(A, 5, sortorder='descending')
>
> the other option would be to auto-detect the order, but then this case might
> surprise some users:
>
> A = [0, 0, 0]
> A = np.sort(A)[::-1]
> print np.searchsorted(A, [1, -1])
> # [3, 0]
>
> This might surprise a user who expects to be searching a descending array

I agree, that result would not really be acceptable (could easily
break various algorithms in very hard to notice cases), so a kwarg
would be better.

-n


From sudheer.joseph at yahoo.com  Tue May  7 05:14:46 2013
From: sudheer.joseph at yahoo.com (Sudheer Joseph)
Date: Tue, 7 May 2013 17:14:46 +0800 (SGT)
Subject: [Numpy-discussion] printing array in tabular form
Message-ID: <1367918086.35474.YahooMailNeo@web193404.mail.sg3.yahoo.com>

Dear experts,
??????????? I need to print few arrays in a tabular form for example below array IL has 25 elements, is there an easy way to print this as 5x5 comma separated table? in python

IL=[]
for i in np.arange(1,bno+1):
?????? IL.append(i)
print(IL)

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
in fortran I could do it as below
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
integer matrix(5,5)
?????? in=0
????? do, k=1,5
????? do, l=1,5
?????? in=in+1
????? matrix(k,l)=in
????? enddo
????? enddo
????? m=5
????? n=5
????? do, i=1,m
????? write(*,"(5i5)") ( matrix(i,j), j=1,n )
????? enddo
????? end
?

***************************************************************
Sudheer Joseph 
Indian National Centre for Ocean Information Services
Ministry of Earth Sciences, Govt. of India
POST BOX NO: 21, IDA Jeedeemetla P.O.
Via Pragathi Nagar,Kukatpally, Hyderabad; Pin:5000 55
Tel:+91-40-23886047(O),Fax:+91-40-23895011(O),
Tel:+91-40-23044600(R),Tel:+91-40-9440832534(Mobile)
E-mail:sjo.India at gmail.com;sudheer.joseph at yahoo.com
Web- http://oppamthadathil.tripod.com
***************************************************************
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20130507/386a8f96/attachment.html>

From Nicolas.Rougier at inria.fr  Tue May  7 05:54:38 2013
From: Nicolas.Rougier at inria.fr (Nicolas Rougier)
Date: Tue, 7 May 2013 11:54:38 +0200
Subject: [Numpy-discussion] Numpy beginner tutorial
Message-ID: <2160ED3D-5471-42A1-9514-3A5C07FCD042@inria.fr>



Hello everybody,


I've written a numpy beginner tutorial that is available from:

http://www.loria.fr/~rougier/teaching/numpy/numpy.html

It has been designed around cellular automata to try to make it fun.


While writing it, I tried to compile a set of exercises and make them progressively harder. For advanced levels, I thought the easiest way would be to extract simple questions (but more importantly answers) from this very mailing list in order to gather them on a single page. The goal would be both to offer a quick reference for new (and old users) and to provide also a set of exercices for those who teach. However, it's a bit harder than I thought since the mailing list is huge.

I made a separate page for this:

http://www.loria.fr/~rougier/teaching/numpy.100/index.html
(Sources are http://www.loria.fr/~rougier/teaching/numpy.100/index.rst)

(The level names came from an old-game: Dungeon Master)


In order to extract questions/answers and I would need some help, if you have some free time to spare...

If you remember having asked or answered a (short) problem, could you send a link to the relevant post (the one with the answer), or better, write directly the formated entry. Here is an example:


#. Find indices of non-zero elements from [1,2,0,0,4,0]

   .. code:: python

      # Author: Somebody

      print np.nonzero([1,2,0,0,4,0])


If you can provide the (assumed) level of the answer, that would be even better.


Nicolas



From sebastian at sipsolutions.net  Tue May  7 07:47:18 2013
From: sebastian at sipsolutions.net (Sebastian Berg)
Date: Tue, 07 May 2013 13:47:18 +0200
Subject: [Numpy-discussion] Really cruel draft of vbench setup for NumPy
 (.add.reduce benchmarks since 2011)
In-Reply-To: <20130506161153.GW5140@onerussian.com>
References: <20130506143241.GV5140@onerussian.com>
	<1367856232.2506.31.camel@sebastian-laptop>
	<20130506161153.GW5140@onerussian.com>
Message-ID: <1367927238.23010.12.camel@sebastian-laptop>

On Mon, 2013-05-06 at 12:11 -0400, Yaroslav Halchenko wrote:
> On Mon, 06 May 2013, Sebastian Berg wrote:
> 
> > > if you care to tune it up/extend and then I could fire it up again on
> > > that box (which doesn't do anything else ATM AFAIK).   Since majority of
> > > time is spent actually building it (did it with ccache though) it would
> > > be neat if you come up with more of benchmarks to run which you might
> > > think could be interesting/important.
> 
> > I think this is pretty cool! Probably would be a while until there are
> > many tests, but if you or someone could set such thing up it could
> > slowly grow when larger code changes are done?
> 
> that is the idea but it would be nice to gather such simple
> benchmark-tests.  if you could hint on the numpy functionality you think
> especially worth benchmarking (I know -- there is a lot of things
> which could be set to be benchmarked) -- that would be a nice starting
> point: just list functionality/functions  you consider of primary
> interest. and either it is worth testing for different types or just a
> gross estimate (e.g. for the selection of types in a loop)
> 
> As for myself -- I guess I will add fancy indexing and slicing tests.
> 

Indexing/assignment was the first thing I thought of too (also because
fancy indexing/assignment really could use some speedups...). Other then
that maybe some timings for small arrays/scalar math, but that might be
nice for that GSoC project.

Maybe array creation functions, just to see if performance bugs should
sneak into something that central. But can't think of something else
that isn't specific functionality.

- Sebastian

> Adding them is quite easy: have a look at
> https://github.com/yarikoptic/numpy-vbench/blob/master/vb_reduce.py
> which is actually a bit more cumbersome because of running them for
> different types.
> This one is more obvious:
> https://github.com/yarikoptic/numpy-vbench/blob/master/vb_io.py
> 




From davidmenhur at gmail.com  Tue May  7 08:09:02 2013
From: davidmenhur at gmail.com (=?UTF-8?B?RGHPgGlk?=)
Date: Tue, 7 May 2013 14:09:02 +0200
Subject: [Numpy-discussion] Really cruel draft of vbench setup for NumPy
 (.add.reduce benchmarks since 2011)
In-Reply-To: <1367927238.23010.12.camel@sebastian-laptop>
References: <20130506143241.GV5140@onerussian.com>
	<1367856232.2506.31.camel@sebastian-laptop>
	<20130506161153.GW5140@onerussian.com>
	<1367927238.23010.12.camel@sebastian-laptop>
Message-ID: <CAJhcF=1a=zHQ7JgXF9f4N+1pHMr_6RESq6Me9oSYewjj62L0MA@mail.gmail.com>

On 7 May 2013 13:47, Sebastian Berg <sebastian at sipsolutions.net> wrote:
> Indexing/assignment was the first thing I thought of too (also because
> fancy indexing/assignment really could use some speedups...). Other then
> that maybe some timings for small arrays/scalar math, but that might be
> nice for that GSoC project.

Why not going bigger? Ufunc operations on big arrays, CPU and memory bound.

Also, what about interfacing with other packages? It may increase the
compiling overhead, but I would like to see Cython in action (say,
only last version, maybe it can be fixed).


From derek at astro.physik.uni-goettingen.de  Tue May  7 09:11:19 2013
From: derek at astro.physik.uni-goettingen.de (Derek Homeier)
Date: Tue, 7 May 2013 15:11:19 +0200
Subject: [Numpy-discussion] printing array in tabular form
In-Reply-To: <1367918086.35474.YahooMailNeo@web193404.mail.sg3.yahoo.com>
References: <1367918086.35474.YahooMailNeo@web193404.mail.sg3.yahoo.com>
Message-ID: <086DEFA5-2C1B-41F2-934F-4EDBE249C3D3@astro.physik.uni-goettingen.de>

Dear Sudheer,

On 07.05.2013, at 11:14AM, Sudheer Joseph <sudheer.joseph at yahoo.com> wrote:

>             I need to print few arrays in a tabular form for example below array IL has 25 elements, is there an easy way to print this as 5x5 comma separated table? in python
> 
> IL=[]
> for i in np.arange(1,bno+1):
>        IL.append(i)
> print(IL)
> 
assuming you want this table printed to a file, savetxt does just what you need. In brief for your case,

np.savetxt("file.txt", IL.reshape(-1,5), fmt='%5d', delimiter=',')

should print it in the requested form; you can refer to the save txt documentation for further options.

HTH,
						Derek



From ben.root at ou.edu  Tue May  7 09:15:27 2013
From: ben.root at ou.edu (Benjamin Root)
Date: Tue, 7 May 2013 09:15:27 -0400
Subject: [Numpy-discussion] searchsorted descending arrays
In-Reply-To: <CAPJVwBkDdrnOk4n44pCU9vCtnD0XfB8GtyNTFCZ4djp5v9D3GQ@mail.gmail.com>
References: <CAHHh6Ca28xAbKNzarWBu62i2U=9b=HfH0uH4c7SaNdn==4DQ9Q@mail.gmail.com>
	<CAPJVwBkDdrnOk4n44pCU9vCtnD0XfB8GtyNTFCZ4djp5v9D3GQ@mail.gmail.com>
Message-ID: <CANNq6F=AgLg2Wmppt3v-=2ZXCCCs7bNb-gP+Mae=0tXoH6mOJw@mail.gmail.com>

On Mon, May 6, 2013 at 7:16 PM, Nathaniel Smith <njs at pobox.com> wrote:

> On Mon, May 6, 2013 at 6:54 PM, Bago <mrbago at gmail.com> wrote:
> > I submitted a patch a little while ago,
> > https://github.com/numpy/numpy/pull/3107, which gave the searchsorted
> > function the ability to search arrays sorted in descending order. At the
> > time my  approach was to detect the sortorder of the array by comparing
> the
> > first and last elements. This works pretty well in most cases, but fails
> in
> > one notable case. After giving it some thought, I think the best way to
> add
> > searching of descending arrays to numpy would be by adding a keyword to
> the
> > searchsorted function. I wanted to know what you guys thought of this
> before
> > updating the pr.
> >
> > I would like to add something like the following to numpy:
> >
> > A = [10, 9, 2, 1]
> > np.searchsorted(A, 5, sortorder='descending')
> >
> > the other option would be to auto-detect the order, but then this case
> might
> > surprise some users:
> >
> > A = [0, 0, 0]
> > A = np.sort(A)[::-1]
> > print np.searchsorted(A, [1, -1])
> > # [3, 0]
> >
> > This might surprise a user who expects to be searching a descending array
>
> I agree, that result would not really be acceptable (could easily
> break various algorithms in very hard to notice cases), so a kwarg
> would be better.
>
> -n
>

+1 on kwarg approach.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20130507/2f027cb0/attachment.html>

From ralf.gommers at gmail.com  Tue May  7 14:34:36 2013
From: ralf.gommers at gmail.com (Ralf Gommers)
Date: Tue, 7 May 2013 20:34:36 +0200
Subject: [Numpy-discussion] Numpy beginner tutorial
In-Reply-To: <2160ED3D-5471-42A1-9514-3A5C07FCD042@inria.fr>
References: <2160ED3D-5471-42A1-9514-3A5C07FCD042@inria.fr>
Message-ID: <CABL7CQigsS6Q8ELqSQOQKh0RE6mDgVYcXJEmupjZMd+5+YeHUw@mail.gmail.com>

On Tue, May 7, 2013 at 11:54 AM, Nicolas Rougier
<Nicolas.Rougier at inria.fr>wrote:

>
>
> Hello everybody,
>
>
> I've written a numpy beginner tutorial that is available from:
>
> http://www.loria.fr/~rougier/teaching/numpy/numpy.html
>
> It has been designed around cellular automata to try to make it fun.
>

Hi Nicolas, that looks good. You're linking to some other tutorials at the
bottom, maybe you can add http://scipy-lectures.github.io/ (has both an
intro and an advanced numpy tutorial).

Ralf
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20130507/5ceec6c0/attachment.html>

From Nicolas.Rougier at inria.fr  Tue May  7 14:44:56 2013
From: Nicolas.Rougier at inria.fr (Nicolas Rougier)
Date: Tue, 7 May 2013 20:44:56 +0200
Subject: [Numpy-discussion] Numpy beginner tutorial
In-Reply-To: <CABL7CQigsS6Q8ELqSQOQKh0RE6mDgVYcXJEmupjZMd+5+YeHUw@mail.gmail.com>
References: <2160ED3D-5471-42A1-9514-3A5C07FCD042@inria.fr>
	<CABL7CQigsS6Q8ELqSQOQKh0RE6mDgVYcXJEmupjZMd+5+YeHUw@mail.gmail.com>
Message-ID: <A8492E1A-BC00-4D17-8B61-9D8D80CE32FC@inria.fr>



Shame on me ! How did I forget this one...
Thanks, just added it.

Nicolas


> 
> Hi Nicolas, that looks good. You're linking to some other tutorials at the bottom, maybe you can add http://scipy-lectures.github.io/ (has both an intro and an advanced numpy tutorial). 
> 
> Ralf



From e.antero.tammi at gmail.com  Tue May  7 15:03:16 2013
From: e.antero.tammi at gmail.com (eat)
Date: Tue, 7 May 2013 22:03:16 +0300
Subject: [Numpy-discussion] Numpy beginner tutorial
In-Reply-To: <2160ED3D-5471-42A1-9514-3A5C07FCD042@inria.fr>
References: <2160ED3D-5471-42A1-9514-3A5C07FCD042@inria.fr>
Message-ID: <CAKa=AYQsmzJEhYMaTu6P7Or9Obk3vnvrfvKODeR6gL4gp8UdCQ@mail.gmail.com>

Hi,

Looks nice tutorial, indeed.


On Tue, May 7, 2013 at 12:54 PM, Nicolas Rougier
<Nicolas.Rougier at inria.fr>wrote:

>
>
> Hello everybody,
>
>
> I've written a numpy beginner tutorial that is available from:
>
> http://www.loria.fr/~rougier/teaching/numpy/numpy.html
>
> It has been designed around cellular automata to try to make it fun.
>
Perhaps you could also link to
http://www.scipy.org/Cookbook/GameOfLifeStrides (at least if you are
planning to have exercises beyond Apprentice level). IMHO it just provides
more natural view of the neighborhood via stride_tricks.

>
>
> While writing it, I tried to compile a set of exercises and make them
> progressively harder. For advanced levels, I thought the easiest way would
> be to extract simple questions (but more importantly answers) from this
> very mailing list in order to gather them on a single page. The goal would
> be both to offer a quick reference for new (and old users) and to provide
> also a set of exercices for those who teach. However, it's a bit harder
> than I thought since the mailing list is huge.
>
> I made a separate page for this:
>
> http://www.loria.fr/~rougier/teaching/numpy.100/index.html
> (Sources are http://www.loria.fr/~rougier/teaching/numpy.100/index.rst)
>
> (The level names came from an old-game: Dungeon Master)
>
>
> In order to extract questions/answers and I would need some help, if you
> have some free time to spare...
>
> If you remember having asked or answered a (short) problem, could you send
> a link to the relevant post (the one with the answer), or better, write
> directly the formated entry. Here is an example:
>
>
> #. Find indices of non-zero elements from [1,2,0,0,4,0]
>
>    .. code:: python
>
>       # Author: Somebody
>
>       print np.nonzero([1,2,0,0,4,0])
>
>
> If you can provide the (assumed) level of the answer, that would be even
> better.

My 2 cents,
-eat

>


> Nicolas
>
> _______________________________________________
> NumPy-Discussion mailing list
> NumPy-Discussion at scipy.org
> http://mail.scipy.org/mailman/listinfo/numpy-discussion
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20130507/5049625d/attachment.html>

From mhpc.edas at gmail.com  Tue May  7 18:33:42 2013
From: mhpc.edas at gmail.com (MHPC 2013)
Date: Wed, 8 May 2013 00:33:42 +0200
Subject: [Numpy-discussion] CfP 2013 Workshop on Middleware for HPC and Big
	Data Systems (MHPC'13)
Message-ID: <CAMDtREDPtK=ZGuc4+mEU-0g3KBZvrV2YaHrsq7tR7qoToy0xMw@mail.gmail.com>

we apologize if you receive multiple copies of this message
===================================================================

CALL FOR PAPERS

2013 Workshop on

Middleware for HPC and Big Data Systems

MHPC '13

as part of Euro-Par 2013, Aachen, Germany

===================================================================

Date: August 27, 2012

Workshop URL: http://m-hpc.org

Springer LNCS

SUBMISSION DEADLINE:

May 31, 2013 - LNCS Full paper submission (rolling abstract submission)
June 28, 2013 - Lightning Talk abstracts


SCOPE

Extremely large, diverse, and complex data sets are generated from
scientific applications, the Internet, social media and other applications.
Data may be physically distributed and shared by an ever larger community.
Collecting, aggregating, storing and analyzing large data volumes
presents major challenges. Processing such amounts of data efficiently
has been an issue to scientific discovery and technological
advancement. In addition, making the data accessible, understandable and
interoperable includes unsolved problems. Novel middleware architectures,
algorithms, and application development frameworks are required.

In this workshop we are particularly interested in original work at the
intersection of HPC and Big Data with regard to middleware handling
and optimizations. Scope is existing and proposed middleware for HPC
and big data, including analytics libraries and frameworks.

The goal of this workshop is to bring together software architects,
middleware and framework developers, data-intensive application developers
as well as users from the scientific and engineering community to exchange
their experience in processing large datasets and to report their scientific
achievement and innovative ideas. The workshop also offers a dedicated forum
for these researchers to access the state of the art, to discuss problems
and requirements, to identify gaps in current and planned designs, and to
collaborate in strategies for scalable data-intensive computing.

The workshop will be one day in length, composed of 20 min paper
presentations, each followed by 10 min discussion sections.
Presentations may be accompanied by interactive demonstrations.


TOPICS

Topics of interest include, but are not limited to:

- Middleware including: Hadoop, Apache Drill, YARN, Spark/Shark, Hive,
Pig, Sqoop,
HBase, HDFS, S4, CIEL, Oozie, Impala, Storm and Hyrack
- Data intensive middleware architecture
 - Libraries/Frameworks including: Apache Mahout, Giraph, UIMA and GraphLab
- NG Databases including Apache Cassandra, MongoDB and CouchDB/Couchbase
- Schedulers including Cascading
- Middleware for optimized data locality/in-place data processing
- Data handling middleware for deployment in virtualized HPC environments
- Parallelization and distributed processing architectures at the
middleware level
- Integration with cloud middleware and application servers
- Runtime environments and system level support for data-intensive computing
- Skeletons and patterns
- Checkpointing
- Programming models and languages
- Big Data ETL
- Stream processing middleware
- In-memory databases for HPC
- Scalability and interoperability
- Large-scale data storage and distributed file systems
- Content-centric addressing and networking
- Execution engines, languages and environments including CIEL/Skywriting
- Performance analysis, evaluation of data-intensive middleware
- In-depth analysis and performance optimizations in existing data-handling
middleware, focusing on indexing/fast storing or retrieval between compute
and storage nodes
- Highly scalable middleware optimized for minimum communication
- Use cases and experience for popular Big Data middleware
- Middleware security, privacy and trust architectures

DATES

Papers:
Rolling abstract submission
May 31, 2013 - Full paper submission
July 8, 2013 - Acceptance notification
October 3, 2013 - Camera-ready version due

Lightning Talks:
June 28, 2013 - Deadline for lightning talk abstracts
July 15, 2013 - Lightning talk notification

August 27, 2013 - Workshop Date


TPC

CHAIR

Michael Alexander (chair), TU Wien, Austria
Anastassios Nanos (co-chair), NTUA, Greece
Jie Tao (co-chair), Karlsruhe Institut of Technology, Germany
Lizhe Wang (co-chair), Chinese Academy of Sciences, China
Gianluigi Zanetti (co-chair), CRS4, Italy

PROGRAM COMMITTEE

Amitanand Aiyer, Facebook, USA
Costas Bekas, IBM, Switzerland
Jakob Blomer, CERN, Switzerland
William Gardner, University of Guelph, Canada
Jos? Gracia, HPC Center of the University of Stuttgart, Germany
Zhenghua Guom,  Indiana University, USA
Marcus Hardt,  Karlsruhe Institute of Technology, Germany
Sverre Jarp, CERN, Switzerland
Christopher Jung,  Karlsruhe Institute of Technology, Germany
Andreas Kn?pfer - Technische Universit?t Dresden, Germany
Nectarios Koziris, National Technical University of Athens, Greece
Yan Ma, Chinese Academy of Sciences, China
Martin Schulz - Lawrence Livermore National Laboratory
Viral Shah, MIT Julia Group, USA
Dimitrios Tsoumakos, Ionian University, Greece
Zhifeng Yun, Louisiana State University, USA


PAPER PUBLICATION

Accepted full papers will be published in the Springer LNCS series.

The best papers of the workshop -- after extension and revision -- will be
published in a Special Issue of the Springer Journal of Scalable Computing.


PAPER SUBMISSION

Papers submitted to the workshop will be reviewed by at least two
members of the program committee and external reviewers. Submissions
should include abstract, key words, the e-mail address of the
corresponding author, and must not exceed 10 pages, including tables
and figures at a main font size no smaller than 11 point. Submission
of a paper should be regarded as a commitment that, should the paper
be accepted, at least one of the authors will register and attend the
conference to present the work.

The format must be according to the Springer LNCS Style. Initial
submissions are in PDF; authors of accepted papers will be requested
to provide source files.


Format Guidelines:
http://www.springer.de/comp/lncs/authors.html

Style template:
ftp://ftp.springer.de/pub/tex/latex/llncs/latex2e/llncs2e.zip

Abstract Registration - Submission Link:
http://edas.info/newPaper.php?c=14763


LIGHTNING TALKS

Talks are strictly limited to 5 minutes. They can be used to gain early
feedback on ongoing research, for demonstrations, to present research
results, early research ideas, perspectives and positions of interest to
the community. Lightning talks should spark discussion with presenters
making themselves available following the lightning talk track.


DURATION: Workshop Duration is one day.


GENERAL INFORMATION

The workshop will be held as part of Euro-Par 2013.

Euro-Par 2013: http://www.europar2013.org


From sudheer.joseph at yahoo.com  Tue May  7 22:13:25 2013
From: sudheer.joseph at yahoo.com (Sudheer Joseph)
Date: Wed, 8 May 2013 10:13:25 +0800 (SGT)
Subject: [Numpy-discussion] printing array in tabular form
In-Reply-To: <086DEFA5-2C1B-41F2-934F-4EDBE249C3D3@astro.physik.uni-goettingen.de>
References: <1367918086.35474.YahooMailNeo@web193404.mail.sg3.yahoo.com>
	<086DEFA5-2C1B-41F2-934F-4EDBE249C3D3@astro.physik.uni-goettingen.de>
Message-ID: <1367979205.1211.YahooMailNeo@web193402.mail.sg3.yahoo.com>

Thank you Derek,
? ? ? ? ? ? ? ? ? ?However I get below error. Please tell me if any thing I am missing.

file "read_reg_grd.py", line 22, in <module>
? ? np.savetxt("file.txt", IL.reshape(-1,5), fmt='%5d', delimiter=',')
AttributeError: 'list' object has no attribute 'reshape'

with best regards,
Sudheer?
***************************************************************
Sudheer Joseph 
Indian National Centre for Ocean Information Services
Ministry of Earth Sciences, Govt. of India
POST BOX NO: 21, IDA Jeedeemetla P.O.
Via Pragathi Nagar,Kukatpally, Hyderabad; Pin:5000 55
Tel:+91-40-23886047(O),Fax:+91-40-23895011(O),
Tel:+91-40-23044600(R),Tel:+91-40-9440832534(Mobile)
E-mail:sjo.India at gmail.com;sudheer.joseph at yahoo.com
Web- http://oppamthadathil.tripod.com
***************************************************************


>________________________________
> From: Derek Homeier <derek at astro.physik.uni-goettingen.de>
>To: Discussion of Numerical Python <numpy-discussion at scipy.org> 
>Sent: Tuesday, 7 May 2013 6:41 PM
>Subject: Re: [Numpy-discussion] printing array in tabular form
> 
>
>Dear Sudheer,
>
>On 07.05.2013, at 11:14AM, Sudheer Joseph <sudheer.joseph at yahoo.com> wrote:
>
>>? ? ? ? ? ?  I need to print few arrays in a tabular form for example below array IL has 25 elements, is there an easy way to print this as 5x5 comma separated table? in python
>> 
>> IL=[]
>> for i in np.arange(1,bno+1):
>>? ? ? ? IL.append(i)
>> print(IL)
>> 
>assuming you want this table printed to a file, savetxt does just what you need. In brief for your case,
>
>np.savetxt("file.txt", IL.reshape(-1,5), fmt='%5d', delimiter=',')
>
>should print it in the requested form; you can refer to the save txt documentation for further options.
>
>HTH,
>??? ??? ??? ??? ??? ??? Derek
>
>_______________________________________________
>NumPy-Discussion mailing list
>NumPy-Discussion at scipy.org
>http://mail.scipy.org/mailman/listinfo/numpy-discussion
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20130508/d9fff5ab/attachment.html>

From heng at cantab.net  Wed May  8 02:53:45 2013
From: heng at cantab.net (Henry Gomersall)
Date: Wed, 08 May 2013 07:53:45 +0100
Subject: [Numpy-discussion] printing array in tabular form
In-Reply-To: <1367979205.1211.YahooMailNeo@web193402.mail.sg3.yahoo.com>
References: <1367918086.35474.YahooMailNeo@web193404.mail.sg3.yahoo.com>
	<086DEFA5-2C1B-41F2-934F-4EDBE249C3D3@astro.physik.uni-goettingen.de>
	<1367979205.1211.YahooMailNeo@web193402.mail.sg3.yahoo.com>
Message-ID: <1367996025.32622.4.camel@farnsworth>

On Wed, 2013-05-08 at 10:13 +0800, Sudheer Joseph wrote:
> However I get below error. Please tell me if any thing I am missing.
> 
> 
> file "read_reg_grd.py", line 22, in <module>
>     np.savetxt("file.txt", IL.reshape(-1,5), fmt='%5d', delimiter=',')
> AttributeError: 'list' object has no attribute 'reshape'

IL is a list, not a numpy array. You can either convert the list to an
array after you've filled it, using np.array(IL), or you can
pre-allocate the array and fill it directly in the loop.

Cheers,

Henry



From pierre.raybaut at gmail.com  Wed May  8 11:15:20 2013
From: pierre.raybaut at gmail.com (Pierre Raybaut)
Date: Wed, 8 May 2013 17:15:20 +0200
Subject: [Numpy-discussion] ANN: Spyder v2.2
Message-ID: <CAKegKuBeX-qbLAyzokOw3vF9jyVeraY=d5nde2oZPQBU5VduLQ@mail.gmail.com>

Hi all,

On the behalf of Spyder's development team (
http://code.google.com/p/spyderlib/people/list), I'm pleased to announce
that Spyder v2.2 has been released and is available for Windows
XP/Vista/7/8, GNU/Linux and MacOS X: http://code.google.com/p/spyderlib/.

This release represents 18 months of development since v2.1 and introduces
major enhancements and new features:
 * Full support for IPython v0.13, including the ability to attach to
existing kernels
 * New MacOS X application
 * Much improved debugging experience
 * Various editor improvements for code completion, zooming, auto
insertion, and syntax highlighting
 * Better looking and faster Object Inspector
 * Single instance mode
 * Spanish tranlation of the interface
 * And many other changes: http://code.google.com/p/spyderlib/wiki/ChangeLog

This is the last release to support Python 2.5:
 * Spyder 2.2 supports Python 2.5 to 2.7
 * Spyder 2.3 will support Python 2.7 and Python 3
 * (Spyder 2.1.14dev4 is a development release which already supports
Python 3)
See also https://code.google.com/p/spyderlib/downloads/list.

Spyder is a free, open-source (MIT license) interactive development
environment for the Python language with advanced editing, interactive
testing, debugging and introspection features. Originally designed to
provide MATLAB-like features (integrated help, interactive console,
variable explorer with GUI-based editors for dictionaries, NumPy arrays,
...), it is strongly oriented towards scientific computing and software
development. Thanks to the `spyderlib` library, Spyder also provides
powerful ready-to-use widgets: embedded Python console (example:
http://packages.python.org/guiqwt/_images/sift3.png), NumPy array editor
(example: http://packages.python.org/guiqwt/_images/sift2.png), dictionary
editor, source code editor, etc.

Description of key features with tasty screenshots can be found at:
http://code.google.com/p/spyderlib/wiki/Features

Don't forget to follow Spyder updates/news:
  * on the project website: http://code.google.com/p/spyderlib/
  * and on our official blog: http://spyder-ide.blogspot.com/

Last, but not least, we welcome any contribution that helps making Spyder
an efficient scientific development/computing environment. Join us to help
creating your favourite environment!
(http://code.google.com/p/spyderlib/wiki/NoteForContributors)

Enjoy!
-Pierre
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20130508/a2dfbdc1/attachment.html>

From pierre.raybaut at gmail.com  Wed May  8 11:26:50 2013
From: pierre.raybaut at gmail.com (Pierre Raybaut)
Date: Wed, 8 May 2013 17:26:50 +0200
Subject: [Numpy-discussion] ANN: WinPython 2.7.4.1 and 3.3.1.1, 32 and 64bit
Message-ID: <CAKegKuC63AOuyJwTi1rBcKT7_YJ0Kio8Nsc1jD+HbEZ4Kn-4yQ@mail.gmail.com>

Hi all,

I am pleased to announce that four new versions of WinPython have been
released yesterday with Python 2.7.4 and 3.3.1, 32 and 64 bits. Many
packages have been added or upgraded (see the automatically-generated
changelogs):
  * WinPython 2.7.4.1 32bit, 64bit -- including new Spyder v2.2:
https://code.google.com/p/winpython/wiki/ChangeLog_27
  * WinPython 3.3.1.1 32bit, 64bit:
https://code.google.com/p/winpython/wiki/ChangeLog_33

Special thanks to Christoph Gohlke for building most of the binary packages
bundled in WinPython.

WinPython is a free open-source portable distribution of Python
for Windows, designed for scientists.

It is a full-featured (see
http://code.google.com/p/winpython/wiki/PackageIndex)
Python-based scientific environment:
  * Designed for scientists (thanks to the integrated libraries
NumPy, SciPy, Matplotlib, guiqwt, etc.:
    * Regular *scientific users*: interactive data processing
and visualization using Python with Spyder
    * *Advanced scientific users and software developers*:
Python applications development with Spyder, version control with
Mercurial and other development tools (like gettext)
  * *Portable*: preconfigured, it should run out of the box on any machine
under Windows (without any installation requirements) and the folder
containing WinPython can be moved to any location (local, network or
removable drive)
  * *Flexible*: one can install (or should I write "use" as it's portable)
as many WinPython versions as necessary (like isolated and self-consistent
environments), even if those versions are running different versions of
Python (2.7, 3.3) or different architectures (32bit or 64bit) on the same
machine
  * *Customizable*: using the integrated package manager (wppm,
as WinPython Package Manager), it's possible to install, uninstall
or upgrade Python packages (see
http://code.google.com/p/winpython/wiki/WPPM for
more details on supported package formats).

*WinPython is not an attempt to replace Python(x,y)*, this is
just something different (see
http://code.google.com/p/winpython/wiki/Roadmap): more flexible, easier to
maintain, movable and less invasive for the OS, but certainly less
user-friendly, with less packages/contents and without any integration to
Windows explorer [*].

[*] Actually there is an optional integration into Windows
explorer, providing the same features as the official Python installer
regarding file associations and context menu entry (this option may be
activated through the WinPython Control Panel), and adding shortcuts to
Windows Start menu.

Enjoy!
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20130508/f451bb78/attachment.html>

From sylvain.corlay at gmail.com  Wed May  8 11:52:35 2013
From: sylvain.corlay at gmail.com (Sylvain Corlay)
Date: Wed, 8 May 2013 08:52:35 -0700 (PDT)
Subject: [Numpy-discussion] ANN: Spyder v2.2
In-Reply-To: <CAKegKuBeX-qbLAyzokOw3vF9jyVeraY=d5nde2oZPQBU5VduLQ@mail.gmail.com>
References: <CAKegKuBeX-qbLAyzokOw3vF9jyVeraY=d5nde2oZPQBU5VduLQ@mail.gmail.com>
Message-ID: <6865fbef-4be3-4f83-9e1c-3747369eea01@googlegroups.com>

Congratulations to Carlos Jed and Pierre for this new release. 

Best, 

Sylvain

On Wednesday, May 8, 2013 11:15:20 AM UTC-4, Pierre Raybaut wrote:
>
> Hi all,
>
> On the behalf of Spyder's development team (
> http://code.google.com/p/spyderlib/people/list), I'm pleased to announce 
> that Spyder v2.2 has been released and is available for Windows 
> XP/Vista/7/8, GNU/Linux and MacOS X: http://code.google.com/p/spyderlib/.
>
> This release represents 18 months of development since v2.1 and introduces 
> major enhancements and new features:
>  * Full support for IPython v0.13, including the ability to attach to 
> existing kernels
>  * New MacOS X application
>  * Much improved debugging experience
>  * Various editor improvements for code completion, zooming, auto 
> insertion, and syntax highlighting
>  * Better looking and faster Object Inspector
>  * Single instance mode
>  * Spanish tranlation of the interface
>  * And many other changes: 
> http://code.google.com/p/spyderlib/wiki/ChangeLog
>
> This is the last release to support Python 2.5:
>  * Spyder 2.2 supports Python 2.5 to 2.7
>  * Spyder 2.3 will support Python 2.7 and Python 3
>  * (Spyder 2.1.14dev4 is a development release which already supports 
> Python 3)
> See also https://code.google.com/p/spyderlib/downloads/list.
>
> Spyder is a free, open-source (MIT license) interactive development 
> environment for the Python language with advanced editing, interactive 
> testing, debugging and introspection features. Originally designed to 
> provide MATLAB-like features (integrated help, interactive console, 
> variable explorer with GUI-based editors for dictionaries, NumPy arrays, 
> ...), it is strongly oriented towards scientific computing and software 
> development. Thanks to the `spyderlib` library, Spyder also provides 
> powerful ready-to-use widgets: embedded Python console (example: 
> http://packages.python.org/guiqwt/_images/sift3.png), NumPy array editor 
> (example: http://packages.python.org/guiqwt/_images/sift2.png), 
> dictionary editor, source code editor, etc.
>
> Description of key features with tasty screenshots can be found at:
> http://code.google.com/p/spyderlib/wiki/Features
>
> Don't forget to follow Spyder updates/news:
>   * on the project website: http://code.google.com/p/spyderlib/
>   * and on our official blog: http://spyder-ide.blogspot.com/
>
> Last, but not least, we welcome any contribution that helps making Spyder 
> an efficient scientific development/computing environment. Join us to help 
> creating your favourite environment!
> (http://code.google.com/p/spyderlib/wiki/NoteForContributors)
>
> Enjoy!
> -Pierre
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20130508/72970b0a/attachment.html>

From sylvain.corlay at gmail.com  Wed May  8 12:13:12 2013
From: sylvain.corlay at gmail.com (Sylvain Corlay)
Date: Wed, 8 May 2013 12:13:12 -0400
Subject: [Numpy-discussion] =?windows-1252?q?Consistency_of_numpy=92s_beha?=
	=?windows-1252?q?vior_with_arrays_of_dimension_0?=
Message-ID: <CAK=Phk5h3tm4_nesaJOFw6SLJo7GSwX7+ziXqk2tWZfPyr0Ndg@mail.gmail.com>

Hello everyone,

0d arrays are very convenient because they allow us to write functions that
are valid for both arrays and scalar-like argument, thanks to Boolean
indexing.

However, most special functions in numpy (and scipy) and most Boolean
operators, when called on 0d arrays, return scalars rather than 0d arrays,
which is quite annoying.

For example, numpy.exp called on a 0d array containing a float number
returns a float, rather than a 0d array, and if x is a 0d array, x > 0
returns a Boolean, rather than a 0d array containing a Boolean.

What I would expect is the following.

-If x is a 0d array containing float, I expect numpy.exp(x) to return a 0d
array, and x>0 to return a 0d array containing a Boolean.

-If x is a scalar, numpy.exp(x) returns the expected scalar type, and x>0
returns the expected Boolean.

Here is an example of a simple function that suffers from this issue (a
corrected version is proposed later)
import numpy
from scipy.stats import norm

def normal_time_value(sig, m, strikes):
    """
    The three arguments are array-like and have the same shape.
    Consider a random variable G ~ N (m , sig^2)
    The function returns          E[(G-K)+] - (E[G]-K)_+
    which is also equal to         E[(K-G)+] - (K-E[G])_+
    """
    sig = numpy.array(sig)
    strikes = numpy.array(strikes)
    m = numpy.array(m)
    tv = numpy.zeros(strikes.shape)
    tv[sig < 0] = numpy.nan         # sig must be nonnegative
    non0 = sig > 0.0
    dev = numpy.where(non0, (m - strikes) / sig, numpy.nan)
    tv[non0] = numpy.where(strikes > m, (m - strikes) * norm.cdf(dev) + sig
* norm.pdf(dev), (strikes - m) * norm.cdf(-dev) + sig * norm.pdf(dev))[non0]
    return tv

This function does not work with scalars or 0d arrays. To make it work, we
need to modify it in the following fashion: reconvert intermediate results
to 0d array to take advantage of the Boolean indexing.

import numpy
from scipy.stats import norm

def normal_time_value(sig, m, strikes):
    """
    The three arguments are array-like and have the same shape.
    Consider a random variable G ~ N (m , sig^2)
    The function returns          E[(G-K)+] - (E[G]-K)_+
    which is also equal to        E[(K-G)+] - (K-E[G])_+
    """
    sig = numpy.array(sig)
    strikes = numpy.array(strikes)
    m = numpy.array(m)
    tv = numpy.zeros(strikes.shape)
    tv[numpy.array(sig < 0)] = numpy.nan         # sig must be nonnegative
    non0 = numpy.array(sig > 0.0)
    dev = numpy.where(non0, (m - strikes) / sig, numpy.nan)
    tv[non0] = numpy.where(numpy.array(strikes > m), (m - strikes) *
norm.cdf(dev) + sig * norm.pdf(dev), (strikes - m) * norm.cdf(-dev) + sig *
norm.pdf(dev))[non0]
    return tv

This problem also affects functions like logical_and, logical_or and
logical_not, which all return numpy.bool_ type rather than 0d array of
dtype bool.
Travis Oliphant opened a ticket in track regarding the same issue some time
ago:  http://projects.scipy.org/numpy/ticket/823 It was transfered to
github https://github.com/numpy/numpy/issues/1421.

Best,

Sylvain
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20130508/9c98d86b/attachment.html>

From Andrew.MACKEITH at 3ds.com  Wed May  8 13:09:42 2013
From: Andrew.MACKEITH at 3ds.com (MACKEITH Andrew)
Date: Wed, 8 May 2013 17:09:42 +0000
Subject: [Numpy-discussion] References to Numerical Recipes in Numpy code
Message-ID: <2922D4459F182B41ABD8C7AE5010642094BB078B@AG-DCC-MBX13.dsone.3ds.com>

The numpy libraries (numpy/core/machar.py, numpy/fft/info.py, numpy/lib/function_base.py) contain references to "Numerical Recipes: The Art of Scientific Computing".  The book contains a licensing section which states "... Unlicensed transfer of Numerical Recipes programs to any other format is strictly prohibited".  From this we understand that use of the book as a reference is permitted, but copying of code for distribution, whether non-commercially or otherwise is not permitted.   We are wondering if anyone has done an analysis of this situation around numpy, and if so what conclusion they might have reached?


Andrew MacKeith
Python and Scripting Development
Dassault Systemes Simulia Corp.


This email and any attachments are intended solely for the use of the individual or entity to whom it is addressed and may be confidential and/or privileged.

If you are not one of the named recipients or have received this email in error,

(i) you should not read, disclose, or copy it,

(ii) please notify sender of your receipt by reply email and delete this email and all attachments,

(iii) Dassault Systemes does not accept or assume any liability or responsibility for any use of or reliance on this email.

For other languages, go to http://www.3ds.com/terms/email-disclaimer
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20130508/064b0ca0/attachment.html>

From robert.kern at gmail.com  Wed May  8 13:18:48 2013
From: robert.kern at gmail.com (Robert Kern)
Date: Wed, 8 May 2013 18:18:48 +0100
Subject: [Numpy-discussion] References to Numerical Recipes in Numpy code
In-Reply-To: <2922D4459F182B41ABD8C7AE5010642094BB078B@AG-DCC-MBX13.dsone.3ds.com>
References: <2922D4459F182B41ABD8C7AE5010642094BB078B@AG-DCC-MBX13.dsone.3ds.com>
Message-ID: <CAF6FJisn+85rP7vT-HnO9P+y+oiwXQqZ7QMiN3zX33Cb_i4S6w@mail.gmail.com>

On Wed, May 8, 2013 at 6:09 PM, MACKEITH Andrew <Andrew.MACKEITH at 3ds.com> wrote:
> The numpy libraries (numpy/core/machar.py, numpy/fft/info.py,
> numpy/lib/function_base.py) contain references to "Numerical Recipes: The
> Art of Scientific Computing".  The book contains a licensing section which
> states "... Unlicensed transfer of Numerical Recipes programs to any other
> format is strictly prohibited".  From this we understand that use of the
> book as a reference is permitted, but copying of code for distribution,
> whether non-commercially or otherwise is not permitted.   We are wondering
> if anyone has done an analysis of this situation around numpy, and if so
> what conclusion they might have reached?

These are references to the book's text for the discussion of the
algorithms and concepts in general, not copies of the code from the
attached disk.

--
Robert Kern


From bin.arthur at gmail.com  Wed May  8 13:47:52 2013
From: bin.arthur at gmail.com (Bin Huang)
Date: Wed, 8 May 2013 11:47:52 -0600
Subject: [Numpy-discussion] Performance degradation when installing Numpy
 and Python not from packaging system?
Message-ID: <CAJkdMQaBso-3qs5EUg+og6oP_oEn3FkuO0VGYDnKAppdtG7LUw@mail.gmail.com>

Hi list,

I have observed slowdown if Numpy and Python are installed manually (i.e.
not from packaging system). The slowdown can be significant when problem
size is large. I wish someone in this mailing list can give me some hints
on this issue. I also post performance results and my source code here.

My platform is a 12-core dual-socket SMP machine running Ubuntu (Linux
kernel 3.2.0). On this platform, I have two sets of installations. One is
pre-installed Python 2.7.3 and Numpy 1.6.1 installed with 'apt-get'. The
other is Python 2.7.3 and Numpy 1.6.1 both compiled freshly from official
tarball release by myself.

In order to compare performance, I wrote a piece of code that simply
performs matrix-matrix multiplication using one of Numpy's functions. The
source code is listed as follows:

*if __name__ == "__main__":
    import numpy, random, time
    import sys

    size = int(sys.argv[1]) # For simplicity, we only test square matrix

    matrix_a = numpy.matrix(numpy.random.randn(size,size))
    matrix_b = numpy.matrix(numpy.random.randn(size,size))

    start_time = time.time()
    result = numpy.dot(matrix_a, matrix_b)
    print '%0.3f ms' % ((time.time() - start_time)*1000.0)*

I ran this code with both pre-installed Python and compiled Python. Here is
performance numbers I got:

Problem size    Pre-installed Python       Compiled Python/Numpy
 Slowdown
256x256           14.165 ms                     44.538 ms
           ~3X
512x512           109.330 ms                   347.143 ms
          ~3X
1024x1024       867.936 ms                    8975.016 ms
        ~10X
2048x2048       7755.084 ms                  84713.997 ms
      ~10X


Here are commands I used to compile Python and Numpy:

a) ~/python2.7.3$ ./configure --prefix=MY_LOCAL_DIR
b) ~/python2.7.3$ make
c) ~/python2.7.3$ make install
e) ~/numpy-1.6.1$ MY_LOCAL_DIR/bin/python setup.py build
f) ~/numpy-1.6.1$ MY_LOCAL_DIR/bin/python setup.py install

Any ideas? Thanks in advance.

Bin Huang
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20130508/4e26a972/attachment.html>

From pav at iki.fi  Wed May  8 14:13:27 2013
From: pav at iki.fi (Pauli Virtanen)
Date: Wed, 08 May 2013 21:13:27 +0300
Subject: [Numpy-discussion] Performance degradation when installing
 Numpy and Python not from packaging system?
In-Reply-To: <CAJkdMQaBso-3qs5EUg+og6oP_oEn3FkuO0VGYDnKAppdtG7LUw@mail.gmail.com>
References: <CAJkdMQaBso-3qs5EUg+og6oP_oEn3FkuO0VGYDnKAppdtG7LUw@mail.gmail.com>
Message-ID: <kme4k3$l3l$1@ger.gmane.org>

Hi,

08.05.2013 20:47, Bin Huang kirjoitti:
[clip]
> In order to compare performance, I wrote a piece of code that simply
> performs matrix-matrix multiplication using one of Numpy's functions.
> The source code is listed as follows:
[clip]

Your benchmark essentially measures the speed of the BLAS library Numpy
is linked with. It appears that the one supplied by the system is linked
with a high-performance BLAS, whereas the one you compiled yourself is not.

You should look at the output of the build command, and look for lines
such as

-------8<-------------
lapack_opt_info:
lapack_mkl_info:
mkl_info:
  libraries mkl,vml,guide not found in ['/usr/local/lib', '/usr/lib',
'/usr/lib/x86_64-linux-gnu']
  NOT AVAILABLE

  NOT AVAILABLE

atlas_threads_info:
Setting PTATLAS=ATLAS
  libraries ptf77blas,ptcblas,atlas not found in /usr/local/lib
  libraries lapack_atlas not found in /usr/local/lib
<class 'numpy.distutils.system_info.atlas_threads_info'>
Setting PTATLAS=ATLAS
Setting PTATLAS=ATLAS
  FOUND:
    language = f77
    include_dirs = ['/usr/include/atlas']
    library_dirs = ['/usr/lib/atlas-base/atlas', '/usr/lib/atlas-base']
    libraries = ['lapack', 'ptf77blas', 'ptcblas', 'atlas']
    define_macros = [('ATLAS_INFO', '"\\"3.8.4\\""')]

  FOUND:
    language = f77
    include_dirs = ['/usr/include/atlas']
    library_dirs = ['/usr/lib/atlas-base/atlas', '/usr/lib/atlas-base']
    libraries = ['lapack', 'ptf77blas', 'ptcblas', 'atlas']
    define_macros = [('ATLAS_INFO', '"\\"3.8.4\\""')]
-------8<-------------

If it says "NOT AVAILABLE" to all the cases, then it does not find a
good BLAS/LAPACK library, and uses one that is not optimized for
specific processors.

See here on how to tell it to use a specific BLAS/LAPACK combination:
http://new.scipy.org/scipylib/building/linux.html#generic-instructions

-- 
Pauli Virtanen



From sudheer.joseph at yahoo.com  Thu May  9 04:06:48 2013
From: sudheer.joseph at yahoo.com (Sudheer Joseph)
Date: Thu, 9 May 2013 16:06:48 +0800 (SGT)
Subject: [Numpy-discussion] printing array in tabular form
In-Reply-To: <1367996025.32622.4.camel@farnsworth>
References: <1367918086.35474.YahooMailNeo@web193404.mail.sg3.yahoo.com>
	<086DEFA5-2C1B-41F2-934F-4EDBE249C3D3@astro.physik.uni-goettingen.de>
	<1367979205.1211.YahooMailNeo@web193402.mail.sg3.yahoo.com>
	<1367996025.32622.4.camel@farnsworth>
Message-ID: <1368086808.92617.YahooMailNeo@web193403.mail.sg3.yahoo.com>

Thank you Gomersall,
????????? However writing a formatted out put looks to be bit tricky with python relative to other programing languages.
For example,
???? If I wanted to print below text in a file (for reading by another program), it looks to be not an easy job....Hope new developments will come and a userfriendly formatted out put method for pyton will evolve.


IL= 1,2,3,4,5
??? 5,5,6,8,0

JL= 1,2,3,4,5
??? 5,5,6,8,0 

KL= 1,2,3,4,5
??? 5,5,6,8,0

with best regards,
Sudheer


?
***************************************************************
Sudheer Joseph 
Indian National Centre for Ocean Information Services
Ministry of Earth Sciences, Govt. of India
POST BOX NO: 21, IDA Jeedeemetla P.O.
Via Pragathi Nagar,Kukatpally, Hyderabad; Pin:5000 55
Tel:+91-40-23886047(O),Fax:+91-40-23895011(O),
Tel:+91-40-23044600(R),Tel:+91-40-9440832534(Mobile)
E-mail:sjo.India at gmail.com;sudheer.joseph at yahoo.com
Web- http://oppamthadathil.tripod.com
***************************************************************


>________________________________
> From: Henry Gomersall <heng at cantab.net>
>To: Discussion of Numerical Python <numpy-discussion at scipy.org> 
>Sent: Wednesday, 8 May 2013 12:23 PM
>Subject: Re: [Numpy-discussion] printing array in tabular form
> 
>
>On Wed, 2013-05-08 at 10:13 +0800, Sudheer Joseph wrote:
>> However I get below error. Please tell me if any thing I am missing.
>> 
>> 
>> file "read_reg_grd.py", line 22, in <module>
>>? ?  np.savetxt("file.txt", IL.reshape(-1,5), fmt='%5d', delimiter=',')
>> AttributeError: 'list' object has no attribute 'reshape'
>
>IL is a list, not a numpy array. You can either convert the list to an
>array after you've filled it, using np.array(IL), or you can
>pre-allocate the array and fill it directly in the loop.
>
>Cheers,
>
>Henry
>
>_______________________________________________
>NumPy-Discussion mailing list
>NumPy-Discussion at scipy.org
>http://mail.scipy.org/mailman/listinfo/numpy-discussion
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20130509/01a099f9/attachment.html>

From davidmenhur at gmail.com  Thu May  9 04:59:37 2013
From: davidmenhur at gmail.com (=?UTF-8?B?RGHPgGlk?=)
Date: Thu, 9 May 2013 10:59:37 +0200
Subject: [Numpy-discussion] printing array in tabular form
In-Reply-To: <1368086808.92617.YahooMailNeo@web193403.mail.sg3.yahoo.com>
References: <1367918086.35474.YahooMailNeo@web193404.mail.sg3.yahoo.com>
	<086DEFA5-2C1B-41F2-934F-4EDBE249C3D3@astro.physik.uni-goettingen.de>
	<1367979205.1211.YahooMailNeo@web193402.mail.sg3.yahoo.com>
	<1367996025.32622.4.camel@farnsworth>
	<1368086808.92617.YahooMailNeo@web193403.mail.sg3.yahoo.com>
Message-ID: <CAJhcF=1SpFUcjYzkWk5=98RBjvSq7iZRJKB4jckqOVv6q6GOfA@mail.gmail.com>

On 9 May 2013 10:06, Sudheer Joseph <sudheer.joseph at yahoo.com> wrote:
> However writing a formatted out put looks to be bit tricky with python
> relative to other programing languages.

If performance is not an issue, you could do it by hand, as you can
always do in any programming language:


savefile = open('data.txt', 'w')
N = len(IL)

for start in xrange(N/5):
   if start+5 > N:
     end = N
   else:
     end = start+5
   print >> savefile, IL[start : end]


But this is actually more verbose, and once you get into NumPy
workflow, it is actually simple.


From heng at cantab.net  Thu May  9 05:21:29 2013
From: heng at cantab.net (Henry Gomersall)
Date: Thu, 09 May 2013 10:21:29 +0100
Subject: [Numpy-discussion] printing array in tabular form
In-Reply-To: <1368086808.92617.YahooMailNeo@web193403.mail.sg3.yahoo.com>
References: <1367918086.35474.YahooMailNeo@web193404.mail.sg3.yahoo.com>
	<086DEFA5-2C1B-41F2-934F-4EDBE249C3D3@astro.physik.uni-goettingen.de>
	<1367979205.1211.YahooMailNeo@web193402.mail.sg3.yahoo.com>
	<1367996025.32622.4.camel@farnsworth>
	<1368086808.92617.YahooMailNeo@web193403.mail.sg3.yahoo.com>
Message-ID: <1368091289.31407.3.camel@farnsworth>

On Thu, 2013-05-09 at 16:06 +0800, Sudheer Joseph wrote:
> If I wanted to print below text in a file (for reading by another
> program), it looks to be not an easy job....Hope new developments will
> come and a userfriendly formatted out put method for pyton will
> evolve.

I don't understand what the problem is - this seems to be trivial to
solve. You gave an example in Fortran; Is the transliteration to Python
not acceptable?

Is the output format specified by the receiving program? If not, there
are loads of options for creating interoperable text files.

Henry



From robert.kern at gmail.com  Thu May  9 06:21:30 2013
From: robert.kern at gmail.com (Robert Kern)
Date: Thu, 9 May 2013 11:21:30 +0100
Subject: [Numpy-discussion] Integer type casting and OverflowError
Message-ID: <CAF6FJisMKn2OL9CVPKNfqFcaywjtUFXGPwib9wRtaOSczJH6Yw@mail.gmail.com>

With master numpy (and back to 1.6.1, at least):

[~]
|1> np.int32(3054212286)
-1240755010


It seems like at one time, this used to raise an OverflowError. We can
see this in at least one place in scipy:

https://github.com/scipy/scipy/blob/master/scipy/interpolate/fitpack.py#L912

Is this a regression in numpy, or should the code in scipy be fixed to
use a different test? Even if it is technically a regression, it might
be one of those "it's been broken so long, it's a feature now"
regressions.

--
Robert Kern


From scott.sinclair.za at gmail.com  Thu May  9 06:38:41 2013
From: scott.sinclair.za at gmail.com (Scott Sinclair)
Date: Thu, 9 May 2013 12:38:41 +0200
Subject: [Numpy-discussion] Integer type casting and OverflowError
In-Reply-To: <CAF6FJisMKn2OL9CVPKNfqFcaywjtUFXGPwib9wRtaOSczJH6Yw@mail.gmail.com>
References: <CAF6FJisMKn2OL9CVPKNfqFcaywjtUFXGPwib9wRtaOSczJH6Yw@mail.gmail.com>
Message-ID: <CA+nsYDtdM=gwZi2Os_=CRrBo2uSv7e_m62yuCi=JUenf16Ra9Q@mail.gmail.com>

On 9 May 2013 12:21, Robert Kern <robert.kern at gmail.com> wrote:
> With master numpy (and back to 1.6.1, at least):
>
> [~]
> |1> np.int32(3054212286)
> -1240755010
>
>
> It seems like at one time, this used to raise an OverflowError. We can
> see this in at least one place in scipy:
>
> https://github.com/scipy/scipy/blob/master/scipy/interpolate/fitpack.py#L912

No doubt I'm missing something, but isn't the OverflowError raised
here https://github.com/scipy/scipy/blob/master/scipy/interpolate/fitpack.py#L40
and not in Numpy?

Cheers,
Scott


From robert.kern at gmail.com  Thu May  9 06:45:19 2013
From: robert.kern at gmail.com (Robert Kern)
Date: Thu, 9 May 2013 11:45:19 +0100
Subject: [Numpy-discussion] Integer type casting and OverflowError
In-Reply-To: <CA+nsYDtdM=gwZi2Os_=CRrBo2uSv7e_m62yuCi=JUenf16Ra9Q@mail.gmail.com>
References: <CAF6FJisMKn2OL9CVPKNfqFcaywjtUFXGPwib9wRtaOSczJH6Yw@mail.gmail.com>
	<CA+nsYDtdM=gwZi2Os_=CRrBo2uSv7e_m62yuCi=JUenf16Ra9Q@mail.gmail.com>
Message-ID: <CAF6FJiu3adJiLQHWmguMg9s+zz1o7t3=Rshw0+TBvuLSCN5-ag@mail.gmail.com>

On Thu, May 9, 2013 at 11:38 AM, Scott Sinclair
<scott.sinclair.za at gmail.com> wrote:
> On 9 May 2013 12:21, Robert Kern <robert.kern at gmail.com> wrote:
>> With master numpy (and back to 1.6.1, at least):
>>
>> [~]
>> |1> np.int32(3054212286)
>> -1240755010
>>
>>
>> It seems like at one time, this used to raise an OverflowError. We can
>> see this in at least one place in scipy:
>>
>> https://github.com/scipy/scipy/blob/master/scipy/interpolate/fitpack.py#L912
>
> No doubt I'm missing something, but isn't the OverflowError raised
> here https://github.com/scipy/scipy/blob/master/scipy/interpolate/fitpack.py#L40
> and not in Numpy?

Heh. I wrote this email before I submitted the PR with that fix. :-)

Here is what the code used to look like:

https://github.com/scipy/scipy/blob/c1625b2cd026553cc9414ad12bb90c74a469e714/scipy/interpolate/fitpack.py#L912

--
Robert Kern


From scott.sinclair.za at gmail.com  Thu May  9 06:48:27 2013
From: scott.sinclair.za at gmail.com (Scott Sinclair)
Date: Thu, 9 May 2013 12:48:27 +0200
Subject: [Numpy-discussion] Integer type casting and OverflowError
In-Reply-To: <CAF6FJiu3adJiLQHWmguMg9s+zz1o7t3=Rshw0+TBvuLSCN5-ag@mail.gmail.com>
References: <CAF6FJisMKn2OL9CVPKNfqFcaywjtUFXGPwib9wRtaOSczJH6Yw@mail.gmail.com>
	<CA+nsYDtdM=gwZi2Os_=CRrBo2uSv7e_m62yuCi=JUenf16Ra9Q@mail.gmail.com>
	<CAF6FJiu3adJiLQHWmguMg9s+zz1o7t3=Rshw0+TBvuLSCN5-ag@mail.gmail.com>
Message-ID: <CA+nsYDvxX5KcjmTGUheTRSrwMAaZYdYTN7YdFC9s6k9=UkdBnw@mail.gmail.com>

On 9 May 2013 12:45, Robert Kern <robert.kern at gmail.com> wrote:
> On Thu, May 9, 2013 at 11:38 AM, Scott Sinclair
> <scott.sinclair.za at gmail.com> wrote:
>> On 9 May 2013 12:21, Robert Kern <robert.kern at gmail.com> wrote:
>>> With master numpy (and back to 1.6.1, at least):
>>>
>>> [~]
>>> |1> np.int32(3054212286)
>>> -1240755010
>>>
>>>
>>> It seems like at one time, this used to raise an OverflowError. We can
>>> see this in at least one place in scipy:
>>>
>>> https://github.com/scipy/scipy/blob/master/scipy/interpolate/fitpack.py#L912
>>
>> No doubt I'm missing something, but isn't the OverflowError raised
>> here https://github.com/scipy/scipy/blob/master/scipy/interpolate/fitpack.py#L40
>> and not in Numpy?
>
> Heh. I wrote this email before I submitted the PR with that fix. :-)

Hah. I should have checked recent commits as well...

Cheers,
Scott


From pav at iki.fi  Thu May  9 06:50:28 2013
From: pav at iki.fi (Pauli Virtanen)
Date: Thu, 09 May 2013 13:50:28 +0300
Subject: [Numpy-discussion] Integer type casting and OverflowError
In-Reply-To: <CAF6FJisMKn2OL9CVPKNfqFcaywjtUFXGPwib9wRtaOSczJH6Yw@mail.gmail.com>
References: <CAF6FJisMKn2OL9CVPKNfqFcaywjtUFXGPwib9wRtaOSczJH6Yw@mail.gmail.com>
Message-ID: <kmfv1f$4c1$1@ger.gmane.org>

09.05.2013 13:21, Robert Kern kirjoitti:
> With master numpy (and back to 1.6.1, at least):
> 
> [~]
> |1> np.int32(3054212286)
> -1240755010
> 
> It seems like at one time, this used to raise an OverflowError. We can
> see this in at least one place in scipy:

It actually still raises an overflowerror if the integer is too big to
fit into int64 (or maybe into int32 on 32-bit systems). The behavior is
the same as in Numpy 1.2.0, so it has perhaps been like this forever.

So most likely, the code in Scipy was correct only for 32-bit systems.

-- 
Pauli Virtanen




From robert.kern at gmail.com  Thu May  9 07:24:37 2013
From: robert.kern at gmail.com (Robert Kern)
Date: Thu, 9 May 2013 12:24:37 +0100
Subject: [Numpy-discussion] Integer type casting and OverflowError
In-Reply-To: <kmfv1f$4c1$1@ger.gmane.org>
References: <CAF6FJisMKn2OL9CVPKNfqFcaywjtUFXGPwib9wRtaOSczJH6Yw@mail.gmail.com>
	<kmfv1f$4c1$1@ger.gmane.org>
Message-ID: <CAF6FJiv=sXd-sNFC8af+u5z7iEHmCezPK8L_GBQHBLim-huirg@mail.gmail.com>

On Thu, May 9, 2013 at 11:50 AM, Pauli Virtanen <pav at iki.fi> wrote:
> 09.05.2013 13:21, Robert Kern kirjoitti:
>> With master numpy (and back to 1.6.1, at least):
>>
>> [~]
>> |1> np.int32(3054212286)
>> -1240755010
>>
>> It seems like at one time, this used to raise an OverflowError. We can
>> see this in at least one place in scipy:
>
> It actually still raises an overflowerror if the integer is too big to
> fit into int64 (or maybe into int32 on 32-bit systems). The behavior is
> the same as in Numpy 1.2.0, so it has perhaps been like this forever.
>
> So most likely, the code in Scipy was correct only for 32-bit systems.

Which does not surprise me in the slightest.

--
Robert Kern


From Tom.KACVINSKY at 3ds.com  Thu May  9 09:10:52 2013
From: Tom.KACVINSKY at 3ds.com (KACVINSKY Tom)
Date: Thu, 9 May 2013 13:10:52 +0000
Subject: [Numpy-discussion] www.scipy.org down?
Message-ID: <B8F5CDE7FC952A4580ABD28F488EF27930DD761D@AG-DCC-MBX11.dsone.3ds.com>

I am looking for a release of numpy for Mac OS, but I can't reach the scipy web server.  IS it down for maintenance?

Thanks,

Tom

This email and any attachments are intended solely for the use of the individual or entity to whom it is addressed and may be confidential and/or privileged.

If you are not one of the named recipients or have received this email in error,

(i) you should not read, disclose, or copy it,

(ii) please notify sender of your receipt by reply email and delete this email and all attachments,

(iii) Dassault Systemes does not accept or assume any liability or responsibility for any use of or reliance on this email.

For other languages, go to http://www.3ds.com/terms/email-disclaimer
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20130509/3c0e8856/attachment.html>

From robert.kern at gmail.com  Thu May  9 09:21:11 2013
From: robert.kern at gmail.com (Robert Kern)
Date: Thu, 9 May 2013 14:21:11 +0100
Subject: [Numpy-discussion] www.scipy.org down?
In-Reply-To: <B8F5CDE7FC952A4580ABD28F488EF27930DD761D@AG-DCC-MBX11.dsone.3ds.com>
References: <B8F5CDE7FC952A4580ABD28F488EF27930DD761D@AG-DCC-MBX11.dsone.3ds.com>
Message-ID: <CAF6FJisR-bJrqyZAbv=xoN-u71h=YO75K5Z6Gbx_WGLStTe-Qg@mail.gmail.com>

On Thu, May 9, 2013 at 2:10 PM, KACVINSKY Tom <Tom.KACVINSKY at 3ds.com> wrote:
> I am looking for a release of numpy for Mac OS, but I can't reach the scipy
> web server.  IS it down for maintenance?

It might have been. It's up now. The PyPI page is the best place to
start looking for downloads:

  https://pypi.python.org/pypi/numpy/

The Mac binaries are hosted on SourceForge:

  http://sourceforge.net/projects/numpy/files/NumPy/

--
Robert Kern


From Tom.KACVINSKY at 3ds.com  Thu May  9 10:25:09 2013
From: Tom.KACVINSKY at 3ds.com (KACVINSKY Tom)
Date: Thu, 9 May 2013 14:25:09 +0000
Subject: [Numpy-discussion] www.scipy.org down?
In-Reply-To: <CAF6FJisR-bJrqyZAbv=xoN-u71h=YO75K5Z6Gbx_WGLStTe-Qg@mail.gmail.com>
References: <B8F5CDE7FC952A4580ABD28F488EF27930DD761D@AG-DCC-MBX11.dsone.3ds.com>
	<CAF6FJisR-bJrqyZAbv=xoN-u71h=YO75K5Z6Gbx_WGLStTe-Qg@mail.gmail.com>
Message-ID: <B8F5CDE7FC952A4580ABD28F488EF27930DD7781@AG-DCC-MBX11.dsone.3ds.com>

Unfortunately, the Mac OS downloads won't work for us as they install into the system Python.  We have a custom built Python (2.7.3) so I compiled from source.  I noticed a few things:

1.  The modules compiled from C source have an extension of .so, not .dylib
2.  I installed nose so I could run the numpy tests, but 0 tests ran.
3.  I configured numpy to use MKL as per the instructions on Intel's site, but the build still used the Accelerate framework provided by Apple.

Given all of this, whom do I speak to regarding numpy builds on Mac OS?

Thanks,

Tom

-----Original Message-----
From: numpy-discussion-bounces at scipy.org [mailto:numpy-discussion-bounces at scipy.org] On Behalf Of Robert Kern
Sent: Thursday, May 09, 2013 9:21 AM
To: Discussion of Numerical Python
Subject: Re: [Numpy-discussion] www.scipy.org down?

On Thu, May 9, 2013 at 2:10 PM, KACVINSKY Tom <Tom.KACVINSKY at 3ds.com> wrote:
> I am looking for a release of numpy for Mac OS, but I can't reach the
> scipy web server.  IS it down for maintenance?

It might have been. It's up now. The PyPI page is the best place to start looking for downloads:

  https://pypi.python.org/pypi/numpy/

The Mac binaries are hosted on SourceForge:

  http://sourceforge.net/projects/numpy/files/NumPy/

--
Robert Kern
_______________________________________________
NumPy-Discussion mailing list
NumPy-Discussion at scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion
This email and any attachments are intended solely for the use of the individual or entity to whom it is addressed and may be confidential and/or privileged.

If you are not one of the named recipients or have received this email in error,

(i) you should not read, disclose, or copy it,

(ii) please notify sender of your receipt by reply email and delete this email and all attachments,

(iii) Dassault Systemes does not accept or assume any liability or responsibility for any use of or reliance on this email.

For other languages, go to http://www.3ds.com/terms/email-disclaimer


From cournape at gmail.com  Thu May  9 10:44:48 2013
From: cournape at gmail.com (David Cournapeau)
Date: Thu, 9 May 2013 15:44:48 +0100
Subject: [Numpy-discussion] www.scipy.org down?
In-Reply-To: <B8F5CDE7FC952A4580ABD28F488EF27930DD7781@AG-DCC-MBX11.dsone.3ds.com>
References: <B8F5CDE7FC952A4580ABD28F488EF27930DD761D@AG-DCC-MBX11.dsone.3ds.com>
	<CAF6FJisR-bJrqyZAbv=xoN-u71h=YO75K5Z6Gbx_WGLStTe-Qg@mail.gmail.com>
	<B8F5CDE7FC952A4580ABD28F488EF27930DD7781@AG-DCC-MBX11.dsone.3ds.com>
Message-ID: <CAGY4rcW_veV_pa7RAf9z7aX72OYnM+FjuP84uFu5DH-uEc76-A@mail.gmail.com>

On Thu, May 9, 2013 at 3:25 PM, KACVINSKY Tom <Tom.KACVINSKY at 3ds.com> wrote:
> Unfortunately, the Mac OS downloads won't work for us as they install into the system Python.

They actually install in a python installed from python.org, not system

>  We have a custom built Python (2.7.3) so I compiled from source.  I noticed a few things:
>
> 1.  The modules compiled from C source have an extension of .so, not .dylib

.so is the usual extension for python extensions on mac os x

> 2.  I installed nose so I could run the numpy tests, but 0 tests ran.

How did you run nose ?

> 3.  I configured numpy to use MKL as per the instructions on Intel's site, but the build still used the Accelerate framework provided by Apple.

Setting up the MKL is a bit tedious, but essentially, you need to
create a site.cfg in the source tree that looks as follows:

[mkl]
library_dirs = "where the libraries are"
include_dirs = "where the headers are"
libpack_libs = mkl_lapack95
mkl_libs = mkl_intel,mkl_intel_thread, mkl_core, mkl_p4m, mkl_p4p

and (that's the undocumented/buggy part), set ATLAS=1 to disable accelerate.

David


From robert.kern at gmail.com  Thu May  9 10:46:25 2013
From: robert.kern at gmail.com (Robert Kern)
Date: Thu, 9 May 2013 15:46:25 +0100
Subject: [Numpy-discussion] www.scipy.org down?
In-Reply-To: <B8F5CDE7FC952A4580ABD28F488EF27930DD7781@AG-DCC-MBX11.dsone.3ds.com>
References: <B8F5CDE7FC952A4580ABD28F488EF27930DD761D@AG-DCC-MBX11.dsone.3ds.com>
	<CAF6FJisR-bJrqyZAbv=xoN-u71h=YO75K5Z6Gbx_WGLStTe-Qg@mail.gmail.com>
	<B8F5CDE7FC952A4580ABD28F488EF27930DD7781@AG-DCC-MBX11.dsone.3ds.com>
Message-ID: <CAF6FJitmm-_tBGxRt5_VuM1qifeqbHnsWqXgPNT++SceO31S8A@mail.gmail.com>

On Thu, May 9, 2013 at 3:25 PM, KACVINSKY Tom <Tom.KACVINSKY at 3ds.com> wrote:
> Unfortunately, the Mac OS downloads won't work for us as they install into the system Python.  We have a custom built Python (2.7.3) so I compiled from source.  I noticed a few things:
>
> 1.  The modules compiled from C source have an extension of .so, not .dylib

As do all Python extension modules.

> 2.  I installed nose so I could run the numpy tests, but 0 tests ran.

Without more information about exactly what command you tried and in
what context (like the current working directory), there is no way for
us to help you except to point you to the relevant section of the
README:

https://github.com/numpy/numpy/blob/master/README.txt#L17

> 3.  I configured numpy to use MKL as per the instructions on Intel's site, but the build still used the Accelerate framework provided by Apple.
>
> Given all of this, whom do I speak to regarding numpy builds on Mac OS?

This mailing list.

--
Robert Kern


From Tom.KACVINSKY at 3ds.com  Thu May  9 11:01:09 2013
From: Tom.KACVINSKY at 3ds.com (KACVINSKY Tom)
Date: Thu, 9 May 2013 15:01:09 +0000
Subject: [Numpy-discussion] www.scipy.org down?
In-Reply-To: <CAF6FJitmm-_tBGxRt5_VuM1qifeqbHnsWqXgPNT++SceO31S8A@mail.gmail.com>
References: <B8F5CDE7FC952A4580ABD28F488EF27930DD761D@AG-DCC-MBX11.dsone.3ds.com>
	<CAF6FJisR-bJrqyZAbv=xoN-u71h=YO75K5Z6Gbx_WGLStTe-Qg@mail.gmail.com>
	<B8F5CDE7FC952A4580ABD28F488EF27930DD7781@AG-DCC-MBX11.dsone.3ds.com>
	<CAF6FJitmm-_tBGxRt5_VuM1qifeqbHnsWqXgPNT++SceO31S8A@mail.gmail.com>
Message-ID: <B8F5CDE7FC952A4580ABD28F488EF27930DD7841@AG-DCC-MBX11.dsone.3ds.com>

I ran the tests as per the instructions:



python -c 'import numpy; numpy.test()'



This is what I get for output:



tkacvins at macomsim> !python

python -Wd -c 'import numpy; numpy.test()'

Running unit tests for numpy

/rd/gen/tky/do_not_delete/Python/macpython27/lib/python2.7/site-packages/nose/util.py:14: DeprecationWarning: The compiler package is deprecated and removed in Python 3.x.

  from compiler.consts import CO_GENERATOR

NumPy version 1.6.2

NumPy is installed in /rd/gen/tky/do_not_delete/Python/macpython27/lib/python2.7/site-packages/numpy

Python version 2.7.3 (default, Oct 10 2012, 14:47:52) [GCC 4.2.1 (Based on Apple Inc. build 5658) (LLVM build 2336.9.00)]

nose version 0.11.4



----------------------------------------------------------------------

Ran 0 tests in 0.148s



OK



Tom



-----Original Message-----
From: numpy-discussion-bounces at scipy.org [mailto:numpy-discussion-bounces at scipy.org] On Behalf Of Robert Kern
Sent: Thursday, May 09, 2013 10:46 AM
To: Discussion of Numerical Python
Subject: Re: [Numpy-discussion] www.scipy.org down?



On Thu, May 9, 2013 at 3:25 PM, KACVINSKY Tom <Tom.KACVINSKY at 3ds.com<mailto:Tom.KACVINSKY at 3ds.com>> wrote:

> Unfortunately, the Mac OS downloads won't work for us as they install into the system Python.  We have a custom built Python (2.7.3) so I compiled from source.  I noticed a few things:

>

> 1.  The modules compiled from C source have an extension of .so, not

> .dylib



As do all Python extension modules.



> 2.  I installed nose so I could run the numpy tests, but 0 tests ran.



Without more information about exactly what command you tried and in what context (like the current working directory), there is no way for us to help you except to point you to the relevant section of the

README:



https://github.com/numpy/numpy/blob/master/README.txt#L17



> 3.  I configured numpy to use MKL as per the instructions on Intel's site, but the build still used the Accelerate framework provided by Apple.

>

> Given all of this, whom do I speak to regarding numpy builds on Mac OS?



This mailing list.



--

Robert Kern

_______________________________________________

NumPy-Discussion mailing list

NumPy-Discussion at scipy.org<mailto:NumPy-Discussion at scipy.org>

http://mail.scipy.org/mailman/listinfo/numpy-discussion

This email and any attachments are intended solely for the use of the individual or entity to whom it is addressed and may be confidential and/or privileged.

If you are not one of the named recipients or have received this email in error,

(i) you should not read, disclose, or copy it,

(ii) please notify sender of your receipt by reply email and delete this email and all attachments,

(iii) Dassault Systemes does not accept or assume any liability or responsibility for any use of or reliance on this email.

For other languages, go to http://www.3ds.com/terms/email-disclaimer
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20130509/b3772905/attachment.html>

From Tom.KACVINSKY at 3ds.com  Thu May  9 11:06:57 2013
From: Tom.KACVINSKY at 3ds.com (KACVINSKY Tom)
Date: Thu, 9 May 2013 15:06:57 +0000
Subject: [Numpy-discussion] www.scipy.org down?
In-Reply-To: <CAGY4rcW_veV_pa7RAf9z7aX72OYnM+FjuP84uFu5DH-uEc76-A@mail.gmail.com>
References: <B8F5CDE7FC952A4580ABD28F488EF27930DD761D@AG-DCC-MBX11.dsone.3ds.com>
	<CAF6FJisR-bJrqyZAbv=xoN-u71h=YO75K5Z6Gbx_WGLStTe-Qg@mail.gmail.com>
	<B8F5CDE7FC952A4580ABD28F488EF27930DD7781@AG-DCC-MBX11.dsone.3ds.com>
	<CAGY4rcW_veV_pa7RAf9z7aX72OYnM+FjuP84uFu5DH-uEc76-A@mail.gmail.com>
Message-ID: <B8F5CDE7FC952A4580ABD28F488EF27930DD788C@AG-DCC-MBX11.dsone.3ds.com>

I assume this is a bottom post list, so my answers are below...

-----Original Message-----
From: numpy-discussion-bounces at scipy.org [mailto:numpy-discussion-bounces at scipy.org] On Behalf Of David Cournapeau
Sent: Thursday, May 09, 2013 10:45 AM
To: Discussion of Numerical Python
Subject: Re: [Numpy-discussion] www.scipy.org down?

On Thu, May 9, 2013 at 3:25 PM, KACVINSKY Tom <Tom.KACVINSKY at 3ds.com> wrote:
>> Unfortunately, the Mac OS downloads won't work for us as they install into the system Python.

>They actually install in a python installed from python.org, not system

>>  We have a custom built Python (2.7.3) so I compiled from source.  I noticed a few things:
>>
>> 1.  The modules compiled from C source have an extension of .so, not .dylib

> .so is the usual extension for python extensions on mac os x

OK, this is good to know.

>> 2.  I installed nose so I could run the numpy tests, but 0 tests ran.

>How did you run nose ?

Per the instructions in the README file.

>> 3.  I configured numpy to use MKL as per the instructions on Intel's site, but the build still used the Accelerate framework provided by Apple.

>Setting up the MKL is a bit tedious, but essentially, you need to create a site.cfg in the source tree that looks as follows:

>[mkl]
>library_dirs = "where the libraries are"
>include_dirs = "where the headers are"
>libpack_libs = mkl_lapack95
>mkl_libs = mkl_intel,mkl_intel_thread, mkl_core, mkl_p4m, mkl_p4p

>and (that's the undocumented/buggy part), set ATLAS=1 to disable accelerate.

This is what I have for my site.cfg file:

[mkl]
include_dirs = /u/users/tky/mklinc
library_dirs = /u/users/tky/mkllib
mkl_libs = mkl_sequential,mkl_intel_lp64,mkl_core
lapack_libs = mkl_sequential,mkl_intel_lp64,mkl_core

I am using MKL 11 update 3, which uses a different set of libraries for linking.  Where do I set ATLAS=1?

David
_______________________________________________
NumPy-Discussion mailing list
NumPy-Discussion at scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion
This email and any attachments are intended solely for the use of the individual or entity to whom it is addressed and may be confidential and/or privileged.

If you are not one of the named recipients or have received this email in error,

(i) you should not read, disclose, or copy it,

(ii) please notify sender of your receipt by reply email and delete this email and all attachments,

(iii) Dassault Systemes does not accept or assume any liability or responsibility for any use of or reliance on this email.

For other languages, go to http://www.3ds.com/terms/email-disclaimer


From chris.barker at noaa.gov  Fri May 10 00:14:47 2013
From: chris.barker at noaa.gov (Chris Barker - NOAA Federal)
Date: Thu, 9 May 2013 21:14:47 -0700
Subject: [Numpy-discussion] printing array in tabular form
In-Reply-To: <1368086808.92617.YahooMailNeo@web193403.mail.sg3.yahoo.com>
References: <1367918086.35474.YahooMailNeo@web193404.mail.sg3.yahoo.com>
	<086DEFA5-2C1B-41F2-934F-4EDBE249C3D3@astro.physik.uni-goettingen.de>
	<1367979205.1211.YahooMailNeo@web193402.mail.sg3.yahoo.com>
	<1367996025.32622.4.camel@farnsworth>
	<1368086808.92617.YahooMailNeo@web193403.mail.sg3.yahoo.com>
Message-ID: <CALGmxEKB+p5vU0km8oRPRN7XR_+sHAsq_2U8FzVrVRLxED0EMA@mail.gmail.com>

On Thu, May 9, 2013 at 1:06 AM, Sudheer Joseph <sudheer.joseph at yahoo.com>wrote:

> Thank you Gomersall,
>           However writing a formatted out put looks to be bit tricky with
> python relative to other programing languages.
>

this is just plain wrong -- working with text in python is as easy, or
easier, than most languages.

Take a little time to learn a bit of pyton, and a bit of numpy, then come
back if you can't figure it out.

But as a rule -- if it's seems hard and/or awkward, you are probably doing
it wrong!

 -Chris
-- 

Christopher Barker, Ph.D.
Oceanographer

Emergency Response Division
NOAA/NOS/OR&R            (206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115       (206) 526-6317   main reception

Chris.Barker at noaa.gov
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20130509/ab5b3882/attachment.html>

From ebressert at gmail.com  Fri May 10 01:47:56 2013
From: ebressert at gmail.com (Eli Bressert)
Date: Fri, 10 May 2013 15:47:56 +1000
Subject: [Numpy-discussion] Name change of the ptp() function
Message-ID: <CA+3+fTaCZODME3ao5_67N9yNgDAmJFbCPz1=Nn0xHSX7TrMn_g@mail.gmail.com>

The peak-to-peak function, ptp(), is a simple and handy function to
use, but it confuses the PEP20 suggestions a bit. See this discussion
for reference: http://d.pr/i/snXG

Based on the discussion, it may be a good idea to make the name more
relevant? Some suggestions include span() or valuerange(). Among few
others who have discussed this on Github
(https://github.com/numpy/numpy/issues/3292#issuecomment-17353454)
valuerange() appears to the best most favored one.

-Eli


From sudheer.joseph at yahoo.com  Fri May 10 05:12:03 2013
From: sudheer.joseph at yahoo.com (Sudheer Joseph)
Date: Fri, 10 May 2013 17:12:03 +0800 (SGT)
Subject: [Numpy-discussion] printing array in tabular form
In-Reply-To: <1368091289.31407.3.camel@farnsworth>
References: <1367918086.35474.YahooMailNeo@web193404.mail.sg3.yahoo.com>
	<086DEFA5-2C1B-41F2-934F-4EDBE249C3D3@astro.physik.uni-goettingen.de>
	<1367979205.1211.YahooMailNeo@web193402.mail.sg3.yahoo.com>
	<1367996025.32622.4.camel@farnsworth>
	<1368086808.92617.YahooMailNeo@web193403.mail.sg3.yahoo.com>
	<1368091289.31407.3.camel@farnsworth>
Message-ID: <1368177123.55356.YahooMailNeo@web193405.mail.sg3.yahoo.com>

Thank you for the response,
???????????????????????? Infact the program which should read this file requires it in specified format which should look like
IL = 1,2,3,4,5
???? 1,2,3,4,5
???? 1,2,3,4,5
?
JL = 1,2,3,4,5
???? 1,2,3,4,5
???? 1,2,3,4,5
so, what I followed from the mail was that np.savetxt can save it in a file and there is no option like
write a string and then leave space and then write the array in above fashion. I searched for a while but could not get any solution.
with best regards,
Sudheer

***************************************************************
Sudheer Joseph 
Indian National Centre for Ocean Information Services
Ministry of Earth Sciences, Govt. of India
POST BOX NO: 21, IDA Jeedeemetla P.O.
Via Pragathi Nagar,Kukatpally, Hyderabad; Pin:5000 55
Tel:+91-40-23886047(O),Fax:+91-40-23895011(O),
Tel:+91-40-23044600(R),Tel:+91-40-9440832534(Mobile)
E-mail:sjo.India at gmail.com;sudheer.joseph at yahoo.com
Web- http://oppamthadathil.tripod.com
***************************************************************


>________________________________
> From: Henry Gomersall <heng at cantab.net>
>To: Discussion of Numerical Python <numpy-discussion at scipy.org> 
>Sent: Thursday, 9 May 2013 2:51 PM
>Subject: Re: [Numpy-discussion] printing array in tabular form
> 
>
>On Thu, 2013-05-09 at 16:06 +0800, Sudheer Joseph wrote:
>> If I wanted to print below text in a file (for reading by another
>> program), it looks to be not an easy job....Hope new developments will
>> come and a userfriendly formatted out put method for pyton will
>> evolve.
>
>I don't understand what the problem is - this seems to be trivial to
>solve. You gave an example in Fortran; Is the transliteration to Python
>not acceptable?
>
>Is the output format specified by the receiving program? If not, there
>are loads of options for creating interoperable text files.
>
>Henry
>
>_______________________________________________
>NumPy-Discussion mailing list
>NumPy-Discussion at scipy.org
>http://mail.scipy.org/mailman/listinfo/numpy-discussion
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20130510/1d3f1629/attachment.html>

From sudheer.joseph at yahoo.com  Fri May 10 05:14:37 2013
From: sudheer.joseph at yahoo.com (Sudheer Joseph)
Date: Fri, 10 May 2013 17:14:37 +0800 (SGT)
Subject: [Numpy-discussion] printing array in tabular form
In-Reply-To: <CAJhcF=1SpFUcjYzkWk5=98RBjvSq7iZRJKB4jckqOVv6q6GOfA@mail.gmail.com>
References: <1367918086.35474.YahooMailNeo@web193404.mail.sg3.yahoo.com>
	<086DEFA5-2C1B-41F2-934F-4EDBE249C3D3@astro.physik.uni-goettingen.de>
	<1367979205.1211.YahooMailNeo@web193402.mail.sg3.yahoo.com>
	<1367996025.32622.4.camel@farnsworth>
	<1368086808.92617.YahooMailNeo@web193403.mail.sg3.yahoo.com>
	<CAJhcF=1SpFUcjYzkWk5=98RBjvSq7iZRJKB4jckqOVv6q6GOfA@mail.gmail.com>
Message-ID: <1368177277.55926.YahooMailNeo@web193405.mail.sg3.yahoo.com>

Thank you,
????????????? But I was looking for? a format statement likw write(*,"(A,5F8.3)")
with best regards,
Sudheer

?
***************************************************************
Sudheer Joseph 
Indian National Centre for Ocean Information Services
Ministry of Earth Sciences, Govt. of India
POST BOX NO: 21, IDA Jeedeemetla P.O.
Via Pragathi Nagar,Kukatpally, Hyderabad; Pin:5000 55
Tel:+91-40-23886047(O),Fax:+91-40-23895011(O),
Tel:+91-40-23044600(R),Tel:+91-40-9440832534(Mobile)
E-mail:sjo.India at gmail.com;sudheer.joseph at yahoo.com
Web- http://oppamthadathil.tripod.com
***************************************************************


>________________________________
> From: Da?id <davidmenhur at gmail.com>
>To: Discussion of Numerical Python <numpy-discussion at scipy.org> 
>Sent: Thursday, 9 May 2013 2:29 PM
>Subject: Re: [Numpy-discussion] printing array in tabular form
> 
>
>On 9 May 2013 10:06, Sudheer Joseph <sudheer.joseph at yahoo.com> wrote:
>> However writing a formatted out put looks to be bit tricky with python
>> relative to other programing languages.
>
>If performance is not an issue, you could do it by hand, as you can
>always do in any programming language:
>
>
>savefile = open('data.txt', 'w')
>N = len(IL)
>
>for start in xrange(N/5):
>?  if start+5 > N:
>? ?  end = N
>?  else:
>? ?  end = start+5
>?  print >> savefile, IL[start : end]
>
>
>But this is actually more verbose, and once you get into NumPy
>workflow, it is actually simple.
>_______________________________________________
>NumPy-Discussion mailing list
>NumPy-Discussion at scipy.org
>http://mail.scipy.org/mailman/listinfo/numpy-discussion
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20130510/82e359c5/attachment.html>

From daniele at grinta.net  Fri May 10 05:42:46 2013
From: daniele at grinta.net (Daniele Nicolodi)
Date: Fri, 10 May 2013 11:42:46 +0200
Subject: [Numpy-discussion] printing array in tabular form
In-Reply-To: <1368177277.55926.YahooMailNeo@web193405.mail.sg3.yahoo.com>
References: <1367918086.35474.YahooMailNeo@web193404.mail.sg3.yahoo.com>
	<086DEFA5-2C1B-41F2-934F-4EDBE249C3D3@astro.physik.uni-goettingen.de>
	<1367979205.1211.YahooMailNeo@web193402.mail.sg3.yahoo.com>
	<1367996025.32622.4.camel@farnsworth>
	<1368086808.92617.YahooMailNeo@web193403.mail.sg3.yahoo.com>
	<CAJhcF=1SpFUcjYzkWk5=98RBjvSq7iZRJKB4jckqOVv6q6GOfA@mail.gmail.com>
	<1368177277.55926.YahooMailNeo@web193405.mail.sg3.yahoo.com>
Message-ID: <518CC116.2080307@grinta.net>

On 10/05/2013 11:14, Sudheer Joseph wrote:

> However writing a formatted out put looks to be bit tricky with
> python relative to other programing languages.

...

> I was looking for a format statement likw write(*,"(A,5F8.3)")

Before denigrating a programming language I would make sure to have a
basic understanding of it.  Every language is going to be tricky if you
approach it with the mindset of Fortran programming.

The output format you are trying to obtain is easy in forrtran because
it is how the default text output formatting is designed.  Of course
obtaining it with a different programming language / numerical library
takes some more effort. But is not tricky at all.


Cheers,
Daniele



From thomas.robitaille at gmail.com  Fri May 10 06:34:13 2013
From: thomas.robitaille at gmail.com (Thomas Robitaille)
Date: Fri, 10 May 2013 11:34:13 +0100
Subject: [Numpy-discussion] Scalar output from sub-classed Numpy array
Message-ID: <CAGMHX_1XiLRW25f05ktNtTtL7=8dy_bjn_v4x6OVwuDB0j=7dA@mail.gmail.com>

Hi everyone,

I am currently trying to write a sub-class of Numpy ndarray, but am
running into issues for functions that return scalar results rather
than array results. For example, in the following case:

    import numpy as np

    class TestClass(np.ndarray):

        def __new__(cls, input_array, unit=None):
            return np.asarray(input_array).view(cls)

        def __array_finalize__(self, obj):
            if obj is None:
                return

        def __array_wrap__(self, out_arr, context=None):
            return np.ndarray.__array_wrap__(self, out_arr, context)

I get:

    In [4]: a = TestClass([1,2,3])

    In [5]: print type(np.dot(a,a))
    <type 'numpy.int64'>

    In [6]: a = TestClass([[1,2],[1,2]])

    In [7]: print type(np.dot(a,a))
    <class '__main__.TestClass'>

that is, in the case where the output is a scalar, it doesn't get
wrapped, while in the case where the output is an array, it does.
Could anyone explain this behavior to me, and most importantly, is
there a way around this and have the above example return a wrapped
0-d TestClass array instead of a Numpy int64?

Thanks,
Tom


From sudheer.joseph at yahoo.com  Fri May 10 07:20:07 2013
From: sudheer.joseph at yahoo.com (Sudheer Joseph)
Date: Fri, 10 May 2013 19:20:07 +0800 (SGT)
Subject: [Numpy-discussion] printing array in tabular form
In-Reply-To: <518CC116.2080307@grinta.net>
References: <1367918086.35474.YahooMailNeo@web193404.mail.sg3.yahoo.com>
	<086DEFA5-2C1B-41F2-934F-4EDBE249C3D3@astro.physik.uni-goettingen.de>
	<1367979205.1211.YahooMailNeo@web193402.mail.sg3.yahoo.com>
	<1367996025.32622.4.camel@farnsworth>
	<1368086808.92617.YahooMailNeo@web193403.mail.sg3.yahoo.com>
	<CAJhcF=1SpFUcjYzkWk5=98RBjvSq7iZRJKB4jckqOVv6q6GOfA@mail.gmail.com>
	<1368177277.55926.YahooMailNeo@web193405.mail.sg3.yahoo.com>
	<518CC116.2080307@grinta.net>
Message-ID: <1368184807.43447.YahooMailNeo@web193405.mail.sg3.yahoo.com>



Hi,
?? I am trying to learn Python after feeling its utility in coding? and also reading a bit about 
its potential only, please do not put words in to my mouth like below.


>> Before denigrating a programming language

If some one has a quick way I would like to learn from them or get a referecence 
where the formatting part is described which was 
my intention while posting here. As I have been using fortran I just tried 
to use it to explain my requirement

with best regards,
Sduheer

From: Daniele Nicolodi <daniele at grinta.net>

To: numpy-discussion at scipy.org 
>Sent: Friday, 10 May 2013 3:12 PM
>Subject: Re: [Numpy-discussion] printing array in tabular form
> 
>
>On 10/05/2013 11:14, Sudheer Joseph wrote:
>
>> However writing a formatted out put looks to be bit tricky with
>> python relative to other programing languages.
>
>...
>
>> I was looking for a format statement likw write(*,"(A,5F8.3)")
>
>Before denigrating a programming language I would make sure to have a
>basic understanding of it.? Every language is going to be tricky if you
>approach it with the mindset of Fortran programming.
>
>The output format you are trying to obtain is easy in forrtran because
>it is how the default text output formatting is designed.? Of course
>obtaining it with a different programming language / numerical library
>takes some more effort. But is not tricky at all.
>
>
>Cheers,
>Daniele
>
>_______________________________________________
>NumPy-Discussion mailing list
>NumPy-Discussion at scipy.org
>http://mail.scipy.org/mailman/listinfo/numpy-discussion
>
>
>


From heng at cantab.net  Fri May 10 07:59:07 2013
From: heng at cantab.net (Henry Gomersall)
Date: Fri, 10 May 2013 12:59:07 +0100
Subject: [Numpy-discussion] printing array in tabular form
In-Reply-To: <1368177277.55926.YahooMailNeo@web193405.mail.sg3.yahoo.com>
References: <1367918086.35474.YahooMailNeo@web193404.mail.sg3.yahoo.com>
	<086DEFA5-2C1B-41F2-934F-4EDBE249C3D3@astro.physik.uni-goettingen.de>
	<1367979205.1211.YahooMailNeo@web193402.mail.sg3.yahoo.com>
	<1367996025.32622.4.camel@farnsworth>
	<1368086808.92617.YahooMailNeo@web193403.mail.sg3.yahoo.com>
	<CAJhcF=1SpFUcjYzkWk5=98RBjvSq7iZRJKB4jckqOVv6q6GOfA@mail.gmail.com>
	<1368177277.55926.YahooMailNeo@web193405.mail.sg3.yahoo.com>
Message-ID: <1368187147.12442.5.camel@farnsworth>

On Fri, 2013-05-10 at 17:14 +0800, Sudheer Joseph wrote:
> Thank you,
>               But I was looking for  a format statement likw
> write(*,"(A,5F8.3)")
> with best regards,
> Sudheer

How about the following:

print('IL = ' + (('%d,' * 5)[:-1] + '\n     ') * 5 % tuple(IL))

If instead of a list IL you had some arbitrary 2D array, a, you could do
(with 2 lines for clarity):

print_string = 'a  = ' + (('%d,' * a.shape[1])[:-1] + '\n     ') *
                          a.shape[0] % tuple(a.T.ravel())
print(print_string)

I'll leave it as an exercise for you to put that into a file.

hen



From derek at astro.physik.uni-goettingen.de  Fri May 10 08:40:15 2013
From: derek at astro.physik.uni-goettingen.de (Derek Homeier)
Date: Fri, 10 May 2013 14:40:15 +0200
Subject: [Numpy-discussion] printing array in tabular form
In-Reply-To: <1368184807.43447.YahooMailNeo@web193405.mail.sg3.yahoo.com>
References: <1367918086.35474.YahooMailNeo@web193404.mail.sg3.yahoo.com>
	<086DEFA5-2C1B-41F2-934F-4EDBE249C3D3@astro.physik.uni-goettingen.de>
	<1367979205.1211.YahooMailNeo@web193402.mail.sg3.yahoo.com>
	<1367996025.32622.4.camel@farnsworth>
	<1368086808.92617.YahooMailNeo@web193403.mail.sg3.yahoo.com>
	<CAJhcF=1SpFUcjYzkWk5=98RBjvSq7iZRJKB4jckqOVv6q6GOfA@mail.gmail.com>
	<1368177277.55926.YahooMailNeo@web193405.mail.sg3.yahoo.com>
	<518CC116.2080307@grinta.net>
	<1368184807.43447.YahooMailNeo@web193405.mail.sg3.yahoo.com>
Message-ID: <6C23BD92-9FFF-4E13-8FD1-8B194707C057@astro.physik.uni-goettingen.de>

On 10.05.2013, at 1:20PM, Sudheer Joseph <sudheer.joseph at yahoo.com> wrote:

> If some one has a quick way I would like to learn from them or get a referecence 
> where the formatting part is described which was 
> my intention while posting here. As I have been using fortran I just tried 
> to use it to explain my requirement
> 
Admittedly the formatting options in Python can be confusing to beginners, precisely
since they are much more powerful than for many other languages. As already pointed
out, formats of the type '(5i5)' are very common to Fortran programs and thus readily
supported by the language. np.savetxt is just a convenience function to support a number
of similarly common output types, and it can create csv, tab-separated, or plenty of other
outputs from a numpy array just out of the box. 
But you added to the confusion as you did not make it clear that you were not just requiring
a plain csv file as your Fortran example would create (and the first version did not even
have the commas); since this is a rather non-standard form you will just have to write a
short loop yourself, wether you are using Fortran or Python.

>                          Infact the program which should read this file requires it in specified format which should look like
> IL = 1,2,3,4,5
>      1,2,3,4,5
>      1,2,3,4,5
> 
The formats are all documented http://docs.python.org/2/library/string.html#format-specification-mini-language
one important thing to know is that you can pretty much add (i.e. concatenate) them like strings:

print(("%6s"+4*"%d,"+"%d\n") % (("IL = ",)+tuple(IL[:5])))

or, perhaps a bit clearer:

fmt = "%6s"+4*"%d,"+"%d\n"
print_t = ("IL = ",)+tuple(IL[:5])
print(fmt % print_t)

The other important bit to keep in mind is that all arguments have to be passed as tuples.
This should allow you to write a loop to print with a "header" or an empty header column
for the subsequent lines as you see fit. 
Except for the string field which is explicitly formatted "%s" here, this is mostly equivalent
to the example Henry just posted.

HTH,
					Derek



From pav at iki.fi  Fri May 10 08:44:27 2013
From: pav at iki.fi (Pauli Virtanen)
Date: Fri, 10 May 2013 15:44:27 +0300
Subject: [Numpy-discussion] Name change of the ptp() function
In-Reply-To: <CA+3+fTaCZODME3ao5_67N9yNgDAmJFbCPz1=Nn0xHSX7TrMn_g@mail.gmail.com>
References: <CA+3+fTaCZODME3ao5_67N9yNgDAmJFbCPz1=Nn0xHSX7TrMn_g@mail.gmail.com>
Message-ID: <kmiq36$giu$1@ger.gmane.org>

10.05.2013 08:47, Eli Bressert kirjoitti:
[clip: renaming ptp]
> valuerange() appears to the best most favored one.

range(), arange(), valuerange()

I'm not really a big fan of changing the name of this function at this
stage, as it seems to me that whether it's a gain or not is somewhat a
matter of taste.

-- 
Pauli Virtanen



From jslavin at cfa.harvard.edu  Fri May 10 08:45:01 2013
From: jslavin at cfa.harvard.edu (Jonathan Slavin)
Date: Fri, 10 May 2013 08:45:01 -0400
Subject: [Numpy-discussion] printing array in tabular form
In-Reply-To: <1368177277.55926.YahooMailNeo@web193405.mail.sg3.yahoo.com>
References: <1367918086.35474.YahooMailNeo@web193404.mail.sg3.yahoo.com>
	<086DEFA5-2C1B-41F2-934F-4EDBE249C3D3@astro.physik.uni-goettingen.de>
	<1367979205.1211.YahooMailNeo@web193402.mail.sg3.yahoo.com>
	<1367996025.32622.4.camel@farnsworth>
	<1368086808.92617.YahooMailNeo@web193403.mail.sg3.yahoo.com>
	<CAJhcF=1SpFUcjYzkWk5=98RBjvSq7iZRJKB4jckqOVv6q6GOfA@mail.gmail.com>
	<1368177277.55926.YahooMailNeo@web193405.mail.sg3.yahoo.com>
Message-ID: <1368189901.19261.16.camel@shevek>

Sudheer,

This is not really numpy specific.  There are many options for output
formatting in python.  For the specific question you have, you could do:

print '{0}{1:8.3f}{2:8.3f}{3:8.3f}{4:8.3f}{5:8.3f}'.format(s,x1,x2,x3,x4,x5)

format is a built-in python string method (see python docs). The one
thing that I will agree with you on is that, as far as I know, there is
no repeat count mechanism.  There are tricky ways around that, e.g.
fmt = '{0}' + ''.join(['{'+str(i)+':8.3f}' for i in range(1,6)])
print fmt.format(s,x1,x2,x3,x4,x5)
though not as simple as the fortran output statement.

Jon

On Fri, 2013-05-10 at 17:14 +0800, Sudheer Joseph wrote:
> Thank you,
>               But I was looking for  a format statement likw
> write(*,"(A,5F8.3)")
> with best regards,
> Sudheer
>  
> ***************************************************************
> Sudheer Joseph 
> Indian National Centre for Ocean Information Services
> Ministry of Earth Sciences, Govt. of India
> POST BOX NO: 21, IDA Jeedeemetla P.O.
> Via Pragathi Nagar,Kukatpally, Hyderabad; Pin:5000 55
> Tel:+91-40-23886047(O),Fax:+91-40-23895011(O),
> Tel:+91-40-23044600(R),Tel:+91-40-9440832534(Mobile)
> E-mail:sjo.India at gmail.com;sudheer.joseph at yahoo.com
> Web- http://oppamthadathil.tripod.com
> ***************************************************************
>         
>         ______________________________________________________________
>         From: Da?id <davidmenhur at gmail.com>
>         To: Discussion of Numerical Python
>         <numpy-discussion at scipy.org> 
>         Sent: Thursday, 9 May 2013 2:29 PM
>         Subject: Re: [Numpy-discussion] printing array in tabular form
>         
>         
>         On 9 May 2013 10:06, Sudheer Joseph <sudheer.joseph at yahoo.com>
>         wrote:
>         > However writing a formatted out put looks to be bit tricky
>         with python
>         > relative to other programing languages.
>         
>         If performance is not an issue, you could do it by hand, as
>         you can
>         always do in any programming language:
>         
>         
>         savefile = open('data.txt', 'w')
>         N = len(IL)
>         
>         for start in xrange(N/5):
>           if start+5 > N:
>             end = N
>           else:
>             end = start+5
>           print >> savefile, IL[start : end]
>         
>         
>         But this is actually more verbose, and once you get into NumPy
>         workflow, it is actually simple.
>         _______________________________________________
>         NumPy-Discussion mailing list
>         NumPy-Discussion at scipy.org
>         http://mail.scipy.org/mailman/listinfo/numpy-discussion
>         
>         
>         

-- 
______________________________________________________________
Jonathan D. Slavin              Harvard-Smithsonian CfA
jslavin at cfa.harvard.edu         60 Garden Street, MS 83
phone: (617) 496-7981           Cambridge, MA 02138-1516
 cell: (781) 363-0035           USA
______________________________________________________________



From daniele at grinta.net  Fri May 10 08:51:55 2013
From: daniele at grinta.net (Daniele Nicolodi)
Date: Fri, 10 May 2013 14:51:55 +0200
Subject: [Numpy-discussion] printing array in tabular form
In-Reply-To: <1368184807.43447.YahooMailNeo@web193405.mail.sg3.yahoo.com>
References: <1367918086.35474.YahooMailNeo@web193404.mail.sg3.yahoo.com>
	<086DEFA5-2C1B-41F2-934F-4EDBE249C3D3@astro.physik.uni-goettingen.de>
	<1367979205.1211.YahooMailNeo@web193402.mail.sg3.yahoo.com>
	<1367996025.32622.4.camel@farnsworth>
	<1368086808.92617.YahooMailNeo@web193403.mail.sg3.yahoo.com>
	<CAJhcF=1SpFUcjYzkWk5=98RBjvSq7iZRJKB4jckqOVv6q6GOfA@mail.gmail.com>
	<1368177277.55926.YahooMailNeo@web193405.mail.sg3.yahoo.com>
	<518CC116.2080307@grinta.net>
	<1368184807.43447.YahooMailNeo@web193405.mail.sg3.yahoo.com>
Message-ID: <518CED6B.8060105@grinta.net>

On 10/05/2013 13:20, Sudheer Joseph wrote:

> Hi,
> I am trying to learn Python after feeling its utility in coding and
> also reading a bit aboutits potential only, please do not put words
> in to my mouth like below.

I didn't put words in your mouth, I simply quoted emails you sent to the
list and gave my interpretation of what you wrote.

>>> Before denigrating a programming language
> 
> If some one has a quick way I would like to learn from them or get a referecence 
> where the formatting part is described which was 
> my intention while posting here. As I have been using fortran I just tried 
> to use it to explain my requirement

For references about string formatting in Python:

http://docs.python.org/2/library/string.html#formatstrings
http://docs.python.org/2/library/stdtypes.html#string-formatting

for the numpy array to text formatting:

http://docs.scipy.org/doc/numpy/reference/generated/numpy.savetxt.html

writing a function to do what you ask is trivial. Unfortunately there is
no "format the thing as I wish" function.

If you wish to format numpy arrays preceding them with a variable name,
the following is a possible solution that gives the same formatting as
in your example:

import numpy as np
import sys

def format(out, v, name):
    header = "{} = ".format(name)
    out.write(header)
    np.savetxt(out, v, fmt="%d", delimiter=", ",
               newline="\n" + " " * len(header))
    out.write("\n")

IL = np.array([range(5), ] * 5)
format(sys.stdout, IL, "IL")


Cheers,
Daniele



From ebressert at gmail.com  Fri May 10 09:04:35 2013
From: ebressert at gmail.com (Eli Bressert)
Date: Fri, 10 May 2013 23:04:35 +1000
Subject: [Numpy-discussion] Name change of the ptp() function
In-Reply-To: <kmiq36$giu$1@ger.gmane.org>
References: <CA+3+fTaCZODME3ao5_67N9yNgDAmJFbCPz1=Nn0xHSX7TrMn_g@mail.gmail.com>
	<kmiq36$giu$1@ger.gmane.org>
Message-ID: <CA+3+fTZDE7+VsZrGG0+QQTi-7e+7_-WhpzEDYunxuF6+2btrKQ@mail.gmail.com>

That's a good point regarding the range function names. But, I think
the issue still stands on the readability of the ptp function.
Regarding PEP20 it's stated that "readability counts."

If you regard what ptp is supposed to replace, array.max() -
array.min(), the aforementioned follows the PEP20 better as it is more
readable. If valuerange() is not an acceptable name, maybe span()?

-Eli


On Fri, May 10, 2013 at 10:44 PM, Pauli Virtanen <pav at iki.fi> wrote:
> 10.05.2013 08:47, Eli Bressert kirjoitti:
> [clip: renaming ptp]
>> valuerange() appears to the best most favored one.
>
> range(), arange(), valuerange()
>
> I'm not really a big fan of changing the name of this function at this
> stage, as it seems to me that whether it's a gain or not is somewhat a
> matter of taste.
>
> --
> Pauli Virtanen
>
> _______________________________________________
> NumPy-Discussion mailing list
> NumPy-Discussion at scipy.org
> http://mail.scipy.org/mailman/listinfo/numpy-discussion


From pav at iki.fi  Fri May 10 09:17:01 2013
From: pav at iki.fi (Pauli Virtanen)
Date: Fri, 10 May 2013 16:17:01 +0300
Subject: [Numpy-discussion] Name change of the ptp() function
In-Reply-To: <CA+3+fTZDE7+VsZrGG0+QQTi-7e+7_-WhpzEDYunxuF6+2btrKQ@mail.gmail.com>
References: <CA+3+fTaCZODME3ao5_67N9yNgDAmJFbCPz1=Nn0xHSX7TrMn_g@mail.gmail.com>
	<kmiq36$giu$1@ger.gmane.org>
	<CA+3+fTZDE7+VsZrGG0+QQTi-7e+7_-WhpzEDYunxuF6+2btrKQ@mail.gmail.com>
Message-ID: <kmis08$5p6$1@ger.gmane.org>

10.05.2013 16:04, Eli Bressert kirjoitti:
> That's a good point regarding the range function names. But, I think
> the issue still stands on the readability of the ptp function.
> Regarding PEP20 it's stated that "readability counts."

I think here it has to be kept in mind that this function has been
called ptp() already in Numeric, i.e., for the last 10+ years. This is
the first proposal to change it that I know of, so I think keeping the
API the same weighs against changing it due to aesthetic reasons.

-- 
Pauli Virtanen



From robert.kern at gmail.com  Fri May 10 09:17:51 2013
From: robert.kern at gmail.com (Robert Kern)
Date: Fri, 10 May 2013 14:17:51 +0100
Subject: [Numpy-discussion] Name change of the ptp() function
In-Reply-To: <CA+3+fTZDE7+VsZrGG0+QQTi-7e+7_-WhpzEDYunxuF6+2btrKQ@mail.gmail.com>
References: <CA+3+fTaCZODME3ao5_67N9yNgDAmJFbCPz1=Nn0xHSX7TrMn_g@mail.gmail.com>
	<kmiq36$giu$1@ger.gmane.org>
	<CA+3+fTZDE7+VsZrGG0+QQTi-7e+7_-WhpzEDYunxuF6+2btrKQ@mail.gmail.com>
Message-ID: <CAF6FJisaiNQ43B2oNSmQr=r_AFz-h8QPonnK=FYRKg6r9G8XyA@mail.gmail.com>

On Fri, May 10, 2013 at 2:04 PM, Eli Bressert <ebressert at gmail.com> wrote:
> That's a good point regarding the range function names. But, I think
> the issue still stands on the readability of the ptp function.
> Regarding PEP20 it's stated that "readability counts."
>
> If you regard what ptp is supposed to replace, array.max() -
> array.min(), the aforementioned follows the PEP20 better as it is more
> readable. If valuerange() is not an acceptable name, maybe span()?

Sure, it's probably more readable, and that would be the controlling
factor if this were a new function. But that's not really the
operative question here. Are the gains in readability worth the
nontrivial costs of deprecating and removing the old name? I, for one,
am generally not in favor of such deprecations.

--
Robert Kern


From ralf.gommers at gmail.com  Fri May 10 09:27:07 2013
From: ralf.gommers at gmail.com (Ralf Gommers)
Date: Fri, 10 May 2013 15:27:07 +0200
Subject: [Numpy-discussion] Name change of the ptp() function
In-Reply-To: <CAF6FJisaiNQ43B2oNSmQr=r_AFz-h8QPonnK=FYRKg6r9G8XyA@mail.gmail.com>
References: <CA+3+fTaCZODME3ao5_67N9yNgDAmJFbCPz1=Nn0xHSX7TrMn_g@mail.gmail.com>
	<kmiq36$giu$1@ger.gmane.org>
	<CA+3+fTZDE7+VsZrGG0+QQTi-7e+7_-WhpzEDYunxuF6+2btrKQ@mail.gmail.com>
	<CAF6FJisaiNQ43B2oNSmQr=r_AFz-h8QPonnK=FYRKg6r9G8XyA@mail.gmail.com>
Message-ID: <CABL7CQjKS6b9acXFa_oeqWy+pYeT73hvuF_Vekk4p7qMJ6UXAA@mail.gmail.com>

On Fri, May 10, 2013 at 3:17 PM, Robert Kern <robert.kern at gmail.com> wrote:

> On Fri, May 10, 2013 at 2:04 PM, Eli Bressert <ebressert at gmail.com> wrote:
> > That's a good point regarding the range function names. But, I think
> > the issue still stands on the readability of the ptp function.
> > Regarding PEP20 it's stated that "readability counts."
> >
> > If you regard what ptp is supposed to replace, array.max() -
> > array.min(), the aforementioned follows the PEP20 better as it is more
> > readable. If valuerange() is not an acceptable name, maybe span()?
>
> Sure, it's probably more readable, and that would be the controlling
> factor if this were a new function. But that's not really the
> operative question here. Are the gains in readability worth the
> nontrivial costs of deprecating and removing the old name? I, for one,
> am generally not in favor of such deprecations.
>

That's not the only option though. I'm -1 on deprecation, but +0 on
renaming and keeping ptp as an alias. The function name is really quite
poor.

Ralf
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20130510/e830fd09/attachment.html>

From derek at astro.physik.uni-goettingen.de  Fri May 10 09:36:38 2013
From: derek at astro.physik.uni-goettingen.de (Derek Homeier)
Date: Fri, 10 May 2013 15:36:38 +0200
Subject: [Numpy-discussion] printing array in tabular form
In-Reply-To: <518CED6B.8060105@grinta.net>
References: <1367918086.35474.YahooMailNeo@web193404.mail.sg3.yahoo.com>
	<086DEFA5-2C1B-41F2-934F-4EDBE249C3D3@astro.physik.uni-goettingen.de>
	<1367979205.1211.YahooMailNeo@web193402.mail.sg3.yahoo.com>
	<1367996025.32622.4.camel@farnsworth>
	<1368086808.92617.YahooMailNeo@web193403.mail.sg3.yahoo.com>
	<CAJhcF=1SpFUcjYzkWk5=98RBjvSq7iZRJKB4jckqOVv6q6GOfA@mail.gmail.com>
	<1368177277.55926.YahooMailNeo@web193405.mail.sg3.yahoo.com>
	<518CC116.2080307@grinta.net>
	<1368184807.43447.YahooMailNeo@web193405.mail.sg3.yahoo.com>
	<518CED6B.8060105@grinta.net>
Message-ID: <5D08B318-6614-4A78-A448-A89936B754A6@astro.physik.uni-goettingen.de>

On 10.05.2013, at 2:51PM, Daniele Nicolodi <daniele at grinta.net> wrote:

> If you wish to format numpy arrays preceding them with a variable name,
> the following is a possible solution that gives the same formatting as
> in your example:
> 
> import numpy as np
> import sys
> 
> def format(out, v, name):
>    header = "{} = ".format(name)
>    out.write(header)
>    np.savetxt(out, v, fmt="%d", delimiter=", ",
>               newline="\n" + " " * len(header))
>    out.write("\n")
> 
> IL = np.array([range(5), ] * 5)
> format(sys.stdout, IL, "IL")

That is a quite ingenuous way to use savetxt functionality to write that extra column!

Only two comments:

Don't call that function "format", as it would mask the 'format' builtin!

In the present version it will only work with a file handle; to print it a to file you would need
to pass it as fformat(open(fname, 'a'), ? or check for that case inside the function.

Cheers,
						Derek
 




From cournape at gmail.com  Fri May 10 09:53:33 2013
From: cournape at gmail.com (David Cournapeau)
Date: Fri, 10 May 2013 14:53:33 +0100
Subject: [Numpy-discussion] Name change of the ptp() function
In-Reply-To: <CABL7CQjKS6b9acXFa_oeqWy+pYeT73hvuF_Vekk4p7qMJ6UXAA@mail.gmail.com>
References: <CA+3+fTaCZODME3ao5_67N9yNgDAmJFbCPz1=Nn0xHSX7TrMn_g@mail.gmail.com>
	<kmiq36$giu$1@ger.gmane.org>
	<CA+3+fTZDE7+VsZrGG0+QQTi-7e+7_-WhpzEDYunxuF6+2btrKQ@mail.gmail.com>
	<CAF6FJisaiNQ43B2oNSmQr=r_AFz-h8QPonnK=FYRKg6r9G8XyA@mail.gmail.com>
	<CABL7CQjKS6b9acXFa_oeqWy+pYeT73hvuF_Vekk4p7qMJ6UXAA@mail.gmail.com>
Message-ID: <CAGY4rcV-Zp0B7=u876vNxDENkLLA3hzSWkEvJaVqZ0epydimQA@mail.gmail.com>

On Fri, May 10, 2013 at 2:27 PM, Ralf Gommers <ralf.gommers at gmail.com> wrote:
>
>
>
> On Fri, May 10, 2013 at 3:17 PM, Robert Kern <robert.kern at gmail.com> wrote:
>>
>> On Fri, May 10, 2013 at 2:04 PM, Eli Bressert <ebressert at gmail.com> wrote:
>> > That's a good point regarding the range function names. But, I think
>> > the issue still stands on the readability of the ptp function.
>> > Regarding PEP20 it's stated that "readability counts."
>> >
>> > If you regard what ptp is supposed to replace, array.max() -
>> > array.min(), the aforementioned follows the PEP20 better as it is more
>> > readable. If valuerange() is not an acceptable name, maybe span()?
>>
>> Sure, it's probably more readable, and that would be the controlling
>> factor if this were a new function. But that's not really the
>> operative question here. Are the gains in readability worth the
>> nontrivial costs of deprecating and removing the old name? I, for one,
>> am generally not in favor of such deprecations.
>
>
> That's not the only option though. I'm -1 on deprecation, but +0 on renaming
> and keeping ptp as an alias. The function name is really quite poor.

I think it is a matter of context. I don't know the history of that
function, but coming from a signal processing background, its meaning
was obvious to me. It is a peak to peak is a very common operations
when dealing with audio file, for example (that's how most wave
display work AFAIK).

I am certainly -1 on the deprecation as well, and -0 on alias.

David


From davidmenhur at gmail.com  Fri May 10 09:55:31 2013
From: davidmenhur at gmail.com (=?UTF-8?B?RGHPgGlk?=)
Date: Fri, 10 May 2013 15:55:31 +0200
Subject: [Numpy-discussion] Name change of the ptp() function
In-Reply-To: <CAF6FJisaiNQ43B2oNSmQr=r_AFz-h8QPonnK=FYRKg6r9G8XyA@mail.gmail.com>
References: <CA+3+fTaCZODME3ao5_67N9yNgDAmJFbCPz1=Nn0xHSX7TrMn_g@mail.gmail.com>
	<kmiq36$giu$1@ger.gmane.org>
	<CA+3+fTZDE7+VsZrGG0+QQTi-7e+7_-WhpzEDYunxuF6+2btrKQ@mail.gmail.com>
	<CAF6FJisaiNQ43B2oNSmQr=r_AFz-h8QPonnK=FYRKg6r9G8XyA@mail.gmail.com>
Message-ID: <CAJhcF=1rh-XmPCXGuq1L=NyszkEDrFM7uGpyyM1SC+_=+dcxHA@mail.gmail.com>

On May 10, 2013 3:18 PM, "Robert Kern" <robert.kern at gmail.com> wrote:
> Sure, it's probably more readable

I am not sure of it. I would have to check the docs to see what it means.
The mathematical term is range, but it already has a meaning in Python, so
it is not a good way to go, being perhaps valuerange the  compromise,  but
not really clear by itself. In some areas, nevertheless, ptp is the
standard notation, as it is in electronics - and maybe that is why it made
its way into Numeric.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20130510/0e569640/attachment.html>

From nouiz at nouiz.org  Fri May 10 12:08:35 2013
From: nouiz at nouiz.org (=?ISO-8859-1?Q?Fr=E9d=E9ric_Bastien?=)
Date: Fri, 10 May 2013 12:08:35 -0400
Subject: [Numpy-discussion] __array_priority__ don't work for gt, lt,
	... operator
Message-ID: <CADKKbtgf8XPgmRYhvA5KYABCbUWV8V+kpKT8v_bjbi16EBAqVw@mail.gmail.com>

Hi,

it popped again on the Theano mailing list that this don't work:

np.arange(10) <= a_theano_vector.

The reason is that __array_priority__ isn't respected for that class of
operation.

This page explain the problem and give a work around:

http://stackoverflow.com/questions/14619449/how-can-i-override-comparisons-between-numpys-ndarray-and-my-type

The work around is to make a python function that will decide witch version
of the comparator to call and do the call. Then we tell NumPy to use that
function instead of its current function with: np.set_numeric_ops(...)

But if we do that, when we import theano, we will slow down all normal
numpy comparison for the user, as when <= is execute, first there will be
numpy c code executed, that will call the python function to decide witch
version to do, then if it is 2 numpy ndarray, it will call again numpy c
code.

That isn't a good solution. We could do the same override in C, but then
theano work the same when there isn't a c++ compiler. That isn't nice.

What do you think of changing them to check for __array_priority__ before
doing the comparison?

Fr?d?ric
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20130510/e0f33097/attachment.html>

From arnaldorusso at gmail.com  Fri May 10 13:32:42 2013
From: arnaldorusso at gmail.com (Arnaldo Russo)
Date: Fri, 10 May 2013 14:32:42 -0300
Subject: [Numpy-discussion] pyhdf packaging
In-Reply-To: <bd3b168e7b9d9ed612f262aa7e426f16.squirrel@srv2.s4y.tournesol-consulting.eu>
References: <bd3b168e7b9d9ed612f262aa7e426f16.squirrel@srv2.s4y.tournesol-consulting.eu>
Message-ID: <CAKUv9ONU8bkVAkxJBenDvNz44d6=Us-X404qeReOo+uqjXu8Pg@mail.gmail.com>

Hi Andreas,
This packaging would be much useful!
How can I help with this?
pyhdf is very important because HDF4-EOS does not open with another
packages, only with pyhdf and gdal.

best,
Arnaldo


---
*Arnaldo D'Amaral Pereira Granja Russo*
Lab. de Estudos dos Oceanos e Clima
Instituto de Oceanografia - FURG




2012/11/6 Andreas Hilboll <lists at hilboll.de>

> Hi,
>
> I would like to package pyhdf for Ubuntu and make the package publicly
> available. Since the license is not totally clear to me (I cannot find any
> information in the sources, and the cheeseshop says "public", which
> doesn't mean anything to me), I tried to contact the maintainer, Andre
> Gosselin, however the email bounces, so I guess he's gone.
>
> Can anyone point me to how to proceed from here?
>
> Cheers, Andreas.
>
> _______________________________________________
> NumPy-Discussion mailing list
> NumPy-Discussion at scipy.org
> http://mail.scipy.org/mailman/listinfo/numpy-discussion
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20130510/36d10a81/attachment.html>

From charlesr.harris at gmail.com  Fri May 10 13:34:26 2013
From: charlesr.harris at gmail.com (Charles R Harris)
Date: Fri, 10 May 2013 11:34:26 -0600
Subject: [Numpy-discussion] __array_priority__ don't work for gt, lt,
	... operator
In-Reply-To: <CADKKbtgf8XPgmRYhvA5KYABCbUWV8V+kpKT8v_bjbi16EBAqVw@mail.gmail.com>
References: <CADKKbtgf8XPgmRYhvA5KYABCbUWV8V+kpKT8v_bjbi16EBAqVw@mail.gmail.com>
Message-ID: <CAB6mnxLZP-3+7pX1K5NK7LABccwqBT0UQb1p8whXXtEGh+J8RA@mail.gmail.com>

On Fri, May 10, 2013 at 10:08 AM, Fr?d?ric Bastien <nouiz at nouiz.org> wrote:

> Hi,
>
> it popped again on the Theano mailing list that this don't work:
>
> np.arange(10) <= a_theano_vector.
>
> The reason is that __array_priority__ isn't respected for that class of
> operation.
>
> This page explain the problem and give a work around:
>
>
> http://stackoverflow.com/questions/14619449/how-can-i-override-comparisons-between-numpys-ndarray-and-my-type
>
> The work around is to make a python function that will decide witch
> version of the comparator to call and do the call. Then we tell NumPy to
> use that function instead of its current function with:
> np.set_numeric_ops(...)
>
> But if we do that, when we import theano, we will slow down all normal
> numpy comparison for the user, as when <= is execute, first there will be
> numpy c code executed, that will call the python function to decide witch
> version to do, then if it is 2 numpy ndarray, it will call again numpy c
> code.
>
> That isn't a good solution. We could do the same override in C, but then
> theano work the same when there isn't a c++ compiler. That isn't nice.
>
> What do you think of changing them to check for __array_priority__ before
> doing the comparison?
>

This looks like an oversight and should be fixed.

Chuck
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20130510/a078d352/attachment.html>

From nouiz at nouiz.org  Fri May 10 15:35:25 2013
From: nouiz at nouiz.org (=?ISO-8859-1?Q?Fr=E9d=E9ric_Bastien?=)
Date: Fri, 10 May 2013 15:35:25 -0400
Subject: [Numpy-discussion] __array_priority__ don't work for gt, lt,
	... operator
In-Reply-To: <CAB6mnxLZP-3+7pX1K5NK7LABccwqBT0UQb1p8whXXtEGh+J8RA@mail.gmail.com>
References: <CADKKbtgf8XPgmRYhvA5KYABCbUWV8V+kpKT8v_bjbi16EBAqVw@mail.gmail.com>
	<CAB6mnxLZP-3+7pX1K5NK7LABccwqBT0UQb1p8whXXtEGh+J8RA@mail.gmail.com>
Message-ID: <CADKKbthRTMOvDK=Ujp-3qL6DEdRvce=J4DSsH4Go8o9=V1vPBg@mail.gmail.com>

I'm trying to do it, but each time I want to test something, it takes a
long time to rebuild numpy to test it. Is there a way to don't recompile
everything for each test?

thanks

Fred


On Fri, May 10, 2013 at 1:34 PM, Charles R Harris <charlesr.harris at gmail.com
> wrote:

>
>
> On Fri, May 10, 2013 at 10:08 AM, Fr?d?ric Bastien <nouiz at nouiz.org>wrote:
>
>> Hi,
>>
>> it popped again on the Theano mailing list that this don't work:
>>
>> np.arange(10) <= a_theano_vector.
>>
>> The reason is that __array_priority__ isn't respected for that class of
>> operation.
>>
>> This page explain the problem and give a work around:
>>
>>
>> http://stackoverflow.com/questions/14619449/how-can-i-override-comparisons-between-numpys-ndarray-and-my-type
>>
>> The work around is to make a python function that will decide witch
>> version of the comparator to call and do the call. Then we tell NumPy to
>> use that function instead of its current function with:
>> np.set_numeric_ops(...)
>>
>> But if we do that, when we import theano, we will slow down all normal
>> numpy comparison for the user, as when <= is execute, first there will be
>> numpy c code executed, that will call the python function to decide witch
>> version to do, then if it is 2 numpy ndarray, it will call again numpy c
>> code.
>>
>> That isn't a good solution. We could do the same override in C, but then
>> theano work the same when there isn't a c++ compiler. That isn't nice.
>>
>> What do you think of changing them to check for __array_priority__ before
>> doing the comparison?
>>
>
> This looks like an oversight and should be fixed.
>
> Chuck
>
> _______________________________________________
> NumPy-Discussion mailing list
> NumPy-Discussion at scipy.org
> http://mail.scipy.org/mailman/listinfo/numpy-discussion
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20130510/418604c5/attachment.html>

From Tom.KACVINSKY at 3ds.com  Fri May 10 15:41:25 2013
From: Tom.KACVINSKY at 3ds.com (KACVINSKY Tom)
Date: Fri, 10 May 2013 19:41:25 +0000
Subject: [Numpy-discussion] tests not running
Message-ID: <B8F5CDE7FC952A4580ABD28F488EF27930DD943A@AG-DCC-MBX11.dsone.3ds.com>

Here is my set up:

Mac OS 10.7.5
Xcode 4.5.1
Intel Fortran 12.1
Python 2.7.3 built from source
Numpy 1.6.2 built from source, using MKL 11.0
nose 0.11.4 installed

I run the numpy tests as documented (python -c 'import numpy; numpy.test()'), but get this output:

tkacvins at macomsim> python -c 'import numpy; numpy.test()'
Running unit tests for numpy
NumPy version 1.6.2
NumPy is installed in /rd/gen/tky/do_not_delete/Python/macpython27/lib/python2.7/site-packages/numpy
Python version 2.7.3 (default, Oct 10 2012, 14:47:52) [GCC 4.2.1 (Based on Apple Inc. build 5658) (LLVM build 2336.9.00)]
nose version 0.11.4

----------------------------------------------------------------------
Ran 0 tests in 0.025s

OK

Any diagnostics or options I can pass to the tests to see what is going on?   This is rather odd, I thought the tests would run and possibly fail.  I didn't expect 0 tests to run.

Thanks,

Tom

This email and any attachments are intended solely for the use of the individual or entity to whom it is addressed and may be confidential and/or privileged.

If you are not one of the named recipients or have received this email in error,

(i) you should not read, disclose, or copy it,

(ii) please notify sender of your receipt by reply email and delete this email and all attachments,

(iii) Dassault Systemes does not accept or assume any liability or responsibility for any use of or reliance on this email.

For other languages, go to http://www.3ds.com/terms/email-disclaimer


From sebastian at sipsolutions.net  Fri May 10 16:03:09 2013
From: sebastian at sipsolutions.net (Sebastian Berg)
Date: Fri, 10 May 2013 22:03:09 +0200
Subject: [Numpy-discussion] __array_priority__ don't work for gt, lt,
 ... operator
In-Reply-To: <CADKKbthRTMOvDK=Ujp-3qL6DEdRvce=J4DSsH4Go8o9=V1vPBg@mail.gmail.com>
References: <CADKKbtgf8XPgmRYhvA5KYABCbUWV8V+kpKT8v_bjbi16EBAqVw@mail.gmail.com>
	<CAB6mnxLZP-3+7pX1K5NK7LABccwqBT0UQb1p8whXXtEGh+J8RA@mail.gmail.com>
	<CADKKbthRTMOvDK=Ujp-3qL6DEdRvce=J4DSsH4Go8o9=V1vPBg@mail.gmail.com>
Message-ID: <1368216189.2567.10.camel@sebastian-laptop>

On Fri, 2013-05-10 at 15:35 -0400, Fr?d?ric Bastien wrote:
> I'm trying to do it, but each time I want to test something, it takes
> a long time to rebuild numpy to test it. Is there a way to don't
> recompile everything for each test?
> 
Are you using current master? It defaults to use
ENABLE_SEPARATE_COMPILATION enviroment variable, which, together with
ccache, makes most changes in numpy compile fast for me.

- Sebastian


> thanks
> 
> Fred
> 
> 
> 
> On Fri, May 10, 2013 at 1:34 PM, Charles R Harris
> <charlesr.harris at gmail.com> wrote:
>         
>         
>         On Fri, May 10, 2013 at 10:08 AM, Fr?d?ric Bastien
>         <nouiz at nouiz.org> wrote:
>                 Hi,
>                 
>                 
>                 it popped again on the Theano mailing list that this
>                 don't work:
>                 
>                 
>                 np.arange(10) <= a_theano_vector.
>                 
>                 
>                 The reason is that __array_priority__ isn't respected
>                 for that class of operation.
>                 
>                 
>                 
>                 This page explain the problem and give a work around:
>                 
>                 
>                 http://stackoverflow.com/questions/14619449/how-can-i-override-comparisons-between-numpys-ndarray-and-my-type
>                 
>                 
>                 The work around is to make a python function that will
>                 decide witch version of the comparator to call and do
>                 the call. Then we tell NumPy to use that function
>                 instead of its current function with:
>                 np.set_numeric_ops(...)
>                 
>                 But if we do that, when we import theano, we will slow
>                 down all normal numpy comparison for the user, as when
>                 <= is execute, first there will be numpy c code
>                 executed, that will call the python function to decide
>                 witch version to do, then if it is 2 numpy ndarray, it
>                 will call again numpy c code.
>                 
>                 
>                 That isn't a good solution. We could do the same
>                 override in C, but then theano work the same when
>                 there isn't a c++ compiler. That isn't nice.
>                 
>                 
>                 What do you think of changing them to check for
>                 __array_priority__ before doing the comparison?
>         
>         This looks like an oversight and should be fixed.
>         
>         Chuck 
>         
>         
>         _______________________________________________
>         NumPy-Discussion mailing list
>         NumPy-Discussion at scipy.org
>         http://mail.scipy.org/mailman/listinfo/numpy-discussion
>         
> 
> 
> _______________________________________________
> NumPy-Discussion mailing list
> NumPy-Discussion at scipy.org
> http://mail.scipy.org/mailman/listinfo/numpy-discussion




From nouiz at nouiz.org  Fri May 10 17:02:48 2013
From: nouiz at nouiz.org (=?ISO-8859-1?Q?Fr=E9d=E9ric_Bastien?=)
Date: Fri, 10 May 2013 17:02:48 -0400
Subject: [Numpy-discussion] __array_priority__ don't work for gt, lt,
	... operator
In-Reply-To: <1368216189.2567.10.camel@sebastian-laptop>
References: <CADKKbtgf8XPgmRYhvA5KYABCbUWV8V+kpKT8v_bjbi16EBAqVw@mail.gmail.com>
	<CAB6mnxLZP-3+7pX1K5NK7LABccwqBT0UQb1p8whXXtEGh+J8RA@mail.gmail.com>
	<CADKKbthRTMOvDK=Ujp-3qL6DEdRvce=J4DSsH4Go8o9=V1vPBg@mail.gmail.com>
	<1368216189.2567.10.camel@sebastian-laptop>
Message-ID: <CADKKbtgVS_5ROZen2WxHxTc2A907ipovj-wOs29D3vBzx3PGig@mail.gmail.com>

thanks, I'll look at it.

I made a PR: https://github.com/numpy/numpy/pull/3324

Where should I put the tests about this?

thanks

Fred


On Fri, May 10, 2013 at 4:03 PM, Sebastian Berg
<sebastian at sipsolutions.net>wrote:

> On Fri, 2013-05-10 at 15:35 -0400, Fr?d?ric Bastien wrote:
> > I'm trying to do it, but each time I want to test something, it takes
> > a long time to rebuild numpy to test it. Is there a way to don't
> > recompile everything for each test?
> >
> Are you using current master? It defaults to use
> ENABLE_SEPARATE_COMPILATION enviroment variable, which, together with
> ccache, makes most changes in numpy compile fast for me.
>
> - Sebastian
>
>
> > thanks
> >
> > Fred
> >
> >
> >
> > On Fri, May 10, 2013 at 1:34 PM, Charles R Harris
> > <charlesr.harris at gmail.com> wrote:
> >
> >
> >         On Fri, May 10, 2013 at 10:08 AM, Fr?d?ric Bastien
> >         <nouiz at nouiz.org> wrote:
> >                 Hi,
> >
> >
> >                 it popped again on the Theano mailing list that this
> >                 don't work:
> >
> >
> >                 np.arange(10) <= a_theano_vector.
> >
> >
> >                 The reason is that __array_priority__ isn't respected
> >                 for that class of operation.
> >
> >
> >
> >                 This page explain the problem and give a work around:
> >
> >
> >
> http://stackoverflow.com/questions/14619449/how-can-i-override-comparisons-between-numpys-ndarray-and-my-type
> >
> >
> >                 The work around is to make a python function that will
> >                 decide witch version of the comparator to call and do
> >                 the call. Then we tell NumPy to use that function
> >                 instead of its current function with:
> >                 np.set_numeric_ops(...)
> >
> >                 But if we do that, when we import theano, we will slow
> >                 down all normal numpy comparison for the user, as when
> >                 <= is execute, first there will be numpy c code
> >                 executed, that will call the python function to decide
> >                 witch version to do, then if it is 2 numpy ndarray, it
> >                 will call again numpy c code.
> >
> >
> >                 That isn't a good solution. We could do the same
> >                 override in C, but then theano work the same when
> >                 there isn't a c++ compiler. That isn't nice.
> >
> >
> >                 What do you think of changing them to check for
> >                 __array_priority__ before doing the comparison?
> >
> >         This looks like an oversight and should be fixed.
> >
> >         Chuck
> >
> >
> >         _______________________________________________
> >         NumPy-Discussion mailing list
> >         NumPy-Discussion at scipy.org
> >         http://mail.scipy.org/mailman/listinfo/numpy-discussion
> >
> >
> >
> > _______________________________________________
> > NumPy-Discussion mailing list
> > NumPy-Discussion at scipy.org
> > http://mail.scipy.org/mailman/listinfo/numpy-discussion
>
>
> _______________________________________________
> NumPy-Discussion mailing list
> NumPy-Discussion at scipy.org
> http://mail.scipy.org/mailman/listinfo/numpy-discussion
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20130510/a1cc95ff/attachment.html>

From ralf.gommers at gmail.com  Fri May 10 17:24:07 2013
From: ralf.gommers at gmail.com (Ralf Gommers)
Date: Fri, 10 May 2013 23:24:07 +0200
Subject: [Numpy-discussion] tests not running
In-Reply-To: <B8F5CDE7FC952A4580ABD28F488EF27930DD943A@AG-DCC-MBX11.dsone.3ds.com>
References: <B8F5CDE7FC952A4580ABD28F488EF27930DD943A@AG-DCC-MBX11.dsone.3ds.com>
Message-ID: <CABL7CQgW=mNrz7uRaRbzOr-8tdyxVpRWaYadiTkGT2wmZxWyCw@mail.gmail.com>

On Fri, May 10, 2013 at 9:41 PM, KACVINSKY Tom <Tom.KACVINSKY at 3ds.com>wrote:

> Here is my set up:
>
> Mac OS 10.7.5
> Xcode 4.5.1
> Intel Fortran 12.1
> Python 2.7.3 built from source
> Numpy 1.6.2 built from source, using MKL 11.0
> nose 0.11.4 installed
>
> I run the numpy tests as documented (python -c 'import numpy;
> numpy.test()'), but get this output:
>
> tkacvins at macomsim> python -c 'import numpy; numpy.test()'
> Running unit tests for numpy
> NumPy version 1.6.2
> NumPy is installed in
> /rd/gen/tky/do_not_delete/Python/macpython27/lib/python2.7/site-packages/numpy
> Python version 2.7.3 (default, Oct 10 2012, 14:47:52) [GCC 4.2.1 (Based on
> Apple Inc. build 5658) (LLVM build 2336.9.00)]
> nose version 0.11.4
>
> ----------------------------------------------------------------------
> Ran 0 tests in 0.025s
>
> OK
>
> Any diagnostics or options I can pass to the tests to see what is going
> on?   This is rather odd, I thought the tests would run and possibly fail.
>  I didn't expect 0 tests to run.
>

That's usually due to the executable bit being set on all files during
install (setuptools does this, nose disapproves). Try ``
np.test(extra_argv='--exe')``.

Ralf
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20130510/f40e7b01/attachment.html>

From ndbecker2 at gmail.com  Fri May 10 19:36:00 2013
From: ndbecker2 at gmail.com (Neal Becker)
Date: Fri, 10 May 2013 19:36 -0400
Subject: [Numpy-discussion] 0-dim arrays inconsistency
Message-ID: <kmk08r$3ec$1@ger.gmane.org>

np.array ((0,0))
Out[10]: array([0, 0])  <<< ok, it's 2 dimensional

In [11]: np.array ((0,0)).shape
Out[11]: (2,)  <<< except, it isn't




From njs at pobox.com  Fri May 10 19:42:57 2013
From: njs at pobox.com (Nathaniel Smith)
Date: Fri, 10 May 2013 19:42:57 -0400
Subject: [Numpy-discussion] 0-dim arrays inconsistency
In-Reply-To: <kmk08r$3ec$1@ger.gmane.org>
References: <kmk08r$3ec$1@ger.gmane.org>
Message-ID: <CAPJVwBkhTZF0FCwsqptKB9C9m7zbkgqroyt17HKkUJmXPAND_Q@mail.gmail.com>

Hi Neal,

On Fri, May 10, 2013 at 7:36 PM, Neal Becker <ndbecker2 at gmail.com> wrote:
> np.array ((0,0))
> Out[10]: array([0, 0])  <<< ok, it's 2 dimensional

Think you may have confused yourself :-). It's 1 dimensional with 2 elements...

> In [11]: np.array ((0,0)).shape
> Out[11]: (2,)  <<< except, it isn't

...as per above. (Not sure where the 0-dim part comes in.)

-n


From ndbecker2 at gmail.com  Fri May 10 19:40:29 2013
From: ndbecker2 at gmail.com (Neal Becker)
Date: Fri, 10 May 2013 19:40:29 -0400
Subject: [Numpy-discussion] 0-dim arrays inconsistency
References: <kmk08r$3ec$1@ger.gmane.org>
Message-ID: <kmk0h8$60o$1@ger.gmane.org>

Neal Becker wrote:

> np.array ((0,0))
> Out[10]: array([0, 0])  <<< ok, it's 2 dimensional
> 
> In [11]: np.array ((0,0)).shape
> Out[11]: (2,)  <<< except, it isn't

Sorry for the stupid question - please ignore



From ndbecker2 at gmail.com  Fri May 10 19:57:21 2013
From: ndbecker2 at gmail.com (Neal Becker)
Date: Fri, 10 May 2013 19:57:21 -0400
Subject: [Numpy-discussion] somewhat less stupid problem with 0-d arrays
Message-ID: <kmk1gt$ebo$1@ger.gmane.org>

It would be convenient if in arithmetic 0-d arrays were just ignored - it would
seem to me to be convenient in generic code where a degenerate array is treated
as "nothing"

np.zeros ((0,0)) + np.ones ((2,2))
---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
<ipython-input-17-27af0e0bbc6f> in <module>()
----> 1 np.zeros ((0,0)) + np.ones ((2,2))

ValueError: operands could not be broadcast together with shapes (0,0) (2,2)





From sudheer.joseph at yahoo.com  Sat May 11 03:53:45 2013
From: sudheer.joseph at yahoo.com (Sudheer Joseph)
Date: Sat, 11 May 2013 15:53:45 +0800 (SGT)
Subject: [Numpy-discussion] printing array in tabular form
In-Reply-To: <1368187147.12442.5.camel@farnsworth>
References: <1367918086.35474.YahooMailNeo@web193404.mail.sg3.yahoo.com>
	<086DEFA5-2C1B-41F2-934F-4EDBE249C3D3@astro.physik.uni-goettingen.de>
	<1367979205.1211.YahooMailNeo@web193402.mail.sg3.yahoo.com>
	<1367996025.32622.4.camel@farnsworth>
	<1368086808.92617.YahooMailNeo@web193403.mail.sg3.yahoo.com>
	<CAJhcF=1SpFUcjYzkWk5=98RBjvSq7iZRJKB4jckqOVv6q6GOfA@mail.gmail.com>
	<1368177277.55926.YahooMailNeo@web193405.mail.sg3.yahoo.com>
	<1368187147.12442.5.camel@farnsworth>
Message-ID: <1368258825.43089.YahooMailNeo@web193404.mail.sg3.yahoo.com>

Thank you very much,
????????????????????? These tips with some effort from me should do it for me.
with best regards,
Sudheer

?
***************************************************************
Sudheer Joseph 
Indian National Centre for Ocean Information Services
Ministry of Earth Sciences, Govt. of India
POST BOX NO: 21, IDA Jeedeemetla P.O.
Via Pragathi Nagar,Kukatpally, Hyderabad; Pin:5000 55
Tel:+91-40-23886047(O),Fax:+91-40-23895011(O),
Tel:+91-40-23044600(R),Tel:+91-40-9440832534(Mobile)
E-mail:sjo.India at gmail.com;sudheer.joseph at yahoo.com
Web- http://oppamthadathil.tripod.com
***************************************************************


----- Original Message -----
> From: Henry Gomersall <heng at cantab.net>
> To: Discussion of Numerical Python <numpy-discussion at scipy.org>
> Cc: 
> Sent: Friday, 10 May 2013 5:29 PM
> Subject: Re: [Numpy-discussion] printing array in tabular form
> 
> On Fri, 2013-05-10 at 17:14 +0800, Sudheer Joseph wrote:
>>  Thank you,
>> ? ? ? ? ? ? ?  But I was looking for? a format statement likw
>>  write(*,"(A,5F8.3)")
>>  with best regards,
>>  Sudheer
> 
> How about the following:
> 
> print('IL = ' + (('%d,' * 5)[:-1] + '\n? ?  ') * 5 % 
> tuple(IL))
> 
> If instead of a list IL you had some arbitrary 2D array, a, you could do
> (with 2 lines for clarity):
> 
> print_string = 'a? = ' + (('%d,' * a.shape[1])[:-1] + 
> '\n? ?  ') *
> ? ? ? ? ? ? ? ? ? ? ? ? ? a.shape[0] % tuple(a.T.ravel())
> print(print_string)
> 
> I'll leave it as an exercise for you to put that into a file.
> 
> hen
> 
> _______________________________________________
> NumPy-Discussion mailing list
> NumPy-Discussion at scipy.org
> http://mail.scipy.org/mailman/listinfo/numpy-discussion
> 


From sudheer.joseph at yahoo.com  Sat May 11 03:55:01 2013
From: sudheer.joseph at yahoo.com (Sudheer Joseph)
Date: Sat, 11 May 2013 15:55:01 +0800 (SGT)
Subject: [Numpy-discussion] printing array in tabular form
In-Reply-To: <6C23BD92-9FFF-4E13-8FD1-8B194707C057@astro.physik.uni-goettingen.de>
References: <1367918086.35474.YahooMailNeo@web193404.mail.sg3.yahoo.com>
	<086DEFA5-2C1B-41F2-934F-4EDBE249C3D3@astro.physik.uni-goettingen.de>
	<1367979205.1211.YahooMailNeo@web193402.mail.sg3.yahoo.com>
	<1367996025.32622.4.camel@farnsworth>
	<1368086808.92617.YahooMailNeo@web193403.mail.sg3.yahoo.com>
	<CAJhcF=1SpFUcjYzkWk5=98RBjvSq7iZRJKB4jckqOVv6q6GOfA@mail.gmail.com>
	<1368177277.55926.YahooMailNeo@web193405.mail.sg3.yahoo.com>
	<518CC116.2080307@grinta.net>
	<1368184807.43447.YahooMailNeo@web193405.mail.sg3.yahoo.com>
	<6C23BD92-9FFF-4E13-8FD1-8B194707C057@astro.physik.uni-goettingen.de>
Message-ID: <1368258901.19204.YahooMailNeo@web193404.mail.sg3.yahoo.com>

Thank you Very much,
???????????????????????? This will help me a lot.
with best regards,
Sudheer


?----- Original Message -----

> From: Derek Homeier <derek at astro.physik.uni-goettingen.de>
> To: Discussion of Numerical Python <numpy-discussion at scipy.org>
> Cc: 
> Sent: Friday, 10 May 2013 6:10 PM
> Subject: Re: [Numpy-discussion] printing array in tabular form
> 
> On 10.05.2013, at 1:20PM, Sudheer Joseph <sudheer.joseph at yahoo.com> wrote:
> 
>>  If some one has a quick way I would like to learn from them or get a 
> referecence 
>>  where the formatting part is described which was 
>>  my intention while posting here. As I have been using fortran I just tried 
>>  to use it to explain my requirement
>> 
> Admittedly the formatting options in Python can be confusing to beginners, 
> precisely
> since they are much more powerful than for many other languages. As already 
> pointed
> out, formats of the type '(5i5)' are very common to Fortran programs and 
> thus readily
> supported by the language. np.savetxt is just a convenience function to support 
> a number
> of similarly common output types, and it can create csv, tab-separated, or 
> plenty of other
> outputs from a numpy array just out of the box. 
> But you added to the confusion as you did not make it clear that you were not 
> just requiring
> a plain csv file as your Fortran example would create (and the first version did 
> not even
> have the commas); since this is a rather non-standard form you will just have to 
> write a
> short loop yourself, wether you are using Fortran or Python.
> 
>> ? ? ? ? ? ? ? ? ? ? ? ? ? Infact the program which should read this file 
> requires it in specified format which should look like
>>  IL = 1,2,3,4,5
>> ? ? ? 1,2,3,4,5
>> ? ? ? 1,2,3,4,5
>> 
> The formats are all documented 
> http://docs.python.org/2/library/string.html#format-specification-mini-language
> one important thing to know is that you can pretty much add (i.e. concatenate) 
> them like strings:
> 
> print(("%6s"+4*"%d,"+"%d\n") % (("IL = 
> ",)+tuple(IL[:5])))
> 
> or, perhaps a bit clearer:
> 
> fmt = "%6s"+4*"%d,"+"%d\n"
> print_t = ("IL = ",)+tuple(IL[:5])
> print(fmt % print_t)
> 
> The other important bit to keep in mind is that all arguments have to be passed 
> as tuples.
> This should allow you to write a loop to print with a "header" or an 
> empty header column
> for the subsequent lines as you see fit. 
> Except for the string field which is explicitly formatted "%s" here, 
> this is mostly equivalent
> to the example Henry just posted.
> 
> HTH,
> ??? ??? ??? ??? ??? Derek
> 
> _______________________________________________
> NumPy-Discussion mailing list
> NumPy-Discussion at scipy.org
> http://mail.scipy.org/mailman/listinfo/numpy-discussion
> 


From sebastian at sipsolutions.net  Sat May 11 05:22:28 2013
From: sebastian at sipsolutions.net (Sebastian Berg)
Date: Sat, 11 May 2013 11:22:28 +0200
Subject: [Numpy-discussion] somewhat less stupid problem with 0-d arrays
In-Reply-To: <kmk1gt$ebo$1@ger.gmane.org>
References: <kmk1gt$ebo$1@ger.gmane.org>
Message-ID: <1368264148.2567.19.camel@sebastian-laptop>

On Fri, 2013-05-10 at 19:57 -0400, Neal Becker wrote:
> It would be convenient if in arithmetic 0-d arrays were just ignored - it would
> seem to me to be convenient in generic code where a degenerate array is treated
> as "nothing"
> 

Small naming detail. A 0-d array is an array with exactly one element
and no dimensions, i.e. np.array(0), and behaves mostly like a scalar.
What you have is an empty array with no elements.

> np.zeros ((0,0)) + np.ones ((2,2))
> ---------------------------------------------------------------------------
> ValueError                                Traceback (most recent call last)
> <ipython-input-17-27af0e0bbc6f> in <module>()
> ----> 1 np.zeros ((0,0)) + np.ones ((2,2))
> 
> ValueError: operands could not be broadcast together with shapes (0,0) (2,2)
> 
> 

I am not sure in what general code you need that, it seems weird to me,
since np.zeros((N, N)) + np.ones((2,2)) would also only work if N=1. And
if N=1, it looks like it might be a reduction result.
Empty arrays *do* support most reductions (making them not empty, like
summing them gives 0). And they do broadcast under the normal
broadcasting rules, such that np.zeros((0,0)) + np.zeros((10,1,1)) gives
np.zeros((10,0,0)).  For the most part, they are not a special case and
just work the same as non-empty arrays, which seems right to me.

- Sebastian

> 
> _______________________________________________
> NumPy-Discussion mailing list
> NumPy-Discussion at scipy.org
> http://mail.scipy.org/mailman/listinfo/numpy-discussion
> 




From sudheer.joseph at yahoo.com  Sat May 11 07:31:43 2013
From: sudheer.joseph at yahoo.com (Sudheer Joseph)
Date: Sat, 11 May 2013 19:31:43 +0800 (SGT)
Subject: [Numpy-discussion] printing array in tabular form
In-Reply-To: <518CED6B.8060105@grinta.net>
References: <1367918086.35474.YahooMailNeo@web193404.mail.sg3.yahoo.com>
	<086DEFA5-2C1B-41F2-934F-4EDBE249C3D3@astro.physik.uni-goettingen.de>
	<1367979205.1211.YahooMailNeo@web193402.mail.sg3.yahoo.com>
	<1367996025.32622.4.camel@farnsworth>
	<1368086808.92617.YahooMailNeo@web193403.mail.sg3.yahoo.com>
	<CAJhcF=1SpFUcjYzkWk5=98RBjvSq7iZRJKB4jckqOVv6q6GOfA@mail.gmail.com>
	<1368177277.55926.YahooMailNeo@web193405.mail.sg3.yahoo.com>
	<518CC116.2080307@grinta.net>
	<1368184807.43447.YahooMailNeo@web193405.mail.sg3.yahoo.com>
	<518CED6B.8060105@grinta.net>
Message-ID: <1368271903.11263.YahooMailNeo@web193405.mail.sg3.yahoo.com>

Thank youNicolodi,
??????????????????? I hope I did not hurt you by my little fight.!!
with best regards,
Sudheer

?

----- Original Message -----
> From: Daniele Nicolodi <daniele at grinta.net>
> To: numpy-discussion at scipy.org
> Cc: 
> Sent: Friday, 10 May 2013 6:21 PM
> Subject: Re: [Numpy-discussion] printing array in tabular form
> 
> On 10/05/2013 13:20, Sudheer Joseph wrote:
> 
>>  Hi,
>>  I am trying to learn Python after feeling its utility in coding and
>>  also reading a bit aboutits potential only, please do not put words
>>  in to my mouth like below.
> 
> I didn't put words in your mouth, I simply quoted emails you sent to the
> list and gave my interpretation of what you wrote.
> 
>>>>  Before denigrating a programming language
>> 
>>  If some one has a quick way I would like to learn from them or get a 
> referecence 
>>  where the formatting part is described which was 
>>  my intention while posting here. As I have been using fortran I just tried 
>>  to use it to explain my requirement
> 
> For references about string formatting in Python:
> 
> http://docs.python.org/2/library/string.html#formatstrings
> http://docs.python.org/2/library/stdtypes.html#string-formatting
> 
> for the numpy array to text formatting:
> 
> http://docs.scipy.org/doc/numpy/reference/generated/numpy.savetxt.html
> 
> writing a function to do what you ask is trivial. Unfortunately there is
> no "format the thing as I wish" function.
> 
> If you wish to format numpy arrays preceding them with a variable name,
> the following is a possible solution that gives the same formatting as
> in your example:
> 
> import numpy as np
> import sys
> 
> def format(out, v, name):
> ? ? header = "{} = ".format(name)
> ? ? out.write(header)
> ? ? np.savetxt(out, v, fmt="%d", delimiter=", ",
> ? ? ? ? ? ? ?  newline="\n" + " " * len(header))
> ? ? out.write("\n")
> 
> IL = np.array([range(5), ] * 5)
> format(sys.stdout, IL, "IL")
> 
> 
> Cheers,
> Daniele
> 
> _______________________________________________
> NumPy-Discussion mailing list
> NumPy-Discussion at scipy.org
> http://mail.scipy.org/mailman/listinfo/numpy-discussion
> 


From ndbecker2 at gmail.com  Sat May 11 08:30:21 2013
From: ndbecker2 at gmail.com (Neal Becker)
Date: Sat, 11 May 2013 08:30:21 -0400
Subject: [Numpy-discussion] somewhat less stupid problem with 0-d arrays
References: <kmk1gt$ebo$1@ger.gmane.org>
	<1368264148.2567.19.camel@sebastian-laptop>
Message-ID: <kmldkq$rk3$1@ger.gmane.org>

Sebastian Berg wrote:

> On Fri, 2013-05-10 at 19:57 -0400, Neal Becker wrote:
>> It would be convenient if in arithmetic 0-d arrays were just ignored - it
>> would seem to me to be convenient in generic code where a degenerate array is
>> treated as "nothing"
>> 
> 
> Small naming detail. A 0-d array is an array with exactly one element
> and no dimensions, i.e. np.array(0), and behaves mostly like a scalar.
> What you have is an empty array with no elements.
> 
>> np.zeros ((0,0)) + np.ones ((2,2))
>> ---------------------------------------------------------------------------
>> ValueError                                Traceback (most recent call last)
>> <ipython-input-17-27af0e0bbc6f> in <module>()
>> ----> 1 np.zeros ((0,0)) + np.ones ((2,2))
>> 
>> ValueError: operands could not be broadcast together with shapes (0,0) (2,2)
>> 
>> 
> 
> I am not sure in what general code you need that, it seems weird to me,
> since np.zeros((N, N)) + np.ones((2,2)) would also only work if N=1. And
> if N=1, it looks like it might be a reduction result.
> Empty arrays *do* support most reductions (making them not empty, like
> summing them gives 0). And they do broadcast under the normal
> broadcasting rules, such that np.zeros((0,0)) + np.zeros((10,1,1)) gives
> np.zeros((10,0,0)).  For the most part, they are not a special case and
> just work the same as non-empty arrays, which seems right to me.
> 
> - Sebastian
> 

OK, my code looks like this:

results = np.dot (a, b) + np.dot (c, d)

I have a case where I want to basically "turn off" that second dot product, and 
I thought if c and d were 0-size it should have that effect.



From sebastian at sipsolutions.net  Sat May 11 09:17:18 2013
From: sebastian at sipsolutions.net (Sebastian Berg)
Date: Sat, 11 May 2013 15:17:18 +0200
Subject: [Numpy-discussion] somewhat less stupid problem with 0-d arrays
In-Reply-To: <kmldkq$rk3$1@ger.gmane.org>
References: <kmk1gt$ebo$1@ger.gmane.org>
	<1368264148.2567.19.camel@sebastian-laptop>
	<kmldkq$rk3$1@ger.gmane.org>
Message-ID: <1368278238.2567.31.camel@sebastian-laptop>

On Sat, 2013-05-11 at 08:30 -0400, Neal Becker wrote:
> Sebastian Berg wrote:
> 
> > On Fri, 2013-05-10 at 19:57 -0400, Neal Becker wrote:
> >> It would be convenient if in arithmetic 0-d arrays were just ignored - it
> >> would seem to me to be convenient in generic code where a degenerate array is
> >> treated as "nothing"
> >> 
> > 
> > Small naming detail. A 0-d array is an array with exactly one element
> > and no dimensions, i.e. np.array(0), and behaves mostly like a scalar.
> > What you have is an empty array with no elements.
> > 
> >> np.zeros ((0,0)) + np.ones ((2,2))
> >> ---------------------------------------------------------------------------
> >> ValueError                                Traceback (most recent call last)
> >> <ipython-input-17-27af0e0bbc6f> in <module>()
> >> ----> 1 np.zeros ((0,0)) + np.ones ((2,2))
> >> 
> >> ValueError: operands could not be broadcast together with shapes (0,0) (2,2)
> >> 
> >> 
> > 
> > I am not sure in what general code you need that, it seems weird to me,
> > since np.zeros((N, N)) + np.ones((2,2)) would also only work if N=1. And
> > if N=1, it looks like it might be a reduction result.
> > Empty arrays *do* support most reductions (making them not empty, like
> > summing them gives 0). And they do broadcast under the normal
> > broadcasting rules, such that np.zeros((0,0)) + np.zeros((10,1,1)) gives
> > np.zeros((10,0,0)).  For the most part, they are not a special case and
> > just work the same as non-empty arrays, which seems right to me.
> > 
> > - Sebastian
> > 
> 
> OK, my code looks like this:
> 
> results = np.dot (a, b) + np.dot (c, d)
> 
> I have a case where I want to basically "turn off" that second dot product, and 
> I thought if c and d were 0-size it should have that effect.
> 

OK, I wouldn't consider that a valid use case to be honest. You could
maybe just set c = d = 0. Heck, you can even have c = np.empty((2, 0)),
d = np.empty((0, 2)) which means that np.dot(c, d) is np.zeros((2,2)),
which for the sake of addition works the same.

But 0-size arrays have way to much meaning to make them identity
operators.

- Sebastian

> _______________________________________________
> NumPy-Discussion mailing list
> NumPy-Discussion at scipy.org
> http://mail.scipy.org/mailman/listinfo/numpy-discussion
> 




From robert.kern at gmail.com  Sat May 11 10:00:31 2013
From: robert.kern at gmail.com (Robert Kern)
Date: Sat, 11 May 2013 15:00:31 +0100
Subject: [Numpy-discussion] somewhat less stupid problem with 0-d arrays
In-Reply-To: <kmk1gt$ebo$1@ger.gmane.org>
References: <kmk1gt$ebo$1@ger.gmane.org>
Message-ID: <CAF6FJittaBbUDNgOs1y+oFUh9RGfrMZLxccc9RZOdLik=Z+ogg@mail.gmail.com>

On Sat, May 11, 2013 at 12:57 AM, Neal Becker <ndbecker2 at gmail.com> wrote:
> It would be convenient if in arithmetic 0-d arrays were just ignored - it would
> seem to me to be convenient in generic code where a degenerate array is treated
> as "nothing"
>
> np.zeros ((0,0)) + np.ones ((2,2))
> ---------------------------------------------------------------------------
> ValueError                                Traceback (most recent call last)
> <ipython-input-17-27af0e0bbc6f> in <module>()
> ----> 1 np.zeros ((0,0)) + np.ones ((2,2))
>
> ValueError: operands could not be broadcast together with shapes (0,0) (2,2)

What would be the result of the following?

  np.zeros((0,0)) - np.ones((2,2))

Would it have positive or negative values? I don't think we can
properly interpret that expression, so I think that the current
behavior is sound.

--
Robert Kern


From sebastian at sipsolutions.net  Sat May 11 11:41:25 2013
From: sebastian at sipsolutions.net (Sebastian Berg)
Date: Sat, 11 May 2013 17:41:25 +0200
Subject: [Numpy-discussion] slight MapIter change
Message-ID: <1368286885.2567.34.camel@sebastian-laptop>

Hey,

(this is only interesting if you know what MapIter and actually use it)

In case anyone already uses the newly exposed mapiter (it was never
released yet). There is a tiny change, which only affects indexes that
start with np.newaxis but otherwise just simplifies a tiny bit. The old
block for swapping axes should be changed like this:

     if ((mit->subspace != NULL) && (mit->consec)) {
-        if (mit->iteraxes[0] > 0) {
-            PyArray_MapIterSwapAxes(mit, (PyArrayObject **)&arr, 0);
-            if (arr == NULL) {
-                return -1;
-            }
+        PyArray_MapIterSwapAxes(mit, (PyArrayObject **)&arr, 0);
+        if (arr == NULL) {
+            return -1;
         }
     }

Regards,

Sebastian



From thomas.robitaille at gmail.com  Sun May 12 03:59:22 2013
From: thomas.robitaille at gmail.com (Thomas Robitaille)
Date: Sun, 12 May 2013 08:59:22 +0100
Subject: [Numpy-discussion] __array_priority__ don't work for gt, lt,
	... operator
In-Reply-To: <CAB6mnxLZP-3+7pX1K5NK7LABccwqBT0UQb1p8whXXtEGh+J8RA@mail.gmail.com>
References: <CADKKbtgf8XPgmRYhvA5KYABCbUWV8V+kpKT8v_bjbi16EBAqVw@mail.gmail.com>
	<CAB6mnxLZP-3+7pX1K5NK7LABccwqBT0UQb1p8whXXtEGh+J8RA@mail.gmail.com>
Message-ID: <CAGMHX_0S_kGCmmFeDRcgbTpxomWwadedoAkmmEmo6QHTbZtuyA@mail.gmail.com>

I've also been having issues with __array_priority__ - the following
code behaves differently for __mul__ and __rmul__:

"""
import numpy as np


class TestClass(object):

    def __init__(self, input_array):
        self.array = input_array

    def __mul__(self, other):
        print "Called __mul__"

    def __rmul__(self, other):
        print "Called __rmul__"

    def __array_wrap__(self, out_arr, context=None):
        print "Called __array_wrap__"
        return TestClass(out_arr)

    def __array__(self):
        print "Called __array__"
        return np.array(self.array)
"""

with output:

"""
In [7]: a = TestClass([1,2,3])

In [8]: print type(np.array([1,2,3]) * a)
Called __array__
Called __array_wrap__
<class '__main__.TestClass'>

In [9]: print type(a * np.array([1,2,3]))
Called __mul__
<type 'NoneType'>
"""

Is this also an oversight? I opened a ticket for it a little while ago:

https://github.com/numpy/numpy/issues/3164

Any ideas?

Cheers,
Tom


On 10 May 2013 18:34, Charles R Harris <charlesr.harris at gmail.com> wrote:
>
>
> On Fri, May 10, 2013 at 10:08 AM, Fr?d?ric Bastien <nouiz at nouiz.org> wrote:
>>
>> Hi,
>>
>> it popped again on the Theano mailing list that this don't work:
>>
>> np.arange(10) <= a_theano_vector.
>>
>> The reason is that __array_priority__ isn't respected for that class of
>> operation.
>>
>> This page explain the problem and give a work around:
>>
>>
>> http://stackoverflow.com/questions/14619449/how-can-i-override-comparisons-between-numpys-ndarray-and-my-type
>>
>> The work around is to make a python function that will decide witch
>> version of the comparator to call and do the call. Then we tell NumPy to use
>> that function instead of its current function with: np.set_numeric_ops(...)
>>
>> But if we do that, when we import theano, we will slow down all normal
>> numpy comparison for the user, as when <= is execute, first there will be
>> numpy c code executed, that will call the python function to decide witch
>> version to do, then if it is 2 numpy ndarray, it will call again numpy c
>> code.
>>
>> That isn't a good solution. We could do the same override in C, but then
>> theano work the same when there isn't a c++ compiler. That isn't nice.
>>
>> What do you think of changing them to check for __array_priority__ before
>> doing the comparison?
>
>
> This looks like an oversight and should be fixed.
>
> Chuck
>
> _______________________________________________
> NumPy-Discussion mailing list
> NumPy-Discussion at scipy.org
> http://mail.scipy.org/mailman/listinfo/numpy-discussion
>


From nouiz at nouiz.org  Mon May 13 09:58:07 2013
From: nouiz at nouiz.org (=?ISO-8859-1?Q?Fr=E9d=E9ric_Bastien?=)
Date: Mon, 13 May 2013 09:58:07 -0400
Subject: [Numpy-discussion] slight MapIter change
In-Reply-To: <1368286885.2567.34.camel@sebastian-laptop>
References: <1368286885.2567.34.camel@sebastian-laptop>
Message-ID: <CADKKbtjqwARKxDCc9eRkY=p33wrd0wRT-+UPByn5ktmnpv2YTA@mail.gmail.com>

Hi,

this is used in Theano. What is the consequence of not doing this? There is
people that use it, the question is how many.
Is there a way to detect witch version need to be used?

thanks

Fred


On Sat, May 11, 2013 at 11:41 AM, Sebastian Berg <sebastian at sipsolutions.net
> wrote:

> Hey,
>
> (this is only interesting if you know what MapIter and actually use it)
>
> In case anyone already uses the newly exposed mapiter (it was never
> released yet). There is a tiny change, which only affects indexes that
> start with np.newaxis but otherwise just simplifies a tiny bit. The old
> block for swapping axes should be changed like this:
>
>      if ((mit->subspace != NULL) && (mit->consec)) {
> -        if (mit->iteraxes[0] > 0) {
> -            PyArray_MapIterSwapAxes(mit, (PyArrayObject **)&arr, 0);
> -            if (arr == NULL) {
> -                return -1;
> -            }
> +        PyArray_MapIterSwapAxes(mit, (PyArrayObject **)&arr, 0);
> +        if (arr == NULL) {
> +            return -1;
>          }
>      }
>
> Regards,
>
> Sebastian
>
> _______________________________________________
> NumPy-Discussion mailing list
> NumPy-Discussion at scipy.org
> http://mail.scipy.org/mailman/listinfo/numpy-discussion
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20130513/477e99a5/attachment.html>

From sebastian at sipsolutions.net  Mon May 13 10:19:07 2013
From: sebastian at sipsolutions.net (Sebastian Berg)
Date: Mon, 13 May 2013 16:19:07 +0200
Subject: [Numpy-discussion] slight MapIter change
In-Reply-To: <CADKKbtjqwARKxDCc9eRkY=p33wrd0wRT-+UPByn5ktmnpv2YTA@mail.gmail.com>
References: <1368286885.2567.34.camel@sebastian-laptop>
	<CADKKbtjqwARKxDCc9eRkY=p33wrd0wRT-+UPByn5ktmnpv2YTA@mail.gmail.com>
Message-ID: <1368454747.2035.15.camel@sebastian-laptop>

On Mon, 2013-05-13 at 09:58 -0400, Fr?d?ric Bastien wrote:
> Hi,
> 
> this is used in Theano. What is the consequence of not doing this?
> There is people that use it, the question is how many.
> 
There are no consequences. Only if you use the equivalent to
`array[np.newaxis, fancy_index, <possibly more>]` the result would be
wrong. But all code that used to work will continue to work, since such
an index was not legal before anyway.

> Is there a way to detect witch version need to be used?
> 
There is no released version of numpy with the other definition. Also
changing it should be safe even for someone who has an older NumPy
development version, because as far as I can tell the check is only an
optimization in the first place.

So just remove that check and you are good. And even if someone uses a
new numpy with an old thaeno development version, they would have to do
weird things to run into problems.

- Sebastian


> thanks
> 
> Fred
> 
> 
> 
> On Sat, May 11, 2013 at 11:41 AM, Sebastian Berg
> <sebastian at sipsolutions.net> wrote:
>         Hey,
>         
>         (this is only interesting if you know what MapIter and
>         actually use it)
>         
>         In case anyone already uses the newly exposed mapiter (it was
>         never
>         released yet). There is a tiny change, which only affects
>         indexes that
>         start with np.newaxis but otherwise just simplifies a tiny
>         bit. The old
>         block for swapping axes should be changed like this:
>         
>              if ((mit->subspace != NULL) && (mit->consec)) {
>         -        if (mit->iteraxes[0] > 0) {
>         -            PyArray_MapIterSwapAxes(mit, (PyArrayObject
>         **)&arr, 0);
>         -            if (arr == NULL) {
>         -                return -1;
>         -            }
>         +        PyArray_MapIterSwapAxes(mit, (PyArrayObject **)&arr,
>         0);
>         +        if (arr == NULL) {
>         +            return -1;
>                  }
>              }
>         
>         Regards,
>         
>         Sebastian
>         
>         _______________________________________________
>         NumPy-Discussion mailing list
>         NumPy-Discussion at scipy.org
>         http://mail.scipy.org/mailman/listinfo/numpy-discussion
> 
> 
> _______________________________________________
> NumPy-Discussion mailing list
> NumPy-Discussion at scipy.org
> http://mail.scipy.org/mailman/listinfo/numpy-discussion




From nouiz at nouiz.org  Mon May 13 13:10:31 2013
From: nouiz at nouiz.org (=?ISO-8859-1?Q?Fr=E9d=E9ric_Bastien?=)
Date: Mon, 13 May 2013 13:10:31 -0400
Subject: [Numpy-discussion] slight MapIter change
In-Reply-To: <1368454747.2035.15.camel@sebastian-laptop>
References: <1368286885.2567.34.camel@sebastian-laptop>
	<CADKKbtjqwARKxDCc9eRkY=p33wrd0wRT-+UPByn5ktmnpv2YTA@mail.gmail.com>
	<1368454747.2035.15.camel@sebastian-laptop>
Message-ID: <CADKKbtjHUt7eH9Z7M7r4W_9CO4pvHrU5Lr=BkA=r03an5uek4Q@mail.gmail.com>

ok,

thanks for the associated PR.

Fred


On Mon, May 13, 2013 at 10:19 AM, Sebastian Berg <sebastian at sipsolutions.net
> wrote:

> On Mon, 2013-05-13 at 09:58 -0400, Fr?d?ric Bastien wrote:
> > Hi,
> >
> > this is used in Theano. What is the consequence of not doing this?
> > There is people that use it, the question is how many.
> >
> There are no consequences. Only if you use the equivalent to
> `array[np.newaxis, fancy_index, <possibly more>]` the result would be
> wrong. But all code that used to work will continue to work, since such
> an index was not legal before anyway.
>
> > Is there a way to detect witch version need to be used?
> >
> There is no released version of numpy with the other definition. Also
> changing it should be safe even for someone who has an older NumPy
> development version, because as far as I can tell the check is only an
> optimization in the first place.
>
> So just remove that check and you are good. And even if someone uses a
> new numpy with an old thaeno development version, they would have to do
> weird things to run into problems.
>
> - Sebastian
>
>
> > thanks
> >
> > Fred
> >
> >
> >
> > On Sat, May 11, 2013 at 11:41 AM, Sebastian Berg
> > <sebastian at sipsolutions.net> wrote:
> >         Hey,
> >
> >         (this is only interesting if you know what MapIter and
> >         actually use it)
> >
> >         In case anyone already uses the newly exposed mapiter (it was
> >         never
> >         released yet). There is a tiny change, which only affects
> >         indexes that
> >         start with np.newaxis but otherwise just simplifies a tiny
> >         bit. The old
> >         block for swapping axes should be changed like this:
> >
> >              if ((mit->subspace != NULL) && (mit->consec)) {
> >         -        if (mit->iteraxes[0] > 0) {
> >         -            PyArray_MapIterSwapAxes(mit, (PyArrayObject
> >         **)&arr, 0);
> >         -            if (arr == NULL) {
> >         -                return -1;
> >         -            }
> >         +        PyArray_MapIterSwapAxes(mit, (PyArrayObject **)&arr,
> >         0);
> >         +        if (arr == NULL) {
> >         +            return -1;
> >                  }
> >              }
> >
> >         Regards,
> >
> >         Sebastian
> >
> >         _______________________________________________
> >         NumPy-Discussion mailing list
> >         NumPy-Discussion at scipy.org
> >         http://mail.scipy.org/mailman/listinfo/numpy-discussion
> >
> >
> > _______________________________________________
> > NumPy-Discussion mailing list
> > NumPy-Discussion at scipy.org
> > http://mail.scipy.org/mailman/listinfo/numpy-discussion
>
>
> _______________________________________________
> NumPy-Discussion mailing list
> NumPy-Discussion at scipy.org
> http://mail.scipy.org/mailman/listinfo/numpy-discussion
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20130513/3bc910ce/attachment.html>

From evgeny.toder at jpmorgan.com  Tue May 14 13:26:13 2013
From: evgeny.toder at jpmorgan.com (Toder, Evgeny)
Date: Tue, 14 May 2013 17:26:13 +0000
Subject: [Numpy-discussion] Integer overflow in test_einsum (1.7.1)
Message-ID: <C0B518458D7A2B479B898CA53524926E361D6461@SBECMX010.exchad.jpmchase.net>

Hello,

One of the test cases in test_einsum causes integer overflow for i2 type. The test goes like this:

>>> import numpy as np
>>> dtype = 'i2'
>>> n = 15
>>> a = np.arange(4*n, dtype=dtype).reshape(4,n)
>>> b = np.arange(n*6, dtype=dtype).reshape(n,6)
>>> c = np.arange(24, dtype=dtype).reshape(4,6)

It then calculates AxB using einsum. The problem is that the values in the last row of the result do not fit into i2:

>>> np.einsum("ij,jk", a, b, dtype='f8', casting='unsafe')
array([[  6090.,   6195.,   6300.,   6405.,   6510.,   6615.],
       [ 15540.,  15870.,  16200.,  16530.,  16860.,  17190.],
       [ 24990.,  25545.,  26100.,  26655.,  27210.,  27765.],
       [ 34440.,  35220.,  36000.,  36780.,  37560.,  38340.]])

In my build this produces different results depending on whether out or .astype is used:

>>> np.einsum("ij,jk", a, b, dtype='f8', casting='unsafe').astype(dtype)
array([[  6090,   6195,   6300,   6405,   6510,   6615],
       [ 15540,  15870,  16200,  16530,  16860,  17190],
       [ 24990,  25545,  26100,  26655,  27210,  27765],
       [-31096, -30316, -29536, -28756, -27976, -27196]], dtype=int16)

>>> np.einsum("ij,jk", a, b, out=c, dtype='f8', casting='unsafe')
array([[  6090,   6195,   6300,   6405,   6510,   6615],
       [ 15540,  15870,  16200,  16530,  16860,  17190],
       [ 24990,  25545,  26100,  26655,  27210,  27765],
       [-32768, -32768, -32768, -32768, -32768, -32768]], dtype=int16)

The test wants these (actually the same using numpy.dot) to be equal, so this difference causes it to fail. Both ways to handle overflow seem reasonable to me.

Does numpy in general assign a defined behavior to integer overflow (e.g. two's complement)?
Is this use of integer overflow in the test intentional and is expected to work, or is my build broken?

Best regards,
Eugene


This email is confidential and subject to important disclaimers and
conditions including on offers for the purchase or sale of
securities, accuracy and completeness of information, viruses,
confidentiality, legal privilege, and legal entity disclaimers,
available at http://www.jpmorgan.com/pages/disclosures/email.  
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20130514/b29f3a94/attachment.html>

From charlesr.harris at gmail.com  Tue May 14 14:25:14 2013
From: charlesr.harris at gmail.com (Charles R Harris)
Date: Tue, 14 May 2013 12:25:14 -0600
Subject: [Numpy-discussion] Integer overflow in test_einsum (1.7.1)
In-Reply-To: <C0B518458D7A2B479B898CA53524926E361D6461@SBECMX010.exchad.jpmchase.net>
References: <C0B518458D7A2B479B898CA53524926E361D6461@SBECMX010.exchad.jpmchase.net>
Message-ID: <CAB6mnxKsUfhgVUp2UA+PNno_+_hdV16LCidg0jZ9T4PWH+erZQ@mail.gmail.com>

On Tue, May 14, 2013 at 11:26 AM, Toder, Evgeny
<evgeny.toder at jpmorgan.com>wrote:

>  Hello,****
>
> ** **
>
> One of the test cases in test_einsum causes integer overflow for i2 type.
> The test goes like this:****
>
> ** **
>
> >>> import numpy as np****
>
> >>> dtype = 'i2'****
>
> >>> n = 15****
>
> >>> a = np.arange(4*n, dtype=dtype).reshape(4,n)****
>
> >>> b = np.arange(n*6, dtype=dtype).reshape(n,6)****
>
> >>> c = np.arange(24, dtype=dtype).reshape(4,6)****
>
> ** **
>
> It then calculates AxB using einsum. The problem is that the values in the
> last row of the result do not fit into i2:****
>
> ** **
>
> >>> np.einsum("ij,jk", a, b, dtype='f8', casting='unsafe')****
>
> array([[  6090.,   6195.,   6300.,   6405.,   6510.,   6615.],****
>
>        [ 15540.,  15870.,  16200.,  16530.,  16860.,  17190.],****
>
>        [ 24990.,  25545.,  26100.,  26655.,  27210.,  27765.],****
>
>        [ 34440.,  35220.,  36000.,  36780.,  37560.,  38340.]])****
>
> ** **
>
> In my build this produces different results depending on whether out or
> .astype is used:****
>
> ** **
>
> >>> np.einsum("ij,jk", a, b, dtype='f8', casting='unsafe').astype(dtype)**
> **
>
> array([[  6090,   6195,   6300,   6405,   6510,   6615],****
>
>        [ 15540,  15870,  16200,  16530,  16860,  17190],****
>
>        [ 24990,  25545,  26100,  26655,  27210,  27765],****
>
>        [-31096, -30316, -29536, -28756, -27976, -27196]], dtype=int16)****
>
> ** **
>
> >>> np.einsum("ij,jk", a, b, out=c, dtype='f8', casting='unsafe')****
>
> array([[  6090,   6195,   6300,   6405,   6510,   6615],****
>
>        [ 15540,  15870,  16200,  16530,  16860,  17190],****
>
>        [ 24990,  25545,  26100,  26655,  27210,  27765],****
>
>        [-32768, -32768, -32768, -32768, -32768, -32768]], dtype=int16)****
>
> ** **
>
> The test wants these (actually the same using numpy.dot) to be equal, so
> this difference causes it to fail. Both ways to handle overflow seem
> reasonable to me. ****
>
> ** **
>
> Does numpy in general assign a defined behavior to integer overflow (e.g.
> two?s complement)?****
>
> Is this use of integer overflow in the test intentional and is expected to
> work, or is my build broken?****
>

I don't know if it was intended in the test, but numpy doesn't guard
against integer overflow. Rather, it takes the C approach and does modular
arithmetic. I don't know how to fix that or even if it could be at this
point given the historical lack of support. I suppose at some point there
could be user type that made such checks, but it would be slower.

Chuck
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20130514/bf2c45dc/attachment.html>

From evgeny.toder at jpmorgan.com  Tue May 14 14:52:51 2013
From: evgeny.toder at jpmorgan.com (Toder, Evgeny)
Date: Tue, 14 May 2013 18:52:51 +0000
Subject: [Numpy-discussion] Integer overflow in test_einsum (1.7.1)
In-Reply-To: <CAB6mnxKsUfhgVUp2UA+PNno_+_hdV16LCidg0jZ9T4PWH+erZQ@mail.gmail.com>
References: <C0B518458D7A2B479B898CA53524926E361D6461@SBECMX010.exchad.jpmchase.net>
	<CAB6mnxKsUfhgVUp2UA+PNno_+_hdV16LCidg0jZ9T4PWH+erZQ@mail.gmail.com>
Message-ID: <C0B518458D7A2B479B898CA53524926E361D679D@SBECMX010.exchad.jpmchase.net>

C only does modular arithmetic for unsigned types. Signed types (like i2 used here, is should be signed short) are not allowed to overflow in C. I.e. not only  is the result not guaranteed to be modular, or not guaranteed to be consistent, the behavior of the whole program is not specified if it allows signed integer overflow. In practice, your often get two's complement arithmetic, because this is what all current CPUs implement.

So, does numpy implement modular (two's complement) arithmetic for signed types, or does it leave it to C compiler, which does not provide any guarantees?

Eugene

From: numpy-discussion-bounces at scipy.org [mailto:numpy-discussion-bounces at scipy.org] On Behalf Of Charles R Harris
Sent: Tuesday, May 14, 2013 2:25 PM
To: Discussion of Numerical Python
Subject: Re: [Numpy-discussion] Integer overflow in test_einsum (1.7.1)


I don't know if it was intended in the test, but numpy doesn't guard against integer overflow. Rather, it takes the C approach and does modular arithmetic. I don't know how to fix that or even if it could be at this point given the historical lack of support. I suppose at some point there could be user type that made such checks, but it would be slower.

Chuck


This email is confidential and subject to important disclaimers and
conditions including on offers for the purchase or sale of
securities, accuracy and completeness of information, viruses,
confidentiality, legal privilege, and legal entity disclaimers,
available at http://www.jpmorgan.com/pages/disclosures/email.  
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20130514/d2952bbb/attachment.html>

From pav at iki.fi  Tue May 14 15:31:13 2013
From: pav at iki.fi (Pauli Virtanen)
Date: Tue, 14 May 2013 22:31:13 +0300
Subject: [Numpy-discussion] Integer overflow in test_einsum (1.7.1)
In-Reply-To: <C0B518458D7A2B479B898CA53524926E361D679D@SBECMX010.exchad.jpmchase.net>
References: <C0B518458D7A2B479B898CA53524926E361D6461@SBECMX010.exchad.jpmchase.net>
	<CAB6mnxKsUfhgVUp2UA+PNno_+_hdV16LCidg0jZ9T4PWH+erZQ@mail.gmail.com>
	<C0B518458D7A2B479B898CA53524926E361D679D@SBECMX010.exchad.jpmchase.net>
Message-ID: <kmu3ds$6ee$1@ger.gmane.org>

14.05.2013 21:52, Toder, Evgeny kirjoitti:
> So, does numpy implement modular (two?s complement)
> arithmetic for signed types
[clip]

Numpy leaves integer arithmetic to the C compiler.

Python and its C modules however by default specify -fwrapv for gcc and
its ilk, so the behavior might seem consistent on these platforms.

-- 
Pauli Virtanen



From evgeny.toder at jpmorgan.com  Tue May 14 15:53:24 2013
From: evgeny.toder at jpmorgan.com (Toder, Evgeny)
Date: Tue, 14 May 2013 19:53:24 +0000
Subject: [Numpy-discussion] Integer overflow in test_einsum (1.7.1)
In-Reply-To: <kmu3ds$6ee$1@ger.gmane.org>
References: <C0B518458D7A2B479B898CA53524926E361D6461@SBECMX010.exchad.jpmchase.net>
	<CAB6mnxKsUfhgVUp2UA+PNno_+_hdV16LCidg0jZ9T4PWH+erZQ@mail.gmail.com>
	<C0B518458D7A2B479B898CA53524926E361D679D@SBECMX010.exchad.jpmchase.net>
	<kmu3ds$6ee$1@ger.gmane.org>
Message-ID: <C0B518458D7A2B479B898CA53524926E361D67FB@SBECMX010.exchad.jpmchase.net>

Thank you, Pauli. I guess this means that the test needs to be fixed?

Eugene

-----Original Message-----
From: numpy-discussion-bounces at scipy.org [mailto:numpy-discussion-bounces at scipy.org] On Behalf Of Pauli Virtanen
Sent: Tuesday, May 14, 2013 3:31 PM
To: numpy-discussion at scipy.org
Subject: Re: [Numpy-discussion] Integer overflow in test_einsum (1.7.1)

14.05.2013 21:52, Toder, Evgeny kirjoitti:
> So, does numpy implement modular (two's complement)
> arithmetic for signed types
[clip]

Numpy leaves integer arithmetic to the C compiler.

Python and its C modules however by default specify -fwrapv for gcc and
its ilk, so the behavior might seem consistent on these platforms.

-- 
Pauli Virtanen

_______________________________________________
NumPy-Discussion mailing list
NumPy-Discussion at scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion
This email is confidential and subject to important disclaimers and
conditions including on offers for the purchase or sale of
securities, accuracy and completeness of information, viruses,
confidentiality, legal privilege, and legal entity disclaimers,
available at http://www.jpmorgan.com/pages/disclosures/email.  


From nouiz at nouiz.org  Wed May 15 13:56:27 2013
From: nouiz at nouiz.org (=?ISO-8859-1?Q?Fr=E9d=E9ric_Bastien?=)
Date: Wed, 15 May 2013 13:56:27 -0400
Subject: [Numpy-discussion] __array_priority__ don't work for gt, lt,
	... operator
In-Reply-To: <CAGMHX_0S_kGCmmFeDRcgbTpxomWwadedoAkmmEmo6QHTbZtuyA@mail.gmail.com>
References: <CADKKbtgf8XPgmRYhvA5KYABCbUWV8V+kpKT8v_bjbi16EBAqVw@mail.gmail.com>
	<CAB6mnxLZP-3+7pX1K5NK7LABccwqBT0UQb1p8whXXtEGh+J8RA@mail.gmail.com>
	<CAGMHX_0S_kGCmmFeDRcgbTpxomWwadedoAkmmEmo6QHTbZtuyA@mail.gmail.com>
Message-ID: <CADKKbtgqDSYcw=rcn0JGXoMK30zv-JEs9KL8GsV6Zzd6SoNq4A@mail.gmail.com>

This is a different issue then mine. Mine is that array_priority is not
implemented for comparison.

Your is that array_priority isn't used when __array__ is defined.

Maybe you can make a new mailing list thread? Issue get more attention when
there is an associated email. I looked rapidly in numpy code, but didn't
find the problem. So you will need to find someone with more time/knowledge
about this to look at it or take a look yourself.

HTH

Fred


On Sun, May 12, 2013 at 3:59 AM, Thomas Robitaille <
thomas.robitaille at gmail.com> wrote:

> I've also been having issues with __array_priority__ - the following
> code behaves differently for __mul__ and __rmul__:
>
> """
> import numpy as np
>
>
> class TestClass(object):
>
>     def __init__(self, input_array):
>         self.array = input_array
>
>     def __mul__(self, other):
>         print "Called __mul__"
>
>     def __rmul__(self, other):
>         print "Called __rmul__"
>
>     def __array_wrap__(self, out_arr, context=None):
>         print "Called __array_wrap__"
>         return TestClass(out_arr)
>
>     def __array__(self):
>         print "Called __array__"
>         return np.array(self.array)
> """
>
> with output:
>
> """
> In [7]: a = TestClass([1,2,3])
>
> In [8]: print type(np.array([1,2,3]) * a)
> Called __array__
> Called __array_wrap__
> <class '__main__.TestClass'>
>
> In [9]: print type(a * np.array([1,2,3]))
> Called __mul__
> <type 'NoneType'>
> """
>
> Is this also an oversight? I opened a ticket for it a little while ago:
>
> https://github.com/numpy/numpy/issues/3164
>
> Any ideas?
>
> Cheers,
> Tom
>
>
> On 10 May 2013 18:34, Charles R Harris <charlesr.harris at gmail.com> wrote:
> >
> >
> > On Fri, May 10, 2013 at 10:08 AM, Fr?d?ric Bastien <nouiz at nouiz.org>
> wrote:
> >>
> >> Hi,
> >>
> >> it popped again on the Theano mailing list that this don't work:
> >>
> >> np.arange(10) <= a_theano_vector.
> >>
> >> The reason is that __array_priority__ isn't respected for that class of
> >> operation.
> >>
> >> This page explain the problem and give a work around:
> >>
> >>
> >>
> http://stackoverflow.com/questions/14619449/how-can-i-override-comparisons-between-numpys-ndarray-and-my-type
> >>
> >> The work around is to make a python function that will decide witch
> >> version of the comparator to call and do the call. Then we tell NumPy
> to use
> >> that function instead of its current function with:
> np.set_numeric_ops(...)
> >>
> >> But if we do that, when we import theano, we will slow down all normal
> >> numpy comparison for the user, as when <= is execute, first there will
> be
> >> numpy c code executed, that will call the python function to decide
> witch
> >> version to do, then if it is 2 numpy ndarray, it will call again numpy c
> >> code.
> >>
> >> That isn't a good solution. We could do the same override in C, but then
> >> theano work the same when there isn't a c++ compiler. That isn't nice.
> >>
> >> What do you think of changing them to check for __array_priority__
> before
> >> doing the comparison?
> >
> >
> > This looks like an oversight and should be fixed.
> >
> > Chuck
> >
> > _______________________________________________
> > NumPy-Discussion mailing list
> > NumPy-Discussion at scipy.org
> > http://mail.scipy.org/mailman/listinfo/numpy-discussion
> >
> _______________________________________________
> NumPy-Discussion mailing list
> NumPy-Discussion at scipy.org
> http://mail.scipy.org/mailman/listinfo/numpy-discussion
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20130515/f641743f/attachment.html>

From martin.raspaud at smhi.se  Thu May 16 03:35:22 2013
From: martin.raspaud at smhi.se (Martin Raspaud)
Date: Thu, 16 May 2013 09:35:22 +0200
Subject: [Numpy-discussion] Strange memory consumption in numpy?
Message-ID: <51948C3A.6000404@smhi.se>

Hi all,

In the context of memory profiling an application (with memory_profiler
module) we came up a strange behaviour in numpy, see for yourselves:

Line #    Mem usage    Increment   Line Contents
================================================
    29                             @profile
    30    23.832 MB     0.000 MB   def main():
    31    46.730 MB    22.898 MB       arr1 = np.random.rand(1000000, 3)
    32    58.180 MB    11.449 MB       arr1s = arr1.astype(np.float32)
    33    35.289 MB   -22.891 MB       del arr1
    34    35.289 MB     0.000 MB       gc.collect()
    35    58.059 MB    22.770 MB       arr2 = np.random.rand(1000000, 3)
    36    69.500 MB    11.441 MB       arr2s = arr2.astype(np.float32)
    37    69.500 MB     0.000 MB       del arr2
    38    69.500 MB     0.000 MB       gc.collect()
    39    69.500 MB     0.000 MB       arr3 = np.random.rand(1000000, 3)
    40    80.945 MB    11.445 MB       arr3s = arr3.astype(np.float32)
    41    80.945 MB     0.000 MB       del arr3
    42    80.945 MB     0.000 MB       gc.collect()
    43    80.945 MB     0.000 MB       return arr1s, arr2s, arr3s


The lines 31-34 are behaving as expected, but then we don't understand
35-38 (why is arr2 not garbage collected ?) and 39-42 (why doesn't the
random allocate any memory ?).

Can anyone give a reasonable explanation ?

I attach the full script for reference.

Best regards,
Martin
-------------- next part --------------
A non-text attachment was scrubbed...
Name: testnumpymem.py
Type: text/x-python
Size: 1235 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20130516/1f0d95dd/attachment.py>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: martin_raspaud.vcf
Type: text/x-vcard
Size: 303 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20130516/1f0d95dd/attachment.vcf>

From robert.kern at gmail.com  Thu May 16 04:26:56 2013
From: robert.kern at gmail.com (Robert Kern)
Date: Thu, 16 May 2013 09:26:56 +0100
Subject: [Numpy-discussion] Strange memory consumption in numpy?
In-Reply-To: <51948C3A.6000404@smhi.se>
References: <51948C3A.6000404@smhi.se>
Message-ID: <CAF6FJiuJt7i+kYsJcXuF_0SJ-iLP5AZKakhsWH2ZoKvZ+knK7A@mail.gmail.com>

On Thu, May 16, 2013 at 8:35 AM, Martin Raspaud <martin.raspaud at smhi.se> wrote:
> Hi all,
>
> In the context of memory profiling an application (with memory_profiler
> module) we came up a strange behaviour in numpy, see for yourselves:
>
> Line #    Mem usage    Increment   Line Contents
> ================================================
>     29                             @profile
>     30    23.832 MB     0.000 MB   def main():
>     31    46.730 MB    22.898 MB       arr1 = np.random.rand(1000000, 3)
>     32    58.180 MB    11.449 MB       arr1s = arr1.astype(np.float32)
>     33    35.289 MB   -22.891 MB       del arr1
>     34    35.289 MB     0.000 MB       gc.collect()
>     35    58.059 MB    22.770 MB       arr2 = np.random.rand(1000000, 3)
>     36    69.500 MB    11.441 MB       arr2s = arr2.astype(np.float32)
>     37    69.500 MB     0.000 MB       del arr2
>     38    69.500 MB     0.000 MB       gc.collect()
>     39    69.500 MB     0.000 MB       arr3 = np.random.rand(1000000, 3)
>     40    80.945 MB    11.445 MB       arr3s = arr3.astype(np.float32)
>     41    80.945 MB     0.000 MB       del arr3
>     42    80.945 MB     0.000 MB       gc.collect()
>     43    80.945 MB     0.000 MB       return arr1s, arr2s, arr3s
>
>
> The lines 31-34 are behaving as expected, but then we don't understand
> 35-38 (why is arr2 not garbage collected ?) and 39-42 (why doesn't the
> random allocate any memory ?).
>
> Can anyone give a reasonable explanation ?

memory_profiler only looks at the amount of memory that the OS has
allocated to the Python process. It cannot measure the amount of
memory actually given to living objects. Python does not always return
memory back to the OS immediately when it frees the memory for an
object. Your two observations are linked. Python freed the memory of
arr2 immediately, but it did not return the memory to the OS, so
memory_profiler could not notice it. When arr3 is allocated, it
happened to fit into the block of memory that arr2 once owned, so
Python's memory allocator just used that block again. Since Python did
not have to go out to the OS to get more memory, memory_profiler could
not notice that, either.

--
Robert Kern


From martin.raspaud at smhi.se  Thu May 16 08:32:09 2013
From: martin.raspaud at smhi.se (Martin Raspaud)
Date: Thu, 16 May 2013 14:32:09 +0200
Subject: [Numpy-discussion] Strange memory consumption in numpy?
In-Reply-To: <CAF6FJiuJt7i+kYsJcXuF_0SJ-iLP5AZKakhsWH2ZoKvZ+knK7A@mail.gmail.com>
References: <51948C3A.6000404@smhi.se>
	<CAF6FJiuJt7i+kYsJcXuF_0SJ-iLP5AZKakhsWH2ZoKvZ+knK7A@mail.gmail.com>
Message-ID: <5194D1C9.1060900@smhi.se>

On 16/05/13 10:26, Robert Kern wrote:

>> Can anyone give a reasonable explanation ?
> 
> memory_profiler only looks at the amount of memory that the OS has
> allocated to the Python process. It cannot measure the amount of
> memory actually given to living objects. Python does not always return
> memory back to the OS immediately when it frees the memory for an
> object. Your two observations are linked. Python freed the memory of
> arr2 immediately, but it did not return the memory to the OS, so
> memory_profiler could not notice it. When arr3 is allocated, it
> happened to fit into the block of memory that arr2 once owned, so
> Python's memory allocator just used that block again. Since Python did
> not have to go out to the OS to get more memory, memory_profiler could
> not notice that, either.

Robert,

Thanks a lot for the clear explanation, it makes perfect sense now.

You're talking about living objects, but as I understand the few memory
profilers I found around the web for python can't track numpy arrays.
Any pointers on something that would work with numpy ?

Best regards,
Martin
-------------- next part --------------
A non-text attachment was scrubbed...
Name: martin_raspaud.vcf
Type: text/x-vcard
Size: 303 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20130516/107279db/attachment.vcf>

From robert.kern at gmail.com  Thu May 16 08:44:57 2013
From: robert.kern at gmail.com (Robert Kern)
Date: Thu, 16 May 2013 13:44:57 +0100
Subject: [Numpy-discussion] Strange memory consumption in numpy?
In-Reply-To: <5194D1C9.1060900@smhi.se>
References: <51948C3A.6000404@smhi.se>
	<CAF6FJiuJt7i+kYsJcXuF_0SJ-iLP5AZKakhsWH2ZoKvZ+knK7A@mail.gmail.com>
	<5194D1C9.1060900@smhi.se>
Message-ID: <CAF6FJiuDN_cBN0KsWgCPw_rVdGu_r-FO7jogYqW7R+RqtdWZTg@mail.gmail.com>

On Thu, May 16, 2013 at 1:32 PM, Martin Raspaud <martin.raspaud at smhi.se> wrote:
> On 16/05/13 10:26, Robert Kern wrote:
>
>>> Can anyone give a reasonable explanation ?
>>
>> memory_profiler only looks at the amount of memory that the OS has
>> allocated to the Python process. It cannot measure the amount of
>> memory actually given to living objects. Python does not always return
>> memory back to the OS immediately when it frees the memory for an
>> object. Your two observations are linked. Python freed the memory of
>> arr2 immediately, but it did not return the memory to the OS, so
>> memory_profiler could not notice it. When arr3 is allocated, it
>> happened to fit into the block of memory that arr2 once owned, so
>> Python's memory allocator just used that block again. Since Python did
>> not have to go out to the OS to get more memory, memory_profiler could
>> not notice that, either.
>
> Robert,
>
> Thanks a lot for the clear explanation, it makes perfect sense now.
>
> You're talking about living objects, but as I understand the few memory
> profilers I found around the web for python can't track numpy arrays.
> Any pointers on something that would work with numpy ?

meliae has special support for numpy.ndarray objects. It's a little
broken, in that it will double-count views, but you can provide a
better specialization if you wish (look for the add_special_size()
function).

https://launchpad.net/meliae

--
Robert Kern


From thomas.robitaille at gmail.com  Thu May 16 09:19:51 2013
From: thomas.robitaille at gmail.com (Thomas Robitaille)
Date: Thu, 16 May 2013 15:19:51 +0200
Subject: [Numpy-discussion] __array_priority__ ignored if __array__ is
	present
Message-ID: <CAGMHX_3Xcn6_NmG1u7t9UVYN9CWN2D3Uj_aBKcgGZG5i9=-qew@mail.gmail.com>

Hi everyone,

(this was posted as part of another topic, but since it was unrelated,
I'm reposting as a separate thread)

I've also been having issues with __array_priority__ - the following
code behaves differently for __mul__ and __rmul__:

"""
import numpy as np


class TestClass(object):

    def __init__(self, input_array):
        self.array = input_array

    def __mul__(self, other):
        print "Called __mul__"

    def __rmul__(self, other):
        print "Called __rmul__"

    def __array_wrap__(self, out_arr, context=None):
        print "Called __array_wrap__"
        return TestClass(out_arr)

    def __array__(self):
        print "Called __array__"
        return np.array(self.array)
"""

with output:

"""
In [7]: a = TestClass([1,2,3])

In [8]: print type(np.array([1,2,3]) * a)
Called __array__
Called __array_wrap__
<class '__main__.TestClass'>

In [9]: print type(a * np.array([1,2,3]))
Called __mul__
<type 'NoneType'>
"""

Is this also an oversight? I opened a ticket for it a little while ago:

https://github.com/numpy/numpy/issues/3164

Any ideas?

Thanks!
Tom


From nouiz at nouiz.org  Thu May 16 09:58:12 2013
From: nouiz at nouiz.org (=?ISO-8859-1?Q?Fr=E9d=E9ric_Bastien?=)
Date: Thu, 16 May 2013 09:58:12 -0400
Subject: [Numpy-discussion] __array_priority__ ignored if __array__ is
	present
In-Reply-To: <CAGMHX_3Xcn6_NmG1u7t9UVYN9CWN2D3Uj_aBKcgGZG5i9=-qew@mail.gmail.com>
References: <CAGMHX_3Xcn6_NmG1u7t9UVYN9CWN2D3Uj_aBKcgGZG5i9=-qew@mail.gmail.com>
Message-ID: <CADKKbtifSh4XKHRb5UxF993gsoK_JJZkPye4VDwzue8EG14a-A@mail.gmail.com>

I looked yesterday rapidly in the code and didn't find the reason (I don't
know it well, that is probably why).

But last night I think of one possible cause. I found this code 2 times in
the file core/src/umath/ufunc_object.c:

    if (nin == 2 && nout == 1 && dtypes[1]->type_num == NPY_OBJECT) {
        PyObject *_obj = PyTuple_GET_ITEM(args, 1);
        if (!PyArray_CheckExact(_obj)) {
            double self_prio, other_prio;
            self_prio = PyArray_GetPriority(PyTuple_GET_ITEM(args, 0),

NPY_SCALAR_PRIORITY);
            other_prio = PyArray_GetPriority(_obj, NPY_SCALAR_PRIORITY);
            if (self_prio < other_prio &&
                            _has_reflected_op(_obj, ufunc_name)) {
                retval = -2;
        goto fail;
            }
    }
    }

It is this code that will call _has_reflected_op() function. The if
condition is:

dtypes[1]->type_num == NPY_OBJECT


I wouldn't be surprised if dtypes[1] isn't NPY_OBJECT when you implement
__array__.

dtypes is set with those line:

    retval = ufunc->type_resolver(ufunc, casting,
                            op, type_tup, dtypes);


HTH

Fred



On Thu, May 16, 2013 at 9:19 AM, Thomas Robitaille <
thomas.robitaille at gmail.com> wrote:

> Hi everyone,
>
> (this was posted as part of another topic, but since it was unrelated,
> I'm reposting as a separate thread)
>
> I've also been having issues with __array_priority__ - the following
> code behaves differently for __mul__ and __rmul__:
>
> """
> import numpy as np
>
>
> class TestClass(object):
>
>     def __init__(self, input_array):
>         self.array = input_array
>
>     def __mul__(self, other):
>         print "Called __mul__"
>
>     def __rmul__(self, other):
>         print "Called __rmul__"
>
>     def __array_wrap__(self, out_arr, context=None):
>         print "Called __array_wrap__"
>         return TestClass(out_arr)
>
>     def __array__(self):
>         print "Called __array__"
>         return np.array(self.array)
> """
>
> with output:
>
> """
> In [7]: a = TestClass([1,2,3])
>
> In [8]: print type(np.array([1,2,3]) * a)
> Called __array__
> Called __array_wrap__
> <class '__main__.TestClass'>
>
> In [9]: print type(a * np.array([1,2,3]))
> Called __mul__
> <type 'NoneType'>
> """
>
> Is this also an oversight? I opened a ticket for it a little while ago:
>
> https://github.com/numpy/numpy/issues/3164
>
> Any ideas?
>
> Thanks!
> Tom
> _______________________________________________
> NumPy-Discussion mailing list
> NumPy-Discussion at scipy.org
> http://mail.scipy.org/mailman/listinfo/numpy-discussion
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20130516/5dacc2ba/attachment.html>

From jtaylor.debian at googlemail.com  Thu May 16 13:42:13 2013
From: jtaylor.debian at googlemail.com (Julian Taylor)
Date: Thu, 16 May 2013 19:42:13 +0200
Subject: [Numpy-discussion] experiments with SSE vectorization
Message-ID: <51951A75.3070105@googlemail.com>

Hi,
I have been experimenting a bit with how applicable SSE vectorization is
to NumPy.
In principle the core of NumPy mostly deals with memory bound
operations, but it turns out on modern machines with large caches you
can still get decent speed ups.

The experiments are available on this fork:
https://github.com/juliantaylor/numpy/tree/simd-experiments
It includes a simple benchmark 'npbench.py' in the top level.
No runtime detection is used, it is only enabled on amd64 systems(which
always has SSE2).

The simd-experiments branch vectorizes the sqrt, basic math operations
and min/max reductions.
For float32 operations you get speedups around 2 (simple ops) - 4 (sqrt).
For double it is around 1.2 - 2, depending on the cpu.
My Phenom(tm) II X4 955 retains a good speedup even for very large
datasizes but on intel cpus (xeon and core2duo) you don't gain anything
if the data is larger than the L3 cache.
The vectorized version was never slower on phenom and xeon.
But on a core2duo the normal addition with very large datasets got 10%
slower. This can be compensated by using aligned load operations, but
its not implemented yet.
I'm interested in your results of npbench.py command on other cpus, so
if you want to try it please send me the output (include /proc/cpuinfo)

The code is a little rough, it can probably be cleaned up a bit by
adapting the code generator used.
Would this be something worth including in NumPy?

Further vectorization targets on my todo list are things like
std/var/mean, basically anything that has a high computation/memory
ration, suggestions are welcome.


Here the detailed results for my phenom:
float32 datasize (2MB)
operation:                         speedup
np.float32 np.max(d)                 3.04
np.float32 np.min(d)                  3.1
np.float32 np.sum(d)                 3.02
np.float32 np.prod(d)                3.04
np.float32 np.add(1, d)              1.44
np.float32 np.add(d, 1)              1.45
np.float32 np.divide(1, d)           3.41
np.float32 np.divide(d, 1)           3.41
np.float32 np.divide(d, d)           3.42
np.float32 np.add(d, d)              1.42
np.float32 np.multiply(d, d)         1.43
np.float32 np.sqrt(d)                4.26

float64 datasize (4MB)
operation:                         speedup
np.float64 np.max(d)                    2
np.float64 np.min(d)                 1.89
np.float64 np.sum(d)                 1.62
np.float64 np.prod(d)                1.63
np.float64 np.add(1, d)              1.08
np.float64 np.add(d, 1)             0.993
np.float64 np.divide(1, d)           1.83
np.float64 np.divide(d, 1)           1.74
np.float64 np.divide(d, d)            1.8
np.float64 np.add(d, d)              1.02
np.float64 np.multiply(d, d)         1.05
np.float64 np.sqrt(d)                2.22

attached the results for intel cpus.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: results.tar.gz
Type: application/gzip
Size: 8668 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20130516/038b759a/attachment.bin>

From ndbecker2 at gmail.com  Thu May 16 14:42:07 2013
From: ndbecker2 at gmail.com (Neal Becker)
Date: Thu, 16 May 2013 14:42:07 -0400
Subject: [Numpy-discussion] RuntimeWarning: divide by zero encountered in log
Message-ID: <kn399q$shh$1@ger.gmane.org>

Is there a way to get a traceback instead of just printing the
line that triggered the error?



From jjhelmus at gmail.com  Thu May 16 14:47:56 2013
From: jjhelmus at gmail.com (Jonathan Helmus)
Date: Thu, 16 May 2013 13:47:56 -0500
Subject: [Numpy-discussion] RuntimeWarning: divide by zero encountered
 in log
In-Reply-To: <kn399q$shh$1@ger.gmane.org>
References: <kn399q$shh$1@ger.gmane.org>
Message-ID: <519529DC.8070905@gmail.com>

On 05/16/2013 01:42 PM, Neal Becker wrote:
> Is there a way to get a traceback instead of just printing the
> line that triggered the error?
>
> _______________________________________________
> NumPy-Discussion mailing list
> NumPy-Discussion at scipy.org
> http://mail.scipy.org/mailman/listinfo/numpy-discussion
Neal,

     Look at the numpy.seterr function.  You can use it to change how 
floating-point errors are handled, including raising a 
FloatingPointError with a traceback as opposed to printing a 
RuntimeWarning.

Example

$ cat foo.py
import numpy as np

np.seterr(divide='raise')

a = np.array([1,1,1], dtype='float32')
a / 0

$ python foo.py
Traceback (most recent call last):
   File "test.py", line 6, in <module>
     a / 0
FloatingPointError: divide by zero encountered in divide

Cheers,

     - Jonathan Helmus


From phillip.m.feldman at gmail.com  Thu May 16 18:09:05 2013
From: phillip.m.feldman at gmail.com (Phillip Feldman)
Date: Thu, 16 May 2013 15:09:05 -0700
Subject: [Numpy-discussion] numpy.nanmin, numpy.nanmax,
	and scipy.stats.nanmean
Message-ID: <CAB2ViTZHvg6dwmtcGbGvMue2fLE0Ahqq=xmzu2D-81PYbodEiA@mail.gmail.com>

It seems odd that `nanmin` and `nanmax` are in NumPy, while `nanmean` is in
SciPy.stats.  I'd like to propose that a `nanmean` function be added to
NumPy.

Phillip
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20130516/ab587232/attachment.html>

From ben.root at ou.edu  Thu May 16 21:04:15 2013
From: ben.root at ou.edu (Benjamin Root)
Date: Thu, 16 May 2013 21:04:15 -0400
Subject: [Numpy-discussion] numpy.nanmin, numpy.nanmax,
	and scipy.stats.nanmean
In-Reply-To: <CAB2ViTZHvg6dwmtcGbGvMue2fLE0Ahqq=xmzu2D-81PYbodEiA@mail.gmail.com>
References: <CAB2ViTZHvg6dwmtcGbGvMue2fLE0Ahqq=xmzu2D-81PYbodEiA@mail.gmail.com>
Message-ID: <CANNq6FmTTvp1Cj+pSJS+L_uYe08S_HoaZX4GhwNL4B2vT3RwyQ@mail.gmail.com>

On Thu, May 16, 2013 at 6:09 PM, Phillip Feldman <
phillip.m.feldman at gmail.com> wrote:

> It seems odd that `nanmin` and `nanmax` are in NumPy, while `nanmean` is
> in SciPy.stats.  I'd like to propose that a `nanmean` function be added to
> NumPy.
>
> Have no fear.  There is already plans for its inclusion in the next
release:

https://github.com/numpy/numpy/pull/3297/files

Cheers!
Ben Root
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20130516/f3a54cc2/attachment.html>

From cjordan1 at uw.edu  Fri May 17 00:18:55 2013
From: cjordan1 at uw.edu (Christopher Jordan-Squire)
Date: Thu, 16 May 2013 21:18:55 -0700
Subject: [Numpy-discussion] experiments with SSE vectorization
In-Reply-To: <51951A75.3070105@googlemail.com>
References: <51951A75.3070105@googlemail.com>
Message-ID: <CAEJxiFpw3TPAZKVkhCkaoZq4spwbmzzcwzgAvTJxqs0S+b0GuQ@mail.gmail.com>

I'd been under the impression that the easiest way to get SSE support
was to have numpy use an optimized blas/lapack. Is that not the case?

On Thu, May 16, 2013 at 10:42 AM, Julian Taylor
<jtaylor.debian at googlemail.com> wrote:
> Hi,
> I have been experimenting a bit with how applicable SSE vectorization is
> to NumPy.
> In principle the core of NumPy mostly deals with memory bound
> operations, but it turns out on modern machines with large caches you
> can still get decent speed ups.
>
> The experiments are available on this fork:
> https://github.com/juliantaylor/numpy/tree/simd-experiments
> It includes a simple benchmark 'npbench.py' in the top level.
> No runtime detection is used, it is only enabled on amd64 systems(which
> always has SSE2).
>
> The simd-experiments branch vectorizes the sqrt, basic math operations
> and min/max reductions.
> For float32 operations you get speedups around 2 (simple ops) - 4 (sqrt).
> For double it is around 1.2 - 2, depending on the cpu.
> My Phenom(tm) II X4 955 retains a good speedup even for very large
> datasizes but on intel cpus (xeon and core2duo) you don't gain anything
> if the data is larger than the L3 cache.
> The vectorized version was never slower on phenom and xeon.
> But on a core2duo the normal addition with very large datasets got 10%
> slower. This can be compensated by using aligned load operations, but
> its not implemented yet.
> I'm interested in your results of npbench.py command on other cpus, so
> if you want to try it please send me the output (include /proc/cpuinfo)
>
> The code is a little rough, it can probably be cleaned up a bit by
> adapting the code generator used.
> Would this be something worth including in NumPy?
>
> Further vectorization targets on my todo list are things like
> std/var/mean, basically anything that has a high computation/memory
> ration, suggestions are welcome.
>
>
> Here the detailed results for my phenom:
> float32 datasize (2MB)
> operation:                         speedup
> np.float32 np.max(d)                 3.04
> np.float32 np.min(d)                  3.1
> np.float32 np.sum(d)                 3.02
> np.float32 np.prod(d)                3.04
> np.float32 np.add(1, d)              1.44
> np.float32 np.add(d, 1)              1.45
> np.float32 np.divide(1, d)           3.41
> np.float32 np.divide(d, 1)           3.41
> np.float32 np.divide(d, d)           3.42
> np.float32 np.add(d, d)              1.42
> np.float32 np.multiply(d, d)         1.43
> np.float32 np.sqrt(d)                4.26
>
> float64 datasize (4MB)
> operation:                         speedup
> np.float64 np.max(d)                    2
> np.float64 np.min(d)                 1.89
> np.float64 np.sum(d)                 1.62
> np.float64 np.prod(d)                1.63
> np.float64 np.add(1, d)              1.08
> np.float64 np.add(d, 1)             0.993
> np.float64 np.divide(1, d)           1.83
> np.float64 np.divide(d, 1)           1.74
> np.float64 np.divide(d, d)            1.8
> np.float64 np.add(d, d)              1.02
> np.float64 np.multiply(d, d)         1.05
> np.float64 np.sqrt(d)                2.22
>
> attached the results for intel cpus.
>
> _______________________________________________
> NumPy-Discussion mailing list
> NumPy-Discussion at scipy.org
> http://mail.scipy.org/mailman/listinfo/numpy-discussion
>


From njs at pobox.com  Fri May 17 00:56:41 2013
From: njs at pobox.com (Nathaniel Smith)
Date: Fri, 17 May 2013 05:56:41 +0100
Subject: [Numpy-discussion] experiments with SSE vectorization
In-Reply-To: <CAEJxiFpw3TPAZKVkhCkaoZq4spwbmzzcwzgAvTJxqs0S+b0GuQ@mail.gmail.com>
References: <51951A75.3070105@googlemail.com>
	<CAEJxiFpw3TPAZKVkhCkaoZq4spwbmzzcwzgAvTJxqs0S+b0GuQ@mail.gmail.com>
Message-ID: <CAPJVwB=Ocf0H77=6aFgbZz504SBxTEiJ8Hywv6UPaR4cXBRL-A@mail.gmail.com>

On 17 May 2013 05:19, "Christopher Jordan-Squire" <cjordan1 at uw.edu> wrote:
>
> I'd been under the impression that the easiest way to get SSE support
> was to have numpy use an optimized blas/lapack. Is that not the case?

Apples and oranges. That's the easiest (only) way to get SSE support for
operations that go through blas/lapack, but there are also lots of
operations in numpy that are implemented directly.

-n
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20130517/2f4d9468/attachment.html>

From njs at pobox.com  Fri May 17 01:06:06 2013
From: njs at pobox.com (Nathaniel Smith)
Date: Fri, 17 May 2013 06:06:06 +0100
Subject: [Numpy-discussion] RuntimeWarning: divide by zero encountered
	in log
In-Reply-To: <519529DC.8070905@gmail.com>
References: <kn399q$shh$1@ger.gmane.org>
	<519529DC.8070905@gmail.com>
Message-ID: <CAPJVwBkcT3F8yWo0+JRkYZSOPeZNGGu22gRgd9AH-3T508ueXw@mail.gmail.com>

On 16 May 2013 19:48, "Jonathan Helmus" <jjhelmus at gmail.com> wrote:
>
> On 05/16/2013 01:42 PM, Neal Becker wrote:
> > Is there a way to get a traceback instead of just printing the
> > line that triggered the error?
> >
> > _______________________________________________
> > NumPy-Discussion mailing list
> > NumPy-Discussion at scipy.org
> > http://mail.scipy.org/mailman/listinfo/numpy-discussion
> Neal,
>
>      Look at the numpy.seterr function.  You can use it to change how
> floating-point errors are handled, including raising a
> FloatingPointError with a traceback as opposed to printing a
> RuntimeWarning.
>
> Example
>
> $ cat foo.py
> import numpy as np
>
> np.seterr(divide='raise')
>
> a = np.array([1,1,1], dtype='float32')
> a / 0
>
> $ python foo.py
> Traceback (most recent call last):
>    File "test.py", line 6, in <module>
>      a / 0
> FloatingPointError: divide by zero encountered in divide

You also have the option of using Python's general ability to customize how
any warning is handled - see the 'warnings' module and -W switch.

If you just want a traceback printed without an exception then I think you
can do that with np.seterr too (using np.seterrcall).

-n
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20130517/82571907/attachment.html>

From ndbecker2 at gmail.com  Fri May 17 09:12:53 2013
From: ndbecker2 at gmail.com (Neal Becker)
Date: Fri, 17 May 2013 09:12:53 -0400
Subject: [Numpy-discussion] RuntimeWarning: divide by zero encountered
	in log
References: <kn399q$shh$1@ger.gmane.org> <519529DC.8070905@gmail.com>
	<CAPJVwBkcT3F8yWo0+JRkYZSOPeZNGGu22gRgd9AH-3T508ueXw@mail.gmail.com>
Message-ID: <kn5ach$e6u$1@ger.gmane.org>

Nathaniel Smith wrote:

> On 16 May 2013 19:48, "Jonathan Helmus" <jjhelmus at gmail.com> wrote:
>>
>> On 05/16/2013 01:42 PM, Neal Becker wrote:
>> > Is there a way to get a traceback instead of just printing the
>> > line that triggered the error?
>> >
>> > _______________________________________________
>> > NumPy-Discussion mailing list
>> > NumPy-Discussion at scipy.org
>> > http://mail.scipy.org/mailman/listinfo/numpy-discussion
>> Neal,
>>
>>      Look at the numpy.seterr function.  You can use it to change how
>> floating-point errors are handled, including raising a
>> FloatingPointError with a traceback as opposed to printing a
>> RuntimeWarning.
>>
>> Example
>>
>> $ cat foo.py
>> import numpy as np
>>
>> np.seterr(divide='raise')
>>
>> a = np.array([1,1,1], dtype='float32')
>> a / 0
>>
>> $ python foo.py
>> Traceback (most recent call last):
>>    File "test.py", line 6, in <module>
>>      a / 0
>> FloatingPointError: divide by zero encountered in divide
> 
> You also have the option of using Python's general ability to customize how
> any warning is handled - see the 'warnings' module and -W switch.
> 
> If you just want a traceback printed without an exception then I think you
> can do that with np.seterr too (using np.seterrcall).
> 
> -n

I tried this:

import traceback

np.seterrcall (lambda a,b: traceback.print_stack)
np.seterr (all='call')
np.seterrcall (lambda a,b: traceback.print_stack)

but it doesn't seem to do anything, I still see numpy warning as before.



From ndbecker2 at gmail.com  Fri May 17 09:14:35 2013
From: ndbecker2 at gmail.com (Neal Becker)
Date: Fri, 17 May 2013 09:14:35 -0400
Subject: [Numpy-discussion] RuntimeWarning: divide by zero encountered
	in log
References: <kn399q$shh$1@ger.gmane.org> <519529DC.8070905@gmail.com>
	<CAPJVwBkcT3F8yWo0+JRkYZSOPeZNGGu22gRgd9AH-3T508ueXw@mail.gmail.com>
	<kn5ach$e6u$1@ger.gmane.org>
Message-ID: <kn5afm$e6u$2@ger.gmane.org>

Neal Becker wrote:

> Nathaniel Smith wrote:
> 
>> On 16 May 2013 19:48, "Jonathan Helmus" <jjhelmus at gmail.com> wrote:
>>>
>>> On 05/16/2013 01:42 PM, Neal Becker wrote:
>>> > Is there a way to get a traceback instead of just printing the
>>> > line that triggered the error?
>>> >
>>> > _______________________________________________
>>> > NumPy-Discussion mailing list
>>> > NumPy-Discussion at scipy.org
>>> > http://mail.scipy.org/mailman/listinfo/numpy-discussion
>>> Neal,
>>>
>>>      Look at the numpy.seterr function.  You can use it to change how
>>> floating-point errors are handled, including raising a
>>> FloatingPointError with a traceback as opposed to printing a
>>> RuntimeWarning.
>>>
>>> Example
>>>
>>> $ cat foo.py
>>> import numpy as np
>>>
>>> np.seterr(divide='raise')
>>>
>>> a = np.array([1,1,1], dtype='float32')
>>> a / 0
>>>
>>> $ python foo.py
>>> Traceback (most recent call last):
>>>    File "test.py", line 6, in <module>
>>>      a / 0
>>> FloatingPointError: divide by zero encountered in divide
>> 
>> You also have the option of using Python's general ability to customize how
>> any warning is handled - see the 'warnings' module and -W switch.
>> 
>> If you just want a traceback printed without an exception then I think you
>> can do that with np.seterr too (using np.seterrcall).
>> 
>> -n
> 
> I tried this:
> 
> import traceback
> 
> np.seterrcall (lambda a,b: traceback.print_stack)
> np.seterr (all='call')
> np.seterrcall (lambda a,b: traceback.print_stack)
> 
> but it doesn't seem to do anything, I still see numpy warning as before.

Here's a trivial test:

import numpy as np

import traceback

np.seterrcall (lambda a,b: traceback.print_stack)
np.seterr (all='call')
np.seterrcall (lambda a,b: traceback.print_stack)

print 1./0




From njs at pobox.com  Fri May 17 09:20:04 2013
From: njs at pobox.com (Nathaniel Smith)
Date: Fri, 17 May 2013 14:20:04 +0100
Subject: [Numpy-discussion] RuntimeWarning: divide by zero encountered
	in log
In-Reply-To: <kn5ach$e6u$1@ger.gmane.org>
References: <kn399q$shh$1@ger.gmane.org> <519529DC.8070905@gmail.com>
	<CAPJVwBkcT3F8yWo0+JRkYZSOPeZNGGu22gRgd9AH-3T508ueXw@mail.gmail.com>
	<kn5ach$e6u$1@ger.gmane.org>
Message-ID: <CAPJVwBkVz2EsYL5OgsDPvz5an-WZw8WBCtxDeznQL7+3na-zvg@mail.gmail.com>

On 17 May 2013 14:17, "Neal Becker" <ndbecker2 at gmail.com> wrote:
>
> Nathaniel Smith wrote:
>
> > On 16 May 2013 19:48, "Jonathan Helmus" <jjhelmus at gmail.com> wrote:
> >>
> >> On 05/16/2013 01:42 PM, Neal Becker wrote:
> >> > Is there a way to get a traceback instead of just printing the
> >> > line that triggered the error?
> >> >
> >> > _______________________________________________
> >> > NumPy-Discussion mailing list
> >> > NumPy-Discussion at scipy.org
> >> > http://mail.scipy.org/mailman/listinfo/numpy-discussion
> >> Neal,
> >>
> >>      Look at the numpy.seterr function.  You can use it to change how
> >> floating-point errors are handled, including raising a
> >> FloatingPointError with a traceback as opposed to printing a
> >> RuntimeWarning.
> >>
> >> Example
> >>
> >> $ cat foo.py
> >> import numpy as np
> >>
> >> np.seterr(divide='raise')
> >>
> >> a = np.array([1,1,1], dtype='float32')
> >> a / 0
> >>
> >> $ python foo.py
> >> Traceback (most recent call last):
> >>    File "test.py", line 6, in <module>
> >>      a / 0
> >> FloatingPointError: divide by zero encountered in divide
> >
> > You also have the option of using Python's general ability to customize
how
> > any warning is handled - see the 'warnings' module and -W switch.
> >
> > If you just want a traceback printed without an exception then I think
you
> > can do that with np.seterr too (using np.seterrcall).
> >
> > -n
>
> I tried this:
>
> import traceback
>
> np.seterrcall (lambda a,b: traceback.print_stack)
> np.seterr (all='call')

Try adding some parens?

np.seterrcall (lambda a,b: traceback.print_stack())

-n
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20130517/49ab6cce/attachment.html>

From rodrigo at rodrigobotafogo.com  Fri May 17 09:20:47 2013
From: rodrigo at rodrigobotafogo.com (Rodrigo Botafogo)
Date: Fri, 17 May 2013 11:20:47 -0200
Subject: [Numpy-discussion] [ANN] Multidimensional Array - MDArray (0.5.0)
Message-ID: <CAAKc=GAq04oY695MhMskN47wr2-8b089oPHHdrbn4h3R4mvgMg@mail.gmail.com>

Although this is not directly connected to NumPy, I believe that it could
be of interest to the NymPy community.  If, by any reason it is inproper to
post this type of announcement on this list, please let me know.

I?m happy to announce a new version of MDArray...


MDArray
=======

MDArray is a multi dimensional array implemented for JRuby inspired by
NumPy (www.numpy.org)
and Narray (narray.rubyforge.org) by Masahiro Tanaka.  MDArray stands on
the shoulders of
Java-NetCDF and Parallel Colt.

NetCDF-Java Library is a Java interface to NetCDF files, as well as to many
other types of
scientific data formats.  It is developed and distributed by Unidata (
http://www.unidata.ucar.edu).

Parallel Colt (sites.google.com/site/piotrwendykier/software/parallelcolt)
is a multithreaded
version of Colt (http://acs.lbl.gov/software/colt/).  Colt provides a set
of Open Source
Libraries for High Performance Scientific and Technical Computing in Java.
Scientific
and technical computing is characterized by demanding problem sizes and a
need for high
performance at reasonably small memory footprint.

MDArray and SciRuby
===================

MDArray subscribes fully to the SciRuby Manifesto (http://sciruby.com/).

"Ruby has for some time had no equivalent to the beautifully constructed
NumPy, SciPy,
and matplotlib libraries for Python.

We believe that the time for a Ruby science and visualization package has
come. Sometimes
when a solution of sugar and water becomes super-saturated, from it
precipitates a pure,
delicious, and diabetes-inducing crystal of sweetness, induced by no more
than the tap
of a finger. So is occurring now, we believe, with numeric and
visualization libraries for Ruby."

Main properties
===============

  + Homogeneous multidimensional array, a table of elements (usually
numbers), all of the
      same type, indexed by a tuple of positive integers;
  + Easy calculation for large numerical multi dimensional arrays;
  + Basic types are: boolean, byte, short, int, long, float, double,
string, structure;
  + Based on JRuby, which allows importing Java libraries;
  + Operator: +,-,*,/,%,**, >, >=, etc.
  + Functions: abs, ceil, floor, truncate, is_zero, square, cube, fourth;
  + Binary Operators: &, |, ^, ~ (binary_ones_complement), <<, >>;
  + Ruby Math functions: acos, acosh, asin, asinh, atan, atan2, atanh,
cbrt, cos, erf, exp,
      gamma, hypot, ldexp, log, log10, log2, sin, sinh, sqrt, tan, tanh,
neg;
  + Boolean operations on boolean arrays: and, or, not;
  + Fast descriptive statistics from Parallel Colt (complete list found
bellow);
  + Easy manipulation of arrays: reshape, reduce dimension, permute,
section, slice, etc.
  + Reading of two dimensional arrays from CSV files (mainly for debugging
and simple
      testing purposes);
  + StatList: a list that can grow/shrink and that can compute Parallel
Colt descriptive
      statistics.

Descriptive statistics methods
==============================

auto_correlation, correlation, covariance, durbin_watson, frequencies,
geometric_mean,
harmonic_mean, kurtosis, lag1, max, mean, mean_deviation, median, min,
moment, moment3,
moment4, pooled_mean, pooled_variance, product, quantile, quantile_inverse,
rank_interpolated, rms, sample_covariance, sample_kurtosis,
sample_kurtosis_standard_error, sample_skew, sample_skew_standard_error,
sample_standard_deviation, sample_variance, sample_weighted_variance, skew,
split,
standard_deviation, standard_error, sum, sum_of_inversions,
sum_of_logarithms,
sum_of_powers, sum_of_power_deviations, sum_of_squares,
sum_of_squared_deviations,
trimmed_mean, variance, weighted_mean, weighted_rms, weighted_sums,
winsorized_mean.

Installation and download
=========================

  + Install Jruby
  + jruby -S gem install mdarray

Contributors
============

  + Contributors are welcome.

Homepages
=========

  + http://rubygems.org/gems/mdarray
  + https://github.com/rbotafogo/mdarray/wiki


HISTORY
=======

  + 16/05/2013: Version 0.5.0: All loops transfered to Java with over 50%
performance
      improvement.  Descriptive statistics from Parallel Colt.
  + 19/04/2013: Version 0.4.3: Fixes a simple (but fatal bug).  No new
features
  + 17/04/2013: Version 0.4.2: Adds simple statistics and boolean operators
  + 05/05/2013: Version 0.4.0: Initial release

-- 
Rodrigo Botafogo
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20130517/8566d297/attachment.html>

From francesc at continuum.io  Fri May 17 14:36:51 2013
From: francesc at continuum.io (Francesc Alted)
Date: Fri, 17 May 2013 20:36:51 +0200
Subject: [Numpy-discussion] ANN: python-blosc 1.1 RC1 available for testing
Message-ID: <519678C3.9080207@continuum.io>

================================
Announcing python-blosc 1.1 RC1
================================

What is it?
===========

python-blosc (http://blosc.pydata.org) is a Python wrapper for the
Blosc compression library.

Blosc (http://blosc.org) is a high performance compressor optimized for
binary data.  It has been designed to transmit data to the processor
cache faster than the traditional, non-compressed, direct memory fetch
approach via a memcpy() OS call.  Whether this is achieved or not
depends of the data compressibility, the number of cores in the system,
and other factors.  See a series of benchmarks conducted for many
different systems: http://blosc.org/trac/wiki/SyntheticBenchmarks.

Blosc works well for compressing numerical arrays that contains data
with relatively low entropy, like sparse data, time series, grids with
regular-spaced values, etc.

There is also a handy command line for Blosc called Bloscpack
(https://github.com/esc/bloscpack) that allows you to compress large
binary datafiles on-disk.  Although the format for Bloscpack has not
stabilized yet, it allows you to effectively use Blosc from your
favorite shell.


What is new?
============

- Added new `compress_ptr` and `decompress_ptr` functions that allows to
   compress and decompress from/to a data pointer.  These are low level
   calls and user must make sure that the pointer data area is safe.

- Since Blosc (the C library) already supports to be installed as an
   standalone library (via cmake), it is also possible to link
   python-blosc against a system Blosc library.

- The Python calls to Blosc are now thread-safe (another consequence of
   recent Blosc library supporting this at C level).

- Many checks on types and ranges of values have been added.  Most of
   the calls will now complain when passed the wrong values.

- Docstrings are much improved. Also, Sphinx-based docs are available
   now.

Many thanks to Valentin H?nel for his impressive work for this release.

For more info, you can see the release notes in:

https://github.com/FrancescAlted/python-blosc/wiki/Release-notes

More docs and examples are available in the documentation site:

http://blosc.pydata.org


Installing
==========

python-blosc is in PyPI repository, so installing it is easy:

$ pip install -U blosc  # yes, you should omit the blosc- prefix


Download sources
================

The sources are managed through github services at:

http://github.com/FrancescAlted/python-blosc


Documentation
=============

There is Sphinx-based documentation site at:

http://blosc.pydata.org/


Mailing list
============

There is an official mailing list for Blosc at:

blosc at googlegroups.com
http://groups.google.es/group/blosc


Licenses
========

Both Blosc and its Python wrapper are distributed using the MIT license.
See:

https://github.com/FrancescAlted/python-blosc/blob/master/LICENSES

for more details.

--
Francesc Alted



From joe13676 at comcast.net  Sat May 18 01:11:16 2013
From: joe13676 at comcast.net (Joe Piccoli)
Date: Fri, 17 May 2013 22:11:16 -0700
Subject: [Numpy-discussion] Newbie trying to install NumPy
Message-ID: <015701ce5386$205a1260$610e3720$@comcast.net>

Hello,

 

I've been trying to install NumPy to run with Eclipse on Windows Vista.
After installing (I thought) NumPy I was seeing:

 

ImportError: Error importing numpy: you should not try to import numpy from

        its source directory; please exit the numpy source tree, and
relaunch

        your python intepreter from there.

 

I next tried to follow the instructions from the scipy.org website and
downloaded and ran:

 

numpy-1.7.1-win32-superpack-python27.exe

 

This started up but I immediately saw the following dialog:

 

---------------------------

Cannot install

---------------------------

Python version 2.7 required, which was not found in the registry.

---------------------------

OK   

---------------------------

 

The next dialog prompted for a Python installation to use but the list box
was empty and it would not allow me to enter a path.

 

Is it absolutely necessary to build NumPy myself or is there a working
installation out there? I know I'm doing something wrong but I don't know
what it is. Any assistance would be greatly appreciated :).

 

Thanks,

Joseph A. Piccoli

joe13676 at comcast.net

 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20130517/f4a2e8ea/attachment.html>

From jtaylor.debian at googlemail.com  Sat May 18 02:12:45 2013
From: jtaylor.debian at googlemail.com (Julian Taylor)
Date: Sat, 18 May 2013 08:12:45 +0200
Subject: [Numpy-discussion] faster (selection based) median, 2013 edition
Message-ID: <51971BDD.9000307@googlemail.com>

hi,

once again I want to bring up the median algorithm which is implemented
in terms of sorting in numpy.
median (and percentile and a couple more functions) can be more
efficiently implemented in terms of a selection algorithm. The
complexity can them be linear instead of linearithmic.

I found numerous discussions of this in the list archives [1, 2, 3] but
I did not find why those attempts failed, the threads all just seemed to
stop.
Did the previous attempts fail due to lack of time or was there a
fundamental reason blocking this change?

In the hope of the former, I went ahead and implemented a prototype of a
partition function (similar to [3] but only one argument) and
implemented median in terms of it.
partition not like C++ partition, its equivalent to nth_element in C++,
maybe its better to name it nth_element?

The code is available here:
https://github.com/juliantaylor/numpy/tree/select-median

the partition interface is:
ndarray.partition(kth, axis=-1)
kth is an integer
The array is transformed so the k-th element of the array is in its
final sorted order, all below are smaller all above are greater, but the
ordering is undefined

Example:
In [1]: d = np.arange(10); np.random.shuffle(d)
In [2]: d
Out[2]: array([1, 7, 0, 2, 5, 6, 8, 9, 3, 4])
In [3]: np.partition(d, 3)
Out[3]: array([0, 1, 2, 3, 4, 6, 8, 9, 7, 5])
In [4]:  _[3] == 3
Out[5]: True

the performance of median improves as expected:
old vs new, 5000, uniform shuffled, out of place:
100us vs 40us
old vs new, 50000, uniform shuffled, out of place:
1.12ms vs 0.265ms
old vs new, 500000, uniform shuffled, out of place:
14ms vs 2.81ms

The implementation is very much still a prototype, apartition is not
exposed (and only implemented as a quicksort) and there is only one
algorithm (quickselect). One could still add median of medians for
better worst case performance.

If no blockers appear I want to fix this up and file a pull request to
have this in numpy 1.8.
Guidance on details of implementation in numpys C api is highly
appreciated, its the first time I'm dealing with it.

Cheers,
Julian Taylor

[1]
http://thread.gmane.org/gmane.comp.python.numeric.general/50931/focus=50941
[2]
http://thread.gmane.org/gmane.comp.python.numeric.general/32507/focus=41716
[3]
http://thread.gmane.org/gmane.comp.python.numeric.general/32341/focus=32348


From johann.cohentanugi at gmail.com  Sat May 18 03:28:01 2013
From: johann.cohentanugi at gmail.com (Johann Cohen-Tanugi)
Date: Sat, 18 May 2013 09:28:01 +0200
Subject: [Numpy-discussion] Newbie trying to install NumPy
In-Reply-To: <015701ce5386$205a1260$610e3720$@comcast.net>
References: <015701ce5386$205a1260$610e3720$@comcast.net>
Message-ID: <51972D81.1080001@gmail.com>

Hello Joe, I am not familiar with windows, but I believe that the 
Python(x,y) is here to help you : http://code.google.com/p/pythonxy/
HTH,
Johann

On 05/18/2013 07:11 AM, Joe Piccoli wrote:
> Hello,
>
> I've been trying to install NumPy to run with Eclipse on Windows Vista.
> After installing (I thought) NumPy I was seeing:
>
> ImportError: Error importing numpy: you should not try to import numpy from
>
>          its source directory; please exit the numpy source tree, and
> relaunch
>
>          your python intepreter from there.
>
> I next tried to follow the instructions from the scipy.org website and
> downloaded and ran:
>
> numpy-1.7.1-win32-superpack-python27.exe
>
> This started up but I immediately saw the following dialog:
>
> ---------------------------
>
> Cannot install
>
> ---------------------------
>
> Python version 2.7 required, which was not found in the registry.
>
> ---------------------------
>
> OK
>
> ---------------------------
>
> The next dialog prompted for a Python installation to use but the list
> box was empty and it would not allow me to enter a path.
>
> Is it absolutely necessary to build NumPy myself or is there a working
> installation out there? I know I'm doing something wrong but I don't
> know what it is. Any assistance would be greatly appreciated :).
>
> Thanks,
>
> Joseph A. Piccoli
>
> joe13676 at comcast.net
>
>
> --
> This message has been scanned for viruses and
> dangerous content by *MailScanner* <http://www.mailscanner.info/>, and is
> believed to be clean.
>
>
> _______________________________________________
> NumPy-Discussion mailing list
> NumPy-Discussion at scipy.org
> http://mail.scipy.org/mailman/listinfo/numpy-discussion
>


From davidmenhur at gmail.com  Sat May 18 04:52:41 2013
From: davidmenhur at gmail.com (=?UTF-8?B?RGHPgGlk?=)
Date: Sat, 18 May 2013 10:52:41 +0200
Subject: [Numpy-discussion] Newbie trying to install NumPy
In-Reply-To: <015701ce5386$205a1260$610e3720$@comcast.net>
References: <015701ce5386$205a1260$610e3720$@comcast.net>
Message-ID: <CAJhcF=08XzQgYH2ggR7mpm-wviU7zkK1qwFL6b3Qb046GhhB1A@mail.gmail.com>

On 18 May 2013 07:11, Joe Piccoli <joe13676 at comcast.net> wrote:
> Python version 2.7 required, which was not found in the registry.

You probably have a different version installed. Grab Python 2.7 from
python.org and install it; then, run the numpy installer. It should
just work out of the box.


From josef.pktd at gmail.com  Sat May 18 07:34:11 2013
From: josef.pktd at gmail.com (josef.pktd at gmail.com)
Date: Sat, 18 May 2013 07:34:11 -0400
Subject: [Numpy-discussion] Newbie trying to install NumPy
In-Reply-To: <015701ce5386$205a1260$610e3720$@comcast.net>
References: <015701ce5386$205a1260$610e3720$@comcast.net>
Message-ID: <CAMMTP+BM5f9-Ww+7V6by0=64HxYDi==pp4D9LGCTr8k0ydT_HQ@mail.gmail.com>

On Sat, May 18, 2013 at 1:11 AM, Joe Piccoli <joe13676 at comcast.net> wrote:

> Hello,****
>
> ** **
>
> I've been trying to install NumPy to run with Eclipse on Windows Vista.
> After installing (I thought) NumPy I was seeing:****
>
> ** **
>
> ImportError: Error importing numpy: you should not try to import numpy from
> ****
>
>         its source directory; please exit the numpy source tree, and
> relaunch****
>
>         your python intepreter from there.
>

Did you try to do this (what it says here)?

Josef



> ****
>
> ** **
>
> I next tried to follow the instructions from the scipy.org website and
> downloaded and ran:****
>
> ** **
>
> numpy-1.7.1-win32-superpack-python27.exe****
>
> ** **
>
> This started up but I immediately saw the following dialog:****
>
> ** **
>
> ---------------------------****
>
> Cannot install****
>
> ---------------------------****
>
> Python version 2.7 required, which was not found in the registry.****
>
> ---------------------------****
>
> OK   ****
>
> ---------------------------****
>
> ** **
>
> The next dialog prompted for a Python installation to use but the list box
> was empty and it would not allow me to enter a path.****
>
> ** **
>
> Is it absolutely necessary to build NumPy myself or is there a working
> installation out there? I know I'm doing something wrong but I don't know
> what it is. Any assistance would be greatly appreciated :).****
>
> ** **
>
> Thanks,****
>
> Joseph A. Piccoli****
>
> joe13676 at comcast.net****
>
> ** **
>
> _______________________________________________
> NumPy-Discussion mailing list
> NumPy-Discussion at scipy.org
> http://mail.scipy.org/mailman/listinfo/numpy-discussion
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20130518/9db4c9f3/attachment.html>

From chris.barker at noaa.gov  Sun May 19 18:14:22 2013
From: chris.barker at noaa.gov (Chris Barker - NOAA Federal)
Date: Sun, 19 May 2013 15:14:22 -0700
Subject: [Numpy-discussion] Newbie trying to install NumPy
In-Reply-To: <CAJhcF=08XzQgYH2ggR7mpm-wviU7zkK1qwFL6b3Qb046GhhB1A@mail.gmail.com>
References: <015701ce5386$205a1260$610e3720$@comcast.net>
	<CAJhcF=08XzQgYH2ggR7mpm-wviU7zkK1qwFL6b3Qb046GhhB1A@mail.gmail.com>
Message-ID: <-8497636677018212631@unknownmsgid>

On May 18, 2013, at 1:53 AM, "Da?id" <davidmenhur at gmail.com> wrote:

> On 18 May 2013 07:11, Joe
> You probably have a different version installed. Grab Python 2.7 from
> python.org and install it;

Make sure you match 32/64 bit. The message is a bit out of date,
you'll get the same error if you try to install a 32 bit numpy and
have a 64 bit python.

-Chris


> then, run the numpy installer. It should
> just work out of the box.
> _______________________________________________
> NumPy-Discussion mailing list
> NumPy-Discussion at scipy.org
> http://mail.scipy.org/mailman/listinfo/numpy-discussion


From bakhtiyor_zokhidov at mail.ru  Mon May 20 11:21:50 2013
From: bakhtiyor_zokhidov at mail.ru (=?UTF-8?B?QmFraHRpeW9yIFpva2hpZG92?=)
Date: Mon, 20 May 2013 19:21:50 +0400
Subject: [Numpy-discussion] =?utf-8?q?Equvalent_function_for_Ceil=28=29_an?=
	=?utf-8?b?ZCBGbG9vcigp?=
Message-ID: <1369063310.820343342@f281.mail.ru>

 Hello,

I am using ceil() and floor() function to get upper and lower value of some numbers. Let's say:

import math
x1 = 0.35
y1 = 4.46
>>>?math.ceil(x1)
1.0
>>> math.floor(y1)
4.0

The problem is that If I want to get upper and lower values for the certain step, for example, step = 0.25, ceil() function should give:
new_ceil(x1, step) => 0.5
new_floor(y1, step) => 4.25
Because, the step is 0.25

Question: How I can I achieve those results by using ceil() and floor() function, or Is there any equvalent function for that?
--? Bakhti
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20130520/e3ef9604/attachment.html>

From njs at pobox.com  Mon May 20 11:36:20 2013
From: njs at pobox.com (Nathaniel Smith)
Date: Mon, 20 May 2013 16:36:20 +0100
Subject: [Numpy-discussion] Equvalent function for Ceil() and Floor()
In-Reply-To: <1369063310.820343342@f281.mail.ru>
References: <1369063310.820343342@f281.mail.ru>
Message-ID: <CAPJVwBk5L8jHaU459k4xbdMjcvNAf-WA7BuG-WxdYriJ6-jQTg@mail.gmail.com>

On Mon, May 20, 2013 at 4:21 PM, Bakhtiyor Zokhidov
<bakhtiyor_zokhidov at mail.ru> wrote:
> Hello,
>
> I am using ceil() and floor() function to get upper and lower value of some
> numbers. Let's say:
>
> import math
> x1 = 0.35
> y1 = 4.46
>>>> math.ceil(x1)
> 1.0
>
>>>> math.floor(y1)
> 4.0
>
> The problem is that If I want to get upper and lower values for the certain
> step, for example, step = 0.25, ceil() function should give:
> new_ceil(x1, step) => 0.5
> new_floor(y1, step) => 4.25
> Because, the step is 0.25
>
> Question: How I can I achieve those results by using ceil() and floor()
> function, or Is there any equvalent function for that?

def new_ceil(x, step):
  return np.ceil(x / step) * step

?

-n


From robert.kern at gmail.com  Mon May 20 11:37:16 2013
From: robert.kern at gmail.com (Robert Kern)
Date: Mon, 20 May 2013 16:37:16 +0100
Subject: [Numpy-discussion] Equvalent function for Ceil() and Floor()
In-Reply-To: <1369063310.820343342@f281.mail.ru>
References: <1369063310.820343342@f281.mail.ru>
Message-ID: <CAF6FJiucd47kRKS36BPAy05uwgK65FGs1b0uc5_e_874fYXP+A@mail.gmail.com>

On Mon, May 20, 2013 at 4:21 PM, Bakhtiyor Zokhidov
<bakhtiyor_zokhidov at mail.ru> wrote:
> Hello,
>
> I am using ceil() and floor() function to get upper and lower value of some
> numbers. Let's say:
>
> import math
> x1 = 0.35
> y1 = 4.46
>>>> math.ceil(x1)
> 1.0
>
>>>> math.floor(y1)
> 4.0
>
> The problem is that If I want to get upper and lower values for the certain
> step, for example, step = 0.25, ceil() function should give:
> new_ceil(x1, step) => 0.5
> new_floor(y1, step) => 4.25
> Because, the step is 0.25
>
> Question: How I can I achieve those results by using ceil() and floor()
> function, or Is there any equvalent function for that?

For most purposes, the following functions suffice:

def new_ceil(x, step):
    return math.ceil(x / step) * step

def new_floor(x, step):
    return math.floor(x / step) * step


Alternately:

def new_ceil(x, step):
    quotient = x // step
    remainder = x % step
    return (quotient + (remainder > 0)) * step

def new_floor(x, step):
    quotient = x // step
    return quotient * step


Floating point representation errors and accumulated floating point
arithmetic inaccuracies may give you unexpected results in many cases,
so be careful.

--
Robert Kern


From bakhtiyor_zokhidov at mail.ru  Mon May 20 11:54:26 2013
From: bakhtiyor_zokhidov at mail.ru (=?UTF-8?B?QmFraHRpeW9yIFpva2hpZG92?=)
Date: Mon, 20 May 2013 19:54:26 +0400
Subject: [Numpy-discussion]
 =?utf-8?q?Equvalent_function_for_Ceil=28=29_an?=
 =?utf-8?b?ZCBGbG9vcigp?=
In-Reply-To: <CAF6FJiucd47kRKS36BPAy05uwgK65FGs1b0uc5_e_874fYXP+A@mail.gmail.com>
References: <1369063310.820343342@f281.mail.ru>
	<CAF6FJiucd47kRKS36BPAy05uwgK65FGs1b0uc5_e_874fYXP+A@mail.gmail.com>
Message-ID: <1369065266.537085267@f202.mail.ru>

 thanks a lot

what about the following example:
>>>new_ceil(-0.24, 0.25)
-0.0
???
thanks in advance for the reply


???????????, 20 ??? 2013, 16:37 +01:00 ?? Robert Kern <robert.kern at gmail.com>:
>On Mon, May 20, 2013 at 4:21 PM, Bakhtiyor Zokhidov
>< bakhtiyor_zokhidov at mail.ru > wrote:
>> Hello,
>>
>> I am using ceil() and floor() function to get upper and lower value of some
>> numbers. Let's say:
>>
>> import math
>> x1 = 0.35
>> y1 = 4.46
>>>>> math.ceil(x1)
>> 1.0
>>
>>>>> math.floor(y1)
>> 4.0
>>
>> The problem is that If I want to get upper and lower values for the certain
>> step, for example, step = 0.25, ceil() function should give:
>> new_ceil(x1, step) => 0.5
>> new_floor(y1, step) => 4.25
>> Because, the step is 0.25
>>
>> Question: How I can I achieve those results by using ceil() and floor()
>> function, or Is there any equvalent function for that?
>
>For most purposes, the following functions suffice:
>
>def new_ceil(x, step):
>????return math.ceil(x / step) * step
>
>def new_floor(x, step):
>????return math.floor(x / step) * step
>
>
>Alternately:
>
>def new_ceil(x, step):
>????quotient = x // step
>????remainder = x % step
>????return (quotient + (remainder > 0)) * step
>
>def new_floor(x, step):
>????quotient = x // step
>????return quotient * step
>
>
>Floating point representation errors and accumulated floating point
>arithmetic inaccuracies may give you unexpected results in many cases,
>so be careful.
>
>--
>Robert Kern
>_______________________________________________
>NumPy-Discussion mailing list
>NumPy-Discussion at scipy.org
>http://mail.scipy.org/mailman/listinfo/numpy-discussion

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20130520/d5a6f136/attachment.html>

From robert.kern at gmail.com  Mon May 20 11:55:47 2013
From: robert.kern at gmail.com (Robert Kern)
Date: Mon, 20 May 2013 16:55:47 +0100
Subject: [Numpy-discussion] Equvalent function for Ceil() and Floor()
In-Reply-To: <1369065266.537085267@f202.mail.ru>
References: <1369063310.820343342@f281.mail.ru>
	<CAF6FJiucd47kRKS36BPAy05uwgK65FGs1b0uc5_e_874fYXP+A@mail.gmail.com>
	<1369065266.537085267@f202.mail.ru>
Message-ID: <CAF6FJisO7dOiMaWUmKpC7uO9wQGp4nkzx-LcFvw5QjnhRi1c9Q@mail.gmail.com>

On Mon, May 20, 2013 at 4:54 PM, Bakhtiyor Zokhidov
<bakhtiyor_zokhidov at mail.ru> wrote:
> thanks a lot
>
> what about the following example:
>>>>new_ceil(-0.24, 0.25)
> -0.0
> ???
> thanks in advance for the reply

What about it? What were you expecting to get? What do you think is
wrong about the answer you did get?

--
Robert Kern


From ndbecker2 at gmail.com  Mon May 20 12:00:31 2013
From: ndbecker2 at gmail.com (Neal Becker)
Date: Mon, 20 May 2013 12:00:31 -0400
Subject: [Numpy-discussion] another indexing question
Message-ID: <kndhar$u5d$1@ger.gmane.org>

I have a system that transmits signals for an alphabet of M symbols
over and additive Gaussian noise channel.  The receiver has a
1-d array of complex received values.  I'd like to find the means
of the received values according to the symbol that was transmitted.

So transmit symbol indexes might be:

x = [0, 1, 2, 1, 3, ...]

and receive output might be:

y = [(1+1j), (1-1j), ...]

Suppose the alphabet was M=4.  Then I'd like to get an array of means

m[0...3] that correspond to the values of y for each of the corresponding
values of x.

I can't think of a better way than manually using loops.  Any tricks here?



From robert.kern at gmail.com  Mon May 20 12:03:55 2013
From: robert.kern at gmail.com (Robert Kern)
Date: Mon, 20 May 2013 17:03:55 +0100
Subject: [Numpy-discussion] another indexing question
In-Reply-To: <kndhar$u5d$1@ger.gmane.org>
References: <kndhar$u5d$1@ger.gmane.org>
Message-ID: <CAF6FJisoHdvVovnPSCJ+mXrn4YbW8fhYCtd=o-T3QbB4nO4wNg@mail.gmail.com>

On Mon, May 20, 2013 at 5:00 PM, Neal Becker <ndbecker2 at gmail.com> wrote:
> I have a system that transmits signals for an alphabet of M symbols
> over and additive Gaussian noise channel.  The receiver has a
> 1-d array of complex received values.  I'd like to find the means
> of the received values according to the symbol that was transmitted.
>
> So transmit symbol indexes might be:
>
> x = [0, 1, 2, 1, 3, ...]
>
> and receive output might be:
>
> y = [(1+1j), (1-1j), ...]
>
> Suppose the alphabet was M=4.  Then I'd like to get an array of means
>
> m[0...3] that correspond to the values of y for each of the corresponding
> values of x.
>
> I can't think of a better way than manually using loops.  Any tricks here?

All you need is a single loop over the alphabet, which is usually not
problematic.


means = np.empty([M])
for i in range(M):
    means[i] = y[x == i].mean()

--
Robert Kern


From chris.barker at noaa.gov  Mon May 20 12:03:53 2013
From: chris.barker at noaa.gov (Chris Barker - NOAA Federal)
Date: Mon, 20 May 2013 09:03:53 -0700
Subject: [Numpy-discussion] Equvalent function for Ceil() and Floor()
In-Reply-To: <1369065266.537085267@f202.mail.ru>
References: <1369063310.820343342@f281.mail.ru>
	<CAF6FJiucd47kRKS36BPAy05uwgK65FGs1b0uc5_e_874fYXP+A@mail.gmail.com>
	<1369065266.537085267@f202.mail.ru>
Message-ID: <CALGmxEL4YbBDsyZ_2QbxFMARuwDDygvP=kZp3qT+MfX8jqTx2Q@mail.gmail.com>

On Mon, May 20, 2013 at 8:54 AM, Bakhtiyor Zokhidov
<bakhtiyor_zokhidov at mail.ru> wrote:

> what about the following example:
>>>>new_ceil(-0.24, 0.25)
> -0.0

ceil rounds toward +inf (and floor towards -inf) -- this is exactly
what you want if you're doing what I think you are...(note that
round() rounds towards and away from zero -- so different for negative
numbers...)

the minus sign means "minus 0.0", which is a seemingly odd floating
point thing -- but FP numbers have a sign bit, which, in this case, is
set to negative, even though, mathematically speaking, 0.0 doesn't
have a sign. But it will compare to 0.0 as you'd expect:

In [8]: -0.0 == 0.0
Out[8]: True

In [9]: -0.0 <  0.0
Out[9]: False

-Chris

-- 

Christopher Barker, Ph.D.
Oceanographer

Emergency Response Division
NOAA/NOS/OR&R            (206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115       (206) 526-6317   main reception

Chris.Barker at noaa.gov


From bakhtiyor_zokhidov at mail.ru  Mon May 20 13:58:13 2013
From: bakhtiyor_zokhidov at mail.ru (=?UTF-8?B?QmFraHRpeW9yIFpva2hpZG92?=)
Date: Mon, 20 May 2013 21:58:13 +0400
Subject: [Numpy-discussion]
 =?utf-8?q?Equvalent_function_for_Ceil=28=29_an?=
 =?utf-8?b?ZCBGbG9vcigp?=
References: <1369063310.820343342@f281.mail.ru>
	<1369065266.537085267@f202.mail.ru>
Message-ID: <1369072693.777993222@f82.mail.ru>

 ok... I think -0.0 is mathematically wrong but in a program it is true.

What I suspect is that if -0.0 doesn't affect on result (e.g., 2*(-0.0 + 2) or (-0.0-2)*2 ) . If it does not affect results it would be good for me
Thanks


???????????, 20 ??? 2013, 9:03 -07:00 ?? Chris Barker - NOAA Federal <chris.barker at noaa.gov>:
>On Mon, May 20, 2013 at 8:54 AM, Bakhtiyor Zokhidov
>< bakhtiyor_zokhidov at mail.ru > wrote:
>
>> what about the following example:
>>>>>new_ceil(-0.24, 0.25)
>> -0.0
>
>ceil rounds toward +inf (and floor towards -inf) -- this is exactly
>what you want if you're doing what I think you are...(note that
>round() rounds towards and away from zero -- so different for negative
>numbers...)
>
>the minus sign means "minus 0.0", which is a seemingly odd floating
>point thing -- but FP numbers have a sign bit, which, in this case, is
>set to negative, even though, mathematically speaking, 0.0 doesn't
>have a sign. But it will compare to 0.0 as you'd expect:
>
>In [8]: -0.0 == 0.0
>Out[8]: True
>
>In [9]: -0.0 <  0.0
>Out[9]: False
>
>-Chris
>
>-- 
>
>Christopher Barker, Ph.D.
>Oceanographer
>
>Emergency Response Division
>NOAA/NOS/OR&R            (206) 526-6959   voice
>7600 Sand Point Way NE   (206) 526-6329   fax
>Seattle, WA  98115       (206) 526-6317   main reception
>
>Chris.Barker at noaa.gov
>_______________________________________________
>NumPy-Discussion mailing list
>NumPy-Discussion at scipy.org
>http://mail.scipy.org/mailman/listinfo/numpy-discussion

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20130520/d6c5ce24/attachment.html>

From robert.kern at gmail.com  Mon May 20 15:51:51 2013
From: robert.kern at gmail.com (Robert Kern)
Date: Mon, 20 May 2013 20:51:51 +0100
Subject: [Numpy-discussion] Equvalent function for Ceil() and Floor()
In-Reply-To: <1369072693.777993222@f82.mail.ru>
References: <1369063310.820343342@f281.mail.ru>
	<1369065266.537085267@f202.mail.ru>
	<1369072693.777993222@f82.mail.ru>
Message-ID: <CAF6FJiv4oGiCL_HqcP7gQUEZAU0hfvGt2784r-vByjWm-65nFw@mail.gmail.com>

On Mon, May 20, 2013 at 6:58 PM, Bakhtiyor Zokhidov
<bakhtiyor_zokhidov at mail.ru> wrote:
> ok... I think -0.0 is mathematically wrong but in a program it is true.
>
> What I suspect is that if -0.0 doesn't affect on result (e.g., 2*(-0.0 + 2)
> or (-0.0-2)*2 ) . If it does not affect results it would be good for me

It doesn't affect those computations, no. It does have different
results for division (1/-0.0 -> -inf), and it will often determine
which branch gets evaluated for a complex function that has a branch
point at the origin. These are usually considered good things and are
the primary reason that floating point includes signed zeros.

http://en.wikipedia.org/wiki/Signed_zero

--
Robert Kern


From mrbago at gmail.com  Mon May 20 22:45:06 2013
From: mrbago at gmail.com (Bago)
Date: Mon, 20 May 2013 19:45:06 -0700
Subject: [Numpy-discussion] another indexing question
In-Reply-To: <CAF6FJisoHdvVovnPSCJ+mXrn4YbW8fhYCtd=o-T3QbB4nO4wNg@mail.gmail.com>
References: <kndhar$u5d$1@ger.gmane.org>
	<CAF6FJisoHdvVovnPSCJ+mXrn4YbW8fhYCtd=o-T3QbB4nO4wNg@mail.gmail.com>
Message-ID: <CAHHh6CY-Q9o3p8bxneEhCQgWVpaOs7L8HDNSS+uQTPmSMeGENw@mail.gmail.com>

You could also try using bincount, (np.bincount(x, y.real) +
1j*np.bincount(x, y.imag)) / np.bincount(x)

Bago


On Mon, May 20, 2013 at 9:03 AM, Robert Kern <robert.kern at gmail.com> wrote:

> On Mon, May 20, 2013 at 5:00 PM, Neal Becker <ndbecker2 at gmail.com> wrote:
> > I have a system that transmits signals for an alphabet of M symbols
> > over and additive Gaussian noise channel.  The receiver has a
> > 1-d array of complex received values.  I'd like to find the means
> > of the received values according to the symbol that was transmitted.
> >
> > So transmit symbol indexes might be:
> >
> > x = [0, 1, 2, 1, 3, ...]
> >
> > and receive output might be:
> >
> > y = [(1+1j), (1-1j), ...]
> >
> > Suppose the alphabet was M=4.  Then I'd like to get an array of means
> >
> > m[0...3] that correspond to the values of y for each of the corresponding
> > values of x.
> >
> > I can't think of a better way than manually using loops.  Any tricks
> here?
>
> All you need is a single loop over the alphabet, which is usually not
> problematic.
>
>
> means = np.empty([M])
> for i in range(M):
>     means[i] = y[x == i].mean()
>
> --
> Robert Kern
> _______________________________________________
> NumPy-Discussion mailing list
> NumPy-Discussion at scipy.org
> http://mail.scipy.org/mailman/listinfo/numpy-discussion
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20130520/8e5b1b9e/attachment.html>

From Nicolas.Rougier at inria.fr  Wed May 22 13:07:47 2013
From: Nicolas.Rougier at inria.fr (Nicolas Rougier)
Date: Wed, 22 May 2013 19:07:47 +0200
Subject: [Numpy-discussion] Possible conversion bug with record array
Message-ID: <37980CD6-23D4-4982-939C-721DB91E4C24@inria.fr>



Hi all,

I got a weird output from the following script:

import numpy as np

U = np.zeros(1, dtype=[('x', np.float32, (4,4))])

U[0] = np.eye(4)
print U[0]
# output:  ([[0.0, 1.875, 0.0, 0.0], [0.0, 0.0, 0.0, 0.0], [0.0, 0.0, 0.0, 1.875], [0.0, 0.0, 0.0, 0.0]],)

U[0] = np.eye(4, dtype=np.float32)
print U[0]
# output:  ([[1.0, 0.0, 0.0, 0.0], [0.0, 1.0, 0.0, 0.0], [0.0, 0.0, 1.0, 0.0], [0.0, 0.0, 0.0, 1.0]],)


The first output is obviously wrong. Can anyone confirm ?
(using numpy 1.7.1 on osx 10.8.3)


Nicolas

From chris.barker at noaa.gov  Wed May 22 13:39:37 2013
From: chris.barker at noaa.gov (Chris Barker - NOAA Federal)
Date: Wed, 22 May 2013 10:39:37 -0700
Subject: [Numpy-discussion] Possible conversion bug with record array
In-Reply-To: <37980CD6-23D4-4982-939C-721DB91E4C24@inria.fr>
References: <37980CD6-23D4-4982-939C-721DB91E4C24@inria.fr>
Message-ID: <CALGmxE+-bamhRvuBi=55wPwh6PrbDUOsRm=aNX4rycP2kp74eA@mail.gmail.com>

On Wed, May 22, 2013 at 10:07 AM, Nicolas Rougier
> U = np.zeros(1, dtype=[('x', np.float32, (4,4))])
>
> U[0] = np.eye(4)
> print U[0]
> # output:  ([[0.0, 1.875, 0.0, 0.0], [0.0, 0.0, 0.0, 0.0], [0.0, 0.0, 0.0, 1.875], [0.0, 0.0, 0.0, 0.0]],)

I get the same thing. Note:

In [86]: U[0].shape
Out[86]: ()

it's a rank-zero array, not a 4X4 array -- some odd tings can happen there...

This works:

In [94]: U = np.zeros(1, dtype=[('x', np.float32, (4,4))])

In [95]: U['x'][0] = np.eye(4)

In [96]: U
Out[96]:
array([ ([[1.0, 0.0, 0.0, 0.0], [0.0, 1.0, 0.0, 0.0], [0.0, 0.0, 1.0,
0.0], [0.0, 0.0, 0.0, 1.0]],)],
      dtype=[('x', '<f4', (4, 4))])

Not that the first version isn't a bug!


-- 

Christopher Barker, Ph.D.
Oceanographer

Emergency Response Division
NOAA/NOS/OR&R            (206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115       (206) 526-6317   main reception

Chris.Barker at noaa.gov


From e.antero.tammi at gmail.com  Wed May 22 14:15:59 2013
From: e.antero.tammi at gmail.com (eat)
Date: Wed, 22 May 2013 21:15:59 +0300
Subject: [Numpy-discussion] Possible conversion bug with record array
In-Reply-To: <37980CD6-23D4-4982-939C-721DB91E4C24@inria.fr>
References: <37980CD6-23D4-4982-939C-721DB91E4C24@inria.fr>
Message-ID: <CAKa=AYSjGjGrQbwYc9cuzFdmEOr+4==Tv6E+kcmxs4-SMXodrA@mail.gmail.com>

Hi,

FWIW, apparently bug related to dtype of np.eye(.)


On Wed, May 22, 2013 at 8:07 PM, Nicolas Rougier
<Nicolas.Rougier at inria.fr>wrote:

>
>
> Hi all,
>
> I got a weird output from the following script:
>
> import numpy as np
>
> U = np.zeros(1, dtype=[('x', np.float32, (4,4))])
>
> U[0] = np.eye(4)
> print U[0]
> # output:  ([[0.0, 1.875, 0.0, 0.0], [0.0, 0.0, 0.0, 0.0], [0.0, 0.0, 0.0,
> 1.875], [0.0, 0.0, 0.0, 0.0]],)
>
> U[0] = np.eye(4, dtype=np.float32)
> print U[0]
> # output:  ([[1.0, 0.0, 0.0, 0.0], [0.0, 1.0, 0.0, 0.0], [0.0, 0.0, 1.0,
> 0.0], [0.0, 0.0, 0.0, 1.0]],)
>
>
> The first output is obviously wrong. Can anyone confirm ?
> (using numpy 1.7.1 on osx 10.8.3)
>
In []: sys.version
Out[]: '2.7.2 (default, Jun 12 2011, 15:08:59) [MSC v.1500 32 bit (Intel)]'
In []: np.__version__
Out[]: '1.6.0'
In []: U= np.zeros(1, dtype= [('x', np.float32, (4, 4))])

In []: U[0]= np.eye(4)
In []: U
Out[]:
array([ ([[0.0, 1.875, 0.0, 0.0], [0.0, 0.0, 0.0, 0.0], [0.0, 0.0, 0.0,
1.875], [0.0, 0.0, 0.0, 0.0]],)],
      dtype=[('x', '<f4', (4, 4))])

In []: U[0]= np.eye(4, dtype= np.float32)
In []: U
Out[]:
array([ ([[1.0, 0.0, 0.0, 0.0], [0.0, 1.0, 0.0, 0.0], [0.0, 0.0, 1.0, 0.0],
[0.0, 0.0, 0.0, 1.0]],)],
      dtype=[('x', '<f4', (4, 4))])

and
In []: sys.version
Out[]: '3.3.0 (v3.3.0:bd8afb90ebf2, Sep 29 2012, 10:57:17) [MSC v.1600 64
bit (AMD64)]'
In []: np.__version__
Out[]: '1.7.0rc1'
In []: U= np.zeros(1, dtype= [('x', np.float32, (4, 4))])

In []: U[0]= np.eye(4)
In []: U
Out[17]:
array([ ([[0.0, 1.875, 0.0, 0.0], [0.0, 0.0, 0.0, 0.0], [0.0, 0.0, 0.0,
1.875], [0.0, 0.0, 0.0, 0.0]],)],
      dtype=[('x', '<f4', (4, 4))])

U[0]= np.eye(4, dtype= np.float32)

In []: U
Out[]:
array([ ([[1.0, 0.0, 0.0, 0.0], [0.0, 1.0, 0.0, 0.0], [0.0, 0.0, 1.0, 0.0],
[0.0, 0.0, 0.0, 1.0]],)],
      dtype=[('x', '<f4', (4, 4))])


My 2 cents,
-eat

>
>
> Nicolas
> _______________________________________________
> NumPy-Discussion mailing list
> NumPy-Discussion at scipy.org
> http://mail.scipy.org/mailman/listinfo/numpy-discussion
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20130522/efe56d01/attachment.html>

From Nicolas.Rougier at inria.fr  Wed May 22 14:30:21 2013
From: Nicolas.Rougier at inria.fr (Nicolas Rougier)
Date: Wed, 22 May 2013 20:30:21 +0200
Subject: [Numpy-discussion] Possible conversion bug with record array
In-Reply-To: <CAKa=AYSjGjGrQbwYc9cuzFdmEOr+4==Tv6E+kcmxs4-SMXodrA@mail.gmail.com>
References: <37980CD6-23D4-4982-939C-721DB91E4C24@inria.fr>
	<CAKa=AYSjGjGrQbwYc9cuzFdmEOr+4==Tv6E+kcmxs4-SMXodrA@mail.gmail.com>
Message-ID: <1A159F81-3713-4BB6-B5D5-708D8BDD1A99@inria.fr>



Thanks, I filed a new issue on the bug tracker.

Nicolas




On May 22, 2013, at 8:15 PM, eat <e.antero.tammi at gmail.com> wrote:

> Hi,
> 
> FWIW, apparently bug related to dtype of np.eye(.)
> 
> 
> On Wed, May 22, 2013 at 8:07 PM, Nicolas Rougier <Nicolas.Rougier at inria.fr> wrote:
> 
> 
> Hi all,
> 
> I got a weird output from the following script:
> 
> import numpy as np
> 
> U = np.zeros(1, dtype=[('x', np.float32, (4,4))])
> 
> U[0] = np.eye(4)
> print U[0]
> # output:  ([[0.0, 1.875, 0.0, 0.0], [0.0, 0.0, 0.0, 0.0], [0.0, 0.0, 0.0, 1.875], [0.0, 0.0, 0.0, 0.0]],)
> 
> U[0] = np.eye(4, dtype=np.float32)
> print U[0]
> # output:  ([[1.0, 0.0, 0.0, 0.0], [0.0, 1.0, 0.0, 0.0], [0.0, 0.0, 1.0, 0.0], [0.0, 0.0, 0.0, 1.0]],)
> 
> 
> The first output is obviously wrong. Can anyone confirm ?
> (using numpy 1.7.1 on osx 10.8.3)
> In []: sys.version
> Out[]: '2.7.2 (default, Jun 12 2011, 15:08:59) [MSC v.1500 32 bit (Intel)]'
> In []: np.__version__
> Out[]: '1.6.0'
> In []: U= np.zeros(1, dtype= [('x', np.float32, (4, 4))])
> 
> In []: U[0]= np.eye(4)
> In []: U
> Out[]: 
> array([ ([[0.0, 1.875, 0.0, 0.0], [0.0, 0.0, 0.0, 0.0], [0.0, 0.0, 0.0, 1.875], [0.0, 0.0, 0.0, 0.0]],)], 
>       dtype=[('x', '<f4', (4, 4))])
> 
> In []: U[0]= np.eye(4, dtype= np.float32)
> In []: U
> Out[]: 
> array([ ([[1.0, 0.0, 0.0, 0.0], [0.0, 1.0, 0.0, 0.0], [0.0, 0.0, 1.0, 0.0], [0.0, 0.0, 0.0, 1.0]],)], 
>       dtype=[('x', '<f4', (4, 4))])
> 
> and 
> In []: sys.version
> Out[]: '3.3.0 (v3.3.0:bd8afb90ebf2, Sep 29 2012, 10:57:17) [MSC v.1600 64 bit (AMD64)]'
> In []: np.__version__
> Out[]: '1.7.0rc1'
> In []: U= np.zeros(1, dtype= [('x', np.float32, (4, 4))])
> 
> In []: U[0]= np.eye(4)
> In []: U
> Out[17]: 
> array([ ([[0.0, 1.875, 0.0, 0.0], [0.0, 0.0, 0.0, 0.0], [0.0, 0.0, 0.0, 1.875], [0.0, 0.0, 0.0, 0.0]],)], 
>       dtype=[('x', '<f4', (4, 4))])
> 
> U[0]= np.eye(4, dtype= np.float32)
> 
> In []: U
> Out[]: 
> array([ ([[1.0, 0.0, 0.0, 0.0], [0.0, 1.0, 0.0, 0.0], [0.0, 0.0, 1.0, 0.0], [0.0, 0.0, 0.0, 1.0]],)], 
>       dtype=[('x', '<f4', (4, 4))]) 
> 
> 
> My 2 cents,
> -eat
> 
> 
> Nicolas
> _______________________________________________
> NumPy-Discussion mailing list
> NumPy-Discussion at scipy.org
> http://mail.scipy.org/mailman/listinfo/numpy-discussion
> 
> _______________________________________________
> NumPy-Discussion mailing list
> NumPy-Discussion at scipy.org
> http://mail.scipy.org/mailman/listinfo/numpy-discussion



From chris.barker at noaa.gov  Wed May 22 18:09:31 2013
From: chris.barker at noaa.gov (Chris Barker - NOAA Federal)
Date: Wed, 22 May 2013 15:09:31 -0700
Subject: [Numpy-discussion] Possible conversion bug with record array
In-Reply-To: <CAKa=AYSjGjGrQbwYc9cuzFdmEOr+4==Tv6E+kcmxs4-SMXodrA@mail.gmail.com>
References: <37980CD6-23D4-4982-939C-721DB91E4C24@inria.fr>
	<CAKa=AYSjGjGrQbwYc9cuzFdmEOr+4==Tv6E+kcmxs4-SMXodrA@mail.gmail.com>
Message-ID: <CALGmxEJR-+ifZuOzgF0AatQYywgigRMgt6TDOdgO7vhYN9jZGg@mail.gmail.com>

On Wed, May 22, 2013 at 11:15 AM, eat <e.antero.tammi at gmail.com> wrote:

> FWIW, apparently bug related to dtype of np.eye(.)

sort of -- the issue shows up when assigning a float64 array (default
for eye()) to a rank-0 array with a custom dtype that has a single
object filed that is an array....numpy should do the right thing
there. If you pull the arra yout of the custom dtype object, it does
do the right thing. See the example I posted earlier.

-Chris


>
> On Wed, May 22, 2013 at 8:07 PM, Nicolas Rougier <Nicolas.Rougier at inria.fr>
> wrote:
>>
>>
>>
>> Hi all,
>>
>> I got a weird output from the following script:
>>
>> import numpy as np
>>
>> U = np.zeros(1, dtype=[('x', np.float32, (4,4))])
>>
>> U[0] = np.eye(4)
>> print U[0]
>> # output:  ([[0.0, 1.875, 0.0, 0.0], [0.0, 0.0, 0.0, 0.0], [0.0, 0.0, 0.0,
>> 1.875], [0.0, 0.0, 0.0, 0.0]],)
>>
>> U[0] = np.eye(4, dtype=np.float32)
>> print U[0]
>> # output:  ([[1.0, 0.0, 0.0, 0.0], [0.0, 1.0, 0.0, 0.0], [0.0, 0.0, 1.0,
>> 0.0], [0.0, 0.0, 0.0, 1.0]],)
>>
>>
>> The first output is obviously wrong. Can anyone confirm ?
>> (using numpy 1.7.1 on osx 10.8.3)
>
> In []: sys.version
> Out[]: '2.7.2 (default, Jun 12 2011, 15:08:59) [MSC v.1500 32 bit (Intel)]'
> In []: np.__version__
> Out[]: '1.6.0'
> In []: U= np.zeros(1, dtype= [('x', np.float32, (4, 4))])
>
> In []: U[0]= np.eye(4)
> In []: U
> Out[]:
> array([ ([[0.0, 1.875, 0.0, 0.0], [0.0, 0.0, 0.0, 0.0], [0.0, 0.0, 0.0,
> 1.875], [0.0, 0.0, 0.0, 0.0]],)],
>       dtype=[('x', '<f4', (4, 4))])
>
> In []: U[0]= np.eye(4, dtype= np.float32)
> In []: U
> Out[]:
> array([ ([[1.0, 0.0, 0.0, 0.0], [0.0, 1.0, 0.0, 0.0], [0.0, 0.0, 1.0, 0.0],
> [0.0, 0.0, 0.0, 1.0]],)],
>       dtype=[('x', '<f4', (4, 4))])
>
> and
> In []: sys.version
> Out[]: '3.3.0 (v3.3.0:bd8afb90ebf2, Sep 29 2012, 10:57:17) [MSC v.1600 64
> bit (AMD64)]'
> In []: np.__version__
> Out[]: '1.7.0rc1'
> In []: U= np.zeros(1, dtype= [('x', np.float32, (4, 4))])
>
> In []: U[0]= np.eye(4)
> In []: U
> Out[17]:
> array([ ([[0.0, 1.875, 0.0, 0.0], [0.0, 0.0, 0.0, 0.0], [0.0, 0.0, 0.0,
> 1.875], [0.0, 0.0, 0.0, 0.0]],)],
>       dtype=[('x', '<f4', (4, 4))])
>
> U[0]= np.eye(4, dtype= np.float32)
>
> In []: U
> Out[]:
> array([ ([[1.0, 0.0, 0.0, 0.0], [0.0, 1.0, 0.0, 0.0], [0.0, 0.0, 1.0, 0.0],
> [0.0, 0.0, 0.0, 1.0]],)],
>       dtype=[('x', '<f4', (4, 4))])
>
>
> My 2 cents,
> -eat
>>
>>
>>
>> Nicolas
>> _______________________________________________
>> NumPy-Discussion mailing list
>> NumPy-Discussion at scipy.org
>> http://mail.scipy.org/mailman/listinfo/numpy-discussion
>
>
>
> _______________________________________________
> NumPy-Discussion mailing list
> NumPy-Discussion at scipy.org
> http://mail.scipy.org/mailman/listinfo/numpy-discussion
>



-- 

Christopher Barker, Ph.D.
Oceanographer

Emergency Response Division
NOAA/NOS/OR&R            (206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115       (206) 526-6317   main reception

Chris.Barker at noaa.gov


From guillaume.viry at gmail.com  Wed May 22 21:12:09 2013
From: guillaume.viry at gmail.com (Guillaume VIRY)
Date: Thu, 23 May 2013 10:12:09 +0900
Subject: [Numpy-discussion] Behavior or finance.npv function
Message-ID: <CANS9D+qnf5AujrQYM0PRfvqydXFTOD7ZQ1tpw5=gRZSmFncgXg@mail.gmail.com>

Good morning all

I have been using the financial functions for a while now, and the npv
function has always bothered me. Indeed neither the documentation nor the
implementation seem to be correct, and actually are not congruent with each
other either.

As stated in the bug report https://github.com/numpy/numpy/issues/649 :

the documentation states that the summation is from 0 to M (should be M-1,
if the input data is of size M by the way) which is the correct definition,
but the implementation computes a summation from 1 to M, which is known to
be wrong, and a mimic of Excel's NPV behavior.

So basically, we would have the following :

RealNPV(data) = data[0] + CurrentNPV(data[1:])

Both behaviors are fine to me, as long as the documentation is clear about
it. Ideally the implementation should be corrected, but in any case there
should be a documentation fix at least.

I'd be glad to have your opinion about it.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20130523/ee085fc9/attachment.html>

From lists at hilboll.de  Thu May 23 06:20:26 2013
From: lists at hilboll.de (Andreas Hilboll)
Date: Thu, 23 May 2013 12:20:26 +0200
Subject: [Numpy-discussion] pyhdf packaging
In-Reply-To: <CAKUv9ONU8bkVAkxJBenDvNz44d6=Us-X404qeReOo+uqjXu8Pg@mail.gmail.com>
References: <bd3b168e7b9d9ed612f262aa7e426f16.squirrel@srv2.s4y.tournesol-consulting.eu>
	<CAKUv9ONU8bkVAkxJBenDvNz44d6=Us-X404qeReOo+uqjXu8Pg@mail.gmail.com>
Message-ID: <519DED6A.6040800@hilboll.de>

On 10.05.2013 19:32, Arnaldo Russo wrote:
> Hi Andreas,
> This packaging would be much useful!
> How can I help with this?
> pyhdf is very important because HDF4-EOS does not open with another
> packages, only with pyhdf and gdal.

Hi Arnaldo,

I actually went ahead and put the package on my PPA:

   https://launchpad.net/~andreas-h/+archive/scicomp-stable

Cheers, Andreas.


> ---
> *Arnaldo D'Amaral Pereira Granja Russo*
> Lab. de Estudos dos Oceanos e Clima
> Instituto de Oceanografia - FURG
> 
> 
> 
> 
> 2012/11/6 Andreas Hilboll <lists at hilboll.de <mailto:lists at hilboll.de>>
> 
>     Hi,
> 
>     I would like to package pyhdf for Ubuntu and make the package publicly
>     available. Since the license is not totally clear to me (I cannot
>     find any
>     information in the sources, and the cheeseshop says "public", which
>     doesn't mean anything to me), I tried to contact the maintainer, Andre
>     Gosselin, however the email bounces, so I guess he's gone.
> 
>     Can anyone point me to how to proceed from here?
> 
>     Cheers, Andreas.
> 
>     _______________________________________________
>     NumPy-Discussion mailing list
>     NumPy-Discussion at scipy.org <mailto:NumPy-Discussion at scipy.org>
>     http://mail.scipy.org/mailman/listinfo/numpy-discussion
> 
> 
> 
> 
> _______________________________________________
> NumPy-Discussion mailing list
> NumPy-Discussion at scipy.org
> http://mail.scipy.org/mailman/listinfo/numpy-discussion
> 



From Nicolas.Rougier at inria.fr  Thu May 23 09:45:45 2013
From: Nicolas.Rougier at inria.fr (Nicolas Rougier)
Date: Thu, 23 May 2013 15:45:45 +0200
Subject: [Numpy-discussion] np.dot and 'out' bug
Message-ID: <43445F94-1E61-4801-86FF-319AA6433F28@inria.fr>


Hi,

From the dot documentation, I tried something simple:

a = np.array([[1, 2], [3, 4]])
b = np.array([[1, 2], [3, 4]])
np.dot(a, b)

-> array([[ 7, 10],
          [15, 22]])

And I got expected result but if I use either a or b as output, results are wrong (and nothing in the dot documentation prevents me from doing this):

a = np.array([[1, 2], [3, 4]])
b = np.array([[1, 2], [3, 4]])
np.dot(a,b,out=a)

-> array([[ 6, 20],
          [15, 46]])

a = np.array([[1, 2], [3, 4]])
b = np.array([[1, 2], [3, 4]])
np.dot(a,b,out=b)

-> array([[ 6, 10],
          [30, 46]])


Can anyone confirm this behavior ? (tested using numpy 1.7.1)


Nicolas

From pierre.haessig at crans.org  Thu May 23 10:16:26 2013
From: pierre.haessig at crans.org (Pierre Haessig)
Date: Thu, 23 May 2013 16:16:26 +0200
Subject: [Numpy-discussion] np.dot and 'out' bug
In-Reply-To: <43445F94-1E61-4801-86FF-319AA6433F28@inria.fr>
References: <43445F94-1E61-4801-86FF-319AA6433F28@inria.fr>
Message-ID: <519E24BA.8090408@crans.org>

Hi Nicolas,

Le 23/05/2013 15:45, Nicolas Rougier a ?crit :
> if I use either a or b as output, results are wrong (and nothing in the dot documentation prevents me from doing this):
>
> a = np.array([[1, 2], [3, 4]])
> b = np.array([[1, 2], [3, 4]])
> np.dot(a,b,out=a)
>
> -> array([[ 6, 20],
>           [15, 46]])
>
>
> Can anyone confirm this behavior ? (tested using numpy 1.7.1)
I just reproduced the same weird results with numpy 1.6.2

best,
Pierre

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 900 bytes
Desc: OpenPGP digital signature
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20130523/6722f4d0/attachment.sig>

From matthieu.brucher at gmail.com  Thu May 23 10:19:23 2013
From: matthieu.brucher at gmail.com (Matthieu Brucher)
Date: Thu, 23 May 2013 15:19:23 +0100
Subject: [Numpy-discussion] np.dot and 'out' bug
In-Reply-To: <519E24BA.8090408@crans.org>
References: <43445F94-1E61-4801-86FF-319AA6433F28@inria.fr>
	<519E24BA.8090408@crans.org>
Message-ID: <CAHCaCkJfP9Jr0ChhXAKoP4UWb14osEwL6a1SBW+OdbO+xv9QeQ@mail.gmail.com>

Hi,

It's to be expected. You are overwritten one of your input vector while it
is still being used.
So not a numpy bug ;)

Matthieu


2013/5/23 Pierre Haessig <pierre.haessig at crans.org>

> Hi Nicolas,
>
> Le 23/05/2013 15:45, Nicolas Rougier a ?crit :
> > if I use either a or b as output, results are wrong (and nothing in the
> dot documentation prevents me from doing this):
> >
> > a = np.array([[1, 2], [3, 4]])
> > b = np.array([[1, 2], [3, 4]])
> > np.dot(a,b,out=a)
> >
> > -> array([[ 6, 20],
> >           [15, 46]])
> >
> >
> > Can anyone confirm this behavior ? (tested using numpy 1.7.1)
> I just reproduced the same weird results with numpy 1.6.2
>
> best,
> Pierre
>
>
> _______________________________________________
> NumPy-Discussion mailing list
> NumPy-Discussion at scipy.org
> http://mail.scipy.org/mailman/listinfo/numpy-discussion
>
>


-- 
Information System Engineer, Ph.D.
Blog: http://matt.eifelle.com
LinkedIn: http://www.linkedin.com/in/matthieubrucher
Music band: http://liliejay.com/
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20130523/91e1f866/attachment.html>

From njs at pobox.com  Thu May 23 10:42:37 2013
From: njs at pobox.com (Nathaniel Smith)
Date: Thu, 23 May 2013 15:42:37 +0100
Subject: [Numpy-discussion] np.dot and 'out' bug
In-Reply-To: <CAHCaCkJfP9Jr0ChhXAKoP4UWb14osEwL6a1SBW+OdbO+xv9QeQ@mail.gmail.com>
References: <43445F94-1E61-4801-86FF-319AA6433F28@inria.fr>
	<519E24BA.8090408@crans.org>
	<CAHCaCkJfP9Jr0ChhXAKoP4UWb14osEwL6a1SBW+OdbO+xv9QeQ@mail.gmail.com>
Message-ID: <CAPJVwBmL61ZikTddP-pMmUZE_P0GLKJZHoVFojnUxukR_X-nRA@mail.gmail.com>

On Thu, May 23, 2013 at 3:19 PM, Matthieu Brucher
<matthieu.brucher at gmail.com> wrote:
> Hi,
>
> It's to be expected. You are overwritten one of your input vector while it
> is still being used.
> So not a numpy bug ;)

Sure, that's clearly what's going on, but numpy shouldn't let you
silently shoot yourself in the foot like that. Re-using input as
output is a very common operation, and usually supported fine.
Probably we should silently make a copy of any input(s) that overlap
with the output? For high-dimensional dot, buffering temprary
subspaces would still be more memory efficient than anything users
could reasonably accomplish by hand.

-n


From Nicolas.Rougier at inria.fr  Thu May 23 10:50:11 2013
From: Nicolas.Rougier at inria.fr (Nicolas Rougier)
Date: Thu, 23 May 2013 16:50:11 +0200
Subject: [Numpy-discussion] np.dot and 'out' bug
In-Reply-To: <CAPJVwBmL61ZikTddP-pMmUZE_P0GLKJZHoVFojnUxukR_X-nRA@mail.gmail.com>
References: <43445F94-1E61-4801-86FF-319AA6433F28@inria.fr>
	<519E24BA.8090408@crans.org>
	<CAHCaCkJfP9Jr0ChhXAKoP4UWb14osEwL6a1SBW+OdbO+xv9QeQ@mail.gmail.com>
	<CAPJVwBmL61ZikTddP-pMmUZE_P0GLKJZHoVFojnUxukR_X-nRA@mail.gmail.com>
Message-ID: <3EF41063-BD7E-4B82-9E48-325D6D28DE7E@inria.fr>


> 
> Sure, that's clearly what's going on, but numpy shouldn't let you
> silently shoot yourself in the foot like that. Re-using input as
> output is a very common operation, and usually supported fine.
> Probably we should silently make a copy of any input(s) that overlap
> with the output? For high-dimensional dot, buffering temprary
> subspaces would still be more memory efficient than anything users
> could reasonably accomplish by hand.



Also, from a user point of view it is difficult to sort out which functions currently allow 'out=a' or  out=b' since nothing in the 'dot' documentation warned me about such problem.


Nicolas





From matthieu.brucher at gmail.com  Thu May 23 10:57:08 2013
From: matthieu.brucher at gmail.com (Matthieu Brucher)
Date: Thu, 23 May 2013 15:57:08 +0100
Subject: [Numpy-discussion] np.dot and 'out' bug
In-Reply-To: <3EF41063-BD7E-4B82-9E48-325D6D28DE7E@inria.fr>
References: <43445F94-1E61-4801-86FF-319AA6433F28@inria.fr>
	<519E24BA.8090408@crans.org>
	<CAHCaCkJfP9Jr0ChhXAKoP4UWb14osEwL6a1SBW+OdbO+xv9QeQ@mail.gmail.com>
	<CAPJVwBmL61ZikTddP-pMmUZE_P0GLKJZHoVFojnUxukR_X-nRA@mail.gmail.com>
	<3EF41063-BD7E-4B82-9E48-325D6D28DE7E@inria.fr>
Message-ID: <CAHCaCkJpL4CcSdvRq66L18aPQ-rY+9Aw4bdwoXYsdUvjzF3q8A@mail.gmail.com>

In my point of view, you should never use an output argument equal to an
input argument. It can impede a lot of optimizations.

Matthieu


2013/5/23 Nicolas Rougier <Nicolas.Rougier at inria.fr>

>
> >
> > Sure, that's clearly what's going on, but numpy shouldn't let you
> > silently shoot yourself in the foot like that. Re-using input as
> > output is a very common operation, and usually supported fine.
> > Probably we should silently make a copy of any input(s) that overlap
> > with the output? For high-dimensional dot, buffering temprary
> > subspaces would still be more memory efficient than anything users
> > could reasonably accomplish by hand.
>
>
>
> Also, from a user point of view it is difficult to sort out which
> functions currently allow 'out=a' or  out=b' since nothing in the 'dot'
> documentation warned me about such problem.
>
>
> Nicolas
>
>
>
> _______________________________________________
> NumPy-Discussion mailing list
> NumPy-Discussion at scipy.org
> http://mail.scipy.org/mailman/listinfo/numpy-discussion
>



-- 
Information System Engineer, Ph.D.
Blog: http://matt.eifelle.com
LinkedIn: http://www.linkedin.com/in/matthieubrucher
Music band: http://liliejay.com/
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20130523/e1e61182/attachment.html>

From njs at pobox.com  Thu May 23 11:14:41 2013
From: njs at pobox.com (Nathaniel Smith)
Date: Thu, 23 May 2013 16:14:41 +0100
Subject: [Numpy-discussion] np.dot and 'out' bug
In-Reply-To: <CAHCaCkJpL4CcSdvRq66L18aPQ-rY+9Aw4bdwoXYsdUvjzF3q8A@mail.gmail.com>
References: <43445F94-1E61-4801-86FF-319AA6433F28@inria.fr>
	<519E24BA.8090408@crans.org>
	<CAHCaCkJfP9Jr0ChhXAKoP4UWb14osEwL6a1SBW+OdbO+xv9QeQ@mail.gmail.com>
	<CAPJVwBmL61ZikTddP-pMmUZE_P0GLKJZHoVFojnUxukR_X-nRA@mail.gmail.com>
	<3EF41063-BD7E-4B82-9E48-325D6D28DE7E@inria.fr>
	<CAHCaCkJpL4CcSdvRq66L18aPQ-rY+9Aw4bdwoXYsdUvjzF3q8A@mail.gmail.com>
Message-ID: <CAPJVwBnMhxuDH1xmvn=ixaHpxSZhpu8PEh19dQzw0NRdHRAdGg@mail.gmail.com>

On Thu, May 23, 2013 at 3:57 PM, Matthieu Brucher
<matthieu.brucher at gmail.com> wrote:
> In my point of view, you should never use an output argument equal to an
> input argument. It can impede a lot of optimizations.

This is a fine philosophy in some cases, but a non-starter in others.
Python doesn't have optimizations in the first place, and in-place
operations are often critical for managing memory usage. '+=' is an
important operator, and in numpy it's just 'np.add(a, b, out=a)' under
the hood.

On Thu, May 23, 2013 at 3:50 PM, Nicolas Rougier
<Nicolas.Rougier at inria.fr> wrote:
> Also, from a user point of view it is difficult to sort out which functions currently allow 'out=a' or  out=b' since nothing in the 'dot' documentation warned me about such problem.

That's because AFAIK all functions allow out=a and out=b, except for
those which contain bugs :-).

Can you file a bug in the bug tracker so this won't get lost?

-n


From Nicolas.Rougier at inria.fr  Thu May 23 11:19:03 2013
From: Nicolas.Rougier at inria.fr (Nicolas Rougier)
Date: Thu, 23 May 2013 17:19:03 +0200
Subject: [Numpy-discussion] np.dot and 'out' bug
In-Reply-To: <CAPJVwBnMhxuDH1xmvn=ixaHpxSZhpu8PEh19dQzw0NRdHRAdGg@mail.gmail.com>
References: <43445F94-1E61-4801-86FF-319AA6433F28@inria.fr>
	<519E24BA.8090408@crans.org>
	<CAHCaCkJfP9Jr0ChhXAKoP4UWb14osEwL6a1SBW+OdbO+xv9QeQ@mail.gmail.com>
	<CAPJVwBmL61ZikTddP-pMmUZE_P0GLKJZHoVFojnUxukR_X-nRA@mail.gmail.com>
	<3EF41063-BD7E-4B82-9E48-325D6D28DE7E@inria.fr>
	<CAHCaCkJpL4CcSdvRq66L18aPQ-rY+9Aw4bdwoXYsdUvjzF3q8A@mail.gmail.com>
	<CAPJVwBnMhxuDH1xmvn=ixaHpxSZhpu8PEh19dQzw0NRdHRAdGg@mail.gmail.com>
Message-ID: <A7F59994-AB81-4CA0-B20F-445FE638CB3B@inria.fr>



> Can you file a bug in the bug tracker so this won't get lost?

Done.


From josef.pktd at gmail.com  Thu May 23 12:12:50 2013
From: josef.pktd at gmail.com (josef.pktd at gmail.com)
Date: Thu, 23 May 2013 12:12:50 -0400
Subject: [Numpy-discussion] np.dot and 'out' bug
In-Reply-To: <CAPJVwBnMhxuDH1xmvn=ixaHpxSZhpu8PEh19dQzw0NRdHRAdGg@mail.gmail.com>
References: <43445F94-1E61-4801-86FF-319AA6433F28@inria.fr>
	<519E24BA.8090408@crans.org>
	<CAHCaCkJfP9Jr0ChhXAKoP4UWb14osEwL6a1SBW+OdbO+xv9QeQ@mail.gmail.com>
	<CAPJVwBmL61ZikTddP-pMmUZE_P0GLKJZHoVFojnUxukR_X-nRA@mail.gmail.com>
	<3EF41063-BD7E-4B82-9E48-325D6D28DE7E@inria.fr>
	<CAHCaCkJpL4CcSdvRq66L18aPQ-rY+9Aw4bdwoXYsdUvjzF3q8A@mail.gmail.com>
	<CAPJVwBnMhxuDH1xmvn=ixaHpxSZhpu8PEh19dQzw0NRdHRAdGg@mail.gmail.com>
Message-ID: <CAMMTP+ACiEsSXTb_q47X364WkUOE569DAZLnaxSntivJQb_qaA@mail.gmail.com>

On Thu, May 23, 2013 at 11:14 AM, Nathaniel Smith <njs at pobox.com> wrote:
> On Thu, May 23, 2013 at 3:57 PM, Matthieu Brucher
> <matthieu.brucher at gmail.com> wrote:
>> In my point of view, you should never use an output argument equal to an
>> input argument. It can impede a lot of optimizations.
>
> This is a fine philosophy in some cases, but a non-starter in others.
> Python doesn't have optimizations in the first place, and in-place
> operations are often critical for managing memory usage. '+=' is an
> important operator, and in numpy it's just 'np.add(a, b, out=a)' under
> the hood.

Isn't dot handled by Lapack, and whatever optimized version is available?

Josef

>
> On Thu, May 23, 2013 at 3:50 PM, Nicolas Rougier
> <Nicolas.Rougier at inria.fr> wrote:
>> Also, from a user point of view it is difficult to sort out which functions currently allow 'out=a' or  out=b' since nothing in the 'dot' documentation warned me about such problem.
>
> That's because AFAIK all functions allow out=a and out=b, except for
> those which contain bugs :-).
>
> Can you file a bug in the bug tracker so this won't get lost?
>
> -n
> _______________________________________________
> NumPy-Discussion mailing list
> NumPy-Discussion at scipy.org
> http://mail.scipy.org/mailman/listinfo/numpy-discussion


From chris.barker at noaa.gov  Thu May 23 14:29:16 2013
From: chris.barker at noaa.gov (Chris Barker - NOAA Federal)
Date: Thu, 23 May 2013 11:29:16 -0700
Subject: [Numpy-discussion] np.dot and 'out' bug
In-Reply-To: <CAHCaCkJfP9Jr0ChhXAKoP4UWb14osEwL6a1SBW+OdbO+xv9QeQ@mail.gmail.com>
References: <43445F94-1E61-4801-86FF-319AA6433F28@inria.fr>
	<519E24BA.8090408@crans.org>
	<CAHCaCkJfP9Jr0ChhXAKoP4UWb14osEwL6a1SBW+OdbO+xv9QeQ@mail.gmail.com>
Message-ID: <CALGmxELEj+sNEuG2siCagOz7pJPrCkG78nGEN9AXi39_EQSxdw@mail.gmail.com>

On Thu, May 23, 2013 at 7:19 AM, Matthieu Brucher
<matthieu.brucher at gmail.com> wrote:

> It's to be expected. You are overwritten one of your input vector while it
> is still being used.
> So not a numpy bug ;)

It's a doc bug, at least.

-Chris


-- 

Christopher Barker, Ph.D.
Oceanographer

Emergency Response Division
NOAA/NOS/OR&R            (206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115       (206) 526-6317   main reception

Chris.Barker at noaa.gov


From sebastian at sipsolutions.net  Thu May 23 15:14:54 2013
From: sebastian at sipsolutions.net (Sebastian Berg)
Date: Thu, 23 May 2013 21:14:54 +0200
Subject: [Numpy-discussion] np.dot and 'out' bug
In-Reply-To: <CAPJVwBmL61ZikTddP-pMmUZE_P0GLKJZHoVFojnUxukR_X-nRA@mail.gmail.com>
References: <43445F94-1E61-4801-86FF-319AA6433F28@inria.fr>
	<519E24BA.8090408@crans.org>
	<CAHCaCkJfP9Jr0ChhXAKoP4UWb14osEwL6a1SBW+OdbO+xv9QeQ@mail.gmail.com>
	<CAPJVwBmL61ZikTddP-pMmUZE_P0GLKJZHoVFojnUxukR_X-nRA@mail.gmail.com>
Message-ID: <1369336494.18396.38.camel@sebastian-laptop>

On Thu, 2013-05-23 at 15:42 +0100, Nathaniel Smith wrote:
> On Thu, May 23, 2013 at 3:19 PM, Matthieu Brucher
> <matthieu.brucher at gmail.com> wrote:
> > Hi,
> >
> > It's to be expected. You are overwritten one of your input vector while it
> > is still being used.
> > So not a numpy bug ;)
> 
> Sure, that's clearly what's going on, but numpy shouldn't let you
> silently shoot yourself in the foot like that. Re-using input as
> output is a very common operation, and usually supported fine.
> Probably we should silently make a copy of any input(s) that overlap
> with the output? For high-dimensional dot, buffering temprary
> subspaces would still be more memory efficient than anything users
> could reasonably accomplish by hand.

Not sure whether you can easily implement buffering of subspaces with
the (g)ufunc machinery, but that shouldn't worry much. The other thing
is, that this problem generally exists for all ufuncs, only that for
normal ones it is more difficult to trigger. This is after all much like
the old, common and completely warranted complaint that `a += a.T` is
not well defined. Only that for np.dot fixing it is a bit more obvious
since nobody would abuse such in-place behaviour.

Personally I think detecting this and at least warning for now in the
(g)ufunc machinery would make a lot of sense. We can just hope someone
feels like attempting such a feature :).

- Sebastian

> 
> -n
> _______________________________________________
> NumPy-Discussion mailing list
> NumPy-Discussion at scipy.org
> http://mail.scipy.org/mailman/listinfo/numpy-discussion
> 




From charlesr.harris at gmail.com  Thu May 23 16:44:21 2013
From: charlesr.harris at gmail.com (Charles R Harris)
Date: Thu, 23 May 2013 14:44:21 -0600
Subject: [Numpy-discussion] stdint.h
Message-ID: <CAB6mnx+tRAqDHAEL+DfRY8CHcoLK9vXkv+bvtqhpqqq=+6GxfA@mail.gmail.com>

Hi all,

Just seeking some info here. The file stdint.h was part of the C99 standard
and has types for integers of specified width and thus could be used to
simplify some of the numpy configuration. I'm curious as to which compilers
might be a problem and what folks think of that possibility.

Chuck
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20130523/fbdf4860/attachment.html>

From jtaylor.debian at googlemail.com  Thu May 23 18:17:17 2013
From: jtaylor.debian at googlemail.com (Julian Taylor)
Date: Fri, 24 May 2013 00:17:17 +0200
Subject: [Numpy-discussion] faster (selection based) median, 2013 edition
In-Reply-To: <51971BDD.9000307@googlemail.com>
References: <51971BDD.9000307@googlemail.com>
Message-ID: <519E956D.4000501@googlemail.com>

On 18.05.2013 08:12, Julian Taylor wrote:
> hi,
> 
> once again I want to bring up the median algorithm which is implemented
> in terms of sorting in numpy.
> median (and percentile and a couple more functions) can be more
> efficiently implemented in terms of a selection algorithm. The
> complexity can them be linear instead of linearithmic.
> 

I filed a work in progress pull request with the code, it now includes
argpartition and crude docstrings:
https://github.com/numpy/numpy/pull/3360

please give it a try.


From chris.barker at noaa.gov  Thu May 23 18:44:44 2013
From: chris.barker at noaa.gov (Chris Barker - NOAA Federal)
Date: Thu, 23 May 2013 15:44:44 -0700
Subject: [Numpy-discussion] stdint.h
In-Reply-To: <CAB6mnx+tRAqDHAEL+DfRY8CHcoLK9vXkv+bvtqhpqqq=+6GxfA@mail.gmail.com>
References: <CAB6mnx+tRAqDHAEL+DfRY8CHcoLK9vXkv+bvtqhpqqq=+6GxfA@mail.gmail.com>
Message-ID: <CALGmxEJTK2Mjh-OVG+RwMFsd1mWwwuYuLA70BHGX-kRCXm+kaw@mail.gmail.com>

On Thu, May 23, 2013 at 1:44 PM, Charles R Harris
<charlesr.harris at gmail.com> wrote:

> Just seeking some info here. The file stdint.h was part of the C99 standard
> and has types for integers of specified width and thus could be used to
> simplify some of the numpy configuration. I'm curious as to which compilers
> might be a problem and what folks think of that possibility.

MSVC 2008, which is the (still!) the standard compiler for Python <=
2.7, does not include that header. Newer MSVC versions do.

However, I'm a big fan of stdint.h, so I think it should be used. I'll
bet we can find a header file for MSVC 2008 that could be included
with numpy.

(OK -- not bet much, but worth looking for!)

-Chris



-- 

Christopher Barker, Ph.D.
Oceanographer

Emergency Response Division
NOAA/NOS/OR&R            (206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115       (206) 526-6317   main reception

Chris.Barker at noaa.gov


From cournape at gmail.com  Thu May 23 21:29:29 2013
From: cournape at gmail.com (David Cournapeau)
Date: Fri, 24 May 2013 02:29:29 +0100
Subject: [Numpy-discussion] stdint.h
In-Reply-To: <CALGmxEJTK2Mjh-OVG+RwMFsd1mWwwuYuLA70BHGX-kRCXm+kaw@mail.gmail.com>
References: <CAB6mnx+tRAqDHAEL+DfRY8CHcoLK9vXkv+bvtqhpqqq=+6GxfA@mail.gmail.com>
	<CALGmxEJTK2Mjh-OVG+RwMFsd1mWwwuYuLA70BHGX-kRCXm+kaw@mail.gmail.com>
Message-ID: <CAGY4rcVUWNDPCufbV1qsmvzsE0iwO=v-R6sPqDrmOrPvXMjUsQ@mail.gmail.com>

On Thu, May 23, 2013 at 11:44 PM, Chris Barker - NOAA Federal
<chris.barker at noaa.gov> wrote:
> On Thu, May 23, 2013 at 1:44 PM, Charles R Harris
> <charlesr.harris at gmail.com> wrote:
>
>> Just seeking some info here. The file stdint.h was part of the C99 standard
>> and has types for integers of specified width and thus could be used to
>> simplify some of the numpy configuration. I'm curious as to which compilers
>> might be a problem and what folks think of that possibility.
>
> MSVC 2008, which is the (still!) the standard compiler for Python <=
> 2.7, does not include that header. Newer MSVC versions do.

There are stdint,h for msvc hanging around, like here:
https://code.google.com/p/msinttypes/

David


From emanuele at relativita.com  Fri May 24 08:02:22 2013
From: emanuele at relativita.com (Emanuele Olivetti)
Date: Fri, 24 May 2013 14:02:22 +0200
Subject: [Numpy-discussion] multivariate_normal issue with 'size' argument
Message-ID: <519F56CE.6000606@relativita.com>

Hi,

I'm using NumPy v1.6.1 shipped with Ubuntu 12.04 (Python 2.7.3). I observed an
odd behavior of the multivariate_normal function, which does not like int64 for
the 'size' argument.
Short example:
"""
import numpy as np
print np.random.multivariate_normal(mean=np.zeros(2), cov=np.eye(2), size=1)
print np.random.multivariate_normal(mean=np.zeros(2), cov=np.eye(2), 
size=np.int64(1))
"""

Which outputs:

"""
$ python2.7 mvn_bug.py
[[ 0.28880655  0.43289446]]
Traceback (most recent call last):
   File "mvn_bug.py", line 3, in <module>
     print np.random.multivariate_normal(mean=np.zeros(2), cov=np.eye(2), 
size=np.int64(1))
   File "mtrand.pyx", line 3990, in mtrand.RandomState.multivariate_normal 
(numpy/random/mtrand/mtrand.c:16663)
IndexError: invalid index to scalar variable.
"""

I had a brief look to the tracker but haven't found any mention of this issue.
It might be already solved in the current NumPy (v1.7.0)... or not.

I'd like to have your feedback before submitting this issue to the bug tracking 
system.

Best,

Emanuele



From francesc at continuum.io  Fri May 24 08:08:34 2013
From: francesc at continuum.io (Francesc Alted)
Date: Fri, 24 May 2013 14:08:34 +0200
Subject: [Numpy-discussion] ANN: python-blosc 1.1 (final) released
Message-ID: <519F5842.2020906@continuum.io>

===========================
Announcing python-blosc 1.1
===========================

What is it?
===========

python-blosc (http://blosc.pydata.org/) is a Python wrapper for the
Blosc compression library.

Blosc (http://blosc.org) is a high performance compressor optimized for
binary data.  It has been designed to transmit data to the processor
cache faster than the traditional, non-compressed, direct memory fetch
approach via a memcpy() OS call.  Whether this is achieved or not
depends of the data compressibility, the number of cores in the system,
and other factors.  See a series of benchmarks conducted for many
different systems: http://blosc.org/trac/wiki/SyntheticBenchmarks.

Blosc works well for compressing numerical arrays that contains data
with relatively low entropy, like sparse data, time series, grids with
regular-spaced values, etc.

There is also a handy command line for Blosc called Bloscpack
(https://github.com/esc/bloscpack) that allows you to compress large
binary datafiles on-disk.  Although the format for Bloscpack has not
stabilized yet, it allows you to effectively use Blosc from your
favorite shell.


What is new?
============

- Added new `compress_ptr` and `decompress_ptr` functions that allows to
   compress and decompress from/to a data pointer, avoiding an
   itermediate copy for maximum speed.  Be careful, as these are low
   level calls, and user must make sure that the pointer data area is
   safe.

- Since Blosc (the C library) already supports to be installed as an
   standalone library (via cmake), it is also possible to link
   python-blosc against a system Blosc library.

- The Python calls to Blosc are now thread-safe (another consequence of
   recent Blosc library supporting this at C level).

- Many checks on types and ranges of values have been added.  Most of
   the calls will now complain when passed the wrong values.

- Docstrings are much improved. Also, Sphinx-based docs are available
   now.

Many thanks to Valentin H?nel for his impressive work for this release.

For more info, you can see the release notes in:

https://github.com/FrancescAlted/python-blosc/wiki/Release-notes

More docs and examples are available in the documentation site:

http://blosc.pydata.org


Installing
==========

python-blosc is in PyPI repository, so installing it is easy:

$ pip install -U blosc  # yes, you should omit the python- prefix


Download sources
================

The sources are managed through github services at:

http://github.com/FrancescAlted/python-blosc


Documentation
=============

There is Sphinx-based documentation site at:

http://blosc.pydata.org/


Mailing list
============

There is an official mailing list for Blosc at:

blosc at googlegroups.com
http://groups.google.es/group/blosc


Licenses
========

Both Blosc and its Python wrapper are distributed using the MIT license.
See:

https://github.com/FrancescAlted/python-blosc/blob/master/LICENSES

for more details.

Enjoy!

--
Francesc Alted


From Nicolas.Rougier at inria.fr  Fri May 24 08:09:07 2013
From: Nicolas.Rougier at inria.fr (Nicolas Rougier)
Date: Fri, 24 May 2013 14:09:07 +0200
Subject: [Numpy-discussion] multivariate_normal issue with 'size'
	argument
In-Reply-To: <519F56CE.6000606@relativita.com>
References: <519F56CE.6000606@relativita.com>
Message-ID: <68CB3562-2A48-459F-9FFD-C879E39070CA@inria.fr>




Works for me (numpy 1.7.1, osx 10.8.3):

>>> import numpy as np
>>> print np.random.multivariate_normal(mean=np.zeros(2), cov=np.eye(2), size=1)
[[-0.55854737 -1.82631485]]
>>> print np.random.multivariate_normal(mean=np.zeros(2), cov=np.eye(2), size=np.int64(1))
[[ 0.40274243 -0.33922682]]



Nicolas

On May 24, 2013, at 2:02 PM, Emanuele Olivetti <emanuele at relativita.com> wrote:

> import numpy as np
> print np.random.multivariate_normal(mean=np.zeros(2), cov=np.eye(2), size=1)
> print np.random.multivariate_normal(mean=np.zeros(2), cov=np.eye(2), 
> size=np.int64(1))



From emanuele at relativita.com  Fri May 24 08:59:12 2013
From: emanuele at relativita.com (Emanuele Olivetti)
Date: Fri, 24 May 2013 14:59:12 +0200
Subject: [Numpy-discussion] multivariate_normal issue with 'size'
	argument
In-Reply-To: <68CB3562-2A48-459F-9FFD-C879E39070CA@inria.fr>
References: <519F56CE.6000606@relativita.com>
	<68CB3562-2A48-459F-9FFD-C879E39070CA@inria.fr>
Message-ID: <519F6420.70609@relativita.com>

Interesting. Anyone able to reproduce what I observe?

Emanuele

On 05/24/2013 02:09 PM, Nicolas Rougier wrote:
>
>
> Works for me (numpy 1.7.1, osx 10.8.3):
>
>>>> import numpy as np
>>>> print np.random.multivariate_normal(mean=np.zeros(2), cov=np.eye(2), size=1)
> [[-0.55854737 -1.82631485]]
>>>> print np.random.multivariate_normal(mean=np.zeros(2), cov=np.eye(2), size=np.int64(1))
> [[ 0.40274243 -0.33922682]]
>
>
>
> Nicolas
>
> On May 24, 2013, at 2:02 PM, Emanuele Olivetti <emanuele at relativita.com> wrote:
>
>> import numpy as np
>> print np.random.multivariate_normal(mean=np.zeros(2), cov=np.eye(2), size=1)
>> print np.random.multivariate_normal(mean=np.zeros(2), cov=np.eye(2),
>> size=np.int64(1))
> _______________________________________________
> NumPy-Discussion mailing list
> NumPy-Discussion at scipy.org
> http://mail.scipy.org/mailman/listinfo/numpy-discussion
>



From warren.weckesser at gmail.com  Fri May 24 09:09:47 2013
From: warren.weckesser at gmail.com (Warren Weckesser)
Date: Fri, 24 May 2013 09:09:47 -0400
Subject: [Numpy-discussion] multivariate_normal issue with 'size'
	argument
In-Reply-To: <519F6420.70609@relativita.com>
References: <519F56CE.6000606@relativita.com>
	<68CB3562-2A48-459F-9FFD-C879E39070CA@inria.fr>
	<519F6420.70609@relativita.com>
Message-ID: <CAGzF1ufYUpU+55Ofb18boO_Db=o7bmyxPrg9jLBeAkm_FDqXpw@mail.gmail.com>

On 5/24/13, Emanuele Olivetti <emanuele at relativita.com> wrote:
> Interesting. Anyone able to reproduce what I observe?


Yes.  I'm also using Ubuntu 12.04.  With numpy 1.6.1, I get the same
error, but it works fine with numpy 1.7.1.

Warren


>
> Emanuele
>
> On 05/24/2013 02:09 PM, Nicolas Rougier wrote:
>>
>>
>> Works for me (numpy 1.7.1, osx 10.8.3):
>>
>>>>> import numpy as np
>>>>> print np.random.multivariate_normal(mean=np.zeros(2), cov=np.eye(2),
>>>>> size=1)
>> [[-0.55854737 -1.82631485]]
>>>>> print np.random.multivariate_normal(mean=np.zeros(2), cov=np.eye(2),
>>>>> size=np.int64(1))
>> [[ 0.40274243 -0.33922682]]
>>
>>
>>
>> Nicolas
>>
>> On May 24, 2013, at 2:02 PM, Emanuele Olivetti <emanuele at relativita.com>
>> wrote:
>>
>>> import numpy as np
>>> print np.random.multivariate_normal(mean=np.zeros(2), cov=np.eye(2),
>>> size=1)
>>> print np.random.multivariate_normal(mean=np.zeros(2), cov=np.eye(2),
>>> size=np.int64(1))
>> _______________________________________________
>> NumPy-Discussion mailing list
>> NumPy-Discussion at scipy.org
>> http://mail.scipy.org/mailman/listinfo/numpy-discussion
>>
>
> _______________________________________________
> NumPy-Discussion mailing list
> NumPy-Discussion at scipy.org
> http://mail.scipy.org/mailman/listinfo/numpy-discussion
>


From daniele at grinta.net  Fri May 24 09:11:42 2013
From: daniele at grinta.net (Daniele Nicolodi)
Date: Fri, 24 May 2013 15:11:42 +0200
Subject: [Numpy-discussion] multivariate_normal issue with 'size'
	argument
In-Reply-To: <519F56CE.6000606@relativita.com>
References: <519F56CE.6000606@relativita.com>
Message-ID: <519F670E.5080506@grinta.net>

On 24/05/2013 14:02, Emanuele Olivetti wrote:
> Hi,
> 
> I'm using NumPy v1.6.1 shipped with Ubuntu 12.04 (Python 2.7.3). I observed an
> odd behavior of the multivariate_normal function, which does not like int64 for
> the 'size' argument.
> Short example:
> """
> import numpy as np
> print np.random.multivariate_normal(mean=np.zeros(2), cov=np.eye(2), size=1)
> print np.random.multivariate_normal(mean=np.zeros(2), cov=np.eye(2), 
> size=np.int64(1))
> """
> 
> Which outputs:
> 
> """
> $ python2.7 mvn_bug.py
> [[ 0.28880655  0.43289446]]
> Traceback (most recent call last):
>    File "mvn_bug.py", line 3, in <module>
>      print np.random.multivariate_normal(mean=np.zeros(2), cov=np.eye(2), 
> size=np.int64(1))
>    File "mtrand.pyx", line 3990, in mtrand.RandomState.multivariate_normal 
> (numpy/random/mtrand/mtrand.c:16663)
> IndexError: invalid index to scalar variable.
> """

Hello Emanuele,

it works for me with numpy 1.6.2 (python 2.7.3 on mac os x 10.6 via
macports but is should not matter).

Cheers,
Daniele



From p.j.a.cock at googlemail.com  Fri May 24 09:12:32 2013
From: p.j.a.cock at googlemail.com (Peter Cock)
Date: Fri, 24 May 2013 14:12:32 +0100
Subject: [Numpy-discussion] multivariate_normal issue with 'size'
	argument
In-Reply-To: <519F6420.70609@relativita.com>
References: <519F56CE.6000606@relativita.com>
	<68CB3562-2A48-459F-9FFD-C879E39070CA@inria.fr>
	<519F6420.70609@relativita.com>
Message-ID: <CAKVJ-_6q8tiExyCEcno5o+gKQdZCJwDUGoA142PctHLuqZ-Q1g@mail.gmail.com>

On Fri, May 24, 2013 at 1:59 PM, Emanuele Olivetti
<emanuele at relativita.com> wrote:
> Interesting. Anyone able to reproduce what I observe?
>
> Emanuele


Yes, I can reproduce this IndexError under Mac OS X:

$ which python2.7
/usr/bin/python2.7
$ python2.7
Python 2.7.2 (default, Oct 11 2012, 20:14:37)
[GCC 4.2.1 Compatible Apple Clang 4.0 (tags/Apple/clang-418.0.60)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import numpy as np
>>> print np.random.multivariate_normal(mean=np.zeros(2), cov=np.eye(2), size=1)
[[ 0.68446902  1.84926031]]
>>> print np.random.multivariate_normal(mean=np.zeros(2), cov=np.eye(2), size=np.int64(1))
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "mtrand.pyx", line 3990, in
mtrand.RandomState.multivariate_normal
(numpy/random/mtrand/mtrand.c:16663)
IndexError: invalid index to scalar variable.
>>> np.__version__
'1.6.1'
>>> quit()

And on a more recent self-compiled Python and NumPy,

$ which python3.3
/Users/pjcock/bin/python3.3
$ python3.3
Python 3.3.1 (default, Apr  8 2013, 17:54:08)
[GCC 4.2.1 Compatible Apple Clang 4.0 ((tags/Apple/clang-421.0.57))] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import numpy as np
>>> print(np.random.multivariate_normal(mean=np.zeros(2), cov=np.eye(2), size=1))
[[-0.57757621  1.09307893]]
>>> print(np.random.multivariate_normal(mean=np.zeros(2), cov=np.eye(2), size=np.int64(1)))
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "mtrand.pyx", line 4161, in
mtrand.RandomState.multivariate_normal
(numpy/random/mtrand/mtrand.c:19140)
IndexError: invalid index to scalar variable.
>>> np.__version__
'1.7.1'

Peter


From robert.kern at gmail.com  Fri May 24 09:15:09 2013
From: robert.kern at gmail.com (Robert Kern)
Date: Fri, 24 May 2013 09:15:09 -0400
Subject: [Numpy-discussion] multivariate_normal issue with 'size'
	argument
In-Reply-To: <CAKVJ-_6q8tiExyCEcno5o+gKQdZCJwDUGoA142PctHLuqZ-Q1g@mail.gmail.com>
References: <519F56CE.6000606@relativita.com>
	<68CB3562-2A48-459F-9FFD-C879E39070CA@inria.fr>
	<519F6420.70609@relativita.com>
	<CAKVJ-_6q8tiExyCEcno5o+gKQdZCJwDUGoA142PctHLuqZ-Q1g@mail.gmail.com>
Message-ID: <CAF6FJiv39DFM8sJUOcsTmHundUPf3mcd2r43w7f==7gAAOXodQ@mail.gmail.com>

On Fri, May 24, 2013 at 9:12 AM, Peter Cock <p.j.a.cock at googlemail.com> wrote:
> On Fri, May 24, 2013 at 1:59 PM, Emanuele Olivetti
> <emanuele at relativita.com> wrote:
>> Interesting. Anyone able to reproduce what I observe?
>>
>> Emanuele
>
>
> Yes, I can reproduce this IndexError under Mac OS X:
>
> $ which python2.7
> /usr/bin/python2.7
> $ python2.7
> Python 2.7.2 (default, Oct 11 2012, 20:14:37)
> [GCC 4.2.1 Compatible Apple Clang 4.0 (tags/Apple/clang-418.0.60)] on darwin
> Type "help", "copyright", "credits" or "license" for more information.

Can everyone please report whether they have a 32-bit build of Python
or a 64-bit build? That's probably the most relevant factor.

--
Robert Kern


From daniele at grinta.net  Fri May 24 09:27:22 2013
From: daniele at grinta.net (Daniele Nicolodi)
Date: Fri, 24 May 2013 15:27:22 +0200
Subject: [Numpy-discussion] multivariate_normal issue with 'size'
	argument
In-Reply-To: <519F670E.5080506@grinta.net>
References: <519F56CE.6000606@relativita.com> <519F670E.5080506@grinta.net>
Message-ID: <519F6ABA.1040009@grinta.net>

On 24/05/2013 15:11, Daniele Nicolodi wrote:
> Hello Emanuele,
> 
> it works for me with numpy 1.6.2 (python 2.7.3 on mac os x 10.6 via
> macports but is should not matter).

64-bit build

Cheers,
Daniele



From p.j.a.cock at googlemail.com  Fri May 24 09:36:30 2013
From: p.j.a.cock at googlemail.com (Peter Cock)
Date: Fri, 24 May 2013 14:36:30 +0100
Subject: [Numpy-discussion] multivariate_normal issue with 'size'
	argument
In-Reply-To: <CAF6FJiv39DFM8sJUOcsTmHundUPf3mcd2r43w7f==7gAAOXodQ@mail.gmail.com>
References: <519F56CE.6000606@relativita.com>
	<68CB3562-2A48-459F-9FFD-C879E39070CA@inria.fr>
	<519F6420.70609@relativita.com>
	<CAKVJ-_6q8tiExyCEcno5o+gKQdZCJwDUGoA142PctHLuqZ-Q1g@mail.gmail.com>
	<CAF6FJiv39DFM8sJUOcsTmHundUPf3mcd2r43w7f==7gAAOXodQ@mail.gmail.com>
Message-ID: <CAKVJ-_5cuCJvZZs6S=_pJ-HoGEWQM+ARU6mC50WDOR91fE4JpA@mail.gmail.com>

On Fri, May 24, 2013 at 2:15 PM, Robert Kern <robert.kern at gmail.com> wrote:
> On Fri, May 24, 2013 at 9:12 AM, Peter Cock <p.j.a.cock at googlemail.com> wrote:
>> On Fri, May 24, 2013 at 1:59 PM, Emanuele Olivetti
>> <emanuele at relativita.com> wrote:
>>> Interesting. Anyone able to reproduce what I observe?
>>>
>>> Emanuele
>>
>>
>> Yes, I can reproduce this IndexError under Mac OS X:
>>
>> $ which python2.7
>> /usr/bin/python2.7
>> $ python2.7
>> Python 2.7.2 (default, Oct 11 2012, 20:14:37)
>> [GCC 4.2.1 Compatible Apple Clang 4.0 (tags/Apple/clang-418.0.60)] on darwin
>> Type "help", "copyright", "credits" or "license" for more information.
>
> Can everyone please report whether they have a 32-bit build of Python
> or a 64-bit build? That's probably the most relevant factor.

It seems to affect all of 32 bit Windows XP, 64 bit Mac, 32 bit Linux,
and 64 bit Linux
for some versions of NumPy...  Thus far the only non-failure I've seen
is 64 bit Linux,
Python 2.6.6 with NumPy 1.6.2 (other Python/NumPy installs on this
machine do fail).

Its a bit strange - I don't see any obvious pattern.

Peter

-------------------------------------------

Failures:

My Python installs on this Mac all seem to be 64bit (and fail),

$ python3.3
Python 3.3.1 (default, Apr  8 2013, 17:54:08)
[GCC 4.2.1 Compatible Apple Clang 4.0 ((tags/Apple/clang-421.0.57))] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import sys;print("%x" % sys.maxsize, sys.maxsize > 2**32)
7fffffffffffffff True
>>> import numpy as np
>>> print(np.random.multivariate_normal(mean=np.zeros(2), cov=np.eye(2), size=1))
[[ 1.80932387  0.85894164]]
>>> print(np.random.multivariate_normal(mean=np.zeros(2), cov=np.eye(2), size=np.int64(1)))
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "mtrand.pyx", line 4161, in
mtrand.RandomState.multivariate_normal
(numpy/random/mtrand/mtrand.c:19140)
IndexError: invalid index to scalar variable.
>>> np.__version__
'1.7.1'
>>> quit()

This also affects NumPy 1.5 so this isn't a recent regression:

$ python3.2
Python 3.2 (r32:88445, Feb 28 2011, 17:04:33)
[GCC 4.2.1 (Apple Inc. build 5664)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import sys;print("%x" % sys.maxsize, sys.maxsize > 2**32)
7fffffffffffffff True
>>> import numpy as np
>>> print(np.random.multivariate_normal(mean=np.zeros(2), cov=np.eye(2), size=1))
[[ 1.11403341 -1.67856405]]
>>> print(np.random.multivariate_normal(mean=np.zeros(2), cov=np.eye(2), size=np.int64(1)))
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "mtrand.pyx", line 3954, in
mtrand.RandomState.multivariate_normal
(numpy/random/mtrand/mtrand.c:17234)
IndexError: invalid index to scalar variable.
>>> np.__version__
'1.5.0'

$ python3.1
Python 3.1.2 (r312:79147, Nov 15 2010, 16:28:52)
[GCC 4.2.1 (Apple Inc. build 5664)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import sys;print("%x" % sys.maxsize, sys.maxsize > 2**32)
7fffffffffffffff True
>>> import numpy as np
>>> print(np.random.multivariate_normal(mean=np.zeros(2), cov=np.eye(2), size=1))
[[ 0.3834108  -0.31124203]]
>>> print(np.random.multivariate_normal(mean=np.zeros(2), cov=np.eye(2), size=np.int64(1)))
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "mtrand.pyx", line 3954, in
mtrand.RandomState.multivariate_normal
(numpy/random/mtrand/mtrand.c:17234)
IndexError: invalid index to scalar variable.
>>> np.__version__
'1.5.0'
>>> quit()

And on my 32 bit Windows XP box,

Python 2.7 (r27:82525, Jul  4 2010, 09:01:59) [MSC v.1500 32 bit
(Intel)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import sys;print("%x" % sys.maxsize, sys.maxsize > 2**32)
('7fffffff', False)
>>> import numpy as np
>>> print(np.random.multivariate_normal(mean=np.zeros(2), cov=np.eye(2), size=1))
[[-0.35072523 -0.58046885]]
>>> print(np.random.multivariate_normal(mean=np.zeros(2), cov=np.eye(2), size=np.int64(1)))
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "mtrand.pyx", line 3954, in
mtrand.RandomState.multivariate_normal
(numpy\random\mtrand\mtrand.c:17234)
IndexError: invalid index to scalar variable.
>>> np.__version__
'1.5.0'
>>>

Python 3.3.0 (v3.3.0:bd8afb90ebf2, Sep 29 2012, 10:55:48) [MSC v.1600
32 bit (Intel)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import numpy as np
>>> print(np.random.multivariate_normal(mean=np.zeros(2), cov=np.eye(2), size=1))
[[-0.00453374  0.2210342 ]]
>>> print(np.random.multivariate_normal(mean=np.zeros(2), cov=np.eye(2), size=np.int64(1)))
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "mtrand.pyx", line 4142, in
mtrand.RandomState.multivariate_normal
(numpy\random\mtrand\mtrand.c:19128)
IndexError: invalid index to scalar variable.
>>>
>>> np.__version__
'1.7.0rc2'

Here's a couple of runs from an old 32 bit Linux machine which also
shows the problem:

$ python2.7
Python 2.7 (r27:82500, Nov 12 2010, 14:19:08)
[GCC 4.1.2 20070626 (Red Hat 4.1.2-13)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import sys;print("%x" % sys.maxsize, sys.maxsize > 2**32)
('7fffffff', False)
>>> import numpy as np
>>> print(np.random.multivariate_normal(mean=np.zeros(2), cov=np.eye(2), size=1))
[[ 0.40887932 -0.09872295]]
>>> print(np.random.multivariate_normal(mean=np.zeros(2), cov=np.eye(2), size=np.int64(1)))
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "mtrand.pyx", line 3952, in
mtrand.RandomState.multivariate_normal
(numpy/random/mtrand/mtrand.c:17027)
IndexError: invalid index to scalar variable.
>>> np.__version__
'1.4.1'
>>> quit()

$ python3.3
Python 3.3.0 (default, Oct 31 2012, 13:19:08)
[GCC 4.1.2 20070626 (Red Hat 4.1.2-13)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import sys;print("%x" % sys.maxsize, sys.maxsize > 2**32)
7fffffff False
>>> import numpy as np
>>> print(np.random.multivariate_normal(mean=np.zeros(2), cov=np.eye(2), size=1))
[[-0.16796654 -1.71312415]]
>>> print(np.random.multivariate_normal(mean=np.zeros(2), cov=np.eye(2), size=np.int64(1)))
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "mtrand.pyx", line 4127, in
mtrand.RandomState.multivariate_normal
(numpy/random/mtrand/mtrand.c:18657)
IndexError: invalid index to scalar variable.
>>> np.__version__
'1.7.0b2'
>>> quit()

And a more up to date 64 bit Linux machine where this fails on
some versions of NumPy:

$ python3.3
Python 3.3.1 (default, Apr  8 2013, 14:38:06)
[GCC 4.4.6 20120305 (Red Hat 4.4.6-4)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import sys;print("%x" % sys.maxsize, sys.maxsize > 2**32)
7fffffffffffffff True
>>> import numpy as np
>>> print(np.random.multivariate_normal(mean=np.zeros(2), cov=np.eye(2), size=1))
[[-0.18862683 -0.92177665]]
>>> print(np.random.multivariate_normal(mean=np.zeros(2), cov=np.eye(2), size=np.int64(1)))
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "mtrand.pyx", line 4127, in
mtrand.RandomState.multivariate_normal
(numpy/random/mtrand/mtrand.c:18657)
IndexError: invalid index to scalar variable.
>>> np.__version__
'1.7.0b2'
>>> quit()

-------------------------------------------
Successes
-------------------------------------------

64 bit Linux:

$ python2.6
Python 2.6.6 (r266:84292, Sep 11 2012, 08:34:23)
[GCC 4.4.6 20120305 (Red Hat 4.4.6-4)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import sys;print("%x" % sys.maxsize, sys.maxsize > 2**32)
('7fffffffffffffff', True)
>>> import numpy as np
>>> print(np.random.multivariate_normal(mean=np.zeros(2), cov=np.eye(2), size=1))
[[-0.27469218 -2.12911784]]
>>> print(np.random.multivariate_normal(mean=np.zeros(2), cov=np.eye(2), size=np.int64(1)))
[[ 0.02609307  0.32485211]]
>>> np.__version__
'1.6.2'
>>> quit()



Peter


From warren.weckesser at gmail.com  Fri May 24 09:47:30 2013
From: warren.weckesser at gmail.com (Warren Weckesser)
Date: Fri, 24 May 2013 09:47:30 -0400
Subject: [Numpy-discussion] multivariate_normal issue with 'size'
	argument
In-Reply-To: <CAKVJ-_5cuCJvZZs6S=_pJ-HoGEWQM+ARU6mC50WDOR91fE4JpA@mail.gmail.com>
References: <519F56CE.6000606@relativita.com>
	<68CB3562-2A48-459F-9FFD-C879E39070CA@inria.fr>
	<519F6420.70609@relativita.com>
	<CAKVJ-_6q8tiExyCEcno5o+gKQdZCJwDUGoA142PctHLuqZ-Q1g@mail.gmail.com>
	<CAF6FJiv39DFM8sJUOcsTmHundUPf3mcd2r43w7f==7gAAOXodQ@mail.gmail.com>
	<CAKVJ-_5cuCJvZZs6S=_pJ-HoGEWQM+ARU6mC50WDOR91fE4JpA@mail.gmail.com>
Message-ID: <CAGzF1udYc3NGUbMKj1OqYC2muQb2h6vm87Q8Nszr-vvnNSO4Tw@mail.gmail.com>

On 5/24/13, Peter Cock <p.j.a.cock at googlemail.com> wrote:
> On Fri, May 24, 2013 at 2:15 PM, Robert Kern <robert.kern at gmail.com> wrote:
>> On Fri, May 24, 2013 at 9:12 AM, Peter Cock <p.j.a.cock at googlemail.com>
>> wrote:
>>> On Fri, May 24, 2013 at 1:59 PM, Emanuele Olivetti
>>> <emanuele at relativita.com> wrote:
>>>> Interesting. Anyone able to reproduce what I observe?
>>>>
>>>> Emanuele
>>>
>>>
>>> Yes, I can reproduce this IndexError under Mac OS X:
>>>
>>> $ which python2.7
>>> /usr/bin/python2.7
>>> $ python2.7
>>> Python 2.7.2 (default, Oct 11 2012, 20:14:37)
>>> [GCC 4.2.1 Compatible Apple Clang 4.0 (tags/Apple/clang-418.0.60)] on
>>> darwin
>>> Type "help", "copyright", "credits" or "license" for more information.
>>
>> Can everyone please report whether they have a 32-bit build of Python
>> or a 64-bit build? That's probably the most relevant factor.
>
> It seems to affect all of 32 bit Windows XP, 64 bit Mac, 32 bit Linux,
> and 64 bit Linux
> for some versions of NumPy...  Thus far the only non-failure I've seen
> is 64 bit Linux,
> Python 2.6.6 with NumPy 1.6.2 (other Python/NumPy installs on this
> machine do fail).
>
> Its a bit strange - I don't see any obvious pattern.
>
> Peter
>
> -------------------------------------------
>
> Failures:
>
> My Python installs on this Mac all seem to be 64bit (and fail),
>
> $ python3.3
> Python 3.3.1 (default, Apr  8 2013, 17:54:08)
> [GCC 4.2.1 Compatible Apple Clang 4.0 ((tags/Apple/clang-421.0.57))] on
> darwin
> Type "help", "copyright", "credits" or "license" for more information.
>>>> import sys;print("%x" % sys.maxsize, sys.maxsize > 2**32)
> 7fffffffffffffff True
>>>> import numpy as np
>>>> print(np.random.multivariate_normal(mean=np.zeros(2), cov=np.eye(2),
>>>> size=1))
> [[ 1.80932387  0.85894164]]
>>>> print(np.random.multivariate_normal(mean=np.zeros(2), cov=np.eye(2),
>>>> size=np.int64(1)))
> Traceback (most recent call last):
>   File "<stdin>", line 1, in <module>
>   File "mtrand.pyx", line 4161, in
> mtrand.RandomState.multivariate_normal
> (numpy/random/mtrand/mtrand.c:19140)
> IndexError: invalid index to scalar variable.
>>>> np.__version__
> '1.7.1'
>>>> quit()
>
> This also affects NumPy 1.5 so this isn't a recent regression:
>
> $ python3.2
> Python 3.2 (r32:88445, Feb 28 2011, 17:04:33)
> [GCC 4.2.1 (Apple Inc. build 5664)] on darwin
> Type "help", "copyright", "credits" or "license" for more information.
>>>> import sys;print("%x" % sys.maxsize, sys.maxsize > 2**32)
> 7fffffffffffffff True
>>>> import numpy as np
>>>> print(np.random.multivariate_normal(mean=np.zeros(2), cov=np.eye(2),
>>>> size=1))
> [[ 1.11403341 -1.67856405]]
>>>> print(np.random.multivariate_normal(mean=np.zeros(2), cov=np.eye(2),
>>>> size=np.int64(1)))
> Traceback (most recent call last):
>   File "<stdin>", line 1, in <module>
>   File "mtrand.pyx", line 3954, in
> mtrand.RandomState.multivariate_normal
> (numpy/random/mtrand/mtrand.c:17234)
> IndexError: invalid index to scalar variable.
>>>> np.__version__
> '1.5.0'
>
> $ python3.1
> Python 3.1.2 (r312:79147, Nov 15 2010, 16:28:52)
> [GCC 4.2.1 (Apple Inc. build 5664)] on darwin
> Type "help", "copyright", "credits" or "license" for more information.
>>>> import sys;print("%x" % sys.maxsize, sys.maxsize > 2**32)
> 7fffffffffffffff True
>>>> import numpy as np
>>>> print(np.random.multivariate_normal(mean=np.zeros(2), cov=np.eye(2),
>>>> size=1))
> [[ 0.3834108  -0.31124203]]
>>>> print(np.random.multivariate_normal(mean=np.zeros(2), cov=np.eye(2),
>>>> size=np.int64(1)))
> Traceback (most recent call last):
>   File "<stdin>", line 1, in <module>
>   File "mtrand.pyx", line 3954, in
> mtrand.RandomState.multivariate_normal
> (numpy/random/mtrand/mtrand.c:17234)
> IndexError: invalid index to scalar variable.
>>>> np.__version__
> '1.5.0'
>>>> quit()
>
> And on my 32 bit Windows XP box,
>
> Python 2.7 (r27:82525, Jul  4 2010, 09:01:59) [MSC v.1500 32 bit
> (Intel)] on win32
> Type "help", "copyright", "credits" or "license" for more information.
>>>> import sys;print("%x" % sys.maxsize, sys.maxsize > 2**32)
> ('7fffffff', False)
>>>> import numpy as np
>>>> print(np.random.multivariate_normal(mean=np.zeros(2), cov=np.eye(2),
>>>> size=1))
> [[-0.35072523 -0.58046885]]
>>>> print(np.random.multivariate_normal(mean=np.zeros(2), cov=np.eye(2),
>>>> size=np.int64(1)))
> Traceback (most recent call last):
>   File "<stdin>", line 1, in <module>
>   File "mtrand.pyx", line 3954, in
> mtrand.RandomState.multivariate_normal
> (numpy\random\mtrand\mtrand.c:17234)
> IndexError: invalid index to scalar variable.
>>>> np.__version__
> '1.5.0'
>>>>
>
> Python 3.3.0 (v3.3.0:bd8afb90ebf2, Sep 29 2012, 10:55:48) [MSC v.1600
> 32 bit (Intel)] on win32
> Type "help", "copyright", "credits" or "license" for more information.
>>>> import numpy as np
>>>> print(np.random.multivariate_normal(mean=np.zeros(2), cov=np.eye(2),
>>>> size=1))
> [[-0.00453374  0.2210342 ]]
>>>> print(np.random.multivariate_normal(mean=np.zeros(2), cov=np.eye(2),
>>>> size=np.int64(1)))
> Traceback (most recent call last):
>   File "<stdin>", line 1, in <module>
>   File "mtrand.pyx", line 4142, in
> mtrand.RandomState.multivariate_normal
> (numpy\random\mtrand\mtrand.c:19128)
> IndexError: invalid index to scalar variable.
>>>>
>>>> np.__version__
> '1.7.0rc2'
>
> Here's a couple of runs from an old 32 bit Linux machine which also
> shows the problem:
>
> $ python2.7
> Python 2.7 (r27:82500, Nov 12 2010, 14:19:08)
> [GCC 4.1.2 20070626 (Red Hat 4.1.2-13)] on linux2
> Type "help", "copyright", "credits" or "license" for more information.
>>>> import sys;print("%x" % sys.maxsize, sys.maxsize > 2**32)
> ('7fffffff', False)
>>>> import numpy as np
>>>> print(np.random.multivariate_normal(mean=np.zeros(2), cov=np.eye(2),
>>>> size=1))
> [[ 0.40887932 -0.09872295]]
>>>> print(np.random.multivariate_normal(mean=np.zeros(2), cov=np.eye(2),
>>>> size=np.int64(1)))
> Traceback (most recent call last):
>   File "<stdin>", line 1, in <module>
>   File "mtrand.pyx", line 3952, in
> mtrand.RandomState.multivariate_normal
> (numpy/random/mtrand/mtrand.c:17027)
> IndexError: invalid index to scalar variable.
>>>> np.__version__
> '1.4.1'
>>>> quit()
>
> $ python3.3
> Python 3.3.0 (default, Oct 31 2012, 13:19:08)
> [GCC 4.1.2 20070626 (Red Hat 4.1.2-13)] on linux
> Type "help", "copyright", "credits" or "license" for more information.
>>>> import sys;print("%x" % sys.maxsize, sys.maxsize > 2**32)
> 7fffffff False
>>>> import numpy as np
>>>> print(np.random.multivariate_normal(mean=np.zeros(2), cov=np.eye(2),
>>>> size=1))
> [[-0.16796654 -1.71312415]]
>>>> print(np.random.multivariate_normal(mean=np.zeros(2), cov=np.eye(2),
>>>> size=np.int64(1)))
> Traceback (most recent call last):
>   File "<stdin>", line 1, in <module>
>   File "mtrand.pyx", line 4127, in
> mtrand.RandomState.multivariate_normal
> (numpy/random/mtrand/mtrand.c:18657)
> IndexError: invalid index to scalar variable.
>>>> np.__version__
> '1.7.0b2'
>>>> quit()
>
> And a more up to date 64 bit Linux machine where this fails on
> some versions of NumPy:
>
> $ python3.3
> Python 3.3.1 (default, Apr  8 2013, 14:38:06)
> [GCC 4.4.6 20120305 (Red Hat 4.4.6-4)] on linux
> Type "help", "copyright", "credits" or "license" for more information.
>>>> import sys;print("%x" % sys.maxsize, sys.maxsize > 2**32)
> 7fffffffffffffff True
>>>> import numpy as np
>>>> print(np.random.multivariate_normal(mean=np.zeros(2), cov=np.eye(2),
>>>> size=1))
> [[-0.18862683 -0.92177665]]
>>>> print(np.random.multivariate_normal(mean=np.zeros(2), cov=np.eye(2),
>>>> size=np.int64(1)))
> Traceback (most recent call last):
>   File "<stdin>", line 1, in <module>
>   File "mtrand.pyx", line 4127, in
> mtrand.RandomState.multivariate_normal
> (numpy/random/mtrand/mtrand.c:18657)
> IndexError: invalid index to scalar variable.
>>>> np.__version__
> '1.7.0b2'
>>>> quit()
>
> -------------------------------------------
> Successes
> -------------------------------------------
>
> 64 bit Linux:
>
> $ python2.6
> Python 2.6.6 (r266:84292, Sep 11 2012, 08:34:23)
> [GCC 4.4.6 20120305 (Red Hat 4.4.6-4)] on linux2
> Type "help", "copyright", "credits" or "license" for more information.
>>>> import sys;print("%x" % sys.maxsize, sys.maxsize > 2**32)
> ('7fffffffffffffff', True)
>>>> import numpy as np
>>>> print(np.random.multivariate_normal(mean=np.zeros(2), cov=np.eye(2),
>>>> size=1))
> [[-0.27469218 -2.12911784]]
>>>> print(np.random.multivariate_normal(mean=np.zeros(2), cov=np.eye(2),
>>>> size=np.int64(1)))
> [[ 0.02609307  0.32485211]]
>>>> np.__version__
> '1.6.2'
>>>> quit()
>
>


Peter: wow, that's a lot of tests!

Two more data points:
On Ubuntu 12.04, using 64 bit builds of Python 2.7.4 (from Anaconda
1.5.0), and numpy built from source: numpy 1.6.1 gives the error, but
1.6.2 does not.

Warren


>
> Peter
> _______________________________________________
> NumPy-Discussion mailing list
> NumPy-Discussion at scipy.org
> http://mail.scipy.org/mailman/listinfo/numpy-discussion
>


From p.j.a.cock at googlemail.com  Fri May 24 09:56:28 2013
From: p.j.a.cock at googlemail.com (Peter Cock)
Date: Fri, 24 May 2013 14:56:28 +0100
Subject: [Numpy-discussion] multivariate_normal issue with 'size'
	argument
In-Reply-To: <CAGzF1udYc3NGUbMKj1OqYC2muQb2h6vm87Q8Nszr-vvnNSO4Tw@mail.gmail.com>
References: <519F56CE.6000606@relativita.com>
	<68CB3562-2A48-459F-9FFD-C879E39070CA@inria.fr>
	<519F6420.70609@relativita.com>
	<CAKVJ-_6q8tiExyCEcno5o+gKQdZCJwDUGoA142PctHLuqZ-Q1g@mail.gmail.com>
	<CAF6FJiv39DFM8sJUOcsTmHundUPf3mcd2r43w7f==7gAAOXodQ@mail.gmail.com>
	<CAKVJ-_5cuCJvZZs6S=_pJ-HoGEWQM+ARU6mC50WDOR91fE4JpA@mail.gmail.com>
	<CAGzF1udYc3NGUbMKj1OqYC2muQb2h6vm87Q8Nszr-vvnNSO4Tw@mail.gmail.com>
Message-ID: <CAKVJ-_4utBN=9+ysqJm8Gf1r14OfCCf9+EoXzrLNfF-J5VZXHw@mail.gmail.com>

On Fri, May 24, 2013 at 2:47 PM, Warren Weckesser
<warren.weckesser at gmail.com> wrote:
>
>Peter wrote:
>> -------------------------------------------
>> Successes
>> -------------------------------------------
>>
>> 64 bit Linux:
>>
>> $ python2.6
>> Python 2.6.6 (r266:84292, Sep 11 2012, 08:34:23)
>> [GCC 4.4.6 20120305 (Red Hat 4.4.6-4)] on linux2
>> Type "help", "copyright", "credits" or "license" for more information.
>>>>> import sys;print("%x" % sys.maxsize, sys.maxsize > 2**32)
>> ('7fffffffffffffff', True)
>>>>> import numpy as np
>>>>> print(np.random.multivariate_normal(mean=np.zeros(2), cov=np.eye(2),
>>>>> size=1))
>> [[-0.27469218 -2.12911784]]
>>>>> print(np.random.multivariate_normal(mean=np.zeros(2), cov=np.eye(2),
>>>>> size=np.int64(1)))
>> [[ 0.02609307  0.32485211]]
>>>>> np.__version__
>> '1.6.2'
>>>>> quit()
>>
>
> Peter: wow, that's a lot of tests!

I try to keep a broad range on hand for testing my own code.

> Two more data points:
> On Ubuntu 12.04, using 64 bit builds of Python 2.7.4 (from Anaconda
> 1.5.0), and numpy built from source: numpy 1.6.1 gives the error, but
> 1.6.2 does not.
>
> Warren

That's interesting - and matches my only success being with NumPy 1.6.2

This suggests this was broken to up 1.6.1, but fixed in the 1.6.2 branch
and not the 1.7 branch. Have anyone tried the current master branch?

Peter


From josef.pktd at gmail.com  Fri May 24 10:00:47 2013
From: josef.pktd at gmail.com (josef.pktd at gmail.com)
Date: Fri, 24 May 2013 10:00:47 -0400
Subject: [Numpy-discussion] multivariate_normal issue with 'size'
	argument
In-Reply-To: <CAGzF1udYc3NGUbMKj1OqYC2muQb2h6vm87Q8Nszr-vvnNSO4Tw@mail.gmail.com>
References: <519F56CE.6000606@relativita.com>
	<68CB3562-2A48-459F-9FFD-C879E39070CA@inria.fr>
	<519F6420.70609@relativita.com>
	<CAKVJ-_6q8tiExyCEcno5o+gKQdZCJwDUGoA142PctHLuqZ-Q1g@mail.gmail.com>
	<CAF6FJiv39DFM8sJUOcsTmHundUPf3mcd2r43w7f==7gAAOXodQ@mail.gmail.com>
	<CAKVJ-_5cuCJvZZs6S=_pJ-HoGEWQM+ARU6mC50WDOR91fE4JpA@mail.gmail.com>
	<CAGzF1udYc3NGUbMKj1OqYC2muQb2h6vm87Q8Nszr-vvnNSO4Tw@mail.gmail.com>
Message-ID: <CAMMTP+B1G6qiuHzxf101b_GSaDGpvPrbA197YjXU_LHtnDRFrA@mail.gmail.com>

On Fri, May 24, 2013 at 9:47 AM, Warren Weckesser
<warren.weckesser at gmail.com> wrote:
> On 5/24/13, Peter Cock <p.j.a.cock at googlemail.com> wrote:
>> On Fri, May 24, 2013 at 2:15 PM, Robert Kern <robert.kern at gmail.com> wrote:
>>> On Fri, May 24, 2013 at 9:12 AM, Peter Cock <p.j.a.cock at googlemail.com>
>>> wrote:
>>>> On Fri, May 24, 2013 at 1:59 PM, Emanuele Olivetti
>>>> <emanuele at relativita.com> wrote:
>>>>> Interesting. Anyone able to reproduce what I observe?
>>>>>
>>>>> Emanuele
>>>>
>>>>
>>>> Yes, I can reproduce this IndexError under Mac OS X:
>>>>
>>>> $ which python2.7
>>>> /usr/bin/python2.7
>>>> $ python2.7
>>>> Python 2.7.2 (default, Oct 11 2012, 20:14:37)
>>>> [GCC 4.2.1 Compatible Apple Clang 4.0 (tags/Apple/clang-418.0.60)] on
>>>> darwin
>>>> Type "help", "copyright", "credits" or "license" for more information.
>>>
>>> Can everyone please report whether they have a 32-bit build of Python
>>> or a 64-bit build? That's probably the most relevant factor.
>>
>> It seems to affect all of 32 bit Windows XP, 64 bit Mac, 32 bit Linux,
>> and 64 bit Linux
>> for some versions of NumPy...  Thus far the only non-failure I've seen
>> is 64 bit Linux,
>> Python 2.6.6 with NumPy 1.6.2 (other Python/NumPy installs on this
>> machine do fail).
>>
>> Its a bit strange - I don't see any obvious pattern.
>>
>> Peter
>>
>> -------------------------------------------
>>
>> Failures:
>>
>> My Python installs on this Mac all seem to be 64bit (and fail),
>>
>> $ python3.3
>> Python 3.3.1 (default, Apr  8 2013, 17:54:08)
>> [GCC 4.2.1 Compatible Apple Clang 4.0 ((tags/Apple/clang-421.0.57))] on
>> darwin
>> Type "help", "copyright", "credits" or "license" for more information.
>>>>> import sys;print("%x" % sys.maxsize, sys.maxsize > 2**32)
>> 7fffffffffffffff True
>>>>> import numpy as np
>>>>> print(np.random.multivariate_normal(mean=np.zeros(2), cov=np.eye(2),
>>>>> size=1))
>> [[ 1.80932387  0.85894164]]
>>>>> print(np.random.multivariate_normal(mean=np.zeros(2), cov=np.eye(2),
>>>>> size=np.int64(1)))
>> Traceback (most recent call last):
>>   File "<stdin>", line 1, in <module>
>>   File "mtrand.pyx", line 4161, in
>> mtrand.RandomState.multivariate_normal
>> (numpy/random/mtrand/mtrand.c:19140)
>> IndexError: invalid index to scalar variable.
>>>>> np.__version__
>> '1.7.1'
>>>>> quit()
>>
>> This also affects NumPy 1.5 so this isn't a recent regression:
>>
>> $ python3.2
>> Python 3.2 (r32:88445, Feb 28 2011, 17:04:33)
>> [GCC 4.2.1 (Apple Inc. build 5664)] on darwin
>> Type "help", "copyright", "credits" or "license" for more information.
>>>>> import sys;print("%x" % sys.maxsize, sys.maxsize > 2**32)
>> 7fffffffffffffff True
>>>>> import numpy as np
>>>>> print(np.random.multivariate_normal(mean=np.zeros(2), cov=np.eye(2),
>>>>> size=1))
>> [[ 1.11403341 -1.67856405]]
>>>>> print(np.random.multivariate_normal(mean=np.zeros(2), cov=np.eye(2),
>>>>> size=np.int64(1)))
>> Traceback (most recent call last):
>>   File "<stdin>", line 1, in <module>
>>   File "mtrand.pyx", line 3954, in
>> mtrand.RandomState.multivariate_normal
>> (numpy/random/mtrand/mtrand.c:17234)
>> IndexError: invalid index to scalar variable.
>>>>> np.__version__
>> '1.5.0'
>>
>> $ python3.1
>> Python 3.1.2 (r312:79147, Nov 15 2010, 16:28:52)
>> [GCC 4.2.1 (Apple Inc. build 5664)] on darwin
>> Type "help", "copyright", "credits" or "license" for more information.
>>>>> import sys;print("%x" % sys.maxsize, sys.maxsize > 2**32)
>> 7fffffffffffffff True
>>>>> import numpy as np
>>>>> print(np.random.multivariate_normal(mean=np.zeros(2), cov=np.eye(2),
>>>>> size=1))
>> [[ 0.3834108  -0.31124203]]
>>>>> print(np.random.multivariate_normal(mean=np.zeros(2), cov=np.eye(2),
>>>>> size=np.int64(1)))
>> Traceback (most recent call last):
>>   File "<stdin>", line 1, in <module>
>>   File "mtrand.pyx", line 3954, in
>> mtrand.RandomState.multivariate_normal
>> (numpy/random/mtrand/mtrand.c:17234)
>> IndexError: invalid index to scalar variable.
>>>>> np.__version__
>> '1.5.0'
>>>>> quit()
>>
>> And on my 32 bit Windows XP box,
>>
>> Python 2.7 (r27:82525, Jul  4 2010, 09:01:59) [MSC v.1500 32 bit
>> (Intel)] on win32
>> Type "help", "copyright", "credits" or "license" for more information.
>>>>> import sys;print("%x" % sys.maxsize, sys.maxsize > 2**32)
>> ('7fffffff', False)
>>>>> import numpy as np
>>>>> print(np.random.multivariate_normal(mean=np.zeros(2), cov=np.eye(2),
>>>>> size=1))
>> [[-0.35072523 -0.58046885]]
>>>>> print(np.random.multivariate_normal(mean=np.zeros(2), cov=np.eye(2),
>>>>> size=np.int64(1)))
>> Traceback (most recent call last):
>>   File "<stdin>", line 1, in <module>
>>   File "mtrand.pyx", line 3954, in
>> mtrand.RandomState.multivariate_normal
>> (numpy\random\mtrand\mtrand.c:17234)
>> IndexError: invalid index to scalar variable.
>>>>> np.__version__
>> '1.5.0'
>>>>>
>>
>> Python 3.3.0 (v3.3.0:bd8afb90ebf2, Sep 29 2012, 10:55:48) [MSC v.1600
>> 32 bit (Intel)] on win32
>> Type "help", "copyright", "credits" or "license" for more information.
>>>>> import numpy as np
>>>>> print(np.random.multivariate_normal(mean=np.zeros(2), cov=np.eye(2),
>>>>> size=1))
>> [[-0.00453374  0.2210342 ]]
>>>>> print(np.random.multivariate_normal(mean=np.zeros(2), cov=np.eye(2),
>>>>> size=np.int64(1)))
>> Traceback (most recent call last):
>>   File "<stdin>", line 1, in <module>
>>   File "mtrand.pyx", line 4142, in
>> mtrand.RandomState.multivariate_normal
>> (numpy\random\mtrand\mtrand.c:19128)
>> IndexError: invalid index to scalar variable.
>>>>>
>>>>> np.__version__
>> '1.7.0rc2'
>>
>> Here's a couple of runs from an old 32 bit Linux machine which also
>> shows the problem:
>>
>> $ python2.7
>> Python 2.7 (r27:82500, Nov 12 2010, 14:19:08)
>> [GCC 4.1.2 20070626 (Red Hat 4.1.2-13)] on linux2
>> Type "help", "copyright", "credits" or "license" for more information.
>>>>> import sys;print("%x" % sys.maxsize, sys.maxsize > 2**32)
>> ('7fffffff', False)
>>>>> import numpy as np
>>>>> print(np.random.multivariate_normal(mean=np.zeros(2), cov=np.eye(2),
>>>>> size=1))
>> [[ 0.40887932 -0.09872295]]
>>>>> print(np.random.multivariate_normal(mean=np.zeros(2), cov=np.eye(2),
>>>>> size=np.int64(1)))
>> Traceback (most recent call last):
>>   File "<stdin>", line 1, in <module>
>>   File "mtrand.pyx", line 3952, in
>> mtrand.RandomState.multivariate_normal
>> (numpy/random/mtrand/mtrand.c:17027)
>> IndexError: invalid index to scalar variable.
>>>>> np.__version__
>> '1.4.1'
>>>>> quit()
>>
>> $ python3.3
>> Python 3.3.0 (default, Oct 31 2012, 13:19:08)
>> [GCC 4.1.2 20070626 (Red Hat 4.1.2-13)] on linux
>> Type "help", "copyright", "credits" or "license" for more information.
>>>>> import sys;print("%x" % sys.maxsize, sys.maxsize > 2**32)
>> 7fffffff False
>>>>> import numpy as np
>>>>> print(np.random.multivariate_normal(mean=np.zeros(2), cov=np.eye(2),
>>>>> size=1))
>> [[-0.16796654 -1.71312415]]
>>>>> print(np.random.multivariate_normal(mean=np.zeros(2), cov=np.eye(2),
>>>>> size=np.int64(1)))
>> Traceback (most recent call last):
>>   File "<stdin>", line 1, in <module>
>>   File "mtrand.pyx", line 4127, in
>> mtrand.RandomState.multivariate_normal
>> (numpy/random/mtrand/mtrand.c:18657)
>> IndexError: invalid index to scalar variable.
>>>>> np.__version__
>> '1.7.0b2'
>>>>> quit()
>>
>> And a more up to date 64 bit Linux machine where this fails on
>> some versions of NumPy:
>>
>> $ python3.3
>> Python 3.3.1 (default, Apr  8 2013, 14:38:06)
>> [GCC 4.4.6 20120305 (Red Hat 4.4.6-4)] on linux
>> Type "help", "copyright", "credits" or "license" for more information.
>>>>> import sys;print("%x" % sys.maxsize, sys.maxsize > 2**32)
>> 7fffffffffffffff True
>>>>> import numpy as np
>>>>> print(np.random.multivariate_normal(mean=np.zeros(2), cov=np.eye(2),
>>>>> size=1))
>> [[-0.18862683 -0.92177665]]
>>>>> print(np.random.multivariate_normal(mean=np.zeros(2), cov=np.eye(2),
>>>>> size=np.int64(1)))
>> Traceback (most recent call last):
>>   File "<stdin>", line 1, in <module>
>>   File "mtrand.pyx", line 4127, in
>> mtrand.RandomState.multivariate_normal
>> (numpy/random/mtrand/mtrand.c:18657)
>> IndexError: invalid index to scalar variable.
>>>>> np.__version__
>> '1.7.0b2'
>>>>> quit()
>>
>> -------------------------------------------
>> Successes
>> -------------------------------------------
>>
>> 64 bit Linux:
>>
>> $ python2.6
>> Python 2.6.6 (r266:84292, Sep 11 2012, 08:34:23)
>> [GCC 4.4.6 20120305 (Red Hat 4.4.6-4)] on linux2
>> Type "help", "copyright", "credits" or "license" for more information.
>>>>> import sys;print("%x" % sys.maxsize, sys.maxsize > 2**32)
>> ('7fffffffffffffff', True)
>>>>> import numpy as np
>>>>> print(np.random.multivariate_normal(mean=np.zeros(2), cov=np.eye(2),
>>>>> size=1))
>> [[-0.27469218 -2.12911784]]
>>>>> print(np.random.multivariate_normal(mean=np.zeros(2), cov=np.eye(2),
>>>>> size=np.int64(1)))
>> [[ 0.02609307  0.32485211]]
>>>>> np.__version__
>> '1.6.2'
>>>>> quit()
>>
>>
>
>
> Peter: wow, that's a lot of tests!
>
> Two more data points:
> On Ubuntu 12.04, using 64 bit builds of Python 2.7.4 (from Anaconda
> 1.5.0), and numpy built from source: numpy 1.6.1 gives the error, but
> 1.6.2 does not.

I also get them on Windows 64 (gohlke binaries)
python 3.2 with numpy 1.6.1
python 3.3 with numpy '1.7.0rc1'

Josef
>
> Warren
>
>
>>
>> Peter
>> _______________________________________________
>> NumPy-Discussion mailing list
>> NumPy-Discussion at scipy.org
>> http://mail.scipy.org/mailman/listinfo/numpy-discussion
>>
> _______________________________________________
> NumPy-Discussion mailing list
> NumPy-Discussion at scipy.org
> http://mail.scipy.org/mailman/listinfo/numpy-discussion


From warren.weckesser at gmail.com  Fri May 24 10:02:18 2013
From: warren.weckesser at gmail.com (Warren Weckesser)
Date: Fri, 24 May 2013 10:02:18 -0400
Subject: [Numpy-discussion] multivariate_normal issue with 'size'
	argument
In-Reply-To: <CAKVJ-_4utBN=9+ysqJm8Gf1r14OfCCf9+EoXzrLNfF-J5VZXHw@mail.gmail.com>
References: <519F56CE.6000606@relativita.com>
	<68CB3562-2A48-459F-9FFD-C879E39070CA@inria.fr>
	<519F6420.70609@relativita.com>
	<CAKVJ-_6q8tiExyCEcno5o+gKQdZCJwDUGoA142PctHLuqZ-Q1g@mail.gmail.com>
	<CAF6FJiv39DFM8sJUOcsTmHundUPf3mcd2r43w7f==7gAAOXodQ@mail.gmail.com>
	<CAKVJ-_5cuCJvZZs6S=_pJ-HoGEWQM+ARU6mC50WDOR91fE4JpA@mail.gmail.com>
	<CAGzF1udYc3NGUbMKj1OqYC2muQb2h6vm87Q8Nszr-vvnNSO4Tw@mail.gmail.com>
	<CAKVJ-_4utBN=9+ysqJm8Gf1r14OfCCf9+EoXzrLNfF-J5VZXHw@mail.gmail.com>
Message-ID: <CAGzF1ucwJ4Uic0TNwAXLWK3SFRdohbgW2m4xick6Y_udTWYhqA@mail.gmail.com>

On 5/24/13, Peter Cock <p.j.a.cock at googlemail.com> wrote:
> On Fri, May 24, 2013 at 2:47 PM, Warren Weckesser
> <warren.weckesser at gmail.com> wrote:
>>
>>Peter wrote:
>>> -------------------------------------------
>>> Successes
>>> -------------------------------------------
>>>
>>> 64 bit Linux:
>>>
>>> $ python2.6
>>> Python 2.6.6 (r266:84292, Sep 11 2012, 08:34:23)
>>> [GCC 4.4.6 20120305 (Red Hat 4.4.6-4)] on linux2
>>> Type "help", "copyright", "credits" or "license" for more information.
>>>>>> import sys;print("%x" % sys.maxsize, sys.maxsize > 2**32)
>>> ('7fffffffffffffff', True)
>>>>>> import numpy as np
>>>>>> print(np.random.multivariate_normal(mean=np.zeros(2), cov=np.eye(2),
>>>>>> size=1))
>>> [[-0.27469218 -2.12911784]]
>>>>>> print(np.random.multivariate_normal(mean=np.zeros(2), cov=np.eye(2),
>>>>>> size=np.int64(1)))
>>> [[ 0.02609307  0.32485211]]
>>>>>> np.__version__
>>> '1.6.2'
>>>>>> quit()
>>>
>>
>> Peter: wow, that's a lot of tests!
>
> I try to keep a broad range on hand for testing my own code.
>
>> Two more data points:
>> On Ubuntu 12.04, using 64 bit builds of Python 2.7.4 (from Anaconda
>> 1.5.0), and numpy built from source: numpy 1.6.1 gives the error, but
>> 1.6.2 does not.
>>
>> Warren
>
> That's interesting - and matches my only success being with NumPy 1.6.2
>
> This suggests this was broken to up 1.6.1, but fixed in the 1.6.2 branch
> and not the 1.7 branch. Have anyone tried the current master branch?
>

Sorry, I should have repeated my earlier report about 1.7.1.  My current summary
(all using 64 bit python 2.7.4 from Anaconda 1.5):

numpy 1.6.1 (built from source) fails.
numpy 1.6.2 (built from source) succeeds.
numpy 1.7.1 (Anaconda package) succeeds.

Warren

> Peter
> _______________________________________________
> NumPy-Discussion mailing list
> NumPy-Discussion at scipy.org
> http://mail.scipy.org/mailman/listinfo/numpy-discussion
>


From rhattersley at gmail.com  Fri May 24 10:12:32 2013
From: rhattersley at gmail.com (Richard Hattersley)
Date: Fri, 24 May 2013 15:12:32 +0100
Subject: [Numpy-discussion] Parameterised dtypes
Message-ID: <CAP=RS9kCSm7343uFc1qYKT=HgHiO9sr9W8K-5gp_NFkM3G27Bw@mail.gmail.com>

Hi all,

I'm in the process of defining some new dtypes to handle non-physical
calendars (such as the 360-day calendar used in the climate modelling
world). This is all going fine[*] so far, but I'd like to know a little bit
more about how much is ultimately possible.

The PyArray_Descr members `metadata` and `c_metadata` allow run-time
parametrisation, but is it possible to hook into the dtype('...') parsing
mechanism to supply those parameters? Or is there some other dtype
mechanism for supplying parameters?

As an example, would it be possible to supply month lengths?
>>> a = np.zeros(n, dtype='my_date[34,33,31,30,30,29,29,30,31,32,34,35]')

Or is the intended use of parametrisation more like:
>>> weird = my_stuff.make_dtype([34,33,31,30,30,29,29,30,31,32,34,35])
>>> a = np.zeros(n, dtype=weird)

[*] The docs could do with updating, and the examples would benefit from
standardising (or at least explaining the significance of the differences).
I intend to post updates where possible.

Richard
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20130524/d11964df/attachment.html>

From warren.weckesser at gmail.com  Fri May 24 10:33:14 2013
From: warren.weckesser at gmail.com (Warren Weckesser)
Date: Fri, 24 May 2013 10:33:14 -0400
Subject: [Numpy-discussion] multivariate_normal issue with 'size'
	argument
In-Reply-To: <CAGzF1ucwJ4Uic0TNwAXLWK3SFRdohbgW2m4xick6Y_udTWYhqA@mail.gmail.com>
References: <519F56CE.6000606@relativita.com>
	<68CB3562-2A48-459F-9FFD-C879E39070CA@inria.fr>
	<519F6420.70609@relativita.com>
	<CAKVJ-_6q8tiExyCEcno5o+gKQdZCJwDUGoA142PctHLuqZ-Q1g@mail.gmail.com>
	<CAF6FJiv39DFM8sJUOcsTmHundUPf3mcd2r43w7f==7gAAOXodQ@mail.gmail.com>
	<CAKVJ-_5cuCJvZZs6S=_pJ-HoGEWQM+ARU6mC50WDOR91fE4JpA@mail.gmail.com>
	<CAGzF1udYc3NGUbMKj1OqYC2muQb2h6vm87Q8Nszr-vvnNSO4Tw@mail.gmail.com>
	<CAKVJ-_4utBN=9+ysqJm8Gf1r14OfCCf9+EoXzrLNfF-J5VZXHw@mail.gmail.com>
	<CAGzF1ucwJ4Uic0TNwAXLWK3SFRdohbgW2m4xick6Y_udTWYhqA@mail.gmail.com>
Message-ID: <CAGzF1uc6uE4YL=fEeo5KPoyFOu4V2jVRmsSxkCPtXDyPCgJHQA@mail.gmail.com>

On 5/24/13, Warren Weckesser <warren.weckesser at gmail.com> wrote:
> On 5/24/13, Peter Cock <p.j.a.cock at googlemail.com> wrote:
>> On Fri, May 24, 2013 at 2:47 PM, Warren Weckesser
>> <warren.weckesser at gmail.com> wrote:
>>>
>>>Peter wrote:
>>>> -------------------------------------------
>>>> Successes
>>>> -------------------------------------------
>>>>
>>>> 64 bit Linux:
>>>>
>>>> $ python2.6
>>>> Python 2.6.6 (r266:84292, Sep 11 2012, 08:34:23)
>>>> [GCC 4.4.6 20120305 (Red Hat 4.4.6-4)] on linux2
>>>> Type "help", "copyright", "credits" or "license" for more information.
>>>>>>> import sys;print("%x" % sys.maxsize, sys.maxsize > 2**32)
>>>> ('7fffffffffffffff', True)
>>>>>>> import numpy as np
>>>>>>> print(np.random.multivariate_normal(mean=np.zeros(2), cov=np.eye(2),
>>>>>>> size=1))
>>>> [[-0.27469218 -2.12911784]]
>>>>>>> print(np.random.multivariate_normal(mean=np.zeros(2), cov=np.eye(2),
>>>>>>> size=np.int64(1)))
>>>> [[ 0.02609307  0.32485211]]
>>>>>>> np.__version__
>>>> '1.6.2'
>>>>>>> quit()
>>>>
>>>
>>> Peter: wow, that's a lot of tests!
>>
>> I try to keep a broad range on hand for testing my own code.
>>
>>> Two more data points:
>>> On Ubuntu 12.04, using 64 bit builds of Python 2.7.4 (from Anaconda
>>> 1.5.0), and numpy built from source: numpy 1.6.1 gives the error, but
>>> 1.6.2 does not.
>>>
>>> Warren
>>
>> That's interesting - and matches my only success being with NumPy 1.6.2
>>
>> This suggests this was broken to up 1.6.1, but fixed in the 1.6.2 branch
>> and not the 1.7 branch. Have anyone tried the current master branch?
>>
>
> Sorry, I should have repeated my earlier report about 1.7.1.  My current
> summary
> (all using 64 bit python 2.7.4 from Anaconda 1.5):
>
> numpy 1.6.1 (built from source) fails.
> numpy 1.6.2 (built from source) succeeds.
> numpy 1.7.1 (Anaconda package) succeeds.
>


Latest summary (all on 64 bit Ubuntu 12.04, and all Numpy packages are
built from source)

64 bit Python 2.7.4 (from Anaconda 1.5.0):
numpy 1.6.1 fails.
numpy 1.6.2 succeeds.
numpy 1.7.0 succeeds.
numpy 1.7.1 succeeds.

64 bit Python 3.3.1 (built from source):
numpy 1.7.1 fails.


Warren


> Warren
>
>> Peter
>> _______________________________________________
>> NumPy-Discussion mailing list
>> NumPy-Discussion at scipy.org
>> http://mail.scipy.org/mailman/listinfo/numpy-discussion
>>
>


From p.j.a.cock at googlemail.com  Fri May 24 10:47:39 2013
From: p.j.a.cock at googlemail.com (Peter Cock)
Date: Fri, 24 May 2013 15:47:39 +0100
Subject: [Numpy-discussion] multivariate_normal issue with 'size'
	argument
In-Reply-To: <CAGzF1ucwJ4Uic0TNwAXLWK3SFRdohbgW2m4xick6Y_udTWYhqA@mail.gmail.com>
References: <519F56CE.6000606@relativita.com>
	<68CB3562-2A48-459F-9FFD-C879E39070CA@inria.fr>
	<519F6420.70609@relativita.com>
	<CAKVJ-_6q8tiExyCEcno5o+gKQdZCJwDUGoA142PctHLuqZ-Q1g@mail.gmail.com>
	<CAF6FJiv39DFM8sJUOcsTmHundUPf3mcd2r43w7f==7gAAOXodQ@mail.gmail.com>
	<CAKVJ-_5cuCJvZZs6S=_pJ-HoGEWQM+ARU6mC50WDOR91fE4JpA@mail.gmail.com>
	<CAGzF1udYc3NGUbMKj1OqYC2muQb2h6vm87Q8Nszr-vvnNSO4Tw@mail.gmail.com>
	<CAKVJ-_4utBN=9+ysqJm8Gf1r14OfCCf9+EoXzrLNfF-J5VZXHw@mail.gmail.com>
	<CAGzF1ucwJ4Uic0TNwAXLWK3SFRdohbgW2m4xick6Y_udTWYhqA@mail.gmail.com>
Message-ID: <CAKVJ-_6Poq3BbsvEpPaesfbhp05GFtG=1phN=-7fvszE7wsKCQ@mail.gmail.com>

On Fri, May 24, 2013 at 3:02 PM, Warren Weckesser
<warren.weckesser at gmail.com> wrote:
> On 5/24/13, Peter Cock <p.j.a.cock at googlemail.com> wrote:
>>Warren wrote:
>>> Two more data points:
>>> On Ubuntu 12.04, using 64 bit builds of Python 2.7.4 (from Anaconda
>>> 1.5.0), and numpy built from source: numpy 1.6.1 gives the error, but
>>> 1.6.2 does not.
>>>
>>> Warren
>>
>> That's interesting - and matches my only success being with NumPy 1.6.2
>>
>> This suggests this was broken to up 1.6.1, but fixed in the 1.6.2 branch
>> and not the 1.7 branch. Have anyone tried the current master branch?
>>
>
> Sorry, I should have repeated my earlier report about 1.7.1.  My current summary
> (all using 64 bit python 2.7.4 from Anaconda 1.5):
>
> numpy 1.6.1 (built from source) fails.
> numpy 1.6.2 (built from source) succeeds.
> numpy 1.7.1 (Anaconda package) succeeds.
>
> Warren

Was this the same numpy 1.7.1 you used earlier, or a different setup?

64 bit Linux machine, Python 2.7.4 compiled from source (recently),
numpy freshly compiled from source today, making sure to remove
the old numpy installation fully between installs - note run in this order,
but I have tested some of these multiple times:

numpy 1.4.1 (built from source) succeeds.
numpy 1.5.1 (built from source) succeeds.
numpy 1.6.0 (built from source) fails.
numpy 1.6.1 (built from source) fails.
numpy 1.6.2 (built from source) succeeds.
numpy 1.7.0b2 (built from source) succeeds.
numpy 1.7.0 (built from source) succeeds.
numpy 1.7.1 (built from source) succeeds.
numpy 1.8.0.dev-e11cd9b (built from source) succeeds.

That all looks nice and neat, but according to my notes (and the
earlier email) the old numpy 1.4.1 on this machine was failing.
(I've erased that install now, but it was likely built with a
older Python 2.7.x and/or slightly older gcc).

Peter


From warren.weckesser at gmail.com  Fri May 24 11:56:28 2013
From: warren.weckesser at gmail.com (Warren Weckesser)
Date: Fri, 24 May 2013 11:56:28 -0400
Subject: [Numpy-discussion] multivariate_normal issue with 'size'
	argument
In-Reply-To: <CAKVJ-_6Poq3BbsvEpPaesfbhp05GFtG=1phN=-7fvszE7wsKCQ@mail.gmail.com>
References: <519F56CE.6000606@relativita.com>
	<68CB3562-2A48-459F-9FFD-C879E39070CA@inria.fr>
	<519F6420.70609@relativita.com>
	<CAKVJ-_6q8tiExyCEcno5o+gKQdZCJwDUGoA142PctHLuqZ-Q1g@mail.gmail.com>
	<CAF6FJiv39DFM8sJUOcsTmHundUPf3mcd2r43w7f==7gAAOXodQ@mail.gmail.com>
	<CAKVJ-_5cuCJvZZs6S=_pJ-HoGEWQM+ARU6mC50WDOR91fE4JpA@mail.gmail.com>
	<CAGzF1udYc3NGUbMKj1OqYC2muQb2h6vm87Q8Nszr-vvnNSO4Tw@mail.gmail.com>
	<CAKVJ-_4utBN=9+ysqJm8Gf1r14OfCCf9+EoXzrLNfF-J5VZXHw@mail.gmail.com>
	<CAGzF1ucwJ4Uic0TNwAXLWK3SFRdohbgW2m4xick6Y_udTWYhqA@mail.gmail.com>
	<CAKVJ-_6Poq3BbsvEpPaesfbhp05GFtG=1phN=-7fvszE7wsKCQ@mail.gmail.com>
Message-ID: <CAGzF1udZT3ZOhcCcF9knueU-a0uagJQQKVH78fn2W7Z_32prtA@mail.gmail.com>

On 5/24/13, Peter Cock <p.j.a.cock at googlemail.com> wrote:
> On Fri, May 24, 2013 at 3:02 PM, Warren Weckesser
> <warren.weckesser at gmail.com> wrote:
>> On 5/24/13, Peter Cock <p.j.a.cock at googlemail.com> wrote:
>>>Warren wrote:
>>>> Two more data points:
>>>> On Ubuntu 12.04, using 64 bit builds of Python 2.7.4 (from Anaconda
>>>> 1.5.0), and numpy built from source: numpy 1.6.1 gives the error, but
>>>> 1.6.2 does not.
>>>>
>>>> Warren
>>>
>>> That's interesting - and matches my only success being with NumPy 1.6.2
>>>
>>> This suggests this was broken to up 1.6.1, but fixed in the 1.6.2 branch
>>> and not the 1.7 branch. Have anyone tried the current master branch?
>>>
>>
>> Sorry, I should have repeated my earlier report about 1.7.1.  My current
>> summary
>> (all using 64 bit python 2.7.4 from Anaconda 1.5):
>>
>> numpy 1.6.1 (built from source) fails.
>> numpy 1.6.2 (built from source) succeeds.
>> numpy 1.7.1 (Anaconda package) succeeds.
>>
>> Warren
>
> Was this the same numpy 1.7.1 you used earlier, or a different setup?
>

With python 2.7.4, numpy 1.7.1 from Anaconda and built from source succeeds.
But with python 3.3.1, numpy 1.7.1 built from source fails.

Warren


> 64 bit Linux machine, Python 2.7.4 compiled from source (recently),
> numpy freshly compiled from source today, making sure to remove
> the old numpy installation fully between installs - note run in this order,
> but I have tested some of these multiple times:
>
> numpy 1.4.1 (built from source) succeeds.
> numpy 1.5.1 (built from source) succeeds.
> numpy 1.6.0 (built from source) fails.
> numpy 1.6.1 (built from source) fails.
> numpy 1.6.2 (built from source) succeeds.
> numpy 1.7.0b2 (built from source) succeeds.
> numpy 1.7.0 (built from source) succeeds.
> numpy 1.7.1 (built from source) succeeds.
> numpy 1.8.0.dev-e11cd9b (built from source) succeeds.
>
> That all looks nice and neat, but according to my notes (and the
> earlier email) the old numpy 1.4.1 on this machine was failing.
> (I've erased that install now, but it was likely built with a
> older Python 2.7.x and/or slightly older gcc).
>
> Peter
> _______________________________________________
> NumPy-Discussion mailing list
> NumPy-Discussion at scipy.org
> http://mail.scipy.org/mailman/listinfo/numpy-discussion
>


From rhattersley at gmail.com  Fri May 24 13:29:25 2013
From: rhattersley at gmail.com (Richard Hattersley)
Date: Fri, 24 May 2013 18:29:25 +0100
Subject: [Numpy-discussion] Parameterised dtypes
In-Reply-To: <CAP=RS9kCSm7343uFc1qYKT=HgHiO9sr9W8K-5gp_NFkM3G27Bw@mail.gmail.com>
References: <CAP=RS9kCSm7343uFc1qYKT=HgHiO9sr9W8K-5gp_NFkM3G27Bw@mail.gmail.com>
Message-ID: <CAP=RS9=aErgVyKHDhAcTLV5iXsFdr4h4fysh6WFpWeiTvzoo+w@mail.gmail.com>

On 24 May 2013 15:12, Richard Hattersley <rhattersley at gmail.com> wrote:

> Or is the intended use of parametrisation more like:
> >>> weird = my_stuff.make_dtype([34,33,31,30,30,29,29,30,31,32,34,35])
> >>> a = np.zeros(n, dtype=weird)
>

Or to put it another way .... I have a working `make_dtype` function (which
could easily be extended to do dtype caching), but is that the right way to
go about things?

Richard
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20130524/a74f181e/attachment.html>

From cgohlke at uci.edu  Fri May 24 15:09:21 2013
From: cgohlke at uci.edu (Christoph Gohlke)
Date: Fri, 24 May 2013 12:09:21 -0700
Subject: [Numpy-discussion] multivariate_normal issue with 'size'
	argument
In-Reply-To: <CAMMTP+B1G6qiuHzxf101b_GSaDGpvPrbA197YjXU_LHtnDRFrA@mail.gmail.com>
References: <519F56CE.6000606@relativita.com>
	<68CB3562-2A48-459F-9FFD-C879E39070CA@inria.fr>
	<519F6420.70609@relativita.com>
	<CAKVJ-_6q8tiExyCEcno5o+gKQdZCJwDUGoA142PctHLuqZ-Q1g@mail.gmail.com>
	<CAF6FJiv39DFM8sJUOcsTmHundUPf3mcd2r43w7f==7gAAOXodQ@mail.gmail.com>
	<CAKVJ-_5cuCJvZZs6S=_pJ-HoGEWQM+ARU6mC50WDOR91fE4JpA@mail.gmail.com>
	<CAGzF1udYc3NGUbMKj1OqYC2muQb2h6vm87Q8Nszr-vvnNSO4Tw@mail.gmail.com>
	<CAMMTP+B1G6qiuHzxf101b_GSaDGpvPrbA197YjXU_LHtnDRFrA@mail.gmail.com>
Message-ID: <519FBAE1.2080107@uci.edu>

On 5/24/2013 7:00 AM, josef.pktd at gmail.com wrote:
> On Fri, May 24, 2013 at 9:47 AM, Warren Weckesser
> <warren.weckesser at gmail.com> wrote:
>> On 5/24/13, Peter Cock <p.j.a.cock at googlemail.com> wrote:
>>> On Fri, May 24, 2013 at 2:15 PM, Robert Kern <robert.kern at gmail.com> wrote:
>>>> On Fri, May 24, 2013 at 9:12 AM, Peter Cock <p.j.a.cock at googlemail.com>
>>>> wrote:
>>>>> On Fri, May 24, 2013 at 1:59 PM, Emanuele Olivetti
>>>>> <emanuele at relativita.com> wrote:
>>>>>> Interesting. Anyone able to reproduce what I observe?
>>>>>>
>>>>>> Emanuele
>>>>>
>>>>>
>>>>> Yes, I can reproduce this IndexError under Mac OS X:
>>>>>
>>>>> $ which python2.7
>>>>> /usr/bin/python2.7
>>>>> $ python2.7
>>>>> Python 2.7.2 (default, Oct 11 2012, 20:14:37)
>>>>> [GCC 4.2.1 Compatible Apple Clang 4.0 (tags/Apple/clang-418.0.60)] on
>>>>> darwin
>>>>> Type "help", "copyright", "credits" or "license" for more information.
>>>>
>>>> Can everyone please report whether they have a 32-bit build of Python
>>>> or a 64-bit build? That's probably the most relevant factor.
>>>
>>> It seems to affect all of 32 bit Windows XP, 64 bit Mac, 32 bit Linux,
>>> and 64 bit Linux
>>> for some versions of NumPy...  Thus far the only non-failure I've seen
>>> is 64 bit Linux,
>>> Python 2.6.6 with NumPy 1.6.2 (other Python/NumPy installs on this
>>> machine do fail).
>>>
>>> Its a bit strange - I don't see any obvious pattern.
>>>
>>> Peter
>>>
>>> -------------------------------------------
>>>
>>> Failures:
>>>
>>> My Python installs on this Mac all seem to be 64bit (and fail),
>>>
>>> $ python3.3
>>> Python 3.3.1 (default, Apr  8 2013, 17:54:08)
>>> [GCC 4.2.1 Compatible Apple Clang 4.0 ((tags/Apple/clang-421.0.57))] on
>>> darwin
>>> Type "help", "copyright", "credits" or "license" for more information.
>>>>>> import sys;print("%x" % sys.maxsize, sys.maxsize > 2**32)
>>> 7fffffffffffffff True
>>>>>> import numpy as np
>>>>>> print(np.random.multivariate_normal(mean=np.zeros(2), cov=np.eye(2),
>>>>>> size=1))
>>> [[ 1.80932387  0.85894164]]
>>>>>> print(np.random.multivariate_normal(mean=np.zeros(2), cov=np.eye(2),
>>>>>> size=np.int64(1)))
>>> Traceback (most recent call last):
>>>    File "<stdin>", line 1, in <module>
>>>    File "mtrand.pyx", line 4161, in
>>> mtrand.RandomState.multivariate_normal
>>> (numpy/random/mtrand/mtrand.c:19140)
>>> IndexError: invalid index to scalar variable.
>>>>>> np.__version__
>>> '1.7.1'
>>>>>> quit()
>>>
>>> This also affects NumPy 1.5 so this isn't a recent regression:
>>>
>>> $ python3.2
>>> Python 3.2 (r32:88445, Feb 28 2011, 17:04:33)
>>> [GCC 4.2.1 (Apple Inc. build 5664)] on darwin
>>> Type "help", "copyright", "credits" or "license" for more information.
>>>>>> import sys;print("%x" % sys.maxsize, sys.maxsize > 2**32)
>>> 7fffffffffffffff True
>>>>>> import numpy as np
>>>>>> print(np.random.multivariate_normal(mean=np.zeros(2), cov=np.eye(2),
>>>>>> size=1))
>>> [[ 1.11403341 -1.67856405]]
>>>>>> print(np.random.multivariate_normal(mean=np.zeros(2), cov=np.eye(2),
>>>>>> size=np.int64(1)))
>>> Traceback (most recent call last):
>>>    File "<stdin>", line 1, in <module>
>>>    File "mtrand.pyx", line 3954, in
>>> mtrand.RandomState.multivariate_normal
>>> (numpy/random/mtrand/mtrand.c:17234)
>>> IndexError: invalid index to scalar variable.
>>>>>> np.__version__
>>> '1.5.0'
>>>
>>> $ python3.1
>>> Python 3.1.2 (r312:79147, Nov 15 2010, 16:28:52)
>>> [GCC 4.2.1 (Apple Inc. build 5664)] on darwin
>>> Type "help", "copyright", "credits" or "license" for more information.
>>>>>> import sys;print("%x" % sys.maxsize, sys.maxsize > 2**32)
>>> 7fffffffffffffff True
>>>>>> import numpy as np
>>>>>> print(np.random.multivariate_normal(mean=np.zeros(2), cov=np.eye(2),
>>>>>> size=1))
>>> [[ 0.3834108  -0.31124203]]
>>>>>> print(np.random.multivariate_normal(mean=np.zeros(2), cov=np.eye(2),
>>>>>> size=np.int64(1)))
>>> Traceback (most recent call last):
>>>    File "<stdin>", line 1, in <module>
>>>    File "mtrand.pyx", line 3954, in
>>> mtrand.RandomState.multivariate_normal
>>> (numpy/random/mtrand/mtrand.c:17234)
>>> IndexError: invalid index to scalar variable.
>>>>>> np.__version__
>>> '1.5.0'
>>>>>> quit()
>>>
>>> And on my 32 bit Windows XP box,
>>>
>>> Python 2.7 (r27:82525, Jul  4 2010, 09:01:59) [MSC v.1500 32 bit
>>> (Intel)] on win32
>>> Type "help", "copyright", "credits" or "license" for more information.
>>>>>> import sys;print("%x" % sys.maxsize, sys.maxsize > 2**32)
>>> ('7fffffff', False)
>>>>>> import numpy as np
>>>>>> print(np.random.multivariate_normal(mean=np.zeros(2), cov=np.eye(2),
>>>>>> size=1))
>>> [[-0.35072523 -0.58046885]]
>>>>>> print(np.random.multivariate_normal(mean=np.zeros(2), cov=np.eye(2),
>>>>>> size=np.int64(1)))
>>> Traceback (most recent call last):
>>>    File "<stdin>", line 1, in <module>
>>>    File "mtrand.pyx", line 3954, in
>>> mtrand.RandomState.multivariate_normal
>>> (numpy\random\mtrand\mtrand.c:17234)
>>> IndexError: invalid index to scalar variable.
>>>>>> np.__version__
>>> '1.5.0'
>>>>>>
>>>
>>> Python 3.3.0 (v3.3.0:bd8afb90ebf2, Sep 29 2012, 10:55:48) [MSC v.1600
>>> 32 bit (Intel)] on win32
>>> Type "help", "copyright", "credits" or "license" for more information.
>>>>>> import numpy as np
>>>>>> print(np.random.multivariate_normal(mean=np.zeros(2), cov=np.eye(2),
>>>>>> size=1))
>>> [[-0.00453374  0.2210342 ]]
>>>>>> print(np.random.multivariate_normal(mean=np.zeros(2), cov=np.eye(2),
>>>>>> size=np.int64(1)))
>>> Traceback (most recent call last):
>>>    File "<stdin>", line 1, in <module>
>>>    File "mtrand.pyx", line 4142, in
>>> mtrand.RandomState.multivariate_normal
>>> (numpy\random\mtrand\mtrand.c:19128)
>>> IndexError: invalid index to scalar variable.
>>>>>>
>>>>>> np.__version__
>>> '1.7.0rc2'
>>>
>>> Here's a couple of runs from an old 32 bit Linux machine which also
>>> shows the problem:
>>>
>>> $ python2.7
>>> Python 2.7 (r27:82500, Nov 12 2010, 14:19:08)
>>> [GCC 4.1.2 20070626 (Red Hat 4.1.2-13)] on linux2
>>> Type "help", "copyright", "credits" or "license" for more information.
>>>>>> import sys;print("%x" % sys.maxsize, sys.maxsize > 2**32)
>>> ('7fffffff', False)
>>>>>> import numpy as np
>>>>>> print(np.random.multivariate_normal(mean=np.zeros(2), cov=np.eye(2),
>>>>>> size=1))
>>> [[ 0.40887932 -0.09872295]]
>>>>>> print(np.random.multivariate_normal(mean=np.zeros(2), cov=np.eye(2),
>>>>>> size=np.int64(1)))
>>> Traceback (most recent call last):
>>>    File "<stdin>", line 1, in <module>
>>>    File "mtrand.pyx", line 3952, in
>>> mtrand.RandomState.multivariate_normal
>>> (numpy/random/mtrand/mtrand.c:17027)
>>> IndexError: invalid index to scalar variable.
>>>>>> np.__version__
>>> '1.4.1'
>>>>>> quit()
>>>
>>> $ python3.3
>>> Python 3.3.0 (default, Oct 31 2012, 13:19:08)
>>> [GCC 4.1.2 20070626 (Red Hat 4.1.2-13)] on linux
>>> Type "help", "copyright", "credits" or "license" for more information.
>>>>>> import sys;print("%x" % sys.maxsize, sys.maxsize > 2**32)
>>> 7fffffff False
>>>>>> import numpy as np
>>>>>> print(np.random.multivariate_normal(mean=np.zeros(2), cov=np.eye(2),
>>>>>> size=1))
>>> [[-0.16796654 -1.71312415]]
>>>>>> print(np.random.multivariate_normal(mean=np.zeros(2), cov=np.eye(2),
>>>>>> size=np.int64(1)))
>>> Traceback (most recent call last):
>>>    File "<stdin>", line 1, in <module>
>>>    File "mtrand.pyx", line 4127, in
>>> mtrand.RandomState.multivariate_normal
>>> (numpy/random/mtrand/mtrand.c:18657)
>>> IndexError: invalid index to scalar variable.
>>>>>> np.__version__
>>> '1.7.0b2'
>>>>>> quit()
>>>
>>> And a more up to date 64 bit Linux machine where this fails on
>>> some versions of NumPy:
>>>
>>> $ python3.3
>>> Python 3.3.1 (default, Apr  8 2013, 14:38:06)
>>> [GCC 4.4.6 20120305 (Red Hat 4.4.6-4)] on linux
>>> Type "help", "copyright", "credits" or "license" for more information.
>>>>>> import sys;print("%x" % sys.maxsize, sys.maxsize > 2**32)
>>> 7fffffffffffffff True
>>>>>> import numpy as np
>>>>>> print(np.random.multivariate_normal(mean=np.zeros(2), cov=np.eye(2),
>>>>>> size=1))
>>> [[-0.18862683 -0.92177665]]
>>>>>> print(np.random.multivariate_normal(mean=np.zeros(2), cov=np.eye(2),
>>>>>> size=np.int64(1)))
>>> Traceback (most recent call last):
>>>    File "<stdin>", line 1, in <module>
>>>    File "mtrand.pyx", line 4127, in
>>> mtrand.RandomState.multivariate_normal
>>> (numpy/random/mtrand/mtrand.c:18657)
>>> IndexError: invalid index to scalar variable.
>>>>>> np.__version__
>>> '1.7.0b2'
>>>>>> quit()
>>>
>>> -------------------------------------------
>>> Successes
>>> -------------------------------------------
>>>
>>> 64 bit Linux:
>>>
>>> $ python2.6
>>> Python 2.6.6 (r266:84292, Sep 11 2012, 08:34:23)
>>> [GCC 4.4.6 20120305 (Red Hat 4.4.6-4)] on linux2
>>> Type "help", "copyright", "credits" or "license" for more information.
>>>>>> import sys;print("%x" % sys.maxsize, sys.maxsize > 2**32)
>>> ('7fffffffffffffff', True)
>>>>>> import numpy as np
>>>>>> print(np.random.multivariate_normal(mean=np.zeros(2), cov=np.eye(2),
>>>>>> size=1))
>>> [[-0.27469218 -2.12911784]]
>>>>>> print(np.random.multivariate_normal(mean=np.zeros(2), cov=np.eye(2),
>>>>>> size=np.int64(1)))
>>> [[ 0.02609307  0.32485211]]
>>>>>> np.__version__
>>> '1.6.2'
>>>>>> quit()
>>>
>>>
>>
>>
>> Peter: wow, that's a lot of tests!
>>
>> Two more data points:
>> On Ubuntu 12.04, using 64 bit builds of Python 2.7.4 (from Anaconda
>> 1.5.0), and numpy built from source: numpy 1.6.1 gives the error, but
>> 1.6.2 does not.
>
> I also get them on Windows 64 (gohlke binaries)
> python 3.2 with numpy 1.6.1
> python 3.3 with numpy '1.7.0rc1'
>
> Josef
>>
>> Warren
>>
>>
>>>
>>> Peter


This patch works for me:

diff --git a/numpy/random/mtrand/mtrand.pyx b/numpy/random/mtrand/mtrand.pyx
index b0de560..233ff52 100644
--- a/numpy/random/mtrand/mtrand.pyx
+++ b/numpy/random/mtrand/mtrand.pyx
@@ -4156,7 +4156,7 @@ cdef class RandomState:
          if mean.shape[0] != cov.shape[0]:
                 raise ValueError("mean and cov must have same length")
          # Compute shape of output
-        if isinstance(shape, int):
+        if isinstance(shape, (int, long, np.integer)):
              shape = [shape]
          final_shape = list(shape[:])
          final_shape.append(mean.shape[0])

--
Christoph


From sudheer.joseph at yahoo.com  Sat May 25 08:19:58 2013
From: sudheer.joseph at yahoo.com (Sudheer Joseph)
Date: Sat, 25 May 2013 20:19:58 +0800 (SGT)
Subject: [Numpy-discussion] array manupulation
Message-ID: <1369484398.86666.YahooMailNeo@web193406.mail.sg3.yahoo.com>

Dear experts,
I can do below operation in matlab but I wanted to do the same in python numpy array.
I tried np.vstak ?and np.concatenate but not getting the desired results. please help. I wanted to add a row of zeros to the beginning of a numpy array. What is ?the available quick way.?

>> cc=[1:10;1:10]

cc =
? ? ?1 ? ? 2 ? ? 3 ? ? 4 ? ? 5 ? ? 6 ? ? 7 ? ? 8 ? ? 9 ? ?10
? ? ?1 ? ? 2 ? ? 3 ? ? 4 ? ? 5 ? ? 6 ? ? 7 ? ? 8 ? ? 9 ? ?10
>> cc=[cc(1,:)*0;cc]
cc =
? ? ?0 ? ? 0 ? ? 0 ? ? 0 ? ? 0 ? ? 0 ? ? 0 ? ? 0 ? ? 0 ? ? 0
? ? ?1 ? ? 2 ? ? 3 ? ? 4 ? ? 5 ? ? 6 ? ? 7 ? ? 8 ? ? 9 ? ?10
? ? ?1 ? ? 2 ? ? 3 ? ? 4 ? ? 5 ? ? 6 ? ? 7 ? ? 8 ? ? 9 ? ?10

wit best regards,
Sudheer?
***************************************************************
Sudheer Joseph 
Indian National Centre for Ocean Information Services
Ministry of Earth Sciences, Govt. of India
POST BOX NO: 21, IDA Jeedeemetla P.O.
Via Pragathi Nagar,Kukatpally, Hyderabad; Pin:5000 55
Tel:+91-40-23886047(O),Fax:+91-40-23895011(O),
Tel:+91-40-23044600(R),Tel:+91-40-9440832534(Mobile)
E-mail:sjo.India at gmail.com;sudheer.joseph at yahoo.com
Web- http://oppamthadathil.tripod.com
***************************************************************


From cournape at gmail.com  Sat May 25 10:23:25 2013
From: cournape at gmail.com (David Cournapeau)
Date: Sat, 25 May 2013 15:23:25 +0100
Subject: [Numpy-discussion] NumPy sprints at Scipy 2013,
	Austin: call for topics and hands to help
Message-ID: <CAGY4rcXRo+jjxxMBpLMhO2vBkFo5d=rL9ZsOTVLw5UeeZruuGQ@mail.gmail.com>

Hi there,

I agreed to help organising NumPy sprints during the scipy 2013
conference in Austin.

As some of you may know, St?fan and me will present a tutorial on
NumPy C code, so if we do our job correctly, we should have a few new
people ready to help out during the sprints.

It would be good to:
  - have some focus topics for improvements
  - know who is going to be there at the sprint to work on things
and/or help newcomers

Things I'd like to work on myself is looking into splitting things
from multiarray, think about a better internal API for dtype
registration/hooks (with the goal to remove any date dtype hardcoding
in both multiarray and ufunc machinery), but I am sure others have
more interesting ideas :)

thanks,
David


From pierre.raybaut at gmail.com  Sat May 25 10:31:11 2013
From: pierre.raybaut at gmail.com (Pierre Raybaut)
Date: Sat, 25 May 2013 16:31:11 +0200
Subject: [Numpy-discussion] ANN: New WinPython with Python 2.7.5 and 3.3.2
	(32/64bit)
Message-ID: <CAKegKuAqXxCAYcJmen4QKu2X+YGmYOC48rErN5XpFaUTLUy4SQ@mail.gmail.com>

Hi all,

I am pleased to announce that four new versions of WinPython have been
released yesterday with Python 2.7.5 and 3.3.2, 32 and 64 bits. Many
packages have been added or upgraded.

Special thanks to Christoph Gohlke for building most of the binary packages
bundled in WinPython.

WinPython is a free open-source portable distribution of Python
for Windows, designed for scientists.

It is a full-featured (see
http://code.google.com/p/winpython/wiki/PackageIndex)
Python-based scientific environment:
  * Designed for scientists (thanks to the integrated libraries
NumPy, SciPy, Matplotlib, guiqwt, etc.:
    * Regular *scientific users*: interactive data processing
and visualization using Python with Spyder
    * *Advanced scientific users and software developers*:
Python applications development with Spyder, version control with
Mercurial and other development tools (like gettext)
  * *Portable*: preconfigured, it should run out of the box on any machine
under Windows (without any installation requirements) and the folder
containing WinPython can be moved to any location (local, network or
removable drive)
  * *Flexible*: one can install (or should I write "use" as it's portable)
as many WinPython versions as necessary (like isolated and self-consistent
environments), even if those versions are running different versions of
Python (2.7, 3.x in the near future) or different architectures (32bit or
64bit) on the same machine
  * *Customizable*: using the integrated package manager (wppm,
as WinPython Package Manager), it's possible to install, uninstall
or upgrade Python packages (see
http://code.google.com/p/winpython/wiki/WPPM for
more details on supported package formats).

*WinPython is not an attempt to replace Python(x,y)*, this is
just something different (see
http://code.google.com/p/winpython/wiki/Roadmap): more flexible, easier to
maintain, movable and less invasive for the OS, but certainly less
user-friendly, with less packages/contents and without any integration to
Windows explorer [*].

[*] Actually there is an optional integration into Windows
explorer, providing the same features as the official Python installer
regarding file associations and context menu entry (this option may be
activated through the WinPython Control Panel), and adding shortcuts to
Windows Start menu.

Enjoy!

-Pierre
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20130525/e0c48b81/attachment.html>

From charlesr.harris at gmail.com  Sat May 25 11:19:33 2013
From: charlesr.harris at gmail.com (Charles R Harris)
Date: Sat, 25 May 2013 09:19:33 -0600
Subject: [Numpy-discussion] NumPy sprints at Scipy 2013,
 Austin: call for topics and hands to help
In-Reply-To: <CAGY4rcXRo+jjxxMBpLMhO2vBkFo5d=rL9ZsOTVLw5UeeZruuGQ@mail.gmail.com>
References: <CAGY4rcXRo+jjxxMBpLMhO2vBkFo5d=rL9ZsOTVLw5UeeZruuGQ@mail.gmail.com>
Message-ID: <CAB6mnxKJ5xw7rdwQx5gUSENHZtEakUZG59ZXEpjO0s4tX-LQJg@mail.gmail.com>

On Sat, May 25, 2013 at 8:23 AM, David Cournapeau <cournape at gmail.com>wrote:

> Hi there,
>
> I agreed to help organising NumPy sprints during the scipy 2013
> conference in Austin.
>
> As some of you may know, St?fan and me will present a tutorial on
> NumPy C code, so if we do our job correctly, we should have a few new
> people ready to help out during the sprints.
>
> It would be good to:
>   - have some focus topics for improvements
>   - know who is going to be there at the sprint to work on things
> and/or help newcomers
>
> Things I'd like to work on myself is looking into splitting things
> from multiarray, think about a better internal API for dtype
> registration/hooks (with the goal to remove any date dtype hardcoding
> in both multiarray and ufunc machinery), but I am sure others have
> more interesting ideas :)
>
>
I'd like to get a 1.8 beta out or at least get to the point where we can
make that leap. There is a lot of new stuff that needs to be tested, PR's
to go through, and I have a suspicion that a memory allocation error might
have crept in somewhere.

Chuck
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20130525/6d9de00a/attachment.html>

From cournape at gmail.com  Sat May 25 11:51:24 2013
From: cournape at gmail.com (David Cournapeau)
Date: Sat, 25 May 2013 16:51:24 +0100
Subject: [Numpy-discussion] NumPy sprints at Scipy 2013,
 Austin: call for topics and hands to help
In-Reply-To: <CAB6mnxKJ5xw7rdwQx5gUSENHZtEakUZG59ZXEpjO0s4tX-LQJg@mail.gmail.com>
References: <CAGY4rcXRo+jjxxMBpLMhO2vBkFo5d=rL9ZsOTVLw5UeeZruuGQ@mail.gmail.com>
	<CAB6mnxKJ5xw7rdwQx5gUSENHZtEakUZG59ZXEpjO0s4tX-LQJg@mail.gmail.com>
Message-ID: <CAGY4rcWDSNUr1ZqM1-LLUD3_Whbc2ZrNFyL7rPYGxpGNCYT-nQ@mail.gmail.com>

On Sat, May 25, 2013 at 4:19 PM, Charles R Harris
<charlesr.harris at gmail.com> wrote:
>
>
> On Sat, May 25, 2013 at 8:23 AM, David Cournapeau <cournape at gmail.com>
> wrote:
>>
>> Hi there,
>>
>> I agreed to help organising NumPy sprints during the scipy 2013
>> conference in Austin.
>>
>> As some of you may know, St?fan and me will present a tutorial on
>> NumPy C code, so if we do our job correctly, we should have a few new
>> people ready to help out during the sprints.
>>
>> It would be good to:
>>   - have some focus topics for improvements
>>   - know who is going to be there at the sprint to work on things
>> and/or help newcomers
>>
>> Things I'd like to work on myself is looking into splitting things
>> from multiarray, think about a better internal API for dtype
>> registration/hooks (with the goal to remove any date dtype hardcoding
>> in both multiarray and ufunc machinery), but I am sure others have
>> more interesting ideas :)
>>
>
> I'd like to get a 1.8 beta out or at least get to the point where we can
> make that leap.

Sure, I am fine doing this in a branch post 1.8.x, I am not in a hurry.

> There is a lot of new stuff that needs to be tested, PR's to
> go through, and I have a suspicion that a memory allocation error might have
> crept in somewhere.

Will you be there at the conference ?

David


From charlesr.harris at gmail.com  Sat May 25 12:37:09 2013
From: charlesr.harris at gmail.com (Charles R Harris)
Date: Sat, 25 May 2013 10:37:09 -0600
Subject: [Numpy-discussion] NumPy sprints at Scipy 2013,
 Austin: call for topics and hands to help
In-Reply-To: <CAGY4rcWDSNUr1ZqM1-LLUD3_Whbc2ZrNFyL7rPYGxpGNCYT-nQ@mail.gmail.com>
References: <CAGY4rcXRo+jjxxMBpLMhO2vBkFo5d=rL9ZsOTVLw5UeeZruuGQ@mail.gmail.com>
	<CAB6mnxKJ5xw7rdwQx5gUSENHZtEakUZG59ZXEpjO0s4tX-LQJg@mail.gmail.com>
	<CAGY4rcWDSNUr1ZqM1-LLUD3_Whbc2ZrNFyL7rPYGxpGNCYT-nQ@mail.gmail.com>
Message-ID: <CAB6mnxJFx5EKZ1ahS-aQzr6CjJw30hkmLUt42EQUsMV9sKsyyQ@mail.gmail.com>

On Sat, May 25, 2013 at 9:51 AM, David Cournapeau <cournape at gmail.com>wrote:

> On Sat, May 25, 2013 at 4:19 PM, Charles R Harris
> <charlesr.harris at gmail.com> wrote:
> >
> >
> > On Sat, May 25, 2013 at 8:23 AM, David Cournapeau <cournape at gmail.com>
> > wrote:
> >>
> >> Hi there,
> >>
> >> I agreed to help organising NumPy sprints during the scipy 2013
> >> conference in Austin.
> >>
> >> As some of you may know, St?fan and me will present a tutorial on
> >> NumPy C code, so if we do our job correctly, we should have a few new
> >> people ready to help out during the sprints.
> >>
> >> It would be good to:
> >>   - have some focus topics for improvements
> >>   - know who is going to be there at the sprint to work on things
> >> and/or help newcomers
> >>
> >> Things I'd like to work on myself is looking into splitting things
> >> from multiarray, think about a better internal API for dtype
> >> registration/hooks (with the goal to remove any date dtype hardcoding
> >> in both multiarray and ufunc machinery), but I am sure others have
> >> more interesting ideas :)
> >>
> >
> > I'd like to get a 1.8 beta out or at least get to the point where we can
> > make that leap.
>
> Sure, I am fine doing this in a branch post 1.8.x, I am not in a hurry.
>
> > There is a lot of new stuff that needs to be tested, PR's to
> > go through, and I have a suspicion that a memory allocation error might
> have
> > crept in somewhere.
>
> Will you be there at the conference ?
>

Yes. I'm not very good at sprinting though. I prefer to amble with a big
screen, nice keyboard, and a cup of coffee ;)

Chuck
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20130525/770616bb/attachment.html>

From bryanv at continuum.io  Sat May 25 14:24:06 2013
From: bryanv at continuum.io (Bryan Van de Ven)
Date: Sat, 25 May 2013 13:24:06 -0500
Subject: [Numpy-discussion] array manupulation
In-Reply-To: <1369484398.86666.YahooMailNeo@web193406.mail.sg3.yahoo.com>
References: <1369484398.86666.YahooMailNeo@web193406.mail.sg3.yahoo.com>
Message-ID: <B575BA4B-29A6-4B4E-8C23-519547918C48@continuum.io>

I'm not sure what you tried, but stack will do what you are seeking:

In [7]: cc = np.ones((2,10)) * np.arange(1, 11)

In [8]: cc
Out[8]: 
array([[  1.,   2.,   3.,   4.,   5.,   6.,   7.,   8.,   9.,  10.],
       [  1.,   2.,   3.,   4.,   5.,   6.,   7.,   8.,   9.,  10.]])

In [9]: np.vstack((np.zeros(10), cc))
Out[9]: 
array([[  0.,   0.,   0.,   0.,   0.,   0.,   0.,   0.,   0.,   0.],
       [  1.,   2.,   3.,   4.,   5.,   6.,   7.,   8.,   9.,  10.],
       [  1.,   2.,   3.,   4.,   5.,   6.,   7.,   8.,   9.,  10.]])

Bryan


On May 25, 2013, at 7:19 AM, Sudheer Joseph <sudheer.joseph at yahoo.com> wrote:

> Dear experts,
> I can do below operation in matlab but I wanted to do the same in python numpy array.
> I tried np.vstak  and np.concatenate but not getting the desired results. please help. I wanted to add a row of zeros to the beginning of a numpy array. What is  the available quick way. 
> 
>>> cc=[1:10;1:10]
> 
> cc =
>      1     2     3     4     5     6     7     8     9    10
>      1     2     3     4     5     6     7     8     9    10
>>> cc=[cc(1,:)*0;cc]
> cc =
>      0     0     0     0     0     0     0     0     0     0
>      1     2     3     4     5     6     7     8     9    10
>      1     2     3     4     5     6     7     8     9    10
> 
> wit best regards,
> Sudheer 
> ***************************************************************
> Sudheer Joseph 
> Indian National Centre for Ocean Information Services
> Ministry of Earth Sciences, Govt. of India
> POST BOX NO: 21, IDA Jeedeemetla P.O.
> Via Pragathi Nagar,Kukatpally, Hyderabad; Pin:5000 55
> Tel:+91-40-23886047(O),Fax:+91-40-23895011(O),
> Tel:+91-40-23044600(R),Tel:+91-40-9440832534(Mobile)
> E-mail:sjo.India at gmail.com;sudheer.joseph at yahoo.com
> Web- http://oppamthadathil.tripod.com
> ***************************************************************
> _______________________________________________
> NumPy-Discussion mailing list
> NumPy-Discussion at scipy.org
> http://mail.scipy.org/mailman/listinfo/numpy-discussion



From sudheer.joseph at yahoo.com  Sat May 25 20:08:28 2013
From: sudheer.joseph at yahoo.com (Sudheer Joseph)
Date: Sun, 26 May 2013 08:08:28 +0800 (SGT)
Subject: [Numpy-discussion] array manupulation
In-Reply-To: <B575BA4B-29A6-4B4E-8C23-519547918C48@continuum.io>
References: <1369484398.86666.YahooMailNeo@web193406.mail.sg3.yahoo.com>
	<B575BA4B-29A6-4B4E-8C23-519547918C48@continuum.io>
Message-ID: <1369526908.58295.YahooMailNeo@web193405.mail.sg3.yahoo.com>

Thank you Brian,
? ? ? ? ? ? ? ? ? ? ?I was missing the outer brackets.!!
with best regards,
Sudheer
?
***************************************************************
Sudheer Joseph 
Indian National Centre for Ocean Information Services
Ministry of Earth Sciences, Govt. of India
POST BOX NO: 21, IDA Jeedeemetla P.O.
Via Pragathi Nagar,Kukatpally, Hyderabad; Pin:5000 55
Tel:+91-40-23886047(O),Fax:+91-40-23895011(O),
Tel:+91-40-23044600(R),Tel:+91-40-9440832534(Mobile)
E-mail:sjo.India at gmail.com;sudheer.joseph at yahoo.com
Web- http://oppamthadathil.tripod.com
***************************************************************


----- Original Message -----
> From: Bryan Van de Ven <bryanv at continuum.io>
> To: Discussion of Numerical Python <numpy-discussion at scipy.org>
> Cc: 
> Sent: Saturday, 25 May 2013 11:54 PM
> Subject: Re: [Numpy-discussion] array manupulation
> 
> I'm not sure what you tried, but stack will do what you are seeking:
> 
> In [7]: cc = np.ones((2,10)) * np.arange(1, 11)
> 
> In [8]: cc
> Out[8]: 
> array([[? 1.,?  2.,?  3.,?  4.,?  5.,?  6.,?  7.,?  8.,?  9.,? 10.],
> ? ? ?  [? 1.,?  2.,?  3.,?  4.,?  5.,?  6.,?  7.,?  8.,?  9.,? 10.]])
> 
> In [9]: np.vstack((np.zeros(10), cc))
> Out[9]: 
> array([[? 0.,?  0.,?  0.,?  0.,?  0.,?  0.,?  0.,?  0.,?  0.,?  0.],
> ? ? ?  [? 1.,?  2.,?  3.,?  4.,?  5.,?  6.,?  7.,?  8.,?  9.,? 10.],
> ? ? ?  [? 1.,?  2.,?  3.,?  4.,?  5.,?  6.,?  7.,?  8.,?  9.,? 10.]])
> 
> Bryan
> 
> 
> On May 25, 2013, at 7:19 AM, Sudheer Joseph <sudheer.joseph at yahoo.com> 
> wrote:
> 
>>  Dear experts,
>>  I can do below operation in matlab but I wanted to do the same in python 
> numpy array.
>>  I tried np.vstak? and np.concatenate but not getting the desired results. 
> please help. I wanted to add a row of zeros to the beginning of a numpy array. 
> What is? the available quick way. 
>> 
>>>>  cc=[1:10;1:10]
>> 
>>  cc =
>> ? ? ? 1? ?  2? ?  3? ?  4? ?  5? ?  6? ?  7? ?  8? ?  9? ? 10
>> ? ? ? 1? ?  2? ?  3? ?  4? ?  5? ?  6? ?  7? ?  8? ?  9? ? 10
>>>>  cc=[cc(1,:)*0;cc]
>>  cc =
>> ? ? ? 0? ?  0? ?  0? ?  0? ?  0? ?  0? ?  0? ?  0? ?  0? ?  0
>> ? ? ? 1? ?  2? ?  3? ?  4? ?  5? ?  6? ?  7? ?  8? ?  9? ? 10
>> ? ? ? 1? ?  2? ?  3? ?  4? ?  5? ?  6? ?  7? ?  8? ?  9? ? 10
>> 
>>  wit best regards,
>>  Sudheer 
>>  ***************************************************************
>>  Sudheer Joseph 
>>  Indian National Centre for Ocean Information Services
>>  Ministry of Earth Sciences, Govt. of India
>>  POST BOX NO: 21, IDA Jeedeemetla P.O.
>>  Via Pragathi Nagar,Kukatpally, Hyderabad; Pin:5000 55
>>  Tel:+91-40-23886047(O),Fax:+91-40-23895011(O),
>>  Tel:+91-40-23044600(R),Tel:+91-40-9440832534(Mobile)
>>  E-mail:sjo.India at gmail.com;sudheer.joseph at yahoo.com
>>  Web- http://oppamthadathil.tripod.com
>>  ***************************************************************
>>  _______________________________________________
>>  NumPy-Discussion mailing list
>>  NumPy-Discussion at scipy.org
>>  http://mail.scipy.org/mailman/listinfo/numpy-discussion
> 
> _______________________________________________
> NumPy-Discussion mailing list
> NumPy-Discussion at scipy.org
> http://mail.scipy.org/mailman/listinfo/numpy-discussion
> 


From sudheer.joseph at yahoo.com  Sat May 25 20:55:35 2013
From: sudheer.joseph at yahoo.com (Sudheer Joseph)
Date: Sun, 26 May 2013 08:55:35 +0800 (SGT)
Subject: [Numpy-discussion] array manupulation
In-Reply-To: <B575BA4B-29A6-4B4E-8C23-519547918C48@continuum.io>
References: <1369484398.86666.YahooMailNeo@web193406.mail.sg3.yahoo.com>
	<B575BA4B-29A6-4B4E-8C23-519547918C48@continuum.io>
Message-ID: <1369529735.86699.YahooMailNeo@web193405.mail.sg3.yahoo.com>

Thank you Brian,
? ? ? ? ? ? ? ? ? ? ?Though the simple example worked for me, there appears to the some logic which I did not capture fully.

I have a 11x5 array named p from which I get a cumulative sum by doing below operations. I tried transposing the zero array to match the dimension( after first failed) but had no luck. It appears I am missing some change happening when I take the cumulative sum along newaxis. Can you please advice me on what I am missing here??

Out[84]:p.shape
Out[85]: (11, 5)
c=p.cumsum(axis=0)

In [86]: c.shape
Out[86]: (11, 5)
In [87]: xx=np.zeros(11)
In [88]: xx

Out[88]: array([ 0., ?0., ?0., ?0., ?0., ?0., ?0., ?0., ?0., ?0., ?0.])
np.vstack((xx,c))

---------------------------------------------------------------------------
ValueError ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?Traceback (most recent call last)
/media/SJOITB/SST_VAL/<ipython-input-89-b81a9a48bb54> in <module>()
----> 1 np.vstack((xx,c))

/usr/local/lib/python2.7/dist-packages/numpy-1.7.0-py2.7-linux-x86_64.egg/numpy/core/shape_base.pyc in vstack(tup)
? ? 224?
? ? 225 ? ? """
--> 226 ? ? return _nx.concatenate(map(atleast_2d,tup),0)
? ? 227?
? ? 228 def hstack(tup):

ValueError: all the input array dimensions except for the concatenation axis must match exactly


----- Original Message -----
> From: Bryan Van de Ven <bryanv at continuum.io>
> To: Discussion of Numerical Python <numpy-discussion at scipy.org>
> Cc: 
> Sent: Saturday, 25 May 2013 11:54 PM
> Subject: Re: [Numpy-discussion] array manupulation
> 
> I'm not sure what you tried, but stack will do what you are seeking:
> 
> In [7]: cc = np.ones((2,10)) * np.arange(1, 11)
> 
> In [8]: cc
> Out[8]: 
> array([[? 1.,?  2.,?  3.,?  4.,?  5.,?  6.,?  7.,?  8.,?  9.,? 10.],
> ? ? ?  [? 1.,?  2.,?  3.,?  4.,?  5.,?  6.,?  7.,?  8.,?  9.,? 10.]])
> 
> In [9]: np.vstack((np.zeros(10), cc))
> Out[9]: 
> array([[? 0.,?  0.,?  0.,?  0.,?  0.,?  0.,?  0.,?  0.,?  0.,?  0.],
> ? ? ?  [? 1.,?  2.,?  3.,?  4.,?  5.,?  6.,?  7.,?  8.,?  9.,? 10.],
> ? ? ?  [? 1.,?  2.,?  3.,?  4.,?  5.,?  6.,?  7.,?  8.,?  9.,? 10.]])
> 
> Bryan
> 
> 
> On May 25, 2013, at 7:19 AM, Sudheer Joseph <sudheer.joseph at yahoo.com> 
> wrote:
> 
>>  Dear experts,
>>  I can do below operation in matlab but I wanted to do the same in python 
> numpy array.
>>  I tried np.vstak? and np.concatenate but not getting the desired results. 
> please help. I wanted to add a row of zeros to the beginning of a numpy array. 
> What is? the available quick way. 
>> 
>>>>  cc=[1:10;1:10]
>> 
>>  cc =
>> ? ? ? 1? ?  2? ?  3? ?  4? ?  5? ?  6? ?  7? ?  8? ?  9? ? 10
>> ? ? ? 1? ?  2? ?  3? ?  4? ?  5? ?  6? ?  7? ?  8? ?  9? ? 10
>>>>  cc=[cc(1,:)*0;cc]
>>  cc =
>> ? ? ? 0? ?  0? ?  0? ?  0? ?  0? ?  0? ?  0? ?  0? ?  0? ?  0
>> ? ? ? 1? ?  2? ?  3? ?  4? ?  5? ?  6? ?  7? ?  8? ?  9? ? 10
>> ? ? ? 1? ?  2? ?  3? ?  4? ?  5? ?  6? ?  7? ?  8? ?  9? ? 10
>> 
>>  wit best regards,
>>  Sudheer 
>>  ***************************************************************
>>  Sudheer Joseph 
>>  Indian National Centre for Ocean Information Services
>>  Ministry of Earth Sciences, Govt. of India
>>  POST BOX NO: 21, IDA Jeedeemetla P.O.
>>  Via Pragathi Nagar,Kukatpally, Hyderabad; Pin:5000 55
>>  Tel:+91-40-23886047(O),Fax:+91-40-23895011(O),
>>  Tel:+91-40-23044600(R),Tel:+91-40-9440832534(Mobile)
>>  E-mail:sjo.India at gmail.com;sudheer.joseph at yahoo.com
>>  Web- http://oppamthadathil.tripod.com
>>  ***************************************************************
>>  _______________________________________________
>>  NumPy-Discussion mailing list
>>  NumPy-Discussion at scipy.org
>>  http://mail.scipy.org/mailman/listinfo/numpy-discussion
> 
> _______________________________________________
> NumPy-Discussion mailing list
> NumPy-Discussion at scipy.org
> http://mail.scipy.org/mailman/listinfo/numpy-discussion
> 


From ben.root at ou.edu  Sat May 25 20:59:00 2013
From: ben.root at ou.edu (Benjamin Root)
Date: Sat, 25 May 2013 20:59:00 -0400
Subject: [Numpy-discussion] NumPy sprints at Scipy 2013,
 Austin: call for topics and hands to help
In-Reply-To: <CAB6mnxJFx5EKZ1ahS-aQzr6CjJw30hkmLUt42EQUsMV9sKsyyQ@mail.gmail.com>
References: <CAGY4rcXRo+jjxxMBpLMhO2vBkFo5d=rL9ZsOTVLw5UeeZruuGQ@mail.gmail.com>
	<CAB6mnxKJ5xw7rdwQx5gUSENHZtEakUZG59ZXEpjO0s4tX-LQJg@mail.gmail.com>
	<CAGY4rcWDSNUr1ZqM1-LLUD3_Whbc2ZrNFyL7rPYGxpGNCYT-nQ@mail.gmail.com>
	<CAB6mnxJFx5EKZ1ahS-aQzr6CjJw30hkmLUt42EQUsMV9sKsyyQ@mail.gmail.com>
Message-ID: <CANNq6Fn+7Jtt3EXRc_TPWivRZc8jGTAFXUhKHPcpcj9wf9vEGg@mail.gmail.com>

On Sat, May 25, 2013 at 12:37 PM, Charles R Harris <
charlesr.harris at gmail.com> wrote:

>
>
> On Sat, May 25, 2013 at 9:51 AM, David Cournapeau <cournape at gmail.com>wrote:
>
>> On Sat, May 25, 2013 at 4:19 PM, Charles R Harris
>> <charlesr.harris at gmail.com> wrote:
>> >
>> >
>> > On Sat, May 25, 2013 at 8:23 AM, David Cournapeau <cournape at gmail.com>
>> > wrote:
>> >>
>> >> Hi there,
>> >>
>> >> I agreed to help organising NumPy sprints during the scipy 2013
>> >> conference in Austin.
>> >>
>> >> As some of you may know, St?fan and me will present a tutorial on
>> >> NumPy C code, so if we do our job correctly, we should have a few new
>> >> people ready to help out during the sprints.
>> >>
>> >> It would be good to:
>> >>   - have some focus topics for improvements
>> >>   - know who is going to be there at the sprint to work on things
>> >> and/or help newcomers
>> >>
>> >> Things I'd like to work on myself is looking into splitting things
>> >> from multiarray, think about a better internal API for dtype
>> >> registration/hooks (with the goal to remove any date dtype hardcoding
>> >> in both multiarray and ufunc machinery), but I am sure others have
>> >> more interesting ideas :)
>> >>
>> >
>> > I'd like to get a 1.8 beta out or at least get to the point where we can
>> > make that leap.
>>
>> Sure, I am fine doing this in a branch post 1.8.x, I am not in a hurry.
>>
>> > There is a lot of new stuff that needs to be tested, PR's to
>> > go through, and I have a suspicion that a memory allocation error might
>> have
>> > crept in somewhere.
>>
>> Will you be there at the conference ?
>>
>
> Yes. I'm not very good at sprinting though. I prefer to amble with a big
> screen, nice keyboard, and a cup of coffee ;)
>
> Chuck
>
>
Oh, I am sure we could get you set up with a projector screen and a nice
bluetooth keyboard... Now, I think you just hit on something with the
coffee.  I don't recall previous sprints having coffee available.

Ben Root
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20130525/7ee2852d/attachment.html>

From charlesr.harris at gmail.com  Sat May 25 22:27:31 2013
From: charlesr.harris at gmail.com (Charles R Harris)
Date: Sat, 25 May 2013 20:27:31 -0600
Subject: [Numpy-discussion] NumPy sprints at Scipy 2013,
 Austin: call for topics and hands to help
In-Reply-To: <CANNq6Fn+7Jtt3EXRc_TPWivRZc8jGTAFXUhKHPcpcj9wf9vEGg@mail.gmail.com>
References: <CAGY4rcXRo+jjxxMBpLMhO2vBkFo5d=rL9ZsOTVLw5UeeZruuGQ@mail.gmail.com>
	<CAB6mnxKJ5xw7rdwQx5gUSENHZtEakUZG59ZXEpjO0s4tX-LQJg@mail.gmail.com>
	<CAGY4rcWDSNUr1ZqM1-LLUD3_Whbc2ZrNFyL7rPYGxpGNCYT-nQ@mail.gmail.com>
	<CAB6mnxJFx5EKZ1ahS-aQzr6CjJw30hkmLUt42EQUsMV9sKsyyQ@mail.gmail.com>
	<CANNq6Fn+7Jtt3EXRc_TPWivRZc8jGTAFXUhKHPcpcj9wf9vEGg@mail.gmail.com>
Message-ID: <CAB6mnxKszgTpTqPkx2FxTKoNJeDmmTKGuT_4xTAMjBTKR7H6og@mail.gmail.com>

On Sat, May 25, 2013 at 6:59 PM, Benjamin Root <ben.root at ou.edu> wrote:

>
> On Sat, May 25, 2013 at 12:37 PM, Charles R Harris <
> charlesr.harris at gmail.com> wrote:
>
>>
>>
>> On Sat, May 25, 2013 at 9:51 AM, David Cournapeau <cournape at gmail.com>wrote:
>>
>>> On Sat, May 25, 2013 at 4:19 PM, Charles R Harris
>>> <charlesr.harris at gmail.com> wrote:
>>> >
>>> >
>>> > On Sat, May 25, 2013 at 8:23 AM, David Cournapeau <cournape at gmail.com>
>>> > wrote:
>>> >>
>>> >> Hi there,
>>> >>
>>> >> I agreed to help organising NumPy sprints during the scipy 2013
>>> >> conference in Austin.
>>> >>
>>> >> As some of you may know, St?fan and me will present a tutorial on
>>> >> NumPy C code, so if we do our job correctly, we should have a few new
>>> >> people ready to help out during the sprints.
>>> >>
>>> >> It would be good to:
>>> >>   - have some focus topics for improvements
>>> >>   - know who is going to be there at the sprint to work on things
>>> >> and/or help newcomers
>>> >>
>>> >> Things I'd like to work on myself is looking into splitting things
>>> >> from multiarray, think about a better internal API for dtype
>>> >> registration/hooks (with the goal to remove any date dtype hardcoding
>>> >> in both multiarray and ufunc machinery), but I am sure others have
>>> >> more interesting ideas :)
>>> >>
>>> >
>>> > I'd like to get a 1.8 beta out or at least get to the point where we
>>> can
>>> > make that leap.
>>>
>>> Sure, I am fine doing this in a branch post 1.8.x, I am not in a hurry.
>>>
>>> > There is a lot of new stuff that needs to be tested, PR's to
>>> > go through, and I have a suspicion that a memory allocation error
>>> might have
>>> > crept in somewhere.
>>>
>>> Will you be there at the conference ?
>>>
>>
>> Yes. I'm not very good at sprinting though. I prefer to amble with a big
>> screen, nice keyboard, and a cup of coffee ;)
>>
>> Chuck
>>
>>
> Oh, I am sure we could get you set up with a projector screen and a nice
> bluetooth keyboard... Now, I think you just hit on something with the
> coffee.  I don't recall previous sprints having coffee available.
>

Arrggghhhh, everyone would see my lousy code. I'd be exposed as the naked
emporer I am ;)

Chuck
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20130525/73305e34/attachment.html>

From sudheer.joseph at yahoo.com  Sat May 25 22:30:24 2013
From: sudheer.joseph at yahoo.com (Sudheer Joseph)
Date: Sun, 26 May 2013 10:30:24 +0800 (SGT)
Subject: [Numpy-discussion] array manupulation
In-Reply-To: <1369529735.86699.YahooMailNeo@web193405.mail.sg3.yahoo.com>
References: <1369484398.86666.YahooMailNeo@web193406.mail.sg3.yahoo.com>
	<B575BA4B-29A6-4B4E-8C23-519547918C48@continuum.io>
	<1369529735.86699.YahooMailNeo@web193405.mail.sg3.yahoo.com>
Message-ID: <1369535424.86470.YahooMailNeo@web193402.mail.sg3.yahoo.com>

Dear Brian,
? ? ? ? ? ? ? ? I even tried below but no luck!
In [138]: xx=np.zeros(11)
In [139]: xx
Out[139]: array([ 0., ?0., ?0., ?0., ?0., ?0., ?0., ?0., ?0., ?0., ?0.])

In [147]: xx.shape
Out[147]: (11,)
In [140]: xx=np.array(xx)[np.newaxis]
In [141]: xx.shape
Out[141]: (1, 11)
In [142]: xx=xx.T
In [143]: xx.shape
Out[143]: (11, 1)
In [144]: csum.shape
Out[144]: (11, 5)
In [145]: np.vstack((xx,csum))
---------------------------------------------------------------------------
ValueError ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?Traceback (most recent call last)
/media/SJOITB/SST_VAL/<ipython-input-145-2a0a60f68737> in <module>()
----> 1 np.vstack((xx,csum))

/usr/local/lib/python2.7/dist-packages/numpy-1.7.0-py2.7-linux-x86_64.egg/numpy/core/shape_base.pyc in vstack(tup)
? ? 224?
? ? 225 ? ? """
--> 226 ? ? return _nx.concatenate(map(atleast_2d,tup),0)
? ? 227?
? ? 228 def hstack(tup):

ValueError: all the input array dimensions except for the concatenation axis must match exactly

?

***************************************************************
Sudheer Joseph 
Indian National Centre for Ocean Information Services
Ministry of Earth Sciences, Govt. of India
POST BOX NO: 21, IDA Jeedeemetla P.O.
Via Pragathi Nagar,Kukatpally, Hyderabad; Pin:5000 55
Tel:+91-40-23886047(O),Fax:+91-40-23895011(O),
Tel:+91-40-23044600(R),Tel:+91-40-9440832534(Mobile)
E-mail:sjo.India at gmail.com;sudheer.joseph at yahoo.com
Web- http://oppamthadathil.tripod.com
***************************************************************


----- Original Message -----
> From: Sudheer Joseph <sudheer.joseph at yahoo.com>
> To: Discussion of Numerical Python <numpy-discussion at scipy.org>
> Cc: 
> Sent: Sunday, 26 May 2013 6:25 AM
> Subject: Re: [Numpy-discussion] array manupulation
> 
>T hank you Brian,
> ? ? ? ? ? ? ? ? ? ? ?Though the simple example worked for me, there appears to 
> the some logic which I did not capture fully.
> 
> I have a 11x5 array named p from which I get a cumulative sum by doing below 
> operations. I tried transposing the zero array to match the dimension( after 
> first failed) but had no luck. It appears I am missing some change happening 
> when I take the cumulative sum along newaxis. Can you please advice me on what I 
> am missing here??
> 
> Out[84]:p.shape
> Out[85]: (11, 5)
> c=p.cumsum(axis=0)
> 
> In [86]: c.shape
> Out[86]: (11, 5)
> In [87]: xx=np.zeros(11)
> In [88]: xx
> 
> Out[88]: array([ 0., ?0., ?0., ?0., ?0., ?0., ?0., ?0., ?0., ?0., ?0.])
> np.vstack((xx,c))
> 
> ---------------------------------------------------------------------------
> ValueError ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?Traceback (most recent call last)
> /media/SJOITB/SST_VAL/<ipython-input-89-b81a9a48bb54> in <module>()
> ----> 1 np.vstack((xx,c))
> 
> /usr/local/lib/python2.7/dist-packages/numpy-1.7.0-py2.7-linux-x86_64.egg/numpy/core/shape_base.pyc 
> in vstack(tup)
> ? ? 224?
> ? ? 225 ? ? """
> --> 226 ? ? return _nx.concatenate(map(atleast_2d,tup),0)
> ? ? 227?
> ? ? 228 def hstack(tup):
> 
> ValueError: all the input array dimensions except for the concatenation axis 
> must match exactly
> 
> 
> ----- Original Message -----
>>  From: Bryan Van de Ven <bryanv at continuum.io>
>>  To: Discussion of Numerical Python <numpy-discussion at scipy.org>
>>  Cc: 
>>  Sent: Saturday, 25 May 2013 11:54 PM
>>  Subject: Re: [Numpy-discussion] array manupulation
>> 
>>  I'm not sure what you tried, but stack will do what you are seeking:
>> 
>>  In [7]: cc = np.ones((2,10)) * np.arange(1, 11)
>> 
>>  In [8]: cc
>>  Out[8]: 
>>  array([[? 1.,?? 2.,?? 3.,?? 4.,?? 5.,?? 6.,?? 7.,?? 8.,?? 9.,? 10.],
>>  ? ? ?? [? 1.,?? 2.,?? 3.,?? 4.,?? 5.,?? 6.,?? 7.,?? 8.,?? 9.,? 10.]])
>> 
>>  In [9]: np.vstack((np.zeros(10), cc))
>>  Out[9]: 
>>  array([[? 0.,?? 0.,?? 0.,?? 0.,?? 0.,?? 0.,?? 0.,?? 0.,?? 0.,?? 0.],
>>  ? ? ?? [? 1.,?? 2.,?? 3.,?? 4.,?? 5.,?? 6.,?? 7.,?? 8.,?? 9.,? 10.],
>>  ? ? ?? [? 1.,?? 2.,?? 3.,?? 4.,?? 5.,?? 6.,?? 7.,?? 8.,?? 9.,? 10.]])
>> 
>>  Bryan
>> 
>> 
>>  On May 25, 2013, at 7:19 AM, Sudheer Joseph 
> <sudheer.joseph at yahoo.com> 
>>  wrote:
>> 
>>> ? Dear experts,
>>> ? I can do below operation in matlab but I wanted to do the same in 
> python 
>>  numpy array.
>>> ? I tried np.vstak? and np.concatenate but not getting the desired 
> results. 
>>  please help. I wanted to add a row of zeros to the beginning of a numpy 
> array. 
>>  What is? the available quick way. 
>>> 
>>>>> ? cc=[1:10;1:10]
>>> 
>>> ? cc =
>>>  ? ? ? 1? ?? 2? ?? 3? ?? 4? ?? 5? ?? 6? ?? 7? ?? 8? ?? 9? ? 10
>>>  ? ? ? 1? ?? 2? ?? 3? ?? 4? ?? 5? ?? 6? ?? 7? ?? 8? ?? 9? ? 10
>>>>> ? cc=[cc(1,:)*0;cc]
>>> ? cc =
>>>  ? ? ? 0? ?? 0? ?? 0? ?? 0? ?? 0? ?? 0? ?? 0? ?? 0? ?? 0? ?? 0
>>>  ? ? ? 1? ?? 2? ?? 3? ?? 4? ?? 5? ?? 6? ?? 7? ?? 8? ?? 9? ? 10
>>>  ? ? ? 1? ?? 2? ?? 3? ?? 4? ?? 5? ?? 6? ?? 7? ?? 8? ?? 9? ? 10
>>> 
>>> ? wit best regards,
>>> ? Sudheer 
>>> ? ***************************************************************
>>> ? Sudheer Joseph 
>>> ? Indian National Centre for Ocean Information Services
>>> ? Ministry of Earth Sciences, Govt. of India
>>> ? POST BOX NO: 21, IDA Jeedeemetla P.O.
>>> ? Via Pragathi Nagar,Kukatpally, Hyderabad; Pin:5000 55
>>> ? Tel:+91-40-23886047(O),Fax:+91-40-23895011(O),
>>> ? Tel:+91-40-23044600(R),Tel:+91-40-9440832534(Mobile)
>>> ? E-mail:sjo.India at gmail.com;sudheer.joseph at yahoo.com
>>> ? Web- http://oppamthadathil.tripod.com
>>> ? ***************************************************************
>>> ? _______________________________________________
>>> ? NumPy-Discussion mailing list
>>> ? NumPy-Discussion at scipy.org
>>> ? http://mail.scipy.org/mailman/listinfo/numpy-discussion
>> 
>>  _______________________________________________
>>  NumPy-Discussion mailing list
>>  NumPy-Discussion at scipy.org
>>  http://mail.scipy.org/mailman/listinfo/numpy-discussion
>> 
> _______________________________________________
> NumPy-Discussion mailing list
> NumPy-Discussion at scipy.org
> http://mail.scipy.org/mailman/listinfo/numpy-discussion
> 


From rhattersley at gmail.com  Sun May 26 03:52:38 2013
From: rhattersley at gmail.com (Richard Hattersley)
Date: Sun, 26 May 2013 08:52:38 +0100
Subject: [Numpy-discussion] NumPy sprints at Scipy 2013,
 Austin: call for topics and hands to help
In-Reply-To: <CAGY4rcXRo+jjxxMBpLMhO2vBkFo5d=rL9ZsOTVLw5UeeZruuGQ@mail.gmail.com>
References: <CAGY4rcXRo+jjxxMBpLMhO2vBkFo5d=rL9ZsOTVLw5UeeZruuGQ@mail.gmail.com>
Message-ID: <CAP=RS9mZ1NiehvxsUnEq14P4HmMK9Cjfay4hvsaDTf3CFo7ZoQ@mail.gmail.com>

Hi David,

On 25 May 2013 15:23, David Cournapeau <cournape at gmail.com> wrote:

> As some of you may know, St?fan and me will present a tutorial on
> NumPy C code, so if we do our job correctly, we should have a few new
> people ready to help out during the sprints.
>

Is there any chance you'll be repeating this at EuroSciPy?


Things I'd like to work on myself is looking into splitting things
> from multiarray, think about a better internal API for dtype
> registration/hooks (with the goal to remove any date dtype hardcoding
> in both multiarray and ufunc machinery), but I am sure others have
> more interesting ideas :)
>

I'm not able to get to SciPy so I understand if my vote of support doesn't
count ;-), but I'm very interested in the work on the dtype API. And if it
was on the radar for EuroSciPy there's a good chance I'd be able to help
out. (The combination of a NumPy C tutorial and dtype API work would make a
pretty compelling case for my managers.)

Regards,
Richard
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20130526/da72b5fa/attachment.html>

From aronne.merrelli at gmail.com  Sun May 26 04:43:56 2013
From: aronne.merrelli at gmail.com (Aronne Merrelli)
Date: Sun, 26 May 2013 10:43:56 +0200
Subject: [Numpy-discussion] array manupulation
In-Reply-To: <1369535424.86470.YahooMailNeo@web193402.mail.sg3.yahoo.com>
References: <1369484398.86666.YahooMailNeo@web193406.mail.sg3.yahoo.com>
	<B575BA4B-29A6-4B4E-8C23-519547918C48@continuum.io>
	<1369529735.86699.YahooMailNeo@web193405.mail.sg3.yahoo.com>
	<1369535424.86470.YahooMailNeo@web193402.mail.sg3.yahoo.com>
Message-ID: <CAHNdQ4LqbCDN+g6yxXUZ2tDkS1Aa-REXBEXV+zgXSa+1Kz4Sdw@mail.gmail.com>

On Sun, May 26, 2013 at 4:30 AM, Sudheer Joseph <sudheer.joseph at yahoo.com>wrote:

> Dear Brian,
>                 I even tried below but no luck!
> In [138]: xx=np.zeros(11)
> In [139]: xx
> Out[139]: array([ 0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.])
>
> In [147]: xx.shape
> Out[147]: (11,)
> In [140]: xx=np.array(xx)[np.newaxis]
> In [141]: xx.shape
> Out[141]: (1, 11)
> In [142]: xx=xx.T
> In [143]: xx.shape
> Out[143]: (11, 1)
> In [144]: csum.shape
> Out[144]: (11, 5)
> In [145]: np.vstack((xx,csum))
> ---------------------------------------------------------------------------
> ValueError                                Traceback (most recent call last)
> /media/SJOITB/SST_VAL/<ipython-input-145-2a0a60f68737> in <module>()
> ----> 1 np.vstack((xx,csum))
>
> /usr/local/lib/python2.7/dist-packages/numpy-1.7.0-py2.7-linux-x86_64.egg/numpy/core/shape_base.pyc
> in vstack(tup)
>     224
>     225     """
> --> 226     return _nx.concatenate(map(atleast_2d,tup),0)
>     227
>     228 def hstack(tup):
>
> ValueError: all the input array dimensions except for the concatenation
> axis must match exactly
>
>
>

You've transposed the arrays, so now you need to stack the other way. So,
you need to use hstack to concatenate arrays with the same column length
(first axis), or vstack to concatenate arrays with the same row length
(second axis). For example:

In [110]: xx1 = np.zeros((1,7)); cc1 = np.ones((3,7))

In [111]: xx2 = np.zeros((7,1)); cc2 = np.ones((7,3))

In [112]: np.vstack((xx1, cc1))
Out[112]:
array([[ 0.,  0.,  0.,  0.,  0.,  0.,  0.],
       [ 1.,  1.,  1.,  1.,  1.,  1.,  1.],
       [ 1.,  1.,  1.,  1.,  1.,  1.,  1.],
       [ 1.,  1.,  1.,  1.,  1.,  1.,  1.]])

In [113]: np.hstack((xx2, cc2))
Out[113]:
array([[ 0.,  1.,  1.,  1.],
       [ 0.,  1.,  1.,  1.],
       [ 0.,  1.,  1.,  1.],
       [ 0.,  1.,  1.,  1.],
       [ 0.,  1.,  1.,  1.],
       [ 0.,  1.,  1.,  1.],
       [ 0.,  1.,  1.,  1.]])


Also, I would highly recommend studying the NumPy for MATLAB users guide:

http://www.scipy.org/NumPy_for_Matlab_Users

These issues (any many more) are discussed there.


Cheers,
Aronne
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20130526/26158775/attachment.html>

From sudheer.joseph at yahoo.com  Sun May 26 06:55:47 2013
From: sudheer.joseph at yahoo.com (Sudheer Joseph)
Date: Sun, 26 May 2013 18:55:47 +0800 (SGT)
Subject: [Numpy-discussion] array manupulation
In-Reply-To: <CAHNdQ4LqbCDN+g6yxXUZ2tDkS1Aa-REXBEXV+zgXSa+1Kz4Sdw@mail.gmail.com>
References: <1369484398.86666.YahooMailNeo@web193406.mail.sg3.yahoo.com>
	<B575BA4B-29A6-4B4E-8C23-519547918C48@continuum.io>
	<1369529735.86699.YahooMailNeo@web193405.mail.sg3.yahoo.com>
	<1369535424.86470.YahooMailNeo@web193402.mail.sg3.yahoo.com>
	<CAHNdQ4LqbCDN+g6yxXUZ2tDkS1Aa-REXBEXV+zgXSa+1Kz4Sdw@mail.gmail.com>
Message-ID: <1369565747.32843.YahooMailNeo@web193401.mail.sg3.yahoo.com>

Thank you Aronne for the helping hand,
? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? I tried the transpose as a check when I could not get it correct other way. I could do it with test arrays, but it appears some thing strange happens when I do the cumsum. So I am attaching here the csum as csum.npy array, where I face problem if your time permits please see what happens with this strange array.!


In [1]: csum=np.load('csum.npy')?should get the array to you.

This ?array is obtained by doing a?
csum=np.cumsum(prcnt), which apparently doing some thing which I am not able to visualize.

with best regards,
Sudheer.

>From:Aronne Merrelli <aronne.merrelli at gmail.com>
>To:Discussion of Numerical Python <numpy-discussion at scipy.org> 
>Sent:Sunday, 26 May 2013 2:13 PM
>Subject:Re: [Numpy-discussion] array manupulation
> 
>
>
>
>
>On Sun, May 26, 2013 at 4:30 AM, Sudheer Joseph <sudheer.joseph at yahoo.com> wrote:
>
>Dear Brian,
>>? ? ? ? ? ? ? ? I even tried below but no luck!
>>In [138]: xx=np.zeros(11)
>>In [139]: xx
>>Out[139]: array([ 0., ?0., ?0., ?0., ?0., ?0., ?0., ?0., ?0., ?0., ?0.])
>>
>>In [147]: xx.shape
>>Out[147]: (11,)
>>In [140]: xx=np.array(xx)[np.newaxis]
>>In [141]: xx.shape
>>Out[141]: (1, 11)
>>In [142]: xx=xx.T
>>In [143]: xx.shape
>>Out[143]: (11, 1)
>>In [144]: csum.shape
>>Out[144]: (11, 5)
>>In [145]: np.vstack((xx,csum))
>>
>>---------------------------------------------------------------------------
>>ValueError ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?Traceback (most recent call last)
>>/media/SJOITB/SST_VAL/<ipython-input-145-2a0a60f68737> in <module>()
>>----> 1 np.vstack((xx,csum))
>>
>>
>>/usr/local/lib/python2.7/dist-packages/numpy-1.7.0-py2.7-linux-x86_64.egg/numpy/core/shape_base.pyc in vstack(tup)
>>? ? 224?
>>? ? 225 ? ? """
>>--> 226 ? ? return _nx.concatenate(map(atleast_2d,tup),0)
>>? ? 227?
>>? ? 228 def hstack(tup):
>>
>>ValueError: all the input array dimensions except for the concatenation axis must match exactly
>>
>>??
>>
>
>
>You've transposed the arrays, so now you need to stack the other way. So, you need to use hstack to concatenate arrays with the same column length (first axis), or vstack to concatenate arrays with the same row length (second axis). For example:
>
>
>In [110]: xx1 = np.zeros((1,7)); cc1 = np.ones((3,7))
>
>
>In [111]: xx2 = np.zeros((7,1)); cc2 = np.ones((7,3))
>
>
>In [112]: np.vstack((xx1, cc1))
>Out[112]:?
>array([[ 0., ?0., ?0., ?0., ?0., ?0., ?0.],
>? ? ? ?[ 1., ?1., ?1., ?1., ?1., ?1., ?1.],
>? ? ? ?[ 1., ?1., ?1., ?1., ?1., ?1., ?1.],
>? ? ? ?[ 1., ?1., ?1., ?1., ?1., ?1., ?1.]])
>
>
>In [113]: np.hstack((xx2, cc2))
>Out[113]:?
>array([[ 0., ?1., ?1., ?1.],
>? ? ? ?[ 0., ?1., ?1., ?1.],
>? ? ? ?[ 0., ?1., ?1., ?1.],
>? ? ? ?[ 0., ?1., ?1., ?1.],
>? ? ? ?[ 0., ?1., ?1., ?1.],
>? ? ? ?[ 0., ?1., ?1., ?1.],
>? ? ? ?[ 0., ?1., ?1., ?1.]])
>
>
>
>
>Also, I would highly recommend studying the NumPy for MATLAB users guide:
>
>
>http://www.scipy.org/NumPy_for_Matlab_Users
>
>
>
>These issues (any many more) are discussed there.
>
>
>
>
>Cheers,
>Aronne
>_______________________________________________
>NumPy-Discussion mailing list
>NumPy-Discussion at scipy.org
>http://mail.scipy.org/mailman/listinfo/numpy-discussion
>
>
>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: csum.npy
Type: application/octet-stream
Size: 520 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20130526/72ef627a/attachment.obj>

From charlesr.harris at gmail.com  Sun May 26 13:51:54 2013
From: charlesr.harris at gmail.com (Charles R Harris)
Date: Sun, 26 May 2013 11:51:54 -0600
Subject: [Numpy-discussion] multivariate_normal issue with 'size'
	argument
In-Reply-To: <519FBAE1.2080107@uci.edu>
References: <519F56CE.6000606@relativita.com>
	<68CB3562-2A48-459F-9FFD-C879E39070CA@inria.fr>
	<519F6420.70609@relativita.com>
	<CAKVJ-_6q8tiExyCEcno5o+gKQdZCJwDUGoA142PctHLuqZ-Q1g@mail.gmail.com>
	<CAF6FJiv39DFM8sJUOcsTmHundUPf3mcd2r43w7f==7gAAOXodQ@mail.gmail.com>
	<CAKVJ-_5cuCJvZZs6S=_pJ-HoGEWQM+ARU6mC50WDOR91fE4JpA@mail.gmail.com>
	<CAGzF1udYc3NGUbMKj1OqYC2muQb2h6vm87Q8Nszr-vvnNSO4Tw@mail.gmail.com>
	<CAMMTP+B1G6qiuHzxf101b_GSaDGpvPrbA197YjXU_LHtnDRFrA@mail.gmail.com>
	<519FBAE1.2080107@uci.edu>
Message-ID: <CAB6mnxKyQL453M_UgfsMZJa6OeaBqdwi3k7=hWRXBz1viaHOzA@mail.gmail.com>

On Fri, May 24, 2013 at 1:09 PM, Christoph Gohlke <cgohlke at uci.edu> wrote:
<snip>


> This patch works for me:
>
> diff --git a/numpy/random/mtrand/mtrand.pyx
> b/numpy/random/mtrand/mtrand.pyx
> index b0de560..233ff52 100644
> --- a/numpy/random/mtrand/mtrand.pyx
> +++ b/numpy/random/mtrand/mtrand.pyx
> @@ -4156,7 +4156,7 @@ cdef class RandomState:
>           if mean.shape[0] != cov.shape[0]:
>                  raise ValueError("mean and cov must have same length")
>           # Compute shape of output
> -        if isinstance(shape, int):
> +        if isinstance(shape, (int, long, np.integer)):
>               shape = [shape]
>           final_shape = list(shape[:])
>           final_shape.append(mean.shape[0])
>
>
Looks right. Can you make a PR for this, maybe with a test that might
otherwise fail for some architectures?

Chuck
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20130526/47da7a0c/attachment.html>

From shish at keba.be  Sun May 26 17:52:13 2013
From: shish at keba.be (Olivier Delalleau)
Date: Sun, 26 May 2013 17:52:13 -0400
Subject: [Numpy-discussion] array manupulation
In-Reply-To: <1369565747.32843.YahooMailNeo@web193401.mail.sg3.yahoo.com>
References: <1369484398.86666.YahooMailNeo@web193406.mail.sg3.yahoo.com>
	<B575BA4B-29A6-4B4E-8C23-519547918C48@continuum.io>
	<1369529735.86699.YahooMailNeo@web193405.mail.sg3.yahoo.com>
	<1369535424.86470.YahooMailNeo@web193402.mail.sg3.yahoo.com>
	<CAHNdQ4LqbCDN+g6yxXUZ2tDkS1Aa-REXBEXV+zgXSa+1Kz4Sdw@mail.gmail.com>
	<1369565747.32843.YahooMailNeo@web193401.mail.sg3.yahoo.com>
Message-ID: <CAFXk4bqngHTSvP870FHFqkLv7h0TsQKpdj2TFEEbygei035FQg@mail.gmail.com>

Your array doesn't seem strange, it looks like a perfectly normal (11 x 5)
matrix of dtype float64.

>>> x = np.load('csum.npy')
>>> np.vstack((np.zeros((1, x.shape[1])), x))
array([[   0.        ,    0.        ,    0.        ,    0.        ,
0.        ],
       [  31.82571459,   29.0629995 ,   27.74400711,   26.6248159 ,
          25.73787976],
       [  59.82231014,   54.27656749,   51.87813602,   50.00937323,
          48.51771275],
       [  80.03460893,   73.46862838,   70.55710765,   68.412796  ,
          66.64323907],
       [  91.12613011,   85.96434025,   83.34633829,   81.36538282,
          79.70197141],
       [  96.11498624,   93.00049572,   91.13864656,   89.61535722,
          88.27247424],
       [  98.22403322,   96.55379518,   95.43277035,   94.39550817,
          93.42804   ],
       [  99.14200421,   98.27546395,   97.64792507,   97.00438205,
          96.3689249 ],
       [  99.55954577,   99.10418687,   98.76971791,   98.39724171,
          98.00386825],
       [  99.76081882,   99.51702755,   99.33960611,   99.13057243,
          98.9007987 ],
       [  99.8617198 ,   99.72882047,   99.63273748,   99.51539561,
          99.38460995],
       [ 100.        ,  100.        ,  100.        ,  100.        ,
100.        ]])

-=- Olivier


2013/5/26 Sudheer Joseph <sudheer.joseph at yahoo.com>

> Thank you Aronne for the helping hand,
>                                       I tried the transpose as a check
> when I could not get it correct other way. I could do it with test arrays,
> but it appears some thing strange happens when I do the cumsum. So I am
> attaching here the csum as csum.npy array, where I face problem if your
> time permits please see what happens with this strange array.!
>
>
> In [1]: csum=np.load('csum.npy') should get the array to you.
>
> This  array is obtained by doing a
> csum=np.cumsum(prcnt), which apparently doing some thing which I am not
> able to visualize.
>
> with best regards,
> Sudheer.
>
> >From:Aronne Merrelli <aronne.merrelli at gmail.com>
> >To:Discussion of Numerical Python <numpy-discussion at scipy.org>
> >Sent:Sunday, 26 May 2013 2:13 PM
> >Subject:Re: [Numpy-discussion] array manupulation
> >
> >
> >
> >
> >
> >On Sun, May 26, 2013 at 4:30 AM, Sudheer Joseph <sudheer.joseph at yahoo.com>
> wrote:
> >
> >Dear Brian,
> >>                I even tried below but no luck!
> >>In [138]: xx=np.zeros(11)
> >>In [139]: xx
> >>Out[139]: array([ 0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.])
> >>
> >>In [147]: xx.shape
> >>Out[147]: (11,)
> >>In [140]: xx=np.array(xx)[np.newaxis]
> >>In [141]: xx.shape
> >>Out[141]: (1, 11)
> >>In [142]: xx=xx.T
> >>In [143]: xx.shape
> >>Out[143]: (11, 1)
> >>In [144]: csum.shape
> >>Out[144]: (11, 5)
> >>In [145]: np.vstack((xx,csum))
> >>
>
> >>---------------------------------------------------------------------------
> >>ValueError                                Traceback (most recent call
> last)
> >>/media/SJOITB/SST_VAL/<ipython-input-145-2a0a60f68737> in <module>()
> >>----> 1 np.vstack((xx,csum))
> >>
> >>
> >>/usr/local/lib/python2.7/dist-packages/numpy-1.7.0-py2.7-linux-x86_64.egg/numpy/core/shape_base.pyc
> in vstack(tup)
> >>    224
> >>    225     """
> >>--> 226     return _nx.concatenate(map(atleast_2d,tup),0)
> >>    227
> >>    228 def hstack(tup):
> >>
> >>ValueError: all the input array dimensions except for the concatenation
> axis must match exactly
> >>
> >>
> >>
> >
> >
> >You've transposed the arrays, so now you need to stack the other way. So,
> you need to use hstack to concatenate arrays with the same column length
> (first axis), or vstack to concatenate arrays with the same row length
> (second axis). For example:
> >
> >
> >In [110]: xx1 = np.zeros((1,7)); cc1 = np.ones((3,7))
> >
> >
> >In [111]: xx2 = np.zeros((7,1)); cc2 = np.ones((7,3))
> >
> >
> >In [112]: np.vstack((xx1, cc1))
> >Out[112]:
> >array([[ 0.,  0.,  0.,  0.,  0.,  0.,  0.],
> >       [ 1.,  1.,  1.,  1.,  1.,  1.,  1.],
> >       [ 1.,  1.,  1.,  1.,  1.,  1.,  1.],
> >       [ 1.,  1.,  1.,  1.,  1.,  1.,  1.]])
> >
> >
> >In [113]: np.hstack((xx2, cc2))
> >Out[113]:
> >array([[ 0.,  1.,  1.,  1.],
> >       [ 0.,  1.,  1.,  1.],
> >       [ 0.,  1.,  1.,  1.],
> >       [ 0.,  1.,  1.,  1.],
> >       [ 0.,  1.,  1.,  1.],
> >       [ 0.,  1.,  1.,  1.],
> >       [ 0.,  1.,  1.,  1.]])
> >
> >
> >
> >
> >Also, I would highly recommend studying the NumPy for MATLAB users guide:
> >
> >
> >http://www.scipy.org/NumPy_for_Matlab_Users
> >
> >
> >
> >These issues (any many more) are discussed there.
> >
> >
> >
> >
> >Cheers,
> >Aronne
> >_______________________________________________
> >NumPy-Discussion mailing list
> >NumPy-Discussion at scipy.org
> >http://mail.scipy.org/mailman/listinfo/numpy-discussion
> >
> >
> >
>
> _______________________________________________
> NumPy-Discussion mailing list
> NumPy-Discussion at scipy.org
> http://mail.scipy.org/mailman/listinfo/numpy-discussion
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20130526/bb5e4555/attachment.html>

From sudheer.joseph at yahoo.com  Sun May 26 20:02:14 2013
From: sudheer.joseph at yahoo.com (Sudheer Joseph)
Date: Mon, 27 May 2013 08:02:14 +0800 (SGT)
Subject: [Numpy-discussion] array manupulation
In-Reply-To: <CAFXk4bqngHTSvP870FHFqkLv7h0TsQKpdj2TFEEbygei035FQg@mail.gmail.com>
References: <1369484398.86666.YahooMailNeo@web193406.mail.sg3.yahoo.com>
	<B575BA4B-29A6-4B4E-8C23-519547918C48@continuum.io>
	<1369529735.86699.YahooMailNeo@web193405.mail.sg3.yahoo.com>
	<1369535424.86470.YahooMailNeo@web193402.mail.sg3.yahoo.com>
	<CAHNdQ4LqbCDN+g6yxXUZ2tDkS1Aa-REXBEXV+zgXSa+1Kz4Sdw@mail.gmail.com>
	<1369565747.32843.YahooMailNeo@web193401.mail.sg3.yahoo.com>
	<CAFXk4bqngHTSvP870FHFqkLv7h0TsQKpdj2TFEEbygei035FQg@mail.gmail.com>
Message-ID: <1369612934.5248.YahooMailNeo@web193401.mail.sg3.yahoo.com>

Thank you very much Oliver,
>? ? ? ? ? It did not occurred to me that this can be done so simple with size of original array itself!
>Thanks a lot.
>with best regards,
>Sudheer
>From: Olivier Delalleau <shish at keba.be>
>To: Discussion of Numerical Python <numpy-discussion at scipy.org> 
>Sent: Monday, 27 May 2013 3:22 AM
>Subject: Re: [Numpy-discussion] array manupulation
> 
>
>
>Your array doesn't seem strange, it looks like a perfectly normal (11 x 5) matrix of dtype float64.
>
>>>> x = np.load('csum.npy')
>>>> np.vstack((np.zeros((1, x.shape[1])), x))
>array([[?? 0.??????? ,??? 0.??????? ,??? 0.??????? ,??? 0.??????? ,??? 0.??????? ],
>?????? [? 31.82571459,?? 29.0629995 ,?? 27.74400711,?? 26.6248159 ,
>????????? 25.73787976],
>?????? [? 59.82231014,?? 54.27656749,?? 51.87813602,?? 50.00937323,
>????????? 48.51771275],
>?????? [? 80.03460893,?? 73.46862838,?? 70.55710765,?? 68.412796? ,
>????????? 66.64323907],
>?????? [? 91.12613011,?? 85.96434025,?? 83.34633829,?? 81.36538282,
>????????? 79.70197141],
>?????? [? 96.11498624,?? 93.00049572,?? 91.13864656,?? 89.61535722,
>????????? 88.27247424],
>?????? [? 98.22403322,?? 96.55379518,?? 95.43277035,?? 94.39550817,
>????????? 93.42804?? ],
>?????? [? 99.14200421,?? 98.27546395,?? 97.64792507,?? 97.00438205,
>????????? 96.3689249 ],
>?????? [? 99.55954577,?? 99.10418687,?? 98.76971791,?? 98.39724171,
>????????? 98.00386825],
>?????? [? 99.76081882,?? 99.51702755,?? 99.33960611,?? 99.13057243,
>????????? 98.9007987 ],
>?????? [? 99.8617198 ,?? 99.72882047,?? 99.63273748,?? 99.51539561,
>????????? 99.38460995],
>?????? [ 100.??????? ,? 100.??????? ,? 100.??????? ,? 100.??????? ,? 100.??????? ]])
>
>-=- Olivier
>
>
>
>
>2013/5/26 Sudheer Joseph <sudheer.joseph at yahoo.com>
>
>Thank you Aronne for the helping hand,
>>? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? I tried the transpose as a check when I could not get it correct other way. I could do it with test arrays, but it appears some thing strange happens when I do the cumsum. So I am attaching here the csum as csum.npy array, where I face problem if your time permits please see what happens with this strange array.!
>>
>>
>>In [1]: csum=np.load('csum.npy')?should get the array to you.
>>
>>This ?array is obtained by doing a?
>>csum=np.cumsum(prcnt), which apparently doing some thing which I am not able to visualize.
>>
>>with best regards,
>>Sudheer.
>>
>>>From:Aronne Merrelli <aronne.merrelli at gmail.com>
>>
>>>To:Discussion of Numerical Python <numpy-discussion at scipy.org>
>>>Sent:Sunday, 26 May 2013 2:13 PM
>>
>>>Subject:Re: [Numpy-discussion] array manupulation
>>>
>>>
>>>
>>>
>>>
>>
>>>On Sun, May 26, 2013 at 4:30 AM, Sudheer Joseph <sudheer.joseph at yahoo.com> wrote:
>>>
>>>Dear Brian,
>>>>? ? ? ? ? ? ? ? I even tried below but no luck!
>>>>In [138]: xx=np.zeros(11)
>>>>In [139]: xx
>>>>Out[139]: array([ 0., ?0., ?0., ?0., ?0., ?0., ?0., ?0., ?0., ?0., ?0.])
>>>>
>>>>In [147]: xx.shape
>>>>Out[147]: (11,)
>>>>In [140]: xx=np.array(xx)[np.newaxis]
>>>>In [141]: xx.shape
>>>>Out[141]: (1, 11)
>>>>In [142]: xx=xx.T
>>>>In [143]: xx.shape
>>>>Out[143]: (11, 1)
>>>>In [144]: csum.shape
>>>>Out[144]: (11, 5)
>>>>In [145]: np.vstack((xx,csum))
>>>>
>>>>---------------------------------------------------------------------------
>>>>ValueError ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?Traceback (most recent call last)
>>>>/media/SJOITB/SST_VAL/<ipython-input-145-2a0a60f68737> in <module>()
>>>>----> 1 np.vstack((xx,csum))
>>>>
>>>>
>>>>/usr/local/lib/python2.7/dist-packages/numpy-1.7.0-py2.7-linux-x86_64.egg/numpy/core/shape_base.pyc in vstack(tup)
>>>>? ? 224?
>>>>? ? 225 ? ? """
>>>>--> 226 ? ? return _nx.concatenate(map(atleast_2d,tup),0)
>>>>? ? 227?
>>>>? ? 228 def hstack(tup):
>>>>
>>>>ValueError: all the input array dimensions except for the concatenation axis must match exactly
>>>>
>>>>??
>>>>
>>>
>>>
>>>You've transposed the arrays, so now you need to stack the other way. So, you need to use hstack to concatenate arrays with the same column length (first axis), or vstack to concatenate arrays with the same row length (second axis). For example:
>>>
>>>
>>>In [110]: xx1 = np.zeros((1,7)); cc1 = np.ones((3,7))
>>>
>>>
>>>In [111]: xx2 = np.zeros((7,1)); cc2 = np.ones((7,3))
>>>
>>>
>>>In [112]: np.vstack((xx1, cc1))
>>>Out[112]:?
>>>array([[ 0., ?0., ?0., ?0., ?0., ?0., ?0.],
>>>? ? ? ?[ 1., ?1., ?1., ?1., ?1., ?1., ?1.],
>>>? ? ? ?[ 1., ?1., ?1., ?1., ?1., ?1., ?1.],
>>>? ? ? ?[ 1., ?1., ?1., ?1., ?1., ?1., ?1.]])
>>>
>>>
>>>In [113]: np.hstack((xx2, cc2))
>>>Out[113]:?
>>>array([[ 0., ?1., ?1., ?1.],
>>>? ? ? ?[ 0., ?1., ?1., ?1.],
>>>? ? ? ?[ 0., ?1., ?1., ?1.],
>>>? ? ? ?[ 0., ?1., ?1., ?1.],
>>>? ? ? ?[ 0., ?1., ?1., ?1.],
>>>? ? ? ?[ 0., ?1., ?1., ?1.],
>>>? ? ? ?[ 0., ?1., ?1., ?1.]])
>>>
>>>
>>>
>>>
>>>Also, I would highly recommend studying the NumPy for MATLAB users guide:
>>>
>>>
>>>http://www.scipy.org/NumPy_for_Matlab_Users
>>>
>>>
>>>
>>>These issues (any many more) are discussed there.
>>>
>>>
>>>
>>>
>>>Cheers,
>>>Aronne
>>
>>>_______________________________________________
>>>NumPy-Discussion mailing list
>>>NumPy-Discussion at scipy.org
>>>http://mail.scipy.org/mailman/listinfo/numpy-discussion
>>>
>>>
>>>
>>_______________________________________________
>>NumPy-Discussion mailing list
>>NumPy-Discussion at scipy.org
>>http://mail.scipy.org/mailman/listinfo/numpy-discussion
>>
>>
>
>_______________________________________________
>NumPy-Discussion mailing list
>NumPy-Discussion at scipy.org
>http://mail.scipy.org/mailman/listinfo/numpy-discussion
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20130527/52a16392/attachment.html>

From dineshbvadhia at hotmail.com  Tue May 28 08:08:41 2013
From: dineshbvadhia at hotmail.com (Dinesh B Vadhia)
Date: Tue, 28 May 2013 05:08:41 -0700
Subject: [Numpy-discussion] Unable to building numpy with openblas using
	bento or distutils
Message-ID: <BAY173-DS10F7F3AB896E3FCA899A1EA3970@phx.gbl>

Maybe this is for the Scipy list but followed the instructions but the command:

$ sudo python setup.py install

to install Scipy failed because it couldn't find Lapack.  However, the export commands were set as:

$ export BLAS=/usr/local/lib/libopenblas.a
$ export LAPACK=/usr/local/lib/libopenblas.a
$ export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/usr/local/lib/

Further, the site.cfg file in the Numpy folder was copied to the Scipy folder.

Numpy built and installed successfully.
Scipy built (with $ python setup.py build) successfully.

Any thoughts?


On Sat, Mar 23, 2013 at 8:44 PM, Skipper Seabold <jsseabold at gmail.com> wrote:
> On Sat, Mar 23, 2013 at 7:26 PM, Ake Sandgren <ake.sandgren at hpc2n.umu.se> wrote:
>> On Sat, 2013-03-23 at 14:19 -0400, Skipper Seabold wrote:
>>> Some help on this would be greatly appreciated. It's been recommended
>>> to use OpenBlas over ATLAS, so I've been trying to build numpy with
>>> openblas and have run into a few problems.
>>
>>>
>>> To truly support OpenBlas, is it maybe necessary to make some
>>> additions to numpy/distutils/system_info.py?
>>
>> Here is how.
>>
>> https://github.com/akesandgren/numpy/commit/363339dd3a9826f3e3e7dc4248c258d3c4dfcd7c
>>
>
>
> Thanks that works well for numpy. Test pass. I hope that makes it into
> a pull request. My site.cfg looks like this. I don't know about the
> lapack_opt section. It doesn't seem to work.
>
> [DEFAULT]
> library_dirs = /home/skipper/.local/lib
> include_dirs = /home/skipper/.local/include
>
> [openblas]
> libraries = openblas
>
> [lapack_opt]
> libraries = openblas
>
> Do you have any idea how to get scipy working too. I have a similar
> site.cfg, but it does not find lapack, which is rolled into
> libopenblas from what I understand. I can do
>
> export LAPACK=~/.local/lib/libopenblas.a
> python setup.py build &> build.log
> sudo -E python setup.py install
>
> There are no obvious failures in the build.log, but scipy is still
> broken because it needs lapack from numpy I guess.

The answer is to

export BLAS=~/.local/lib/libopenblas.a
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:~/.local/lib/

before building and installing. Now everything works. Whew. Thanks a
lot for the help.

>
>>>> import numpy as np
>>>> np.show_config()
> lapack_info:
>   NOT AVAILABLE
> atlas_threads_info:
>   NOT AVAILABLE
> blas_opt_info:
>     libraries = ['openblas', 'openblas']
>     library_dirs = ['/home/skipper/.local/lib']
>     language = f77
> lapack_src_info:
>   NOT AVAILABLE
> openblas_info:
>     libraries = ['openblas', 'openblas']
>     library_dirs = ['/home/skipper/.local/lib']
>     language = f77
> lapack_opt_info:
>   NOT AVAILABLE
> atlas_info:
>   NOT AVAILABLE
> lapack_mkl_info:
>   NOT AVAILABLE
> blas_mkl_info:
>   NOT AVAILABLE
> mkl_info:
>   NOT AVAILABLE
>>>> from scipy import stats
> Traceback (most recent call last):
>   File "<stdin>", line 1, in <module>
>   File "/usr/local/lib/python2.7/dist-packages/scipy/stats/__init__.py",
> line 320, in <module>
>     from .stats import *
>   File "/usr/local/lib/python2.7/dist-packages/scipy/stats/stats.py",
> line 242, in <module>
>     import scipy.linalg as linalg
>   File "/usr/local/lib/python2.7/dist-packages/scipy/linalg/__init__.py",
> line 147, in <module>
>     from .misc import *
>   File "/usr/local/lib/python2.7/dist-packages/scipy/linalg/misc.py",
> line 5, in <module>
>     from . import blas
>   File "/usr/local/lib/python2.7/dist-packages/scipy/linalg/blas.py",
> line 113, in <module>
>     from scipy.linalg import _fblas
> ImportError: libopenblas.so.0: cannot open shared object file: No such
> file or directory
>
> Skipper
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20130528/a3dac4f6/attachment.html>

From andrew.collette at gmail.com  Tue May 28 14:47:15 2013
From: andrew.collette at gmail.com (Andrew Collette)
Date: Tue, 28 May 2013 12:47:15 -0600
Subject: [Numpy-discussion] Parameterised dtypes
In-Reply-To: <CAP=RS9kCSm7343uFc1qYKT=HgHiO9sr9W8K-5gp_NFkM3G27Bw@mail.gmail.com>
References: <CAP=RS9kCSm7343uFc1qYKT=HgHiO9sr9W8K-5gp_NFkM3G27Bw@mail.gmail.com>
Message-ID: <CALmrCV1e7WAXiC1ytJV1hQhCCKTHd-1sKH3vE=PzhuUvrTf3BA@mail.gmail.com>

Hi Richard,

> I'm in the process of defining some new dtypes to handle non-physical
> calendars (such as the 360-day calendar used in the climate modelling
> world). This is all going fine[*] so far, but I'd like to know a little bit
> more about how much is ultimately possible.
>
> The PyArray_Descr members `metadata` and `c_metadata` allow run-time
> parametrisation, but is it possible to hook into the dtype('...') parsing
> mechanism to supply those parameters? Or is there some other dtype mechanism
> for supplying parameters?

I would also love to know the answer to this question.  Right now h5py
is using field titles to hold metadata, because it's the most
straightforward way I could find (with my limited NumPy-internals
experience)  to attach metadata that's preserved when dtypes are
passed around, copied, etc.  It works but is awkward.  If there's a
non-C way to deal with this that would be great.

> [*] The docs could do with updating, and the examples would benefit from
> standardising (or at least explaining the significance of the differences).
> I intend to post updates where possible.

Maybe a stupid question, but do you know a reference I could look at
for the metadata and c_metadata fields you described?

Andrew


From njs at pobox.com  Tue May 28 16:06:18 2013
From: njs at pobox.com (Nathaniel Smith)
Date: Tue, 28 May 2013 21:06:18 +0100
Subject: [Numpy-discussion] Parameterised dtypes
In-Reply-To: <CAP=RS9kCSm7343uFc1qYKT=HgHiO9sr9W8K-5gp_NFkM3G27Bw@mail.gmail.com>
References: <CAP=RS9kCSm7343uFc1qYKT=HgHiO9sr9W8K-5gp_NFkM3G27Bw@mail.gmail.com>
Message-ID: <CAPJVwB=5ibVyk2cAoTSpy4fDwAbN71CYysmp4EpGJRaf_0SwSw@mail.gmail.com>

On Fri, May 24, 2013 at 3:12 PM, Richard Hattersley
<rhattersley at gmail.com> wrote:
> Hi all,
>
> I'm in the process of defining some new dtypes to handle non-physical
> calendars (such as the 360-day calendar used in the climate modelling
> world). This is all going fine[*] so far, but I'd like to know a little bit
> more about how much is ultimately possible.
>
> The PyArray_Descr members `metadata` and `c_metadata` allow run-time
> parametrisation, but is it possible to hook into the dtype('...') parsing
> mechanism to supply those parameters? Or is there some other dtype mechanism
> for supplying parameters?
>
> As an example, would it be possible to supply month lengths?
>>>> a = np.zeros(n, dtype='my_date[34,33,31,30,30,29,29,30,31,32,34,35]')
>
> Or is the intended use of parametrisation more like:
>>>> weird = my_stuff.make_dtype([34,33,31,30,30,29,29,30,31,32,34,35])
>>>> a = np.zeros(n, dtype=weird)

I don't think there's any "intended use" really. AFAICT numpy was
originally designed with the assumption that there were a fixed set of
dtypes stateless dtypes, and then the ability to add new dtypes, the
ability to add state (parametrize them), etc., have gradually been
kluged in as needed to solve whatever immediate problem someone faced.
E.g., dtype callbacks and ufuncs don't in general get access to the
dtype object, so they can't access whatever parameters exist, and the
builtin dtypes that *are* parametrized (strings, structs, etc.) all
have special case code scattered all around numpy.

You don't even need 'metadata' or 'c_metadata' -- this is Python, we
already have a totally standard way to add new fields, just subclass
the dumb thing. Instead we have this baffling system of 'NpyAuxData'
which invents its own tiny little refcounted object system from
scratch, and 'issubdtype' which invents its own way of representing
inheritance hierarchies independent of Python's object system, and so
forth.

Anyway!

1) No, you can't hook into the dtype string parser. Though, are you
sure you really want to? Surely it's nicer to use Python syntax
instead of inventing a new syntax and then having to write a parser
for it from scratch?

2) I have some vague plans worked out to fix all this so dtypes are
just ordinary python objects, but haven't written it down yet due to a
combination of lack of time to do so, and lack of anyone with time to
actually implement the plan even if it were written down. I mention
this just in case someone wants to volunteer, which would move it up
my stack.

-n


From ralf.gommers at gmail.com  Tue May 28 17:10:43 2013
From: ralf.gommers at gmail.com (Ralf Gommers)
Date: Tue, 28 May 2013 23:10:43 +0200
Subject: [Numpy-discussion] Behavior or finance.npv function
In-Reply-To: <CANS9D+qnf5AujrQYM0PRfvqydXFTOD7ZQ1tpw5=gRZSmFncgXg@mail.gmail.com>
References: <CANS9D+qnf5AujrQYM0PRfvqydXFTOD7ZQ1tpw5=gRZSmFncgXg@mail.gmail.com>
Message-ID: <CABL7CQiyKOq57J-SapAnqmVRZNEFC9aHRi-obTNhRJP0UQoGuw@mail.gmail.com>

On Thu, May 23, 2013 at 3:12 AM, Guillaume VIRY <guillaume.viry at gmail.com>wrote:

> Good morning all
>
> I have been using the financial functions for a while now, and the npv
> function has always bothered me.
>
All of them have always bothered me:)

> Indeed neither the documentation nor the implementation seem to be
> correct, and actually are not congruent with each other either.
>
> As stated in the bug report https://github.com/numpy/numpy/issues/649 :
>
> the documentation states that the summation is from 0 to M (should be M-1,
> if the input data is of size M by the way) which is the correct definition,
> but the implementation computes a summation from 1 to M, which is known to
> be wrong, and a mimic of Excel's NPV behavior.
>
> So basically, we would have the following :
>
> RealNPV(data) = data[0] + CurrentNPV(data[1:])
>
> Both behaviors are fine to me, as long as the documentation is clear about
> it. Ideally the implementation should be corrected, but in any case there
> should be a documentation fix at least.
>
> I'd be glad to have your opinion about it.
>
Looks like the code needs to be be changed. The current sum starting at 1
is different from what I see on wikipedia and other definitions scattered
over the internet.

Unless someone complains, I think just considering this a bug and changing
the code is OK.

Ralf
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20130528/21ae72af/attachment.html>

From charlesr.harris at gmail.com  Tue May 28 18:31:28 2013
From: charlesr.harris at gmail.com (Charles R Harris)
Date: Tue, 28 May 2013 16:31:28 -0600
Subject: [Numpy-discussion] quickselect
Message-ID: <CAB6mnxJtcm_VdqgLtMtxszQ1+MUW6kvf2d82noLW=fuxOe8EsA@mail.gmail.com>

Hi All,

There is a PR <https://github.com/numpy/numpy/pull/3360> adding quickselect
to numpy as a function `partition`. Comments on name and exposure in the
numpy API are welcome.

Chuck
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20130528/f0233a33/attachment.html>

From guillaume.viry at gmail.com  Tue May 28 20:49:52 2013
From: guillaume.viry at gmail.com (Guillaume VIRY)
Date: Wed, 29 May 2013 09:49:52 +0900
Subject: [Numpy-discussion] Behavior or finance.npv function
In-Reply-To: <CABL7CQiyKOq57J-SapAnqmVRZNEFC9aHRi-obTNhRJP0UQoGuw@mail.gmail.com>
References: <CANS9D+qnf5AujrQYM0PRfvqydXFTOD7ZQ1tpw5=gRZSmFncgXg@mail.gmail.com>
	<CABL7CQiyKOq57J-SapAnqmVRZNEFC9aHRi-obTNhRJP0UQoGuw@mail.gmail.com>
Message-ID: <CANS9D+qZsaFFNKbwC_-96hgpU_J-FHLDga6Ah8o+2ameEnbJTA@mail.gmail.com>

Glad to see I'm not the only one who cares :)
I've actually made a pull request with the proper documentation / source /
test update, so I guess I shall just wait for it to be accepted now !
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20130529/f81fb6e7/attachment.html>

From josef.pktd at gmail.com  Wed May 29 00:12:11 2013
From: josef.pktd at gmail.com (josef.pktd at gmail.com)
Date: Wed, 29 May 2013 00:12:11 -0400
Subject: [Numpy-discussion] quickselect
In-Reply-To: <CAB6mnxJtcm_VdqgLtMtxszQ1+MUW6kvf2d82noLW=fuxOe8EsA@mail.gmail.com>
References: <CAB6mnxJtcm_VdqgLtMtxszQ1+MUW6kvf2d82noLW=fuxOe8EsA@mail.gmail.com>
Message-ID: <CAMMTP+ArWn1hVswH6GXfJSbBtsAg3j9RZd2R52GzcjAeMjynEQ@mail.gmail.com>

On Tue, May 28, 2013 at 6:31 PM, Charles R Harris
<charlesr.harris at gmail.com> wrote:
> Hi All,
>
> There is a PR adding quickselect to numpy as a function `partition`.
> Comments on name and exposure in the numpy API are welcome.

I think the name is fine. It's possible to get used to it.

One possible use I can think of is if I want to sort afterwards only
one of the tails (like largest 15%).

Limitations:

medium: like percentiles it would be nice to partition into more than
two sets, if that can be done more efficiently than calling the
function twice.

major:
it won't work to calculate the median or other quantiles since those interpolate
np.median: "When N is even, it is the average of the two middle values
of V_sorted"

AFAICS based on the docstring (I closed my eyes when the c code scrolled by)

Josef

>
> Chuck
>
> _______________________________________________
> NumPy-Discussion mailing list
> NumPy-Discussion at scipy.org
> http://mail.scipy.org/mailman/listinfo/numpy-discussion
>


From rhattersley at gmail.com  Wed May 29 05:47:10 2013
From: rhattersley at gmail.com (Richard Hattersley)
Date: Wed, 29 May 2013 10:47:10 +0100
Subject: [Numpy-discussion] Parameterised dtypes
In-Reply-To: <CAPJVwB=5ibVyk2cAoTSpy4fDwAbN71CYysmp4EpGJRaf_0SwSw@mail.gmail.com>
References: <CAP=RS9kCSm7343uFc1qYKT=HgHiO9sr9W8K-5gp_NFkM3G27Bw@mail.gmail.com>
	<CAPJVwB=5ibVyk2cAoTSpy4fDwAbN71CYysmp4EpGJRaf_0SwSw@mail.gmail.com>
Message-ID: <CAP=RS9nOt+vRtyP5fJreC=mHYPkyhYAkseXokCTrhCv3YPnu+Q@mail.gmail.com>

Hi Nathaniel,

Thanks for the useful feedback - it'll definitely save me some time
chasing around the code base.

> dtype callbacks and ufuncs don't in general get access to the
> dtype object, so they can't access whatever parameters exist

Indeed - it is a little awkward. But I'm hoping I can use the `data`
argument to supply this.

> You don't even need 'metadata' or 'c_metadata' -- this is Python, we
> already have a totally standard way to add new fields, just subclass
> the dumb thing.

That would be nice ... but Py_TPFLAGS_BASETYPE is not set for
PyArrayDescr_Type so that class is "final".

> 1) No, you can't hook into the dtype string parser. Though, are you
> sure you really want to? Surely it's nicer to use Python syntax
> instead of inventing a new syntax and then having to write a parser
> for it from scratch?

Thank you - that's good to know. As you say, I'd *far* rather avoid
parsing, but I'd like my dtype to be a "good citizen" so I was asking
out of completeness. (Off at a tangent: The blaze project is a good
example of what happens if you do add more parsing. In my opinion it's
not the way to go.)

> 2) I have some vague plans worked out to fix all this so dtypes are
> just ordinary python objects, but haven't written it down yet due to a
> combination of lack of time to do so, and lack of anyone with time to
> actually implement the plan even if it were written down. I mention
> this just in case someone wants to volunteer, which would move it up
> my stack.

Would you have the time to sketch out the intended benefits?

Richard


From rhattersley at gmail.com  Wed May 29 05:55:01 2013
From: rhattersley at gmail.com (Richard Hattersley)
Date: Wed, 29 May 2013 10:55:01 +0100
Subject: [Numpy-discussion] Parameterised dtypes
In-Reply-To: <CALmrCV1e7WAXiC1ytJV1hQhCCKTHd-1sKH3vE=PzhuUvrTf3BA@mail.gmail.com>
References: <CAP=RS9kCSm7343uFc1qYKT=HgHiO9sr9W8K-5gp_NFkM3G27Bw@mail.gmail.com>
	<CALmrCV1e7WAXiC1ytJV1hQhCCKTHd-1sKH3vE=PzhuUvrTf3BA@mail.gmail.com>
Message-ID: <CAP=RS9mDQCp3S8KYBSmqOTo=Gn8xgkP3u7btGh2Lx3mdfuWZ4g@mail.gmail.com>

Hi Andrew,

> Maybe a stupid question, but do you know a reference I could look at
> for the metadata and c_metadata fields you described?

Sorry ... no. I've not found anything. :-(

If I remember correctly, I got wind of the metadata aspect from the
mailing list discussions of datetime64. So for my current work I've
just been scratching around in the datetime64 code looking for example
usage.

Regards,
Richard


From cournape at gmail.com  Wed May 29 07:01:34 2013
From: cournape at gmail.com (David Cournapeau)
Date: Wed, 29 May 2013 12:01:34 +0100
Subject: [Numpy-discussion] Parameterised dtypes
In-Reply-To: <CAPJVwB=5ibVyk2cAoTSpy4fDwAbN71CYysmp4EpGJRaf_0SwSw@mail.gmail.com>
References: <CAP=RS9kCSm7343uFc1qYKT=HgHiO9sr9W8K-5gp_NFkM3G27Bw@mail.gmail.com>
	<CAPJVwB=5ibVyk2cAoTSpy4fDwAbN71CYysmp4EpGJRaf_0SwSw@mail.gmail.com>
Message-ID: <CAGY4rcXPNZjTOzCE_55Rg3qGCTX9WXgB3ijy5QWxDaFc9HueLQ@mail.gmail.com>

On Tue, May 28, 2013 at 9:06 PM, Nathaniel Smith <njs at pobox.com> wrote:
> On Fri, May 24, 2013 at 3:12 PM, Richard Hattersley
> <rhattersley at gmail.com> wrote:
>> Hi all,
>>
>> I'm in the process of defining some new dtypes to handle non-physical
>> calendars (such as the 360-day calendar used in the climate modelling
>> world). This is all going fine[*] so far, but I'd like to know a little bit
>> more about how much is ultimately possible.
>>
>> The PyArray_Descr members `metadata` and `c_metadata` allow run-time
>> parametrisation, but is it possible to hook into the dtype('...') parsing
>> mechanism to supply those parameters? Or is there some other dtype mechanism
>> for supplying parameters?
>>
>> As an example, would it be possible to supply month lengths?
>>>>> a = np.zeros(n, dtype='my_date[34,33,31,30,30,29,29,30,31,32,34,35]')
>>
>> Or is the intended use of parametrisation more like:
>>>>> weird = my_stuff.make_dtype([34,33,31,30,30,29,29,30,31,32,34,35])
>>>>> a = np.zeros(n, dtype=weird)
>
> I don't think there's any "intended use" really. AFAICT numpy was
> originally designed with the assumption that there were a fixed set of
> dtypes stateless dtypes, and then the ability to add new dtypes, the
> ability to add state (parametrize them), etc., have gradually been
> kluged in as needed to solve whatever immediate problem someone faced.
> E.g., dtype callbacks and ufuncs don't in general get access to the
> dtype object, so they can't access whatever parameters exist, and the
> builtin dtypes that *are* parametrized (strings, structs, etc.) all
> have special case code scattered all around numpy.
>
> You don't even need 'metadata' or 'c_metadata' -- this is Python, we
> already have a totally standard way to add new fields, just subclass
> the dumb thing. Instead we have this baffling system of 'NpyAuxData'
> which invents its own tiny little refcounted object system from
> scratch, and 'issubdtype' which invents its own way of representing
> inheritance hierarchies independent of Python's object system, and so
> forth.
>
> Anyway!
>
> 1) No, you can't hook into the dtype string parser. Though, are you
> sure you really want to? Surely it's nicer to use Python syntax
> instead of inventing a new syntax and then having to write a parser
> for it from scratch?
>
> 2) I have some vague plans worked out to fix all this so dtypes are
> just ordinary python objects, but haven't written it down yet due to a
> combination of lack of time to do so, and lack of anyone with time to
> actually implement the plan even if it were written down. I mention
> this just in case someone wants to volunteer, which would move it up
> my stack.

Nathan, will you be at scipy conference this year ? That's something
I'd like to improve/refactor myself as well.

I have not thought much about making dtype ordinary object yet, but is
what you had in mind 'mostly' backward incompatible ?

David


From jtaylor.debian at googlemail.com  Wed May 29 12:25:51 2013
From: jtaylor.debian at googlemail.com (Julian Taylor)
Date: Wed, 29 May 2013 18:25:51 +0200
Subject: [Numpy-discussion] quickselect
In-Reply-To: <CAMMTP+ArWn1hVswH6GXfJSbBtsAg3j9RZd2R52GzcjAeMjynEQ@mail.gmail.com>
References: <CAB6mnxJtcm_VdqgLtMtxszQ1+MUW6kvf2d82noLW=fuxOe8EsA@mail.gmail.com>
	<CAMMTP+ArWn1hVswH6GXfJSbBtsAg3j9RZd2R52GzcjAeMjynEQ@mail.gmail.com>
Message-ID: <51A62C0F.6040109@googlemail.com>

On 05/29/2013 06:12 AM, josef.pktd at gmail.com wrote:
> On Tue, May 28, 2013 at 6:31 PM, Charles R Harris
> <charlesr.harris at gmail.com> wrote:
>> Hi All,
>>
>> There is a PR adding quickselect to numpy as a function `partition`.
>> Comments on name and exposure in the numpy API are welcome.
>
> I think the name is fine. It's possible to get used to it.
>
> One possible use I can think of is if I want to sort afterwards only
> one of the tails (like largest 15%).
>
> Limitations:
>
> medium: like percentiles it would be nice to partition into more than
> two sets, if that can be done more efficiently than calling the
> function twice.

I'm not aware of an algorithm that is more efficient to generate 
multiple partitions opposed to partitioning twice.
Additional partitions can work on subranges so so smaller the remainign 
range the faster it gets.
I did a coarse benchmark with percentile partitioning multiple times 
instead of sorting once.
partition beats sorting even when you want to select 500 percentiles, so 
we probably can change it completely.
If people frequently select even more we could add a heuristic cut off 
to sort instead.

>
> major:
> it won't work to calculate the median or other quantiles since those interpolate
> np.median: "When N is even, it is the average of the two middle values
> of V_sorted"
>

this works fine, quickselect/partition moves the kth element into its 
final sorted order, so to get the median of even elements you can just 
partition twice, to get the two middle values. Then take the mean as usual.

This is already implemented in the pull request.
It includes the additional optimization that the second middle value is 
just a minimum of the second partition.



From josef.pktd at gmail.com  Wed May 29 15:19:57 2013
From: josef.pktd at gmail.com (josef.pktd at gmail.com)
Date: Wed, 29 May 2013 15:19:57 -0400
Subject: [Numpy-discussion] quickselect
In-Reply-To: <51A62C0F.6040109@googlemail.com>
References: <CAB6mnxJtcm_VdqgLtMtxszQ1+MUW6kvf2d82noLW=fuxOe8EsA@mail.gmail.com>
	<CAMMTP+ArWn1hVswH6GXfJSbBtsAg3j9RZd2R52GzcjAeMjynEQ@mail.gmail.com>
	<51A62C0F.6040109@googlemail.com>
Message-ID: <CAMMTP+D_hjjotGHcgG1PxFrS-yjEB2z5LXL=qYasi-E226NLZg@mail.gmail.com>

On Wed, May 29, 2013 at 12:25 PM, Julian Taylor
<jtaylor.debian at googlemail.com> wrote:
> On 05/29/2013 06:12 AM, josef.pktd at gmail.com wrote:
>>
>> On Tue, May 28, 2013 at 6:31 PM, Charles R Harris
>> <charlesr.harris at gmail.com> wrote:
>>>
>>> Hi All,
>>>
>>> There is a PR adding quickselect to numpy as a function `partition`.
>>> Comments on name and exposure in the numpy API are welcome.
>>
>>
>> I think the name is fine. It's possible to get used to it.
>>
>> One possible use I can think of is if I want to sort afterwards only
>> one of the tails (like largest 15%).
>>
>> Limitations:
>>
>> medium: like percentiles it would be nice to partition into more than
>> two sets, if that can be done more efficiently than calling the
>> function twice.
>
>
> I'm not aware of an algorithm that is more efficient to generate multiple
> partitions opposed to partitioning twice.
> Additional partitions can work on subranges so so smaller the remainign
> range the faster it gets.
> I did a coarse benchmark with percentile partitioning multiple times instead
> of sorting once.
> partition beats sorting even when you want to select 500 percentiles, so we
> probably can change it completely.
> If people frequently select even more we could add a heuristic cut off to
> sort instead.

I only know a few places in statistics where a medium large number of
percentiles or partitions are used, most of the time it's either just
a few or a full sort.
This would work well then.

Actually, I just thought about another application:
If I want to do a coarse quantization of an array by percentiles, then
I could partition it and assign categories, instead of doing a full
sort. The number of bins could be large in this case.

>
>
>>
>> major:
>> it won't work to calculate the median or other quantiles since those
>> interpolate
>> np.median: "When N is even, it is the average of the two middle values
>> of V_sorted"
>>
>
> this works fine, quickselect/partition moves the kth element into its final
> sorted order, so to get the median of even elements you can just partition
> twice, to get the two middle values. Then take the mean as usual.
>
> This is already implemented in the pull request.
> It includes the additional optimization that the second middle value is just
> a minimum of the second partition.
>

Thanks for the explanation. I was only looking for docstrings and
didn't see the implementation of median.
Using min on the second partitions looks like a nice trick, (one to remember).

As a future user, I think this sounds good, faster median and
percentiles and a nice new partition function.

Josef


From ndbecker2 at gmail.com  Thu May 30 07:44:19 2013
From: ndbecker2 at gmail.com (Neal Becker)
Date: Thu, 30 May 2013 07:44:19 -0400
Subject: [Numpy-discussion] forwarded article (embracing tensors)
Message-ID: <ko7e2e$9uj$1@ger.gmane.org>

I thought the topic of this article might be of interest here:

https://groups.google.com/forum/?fromgroups#!topic/julia-dev/GAdcYzmibyo



From msuzen at gmail.com  Thu May 30 08:37:09 2013
From: msuzen at gmail.com (Suzen, Mehmet)
Date: Thu, 30 May 2013 14:37:09 +0200
Subject: [Numpy-discussion] forwarded article (embracing tensors)
In-Reply-To: <ko7e2e$9uj$1@ger.gmane.org>
References: <ko7e2e$9uj$1@ger.gmane.org>
Message-ID: <CAPtbhHwfoQ-4FwEcQ2xOuNRjSGsJMw_Ov4icpVDVS-MS-cCKJA@mail.gmail.com>

On 30 May 2013 13:44, Neal Becker <ndbecker2 at gmail.com> wrote:
> I thought the topic of this article might be of interest here:
>
> https://groups.google.com/forum/?fromgroups#!topic/julia-dev/GAdcYzmibyo
>

>"I think Julia has great potential, and can, unlike Matlab, rid itself of its tunnel vision on matrices and embrace tensors from the ground >up"

Trying to discredit Matlab for tensors? I appreciate the enthusiasm
but consider that there are high quality extensions before trying to
discredit other people's work:
http://www.sandia.gov/~tgkolda/TensorToolbox/index-2.5.html


From thomas.robitaille at gmail.com  Thu May 30 15:28:42 2013
From: thomas.robitaille at gmail.com (Thomas Robitaille)
Date: Thu, 30 May 2013 21:28:42 +0200
Subject: [Numpy-discussion] __array_priority__ ignored if __array__ is
	present
In-Reply-To: <CADKKbtifSh4XKHRb5UxF993gsoK_JJZkPye4VDwzue8EG14a-A@mail.gmail.com>
References: <CAGMHX_3Xcn6_NmG1u7t9UVYN9CWN2D3Uj_aBKcgGZG5i9=-qew@mail.gmail.com>
	<CADKKbtifSh4XKHRb5UxF993gsoK_JJZkPye4VDwzue8EG14a-A@mail.gmail.com>
Message-ID: <CAGMHX_23R_C2Z7iKKYXDjdxkbaQkJpB0VSw9=sDL6DDPipFtxg@mail.gmail.com>

Hi Frederic,

On 16 May 2013 15:58, Fr?d?ric Bastien <nouiz at nouiz.org> wrote:
> I looked yesterday rapidly in the code and didn't find the reason (I don't
> know it well, that is probably why).
>
> But last night I think of one possible cause. I found this code 2 times in
> the file core/src/umath/ufunc_object.c:
>
>     if (nin == 2 && nout == 1 && dtypes[1]->type_num == NPY_OBJECT) {
>         PyObject *_obj = PyTuple_GET_ITEM(args, 1);
>         if (!PyArray_CheckExact(_obj)) {
>             double self_prio, other_prio;
>             self_prio = PyArray_GetPriority(PyTuple_GET_ITEM(args, 0),
>
> NPY_SCALAR_PRIORITY);
>             other_prio = PyArray_GetPriority(_obj, NPY_SCALAR_PRIORITY);
>             if (self_prio < other_prio &&
>                             _has_reflected_op(_obj, ufunc_name)) {
>                 retval = -2;
>         goto fail;
>             }
>     }
>     }
>
> It is this code that will call _has_reflected_op() function. The if
> condition is:
>
> dtypes[1]->type_num == NPY_OBJECT
>
>
> I wouldn't be surprised if dtypes[1] isn't NPY_OBJECT when you implement
> __array__.
>
> dtypes is set with those line:
>
>     retval = ufunc->type_resolver(ufunc, casting,
>                             op, type_tup, dtypes);

Thanks for looking into this - should this be considered a bug?

Tom

>
>
> HTH
>
> Fred
>
>
>
> On Thu, May 16, 2013 at 9:19 AM, Thomas Robitaille
> <thomas.robitaille at gmail.com> wrote:
>>
>> Hi everyone,
>>
>> (this was posted as part of another topic, but since it was unrelated,
>> I'm reposting as a separate thread)
>>
>> I've also been having issues with __array_priority__ - the following
>> code behaves differently for __mul__ and __rmul__:
>>
>> """
>> import numpy as np
>>
>>
>> class TestClass(object):
>>
>>     def __init__(self, input_array):
>>         self.array = input_array
>>
>>     def __mul__(self, other):
>>         print "Called __mul__"
>>
>>     def __rmul__(self, other):
>>         print "Called __rmul__"
>>
>>     def __array_wrap__(self, out_arr, context=None):
>>         print "Called __array_wrap__"
>>         return TestClass(out_arr)
>>
>>     def __array__(self):
>>         print "Called __array__"
>>         return np.array(self.array)
>> """
>>
>> with output:
>>
>> """
>> In [7]: a = TestClass([1,2,3])
>>
>> In [8]: print type(np.array([1,2,3]) * a)
>> Called __array__
>> Called __array_wrap__
>> <class '__main__.TestClass'>
>>
>> In [9]: print type(a * np.array([1,2,3]))
>> Called __mul__
>> <type 'NoneType'>
>> """
>>
>> Is this also an oversight? I opened a ticket for it a little while ago:
>>
>> https://github.com/numpy/numpy/issues/3164
>>
>> Any ideas?
>>
>> Thanks!
>> Tom
>> _______________________________________________
>> NumPy-Discussion mailing list
>> NumPy-Discussion at scipy.org
>> http://mail.scipy.org/mailman/listinfo/numpy-discussion
>
>
>
> _______________________________________________
> NumPy-Discussion mailing list
> NumPy-Discussion at scipy.org
> http://mail.scipy.org/mailman/listinfo/numpy-discussion
>


From aldcroft at head.cfa.harvard.edu  Thu May 30 16:21:36 2013
From: aldcroft at head.cfa.harvard.edu (Aldcroft, Thomas)
Date: Thu, 30 May 2013 16:21:36 -0400
Subject: [Numpy-discussion] __getitem__ and creating an array of objects
Message-ID: <CAMtEP6wsY1GUYTL_z-_Yk5T_hKkeHNb_f1HzaeMN41_ALNJmmg@mail.gmail.com>

I'm seeing some behavior that I can't understand when creating a numpy
array of Python objects.  Basically it seems that np.array() is calling the
object __getitem__ method for one object class but not another class, and I
can't understand the difference.

Here is an example, starting with a simple class where __getitem__ does NOT
get called:

>>> class Foo(object):
...     def __getitem__(self, item):
...         return 1/0
...     def __len__(self, item):
...         return 1
...
>>> f = Foo()
>>> x = np.array([f])
>>> x
array([<__main__.Foo object at 0x11a1ff10>], dtype=object)

Now move to the complicated class (astropy.time.Time) which has a lot of
stuff in it (including __new__, __init__, and __len__ methods, but
initially no __getitem__), but is otherwise an ordinary class derived from
object. This works as expected:

>>> from astropy.time import Time
>>> t = Time('2001-01-01', scale='utc')
>>> x = np.array([t])
>>> x
array([2001-01-01 00:00:00.000], dtype=object)

Now inject a __getitem__ that will fail and try again:

>>> Time.__getitem__ = lambda self, item: 1 / 0
>>> x = np.array([t])
ERROR: ZeroDivisionError: integer division or modulo by zero

Any ideas on what is driving this difference in behavior?  BTW, the value
of item in the latter case is 0.

Thanks,
Tom
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20130530/b6e4c54d/attachment.html>

From nouiz at nouiz.org  Thu May 30 16:26:35 2013
From: nouiz at nouiz.org (=?ISO-8859-1?Q?Fr=E9d=E9ric_Bastien?=)
Date: Thu, 30 May 2013 16:26:35 -0400
Subject: [Numpy-discussion] __array_priority__ ignored if __array__ is
	present
In-Reply-To: <CAGMHX_23R_C2Z7iKKYXDjdxkbaQkJpB0VSw9=sDL6DDPipFtxg@mail.gmail.com>
References: <CAGMHX_3Xcn6_NmG1u7t9UVYN9CWN2D3Uj_aBKcgGZG5i9=-qew@mail.gmail.com>
	<CADKKbtifSh4XKHRb5UxF993gsoK_JJZkPye4VDwzue8EG14a-A@mail.gmail.com>
	<CAGMHX_23R_C2Z7iKKYXDjdxkbaQkJpB0VSw9=sDL6DDPipFtxg@mail.gmail.com>
Message-ID: <CADKKbth-Tf1qorHB2KF5=iV2vOk35GRZip1zUha58N=c6D12sw@mail.gmail.com>

I think so.

Changing the order between "np.array([1,2,3]) * a" and "a *
np.array([1,2,3])" should return the same type I think, specificaly when
array_priority is defined.

Fred


On Thu, May 30, 2013 at 3:28 PM, Thomas Robitaille <
thomas.robitaille at gmail.com> wrote:

> Hi Frederic,
>
> On 16 May 2013 15:58, Fr?d?ric Bastien <nouiz at nouiz.org> wrote:
> > I looked yesterday rapidly in the code and didn't find the reason (I
> don't
> > know it well, that is probably why).
> >
> > But last night I think of one possible cause. I found this code 2 times
> in
> > the file core/src/umath/ufunc_object.c:
> >
> >     if (nin == 2 && nout == 1 && dtypes[1]->type_num == NPY_OBJECT) {
> >         PyObject *_obj = PyTuple_GET_ITEM(args, 1);
> >         if (!PyArray_CheckExact(_obj)) {
> >             double self_prio, other_prio;
> >             self_prio = PyArray_GetPriority(PyTuple_GET_ITEM(args, 0),
> >
> > NPY_SCALAR_PRIORITY);
> >             other_prio = PyArray_GetPriority(_obj, NPY_SCALAR_PRIORITY);
> >             if (self_prio < other_prio &&
> >                             _has_reflected_op(_obj, ufunc_name)) {
> >                 retval = -2;
> >         goto fail;
> >             }
> >     }
> >     }
> >
> > It is this code that will call _has_reflected_op() function. The if
> > condition is:
> >
> > dtypes[1]->type_num == NPY_OBJECT
> >
> >
> > I wouldn't be surprised if dtypes[1] isn't NPY_OBJECT when you implement
> > __array__.
> >
> > dtypes is set with those line:
> >
> >     retval = ufunc->type_resolver(ufunc, casting,
> >                             op, type_tup, dtypes);
>
> Thanks for looking into this - should this be considered a bug?
>
> Tom
>
> >
> >
> > HTH
> >
> > Fred
> >
> >
> >
> > On Thu, May 16, 2013 at 9:19 AM, Thomas Robitaille
> > <thomas.robitaille at gmail.com> wrote:
> >>
> >> Hi everyone,
> >>
> >> (this was posted as part of another topic, but since it was unrelated,
> >> I'm reposting as a separate thread)
> >>
> >> I've also been having issues with __array_priority__ - the following
> >> code behaves differently for __mul__ and __rmul__:
> >>
> >> """
> >> import numpy as np
> >>
> >>
> >> class TestClass(object):
> >>
> >>     def __init__(self, input_array):
> >>         self.array = input_array
> >>
> >>     def __mul__(self, other):
> >>         print "Called __mul__"
> >>
> >>     def __rmul__(self, other):
> >>         print "Called __rmul__"
> >>
> >>     def __array_wrap__(self, out_arr, context=None):
> >>         print "Called __array_wrap__"
> >>         return TestClass(out_arr)
> >>
> >>     def __array__(self):
> >>         print "Called __array__"
> >>         return np.array(self.array)
> >> """
> >>
> >> with output:
> >>
> >> """
> >> In [7]: a = TestClass([1,2,3])
> >>
> >> In [8]: print type(np.array([1,2,3]) * a)
> >> Called __array__
> >> Called __array_wrap__
> >> <class '__main__.TestClass'>
> >>
> >> In [9]: print type(a * np.array([1,2,3]))
> >> Called __mul__
> >> <type 'NoneType'>
> >> """
> >>
> >> Is this also an oversight? I opened a ticket for it a little while ago:
> >>
> >> https://github.com/numpy/numpy/issues/3164
> >>
> >> Any ideas?
> >>
> >> Thanks!
> >> Tom
> >> _______________________________________________
> >> NumPy-Discussion mailing list
> >> NumPy-Discussion at scipy.org
> >> http://mail.scipy.org/mailman/listinfo/numpy-discussion
> >
> >
> >
> > _______________________________________________
> > NumPy-Discussion mailing list
> > NumPy-Discussion at scipy.org
> > http://mail.scipy.org/mailman/listinfo/numpy-discussion
> >
> _______________________________________________
> NumPy-Discussion mailing list
> NumPy-Discussion at scipy.org
> http://mail.scipy.org/mailman/listinfo/numpy-discussion
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20130530/fa6e1453/attachment.html>

From robert.kern at gmail.com  Thu May 30 16:27:20 2013
From: robert.kern at gmail.com (Robert Kern)
Date: Thu, 30 May 2013 21:27:20 +0100
Subject: [Numpy-discussion] __getitem__ and creating an array of objects
In-Reply-To: <CAMtEP6wsY1GUYTL_z-_Yk5T_hKkeHNb_f1HzaeMN41_ALNJmmg@mail.gmail.com>
References: <CAMtEP6wsY1GUYTL_z-_Yk5T_hKkeHNb_f1HzaeMN41_ALNJmmg@mail.gmail.com>
Message-ID: <CAF6FJiuJLiF27_PSB+bn=8hpXM_A0gQO-cXuyZputLa=u174Tg@mail.gmail.com>

On Thu, May 30, 2013 at 9:21 PM, Aldcroft, Thomas
<aldcroft at head.cfa.harvard.edu> wrote:
> I'm seeing some behavior that I can't understand when creating a numpy array
> of Python objects.  Basically it seems that np.array() is calling the object
> __getitem__ method for one object class but not another class, and I can't
> understand the difference.
>
> Here is an example, starting with a simple class where __getitem__ does NOT
> get called:
>
>>>> class Foo(object):
> ...     def __getitem__(self, item):
> ...         return 1/0
> ...     def __len__(self, item):
> ...         return 1
> ...
>>>> f = Foo()
>>>> x = np.array([f])
>>>> x
> array([<__main__.Foo object at 0x11a1ff10>], dtype=object)
>
> Now move to the complicated class (astropy.time.Time) which has a lot of
> stuff in it (including __new__, __init__, and __len__ methods, but initially
> no __getitem__), but is otherwise an ordinary class derived from object.
> This works as expected:
>
>>>> from astropy.time import Time
>>>> t = Time('2001-01-01', scale='utc')
>>>> x = np.array([t])
>>>> x
> array([2001-01-01 00:00:00.000], dtype=object)
>
> Now inject a __getitem__ that will fail and try again:
>
>>>> Time.__getitem__ = lambda self, item: 1 / 0
>>>> x = np.array([t])
> ERROR: ZeroDivisionError: integer division or modulo by zero
>
> Any ideas on what is driving this difference in behavior?  BTW, the value of
> item in the latter case is 0.

What is len(t) in the latter case?

--
Robert Kern


From aldcroft at head.cfa.harvard.edu  Thu May 30 16:58:54 2013
From: aldcroft at head.cfa.harvard.edu (Aldcroft, Thomas)
Date: Thu, 30 May 2013 16:58:54 -0400
Subject: [Numpy-discussion] __getitem__ and creating an array of objects
In-Reply-To: <CAF6FJiuJLiF27_PSB+bn=8hpXM_A0gQO-cXuyZputLa=u174Tg@mail.gmail.com>
References: <CAMtEP6wsY1GUYTL_z-_Yk5T_hKkeHNb_f1HzaeMN41_ALNJmmg@mail.gmail.com>
	<CAF6FJiuJLiF27_PSB+bn=8hpXM_A0gQO-cXuyZputLa=u174Tg@mail.gmail.com>
Message-ID: <CAMtEP6xt7uJPx1P_wttbHfeJ2aqu3qY78azH7u0FmU2SMXsNqw@mail.gmail.com>

On Thu, May 30, 2013 at 4:27 PM, Robert Kern <robert.kern at gmail.com> wrote:

> On Thu, May 30, 2013 at 9:21 PM, Aldcroft, Thomas
> <aldcroft at head.cfa.harvard.edu> wrote:
> > I'm seeing some behavior that I can't understand when creating a numpy
> array
> > of Python objects.  Basically it seems that np.array() is calling the
> object
> > __getitem__ method for one object class but not another class, and I
> can't
> > understand the difference.
> >
> > Here is an example, starting with a simple class where __getitem__ does
> NOT
> > get called:
> >
> >>>> class Foo(object):
> > ...     def __getitem__(self, item):
> > ...         return 1/0
> > ...     def __len__(self, item):
> > ...         return 1
> > ...
> >>>> f = Foo()
> >>>> x = np.array([f])
> >>>> x
> > array([<__main__.Foo object at 0x11a1ff10>], dtype=object)
> >
> > Now move to the complicated class (astropy.time.Time) which has a lot of
> > stuff in it (including __new__, __init__, and __len__ methods, but
> initially
> > no __getitem__), but is otherwise an ordinary class derived from object.
> > This works as expected:
> >
> >>>> from astropy.time import Time
> >>>> t = Time('2001-01-01', scale='utc')
> >>>> x = np.array([t])
> >>>> x
> > array([2001-01-01 00:00:00.000], dtype=object)
> >
> > Now inject a __getitem__ that will fail and try again:
> >
> >>>> Time.__getitem__ = lambda self, item: 1 / 0
> >>>> x = np.array([t])
> > ERROR: ZeroDivisionError: integer division or modulo by zero
> >
> > Any ideas on what is driving this difference in behavior?  BTW, the
> value of
> > item in the latter case is 0.
>
> What is len(t) in the latter case?
>

>>> len(t)
1

Prompted by this question I looked again at the Foo() definition and see
that I had copy/pasted the definition of __len__ from __getitem__ and
forgot to remove the `item` arg.  As written `len(Foo())` would fail.
 Interestingly, once `Foo.__len__` is valid and succeeds, then
`f.__getitem__` does indeed get called and `np.array([f])` fails in the
same way.

Sorry for the noise, though this is still just slightly curious to me.  I
guess internally maybe there is try/except block that is trying to get a
len and if that fails then it moves on.

- Tom



>
> --
> Robert Kern
> _______________________________________________
> NumPy-Discussion mailing list
> NumPy-Discussion at scipy.org
> http://mail.scipy.org/mailman/listinfo/numpy-discussion
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20130530/60dd1794/attachment.html>

From aldcroft at head.cfa.harvard.edu  Thu May 30 17:31:49 2013
From: aldcroft at head.cfa.harvard.edu (Aldcroft, Thomas)
Date: Thu, 30 May 2013 17:31:49 -0400
Subject: [Numpy-discussion] __getitem__ and creating an array of objects
In-Reply-To: <CAMtEP6xt7uJPx1P_wttbHfeJ2aqu3qY78azH7u0FmU2SMXsNqw@mail.gmail.com>
References: <CAMtEP6wsY1GUYTL_z-_Yk5T_hKkeHNb_f1HzaeMN41_ALNJmmg@mail.gmail.com>
	<CAF6FJiuJLiF27_PSB+bn=8hpXM_A0gQO-cXuyZputLa=u174Tg@mail.gmail.com>
	<CAMtEP6xt7uJPx1P_wttbHfeJ2aqu3qY78azH7u0FmU2SMXsNqw@mail.gmail.com>
Message-ID: <CAMtEP6wrJdCgiefKp54B40kqNGoi-eXMqyKUqA9hiRD3ZJyqhw@mail.gmail.com>

On Thu, May 30, 2013 at 4:58 PM, Aldcroft, Thomas <
aldcroft at head.cfa.harvard.edu> wrote:

>
>
>
> On Thu, May 30, 2013 at 4:27 PM, Robert Kern <robert.kern at gmail.com>wrote:
>
>> On Thu, May 30, 2013 at 9:21 PM, Aldcroft, Thomas
>> <aldcroft at head.cfa.harvard.edu> wrote:
>> > I'm seeing some behavior that I can't understand when creating a numpy
>> array
>> > of Python objects.  Basically it seems that np.array() is calling the
>> object
>> > __getitem__ method for one object class but not another class, and I
>> can't
>> > understand the difference.
>> >
>> > Here is an example, starting with a simple class where __getitem__ does
>> NOT
>> > get called:
>> >
>> >>>> class Foo(object):
>> > ...     def __getitem__(self, item):
>> > ...         return 1/0
>> > ...     def __len__(self, item):
>> > ...         return 1
>> > ...
>> >>>> f = Foo()
>> >>>> x = np.array([f])
>> >>>> x
>> > array([<__main__.Foo object at 0x11a1ff10>], dtype=object)
>> >
>> > Now move to the complicated class (astropy.time.Time) which has a lot of
>> > stuff in it (including __new__, __init__, and __len__ methods, but
>> initially
>> > no __getitem__), but is otherwise an ordinary class derived from object.
>> > This works as expected:
>> >
>> >>>> from astropy.time import Time
>> >>>> t = Time('2001-01-01', scale='utc')
>> >>>> x = np.array([t])
>> >>>> x
>> > array([2001-01-01 00:00:00.000], dtype=object)
>> >
>> > Now inject a __getitem__ that will fail and try again:
>> >
>> >>>> Time.__getitem__ = lambda self, item: 1 / 0
>> >>>> x = np.array([t])
>> > ERROR: ZeroDivisionError: integer division or modulo by zero
>> >
>> > Any ideas on what is driving this difference in behavior?  BTW, the
>> value of
>> > item in the latter case is 0.
>>
>> What is len(t) in the latter case?
>>
>
> >>> len(t)
> 1
>
> Prompted by this question I looked again at the Foo() definition and see
> that I had copy/pasted the definition of __len__ from __getitem__ and
> forgot to remove the `item` arg.  As written `len(Foo())` would fail.
>  Interestingly, once `Foo.__len__` is valid and succeeds, then
> `f.__getitem__` does indeed get called and `np.array([f])` fails in the
> same way.
>
> Sorry for the noise, though this is still just slightly curious to me.  I
> guess internally maybe there is try/except block that is trying to get a
> len and if that fails then it moves on.
>


Now I realize what seemed curious.  Here is a related example which shows
that when initializing a numpy array of objects where __getitem__ and
__len__ exist, np.array introspects the object item values for item in
range(len(object)) and appears to replace the input object with an ndarray
of the object values.

>>> class Foo(object):
...     attr = 'hello'
...     def __getitem__(self, item):
...         return item
...     def __len__(self):
...         return 5

>>> f = Foo()
>>> f.attr
'hello'

>>> x = np.array([f, f], dtype=object)
>>> x
array([[0, 1, 2, 3, 4],
       [0, 1, 2, 3, 4]], dtype=object)
>>> x[0]
array([0, 1, 2, 3, 4], dtype=object)
>>> type(x[0])
<type 'numpy.ndarray'>
>>> x[0].attr
Traceback (most recent call last):
  File "<ipython-input-16-7b865674a0b6>", line 1, in <module>
    x[0].attr
AttributeError: 'numpy.ndarray' object has no attribute 'attr'

The actual objects that I passed in seem to be lost, which is not the
behavior I expected.

Thanks,
Tom



> - Tom
>
>
>
>>
>> --
>> Robert Kern
>> _______________________________________________
>> NumPy-Discussion mailing list
>> NumPy-Discussion at scipy.org
>> http://mail.scipy.org/mailman/listinfo/numpy-discussion
>>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20130530/02a4b3a1/attachment.html>

From robert.kern at gmail.com  Thu May 30 17:47:09 2013
From: robert.kern at gmail.com (Robert Kern)
Date: Thu, 30 May 2013 22:47:09 +0100
Subject: [Numpy-discussion] __getitem__ and creating an array of objects
In-Reply-To: <CAMtEP6wrJdCgiefKp54B40kqNGoi-eXMqyKUqA9hiRD3ZJyqhw@mail.gmail.com>
References: <CAMtEP6wsY1GUYTL_z-_Yk5T_hKkeHNb_f1HzaeMN41_ALNJmmg@mail.gmail.com>
	<CAF6FJiuJLiF27_PSB+bn=8hpXM_A0gQO-cXuyZputLa=u174Tg@mail.gmail.com>
	<CAMtEP6xt7uJPx1P_wttbHfeJ2aqu3qY78azH7u0FmU2SMXsNqw@mail.gmail.com>
	<CAMtEP6wrJdCgiefKp54B40kqNGoi-eXMqyKUqA9hiRD3ZJyqhw@mail.gmail.com>
Message-ID: <CAF6FJitFwWK0UMaPZz-Kx2bi5dZjM=b7F5JEoNA+42P5K86Knw@mail.gmail.com>

On Thu, May 30, 2013 at 10:31 PM, Aldcroft, Thomas
<aldcroft at head.cfa.harvard.edu> wrote:

> Now I realize what seemed curious.  Here is a related example which shows
> that when initializing a numpy array of objects where __getitem__ and
> __len__ exist, np.array introspects the object item values for item in
> range(len(object)) and appears to replace the input object with an ndarray
> of the object values.
>
>>>> class Foo(object):
> ...     attr = 'hello'
> ...     def __getitem__(self, item):
> ...         return item
> ...     def __len__(self):
> ...         return 5
>
>>>> f = Foo()
>>>> f.attr
> 'hello'
>
>>>> x = np.array([f, f], dtype=object)
>>>> x
> array([[0, 1, 2, 3, 4],
>        [0, 1, 2, 3, 4]], dtype=object)
>>>> x[0]
> array([0, 1, 2, 3, 4], dtype=object)
>>>> type(x[0])
> <type 'numpy.ndarray'>
>>>> x[0].attr
> Traceback (most recent call last):
>   File "<ipython-input-16-7b865674a0b6>", line 1, in <module>
>     x[0].attr
> AttributeError: 'numpy.ndarray' object has no attribute 'attr'
>
> The actual objects that I passed in seem to be lost, which is not the
> behavior I expected.

By defining __len__ and __getitem__, you are making your objects look
like containers that array() should traverse down into, like a list,
rather than atomic items like a float. array() has to do a lot of
magic to determine what shape of an array you actually want. If you
have objects that look like containers but you don't want them to be
the atomic items, then you should first construct an empty object
array of the right shape, then populate it using slice assignment.

--
Robert Kern


From sebastian at sipsolutions.net  Fri May 31 14:08:13 2013
From: sebastian at sipsolutions.net (Sebastian Berg)
Date: Fri, 31 May 2013 20:08:13 +0200
Subject: [Numpy-discussion] Effect of deprecating non-integers for integer
	function arguments
Message-ID: <1370023693.2584.25.camel@sebastian-laptop>

Hi,

the current numpy master has deprecated non-integers for the use of
indexing (not-fancy yet). However I think this should be moved further
down in the numpy machinery which means that the conversion utils
provided by numpy would generally raise warnings for non-integers.

This means that for most numpy functions such as reshape, etc. the use
of non-integers for arguments that are naturally integers is deprecated,
which may also affect third party code in principle.

Testing this against current SciPy appearently causes 56 failures (with
deprecation warnings being raised). Are other project similarly affect
by this change?

The branch implementing this can be found in the PR
https://github.com/numpy/numpy/pull/3243

Regards,

Sebastian



From josef.pktd at gmail.com  Fri May 31 16:32:20 2013
From: josef.pktd at gmail.com (josef.pktd at gmail.com)
Date: Fri, 31 May 2013 16:32:20 -0400
Subject: [Numpy-discussion] Effect of deprecating non-integers for
 integer function arguments
In-Reply-To: <1370023693.2584.25.camel@sebastian-laptop>
References: <1370023693.2584.25.camel@sebastian-laptop>
Message-ID: <CAMMTP+CqBYnXEiiOYQgufD-L9mdqJexEWmgOxfH7=JCENh-07A@mail.gmail.com>

On Fri, May 31, 2013 at 2:08 PM, Sebastian Berg
<sebastian at sipsolutions.net> wrote:
> Hi,
>
> the current numpy master has deprecated non-integers for the use of
> indexing (not-fancy yet). However I think this should be moved further
> down in the numpy machinery which means that the conversion utils
> provided by numpy would generally raise warnings for non-integers.
>
> This means that for most numpy functions such as reshape, etc. the use
> of non-integers for arguments that are naturally integers is deprecated,
> which may also affect third party code in principle.
>
> Testing this against current SciPy appearently causes 56 failures (with
> deprecation warnings being raised). Are other project similarly affect
> by this change?

I haven't checked for statsmodels, but I would consider it a bug on
our side if we have implicit conversion to integers. We might still
have some left in older code but changed already most of them.
I just wrote code like this (upp are quantiles in (0, 1))

m_upp = [x[:int(np.round(upp_ * nobs))].mean(0) for upp_ in upp]

I don't think we use non-integers for reshape.

Josef

>
> The branch implementing this can be found in the PR
> https://github.com/numpy/numpy/pull/3243
>
> Regards,
>
> Sebastian
>
> _______________________________________________
> NumPy-Discussion mailing list
> NumPy-Discussion at scipy.org
> http://mail.scipy.org/mailman/listinfo/numpy-discussion


From albert.kottke at gmail.com  Fri May 31 17:08:11 2013
From: albert.kottke at gmail.com (Albert Kottke)
Date: Fri, 31 May 2013 14:08:11 -0700
Subject: [Numpy-discussion] genfromtxt() skips comments
Message-ID: <CAKoEZq_gMXdQ+n7iMozCUjhT-oY3hWycND_zeyFM5bK-uDBfjQ@mail.gmail.com>

I noticed that genfromtxt() did not skip comments if the keyword names is
not True. If names is True, then genfromtxt() would take the first line as
the names. I am proposing a fix to genfromtxt that skips all of the
comments in a file, and potentially using the last comment line for names.
This will allow reading files with and without comments and/or names.

The difference is here:
https://github.com/arkottke/numpy/compare/my-genfromtxt

Albert

p.s. insert some disclaimer about my first pull request
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20130531/7f895a1d/attachment.html>

From ben.root at ou.edu  Fri May 31 17:14:40 2013
From: ben.root at ou.edu (Benjamin Root)
Date: Fri, 31 May 2013 17:14:40 -0400
Subject: [Numpy-discussion] genfromtxt() skips comments
In-Reply-To: <CAKoEZq_gMXdQ+n7iMozCUjhT-oY3hWycND_zeyFM5bK-uDBfjQ@mail.gmail.com>
References: <CAKoEZq_gMXdQ+n7iMozCUjhT-oY3hWycND_zeyFM5bK-uDBfjQ@mail.gmail.com>
Message-ID: <CANNq6FmF=PHLpgK8sPK_SrNZG82RkfskOaAZaaEg7gqUkJ8+Mg@mail.gmail.com>

On Fri, May 31, 2013 at 5:08 PM, Albert Kottke <albert.kottke at gmail.com>wrote:

> I noticed that genfromtxt() did not skip comments if the keyword names is
> not True. If names is True, then genfromtxt() would take the first line as
> the names. I am proposing a fix to genfromtxt that skips all of the
> comments in a file, and potentially using the last comment line for names.
> This will allow reading files with and without comments and/or names.
>
> The difference is here:
> https://github.com/arkottke/numpy/compare/my-genfromtxt
>
>
Careful with semantics here.  First off, using the last comment line as the
source for names might initially make sense, except when there are comments
within the data file.  I would suggest going for "last comment line before
the first line of data".  Second, sometimes the names come from an
un-commented first line, but comments are still used within the file
elsewhere.

Just some food for thought.  I don't know if the current design is best or
not.

Ben Root
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20130531/2591f8ae/attachment.html>

From albert.kottke at gmail.com  Fri May 31 17:30:31 2013
From: albert.kottke at gmail.com (Albert Kottke)
Date: Fri, 31 May 2013 14:30:31 -0700
Subject: [Numpy-discussion] genfromtxt() skips comments
In-Reply-To: <CANNq6FmF=PHLpgK8sPK_SrNZG82RkfskOaAZaaEg7gqUkJ8+Mg@mail.gmail.com>
References: <CAKoEZq_gMXdQ+n7iMozCUjhT-oY3hWycND_zeyFM5bK-uDBfjQ@mail.gmail.com>
	<CANNq6FmF=PHLpgK8sPK_SrNZG82RkfskOaAZaaEg7gqUkJ8+Mg@mail.gmail.com>
Message-ID: <CAKoEZq_70Y_HFLzw2MBmS_o5NtTnc=vV+ddXZZ3jvb5p53N8Eg@mail.gmail.com>

I agree that "last comment line before the first line of data" is more
descriptive.

Regarding the location of the names. I thought taking it from the last
comment line before the first line of data made sense because it would
permit reading of just the data with np.loadtxt(), but also permit creating
records with np.recfromtxt(). It would also be good to consider other
implementations. For example, pandas and R both use names without a comment
character.

Albert


On Fri, May 31, 2013 at 2:14 PM, Benjamin Root <ben.root at ou.edu> wrote:

>
> On Fri, May 31, 2013 at 5:08 PM, Albert Kottke <albert.kottke at gmail.com>wrote:
>
>> I noticed that genfromtxt() did not skip comments if the keyword names is
>> not True. If names is True, then genfromtxt() would take the first line as
>> the names. I am proposing a fix to genfromtxt that skips all of the
>> comments in a file, and potentially using the last comment line for names.
>> This will allow reading files with and without comments and/or names.
>>
>> The difference is here:
>> https://github.com/arkottke/numpy/compare/my-genfromtxt
>>
>>
> Careful with semantics here.  First off, using the last comment line as
> the source for names might initially make sense, except when there are
> comments within the data file.  I would suggest going for "last comment
> line before the first line of data".  Second, sometimes the names come from
> an un-commented first line, but comments are still used within the file
> elsewhere.
>
> Just some food for thought.  I don't know if the current design is best or
> not.
>
> Ben Root
>
> _______________________________________________
> NumPy-Discussion mailing list
> NumPy-Discussion at scipy.org
> http://mail.scipy.org/mailman/listinfo/numpy-discussion
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20130531/f828683e/attachment.html>

From charlesr.harris at gmail.com  Fri May 31 17:47:48 2013
From: charlesr.harris at gmail.com (Charles R Harris)
Date: Fri, 31 May 2013 15:47:48 -0600
Subject: [Numpy-discussion] 1.8 release update
Message-ID: <CAB6mnxJy=hOBvowjOg-TaDgXyGmt8X1HM-Pjq94PZ5NVB8huJA@mail.gmail.com>

Hi All,

Most of the PR's that were mentioned as desirable for the 1.8 release have
been merged, or look to be merged in the next week or two. The current list
of blockers<https://github.com/numpy/numpy/issues?milestone=1&page=1&state=open>doesn't
look to severe to me but I suspect that it is incomplete. The major
outstanding issue I see is datetime and I'd like to get a small group
together to work that out. As a start I think such a group should include
Christopher Barker and Wes McKinney. Suggestions for other folks, or even
volunteers, to be part of such a group are welcome.

A lot of stuff has piled up over the last year for inclusion in the 1.8
release, and I'm sure some bugs and regressions  have crept in along with
the new code. On that account a lengthy wringing out period is probably
going to be needed, so the sooner we can get the 1.8 branch tagged the
better. I'd like to shoot for getting that done in 2-3 weeks. If there look
to be difficult issues remaining, perhaps they can be worked out at
scipy2013 if there is enough interest.

Thoughts?

Chuck
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20130531/7da07b3e/attachment.html>

From pgmdevlist at gmail.com  Fri May 31 18:02:19 2013
From: pgmdevlist at gmail.com (Pierre GM)
Date: Sat, 1 Jun 2013 00:02:19 +0200
Subject: [Numpy-discussion] =?utf-8?q?_Re=3A__genfromtxt=28=29_skips_comme?=
 =?utf-8?q?nts?=
In-Reply-To: <CAKoEZq_gMXdQ+n7iMozCUjhT-oY3hWycND_zeyFM5bK-uDBfjQ@mail.gmail.com>
References: <CAKoEZq_gMXdQ+n7iMozCUjhT-oY3hWycND_zeyFM5bK-uDBfjQ@mail.gmail.com>
Message-ID: <etPan.51a91deb.6b8b4567.c85c@platypus.local>

On May 31, 2013 at 23:08:18 , Albert Kottke (albert.kottke at gmail.com) wrote:
I noticed that genfromtxt() did not skip comments if the keyword names is not True. If names is True, then genfromtxt() would take the first line as the names. I am proposing a fix to genfromtxt that skips all of the comments in a file, and potentially using the last comment line for names.
I'm quite surprised, as comments are already skipped in my standard numpy version (1.7.0). For example:
>>> S=StringIO("!blah\n!blah\n!blah\n!A:B:C\n1:2:3\n4:5:6\n")
>>> np.genfromtxt(S, delimiter=":", comments="!", names=("A","B","C"))
Works as expected, even when using the default `names=None`.
Comments are taken care of with the `split_line` function (an instance of `_iotools.LineSplitter`).


From albert.kottke at gmail.com  Fri May 31 18:24:39 2013
From: albert.kottke at gmail.com (Albert Kottke)
Date: Fri, 31 May 2013 15:24:39 -0700
Subject: [Numpy-discussion] genfromtxt() skips comments
In-Reply-To: <etPan.51a91deb.6b8b4567.c85c@platypus.local>
References: <CAKoEZq_gMXdQ+n7iMozCUjhT-oY3hWycND_zeyFM5bK-uDBfjQ@mail.gmail.com>
	<etPan.51a91deb.6b8b4567.c85c@platypus.local>
Message-ID: <CAKoEZq_FRxs+inRT-rBve=L_K9+iAUENp7Gbijz=P2KN1PKvvw@mail.gmail.com>

Now try the same thing with np.recfromcsv().

I get the following (Python 3.3):
>>> import io
>>> b = io.BytesIO(b"!blah\n!blah\n!blah\n!A:B:C\n1:2:3\n4:5:6\n")
>>> np.recfromcsv(b, delimiter=':', comments='!')
...
ValueError: Some errors were detected !
    Line #5 (got 3 columns instead of 1)
    Line #6 (got 3 columns instead of 1)


On Fri, May 31, 2013 at 3:02 PM, Pierre GM <pgmdevlist at gmail.com> wrote:

> On May 31, 2013 at 23:08:18 , Albert Kottke (albert.kottke at gmail.com)
> wrote:
> I noticed that genfromtxt() did not skip comments if the keyword names is
> not True. If names is True, then genfromtxt() would take the first line as
> the names. I am proposing a fix to genfromtxt that skips all of the
> comments in a file, and potentially using the last comment line for names.
> I'm quite surprised, as comments are already skipped in my standard numpy
> version (1.7.0). For example:
> >>> S=StringIO("!blah\n!blah\n!blah\n!A:B:C\n1:2:3\n4:5:6\n")
> >>> np.genfromtxt(S, delimiter=":", comments="!", names=("A","B","C"))
> Works as expected, even when using the default `names=None`.
> Comments are taken care of with the `split_line` function (an instance of
> `_iotools.LineSplitter`).
> _______________________________________________
> NumPy-Discussion mailing list
> NumPy-Discussion at scipy.org
> http://mail.scipy.org/mailman/listinfo/numpy-discussion
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20130531/29dd2476/attachment.html>