From Rob  Thu Nov 16 16:50:39 2006
From: Rob (Rob)
Date: 09/04/2001 18:02:05
Subject: [Numpy-discussion] Python and Numpy compiled on Athlon
Message-ID: <mailman.4.1490332292.18468.numpy-discussion@python.org>

optimized gcc3.01

Just for kicks last night I installed gcc3.01 which claims to have Athlon
optimization, ie. -march=athlon.  I recompiled Python and Numpy, and then
ran a big simulation.  The new compiler ran 200 seconds slower than the
old gcc2.95 with plain -march=pentium.  I need to go to the gcc website
and see just what optimization they are claiming.  Maybe I should have
also used -02.   Rob.


-- 
The Numeric Python EM Project

http://www.members.home.net/europax





From bogus@does.not.exist.com  Thu Nov 16 16:50:39 2006
From: bogus@does.not.exist.com ()
Date: Thu, 16 Nov 2006 21:50:39 -0000
Subject: No subject
Message-ID: <mailman.6.1490332297.18468.numpy-discussion@python.org>

Dear All,

thank you for the swift responses and the help!
NumPy now works for me on OSX. Let me summarize:

1. Python 2.2.1 compiles without errors.

2. NumPy 20 does, too - special thanks to Jack!

3. Fink does not have a good UI, I dont't use it.

4. gcc is a hassle to get.

The next step ;-) is now to compile gnuplot in Darwin
which fails. I do have a running version that works
through the OSX interface but do not know how to call
this from the the terminal. If someone has any hints,
I would appreciate an e-mail message. Thanks again!

Best regards,
Thomas

__________________________________________________
Do You Yahoo!?
Yahoo! Health - Feel better, live better
http://health.yahoo.com



From falted at openlc.org  Thu Nov 16 16:50:40 2006
From: falted at openlc.org (Francesc Alted)
Date: Thu, 16 Nov 2006 21:50:40 -0000
Subject: [Numpy-discussion] Conversion functions between Numeric and
Message-ID: <mailman.7.1490332298.18468.numpy-discussion@python.org>

numarray arrays?


Hi,

Anybody is aware of any function (either in C or Python or a mixture of
both) to easily convert Numerical Python arrays from/to numarray arrays?

I mean, I would like to use such a funtion that, without having to copy
element by element all the data, be able to copy the data buffer (or even
use the same if possible at all) from one object to the other.

Thanks,






From nadavh at VisionSense.com  Thu Nov 16 16:50:40 2006
From: nadavh at VisionSense.com (Nadav Horesh)
Date: Thu, 16 Nov 2006 21:50:40 -0000
Subject: [Numpy-discussion] numarray bug !? astype with 2d array
Message-ID: <mailman.9.1490332302.18468.numpy-discussion@python.org>

givestransform ??


> I do not see any of these problems in numarray 0.8 (from CVS).
>
>   Nadav

That is good to hear !
Was this actually a known bug ?
I'm still trying to advertive the use of Numarray (instead of buying Matlab)
in our Lab. We really benefit from the support of UInt16 and single
precision floats when analysing image data. But this "implicit transpose
feature" would have made it unusable ...

Thanks for the work,
Sebastian Haase





> On Mon, 2003-10-13 at 18:11, Sebastian Haase wrote:
> > Hi all,
> > Could someone maybe confirm this or say that numarray 0.7 has fixed this
?
> > I also found more problems when type conversions are involved:  I had a
3d
> > stack of UInt16  data and wanted to compute the 2d-fft of different
> > sections -> I got identical ffts for different sections ;-(
> >
> > Please help,
> > Sebastian Haase
> >
> > ----- Original Message ----- 
> > From: "Sebastian Haase" <haase at msg.ucsf.edu>
> > To: <numpy-discussion at lists.sourceforge.net>
> > Sent: Thursday, October 09, 2003 10:25 AM
> > Subject: [Numpy-discussion] numarray bug !? astype with 2d array gives
> > transform ??
> >
> >
> > > Hi !
> > > I just discovered this: (I'm using numarray 0.6 [on windows])
> > >
> > > >>> dy = na.fromfunction(lambda y,x: y, (3,3))
> > > >>> dx = na.fromfunction(lambda y,x: x, (3,3))
> > > >>> dy
> > > array([[0, 0, 0],
> > >        [1, 1, 1],
> > >        [2, 2, 2]])
> > > >>> dx
> > > array([[0, 1, 2],
> > >        [0, 1, 2],
> > >        [0, 1, 2]])
> > > >>> dx.type()
> > > Int32
> > > >>> dx.astype(na.Int8)
> > > array([[0, 0, 0],
> > >        [1, 1, 1],
> > >        [2, 2, 2]], type=Int8)
> > > >>> dx.astype(na.Int16)
> > > array([[0, 0, 0],
> > >        [1, 1, 1],
> > >        [2, 2, 2]], type=Int16)
> > > >>> dx.astype(na.Float)
> > > array([[ 0.,  0.,  0.],
> > >        [ 1.,  1.,  1.],
> > >        [ 2.,  2.,  2.]])
> > > >>> dx.astype(na.Float32)
> > > array([[ 0.,  0.,  0.],
> > >        [ 1.,  1.,  1.],
> > >        [ 2.,  2.,  2.]], type=Float32)
> > >
> > > What does this mean ? Am I missing something ?
> > >
> > > Thanks,
> > > Sebastian Haase
> > >
> > >
> > >
> > > -------------------------------------------------------
> > > This SF.net email is sponsored by: SF.net Giveback Program.
> > > SourceForge.net hosts over 70,000 Open Source Projects.
> > > See the people who have HELPED US provide better services:
> > > Click here: http://sourceforge.net/supporters.php
> > > _______________________________________________
> > > Numpy-discussion mailing list
> > > Numpy-discussion at lists.sourceforge.net
> > > https://lists.sourceforge.net/lists/listinfo/numpy-discussion
> > >
> >
> >
> >
> > -------------------------------------------------------
> > This SF.net email is sponsored by: SF.net Giveback Program.
> > SourceForge.net hosts over 70,000 Open Source Projects.
> > See the people who have HELPED US provide better services:
> > Click here: http://sourceforge.net/supporters.php
> > _______________________________________________
> > Numpy-discussion mailing list
> > Numpy-discussion at lists.sourceforge.net
> > https://lists.sourceforge.net/lists/listinfo/numpy-discussion
>
>




From bogus@does.not.exist.com  Thu Nov 16 16:50:40 2006
From: bogus@does.not.exist.com ()
Date: Thu, 16 Nov 2006 21:50:40 -0000
Subject: No subject
Message-ID: <mailman.18.1490332313.18468.numpy-discussion@python.org>

Ray Schumacher  writes:

> At 06:31 PM 12/15/2004 +0100, Thomas Heller wrote:
>>>   buf = (c_ulong * 2000)()
>>>   numArray[pntr:pntr+2000] = map(None, buf)
>>
>>Should work, but builds a large temporary list.  Best would be to avoid
>>temporary objects at all, like this (for memmove, you need the latest
>>ctypes release) (I assume that it is possible to get the address of a
>>numarray buffer in a similar way as is possible for a normal array
>>instance):
>
> I'm not sure it will be able to work...
> The closest thing to .buffer_info() for numarray-addressing seems to
> be info(), but it returns text:
>
>  >>> src = numarray.array(range(32), Float)
>  >>> s = info(src)
> class: <class 'numarray.numarraycore.NumArray'>
> shape: (32,)
> strides: (8,)
> byteoffset: 0
> bytestride: 8
> itemsize: 8
> aligned: 1
> contiguous: 1
> data: <memory at 00815880 with size:256 held by object 00815860 aliasing object
> 00000000>
> byteorder: little
> byteswap: 0
> type: Float64
>  >>> src._byteoffset
> 0

What a pity.  (You could try to parse the <memory at 00815880> to get the
address).

Maybe you should ask to make sure that there's no way to copy between
objects implementing the buffer protocol with some Python function that
I do not know about?

If such a function doesn't exist, ctypes' memmove function should be
changed to accept objects implementing the buffer interface.

Thomas





Secret anti-spam filter-passing text. Include with reply: 
qwertyuiop 
--=====================_131324106==.ALT
Content-Type: text/html; charset="us-ascii"

<html>
<body>
I'm posting here to see if numarray has a method that could work like
array.buffer_info().&nbsp; numarray.info() returns a text output that
can't be used like the array method is to memmove() between ctypes and
Python arrays without parsing, apparently.<br><br>
ctypes thread with Thomas Heller below. His main question: &quot;Maybe
you should ask to make sure that there's no way to copy between<br>
objects implementing the buffer protocol with some Python function
that<br>
I do not know about?&quot;<br><br>
Cc: ctypes-users at lists.sourceforge.net<br>
Subject: Re: [ctypes-users] copying/slicing ctypes arrays, (c_ulong
*n)()<br>
In-Reply-To: &lt;5.2.0.4.2.20041215100117.0dff97b0 at blue-cove.com&gt;
(Ray<br>
&nbsp;Schumacher's message of &quot;Wed, 15 Dec 2004 10:57:45
-0800&quot;)<br>
References:
&lt;5.2.1.1.2.20041214202542.05f02920 at pop-server.san.rr.com&gt;<br>
<x-tab>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</x-tab>&lt;5.2.1.1.2.20041214202542.05f02920 at pop-server.san.rr.com&gt;<br>
<x-tab>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</x-tab>&lt;5.2.0.4.2.20041215100117.0dff97b0 at blue-cove.com&gt;<br>
Date: Wed, 15 Dec 2004 20:25:33 +0100<br>
Message-ID: &lt;is73z7ma.fsf at python.net&gt;<br><br>
MIME-Version: 1.0<br>
Content-Type: text/plain; charset=us-ascii<br>
X-UIDL: /O~!!CW8&quot;!g],&quot;!(Ec&quot;!<br>
Ray Schumacher&nbsp; writes:<br><br>
&gt; At 06:31 PM 12/15/2004 +0100, Thomas Heller wrote:<br>
&gt;&gt;&gt;&nbsp;&nbsp; buf = (c_ulong * 2000)()<br>
&gt;&gt;&gt;&nbsp;&nbsp; numArray[pntr:pntr+2000] = map(None, buf)<br>
&gt;&gt;<br>
&gt;&gt;Should work, but builds a large temporary list.&nbsp; Best would
be to avoid<br>
&gt;&gt;temporary objects at all, like this (for memmove, you need the
latest<br>
&gt;&gt;ctypes release) (I assume that it is possible to get the address
of a<br>
&gt;&gt;numarray buffer in a similar way as is possible for a normal
array<br>
&gt;&gt;instance):<br>
&gt;<br>
&gt; I'm not sure it will be able to work...<br>
&gt; The closest thing to .buffer_info() for numarray-addressing seems
to<br>
&gt; be info(), but it returns text:<br>
&gt;<br>
&gt;&nbsp; &gt;&gt;&gt; src = numarray.array(range(32), Float)<br>
&gt;&nbsp; &gt;&gt;&gt; s = info(src)<br>
&gt; class: &lt;class 'numarray.numarraycore.NumArray'&gt;<br>
&gt; shape: (32,)<br>
&gt; strides: (8,)<br>
&gt; byteoffset: 0<br>
&gt; bytestride: 8<br>
&gt; itemsize: 8<br>
&gt; aligned: 1<br>
&gt; contiguous: 1<br>
&gt; data: &lt;memory at 00815880 with size:256 held by object 00815860
aliasing object<br>
&gt; 00000000&gt;<br>
&gt; byteorder: little<br>
&gt; byteswap: 0<br>
&gt; type: Float64<br>
&gt;&nbsp; &gt;&gt;&gt; src._byteoffset<br>
&gt; 0<br><br>
What a pity.&nbsp; (You could try to parse the &lt;memory at 00815880&gt;
to get the<br>
address).<br><br>
Maybe you should ask to make sure that there's no way to copy
between<br>
objects implementing the buffer protocol with some Python function
that<br>
I do not know about?<br><br>
If such a function doesn't exist, ctypes' memmove function should 
be<br>
changed to accept objects implementing the buffer interface.<br><br>
Thomas<br><br>
<br><br>
<br>
<x-sigsep><p></x-sigsep>
Secret anti-spam filter-passing text. Include with reply: <br>
qwertyuiop</body>
</html>

--=====================_131324106==.ALT--




From bogus@does.not.exist.com  Thu Nov 16 16:50:41 2006
From: bogus@does.not.exist.com ()
Date: Thu, 16 Nov 2006 21:50:41 -0000
Subject: No subject
Message-ID: <mailman.20.1490332322.18468.numpy-discussion@python.org>

--j52DDRoG076339.1117719727/mail.grebennikov.ru
Content-Type: text/rfc822-headers


From bogus@does.not.exist.com  Thu Nov 16 16:50:41 2006
From: bogus@does.not.exist.com ()
Date: Thu, 16 Nov 2006 21:50:41 -0000
Subject: No subject
Message-ID: <mailman.21.1490332322.18468.numpy-discussion@python.org>

--j573DcGi094590.1118136822/mail.grebennikov.ru
Content-Type: text/rfc822-headers


From tchur at optushome.com.au  Thu Nov 16 16:50:41 2006
From: tchur at optushome.com.au (Tim Churches)
Date: Apr 12 2005 - 7:51am
Subject: No subject
Message-ID: <mailman.22.1490332327.18468.numpy-discussion@python.org>

Todd Miller wrote:
> On Sun, 2005-04-10 at 10:23 +1000, Tim Churches wrote:
>
>>I just got caught by code equivalent to this (with NumPy 23.8 on 32 bit
>>Linux):
>>
>> >>> import Numeric as N
>> >>> a = N.array((2000000000,1000000000),typecode=N.Int32)
>> >>> N.add.reduce(a)
>>-1294967296
>>
>>OK, it is an elementary mistake, but the silent overflow caught me
>>unawares. casting the array to Float64 before summing it avoids the
>>error, but in my instance the actual data is a rank-1 array of 21
>>million integers with a mean value of about 140 (which adds up more than
>>sys.maxint), and casting to Float64 will use quite a lot of memory (as
>>well as taking some time).
>>
>>Any advice for catching or avoiding such overflow without always
>>incurring a performance and memory hit by always casting to Float64?
>
>
> Here's what numarray does:
>
>
>>>>import numarray as N
>>>>a = N.array((2000000000,1000000000),typecode=N.Int32)
>>>>N.add.reduce(a)
>
> -1294967296
>
> So basic reductions in numarray have the same "careful while you're
> shaving" behavior as Numeric; it's fast but easy to screw up.

Sure, but how does one be careful? It seems that for any array of two
integers or more which could sum to more than sys.maxint or less than
-sys.maxint, add.reduce() in both NumPy and Numeric will give either a)
the correct answer or b) the incorrect answer, and short of adding up
the array using a safer but much slower method, there is no way of
determining if the answer provided (quickly) by add.reduce is right or
wrong? Which seems to make it fast but useless (for integer arrays, at
least? Is that an unfair summary? Can anyone point me towards a method
for using add.reduce() on small arrays of large integers with values in
the billions, or on large arrays of fairly small integer values, which
will not suddenly and without warning give the wrong answer?

>
> But:
>
>
>>>>a.sum()
>
> 3000000000L
>
>>>>a.sum(type='d')
>
> 3000000000.0
>
> a.sum() blockwise upcasts to the largest type of kind on the fly, in
> this case, Int64. This avoids the storage overhead of typecasting the
> entire array.

That's on a 64-bit platform, right? The same method could be used to
cast the accumulator to a Float64 on a 32-bit platform to avoid casting
the entire array?

> A better name for the method would have been sumall() since it sums all
> elements of a multi-dimensional array. The flattening process reduces
> on one dimension before flattening preventing a full copy of a
> discontiguous array. It could be smarter about choosing the dimension
> of the initial reduction.

OK, thanks. Unfortunately it is not possible for us to port our
application to numarray at the moment. But the insight is most helpful.

Tim C



From bogus@does.not.exist.com  Thu Nov 16 16:50:43 2006
From: bogus@does.not.exist.com ()
Date: Thu, 16 Nov 2006 21:50:43 -0000
Subject: No subject
Message-ID: <mailman.23.1490332340.18468.numpy-discussion@python.org>

claration for identifier "ushort".
cc: "numpy/core/include/numpy/arrayobject.h", line 140: error 1713: Illegal=
 rede
claration for identifier "uint".
cc: "numpy/core/src/arrayobject.c", line 2420: warning 562: Redeclaration o=
f "Py
Array_SetNumericOps" with a different storage class specifier: "PyArray_Set=
Numer
icOps" will have internal linkage.
cc: "numpy/core/include/numpy/arrayobject.h", line 139: error 1713: Illegal=
 rede
claration for identifier "ushort".
cc: "numpy/core/include/numpy/arrayobject.h", line 140: error 1713: Illegal=
 rede
claration for identifier "uint".
cc: "numpy/core/src/arrayobject.c", line 2420: warning 562: Redeclaration o=
f "Py
Array_SetNumericOps" with a different storage class specifier: "PyArray_Set=
Numer
icOps" will have internal linkage.
error: Command "cc -Ae -DNDEBUG -O +z -Ibuild/src/numpy/core/src -Inumpy/co=
re/in
clude -Ibuild/src/numpy/core -Inumpy/core/src -Inumpy/core/include -I/usr/l=
ocal/
include/python2.4 -c numpy/core/src/multiarraymodule.c -o build/temp.hp-ux-=
B.11.
11-9000/785-2.4/numpy/core/src/multiarraymodule.o" failed with exit status =
1

And this error message when trying to use 'gcc' instead:

error: don't know how to compile C/C++ code on platform 'posix' with 'gcc' =
comp
iler

Has this been fixed in the current SVN or should can someone suggest a fix?

 -- Paul



From prabhu at aero.iitb.ac.in  Wed Nov  1 04:53:21 2006
From: prabhu at aero.iitb.ac.in (Prabhu Ramachandran)
Date: Wed, 1 Nov 2006 15:23:21 +0530
Subject: [SciPy-user] Need more comments from
	scientific community on python-dev
In-Reply-To: <45478230.2050709@ee.byu.edu>
References: <45474841.9030802@ieee.org>
	<Mahogany-0.67.0-3016-20061031-111400.00@CASC64043305.american.edu>
	<Pine.LNX.4.64.0610310840100.18434@salmo.appl-ecosys.com>
	<db6b5ecc0610310847u331fb8daud97a6a13964c23ad@mail.gmail.com>
	<45478230.2050709@ee.byu.edu>
Message-ID: <17736.28305.154947.629130@prpc.aero.iitb.ac.in>

>>>>> "Travis" == Travis Oliphant <oliphant at ee.byu.edu> writes:

    Travis> 2) Examples of sharing memory between two objects.  PIL is
    Travis> the classic example and has some merit, but because the
    Travis> internal memory layout of the PIL is 'pointer-to-pointers'
    Travis> instead of 'big-chunk-of-memory' it's not a 1-1 match to
    Travis> NumPy and the array interface only can comunicate
    Travis> information about the "mode."  But, I can see other
    Travis> examples.  PyMedia, PyGame, PyVideo?  CVXOPT, PyVoxel.

VTK-Python and TVTK use this.  Right now these use the buffer protocol
to allow users to pass numpy data into a vtkDataArray.  It works
reasonably well.  The VTK data arrays have type information so it is
not absolutely essential for an extended buffer protocol here.  TVTK
uses this underlying functionality in VTK to present a cleaner
interface.  The idea is to have VTK use the data setup and manipulated
by numpy without an expensive copy.  The only caveat is that you need
to hang on to a reference of the numpy array if not you will end up
with a segfault.

I just thought this might be a useful data point.

FWIW, I am all for ndarrays/extended buffers in Python core.  It would
encourage a standard way to pass data in and out of wrapped code.

cheers,
prabhu


-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From faltet at carabos.com  Wed Nov  1 07:03:05 2006
From: faltet at carabos.com (Francesc Altet)
Date: Wed, 01 Nov 2006 13:03:05 +0100
Subject: Reading records from file and sorting
In-Reply-To: <1162337923.747767.323100@b28g2000cwb.googlegroups.com>
References: <1162337923.747767.323100@b28g2000cwb.googlegroups.com>
Message-ID: <1162382586.3966.44.camel@localhost.localdomain>

El dt 31 de 10 del 2006 a les 23:38 +0000, en/na George Sakkis va
escriure:
> Is there a more elegant and/or faster way to read some records from a
> file and then sort them by different fields ? What I have now is too
> specific and error-prone in general:
> 
> import numpy as N
> records = N.fromfile(a_file, dtype=N.dtype('i2,i4'))
> records_by_f0 = records.take(records.getfield('i2').argsort())
> records_by_f1 = records.take(records.getfield('i4',2).argsort())
> 
> If there's a better way, I'd like to see it; bonus points for in-place
> sorting.

Why this is too specific or error-prone? I think your solution is quite
good. If what you want is a more compact way to write the above, you can
try with:

In [56]:records=numpy.array([(1,1),(0,2)], dtype="i2,i4")
In [57]:records[records['f0'].argsort()]
Out[57]:
array([(0, 2), (1, 1)],
      dtype=[('f0', '<i2'), ('f1', '<i4')])
In [58]:records[records['f1'].argsort()]
Out[58]:
array([(1, 1), (0, 2)],
      dtype=[('f0', '<i2'), ('f1', '<i4')])

HTH,

-- 
Francesc Altet    |  Be careful about using the following code --
Carabos Coop. V.  |  I've only proven that it works, 
www.carabos.com   |  I haven't tested it. -- Donald Knuth


-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From dalcinl at gmail.com  Wed Nov  1 10:02:04 2006
From: dalcinl at gmail.com (Lisandro Dalcin)
Date: Wed, 1 Nov 2006 12:02:04 -0300
Subject: Need more comments from scientific community
	on python-dev
In-Reply-To: <e86a5fd00610311626v11c3d554u5af63401b50e030c@mail.gmail.com>
References: <45474841.9030802@ieee.org>
	<e86a5fd00610311626v11c3d554u5af63401b50e030c@mail.gmail.com>
Message-ID: <e7ba66e40611010702n253f0f8ah14927f5513ec4695@mail.gmail.com>

On 10/31/06, Bill Baxter <wbaxter at gmail.com> wrote:
> One thing I see with PIL and Matplotlib and PyOpenGL is that they're having
> to add specific support for Numpy and numeric and numarray and PIL and
> ctypes etc.

This is a real problem, and a waste of developing time, because we
lack a **general** and **standard** mechanism.

> would the proposed extensions make it
> possible for all these different packages to just have to support "buffer
> protocol" and magically be able to work with numpy or numeric without having
> to introduce explicit dependencies and support code for all those other
> packages?

I think this is the actual reason of Travis's proposal, and of all
other people like me supporting him.

> If I've got the facts straight then I'd be happy to go and say
> that on comp.lang.python.
>

Please, go on!



-- 
Lisandro Dalc?n
---------------
Centro Internacional de M?todos Computacionales en Ingenier?a (CIMEC)
Instituto de Desarrollo Tecnol?gico para la Industria Qu?mica (INTEC)
Consejo Nacional de Investigaciones Cient?ficas y T?cnicas (CONICET)
PTLC - G?emes 3450, (3000) Santa Fe, Argentina
Tel/Fax: +54-(0)342-451.1594

-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From wbaxter at gmail.com  Wed Nov  1 10:07:30 2006
From: wbaxter at gmail.com (Bill Baxter)
Date: Thu, 2 Nov 2006 00:07:30 +0900
Subject: Need more comments from scientific community
	on python-dev
In-Reply-To: <e7ba66e40611010702n253f0f8ah14927f5513ec4695@mail.gmail.com>
References: <45474841.9030802@ieee.org>
	<e86a5fd00610311626v11c3d554u5af63401b50e030c@mail.gmail.com>
	<e7ba66e40611010702n253f0f8ah14927f5513ec4695@mail.gmail.com>
Message-ID: <e86a5fd00611010707x744c71d1h2ae714eda07627d7@mail.gmail.com>

Grr, I wrote something up (it was wonderfully eloquent and convincing --
trust me!), but the gmane interface seems to have lost it (I posted two
things before responding to the two "authorization requested" emails, and
only one of the two things ever got posted it seems.)

--bb

On 11/2/06, Lisandro Dalcin <dalcinl at gmail.com> wrote:
>
> On 10/31/06, Bill Baxter <wbaxter at gmail.com> wrote:
> > One thing I see with PIL and Matplotlib and PyOpenGL is that they're
> having
> > to add specific support for Numpy and numeric and numarray and PIL and
> > ctypes etc.
>
> This is a real problem, and a waste of developing time, because we
> lack a **general** and **standard** mechanism.
>
> > would the proposed extensions make it
> > possible for all these different packages to just have to support
> "buffer
> > protocol" and magically be able to work with numpy or numeric without
> having
> > to introduce explicit dependencies and support code for all those other
> > packages?
>
> I think this is the actual reason of Travis's proposal, and of all
> other people like me supporting him.
>
> > If I've got the facts straight then I'd be happy to go and say
> > that on comp.lang.python.
> >
>
> Please, go on!
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20061102/92c59f37/attachment.html>
-------------- next part --------------
-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
-------------- next part --------------
_______________________________________________
Numpy-discussion mailing list
Numpy-discussion at lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/numpy-discussion

From george.sakkis at gmail.com  Wed Nov  1 10:18:40 2006
From: george.sakkis at gmail.com (George Sakkis)
Date: Wed, 01 Nov 2006 15:18:40 -0000
Subject: Reading records from file and sorting
References: <1162337923.747767.323100@b28g2000cwb.googlegroups.com>
	<1162382586.3966.44.camel@localhost.localdomain>
Message-ID: <1162394320.550065.80510@m73g2000cwd.googlegroups.com>

Francesc Altet wrote:

> El dt 31 de 10 del 2006 a les 23:38 +0000, en/na George Sakkis va
> escriure:
> > Is there a more elegant and/or faster way to read some records from a
> > file and then sort them by different fields ? What I have now is too
> > specific and error-prone in general:
> >
> > import numpy as N
> > records = N.fromfile(a_file, dtype=N.dtype('i2,i4'))
> > records_by_f0 = records.take(records.getfield('i2').argsort())
> > records_by_f1 = records.take(records.getfield('i4',2).argsort())
> >
> > If there's a better way, I'd like to see it; bonus points for in-place
> > sorting.
>
> Why this is too specific or error-prone?

Because it
1. repeats the field types
2. requires adding up the length of all previous fields as offset.

If you're not convinced yet, try writing this in less than 3 seconds
;-):
records = N.fromfile(a_file, dtype=N.dtype('i2,i4,f4,S5,B,Q'))
records_by_f5 = ??

> I think your solution is quite good.If what you want is a more compact way to
> write the above, you can
> try with:
>
> In [56]:records=numpy.array([(1,1),(0,2)], dtype="i2,i4")
> In [57]:records[records['f0'].argsort()]
> Out[57]:
> array([(0, 2), (1, 1)],
>       dtype=[('f0', '<i2'), ('f1', '<i4')])
> In [58]:records[records['f1'].argsort()]
> Out[58]:
> array([(1, 1), (0, 2)],
>       dtype=[('f0', '<i2'), ('f1', '<i4')])

Ah, much better; I didn't know you can index a normal array (not
recarray) by label. Now, if there's a way to do the sorting in place
(records.sort('f1') doesn't work unfortunately), that would be perfect.

Thanks,
George


-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From ndarray at mac.com  Wed Nov  1 10:27:48 2006
From: ndarray at mac.com (Sasha)
Date: Wed, 1 Nov 2006 10:27:48 -0500
Subject: Need more comments from scientific community
	on python-dev
In-Reply-To: <4547D2B3.5000808@ee.byu.edu>
References: <45474841.9030802@ieee.org>
	<d38f5330610311210i5141b495s5bb73f8da6484629@mail.gmail.com>
	<4547B235.6050904@ee.byu.edu>
	<d38f5330610311336w7538e0b2v994ee277da64f6cf@mail.gmail.com>
	<4547D2B3.5000808@ee.byu.edu>
Message-ID: <d38f5330611010727x9d1b531odbc597a15df0ce20@mail.gmail.com>

On 10/31/06, Travis Oliphant <oliphant at ee.byu.edu> wrote:
> No, it's not that simple.  We have a headache whenever we want to do
> something like I just did and separate out the concepts of what makes a
> Python Object a Python object.  Now, we don't just modify a simple
> C-structure (PyArray_Descr *), we have to modify a "meta-type" or a
> altered dictionary and get that change put in to ctypes.

I think I am starting to understand. Forgive me for being slow.

Is it correct that you don't mind writing c_int * 10 instead of
dtype(('<i4', 10)) as long as the result does not have to be a
PyTypeObject at the C level? If this is the case, I would suggest to
merge ctypes syntax with your implementation.  This may help to make
the case for the python-dev crowd.  I believe very few people
understand the subtle problems in inheriting from PyTypeObject and
resist your proposal simply because they like c_int * 10  better than
dtype(('<i4', 10)).

There is no reason why ctypes should be implemented the way it is. It
is not necessary that type(c_int()) is c_int.  If a type with almost
the same properties gets into the core, ctypes developers may see an
advantage of simply inheriting from it and adding a factory __call__
method.  Meanwhile, users familiar with ctypes will not need to learn
yet another type specification syntax.

-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From dalcinl at gmail.com  Wed Nov  1 10:47:29 2006
From: dalcinl at gmail.com (Lisandro Dalcin)
Date: Wed, 1 Nov 2006 12:47:29 -0300
Subject: Need more comments from scientific community
	on python-dev
In-Reply-To: <45474841.9030802@ieee.org>
References: <45474841.9030802@ieee.org>
Message-ID: <e7ba66e40611010747u2f80608flc4e1bf6b2b19fb5a@mail.gmail.com>

On 10/31/06, Travis Oliphant <oliphant.travis at ieee.org> wrote:
>
> I'm recruiting more comments on python-dev regarding my two proposals
> for improving Python's native ability to share ndarray-like information.
>

I believe there are another really important reason to support
Travis's proposal.

Python should have a general, simple, powerful and standard way to
associate type metadata to binary streams. This enables efficient
share or memory, but also can help a lot to achieve inter-language
capabilities.

As an example, I would to make some comments about MPI, the "de facto"
standard for message-passing in parallel computing. When MPI
specification was being developed, there were many different,
incompatible libraries for message passing, and many of them (ie. PVM)
usually communicated binary data following a pack/unpack approach. The
MPI Forum decided to add an alternative approach, adding a class
'MPI_Datatype', and many predefined instances of this class (MPI_INT,
MPI_FLOAT, etc.) in order to represent the basic datatypes (integers,
floats, strings, complex numbers) in C, Fortran, and C++. Predefined
MPI datatypes can be used as building blocks for describing very
complicated memory layouts and derived,nested structures. This way,
any implementations of this standard can support interprocess
communication in a multi-language, muti-architecture approach. MPI
datatypes can also be "decoded" in order to get the basic, predefined
types contained in a user-defined datatype.

I think the MPI example, despite being taken from a very specific
domain with particular needs, has a strong connection with Travis's
proposal. Pack/unpack degrades performance, but if you want to
efficiently share binary data (with other languages/architecture),
then a general and extensible mechanism for attaching type metadata to
binary streams must me defined.

I would like to know your opinions about this. Travis's datatypes in
Python should be more that something to share data, they should be
able to **describe** data. And I am not sure if ctypes can be be the
way. Can Jython take advantage of ctypes??? Or in the near future
CPython/Jython will have a 'jtypes' module? I forget! Travis/Pearu,
why don't you develop a 'ftypes' for Fortran binary data, and next ask
Python core developers to include it in standard library (a joke, of
course)?

Travis, if you think my previous example about MPI can help, I can
resend this post to Python-Dev. I ask you this because I really want
to help and not to make noise.

Regars,


-- 
Lisandro Dalc?n
---------------
Centro Internacional de M?todos Computacionales en Ingenier?a (CIMEC)
Instituto de Desarrollo Tecnol?gico para la Industria Qu?mica (INTEC)
Consejo Nacional de Investigaciones Cient?ficas y T?cnicas (CONICET)
PTLC - G?emes 3450, (3000) Santa Fe, Argentina
Tel/Fax: +54-(0)342-451.1594

-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From oliphant.travis at ieee.org  Wed Nov  1 10:57:10 2006
From: oliphant.travis at ieee.org (Travis Oliphant)
Date: Wed, 01 Nov 2006 08:57:10 -0700
Subject: Need more comments from scientific community
 on python-dev
In-Reply-To: <d38f5330610311551u4e71cdd7ndc0b93de16dc6d80@mail.gmail.com>
References: <45474841.9030802@ieee.org>	<d38f5330610311210i5141b495s5bb73f8da6484629@mail.gmail.com>	<4547B235.6050904@ee.byu.edu>	<d38f5330610311336w7538e0b2v994ee277da64f6cf@mail.gmail.com>	<4547D2B3.5000808@ee.byu.edu>
	<d38f5330610311551u4e71cdd7ndc0b93de16dc6d80@mail.gmail.com>
Message-ID: <4548C3D6.40100@ieee.org>

Alexander Belopolsky wrote:
> On 10/31/06, Travis Oliphant <oliphant at ee.byu.edu> wrote:
> ...
>   
>> Please understand what I meant.  I meant putting effort into getting
>> PyArray_DescrConverter to allow ctypes inputs and convert to the
>> appropriate PyArray_Descr * structure.   I already understand ctypes
>> objects.  I want the dtype() command to also understand them.
>>
>>     
>
> I think I am starting to understand. Forgive me for being slow.
>   

And I think I'm beginning to understand why some people are resistant to 
what I'm saying.   I have no problem converting from ctypes types to the 
underlying PyArray_Descr structure.  I don't necessarily want that to be 
the only way to describe new types but I sure don't mind people using 
that approach.

I suspect it's the concern about "how to describe data-formats on the 
Python level" that have people concerned.  

> Is it correct that you don't mind writing c_int * 10 instead of
> dtype(('<i4', 10)) as long as the result does not have to be a
> PyTypeObject at the C level?
That is correct. 

>  If this is the case, I would suggest to
> merge ctypes syntax with your implementation.  
> This may help to make
> the case for the python-dev crowd.
Thanks for this suggestion.  You could very probably be right.
>   I believe very few people
> understand the subtle problems in inheriting from PyTypeObject and
> resist your proposal simply because they like c_int * 10  better than
> dtype(('<i4', 10)).
>
> There is no reason why ctypes should be implemented the way it is. It
> is not necessary that type(c_int()) is c_int.  If a type with almost
> the same properties gets into the core, ctypes developers may see an
> advantage of simply inheriting from it and adding a factory __call__
> method.  Meanwhile, users familiar with ctypes will not need to learn
> yet another type specification syntax.
>
>   
It would actually be "easier" for NumPy to understand ctypes syntax if 
something like a data-type object were.  As it is, the converter to a 
PyArray_Descr structure will probably have to check the tp_name of the 
ctypes.  Maybe this will help with the PEP to describe the difficulty of 
using ctypes constructs inside of other programs.

And it may be a good idea to also have a get_ctype method or some-such 
on the ctypes attribute so that  one could get a "ctypes description" 
from the NumPy data-type.

-Travis


-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From faltet at carabos.com  Wed Nov  1 11:29:31 2006
From: faltet at carabos.com (Francesc Altet)
Date: Wed, 01 Nov 2006 17:29:31 +0100
Subject: Reading records from file and sorting
In-Reply-To: <1162394320.550065.80510@m73g2000cwd.googlegroups.com>
References: <1162337923.747767.323100@b28g2000cwb.googlegroups.com>
	<1162382586.3966.44.camel@localhost.localdomain>
	<1162394320.550065.80510@m73g2000cwd.googlegroups.com>
Message-ID: <1162398571.3967.3.camel@localhost.localdomain>

El dc 01 de 11 del 2006 a les 15:18 +0000, en/na George Sakkis va
escriure:
> > Why this is too specific or error-prone?
> 
> Because it
> 1. repeats the field types
> 2. requires adding up the length of all previous fields as offset.
> 
> If you're not convinced yet, try writing this in less than 3 seconds
> ;-):
> records = N.fromfile(a_file, dtype=N.dtype('i2,i4,f4,S5,B,Q'))
> records_by_f5 = ??

Ah, I see your point :)

> > I think your solution is quite good.If what you want is a more compact way to
> > write the above, you can
> > try with:
> >
> > In [56]:records=numpy.array([(1,1),(0,2)], dtype="i2,i4")
> > In [57]:records[records['f0'].argsort()]
> > Out[57]:
> > array([(0, 2), (1, 1)],
> >       dtype=[('f0', '<i2'), ('f1', '<i4')])
> > In [58]:records[records['f1'].argsort()]
> > Out[58]:
> > array([(1, 1), (0, 2)],
> >       dtype=[('f0', '<i2'), ('f1', '<i4')])
> 
> Ah, much better; I didn't know you can index a normal array (not
> recarray) by label. Now, if there's a way to do the sorting in place
> (records.sort('f1') doesn't work unfortunately), that would be perfect.

Yes, I agree that having the possibility to do records.sort('f1') would
be a great addition (both in terms of usability but also efficiency).

Cheers,

-- 
Francesc Altet    |  Be careful about using the following code --
Carabos Coop. V.  |  I've only proven that it works, 
www.carabos.com   |  I haven't tested it. -- Donald Knuth


-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From peridot.faceted at gmail.com  Wed Nov  1 13:13:05 2006
From: peridot.faceted at gmail.com (A. M. Archibald)
Date: Wed, 1 Nov 2006 13:13:05 -0500
Subject: Need more comments from scientific community
	on python-dev
In-Reply-To: <4548C3D6.40100@ieee.org>
References: <45474841.9030802@ieee.org>
	<d38f5330610311210i5141b495s5bb73f8da6484629@mail.gmail.com>
	<4547B235.6050904@ee.byu.edu>
	<d38f5330610311336w7538e0b2v994ee277da64f6cf@mail.gmail.com>
	<4547D2B3.5000808@ee.byu.edu>
	<d38f5330610311551u4e71cdd7ndc0b93de16dc6d80@mail.gmail.com>
	<4548C3D6.40100@ieee.org>
Message-ID: <ce557a360611011013v4e883251l46dc6e2c0c85b854@mail.gmail.com>

On 01/11/06, Travis Oliphant <oliphant.travis at ieee.org> wrote:

> And it may be a good idea to also have a get_ctype method or some-such
> on the ctypes attribute so that  one could get a "ctypes description"
> from the NumPy data-type.

It seems to me that at the python level, there's not much reason to
choose the dtypes proposal over ctypes. There is one at the C level,
it seems (though I, like perhaps most of the people on python-dev,
have never actually tried using either). So perhaps, to persuade the
python-dev folks, what is needed is a comparison of what has to be
done at the C level. What would it take to rewrite numpy to use
ctypes? There seems to be some problem with extending the type objects
used by ctypes, but it's not very clear to me what that problem is
(what the extensions are supposed to do).

The argument that *some* datatypes format should become standard is much easier.

A. M. Archibald

-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From wbaxter at gmail.com  Wed Nov  1 13:19:15 2006
From: wbaxter at gmail.com (Bill Baxter)
Date: Thu, 2 Nov 2006 03:19:15 +0900
Subject: Need more comments from scientific community
	on python-dev
In-Reply-To: <ce557a360611011013v4e883251l46dc6e2c0c85b854@mail.gmail.com>
References: <45474841.9030802@ieee.org>
	<d38f5330610311210i5141b495s5bb73f8da6484629@mail.gmail.com>
	<4547B235.6050904@ee.byu.edu>
	<d38f5330610311336w7538e0b2v994ee277da64f6cf@mail.gmail.com>
	<4547D2B3.5000808@ee.byu.edu>
	<d38f5330610311551u4e71cdd7ndc0b93de16dc6d80@mail.gmail.com>
	<4548C3D6.40100@ieee.org>
	<ce557a360611011013v4e883251l46dc6e2c0c85b854@mail.gmail.com>
Message-ID: <e86a5fd00611011019p5a39553cwf55fd06e585e58e0@mail.gmail.com>

On 11/2/06, A. M. Archibald <peridot.faceted at gmail.com> wrote:
>
> On 01/11/06, Travis Oliphant <oliphant.travis at ieee.org> wrote:
>
> > And it may be a good idea to also have a get_ctype method or some-such
> > on the ctypes attribute so that  one could get a "ctypes description"
> > from the NumPy data-type.
>
> It seems to me that at the python level, there's not much reason to
> choose the dtypes proposal over ctypes. There is one at the C level,
> it seems (though I, like perhaps most of the people on python-dev,
> have never actually tried using either). So perhaps, to persuade the
> python-dev folks, what is needed is a comparison of what has to be
> done at the C level. What would it take to rewrite numpy to use
> ctypes? There seems to be some problem with extending the type objects
> used by ctypes, but it's not very clear to me what that problem is
> (what the extensions are supposed to do).



I posted a message to the thread trying to prod things in that direction.
I.e. can we see a simple concrete example of the complications involved in
using ctypes interface code, vs the presumably much nicer
numpy/data-descriptor code.  I too think that would help convince people.
Just saying it's more compilcated, trust me, doesn't help when most people
reading the list have never had to write any sort of C extension.

--bb
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20061102/ddf05221/attachment.html>
-------------- next part --------------
-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
-------------- next part --------------
_______________________________________________
Numpy-discussion mailing list
Numpy-discussion at lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/numpy-discussion

From Mailer-Daemon at lists.sourceforge.net  Wed Nov  1 13:43:39 2006
From: Mailer-Daemon at lists.sourceforge.net (Mail Delivery System)
Date: Wed, 01 Nov 2006 10:43:39 -0800
Subject: Warning: message 1GfAoO-0007im-Sh delayed 10
	hours
Message-ID: <E1GfL3r-0007Au-8b@externalmx-1.sourceforge.net>

This message was created automatically by mail delivery software.
A message that you sent has not yet been delivered to one or more of its
recipients after more than 10 hours on the queue on externalmx-1.sourceforge.net.

The message identifier is:     1GfAoO-0007im-Sh
The subject of the message is: looked like me the full sized HARLEY and began to her, with this but as
The date of the message is:    Wed, 01 Nov 2006 15:46:73 +0800

The address to which the message has not yet been delivered is:

  numpy-discussion at lists.sourceforge.net
    Delay reason: SMTP error from remote mailer after RCPT TO:<numpy-discussion at lists.sourceforge.net>:
    host mail.sourceforge.net [66.35.250.206]: 451-Could not complete sender verify callout
    451-Could not complete sender verify callout for
    451-<numpy-discussion at lists.sourceforge.net>.
    451-The mail server(s) for the domain may be temporarily unreachable, or
    451-they may be permanently unreachable from this server. In the latter case,
    451-you need to change the address or create an MX record for its domain
    451-if it is 

No action is required on your part. Delivery attempts will continue for
some time, and this warning may be repeated at intervals if the message
remains undelivered. Eventually the mail delivery software will give up,
and when that happens, the message will be returned to you.

-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From kwgoodman at gmail.com  Wed Nov  1 15:33:05 2006
From: kwgoodman at gmail.com (Keith Goodman)
Date: Wed, 1 Nov 2006 12:33:05 -0800
Subject: Converting bool to float
Message-ID: <f4f93d420611011233i60c8128ew63288d5620477c9c@mail.gmail.com>

I had a hard time tracing a bug in my code. The culprit was this difference:

>> x

matrix([[True],
        [True],
        [True]], dtype=bool)
>> 1.0 - x

matrix([[ 0.],
        [ 0.],
        [ 0.]], dtype=float32)     <------- float32
>> 1.0*x

matrix([[ 1.],
        [ 1.],
        [ 1.]])                            <-------- float64

>> numpy.__version__
'1.0rc1'

Any chance that 1.0 - x could return dtype = float64?

-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From oliphant at ee.byu.edu  Wed Nov  1 16:43:38 2006
From: oliphant at ee.byu.edu (Travis Oliphant)
Date: Wed, 01 Nov 2006 14:43:38 -0700
Subject: Converting bool to float
In-Reply-To: <f4f93d420611011233i60c8128ew63288d5620477c9c@mail.gmail.com>
References: <f4f93d420611011233i60c8128ew63288d5620477c9c@mail.gmail.com>
Message-ID: <4549150A.4010404@ee.byu.edu>

Keith Goodman wrote:

>I had a hard time tracing a bug in my code. The culprit was this difference:
>
>  
>
>>>x
>>>      
>>>
>
>matrix([[True],
>        [True],
>        [True]], dtype=bool)
>  
>
>>>1.0 - x
>>>      
>>>
>
>matrix([[ 0.],
>        [ 0.],
>        [ 0.]], dtype=float32)     <------- float32
>  
>
>>>1.0*x
>>>      
>>>
>
>matrix([[ 1.],
>        [ 1.],
>        [ 1.]])                            <-------- float64
>
>  
>
>>>numpy.__version__
>>>      
>>>
>'1.0rc1'
>
>Any chance that 1.0 - x could return dtype = float64?
>  
>
I'm surprised it doesn't.   Both should follow bascially the same 
code-path.  Perhaps there is a missing function loop or something. 


I'll look in to it.

-Travis


-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From oliphant at ee.byu.edu  Wed Nov  1 16:51:40 2006
From: oliphant at ee.byu.edu (Travis Oliphant)
Date: Wed, 01 Nov 2006 14:51:40 -0700
Subject: Converting bool to float
In-Reply-To: <f4f93d420611011233i60c8128ew63288d5620477c9c@mail.gmail.com>
References: <f4f93d420611011233i60c8128ew63288d5620477c9c@mail.gmail.com>
Message-ID: <454916EC.6010003@ee.byu.edu>

Keith Goodman wrote:

>I had a hard time tracing a bug in my code. The culprit was this difference:
>
>  
>
>>>x
>>>      
>>>
>
>matrix([[True],
>        [True],
>        [True]], dtype=bool)
>  
>
>>>1.0 - x
>>>      
>>>
>
>matrix([[ 0.],
>        [ 0.],
>        [ 0.]], dtype=float32)     <------- float32
>  
>
>>>1.0*x
>>>      
>>>
>
>matrix([[ 1.],
>        [ 1.],
>        [ 1.]])                            <-------- float64
>
>  
>
>>>numpy.__version__
>>>      
>>>
>'1.0rc1'
>
>Any chance that 1.0 - x could return dtype = float64?
>  
>

It looks like 1.0-x is doing the right thing.

The problem is 1.0*x for matrices is going to float64.  For arrays it 
returns float32 just like the 1.0-x

This can't be changed at this point until 1.1

We will fix the bug in 1.0*x producing float64, however.  I'm still not 
sure what's causing it, though.

-Travis


-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From robert.kern at gmail.com  Wed Nov  1 16:57:19 2006
From: robert.kern at gmail.com (Robert Kern)
Date: Wed, 01 Nov 2006 13:57:19 -0800
Subject: Converting bool to float
In-Reply-To: <454916EC.6010003@ee.byu.edu>
References: <f4f93d420611011233i60c8128ew63288d5620477c9c@mail.gmail.com>
	<454916EC.6010003@ee.byu.edu>
Message-ID: <eib580$qdt$1@sea.gmane.org>

Travis Oliphant wrote:

> It looks like 1.0-x is doing the right thing.
> 
> The problem is 1.0*x for matrices is going to float64.  For arrays it 
> returns float32 just like the 1.0-x

Why is this the right thing? Python floats are float64.

-- 
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless enigma
  that is made terrible by our own mad attempt to interpret it as though it had
  an underlying truth."
   -- Umberto Eco


-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From hetland at tamu.edu  Wed Nov  1 18:14:57 2006
From: hetland at tamu.edu (Rob Hetland)
Date: Wed, 1 Nov 2006 17:14:57 -0600
Subject: pyaudio 0.3, with docs !
In-Reply-To: <4547519B.4040201@ar.media.kyoto-u.ac.jp>
References: <4547519B.4040201@ar.media.kyoto-u.ac.jp>
Message-ID: <92DCBC0B-10B3-469D-A45D-172C02B3F9D2@tamu.edu>


One small change -- on my intel mac, I need to link to

/usr/local/lib/libsndfile.1.dylib

instead of the .so file.

Then everything works!!!  This is truely a great thing -- my two  
loves, numpy and audio files, together forever...

-r

On Oct 31, 2006, at 7:37 AM, David Cournapeau wrote:

> Hi,
>
>     I improved pyaudio last WE using indications given by various  
> people
> on the list or privately, and as I finally got the motivation to  
> set-up
> something which looks like a webpage, there is a doc with examples  
> which
> show how to use it. The API to open files for writing is much  
> saner, and
> the setup.py should be smart enough to grab all informations necessary
> to the wrapper, including the location of the shared libsndfile:
>
>     download:
> http://www.ar.media.kyoto-u.ac.jp/members/david/softwares/pyaudio/ 
> #installation
>     doc + examples:
> http://www.ar.media.kyoto-u.ac.jp/members/david/softwares/pyaudio/
>
>     I would appreciate to hear reports on platforms which are not  
> linux
> (windows, mac os X) to see if my the setup.py works there,
>
>     Cheers,
>
>     David
> _______________________________________________
> SciPy-user mailing list
> SciPy-user at scipy.org
> http://projects.scipy.org/mailman/listinfo/scipy-user

----
Rob Hetland, Associate Professor
Dept. of Oceanography, Texas A&M University
http://pong.tamu.edu/~rob
phone: 979-458-0096, fax: 979-845-6331



From kwgoodman at gmail.com  Wed Nov  1 20:50:15 2006
From: kwgoodman at gmail.com (Keith Goodman)
Date: Wed, 1 Nov 2006 17:50:15 -0800
Subject: Converting bool to float
In-Reply-To: <454916EC.6010003@ee.byu.edu>
References: <f4f93d420611011233i60c8128ew63288d5620477c9c@mail.gmail.com>
	<454916EC.6010003@ee.byu.edu>
Message-ID: <f4f93d420611011750j78f8fa26h9d048821e7dd0f05@mail.gmail.com>

On 11/1/06, Travis Oliphant <oliphant at ee.byu.edu> wrote:
> It looks like 1.0-x is doing the right thing.
>
> The problem is 1.0*x for matrices is going to float64.  For arrays it
> returns float32 just like the 1.0-x
>
> This can't be changed at this point until 1.1
>
> We will fix the bug in 1.0*x producing float64, however.  I'm still not
> sure what's causing it, though.

I think it would be great if float64 was the default in numpy. That
way most people wouldn't have to worry about dtypes when crunching
numbers. And then numpy could apply for a trademark on 'it just
works'.

Having to worry about dtypes makes users (me) nervous.

I imagine a change like this would not be an overnight change, more of
a long-term goal.

This one, from a previous thread, also makes me nervous:

>> sum(M.ones((300,1)) == 1)
matrix([[44]], dtype=int8)

But float64 might not make sense here.

-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From wbaxter at gmail.com  Wed Nov  1 20:44:38 2006
From: wbaxter at gmail.com (Bill Baxter)
Date: Thu, 2 Nov 2006 10:44:38 +0900
Subject: Compressed index sets, generators and iterators
Message-ID: <e86a5fd00611011744sbae3b89oada17ea4718837b2@mail.gmail.com>

Has any thought been given to using compressed or functional representations
of index sets?

For instance there could be a where-like function that returns an object
that can generate a set of indexes on the fly, rather than explicitly
allocating arrays and enumerating all of the indices.

What's the reason iterators are not supported currently?
For instance  A[range(0,4)] works for a 1d A, but A[xrange(0,4)] does not.
Are iterators just too inefficient to bother with?

I could imagine an iterator that generates a set of N-tuples for an N-d
array being a legal indexing construct.  Even something like
A[idx_iterator] = value_iterator  would seem to make sense.

--bb
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20061102/9873c122/attachment.html>
-------------- next part --------------
-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
-------------- next part --------------
_______________________________________________
Numpy-discussion mailing list
Numpy-discussion at lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/numpy-discussion

From nqhbtgkjppg at shawcable.net  Wed Nov  1 18:37:58 2006
From: nqhbtgkjppg at shawcable.net (Nov.)
Date: Wed, 1 Nov 2006 15:37:58 -0800
Subject: NowPlaying Topic
Message-ID: <000b01c6fe0e$c2f289f0$ed6f5218@kathy>


Web Tools or Html easiest Html guide in beginners. Cashfall Previewtop am ffb Bluegrass Dates?
Brought in discovery Monsters Waiting Echo ca slipped.
Business apply throughout! Crash Fuel Saving a Minivans of.
Business apply throughout!
Zo Poule Place net Stel am. Rate Sign Multimedia or maken heeft.
Waiting Echo ca slipped apropos showcase. Jackson School Karl Mamola State Univeristy in?
Waiting Echo ca slipped apropos showcase? Etc shareware is Burner Webedit Webber or friendly flexible in?
Adoredinto a Pianos night Annalee of Ferys breathy. Golive new name yes bought software. Did my views.
Roses Wanna am Adoredinto Pianos night! Zijn utc or sinds am.
People editor a included Splash easy. Ronde hulpstuk nba Gisteren Overige a Alles zonder.
Web Tools or Html easiest Html guide in beginners. Cntry Grammys Winners a.
Online is gebruikers in hebben totaal am. Zin rondje kerk Basso mag weer wedstri am literatuur am.
Cars Crash Fuel Saving!
Than sites or intend beginner supports image creations files or offers. Schepper in Sporten Live Blijf hoogte van Deze keer bezocht of. Lot longer separate in tools insert!
Claris Home focus mainly single pages.
One additional received a November accepted of until filled Dickison Physical.
History Articles Picks. Cashfall Previewtop am ffb Bluegrass Dates?
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20061101/e377463e/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: Pareles.gif
Type: image/gif
Size: 9519 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20061101/e377463e/attachment.gif>
-------------- next part --------------
-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
-------------- next part --------------
_______________________________________________
Numpy-discussion mailing list
Numpy-discussion at lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/numpy-discussion

From fullung at gmail.com  Wed Nov  1 17:46:20 2006
From: fullung at gmail.com (Albert Strasheim)
Date: Thu, 2 Nov 2006 00:46:20 +0200
Subject: Reading records from file and sorting
In-Reply-To: <1162337923.747767.323100@b28g2000cwb.googlegroups.com>
References: <1162337923.747767.323100@b28g2000cwb.googlegroups.com>
Message-ID: <20061101224620.GA29602@dogbert.sdsl.sun.ac.za>

Hey George

On Tue, 31 Oct 2006, George Sakkis wrote:

> Is there a more elegant and/or faster way to read some records from a
> file and then sort them by different fields ? What I have now is too
> specific and error-prone in general:
> 
> import numpy as N
> records = N.fromfile(a_file, dtype=N.dtype('i2,i4'))
> records_by_f0 = records.take(records.getfield('i2').argsort())
> records_by_f1 = records.take(records.getfield('i4',2).argsort())
> 
> If there's a better way, I'd like to see it; bonus points for in-place
> sorting.

Check the thread "Strange results when sorting array with fields" from 
about a week back. Travis made some changes to sorting in the presence 
of fields that should solve your problem, assuming your fields appear in 
the order you want to sort (i.e. you want to sort f1, f2, f3 and not 
something like f1, f3, f2).

Cheers,

Albert

-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From oliphant at ee.byu.edu  Wed Nov  1 17:44:09 2006
From: oliphant at ee.byu.edu (Travis Oliphant)
Date: Wed, 01 Nov 2006 15:44:09 -0700
Subject: Converting bool to float
In-Reply-To: <eib580$qdt$1@sea.gmane.org>
References: <f4f93d420611011233i60c8128ew63288d5620477c9c@mail.gmail.com>	<454916EC.6010003@ee.byu.edu>
	<eib580$qdt$1@sea.gmane.org>
Message-ID: <45492339.6050302@ee.byu.edu>

Robert Kern wrote:

>Travis Oliphant wrote:
>
>  
>
>>It looks like 1.0-x is doing the right thing.
>>
>>The problem is 1.0*x for matrices is going to float64.  For arrays it 
>>returns float32 just like the 1.0-x
>>    
>>
>
>Why is this the right thing? Python floats are float64.
>  
>
Yeah, why indeed.  Must be something with the scalar coercion code...

-Travis



-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From charlesr.harris at gmail.com  Wed Nov  1 17:21:41 2006
From: charlesr.harris at gmail.com (Charles R Harris)
Date: Wed, 1 Nov 2006 15:21:41 -0700
Subject: Converting bool to float
In-Reply-To: <eib580$qdt$1@sea.gmane.org>
References: <f4f93d420611011233i60c8128ew63288d5620477c9c@mail.gmail.com>
	<454916EC.6010003@ee.byu.edu> <eib580$qdt$1@sea.gmane.org>
Message-ID: <e06186140611011421k118fd7b7j97bd9614c181ec5b@mail.gmail.com>

On 11/1/06, Robert Kern <robert.kern at gmail.com> wrote:
>
> Travis Oliphant wrote:
>
> > It looks like 1.0-x is doing the right thing.
> >
> > The problem is 1.0*x for matrices is going to float64.  For arrays it
> > returns float32 just like the 1.0-x
>
> Why is this the right thing? Python floats are float64.


Same question here. Float32 is a designer float for special occasions,
float64 is for everyday use.

Chuck
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20061101/64c31235/attachment.html>
-------------- next part --------------
-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
-------------- next part --------------
_______________________________________________
Numpy-discussion mailing list
Numpy-discussion at lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/numpy-discussion

From tim.hochberg at ieee.org  Wed Nov  1 22:16:59 2006
From: tim.hochberg at ieee.org (Tim Hochberg)
Date: Wed, 01 Nov 2006 20:16:59 -0700
Subject: Converting bool to float
In-Reply-To: <45492339.6050302@ee.byu.edu>
References: <f4f93d420611011233i60c8128ew63288d5620477c9c@mail.gmail.com>	<454916EC.6010003@ee.byu.edu>	<eib580$qdt$1@sea.gmane.org>
	<45492339.6050302@ee.byu.edu>
Message-ID: <4549632B.4010001@ieee.org>

Travis Oliphant wrote:
> Robert Kern wrote:
>
>   
>> Travis Oliphant wrote:
>>
>>  
>>
>>     
>>> It looks like 1.0-x is doing the right thing.
>>>
>>> The problem is 1.0*x for matrices is going to float64.  For arrays it 
>>> returns float32 just like the 1.0-x
>>>    
>>>
>>>       
>> Why is this the right thing? Python floats are float64.
>>  
>>
>>     
> Yeah, why indeed.  Must be something with the scalar coercion code...

This is one of those things that pops up every few years. I suspect that 
the best thing to do here is to treat 1.0, and all Python floats as 
having a kind (float), but no precision. Or, equivalently treat them as 
the smallest precision floating point value. The rationale behind this 
is that otherwise float32 array will be promoted whenever they are 
multiplied by Python floating point scalars. If Python floats are 
treated as Float64 for purposes of determining output precision then 
anyone using float32 arrays is going to have to wrap all of their 
literals in float32 to prevent inadvertent upcasting to float64. This 
was the origin of the (rather clunky) numarray spacesaver flag.

It's no skin off my nose either way, since I pretty much never use 
float32, but I suspect that treating python floats equivalently to 
float64 scalars would be a mistake. At the very least it deserves a bit 
of discussion.

-tim


-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From charlesr.harris at gmail.com  Wed Nov  1 22:19:45 2006
From: charlesr.harris at gmail.com (Charles R Harris)
Date: Wed, 1 Nov 2006 20:19:45 -0700
Subject: Converting bool to float
In-Reply-To: <f4f93d420611011750j78f8fa26h9d048821e7dd0f05@mail.gmail.com>
References: <f4f93d420611011233i60c8128ew63288d5620477c9c@mail.gmail.com>
	<454916EC.6010003@ee.byu.edu>
	<f4f93d420611011750j78f8fa26h9d048821e7dd0f05@mail.gmail.com>
Message-ID: <e06186140611011919n70d09aebsa1fdbfaefdc1ae8b@mail.gmail.com>

On 11/1/06, Keith Goodman <kwgoodman at gmail.com> wrote:
>
> On 11/1/06, Travis Oliphant <oliphant at ee.byu.edu> wrote:
> > It looks like 1.0-x is doing the right thing.
> >
> > The problem is 1.0*x for matrices is going to float64.  For arrays it
> > returns float32 just like the 1.0-x
> >
> > This can't be changed at this point until 1.1
> >
> > We will fix the bug in 1.0*x producing float64, however.  I'm still not
> > sure what's causing it, though.
>
> I think it would be great if float64 was the default in numpy. That
> way most people wouldn't have to worry about dtypes when crunching
> numbers. And then numpy could apply for a trademark on 'it just
> works'.
>
> Having to worry about dtypes makes users (me) nervous.
>
> I imagine a change like this would not be an overnight change, more of
> a long-term goal.
>
> This one, from a previous thread, also makes me nervous:
>
> >> sum(M.ones((300,1)) == 1)
> matrix([[44]], dtype=int8)


That one seems to be fixed:

In [1]: sum(ones((300,1)) == 1)
Out[1]: 300

In [2]: (ones((300,1)) == 1).sum()
Out[2]: 300

The matrix version also returns a numpy scalar, however.

In [20]: sum(matrix(ones((300,1)) == 1))
Out[20]: 300

I wonder if that is expected?

Chuck
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20061101/c696861c/attachment.html>
-------------- next part --------------
-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
-------------- next part --------------
_______________________________________________
Numpy-discussion mailing list
Numpy-discussion at lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/numpy-discussion

From peridot.faceted at gmail.com  Wed Nov  1 22:20:27 2006
From: peridot.faceted at gmail.com (A. M. Archibald)
Date: Wed, 1 Nov 2006 22:20:27 -0500
Subject: Compressed index sets,
	generators and iterators
In-Reply-To: <e86a5fd00611011744sbae3b89oada17ea4718837b2@mail.gmail.com>
References: <e86a5fd00611011744sbae3b89oada17ea4718837b2@mail.gmail.com>
Message-ID: <ce557a360611011920k7b79d9a6v36b0bace92955436@mail.gmail.com>

On 01/11/06, Bill Baxter <wbaxter at gmail.com> wrote:

> What's the reason iterators are not supported currently?
> For instance  A[range(0,4)] works for a 1d A, but A[xrange(0,4)] does not.
> Are iterators just too inefficient to bother with?

If you want an iterator back, a generator comprehension will do it:
(A[i] for i in xrange(0,4))

If the result is to be a numpy array, the size must be known ahead of
time (so that the memory can be allocated) and specified. At this
point, and considering the overhead in calling back and forth to the
generator, the cost of converting to a list (after all,
A[list(xrange(0,40)] should work fine) isn't necessarily worth the
trouble.

A. M. Archibald

-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From tim.hochberg at ieee.org  Wed Nov  1 22:27:22 2006
From: tim.hochberg at ieee.org (Tim Hochberg)
Date: Wed, 01 Nov 2006 20:27:22 -0700
Subject: Compressed index sets,
	generators and iterators
In-Reply-To: <ce557a360611011920k7b79d9a6v36b0bace92955436@mail.gmail.com>
References: <e86a5fd00611011744sbae3b89oada17ea4718837b2@mail.gmail.com>
	<ce557a360611011920k7b79d9a6v36b0bace92955436@mail.gmail.com>
Message-ID: <4549659A.6020506@ieee.org>

A. M. Archibald wrote:
> On 01/11/06, Bill Baxter <wbaxter at gmail.com> wrote:
>
>   
>> What's the reason iterators are not supported currently?
>> For instance  A[range(0,4)] works for a 1d A, but A[xrange(0,4)] does not.
>> Are iterators just too inefficient to bother with?
>>     
>
> If you want an iterator back, a generator comprehension will do it:
> (A[i] for i in xrange(0,4))
>
> If the result is to be a numpy array, the size must be known ahead of
> time (so that the memory can be allocated) and specified. At this
> point, and considering the overhead in calling back and forth to the
> generator, the cost of converting to a list (after all,
> A[list(xrange(0,40)] should work fine) isn't necessarily worth the
> trouble.
>   
Another option for converting an iterator to a sequence that might be 
helpful, particularly if result is large, is to use fromiter. For example:

    a[numpy.fromiter(xrange(0,40, 2), int)]


-tim


-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From charlesr.harris at gmail.com  Wed Nov  1 22:36:48 2006
From: charlesr.harris at gmail.com (Charles R Harris)
Date: Wed, 1 Nov 2006 20:36:48 -0700
Subject: Converting bool to float
In-Reply-To: <4549632B.4010001@ieee.org>
References: <f4f93d420611011233i60c8128ew63288d5620477c9c@mail.gmail.com>
	<454916EC.6010003@ee.byu.edu> <eib580$qdt$1@sea.gmane.org>
	<45492339.6050302@ee.byu.edu> <4549632B.4010001@ieee.org>
Message-ID: <e06186140611011936n6bd37065nbc0138f5d9f2a4c6@mail.gmail.com>

On 11/1/06, Tim Hochberg <tim.hochberg at ieee.org> wrote:
>
> Travis Oliphant wrote:
> > Robert Kern wrote:
> >
> >
> >> Travis Oliphant wrote:
> >>
> >>
> >>
> >>
> >>> It looks like 1.0-x is doing the right thing.
> >>>
> >>> The problem is 1.0*x for matrices is going to float64.  For arrays it
> >>> returns float32 just like the 1.0-x
> >>>
> >>>
> >>>
> >> Why is this the right thing? Python floats are float64.
> >>
> >>
> >>
> > Yeah, why indeed.  Must be something with the scalar coercion code...
>
> This is one of those things that pops up every few years. I suspect that
> the best thing to do here is to treat 1.0, and all Python floats as
> having a kind (float), but no precision. Or, equivalently treat them as
> the smallest precision floating point value. The rationale behind this
> is that otherwise float32 array will be promoted whenever they are
> multiplied by Python floating point scalars. If Python floats are
> treated as Float64 for purposes of determining output precision then
> anyone using float32 arrays is going to have to wrap all of their
> literals in float32 to prevent inadvertent upcasting to float64. This
> was the origin of the (rather clunky) numarray spacesaver flag.
>
> It's no skin off my nose either way, since I pretty much never use
> float32, but I suspect that treating python floats equivalently to
> float64 scalars would be a mistake. At the very least it deserves a bit
> of discussion.


Well, I think that the present convention of having the array float type
determine the output type when doing a binary op with a scalar makes sense.
The question is what to do when the initial array is an integer type and
needs to be promoted. Now I could see

1) coercing the scalar float to integer, which is probably consistent with
the treatment of integer types. (boo)
2) requiring explicit use of float types, i.e., float64(1.0), which is a bit
clumsy.
3) promoting to float64 by default and expecting the user to specify
float32(1.0) when needed.

I prefer 3, as float32 is probably not the most used data type. So the rule
would be

numpy_int array + python_int -- type numpy_int
numpy_int array + python_flt -- type float64
numpy_int array + numpy_flt -- type numpy_flt
numpy_flt array + python_flt --  type numpy_flt

Seems a bit much to remember, but things always get complicated when you
want to control the types. Mind that going from int64 to float64 can lead to
loss of precision.

Chuck
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20061101/f6125458/attachment.html>
-------------- next part --------------
-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
-------------- next part --------------
_______________________________________________
Numpy-discussion mailing list
Numpy-discussion at lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/numpy-discussion

From oliphant.travis at ieee.org  Wed Nov  1 23:41:34 2006
From: oliphant.travis at ieee.org (Travis Oliphant)
Date: Wed, 01 Nov 2006 21:41:34 -0700
Subject: dgesdd error
Message-ID: <454976FE.20504@ieee.org>


Apparently some dgesdd libraries don't actually compute the correct 
value for the work-space size if requested. 

This results in an ** ILLEGAL value and program termination from LAPACK.

I've added code in the latest SVN to that particular wrapper to make 
sure the query gives back the minimums at least, but I'd like to see who 
else has the problem.

On my system, doing numpy.linalg.svd on a 25 x 50 array pulled the trigger.

Anybody else have that problem with their lapack library?

-Travis


-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From george.sakkis at gmail.com  Wed Nov  1 23:42:14 2006
From: george.sakkis at gmail.com (George Sakkis)
Date: Thu, 02 Nov 2006 04:42:14 -0000
Subject: Reading records from file and sorting
References: <1162337923.747767.323100@b28g2000cwb.googlegroups.com>
	<20061101224620.GA29602@dogbert.sdsl.sun.ac.za>
Message-ID: <1162442534.508320.151770@e3g2000cwe.googlegroups.com>

Albert Strasheim wrote:

> Check the thread "Strange results when sorting array with fields" from
> about a week back. Travis made some changes to sorting in the presence
> of fields that should solve your problem, assuming your fields appear in
> the order you want to sort (i.e. you want to sort f1, f2, f3 and not
> something like f1, f3, f2).

I'm afraid this won't help in my case; I want to sort twice, once by f1
and once by f2. I guess I could make a second file with the fields
swapped but this seems more messy and inefficient than Francesc's
suggestion.

George


-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From fperez.net at gmail.com  Wed Nov  1 23:43:24 2006
From: fperez.net at gmail.com (Fernando Perez)
Date: Wed, 1 Nov 2006 21:43:24 -0700
Subject: dgesdd error
In-Reply-To: <454976FE.20504@ieee.org>
References: <454976FE.20504@ieee.org>
Message-ID: <db6b5ecc0611012043l579ed982if8d8c27ea3dede8@mail.gmail.com>

On 11/1/06, Travis Oliphant <oliphant.travis at ieee.org> wrote:
>
> Apparently some dgesdd libraries don't actually compute the correct
> value for the work-space size if requested.
>
> This results in an ** ILLEGAL value and program termination from LAPACK.
>
> I've added code in the latest SVN to that particular wrapper to make
> sure the query gives back the minimums at least, but I'd like to see who
> else has the problem.
>
> On my system, doing numpy.linalg.svd on a 25 x 50 array pulled the trigger.
>
> Anybody else have that problem with their lapack library?

Seems fine here:

In [4]: N.linalg.svd(N.random.rand(25,50));

In [5]: N.linalg.svd(N.random.rand(25,50));

In [6]: N.__version__
Out[6]: '1.0.1.dev3423'

Ubuntu Dapper (6.06) box, up to date.  I'm using the ubuntu-supplied
ATLAS and LAPACK, I can give you more info if needed.

Cheers,

f

-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From charlesr.harris at gmail.com  Wed Nov  1 23:52:34 2006
From: charlesr.harris at gmail.com (Charles R Harris)
Date: Wed, 1 Nov 2006 21:52:34 -0700
Subject: Reading records from file and sorting
In-Reply-To: <1162442534.508320.151770@e3g2000cwe.googlegroups.com>
References: <1162337923.747767.323100@b28g2000cwb.googlegroups.com>
	<20061101224620.GA29602@dogbert.sdsl.sun.ac.za>
	<1162442534.508320.151770@e3g2000cwe.googlegroups.com>
Message-ID: <e06186140611012052u16743231t9a508338f883b4@mail.gmail.com>

On 11/1/06, George Sakkis <george.sakkis at gmail.com> wrote:
>
> Albert Strasheim wrote:
>
> > Check the thread "Strange results when sorting array with fields" from
> > about a week back. Travis made some changes to sorting in the presence
> > of fields that should solve your problem, assuming your fields appear in
> > the order you want to sort (i.e. you want to sort f1, f2, f3 and not
> > something like f1, f3, f2).
>
> I'm afraid this won't help in my case; I want to sort twice, once by f1
> and once by f2. I guess I could make a second file with the fields
> swapped but this seems more messy and inefficient than Francesc's
> suggestion.


Do you actually want the two different orders, or do you want to sort on the
first field, then sort all the items with the same first field on the second
field?

Chuck
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20061101/4cdbee6b/attachment.html>
-------------- next part --------------
-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
-------------- next part --------------
_______________________________________________
Numpy-discussion mailing list
Numpy-discussion at lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/numpy-discussion

From sransom at nrao.edu  Wed Nov  1 23:55:27 2006
From: sransom at nrao.edu (Scott Ransom)
Date: Wed, 1 Nov 2006 23:55:27 -0500
Subject: Converting bool to float
In-Reply-To: <4549632B.4010001@ieee.org>
References: <f4f93d420611011233i60c8128ew63288d5620477c9c@mail.gmail.com>
	<454916EC.6010003@ee.byu.edu> <eib580$qdt$1@sea.gmane.org>
	<45492339.6050302@ee.byu.edu> <4549632B.4010001@ieee.org>
Message-ID: <20061102045527.GB11421@ssh.cv.nrao.edu>

On Wed, Nov 01, 2006 at 08:16:59PM -0700, Tim Hochberg wrote:
> Travis Oliphant wrote:
> > Robert Kern wrote:
> >> Travis Oliphant wrote:
> >>
> >>> It looks like 1.0-x is doing the right thing.
> >>>
> >>> The problem is 1.0*x for matrices is going to float64.  For arrays it 
> >>> returns float32 just like the 1.0-x
> >>>       
> >> Why is this the right thing? Python floats are float64.
> >>  
> > Yeah, why indeed.  Must be something with the scalar coercion code...
> 
> This is one of those things that pops up every few years. I suspect that 
> the best thing to do here is to treat 1.0, and all Python floats as 
> having a kind (float), but no precision. Or, equivalently treat them as 
> the smallest precision floating point value. The rationale behind this 
> is that otherwise float32 array will be promoted whenever they are 
> multiplied by Python floating point scalars. If Python floats are 
> treated as Float64 for purposes of determining output precision then 
> anyone using float32 arrays is going to have to wrap all of their 
> literals in float32 to prevent inadvertent upcasting to float64. This 
> was the origin of the (rather clunky) numarray spacesaver flag.

I'm one of those people who made serious use of that clunky
spacesaver flag for precisely this reason.  I deal with several GB
arrays of 32-bit floats (or 32-bit x2 complex numbers) on a
regular basis.  Having automatic upcasting from scalar operations
can be a royal pain.

Scott

-- 
Scott M. Ransom            Address:  NRAO
Phone:  (434) 296-0320               520 Edgemont Rd.
email:  sransom at nrao.edu             Charlottesville, VA 22903 USA
GPG Fingerprint: 06A9 9553 78BE 16DB 407B  FFCA 9BFA B6FF FFD3 2989

-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From charlesr.harris at gmail.com  Wed Nov  1 23:58:10 2006
From: charlesr.harris at gmail.com (Charles R Harris)
Date: Wed, 1 Nov 2006 21:58:10 -0700
Subject: dgesdd error
In-Reply-To: <db6b5ecc0611012043l579ed982if8d8c27ea3dede8@mail.gmail.com>
References: <454976FE.20504@ieee.org>
	<db6b5ecc0611012043l579ed982if8d8c27ea3dede8@mail.gmail.com>
Message-ID: <e06186140611012058j654c4432wf0c15b2f6bd9ac1b@mail.gmail.com>

On 11/1/06, Fernando Perez <fperez.net at gmail.com> wrote:
>
> On 11/1/06, Travis Oliphant <oliphant.travis at ieee.org> wrote:
> >
> > Apparently some dgesdd libraries don't actually compute the correct
> > value for the work-space size if requested.
> >
> > This results in an ** ILLEGAL value and program termination from LAPACK.
> >
> > I've added code in the latest SVN to that particular wrapper to make
> > sure the query gives back the minimums at least, but I'd like to see who
> > else has the problem.
> >
> > On my system, doing numpy.linalg.svd on a 25 x 50 array pulled the
> trigger.
> >
> > Anybody else have that problem with their lapack library?
>
> Seems fine here:


Fine here also: 1.0.1.dev3416

Fedora 6, ATLAS and LAPACK from extras.

Chuck
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20061101/ba081e24/attachment.html>
-------------- next part --------------
-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
-------------- next part --------------
_______________________________________________
Numpy-discussion mailing list
Numpy-discussion at lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/numpy-discussion

From charlesr.harris at gmail.com  Thu Nov  2 00:00:05 2006
From: charlesr.harris at gmail.com (Charles R Harris)
Date: Wed, 1 Nov 2006 22:00:05 -0700
Subject: dgesdd error
In-Reply-To: <e06186140611012058j654c4432wf0c15b2f6bd9ac1b@mail.gmail.com>
References: <454976FE.20504@ieee.org>
	<db6b5ecc0611012043l579ed982if8d8c27ea3dede8@mail.gmail.com>
	<e06186140611012058j654c4432wf0c15b2f6bd9ac1b@mail.gmail.com>
Message-ID: <e06186140611012100q1b72addeh495d886ddc60233c@mail.gmail.com>

On 11/1/06, Charles R Harris <charlesr.harris at gmail.com> wrote:
>
>
>
> On 11/1/06, Fernando Perez <fperez.net at gmail.com> wrote:
> >
> > On 11/1/06, Travis Oliphant <oliphant.travis at ieee.org> wrote:
> > >
> > > Apparently some dgesdd libraries don't actually compute the correct
> > > value for the work-space size if requested.
> > >
> > > This results in an ** ILLEGAL value and program termination from
> > LAPACK.
> > >
> > > I've added code in the latest SVN to that particular wrapper to make
> > > sure the query gives back the minimums at least, but I'd like to see
> > who
> > > else has the problem.
> > >
> > > On my system, doing numpy.linalg.svd on a 25 x 50 array pulled the
> > trigger.
> > >
> > > Anybody else have that problem with their lapack library?
> >
> > Seems fine here:
>
>
> Fine here also: 1.0.1.dev3416
>
> Fedora 6, ATLAS and LAPACK from extras.
>
> Chuck
>

Oh, and 32 bit.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20061101/2ff70265/attachment.html>
-------------- next part --------------
-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
-------------- next part --------------
_______________________________________________
Numpy-discussion mailing list
Numpy-discussion at lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/numpy-discussion

From robert.kern at gmail.com  Thu Nov  2 00:17:27 2006
From: robert.kern at gmail.com (Robert Kern)
Date: Wed, 01 Nov 2006 21:17:27 -0800
Subject: Converting bool to float
In-Reply-To: <4549632B.4010001@ieee.org>
References: <f4f93d420611011233i60c8128ew63288d5620477c9c@mail.gmail.com>	<454916EC.6010003@ee.byu.edu>	<eib580$qdt$1@sea.gmane.org>	<45492339.6050302@ee.byu.edu>
	<4549632B.4010001@ieee.org>
Message-ID: <eibv18$sj3$1@sea.gmane.org>

Tim Hochberg wrote:
> Travis Oliphant wrote:
>> Robert Kern wrote:
>>> Travis Oliphant wrote:   
>>>> It looks like 1.0-x is doing the right thing.
>>>>
>>>> The problem is 1.0*x for matrices is going to float64.  For arrays it 
>>>> returns float32 just like the 1.0-x
>>>>       
>>> Why is this the right thing? Python floats are float64.
>>>  
>> Yeah, why indeed.  Must be something with the scalar coercion code...
> 
> This is one of those things that pops up every few years. I suspect that 
> the best thing to do here is to treat 1.0, and all Python floats as 
> having a kind (float), but no precision. Or, equivalently treat them as 
> the smallest precision floating point value. The rationale behind this 
> is that otherwise float32 array will be promoted whenever they are 
> multiplied by Python floating point scalars. If Python floats are 
> treated as Float64 for purposes of determining output precision then 
> anyone using float32 arrays is going to have to wrap all of their 
> literals in float32 to prevent inadvertent upcasting to float64. This 
> was the origin of the (rather clunky) numarray spacesaver flag.
> 
> It's no skin off my nose either way, since I pretty much never use 
> float32, but I suspect that treating python floats equivalently to 
> float64 scalars would be a mistake. At the very least it deserves a bit 
> of discussion.

Well, they *are* 64-bit floating point numbers. You simply can't get around 
that. That's why we now have all of the scalar types: you can get any precision 
scalars that you want as long as you are explicit about it (and explicit is 
better than implicit). The spacesaver flag was the only solution before the 
various scalar types existed. I'd like to suggest that the discussion already 
occurred some time ago and has concluded in favor of the scalar types. 
Downcasting should be explicit.

However, whether or not float32 arrays operated with Python float scalars give 
float32 or float64 arrays is tangential to my question. Does anyone actually 
think that a Python float operated with a boolean array should give a float32 
result? Must we *up*cast a boolean array to float64 to preserve the precision of 
the scalar?

-- 
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless enigma
  that is made terrible by our own mad attempt to interpret it as though it had
  an underlying truth."
   -- Umberto Eco


-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From david at ar.media.kyoto-u.ac.jp  Thu Nov  2 00:22:10 2006
From: david at ar.media.kyoto-u.ac.jp (David Cournapeau)
Date: Thu, 02 Nov 2006 14:22:10 +0900
Subject: numpy.random.ranomd_integers returning only
	negative numbers ?
Message-ID: <45498082.1090201@ar.media.kyoto-u.ac.jp>

Hi,

    I want to generate some random integers,let's say in the range [- 
2^15, 2^16]. Why doing:

    noise = numpy.random.random_integers(- (2 ** 15), (2 * 15), 22050)

    gives only negative numbers ?

    Cheers,

    David


-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From fperez.net at gmail.com  Thu Nov  2 00:25:57 2006
From: fperez.net at gmail.com (Fernando Perez)
Date: Wed, 1 Nov 2006 22:25:57 -0700
Subject: numpy.random.ranomd_integers returning only
	negative numbers ?
In-Reply-To: <45498082.1090201@ar.media.kyoto-u.ac.jp>
References: <45498082.1090201@ar.media.kyoto-u.ac.jp>
Message-ID: <db6b5ecc0611012125y3df72d4fmdfa87179695b4aef@mail.gmail.com>

On 11/1/06, David Cournapeau <david at ar.media.kyoto-u.ac.jp> wrote:
> Hi,
>
>     I want to generate some random integers,let's say in the range [-
> 2^15, 2^16]. Why doing:
>
>     noise = numpy.random.random_integers(- (2 ** 15), (2 * 15), 22050)
>
>     gives only negative numbers ?

In [3]: noise = numpy.random.random_integers(- (2 ** 15), (2 * 15), 22050)

In [4]: noise[noise>0].shape
Out[4]: (17,)

In [5]: noise[noise<0].shape
Out[5]: (22033,)

In [6]: noise = numpy.random.random_integers(-(2**15), (2 ** 15), 22050)

In [7]: noise[noise>0].shape
Out[7]: (11006,)

In [8]: noise[noise>0].shape
Out[8]: (11006,)

In [9]: 17./22033
Out[9]: 0.00077156991785049694

In [10]: 2.0*15/2**15
Out[10]: 0.00091552734375

close enough ;)

Cheers,

f

-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From kwgoodman at gmail.com  Thu Nov  2 00:29:31 2006
From: kwgoodman at gmail.com (Keith Goodman)
Date: Wed, 1 Nov 2006 21:29:31 -0800
Subject: numpy.random.ranomd_integers returning only
	negative numbers ?
In-Reply-To: <45498082.1090201@ar.media.kyoto-u.ac.jp>
References: <45498082.1090201@ar.media.kyoto-u.ac.jp>
Message-ID: <f4f93d420611012129x765737fax9bc0b65ed7400345@mail.gmail.com>

On 11/1/06, David Cournapeau <david at ar.media.kyoto-u.ac.jp> wrote:
> Hi,
>
>     I want to generate some random integers,let's say in the range [-
> 2^15, 2^16]. Why doing:
>
>     noise = numpy.random.random_integers(- (2 ** 15), (2 * 15), 22050)
>
>     gives only negative numbers ?

I guess 2^15 is too big to be an int.

Would this work?

int(N.random.uniform(1e15,1e16))

-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From robert.kern at gmail.com  Thu Nov  2 00:42:06 2006
From: robert.kern at gmail.com (Robert Kern)
Date: Wed, 01 Nov 2006 21:42:06 -0800
Subject: numpy.random.ranomd_integers returning only
 negative numbers ?
In-Reply-To: <f4f93d420611012129x765737fax9bc0b65ed7400345@mail.gmail.com>
References: <45498082.1090201@ar.media.kyoto-u.ac.jp>
	<f4f93d420611012129x765737fax9bc0b65ed7400345@mail.gmail.com>
Message-ID: <eic0fe$v8o$1@sea.gmane.org>

Keith Goodman wrote:
> On 11/1/06, David Cournapeau <david at ar.media.kyoto-u.ac.jp> wrote:
>> Hi,
>>
>>     I want to generate some random integers,let's say in the range [-
>> 2^15, 2^16]. Why doing:
>>
>>     noise = numpy.random.random_integers(- (2 ** 15), (2 * 15), 22050)
>>
>>     gives only negative numbers ?
> 
> I guess 2^15 is too big to be an int.

No, it isn't. As Fernando pointed out, if David's code is what he typed here, 
then his upper bound is (2 * 15) == 30 not (2 ** 15) == 32768 as he intended 
intended.

-- 
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless enigma
  that is made terrible by our own mad attempt to interpret it as though it had
  an underlying truth."
   -- Umberto Eco


-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From david at ar.media.kyoto-u.ac.jp  Thu Nov  2 00:41:46 2006
From: david at ar.media.kyoto-u.ac.jp (David Cournapeau)
Date: Thu, 02 Nov 2006 14:41:46 +0900
Subject: numpy.random.ranomd_integers returning only
 negative numbers ?
In-Reply-To: <db6b5ecc0611012125y3df72d4fmdfa87179695b4aef@mail.gmail.com>
References: <45498082.1090201@ar.media.kyoto-u.ac.jp>
	<db6b5ecc0611012125y3df72d4fmdfa87179695b4aef@mail.gmail.com>
Message-ID: <4549851A.4050505@ar.media.kyoto-u.ac.jp>

Fernando Perez wrote:
> On 11/1/06, David Cournapeau <david at ar.media.kyoto-u.ac.jp> wrote:
>   
>> Hi,
>>
>>     I want to generate some random integers,let's say in the range [-
>> 2^15, 2^16]. Why doing:
>>
>>     noise = numpy.random.random_integers(- (2 ** 15), (2 * 15), 22050)
>>
>>     gives only negative numbers ?
>>     
>
> In [3]: noise = numpy.random.random_integers(- (2 ** 15), (2 * 15), 22050)
>
> In [4]: noise[noise>0].shape
> Out[4]: (17,)
>
> In [5]: noise[noise<0].shape
> Out[5]: (22033,)
>
> In [6]: noise = numpy.random.random_integers(-(2**15), (2 ** 15), 22050)
>
> In [7]: noise[noise>0].shape
> Out[7]: (11006,)
>
> In [8]: noise[noise>0].shape
> Out[8]: (11006,)
>
> In [9]: 17./22033
> Out[9]: 0.00077156991785049694
>
> In [10]: 2.0*15/2**15
> Out[10]: 0.00091552734375
>
> close enough ;)
>   
Ok, I deserve my email to be called the most stupid email of the week... 
Sorry for the noise,

David

-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From charlesr.harris at gmail.com  Thu Nov  2 00:55:39 2006
From: charlesr.harris at gmail.com (Charles R Harris)
Date: Wed, 1 Nov 2006 22:55:39 -0700
Subject: Converting bool to float
In-Reply-To: <eibv18$sj3$1@sea.gmane.org>
References: <f4f93d420611011233i60c8128ew63288d5620477c9c@mail.gmail.com>
	<454916EC.6010003@ee.byu.edu> <eib580$qdt$1@sea.gmane.org>
	<45492339.6050302@ee.byu.edu> <4549632B.4010001@ieee.org>
	<eibv18$sj3$1@sea.gmane.org>
Message-ID: <e06186140611012155x38f47a9ard4c8126a13a344d7@mail.gmail.com>

On 11/1/06, Robert Kern <robert.kern at gmail.com> wrote:
>
> Tim Hochberg wrote:
> > Travis Oliphant wrote:


<snip>

However, whether or not float32 arrays operated with Python float scalars
> give
> float32 or float64 arrays is tangential to my question. Does anyone
> actually
> think that a Python float operated with a boolean array should give a
> float32
> result? Must we *up*cast a boolean array to float64 to preserve the
> precision of
> the scalar?


Probably doesn't matter most of the time, I suppose, who is going to check?
I tend to think doubles because they are a bit faster on the x86
architecture and because they are a pretty common default.

Chuck
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20061101/88fa6bc3/attachment.html>
-------------- next part --------------
-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
-------------- next part --------------
_______________________________________________
Numpy-discussion mailing list
Numpy-discussion at lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/numpy-discussion

From kgyhmgt at paniere.com  Thu Nov  2 01:10:10 2006
From: kgyhmgt at paniere.com (G.)
Date: Thu, 2 Nov 2006 14:10:10 +0800
Subject: Wishesquot case
Message-ID: <001101c6fe45$8d93f3c0$9fef84db@pj3fe882826648>


So Whenever of Call of. Help Need Else Close a.
Girl Crash of Burn Wide Maybe Promise.
Now share in life is dream forever let. Began or to live again dedicate these pages.
Now share in life is dream forever let.
Stand Every of Falling Mend Broken Endless Whiter Shade Pale! Access links Featured or Week Thank Dido Wake me.
Music Audio Search site particular is song Crescendo download plugin. Others a found comfort or.
Angel red Lucky Mind Youre Tomorrow Comes. See pictures contains lyrics.
Version Freddie Aguilar am. Work in Keep Alive Truly Madly Deeply Everytime go Away. Time Ever saw Want Christmas am Wanna Miss.
How do Frozen Lady? Beneath Wings at Moment Make Feel.
Need Else Close am. S Mclachlan a First Time Ever is saw Want in Christmas Wanna.
So Whenever of Call of. Eyes Sometimes am Touch in More Than.
Began or to live in again dedicate these? Meditate or enjoy in Remember reload is updates Important am Copyright Notice if.
Hope in Colors Tell him Song Brought Hearts of Rain in.
Being solely Internet mails chats now share life of. Spend or Lifetime Young a! Hate Swear try Into is Shoulder of Annies Morn Vision a Hard.
Make of Feel Sweet a Friends or mws or ill is Take Breath.
Sang Especially Help Need Else Close Lost Babe.
Chords music of quotes excerpts from or messages net. Help Need Else Close a.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20061102/8580c6d7/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: Shade.gif
Type: image/gif
Size: 9663 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20061102/8580c6d7/attachment.gif>
-------------- next part --------------
-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
-------------- next part --------------
_______________________________________________
Numpy-discussion mailing list
Numpy-discussion at lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/numpy-discussion

From david at ar.media.kyoto-u.ac.jp  Thu Nov  2 02:12:00 2006
From: david at ar.media.kyoto-u.ac.jp (David Cournapeau)
Date: Thu, 02 Nov 2006 16:12:00 +0900
Subject: [ANN[ pysamplerate,
	a library to do high quality samplerate convertion
Message-ID: <45499A40.8070204@ar.media.kyoto-u.ac.jp>

(if those announcements are not welcome on the lists, please tell me)

Hi there,

    I've just developed a small package to wrap Source Rabbit Code from 
Erik de Castro de Lopo, which is a library to do high quality samplerate 
conversion (http://www.mega-nerd.com/SRC/). You can download it here:

    
http://www.ar.media.kyoto-u.ac.jp/members/david/softwares/pysamplerate/pysamplerate-0.1.tar.gz
    
http://www.ar.media.kyoto-u.ac.jp/members/david/softwares/pysamplerate/pysamplerate-0.1.zip

    Example of error conversions:
   
    
http://www.ar.media.kyoto-u.ac.jp/members/david/softwares/pysamplerate/example1.png

    Basically, you just import resample from pysamplerate, and 
resample(sins, fr/fs) gives you a converted signal at sampling rate fr, 
with fs the original sampling rate. You can also select the conversion 
method (eg 'sinc_best', etc...). The pydocs + examples should be enough
   
    This package can be useful for people who deal with music signal 
processing and co, when scipy.resample is not enough. Only float32 and 
float64 input are supported for now, I will polish the package when I 
will have some more time,

    cheers,

    David

-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From fperez.net at gmail.com  Thu Nov  2 02:23:49 2006
From: fperez.net at gmail.com (Fernando Perez)
Date: Thu, 2 Nov 2006 00:23:49 -0700
Subject: [SciPy-user] [ANN[ pysamplerate,
	a library to do high quality samplerate convertion
In-Reply-To: <45499A40.8070204@ar.media.kyoto-u.ac.jp>
References: <45499A40.8070204@ar.media.kyoto-u.ac.jp>
Message-ID: <db6b5ecc0611012323k4f9813cfl712aeac485945494@mail.gmail.com>

On 11/2/06, David Cournapeau <david at ar.media.kyoto-u.ac.jp> wrote:
> (if those announcements are not welcome on the lists, please tell me)

Frankly, if an announcement for a free Python signal processing
library is not welcome on the scipy lists, I don't know where it would
be ;)

As a minor note though: please remember to make a note of this in the
Topical Software wiki; it's good for that page to remain up to date
with newly released packages and libraries.

Thanks!

f

-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From Mailer-Daemon at lists.sourceforge.net  Thu Nov  2 06:06:46 2006
From: Mailer-Daemon at lists.sourceforge.net (Mail Delivery System)
Date: Thu, 02 Nov 2006 03:06:46 -0800
Subject: Warning: message 1GfAoO-0007im-Sh delayed 24
	hours
Message-ID: <E1GfaPG-0000jm-La@externalmx-1.sourceforge.net>

This message was created automatically by mail delivery software.
A message that you sent has not yet been delivered to one or more of its
recipients after more than 24 hours on the queue on externalmx-1.sourceforge.net.

The message identifier is:     1GfAoO-0007im-Sh
The subject of the message is: looked like me the full sized HARLEY and began to her, with this but as
The date of the message is:    Wed, 01 Nov 2006 15:46:73 +0800

The address to which the message has not yet been delivered is:

  numpy-discussion at lists.sourceforge.net
    Delay reason: SMTP error from remote mailer after RCPT TO:<numpy-discussion at lists.sourceforge.net>:
    host mail.sourceforge.net [66.35.250.206]: 451-Could not complete sender verify callout
    451-Could not complete sender verify callout for
    451-<numpy-discussion at lists.sourceforge.net>.
    451-The mail server(s) for the domain may be temporarily unreachable, or
    451-they may be permanently unreachable from this server. In the latter case,
    451-you need to change the address or create an MX record for its domain
    451-if it is 

No action is required on your part. Delivery attempts will continue for
some time, and this warning may be repeated at intervals if the message
remains undelivered. Eventually the mail delivery software will give up,
and when that happens, the message will be returned to you.

-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From svetosch at gmx.net  Thu Nov  2 07:23:38 2006
From: svetosch at gmx.net (Sven Schreiber)
Date: Thu, 02 Nov 2006 13:23:38 +0100
Subject: numpy book
Message-ID: <4549E34A.4060507@gmx.net>

Hi,
now that 1.0 is out I would like to ask again about the status of the
numpy ebook. Is there any mechanism in place where owners can get the
updates?

Note that this is not a request to Travis to send me the latest version
by private email. That would be inefficient and my need is not that
urgent. Nevertheless I think that issue should be settled.

Excuse me if I've missed something obvious in that respect.

Thanks,
Sven

-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From markusro at element.fkp.physik.tu-darmstadt.de  Thu Nov  2 10:13:34 2006
From: markusro at element.fkp.physik.tu-darmstadt.de (Markus Rosenstihl)
Date: Thu, 2 Nov 2006 16:13:34 +0100
Subject: Can' compile numpy 1.02rc3 on OSX 10.3.9
In-Reply-To: <5228203bb5a6003c571d52a5e94ef370@element.fkp.physik.tu-darmstadt.de>
References: <5228203bb5a6003c571d52a5e94ef370@element.fkp.physik.tu-darmstadt.de>
Message-ID: <089ba8ec7c78b92f4563ce0c7eba4d85@element.fkp.physik.tu-darmstadt.de>

Hi!
This, and similar problems with other programs were fixed when I  
installed
10.4 SDK from Apple Developer Conenction.
Yes, the 10.4 SDK can be installed on 10.3.9.

Regards
Markus


Am 19.10.2006 um 23:56 schrieb Markus Rosenstihl:

> Hi!
> I try to compile numpy rc3 on Panther and get following errors.
> (I start build with "python2.3 setup.py build" to be sure to use the
> python shipped with OS X. I din't manage to compile Python2.5 either
> yet with similar errors)
> Does anynbody has an Idea?
> gcc-3.3
> XCode 1.5
> November gcc updater is installed
>
>
> regards
>
> Markus
>
> python2.3 setup.py build
> ...
> compile options:
> '-Ibuild/src.darwin-7.9.0-Power_Macintosh-2.3/numpy/core/src
> -Inumpy/core/include
> -Ibuild/src.darwin-7.9.0-Power_Macintosh-2.3/numpy/core
> -Inumpy/core/src -Inumpy/core/include
> -I/System/Library/Frameworks/Python.framework/Versions/2.3/include/
> python2.3 -c'
> gcc: numpy/core/src/multiarraymodule.c
> In file included from numpy/core/src/arrayobject.c:511,
>                   from numpy/core/src/multiarraymodule.c:63:
> numpy/core/src/arraytypes.inc.src: In function `LONGDOUBLE_scan':
> numpy/core/src/arraytypes.inc.src:883: warning: long double format,
> npy_longdouble arg (arg 3)
> gcc -Wl,-F. -Wl,-F. -bundle -framework Python
> build/temp.darwin-7.9.0-Power_Macintosh-2.3/numpy/core/src/
> multiarraymodule.o -o
> build/lib.darwin-7.9.0-Power_Macintosh-2.3/numpy/core/multiarray.so
> ld: Undefined symbols:
> _fstatvfs referenced from Python expected to be defined in libSystem
> _lchown referenced from Python expected to be defined in libSystem
> _statvfs referenced from Python expected to be defined in libSystem
> ld: Undefined symbols:
> _fstatvfs referenced from Python expected to be defined in libSystem
> _lchown referenced from Python expected to be defined in libSystem
> _statvfs referenced from Python expected to be defined in libSystem
> error: Command "gcc -Wl,-F. -Wl,-F. -bundle -framework Python
> build/temp.darwin-7.9.0-Power_Macintosh-2.3/numpy/core/src/
> multiarraymodule.o -o
> build/lib.darwin-7.9.0-Power_Macintosh-2.3/numpy/core/multiarray.so"
> failed with exit status 1
>
>
> ----------------------------------------------------------------------- 
> --
> Using Tomcat but need to do more? Need to support web services,  
> security?
> Get stuff done quickly with pre-integrated technology to make your job  
> easier
> Download IBM WebSphere Application Server v.1.0.1 based on Apache  
> Geronimo
> http://sel.as-us.falkag.net/sel? 
> cmd=lnk&kid=120709&bid=263057&dat=121642
> _______________________________________________
> Numpy-discussion mailing list
> Numpy-discussion at lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/numpy-discussion


-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From oliphant at ee.byu.edu  Thu Nov  2 10:41:11 2006
From: oliphant at ee.byu.edu (Travis Oliphant)
Date: Thu, 02 Nov 2006 08:41:11 -0700
Subject: numpy book
In-Reply-To: <4549E34A.4060507@gmx.net>
References: <4549E34A.4060507@gmx.net>
Message-ID: <454A1197.1020606@ee.byu.edu>


>Note that this is not a request to Travis to send me the latest version
>by private email. That would be inefficient and my need is not that
>urgent. Nevertheless I think that issue should be settled.
>  
>
There will be an update, soon. I'm currently working on the index, 
corrections, and formatting issues.

The update will be sent in conjunction with the release of 1.0.1 which I 
am targetting in 2 weeks.

Thanks for everybody's patience.  If you need an update now, I'm happy 
to send it to you by private email. 

Best regards,

-Travis O.



-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From oliphant at ee.byu.edu  Thu Nov  2 10:48:47 2006
From: oliphant at ee.byu.edu (Travis Oliphant)
Date: Thu, 02 Nov 2006 08:48:47 -0700
Subject: Converting bool to float
In-Reply-To: <eibv18$sj3$1@sea.gmane.org>
References: <f4f93d420611011233i60c8128ew63288d5620477c9c@mail.gmail.com>	<454916EC.6010003@ee.byu.edu>	<eib580$qdt$1@sea.gmane.org>	<45492339.6050302@ee.byu.edu>	<4549632B.4010001@ieee.org>
	<eibv18$sj3$1@sea.gmane.org>
Message-ID: <454A135F.2040009@ee.byu.edu>

Robert Kern wrote:

>Tim Hochberg wrote:
>  
>
>>Travis Oliphant wrote:
>>    
>>
>>>Robert Kern wrote:
>>>      
>>>
>>>>Travis Oliphant wrote:   
>>>>        
>>>>
>>>>>It looks like 1.0-x is doing the right thing.
>>>>>
>>>>>The problem is 1.0*x for matrices is going to float64.  For arrays it 
>>>>>returns float32 just like the 1.0-x
>>>>>      
>>>>>          
>>>>>
>>>>Why is this the right thing? Python floats are float64.
>>>> 
>>>>        
>>>>
>>>Yeah, why indeed.  Must be something with the scalar coercion code...
>>>      
>>>
>>This is one of those things that pops up every few years. I suspect that 
>>the best thing to do here is to treat 1.0, and all Python floats as 
>>having a kind (float), but no precision. Or, equivalently treat them as 
>>the smallest precision floating point value. The rationale behind this 
>>is that otherwise float32 array will be promoted whenever they are 
>>multiplied by Python floating point scalars. If Python floats are 
>>treated as Float64 for purposes of determining output precision then 
>>anyone using float32 arrays is going to have to wrap all of their 
>>literals in float32 to prevent inadvertent upcasting to float64. This 
>>was the origin of the (rather clunky) numarray spacesaver flag.
>>
>>It's no skin off my nose either way, since I pretty much never use 
>>float32, but I suspect that treating python floats equivalently to 
>>float64 scalars would be a mistake. At the very least it deserves a bit 
>>of discussion.
>>    
>>
>
>Well, they *are* 64-bit floating point numbers. You simply can't get around 
>that. That's why we now have all of the scalar types: you can get any precision 
>scalars that you want as long as you are explicit about it (and explicit is 
>better than implicit). The spacesaver flag was the only solution before the 
>various scalar types existed. I'd like to suggest that the discussion already 
>occurred some time ago and has concluded in favor of the scalar types. 
>Downcasting should be explicit.
>
>However, whether or not float32 arrays operated with Python float scalars give 
>float32 or float64 arrays is tangential to my question. Does anyone actually 
>think that a Python float operated with a boolean array should give a float32 
>result? Must we *up*cast a boolean array to float64 to preserve the precision of 
>the scalar?
>
>  
>
The first basic rule is that scalars don't control the precision of the 
output when doing mixed-type calculations *except* when they are of a 
fundamentally different kind. 

Then (if a different kind of scalar is used), the rule is that the 
arrays will be upcast to the "lowest" precision in the group to preserve 
overall precision.   So, when a bool is combined with a "float" kind of 
scalar, the result is float32 because that preserves precision of the 
bool.  Remember it is array precision that takes precedence over scalars 
in mixed type array-scalar operations.

This is the rule.  I agree that this rule is probably flawed in certain 
circumstances.

So, what should be done about it at this point?  Do you think a change 
is acceptable for 1.0.1 or does it need to wait a year until 1.1?

-Travis


















-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From jontwang at gmail.com  Thu Nov  2 10:52:08 2006
From: jontwang at gmail.com (Jonathan Wang)
Date: Thu, 2 Nov 2006 09:52:08 -0600
Subject: Argmax and max inconsistent for object types
Message-ID: <4f7636bf0611020752t3e079d06h1f41e3a12e226de2@mail.gmail.com>

On numpy version 1.0, argmax and max give inconsistent results for an array
of objects. I've seen this problem in both Python native datetime and
mx.DateTime types:

In [22]: print nativeDates
[datetime.datetime(2006, 10, 18, 10, 11, 27), datetime.datetime(2006, 10,
18, 10, 16, 20), datetime.datetime(2006, 10, 18, 10, 21, 23),
datetime.datetime(2006, 10, 18, 10, 31, 13), datetime.datetime(2006, 10, 18,
10, 39, 49), datetime.datetime(2006, 10, 18, 10, 53, 19),
datetime.datetime(2006,
10, 18, 11, 23, 18), datetime.datetime(2006, 10, 18, 17, 18, 43),
datetime.datetime(2006, 10, 18, 17, 21, 49), datetime.datetime(2006, 10, 18,
17, 24, 28), datetime.datetime(2006, 10, 18, 17, 28, 29),
datetime.datetime(2006,
10, 18, 17, 31, 7), datetime.datetime(2006, 10, 18, 17, 36, 26),
datetime.datetime(2006, 10, 19, 10, 17, 45), datetime.datetime(2006, 10, 19,
11, 23, 19), datetime.datetime(2006, 10, 19, 11, 58, 18),
datetime.datetime(2006,
10, 19, 10, 27, 40), datetime.datetime(2006, 10, 19, 13, 17, 14),
datetime.datetime(2006, 10, 19, 13, 21, 17), datetime.datetime(2006, 10, 19,
13, 23, 52), datetime.datetime(2006, 10, 19, 13, 29, 1)]

In [23]: numpy.argmax(nativeDates)
Out[23]: 0

In [24]: numpy.max(nativeDates)
Out[24]: datetime.datetime(2006, 10, 19, 13, 29, 1)

In [25]: nativeDates[0]
Out[25]: datetime.datetime(2006, 10, 18, 10, 11, 27)

I get the same results if I create an array from the list first:

In [28]: dateArr = numpy.array(nativeDates, dtype=object)

In [29]: print dateArr
[2006-10-18 10:11:27 2006-10-18 10:16:20 2006-10-18 10:21:23
 2006-10-18 10:31:13 2006-10-18 10:39:49 2006-10-18 10:53:19
 2006-10-18 11:23:18 2006-10-18 17:18:43 2006-10-18 17:21:49
 2006-10-18 17:24:28 2006-10-18 17:28:29 2006-10-18 17:31:07
 2006-10-18 17:36:26 2006-10-19 10:17:45 2006-10-19 11:23:19
 2006-10-19 11:58:18 2006-10-19 10:27:40 2006-10-19 13:17:14
 2006-10-19 13:21:17 2006-10-19 13:23:52 2006-10-19 13:29:01]

In [30]: numpy.argmax(dateArr)
Out[30]: 0

In [31]: numpy.max(dateArr)
Out[31]: datetime.datetime(2006, 10, 19, 13, 29, 1)

In [32]: dateArr[0]
Out[32]: datetime.datetime(2006, 10, 18, 10, 11, 27)

My guess is that it's related to some underlying memory layout; I've gotten
different results when running this.

Jonathan
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20061102/565ff27a/attachment.html>
-------------- next part --------------
-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
-------------- next part --------------
_______________________________________________
Numpy-discussion mailing list
Numpy-discussion at lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/numpy-discussion

From oliphant at ee.byu.edu  Thu Nov  2 10:55:09 2006
From: oliphant at ee.byu.edu (Travis Oliphant)
Date: Thu, 02 Nov 2006 08:55:09 -0700
Subject: Argmax and max inconsistent for object types
In-Reply-To: <4f7636bf0611020752t3e079d06h1f41e3a12e226de2@mail.gmail.com>
References: <4f7636bf0611020752t3e079d06h1f41e3a12e226de2@mail.gmail.com>
Message-ID: <454A14DD.3000502@ee.byu.edu>

Jonathan Wang wrote:

> On numpy version 1.0, argmax and max give inconsistent results for an 
> array of objects. I've seen this problem in both Python native 
> datetime and mx.DateTime types:


There is a bug in argmax for OBJECT arrays in 1.0  (it's fixed in SVN 
and will be in 1.0.1)

-Travis


-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From fperez.net at gmail.com  Thu Nov  2 11:00:31 2006
From: fperez.net at gmail.com (Fernando Perez)
Date: Thu, 2 Nov 2006 09:00:31 -0700
Subject: numpy book
In-Reply-To: <454A1197.1020606@ee.byu.edu>
References: <4549E34A.4060507@gmx.net> <454A1197.1020606@ee.byu.edu>
Message-ID: <db6b5ecc0611020800u367eb6fdvccf194c6e1aecd14@mail.gmail.com>

On 11/2/06, Travis Oliphant <oliphant at ee.byu.edu> wrote:
>
> >Note that this is not a request to Travis to send me the latest version
> >by private email. That would be inefficient and my need is not that
> >urgent. Nevertheless I think that issue should be settled.
> >
> >
> There will be an update, soon. I'm currently working on the index,
> corrections, and formatting issues.

If I may make a suggestion, it would greatly increase the usability of
the PDF if it had internal and external clickable links, as well as a
PDF table of contents (the one that appears on the left sidebar of
acroread).

Since I know the original is in Lyx, here's my standard preamble for
these kinds of documents, which includes all the goodies for
generating such a PDF, with some color and formatting tweaks so that
it prints legibly on black and white as well as looking nice on
screen.  If you add this to your preamble, a simple 'view->pdflatex'
should work.  I'll be happy to help if you have any issues.

Cheers,

f

%%% My preamble; tweak and season to taste:

% This gives us a better font in URL links (otherwise the default
% MonoSpace font is bitmapped, and it looks horrible in PDF)
\usepackage{courier}

\usepackage{fullpage}

\usepackage{color}  % so we can use red for the fixme warnings

% The hyperref package gives us a pdf with properly built
% internal navigation ('pdf bookmarks' for the table of contents,
% internal cross-reference links, web links for URLs, etc.)

% A few colors to replace the defaults for certain link types
\definecolor{darkorange}{rgb}{.71,0.21,0.01}
\definecolor{darkgreen}{rgb}{.12,.54,.11}

\usepackage[
  %pdftex,  % needed for pdflatex
  breaklinks=true,  % so long urls are correctly broken across lines
  colorlinks=true,
  urlcolor=blue,
  linkcolor=darkorange,
  citecolor=darkgreen,
  ]{hyperref}

% This helps prevent overly long lines that stretch beyond the margins
\sloppy

% Define a \fixme command to mark visually things needing fixing in the draft.
% For final printing or to simply disable these bright warnings, simply
% uncomment the \renewcommand redefinition below

\newcommand{\fixme}[1] {
\textcolor{red}{
{\fbox{ {\bf FIX}
\ensuremath{\blacktriangleright \blacktriangleright \blacktriangleright}}
{\bf #1}
\fbox{\ensuremath{ \blacktriangleleft \blacktriangleleft \blacktriangleleft }
} } }
}
% Uncomment the next line to make the \fixme command be a no-op
%\renewcommand{\fixme}[1]{}

%%% If you also want to use the listings package for nicely formatted
%%% Python source code, this configuration produces good on-paper and
%%% on-screen results:

\definecolor{orange}{cmyk}{0,0.4,0.8,0.2}
% Use and configure listings package for nicely formatted code
\usepackage{listings}
\lstset{
  language=Python,
  basicstyle=\small\ttfamily,
  commentstyle=\ttfamily\color{blue},
  stringstyle=\ttfamily\color{orange},
  showstringspaces=false,
  breaklines=true,
  postbreak = \space\dots
}

-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From oliphant at ee.byu.edu  Thu Nov  2 11:03:20 2006
From: oliphant at ee.byu.edu (Travis Oliphant)
Date: Thu, 02 Nov 2006 09:03:20 -0700
Subject: numpy book
In-Reply-To: <db6b5ecc0611020800u367eb6fdvccf194c6e1aecd14@mail.gmail.com>
References: <4549E34A.4060507@gmx.net> <454A1197.1020606@ee.byu.edu>
	<db6b5ecc0611020800u367eb6fdvccf194c6e1aecd14@mail.gmail.com>
Message-ID: <454A16C8.7060907@ee.byu.edu>

Fernando Perez wrote:

>On 11/2/06, Travis Oliphant <oliphant at ee.byu.edu> wrote:
>  
>
>>>Note that this is not a request to Travis to send me the latest version
>>>by private email. That would be inefficient and my need is not that
>>>urgent. Nevertheless I think that issue should be settled.
>>>
>>>
>>>      
>>>
>>There will be an update, soon. I'm currently working on the index,
>>corrections, and formatting issues.
>>    
>>
>
>If I may make a suggestion, it would greatly increase the usability of
>the PDF if it had internal and external clickable links, as well as a
>PDF table of contents (the one that appears on the left sidebar of
>acroread).
>
>Since I know the original is in Lyx, here's my standard preamble for
>these kinds of documents, which includes all the goodies for
>generating such a PDF, with some color and formatting tweaks so that
>it prints legibly on black and white as well as looking nice on
>screen.  If you add this to your preamble, a simple 'view->pdflatex'
>should work.  I'll be happy to help if you have any issues.
>  
>

Thanks for the pre-amble.  Does it require pdflatex?  I use ps2pdf 
because to generate the shaded boxes and graphics.  I could probably try 
to do it with pdflatex and png files but I haven't tried, yet.

-Travis



-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From fperez.net at gmail.com  Thu Nov  2 11:07:21 2006
From: fperez.net at gmail.com (Fernando Perez)
Date: Thu, 2 Nov 2006 09:07:21 -0700
Subject: numpy book
In-Reply-To: <454A16C8.7060907@ee.byu.edu>
References: <4549E34A.4060507@gmx.net> <454A1197.1020606@ee.byu.edu>
	<db6b5ecc0611020800u367eb6fdvccf194c6e1aecd14@mail.gmail.com>
	<454A16C8.7060907@ee.byu.edu>
Message-ID: <db6b5ecc0611020807y33361015p2959afdaeee020cb@mail.gmail.com>

On 11/2/06, Travis Oliphant <oliphant at ee.byu.edu> wrote:

> Thanks for the pre-amble.  Does it require pdflatex?  I use ps2pdf
> because to generate the shaded boxes and graphics.  I could probably try
> to do it with pdflatex and png files but I haven't tried, yet.

I just tested it with ps2pdf on a document, and it worked fine.  If
you have problems, play with commenting (or not) the first option line
to hyperref:

\usepackage[
  %pdftex,  % needed for pdflatex


In the copy I gave you it's commented out, so it /should/ work for
ps2pdf.  But I've had documents where I've had to uncomment it, and I
think at that point they stop working with ps2pdf, and yes, you lose
the usage of PostScript specials in that case (which I use for things
like whole-page DRAFT watermarks).

Cheers,

f

-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From tim.hochberg at ieee.org  Thu Nov  2 11:15:00 2006
From: tim.hochberg at ieee.org (Tim Hochberg)
Date: Thu, 02 Nov 2006 09:15:00 -0700
Subject: Converting bool to float
In-Reply-To: <454A135F.2040009@ee.byu.edu>
References: <f4f93d420611011233i60c8128ew63288d5620477c9c@mail.gmail.com>	<454916EC.6010003@ee.byu.edu>	<eib580$qdt$1@sea.gmane.org>	<45492339.6050302@ee.byu.edu>	<4549632B.4010001@ieee.org>	<eibv18$sj3$1@sea.gmane.org>
	<454A135F.2040009@ee.byu.edu>
Message-ID: <454A1984.7030205@ieee.org>

Travis Oliphant wrote:
> Robert Kern wrote:
>
>   
>> Tim Hochberg wrote:
>>  
>>
>>     
>>> Travis Oliphant wrote:
>>>    
>>>
>>>       
>>>> Robert Kern wrote:
>>>>      
>>>>
>>>>         
>>>>> Travis Oliphant wrote:   
>>>>>        
>>>>>
>>>>>           
>>>>>> It looks like 1.0-x is doing the right thing.
>>>>>>
>>>>>> The problem is 1.0*x for matrices is going to float64.  For arrays it 
>>>>>> returns float32 just like the 1.0-x
>>>>>>      
>>>>>>          
>>>>>>
>>>>>>             
>>>>> Why is this the right thing? Python floats are float64.
>>>>>
>>>>>        
>>>>>
>>>>>           
>>>> Yeah, why indeed.  Must be something with the scalar coercion code...
>>>>      
>>>>
>>>>         
>>> This is one of those things that pops up every few years. I suspect that 
>>> the best thing to do here is to treat 1.0, and all Python floats as 
>>> having a kind (float), but no precision. Or, equivalently treat them as 
>>> the smallest precision floating point value. The rationale behind this 
>>> is that otherwise float32 array will be promoted whenever they are 
>>> multiplied by Python floating point scalars. If Python floats are 
>>> treated as Float64 for purposes of determining output precision then 
>>> anyone using float32 arrays is going to have to wrap all of their 
>>> literals in float32 to prevent inadvertent upcasting to float64. This 
>>> was the origin of the (rather clunky) numarray spacesaver flag.
>>>
>>> It's no skin off my nose either way, since I pretty much never use 
>>> float32, but I suspect that treating python floats equivalently to 
>>> float64 scalars would be a mistake. At the very least it deserves a bit 
>>> of discussion.
>>>    
>>>
>>>       
>> Well, they *are* 64-bit floating point numbers. You simply can't get around 
>> that. That's why we now have all of the scalar types: you can get any precision 
>> scalars that you want as long as you are explicit about it (and explicit is 
>> better than implicit). The spacesaver flag was the only solution before the 
>> various scalar types existed. I'd like to suggest that the discussion already 
>> occurred some time ago and has concluded in favor of the scalar types. 
>> Downcasting should be explicit.
>>
>> However, whether or not float32 arrays operated with Python float scalars give 
>> float32 or float64 arrays is tangential to my question. Does anyone actually 
>> think that a Python float operated with a boolean array should give a float32 
>> result? Must we *up*cast a boolean array to float64 to preserve the precision of 
>> the scalar?
>>
>>  
>>
>>     
> The first basic rule is that scalars don't control the precision of the 
> output when doing mixed-type calculations *except* when they are of a 
> fundamentally different kind. 
>
> Then (if a different kind of scalar is used), the rule is that the 
> arrays will be upcast to the "lowest" precision in the group to preserve 
> overall precision.   So, when a bool is combined with a "float" kind of 
> scalar, the result is float32 because that preserves precision of the 
> bool.  Remember it is array precision that takes precedence over scalars 
> in mixed type array-scalar operations.
>
> This is the rule.  I agree that this rule is probably flawed in certain 
> circumstances.
>   
I think any rule will be flawed in certain circumstances. This 
particular rule has the advantage of being relatively straightforward 
and the circumstances that I can think of where it could cause problems 
are relatively limited and relatively easy to address. The obvious 
"fixes" to this rule that I  have I've thought of all have problems that 
are as bad or worse as the current rule and have the added disadvantage 
of being more complicated.

At the very least, any replacement rule should get some serious 
discussion here before being implemented. We should particularly solicit 
the input of numarray users since that package had more infrastructure 
in place to support the use of lower precision arrays.

> So, what should be done about it at this point?  Do you think a change 
> is acceptable for 1.0.1 or does it need to wait a year until 1.1?
>   
Unless someone can come up with a convincingly better solution, I say 
leave things as is indefinitely.

-tim



-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From george.sakkis at gmail.com  Thu Nov  2 11:18:25 2006
From: george.sakkis at gmail.com (George Sakkis)
Date: Thu, 02 Nov 2006 16:18:25 -0000
Subject: Reading records from file and sorting
References: <1162337923.747767.323100@b28g2000cwb.googlegroups.com>
	<20061101224620.GA29602@dogbert.sdsl.sun.ac.za>
	<1162442534.508320.151770@e3g2000cwe.googlegroups.com>
	<e06186140611012052u16743231t9a508338f883b4@mail.gmail.com>
Message-ID: <1162484305.382176.128300@m7g2000cwm.googlegroups.com>

Charles R Harris wrote:

> On 11/1/06, George Sakkis <george.sakkis at gmail.com> wrote:
> >
> > Albert Strasheim wrote:
> >
> > > Check the thread "Strange results when sorting array with fields" from
> > > about a week back. Travis made some changes to sorting in the presence
> > > of fields that should solve your problem, assuming your fields appear in
> > > the order you want to sort (i.e. you want to sort f1, f2, f3 and not
> > > something like f1, f3, f2).
> >
> > I'm afraid this won't help in my case; I want to sort twice, once by f1
> > and once by f2. I guess I could make a second file with the fields
> > swapped but this seems more messy and inefficient than Francesc's
> > suggestion.
>
>
> Do you actually want the two different orders, or do you want to sort on the
> first field, then sort all the items with the same first field on the second
> field?
 
The former.

George


-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From oliphant at ee.byu.edu  Thu Nov  2 11:32:51 2006
From: oliphant at ee.byu.edu (Travis Oliphant)
Date: Thu, 02 Nov 2006 09:32:51 -0700
Subject: Reading records from file and sorting
In-Reply-To: <1162484305.382176.128300@m7g2000cwm.googlegroups.com>
References: <1162337923.747767.323100@b28g2000cwb.googlegroups.com>	<20061101224620.GA29602@dogbert.sdsl.sun.ac.za>	<1162442534.508320.151770@e3g2000cwe.googlegroups.com>	<e06186140611012052u16743231t9a508338f883b4@mail.gmail.com>
	<1162484305.382176.128300@m7g2000cwm.googlegroups.com>
Message-ID: <454A1DB3.4090002@ee.byu.edu>

George Sakkis wrote:

>Charles R Harris wrote:
>
>  
>
>>On 11/1/06, George Sakkis <george.sakkis at gmail.com> wrote:
>>    
>>
>>>Albert Strasheim wrote:
>>>
>>>      
>>>
>>>>Check the thread "Strange results when sorting array with fields" from
>>>>about a week back. Travis made some changes to sorting in the presence
>>>>of fields that should solve your problem, assuming your fields appear in
>>>>the order you want to sort (i.e. you want to sort f1, f2, f3 and not
>>>>something like f1, f3, f2).
>>>>        
>>>>
>>>I'm afraid this won't help in my case; I want to sort twice, once by f1
>>>and once by f2. I guess I could make a second file with the fields
>>>swapped but this seems more messy and inefficient than Francesc's
>>>suggestion.
>>>      
>>>
>>Do you actually want the two different orders, or do you want to sort on the
>>first field, then sort all the items with the same first field on the second
>>field?
>>    
>>
> 
>The former.
>  
>
Sorting on a particular field in-place would be possible if there were 
some-way to indicate to VOID_compare the field-order you wanted to use 
to compare on.

There are a few ways I could think of doing this.

1) Use a  thread-specific global variable (this doesn't recurse very 
easily).
2) Use the meta-object in the field specifier to indicate the order (the 
interface could still be something like
.sort(order='f1')  and a temporary data-type object is created and used).
3) Use a special key in the fields dictionary although this would 
require some fixes to all the code that cycles through the fields 
dictionary to recurse on structures.

4) Overload any of the other variables in the PyArray_Descr * structure.

5) Add a sort-order to the end of the PyArray_Descr * structure and a 
flag to the hasobject flag bits (that would be the last one available) 
that states that the Data-type object has the sort-order defined (so 
binary compatibilty is retained but the new feature can be used going 
forward).

Any other ideas?

-Travis




-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From drife at ucar.edu  Thu Nov  2 11:54:43 2006
From: drife at ucar.edu (Daran Rife)
Date: Thu, 2 Nov 2006 09:54:43 -0700
Subject: numpy book
Message-ID: <10D7CE3B-1A80-4B02-B9A3-A9BFCCC250E2@ucar.edu>

Hi Travis,

I have one question about the fee-based "Guide to NumPy"
that doesn't seem to be addressed on the Web site:

    <http://www.tramy.us/guidetoscipy.html>

or in the FAQ.

Once a user purchases the document, how does s/he obtain
the latest version, when it becomes available?

Looking forward to purchasing a copy, to show support for
NumPy--one of the best (free) scientific computing packages
available.


Thanks for fielding my question,


Daran

--


From oliphant at ee.byu.edu  Thu Nov  2 10:41:11 2006
From: oliphant at ee.byu.edu (Travis Oliphant)
Date: Thu, 02 Nov 2006 08:41:11 -0700
Subject: [Numpy-discussion] numpy book
Message-ID: <454A1197.1020606@ee.byu.edu>



> Note that this is not a request to Travis to send me the latest  
> version
> by private email. That would be inefficient and my need is not that
> urgent. Nevertheless I think that issue should be settled.
>
>
>
There will be an update, soon. I'm currently working on the index,
corrections, and formatting issues.

The update will be sent in conjunction with the release of 1.0.1 which I
am targetting in 2 weeks.

Thanks for everybody's patience.  If you need an update now, I'm happy
to send it to you by private email.

Best regards,

-Travis O.

-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From aisaac at american.edu  Thu Nov  2 12:13:34 2006
From: aisaac at american.edu (Alan G Isaac)
Date: Thu, 2 Nov 2006 12:13:34 -0500
Subject: numpy book
In-Reply-To: <10D7CE3B-1A80-4B02-B9A3-A9BFCCC250E2@ucar.edu>
References: <10D7CE3B-1A80-4B02-B9A3-A9BFCCC250E2@ucar.edu>
Message-ID: <Mahogany-0.67.0-1408-20061102-121334.00@american.edu>

On Thu, 2 Nov 2006, Daran Rife apparently wrote: 
> Once a user purchases the document, how does s/he obtain 
> the latest version, when it becomes available? 

In addition to providing updates on an "as requested" basis,
Travis has made it clear that he will occasionally do a mass
mailing.  (At least it seems clear to me...)

I have a different suggestion for Travis:
I suggest each purchaser receive an ID which can be used for 
a download twice a year.  That will put these enquiries to 
rest.


> Looking forward to purchasing a copy, to show support for 
> NumPy--one of the best (free) scientific computing 
> packages available. 

It is also a very nice book.  If you had to repurchase it 
once every year or two, which you do not, it would still be 
a good deal.  So don't hesitate!  And make sure your library
and labs buy copies too!

Cheers,
Alan Isaac



-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From fperez.net at gmail.com  Thu Nov  2 14:07:42 2006
From: fperez.net at gmail.com (Fernando Perez)
Date: Thu, 2 Nov 2006 12:07:42 -0700
Subject: Strange and hard to reproduce crash
In-Reply-To: <453D4AD1.5060508@ieee.org>
References: <db6b5ecc0610231439o1217af98j11dcb45e2ae5e505@mail.gmail.com>
	<453D3DD7.50506@ieee.org>
	<db6b5ecc0610231544g7413491du9e8b0bd6c8c20e6f@mail.gmail.com>
	<453D4AD1.5060508@ieee.org>
Message-ID: <db6b5ecc0611021107h3fed690dycbd0c608728b3b95@mail.gmail.com>

On 10/23/06, Travis Oliphant <oliphant.travis at ieee.org> wrote:

> I've placed them in SVN (r3384):
>
> arraydescr_dealloc needs to do something like.
>
> if (self->fields == Py_None) {
>     print something
>     incref(self)
>     return;
> }
>
> Most likely there is a missing Py_INCREF() before some call that uses
> the data-type object (and consumes it's reference count) --- do you have
> any Pyrex code (it's harder to get it right with Pyrex).

OK, we've completed another long run (several days), and this time it
didn't crash.  But I think there are still refcount problems.  I'm
attaching the full log file and a plot of the refcount.  It's wrapping
around, and after some point the increase switches to a pefectly
linear pattern, I'm not exactly sure why (it could be a change in the
underlying code after the initialization phase; it's not my code so I
don't know its internals).

I hope this helps, it would be nice to track this down before 1.0.1 is out.

Cheers,

f
-------------- next part --------------
A non-text attachment was scrubbed...
Name: refcounts.png
Type: image/png
Size: 27895 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20061102/247f3033/attachment.png>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: numpy_report.gz
Type: application/x-gzip
Size: 25416 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20061102/247f3033/attachment.bin>
-------------- next part --------------
-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
-------------- next part --------------
_______________________________________________
Numpy-discussion mailing list
Numpy-discussion at lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/numpy-discussion

From torgil.svensson at gmail.com  Thu Nov  2 14:32:44 2006
From: torgil.svensson at gmail.com (Torgil Svensson)
Date: Thu, 2 Nov 2006 20:32:44 +0100
Subject: numpy book
In-Reply-To: <db6b5ecc0611020800u367eb6fdvccf194c6e1aecd14@mail.gmail.com>
References: <4549E34A.4060507@gmx.net> <454A1197.1020606@ee.byu.edu>
	<db6b5ecc0611020800u367eb6fdvccf194c6e1aecd14@mail.gmail.com>
Message-ID: <e7bda7770611021132y60c9a6abta182ed16e202cb85@mail.gmail.com>

> generating such a PDF, with some color and formatting tweaks so that
> it prints legibly on black and white as well as looking nice on
> screen.

This sounds promising. I actually have had problems printing the
guide, every information box showed up completely black except for the
emblem.

On 11/2/06, Fernando Perez <fperez.net at gmail.com> wrote:
> On 11/2/06, Travis Oliphant <oliphant at ee.byu.edu> wrote:
> >
> > >Note that this is not a request to Travis to send me the latest version
> > >by private email. That would be inefficient and my need is not that
> > >urgent. Nevertheless I think that issue should be settled.
> > >
> > >
> > There will be an update, soon. I'm currently working on the index,
> > corrections, and formatting issues.
>
> If I may make a suggestion, it would greatly increase the usability of
> the PDF if it had internal and external clickable links, as well as a
> PDF table of contents (the one that appears on the left sidebar of
> acroread).
>
> Since I know the original is in Lyx, here's my standard preamble for
> these kinds of documents, which includes all the goodies for
> generating such a PDF, with some color and formatting tweaks so that
> it prints legibly on black and white as well as looking nice on
> screen.  If you add this to your preamble, a simple 'view->pdflatex'
> should work.  I'll be happy to help if you have any issues.
>
> Cheers,
>
> f
>
> %%% My preamble; tweak and season to taste:
>
> % This gives us a better font in URL links (otherwise the default
> % MonoSpace font is bitmapped, and it looks horrible in PDF)
> \usepackage{courier}
>
> \usepackage{fullpage}
>
> \usepackage{color}  % so we can use red for the fixme warnings
>
> % The hyperref package gives us a pdf with properly built
> % internal navigation ('pdf bookmarks' for the table of contents,
> % internal cross-reference links, web links for URLs, etc.)
>
> % A few colors to replace the defaults for certain link types
> \definecolor{darkorange}{rgb}{.71,0.21,0.01}
> \definecolor{darkgreen}{rgb}{.12,.54,.11}
>
> \usepackage[
>   %pdftex,  % needed for pdflatex
>   breaklinks=true,  % so long urls are correctly broken across lines
>   colorlinks=true,
>   urlcolor=blue,
>   linkcolor=darkorange,
>   citecolor=darkgreen,
>   ]{hyperref}
>
> % This helps prevent overly long lines that stretch beyond the margins
> \sloppy
>
> % Define a \fixme command to mark visually things needing fixing in the draft.
> % For final printing or to simply disable these bright warnings, simply
> % uncomment the \renewcommand redefinition below
>
> \newcommand{\fixme}[1] {
> \textcolor{red}{
> {\fbox{ {\bf FIX}
> \ensuremath{\blacktriangleright \blacktriangleright \blacktriangleright}}
> {\bf #1}
> \fbox{\ensuremath{ \blacktriangleleft \blacktriangleleft \blacktriangleleft }
> } } }
> }
> % Uncomment the next line to make the \fixme command be a no-op
> %\renewcommand{\fixme}[1]{}
>
> %%% If you also want to use the listings package for nicely formatted
> %%% Python source code, this configuration produces good on-paper and
> %%% on-screen results:
>
> \definecolor{orange}{cmyk}{0,0.4,0.8,0.2}
> % Use and configure listings package for nicely formatted code
> \usepackage{listings}
> \lstset{
>   language=Python,
>   basicstyle=\small\ttfamily,
>   commentstyle=\ttfamily\color{blue},
>   stringstyle=\ttfamily\color{orange},
>   showstringspaces=false,
>   breaklines=true,
>   postbreak = \space\dots
> }
>
> -------------------------------------------------------------------------
> Using Tomcat but need to do more? Need to support web services, security?
> Get stuff done quickly with pre-integrated technology to make your job easier
> Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
> http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
> _______________________________________________
> Numpy-discussion mailing list
> Numpy-discussion at lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/numpy-discussion
>

-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From faltet at carabos.com  Thu Nov  2 14:26:25 2006
From: faltet at carabos.com (Francesc Altet)
Date: Thu, 02 Nov 2006 20:26:25 +0100
Subject: numarray indexing problems with Python2.5 and
	64-bit platforms
Message-ID: <1162495586.3960.19.camel@localhost.localdomain>

Hi,

I've detected that numarray (1.5.2) seems to be bitten by the change in
Python 2.5 for indexes (http://docs.python.org/whatsnew/pep-353.html). 
In a Linux64 machine (using Python 2.5), I get this:

>>> a=numarray.array([1,2,3])
>>> a[1:2]
array([1, 2])  # !

However, both Numeric and numpy seems to work well with the same
scenario.

>>> b=Numeric.array([1,2,3])
>>> b[1:2]
array([2])
>>> c=numpy.array([1,2])
>>> c[1:2]
array([2])

I see this as a major issue in numarray and poses in great danger the
intended support of PyTables for numarray that we planned for some time
(until end of 2007). It would be nice to know if the numarray crew would
be willing to address this, or, now that NumPy 1.0 is out, they have
decided to completely drop the support for it.

We'd really like to continue offering support for numarray (in the end,
it is a very good piece of software) in PyTables, but don't having a
solution for this problem anytime soon, will make this very problematic
to us.

Thanks,

-- 
Francesc Altet    |  Be careful about using the following code --
Carabos Coop. V.  |  I've only proven that it works, 
www.carabos.com   |  I haven't tested it. -- Donald Knuth


-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From NPadmanabhan at lbl.gov  Thu Nov  2 15:35:33 2006
From: NPadmanabhan at lbl.gov (Nikhil Padmanabhan)
Date: Thu, 2 Nov 2006 12:35:33 -0800
Subject: Segfault in Python 2.5
Message-ID: <FAFFEFE2-07AF-4D62-9B23-A1BDD07659C7@lbl.gov>

Hi,

I recently tried to upgrade to Python 2.5 (MacPython), and I am  
getting segfaults on some of my f2py wrapped fortran codes. I'm  
running OS X 10.4.8 (ppc), and this occurs for both numpy 1.0 and  
1.0rc2 (the two versions I tried). Running with Python 2.4 does not  
give me this error. This is using the IBM xlf compiler....

A simple test case is below.....

Not sure where to go from here -- any help would be great!
Thanks,

Here is a stripped down version that still crashes :

subroutine foo(xx, nxx, bj)
   implicit none
   integer nxx
   real*8 xx(nxx),bj(nxx)
!f2py intent(out) bj

   bj = xx

end subroutine foo

Compiling this to foo.so, I get :
Python 2.5 (r25:51918, Sep 19 2006, 08:49:13)
[GCC 4.0.1 (Apple Computer, Inc. build 5341)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
 >>> from numpy import *
 >>> import foo as p
 >>> a = arange(10)
 >>> p.foo(a)
Segmentation fault

Intriguingly, swapping the import orders fixes this :
Python 2.5 (r25:51918, Sep 19 2006, 08:49:13)
[GCC 4.0.1 (Apple Computer, Inc. build 5341)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
 >>> import foo as p
 >>> from numpy import *
 >>> a = arange(10)
 >>> p.foo(a)
array([ 0.,  1.,  2.,  3.,  4.,  5.,  6.,  7.,  8.,  9.])
 >>>



------------------------------------
Nikhil Padmanabhan
NPadmanabhan at lbl.gov
nikhil@(510) 495-2943




-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From jmiller at stsci.edu  Thu Nov  2 16:04:43 2006
From: jmiller at stsci.edu (Todd Miller)
Date: Thu, 02 Nov 2006 16:04:43 -0500
Subject: numarray indexing problems with Python2.5
 and	64-bit platforms
In-Reply-To: <1162495586.3960.19.camel@localhost.localdomain>
References: <1162495586.3960.19.camel@localhost.localdomain>
Message-ID: <454A5D6B.4020505@stsci.edu>

Here's a stab at a solution but I don't have easy access to 64-bit linux 
at the moment so it is untested.   If someone is willing to test it 
(and/or fix it) I'll add it to the next numarray release.  It should be 
noted that numarray is not 64-bit enabled (it compiles as a 32-bit 
program in terms of arrays) which is another motivation to switch to numpy.

Regards,
Todd

Francesc Altet wrote:
> Hi,
>
> I've detected that numarray (1.5.2) seems to be bitten by the change in
> Python 2.5 for indexes (http://docs.python.org/whatsnew/pep-353.html). 
> In a Linux64 machine (using Python 2.5), I get this:
>
>   
>>>> a=numarray.array([1,2,3])
>>>> a[1:2]
>>>>         
> array([1, 2])  # !
>
> However, both Numeric and numpy seems to work well with the same
> scenario.
>
>   
>>>> b=Numeric.array([1,2,3])
>>>> b[1:2]
>>>>         
> array([2])
>   
>>>> c=numpy.array([1,2])
>>>> c[1:2]
>>>>         
> array([2])
>
> I see this as a major issue in numarray and poses in great danger the
> intended support of PyTables for numarray that we planned for some time
> (until end of 2007). It would be nice to know if the numarray crew would
> be willing to address this, or, now that NumPy 1.0 is out, they have
> decided to completely drop the support for it.
>
> We'd really like to continue offering support for numarray (in the end,
> it is a very good piece of software) in PyTables, but don't having a
> solution for this problem anytime soon, will make this very problematic
> to us.
>
> Thanks,
>
>   

-------------- next part --------------
An embedded and charset-unspecified text was scrubbed...
Name: slice64.patch
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20061102/80bd33c4/attachment.ksh>
-------------- next part --------------
-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
-------------- next part --------------
_______________________________________________
Numpy-discussion mailing list
Numpy-discussion at lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/numpy-discussion

From peridot.faceted at gmail.com  Thu Nov  2 16:26:07 2006
From: peridot.faceted at gmail.com (A. M. Archibald)
Date: Thu, 2 Nov 2006 16:26:07 -0500
Subject: numarray indexing problems with Python2.5
	and 64-bit platforms
In-Reply-To: <1162495586.3960.19.camel@localhost.localdomain>
References: <1162495586.3960.19.camel@localhost.localdomain>
Message-ID: <ce557a360611021326o4205fbdg5709859f57d02d2e@mail.gmail.com>

On 02/11/06, Francesc Altet <faltet at carabos.com> wrote:

> I see this as a major issue in numarray and poses in great danger the
> intended support of PyTables for numarray that we planned for some time
> (until end of 2007). It would be nice to know if the numarray crew would
> be willing to address this, or, now that NumPy 1.0 is out, they have
> decided to completely drop the support for it.
>
> We'd really like to continue offering support for numarray (in the end,
> it is a very good piece of software) in PyTables, but don't having a
> solution for this problem anytime soon, will make this very problematic
> to us.

Someone has to say it: you could just drop support for the obsolete
numarray and provide only support for its successor, numpy.

A. M. Archibald

-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From sz13544007750 at 163.com  Thu Nov  2 19:48:53 2006
From: sz13544007750 at 163.com (=?GB2312?B?xvPStdPQwPs=?=)
Date: Fri, 3 Nov 2006 08:48:53 +0800
Subject: =?GB2312?B?xvPStdPQwPs=?=
Message-ID: <E1GfnEx-0003wS-Q0@mail.sourceforge.net>

???/????
??         ??! 
?????????????????????????????????????
?????????????????????????????????????0.8-2%??????6%-8%???????2-4%????????????!
??????????????????????????????????????????????????????????????????????????????????????????????????????????
?????????????????????????????????????????????????????????????
?????????????????????????????????
?????? ???  13544007750  
E-mail? jason-7750 at 163.com
??????????????????

-------------- next part --------------
-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
-------------- next part --------------
_______________________________________________
Numpy-discussion mailing list
Numpy-discussion at lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/numpy-discussion

From wsa532 at 163.com  Thu Nov  2 22:41:58 2006
From: wsa532 at 163.com (=?GB2312?B?IjEx1MIxMS0xMsjVL8nPuqMi?=)
Date: Fri, 3 Nov 2006 03:41:58 +0000 (UTC)
Subject: =?GB2312?B?t8eyxs7xvq3A7bXEssbO8bncwO0tybPFzMSjxOI=?=
Message-ID: <E1GfpwC-0008OV-OQ@mail.sourceforge.net>

An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20061103/5efabdcc/attachment.html>
-------------- next part --------------
-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
-------------- next part --------------
_______________________________________________
Numpy-discussion mailing list
Numpy-discussion at lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/numpy-discussion

From mhkcptinw at dgcsystems.net  Thu Nov  2 23:39:38 2006
From: mhkcptinw at dgcsystems.net (cannot connection)
Date: Fri, 3 Nov 2006 05:39:38 +0100
Subject: hearts.MY tammy rowdy
Message-ID: <001301c6ff02$12529fa0$98dfb53e@laxp1>


Interstate Highways numbers or.
German rockers who big pals with? Openwfe convert in pdf a svg. Top Larrys work enjoying board is.
Openwfe convert in pdf a svg. Laudio Jxxyy Kxxyy Lxxyy!
Style pagesmost comply mos rarely is directly pages. Ends needs am generated specify location am.
Video Converter Diablo in? Mckerchar doing while customers or.
Enoughthe a awesome milestone worthy White in checks of Lift of. Explore paranormal phenomena evp?
Winners correct answer is Edgar Allan poe Sign of hsw of Newsletter. Theres chance is itself perform.
Continuous is Xmldecoder Sergey Malenkov in start. Comparison readily erased Firmware.
Interstate Highways numbers or. Moog am intro songs. Wing episode in Odyssey Ncaa is Tournament?
Todayi loved forand sense peacethat or.
Xmltask developed mechanism am. Them Basics Nuclear Bombs is.
There in discover in iterate keys in isnt returning values am yet or dealloc of.
Fantasy Lineage Middle in Earth Nights Galaxies.  Difficult reduce is what.
Graph memory thisobject translated of calls! Agreement bound Oreilly Collabnet. Profile rackmount of chassis accepts in?
Edgar a Allan poe Sign. Agreement bound Oreilly Collabnet. Page pleased Mobility Pack fcs a Profiler Fcscc Beta?
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20061103/c49079a5/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: uses:.gif
Type: image/gif
Size: 9519 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20061103/c49079a5/attachment.gif>
-------------- next part --------------
-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
-------------- next part --------------
_______________________________________________
Numpy-discussion mailing list
Numpy-discussion at lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/numpy-discussion

From torgil.svensson at gmail.com  Fri Nov  3 01:38:29 2006
From: torgil.svensson at gmail.com (Torgil Svensson)
Date: Fri, 3 Nov 2006 07:38:29 +0100
Subject: Need more comments from scientific community
	on python-dev
In-Reply-To: <e86a5fd00611011019p5a39553cwf55fd06e585e58e0@mail.gmail.com>
References: <45474841.9030802@ieee.org>
	<d38f5330610311210i5141b495s5bb73f8da6484629@mail.gmail.com>
	<4547B235.6050904@ee.byu.edu>
	<d38f5330610311336w7538e0b2v994ee277da64f6cf@mail.gmail.com>
	<4547D2B3.5000808@ee.byu.edu>
	<d38f5330610311551u4e71cdd7ndc0b93de16dc6d80@mail.gmail.com>
	<4548C3D6.40100@ieee.org>
	<ce557a360611011013v4e883251l46dc6e2c0c85b854@mail.gmail.com>
	<e86a5fd00611011019p5a39553cwf55fd06e585e58e0@mail.gmail.com>
Message-ID: <e7bda7770611022238y2cb15e75s3739d2b4b652129c@mail.gmail.com>

> > It seems to me that at the python level, there's not much reason to
> > choose the dtypes proposal over ctypes.

I disagree. I can see a point in unifying ctypes and dtypes but in my
mindset they've got two different scopes.

ctypes is an interface to the c language
dtype is a data description for the python language

The biggest advantages python has over c in my mind (except the script
vs compile) is that it's dynamic and expressive. I think basing dtype
on ctypes is in that mind is simply destructive and against what I
like best with python.

ctypes mimics c's static behaviour:

class struct_type(Structure):
    _fields_ = [....]

...

... which is somewhat static in nature. How do you create "structures"
dynamically?

How do you do things like this in ctypes:

mytype=filter(lambda x: is_little_endian(x),map(lambda x: x==int32 and
int64 or x, old_type))
b=['gg',[1,2,3]]
mytype=dtype([{str: uint64 , list:another_type }[type(x)] for x in b])

I'm sure it's possibl but I'm also suspecting it not at all as clean
and expressive (my second argument for choosing python over c).

Since we creating something new in python isn't it best we do
something the python way and later see if we can adapt the c structure
to that rather than the other way around?  If we can't adapt ctypes to
this new thing i'm perfectly happy with leaving ctypes as is.

ctype is doing a great job on the c interaction level but is plain
ugly on the python level.


On 11/1/06, Bill Baxter <wbaxter at gmail.com> wrote:
>
> On 11/2/06, A. M. Archibald <peridot.faceted at gmail.com> wrote:
> > On 01/11/06, Travis Oliphant <oliphant.travis at ieee.org> wrote:
> >
> > > And it may be a good idea to also have a get_ctype method or some-such
> > > on the ctypes attribute so that  one could get a "ctypes description"
> > > from the NumPy data-type.
> >
> > It seems to me that at the python level, there's not much reason to
> > choose the dtypes proposal over ctypes. There is one at the C level,
> > it seems (though I, like perhaps most of the people on python-dev,
> > have never actually tried using either). So perhaps, to persuade the
> > python-dev folks, what is needed is a comparison of what has to be
> > done at the C level. What would it take to rewrite numpy to use
> > ctypes? There seems to be some problem with extending the type objects
> > used by ctypes, but it's not very clear to me what that problem is
> > (what the extensions are supposed to do).
>
>
> I posted a message to the thread trying to prod things in that direction.
> I.e. can we see a simple concrete example of the complications involved in
> using ctypes interface code, vs the presumably much nicer
> numpy/data-descriptor code.  I too think that would help convince people.
> Just saying it's more compilcated, trust me, doesn't help when most people
> reading the list have never had to write any sort of C extension.
>
> --bb
>
>
> -------------------------------------------------------------------------
> Using Tomcat but need to do more? Need to support web services, security?
> Get stuff done quickly with pre-integrated technology to make your job
> easier
> Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
> http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
>
> _______________________________________________
> Numpy-discussion mailing list
> Numpy-discussion at lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/numpy-discussion
>
>
>

-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From Mailer-Daemon at lists.sourceforge.net  Fri Nov  3 07:11:56 2006
From: Mailer-Daemon at lists.sourceforge.net (Mail Delivery System)
Date: Fri, 03 Nov 2006 04:11:56 -0800
Subject: Warning: message 1GfAoO-0007im-Sh delayed 48
	hours
Message-ID: <E1Gfxts-0003oY-44@externalmx-1.sourceforge.net>

This message was created automatically by mail delivery software.
A message that you sent has not yet been delivered to one or more of its
recipients after more than 48 hours on the queue on externalmx-1.sourceforge.net.

The message identifier is:     1GfAoO-0007im-Sh
The subject of the message is: looked like me the full sized HARLEY and began to her, with this but as
The date of the message is:    Wed, 01 Nov 2006 15:46:73 +0800

The address to which the message has not yet been delivered is:

  numpy-discussion at lists.sourceforge.net
    Delay reason: SMTP error from remote mailer after RCPT TO:<numpy-discussion at lists.sourceforge.net>:
    host mail.sourceforge.net [66.35.250.206]: 451-Could not complete sender verify callout
    451-Could not complete sender verify callout for
    451-<numpy-discussion at lists.sourceforge.net>.
    451-The mail server(s) for the domain may be temporarily unreachable, or
    451-they may be permanently unreachable from this server. In the latter case,
    451-you need to change the address or create an MX record for its domain
    451-if it is 

No action is required on your part. Delivery attempts will continue for
some time, and this warning may be repeated at intervals if the message
remains undelivered. Eventually the mail delivery software will give up,
and when that happens, the message will be returned to you.

-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From faltet at carabos.com  Fri Nov  3 10:00:10 2006
From: faltet at carabos.com (Francesc Altet)
Date: Fri, 3 Nov 2006 16:00:10 +0100
Subject: =?utf-8?q?numarray_indexing_problems_with_Pyth?=
	=?utf-8?q?on2=2E5_and=0964-bit_platforms?=
In-Reply-To: <454A5D6B.4020505@stsci.edu>
References: <1162495586.3960.19.camel@localhost.localdomain>
	<454A5D6B.4020505@stsci.edu>
Message-ID: <200611031600.10980.faltet@carabos.com>

Hi Todd,

A Dijous 02 Novembre 2006 22:04, Todd Miller escrigu?:
> Here's a stab at a solution but I don't have easy access to 64-bit linux
> at the moment so it is untested.   If someone is willing to test it
> (and/or fix it) I'll add it to the next numarray release.  It should be
> noted that numarray is not 64-bit enabled (it compiles as a 32-bit
> program in terms of arrays) which is another motivation to switch to numpy.

Many thanks for the patch. The good news is that it seems to fix the
main issue:

>>> a=numarray.array([1,2,3])
>>> a[1:2]
array([2])

and moreover, all the tests in PyTables (with numarray at its core) do
pass in 64-bit platforms and Python 2.5 run ok (except for the known
bug that also happens in 32-bit platforms; see [1]).

The bad news is that the numarray tests seems more finicky than
PyTables ones and they are throwing a "segmentation fault":

>>> import numarray.testall as testall
>>> testall.test()
Testing numarray 1.5.2 on normal Python (2, 5, 0, 'final', 0) on platform 
linux2 with builtin linear algebra support
Violaci?n de segmento

I've run this through valgrind, and here it is the offending line:

unhandled opc_aux = 0x 7
first_opcode == 0xDD
vex amd64->IR: unhandled instruction bytes: 0xDD 0x7C 0x24 0xFC
==29710==
==29710== Process terminating with default action of signal 4 (SIGILL)
==29710==  Illegal opcode at address 0x11F603A0
==29710==    at 0x11F603A0: fetestexcept (in /lib64/tls/libm-2.3.5.so)
==29710==    by 0x1272E21D: NA_clearFPErrors (libnumarraymodule.c:1949)
==29710==    by 0x13465311: _Py_cum_exec (_ufuncmodule.c:998)
==29710==    by 0x1346562C: _cum_cached (_ufuncmodule.c:1087)
==29710==    by 0x134659E7: _cum_swapped (_ufuncmodule.c:1199)
==29710==    by 0x13465ED0: _Py_reduce (_ufuncmodule.c:1374)
==29710==    by 0x483166: PyEval_EvalFrameEx (ceval.c:3566)
==29710==    by 0x4845B0: PyEval_EvalCodeEx (ceval.c:2833)
==29710==    by 0x48331D: PyEval_EvalFrameEx (ceval.c:3662)
==29710==    by 0x4845B0: PyEval_EvalCodeEx (ceval.c:2833)
==29710==    by 0x48331D: PyEval_EvalFrameEx (ceval.c:3662)
==29710==    by 0x4845B0: PyEval_EvalCodeEx (ceval.c:2833)
==29710==    by 0x48331D: PyEval_EvalFrameEx (ceval.c:3662)
==29710==    by 0x4845B0: PyEval_EvalCodeEx (ceval.c:2833)
==29710==    by 0x4CF0D3: function_call (funcobject.c:517)
==29710==    by 0x415C32: PyObject_Call (abstract.c:1860)
==29710==    by 0x41D0A2: instancemethod_call (classobject.c:2493)
==29710==    by 0x415C32: PyObject_Call (abstract.c:1860)
==29710==    by 0x47C660: PyEval_CallObjectWithKeywords (ceval.c:3435)
==29710==    by 0x454823: slot_tp_repr (typeobject.c:4504)

it seems to my inexpert eyes that the compiler has generated some code
that it is not understood by the amd64 architecture (?).

[1] 
<http://sourceforge.net/tracker/index.php?func=detail&aid=1565683&group_id=1369&atid=450446>

Regards,

-- 
>0,0<   Francesc Altet ? ? http://www.carabos.com/
V   V   C?rabos Coop. V. ??Enjoy Data
 "-"



-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
_______________________________________________
Numpy-discussion mailing list
Numpy-discussion at lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/numpy-discussion

From numpy-discussion at mlists.thewrittenword.com  Fri Nov  3 10:09:43 2006
From: numpy-discussion at mlists.thewrittenword.com (Albert Chin)
Date: Fri, 3 Nov 2006 09:09:43 -0600
Subject: Setting runtime_library_dirs while building numpy
Message-ID: <20061103150943.GA35289@mail1.thewrittenword.com>

We have atlas-3.6.0 installed in a non-standard location. When
building the numpy-1.0 shared libraries, we'd like to encode the
non-standard location in the RPATH of the executable. We do this with
other Python modules by adding the following to setup.cfg:
  [build_ext]
  rpath=<path to library>

But, this doesn't work for numpy-1.0. Is there another way to do this?

-- 
albert chin (china at thewrittenword.com)

-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From karol.langner at kn.pl  Fri Nov  3 11:19:09 2006
From: karol.langner at kn.pl (Karol Langner)
Date: Fri, 3 Nov 2006 17:19:09 +0100
Subject: Setting runtime_library_dirs while building
	numpy
In-Reply-To: <20061103150943.GA35289@mail1.thewrittenword.com>
References: <20061103150943.GA35289@mail1.thewrittenword.com>
Message-ID: <200611031719.09388.karol.langner@kn.pl>

On Friday 03 of November 2006 16:09, Albert Chin wrote:
> We have atlas-3.6.0 installed in a non-standard location. When
> building the numpy-1.0 shared libraries, we'd like to encode the
> non-standard location in the RPATH of the executable. We do this with
> other Python modules by adding the following to setup.cfg:
>   [build_ext]
>   rpath=<path to library>
>
> But, this doesn't work for numpy-1.0. Is there another way to do this?

try something like this in your site.cfg:

[atlas]
library_dirs = <paths to libraries>
include_dirs = <paths to include dirs>
atlas_libs = <names of libraries>

Karol

-- 
written by Karol Langner
Fri Nov  3 17:18:38 CET 2006

-- 
written by Karol Langner
Fri Nov  3 17:19:08 CET 2006

-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From faltet at carabos.com  Fri Nov  3 10:21:34 2006
From: faltet at carabos.com (Francesc Altet)
Date: Fri, 3 Nov 2006 16:21:34 +0100
Subject: Numeric 24.2 indexing problems with Python 2.5
	and 64-bit platforms
Message-ID: <200611031621.35721.faltet@carabos.com>

Hi,

This is just to inform you that I've detected also problems with
indexation code in Numeric (24.2) with Python 2.5 and 64-bit
platforms. The next reproduces the issue:

>>> a=Numeric.array([1,0])
>>> a
array([1, 0])
>>> a[:]
zeros((0,), 'l')  # ??
# The order of values in the array doesn't seem to affect
>>> a=Numeric.array([0,1])
>>> a
array([0, 1])
>>> a[:]
zeros((0,), 'l')  # ?


I know that support for Numeric has officially ceased. This is just to
let others know that using latest Numeric with Python 2.5 and 64-bit
platforms (or, at very least, Linux64 on top of AMD64 :P) can lend to
pretty scaring results.

Cheers,

-- 
>0,0<   Francesc Altet ? ? http://www.carabos.com/
V   V   C?rabos Coop. V. ??Enjoy Data
 "-"


-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From faltet at carabos.com  Fri Nov  3 10:51:56 2006
From: faltet at carabos.com (Francesc Altet)
Date: Fri, 3 Nov 2006 16:51:56 +0100
Subject: numarray indexing problems with Python2.5
	and 64-bit platforms
In-Reply-To: <ce557a360611021326o4205fbdg5709859f57d02d2e@mail.gmail.com>
References: <1162495586.3960.19.camel@localhost.localdomain>
	<ce557a360611021326o4205fbdg5709859f57d02d2e@mail.gmail.com>
Message-ID: <200611031652.01465.faltet@carabos.com>

A Dijous 02 Novembre 2006 22:26, A. M. Archibald escrigu?:
> On 02/11/06, Francesc Altet <faltet at carabos.com> wrote:
> > I see this as a major issue in numarray and poses in great danger the
> > intended support of PyTables for numarray that we planned for some time
> > (until end of 2007). It would be nice to know if the numarray crew would
> > be willing to address this, or, now that NumPy 1.0 is out, they have
> > decided to completely drop the support for it.
> >
> > We'd really like to continue offering support for numarray (in the end,
> > it is a very good piece of software) in PyTables, but don't having a
> > solution for this problem anytime soon, will make this very problematic
> > to us.
>
> Someone has to say it: you could just drop support for the obsolete
> numarray and provide only support for its successor, numpy.

Yeah, that would be the easy path, agreed.  However, only God knows
how many code out there exists that still uses numarray (or Numeric).
We were optimistic that, given the implementation of the stunningly
efficient and easy-to-use array protocol in the three packages (NumPy,
numarray and Numeric), that very much boosted the interoperability
between them, support for all three would be a relatively easy thing
to keep in next release PyTables (with NumPy at its core).

Unfortunately, time has demonstrated how easy for a package to become
obsolete (and hence, useless) is. Just a new version of Python (2.5
this time) and the advent of 64-bit platforms has rendered obsolete
both numarray and Numeric in one shot. Mmm, perhaps it would make more
sense to support just numarray (and Numeric up to an extend) in just
32-bit platforms, but again, even that could be risky. Well, "que
sera, sera..." only time will say (as the song says).

Cheers,

-- 
>0,0<   Francesc Altet ? ? http://www.carabos.com/
V   V   C?rabos Coop. V. ??Enjoy Data
 "-"


-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From numpy-discussion at mlists.thewrittenword.com  Fri Nov  3 10:54:17 2006
From: numpy-discussion at mlists.thewrittenword.com (Albert Chin)
Date: Fri, 3 Nov 2006 09:54:17 -0600
Subject: Setting runtime_library_dirs while building
	numpy
In-Reply-To: <200611031719.09388.karol.langner@kn.pl>
References: <20061103150943.GA35289@mail1.thewrittenword.com>
	<200611031719.09388.karol.langner@kn.pl>
Message-ID: <20061103155417.GB35289@mail1.thewrittenword.com>

On Fri, Nov 03, 2006 at 05:19:09PM +0100, Karol Langner wrote:
> On Friday 03 of November 2006 16:09, Albert Chin wrote:
> > We have atlas-3.6.0 installed in a non-standard location. When
> > building the numpy-1.0 shared libraries, we'd like to encode the
> > non-standard location in the RPATH of the executable. We do this with
> > other Python modules by adding the following to setup.cfg:
> >   [build_ext]
> >   rpath=<path to library>
> >
> > But, this doesn't work for numpy-1.0. Is there another way to do this?
> 
> try something like this in your site.cfg:
> 
> [atlas]
> library_dirs = <paths to libraries>
> include_dirs = <paths to include dirs>
> atlas_libs = <names of libraries>

The library_dirs variable will simply set the path for the linker to
search for the library (i.e. -L<paths to libraries>). However, it
won't add the -Wl,-rpath,<paths to libraries> on Linux that we want
(or -R<paths to libraries> on Solaris, etc.).

-- 
albert chin (china at thewrittenword.com)

-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From alexander.belopolsky at gmail.com  Fri Nov  3 10:56:53 2006
From: alexander.belopolsky at gmail.com (Alexander Belopolsky)
Date: Fri, 3 Nov 2006 10:56:53 -0500
Subject: Need more comments from scientific community
	on python-dev
In-Reply-To: <e7bda7770611022238y2cb15e75s3739d2b4b652129c@mail.gmail.com>
References: <45474841.9030802@ieee.org>
	<d38f5330610311210i5141b495s5bb73f8da6484629@mail.gmail.com>
	<4547B235.6050904@ee.byu.edu>
	<d38f5330610311336w7538e0b2v994ee277da64f6cf@mail.gmail.com>
	<4547D2B3.5000808@ee.byu.edu>
	<d38f5330610311551u4e71cdd7ndc0b93de16dc6d80@mail.gmail.com>
	<4548C3D6.40100@ieee.org>
	<ce557a360611011013v4e883251l46dc6e2c0c85b854@mail.gmail.com>
	<e86a5fd00611011019p5a39553cwf55fd06e585e58e0@mail.gmail.com>
	<e7bda7770611022238y2cb15e75s3739d2b4b652129c@mail.gmail.com>
Message-ID: <d38f5330611030756w33b5f026t92282f6ed571e66a@mail.gmail.com>

On 11/3/06, Torgil Svensson <torgil.svensson at gmail.com> wrote:
> class struct_type(Structure):
>     _fields_ = [....]
>
> ...
> ... which is somewhat static in nature. How do you create "structures"
> dynamically?
>

You can put the above in a function that takes fields as an argument, or

    type('struct_type', (Structure,), {'_fields_':[('a', c_int), ('b',
c_long)]})

-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From shane.holloway at ieee.org  Fri Nov  3 11:27:30 2006
From: shane.holloway at ieee.org (Shane Holloway)
Date: Fri, 3 Nov 2006 09:27:30 -0700
Subject: Getting an array of indexes from a unicode string
Message-ID: <8CFCD207-9528-4689-83BE-2BCEB27B6104@ieee.org>

I'm looking for a better way to map the characters of a unicode  
string to indexes into an array of geometry.  The following code is  
functional, but it seems sub-optimal with all that numpy has to offer::

     textOrds = map(ord, text.encode('utf-8'))
     idx = indexMap[textOrds]
     textGeo = geometry[idx]

text is a simple python string coming in.  I then manually covert it  
to unicode ordinals.  Those are then mapped through indexMap, which  
happens to be a 1-to-1 mapping between unicode ordinals and valid  
indexes into geometry.  I then use the idx array to take a selection  
from geometry for the text.

As I mentioned before, this works alright, however two things seem  
inefficient.  First is the manual mapping to unicode ordinals.  Is  
there a way to have numpy do that for me?  Secondly is the mapping  
through indexMap, because it is only sparsely populated -- usually  
only a 2-5 thousand entries out of the 64 thousand allocated.  I've  
thought of using unicode.translate, but characters cannot be used for  
indexes in numpy.

What are your collective thoughts on making this cleaner and more  
efficient?

Thanks,
-Shane Holloway

-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From george.sakkis at gmail.com  Fri Nov  3 14:10:54 2006
From: george.sakkis at gmail.com (George Sakkis)
Date: Fri, 03 Nov 2006 19:10:54 -0000
Subject: numpy.repeat TypeError: array cannot be safely
	cast to required type
Message-ID: <1162581054.217811.174140@k70g2000cwa.googlegroups.com>

Can anyone explain this ?

>>> import numpy as N
>>> x = N.arange(1,6,dtype='B')
>>> x
array([1, 2, 3, 4, 5], dtype=uint8)
>>> N.repeat(x, N.ones(5,'H'))
array([1, 2, 3, 4, 5], dtype=uint8)
>>> N.repeat(x, N.ones(5,'l'))
array([1, 2, 3, 4, 5], dtype=uint8)
>>> N.repeat(x, N.ones(5,'L'))
Traceback (most recent call last):
  File "<stdin>", line 1, in ?
  File
"/usr/local/lib/python2.4/site-packages/numpy/core/fromnumeric.py",
line 83, in repeat
    return repeat(repeats, axis)
TypeError: array cannot be safely cast to required type

Thanks,
George


-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From tim.hochberg at ieee.org  Fri Nov  3 14:43:25 2006
From: tim.hochberg at ieee.org (Tim Hochberg)
Date: Fri, 03 Nov 2006 12:43:25 -0700
Subject: numpy.repeat TypeError: array cannot be
 safely cast to required type
In-Reply-To: <1162581054.217811.174140@k70g2000cwa.googlegroups.com>
References: <1162581054.217811.174140@k70g2000cwa.googlegroups.com>
Message-ID: <454B9BDD.2060101@ieee.org>

George Sakkis wrote:
> Can anyone explain this ?
>
>   
>>>> import numpy as N
>>>> x = N.arange(1,6,dtype='B')
>>>> x
>>>>         
> array([1, 2, 3, 4, 5], dtype=uint8)
>   
>>>> N.repeat(x, N.ones(5,'H'))
>>>>         
> array([1, 2, 3, 4, 5], dtype=uint8)
>   
>>>> N.repeat(x, N.ones(5,'l'))
>>>>         
> array([1, 2, 3, 4, 5], dtype=uint8)
>   
>>>> N.repeat(x, N.ones(5,'L'))
>>>>         
> Traceback (most recent call last):
>   File "<stdin>", line 1, in ?
>   File
> "/usr/local/lib/python2.4/site-packages/numpy/core/fromnumeric.py",
> line 83, in repeat
>     return repeat(repeats, axis)
> TypeError: array cannot be safely cast to required type
>   
It means you can't safely a value of type uint32 ('L') to a value of 
type int32 ('i').  The second argument of repeat needs to be castable to 
type int32.

A style note: please use the named dtypes (int32, uint32, etc) rather 
than the old-style letter codes; the former is much clearer. The answer 
to your question might  have been immediately apparent had you been 
using named dtypes.

 Personally, I'd also prefer people use the "ones([n])" syntax instead 
of the I-wish-it-were-deprecated-but-it's-too-much-to-hope-for "ones(n)" 
syntax.  T

-tim


-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From george.sakkis at gmail.com  Fri Nov  3 15:25:54 2006
From: george.sakkis at gmail.com (George Sakkis)
Date: Fri, 03 Nov 2006 20:25:54 -0000
Subject: numpy.repeat TypeError: array cannot be
	safely cast to required type
References: <1162581054.217811.174140@k70g2000cwa.googlegroups.com>
	<454B9BDD.2060101@ieee.org>
Message-ID: <1162585554.214708.108210@k70g2000cwa.googlegroups.com>

Tim Hochberg wrote:

> George Sakkis wrote:
> > Can anyone explain this ?
> >
> >
> >>>> import numpy as N
> >>>> x = N.arange(1,6,dtype='B')
> >>>> x
> >>>>
> > array([1, 2, 3, 4, 5], dtype=uint8)
> >
> >>>> N.repeat(x, N.ones(5,'H'))
> >>>>
> > array([1, 2, 3, 4, 5], dtype=uint8)
> >
> >>>> N.repeat(x, N.ones(5,'l'))
> >>>>
> > array([1, 2, 3, 4, 5], dtype=uint8)
> >
> >>>> N.repeat(x, N.ones(5,'L'))
> >>>>
> > Traceback (most recent call last):
> >   File "<stdin>", line 1, in ?
> >   File
> > "/usr/local/lib/python2.4/site-packages/numpy/core/fromnumeric.py",
> > line 83, in repeat
> >     return repeat(repeats, axis)
> > TypeError: array cannot be safely cast to required type
> >
> It means you can't safely a value of type uint32 ('L') to a value of
> type int32 ('i').  The second argument of repeat needs to be castable to
> type int32.

I see. I suppose there is a better reason for this than a "why on earth
would anyone repeat a value more than 2**32-1 times". Besides, why
should it be castable to a signed type since we're talking about number
of repetitions ?

> A style note: please use the named dtypes (int32, uint32, etc) rather
> than the old-style letter codes; the former is much clearer. The answer
> to your question might  have been immediately apparent had you been
> using named dtypes.

I'm using numpy along with the struct module, so the letter codes are
convenient for talking to both modules.

George


-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From oliphant at ee.byu.edu  Fri Nov  3 15:26:23 2006
From: oliphant at ee.byu.edu (Travis Oliphant)
Date: Fri, 03 Nov 2006 13:26:23 -0700
Subject: I won't be as visible for a few weeks
Message-ID: <454BA5EF.6070309@ee.byu.edu>



I'm writing this to indicate that I won't be as visible on the lists for 
a few weeks as I have some pressing matters to attend to that will take 
more of my time.

I apologize for not being able to help more right now.

Best regards,

-Travis Oliphant


-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From tim.hochberg at ieee.org  Fri Nov  3 15:39:15 2006
From: tim.hochberg at ieee.org (Tim Hochberg)
Date: Fri, 03 Nov 2006 13:39:15 -0700
Subject: numpy.repeat TypeError: array cannot
 be	safely cast to required type
In-Reply-To: <1162585554.214708.108210@k70g2000cwa.googlegroups.com>
References: <1162581054.217811.174140@k70g2000cwa.googlegroups.com>	<454B9BDD.2060101@ieee.org>
	<1162585554.214708.108210@k70g2000cwa.googlegroups.com>
Message-ID: <454BA8F3.3020609@ieee.org>

George Sakkis wrote:
> Tim Hochberg wrote:
>
>   
>> George Sakkis wrote:
>>     
>>> Can anyone explain this ?
>>>
>>>
>>>       
>>>>>> import numpy as N
>>>>>> x = N.arange(1,6,dtype='B')
>>>>>> x
>>>>>>
>>>>>>             
>>> array([1, 2, 3, 4, 5], dtype=uint8)
>>>
>>>       
>>>>>> N.repeat(x, N.ones(5,'H'))
>>>>>>
>>>>>>             
>>> array([1, 2, 3, 4, 5], dtype=uint8)
>>>
>>>       
>>>>>> N.repeat(x, N.ones(5,'l'))
>>>>>>
>>>>>>             
>>> array([1, 2, 3, 4, 5], dtype=uint8)
>>>
>>>       
>>>>>> N.repeat(x, N.ones(5,'L'))
>>>>>>
>>>>>>             
>>> Traceback (most recent call last):
>>>   File "<stdin>", line 1, in ?
>>>   File
>>> "/usr/local/lib/python2.4/site-packages/numpy/core/fromnumeric.py",
>>> line 83, in repeat
>>>     return repeat(repeats, axis)
>>> TypeError: array cannot be safely cast to required type
>>>
>>>       
>> It means you can't safely a value of type uint32 ('L') to a value of
>> type int32 ('i').  The second argument of repeat needs to be castable to
>> type int32.
>>     
>
> I see. I suppose there is a better reason for this than a "why on earth
> would anyone repeat a value more than 2**32-1 times". Besides, why
> should it be castable to a signed type since we're talking about number
> of repetitions ?
>   
I suspect it's just a matter of simplicity although I haven't looked at 
the code in question. On the way in all of the arrays are going to need 
to be cast to some standard integral type. While unit32 might make 
sense, int32 is by far the more common type. If one was to use uint32 as 
the type, before  casting int32 to uint32, one would have to scan the 
array looking for negative values lest they get cast into really big 
positive values. An analogous problem exists for casting uint32 to 
int32, but since unit32 is rather uncommon, one can just make users do 
the necessary checking and casting by hand and probably someone will 
only notice every couple of years.


>> A style note: please use the named dtypes (int32, uint32, etc) rather
>> than the old-style letter codes; the former is much clearer. The answer
>> to your question might  have been immediately apparent had you been
>> using named dtypes.
>>     
>
> I'm using numpy along with the struct module, so the letter codes are
> convenient for talking to both modules.
>   
I imagine they are, but they're harder for me, and I suspect many 
others, to remember. Your more likely to get a useful answer if your 
readers don't have to go look stuff up.

-tim




-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From ahmed7 at nyu.edu  Fri Nov  3 20:35:40 2006
From: ahmed7 at nyu.edu (Info)
Date: Sat, 04 Nov 2006 04:35:40 +0300
Subject: =?windows-1251?b?xOXt/CDz5ODr8f8=?=
Message-ID: <6bca01c6ffb1$6cdf2d88$aa88ca2c@nyu.edu>

?????? ????!


-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
_______________________________________________
Numpy-discussion mailing list
Numpy-discussion at lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/numpy-discussion

From oliphant.travis at ieee.org  Fri Nov  3 20:56:39 2006
From: oliphant.travis at ieee.org (Travis Oliphant)
Date: Fri, 03 Nov 2006 18:56:39 -0700
Subject: Reading records from file and sorting
In-Reply-To: <1162442534.508320.151770@e3g2000cwe.googlegroups.com>
References: <1162337923.747767.323100@b28g2000cwb.googlegroups.com>	<20061101224620.GA29602@dogbert.sdsl.sun.ac.za>
	<1162442534.508320.151770@e3g2000cwe.googlegroups.com>
Message-ID: <454BF357.9020903@ieee.org>

George Sakkis wrote:
> Albert Strasheim wrote:
>
>   
>> Check the thread "Strange results when sorting array with fields" from
>> about a week back. Travis made some changes to sorting in the presence
>> of fields that should solve your problem, assuming your fields appear in
>> the order you want to sort (i.e. you want to sort f1, f2, f3 and not
>> something like f1, f3, f2).
>>     
>
> I'm afraid this won't help in my case; I want to sort twice, once by f1
> and once by f2. I guess I could make a second file with the fields
> swapped but this seems more messy and inefficient than Francesc's
> suggestion.
>   

As a final contribution before I become more quiet for a few weeks (and 
aside from hopefully releasing 1.0.1 soon), I've added a feature to 
allow specifying the sorting order for record arrays.

It's added to the sort method as the order= keyword.  You can pass in a 
string (specifying which field comes first --- all other fields will 
stay in the same order) or you can pass in a list or tuple which 
indicates the field order (any fields un-specified will stay in their 
same relative order).

It works be creating a new data-type object with the .names attribute 
replaced with a newly ordered one and then calling sort on a view of the 
array with that data-type.  The VOID_compare uses the names tuple to 
determine the ordering.

This was the un-named option in my previous list.  It's a better 
solution than all of the others, I think.  The newly created data-type 
is discarded after the sorting is complete.

-Travis


-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From lyfebwkyrrh at optical-center.com  Fri Nov  3 23:22:35 2006
From: lyfebwkyrrh at optical-center.com (require)
Date: Sat, 4 Nov 2006 05:22:35 +0100
Subject: Americans highly declined percent
Message-ID: <001201c6ffc8$da888540$73cb00c3@gustavson>


Ee in spec task potential hazards model.
Aristocats Rescuers Newhart Pocahontas! Accounting Admin Clerical Banking Biotech Consultant a Customer Education a.
Aristocats Rescuers Newhart Pocahontas!
Shell services committed of furthering volunteers or Compliance. Img off Pingbacks. Muffin am Crash Vids Webndex Gorilla is Mask Shavehead Uber a Ytmnd.
Rescuers Newhart Pocahontas Irene Bedard Cast Crew Castclint Howard of.
Pdtaol Usersjj or Pdtwhose a itvolterwd. Ipod trademark Apple partner sponsor emusiccom or Skinsmore Classicz is Alien.
Copyleft resulting gpl gpl wider. Avoider in Deluxe or fiddly of.
Bca wide in Greece or influenced became prominent Greeks or. Jfrancis in Daniel in Glazman Neil Rashbrook of parkwaycc uk.
More capital billed. Hop Anime raquoaol Launches Enhanced.
Currently of ant Note h Quicktime Mplayer same in video.
Compile sourcesuse command in ipodsa indepth am. More capital billed.
Dialogs am documents subscribe.
Halloway hearing or sleep scene occurs duplicate. Ltlt lt Prev gt?
Protecting crucial Attorney speaks Patches is holes emerge of Rivals.
Claws of claws so damage scratches Cats Saves am Furniture. Hypnotic Wannabe Webcam cutie assets Lust am Bust.
Steps millions a service Dulles Vabased.
Servers analogue Millennium Dmcpa Dmca prohibits indirect brutally. Reading draft Platform Enterprise Edition aka jsr of noticed or nothing.
Systems contains verify cost world homepages tracking ftp.
Halloway hearing sleep scene occurs duplicate or earlier one. Reading draft Platform Enterprise Edition aka jsr of noticed or nothing.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20061104/3ed4c9e5/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: therefore.gif
Type: image/gif
Size: 7982 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20061104/3ed4c9e5/attachment.gif>
-------------- next part --------------
-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
-------------- next part --------------
_______________________________________________
Numpy-discussion mailing list
Numpy-discussion at lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/numpy-discussion

From bjrtjhnbv at pbna.com  Sat Nov  4 02:42:52 2006
From: bjrtjhnbv at pbna.com (Pages)
Date: Sat, 4 Nov 2006 03:42:52 -0400
Subject: expressed dislike
Message-ID: <001201c6ffe4$d5a478b0$4ef9c4a1@KCGT4G2>


Ironware and up ant reg?
Idiom is humorous effect criticized adhering expressed. Middotsign middothelp am Username forin postsin tagsin is.
Idiom is humorous effect criticized adhering expressed.
Xen in Paul rage days Advances well increasing energy make. Person a Cutter lossless am wav cutting Font Fitting cache. Wikimedia Foundation Seattle Times Comics Gameson error in resume.
Dion or Peter Cetera list or Sounds ros a commentyou.
Patent Leather appeared featured of unknown Depeche produced Daniel. Warnings Shoutbox Folding Processors Memory.
Minute principle odds finest courage. Ebuild binary summed succinctly postwhat now generic non is.
Return Type nsieditor numerous? Developers encourage in do discourage jackass am sole.
Intheknow Donrsquot tomorrow is guarantee. Adams Morgansix blue interior or ne Dcthu reclyner or woodbridge.
Gone taking in Roberts suggestion entered title.
Madness disc finishing exe cd inserted or should? Intheknow Donrsquot tomorrow is guarantee.
Newspaper Ostg or Thinkgeek or Itmj Linuxcom am!
Background university nascent in timeedit! Fellow musicians trends favorite pmchaz.
Dog in distro is kai Staats ceo.
Giving Labs James Wards is brief or. Crazy longtime intrigued presents of tag am.
Weak toolsread Printinghp.
Terminate withdraw deposit deliver process departing lti in package of. Enrogue in tricky right snippet workxul var null me.
Digitalb nddigitalb a a Galaxie ndgalaxie.
Rules Procedures Integrated Sustained Judging Wondering achieved. Enrogue in tricky right snippet workxul var null me.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20061104/8a0bf29c/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: central.gif
Type: image/gif
Size: 7758 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20061104/8a0bf29c/attachment.gif>
-------------- next part --------------
-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
-------------- next part --------------
_______________________________________________
Numpy-discussion mailing list
Numpy-discussion at lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/numpy-discussion

From Mailer-Daemon at lists.sourceforge.net  Sat Nov  4 03:44:41 2006
From: Mailer-Daemon at lists.sourceforge.net (Mail Delivery System)
Date: Sat, 04 Nov 2006 00:44:41 -0800
Subject: Warning: message 1GfAoO-0007im-Sh delayed 72
	hours
Message-ID: <E1GgH8r-0007q7-05@externalmx-1.sourceforge.net>

This message was created automatically by mail delivery software.
A message that you sent has not yet been delivered to one or more of its
recipients after more than 72 hours on the queue on externalmx-1.sourceforge.net.

The message identifier is:     1GfAoO-0007im-Sh
The subject of the message is: looked like me the full sized HARLEY and began to her, with this but as
The date of the message is:    Wed, 01 Nov 2006 15:46:73 +0800

The address to which the message has not yet been delivered is:

  numpy-discussion at lists.sourceforge.net
    Delay reason: SMTP error from remote mailer after RCPT TO:<numpy-discussion at lists.sourceforge.net>:
    host mail.sourceforge.net [66.35.250.206]: 451-Could not complete sender verify callout
    451-Could not complete sender verify callout for
    451-<numpy-discussion at lists.sourceforge.net>.
    451-The mail server(s) for the domain may be temporarily unreachable, or
    451-they may be permanently unreachable from this server. In the latter case,
    451-you need to change the address or create an MX record for its domain
    451-if it is 

No action is required on your part. Delivery attempts will continue for
some time, and this warning may be repeated at intervals if the message
remains undelivered. Eventually the mail delivery software will give up,
and when that happens, the message will be returned to you.

-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From cjw at sympatico.ca  Sat Nov  4 07:30:07 2006
From: cjw at sympatico.ca (Colin J. Williams)
Date: Sat, 04 Nov 2006 07:30:07 -0500
Subject: Style - was Re: numpy.repeat TypeError: array
 cannot be safely cast to required type
In-Reply-To: <454B9BDD.2060101@ieee.org>
References: <1162581054.217811.174140@k70g2000cwa.googlegroups.com>
	<454B9BDD.2060101@ieee.org>
Message-ID: <454C87CF.7030500@sympatico.ca>



Tim Hochberg wrote:
[snip]
> A style note: please use the named dtypes (int32, uint32, etc) rather 
> than the old-style letter codes; the former is much clearer. The answer 
> to your question might  have been immediately apparent had you been 
> using named dtypes.
>
>   
+1
>  Personally, I'd also prefer people use the "ones([n])" syntax instead 
> of the I-wish-it-were-deprecated-but-it's-too-much-to-hope-for "ones(n)" 
> syntax.  T
>
> -tim
>
>   
Could you elaborate please?

Colin W.

-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From mcfletch at vrplumber.com  Sat Nov  4 11:21:12 2006
From: mcfletch at vrplumber.com (Mike C. Fletcher)
Date: Sat, 04 Nov 2006 11:21:12 -0500
Subject: [PyOpenGL-Devel] Need more comments from
 scientific community on python-dev
In-Reply-To: <C23A1918-5E8B-4033-8969-45B20B098CB6@gmail.com>
References: <mailman.90862.1162320870.22397.numpy-discussion@lists.sourceforge.net>
	<C23A1918-5E8B-4033-8969-45B20B098CB6@gmail.com>
Message-ID: <454CBDF8.6050807@vrplumber.com>

Josh Marshall wrote:
> On 11/31/06, Fernando Perez <fperez.net at gmail.com> wrote:
>
>   
>> Fernando Perez wrote:
>> ps - one more thing.  This guy:
>>
>> http://blog.vrplumber.com/
>>
>> has been rewriting the OpenGL bindings using ctypes, and I've seen
>> posts from him about numpy (in his blog).  He might be able to
>> contribute something...
>>     
...
> Now, I haven't spent much time looking at these parts of OpenGL- 
> ctypes, as they have just worked for me. I would think that it would  
> be trivial to write a FormatHandler which uses the ndarray interface  
> and data type description to use any object implenting it as an input  
> for OpenGL. This would include things such as PIL images.
>   
Yes, this is the purpose of the design, to allow users to write new 
format handlers that can be registered at run-time and act as 
first-class data-formats within the system.  PIL images, custom vector 
objects, memory-mapped files, media buffers and the like are all 
contemplated as useful array storage formats.

> Mike, can you give us your opinion on how a standardised data type  
> descriptor would be helpful for PyOpenGL? The PEP and some  
> information about it can be found here:
> http://www.scipy.org/ArrayInterfacePEP
>   
A standardised data-type descriptor would be of some benefit for us, as 
it would allow a single handler to deal with larger numbers of storage 
formats (reducing the amount of coding required).  That said, the spec 
as proposed has far more features than *most* OpenGL users will use 
(OpenGL being largely (though not exclusively) focused on simple, 
homogeneous data-types), and the handler mechanism largely abstracts 
away the problem for *us* at the moment.  We wouldn't be able to get rid 
of the abstraction mechanism entirely, as we would still have data-types 
such as lists-of-lists-of-integers that wouldn't support the protocol.

The likely efficiency of accessing the metadata (versus current 
implementation where in some cases we wind up producing a whole 
dictionary object with meta-data just to pull out a single value) would 
be attractive.

We also need functionality to get at the data-pointer for the objects to 
make any description useful, though I suppose that's out of scope for 
this particular PEP.

HTH,
Mike

-- 
________________________________________________
  Mike C. Fletcher
  Designer, VR Plumber, Coder
  http://www.vrplumber.com
  http://blog.vrplumber.com


-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From ndarray at mac.com  Sat Nov  4 13:03:24 2006
From: ndarray at mac.com (Sasha)
Date: Sat, 4 Nov 2006 13:03:24 -0500
Subject: Need more comments from scientific community
	on python-dev
In-Reply-To: <d38f5330611030756w33b5f026t92282f6ed571e66a@mail.gmail.com>
References: <45474841.9030802@ieee.org> <4547B235.6050904@ee.byu.edu>
	<d38f5330610311336w7538e0b2v994ee277da64f6cf@mail.gmail.com>
	<4547D2B3.5000808@ee.byu.edu>
	<d38f5330610311551u4e71cdd7ndc0b93de16dc6d80@mail.gmail.com>
	<4548C3D6.40100@ieee.org>
	<ce557a360611011013v4e883251l46dc6e2c0c85b854@mail.gmail.com>
	<e86a5fd00611011019p5a39553cwf55fd06e585e58e0@mail.gmail.com>
	<e7bda7770611022238y2cb15e75s3739d2b4b652129c@mail.gmail.com>
	<d38f5330611030756w33b5f026t92282f6ed571e66a@mail.gmail.com>
Message-ID: <d38f5330611041003v6c81f856qda00d518cef62f38@mail.gmail.com>

On 11/3/06, Torgil Svensson <torgil.svensson at gmail.com> wrote:
> class struct_type(Structure):
>     _fields_ = [....]
>
> ...
> ... which is somewhat static in nature. How do you create "structures"
> dynamically?
>

You can put the above in a function that takes fields as an argument, or

    type('struct_type', (Structure,), {'_fields_':[('a', c_int), ('b',
c_long)]})

-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From tim.hochberg at ieee.org  Sat Nov  4 15:23:11 2006
From: tim.hochberg at ieee.org (Tim Hochberg)
Date: Sat, 04 Nov 2006 13:23:11 -0700
Subject: Style - was Re: numpy.repeat TypeError:
 array cannot be safely cast to required type
In-Reply-To: <454C87CF.7030500@sympatico.ca>
References: <1162581054.217811.174140@k70g2000cwa.googlegroups.com>	<454B9BDD.2060101@ieee.org>
	<454C87CF.7030500@sympatico.ca>
Message-ID: <454CF6AF.10205@ieee.org>

Colin J. Williams wrote:
> Tim Hochberg wrote:
> [snip]
>   
>> A style note: please use the named dtypes (int32, uint32, etc) rather 
>> than the old-style letter codes; the former is much clearer. The answer 
>> to your question might  have been immediately apparent had you been 
>> using named dtypes.
>>
>>   
>>     
> +1
>   
>>  Personally, I'd also prefer people use the "ones([n])" syntax instead 
>> of the I-wish-it-were-deprecated-but-it's-too-much-to-hope-for "ones(n)" 
>> syntax.  T
>>
>> -tim
>>
>>   
>>     
> Could you elaborate please?
>
>   
Sure. The general form of the zeros function, and several others[1], is:

    zeros(shape, dtype)

 Here 'shape' is a sequence of one sort or another. There's also a 
second form that's applicable only to one dimensional arrays:

    zeros(length, dtype)

Where length is an integer. I don't recall if this is a historical 
legacy or is intended as a convenience function or a bit of both. Either 
way, the result is that there are two ways to spell "give me a 1D array 
of zeros with a given length and dtype":

    zeros([length], dtype)

and

    zeros(length, dtype)


I have two issues with having this second spelling. First, it's one more 
thing to remember. Whenever I see the scalar spelling I have to use a 
little bit extra of my limited brainpower to remember that it is not in 
fact a typo, but is instead a shortcut. The second issue is pedagogical. 
If people are initially exposed to the first form, the extension to 
multiple dimensions is straightforward. They'll probably guess the 
correct way right off the bat, and if not, they'll get it right away 
when it's explained. On the other hand, if they are initially exposed to 
the second form, the multidimensional form is far from obvious. In 
addition, they'll probably spend a long time thinking that the 
one-dimensional way is the normal way, but that we have to jump through 
weird hoops to get multidimensional arrays to work. That's bad 
propaganda for numpy. This all seems like a rather large price to pay to 
avoid typing the occasional pair of brackets.

That's my two cents. Just say not to form #2.

-tim



... [1] Yes, I'm neglecting the order parameter; I don't think it 
matters for this discussion.



-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From dalcinl at gmail.com  Sat Nov  4 16:12:01 2006
From: dalcinl at gmail.com (Lisandro Dalcin)
Date: Sat, 4 Nov 2006 18:12:01 -0300
Subject: [PyOpenGL-Devel] Need more comments from
	scientific community on python-dev
In-Reply-To: <454CBDF8.6050807@vrplumber.com>
References: <mailman.90862.1162320870.22397.numpy-discussion@lists.sourceforge.net>
	<C23A1918-5E8B-4033-8969-45B20B098CB6@gmail.com>
	<454CBDF8.6050807@vrplumber.com>
Message-ID: <e7ba66e40611041312q2973015elbdd18b41b2698999@mail.gmail.com>

On 11/4/06, Mike C. Fletcher <mcfletch at vrplumber.com> wrote:
>  We wouldn't be able to get rid
> of the abstraction mechanism entirely, as we would still have data-types
> such as lists-of-lists-of-integers that wouldn't support the protocol.
>

Perhaps this will sound stupid, but I was thinking about extending
some builtin Python types to provide datatype info. A list/tuple is an
array of Python objects, an int/float/complex could return a datatype
upon query. This could be very efficient if some predefined datatypes
were already defined in the C-side. MPI provides predefined datatypes
(MPI_INT, MPI_FLOAT) and new user-defined datatypes are created from
them.


-- 
Lisandro Dalc?n
---------------
Centro Internacional de M?todos Computacionales en Ingenier?a (CIMEC)
Instituto de Desarrollo Tecnol?gico para la Industria Qu?mica (INTEC)
Consejo Nacional de Investigaciones Cient?ficas y T?cnicas (CONICET)
PTLC - G?emes 3450, (3000) Santa Fe, Argentina
Tel/Fax: +54-(0)342-451.1594

-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From arildna at stud.ntnu.no  Sat Nov  4 16:22:19 2006
From: arildna at stud.ntnu.no (arildna at stud.ntnu.no)
Date: Sat,  4 Nov 2006 22:22:19 +0100
Subject: problems installing NumPy on OSX
Message-ID: <20061104222219.5v1dj889q8cgsg8c@webmail.ntnu.no>

Hi,

I'm sorry if this might seem like a stupid question to some of you,  
but I have been struggling for the better part of the afternoon trying  
to install NumPy on my G4 iBook, so I hope somebody can take the time  
to lend me a helping hand. I have searched a little in the mail  
archives and tried to follow the instructions on this page:
http://www.scipy.org/Installing_SciPy/Mac_OS_X

Python 2.5 is installed, and I have installed Apple's Developer's  
Tools (i.e. the package "December 2002 Mac OS X Developer Tools"). The  
GCC/G77-installation seemed to go ok.

When I try to install the FFW libraries however, I get an error  
message telling me that my "C compiler cannot create executables"  
(full output below).

After that I didn't really expect the installation of NumPy to work,  
but I got a "permission denied"-error message that I'm not so sure is  
connected to the missing FFW.

So I'm at loss... Anyone?

Many thanks,

    Arild N?ss

===============================================================

%pwd
/Users/arildnss/Desktop/fftw-3.1.2

%./configure
checking for a BSD-compatible install... /usr/bin/install -c
checking whether build environment is sane... yes
checking for gawk... no
checking for mawk... no
checking for nawk... no
checking for awk... awk
checking whether make sets $(MAKE)... yes
checking whether to enable maintainer-specific portions of Makefiles... no
checking build system type... powerpc-apple-darwin8.8.0
checking host system type... powerpc-apple-darwin8.8.0
checking for gcc... gcc
checking for C compiler default output file name... configure: error:  
C compiler cannot create executables
See `config.log' for more details.


%pwd
/Users/arildnss/Desktop/numpy-1.0

%python setup.py build
Running from numpy source directory.
F2PY Version 2_3396
blas_opt_info:
   FOUND:
     extra_link_args = ['-Wl,-framework', '-Wl,Accelerate']
     define_macros = [('NO_ATLAS_INFO', 3)]
     extra_compile_args = ['-faltivec',  
'-I/System/Library/Frameworks/vecLib.framework/Headers']

lapack_opt_info:
   FOUND:
     extra_link_args = ['-Wl,-framework', '-Wl,Accelerate']
     define_macros = [('NO_ATLAS_INFO', 3)]
     extra_compile_args = ['-faltivec']

running build
running config_fc
running build_src
building py_modules sources
error: build/src.macosx-10.3-fat-2.5/numpy/distutils/__config__.py:  
Permission denied



-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From robert.kern at gmail.com  Sat Nov  4 16:58:35 2006
From: robert.kern at gmail.com (Robert Kern)
Date: Sat, 04 Nov 2006 15:58:35 -0600
Subject: problems installing NumPy on OSX
In-Reply-To: <20061104222219.5v1dj889q8cgsg8c@webmail.ntnu.no>
References: <20061104222219.5v1dj889q8cgsg8c@webmail.ntnu.no>
Message-ID: <eij2ej$245$1@sea.gmane.org>

arildna at stud.ntnu.no wrote:
> Hi,
> 
> I'm sorry if this might seem like a stupid question to some of you,  
> but I have been struggling for the better part of the afternoon trying  
> to install NumPy on my G4 iBook, so I hope somebody can take the time  
> to lend me a helping hand. I have searched a little in the mail  
> archives and tried to follow the instructions on this page:
> http://www.scipy.org/Installing_SciPy/Mac_OS_X

Note that these are instructions for installing scipy, so there is a lot there
that you don't need to do just to install numpy. Notably, some of the things
that you are having problems with.

> Python 2.5 is installed, and I have installed Apple's Developer's  
> Tools (i.e. the package "December 2002 Mac OS X Developer Tools"). The  
> GCC/G77-installation seemed to go ok.

You probably can't use g77 along with Python 2.5. Python 2.5 is built with gcc
4.0 but g77 never got ported to the 4.x platform. However, it's not at all
necessary for numpy.

> When I try to install the FFW libraries however, I get an error  
> message telling me that my "C compiler cannot create executables"  
> (full output below).

numpy does not use FFTW at all, so don't let this hold you up. If you want to
troubleshoot the problem, however, look at the config.log file for more
information. grep for the string "cannot create executables".

> After that I didn't really expect the installation of NumPy to work,  
> but I got a "permission denied"-error message that I'm not so sure is  
> connected to the missing FFW.

You probably tried to run a previous build or install as root. Delete the build/
directory (probably as root) and try another build as a regular user. Don't use
root until you actually want to install (and then, only if you need to).

-- 
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless enigma
 that is made terrible by our own mad attempt to interpret it as though it had
 an underlying truth."
  -- Umberto Eco


-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From arildna at stud.ntnu.no  Sat Nov  4 17:32:10 2006
From: arildna at stud.ntnu.no (arildna at stud.ntnu.no)
Date: Sat,  4 Nov 2006 23:32:10 +0100
Subject: problems installing NumPy on OSX
In-Reply-To: <eij2ej$245$1@sea.gmane.org>
References: <20061104222219.5v1dj889q8cgsg8c@webmail.ntnu.no>
	<eij2ej$245$1@sea.gmane.org>
Message-ID: <20061104233210.xfx6c75uswss0gsw@webmail.ntnu.no>

Siterer Robert Kern <robert.kern at gmail.com>:

> arildna at stud.ntnu.no wrote:
>> Hi,
>>
>> I'm sorry if this might seem like a stupid question to some of you,
>> but I have been struggling for the better part of the afternoon trying
>> to install NumPy on my G4 iBook, so I hope somebody can take the time
>> to lend me a helping hand. I have searched a little in the mail
>> archives and tried to follow the instructions on this page:
>> http://www.scipy.org/Installing_SciPy/Mac_OS_X
>
> Note that these are instructions for installing scipy, so there is a  
>  lot there
> that you don't need to do just to install numpy. Notably, some of the things
> that you are having problems with.

I did get the impression that it was kind of an overkill...

>> After that I didn't really expect the installation of NumPy to work,
>> but I got a "permission denied"-error message that I'm not so sure is
>> connected to the missing FFW.
>
> You probably tried to run a previous build or install as root.   
> Delete the build/
> directory (probably as root) and try another build as a regular   
> user. Don't use
> root until you actually want to install (and then, only if you need to).

That got the installation a lot further, but it still halts with an  
error message, this time about failing to test the configuration.

    Arild N?ss


==========================================================================

% python setup.py build

Running from numpy source directory.
F2PY Version 2_3396
blas_opt_info:
   FOUND:
     extra_link_args = ['-Wl,-framework', '-Wl,Accelerate']
     define_macros = [('NO_ATLAS_INFO', 3)]
     extra_compile_args = ['-faltivec',  
'-I/System/Library/Frameworks/vecLib.framework/Headers']

lapack_opt_info:
   FOUND:
     extra_link_args = ['-Wl,-framework', '-Wl,Accelerate']
     define_macros = [('NO_ATLAS_INFO', 3)]
     extra_compile_args = ['-faltivec']

running build
running config_fc
running build_src
building py_modules sources
creating build
creating build/src.macosx-10.3-fat-2.5
creating build/src.macosx-10.3-fat-2.5/numpy
creating build/src.macosx-10.3-fat-2.5/numpy/distutils
building extension "numpy.core.multiarray" sources
creating build/src.macosx-10.3-fat-2.5/numpy/core
Generating build/src.macosx-10.3-fat-2.5/numpy/core/config.h
customize NAGFCompiler
customize AbsoftFCompiler
customize IbmFCompiler
Could not locate executable g77
Could not locate executable f77
Could not locate executable gfortran
Could not locate executable f95
customize GnuFCompiler
customize Gnu95FCompiler
customize G95FCompiler
customize GnuFCompiler
customize Gnu95FCompiler
customize NAGFCompiler
customize NAGFCompiler using config
C compiler: gcc -arch ppc -arch i386 -isysroot  
/Developer/SDKs/MacOSX10.4u.sdk -fno-strict-aliasing -Wno-long-double  
-no-cpp-precomp -mno-fused-madd -fno-common -dynamic -DNDEBUG -g -O3

compile options:  
'-I/Library/Frameworks/Python.framework/Versions/2.5/include/python2.5  
-Inumpy/core/src -Inumpy/core/include  
-I/Library/Frameworks/Python.framework/Versions/2.5/include/python2.5  
-c'
gcc: _configtest.c
gcc: cannot specify -o with -c or -S and multiple compilations
gcc: cannot specify -o with -c or -S and multiple compilations
failure.
removing: _configtest.c _configtest.o
numpy/core/setup.py:50: DeprecationWarning: raising a string exception  
is deprecated
   raise "ERROR: Failed to test configuration"
Traceback (most recent call last):
   File "setup.py", line 89, in <module>
     setup_package()
   File "setup.py", line 82, in setup_package
     configuration=configuration )
   File "/Users/arildnss/Desktop/numpy-1.0/numpy/distutils/core.py",  
line 174, in setup
     return old_setup(**new_attr)
   File  
"/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/distutils/core.py", line 151, in  
setup
     dist.run_commands()
   File  
"/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/distutils/dist.py", line 974, in  
run_commands
     self.run_command(cmd)
   File  
"/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/distutils/dist.py", line 994, in  
run_command
     cmd_obj.run()
   File  
"/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/distutils/command/build.py", line 112, in  
run
     self.run_command(cmd_name)
   File  
"/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/distutils/cmd.py", line 333, in  
run_command
     self.distribution.run_command(command)
   File  
"/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/distutils/dist.py", line 994, in  
run_command
     cmd_obj.run()
   File  
"/Users/arildnss/Desktop/numpy-1.0/numpy/distutils/command/build_src.py", line  
87, in run
     self.build_sources()
   File  
"/Users/arildnss/Desktop/numpy-1.0/numpy/distutils/command/build_src.py", line  
106, in build_sources
     self.build_extension_sources(ext)
   File  
"/Users/arildnss/Desktop/numpy-1.0/numpy/distutils/command/build_src.py", line  
212, in build_extension_sources
     sources = self.generate_sources(sources, ext)
   File  
"/Users/arildnss/Desktop/numpy-1.0/numpy/distutils/command/build_src.py", line  
270, in generate_sources
     source = func(extension, build_dir)
   File "numpy/core/setup.py", line 50, in generate_config_h
     raise "ERROR: Failed to test configuration"
ERROR: Failed to test configuration


-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From robert.kern at gmail.com  Sat Nov  4 17:36:04 2006
From: robert.kern at gmail.com (Robert Kern)
Date: Sat, 04 Nov 2006 16:36:04 -0600
Subject: problems installing NumPy on OSX
In-Reply-To: <20061104233210.xfx6c75uswss0gsw@webmail.ntnu.no>
References: <20061104222219.5v1dj889q8cgsg8c@webmail.ntnu.no>	<eij2ej$245$1@sea.gmane.org>
	<20061104233210.xfx6c75uswss0gsw@webmail.ntnu.no>
Message-ID: <eij4kr$7l2$1@sea.gmane.org>

arildna at stud.ntnu.no wrote:
> gcc: _configtest.c
> gcc: cannot specify -o with -c or -S and multiple compilations
> gcc: cannot specify -o with -c or -S and multiple compilations
> failure.

You're using OS X 10.3.9? There is a bug in distutils on that platform. It was
fixed in Python's trunk, but I'm not sure where to find it.

-- 
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless enigma
 that is made terrible by our own mad attempt to interpret it as though it had
 an underlying truth."
  -- Umberto Eco


-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From robert.kern at gmail.com  Sat Nov  4 17:37:49 2006
From: robert.kern at gmail.com (Robert Kern)
Date: Sat, 04 Nov 2006 16:37:49 -0600
Subject: problems installing NumPy on OSX
In-Reply-To: <20061104222219.5v1dj889q8cgsg8c@webmail.ntnu.no>
References: <20061104222219.5v1dj889q8cgsg8c@webmail.ntnu.no>
Message-ID: <eij4o3$7l2$2@sea.gmane.org>

arildna at stud.ntnu.no wrote:

> Python 2.5 is installed, and I have installed Apple's Developer's  
> Tools (i.e. the package "December 2002 Mac OS X Developer Tools").

By the way, this is *really* old. Long before there were Universal binaries
(which is what Python 2.5 is). I believe you will want to install Xcode 2.4.1
which was released on Tuesday.

-- 
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless enigma
 that is made terrible by our own mad attempt to interpret it as though it had
 an underlying truth."
  -- Umberto Eco


-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From arildna at stud.ntnu.no  Sun Nov  5 03:09:43 2006
From: arildna at stud.ntnu.no (arildna at stud.ntnu.no)
Date: Sun,  5 Nov 2006 09:09:43 +0100
Subject: problems installing NumPy on OSX
In-Reply-To: <eij4kr$7l2$1@sea.gmane.org>
References: <20061104222219.5v1dj889q8cgsg8c@webmail.ntnu.no>
	<eij2ej$245$1@sea.gmane.org>
	<20061104233210.xfx6c75uswss0gsw@webmail.ntnu.no>
	<eij4kr$7l2$1@sea.gmane.org>
Message-ID: <20061105090943.e33g3827q8oco8s0@webmail.ntnu.no>

Siterer Robert Kern <robert.kern at gmail.com>:

> arildna at stud.ntnu.no wrote:
>> gcc: _configtest.c
>> gcc: cannot specify -o with -c or -S and multiple compilations
>> gcc: cannot specify -o with -c or -S and multiple compilations
>> failure.
>
> You're using OS X 10.3.9? There is a bug in distutils on that   
> platform. It was
> fixed in Python's trunk, but I'm not sure where to find it.
>
> --
> Robert Kern

No, I'm on OS X 10.4.8, so my OS version shouldn't be the problem

    Arild N?ss


-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From arildna at stud.ntnu.no  Sun Nov  5 03:17:27 2006
From: arildna at stud.ntnu.no (arildna at stud.ntnu.no)
Date: Sun,  5 Nov 2006 09:17:27 +0100
Subject: problems installing NumPy on OSX
In-Reply-To: <eij4o3$7l2$2@sea.gmane.org>
References: <20061104222219.5v1dj889q8cgsg8c@webmail.ntnu.no>
	<eij4o3$7l2$2@sea.gmane.org>
Message-ID: <20061105091727.ouibxoltwk40cksg@webmail.ntnu.no>

Siterer Robert Kern <robert.kern at gmail.com>:

> arildna at stud.ntnu.no wrote:
>
>> Python 2.5 is installed, and I have installed Apple's Developer's
>> Tools (i.e. the package "December 2002 Mac OS X Developer Tools").
>
> By the way, this is *really* old. Long before there were Universal binaries
> (which is what Python 2.5 is). I believe you will want to install Xcode 2.4.1
> which was released on Tuesday.
>
> --
> Robert Kern

I was wondering about that, the instructions page on scipy.org wasn't  
very specific. I don't need anything else than Xcode 2.4.1, so I can  
remove the Dec2002 package? And do I need even to run numpy (and not  
scipy)?

    Arild N?ss


-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From robert.kern at gmail.com  Sun Nov  5 03:47:12 2006
From: robert.kern at gmail.com (Robert Kern)
Date: Sun, 05 Nov 2006 02:47:12 -0600
Subject: problems installing NumPy on OSX
In-Reply-To: <20061105091727.ouibxoltwk40cksg@webmail.ntnu.no>
References: <20061104222219.5v1dj889q8cgsg8c@webmail.ntnu.no>	<eij4o3$7l2$2@sea.gmane.org>
	<20061105091727.ouibxoltwk40cksg@webmail.ntnu.no>
Message-ID: <eik8eo$f8l$1@sea.gmane.org>

arildna at stud.ntnu.no wrote:
> Siterer Robert Kern <robert.kern at gmail.com>:
> 
>> arildna at stud.ntnu.no wrote:
>>
>>> Python 2.5 is installed, and I have installed Apple's Developer's
>>> Tools (i.e. the package "December 2002 Mac OS X Developer Tools").
>> By the way, this is *really* old. Long before there were Universal binaries
>> (which is what Python 2.5 is). I believe you will want to install Xcode 2.4.1
>> which was released on Tuesday.
>>
>> --
>> Robert Kern
> 
> I was wondering about that, the instructions page on scipy.org wasn't  
> very specific. I don't need anything else than Xcode 2.4.1, so I can  
> remove the Dec2002 package? And do I need even to run numpy (and not  
> scipy)?

Yes, remove the Dec2002 package and install Xcode 2.4.1.

-- 
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless enigma
 that is made terrible by our own mad attempt to interpret it as though it had
 an underlying truth."
  -- Umberto Eco


-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From Mailer-Daemon at lists.sourceforge.net  Sun Nov  5 05:35:54 2006
From: Mailer-Daemon at lists.sourceforge.net (Mail Delivery System)
Date: Sun, 05 Nov 2006 02:35:54 -0800
Subject: Mail delivery failed: returning message to sender
Message-ID: <E1GgfM2-0002Lb-2g@externalmx-1.sourceforge.net>

This message was created automatically by mail delivery software.

A message that you sent could not be delivered to one or more of its
recipients. This is a permanent error. The following address(es) failed:

  numpy-discussion at lists.sourceforge.net
    SMTP error from remote mailer after RCPT TO:<numpy-discussion at lists.sourceforge.net>:
    host mail.sourceforge.net [66.35.250.206]: 451-Could not complete sender verify callout
    451-Could not complete sender verify callout for
    451-<numpy-discussion at lists.sourceforge.net>.
    451-The mail server(s) for the domain may be temporarily unreachable, or
    451-they may be permanently unreachable from this server. In the latter case,
    451-you need to change the address or create an MX record for its domain
    451-if it is : retry timeout exceeded

------ This is a copy of the message, including all the headers. ------
------ The body of the message is 45596 characters long; only the first
------ 16384 or so are included here.


From lspjsa at ohioflyers.com  Wed Nov  1 02:47:13 2006
From: lspjsa at ohioflyers.com (Alan White)
Date: Wed, 01 Nov 2006 15:46:73 +0800
Subject: looked like me the full sized HARLEY and began to her, with this but as
Message-ID: <000a01c6fd89$e64f9580$0199aa50@qhjxhpymmkur>

An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20061116/843899ef/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: jckbnhtc.png
Type: image/png
Size: 11480 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20061116/843899ef/attachment.png>

From lists.steve at arachnedesign.net  Sun Nov  5 11:16:18 2006
From: lists.steve at arachnedesign.net (Steve Lianoglou)
Date: Sun, 5 Nov 2006 11:16:18 -0500
Subject: problems installing NumPy on OSX
In-Reply-To: <20061105091727.ouibxoltwk40cksg@webmail.ntnu.no>
References: <20061104222219.5v1dj889q8cgsg8c@webmail.ntnu.no>
	<eij4o3$7l2$2@sea.gmane.org>
	<20061105091727.ouibxoltwk40cksg@webmail.ntnu.no>
Message-ID: <CA13C03A-6B1C-483B-B9FF-66965D9E8DAF@arachnedesign.net>

Hi,

> And do I need even to run numpy (and not scipy)?

I'm not sure what you mean with you last question here, but just in  
case it's not clear:

If you want to use scipy, you need to install numpy.
You do *not* have to install scipy if all you need is numpy.

-steve

-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From Mailer-Daemon at lists.sourceforge.net  Sun Nov  5 12:45:22 2006
From: Mailer-Daemon at lists.sourceforge.net (Mail Delivery System)
Date: Sun, 05 Nov 2006 09:45:22 -0800
Subject: Warning: message 1Ggbsf-00056T-Ol delayed 10
	hours
Message-ID: <E1Ggm3e-0005Gu-LP@externalmx-1.sourceforge.net>

This message was created automatically by mail delivery software.
A message that you sent has not yet been delivered to one or more of its
recipients after more than 10 hours on the queue on externalmx-1.sourceforge.net.

The message identifier is:     1Ggbsf-00056T-Ol
The subject of the message is: faces and their former position.  To be, sure.  But question
The date of the message is:    Sun, 05 Nov 2006 16:01:50 +0900

The address to which the message has not yet been delivered is:

  numpy-discussion at lists.sourceforge.net
    Delay reason: SMTP error from remote mailer after RCPT TO:<numpy-discussion at lists.sourceforge.net>:
    host mail.sourceforge.net [66.35.250.206]: 451-Could not complete sender verify callout
    451-Could not complete sender verify callout for
    451-<numpy-discussion at lists.sourceforge.net>.
    451-The mail server(s) for the domain may be temporarily unreachable, or
    451-they may be permanently unreachable from this server. In the latter case,
    451-you need to change the address or create an MX record for its domain
    451-if it is 

No action is required on your part. Delivery attempts will continue for
some time, and this warning may be repeated at intervals if the message
remains undelivered. Eventually the mail delivery software will give up,
and when that happens, the message will be returned to you.

-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From arildna at stud.ntnu.no  Sun Nov  5 13:32:59 2006
From: arildna at stud.ntnu.no (arildna at stud.ntnu.no)
Date: Sun,  5 Nov 2006 19:32:59 +0100
Subject: problems installing NumPy on OSX
In-Reply-To: <CA13C03A-6B1C-483B-B9FF-66965D9E8DAF@arachnedesign.net>
References: <20061104222219.5v1dj889q8cgsg8c@webmail.ntnu.no>
	<eij4o3$7l2$2@sea.gmane.org>
	<20061105091727.ouibxoltwk40cksg@webmail.ntnu.no>
	<CA13C03A-6B1C-483B-B9FF-66965D9E8DAF@arachnedesign.net>
Message-ID: <20061105193259.lbxo4gkzk44wsw4c@webmail.ntnu.no>

Siterer Steve Lianoglou <lists.steve at arachnedesign.net>:

> Hi,
>
>> And do I need even to run numpy (and not scipy)?
>
> I'm not sure what you mean with you last question here, but just in
> case it's not clear:
>
> If you want to use scipy, you need to install numpy.
> You do *not* have to install scipy if all you need is numpy.
>
> -steve

I'm sorry, I was a tad too quick typing there. I meant to say "And do  
I even need to [install Xcode] to run numpy?" Robert pointed out that  
a lot things mentioned in the install guide were necessary to run  
scipy, but that you could run numpy without them.

Therefore I was wondering if installing the newest Xcode package was  
likely to fix the error message I am now getting when trying to  
install numpy:

  File "numpy/core/setup.py", line 50, in generate_config_h
     raise "ERROR: Failed to test configuration"
ERROR: Failed to test configuration

    Arild


-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From lists.steve at arachnedesign.net  Sun Nov  5 14:26:49 2006
From: lists.steve at arachnedesign.net (Steve Lianoglou)
Date: Sun, 5 Nov 2006 14:26:49 -0500
Subject: problems installing NumPy on OSX
In-Reply-To: <20061105193259.lbxo4gkzk44wsw4c@webmail.ntnu.no>
References: <20061104222219.5v1dj889q8cgsg8c@webmail.ntnu.no>
	<eij4o3$7l2$2@sea.gmane.org>
	<20061105091727.ouibxoltwk40cksg@webmail.ntnu.no>
	<CA13C03A-6B1C-483B-B9FF-66965D9E8DAF@arachnedesign.net>
	<20061105193259.lbxo4gkzk44wsw4c@webmail.ntnu.no>
Message-ID: <B5D87503-56A4-462D-B9F2-7EB5F303A5A8@arachnedesign.net>

> I'm sorry, I was a tad too quick typing there. I meant to say "And do
> I even need to [install Xcode] to run numpy?" Robert pointed out that
> a lot things mentioned in the install guide were necessary to run
> scipy, but that you could run numpy without them.
>
> Therefore I was wondering if installing the newest Xcode package was
> likely to fix the error message I am now getting when trying to
> install numpy:

I think Robert may have suggested to install the newest XCode because  
it will give you a newer gcc that can have a better chance compiling  
numpy correctly (or at least will remove another "unkown" to help  
find your true problem).

Maybe there'd be some "Universal Binary-aware"ness that the old xcode  
gcc might be missing that you'll get w/ the new one and since Python  
2.5 is universal, this might be it. Getting the new xcode would be  
the simplest part of the install anyway, so .. why not :-)

-steve



-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From oyffuwebeb at orchardhillchurch.com  Sun Nov  5 23:22:19 2006
From: oyffuwebeb at orchardhillchurch.com (Mambo Mazurka)
Date: Mon, 6 Nov 2006 12:22:19 +0800
Subject: playing
Message-ID: <000e01c7015b$264e8400$da525cde@xswangxl>


Applied diagram effects valueit.
Em in Poker Chip of set.
Holiday Dinner of Kidscope Comic of Playskool Monday or appliance Chocolate am.
Readme License Third Party Licenses jre. Concerti grossi Country.
Puppy Ziggies of yes! Express Sample Robson dvd Video. Sale increases of Booksh increase in if.
Stores Songs Plays hrs. Amp Malicetip Poolspite Maliceshow raquo gv.
Cellfire Hundreds Freebies in Victorias Secret Gift Card? Usedas per am share value balance or sheet divided is number or?
Multimedia Memory gb. Shotgun g greasegun mp ppsh.
Own decrease of more paid them of was received originally out.
Multimedia Memory gb. May or left over owners after of debts paid?
Cabaret Candombe Carnatic Chimurenga Choro Christian. Gas or Toppik Hair Building Airline is.
Xpatches Downloads Extension.
Multimedia Memory gb. June newsabout ushome Hunting jobs Looking Welcome is Blogjob.
Share in value balance sheet divided number shares at date. Em in Poker Chip of set.
Eligible barring regional in local post resume in. Views in job Seekers. Including provided Content!  Begging in mercyzuma quick hand of.
Industries a trade am lower. Views in job Seekers.  Loss Trial of Texas Hold em Poker Chip.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20061106/4d86172b/attachment.html>

From eydntzopip at parenting.com  Sun Nov  5 23:34:45 2006
From: eydntzopip at parenting.com (Engineer Peter)
Date: Mon, 6 Nov 2006 12:34:45 +0800
Subject: love ones.
Message-ID: <000d01c7015c$e2616080$313da4dc@4a5d5db1e75a411>


Signup am Today am Summaries Sites.
Next a year Board found themselves a.
Chopper bicyclehow sew circuit knee Brewee fc a.
Pardoned because didnt ram Jethmalani defend. Indies experience or mattered.
Resonates people age beset a scandal war is. Hang Santosh was faulty he pardoned of because didnt ram. Storozuka Allena rob Farrella Carettaa Doug.
Archives of Editorial Letters am Editor Sunday in Deep Focus? Page play many is like Todays Crossword ways or bring am?
Sexual Immorality Exit Polls win of Night Games Predicted Problems? Racers causes teams join forces Detective Jeffries rookie am murder.
Wonderful conference of over it time start planning in next in year. Entry Blog Youll find date happenings Maker.
Playlist cb in Russell Crowe Temper a.
Wonderful conference of over it time start planning in next in year. To families interested in the care.
End or summer kids will soon be. Us trying do in this in without them.
Nanak Jayanti Champions Trophy victors Bagchi am.
Menopause Womens a Heart Disease see Nelson Cares. Clinics sign contract review Denver want let.
Love ones size of pin. Next a year Board found themselves a.
Craft Robert Redford. Bcs Sometimes of past secrets Cold Case of Learn. Friends love or ones size pin rdquo Only Shipping in Included or.  Night of Games Predicted Problems gop Wins!
Team a topdown in David Fortier Engineer a Peter Gormley general Manager! Bcs Sometimes of past secrets Cold Case of Learn.  Nov Updated at hrs ist Searchin India Indiatimes!
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20061106/10df1e6d/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: St.gif
Type: image/gif
Size: 7843 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20061106/10df1e6d/attachment.gif>
-------------- next part --------------
-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
-------------- next part --------------
_______________________________________________
Numpy-discussion mailing list
Numpy-discussion at lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/numpy-discussion

From Mailer-Daemon at lists.sourceforge.net  Mon Nov  6 04:06:29 2006
From: Mailer-Daemon at lists.sourceforge.net (Mail Delivery System)
Date: Mon, 06 Nov 2006 01:06:29 -0800
Subject: Warning: message 1Ggbsf-00056T-Ol delayed 24
	hours
Message-ID: <E1Gh0R3-0004Up-Or@externalmx-1.sourceforge.net>

This message was created automatically by mail delivery software.
A message that you sent has not yet been delivered to one or more of its
recipients after more than 24 hours on the queue on externalmx-1.sourceforge.net.

The message identifier is:     1Ggbsf-00056T-Ol
The subject of the message is: faces and their former position.  To be, sure.  But question
The date of the message is:    Sun, 05 Nov 2006 16:01:50 +0900

The address to which the message has not yet been delivered is:

  numpy-discussion at lists.sourceforge.net
    Delay reason: SMTP error from remote mailer after RCPT TO:<numpy-discussion at lists.sourceforge.net>:
    host mail.sourceforge.net [66.35.250.206]: 451-Could not complete sender verify callout
    451-Could not complete sender verify callout for
    451-<numpy-discussion at lists.sourceforge.net>.
    451-The mail server(s) for the domain may be temporarily unreachable, or
    451-they may be permanently unreachable from this server. In the latter case,
    451-you need to change the address or create an MX record for its domain
    451-if it is 

No action is required on your part. Delivery attempts will continue for
some time, and this warning may be repeated at intervals if the message
remains undelivered. Eventually the mail delivery software will give up,
and when that happens, the message will be returned to you.

-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From Chris.Barker at noaa.gov  Mon Nov  6 12:37:57 2006
From: Chris.Barker at noaa.gov (Christopher Barker)
Date: Mon, 06 Nov 2006 09:37:57 -0800
Subject: problems installing NumPy on OSX
In-Reply-To: <eik8eo$f8l$1@sea.gmane.org>
References: <20061104222219.5v1dj889q8cgsg8c@webmail.ntnu.no>
	<eij4o3$7l2$2@sea.gmane.org>
	<20061105091727.ouibxoltwk40cksg@webmail.ntnu.no>
	<eik8eo$f8l$1@sea.gmane.org>
Message-ID: <454F72F5.2020009@noaa.gov>


I think it's time to get MacPython2.5 and a set up packages on PythonMac 
org, and build a numpy for it (and SciPy and Matplotlib too...)

Bob, can you set up the page, and then we can start populating it?

-Chris


-- 
Christopher Barker, Ph.D.
Oceanographer
                                     		
NOAA/OR&R/HAZMAT         (206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115       (206) 526-6317   main reception

Chris.Barker at noaa.gov

-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From Chris.Barker at noaa.gov  Mon Nov  6 12:41:18 2006
From: Chris.Barker at noaa.gov (Christopher Barker)
Date: Mon, 06 Nov 2006 09:41:18 -0800
Subject: problems installing NumPy on OSX
In-Reply-To: <20061105193259.lbxo4gkzk44wsw4c@webmail.ntnu.no>
References: <20061104222219.5v1dj889q8cgsg8c@webmail.ntnu.no>
	<eij4o3$7l2$2@sea.gmane.org>
	<20061105091727.ouibxoltwk40cksg@webmail.ntnu.no>
	<CA13C03A-6B1C-483B-B9FF-66965D9E8DAF@arachnedesign.net>
	<20061105193259.lbxo4gkzk44wsw4c@webmail.ntnu.no>
Message-ID: <454F73BE.30503@noaa.gov>

arildna at stud.ntnu.no wrote:
> I'm sorry, I was a tad too quick typing there. I meant to say "And do  
> I even need to [install Xcode] to run numpy?"

You need Xcode to build numpy (or anything else). If you can find a 
binary, then you should be able to just run that. One of us should get a 
binary for 2.5 out there soon.

by the way, if you can run 2.4 instead, you'll find a lot of binaries 
for various packages here:

http://www.pythonmac.org/packages/py24-fat/index.html

-Chris

-- 
Christopher Barker, Ph.D.
Oceanographer
                                     		
NOAA/OR&R/HAZMAT         (206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115       (206) 526-6317   main reception

Chris.Barker at noaa.gov

-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From bob at redivi.com  Mon Nov  6 12:42:31 2006
From: bob at redivi.com (Bob Ippolito)
Date: Mon, 6 Nov 2006 09:42:31 -0800
Subject: problems installing NumPy on OSX
In-Reply-To: <454F72F5.2020009@noaa.gov>
References: <20061104222219.5v1dj889q8cgsg8c@webmail.ntnu.no>
	<eij4o3$7l2$2@sea.gmane.org>
	<20061105091727.ouibxoltwk40cksg@webmail.ntnu.no>
	<eik8eo$f8l$1@sea.gmane.org> <454F72F5.2020009@noaa.gov>
Message-ID: <6a36e7290611060942x4b3bb30am56aaec599d14d090@mail.gmail.com>

On 11/6/06, Christopher Barker <Chris.Barker at noaa.gov> wrote:
>
> I think it's time to get MacPython2.5 and a set up packages on PythonMac
> org, and build a numpy for it (and SciPy and Matplotlib too...)
>
> Bob, can you set up the page, and then we can start populating it?

I'll put it up when there's something to put there. Doesn't make sense
to have an empty page.

-bob

-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From aisaac at american.edu  Mon Nov  6 15:09:12 2006
From: aisaac at american.edu (Alan Isaac)
Date: Mon, 06 Nov 2006 15:09:12 -0500
Subject: compile numpy on Mac
Message-ID: <454F9668.5030007@american.edu>

Experience of a brand new Mac user.
Barebones numpy install on Python 2.5 on a MacBook.

1. Where is my shell?

By default, bash is 'Terminal.app' in you applications folder.

2. Where is my editor?

You have many choices, but you can always start 'vim' from your bash shell.
Howevever, I just used the default configuration,
so no editor was needed. (Comment: I do not know if
this was a good decision, but I ended up with a working numpy.)

3. Where is my compiler?

The default OSX install does not include a compiler.
Go to http://developer.apple.com/tools/xcode and download the most 
recent Xcode.
(This is a very large download.)
Follow the easy installation directions in the accompanying 'readme' file.
(You must have administrative privileges.)

You will have to sign up for a free ADC account.

4. How to compile?

 From your bash shell, change to your build directory.
(I.e., wherever you unzipped your numpy tarball to.)
Do the usual:
python setup.py install
Compilation and installation takes a few minutes.
Then exit your shell.
You should be ready to go.


-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From cjw at sympatico.ca  Mon Nov  6 15:10:20 2006
From: cjw at sympatico.ca (Colin J. Williams)
Date: Mon, 06 Nov 2006 15:10:20 -0500
Subject: Style - was Re: numpy.repeat TypeError:
 array cannot be safely cast to required type
In-Reply-To: <454CF6AF.10205@ieee.org>
References: <1162581054.217811.174140@k70g2000cwa.googlegroups.com>	<454B9BDD.2060101@ieee.org>	<454C87CF.7030500@sympatico.ca>
	<454CF6AF.10205@ieee.org>
Message-ID: <454F96AC.5020003@sympatico.ca>

An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20061106/fbcf9f68/attachment.html>
-------------- next part --------------
-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
-------------- next part --------------
_______________________________________________
Numpy-discussion mailing list
Numpy-discussion at lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/numpy-discussion

From jdhunter at ace.bsd.uchicago.edu  Mon Nov  6 15:09:32 2006
From: jdhunter at ace.bsd.uchicago.edu (John Hunter)
Date: Mon, 06 Nov 2006 14:09:32 -0600
Subject: ctypes warning
Message-ID: <87lkmofibn.fsf@peds-pc311.bsd.uchicago.edu>


A simple import of numpy with the latest svn triggers a ctypes warning

In [1]: import numpy
/usr/lib/python2.4/site-packages/numpy/ctypeslib.py:12: UserWarning:
All features of ctypes interface may not work with ctypes < 1.0.1
  warnings.warn("All features of ctypes interface may not work with "
 

This is a bit of an annoyance.  Even if the warning level is
configurable, I think the bulk of matplotlib users, who get the
warning when using numpy through mpl but make no use of ctypes, will
be confused by this.  Is it right and good that this message gets
triggered by default for any import of numpy, even those not using the
ctypes features?

JDH

-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From stefan at sun.ac.za  Mon Nov  6 16:23:17 2006
From: stefan at sun.ac.za (Stefan van der Walt)
Date: Mon, 6 Nov 2006 23:23:17 +0200
Subject: ctypes warning
In-Reply-To: <87lkmofibn.fsf@peds-pc311.bsd.uchicago.edu>
References: <87lkmofibn.fsf@peds-pc311.bsd.uchicago.edu>
Message-ID: <20061106212317.GH3380@mentat.za.net>

On Mon, Nov 06, 2006 at 02:09:32PM -0600, John Hunter wrote:
> 
> A simple import of numpy with the latest svn triggers a ctypes warning
> 
> In [1]: import numpy
> /usr/lib/python2.4/site-packages/numpy/ctypeslib.py:12: UserWarning:
> All features of ctypes interface may not work with ctypes < 1.0.1
>   warnings.warn("All features of ctypes interface may not work with "
>  
> 
> This is a bit of an annoyance.  Even if the warning level is
> configurable, I think the bulk of matplotlib users, who get the
> warning when using numpy through mpl but make no use of ctypes, will
> be confused by this.  Is it right and good that this message gets
> triggered by default for any import of numpy, even those not using the
> ctypes features?

I moved the warning to inside ctypeslib.load_library.  I think most
people using ctypes in numpy come via that route.  Soon, we can remove
the warning, but for now I think it should remain -- the bugs
resulting from using older versions of ctypes are fairly hard to track
down.

Regards
St?fan

-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From charlesr.harris at gmail.com  Mon Nov  6 16:30:30 2006
From: charlesr.harris at gmail.com (Charles R Harris)
Date: Mon, 6 Nov 2006 14:30:30 -0700
Subject: Style - was Re: numpy.repeat TypeError:
	array cannot be safely cast to required type
In-Reply-To: <454F96AC.5020003@sympatico.ca>
References: <1162581054.217811.174140@k70g2000cwa.googlegroups.com>
	<454B9BDD.2060101@ieee.org> <454C87CF.7030500@sympatico.ca>
	<454CF6AF.10205@ieee.org> <454F96AC.5020003@sympatico.ca>
Message-ID: <e06186140611061330v41f3ce65o3d594f2a565bbd5e@mail.gmail.com>

On 11/6/06, Colin J. Williams <cjw at sympatico.ca> wrote:
<snip>

Nobody has proposed using English yet - zeroes.
>

It seemed a bridge too far.

Chuck
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20061106/b3ea8e49/attachment.html>
-------------- next part --------------
-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
-------------- next part --------------
_______________________________________________
Numpy-discussion mailing list
Numpy-discussion at lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/numpy-discussion

From stefan at sun.ac.za  Mon Nov  6 16:40:45 2006
From: stefan at sun.ac.za (Stefan van der Walt)
Date: Mon, 6 Nov 2006 23:40:45 +0200
Subject: Style - was Re: numpy.repeat TypeError:
	array cannot be safely cast to required type
In-Reply-To: <454F96AC.5020003@sympatico.ca>
References: <1162581054.217811.174140@k70g2000cwa.googlegroups.com>
	<454B9BDD.2060101@ieee.org> <454C87CF.7030500@sympatico.ca>
	<454CF6AF.10205@ieee.org> <454F96AC.5020003@sympatico.ca>
Message-ID: <20061106214045.GI3380@mentat.za.net>

On Mon, Nov 06, 2006 at 03:10:20PM -0500, Colin J. Williams wrote:
> Many thanks.  In general, there is sense in the Python dictum about having one
> way to do things.  Although, in this case [length] vs length for one dimension
> doesn't
> exercise me greatly.  I would be a bit more concerned about synonyms.
> 
> Nobody has proposed using English yet - zeroes.

If Cambridge Advanced Learner's is happy with zeros, then so we should
be:

http://dictionary.cambridge.org/define.asp?key=92164&dict=CALD

zero   Show phonetics
noun plural zeros or zeroes
1 [C or U] (the number) 0; nothing:

I think the American-english speakers on the list outnumber the rest
of us, so we'd better be careful :)

Cheers
St?fan

-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From oliphant at ee.byu.edu  Mon Nov  6 17:14:16 2006
From: oliphant at ee.byu.edu (Travis Oliphant)
Date: Mon, 06 Nov 2006 15:14:16 -0700
Subject: Scholarly contribution of NumPy : request for
	review
Message-ID: <454FB3B8.4030108@ee.byu.edu>

I apologize for this bit of spam.

If there is anyone on this list (preferrably of Associate Professor or 
similar rank) who can provide an honest assessment of the scholarly 
contribution of NumPy and/or SciPy (and has not already done so)? 

If you can do this and don't mind me quoting you to a tenure-review 
committee, could you please email me your assessment by Friday morning 
(Nov. 10).   Please include information about your position in the 
mail.  I welcome all assessments (both positive and negative). 

Thank you very much.


Travis Oliphant
Assistant Professor
Brigham Young University
oliphant at ee.byu.edu
801-422-3108






-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From laidler at stsci.edu  Mon Nov  6 17:59:54 2006
From: laidler at stsci.edu (Victoria G. Laidler)
Date: Mon, 06 Nov 2006 17:59:54 -0500
Subject: Style - was Re: numpy.repeat
 TypeError:	array cannot be safely cast to required type
In-Reply-To: <20061106214045.GI3380@mentat.za.net>
References: <1162581054.217811.174140@k70g2000cwa.googlegroups.com>	<454B9BDD.2060101@ieee.org>
	<454C87CF.7030500@sympatico.ca>	<454CF6AF.10205@ieee.org>
	<454F96AC.5020003@sympatico.ca>
	<20061106214045.GI3380@mentat.za.net>
Message-ID: <454FBE6A.4000704@stsci.edu>

Stefan van der Walt wrote:
> On Mon, Nov 06, 2006 at 03:10:20PM -0500, Colin J. Williams wrote:
>   
>> Many thanks.  In general, there is sense in the Python dictum about having one
>> way to do things.  Although, in this case [length] vs length for one dimension
>> doesn't
>> exercise me greatly.  I would be a bit more concerned about synonyms.
>>
>> Nobody has proposed using English yet - zeroes.
>>     
>
> If Cambridge Advanced Learner's is happy with zeros, then so we should
> be:
>
> http://dictionary.cambridge.org/define.asp?key=92164&dict=CALD
>
> zero   Show phonetics
> noun plural zeros or zeroes
> 1 [C or U] (the number) 0; nothing:
>
> I think the American-english speakers on the list outnumber the rest
> of us, so we'd better be careful :)
>   
You know, I spell this wrong at least every other time I use it.
But of course, if we changed it to "zeroes", I'd -still- spell it wrong 
every other time I used it.

Are synonyms really *always* evil????

wistfully,
Vicki Laidler

-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From fperez.net at gmail.com  Mon Nov  6 18:38:54 2006
From: fperez.net at gmail.com (Fernando Perez)
Date: Mon, 6 Nov 2006 16:38:54 -0700
Subject: Strange and hard to reproduce crash
In-Reply-To: <453D4AD1.5060508@ieee.org>
References: <db6b5ecc0610231439o1217af98j11dcb45e2ae5e505@mail.gmail.com>
	<453D3DD7.50506@ieee.org>
	<db6b5ecc0610231544g7413491du9e8b0bd6c8c20e6f@mail.gmail.com>
	<453D4AD1.5060508@ieee.org>
Message-ID: <db6b5ecc0611061538q46ba6efoc82aacfd73f10085@mail.gmail.com>

On 10/23/06, Travis Oliphant <oliphant.travis at ieee.org> wrote:

> I've placed them in SVN (r3384):
>
> arraydescr_dealloc needs to do something like.
>
> if (self->fields == Py_None) {
>     print something
>     incref(self)
>     return;
> }

Travis, I know you're busy right now, so this message is just so that
the archives have this info, for whenever you revisit the problem.

A long run of our code is now producing the following output:

*** Reference count error detected:
an attempt was made to deallocate 12 (d) ***
*** Reference count error detected:
an attempt was made to deallocate 12 (d) ***

etc.

Thanks to your changes it does not crash anymore, so it's not a big
deal for us.  Whenever you want further details, I can try to collect
them.

Regards,

f

-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From dlaney at llnl.gov  Mon Nov  6 20:57:51 2006
From: dlaney at llnl.gov (dlaney at llnl.gov)
Date: Mon, 6 Nov 2006 17:57:51 -0800 (PST)
Subject: VisIt Usage at LLNL
Message-ID: <Pine.LNX.4.64.0611061748360.32554@tux219.llnl.gov>

Paul Dubois asked me to post a short summary about how VisIt is used
at LLNL.

First, I am not a VisIt developer, but I have worked with it and those
who use it in day to day work.  I have also used it to generate high
resolution movies of medium to extremely large (~ 4 TeraByte) data sets.

VisIt is based on VTK (Visualization Tool Kit, check their webpage via
google search).  However, several features have been re-implemented for
performance reasons or because they did (or do not) exist in VTK.

VisIt is open source, but at present time does not have a true open source
development model (no sourceforge home page for example).  This may change in
the future, but I recommend contacting the VisIt team directly to find out
about official plans in this direction.

>From my perspective, VisIt is used in three ways at LLNL:
   1) Debugging    (GUI-based, but could use the Python interface)
   2) Exploration  (GUI-based)
   3) Movies       (Usually heavier use of the python interface)

Debugging:

    VisIt contains many operators that can be used to find bad cells and and
    display information about them.  A typical debugging session might start
    with a bad zonal quantity.  The user first creates plots of fields of
    interest using pseudocolor plots for example.  Then the user applies an
    OnionPeel operator and enters the domain and zone number from the error
    message generated by the code.  The user can then vary the number of
    additional layers of cells displayed around the bad cell.
    Finally, this small set of cells can be picked or queried, and investigated
    at different time steps.

    VisIt can also plot various quantities over time.  The expression engine
    in VisIt can be used to compute many derived quantities.

    VisIt also handles cells containing multiple materials in a robust way.

    There is interest in having physics codes dump Python
    scripts for VisIt that would automatically visualize errors in
    simulations.

Exploration:

    VisIt supports a large set of expressions, including the expected
    arithmetic operations along with vector operations (cross, dot, etc.),
    conditional operations (if-then-else), and many other useful functions.
    This allows users to create a large set of derived fields and then
    interact with them in VisIt like any other field.  I have seen users
    generate quite interesting plots by carefully creating derived quantities
    from the raw simulation dump files.

    VisIt has a nice set of functionality for creating 2-D plots.

Movies:

    The way I typically approach the problem of generating a movie of a given
    visualization with a static view-point is as follows.

    1) First I set up the visualization with the domain scientist(s).  This
       process is iterative by nature and is entirely GUI based.  VisIt can save
       'session' files that record everything about a given session, allowing me
       to create multiple visualizations and re-use them later.

    2) Once everything is set up, VisIt can generate the movie from within
       the GUI, or the session file can be used in a python script to drive
       VisIt.  I usually use the Python interface because that gives direct
       control over camera position and other features.  For large movies,
       the frames must be generated on a cluster and the batch system is
       necessary.  Python can be used to build scripts that play nice with
       the batch-system at LLNL.

    It is also possible to build an entire movie using Python by specifying
    exactly which operations to apply.  The end-result is a self-contained
    script for generating frames.  However, this is a labor intensive
    approach and is only needed for movies with complex camera movements or
    other effects.  Even for such movies, VisIt has a key-framing system
    that can handle some common cases for movies.

For more information I suggest emailing the VisIt team directly at:
visit-help at llnl.gov

Regards

-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From nzlofmyj at pacificsportsllc.com  Mon Nov  6 21:18:02 2006
From: nzlofmyj at pacificsportsllc.com (near)
Date: Tue, 7 Nov 2006 11:18:02 +0900
Subject: EntryCan Monkey Posted //Comments
Message-ID: <000701c70212$f3cb43c0$66b1bad3@a>


Processes strataud mustreads a Franz Woyzeck.
Trends is comics a juvenile pranks practical jokes you. Many find idiotic everyone else waste is.
When a insult of reveals a inner.
Mission a Calendar Stickers Concerts!
Sharon leaves icu or condition improves Israeli lawmaker sues in Iran is. Inc Maxim a Dennis Publishing Esquire Hearst in Blender.
Ship they is qualify of rest a order mailing address. Bb upi discuss of demands stumps.
Primary audience or age group am many of find of idiotic of everyone else. Average one am is reviewswhy in pay full!
Fhm Total Today Customers a bought is also Emap! Md its strategy!
Florida Arkansas Texas or amahold cut a Giants prices sell.
Sgt Shaft am Culture etc World! Atop Taishan Huangshan Picks here less buggy. Japan Hiroko clamps is James officers. Pricing Martin Baxter collected or actual is!
Florida Arkansas Texas or amahold cut a Giants prices sell.
Tn sister in magazine several ways Both feature silly. Company owns Giant said of today.
Say months ugly battles tomorrows will.
Involves lot analysis Having practiced downright easy breezed rounds is.
Can shipped within Review.
List analytic of before reading los Angeles targeted. Trends is comics a juvenile pranks practical jokes you.
Bytimothy sellers in Offers Unknown Binding Version Finance is Voitle.
Quant Physics Emanuel Derman Fixed Income Markets is.
Journals finance degrees diploma dr taught university or level. Quant Physics Emanuel Derman Fixed Income Markets is.
Simple a cases appear complex teaches examining.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20061107/cdb54895/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: Join.gif
Type: image/gif
Size: 8379 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20061107/cdb54895/attachment.gif>
-------------- next part --------------
-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
-------------- next part --------------
_______________________________________________
Numpy-discussion mailing list
Numpy-discussion at lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/numpy-discussion

From david at ar.media.kyoto-u.ac.jp  Mon Nov  6 21:23:29 2006
From: david at ar.media.kyoto-u.ac.jp (David Cournapeau)
Date: Tue, 07 Nov 2006 11:23:29 +0900
Subject: Passing numpy arrays to matlab
Message-ID: <454FEE21.7040205@ar.media.kyoto-u.ac.jp>

Hi,

    I am trying to find a nice way to communicate between matlab and 
python. I am aware of pymat, which does that, but the code is 
deprecated, and I thing basing the code on ctypes would lead to much 
more robust code.

    http://claymore.engineer.gvsu.edu/%7Esteriana/Software/pymat.html

I have a really simple prototype which can send and get back data from 
matlab, but I was wondering if it would be possible to use a scheme 
similar to ctypes instead of having to convert it by hand. Let me 
present the way communication with matlab is done:

    - open a matlab session, which on unix launch a matlab process, and 
set up pipes between the calling process and matlab process for 
communication
    - To send data from the calling process to matlab, you first have to 
create a mxArray, which is the basic matlab handler of a matlab array, 
and populating it. Using mxArray is very ackward : you cannot create 
mxArray from existing data, you have to copy data to them, etc... (it is 
one of the reason I started looking in python in the first place: 
interfacing matlab with foreign code is really not a nice experience, 
and the API is not rich enough to do many interesting things; I am 
actually amazed how poor this part of matlab is, a product which is now 
20 years old).

    I was wondering if there was a way to extend a numpy array so that I 
could send directly numpy arrays to matlab C functions expecting a , 
with ctypes doing the hard work. For example, now, to send data to 
matlab, I do:

    session   = MatlabEngine()
    # data is a numpy array, 'dataname' a string with its name inside 
matlab interpreter
    session.put(data, 'dataname')

The put function has do to a lot for work:
    - first, getting metadata from data (dimensions, real or complex, 
etc...)
    - then creating a mxArray with the same metadata
    - then populating the mxarray by copying the data from the numpy array.

And of course, taking care that all mxarrays are detroyed, otherwise, 
memory leak... That's why I was thinking about something a bit smarter: 
creating a mxarray class which implements the numpy interface. I could 
create a mxarray from a numpy array easily, send a mxarray directly to C 
function with ctypes, etc... Is this doable ? Would this take care of 
correct destruction of mxarrays ? I really don't know much about the 
internals of numpy arrays, so I don't really know how to start,

    cheers,

    David

-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From strawman at astraw.com  Mon Nov  6 23:56:18 2006
From: strawman at astraw.com (Andrew Straw)
Date: Mon, 06 Nov 2006 20:56:18 -0800
Subject: ctypes warning
In-Reply-To: <20061106212317.GH3380@mentat.za.net>
References: <87lkmofibn.fsf@peds-pc311.bsd.uchicago.edu>
	<20061106212317.GH3380@mentat.za.net>
Message-ID: <455011F2.7010204@astraw.com>

Stefan van der Walt wrote:
> On Mon, Nov 06, 2006 at 02:09:32PM -0600, John Hunter wrote:
>   
>> A simple import of numpy with the latest svn triggers a ctypes warning
>>
>> In [1]: import numpy
>> /usr/lib/python2.4/site-packages/numpy/ctypeslib.py:12: UserWarning:
>> All features of ctypes interface may not work with ctypes < 1.0.1
>>   warnings.warn("All features of ctypes interface may not work with "
>>  
>>
>> This is a bit of an annoyance.  Even if the warning level is
>> configurable, I think the bulk of matplotlib users, who get the
>> warning when using numpy through mpl but make no use of ctypes, will
>> be confused by this.  Is it right and good that this message gets
>> triggered by default for any import of numpy, even those not using the
>> ctypes features?
>>     
>
> I moved the warning to inside ctypeslib.load_library.  I think most
> people using ctypes in numpy come via that route. 
I don't come in via that route. How hard is it to trigger on a getattr()
for the .ctypes attribute?

-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From strawman at astraw.com  Tue Nov  7 00:03:18 2006
From: strawman at astraw.com (Andrew Straw)
Date: Mon, 06 Nov 2006 21:03:18 -0800
Subject: Passing numpy arrays to matlab
In-Reply-To: <454FEE21.7040205@ar.media.kyoto-u.ac.jp>
References: <454FEE21.7040205@ar.media.kyoto-u.ac.jp>
Message-ID: <45501396.3070908@astraw.com>

David Cournapeau wrote:
>     - To send data from the calling process to matlab, you first have to 
> create a mxArray, which is the basic matlab handler of a matlab array, 
> and populating it. Using mxArray is very ackward : you cannot create 
> mxArray from existing data, you have to copy data to them, etc... 
My understanding, never having done it, but from reading the docs, is
that you can create a "hybrid array" where you manage the memory. Thus,
you can create an mxArray from existing data. However, the docs
basically say that this is too hard for most mortals (and they may well
be right -- too painful for me, anyway)!

-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From josh.p.marshall at gmail.com  Mon Nov  6 23:59:25 2006
From: josh.p.marshall at gmail.com (Josh Marshall)
Date: Tue, 7 Nov 2006 15:59:25 +1100
Subject: Passing numpy arrays to matlab
In-Reply-To: <mailman.119602.1162866273.22397.numpy-discussion@lists.sourceforge.net>
References: <mailman.119602.1162866273.22397.numpy-discussion@lists.sourceforge.net>
Message-ID: <E50CA0F6-3ED5-4ED0-92BA-B55991001504@gmail.com>

Hi David,

Did you have a look at mlabwrap? It's quite hard to find on the net,  
which is a shame, since it is a much more up to date version,  
enhancing pymat with the things that you are trying to do. It allows  
passing arrays and getting arrays back.

http://mlabwrap.sourceforge.net/

However, I do agree with you that it would be good to have a bridge  
based on ctypes. I can't really help since I long ago moved all of my  
code from Matlab to Numpy and Matplotlib.

Cheers,
Josh M

David Cournapeau <david at ar.media.kyoto-u.ac.jp> wrote:
>
> Hi,
>
>     I am trying to find a nice way to communicate between matlab and
> python. I am aware of pymat, which does that, but the code is
> deprecated, and I thing basing the code on ctypes would lead to much
> more robust code.
>
>     http://claymore.engineer.gvsu.edu/%7Esteriana/Software/pymat.html
>
> I have a really simple prototype which can send and get back data from
> matlab, but I was wondering if it would be possible to use a scheme
> similar to ctypes instead of having to convert it by hand. Let me
> present the way communication with matlab is done:
>
>     - open a matlab session, which on unix launch a matlab process,  
> and
> set up pipes between the calling process and matlab process for
> communication
>     - To send data from the calling process to matlab, you first  
> have to
> create a mxArray, which is the basic matlab handler of a matlab array,
> and populating it. Using mxArray is very ackward : you cannot create
> mxArray from existing data, you have to copy data to them, etc...  
> (it is
> one of the reason I started looking in python in the first place:
> interfacing matlab with foreign code is really not a nice experience,
> and the API is not rich enough to do many interesting things; I am
> actually amazed how poor this part of matlab is, a product which is  
> now
> 20 years old).
>
>     I was wondering if there was a way to extend a numpy array so  
> that I
> could send directly numpy arrays to matlab C functions expecting a ,
> with ctypes doing the hard work. For example, now, to send data to
> matlab, I do:
>
>     session   = MatlabEngine()
>     # data is a numpy array, 'dataname' a string with its name inside
> matlab interpreter
>     session.put(data, 'dataname')
>
> The put function has do to a lot for work:
>     - first, getting metadata from data (dimensions, real or complex,
> etc...)
>     - then creating a mxArray with the same metadata
>     - then populating the mxarray by copying the data from the  
> numpy array.
>
> And of course, taking care that all mxarrays are detroyed, otherwise,
> memory leak... That's why I was thinking about something a bit  
> smarter:
> creating a mxarray class which implements the numpy interface. I could
> create a mxarray from a numpy array easily, send a mxarray directly  
> to C
> function with ctypes, etc... Is this doable ? Would this take care of
> correct destruction of mxarrays ? I really don't know much about the
> internals of numpy arrays, so I don't really know how to start,
>
>     cheers,
>
>     David
>
>


-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From david at ar.media.kyoto-u.ac.jp  Mon Nov  6 23:59:43 2006
From: david at ar.media.kyoto-u.ac.jp (David Cournapeau)
Date: Tue, 07 Nov 2006 13:59:43 +0900
Subject: Passing numpy arrays to matlab
In-Reply-To: <45501396.3070908@astraw.com>
References: <454FEE21.7040205@ar.media.kyoto-u.ac.jp>
	<45501396.3070908@astraw.com>
Message-ID: <455012BF.1030105@ar.media.kyoto-u.ac.jp>

Andrew Straw wrote:
> David Cournapeau wrote:
>   
>>     - To send data from the calling process to matlab, you first have to 
>> create a mxArray, which is the basic matlab handler of a matlab array, 
>> and populating it. Using mxArray is very ackward : you cannot create 
>> mxArray from existing data, you have to copy data to them, etc... 
>>     
> My understanding, never having done it, but from reading the docs, is
> that you can create a "hybrid array" where you manage the memory. Thus,
> you can create an mxArray from existing data. However, the docs
> basically say that this is too hard for most mortals (and they may well
> be right -- too painful for me, anyway)!
>   
Would you mind telling me where you found that information ? Because 
right now, I am wasting a lot of cycles because of memory copy in both 
directions, and it is sometimes slow enough so that it is annoying,

cheers,

David

-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From bur at carrabbas.com  Tue Nov  7 00:26:10 2006
From: bur at carrabbas.com (Fran Kellerhouse)
Date: Mon, 6 Nov 2006 21:26:10 -0800
Subject: new 874
Message-ID: <000001c7022d$3c2d63e0$c316a8c0@hbes>

Hi,

PHAjjRMACY for LESS http://www.mitujunhertshieasde.com

 

Your slightest wish is our command.


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20061106/88de1605/attachment.html>
-------------- next part --------------
-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
-------------- next part --------------
_______________________________________________
Numpy-discussion mailing list
Numpy-discussion at lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/numpy-discussion

From strawman at astraw.com  Tue Nov  7 00:49:36 2006
From: strawman at astraw.com (Andrew Straw)
Date: Mon, 06 Nov 2006 21:49:36 -0800
Subject: Passing numpy arrays to matlab
In-Reply-To: <455012BF.1030105@ar.media.kyoto-u.ac.jp>
References: <454FEE21.7040205@ar.media.kyoto-u.ac.jp>	<45501396.3070908@astraw.com>
	<455012BF.1030105@ar.media.kyoto-u.ac.jp>
Message-ID: <45501E70.8090609@astraw.com>

David Cournapeau wrote:
> Andrew Straw wrote:
>   
>> David Cournapeau wrote:
>>   
>>     
>>>     - To send data from the calling process to matlab, you first have to 
>>> create a mxArray, which is the basic matlab handler of a matlab array, 
>>> and populating it. Using mxArray is very ackward : you cannot create 
>>> mxArray from existing data, you have to copy data to them, etc... 
>>>     
>>>       
>> My understanding, never having done it, but from reading the docs, is
>> that you can create a "hybrid array" where you manage the memory. Thus,
>> you can create an mxArray from existing data. However, the docs
>> basically say that this is too hard for most mortals (and they may well
>> be right -- too painful for me, anyway)!
>>   
>>     
> Would you mind telling me where you found that information ? Because 
> right now, I am wasting a lot of cycles because of memory copy in both 
> directions, and it is sometimes slow enough so that it is annoying,
>   
I found it reading through the in-program help (the C-API section,
whatever it's called) on a Matlab installation at my university. I guess
this was Matlab 2006A. A quick Google search turns this up:

http://www.mathworks.com/access/helpdesk/help/techdoc/matlab_external/index.html?/access/helpdesk/help/techdoc/matlab_external/f25255.html

They give the following example, which seems to create a Matlab array
"pArray" with data owned by the C variable "data":

mxArray *pArray = mxCreateDoubleMatrix(0, 0, mxREAL);
double data[10];

mxSetPr(pArray, data);
mxSetM(pArray, 1);
mxSetN(pArray, 10);


-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From david at ar.media.kyoto-u.ac.jp  Tue Nov  7 00:52:56 2006
From: david at ar.media.kyoto-u.ac.jp (David Cournapeau)
Date: Tue, 07 Nov 2006 14:52:56 +0900
Subject: Passing numpy arrays to matlab
In-Reply-To: <45501E70.8090609@astraw.com>
References: <454FEE21.7040205@ar.media.kyoto-u.ac.jp>	<45501396.3070908@astraw.com>	<455012BF.1030105@ar.media.kyoto-u.ac.jp>
	<45501E70.8090609@astraw.com>
Message-ID: <45501F38.3090201@ar.media.kyoto-u.ac.jp>

Andrew Straw wrote:
> David Cournapeau wrote:
>   
>> Andrew Straw wrote:
>>   
>>     
>>> David Cournapeau wrote:
>>>   
>>>     
>>>       
>>>>     - To send data from the calling process to matlab, you first have to 
>>>> create a mxArray, which is the basic matlab handler of a matlab array, 
>>>> and populating it. Using mxArray is very ackward : you cannot create 
>>>> mxArray from existing data, you have to copy data to them, etc... 
>>>>     
>>>>       
>>>>         
>>> My understanding, never having done it, but from reading the docs, is
>>> that you can create a "hybrid array" where you manage the memory. Thus,
>>> you can create an mxArray from existing data. However, the docs
>>> basically say that this is too hard for most mortals (and they may well
>>> be right -- too painful for me, anyway)!
>>>   
>>>     
>>>       
>> Would you mind telling me where you found that information ? Because 
>> right now, I am wasting a lot of cycles because of memory copy in both 
>> directions, and it is sometimes slow enough so that it is annoying,
>>   
>>     
> I found it reading through the in-program help (the C-API section,
> whatever it's called) on a Matlab installation at my university. I guess
> this was Matlab 2006A. A quick Google search turns this up:
>
> http://www.mathworks.com/access/helpdesk/help/techdoc/matlab_external/index.html?/access/helpdesk/help/techdoc/matlab_external/f25255.html
>
> They give the following example, which seems to create a Matlab array
> "pArray" with data owned by the C variable "data":
>
> mxArray *pArray = mxCreateDoubleMatrix(0, 0, mxREAL);
> double data[10];
>
> mxSetPr(pArray, data);
> mxSetM(pArray, 1);
> mxSetN(pArray, 10);
>   
Thank you very much, I think this added documentation is pretty recent; 
I have never seen it before, and I did a lot a mex programming at some 
point... This whole mxarray nonsense reminds me why I gave up on matlab :),

cheers,

David

-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From Mailer-Daemon at lists.sourceforge.net  Tue Nov  7 04:15:27 2006
From: Mailer-Daemon at lists.sourceforge.net (Mail Delivery System)
Date: Tue, 07 Nov 2006 01:15:27 -0800
Subject: Warning: message 1Ggbsf-00056T-Ol delayed 48
	hours
Message-ID: <E1GhN3H-0004h3-MU@externalmx-1.sourceforge.net>

This message was created automatically by mail delivery software.
A message that you sent has not yet been delivered to one or more of its
recipients after more than 48 hours on the queue on externalmx-1.sourceforge.net.

The message identifier is:     1Ggbsf-00056T-Ol
The subject of the message is: faces and their former position.  To be, sure.  But question
The date of the message is:    Sun, 05 Nov 2006 16:01:50 +0900

The address to which the message has not yet been delivered is:

  numpy-discussion at lists.sourceforge.net
    Delay reason: SMTP error from remote mailer after RCPT TO:<numpy-discussion at lists.sourceforge.net>:
    host mail.sourceforge.net [66.35.250.206]: 451-Could not complete sender verify callout
    451-Could not complete sender verify callout for
    451-<numpy-discussion at lists.sourceforge.net>.
    451-The mail server(s) for the domain may be temporarily unreachable, or
    451-they may be permanently unreachable from this server. In the latter case,
    451-you need to change the address or create an MX record for its domain
    451-if it is 

No action is required on your part. Delivery attempts will continue for
some time, and this warning may be repeated at intervals if the message
remains undelivered. Eventually the mail delivery software will give up,
and when that happens, the message will be returned to you.

-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From olivetti at itc.it  Tue Nov  7 08:29:13 2006
From: olivetti at itc.it (Emanuele Olivetti)
Date: Tue, 07 Nov 2006 14:29:13 +0100
Subject: numpy on wince
Message-ID: <45508A29.6060302@itc.it>

Hi all,
do you know if numpy/Numeric is available on WinCE ?
I'm thinking about the use of a numpy/Numeric app on a palmtop
and I'm not sure it will be deployed on linux only devices.

Thanks in advance,

Emanuele

P.S.: I know that Python is available on many many platforms, but
I not so confident about many external libraries.

-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From gurjptwydsh at otisinstruments.com  Tue Nov  7 09:10:17 2006
From: gurjptwydsh at otisinstruments.com (Miracle Vol.)
Date: Tue, 7 Nov 2006 22:10:17 +0800
Subject: revealed in
Message-ID: <001101c70276$73a55500$c545eadd@e29e69d898f14ac>


Harper lee Charles.
Blogin Creasein Dialon Ranttable in Weighing Incorpus or Christi is Hooksortiz a Becoming. Shelly Lewis East reg District am West Coast Windows.
Heb of Plus signing before a continuing Parkdale.
Costa than developed nations holds is its am certainty god or created. Nov Bonanza Street Locust a streets Walnut Creek. News Archives web for Jobs Cars am.
Say they checks but touched hug. Nineteen feature of films have a!
Project Cannon crew is kick ass?
Everything better Thats secret revealed. Reminder horrible dayits gonea brilliant example Aberdeen Dakota Call.
As guide browsers though liked specific. Mccain John Mayer Mikes band.
Windows is Stewart Heating.
Bought guitar learned in chords writing. Wilson Winter Englands bestloved writer .
Windows is Stewart Heating.
Last chance reconnect restore dead Current Selection book. Accurate Game Browser Updated of constantly popular servers listed browsing.
Packed someone ive a known is talked soldiers giving is lives privileges?
Values Moushumi had whole different set issues deal after. Inked Warner coauthor?
Dr jay a Goble dds in. Blogin Creasein Dialon Ranttable in Weighing Incorpus or Christi is Hooksortiz a Becoming.
Friends family their.
Passion Printpay Forward of.
Tommy Brooks is Illinois. Passion Printpay Forward of.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20061107/f17b9fa7/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: Severance.gif
Type: image/gif
Size: 8535 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20061107/f17b9fa7/attachment.gif>
-------------- next part --------------
-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
-------------- next part --------------
_______________________________________________
Numpy-discussion mailing list
Numpy-discussion at lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/numpy-discussion

From ivan.vinogradov at gmail.com  Tue Nov  7 10:37:32 2006
From: ivan.vinogradov at gmail.com (IV)
Date: Tue, 07 Nov 2006 15:37:32 -0000
Subject: help installing numpy1.0 on sgi irix
Message-ID: <1162913852.488520.204670@e3g2000cwe.googlegroups.com>

Dear All,

has anyone successfully install numpy-1.0 on irix 6.5?

The GCC is 3.3 and is the latest from freeware.sgi.com.
Building gcc 4.1.1 from source did not work for me, and I am afraid to
force it.

Latest Python 2.5 installed fine from source.

Numpy gives this error(s):
$ python setup.py install
Running from numpy source directory.
F2PY Version 2_3396
blas_opt_info:
blas_mkl_info:
  libraries mkl,vml,guide not found in /usr/local/lib
  NOT AVAILABLE

atlas_blas_threads_info:
Setting PTATLAS=ATLAS
  libraries lapack,blas not found in /usr/local/lib
  NOT AVAILABLE

atlas_blas_info:
  libraries lapack,blas not found in /usr/local/lib
  NOT AVAILABLE

/usr/people/ivan/numpy-1.0/numpy/distutils/system_info.py:1301:
UserWarning:
    Atlas (http://math-atlas.sourceforge.net/) libraries not found.
    Directories to search for the libraries can be specified in the
    numpy/distutils/site.cfg file (section [atlas]) or by setting
    the ATLAS environment variable.
  warnings.warn(AtlasNotFoundError.__doc__)
blas_info:
  libraries blas not found in /usr/local/lib
  NOT AVAILABLE

/usr/people/ivan/numpy-1.0/numpy/distutils/system_info.py:1310:
UserWarning:
    Blas (http://www.netlib.org/blas/) libraries not found.
    Directories to search for the libraries can be specified in the
    numpy/distutils/site.cfg file (section [blas]) or by setting
    the BLAS environment variable.
  warnings.warn(BlasNotFoundError.__doc__)
blas_src_info:
  NOT AVAILABLE

/usr/people/ivan/numpy-1.0/numpy/distutils/system_info.py:1313:
UserWarning:
    Blas (http://www.netlib.org/blas/) sources not found.
    Directories to search for the sources can be specified in the
    numpy/distutils/site.cfg file (section [blas_src]) or by setting
    the BLAS_SRC environment variable.
  warnings.warn(BlasSrcNotFoundError.__doc__)
  NOT AVAILABLE

lapack_opt_info:
lapack_mkl_info:
mkl_info:
  libraries mkl,vml,guide not found in /usr/local/lib
  NOT AVAILABLE

  NOT AVAILABLE

atlas_threads_info:
Setting PTATLAS=ATLAS
  libraries lapack,blas not found in /usr/local/lib
  libraries lapack_atlas not found in /usr/local/lib
numpy.distutils.system_info.atlas_threads_info
  NOT AVAILABLE

atlas_info:
  libraries lapack,blas not found in /usr/local/lib
  libraries lapack_atlas not found in /usr/local/lib
numpy.distutils.system_info.atlas_info
  NOT AVAILABLE

/usr/people/ivan/numpy-1.0/numpy/distutils/system_info.py:1210:
UserWarning:
    Atlas (http://math-atlas.sourceforge.net/) libraries not found.
    Directories to search for the libraries can be specified in the
    numpy/distutils/site.cfg file (section [atlas]) or by setting
    the ATLAS environment variable.
  warnings.warn(AtlasNotFoundError.__doc__)
lapack_info:
  libraries lapack not found in /usr/local/lib
  NOT AVAILABLE

/usr/people/ivan/numpy-1.0/numpy/distutils/system_info.py:1221:
UserWarning:
    Lapack (http://www.netlib.org/lapack/) libraries not found.
    Directories to search for the libraries can be specified in the
    numpy/distutils/site.cfg file (section [lapack]) or by setting
    the LAPACK environment variable.
  warnings.warn(LapackNotFoundError.__doc__)
lapack_src_info:
  NOT AVAILABLE

/usr/people/ivan/numpy-1.0/numpy/distutils/system_info.py:1224:
UserWarning:
    Lapack (http://www.netlib.org/lapack/) sources not found.
    Directories to search for the sources can be specified in the
    numpy/distutils/site.cfg file (section [lapack_src]) or by setting
    the LAPACK_SRC environment variable.
  warnings.warn(LapackSrcNotFoundError.__doc__)
  NOT AVAILABLE

running install
running build
running config_fc
running build_src
building py_modules sources
building extension "numpy.core.multiarray" sources
Generating build/src.irix64-6.5-2.5/numpy/core/config.h
customize MipsFCompiler
customize MipsFCompiler
customize MipsFCompiler using config
C compiler: gcc -fno-strict-aliasing -DNDEBUG -g -O3 -Wall
-Wstrict-prototypes -shared

compile options: '-I/usr/local/include/python2.5 -Inumpy/core/src
-Inumpy/core/include -I/usr/local/include/python2.5 -c'
gcc: _configtest.c
gcc _configtest.o -L/usr/local/lib -L/usr/lib -o _configtest
ld32: FATAL   12 : Expecting n32 objects: /usr/lib/libc.so is o32.
collect2: ld returned 4 exit status
ld32: FATAL   12 : Expecting n32 objects: /usr/lib/libc.so is o32.
collect2: ld returned 4 exit status
failure.
removing: _configtest.c _configtest.o
numpy/core/setup.py:50: DeprecationWarning: raising a string exception
is deprecated
  raise "ERROR: Failed to test configuration"
Traceback (most recent call last):
  File "setup.py", line 89, in <module>
    setup_package()
  File "setup.py", line 82, in setup_package
    configuration=configuration )
  File "/usr/people/ivan/numpy-1.0/numpy/distutils/core.py", line 174,
in setup
    return old_setup(**new_attr)
  File "/usr/local/lib/python2.5/distutils/core.py", line 151, in setup
    dist.run_commands()
  File "/usr/local/lib/python2.5/distutils/dist.py", line 974, in
run_commands
    self.run_command(cmd)
  File "/usr/local/lib/python2.5/distutils/dist.py", line 994, in
run_command
    cmd_obj.run()
  File "/usr/people/ivan/numpy-1.0/numpy/distutils/command/install.py",
line 16, in run
    r = old_install.run(self)
  File "/usr/local/lib/python2.5/distutils/command/install.py", line
506, in run
    self.run_command('build')
  File "/usr/local/lib/python2.5/distutils/cmd.py", line 333, in
run_command
    self.distribution.run_command(command)
  File "/usr/local/lib/python2.5/distutils/dist.py", line 994, in
run_command
    cmd_obj.run()
  File "/usr/local/lib/python2.5/distutils/command/build.py", line 112,
in run
    self.run_command(cmd_name)
  File "/usr/local/lib/python2.5/distutils/cmd.py", line 333, in
run_command
    self.distribution.run_command(command)
  File "/usr/local/lib/python2.5/distutils/dist.py", line 994, in
run_command
    cmd_obj.run()
  File
"/usr/people/ivan/numpy-1.0/numpy/distutils/command/build_src.py", line
87, in run
    self.build_sources()
  File
"/usr/people/ivan/numpy-1.0/numpy/distutils/command/build_src.py", line
106, in build_sources
    self.build_extension_sources(ext)
  File
"/usr/people/ivan/numpy-1.0/numpy/distutils/command/build_src.py", line
212, in build_extension_sources
    sources = self.generate_sources(sources, ext)
  File
"/usr/people/ivan/numpy-1.0/numpy/distutils/command/build_src.py", line
270, in generate_sources
    source = func(extension, build_dir)
  File "numpy/core/setup.py", line 50, in generate_config_h
    raise "ERROR: Failed to test configuration"
ERROR: Failed to test configuration 


Please advise :)


-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From ivilata at carabos.com  Tue Nov  7 12:19:37 2006
From: ivilata at carabos.com (Ivan Vilata i Balaguer)
Date: Tue, 7 Nov 2006 18:19:37 +0100
Subject: More on Numexpr-unsupported objects
Message-ID: <20061107171937.GI22979@tardis.terramar.selidor.net>

This is somehow related with this previous thread about raising
``TypeError`` on unsupported objects in Numexpr:
http://www.mail-archive.com/numpy-discussion at lists.sourceforge.net/msg03146.html

There is still a case where unsupported objects can get into expressions
without Numexpr noticing.  For instance:

>>> import numexpr
>>> numexpr.evaluate('[]')
Traceback (most recent call last):
  File "<stdin>", line 1, in ?
  File "numexpr/compiler.py", line 594, in evaluate
    _names_cache[expr_key] = getExprNames(ex, context)
  File "numexpr/compiler.py", line 570, in getExprNames
    ast = expressionToAST(ex)
  File "numexpr/compiler.py", line 84, in expressionToAST
    this_ast = ASTNode(ex.astType, ex.astKind, ex.value,
AttributeError: 'list' object has no attribute 'astType'

The attached patch makes the error clearer and more consistent with the
error added in the aforementioned thread:

>>> import numpy
>>> import numexpr
>>> numexpr.evaluate('[]')
Traceback (most recent call last):
  File "<stdin>", line 1, in ?
  File "numexpr/compiler.py", line 596, in evaluate
    _names_cache[expr_key] = getExprNames(ex, context)
  File "numexpr/compiler.py", line 571, in getExprNames
    ex = stringToExpression(text, {}, context)
  File "numexpr/compiler.py", line 229, in stringToExpression
    raise TypeError("unsupported expression type: %s" % type(ex))
TypeError: unsupported expression type: <type 'list'>

Though I admit it may be strange for this error to be triggered.

(I had a little mess with using ``expressions.ExpressionNode`` instead
of ``expr.ExpressionNode``...  I still don't see why the private copy of
the module is necessary.)

::

	Ivan Vilata i Balaguer   >qo<   http://www.carabos.com/
	       C?rabos Coop. V.  V  V   Enjoy Data
	                          ""
-------------- next part --------------
Index: compiler.py
===================================================================
--- compiler.py	(revisi?n: 2307)
+++ compiler.py	(copia de trabajo)
@@ -225,6 +225,8 @@
     ex = eval(c, names)
     if expressions.isConstant(ex):
         ex = expr.ConstantNode(ex, expressions.getKind(ex))
+    elif not isinstance(ex, expr.ExpressionNode):
+        raise TypeError("unsupported expression type: %s" % type(ex))
     return ex
 
 
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 309 bytes
Desc: Digital signature
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20061107/bdf54192/attachment.sig>
-------------- next part --------------
-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
-------------- next part --------------
_______________________________________________
Numpy-discussion mailing list
Numpy-discussion at lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/numpy-discussion

From Chris.Barker at noaa.gov  Tue Nov  7 13:56:56 2006
From: Chris.Barker at noaa.gov (Christopher Barker)
Date: Tue, 07 Nov 2006 10:56:56 -0800
Subject: Stacking arrays...
Message-ID: <4550D6F8.80001@noaa.gov>

HI all,

I'm trying to get the hang of this new r_ and c_ stuff.

I need to take two MxN arrays, and stack them together into one MxNx2 
arrays. I found this works:

 >>> a = N.ones((3,4))
 >>> b = N.ones((3,4)) * 2
 >>> a
array([[ 1.,  1.,  1.,  1.],
        [ 1.,  1.,  1.,  1.],
        [ 1.,  1.,  1.,  1.]])
 >>> b
array([[ 2.,  2.,  2.,  2.],
        [ 2.,  2.,  2.,  2.],
        [ 2.,  2.,  2.,  2.]])
 >>> a.shape = (3,4,1)
 >>> b.shape = (3,4,1)
 >>> c = N.c_[a,b]
 >>> c.shape
(3, 4, 2)
 >>> c[0,0]
array([ 1.,  2.])


What I'm wondering is if there is a way to do that without explicitly 
adding the extra dimension to a and b before calling c_ ?

-Chris


-- 
Christopher Barker, Ph.D.
Oceanographer
                                     		
NOAA/OR&R/HAZMAT         (206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115       (206) 526-6317   main reception

Chris.Barker at noaa.gov

-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From matthew.brett at gmail.com  Tue Nov  7 13:57:21 2006
From: matthew.brett at gmail.com (Matthew Brett)
Date: Tue, 7 Nov 2006 10:57:21 -0800
Subject: Passing numpy arrays to matlab
In-Reply-To: <45501F38.3090201@ar.media.kyoto-u.ac.jp>
References: <454FEE21.7040205@ar.media.kyoto-u.ac.jp>
	<45501396.3070908@astraw.com>
	<455012BF.1030105@ar.media.kyoto-u.ac.jp>
	<45501E70.8090609@astraw.com>
	<45501F38.3090201@ar.media.kyoto-u.ac.jp>
Message-ID: <1e2af89e0611071057h70b2f575u3943efbc142ca9c8@mail.gmail.com>

Hi,

> Thank you very much, I think this added documentation is pretty recent;
> I have never seen it before, and I did a lot a mex programming at some
> point... This whole mxarray nonsense reminds me why I gave up on matlab :),

I would be very happy to help with this.  It would be great if we
could get a standard well-maintained library of some sort towards
scipy - we (http://neuroimaging.scipy.org/) have a great deal of
matlab integration to do.

Best,

Matthew

-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From pauli.virtanen at iki.fi  Tue Nov  7 13:59:45 2006
From: pauli.virtanen at iki.fi (Pauli Virtanen)
Date: Tue, 07 Nov 2006 20:59:45 +0200
Subject: Passing numpy arrays to matlab
In-Reply-To: <454FEE21.7040205@ar.media.kyoto-u.ac.jp>
References: <454FEE21.7040205@ar.media.kyoto-u.ac.jp>
Message-ID: <1162925985.6891.6.camel@localhost.localdomain>

Hi all,

ti, 2006-11-07 kello 11:23 +0900, David Cournapeau kirjoitti:
>     I am trying to find a nice way to communicate between matlab and 
> python. I am aware of pymat, which does that, but the code is 
> deprecated, and I thing basing the code on ctypes would lead to much 
> more robust code.
> 
>     http://claymore.engineer.gvsu.edu/%7Esteriana/Software/pymat.html
> 
> I have a really simple prototype which can send and get back data from 
> matlab, but I was wondering if it would be possible to use a scheme 
> similar to ctypes instead of having to convert it by hand.

A while ago I wrote a mex extension to embed the Python interpreter
inside Matlab:

	http://www.iki.fi/pav/pythoncall

I guess it's something like an inverse of pymat :)

But I guess this is not really what you are looking for, since at
present it just does a memory copy when passing arrays between Matlab
and Python. Though, shared arrays might be just possible to implement if
memory management is done carefully.

BR,

	Pauli Virtanen

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 189 bytes
Desc: Digitaalisesti allekirjoitettu viestin osa
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20061107/2d09360e/attachment.sig>
-------------- next part --------------
-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
-------------- next part --------------
_______________________________________________
Numpy-discussion mailing list
Numpy-discussion at lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/numpy-discussion

From Chris.Barker at noaa.gov  Tue Nov  7 19:53:47 2006
From: Chris.Barker at noaa.gov (Christopher Barker)
Date: Tue, 07 Nov 2006 16:53:47 -0800
Subject: "Slicing" recarrays
Message-ID: <45512A9B.9030809@noaa.gov>

Hi all,

I'm using a recarray to read a bunch of binary data out of a file. It's 
working great, but it seems there should be a more efficient way to 
"slice" the data. Here's what I've got:

The binary data is essentially a dump of a 2-d array of structs. So I 
read it like this:

DataType = N.dtype([("long","i4"), ("lat", "i4"), ("flag","b1")])

data = N.fromfile(file, DataType)

data.shape = (M, N)

So I now have a MxN array of the structs. What I would like to do is 
extract a MxNx2 array of just the two 4-byte integers. It seems that I 
should be able to do that without copying -- by using a view into the 
original data. I can't figure out how, however. What I am doing is:

LEs = N.empty((M, N, 2), dtype=N.int32)
LEs[:,:,0] = data['long']
LEs[:,:,1] = data['lat']

This works, but these are BIG files, so it would be nice not to be doing 
that extra copying. Is that possible?

Thanks,

-Chris







-- 
Christopher Barker, Ph.D.
Oceanographer
                                     		
NOAA/OR&R/HAZMAT         (206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115       (206) 526-6317   main reception

Chris.Barker at noaa.gov

-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From wbaxter at gmail.com  Tue Nov  7 18:41:37 2006
From: wbaxter at gmail.com (Bill Baxter)
Date: Wed, 8 Nov 2006 08:41:37 +0900
Subject: Stacking arrays...
In-Reply-To: <4550D6F8.80001@noaa.gov>
References: <4550D6F8.80001@noaa.gov>
Message-ID: <e86a5fd00611071541x701aa51cqf7c74533869c99e@mail.gmail.com>

dstack maybe? or does that add a dim on the other end?
--bb

On 11/8/06, Christopher Barker <Chris.Barker at noaa.gov> wrote:
>
> HI all,
>
> I'm trying to get the hang of this new r_ and c_ stuff.
>
> I need to take two MxN arrays, and stack them together into one MxNx2
> arrays. I found this works:
>
> >>> a = N.ones((3,4))
> >>> b = N.ones((3,4)) * 2
> >>> a
> array([[ 1.,  1.,  1.,  1.],
>         [ 1.,  1.,  1.,  1.],
>         [ 1.,  1.,  1.,  1.]])
> >>> b
> array([[ 2.,  2.,  2.,  2.],
>         [ 2.,  2.,  2.,  2.],
>         [ 2.,  2.,  2.,  2.]])
> >>> a.shape = (3,4,1)
> >>> b.shape = (3,4,1)
> >>> c = N.c_[a,b]
> >>> c.shape
> (3, 4, 2)
> >>> c[0,0]
> array([ 1.,  2.])
>
>
> What I'm wondering is if there is a way to do that without explicitly
> adding the extra dimension to a and b before calling c_ ?
>
> -Chris
>
>
> --
> Christopher Barker, Ph.D.
> Oceanographer
>
> NOAA/OR&R/HAZMAT         (206) 526-6959   voice
> 7600 Sand Point Way NE   (206) 526-6329   fax
> Seattle, WA  98115       (206) 526-6317   main reception
>
> Chris.Barker at noaa.gov
>
> -------------------------------------------------------------------------
> Using Tomcat but need to do more? Need to support web services, security?
> Get stuff done quickly with pre-integrated technology to make your job
> easier
> Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
> http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
> _______________________________________________
> Numpy-discussion mailing list
> Numpy-discussion at lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/numpy-discussion
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20061108/98a80663/attachment.html>
-------------- next part --------------
-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
-------------- next part --------------
_______________________________________________
Numpy-discussion mailing list
Numpy-discussion at lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/numpy-discussion

From michael.sorich at gmail.com  Tue Nov  7 20:11:24 2006
From: michael.sorich at gmail.com (Michael Sorich)
Date: Wed, 8 Nov 2006 11:41:24 +1030
Subject: A reimplementation of MaskedArray
In-Reply-To: <200610241408.40415.pgmdevlist@gmail.com>
References: <200610160232.20888.pgmdevlist@mailcan.com>
	<16761e100610232350u28329c5bp7944ac325030d0ba@mail.gmail.com>
	<200610241408.40415.pgmdevlist@gmail.com>
Message-ID: <16761e100611071711q6b9cc464p85708d7a642b9a8a@mail.gmail.com>

On 10/25/06, Pierre GM <pgmdevlist at gmail.com> wrote:
> On Tuesday 24 October 2006 02:50, Michael Sorich wrote:
> > I am currently running numpy rc2 (I haven't tried your
> > reimplementation yet as I am still using python 2.3). I am wondering
> > whether the new maskedarray is able to handle construction of arrays
> > from masked scalar values (not sure if this is the correct term).
>
> The answer is no, unfortunately: both the new and old implementations fail at
> the same point, raising a TypeError: lists are processed through
> numpy.core.numeric.array, which doesn't handle that.

I have finally gotten around to upgrading to python 2.4 and have had a
chance to play with your new version of the MaskedArray. It is great
to see that someone is working on this. I have a few thoughts on
masked arrays that may or may no warrant discusion

1. It would be nice if the masked_singleton could be passed into a
ndarray, as this would allow it to be passed into the MaskedArray e.g.

import numpy as N
import ma.maskedarray as MA
test = N.array([1,2,MA.masked])
>> ValueError: setting an array element with a sequence

If the masked_singleton was implemented as an object that is not a
MakedArray (which is a sequence that numpy.array chokes on), then a
valid numpy array with an object dtype could be produced. e.g.

class MaskedScalar:
    def __str__(self):
        return 'masked'
masked = MaskedScalar()

test = N.array([1,2,masked])
print test.dtype, test
>>object [1 2 masked]
print test == masked
>>[False False True]
print test[2] == masked
>>True
print test[2] is masked
>>True

Then it would be possible to alternatively define a masked array as
MA.array([1,2,masked]) or MA.array(N.array([1,2,masked])). In the
__init__ of the MaskedArray if the ndarray has an object dtype simply
calculate the mask from a==masked.

2. What happens if a masked array is passed into a ndarray or passed
into a MaskedArray with another mask?

test_ma1 = MA.array([1,2,3], mask=[False, False, True])
print test_ma1
>>[1 2 --]

print N.array(test_ma1)
>>[1 2 3]

test_ma2 = MA.array(test_ma1, mask=[True, False, False])
print test_ma2
>>[-- 2 3]

I suppose it depends on whether you are masking useful data, or the
masks represent missing data. In the former it may make sense to
change or remove the mask. However in the latter case the original
data entered is a bogus value which should never be unmasked. In this
case, when converting to a ndarray I think it make more sense to make
an object ndarray with the missing value containing the masked
singleton. Additionally, if the MaskedArray is holding missing data,
it does not make much sense to be able to pass in to the MA
constructor both an existing ma and a mask.

-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From tim.hochberg at ieee.org  Tue Nov  7 15:38:39 2006
From: tim.hochberg at ieee.org (Tim Hochberg)
Date: Tue, 07 Nov 2006 13:38:39 -0700
Subject: More on Numexpr-unsupported objects
In-Reply-To: <20061107171937.GI22979@tardis.terramar.selidor.net>
References: <20061107171937.GI22979@tardis.terramar.selidor.net>
Message-ID: <4550EECF.9070406@ieee.org>

Ivan Vilata i Balaguer wrote:


[SNIP]
>
> Though I admit it may be strange for this error to be triggered.
>
> (I had a little mess with using ``expressions.ExpressionNode`` instead
> of ``expr.ExpressionNode``...  I still don't see why the private copy of
> the module is necessary.)
>
>   
It's an attempt to make things thread safe. Since we are mucking around 
with the internals of the expression module, if there were multiple 
threads one might thread might change get_context out from under the 
other one. Giving each call to stringToExpression its own copy of the 
expression module prevents this.

It is,  admittedly, a kind of stupid way to do this. A more sane thing 
to do would probably to have some sort of expression object that we 
instantiate for each call to stringToExpression instead of mucking 
around with modules, which really aren't meant to be (ab)used this way. 
However, this would require a major rewrite of the expression module and 
I was feeling lazy at the time (and still am), so I resorted to 
questionable hackery.

-tim


-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From pau.gargallo at gmail.com  Tue Nov  7 17:15:03 2006
From: pau.gargallo at gmail.com (Pau Gargallo)
Date: Tue, 7 Nov 2006 23:15:03 +0100
Subject: Stacking arrays...
In-Reply-To: <4550D6F8.80001@noaa.gov>
References: <4550D6F8.80001@noaa.gov>
Message-ID: <6ef8f3380611071415k7e3c59d5pba7ff9338abec60f@mail.gmail.com>

On 11/7/06, Christopher Barker <Chris.Barker at noaa.gov> wrote:
> HI all,
>
> I'm trying to get the hang of this new r_ and c_ stuff.
>
> I need to take two MxN arrays, and stack them together into one MxNx2
> arrays. I found this works:
>
>  >>> a = N.ones((3,4))
>  >>> b = N.ones((3,4)) * 2
>  >>> a
> array([[ 1.,  1.,  1.,  1.],
>         [ 1.,  1.,  1.,  1.],
>         [ 1.,  1.,  1.,  1.]])
>  >>> b
> array([[ 2.,  2.,  2.,  2.],
>         [ 2.,  2.,  2.,  2.],
>         [ 2.,  2.,  2.,  2.]])
>  >>> a.shape = (3,4,1)
>  >>> b.shape = (3,4,1)
>  >>> c = N.c_[a,b]
>  >>> c.shape
> (3, 4, 2)
>  >>> c[0,0]
> array([ 1.,  2.])
>
>
> What I'm wondering is if there is a way to do that without explicitly
> adding the extra dimension to a and b before calling c_ ?
>

c=N.dstack([a,b]) seems to do the trick.

pau



> -Chris
>
>
> --
> Christopher Barker, Ph.D.
> Oceanographer
>
> NOAA/OR&R/HAZMAT         (206) 526-6959   voice
> 7600 Sand Point Way NE   (206) 526-6329   fax
> Seattle, WA  98115       (206) 526-6317   main reception
>
> Chris.Barker at noaa.gov
>
> -------------------------------------------------------------------------
> Using Tomcat but need to do more? Need to support web services, security?
> Get stuff done quickly with pre-integrated technology to make your job easier
> Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
> http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
> _______________________________________________
> Numpy-discussion mailing list
> Numpy-discussion at lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/numpy-discussion
>

-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From oliphant at ee.byu.edu  Tue Nov  7 21:01:21 2006
From: oliphant at ee.byu.edu (Travis Oliphant)
Date: Tue, 07 Nov 2006 19:01:21 -0700
Subject: "Slicing" recarrays
In-Reply-To: <45512A9B.9030809@noaa.gov>
References: <45512A9B.9030809@noaa.gov>
Message-ID: <45513A71.3060603@ee.byu.edu>

Christopher Barker wrote:

>Hi all,
>
>I'm using a recarray to read a bunch of binary data out of a file. It's 
>working great, but it seems there should be a more efficient way to 
>"slice" the data. Here's what I've got:
>
>The binary data is essentially a dump of a 2-d array of structs. So I 
>read it like this:
>
>DataType = N.dtype([("long","i4"), ("lat", "i4"), ("flag","b1")])
>
>data = N.fromfile(file, DataType)
>
>data.shape = (M, N)
>
>So I now have a MxN array of the structs. What I would like to do is 
>extract a MxNx2 array of just the two 4-byte integers. It seems that I 
>should be able to do that without copying -- by using a view into the 
>original data. I can't figure out how, however. What I am doing is:
>
>LEs = N.empty((M, N, 2), dtype=N.int32)
>LEs[:,:,0] = data['long']
>LEs[:,:,1] = data['lat']
>
>This works, but these are BIG files, so it would be nice not to be doing 
>that extra copying. Is that possible?
>  
>

How about

newdtype = N.dtype([('both','2i4'),('','b1')])
res = data.view(newdtype)['both']


-Travis


-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From david at ar.media.kyoto-u.ac.jp  Tue Nov  7 21:43:21 2006
From: david at ar.media.kyoto-u.ac.jp (David Cournapeau)
Date: Wed, 08 Nov 2006 11:43:21 +0900
Subject: Passing numpy arrays to matlab
In-Reply-To: <1162925985.6891.6.camel@localhost.localdomain>
References: <454FEE21.7040205@ar.media.kyoto-u.ac.jp>
	<1162925985.6891.6.camel@localhost.localdomain>
Message-ID: <45514449.1010803@ar.media.kyoto-u.ac.jp>

Pauli Virtanen wrote:
> Hi all,
>
> ti, 2006-11-07 kello 11:23 +0900, David Cournapeau kirjoitti:
>   
>>     I am trying to find a nice way to communicate between matlab and 
>> python. I am aware of pymat, which does that, but the code is 
>> deprecated, and I thing basing the code on ctypes would lead to much 
>> more robust code.
>>
>>     http://claymore.engineer.gvsu.edu/%7Esteriana/Software/pymat.html
>>
>> I have a really simple prototype which can send and get back data from 
>> matlab, but I was wondering if it would be possible to use a scheme 
>> similar to ctypes instead of having to convert it by hand.
>>     
>
> A while ago I wrote a mex extension to embed the Python interpreter
> inside Matlab:
>
> 	http://www.iki.fi/pav/pythoncall
>
> I guess it's something like an inverse of pymat :)
>
>   
Yes, but at the end, I think they enable similar things. Thanks for the 
link !
> But I guess this is not really what you are looking for, since at
> present it just does a memory copy when passing arrays between Matlab
> and Python. Though, shared arrays might be just possible to implement if
> memory management is done carefully.
>   
In my case, it is much worse:

    1 first, you have numpy data that you have to copy to mxArray, the 
structure representing arrays in matlab C api.
    2 then when you "send" data to the matlab engine, this is done 
automatically through pipe  by the matlab engine API (maybe pipe does 
not imply copying; I don't know much about pipe from a programming point 
of view, actually)
    3 The arrays you get back from matlab are in matlab mxArray 
structures: right now, I copy their data to new numpy arrays.

    At first, I just developed a prototype without thinking too much, 
and the result was much slower than I thought: sending a numpy  with 
2e5x10 double takes around 100 ms on my quite powerful machine (around 
14 cycles per item for the best case). I suspect it is because I copy 
memory in a non contiguous manner (matlab arrays have a internal F 
storage for real arrays, but complex arrays are really two different 
arrays, which is different than Fortran convention I think, making the 
copy cost really expensive for complex arrays). To see if I was doing 
something wrong, I compared with numpy.require(ar, requirements = 
'F_CONTIGUOUS'), which is even much slower
    There is not much I can do about 2, it looks like there is a way to 
avoid copying for 1, and my question was more specific to 3 (but 
reusable in 1, maybe, if I am smart enough). Basically:

     * how to create an object which has the same interface than numpy 
arrays, but owns the data from a foreign structure, which data are 
availble when building the object (The idea was to create a class which 
implements the array interface from python, kind of proxy class, which 
owns the data from mxArray; owns here is from a memory management point 
of view).

    David

-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From david at ar.media.kyoto-u.ac.jp  Tue Nov  7 22:07:08 2006
From: david at ar.media.kyoto-u.ac.jp (David Cournapeau)
Date: Wed, 08 Nov 2006 12:07:08 +0900
Subject: Passing numpy arrays to matlab
In-Reply-To: <E50CA0F6-3ED5-4ED0-92BA-B55991001504@gmail.com>
References: <mailman.119602.1162866273.22397.numpy-discussion@lists.sourceforge.net>
	<E50CA0F6-3ED5-4ED0-92BA-B55991001504@gmail.com>
Message-ID: <455149DC.3090701@ar.media.kyoto-u.ac.jp>

Josh Marshall wrote:
> Hi David,
>
> Did you have a look at mlabwrap? It's quite hard to find on the net,  
> which is a shame, since it is a much more up to date version,  
> enhancing pymat with the things that you are trying to do. It allows  
> passing arrays and getting arrays back.
>
> http://mlabwrap.sourceforge.net/
>   
I didn't know that, thanks. Unfortunately, it is not really what I am 
trying to do: mlabwrap is just a python interface a bit more high level 
than pymat, with many fancy tricks, but still do copies. What I would 
like is to avoid completely the copying by using proxy classes around 
data from numpy so that I can pass "automatically" numpy arrays to 
matlab C api, and a proxy class around data from matlab so that they 
look like numpy arrays.
    I don't care that much about the actual api from python point of 
view, because I intend to use this mainly to compare matlab vs numpy 
implementation, not as a way to use matlab inside python regularly. And 
once the copy problem is solved, adding syntactic sugar using python is 
easy anyway, I think (it should be easy to do something similar to 
mlabwrap at that point),

    cheers,

    David

-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From fperez.net at gmail.com  Wed Nov  8 03:32:44 2006
From: fperez.net at gmail.com (Fernando Perez)
Date: Wed, 8 Nov 2006 01:32:44 -0700
Subject: Profiling Python codes with hotshot and
	KCachegrind
Message-ID: <db6b5ecc0611080032n10bfa84fg6edb1e97f48f04ba@mail.gmail.com>

Hi all,

in the past, Arnd Baecker has made a number of very useful posts on
this matter, and provided some nice utilities to do it.  I now needed
to profile some fairly complex codes prior to a big optimization push,
so I went over his material and wrote a little tool to make the whole
process as painless as possible.  Here it is, hoping others may find
it useful:

http://amath.colorado.edu/faculty/fperez/python/profiling/

Cheers,

f

-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From pgmdevlist at gmail.com  Wed Nov  8 05:10:13 2006
From: pgmdevlist at gmail.com (Pierre GM)
Date: Wed, 8 Nov 2006 05:10:13 -0500
Subject: A reimplementation of MaskedArray
In-Reply-To: <16761e100611071711q6b9cc464p85708d7a642b9a8a@mail.gmail.com>
References: <200610160232.20888.pgmdevlist@mailcan.com>
	<200610241408.40415.pgmdevlist@gmail.com>
	<16761e100611071711q6b9cc464p85708d7a642b9a8a@mail.gmail.com>
Message-ID: <200611080510.13565.pgmdevlist@gmail.com>

Michael,
First of all, thanks for your interest in the exercise of style the new 
implementation of MaskedArray is basically nothing but.

On Tuesday 07 November 2006 20:11, Michael Sorich wrote:
> 1. It would be nice if the masked_singleton could be passed into a
> ndarray, as this would allow it to be passed into the MaskedArray e.g.
>
> import numpy as N
> import ma.maskedarray as MA
> test = N.array([1,2,MA.masked])
>
> >> ValueError: setting an array element with a sequence

I like your idea, but not its implementation. If MA.masked_singleton is 
defined as an object, as you suggest, then the dtype of the ndarray it is 
passed to becomes 'object', as you pointed out, and that is not something one 
would naturally expec, as basic numerical functions don't  work well  with the 
'object' dtype (just try  N.sqrt(N.array([1],dtype=N.object)) to see what I 
mean).
Even if we can construct a mask rather easily at the creation of the masked 
array, following your 'a==masked' suggestion, we still need to get the dtype 
of the non-masked section, and that doesn't seem trivial...

I guess that a simple solution is to use MA.masked_values. 
Make sure to use a numerical value for the masked data, else you'll end up 
with yet another object array.  


> 2. What happens if a masked array is passed into a ndarray or passed
> into a MaskedArray with another mask?
>>> test_ma1 = MA.array([1,2,3], mask=[False, False, True])
>>> print test_ma1, N.array(test_ma1), 
[1 2 --] [1 2 3]
>>> MA.array(test_ma1, mask=[True, False, False]) 
[-- 2 3]

Let me precise that my objective was to get an implementation as close to the 
original numpy.core.ma as possible, for 'backward compatibility'. I'm not 
sure it'd be wise to change it at this point, but that could be discussed.

As you've noticed, when creating a new masked array from an existing one, the 
'mask' argument supersedes the initial mask. That's ideal when you want to 
focus on a fraction of the initial data: you just mask what you don't need, 
and are still able to retrieve it when you need it. I agree that this default 
behavior is a bit strange when you have missing data: in that case, one would 
expect the new mask to be a combination of the 'mask' argument and the old 
mask. 
A possibility would then be to add a 'keep_mask' flag: a default of False 
would give the current behavior, a value of True would force the new mask to 
be a combination. I think that feelings are mixed on that list about extra 
flags, but the users of maskedarray are only a minority anyway (hopefully, 
only for the moment).

About the conversion to ndarray: 
By default, the result should have the same dtype as the _data section. 
For this reason, I disagree with your idea of "(returning) an object ndarray 
with the missing value containing the masked singleton". If you really want 
an object ndarray, you can use the filled method or the filled function, with 
your own definition of the filling value (such as your MaskedScalar). 







-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From ejaxvxjkxw at panoramapoint.com  Wed Nov  8 06:52:00 2006
From: ejaxvxjkxw at panoramapoint.com (other)
Date: Wed, 8 Nov 2006 12:52:00 +0100
Subject: Online Policy updated
Message-ID: <000701c7032c$4d0d9a90$6a2ca45b@moiax5xk5ayd7x>


Version recommends that upgrade most recent of through Download a Center.
Legacy Operating Systems nt Macintosh os x am Powerpc Safari.
Powerpc Safari Intel Linux Solaris Copyright Macromedia Software is llc. Designs animation user in? Center take advantage security updates Platform Browser Windows of Explorer of. Robust runtime am for rich media is is the standard. Adobeyour Guestyour States Management of Internet and in Audioweb Industries amp.
Rights Reserved Protected! Buy Searchyou may not have everything you.
Highimpact web content Designs animation a user.
Storestore Ways to buy Searchyou? Designs animation user or interfaces.
Rights Reserved Protected by a us is. Operating Systems nt am Macintosh os x a Powerpc!
Opera in Legacy Operating Systems nt a.
Controls plugins Firefox Mozilla of. Legacy Operating Systems a nt Macintosh!
Opera in Legacy Operating Systems nt a.
Updates Platform Browser Windows Explorer other? Web content Designs or animation user interfaces.
Are deployed across all browsers platforms attracting engaging users.
Media is the or standard. X Powerpc Safari Intel Linux Solaris Copyright Macromedia a!
Content Designs is animation user am interfaces are or deployed am.
Legacy Operating Systems nt Macintosh os x am Powerpc Safari.
Feedback this website or signifies your or agreement Terms.
Of Adobecom Please see of our or site Home.
Robust in runtime for rich a media in. Of Adobecom Please see of our or site Home.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20061108/18fe4ee3/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: contains.gif
Type: image/gif
Size: 8304 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20061108/18fe4ee3/attachment.gif>
-------------- next part --------------
-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
-------------- next part --------------
_______________________________________________
Numpy-discussion mailing list
Numpy-discussion at lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/numpy-discussion

From Mailer-Daemon at lists.sourceforge.net  Wed Nov  8 07:08:21 2006
From: Mailer-Daemon at lists.sourceforge.net (Mail Delivery System)
Date: Wed, 08 Nov 2006 04:08:21 -0800
Subject: Warning: message 1Ggbsf-00056T-Ol delayed 72
	hours
Message-ID: <E1GhmE9-0008RB-Ox@externalmx-1.sourceforge.net>

This message was created automatically by mail delivery software.
A message that you sent has not yet been delivered to one or more of its
recipients after more than 72 hours on the queue on externalmx-1.sourceforge.net.

The message identifier is:     1Ggbsf-00056T-Ol
The subject of the message is: faces and their former position.  To be, sure.  But question
The date of the message is:    Sun, 05 Nov 2006 16:01:50 +0900

The address to which the message has not yet been delivered is:

  numpy-discussion at lists.sourceforge.net
    Delay reason: SMTP error from remote mailer after RCPT TO:<numpy-discussion at lists.sourceforge.net>:
    host mail.sourceforge.net [66.35.250.206]: 451-Could not complete sender verify callout
    451-Could not complete sender verify callout for
    451-<numpy-discussion at lists.sourceforge.net>.
    451-The mail server(s) for the domain may be temporarily unreachable, or
    451-they may be permanently unreachable from this server. In the latter case,
    451-you need to change the address or create an MX record for its domain
    451-if it is 

No action is required on your part. Delivery attempts will continue for
some time, and this warning may be repeated at intervals if the message
remains undelivered. Eventually the mail delivery software will give up,
and when that happens, the message will be returned to you.

-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From amit_605 at yahoo.co.in  Wed Nov  8 07:42:39 2006
From: amit_605 at yahoo.co.in (amit soni)
Date: Wed, 8 Nov 2006 04:42:39 -0800 (PST)
Subject: reading matrix from a file
Message-ID: <20061108124239.20230.qmail@web8409.mail.in.yahoo.com>

Hi,
  i have a file with following format: 
 1 2 
 3 9 
 2 3 
 4 4 
   I want to read it and then store the values into two matrices, s.t. 
 A=[1 2;3 9] 
 B=[2 3;4 4] 
 
  Can anyone tell me how to do this in python?
 thanks 
 Amit 
  
 
---------------------------------
Check out the all-new Yahoo! Mail - Fire up a more powerful email and get things done faster.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20061108/1b3df0b8/attachment.html>
-------------- next part --------------
-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
-------------- next part --------------
_______________________________________________
Numpy-discussion mailing list
Numpy-discussion at lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/numpy-discussion

From izakmarais at yahoo.com  Wed Nov  8 08:54:17 2006
From: izakmarais at yahoo.com (izak marais)
Date: Wed, 8 Nov 2006 05:54:17 -0800 (PST)
Subject: matrix multiplication (newbie question)
Message-ID: <20061108135417.6906.qmail@web50913.mail.yahoo.com>

Hi

Sorry if this is an obvious question, but what is the easiest way to multiply matrices in numpy? Suppose I want to do A=B*C*D. The ' * ' operator apparently does element wise multiplication, as does the 'multiply' ufunc. All I could find was the numeric function 'matrix_multiply, but this only takes two arguments.

Thanks in advance!
Izak

 
---------------------------------
Sponsored Link

Degrees online in as fast as 1 Yr - MBA, Bachelor's, Master's, Associate - Click now to apply
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20061108/0b9c79bf/attachment.html>
-------------- next part --------------
-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
-------------- next part --------------
_______________________________________________
Numpy-discussion mailing list
Numpy-discussion at lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/numpy-discussion

From kwgoodman at gmail.com  Wed Nov  8 09:32:00 2006
From: kwgoodman at gmail.com (Keith Goodman)
Date: Wed, 8 Nov 2006 06:32:00 -0800
Subject: matrix multiplication (newbie question)
In-Reply-To: <20061108135417.6906.qmail@web50913.mail.yahoo.com>
References: <20061108135417.6906.qmail@web50913.mail.yahoo.com>
Message-ID: <f4f93d420611080632h3b3eb058qd3d0d0575d116988@mail.gmail.com>

On 11/8/06, izak marais <izakmarais at yahoo.com> wrote:

> Sorry if this is an obvious question, but what is the easiest way to
> multiply matrices in numpy? Suppose I want to do A=B*C*D. The ' * ' operator
> apparently does element wise multiplication, as does the 'multiply' ufunc.
> All I could find was the numeric function 'matrix_multiply, but this only
> takes two arguments.

If B and C and D are matrices, then '*' is matrix multiplication.

-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From stefan at sun.ac.za  Wed Nov  8 09:38:27 2006
From: stefan at sun.ac.za (Stefan van der Walt)
Date: Wed, 8 Nov 2006 16:38:27 +0200
Subject: matrix multiplication (newbie question)
In-Reply-To: <20061108135417.6906.qmail@web50913.mail.yahoo.com>
References: <20061108135417.6906.qmail@web50913.mail.yahoo.com>
Message-ID: <20061108143827.GO3380@mentat.za.net>

On Wed, Nov 08, 2006 at 05:54:17AM -0800, izak marais wrote:
> Hi
> 
> Sorry if this is an obvious question, but what is the easiest way to multiply
> matrices in numpy? Suppose I want to do A=B*C*D. The ' * ' operator apparently
> does element wise multiplication, as does the 'multiply' ufunc. All I could
> find was the numeric function 'matrix_multiply, but this only takes two
> arguments.

You can also do N.dot(B, N.dot(C,D)).

Cheers
St?fan



-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From nadavh at visionsense.com  Wed Nov  8 09:43:46 2006
From: nadavh at visionsense.com (Nadav Horesh)
Date: Wed, 8 Nov 2006 16:43:46 +0200
Subject: matrix multiplication (newbie question)
Message-ID: <07C6A61102C94148B8104D42DE95F7E8FCB333@exchange2k.envision.co.il>

Make A,B,... matrices instead of arrays, so instead

A = array((.....))

Write

A = matrix((....))

 

Assuming you had

>From numpy import *

 

  Nadav

 

  _____  

From: numpy-discussion-bounces at lists.sourceforge.net
[mailto:numpy-discussion-bounces at lists.sourceforge.net] On Behalf Of
izak marais
Sent: Wednesday, November 08, 2006 15:54
To: numpy-discussion at lists.sourceforge.net
Subject: [Numpy-discussion] matrix multiplication (newbie question)

 

Hi

Sorry if this is an obvious question, but what is the easiest way to
multiply matrices in numpy? Suppose I want to do A=B*C*D. The ' * '
operator apparently does element wise multiplication, as does the
'multiply' ufunc. All I could find was the numeric function
'matrix_multiply, but this only takes two arguments.

Thanks in advance!
Izak

  

  _____  

Sponsored Link

Degrees online in as fast as 1 Yr - MBA, Bachelor's, Master's, Associate
- Click now to apply
<http://online.degrees.info/forms/form.jsp?ADEL=D3sPPz6WYLD7%2FD1DxMxP%2
BbXooR4YcdmONmTWaUizNjh6q2wGRcnxuw%3D%3D&PI=y_mail_textlink> 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20061108/6ffe4425/attachment.html>
-------------- next part --------------
-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
-------------- next part --------------
_______________________________________________
Numpy-discussion mailing list
Numpy-discussion at lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/numpy-discussion

From david at ar.media.kyoto-u.ac.jp  Wed Nov  8 09:43:04 2006
From: david at ar.media.kyoto-u.ac.jp (David Cournapeau)
Date: Wed, 08 Nov 2006 23:43:04 +0900
Subject: Passing numpy arrays to matlab
In-Reply-To: <45501396.3070908@astraw.com>
References: <454FEE21.7040205@ar.media.kyoto-u.ac.jp>
	<45501396.3070908@astraw.com>
Message-ID: <4551ECF8.90806@ar.media.kyoto-u.ac.jp>

Andrew Straw wrote:
> David Cournapeau wrote:
>   
>>     - To send data from the calling process to matlab, you first have to 
>> create a mxArray, which is the basic matlab handler of a matlab array, 
>> and populating it. Using mxArray is very ackward : you cannot create 
>> mxArray from existing data, you have to copy data to them, etc... 
>>     
> My understanding, never having done it, but from reading the docs, is
> that you can create a "hybrid array" where you manage the memory. Thus,
> you can create an mxArray from existing data. However, the docs
> basically say that this is too hard for most mortals (and they may well
> be right -- too painful for me, anyway)!
>   
Ok, I have looked at it. It is not hard, it is just totally brain 
damaged: there is no way to destroy a mxArray without destroying the 
data it is holding, even after a call with mxSetPr. So the data 
referenced by the pointer given to mxSetPr is always destroyed by 
mxDestroyArray; I don't see any way to use this to avoid copy... They 
could at least have given a function which frees the data buffer and one 
which destroys the other stuff; as it is, it is totally useless, unless 
you don't mind memory leaks.

David

-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From faltet at carabos.com  Wed Nov  8 09:44:58 2006
From: faltet at carabos.com (Francesc Altet)
Date: Wed, 8 Nov 2006 15:44:58 +0100
Subject: reading matrix from a file
In-Reply-To: <20061108124239.20230.qmail@web8409.mail.in.yahoo.com>
References: <20061108124239.20230.qmail@web8409.mail.in.yahoo.com>
Message-ID: <200611081544.59856.faltet@carabos.com>

A Dimecres 08 Novembre 2006 13:42, amit soni escrigu?:
> Hi,
>   i have a file with following format:
>  1 2
>  3 9
>  2 3
>  4 4
>    I want to read it and then store the values into two matrices, s.t.
>  A=[1 2;3 9]
>  B=[2 3;4 4]
>
>   Can anyone tell me how to do this in python?
>  thanks
>  Amit

There are many possibilities. One of them could be:

In [64]: a = []; b = []

In [65]: for i, line in enumerate(file("/tmp/data.txt")):
   ....:     if i < 2:
   ....:         a.extend([float(n) for n in line.split()])
   ....:     else:
   ....:         b.extend([float(n) for n in line.split()])
   ....:

In [66]: A=numpy.array(a).reshape(2,2); B=numpy.array(b).reshape(2,2)

In [67]: A, B
Out[67]:
(array([[ 1.,  2.],
       [ 3.,  9.]]),
 array([[ 2.,  3.],
       [ 4.,  4.]]))


HTH,

-- 
>0,0<   Francesc Altet ? ? http://www.carabos.com/
V   V   C?rabos Coop. V. ??Enjoy Data
 "-"


-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
_______________________________________________
Numpy-discussion mailing list
Numpy-discussion at lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/numpy-discussion

From roberto at dealmeida.net  Wed Nov  8 09:45:15 2006
From: roberto at dealmeida.net (Roberto De Almeida)
Date: Wed, 8 Nov 2006 10:45:15 -0400
Subject: matrix multiplication (newbie question)
In-Reply-To: <f4f93d420611080632h3b3eb058qd3d0d0575d116988@mail.gmail.com>
References: <20061108135417.6906.qmail@web50913.mail.yahoo.com>
	<f4f93d420611080632h3b3eb058qd3d0d0575d116988@mail.gmail.com>
Message-ID: <10c662fe0611080645p6af724b1w2983e599d0157f81@mail.gmail.com>

On 11/8/06, Keith Goodman <kwgoodman at gmail.com> wrote:
> > Sorry if this is an obvious question, but what is the easiest way to
> > multiply matrices in numpy? Suppose I want to do A=B*C*D.
> If B and C and D are matrices, then '*' is matrix multiplication.

I think the difference between arrays and matrices is not clear for
him (he's clearly multiplying arrays).

Izak, you should first convert you arrays to matrices using ``numpy.matrix``.

--Rob

-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From aswart at dip.sun.ac.za  Wed Nov  8 09:47:33 2006
From: aswart at dip.sun.ac.za (Albert Swart)
Date: Wed, 08 Nov 2006 16:47:33 +0200
Subject: matrix multiplication (newbie question)
In-Reply-To: <10c662fe0611080645p6af724b1w2983e599d0157f81@mail.gmail.com>
References: <20061108135417.6906.qmail@web50913.mail.yahoo.com>
	<f4f93d420611080632h3b3eb058qd3d0d0575d116988@mail.gmail.com>
	<10c662fe0611080645p6af724b1w2983e599d0157f81@mail.gmail.com>
Message-ID: <1162997254.5646.19.camel@localhost>


> Izak, you should first convert you arrays to matrices using ``numpy.matrix``.

or numpy.asmatrix()
> 
> --Rob
> 
> -------------------------------------------------------------------------
> Using Tomcat but need to do more? Need to support web services, security?
> Get stuff done quickly with pre-integrated technology to make your job easier
> Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
> http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
> _______________________________________________
> Numpy-discussion mailing list
> Numpy-discussion at lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/numpy-discussion


-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From charlesr.harris at gmail.com  Wed Nov  8 09:48:48 2006
From: charlesr.harris at gmail.com (Charles R Harris)
Date: Wed, 8 Nov 2006 07:48:48 -0700
Subject: matrix multiplication (newbie question)
In-Reply-To: <f4f93d420611080632h3b3eb058qd3d0d0575d116988@mail.gmail.com>
References: <20061108135417.6906.qmail@web50913.mail.yahoo.com>
	<f4f93d420611080632h3b3eb058qd3d0d0575d116988@mail.gmail.com>
Message-ID: <e06186140611080648o7e6869a9vca150c7a9b506aa9@mail.gmail.com>

On 11/8/06, Keith Goodman <kwgoodman at gmail.com> wrote:
>
> On 11/8/06, izak marais <izakmarais at yahoo.com> wrote:
>
> > Sorry if this is an obvious question, but what is the easiest way to
> > multiply matrices in numpy? Suppose I want to do A=B*C*D. The ' * '
> operator
> > apparently does element wise multiplication, as does the 'multiply'
> ufunc.
> > All I could find was the numeric function 'matrix_multiply, but this
> only
> > takes two arguments.


Same with the operator *, it takes two arguments but is in infix order, i.e.,
left side and right side.

If B and C and D are matrices, then '*' is matrix multiplication.


And if they are arrays:

A = dot(B,dot(C,D))

Python has a dearth of recognized operators which makes this necessary once
'*' is used for elementwise multiplication, it's a long standing complaint.
You can use matrices in numpy, in which case '*' is used for matrix
multiplication like in matlab, but I think it would be better to get used to
using arrays as they are the numpy core.

Chuck
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20061108/a9a0fcdf/attachment.html>
-------------- next part --------------
-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
-------------- next part --------------
_______________________________________________
Numpy-discussion mailing list
Numpy-discussion at lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/numpy-discussion

From svetosch at gmx.net  Wed Nov  8 09:51:09 2006
From: svetosch at gmx.net (Sven Schreiber)
Date: Wed, 08 Nov 2006 15:51:09 +0100
Subject: matrix multiplication (newbie question)
In-Reply-To: <20061108135417.6906.qmail@web50913.mail.yahoo.com>
References: <20061108135417.6906.qmail@web50913.mail.yahoo.com>
Message-ID: <4551EEDD.8010405@gmx.net>

izak marais schrieb:
> Hi
> 
> Sorry if this is an obvious question, but what is the easiest way to
> multiply matrices in numpy? Suppose I want to do A=B*C*D. The ' * '
> operator apparently does element wise multiplication, as does the
> 'multiply' ufunc. All I could find was the numeric function
> 'matrix_multiply, but this only takes two arguments.
> 
> Thanks in advance!
> Izak

There are (at least) two ways:

You can use 'dot', possibly nested.

or you can convert your arrays into the matrix subclass, for which '*'
is matrix multiplication. I.e. mat(B)*mat(C)*mat(D) does what you want.

If you "only" deal with algebra-style matrices (2d-arrays), consider
using the matrix subclass as much as you can. E.g. use the functions in
numpy.matlib to build your inputs.
-sven

-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From charlesr.harris at gmail.com  Wed Nov  8 09:55:41 2006
From: charlesr.harris at gmail.com (Charles R Harris)
Date: Wed, 8 Nov 2006 07:55:41 -0700
Subject: reading matrix from a file
In-Reply-To: <200611081544.59856.faltet@carabos.com>
References: <20061108124239.20230.qmail@web8409.mail.in.yahoo.com>
	<200611081544.59856.faltet@carabos.com>
Message-ID: <e06186140611080655h604a71f8q6dcd046ac248033e@mail.gmail.com>

On 11/8/06, Francesc Altet <faltet at carabos.com> wrote:
>
> A Dimecres 08 Novembre 2006 13:42, amit soni escrigu?:
> > Hi,
> >   i have a file with following format:
> >  1 2
> >  3 9
> >  2 3
> >  4 4
> >    I want to read it and then store the values into two matrices, s.t.
> >  A=[1 2;3 9]
> >  B=[2 3;4 4]
> >
> >   Can anyone tell me how to do this in python?
> >  thanks
> >  Amit


Try

In [8]: tmp = fromfile('tmp.txt', sep=' ', dtype=int)

In [9]: a = tmp[:4].reshape(2,2)

In [10]: b = tmp[4:].reshape(2,2)

In [11]: a
Out[11]:
array([[1, 2],
       [3, 9]])

In [12]: b
Out[12]:
array([[2, 3],
       [4, 4]])

Chuck
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20061108/fe8dbf12/attachment.html>
-------------- next part --------------
-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
-------------- next part --------------
_______________________________________________
Numpy-discussion mailing list
Numpy-discussion at lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/numpy-discussion

From joris at ster.kuleuven.be  Wed Nov  8 09:00:00 2006
From: joris at ster.kuleuven.be (Joris De Ridder)
Date: Wed, 8 Nov 2006 15:00:00 +0100
Subject: matrix multiplication (newbie question)
In-Reply-To: <20061108135417.6906.qmail@web50913.mail.yahoo.com>
References: <20061108135417.6906.qmail@web50913.mail.yahoo.com>
Message-ID: <200611081500.00355.joris@ster.kuleuven.be>


   [im]: Sorry if this is an obvious question, but what is the easiest way to multiply matrices in numpy? Suppose I want to do A=B*C*D. The ' * ' operator apparently does element wise multiplication, as does the 'multiply' ufunc. 
   [im] All I could find was the numeric function 'matrix_multiply, but this only takes two arguments.

Have a look at the examples "dot()" and "mat()" in the Numpy Example List.
http://www.scipy.org/Numpy_Example_List

J.

Disclaimer: http://www.kuleuven.be/cwis/email_disclaimer.htm


-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From faltet at carabos.com  Wed Nov  8 10:01:31 2006
From: faltet at carabos.com (Francesc Altet)
Date: Wed, 8 Nov 2006 16:01:31 +0100
Subject: reading matrix from a file
In-Reply-To: <e06186140611080655h604a71f8q6dcd046ac248033e@mail.gmail.com>
References: <20061108124239.20230.qmail@web8409.mail.in.yahoo.com>
	<200611081544.59856.faltet@carabos.com>
	<e06186140611080655h604a71f8q6dcd046ac248033e@mail.gmail.com>
Message-ID: <200611081601.32248.faltet@carabos.com>

A Dimecres 08 Novembre 2006 15:55, Charles R Harris escrigu?:
> Try
>
> In [8]: tmp = fromfile('tmp.txt', sep=' ', dtype=int)
>
> In [9]: a = tmp[:4].reshape(2,2)
>
> In [10]: b = tmp[4:].reshape(2,2)
>
> In [11]: a
> Out[11]:
> array([[1, 2],
>        [3, 9]])
>
> In [12]: b
> Out[12]:
> array([[2, 3],
>        [4, 4]])

Yeah. Much, much better indeed.

-- 
>0,0<   Francesc Altet ? ? http://www.carabos.com/
V   V   C?rabos Coop. V. ??Enjoy Data
 "-"


-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
_______________________________________________
Numpy-discussion mailing list
Numpy-discussion at lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/numpy-discussion

From stefan at sun.ac.za  Wed Nov  8 10:20:02 2006
From: stefan at sun.ac.za (Stefan van der Walt)
Date: Wed, 8 Nov 2006 17:20:02 +0200
Subject: Profiling Python codes with hotshot and
	KCachegrind
In-Reply-To: <db6b5ecc0611080032n10bfa84fg6edb1e97f48f04ba@mail.gmail.com>
References: <db6b5ecc0611080032n10bfa84fg6edb1e97f48f04ba@mail.gmail.com>
Message-ID: <20061108152002.GP3380@mentat.za.net>

On Wed, Nov 08, 2006 at 01:32:44AM -0700, Fernando Perez wrote:
> Hi all,
> 
> in the past, Arnd Baecker has made a number of very useful posts on
> this matter, and provided some nice utilities to do it.  I now needed
> to profile some fairly complex codes prior to a big optimization push,
> so I went over his material and wrote a little tool to make the whole
> process as painless as possible.  Here it is, hoping others may find
> it useful:
> 
> http://amath.colorado.edu/faculty/fperez/python/profiling/

This looks very interesting.  It works for me on simple scripts, but
whenever I include the lines

from numpy.testing import set_local_path
set_local_path('../../..')

in the input, pycachegrind aborts with

  File "/home/stefan//lib/python2.4/site-packages/numpy/testing/numpytest.py", line 68, in set_local_path
    if f.f_locals['__name__']=='__main__':
KeyError: '__name__'

I guess this is because the script is run in a separate namespace.
I've managed to work around the problem by changing the definition of
'run' to:

def run(code):
    loc = locals()
    loc['__name__'] = '__main__'
    loc['__file__'] = sys.argv[0]

    exec code in locals()


Cheers
St?fan

-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From a.u.r.e.l.i.a.n at gmx.net  Wed Nov  8 11:16:17 2006
From: a.u.r.e.l.i.a.n at gmx.net (Johannes Loehnert)
Date: Wed, 8 Nov 2006 17:16:17 +0100
Subject: matrix multiplication (newbie question)
In-Reply-To: <20061108135417.6906.qmail@web50913.mail.yahoo.com>
References: <20061108135417.6906.qmail@web50913.mail.yahoo.com>
Message-ID: <200611081716.18422.a.u.r.e.l.i.a.n@gmx.net>

Hi,

in extension to the previous answers, I'd like to say that it is strongly 
preferable to use dot(A,dot(B,C)) or dot(dot(A,B),C) instead of A*B*C.

The reason is that with dot(), you can control of which operation is performed 
first, which can *massively* influence the time needed, depending on the 
involved matrices. A*B*C will always be evaluated left-to-right (if I 
remember correctly).

Johannes

-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From strawman at astraw.com  Wed Nov  8 11:57:44 2006
From: strawman at astraw.com (Andrew Straw)
Date: Wed, 08 Nov 2006 08:57:44 -0800
Subject: Passing numpy arrays to matlab
In-Reply-To: <4551ECF8.90806@ar.media.kyoto-u.ac.jp>
References: <454FEE21.7040205@ar.media.kyoto-u.ac.jp>	<45501396.3070908@astraw.com>
	<4551ECF8.90806@ar.media.kyoto-u.ac.jp>
Message-ID: <45520C88.6020902@astraw.com>

David Cournapeau wrote:
> Andrew Straw wrote:
>   
>> David Cournapeau wrote:
>>   
>>     
>>>     - To send data from the calling process to matlab, you first have to 
>>> create a mxArray, which is the basic matlab handler of a matlab array, 
>>> and populating it. Using mxArray is very ackward : you cannot create 
>>> mxArray from existing data, you have to copy data to them, etc... 
>>>     
>>>       
>> My understanding, never having done it, but from reading the docs, is
>> that you can create a "hybrid array" where you manage the memory. Thus,
>> you can create an mxArray from existing data. However, the docs
>> basically say that this is too hard for most mortals (and they may well
>> be right -- too painful for me, anyway)!
>>   
>>     
> Ok, I have looked at it. It is not hard, it is just totally brain 
> damaged: there is no way to destroy a mxArray without destroying the 
> data it is holding, even after a call with mxSetPr. So the data 
> referenced by the pointer given to mxSetPr is always destroyed by 
> mxDestroyArray; I don't see any way to use this to avoid copy... They 
> could at least have given a function which frees the data buffer and one 
> which destroys the other stuff; as it is, it is totally useless, unless 
> you don't mind memory leaks.
>   
It does sound brain damaged, I agree. But here's a suggestion: can you
keep a pool of unused mxArrays rather than calling mxDestroyArray? I
guess without the payload, they're just a few bytes and shouldn't take
up that much space.

-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From fperez.net at gmail.com  Wed Nov  8 11:57:28 2006
From: fperez.net at gmail.com (Fernando Perez)
Date: Wed, 8 Nov 2006 09:57:28 -0700
Subject: Profiling Python codes with hotshot and
	KCachegrind
In-Reply-To: <20061108152002.GP3380@mentat.za.net>
References: <db6b5ecc0611080032n10bfa84fg6edb1e97f48f04ba@mail.gmail.com>
	<20061108152002.GP3380@mentat.za.net>
Message-ID: <db6b5ecc0611080857p4ef97b52v2e22b54e4f7801d@mail.gmail.com>

On 11/8/06, Stefan van der Walt <stefan at sun.ac.za> wrote:

> This looks very interesting.  It works for me on simple scripts, but
> whenever I include the lines
>
> from numpy.testing import set_local_path
> set_local_path('../../..')
>
> in the input, pycachegrind aborts with
>
>   File "/home/stefan//lib/python2.4/site-packages/numpy/testing/numpytest.py", line 68, in set_local_path
>     if f.f_locals['__name__']=='__main__':
> KeyError: '__name__'
>
> I guess this is because the script is run in a separate namespace.
> I've managed to work around the problem by changing the definition of
> 'run' to:

Good catch, thanks, I've fixed the public version with your changes.

Best,

f

-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From amit_605 at yahoo.co.in  Wed Nov  8 12:36:14 2006
From: amit_605 at yahoo.co.in (amit soni)
Date: Wed, 8 Nov 2006 09:36:14 -0800 (PST)
Subject: Calculating tan inverse
Message-ID: <20061108173614.6777.qmail@web8414.mail.in.yahoo.com>

how can I calculate arctan of a number in python?
thanks
Amit

 
---------------------------------
Sponsored Link

Mortgage rates near 39yr lows. $420,000 Mortgage for $1,399/mo - Calculate new house payment
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20061108/b44ecd10/attachment.html>
-------------- next part --------------
-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
-------------- next part --------------
_______________________________________________
Numpy-discussion mailing list
Numpy-discussion at lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/numpy-discussion

From nadavh at visionsense.com  Wed Nov  8 12:46:52 2006
From: nadavh at visionsense.com (Nadav Horesh)
Date: Wed, 8 Nov 2006 19:46:52 +0200
Subject: Calculating tan inverse
Message-ID: <07C6A61102C94148B8104D42DE95F7E8FCB351@exchange2k.envision.co.il>

There is arctan function in numpy, and in math (atan, atan2)

 

  Nadav.

 

  _____  

From: numpy-discussion-bounces at lists.sourceforge.net
[mailto:numpy-discussion-bounces at lists.sourceforge.net] On Behalf Of
amit soni
Sent: Wednesday, November 08, 2006 19:36
To: numpy-discussion at lists.sourceforge.net
Subject: [Numpy-discussion] Calculating tan inverse

 

how can I calculate arctan of a number in python?
thanks
Amit

  

  _____  

Sponsored Link

Mortgage rates near 39yr lows. $420,000 Mortgage for $1,399/mo -
Calculate new house payment
<http://www.lowermybills.com/lre/index.jsp?sourceid=lmb-9069-16273&moid=
4116> 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20061108/dae0706c/attachment.html>
-------------- next part --------------
-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
-------------- next part --------------
_______________________________________________
Numpy-discussion mailing list
Numpy-discussion at lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/numpy-discussion

From faltet at carabos.com  Wed Nov  8 13:00:08 2006
From: faltet at carabos.com (Francesc Altet)
Date: Wed, 8 Nov 2006 19:00:08 +0100
Subject: Calculating tan inverse
In-Reply-To: <20061108173614.6777.qmail@web8414.mail.in.yahoo.com>
References: <20061108173614.6777.qmail@web8414.mail.in.yahoo.com>
Message-ID: <200611081900.08524.faltet@carabos.com>

A Dimecres 08 Novembre 2006 18:36, amit soni escrigu?:
> how can I calculate arctan of a number in python?
> thanks
> Amit

Have you tried to read the numpy docs? some googling? In general, you
can get a lot of insight by querying this to Google:

"your words" site:www.scipy.org

in particular, try:

arctan site:www.scipy.org

Cheers,

-- 
>0,0<   Francesc Altet ? ? http://www.carabos.com/
V   V   C?rabos Coop. V. ??Enjoy Data
 "-"



-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
_______________________________________________
Numpy-discussion mailing list
Numpy-discussion at lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/numpy-discussion

From peridot.faceted at gmail.com  Wed Nov  8 13:41:55 2006
From: peridot.faceted at gmail.com (A. M. Archibald)
Date: Wed, 8 Nov 2006 13:41:55 -0500
Subject: A reimplementation of MaskedArray
In-Reply-To: <200611080510.13565.pgmdevlist@gmail.com>
References: <200610160232.20888.pgmdevlist@mailcan.com>
	<200610241408.40415.pgmdevlist@gmail.com>
	<16761e100611071711q6b9cc464p85708d7a642b9a8a@mail.gmail.com>
	<200611080510.13565.pgmdevlist@gmail.com>
Message-ID: <ce557a360611081041t6748bcdj953aa6aaea9cab18@mail.gmail.com>

On 08/11/06, Pierre GM <pgmdevlist at gmail.com> wrote:

> I like your idea, but not its implementation. If MA.masked_singleton is
> defined as an object, as you suggest, then the dtype of the ndarray it is
> passed to becomes 'object', as you pointed out, and that is not something one
> would naturally expec, as basic numerical functions don't  work well  with the
> 'object' dtype (just try  N.sqrt(N.array([1],dtype=N.object)) to see what I
> mean).
> Even if we can construct a mask rather easily at the creation of the masked
> array, following your 'a==masked' suggestion, we still need to get the dtype
> of the non-masked section, and that doesn't seem trivial...

A good candidate for "should be masked" marked is NaN. It is supposed
to mean, more or less, "no sensible value". Unfortunately, integer
types do not have such a special value. It's also conceivable that
some user might want to keep NaNs in their array separate from the
mask. Finally, on some hardware, operations with NaN are very slow (so
leaving them in the array, even masked, might not be a good idea).

The reason I suggest this is that in the last major application I had
for numpy, one stage of the problem would occasionally result in NaNs
for certain values, but the best thing I could do was leave them in
place to represent "no data". Switching to a MaskedArray might have
been a better idea, but the NaNs were a rare occurrence.

> About the conversion to ndarray:
> By default, the result should have the same dtype as the _data section.
> For this reason, I disagree with your idea of "(returning) an object ndarray
> with the missing value containing the masked singleton". If you really want
> an object ndarray, you can use the filled method or the filled function, with
> your own definition of the filling value (such as your MaskedScalar).

If you've got floating point, you can again fill in NaNs, but you have
a good point about wanting to extract the original values that were
masked out. Depending on what one is doing, one might want one or the
other.

A. M. Archibald

-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From pgmdevlist at gmail.com  Wed Nov  8 16:08:25 2006
From: pgmdevlist at gmail.com (Pierre GM)
Date: Wed, 8 Nov 2006 16:08:25 -0500
Subject: A reimplementation of MaskedArray
In-Reply-To: <ce557a360611081041t6748bcdj953aa6aaea9cab18@mail.gmail.com>
References: <200610160232.20888.pgmdevlist@mailcan.com>
	<200611080510.13565.pgmdevlist@gmail.com>
	<ce557a360611081041t6748bcdj953aa6aaea9cab18@mail.gmail.com>
Message-ID: <200611081608.25872.pgmdevlist@gmail.com>

> A good candidate for "should be masked" marked is NaN. It is supposed
> to mean, more or less, "no sensible value". 

Which might turn out out to be the best indeed. Michael's application would 
then look like
>>> import numpy as N
>>> import maskedarray as MA
>>> maskit = N.nan
>>> test = N.array([1,2,maskit])
>>> test_ma1 = MA.array(x,mask=N.isnan(x))

> Switching to a MaskedArray might have
> been a better idea, but the NaNs were a rare occurrence.
Once again, that's a situation when one would use masked arrays. 

> If you've got floating point, you can again fill in NaNs, but you have
> a good point about wanting to extract the original values that were
> masked out. Depending on what one is doing, one might want one or the
> other.

In any case, I think we should stick to the numpy.core.ma default behavior for 
backwards compatibility. If you really wanht to distinguish between several 
kind of masks (one for missing data, one for data to discard temporarily), 
that could be done by defining a special subclass. But is it really needed ? 
A smart use of filled and masked_values should do the trick in most cases.

-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From svetosch at gmx.net  Wed Nov  8 17:00:03 2006
From: svetosch at gmx.net (Sven Schreiber)
Date: Wed, 08 Nov 2006 23:00:03 +0100
Subject: matrix multiplication (newbie question)
In-Reply-To: <200611081716.18422.a.u.r.e.l.i.a.n@gmx.net>
References: <20061108135417.6906.qmail@web50913.mail.yahoo.com>
	<200611081716.18422.a.u.r.e.l.i.a.n@gmx.net>
Message-ID: <45525363.1030404@gmx.net>

Johannes Loehnert schrieb:
> Hi,
> 
> in extension to the previous answers, I'd like to say that it is strongly 
> preferable to use dot(A,dot(B,C)) or dot(dot(A,B),C) instead of A*B*C.
> 
> The reason is that with dot(), you can control of which operation is performed 
> first, which can *massively* influence the time needed, depending on the 
> involved matrices. A*B*C will always be evaluated left-to-right (if I 
> remember correctly).
> 

Well what about A*(B*C)?

-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From koara at atlas.cz  Wed Nov  8 17:36:33 2006
From: koara at atlas.cz (koara)
Date: Wed, 08 Nov 2006 14:36:33 -0800
Subject: slice assignment: strange behaviour
Message-ID: <1163025393.211885.161470@e3g2000cwe.googlegroups.com>

Hello,

a piece of my code started giving strange results with certain data; i
managed to track down the cause to a slice array assignment. In the
following code snip; 'mat' is a numpy.array with shape=(22973, 1009),
'vec' is a numpy.array with shape=(22973,), both of type int:

for i in xrange(1009):
    ...
    fr = vec[10001]
    mat[:, i] = vec # assign whole column
    if mat[10001, i] != fr:
        print "how come?"
     ...

for elements beyond index 10000, nothing is assigned (ie,
numpy.sum(mat[row, :]) is zero for any row > 10000). As soon as i
replace the assignment with a cycle that assigns each element
explicitly (for j in xrange(22973): result[j, i] = vec[j]),
everything's OK. With some matrices, the above seems to work fine
though, as well as for smaller dimensions, so i am unable to provide a
full simple example.

Any ideas? I am using enthought python (python 2.4.3) which uses numpy
version 0.9.9.


-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From vincent.broman at navy.mil  Wed Nov  8 17:20:55 2006
From: vincent.broman at navy.mil (Vincent Broman)
Date: Wed, 8 Nov 2006 22:20:55 +0000 (UTC)
Subject: building fc6 rpm of numpy
Message-ID: <loom.20061108T230408-554@post.gmane.org>

Building an rpm of numpy-1.0.1.dev3432-1 on fedora core 6 is failing for me.
With either python-2.4.3 or 2.4.4 I try "python setup.py bdist_rpm"
inside the source directory, and everything seems to go well except for
many "File listed twice" messages for all kinds of files,
and then at the end there is the error message:

    error: Installed (but unpackaged) file(s) found

with a long list of .pyc and .pyo files.
The same seems to happen to me with the source rpm I downloaded
from sourceforge. And I vaguely recall the same happening with
my fedora core 4 + updates, which I had till my recent upgrade to fc6.

The list or lists of files to install are not in the %files section,
but are generated dynamically somehow by setup.py, but I don't
know enough about distutils to debug that.

Anyone succeed with fc6 rpms or have any suggestion?

Vincent Broman
SSC-SD



-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From koara at atlas.cz  Wed Nov  8 18:08:27 2006
From: koara at atlas.cz (koara)
Date: Wed, 08 Nov 2006 15:08:27 -0800
Subject: slice assignment: strange behaviour
In-Reply-To: <1163025393.211885.161470@e3g2000cwe.googlegroups.com>
References: <1163025393.211885.161470@e3g2000cwe.googlegroups.com>
Message-ID: <1163027307.938949.183140@m73g2000cwd.googlegroups.com>


koara wrote:
> Hello,
>
> a piece of my code started giving strange results with certain data; i
> managed to track down the cause to a slice array assignment. In the
> ....

Also if i first build a sequence of columns and then use
numpy.transpose(numpy.vstack(sequence)) the result is ok. But the
arrays can be quite big so in this case i fear space overhead -- or am
i wrong?

Cheers.


-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From fullung at gmail.com  Wed Nov  8 18:14:28 2006
From: fullung at gmail.com (Albert Strasheim)
Date: Thu, 9 Nov 2006 01:14:28 +0200
Subject: building fc6 rpm of numpy
In-Reply-To: <loom.20061108T230408-554@post.gmane.org>
References: <loom.20061108T230408-554@post.gmane.org>
Message-ID: <20061108231428.GA6848@dogbert.sdsl.sun.ac.za>

Howdy

On Wed, 08 Nov 2006, Vincent Broman wrote:

> Building an rpm of numpy-1.0.1.dev3432-1 on fedora core 6 is failing for me.
> With either python-2.4.3 or 2.4.4 I try "python setup.py bdist_rpm"
> inside the source directory, and everything seems to go well except for
> many "File listed twice" messages for all kinds of files,
> and then at the end there is the error message:
> 
>     error: Installed (but unpackaged) file(s) found
> 
> with a long list of .pyc and .pyo files.
> The same seems to happen to me with the source rpm I downloaded
> from sourceforge. And I vaguely recall the same happening with
> my fedora core 4 + updates, which I had till my recent upgrade to fc6.
> 
> The list or lists of files to install are not in the %files section,
> but are generated dynamically somehow by setup.py, but I don't
> know enough about distutils to debug that.
> 
> Anyone succeed with fc6 rpms or have any suggestion?

I also ran into this problem a few days ago. I'm guessing it's a bug in 
either distutils or numpy.distutils.

Anyway, you can hack it to work by putting

%_unpackaged_files_terminate_build 1

in your ~/.rpmmacros. More details here:

http://www.rpm.org/hintskinks/unpackaged-files/

With that in place, bdist_rpm works on my FC6 machine. rpm -qpl on the 
RPM seems to indicate a "sane" RPM.

Cheers,

Albert

-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From fullung at gmail.com  Wed Nov  8 18:18:52 2006
From: fullung at gmail.com (Albert Strasheim)
Date: Thu, 9 Nov 2006 01:18:52 +0200
Subject: building fc6 rpm of numpy
In-Reply-To: <20061108231428.GA6848@dogbert.sdsl.sun.ac.za>
References: <loom.20061108T230408-554@post.gmane.org>
	<20061108231428.GA6848@dogbert.sdsl.sun.ac.za>
Message-ID: <20061108231852.GB6848@dogbert.sdsl.sun.ac.za>

Argh,

On Thu, 09 Nov 2006, Albert Strasheim wrote:

> %_unpackaged_files_terminate_build 1

Cut and paste error. Make that

%_unpackaged_files_terminate_build 0

Cheers,

Albert

-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From robert.kern at gmail.com  Wed Nov  8 18:26:23 2006
From: robert.kern at gmail.com (Robert Kern)
Date: Wed, 08 Nov 2006 17:26:23 -0600
Subject: slice assignment: strange behaviour
In-Reply-To: <1163027307.938949.183140@m73g2000cwd.googlegroups.com>
References: <1163025393.211885.161470@e3g2000cwe.googlegroups.com>
	<1163027307.938949.183140@m73g2000cwd.googlegroups.com>
Message-ID: <eitp38$ro7$1@sea.gmane.org>

koara wrote:
> koara wrote:
>> Hello,
>>
>> a piece of my code started giving strange results with certain data; i
>> managed to track down the cause to a slice array assignment. In the
>> ....
> 
> Also if i first build a sequence of columns and then use
> numpy.transpose(numpy.vstack(sequence)) the result is ok. But the
> arrays can be quite big so in this case i fear space overhead -- or am
> i wrong?

You might want to use column_stack() instead, for clarity if nothing else.

-- 
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless enigma
 that is made terrible by our own mad attempt to interpret it as though it had
 an underlying truth."
  -- Umberto Eco


-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From wbaxter at gmail.com  Wed Nov  8 18:39:07 2006
From: wbaxter at gmail.com (Bill Baxter)
Date: Thu, 9 Nov 2006 08:39:07 +0900
Subject: matrix multiplication (newbie question)
In-Reply-To: <200611081500.00355.joris@ster.kuleuven.be>
References: <20061108135417.6906.qmail@web50913.mail.yahoo.com>
	<200611081500.00355.joris@ster.kuleuven.be>
Message-ID: <e86a5fd00611081539m6ccceb0aj7f59a826e55f4b3b@mail.gmail.com>

Also have a look at the section on Arrays vs Matrices in the Numpy for
Matlab users page.  That particular section has nothing to do with
Matlab, really.
http://www.scipy.org/NumPy_for_Matlab_Users
Most of the suggestions and comments made here are already on that page.

--bb

On 11/8/06, Joris De Ridder <joris at ster.kuleuven.be> wrote:
>
>    [im]: Sorry if this is an obvious question, but what is the easiest way to multiply matrices in numpy? Suppose I want to do A=B*C*D. The ' * ' operator apparently does element wise multiplication, as does the 'multiply' ufunc.
>    [im] All I could find was the numeric function 'matrix_multiply, but this only takes two arguments.
>
> Have a look at the examples "dot()" and "mat()" in the Numpy Example List.
> http://www.scipy.org/Numpy_Example_List
>
> J.
>
> Disclaimer: http://www.kuleuven.be/cwis/email_disclaimer.htm
>
>
> -------------------------------------------------------------------------
> Using Tomcat but need to do more? Need to support web services, security?
> Get stuff done quickly with pre-integrated technology to make your job easier
> Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
> http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
> _______________________________________________
> Numpy-discussion mailing list
> Numpy-discussion at lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/numpy-discussion
>

-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From ndbecker2 at gmail.com  Wed Nov  8 18:42:52 2006
From: ndbecker2 at gmail.com (Neal Becker)
Date: Wed, 08 Nov 2006 18:42:52 -0500
Subject: building fc6 rpm of numpy
References: <loom.20061108T230408-554@post.gmane.org>
Message-ID: <eitq1h$uei$1@sea.gmane.org>

Try the enclosed spec file.  Also, you can use the one from Fedora devel (I
think the one I enclosed is the same).
-------------- next part --------------
An embedded and charset-unspecified text was scrubbed...
Name: numpy.spec
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20061108/8f7ddced/attachment.ksh>
-------------- next part --------------
-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
-------------- next part --------------
_______________________________________________
Numpy-discussion mailing list
Numpy-discussion at lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/numpy-discussion

From david at ar.media.kyoto-u.ac.jp  Wed Nov  8 21:31:07 2006
From: david at ar.media.kyoto-u.ac.jp (David Cournapeau)
Date: Thu, 09 Nov 2006 11:31:07 +0900
Subject: Passing numpy arrays to matlab
In-Reply-To: <1e2af89e0611071057h70b2f575u3943efbc142ca9c8@mail.gmail.com>
References: <454FEE21.7040205@ar.media.kyoto-u.ac.jp>	<45501396.3070908@astraw.com>	<455012BF.1030105@ar.media.kyoto-u.ac.jp>	<45501E70.8090609@astraw.com>	<45501F38.3090201@ar.media.kyoto-u.ac.jp>
	<1e2af89e0611071057h70b2f575u3943efbc142ca9c8@mail.gmail.com>
Message-ID: <455292EB.4050506@ar.media.kyoto-u.ac.jp>

Matthew Brett wrote:
>
> I would be very happy to help with this.  It would be great if we
> could get a standard well-maintained library of some sort towards
> scipy - we (http://neuroimaging.scipy.org/) have a great deal of
> matlab integration to do.
>   
I am a bit busy and late on my PhD schedule; I will try to release 
something remotely usable by other people within the end of the week, so 
other people can start hacking on it too,

cheers,

David

-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From ylsjeqgh at parrishmed.com  Wed Nov  8 22:44:24 2006
From: ylsjeqgh at parrishmed.com (MONTANA NEBRASKA)
Date: Thu, 9 Nov 2006 04:44:24 +0100
Subject: reason
Message-ID: <000d01c703b1$59511810$3115a555@petter>


Spam postage charges or strings attached Whyi!
Drugs often in unable even to once. Visitor Hereplain simple.
Threads Posts am Members or Welcome newest member spacerocks.
Updated program French lot called mtm available address.
Tsig Vermonter worzel Statistics Threads Posts Members Welcome newest.
Even to or once though ghetto popular among younger affluent.
Apply Name Title City Alabama Alaska Arizona Arkansas or. Put thisin interview fired back article such behavior.
Chamber guarded by grinning brutal orcsin adventure you a.
Anything site is Regular shaped a cookies soft or chocolate. If visit of sure check faq.
Serenitude or Against Mainstream. Approver rss dc Free in?
Often unable even to.
Wont stopjones action of! Nice in still havent or.
Often unable even to. Equipped developed opensource understand in reason anonymity.
Hardware in Benefits Feeds Subscribe Events in or Tech am.
Support Billing amp Account am Features.
Import url Click here or sign?
Omissions of page may reported Copyright copy Mobygames all. Discuss stories posted Today than a!
Drugs often in unable even to once.
Courtyards is in Tower am Cirith Ungol horrifying torture chamber. Members List in Calendar.
Credits There in file of release game Everything usersadd Errors.
Members List in Calendar.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20061109/5afcc4ab/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: give.gif
Type: image/gif
Size: 10326 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20061109/5afcc4ab/attachment.gif>
-------------- next part --------------
-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
-------------- next part --------------
_______________________________________________
Numpy-discussion mailing list
Numpy-discussion at lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/numpy-discussion

From pfdubois at gmail.com  Wed Nov  8 23:00:58 2006
From: pfdubois at gmail.com (Paul Dubois)
Date: Wed, 8 Nov 2006 20:00:58 -0800
Subject: A reimplementation of MaskedArray
In-Reply-To: <200611081608.25872.pgmdevlist@gmail.com>
References: <200610160232.20888.pgmdevlist@mailcan.com>
	<200611080510.13565.pgmdevlist@gmail.com>
	<ce557a360611081041t6748bcdj953aa6aaea9cab18@mail.gmail.com>
	<200611081608.25872.pgmdevlist@gmail.com>
Message-ID: <f74a6c2f0611082000k31f8b596r18e635d687e4866d@mail.gmail.com>

2 cents from the author of the first folio:

The intent was to allow creation of masked arrays with copy=no, so that the
original data could be retrieved from it if desired. But I was quite, quite
rigorous about NEVER assuming the data in a masked slot made any sense
whatsoever.

The intention was that there are two ways to get a numeric array out of a
masked one:

1. Get the data field
2. m.filled()

(1) is strictly at your own risk.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20061108/0932feb3/attachment.html>
-------------- next part --------------
-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
-------------- next part --------------
_______________________________________________
Numpy-discussion mailing list
Numpy-discussion at lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/numpy-discussion

From josh.p.marshall at gmail.com  Wed Nov  8 23:18:54 2006
From: josh.p.marshall at gmail.com (Josh Marshall)
Date: Thu, 9 Nov 2006 15:18:54 +1100
Subject: Passing numpy arrays to matlab
In-Reply-To: <mailman.126689.1162987742.22397.numpy-discussion@lists.sourceforge.net>
References: <mailman.126689.1162987742.22397.numpy-discussion@lists.sourceforge.net>
Message-ID: <8B4A9501-180E-4714-85F9-AB483B59099B@gmail.com>

Hi David,

Sorry for the late reply. Can you CC any reply to me as well, as I  
just get the digests and read them every few days.

On 08/11/2006, at 11:09 PM, David Cournapeau wrote:

> I didn't know that, thanks. Unfortunately, it is not really what I am
> trying to do: mlabwrap is just a python interface a bit more high  
> level
> than pymat, with many fancy tricks, but still do copies. What I would
> like is to avoid completely the copying by using proxy classes around
> data from numpy so that I can pass "automatically" numpy arrays to
> matlab C api, and a proxy class around data from matlab so that they
> look like numpy arrays.

I don't see how you are going to get around doing the copies. Matlab  
is in a separate process from the Python interpreter, and there is no  
shared memory. In what way do you want these proxy classes to "look  
like numpy arrays"?

Note that mlabwrap creates proxy arrays, and only copies the data if  
you actually request it to. (AFAIRemember) Otherwise you aren't  
losing any speed, because there aren't going to be any copies.

>     I don't care that much about the actual api from python point of
> view, because I intend to use this mainly to compare matlab vs numpy
> implementation, not as a way to use matlab inside python regularly.  
> And
> once the copy problem is solved, adding syntactic sugar using  
> python is
> easy anyway, I think (it should be easy to do something similar to
> mlabwrap at that point),

What could be possible to do is add an array interface to the  
mlabwrap proxy classes so they can be used as numpy arrays when  
required for passing to numpy functions (or PIL, etc). Thus we only  
copy when we want to use numpy functions. Then we could define the  
operators on the proxy class to perform their operations on the other  
side of the bridge.

Cheers,
Josh


-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 2433 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20061109/fb34db35/attachment.bin>
-------------- next part --------------
-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
-------------- next part --------------
_______________________________________________
Numpy-discussion mailing list
Numpy-discussion at lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/numpy-discussion

From tim.hochberg at ieee.org  Wed Nov  8 14:24:34 2006
From: tim.hochberg at ieee.org (Tim Hochberg)
Date: Wed, 08 Nov 2006 12:24:34 -0700
Subject: A reimplementation of MaskedArray
In-Reply-To: <ce557a360611081041t6748bcdj953aa6aaea9cab18@mail.gmail.com>
References: <200610160232.20888.pgmdevlist@mailcan.com>	<200610241408.40415.pgmdevlist@gmail.com>	<16761e100611071711q6b9cc464p85708d7a642b9a8a@mail.gmail.com>	<200611080510.13565.pgmdevlist@gmail.com>
	<ce557a360611081041t6748bcdj953aa6aaea9cab18@mail.gmail.com>
Message-ID: <45522EF2.8000603@ieee.org>

A. M. Archibald wrote:
> On 08/11/06, Pierre GM <pgmdevlist at gmail.com> wrote:
>
>   
>> I like your idea, but not its implementation. If MA.masked_singleton is
>> defined as an object, as you suggest, then the dtype of the ndarray it is
>> passed to becomes 'object', as you pointed out, and that is not something one
>> would naturally expec, as basic numerical functions don't  work well  with the
>> 'object' dtype (just try  N.sqrt(N.array([1],dtype=N.object)) to see what I
>> mean).
>> Even if we can construct a mask rather easily at the creation of the masked
>> array, following your 'a==masked' suggestion, we still need to get the dtype
>> of the non-masked section, and that doesn't seem trivial...
>>     
>
> A good candidate for "should be masked" marked is NaN. It is supposed
> to mean, more or less, "no sensible value". Unfortunately, integer
> types do not have such a special value. It's also conceivable that
> some user might want to keep NaNs in their array separate from the
> mask. Finally, on some hardware, operations with NaN are very slow (so
> leaving them in the array, even masked, might not be a good idea).
>   
It has always been my experience (on various flavors or Pentium) that 
operating on NANs is extremely slow. Does anyone know on what hardware 
NANs are *not* slow? Of course it's always possible I just never notice 
NANs on hardware where they aren't slow.


[SNIP]

-tim


-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From David.L.Goldsmith at noaa.gov  Wed Nov  8 22:06:21 2006
From: David.L.Goldsmith at noaa.gov (David L Goldsmith)
Date: Wed, 08 Nov 2006 19:06:21 -0800
Subject: Help, me emails aren't getting through!
Message-ID: <45529B2D.2010505@noaa.gov>

If the subject line is true, no one will see this; if it's false, please 
ignore! :-)

DG
-- 
HMRD/ORR/NOS/NOAA <http://response.restoration.noaa.gov/emergencyresponse/>

-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From David.L.Goldsmith at noaa.gov  Wed Nov  8 19:08:17 2006
From: David.L.Goldsmith at noaa.gov (David L Goldsmith)
Date: Wed, 08 Nov 2006 16:08:17 -0800
Subject: Anyone have a "little" shooting-method function
	to share
Message-ID: <45527171.8030903@noaa.gov>

Hi!  I tried to send this earlier: it made it into my sent mail folder, 
but does not appear to have made it to the list.

I need to numerically solve:
    (1-t)x" + x' - x = f(t), x(0) = x0, x(1) = x1
I've been trying to use (because it's the approach I inherited) an 
elementary finite-difference discretization, but unit tests have shown 
that that approach isn't working.  After a little review, I believe I 
understand the problem: near t=0, the thing is like an 
advection-diffusion equation with diffusion as strong as advection (a 
case I'm sure is treated somewhere in the literature, but not as easily 
findable as:) near t=1, the thing is like the advection-diffusion 
equation I did easily find treated, namely one where advection 
dominates.  Based on that treatment, I understand why an FD approach 
(with a uniform grid) would fail around t=0, and from there it is easy 
to accept that the fact that the thing changes its "nature" over the 
course of its "life," implies that (a uniform grid) FD is probably not a 
very good approach for this equation anywhere on its domain.  I could 
try (and maybe will have to) a variable grid FD approach, but I'd first 
like to try a "shooting" method (for some reason my intuition is telling 
me that in this case this might be more efficient).  This is where you 
all come in: I understand the algorithm and could (may have to) code it 
myself, but if anyone out there already has code for this (in Python 
using numpy, and preferably already under test), might you be willing to 
share?  Thanks in advance,

David Goldsmith

PS: In the interim, I realized that if something like this "pre-exists" 
in a module, that module might be scipy.  Sure enough, scipy has 
integrate.ode and integrate.odeint; Google-ing scipy: integrate.odeint 
help led me to Travis's 2004 "SciPy Tutorial," where I find "There are 
many optional inputs and outputs available when using odeint which can 
help tune the solver.  These additional inputs and outputs are not 
needed much of the time, however..."  Well, is one of those inputs an 
algorithm specifier?  Alternatively, is there a "shooting" algorithm 
implemented elsewhere in scipy?


-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From David.L.Goldsmith at noaa.gov  Wed Nov  8 16:49:00 2006
From: David.L.Goldsmith at noaa.gov (David L Goldsmith)
Date: Wed, 08 Nov 2006 13:49:00 -0800
Subject: Anyone have a "little" shooting-method function
	to share
Message-ID: <455250CC.4040608@noaa.gov>

Hi!  I need to numerically solve:
     (1-t)x" + x' - x = f(t), x(0) = x0, x(1) = x1
I've been trying to use (because it's the approach I inherited) an 
elementary finite-difference discretization, but unit tests have shown 
that that approach isn't working.  After a little review, I believe I 
understand the problem: near t=0, the thing is like an 
advection-diffusion equation with diffusion as strong as advection (a 
case I'm sure is treated somewhere in the literature, but not as easily 
findable as:) near t=1, the thing is like the advection-diffusion 
equation I did easily find treated, namely one where advection 
dominates.  Based on that treatment, I understand why an FD approach 
(with a uniform grid) would fail around t=0, and from there it is easy 
to accept that the fact that the thing changes its "nature" over the 
course of its "life," implies that (a uniform grid) FD is probably not a 
very good approach for this equation anywhere on its domain.  I could 
try (and maybe will have to) a variable grid FD approach, but I'd first 
like to try a "shooting" method (for some reason my intuition is telling 
me that in this case this might be more efficient).  This is where you 
all come in: I understand the algorithm and could (may have to) code it 
myself, but if anyone out there already has code for this (in Python 
using numpy, and preferably already under test), might you be willing to 
share?  Thanks in advance,

David Goldsmith
-- 
HMRD/ORR/NOS/NOAA <http://response.restoration.noaa.gov/emergencyresponse/>

-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From david at ar.media.kyoto-u.ac.jp  Thu Nov  9 01:32:04 2006
From: david at ar.media.kyoto-u.ac.jp (David Cournapeau)
Date: Thu, 09 Nov 2006 15:32:04 +0900
Subject: Passing numpy arrays to matlab
In-Reply-To: <8B4A9501-180E-4714-85F9-AB483B59099B@gmail.com>
References: <mailman.126689.1162987742.22397.numpy-discussion@lists.sourceforge.net>
	<8B4A9501-180E-4714-85F9-AB483B59099B@gmail.com>
Message-ID: <4552CB64.2080604@ar.media.kyoto-u.ac.jp>

Josh Marshall wrote:
>
>
> I don't see how you are going to get around doing the copies. Matlab 
> is in a separate process from the Python interpreter, and there is no 
> shared memory. In what way do you want these proxy classes to "look 
> like numpy arrays"?
I am not talking about the copy in the matlab <-> python interaction. 
This is done through pipe, handled by the OS; I don't know the details, 
but I know that communication through pipe is quite fast under linux 
(see below), and is not the bottleneck.

>
> Note that mlabwrap creates proxy arrays, and only copies the data if 
> you actually request it to. (AFAIRemember) Otherwise you aren't losing 
> any speed, because there aren't going to be any copies.
There may be no copy for returned data you don't need, but that's not 
the case I am talking about. For all other cases, I don't think this is 
what's happening: if you take a look at mlabwrap, in the C mlabraw 
module, the function mlabraw_put always calls numeric2mx for arrays, 
which itself always calls makeMxFromNumeric, which makes a copy. Same in 
the other direction once you call mlabwrap_get. I am doing the same in 
my module, because that's the simplest thing to do.

The problem is that when you are using the function engPutVariable of 
the matlab engine API, you need to give a pointer to a mxArray 
structure, which is the C representation of a matlab array. You cannot 
say (this is one of the brain damaged thing of matlab C api I was 
talking in an other mail): build a mxArray from existing data: this is 
the copy I am talking about, and this is one expensive. In the best case 
(real numpy arrays with fortran storage), you can do a memcpy, but in 
most cases, you need to do something which takes strides into account 
(because complex matlab arrays are actually not fortran, or because by 
default, most numpy arrays are C storage, and this makes a difference 
for rank >= 2), which implies non-contiguous memory access, which is 
*really* expensive (around 2 cycles/byte at best, on my bi Xeon 3.2 Ghz).

Basically, if you want to do something like calling the resample 
function of matlab on an numpy array and using the result later in 
numpy, here is what's happening right now:
   
    1 copy numpy (or numarray in the case of mlabwrap, but this should 
not matter, I guess) data into an mxArray
    2 send the mxArray to matlab engine: done with pipe (imply copy ? At 
least, it is contiguous array copy)
    3 compute the thing into matlab
    4 send the result to python mxArray
    5 copy the data of the mxArray to numpy array

A quick profiling show that if you don't do any processing in matlab, 
just sending and getting an array back, 1 and 5 takes roughly 80-90 % of 
the time in my implementation (which is faster than mlabwrap, but I 
think this is just caused by the much fancier API of mlabwrap, ie the 
core mecanism to pass arrays should be roughly the same, as mlabwrap 
uses the C function makeMxFromNumeric, and I am using a similar function 
myself through ctypes), the 10-20% are used for the communication 
through the pipe. I believe that most typical usage cases involve 1 and 5.

5 should be avoidable in many cases if I know how to build a proxy class 
around the mxArray so that the the proxy behaves as a numpy array, with 
the buffer owned by the mxArray; but I don't know how to do that 
(particularly, how to handle the destruction of data, as the proxy 
should destroy the mxArray once the proxy  object is garbage collected). 
1 would be easy if the C matlab API was sane, which is not the case; 
they give functions which are impossible to use correctly (mxSetPr and 
mxSetData).
>
> What could be possible to do is add an array interface to the mlabwrap 
> proxy classes so they can be used as numpy arrays when required for 
> passing to numpy functions (or PIL, etc). Thus we only copy when we 
> want to use numpy functions. Then we could define the operators on the 
> proxy class to perform their operations on the other side of the bridge.
Yes, that's what I want to do, and in theory, this should be possible 
without copy; my initial question in the beginning of the thread is how 
to build a numpy proxy class from existing buffer of data, with the 
proxy becoming the owner of the data (ie should do all the deallocation, 
including here cleaning mxArray structures).

cheers,

David

-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From peridot.faceted at gmail.com  Thu Nov  9 02:27:38 2006
From: peridot.faceted at gmail.com (A. M. Archibald)
Date: Thu, 9 Nov 2006 02:27:38 -0500
Subject: A reimplementation of MaskedArray
In-Reply-To: <45522EF2.8000603@ieee.org>
References: <200610160232.20888.pgmdevlist@mailcan.com>
	<200610241408.40415.pgmdevlist@gmail.com>
	<16761e100611071711q6b9cc464p85708d7a642b9a8a@mail.gmail.com>
	<200611080510.13565.pgmdevlist@gmail.com>
	<ce557a360611081041t6748bcdj953aa6aaea9cab18@mail.gmail.com>
	<45522EF2.8000603@ieee.org>
Message-ID: <ce557a360611082327i6560c75dy18bcb2bdc03833f2@mail.gmail.com>

On 08/11/06, Tim Hochberg <tim.hochberg at ieee.org> wrote:

> It has always been my experience (on various flavors or Pentium) that
> operating on NANs is extremely slow. Does anyone know on what hardware
> NANs are *not* slow? Of course it's always possible I just never notice
> NANs on hardware where they aren't slow.

On an opteron machine I have access to, they appear to be no slower
(and even faster for some transcendental functions) than ordinary
floats:

In [13]: a=zeros(1000000)

In [14]: %time for i in xrange(1000): a += 1.1
CPU times: user 6.87 s, sys: 0.00 s, total: 6.87 s
Wall time: 6.87

In [15]: a *= NaN

In [16]: %time for i in xrange(1000): a += 1.1
CPU times: user 6.86 s, sys: 0.00 s, total: 6.86 s
Wall time: 6.87

On my Pentium M, they are indeed significantly slower (three times? I
didn't really do enough testing to say how much). I am actually rather
offended by this unfair discrimination against a citizen in good
standing of the IEEE floating point community.

A. M. Archibald

-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From a.u.r.e.l.i.a.n at gmx.net  Thu Nov  9 03:00:48 2006
From: a.u.r.e.l.i.a.n at gmx.net (Johannes Loehnert)
Date: Thu, 9 Nov 2006 09:00:48 +0100
Subject: matrix multiplication (newbie question)
In-Reply-To: <45525363.1030404@gmx.net>
References: <20061108135417.6906.qmail@web50913.mail.yahoo.com>
	<200611081716.18422.a.u.r.e.l.i.a.n@gmx.net>
	<45525363.1030404@gmx.net>
Message-ID: <200611090900.48824.a.u.r.e.l.i.a.n@gmx.net>


> Well what about A*(B*C)?

Point taken...

Johannes

-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From cjw at sympatico.ca  Thu Nov  9 03:18:57 2006
From: cjw at sympatico.ca (Colin J. Williams)
Date: Thu, 09 Nov 2006 03:18:57 -0500
Subject: Grief with a complex value
Message-ID: <4552E471.2020500@sympatico.ca>


 >>> import numpy.core as _n
 >>> _nt= _n.numerictypes
 >>> value= 
'\x00\x00\x00\x00\x00\x00\xf0?\x00\x00\x00\x00\x00\x00\x00@\x00\x00\x00\x00\x00\x00\x08@\x00\x00\x00\x00\x00\x00\x10@\x00\x00\x00\x00\x00\x00\x14@\x00\x00\x00\x00\x00\x00\x18@'
 >>> _n.array(value, dtype= _nt.complex128, copy=True)
Traceback (most recent call last):
  File "<interactive input>", line 1, in <module>
TypeError: a float is required
 >>>

A bug?

Colin W.

-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From robert.kern at gmail.com  Thu Nov  9 03:34:44 2006
From: robert.kern at gmail.com (Robert Kern)
Date: Thu, 09 Nov 2006 02:34:44 -0600
Subject: Grief with a complex value
In-Reply-To: <4552E471.2020500@sympatico.ca>
References: <4552E471.2020500@sympatico.ca>
Message-ID: <eiup7d$c6k$1@sea.gmane.org>

Colin J. Williams wrote:
>  >>> import numpy.core as _n
>  >>> _nt= _n.numerictypes
>  >>> value= 
> '\x00\x00\x00\x00\x00\x00\xf0?\x00\x00\x00\x00\x00\x00\x00@\x00\x00\x00\x00\x00\x00\x08@\x00\x00\x00\x00\x00\x00\x10@\x00\x00\x00\x00\x00\x00\x14@\x00\x00\x00\x00\x00\x00\x18@'
>  >>> _n.array(value, dtype= _nt.complex128, copy=True)
> Traceback (most recent call last):
>   File "<interactive input>", line 1, in <module>
> TypeError: a float is required
>  >>>
> 
> A bug?

No. array() takes sequences, not data buffers. You want frombuffer().

-- 
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless enigma
 that is made terrible by our own mad attempt to interpret it as though it had
 an underlying truth."
  -- Umberto Eco


-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From fperez.net at gmail.com  Thu Nov  9 03:40:33 2006
From: fperez.net at gmail.com (Fernando Perez)
Date: Thu, 9 Nov 2006 01:40:33 -0700
Subject: Int32s, scalar operations and Pyhon longs
Message-ID: <db6b5ecc0611090040m570b634fl312b21222b4fe108@mail.gmail.com>

I understand why this happens, but I wonder if it should be in any way
'fixed' (if that is even feasible without introducing other problems):

In [28]: x = 999999

In [29]: y = numpy.array([x])

In [30]: z = y[0]

In [31]: x==z
Out[31]: True

In [32]: x
Out[32]: 999999

In [33]: z
Out[33]: 999999

In [34]: x*x
Out[34]: 999998000001L

In [35]: z*z
Warning: overflow encountered in long_scalars
Out[35]: -729379967

I am sure it will be, to say the least, pretty surprising (and I can
imagine subtle bugs being caused by this).

Regards,

f

-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From cjw at sympatico.ca  Thu Nov  9 03:54:05 2006
From: cjw at sympatico.ca (Colin J. Williams)
Date: Thu, 09 Nov 2006 03:54:05 -0500
Subject: Grief with a complex value
In-Reply-To: <eiup7d$c6k$1@sea.gmane.org>
References: <4552E471.2020500@sympatico.ca> <eiup7d$c6k$1@sea.gmane.org>
Message-ID: <4552ECAD.6090005@sympatico.ca>



Robert Kern wrote:
> Colin J. Williams wrote:
>   
>>  >>> import numpy.core as _n
>>  >>> _nt= _n.numerictypes
>>  >>> value= 
>> '\x00\x00\x00\x00\x00\x00\xf0?\x00\x00\x00\x00\x00\x00\x00@\x00\x00\x00\x00\x00\x00\x08@\x00\x00\x00\x00\x00\x00\x10@\x00\x00\x00\x00\x00\x00\x14@\x00\x00\x00\x00\x00\x00\x18@'
>>  >>> _n.array(value, dtype= _nt.complex128, copy=True)
>> Traceback (most recent call last):
>>   File "<interactive input>", line 1, in <module>
>> TypeError: a float is required
>>  >>>
>>
>> A bug?
>>     
>
> No. array() takes sequences, not data buffers. You want frombuffer().
>   
Thanks.

Colin W.

-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From stefan at sun.ac.za  Thu Nov  9 03:56:38 2006
From: stefan at sun.ac.za (Stefan van der Walt)
Date: Thu, 9 Nov 2006 10:56:38 +0200
Subject: Grief with a complex value
In-Reply-To: <4552E471.2020500@sympatico.ca>
References: <4552E471.2020500@sympatico.ca>
Message-ID: <20061109085638.GA25499@mentat.za.net>

On Thu, Nov 09, 2006 at 03:18:57AM -0500, Colin J. Williams wrote:
> 
>  >>> import numpy.core as _n
>  >>> _nt= _n.numerictypes
>  >>> value= 
> '\x00\x00\x00\x00\x00\x00\xf0?\x00\x00\x00\x00\x00\x00\x00@\x00\x00\x00\x00\x00\x00\x08@\x00\x00\x00\x00\x00\x00\x10@\x00\x00\x00\x00\x00\x00\x14@\x00\x00\x00\x00\x00\x00\x18@'
>  >>> _n.array(value, dtype= _nt.complex128, copy=True)
> Traceback (most recent call last):
>   File "<interactive input>", line 1, in <module>
> TypeError: a float is required
>  >>>

If I understand correctly, you would like to create an array from a
buffer:

In [12]: N.frombuffer(value,dtype=N.complex128)
Out[12]: array([ 1.+2.j,  3.+4.j,  5.+6.j])

Cheers
St?fan

-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From Mailer-Daemon at lists.sourceforge.net  Thu Nov  9 04:29:15 2006
From: Mailer-Daemon at lists.sourceforge.net (Mail Delivery System)
Date: Thu, 09 Nov 2006 01:29:15 -0800
Subject: Mail delivery failed: returning message to sender
Message-ID: <E1Gi6Dj-0000On-GV@externalmx-1.sourceforge.net>

This message was created automatically by mail delivery software.

A message that you sent could not be delivered to one or more of its
recipients. This is a permanent error. The following address(es) failed:

  numpy-discussion at lists.sourceforge.net
    SMTP error from remote mailer after RCPT TO:<numpy-discussion at lists.sourceforge.net>:
    host mail.sourceforge.net [66.35.250.206]: 451-Could not complete sender verify callout
    451-Could not complete sender verify callout for
    451-<numpy-discussion at lists.sourceforge.net>.
    451-The mail server(s) for the domain may be temporarily unreachable, or
    451-they may be permanently unreachable from this server. In the latter case,
    451-you need to change the address or create an MX record for its domain
    451-if it is : retry timeout exceeded

------ This is a copy of the message, including all the headers. ------
------ The body of the message is 56616 characters long; only the first
------ 16384 or so are included here.


From xswbrx at kpslm.co.uk  Sun Nov  5 02:01:50 2006
From: xswbrx at kpslm.co.uk (Jay Porter)
Date: Sun, 05 Nov 2006 16:01:50 +0900
Subject: faces and their former position.  To be, sure.  But question
Message-ID: <000901c700a8$3bc01be0$01bfaa58@tldaduattuwq>

An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20061105/d4cb1742/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: eodabc.gif
Type: image/gif
Size: 11480 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20061105/d4cb1742/attachment.gif>

From johanna at chez.com  Thu Nov  9 09:07:08 2006
From: johanna at chez.com (=?windows-1251?B?0/fo7CDP6+Dt6PDu4uDy/A==?=)
Date: Thu, 09 Nov 2006 17:07:08 +0300
Subject: =?windows-1251?b?0/fl4e376SDq8/DxOqvB6Oft5fEg?=
 =?windows-1251?b?7+vg7Swg0t3OLCCWIPDg5/Dg4e7y6uAg6CDg7eDr6Oe7Lg==?=
Message-ID: <de7b01c70408$32a3a4fa$636693e0@chez.com>

  ???????????? ???????. ?????????? ?????? ? ?????, ???.

 

 

???? ????????!!! ????? ??????:

??? ?????????? ? ???????????? ?????, ??? 2 ??? ? ???? ??????????, ?????
????????????? (?????????? ? ?????????? ????????????), ??? ??? ??? ?? ?????????
?????????? ??????????? ????????????? ????????, ????????? ?????? ? ??? ???????????
??? ?????????? ???????,  ?????????? ?? ?? ?????????? ?????????? ??????????????????
??????????? ??????????. ??? ???????? ????, ? ?????? ????. ?? ?????????
????? ??? ???????? ????????????, ???????????? ?????????????????? ???????
?????????, ??? ?????????? ?????? ?????? ? ?????????????? ????????, ?? ??????????
?? ???????????, ????????????? ??????? ?? ???? ????????? ? ????????? ?????
???????? (??? ?????? ???????? ????) - ??? ???! ??????! ?? ?????? ??????????????!

 
??????? ?????????? ? ????????:

 

???? ?????????? ????????: ? I? ?? I4 ?????? 2006 ?.



 
????????? ??????????:

 

?????????? ?????????? ????????????? ???????? ????? ???????? ?? ???????????
????????? ? ?????????????? ?????????? (?????????????? ????) ?LOGICAL -
J 3000? ? ???????? ????????????????. 

?????????? ???????????? ????????? ? ???????????? ? ??????????? ???????????
??????????? ???????? ?????? ??????? ????????? ???????? ? ????? ??????????????
????????????, ???????? ???????? ???????????????? ?????? ? ?????????? ?????????????
??????, ??????? ??????? ???????? ???? ??????? ????????????, ???????????
????????? ???????????????? ???????????? ? ??????????? ?? ?????? ????????????
?????????.

 

??????? ????????? ????????: 

??? ?????????? ???????? ?????? ? ????????? ???????? ???????????? ? ????????
?????????? ????? ?? ??????????? ??????????????? ????????? ????????? ????????????
????????? ? ?????????:
 ?????????????????? ?????????- ???????????????? ?????????? ????????????!
??????????? ??????? ???????? ?? ?????? ????????????? ?????????? ????????????
?? ???????????????? ??????, ? ????? ?????? ??????????? ???????????? ???????????
??? ???????????? ???????? ??????? ???????????. ????? - ??????????? ??????
????? - ???????????? ????????????. ? ????????? ? ?????? ???????????? ?????
?????? ?????????????????? ????????? ?? ???????? ?????????? ?????? ??????. (????????
?????????? ????????????? ????????????. ???????? ?????????? ????????? -
???????, ????????????, ???????????????? ?????????? ????????????? ????????????
??? ????????????? ? ????????? ?????????, ??? ? ??????????, ??? ? ? ?????????
??????? ??????.). ???????????? ???????? ???????? (????????????? ????? ?????????)
??? ??????????????? ?????????? ??????????? ?????? ?????. ??????????? ??????
(?? CD) ????????????? ????????? ??? ??????????????? ?????????? ??????????
????????.  



??????? ???????????? ??:

????????????? ???????????, ????????????? ???????-????????????? ?????, ????????????
?? ?????????? ?????????, ?????????????, ??????????, ???????????? ?????????
??????????? ? ??????? ???????????????? ??????????. ????????????, ??????????????
?????????? ? ???????? ???????, ????????? ? ???????????? ? ?????????????
????????????? ???????????, ???????????????, ??????? ?????????, ???????????
?????????? ? ??????? ?????????? ????????????, ?????????? ?????????????
???????????? ???????????? ???????????.

 

? ?????????? ????????? ???????? ???????? ?????????? ????????? ?????? ?
??????: 
 ?????? ????????????? ? ????????? ??????????????? ???????, ??????-?????,
??? ????????? ?????????? ? ??????? ???????, ? ????? ???????? ??????????????
????????????? ?????? ????????????;  ??? ???????? ??????????? ???????????,
????????????? ? ?????????? ??????? ? ???????? ?? ??????????? ???????????;
??? ?????????? ?????????? ??????? ?? ???????? ?? ????????? ? ?????????????,
????????????? ???????????, ?? ???????????; ???????? ?????? ?????? ??, ???????????
?? ??????????? ???????????;  ??????????? ??????? ????????????? ????????
(??? ?????? ????????? ???????????);  ???????? ?????? ????????;  ????? ???????
???????? ? ???????? ? ????????????? ???????????;  ??? ????????? ???????????
???????????? (?????????) ????? ???????; ??? ????????? ??????????? ??????????
????? ????????????;  ??? ????????? ???????? ? ??????????? ????????;  ???
???????????? ????????? ??????????? ???????;   
 

? ???????????? ????????:
 ????????????? ? ?????????? ????????? ????????? ??????? ???????????;  
??? ????????? ??????????? ???????? ?? ????????? ???????????? ?????? ? ???????????
??????? ??????? ????????????;  ?????????? ??????????? ???????? ? ?????????????
??????????? ?? ??????????? (????? ???????); ????????? ?????????? ????????????
(?????????) ????? ??????????? ?????????;  ?????? ?????????? ??????????
? ?? ??????? ???????????; ????? ?????????? ??? ????????????? ????????????; 
????????? ?????????? ??? ????????????? ???????????? - ?????????? ???????
??????????, ???????, ??? ????? ?????????????? ???????????? ? ???????????
????????? ? ????????????? ???????????? ?????????? ? ????????? ?????? ?????,
??????????????? ???????;  ?????????? ?????? ??????????? ???????, ? ?????????
????????????. ?????????? ?????????? ????? ????????????:  ????? ? ??????????
? ????????;  ??? ?????????????? ??????????? ?????????????? ???????????
???????;  ????? ???????? ? ?? ???????? ? ?????????? ?????;  ????????????
?????????? ??????????? ? ?? ????????????? ? ?????? ?????????? ?????????
????????;  ?????? ???????? ???????? ??????: ??????????? ???????? ??????????
??? ?????? ??????? -  ??????????? ???????? ? ????????? ??????, ?? ????????????
?? ??????? ???:    ????? ??????????? ???????????  ???????? ?? ???????????
????????????.    ?????????? ?????? ?????????????? ??????.  ?????????? ???????????
????????? ??????????????: ??????? ????????, ??????????? ???????? ???????????. 
???????????? ?????????? ????????????? ??????? ?? ?????? ??????? (?????,
?????)  ??????????? ?????? ??????????: ???????? ??????????, ???????????????
?????????? ???????????? ??????????? (??????????????, ????????????) ? ??
????????????? ??? ????? ??????? ?????????? ???????????.  ???????? ???????????
?????? ?????? ? ???????? ? ???????, ???????????? ?????? ? ?????????? ?????????
+ ????????????? ??????.  ?????????? ? ?????????? ??????? ??????? ????????
? ???????? ?? ?????????? ?????????? ???????.  ???????????? ?????? ????????????????
?????????????? ?????? ? ?????????????? ??????? ? ???????????? ?????????
?????????????.  ??????? ???????? ?????? ?????? ????????? ????????? ? ?????
?????? ??????????? ?? ??????????? ????????? ???????????? ??? ?????? ? ??????????
?????????????.  


?? ???????? ??????????? ??????????? ?? ???:  (495) 235-58 88, 235-12-35


????




 
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20061109/c0ed5357/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: fsxbl.jpg
Type: image/jpeg
Size: 6148 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20061109/c0ed5357/attachment.jpg>

From aisaac at american.edu  Thu Nov  9 11:43:22 2006
From: aisaac at american.edu (Alan G Isaac)
Date: Thu, 9 Nov 2006 11:43:22 -0500
Subject: slice assignment: strange behaviour
In-Reply-To: <1163025393.211885.161470@e3g2000cwe.googlegroups.com>
References: <1163025393.211885.161470@e3g2000cwe.googlegroups.com>
Message-ID: <Mahogany-0.67.0-660-20061109-114322.00@american.edu>

On Wed, 08 Nov 2006, koara apparently wrote: 
> 'mat' is a numpy.array with shape=(22973, 1009), 
> 'vec' is a numpy.array with shape=(22973,), both of type int: 
> for i in xrange(1009): 
>     ... 
>     fr = vec[10001] 
>     mat[:, i] = vec # assign whole column 
>     if mat[10001, i] != fr: 
>         print "how come?" 
>      ... 
> for elements beyond index 10000, nothing is assigned (ie, 
> numpy.sum(mat[row, :]) is zero for any row > 10000). 

Using numpy 1.0 and trying the code below,
I do not see the problem.

import numpy
from numpy import random
m = random.randint(0,1000,(22973,1009))
v =  random.randint(0,1000,(22973,))
fr = v[10001]
for i in xrange(1009):
    m[:,i] = v
    if m[10001, i] != fr:
        print m[10001,i], fr

Upgrading to numpy 1.0 should be easy enough and may fix 
your problem.

Cheers,
Alan Isaac





-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From robert.kern at gmail.com  Thu Nov  9 11:43:45 2006
From: robert.kern at gmail.com (Robert Kern)
Date: Thu, 09 Nov 2006 10:43:45 -0600
Subject: Int32s, scalar operations and Pyhon longs
In-Reply-To: <db6b5ecc0611090040m570b634fl312b21222b4fe108@mail.gmail.com>
References: <db6b5ecc0611090040m570b634fl312b21222b4fe108@mail.gmail.com>
Message-ID: <eivlsb$mco$1@sea.gmane.org>

Fernando Perez wrote:
> I understand why this happens, but I wonder if it should be in any way
> 'fixed' (if that is even feasible without introducing other problems):
> 
> In [28]: x = 999999
> 
> In [29]: y = numpy.array([x])
> 
> In [30]: z = y[0]
> 
> In [31]: x==z
> Out[31]: True
> 
> In [32]: x
> Out[32]: 999999
> 
> In [33]: z
> Out[33]: 999999
> 
> In [34]: x*x
> Out[34]: 999998000001L
> 
> In [35]: z*z
> Warning: overflow encountered in long_scalars
> Out[35]: -729379967
> 
> I am sure it will be, to say the least, pretty surprising (and I can
> imagine subtle bugs being caused by this).

I think we decided long ago that an int32 really is an array of 32-bit integers
and behaves like one. That's precisely why one uses int32 arrays rather than
object arrays. There are algorithms that do need the wraparound, and the Python
int behavior is always available through object arrays.

-- 
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless enigma
 that is made terrible by our own mad attempt to interpret it as though it had
 an underlying truth."
  -- Umberto Eco


-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From fperez.net at gmail.com  Thu Nov  9 12:07:05 2006
From: fperez.net at gmail.com (Fernando Perez)
Date: Thu, 9 Nov 2006 10:07:05 -0700
Subject: Int32s, scalar operations and Pyhon longs
In-Reply-To: <eivlsb$mco$1@sea.gmane.org>
References: <db6b5ecc0611090040m570b634fl312b21222b4fe108@mail.gmail.com>
	<eivlsb$mco$1@sea.gmane.org>
Message-ID: <db6b5ecc0611090907hfd63b41m6f94201aff4591ad@mail.gmail.com>

On 11/9/06, Robert Kern <robert.kern at gmail.com> wrote:
> Fernando Perez wrote:
> > I understand why this happens, but I wonder if it should be in any way
> > 'fixed' (if that is even feasible without introducing other problems):

[...]

> > I am sure it will be, to say the least, pretty surprising (and I can
> > imagine subtle bugs being caused by this).
>
> I think we decided long ago that an int32 really is an array of 32-bit integers
> and behaves like one. That's precisely why one uses int32 arrays rather than
> object arrays. There are algorithms that do need the wraparound, and the Python
> int behavior is always available through object arrays.

Mkay, fine by me.  Novus caveat emptor.

Cheers,

f

-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From pauli.virtanen at iki.fi  Thu Nov  9 12:47:17 2006
From: pauli.virtanen at iki.fi (Pauli Virtanen)
Date: Thu, 09 Nov 2006 19:47:17 +0200
Subject: Anyone have a "little"
	shooting-method	function	to share
In-Reply-To: <45527171.8030903@noaa.gov>
References: <45527171.8030903@noaa.gov>
Message-ID: <1163094437.6376.6.camel@localhost.localdomain>

ke, 2006-11-08 kello 16:08 -0800, David L Goldsmith kirjoitti:
> Hi!  I tried to send this earlier: it made it into my sent mail folder, 
> but does not appear to have made it to the list.
> 
> I need to numerically solve:
>     (1-t)x" + x' - x = f(t), x(0) = x0, x(1) = x1
> I've been trying to use (because it's the approach I inherited) an 
> elementary finite-difference discretization, but unit tests have shown 
> that that approach isn't working.  
[clip]

You could try to use some of the pre-existing www.netlib.org codes for
solving BVPs. This only requires writing a wrapper.

However, I have written a wrapper for the COLNEW code, which is a
finite-difference method written by Ascher & Bader in '87, having an
adaptive mesh selection. You can find it here:

	http://www.iki.fi/pav/bvp

As I understand, COLNEW does not have any special handling for singular
coefficients, but nevertheless it seems to be able to solve your
problem. The code goes as follows:

----------------------------------------------------
import scipy as N
import bvp

u0 = 1
u1 = 2

def f(t):
    return N.sin(2*t)

def fsub(t, z):
    u, du = z
    return N.array([(f(t) + u - du)/(1-t)])

def gsub(z):
    u, du = z
    return N.array([u[0] - u0, u[1] - u1])

tol = [1e-5, 1e-5]
boundary_points = [0, 1]

solution = bvp.colnew.solve(boundary_points, [2], fsub, gsub,
    is_linear=True, tolerances=tol,
    vectorized=True, maximum_mesh_size=300)

import pylab

x = solution.mesh
pylab.plot(x, solution(x)[:,0])
pylab.savefig('solution.png')
----------------------------------------------------

BR,

	Pauli Virtanen

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 189 bytes
Desc: Digitaalisesti allekirjoitettu viestin osa
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20061109/4287a0a4/attachment.sig>
-------------- next part --------------
-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
-------------- next part --------------
_______________________________________________
Numpy-discussion mailing list
Numpy-discussion at lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/numpy-discussion

From tim.hochberg at ieee.org  Thu Nov  9 12:11:37 2006
From: tim.hochberg at ieee.org (Tim Hochberg)
Date: Thu, 09 Nov 2006 10:11:37 -0700
Subject: A reimplementation of MaskedArray
In-Reply-To: <ce557a360611082327i6560c75dy18bcb2bdc03833f2@mail.gmail.com>
References: <200610160232.20888.pgmdevlist@mailcan.com>	<200610241408.40415.pgmdevlist@gmail.com>	<16761e100611071711q6b9cc464p85708d7a642b9a8a@mail.gmail.com>	<200611080510.13565.pgmdevlist@gmail.com>	<ce557a360611081041t6748bcdj953aa6aaea9cab18@mail.gmail.com>	<45522EF2.8000603@ieee.org>
	<ce557a360611082327i6560c75dy18bcb2bdc03833f2@mail.gmail.com>
Message-ID: <45536149.6040409@ieee.org>

A. M. Archibald wrote:
> On 08/11/06, Tim Hochberg <tim.hochberg at ieee.org> wrote:
>
>   
>> It has always been my experience (on various flavors or Pentium) that
>> operating on NANs is extremely slow. Does anyone know on what hardware
>> NANs are *not* slow? Of course it's always possible I just never notice
>> NANs on hardware where they aren't slow.
>>     
>
> On an opteron machine I have access to, they appear to be no slower
> (and even faster for some transcendental functions) than ordinary
> floats:
>
> In [13]: a=zeros(1000000)
>
> In [14]: %time for i in xrange(1000): a += 1.1
> CPU times: user 6.87 s, sys: 0.00 s, total: 6.87 s
> Wall time: 6.87
>
> In [15]: a *= NaN
>
> In [16]: %time for i in xrange(1000): a += 1.1
> CPU times: user 6.86 s, sys: 0.00 s, total: 6.86 s
> Wall time: 6.87
>
> On my Pentium M, they are indeed significantly slower (three times? I
> didn't really do enough testing to say how much). I am actually rather
> offended by this unfair discrimination against a citizen in good
> standing of the IEEE floating point community.
>   
If they're only 3x slower you're doing better than I am. On my core duo 
box they appear to be nearly 20x slower for both addition and 
multiplication. This more or less matches what I recall from earlier boxes.

     >>> Timer("a.prod()", "import numpy as np; a = np.ones(4096); a[0]
    = np.nan").timeit(10000)
    5.6495738983585397
     >>> Timer("a.prod()", "import numpy as np; a =
    np.ones(4096)").timeit(10000)
    0.34439041833525152
     >>> Timer("a.sum()", "import numpy as np; a = np.ones(4096); a[0] =
    np.nan").timeit(10000)
    6.0985655998179027
     >>> Timer("a.sum()", "import numpy as np; a =
    np.ones(4096)").timeit(10000)
    0.32354363473564263

I've been told that operations on NANs are slow because they aren't 
always implemented in the FPU hardware. Instead they are trapped and 
implemented software or firmware or something or other. That may well be 
bogus 42nd hand information though.

-tim


-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From bpederse at gmail.com  Thu Nov  9 14:01:59 2006
From: bpederse at gmail.com (Brent Pedersen)
Date: Thu, 9 Nov 2006 11:01:59 -0800
Subject: filter a recarray
Message-ID: <e183a99d0611091101q5e611202tb41a94427778ab3c@mail.gmail.com>

hi, i have a recarray of  > 60K  records and i'm wondering if there's a
numpy/vectorized way to the following.

get a new array where there will be unique column0 + column1 rows with the
row that remains being chosen because it has the highest value in the last
column. so in the paste below, there are 4
'AT1G01070', 'AT1G11450' rows, but i only want to keep the row:
('AT1G01070', 'AT1G11450', 78.660003662109375, 717, 140, 5, 1129L, 1838L,
1098L, 1808L, 1e-168, 592.0)

because 592.0 is the highest value.  i can do this using a hash and looping,
but i'm guessing there's a more efficient way. any tips?
thanks.
-brent


[  ('AT1G01030', 'AT1G13260', 70.339996337890625, 263, 75, 1, 835L, 1094L,
778L, 1040L, 5.9999999999999998e-38, 158.0)
 ('AT1G01040', 'AT1G01040', 100.0, 8019, 0, 0, 1L, 8019L, 1L, 8019L, 0.0,
12620.0)
 ('AT1G01050', 'AT1G01050', 100.0, 1968, 0, 0, 1L, 1968L, 1L, 1968L, 0.0,
3090.0)
 ('AT1G01060', 'AT1G01060', 100.0, 4175, 0, 0, 1L, 4175L, 1L, 4175L, 0.0,
6355.0)
 ('AT1G01070', 'AT1G01070', 100.0, 2192, 0, 0, 1L, 2192L, 1L, 2192L, 0.0,
3426.0)
 ('AT1G01070', 'AT1G11450', 78.660003662109375, 717, 140, 5, 1129L, 1838L,
1098L, 1808L, 1e-168, 592.0)
 ('AT1G01070', 'AT1G11450', 79.870002746582031, 303, 51, 2, 1L, 293L, 1L,
303L, 1.9999999999999999e-67 , 256.0)
 ('AT1G01070', 'AT1G11450', 88.400001525878906, 181, 21, 0, 587L, 767L,
724L, 904L, 6e-57, 221.0)
 ('AT1G01070', 'AT1G11450', 83.209999084472656, 131, 19, 1, 1875L, 2002L,
1878L, 2008L, 1.9999999999999999e-28 , 126.0)
 ('AT1G01070', 'AT1G11460', 82.919998168945312, 480, 77, 3, 1129L, 1607L,
1216L, 1691L, 6.9999999999999999e-132, 470.0)]
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20061109/5d10a4dc/attachment.html>
-------------- next part --------------
-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
-------------- next part --------------
_______________________________________________
Numpy-discussion mailing list
Numpy-discussion at lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/numpy-discussion

From cjw at sympatico.ca  Thu Nov  9 14:01:15 2006
From: cjw at sympatico.ca (Colin J. Williams)
Date: Thu, 09 Nov 2006 14:01:15 -0500
Subject: Grief with a complex value
In-Reply-To: <20061109085638.GA25499@mentat.za.net>
References: <4552E471.2020500@sympatico.ca>
	<20061109085638.GA25499@mentat.za.net>
Message-ID: <45537AFB.3080709@sympatico.ca>



Stefan van der Walt wrote:
> On Thu, Nov 09, 2006 at 03:18:57AM -0500, Colin J. Williams wrote:
>   
>>  >>> import numpy.core as _n
>>  >>> _nt= _n.numerictypes
>>  >>> value= 
>> '\x00\x00\x00\x00\x00\x00\xf0?\x00\x00\x00\x00\x00\x00\x00@\x00\x00\x00\x00\x00\x00\x08@\x00\x00\x00\x00\x00\x00\x10@\x00\x00\x00\x00\x00\x00\x14@\x00\x00\x00\x00\x00\x00\x18@'
>>  >>> _n.array(value, dtype= _nt.complex128, copy=True)
>> Traceback (most recent call last):
>>   File "<interactive input>", line 1, in <module>
>> TypeError: a float is required
>>  >>>
>>     
>
> If I understand correctly, you would like to create an array from a
> buffer:
>
> In [12]: N.frombuffer(value,dtype=N.complex128)
> Out[12]: array([ 1.+2.j,  3.+4.j,  5.+6.j])
>
> Cheers
> St?fan
>   
Thanks, I didn't find "frombuffer" in the Numpy Examples List. 

Incidentally, the List is a big help but it would be even better if it 
included the signatures of the functions.

Colin W.
> -------------------------------------------------------------------------
> Using Tomcat but need to do more? Need to support web services, security?
> Get stuff done quickly with pre-integrated technology to make your job easier
> Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
> http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
> _______________________________________________
> Numpy-discussion mailing list
> Numpy-discussion at lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/numpy-discussion
>
>   

-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From David.L.Goldsmith at noaa.gov  Thu Nov  9 16:02:33 2006
From: David.L.Goldsmith at noaa.gov (David L Goldsmith)
Date: Thu, 09 Nov 2006 13:02:33 -0800
Subject: Anyone have a
 "little"	shooting-method	function to share
In-Reply-To: <1163094437.6376.6.camel@localhost.localdomain>
References: <45527171.8030903@noaa.gov>
	<1163094437.6376.6.camel@localhost.localdomain>
Message-ID: <45539769.6030306@noaa.gov>

Wow, thanks!

DG

Pauli Virtanen wrote:
> ke, 2006-11-08 kello 16:08 -0800, David L Goldsmith kirjoitti:
>   
>> Hi!  I tried to send this earlier: it made it into my sent mail folder, 
>> but does not appear to have made it to the list.
>>
>> I need to numerically solve:
>>     (1-t)x" + x' - x = f(t), x(0) = x0, x(1) = x1
>> I've been trying to use (because it's the approach I inherited) an 
>> elementary finite-difference discretization, but unit tests have shown 
>> that that approach isn't working.  
>>     
> [clip]
>
> You could try to use some of the pre-existing www.netlib.org codes for
> solving BVPs. This only requires writing a wrapper.
>
> However, I have written a wrapper for the COLNEW code, which is a
> finite-difference method written by Ascher & Bader in '87, having an
> adaptive mesh selection. You can find it here:
>
> 	http://www.iki.fi/pav/bvp
>
> As I understand, COLNEW does not have any special handling for singular
> coefficients, but nevertheless it seems to be able to solve your
> problem. The code goes as follows:
>
> ----------------------------------------------------
> import scipy as N
> import bvp
>
> u0 = 1
> u1 = 2
>
> def f(t):
>     return N.sin(2*t)
>
> def fsub(t, z):
>     u, du = z
>     return N.array([(f(t) + u - du)/(1-t)])
>
> def gsub(z):
>     u, du = z
>     return N.array([u[0] - u0, u[1] - u1])
>
> tol = [1e-5, 1e-5]
> boundary_points = [0, 1]
>
> solution = bvp.colnew.solve(boundary_points, [2], fsub, gsub,
>     is_linear=True, tolerances=tol,
>     vectorized=True, maximum_mesh_size=300)
>
> import pylab
>
> x = solution.mesh
> pylab.plot(x, solution(x)[:,0])
> pylab.savefig('solution.png')
> ----------------------------------------------------
>
> BR,
>
> 	Pauli Virtanen
>
>   
> ------------------------------------------------------------------------
>
> -------------------------------------------------------------------------
> Using Tomcat but need to do more? Need to support web services, security?
> Get stuff done quickly with pre-integrated technology to make your job easier
> Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
> http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
> ------------------------------------------------------------------------
>
> _______________________________________________
> Numpy-discussion mailing list
> Numpy-discussion at lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/numpy-discussion
>   


-- 
HMRD/ORR/NOS/NOAA <http://response.restoration.noaa.gov/emergencyresponse/>

-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From Chris.Barker at noaa.gov  Thu Nov  9 13:34:35 2006
From: Chris.Barker at noaa.gov (Christopher Barker)
Date: Thu, 09 Nov 2006 10:34:35 -0800
Subject: Int32s, scalar operations and Pyhon longs
In-Reply-To: <eivlsb$mco$1@sea.gmane.org>
References: <db6b5ecc0611090040m570b634fl312b21222b4fe108@mail.gmail.com>
	<eivlsb$mco$1@sea.gmane.org>
Message-ID: <455374BB.8070000@noaa.gov>

Robert Kern wrote:

> I think we decided long ago that an int32 really is an array of 32-bit integers
> and behaves like one.

That would apply to y*y:

 >>> x = 999999
 >>> y = numpy.array([x])
 >>> x*x
999998000001L

So Python ints automatically convert to Python longs on overflow.

 >>> y*y
array([-729379967])

numpy int32 arrays wrap...

 >>> z = y[0]
 >>> type(z)
<type 'numpy.int32'>
 >>> z*z
Warning: overflow encountered in long_scalars
-729379967

int32 scalars wrap, but give a warning -- (why the warning here, but not 
with the array calculation?)

I'm a bit confused, because I thought that when you extracted a scalar 
from an array, you got regular python scalar for the datatypes that are 
supported. This made it clear that you always get a numpy Scalar, which, 
in a few situations, behaves differently than a seemingly equivalent 
Python scalar.

Have I got that right?

-Chris

-- 
Christopher Barker, Ph.D.
Oceanographer
                                     		
NOAA/OR&R/HAZMAT         (206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115       (206) 526-6317   main reception

Chris.Barker at noaa.gov

-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From tim.hochberg at ieee.org  Thu Nov  9 12:00:57 2006
From: tim.hochberg at ieee.org (Tim Hochberg)
Date: Thu, 09 Nov 2006 10:00:57 -0700
Subject: Int32s, scalar operations and Pyhon longs
In-Reply-To: <eivlsb$mco$1@sea.gmane.org>
References: <db6b5ecc0611090040m570b634fl312b21222b4fe108@mail.gmail.com>
	<eivlsb$mco$1@sea.gmane.org>
Message-ID: <45535EC9.80901@ieee.org>

Robert Kern wrote:
> Fernando Perez wrote:
>   
>> I understand why this happens, but I wonder if it should be in any way
>> 'fixed' (if that is even feasible without introducing other problems):
>>
>> In [28]: x = 999999
>>
>> In [29]: y = numpy.array([x])
>>
>> In [30]: z = y[0]
>>
>> In [31]: x==z
>> Out[31]: True
>>
>> In [32]: x
>> Out[32]: 999999
>>
>> In [33]: z
>> Out[33]: 999999
>>
>> In [34]: x*x
>> Out[34]: 999998000001L
>>
>> In [35]: z*z
>> Warning: overflow encountered in long_scalars
>> Out[35]: -729379967
>>
>> I am sure it will be, to say the least, pretty surprising (and I can
>> imagine subtle bugs being caused by this).
>>     
>
> I think we decided long ago that an int32 really is an array of 32-bit integers
> and behaves like one. That's precisely why one uses int32 arrays rather than
> object arrays. There are algorithms that do need the wraparound, and the Python
> int behavior is always available through object arrays.
>
>   
Let me add that I can't imagine that the bugs will be all that subtle 
given that numpy now spits out a warning on overflow.
If you're really worried about this I suggest you crank up the error 
mode to make this an error - then you really won't be able to miss this 
kind of overflow.

-tim




-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From efiring at hawaii.edu  Thu Nov  9 14:27:22 2006
From: efiring at hawaii.edu (Eric Firing)
Date: Thu, 09 Nov 2006 09:27:22 -1000
Subject: A reimplementation of MaskedArray
In-Reply-To: <45536149.6040409@ieee.org>
References: <200610160232.20888.pgmdevlist@mailcan.com>
	<200610241408.40415.pgmdevlist@gmail.com>
	<16761e100611071711q6b9cc464p85708d7a642b9a8a@mail.gmail.com>
	<200611080510.13565.pgmdevlist@gmail.com>
	<ce557a360611081041t6748bcdj953aa6aaea9cab18@mail.gmail.com>
	<45522EF2.8000603@ieee.org>
	<ce557a360611082327i6560c75dy18bcb2bdc03833f2@mail.gmail.com>
	<45536149.6040409@ieee.org>
Message-ID: <4553811A.4030606@hawaii.edu>

It looks like on my Pentium M multiplication with NaNs is slow, but 
using a masked array ranges from slightly faster (with only one value 
masked) to twice as slow (with all values masked):


In [15]:Timer("a.prod()", "import numpy as np; aa = np.ones(4096); a = 
np.ma.masked_greater(aa,0)").timeit(10000)
Out[15]:9.4012830257415771

In [16]:Timer("a.prod()", "import numpy as np; a = np.ones(4096); a[0]= 
np.nan").timeit(10000)
Out[16]:5.5737850666046143

In [17]:Timer("a.prod()", "import numpy as np; a = 
np.ones(4096)").timeit(10000)
Out[17]:0.40796804428100586

In [18]:Timer("a.prod()", "import numpy as np; aa = np.ones(4096); aa[0] 
= 2; a = np.ma.masked_greater(aa,1)").timeit(10000)
Out[18]:4.1544749736785889

In [19]:Timer("a.prod()", "import numpy as np; a = np.ones(4096); a[:]= 
np.nan").timeit(10000)Out[19]:5.8589630126953125

For transcendentals, nans or masks don't make much difference, although 
masks are slightly faster than nans:

In [20]:Timer("np.sin(a)", "import numpy as np; a = np.ones(4096); a[:]= 
np.nan").timeit(10000)
Out[20]:4.5575671195983887

In [21]:Timer("np.ma.sin(a)", "import numpy as np; aa = np.ones(4096); a 
= np.ma.masked_greater(aa,0)").timeit(10000)
Out[21]:4.4125270843505859

In [22]:Timer("b=np.sin(a)", "import numpy as np; a = 
np.ones(4096)").timeit(10000)
Out[22]:3.5793929100036621


Eric



Tim Hochberg wrote:
> A. M. Archibald wrote:
>> On 08/11/06, Tim Hochberg <tim.hochberg at ieee.org> wrote:
>>
>>   
>>> It has always been my experience (on various flavors or Pentium) that
>>> operating on NANs is extremely slow. Does anyone know on what hardware
>>> NANs are *not* slow? Of course it's always possible I just never notice
>>> NANs on hardware where they aren't slow.
>>>     
>> On an opteron machine I have access to, they appear to be no slower
>> (and even faster for some transcendental functions) than ordinary
>> floats:
>>
>> In [13]: a=zeros(1000000)
>>
>> In [14]: %time for i in xrange(1000): a += 1.1
>> CPU times: user 6.87 s, sys: 0.00 s, total: 6.87 s
>> Wall time: 6.87
>>
>> In [15]: a *= NaN
>>
>> In [16]: %time for i in xrange(1000): a += 1.1
>> CPU times: user 6.86 s, sys: 0.00 s, total: 6.86 s
>> Wall time: 6.87
>>
>> On my Pentium M, they are indeed significantly slower (three times? I
>> didn't really do enough testing to say how much). I am actually rather
>> offended by this unfair discrimination against a citizen in good
>> standing of the IEEE floating point community.
>>   
> If they're only 3x slower you're doing better than I am. On my core duo 
> box they appear to be nearly 20x slower for both addition and 
> multiplication. This more or less matches what I recall from earlier boxes.
> 
>      >>> Timer("a.prod()", "import numpy as np; a = np.ones(4096); a[0]
>     = np.nan").timeit(10000)
>     5.6495738983585397
>      >>> Timer("a.prod()", "import numpy as np; a =
>     np.ones(4096)").timeit(10000)
>     0.34439041833525152
>      >>> Timer("a.sum()", "import numpy as np; a = np.ones(4096); a[0] =
>     np.nan").timeit(10000)
>     6.0985655998179027
>      >>> Timer("a.sum()", "import numpy as np; a =
>     np.ones(4096)").timeit(10000)
>     0.32354363473564263
> 
> I've been told that operations on NANs are slow because they aren't 
> always implemented in the FPU hardware. Instead they are trapped and 
> implemented software or firmware or something or other. That may well be 
> bogus 42nd hand information though.
> 
> -tim
> 
> 
> -------------------------------------------------------------------------
> Using Tomcat but need to do more? Need to support web services, security?
> Get stuff done quickly with pre-integrated technology to make your job easier
> Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
> http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
> _______________________________________________
> Numpy-discussion mailing list
> Numpy-discussion at lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/numpy-discussion


-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From Chris.Barker at noaa.gov  Thu Nov  9 13:53:19 2006
From: Chris.Barker at noaa.gov (Christopher Barker)
Date: Thu, 09 Nov 2006 10:53:19 -0800
Subject: A reimplementation of MaskedArray
In-Reply-To: <45536149.6040409@ieee.org>
References: <200610160232.20888.pgmdevlist@mailcan.com>
	<200610241408.40415.pgmdevlist@gmail.com>
	<16761e100611071711q6b9cc464p85708d7a642b9a8a@mail.gmail.com>
	<200611080510.13565.pgmdevlist@gmail.com>
	<ce557a360611081041t6748bcdj953aa6aaea9cab18@mail.gmail.com>
	<45522EF2.8000603@ieee.org>
	<ce557a360611082327i6560c75dy18bcb2bdc03833f2@mail.gmail.com>
	<45536149.6040409@ieee.org>
Message-ID: <4553791F.4010906@noaa.gov>

Tim Hochberg wrote:
> I've been told that operations on NANs are slow because they aren't 
> always implemented in the FPU hardware. Instead they are trapped and 
> implemented software or firmware or something or other. 

which still doesn't make sense -- doesn't ANY operation with a NaN 
return NaN? how hard could that be?

I share in the disappointment that NaN's are not first class citizens in 
common hardware....

-Chris


-- 
Christopher Barker, Ph.D.
Oceanographer
                                     		
NOAA/OR&R/HAZMAT         (206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115       (206) 526-6317   main reception

Chris.Barker at noaa.gov

-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From tim.hochberg at ieee.org  Thu Nov  9 17:13:56 2006
From: tim.hochberg at ieee.org (Tim Hochberg)
Date: Thu, 09 Nov 2006 15:13:56 -0700
Subject: A reimplementation of MaskedArray
In-Reply-To: <4553791F.4010906@noaa.gov>
References: <200610160232.20888.pgmdevlist@mailcan.com>	<200610241408.40415.pgmdevlist@gmail.com>	<16761e100611071711q6b9cc464p85708d7a642b9a8a@mail.gmail.com>	<200611080510.13565.pgmdevlist@gmail.com>	<ce557a360611081041t6748bcdj953aa6aaea9cab18@mail.gmail.com>	<45522EF2.8000603@ieee.org>	<ce557a360611082327i6560c75dy18bcb2bdc03833f2@mail.gmail.com>	<45536149.6040409@ieee.org>
	<4553791F.4010906@noaa.gov>
Message-ID: <4553A824.6070108@ieee.org>

Christopher Barker wrote:
> Tim Hochberg wrote:
>   
>> I've been told that operations on NANs are slow because they aren't 
>> always implemented in the FPU hardware. Instead they are trapped and 
>> implemented software or firmware or something or other. 
>>     
>
> which still doesn't make sense -- doesn't ANY operation with a NaN 
> return NaN? how hard could that be?
>   
Well, it's not free. You still have to recognize a particular bit 
pattern corresponding to QNaN and then copy the result in that case. All 
that probably burns more silicon for a case that most people don't care 
about. Do NaNs make Half Life run any faster? No I didn't think so.

I suspect that instead they just look for the case where the exponent is 
all 1s, in which case the result is either +-Inf, SNaN or QNaN and then 
drop into software to handle those infrequent cases.
> I share in the disappointment that NaN's are not first class citizens in 
> common hardware....
>   
It would be nice to see IEEE 754 supported more faithfully, but there 
seems to be more of a problem on the software side than on the hardware 
side in practice.

I still can't help feeling that using NaNs for mask values would be 
abuse even if it weren't slow and eventually it would reach around and 
bite you someplace tender.

-tim


-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From robert.kern at gmail.com  Thu Nov  9 17:14:47 2006
From: robert.kern at gmail.com (Robert Kern)
Date: Thu, 09 Nov 2006 16:14:47 -0600
Subject: Int32s, scalar operations and Pyhon longs
In-Reply-To: <455374BB.8070000@noaa.gov>
References: <db6b5ecc0611090040m570b634fl312b21222b4fe108@mail.gmail.com>	<eivlsb$mco$1@sea.gmane.org>
	<455374BB.8070000@noaa.gov>
Message-ID: <ej0990$vpm$1@sea.gmane.org>

Christopher Barker wrote:
> I'm a bit confused, because I thought that when you extracted a scalar 
> from an array, you got regular python scalar for the datatypes that are 
> supported. This made it clear that you always get a numpy Scalar, which, 
> in a few situations, behaves differently than a seemingly equivalent 
> Python scalar.

Part of the point of adding scalar types was to ensure that a[0], for example,
always exposes the array interface (.shape, .dtype, etc.) whether a.shape is
(10,) or (10, 10). This uniformity aids generic programming.

-- 
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless enigma
 that is made terrible by our own mad attempt to interpret it as though it had
 an underlying truth."
  -- Umberto Eco


-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From fperez.net at gmail.com  Thu Nov  9 17:34:05 2006
From: fperez.net at gmail.com (Fernando Perez)
Date: Thu, 9 Nov 2006 15:34:05 -0700
Subject: Int32s, scalar operations and Pyhon longs
In-Reply-To: <45535EC9.80901@ieee.org>
References: <db6b5ecc0611090040m570b634fl312b21222b4fe108@mail.gmail.com>
	<eivlsb$mco$1@sea.gmane.org> <45535EC9.80901@ieee.org>
Message-ID: <db6b5ecc0611091434p6772ec3dkd82352742cc80a97@mail.gmail.com>

On 11/9/06, Tim Hochberg <tim.hochberg at ieee.org> wrote:

> Let me add that I can't imagine that the bugs will be all that subtle
> given that numpy now spits out a warning on overflow.
> If you're really worried about this I suggest you crank up the error
> mode to make this an error - then you really won't be able to miss this
> kind of overflow.

Not all computations are run interactively with a human in front of
the console to actually see the warnings go by.  If they get buried in
a log file on a cluster, there is a possibility you might miss them.

Cheers,

f

-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From Chris.Barker at noaa.gov  Wed Nov  8 16:37:13 2006
From: Chris.Barker at noaa.gov (Christopher Barker)
Date: Wed, 08 Nov 2006 13:37:13 -0800
Subject: "Slicing" recarrays
In-Reply-To: <45513A71.3060603@ee.byu.edu>
References: <45512A9B.9030809@noaa.gov> <45513A71.3060603@ee.byu.edu>
Message-ID: <45524E09.6030806@noaa.gov>

Travis Oliphant wrote:

> How about
> 
> newdtype = N.dtype([('both','2i4'),('','b1')])
> res = data.view(newdtype)['both']

That works great! thanks,

-Chris

-- 
Christopher Barker, Ph.D.
Oceanographer
                                     		
NOAA/OR&R/HAZMAT         (206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115       (206) 526-6317   main reception

Chris.Barker at noaa.gov

-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From pfdubois at gmail.com  Thu Nov  9 18:00:18 2006
From: pfdubois at gmail.com (Paul Dubois)
Date: Thu, 9 Nov 2006 15:00:18 -0800
Subject: A reimplementation of MaskedArray
In-Reply-To: <4553791F.4010906@noaa.gov>
References: <200610160232.20888.pgmdevlist@mailcan.com>
	<200610241408.40415.pgmdevlist@gmail.com>
	<16761e100611071711q6b9cc464p85708d7a642b9a8a@mail.gmail.com>
	<200611080510.13565.pgmdevlist@gmail.com>
	<ce557a360611081041t6748bcdj953aa6aaea9cab18@mail.gmail.com>
	<45522EF2.8000603@ieee.org>
	<ce557a360611082327i6560c75dy18bcb2bdc03833f2@mail.gmail.com>
	<45536149.6040409@ieee.org> <4553791F.4010906@noaa.gov>
Message-ID: <f74a6c2f0611091500s10a22e65u700441529df7134a@mail.gmail.com>

Disappointed in NaN land?

Since the function of old retired persons is to tell youngsters stories
around the campfile:

A supercomputer hardware designer told me that when forced to add IEEE
arithmetic to his designs that it decreased performance substantially, maybe
25-30%; it wasn't that doing the operations took so much longer, it was that
the increased physical space needed for that circuitry pushed the memory
farther away. Doubtless this inspired doing some of it in software instead.

No standard for controlling the behaviors exists, either, so you can find
out the hard way that underflow-to-zero is being done in software by
default, and that you are doing a lot of it. Or that your code doesn't have
the same behavior on different platforms.

To my mind, all that was really accomplished was to convince said youngsters
that somehow this NaN stuff was the solution to some problem. In reality,
computing for 3 days and having it print out 1000 NaNs is not exactly
satisfying. I think this stuff was essentially a mistake in the sense that
it is a nice ivory-tower idea that costs more in practice than it is worth.
I do not think a properly thought-out and coded algorithm ought to do
anything that this stuff is supposed to 'help' with, and if it does do it,
the code should stop executing.

Anyway, if I thought it would do the job I wouldn't have written MA in the
first place.

Rant off. Nap. Grumble. Stupid kids. (:->
-- Paul
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20061109/79338e77/attachment.html>
-------------- next part --------------
-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
-------------- next part --------------
_______________________________________________
Numpy-discussion mailing list
Numpy-discussion at lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/numpy-discussion

From koara at atlas.cz  Thu Nov  9 18:24:23 2006
From: koara at atlas.cz (koara)
Date: Thu, 09 Nov 2006 15:24:23 -0800
Subject: slice assignment: strange behaviour
In-Reply-To: <Mahogany-0.67.0-660-20061109-114322.00@american.edu>
References: <1163025393.211885.161470@e3g2000cwe.googlegroups.com>
	<Mahogany-0.67.0-660-20061109-114322.00@american.edu>
Message-ID: <1163114663.842976.124280@f16g2000cwb.googlegroups.com>

Ok, so i installed the newest numpy and scipy over enthought defaults.
The original problem indeed disappeared! Given the highly suspicous
magic threshold number of 10000, i suspect it was a bug in old numpy.
Makes me wonder if it's really been solved or just stopped manifesting
itself in my test data :>

Btw numpy.sum now behaves differently when axis is left unspecified: an
example listed in help(numpy.sum) doesn't give the result the help
claims it does, as numpy.sum([[0, 1], [0, 5]]) now gives 6. Someone
ought to either change the description or the code 8)

Cheers


-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From joris at ster.kuleuven.ac.be  Thu Nov  9 17:45:21 2006
From: joris at ster.kuleuven.ac.be (joris at ster.kuleuven.ac.be)
Date: Thu,  9 Nov 2006 23:45:21 +0100
Subject: Grief with a complex value
Message-ID: <1163112321.4553af8165fdb@webmail.ster.kuleuven.be>


   [CJ]: I didn't find "frombuffer" in the Numpy Examples List. 

I just added a frombuffer() example (the 197th example!) which
I extracted from your mail.

 
   [CJ]: Incidentally, the List is a big help but it would be even better if 
   [CJ]: included the signatures of the functions.
 
Good idea! There are some other things, though, that are slightly higher
on my priority list. First, the NumPy Example List could be more complete.
Secondly, the alphabetic hyperlink list of NumPy functions is getting so long 
that it's becoming difficult for new people to grasp what NumPy can do. 
So I would like to add a categorized overview of the NumPy functions.

Cheers,
Joris


Disclaimer: http://www.kuleuven.be/cwis/email_disclaimer.htm


-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From miki at yahoo.co.jp  Thu Nov  9 19:29:58 2006
From: miki at yahoo.co.jp (=?iso-2022-jp?B?bWlraQ==?=)
Date: Thu,  9 Nov 2006 16:29:58 -0800 (PST)
Subject: (no subject)
Message-ID: <E1GiKEc-0006he-4z@mail.sourceforge.net>

:?? INFORMATION  ?????????????????????????:
 ??????????????????????
 ????????????   http://love-match.bz/pc/04
:??????????????????????????????????:

*????*:.?. .?.:*????*:.?..?:*????*:.?..?:**????*


???????????????????????????????????
??? ???????????????????Love?Match?
?-----------------------------------------------------------------
???????????????????????
???????????????????????
???????????????????????
???????????????????????
???????????????????????
???????????????????????
?-----------------------------------------------------------------
????????????????http://love-match.bz/pc/04

???????????????????????????????????
??? ??????????????????????

?-----------------------------------------------------------------
????????????????????????????
?-----------------------------------------------------------------
?????????????????????????????
???????????????????????????????
?http://love-match.bz/pc/04

?-----------------------------------------------------------------
????????????????????????????????
?-----------------------------------------------------------------
????????????????????????????????
?????????????????????
?http://love-match.bz/pc/04
?-----------------------------------------------------------------
????????????????????
?-----------------------------------------------------------------
????????????????????????
??????????????????????????????????
?http://love-match.bz/pc/04

?-----------------------------------------------------------------
????????????????????????????
?-----------------------------------------------------------------
???????????????????????????
?????????????????????????????????
?http://love-match.bz/pc/04

?-----------------------------------------------------------------
?????????????????????????
?-----------------------------------------------------------------
?????????????????????????
?????????????????????????????????
?http://love-match.bz/pc/04


???????????????????????????????????
??? ??500????????????????

?-----------------------------------------------------------------

???????/????
????????????????????
?????????????????????????????????
????????????????????????????????
??????????????????????????
??????????????????????????????
?[????] http://love-match.bz/pc/04

?-----------------------------------------------------------------

???????/??????
??????????????????????????????????
???????????????????????????????????
??????????
?[????] http://love-match.bz/pc/04

?-----------------------------------------------------------------

???????/?????
??????????????????????????????????
????????????????????????????????
?????????????????????????(^^)
?[????] http://love-match.bz/pc/04

?-----------------------------------------------------------------

???????/????
???????????????????????????????
??????????????????????????????
??????????????????????????????
????????
?[????] http://love-match.bz/pc/04

?-----------------------------------------------------------------

????????/???
???????????????1???
?????????????????????????
?????????????????????????
?[????] http://love-match.bz/pc/04

?-----------------------------------------------------------------

???????/???????
????18??????????????????????????
?????????????????????????????
?????????????????????????????
?[????] http://love-match.bz/pc/04

?-----------------------------------------------------------------

???`????/???
?????????????????????
??????????????????????
??????????????
?[????] http://love-match.bz/pc/04

?-----------------------------------------------------------------


   ????????????????????
??????????????????????????????????
?????????????
??-------------------------------------------------------------
????????????????????????????????
??[??????????]?http://love-match.bz/pc/?04
??-------------------------------------------------------------
?????????????????????
???????????????????????????
???????????????????
???????????????????????????????
??[??????????]?http://love-match.bz/pc/04
??????????????????????????????????

??????????3-6-4-533
??????
139-3668-7892







-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From mkg at cs.nyu.edu  Thu Nov  9 20:49:02 2006
From: mkg at cs.nyu.edu (Matthew Koichi Grimes)
Date: Thu, 09 Nov 2006 20:49:02 -0500
Subject: automatic differentiation using custom ndarray?
Message-ID: <4553DA8E.7010109@cs.nyu.edu>

I'd love to be able to do automatic differentiation using numpy
ndarrays. There are some AD libraries in C++ that work through operator
overloading on custom numeric types (e.g. "adouble"). I have no
experience in in creating custom ndarrays, but would it be a huge
project (or even possible) to make a custom ndarray that used such
augmented numeric types? From reading Jonathan Wang's thread
(http://www.mail-archive.com/numpy-discussion at lists.sourceforge.net/msg03487.html), 

it seemed like ndarrays with custom numeric types are still a work in
progress.

I'm thinking specifically of the ADOL-C library, but if people with
experience with various AD libraries have a better recommendation, I'd
be all ears. I've also looked at CPPAD, but it doesn't seem to operate
nearly as transparently.

-- Matt


-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From Chris.Barker at noaa.gov  Wed Nov  8 12:27:46 2006
From: Chris.Barker at noaa.gov (Christopher Barker)
Date: Wed, 08 Nov 2006 09:27:46 -0800
Subject: Stacking arrays...
In-Reply-To: <6ef8f3380611071415k7e3c59d5pba7ff9338abec60f@mail.gmail.com>
References: <4550D6F8.80001@noaa.gov>
	<6ef8f3380611071415k7e3c59d5pba7ff9338abec60f@mail.gmail.com>
Message-ID: <45521392.3000204@noaa.gov>

Pau Gargallo wrote:
> c=N.dstack([a,b]) seems to do the trick.

indeed it does. Thanks.

-Chris

-- 
Christopher Barker, Ph.D.
Oceanographer
                                     		
NOAA/OR&R/HAZMAT         (206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115       (206) 526-6317   main reception

Chris.Barker at noaa.gov

-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From fperez.net at gmail.com  Fri Nov 10 00:51:09 2006
From: fperez.net at gmail.com (Fernando Perez)
Date: Thu, 9 Nov 2006 22:51:09 -0700
Subject: [OT] Why Python 'fits your brain'
Message-ID: <db6b5ecc0611092151q4489ec46kbc81652e24c27a7b@mail.gmail.com>

Please forgive the not-specifically-numpy post. I'll keep it short.

Some of us often, when trying to explain to newcomers the benefits of
Python for scientific work, use expressions like the famous 'it fits
your brain'.  This is an attempt at conveying why it seems like such a
natural tool for expressing, concisely and without extraneous noise,
but simultaneously with ease, the nature of many algorithmic problems.

I think these four images

http://flickr.com/photos/nicksieger/281055485/
http://flickr.com/photos/nicksieger/281055530/
http://flickr.com/photos/nicksieger/280662707/
http://flickr.com/photos/nicksieger/280661836/

may provide a useful visual aid.  I certainly intend to use them (with
attribution) the next time I need to give a talk along those lines.

Regards,

f

-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From wbaxter at gmail.com  Fri Nov 10 01:12:09 2006
From: wbaxter at gmail.com (Bill Baxter)
Date: Fri, 10 Nov 2006 15:12:09 +0900
Subject: [OT] Why Python 'fits your brain'
In-Reply-To: <db6b5ecc0611092151q4489ec46kbc81652e24c27a7b@mail.gmail.com>
References: <db6b5ecc0611092151q4489ec46kbc81652e24c27a7b@mail.gmail.com>
Message-ID: <e86a5fd00611092212n6ae681b0vbbec422cc2412f57@mail.gmail.com>

I think Ruby users say the same about Ruby, maybe even more
emphatically than Python users, and Ruby's chart looks like just about
the most complicated one there.  C and Python look to be about on par.
 Also I suspect a chart of Lisp's grammar would be even simpler than
any of those up there, but I doubt many scientists would say that Lisp
really fits their brain.

So I'm not really sure what conclusions you can draw from such charts.
The human brain is a complex network too.  Probably what matters more
is that the connections in those dependency charts somehow match the
connections in our brains.

--bb

On 11/10/06, Fernando Perez <fperez.net at gmail.com> wrote:
> Please forgive the not-specifically-numpy post. I'll keep it short.
>
> Some of us often, when trying to explain to newcomers the benefits of
> Python for scientific work, use expressions like the famous 'it fits
> your brain'.  This is an attempt at conveying why it seems like such a
> natural tool for expressing, concisely and without extraneous noise,
> but simultaneously with ease, the nature of many algorithmic problems.
>
> I think these four images
>
> http://flickr.com/photos/nicksieger/281055485/
> http://flickr.com/photos/nicksieger/281055530/
> http://flickr.com/photos/nicksieger/280662707/
> http://flickr.com/photos/nicksieger/280661836/
>
> may provide a useful visual aid.  I certainly intend to use them (with
> attribution) the next time I need to give a talk along those lines.
>
> Regards,
>

-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From peridot.faceted at gmail.com  Fri Nov 10 01:42:38 2006
From: peridot.faceted at gmail.com (A. M. Archibald)
Date: Fri, 10 Nov 2006 01:42:38 -0500
Subject: A reimplementation of MaskedArray
In-Reply-To: <f74a6c2f0611091500s10a22e65u700441529df7134a@mail.gmail.com>
References: <200610160232.20888.pgmdevlist@mailcan.com>
	<200610241408.40415.pgmdevlist@gmail.com>
	<16761e100611071711q6b9cc464p85708d7a642b9a8a@mail.gmail.com>
	<200611080510.13565.pgmdevlist@gmail.com>
	<ce557a360611081041t6748bcdj953aa6aaea9cab18@mail.gmail.com>
	<45522EF2.8000603@ieee.org>
	<ce557a360611082327i6560c75dy18bcb2bdc03833f2@mail.gmail.com>
	<45536149.6040409@ieee.org> <4553791F.4010906@noaa.gov>
	<f74a6c2f0611091500s10a22e65u700441529df7134a@mail.gmail.com>
Message-ID: <ce557a360611092242m6b81bcdale40956ba1eb38c0@mail.gmail.com>

On 09/11/06, Paul Dubois <pfdubois at gmail.com> wrote:

> Since the function of old retired persons is to tell youngsters stories
> around the campfile:

I'll pull up a log. But since I'm uppity, and not especially young, I
hope you won't mind if I heckle.

> A supercomputer hardware designer told me that when forced to add IEEE
> arithmetic to his designs that it decreased performance substantially, maybe
> 25-30%; it wasn't that doing the operations took so much longer, it was that
> the increased physical space needed for that circuitry pushed the memory
> farther away. Doubtless this inspired doing some of it in software instead.

The goal of IEEE floats is not to be faster but to make doing correct
numerical programming easier. (Numerical analysts can write robust
numerical code for almost any kind of float, but the vast majority of
numerical code is written by scientists who know the bare minimum or
less about numerical analysis, so a good system makes such code work
as well as possible.)

The other reason IEEE floats are good is, no disrespect to your
hardware designer friend intended, that they were designed with some
care. Reimplementations are liable to get some of the finicky details
wrong (round-to-even, denormalized numbers, what have you...). I found
Kahan's "How Java's Floating-point Hurts Everyone Everywhere" (
http://www.cs.berkeley.edu/~wkahan/JAVAhurt.pdf ) very educational
when it comes to "what can go wrong with platform-dependent
floating-point".

> No standard for controlling the behaviors exists, either, so you can find
> out the hard way that underflow-to-zero is being done in software by
> default, and that you are doing a lot of it. Or that your code doesn't have
> the same behavior on different platforms.

Well, if the platform is IEEE-compliant, you can trust it to behave
the same way logicially, even if some applications are slower on some
systems than others. Or did you mean that there's no standard way to
set the various IEEE flags? That seems like a language/compiler issue,
to me (which is not to minimize the pain it causes!)

> To my mind, all that was really accomplished was to convince said youngsters
> that somehow this NaN stuff was the solution to some problem. In reality,
> computing for 3 days and having it print out 1000 NaNs is not exactly
> satisfying. I think this stuff was essentially a mistake in the sense that
> it is a nice ivory-tower idea that costs more in practice than it is worth.
> I do not think a properly thought-out and coded algorithm ought to do
> anything that this stuff is supposed to 'help' with, and if it does do it,
> the code should stop executing.

Well, then turn on floating-point exceptions. I find a few NaNs in my
calculations relatively benign. If I'm doing a calculation where
they'll propagate and destroy everything, I trap them. But it happens
just as often that I launch a job, a NaN appears early on, but I come
back in the morning to find that instead of aborting, the job has
completed except for a few bad data points.

If you want an algorithm that takes advantage of NaNs, I have one. I
was simulating light rays around a black hole, generating a map of the
bending for various view angles. So I naturally did a lot of exploring
of grazing incidence, where the calculations would often yield NaNs.
So I used the NaNs to fill in the gap between the internal bending of
light and the external bending of the light; no extra programming
effort required, since that was what came out of the ray tracer
anyway. The rendering step just returned NaNs for the image pixels
that came from interpolating in the NaN-filled regions, which came out
black; just what I wanted. I just wrote the program ignoring what
would happen if a NaN appeared, and it came out right. Which is, I
think, the goal - to save the programmer having to think too much
about numerical-analysis headaches.

> Anyway, if I thought it would do the job I wouldn't have written MA in the
> first place.

It's certainly not always a good solution; in particular it's hard to
control the behaviour of things like where(). There's also no NaN for
integer types, which makes MaskedArray more useful.

Anyway, I am pleased that numpy has both controllable IEEE floats and
MaskedArray, and I apologize for a basically off-topic post.

A. M. Archibald

-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From gruben at bigpond.net.au  Fri Nov 10 07:40:54 2006
From: gruben at bigpond.net.au (Gary Ruben)
Date: Fri, 10 Nov 2006 23:40:54 +1100
Subject: Grief with a complex value
In-Reply-To: <1163112321.4553af8165fdb@webmail.ster.kuleuven.be>
References: <1163112321.4553af8165fdb@webmail.ster.kuleuven.be>
Message-ID: <45547356.6040106@bigpond.net.au>

Does anyone think it might be a good idea to split up the examples list? 
It's getting very big now. I pity any poor modem users (I'm sure they 
still exist). However, I'm not sure that there's a good way to do it.

Gary R.

joris at ster.kuleuven.ac.be wrote:
>    [CJ]: I didn't find "frombuffer" in the Numpy Examples List. 
> 
> I just added a frombuffer() example (the 197th example!) which
> I extracted from your mail.
> 
>  
>    [CJ]: Incidentally, the List is a big help but it would be even better if 
>    [CJ]: included the signatures of the functions.
>  
> Good idea! There are some other things, though, that are slightly higher
> on my priority list. First, the NumPy Example List could be more complete.
> Secondly, the alphabetic hyperlink list of NumPy functions is getting so long 
> that it's becoming difficult for new people to grasp what NumPy can do. 
> So I would like to add a categorized overview of the NumPy functions.
> 
> Cheers,
> Joris

-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From joris at ster.kuleuven.be  Fri Nov 10 07:51:05 2006
From: joris at ster.kuleuven.be (Joris De Ridder)
Date: Fri, 10 Nov 2006 13:51:05 +0100
Subject: Grief with a complex value
In-Reply-To: <45547356.6040106@bigpond.net.au>
References: <1163112321.4553af8165fdb@webmail.ster.kuleuven.be>
	<45547356.6040106@bigpond.net.au>
Message-ID: <200611101351.05502.joris@ster.kuleuven.be>



On Friday 10 November 2006 13:40, Gary Ruben wrote:
   [GR]: Does anyone think it might be a good idea to split up the examples list? 
   [GR]: It's getting very big now. I pity any poor modem users (I'm sure they 
   [GR]: still exist). However, I'm not sure that there's a good way to do it.

They could download the Example List once and save it in a HTML file on their
local disk...

Joris

Disclaimer: http://www.kuleuven.be/cwis/email_disclaimer.htm


-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From steve at shrogers.com  Fri Nov 10 08:29:58 2006
From: steve at shrogers.com (Steven H. Rogers)
Date: Fri, 10 Nov 2006 06:29:58 -0700
Subject: [OT] Why Python 'fits your brain'
In-Reply-To: <e86a5fd00611092212n6ae681b0vbbec422cc2412f57@mail.gmail.com>
References: <db6b5ecc0611092151q4489ec46kbc81652e24c27a7b@mail.gmail.com>
	<e86a5fd00611092212n6ae681b0vbbec422cc2412f57@mail.gmail.com>
Message-ID: <45547ED6.1080004@shrogers.com>

Lisp has minimal syntax, as does APL, which does make them easy to learn if 
you have no preconceptions about what a programming language should be. 
They're rarely, if ever the first language people learn, so the strange 
appearance is a barrier that offsets the basic simplicity.  They both also 
have a large number of primitive functions, which adds to the cognitive 
load.  Of course, when you include necessary libraries, C and Java get 
pretty big too.

Maybe the types of connections are the deciding factor.  Hierarchical name 
spaces may help as well, so that it's more obvious that you only need to fit 
part of it into your brain as well.

Regards,
Steve
////////////////

Bill Baxter wrote:
> I think Ruby users say the same about Ruby, maybe even more
> emphatically than Python users, and Ruby's chart looks like just about
> the most complicated one there.  C and Python look to be about on par.
>  Also I suspect a chart of Lisp's grammar would be even simpler than
> any of those up there, but I doubt many scientists would say that Lisp
> really fits their brain.
> 
> So I'm not really sure what conclusions you can draw from such charts.
> The human brain is a complex network too.  Probably what matters more
> is that the connections in those dependency charts somehow match the
> connections in our brains.
> 
> --bb
> 
> On 11/10/06, Fernando Perez <fperez.net at gmail.com> wrote:
>> Please forgive the not-specifically-numpy post. I'll keep it short.
>>
>> Some of us often, when trying to explain to newcomers the benefits of
>> Python for scientific work, use expressions like the famous 'it fits
>> your brain'.  This is an attempt at conveying why it seems like such a
>> natural tool for expressing, concisely and without extraneous noise,
>> but simultaneously with ease, the nature of many algorithmic problems.
>>
>> I think these four images
>>
>> http://flickr.com/photos/nicksieger/281055485/
>> http://flickr.com/photos/nicksieger/281055530/
>> http://flickr.com/photos/nicksieger/280662707/
>> http://flickr.com/photos/nicksieger/280661836/
>>
>> may provide a useful visual aid.  I certainly intend to use them (with
>> attribution) the next time I need to give a talk along those lines.
>>
>> Regards,
>>
> 
> -------------------------------------------------------------------------
> Using Tomcat but need to do more? Need to support web services, security?
> Get stuff done quickly with pre-integrated technology to make your job easier
> Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
> http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
> _______________________________________________
> Numpy-discussion mailing list
> Numpy-discussion at lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/numpy-discussion
> 
> 

-- 
Steven H. Rogers, Ph.D., steve at shrogers.com
Weblog: http://shrogers.com/weblog
"He who refuses to do arithmetic is doomed to talk nonsense."
-- John McCarthy


-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From theller at ctypes.org  Fri Nov 10 09:32:13 2006
From: theller at ctypes.org (Thomas Heller)
Date: Fri, 10 Nov 2006 15:32:13 +0100
Subject: [OT] Why Python 'fits your brain'
In-Reply-To: <45547ED6.1080004@shrogers.com>
References: <db6b5ecc0611092151q4489ec46kbc81652e24c27a7b@mail.gmail.com>	<e86a5fd00611092212n6ae681b0vbbec422cc2412f57@mail.gmail.com>
	<45547ED6.1080004@shrogers.com>
Message-ID: <45548D6D.8030005@ctypes.org>

Steven H. Rogers schrieb:
> Lisp has minimal syntax, as does APL, which does make them easy to learn if 
> you have no preconceptions about what a programming language should be. 
> They're rarely, if ever the first language people learn, so the strange 
> appearance is a barrier that offsets the basic simplicity.

I really believe the world would be a somewhat better place if Lisp (or better yet, Scheme)
would be the first language people learned.  They need a better editor than notepad
for them, though.

Thomas


-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From pfdubois at gmail.com  Fri Nov 10 12:10:59 2006
From: pfdubois at gmail.com (Paul Dubois)
Date: Fri, 10 Nov 2006 09:10:59 -0800
Subject: [OT] was about masked array and IEEE arithmetic
Message-ID: <f74a6c2f0611100910y2cba22ftc526b3f76dd7ee4f@mail.gmail.com>

On 09 Nov 2006 22:50:44 -0800, A. M. Archibald <peridot.faceted at gmail.com>
wrote:
>
> On 09/11/06, Paul Dubois <pfdubois at gmail.com> wrote:
>
> > Since the function of old retired persons is to tell youngsters stories
> > around the campfile:
>
> I'll pull up a log. But since I'm uppity, and not especially young, I
> hope you won't mind if I heckle.
>
> > A supercomputer hardware designer told me that when forced to add IEEE
> > arithmetic to his designs that it decreased performance substantially,
> maybe
> > 25-30%; it wasn't that doing the operations took so much longer, it was
> that
> > the increased physical space needed for that circuitry pushed the memory
> > farther away. Doubtless this inspired doing some of it in software
> instead.
>
> The goal of IEEE floats is not to be faster but to make doing correct
> numerical programming easier. (Numerical analysts can write robust
> numerical code for almost any kind of float, but the vast majority of
> numerical code is written by scientists who know the bare minimum or
> less about numerical analysis, so a good system makes such code work
> as well as possible.)


This is an urban legend propagated to support the IEEE argument. I've worked
intimately with scientific programmers for 30 years and very few of them
have the characteristics you describe. It is their professional business to
know the basics of numerical analysis. They use packages written by the
numerical analysis community a lot, too.

The other reason IEEE floats are good is, no disrespect to your
> hardware designer friend intended, that they were designed with some
> care. Reimplementations are liable to get some of the finicky details
> wrong (round-to-even, denormalized numbers, what have you...). I found
> Kahan's "How Java's Floating-point Hurts Everyone Everywhere" (
> http://www.cs.berkeley.edu/~wkahan/JAVAhurt.pdf ) very educational
> when it comes to "what can go wrong with platform-dependent
> floating-point".



Well, Kahan at one point went around showing how using an early HP
calculator to navigate your plane was a bad idea. The pilots in the audience
were smirking -- he had an extremely long thin triangle in his example. They
aren't that dumb. I think these alleged difficulties don't exist in
practice.

> No standard for controlling the behaviors exists, either, so you can find
> > out the hard way that underflow-to-zero is being done in software by
> > default, and that you are doing a lot of it. Or that your code doesn't
> have
> > the same behavior on different platforms.
>
> Well, if the platform is IEEE-compliant, you can trust it to behave
> the same way logicially, even if some applications are slower on some
> systems than others. Or did you mean that there's no standard way to
> set the various IEEE flags? That seems like a language/compiler issue,
> to me (which is not to minimize the pain it causes!)


I meant both. It may seem like a language issue to you (and me) but it
doesn't to the language standard committees; just like they won't take
ownership of the command line to run the compiler no matter how much the
variances hurt people.

> To my mind, all that was really accomplished was to convince said
> youngsters
> > that somehow this NaN stuff was the solution to some problem. In
> reality,
> > computing for 3 days and having it print out 1000 NaNs is not exactly
> > satisfying. I think this stuff was essentially a mistake in the sense
> that
> > it is a nice ivory-tower idea that costs more in practice than it is
> worth.
> > I do not think a properly thought-out and coded algorithm ought to do
> > anything that this stuff is supposed to 'help' with, and if it does do
> it,
> > the code should stop executing.
>
> Well, then turn on floating-point exceptions. I find a few NaNs in my
> calculations relatively benign. If I'm doing a calculation where
> they'll propagate and destroy everything, I trap them. But it happens
> just as often that I launch a job, a NaN appears early on, but I come
> back in the morning to find that instead of aborting, the job has
> completed except for a few bad data points.


This sounds like "the software is done but we just haven't gotten it running
yet."

If you want an algorithm that takes advantage of NaNs, I have one. I
> was simulating light rays around a black hole, generating a map of the
> bending for various view angles. So I naturally did a lot of exploring
> of grazing incidence, where the calculations would often yield NaNs.
> So I used the NaNs to fill in the gap between the internal bending of
> light and the external bending of the light; no extra programming
> effort required, since that was what came out of the ray tracer
> anyway. The rendering step just returned NaNs for the image pixels
> that came from interpolating in the NaN-filled regions, which came out
> black; just what I wanted. I just wrote the program ignoring what
> would happen if a NaN appeared, and it came out right. Which is, I
> think, the goal - to save the programmer having to think too much
> about numerical-analysis headaches.


How do you know it came out right? (:->

Anyway, costing those who can think 30% of performance to save you a few
headaches is not worth it. That's what I meant, not that there were no cases
of benefit.

> Anyway, if I thought it would do the job I wouldn't have written MA in the
> > first place.
>
> It's certainly not always a good solution; in particular it's hard to
> control the behaviour of things like where(). There's also no NaN for
> integer types, which makes MaskedArray more useful.
>
> Anyway, I am pleased that numpy has both controllable IEEE floats and
> MaskedArray, and I apologize for a basically off-topic post.
>
> A. M. Archibald



I guess it was me who was off topic, so I'll be quiet now.

-------------------------------------------------------------------------
> Using Tomcat but need to do more? Need to support web services, security?
> Get stuff done quickly with pre-integrated technology to make your job
> easier
> Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
> http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
> _______________________________________________
> Numpy-discussion mailing list
> Numpy-discussion at lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/numpy-discussion
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20061110/05f9b7d0/attachment.html>
-------------- next part --------------
-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
-------------- next part --------------
_______________________________________________
Numpy-discussion mailing list
Numpy-discussion at lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/numpy-discussion

From oubtevephs at originalwebdesign.com  Fri Nov 10 20:15:06 2006
From: oubtevephs at originalwebdesign.com (NETBALL)
Date: Fri, 10 Nov 2006 22:15:06 -0300
Subject: Faith Hill Vs Carrie
References: <000401c7052e$d2e86ab0$00000000@meche>
Message-ID: <000401c7052e$d2e86ab0$00000000@meche>


  ----- Original Message ----- 
  From: numpy-discussion at lists.sourceforge.net 
  To: oubtevephs at originalwebdesign.com 
  Sent: Saturday, November 01:13:02 PM
  Subject: Faith Hill Vs Carrie


  Front page forglobe Headlines.
  Woodhall gb sprintings low Relief sun or nations. Fog in dirty coke jackedup am living is couple havent formally.
  Gary Geneva Glen Ellyn! Secrets Fights Runways Vincent of Maroulis focus. Trip a Pakistan Shocked. Written produced edited Dehnartfor of reprinting content pithy quotes.
  Spam Address a comment. Crackin Americas in foremost popstar wasting fog dirty of coke. Pages or under Creative.
  Chicago Belmont Cragin Edgebrook Edison. Sun of nations incredible Police.
  Account of vacuum vulturous leeching off brought. Delicious Digg itrss is.
  Macintosh os a earlier Wear Newsletter javacom Technology Partner.
  Showdown psp am Blues Bash Pico North. Delicious Digg itrss is.
  How they best react. Fare enjoys is gardening add ownbuy ships hours Bodyguard a? Subscribe Contact recent stories begins.
  News Policies in Affiliate Program of Link Exchange Center in Feedback am. Niles Norridge Northbrook am Northlake Northwest of.
  Supporters impeccable pedigree needs drop pounds whatever! Main Menu raquo General.
  Crestwood Crown or Darien Deerfield in des Plaines or Downers!
  Woodhall gb sprintings low Relief sun or nations.
  Quiz countries am part cgc Council ni.
  Mora of Manfredo again Saturday Directors Guild in.
  Bet Platinum Soft Power Boston Globe.
  Mora of Manfredo again Saturday Directors Guild in.
  Files Divorce Federline Takes a cma a Awards!
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20061110/d41b0f51/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: West.gif
Type: image/gif
Size: 14283 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20061110/d41b0f51/attachment.gif>
-------------- next part --------------
-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
-------------- next part --------------
_______________________________________________
Numpy-discussion mailing list
Numpy-discussion at lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/numpy-discussion

From daloveuw at ont.sysco.com  Fri Nov 10 19:56:10 2006
From: daloveuw at ont.sysco.com (devices across)
Date: Sat, 11 Nov 2006 08:56:10 +0800
Subject: down Combines advanced freeKVM
References: <000901c7052c$2d4decd0$00000000@HAIER>
Message-ID: <000901c7052c$2d4decd0$00000000@HAIER>


  ----- Original Message ----- 
  From: numpy-discussion at lists.sourceforge.net 
  To: daloveuw at ont.sysco.com 
  Sent: Sunday, November 06:47:05 AM
  Subject: down Combines advanced freeKVM


  Embedded Solutions Home.
  Digi Sites Deutsch Chinese Japanese Rabbit Maxstream Products Cellular in. Remotely onoff measure electrical a load.
  Ethernet dc ac is options a Securely it port Inband outofband. Tasks rj of dual Ethernet dc ac. Deutsch Chinese Japanese Rabbit of Maxstream. Try online demo am now Warranty Where buy direct from.
  Load more alarms for quotreal timequot of of? Common equipment monitoring am. Management Contact us Partner login or Digi Sites Deutsch.
  Reliable Automates common or equipment a monitoring tasks rj. Fe Console Servers a Secure and Easy.
  Measure electrical of load more alarms for quotreal. Enable in you am to access is monitor or manage in.
  Remote Power Manager Serial of Cards or Embedded Solutions.
  Their ports am even when is down Combines advanced. Enable in you am to access is monitor or manage in.
  Fe Console Servers Secure and Easy Management Contact. Embedded Solutions Home enable you is to in. Fe is Console Servers Secure and Easy Management Contact!
  Freekvm Ipmi is support a in one reliable? Both or over the.
  Automates common equipment monitoring tasks. Conditions Product am Literature Solution guide or Related?
  Remote Power Manager Serial a Cards Embedded.
  Digi Sites Deutsch Chinese Japanese Rabbit Maxstream Products Cellular in.
  Cards Embedded is Solutions a Home is enable you of to.
  And Easy Management Contact us Partner in login Digi.
  Access monitor manage network.
  And Easy Management Contact us Partner in login Digi.
  Demo is now of Warranty Where buy direct from.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20061111/67a17164/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: more.gif
Type: image/gif
Size: 14076 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20061111/67a17164/attachment.gif>
-------------- next part --------------
-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
-------------- next part --------------
_______________________________________________
Numpy-discussion mailing list
Numpy-discussion at lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/numpy-discussion

From kwgoodman at gmail.com  Sat Nov 11 13:40:22 2006
From: kwgoodman at gmail.com (Keith Goodman)
Date: Sat, 11 Nov 2006 10:40:22 -0800
Subject: Assign NaN, get zero
Message-ID: <f4f93d420611111040u3ea1988el6b6baf01aba4891e@mail.gmail.com>

I accidentally wrote a unit test using int32 instead of float64 and
ran into this problem:

>> x = M.matrix([[1, 2, 3]])
>> x[0,1] = M.nan
>> x
matrix([[1, 0, 3]])   <--- Got 0 instead of NaN

But this, of course, works:

>> x = M.matrix([[1.0, 2.0, 3.0]])
>> x[0,1] = M.nan
>> x
matrix([[ 1.        ,         nan,  3.        ]])

Is returning a 0 instead of NaN the expected behavior?

If so do I need to check dtype before inserting NaNs? And if the dtype
is int32, then raise an exception?

I'm using 1.0rc1 from debian etch.

-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From stefan at sun.ac.za  Sat Nov 11 15:36:39 2006
From: stefan at sun.ac.za (Stefan van der Walt)
Date: Sat, 11 Nov 2006 22:36:39 +0200
Subject: Assign NaN, get zero
In-Reply-To: <f4f93d420611111040u3ea1988el6b6baf01aba4891e@mail.gmail.com>
References: <f4f93d420611111040u3ea1988el6b6baf01aba4891e@mail.gmail.com>
Message-ID: <20061111203639.GB22096@mentat.za.net>

On Sat, Nov 11, 2006 at 10:40:22AM -0800, Keith Goodman wrote:
> I accidentally wrote a unit test using int32 instead of float64 and
> ran into this problem:
> 
> >> x = M.matrix([[1, 2, 3]])
> >> x[0,1] = M.nan
> >> x
> matrix([[1, 0, 3]])   <--- Got 0 instead of NaN
> 
> But this, of course, works:
> 
> >> x = M.matrix([[1.0, 2.0, 3.0]])
> >> x[0,1] = M.nan
> >> x
> matrix([[ 1.        ,         nan,  3.        ]])
> 
> Is returning a 0 instead of NaN the expected behavior?

NaN (or inf) is a floating point number, so seeing a zero in integer
representation seems correct:

In [2]: int(N.nan)
Out[2]: 0L

Cheers
St?fan

-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From dalcinl at gmail.com  Sat Nov 11 16:30:06 2006
From: dalcinl at gmail.com (Lisandro Dalcin)
Date: Sat, 11 Nov 2006 18:30:06 -0300
Subject: Assign NaN, get zero
In-Reply-To: <20061111203639.GB22096@mentat.za.net>
References: <f4f93d420611111040u3ea1988el6b6baf01aba4891e@mail.gmail.com>
	<20061111203639.GB22096@mentat.za.net>
Message-ID: <e7ba66e40611111330x4e8f9c65x9db05792cac98fbe@mail.gmail.com>

On 11/11/06, Stefan van der Walt <stefan at sun.ac.za> wrote:
> NaN (or inf) is a floating point number, so seeing a zero in integer
> representation seems correct:
>
> In [2]: int(N.nan)
> Out[2]: 0L
>

Just to learn myself: Why int(N.nan) should be 0? Is it C behavior?

-- 
Lisandro Dalc?n
---------------
Centro Internacional de M?todos Computacionales en Ingenier?a (CIMEC)
Instituto de Desarrollo Tecnol?gico para la Industria Qu?mica (INTEC)
Consejo Nacional de Investigaciones Cient?ficas y T?cnicas (CONICET)
PTLC - G?emes 3450, (3000) Santa Fe, Argentina
Tel/Fax: +54-(0)342-451.1594

-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From charlesr.harris at gmail.com  Sat Nov 11 16:56:33 2006
From: charlesr.harris at gmail.com (Charles R Harris)
Date: Sat, 11 Nov 2006 14:56:33 -0700
Subject: Assign NaN, get zero
In-Reply-To: <e7ba66e40611111330x4e8f9c65x9db05792cac98fbe@mail.gmail.com>
References: <f4f93d420611111040u3ea1988el6b6baf01aba4891e@mail.gmail.com>
	<20061111203639.GB22096@mentat.za.net>
	<e7ba66e40611111330x4e8f9c65x9db05792cac98fbe@mail.gmail.com>
Message-ID: <e06186140611111356n1b02b48k68e57b7ce990d863@mail.gmail.com>

On 11/11/06, Lisandro Dalcin <dalcinl at gmail.com> wrote:
>
> On 11/11/06, Stefan van der Walt <stefan at sun.ac.za> wrote:
> > NaN (or inf) is a floating point number, so seeing a zero in integer
> > representation seems correct:
> >
> > In [2]: int(N.nan)
> > Out[2]: 0L
> >
>
> Just to learn myself: Why int(N.nan) should be 0? Is it C behavior?


In [1]: int32(0)/int32(0)
Warning: divide by zero encountered in long_scalars
Out[1]: 0

In [2]: float32(0)/float32(0)
Out[2]: nan

In [3]: int(nan)
Out[3]: 0L

I think it was just a default for numpy. Hmmm, numpy now warns on integer
division by zero, didn't used to.  Looks like a warning should also be
raised when casting nan to integer. It is probably a small bug not to. I
also suspect int(nan) should return a normal python zero, not 0L.

Chuck
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20061111/63228828/attachment.html>
-------------- next part --------------
-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
-------------- next part --------------
_______________________________________________
Numpy-discussion mailing list
Numpy-discussion at lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/numpy-discussion

From stefan at sun.ac.za  Sat Nov 11 16:54:49 2006
From: stefan at sun.ac.za (Stefan van der Walt)
Date: Sat, 11 Nov 2006 23:54:49 +0200
Subject: Assign NaN, get zero
In-Reply-To: <e7ba66e40611111330x4e8f9c65x9db05792cac98fbe@mail.gmail.com>
References: <f4f93d420611111040u3ea1988el6b6baf01aba4891e@mail.gmail.com>
	<20061111203639.GB22096@mentat.za.net>
	<e7ba66e40611111330x4e8f9c65x9db05792cac98fbe@mail.gmail.com>
Message-ID: <20061111215448.GD23231@mentat.za.net>

On Sat, Nov 11, 2006 at 06:30:06PM -0300, Lisandro Dalcin wrote:
> On 11/11/06, Stefan van der Walt <stefan at sun.ac.za> wrote:
> > NaN (or inf) is a floating point number, so seeing a zero in integer
> > representation seems correct:
> >
> > In [2]: int(N.nan)
> > Out[2]: 0L
> >
> 
> Just to learn myself: Why int(N.nan) should be 0? Is it C behavior?

As far as I know (and please correct me if I'm wrong), nan's are just
a specific bit pattern set in memory when an invalid floating point
operation occurs (in IEEE 754 nan's are represented by an exponent of
all 1's and a non-zero mantissa).

Most integer representations have no way of indication an invalid
result (and C provides no such conversion, as far as I am aware), so
nan's are interpreted as 0 (which could have been any arbitrary number
for that matter, although 0 seems a logical choice).

Regards
St?fan

-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From kwgoodman at gmail.com  Sat Nov 11 16:59:40 2006
From: kwgoodman at gmail.com (Keith Goodman)
Date: Sat, 11 Nov 2006 13:59:40 -0800
Subject: Assign NaN, get zero
In-Reply-To: <20061111203639.GB22096@mentat.za.net>
References: <f4f93d420611111040u3ea1988el6b6baf01aba4891e@mail.gmail.com>
	<20061111203639.GB22096@mentat.za.net>
Message-ID: <f4f93d420611111359v1f95f928sa2f16eec54111aea@mail.gmail.com>

On 11/11/06, Stefan van der Walt <stefan at sun.ac.za> wrote:
> On Sat, Nov 11, 2006 at 10:40:22AM -0800, Keith Goodman wrote:
> > I accidentally wrote a unit test using int32 instead of float64 and
> > ran into this problem:
> >
> > >> x = M.matrix([[1, 2, 3]])
> > >> x[0,1] = M.nan
> > >> x
> > matrix([[1, 0, 3]])   <--- Got 0 instead of NaN
> >
> > But this, of course, works:
> >
> > >> x = M.matrix([[1.0, 2.0, 3.0]])
> > >> x[0,1] = M.nan
> > >> x
> > matrix([[ 1.        ,         nan,  3.        ]])
> >
> > Is returning a 0 instead of NaN the expected behavior?
>
> NaN (or inf) is a floating point number, so seeing a zero in integer
> representation seems correct:
>
> In [2]: int(N.nan)
> Out[2]: 0L

Would it make sense to upcast instead of downcast?

This upcasts:

>> x = M.matrix([[1, M.nan, 3]])
>> x
matrix([[ 1.        ,         nan,  3.        ]])

But this doesn't:

>> x = M.matrix([[1, 2, 3]])
>> x[0,1] = M.nan
>> x
matrix([[1, 0, 3]])

(BTW, how do you represent missing integers if you can't use NaN?)

-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From stefan at sun.ac.za  Sat Nov 11 17:10:46 2006
From: stefan at sun.ac.za (Stefan van der Walt)
Date: Sun, 12 Nov 2006 00:10:46 +0200
Subject: Assign NaN, get zero
In-Reply-To: <f4f93d420611111359v1f95f928sa2f16eec54111aea@mail.gmail.com>
References: <f4f93d420611111040u3ea1988el6b6baf01aba4891e@mail.gmail.com>
	<20061111203639.GB22096@mentat.za.net>
	<f4f93d420611111359v1f95f928sa2f16eec54111aea@mail.gmail.com>
Message-ID: <20061111221046.GE23231@mentat.za.net>

On Sat, Nov 11, 2006 at 01:59:40PM -0800, Keith Goodman wrote:
> Would it make sense to upcast instead of downcast?
> 
> This upcasts:
> 
> >> x = M.matrix([[1, M.nan, 3]])
> >> x
> matrix([[ 1.        ,         nan,  3.        ]])
> 
> But this doesn't:
> 
> >> x = M.matrix([[1, 2, 3]])
> >> x[0,1] = M.nan
> >> x
> matrix([[1, 0, 3]])

This behaviour is consistent with

x = N.array([[1,2.0,3]])

vs

x = N.array([1,2,3])
x[0,1] = 2.

> (BTW, how do you represent missing integers if you can't use NaN?)

I think masked arrays should work on integer arrays (alternatively, if
you have enough memory, cast your array to float).

Regards
St?fan

-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From koara at atlas.cz  Sat Nov 11 17:41:47 2006
From: koara at atlas.cz (koara)
Date: Sat, 11 Nov 2006 14:41:47 -0800
Subject: sparse SVD
Message-ID: <1163284907.662729.265470@m73g2000cwd.googlegroups.com>

Hello, is there a way to do SVD of large sparse matrices efficiently in
python? All i found was scipy (little sparse support, no SVD), pysparse
(no SVD) and PROPACK (no python). Out of these, PROPACK seems the most
enticing choice, but i have no experience porting fortran code and this
seems too big a bite. Any pointers or suggestions are most welcome!

Cheers.


-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From peridot.faceted at gmail.com  Sat Nov 11 17:57:24 2006
From: peridot.faceted at gmail.com (A. M. Archibald)
Date: Sat, 11 Nov 2006 17:57:24 -0500
Subject: sparse SVD
In-Reply-To: <1163284907.662729.265470@m73g2000cwd.googlegroups.com>
References: <1163284907.662729.265470@m73g2000cwd.googlegroups.com>
Message-ID: <ce557a360611111457j51be99a0j2e33db2fe5e27cbd@mail.gmail.com>

On 11/11/06, koara <koara at atlas.cz> wrote:
> Hello, is there a way to do SVD of large sparse matrices efficiently in
> python? All i found was scipy (little sparse support, no SVD), pysparse
> (no SVD) and PROPACK (no python). Out of these, PROPACK seems the most
> enticing choice, but i have no experience porting fortran code and this
> seems too big a bite. Any pointers or suggestions are most welcome!

Numpy includes the program "f2py" for generating python wrappers for
fortran code; it's not too difficult to use, even for a non-fortran
programmer like me.

If you do write wrappers, please send them to the scipy list - it's
always good to build up the library.

A. M. Archibald

-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From kwgoodman at gmail.com  Sat Nov 11 18:04:29 2006
From: kwgoodman at gmail.com (Keith Goodman)
Date: Sat, 11 Nov 2006 15:04:29 -0800
Subject: x.min() depends on ordering
Message-ID: <f4f93d420611111504k77908fdfqa630adf6138025c7@mail.gmail.com>

x.min() and x.max() depend on the ordering of the elements:

>> x = M.matrix([[ M.nan, 2.0, 1.0]])
>> x.min()
nan

>> x = M.matrix([[ 1.0, 2.0, M.nan]])
>> x.min()
1.0

If I were to try the latter in ipython, I'd assume, great, min()
ignores NaNs. But then the former would be a bug in my program.

Is this related to how sort works?

>> x = M.matrix([[ M.nan, 2.0, 1.0]])
>> x.sort()
>> x
matrix([[                  nan,            1.        ,            2.        ]])

>> x = M.matrix([[ 1.0, 2.0, M.nan]])
>> x.sort()
>> x
matrix([[ 1.        ,  2.        ,         nan]])

-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From robert.kern at gmail.com  Sat Nov 11 18:14:30 2006
From: robert.kern at gmail.com (Robert Kern)
Date: Sat, 11 Nov 2006 17:14:30 -0600
Subject: x.min() depends on ordering
In-Reply-To: <f4f93d420611111504k77908fdfqa630adf6138025c7@mail.gmail.com>
References: <f4f93d420611111504k77908fdfqa630adf6138025c7@mail.gmail.com>
Message-ID: <ej5lh1$f4p$1@sea.gmane.org>

Keith Goodman wrote:
> x.min() and x.max() depend on the ordering of the elements:
> 
>>> x = M.matrix([[ M.nan, 2.0, 1.0]])
>>> x.min()
> nan
> 
>>> x = M.matrix([[ 1.0, 2.0, M.nan]])
>>> x.min()
> 1.0
> 
> If I were to try the latter in ipython, I'd assume, great, min()
> ignores NaNs. But then the former would be a bug in my program.
> 
> Is this related to how sort works?

Not really. sort() is a more complicated algorithm that does a number of
different comparisons in an order that is difficult to determine beforehand.
x.min() should just be a straight pass through all of the elements. However, the
core problem is the same: a < nan, a > nan, a == nan are all False for any a.

Barring a clever solution (at least cleverer than I feel like being
immediately), the way to solve this would be to check for nans in the array and
deal with them separately (or simply ignore them in the case of x.min()).
However, this checking would slow down the common case that has no nans (sans
nans, if you will).

-- 
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless enigma
 that is made terrible by our own mad attempt to interpret it as though it had
 an underlying truth."
  -- Umberto Eco


-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From kwgoodman at gmail.com  Sat Nov 11 18:25:12 2006
From: kwgoodman at gmail.com (Keith Goodman)
Date: Sat, 11 Nov 2006 15:25:12 -0800
Subject: x.min() depends on ordering
In-Reply-To: <ej5lh1$f4p$1@sea.gmane.org>
References: <f4f93d420611111504k77908fdfqa630adf6138025c7@mail.gmail.com>
	<ej5lh1$f4p$1@sea.gmane.org>
Message-ID: <f4f93d420611111525o76503decp130a312742ef7922@mail.gmail.com>

On 11/11/06, Robert Kern <robert.kern at gmail.com> wrote:

> Barring a clever solution (at least cleverer than I feel like being
> immediately), the way to solve this would be to check for nans in the array and
> deal with them separately (or simply ignore them in the case of x.min()).
> However, this checking would slow down the common case that has no nans
> (sans nans, if you will).

I'm not one of the fans of sans nans.

I'd prefer a slower min() that ignored nans. But I'm probably in the minority.

How about a nanmin() function?

-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From charlesr.harris at gmail.com  Sat Nov 11 18:35:43 2006
From: charlesr.harris at gmail.com (Charles R Harris)
Date: Sat, 11 Nov 2006 16:35:43 -0700
Subject: x.min() depends on ordering
In-Reply-To: <ej5lh1$f4p$1@sea.gmane.org>
References: <f4f93d420611111504k77908fdfqa630adf6138025c7@mail.gmail.com>
	<ej5lh1$f4p$1@sea.gmane.org>
Message-ID: <e06186140611111535u7d7c273ek2ac0aa35fc82c22@mail.gmail.com>

On 11/11/06, Robert Kern <robert.kern at gmail.com> wrote:
>
> Keith Goodman wrote:
> > x.min() and x.max() depend on the ordering of the elements:
> >
> >>> x = M.matrix([[ M.nan, 2.0, 1.0]])
> >>> x.min()
> > nan
> >
> >>> x = M.matrix([[ 1.0, 2.0, M.nan]])
> >>> x.min()
> > 1.0
> >
> > If I were to try the latter in ipython, I'd assume, great, min()
> > ignores NaNs. But then the former would be a bug in my program.
> >
> > Is this related to how sort works?
>
> Not really. sort() is a more complicated algorithm that does a number of
> different comparisons in an order that is difficult to determine
> beforehand.
> x.min() should just be a straight pass through all of the elements.
> However, the
> core problem is the same: a < nan, a > nan, a == nan are all False for any
> a.
>
> Barring a clever solution (at least cleverer than I feel like being
> immediately), the way to solve this would be to check for nans in the
> array and
> deal with them separately (or simply ignore them in the case of x.min()).
> However, this checking would slow down the common case that has no nans
> (sans
> nans, if you will).


I think the problem is that the max and min functions use the first value in
the array as the starting point. That could be fixed by using the first
non-nan and returning nan if there aren't any "real" numbers.  But it
probably isn't worth the effort as the behavior becomes more complicated. A
better rule of thumb is to note that comparisons involving nans are
basically invalid because nans aren't comparable -- the comparison violates
trichotomy. Don't really know what to do about that.

Chuck
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20061111/a86bfdc6/attachment.html>
-------------- next part --------------
-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
-------------- next part --------------
_______________________________________________
Numpy-discussion mailing list
Numpy-discussion at lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/numpy-discussion

From robert.kern at gmail.com  Sat Nov 11 18:39:13 2006
From: robert.kern at gmail.com (Robert Kern)
Date: Sat, 11 Nov 2006 17:39:13 -0600
Subject: x.min() depends on ordering
In-Reply-To: <f4f93d420611111525o76503decp130a312742ef7922@mail.gmail.com>
References: <f4f93d420611111504k77908fdfqa630adf6138025c7@mail.gmail.com>	<ej5lh1$f4p$1@sea.gmane.org>
	<f4f93d420611111525o76503decp130a312742ef7922@mail.gmail.com>
Message-ID: <ej5mvc$ibm$1@sea.gmane.org>

Keith Goodman wrote:
> How about a nanmin() function?

Already there.

In [2]: nanmin?
Type:           function
Base Class:     <type 'function'>
Namespace:      Interactive
File:
/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/site-packages/numpy-1.0.1.dev3432-py2.5-macosx-10.4-i386.egg/numpy/lib/function_base.py
Definition:     nanmin(a, axis=None)
Docstring:
    Find the minimium over the given axis, ignoring NaNs.

-- 
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless enigma
 that is made terrible by our own mad attempt to interpret it as though it had
 an underlying truth."
  -- Umberto Eco


-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From tim.hochberg at ieee.org  Sat Nov 11 18:46:47 2006
From: tim.hochberg at ieee.org (Tim Hochberg)
Date: Sat, 11 Nov 2006 16:46:47 -0700
Subject: x.min() depends on ordering
In-Reply-To: <ej5lh1$f4p$1@sea.gmane.org>
References: <f4f93d420611111504k77908fdfqa630adf6138025c7@mail.gmail.com>
	<ej5lh1$f4p$1@sea.gmane.org>
Message-ID: <455660E7.2010905@ieee.org>

Robert Kern wrote:
> Keith Goodman wrote:
>   
>> x.min() and x.max() depend on the ordering of the elements:
>>
>>     
>>>> x = M.matrix([[ M.nan, 2.0, 1.0]])
>>>> x.min()
>>>>         
>> nan
>>
>>     
>>>> x = M.matrix([[ 1.0, 2.0, M.nan]])
>>>> x.min()
>>>>         
>> 1.0
>>
>> If I were to try the latter in ipython, I'd assume, great, min()
>> ignores NaNs. But then the former would be a bug in my program.
>>
>> Is this related to how sort works?
>>     
>
> Not really. sort() is a more complicated algorithm that does a number of
> different comparisons in an order that is difficult to determine beforehand.
> x.min() should just be a straight pass through all of the elements. However, the
> core problem is the same: a < nan, a > nan, a == nan are all False for any a.
>
> Barring a clever solution (at least cleverer than I feel like being
> immediately), the way to solve this would be to check for nans in the array and
> deal with them separately (or simply ignore them in the case of x.min()).
> However, this checking would slow down the common case that has no nans (sans
> nans, if you will).
>   
For ignoring NaNs, isn't is simply a matter of scanning through the 
array till you find the first non NaN the proceeding as normal? In the 
common case, this requires one extra compare (or rather is_nan) which 
should be negligible in most circumstances. Only when you have an array 
with a load of NaNs at the beginning would it be slow. One would have to 
decide whether to return NaN or raise an error when there were no real 
numbers.

My preference would be to raise an error / warning when there is a nan 
in the array. Technically, there is no minimum value when a nan is 
present. I believe that this would be feasible be swapping the compare 
from 'a < b' to '!(a >= b)'. This should return NaN if any NaNs are 
present and I suspect the extra '!' will have minimal performance impact 
but it would have to be tested. Then a warning or error could be issued 
on the way out depending on the erstate. Arguably returning NaN is more 
correct than returning the smallest non NaN anyway.

As for Keith Goodmans request for a NaN ignoring min function, I suggest:

    a[~np.isnan(a)].min()


Or better yet, stop generating so many NaN's.

-tim




-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From kwgoodman at gmail.com  Sat Nov 11 18:52:31 2006
From: kwgoodman at gmail.com (Keith Goodman)
Date: Sat, 11 Nov 2006 15:52:31 -0800
Subject: x.min() depends on ordering
In-Reply-To: <ej5mvc$ibm$1@sea.gmane.org>
References: <f4f93d420611111504k77908fdfqa630adf6138025c7@mail.gmail.com>
	<ej5lh1$f4p$1@sea.gmane.org>
	<f4f93d420611111525o76503decp130a312742ef7922@mail.gmail.com>
	<ej5mvc$ibm$1@sea.gmane.org>
Message-ID: <f4f93d420611111552n3e523dd5p77a23d323451c2a5@mail.gmail.com>

On 11/11/06, Robert Kern <robert.kern at gmail.com> wrote:
> Keith Goodman wrote:
> > How about a nanmin() function?
>
> Already there.
>
> In [2]: nanmin?
> Type:           function
> Base Class:     <type 'function'>
> Namespace:      Interactive
> File:
> /Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/site-packages/numpy-1.0.1.dev3432-py2.5-macosx-10.4-i386.egg/numpy/lib/function_base.py
> Definition:     nanmin(a, axis=None)
> Docstring:
>     Find the minimium over the given axis, ignoring NaNs.

Thank you! I was just about to write my own using Chuck's email as a guide.

-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From peridot.faceted at gmail.com  Sat Nov 11 18:58:36 2006
From: peridot.faceted at gmail.com (A. M. Archibald)
Date: Sat, 11 Nov 2006 18:58:36 -0500
Subject: x.min() depends on ordering
In-Reply-To: <e06186140611111535u7d7c273ek2ac0aa35fc82c22@mail.gmail.com>
References: <f4f93d420611111504k77908fdfqa630adf6138025c7@mail.gmail.com>
	<ej5lh1$f4p$1@sea.gmane.org>
	<e06186140611111535u7d7c273ek2ac0aa35fc82c22@mail.gmail.com>
Message-ID: <ce557a360611111558k61027bf0v67cb7dbb01f8c5a2@mail.gmail.com>

On 11/11/06, Charles R Harris <charlesr.harris at gmail.com> wrote:

> I think the problem is that the max and min functions use the first value in
> the array as the starting point. That could be fixed by using the first
> non-nan and returning nan if there aren't any "real" numbers.  But it
> probably isn't worth the effort as the behavior becomes more complicated. A
> better rule of thumb is to note that comparisons involving nans are
> basically invalid because nans aren't comparable -- the comparison violates
> trichotomy. Don't really know what to do about that.

Well, we could get simple consistent behaviour by taking inf as the
initial value for min and -inf as the first value for max, then
reducing as normal. This would then, depending on how max and min are
implemented, either return NaN if any are present, or return the
smallest/largest non-NaN value (or inf/-inf if there are none)

A. M. Archibald

-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From charlesr.harris at gmail.com  Sat Nov 11 19:09:44 2006
From: charlesr.harris at gmail.com (Charles R Harris)
Date: Sat, 11 Nov 2006 17:09:44 -0700
Subject: x.min() depends on ordering
In-Reply-To: <455660E7.2010905@ieee.org>
References: <f4f93d420611111504k77908fdfqa630adf6138025c7@mail.gmail.com>
	<ej5lh1$f4p$1@sea.gmane.org> <455660E7.2010905@ieee.org>
Message-ID: <e06186140611111609m66afb5dey7a71679bcccc0a09@mail.gmail.com>

On 11/11/06, Tim Hochberg <tim.hochberg at ieee.org> wrote:
>
> Robert Kern wrote:


<snip>

My preference would be to raise an error / warning when there is a nan
> in the array. Technically, there is no minimum value when a nan is
> present. I believe that this would be feasible be swapping the compare
> from 'a < b' to '!(a >= b)'. This should return NaN if any NaNs are
> present and I suspect the extra '!' will have minimal performance impact
> but it would have to be tested. Then a warning or error could be issued
> on the way out depending on the erstate. Arguably returning NaN is more
> correct than returning the smallest non NaN anyway.
>

No telling what compiler optimizations might do with  '!(a >= b)' if they
assume that '!(a >= b)' == 'a < b'. For instance,

if !(a >= b)
    do something;
else
    do otherwise;

might branch to the second statement on  'a <b' and fall through to the
first otherwise.

Chuck
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20061111/272945c6/attachment.html>
-------------- next part --------------
-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
-------------- next part --------------
_______________________________________________
Numpy-discussion mailing list
Numpy-discussion at lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/numpy-discussion

From charlesr.harris at gmail.com  Sat Nov 11 19:49:21 2006
From: charlesr.harris at gmail.com (Charles R Harris)
Date: Sat, 11 Nov 2006 17:49:21 -0700
Subject: x.min() depends on ordering
In-Reply-To: <e06186140611111609m66afb5dey7a71679bcccc0a09@mail.gmail.com>
References: <f4f93d420611111504k77908fdfqa630adf6138025c7@mail.gmail.com>
	<ej5lh1$f4p$1@sea.gmane.org> <455660E7.2010905@ieee.org>
	<e06186140611111609m66afb5dey7a71679bcccc0a09@mail.gmail.com>
Message-ID: <e06186140611111649o52843c6ey80b9751096e90a14@mail.gmail.com>

On 11/11/06, Charles R Harris <charlesr.harris at gmail.com> wrote:
>
>
>
> On 11/11/06, Tim Hochberg <tim.hochberg at ieee.org> wrote:
> >
> > Robert Kern wrote:
>
>
> <snip>
>
> My preference would be to raise an error / warning when there is a nan
> > in the array. Technically, there is no minimum value when a nan is
> > present. I believe that this would be feasible be swapping the compare
> > from 'a < b' to '!(a >= b)'. This should return NaN if any NaNs are
> > present and I suspect the extra '!' will have minimal performance impact
> > but it would have to be tested. Then a warning or error could be issued
> > on the way out depending on the erstate. Arguably returning NaN is more
> > correct than returning the smallest non NaN anyway.
> >
>
> No telling what compiler optimizations might do with  '!(a >= b)' if they
> assume that '!(a >= b)' == 'a < b'. For instance,
>
> if !(a >= b)
>     do something;
> else
>     do otherwise;
>

This made me curious. Here is the code

int test(double a, double b)
{
    if (a > b)
        return 0;
    return 1;
}

Here is the relevant part of the assembly code when compiled with no
optimizations

        fucompp
        fnstsw  %ax
        sahf
        ja      .L4
        jmp     .L2
.L4:
        movl    $0, -20(%ebp)
        jmp     .L5
.L2:
        movl    $1, -20(%ebp)
.L5:
        movl    -20(%ebp), %eax
        leave
        ret

Which jumps to the right place on a > b (ja)

Here is the relevant part of the assembly code when compiled with -O3

        fucompp
        fnstsw  %ax
        popl    %ebp
        sahf
        setbe   %al
        movzbl  %al, %eax
        ret

Which sets the value of the return to the logical value of a <= b (setbe),
which won't work right with NaNs. Maybe the compiler needs another flag to
deal with the possibility of NaN's because the generated code is actually
incorrect. Or maybe I just discovered a compiler bug. But boy, that compiler
is awesome clever. Those optimizers are getting better all the time.

Chuck
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20061111/f55488d4/attachment.html>
-------------- next part --------------
-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
-------------- next part --------------
_______________________________________________
Numpy-discussion mailing list
Numpy-discussion at lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/numpy-discussion

From tim.hochberg at ieee.org  Sat Nov 11 19:49:26 2006
From: tim.hochberg at ieee.org (Tim Hochberg)
Date: Sat, 11 Nov 2006 17:49:26 -0700
Subject: x.min() depends on ordering
In-Reply-To: <ej5mvc$ibm$1@sea.gmane.org>
References: <f4f93d420611111504k77908fdfqa630adf6138025c7@mail.gmail.com>	<ej5lh1$f4p$1@sea.gmane.org>	<f4f93d420611111525o76503decp130a312742ef7922@mail.gmail.com>
	<ej5mvc$ibm$1@sea.gmane.org>
Message-ID: <45566F96.2090603@ieee.org>

Robert Kern wrote:
> Keith Goodman wrote:
>   
>> How about a nanmin() function?
>>     
>
> Already there.
>
> In [2]: nanmin?
> Type:           function
> Base Class:     <type 'function'>
> Namespace:      Interactive
> File:
> /Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/site-packages/numpy-1.0.1.dev3432-py2.5-macosx-10.4-i386.egg/numpy/lib/function_base.py
> Definition:     nanmin(a, axis=None)
> Docstring:
>     Find the minimium over the given axis, ignoring NaNs.
>
>   
Bah! That seems excessive. If I was king of the world one of the many 
things that I would stuff into a submodule is all the stuff relating to 
special values (nan, inf, isnan, isinf, namin, nanmax, nansum, 
nanargmax, nanargmin, nan_to_num, infty,  isneginf, isposinf and 
probably some others that I'm missing). First, these are mostly clutter 
in the base namespace. Not only would it make the main namespace easier 
to navigate in (although there's much clean up that would have to be 
done before it would be anything approaching easy to navigate). Second, 
for those who actually do need them, they'd be easier to find if they 
were all grouped together -- Keith for example would almost certainly 
have immediately found nanmin. Third, and this is perhaps a matter of 
opinion, there seems to be a sudden urge to abuse NaNs. Perhaps if they 
were shunted a bit off to the side, this temptation would be lifted.

Curmudgeonly yours,

-tim


-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From charlesr.harris at gmail.com  Sat Nov 11 20:06:57 2006
From: charlesr.harris at gmail.com (Charles R Harris)
Date: Sat, 11 Nov 2006 18:06:57 -0700
Subject: x.min() depends on ordering
In-Reply-To: <45566F96.2090603@ieee.org>
References: <f4f93d420611111504k77908fdfqa630adf6138025c7@mail.gmail.com>
	<ej5lh1$f4p$1@sea.gmane.org>
	<f4f93d420611111525o76503decp130a312742ef7922@mail.gmail.com>
	<ej5mvc$ibm$1@sea.gmane.org> <45566F96.2090603@ieee.org>
Message-ID: <e06186140611111706u34509991j9f08dbedcdeddce@mail.gmail.com>

On 11/11/06, Tim Hochberg <tim.hochberg at ieee.org> wrote:
>
> Robert Kern wrote:
> > Keith Goodman wrote:
> >
> >> How about a nanmin() function?
> >>
> >
> > Already there.
> >
> > In [2]: nanmin?
> > Type:           function
> > Base Class:     <type 'function'>
> > Namespace:      Interactive
> > File:
> >
> /Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/site-packages/numpy-
> 1.0.1.dev3432-py2.5-macosx-10.4-i386.egg/numpy/lib/function_base.py
> > Definition:     nanmin(a, axis=None)
> > Docstring:
> >     Find the minimium over the given axis, ignoring NaNs.
> >
> >
> Bah! That seems excessive. If I was king of the world one of the many
> things that I would stuff into a submodule is all the stuff relating to
> special values (nan, inf, isnan, isinf, namin, nanmax, nansum,
> nanargmax, nanargmin, nan_to_num, infty,  isneginf, isposinf and
> probably some others that I'm missing). First, these are mostly clutter
> in the base namespace. Not only would it make the main namespace easier
> to navigate in (although there's much clean up that would have to be
> done before it would be anything approaching easy to navigate). Second,
> for those who actually do need them, they'd be easier to find if they
> were all grouped together -- Keith for example would almost certainly
> have immediately found nanmin. Third, and this is perhaps a matter of
> opinion, there seems to be a sudden urge to abuse NaNs. Perhaps if they
> were shunted a bit off to the side, this temptation would be lifted.
>
> Curmudgeonly yours,


Oh yes. And let's reserve a bit of abuse for whoever mixed up the nans with
the rest. I mean, the infs and such actually make some sense as numbers, but
nans are, well, nans. So it would have been nice to enable everything
*except* nans, and have an errorflag set whenever the latter turned up.  Or
if folks just have to have nans, make them compare less than anything else.
If isnan were fast and easy one could use LT(a,b) := isnan(a) || a < b.

Chuck
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20061111/1535e69f/attachment.html>
-------------- next part --------------
-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
-------------- next part --------------
_______________________________________________
Numpy-discussion mailing list
Numpy-discussion at lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/numpy-discussion

From tim.hochberg at ieee.org  Sat Nov 11 20:55:21 2006
From: tim.hochberg at ieee.org (Tim Hochberg)
Date: Sat, 11 Nov 2006 18:55:21 -0700
Subject: x.min() depends on ordering
In-Reply-To: <e06186140611111706u34509991j9f08dbedcdeddce@mail.gmail.com>
References: <f4f93d420611111504k77908fdfqa630adf6138025c7@mail.gmail.com>	<ej5lh1$f4p$1@sea.gmane.org>	<f4f93d420611111525o76503decp130a312742ef7922@mail.gmail.com>	<ej5mvc$ibm$1@sea.gmane.org>
	<45566F96.2090603@ieee.org>
	<e06186140611111706u34509991j9f08dbedcdeddce@mail.gmail.com>
Message-ID: <45567F09.3030504@ieee.org>

Charles R Harris wrote:
>
>
> On 11/11/06, *Tim Hochberg* <tim.hochberg at ieee.org 
> <mailto:tim.hochberg at ieee.org>> wrote:
>
>     Robert Kern wrote:
>     > Keith Goodman wrote:
>     >
>     >> How about a nanmin() function?
>     >>
>     >
>     > Already there.
>     >
>     > In [2]: nanmin?
>     > Type:           function
>     > Base Class:     <type 'function'>
>     > Namespace:      Interactive
>     > File:
>     >
>     /Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/site-packages/numpy-1.0.1.dev3432-py2.5-macosx-10.4-i386.egg/numpy/lib/function_base.py
>     > Definition:     nanmin(a, axis=None)
>     > Docstring:
>     >     Find the minimium over the given axis, ignoring NaNs.
>     >
>     >
>     Bah! That seems excessive. If I was king of the world one of the many
>     things that I would stuff into a submodule is all the stuff
>     relating to
>     special values (nan, inf, isnan, isinf, namin, nanmax, nansum,
>     nanargmax, nanargmin, nan_to_num, infty,  isneginf, isposinf and
>     probably some others that I'm missing). First, these are mostly
>     clutter
>     in the base namespace. Not only would it make the main namespace
>     easier
>     to navigate in (although there's much clean up that would have to be
>     done before it would be anything approaching easy to navigate).
>     Second,
>     for those who actually do need them, they'd be easier to find if they
>     were all grouped together -- Keith for example would almost certainly
>     have immediately found nanmin. Third, and this is perhaps a matter of
>     opinion, there seems to be a sudden urge to abuse NaNs. Perhaps if
>     they
>     were shunted a bit off to the side, this temptation would be lifted.
>
>     Curmudgeonly yours,
>
>
> Oh yes. And let's reserve a bit of abuse for whoever mixed up the nans 
> with the rest. I mean, the infs and such actually make some sense as 
> numbers, but nans are, well, nans. So it would have been nice to 
> enable everything *except* nans, and have an errorflag set whenever 
> the latter turned up.
Actually you can do this. At least for the most part, I'm sure there are 
some corners that aren't working right yet.  Operations on NaNs set the 
invalid flag while infinities set the overflow flag (actually the flags 
are only set when you first get the infinity or NaN, at least on my 
platform). So, you can make invalid a flag and ignore overflow by using:

np.seterr(over='ignore', invalid='raise')

I doubt this does *exactly* what you want, but it may be close.

-tim


[snip]




-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From fingerslip at yahoo.com  Sun Nov 12 02:17:32 2006
From: fingerslip at yahoo.com (Abhishek Roy)
Date: Sun, 12 Nov 2006 07:17:32 +0000 (GMT)
Subject: Compiling numpy with lapack and atlas
Message-ID: <20061112071732.84470.qmail@web38906.mail.mud.yahoo.com>

Hello,
      I am trying to compile Numpy with lapack & atlas. In site.cfg I put,
[atlas]
library_dirs = /usr/local/lib/atlas/
atlas_libs = cblas, lapack, f77blas, atlas

and installation seems to go well. It says,
  FOUND:
    libraries = ['cblas', 'lapack', 'f77blas', 'atlas']
    library_dirs = ['/usr/local/lib/atlas/']
    language = c
    include_dirs = ['/usr/local/lib/atlas']
and the lines with gcc <something> -lcblas -llapack don't give an error. But
afterwards running,
$ ldd ~/numpy-1.0/build/lib.linux-i686-2.4/numpy/linalg/lapack_lite.so
linux-gate.so.1 =>  (0xffffe000)
libpthread.so.0 => /lib/libpthread.so.0 (0x40238000)
libc.so.6 => /lib/libc.so.6 (0x4028b000)
/lib/ld-linux.so.2 (0x80000000)

which I think means libcblas and libatlas are not linked. Can someone tell me
what's going wrong here?

Thanks,
Abhishek

Send instant messages to your online friends http://uk.messenger.yahoo.com 

-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From robert.kern at gmail.com  Sun Nov 12 02:34:53 2006
From: robert.kern at gmail.com (Robert Kern)
Date: Sun, 12 Nov 2006 01:34:53 -0600
Subject: Compiling numpy with lapack and atlas
In-Reply-To: <20061112071732.84470.qmail@web38906.mail.mud.yahoo.com>
References: <20061112071732.84470.qmail@web38906.mail.mud.yahoo.com>
Message-ID: <ej6ir8$72b$1@sea.gmane.org>

Abhishek Roy wrote:
> Hello,
>       I am trying to compile Numpy with lapack & atlas. In site.cfg I put,
> [atlas]
> library_dirs = /usr/local/lib/atlas/
> atlas_libs = cblas, lapack, f77blas, atlas
> 
> and installation seems to go well. It says,
>   FOUND:
>     libraries = ['cblas', 'lapack', 'f77blas', 'atlas']
>     library_dirs = ['/usr/local/lib/atlas/']
>     language = c
>     include_dirs = ['/usr/local/lib/atlas']
> and the lines with gcc <something> -lcblas -llapack don't give an error. But
> afterwards running,
> $ ldd ~/numpy-1.0/build/lib.linux-i686-2.4/numpy/linalg/lapack_lite.so
> linux-gate.so.1 =>  (0xffffe000)
> libpthread.so.0 => /lib/libpthread.so.0 (0x40238000)
> libc.so.6 => /lib/libc.so.6 (0x4028b000)
> /lib/ld-linux.so.2 (0x80000000)
> 
> which I think means libcblas and libatlas are not linked. Can someone tell me
> what's going wrong here?

Are your ATLAS libraries shared or static (i.e., are they libatlas.so or
libatlas.a)? ldd(1) only lists the shared libraries that are linked. ATLAS is
usually installed as static libraries.

Does that module import? If so, then you have no problems. If not, then copy the
message here. It should tell you one of the symbols that are missing.

Note that libraries should be listed in the order in which they depend on each
other. lapack depends on f77blas and cblas which both depend on atlas.

  lapack,f77blas,cblas,atlas

-- 
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless enigma
 that is made terrible by our own mad attempt to interpret it as though it had
 an underlying truth."
  -- Umberto Eco


-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From lbixeqb at party1.com  Sun Nov 12 02:56:47 2006
From: lbixeqb at party1.com (Center Developing)
Date: Sun, 12 Nov 2006 08:56:47 +0100
Subject: Scan MAC Repair
References: <000a01c70630$1a45d710$00000000@karpik>
Message-ID: <000a01c70630$1a45d710$00000000@karpik>


  ----- Original Message ----- 
  From: numpy-discussion at lists.sourceforge.net 
  To: lbixeqb at party1.com 
  Sent: Monday, November 02:18:15 AM
  Subject: Scan MAC Repair


  Apos Transfer Weigh Benefits Ltpipsec Pros Cons Advantages.
  Ghost Solve is Winktech Enter a event in id Winternals. Csgew a Failure am Launching or Published Citrix Secure Adminpak.
  Geekcom Geek favorite tv show in aposnet Whirlpool of Thread a. Buycom Befvp Etherfast is external am Configure Ipsec.
  Home lan diy Share an Internet Connection in Linksys Cabledsl. Second cpu Virtualdr Forums.
  Right Cracks anytime. Bbs Deleating accessing Pack. Auto Cannot Export Fields.
  Cars greeting Strassberg. Real Audio Midiaposs Multi or.
  Pcdocrx a Oncall fixed is our diagnostic test or.
  Vpnclient or Clienteasy Xauth Split Tunnel Systems a Active.
  Remote User Manual Tutorial Acidcool Cracking Answer. Fiction Trialware United States.
  Befvp is Etherfast external Configure Ipsec Tunneling? Should am Know is Security Bulletin.
  Rules Wizard Contacts or Display Same. Real Audio Midiaposs Multi or.
  Ghost Solve is Winktech Enter a event in id Winternals.
  Guarantee of Appligent Apcrypt finance. Windowsx Filetracer am finds.
  Vobdvdfile Ripping Vobaudio Ssrc Pinoyaposs Neue? Recovery Intuit vpn Solution Find in Virtual.
  Tqm iso six Sigma Projectpix a Visio onproject Webbased in.
  Vobdvdfile Ripping Vobaudio Ssrc Pinoyaposs Neue? Activex Controls ssl Strategy Hotfix Csgew. Password Protection System Lite Version.
  Stopsign tm Stopsigntm Escorcher Present. Developing Effective Level Management.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20061112/b6c47d2d/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: Resource.gif
Type: image/gif
Size: 14125 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20061112/b6c47d2d/attachment.gif>
-------------- next part --------------
-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
-------------- next part --------------
_______________________________________________
Numpy-discussion mailing list
Numpy-discussion at lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/numpy-discussion

From ndbecker2 at gmail.com  Sun Nov 12 06:25:29 2006
From: ndbecker2 at gmail.com (Neal Becker)
Date: Sun, 12 Nov 2006 06:25:29 -0500
Subject: Compiling numpy with lapack and atlas
References: <20061112071732.84470.qmail@web38906.mail.mud.yahoo.com>
Message-ID: <ej70b5$17a$2@sea.gmane.org>

I see you're on linux.  If this is Fedora, atlas (and the other libaries)
are available from standard repos (core + extras)


-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From gael.varoquaux at normalesup.org  Sun Nov 12 10:51:02 2006
From: gael.varoquaux at normalesup.org (Gael Varoquaux)
Date: Sun, 12 Nov 2006 16:51:02 +0100
Subject: Helper function to "unroll" a array
In-Reply-To: <20061020112848.GB3555@clipper.ens.fr>
References: <20061020112848.GB3555@clipper.ens.fr>
Message-ID: <20061112155102.GB29963@clipper.ens.fr>

    Hi all,

I didn't get any answers to this email. Is it because the proposed
addition to numpy is not of any interest to anybody apart from me ?
Maybe the way I introduced this is wrong. Please tell me what is wrong
with this proposition.

Regards,

Ga?l

On Fri, Oct 20, 2006 at 01:28:52PM +0200, Gael Varoquaux wrote:
> Hi,

> There is an operation I do a lot, I would call it "unrolling" a array.
> The best way to describe it is probably to give the code:

> def unroll(M):
>     """ Flattens the array M and returns a 2D array with the first columns 
> 	being the indices of M, and the last column the flatten M.
>     """
>     return hstack((indices(M.shape).reshape(-1,M.ndim),M.reshape(-1,1)))

> Example:

> >>> M
> array([[ 0.73530097,  0.3553424 ,  0.3719772 ],
>        [ 0.83353373,  0.74622133,  0.14748905],
>        [ 0.72023762,  0.32306969,  0.19142366]])

> >>> unroll(M)
> array([[ 0.        ,  0.        ,  0.73530097],
>        [ 0.        ,  1.        ,  0.3553424 ],
>        [ 1.        ,  1.        ,  0.3719772 ],
>        [ 2.        ,  2.        ,  0.83353373],
>        [ 2.        ,  0.        ,  0.74622133],
>        [ 1.        ,  2.        ,  0.14748905],
>        [ 0.        ,  1.        ,  0.72023762],
>        [ 2.        ,  0.        ,  0.32306969],
>        [ 1.        ,  2.        ,  0.19142366]])


> The docstring sucks. The function is trivial (when you know numpy a bit).
> Maybe this function already exists in numpy, if so I couldn't find it.
> Elsewhere I propose it for inclusion.

> Cheers,

> Ga?l

> -------------------------------------------------------------------------
> Using Tomcat but need to do more? Need to support web services, security?
> Get stuff done quickly with pre-integrated technology to make your job easier
> Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
> http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
> _______________________________________________
> Numpy-discussion mailing list
> Numpy-discussion at lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/numpy-discussion

-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From svetosch at gmx.net  Sun Nov 12 12:07:30 2006
From: svetosch at gmx.net (Sven Schreiber)
Date: Sun, 12 Nov 2006 18:07:30 +0100
Subject: Helper function to "unroll" a array
In-Reply-To: <20061112155102.GB29963@clipper.ens.fr>
References: <20061020112848.GB3555@clipper.ens.fr>
	<20061112155102.GB29963@clipper.ens.fr>
Message-ID: <455754D2.3050309@gmx.net>

Gael Varoquaux schrieb:
>     Hi all,
> 
> I didn't get any answers to this email. Is it because the proposed
> addition to numpy is not of any interest to anybody apart from me ?
> Maybe the way I introduced this is wrong. Please tell me what is wrong
> with this proposition.
> 

Well you didn't mentioned why it would be useful for a broad audience,
only that you yourself use it alot. Together with the fact that it's a
one-liner this presumably means that it's not exactly a prime candidate
for adoption. (I'm just another user, so I'm speculating a bit here
based on my reading of past postings on this list.)
-sven

-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From gael.varoquaux at normalesup.org  Sun Nov 12 12:12:31 2006
From: gael.varoquaux at normalesup.org (Gael Varoquaux)
Date: Sun, 12 Nov 2006 18:12:31 +0100
Subject: Helper function to "unroll" a array
In-Reply-To: <455754D2.3050309@gmx.net>
References: <20061020112848.GB3555@clipper.ens.fr>
	<20061112155102.GB29963@clipper.ens.fr> <455754D2.3050309@gmx.net>
Message-ID: <20061112171230.GC29963@clipper.ens.fr>

You're probably right. Well it would most definitely be useful for all
the lads in my lab, but I am not sure this is a broad audience. The use
case is when you have a array representing data in an "mgrid" way, and
you wnat to apply transformations to the coordinates. It is something I
have done and seen done quite often.

If I am the only one a the list who see a use for this, then forget it.

Ga?l

On Sun, Nov 12, 2006 at 06:07:30PM +0100, Sven Schreiber wrote:
> Well you didn't mentioned why it would be useful for a broad audience,
> only that you yourself use it alot. Together with the fact that it's a
> one-liner this presumably means that it's not exactly a prime candidate
> for adoption. (I'm just another user, so I'm speculating a bit here
> based on my reading of past postings on this list.)


-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From george.sakkis at gmail.com  Sun Nov 12 14:16:25 2006
From: george.sakkis at gmail.com (George Sakkis)
Date: Sun, 12 Nov 2006 19:16:25 -0000
Subject: Equivalent to list.index ?
Message-ID: <1163358985.920640.306720@k70g2000cwa.googlegroups.com>

This must be pretty trivial but I couldn't find it in the docs: what's
the "numpythonic" way to find the (first) index of an element, i.e. the
equivalent to list.index ?

Thanks,
George


-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From kwgoodman at gmail.com  Sun Nov 12 14:21:43 2006
From: kwgoodman at gmail.com (Keith Goodman)
Date: Sun, 12 Nov 2006 11:21:43 -0800
Subject: Equivalent to list.index ?
In-Reply-To: <1163358985.920640.306720@k70g2000cwa.googlegroups.com>
References: <1163358985.920640.306720@k70g2000cwa.googlegroups.com>
Message-ID: <f4f93d420611121121y2620c6fdj60241acef01e30b4@mail.gmail.com>

On 11/12/06, George Sakkis <george.sakkis at gmail.com> wrote:
> This must be pretty trivial but I couldn't find it in the docs: what's
> the "numpythonic" way to find the (first) index of an element, i.e. the
> equivalent to list.index ?

Does where work?

-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From kwgoodman at gmail.com  Sun Nov 12 15:12:42 2006
From: kwgoodman at gmail.com (Keith Goodman)
Date: Sun, 12 Nov 2006 12:12:42 -0800
Subject: I can slice but I can't assign
Message-ID: <f4f93d420611121212m5f09f1f3x6790224a09b9a339@mail.gmail.com>

>> x = M.matrix([[1, 2], [3, 4]])
>> x
matrix([[1, 2],
        [3, 4]])

>> y = M.matrix([[5], [6]])
>> y
matrix([[5],
        [6]])

>> idx = M.where(x[:,0].A < 100)[0]
>> idx
array([0, 1])


>> x[idx,0]   # <---- This works
matrix([[1],
        [3]])

>> x[idx,0] = y    # <---- But this doesn't work. How can I make it work?

ValueError: array is not broadcastable to correct shape

-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From kwgoodman at gmail.com  Sun Nov 12 15:18:51 2006
From: kwgoodman at gmail.com (Keith Goodman)
Date: Sun, 12 Nov 2006 12:18:51 -0800
Subject: I can slice but I can't assign
In-Reply-To: <f4f93d420611121212m5f09f1f3x6790224a09b9a339@mail.gmail.com>
References: <f4f93d420611121212m5f09f1f3x6790224a09b9a339@mail.gmail.com>
Message-ID: <f4f93d420611121218n5fef3a3bic134e0bf39be2d3e@mail.gmail.com>

On 11/12/06, Keith Goodman <kwgoodman at gmail.com> wrote:
> >> x = M.matrix([[1, 2], [3, 4]])
> >> x
> matrix([[1, 2],
>         [3, 4]])
>
> >> y = M.matrix([[5], [6]])
> >> y
> matrix([[5],
>         [6]])
>
> >> idx = M.where(x[:,0].A < 100)[0]
> >> idx
> array([0, 1])
>
>
> >> x[idx,0]   # <---- This works
> matrix([[1],
>         [3]])
>
> >> x[idx,0] = y    # <---- But this doesn't work. How can I make it work?
>
> ValueError: array is not broadcastable to correct shape

Awkward, but it works:

>> idx = M.asmatrix(idx)
>> idx
matrix([[0, 1]])
>> idx = idx.T
>> idx
matrix([[0],
        [1]])

>> x[idx,0] = y

But it does seem odd to me that the array idx in the original example
works for a slice but for not an assignment.

-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From george.sakkis at gmail.com  Sun Nov 12 15:19:03 2006
From: george.sakkis at gmail.com (George Sakkis)
Date: Sun, 12 Nov 2006 20:19:03 -0000
Subject: Equivalent to list.index ?
References: <1163358985.920640.306720@k70g2000cwa.googlegroups.com>
	<f4f93d420611121121y2620c6fdj60241acef01e30b4@mail.gmail.com>
Message-ID: <1163362743.775427.260100@f16g2000cwb.googlegroups.com>

Keith Goodman wrote:

> On 11/12/06, George Sakkis <george.sakkis at gmail.com> wrote:
> > This must be pretty trivial but I couldn't find it in the docs: what's
> > the "numpythonic" way to find the (first) index of an element, i.e. the
> > equivalent to list.index ?
>
> Does where work?

def index(a, value):
    return N.where(a==value)[0][0]

This works but seems clunky and less efficient than necessary. If there
isn't a better alternative, I would welcome a new index()
function/method in the next numpy release.

George


-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From fingerslip at yahoo.com  Sun Nov 12 15:36:30 2006
From: fingerslip at yahoo.com (Abhishek Roy)
Date: Sun, 12 Nov 2006 20:36:30 +0000 (GMT)
Subject: Compiling numpy with lapack and atlas
In-Reply-To: <ej6ir8$72b$1@sea.gmane.org>
References: <ej6ir8$72b$1@sea.gmane.org>
Message-ID: <20061112203630.98834.qmail@web38902.mail.mud.yahoo.com>

--- Robert Kern <robert.kern at gmail.com> wrote:
> Are your ATLAS libraries shared or static (i.e., are they libatlas.so or
> libatlas.a)? ldd(1) only lists the shared libraries that are linked. ATLAS is
> usually installed as static libraries.
They are static. Thanks, I was mindlessly copying a suggestion I saw elsewhere.
 
> Does that module import? 
I got,
ImportError: /usr/lib/python2.4/site-packages/numpy/linalg/lapack_lite.so:
undefined symbol: s_cat

even after changing the order of libraries in site.cfg to,
>   lapack,f77blas,cblas,atlas

Abhishek

Send instant messages to your online friends http://uk.messenger.yahoo.com 

-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From tim.hochberg at ieee.org  Sun Nov 12 16:39:42 2006
From: tim.hochberg at ieee.org (Tim Hochberg)
Date: Sun, 12 Nov 2006 14:39:42 -0700
Subject: Equivalent to list.index ?
In-Reply-To: <1163362743.775427.260100@f16g2000cwb.googlegroups.com>
References: <1163358985.920640.306720@k70g2000cwa.googlegroups.com>	<f4f93d420611121121y2620c6fdj60241acef01e30b4@mail.gmail.com>
	<1163362743.775427.260100@f16g2000cwb.googlegroups.com>
Message-ID: <4557949E.5060009@ieee.org>

George Sakkis wrote:
> def index(a, value):
>     return N.where(a==value)[0][0]
>   
Or

def index(a, value):
    return argmax(a == value)


> This works but seems clunky and less efficient than necessary. If there
> isn't a better alternative, I would welcome a new index()
> function/method in the next numpy release.
>
>   

Without commenting on the desirability of this particular function, 
people seem awfully eager to add functions and methods everywhere. These 
aren't free you know. They cost in terms of maintenance and potential 
bugs, and in terms of making the already large function set even harder 
to wrap ones head around. That's partly an organizational problem but 
partly an issue of sheer size, neither of which are helped by adding a 
bunch more stuff.

    Not every two-line Python function has to come pre-written 
        --Tim Peters

In the case of this particular function, what are the use cases? Are 
they common (not just with you but with other numpy users)? Are the uses 
speed critical? Is it a good building block for other numeric 
functions?  I'm skeptical of the above implementation as it doesn't fit 
well with other similar numpy functions (argmin and argmax for example) 
and I personally don't see much in the way of compelling uses for this, 
but feel free to try to convince me otherwise.

-tim







-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From kwgoodman at gmail.com  Sun Nov 12 16:57:17 2006
From: kwgoodman at gmail.com (Keith Goodman)
Date: Sun, 12 Nov 2006 13:57:17 -0800
Subject: Could numpy.matlib.nanmax return a matrix?
Message-ID: <f4f93d420611121357o5dda5950g4fdcdc2f657ee654@mail.gmail.com>

Is anybody interested in making x.max() and nanmax() behave the same
for matrices, except for the NaN part? That is, make
numpy.matlib.nanmax return a matrix instead of an array.

Example
=====

>> import numpy.matlib as M
>> x = M.rand(3,2)

>> x.max(0)
matrix([[ 0.91749823,  0.94942954]])

>> M.nanmax(x, 0)
array([ 0.91749823,  0.94942954])

-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From peridot.faceted at gmail.com  Sun Nov 12 17:08:34 2006
From: peridot.faceted at gmail.com (A. M. Archibald)
Date: Sun, 12 Nov 2006 17:08:34 -0500
Subject: Could numpy.matlib.nanmax return a matrix?
In-Reply-To: <f4f93d420611121357o5dda5950g4fdcdc2f657ee654@mail.gmail.com>
References: <f4f93d420611121357o5dda5950g4fdcdc2f657ee654@mail.gmail.com>
Message-ID: <ce557a360611121408k6bae03fds9b2e979d786bb74e@mail.gmail.com>

On 12/11/06, Keith Goodman <kwgoodman at gmail.com> wrote:
> Is anybody interested in making x.max() and nanmax() behave the same
> for matrices, except for the NaN part? That is, make
> numpy.matlib.nanmax return a matrix instead of an array.

Sounds good to me; maybe put a patch in the tracker?

A. M. Archibald

-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From cjw at sympatico.ca  Sun Nov 12 17:53:59 2006
From: cjw at sympatico.ca (Colin J. Williams)
Date: Sun, 12 Nov 2006 17:53:59 -0500
Subject: Equivalent to list.index ?
In-Reply-To: <4557949E.5060009@ieee.org>
References: <1163358985.920640.306720@k70g2000cwa.googlegroups.com>	<f4f93d420611121121y2620c6fdj60241acef01e30b4@mail.gmail.com>	<1163362743.775427.260100@f16g2000cwb.googlegroups.com>
	<4557949E.5060009@ieee.org>
Message-ID: <4557A607.5060804@sympatico.ca>

An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20061112/6c6bc99f/attachment.html>
-------------- next part --------------
-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
-------------- next part --------------
_______________________________________________
Numpy-discussion mailing list
Numpy-discussion at lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/numpy-discussion

From pgmdevlist at gmail.com  Sun Nov 12 18:08:50 2006
From: pgmdevlist at gmail.com (Pierre GM)
Date: Sun, 12 Nov 2006 18:08:50 -0500
Subject: Could numpy.matlib.nanmax return a matrix?
In-Reply-To: <ce557a360611121408k6bae03fds9b2e979d786bb74e@mail.gmail.com>
References: <f4f93d420611121357o5dda5950g4fdcdc2f657ee654@mail.gmail.com>
	<ce557a360611121408k6bae03fds9b2e979d786bb74e@mail.gmail.com>
Message-ID: <200611121808.51298.pgmdevlist@gmail.com>

On Sunday 12 November 2006 17:08, A. M. Archibald wrote:
> On 12/11/06, Keith Goodman <kwgoodman at gmail.com> wrote:
> > Is anybody interested in making x.max() and nanmax() behave the same
> > for matrices, except for the NaN part? That is, make
> > numpy.matlib.nanmax return a matrix instead of an array.

Or, you could use masked arrays... In the new implementation, you can add a 
mask to a subclassed array (such as matrix) to get a regular masked array. If 
you fill this masked array, you get an array of the same subclass.

>>> import numpy as N
>>> import numpy.matlib as M
>>> import maskedarray as MA
>>> x=M.rand(3,3)
>>> assert isinstance(x.max(0), M.matrix)
>>> assert isinstance(N.max(x,0), M.matrix)
>>> assert  isinstance(MA.max(x,0).filled(0), M.matrix)
>>> assert isinstance(MA.max(x,0)._data, M.matrix)

>>> x[-1,-1] = N.nan
>>> tmp = MA.max(MA.array(x,mask=N.isnan(x)), 0)
>>> assert (tmp == N.nanmax(x,0)).all()
>>> assert isinstance(tmp.filled(0), M.matrix)

-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From robert.kern at gmail.com  Sun Nov 12 18:13:25 2006
From: robert.kern at gmail.com (Robert Kern)
Date: Sun, 12 Nov 2006 17:13:25 -0600
Subject: Compiling numpy with lapack and atlas
In-Reply-To: <20061112203630.98834.qmail@web38902.mail.mud.yahoo.com>
References: <ej6ir8$72b$1@sea.gmane.org>
	<20061112203630.98834.qmail@web38902.mail.mud.yahoo.com>
Message-ID: <ej89qu$o7o$1@sea.gmane.org>

Abhishek Roy wrote:
> --- Robert Kern <robert.kern at gmail.com> wrote:
>> Are your ATLAS libraries shared or static (i.e., are they libatlas.so or
>> libatlas.a)? ldd(1) only lists the shared libraries that are linked. ATLAS is
>> usually installed as static libraries.
> They are static. Thanks, I was mindlessly copying a suggestion I saw elsewhere.
>  
>> Does that module import? 
> I got,
> ImportError: /usr/lib/python2.4/site-packages/numpy/linalg/lapack_lite.so:
> undefined symbol: s_cat

Also add g2c to your library list, then. It is g77's FORTRAN runtime library.

-- 
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless enigma
 that is made terrible by our own mad attempt to interpret it as though it had
 an underlying truth."
  -- Umberto Eco


-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From erin.sheldon at gmail.com  Sun Nov 12 18:56:29 2006
From: erin.sheldon at gmail.com (Erin Sheldon)
Date: Sun, 12 Nov 2006 18:56:29 -0500
Subject: array from list of lists
Message-ID: <20061112235629.GB5596@66-234-56-83.nyc.cable.nyct.net>

Hi all-

I want to take the result from a database query,
and create a numpy array with field names and types 
corresponding to the returned columns.

The DBI 2.0 compliant interfaces return lists
of lists. E.g.


[[94137100072000193L, 94, 345.57215100000002, -0.83673208099999996],
 [94137100072000368L, 94, 345.60217299999999, -0.83766954299999996],
 ....
 [94137100083000157L, 94, 347.21668099999999, -0.83572582399999995],
 [94137100084000045L, 94, 347.45524799999998, -0.829750074]]


But the only examples I have found for creating an inhomogeneous
array with fields involves lists of tuples.  e.g.

>>> mydescriptor = {'names': ('gender','age','weight'), 'formats':('S1',
>>> 'f4', 'f4')}
>>> a = array([('M',64.0,75.0),('F',25.0,60.0)], dtype=mydescriptor)

Trying something like this with a list of lists results in
the following error:

TypeError: expected a readable buffer object

Now I could create the array and run a loop, copying
in, but this would be less efficient.  Is there a way
to do this in one step?

Thanks,
Erin


-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From erin.sheldon at gmail.com  Sun Nov 12 19:09:18 2006
From: erin.sheldon at gmail.com (Erin Sheldon)
Date: Sun, 12 Nov 2006 19:09:18 -0500
Subject: array from list of lists
In-Reply-To: <20061112235629.GB5596@66-234-56-83.nyc.cable.nyct.net>
References: <20061112235629.GB5596@66-234-56-83.nyc.cable.nyct.net>
Message-ID: <20061113000918.GC5596@66-234-56-83.nyc.cable.nyct.net>

I have to not ammend my statement a bit:
  
DBI 2.0 actually returns a lists of tuples, which would
work.  It appears to just be pgdb, the postgres interface,
that is returning lists of lists.  Still, I need to
interact with this database.

Erin

On Sun, Nov 12, 2006 at 06:56:29PM -0500, Erin Sheldon wrote:
> Hi all-
> 
> I want to take the result from a database query,
> and create a numpy array with field names and types 
> corresponding to the returned columns.
> 
> The DBI 2.0 compliant interfaces return lists
> of lists. E.g.
> 
> 
> [[94137100072000193L, 94, 345.57215100000002, -0.83673208099999996],
>  [94137100072000368L, 94, 345.60217299999999, -0.83766954299999996],
>  ....
>  [94137100083000157L, 94, 347.21668099999999, -0.83572582399999995],
>  [94137100084000045L, 94, 347.45524799999998, -0.829750074]]
> 
> 
> But the only examples I have found for creating an inhomogeneous
> array with fields involves lists of tuples.  e.g.
> 
> >>> mydescriptor = {'names': ('gender','age','weight'), 'formats':('S1',
> >>> 'f4', 'f4')}
> >>> a = array([('M',64.0,75.0),('F',25.0,60.0)], dtype=mydescriptor)
> 
> Trying something like this with a list of lists results in
> the following error:
> 
> TypeError: expected a readable buffer object
> 
> Now I could create the array and run a loop, copying
> in, but this would be less efficient.  Is there a way
> to do this in one step?
> 
> Thanks,
> Erin
> 

-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From pgmdevlist at gmail.com  Sun Nov 12 19:14:29 2006
From: pgmdevlist at gmail.com (Pierre GM)
Date: Sun, 12 Nov 2006 19:14:29 -0500
Subject: array from list of lists
In-Reply-To: <20061112235629.GB5596@66-234-56-83.nyc.cable.nyct.net>
References: <20061112235629.GB5596@66-234-56-83.nyc.cable.nyct.net>
Message-ID: <200611121914.30182.pgmdevlist@gmail.com>


You could try the fromarrays function of numpy.core.records

>>> mydescriptor = {'names': (a','b','c','d'), 'formats':('f4', 'f4', 'f4', 
'f4')}
>>> a = N.core.records.fromarrays(N.transpose(yourlist), dtype=mydescriptor)

The 'transpose' function ensures that 'fromarrays' sees 4 arrays (one for each 
column).

-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From erin.sheldon at gmail.com  Sun Nov 12 19:50:39 2006
From: erin.sheldon at gmail.com (Erin Sheldon)
Date: Sun, 12 Nov 2006 19:50:39 -0500
Subject: array from list of lists
In-Reply-To: <200611121914.30182.pgmdevlist@gmail.com>
References: <20061112235629.GB5596@66-234-56-83.nyc.cable.nyct.net>
	<200611121914.30182.pgmdevlist@gmail.com>
Message-ID: <331116dc0611121650m812119es1c4381d7d4faa2ce@mail.gmail.com>

On 11/12/06, Pierre GM <pgmdevlist at gmail.com> wrote:
>
> You could try the fromarrays function of numpy.core.records
>
> >>> mydescriptor = {'names': (a','b','c','d'), 'formats':('f4', 'f4', 'f4',
> 'f4')}
> >>> a = N.core.records.fromarrays(N.transpose(yourlist), dtype=mydescriptor)
>
> The 'transpose' function ensures that 'fromarrays' sees 4 arrays (one for each
> column).
>

That worked as advertised. Thanks.
Erin

-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From erin.sheldon at gmail.com  Sun Nov 12 20:10:17 2006
From: erin.sheldon at gmail.com (Erin Sheldon)
Date: Sun, 12 Nov 2006 20:10:17 -0500
Subject: array from list of lists
In-Reply-To: <331116dc0611121650m812119es1c4381d7d4faa2ce@mail.gmail.com>
References: <20061112235629.GB5596@66-234-56-83.nyc.cable.nyct.net>
	<200611121914.30182.pgmdevlist@gmail.com>
	<331116dc0611121650m812119es1c4381d7d4faa2ce@mail.gmail.com>
Message-ID: <331116dc0611121710k7cc2c965pbec26543224e8dc9@mail.gmail.com>

On 11/12/06, Erin Sheldon <erin.sheldon at gmail.com> wrote:
> On 11/12/06, Pierre GM <pgmdevlist at gmail.com> wrote:
> >
> > You could try the fromarrays function of numpy.core.records
> >
> > >>> mydescriptor = {'names': (a','b','c','d'), 'formats':('f4', 'f4', 'f4',
> > 'f4')}
> > >>> a = N.core.records.fromarrays(N.transpose(yourlist), dtype=mydescriptor)
> >
> > The 'transpose' function ensures that 'fromarrays' sees 4 arrays (one for each
> > column).

Actually, there is a problem with that approach.  It first converts
the entire array to a single type, by default a floating type.  For
very large integers this precision is insufficient.  For example, I
have the following integer in my arrays:
 94137100072000193L
which ends up as
  94137100072000192
after going to a float and then back to an integer.

Erin

-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From charlesr.harris at gmail.com  Sun Nov 12 20:15:10 2006
From: charlesr.harris at gmail.com (Charles R Harris)
Date: Sun, 12 Nov 2006 18:15:10 -0700
Subject: array from list of lists
In-Reply-To: <331116dc0611121710k7cc2c965pbec26543224e8dc9@mail.gmail.com>
References: <20061112235629.GB5596@66-234-56-83.nyc.cable.nyct.net>
	<200611121914.30182.pgmdevlist@gmail.com>
	<331116dc0611121650m812119es1c4381d7d4faa2ce@mail.gmail.com>
	<331116dc0611121710k7cc2c965pbec26543224e8dc9@mail.gmail.com>
Message-ID: <e06186140611121715m4e84de49he4c65c2c69bc73de@mail.gmail.com>

On 11/12/06, Erin Sheldon <erin.sheldon at gmail.com> wrote:
>
> On 11/12/06, Erin Sheldon <erin.sheldon at gmail.com> wrote:
> > On 11/12/06, Pierre GM <pgmdevlist at gmail.com> wrote:
> > >
> > > You could try the fromarrays function of numpy.core.records
> > >
> > > >>> mydescriptor = {'names': (a','b','c','d'), 'formats':('f4', 'f4',
> 'f4',
> > > 'f4')}
> > > >>> a = N.core.records.fromarrays(N.transpose(yourlist),
> dtype=mydescriptor)
> > >
> > > The 'transpose' function ensures that 'fromarrays' sees 4 arrays (one
> for each
> > > column).
>
> Actually, there is a problem with that approach.  It first converts
> the entire array to a single type, by default a floating type.  For
> very large integers this precision is insufficient.  For example, I
> have the following integer in my arrays:
> 94137100072000193L
> which ends up as
>   94137100072000192
> after going to a float and then back to an integer.


Out of curiosity, where does that large integer come from?

Chuck
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20061112/4a0a3969/attachment.html>
-------------- next part --------------
-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
-------------- next part --------------
_______________________________________________
Numpy-discussion mailing list
Numpy-discussion at lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/numpy-discussion

From erin.sheldon at gmail.com  Sun Nov 12 20:20:29 2006
From: erin.sheldon at gmail.com (Erin Sheldon)
Date: Sun, 12 Nov 2006 20:20:29 -0500
Subject: array from list of lists
In-Reply-To: <e06186140611121715m4e84de49he4c65c2c69bc73de@mail.gmail.com>
References: <20061112235629.GB5596@66-234-56-83.nyc.cable.nyct.net>
	<200611121914.30182.pgmdevlist@gmail.com>
	<331116dc0611121650m812119es1c4381d7d4faa2ce@mail.gmail.com>
	<331116dc0611121710k7cc2c965pbec26543224e8dc9@mail.gmail.com>
	<e06186140611121715m4e84de49he4c65c2c69bc73de@mail.gmail.com>
Message-ID: <331116dc0611121720o3fa53d53xfca405f62f763d14@mail.gmail.com>

On 11/12/06, Charles R Harris <charlesr.harris at gmail.com> wrote:
> > 94137100072000193L
> > which ends up as
> >   94137100072000192
> > after going to a float and then back to an integer.
>
> Out of curiosity, where does that large integer come from?

It is a unique object identifier.  It is a combination of various
numbers related to an astronomical object detected by
the sloan digital sky survey. (www.sdss.org).

cheers,
Erin

-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From jerald433 at brazosnet.com  Sun Nov 12 21:15:23 2006
From: jerald433 at brazosnet.com (=?windows-1251?B?zOXm5PPt4PDu5O375SDx8uDt5ODw8vsg4fP14+Dr8uXx6u7j7iDz9+Xy4A==?=)
Date: Mon, 13 Nov 2006 05:15:23 +0300
Subject: =?windows-1251?b?z3B17OXt5e115SDs5ebkee3gcO7k?=
 =?windows-1251?b?7ft4IPHy4O3k4HDy4Hgg9HXt4O3x7uIw6SDu8vfl8u3u8fJ1IHUg4Xl4?=
 =?windows-1251?b?4+Dr8uVw8eow4zAgeffl8uA=?=
Message-ID: <cc2f01c706c9$035c31cc$f4e28aef@brazosnet.com>

 
     ????y??p????? c?????p?? ?y??????pc???? y????.
      0????o??e?ue c ??u??u???u ?o??u?o???u? ?u???co?o? o??e??oc?u ?o M??0
(M???) u ?o????u ??e?c????e?u? o??e??oc?u. ??uo??e?e?ue ?????u?ec?ux ?????o?
y?e?? o??e????x ?u?o? ???u?o? u o?????e??c?? ? coo??e?c??uu c M???. ??u?e??
coc????e?u? ?u???co??x o??e?o?. 
         ???? ??o?e?e?u?:   14 - I5 Ho??p?, ?o 8 ????e?u?ec?ux ??co? ?
?e??, c ?e?e???o? ?? o?e? u ?o?e-?????. 
  Mec?o ??o?e?e?u?:  ?. Moc???   Me?o?u?ec?ue u ??????o???e ???e?u???,
o?e??-?o?e-?????u ?????e?? ? c?ou?oc??. 
    ?y?c o?ue??u?o??? ?? ?y?o?o?u?e?e? ?u???co?o-??o?o?u?ec?ux c?y?? o????u???u?,
??????x ?yx????e?o?, ? ????e ?cex ?e????ux ?o?y?u?? ?????u?ec?ue ????u?
?o ??u?e?e?u? M??0. 
   ?o?e????ue: 
     0??ee ????o?c??o c Me??y???o????u c?????????u ?u???co?o? o??e??oc?u
u ?yx????e?c?o?o y?e??. ??u??u?? ?o??u?o???u? c???????o? u c??y??y?? c???????o?.
?uc?e?? ?o????u???x ?o?y?e??o? ?o?e?? ?o M??0: 
  0??ue ??u??u?? (IFRS Frameworks) 
 Me??y???o???e c???????? ?u???co?o? o??e??oc?u (M??0 / IFRS) 
 Me??y???o???e c???????? ?yx????e?c?o?o y?e?? (M??? / IAS) 
 ???e???e???uu ? c?????????, ???y?e???e ?o?u?e?o? ?o u??e???e???uu ?e??y???o??o?
?u???co?o? o??e??oc?u (IFRIC) 
 ???e???e???uu ? c????????? ?oc?o???o?o ?o?u?e?? ?o u??e???e???uu (SIC)

 ??oe??? ?o?y?e??o? 
 ?uc?yccuo???e ???e?u??? o ?oo??o?e?ue M??0 u US GAAP 
  ??e?e??? ?u???co?o? o??e??oc?u ?o M??0 (M??? 1, M??? 8, M??? 10, M???
18): 
  ?yx????e?c?u? ?????c (Balance Sheet) 
 0??e? o ??u????x u y?????x (ProfitLoss Statement) 
 0??e? o ??u?e?uu ?e?e???x c?e?c?? (Cash Flow Statement) 
 0??e? o? u??e?e?uu co?c??e??o?o ???u???? (Changes of Equity Statement)

 ??u?e???u? / ?o?c?u?e????? ???uc?? ? ?u???co?o? o??e??oc?u (Disclosures)
o ??e???? ?o?u?u?? (Accounting Policies) 
  ?o??? ??e?c????e?u? ?u???co?o? o??e??oc?u 
  ?????c 
  ?uc??e ???u?? (???u?? - o?????e??c???) = ???u??? 
 ???u?? = ???u??? + 0?????e??c??? 
  0??e? o ??u????x u y?????x 
  ?o ?e?o?y x?????e?? ??cxo?o? 
 ?o ?e?o?y ?y???u? ??cxo?o? 
  0??e? o ??u?e?uu ?e?e???x c?e?c?? 
  ????o? ?e?o? 
 ?oc?e???? ?e?o? 
  0??e? o? u??e?e?uu co?c??e??o?o ???u???? 
  0c?o??o? ?o?xo? ("????u?? u??e?e?u? ?e?e??o?") 
 ????e????u???? ?o?xo? (???e?u?o?????? c ??c????u??u ? ?o?c?u?e???o? ???uc?e)

  ??u?e? ?o???e??? ?o?? ?u???co?o? o??e??oc?u 
  T?e?o???u? M??0 ??u ?o??u?o???uu ?u???co?o? o??e??oc?u ?o ?e??y???o????
c????????? ??e???e. M??0 1 
 ??e? o??e????x ?u?o? ???u?o? u o?????e??c?? 
  0??ue ??u??u?? y?e?? 
 ?o???o? ??u????u? u o?e??u o??e????x ?u?o? ???u?o? u o?????e??c?? 
 ?????e?ue c ?occu?c?u?u c?????????u ?yx????e?c?o?o y?e?? 
 ?o???o? ??c????u? u??o????uu o? o??e????x ?u??x ???u?o? u o?????e??c??
? ?o?c?u?e???o? ???uc?e 
 ??e? ??eo?o?o???x ???u?o? 
  0c?o???e c?e?c???, u??ec?u?uo???? co?c??e??oc?? u ??eo?o?o???e ???u??,
??e???????e???e ??? ??o???u (M??? 16, M??? 40, M??0 5) 
 ??e? ?e???e?u?????x ???u?o? (M??? 38) 
 ??o??u???u? ??eo?o?o???x ???u?o? 
 ?e?eo?e??? / 0?ec?e?e?ue ??eo?o?o???x ???u?o? (M??? 36) 
  ??e? ????co? (M??? 2) 
  ?e??o????????? o?e??? 
 ?????e???? ?e?eo?e??? ????co? (M??? 36) 
 ??uc??ue ??u ??o???e u ?????uu ????co? 
  ??c?e?? c ?e?u?o???u u ??e?u?o???u 
 ??e? ????o? u ??e?u?o? (M??? 23) 
 B??y??? u ???o? ?? ??u???? 
  B??y??? (M??? 18) 
 ??e? ?y?co??x ????u? (M??? 21) 
 ?uc??? ??u???? / y???o? ?? o??e???? ?e?uo? (M??? 8) 
 ??e? ???o?? ?? ??u???? (M??? 12) 
 ??c????ue u??o????uu o co???u?x, ??ouc?e??ux ?oc?e o??e??o? ???? (M???
10) 
  ???u??? u ?e?e??? (M??? 37) 
  ??u?e?? coc????e?u? ?u???co??x o??e?o? ? coo??e?c??uu c M??0 
  ??u?e? 1: ??ou??o?c??e???? ?o????u? 
 ??u?e? 2: To??o?o-c???o??? ?o????u? 
  ??u??u?? u ?????u?ec?ue ?o?xo?? ? ????c?o????uu ?occu?c?o? ?yx????e?c?o?
o??e??oc?u ? ?u???co??e o??e?? ?o M??0 
  ?o??u?o???ue ???o?e?o ????? c?e?o? 
 T???c?o????u? o?o?o??o?o ?????c? 
 ?o???o? y?e?? ?o??e??u?o?o? ??u ??o?e?e?uu ????c?o????uu o?o?o??o?o ?????c?

 ??u?e? ????c?o????uu u ?o??u?o???u? ?u???co??x o??e?o? ? coo??e?c??uu
c M??0 ??? ?o??o?o-c???o?o? ?o????uu 
  ?o ?o??oc?? ?e?uc????uu o???????ec? ?o ?e?: (495) 2?5-58-88, (495) 2?5-I2?5
  ????
 
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20061113/2e4365b8/attachment.html>

From fingerslip at yahoo.com  Sun Nov 12 22:06:24 2006
From: fingerslip at yahoo.com (Abhishek Roy)
Date: Mon, 13 Nov 2006 03:06:24 +0000 (GMT)
Subject: Compiling numpy with lapack and atlas
In-Reply-To: <ej89qu$o7o$1@sea.gmane.org>
References: <ej89qu$o7o$1@sea.gmane.org>
Message-ID: <20061113030624.18642.qmail@web38903.mail.mud.yahoo.com>

--- Robert Kern <robert.kern at gmail.com> wrote:
> Also add g2c to your library list, then. It is g77's FORTRAN runtime library
Okay I put libg2c.a in the atlas directory and made the line in site.cfg,
atlas_libs = lapack, f77blas, cblas, atlas, g2c

Now numpy imports but numpy.linalg.test() as well as any linalg function gives,
>>> numpy.linalg.test()
  Found 32 tests for numpy.linalg
  Found 0 tests for __main__
.. ** On entry to DGEEV  parameter number 13 had an illegal value

I am using prebuilt a lapack library from netlib.org. 

Thanks for all your help,
Abhishek

Send instant messages to your online friends http://uk.messenger.yahoo.com 

-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From tim.hochberg at ieee.org  Sun Nov 12 22:07:50 2006
From: tim.hochberg at ieee.org (Tim Hochberg)
Date: Sun, 12 Nov 2006 20:07:50 -0700
Subject: array from list of lists
In-Reply-To: <331116dc0611121710k7cc2c965pbec26543224e8dc9@mail.gmail.com>
References: <20061112235629.GB5596@66-234-56-83.nyc.cable.nyct.net>	<200611121914.30182.pgmdevlist@gmail.com>	<331116dc0611121650m812119es1c4381d7d4faa2ce@mail.gmail.com>
	<331116dc0611121710k7cc2c965pbec26543224e8dc9@mail.gmail.com>
Message-ID: <4557E186.6030307@ieee.org>

Erin Sheldon wrote:
> On 11/12/06, Erin Sheldon <erin.sheldon at gmail.com> wrote:
>   
>> On 11/12/06, Pierre GM <pgmdevlist at gmail.com> wrote:
>>     
>>> You could try the fromarrays function of numpy.core.records
>>>
>>>       
>>>>>> mydescriptor = {'names': (a','b','c','d'), 'formats':('f4', 'f4', 'f4',
>>>>>>             
>>> 'f4')}
>>>       
>>>>>> a = N.core.records.fromarrays(N.transpose(yourlist), dtype=mydescriptor)
>>>>>>             
>>> The 'transpose' function ensures that 'fromarrays' sees 4 arrays (one for each
>>> column).
>>>       
>
> Actually, there is a problem with that approach.  It first converts
> the entire array to a single type, by default a floating type.  For
> very large integers this precision is insufficient.  For example, I
> have the following integer in my arrays:
>  94137100072000193L
> which ends up as
>   94137100072000192
> after going to a float and then back to an integer.
>   

I haven't been following this too closely, but if you need to transpose 
your data without converting all to one type, I can think of a couple of 
different approaches:

    1.  zip(*yourlist)
    2. numpy.transpose(numpy.array(yourlist, dtype=object)

I haven't tested them though (particularly the second one), so caveat 
emptor, etc, etc.

-tim


-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From erin.sheldon at gmail.com  Sun Nov 12 22:17:17 2006
From: erin.sheldon at gmail.com (Erin Sheldon)
Date: Sun, 12 Nov 2006 22:17:17 -0500
Subject: array from list of lists
In-Reply-To: <4557E186.6030307@ieee.org>
References: <20061112235629.GB5596@66-234-56-83.nyc.cable.nyct.net>
	<200611121914.30182.pgmdevlist@gmail.com>
	<331116dc0611121650m812119es1c4381d7d4faa2ce@mail.gmail.com>
	<331116dc0611121710k7cc2c965pbec26543224e8dc9@mail.gmail.com>
	<4557E186.6030307@ieee.org>
Message-ID: <331116dc0611121917oe71d9bsc8fd684e975a014@mail.gmail.com>

On 11/12/06, Tim Hochberg <tim.hochberg at ieee.org> wrote:
> I haven't been following this too closely, but if you need to transpose
> your data without converting all to one type, I can think of a couple of
> different approaches:
>
>     1.  zip(*yourlist)
>     2. numpy.transpose(numpy.array(yourlist, dtype=object)
>
> I haven't tested them though (particularly the second one), so caveat
> emptor, etc, etc.

Its not that I want to transpose data.

I'm trying to convert the output of a pgdb postgres query into
an array with fields and types corresponding to the columns
I have selected.  The problem is pgdb does not return a list
of tuples as it should according to DB 2.0, but instead
a list of lists.   So numpy.array(lol, dtype=) fails, and so will your
solution #2.   I don't want to copy the data more than once
obviously, so I'm looking for a way to call array() with a lists
of lists.

Erin

-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From peridot.faceted at gmail.com  Sun Nov 12 22:18:19 2006
From: peridot.faceted at gmail.com (A. M. Archibald)
Date: Sun, 12 Nov 2006 22:18:19 -0500
Subject: array from list of lists
In-Reply-To: <331116dc0611121710k7cc2c965pbec26543224e8dc9@mail.gmail.com>
References: <20061112235629.GB5596@66-234-56-83.nyc.cable.nyct.net>
	<200611121914.30182.pgmdevlist@gmail.com>
	<331116dc0611121650m812119es1c4381d7d4faa2ce@mail.gmail.com>
	<331116dc0611121710k7cc2c965pbec26543224e8dc9@mail.gmail.com>
Message-ID: <ce557a360611121918y612ce38fu68c51fbc20546dc1@mail.gmail.com>

On 12/11/06, Erin Sheldon <erin.sheldon at gmail.com> wrote:

> Actually, there is a problem with that approach.  It first converts
> the entire array to a single type, by default a floating type.  For
> very large integers this precision is insufficient.  For example, I
> have the following integer in my arrays:
>  94137100072000193L
> which ends up as
>   94137100072000192
> after going to a float and then back to an integer.

That's an unfortunate limitation of numpy; it views double-precision
floats as higher precision than 64-bit integers, but of course they
aren't. If you want to put all your data in a record array, you could
try transposing the lists using a list comprehension - numpy is not
always as much faster than pure python as it looks. You could then
convert that to a list of four arrays and do the assignment as
appropriate. Alternatively, you could convert your array into a
higher-precision floating-point format (if one is available on your
machine) before transposing and storing in a record array.

A. M. Archibald

-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From george.sakkis at gmail.com  Sun Nov 12 22:25:05 2006
From: george.sakkis at gmail.com (George Sakkis)
Date: Mon, 13 Nov 2006 03:25:05 -0000
Subject: Equivalent to list.index ?
References: <1163358985.920640.306720@k70g2000cwa.googlegroups.com>
	<f4f93d420611121121y2620c6fdj60241acef01e30b4@mail.gmail.com>
	<1163362743.775427.260100@f16g2000cwb.googlegroups.com>
	<4557949E.5060009@ieee.org>
Message-ID: <1163388305.110953.216520@m7g2000cwm.googlegroups.com>

Tim Hochberg wrote:

> George Sakkis wrote:
> > def index(a, value):
> >     return N.where(a==value)[0][0]
> >
> Or
>
> def index(a, value):
>     return argmax(a == value)

That's a bit easier to write and a bit harder to grok; that's ok, I can
live with it.

> > This works but seems clunky and less efficient than necessary. If there
> > isn't a better alternative, I would welcome a new index()
> > function/method in the next numpy release.
> >
> >
>
> Without commenting on the desirability of this particular function,
> people seem awfully eager to add functions and methods everywhere. These
> aren't free you know. They cost in terms of maintenance and potential
> bugs, and in terms of making the already large function set even harder
> to wrap ones head around. That's partly an organizational problem but
> partly an issue of sheer size, neither of which are helped by adding a
> bunch more stuff.
>
>     Not every two-line Python function has to come pre-written
>         --Tim Peters

I think the organizational problem is more serious in Numpy's case; any
package with about 200 functions (according to
http://www.scipy.org/Numpy_Example_List) in an almost flat namespace
would feel the maintainability problem build up.

> In the case of this particular function, what are the use cases? Are
> they common (not just with you but with other numpy users)? Are the uses
> speed critical? Is it a good building block for other numeric
> functions?  I'm skeptical of the above implementation as it doesn't fit
> well with other similar numpy functions (argmin and argmax for example)
> and I personally don't see much in the way of compelling uses for this,
> but feel free to try to convince me otherwise.
>
> -tim

As I said, I would welcome the addition of this function and I would
use it if it was there, but I'm not particularly fervent about it. If I
were to convince you, I'd argue about how numpy arrays can be
considered generalizations of (1-d) sequences in many ways and how
convenient would it be for users whose code expects python lists to
work as is (thanks to duck typing) with 1-d numpy arrays. Another
(indirect) argument would be the addition of functions that implement
set-theoretic operations, and certainly lists are far more common than
sets.

I have to agree though that the flat namespace at its current size
makes me uneasy too; I typically use "import numpy as N" instead of
"from numpy import *".

George


-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From tim.hochberg at ieee.org  Sun Nov 12 22:50:13 2006
From: tim.hochberg at ieee.org (Tim Hochberg)
Date: Sun, 12 Nov 2006 20:50:13 -0700
Subject: Equivalent to list.index ?
In-Reply-To: <1163388305.110953.216520@m7g2000cwm.googlegroups.com>
References: <1163358985.920640.306720@k70g2000cwa.googlegroups.com>	<f4f93d420611121121y2620c6fdj60241acef01e30b4@mail.gmail.com>	<1163362743.775427.260100@f16g2000cwb.googlegroups.com>	<4557949E.5060009@ieee.org>
	<1163388305.110953.216520@m7g2000cwm.googlegroups.com>
Message-ID: <4557EB75.5030505@ieee.org>

George Sakkis wrote:
> Tim Hochberg wrote:
>
>   
>> George Sakkis wrote:
>>     
>>> def index(a, value):
>>>     return N.where(a==value)[0][0]
>>>
>>>       
>> Or
>>
>> def index(a, value):
>>     return argmax(a == value)
>>     
>
> That's a bit easier to write and a bit harder to grok; that's ok, I can
> live with it.
>   
I'll add a little cautionary note as well: it won't work correctly if 
there are no elements of a that equal value (in which case the max of 
a==values is 0 and you'll just get the first values). It would be easy 
enough to make this bullet proof:

    def index(a, value):
        arg = argmax(a == value)
        if a[arg] != value:
           raise IndexError("or whatever")
        return arg


Unfortunately, that's no longer two lines and I wouldn't have been able 
to use that Tim Peter's quote which needs to be thrown out from time to 
time just on general principles.

>   
>> [snip]
>> Peters
>>     
>
> I think the organizational problem is more serious in Numpy's case; any
> package with about 200 functions (according to
> http://www.scipy.org/Numpy_Example_List) in an almost flat namespace
> would feel the maintainability problem build up.
>   
I agree with that for the most part.

>   
>> In the case of this particular function, what are the use cases? Are
>> they common (not just with you but with other numpy users)? Are the uses
>> speed critical? Is it a good building block for other numeric
>> functions?  I'm skeptical of the above implementation as it doesn't fit
>> well with other similar numpy functions (argmin and argmax for example)
>> and I personally don't see much in the way of compelling uses for this,
>> but feel free to try to convince me otherwise.
>>
>> -tim
>>     
>
> As I said, I would welcome the addition of this function and I would
> use it if it was there, but I'm not particularly fervent about it. If I
> were to convince you, I'd argue about how numpy arrays can be
> considered generalizations of (1-d) sequences in many ways and how
> convenient would it be for users whose code expects python lists to
> work as is (thanks to duck typing) with 1-d numpy arrays. Another
> (indirect) argument would be the addition of functions that implement
> set-theoretic operations, and certainly lists are far more common than
> sets.
>   
Well, if you were to try to convince me like that, I would try to 
convince you of your errant ways.by noting that numpy arrays are quite 
dissimilar from lists in so many ways, that making them closer would 
just lead to confusion as people asked for more and more list-like 
behaviors (append, in, etc etc.). Lists and arrays aren't close enough 
that conflating them could be made to work in any halfway sensible manner.

Lists and arrays are both examples of python sequences and they both 
fully implement the sequence interface (and the iterarable interface for 
that matter) but that's really the extent of their similarity.

Now whether an object based on array that implemented list semantics but 
kept a numpy arrays memory frugality would be useful I cannot say. 
However, that seems to be the kind of thing that should be a separate 
type of object, perhaps reusing much of the array code, but having a 
completely different interface. It also seems the kind of thing that 
would likely have fairly limited use in practice, so if someone really 
wants it I'd like to see it exist as a separate package for some period 
of time to verify that it actually has a user base before it got 
incorporated into numpy.

Not that I control these things.

> I have to agree though that the flat namespace at its current size
> makes me uneasy too; I typically use "import numpy as N" instead of
> "from numpy import *".
>   
Oh yeah, you should almost never use "from anything import *" IMO. 'N' 
seems to be the favored abreviation these days although I've always used 
'np' (this helped when I was transition from Numeric to numarray and 
then back to numpy as I used 'na' for numarray and np for numpy).

-tim




-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From tim.hochberg at ieee.org  Sun Nov 12 23:01:26 2006
From: tim.hochberg at ieee.org (Tim Hochberg)
Date: Sun, 12 Nov 2006 21:01:26 -0700
Subject: array from list of lists
In-Reply-To: <331116dc0611121917oe71d9bsc8fd684e975a014@mail.gmail.com>
References: <20061112235629.GB5596@66-234-56-83.nyc.cable.nyct.net>	<200611121914.30182.pgmdevlist@gmail.com>	<331116dc0611121650m812119es1c4381d7d4faa2ce@mail.gmail.com>	<331116dc0611121710k7cc2c965pbec26543224e8dc9@mail.gmail.com>	<4557E186.6030307@ieee.org>
	<331116dc0611121917oe71d9bsc8fd684e975a014@mail.gmail.com>
Message-ID: <4557EE16.3080604@ieee.org>

Erin Sheldon wrote:
> On 11/12/06, Tim Hochberg <tim.hochberg at ieee.org> wrote:
>   
>> I haven't been following this too closely, but if you need to transpose
>> your data without converting all to one type, I can think of a couple of
>> different approaches:
>>
>>     1.  zip(*yourlist)
>>     2. numpy.transpose(numpy.array(yourlist, dtype=object)
>>
>> I haven't tested them though (particularly the second one), so caveat
>> emptor, etc, etc.
>>     
>
> Its not that I want to transpose data.
>
> I'm trying to convert the output of a pgdb postgres query into
> an array with fields and types corresponding to the columns
> I have selected.  The problem is pgdb does not return a list
> of tuples as it should according to DB 2.0, but instead
> a list of lists.   So numpy.array(lol, dtype=) fails, and so will your
> solution #2. 
In that case, I suggest just using a list comprehension or map, 
[tuple(x) for x in lol] for example.

>   I don't want to copy the data more than once
> obviously, so I'm looking for a way to call array() with a lists
> of lists.
>   
It's probably pointless to worry about his. You are already allocating 
5*N python objects (all those Python floats and integers as well as the 
lists themselves). I believe the list comprehension above is only going 
to allocate an additional N objects (the new tuples). Admittedly, the 
objects aren't all the same size, but in this case they are close enough 
that I doubt it'll matter.

-tim




-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From pgmdevlist at gmail.com  Sun Nov 12 23:06:33 2006
From: pgmdevlist at gmail.com (Pierre GM)
Date: Sun, 12 Nov 2006 23:06:33 -0500
Subject: array from list of lists
In-Reply-To: <331116dc0611121710k7cc2c965pbec26543224e8dc9@mail.gmail.com>
References: <20061112235629.GB5596@66-234-56-83.nyc.cable.nyct.net>
	<331116dc0611121650m812119es1c4381d7d4faa2ce@mail.gmail.com>
	<331116dc0611121710k7cc2c965pbec26543224e8dc9@mail.gmail.com>
Message-ID: <200611122306.34072.pgmdevlist@gmail.com>

On Sunday 12 November 2006 20:10, Erin Sheldon wrote:
> Actually, there is a problem with that approach.  It first converts
> the entire array to a single type, by default a floating type.  

As A.M. Archibald suggested, you can use list comprehension:
N.array([(a,b,c,d,) for (a,b,c,d) in yourlist], dtype=yourdesc)

or

N.fromiter(((a,b,c,d) for (a,b,c,d,) in yourlist), dtype=yourdesc)

Would you mind trying that, and let us know which one works best ? That could 
be put on the wiki somewhere...


-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From robert.kern at gmail.com  Sun Nov 12 23:27:54 2006
From: robert.kern at gmail.com (Robert Kern)
Date: Sun, 12 Nov 2006 22:27:54 -0600
Subject: array from list of lists
In-Reply-To: <200611122306.34072.pgmdevlist@gmail.com>
References: <20061112235629.GB5596@66-234-56-83.nyc.cable.nyct.net>	<331116dc0611121650m812119es1c4381d7d4faa2ce@mail.gmail.com>	<331116dc0611121710k7cc2c965pbec26543224e8dc9@mail.gmail.com>
	<200611122306.34072.pgmdevlist@gmail.com>
Message-ID: <ej8s8l$1fr$1@sea.gmane.org>

Pierre GM wrote:
> On Sunday 12 November 2006 20:10, Erin Sheldon wrote:
>> Actually, there is a problem with that approach.  It first converts
>> the entire array to a single type, by default a floating type.  
> 
> As A.M. Archibald suggested, you can use list comprehension:
> N.array([(a,b,c,d,) for (a,b,c,d) in yourlist], dtype=yourdesc)
> 
> or
> 
> N.fromiter(((a,b,c,d) for (a,b,c,d,) in yourlist), dtype=yourdesc)
> 
> Would you mind trying that, and let us know which one works best ? That could 
> be put on the wiki somewhere...

N.array(map(tuple, yourlist), dtype=yourdesc)

is probably the best option.

-- 
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless enigma
 that is made terrible by our own mad attempt to interpret it as though it had
 an underlying truth."
  -- Umberto Eco


-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From erin.sheldon at gmail.com  Mon Nov 13 00:56:27 2006
From: erin.sheldon at gmail.com (Erin Sheldon)
Date: Mon, 13 Nov 2006 00:56:27 -0500
Subject: array from list of lists
In-Reply-To: <ej8s8l$1fr$1@sea.gmane.org>
References: <20061112235629.GB5596@66-234-56-83.nyc.cable.nyct.net>
	<331116dc0611121650m812119es1c4381d7d4faa2ce@mail.gmail.com>
	<331116dc0611121710k7cc2c965pbec26543224e8dc9@mail.gmail.com>
	<200611122306.34072.pgmdevlist@gmail.com> <ej8s8l$1fr$1@sea.gmane.org>
Message-ID: <331116dc0611122156m7df55cf5r939d0acab84cdeb@mail.gmail.com>

Hi all -

Thanks to everyone for the suggestions.
I think map(tuple, list) is probably the most compact,
but the list comprehension also works well.

Because map() is proably going to disappear someday, I'll
stick with the list comprehension.
  array( [tuple(row) for row in result], dtype=dtype)

That said, is there some compelling reason that the array
function doesn't support this operation?

Thanks again,
Erin

On 11/12/06, Robert Kern <robert.kern at gmail.com> wrote:
> Pierre GM wrote:
> > On Sunday 12 November 2006 20:10, Erin Sheldon wrote:
> >> Actually, there is a problem with that approach.  It first converts
> >> the entire array to a single type, by default a floating type.
> >
> > As A.M. Archibald suggested, you can use list comprehension:
> > N.array([(a,b,c,d,) for (a,b,c,d) in yourlist], dtype=yourdesc)
> >
> > or
> >
> > N.fromiter(((a,b,c,d) for (a,b,c,d,) in yourlist), dtype=yourdesc)
> >
> > Would you mind trying that, and let us know which one works best ? That could
> > be put on the wiki somewhere...
>
> N.array(map(tuple, yourlist), dtype=yourdesc)
>
> is probably the best option.
>
> --
> Robert Kern
>
> "I have come to believe that the whole world is an enigma, a harmless enigma
>  that is made terrible by our own mad attempt to interpret it as though it had
>  an underlying truth."
>   -- Umberto Eco
>
>
> -------------------------------------------------------------------------
> Using Tomcat but need to do more? Need to support web services, security?
> Get stuff done quickly with pre-integrated technology to make your job easier
> Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
> http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
> _______________________________________________
> Numpy-discussion mailing list
> Numpy-discussion at lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/numpy-discussion
>

-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From charlesr.harris at gmail.com  Mon Nov 13 01:18:22 2006
From: charlesr.harris at gmail.com (Charles R Harris)
Date: Sun, 12 Nov 2006 23:18:22 -0700
Subject: array from list of lists
In-Reply-To: <331116dc0611122156m7df55cf5r939d0acab84cdeb@mail.gmail.com>
References: <20061112235629.GB5596@66-234-56-83.nyc.cable.nyct.net>
	<331116dc0611121650m812119es1c4381d7d4faa2ce@mail.gmail.com>
	<331116dc0611121710k7cc2c965pbec26543224e8dc9@mail.gmail.com>
	<200611122306.34072.pgmdevlist@gmail.com> <ej8s8l$1fr$1@sea.gmane.org>
	<331116dc0611122156m7df55cf5r939d0acab84cdeb@mail.gmail.com>
Message-ID: <e06186140611122218i47032882r95a20b789ae2630a@mail.gmail.com>

On 11/12/06, Erin Sheldon <erin.sheldon at gmail.com> wrote:
>
> Hi all -
>
> Thanks to everyone for the suggestions.
> I think map(tuple, list) is probably the most compact,
> but the list comprehension also works well.
>
> Because map() is proably going to disappear someday, I'll
> stick with the list comprehension.
>   array( [tuple(row) for row in result], dtype=dtype)
>
> That said, is there some compelling reason that the array
> function doesn't support this operation?


My understanding is that the array needs to be allocated up front. Since the
list comprehension is iterative it is impossible to know how big the result
is going to be.

BTW, it might be possible to use fromfile('name', dtype=dtype) to do what
you want if the data is stored by rows in a file.

Chuck
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20061112/591667ae/attachment.html>
-------------- next part --------------
-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
-------------- next part --------------
_______________________________________________
Numpy-discussion mailing list
Numpy-discussion at lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/numpy-discussion

From erin.sheldon at gmail.com  Mon Nov 13 02:07:29 2006
From: erin.sheldon at gmail.com (Erin Sheldon)
Date: Mon, 13 Nov 2006 02:07:29 -0500
Subject: array from list of lists
In-Reply-To: <e06186140611122218i47032882r95a20b789ae2630a@mail.gmail.com>
References: <20061112235629.GB5596@66-234-56-83.nyc.cable.nyct.net>
	<331116dc0611121650m812119es1c4381d7d4faa2ce@mail.gmail.com>
	<331116dc0611121710k7cc2c965pbec26543224e8dc9@mail.gmail.com>
	<200611122306.34072.pgmdevlist@gmail.com> <ej8s8l$1fr$1@sea.gmane.org>
	<331116dc0611122156m7df55cf5r939d0acab84cdeb@mail.gmail.com>
	<e06186140611122218i47032882r95a20b789ae2630a@mail.gmail.com>
Message-ID: <331116dc0611122307p23705f5fjd2a038812d824044@mail.gmail.com>

On 11/13/06, Charles R Harris <charlesr.harris at gmail.com> wrote:
>
>
> On 11/12/06, Erin Sheldon <erin.sheldon at gmail.com> wrote:
> > Hi all -
> >
> > Thanks to everyone for the suggestions.
> > I think map(tuple, list) is probably the most compact,
> > but the list comprehension also works well.
> >
> > Because map() is proably going to disappear someday, I'll
> > stick with the list comprehension.
> >   array( [tuple(row) for row in result], dtype=dtype)
> >
> > That said, is there some compelling reason that the array
> > function doesn't support this operation?
>
> My understanding is that the array needs to be allocated up front. Since the
> list comprehension is iterative it is impossible to know how big the result
> is going to be.

Isn't it the same with a list of tuples?  But you can send that directly to the
array constructor.  I don't see the fundamental difference, except that the
code might be simpler to write.

>
> BTW, it might be possible to use fromfile('name', dtype=dtype) to do what
> you want if the data is stored by rows in a file.

I'm reading from a database.
Erin

-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From faltet at carabos.com  Mon Nov 13 03:19:17 2006
From: faltet at carabos.com (Francesc Altet)
Date: Mon, 13 Nov 2006 09:19:17 +0100
Subject: array from list of lists
In-Reply-To: <331116dc0611122307p23705f5fjd2a038812d824044@mail.gmail.com>
References: <20061112235629.GB5596@66-234-56-83.nyc.cable.nyct.net>
	<331116dc0611121650m812119es1c4381d7d4faa2ce@mail.gmail.com>
	<331116dc0611121710k7cc2c965pbec26543224e8dc9@mail.gmail.com>
	<200611122306.34072.pgmdevlist@gmail.com> <ej8s8l$1fr$1@sea.gmane.org>
	<331116dc0611122156m7df55cf5r939d0acab84cdeb@mail.gmail.com>
	<e06186140611122218i47032882r95a20b789ae2630a@mail.gmail.com>
	<331116dc0611122307p23705f5fjd2a038812d824044@mail.gmail.com>
Message-ID: <1163405957.4221.25.camel@localhost.localdomain>

El dl 13 de 11 del 2006 a les 02:07 -0500, en/na Erin Sheldon va
escriure:
> On 11/13/06, Charles R Harris <charlesr.harris at gmail.com> wrote:
> >
> >
> > On 11/12/06, Erin Sheldon <erin.sheldon at gmail.com> wrote:
> > > Hi all -
> > >
> > > Thanks to everyone for the suggestions.
> > > I think map(tuple, list) is probably the most compact,
> > > but the list comprehension also works well.
> > >
> > > Because map() is proably going to disappear someday, I'll
> > > stick with the list comprehension.
> > >   array( [tuple(row) for row in result], dtype=dtype)
> > >
> > > That said, is there some compelling reason that the array
> > > function doesn't support this operation?
> >
> > My understanding is that the array needs to be allocated up front. Since the
> > list comprehension is iterative it is impossible to know how big the result
> > is going to be.
> 
> Isn't it the same with a list of tuples?  But you can send that directly to the
> array constructor.  I don't see the fundamental difference, except that the
> code might be simpler to write.

I think that the correct explanation is that Travis has chosen a tuple
as the way to refer to a inhomogeneous list of values (a record) and a
list as the way to refer to homogenous list of values. I'm not
completely sure why he did this, but I guess the reason was to be able
to distinguish the records in scenarios where nested records do appear.

In any case, you can also use rec.fromrecords for build recarrays from
lists of lists. This breaks the aforementioned rule, but Travis allowed
this because rec.* had to mimic numarray behaviour as much as possible.
Here is an example of use:

In [46]:mydescriptor = {'names': ('gender','age','weight'),
'formats':('S1','f4', 'f4')}
In [47]:results=[['M',64.0,75.0],['F',25.0,60.0]]
In [48]:a = numpy.rec.fromrecords(results, dtype=mydescriptor)
In [49]:b = numpy.array([tuple(row) for row in results],
dtype=mydescriptor)
In [50]:a==b
Out[50]:recarray([True, True], dtype=bool)

OTOH, it is said in the docs that fromrecords is discouraged because it
is somewhat slow, but apparently it has similar performance than using
comprehensions lists:

In [51]:Timer("numpy.rec.fromrecords(results, dtype=mydescriptor)",
"import numpy; results = [['M',64.0,75.0]]*10000; mydescriptor =
{'names': ('gender','age','weight'), 'formats':('S1','f4',
'f4')}").repeat(3,10)
Out[51]:[0.44204592704772949, 0.43584394454956055, 0.50145101547241211]

In [52]:Timer("numpy.array([tuple(row) for row in results],
dtype=mydescriptor)", "import numpy; results = [['M',64.0,75.0]]*10000;
mydescriptor = {'names': ('gender','age','weight'),
'formats':('S1','f4', 'f4')}").repeat(3,10)
Out[52]:[0.49885106086730957, 0.4325258731842041, 0.43297886848449707]


HTH,

-- 
Francesc Altet    |  Be careful about using the following code --
Carabos Coop. V.  |  I've only proven that it works, 
www.carabos.com   |  I haven't tested it. -- Donald Knuth


-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From svetosch at gmx.net  Mon Nov 13 05:02:47 2006
From: svetosch at gmx.net (Sven Schreiber)
Date: Mon, 13 Nov 2006 11:02:47 +0100
Subject: Could numpy.matlib.nanmax return a matrix?
In-Reply-To: <200611121808.51298.pgmdevlist@gmail.com>
References: <f4f93d420611121357o5dda5950g4fdcdc2f657ee654@mail.gmail.com>	<ce557a360611121408k6bae03fds9b2e979d786bb74e@mail.gmail.com>
	<200611121808.51298.pgmdevlist@gmail.com>
Message-ID: <455842C7.3050604@gmx.net>

Pierre GM schrieb:
> On Sunday 12 November 2006 17:08, A. M. Archibald wrote:
>> On 12/11/06, Keith Goodman <kwgoodman at gmail.com> wrote:
>>> Is anybody interested in making x.max() and nanmax() behave the same
>>> for matrices, except for the NaN part? That is, make
>>> numpy.matlib.nanmax return a matrix instead of an array.
> 
> Or, you could use masked arrays... In the new implementation, you can add a 
> mask to a subclassed array (such as matrix) to get a regular masked array. If 
> you fill this masked array, you get an array of the same subclass.
> 

That is very interesting, but I agree with Keith and would actually call
this a bug. (If still present in 1.0, that is, haven't checked, I think
Keith used some rc?.) One proclaimed goal of numpy for the 1.0 release
has been to be as matrix-friendly as possible, for which I am very
grateful.

Still, the use of masked arrays looks more attractive every time they're
mentioned...

-sven

-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From tim.hochberg at ieee.org  Mon Nov 13 10:03:33 2006
From: tim.hochberg at ieee.org (Tim Hochberg)
Date: Mon, 13 Nov 2006 08:03:33 -0700
Subject: array from list of lists
In-Reply-To: <1163405957.4221.25.camel@localhost.localdomain>
References: <20061112235629.GB5596@66-234-56-83.nyc.cable.nyct.net>	<331116dc0611121650m812119es1c4381d7d4faa2ce@mail.gmail.com>	<331116dc0611121710k7cc2c965pbec26543224e8dc9@mail.gmail.com>	<200611122306.34072.pgmdevlist@gmail.com>
	<ej8s8l$1fr$1@sea.gmane.org>	<331116dc0611122156m7df55cf5r939d0acab84cdeb@mail.gmail.com>	<e06186140611122218i47032882r95a20b789ae2630a@mail.gmail.com>	<331116dc0611122307p23705f5fjd2a038812d824044@mail.gmail.com>
	<1163405957.4221.25.camel@localhost.localdomain>
Message-ID: <45588945.7030703@ieee.org>

Francesc Altet wrote:
> El dl 13 de 11 del 2006 a les 02:07 -0500, en/na Erin Sheldon va
> escriure:
>   
>> On 11/13/06, Charles R Harris <charlesr.harris at gmail.com> wrote:
>>     
>>> On 11/12/06, Erin Sheldon <erin.sheldon at gmail.com> wrote:
>>>       
>>>> Hi all -
>>>>
>>>> Thanks to everyone for the suggestions.
>>>> I think map(tuple, list) is probably the most compact,
>>>> but the list comprehension also works well.
>>>>
>>>> Because map() is proably going to disappear someday, I'll
>>>> stick with the list comprehension.
>>>>   array( [tuple(row) for row in result], dtype=dtype)
>>>>
>>>> That said, is there some compelling reason that the array
>>>> function doesn't support this operation?
>>>>         
>>> My understanding is that the array needs to be allocated up front. Since the
>>> list comprehension is iterative it is impossible to know how big the result
>>> is going to be.
>>>       
>> Isn't it the same with a list of tuples?  But you can send that directly to the
>> array constructor.  I don't see the fundamental difference, except that the
>> code might be simpler to write.
>>     
>
> I think that the correct explanation is that Travis has chosen a tuple
> as the way to refer to a inhomogeneous list of values (a record) and a
> list as the way to refer to homogenous list of values. 
Just for the record, this is the officially blessed usage of tuple and 
lists for all of Python (by Guido himself). On the other hand, it's 
honored more in the breach than in reality. Since other factors, such as 
mutability/immutability or the mistaken belief that using tuples 
everywhere will make code noticeably faster or more memory frugal or 
something.

> I'm not
> completely sure why he did this, but I guess the reason was to be able
> to distinguish the records in scenarios where nested records do appear.
>   
I suspect that this could be made a little more forgiving, without 
loosing rigor. As long as none of the fields are objects of course in 
which case nearly all bets are off. Then again, the rule that tuple 
designate records is a lot simpler than something like tuples designate 
records, but you can use lists too, unless of course you have an object 
field in your array, in which case you really need to use tuples, except 
sometimes lists will work anyway, depending on where the object is 
fields is. So, maybe it's best just to keep it strict.

> In any case, you can also use rec.fromrecords for build recarrays from
> lists of lists. This breaks the aforementioned rule, but Travis allowed
> this because rec.* had to mimic numarray behaviour as much as possible.
> Here is an example of use:
> [SNIP]
>   

Just for completeness, I benchmarked the fromiter and map(tuple, 
results)  solutions as well. Map is fastest, followed by fromiter, list 
comprehension and then fromrecords. The differences are pretty minor 
however, so I'd stick with whatever seems clearest.

-tim


print Timer("numpy.rec.fromrecords(results, dtype=mydescriptor)",
"""import numpy; results = [['M',64.0,75.0]]*100000; mydescriptor = 
{'names': ('gender','age','weight'), 
'formats':('S1','f4','f4')}""").repeat(3,10)

print Timer("numpy.array([tuple(row) for row in results], 
dtype=mydescriptor)",
"""import numpy; results = [['M',64.0,75.0]]*100000; mydescriptor = 
{'names': ('gender','age','weight'),'formats':('S1','f4', 
'f4')}""").repeat(3,10)

print Timer("numpy.fromiter((tuple(x) for x in results), 
dtype=mydescriptor, count=len(results))",
"""import numpy; results = [['M',64.0,75.0]]*100000; mydescriptor = 
{'names': ('gender','age','weight'),'formats':('S1','f4', 
'f4')}""").repeat(3,10)

print Timer("numpy.array(map(tuple, results), dtype=mydescriptor)",
"""import numpy; results = [['M',64.0,75.0]]*100000; mydescriptor = 
{'names': ('gender','age','weight'),'formats':('S1','f4', 
'f4')}""").repeat(3,10)

===>

[1.3928521641717035, 1.3892659541925021, 1.3949996438094785]
[1.344854164425926, 1.3157404083479882, 1.3207066819944986]
[1.2768430065832401, 1.2742884919731416, 1.2736657871321633]
[1.2081393026208644, 1.2025276955590734, 1.205871416618594]


-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From erin.sheldon at gmail.com  Mon Nov 13 10:10:42 2006
From: erin.sheldon at gmail.com (Erin Sheldon)
Date: Mon, 13 Nov 2006 10:10:42 -0500
Subject: array from list of lists
In-Reply-To: <1163405957.4221.25.camel@localhost.localdomain>
References: <20061112235629.GB5596@66-234-56-83.nyc.cable.nyct.net>
	<331116dc0611121650m812119es1c4381d7d4faa2ce@mail.gmail.com>
	<331116dc0611121710k7cc2c965pbec26543224e8dc9@mail.gmail.com>
	<200611122306.34072.pgmdevlist@gmail.com> <ej8s8l$1fr$1@sea.gmane.org>
	<331116dc0611122156m7df55cf5r939d0acab84cdeb@mail.gmail.com>
	<e06186140611122218i47032882r95a20b789ae2630a@mail.gmail.com>
	<331116dc0611122307p23705f5fjd2a038812d824044@mail.gmail.com>
	<1163405957.4221.25.camel@localhost.localdomain>
Message-ID: <331116dc0611130710y15a165dxf0feaa9884818160@mail.gmail.com>

On 11/13/06, Francesc Altet <faltet at carabos.com> wrote:
> In any case, you can also use rec.fromrecords for build recarrays from
> lists of lists. This breaks the aforementioned rule, but Travis allowed
> this because rec.* had to mimic numarray behaviour as much as possible.
> Here is an example of use:
>
> In [46]:mydescriptor = {'names': ('gender','age','weight'),
> 'formats':('S1','f4', 'f4')}
> In [47]:results=[['M',64.0,75.0],['F',25.0,60.0]]
> In [48]:a = numpy.rec.fromrecords(results, dtype=mydescriptor)
> In [49]:b = numpy.array([tuple(row) for row in results],
> dtype=mydescriptor)
> In [50]:a==b
> Out[50]:recarray([True, True], dtype=bool)
>
> OTOH, it is said in the docs that fromrecords is discouraged because it
> is somewhat slow, but apparently it has similar performance than using
> comprehensions lists:
>
> In [51]:Timer("numpy.rec.fromrecords(results, dtype=mydescriptor)",
> "import numpy; results = [['M',64.0,75.0]]*10000; mydescriptor =
> {'names': ('gender','age','weight'), 'formats':('S1','f4',
> 'f4')}").repeat(3,10)
> Out[51]:[0.44204592704772949, 0.43584394454956055, 0.50145101547241211]
>
> In [52]:Timer("numpy.array([tuple(row) for row in results],
> dtype=mydescriptor)", "import numpy; results = [['M',64.0,75.0]]*10000;
> mydescriptor = {'names': ('gender','age','weight'),
> 'formats':('S1','f4', 'f4')}").repeat(3,10)
> Out[52]:[0.49885106086730957, 0.4325258731842041, 0.43297886848449707]

I checked the code.  For lists of lists it just creates the recarray
and runs a loop copying in the data row by row.

The fact that they are of similar speed is actually good news
because the list comprehension was making an
extra copy of the data in memory.  For large memory usage,
which is my case, this 50% overhead would have been an issue.

Erin

-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From skokot at po.opole.pl  Mon Nov 13 11:58:22 2006
From: skokot at po.opole.pl (Seweryn Kokot)
Date: Mon, 13 Nov 2006 17:58:22 +0100
Subject: why the difference between ipython and python
	shell?
Message-ID: <877ixzl1w1.fsf@poczta.po.opole.pl>

Hello,

Why ipython and python interactive shell give two different information?

--- ipython 
Python 2.4.4 (#2, Oct 20 2006, 00:23:25) 
Type "copyright", "credits" or "license" for more information.

IPython 0.7.2 -- An enhanced Interactive Python.
?       -> Introduction to IPython's features.
%magic  -> Information about IPython's 'magic' % functions.
help    -> Python's own help system.
object? -> Details about 'object'. ?object also works, ?? prints more.

In [1]: from scipy import linalg

In [2]: help(linalg.eig)

Help on function eig in module numpy.linalg.linalg:

eig(a)
    eig(a) returns u,v  where u is the eigenvalues and
    v is a matrix of eigenvectors with vector v[:,i] corresponds to
    eigenvalue u[i].  Satisfies the equation dot(a, v[:,i]) =
    u[i]*v[:,i]
--- ipython 

while

--- python
Python 2.4.4 (#2, Oct 20 2006, 00:23:25) 
[GCC 4.1.2 20061015 (prerelease) (Debian 4.1.1-16.1)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> from scipy import linalg
>>> help(linalg.eig)

Help on function eig in module scipy.linalg.decomp:

eig(a, b=None, left=False, right=True, overwrite_a=False, overwrite_b=False)
    Solve ordinary and generalized eigenvalue problem
    of a square matrix.
    
    Inputs:
    
      a     -- An N x N matrix.
      b     -- An N x N matrix [default is identity(N)].
      left  -- Return left eigenvectors [disabled].
      right -- Return right eigenvectors [enabled].
      overwrite_a, overwrite_b -- save space by overwriting the a and/or
                                  b matrices (both False by default)
    
    Outputs:
    
      w      -- eigenvalues [left==right==False].
      w,vr   -- w and right eigenvectors [left==False,right=True].
      w,vl   -- w and left eigenvectors [left==True,right==False].
      w,vl,vr  -- [left==right==True].
    
    Definitions:
...
--- python

Any idea?
regards
SK


-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From tim.hochberg at ieee.org  Mon Nov 13 12:45:55 2006
From: tim.hochberg at ieee.org (Tim Hochberg)
Date: Mon, 13 Nov 2006 10:45:55 -0700
Subject: array from list of lists
In-Reply-To: <45588945.7030703@ieee.org>
References: <20061112235629.GB5596@66-234-56-83.nyc.cable.nyct.net>	<331116dc0611121650m812119es1c4381d7d4faa2ce@mail.gmail.com>	<331116dc0611121710k7cc2c965pbec26543224e8dc9@mail.gmail.com>	<200611122306.34072.pgmdevlist@gmail.com>	<ej8s8l$1fr$1@sea.gmane.org>	<331116dc0611122156m7df55cf5r939d0acab84cdeb@mail.gmail.com>	<e06186140611122218i47032882r95a20b789ae2630a@mail.gmail.com>	<331116dc0611122307p23705f5fjd2a038812d824044@mail.gmail.com>	<1163405957.4221.25.camel@localhost.localdomain>
	<45588945.7030703@ieee.org>
Message-ID: <4558AF53.8060908@ieee.org>

Tim Hochberg wrote:
> [SNIP]
>>   
>>     
>
> Just for completeness, I benchmarked the fromiter and map(tuple, 
> results)  solutions as well. Map is fastest, followed by fromiter, list 
> comprehension and then fromrecords. The differences are pretty minor 
> however, so I'd stick with whatever seems clearest.
>
> -tim
>
>   
Here's one more approach that's marginally faster than the map based 
solution and also won't chew up an extra memory since it's based on from 
iter:

numpy.fromiter(itertools.imap(tuple, results), dtype=mydescriptor, 
count=len(results))


[SNIP]

-tim


-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From charlesr.harris at gmail.com  Mon Nov 13 14:13:07 2006
From: charlesr.harris at gmail.com (Charles R Harris)
Date: Mon, 13 Nov 2006 12:13:07 -0700
Subject: why the difference between ipython and
	python shell?
In-Reply-To: <877ixzl1w1.fsf@poczta.po.opole.pl>
References: <877ixzl1w1.fsf@poczta.po.opole.pl>
Message-ID: <e06186140611131113v10c07711s49a3e69e09810116@mail.gmail.com>

On 11/13/06, Seweryn Kokot <skokot at po.opole.pl> wrote:
>
> Hello,
>
> Why ipython and python interactive shell give two different information?
>
> --- ipython
> Python 2.4.4 (#2, Oct 20 2006, 00:23:25)
> Type "copyright", "credits" or "license" for more information.
>
> IPython 0.7.2 -- An enhanced Interactive Python.
> ?       -> Introduction to IPython's features.
> %magic  -> Information about IPython's 'magic' % functions.
> help    -> Python's own help system.
> object? -> Details about 'object'. ?object also works, ?? prints more.
>
> In [1]: from scipy import linalg
>
> In [2]: help(linalg.eig)
>
> Help on function eig in module numpy.linalg.linalg:
>
> eig(a)
>     eig(a) returns u,v  where u is the eigenvalues and
>     v is a matrix of eigenvectors with vector v[:,i] corresponds to
>     eigenvalue u[i].  Satisfies the equation dot(a, v[:,i]) =
>     u[i]*v[:,i]
> --- ipython
>
> while
>
> --- python
> Python 2.4.4 (#2, Oct 20 2006, 00:23:25)
> [GCC 4.1.2 20061015 (prerelease) (Debian 4.1.1-16.1)] on linux2
> Type "help", "copyright", "credits" or "license" for more information.
> >>> from scipy import linalg
> >>> help(linalg.eig)
>
> Help on function eig in module scipy.linalg.decomp:
>
> eig(a, b=None, left=False, right=True, overwrite_a=False,
> overwrite_b=False)
>     Solve ordinary and generalized eigenvalue problem
>     of a square matrix.
>
>     Inputs:
>
>       a     -- An N x N matrix.
>       b     -- An N x N matrix [default is identity(N)].
>       left  -- Return left eigenvectors [disabled].
>       right -- Return right eigenvectors [enabled].
>       overwrite_a, overwrite_b -- save space by overwriting the a and/or
>                                   b matrices (both False by default)
>
>     Outputs:
>
>       w      -- eigenvalues [left==right==False].
>       w,vr   -- w and right eigenvectors [left==False,right=True].
>       w,vl   -- w and left eigenvectors [left==True,right==False].
>       w,vl,vr  -- [left==right==True].
>
>     Definitions:
> ...
> --- python
>
> Any idea?


I expect scipy.linalg and numpy.linalg are different modules containing
different functions. That said, the documentation in scipy.linalg looks
quite a bit more complete.

Chuck
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20061113/5f3d8c77/attachment.html>
-------------- next part --------------
-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
-------------- next part --------------
_______________________________________________
Numpy-discussion mailing list
Numpy-discussion at lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/numpy-discussion

From erin.sheldon at gmail.com  Mon Nov 13 14:43:20 2006
From: erin.sheldon at gmail.com (Erin Sheldon)
Date: Mon, 13 Nov 2006 14:43:20 -0500
Subject: array from list of lists
In-Reply-To: <4558AF53.8060908@ieee.org>
References: <20061112235629.GB5596@66-234-56-83.nyc.cable.nyct.net>
	<331116dc0611121710k7cc2c965pbec26543224e8dc9@mail.gmail.com>
	<200611122306.34072.pgmdevlist@gmail.com> <ej8s8l$1fr$1@sea.gmane.org>
	<331116dc0611122156m7df55cf5r939d0acab84cdeb@mail.gmail.com>
	<e06186140611122218i47032882r95a20b789ae2630a@mail.gmail.com>
	<331116dc0611122307p23705f5fjd2a038812d824044@mail.gmail.com>
	<1163405957.4221.25.camel@localhost.localdomain>
	<45588945.7030703@ieee.org> <4558AF53.8060908@ieee.org>
Message-ID: <331116dc0611131143h7e6e3606ra71646e9d0ded078@mail.gmail.com>

On 11/13/06, Tim Hochberg <tim.hochberg at ieee.org> wrote:
> Here's one more approach that's marginally faster than the map based
> solution and also won't chew up an extra memory since it's based on from
> iter:
>
> numpy.fromiter(itertools.imap(tuple, results), dtype=mydescriptor,
> count=len(results))

Yes, this is what I need.  BTW, there is no doc string for this.  I just added
an example the the Numpy Example List.

Thanks,
Erin

-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From myeates at jpl.nasa.gov  Mon Nov 13 16:25:29 2006
From: myeates at jpl.nasa.gov (Mathew Yeates)
Date: Mon, 13 Nov 2006 13:25:29 -0800
Subject: memmap working?
Message-ID: <4558E2C9.6030007@jpl.nasa.gov>

I have a memory mapped array. When I try and assign data, my mem usage 
goes through the roof.
example: outdat[filenum,:]=outarr where outdat is memory mapped.

Anybody know how to avoid this?

Mathew



-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From tim.hochberg at ieee.org  Mon Nov 13 16:29:11 2006
From: tim.hochberg at ieee.org (Tim Hochberg)
Date: Mon, 13 Nov 2006 14:29:11 -0700
Subject: array from list of lists
In-Reply-To: <331116dc0611131143h7e6e3606ra71646e9d0ded078@mail.gmail.com>
References: <20061112235629.GB5596@66-234-56-83.nyc.cable.nyct.net>	<331116dc0611121710k7cc2c965pbec26543224e8dc9@mail.gmail.com>	<200611122306.34072.pgmdevlist@gmail.com>
	<ej8s8l$1fr$1@sea.gmane.org>	<331116dc0611122156m7df55cf5r939d0acab84cdeb@mail.gmail.com>	<e06186140611122218i47032882r95a20b789ae2630a@mail.gmail.com>	<331116dc0611122307p23705f5fjd2a038812d824044@mail.gmail.com>	<1163405957.4221.25.camel@localhost.localdomain>	<45588945.7030703@ieee.org>
	<4558AF53.8060908@ieee.org>
	<331116dc0611131143h7e6e3606ra71646e9d0ded078@mail.gmail.com>
Message-ID: <4558E3A7.1000103@ieee.org>

Erin Sheldon wrote:
> On 11/13/06, Tim Hochberg <tim.hochberg at ieee.org> wrote:
>   
>> Here's one more approach that's marginally faster than the map based
>> solution and also won't chew up an extra memory since it's based on from
>> iter:
>>
>> numpy.fromiter(itertools.imap(tuple, results), dtype=mydescriptor,
>> count=len(results))
>>     
>
> Yes, this is what I need.  BTW, there is no doc string for this. 
Yeah, I noticed that too. I swear I wrote one at one point I'm not sure 
what happened to it. Sigh.

>  I just added
> an example the the Numpy Example List.
>   
Great.

-tim




-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From stefan at sun.ac.za  Mon Nov 13 16:50:13 2006
From: stefan at sun.ac.za (Stefan van der Walt)
Date: Mon, 13 Nov 2006 23:50:13 +0200
Subject: array from list of lists
In-Reply-To: <4558E3A7.1000103@ieee.org>
References: <200611122306.34072.pgmdevlist@gmail.com>
	<ej8s8l$1fr$1@sea.gmane.org>
	<331116dc0611122156m7df55cf5r939d0acab84cdeb@mail.gmail.com>
	<e06186140611122218i47032882r95a20b789ae2630a@mail.gmail.com>
	<331116dc0611122307p23705f5fjd2a038812d824044@mail.gmail.com>
	<1163405957.4221.25.camel@localhost.localdomain>
	<45588945.7030703@ieee.org> <4558AF53.8060908@ieee.org>
	<331116dc0611131143h7e6e3606ra71646e9d0ded078@mail.gmail.com>
	<4558E3A7.1000103@ieee.org>
Message-ID: <20061113215013.GE10559@mentat.za.net>

On Mon, Nov 13, 2006 at 02:29:11PM -0700, Tim Hochberg wrote:
> Erin Sheldon wrote:
> > On 11/13/06, Tim Hochberg <tim.hochberg at ieee.org> wrote:
> >   
> >> Here's one more approach that's marginally faster than the map based
> >> solution and also won't chew up an extra memory since it's based on from
> >> iter:
> >>
> >> numpy.fromiter(itertools.imap(tuple, results), dtype=mydescriptor,
> >> count=len(results))
> >>     
> >
> > Yes, this is what I need.  BTW, there is no doc string for this. 
> Yeah, I noticed that too. I swear I wrote one at one point I'm not sure 
> what happened to it. Sigh.

A typo slipped into add_newdocs.py.  Fixed in SVN.

Cheers
St?fan

-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From charlesr.harris at gmail.com  Mon Nov 13 17:03:38 2006
From: charlesr.harris at gmail.com (Charles R Harris)
Date: Mon, 13 Nov 2006 15:03:38 -0700
Subject: memmap working?
In-Reply-To: <4558E2C9.6030007@jpl.nasa.gov>
References: <4558E2C9.6030007@jpl.nasa.gov>
Message-ID: <e06186140611131403o3b788140m54b4c65f4954286@mail.gmail.com>

On 11/13/06, Mathew Yeates <myeates at jpl.nasa.gov> wrote:
>
> I have a memory mapped array. When I try and assign data, my mem usage
> goes through the roof.


Is it cache memory or process memory? I think a memory mapped file will keep
pages cached in memory until the space is needed so as to avoid unneeded io.
At least that is what happens in linux.

Chuck
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20061113/98b78d90/attachment.html>
-------------- next part --------------
-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
-------------- next part --------------
_______________________________________________
Numpy-discussion mailing list
Numpy-discussion at lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/numpy-discussion

From myeates at jpl.nasa.gov  Mon Nov 13 17:25:41 2006
From: myeates at jpl.nasa.gov (Mathew Yeates)
Date: Mon, 13 Nov 2006 14:25:41 -0800
Subject: memmap working?
In-Reply-To: <e06186140611131403o3b788140m54b4c65f4954286@mail.gmail.com>
References: <4558E2C9.6030007@jpl.nasa.gov>
	<e06186140611131403o3b788140m54b4c65f4954286@mail.gmail.com>
Message-ID: <4558F0E5.4090108@jpl.nasa.gov>

Not sure. When I run "top" I see the line
Memory: 6016M real, 2895M free, 4174M swap in use, 2427M swap free

Its the second number that drops like a rock. Plus, it never comes back 
until I quit the program. This is a great way to turn my machine into a 
nice desk ornament!

Mathew


Charles R Harris wrote:
>
>
> On 11/13/06, *Mathew Yeates* <myeates at jpl.nasa.gov 
> <mailto:myeates at jpl.nasa.gov>> wrote:
>
>     I have a memory mapped array. When I try and assign data, my mem usage
>     goes through the roof.
>
>
> Is it cache memory or process memory? I think a memory mapped file 
> will keep pages cached in memory until the space is needed so as to 
> avoid unneeded io. At least that is what happens in linux.
>
> Chuck
>
>
> ------------------------------------------------------------------------
>
> -------------------------------------------------------------------------
> Using Tomcat but need to do more? Need to support web services, security?
> Get stuff done quickly with pre-integrated technology to make your job easier
> Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
> http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
> ------------------------------------------------------------------------
>
> _______________________________________________
> Numpy-discussion mailing list
> Numpy-discussion at lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/numpy-discussion
>   



-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From vallis.35530053 at bloglines.com  Mon Nov 13 20:32:24 2006
From: vallis.35530053 at bloglines.com (vallis.35530053 at bloglines.com)
Date: 14 Nov 2006 01:32:24 -0000
Subject: numpy.where behavior
Message-ID: <1163467944.2853004636.6699.sendItem@bloglines.com>

Using numpy 1.0, why does



>>> a = numpy.array([0.0,1.0,2.0],'d')

>>> numpy.where(a
== 0.0,1,1/a)



give the correct result, but with the warning "Warning: divide
by zero encountered in divide"?



? I thought that the point of where was
that the second expression is never used for the elements where the condition
evaluates true. 



If this is the desired behavior, is there a way to suppress
the warning?



Thanks!



Michele

-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From robert.kern at gmail.com  Mon Nov 13 20:42:40 2006
From: robert.kern at gmail.com (Robert Kern)
Date: Mon, 13 Nov 2006 19:42:40 -0600
Subject: numpy.where behavior
In-Reply-To: <1163467944.2853004636.6699.sendItem@bloglines.com>
References: <1163467944.2853004636.6699.sendItem@bloglines.com>
Message-ID: <ejb6us$i5$1@sea.gmane.org>

vallis.35530053 at bloglines.com wrote:
> ? I thought that the point of where was
> that the second expression is never used for the elements where the condition
> evaluates true. 

It is not used, but the expression still gets evaluated. There's really no way
around that.

> If this is the desired behavior, is there a way to suppress
> the warning?

In [1]: from numpy import *

In [2]: a = zeros(3)

In [3]: 1/a
Warning: divide by zero encountered in divide
Warning: invalid value encountered in double_scalars
Out[3]: array([              inf,               inf,               inf])

In [4]: seterr(divide='ignore', invalid='ignore')
Out[4]: {'divide': 'print', 'invalid': 'print', 'over': 'print', 'under': 'ignore'}

In [5]: 1/a
Out[5]: array([              inf,               inf,               inf])

In [6]: seterr?
Type:           function
Base Class:     <type 'function'>
Namespace:      Interactive
File:
/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/site-packages/numpy-1.0.1.dev3432-py2.5-macosx-10.4-i386.egg
/numpy/core/numeric.py
Definition:     seterr(all=None, divide=None, over=None, under=None, invalid=None)
Docstring:
    Set how floating-point errors are handled.

    Valid values for each type of error are the strings
    "ignore", "warn", "raise", and "call". Returns the old settings.
    If 'all' is specified, values that are not otherwise specified
    will be set to 'all', otherwise they will retain their old
    values.

    Note that operations on integer scalar types (such as int16) are
    handled like floating point, and are affected by these settings.

    Example:

    >>> seterr(over='raise')
    {'over': 'ignore', 'divide': 'ignore', 'invalid': 'ignore', 'under': 'ignore'}
    >>> seterr(all='warn', over='raise')
    {'over': 'raise', 'divide': 'ignore', 'invalid': 'ignore', 'under': 'ignore'}
    >>> int16(32000) * int16(3)
    Traceback (most recent call last):
          File "<stdin>", line 1, in ?
    FloatingPointError: overflow encountered in short_scalars
    >>> seterr(all='ignore')
    {'over': 'ignore', 'divide': 'ignore', 'invalid': 'ignore', 'under': 'ignore'}


-- 
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless enigma
 that is made terrible by our own mad attempt to interpret it as though it had
 an underlying truth."
  -- Umberto Eco


-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From charlesr.harris at gmail.com  Mon Nov 13 20:54:20 2006
From: charlesr.harris at gmail.com (Charles R Harris)
Date: Mon, 13 Nov 2006 18:54:20 -0700
Subject: memmap working?
In-Reply-To: <4558F0E5.4090108@jpl.nasa.gov>
References: <4558E2C9.6030007@jpl.nasa.gov>
	<e06186140611131403o3b788140m54b4c65f4954286@mail.gmail.com>
	<4558F0E5.4090108@jpl.nasa.gov>
Message-ID: <e06186140611131754r29eee3d6re4d7e71feb2c902@mail.gmail.com>

On 11/13/06, Mathew Yeates <myeates at jpl.nasa.gov> wrote:
>
> Not sure. When I run "top" I see the line
> Memory: 6016M real, 2895M free, 4174M swap in use, 2427M swap free
>
> Its the second number that drops like a rock. Plus, it never comes back
> until I quit the program. This is a great way to turn my machine into a
> nice desk ornament!


try free:

$[charris at fedora ~]$ free
             total       used       free     shared    buffers     cached
Mem:       1034952     995212      39740          0     126616     328124
-/+ buffers/cache:     540472     494480
Swap:       979956        152     979804

The second line under 'used' shows actual program use, i.e. used - buffers -
cached from the first line. But if your system is dying I don't know what to
say. My knowledge of these things is a bit sketchy.

Chuck
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20061113/e32edbda/attachment.html>
-------------- next part --------------
-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
-------------- next part --------------
_______________________________________________
Numpy-discussion mailing list
Numpy-discussion at lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/numpy-discussion

From tim.hochberg at ieee.org  Mon Nov 13 22:10:09 2006
From: tim.hochberg at ieee.org (Tim Hochberg)
Date: Mon, 13 Nov 2006 20:10:09 -0700
Subject: numpy.where behavior
In-Reply-To: <1163467944.2853004636.6699.sendItem@bloglines.com>
References: <1163467944.2853004636.6699.sendItem@bloglines.com>
Message-ID: <45593391.4040001@ieee.org>

vallis.35530053 at bloglines.com wrote:
> Using numpy 1.0, why does
>
>
>
>   
>>>> a = numpy.array([0.0,1.0,2.0],'d')
>>>>         
>
>   
>>>> numpy.where(a
>>>>         
> == 0.0,1,1/a)
>
>
>
> give the correct result, but with the warning "Warning: divide
> by zero encountered in divide"?
>
>
>
> ? I thought that the point of where was
> that the second expression is never used for the elements where the condition
> evaluates true. 
>
>
>
> If this is the desired behavior, is there a way to suppress
> the warning?
>   
Robert Kern has already pointed you to seterr. If you are using Python 
2.5, you also have the option using the with statement, which is more 
convenient if you want to temporarily change the error state. You'll 
need a "from __future__ import with_statement" at the top of your file. 
Then you can temporarily disable errors as shown:

     >>> a = zeros([3])
     >>> b = 1/a  # This will warn
    Warning: divide by zero encountered in divide
     >>> with errstate(divide='ignore'): # But this will not
    ...     c = 1/a
    ...
     >>> d = 1/a # And this will warn again since the error state is
    restored when we exit the block
    Warning: divide by zero encountered in divide


Another little tidbit: this is not as general as where, and could 
probably be considered a little too clever to be clear, but:

        b = 1 / (a + (a==0.0))

is faster than using where in this particular case and sidesteps the 
divide by zero issue altogether.

-tim



-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From pfdubois at gmail.com  Mon Nov 13 23:13:39 2006
From: pfdubois at gmail.com (Paul Dubois)
Date: Mon, 13 Nov 2006 20:13:39 -0800
Subject: numpy.where behavior
In-Reply-To: <45593391.4040001@ieee.org>
References: <1163467944.2853004636.6699.sendItem@bloglines.com>
	<45593391.4040001@ieee.org>
Message-ID: <f74a6c2f0611132013k4e32ca09i69b8c97f27bd9e04@mail.gmail.com>

Unfortunately, where does not have the behavior of not evaluating the second
argument where the first one is true. That would be nice (if the speed were
ok) but it isn't possible unless where is built into the language, since
where doesn't even get called until the arguments have all been calculated.
It was intended as having a different use than avoiding zero-divide. The ma
package can calculate 1/a without problem, resulting in masked results where
a is 0.0.

I put where into numeric after it had proved invaluable in Basis, even
though it has this limitation; it takes care of doing both merge and
compress.

On 13 Nov 2006 20:02:31 -0800, Tim Hochberg <tim.hochberg at ieee.org> wrote:
>
> vallis.35530053 at bloglines.com wrote:
> > Using numpy 1.0, why does
> >
> >
> >
> >
> >>>> a = numpy.array([0.0,1.0,2.0],'d')
> >>>>
> >
> >
> >>>> numpy.where(a
> >>>>
> > == 0.0,1,1/a)
> >
> >
> >
> > give the correct result, but with the warning "Warning: divide
> > by zero encountered in divide"?
> >
> >
> >
> > ? I thought that the point of where was
> > that the second expression is never used for the elements where the
> condition
> > evaluates true.
> >
> >
> >
> > If this is the desired behavior, is there a way to suppress
> > the warning?
> >
> Robert Kern has already pointed you to seterr. If you are using Python
> 2.5, you also have the option using the with statement, which is more
> convenient if you want to temporarily change the error state. You'll
> need a "from __future__ import with_statement" at the top of your file.
> Then you can temporarily disable errors as shown:
>
>      >>> a = zeros([3])
>      >>> b = 1/a  # This will warn
>     Warning: divide by zero encountered in divide
>      >>> with errstate(divide='ignore'): # But this will not
>     ...     c = 1/a
>     ...
>      >>> d = 1/a # And this will warn again since the error state is
>     restored when we exit the block
>     Warning: divide by zero encountered in divide
>
>
> Another little tidbit: this is not as general as where, and could
> probably be considered a little too clever to be clear, but:
>
>         b = 1 / (a + (a==0.0))
>
> is faster than using where in this particular case and sidesteps the
> divide by zero issue altogether.
>
> -tim
>
>
>
> -------------------------------------------------------------------------
> Using Tomcat but need to do more? Need to support web services, security?
> Get stuff done quickly with pre-integrated technology to make your job
> easier
> Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
> http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
> _______________________________________________
> Numpy-discussion mailing list
> Numpy-discussion at lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/numpy-discussion
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20061113/c487d10d/attachment.html>
-------------- next part --------------
-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
-------------- next part --------------
_______________________________________________
Numpy-discussion mailing list
Numpy-discussion at lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/numpy-discussion

From robert.kern at gmail.com  Mon Nov 13 23:16:55 2006
From: robert.kern at gmail.com (Robert Kern)
Date: Mon, 13 Nov 2006 22:16:55 -0600
Subject: numpy.where behavior
In-Reply-To: <45593391.4040001@ieee.org>
References: <1163467944.2853004636.6699.sendItem@bloglines.com>
	<45593391.4040001@ieee.org>
Message-ID: <ejbg02$kqo$1@sea.gmane.org>

Tim Hochberg wrote:
> Another little tidbit: this is not as general as where, and could 
> probably be considered a little too clever to be clear, but:
> 
>         b = 1 / (a + (a==0.0))
> 
> is faster than using where in this particular case and sidesteps the 
> divide by zero issue altogether.

A less clever approach that does much the same thing:

  b = 1.0 / where(a==0, 1.0, a)

-- 
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless enigma
 that is made terrible by our own mad attempt to interpret it as though it had
 an underlying truth."
  -- Umberto Eco


-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From drrfqwrif at pace-engineering.com  Tue Nov 14 01:26:00 2006
From: drrfqwrif at pace-engineering.com (underwater stolen)
Date: Tue, 14 Nov 2006 14:26:00 +0800
Subject: long rest
References: <000701c707b5$c0db48e0$00000000@b5f8e7b051d247b>
Message-ID: <000701c707b5$c0db48e0$00000000@b5f8e7b051d247b>


  ----- Original Message ----- 
  From: numpy-discussion at lists.sourceforge.net 
  To: drrfqwrif at pace-engineering.com 
  Sent: Wednesday, November 00:20:00 PM
  Subject: long rest


  Obtained is Mars Only am person.
  Beetle ju Boulder is Dash Brave Dwarves of gamefree Apple mac is.
  Legendary of boasts superior physics concept advanced collision system vibrant.
  Anything pure adrenaline of rush Buckleup because. While is lawns green or appearance!
  Future fears behind set am mind galaga shareware Nether Star.
  Hand in Foot partner outwit opponent just hands. Fantastic routes is will not choose a one fastest superbikes! Need fb Best fe pages Next in gtshowing Domination.
  We advise Black Jack gives every player.
  Physics concept advanced collision am system vibrant cool? Form row Make sure room could.
  Returns remake famous Dyna four castle. Explosive solitaire peg am.
  Wont feel anything pure am. Contact Copyright copy rights design.
  Not is choose one.
  Atmosphere is enjoy freeware.
  Ten times addition energizing gameplay legendary boasts superior physics. Form row Make sure room could.
  Beetle ju Boulder is Dash Brave Dwarves of gamefree Apple mac is.
  Move colored in blocks around form row Make. Brave Dwarves gamefree Apple!
  Dont in think its. Limited Pick bonuses Geeks Devil.
  Submit Advertise in Privacy is Policy Disclaimer or Contact Copyright copy!
  Dont in think its.
  Brains come in with different strategies can?
  Cut grass statue gtleave News.
  Both fun addictive Move colored blocks around am form.
  On hex am grid using?
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20061114/2503165c/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: while.gif
Type: image/gif
Size: 21684 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20061114/2503165c/attachment.gif>
-------------- next part --------------
-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
-------------- next part --------------
_______________________________________________
Numpy-discussion mailing list
Numpy-discussion at lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/numpy-discussion

From svetosch at gmx.net  Tue Nov 14 04:50:44 2006
From: svetosch at gmx.net (Sven Schreiber)
Date: Tue, 14 Nov 2006 10:50:44 +0100
Subject: why the difference between ipython
 and	python shell?
In-Reply-To: <e06186140611131113v10c07711s49a3e69e09810116@mail.gmail.com>
References: <877ixzl1w1.fsf@poczta.po.opole.pl>
	<e06186140611131113v10c07711s49a3e69e09810116@mail.gmail.com>
Message-ID: <45599174.4090906@gmx.net>

Charles R Harris schrieb:

>     In [1]: from scipy import linalg
> 
>     In [2]: help(linalg.eig)
> 

>
>     >>> from scipy import linalg
>     >>> help(linalg.eig)
> 
>     Help on function eig in module scipy.linalg.decomp:
>
> 
> I expect scipy.linalg and numpy.linalg are different modules containing
> different functions. That said, the documentation in scipy.linalg looks
> quite a bit more complete.
> 
> Chuck

Sure, but Seweryn used the same import statement from scipy and never
explicitly referred to numpy, so there must be some subtle import voodoo
going on. Or did you not show us everything, Seweryn?

-sven

-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From skokot at po.opole.pl  Tue Nov 14 05:05:19 2006
From: skokot at po.opole.pl (Seweryn Kokot)
Date: Tue, 14 Nov 2006 11:05:19 +0100
Subject: why the difference between ipython
	and	python shell?
References: <877ixzl1w1.fsf@poczta.po.opole.pl>
	<e06186140611131113v10c07711s49a3e69e09810116@mail.gmail.com>
	<45599174.4090906@gmx.net>
Message-ID: <87lkmewdgg.fsf@poczta.po.opole.pl>

Sven Schreiber <svetosch at gmx.net> writes:
>
> Sure, but Seweryn used the same import statement from scipy and never
> explicitly referred to numpy, so there must be some subtle import voodoo
> going on. Or did you not show us everything, Seweryn?

It's all ok now, It was my mistake. The problem was that in ipython I
typed "from scipy import linalg" which is wrong and being surprised by the
output I open python shell and tried different combinations of import,
among others "from scipy import *" and this is the reason of the
difference. So now in ipython I get the same output when typing
help(linalg.eig).
Sorry for bothering you,
regards,
SK


-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From meesters at uni-mainz.de  Tue Nov 14 06:09:23 2006
From: meesters at uni-mainz.de (Christian Meesters)
Date: Tue, 14 Nov 2006 12:09:23 +0100
Subject: accessing FORTRAN from Python
Message-ID: <200611141209.23481.meesters@uni-mainz.de>

Hoi,

thanks to Robert Kern who helped me out yesterday on the f2py-list, I was able 
to make some progress in accessing FORTRAN from Python. But only some 
progress ...

If I have the following code, named 'hello.f':
C File hello.f
      subroutine foo (a)
      integer a
      print*, "Hello from Fortran!"
      print*, "a=",a
      end

and compile it with g77 -shared -fPIC hello.f -o hello.so

and then start python, I get the following:
>>> from numpy import *
>>> from ctypes import c_int,  POINTER, byref
>>> hellolib = ctypeslib.load_library('hello', '.')
>>> hello = hellolib.foo_
>>> hello(42)
 Hello from Fortran!
Segmentation fault

Can anybody tell me where my mistake is? (Currently python 2.4.1 (no intention 
to update soon), the most recent ctypes, and numpy '1.0.dev3341' from svn.)

And a second question: Are there simple examples around which show how to pass 
and retrieve lists, numpy arrays, and dicts to and from FORTRAN? Despite an 
intensive web search I couldn't find anything.

TIA
Christian

-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From gnata at obs.univ-lyon1.fr  Tue Nov 14 06:31:41 2006
From: gnata at obs.univ-lyon1.fr (Xavier Gnata)
Date: Tue, 14 Nov 2006 12:31:41 +0100
Subject: A way to compute a 2d histogram
In-Reply-To: <45472DC9.2090802@obs.univ-lyon1.fr>
References: <453E3FE5.1080201@obs.univ-lyon1.fr>	<91cf711d0610241002u765170b5hcb9bacc850a205e0@mail.gmail.com>	<91cf711d0610241340o3fbcc93x235849b8d63ffc16@mail.gmail.com>	<453F57C6.2020101@obs.univ-lyon1.fr>	<91cf711d0610250613q58056568p98dbc342728212fc@mail.gmail.com>
	<45472DC9.2090802@obs.univ-lyon1.fr>
Message-ID: <4559A91D.9020701@obs.univ-lyon1.fr>

Hi,

IFAICS these new histograms versions have not yet been merged to svn.
Are they problems to be solve before to be able to merge them?
How could we help?

Xavier

> Hi,
>
> Your histograms functions look fine for me :)
> As it is a quite usual  operation on an array, I would suggest  to put 
> it in numpy as numpy.histogram. IMHO, there is no point to create an 
> numpy.stats only for histograms (or do you have plans to move other 
> stats related function to numpy.stats?)
>
> Xavier.
>
>   
>> Nicolas, thanks for the bug report, I fooled around with argument 
>> passing and should have checked every case.
>>
>> You'll find the histogram function that deals with weights on the 
>> numpy trac ticket 189, <http://projects.scipy.org/scipy/numpy/ticket/189>
>> I'm waiting for some hints as to where the histogram function should 
>> reside (numpy.histogram, numpy.stats.histogram, ...) before submitting 
>> a patch .
>>
>> Salut,
>> David
>>
>>
>> 2006/10/25, Nicolas Champavert <nicolas.champavert at obs.univ-lyon1.fr 
>> <mailto:nicolas.champavert at obs.univ-lyon1.fr>>:
>>
>>     Hi,
>>
>>         it would be great if you could add the weight option in the 1D
>>     histogram too.
>>
>>     Nicolas
>>
>>     David Huard a ?crit :
>>     > Xavier,
>>     > Here is the patch against svn. Please report any bug. I haven't had
>>     > the time to test it extensively, something that should be done
>>     before
>>     > commiting the patch to the repo. I'd appreciate your feedback.
>>     >
>>     > David
>>     >
>>     > 2006/10/24, David Huard < david.huard at gmail.com
>>     <mailto:david.huard at gmail.com>
>>     > <mailto:david.huard at gmail.com <mailto:david.huard at gmail.com>>>:
>>     >
>>     >     Hi Xavier,
>>     >
>>     >     You could tweak histogram2d to do what you want, or you
>>     could give
>>     >     me a couple of days and I'll do it and let you know. If you want
>>     >     to help, you could write a test using your particular
>>     application
>>     >     and data.
>>     >
>>     >     David
>>     >
>>     >
>>     >     2006/10/24, Xavier Gnata < gnata at obs.univ-lyon1.fr
>>     <mailto:gnata at obs.univ-lyon1.fr>
>>     >     <mailto:gnata at obs.univ-lyon1.fr
>>     <mailto:gnata at obs.univ-lyon1.fr>>>:
>>     >
>>     >         Hi,
>>     >
>>     >         I have a set of 3 1D large arrays.
>>     >         The first 2 one stand for the coordinates of particules and
>>     >         the last one
>>     >         for their masses.
>>     >         I would like to be able to plot this data ie to compute
>>     a 2D
>>     >         histogram
>>     >         summing the masses in each bin.
>>     >         I cannot find a way to do that without any loop on the
>>     indices
>>     >         resulting
>>     >         too a very slow function.
>>     >
>>     >         I'm looking for an elegant way to do that with numpy (or
>>     >         scipy??) function.
>>     >
>>     >         For instance, scipy.histogram2d cannot do the job because it
>>     >         only counts
>>     >         the number of samples in each bin.
>>     >         There is no way to deal with weights.
>>     >
>>     >         Xavier.
>>     >
>>     >
>>     >         --
>>     >         ############################################
>>     >         Xavier Gnata
>>     >         CRAL - Observatoire de Lyon
>>     >         9, avenue Charles Andr?
>>     >         69561 Saint Genis Laval cedex
>>     >         Phone: +33 4 78 86 85 28
>>     >         Fax: +33 4 78 86 83 86
>>     >         E-mail: gnata at obs.univ-lyon1.fr
>>     <mailto:gnata at obs.univ-lyon1.fr> <mailto:gnata at obs.univ-lyon1.fr
>>     <mailto:gnata at obs.univ-lyon1.fr>>
>>     >         ############################################
>>     >
>>     >
>>     >        
>>     -------------------------------------------------------------------------
>>     >         Using Tomcat but need to do more? Need to support web
>>     >         services, security?
>>     >         Get stuff done quickly with pre-integrated technology to
>>     make
>>     >         your job easier
>>     >         Download IBM WebSphere Application Server v.1.0.1 based on
>>     >         Apache Geronimo
>>     >        
>>     http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
>>     <http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642>
>>     >        
>>     <http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
>>     <http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642>>
>>     >         _______________________________________________
>>     >         Numpy-discussion mailing list
>>     >         Numpy-discussion at lists.sourceforge.net
>>     <mailto:Numpy-discussion at lists.sourceforge.net>
>>     >         <mailto:Numpy-discussion at lists.sourceforge.net
>>     <mailto:Numpy-discussion at lists.sourceforge.net>>
>>     >        
>>     https://lists.sourceforge.net/lists/listinfo/numpy-discussion
>>     >
>>     >
>>     >
>>     >
>>
>>     -------------------------------------------------------------------------
>>     Using Tomcat but need to do more? Need to support web services,
>>     security?
>>     Get stuff done quickly with pre-integrated technology to make your
>>     job easier
>>     Download IBM WebSphere Application Server v.1.0.1 based on Apache
>>     Geronimo
>>     http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
>>     <http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642>
>>     _______________________________________________
>>     Numpy-discussion mailing list
>>     Numpy-discussion at lists.sourceforge.net
>>     <mailto:Numpy-discussion at lists.sourceforge.net>
>>     https://lists.sourceforge.net/lists/listinfo/numpy-discussion
>>
>>
>> ------------------------------------------------------------------------
>>
>> -------------------------------------------------------------------------
>> Using Tomcat but need to do more? Need to support web services, security?
>> Get stuff done quickly with pre-integrated technology to make your job easier
>> Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
>> http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
>> ------------------------------------------------------------------------
>>
>> _______________________________________________
>> Numpy-discussion mailing list
>> Numpy-discussion at lists.sourceforge.net
>> https://lists.sourceforge.net/lists/listinfo/numpy-discussion
>>   
>>     
>
>
>   


-- 
############################################
Xavier Gnata
CRAL - Observatoire de Lyon
9, avenue Charles Andr?
69561 Saint Genis Laval cedex
Phone: +33 4 78 86 85 28
Fax: +33 4 78 86 83 86
E-mail: gnata at obs.univ-lyon1.fr
############################################ 


-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From nmarais at sun.ac.za  Tue Nov 14 09:37:52 2006
From: nmarais at sun.ac.za (Neilen Marais)
Date: Tue, 14 Nov 2006 16:37:52 +0200
Subject: 32bit integers as list indices
Message-ID: <pan.2006.11.14.14.37.49.702134@sun.ac.za>

Hi,

I'm not sure if the following is expected to work on a 64bit machine:

In [381]: import numpy as N
In [382]: l = range(3)
In [383]: i32 = N.array([0,2], N.int32)
In [384]: i64 = N.array([0,2], N.int64)
In [385]: l[i32[0]]
---------------------------------------------------------------------------
exceptions.TypeError                           Traceback (most recent call last)

/home/brick/akademie/NewCode/working/<ipython console> 

TypeError: list indices must be integers

In [386]: l[i64[0]]
Out[386]: 0

I'd expect the 32-bit indices to work since they can be upcast to 64bit without
loss. Am I silly for thinking this way, or is it something numpy can/should
address? This came up while working with sparse matrices:

http://projects.scipy.org/scipy/scipy/ticket/307

Regards
Neilen

-- 
you know its kind of tragic 
we live in the new world
but we've lost the magic
-- Battery 9 (www.battery9.co.za)


-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From david.huard at gmail.com  Tue Nov 14 09:53:27 2006
From: david.huard at gmail.com (David Huard)
Date: Tue, 14 Nov 2006 09:53:27 -0500
Subject: A way to compute a 2d histogram
In-Reply-To: <4559A91D.9020701@obs.univ-lyon1.fr>
References: <453E3FE5.1080201@obs.univ-lyon1.fr>
	<91cf711d0610241002u765170b5hcb9bacc850a205e0@mail.gmail.com>
	<91cf711d0610241340o3fbcc93x235849b8d63ffc16@mail.gmail.com>
	<453F57C6.2020101@obs.univ-lyon1.fr>
	<91cf711d0610250613q58056568p98dbc342728212fc@mail.gmail.com>
	<45472DC9.2090802@obs.univ-lyon1.fr>
	<4559A91D.9020701@obs.univ-lyon1.fr>
Message-ID: <91cf711d0611140653q5b3730fdy2d8e57d2edbe4e58@mail.gmail.com>

2006/11/14, Xavier Gnata <gnata at obs.univ-lyon1.fr>:
>
> Hi,
>
> IFAICS these new histograms versions have not yet been merged to svn.
> Are they problems to be solve before to be able to merge them?
> How could we help?


No, I'm just overloaded with other stuff, I'll submit a patch on Trac today
or tomorrow. Up to now, Travis did the merges, but since he looks pretty
busy right now, I don't know when it'll show up on svn.

David



Xavier
>
> > Hi,
> >
> > Your histograms functions look fine for me :)
> > As it is a quite usual  operation on an array, I would suggest  to put
> > it in numpy as numpy.histogram. IMHO, there is no point to create an
> > numpy.stats only for histograms (or do you have plans to move other
> > stats related function to numpy.stats?)
> >
> > Xavier.
> >
> >
> >> Nicolas, thanks for the bug report, I fooled around with argument
> >> passing and should have checked every case.
> >>
> >> You'll find the histogram function that deals with weights on the
> >> numpy trac ticket 189, <
> http://projects.scipy.org/scipy/numpy/ticket/189>
> >> I'm waiting for some hints as to where the histogram function should
> >> reside (numpy.histogram, numpy.stats.histogram, ...) before submitting
> >> a patch .
> >>
> >> Salut,
> >> David
> >>
> >>
> >> 2006/10/25, Nicolas Champavert <nicolas.champavert at obs.univ-lyon1.fr
> >> <mailto:nicolas.champavert at obs.univ-lyon1.fr>>:
> >>
> >>     Hi,
> >>
> >>         it would be great if you could add the weight option in the 1D
> >>     histogram too.
> >>
> >>     Nicolas
> >>
> >>     David Huard a ?crit :
> >>     > Xavier,
> >>     > Here is the patch against svn. Please report any bug. I haven't
> had
> >>     > the time to test it extensively, something that should be done
> >>     before
> >>     > commiting the patch to the repo. I'd appreciate your feedback.
> >>     >
> >>     > David
> >>     >
> >>     > 2006/10/24, David Huard < david.huard at gmail.com
> >>     <mailto:david.huard at gmail.com>
> >>     > <mailto:david.huard at gmail.com <mailto:david.huard at gmail.com>>>:
> >>     >
> >>     >     Hi Xavier,
> >>     >
> >>     >     You could tweak histogram2d to do what you want, or you
> >>     could give
> >>     >     me a couple of days and I'll do it and let you know. If you
> want
> >>     >     to help, you could write a test using your particular
> >>     application
> >>     >     and data.
> >>     >
> >>     >     David
> >>     >
> >>     >
> >>     >     2006/10/24, Xavier Gnata < gnata at obs.univ-lyon1.fr
> >>     <mailto:gnata at obs.univ-lyon1.fr>
> >>     >     <mailto:gnata at obs.univ-lyon1.fr
> >>     <mailto:gnata at obs.univ-lyon1.fr>>>:
> >>     >
> >>     >         Hi,
> >>     >
> >>     >         I have a set of 3 1D large arrays.
> >>     >         The first 2 one stand for the coordinates of particules
> and
> >>     >         the last one
> >>     >         for their masses.
> >>     >         I would like to be able to plot this data ie to compute
> >>     a 2D
> >>     >         histogram
> >>     >         summing the masses in each bin.
> >>     >         I cannot find a way to do that without any loop on the
> >>     indices
> >>     >         resulting
> >>     >         too a very slow function.
> >>     >
> >>     >         I'm looking for an elegant way to do that with numpy (or
> >>     >         scipy??) function.
> >>     >
> >>     >         For instance, scipy.histogram2d cannot do the job because
> it
> >>     >         only counts
> >>     >         the number of samples in each bin.
> >>     >         There is no way to deal with weights.
> >>     >
> >>     >         Xavier.
> >>     >
> >>     >
> >>     >         --
> >>     >         ############################################
> >>     >         Xavier Gnata
> >>     >         CRAL - Observatoire de Lyon
> >>     >         9, avenue Charles Andr?
> >>     >         69561 Saint Genis Laval cedex
> >>     >         Phone: +33 4 78 86 85 28
> >>     >         Fax: +33 4 78 86 83 86
> >>     >         E-mail: gnata at obs.univ-lyon1.fr
> >>     <mailto:gnata at obs.univ-lyon1.fr> <mailto:gnata at obs.univ-lyon1.fr
> >>     <mailto:gnata at obs.univ-lyon1.fr>>
> >>     >         ############################################
> >>     >
> >>     >
> >>     >
> >>
> -------------------------------------------------------------------------
> >>     >         Using Tomcat but need to do more? Need to support web
> >>     >         services, security?
> >>     >         Get stuff done quickly with pre-integrated technology to
> >>     make
> >>     >         your job easier
> >>     >         Download IBM WebSphere Application Server v.1.0.1 based
> on
> >>     >         Apache Geronimo
> >>     >
> >>
> http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
> >>     <
> http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642>
> >>     >
> >>     <
> http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
> >>     <
> http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642>>
> >>     >         _______________________________________________
> >>     >         Numpy-discussion mailing list
> >>     >         Numpy-discussion at lists.sourceforge.net
> >>     <mailto:Numpy-discussion at lists.sourceforge.net>
> >>     >         <mailto:Numpy-discussion at lists.sourceforge.net
> >>     <mailto:Numpy-discussion at lists.sourceforge.net>>
> >>     >
> >>     https://lists.sourceforge.net/lists/listinfo/numpy-discussion
> >>     >
> >>     >
> >>     >
> >>     >
> >>
> >>
> -------------------------------------------------------------------------
> >>     Using Tomcat but need to do more? Need to support web services,
> >>     security?
> >>     Get stuff done quickly with pre-integrated technology to make your
> >>     job easier
> >>     Download IBM WebSphere Application Server v.1.0.1 based on Apache
> >>     Geronimo
> >>
> http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
> >>     <
> http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642>
> >>     _______________________________________________
> >>     Numpy-discussion mailing list
> >>     Numpy-discussion at lists.sourceforge.net
> >>     <mailto:Numpy-discussion at lists.sourceforge.net>
> >>     https://lists.sourceforge.net/lists/listinfo/numpy-discussion
> >>
> >>
> >>
> ------------------------------------------------------------------------
> >>
> >>
> -------------------------------------------------------------------------
> >> Using Tomcat but need to do more? Need to support web services,
> security?
> >> Get stuff done quickly with pre-integrated technology to make your job
> easier
> >> Download IBM WebSphere Application Server v.1.0.1 based on Apache
> Geronimo
> >>
> http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
> >>
> ------------------------------------------------------------------------
> >>
> >> _______________________________________________
> >> Numpy-discussion mailing list
> >> Numpy-discussion at lists.sourceforge.net
> >> https://lists.sourceforge.net/lists/listinfo/numpy-discussion
> >>
> >>
> >
> >
> >
>
>
> --
> ############################################
> Xavier Gnata
> CRAL - Observatoire de Lyon
> 9, avenue Charles Andr?
> 69561 Saint Genis Laval cedex
> Phone: +33 4 78 86 85 28
> Fax: +33 4 78 86 83 86
> E-mail: gnata at obs.univ-lyon1.fr
> ############################################
>
>
> -------------------------------------------------------------------------
> Using Tomcat but need to do more? Need to support web services, security?
> Get stuff done quickly with pre-integrated technology to make your job
> easier
> Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
> http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
> _______________________________________________
> Numpy-discussion mailing list
> Numpy-discussion at lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/numpy-discussion
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20061114/70e08098/attachment.html>
-------------- next part --------------
-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
-------------- next part --------------
_______________________________________________
Numpy-discussion mailing list
Numpy-discussion at lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/numpy-discussion

From oliphant at ee.byu.edu  Tue Nov 14 10:56:14 2006
From: oliphant at ee.byu.edu (Travis Oliphant)
Date: Tue, 14 Nov 2006 08:56:14 -0700
Subject: 32bit integers as list indices
In-Reply-To: <pan.2006.11.14.14.37.49.702134@sun.ac.za>
References: <pan.2006.11.14.14.37.49.702134@sun.ac.za>
Message-ID: <4559E71E.4070805@ee.byu.edu>

Neilen Marais wrote:

>Hi,
>
>I'm not sure if the following is expected to work on a 64bit machine:
>
>In [381]: import numpy as N
>In [382]: l = range(3)
>In [383]: i32 = N.array([0,2], N.int32)
>In [384]: i64 = N.array([0,2], N.int64)
>In [385]: l[i32[0]]
>---------------------------------------------------------------------------
>exceptions.TypeError                           Traceback (most recent call last)
>
>/home/brick/akademie/NewCode/working/<ipython console> 
>
>TypeError: list indices must be integers
>
>In [386]: l[i64[0]]
>Out[386]: 0
>
>I'd expect the 32-bit indices to work since they can be upcast to 64bit without
>loss. Am I silly for thinking this way, or is it something numpy can/should
>address? This came up while working with sparse matrices:
>
>http://projects.scipy.org/scipy/scipy/ticket/307
>  
>

It's addressed with Python 2.5  

We can't do anything about it for Python 2.4

-Travis


-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From robert.kern at gmail.com  Tue Nov 14 12:01:09 2006
From: robert.kern at gmail.com (Robert Kern)
Date: Tue, 14 Nov 2006 11:01:09 -0600
Subject: accessing FORTRAN from Python
In-Reply-To: <200611141209.23481.meesters@uni-mainz.de>
References: <200611141209.23481.meesters@uni-mainz.de>
Message-ID: <ejcsp1$atk$1@sea.gmane.org>

Christian Meesters wrote:
> Hoi,
> 
> thanks to Robert Kern who helped me out yesterday on the f2py-list, I was able 
> to make some progress in accessing FORTRAN from Python. But only some 
> progress ...
> 
> If I have the following code, named 'hello.f':
> C File hello.f
>       subroutine foo (a)
>       integer a
>       print*, "Hello from Fortran!"
>       print*, "a=",a
>       end
> 
> and compile it with g77 -shared -fPIC hello.f -o hello.so
> 
> and then start python, I get the following:
>>>> from numpy import *
>>>> from ctypes import c_int,  POINTER, byref
>>>> hellolib = ctypeslib.load_library('hello', '.')
>>>> hello = hellolib.foo_
>>>> hello(42)
>  Hello from Fortran!
> Segmentation fault
> 
> Can anybody tell me where my mistake is? (Currently python 2.4.1 (no intention 
> to update soon), the most recent ctypes, and numpy '1.0.dev3341' from svn.)
> 
> And a second question: Are there simple examples around which show how to pass 
> and retrieve lists, numpy arrays, and dicts to and from FORTRAN? Despite an 
> intensive web search I couldn't find anything.

I don't believe anyone has posted anything about calling FORTRAN code using
ctypes. Why aren't you using f2py?

-- 
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless enigma
 that is made terrible by our own mad attempt to interpret it as though it had
 an underlying truth."
  -- Umberto Eco


-------------------------------------------------------------------------
SF.net email is sponsored by: A Better Job is Waiting for You - Find it Now.
Check out Slashdot's new job board. Browse through tons of technical jobs
posted by companies looking to hire people just like you.
http://jobs.slashdot.org/



From meesters at uni-mainz.de  Tue Nov 14 12:19:41 2006
From: meesters at uni-mainz.de (Christian Meesters)
Date: Tue, 14 Nov 2006 18:19:41 +0100
Subject: accessing FORTRAN from Python
In-Reply-To: <ejcsp1$atk$1@sea.gmane.org>
References: <200611141209.23481.meesters@uni-mainz.de>
	<ejcsp1$atk$1@sea.gmane.org>
Message-ID: <200611141819.41699.meesters@uni-mainz.de>

On Tuesday 14 November 2006 18:01, Robert Kern wrote:
>
> I don't believe anyone has posted anything about calling FORTRAN code using
> ctypes. Why aren't you using f2py?
What if we just forget about my last post? (I guess I was doing way too many 
things in parallel, stumbled across the g95 site and tried to apply what's 
posted there in the way described there ...)

Sorry for this confusion and thanks, Robert, that again you brought me back on 
track, things are working now.

Christian

-------------------------------------------------------------------------
SF.net email is sponsored by: A Better Job is Waiting for You - Find it Now.
Check out Slashdot's new job board. Browse through tons of technical jobs
posted by companies looking to hire people just like you.
http://jobs.slashdot.org/



From david.huard at gmail.com  Tue Nov 14 13:50:14 2006
From: david.huard at gmail.com (David Huard)
Date: Tue, 14 Nov 2006 13:50:14 -0500
Subject: A way to compute a 2d histogram
In-Reply-To: <91cf711d0611140653q5b3730fdy2d8e57d2edbe4e58@mail.gmail.com>
References: <453E3FE5.1080201@obs.univ-lyon1.fr>
	<91cf711d0610241002u765170b5hcb9bacc850a205e0@mail.gmail.com>
	<91cf711d0610241340o3fbcc93x235849b8d63ffc16@mail.gmail.com>
	<453F57C6.2020101@obs.univ-lyon1.fr>
	<91cf711d0610250613q58056568p98dbc342728212fc@mail.gmail.com>
	<45472DC9.2090802@obs.univ-lyon1.fr>
	<4559A91D.9020701@obs.univ-lyon1.fr>
	<91cf711d0611140653q5b3730fdy2d8e57d2edbe4e58@mail.gmail.com>
Message-ID: <91cf711d0611141050ka8b18abl7a84063db241269a@mail.gmail.com>

I put the patch on Trac. Ticket 189.

2006/11/14, Xavier Gnata <gnata at obs.univ-lyon1.fr>:
>
> Hi,
> >
> > IFAICS these new histograms versions have not yet been merged to svn.
> > Are they problems to be solve before to be able to merge them?
> > How could we help?
>
>
Voice you support on Trac to replace histogram with the upgraded version.
Thanks.


David
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20061114/beae1a81/attachment.html>
-------------- next part --------------
-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your
opinions on IT & business topics through brief surveys - and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV
-------------- next part --------------
_______________________________________________
Numpy-discussion mailing list
Numpy-discussion at lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/numpy-discussion

From jdhunter at ace.bsd.uchicago.edu  Tue Nov 14 16:17:29 2006
From: jdhunter at ace.bsd.uchicago.edu (John Hunter)
Date: Tue, 14 Nov 2006 15:17:29 -0600
Subject: mysql -> record array
Message-ID: <87slglwwwm.fsf@peds-pc311.bsd.uchicago.edu>


Has anyone written any code to facilitate dumping mysql query results
(mainly arrays of floats) into numpy arrays directly at the extension
code layer.  The query results->list->array conversion can be slow.

Ideally, one could do this semi-automagically with record arrays and
table introspection....

JDH

-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your
opinions on IT & business topics through brief surveys - and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV



From jstrunk at enthought.com  Tue Nov 14 16:42:53 2006
From: jstrunk at enthought.com (Jeff Strunk)
Date: Tue, 14 Nov 2006 15:42:53 -0600
Subject: List migration
Message-ID: <200611141542.53287.jstrunk@enthought.com>

Good afternoon,

We will be performing the migration of this mailing list from Sourceforge to 
SciPy on Thursday at 2pm central.

After this time, the new mailing list address will be 
numpy-discussion at scipy.org . Any mail sent to the Sourceforge address will 
NOT be forwarded. The sender will receive a message with the new address.

Thank you for your patience.
Jeff Strunk
IT Administrator
Enthought, Inc.

-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your
opinions on IT & business topics through brief surveys - and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV



From kwgoodman at gmail.com  Tue Nov 14 16:49:02 2006
From: kwgoodman at gmail.com (Keith Goodman)
Date: Tue, 14 Nov 2006 13:49:02 -0800
Subject: List migration
In-Reply-To: <200611141542.53287.jstrunk@enthought.com>
References: <200611141542.53287.jstrunk@enthought.com>
Message-ID: <f4f93d420611141349xc64c707iaf4153d9fcfe450e@mail.gmail.com>

On 11/14/06, Jeff Strunk <jstrunk at enthought.com> wrote:
> Good afternoon,
>
> We will be performing the migration of this mailing list from Sourceforge to
> SciPy on Thursday at 2pm central.
>
> After this time, the new mailing list address will be
> numpy-discussion at scipy.org . Any mail sent to the Sourceforge address will
> NOT be forwarded. The sender will receive a message with the new address.
>
> Thank you for your patience.
> Jeff Strunk
> IT Administrator
> Enthought, Inc.
>
> -------------------------------------------------------------------------
> Take Surveys. Earn Cash. Influence the Future of IT
> Join SourceForge.net's Techsay panel and you'll get the chance to share your
> opinions on IT & business topics through brief surveys - and earn cash

So is this my last chance to Take Surveys. Earn Cash. Influence the
Future of IT?

That's great. Thank you for hosting.

-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your
opinions on IT & business topics through brief surveys - and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV



From klemm at phys.ethz.ch  Tue Nov 14 16:52:37 2006
From: klemm at phys.ethz.ch (Hanno Klemm)
Date: Tue, 14 Nov 2006 22:52:37 +0100
Subject: accessing FORTRAN from Python
In-Reply-To: <200611141209.23481.meesters@uni-mainz.de>
References: <200611141209.23481.meesters@uni-mainz.de>
Message-ID: <4f7439c24e2fb96f149bc34754af8bfe@phys.ethz.ch>

Hi Christian,

I send this off-list as there are probably a lot more knowledgeable  
people around there.

However, I don't entirely understand your problem (I'm not on the f2py  
list).

What happens if you try:

C file hello.f
       subroutine foo(a)
       integer a
Cf2py intent(in) a
       print*, "Hello from  Fortran!"
       print*, "a=",a
       end


f2py -m -c hello hello.f

That usually did the trick for me. What are your error messages, if you  
try the above?

Best regards,
Hanno


On Nov 14, 2006, at 12:09 PM, Christian Meesters wrote:

> Hoi,
>
> thanks to Robert Kern who helped me out yesterday on the f2py-list, I  
> was able
> to make some progress in accessing FORTRAN from Python. But only some
> progress ...
>
> If I have the following code, named 'hello.f':
> C File hello.f
>       subroutine foo (a)
>       integer a
>       print*, "Hello from Fortran!"
>       print*, "a=",a
>       end
>
> and compile it with g77 -shared -fPIC hello.f -o hello.so
>
> and then start python, I get the following:
>>>> from numpy import *
>>>> from ctypes import c_int,  POINTER, byref
>>>> hellolib = ctypeslib.load_library('hello', '.')
>>>> hello = hellolib.foo_
>>>> hello(42)
>  Hello from Fortran!
> Segmentation fault
>
> Can anybody tell me where my mistake is? (Currently python 2.4.1 (no  
> intention
> to update soon), the most recent ctypes, and numpy '1.0.dev3341' from  
> svn.)
>
> And a second question: Are there simple examples around which show how  
> to pass
> and retrieve lists, numpy arrays, and dicts to and from FORTRAN?  
> Despite an
> intensive web search I couldn't find anything.
>
> TIA
> Christian
>
> ----------------------------------------------------------------------- 
> --
> Using Tomcat but need to do more? Need to support web services,  
> security?
> Get stuff done quickly with pre-integrated technology to make your job  
> easier
> Download IBM WebSphere Application Server v.1.0.1 based on Apache  
> Geronimo
> http://sel.as-us.falkag.net/sel? 
> cmd=lnk&kid=120709&bid=263057&dat=121642
> _______________________________________________
> Numpy-discussion mailing list
> Numpy-discussion at lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/numpy-discussion
>
-- 
Hanno Klemm							klemm at itp.phys.ethz.ch
ETH Zurich 							tel: +41-1-6332580
Institute for theoretical physics	mobile: +41-79-4500428
									http://www.mth.kcl.ac.uk/~klemm


-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your
opinions on IT & business topics through brief surveys - and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV



From erin.sheldon at gmail.com  Tue Nov 14 17:08:52 2006
From: erin.sheldon at gmail.com (Erin Sheldon)
Date: Tue, 14 Nov 2006 17:08:52 -0500
Subject: mysql -> record array
In-Reply-To: <87slglwwwm.fsf@peds-pc311.bsd.uchicago.edu>
References: <87slglwwwm.fsf@peds-pc311.bsd.uchicago.edu>
Message-ID: <331116dc0611141408q2d7c9431s38a0f4c4b3f2db25@mail.gmail.com>

On 11/14/06, John Hunter <jdhunter at ace.bsd.uchicago.edu> wrote:
>
> Has anyone written any code to facilitate dumping mysql query results
> (mainly arrays of floats) into numpy arrays directly at the extension
> code layer.  The query results->list->array conversion can be slow.
>
> Ideally, one could do this semi-automagically with record arrays and
> table introspection....

I've been considering this as well.  I use both postgres and Oracle
in my work, and I have been using the python interfaces (cx_Oracle
and pgdb) to get result lists and convert to numpy arrays.

The question I have been asking myself is "what is the advantage
of such an approach?".  It would be faster, but by how
much?  Presumably the bottleneck for most applications will
be data retrieval rather than data copying in memory.  The
process numpy.array(results, dtype=) is pretty fast and simple
if the client is DB 2.0 compliant and returns a list of tuples
(pgdb does not sadly).  Also the memory usage will be about
the same since a copy must be made in order to create the
lists or python arrays in either case.

On the other hand, the database access modules for all major
databases, with DB 2.0 semicomplience, have already been written.
This is not an insignificant amount of work.  Writing our own
interfaces for each  of our favorite databases would require an
equivalent amount of work.

I think a set of timing tests would be useful.  I will try some
using Oracle or postgres over the next few days.  Perhaps
you could do the same with mysql.

Erin

-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your
opinions on IT & business topics through brief surveys - and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV



From jdhunter at ace.bsd.uchicago.edu  Tue Nov 14 18:02:07 2006
From: jdhunter at ace.bsd.uchicago.edu (John Hunter)
Date: Tue, 14 Nov 2006 17:02:07 -0600
Subject: mysql -> record array
In-Reply-To: <331116dc0611141408q2d7c9431s38a0f4c4b3f2db25@mail.gmail.com>
	("Erin Sheldon"'s message of "Tue, 14 Nov 2006 17:08:52 -0500")
References: <87slglwwwm.fsf@peds-pc311.bsd.uchicago.edu>
	<331116dc0611141408q2d7c9431s38a0f4c4b3f2db25@mail.gmail.com>
Message-ID: <87velh4opc.fsf@peds-pc311.bsd.uchicago.edu>

>>>>> "Erin" == Erin Sheldon <erin.sheldon at gmail.com> writes:

    Erin> The question I have been asking myself is "what is the
    Erin> advantage of such an approach?".  It would be faster, but by

In the use case that prompted this message, the pull from mysql took
almost 3 seconds, and the conversion from lists to numpy arrays took
more that 4 seconds.  We have a list of about 500000 2 tuples of
floats.

Digging in a little bit, we found that numpy is about 3x slower than
Numeric here

  peds-pc311:~> python test.py
  with dtype: 4.25 elapsed seconds
  w/o dtype 5.79 elapsed seconds
  Numeric  1.58 elapsed seconds
  24.0b2
  1.0.1.dev3432

Hmm... So maybe the question is -- is there some low hanging fruit
here to get numpy speeds up?

import time
import numpy
import numpy.random
rand = numpy.random.rand

x = [(rand(), rand()) for i in xrange(500000)]
tnow = time.time()
y = numpy.array(x, dtype=numpy.float_)
tdone = time.time()
print 'with dtype: %1.2f elapsed seconds'%(tdone - tnow)

tnow = time.time()
y = numpy.array(x)
tdone = time.time()
print 'w/o dtype %1.2f elapsed seconds'%(tdone - tnow)

import Numeric
tnow = time.time()
y = Numeric.array(x, Numeric.Float)
tdone = time.time()
print 'Numeric  %1.2f elapsed seconds'%(tdone - tnow)

print Numeric.__version__
print numpy.__version__

-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your
opinions on IT & business topics through brief surveys - and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV



From jdhunter at ace.bsd.uchicago.edu  Tue Nov 14 18:14:27 2006
From: jdhunter at ace.bsd.uchicago.edu (John Hunter)
Date: Tue, 14 Nov 2006 17:14:27 -0600
Subject: mysql -> record array
In-Reply-To: <87velh4opc.fsf@peds-pc311.bsd.uchicago.edu> (John Hunter's
	message of "Tue, 14 Nov 2006 17:02:07 -0600")
References: <87slglwwwm.fsf@peds-pc311.bsd.uchicago.edu>
	<331116dc0611141408q2d7c9431s38a0f4c4b3f2db25@mail.gmail.com>
	<87velh4opc.fsf@peds-pc311.bsd.uchicago.edu>
Message-ID: <87psbpvcx8.fsf@peds-pc311.bsd.uchicago.edu>

>>>>> "John" == John Hunter <jdhunter at ace.bsd.uchicago.edu> writes:

>>>>> "Erin" == Erin Sheldon <erin.sheldon at gmail.com> writes:
    Erin> The question I have been asking myself is "what is the
    Erin> advantage of such an approach?".  It would be faster, but by

    John> In the use case that prompted this message, the pull from
    John> mysql took almost 3 seconds, and the conversion from lists
    John> to numpy arrays took more that 4 seconds.  We have a list of
    John> about 500000 2 tuples of floats.

    John> Digging in a little bit, we found that numpy is about 3x
    John> slower than Numeric here

    John>   peds-pc311:~> python test.py with dtype: 4.25 elapsed
    John> seconds w/o dtype 5.79 elapsed seconds Numeric 1.58 elapsed
    John> seconds 24.0b2 1.0.1.dev3432

    John> Hmm... So maybe the question is -- is there some low hanging
    John> fruit here to get numpy speeds up?

And for reference, numarray is 5 times faster than Numeric here and 15
times faster than numpy

  peds-pc311:~> python test.py
  with dtype: 4.20 elapsed seconds
  w/o dtype 5.71 elapsed seconds
  Numeric  1.60 elapsed seconds
  numarray  0.30 elapsed seconds
  24.0b2
  1.0.1.dev3432
  1.5.1

import numarray
tnow = time.time()
y = numarray.array(x, numarray.Float)
tdone = time.time()
print 'numarray  %1.2f elapsed seconds'%(tdone - tnow)

print numarray.__version__


-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your
opinions on IT & business topics through brief surveys - and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV



From pgmdevlist at gmail.com  Tue Nov 14 18:33:15 2006
From: pgmdevlist at gmail.com (Pierre GM)
Date: Tue, 14 Nov 2006 18:33:15 -0500
Subject: mysql -> record array
In-Reply-To: <87psbpvcx8.fsf@peds-pc311.bsd.uchicago.edu>
References: <87slglwwwm.fsf@peds-pc311.bsd.uchicago.edu>
	<87velh4opc.fsf@peds-pc311.bsd.uchicago.edu>
	<87psbpvcx8.fsf@peds-pc311.bsd.uchicago.edu>
Message-ID: <200611141833.15877.pgmdevlist@gmail.com>

John,
I just added the following to your example:
...................................
tnow = time.time()
y = numpy.fromiter((tuple(i) for i in x), dtype=[('a',numpy.float_), ('b', 
numpy.float_)])
tdone = time.time()
print 'Numpy.fromiter %1.2f elapsed seconds'%(tdone - tnow)
...................................
Here are my results
with dtype: 4.43 elapsed seconds
w/o dtype 5.78 elapsed seconds
Numeric ?1.17 elapsed seconds
Numpy.fromiter 0.62 elapsed seconds
23.7
1.0

Numpy, one point.

-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your
opinions on IT & business topics through brief surveys - and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV



From kortmann at ideaworks.com  Tue Nov 14 18:48:47 2006
From: kortmann at ideaworks.com (Kenny Ortmann)
Date: Tue, 14 Nov 2006 17:48:47 -0600
Subject: numpy, matplotlib, scipy
References: <mailman.70135.1161983292.22397.numpy-discussion@lists.sourceforge.net>
Message-ID: <03c401c70847$6def0670$0800a8c0@yairlap>

I am sorry if this has come up before, I've found some stuff on it but just 
want to be clear.
What are the latest versions of numpy matplotlib and scipy that work 
together.
I have seen that numpy 1.0rc2 works with the latest scipy until 0.5.2 comes 
out.
now with numpy and matplotlib?  I saw that matplotlib 87.5 works with i 
think the 1.0rc2 also, but i read that mpl was going to release right after 
numpy 1.0 was released so that they would be compatible.
I am just trying to upgrade these packages before I create an executable of 
the program I am working on, and I am running into problems with
"from matplotlib._ns_nxutils import *
   ImportError: numpy.core.multiarray failed to imort"

-Kenny 


-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your
opinions on IT & business topics through brief surveys - and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV



From oliphant at ee.byu.edu  Tue Nov 14 18:50:03 2006
From: oliphant at ee.byu.edu (Travis Oliphant)
Date: Tue, 14 Nov 2006 16:50:03 -0700
Subject: mysql -> record array
In-Reply-To: <87psbpvcx8.fsf@peds-pc311.bsd.uchicago.edu>
References: <87slglwwwm.fsf@peds-pc311.bsd.uchicago.edu>	<331116dc0611141408q2d7c9431s38a0f4c4b3f2db25@mail.gmail.com>	<87velh4opc.fsf@peds-pc311.bsd.uchicago.edu>
	<87psbpvcx8.fsf@peds-pc311.bsd.uchicago.edu>
Message-ID: <455A562B.30303@ee.byu.edu>

John Hunter wrote:

>>>>>>"John" == John Hunter <jdhunter at ace.bsd.uchicago.edu> writes:
>>>>>>            
>>>>>>
>
>  
>
>>>>>>"Erin" == Erin Sheldon <erin.sheldon at gmail.com> writes:
>>>>>>            
>>>>>>
>    Erin> The question I have been asking myself is "what is the
>    Erin> advantage of such an approach?".  It would be faster, but by
>
>    John> In the use case that prompted this message, the pull from
>    John> mysql took almost 3 seconds, and the conversion from lists
>    John> to numpy arrays took more that 4 seconds.  We have a list of
>    John> about 500000 2 tuples of floats.
>
>    John> Digging in a little bit, we found that numpy is about 3x
>    John> slower than Numeric here
>
>    John>   peds-pc311:~> python test.py with dtype: 4.25 elapsed
>    John> seconds w/o dtype 5.79 elapsed seconds Numeric 1.58 elapsed
>    John> seconds 24.0b2 1.0.1.dev3432
>
>    John> Hmm... So maybe the question is -- is there some low hanging
>    John> fruit here to get numpy speeds up?
>
>And for reference, numarray is 5 times faster than Numeric here and 15
>times faster than numpy
>
>  peds-pc311:~> python test.py
>  with dtype: 4.20 elapsed seconds
>  w/o dtype 5.71 elapsed seconds
>  Numeric  1.60 elapsed seconds
>  numarray  0.30 elapsed seconds
>  24.0b2
>  1.0.1.dev3432
>  1.5.1
>
>import numarray
>tnow = time.time()
>y = numarray.array(x, numarray.Float)
>tdone = time.time()
>  
>
This sounds like it could be definitely be sped up, then.      
Assign_Array is the relevant code (it then calls PySequence_SetItem) so 
that basically

for k in a.shape[0]:
      a[k] = x[k]

is what is being done.  Thus, it might be the indexing code that is 
causing this to be a slower operation.  We should look at what numarray 
is doing --- it has provided important speed-ups in the past.

I still don't have time to look at this, but please file a ticket as we 
should fix this one.  Reference the faster numarray implementation.

-Travis



-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your
opinions on IT & business topics through brief surveys - and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV



From tim.hochberg at ieee.org  Tue Nov 14 19:44:48 2006
From: tim.hochberg at ieee.org (Tim Hochberg)
Date: Tue, 14 Nov 2006 17:44:48 -0700
Subject: mysql -> record array
In-Reply-To: <87velh4opc.fsf@peds-pc311.bsd.uchicago.edu>
References: <87slglwwwm.fsf@peds-pc311.bsd.uchicago.edu>	<331116dc0611141408q2d7c9431s38a0f4c4b3f2db25@mail.gmail.com>
	<87velh4opc.fsf@peds-pc311.bsd.uchicago.edu>
Message-ID: <455A62FF.6040706@ieee.org>

John Hunter wrote:
>>>>>> "Erin" == Erin Sheldon <erin.sheldon at gmail.com> writes:
>>>>>>             
>
>     Erin> The question I have been asking myself is "what is the
>     Erin> advantage of such an approach?".  It would be faster, but by
>
> In the use case that prompted this message, the pull from mysql took
> almost 3 seconds, and the conversion from lists to numpy arrays took
> more that 4 seconds.  We have a list of about 500000 2 tuples of
> floats.
>   
I'm no database user, but a glance at the at the docs seems to indicate 
that you can get your data via an iterator (by iterating over the cursor 
or some such db mumbo jumbo) rather than slurping up the whole list up 
at once. If so, then you'll save a lot of memory by passing the iterator 
straight to fromiter. It may even be faster, who knows.

Accessing the db via the iterator could be a performance killer, but 
it's almost certainly worth trying as it could a few megabytes of 
storage and that in turn might speed things up.

-tim

> Digging in a little bit, we found that numpy is about 3x slower than
> Numeric here
>
>   peds-pc311:~> python test.py
>   with dtype: 4.25 elapsed seconds
>   w/o dtype 5.79 elapsed seconds
>   Numeric  1.58 elapsed seconds
>   24.0b2
>   1.0.1.dev3432
>
> Hmm... So maybe the question is -- is there some low hanging fruit
> here to get numpy speeds up?
>
> import time
> import numpy
> import numpy.random
> rand = numpy.random.rand
>
> x = [(rand(), rand()) for i in xrange(500000)]
> tnow = time.time()
> y = numpy.array(x, dtype=numpy.float_)
> tdone = time.time()
> print 'with dtype: %1.2f elapsed seconds'%(tdone - tnow)
>
> tnow = time.time()
> y = numpy.array(x)
> tdone = time.time()
> print 'w/o dtype %1.2f elapsed seconds'%(tdone - tnow)
>
> import Numeric
> tnow = time.time()
> y = Numeric.array(x, Numeric.Float)
> tdone = time.time()
> print 'Numeric  %1.2f elapsed seconds'%(tdone - tnow)
>
> print Numeric.__version__
> print numpy.__version__
>
> -------------------------------------------------------------------------
> Take Surveys. Earn Cash. Influence the Future of IT
> Join SourceForge.net's Techsay panel and you'll get the chance to share your
> opinions on IT & business topics through brief surveys - and earn cash
> http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV
> _______________________________________________
> Numpy-discussion mailing list
> Numpy-discussion at lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/numpy-discussion
>
>
>   



-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your
opinions on IT & business topics through brief surveys - and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV



From tim.hochberg at ieee.org  Tue Nov 14 21:09:09 2006
From: tim.hochberg at ieee.org (Tim Hochberg)
Date: Tue, 14 Nov 2006 19:09:09 -0700
Subject: mysql -> record array
In-Reply-To: <455A62FF.6040706@ieee.org>
References: <87slglwwwm.fsf@peds-pc311.bsd.uchicago.edu>	<331116dc0611141408q2d7c9431s38a0f4c4b3f2db25@mail.gmail.com>	<87velh4opc.fsf@peds-pc311.bsd.uchicago.edu>
	<455A62FF.6040706@ieee.org>
Message-ID: <455A76C5.9050508@ieee.org>

Tim Hochberg wrote:
> John Hunter wrote:
>   
>>>>>>> "Erin" == Erin Sheldon <erin.sheldon at gmail.com> writes:
>>>>>>>             
>>>>>>>               
>>     Erin> The question I have been asking myself is "what is the
>>     Erin> advantage of such an approach?".  It would be faster, but by
>>
>> In the use case that prompted this message, the pull from mysql took
>> almost 3 seconds, and the conversion from lists to numpy arrays took
>> more that 4 seconds.  We have a list of about 500000 2 tuples of
>> floats.
>>   
>>     
> I'm no database user, but a glance at the at the docs seems to indicate 
> that you can get your data via an iterator (by iterating over the cursor 
> or some such db mumbo jumbo) rather than slurping up the whole list up 
> at once. If so, then you'll save a lot of memory by passing the iterator 
> straight to fromiter. It may even be faster, who knows.
>
> Accessing the db via the iterator could be a performance killer, but 
> it's almost certainly worth trying as it could a few megabytes of 
> storage and that in turn might speed things up.
>
>   
Assuming that I didn't mess this up too badly, it appears that using the 
iterator directly with fromiter is significantly faster than the next 
best solution (about 45%). The fromiter wrapping a list solution come in 
second, followed by numarray.array and finally way in the back, 
numpy.array. Here's the numbers:

    retrieve1 took 0.902922857514 seconds
    retrieve2 took 1.31245870634 seconds
    retrieve3 took 1.51207569677 seconds
    retrieve4 took 8.71539930354 seconds

And here is the code:

    import sqlite3, numpy as np, numarray as na, time

    N = 500000

    def setup(conn):
        c = conn.cursor()
        c.execute('''create table demo (x real, y real)''')
        data = np.random.rand(N, 2)
        c.executemany("""insert into demo values (?, ?)""", data)

    def retrieve1(conn):
        c = conn.cursor()
        c.execute('select * from demo')
        y = np.fromiter(c, dtype=[('a',float), ('b', float)])
        return y

    def retrieve2(conn):
        c = conn.cursor()
        c.execute('select * from demo')
        y = np.fromiter(c.fetchall(), dtype=[('a',float), ('b', float)])
        return y

    def retrieve3(conn):
        c = conn.cursor()
        c.execute('select * from demo')
        y = na.array(c.fetchall())
        return y

    def retrieve4(conn):
        c = conn.cursor()
        c.execute('select * from demo')
        y = np.array(c.fetchall())
        return y

    conn = sqlite3.connect(':memory:')

    setup(conn)

    t0 = time.clock()
    y1 = retrieve1(conn)
    t1 = time.clock()
    y2 = retrieve2(conn)
    t2 = time.clock()
    y3 = retrieve3(conn)
    t3 = time.clock()
    y4 = retrieve4(conn)
    t4 = time.clock()

    assert y1.shape == y2.shape == y3.shape[:1] == y4.shape[:1] == (N,)
    assert np.alltrue(y1 == y2)

    print "retrieve1 took", t1-t0, "seconds"
    print "retrieve2 took", t2-t1, "seconds"
    print "retrieve3 took", t3-t2, "seconds"
    print "retrieve4 took", t4-t3, "seconds"







-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your
opinions on IT & business topics through brief surveys - and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV



From erin.sheldon at gmail.com  Tue Nov 14 21:26:48 2006
From: erin.sheldon at gmail.com (Erin Sheldon)
Date: Tue, 14 Nov 2006 21:26:48 -0500
Subject: mysql -> record array
In-Reply-To: <455A76C5.9050508@ieee.org>
References: <87slglwwwm.fsf@peds-pc311.bsd.uchicago.edu>
	<331116dc0611141408q2d7c9431s38a0f4c4b3f2db25@mail.gmail.com>
	<87velh4opc.fsf@peds-pc311.bsd.uchicago.edu>
	<455A62FF.6040706@ieee.org> <455A76C5.9050508@ieee.org>
Message-ID: <331116dc0611141826x47ed9ccbpf5ff038c79967a44@mail.gmail.com>

On 11/14/06, Tim Hochberg <tim.hochberg at ieee.org> wrote:
> Tim Hochberg wrote:
> > John Hunter wrote:
> >
> >>>>>>> "Erin" == Erin Sheldon <erin.sheldon at gmail.com> writes:
> >>>>>>>
> >>>>>>>
> >>     Erin> The question I have been asking myself is "what is the
> >>     Erin> advantage of such an approach?".  It would be faster, but by
> >>
> >> In the use case that prompted this message, the pull from mysql took
> >> almost 3 seconds, and the conversion from lists to numpy arrays took
> >> more that 4 seconds.  We have a list of about 500000 2 tuples of
> >> floats.
> >>
> >>
> > I'm no database user, but a glance at the at the docs seems to indicate
> > that you can get your data via an iterator (by iterating over the cursor
> > or some such db mumbo jumbo) rather than slurping up the whole list up
> > at once. If so, then you'll save a lot of memory by passing the iterator
> > straight to fromiter. It may even be faster, who knows.
> >
> > Accessing the db via the iterator could be a performance killer, but
> > it's almost certainly worth trying as it could a few megabytes of
> > storage and that in turn might speed things up.
> >
> >
> Assuming that I didn't mess this up too badly, it appears that using the
> iterator directly with fromiter is significantly faster than the next
> best solution (about 45%). The fromiter wrapping a list solution come in
> second, followed by numarray.array and finally way in the back,
> numpy.array. Here's the numbers:
>
>     retrieve1 took 0.902922857514 seconds
>     retrieve2 took 1.31245870634 seconds
>     retrieve3 took 1.51207569677 seconds
>     retrieve4 took 8.71539930354 seconds
>
>

Interesting results Tim.  From Pierre's results
we saw that fromiter is the fastest way to get data
into arrays.  With your results we see there is a
difference between iterating over the cursor and
doing a fetchall() as well. Surprisingly, running
the cursor is faster.

This must come not from the data retrieval rate but
from creating the copies in memory. But just in case
I think there is one more thing to check.
I haven't used sqlite, but with other databases I have
used there is often a large variance in times from
one select to the next.  Can you
repeat these tests with a timeit().repeat  and give the
minimum?

As an aside, your database is running on a local disk, right, so
the overehead of retrieving data is minimized here?
For my tests I think I am data retrieval limited because I
get exactly the same time for the equivalent of retrieve1
and retrieve2.

Erin

-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your
opinions on IT & business topics through brief surveys - and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV



From david at ar.media.kyoto-u.ac.jp  Tue Nov 14 21:50:28 2006
From: david at ar.media.kyoto-u.ac.jp (David Cournapeau)
Date: Wed, 15 Nov 2006 11:50:28 +0900
Subject: numpy, matplotlib, scipy
In-Reply-To: <03c401c70847$6def0670$0800a8c0@yairlap>
References: <mailman.70135.1161983292.22397.numpy-discussion@lists.sourceforge.net>
	<03c401c70847$6def0670$0800a8c0@yairlap>
Message-ID: <455A8074.5010400@ar.media.kyoto-u.ac.jp>

Kenny Ortmann wrote:
> I am sorry if this has come up before, I've found some stuff on it but just 
> want to be clear.
> What are the latest versions of numpy matplotlib and scipy that work 
> together.
> I have seen that numpy 1.0rc2 works with the latest scipy until 0.5.2 comes 
> out.
> now with numpy and matplotlib?  I saw that matplotlib 87.5 works with i 
> think the 1.0rc2 also, but i read that mpl was going to release right after 
> numpy 1.0 was released so that they would be compatible.
> I am just trying to upgrade these packages before I create an executable of 
> the program I am working on, and I am running into problems with
> "from matplotlib._ns_nxutils import *
>    ImportError: numpy.core.multiarray failed to imort"
It looks like you have a problem with numpy. First, I would try import 
numpy from a python prompt, to see what's wrong. I would also check that 
numpy is set as the array package in matplolibrc file.

Second, I would check that I install everything "cleanly", that is:
    - first, remove all packages (numpy, scipy and mpl) in site-packages 
directory
    - remove the build directory in each package
    - then build from scratch + test  each package: numpy first, then 
mpl, then scipy.


Concerning versions, I regularly rebuild scipy and numpy from SVN (but 
always test them with import package; package.test(100) to test 
everything), but uses the latest release of mpl (0.87.7 as we speak), on 
linux, without any problem

cheers,

David
>
> -Kenny 
>
>
> -------------------------------------------------------------------------
> Take Surveys. Earn Cash. Influence the Future of IT
> Join SourceForge.net's Techsay panel and you'll get the chance to share your
> opinions on IT & business topics through brief surveys - and earn cash
> http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV
> _______________________________________________
> Numpy-discussion mailing list
> Numpy-discussion at lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/numpy-discussion
>
>


-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your
opinions on IT & business topics through brief surveys - and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV



From kwgoodman at gmail.com  Tue Nov 14 21:50:20 2006
From: kwgoodman at gmail.com (Keith Goodman)
Date: Tue, 14 Nov 2006 18:50:20 -0800
Subject: mysql -> record array
In-Reply-To: <331116dc0611141826x47ed9ccbpf5ff038c79967a44@mail.gmail.com>
References: <87slglwwwm.fsf@peds-pc311.bsd.uchicago.edu>
	<331116dc0611141408q2d7c9431s38a0f4c4b3f2db25@mail.gmail.com>
	<87velh4opc.fsf@peds-pc311.bsd.uchicago.edu>
	<455A62FF.6040706@ieee.org> <455A76C5.9050508@ieee.org>
	<331116dc0611141826x47ed9ccbpf5ff038c79967a44@mail.gmail.com>
Message-ID: <f4f93d420611141850v5c3fa8d9nf4f047072ee3281c@mail.gmail.com>

On 11/14/06, Erin Sheldon <erin.sheldon at gmail.com> wrote:

> As an aside, your database is running on a local disk, right, so
> the overehead of retrieving data is minimized here?
> For my tests I think I am data retrieval limited because I
> get exactly the same time for the equivalent of retrieve1
> and retrieve2.

Tim created the database in memory (conn =
sqlite3.connect(':memory:')), which is much faster than disk.

-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your
opinions on IT & business topics through brief surveys - and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV



From tim.hochberg at ieee.org  Tue Nov 14 23:14:10 2006
From: tim.hochberg at ieee.org (Tim Hochberg)
Date: Tue, 14 Nov 2006 21:14:10 -0700
Subject: mysql -> record array
In-Reply-To: <331116dc0611141826x47ed9ccbpf5ff038c79967a44@mail.gmail.com>
References: <87slglwwwm.fsf@peds-pc311.bsd.uchicago.edu>	<331116dc0611141408q2d7c9431s38a0f4c4b3f2db25@mail.gmail.com>	<87velh4opc.fsf@peds-pc311.bsd.uchicago.edu>	<455A62FF.6040706@ieee.org>
	<455A76C5.9050508@ieee.org>
	<331116dc0611141826x47ed9ccbpf5ff038c79967a44@mail.gmail.com>
Message-ID: <455A9412.5020008@ieee.org>

Erin Sheldon wrote:
> On 11/14/06, Tim Hochberg <tim.hochberg at ieee.org> wrote:
>   
>> Tim Hochberg wrote:
>>     
>>> [SNIP]
>>>
>>> I'm no database user, but a glance at the at the docs seems to indicate
>>> that you can get your data via an iterator (by iterating over the cursor
>>> or some such db mumbo jumbo) rather than slurping up the whole list up
>>> at once. If so, then you'll save a lot of memory by passing the iterator
>>> straight to fromiter. It may even be faster, who knows.
>>>
>>> Accessing the db via the iterator could be a performance killer, but
>>> it's almost certainly worth trying as it could a few megabytes of
>>> storage and that in turn might speed things up.
>>>       
>> Assuming that I didn't mess this up too badly, it appears that using the
>> iterator directly with fromiter is significantly faster than the next
>> best solution (about 45%). The fromiter wrapping a list solution come in
>> second, followed by numarray.array and finally way in the back,
>> numpy.array. Here's the numbers:
>>
>>     retrieve1 took 0.902922857514 seconds
>>     retrieve2 took 1.31245870634 seconds
>>     retrieve3 took 1.51207569677 seconds
>>     retrieve4 took 8.71539930354 seconds
>>     
>
> Interesting results Tim.  From Pierre's results
> we saw that fromiter is the fastest way to get data
> into arrays.  With your results we see there is a
> difference between iterating over the cursor and
> doing a fetchall() as well. Surprisingly, running
> the cursor is faster.
>
> This must come not from the data retrieval rate but
> from creating the copies in memory.
I imagine that is correct. In particular, skipping the making of the 
list avoids the creation of 1e6 Python floats, which is going to result 
in a lot of memory allocation.

>  But just in case
> I think there is one more thing to check.
> I haven't used sqlite, but with other databases I have
> used there is often a large variance in times from
> one select to the next.  Can you
> repeat these tests with a timeit().repeat  and give the
> minimum?
>   
Sure. Here's two sets of numbers. The first is for repeat(3,1) and the 
second for repeat (3,3).

    retrieve1 [0.91198546183942375, 0.9042411814909439, 0.90411518782415001]
    retrieve2 [0.98355349632425515, 0.95424502276127754,
    0.94714328217692412]
    retrieve3 [1.2227562441595268, 1.2195848913758596, 1.2206193803961156]
    retrieve4 [8.4344040932576547, 8.3556245276983532, 8.3568341786456131]

    retrieve1 [2.7317457945074026, 2.7274656415829384, 2.7250913174719109]
    retrieve2 [2.8857103346933783, 2.8379299603720582, 2.8386803350705136]
    retrieve3 [3.6870535221655203, 3.8980253076857565, 3.7002303365371887]
    retrieve4 [25.138646950939304, 25.06737169109482, 25.052789390830412]

The timings of these are pretty consistent with each other with the 
previous runs except that the difference between retrieve1 and retrieve2 
has disappeared. In fact, all of the runs that produce lists have gotten 
faster by about the same amount.. Odd! A little digging reveals that 
timeit turns off garbage collection to make things more repeatable. 
Turning gc back on yields the following numbers for repeat(3,1):

    retrieve1 [0.92517736192728406, 0.92109667569481601,
    0.92390960303614023]
    retrieve2 [1.3018456256311914, 1.2277141368525903, 1.2929785768861706]
    retrieve3 [1.5309831277438946, 1.4998853206203577, 1.5601200711263488]
    retrieve4 [8.6400394463542227, 8.7022300320292061, 8.6807761880350682]

So there we are, back to our original numbers. This also reveals that 
the majority of the time difference between retrieve1 and retrieve2 *is* 
memory related. However, it's the deallocation (or more precisely 
garbage collection) of all those floats that is the killer. Here's what 
the timeit routines looked like:

    if __name__ == "__main__":
        for name in ['retrieve1', 'retrieve2', 'retrieve3', 'retrieve4']:
            print name, timeit.Timer("%s(conn)" % name, "gc.enable();
    from scratch import sqlite3, %s, setup; conn =
    sqlite3.connect(':memory:'); setup(conn)" % name).repeat(3, 1)

> As an aside, your database is running on a local disk, right, so
> the overehead of retrieving data is minimized here?
> For my tests I think I am data retrieval limited because I
> get exactly the same time for the equivalent of retrieve1
> and retrieve2.
>   
As Keith pointed out, I'm keeping the database in memory (although 
there's a very good chance some of it is actually swapped to disk) so 
it's probably relatively fast. On the other hand, if you are using 
timeit to make your measurements you could be running into the (lack of) 
garbage collection issue I mention above.

-tim



-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your
opinions on IT & business topics through brief surveys - and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV



From erin.sheldon at gmail.com  Tue Nov 14 23:47:05 2006
From: erin.sheldon at gmail.com (Erin Sheldon)
Date: Tue, 14 Nov 2006 23:47:05 -0500
Subject: mysql -> record array
In-Reply-To: <455A9412.5020008@ieee.org>
References: <87slglwwwm.fsf@peds-pc311.bsd.uchicago.edu>
	<331116dc0611141408q2d7c9431s38a0f4c4b3f2db25@mail.gmail.com>
	<87velh4opc.fsf@peds-pc311.bsd.uchicago.edu>
	<455A62FF.6040706@ieee.org> <455A76C5.9050508@ieee.org>
	<331116dc0611141826x47ed9ccbpf5ff038c79967a44@mail.gmail.com>
	<455A9412.5020008@ieee.org>
Message-ID: <331116dc0611142047t2be5cdcs8c24182e5afe4407@mail.gmail.com>

On 11/14/06, Tim Hochberg <tim.hochberg at ieee.org> wrote:
SNIP
> > Interesting results Tim.  From Pierre's results
> > we saw that fromiter is the fastest way to get data
> > into arrays.  With your results we see there is a
> > difference between iterating over the cursor and
> > doing a fetchall() as well. Surprisingly, running
> > the cursor is faster.
> >
> > This must come not from the data retrieval rate but
> > from creating the copies in memory.
> I imagine that is correct. In particular, skipping the making of the
> list avoids the creation of 1e6 Python floats, which is going to result
> in a lot of memory allocation.
>
> >  But just in case
> > I think there is one more thing to check.
> > I haven't used sqlite, but with other databases I have
> > used there is often a large variance in times from
> > one select to the next.  Can you
> > repeat these tests with a timeit().repeat  and give the
> > minimum?
> >
> Sure. Here's two sets of numbers. The first is for repeat(3,1) and the
> second for repeat (3,3).
>
>     retrieve1 [0.91198546183942375, 0.9042411814909439, 0.90411518782415001]
>     retrieve2 [0.98355349632425515, 0.95424502276127754,
>     0.94714328217692412]
>     retrieve3 [1.2227562441595268, 1.2195848913758596, 1.2206193803961156]
>     retrieve4 [8.4344040932576547, 8.3556245276983532, 8.3568341786456131]
>
>     retrieve1 [2.7317457945074026, 2.7274656415829384, 2.7250913174719109]
>     retrieve2 [2.8857103346933783, 2.8379299603720582, 2.8386803350705136]
>     retrieve3 [3.6870535221655203, 3.8980253076857565, 3.7002303365371887]
>     retrieve4 [25.138646950939304, 25.06737169109482, 25.052789390830412]
>
> The timings of these are pretty consistent with each other with the
> previous runs except that the difference between retrieve1 and retrieve2
> has disappeared. In fact, all of the runs that produce lists have gotten
> faster by about the same amount.. Odd! A little digging reveals that
> timeit turns off garbage collection to make things more repeatable.
> Turning gc back on yields the following numbers for repeat(3,1):
>
>     retrieve1 [0.92517736192728406, 0.92109667569481601,
>     0.92390960303614023]
>     retrieve2 [1.3018456256311914, 1.2277141368525903, 1.2929785768861706]
>     retrieve3 [1.5309831277438946, 1.4998853206203577, 1.5601200711263488]
>     retrieve4 [8.6400394463542227, 8.7022300320292061, 8.6807761880350682]
>
> So there we are, back to our original numbers. This also reveals that
> the majority of the time difference between retrieve1 and retrieve2 *is*
> memory related. However, it's the deallocation (or more precisely
> garbage collection) of all those floats that is the killer. Here's what
> the timeit routines looked like:
>
>     if __name__ == "__main__":
>         for name in ['retrieve1', 'retrieve2', 'retrieve3', 'retrieve4']:
>             print name, timeit.Timer("%s(conn)" % name, "gc.enable();
>     from scratch import sqlite3, %s, setup; conn =
>     sqlite3.connect(':memory:'); setup(conn)" % name).repeat(3, 1)
>
> > As an aside, your database is running on a local disk, right, so
> > the overehead of retrieving data is minimized here?
> > For my tests I think I am data retrieval limited because I
> > get exactly the same time for the equivalent of retrieve1
> > and retrieve2.
> >
> As Keith pointed out, I'm keeping the database in memory (although
> there's a very good chance some of it is actually swapped to disk) so
> it's probably relatively fast. On the other hand, if you are using
> timeit to make your measurements you could be running into the (lack of)
> garbage collection issue I mention above.
>

I checked and for my real situation I am totally limited by the time
to retrieve the data.  From these tests I think this will
probably be true even if the data is  on a local disk.

I think these experiments show that iterating over the
cursor is the best approach.  It is better from a memory
point of view and is probably also the fastest.



We should still resolve the slowness for the array()
function however when converting lists of tuples.
I will file a ticket if no one else has.

Erin

-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your
opinions on IT & business topics through brief surveys - and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV



From tim.hochberg at ieee.org  Tue Nov 14 23:55:56 2006
From: tim.hochberg at ieee.org (Tim Hochberg)
Date: Tue, 14 Nov 2006 21:55:56 -0700
Subject: mysql -> record array
In-Reply-To: <455A9412.5020008@ieee.org>
References: <87slglwwwm.fsf@peds-pc311.bsd.uchicago.edu>	<331116dc0611141408q2d7c9431s38a0f4c4b3f2db25@mail.gmail.com>	<87velh4opc.fsf@peds-pc311.bsd.uchicago.edu>	<455A62FF.6040706@ieee.org>	<455A76C5.9050508@ieee.org>	<331116dc0611141826x47ed9ccbpf5ff038c79967a44@mail.gmail.com>
	<455A9412.5020008@ieee.org>
Message-ID: <455A9DDC.9040700@ieee.org>

Tim Hochberg wrote:
> [CHOP]
>
> The timings of these are pretty consistent with each other with the 
> previous runs except that the difference between retrieve1 and retrieve2 
> has disappeared. In fact, all of the runs that produce lists have gotten 
> faster by about the same amount.. Odd! A little digging reveals that 
> timeit turns off garbage collection to make things more repeatable. 
> Turning gc back on yields the following numbers for repeat(3,1):
>
>     retrieve1 [0.92517736192728406, 0.92109667569481601,
>     0.92390960303614023]
>     retrieve2 [1.3018456256311914, 1.2277141368525903, 1.2929785768861706]
>     retrieve3 [1.5309831277438946, 1.4998853206203577, 1.5601200711263488]
>     retrieve4 [8.6400394463542227, 8.7022300320292061, 8.6807761880350682]
>
> So there we are, back to our original numbers. This also reveals that 
> the majority of the time difference between retrieve1 and retrieve2 *is* 
> memory related. However, it's the deallocation (or more precisely 
> garbage collection) of all those floats that is the killer. 
I just realized that this sounds sort of misleading. In both cases a 
million floats are allocated and deallocated. However, in retrieve1 only 
two of those million are alive at any one time, so Python will just keep 
reusing the same two chunks of memory for all 500,000 pairs (ditto for 
the 500,000 tuples that are created). In the other cases, all million 
floats will be alive at once, requiring much more memory and possibly 
swapping to disk. Unsurprisingly, the second case is slower, but the 
details aren't clear. In particular why is it the deallocation that is slow?

Another mystery is why gc matters at all. None of the obvious actors are 
involved in cycles so they would normally go away due to reference 
counting even with gc turned off. My rather uninformed guess is that the 
cursor or the connection holds onto the list (caching it for later 
perhaps) and that cursor/connection is involved in some sort of cycle. 
This would keep the list alive until the gc ran.

-tim


[CHOP]




-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your
opinions on IT & business topics through brief surveys - and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV



From kwgoodman at gmail.com  Wed Nov 15 12:55:38 2006
From: kwgoodman at gmail.com (Keith Goodman)
Date: Wed, 15 Nov 2006 09:55:38 -0800
Subject: Could numpy.matlib.nanmax return a matrix?
In-Reply-To: <200611121808.51298.pgmdevlist@gmail.com>
References: <f4f93d420611121357o5dda5950g4fdcdc2f657ee654@mail.gmail.com>
	<ce557a360611121408k6bae03fds9b2e979d786bb74e@mail.gmail.com>
	<200611121808.51298.pgmdevlist@gmail.com>
Message-ID: <f4f93d420611150955u537f5bcewc9567ed041ac1d43@mail.gmail.com>

On 11/12/06, Pierre GM <pgmdevlist at gmail.com> wrote:
> On Sunday 12 November 2006 17:08, A. M. Archibald wrote:
> > On 12/11/06, Keith Goodman <kwgoodman at gmail.com> wrote:
> > > Is anybody interested in making x.max() and nanmax() behave the same
> > > for matrices, except for the NaN part? That is, make
> > > numpy.matlib.nanmax return a matrix instead of an array.
>
> Or, you could use masked arrays... In the new implementation, you can add a
> mask to a subclassed array (such as matrix) to get a regular masked array. If
> you fill this masked array, you get an array of the same subclass.
>
> >>> import numpy as N
> >>> import numpy.matlib as M
> >>> import maskedarray as MA
> >>> x=M.rand(3,3)
> >>> assert isinstance(x.max(0), M.matrix)
> >>> assert isinstance(N.max(x,0), M.matrix)
> >>> assert  isinstance(MA.max(x,0).filled(0), M.matrix)
> >>> assert isinstance(MA.max(x,0)._data, M.matrix)
>
> >>> x[-1,-1] = N.nan
> >>> tmp = MA.max(MA.array(x,mask=N.isnan(x)), 0)
> >>> assert (tmp == N.nanmax(x,0)).all()
> >>> assert isinstance(tmp.filled(0), M.matrix)

I didn't know you could use masked arrays with matrices. I guess I
took the name literally.

I think an easier way to use masked arrays would be to introduce a new
thing called mis.

I could make a regular matrix

x  = M.rand(3,3)

and assign a missing value

x[0,0] = M.mis

x would then behave as a missing array matrix.

I could also do

x[M.isnan(x)] = M.mis

or

x[mask] = M.mis

To get the mask from x:

x.mask

or

M.ismis(x)

I think that would make missing arrays accessible to everyone.

-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your
opinions on IT & business topics through brief surveys - and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV



From Korea at ophouston.com  Wed Nov 15 14:43:51 2006
From: Korea at ophouston.com (team playing)
Date: Wed, 15 Nov 2006 19:43:51 -0000
Subject: increase success Norways
Message-ID: <000c01c708ee$606467a0$00000000@helm>


Few or like Rajasthan Permitted?
Wage wages above in. Parenthood policies nb bbc means Firsttime am us Articles unsourced.
Five event serious homethe Gambia!
Related day is private! Excited of Clemence dismisses England a Play back foot.
Months per cost being of shared between State encourage.
Their current employer.
Wife gives birthiran weeksiraq Israel. Designed halt decline malefemale.
Parentsas must been a taking expires eighth Maximum.
Davydenko Highlights Features am.
Parental leave Wikipedia the am. Flat rate weeksfour South Three leavesudan a Swaziland Togo ten or.
Known in do or exist Icelandic no Esttl India Ministry?
Patriots name Test. Flat rate weeksfour South Three leavesudan a Swaziland Togo ten or.
Ashes callup Graveney discuss Pakistan doping or. Burundi Cameroon up!
Islands sri or Lanka a Syria am.
Column Pick Comment in fire Wednesdays photos am. Syria am Thailand then daysunited Emirates of Vietnam Yemen Belarus is Belgium.
Birthiran weeksiraq Israel in Japan Jordan is Korea.
Such as Australia United States in was greatly expanded or.
Week Belize Bolivia national min wage wages is?
Wage wages above in.
Kk What people sayingfans views bc.
Labor a Lawscanada indicates.
American Veteran Testaverde joins Patriots of name Test squad!
Labor a Lawscanada indicates.
Malta first after Portugal.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20061115/474001db/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: only.gif
Type: image/gif
Size: 13686 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20061115/474001db/attachment.gif>
-------------- next part --------------
-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your
opinions on IT & business topics through brief surveys - and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV
-------------- next part --------------
_______________________________________________
Numpy-discussion mailing list
Numpy-discussion at lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/numpy-discussion

From myeates at jpl.nasa.gov  Wed Nov 15 15:30:27 2006
From: myeates at jpl.nasa.gov (Mathew Yeates)
Date: Wed, 15 Nov 2006 12:30:27 -0800
Subject: reference count problems
Message-ID: <455B78E3.6020402@jpl.nasa.gov>

Hi
I'm running a 64 bit Python 2.5 on an x86 with Solaris. I have a 
function I call over 2^32 times and eventually I run out of memory.

The function is
def make_B(deltadates):
    numcols=deltadates.shape[0]
    B=numpy.zeros((numcols,numcols))
    for ind in range(0,numcols): #comment out this loop and all is good
        B[ind,0:numcols] = deltadates[0:numcols]
    return B


If I comment out the loop lines, my memory is okay. I'm guessing that a 
reference is being added to "deltadates" and that the reference count is 
going above 2^32 and reseting. Anybody have any ideas about how I can 
cure this? Is Numpy increasing the reference count here?

Mathew



-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your
opinions on IT & business topics through brief surveys - and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV



From robert.kern at gmail.com  Wed Nov 15 15:33:52 2006
From: robert.kern at gmail.com (Robert Kern)
Date: Wed, 15 Nov 2006 14:33:52 -0600
Subject: reference count problems
In-Reply-To: <455B78E3.6020402@jpl.nasa.gov>
References: <455B78E3.6020402@jpl.nasa.gov>
Message-ID: <ejftjs$9kq$1@sea.gmane.org>

Mathew Yeates wrote:
> Hi
> I'm running a 64 bit Python 2.5 on an x86 with Solaris. I have a 
> function I call over 2^32 times and eventually I run out of memory.
> 
> The function is
> def make_B(deltadates):
>     numcols=deltadates.shape[0]
>     B=numpy.zeros((numcols,numcols))
>     for ind in range(0,numcols): #comment out this loop and all is good
>         B[ind,0:numcols] = deltadates[0:numcols]
>     return B
> 
> 
> If I comment out the loop lines, my memory is okay. I'm guessing that a 
> reference is being added to "deltadates" and that the reference count is 
> going above 2^32 and reseting. Anybody have any ideas about how I can 
> cure this? Is Numpy increasing the reference count here?

Can you give us a small but complete and self-contained script that demonstrates
the problem?

-- 
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless enigma
 that is made terrible by our own mad attempt to interpret it as though it had
 an underlying truth."
  -- Umberto Eco


-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your
opinions on IT & business topics through brief surveys - and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV



From myeates at jpl.nasa.gov  Wed Nov 15 15:45:58 2006
From: myeates at jpl.nasa.gov (Mathew Yeates)
Date: Wed, 15 Nov 2006 12:45:58 -0800
Subject: reference count problems
In-Reply-To: <ejftjs$9kq$1@sea.gmane.org>
References: <455B78E3.6020402@jpl.nasa.gov> <ejftjs$9kq$1@sea.gmane.org>
Message-ID: <455B7C86.7090609@jpl.nasa.gov>

Robert Kern wrote:
> Mathew Yeates wrote:
>   
>> Hi
>> I'm running a 64 bit Python 2.5 on an x86 with Solaris. I have a 
>> function I call over 2^32 times and eventually I run out of memory.
>>
>> The function is
>> def make_B(deltadates):
>>     numcols=deltadates.shape[0]
>>     B=numpy.zeros((numcols,numcols))
>>     for ind in range(0,numcols): #comment out this loop and all is good
>>         B[ind,0:numcols] = deltadates[0:numcols]
>>     return B
>>
>>
>> If I comment out the loop lines, my memory is okay. I'm guessing that a 
>> reference is being added to "deltadates" and that the reference count is 
>> going above 2^32 and reseting. Anybody have any ideas about how I can 
>> cure this? Is Numpy increasing the reference count here?
>>     
>
> Can you give us a small but complete and self-contained script that demonstrates
> the problem?
>
>   
I'll try. But its in a complex program. BTW - I tried
B[ind,0:numcols] = deltadates[0:numcols].copy()
but that didn't work either.

Mathew


-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your
opinions on IT & business topics through brief surveys - and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV



From stefan at sun.ac.za  Wed Nov 15 15:51:49 2006
From: stefan at sun.ac.za (Stefan van der Walt)
Date: Wed, 15 Nov 2006 22:51:49 +0200
Subject: reference count problems
In-Reply-To: <ejftjs$9kq$1@sea.gmane.org>
References: <455B78E3.6020402@jpl.nasa.gov> <ejftjs$9kq$1@sea.gmane.org>
Message-ID: <20061115205149.GD1975@mentat.za.net>

On Wed, Nov 15, 2006 at 02:33:52PM -0600, Robert Kern wrote:
> Mathew Yeates wrote:
> > Hi
> > I'm running a 64 bit Python 2.5 on an x86 with Solaris. I have a 
> > function I call over 2^32 times and eventually I run out of memory.
> > 
> > The function is
> > def make_B(deltadates):
> >     numcols=deltadates.shape[0]
> >     B=numpy.zeros((numcols,numcols))
> >     for ind in range(0,numcols): #comment out this loop and all is good
> >         B[ind,0:numcols] = deltadates[0:numcols]
> >     return B
> > 
> > 
> > If I comment out the loop lines, my memory is okay. I'm guessing that a 
> > reference is being added to "deltadates" and that the reference count is 
> > going above 2^32 and reseting. Anybody have any ideas about how I can 
> > cure this? Is Numpy increasing the reference count here?
> 
> Can you give us a small but complete and self-contained script that demonstrates
> the problem?

I think this might be related to ticket #378:

http://projects.scipy.org/scipy/numpy/ticket/378

Cheers
St?fan

-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your
opinions on IT & business topics through brief surveys - and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV



From myeates at jpl.nasa.gov  Wed Nov 15 16:37:24 2006
From: myeates at jpl.nasa.gov (Mathew Yeates)
Date: Wed, 15 Nov 2006 13:37:24 -0800
Subject: program attached Re:  reference count problems
In-Reply-To: <20061115205149.GD1975@mentat.za.net>
References: <455B78E3.6020402@jpl.nasa.gov> <ejftjs$9kq$1@sea.gmane.org>
	<20061115205149.GD1975@mentat.za.net>
Message-ID: <455B8894.7000904@jpl.nasa.gov>

Stefan van der Walt wrote:
> On Wed, Nov 15, 2006 at 02:33:52PM -0600, Robert Kern wrote:
>   
>> Mathew Yeates wrote:
>>     
>>> Hi
>>> I'm running a 64 bit Python 2.5 on an x86 with Solaris. I have a 
>>> function I call over 2^32 times and eventually I run out of memory.
>>>
>>> The function is
>>> def make_B(deltadates):
>>>     numcols=deltadates.shape[0]
>>>     B=numpy.zeros((numcols,numcols))
>>>     for ind in range(0,numcols): #comment out this loop and all is good
>>>         B[ind,0:numcols] = deltadates[0:numcols]
>>>     return B
>>>
>>>
>>> If I comment out the loop lines, my memory is okay. I'm guessing that a 
>>> reference is being added to "deltadates" and that the reference count is 
>>> going above 2^32 and reseting. Anybody have any ideas about how I can 
>>> cure this? Is Numpy increasing the reference count here?
>>>       
>> Can you give us a small but complete and self-contained script that demonstrates
>> the problem?
>>     
>
> I think this might be related to ticket #378:
>
> http://projects.scipy.org/scipy/numpy/ticket/378
>
> Cheers
> St?fan
>   
okay. attached is the smallest program I could make. Before running you 
will need to create a file named biggie with 669009000 non zero floats.

Mathew
-------------- next part --------------
An embedded and charset-unspecified text was scrubbed...
Name: memsuck.py
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20061115/d59424e4/attachment.ksh>
-------------- next part --------------
-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your
opinions on IT & business topics through brief surveys - and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV
-------------- next part --------------
_______________________________________________
Numpy-discussion mailing list
Numpy-discussion at lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/numpy-discussion

From robert.kern at gmail.com  Wed Nov 15 16:41:44 2006
From: robert.kern at gmail.com (Robert Kern)
Date: Wed, 15 Nov 2006 15:41:44 -0600
Subject: program attached Re: reference count problems
In-Reply-To: <455B8894.7000904@jpl.nasa.gov>
References: <455B78E3.6020402@jpl.nasa.gov>
	<ejftjs$9kq$1@sea.gmane.org>	<20061115205149.GD1975@mentat.za.net>
	<455B8894.7000904@jpl.nasa.gov>
Message-ID: <ejg1j4$o1k$1@sea.gmane.org>

Mathew Yeates wrote:

> def delta2day1(delta):
>     return delta.days/365.0
> deltas2days=numpy.frompyfunc(delta2day1,1,1)

If I had to guess where the problem is, it's here. frompyfunc() and vectorize()
have always been tricky beasts to get right.

-- 
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless enigma
 that is made terrible by our own mad attempt to interpret it as though it had
 an underlying truth."
  -- Umberto Eco


-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your
opinions on IT & business topics through brief surveys - and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV



From tim.hochberg at ieee.org  Wed Nov 15 17:15:18 2006
From: tim.hochberg at ieee.org (Tim Hochberg)
Date: Wed, 15 Nov 2006 15:15:18 -0700
Subject: program attached Re: reference count problems
In-Reply-To: <ejg1j4$o1k$1@sea.gmane.org>
References: <455B78E3.6020402@jpl.nasa.gov>	<ejftjs$9kq$1@sea.gmane.org>	<20061115205149.GD1975@mentat.za.net>	<455B8894.7000904@jpl.nasa.gov>
	<ejg1j4$o1k$1@sea.gmane.org>
Message-ID: <455B9176.9020406@ieee.org>

Robert Kern wrote:
> Mathew Yeates wrote:
>
>   
>> def delta2day1(delta):
>>     return delta.days/365.0
>> deltas2days=numpy.frompyfunc(delta2day1,1,1)
>>     
>
> If I had to guess where the problem is, it's here. frompyfunc() and vectorize()
> have always been tricky beasts to get right.
>   
<curnudgen mode>

IMO, frompyfunc is an attractive nuisance. It doesn't magically make 
scalar Python functions fast, as people seem to assume, and it prevents 
people from figuring out how to write vectorized functions in the many 
cases where that's practicable. And it sounds like it may be buggy to boot.

In this case, I don't know that you can easily vectorize this by hand, 
but there are many ways that it could be rewritten to avoid frompyfunc. 
For example:

    def deltas2days(seq):
        return numpy.fromter((x.days for x in seq), dtype=float,
    count=len(seq))

One line shorter, about equally opaque and less likely to have 
mysterious bugs.

</curmudgen mode>

-tim


-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your
opinions on IT & business topics through brief surveys - and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV



From myeates at jpl.nasa.gov  Wed Nov 15 17:20:37 2006
From: myeates at jpl.nasa.gov (Mathew Yeates)
Date: Wed, 15 Nov 2006 14:20:37 -0800
Subject: program attached Re: reference count problems
In-Reply-To: <ejg1j4$o1k$1@sea.gmane.org>
References: <455B78E3.6020402@jpl.nasa.gov>	<ejftjs$9kq$1@sea.gmane.org>	<20061115205149.GD1975@mentat.za.net>	<455B8894.7000904@jpl.nasa.gov>
	<ejg1j4$o1k$1@sea.gmane.org>
Message-ID: <455B92B5.4050908@jpl.nasa.gov>

Robert Kern wrote:
> Mathew Yeates wrote:
>
>   
>> def delta2day1(delta):
>>     return delta.days/365.0
>> deltas2days=numpy.frompyfunc(delta2day1,1,1)
>>     
>
> If I had to guess where the problem is, it's here. frompyfunc() and vectorize()
> have always been tricky beasts to get right.
>
>   
It appears the problem is, in fact, with frompyfunc. I'm still running 
but I'm not seeing the immediate loss of memory as I was before. I 
replaced the frompyfunc with a simple loop.

Thanks for all  the help.
Mathew



-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your
opinions on IT & business topics through brief surveys - and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV



From pgmdevlist at gmail.com  Wed Nov 15 21:00:20 2006
From: pgmdevlist at gmail.com (Pierre GM)
Date: Wed, 15 Nov 2006 21:00:20 -0500
Subject: Could numpy.matlib.nanmax return a matrix?
In-Reply-To: <f4f93d420611150955u537f5bcewc9567ed041ac1d43@mail.gmail.com>
References: <f4f93d420611121357o5dda5950g4fdcdc2f657ee654@mail.gmail.com>
	<200611121808.51298.pgmdevlist@gmail.com>
	<f4f93d420611150955u537f5bcewc9567ed041ac1d43@mail.gmail.com>
Message-ID: <200611152100.20522.pgmdevlist@gmail.com>

On Wednesday 15 November 2006 12:55, Keith Goodman wrote:
> I didn't know you could use masked arrays with matrices. I guess I
> took the name literally.

:) 
Please check the developer zone:
http://projects.scipy.org/scipy/numpy/wiki/MaskedArray
for an alternative implementation of masked arrays that support subclasses of 
ndarray.

> I think an easier way to use masked arrays would be to introduce a new
> thing called mis.
>
> I could make a regular matrix
>
> x  = M.rand(3,3)
>
> and assign a missing value
>
> x[0,0] = M.mis
>
> x would then behave as a missing array matrix.
....
> I think that would make missing arrays accessible to everyone.

Well, there's already something like that, sort of: MA.masked, or 
MA.masked_singleton. The emphasis here is on "sort of".

That works well if x is already a masked array. Else, a "ValueError: setting 
an array element with a sequence" is raised. I haven't tried to find where 
the problem comes from (ndarray.__setitem__ ? The masked_singleton larger 
than it seems ?), but I wonder whether it's an issue worth solving.

If you want to get a masked_matrix from x, just type x=masked_array(x). You 
won't be able to access some specific matrix attributes (A, T), at least 
directly, but you can fill your masked_matrix and get a matrix back. And 
multiplication of two masked_matrices work as expected ! 

The main advantage of this approach is that we don't overload ndarray or 
matrices, the work is solely on the masked_array side.


-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your
opinions on IT & business topics through brief surveys - and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV



From pgmdevlist at gmail.com  Thu Nov 16 12:01:08 2006
From: pgmdevlist at gmail.com (Pierre GM)
Date: Thu, 16 Nov 2006 12:01:08 -0500
Subject: Defining custom types
In-Reply-To: <20061116164445.GB27681@crater.logilab.fr>
References: <4f7636bf0610261526t5b7b4191x370308df775799a9@mail.gmail.com>
	<20061116164445.GB27681@crater.logilab.fr>
Message-ID: <200611161201.09119.pgmdevlist@gmail.com>

On Thursday 16 November 2006 11:44, David Douard wrote:
> Hi, just to ask you: how is the work going on encapsulatinsg mx.DateTime
> as a native numpy type?
> And most important: is the code available somewhere? I am also
> interested in using DateTime objects in numpy arrays. For now, I've
> always used arrays of floats (using gmticks values of dates).

And I, as arrays of objects (well, I wrote a subclass to deal with dates, 
where each element is a datetime object, with methods to translate to floats 
or strings , but it's far from optimal...). I'd also be quite interested in 
checking what has been done.

-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your
opinions on IT & business topics through brief surveys - and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV



From cjw at sympatico.ca  Thu Nov 16 12:40:25 2006
From: cjw at sympatico.ca (Colin J. Williams)
Date: Thu, 16 Nov 2006 12:40:25 -0500
Subject: Defining custom types
In-Reply-To: <20061116164445.GB27681@crater.logilab.fr>
References: <4f7636bf0610261526t5b7b4191x370308df775799a9@mail.gmail.com>
	<20061116164445.GB27681@crater.logilab.fr>
Message-ID: <455CA289.1030101@sympatico.ca>

An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20061116/bdb2c6a2/attachment.html>
-------------- next part --------------
-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your
opinions on IT & business topics through brief surveys - and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV
-------------- next part --------------
_______________________________________________
Numpy-discussion mailing list
Numpy-discussion at lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/numpy-discussion

From tim.hochberg at ieee.org  Thu Nov 16 13:05:22 2006
From: tim.hochberg at ieee.org (Tim Hochberg)
Date: Thu, 16 Nov 2006 11:05:22 -0700
Subject: mysql -> record array
In-Reply-To: <200611161413.31917.faltet@carabos.com>
References: <87slglwwwm.fsf@peds-pc311.bsd.uchicago.edu>	<331116dc0611141408q2d7c9431s38a0f4c4b3f2db25@mail.gmail.com>
	<200611161413.31917.faltet@carabos.com>
Message-ID: <455CA862.2060305@ieee.org>

Francesc Altet wrote:
> A Dimarts 14 Novembre 2006 23:08, Erin Sheldon escrigu?:
>   
>> On 11/14/06, John Hunter <jdhunter at ace.bsd.uchicago.edu> wrote:
>>     
>>> Has anyone written any code to facilitate dumping mysql query results
>>> (mainly arrays of floats) into numpy arrays directly at the extension
>>> code layer.  The query results->list->array conversion can be slow.
>>>
>>> Ideally, one could do this semi-automagically with record arrays and
>>> table introspection....
>>>       
>> I've been considering this as well.  I use both postgres and Oracle
>> in my work, and I have been using the python interfaces (cx_Oracle
>> and pgdb) to get result lists and convert to numpy arrays.
>>
>> The question I have been asking myself is "what is the advantage
>> of such an approach?".  It would be faster, but by how
>> much?  Presumably the bottleneck for most applications will
>> be data retrieval rather than data copying in memory.
>>     
>
> Well, that largely depends on your pattern to access the data in your
> database. If you are accessing to regions of your database that have a
> high degree of spatial locality (i.e. they are located in equal or
> very similar places), the data is most probably already in memory (in
> your filesystem cache or maybe in your database cache) and the
> bottleneck will become the memory access. Of course, if you don't have
> such a spatial locality in the access pattern, then the bottleneck
> will be the disk.
>
> Just to see how DB 2.0 could benefit from adopting record arrays as
> input buffers, I've done a comparison between SQLite3 and PyTables.
> PyTables doesn't suport DB 2.0 as such, but it does use record arrays
> as buffers internally so as to read data in an efficient way (there
> should be other databases that features this, but I know PyTables best
> ;)
>
> For this, I've used a modified version of a small benchmarking program
> posted by Tim Hochberg in this same thread (it is listed at the end
> of the message). Here are the results:
>
> setup SQLite took 23.5661110878 seconds
> retrieve SQLite took 3.26717996597 seconds
> setup PyTables took 0.139157056808 seconds
> retrieve PyTables took 0.13444685936 seconds
>
> [SQLite results were obtained using an in-memory database, while
> PyTables used an on-disk one. See the code.]
>
> So, yes, if your access pattern exhibits a high degree of locality,
> you can expect a huge difference on the reading speed (more than 20x
> for this example, but as this depends on the dataset size, it can be
> even higher for larger datasets).
>   
One weakness of this benchmark is that it doesn't break out how much of 
the sqlite3 overhead is inherent to the sqlite3 engine, which I expect 
is somewhat more complicated internally than PyTables, and how much is 
due to all the extra layers we go through to get the data into an array 
(native[in database]->Python Objects->Native[In record array]). To try 
to get at least a little handle on this, I add this test:

    def querySQLite(conn):
        c = conn.cursor()
        c.execute('select * from demo where x = 0.0')
        y = np.fromiter(c, dtype=dtype)
        return y

This returns very little data (in the cases I ran it actually returned 
no data). However is still needs to loop over all the records and 
examine them. Here's what the timings looked like:

    setup SQLite took 9.71799993515 seconds
    retrieve SQLite took 0.921999931335 seconds
    query SQLite took 0.313000202179 seconds

I'm reluctant to conclude to conclude that 1/3 of the time is spent in 
traversing the database and 2/3 of the time in creating the data solely 
because databases are big voodoo to me. Still, we can probably conclude 
that traversing the data itself is pretty expensive and we would be 
unlikely to approach PyTables speed even if we didn't have the extra 
overhead. On the other hand, there's a factor of three or so improvement 
that could be realized by reducing overhead.

Or maybe not. I think that the database has to return it's data a row at 
a time, so there's intrinsically a lot of copying that's going to 
happen. So, I think it's unclear whether getting the data directly in 
native format would be significantly cheaper. I suppose that the way to 
definitively test it would be to rewrite one of these tests in C. Any 
volunteers?

I think it's probably safe to say that either way PyTables will cream 
sqllite3 in those fields where it's applicable. One of these days I 
really need to dig into PyTables. I'm sure I could use it for something.

[snip]

-tim




-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your
opinions on IT & business topics through brief surveys - and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV



From Webroot at overwet.com  Thu Nov 16 13:21:22 2006
From: Webroot at overwet.com (Pelosi)
Date: Thu, 16 Nov 2006 19:21:22 +0100
Subject: Premiere WeekThis MonthThis
Message-ID: <000b01c709ac$04b5d9c0$00000000@pctitan>


Suita in killed pairblogs ca Paper. Joinforgot password in ce Security Spyware Utilities Tools of Desktop Business am.
Shots Knowledge Base a Overview Skin Demos Column Satire of Humor.
Prizetroop of pullout uneasyuns Darfur proposal calls?
Gt famous trackwatch Rodney Mullen. Attaching about quibble.
Loves am chase going!
Os try Toolbar am message browser!
Cnncom Breaking of est cnn a Deadly? Legally mainly certain or multipacks is within cases a.
Not but also in appearance featuring little than new.
Bank Lapd cau Space re Kanye West or Diddy!
Angeles make sure or ideas in tune said must!
Enoughmost in spinning solution problem marginally background hiss attaching about. Crosswords Horoscopes Lottery Stats in. Larger heavier or iriver fairly compact. Enoughmost in spinning solution problem marginally background hiss attaching about.
Cnntogo Pipeline window a endorsed is Cnncompay. Gameinfo wikithe World of article Mean page lists.
Flying am Squirrel bed a incredible!
Luxuries Countdown Gear Clix Zune. Sports platform Rumble Stations Color or Dreams or Sachens Qboy!
Check out decor is store near cant wait Canada Show.
Heidi Hemmat reports.
Leftto hr jm or Dodd Craig Harris in fun Harristhis.
Joinforgot password in ce Security Spyware Utilities Tools of Desktop Business am. Januarytry in un expected vote Cnnmoney of?
Made available since or height a late technology employed has.
Monitoring am sites likeminded Solitaire.
Made available since or height a late technology employed has. Videostv Breakgame Soft.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20061116/aa57f7a2/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: Pakistan.gif
Type: image/gif
Size: 14085 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20061116/aa57f7a2/attachment.gif>
-------------- next part --------------
-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your
opinions on IT & business topics through brief surveys - and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV
-------------- next part --------------
_______________________________________________
Numpy-discussion mailing list
Numpy-discussion at lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/numpy-discussion

From mattknox_ca at hotmail.com  Thu Nov 16 13:37:31 2006
From: mattknox_ca at hotmail.com (Matt Knox)
Date: Thu, 16 Nov 2006 13:37:31 -0500
Subject: Defining custom types
Message-ID: <BAY124-W51DF4DFC8284EBE1AE2518CE90@phx.gbl>

> On Thursday 16 November 2006 11:44, David Douard wrote:> > Hi, just to ask you: how is the work going on encapsulatinsg mx.DateTime> > as a native numpy type?> > And most important: is the code available somewhere? I am also> > interested in using DateTime objects in numpy arrays. For now, I've> > always used arrays of floats (using gmticks values of dates).> And I, as arrays of objects (well, I wrote a subclass to deal with dates, > where each element is a datetime object, with methods to translate to floats > or strings , but it's far from optimal...). I'd also be quite interested in > checking what has been done.
 
I'm also very interested in the results of this. I need to do something very similar and am currently relying on an ugly hack to achieve the desired result.
 
- Matt Knox
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20061116/26a6b9ae/attachment.html>
-------------- next part --------------
-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your
opinions on IT & business topics through brief surveys - and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV
-------------- next part --------------
_______________________________________________
Numpy-discussion mailing list
Numpy-discussion at lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/numpy-discussion

From jontwang at gmail.com  Thu Nov 16 14:28:25 2006
From: jontwang at gmail.com (Jonathan Wang)
Date: Thu, 16 Nov 2006 13:28:25 -0600
Subject: Defining custom types
In-Reply-To: <BAY124-W51DF4DFC8284EBE1AE2518CE90@phx.gbl>
References: <BAY124-W51DF4DFC8284EBE1AE2518CE90@phx.gbl>
Message-ID: <4f7636bf0611161128je184825n7fc35bb2cfc1acbf@mail.gmail.com>

Hi all,

I've gotten to the point where Numpy recognizes the objects (represented as
doubles), but I haven't figured out how to register ufunc loops on the
custom type. It seems like Numpy should be able to check that the scalarkind
variable in the numpy type descriptor is set to float and use the float
ufuncs on the custom object. Barring that, does anyone know if the symbols
for the ufuncs are publicly accessible (and where they are) so that I can
register them with Numpy on the custom type?

As for sharing code, I've been working on this for a project at work. There
is a possibility that it will be released to the Numpy community, but that's
not clear yet.

Thanks,
Jonathan

On 11/16/06, Matt Knox <mattknox_ca at hotmail.com> wrote:
>
> > On Thursday 16 November 2006 11:44, David Douard wrote:
> > > Hi, just to ask you: how is the work going on encapsulatinsg
> mx.DateTime
> > > as a native numpy type?
> > > And most important: is the code available somewhere? I am also
> > > interested in using DateTime objects in numpy arrays. For now, I've
> > > always used arrays of floats (using gmticks values of dates).
>
> > And I, as arrays of objects (well, I wrote a subclass to deal with
> dates,
> > where each element is a datetime object, with methods to translate to
> floats
> > or strings , but it's far from optimal...). I'd also be quite interested
> in
> > checking what has been done.
>
> I'm also very interested in the results of this. I need to do something
> very similar and am currently relying on an ugly hack to achieve the desired
> result.
>
> - Matt Knox
>
> -------------------------------------------------------------------------
> Take Surveys. Earn Cash. Influence the Future of IT
> Join SourceForge.net's Techsay panel and you'll get the chance to share
> your
> opinions on IT & business topics through brief surveys - and earn cash
> http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV
>
> _______________________________________________
> Numpy-discussion mailing list
> Numpy-discussion at lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/numpy-discussion
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20061116/9208086b/attachment.html>
-------------- next part --------------
-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your
opinions on IT & business topics through brief surveys - and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV
-------------- next part --------------
_______________________________________________
Numpy-discussion mailing list
Numpy-discussion at lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/numpy-discussion

From jstrunk at enthought.com  Thu Nov 16 14:55:37 2006
From: jstrunk at enthought.com (Jeff Strunk)
Date: Thu, 16 Nov 2006 13:55:37 -0600
Subject: New list address
Message-ID: <200611161355.37519.jstrunk@enthought.com>

Good afternoon,

In a few minutes, this list will have a new address.

Please send all posts to numpy-discussion at scipy.org .

Thank you,
Jeff

-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your
opinions on IT & business topics through brief surveys - and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV



From robert.kern at gmail.com  Thu Nov 16 15:21:24 2006
From: robert.kern at gmail.com (Robert Kern)
Date: Thu, 16 Nov 2006 14:21:24 -0600
Subject: Test - Please ignore
Message-ID: <455CC844.4000208@gmail.com>

This should be rejected.

-- 
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless enigma
 that is made terrible by our own mad attempt to interpret it as though it had
 an underlying truth."
  -- Umberto Eco

-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your
opinions on IT & business topics through brief surveys - and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV



From jstrunk at enthought.com  Thu Nov 16 15:36:14 2006
From: jstrunk at enthought.com (Jeff Strunk)
Date: Thu, 16 Nov 2006 14:36:14 -0600
Subject: List migration complete
Message-ID: <200611161436.14512.jstrunk@enthought.com>

Good afternoon,

The list migration has completed successfully. numpy-discussion at scipy.org is 
the new address for this list.

Thank you,
Jeff



From iain at day-online.org.uk.invalid  Thu Nov 16 15:34:51 2006
From: iain at day-online.org.uk.invalid (Iain Day)
Date: Thu, 16 Nov 2006 20:34:51 +0000
Subject: Precision problem Mac OS X (PPC)
Message-ID: <ejii1b$ifg$2@sea.gmane.org>

Hi,

I posted this query to the scipy list (which you may have seen). I have 
been having some precision issues with some code I've got. I've searched 
the list archives and came across these tests. SciPy and numpy are 
installed from Fink, and running on a 1.5GHz G4 with Mac os x 10.4.8.

 >>> import numpy
 >>> import scipy
 >>> numpy.__version__
'1.0'
 >>> scipy.__version__
'0.5.1'
 >>> print numpy.finfo(numpy.float32).min
-3.40282346639e+38
 >>> print numpy.finfo(numpy.float32).min, 
numpy.finfo(numpy.float32).max, numpy.finfo(numpy.float32).eps
-3.40282346639e+38 3.40282346639e+38 1.19209289551e-07
 >>> print numpy.finfo(numpy.float64).min, 
numpy.finfo(numpy.float64).max, numpy.finfo(numpy.float64).eps
-1.79769313486e+308 1.79769313486e+308 2.22044604925e-16
 >>> print numpy.finfo(numpy.float128).min, 
numpy.finfo(numpy.float128).max, numpy.finfo(numpy.float128).eps
Warning: overflow encountered in add
Warning: invalid value encountered in subtract
Warning: invalid value encountered in subtract
Warning: overflow encountered in add
Warning: invalid value encountered in subtract
Warning: invalid value encountered in subtract
9223372034707292160.0 -9223372034707292160.0 1.38178697010200053818e-76

Any thoughts? I think this is a bug? If so, where do I report it?

Iain



From Rob  Thu Nov 16 16:51:16 2006
From: Rob (Rob)
Date: 09/04/2001 18:02:05
Subject: [Numpy-discussion] Python and Numpy compiled on Athlon
Message-ID: <mailman.28.1490332390.18468.numpy-discussion@python.org>

optimized gcc3.01

Just for kicks last night I installed gcc3.01 which claims to have Athlon
optimization, ie. -march=athlon.  I recompiled Python and Numpy, and then
ran a big simulation.  The new compiler ran 200 seconds slower than the
old gcc2.95 with plain -march=pentium.  I need to go to the gcc website
and see just what optimization they are claiming.  Maybe I should have
also used -02.   Rob.


-- 
The Numeric Python EM Project

http://www.members.home.net/europax





From bogus@does.not.exist.com  Thu Nov 16 16:51:16 2006
From: bogus@does.not.exist.com ()
Date: Thu, 16 Nov 2006 21:51:16 -0000
Subject: No subject
Message-ID: <mailman.30.1490332395.18468.numpy-discussion@python.org>

Dear All,

thank you for the swift responses and the help!
NumPy now works for me on OSX. Let me summarize:

1. Python 2.2.1 compiles without errors.

2. NumPy 20 does, too - special thanks to Jack!

3. Fink does not have a good UI, I dont't use it.

4. gcc is a hassle to get.

The next step ;-) is now to compile gnuplot in Darwin
which fails. I do have a running version that works
through the OSX interface but do not know how to call
this from the the terminal. If someone has any hints,
I would appreciate an e-mail message. Thanks again!

Best regards,
Thomas

__________________________________________________
Do You Yahoo!?
Yahoo! Health - Feel better, live better
http://health.yahoo.com



From falted at openlc.org  Thu Nov 16 16:51:17 2006
From: falted at openlc.org (Francesc Alted)
Date: Thu, 16 Nov 2006 21:51:17 -0000
Subject: [Numpy-discussion] Conversion functions between Numeric and
Message-ID: <mailman.31.1490332398.18468.numpy-discussion@python.org>

numarray arrays?


Hi,

Anybody is aware of any function (either in C or Python or a mixture of
both) to easily convert Numerical Python arrays from/to numarray arrays?

I mean, I would like to use such a funtion that, without having to copy
element by element all the data, be able to copy the data buffer (or even
use the same if possible at all) from one object to the other.

Thanks,






From nadavh at VisionSense.com  Thu Nov 16 16:51:17 2006
From: nadavh at VisionSense.com (Nadav Horesh)
Date: Thu, 16 Nov 2006 21:51:17 -0000
Subject: [Numpy-discussion] numarray bug !? astype with 2d array
Message-ID: <mailman.33.1490332400.18468.numpy-discussion@python.org>

givestransform ??


> I do not see any of these problems in numarray 0.8 (from CVS).
>
>   Nadav

That is good to hear !
Was this actually a known bug ?
I'm still trying to advertive the use of Numarray (instead of buying Matlab)
in our Lab. We really benefit from the support of UInt16 and single
precision floats when analysing image data. But this "implicit transpose
feature" would have made it unusable ...

Thanks for the work,
Sebastian Haase





> On Mon, 2003-10-13 at 18:11, Sebastian Haase wrote:
> > Hi all,
> > Could someone maybe confirm this or say that numarray 0.7 has fixed this
?
> > I also found more problems when type conversions are involved:  I had a
3d
> > stack of UInt16  data and wanted to compute the 2d-fft of different
> > sections -> I got identical ffts for different sections ;-(
> >
> > Please help,
> > Sebastian Haase
> >
> > ----- Original Message ----- 
> > From: "Sebastian Haase" <haase at msg.ucsf.edu>
> > To: <numpy-discussion at lists.sourceforge.net>
> > Sent: Thursday, October 09, 2003 10:25 AM
> > Subject: [Numpy-discussion] numarray bug !? astype with 2d array gives
> > transform ??
> >
> >
> > > Hi !
> > > I just discovered this: (I'm using numarray 0.6 [on windows])
> > >
> > > >>> dy = na.fromfunction(lambda y,x: y, (3,3))
> > > >>> dx = na.fromfunction(lambda y,x: x, (3,3))
> > > >>> dy
> > > array([[0, 0, 0],
> > >        [1, 1, 1],
> > >        [2, 2, 2]])
> > > >>> dx
> > > array([[0, 1, 2],
> > >        [0, 1, 2],
> > >        [0, 1, 2]])
> > > >>> dx.type()
> > > Int32
> > > >>> dx.astype(na.Int8)
> > > array([[0, 0, 0],
> > >        [1, 1, 1],
> > >        [2, 2, 2]], type=Int8)
> > > >>> dx.astype(na.Int16)
> > > array([[0, 0, 0],
> > >        [1, 1, 1],
> > >        [2, 2, 2]], type=Int16)
> > > >>> dx.astype(na.Float)
> > > array([[ 0.,  0.,  0.],
> > >        [ 1.,  1.,  1.],
> > >        [ 2.,  2.,  2.]])
> > > >>> dx.astype(na.Float32)
> > > array([[ 0.,  0.,  0.],
> > >        [ 1.,  1.,  1.],
> > >        [ 2.,  2.,  2.]], type=Float32)
> > >
> > > What does this mean ? Am I missing something ?
> > >
> > > Thanks,
> > > Sebastian Haase
> > >
> > >
> > >
> > > -------------------------------------------------------
> > > This SF.net email is sponsored by: SF.net Giveback Program.
> > > SourceForge.net hosts over 70,000 Open Source Projects.
> > > See the people who have HELPED US provide better services:
> > > Click here: http://sourceforge.net/supporters.php
> > > _______________________________________________
> > > Numpy-discussion mailing list
> > > Numpy-discussion at lists.sourceforge.net
> > > https://lists.sourceforge.net/lists/listinfo/numpy-discussion
> > >
> >
> >
> >
> > -------------------------------------------------------
> > This SF.net email is sponsored by: SF.net Giveback Program.
> > SourceForge.net hosts over 70,000 Open Source Projects.
> > See the people who have HELPED US provide better services:
> > Click here: http://sourceforge.net/supporters.php
> > _______________________________________________
> > Numpy-discussion mailing list
> > Numpy-discussion at lists.sourceforge.net
> > https://lists.sourceforge.net/lists/listinfo/numpy-discussion
>
>




From bogus@does.not.exist.com  Thu Nov 16 16:51:17 2006
From: bogus@does.not.exist.com ()
Date: Thu, 16 Nov 2006 21:51:17 -0000
Subject: No subject
Message-ID: <mailman.42.1490332409.18468.numpy-discussion@python.org>

Ray Schumacher  writes:

> At 06:31 PM 12/15/2004 +0100, Thomas Heller wrote:
>>>   buf = (c_ulong * 2000)()
>>>   numArray[pntr:pntr+2000] = map(None, buf)
>>
>>Should work, but builds a large temporary list.  Best would be to avoid
>>temporary objects at all, like this (for memmove, you need the latest
>>ctypes release) (I assume that it is possible to get the address of a
>>numarray buffer in a similar way as is possible for a normal array
>>instance):
>
> I'm not sure it will be able to work...
> The closest thing to .buffer_info() for numarray-addressing seems to
> be info(), but it returns text:
>
>  >>> src = numarray.array(range(32), Float)
>  >>> s = info(src)
> class: <class 'numarray.numarraycore.NumArray'>
> shape: (32,)
> strides: (8,)
> byteoffset: 0
> bytestride: 8
> itemsize: 8
> aligned: 1
> contiguous: 1
> data: <memory at 00815880 with size:256 held by object 00815860 aliasing object
> 00000000>
> byteorder: little
> byteswap: 0
> type: Float64
>  >>> src._byteoffset
> 0

What a pity.  (You could try to parse the <memory at 00815880> to get the
address).

Maybe you should ask to make sure that there's no way to copy between
objects implementing the buffer protocol with some Python function that
I do not know about?

If such a function doesn't exist, ctypes' memmove function should be
changed to accept objects implementing the buffer interface.

Thomas





Secret anti-spam filter-passing text. Include with reply: 
qwertyuiop 
--=====================_131324106==.ALT
Content-Type: text/html; charset="us-ascii"

<html>
<body>
I'm posting here to see if numarray has a method that could work like
array.buffer_info().&nbsp; numarray.info() returns a text output that
can't be used like the array method is to memmove() between ctypes and
Python arrays without parsing, apparently.<br><br>
ctypes thread with Thomas Heller below. His main question: &quot;Maybe
you should ask to make sure that there's no way to copy between<br>
objects implementing the buffer protocol with some Python function
that<br>
I do not know about?&quot;<br><br>
Cc: ctypes-users at lists.sourceforge.net<br>
Subject: Re: [ctypes-users] copying/slicing ctypes arrays, (c_ulong
*n)()<br>
In-Reply-To: &lt;5.2.0.4.2.20041215100117.0dff97b0 at blue-cove.com&gt;
(Ray<br>
&nbsp;Schumacher's message of &quot;Wed, 15 Dec 2004 10:57:45
-0800&quot;)<br>
References:
&lt;5.2.1.1.2.20041214202542.05f02920 at pop-server.san.rr.com&gt;<br>
<x-tab>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</x-tab>&lt;5.2.1.1.2.20041214202542.05f02920 at pop-server.san.rr.com&gt;<br>
<x-tab>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</x-tab>&lt;5.2.0.4.2.20041215100117.0dff97b0 at blue-cove.com&gt;<br>
Date: Wed, 15 Dec 2004 20:25:33 +0100<br>
Message-ID: &lt;is73z7ma.fsf at python.net&gt;<br><br>
MIME-Version: 1.0<br>
Content-Type: text/plain; charset=us-ascii<br>
X-UIDL: /O~!!CW8&quot;!g],&quot;!(Ec&quot;!<br>
Ray Schumacher&nbsp; writes:<br><br>
&gt; At 06:31 PM 12/15/2004 +0100, Thomas Heller wrote:<br>
&gt;&gt;&gt;&nbsp;&nbsp; buf = (c_ulong * 2000)()<br>
&gt;&gt;&gt;&nbsp;&nbsp; numArray[pntr:pntr+2000] = map(None, buf)<br>
&gt;&gt;<br>
&gt;&gt;Should work, but builds a large temporary list.&nbsp; Best would
be to avoid<br>
&gt;&gt;temporary objects at all, like this (for memmove, you need the
latest<br>
&gt;&gt;ctypes release) (I assume that it is possible to get the address
of a<br>
&gt;&gt;numarray buffer in a similar way as is possible for a normal
array<br>
&gt;&gt;instance):<br>
&gt;<br>
&gt; I'm not sure it will be able to work...<br>
&gt; The closest thing to .buffer_info() for numarray-addressing seems
to<br>
&gt; be info(), but it returns text:<br>
&gt;<br>
&gt;&nbsp; &gt;&gt;&gt; src = numarray.array(range(32), Float)<br>
&gt;&nbsp; &gt;&gt;&gt; s = info(src)<br>
&gt; class: &lt;class 'numarray.numarraycore.NumArray'&gt;<br>
&gt; shape: (32,)<br>
&gt; strides: (8,)<br>
&gt; byteoffset: 0<br>
&gt; bytestride: 8<br>
&gt; itemsize: 8<br>
&gt; aligned: 1<br>
&gt; contiguous: 1<br>
&gt; data: &lt;memory at 00815880 with size:256 held by object 00815860
aliasing object<br>
&gt; 00000000&gt;<br>
&gt; byteorder: little<br>
&gt; byteswap: 0<br>
&gt; type: Float64<br>
&gt;&nbsp; &gt;&gt;&gt; src._byteoffset<br>
&gt; 0<br><br>
What a pity.&nbsp; (You could try to parse the &lt;memory at 00815880&gt;
to get the<br>
address).<br><br>
Maybe you should ask to make sure that there's no way to copy
between<br>
objects implementing the buffer protocol with some Python function
that<br>
I do not know about?<br><br>
If such a function doesn't exist, ctypes' memmove function should 
be<br>
changed to accept objects implementing the buffer interface.<br><br>
Thomas<br><br>
<br><br>
<br>
<x-sigsep><p></x-sigsep>
Secret anti-spam filter-passing text. Include with reply: <br>
qwertyuiop</body>
</html>

--=====================_131324106==.ALT--




From bogus@does.not.exist.com  Thu Nov 16 16:51:17 2006
From: bogus@does.not.exist.com ()
Date: Thu, 16 Nov 2006 21:51:17 -0000
Subject: No subject
Message-ID: <mailman.44.1490332418.18468.numpy-discussion@python.org>

--j52DDRoG076339.1117719727/mail.grebennikov.ru
Content-Type: text/rfc822-headers


From bogus@does.not.exist.com  Thu Nov 16 16:51:17 2006
From: bogus@does.not.exist.com ()
Date: Thu, 16 Nov 2006 21:51:17 -0000
Subject: No subject
Message-ID: <mailman.45.1490332418.18468.numpy-discussion@python.org>

--j573DcGi094590.1118136822/mail.grebennikov.ru
Content-Type: text/rfc822-headers


From tchur at optushome.com.au  Thu Nov 16 16:51:18 2006
From: tchur at optushome.com.au (Tim Churches)
Date: Apr 12 2005 - 7:51am
Subject: No subject
Message-ID: <mailman.46.1490332423.18468.numpy-discussion@python.org>

Todd Miller wrote:
> On Sun, 2005-04-10 at 10:23 +1000, Tim Churches wrote:
>
>>I just got caught by code equivalent to this (with NumPy 23.8 on 32 bit
>>Linux):
>>
>> >>> import Numeric as N
>> >>> a = N.array((2000000000,1000000000),typecode=N.Int32)
>> >>> N.add.reduce(a)
>>-1294967296
>>
>>OK, it is an elementary mistake, but the silent overflow caught me
>>unawares. casting the array to Float64 before summing it avoids the
>>error, but in my instance the actual data is a rank-1 array of 21
>>million integers with a mean value of about 140 (which adds up more than
>>sys.maxint), and casting to Float64 will use quite a lot of memory (as
>>well as taking some time).
>>
>>Any advice for catching or avoiding such overflow without always
>>incurring a performance and memory hit by always casting to Float64?
>
>
> Here's what numarray does:
>
>
>>>>import numarray as N
>>>>a = N.array((2000000000,1000000000),typecode=N.Int32)
>>>>N.add.reduce(a)
>
> -1294967296
>
> So basic reductions in numarray have the same "careful while you're
> shaving" behavior as Numeric; it's fast but easy to screw up.

Sure, but how does one be careful? It seems that for any array of two
integers or more which could sum to more than sys.maxint or less than
-sys.maxint, add.reduce() in both NumPy and Numeric will give either a)
the correct answer or b) the incorrect answer, and short of adding up
the array using a safer but much slower method, there is no way of
determining if the answer provided (quickly) by add.reduce is right or
wrong? Which seems to make it fast but useless (for integer arrays, at
least? Is that an unfair summary? Can anyone point me towards a method
for using add.reduce() on small arrays of large integers with values in
the billions, or on large arrays of fairly small integer values, which
will not suddenly and without warning give the wrong answer?

>
> But:
>
>
>>>>a.sum()
>
> 3000000000L
>
>>>>a.sum(type='d')
>
> 3000000000.0
>
> a.sum() blockwise upcasts to the largest type of kind on the fly, in
> this case, Int64. This avoids the storage overhead of typecasting the
> entire array.

That's on a 64-bit platform, right? The same method could be used to
cast the accumulator to a Float64 on a 32-bit platform to avoid casting
the entire array?

> A better name for the method would have been sumall() since it sums all
> elements of a multi-dimensional array. The flattening process reduces
> on one dimension before flattening preventing a full copy of a
> discontiguous array. It could be smarter about choosing the dimension
> of the initial reduction.

OK, thanks. Unfortunately it is not possible for us to port our
application to numarray at the moment. But the insight is most helpful.

Tim C



From bogus@does.not.exist.com  Thu Nov 16 16:51:18 2006
From: bogus@does.not.exist.com ()
Date: Thu, 16 Nov 2006 21:51:18 -0000
Subject: No subject
Message-ID: <mailman.47.1490332438.18468.numpy-discussion@python.org>

claration for identifier "ushort".
cc: "numpy/core/include/numpy/arrayobject.h", line 140: error 1713: Illegal=
 rede
claration for identifier "uint".
cc: "numpy/core/src/arrayobject.c", line 2420: warning 562: Redeclaration o=
f "Py
Array_SetNumericOps" with a different storage class specifier: "PyArray_Set=
Numer
icOps" will have internal linkage.
cc: "numpy/core/include/numpy/arrayobject.h", line 139: error 1713: Illegal=
 rede
claration for identifier "ushort".
cc: "numpy/core/include/numpy/arrayobject.h", line 140: error 1713: Illegal=
 rede
claration for identifier "uint".
cc: "numpy/core/src/arrayobject.c", line 2420: warning 562: Redeclaration o=
f "Py
Array_SetNumericOps" with a different storage class specifier: "PyArray_Set=
Numer
icOps" will have internal linkage.
error: Command "cc -Ae -DNDEBUG -O +z -Ibuild/src/numpy/core/src -Inumpy/co=
re/in
clude -Ibuild/src/numpy/core -Inumpy/core/src -Inumpy/core/include -I/usr/l=
ocal/
include/python2.4 -c numpy/core/src/multiarraymodule.c -o build/temp.hp-ux-=
B.11.
11-9000/785-2.4/numpy/core/src/multiarraymodule.o" failed with exit status =
1

And this error message when trying to use 'gcc' instead:

error: don't know how to compile C/C++ code on platform 'posix' with 'gcc' =
comp
iler

Has this been fixed in the current SVN or should can someone suggest a fix?

 -- Paul



From prabhu at aero.iitb.ac.in  Wed Nov  1 04:53:21 2006
From: prabhu at aero.iitb.ac.in (Prabhu Ramachandran)
Date: Wed, 1 Nov 2006 15:23:21 +0530
Subject: [SciPy-user] Need more comments from
	scientific community on python-dev
In-Reply-To: <45478230.2050709@ee.byu.edu>
References: <45474841.9030802@ieee.org>
	<Mahogany-0.67.0-3016-20061031-111400.00@CASC64043305.american.edu>
	<Pine.LNX.4.64.0610310840100.18434@salmo.appl-ecosys.com>
	<db6b5ecc0610310847u331fb8daud97a6a13964c23ad@mail.gmail.com>
	<45478230.2050709@ee.byu.edu>
Message-ID: <17736.28305.154947.629130@prpc.aero.iitb.ac.in>

>>>>> "Travis" == Travis Oliphant <oliphant at ee.byu.edu> writes:

    Travis> 2) Examples of sharing memory between two objects.  PIL is
    Travis> the classic example and has some merit, but because the
    Travis> internal memory layout of the PIL is 'pointer-to-pointers'
    Travis> instead of 'big-chunk-of-memory' it's not a 1-1 match to
    Travis> NumPy and the array interface only can comunicate
    Travis> information about the "mode."  But, I can see other
    Travis> examples.  PyMedia, PyGame, PyVideo?  CVXOPT, PyVoxel.

VTK-Python and TVTK use this.  Right now these use the buffer protocol
to allow users to pass numpy data into a vtkDataArray.  It works
reasonably well.  The VTK data arrays have type information so it is
not absolutely essential for an extended buffer protocol here.  TVTK
uses this underlying functionality in VTK to present a cleaner
interface.  The idea is to have VTK use the data setup and manipulated
by numpy without an expensive copy.  The only caveat is that you need
to hang on to a reference of the numpy array if not you will end up
with a segfault.

I just thought this might be a useful data point.

FWIW, I am all for ndarrays/extended buffers in Python core.  It would
encourage a standard way to pass data in and out of wrapped code.

cheers,
prabhu


-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From faltet at carabos.com  Wed Nov  1 07:03:05 2006
From: faltet at carabos.com (Francesc Altet)
Date: Wed, 01 Nov 2006 13:03:05 +0100
Subject: Reading records from file and sorting
In-Reply-To: <1162337923.747767.323100@b28g2000cwb.googlegroups.com>
References: <1162337923.747767.323100@b28g2000cwb.googlegroups.com>
Message-ID: <1162382586.3966.44.camel@localhost.localdomain>

El dt 31 de 10 del 2006 a les 23:38 +0000, en/na George Sakkis va
escriure:
> Is there a more elegant and/or faster way to read some records from a
> file and then sort them by different fields ? What I have now is too
> specific and error-prone in general:
> 
> import numpy as N
> records = N.fromfile(a_file, dtype=N.dtype('i2,i4'))
> records_by_f0 = records.take(records.getfield('i2').argsort())
> records_by_f1 = records.take(records.getfield('i4',2).argsort())
> 
> If there's a better way, I'd like to see it; bonus points for in-place
> sorting.

Why this is too specific or error-prone? I think your solution is quite
good. If what you want is a more compact way to write the above, you can
try with:

In [56]:records=numpy.array([(1,1),(0,2)], dtype="i2,i4")
In [57]:records[records['f0'].argsort()]
Out[57]:
array([(0, 2), (1, 1)],
      dtype=[('f0', '<i2'), ('f1', '<i4')])
In [58]:records[records['f1'].argsort()]
Out[58]:
array([(1, 1), (0, 2)],
      dtype=[('f0', '<i2'), ('f1', '<i4')])

HTH,

-- 
Francesc Altet    |  Be careful about using the following code --
Carabos Coop. V.  |  I've only proven that it works, 
www.carabos.com   |  I haven't tested it. -- Donald Knuth


-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From dalcinl at gmail.com  Wed Nov  1 10:02:04 2006
From: dalcinl at gmail.com (Lisandro Dalcin)
Date: Wed, 1 Nov 2006 12:02:04 -0300
Subject: Need more comments from scientific community
	on python-dev
In-Reply-To: <e86a5fd00610311626v11c3d554u5af63401b50e030c@mail.gmail.com>
References: <45474841.9030802@ieee.org>
	<e86a5fd00610311626v11c3d554u5af63401b50e030c@mail.gmail.com>
Message-ID: <e7ba66e40611010702n253f0f8ah14927f5513ec4695@mail.gmail.com>

On 10/31/06, Bill Baxter <wbaxter at gmail.com> wrote:
> One thing I see with PIL and Matplotlib and PyOpenGL is that they're having
> to add specific support for Numpy and numeric and numarray and PIL and
> ctypes etc.

This is a real problem, and a waste of developing time, because we
lack a **general** and **standard** mechanism.

> would the proposed extensions make it
> possible for all these different packages to just have to support "buffer
> protocol" and magically be able to work with numpy or numeric without having
> to introduce explicit dependencies and support code for all those other
> packages?

I think this is the actual reason of Travis's proposal, and of all
other people like me supporting him.

> If I've got the facts straight then I'd be happy to go and say
> that on comp.lang.python.
>

Please, go on!



-- 
Lisandro Dalc?n
---------------
Centro Internacional de M?todos Computacionales en Ingenier?a (CIMEC)
Instituto de Desarrollo Tecnol?gico para la Industria Qu?mica (INTEC)
Consejo Nacional de Investigaciones Cient?ficas y T?cnicas (CONICET)
PTLC - G?emes 3450, (3000) Santa Fe, Argentina
Tel/Fax: +54-(0)342-451.1594

-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From wbaxter at gmail.com  Wed Nov  1 10:07:30 2006
From: wbaxter at gmail.com (Bill Baxter)
Date: Thu, 2 Nov 2006 00:07:30 +0900
Subject: Need more comments from scientific community
	on python-dev
In-Reply-To: <e7ba66e40611010702n253f0f8ah14927f5513ec4695@mail.gmail.com>
References: <45474841.9030802@ieee.org>
	<e86a5fd00610311626v11c3d554u5af63401b50e030c@mail.gmail.com>
	<e7ba66e40611010702n253f0f8ah14927f5513ec4695@mail.gmail.com>
Message-ID: <e86a5fd00611010707x744c71d1h2ae714eda07627d7@mail.gmail.com>

Grr, I wrote something up (it was wonderfully eloquent and convincing --
trust me!), but the gmane interface seems to have lost it (I posted two
things before responding to the two "authorization requested" emails, and
only one of the two things ever got posted it seems.)

--bb

On 11/2/06, Lisandro Dalcin <dalcinl at gmail.com> wrote:
>
> On 10/31/06, Bill Baxter <wbaxter at gmail.com> wrote:
> > One thing I see with PIL and Matplotlib and PyOpenGL is that they're
> having
> > to add specific support for Numpy and numeric and numarray and PIL and
> > ctypes etc.
>
> This is a real problem, and a waste of developing time, because we
> lack a **general** and **standard** mechanism.
>
> > would the proposed extensions make it
> > possible for all these different packages to just have to support
> "buffer
> > protocol" and magically be able to work with numpy or numeric without
> having
> > to introduce explicit dependencies and support code for all those other
> > packages?
>
> I think this is the actual reason of Travis's proposal, and of all
> other people like me supporting him.
>
> > If I've got the facts straight then I'd be happy to go and say
> > that on comp.lang.python.
> >
>
> Please, go on!
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20061102/92c59f37/attachment-0001.html>
-------------- next part --------------
-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
-------------- next part --------------
_______________________________________________
Numpy-discussion mailing list
Numpy-discussion at lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/numpy-discussion

From george.sakkis at gmail.com  Wed Nov  1 10:18:40 2006
From: george.sakkis at gmail.com (George Sakkis)
Date: Wed, 01 Nov 2006 15:18:40 -0000
Subject: Reading records from file and sorting
References: <1162337923.747767.323100@b28g2000cwb.googlegroups.com>
	<1162382586.3966.44.camel@localhost.localdomain>
Message-ID: <1162394320.550065.80510@m73g2000cwd.googlegroups.com>

Francesc Altet wrote:

> El dt 31 de 10 del 2006 a les 23:38 +0000, en/na George Sakkis va
> escriure:
> > Is there a more elegant and/or faster way to read some records from a
> > file and then sort them by different fields ? What I have now is too
> > specific and error-prone in general:
> >
> > import numpy as N
> > records = N.fromfile(a_file, dtype=N.dtype('i2,i4'))
> > records_by_f0 = records.take(records.getfield('i2').argsort())
> > records_by_f1 = records.take(records.getfield('i4',2).argsort())
> >
> > If there's a better way, I'd like to see it; bonus points for in-place
> > sorting.
>
> Why this is too specific or error-prone?

Because it
1. repeats the field types
2. requires adding up the length of all previous fields as offset.

If you're not convinced yet, try writing this in less than 3 seconds
;-):
records = N.fromfile(a_file, dtype=N.dtype('i2,i4,f4,S5,B,Q'))
records_by_f5 = ??

> I think your solution is quite good.If what you want is a more compact way to
> write the above, you can
> try with:
>
> In [56]:records=numpy.array([(1,1),(0,2)], dtype="i2,i4")
> In [57]:records[records['f0'].argsort()]
> Out[57]:
> array([(0, 2), (1, 1)],
>       dtype=[('f0', '<i2'), ('f1', '<i4')])
> In [58]:records[records['f1'].argsort()]
> Out[58]:
> array([(1, 1), (0, 2)],
>       dtype=[('f0', '<i2'), ('f1', '<i4')])

Ah, much better; I didn't know you can index a normal array (not
recarray) by label. Now, if there's a way to do the sorting in place
(records.sort('f1') doesn't work unfortunately), that would be perfect.

Thanks,
George


-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From ndarray at mac.com  Wed Nov  1 10:27:48 2006
From: ndarray at mac.com (Sasha)
Date: Wed, 1 Nov 2006 10:27:48 -0500
Subject: Need more comments from scientific community
	on python-dev
In-Reply-To: <4547D2B3.5000808@ee.byu.edu>
References: <45474841.9030802@ieee.org>
	<d38f5330610311210i5141b495s5bb73f8da6484629@mail.gmail.com>
	<4547B235.6050904@ee.byu.edu>
	<d38f5330610311336w7538e0b2v994ee277da64f6cf@mail.gmail.com>
	<4547D2B3.5000808@ee.byu.edu>
Message-ID: <d38f5330611010727x9d1b531odbc597a15df0ce20@mail.gmail.com>

On 10/31/06, Travis Oliphant <oliphant at ee.byu.edu> wrote:
> No, it's not that simple.  We have a headache whenever we want to do
> something like I just did and separate out the concepts of what makes a
> Python Object a Python object.  Now, we don't just modify a simple
> C-structure (PyArray_Descr *), we have to modify a "meta-type" or a
> altered dictionary and get that change put in to ctypes.

I think I am starting to understand. Forgive me for being slow.

Is it correct that you don't mind writing c_int * 10 instead of
dtype(('<i4', 10)) as long as the result does not have to be a
PyTypeObject at the C level? If this is the case, I would suggest to
merge ctypes syntax with your implementation.  This may help to make
the case for the python-dev crowd.  I believe very few people
understand the subtle problems in inheriting from PyTypeObject and
resist your proposal simply because they like c_int * 10  better than
dtype(('<i4', 10)).

There is no reason why ctypes should be implemented the way it is. It
is not necessary that type(c_int()) is c_int.  If a type with almost
the same properties gets into the core, ctypes developers may see an
advantage of simply inheriting from it and adding a factory __call__
method.  Meanwhile, users familiar with ctypes will not need to learn
yet another type specification syntax.

-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From dalcinl at gmail.com  Wed Nov  1 10:47:29 2006
From: dalcinl at gmail.com (Lisandro Dalcin)
Date: Wed, 1 Nov 2006 12:47:29 -0300
Subject: Need more comments from scientific community
	on python-dev
In-Reply-To: <45474841.9030802@ieee.org>
References: <45474841.9030802@ieee.org>
Message-ID: <e7ba66e40611010747u2f80608flc4e1bf6b2b19fb5a@mail.gmail.com>

On 10/31/06, Travis Oliphant <oliphant.travis at ieee.org> wrote:
>
> I'm recruiting more comments on python-dev regarding my two proposals
> for improving Python's native ability to share ndarray-like information.
>

I believe there are another really important reason to support
Travis's proposal.

Python should have a general, simple, powerful and standard way to
associate type metadata to binary streams. This enables efficient
share or memory, but also can help a lot to achieve inter-language
capabilities.

As an example, I would to make some comments about MPI, the "de facto"
standard for message-passing in parallel computing. When MPI
specification was being developed, there were many different,
incompatible libraries for message passing, and many of them (ie. PVM)
usually communicated binary data following a pack/unpack approach. The
MPI Forum decided to add an alternative approach, adding a class
'MPI_Datatype', and many predefined instances of this class (MPI_INT,
MPI_FLOAT, etc.) in order to represent the basic datatypes (integers,
floats, strings, complex numbers) in C, Fortran, and C++. Predefined
MPI datatypes can be used as building blocks for describing very
complicated memory layouts and derived,nested structures. This way,
any implementations of this standard can support interprocess
communication in a multi-language, muti-architecture approach. MPI
datatypes can also be "decoded" in order to get the basic, predefined
types contained in a user-defined datatype.

I think the MPI example, despite being taken from a very specific
domain with particular needs, has a strong connection with Travis's
proposal. Pack/unpack degrades performance, but if you want to
efficiently share binary data (with other languages/architecture),
then a general and extensible mechanism for attaching type metadata to
binary streams must me defined.

I would like to know your opinions about this. Travis's datatypes in
Python should be more that something to share data, they should be
able to **describe** data. And I am not sure if ctypes can be be the
way. Can Jython take advantage of ctypes??? Or in the near future
CPython/Jython will have a 'jtypes' module? I forget! Travis/Pearu,
why don't you develop a 'ftypes' for Fortran binary data, and next ask
Python core developers to include it in standard library (a joke, of
course)?

Travis, if you think my previous example about MPI can help, I can
resend this post to Python-Dev. I ask you this because I really want
to help and not to make noise.

Regars,


-- 
Lisandro Dalc?n
---------------
Centro Internacional de M?todos Computacionales en Ingenier?a (CIMEC)
Instituto de Desarrollo Tecnol?gico para la Industria Qu?mica (INTEC)
Consejo Nacional de Investigaciones Cient?ficas y T?cnicas (CONICET)
PTLC - G?emes 3450, (3000) Santa Fe, Argentina
Tel/Fax: +54-(0)342-451.1594

-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From oliphant.travis at ieee.org  Wed Nov  1 10:57:10 2006
From: oliphant.travis at ieee.org (Travis Oliphant)
Date: Wed, 01 Nov 2006 08:57:10 -0700
Subject: Need more comments from scientific community
 on python-dev
In-Reply-To: <d38f5330610311551u4e71cdd7ndc0b93de16dc6d80@mail.gmail.com>
References: <45474841.9030802@ieee.org>	<d38f5330610311210i5141b495s5bb73f8da6484629@mail.gmail.com>	<4547B235.6050904@ee.byu.edu>	<d38f5330610311336w7538e0b2v994ee277da64f6cf@mail.gmail.com>	<4547D2B3.5000808@ee.byu.edu>
	<d38f5330610311551u4e71cdd7ndc0b93de16dc6d80@mail.gmail.com>
Message-ID: <4548C3D6.40100@ieee.org>

Alexander Belopolsky wrote:
> On 10/31/06, Travis Oliphant <oliphant at ee.byu.edu> wrote:
> ...
>   
>> Please understand what I meant.  I meant putting effort into getting
>> PyArray_DescrConverter to allow ctypes inputs and convert to the
>> appropriate PyArray_Descr * structure.   I already understand ctypes
>> objects.  I want the dtype() command to also understand them.
>>
>>     
>
> I think I am starting to understand. Forgive me for being slow.
>   

And I think I'm beginning to understand why some people are resistant to 
what I'm saying.   I have no problem converting from ctypes types to the 
underlying PyArray_Descr structure.  I don't necessarily want that to be 
the only way to describe new types but I sure don't mind people using 
that approach.

I suspect it's the concern about "how to describe data-formats on the 
Python level" that have people concerned.  

> Is it correct that you don't mind writing c_int * 10 instead of
> dtype(('<i4', 10)) as long as the result does not have to be a
> PyTypeObject at the C level?
That is correct. 

>  If this is the case, I would suggest to
> merge ctypes syntax with your implementation.  
> This may help to make
> the case for the python-dev crowd.
Thanks for this suggestion.  You could very probably be right.
>   I believe very few people
> understand the subtle problems in inheriting from PyTypeObject and
> resist your proposal simply because they like c_int * 10  better than
> dtype(('<i4', 10)).
>
> There is no reason why ctypes should be implemented the way it is. It
> is not necessary that type(c_int()) is c_int.  If a type with almost
> the same properties gets into the core, ctypes developers may see an
> advantage of simply inheriting from it and adding a factory __call__
> method.  Meanwhile, users familiar with ctypes will not need to learn
> yet another type specification syntax.
>
>   
It would actually be "easier" for NumPy to understand ctypes syntax if 
something like a data-type object were.  As it is, the converter to a 
PyArray_Descr structure will probably have to check the tp_name of the 
ctypes.  Maybe this will help with the PEP to describe the difficulty of 
using ctypes constructs inside of other programs.

And it may be a good idea to also have a get_ctype method or some-such 
on the ctypes attribute so that  one could get a "ctypes description" 
from the NumPy data-type.

-Travis


-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From faltet at carabos.com  Wed Nov  1 11:29:31 2006
From: faltet at carabos.com (Francesc Altet)
Date: Wed, 01 Nov 2006 17:29:31 +0100
Subject: Reading records from file and sorting
In-Reply-To: <1162394320.550065.80510@m73g2000cwd.googlegroups.com>
References: <1162337923.747767.323100@b28g2000cwb.googlegroups.com>
	<1162382586.3966.44.camel@localhost.localdomain>
	<1162394320.550065.80510@m73g2000cwd.googlegroups.com>
Message-ID: <1162398571.3967.3.camel@localhost.localdomain>

El dc 01 de 11 del 2006 a les 15:18 +0000, en/na George Sakkis va
escriure:
> > Why this is too specific or error-prone?
> 
> Because it
> 1. repeats the field types
> 2. requires adding up the length of all previous fields as offset.
> 
> If you're not convinced yet, try writing this in less than 3 seconds
> ;-):
> records = N.fromfile(a_file, dtype=N.dtype('i2,i4,f4,S5,B,Q'))
> records_by_f5 = ??

Ah, I see your point :)

> > I think your solution is quite good.If what you want is a more compact way to
> > write the above, you can
> > try with:
> >
> > In [56]:records=numpy.array([(1,1),(0,2)], dtype="i2,i4")
> > In [57]:records[records['f0'].argsort()]
> > Out[57]:
> > array([(0, 2), (1, 1)],
> >       dtype=[('f0', '<i2'), ('f1', '<i4')])
> > In [58]:records[records['f1'].argsort()]
> > Out[58]:
> > array([(1, 1), (0, 2)],
> >       dtype=[('f0', '<i2'), ('f1', '<i4')])
> 
> Ah, much better; I didn't know you can index a normal array (not
> recarray) by label. Now, if there's a way to do the sorting in place
> (records.sort('f1') doesn't work unfortunately), that would be perfect.

Yes, I agree that having the possibility to do records.sort('f1') would
be a great addition (both in terms of usability but also efficiency).

Cheers,

-- 
Francesc Altet    |  Be careful about using the following code --
Carabos Coop. V.  |  I've only proven that it works, 
www.carabos.com   |  I haven't tested it. -- Donald Knuth


-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From peridot.faceted at gmail.com  Wed Nov  1 13:13:05 2006
From: peridot.faceted at gmail.com (A. M. Archibald)
Date: Wed, 1 Nov 2006 13:13:05 -0500
Subject: Need more comments from scientific community
	on python-dev
In-Reply-To: <4548C3D6.40100@ieee.org>
References: <45474841.9030802@ieee.org>
	<d38f5330610311210i5141b495s5bb73f8da6484629@mail.gmail.com>
	<4547B235.6050904@ee.byu.edu>
	<d38f5330610311336w7538e0b2v994ee277da64f6cf@mail.gmail.com>
	<4547D2B3.5000808@ee.byu.edu>
	<d38f5330610311551u4e71cdd7ndc0b93de16dc6d80@mail.gmail.com>
	<4548C3D6.40100@ieee.org>
Message-ID: <ce557a360611011013v4e883251l46dc6e2c0c85b854@mail.gmail.com>

On 01/11/06, Travis Oliphant <oliphant.travis at ieee.org> wrote:

> And it may be a good idea to also have a get_ctype method or some-such
> on the ctypes attribute so that  one could get a "ctypes description"
> from the NumPy data-type.

It seems to me that at the python level, there's not much reason to
choose the dtypes proposal over ctypes. There is one at the C level,
it seems (though I, like perhaps most of the people on python-dev,
have never actually tried using either). So perhaps, to persuade the
python-dev folks, what is needed is a comparison of what has to be
done at the C level. What would it take to rewrite numpy to use
ctypes? There seems to be some problem with extending the type objects
used by ctypes, but it's not very clear to me what that problem is
(what the extensions are supposed to do).

The argument that *some* datatypes format should become standard is much easier.

A. M. Archibald

-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From wbaxter at gmail.com  Wed Nov  1 13:19:15 2006
From: wbaxter at gmail.com (Bill Baxter)
Date: Thu, 2 Nov 2006 03:19:15 +0900
Subject: Need more comments from scientific community
	on python-dev
In-Reply-To: <ce557a360611011013v4e883251l46dc6e2c0c85b854@mail.gmail.com>
References: <45474841.9030802@ieee.org>
	<d38f5330610311210i5141b495s5bb73f8da6484629@mail.gmail.com>
	<4547B235.6050904@ee.byu.edu>
	<d38f5330610311336w7538e0b2v994ee277da64f6cf@mail.gmail.com>
	<4547D2B3.5000808@ee.byu.edu>
	<d38f5330610311551u4e71cdd7ndc0b93de16dc6d80@mail.gmail.com>
	<4548C3D6.40100@ieee.org>
	<ce557a360611011013v4e883251l46dc6e2c0c85b854@mail.gmail.com>
Message-ID: <e86a5fd00611011019p5a39553cwf55fd06e585e58e0@mail.gmail.com>

On 11/2/06, A. M. Archibald <peridot.faceted at gmail.com> wrote:
>
> On 01/11/06, Travis Oliphant <oliphant.travis at ieee.org> wrote:
>
> > And it may be a good idea to also have a get_ctype method or some-such
> > on the ctypes attribute so that  one could get a "ctypes description"
> > from the NumPy data-type.
>
> It seems to me that at the python level, there's not much reason to
> choose the dtypes proposal over ctypes. There is one at the C level,
> it seems (though I, like perhaps most of the people on python-dev,
> have never actually tried using either). So perhaps, to persuade the
> python-dev folks, what is needed is a comparison of what has to be
> done at the C level. What would it take to rewrite numpy to use
> ctypes? There seems to be some problem with extending the type objects
> used by ctypes, but it's not very clear to me what that problem is
> (what the extensions are supposed to do).



I posted a message to the thread trying to prod things in that direction.
I.e. can we see a simple concrete example of the complications involved in
using ctypes interface code, vs the presumably much nicer
numpy/data-descriptor code.  I too think that would help convince people.
Just saying it's more compilcated, trust me, doesn't help when most people
reading the list have never had to write any sort of C extension.

--bb
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20061102/ddf05221/attachment-0001.html>
-------------- next part --------------
-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
-------------- next part --------------
_______________________________________________
Numpy-discussion mailing list
Numpy-discussion at lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/numpy-discussion

From Mailer-Daemon at lists.sourceforge.net  Wed Nov  1 13:43:39 2006
From: Mailer-Daemon at lists.sourceforge.net (Mail Delivery System)
Date: Wed, 01 Nov 2006 10:43:39 -0800
Subject: Warning: message 1GfAoO-0007im-Sh delayed 10
	hours
Message-ID: <E1GfL3r-0007Au-8b@externalmx-1.sourceforge.net>

This message was created automatically by mail delivery software.
A message that you sent has not yet been delivered to one or more of its
recipients after more than 10 hours on the queue on externalmx-1.sourceforge.net.

The message identifier is:     1GfAoO-0007im-Sh
The subject of the message is: looked like me the full sized HARLEY and began to her, with this but as
The date of the message is:    Wed, 01 Nov 2006 15:46:73 +0800

The address to which the message has not yet been delivered is:

  numpy-discussion at lists.sourceforge.net
    Delay reason: SMTP error from remote mailer after RCPT TO:<numpy-discussion at lists.sourceforge.net>:
    host mail.sourceforge.net [66.35.250.206]: 451-Could not complete sender verify callout
    451-Could not complete sender verify callout for
    451-<numpy-discussion at lists.sourceforge.net>.
    451-The mail server(s) for the domain may be temporarily unreachable, or
    451-they may be permanently unreachable from this server. In the latter case,
    451-you need to change the address or create an MX record for its domain
    451-if it is 

No action is required on your part. Delivery attempts will continue for
some time, and this warning may be repeated at intervals if the message
remains undelivered. Eventually the mail delivery software will give up,
and when that happens, the message will be returned to you.

-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From kwgoodman at gmail.com  Wed Nov  1 15:33:05 2006
From: kwgoodman at gmail.com (Keith Goodman)
Date: Wed, 1 Nov 2006 12:33:05 -0800
Subject: Converting bool to float
Message-ID: <f4f93d420611011233i60c8128ew63288d5620477c9c@mail.gmail.com>

I had a hard time tracing a bug in my code. The culprit was this difference:

>> x

matrix([[True],
        [True],
        [True]], dtype=bool)
>> 1.0 - x

matrix([[ 0.],
        [ 0.],
        [ 0.]], dtype=float32)     <------- float32
>> 1.0*x

matrix([[ 1.],
        [ 1.],
        [ 1.]])                            <-------- float64

>> numpy.__version__
'1.0rc1'

Any chance that 1.0 - x could return dtype = float64?

-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From oliphant at ee.byu.edu  Wed Nov  1 16:43:38 2006
From: oliphant at ee.byu.edu (Travis Oliphant)
Date: Wed, 01 Nov 2006 14:43:38 -0700
Subject: Converting bool to float
In-Reply-To: <f4f93d420611011233i60c8128ew63288d5620477c9c@mail.gmail.com>
References: <f4f93d420611011233i60c8128ew63288d5620477c9c@mail.gmail.com>
Message-ID: <4549150A.4010404@ee.byu.edu>

Keith Goodman wrote:

>I had a hard time tracing a bug in my code. The culprit was this difference:
>
>  
>
>>>x
>>>      
>>>
>
>matrix([[True],
>        [True],
>        [True]], dtype=bool)
>  
>
>>>1.0 - x
>>>      
>>>
>
>matrix([[ 0.],
>        [ 0.],
>        [ 0.]], dtype=float32)     <------- float32
>  
>
>>>1.0*x
>>>      
>>>
>
>matrix([[ 1.],
>        [ 1.],
>        [ 1.]])                            <-------- float64
>
>  
>
>>>numpy.__version__
>>>      
>>>
>'1.0rc1'
>
>Any chance that 1.0 - x could return dtype = float64?
>  
>
I'm surprised it doesn't.   Both should follow bascially the same 
code-path.  Perhaps there is a missing function loop or something. 


I'll look in to it.

-Travis


-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From oliphant at ee.byu.edu  Wed Nov  1 16:51:40 2006
From: oliphant at ee.byu.edu (Travis Oliphant)
Date: Wed, 01 Nov 2006 14:51:40 -0700
Subject: Converting bool to float
In-Reply-To: <f4f93d420611011233i60c8128ew63288d5620477c9c@mail.gmail.com>
References: <f4f93d420611011233i60c8128ew63288d5620477c9c@mail.gmail.com>
Message-ID: <454916EC.6010003@ee.byu.edu>

Keith Goodman wrote:

>I had a hard time tracing a bug in my code. The culprit was this difference:
>
>  
>
>>>x
>>>      
>>>
>
>matrix([[True],
>        [True],
>        [True]], dtype=bool)
>  
>
>>>1.0 - x
>>>      
>>>
>
>matrix([[ 0.],
>        [ 0.],
>        [ 0.]], dtype=float32)     <------- float32
>  
>
>>>1.0*x
>>>      
>>>
>
>matrix([[ 1.],
>        [ 1.],
>        [ 1.]])                            <-------- float64
>
>  
>
>>>numpy.__version__
>>>      
>>>
>'1.0rc1'
>
>Any chance that 1.0 - x could return dtype = float64?
>  
>

It looks like 1.0-x is doing the right thing.

The problem is 1.0*x for matrices is going to float64.  For arrays it 
returns float32 just like the 1.0-x

This can't be changed at this point until 1.1

We will fix the bug in 1.0*x producing float64, however.  I'm still not 
sure what's causing it, though.

-Travis


-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From robert.kern at gmail.com  Wed Nov  1 16:57:19 2006
From: robert.kern at gmail.com (Robert Kern)
Date: Wed, 01 Nov 2006 13:57:19 -0800
Subject: Converting bool to float
In-Reply-To: <454916EC.6010003@ee.byu.edu>
References: <f4f93d420611011233i60c8128ew63288d5620477c9c@mail.gmail.com>
	<454916EC.6010003@ee.byu.edu>
Message-ID: <eib580$qdt$1@sea.gmane.org>

Travis Oliphant wrote:

> It looks like 1.0-x is doing the right thing.
> 
> The problem is 1.0*x for matrices is going to float64.  For arrays it 
> returns float32 just like the 1.0-x

Why is this the right thing? Python floats are float64.

-- 
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless enigma
  that is made terrible by our own mad attempt to interpret it as though it had
  an underlying truth."
   -- Umberto Eco


-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From hetland at tamu.edu  Wed Nov  1 18:14:57 2006
From: hetland at tamu.edu (Rob Hetland)
Date: Wed, 1 Nov 2006 17:14:57 -0600
Subject: pyaudio 0.3, with docs !
In-Reply-To: <4547519B.4040201@ar.media.kyoto-u.ac.jp>
References: <4547519B.4040201@ar.media.kyoto-u.ac.jp>
Message-ID: <92DCBC0B-10B3-469D-A45D-172C02B3F9D2@tamu.edu>


One small change -- on my intel mac, I need to link to

/usr/local/lib/libsndfile.1.dylib

instead of the .so file.

Then everything works!!!  This is truely a great thing -- my two  
loves, numpy and audio files, together forever...

-r

On Oct 31, 2006, at 7:37 AM, David Cournapeau wrote:

> Hi,
>
>     I improved pyaudio last WE using indications given by various  
> people
> on the list or privately, and as I finally got the motivation to  
> set-up
> something which looks like a webpage, there is a doc with examples  
> which
> show how to use it. The API to open files for writing is much  
> saner, and
> the setup.py should be smart enough to grab all informations necessary
> to the wrapper, including the location of the shared libsndfile:
>
>     download:
> http://www.ar.media.kyoto-u.ac.jp/members/david/softwares/pyaudio/ 
> #installation
>     doc + examples:
> http://www.ar.media.kyoto-u.ac.jp/members/david/softwares/pyaudio/
>
>     I would appreciate to hear reports on platforms which are not  
> linux
> (windows, mac os X) to see if my the setup.py works there,
>
>     Cheers,
>
>     David
> _______________________________________________
> SciPy-user mailing list
> SciPy-user at scipy.org
> http://projects.scipy.org/mailman/listinfo/scipy-user

----
Rob Hetland, Associate Professor
Dept. of Oceanography, Texas A&M University
http://pong.tamu.edu/~rob
phone: 979-458-0096, fax: 979-845-6331



From kwgoodman at gmail.com  Wed Nov  1 20:50:15 2006
From: kwgoodman at gmail.com (Keith Goodman)
Date: Wed, 1 Nov 2006 17:50:15 -0800
Subject: Converting bool to float
In-Reply-To: <454916EC.6010003@ee.byu.edu>
References: <f4f93d420611011233i60c8128ew63288d5620477c9c@mail.gmail.com>
	<454916EC.6010003@ee.byu.edu>
Message-ID: <f4f93d420611011750j78f8fa26h9d048821e7dd0f05@mail.gmail.com>

On 11/1/06, Travis Oliphant <oliphant at ee.byu.edu> wrote:
> It looks like 1.0-x is doing the right thing.
>
> The problem is 1.0*x for matrices is going to float64.  For arrays it
> returns float32 just like the 1.0-x
>
> This can't be changed at this point until 1.1
>
> We will fix the bug in 1.0*x producing float64, however.  I'm still not
> sure what's causing it, though.

I think it would be great if float64 was the default in numpy. That
way most people wouldn't have to worry about dtypes when crunching
numbers. And then numpy could apply for a trademark on 'it just
works'.

Having to worry about dtypes makes users (me) nervous.

I imagine a change like this would not be an overnight change, more of
a long-term goal.

This one, from a previous thread, also makes me nervous:

>> sum(M.ones((300,1)) == 1)
matrix([[44]], dtype=int8)

But float64 might not make sense here.

-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From wbaxter at gmail.com  Wed Nov  1 20:44:38 2006
From: wbaxter at gmail.com (Bill Baxter)
Date: Thu, 2 Nov 2006 10:44:38 +0900
Subject: Compressed index sets, generators and iterators
Message-ID: <e86a5fd00611011744sbae3b89oada17ea4718837b2@mail.gmail.com>

Has any thought been given to using compressed or functional representations
of index sets?

For instance there could be a where-like function that returns an object
that can generate a set of indexes on the fly, rather than explicitly
allocating arrays and enumerating all of the indices.

What's the reason iterators are not supported currently?
For instance  A[range(0,4)] works for a 1d A, but A[xrange(0,4)] does not.
Are iterators just too inefficient to bother with?

I could imagine an iterator that generates a set of N-tuples for an N-d
array being a legal indexing construct.  Even something like
A[idx_iterator] = value_iterator  would seem to make sense.

--bb
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20061102/9873c122/attachment-0001.html>
-------------- next part --------------
-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
-------------- next part --------------
_______________________________________________
Numpy-discussion mailing list
Numpy-discussion at lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/numpy-discussion

From nqhbtgkjppg at shawcable.net  Wed Nov  1 18:37:58 2006
From: nqhbtgkjppg at shawcable.net (Nov.)
Date: Wed, 1 Nov 2006 15:37:58 -0800
Subject: NowPlaying Topic
Message-ID: <000b01c6fe0e$c2f289f0$ed6f5218@kathy>


Web Tools or Html easiest Html guide in beginners. Cashfall Previewtop am ffb Bluegrass Dates?
Brought in discovery Monsters Waiting Echo ca slipped.
Business apply throughout! Crash Fuel Saving a Minivans of.
Business apply throughout!
Zo Poule Place net Stel am. Rate Sign Multimedia or maken heeft.
Waiting Echo ca slipped apropos showcase. Jackson School Karl Mamola State Univeristy in?
Waiting Echo ca slipped apropos showcase? Etc shareware is Burner Webedit Webber or friendly flexible in?
Adoredinto a Pianos night Annalee of Ferys breathy. Golive new name yes bought software. Did my views.
Roses Wanna am Adoredinto Pianos night! Zijn utc or sinds am.
People editor a included Splash easy. Ronde hulpstuk nba Gisteren Overige a Alles zonder.
Web Tools or Html easiest Html guide in beginners. Cntry Grammys Winners a.
Online is gebruikers in hebben totaal am. Zin rondje kerk Basso mag weer wedstri am literatuur am.
Cars Crash Fuel Saving!
Than sites or intend beginner supports image creations files or offers. Schepper in Sporten Live Blijf hoogte van Deze keer bezocht of. Lot longer separate in tools insert!
Claris Home focus mainly single pages.
One additional received a November accepted of until filled Dickison Physical.
History Articles Picks. Cashfall Previewtop am ffb Bluegrass Dates?
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20061101/e377463e/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: Pareles.gif
Type: image/gif
Size: 9519 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20061101/e377463e/attachment-0001.gif>
-------------- next part --------------
-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
-------------- next part --------------
_______________________________________________
Numpy-discussion mailing list
Numpy-discussion at lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/numpy-discussion

From fullung at gmail.com  Wed Nov  1 17:46:20 2006
From: fullung at gmail.com (Albert Strasheim)
Date: Thu, 2 Nov 2006 00:46:20 +0200
Subject: Reading records from file and sorting
In-Reply-To: <1162337923.747767.323100@b28g2000cwb.googlegroups.com>
References: <1162337923.747767.323100@b28g2000cwb.googlegroups.com>
Message-ID: <20061101224620.GA29602@dogbert.sdsl.sun.ac.za>

Hey George

On Tue, 31 Oct 2006, George Sakkis wrote:

> Is there a more elegant and/or faster way to read some records from a
> file and then sort them by different fields ? What I have now is too
> specific and error-prone in general:
> 
> import numpy as N
> records = N.fromfile(a_file, dtype=N.dtype('i2,i4'))
> records_by_f0 = records.take(records.getfield('i2').argsort())
> records_by_f1 = records.take(records.getfield('i4',2).argsort())
> 
> If there's a better way, I'd like to see it; bonus points for in-place
> sorting.

Check the thread "Strange results when sorting array with fields" from 
about a week back. Travis made some changes to sorting in the presence 
of fields that should solve your problem, assuming your fields appear in 
the order you want to sort (i.e. you want to sort f1, f2, f3 and not 
something like f1, f3, f2).

Cheers,

Albert

-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From oliphant at ee.byu.edu  Wed Nov  1 17:44:09 2006
From: oliphant at ee.byu.edu (Travis Oliphant)
Date: Wed, 01 Nov 2006 15:44:09 -0700
Subject: Converting bool to float
In-Reply-To: <eib580$qdt$1@sea.gmane.org>
References: <f4f93d420611011233i60c8128ew63288d5620477c9c@mail.gmail.com>	<454916EC.6010003@ee.byu.edu>
	<eib580$qdt$1@sea.gmane.org>
Message-ID: <45492339.6050302@ee.byu.edu>

Robert Kern wrote:

>Travis Oliphant wrote:
>
>  
>
>>It looks like 1.0-x is doing the right thing.
>>
>>The problem is 1.0*x for matrices is going to float64.  For arrays it 
>>returns float32 just like the 1.0-x
>>    
>>
>
>Why is this the right thing? Python floats are float64.
>  
>
Yeah, why indeed.  Must be something with the scalar coercion code...

-Travis



-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From charlesr.harris at gmail.com  Wed Nov  1 17:21:41 2006
From: charlesr.harris at gmail.com (Charles R Harris)
Date: Wed, 1 Nov 2006 15:21:41 -0700
Subject: Converting bool to float
In-Reply-To: <eib580$qdt$1@sea.gmane.org>
References: <f4f93d420611011233i60c8128ew63288d5620477c9c@mail.gmail.com>
	<454916EC.6010003@ee.byu.edu> <eib580$qdt$1@sea.gmane.org>
Message-ID: <e06186140611011421k118fd7b7j97bd9614c181ec5b@mail.gmail.com>

On 11/1/06, Robert Kern <robert.kern at gmail.com> wrote:
>
> Travis Oliphant wrote:
>
> > It looks like 1.0-x is doing the right thing.
> >
> > The problem is 1.0*x for matrices is going to float64.  For arrays it
> > returns float32 just like the 1.0-x
>
> Why is this the right thing? Python floats are float64.


Same question here. Float32 is a designer float for special occasions,
float64 is for everyday use.

Chuck
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20061101/64c31235/attachment-0001.html>
-------------- next part --------------
-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
-------------- next part --------------
_______________________________________________
Numpy-discussion mailing list
Numpy-discussion at lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/numpy-discussion

From tim.hochberg at ieee.org  Wed Nov  1 22:16:59 2006
From: tim.hochberg at ieee.org (Tim Hochberg)
Date: Wed, 01 Nov 2006 20:16:59 -0700
Subject: Converting bool to float
In-Reply-To: <45492339.6050302@ee.byu.edu>
References: <f4f93d420611011233i60c8128ew63288d5620477c9c@mail.gmail.com>	<454916EC.6010003@ee.byu.edu>	<eib580$qdt$1@sea.gmane.org>
	<45492339.6050302@ee.byu.edu>
Message-ID: <4549632B.4010001@ieee.org>

Travis Oliphant wrote:
> Robert Kern wrote:
>
>   
>> Travis Oliphant wrote:
>>
>>  
>>
>>     
>>> It looks like 1.0-x is doing the right thing.
>>>
>>> The problem is 1.0*x for matrices is going to float64.  For arrays it 
>>> returns float32 just like the 1.0-x
>>>    
>>>
>>>       
>> Why is this the right thing? Python floats are float64.
>>  
>>
>>     
> Yeah, why indeed.  Must be something with the scalar coercion code...

This is one of those things that pops up every few years. I suspect that 
the best thing to do here is to treat 1.0, and all Python floats as 
having a kind (float), but no precision. Or, equivalently treat them as 
the smallest precision floating point value. The rationale behind this 
is that otherwise float32 array will be promoted whenever they are 
multiplied by Python floating point scalars. If Python floats are 
treated as Float64 for purposes of determining output precision then 
anyone using float32 arrays is going to have to wrap all of their 
literals in float32 to prevent inadvertent upcasting to float64. This 
was the origin of the (rather clunky) numarray spacesaver flag.

It's no skin off my nose either way, since I pretty much never use 
float32, but I suspect that treating python floats equivalently to 
float64 scalars would be a mistake. At the very least it deserves a bit 
of discussion.

-tim


-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From charlesr.harris at gmail.com  Wed Nov  1 22:19:45 2006
From: charlesr.harris at gmail.com (Charles R Harris)
Date: Wed, 1 Nov 2006 20:19:45 -0700
Subject: Converting bool to float
In-Reply-To: <f4f93d420611011750j78f8fa26h9d048821e7dd0f05@mail.gmail.com>
References: <f4f93d420611011233i60c8128ew63288d5620477c9c@mail.gmail.com>
	<454916EC.6010003@ee.byu.edu>
	<f4f93d420611011750j78f8fa26h9d048821e7dd0f05@mail.gmail.com>
Message-ID: <e06186140611011919n70d09aebsa1fdbfaefdc1ae8b@mail.gmail.com>

On 11/1/06, Keith Goodman <kwgoodman at gmail.com> wrote:
>
> On 11/1/06, Travis Oliphant <oliphant at ee.byu.edu> wrote:
> > It looks like 1.0-x is doing the right thing.
> >
> > The problem is 1.0*x for matrices is going to float64.  For arrays it
> > returns float32 just like the 1.0-x
> >
> > This can't be changed at this point until 1.1
> >
> > We will fix the bug in 1.0*x producing float64, however.  I'm still not
> > sure what's causing it, though.
>
> I think it would be great if float64 was the default in numpy. That
> way most people wouldn't have to worry about dtypes when crunching
> numbers. And then numpy could apply for a trademark on 'it just
> works'.
>
> Having to worry about dtypes makes users (me) nervous.
>
> I imagine a change like this would not be an overnight change, more of
> a long-term goal.
>
> This one, from a previous thread, also makes me nervous:
>
> >> sum(M.ones((300,1)) == 1)
> matrix([[44]], dtype=int8)


That one seems to be fixed:

In [1]: sum(ones((300,1)) == 1)
Out[1]: 300

In [2]: (ones((300,1)) == 1).sum()
Out[2]: 300

The matrix version also returns a numpy scalar, however.

In [20]: sum(matrix(ones((300,1)) == 1))
Out[20]: 300

I wonder if that is expected?

Chuck
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20061101/c696861c/attachment-0001.html>
-------------- next part --------------
-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
-------------- next part --------------
_______________________________________________
Numpy-discussion mailing list
Numpy-discussion at lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/numpy-discussion

From peridot.faceted at gmail.com  Wed Nov  1 22:20:27 2006
From: peridot.faceted at gmail.com (A. M. Archibald)
Date: Wed, 1 Nov 2006 22:20:27 -0500
Subject: Compressed index sets,
	generators and iterators
In-Reply-To: <e86a5fd00611011744sbae3b89oada17ea4718837b2@mail.gmail.com>
References: <e86a5fd00611011744sbae3b89oada17ea4718837b2@mail.gmail.com>
Message-ID: <ce557a360611011920k7b79d9a6v36b0bace92955436@mail.gmail.com>

On 01/11/06, Bill Baxter <wbaxter at gmail.com> wrote:

> What's the reason iterators are not supported currently?
> For instance  A[range(0,4)] works for a 1d A, but A[xrange(0,4)] does not.
> Are iterators just too inefficient to bother with?

If you want an iterator back, a generator comprehension will do it:
(A[i] for i in xrange(0,4))

If the result is to be a numpy array, the size must be known ahead of
time (so that the memory can be allocated) and specified. At this
point, and considering the overhead in calling back and forth to the
generator, the cost of converting to a list (after all,
A[list(xrange(0,40)] should work fine) isn't necessarily worth the
trouble.

A. M. Archibald

-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From tim.hochberg at ieee.org  Wed Nov  1 22:27:22 2006
From: tim.hochberg at ieee.org (Tim Hochberg)
Date: Wed, 01 Nov 2006 20:27:22 -0700
Subject: Compressed index sets,
	generators and iterators
In-Reply-To: <ce557a360611011920k7b79d9a6v36b0bace92955436@mail.gmail.com>
References: <e86a5fd00611011744sbae3b89oada17ea4718837b2@mail.gmail.com>
	<ce557a360611011920k7b79d9a6v36b0bace92955436@mail.gmail.com>
Message-ID: <4549659A.6020506@ieee.org>

A. M. Archibald wrote:
> On 01/11/06, Bill Baxter <wbaxter at gmail.com> wrote:
>
>   
>> What's the reason iterators are not supported currently?
>> For instance  A[range(0,4)] works for a 1d A, but A[xrange(0,4)] does not.
>> Are iterators just too inefficient to bother with?
>>     
>
> If you want an iterator back, a generator comprehension will do it:
> (A[i] for i in xrange(0,4))
>
> If the result is to be a numpy array, the size must be known ahead of
> time (so that the memory can be allocated) and specified. At this
> point, and considering the overhead in calling back and forth to the
> generator, the cost of converting to a list (after all,
> A[list(xrange(0,40)] should work fine) isn't necessarily worth the
> trouble.
>   
Another option for converting an iterator to a sequence that might be 
helpful, particularly if result is large, is to use fromiter. For example:

    a[numpy.fromiter(xrange(0,40, 2), int)]


-tim


-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From charlesr.harris at gmail.com  Wed Nov  1 22:36:48 2006
From: charlesr.harris at gmail.com (Charles R Harris)
Date: Wed, 1 Nov 2006 20:36:48 -0700
Subject: Converting bool to float
In-Reply-To: <4549632B.4010001@ieee.org>
References: <f4f93d420611011233i60c8128ew63288d5620477c9c@mail.gmail.com>
	<454916EC.6010003@ee.byu.edu> <eib580$qdt$1@sea.gmane.org>
	<45492339.6050302@ee.byu.edu> <4549632B.4010001@ieee.org>
Message-ID: <e06186140611011936n6bd37065nbc0138f5d9f2a4c6@mail.gmail.com>

On 11/1/06, Tim Hochberg <tim.hochberg at ieee.org> wrote:
>
> Travis Oliphant wrote:
> > Robert Kern wrote:
> >
> >
> >> Travis Oliphant wrote:
> >>
> >>
> >>
> >>
> >>> It looks like 1.0-x is doing the right thing.
> >>>
> >>> The problem is 1.0*x for matrices is going to float64.  For arrays it
> >>> returns float32 just like the 1.0-x
> >>>
> >>>
> >>>
> >> Why is this the right thing? Python floats are float64.
> >>
> >>
> >>
> > Yeah, why indeed.  Must be something with the scalar coercion code...
>
> This is one of those things that pops up every few years. I suspect that
> the best thing to do here is to treat 1.0, and all Python floats as
> having a kind (float), but no precision. Or, equivalently treat them as
> the smallest precision floating point value. The rationale behind this
> is that otherwise float32 array will be promoted whenever they are
> multiplied by Python floating point scalars. If Python floats are
> treated as Float64 for purposes of determining output precision then
> anyone using float32 arrays is going to have to wrap all of their
> literals in float32 to prevent inadvertent upcasting to float64. This
> was the origin of the (rather clunky) numarray spacesaver flag.
>
> It's no skin off my nose either way, since I pretty much never use
> float32, but I suspect that treating python floats equivalently to
> float64 scalars would be a mistake. At the very least it deserves a bit
> of discussion.


Well, I think that the present convention of having the array float type
determine the output type when doing a binary op with a scalar makes sense.
The question is what to do when the initial array is an integer type and
needs to be promoted. Now I could see

1) coercing the scalar float to integer, which is probably consistent with
the treatment of integer types. (boo)
2) requiring explicit use of float types, i.e., float64(1.0), which is a bit
clumsy.
3) promoting to float64 by default and expecting the user to specify
float32(1.0) when needed.

I prefer 3, as float32 is probably not the most used data type. So the rule
would be

numpy_int array + python_int -- type numpy_int
numpy_int array + python_flt -- type float64
numpy_int array + numpy_flt -- type numpy_flt
numpy_flt array + python_flt --  type numpy_flt

Seems a bit much to remember, but things always get complicated when you
want to control the types. Mind that going from int64 to float64 can lead to
loss of precision.

Chuck
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20061101/f6125458/attachment-0001.html>
-------------- next part --------------
-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
-------------- next part --------------
_______________________________________________
Numpy-discussion mailing list
Numpy-discussion at lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/numpy-discussion

From oliphant.travis at ieee.org  Wed Nov  1 23:41:34 2006
From: oliphant.travis at ieee.org (Travis Oliphant)
Date: Wed, 01 Nov 2006 21:41:34 -0700
Subject: dgesdd error
Message-ID: <454976FE.20504@ieee.org>


Apparently some dgesdd libraries don't actually compute the correct 
value for the work-space size if requested. 

This results in an ** ILLEGAL value and program termination from LAPACK.

I've added code in the latest SVN to that particular wrapper to make 
sure the query gives back the minimums at least, but I'd like to see who 
else has the problem.

On my system, doing numpy.linalg.svd on a 25 x 50 array pulled the trigger.

Anybody else have that problem with their lapack library?

-Travis


-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From george.sakkis at gmail.com  Wed Nov  1 23:42:14 2006
From: george.sakkis at gmail.com (George Sakkis)
Date: Thu, 02 Nov 2006 04:42:14 -0000
Subject: Reading records from file and sorting
References: <1162337923.747767.323100@b28g2000cwb.googlegroups.com>
	<20061101224620.GA29602@dogbert.sdsl.sun.ac.za>
Message-ID: <1162442534.508320.151770@e3g2000cwe.googlegroups.com>

Albert Strasheim wrote:

> Check the thread "Strange results when sorting array with fields" from
> about a week back. Travis made some changes to sorting in the presence
> of fields that should solve your problem, assuming your fields appear in
> the order you want to sort (i.e. you want to sort f1, f2, f3 and not
> something like f1, f3, f2).

I'm afraid this won't help in my case; I want to sort twice, once by f1
and once by f2. I guess I could make a second file with the fields
swapped but this seems more messy and inefficient than Francesc's
suggestion.

George


-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From fperez.net at gmail.com  Wed Nov  1 23:43:24 2006
From: fperez.net at gmail.com (Fernando Perez)
Date: Wed, 1 Nov 2006 21:43:24 -0700
Subject: dgesdd error
In-Reply-To: <454976FE.20504@ieee.org>
References: <454976FE.20504@ieee.org>
Message-ID: <db6b5ecc0611012043l579ed982if8d8c27ea3dede8@mail.gmail.com>

On 11/1/06, Travis Oliphant <oliphant.travis at ieee.org> wrote:
>
> Apparently some dgesdd libraries don't actually compute the correct
> value for the work-space size if requested.
>
> This results in an ** ILLEGAL value and program termination from LAPACK.
>
> I've added code in the latest SVN to that particular wrapper to make
> sure the query gives back the minimums at least, but I'd like to see who
> else has the problem.
>
> On my system, doing numpy.linalg.svd on a 25 x 50 array pulled the trigger.
>
> Anybody else have that problem with their lapack library?

Seems fine here:

In [4]: N.linalg.svd(N.random.rand(25,50));

In [5]: N.linalg.svd(N.random.rand(25,50));

In [6]: N.__version__
Out[6]: '1.0.1.dev3423'

Ubuntu Dapper (6.06) box, up to date.  I'm using the ubuntu-supplied
ATLAS and LAPACK, I can give you more info if needed.

Cheers,

f

-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From charlesr.harris at gmail.com  Wed Nov  1 23:52:34 2006
From: charlesr.harris at gmail.com (Charles R Harris)
Date: Wed, 1 Nov 2006 21:52:34 -0700
Subject: Reading records from file and sorting
In-Reply-To: <1162442534.508320.151770@e3g2000cwe.googlegroups.com>
References: <1162337923.747767.323100@b28g2000cwb.googlegroups.com>
	<20061101224620.GA29602@dogbert.sdsl.sun.ac.za>
	<1162442534.508320.151770@e3g2000cwe.googlegroups.com>
Message-ID: <e06186140611012052u16743231t9a508338f883b4@mail.gmail.com>

On 11/1/06, George Sakkis <george.sakkis at gmail.com> wrote:
>
> Albert Strasheim wrote:
>
> > Check the thread "Strange results when sorting array with fields" from
> > about a week back. Travis made some changes to sorting in the presence
> > of fields that should solve your problem, assuming your fields appear in
> > the order you want to sort (i.e. you want to sort f1, f2, f3 and not
> > something like f1, f3, f2).
>
> I'm afraid this won't help in my case; I want to sort twice, once by f1
> and once by f2. I guess I could make a second file with the fields
> swapped but this seems more messy and inefficient than Francesc's
> suggestion.


Do you actually want the two different orders, or do you want to sort on the
first field, then sort all the items with the same first field on the second
field?

Chuck
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20061101/4cdbee6b/attachment-0001.html>
-------------- next part --------------
-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
-------------- next part --------------
_______________________________________________
Numpy-discussion mailing list
Numpy-discussion at lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/numpy-discussion

From sransom at nrao.edu  Wed Nov  1 23:55:27 2006
From: sransom at nrao.edu (Scott Ransom)
Date: Wed, 1 Nov 2006 23:55:27 -0500
Subject: Converting bool to float
In-Reply-To: <4549632B.4010001@ieee.org>
References: <f4f93d420611011233i60c8128ew63288d5620477c9c@mail.gmail.com>
	<454916EC.6010003@ee.byu.edu> <eib580$qdt$1@sea.gmane.org>
	<45492339.6050302@ee.byu.edu> <4549632B.4010001@ieee.org>
Message-ID: <20061102045527.GB11421@ssh.cv.nrao.edu>

On Wed, Nov 01, 2006 at 08:16:59PM -0700, Tim Hochberg wrote:
> Travis Oliphant wrote:
> > Robert Kern wrote:
> >> Travis Oliphant wrote:
> >>
> >>> It looks like 1.0-x is doing the right thing.
> >>>
> >>> The problem is 1.0*x for matrices is going to float64.  For arrays it 
> >>> returns float32 just like the 1.0-x
> >>>       
> >> Why is this the right thing? Python floats are float64.
> >>  
> > Yeah, why indeed.  Must be something with the scalar coercion code...
> 
> This is one of those things that pops up every few years. I suspect that 
> the best thing to do here is to treat 1.0, and all Python floats as 
> having a kind (float), but no precision. Or, equivalently treat them as 
> the smallest precision floating point value. The rationale behind this 
> is that otherwise float32 array will be promoted whenever they are 
> multiplied by Python floating point scalars. If Python floats are 
> treated as Float64 for purposes of determining output precision then 
> anyone using float32 arrays is going to have to wrap all of their 
> literals in float32 to prevent inadvertent upcasting to float64. This 
> was the origin of the (rather clunky) numarray spacesaver flag.

I'm one of those people who made serious use of that clunky
spacesaver flag for precisely this reason.  I deal with several GB
arrays of 32-bit floats (or 32-bit x2 complex numbers) on a
regular basis.  Having automatic upcasting from scalar operations
can be a royal pain.

Scott

-- 
Scott M. Ransom            Address:  NRAO
Phone:  (434) 296-0320               520 Edgemont Rd.
email:  sransom at nrao.edu             Charlottesville, VA 22903 USA
GPG Fingerprint: 06A9 9553 78BE 16DB 407B  FFCA 9BFA B6FF FFD3 2989

-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From charlesr.harris at gmail.com  Wed Nov  1 23:58:10 2006
From: charlesr.harris at gmail.com (Charles R Harris)
Date: Wed, 1 Nov 2006 21:58:10 -0700
Subject: dgesdd error
In-Reply-To: <db6b5ecc0611012043l579ed982if8d8c27ea3dede8@mail.gmail.com>
References: <454976FE.20504@ieee.org>
	<db6b5ecc0611012043l579ed982if8d8c27ea3dede8@mail.gmail.com>
Message-ID: <e06186140611012058j654c4432wf0c15b2f6bd9ac1b@mail.gmail.com>

On 11/1/06, Fernando Perez <fperez.net at gmail.com> wrote:
>
> On 11/1/06, Travis Oliphant <oliphant.travis at ieee.org> wrote:
> >
> > Apparently some dgesdd libraries don't actually compute the correct
> > value for the work-space size if requested.
> >
> > This results in an ** ILLEGAL value and program termination from LAPACK.
> >
> > I've added code in the latest SVN to that particular wrapper to make
> > sure the query gives back the minimums at least, but I'd like to see who
> > else has the problem.
> >
> > On my system, doing numpy.linalg.svd on a 25 x 50 array pulled the
> trigger.
> >
> > Anybody else have that problem with their lapack library?
>
> Seems fine here:


Fine here also: 1.0.1.dev3416

Fedora 6, ATLAS and LAPACK from extras.

Chuck
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20061101/ba081e24/attachment-0001.html>
-------------- next part --------------
-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
-------------- next part --------------
_______________________________________________
Numpy-discussion mailing list
Numpy-discussion at lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/numpy-discussion

From charlesr.harris at gmail.com  Thu Nov  2 00:00:05 2006
From: charlesr.harris at gmail.com (Charles R Harris)
Date: Wed, 1 Nov 2006 22:00:05 -0700
Subject: dgesdd error
In-Reply-To: <e06186140611012058j654c4432wf0c15b2f6bd9ac1b@mail.gmail.com>
References: <454976FE.20504@ieee.org>
	<db6b5ecc0611012043l579ed982if8d8c27ea3dede8@mail.gmail.com>
	<e06186140611012058j654c4432wf0c15b2f6bd9ac1b@mail.gmail.com>
Message-ID: <e06186140611012100q1b72addeh495d886ddc60233c@mail.gmail.com>

On 11/1/06, Charles R Harris <charlesr.harris at gmail.com> wrote:
>
>
>
> On 11/1/06, Fernando Perez <fperez.net at gmail.com> wrote:
> >
> > On 11/1/06, Travis Oliphant <oliphant.travis at ieee.org> wrote:
> > >
> > > Apparently some dgesdd libraries don't actually compute the correct
> > > value for the work-space size if requested.
> > >
> > > This results in an ** ILLEGAL value and program termination from
> > LAPACK.
> > >
> > > I've added code in the latest SVN to that particular wrapper to make
> > > sure the query gives back the minimums at least, but I'd like to see
> > who
> > > else has the problem.
> > >
> > > On my system, doing numpy.linalg.svd on a 25 x 50 array pulled the
> > trigger.
> > >
> > > Anybody else have that problem with their lapack library?
> >
> > Seems fine here:
>
>
> Fine here also: 1.0.1.dev3416
>
> Fedora 6, ATLAS and LAPACK from extras.
>
> Chuck
>

Oh, and 32 bit.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20061101/2ff70265/attachment-0001.html>
-------------- next part --------------
-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
-------------- next part --------------
_______________________________________________
Numpy-discussion mailing list
Numpy-discussion at lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/numpy-discussion

From robert.kern at gmail.com  Thu Nov  2 00:17:27 2006
From: robert.kern at gmail.com (Robert Kern)
Date: Wed, 01 Nov 2006 21:17:27 -0800
Subject: Converting bool to float
In-Reply-To: <4549632B.4010001@ieee.org>
References: <f4f93d420611011233i60c8128ew63288d5620477c9c@mail.gmail.com>	<454916EC.6010003@ee.byu.edu>	<eib580$qdt$1@sea.gmane.org>	<45492339.6050302@ee.byu.edu>
	<4549632B.4010001@ieee.org>
Message-ID: <eibv18$sj3$1@sea.gmane.org>

Tim Hochberg wrote:
> Travis Oliphant wrote:
>> Robert Kern wrote:
>>> Travis Oliphant wrote:   
>>>> It looks like 1.0-x is doing the right thing.
>>>>
>>>> The problem is 1.0*x for matrices is going to float64.  For arrays it 
>>>> returns float32 just like the 1.0-x
>>>>       
>>> Why is this the right thing? Python floats are float64.
>>>  
>> Yeah, why indeed.  Must be something with the scalar coercion code...
> 
> This is one of those things that pops up every few years. I suspect that 
> the best thing to do here is to treat 1.0, and all Python floats as 
> having a kind (float), but no precision. Or, equivalently treat them as 
> the smallest precision floating point value. The rationale behind this 
> is that otherwise float32 array will be promoted whenever they are 
> multiplied by Python floating point scalars. If Python floats are 
> treated as Float64 for purposes of determining output precision then 
> anyone using float32 arrays is going to have to wrap all of their 
> literals in float32 to prevent inadvertent upcasting to float64. This 
> was the origin of the (rather clunky) numarray spacesaver flag.
> 
> It's no skin off my nose either way, since I pretty much never use 
> float32, but I suspect that treating python floats equivalently to 
> float64 scalars would be a mistake. At the very least it deserves a bit 
> of discussion.

Well, they *are* 64-bit floating point numbers. You simply can't get around 
that. That's why we now have all of the scalar types: you can get any precision 
scalars that you want as long as you are explicit about it (and explicit is 
better than implicit). The spacesaver flag was the only solution before the 
various scalar types existed. I'd like to suggest that the discussion already 
occurred some time ago and has concluded in favor of the scalar types. 
Downcasting should be explicit.

However, whether or not float32 arrays operated with Python float scalars give 
float32 or float64 arrays is tangential to my question. Does anyone actually 
think that a Python float operated with a boolean array should give a float32 
result? Must we *up*cast a boolean array to float64 to preserve the precision of 
the scalar?

-- 
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless enigma
  that is made terrible by our own mad attempt to interpret it as though it had
  an underlying truth."
   -- Umberto Eco


-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From david at ar.media.kyoto-u.ac.jp  Thu Nov  2 00:22:10 2006
From: david at ar.media.kyoto-u.ac.jp (David Cournapeau)
Date: Thu, 02 Nov 2006 14:22:10 +0900
Subject: numpy.random.ranomd_integers returning only
	negative numbers ?
Message-ID: <45498082.1090201@ar.media.kyoto-u.ac.jp>

Hi,

    I want to generate some random integers,let's say in the range [- 
2^15, 2^16]. Why doing:

    noise = numpy.random.random_integers(- (2 ** 15), (2 * 15), 22050)

    gives only negative numbers ?

    Cheers,

    David


-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From fperez.net at gmail.com  Thu Nov  2 00:25:57 2006
From: fperez.net at gmail.com (Fernando Perez)
Date: Wed, 1 Nov 2006 22:25:57 -0700
Subject: numpy.random.ranomd_integers returning only
	negative numbers ?
In-Reply-To: <45498082.1090201@ar.media.kyoto-u.ac.jp>
References: <45498082.1090201@ar.media.kyoto-u.ac.jp>
Message-ID: <db6b5ecc0611012125y3df72d4fmdfa87179695b4aef@mail.gmail.com>

On 11/1/06, David Cournapeau <david at ar.media.kyoto-u.ac.jp> wrote:
> Hi,
>
>     I want to generate some random integers,let's say in the range [-
> 2^15, 2^16]. Why doing:
>
>     noise = numpy.random.random_integers(- (2 ** 15), (2 * 15), 22050)
>
>     gives only negative numbers ?

In [3]: noise = numpy.random.random_integers(- (2 ** 15), (2 * 15), 22050)

In [4]: noise[noise>0].shape
Out[4]: (17,)

In [5]: noise[noise<0].shape
Out[5]: (22033,)

In [6]: noise = numpy.random.random_integers(-(2**15), (2 ** 15), 22050)

In [7]: noise[noise>0].shape
Out[7]: (11006,)

In [8]: noise[noise>0].shape
Out[8]: (11006,)

In [9]: 17./22033
Out[9]: 0.00077156991785049694

In [10]: 2.0*15/2**15
Out[10]: 0.00091552734375

close enough ;)

Cheers,

f

-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From kwgoodman at gmail.com  Thu Nov  2 00:29:31 2006
From: kwgoodman at gmail.com (Keith Goodman)
Date: Wed, 1 Nov 2006 21:29:31 -0800
Subject: numpy.random.ranomd_integers returning only
	negative numbers ?
In-Reply-To: <45498082.1090201@ar.media.kyoto-u.ac.jp>
References: <45498082.1090201@ar.media.kyoto-u.ac.jp>
Message-ID: <f4f93d420611012129x765737fax9bc0b65ed7400345@mail.gmail.com>

On 11/1/06, David Cournapeau <david at ar.media.kyoto-u.ac.jp> wrote:
> Hi,
>
>     I want to generate some random integers,let's say in the range [-
> 2^15, 2^16]. Why doing:
>
>     noise = numpy.random.random_integers(- (2 ** 15), (2 * 15), 22050)
>
>     gives only negative numbers ?

I guess 2^15 is too big to be an int.

Would this work?

int(N.random.uniform(1e15,1e16))

-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From robert.kern at gmail.com  Thu Nov  2 00:42:06 2006
From: robert.kern at gmail.com (Robert Kern)
Date: Wed, 01 Nov 2006 21:42:06 -0800
Subject: numpy.random.ranomd_integers returning only
 negative numbers ?
In-Reply-To: <f4f93d420611012129x765737fax9bc0b65ed7400345@mail.gmail.com>
References: <45498082.1090201@ar.media.kyoto-u.ac.jp>
	<f4f93d420611012129x765737fax9bc0b65ed7400345@mail.gmail.com>
Message-ID: <eic0fe$v8o$1@sea.gmane.org>

Keith Goodman wrote:
> On 11/1/06, David Cournapeau <david at ar.media.kyoto-u.ac.jp> wrote:
>> Hi,
>>
>>     I want to generate some random integers,let's say in the range [-
>> 2^15, 2^16]. Why doing:
>>
>>     noise = numpy.random.random_integers(- (2 ** 15), (2 * 15), 22050)
>>
>>     gives only negative numbers ?
> 
> I guess 2^15 is too big to be an int.

No, it isn't. As Fernando pointed out, if David's code is what he typed here, 
then his upper bound is (2 * 15) == 30 not (2 ** 15) == 32768 as he intended 
intended.

-- 
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless enigma
  that is made terrible by our own mad attempt to interpret it as though it had
  an underlying truth."
   -- Umberto Eco


-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From david at ar.media.kyoto-u.ac.jp  Thu Nov  2 00:41:46 2006
From: david at ar.media.kyoto-u.ac.jp (David Cournapeau)
Date: Thu, 02 Nov 2006 14:41:46 +0900
Subject: numpy.random.ranomd_integers returning only
 negative numbers ?
In-Reply-To: <db6b5ecc0611012125y3df72d4fmdfa87179695b4aef@mail.gmail.com>
References: <45498082.1090201@ar.media.kyoto-u.ac.jp>
	<db6b5ecc0611012125y3df72d4fmdfa87179695b4aef@mail.gmail.com>
Message-ID: <4549851A.4050505@ar.media.kyoto-u.ac.jp>

Fernando Perez wrote:
> On 11/1/06, David Cournapeau <david at ar.media.kyoto-u.ac.jp> wrote:
>   
>> Hi,
>>
>>     I want to generate some random integers,let's say in the range [-
>> 2^15, 2^16]. Why doing:
>>
>>     noise = numpy.random.random_integers(- (2 ** 15), (2 * 15), 22050)
>>
>>     gives only negative numbers ?
>>     
>
> In [3]: noise = numpy.random.random_integers(- (2 ** 15), (2 * 15), 22050)
>
> In [4]: noise[noise>0].shape
> Out[4]: (17,)
>
> In [5]: noise[noise<0].shape
> Out[5]: (22033,)
>
> In [6]: noise = numpy.random.random_integers(-(2**15), (2 ** 15), 22050)
>
> In [7]: noise[noise>0].shape
> Out[7]: (11006,)
>
> In [8]: noise[noise>0].shape
> Out[8]: (11006,)
>
> In [9]: 17./22033
> Out[9]: 0.00077156991785049694
>
> In [10]: 2.0*15/2**15
> Out[10]: 0.00091552734375
>
> close enough ;)
>   
Ok, I deserve my email to be called the most stupid email of the week... 
Sorry for the noise,

David

-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From charlesr.harris at gmail.com  Thu Nov  2 00:55:39 2006
From: charlesr.harris at gmail.com (Charles R Harris)
Date: Wed, 1 Nov 2006 22:55:39 -0700
Subject: Converting bool to float
In-Reply-To: <eibv18$sj3$1@sea.gmane.org>
References: <f4f93d420611011233i60c8128ew63288d5620477c9c@mail.gmail.com>
	<454916EC.6010003@ee.byu.edu> <eib580$qdt$1@sea.gmane.org>
	<45492339.6050302@ee.byu.edu> <4549632B.4010001@ieee.org>
	<eibv18$sj3$1@sea.gmane.org>
Message-ID: <e06186140611012155x38f47a9ard4c8126a13a344d7@mail.gmail.com>

On 11/1/06, Robert Kern <robert.kern at gmail.com> wrote:
>
> Tim Hochberg wrote:
> > Travis Oliphant wrote:


<snip>

However, whether or not float32 arrays operated with Python float scalars
> give
> float32 or float64 arrays is tangential to my question. Does anyone
> actually
> think that a Python float operated with a boolean array should give a
> float32
> result? Must we *up*cast a boolean array to float64 to preserve the
> precision of
> the scalar?


Probably doesn't matter most of the time, I suppose, who is going to check?
I tend to think doubles because they are a bit faster on the x86
architecture and because they are a pretty common default.

Chuck
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20061101/88fa6bc3/attachment-0001.html>
-------------- next part --------------
-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
-------------- next part --------------
_______________________________________________
Numpy-discussion mailing list
Numpy-discussion at lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/numpy-discussion

From kgyhmgt at paniere.com  Thu Nov  2 01:10:10 2006
From: kgyhmgt at paniere.com (G.)
Date: Thu, 2 Nov 2006 14:10:10 +0800
Subject: Wishesquot case
Message-ID: <001101c6fe45$8d93f3c0$9fef84db@pj3fe882826648>


So Whenever of Call of. Help Need Else Close a.
Girl Crash of Burn Wide Maybe Promise.
Now share in life is dream forever let. Began or to live again dedicate these pages.
Now share in life is dream forever let.
Stand Every of Falling Mend Broken Endless Whiter Shade Pale! Access links Featured or Week Thank Dido Wake me.
Music Audio Search site particular is song Crescendo download plugin. Others a found comfort or.
Angel red Lucky Mind Youre Tomorrow Comes. See pictures contains lyrics.
Version Freddie Aguilar am. Work in Keep Alive Truly Madly Deeply Everytime go Away. Time Ever saw Want Christmas am Wanna Miss.
How do Frozen Lady? Beneath Wings at Moment Make Feel.
Need Else Close am. S Mclachlan a First Time Ever is saw Want in Christmas Wanna.
So Whenever of Call of. Eyes Sometimes am Touch in More Than.
Began or to live in again dedicate these? Meditate or enjoy in Remember reload is updates Important am Copyright Notice if.
Hope in Colors Tell him Song Brought Hearts of Rain in.
Being solely Internet mails chats now share life of. Spend or Lifetime Young a! Hate Swear try Into is Shoulder of Annies Morn Vision a Hard.
Make of Feel Sweet a Friends or mws or ill is Take Breath.
Sang Especially Help Need Else Close Lost Babe.
Chords music of quotes excerpts from or messages net. Help Need Else Close a.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20061102/8580c6d7/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: Shade.gif
Type: image/gif
Size: 9663 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20061102/8580c6d7/attachment-0001.gif>
-------------- next part --------------
-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
-------------- next part --------------
_______________________________________________
Numpy-discussion mailing list
Numpy-discussion at lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/numpy-discussion

From david at ar.media.kyoto-u.ac.jp  Thu Nov  2 02:12:00 2006
From: david at ar.media.kyoto-u.ac.jp (David Cournapeau)
Date: Thu, 02 Nov 2006 16:12:00 +0900
Subject: [ANN[ pysamplerate,
	a library to do high quality samplerate convertion
Message-ID: <45499A40.8070204@ar.media.kyoto-u.ac.jp>

(if those announcements are not welcome on the lists, please tell me)

Hi there,

    I've just developed a small package to wrap Source Rabbit Code from 
Erik de Castro de Lopo, which is a library to do high quality samplerate 
conversion (http://www.mega-nerd.com/SRC/). You can download it here:

    
http://www.ar.media.kyoto-u.ac.jp/members/david/softwares/pysamplerate/pysamplerate-0.1.tar.gz
    
http://www.ar.media.kyoto-u.ac.jp/members/david/softwares/pysamplerate/pysamplerate-0.1.zip

    Example of error conversions:
   
    
http://www.ar.media.kyoto-u.ac.jp/members/david/softwares/pysamplerate/example1.png

    Basically, you just import resample from pysamplerate, and 
resample(sins, fr/fs) gives you a converted signal at sampling rate fr, 
with fs the original sampling rate. You can also select the conversion 
method (eg 'sinc_best', etc...). The pydocs + examples should be enough
   
    This package can be useful for people who deal with music signal 
processing and co, when scipy.resample is not enough. Only float32 and 
float64 input are supported for now, I will polish the package when I 
will have some more time,

    cheers,

    David

-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From fperez.net at gmail.com  Thu Nov  2 02:23:49 2006
From: fperez.net at gmail.com (Fernando Perez)
Date: Thu, 2 Nov 2006 00:23:49 -0700
Subject: [SciPy-user] [ANN[ pysamplerate,
	a library to do high quality samplerate convertion
In-Reply-To: <45499A40.8070204@ar.media.kyoto-u.ac.jp>
References: <45499A40.8070204@ar.media.kyoto-u.ac.jp>
Message-ID: <db6b5ecc0611012323k4f9813cfl712aeac485945494@mail.gmail.com>

On 11/2/06, David Cournapeau <david at ar.media.kyoto-u.ac.jp> wrote:
> (if those announcements are not welcome on the lists, please tell me)

Frankly, if an announcement for a free Python signal processing
library is not welcome on the scipy lists, I don't know where it would
be ;)

As a minor note though: please remember to make a note of this in the
Topical Software wiki; it's good for that page to remain up to date
with newly released packages and libraries.

Thanks!

f

-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From Mailer-Daemon at lists.sourceforge.net  Thu Nov  2 06:06:46 2006
From: Mailer-Daemon at lists.sourceforge.net (Mail Delivery System)
Date: Thu, 02 Nov 2006 03:06:46 -0800
Subject: Warning: message 1GfAoO-0007im-Sh delayed 24
	hours
Message-ID: <E1GfaPG-0000jm-La@externalmx-1.sourceforge.net>

This message was created automatically by mail delivery software.
A message that you sent has not yet been delivered to one or more of its
recipients after more than 24 hours on the queue on externalmx-1.sourceforge.net.

The message identifier is:     1GfAoO-0007im-Sh
The subject of the message is: looked like me the full sized HARLEY and began to her, with this but as
The date of the message is:    Wed, 01 Nov 2006 15:46:73 +0800

The address to which the message has not yet been delivered is:

  numpy-discussion at lists.sourceforge.net
    Delay reason: SMTP error from remote mailer after RCPT TO:<numpy-discussion at lists.sourceforge.net>:
    host mail.sourceforge.net [66.35.250.206]: 451-Could not complete sender verify callout
    451-Could not complete sender verify callout for
    451-<numpy-discussion at lists.sourceforge.net>.
    451-The mail server(s) for the domain may be temporarily unreachable, or
    451-they may be permanently unreachable from this server. In the latter case,
    451-you need to change the address or create an MX record for its domain
    451-if it is 

No action is required on your part. Delivery attempts will continue for
some time, and this warning may be repeated at intervals if the message
remains undelivered. Eventually the mail delivery software will give up,
and when that happens, the message will be returned to you.

-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From svetosch at gmx.net  Thu Nov  2 07:23:38 2006
From: svetosch at gmx.net (Sven Schreiber)
Date: Thu, 02 Nov 2006 13:23:38 +0100
Subject: numpy book
Message-ID: <4549E34A.4060507@gmx.net>

Hi,
now that 1.0 is out I would like to ask again about the status of the
numpy ebook. Is there any mechanism in place where owners can get the
updates?

Note that this is not a request to Travis to send me the latest version
by private email. That would be inefficient and my need is not that
urgent. Nevertheless I think that issue should be settled.

Excuse me if I've missed something obvious in that respect.

Thanks,
Sven

-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From markusro at element.fkp.physik.tu-darmstadt.de  Thu Nov  2 10:13:34 2006
From: markusro at element.fkp.physik.tu-darmstadt.de (Markus Rosenstihl)
Date: Thu, 2 Nov 2006 16:13:34 +0100
Subject: Can' compile numpy 1.02rc3 on OSX 10.3.9
In-Reply-To: <5228203bb5a6003c571d52a5e94ef370@element.fkp.physik.tu-darmstadt.de>
References: <5228203bb5a6003c571d52a5e94ef370@element.fkp.physik.tu-darmstadt.de>
Message-ID: <089ba8ec7c78b92f4563ce0c7eba4d85@element.fkp.physik.tu-darmstadt.de>

Hi!
This, and similar problems with other programs were fixed when I  
installed
10.4 SDK from Apple Developer Conenction.
Yes, the 10.4 SDK can be installed on 10.3.9.

Regards
Markus


Am 19.10.2006 um 23:56 schrieb Markus Rosenstihl:

> Hi!
> I try to compile numpy rc3 on Panther and get following errors.
> (I start build with "python2.3 setup.py build" to be sure to use the
> python shipped with OS X. I din't manage to compile Python2.5 either
> yet with similar errors)
> Does anynbody has an Idea?
> gcc-3.3
> XCode 1.5
> November gcc updater is installed
>
>
> regards
>
> Markus
>
> python2.3 setup.py build
> ...
> compile options:
> '-Ibuild/src.darwin-7.9.0-Power_Macintosh-2.3/numpy/core/src
> -Inumpy/core/include
> -Ibuild/src.darwin-7.9.0-Power_Macintosh-2.3/numpy/core
> -Inumpy/core/src -Inumpy/core/include
> -I/System/Library/Frameworks/Python.framework/Versions/2.3/include/
> python2.3 -c'
> gcc: numpy/core/src/multiarraymodule.c
> In file included from numpy/core/src/arrayobject.c:511,
>                   from numpy/core/src/multiarraymodule.c:63:
> numpy/core/src/arraytypes.inc.src: In function `LONGDOUBLE_scan':
> numpy/core/src/arraytypes.inc.src:883: warning: long double format,
> npy_longdouble arg (arg 3)
> gcc -Wl,-F. -Wl,-F. -bundle -framework Python
> build/temp.darwin-7.9.0-Power_Macintosh-2.3/numpy/core/src/
> multiarraymodule.o -o
> build/lib.darwin-7.9.0-Power_Macintosh-2.3/numpy/core/multiarray.so
> ld: Undefined symbols:
> _fstatvfs referenced from Python expected to be defined in libSystem
> _lchown referenced from Python expected to be defined in libSystem
> _statvfs referenced from Python expected to be defined in libSystem
> ld: Undefined symbols:
> _fstatvfs referenced from Python expected to be defined in libSystem
> _lchown referenced from Python expected to be defined in libSystem
> _statvfs referenced from Python expected to be defined in libSystem
> error: Command "gcc -Wl,-F. -Wl,-F. -bundle -framework Python
> build/temp.darwin-7.9.0-Power_Macintosh-2.3/numpy/core/src/
> multiarraymodule.o -o
> build/lib.darwin-7.9.0-Power_Macintosh-2.3/numpy/core/multiarray.so"
> failed with exit status 1
>
>
> ----------------------------------------------------------------------- 
> --
> Using Tomcat but need to do more? Need to support web services,  
> security?
> Get stuff done quickly with pre-integrated technology to make your job  
> easier
> Download IBM WebSphere Application Server v.1.0.1 based on Apache  
> Geronimo
> http://sel.as-us.falkag.net/sel? 
> cmd=lnk&kid=120709&bid=263057&dat=121642
> _______________________________________________
> Numpy-discussion mailing list
> Numpy-discussion at lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/numpy-discussion


-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From oliphant at ee.byu.edu  Thu Nov  2 10:41:11 2006
From: oliphant at ee.byu.edu (Travis Oliphant)
Date: Thu, 02 Nov 2006 08:41:11 -0700
Subject: numpy book
In-Reply-To: <4549E34A.4060507@gmx.net>
References: <4549E34A.4060507@gmx.net>
Message-ID: <454A1197.1020606@ee.byu.edu>


>Note that this is not a request to Travis to send me the latest version
>by private email. That would be inefficient and my need is not that
>urgent. Nevertheless I think that issue should be settled.
>  
>
There will be an update, soon. I'm currently working on the index, 
corrections, and formatting issues.

The update will be sent in conjunction with the release of 1.0.1 which I 
am targetting in 2 weeks.

Thanks for everybody's patience.  If you need an update now, I'm happy 
to send it to you by private email. 

Best regards,

-Travis O.



-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From oliphant at ee.byu.edu  Thu Nov  2 10:48:47 2006
From: oliphant at ee.byu.edu (Travis Oliphant)
Date: Thu, 02 Nov 2006 08:48:47 -0700
Subject: Converting bool to float
In-Reply-To: <eibv18$sj3$1@sea.gmane.org>
References: <f4f93d420611011233i60c8128ew63288d5620477c9c@mail.gmail.com>	<454916EC.6010003@ee.byu.edu>	<eib580$qdt$1@sea.gmane.org>	<45492339.6050302@ee.byu.edu>	<4549632B.4010001@ieee.org>
	<eibv18$sj3$1@sea.gmane.org>
Message-ID: <454A135F.2040009@ee.byu.edu>

Robert Kern wrote:

>Tim Hochberg wrote:
>  
>
>>Travis Oliphant wrote:
>>    
>>
>>>Robert Kern wrote:
>>>      
>>>
>>>>Travis Oliphant wrote:   
>>>>        
>>>>
>>>>>It looks like 1.0-x is doing the right thing.
>>>>>
>>>>>The problem is 1.0*x for matrices is going to float64.  For arrays it 
>>>>>returns float32 just like the 1.0-x
>>>>>      
>>>>>          
>>>>>
>>>>Why is this the right thing? Python floats are float64.
>>>> 
>>>>        
>>>>
>>>Yeah, why indeed.  Must be something with the scalar coercion code...
>>>      
>>>
>>This is one of those things that pops up every few years. I suspect that 
>>the best thing to do here is to treat 1.0, and all Python floats as 
>>having a kind (float), but no precision. Or, equivalently treat them as 
>>the smallest precision floating point value. The rationale behind this 
>>is that otherwise float32 array will be promoted whenever they are 
>>multiplied by Python floating point scalars. If Python floats are 
>>treated as Float64 for purposes of determining output precision then 
>>anyone using float32 arrays is going to have to wrap all of their 
>>literals in float32 to prevent inadvertent upcasting to float64. This 
>>was the origin of the (rather clunky) numarray spacesaver flag.
>>
>>It's no skin off my nose either way, since I pretty much never use 
>>float32, but I suspect that treating python floats equivalently to 
>>float64 scalars would be a mistake. At the very least it deserves a bit 
>>of discussion.
>>    
>>
>
>Well, they *are* 64-bit floating point numbers. You simply can't get around 
>that. That's why we now have all of the scalar types: you can get any precision 
>scalars that you want as long as you are explicit about it (and explicit is 
>better than implicit). The spacesaver flag was the only solution before the 
>various scalar types existed. I'd like to suggest that the discussion already 
>occurred some time ago and has concluded in favor of the scalar types. 
>Downcasting should be explicit.
>
>However, whether or not float32 arrays operated with Python float scalars give 
>float32 or float64 arrays is tangential to my question. Does anyone actually 
>think that a Python float operated with a boolean array should give a float32 
>result? Must we *up*cast a boolean array to float64 to preserve the precision of 
>the scalar?
>
>  
>
The first basic rule is that scalars don't control the precision of the 
output when doing mixed-type calculations *except* when they are of a 
fundamentally different kind. 

Then (if a different kind of scalar is used), the rule is that the 
arrays will be upcast to the "lowest" precision in the group to preserve 
overall precision.   So, when a bool is combined with a "float" kind of 
scalar, the result is float32 because that preserves precision of the 
bool.  Remember it is array precision that takes precedence over scalars 
in mixed type array-scalar operations.

This is the rule.  I agree that this rule is probably flawed in certain 
circumstances.

So, what should be done about it at this point?  Do you think a change 
is acceptable for 1.0.1 or does it need to wait a year until 1.1?

-Travis


















-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From jontwang at gmail.com  Thu Nov  2 10:52:08 2006
From: jontwang at gmail.com (Jonathan Wang)
Date: Thu, 2 Nov 2006 09:52:08 -0600
Subject: Argmax and max inconsistent for object types
Message-ID: <4f7636bf0611020752t3e079d06h1f41e3a12e226de2@mail.gmail.com>

On numpy version 1.0, argmax and max give inconsistent results for an array
of objects. I've seen this problem in both Python native datetime and
mx.DateTime types:

In [22]: print nativeDates
[datetime.datetime(2006, 10, 18, 10, 11, 27), datetime.datetime(2006, 10,
18, 10, 16, 20), datetime.datetime(2006, 10, 18, 10, 21, 23),
datetime.datetime(2006, 10, 18, 10, 31, 13), datetime.datetime(2006, 10, 18,
10, 39, 49), datetime.datetime(2006, 10, 18, 10, 53, 19),
datetime.datetime(2006,
10, 18, 11, 23, 18), datetime.datetime(2006, 10, 18, 17, 18, 43),
datetime.datetime(2006, 10, 18, 17, 21, 49), datetime.datetime(2006, 10, 18,
17, 24, 28), datetime.datetime(2006, 10, 18, 17, 28, 29),
datetime.datetime(2006,
10, 18, 17, 31, 7), datetime.datetime(2006, 10, 18, 17, 36, 26),
datetime.datetime(2006, 10, 19, 10, 17, 45), datetime.datetime(2006, 10, 19,
11, 23, 19), datetime.datetime(2006, 10, 19, 11, 58, 18),
datetime.datetime(2006,
10, 19, 10, 27, 40), datetime.datetime(2006, 10, 19, 13, 17, 14),
datetime.datetime(2006, 10, 19, 13, 21, 17), datetime.datetime(2006, 10, 19,
13, 23, 52), datetime.datetime(2006, 10, 19, 13, 29, 1)]

In [23]: numpy.argmax(nativeDates)
Out[23]: 0

In [24]: numpy.max(nativeDates)
Out[24]: datetime.datetime(2006, 10, 19, 13, 29, 1)

In [25]: nativeDates[0]
Out[25]: datetime.datetime(2006, 10, 18, 10, 11, 27)

I get the same results if I create an array from the list first:

In [28]: dateArr = numpy.array(nativeDates, dtype=object)

In [29]: print dateArr
[2006-10-18 10:11:27 2006-10-18 10:16:20 2006-10-18 10:21:23
 2006-10-18 10:31:13 2006-10-18 10:39:49 2006-10-18 10:53:19
 2006-10-18 11:23:18 2006-10-18 17:18:43 2006-10-18 17:21:49
 2006-10-18 17:24:28 2006-10-18 17:28:29 2006-10-18 17:31:07
 2006-10-18 17:36:26 2006-10-19 10:17:45 2006-10-19 11:23:19
 2006-10-19 11:58:18 2006-10-19 10:27:40 2006-10-19 13:17:14
 2006-10-19 13:21:17 2006-10-19 13:23:52 2006-10-19 13:29:01]

In [30]: numpy.argmax(dateArr)
Out[30]: 0

In [31]: numpy.max(dateArr)
Out[31]: datetime.datetime(2006, 10, 19, 13, 29, 1)

In [32]: dateArr[0]
Out[32]: datetime.datetime(2006, 10, 18, 10, 11, 27)

My guess is that it's related to some underlying memory layout; I've gotten
different results when running this.

Jonathan
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20061102/565ff27a/attachment-0001.html>
-------------- next part --------------
-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
-------------- next part --------------
_______________________________________________
Numpy-discussion mailing list
Numpy-discussion at lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/numpy-discussion

From oliphant at ee.byu.edu  Thu Nov  2 10:55:09 2006
From: oliphant at ee.byu.edu (Travis Oliphant)
Date: Thu, 02 Nov 2006 08:55:09 -0700
Subject: Argmax and max inconsistent for object types
In-Reply-To: <4f7636bf0611020752t3e079d06h1f41e3a12e226de2@mail.gmail.com>
References: <4f7636bf0611020752t3e079d06h1f41e3a12e226de2@mail.gmail.com>
Message-ID: <454A14DD.3000502@ee.byu.edu>

Jonathan Wang wrote:

> On numpy version 1.0, argmax and max give inconsistent results for an 
> array of objects. I've seen this problem in both Python native 
> datetime and mx.DateTime types:


There is a bug in argmax for OBJECT arrays in 1.0  (it's fixed in SVN 
and will be in 1.0.1)

-Travis


-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From fperez.net at gmail.com  Thu Nov  2 11:00:31 2006
From: fperez.net at gmail.com (Fernando Perez)
Date: Thu, 2 Nov 2006 09:00:31 -0700
Subject: numpy book
In-Reply-To: <454A1197.1020606@ee.byu.edu>
References: <4549E34A.4060507@gmx.net> <454A1197.1020606@ee.byu.edu>
Message-ID: <db6b5ecc0611020800u367eb6fdvccf194c6e1aecd14@mail.gmail.com>

On 11/2/06, Travis Oliphant <oliphant at ee.byu.edu> wrote:
>
> >Note that this is not a request to Travis to send me the latest version
> >by private email. That would be inefficient and my need is not that
> >urgent. Nevertheless I think that issue should be settled.
> >
> >
> There will be an update, soon. I'm currently working on the index,
> corrections, and formatting issues.

If I may make a suggestion, it would greatly increase the usability of
the PDF if it had internal and external clickable links, as well as a
PDF table of contents (the one that appears on the left sidebar of
acroread).

Since I know the original is in Lyx, here's my standard preamble for
these kinds of documents, which includes all the goodies for
generating such a PDF, with some color and formatting tweaks so that
it prints legibly on black and white as well as looking nice on
screen.  If you add this to your preamble, a simple 'view->pdflatex'
should work.  I'll be happy to help if you have any issues.

Cheers,

f

%%% My preamble; tweak and season to taste:

% This gives us a better font in URL links (otherwise the default
% MonoSpace font is bitmapped, and it looks horrible in PDF)
\usepackage{courier}

\usepackage{fullpage}

\usepackage{color}  % so we can use red for the fixme warnings

% The hyperref package gives us a pdf with properly built
% internal navigation ('pdf bookmarks' for the table of contents,
% internal cross-reference links, web links for URLs, etc.)

% A few colors to replace the defaults for certain link types
\definecolor{darkorange}{rgb}{.71,0.21,0.01}
\definecolor{darkgreen}{rgb}{.12,.54,.11}

\usepackage[
  %pdftex,  % needed for pdflatex
  breaklinks=true,  % so long urls are correctly broken across lines
  colorlinks=true,
  urlcolor=blue,
  linkcolor=darkorange,
  citecolor=darkgreen,
  ]{hyperref}

% This helps prevent overly long lines that stretch beyond the margins
\sloppy

% Define a \fixme command to mark visually things needing fixing in the draft.
% For final printing or to simply disable these bright warnings, simply
% uncomment the \renewcommand redefinition below

\newcommand{\fixme}[1] {
\textcolor{red}{
{\fbox{ {\bf FIX}
\ensuremath{\blacktriangleright \blacktriangleright \blacktriangleright}}
{\bf #1}
\fbox{\ensuremath{ \blacktriangleleft \blacktriangleleft \blacktriangleleft }
} } }
}
% Uncomment the next line to make the \fixme command be a no-op
%\renewcommand{\fixme}[1]{}

%%% If you also want to use the listings package for nicely formatted
%%% Python source code, this configuration produces good on-paper and
%%% on-screen results:

\definecolor{orange}{cmyk}{0,0.4,0.8,0.2}
% Use and configure listings package for nicely formatted code
\usepackage{listings}
\lstset{
  language=Python,
  basicstyle=\small\ttfamily,
  commentstyle=\ttfamily\color{blue},
  stringstyle=\ttfamily\color{orange},
  showstringspaces=false,
  breaklines=true,
  postbreak = \space\dots
}

-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From oliphant at ee.byu.edu  Thu Nov  2 11:03:20 2006
From: oliphant at ee.byu.edu (Travis Oliphant)
Date: Thu, 02 Nov 2006 09:03:20 -0700
Subject: numpy book
In-Reply-To: <db6b5ecc0611020800u367eb6fdvccf194c6e1aecd14@mail.gmail.com>
References: <4549E34A.4060507@gmx.net> <454A1197.1020606@ee.byu.edu>
	<db6b5ecc0611020800u367eb6fdvccf194c6e1aecd14@mail.gmail.com>
Message-ID: <454A16C8.7060907@ee.byu.edu>

Fernando Perez wrote:

>On 11/2/06, Travis Oliphant <oliphant at ee.byu.edu> wrote:
>  
>
>>>Note that this is not a request to Travis to send me the latest version
>>>by private email. That would be inefficient and my need is not that
>>>urgent. Nevertheless I think that issue should be settled.
>>>
>>>
>>>      
>>>
>>There will be an update, soon. I'm currently working on the index,
>>corrections, and formatting issues.
>>    
>>
>
>If I may make a suggestion, it would greatly increase the usability of
>the PDF if it had internal and external clickable links, as well as a
>PDF table of contents (the one that appears on the left sidebar of
>acroread).
>
>Since I know the original is in Lyx, here's my standard preamble for
>these kinds of documents, which includes all the goodies for
>generating such a PDF, with some color and formatting tweaks so that
>it prints legibly on black and white as well as looking nice on
>screen.  If you add this to your preamble, a simple 'view->pdflatex'
>should work.  I'll be happy to help if you have any issues.
>  
>

Thanks for the pre-amble.  Does it require pdflatex?  I use ps2pdf 
because to generate the shaded boxes and graphics.  I could probably try 
to do it with pdflatex and png files but I haven't tried, yet.

-Travis



-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From fperez.net at gmail.com  Thu Nov  2 11:07:21 2006
From: fperez.net at gmail.com (Fernando Perez)
Date: Thu, 2 Nov 2006 09:07:21 -0700
Subject: numpy book
In-Reply-To: <454A16C8.7060907@ee.byu.edu>
References: <4549E34A.4060507@gmx.net> <454A1197.1020606@ee.byu.edu>
	<db6b5ecc0611020800u367eb6fdvccf194c6e1aecd14@mail.gmail.com>
	<454A16C8.7060907@ee.byu.edu>
Message-ID: <db6b5ecc0611020807y33361015p2959afdaeee020cb@mail.gmail.com>

On 11/2/06, Travis Oliphant <oliphant at ee.byu.edu> wrote:

> Thanks for the pre-amble.  Does it require pdflatex?  I use ps2pdf
> because to generate the shaded boxes and graphics.  I could probably try
> to do it with pdflatex and png files but I haven't tried, yet.

I just tested it with ps2pdf on a document, and it worked fine.  If
you have problems, play with commenting (or not) the first option line
to hyperref:

\usepackage[
  %pdftex,  % needed for pdflatex


In the copy I gave you it's commented out, so it /should/ work for
ps2pdf.  But I've had documents where I've had to uncomment it, and I
think at that point they stop working with ps2pdf, and yes, you lose
the usage of PostScript specials in that case (which I use for things
like whole-page DRAFT watermarks).

Cheers,

f

-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From tim.hochberg at ieee.org  Thu Nov  2 11:15:00 2006
From: tim.hochberg at ieee.org (Tim Hochberg)
Date: Thu, 02 Nov 2006 09:15:00 -0700
Subject: Converting bool to float
In-Reply-To: <454A135F.2040009@ee.byu.edu>
References: <f4f93d420611011233i60c8128ew63288d5620477c9c@mail.gmail.com>	<454916EC.6010003@ee.byu.edu>	<eib580$qdt$1@sea.gmane.org>	<45492339.6050302@ee.byu.edu>	<4549632B.4010001@ieee.org>	<eibv18$sj3$1@sea.gmane.org>
	<454A135F.2040009@ee.byu.edu>
Message-ID: <454A1984.7030205@ieee.org>

Travis Oliphant wrote:
> Robert Kern wrote:
>
>   
>> Tim Hochberg wrote:
>>  
>>
>>     
>>> Travis Oliphant wrote:
>>>    
>>>
>>>       
>>>> Robert Kern wrote:
>>>>      
>>>>
>>>>         
>>>>> Travis Oliphant wrote:   
>>>>>        
>>>>>
>>>>>           
>>>>>> It looks like 1.0-x is doing the right thing.
>>>>>>
>>>>>> The problem is 1.0*x for matrices is going to float64.  For arrays it 
>>>>>> returns float32 just like the 1.0-x
>>>>>>      
>>>>>>          
>>>>>>
>>>>>>             
>>>>> Why is this the right thing? Python floats are float64.
>>>>>
>>>>>        
>>>>>
>>>>>           
>>>> Yeah, why indeed.  Must be something with the scalar coercion code...
>>>>      
>>>>
>>>>         
>>> This is one of those things that pops up every few years. I suspect that 
>>> the best thing to do here is to treat 1.0, and all Python floats as 
>>> having a kind (float), but no precision. Or, equivalently treat them as 
>>> the smallest precision floating point value. The rationale behind this 
>>> is that otherwise float32 array will be promoted whenever they are 
>>> multiplied by Python floating point scalars. If Python floats are 
>>> treated as Float64 for purposes of determining output precision then 
>>> anyone using float32 arrays is going to have to wrap all of their 
>>> literals in float32 to prevent inadvertent upcasting to float64. This 
>>> was the origin of the (rather clunky) numarray spacesaver flag.
>>>
>>> It's no skin off my nose either way, since I pretty much never use 
>>> float32, but I suspect that treating python floats equivalently to 
>>> float64 scalars would be a mistake. At the very least it deserves a bit 
>>> of discussion.
>>>    
>>>
>>>       
>> Well, they *are* 64-bit floating point numbers. You simply can't get around 
>> that. That's why we now have all of the scalar types: you can get any precision 
>> scalars that you want as long as you are explicit about it (and explicit is 
>> better than implicit). The spacesaver flag was the only solution before the 
>> various scalar types existed. I'd like to suggest that the discussion already 
>> occurred some time ago and has concluded in favor of the scalar types. 
>> Downcasting should be explicit.
>>
>> However, whether or not float32 arrays operated with Python float scalars give 
>> float32 or float64 arrays is tangential to my question. Does anyone actually 
>> think that a Python float operated with a boolean array should give a float32 
>> result? Must we *up*cast a boolean array to float64 to preserve the precision of 
>> the scalar?
>>
>>  
>>
>>     
> The first basic rule is that scalars don't control the precision of the 
> output when doing mixed-type calculations *except* when they are of a 
> fundamentally different kind. 
>
> Then (if a different kind of scalar is used), the rule is that the 
> arrays will be upcast to the "lowest" precision in the group to preserve 
> overall precision.   So, when a bool is combined with a "float" kind of 
> scalar, the result is float32 because that preserves precision of the 
> bool.  Remember it is array precision that takes precedence over scalars 
> in mixed type array-scalar operations.
>
> This is the rule.  I agree that this rule is probably flawed in certain 
> circumstances.
>   
I think any rule will be flawed in certain circumstances. This 
particular rule has the advantage of being relatively straightforward 
and the circumstances that I can think of where it could cause problems 
are relatively limited and relatively easy to address. The obvious 
"fixes" to this rule that I  have I've thought of all have problems that 
are as bad or worse as the current rule and have the added disadvantage 
of being more complicated.

At the very least, any replacement rule should get some serious 
discussion here before being implemented. We should particularly solicit 
the input of numarray users since that package had more infrastructure 
in place to support the use of lower precision arrays.

> So, what should be done about it at this point?  Do you think a change 
> is acceptable for 1.0.1 or does it need to wait a year until 1.1?
>   
Unless someone can come up with a convincingly better solution, I say 
leave things as is indefinitely.

-tim



-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From george.sakkis at gmail.com  Thu Nov  2 11:18:25 2006
From: george.sakkis at gmail.com (George Sakkis)
Date: Thu, 02 Nov 2006 16:18:25 -0000
Subject: Reading records from file and sorting
References: <1162337923.747767.323100@b28g2000cwb.googlegroups.com>
	<20061101224620.GA29602@dogbert.sdsl.sun.ac.za>
	<1162442534.508320.151770@e3g2000cwe.googlegroups.com>
	<e06186140611012052u16743231t9a508338f883b4@mail.gmail.com>
Message-ID: <1162484305.382176.128300@m7g2000cwm.googlegroups.com>

Charles R Harris wrote:

> On 11/1/06, George Sakkis <george.sakkis at gmail.com> wrote:
> >
> > Albert Strasheim wrote:
> >
> > > Check the thread "Strange results when sorting array with fields" from
> > > about a week back. Travis made some changes to sorting in the presence
> > > of fields that should solve your problem, assuming your fields appear in
> > > the order you want to sort (i.e. you want to sort f1, f2, f3 and not
> > > something like f1, f3, f2).
> >
> > I'm afraid this won't help in my case; I want to sort twice, once by f1
> > and once by f2. I guess I could make a second file with the fields
> > swapped but this seems more messy and inefficient than Francesc's
> > suggestion.
>
>
> Do you actually want the two different orders, or do you want to sort on the
> first field, then sort all the items with the same first field on the second
> field?
 
The former.

George


-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From oliphant at ee.byu.edu  Thu Nov  2 11:32:51 2006
From: oliphant at ee.byu.edu (Travis Oliphant)
Date: Thu, 02 Nov 2006 09:32:51 -0700
Subject: Reading records from file and sorting
In-Reply-To: <1162484305.382176.128300@m7g2000cwm.googlegroups.com>
References: <1162337923.747767.323100@b28g2000cwb.googlegroups.com>	<20061101224620.GA29602@dogbert.sdsl.sun.ac.za>	<1162442534.508320.151770@e3g2000cwe.googlegroups.com>	<e06186140611012052u16743231t9a508338f883b4@mail.gmail.com>
	<1162484305.382176.128300@m7g2000cwm.googlegroups.com>
Message-ID: <454A1DB3.4090002@ee.byu.edu>

George Sakkis wrote:

>Charles R Harris wrote:
>
>  
>
>>On 11/1/06, George Sakkis <george.sakkis at gmail.com> wrote:
>>    
>>
>>>Albert Strasheim wrote:
>>>
>>>      
>>>
>>>>Check the thread "Strange results when sorting array with fields" from
>>>>about a week back. Travis made some changes to sorting in the presence
>>>>of fields that should solve your problem, assuming your fields appear in
>>>>the order you want to sort (i.e. you want to sort f1, f2, f3 and not
>>>>something like f1, f3, f2).
>>>>        
>>>>
>>>I'm afraid this won't help in my case; I want to sort twice, once by f1
>>>and once by f2. I guess I could make a second file with the fields
>>>swapped but this seems more messy and inefficient than Francesc's
>>>suggestion.
>>>      
>>>
>>Do you actually want the two different orders, or do you want to sort on the
>>first field, then sort all the items with the same first field on the second
>>field?
>>    
>>
> 
>The former.
>  
>
Sorting on a particular field in-place would be possible if there were 
some-way to indicate to VOID_compare the field-order you wanted to use 
to compare on.

There are a few ways I could think of doing this.

1) Use a  thread-specific global variable (this doesn't recurse very 
easily).
2) Use the meta-object in the field specifier to indicate the order (the 
interface could still be something like
.sort(order='f1')  and a temporary data-type object is created and used).
3) Use a special key in the fields dictionary although this would 
require some fixes to all the code that cycles through the fields 
dictionary to recurse on structures.

4) Overload any of the other variables in the PyArray_Descr * structure.

5) Add a sort-order to the end of the PyArray_Descr * structure and a 
flag to the hasobject flag bits (that would be the last one available) 
that states that the Data-type object has the sort-order defined (so 
binary compatibilty is retained but the new feature can be used going 
forward).

Any other ideas?

-Travis




-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From drife at ucar.edu  Thu Nov  2 11:54:43 2006
From: drife at ucar.edu (Daran Rife)
Date: Thu, 2 Nov 2006 09:54:43 -0700
Subject: numpy book
Message-ID: <10D7CE3B-1A80-4B02-B9A3-A9BFCCC250E2@ucar.edu>

Hi Travis,

I have one question about the fee-based "Guide to NumPy"
that doesn't seem to be addressed on the Web site:

    <http://www.tramy.us/guidetoscipy.html>

or in the FAQ.

Once a user purchases the document, how does s/he obtain
the latest version, when it becomes available?

Looking forward to purchasing a copy, to show support for
NumPy--one of the best (free) scientific computing packages
available.


Thanks for fielding my question,


Daran

--


From oliphant at ee.byu.edu  Thu Nov  2 10:41:11 2006
From: oliphant at ee.byu.edu (Travis Oliphant)
Date: Thu, 02 Nov 2006 08:41:11 -0700
Subject: [Numpy-discussion] numpy book
Message-ID: <454A1197.1020606@ee.byu.edu>



> Note that this is not a request to Travis to send me the latest  
> version
> by private email. That would be inefficient and my need is not that
> urgent. Nevertheless I think that issue should be settled.
>
>
>
There will be an update, soon. I'm currently working on the index,
corrections, and formatting issues.

The update will be sent in conjunction with the release of 1.0.1 which I
am targetting in 2 weeks.

Thanks for everybody's patience.  If you need an update now, I'm happy
to send it to you by private email.

Best regards,

-Travis O.

-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From aisaac at american.edu  Thu Nov  2 12:13:34 2006
From: aisaac at american.edu (Alan G Isaac)
Date: Thu, 2 Nov 2006 12:13:34 -0500
Subject: numpy book
In-Reply-To: <10D7CE3B-1A80-4B02-B9A3-A9BFCCC250E2@ucar.edu>
References: <10D7CE3B-1A80-4B02-B9A3-A9BFCCC250E2@ucar.edu>
Message-ID: <Mahogany-0.67.0-1408-20061102-121334.00@american.edu>

On Thu, 2 Nov 2006, Daran Rife apparently wrote: 
> Once a user purchases the document, how does s/he obtain 
> the latest version, when it becomes available? 

In addition to providing updates on an "as requested" basis,
Travis has made it clear that he will occasionally do a mass
mailing.  (At least it seems clear to me...)

I have a different suggestion for Travis:
I suggest each purchaser receive an ID which can be used for 
a download twice a year.  That will put these enquiries to 
rest.


> Looking forward to purchasing a copy, to show support for 
> NumPy--one of the best (free) scientific computing 
> packages available. 

It is also a very nice book.  If you had to repurchase it 
once every year or two, which you do not, it would still be 
a good deal.  So don't hesitate!  And make sure your library
and labs buy copies too!

Cheers,
Alan Isaac



-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From fperez.net at gmail.com  Thu Nov  2 14:07:42 2006
From: fperez.net at gmail.com (Fernando Perez)
Date: Thu, 2 Nov 2006 12:07:42 -0700
Subject: Strange and hard to reproduce crash
In-Reply-To: <453D4AD1.5060508@ieee.org>
References: <db6b5ecc0610231439o1217af98j11dcb45e2ae5e505@mail.gmail.com>
	<453D3DD7.50506@ieee.org>
	<db6b5ecc0610231544g7413491du9e8b0bd6c8c20e6f@mail.gmail.com>
	<453D4AD1.5060508@ieee.org>
Message-ID: <db6b5ecc0611021107h3fed690dycbd0c608728b3b95@mail.gmail.com>

On 10/23/06, Travis Oliphant <oliphant.travis at ieee.org> wrote:

> I've placed them in SVN (r3384):
>
> arraydescr_dealloc needs to do something like.
>
> if (self->fields == Py_None) {
>     print something
>     incref(self)
>     return;
> }
>
> Most likely there is a missing Py_INCREF() before some call that uses
> the data-type object (and consumes it's reference count) --- do you have
> any Pyrex code (it's harder to get it right with Pyrex).

OK, we've completed another long run (several days), and this time it
didn't crash.  But I think there are still refcount problems.  I'm
attaching the full log file and a plot of the refcount.  It's wrapping
around, and after some point the increase switches to a pefectly
linear pattern, I'm not exactly sure why (it could be a change in the
underlying code after the initialization phase; it's not my code so I
don't know its internals).

I hope this helps, it would be nice to track this down before 1.0.1 is out.

Cheers,

f
-------------- next part --------------
A non-text attachment was scrubbed...
Name: refcounts.png
Type: image/png
Size: 27895 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20061102/247f3033/attachment-0001.png>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: numpy_report.gz
Type: application/x-gzip
Size: 25416 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20061102/247f3033/attachment-0001.bin>
-------------- next part --------------
-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
-------------- next part --------------
_______________________________________________
Numpy-discussion mailing list
Numpy-discussion at lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/numpy-discussion

From torgil.svensson at gmail.com  Thu Nov  2 14:32:44 2006
From: torgil.svensson at gmail.com (Torgil Svensson)
Date: Thu, 2 Nov 2006 20:32:44 +0100
Subject: numpy book
In-Reply-To: <db6b5ecc0611020800u367eb6fdvccf194c6e1aecd14@mail.gmail.com>
References: <4549E34A.4060507@gmx.net> <454A1197.1020606@ee.byu.edu>
	<db6b5ecc0611020800u367eb6fdvccf194c6e1aecd14@mail.gmail.com>
Message-ID: <e7bda7770611021132y60c9a6abta182ed16e202cb85@mail.gmail.com>

> generating such a PDF, with some color and formatting tweaks so that
> it prints legibly on black and white as well as looking nice on
> screen.

This sounds promising. I actually have had problems printing the
guide, every information box showed up completely black except for the
emblem.

On 11/2/06, Fernando Perez <fperez.net at gmail.com> wrote:
> On 11/2/06, Travis Oliphant <oliphant at ee.byu.edu> wrote:
> >
> > >Note that this is not a request to Travis to send me the latest version
> > >by private email. That would be inefficient and my need is not that
> > >urgent. Nevertheless I think that issue should be settled.
> > >
> > >
> > There will be an update, soon. I'm currently working on the index,
> > corrections, and formatting issues.
>
> If I may make a suggestion, it would greatly increase the usability of
> the PDF if it had internal and external clickable links, as well as a
> PDF table of contents (the one that appears on the left sidebar of
> acroread).
>
> Since I know the original is in Lyx, here's my standard preamble for
> these kinds of documents, which includes all the goodies for
> generating such a PDF, with some color and formatting tweaks so that
> it prints legibly on black and white as well as looking nice on
> screen.  If you add this to your preamble, a simple 'view->pdflatex'
> should work.  I'll be happy to help if you have any issues.
>
> Cheers,
>
> f
>
> %%% My preamble; tweak and season to taste:
>
> % This gives us a better font in URL links (otherwise the default
> % MonoSpace font is bitmapped, and it looks horrible in PDF)
> \usepackage{courier}
>
> \usepackage{fullpage}
>
> \usepackage{color}  % so we can use red for the fixme warnings
>
> % The hyperref package gives us a pdf with properly built
> % internal navigation ('pdf bookmarks' for the table of contents,
> % internal cross-reference links, web links for URLs, etc.)
>
> % A few colors to replace the defaults for certain link types
> \definecolor{darkorange}{rgb}{.71,0.21,0.01}
> \definecolor{darkgreen}{rgb}{.12,.54,.11}
>
> \usepackage[
>   %pdftex,  % needed for pdflatex
>   breaklinks=true,  % so long urls are correctly broken across lines
>   colorlinks=true,
>   urlcolor=blue,
>   linkcolor=darkorange,
>   citecolor=darkgreen,
>   ]{hyperref}
>
> % This helps prevent overly long lines that stretch beyond the margins
> \sloppy
>
> % Define a \fixme command to mark visually things needing fixing in the draft.
> % For final printing or to simply disable these bright warnings, simply
> % uncomment the \renewcommand redefinition below
>
> \newcommand{\fixme}[1] {
> \textcolor{red}{
> {\fbox{ {\bf FIX}
> \ensuremath{\blacktriangleright \blacktriangleright \blacktriangleright}}
> {\bf #1}
> \fbox{\ensuremath{ \blacktriangleleft \blacktriangleleft \blacktriangleleft }
> } } }
> }
> % Uncomment the next line to make the \fixme command be a no-op
> %\renewcommand{\fixme}[1]{}
>
> %%% If you also want to use the listings package for nicely formatted
> %%% Python source code, this configuration produces good on-paper and
> %%% on-screen results:
>
> \definecolor{orange}{cmyk}{0,0.4,0.8,0.2}
> % Use and configure listings package for nicely formatted code
> \usepackage{listings}
> \lstset{
>   language=Python,
>   basicstyle=\small\ttfamily,
>   commentstyle=\ttfamily\color{blue},
>   stringstyle=\ttfamily\color{orange},
>   showstringspaces=false,
>   breaklines=true,
>   postbreak = \space\dots
> }
>
> -------------------------------------------------------------------------
> Using Tomcat but need to do more? Need to support web services, security?
> Get stuff done quickly with pre-integrated technology to make your job easier
> Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
> http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
> _______________________________________________
> Numpy-discussion mailing list
> Numpy-discussion at lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/numpy-discussion
>

-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From faltet at carabos.com  Thu Nov  2 14:26:25 2006
From: faltet at carabos.com (Francesc Altet)
Date: Thu, 02 Nov 2006 20:26:25 +0100
Subject: numarray indexing problems with Python2.5 and
	64-bit platforms
Message-ID: <1162495586.3960.19.camel@localhost.localdomain>

Hi,

I've detected that numarray (1.5.2) seems to be bitten by the change in
Python 2.5 for indexes (http://docs.python.org/whatsnew/pep-353.html). 
In a Linux64 machine (using Python 2.5), I get this:

>>> a=numarray.array([1,2,3])
>>> a[1:2]
array([1, 2])  # !

However, both Numeric and numpy seems to work well with the same
scenario.

>>> b=Numeric.array([1,2,3])
>>> b[1:2]
array([2])
>>> c=numpy.array([1,2])
>>> c[1:2]
array([2])

I see this as a major issue in numarray and poses in great danger the
intended support of PyTables for numarray that we planned for some time
(until end of 2007). It would be nice to know if the numarray crew would
be willing to address this, or, now that NumPy 1.0 is out, they have
decided to completely drop the support for it.

We'd really like to continue offering support for numarray (in the end,
it is a very good piece of software) in PyTables, but don't having a
solution for this problem anytime soon, will make this very problematic
to us.

Thanks,

-- 
Francesc Altet    |  Be careful about using the following code --
Carabos Coop. V.  |  I've only proven that it works, 
www.carabos.com   |  I haven't tested it. -- Donald Knuth


-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From NPadmanabhan at lbl.gov  Thu Nov  2 15:35:33 2006
From: NPadmanabhan at lbl.gov (Nikhil Padmanabhan)
Date: Thu, 2 Nov 2006 12:35:33 -0800
Subject: Segfault in Python 2.5
Message-ID: <FAFFEFE2-07AF-4D62-9B23-A1BDD07659C7@lbl.gov>

Hi,

I recently tried to upgrade to Python 2.5 (MacPython), and I am  
getting segfaults on some of my f2py wrapped fortran codes. I'm  
running OS X 10.4.8 (ppc), and this occurs for both numpy 1.0 and  
1.0rc2 (the two versions I tried). Running with Python 2.4 does not  
give me this error. This is using the IBM xlf compiler....

A simple test case is below.....

Not sure where to go from here -- any help would be great!
Thanks,

Here is a stripped down version that still crashes :

subroutine foo(xx, nxx, bj)
   implicit none
   integer nxx
   real*8 xx(nxx),bj(nxx)
!f2py intent(out) bj

   bj = xx

end subroutine foo

Compiling this to foo.so, I get :
Python 2.5 (r25:51918, Sep 19 2006, 08:49:13)
[GCC 4.0.1 (Apple Computer, Inc. build 5341)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
 >>> from numpy import *
 >>> import foo as p
 >>> a = arange(10)
 >>> p.foo(a)
Segmentation fault

Intriguingly, swapping the import orders fixes this :
Python 2.5 (r25:51918, Sep 19 2006, 08:49:13)
[GCC 4.0.1 (Apple Computer, Inc. build 5341)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
 >>> import foo as p
 >>> from numpy import *
 >>> a = arange(10)
 >>> p.foo(a)
array([ 0.,  1.,  2.,  3.,  4.,  5.,  6.,  7.,  8.,  9.])
 >>>



------------------------------------
Nikhil Padmanabhan
NPadmanabhan at lbl.gov
nikhil@(510) 495-2943




-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From jmiller at stsci.edu  Thu Nov  2 16:04:43 2006
From: jmiller at stsci.edu (Todd Miller)
Date: Thu, 02 Nov 2006 16:04:43 -0500
Subject: numarray indexing problems with Python2.5
 and	64-bit platforms
In-Reply-To: <1162495586.3960.19.camel@localhost.localdomain>
References: <1162495586.3960.19.camel@localhost.localdomain>
Message-ID: <454A5D6B.4020505@stsci.edu>

Here's a stab at a solution but I don't have easy access to 64-bit linux 
at the moment so it is untested.   If someone is willing to test it 
(and/or fix it) I'll add it to the next numarray release.  It should be 
noted that numarray is not 64-bit enabled (it compiles as a 32-bit 
program in terms of arrays) which is another motivation to switch to numpy.

Regards,
Todd

Francesc Altet wrote:
> Hi,
>
> I've detected that numarray (1.5.2) seems to be bitten by the change in
> Python 2.5 for indexes (http://docs.python.org/whatsnew/pep-353.html). 
> In a Linux64 machine (using Python 2.5), I get this:
>
>   
>>>> a=numarray.array([1,2,3])
>>>> a[1:2]
>>>>         
> array([1, 2])  # !
>
> However, both Numeric and numpy seems to work well with the same
> scenario.
>
>   
>>>> b=Numeric.array([1,2,3])
>>>> b[1:2]
>>>>         
> array([2])
>   
>>>> c=numpy.array([1,2])
>>>> c[1:2]
>>>>         
> array([2])
>
> I see this as a major issue in numarray and poses in great danger the
> intended support of PyTables for numarray that we planned for some time
> (until end of 2007). It would be nice to know if the numarray crew would
> be willing to address this, or, now that NumPy 1.0 is out, they have
> decided to completely drop the support for it.
>
> We'd really like to continue offering support for numarray (in the end,
> it is a very good piece of software) in PyTables, but don't having a
> solution for this problem anytime soon, will make this very problematic
> to us.
>
> Thanks,
>
>   

-------------- next part --------------
An embedded and charset-unspecified text was scrubbed...
Name: slice64.patch
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20061102/80bd33c4/attachment-0001.ksh>
-------------- next part --------------
-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
-------------- next part --------------
_______________________________________________
Numpy-discussion mailing list
Numpy-discussion at lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/numpy-discussion

From peridot.faceted at gmail.com  Thu Nov  2 16:26:07 2006
From: peridot.faceted at gmail.com (A. M. Archibald)
Date: Thu, 2 Nov 2006 16:26:07 -0500
Subject: numarray indexing problems with Python2.5
	and 64-bit platforms
In-Reply-To: <1162495586.3960.19.camel@localhost.localdomain>
References: <1162495586.3960.19.camel@localhost.localdomain>
Message-ID: <ce557a360611021326o4205fbdg5709859f57d02d2e@mail.gmail.com>

On 02/11/06, Francesc Altet <faltet at carabos.com> wrote:

> I see this as a major issue in numarray and poses in great danger the
> intended support of PyTables for numarray that we planned for some time
> (until end of 2007). It would be nice to know if the numarray crew would
> be willing to address this, or, now that NumPy 1.0 is out, they have
> decided to completely drop the support for it.
>
> We'd really like to continue offering support for numarray (in the end,
> it is a very good piece of software) in PyTables, but don't having a
> solution for this problem anytime soon, will make this very problematic
> to us.

Someone has to say it: you could just drop support for the obsolete
numarray and provide only support for its successor, numpy.

A. M. Archibald

-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From sz13544007750 at 163.com  Thu Nov  2 19:48:53 2006
From: sz13544007750 at 163.com (=?GB2312?B?xvPStdPQwPs=?=)
Date: Fri, 3 Nov 2006 08:48:53 +0800
Subject: =?GB2312?B?xvPStdPQwPs=?=
Message-ID: <E1GfnEx-0003wS-Q0@mail.sourceforge.net>

???/????
??         ??! 
?????????????????????????????????????
?????????????????????????????????????0.8-2%??????6%-8%???????2-4%????????????!
??????????????????????????????????????????????????????????????????????????????????????????????????????????
?????????????????????????????????????????????????????????????
?????????????????????????????????
?????? ???  13544007750  
E-mail? jason-7750 at 163.com
??????????????????

-------------- next part --------------
-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
-------------- next part --------------
_______________________________________________
Numpy-discussion mailing list
Numpy-discussion at lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/numpy-discussion

From wsa532 at 163.com  Thu Nov  2 22:41:58 2006
From: wsa532 at 163.com (=?GB2312?B?IjEx1MIxMS0xMsjVL8nPuqMi?=)
Date: Fri, 3 Nov 2006 03:41:58 +0000 (UTC)
Subject: =?GB2312?B?t8eyxs7xvq3A7bXEssbO8bncwO0tybPFzMSjxOI=?=
Message-ID: <E1GfpwC-0008OV-OQ@mail.sourceforge.net>

An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20061103/5efabdcc/attachment-0001.html>
-------------- next part --------------
-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
-------------- next part --------------
_______________________________________________
Numpy-discussion mailing list
Numpy-discussion at lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/numpy-discussion

From mhkcptinw at dgcsystems.net  Thu Nov  2 23:39:38 2006
From: mhkcptinw at dgcsystems.net (cannot connection)
Date: Fri, 3 Nov 2006 05:39:38 +0100
Subject: hearts.MY tammy rowdy
Message-ID: <001301c6ff02$12529fa0$98dfb53e@laxp1>


Interstate Highways numbers or.
German rockers who big pals with? Openwfe convert in pdf a svg. Top Larrys work enjoying board is.
Openwfe convert in pdf a svg. Laudio Jxxyy Kxxyy Lxxyy!
Style pagesmost comply mos rarely is directly pages. Ends needs am generated specify location am.
Video Converter Diablo in? Mckerchar doing while customers or.
Enoughthe a awesome milestone worthy White in checks of Lift of. Explore paranormal phenomena evp?
Winners correct answer is Edgar Allan poe Sign of hsw of Newsletter. Theres chance is itself perform.
Continuous is Xmldecoder Sergey Malenkov in start. Comparison readily erased Firmware.
Interstate Highways numbers or. Moog am intro songs. Wing episode in Odyssey Ncaa is Tournament?
Todayi loved forand sense peacethat or.
Xmltask developed mechanism am. Them Basics Nuclear Bombs is.
There in discover in iterate keys in isnt returning values am yet or dealloc of.
Fantasy Lineage Middle in Earth Nights Galaxies.  Difficult reduce is what.
Graph memory thisobject translated of calls! Agreement bound Oreilly Collabnet. Profile rackmount of chassis accepts in?
Edgar a Allan poe Sign. Agreement bound Oreilly Collabnet. Page pleased Mobility Pack fcs a Profiler Fcscc Beta?
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20061103/c49079a5/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: uses:.gif
Type: image/gif
Size: 9519 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20061103/c49079a5/attachment-0001.gif>
-------------- next part --------------
-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
-------------- next part --------------
_______________________________________________
Numpy-discussion mailing list
Numpy-discussion at lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/numpy-discussion

From torgil.svensson at gmail.com  Fri Nov  3 01:38:29 2006
From: torgil.svensson at gmail.com (Torgil Svensson)
Date: Fri, 3 Nov 2006 07:38:29 +0100
Subject: Need more comments from scientific community
	on python-dev
In-Reply-To: <e86a5fd00611011019p5a39553cwf55fd06e585e58e0@mail.gmail.com>
References: <45474841.9030802@ieee.org>
	<d38f5330610311210i5141b495s5bb73f8da6484629@mail.gmail.com>
	<4547B235.6050904@ee.byu.edu>
	<d38f5330610311336w7538e0b2v994ee277da64f6cf@mail.gmail.com>
	<4547D2B3.5000808@ee.byu.edu>
	<d38f5330610311551u4e71cdd7ndc0b93de16dc6d80@mail.gmail.com>
	<4548C3D6.40100@ieee.org>
	<ce557a360611011013v4e883251l46dc6e2c0c85b854@mail.gmail.com>
	<e86a5fd00611011019p5a39553cwf55fd06e585e58e0@mail.gmail.com>
Message-ID: <e7bda7770611022238y2cb15e75s3739d2b4b652129c@mail.gmail.com>

> > It seems to me that at the python level, there's not much reason to
> > choose the dtypes proposal over ctypes.

I disagree. I can see a point in unifying ctypes and dtypes but in my
mindset they've got two different scopes.

ctypes is an interface to the c language
dtype is a data description for the python language

The biggest advantages python has over c in my mind (except the script
vs compile) is that it's dynamic and expressive. I think basing dtype
on ctypes is in that mind is simply destructive and against what I
like best with python.

ctypes mimics c's static behaviour:

class struct_type(Structure):
    _fields_ = [....]

...

... which is somewhat static in nature. How do you create "structures"
dynamically?

How do you do things like this in ctypes:

mytype=filter(lambda x: is_little_endian(x),map(lambda x: x==int32 and
int64 or x, old_type))
b=['gg',[1,2,3]]
mytype=dtype([{str: uint64 , list:another_type }[type(x)] for x in b])

I'm sure it's possibl but I'm also suspecting it not at all as clean
and expressive (my second argument for choosing python over c).

Since we creating something new in python isn't it best we do
something the python way and later see if we can adapt the c structure
to that rather than the other way around?  If we can't adapt ctypes to
this new thing i'm perfectly happy with leaving ctypes as is.

ctype is doing a great job on the c interaction level but is plain
ugly on the python level.


On 11/1/06, Bill Baxter <wbaxter at gmail.com> wrote:
>
> On 11/2/06, A. M. Archibald <peridot.faceted at gmail.com> wrote:
> > On 01/11/06, Travis Oliphant <oliphant.travis at ieee.org> wrote:
> >
> > > And it may be a good idea to also have a get_ctype method or some-such
> > > on the ctypes attribute so that  one could get a "ctypes description"
> > > from the NumPy data-type.
> >
> > It seems to me that at the python level, there's not much reason to
> > choose the dtypes proposal over ctypes. There is one at the C level,
> > it seems (though I, like perhaps most of the people on python-dev,
> > have never actually tried using either). So perhaps, to persuade the
> > python-dev folks, what is needed is a comparison of what has to be
> > done at the C level. What would it take to rewrite numpy to use
> > ctypes? There seems to be some problem with extending the type objects
> > used by ctypes, but it's not very clear to me what that problem is
> > (what the extensions are supposed to do).
>
>
> I posted a message to the thread trying to prod things in that direction.
> I.e. can we see a simple concrete example of the complications involved in
> using ctypes interface code, vs the presumably much nicer
> numpy/data-descriptor code.  I too think that would help convince people.
> Just saying it's more compilcated, trust me, doesn't help when most people
> reading the list have never had to write any sort of C extension.
>
> --bb
>
>
> -------------------------------------------------------------------------
> Using Tomcat but need to do more? Need to support web services, security?
> Get stuff done quickly with pre-integrated technology to make your job
> easier
> Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
> http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
>
> _______________________________________________
> Numpy-discussion mailing list
> Numpy-discussion at lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/numpy-discussion
>
>
>

-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From Mailer-Daemon at lists.sourceforge.net  Fri Nov  3 07:11:56 2006
From: Mailer-Daemon at lists.sourceforge.net (Mail Delivery System)
Date: Fri, 03 Nov 2006 04:11:56 -0800
Subject: Warning: message 1GfAoO-0007im-Sh delayed 48
	hours
Message-ID: <E1Gfxts-0003oY-44@externalmx-1.sourceforge.net>

This message was created automatically by mail delivery software.
A message that you sent has not yet been delivered to one or more of its
recipients after more than 48 hours on the queue on externalmx-1.sourceforge.net.

The message identifier is:     1GfAoO-0007im-Sh
The subject of the message is: looked like me the full sized HARLEY and began to her, with this but as
The date of the message is:    Wed, 01 Nov 2006 15:46:73 +0800

The address to which the message has not yet been delivered is:

  numpy-discussion at lists.sourceforge.net
    Delay reason: SMTP error from remote mailer after RCPT TO:<numpy-discussion at lists.sourceforge.net>:
    host mail.sourceforge.net [66.35.250.206]: 451-Could not complete sender verify callout
    451-Could not complete sender verify callout for
    451-<numpy-discussion at lists.sourceforge.net>.
    451-The mail server(s) for the domain may be temporarily unreachable, or
    451-they may be permanently unreachable from this server. In the latter case,
    451-you need to change the address or create an MX record for its domain
    451-if it is 

No action is required on your part. Delivery attempts will continue for
some time, and this warning may be repeated at intervals if the message
remains undelivered. Eventually the mail delivery software will give up,
and when that happens, the message will be returned to you.

-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From faltet at carabos.com  Fri Nov  3 10:00:10 2006
From: faltet at carabos.com (Francesc Altet)
Date: Fri, 3 Nov 2006 16:00:10 +0100
Subject: =?utf-8?q?numarray_indexing_problems_with_Pyth?=
	=?utf-8?q?on2=2E5_and=0964-bit_platforms?=
In-Reply-To: <454A5D6B.4020505@stsci.edu>
References: <1162495586.3960.19.camel@localhost.localdomain>
	<454A5D6B.4020505@stsci.edu>
Message-ID: <200611031600.10980.faltet@carabos.com>

Hi Todd,

A Dijous 02 Novembre 2006 22:04, Todd Miller escrigu?:
> Here's a stab at a solution but I don't have easy access to 64-bit linux
> at the moment so it is untested.   If someone is willing to test it
> (and/or fix it) I'll add it to the next numarray release.  It should be
> noted that numarray is not 64-bit enabled (it compiles as a 32-bit
> program in terms of arrays) which is another motivation to switch to numpy.

Many thanks for the patch. The good news is that it seems to fix the
main issue:

>>> a=numarray.array([1,2,3])
>>> a[1:2]
array([2])

and moreover, all the tests in PyTables (with numarray at its core) do
pass in 64-bit platforms and Python 2.5 run ok (except for the known
bug that also happens in 32-bit platforms; see [1]).

The bad news is that the numarray tests seems more finicky than
PyTables ones and they are throwing a "segmentation fault":

>>> import numarray.testall as testall
>>> testall.test()
Testing numarray 1.5.2 on normal Python (2, 5, 0, 'final', 0) on platform 
linux2 with builtin linear algebra support
Violaci?n de segmento

I've run this through valgrind, and here it is the offending line:

unhandled opc_aux = 0x 7
first_opcode == 0xDD
vex amd64->IR: unhandled instruction bytes: 0xDD 0x7C 0x24 0xFC
==29710==
==29710== Process terminating with default action of signal 4 (SIGILL)
==29710==  Illegal opcode at address 0x11F603A0
==29710==    at 0x11F603A0: fetestexcept (in /lib64/tls/libm-2.3.5.so)
==29710==    by 0x1272E21D: NA_clearFPErrors (libnumarraymodule.c:1949)
==29710==    by 0x13465311: _Py_cum_exec (_ufuncmodule.c:998)
==29710==    by 0x1346562C: _cum_cached (_ufuncmodule.c:1087)
==29710==    by 0x134659E7: _cum_swapped (_ufuncmodule.c:1199)
==29710==    by 0x13465ED0: _Py_reduce (_ufuncmodule.c:1374)
==29710==    by 0x483166: PyEval_EvalFrameEx (ceval.c:3566)
==29710==    by 0x4845B0: PyEval_EvalCodeEx (ceval.c:2833)
==29710==    by 0x48331D: PyEval_EvalFrameEx (ceval.c:3662)
==29710==    by 0x4845B0: PyEval_EvalCodeEx (ceval.c:2833)
==29710==    by 0x48331D: PyEval_EvalFrameEx (ceval.c:3662)
==29710==    by 0x4845B0: PyEval_EvalCodeEx (ceval.c:2833)
==29710==    by 0x48331D: PyEval_EvalFrameEx (ceval.c:3662)
==29710==    by 0x4845B0: PyEval_EvalCodeEx (ceval.c:2833)
==29710==    by 0x4CF0D3: function_call (funcobject.c:517)
==29710==    by 0x415C32: PyObject_Call (abstract.c:1860)
==29710==    by 0x41D0A2: instancemethod_call (classobject.c:2493)
==29710==    by 0x415C32: PyObject_Call (abstract.c:1860)
==29710==    by 0x47C660: PyEval_CallObjectWithKeywords (ceval.c:3435)
==29710==    by 0x454823: slot_tp_repr (typeobject.c:4504)

it seems to my inexpert eyes that the compiler has generated some code
that it is not understood by the amd64 architecture (?).

[1] 
<http://sourceforge.net/tracker/index.php?func=detail&aid=1565683&group_id=1369&atid=450446>

Regards,

-- 
>0,0<   Francesc Altet ? ? http://www.carabos.com/
V   V   C?rabos Coop. V. ??Enjoy Data
 "-"



-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
_______________________________________________
Numpy-discussion mailing list
Numpy-discussion at lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/numpy-discussion

From numpy-discussion at mlists.thewrittenword.com  Fri Nov  3 10:09:43 2006
From: numpy-discussion at mlists.thewrittenword.com (Albert Chin)
Date: Fri, 3 Nov 2006 09:09:43 -0600
Subject: Setting runtime_library_dirs while building numpy
Message-ID: <20061103150943.GA35289@mail1.thewrittenword.com>

We have atlas-3.6.0 installed in a non-standard location. When
building the numpy-1.0 shared libraries, we'd like to encode the
non-standard location in the RPATH of the executable. We do this with
other Python modules by adding the following to setup.cfg:
  [build_ext]
  rpath=<path to library>

But, this doesn't work for numpy-1.0. Is there another way to do this?

-- 
albert chin (china at thewrittenword.com)

-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From karol.langner at kn.pl  Fri Nov  3 11:19:09 2006
From: karol.langner at kn.pl (Karol Langner)
Date: Fri, 3 Nov 2006 17:19:09 +0100
Subject: Setting runtime_library_dirs while building
	numpy
In-Reply-To: <20061103150943.GA35289@mail1.thewrittenword.com>
References: <20061103150943.GA35289@mail1.thewrittenword.com>
Message-ID: <200611031719.09388.karol.langner@kn.pl>

On Friday 03 of November 2006 16:09, Albert Chin wrote:
> We have atlas-3.6.0 installed in a non-standard location. When
> building the numpy-1.0 shared libraries, we'd like to encode the
> non-standard location in the RPATH of the executable. We do this with
> other Python modules by adding the following to setup.cfg:
>   [build_ext]
>   rpath=<path to library>
>
> But, this doesn't work for numpy-1.0. Is there another way to do this?

try something like this in your site.cfg:

[atlas]
library_dirs = <paths to libraries>
include_dirs = <paths to include dirs>
atlas_libs = <names of libraries>

Karol

-- 
written by Karol Langner
Fri Nov  3 17:18:38 CET 2006

-- 
written by Karol Langner
Fri Nov  3 17:19:08 CET 2006

-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From faltet at carabos.com  Fri Nov  3 10:21:34 2006
From: faltet at carabos.com (Francesc Altet)
Date: Fri, 3 Nov 2006 16:21:34 +0100
Subject: Numeric 24.2 indexing problems with Python 2.5
	and 64-bit platforms
Message-ID: <200611031621.35721.faltet@carabos.com>

Hi,

This is just to inform you that I've detected also problems with
indexation code in Numeric (24.2) with Python 2.5 and 64-bit
platforms. The next reproduces the issue:

>>> a=Numeric.array([1,0])
>>> a
array([1, 0])
>>> a[:]
zeros((0,), 'l')  # ??
# The order of values in the array doesn't seem to affect
>>> a=Numeric.array([0,1])
>>> a
array([0, 1])
>>> a[:]
zeros((0,), 'l')  # ?


I know that support for Numeric has officially ceased. This is just to
let others know that using latest Numeric with Python 2.5 and 64-bit
platforms (or, at very least, Linux64 on top of AMD64 :P) can lend to
pretty scaring results.

Cheers,

-- 
>0,0<   Francesc Altet ? ? http://www.carabos.com/
V   V   C?rabos Coop. V. ??Enjoy Data
 "-"


-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From faltet at carabos.com  Fri Nov  3 10:51:56 2006
From: faltet at carabos.com (Francesc Altet)
Date: Fri, 3 Nov 2006 16:51:56 +0100
Subject: numarray indexing problems with Python2.5
	and 64-bit platforms
In-Reply-To: <ce557a360611021326o4205fbdg5709859f57d02d2e@mail.gmail.com>
References: <1162495586.3960.19.camel@localhost.localdomain>
	<ce557a360611021326o4205fbdg5709859f57d02d2e@mail.gmail.com>
Message-ID: <200611031652.01465.faltet@carabos.com>

A Dijous 02 Novembre 2006 22:26, A. M. Archibald escrigu?:
> On 02/11/06, Francesc Altet <faltet at carabos.com> wrote:
> > I see this as a major issue in numarray and poses in great danger the
> > intended support of PyTables for numarray that we planned for some time
> > (until end of 2007). It would be nice to know if the numarray crew would
> > be willing to address this, or, now that NumPy 1.0 is out, they have
> > decided to completely drop the support for it.
> >
> > We'd really like to continue offering support for numarray (in the end,
> > it is a very good piece of software) in PyTables, but don't having a
> > solution for this problem anytime soon, will make this very problematic
> > to us.
>
> Someone has to say it: you could just drop support for the obsolete
> numarray and provide only support for its successor, numpy.

Yeah, that would be the easy path, agreed.  However, only God knows
how many code out there exists that still uses numarray (or Numeric).
We were optimistic that, given the implementation of the stunningly
efficient and easy-to-use array protocol in the three packages (NumPy,
numarray and Numeric), that very much boosted the interoperability
between them, support for all three would be a relatively easy thing
to keep in next release PyTables (with NumPy at its core).

Unfortunately, time has demonstrated how easy for a package to become
obsolete (and hence, useless) is. Just a new version of Python (2.5
this time) and the advent of 64-bit platforms has rendered obsolete
both numarray and Numeric in one shot. Mmm, perhaps it would make more
sense to support just numarray (and Numeric up to an extend) in just
32-bit platforms, but again, even that could be risky. Well, "que
sera, sera..." only time will say (as the song says).

Cheers,

-- 
>0,0<   Francesc Altet ? ? http://www.carabos.com/
V   V   C?rabos Coop. V. ??Enjoy Data
 "-"


-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From numpy-discussion at mlists.thewrittenword.com  Fri Nov  3 10:54:17 2006
From: numpy-discussion at mlists.thewrittenword.com (Albert Chin)
Date: Fri, 3 Nov 2006 09:54:17 -0600
Subject: Setting runtime_library_dirs while building
	numpy
In-Reply-To: <200611031719.09388.karol.langner@kn.pl>
References: <20061103150943.GA35289@mail1.thewrittenword.com>
	<200611031719.09388.karol.langner@kn.pl>
Message-ID: <20061103155417.GB35289@mail1.thewrittenword.com>

On Fri, Nov 03, 2006 at 05:19:09PM +0100, Karol Langner wrote:
> On Friday 03 of November 2006 16:09, Albert Chin wrote:
> > We have atlas-3.6.0 installed in a non-standard location. When
> > building the numpy-1.0 shared libraries, we'd like to encode the
> > non-standard location in the RPATH of the executable. We do this with
> > other Python modules by adding the following to setup.cfg:
> >   [build_ext]
> >   rpath=<path to library>
> >
> > But, this doesn't work for numpy-1.0. Is there another way to do this?
> 
> try something like this in your site.cfg:
> 
> [atlas]
> library_dirs = <paths to libraries>
> include_dirs = <paths to include dirs>
> atlas_libs = <names of libraries>

The library_dirs variable will simply set the path for the linker to
search for the library (i.e. -L<paths to libraries>). However, it
won't add the -Wl,-rpath,<paths to libraries> on Linux that we want
(or -R<paths to libraries> on Solaris, etc.).

-- 
albert chin (china at thewrittenword.com)

-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From alexander.belopolsky at gmail.com  Fri Nov  3 10:56:53 2006
From: alexander.belopolsky at gmail.com (Alexander Belopolsky)
Date: Fri, 3 Nov 2006 10:56:53 -0500
Subject: Need more comments from scientific community
	on python-dev
In-Reply-To: <e7bda7770611022238y2cb15e75s3739d2b4b652129c@mail.gmail.com>
References: <45474841.9030802@ieee.org>
	<d38f5330610311210i5141b495s5bb73f8da6484629@mail.gmail.com>
	<4547B235.6050904@ee.byu.edu>
	<d38f5330610311336w7538e0b2v994ee277da64f6cf@mail.gmail.com>
	<4547D2B3.5000808@ee.byu.edu>
	<d38f5330610311551u4e71cdd7ndc0b93de16dc6d80@mail.gmail.com>
	<4548C3D6.40100@ieee.org>
	<ce557a360611011013v4e883251l46dc6e2c0c85b854@mail.gmail.com>
	<e86a5fd00611011019p5a39553cwf55fd06e585e58e0@mail.gmail.com>
	<e7bda7770611022238y2cb15e75s3739d2b4b652129c@mail.gmail.com>
Message-ID: <d38f5330611030756w33b5f026t92282f6ed571e66a@mail.gmail.com>

On 11/3/06, Torgil Svensson <torgil.svensson at gmail.com> wrote:
> class struct_type(Structure):
>     _fields_ = [....]
>
> ...
> ... which is somewhat static in nature. How do you create "structures"
> dynamically?
>

You can put the above in a function that takes fields as an argument, or

    type('struct_type', (Structure,), {'_fields_':[('a', c_int), ('b',
c_long)]})

-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From shane.holloway at ieee.org  Fri Nov  3 11:27:30 2006
From: shane.holloway at ieee.org (Shane Holloway)
Date: Fri, 3 Nov 2006 09:27:30 -0700
Subject: Getting an array of indexes from a unicode string
Message-ID: <8CFCD207-9528-4689-83BE-2BCEB27B6104@ieee.org>

I'm looking for a better way to map the characters of a unicode  
string to indexes into an array of geometry.  The following code is  
functional, but it seems sub-optimal with all that numpy has to offer::

     textOrds = map(ord, text.encode('utf-8'))
     idx = indexMap[textOrds]
     textGeo = geometry[idx]

text is a simple python string coming in.  I then manually covert it  
to unicode ordinals.  Those are then mapped through indexMap, which  
happens to be a 1-to-1 mapping between unicode ordinals and valid  
indexes into geometry.  I then use the idx array to take a selection  
from geometry for the text.

As I mentioned before, this works alright, however two things seem  
inefficient.  First is the manual mapping to unicode ordinals.  Is  
there a way to have numpy do that for me?  Secondly is the mapping  
through indexMap, because it is only sparsely populated -- usually  
only a 2-5 thousand entries out of the 64 thousand allocated.  I've  
thought of using unicode.translate, but characters cannot be used for  
indexes in numpy.

What are your collective thoughts on making this cleaner and more  
efficient?

Thanks,
-Shane Holloway

-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From george.sakkis at gmail.com  Fri Nov  3 14:10:54 2006
From: george.sakkis at gmail.com (George Sakkis)
Date: Fri, 03 Nov 2006 19:10:54 -0000
Subject: numpy.repeat TypeError: array cannot be safely
	cast to required type
Message-ID: <1162581054.217811.174140@k70g2000cwa.googlegroups.com>

Can anyone explain this ?

>>> import numpy as N
>>> x = N.arange(1,6,dtype='B')
>>> x
array([1, 2, 3, 4, 5], dtype=uint8)
>>> N.repeat(x, N.ones(5,'H'))
array([1, 2, 3, 4, 5], dtype=uint8)
>>> N.repeat(x, N.ones(5,'l'))
array([1, 2, 3, 4, 5], dtype=uint8)
>>> N.repeat(x, N.ones(5,'L'))
Traceback (most recent call last):
  File "<stdin>", line 1, in ?
  File
"/usr/local/lib/python2.4/site-packages/numpy/core/fromnumeric.py",
line 83, in repeat
    return repeat(repeats, axis)
TypeError: array cannot be safely cast to required type

Thanks,
George


-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From tim.hochberg at ieee.org  Fri Nov  3 14:43:25 2006
From: tim.hochberg at ieee.org (Tim Hochberg)
Date: Fri, 03 Nov 2006 12:43:25 -0700
Subject: numpy.repeat TypeError: array cannot be
 safely cast to required type
In-Reply-To: <1162581054.217811.174140@k70g2000cwa.googlegroups.com>
References: <1162581054.217811.174140@k70g2000cwa.googlegroups.com>
Message-ID: <454B9BDD.2060101@ieee.org>

George Sakkis wrote:
> Can anyone explain this ?
>
>   
>>>> import numpy as N
>>>> x = N.arange(1,6,dtype='B')
>>>> x
>>>>         
> array([1, 2, 3, 4, 5], dtype=uint8)
>   
>>>> N.repeat(x, N.ones(5,'H'))
>>>>         
> array([1, 2, 3, 4, 5], dtype=uint8)
>   
>>>> N.repeat(x, N.ones(5,'l'))
>>>>         
> array([1, 2, 3, 4, 5], dtype=uint8)
>   
>>>> N.repeat(x, N.ones(5,'L'))
>>>>         
> Traceback (most recent call last):
>   File "<stdin>", line 1, in ?
>   File
> "/usr/local/lib/python2.4/site-packages/numpy/core/fromnumeric.py",
> line 83, in repeat
>     return repeat(repeats, axis)
> TypeError: array cannot be safely cast to required type
>   
It means you can't safely a value of type uint32 ('L') to a value of 
type int32 ('i').  The second argument of repeat needs to be castable to 
type int32.

A style note: please use the named dtypes (int32, uint32, etc) rather 
than the old-style letter codes; the former is much clearer. The answer 
to your question might  have been immediately apparent had you been 
using named dtypes.

 Personally, I'd also prefer people use the "ones([n])" syntax instead 
of the I-wish-it-were-deprecated-but-it's-too-much-to-hope-for "ones(n)" 
syntax.  T

-tim


-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From george.sakkis at gmail.com  Fri Nov  3 15:25:54 2006
From: george.sakkis at gmail.com (George Sakkis)
Date: Fri, 03 Nov 2006 20:25:54 -0000
Subject: numpy.repeat TypeError: array cannot be
	safely cast to required type
References: <1162581054.217811.174140@k70g2000cwa.googlegroups.com>
	<454B9BDD.2060101@ieee.org>
Message-ID: <1162585554.214708.108210@k70g2000cwa.googlegroups.com>

Tim Hochberg wrote:

> George Sakkis wrote:
> > Can anyone explain this ?
> >
> >
> >>>> import numpy as N
> >>>> x = N.arange(1,6,dtype='B')
> >>>> x
> >>>>
> > array([1, 2, 3, 4, 5], dtype=uint8)
> >
> >>>> N.repeat(x, N.ones(5,'H'))
> >>>>
> > array([1, 2, 3, 4, 5], dtype=uint8)
> >
> >>>> N.repeat(x, N.ones(5,'l'))
> >>>>
> > array([1, 2, 3, 4, 5], dtype=uint8)
> >
> >>>> N.repeat(x, N.ones(5,'L'))
> >>>>
> > Traceback (most recent call last):
> >   File "<stdin>", line 1, in ?
> >   File
> > "/usr/local/lib/python2.4/site-packages/numpy/core/fromnumeric.py",
> > line 83, in repeat
> >     return repeat(repeats, axis)
> > TypeError: array cannot be safely cast to required type
> >
> It means you can't safely a value of type uint32 ('L') to a value of
> type int32 ('i').  The second argument of repeat needs to be castable to
> type int32.

I see. I suppose there is a better reason for this than a "why on earth
would anyone repeat a value more than 2**32-1 times". Besides, why
should it be castable to a signed type since we're talking about number
of repetitions ?

> A style note: please use the named dtypes (int32, uint32, etc) rather
> than the old-style letter codes; the former is much clearer. The answer
> to your question might  have been immediately apparent had you been
> using named dtypes.

I'm using numpy along with the struct module, so the letter codes are
convenient for talking to both modules.

George


-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From oliphant at ee.byu.edu  Fri Nov  3 15:26:23 2006
From: oliphant at ee.byu.edu (Travis Oliphant)
Date: Fri, 03 Nov 2006 13:26:23 -0700
Subject: I won't be as visible for a few weeks
Message-ID: <454BA5EF.6070309@ee.byu.edu>



I'm writing this to indicate that I won't be as visible on the lists for 
a few weeks as I have some pressing matters to attend to that will take 
more of my time.

I apologize for not being able to help more right now.

Best regards,

-Travis Oliphant


-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From tim.hochberg at ieee.org  Fri Nov  3 15:39:15 2006
From: tim.hochberg at ieee.org (Tim Hochberg)
Date: Fri, 03 Nov 2006 13:39:15 -0700
Subject: numpy.repeat TypeError: array cannot
 be	safely cast to required type
In-Reply-To: <1162585554.214708.108210@k70g2000cwa.googlegroups.com>
References: <1162581054.217811.174140@k70g2000cwa.googlegroups.com>	<454B9BDD.2060101@ieee.org>
	<1162585554.214708.108210@k70g2000cwa.googlegroups.com>
Message-ID: <454BA8F3.3020609@ieee.org>

George Sakkis wrote:
> Tim Hochberg wrote:
>
>   
>> George Sakkis wrote:
>>     
>>> Can anyone explain this ?
>>>
>>>
>>>       
>>>>>> import numpy as N
>>>>>> x = N.arange(1,6,dtype='B')
>>>>>> x
>>>>>>
>>>>>>             
>>> array([1, 2, 3, 4, 5], dtype=uint8)
>>>
>>>       
>>>>>> N.repeat(x, N.ones(5,'H'))
>>>>>>
>>>>>>             
>>> array([1, 2, 3, 4, 5], dtype=uint8)
>>>
>>>       
>>>>>> N.repeat(x, N.ones(5,'l'))
>>>>>>
>>>>>>             
>>> array([1, 2, 3, 4, 5], dtype=uint8)
>>>
>>>       
>>>>>> N.repeat(x, N.ones(5,'L'))
>>>>>>
>>>>>>             
>>> Traceback (most recent call last):
>>>   File "<stdin>", line 1, in ?
>>>   File
>>> "/usr/local/lib/python2.4/site-packages/numpy/core/fromnumeric.py",
>>> line 83, in repeat
>>>     return repeat(repeats, axis)
>>> TypeError: array cannot be safely cast to required type
>>>
>>>       
>> It means you can't safely a value of type uint32 ('L') to a value of
>> type int32 ('i').  The second argument of repeat needs to be castable to
>> type int32.
>>     
>
> I see. I suppose there is a better reason for this than a "why on earth
> would anyone repeat a value more than 2**32-1 times". Besides, why
> should it be castable to a signed type since we're talking about number
> of repetitions ?
>   
I suspect it's just a matter of simplicity although I haven't looked at 
the code in question. On the way in all of the arrays are going to need 
to be cast to some standard integral type. While unit32 might make 
sense, int32 is by far the more common type. If one was to use uint32 as 
the type, before  casting int32 to uint32, one would have to scan the 
array looking for negative values lest they get cast into really big 
positive values. An analogous problem exists for casting uint32 to 
int32, but since unit32 is rather uncommon, one can just make users do 
the necessary checking and casting by hand and probably someone will 
only notice every couple of years.


>> A style note: please use the named dtypes (int32, uint32, etc) rather
>> than the old-style letter codes; the former is much clearer. The answer
>> to your question might  have been immediately apparent had you been
>> using named dtypes.
>>     
>
> I'm using numpy along with the struct module, so the letter codes are
> convenient for talking to both modules.
>   
I imagine they are, but they're harder for me, and I suspect many 
others, to remember. Your more likely to get a useful answer if your 
readers don't have to go look stuff up.

-tim




-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From ahmed7 at nyu.edu  Fri Nov  3 20:35:40 2006
From: ahmed7 at nyu.edu (Info)
Date: Sat, 04 Nov 2006 04:35:40 +0300
Subject: =?windows-1251?b?xOXt/CDz5ODr8f8=?=
Message-ID: <6bca01c6ffb1$6cdf2d88$aa88ca2c@nyu.edu>

?????? ????!


-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
_______________________________________________
Numpy-discussion mailing list
Numpy-discussion at lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/numpy-discussion

From oliphant.travis at ieee.org  Fri Nov  3 20:56:39 2006
From: oliphant.travis at ieee.org (Travis Oliphant)
Date: Fri, 03 Nov 2006 18:56:39 -0700
Subject: Reading records from file and sorting
In-Reply-To: <1162442534.508320.151770@e3g2000cwe.googlegroups.com>
References: <1162337923.747767.323100@b28g2000cwb.googlegroups.com>	<20061101224620.GA29602@dogbert.sdsl.sun.ac.za>
	<1162442534.508320.151770@e3g2000cwe.googlegroups.com>
Message-ID: <454BF357.9020903@ieee.org>

George Sakkis wrote:
> Albert Strasheim wrote:
>
>   
>> Check the thread "Strange results when sorting array with fields" from
>> about a week back. Travis made some changes to sorting in the presence
>> of fields that should solve your problem, assuming your fields appear in
>> the order you want to sort (i.e. you want to sort f1, f2, f3 and not
>> something like f1, f3, f2).
>>     
>
> I'm afraid this won't help in my case; I want to sort twice, once by f1
> and once by f2. I guess I could make a second file with the fields
> swapped but this seems more messy and inefficient than Francesc's
> suggestion.
>   

As a final contribution before I become more quiet for a few weeks (and 
aside from hopefully releasing 1.0.1 soon), I've added a feature to 
allow specifying the sorting order for record arrays.

It's added to the sort method as the order= keyword.  You can pass in a 
string (specifying which field comes first --- all other fields will 
stay in the same order) or you can pass in a list or tuple which 
indicates the field order (any fields un-specified will stay in their 
same relative order).

It works be creating a new data-type object with the .names attribute 
replaced with a newly ordered one and then calling sort on a view of the 
array with that data-type.  The VOID_compare uses the names tuple to 
determine the ordering.

This was the un-named option in my previous list.  It's a better 
solution than all of the others, I think.  The newly created data-type 
is discarded after the sorting is complete.

-Travis


-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From lyfebwkyrrh at optical-center.com  Fri Nov  3 23:22:35 2006
From: lyfebwkyrrh at optical-center.com (require)
Date: Sat, 4 Nov 2006 05:22:35 +0100
Subject: Americans highly declined percent
Message-ID: <001201c6ffc8$da888540$73cb00c3@gustavson>


Ee in spec task potential hazards model.
Aristocats Rescuers Newhart Pocahontas! Accounting Admin Clerical Banking Biotech Consultant a Customer Education a.
Aristocats Rescuers Newhart Pocahontas!
Shell services committed of furthering volunteers or Compliance. Img off Pingbacks. Muffin am Crash Vids Webndex Gorilla is Mask Shavehead Uber a Ytmnd.
Rescuers Newhart Pocahontas Irene Bedard Cast Crew Castclint Howard of.
Pdtaol Usersjj or Pdtwhose a itvolterwd. Ipod trademark Apple partner sponsor emusiccom or Skinsmore Classicz is Alien.
Copyleft resulting gpl gpl wider. Avoider in Deluxe or fiddly of.
Bca wide in Greece or influenced became prominent Greeks or. Jfrancis in Daniel in Glazman Neil Rashbrook of parkwaycc uk.
More capital billed. Hop Anime raquoaol Launches Enhanced.
Currently of ant Note h Quicktime Mplayer same in video.
Compile sourcesuse command in ipodsa indepth am. More capital billed.
Dialogs am documents subscribe.
Halloway hearing or sleep scene occurs duplicate. Ltlt lt Prev gt?
Protecting crucial Attorney speaks Patches is holes emerge of Rivals.
Claws of claws so damage scratches Cats Saves am Furniture. Hypnotic Wannabe Webcam cutie assets Lust am Bust.
Steps millions a service Dulles Vabased.
Servers analogue Millennium Dmcpa Dmca prohibits indirect brutally. Reading draft Platform Enterprise Edition aka jsr of noticed or nothing.
Systems contains verify cost world homepages tracking ftp.
Halloway hearing sleep scene occurs duplicate or earlier one. Reading draft Platform Enterprise Edition aka jsr of noticed or nothing.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20061104/3ed4c9e5/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: therefore.gif
Type: image/gif
Size: 7982 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20061104/3ed4c9e5/attachment-0001.gif>
-------------- next part --------------
-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
-------------- next part --------------
_______________________________________________
Numpy-discussion mailing list
Numpy-discussion at lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/numpy-discussion

From bjrtjhnbv at pbna.com  Sat Nov  4 02:42:52 2006
From: bjrtjhnbv at pbna.com (Pages)
Date: Sat, 4 Nov 2006 03:42:52 -0400
Subject: expressed dislike
Message-ID: <001201c6ffe4$d5a478b0$4ef9c4a1@KCGT4G2>


Ironware and up ant reg?
Idiom is humorous effect criticized adhering expressed. Middotsign middothelp am Username forin postsin tagsin is.
Idiom is humorous effect criticized adhering expressed.
Xen in Paul rage days Advances well increasing energy make. Person a Cutter lossless am wav cutting Font Fitting cache. Wikimedia Foundation Seattle Times Comics Gameson error in resume.
Dion or Peter Cetera list or Sounds ros a commentyou.
Patent Leather appeared featured of unknown Depeche produced Daniel. Warnings Shoutbox Folding Processors Memory.
Minute principle odds finest courage. Ebuild binary summed succinctly postwhat now generic non is.
Return Type nsieditor numerous? Developers encourage in do discourage jackass am sole.
Intheknow Donrsquot tomorrow is guarantee. Adams Morgansix blue interior or ne Dcthu reclyner or woodbridge.
Gone taking in Roberts suggestion entered title.
Madness disc finishing exe cd inserted or should? Intheknow Donrsquot tomorrow is guarantee.
Newspaper Ostg or Thinkgeek or Itmj Linuxcom am!
Background university nascent in timeedit! Fellow musicians trends favorite pmchaz.
Dog in distro is kai Staats ceo.
Giving Labs James Wards is brief or. Crazy longtime intrigued presents of tag am.
Weak toolsread Printinghp.
Terminate withdraw deposit deliver process departing lti in package of. Enrogue in tricky right snippet workxul var null me.
Digitalb nddigitalb a a Galaxie ndgalaxie.
Rules Procedures Integrated Sustained Judging Wondering achieved. Enrogue in tricky right snippet workxul var null me.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20061104/8a0bf29c/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: central.gif
Type: image/gif
Size: 7758 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20061104/8a0bf29c/attachment-0001.gif>
-------------- next part --------------
-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
-------------- next part --------------
_______________________________________________
Numpy-discussion mailing list
Numpy-discussion at lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/numpy-discussion

From Mailer-Daemon at lists.sourceforge.net  Sat Nov  4 03:44:41 2006
From: Mailer-Daemon at lists.sourceforge.net (Mail Delivery System)
Date: Sat, 04 Nov 2006 00:44:41 -0800
Subject: Warning: message 1GfAoO-0007im-Sh delayed 72
	hours
Message-ID: <E1GgH8r-0007q7-05@externalmx-1.sourceforge.net>

This message was created automatically by mail delivery software.
A message that you sent has not yet been delivered to one or more of its
recipients after more than 72 hours on the queue on externalmx-1.sourceforge.net.

The message identifier is:     1GfAoO-0007im-Sh
The subject of the message is: looked like me the full sized HARLEY and began to her, with this but as
The date of the message is:    Wed, 01 Nov 2006 15:46:73 +0800

The address to which the message has not yet been delivered is:

  numpy-discussion at lists.sourceforge.net
    Delay reason: SMTP error from remote mailer after RCPT TO:<numpy-discussion at lists.sourceforge.net>:
    host mail.sourceforge.net [66.35.250.206]: 451-Could not complete sender verify callout
    451-Could not complete sender verify callout for
    451-<numpy-discussion at lists.sourceforge.net>.
    451-The mail server(s) for the domain may be temporarily unreachable, or
    451-they may be permanently unreachable from this server. In the latter case,
    451-you need to change the address or create an MX record for its domain
    451-if it is 

No action is required on your part. Delivery attempts will continue for
some time, and this warning may be repeated at intervals if the message
remains undelivered. Eventually the mail delivery software will give up,
and when that happens, the message will be returned to you.

-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From cjw at sympatico.ca  Sat Nov  4 07:30:07 2006
From: cjw at sympatico.ca (Colin J. Williams)
Date: Sat, 04 Nov 2006 07:30:07 -0500
Subject: Style - was Re: numpy.repeat TypeError: array
 cannot be safely cast to required type
In-Reply-To: <454B9BDD.2060101@ieee.org>
References: <1162581054.217811.174140@k70g2000cwa.googlegroups.com>
	<454B9BDD.2060101@ieee.org>
Message-ID: <454C87CF.7030500@sympatico.ca>



Tim Hochberg wrote:
[snip]
> A style note: please use the named dtypes (int32, uint32, etc) rather 
> than the old-style letter codes; the former is much clearer. The answer 
> to your question might  have been immediately apparent had you been 
> using named dtypes.
>
>   
+1
>  Personally, I'd also prefer people use the "ones([n])" syntax instead 
> of the I-wish-it-were-deprecated-but-it's-too-much-to-hope-for "ones(n)" 
> syntax.  T
>
> -tim
>
>   
Could you elaborate please?

Colin W.

-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From mcfletch at vrplumber.com  Sat Nov  4 11:21:12 2006
From: mcfletch at vrplumber.com (Mike C. Fletcher)
Date: Sat, 04 Nov 2006 11:21:12 -0500
Subject: [PyOpenGL-Devel] Need more comments from
 scientific community on python-dev
In-Reply-To: <C23A1918-5E8B-4033-8969-45B20B098CB6@gmail.com>
References: <mailman.90862.1162320870.22397.numpy-discussion@lists.sourceforge.net>
	<C23A1918-5E8B-4033-8969-45B20B098CB6@gmail.com>
Message-ID: <454CBDF8.6050807@vrplumber.com>

Josh Marshall wrote:
> On 11/31/06, Fernando Perez <fperez.net at gmail.com> wrote:
>
>   
>> Fernando Perez wrote:
>> ps - one more thing.  This guy:
>>
>> http://blog.vrplumber.com/
>>
>> has been rewriting the OpenGL bindings using ctypes, and I've seen
>> posts from him about numpy (in his blog).  He might be able to
>> contribute something...
>>     
...
> Now, I haven't spent much time looking at these parts of OpenGL- 
> ctypes, as they have just worked for me. I would think that it would  
> be trivial to write a FormatHandler which uses the ndarray interface  
> and data type description to use any object implenting it as an input  
> for OpenGL. This would include things such as PIL images.
>   
Yes, this is the purpose of the design, to allow users to write new 
format handlers that can be registered at run-time and act as 
first-class data-formats within the system.  PIL images, custom vector 
objects, memory-mapped files, media buffers and the like are all 
contemplated as useful array storage formats.

> Mike, can you give us your opinion on how a standardised data type  
> descriptor would be helpful for PyOpenGL? The PEP and some  
> information about it can be found here:
> http://www.scipy.org/ArrayInterfacePEP
>   
A standardised data-type descriptor would be of some benefit for us, as 
it would allow a single handler to deal with larger numbers of storage 
formats (reducing the amount of coding required).  That said, the spec 
as proposed has far more features than *most* OpenGL users will use 
(OpenGL being largely (though not exclusively) focused on simple, 
homogeneous data-types), and the handler mechanism largely abstracts 
away the problem for *us* at the moment.  We wouldn't be able to get rid 
of the abstraction mechanism entirely, as we would still have data-types 
such as lists-of-lists-of-integers that wouldn't support the protocol.

The likely efficiency of accessing the metadata (versus current 
implementation where in some cases we wind up producing a whole 
dictionary object with meta-data just to pull out a single value) would 
be attractive.

We also need functionality to get at the data-pointer for the objects to 
make any description useful, though I suppose that's out of scope for 
this particular PEP.

HTH,
Mike

-- 
________________________________________________
  Mike C. Fletcher
  Designer, VR Plumber, Coder
  http://www.vrplumber.com
  http://blog.vrplumber.com


-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From ndarray at mac.com  Sat Nov  4 13:03:24 2006
From: ndarray at mac.com (Sasha)
Date: Sat, 4 Nov 2006 13:03:24 -0500
Subject: Need more comments from scientific community
	on python-dev
In-Reply-To: <d38f5330611030756w33b5f026t92282f6ed571e66a@mail.gmail.com>
References: <45474841.9030802@ieee.org> <4547B235.6050904@ee.byu.edu>
	<d38f5330610311336w7538e0b2v994ee277da64f6cf@mail.gmail.com>
	<4547D2B3.5000808@ee.byu.edu>
	<d38f5330610311551u4e71cdd7ndc0b93de16dc6d80@mail.gmail.com>
	<4548C3D6.40100@ieee.org>
	<ce557a360611011013v4e883251l46dc6e2c0c85b854@mail.gmail.com>
	<e86a5fd00611011019p5a39553cwf55fd06e585e58e0@mail.gmail.com>
	<e7bda7770611022238y2cb15e75s3739d2b4b652129c@mail.gmail.com>
	<d38f5330611030756w33b5f026t92282f6ed571e66a@mail.gmail.com>
Message-ID: <d38f5330611041003v6c81f856qda00d518cef62f38@mail.gmail.com>

On 11/3/06, Torgil Svensson <torgil.svensson at gmail.com> wrote:
> class struct_type(Structure):
>     _fields_ = [....]
>
> ...
> ... which is somewhat static in nature. How do you create "structures"
> dynamically?
>

You can put the above in a function that takes fields as an argument, or

    type('struct_type', (Structure,), {'_fields_':[('a', c_int), ('b',
c_long)]})

-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From tim.hochberg at ieee.org  Sat Nov  4 15:23:11 2006
From: tim.hochberg at ieee.org (Tim Hochberg)
Date: Sat, 04 Nov 2006 13:23:11 -0700
Subject: Style - was Re: numpy.repeat TypeError:
 array cannot be safely cast to required type
In-Reply-To: <454C87CF.7030500@sympatico.ca>
References: <1162581054.217811.174140@k70g2000cwa.googlegroups.com>	<454B9BDD.2060101@ieee.org>
	<454C87CF.7030500@sympatico.ca>
Message-ID: <454CF6AF.10205@ieee.org>

Colin J. Williams wrote:
> Tim Hochberg wrote:
> [snip]
>   
>> A style note: please use the named dtypes (int32, uint32, etc) rather 
>> than the old-style letter codes; the former is much clearer. The answer 
>> to your question might  have been immediately apparent had you been 
>> using named dtypes.
>>
>>   
>>     
> +1
>   
>>  Personally, I'd also prefer people use the "ones([n])" syntax instead 
>> of the I-wish-it-were-deprecated-but-it's-too-much-to-hope-for "ones(n)" 
>> syntax.  T
>>
>> -tim
>>
>>   
>>     
> Could you elaborate please?
>
>   
Sure. The general form of the zeros function, and several others[1], is:

    zeros(shape, dtype)

 Here 'shape' is a sequence of one sort or another. There's also a 
second form that's applicable only to one dimensional arrays:

    zeros(length, dtype)

Where length is an integer. I don't recall if this is a historical 
legacy or is intended as a convenience function or a bit of both. Either 
way, the result is that there are two ways to spell "give me a 1D array 
of zeros with a given length and dtype":

    zeros([length], dtype)

and

    zeros(length, dtype)


I have two issues with having this second spelling. First, it's one more 
thing to remember. Whenever I see the scalar spelling I have to use a 
little bit extra of my limited brainpower to remember that it is not in 
fact a typo, but is instead a shortcut. The second issue is pedagogical. 
If people are initially exposed to the first form, the extension to 
multiple dimensions is straightforward. They'll probably guess the 
correct way right off the bat, and if not, they'll get it right away 
when it's explained. On the other hand, if they are initially exposed to 
the second form, the multidimensional form is far from obvious. In 
addition, they'll probably spend a long time thinking that the 
one-dimensional way is the normal way, but that we have to jump through 
weird hoops to get multidimensional arrays to work. That's bad 
propaganda for numpy. This all seems like a rather large price to pay to 
avoid typing the occasional pair of brackets.

That's my two cents. Just say not to form #2.

-tim



... [1] Yes, I'm neglecting the order parameter; I don't think it 
matters for this discussion.



-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From dalcinl at gmail.com  Sat Nov  4 16:12:01 2006
From: dalcinl at gmail.com (Lisandro Dalcin)
Date: Sat, 4 Nov 2006 18:12:01 -0300
Subject: [PyOpenGL-Devel] Need more comments from
	scientific community on python-dev
In-Reply-To: <454CBDF8.6050807@vrplumber.com>
References: <mailman.90862.1162320870.22397.numpy-discussion@lists.sourceforge.net>
	<C23A1918-5E8B-4033-8969-45B20B098CB6@gmail.com>
	<454CBDF8.6050807@vrplumber.com>
Message-ID: <e7ba66e40611041312q2973015elbdd18b41b2698999@mail.gmail.com>

On 11/4/06, Mike C. Fletcher <mcfletch at vrplumber.com> wrote:
>  We wouldn't be able to get rid
> of the abstraction mechanism entirely, as we would still have data-types
> such as lists-of-lists-of-integers that wouldn't support the protocol.
>

Perhaps this will sound stupid, but I was thinking about extending
some builtin Python types to provide datatype info. A list/tuple is an
array of Python objects, an int/float/complex could return a datatype
upon query. This could be very efficient if some predefined datatypes
were already defined in the C-side. MPI provides predefined datatypes
(MPI_INT, MPI_FLOAT) and new user-defined datatypes are created from
them.


-- 
Lisandro Dalc?n
---------------
Centro Internacional de M?todos Computacionales en Ingenier?a (CIMEC)
Instituto de Desarrollo Tecnol?gico para la Industria Qu?mica (INTEC)
Consejo Nacional de Investigaciones Cient?ficas y T?cnicas (CONICET)
PTLC - G?emes 3450, (3000) Santa Fe, Argentina
Tel/Fax: +54-(0)342-451.1594

-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From arildna at stud.ntnu.no  Sat Nov  4 16:22:19 2006
From: arildna at stud.ntnu.no (arildna at stud.ntnu.no)
Date: Sat,  4 Nov 2006 22:22:19 +0100
Subject: problems installing NumPy on OSX
Message-ID: <20061104222219.5v1dj889q8cgsg8c@webmail.ntnu.no>

Hi,

I'm sorry if this might seem like a stupid question to some of you,  
but I have been struggling for the better part of the afternoon trying  
to install NumPy on my G4 iBook, so I hope somebody can take the time  
to lend me a helping hand. I have searched a little in the mail  
archives and tried to follow the instructions on this page:
http://www.scipy.org/Installing_SciPy/Mac_OS_X

Python 2.5 is installed, and I have installed Apple's Developer's  
Tools (i.e. the package "December 2002 Mac OS X Developer Tools"). The  
GCC/G77-installation seemed to go ok.

When I try to install the FFW libraries however, I get an error  
message telling me that my "C compiler cannot create executables"  
(full output below).

After that I didn't really expect the installation of NumPy to work,  
but I got a "permission denied"-error message that I'm not so sure is  
connected to the missing FFW.

So I'm at loss... Anyone?

Many thanks,

    Arild N?ss

===============================================================

%pwd
/Users/arildnss/Desktop/fftw-3.1.2

%./configure
checking for a BSD-compatible install... /usr/bin/install -c
checking whether build environment is sane... yes
checking for gawk... no
checking for mawk... no
checking for nawk... no
checking for awk... awk
checking whether make sets $(MAKE)... yes
checking whether to enable maintainer-specific portions of Makefiles... no
checking build system type... powerpc-apple-darwin8.8.0
checking host system type... powerpc-apple-darwin8.8.0
checking for gcc... gcc
checking for C compiler default output file name... configure: error:  
C compiler cannot create executables
See `config.log' for more details.


%pwd
/Users/arildnss/Desktop/numpy-1.0

%python setup.py build
Running from numpy source directory.
F2PY Version 2_3396
blas_opt_info:
   FOUND:
     extra_link_args = ['-Wl,-framework', '-Wl,Accelerate']
     define_macros = [('NO_ATLAS_INFO', 3)]
     extra_compile_args = ['-faltivec',  
'-I/System/Library/Frameworks/vecLib.framework/Headers']

lapack_opt_info:
   FOUND:
     extra_link_args = ['-Wl,-framework', '-Wl,Accelerate']
     define_macros = [('NO_ATLAS_INFO', 3)]
     extra_compile_args = ['-faltivec']

running build
running config_fc
running build_src
building py_modules sources
error: build/src.macosx-10.3-fat-2.5/numpy/distutils/__config__.py:  
Permission denied



-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From robert.kern at gmail.com  Sat Nov  4 16:58:35 2006
From: robert.kern at gmail.com (Robert Kern)
Date: Sat, 04 Nov 2006 15:58:35 -0600
Subject: problems installing NumPy on OSX
In-Reply-To: <20061104222219.5v1dj889q8cgsg8c@webmail.ntnu.no>
References: <20061104222219.5v1dj889q8cgsg8c@webmail.ntnu.no>
Message-ID: <eij2ej$245$1@sea.gmane.org>

arildna at stud.ntnu.no wrote:
> Hi,
> 
> I'm sorry if this might seem like a stupid question to some of you,  
> but I have been struggling for the better part of the afternoon trying  
> to install NumPy on my G4 iBook, so I hope somebody can take the time  
> to lend me a helping hand. I have searched a little in the mail  
> archives and tried to follow the instructions on this page:
> http://www.scipy.org/Installing_SciPy/Mac_OS_X

Note that these are instructions for installing scipy, so there is a lot there
that you don't need to do just to install numpy. Notably, some of the things
that you are having problems with.

> Python 2.5 is installed, and I have installed Apple's Developer's  
> Tools (i.e. the package "December 2002 Mac OS X Developer Tools"). The  
> GCC/G77-installation seemed to go ok.

You probably can't use g77 along with Python 2.5. Python 2.5 is built with gcc
4.0 but g77 never got ported to the 4.x platform. However, it's not at all
necessary for numpy.

> When I try to install the FFW libraries however, I get an error  
> message telling me that my "C compiler cannot create executables"  
> (full output below).

numpy does not use FFTW at all, so don't let this hold you up. If you want to
troubleshoot the problem, however, look at the config.log file for more
information. grep for the string "cannot create executables".

> After that I didn't really expect the installation of NumPy to work,  
> but I got a "permission denied"-error message that I'm not so sure is  
> connected to the missing FFW.

You probably tried to run a previous build or install as root. Delete the build/
directory (probably as root) and try another build as a regular user. Don't use
root until you actually want to install (and then, only if you need to).

-- 
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless enigma
 that is made terrible by our own mad attempt to interpret it as though it had
 an underlying truth."
  -- Umberto Eco


-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From arildna at stud.ntnu.no  Sat Nov  4 17:32:10 2006
From: arildna at stud.ntnu.no (arildna at stud.ntnu.no)
Date: Sat,  4 Nov 2006 23:32:10 +0100
Subject: problems installing NumPy on OSX
In-Reply-To: <eij2ej$245$1@sea.gmane.org>
References: <20061104222219.5v1dj889q8cgsg8c@webmail.ntnu.no>
	<eij2ej$245$1@sea.gmane.org>
Message-ID: <20061104233210.xfx6c75uswss0gsw@webmail.ntnu.no>

Siterer Robert Kern <robert.kern at gmail.com>:

> arildna at stud.ntnu.no wrote:
>> Hi,
>>
>> I'm sorry if this might seem like a stupid question to some of you,
>> but I have been struggling for the better part of the afternoon trying
>> to install NumPy on my G4 iBook, so I hope somebody can take the time
>> to lend me a helping hand. I have searched a little in the mail
>> archives and tried to follow the instructions on this page:
>> http://www.scipy.org/Installing_SciPy/Mac_OS_X
>
> Note that these are instructions for installing scipy, so there is a  
>  lot there
> that you don't need to do just to install numpy. Notably, some of the things
> that you are having problems with.

I did get the impression that it was kind of an overkill...

>> After that I didn't really expect the installation of NumPy to work,
>> but I got a "permission denied"-error message that I'm not so sure is
>> connected to the missing FFW.
>
> You probably tried to run a previous build or install as root.   
> Delete the build/
> directory (probably as root) and try another build as a regular   
> user. Don't use
> root until you actually want to install (and then, only if you need to).

That got the installation a lot further, but it still halts with an  
error message, this time about failing to test the configuration.

    Arild N?ss


==========================================================================

% python setup.py build

Running from numpy source directory.
F2PY Version 2_3396
blas_opt_info:
   FOUND:
     extra_link_args = ['-Wl,-framework', '-Wl,Accelerate']
     define_macros = [('NO_ATLAS_INFO', 3)]
     extra_compile_args = ['-faltivec',  
'-I/System/Library/Frameworks/vecLib.framework/Headers']

lapack_opt_info:
   FOUND:
     extra_link_args = ['-Wl,-framework', '-Wl,Accelerate']
     define_macros = [('NO_ATLAS_INFO', 3)]
     extra_compile_args = ['-faltivec']

running build
running config_fc
running build_src
building py_modules sources
creating build
creating build/src.macosx-10.3-fat-2.5
creating build/src.macosx-10.3-fat-2.5/numpy
creating build/src.macosx-10.3-fat-2.5/numpy/distutils
building extension "numpy.core.multiarray" sources
creating build/src.macosx-10.3-fat-2.5/numpy/core
Generating build/src.macosx-10.3-fat-2.5/numpy/core/config.h
customize NAGFCompiler
customize AbsoftFCompiler
customize IbmFCompiler
Could not locate executable g77
Could not locate executable f77
Could not locate executable gfortran
Could not locate executable f95
customize GnuFCompiler
customize Gnu95FCompiler
customize G95FCompiler
customize GnuFCompiler
customize Gnu95FCompiler
customize NAGFCompiler
customize NAGFCompiler using config
C compiler: gcc -arch ppc -arch i386 -isysroot  
/Developer/SDKs/MacOSX10.4u.sdk -fno-strict-aliasing -Wno-long-double  
-no-cpp-precomp -mno-fused-madd -fno-common -dynamic -DNDEBUG -g -O3

compile options:  
'-I/Library/Frameworks/Python.framework/Versions/2.5/include/python2.5  
-Inumpy/core/src -Inumpy/core/include  
-I/Library/Frameworks/Python.framework/Versions/2.5/include/python2.5  
-c'
gcc: _configtest.c
gcc: cannot specify -o with -c or -S and multiple compilations
gcc: cannot specify -o with -c or -S and multiple compilations
failure.
removing: _configtest.c _configtest.o
numpy/core/setup.py:50: DeprecationWarning: raising a string exception  
is deprecated
   raise "ERROR: Failed to test configuration"
Traceback (most recent call last):
   File "setup.py", line 89, in <module>
     setup_package()
   File "setup.py", line 82, in setup_package
     configuration=configuration )
   File "/Users/arildnss/Desktop/numpy-1.0/numpy/distutils/core.py",  
line 174, in setup
     return old_setup(**new_attr)
   File  
"/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/distutils/core.py", line 151, in  
setup
     dist.run_commands()
   File  
"/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/distutils/dist.py", line 974, in  
run_commands
     self.run_command(cmd)
   File  
"/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/distutils/dist.py", line 994, in  
run_command
     cmd_obj.run()
   File  
"/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/distutils/command/build.py", line 112, in  
run
     self.run_command(cmd_name)
   File  
"/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/distutils/cmd.py", line 333, in  
run_command
     self.distribution.run_command(command)
   File  
"/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/distutils/dist.py", line 994, in  
run_command
     cmd_obj.run()
   File  
"/Users/arildnss/Desktop/numpy-1.0/numpy/distutils/command/build_src.py", line  
87, in run
     self.build_sources()
   File  
"/Users/arildnss/Desktop/numpy-1.0/numpy/distutils/command/build_src.py", line  
106, in build_sources
     self.build_extension_sources(ext)
   File  
"/Users/arildnss/Desktop/numpy-1.0/numpy/distutils/command/build_src.py", line  
212, in build_extension_sources
     sources = self.generate_sources(sources, ext)
   File  
"/Users/arildnss/Desktop/numpy-1.0/numpy/distutils/command/build_src.py", line  
270, in generate_sources
     source = func(extension, build_dir)
   File "numpy/core/setup.py", line 50, in generate_config_h
     raise "ERROR: Failed to test configuration"
ERROR: Failed to test configuration


-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From robert.kern at gmail.com  Sat Nov  4 17:36:04 2006
From: robert.kern at gmail.com (Robert Kern)
Date: Sat, 04 Nov 2006 16:36:04 -0600
Subject: problems installing NumPy on OSX
In-Reply-To: <20061104233210.xfx6c75uswss0gsw@webmail.ntnu.no>
References: <20061104222219.5v1dj889q8cgsg8c@webmail.ntnu.no>	<eij2ej$245$1@sea.gmane.org>
	<20061104233210.xfx6c75uswss0gsw@webmail.ntnu.no>
Message-ID: <eij4kr$7l2$1@sea.gmane.org>

arildna at stud.ntnu.no wrote:
> gcc: _configtest.c
> gcc: cannot specify -o with -c or -S and multiple compilations
> gcc: cannot specify -o with -c or -S and multiple compilations
> failure.

You're using OS X 10.3.9? There is a bug in distutils on that platform. It was
fixed in Python's trunk, but I'm not sure where to find it.

-- 
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless enigma
 that is made terrible by our own mad attempt to interpret it as though it had
 an underlying truth."
  -- Umberto Eco


-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From robert.kern at gmail.com  Sat Nov  4 17:37:49 2006
From: robert.kern at gmail.com (Robert Kern)
Date: Sat, 04 Nov 2006 16:37:49 -0600
Subject: problems installing NumPy on OSX
In-Reply-To: <20061104222219.5v1dj889q8cgsg8c@webmail.ntnu.no>
References: <20061104222219.5v1dj889q8cgsg8c@webmail.ntnu.no>
Message-ID: <eij4o3$7l2$2@sea.gmane.org>

arildna at stud.ntnu.no wrote:

> Python 2.5 is installed, and I have installed Apple's Developer's  
> Tools (i.e. the package "December 2002 Mac OS X Developer Tools").

By the way, this is *really* old. Long before there were Universal binaries
(which is what Python 2.5 is). I believe you will want to install Xcode 2.4.1
which was released on Tuesday.

-- 
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless enigma
 that is made terrible by our own mad attempt to interpret it as though it had
 an underlying truth."
  -- Umberto Eco


-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From arildna at stud.ntnu.no  Sun Nov  5 03:09:43 2006
From: arildna at stud.ntnu.no (arildna at stud.ntnu.no)
Date: Sun,  5 Nov 2006 09:09:43 +0100
Subject: problems installing NumPy on OSX
In-Reply-To: <eij4kr$7l2$1@sea.gmane.org>
References: <20061104222219.5v1dj889q8cgsg8c@webmail.ntnu.no>
	<eij2ej$245$1@sea.gmane.org>
	<20061104233210.xfx6c75uswss0gsw@webmail.ntnu.no>
	<eij4kr$7l2$1@sea.gmane.org>
Message-ID: <20061105090943.e33g3827q8oco8s0@webmail.ntnu.no>

Siterer Robert Kern <robert.kern at gmail.com>:

> arildna at stud.ntnu.no wrote:
>> gcc: _configtest.c
>> gcc: cannot specify -o with -c or -S and multiple compilations
>> gcc: cannot specify -o with -c or -S and multiple compilations
>> failure.
>
> You're using OS X 10.3.9? There is a bug in distutils on that   
> platform. It was
> fixed in Python's trunk, but I'm not sure where to find it.
>
> --
> Robert Kern

No, I'm on OS X 10.4.8, so my OS version shouldn't be the problem

    Arild N?ss


-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From arildna at stud.ntnu.no  Sun Nov  5 03:17:27 2006
From: arildna at stud.ntnu.no (arildna at stud.ntnu.no)
Date: Sun,  5 Nov 2006 09:17:27 +0100
Subject: problems installing NumPy on OSX
In-Reply-To: <eij4o3$7l2$2@sea.gmane.org>
References: <20061104222219.5v1dj889q8cgsg8c@webmail.ntnu.no>
	<eij4o3$7l2$2@sea.gmane.org>
Message-ID: <20061105091727.ouibxoltwk40cksg@webmail.ntnu.no>

Siterer Robert Kern <robert.kern at gmail.com>:

> arildna at stud.ntnu.no wrote:
>
>> Python 2.5 is installed, and I have installed Apple's Developer's
>> Tools (i.e. the package "December 2002 Mac OS X Developer Tools").
>
> By the way, this is *really* old. Long before there were Universal binaries
> (which is what Python 2.5 is). I believe you will want to install Xcode 2.4.1
> which was released on Tuesday.
>
> --
> Robert Kern

I was wondering about that, the instructions page on scipy.org wasn't  
very specific. I don't need anything else than Xcode 2.4.1, so I can  
remove the Dec2002 package? And do I need even to run numpy (and not  
scipy)?

    Arild N?ss


-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From robert.kern at gmail.com  Sun Nov  5 03:47:12 2006
From: robert.kern at gmail.com (Robert Kern)
Date: Sun, 05 Nov 2006 02:47:12 -0600
Subject: problems installing NumPy on OSX
In-Reply-To: <20061105091727.ouibxoltwk40cksg@webmail.ntnu.no>
References: <20061104222219.5v1dj889q8cgsg8c@webmail.ntnu.no>	<eij4o3$7l2$2@sea.gmane.org>
	<20061105091727.ouibxoltwk40cksg@webmail.ntnu.no>
Message-ID: <eik8eo$f8l$1@sea.gmane.org>

arildna at stud.ntnu.no wrote:
> Siterer Robert Kern <robert.kern at gmail.com>:
> 
>> arildna at stud.ntnu.no wrote:
>>
>>> Python 2.5 is installed, and I have installed Apple's Developer's
>>> Tools (i.e. the package "December 2002 Mac OS X Developer Tools").
>> By the way, this is *really* old. Long before there were Universal binaries
>> (which is what Python 2.5 is). I believe you will want to install Xcode 2.4.1
>> which was released on Tuesday.
>>
>> --
>> Robert Kern
> 
> I was wondering about that, the instructions page on scipy.org wasn't  
> very specific. I don't need anything else than Xcode 2.4.1, so I can  
> remove the Dec2002 package? And do I need even to run numpy (and not  
> scipy)?

Yes, remove the Dec2002 package and install Xcode 2.4.1.

-- 
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless enigma
 that is made terrible by our own mad attempt to interpret it as though it had
 an underlying truth."
  -- Umberto Eco


-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From Mailer-Daemon at lists.sourceforge.net  Sun Nov  5 05:35:54 2006
From: Mailer-Daemon at lists.sourceforge.net (Mail Delivery System)
Date: Sun, 05 Nov 2006 02:35:54 -0800
Subject: Mail delivery failed: returning message to sender
Message-ID: <E1GgfM2-0002Lb-2g@externalmx-1.sourceforge.net>

This message was created automatically by mail delivery software.

A message that you sent could not be delivered to one or more of its
recipients. This is a permanent error. The following address(es) failed:

  numpy-discussion at lists.sourceforge.net
    SMTP error from remote mailer after RCPT TO:<numpy-discussion at lists.sourceforge.net>:
    host mail.sourceforge.net [66.35.250.206]: 451-Could not complete sender verify callout
    451-Could not complete sender verify callout for
    451-<numpy-discussion at lists.sourceforge.net>.
    451-The mail server(s) for the domain may be temporarily unreachable, or
    451-they may be permanently unreachable from this server. In the latter case,
    451-you need to change the address or create an MX record for its domain
    451-if it is : retry timeout exceeded

------ This is a copy of the message, including all the headers. ------
------ The body of the message is 45596 characters long; only the first
------ 16384 or so are included here.


From lspjsa at ohioflyers.com  Wed Nov  1 02:47:13 2006
From: lspjsa at ohioflyers.com (Alan White)
Date: Wed, 01 Nov 2006 15:46:73 +0800
Subject: looked like me the full sized HARLEY and began to her, with this but as
Message-ID: <000a01c6fd89$e64f9580$0199aa50@qhjxhpymmkur>

An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20061116/843899ef/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: jckbnhtc.png
Type: image/png
Size: 11480 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20061116/843899ef/attachment-0001.png>

From lists.steve at arachnedesign.net  Sun Nov  5 11:16:18 2006
From: lists.steve at arachnedesign.net (Steve Lianoglou)
Date: Sun, 5 Nov 2006 11:16:18 -0500
Subject: problems installing NumPy on OSX
In-Reply-To: <20061105091727.ouibxoltwk40cksg@webmail.ntnu.no>
References: <20061104222219.5v1dj889q8cgsg8c@webmail.ntnu.no>
	<eij4o3$7l2$2@sea.gmane.org>
	<20061105091727.ouibxoltwk40cksg@webmail.ntnu.no>
Message-ID: <CA13C03A-6B1C-483B-B9FF-66965D9E8DAF@arachnedesign.net>

Hi,

> And do I need even to run numpy (and not scipy)?

I'm not sure what you mean with you last question here, but just in  
case it's not clear:

If you want to use scipy, you need to install numpy.
You do *not* have to install scipy if all you need is numpy.

-steve

-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From Mailer-Daemon at lists.sourceforge.net  Sun Nov  5 12:45:22 2006
From: Mailer-Daemon at lists.sourceforge.net (Mail Delivery System)
Date: Sun, 05 Nov 2006 09:45:22 -0800
Subject: Warning: message 1Ggbsf-00056T-Ol delayed 10
	hours
Message-ID: <E1Ggm3e-0005Gu-LP@externalmx-1.sourceforge.net>

This message was created automatically by mail delivery software.
A message that you sent has not yet been delivered to one or more of its
recipients after more than 10 hours on the queue on externalmx-1.sourceforge.net.

The message identifier is:     1Ggbsf-00056T-Ol
The subject of the message is: faces and their former position.  To be, sure.  But question
The date of the message is:    Sun, 05 Nov 2006 16:01:50 +0900

The address to which the message has not yet been delivered is:

  numpy-discussion at lists.sourceforge.net
    Delay reason: SMTP error from remote mailer after RCPT TO:<numpy-discussion at lists.sourceforge.net>:
    host mail.sourceforge.net [66.35.250.206]: 451-Could not complete sender verify callout
    451-Could not complete sender verify callout for
    451-<numpy-discussion at lists.sourceforge.net>.
    451-The mail server(s) for the domain may be temporarily unreachable, or
    451-they may be permanently unreachable from this server. In the latter case,
    451-you need to change the address or create an MX record for its domain
    451-if it is 

No action is required on your part. Delivery attempts will continue for
some time, and this warning may be repeated at intervals if the message
remains undelivered. Eventually the mail delivery software will give up,
and when that happens, the message will be returned to you.

-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From arildna at stud.ntnu.no  Sun Nov  5 13:32:59 2006
From: arildna at stud.ntnu.no (arildna at stud.ntnu.no)
Date: Sun,  5 Nov 2006 19:32:59 +0100
Subject: problems installing NumPy on OSX
In-Reply-To: <CA13C03A-6B1C-483B-B9FF-66965D9E8DAF@arachnedesign.net>
References: <20061104222219.5v1dj889q8cgsg8c@webmail.ntnu.no>
	<eij4o3$7l2$2@sea.gmane.org>
	<20061105091727.ouibxoltwk40cksg@webmail.ntnu.no>
	<CA13C03A-6B1C-483B-B9FF-66965D9E8DAF@arachnedesign.net>
Message-ID: <20061105193259.lbxo4gkzk44wsw4c@webmail.ntnu.no>

Siterer Steve Lianoglou <lists.steve at arachnedesign.net>:

> Hi,
>
>> And do I need even to run numpy (and not scipy)?
>
> I'm not sure what you mean with you last question here, but just in
> case it's not clear:
>
> If you want to use scipy, you need to install numpy.
> You do *not* have to install scipy if all you need is numpy.
>
> -steve

I'm sorry, I was a tad too quick typing there. I meant to say "And do  
I even need to [install Xcode] to run numpy?" Robert pointed out that  
a lot things mentioned in the install guide were necessary to run  
scipy, but that you could run numpy without them.

Therefore I was wondering if installing the newest Xcode package was  
likely to fix the error message I am now getting when trying to  
install numpy:

  File "numpy/core/setup.py", line 50, in generate_config_h
     raise "ERROR: Failed to test configuration"
ERROR: Failed to test configuration

    Arild


-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From lists.steve at arachnedesign.net  Sun Nov  5 14:26:49 2006
From: lists.steve at arachnedesign.net (Steve Lianoglou)
Date: Sun, 5 Nov 2006 14:26:49 -0500
Subject: problems installing NumPy on OSX
In-Reply-To: <20061105193259.lbxo4gkzk44wsw4c@webmail.ntnu.no>
References: <20061104222219.5v1dj889q8cgsg8c@webmail.ntnu.no>
	<eij4o3$7l2$2@sea.gmane.org>
	<20061105091727.ouibxoltwk40cksg@webmail.ntnu.no>
	<CA13C03A-6B1C-483B-B9FF-66965D9E8DAF@arachnedesign.net>
	<20061105193259.lbxo4gkzk44wsw4c@webmail.ntnu.no>
Message-ID: <B5D87503-56A4-462D-B9F2-7EB5F303A5A8@arachnedesign.net>

> I'm sorry, I was a tad too quick typing there. I meant to say "And do
> I even need to [install Xcode] to run numpy?" Robert pointed out that
> a lot things mentioned in the install guide were necessary to run
> scipy, but that you could run numpy without them.
>
> Therefore I was wondering if installing the newest Xcode package was
> likely to fix the error message I am now getting when trying to
> install numpy:

I think Robert may have suggested to install the newest XCode because  
it will give you a newer gcc that can have a better chance compiling  
numpy correctly (or at least will remove another "unkown" to help  
find your true problem).

Maybe there'd be some "Universal Binary-aware"ness that the old xcode  
gcc might be missing that you'll get w/ the new one and since Python  
2.5 is universal, this might be it. Getting the new xcode would be  
the simplest part of the install anyway, so .. why not :-)

-steve



-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From oyffuwebeb at orchardhillchurch.com  Sun Nov  5 23:22:19 2006
From: oyffuwebeb at orchardhillchurch.com (Mambo Mazurka)
Date: Mon, 6 Nov 2006 12:22:19 +0800
Subject: playing
Message-ID: <000e01c7015b$264e8400$da525cde@xswangxl>


Applied diagram effects valueit.
Em in Poker Chip of set.
Holiday Dinner of Kidscope Comic of Playskool Monday or appliance Chocolate am.
Readme License Third Party Licenses jre. Concerti grossi Country.
Puppy Ziggies of yes! Express Sample Robson dvd Video. Sale increases of Booksh increase in if.
Stores Songs Plays hrs. Amp Malicetip Poolspite Maliceshow raquo gv.
Cellfire Hundreds Freebies in Victorias Secret Gift Card? Usedas per am share value balance or sheet divided is number or?
Multimedia Memory gb. Shotgun g greasegun mp ppsh.
Own decrease of more paid them of was received originally out.
Multimedia Memory gb. May or left over owners after of debts paid?
Cabaret Candombe Carnatic Chimurenga Choro Christian. Gas or Toppik Hair Building Airline is.
Xpatches Downloads Extension.
Multimedia Memory gb. June newsabout ushome Hunting jobs Looking Welcome is Blogjob.
Share in value balance sheet divided number shares at date. Em in Poker Chip of set.
Eligible barring regional in local post resume in. Views in job Seekers. Including provided Content!  Begging in mercyzuma quick hand of.
Industries a trade am lower. Views in job Seekers.  Loss Trial of Texas Hold em Poker Chip.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20061106/4d86172b/attachment-0001.html>

From eydntzopip at parenting.com  Sun Nov  5 23:34:45 2006
From: eydntzopip at parenting.com (Engineer Peter)
Date: Mon, 6 Nov 2006 12:34:45 +0800
Subject: love ones.
Message-ID: <000d01c7015c$e2616080$313da4dc@4a5d5db1e75a411>


Signup am Today am Summaries Sites.
Next a year Board found themselves a.
Chopper bicyclehow sew circuit knee Brewee fc a.
Pardoned because didnt ram Jethmalani defend. Indies experience or mattered.
Resonates people age beset a scandal war is. Hang Santosh was faulty he pardoned of because didnt ram. Storozuka Allena rob Farrella Carettaa Doug.
Archives of Editorial Letters am Editor Sunday in Deep Focus? Page play many is like Todays Crossword ways or bring am?
Sexual Immorality Exit Polls win of Night Games Predicted Problems? Racers causes teams join forces Detective Jeffries rookie am murder.
Wonderful conference of over it time start planning in next in year. Entry Blog Youll find date happenings Maker.
Playlist cb in Russell Crowe Temper a.
Wonderful conference of over it time start planning in next in year. To families interested in the care.
End or summer kids will soon be. Us trying do in this in without them.
Nanak Jayanti Champions Trophy victors Bagchi am.
Menopause Womens a Heart Disease see Nelson Cares. Clinics sign contract review Denver want let.
Love ones size of pin. Next a year Board found themselves a.
Craft Robert Redford. Bcs Sometimes of past secrets Cold Case of Learn. Friends love or ones size pin rdquo Only Shipping in Included or.  Night of Games Predicted Problems gop Wins!
Team a topdown in David Fortier Engineer a Peter Gormley general Manager! Bcs Sometimes of past secrets Cold Case of Learn.  Nov Updated at hrs ist Searchin India Indiatimes!
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20061106/10df1e6d/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: St.gif
Type: image/gif
Size: 7843 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20061106/10df1e6d/attachment-0001.gif>
-------------- next part --------------
-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
-------------- next part --------------
_______________________________________________
Numpy-discussion mailing list
Numpy-discussion at lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/numpy-discussion

From Mailer-Daemon at lists.sourceforge.net  Mon Nov  6 04:06:29 2006
From: Mailer-Daemon at lists.sourceforge.net (Mail Delivery System)
Date: Mon, 06 Nov 2006 01:06:29 -0800
Subject: Warning: message 1Ggbsf-00056T-Ol delayed 24
	hours
Message-ID: <E1Gh0R3-0004Up-Or@externalmx-1.sourceforge.net>

This message was created automatically by mail delivery software.
A message that you sent has not yet been delivered to one or more of its
recipients after more than 24 hours on the queue on externalmx-1.sourceforge.net.

The message identifier is:     1Ggbsf-00056T-Ol
The subject of the message is: faces and their former position.  To be, sure.  But question
The date of the message is:    Sun, 05 Nov 2006 16:01:50 +0900

The address to which the message has not yet been delivered is:

  numpy-discussion at lists.sourceforge.net
    Delay reason: SMTP error from remote mailer after RCPT TO:<numpy-discussion at lists.sourceforge.net>:
    host mail.sourceforge.net [66.35.250.206]: 451-Could not complete sender verify callout
    451-Could not complete sender verify callout for
    451-<numpy-discussion at lists.sourceforge.net>.
    451-The mail server(s) for the domain may be temporarily unreachable, or
    451-they may be permanently unreachable from this server. In the latter case,
    451-you need to change the address or create an MX record for its domain
    451-if it is 

No action is required on your part. Delivery attempts will continue for
some time, and this warning may be repeated at intervals if the message
remains undelivered. Eventually the mail delivery software will give up,
and when that happens, the message will be returned to you.

-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From Chris.Barker at noaa.gov  Mon Nov  6 12:37:57 2006
From: Chris.Barker at noaa.gov (Christopher Barker)
Date: Mon, 06 Nov 2006 09:37:57 -0800
Subject: problems installing NumPy on OSX
In-Reply-To: <eik8eo$f8l$1@sea.gmane.org>
References: <20061104222219.5v1dj889q8cgsg8c@webmail.ntnu.no>
	<eij4o3$7l2$2@sea.gmane.org>
	<20061105091727.ouibxoltwk40cksg@webmail.ntnu.no>
	<eik8eo$f8l$1@sea.gmane.org>
Message-ID: <454F72F5.2020009@noaa.gov>


I think it's time to get MacPython2.5 and a set up packages on PythonMac 
org, and build a numpy for it (and SciPy and Matplotlib too...)

Bob, can you set up the page, and then we can start populating it?

-Chris


-- 
Christopher Barker, Ph.D.
Oceanographer
                                     		
NOAA/OR&R/HAZMAT         (206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115       (206) 526-6317   main reception

Chris.Barker at noaa.gov

-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From Chris.Barker at noaa.gov  Mon Nov  6 12:41:18 2006
From: Chris.Barker at noaa.gov (Christopher Barker)
Date: Mon, 06 Nov 2006 09:41:18 -0800
Subject: problems installing NumPy on OSX
In-Reply-To: <20061105193259.lbxo4gkzk44wsw4c@webmail.ntnu.no>
References: <20061104222219.5v1dj889q8cgsg8c@webmail.ntnu.no>
	<eij4o3$7l2$2@sea.gmane.org>
	<20061105091727.ouibxoltwk40cksg@webmail.ntnu.no>
	<CA13C03A-6B1C-483B-B9FF-66965D9E8DAF@arachnedesign.net>
	<20061105193259.lbxo4gkzk44wsw4c@webmail.ntnu.no>
Message-ID: <454F73BE.30503@noaa.gov>

arildna at stud.ntnu.no wrote:
> I'm sorry, I was a tad too quick typing there. I meant to say "And do  
> I even need to [install Xcode] to run numpy?"

You need Xcode to build numpy (or anything else). If you can find a 
binary, then you should be able to just run that. One of us should get a 
binary for 2.5 out there soon.

by the way, if you can run 2.4 instead, you'll find a lot of binaries 
for various packages here:

http://www.pythonmac.org/packages/py24-fat/index.html

-Chris

-- 
Christopher Barker, Ph.D.
Oceanographer
                                     		
NOAA/OR&R/HAZMAT         (206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115       (206) 526-6317   main reception

Chris.Barker at noaa.gov

-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From bob at redivi.com  Mon Nov  6 12:42:31 2006
From: bob at redivi.com (Bob Ippolito)
Date: Mon, 6 Nov 2006 09:42:31 -0800
Subject: problems installing NumPy on OSX
In-Reply-To: <454F72F5.2020009@noaa.gov>
References: <20061104222219.5v1dj889q8cgsg8c@webmail.ntnu.no>
	<eij4o3$7l2$2@sea.gmane.org>
	<20061105091727.ouibxoltwk40cksg@webmail.ntnu.no>
	<eik8eo$f8l$1@sea.gmane.org> <454F72F5.2020009@noaa.gov>
Message-ID: <6a36e7290611060942x4b3bb30am56aaec599d14d090@mail.gmail.com>

On 11/6/06, Christopher Barker <Chris.Barker at noaa.gov> wrote:
>
> I think it's time to get MacPython2.5 and a set up packages on PythonMac
> org, and build a numpy for it (and SciPy and Matplotlib too...)
>
> Bob, can you set up the page, and then we can start populating it?

I'll put it up when there's something to put there. Doesn't make sense
to have an empty page.

-bob

-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From aisaac at american.edu  Mon Nov  6 15:09:12 2006
From: aisaac at american.edu (Alan Isaac)
Date: Mon, 06 Nov 2006 15:09:12 -0500
Subject: compile numpy on Mac
Message-ID: <454F9668.5030007@american.edu>

Experience of a brand new Mac user.
Barebones numpy install on Python 2.5 on a MacBook.

1. Where is my shell?

By default, bash is 'Terminal.app' in you applications folder.

2. Where is my editor?

You have many choices, but you can always start 'vim' from your bash shell.
Howevever, I just used the default configuration,
so no editor was needed. (Comment: I do not know if
this was a good decision, but I ended up with a working numpy.)

3. Where is my compiler?

The default OSX install does not include a compiler.
Go to http://developer.apple.com/tools/xcode and download the most 
recent Xcode.
(This is a very large download.)
Follow the easy installation directions in the accompanying 'readme' file.
(You must have administrative privileges.)

You will have to sign up for a free ADC account.

4. How to compile?

 From your bash shell, change to your build directory.
(I.e., wherever you unzipped your numpy tarball to.)
Do the usual:
python setup.py install
Compilation and installation takes a few minutes.
Then exit your shell.
You should be ready to go.


-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From cjw at sympatico.ca  Mon Nov  6 15:10:20 2006
From: cjw at sympatico.ca (Colin J. Williams)
Date: Mon, 06 Nov 2006 15:10:20 -0500
Subject: Style - was Re: numpy.repeat TypeError:
 array cannot be safely cast to required type
In-Reply-To: <454CF6AF.10205@ieee.org>
References: <1162581054.217811.174140@k70g2000cwa.googlegroups.com>	<454B9BDD.2060101@ieee.org>	<454C87CF.7030500@sympatico.ca>
	<454CF6AF.10205@ieee.org>
Message-ID: <454F96AC.5020003@sympatico.ca>

An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20061106/fbcf9f68/attachment-0001.html>
-------------- next part --------------
-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
-------------- next part --------------
_______________________________________________
Numpy-discussion mailing list
Numpy-discussion at lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/numpy-discussion

From jdhunter at ace.bsd.uchicago.edu  Mon Nov  6 15:09:32 2006
From: jdhunter at ace.bsd.uchicago.edu (John Hunter)
Date: Mon, 06 Nov 2006 14:09:32 -0600
Subject: ctypes warning
Message-ID: <87lkmofibn.fsf@peds-pc311.bsd.uchicago.edu>


A simple import of numpy with the latest svn triggers a ctypes warning

In [1]: import numpy
/usr/lib/python2.4/site-packages/numpy/ctypeslib.py:12: UserWarning:
All features of ctypes interface may not work with ctypes < 1.0.1
  warnings.warn("All features of ctypes interface may not work with "
 

This is a bit of an annoyance.  Even if the warning level is
configurable, I think the bulk of matplotlib users, who get the
warning when using numpy through mpl but make no use of ctypes, will
be confused by this.  Is it right and good that this message gets
triggered by default for any import of numpy, even those not using the
ctypes features?

JDH

-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From stefan at sun.ac.za  Mon Nov  6 16:23:17 2006
From: stefan at sun.ac.za (Stefan van der Walt)
Date: Mon, 6 Nov 2006 23:23:17 +0200
Subject: ctypes warning
In-Reply-To: <87lkmofibn.fsf@peds-pc311.bsd.uchicago.edu>
References: <87lkmofibn.fsf@peds-pc311.bsd.uchicago.edu>
Message-ID: <20061106212317.GH3380@mentat.za.net>

On Mon, Nov 06, 2006 at 02:09:32PM -0600, John Hunter wrote:
> 
> A simple import of numpy with the latest svn triggers a ctypes warning
> 
> In [1]: import numpy
> /usr/lib/python2.4/site-packages/numpy/ctypeslib.py:12: UserWarning:
> All features of ctypes interface may not work with ctypes < 1.0.1
>   warnings.warn("All features of ctypes interface may not work with "
>  
> 
> This is a bit of an annoyance.  Even if the warning level is
> configurable, I think the bulk of matplotlib users, who get the
> warning when using numpy through mpl but make no use of ctypes, will
> be confused by this.  Is it right and good that this message gets
> triggered by default for any import of numpy, even those not using the
> ctypes features?

I moved the warning to inside ctypeslib.load_library.  I think most
people using ctypes in numpy come via that route.  Soon, we can remove
the warning, but for now I think it should remain -- the bugs
resulting from using older versions of ctypes are fairly hard to track
down.

Regards
St?fan

-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From charlesr.harris at gmail.com  Mon Nov  6 16:30:30 2006
From: charlesr.harris at gmail.com (Charles R Harris)
Date: Mon, 6 Nov 2006 14:30:30 -0700
Subject: Style - was Re: numpy.repeat TypeError:
	array cannot be safely cast to required type
In-Reply-To: <454F96AC.5020003@sympatico.ca>
References: <1162581054.217811.174140@k70g2000cwa.googlegroups.com>
	<454B9BDD.2060101@ieee.org> <454C87CF.7030500@sympatico.ca>
	<454CF6AF.10205@ieee.org> <454F96AC.5020003@sympatico.ca>
Message-ID: <e06186140611061330v41f3ce65o3d594f2a565bbd5e@mail.gmail.com>

On 11/6/06, Colin J. Williams <cjw at sympatico.ca> wrote:
<snip>

Nobody has proposed using English yet - zeroes.
>

It seemed a bridge too far.

Chuck
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20061106/b3ea8e49/attachment-0001.html>
-------------- next part --------------
-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
-------------- next part --------------
_______________________________________________
Numpy-discussion mailing list
Numpy-discussion at lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/numpy-discussion

From stefan at sun.ac.za  Mon Nov  6 16:40:45 2006
From: stefan at sun.ac.za (Stefan van der Walt)
Date: Mon, 6 Nov 2006 23:40:45 +0200
Subject: Style - was Re: numpy.repeat TypeError:
	array cannot be safely cast to required type
In-Reply-To: <454F96AC.5020003@sympatico.ca>
References: <1162581054.217811.174140@k70g2000cwa.googlegroups.com>
	<454B9BDD.2060101@ieee.org> <454C87CF.7030500@sympatico.ca>
	<454CF6AF.10205@ieee.org> <454F96AC.5020003@sympatico.ca>
Message-ID: <20061106214045.GI3380@mentat.za.net>

On Mon, Nov 06, 2006 at 03:10:20PM -0500, Colin J. Williams wrote:
> Many thanks.  In general, there is sense in the Python dictum about having one
> way to do things.  Although, in this case [length] vs length for one dimension
> doesn't
> exercise me greatly.  I would be a bit more concerned about synonyms.
> 
> Nobody has proposed using English yet - zeroes.

If Cambridge Advanced Learner's is happy with zeros, then so we should
be:

http://dictionary.cambridge.org/define.asp?key=92164&dict=CALD

zero   Show phonetics
noun plural zeros or zeroes
1 [C or U] (the number) 0; nothing:

I think the American-english speakers on the list outnumber the rest
of us, so we'd better be careful :)

Cheers
St?fan

-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From oliphant at ee.byu.edu  Mon Nov  6 17:14:16 2006
From: oliphant at ee.byu.edu (Travis Oliphant)
Date: Mon, 06 Nov 2006 15:14:16 -0700
Subject: Scholarly contribution of NumPy : request for
	review
Message-ID: <454FB3B8.4030108@ee.byu.edu>

I apologize for this bit of spam.

If there is anyone on this list (preferrably of Associate Professor or 
similar rank) who can provide an honest assessment of the scholarly 
contribution of NumPy and/or SciPy (and has not already done so)? 

If you can do this and don't mind me quoting you to a tenure-review 
committee, could you please email me your assessment by Friday morning 
(Nov. 10).   Please include information about your position in the 
mail.  I welcome all assessments (both positive and negative). 

Thank you very much.


Travis Oliphant
Assistant Professor
Brigham Young University
oliphant at ee.byu.edu
801-422-3108






-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From laidler at stsci.edu  Mon Nov  6 17:59:54 2006
From: laidler at stsci.edu (Victoria G. Laidler)
Date: Mon, 06 Nov 2006 17:59:54 -0500
Subject: Style - was Re: numpy.repeat
 TypeError:	array cannot be safely cast to required type
In-Reply-To: <20061106214045.GI3380@mentat.za.net>
References: <1162581054.217811.174140@k70g2000cwa.googlegroups.com>	<454B9BDD.2060101@ieee.org>
	<454C87CF.7030500@sympatico.ca>	<454CF6AF.10205@ieee.org>
	<454F96AC.5020003@sympatico.ca>
	<20061106214045.GI3380@mentat.za.net>
Message-ID: <454FBE6A.4000704@stsci.edu>

Stefan van der Walt wrote:
> On Mon, Nov 06, 2006 at 03:10:20PM -0500, Colin J. Williams wrote:
>   
>> Many thanks.  In general, there is sense in the Python dictum about having one
>> way to do things.  Although, in this case [length] vs length for one dimension
>> doesn't
>> exercise me greatly.  I would be a bit more concerned about synonyms.
>>
>> Nobody has proposed using English yet - zeroes.
>>     
>
> If Cambridge Advanced Learner's is happy with zeros, then so we should
> be:
>
> http://dictionary.cambridge.org/define.asp?key=92164&dict=CALD
>
> zero   Show phonetics
> noun plural zeros or zeroes
> 1 [C or U] (the number) 0; nothing:
>
> I think the American-english speakers on the list outnumber the rest
> of us, so we'd better be careful :)
>   
You know, I spell this wrong at least every other time I use it.
But of course, if we changed it to "zeroes", I'd -still- spell it wrong 
every other time I used it.

Are synonyms really *always* evil????

wistfully,
Vicki Laidler

-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From fperez.net at gmail.com  Mon Nov  6 18:38:54 2006
From: fperez.net at gmail.com (Fernando Perez)
Date: Mon, 6 Nov 2006 16:38:54 -0700
Subject: Strange and hard to reproduce crash
In-Reply-To: <453D4AD1.5060508@ieee.org>
References: <db6b5ecc0610231439o1217af98j11dcb45e2ae5e505@mail.gmail.com>
	<453D3DD7.50506@ieee.org>
	<db6b5ecc0610231544g7413491du9e8b0bd6c8c20e6f@mail.gmail.com>
	<453D4AD1.5060508@ieee.org>
Message-ID: <db6b5ecc0611061538q46ba6efoc82aacfd73f10085@mail.gmail.com>

On 10/23/06, Travis Oliphant <oliphant.travis at ieee.org> wrote:

> I've placed them in SVN (r3384):
>
> arraydescr_dealloc needs to do something like.
>
> if (self->fields == Py_None) {
>     print something
>     incref(self)
>     return;
> }

Travis, I know you're busy right now, so this message is just so that
the archives have this info, for whenever you revisit the problem.

A long run of our code is now producing the following output:

*** Reference count error detected:
an attempt was made to deallocate 12 (d) ***
*** Reference count error detected:
an attempt was made to deallocate 12 (d) ***

etc.

Thanks to your changes it does not crash anymore, so it's not a big
deal for us.  Whenever you want further details, I can try to collect
them.

Regards,

f

-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From dlaney at llnl.gov  Mon Nov  6 20:57:51 2006
From: dlaney at llnl.gov (dlaney at llnl.gov)
Date: Mon, 6 Nov 2006 17:57:51 -0800 (PST)
Subject: VisIt Usage at LLNL
Message-ID: <Pine.LNX.4.64.0611061748360.32554@tux219.llnl.gov>

Paul Dubois asked me to post a short summary about how VisIt is used
at LLNL.

First, I am not a VisIt developer, but I have worked with it and those
who use it in day to day work.  I have also used it to generate high
resolution movies of medium to extremely large (~ 4 TeraByte) data sets.

VisIt is based on VTK (Visualization Tool Kit, check their webpage via
google search).  However, several features have been re-implemented for
performance reasons or because they did (or do not) exist in VTK.

VisIt is open source, but at present time does not have a true open source
development model (no sourceforge home page for example).  This may change in
the future, but I recommend contacting the VisIt team directly to find out
about official plans in this direction.

>From my perspective, VisIt is used in three ways at LLNL:
   1) Debugging    (GUI-based, but could use the Python interface)
   2) Exploration  (GUI-based)
   3) Movies       (Usually heavier use of the python interface)

Debugging:

    VisIt contains many operators that can be used to find bad cells and and
    display information about them.  A typical debugging session might start
    with a bad zonal quantity.  The user first creates plots of fields of
    interest using pseudocolor plots for example.  Then the user applies an
    OnionPeel operator and enters the domain and zone number from the error
    message generated by the code.  The user can then vary the number of
    additional layers of cells displayed around the bad cell.
    Finally, this small set of cells can be picked or queried, and investigated
    at different time steps.

    VisIt can also plot various quantities over time.  The expression engine
    in VisIt can be used to compute many derived quantities.

    VisIt also handles cells containing multiple materials in a robust way.

    There is interest in having physics codes dump Python
    scripts for VisIt that would automatically visualize errors in
    simulations.

Exploration:

    VisIt supports a large set of expressions, including the expected
    arithmetic operations along with vector operations (cross, dot, etc.),
    conditional operations (if-then-else), and many other useful functions.
    This allows users to create a large set of derived fields and then
    interact with them in VisIt like any other field.  I have seen users
    generate quite interesting plots by carefully creating derived quantities
    from the raw simulation dump files.

    VisIt has a nice set of functionality for creating 2-D plots.

Movies:

    The way I typically approach the problem of generating a movie of a given
    visualization with a static view-point is as follows.

    1) First I set up the visualization with the domain scientist(s).  This
       process is iterative by nature and is entirely GUI based.  VisIt can save
       'session' files that record everything about a given session, allowing me
       to create multiple visualizations and re-use them later.

    2) Once everything is set up, VisIt can generate the movie from within
       the GUI, or the session file can be used in a python script to drive
       VisIt.  I usually use the Python interface because that gives direct
       control over camera position and other features.  For large movies,
       the frames must be generated on a cluster and the batch system is
       necessary.  Python can be used to build scripts that play nice with
       the batch-system at LLNL.

    It is also possible to build an entire movie using Python by specifying
    exactly which operations to apply.  The end-result is a self-contained
    script for generating frames.  However, this is a labor intensive
    approach and is only needed for movies with complex camera movements or
    other effects.  Even for such movies, VisIt has a key-framing system
    that can handle some common cases for movies.

For more information I suggest emailing the VisIt team directly at:
visit-help at llnl.gov

Regards

-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From nzlofmyj at pacificsportsllc.com  Mon Nov  6 21:18:02 2006
From: nzlofmyj at pacificsportsllc.com (near)
Date: Tue, 7 Nov 2006 11:18:02 +0900
Subject: EntryCan Monkey Posted //Comments
Message-ID: <000701c70212$f3cb43c0$66b1bad3@a>


Processes strataud mustreads a Franz Woyzeck.
Trends is comics a juvenile pranks practical jokes you. Many find idiotic everyone else waste is.
When a insult of reveals a inner.
Mission a Calendar Stickers Concerts!
Sharon leaves icu or condition improves Israeli lawmaker sues in Iran is. Inc Maxim a Dennis Publishing Esquire Hearst in Blender.
Ship they is qualify of rest a order mailing address. Bb upi discuss of demands stumps.
Primary audience or age group am many of find of idiotic of everyone else. Average one am is reviewswhy in pay full!
Fhm Total Today Customers a bought is also Emap! Md its strategy!
Florida Arkansas Texas or amahold cut a Giants prices sell.
Sgt Shaft am Culture etc World! Atop Taishan Huangshan Picks here less buggy. Japan Hiroko clamps is James officers. Pricing Martin Baxter collected or actual is!
Florida Arkansas Texas or amahold cut a Giants prices sell.
Tn sister in magazine several ways Both feature silly. Company owns Giant said of today.
Say months ugly battles tomorrows will.
Involves lot analysis Having practiced downright easy breezed rounds is.
Can shipped within Review.
List analytic of before reading los Angeles targeted. Trends is comics a juvenile pranks practical jokes you.
Bytimothy sellers in Offers Unknown Binding Version Finance is Voitle.
Quant Physics Emanuel Derman Fixed Income Markets is.
Journals finance degrees diploma dr taught university or level. Quant Physics Emanuel Derman Fixed Income Markets is.
Simple a cases appear complex teaches examining.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20061107/cdb54895/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: Join.gif
Type: image/gif
Size: 8379 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20061107/cdb54895/attachment-0001.gif>
-------------- next part --------------
-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
-------------- next part --------------
_______________________________________________
Numpy-discussion mailing list
Numpy-discussion at lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/numpy-discussion

From david at ar.media.kyoto-u.ac.jp  Mon Nov  6 21:23:29 2006
From: david at ar.media.kyoto-u.ac.jp (David Cournapeau)
Date: Tue, 07 Nov 2006 11:23:29 +0900
Subject: Passing numpy arrays to matlab
Message-ID: <454FEE21.7040205@ar.media.kyoto-u.ac.jp>

Hi,

    I am trying to find a nice way to communicate between matlab and 
python. I am aware of pymat, which does that, but the code is 
deprecated, and I thing basing the code on ctypes would lead to much 
more robust code.

    http://claymore.engineer.gvsu.edu/%7Esteriana/Software/pymat.html

I have a really simple prototype which can send and get back data from 
matlab, but I was wondering if it would be possible to use a scheme 
similar to ctypes instead of having to convert it by hand. Let me 
present the way communication with matlab is done:

    - open a matlab session, which on unix launch a matlab process, and 
set up pipes between the calling process and matlab process for 
communication
    - To send data from the calling process to matlab, you first have to 
create a mxArray, which is the basic matlab handler of a matlab array, 
and populating it. Using mxArray is very ackward : you cannot create 
mxArray from existing data, you have to copy data to them, etc... (it is 
one of the reason I started looking in python in the first place: 
interfacing matlab with foreign code is really not a nice experience, 
and the API is not rich enough to do many interesting things; I am 
actually amazed how poor this part of matlab is, a product which is now 
20 years old).

    I was wondering if there was a way to extend a numpy array so that I 
could send directly numpy arrays to matlab C functions expecting a , 
with ctypes doing the hard work. For example, now, to send data to 
matlab, I do:

    session   = MatlabEngine()
    # data is a numpy array, 'dataname' a string with its name inside 
matlab interpreter
    session.put(data, 'dataname')

The put function has do to a lot for work:
    - first, getting metadata from data (dimensions, real or complex, 
etc...)
    - then creating a mxArray with the same metadata
    - then populating the mxarray by copying the data from the numpy array.

And of course, taking care that all mxarrays are detroyed, otherwise, 
memory leak... That's why I was thinking about something a bit smarter: 
creating a mxarray class which implements the numpy interface. I could 
create a mxarray from a numpy array easily, send a mxarray directly to C 
function with ctypes, etc... Is this doable ? Would this take care of 
correct destruction of mxarrays ? I really don't know much about the 
internals of numpy arrays, so I don't really know how to start,

    cheers,

    David

-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From strawman at astraw.com  Mon Nov  6 23:56:18 2006
From: strawman at astraw.com (Andrew Straw)
Date: Mon, 06 Nov 2006 20:56:18 -0800
Subject: ctypes warning
In-Reply-To: <20061106212317.GH3380@mentat.za.net>
References: <87lkmofibn.fsf@peds-pc311.bsd.uchicago.edu>
	<20061106212317.GH3380@mentat.za.net>
Message-ID: <455011F2.7010204@astraw.com>

Stefan van der Walt wrote:
> On Mon, Nov 06, 2006 at 02:09:32PM -0600, John Hunter wrote:
>   
>> A simple import of numpy with the latest svn triggers a ctypes warning
>>
>> In [1]: import numpy
>> /usr/lib/python2.4/site-packages/numpy/ctypeslib.py:12: UserWarning:
>> All features of ctypes interface may not work with ctypes < 1.0.1
>>   warnings.warn("All features of ctypes interface may not work with "
>>  
>>
>> This is a bit of an annoyance.  Even if the warning level is
>> configurable, I think the bulk of matplotlib users, who get the
>> warning when using numpy through mpl but make no use of ctypes, will
>> be confused by this.  Is it right and good that this message gets
>> triggered by default for any import of numpy, even those not using the
>> ctypes features?
>>     
>
> I moved the warning to inside ctypeslib.load_library.  I think most
> people using ctypes in numpy come via that route. 
I don't come in via that route. How hard is it to trigger on a getattr()
for the .ctypes attribute?

-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From strawman at astraw.com  Tue Nov  7 00:03:18 2006
From: strawman at astraw.com (Andrew Straw)
Date: Mon, 06 Nov 2006 21:03:18 -0800
Subject: Passing numpy arrays to matlab
In-Reply-To: <454FEE21.7040205@ar.media.kyoto-u.ac.jp>
References: <454FEE21.7040205@ar.media.kyoto-u.ac.jp>
Message-ID: <45501396.3070908@astraw.com>

David Cournapeau wrote:
>     - To send data from the calling process to matlab, you first have to 
> create a mxArray, which is the basic matlab handler of a matlab array, 
> and populating it. Using mxArray is very ackward : you cannot create 
> mxArray from existing data, you have to copy data to them, etc... 
My understanding, never having done it, but from reading the docs, is
that you can create a "hybrid array" where you manage the memory. Thus,
you can create an mxArray from existing data. However, the docs
basically say that this is too hard for most mortals (and they may well
be right -- too painful for me, anyway)!

-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From josh.p.marshall at gmail.com  Mon Nov  6 23:59:25 2006
From: josh.p.marshall at gmail.com (Josh Marshall)
Date: Tue, 7 Nov 2006 15:59:25 +1100
Subject: Passing numpy arrays to matlab
In-Reply-To: <mailman.119602.1162866273.22397.numpy-discussion@lists.sourceforge.net>
References: <mailman.119602.1162866273.22397.numpy-discussion@lists.sourceforge.net>
Message-ID: <E50CA0F6-3ED5-4ED0-92BA-B55991001504@gmail.com>

Hi David,

Did you have a look at mlabwrap? It's quite hard to find on the net,  
which is a shame, since it is a much more up to date version,  
enhancing pymat with the things that you are trying to do. It allows  
passing arrays and getting arrays back.

http://mlabwrap.sourceforge.net/

However, I do agree with you that it would be good to have a bridge  
based on ctypes. I can't really help since I long ago moved all of my  
code from Matlab to Numpy and Matplotlib.

Cheers,
Josh M

David Cournapeau <david at ar.media.kyoto-u.ac.jp> wrote:
>
> Hi,
>
>     I am trying to find a nice way to communicate between matlab and
> python. I am aware of pymat, which does that, but the code is
> deprecated, and I thing basing the code on ctypes would lead to much
> more robust code.
>
>     http://claymore.engineer.gvsu.edu/%7Esteriana/Software/pymat.html
>
> I have a really simple prototype which can send and get back data from
> matlab, but I was wondering if it would be possible to use a scheme
> similar to ctypes instead of having to convert it by hand. Let me
> present the way communication with matlab is done:
>
>     - open a matlab session, which on unix launch a matlab process,  
> and
> set up pipes between the calling process and matlab process for
> communication
>     - To send data from the calling process to matlab, you first  
> have to
> create a mxArray, which is the basic matlab handler of a matlab array,
> and populating it. Using mxArray is very ackward : you cannot create
> mxArray from existing data, you have to copy data to them, etc...  
> (it is
> one of the reason I started looking in python in the first place:
> interfacing matlab with foreign code is really not a nice experience,
> and the API is not rich enough to do many interesting things; I am
> actually amazed how poor this part of matlab is, a product which is  
> now
> 20 years old).
>
>     I was wondering if there was a way to extend a numpy array so  
> that I
> could send directly numpy arrays to matlab C functions expecting a ,
> with ctypes doing the hard work. For example, now, to send data to
> matlab, I do:
>
>     session   = MatlabEngine()
>     # data is a numpy array, 'dataname' a string with its name inside
> matlab interpreter
>     session.put(data, 'dataname')
>
> The put function has do to a lot for work:
>     - first, getting metadata from data (dimensions, real or complex,
> etc...)
>     - then creating a mxArray with the same metadata
>     - then populating the mxarray by copying the data from the  
> numpy array.
>
> And of course, taking care that all mxarrays are detroyed, otherwise,
> memory leak... That's why I was thinking about something a bit  
> smarter:
> creating a mxarray class which implements the numpy interface. I could
> create a mxarray from a numpy array easily, send a mxarray directly  
> to C
> function with ctypes, etc... Is this doable ? Would this take care of
> correct destruction of mxarrays ? I really don't know much about the
> internals of numpy arrays, so I don't really know how to start,
>
>     cheers,
>
>     David
>
>


-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From david at ar.media.kyoto-u.ac.jp  Mon Nov  6 23:59:43 2006
From: david at ar.media.kyoto-u.ac.jp (David Cournapeau)
Date: Tue, 07 Nov 2006 13:59:43 +0900
Subject: Passing numpy arrays to matlab
In-Reply-To: <45501396.3070908@astraw.com>
References: <454FEE21.7040205@ar.media.kyoto-u.ac.jp>
	<45501396.3070908@astraw.com>
Message-ID: <455012BF.1030105@ar.media.kyoto-u.ac.jp>

Andrew Straw wrote:
> David Cournapeau wrote:
>   
>>     - To send data from the calling process to matlab, you first have to 
>> create a mxArray, which is the basic matlab handler of a matlab array, 
>> and populating it. Using mxArray is very ackward : you cannot create 
>> mxArray from existing data, you have to copy data to them, etc... 
>>     
> My understanding, never having done it, but from reading the docs, is
> that you can create a "hybrid array" where you manage the memory. Thus,
> you can create an mxArray from existing data. However, the docs
> basically say that this is too hard for most mortals (and they may well
> be right -- too painful for me, anyway)!
>   
Would you mind telling me where you found that information ? Because 
right now, I am wasting a lot of cycles because of memory copy in both 
directions, and it is sometimes slow enough so that it is annoying,

cheers,

David

-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From bur at carrabbas.com  Tue Nov  7 00:26:10 2006
From: bur at carrabbas.com (Fran Kellerhouse)
Date: Mon, 6 Nov 2006 21:26:10 -0800
Subject: new 874
Message-ID: <000001c7022d$3c2d63e0$c316a8c0@hbes>

Hi,

PHAjjRMACY for LESS http://www.mitujunhertshieasde.com

 

Your slightest wish is our command.


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20061106/88de1605/attachment-0001.html>
-------------- next part --------------
-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
-------------- next part --------------
_______________________________________________
Numpy-discussion mailing list
Numpy-discussion at lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/numpy-discussion

From strawman at astraw.com  Tue Nov  7 00:49:36 2006
From: strawman at astraw.com (Andrew Straw)
Date: Mon, 06 Nov 2006 21:49:36 -0800
Subject: Passing numpy arrays to matlab
In-Reply-To: <455012BF.1030105@ar.media.kyoto-u.ac.jp>
References: <454FEE21.7040205@ar.media.kyoto-u.ac.jp>	<45501396.3070908@astraw.com>
	<455012BF.1030105@ar.media.kyoto-u.ac.jp>
Message-ID: <45501E70.8090609@astraw.com>

David Cournapeau wrote:
> Andrew Straw wrote:
>   
>> David Cournapeau wrote:
>>   
>>     
>>>     - To send data from the calling process to matlab, you first have to 
>>> create a mxArray, which is the basic matlab handler of a matlab array, 
>>> and populating it. Using mxArray is very ackward : you cannot create 
>>> mxArray from existing data, you have to copy data to them, etc... 
>>>     
>>>       
>> My understanding, never having done it, but from reading the docs, is
>> that you can create a "hybrid array" where you manage the memory. Thus,
>> you can create an mxArray from existing data. However, the docs
>> basically say that this is too hard for most mortals (and they may well
>> be right -- too painful for me, anyway)!
>>   
>>     
> Would you mind telling me where you found that information ? Because 
> right now, I am wasting a lot of cycles because of memory copy in both 
> directions, and it is sometimes slow enough so that it is annoying,
>   
I found it reading through the in-program help (the C-API section,
whatever it's called) on a Matlab installation at my university. I guess
this was Matlab 2006A. A quick Google search turns this up:

http://www.mathworks.com/access/helpdesk/help/techdoc/matlab_external/index.html?/access/helpdesk/help/techdoc/matlab_external/f25255.html

They give the following example, which seems to create a Matlab array
"pArray" with data owned by the C variable "data":

mxArray *pArray = mxCreateDoubleMatrix(0, 0, mxREAL);
double data[10];

mxSetPr(pArray, data);
mxSetM(pArray, 1);
mxSetN(pArray, 10);


-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From david at ar.media.kyoto-u.ac.jp  Tue Nov  7 00:52:56 2006
From: david at ar.media.kyoto-u.ac.jp (David Cournapeau)
Date: Tue, 07 Nov 2006 14:52:56 +0900
Subject: Passing numpy arrays to matlab
In-Reply-To: <45501E70.8090609@astraw.com>
References: <454FEE21.7040205@ar.media.kyoto-u.ac.jp>	<45501396.3070908@astraw.com>	<455012BF.1030105@ar.media.kyoto-u.ac.jp>
	<45501E70.8090609@astraw.com>
Message-ID: <45501F38.3090201@ar.media.kyoto-u.ac.jp>

Andrew Straw wrote:
> David Cournapeau wrote:
>   
>> Andrew Straw wrote:
>>   
>>     
>>> David Cournapeau wrote:
>>>   
>>>     
>>>       
>>>>     - To send data from the calling process to matlab, you first have to 
>>>> create a mxArray, which is the basic matlab handler of a matlab array, 
>>>> and populating it. Using mxArray is very ackward : you cannot create 
>>>> mxArray from existing data, you have to copy data to them, etc... 
>>>>     
>>>>       
>>>>         
>>> My understanding, never having done it, but from reading the docs, is
>>> that you can create a "hybrid array" where you manage the memory. Thus,
>>> you can create an mxArray from existing data. However, the docs
>>> basically say that this is too hard for most mortals (and they may well
>>> be right -- too painful for me, anyway)!
>>>   
>>>     
>>>       
>> Would you mind telling me where you found that information ? Because 
>> right now, I am wasting a lot of cycles because of memory copy in both 
>> directions, and it is sometimes slow enough so that it is annoying,
>>   
>>     
> I found it reading through the in-program help (the C-API section,
> whatever it's called) on a Matlab installation at my university. I guess
> this was Matlab 2006A. A quick Google search turns this up:
>
> http://www.mathworks.com/access/helpdesk/help/techdoc/matlab_external/index.html?/access/helpdesk/help/techdoc/matlab_external/f25255.html
>
> They give the following example, which seems to create a Matlab array
> "pArray" with data owned by the C variable "data":
>
> mxArray *pArray = mxCreateDoubleMatrix(0, 0, mxREAL);
> double data[10];
>
> mxSetPr(pArray, data);
> mxSetM(pArray, 1);
> mxSetN(pArray, 10);
>   
Thank you very much, I think this added documentation is pretty recent; 
I have never seen it before, and I did a lot a mex programming at some 
point... This whole mxarray nonsense reminds me why I gave up on matlab :),

cheers,

David

-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From Mailer-Daemon at lists.sourceforge.net  Tue Nov  7 04:15:27 2006
From: Mailer-Daemon at lists.sourceforge.net (Mail Delivery System)
Date: Tue, 07 Nov 2006 01:15:27 -0800
Subject: Warning: message 1Ggbsf-00056T-Ol delayed 48
	hours
Message-ID: <E1GhN3H-0004h3-MU@externalmx-1.sourceforge.net>

This message was created automatically by mail delivery software.
A message that you sent has not yet been delivered to one or more of its
recipients after more than 48 hours on the queue on externalmx-1.sourceforge.net.

The message identifier is:     1Ggbsf-00056T-Ol
The subject of the message is: faces and their former position.  To be, sure.  But question
The date of the message is:    Sun, 05 Nov 2006 16:01:50 +0900

The address to which the message has not yet been delivered is:

  numpy-discussion at lists.sourceforge.net
    Delay reason: SMTP error from remote mailer after RCPT TO:<numpy-discussion at lists.sourceforge.net>:
    host mail.sourceforge.net [66.35.250.206]: 451-Could not complete sender verify callout
    451-Could not complete sender verify callout for
    451-<numpy-discussion at lists.sourceforge.net>.
    451-The mail server(s) for the domain may be temporarily unreachable, or
    451-they may be permanently unreachable from this server. In the latter case,
    451-you need to change the address or create an MX record for its domain
    451-if it is 

No action is required on your part. Delivery attempts will continue for
some time, and this warning may be repeated at intervals if the message
remains undelivered. Eventually the mail delivery software will give up,
and when that happens, the message will be returned to you.

-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From olivetti at itc.it  Tue Nov  7 08:29:13 2006
From: olivetti at itc.it (Emanuele Olivetti)
Date: Tue, 07 Nov 2006 14:29:13 +0100
Subject: numpy on wince
Message-ID: <45508A29.6060302@itc.it>

Hi all,
do you know if numpy/Numeric is available on WinCE ?
I'm thinking about the use of a numpy/Numeric app on a palmtop
and I'm not sure it will be deployed on linux only devices.

Thanks in advance,

Emanuele

P.S.: I know that Python is available on many many platforms, but
I not so confident about many external libraries.

-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From gurjptwydsh at otisinstruments.com  Tue Nov  7 09:10:17 2006
From: gurjptwydsh at otisinstruments.com (Miracle Vol.)
Date: Tue, 7 Nov 2006 22:10:17 +0800
Subject: revealed in
Message-ID: <001101c70276$73a55500$c545eadd@e29e69d898f14ac>


Harper lee Charles.
Blogin Creasein Dialon Ranttable in Weighing Incorpus or Christi is Hooksortiz a Becoming. Shelly Lewis East reg District am West Coast Windows.
Heb of Plus signing before a continuing Parkdale.
Costa than developed nations holds is its am certainty god or created. Nov Bonanza Street Locust a streets Walnut Creek. News Archives web for Jobs Cars am.
Say they checks but touched hug. Nineteen feature of films have a!
Project Cannon crew is kick ass?
Everything better Thats secret revealed. Reminder horrible dayits gonea brilliant example Aberdeen Dakota Call.
As guide browsers though liked specific. Mccain John Mayer Mikes band.
Windows is Stewart Heating.
Bought guitar learned in chords writing. Wilson Winter Englands bestloved writer .
Windows is Stewart Heating.
Last chance reconnect restore dead Current Selection book. Accurate Game Browser Updated of constantly popular servers listed browsing.
Packed someone ive a known is talked soldiers giving is lives privileges?
Values Moushumi had whole different set issues deal after. Inked Warner coauthor?
Dr jay a Goble dds in. Blogin Creasein Dialon Ranttable in Weighing Incorpus or Christi is Hooksortiz a Becoming.
Friends family their.
Passion Printpay Forward of.
Tommy Brooks is Illinois. Passion Printpay Forward of.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20061107/f17b9fa7/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: Severance.gif
Type: image/gif
Size: 8535 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20061107/f17b9fa7/attachment-0001.gif>
-------------- next part --------------
-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
-------------- next part --------------
_______________________________________________
Numpy-discussion mailing list
Numpy-discussion at lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/numpy-discussion

From ivan.vinogradov at gmail.com  Tue Nov  7 10:37:32 2006
From: ivan.vinogradov at gmail.com (IV)
Date: Tue, 07 Nov 2006 15:37:32 -0000
Subject: help installing numpy1.0 on sgi irix
Message-ID: <1162913852.488520.204670@e3g2000cwe.googlegroups.com>

Dear All,

has anyone successfully install numpy-1.0 on irix 6.5?

The GCC is 3.3 and is the latest from freeware.sgi.com.
Building gcc 4.1.1 from source did not work for me, and I am afraid to
force it.

Latest Python 2.5 installed fine from source.

Numpy gives this error(s):
$ python setup.py install
Running from numpy source directory.
F2PY Version 2_3396
blas_opt_info:
blas_mkl_info:
  libraries mkl,vml,guide not found in /usr/local/lib
  NOT AVAILABLE

atlas_blas_threads_info:
Setting PTATLAS=ATLAS
  libraries lapack,blas not found in /usr/local/lib
  NOT AVAILABLE

atlas_blas_info:
  libraries lapack,blas not found in /usr/local/lib
  NOT AVAILABLE

/usr/people/ivan/numpy-1.0/numpy/distutils/system_info.py:1301:
UserWarning:
    Atlas (http://math-atlas.sourceforge.net/) libraries not found.
    Directories to search for the libraries can be specified in the
    numpy/distutils/site.cfg file (section [atlas]) or by setting
    the ATLAS environment variable.
  warnings.warn(AtlasNotFoundError.__doc__)
blas_info:
  libraries blas not found in /usr/local/lib
  NOT AVAILABLE

/usr/people/ivan/numpy-1.0/numpy/distutils/system_info.py:1310:
UserWarning:
    Blas (http://www.netlib.org/blas/) libraries not found.
    Directories to search for the libraries can be specified in the
    numpy/distutils/site.cfg file (section [blas]) or by setting
    the BLAS environment variable.
  warnings.warn(BlasNotFoundError.__doc__)
blas_src_info:
  NOT AVAILABLE

/usr/people/ivan/numpy-1.0/numpy/distutils/system_info.py:1313:
UserWarning:
    Blas (http://www.netlib.org/blas/) sources not found.
    Directories to search for the sources can be specified in the
    numpy/distutils/site.cfg file (section [blas_src]) or by setting
    the BLAS_SRC environment variable.
  warnings.warn(BlasSrcNotFoundError.__doc__)
  NOT AVAILABLE

lapack_opt_info:
lapack_mkl_info:
mkl_info:
  libraries mkl,vml,guide not found in /usr/local/lib
  NOT AVAILABLE

  NOT AVAILABLE

atlas_threads_info:
Setting PTATLAS=ATLAS
  libraries lapack,blas not found in /usr/local/lib
  libraries lapack_atlas not found in /usr/local/lib
numpy.distutils.system_info.atlas_threads_info
  NOT AVAILABLE

atlas_info:
  libraries lapack,blas not found in /usr/local/lib
  libraries lapack_atlas not found in /usr/local/lib
numpy.distutils.system_info.atlas_info
  NOT AVAILABLE

/usr/people/ivan/numpy-1.0/numpy/distutils/system_info.py:1210:
UserWarning:
    Atlas (http://math-atlas.sourceforge.net/) libraries not found.
    Directories to search for the libraries can be specified in the
    numpy/distutils/site.cfg file (section [atlas]) or by setting
    the ATLAS environment variable.
  warnings.warn(AtlasNotFoundError.__doc__)
lapack_info:
  libraries lapack not found in /usr/local/lib
  NOT AVAILABLE

/usr/people/ivan/numpy-1.0/numpy/distutils/system_info.py:1221:
UserWarning:
    Lapack (http://www.netlib.org/lapack/) libraries not found.
    Directories to search for the libraries can be specified in the
    numpy/distutils/site.cfg file (section [lapack]) or by setting
    the LAPACK environment variable.
  warnings.warn(LapackNotFoundError.__doc__)
lapack_src_info:
  NOT AVAILABLE

/usr/people/ivan/numpy-1.0/numpy/distutils/system_info.py:1224:
UserWarning:
    Lapack (http://www.netlib.org/lapack/) sources not found.
    Directories to search for the sources can be specified in the
    numpy/distutils/site.cfg file (section [lapack_src]) or by setting
    the LAPACK_SRC environment variable.
  warnings.warn(LapackSrcNotFoundError.__doc__)
  NOT AVAILABLE

running install
running build
running config_fc
running build_src
building py_modules sources
building extension "numpy.core.multiarray" sources
Generating build/src.irix64-6.5-2.5/numpy/core/config.h
customize MipsFCompiler
customize MipsFCompiler
customize MipsFCompiler using config
C compiler: gcc -fno-strict-aliasing -DNDEBUG -g -O3 -Wall
-Wstrict-prototypes -shared

compile options: '-I/usr/local/include/python2.5 -Inumpy/core/src
-Inumpy/core/include -I/usr/local/include/python2.5 -c'
gcc: _configtest.c
gcc _configtest.o -L/usr/local/lib -L/usr/lib -o _configtest
ld32: FATAL   12 : Expecting n32 objects: /usr/lib/libc.so is o32.
collect2: ld returned 4 exit status
ld32: FATAL   12 : Expecting n32 objects: /usr/lib/libc.so is o32.
collect2: ld returned 4 exit status
failure.
removing: _configtest.c _configtest.o
numpy/core/setup.py:50: DeprecationWarning: raising a string exception
is deprecated
  raise "ERROR: Failed to test configuration"
Traceback (most recent call last):
  File "setup.py", line 89, in <module>
    setup_package()
  File "setup.py", line 82, in setup_package
    configuration=configuration )
  File "/usr/people/ivan/numpy-1.0/numpy/distutils/core.py", line 174,
in setup
    return old_setup(**new_attr)
  File "/usr/local/lib/python2.5/distutils/core.py", line 151, in setup
    dist.run_commands()
  File "/usr/local/lib/python2.5/distutils/dist.py", line 974, in
run_commands
    self.run_command(cmd)
  File "/usr/local/lib/python2.5/distutils/dist.py", line 994, in
run_command
    cmd_obj.run()
  File "/usr/people/ivan/numpy-1.0/numpy/distutils/command/install.py",
line 16, in run
    r = old_install.run(self)
  File "/usr/local/lib/python2.5/distutils/command/install.py", line
506, in run
    self.run_command('build')
  File "/usr/local/lib/python2.5/distutils/cmd.py", line 333, in
run_command
    self.distribution.run_command(command)
  File "/usr/local/lib/python2.5/distutils/dist.py", line 994, in
run_command
    cmd_obj.run()
  File "/usr/local/lib/python2.5/distutils/command/build.py", line 112,
in run
    self.run_command(cmd_name)
  File "/usr/local/lib/python2.5/distutils/cmd.py", line 333, in
run_command
    self.distribution.run_command(command)
  File "/usr/local/lib/python2.5/distutils/dist.py", line 994, in
run_command
    cmd_obj.run()
  File
"/usr/people/ivan/numpy-1.0/numpy/distutils/command/build_src.py", line
87, in run
    self.build_sources()
  File
"/usr/people/ivan/numpy-1.0/numpy/distutils/command/build_src.py", line
106, in build_sources
    self.build_extension_sources(ext)
  File
"/usr/people/ivan/numpy-1.0/numpy/distutils/command/build_src.py", line
212, in build_extension_sources
    sources = self.generate_sources(sources, ext)
  File
"/usr/people/ivan/numpy-1.0/numpy/distutils/command/build_src.py", line
270, in generate_sources
    source = func(extension, build_dir)
  File "numpy/core/setup.py", line 50, in generate_config_h
    raise "ERROR: Failed to test configuration"
ERROR: Failed to test configuration 


Please advise :)


-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From ivilata at carabos.com  Tue Nov  7 12:19:37 2006
From: ivilata at carabos.com (Ivan Vilata i Balaguer)
Date: Tue, 7 Nov 2006 18:19:37 +0100
Subject: More on Numexpr-unsupported objects
Message-ID: <20061107171937.GI22979@tardis.terramar.selidor.net>

This is somehow related with this previous thread about raising
``TypeError`` on unsupported objects in Numexpr:
http://www.mail-archive.com/numpy-discussion at lists.sourceforge.net/msg03146.html

There is still a case where unsupported objects can get into expressions
without Numexpr noticing.  For instance:

>>> import numexpr
>>> numexpr.evaluate('[]')
Traceback (most recent call last):
  File "<stdin>", line 1, in ?
  File "numexpr/compiler.py", line 594, in evaluate
    _names_cache[expr_key] = getExprNames(ex, context)
  File "numexpr/compiler.py", line 570, in getExprNames
    ast = expressionToAST(ex)
  File "numexpr/compiler.py", line 84, in expressionToAST
    this_ast = ASTNode(ex.astType, ex.astKind, ex.value,
AttributeError: 'list' object has no attribute 'astType'

The attached patch makes the error clearer and more consistent with the
error added in the aforementioned thread:

>>> import numpy
>>> import numexpr
>>> numexpr.evaluate('[]')
Traceback (most recent call last):
  File "<stdin>", line 1, in ?
  File "numexpr/compiler.py", line 596, in evaluate
    _names_cache[expr_key] = getExprNames(ex, context)
  File "numexpr/compiler.py", line 571, in getExprNames
    ex = stringToExpression(text, {}, context)
  File "numexpr/compiler.py", line 229, in stringToExpression
    raise TypeError("unsupported expression type: %s" % type(ex))
TypeError: unsupported expression type: <type 'list'>

Though I admit it may be strange for this error to be triggered.

(I had a little mess with using ``expressions.ExpressionNode`` instead
of ``expr.ExpressionNode``...  I still don't see why the private copy of
the module is necessary.)

::

	Ivan Vilata i Balaguer   >qo<   http://www.carabos.com/
	       C?rabos Coop. V.  V  V   Enjoy Data
	                          ""
-------------- next part --------------
Index: compiler.py
===================================================================
--- compiler.py	(revisi?n: 2307)
+++ compiler.py	(copia de trabajo)
@@ -225,6 +225,8 @@
     ex = eval(c, names)
     if expressions.isConstant(ex):
         ex = expr.ConstantNode(ex, expressions.getKind(ex))
+    elif not isinstance(ex, expr.ExpressionNode):
+        raise TypeError("unsupported expression type: %s" % type(ex))
     return ex
 
 
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 309 bytes
Desc: Digital signature
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20061107/bdf54192/attachment-0001.sig>
-------------- next part --------------
-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
-------------- next part --------------
_______________________________________________
Numpy-discussion mailing list
Numpy-discussion at lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/numpy-discussion

From Chris.Barker at noaa.gov  Tue Nov  7 13:56:56 2006
From: Chris.Barker at noaa.gov (Christopher Barker)
Date: Tue, 07 Nov 2006 10:56:56 -0800
Subject: Stacking arrays...
Message-ID: <4550D6F8.80001@noaa.gov>

HI all,

I'm trying to get the hang of this new r_ and c_ stuff.

I need to take two MxN arrays, and stack them together into one MxNx2 
arrays. I found this works:

 >>> a = N.ones((3,4))
 >>> b = N.ones((3,4)) * 2
 >>> a
array([[ 1.,  1.,  1.,  1.],
        [ 1.,  1.,  1.,  1.],
        [ 1.,  1.,  1.,  1.]])
 >>> b
array([[ 2.,  2.,  2.,  2.],
        [ 2.,  2.,  2.,  2.],
        [ 2.,  2.,  2.,  2.]])
 >>> a.shape = (3,4,1)
 >>> b.shape = (3,4,1)
 >>> c = N.c_[a,b]
 >>> c.shape
(3, 4, 2)
 >>> c[0,0]
array([ 1.,  2.])


What I'm wondering is if there is a way to do that without explicitly 
adding the extra dimension to a and b before calling c_ ?

-Chris


-- 
Christopher Barker, Ph.D.
Oceanographer
                                     		
NOAA/OR&R/HAZMAT         (206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115       (206) 526-6317   main reception

Chris.Barker at noaa.gov

-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From matthew.brett at gmail.com  Tue Nov  7 13:57:21 2006
From: matthew.brett at gmail.com (Matthew Brett)
Date: Tue, 7 Nov 2006 10:57:21 -0800
Subject: Passing numpy arrays to matlab
In-Reply-To: <45501F38.3090201@ar.media.kyoto-u.ac.jp>
References: <454FEE21.7040205@ar.media.kyoto-u.ac.jp>
	<45501396.3070908@astraw.com>
	<455012BF.1030105@ar.media.kyoto-u.ac.jp>
	<45501E70.8090609@astraw.com>
	<45501F38.3090201@ar.media.kyoto-u.ac.jp>
Message-ID: <1e2af89e0611071057h70b2f575u3943efbc142ca9c8@mail.gmail.com>

Hi,

> Thank you very much, I think this added documentation is pretty recent;
> I have never seen it before, and I did a lot a mex programming at some
> point... This whole mxarray nonsense reminds me why I gave up on matlab :),

I would be very happy to help with this.  It would be great if we
could get a standard well-maintained library of some sort towards
scipy - we (http://neuroimaging.scipy.org/) have a great deal of
matlab integration to do.

Best,

Matthew

-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From pauli.virtanen at iki.fi  Tue Nov  7 13:59:45 2006
From: pauli.virtanen at iki.fi (Pauli Virtanen)
Date: Tue, 07 Nov 2006 20:59:45 +0200
Subject: Passing numpy arrays to matlab
In-Reply-To: <454FEE21.7040205@ar.media.kyoto-u.ac.jp>
References: <454FEE21.7040205@ar.media.kyoto-u.ac.jp>
Message-ID: <1162925985.6891.6.camel@localhost.localdomain>

Hi all,

ti, 2006-11-07 kello 11:23 +0900, David Cournapeau kirjoitti:
>     I am trying to find a nice way to communicate between matlab and 
> python. I am aware of pymat, which does that, but the code is 
> deprecated, and I thing basing the code on ctypes would lead to much 
> more robust code.
> 
>     http://claymore.engineer.gvsu.edu/%7Esteriana/Software/pymat.html
> 
> I have a really simple prototype which can send and get back data from 
> matlab, but I was wondering if it would be possible to use a scheme 
> similar to ctypes instead of having to convert it by hand.

A while ago I wrote a mex extension to embed the Python interpreter
inside Matlab:

	http://www.iki.fi/pav/pythoncall

I guess it's something like an inverse of pymat :)

But I guess this is not really what you are looking for, since at
present it just does a memory copy when passing arrays between Matlab
and Python. Though, shared arrays might be just possible to implement if
memory management is done carefully.

BR,

	Pauli Virtanen

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 189 bytes
Desc: Digitaalisesti allekirjoitettu viestin osa
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20061107/2d09360e/attachment-0001.sig>
-------------- next part --------------
-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
-------------- next part --------------
_______________________________________________
Numpy-discussion mailing list
Numpy-discussion at lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/numpy-discussion

From Chris.Barker at noaa.gov  Tue Nov  7 19:53:47 2006
From: Chris.Barker at noaa.gov (Christopher Barker)
Date: Tue, 07 Nov 2006 16:53:47 -0800
Subject: "Slicing" recarrays
Message-ID: <45512A9B.9030809@noaa.gov>

Hi all,

I'm using a recarray to read a bunch of binary data out of a file. It's 
working great, but it seems there should be a more efficient way to 
"slice" the data. Here's what I've got:

The binary data is essentially a dump of a 2-d array of structs. So I 
read it like this:

DataType = N.dtype([("long","i4"), ("lat", "i4"), ("flag","b1")])

data = N.fromfile(file, DataType)

data.shape = (M, N)

So I now have a MxN array of the structs. What I would like to do is 
extract a MxNx2 array of just the two 4-byte integers. It seems that I 
should be able to do that without copying -- by using a view into the 
original data. I can't figure out how, however. What I am doing is:

LEs = N.empty((M, N, 2), dtype=N.int32)
LEs[:,:,0] = data['long']
LEs[:,:,1] = data['lat']

This works, but these are BIG files, so it would be nice not to be doing 
that extra copying. Is that possible?

Thanks,

-Chris







-- 
Christopher Barker, Ph.D.
Oceanographer
                                     		
NOAA/OR&R/HAZMAT         (206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115       (206) 526-6317   main reception

Chris.Barker at noaa.gov

-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From wbaxter at gmail.com  Tue Nov  7 18:41:37 2006
From: wbaxter at gmail.com (Bill Baxter)
Date: Wed, 8 Nov 2006 08:41:37 +0900
Subject: Stacking arrays...
In-Reply-To: <4550D6F8.80001@noaa.gov>
References: <4550D6F8.80001@noaa.gov>
Message-ID: <e86a5fd00611071541x701aa51cqf7c74533869c99e@mail.gmail.com>

dstack maybe? or does that add a dim on the other end?
--bb

On 11/8/06, Christopher Barker <Chris.Barker at noaa.gov> wrote:
>
> HI all,
>
> I'm trying to get the hang of this new r_ and c_ stuff.
>
> I need to take two MxN arrays, and stack them together into one MxNx2
> arrays. I found this works:
>
> >>> a = N.ones((3,4))
> >>> b = N.ones((3,4)) * 2
> >>> a
> array([[ 1.,  1.,  1.,  1.],
>         [ 1.,  1.,  1.,  1.],
>         [ 1.,  1.,  1.,  1.]])
> >>> b
> array([[ 2.,  2.,  2.,  2.],
>         [ 2.,  2.,  2.,  2.],
>         [ 2.,  2.,  2.,  2.]])
> >>> a.shape = (3,4,1)
> >>> b.shape = (3,4,1)
> >>> c = N.c_[a,b]
> >>> c.shape
> (3, 4, 2)
> >>> c[0,0]
> array([ 1.,  2.])
>
>
> What I'm wondering is if there is a way to do that without explicitly
> adding the extra dimension to a and b before calling c_ ?
>
> -Chris
>
>
> --
> Christopher Barker, Ph.D.
> Oceanographer
>
> NOAA/OR&R/HAZMAT         (206) 526-6959   voice
> 7600 Sand Point Way NE   (206) 526-6329   fax
> Seattle, WA  98115       (206) 526-6317   main reception
>
> Chris.Barker at noaa.gov
>
> -------------------------------------------------------------------------
> Using Tomcat but need to do more? Need to support web services, security?
> Get stuff done quickly with pre-integrated technology to make your job
> easier
> Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
> http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
> _______________________________________________
> Numpy-discussion mailing list
> Numpy-discussion at lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/numpy-discussion
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20061108/98a80663/attachment-0001.html>
-------------- next part --------------
-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
-------------- next part --------------
_______________________________________________
Numpy-discussion mailing list
Numpy-discussion at lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/numpy-discussion

From michael.sorich at gmail.com  Tue Nov  7 20:11:24 2006
From: michael.sorich at gmail.com (Michael Sorich)
Date: Wed, 8 Nov 2006 11:41:24 +1030
Subject: A reimplementation of MaskedArray
In-Reply-To: <200610241408.40415.pgmdevlist@gmail.com>
References: <200610160232.20888.pgmdevlist@mailcan.com>
	<16761e100610232350u28329c5bp7944ac325030d0ba@mail.gmail.com>
	<200610241408.40415.pgmdevlist@gmail.com>
Message-ID: <16761e100611071711q6b9cc464p85708d7a642b9a8a@mail.gmail.com>

On 10/25/06, Pierre GM <pgmdevlist at gmail.com> wrote:
> On Tuesday 24 October 2006 02:50, Michael Sorich wrote:
> > I am currently running numpy rc2 (I haven't tried your
> > reimplementation yet as I am still using python 2.3). I am wondering
> > whether the new maskedarray is able to handle construction of arrays
> > from masked scalar values (not sure if this is the correct term).
>
> The answer is no, unfortunately: both the new and old implementations fail at
> the same point, raising a TypeError: lists are processed through
> numpy.core.numeric.array, which doesn't handle that.

I have finally gotten around to upgrading to python 2.4 and have had a
chance to play with your new version of the MaskedArray. It is great
to see that someone is working on this. I have a few thoughts on
masked arrays that may or may no warrant discusion

1. It would be nice if the masked_singleton could be passed into a
ndarray, as this would allow it to be passed into the MaskedArray e.g.

import numpy as N
import ma.maskedarray as MA
test = N.array([1,2,MA.masked])
>> ValueError: setting an array element with a sequence

If the masked_singleton was implemented as an object that is not a
MakedArray (which is a sequence that numpy.array chokes on), then a
valid numpy array with an object dtype could be produced. e.g.

class MaskedScalar:
    def __str__(self):
        return 'masked'
masked = MaskedScalar()

test = N.array([1,2,masked])
print test.dtype, test
>>object [1 2 masked]
print test == masked
>>[False False True]
print test[2] == masked
>>True
print test[2] is masked
>>True

Then it would be possible to alternatively define a masked array as
MA.array([1,2,masked]) or MA.array(N.array([1,2,masked])). In the
__init__ of the MaskedArray if the ndarray has an object dtype simply
calculate the mask from a==masked.

2. What happens if a masked array is passed into a ndarray or passed
into a MaskedArray with another mask?

test_ma1 = MA.array([1,2,3], mask=[False, False, True])
print test_ma1
>>[1 2 --]

print N.array(test_ma1)
>>[1 2 3]

test_ma2 = MA.array(test_ma1, mask=[True, False, False])
print test_ma2
>>[-- 2 3]

I suppose it depends on whether you are masking useful data, or the
masks represent missing data. In the former it may make sense to
change or remove the mask. However in the latter case the original
data entered is a bogus value which should never be unmasked. In this
case, when converting to a ndarray I think it make more sense to make
an object ndarray with the missing value containing the masked
singleton. Additionally, if the MaskedArray is holding missing data,
it does not make much sense to be able to pass in to the MA
constructor both an existing ma and a mask.

-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From tim.hochberg at ieee.org  Tue Nov  7 15:38:39 2006
From: tim.hochberg at ieee.org (Tim Hochberg)
Date: Tue, 07 Nov 2006 13:38:39 -0700
Subject: More on Numexpr-unsupported objects
In-Reply-To: <20061107171937.GI22979@tardis.terramar.selidor.net>
References: <20061107171937.GI22979@tardis.terramar.selidor.net>
Message-ID: <4550EECF.9070406@ieee.org>

Ivan Vilata i Balaguer wrote:


[SNIP]
>
> Though I admit it may be strange for this error to be triggered.
>
> (I had a little mess with using ``expressions.ExpressionNode`` instead
> of ``expr.ExpressionNode``...  I still don't see why the private copy of
> the module is necessary.)
>
>   
It's an attempt to make things thread safe. Since we are mucking around 
with the internals of the expression module, if there were multiple 
threads one might thread might change get_context out from under the 
other one. Giving each call to stringToExpression its own copy of the 
expression module prevents this.

It is,  admittedly, a kind of stupid way to do this. A more sane thing 
to do would probably to have some sort of expression object that we 
instantiate for each call to stringToExpression instead of mucking 
around with modules, which really aren't meant to be (ab)used this way. 
However, this would require a major rewrite of the expression module and 
I was feeling lazy at the time (and still am), so I resorted to 
questionable hackery.

-tim


-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From pau.gargallo at gmail.com  Tue Nov  7 17:15:03 2006
From: pau.gargallo at gmail.com (Pau Gargallo)
Date: Tue, 7 Nov 2006 23:15:03 +0100
Subject: Stacking arrays...
In-Reply-To: <4550D6F8.80001@noaa.gov>
References: <4550D6F8.80001@noaa.gov>
Message-ID: <6ef8f3380611071415k7e3c59d5pba7ff9338abec60f@mail.gmail.com>

On 11/7/06, Christopher Barker <Chris.Barker at noaa.gov> wrote:
> HI all,
>
> I'm trying to get the hang of this new r_ and c_ stuff.
>
> I need to take two MxN arrays, and stack them together into one MxNx2
> arrays. I found this works:
>
>  >>> a = N.ones((3,4))
>  >>> b = N.ones((3,4)) * 2
>  >>> a
> array([[ 1.,  1.,  1.,  1.],
>         [ 1.,  1.,  1.,  1.],
>         [ 1.,  1.,  1.,  1.]])
>  >>> b
> array([[ 2.,  2.,  2.,  2.],
>         [ 2.,  2.,  2.,  2.],
>         [ 2.,  2.,  2.,  2.]])
>  >>> a.shape = (3,4,1)
>  >>> b.shape = (3,4,1)
>  >>> c = N.c_[a,b]
>  >>> c.shape
> (3, 4, 2)
>  >>> c[0,0]
> array([ 1.,  2.])
>
>
> What I'm wondering is if there is a way to do that without explicitly
> adding the extra dimension to a and b before calling c_ ?
>

c=N.dstack([a,b]) seems to do the trick.

pau



> -Chris
>
>
> --
> Christopher Barker, Ph.D.
> Oceanographer
>
> NOAA/OR&R/HAZMAT         (206) 526-6959   voice
> 7600 Sand Point Way NE   (206) 526-6329   fax
> Seattle, WA  98115       (206) 526-6317   main reception
>
> Chris.Barker at noaa.gov
>
> -------------------------------------------------------------------------
> Using Tomcat but need to do more? Need to support web services, security?
> Get stuff done quickly with pre-integrated technology to make your job easier
> Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
> http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
> _______________________________________________
> Numpy-discussion mailing list
> Numpy-discussion at lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/numpy-discussion
>

-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From oliphant at ee.byu.edu  Tue Nov  7 21:01:21 2006
From: oliphant at ee.byu.edu (Travis Oliphant)
Date: Tue, 07 Nov 2006 19:01:21 -0700
Subject: "Slicing" recarrays
In-Reply-To: <45512A9B.9030809@noaa.gov>
References: <45512A9B.9030809@noaa.gov>
Message-ID: <45513A71.3060603@ee.byu.edu>

Christopher Barker wrote:

>Hi all,
>
>I'm using a recarray to read a bunch of binary data out of a file. It's 
>working great, but it seems there should be a more efficient way to 
>"slice" the data. Here's what I've got:
>
>The binary data is essentially a dump of a 2-d array of structs. So I 
>read it like this:
>
>DataType = N.dtype([("long","i4"), ("lat", "i4"), ("flag","b1")])
>
>data = N.fromfile(file, DataType)
>
>data.shape = (M, N)
>
>So I now have a MxN array of the structs. What I would like to do is 
>extract a MxNx2 array of just the two 4-byte integers. It seems that I 
>should be able to do that without copying -- by using a view into the 
>original data. I can't figure out how, however. What I am doing is:
>
>LEs = N.empty((M, N, 2), dtype=N.int32)
>LEs[:,:,0] = data['long']
>LEs[:,:,1] = data['lat']
>
>This works, but these are BIG files, so it would be nice not to be doing 
>that extra copying. Is that possible?
>  
>

How about

newdtype = N.dtype([('both','2i4'),('','b1')])
res = data.view(newdtype)['both']


-Travis


-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From david at ar.media.kyoto-u.ac.jp  Tue Nov  7 21:43:21 2006
From: david at ar.media.kyoto-u.ac.jp (David Cournapeau)
Date: Wed, 08 Nov 2006 11:43:21 +0900
Subject: Passing numpy arrays to matlab
In-Reply-To: <1162925985.6891.6.camel@localhost.localdomain>
References: <454FEE21.7040205@ar.media.kyoto-u.ac.jp>
	<1162925985.6891.6.camel@localhost.localdomain>
Message-ID: <45514449.1010803@ar.media.kyoto-u.ac.jp>

Pauli Virtanen wrote:
> Hi all,
>
> ti, 2006-11-07 kello 11:23 +0900, David Cournapeau kirjoitti:
>   
>>     I am trying to find a nice way to communicate between matlab and 
>> python. I am aware of pymat, which does that, but the code is 
>> deprecated, and I thing basing the code on ctypes would lead to much 
>> more robust code.
>>
>>     http://claymore.engineer.gvsu.edu/%7Esteriana/Software/pymat.html
>>
>> I have a really simple prototype which can send and get back data from 
>> matlab, but I was wondering if it would be possible to use a scheme 
>> similar to ctypes instead of having to convert it by hand.
>>     
>
> A while ago I wrote a mex extension to embed the Python interpreter
> inside Matlab:
>
> 	http://www.iki.fi/pav/pythoncall
>
> I guess it's something like an inverse of pymat :)
>
>   
Yes, but at the end, I think they enable similar things. Thanks for the 
link !
> But I guess this is not really what you are looking for, since at
> present it just does a memory copy when passing arrays between Matlab
> and Python. Though, shared arrays might be just possible to implement if
> memory management is done carefully.
>   
In my case, it is much worse:

    1 first, you have numpy data that you have to copy to mxArray, the 
structure representing arrays in matlab C api.
    2 then when you "send" data to the matlab engine, this is done 
automatically through pipe  by the matlab engine API (maybe pipe does 
not imply copying; I don't know much about pipe from a programming point 
of view, actually)
    3 The arrays you get back from matlab are in matlab mxArray 
structures: right now, I copy their data to new numpy arrays.

    At first, I just developed a prototype without thinking too much, 
and the result was much slower than I thought: sending a numpy  with 
2e5x10 double takes around 100 ms on my quite powerful machine (around 
14 cycles per item for the best case). I suspect it is because I copy 
memory in a non contiguous manner (matlab arrays have a internal F 
storage for real arrays, but complex arrays are really two different 
arrays, which is different than Fortran convention I think, making the 
copy cost really expensive for complex arrays). To see if I was doing 
something wrong, I compared with numpy.require(ar, requirements = 
'F_CONTIGUOUS'), which is even much slower
    There is not much I can do about 2, it looks like there is a way to 
avoid copying for 1, and my question was more specific to 3 (but 
reusable in 1, maybe, if I am smart enough). Basically:

     * how to create an object which has the same interface than numpy 
arrays, but owns the data from a foreign structure, which data are 
availble when building the object (The idea was to create a class which 
implements the array interface from python, kind of proxy class, which 
owns the data from mxArray; owns here is from a memory management point 
of view).

    David

-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From david at ar.media.kyoto-u.ac.jp  Tue Nov  7 22:07:08 2006
From: david at ar.media.kyoto-u.ac.jp (David Cournapeau)
Date: Wed, 08 Nov 2006 12:07:08 +0900
Subject: Passing numpy arrays to matlab
In-Reply-To: <E50CA0F6-3ED5-4ED0-92BA-B55991001504@gmail.com>
References: <mailman.119602.1162866273.22397.numpy-discussion@lists.sourceforge.net>
	<E50CA0F6-3ED5-4ED0-92BA-B55991001504@gmail.com>
Message-ID: <455149DC.3090701@ar.media.kyoto-u.ac.jp>

Josh Marshall wrote:
> Hi David,
>
> Did you have a look at mlabwrap? It's quite hard to find on the net,  
> which is a shame, since it is a much more up to date version,  
> enhancing pymat with the things that you are trying to do. It allows  
> passing arrays and getting arrays back.
>
> http://mlabwrap.sourceforge.net/
>   
I didn't know that, thanks. Unfortunately, it is not really what I am 
trying to do: mlabwrap is just a python interface a bit more high level 
than pymat, with many fancy tricks, but still do copies. What I would 
like is to avoid completely the copying by using proxy classes around 
data from numpy so that I can pass "automatically" numpy arrays to 
matlab C api, and a proxy class around data from matlab so that they 
look like numpy arrays.
    I don't care that much about the actual api from python point of 
view, because I intend to use this mainly to compare matlab vs numpy 
implementation, not as a way to use matlab inside python regularly. And 
once the copy problem is solved, adding syntactic sugar using python is 
easy anyway, I think (it should be easy to do something similar to 
mlabwrap at that point),

    cheers,

    David

-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From fperez.net at gmail.com  Wed Nov  8 03:32:44 2006
From: fperez.net at gmail.com (Fernando Perez)
Date: Wed, 8 Nov 2006 01:32:44 -0700
Subject: Profiling Python codes with hotshot and
	KCachegrind
Message-ID: <db6b5ecc0611080032n10bfa84fg6edb1e97f48f04ba@mail.gmail.com>

Hi all,

in the past, Arnd Baecker has made a number of very useful posts on
this matter, and provided some nice utilities to do it.  I now needed
to profile some fairly complex codes prior to a big optimization push,
so I went over his material and wrote a little tool to make the whole
process as painless as possible.  Here it is, hoping others may find
it useful:

http://amath.colorado.edu/faculty/fperez/python/profiling/

Cheers,

f

-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From pgmdevlist at gmail.com  Wed Nov  8 05:10:13 2006
From: pgmdevlist at gmail.com (Pierre GM)
Date: Wed, 8 Nov 2006 05:10:13 -0500
Subject: A reimplementation of MaskedArray
In-Reply-To: <16761e100611071711q6b9cc464p85708d7a642b9a8a@mail.gmail.com>
References: <200610160232.20888.pgmdevlist@mailcan.com>
	<200610241408.40415.pgmdevlist@gmail.com>
	<16761e100611071711q6b9cc464p85708d7a642b9a8a@mail.gmail.com>
Message-ID: <200611080510.13565.pgmdevlist@gmail.com>

Michael,
First of all, thanks for your interest in the exercise of style the new 
implementation of MaskedArray is basically nothing but.

On Tuesday 07 November 2006 20:11, Michael Sorich wrote:
> 1. It would be nice if the masked_singleton could be passed into a
> ndarray, as this would allow it to be passed into the MaskedArray e.g.
>
> import numpy as N
> import ma.maskedarray as MA
> test = N.array([1,2,MA.masked])
>
> >> ValueError: setting an array element with a sequence

I like your idea, but not its implementation. If MA.masked_singleton is 
defined as an object, as you suggest, then the dtype of the ndarray it is 
passed to becomes 'object', as you pointed out, and that is not something one 
would naturally expec, as basic numerical functions don't  work well  with the 
'object' dtype (just try  N.sqrt(N.array([1],dtype=N.object)) to see what I 
mean).
Even if we can construct a mask rather easily at the creation of the masked 
array, following your 'a==masked' suggestion, we still need to get the dtype 
of the non-masked section, and that doesn't seem trivial...

I guess that a simple solution is to use MA.masked_values. 
Make sure to use a numerical value for the masked data, else you'll end up 
with yet another object array.  


> 2. What happens if a masked array is passed into a ndarray or passed
> into a MaskedArray with another mask?
>>> test_ma1 = MA.array([1,2,3], mask=[False, False, True])
>>> print test_ma1, N.array(test_ma1), 
[1 2 --] [1 2 3]
>>> MA.array(test_ma1, mask=[True, False, False]) 
[-- 2 3]

Let me precise that my objective was to get an implementation as close to the 
original numpy.core.ma as possible, for 'backward compatibility'. I'm not 
sure it'd be wise to change it at this point, but that could be discussed.

As you've noticed, when creating a new masked array from an existing one, the 
'mask' argument supersedes the initial mask. That's ideal when you want to 
focus on a fraction of the initial data: you just mask what you don't need, 
and are still able to retrieve it when you need it. I agree that this default 
behavior is a bit strange when you have missing data: in that case, one would 
expect the new mask to be a combination of the 'mask' argument and the old 
mask. 
A possibility would then be to add a 'keep_mask' flag: a default of False 
would give the current behavior, a value of True would force the new mask to 
be a combination. I think that feelings are mixed on that list about extra 
flags, but the users of maskedarray are only a minority anyway (hopefully, 
only for the moment).

About the conversion to ndarray: 
By default, the result should have the same dtype as the _data section. 
For this reason, I disagree with your idea of "(returning) an object ndarray 
with the missing value containing the masked singleton". If you really want 
an object ndarray, you can use the filled method or the filled function, with 
your own definition of the filling value (such as your MaskedScalar). 







-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From ejaxvxjkxw at panoramapoint.com  Wed Nov  8 06:52:00 2006
From: ejaxvxjkxw at panoramapoint.com (other)
Date: Wed, 8 Nov 2006 12:52:00 +0100
Subject: Online Policy updated
Message-ID: <000701c7032c$4d0d9a90$6a2ca45b@moiax5xk5ayd7x>


Version recommends that upgrade most recent of through Download a Center.
Legacy Operating Systems nt Macintosh os x am Powerpc Safari.
Powerpc Safari Intel Linux Solaris Copyright Macromedia Software is llc. Designs animation user in? Center take advantage security updates Platform Browser Windows of Explorer of. Robust runtime am for rich media is is the standard. Adobeyour Guestyour States Management of Internet and in Audioweb Industries amp.
Rights Reserved Protected! Buy Searchyou may not have everything you.
Highimpact web content Designs animation a user.
Storestore Ways to buy Searchyou? Designs animation user or interfaces.
Rights Reserved Protected by a us is. Operating Systems nt am Macintosh os x a Powerpc!
Opera in Legacy Operating Systems nt a.
Controls plugins Firefox Mozilla of. Legacy Operating Systems a nt Macintosh!
Opera in Legacy Operating Systems nt a.
Updates Platform Browser Windows Explorer other? Web content Designs or animation user interfaces.
Are deployed across all browsers platforms attracting engaging users.
Media is the or standard. X Powerpc Safari Intel Linux Solaris Copyright Macromedia a!
Content Designs is animation user am interfaces are or deployed am.
Legacy Operating Systems nt Macintosh os x am Powerpc Safari.
Feedback this website or signifies your or agreement Terms.
Of Adobecom Please see of our or site Home.
Robust in runtime for rich a media in. Of Adobecom Please see of our or site Home.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20061108/18fe4ee3/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: contains.gif
Type: image/gif
Size: 8304 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20061108/18fe4ee3/attachment-0001.gif>
-------------- next part --------------
-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
-------------- next part --------------
_______________________________________________
Numpy-discussion mailing list
Numpy-discussion at lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/numpy-discussion

From Mailer-Daemon at lists.sourceforge.net  Wed Nov  8 07:08:21 2006
From: Mailer-Daemon at lists.sourceforge.net (Mail Delivery System)
Date: Wed, 08 Nov 2006 04:08:21 -0800
Subject: Warning: message 1Ggbsf-00056T-Ol delayed 72
	hours
Message-ID: <E1GhmE9-0008RB-Ox@externalmx-1.sourceforge.net>

This message was created automatically by mail delivery software.
A message that you sent has not yet been delivered to one or more of its
recipients after more than 72 hours on the queue on externalmx-1.sourceforge.net.

The message identifier is:     1Ggbsf-00056T-Ol
The subject of the message is: faces and their former position.  To be, sure.  But question
The date of the message is:    Sun, 05 Nov 2006 16:01:50 +0900

The address to which the message has not yet been delivered is:

  numpy-discussion at lists.sourceforge.net
    Delay reason: SMTP error from remote mailer after RCPT TO:<numpy-discussion at lists.sourceforge.net>:
    host mail.sourceforge.net [66.35.250.206]: 451-Could not complete sender verify callout
    451-Could not complete sender verify callout for
    451-<numpy-discussion at lists.sourceforge.net>.
    451-The mail server(s) for the domain may be temporarily unreachable, or
    451-they may be permanently unreachable from this server. In the latter case,
    451-you need to change the address or create an MX record for its domain
    451-if it is 

No action is required on your part. Delivery attempts will continue for
some time, and this warning may be repeated at intervals if the message
remains undelivered. Eventually the mail delivery software will give up,
and when that happens, the message will be returned to you.

-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From amit_605 at yahoo.co.in  Wed Nov  8 07:42:39 2006
From: amit_605 at yahoo.co.in (amit soni)
Date: Wed, 8 Nov 2006 04:42:39 -0800 (PST)
Subject: reading matrix from a file
Message-ID: <20061108124239.20230.qmail@web8409.mail.in.yahoo.com>

Hi,
  i have a file with following format: 
 1 2 
 3 9 
 2 3 
 4 4 
   I want to read it and then store the values into two matrices, s.t. 
 A=[1 2;3 9] 
 B=[2 3;4 4] 
 
  Can anyone tell me how to do this in python?
 thanks 
 Amit 
  
 
---------------------------------
Check out the all-new Yahoo! Mail - Fire up a more powerful email and get things done faster.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20061108/1b3df0b8/attachment-0001.html>
-------------- next part --------------
-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
-------------- next part --------------
_______________________________________________
Numpy-discussion mailing list
Numpy-discussion at lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/numpy-discussion

From izakmarais at yahoo.com  Wed Nov  8 08:54:17 2006
From: izakmarais at yahoo.com (izak marais)
Date: Wed, 8 Nov 2006 05:54:17 -0800 (PST)
Subject: matrix multiplication (newbie question)
Message-ID: <20061108135417.6906.qmail@web50913.mail.yahoo.com>

Hi

Sorry if this is an obvious question, but what is the easiest way to multiply matrices in numpy? Suppose I want to do A=B*C*D. The ' * ' operator apparently does element wise multiplication, as does the 'multiply' ufunc. All I could find was the numeric function 'matrix_multiply, but this only takes two arguments.

Thanks in advance!
Izak

 
---------------------------------
Sponsored Link

Degrees online in as fast as 1 Yr - MBA, Bachelor's, Master's, Associate - Click now to apply
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20061108/0b9c79bf/attachment-0001.html>
-------------- next part --------------
-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
-------------- next part --------------
_______________________________________________
Numpy-discussion mailing list
Numpy-discussion at lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/numpy-discussion

From kwgoodman at gmail.com  Wed Nov  8 09:32:00 2006
From: kwgoodman at gmail.com (Keith Goodman)
Date: Wed, 8 Nov 2006 06:32:00 -0800
Subject: matrix multiplication (newbie question)
In-Reply-To: <20061108135417.6906.qmail@web50913.mail.yahoo.com>
References: <20061108135417.6906.qmail@web50913.mail.yahoo.com>
Message-ID: <f4f93d420611080632h3b3eb058qd3d0d0575d116988@mail.gmail.com>

On 11/8/06, izak marais <izakmarais at yahoo.com> wrote:

> Sorry if this is an obvious question, but what is the easiest way to
> multiply matrices in numpy? Suppose I want to do A=B*C*D. The ' * ' operator
> apparently does element wise multiplication, as does the 'multiply' ufunc.
> All I could find was the numeric function 'matrix_multiply, but this only
> takes two arguments.

If B and C and D are matrices, then '*' is matrix multiplication.

-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From stefan at sun.ac.za  Wed Nov  8 09:38:27 2006
From: stefan at sun.ac.za (Stefan van der Walt)
Date: Wed, 8 Nov 2006 16:38:27 +0200
Subject: matrix multiplication (newbie question)
In-Reply-To: <20061108135417.6906.qmail@web50913.mail.yahoo.com>
References: <20061108135417.6906.qmail@web50913.mail.yahoo.com>
Message-ID: <20061108143827.GO3380@mentat.za.net>

On Wed, Nov 08, 2006 at 05:54:17AM -0800, izak marais wrote:
> Hi
> 
> Sorry if this is an obvious question, but what is the easiest way to multiply
> matrices in numpy? Suppose I want to do A=B*C*D. The ' * ' operator apparently
> does element wise multiplication, as does the 'multiply' ufunc. All I could
> find was the numeric function 'matrix_multiply, but this only takes two
> arguments.

You can also do N.dot(B, N.dot(C,D)).

Cheers
St?fan



-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From nadavh at visionsense.com  Wed Nov  8 09:43:46 2006
From: nadavh at visionsense.com (Nadav Horesh)
Date: Wed, 8 Nov 2006 16:43:46 +0200
Subject: matrix multiplication (newbie question)
Message-ID: <07C6A61102C94148B8104D42DE95F7E8FCB333@exchange2k.envision.co.il>

Make A,B,... matrices instead of arrays, so instead

A = array((.....))

Write

A = matrix((....))

 

Assuming you had

>From numpy import *

 

  Nadav

 

  _____  

From: numpy-discussion-bounces at lists.sourceforge.net
[mailto:numpy-discussion-bounces at lists.sourceforge.net] On Behalf Of
izak marais
Sent: Wednesday, November 08, 2006 15:54
To: numpy-discussion at lists.sourceforge.net
Subject: [Numpy-discussion] matrix multiplication (newbie question)

 

Hi

Sorry if this is an obvious question, but what is the easiest way to
multiply matrices in numpy? Suppose I want to do A=B*C*D. The ' * '
operator apparently does element wise multiplication, as does the
'multiply' ufunc. All I could find was the numeric function
'matrix_multiply, but this only takes two arguments.

Thanks in advance!
Izak

  

  _____  

Sponsored Link

Degrees online in as fast as 1 Yr - MBA, Bachelor's, Master's, Associate
- Click now to apply
<http://online.degrees.info/forms/form.jsp?ADEL=D3sPPz6WYLD7%2FD1DxMxP%2
BbXooR4YcdmONmTWaUizNjh6q2wGRcnxuw%3D%3D&PI=y_mail_textlink> 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20061108/6ffe4425/attachment-0001.html>
-------------- next part --------------
-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
-------------- next part --------------
_______________________________________________
Numpy-discussion mailing list
Numpy-discussion at lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/numpy-discussion

From david at ar.media.kyoto-u.ac.jp  Wed Nov  8 09:43:04 2006
From: david at ar.media.kyoto-u.ac.jp (David Cournapeau)
Date: Wed, 08 Nov 2006 23:43:04 +0900
Subject: Passing numpy arrays to matlab
In-Reply-To: <45501396.3070908@astraw.com>
References: <454FEE21.7040205@ar.media.kyoto-u.ac.jp>
	<45501396.3070908@astraw.com>
Message-ID: <4551ECF8.90806@ar.media.kyoto-u.ac.jp>

Andrew Straw wrote:
> David Cournapeau wrote:
>   
>>     - To send data from the calling process to matlab, you first have to 
>> create a mxArray, which is the basic matlab handler of a matlab array, 
>> and populating it. Using mxArray is very ackward : you cannot create 
>> mxArray from existing data, you have to copy data to them, etc... 
>>     
> My understanding, never having done it, but from reading the docs, is
> that you can create a "hybrid array" where you manage the memory. Thus,
> you can create an mxArray from existing data. However, the docs
> basically say that this is too hard for most mortals (and they may well
> be right -- too painful for me, anyway)!
>   
Ok, I have looked at it. It is not hard, it is just totally brain 
damaged: there is no way to destroy a mxArray without destroying the 
data it is holding, even after a call with mxSetPr. So the data 
referenced by the pointer given to mxSetPr is always destroyed by 
mxDestroyArray; I don't see any way to use this to avoid copy... They 
could at least have given a function which frees the data buffer and one 
which destroys the other stuff; as it is, it is totally useless, unless 
you don't mind memory leaks.

David

-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From faltet at carabos.com  Wed Nov  8 09:44:58 2006
From: faltet at carabos.com (Francesc Altet)
Date: Wed, 8 Nov 2006 15:44:58 +0100
Subject: reading matrix from a file
In-Reply-To: <20061108124239.20230.qmail@web8409.mail.in.yahoo.com>
References: <20061108124239.20230.qmail@web8409.mail.in.yahoo.com>
Message-ID: <200611081544.59856.faltet@carabos.com>

A Dimecres 08 Novembre 2006 13:42, amit soni escrigu?:
> Hi,
>   i have a file with following format:
>  1 2
>  3 9
>  2 3
>  4 4
>    I want to read it and then store the values into two matrices, s.t.
>  A=[1 2;3 9]
>  B=[2 3;4 4]
>
>   Can anyone tell me how to do this in python?
>  thanks
>  Amit

There are many possibilities. One of them could be:

In [64]: a = []; b = []

In [65]: for i, line in enumerate(file("/tmp/data.txt")):
   ....:     if i < 2:
   ....:         a.extend([float(n) for n in line.split()])
   ....:     else:
   ....:         b.extend([float(n) for n in line.split()])
   ....:

In [66]: A=numpy.array(a).reshape(2,2); B=numpy.array(b).reshape(2,2)

In [67]: A, B
Out[67]:
(array([[ 1.,  2.],
       [ 3.,  9.]]),
 array([[ 2.,  3.],
       [ 4.,  4.]]))


HTH,

-- 
>0,0<   Francesc Altet ? ? http://www.carabos.com/
V   V   C?rabos Coop. V. ??Enjoy Data
 "-"


-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
_______________________________________________
Numpy-discussion mailing list
Numpy-discussion at lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/numpy-discussion

From roberto at dealmeida.net  Wed Nov  8 09:45:15 2006
From: roberto at dealmeida.net (Roberto De Almeida)
Date: Wed, 8 Nov 2006 10:45:15 -0400
Subject: matrix multiplication (newbie question)
In-Reply-To: <f4f93d420611080632h3b3eb058qd3d0d0575d116988@mail.gmail.com>
References: <20061108135417.6906.qmail@web50913.mail.yahoo.com>
	<f4f93d420611080632h3b3eb058qd3d0d0575d116988@mail.gmail.com>
Message-ID: <10c662fe0611080645p6af724b1w2983e599d0157f81@mail.gmail.com>

On 11/8/06, Keith Goodman <kwgoodman at gmail.com> wrote:
> > Sorry if this is an obvious question, but what is the easiest way to
> > multiply matrices in numpy? Suppose I want to do A=B*C*D.
> If B and C and D are matrices, then '*' is matrix multiplication.

I think the difference between arrays and matrices is not clear for
him (he's clearly multiplying arrays).

Izak, you should first convert you arrays to matrices using ``numpy.matrix``.

--Rob

-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From aswart at dip.sun.ac.za  Wed Nov  8 09:47:33 2006
From: aswart at dip.sun.ac.za (Albert Swart)
Date: Wed, 08 Nov 2006 16:47:33 +0200
Subject: matrix multiplication (newbie question)
In-Reply-To: <10c662fe0611080645p6af724b1w2983e599d0157f81@mail.gmail.com>
References: <20061108135417.6906.qmail@web50913.mail.yahoo.com>
	<f4f93d420611080632h3b3eb058qd3d0d0575d116988@mail.gmail.com>
	<10c662fe0611080645p6af724b1w2983e599d0157f81@mail.gmail.com>
Message-ID: <1162997254.5646.19.camel@localhost>


> Izak, you should first convert you arrays to matrices using ``numpy.matrix``.

or numpy.asmatrix()
> 
> --Rob
> 
> -------------------------------------------------------------------------
> Using Tomcat but need to do more? Need to support web services, security?
> Get stuff done quickly with pre-integrated technology to make your job easier
> Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
> http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
> _______________________________________________
> Numpy-discussion mailing list
> Numpy-discussion at lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/numpy-discussion


-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From charlesr.harris at gmail.com  Wed Nov  8 09:48:48 2006
From: charlesr.harris at gmail.com (Charles R Harris)
Date: Wed, 8 Nov 2006 07:48:48 -0700
Subject: matrix multiplication (newbie question)
In-Reply-To: <f4f93d420611080632h3b3eb058qd3d0d0575d116988@mail.gmail.com>
References: <20061108135417.6906.qmail@web50913.mail.yahoo.com>
	<f4f93d420611080632h3b3eb058qd3d0d0575d116988@mail.gmail.com>
Message-ID: <e06186140611080648o7e6869a9vca150c7a9b506aa9@mail.gmail.com>

On 11/8/06, Keith Goodman <kwgoodman at gmail.com> wrote:
>
> On 11/8/06, izak marais <izakmarais at yahoo.com> wrote:
>
> > Sorry if this is an obvious question, but what is the easiest way to
> > multiply matrices in numpy? Suppose I want to do A=B*C*D. The ' * '
> operator
> > apparently does element wise multiplication, as does the 'multiply'
> ufunc.
> > All I could find was the numeric function 'matrix_multiply, but this
> only
> > takes two arguments.


Same with the operator *, it takes two arguments but is in infix order, i.e.,
left side and right side.

If B and C and D are matrices, then '*' is matrix multiplication.


And if they are arrays:

A = dot(B,dot(C,D))

Python has a dearth of recognized operators which makes this necessary once
'*' is used for elementwise multiplication, it's a long standing complaint.
You can use matrices in numpy, in which case '*' is used for matrix
multiplication like in matlab, but I think it would be better to get used to
using arrays as they are the numpy core.

Chuck
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20061108/a9a0fcdf/attachment-0001.html>
-------------- next part --------------
-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
-------------- next part --------------
_______________________________________________
Numpy-discussion mailing list
Numpy-discussion at lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/numpy-discussion

From svetosch at gmx.net  Wed Nov  8 09:51:09 2006
From: svetosch at gmx.net (Sven Schreiber)
Date: Wed, 08 Nov 2006 15:51:09 +0100
Subject: matrix multiplication (newbie question)
In-Reply-To: <20061108135417.6906.qmail@web50913.mail.yahoo.com>
References: <20061108135417.6906.qmail@web50913.mail.yahoo.com>
Message-ID: <4551EEDD.8010405@gmx.net>

izak marais schrieb:
> Hi
> 
> Sorry if this is an obvious question, but what is the easiest way to
> multiply matrices in numpy? Suppose I want to do A=B*C*D. The ' * '
> operator apparently does element wise multiplication, as does the
> 'multiply' ufunc. All I could find was the numeric function
> 'matrix_multiply, but this only takes two arguments.
> 
> Thanks in advance!
> Izak

There are (at least) two ways:

You can use 'dot', possibly nested.

or you can convert your arrays into the matrix subclass, for which '*'
is matrix multiplication. I.e. mat(B)*mat(C)*mat(D) does what you want.

If you "only" deal with algebra-style matrices (2d-arrays), consider
using the matrix subclass as much as you can. E.g. use the functions in
numpy.matlib to build your inputs.
-sven

-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From charlesr.harris at gmail.com  Wed Nov  8 09:55:41 2006
From: charlesr.harris at gmail.com (Charles R Harris)
Date: Wed, 8 Nov 2006 07:55:41 -0700
Subject: reading matrix from a file
In-Reply-To: <200611081544.59856.faltet@carabos.com>
References: <20061108124239.20230.qmail@web8409.mail.in.yahoo.com>
	<200611081544.59856.faltet@carabos.com>
Message-ID: <e06186140611080655h604a71f8q6dcd046ac248033e@mail.gmail.com>

On 11/8/06, Francesc Altet <faltet at carabos.com> wrote:
>
> A Dimecres 08 Novembre 2006 13:42, amit soni escrigu?:
> > Hi,
> >   i have a file with following format:
> >  1 2
> >  3 9
> >  2 3
> >  4 4
> >    I want to read it and then store the values into two matrices, s.t.
> >  A=[1 2;3 9]
> >  B=[2 3;4 4]
> >
> >   Can anyone tell me how to do this in python?
> >  thanks
> >  Amit


Try

In [8]: tmp = fromfile('tmp.txt', sep=' ', dtype=int)

In [9]: a = tmp[:4].reshape(2,2)

In [10]: b = tmp[4:].reshape(2,2)

In [11]: a
Out[11]:
array([[1, 2],
       [3, 9]])

In [12]: b
Out[12]:
array([[2, 3],
       [4, 4]])

Chuck
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20061108/fe8dbf12/attachment-0001.html>
-------------- next part --------------
-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
-------------- next part --------------
_______________________________________________
Numpy-discussion mailing list
Numpy-discussion at lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/numpy-discussion

From joris at ster.kuleuven.be  Wed Nov  8 09:00:00 2006
From: joris at ster.kuleuven.be (Joris De Ridder)
Date: Wed, 8 Nov 2006 15:00:00 +0100
Subject: matrix multiplication (newbie question)
In-Reply-To: <20061108135417.6906.qmail@web50913.mail.yahoo.com>
References: <20061108135417.6906.qmail@web50913.mail.yahoo.com>
Message-ID: <200611081500.00355.joris@ster.kuleuven.be>


   [im]: Sorry if this is an obvious question, but what is the easiest way to multiply matrices in numpy? Suppose I want to do A=B*C*D. The ' * ' operator apparently does element wise multiplication, as does the 'multiply' ufunc. 
   [im] All I could find was the numeric function 'matrix_multiply, but this only takes two arguments.

Have a look at the examples "dot()" and "mat()" in the Numpy Example List.
http://www.scipy.org/Numpy_Example_List

J.

Disclaimer: http://www.kuleuven.be/cwis/email_disclaimer.htm


-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From faltet at carabos.com  Wed Nov  8 10:01:31 2006
From: faltet at carabos.com (Francesc Altet)
Date: Wed, 8 Nov 2006 16:01:31 +0100
Subject: reading matrix from a file
In-Reply-To: <e06186140611080655h604a71f8q6dcd046ac248033e@mail.gmail.com>
References: <20061108124239.20230.qmail@web8409.mail.in.yahoo.com>
	<200611081544.59856.faltet@carabos.com>
	<e06186140611080655h604a71f8q6dcd046ac248033e@mail.gmail.com>
Message-ID: <200611081601.32248.faltet@carabos.com>

A Dimecres 08 Novembre 2006 15:55, Charles R Harris escrigu?:
> Try
>
> In [8]: tmp = fromfile('tmp.txt', sep=' ', dtype=int)
>
> In [9]: a = tmp[:4].reshape(2,2)
>
> In [10]: b = tmp[4:].reshape(2,2)
>
> In [11]: a
> Out[11]:
> array([[1, 2],
>        [3, 9]])
>
> In [12]: b
> Out[12]:
> array([[2, 3],
>        [4, 4]])

Yeah. Much, much better indeed.

-- 
>0,0<   Francesc Altet ? ? http://www.carabos.com/
V   V   C?rabos Coop. V. ??Enjoy Data
 "-"


-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
_______________________________________________
Numpy-discussion mailing list
Numpy-discussion at lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/numpy-discussion

From stefan at sun.ac.za  Wed Nov  8 10:20:02 2006
From: stefan at sun.ac.za (Stefan van der Walt)
Date: Wed, 8 Nov 2006 17:20:02 +0200
Subject: Profiling Python codes with hotshot and
	KCachegrind
In-Reply-To: <db6b5ecc0611080032n10bfa84fg6edb1e97f48f04ba@mail.gmail.com>
References: <db6b5ecc0611080032n10bfa84fg6edb1e97f48f04ba@mail.gmail.com>
Message-ID: <20061108152002.GP3380@mentat.za.net>

On Wed, Nov 08, 2006 at 01:32:44AM -0700, Fernando Perez wrote:
> Hi all,
> 
> in the past, Arnd Baecker has made a number of very useful posts on
> this matter, and provided some nice utilities to do it.  I now needed
> to profile some fairly complex codes prior to a big optimization push,
> so I went over his material and wrote a little tool to make the whole
> process as painless as possible.  Here it is, hoping others may find
> it useful:
> 
> http://amath.colorado.edu/faculty/fperez/python/profiling/

This looks very interesting.  It works for me on simple scripts, but
whenever I include the lines

from numpy.testing import set_local_path
set_local_path('../../..')

in the input, pycachegrind aborts with

  File "/home/stefan//lib/python2.4/site-packages/numpy/testing/numpytest.py", line 68, in set_local_path
    if f.f_locals['__name__']=='__main__':
KeyError: '__name__'

I guess this is because the script is run in a separate namespace.
I've managed to work around the problem by changing the definition of
'run' to:

def run(code):
    loc = locals()
    loc['__name__'] = '__main__'
    loc['__file__'] = sys.argv[0]

    exec code in locals()


Cheers
St?fan

-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From a.u.r.e.l.i.a.n at gmx.net  Wed Nov  8 11:16:17 2006
From: a.u.r.e.l.i.a.n at gmx.net (Johannes Loehnert)
Date: Wed, 8 Nov 2006 17:16:17 +0100
Subject: matrix multiplication (newbie question)
In-Reply-To: <20061108135417.6906.qmail@web50913.mail.yahoo.com>
References: <20061108135417.6906.qmail@web50913.mail.yahoo.com>
Message-ID: <200611081716.18422.a.u.r.e.l.i.a.n@gmx.net>

Hi,

in extension to the previous answers, I'd like to say that it is strongly 
preferable to use dot(A,dot(B,C)) or dot(dot(A,B),C) instead of A*B*C.

The reason is that with dot(), you can control of which operation is performed 
first, which can *massively* influence the time needed, depending on the 
involved matrices. A*B*C will always be evaluated left-to-right (if I 
remember correctly).

Johannes

-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From strawman at astraw.com  Wed Nov  8 11:57:44 2006
From: strawman at astraw.com (Andrew Straw)
Date: Wed, 08 Nov 2006 08:57:44 -0800
Subject: Passing numpy arrays to matlab
In-Reply-To: <4551ECF8.90806@ar.media.kyoto-u.ac.jp>
References: <454FEE21.7040205@ar.media.kyoto-u.ac.jp>	<45501396.3070908@astraw.com>
	<4551ECF8.90806@ar.media.kyoto-u.ac.jp>
Message-ID: <45520C88.6020902@astraw.com>

David Cournapeau wrote:
> Andrew Straw wrote:
>   
>> David Cournapeau wrote:
>>   
>>     
>>>     - To send data from the calling process to matlab, you first have to 
>>> create a mxArray, which is the basic matlab handler of a matlab array, 
>>> and populating it. Using mxArray is very ackward : you cannot create 
>>> mxArray from existing data, you have to copy data to them, etc... 
>>>     
>>>       
>> My understanding, never having done it, but from reading the docs, is
>> that you can create a "hybrid array" where you manage the memory. Thus,
>> you can create an mxArray from existing data. However, the docs
>> basically say that this is too hard for most mortals (and they may well
>> be right -- too painful for me, anyway)!
>>   
>>     
> Ok, I have looked at it. It is not hard, it is just totally brain 
> damaged: there is no way to destroy a mxArray without destroying the 
> data it is holding, even after a call with mxSetPr. So the data 
> referenced by the pointer given to mxSetPr is always destroyed by 
> mxDestroyArray; I don't see any way to use this to avoid copy... They 
> could at least have given a function which frees the data buffer and one 
> which destroys the other stuff; as it is, it is totally useless, unless 
> you don't mind memory leaks.
>   
It does sound brain damaged, I agree. But here's a suggestion: can you
keep a pool of unused mxArrays rather than calling mxDestroyArray? I
guess without the payload, they're just a few bytes and shouldn't take
up that much space.

-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From fperez.net at gmail.com  Wed Nov  8 11:57:28 2006
From: fperez.net at gmail.com (Fernando Perez)
Date: Wed, 8 Nov 2006 09:57:28 -0700
Subject: Profiling Python codes with hotshot and
	KCachegrind
In-Reply-To: <20061108152002.GP3380@mentat.za.net>
References: <db6b5ecc0611080032n10bfa84fg6edb1e97f48f04ba@mail.gmail.com>
	<20061108152002.GP3380@mentat.za.net>
Message-ID: <db6b5ecc0611080857p4ef97b52v2e22b54e4f7801d@mail.gmail.com>

On 11/8/06, Stefan van der Walt <stefan at sun.ac.za> wrote:

> This looks very interesting.  It works for me on simple scripts, but
> whenever I include the lines
>
> from numpy.testing import set_local_path
> set_local_path('../../..')
>
> in the input, pycachegrind aborts with
>
>   File "/home/stefan//lib/python2.4/site-packages/numpy/testing/numpytest.py", line 68, in set_local_path
>     if f.f_locals['__name__']=='__main__':
> KeyError: '__name__'
>
> I guess this is because the script is run in a separate namespace.
> I've managed to work around the problem by changing the definition of
> 'run' to:

Good catch, thanks, I've fixed the public version with your changes.

Best,

f

-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From amit_605 at yahoo.co.in  Wed Nov  8 12:36:14 2006
From: amit_605 at yahoo.co.in (amit soni)
Date: Wed, 8 Nov 2006 09:36:14 -0800 (PST)
Subject: Calculating tan inverse
Message-ID: <20061108173614.6777.qmail@web8414.mail.in.yahoo.com>

how can I calculate arctan of a number in python?
thanks
Amit

 
---------------------------------
Sponsored Link

Mortgage rates near 39yr lows. $420,000 Mortgage for $1,399/mo - Calculate new house payment
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20061108/b44ecd10/attachment-0001.html>
-------------- next part --------------
-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
-------------- next part --------------
_______________________________________________
Numpy-discussion mailing list
Numpy-discussion at lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/numpy-discussion

From nadavh at visionsense.com  Wed Nov  8 12:46:52 2006
From: nadavh at visionsense.com (Nadav Horesh)
Date: Wed, 8 Nov 2006 19:46:52 +0200
Subject: Calculating tan inverse
Message-ID: <07C6A61102C94148B8104D42DE95F7E8FCB351@exchange2k.envision.co.il>

There is arctan function in numpy, and in math (atan, atan2)

 

  Nadav.

 

  _____  

From: numpy-discussion-bounces at lists.sourceforge.net
[mailto:numpy-discussion-bounces at lists.sourceforge.net] On Behalf Of
amit soni
Sent: Wednesday, November 08, 2006 19:36
To: numpy-discussion at lists.sourceforge.net
Subject: [Numpy-discussion] Calculating tan inverse

 

how can I calculate arctan of a number in python?
thanks
Amit

  

  _____  

Sponsored Link

Mortgage rates near 39yr lows. $420,000 Mortgage for $1,399/mo -
Calculate new house payment
<http://www.lowermybills.com/lre/index.jsp?sourceid=lmb-9069-16273&moid=
4116> 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20061108/dae0706c/attachment-0001.html>
-------------- next part --------------
-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
-------------- next part --------------
_______________________________________________
Numpy-discussion mailing list
Numpy-discussion at lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/numpy-discussion

From faltet at carabos.com  Wed Nov  8 13:00:08 2006
From: faltet at carabos.com (Francesc Altet)
Date: Wed, 8 Nov 2006 19:00:08 +0100
Subject: Calculating tan inverse
In-Reply-To: <20061108173614.6777.qmail@web8414.mail.in.yahoo.com>
References: <20061108173614.6777.qmail@web8414.mail.in.yahoo.com>
Message-ID: <200611081900.08524.faltet@carabos.com>

A Dimecres 08 Novembre 2006 18:36, amit soni escrigu?:
> how can I calculate arctan of a number in python?
> thanks
> Amit

Have you tried to read the numpy docs? some googling? In general, you
can get a lot of insight by querying this to Google:

"your words" site:www.scipy.org

in particular, try:

arctan site:www.scipy.org

Cheers,

-- 
>0,0<   Francesc Altet ? ? http://www.carabos.com/
V   V   C?rabos Coop. V. ??Enjoy Data
 "-"



-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
_______________________________________________
Numpy-discussion mailing list
Numpy-discussion at lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/numpy-discussion

From peridot.faceted at gmail.com  Wed Nov  8 13:41:55 2006
From: peridot.faceted at gmail.com (A. M. Archibald)
Date: Wed, 8 Nov 2006 13:41:55 -0500
Subject: A reimplementation of MaskedArray
In-Reply-To: <200611080510.13565.pgmdevlist@gmail.com>
References: <200610160232.20888.pgmdevlist@mailcan.com>
	<200610241408.40415.pgmdevlist@gmail.com>
	<16761e100611071711q6b9cc464p85708d7a642b9a8a@mail.gmail.com>
	<200611080510.13565.pgmdevlist@gmail.com>
Message-ID: <ce557a360611081041t6748bcdj953aa6aaea9cab18@mail.gmail.com>

On 08/11/06, Pierre GM <pgmdevlist at gmail.com> wrote:

> I like your idea, but not its implementation. If MA.masked_singleton is
> defined as an object, as you suggest, then the dtype of the ndarray it is
> passed to becomes 'object', as you pointed out, and that is not something one
> would naturally expec, as basic numerical functions don't  work well  with the
> 'object' dtype (just try  N.sqrt(N.array([1],dtype=N.object)) to see what I
> mean).
> Even if we can construct a mask rather easily at the creation of the masked
> array, following your 'a==masked' suggestion, we still need to get the dtype
> of the non-masked section, and that doesn't seem trivial...

A good candidate for "should be masked" marked is NaN. It is supposed
to mean, more or less, "no sensible value". Unfortunately, integer
types do not have such a special value. It's also conceivable that
some user might want to keep NaNs in their array separate from the
mask. Finally, on some hardware, operations with NaN are very slow (so
leaving them in the array, even masked, might not be a good idea).

The reason I suggest this is that in the last major application I had
for numpy, one stage of the problem would occasionally result in NaNs
for certain values, but the best thing I could do was leave them in
place to represent "no data". Switching to a MaskedArray might have
been a better idea, but the NaNs were a rare occurrence.

> About the conversion to ndarray:
> By default, the result should have the same dtype as the _data section.
> For this reason, I disagree with your idea of "(returning) an object ndarray
> with the missing value containing the masked singleton". If you really want
> an object ndarray, you can use the filled method or the filled function, with
> your own definition of the filling value (such as your MaskedScalar).

If you've got floating point, you can again fill in NaNs, but you have
a good point about wanting to extract the original values that were
masked out. Depending on what one is doing, one might want one or the
other.

A. M. Archibald

-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From pgmdevlist at gmail.com  Wed Nov  8 16:08:25 2006
From: pgmdevlist at gmail.com (Pierre GM)
Date: Wed, 8 Nov 2006 16:08:25 -0500
Subject: A reimplementation of MaskedArray
In-Reply-To: <ce557a360611081041t6748bcdj953aa6aaea9cab18@mail.gmail.com>
References: <200610160232.20888.pgmdevlist@mailcan.com>
	<200611080510.13565.pgmdevlist@gmail.com>
	<ce557a360611081041t6748bcdj953aa6aaea9cab18@mail.gmail.com>
Message-ID: <200611081608.25872.pgmdevlist@gmail.com>

> A good candidate for "should be masked" marked is NaN. It is supposed
> to mean, more or less, "no sensible value". 

Which might turn out out to be the best indeed. Michael's application would 
then look like
>>> import numpy as N
>>> import maskedarray as MA
>>> maskit = N.nan
>>> test = N.array([1,2,maskit])
>>> test_ma1 = MA.array(x,mask=N.isnan(x))

> Switching to a MaskedArray might have
> been a better idea, but the NaNs were a rare occurrence.
Once again, that's a situation when one would use masked arrays. 

> If you've got floating point, you can again fill in NaNs, but you have
> a good point about wanting to extract the original values that were
> masked out. Depending on what one is doing, one might want one or the
> other.

In any case, I think we should stick to the numpy.core.ma default behavior for 
backwards compatibility. If you really wanht to distinguish between several 
kind of masks (one for missing data, one for data to discard temporarily), 
that could be done by defining a special subclass. But is it really needed ? 
A smart use of filled and masked_values should do the trick in most cases.

-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From svetosch at gmx.net  Wed Nov  8 17:00:03 2006
From: svetosch at gmx.net (Sven Schreiber)
Date: Wed, 08 Nov 2006 23:00:03 +0100
Subject: matrix multiplication (newbie question)
In-Reply-To: <200611081716.18422.a.u.r.e.l.i.a.n@gmx.net>
References: <20061108135417.6906.qmail@web50913.mail.yahoo.com>
	<200611081716.18422.a.u.r.e.l.i.a.n@gmx.net>
Message-ID: <45525363.1030404@gmx.net>

Johannes Loehnert schrieb:
> Hi,
> 
> in extension to the previous answers, I'd like to say that it is strongly 
> preferable to use dot(A,dot(B,C)) or dot(dot(A,B),C) instead of A*B*C.
> 
> The reason is that with dot(), you can control of which operation is performed 
> first, which can *massively* influence the time needed, depending on the 
> involved matrices. A*B*C will always be evaluated left-to-right (if I 
> remember correctly).
> 

Well what about A*(B*C)?

-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From koara at atlas.cz  Wed Nov  8 17:36:33 2006
From: koara at atlas.cz (koara)
Date: Wed, 08 Nov 2006 14:36:33 -0800
Subject: slice assignment: strange behaviour
Message-ID: <1163025393.211885.161470@e3g2000cwe.googlegroups.com>

Hello,

a piece of my code started giving strange results with certain data; i
managed to track down the cause to a slice array assignment. In the
following code snip; 'mat' is a numpy.array with shape=(22973, 1009),
'vec' is a numpy.array with shape=(22973,), both of type int:

for i in xrange(1009):
    ...
    fr = vec[10001]
    mat[:, i] = vec # assign whole column
    if mat[10001, i] != fr:
        print "how come?"
     ...

for elements beyond index 10000, nothing is assigned (ie,
numpy.sum(mat[row, :]) is zero for any row > 10000). As soon as i
replace the assignment with a cycle that assigns each element
explicitly (for j in xrange(22973): result[j, i] = vec[j]),
everything's OK. With some matrices, the above seems to work fine
though, as well as for smaller dimensions, so i am unable to provide a
full simple example.

Any ideas? I am using enthought python (python 2.4.3) which uses numpy
version 0.9.9.


-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From vincent.broman at navy.mil  Wed Nov  8 17:20:55 2006
From: vincent.broman at navy.mil (Vincent Broman)
Date: Wed, 8 Nov 2006 22:20:55 +0000 (UTC)
Subject: building fc6 rpm of numpy
Message-ID: <loom.20061108T230408-554@post.gmane.org>

Building an rpm of numpy-1.0.1.dev3432-1 on fedora core 6 is failing for me.
With either python-2.4.3 or 2.4.4 I try "python setup.py bdist_rpm"
inside the source directory, and everything seems to go well except for
many "File listed twice" messages for all kinds of files,
and then at the end there is the error message:

    error: Installed (but unpackaged) file(s) found

with a long list of .pyc and .pyo files.
The same seems to happen to me with the source rpm I downloaded
from sourceforge. And I vaguely recall the same happening with
my fedora core 4 + updates, which I had till my recent upgrade to fc6.

The list or lists of files to install are not in the %files section,
but are generated dynamically somehow by setup.py, but I don't
know enough about distutils to debug that.

Anyone succeed with fc6 rpms or have any suggestion?

Vincent Broman
SSC-SD



-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From koara at atlas.cz  Wed Nov  8 18:08:27 2006
From: koara at atlas.cz (koara)
Date: Wed, 08 Nov 2006 15:08:27 -0800
Subject: slice assignment: strange behaviour
In-Reply-To: <1163025393.211885.161470@e3g2000cwe.googlegroups.com>
References: <1163025393.211885.161470@e3g2000cwe.googlegroups.com>
Message-ID: <1163027307.938949.183140@m73g2000cwd.googlegroups.com>


koara wrote:
> Hello,
>
> a piece of my code started giving strange results with certain data; i
> managed to track down the cause to a slice array assignment. In the
> ....

Also if i first build a sequence of columns and then use
numpy.transpose(numpy.vstack(sequence)) the result is ok. But the
arrays can be quite big so in this case i fear space overhead -- or am
i wrong?

Cheers.


-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From fullung at gmail.com  Wed Nov  8 18:14:28 2006
From: fullung at gmail.com (Albert Strasheim)
Date: Thu, 9 Nov 2006 01:14:28 +0200
Subject: building fc6 rpm of numpy
In-Reply-To: <loom.20061108T230408-554@post.gmane.org>
References: <loom.20061108T230408-554@post.gmane.org>
Message-ID: <20061108231428.GA6848@dogbert.sdsl.sun.ac.za>

Howdy

On Wed, 08 Nov 2006, Vincent Broman wrote:

> Building an rpm of numpy-1.0.1.dev3432-1 on fedora core 6 is failing for me.
> With either python-2.4.3 or 2.4.4 I try "python setup.py bdist_rpm"
> inside the source directory, and everything seems to go well except for
> many "File listed twice" messages for all kinds of files,
> and then at the end there is the error message:
> 
>     error: Installed (but unpackaged) file(s) found
> 
> with a long list of .pyc and .pyo files.
> The same seems to happen to me with the source rpm I downloaded
> from sourceforge. And I vaguely recall the same happening with
> my fedora core 4 + updates, which I had till my recent upgrade to fc6.
> 
> The list or lists of files to install are not in the %files section,
> but are generated dynamically somehow by setup.py, but I don't
> know enough about distutils to debug that.
> 
> Anyone succeed with fc6 rpms or have any suggestion?

I also ran into this problem a few days ago. I'm guessing it's a bug in 
either distutils or numpy.distutils.

Anyway, you can hack it to work by putting

%_unpackaged_files_terminate_build 1

in your ~/.rpmmacros. More details here:

http://www.rpm.org/hintskinks/unpackaged-files/

With that in place, bdist_rpm works on my FC6 machine. rpm -qpl on the 
RPM seems to indicate a "sane" RPM.

Cheers,

Albert

-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From fullung at gmail.com  Wed Nov  8 18:18:52 2006
From: fullung at gmail.com (Albert Strasheim)
Date: Thu, 9 Nov 2006 01:18:52 +0200
Subject: building fc6 rpm of numpy
In-Reply-To: <20061108231428.GA6848@dogbert.sdsl.sun.ac.za>
References: <loom.20061108T230408-554@post.gmane.org>
	<20061108231428.GA6848@dogbert.sdsl.sun.ac.za>
Message-ID: <20061108231852.GB6848@dogbert.sdsl.sun.ac.za>

Argh,

On Thu, 09 Nov 2006, Albert Strasheim wrote:

> %_unpackaged_files_terminate_build 1

Cut and paste error. Make that

%_unpackaged_files_terminate_build 0

Cheers,

Albert

-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From robert.kern at gmail.com  Wed Nov  8 18:26:23 2006
From: robert.kern at gmail.com (Robert Kern)
Date: Wed, 08 Nov 2006 17:26:23 -0600
Subject: slice assignment: strange behaviour
In-Reply-To: <1163027307.938949.183140@m73g2000cwd.googlegroups.com>
References: <1163025393.211885.161470@e3g2000cwe.googlegroups.com>
	<1163027307.938949.183140@m73g2000cwd.googlegroups.com>
Message-ID: <eitp38$ro7$1@sea.gmane.org>

koara wrote:
> koara wrote:
>> Hello,
>>
>> a piece of my code started giving strange results with certain data; i
>> managed to track down the cause to a slice array assignment. In the
>> ....
> 
> Also if i first build a sequence of columns and then use
> numpy.transpose(numpy.vstack(sequence)) the result is ok. But the
> arrays can be quite big so in this case i fear space overhead -- or am
> i wrong?

You might want to use column_stack() instead, for clarity if nothing else.

-- 
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless enigma
 that is made terrible by our own mad attempt to interpret it as though it had
 an underlying truth."
  -- Umberto Eco


-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From wbaxter at gmail.com  Wed Nov  8 18:39:07 2006
From: wbaxter at gmail.com (Bill Baxter)
Date: Thu, 9 Nov 2006 08:39:07 +0900
Subject: matrix multiplication (newbie question)
In-Reply-To: <200611081500.00355.joris@ster.kuleuven.be>
References: <20061108135417.6906.qmail@web50913.mail.yahoo.com>
	<200611081500.00355.joris@ster.kuleuven.be>
Message-ID: <e86a5fd00611081539m6ccceb0aj7f59a826e55f4b3b@mail.gmail.com>

Also have a look at the section on Arrays vs Matrices in the Numpy for
Matlab users page.  That particular section has nothing to do with
Matlab, really.
http://www.scipy.org/NumPy_for_Matlab_Users
Most of the suggestions and comments made here are already on that page.

--bb

On 11/8/06, Joris De Ridder <joris at ster.kuleuven.be> wrote:
>
>    [im]: Sorry if this is an obvious question, but what is the easiest way to multiply matrices in numpy? Suppose I want to do A=B*C*D. The ' * ' operator apparently does element wise multiplication, as does the 'multiply' ufunc.
>    [im] All I could find was the numeric function 'matrix_multiply, but this only takes two arguments.
>
> Have a look at the examples "dot()" and "mat()" in the Numpy Example List.
> http://www.scipy.org/Numpy_Example_List
>
> J.
>
> Disclaimer: http://www.kuleuven.be/cwis/email_disclaimer.htm
>
>
> -------------------------------------------------------------------------
> Using Tomcat but need to do more? Need to support web services, security?
> Get stuff done quickly with pre-integrated technology to make your job easier
> Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
> http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
> _______________________________________________
> Numpy-discussion mailing list
> Numpy-discussion at lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/numpy-discussion
>

-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From ndbecker2 at gmail.com  Wed Nov  8 18:42:52 2006
From: ndbecker2 at gmail.com (Neal Becker)
Date: Wed, 08 Nov 2006 18:42:52 -0500
Subject: building fc6 rpm of numpy
References: <loom.20061108T230408-554@post.gmane.org>
Message-ID: <eitq1h$uei$1@sea.gmane.org>

Try the enclosed spec file.  Also, you can use the one from Fedora devel (I
think the one I enclosed is the same).
-------------- next part --------------
An embedded and charset-unspecified text was scrubbed...
Name: numpy.spec
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20061108/8f7ddced/attachment-0001.ksh>
-------------- next part --------------
-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
-------------- next part --------------
_______________________________________________
Numpy-discussion mailing list
Numpy-discussion at lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/numpy-discussion

From david at ar.media.kyoto-u.ac.jp  Wed Nov  8 21:31:07 2006
From: david at ar.media.kyoto-u.ac.jp (David Cournapeau)
Date: Thu, 09 Nov 2006 11:31:07 +0900
Subject: Passing numpy arrays to matlab
In-Reply-To: <1e2af89e0611071057h70b2f575u3943efbc142ca9c8@mail.gmail.com>
References: <454FEE21.7040205@ar.media.kyoto-u.ac.jp>	<45501396.3070908@astraw.com>	<455012BF.1030105@ar.media.kyoto-u.ac.jp>	<45501E70.8090609@astraw.com>	<45501F38.3090201@ar.media.kyoto-u.ac.jp>
	<1e2af89e0611071057h70b2f575u3943efbc142ca9c8@mail.gmail.com>
Message-ID: <455292EB.4050506@ar.media.kyoto-u.ac.jp>

Matthew Brett wrote:
>
> I would be very happy to help with this.  It would be great if we
> could get a standard well-maintained library of some sort towards
> scipy - we (http://neuroimaging.scipy.org/) have a great deal of
> matlab integration to do.
>   
I am a bit busy and late on my PhD schedule; I will try to release 
something remotely usable by other people within the end of the week, so 
other people can start hacking on it too,

cheers,

David

-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From ylsjeqgh at parrishmed.com  Wed Nov  8 22:44:24 2006
From: ylsjeqgh at parrishmed.com (MONTANA NEBRASKA)
Date: Thu, 9 Nov 2006 04:44:24 +0100
Subject: reason
Message-ID: <000d01c703b1$59511810$3115a555@petter>


Spam postage charges or strings attached Whyi!
Drugs often in unable even to once. Visitor Hereplain simple.
Threads Posts am Members or Welcome newest member spacerocks.
Updated program French lot called mtm available address.
Tsig Vermonter worzel Statistics Threads Posts Members Welcome newest.
Even to or once though ghetto popular among younger affluent.
Apply Name Title City Alabama Alaska Arizona Arkansas or. Put thisin interview fired back article such behavior.
Chamber guarded by grinning brutal orcsin adventure you a.
Anything site is Regular shaped a cookies soft or chocolate. If visit of sure check faq.
Serenitude or Against Mainstream. Approver rss dc Free in?
Often unable even to.
Wont stopjones action of! Nice in still havent or.
Often unable even to. Equipped developed opensource understand in reason anonymity.
Hardware in Benefits Feeds Subscribe Events in or Tech am.
Support Billing amp Account am Features.
Import url Click here or sign?
Omissions of page may reported Copyright copy Mobygames all. Discuss stories posted Today than a!
Drugs often in unable even to once.
Courtyards is in Tower am Cirith Ungol horrifying torture chamber. Members List in Calendar.
Credits There in file of release game Everything usersadd Errors.
Members List in Calendar.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20061109/5afcc4ab/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: give.gif
Type: image/gif
Size: 10326 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20061109/5afcc4ab/attachment-0001.gif>
-------------- next part --------------
-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
-------------- next part --------------
_______________________________________________
Numpy-discussion mailing list
Numpy-discussion at lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/numpy-discussion

From pfdubois at gmail.com  Wed Nov  8 23:00:58 2006
From: pfdubois at gmail.com (Paul Dubois)
Date: Wed, 8 Nov 2006 20:00:58 -0800
Subject: A reimplementation of MaskedArray
In-Reply-To: <200611081608.25872.pgmdevlist@gmail.com>
References: <200610160232.20888.pgmdevlist@mailcan.com>
	<200611080510.13565.pgmdevlist@gmail.com>
	<ce557a360611081041t6748bcdj953aa6aaea9cab18@mail.gmail.com>
	<200611081608.25872.pgmdevlist@gmail.com>
Message-ID: <f74a6c2f0611082000k31f8b596r18e635d687e4866d@mail.gmail.com>

2 cents from the author of the first folio:

The intent was to allow creation of masked arrays with copy=no, so that the
original data could be retrieved from it if desired. But I was quite, quite
rigorous about NEVER assuming the data in a masked slot made any sense
whatsoever.

The intention was that there are two ways to get a numeric array out of a
masked one:

1. Get the data field
2. m.filled()

(1) is strictly at your own risk.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20061108/0932feb3/attachment-0001.html>
-------------- next part --------------
-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
-------------- next part --------------
_______________________________________________
Numpy-discussion mailing list
Numpy-discussion at lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/numpy-discussion

From josh.p.marshall at gmail.com  Wed Nov  8 23:18:54 2006
From: josh.p.marshall at gmail.com (Josh Marshall)
Date: Thu, 9 Nov 2006 15:18:54 +1100
Subject: Passing numpy arrays to matlab
In-Reply-To: <mailman.126689.1162987742.22397.numpy-discussion@lists.sourceforge.net>
References: <mailman.126689.1162987742.22397.numpy-discussion@lists.sourceforge.net>
Message-ID: <8B4A9501-180E-4714-85F9-AB483B59099B@gmail.com>

Hi David,

Sorry for the late reply. Can you CC any reply to me as well, as I  
just get the digests and read them every few days.

On 08/11/2006, at 11:09 PM, David Cournapeau wrote:

> I didn't know that, thanks. Unfortunately, it is not really what I am
> trying to do: mlabwrap is just a python interface a bit more high  
> level
> than pymat, with many fancy tricks, but still do copies. What I would
> like is to avoid completely the copying by using proxy classes around
> data from numpy so that I can pass "automatically" numpy arrays to
> matlab C api, and a proxy class around data from matlab so that they
> look like numpy arrays.

I don't see how you are going to get around doing the copies. Matlab  
is in a separate process from the Python interpreter, and there is no  
shared memory. In what way do you want these proxy classes to "look  
like numpy arrays"?

Note that mlabwrap creates proxy arrays, and only copies the data if  
you actually request it to. (AFAIRemember) Otherwise you aren't  
losing any speed, because there aren't going to be any copies.

>     I don't care that much about the actual api from python point of
> view, because I intend to use this mainly to compare matlab vs numpy
> implementation, not as a way to use matlab inside python regularly.  
> And
> once the copy problem is solved, adding syntactic sugar using  
> python is
> easy anyway, I think (it should be easy to do something similar to
> mlabwrap at that point),

What could be possible to do is add an array interface to the  
mlabwrap proxy classes so they can be used as numpy arrays when  
required for passing to numpy functions (or PIL, etc). Thus we only  
copy when we want to use numpy functions. Then we could define the  
operators on the proxy class to perform their operations on the other  
side of the bridge.

Cheers,
Josh


-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 2433 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20061109/fb34db35/attachment-0001.bin>
-------------- next part --------------
-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
-------------- next part --------------
_______________________________________________
Numpy-discussion mailing list
Numpy-discussion at lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/numpy-discussion

From tim.hochberg at ieee.org  Wed Nov  8 14:24:34 2006
From: tim.hochberg at ieee.org (Tim Hochberg)
Date: Wed, 08 Nov 2006 12:24:34 -0700
Subject: A reimplementation of MaskedArray
In-Reply-To: <ce557a360611081041t6748bcdj953aa6aaea9cab18@mail.gmail.com>
References: <200610160232.20888.pgmdevlist@mailcan.com>	<200610241408.40415.pgmdevlist@gmail.com>	<16761e100611071711q6b9cc464p85708d7a642b9a8a@mail.gmail.com>	<200611080510.13565.pgmdevlist@gmail.com>
	<ce557a360611081041t6748bcdj953aa6aaea9cab18@mail.gmail.com>
Message-ID: <45522EF2.8000603@ieee.org>

A. M. Archibald wrote:
> On 08/11/06, Pierre GM <pgmdevlist at gmail.com> wrote:
>
>   
>> I like your idea, but not its implementation. If MA.masked_singleton is
>> defined as an object, as you suggest, then the dtype of the ndarray it is
>> passed to becomes 'object', as you pointed out, and that is not something one
>> would naturally expec, as basic numerical functions don't  work well  with the
>> 'object' dtype (just try  N.sqrt(N.array([1],dtype=N.object)) to see what I
>> mean).
>> Even if we can construct a mask rather easily at the creation of the masked
>> array, following your 'a==masked' suggestion, we still need to get the dtype
>> of the non-masked section, and that doesn't seem trivial...
>>     
>
> A good candidate for "should be masked" marked is NaN. It is supposed
> to mean, more or less, "no sensible value". Unfortunately, integer
> types do not have such a special value. It's also conceivable that
> some user might want to keep NaNs in their array separate from the
> mask. Finally, on some hardware, operations with NaN are very slow (so
> leaving them in the array, even masked, might not be a good idea).
>   
It has always been my experience (on various flavors or Pentium) that 
operating on NANs is extremely slow. Does anyone know on what hardware 
NANs are *not* slow? Of course it's always possible I just never notice 
NANs on hardware where they aren't slow.


[SNIP]

-tim


-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From David.L.Goldsmith at noaa.gov  Wed Nov  8 22:06:21 2006
From: David.L.Goldsmith at noaa.gov (David L Goldsmith)
Date: Wed, 08 Nov 2006 19:06:21 -0800
Subject: Help, me emails aren't getting through!
Message-ID: <45529B2D.2010505@noaa.gov>

If the subject line is true, no one will see this; if it's false, please 
ignore! :-)

DG
-- 
HMRD/ORR/NOS/NOAA <http://response.restoration.noaa.gov/emergencyresponse/>

-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From David.L.Goldsmith at noaa.gov  Wed Nov  8 19:08:17 2006
From: David.L.Goldsmith at noaa.gov (David L Goldsmith)
Date: Wed, 08 Nov 2006 16:08:17 -0800
Subject: Anyone have a "little" shooting-method function
	to share
Message-ID: <45527171.8030903@noaa.gov>

Hi!  I tried to send this earlier: it made it into my sent mail folder, 
but does not appear to have made it to the list.

I need to numerically solve:
    (1-t)x" + x' - x = f(t), x(0) = x0, x(1) = x1
I've been trying to use (because it's the approach I inherited) an 
elementary finite-difference discretization, but unit tests have shown 
that that approach isn't working.  After a little review, I believe I 
understand the problem: near t=0, the thing is like an 
advection-diffusion equation with diffusion as strong as advection (a 
case I'm sure is treated somewhere in the literature, but not as easily 
findable as:) near t=1, the thing is like the advection-diffusion 
equation I did easily find treated, namely one where advection 
dominates.  Based on that treatment, I understand why an FD approach 
(with a uniform grid) would fail around t=0, and from there it is easy 
to accept that the fact that the thing changes its "nature" over the 
course of its "life," implies that (a uniform grid) FD is probably not a 
very good approach for this equation anywhere on its domain.  I could 
try (and maybe will have to) a variable grid FD approach, but I'd first 
like to try a "shooting" method (for some reason my intuition is telling 
me that in this case this might be more efficient).  This is where you 
all come in: I understand the algorithm and could (may have to) code it 
myself, but if anyone out there already has code for this (in Python 
using numpy, and preferably already under test), might you be willing to 
share?  Thanks in advance,

David Goldsmith

PS: In the interim, I realized that if something like this "pre-exists" 
in a module, that module might be scipy.  Sure enough, scipy has 
integrate.ode and integrate.odeint; Google-ing scipy: integrate.odeint 
help led me to Travis's 2004 "SciPy Tutorial," where I find "There are 
many optional inputs and outputs available when using odeint which can 
help tune the solver.  These additional inputs and outputs are not 
needed much of the time, however..."  Well, is one of those inputs an 
algorithm specifier?  Alternatively, is there a "shooting" algorithm 
implemented elsewhere in scipy?


-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From David.L.Goldsmith at noaa.gov  Wed Nov  8 16:49:00 2006
From: David.L.Goldsmith at noaa.gov (David L Goldsmith)
Date: Wed, 08 Nov 2006 13:49:00 -0800
Subject: Anyone have a "little" shooting-method function
	to share
Message-ID: <455250CC.4040608@noaa.gov>

Hi!  I need to numerically solve:
     (1-t)x" + x' - x = f(t), x(0) = x0, x(1) = x1
I've been trying to use (because it's the approach I inherited) an 
elementary finite-difference discretization, but unit tests have shown 
that that approach isn't working.  After a little review, I believe I 
understand the problem: near t=0, the thing is like an 
advection-diffusion equation with diffusion as strong as advection (a 
case I'm sure is treated somewhere in the literature, but not as easily 
findable as:) near t=1, the thing is like the advection-diffusion 
equation I did easily find treated, namely one where advection 
dominates.  Based on that treatment, I understand why an FD approach 
(with a uniform grid) would fail around t=0, and from there it is easy 
to accept that the fact that the thing changes its "nature" over the 
course of its "life," implies that (a uniform grid) FD is probably not a 
very good approach for this equation anywhere on its domain.  I could 
try (and maybe will have to) a variable grid FD approach, but I'd first 
like to try a "shooting" method (for some reason my intuition is telling 
me that in this case this might be more efficient).  This is where you 
all come in: I understand the algorithm and could (may have to) code it 
myself, but if anyone out there already has code for this (in Python 
using numpy, and preferably already under test), might you be willing to 
share?  Thanks in advance,

David Goldsmith
-- 
HMRD/ORR/NOS/NOAA <http://response.restoration.noaa.gov/emergencyresponse/>

-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From david at ar.media.kyoto-u.ac.jp  Thu Nov  9 01:32:04 2006
From: david at ar.media.kyoto-u.ac.jp (David Cournapeau)
Date: Thu, 09 Nov 2006 15:32:04 +0900
Subject: Passing numpy arrays to matlab
In-Reply-To: <8B4A9501-180E-4714-85F9-AB483B59099B@gmail.com>
References: <mailman.126689.1162987742.22397.numpy-discussion@lists.sourceforge.net>
	<8B4A9501-180E-4714-85F9-AB483B59099B@gmail.com>
Message-ID: <4552CB64.2080604@ar.media.kyoto-u.ac.jp>

Josh Marshall wrote:
>
>
> I don't see how you are going to get around doing the copies. Matlab 
> is in a separate process from the Python interpreter, and there is no 
> shared memory. In what way do you want these proxy classes to "look 
> like numpy arrays"?
I am not talking about the copy in the matlab <-> python interaction. 
This is done through pipe, handled by the OS; I don't know the details, 
but I know that communication through pipe is quite fast under linux 
(see below), and is not the bottleneck.

>
> Note that mlabwrap creates proxy arrays, and only copies the data if 
> you actually request it to. (AFAIRemember) Otherwise you aren't losing 
> any speed, because there aren't going to be any copies.
There may be no copy for returned data you don't need, but that's not 
the case I am talking about. For all other cases, I don't think this is 
what's happening: if you take a look at mlabwrap, in the C mlabraw 
module, the function mlabraw_put always calls numeric2mx for arrays, 
which itself always calls makeMxFromNumeric, which makes a copy. Same in 
the other direction once you call mlabwrap_get. I am doing the same in 
my module, because that's the simplest thing to do.

The problem is that when you are using the function engPutVariable of 
the matlab engine API, you need to give a pointer to a mxArray 
structure, which is the C representation of a matlab array. You cannot 
say (this is one of the brain damaged thing of matlab C api I was 
talking in an other mail): build a mxArray from existing data: this is 
the copy I am talking about, and this is one expensive. In the best case 
(real numpy arrays with fortran storage), you can do a memcpy, but in 
most cases, you need to do something which takes strides into account 
(because complex matlab arrays are actually not fortran, or because by 
default, most numpy arrays are C storage, and this makes a difference 
for rank >= 2), which implies non-contiguous memory access, which is 
*really* expensive (around 2 cycles/byte at best, on my bi Xeon 3.2 Ghz).

Basically, if you want to do something like calling the resample 
function of matlab on an numpy array and using the result later in 
numpy, here is what's happening right now:
   
    1 copy numpy (or numarray in the case of mlabwrap, but this should 
not matter, I guess) data into an mxArray
    2 send the mxArray to matlab engine: done with pipe (imply copy ? At 
least, it is contiguous array copy)
    3 compute the thing into matlab
    4 send the result to python mxArray
    5 copy the data of the mxArray to numpy array

A quick profiling show that if you don't do any processing in matlab, 
just sending and getting an array back, 1 and 5 takes roughly 80-90 % of 
the time in my implementation (which is faster than mlabwrap, but I 
think this is just caused by the much fancier API of mlabwrap, ie the 
core mecanism to pass arrays should be roughly the same, as mlabwrap 
uses the C function makeMxFromNumeric, and I am using a similar function 
myself through ctypes), the 10-20% are used for the communication 
through the pipe. I believe that most typical usage cases involve 1 and 5.

5 should be avoidable in many cases if I know how to build a proxy class 
around the mxArray so that the the proxy behaves as a numpy array, with 
the buffer owned by the mxArray; but I don't know how to do that 
(particularly, how to handle the destruction of data, as the proxy 
should destroy the mxArray once the proxy  object is garbage collected). 
1 would be easy if the C matlab API was sane, which is not the case; 
they give functions which are impossible to use correctly (mxSetPr and 
mxSetData).
>
> What could be possible to do is add an array interface to the mlabwrap 
> proxy classes so they can be used as numpy arrays when required for 
> passing to numpy functions (or PIL, etc). Thus we only copy when we 
> want to use numpy functions. Then we could define the operators on the 
> proxy class to perform their operations on the other side of the bridge.
Yes, that's what I want to do, and in theory, this should be possible 
without copy; my initial question in the beginning of the thread is how 
to build a numpy proxy class from existing buffer of data, with the 
proxy becoming the owner of the data (ie should do all the deallocation, 
including here cleaning mxArray structures).

cheers,

David

-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From peridot.faceted at gmail.com  Thu Nov  9 02:27:38 2006
From: peridot.faceted at gmail.com (A. M. Archibald)
Date: Thu, 9 Nov 2006 02:27:38 -0500
Subject: A reimplementation of MaskedArray
In-Reply-To: <45522EF2.8000603@ieee.org>
References: <200610160232.20888.pgmdevlist@mailcan.com>
	<200610241408.40415.pgmdevlist@gmail.com>
	<16761e100611071711q6b9cc464p85708d7a642b9a8a@mail.gmail.com>
	<200611080510.13565.pgmdevlist@gmail.com>
	<ce557a360611081041t6748bcdj953aa6aaea9cab18@mail.gmail.com>
	<45522EF2.8000603@ieee.org>
Message-ID: <ce557a360611082327i6560c75dy18bcb2bdc03833f2@mail.gmail.com>

On 08/11/06, Tim Hochberg <tim.hochberg at ieee.org> wrote:

> It has always been my experience (on various flavors or Pentium) that
> operating on NANs is extremely slow. Does anyone know on what hardware
> NANs are *not* slow? Of course it's always possible I just never notice
> NANs on hardware where they aren't slow.

On an opteron machine I have access to, they appear to be no slower
(and even faster for some transcendental functions) than ordinary
floats:

In [13]: a=zeros(1000000)

In [14]: %time for i in xrange(1000): a += 1.1
CPU times: user 6.87 s, sys: 0.00 s, total: 6.87 s
Wall time: 6.87

In [15]: a *= NaN

In [16]: %time for i in xrange(1000): a += 1.1
CPU times: user 6.86 s, sys: 0.00 s, total: 6.86 s
Wall time: 6.87

On my Pentium M, they are indeed significantly slower (three times? I
didn't really do enough testing to say how much). I am actually rather
offended by this unfair discrimination against a citizen in good
standing of the IEEE floating point community.

A. M. Archibald

-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From a.u.r.e.l.i.a.n at gmx.net  Thu Nov  9 03:00:48 2006
From: a.u.r.e.l.i.a.n at gmx.net (Johannes Loehnert)
Date: Thu, 9 Nov 2006 09:00:48 +0100
Subject: matrix multiplication (newbie question)
In-Reply-To: <45525363.1030404@gmx.net>
References: <20061108135417.6906.qmail@web50913.mail.yahoo.com>
	<200611081716.18422.a.u.r.e.l.i.a.n@gmx.net>
	<45525363.1030404@gmx.net>
Message-ID: <200611090900.48824.a.u.r.e.l.i.a.n@gmx.net>


> Well what about A*(B*C)?

Point taken...

Johannes

-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From cjw at sympatico.ca  Thu Nov  9 03:18:57 2006
From: cjw at sympatico.ca (Colin J. Williams)
Date: Thu, 09 Nov 2006 03:18:57 -0500
Subject: Grief with a complex value
Message-ID: <4552E471.2020500@sympatico.ca>


 >>> import numpy.core as _n
 >>> _nt= _n.numerictypes
 >>> value= 
'\x00\x00\x00\x00\x00\x00\xf0?\x00\x00\x00\x00\x00\x00\x00@\x00\x00\x00\x00\x00\x00\x08@\x00\x00\x00\x00\x00\x00\x10@\x00\x00\x00\x00\x00\x00\x14@\x00\x00\x00\x00\x00\x00\x18@'
 >>> _n.array(value, dtype= _nt.complex128, copy=True)
Traceback (most recent call last):
  File "<interactive input>", line 1, in <module>
TypeError: a float is required
 >>>

A bug?

Colin W.

-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From robert.kern at gmail.com  Thu Nov  9 03:34:44 2006
From: robert.kern at gmail.com (Robert Kern)
Date: Thu, 09 Nov 2006 02:34:44 -0600
Subject: Grief with a complex value
In-Reply-To: <4552E471.2020500@sympatico.ca>
References: <4552E471.2020500@sympatico.ca>
Message-ID: <eiup7d$c6k$1@sea.gmane.org>

Colin J. Williams wrote:
>  >>> import numpy.core as _n
>  >>> _nt= _n.numerictypes
>  >>> value= 
> '\x00\x00\x00\x00\x00\x00\xf0?\x00\x00\x00\x00\x00\x00\x00@\x00\x00\x00\x00\x00\x00\x08@\x00\x00\x00\x00\x00\x00\x10@\x00\x00\x00\x00\x00\x00\x14@\x00\x00\x00\x00\x00\x00\x18@'
>  >>> _n.array(value, dtype= _nt.complex128, copy=True)
> Traceback (most recent call last):
>   File "<interactive input>", line 1, in <module>
> TypeError: a float is required
>  >>>
> 
> A bug?

No. array() takes sequences, not data buffers. You want frombuffer().

-- 
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless enigma
 that is made terrible by our own mad attempt to interpret it as though it had
 an underlying truth."
  -- Umberto Eco


-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From fperez.net at gmail.com  Thu Nov  9 03:40:33 2006
From: fperez.net at gmail.com (Fernando Perez)
Date: Thu, 9 Nov 2006 01:40:33 -0700
Subject: Int32s, scalar operations and Pyhon longs
Message-ID: <db6b5ecc0611090040m570b634fl312b21222b4fe108@mail.gmail.com>

I understand why this happens, but I wonder if it should be in any way
'fixed' (if that is even feasible without introducing other problems):

In [28]: x = 999999

In [29]: y = numpy.array([x])

In [30]: z = y[0]

In [31]: x==z
Out[31]: True

In [32]: x
Out[32]: 999999

In [33]: z
Out[33]: 999999

In [34]: x*x
Out[34]: 999998000001L

In [35]: z*z
Warning: overflow encountered in long_scalars
Out[35]: -729379967

I am sure it will be, to say the least, pretty surprising (and I can
imagine subtle bugs being caused by this).

Regards,

f

-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From cjw at sympatico.ca  Thu Nov  9 03:54:05 2006
From: cjw at sympatico.ca (Colin J. Williams)
Date: Thu, 09 Nov 2006 03:54:05 -0500
Subject: Grief with a complex value
In-Reply-To: <eiup7d$c6k$1@sea.gmane.org>
References: <4552E471.2020500@sympatico.ca> <eiup7d$c6k$1@sea.gmane.org>
Message-ID: <4552ECAD.6090005@sympatico.ca>



Robert Kern wrote:
> Colin J. Williams wrote:
>   
>>  >>> import numpy.core as _n
>>  >>> _nt= _n.numerictypes
>>  >>> value= 
>> '\x00\x00\x00\x00\x00\x00\xf0?\x00\x00\x00\x00\x00\x00\x00@\x00\x00\x00\x00\x00\x00\x08@\x00\x00\x00\x00\x00\x00\x10@\x00\x00\x00\x00\x00\x00\x14@\x00\x00\x00\x00\x00\x00\x18@'
>>  >>> _n.array(value, dtype= _nt.complex128, copy=True)
>> Traceback (most recent call last):
>>   File "<interactive input>", line 1, in <module>
>> TypeError: a float is required
>>  >>>
>>
>> A bug?
>>     
>
> No. array() takes sequences, not data buffers. You want frombuffer().
>   
Thanks.

Colin W.

-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From stefan at sun.ac.za  Thu Nov  9 03:56:38 2006
From: stefan at sun.ac.za (Stefan van der Walt)
Date: Thu, 9 Nov 2006 10:56:38 +0200
Subject: Grief with a complex value
In-Reply-To: <4552E471.2020500@sympatico.ca>
References: <4552E471.2020500@sympatico.ca>
Message-ID: <20061109085638.GA25499@mentat.za.net>

On Thu, Nov 09, 2006 at 03:18:57AM -0500, Colin J. Williams wrote:
> 
>  >>> import numpy.core as _n
>  >>> _nt= _n.numerictypes
>  >>> value= 
> '\x00\x00\x00\x00\x00\x00\xf0?\x00\x00\x00\x00\x00\x00\x00@\x00\x00\x00\x00\x00\x00\x08@\x00\x00\x00\x00\x00\x00\x10@\x00\x00\x00\x00\x00\x00\x14@\x00\x00\x00\x00\x00\x00\x18@'
>  >>> _n.array(value, dtype= _nt.complex128, copy=True)
> Traceback (most recent call last):
>   File "<interactive input>", line 1, in <module>
> TypeError: a float is required
>  >>>

If I understand correctly, you would like to create an array from a
buffer:

In [12]: N.frombuffer(value,dtype=N.complex128)
Out[12]: array([ 1.+2.j,  3.+4.j,  5.+6.j])

Cheers
St?fan

-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From Mailer-Daemon at lists.sourceforge.net  Thu Nov  9 04:29:15 2006
From: Mailer-Daemon at lists.sourceforge.net (Mail Delivery System)
Date: Thu, 09 Nov 2006 01:29:15 -0800
Subject: Mail delivery failed: returning message to sender
Message-ID: <E1Gi6Dj-0000On-GV@externalmx-1.sourceforge.net>

This message was created automatically by mail delivery software.

A message that you sent could not be delivered to one or more of its
recipients. This is a permanent error. The following address(es) failed:

  numpy-discussion at lists.sourceforge.net
    SMTP error from remote mailer after RCPT TO:<numpy-discussion at lists.sourceforge.net>:
    host mail.sourceforge.net [66.35.250.206]: 451-Could not complete sender verify callout
    451-Could not complete sender verify callout for
    451-<numpy-discussion at lists.sourceforge.net>.
    451-The mail server(s) for the domain may be temporarily unreachable, or
    451-they may be permanently unreachable from this server. In the latter case,
    451-you need to change the address or create an MX record for its domain
    451-if it is : retry timeout exceeded

------ This is a copy of the message, including all the headers. ------
------ The body of the message is 56616 characters long; only the first
------ 16384 or so are included here.


From xswbrx at kpslm.co.uk  Sun Nov  5 02:01:50 2006
From: xswbrx at kpslm.co.uk (Jay Porter)
Date: Sun, 05 Nov 2006 16:01:50 +0900
Subject: faces and their former position.  To be, sure.  But question
Message-ID: <000901c700a8$3bc01be0$01bfaa58@tldaduattuwq>

An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20061105/d4cb1742/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: eodabc.gif
Type: image/gif
Size: 11480 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20061105/d4cb1742/attachment-0001.gif>

From johanna at chez.com  Thu Nov  9 09:07:08 2006
From: johanna at chez.com (=?windows-1251?B?0/fo7CDP6+Dt6PDu4uDy/A==?=)
Date: Thu, 09 Nov 2006 17:07:08 +0300
Subject: =?windows-1251?b?0/fl4e376SDq8/DxOqvB6Oft5fEg?=
 =?windows-1251?b?7+vg7Swg0t3OLCCWIPDg5/Dg4e7y6uAg6CDg7eDr6Oe7Lg==?=
Message-ID: <de7b01c70408$32a3a4fa$636693e0@chez.com>

  ???????????? ???????. ?????????? ?????? ? ?????, ???.

 

 

???? ????????!!! ????? ??????:

??? ?????????? ? ???????????? ?????, ??? 2 ??? ? ???? ??????????, ?????
????????????? (?????????? ? ?????????? ????????????), ??? ??? ??? ?? ?????????
?????????? ??????????? ????????????? ????????, ????????? ?????? ? ??? ???????????
??? ?????????? ???????,  ?????????? ?? ?? ?????????? ?????????? ??????????????????
??????????? ??????????. ??? ???????? ????, ? ?????? ????. ?? ?????????
????? ??? ???????? ????????????, ???????????? ?????????????????? ???????
?????????, ??? ?????????? ?????? ?????? ? ?????????????? ????????, ?? ??????????
?? ???????????, ????????????? ??????? ?? ???? ????????? ? ????????? ?????
???????? (??? ?????? ???????? ????) - ??? ???! ??????! ?? ?????? ??????????????!

 
??????? ?????????? ? ????????:

 

???? ?????????? ????????: ? I? ?? I4 ?????? 2006 ?.



 
????????? ??????????:

 

?????????? ?????????? ????????????? ???????? ????? ???????? ?? ???????????
????????? ? ?????????????? ?????????? (?????????????? ????) ?LOGICAL -
J 3000? ? ???????? ????????????????. 

?????????? ???????????? ????????? ? ???????????? ? ??????????? ???????????
??????????? ???????? ?????? ??????? ????????? ???????? ? ????? ??????????????
????????????, ???????? ???????? ???????????????? ?????? ? ?????????? ?????????????
??????, ??????? ??????? ???????? ???? ??????? ????????????, ???????????
????????? ???????????????? ???????????? ? ??????????? ?? ?????? ????????????
?????????.

 

??????? ????????? ????????: 

??? ?????????? ???????? ?????? ? ????????? ???????? ???????????? ? ????????
?????????? ????? ?? ??????????? ??????????????? ????????? ????????? ????????????
????????? ? ?????????:
 ?????????????????? ?????????- ???????????????? ?????????? ????????????!
??????????? ??????? ???????? ?? ?????? ????????????? ?????????? ????????????
?? ???????????????? ??????, ? ????? ?????? ??????????? ???????????? ???????????
??? ???????????? ???????? ??????? ???????????. ????? - ??????????? ??????
????? - ???????????? ????????????. ? ????????? ? ?????? ???????????? ?????
?????? ?????????????????? ????????? ?? ???????? ?????????? ?????? ??????. (????????
?????????? ????????????? ????????????. ???????? ?????????? ????????? -
???????, ????????????, ???????????????? ?????????? ????????????? ????????????
??? ????????????? ? ????????? ?????????, ??? ? ??????????, ??? ? ? ?????????
??????? ??????.). ???????????? ???????? ???????? (????????????? ????? ?????????)
??? ??????????????? ?????????? ??????????? ?????? ?????. ??????????? ??????
(?? CD) ????????????? ????????? ??? ??????????????? ?????????? ??????????
????????.  



??????? ???????????? ??:

????????????? ???????????, ????????????? ???????-????????????? ?????, ????????????
?? ?????????? ?????????, ?????????????, ??????????, ???????????? ?????????
??????????? ? ??????? ???????????????? ??????????. ????????????, ??????????????
?????????? ? ???????? ???????, ????????? ? ???????????? ? ?????????????
????????????? ???????????, ???????????????, ??????? ?????????, ???????????
?????????? ? ??????? ?????????? ????????????, ?????????? ?????????????
???????????? ???????????? ???????????.

 

? ?????????? ????????? ???????? ???????? ?????????? ????????? ?????? ?
??????: 
 ?????? ????????????? ? ????????? ??????????????? ???????, ??????-?????,
??? ????????? ?????????? ? ??????? ???????, ? ????? ???????? ??????????????
????????????? ?????? ????????????;  ??? ???????? ??????????? ???????????,
????????????? ? ?????????? ??????? ? ???????? ?? ??????????? ???????????;
??? ?????????? ?????????? ??????? ?? ???????? ?? ????????? ? ?????????????,
????????????? ???????????, ?? ???????????; ???????? ?????? ?????? ??, ???????????
?? ??????????? ???????????;  ??????????? ??????? ????????????? ????????
(??? ?????? ????????? ???????????);  ???????? ?????? ????????;  ????? ???????
???????? ? ???????? ? ????????????? ???????????;  ??? ????????? ???????????
???????????? (?????????) ????? ???????; ??? ????????? ??????????? ??????????
????? ????????????;  ??? ????????? ???????? ? ??????????? ????????;  ???
???????????? ????????? ??????????? ???????;   
 

? ???????????? ????????:
 ????????????? ? ?????????? ????????? ????????? ??????? ???????????;  
??? ????????? ??????????? ???????? ?? ????????? ???????????? ?????? ? ???????????
??????? ??????? ????????????;  ?????????? ??????????? ???????? ? ?????????????
??????????? ?? ??????????? (????? ???????); ????????? ?????????? ????????????
(?????????) ????? ??????????? ?????????;  ?????? ?????????? ??????????
? ?? ??????? ???????????; ????? ?????????? ??? ????????????? ????????????; 
????????? ?????????? ??? ????????????? ???????????? - ?????????? ???????
??????????, ???????, ??? ????? ?????????????? ???????????? ? ???????????
????????? ? ????????????? ???????????? ?????????? ? ????????? ?????? ?????,
??????????????? ???????;  ?????????? ?????? ??????????? ???????, ? ?????????
????????????. ?????????? ?????????? ????? ????????????:  ????? ? ??????????
? ????????;  ??? ?????????????? ??????????? ?????????????? ???????????
???????;  ????? ???????? ? ?? ???????? ? ?????????? ?????;  ????????????
?????????? ??????????? ? ?? ????????????? ? ?????? ?????????? ?????????
????????;  ?????? ???????? ???????? ??????: ??????????? ???????? ??????????
??? ?????? ??????? -  ??????????? ???????? ? ????????? ??????, ?? ????????????
?? ??????? ???:    ????? ??????????? ???????????  ???????? ?? ???????????
????????????.    ?????????? ?????? ?????????????? ??????.  ?????????? ???????????
????????? ??????????????: ??????? ????????, ??????????? ???????? ???????????. 
???????????? ?????????? ????????????? ??????? ?? ?????? ??????? (?????,
?????)  ??????????? ?????? ??????????: ???????? ??????????, ???????????????
?????????? ???????????? ??????????? (??????????????, ????????????) ? ??
????????????? ??? ????? ??????? ?????????? ???????????.  ???????? ???????????
?????? ?????? ? ???????? ? ???????, ???????????? ?????? ? ?????????? ?????????
+ ????????????? ??????.  ?????????? ? ?????????? ??????? ??????? ????????
? ???????? ?? ?????????? ?????????? ???????.  ???????????? ?????? ????????????????
?????????????? ?????? ? ?????????????? ??????? ? ???????????? ?????????
?????????????.  ??????? ???????? ?????? ?????? ????????? ????????? ? ?????
?????? ??????????? ?? ??????????? ????????? ???????????? ??? ?????? ? ??????????
?????????????.  


?? ???????? ??????????? ??????????? ?? ???:  (495) 235-58 88, 235-12-35


????




 
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20061109/c0ed5357/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: fsxbl.jpg
Type: image/jpeg
Size: 6148 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20061109/c0ed5357/attachment-0001.jpg>

From aisaac at american.edu  Thu Nov  9 11:43:22 2006
From: aisaac at american.edu (Alan G Isaac)
Date: Thu, 9 Nov 2006 11:43:22 -0500
Subject: slice assignment: strange behaviour
In-Reply-To: <1163025393.211885.161470@e3g2000cwe.googlegroups.com>
References: <1163025393.211885.161470@e3g2000cwe.googlegroups.com>
Message-ID: <Mahogany-0.67.0-660-20061109-114322.00@american.edu>

On Wed, 08 Nov 2006, koara apparently wrote: 
> 'mat' is a numpy.array with shape=(22973, 1009), 
> 'vec' is a numpy.array with shape=(22973,), both of type int: 
> for i in xrange(1009): 
>     ... 
>     fr = vec[10001] 
>     mat[:, i] = vec # assign whole column 
>     if mat[10001, i] != fr: 
>         print "how come?" 
>      ... 
> for elements beyond index 10000, nothing is assigned (ie, 
> numpy.sum(mat[row, :]) is zero for any row > 10000). 

Using numpy 1.0 and trying the code below,
I do not see the problem.

import numpy
from numpy import random
m = random.randint(0,1000,(22973,1009))
v =  random.randint(0,1000,(22973,))
fr = v[10001]
for i in xrange(1009):
    m[:,i] = v
    if m[10001, i] != fr:
        print m[10001,i], fr

Upgrading to numpy 1.0 should be easy enough and may fix 
your problem.

Cheers,
Alan Isaac





-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From robert.kern at gmail.com  Thu Nov  9 11:43:45 2006
From: robert.kern at gmail.com (Robert Kern)
Date: Thu, 09 Nov 2006 10:43:45 -0600
Subject: Int32s, scalar operations and Pyhon longs
In-Reply-To: <db6b5ecc0611090040m570b634fl312b21222b4fe108@mail.gmail.com>
References: <db6b5ecc0611090040m570b634fl312b21222b4fe108@mail.gmail.com>
Message-ID: <eivlsb$mco$1@sea.gmane.org>

Fernando Perez wrote:
> I understand why this happens, but I wonder if it should be in any way
> 'fixed' (if that is even feasible without introducing other problems):
> 
> In [28]: x = 999999
> 
> In [29]: y = numpy.array([x])
> 
> In [30]: z = y[0]
> 
> In [31]: x==z
> Out[31]: True
> 
> In [32]: x
> Out[32]: 999999
> 
> In [33]: z
> Out[33]: 999999
> 
> In [34]: x*x
> Out[34]: 999998000001L
> 
> In [35]: z*z
> Warning: overflow encountered in long_scalars
> Out[35]: -729379967
> 
> I am sure it will be, to say the least, pretty surprising (and I can
> imagine subtle bugs being caused by this).

I think we decided long ago that an int32 really is an array of 32-bit integers
and behaves like one. That's precisely why one uses int32 arrays rather than
object arrays. There are algorithms that do need the wraparound, and the Python
int behavior is always available through object arrays.

-- 
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless enigma
 that is made terrible by our own mad attempt to interpret it as though it had
 an underlying truth."
  -- Umberto Eco


-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From fperez.net at gmail.com  Thu Nov  9 12:07:05 2006
From: fperez.net at gmail.com (Fernando Perez)
Date: Thu, 9 Nov 2006 10:07:05 -0700
Subject: Int32s, scalar operations and Pyhon longs
In-Reply-To: <eivlsb$mco$1@sea.gmane.org>
References: <db6b5ecc0611090040m570b634fl312b21222b4fe108@mail.gmail.com>
	<eivlsb$mco$1@sea.gmane.org>
Message-ID: <db6b5ecc0611090907hfd63b41m6f94201aff4591ad@mail.gmail.com>

On 11/9/06, Robert Kern <robert.kern at gmail.com> wrote:
> Fernando Perez wrote:
> > I understand why this happens, but I wonder if it should be in any way
> > 'fixed' (if that is even feasible without introducing other problems):

[...]

> > I am sure it will be, to say the least, pretty surprising (and I can
> > imagine subtle bugs being caused by this).
>
> I think we decided long ago that an int32 really is an array of 32-bit integers
> and behaves like one. That's precisely why one uses int32 arrays rather than
> object arrays. There are algorithms that do need the wraparound, and the Python
> int behavior is always available through object arrays.

Mkay, fine by me.  Novus caveat emptor.

Cheers,

f

-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From pauli.virtanen at iki.fi  Thu Nov  9 12:47:17 2006
From: pauli.virtanen at iki.fi (Pauli Virtanen)
Date: Thu, 09 Nov 2006 19:47:17 +0200
Subject: Anyone have a "little"
	shooting-method	function	to share
In-Reply-To: <45527171.8030903@noaa.gov>
References: <45527171.8030903@noaa.gov>
Message-ID: <1163094437.6376.6.camel@localhost.localdomain>

ke, 2006-11-08 kello 16:08 -0800, David L Goldsmith kirjoitti:
> Hi!  I tried to send this earlier: it made it into my sent mail folder, 
> but does not appear to have made it to the list.
> 
> I need to numerically solve:
>     (1-t)x" + x' - x = f(t), x(0) = x0, x(1) = x1
> I've been trying to use (because it's the approach I inherited) an 
> elementary finite-difference discretization, but unit tests have shown 
> that that approach isn't working.  
[clip]

You could try to use some of the pre-existing www.netlib.org codes for
solving BVPs. This only requires writing a wrapper.

However, I have written a wrapper for the COLNEW code, which is a
finite-difference method written by Ascher & Bader in '87, having an
adaptive mesh selection. You can find it here:

	http://www.iki.fi/pav/bvp

As I understand, COLNEW does not have any special handling for singular
coefficients, but nevertheless it seems to be able to solve your
problem. The code goes as follows:

----------------------------------------------------
import scipy as N
import bvp

u0 = 1
u1 = 2

def f(t):
    return N.sin(2*t)

def fsub(t, z):
    u, du = z
    return N.array([(f(t) + u - du)/(1-t)])

def gsub(z):
    u, du = z
    return N.array([u[0] - u0, u[1] - u1])

tol = [1e-5, 1e-5]
boundary_points = [0, 1]

solution = bvp.colnew.solve(boundary_points, [2], fsub, gsub,
    is_linear=True, tolerances=tol,
    vectorized=True, maximum_mesh_size=300)

import pylab

x = solution.mesh
pylab.plot(x, solution(x)[:,0])
pylab.savefig('solution.png')
----------------------------------------------------

BR,

	Pauli Virtanen

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 189 bytes
Desc: Digitaalisesti allekirjoitettu viestin osa
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20061109/4287a0a4/attachment-0001.sig>
-------------- next part --------------
-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
-------------- next part --------------
_______________________________________________
Numpy-discussion mailing list
Numpy-discussion at lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/numpy-discussion

From tim.hochberg at ieee.org  Thu Nov  9 12:11:37 2006
From: tim.hochberg at ieee.org (Tim Hochberg)
Date: Thu, 09 Nov 2006 10:11:37 -0700
Subject: A reimplementation of MaskedArray
In-Reply-To: <ce557a360611082327i6560c75dy18bcb2bdc03833f2@mail.gmail.com>
References: <200610160232.20888.pgmdevlist@mailcan.com>	<200610241408.40415.pgmdevlist@gmail.com>	<16761e100611071711q6b9cc464p85708d7a642b9a8a@mail.gmail.com>	<200611080510.13565.pgmdevlist@gmail.com>	<ce557a360611081041t6748bcdj953aa6aaea9cab18@mail.gmail.com>	<45522EF2.8000603@ieee.org>
	<ce557a360611082327i6560c75dy18bcb2bdc03833f2@mail.gmail.com>
Message-ID: <45536149.6040409@ieee.org>

A. M. Archibald wrote:
> On 08/11/06, Tim Hochberg <tim.hochberg at ieee.org> wrote:
>
>   
>> It has always been my experience (on various flavors or Pentium) that
>> operating on NANs is extremely slow. Does anyone know on what hardware
>> NANs are *not* slow? Of course it's always possible I just never notice
>> NANs on hardware where they aren't slow.
>>     
>
> On an opteron machine I have access to, they appear to be no slower
> (and even faster for some transcendental functions) than ordinary
> floats:
>
> In [13]: a=zeros(1000000)
>
> In [14]: %time for i in xrange(1000): a += 1.1
> CPU times: user 6.87 s, sys: 0.00 s, total: 6.87 s
> Wall time: 6.87
>
> In [15]: a *= NaN
>
> In [16]: %time for i in xrange(1000): a += 1.1
> CPU times: user 6.86 s, sys: 0.00 s, total: 6.86 s
> Wall time: 6.87
>
> On my Pentium M, they are indeed significantly slower (three times? I
> didn't really do enough testing to say how much). I am actually rather
> offended by this unfair discrimination against a citizen in good
> standing of the IEEE floating point community.
>   
If they're only 3x slower you're doing better than I am. On my core duo 
box they appear to be nearly 20x slower for both addition and 
multiplication. This more or less matches what I recall from earlier boxes.

     >>> Timer("a.prod()", "import numpy as np; a = np.ones(4096); a[0]
    = np.nan").timeit(10000)
    5.6495738983585397
     >>> Timer("a.prod()", "import numpy as np; a =
    np.ones(4096)").timeit(10000)
    0.34439041833525152
     >>> Timer("a.sum()", "import numpy as np; a = np.ones(4096); a[0] =
    np.nan").timeit(10000)
    6.0985655998179027
     >>> Timer("a.sum()", "import numpy as np; a =
    np.ones(4096)").timeit(10000)
    0.32354363473564263

I've been told that operations on NANs are slow because they aren't 
always implemented in the FPU hardware. Instead they are trapped and 
implemented software or firmware or something or other. That may well be 
bogus 42nd hand information though.

-tim


-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From bpederse at gmail.com  Thu Nov  9 14:01:59 2006
From: bpederse at gmail.com (Brent Pedersen)
Date: Thu, 9 Nov 2006 11:01:59 -0800
Subject: filter a recarray
Message-ID: <e183a99d0611091101q5e611202tb41a94427778ab3c@mail.gmail.com>

hi, i have a recarray of  > 60K  records and i'm wondering if there's a
numpy/vectorized way to the following.

get a new array where there will be unique column0 + column1 rows with the
row that remains being chosen because it has the highest value in the last
column. so in the paste below, there are 4
'AT1G01070', 'AT1G11450' rows, but i only want to keep the row:
('AT1G01070', 'AT1G11450', 78.660003662109375, 717, 140, 5, 1129L, 1838L,
1098L, 1808L, 1e-168, 592.0)

because 592.0 is the highest value.  i can do this using a hash and looping,
but i'm guessing there's a more efficient way. any tips?
thanks.
-brent


[  ('AT1G01030', 'AT1G13260', 70.339996337890625, 263, 75, 1, 835L, 1094L,
778L, 1040L, 5.9999999999999998e-38, 158.0)
 ('AT1G01040', 'AT1G01040', 100.0, 8019, 0, 0, 1L, 8019L, 1L, 8019L, 0.0,
12620.0)
 ('AT1G01050', 'AT1G01050', 100.0, 1968, 0, 0, 1L, 1968L, 1L, 1968L, 0.0,
3090.0)
 ('AT1G01060', 'AT1G01060', 100.0, 4175, 0, 0, 1L, 4175L, 1L, 4175L, 0.0,
6355.0)
 ('AT1G01070', 'AT1G01070', 100.0, 2192, 0, 0, 1L, 2192L, 1L, 2192L, 0.0,
3426.0)
 ('AT1G01070', 'AT1G11450', 78.660003662109375, 717, 140, 5, 1129L, 1838L,
1098L, 1808L, 1e-168, 592.0)
 ('AT1G01070', 'AT1G11450', 79.870002746582031, 303, 51, 2, 1L, 293L, 1L,
303L, 1.9999999999999999e-67 , 256.0)
 ('AT1G01070', 'AT1G11450', 88.400001525878906, 181, 21, 0, 587L, 767L,
724L, 904L, 6e-57, 221.0)
 ('AT1G01070', 'AT1G11450', 83.209999084472656, 131, 19, 1, 1875L, 2002L,
1878L, 2008L, 1.9999999999999999e-28 , 126.0)
 ('AT1G01070', 'AT1G11460', 82.919998168945312, 480, 77, 3, 1129L, 1607L,
1216L, 1691L, 6.9999999999999999e-132, 470.0)]
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20061109/5d10a4dc/attachment-0001.html>
-------------- next part --------------
-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
-------------- next part --------------
_______________________________________________
Numpy-discussion mailing list
Numpy-discussion at lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/numpy-discussion

From cjw at sympatico.ca  Thu Nov  9 14:01:15 2006
From: cjw at sympatico.ca (Colin J. Williams)
Date: Thu, 09 Nov 2006 14:01:15 -0500
Subject: Grief with a complex value
In-Reply-To: <20061109085638.GA25499@mentat.za.net>
References: <4552E471.2020500@sympatico.ca>
	<20061109085638.GA25499@mentat.za.net>
Message-ID: <45537AFB.3080709@sympatico.ca>



Stefan van der Walt wrote:
> On Thu, Nov 09, 2006 at 03:18:57AM -0500, Colin J. Williams wrote:
>   
>>  >>> import numpy.core as _n
>>  >>> _nt= _n.numerictypes
>>  >>> value= 
>> '\x00\x00\x00\x00\x00\x00\xf0?\x00\x00\x00\x00\x00\x00\x00@\x00\x00\x00\x00\x00\x00\x08@\x00\x00\x00\x00\x00\x00\x10@\x00\x00\x00\x00\x00\x00\x14@\x00\x00\x00\x00\x00\x00\x18@'
>>  >>> _n.array(value, dtype= _nt.complex128, copy=True)
>> Traceback (most recent call last):
>>   File "<interactive input>", line 1, in <module>
>> TypeError: a float is required
>>  >>>
>>     
>
> If I understand correctly, you would like to create an array from a
> buffer:
>
> In [12]: N.frombuffer(value,dtype=N.complex128)
> Out[12]: array([ 1.+2.j,  3.+4.j,  5.+6.j])
>
> Cheers
> St?fan
>   
Thanks, I didn't find "frombuffer" in the Numpy Examples List. 

Incidentally, the List is a big help but it would be even better if it 
included the signatures of the functions.

Colin W.
> -------------------------------------------------------------------------
> Using Tomcat but need to do more? Need to support web services, security?
> Get stuff done quickly with pre-integrated technology to make your job easier
> Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
> http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
> _______________________________________________
> Numpy-discussion mailing list
> Numpy-discussion at lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/numpy-discussion
>
>   

-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From David.L.Goldsmith at noaa.gov  Thu Nov  9 16:02:33 2006
From: David.L.Goldsmith at noaa.gov (David L Goldsmith)
Date: Thu, 09 Nov 2006 13:02:33 -0800
Subject: Anyone have a
 "little"	shooting-method	function to share
In-Reply-To: <1163094437.6376.6.camel@localhost.localdomain>
References: <45527171.8030903@noaa.gov>
	<1163094437.6376.6.camel@localhost.localdomain>
Message-ID: <45539769.6030306@noaa.gov>

Wow, thanks!

DG

Pauli Virtanen wrote:
> ke, 2006-11-08 kello 16:08 -0800, David L Goldsmith kirjoitti:
>   
>> Hi!  I tried to send this earlier: it made it into my sent mail folder, 
>> but does not appear to have made it to the list.
>>
>> I need to numerically solve:
>>     (1-t)x" + x' - x = f(t), x(0) = x0, x(1) = x1
>> I've been trying to use (because it's the approach I inherited) an 
>> elementary finite-difference discretization, but unit tests have shown 
>> that that approach isn't working.  
>>     
> [clip]
>
> You could try to use some of the pre-existing www.netlib.org codes for
> solving BVPs. This only requires writing a wrapper.
>
> However, I have written a wrapper for the COLNEW code, which is a
> finite-difference method written by Ascher & Bader in '87, having an
> adaptive mesh selection. You can find it here:
>
> 	http://www.iki.fi/pav/bvp
>
> As I understand, COLNEW does not have any special handling for singular
> coefficients, but nevertheless it seems to be able to solve your
> problem. The code goes as follows:
>
> ----------------------------------------------------
> import scipy as N
> import bvp
>
> u0 = 1
> u1 = 2
>
> def f(t):
>     return N.sin(2*t)
>
> def fsub(t, z):
>     u, du = z
>     return N.array([(f(t) + u - du)/(1-t)])
>
> def gsub(z):
>     u, du = z
>     return N.array([u[0] - u0, u[1] - u1])
>
> tol = [1e-5, 1e-5]
> boundary_points = [0, 1]
>
> solution = bvp.colnew.solve(boundary_points, [2], fsub, gsub,
>     is_linear=True, tolerances=tol,
>     vectorized=True, maximum_mesh_size=300)
>
> import pylab
>
> x = solution.mesh
> pylab.plot(x, solution(x)[:,0])
> pylab.savefig('solution.png')
> ----------------------------------------------------
>
> BR,
>
> 	Pauli Virtanen
>
>   
> ------------------------------------------------------------------------
>
> -------------------------------------------------------------------------
> Using Tomcat but need to do more? Need to support web services, security?
> Get stuff done quickly with pre-integrated technology to make your job easier
> Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
> http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
> ------------------------------------------------------------------------
>
> _______________________________________________
> Numpy-discussion mailing list
> Numpy-discussion at lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/numpy-discussion
>   


-- 
HMRD/ORR/NOS/NOAA <http://response.restoration.noaa.gov/emergencyresponse/>

-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From Chris.Barker at noaa.gov  Thu Nov  9 13:34:35 2006
From: Chris.Barker at noaa.gov (Christopher Barker)
Date: Thu, 09 Nov 2006 10:34:35 -0800
Subject: Int32s, scalar operations and Pyhon longs
In-Reply-To: <eivlsb$mco$1@sea.gmane.org>
References: <db6b5ecc0611090040m570b634fl312b21222b4fe108@mail.gmail.com>
	<eivlsb$mco$1@sea.gmane.org>
Message-ID: <455374BB.8070000@noaa.gov>

Robert Kern wrote:

> I think we decided long ago that an int32 really is an array of 32-bit integers
> and behaves like one.

That would apply to y*y:

 >>> x = 999999
 >>> y = numpy.array([x])
 >>> x*x
999998000001L

So Python ints automatically convert to Python longs on overflow.

 >>> y*y
array([-729379967])

numpy int32 arrays wrap...

 >>> z = y[0]
 >>> type(z)
<type 'numpy.int32'>
 >>> z*z
Warning: overflow encountered in long_scalars
-729379967

int32 scalars wrap, but give a warning -- (why the warning here, but not 
with the array calculation?)

I'm a bit confused, because I thought that when you extracted a scalar 
from an array, you got regular python scalar for the datatypes that are 
supported. This made it clear that you always get a numpy Scalar, which, 
in a few situations, behaves differently than a seemingly equivalent 
Python scalar.

Have I got that right?

-Chris

-- 
Christopher Barker, Ph.D.
Oceanographer
                                     		
NOAA/OR&R/HAZMAT         (206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115       (206) 526-6317   main reception

Chris.Barker at noaa.gov

-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From tim.hochberg at ieee.org  Thu Nov  9 12:00:57 2006
From: tim.hochberg at ieee.org (Tim Hochberg)
Date: Thu, 09 Nov 2006 10:00:57 -0700
Subject: Int32s, scalar operations and Pyhon longs
In-Reply-To: <eivlsb$mco$1@sea.gmane.org>
References: <db6b5ecc0611090040m570b634fl312b21222b4fe108@mail.gmail.com>
	<eivlsb$mco$1@sea.gmane.org>
Message-ID: <45535EC9.80901@ieee.org>

Robert Kern wrote:
> Fernando Perez wrote:
>   
>> I understand why this happens, but I wonder if it should be in any way
>> 'fixed' (if that is even feasible without introducing other problems):
>>
>> In [28]: x = 999999
>>
>> In [29]: y = numpy.array([x])
>>
>> In [30]: z = y[0]
>>
>> In [31]: x==z
>> Out[31]: True
>>
>> In [32]: x
>> Out[32]: 999999
>>
>> In [33]: z
>> Out[33]: 999999
>>
>> In [34]: x*x
>> Out[34]: 999998000001L
>>
>> In [35]: z*z
>> Warning: overflow encountered in long_scalars
>> Out[35]: -729379967
>>
>> I am sure it will be, to say the least, pretty surprising (and I can
>> imagine subtle bugs being caused by this).
>>     
>
> I think we decided long ago that an int32 really is an array of 32-bit integers
> and behaves like one. That's precisely why one uses int32 arrays rather than
> object arrays. There are algorithms that do need the wraparound, and the Python
> int behavior is always available through object arrays.
>
>   
Let me add that I can't imagine that the bugs will be all that subtle 
given that numpy now spits out a warning on overflow.
If you're really worried about this I suggest you crank up the error 
mode to make this an error - then you really won't be able to miss this 
kind of overflow.

-tim




-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From efiring at hawaii.edu  Thu Nov  9 14:27:22 2006
From: efiring at hawaii.edu (Eric Firing)
Date: Thu, 09 Nov 2006 09:27:22 -1000
Subject: A reimplementation of MaskedArray
In-Reply-To: <45536149.6040409@ieee.org>
References: <200610160232.20888.pgmdevlist@mailcan.com>
	<200610241408.40415.pgmdevlist@gmail.com>
	<16761e100611071711q6b9cc464p85708d7a642b9a8a@mail.gmail.com>
	<200611080510.13565.pgmdevlist@gmail.com>
	<ce557a360611081041t6748bcdj953aa6aaea9cab18@mail.gmail.com>
	<45522EF2.8000603@ieee.org>
	<ce557a360611082327i6560c75dy18bcb2bdc03833f2@mail.gmail.com>
	<45536149.6040409@ieee.org>
Message-ID: <4553811A.4030606@hawaii.edu>

It looks like on my Pentium M multiplication with NaNs is slow, but 
using a masked array ranges from slightly faster (with only one value 
masked) to twice as slow (with all values masked):


In [15]:Timer("a.prod()", "import numpy as np; aa = np.ones(4096); a = 
np.ma.masked_greater(aa,0)").timeit(10000)
Out[15]:9.4012830257415771

In [16]:Timer("a.prod()", "import numpy as np; a = np.ones(4096); a[0]= 
np.nan").timeit(10000)
Out[16]:5.5737850666046143

In [17]:Timer("a.prod()", "import numpy as np; a = 
np.ones(4096)").timeit(10000)
Out[17]:0.40796804428100586

In [18]:Timer("a.prod()", "import numpy as np; aa = np.ones(4096); aa[0] 
= 2; a = np.ma.masked_greater(aa,1)").timeit(10000)
Out[18]:4.1544749736785889

In [19]:Timer("a.prod()", "import numpy as np; a = np.ones(4096); a[:]= 
np.nan").timeit(10000)Out[19]:5.8589630126953125

For transcendentals, nans or masks don't make much difference, although 
masks are slightly faster than nans:

In [20]:Timer("np.sin(a)", "import numpy as np; a = np.ones(4096); a[:]= 
np.nan").timeit(10000)
Out[20]:4.5575671195983887

In [21]:Timer("np.ma.sin(a)", "import numpy as np; aa = np.ones(4096); a 
= np.ma.masked_greater(aa,0)").timeit(10000)
Out[21]:4.4125270843505859

In [22]:Timer("b=np.sin(a)", "import numpy as np; a = 
np.ones(4096)").timeit(10000)
Out[22]:3.5793929100036621


Eric



Tim Hochberg wrote:
> A. M. Archibald wrote:
>> On 08/11/06, Tim Hochberg <tim.hochberg at ieee.org> wrote:
>>
>>   
>>> It has always been my experience (on various flavors or Pentium) that
>>> operating on NANs is extremely slow. Does anyone know on what hardware
>>> NANs are *not* slow? Of course it's always possible I just never notice
>>> NANs on hardware where they aren't slow.
>>>     
>> On an opteron machine I have access to, they appear to be no slower
>> (and even faster for some transcendental functions) than ordinary
>> floats:
>>
>> In [13]: a=zeros(1000000)
>>
>> In [14]: %time for i in xrange(1000): a += 1.1
>> CPU times: user 6.87 s, sys: 0.00 s, total: 6.87 s
>> Wall time: 6.87
>>
>> In [15]: a *= NaN
>>
>> In [16]: %time for i in xrange(1000): a += 1.1
>> CPU times: user 6.86 s, sys: 0.00 s, total: 6.86 s
>> Wall time: 6.87
>>
>> On my Pentium M, they are indeed significantly slower (three times? I
>> didn't really do enough testing to say how much). I am actually rather
>> offended by this unfair discrimination against a citizen in good
>> standing of the IEEE floating point community.
>>   
> If they're only 3x slower you're doing better than I am. On my core duo 
> box they appear to be nearly 20x slower for both addition and 
> multiplication. This more or less matches what I recall from earlier boxes.
> 
>      >>> Timer("a.prod()", "import numpy as np; a = np.ones(4096); a[0]
>     = np.nan").timeit(10000)
>     5.6495738983585397
>      >>> Timer("a.prod()", "import numpy as np; a =
>     np.ones(4096)").timeit(10000)
>     0.34439041833525152
>      >>> Timer("a.sum()", "import numpy as np; a = np.ones(4096); a[0] =
>     np.nan").timeit(10000)
>     6.0985655998179027
>      >>> Timer("a.sum()", "import numpy as np; a =
>     np.ones(4096)").timeit(10000)
>     0.32354363473564263
> 
> I've been told that operations on NANs are slow because they aren't 
> always implemented in the FPU hardware. Instead they are trapped and 
> implemented software or firmware or something or other. That may well be 
> bogus 42nd hand information though.
> 
> -tim
> 
> 
> -------------------------------------------------------------------------
> Using Tomcat but need to do more? Need to support web services, security?
> Get stuff done quickly with pre-integrated technology to make your job easier
> Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
> http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
> _______________________________________________
> Numpy-discussion mailing list
> Numpy-discussion at lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/numpy-discussion


-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From Chris.Barker at noaa.gov  Thu Nov  9 13:53:19 2006
From: Chris.Barker at noaa.gov (Christopher Barker)
Date: Thu, 09 Nov 2006 10:53:19 -0800
Subject: A reimplementation of MaskedArray
In-Reply-To: <45536149.6040409@ieee.org>
References: <200610160232.20888.pgmdevlist@mailcan.com>
	<200610241408.40415.pgmdevlist@gmail.com>
	<16761e100611071711q6b9cc464p85708d7a642b9a8a@mail.gmail.com>
	<200611080510.13565.pgmdevlist@gmail.com>
	<ce557a360611081041t6748bcdj953aa6aaea9cab18@mail.gmail.com>
	<45522EF2.8000603@ieee.org>
	<ce557a360611082327i6560c75dy18bcb2bdc03833f2@mail.gmail.com>
	<45536149.6040409@ieee.org>
Message-ID: <4553791F.4010906@noaa.gov>

Tim Hochberg wrote:
> I've been told that operations on NANs are slow because they aren't 
> always implemented in the FPU hardware. Instead they are trapped and 
> implemented software or firmware or something or other. 

which still doesn't make sense -- doesn't ANY operation with a NaN 
return NaN? how hard could that be?

I share in the disappointment that NaN's are not first class citizens in 
common hardware....

-Chris


-- 
Christopher Barker, Ph.D.
Oceanographer
                                     		
NOAA/OR&R/HAZMAT         (206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115       (206) 526-6317   main reception

Chris.Barker at noaa.gov

-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From tim.hochberg at ieee.org  Thu Nov  9 17:13:56 2006
From: tim.hochberg at ieee.org (Tim Hochberg)
Date: Thu, 09 Nov 2006 15:13:56 -0700
Subject: A reimplementation of MaskedArray
In-Reply-To: <4553791F.4010906@noaa.gov>
References: <200610160232.20888.pgmdevlist@mailcan.com>	<200610241408.40415.pgmdevlist@gmail.com>	<16761e100611071711q6b9cc464p85708d7a642b9a8a@mail.gmail.com>	<200611080510.13565.pgmdevlist@gmail.com>	<ce557a360611081041t6748bcdj953aa6aaea9cab18@mail.gmail.com>	<45522EF2.8000603@ieee.org>	<ce557a360611082327i6560c75dy18bcb2bdc03833f2@mail.gmail.com>	<45536149.6040409@ieee.org>
	<4553791F.4010906@noaa.gov>
Message-ID: <4553A824.6070108@ieee.org>

Christopher Barker wrote:
> Tim Hochberg wrote:
>   
>> I've been told that operations on NANs are slow because they aren't 
>> always implemented in the FPU hardware. Instead they are trapped and 
>> implemented software or firmware or something or other. 
>>     
>
> which still doesn't make sense -- doesn't ANY operation with a NaN 
> return NaN? how hard could that be?
>   
Well, it's not free. You still have to recognize a particular bit 
pattern corresponding to QNaN and then copy the result in that case. All 
that probably burns more silicon for a case that most people don't care 
about. Do NaNs make Half Life run any faster? No I didn't think so.

I suspect that instead they just look for the case where the exponent is 
all 1s, in which case the result is either +-Inf, SNaN or QNaN and then 
drop into software to handle those infrequent cases.
> I share in the disappointment that NaN's are not first class citizens in 
> common hardware....
>   
It would be nice to see IEEE 754 supported more faithfully, but there 
seems to be more of a problem on the software side than on the hardware 
side in practice.

I still can't help feeling that using NaNs for mask values would be 
abuse even if it weren't slow and eventually it would reach around and 
bite you someplace tender.

-tim


-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From robert.kern at gmail.com  Thu Nov  9 17:14:47 2006
From: robert.kern at gmail.com (Robert Kern)
Date: Thu, 09 Nov 2006 16:14:47 -0600
Subject: Int32s, scalar operations and Pyhon longs
In-Reply-To: <455374BB.8070000@noaa.gov>
References: <db6b5ecc0611090040m570b634fl312b21222b4fe108@mail.gmail.com>	<eivlsb$mco$1@sea.gmane.org>
	<455374BB.8070000@noaa.gov>
Message-ID: <ej0990$vpm$1@sea.gmane.org>

Christopher Barker wrote:
> I'm a bit confused, because I thought that when you extracted a scalar 
> from an array, you got regular python scalar for the datatypes that are 
> supported. This made it clear that you always get a numpy Scalar, which, 
> in a few situations, behaves differently than a seemingly equivalent 
> Python scalar.

Part of the point of adding scalar types was to ensure that a[0], for example,
always exposes the array interface (.shape, .dtype, etc.) whether a.shape is
(10,) or (10, 10). This uniformity aids generic programming.

-- 
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless enigma
 that is made terrible by our own mad attempt to interpret it as though it had
 an underlying truth."
  -- Umberto Eco


-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From fperez.net at gmail.com  Thu Nov  9 17:34:05 2006
From: fperez.net at gmail.com (Fernando Perez)
Date: Thu, 9 Nov 2006 15:34:05 -0700
Subject: Int32s, scalar operations and Pyhon longs
In-Reply-To: <45535EC9.80901@ieee.org>
References: <db6b5ecc0611090040m570b634fl312b21222b4fe108@mail.gmail.com>
	<eivlsb$mco$1@sea.gmane.org> <45535EC9.80901@ieee.org>
Message-ID: <db6b5ecc0611091434p6772ec3dkd82352742cc80a97@mail.gmail.com>

On 11/9/06, Tim Hochberg <tim.hochberg at ieee.org> wrote:

> Let me add that I can't imagine that the bugs will be all that subtle
> given that numpy now spits out a warning on overflow.
> If you're really worried about this I suggest you crank up the error
> mode to make this an error - then you really won't be able to miss this
> kind of overflow.

Not all computations are run interactively with a human in front of
the console to actually see the warnings go by.  If they get buried in
a log file on a cluster, there is a possibility you might miss them.

Cheers,

f

-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From Chris.Barker at noaa.gov  Wed Nov  8 16:37:13 2006
From: Chris.Barker at noaa.gov (Christopher Barker)
Date: Wed, 08 Nov 2006 13:37:13 -0800
Subject: "Slicing" recarrays
In-Reply-To: <45513A71.3060603@ee.byu.edu>
References: <45512A9B.9030809@noaa.gov> <45513A71.3060603@ee.byu.edu>
Message-ID: <45524E09.6030806@noaa.gov>

Travis Oliphant wrote:

> How about
> 
> newdtype = N.dtype([('both','2i4'),('','b1')])
> res = data.view(newdtype)['both']

That works great! thanks,

-Chris

-- 
Christopher Barker, Ph.D.
Oceanographer
                                     		
NOAA/OR&R/HAZMAT         (206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115       (206) 526-6317   main reception

Chris.Barker at noaa.gov

-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From pfdubois at gmail.com  Thu Nov  9 18:00:18 2006
From: pfdubois at gmail.com (Paul Dubois)
Date: Thu, 9 Nov 2006 15:00:18 -0800
Subject: A reimplementation of MaskedArray
In-Reply-To: <4553791F.4010906@noaa.gov>
References: <200610160232.20888.pgmdevlist@mailcan.com>
	<200610241408.40415.pgmdevlist@gmail.com>
	<16761e100611071711q6b9cc464p85708d7a642b9a8a@mail.gmail.com>
	<200611080510.13565.pgmdevlist@gmail.com>
	<ce557a360611081041t6748bcdj953aa6aaea9cab18@mail.gmail.com>
	<45522EF2.8000603@ieee.org>
	<ce557a360611082327i6560c75dy18bcb2bdc03833f2@mail.gmail.com>
	<45536149.6040409@ieee.org> <4553791F.4010906@noaa.gov>
Message-ID: <f74a6c2f0611091500s10a22e65u700441529df7134a@mail.gmail.com>

Disappointed in NaN land?

Since the function of old retired persons is to tell youngsters stories
around the campfile:

A supercomputer hardware designer told me that when forced to add IEEE
arithmetic to his designs that it decreased performance substantially, maybe
25-30%; it wasn't that doing the operations took so much longer, it was that
the increased physical space needed for that circuitry pushed the memory
farther away. Doubtless this inspired doing some of it in software instead.

No standard for controlling the behaviors exists, either, so you can find
out the hard way that underflow-to-zero is being done in software by
default, and that you are doing a lot of it. Or that your code doesn't have
the same behavior on different platforms.

To my mind, all that was really accomplished was to convince said youngsters
that somehow this NaN stuff was the solution to some problem. In reality,
computing for 3 days and having it print out 1000 NaNs is not exactly
satisfying. I think this stuff was essentially a mistake in the sense that
it is a nice ivory-tower idea that costs more in practice than it is worth.
I do not think a properly thought-out and coded algorithm ought to do
anything that this stuff is supposed to 'help' with, and if it does do it,
the code should stop executing.

Anyway, if I thought it would do the job I wouldn't have written MA in the
first place.

Rant off. Nap. Grumble. Stupid kids. (:->
-- Paul
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20061109/79338e77/attachment-0001.html>
-------------- next part --------------
-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
-------------- next part --------------
_______________________________________________
Numpy-discussion mailing list
Numpy-discussion at lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/numpy-discussion

From koara at atlas.cz  Thu Nov  9 18:24:23 2006
From: koara at atlas.cz (koara)
Date: Thu, 09 Nov 2006 15:24:23 -0800
Subject: slice assignment: strange behaviour
In-Reply-To: <Mahogany-0.67.0-660-20061109-114322.00@american.edu>
References: <1163025393.211885.161470@e3g2000cwe.googlegroups.com>
	<Mahogany-0.67.0-660-20061109-114322.00@american.edu>
Message-ID: <1163114663.842976.124280@f16g2000cwb.googlegroups.com>

Ok, so i installed the newest numpy and scipy over enthought defaults.
The original problem indeed disappeared! Given the highly suspicous
magic threshold number of 10000, i suspect it was a bug in old numpy.
Makes me wonder if it's really been solved or just stopped manifesting
itself in my test data :>

Btw numpy.sum now behaves differently when axis is left unspecified: an
example listed in help(numpy.sum) doesn't give the result the help
claims it does, as numpy.sum([[0, 1], [0, 5]]) now gives 6. Someone
ought to either change the description or the code 8)

Cheers


-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From joris at ster.kuleuven.ac.be  Thu Nov  9 17:45:21 2006
From: joris at ster.kuleuven.ac.be (joris at ster.kuleuven.ac.be)
Date: Thu,  9 Nov 2006 23:45:21 +0100
Subject: Grief with a complex value
Message-ID: <1163112321.4553af8165fdb@webmail.ster.kuleuven.be>


   [CJ]: I didn't find "frombuffer" in the Numpy Examples List. 

I just added a frombuffer() example (the 197th example!) which
I extracted from your mail.

 
   [CJ]: Incidentally, the List is a big help but it would be even better if 
   [CJ]: included the signatures of the functions.
 
Good idea! There are some other things, though, that are slightly higher
on my priority list. First, the NumPy Example List could be more complete.
Secondly, the alphabetic hyperlink list of NumPy functions is getting so long 
that it's becoming difficult for new people to grasp what NumPy can do. 
So I would like to add a categorized overview of the NumPy functions.

Cheers,
Joris


Disclaimer: http://www.kuleuven.be/cwis/email_disclaimer.htm


-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From miki at yahoo.co.jp  Thu Nov  9 19:29:58 2006
From: miki at yahoo.co.jp (=?iso-2022-jp?B?bWlraQ==?=)
Date: Thu,  9 Nov 2006 16:29:58 -0800 (PST)
Subject: (no subject)
Message-ID: <E1GiKEc-0006he-4z@mail.sourceforge.net>

:?? INFORMATION  ?????????????????????????:
 ??????????????????????
 ????????????   http://love-match.bz/pc/04
:??????????????????????????????????:

*????*:.?. .?.:*????*:.?..?:*????*:.?..?:**????*


???????????????????????????????????
??? ???????????????????Love?Match?
?-----------------------------------------------------------------
???????????????????????
???????????????????????
???????????????????????
???????????????????????
???????????????????????
???????????????????????
?-----------------------------------------------------------------
????????????????http://love-match.bz/pc/04

???????????????????????????????????
??? ??????????????????????

?-----------------------------------------------------------------
????????????????????????????
?-----------------------------------------------------------------
?????????????????????????????
???????????????????????????????
?http://love-match.bz/pc/04

?-----------------------------------------------------------------
????????????????????????????????
?-----------------------------------------------------------------
????????????????????????????????
?????????????????????
?http://love-match.bz/pc/04
?-----------------------------------------------------------------
????????????????????
?-----------------------------------------------------------------
????????????????????????
??????????????????????????????????
?http://love-match.bz/pc/04

?-----------------------------------------------------------------
????????????????????????????
?-----------------------------------------------------------------
???????????????????????????
?????????????????????????????????
?http://love-match.bz/pc/04

?-----------------------------------------------------------------
?????????????????????????
?-----------------------------------------------------------------
?????????????????????????
?????????????????????????????????
?http://love-match.bz/pc/04


???????????????????????????????????
??? ??500????????????????

?-----------------------------------------------------------------

???????/????
????????????????????
?????????????????????????????????
????????????????????????????????
??????????????????????????
??????????????????????????????
?[????] http://love-match.bz/pc/04

?-----------------------------------------------------------------

???????/??????
??????????????????????????????????
???????????????????????????????????
??????????
?[????] http://love-match.bz/pc/04

?-----------------------------------------------------------------

???????/?????
??????????????????????????????????
????????????????????????????????
?????????????????????????(^^)
?[????] http://love-match.bz/pc/04

?-----------------------------------------------------------------

???????/????
???????????????????????????????
??????????????????????????????
??????????????????????????????
????????
?[????] http://love-match.bz/pc/04

?-----------------------------------------------------------------

????????/???
???????????????1???
?????????????????????????
?????????????????????????
?[????] http://love-match.bz/pc/04

?-----------------------------------------------------------------

???????/???????
????18??????????????????????????
?????????????????????????????
?????????????????????????????
?[????] http://love-match.bz/pc/04

?-----------------------------------------------------------------

???`????/???
?????????????????????
??????????????????????
??????????????
?[????] http://love-match.bz/pc/04

?-----------------------------------------------------------------


   ????????????????????
??????????????????????????????????
?????????????
??-------------------------------------------------------------
????????????????????????????????
??[??????????]?http://love-match.bz/pc/?04
??-------------------------------------------------------------
?????????????????????
???????????????????????????
???????????????????
???????????????????????????????
??[??????????]?http://love-match.bz/pc/04
??????????????????????????????????

??????????3-6-4-533
??????
139-3668-7892







-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From mkg at cs.nyu.edu  Thu Nov  9 20:49:02 2006
From: mkg at cs.nyu.edu (Matthew Koichi Grimes)
Date: Thu, 09 Nov 2006 20:49:02 -0500
Subject: automatic differentiation using custom ndarray?
Message-ID: <4553DA8E.7010109@cs.nyu.edu>

I'd love to be able to do automatic differentiation using numpy
ndarrays. There are some AD libraries in C++ that work through operator
overloading on custom numeric types (e.g. "adouble"). I have no
experience in in creating custom ndarrays, but would it be a huge
project (or even possible) to make a custom ndarray that used such
augmented numeric types? From reading Jonathan Wang's thread
(http://www.mail-archive.com/numpy-discussion at lists.sourceforge.net/msg03487.html), 

it seemed like ndarrays with custom numeric types are still a work in
progress.

I'm thinking specifically of the ADOL-C library, but if people with
experience with various AD libraries have a better recommendation, I'd
be all ears. I've also looked at CPPAD, but it doesn't seem to operate
nearly as transparently.

-- Matt


-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From Chris.Barker at noaa.gov  Wed Nov  8 12:27:46 2006
From: Chris.Barker at noaa.gov (Christopher Barker)
Date: Wed, 08 Nov 2006 09:27:46 -0800
Subject: Stacking arrays...
In-Reply-To: <6ef8f3380611071415k7e3c59d5pba7ff9338abec60f@mail.gmail.com>
References: <4550D6F8.80001@noaa.gov>
	<6ef8f3380611071415k7e3c59d5pba7ff9338abec60f@mail.gmail.com>
Message-ID: <45521392.3000204@noaa.gov>

Pau Gargallo wrote:
> c=N.dstack([a,b]) seems to do the trick.

indeed it does. Thanks.

-Chris

-- 
Christopher Barker, Ph.D.
Oceanographer
                                     		
NOAA/OR&R/HAZMAT         (206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115       (206) 526-6317   main reception

Chris.Barker at noaa.gov

-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From fperez.net at gmail.com  Fri Nov 10 00:51:09 2006
From: fperez.net at gmail.com (Fernando Perez)
Date: Thu, 9 Nov 2006 22:51:09 -0700
Subject: [OT] Why Python 'fits your brain'
Message-ID: <db6b5ecc0611092151q4489ec46kbc81652e24c27a7b@mail.gmail.com>

Please forgive the not-specifically-numpy post. I'll keep it short.

Some of us often, when trying to explain to newcomers the benefits of
Python for scientific work, use expressions like the famous 'it fits
your brain'.  This is an attempt at conveying why it seems like such a
natural tool for expressing, concisely and without extraneous noise,
but simultaneously with ease, the nature of many algorithmic problems.

I think these four images

http://flickr.com/photos/nicksieger/281055485/
http://flickr.com/photos/nicksieger/281055530/
http://flickr.com/photos/nicksieger/280662707/
http://flickr.com/photos/nicksieger/280661836/

may provide a useful visual aid.  I certainly intend to use them (with
attribution) the next time I need to give a talk along those lines.

Regards,

f

-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From wbaxter at gmail.com  Fri Nov 10 01:12:09 2006
From: wbaxter at gmail.com (Bill Baxter)
Date: Fri, 10 Nov 2006 15:12:09 +0900
Subject: [OT] Why Python 'fits your brain'
In-Reply-To: <db6b5ecc0611092151q4489ec46kbc81652e24c27a7b@mail.gmail.com>
References: <db6b5ecc0611092151q4489ec46kbc81652e24c27a7b@mail.gmail.com>
Message-ID: <e86a5fd00611092212n6ae681b0vbbec422cc2412f57@mail.gmail.com>

I think Ruby users say the same about Ruby, maybe even more
emphatically than Python users, and Ruby's chart looks like just about
the most complicated one there.  C and Python look to be about on par.
 Also I suspect a chart of Lisp's grammar would be even simpler than
any of those up there, but I doubt many scientists would say that Lisp
really fits their brain.

So I'm not really sure what conclusions you can draw from such charts.
The human brain is a complex network too.  Probably what matters more
is that the connections in those dependency charts somehow match the
connections in our brains.

--bb

On 11/10/06, Fernando Perez <fperez.net at gmail.com> wrote:
> Please forgive the not-specifically-numpy post. I'll keep it short.
>
> Some of us often, when trying to explain to newcomers the benefits of
> Python for scientific work, use expressions like the famous 'it fits
> your brain'.  This is an attempt at conveying why it seems like such a
> natural tool for expressing, concisely and without extraneous noise,
> but simultaneously with ease, the nature of many algorithmic problems.
>
> I think these four images
>
> http://flickr.com/photos/nicksieger/281055485/
> http://flickr.com/photos/nicksieger/281055530/
> http://flickr.com/photos/nicksieger/280662707/
> http://flickr.com/photos/nicksieger/280661836/
>
> may provide a useful visual aid.  I certainly intend to use them (with
> attribution) the next time I need to give a talk along those lines.
>
> Regards,
>

-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From peridot.faceted at gmail.com  Fri Nov 10 01:42:38 2006
From: peridot.faceted at gmail.com (A. M. Archibald)
Date: Fri, 10 Nov 2006 01:42:38 -0500
Subject: A reimplementation of MaskedArray
In-Reply-To: <f74a6c2f0611091500s10a22e65u700441529df7134a@mail.gmail.com>
References: <200610160232.20888.pgmdevlist@mailcan.com>
	<200610241408.40415.pgmdevlist@gmail.com>
	<16761e100611071711q6b9cc464p85708d7a642b9a8a@mail.gmail.com>
	<200611080510.13565.pgmdevlist@gmail.com>
	<ce557a360611081041t6748bcdj953aa6aaea9cab18@mail.gmail.com>
	<45522EF2.8000603@ieee.org>
	<ce557a360611082327i6560c75dy18bcb2bdc03833f2@mail.gmail.com>
	<45536149.6040409@ieee.org> <4553791F.4010906@noaa.gov>
	<f74a6c2f0611091500s10a22e65u700441529df7134a@mail.gmail.com>
Message-ID: <ce557a360611092242m6b81bcdale40956ba1eb38c0@mail.gmail.com>

On 09/11/06, Paul Dubois <pfdubois at gmail.com> wrote:

> Since the function of old retired persons is to tell youngsters stories
> around the campfile:

I'll pull up a log. But since I'm uppity, and not especially young, I
hope you won't mind if I heckle.

> A supercomputer hardware designer told me that when forced to add IEEE
> arithmetic to his designs that it decreased performance substantially, maybe
> 25-30%; it wasn't that doing the operations took so much longer, it was that
> the increased physical space needed for that circuitry pushed the memory
> farther away. Doubtless this inspired doing some of it in software instead.

The goal of IEEE floats is not to be faster but to make doing correct
numerical programming easier. (Numerical analysts can write robust
numerical code for almost any kind of float, but the vast majority of
numerical code is written by scientists who know the bare minimum or
less about numerical analysis, so a good system makes such code work
as well as possible.)

The other reason IEEE floats are good is, no disrespect to your
hardware designer friend intended, that they were designed with some
care. Reimplementations are liable to get some of the finicky details
wrong (round-to-even, denormalized numbers, what have you...). I found
Kahan's "How Java's Floating-point Hurts Everyone Everywhere" (
http://www.cs.berkeley.edu/~wkahan/JAVAhurt.pdf ) very educational
when it comes to "what can go wrong with platform-dependent
floating-point".

> No standard for controlling the behaviors exists, either, so you can find
> out the hard way that underflow-to-zero is being done in software by
> default, and that you are doing a lot of it. Or that your code doesn't have
> the same behavior on different platforms.

Well, if the platform is IEEE-compliant, you can trust it to behave
the same way logicially, even if some applications are slower on some
systems than others. Or did you mean that there's no standard way to
set the various IEEE flags? That seems like a language/compiler issue,
to me (which is not to minimize the pain it causes!)

> To my mind, all that was really accomplished was to convince said youngsters
> that somehow this NaN stuff was the solution to some problem. In reality,
> computing for 3 days and having it print out 1000 NaNs is not exactly
> satisfying. I think this stuff was essentially a mistake in the sense that
> it is a nice ivory-tower idea that costs more in practice than it is worth.
> I do not think a properly thought-out and coded algorithm ought to do
> anything that this stuff is supposed to 'help' with, and if it does do it,
> the code should stop executing.

Well, then turn on floating-point exceptions. I find a few NaNs in my
calculations relatively benign. If I'm doing a calculation where
they'll propagate and destroy everything, I trap them. But it happens
just as often that I launch a job, a NaN appears early on, but I come
back in the morning to find that instead of aborting, the job has
completed except for a few bad data points.

If you want an algorithm that takes advantage of NaNs, I have one. I
was simulating light rays around a black hole, generating a map of the
bending for various view angles. So I naturally did a lot of exploring
of grazing incidence, where the calculations would often yield NaNs.
So I used the NaNs to fill in the gap between the internal bending of
light and the external bending of the light; no extra programming
effort required, since that was what came out of the ray tracer
anyway. The rendering step just returned NaNs for the image pixels
that came from interpolating in the NaN-filled regions, which came out
black; just what I wanted. I just wrote the program ignoring what
would happen if a NaN appeared, and it came out right. Which is, I
think, the goal - to save the programmer having to think too much
about numerical-analysis headaches.

> Anyway, if I thought it would do the job I wouldn't have written MA in the
> first place.

It's certainly not always a good solution; in particular it's hard to
control the behaviour of things like where(). There's also no NaN for
integer types, which makes MaskedArray more useful.

Anyway, I am pleased that numpy has both controllable IEEE floats and
MaskedArray, and I apologize for a basically off-topic post.

A. M. Archibald

-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From gruben at bigpond.net.au  Fri Nov 10 07:40:54 2006
From: gruben at bigpond.net.au (Gary Ruben)
Date: Fri, 10 Nov 2006 23:40:54 +1100
Subject: Grief with a complex value
In-Reply-To: <1163112321.4553af8165fdb@webmail.ster.kuleuven.be>
References: <1163112321.4553af8165fdb@webmail.ster.kuleuven.be>
Message-ID: <45547356.6040106@bigpond.net.au>

Does anyone think it might be a good idea to split up the examples list? 
It's getting very big now. I pity any poor modem users (I'm sure they 
still exist). However, I'm not sure that there's a good way to do it.

Gary R.

joris at ster.kuleuven.ac.be wrote:
>    [CJ]: I didn't find "frombuffer" in the Numpy Examples List. 
> 
> I just added a frombuffer() example (the 197th example!) which
> I extracted from your mail.
> 
>  
>    [CJ]: Incidentally, the List is a big help but it would be even better if 
>    [CJ]: included the signatures of the functions.
>  
> Good idea! There are some other things, though, that are slightly higher
> on my priority list. First, the NumPy Example List could be more complete.
> Secondly, the alphabetic hyperlink list of NumPy functions is getting so long 
> that it's becoming difficult for new people to grasp what NumPy can do. 
> So I would like to add a categorized overview of the NumPy functions.
> 
> Cheers,
> Joris

-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From joris at ster.kuleuven.be  Fri Nov 10 07:51:05 2006
From: joris at ster.kuleuven.be (Joris De Ridder)
Date: Fri, 10 Nov 2006 13:51:05 +0100
Subject: Grief with a complex value
In-Reply-To: <45547356.6040106@bigpond.net.au>
References: <1163112321.4553af8165fdb@webmail.ster.kuleuven.be>
	<45547356.6040106@bigpond.net.au>
Message-ID: <200611101351.05502.joris@ster.kuleuven.be>



On Friday 10 November 2006 13:40, Gary Ruben wrote:
   [GR]: Does anyone think it might be a good idea to split up the examples list? 
   [GR]: It's getting very big now. I pity any poor modem users (I'm sure they 
   [GR]: still exist). However, I'm not sure that there's a good way to do it.

They could download the Example List once and save it in a HTML file on their
local disk...

Joris

Disclaimer: http://www.kuleuven.be/cwis/email_disclaimer.htm


-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From steve at shrogers.com  Fri Nov 10 08:29:58 2006
From: steve at shrogers.com (Steven H. Rogers)
Date: Fri, 10 Nov 2006 06:29:58 -0700
Subject: [OT] Why Python 'fits your brain'
In-Reply-To: <e86a5fd00611092212n6ae681b0vbbec422cc2412f57@mail.gmail.com>
References: <db6b5ecc0611092151q4489ec46kbc81652e24c27a7b@mail.gmail.com>
	<e86a5fd00611092212n6ae681b0vbbec422cc2412f57@mail.gmail.com>
Message-ID: <45547ED6.1080004@shrogers.com>

Lisp has minimal syntax, as does APL, which does make them easy to learn if 
you have no preconceptions about what a programming language should be. 
They're rarely, if ever the first language people learn, so the strange 
appearance is a barrier that offsets the basic simplicity.  They both also 
have a large number of primitive functions, which adds to the cognitive 
load.  Of course, when you include necessary libraries, C and Java get 
pretty big too.

Maybe the types of connections are the deciding factor.  Hierarchical name 
spaces may help as well, so that it's more obvious that you only need to fit 
part of it into your brain as well.

Regards,
Steve
////////////////

Bill Baxter wrote:
> I think Ruby users say the same about Ruby, maybe even more
> emphatically than Python users, and Ruby's chart looks like just about
> the most complicated one there.  C and Python look to be about on par.
>  Also I suspect a chart of Lisp's grammar would be even simpler than
> any of those up there, but I doubt many scientists would say that Lisp
> really fits their brain.
> 
> So I'm not really sure what conclusions you can draw from such charts.
> The human brain is a complex network too.  Probably what matters more
> is that the connections in those dependency charts somehow match the
> connections in our brains.
> 
> --bb
> 
> On 11/10/06, Fernando Perez <fperez.net at gmail.com> wrote:
>> Please forgive the not-specifically-numpy post. I'll keep it short.
>>
>> Some of us often, when trying to explain to newcomers the benefits of
>> Python for scientific work, use expressions like the famous 'it fits
>> your brain'.  This is an attempt at conveying why it seems like such a
>> natural tool for expressing, concisely and without extraneous noise,
>> but simultaneously with ease, the nature of many algorithmic problems.
>>
>> I think these four images
>>
>> http://flickr.com/photos/nicksieger/281055485/
>> http://flickr.com/photos/nicksieger/281055530/
>> http://flickr.com/photos/nicksieger/280662707/
>> http://flickr.com/photos/nicksieger/280661836/
>>
>> may provide a useful visual aid.  I certainly intend to use them (with
>> attribution) the next time I need to give a talk along those lines.
>>
>> Regards,
>>
> 
> -------------------------------------------------------------------------
> Using Tomcat but need to do more? Need to support web services, security?
> Get stuff done quickly with pre-integrated technology to make your job easier
> Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
> http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
> _______________________________________________
> Numpy-discussion mailing list
> Numpy-discussion at lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/numpy-discussion
> 
> 

-- 
Steven H. Rogers, Ph.D., steve at shrogers.com
Weblog: http://shrogers.com/weblog
"He who refuses to do arithmetic is doomed to talk nonsense."
-- John McCarthy


-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From theller at ctypes.org  Fri Nov 10 09:32:13 2006
From: theller at ctypes.org (Thomas Heller)
Date: Fri, 10 Nov 2006 15:32:13 +0100
Subject: [OT] Why Python 'fits your brain'
In-Reply-To: <45547ED6.1080004@shrogers.com>
References: <db6b5ecc0611092151q4489ec46kbc81652e24c27a7b@mail.gmail.com>	<e86a5fd00611092212n6ae681b0vbbec422cc2412f57@mail.gmail.com>
	<45547ED6.1080004@shrogers.com>
Message-ID: <45548D6D.8030005@ctypes.org>

Steven H. Rogers schrieb:
> Lisp has minimal syntax, as does APL, which does make them easy to learn if 
> you have no preconceptions about what a programming language should be. 
> They're rarely, if ever the first language people learn, so the strange 
> appearance is a barrier that offsets the basic simplicity.

I really believe the world would be a somewhat better place if Lisp (or better yet, Scheme)
would be the first language people learned.  They need a better editor than notepad
for them, though.

Thomas


-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From pfdubois at gmail.com  Fri Nov 10 12:10:59 2006
From: pfdubois at gmail.com (Paul Dubois)
Date: Fri, 10 Nov 2006 09:10:59 -0800
Subject: [OT] was about masked array and IEEE arithmetic
Message-ID: <f74a6c2f0611100910y2cba22ftc526b3f76dd7ee4f@mail.gmail.com>

On 09 Nov 2006 22:50:44 -0800, A. M. Archibald <peridot.faceted at gmail.com>
wrote:
>
> On 09/11/06, Paul Dubois <pfdubois at gmail.com> wrote:
>
> > Since the function of old retired persons is to tell youngsters stories
> > around the campfile:
>
> I'll pull up a log. But since I'm uppity, and not especially young, I
> hope you won't mind if I heckle.
>
> > A supercomputer hardware designer told me that when forced to add IEEE
> > arithmetic to his designs that it decreased performance substantially,
> maybe
> > 25-30%; it wasn't that doing the operations took so much longer, it was
> that
> > the increased physical space needed for that circuitry pushed the memory
> > farther away. Doubtless this inspired doing some of it in software
> instead.
>
> The goal of IEEE floats is not to be faster but to make doing correct
> numerical programming easier. (Numerical analysts can write robust
> numerical code for almost any kind of float, but the vast majority of
> numerical code is written by scientists who know the bare minimum or
> less about numerical analysis, so a good system makes such code work
> as well as possible.)


This is an urban legend propagated to support the IEEE argument. I've worked
intimately with scientific programmers for 30 years and very few of them
have the characteristics you describe. It is their professional business to
know the basics of numerical analysis. They use packages written by the
numerical analysis community a lot, too.

The other reason IEEE floats are good is, no disrespect to your
> hardware designer friend intended, that they were designed with some
> care. Reimplementations are liable to get some of the finicky details
> wrong (round-to-even, denormalized numbers, what have you...). I found
> Kahan's "How Java's Floating-point Hurts Everyone Everywhere" (
> http://www.cs.berkeley.edu/~wkahan/JAVAhurt.pdf ) very educational
> when it comes to "what can go wrong with platform-dependent
> floating-point".



Well, Kahan at one point went around showing how using an early HP
calculator to navigate your plane was a bad idea. The pilots in the audience
were smirking -- he had an extremely long thin triangle in his example. They
aren't that dumb. I think these alleged difficulties don't exist in
practice.

> No standard for controlling the behaviors exists, either, so you can find
> > out the hard way that underflow-to-zero is being done in software by
> > default, and that you are doing a lot of it. Or that your code doesn't
> have
> > the same behavior on different platforms.
>
> Well, if the platform is IEEE-compliant, you can trust it to behave
> the same way logicially, even if some applications are slower on some
> systems than others. Or did you mean that there's no standard way to
> set the various IEEE flags? That seems like a language/compiler issue,
> to me (which is not to minimize the pain it causes!)


I meant both. It may seem like a language issue to you (and me) but it
doesn't to the language standard committees; just like they won't take
ownership of the command line to run the compiler no matter how much the
variances hurt people.

> To my mind, all that was really accomplished was to convince said
> youngsters
> > that somehow this NaN stuff was the solution to some problem. In
> reality,
> > computing for 3 days and having it print out 1000 NaNs is not exactly
> > satisfying. I think this stuff was essentially a mistake in the sense
> that
> > it is a nice ivory-tower idea that costs more in practice than it is
> worth.
> > I do not think a properly thought-out and coded algorithm ought to do
> > anything that this stuff is supposed to 'help' with, and if it does do
> it,
> > the code should stop executing.
>
> Well, then turn on floating-point exceptions. I find a few NaNs in my
> calculations relatively benign. If I'm doing a calculation where
> they'll propagate and destroy everything, I trap them. But it happens
> just as often that I launch a job, a NaN appears early on, but I come
> back in the morning to find that instead of aborting, the job has
> completed except for a few bad data points.


This sounds like "the software is done but we just haven't gotten it running
yet."

If you want an algorithm that takes advantage of NaNs, I have one. I
> was simulating light rays around a black hole, generating a map of the
> bending for various view angles. So I naturally did a lot of exploring
> of grazing incidence, where the calculations would often yield NaNs.
> So I used the NaNs to fill in the gap between the internal bending of
> light and the external bending of the light; no extra programming
> effort required, since that was what came out of the ray tracer
> anyway. The rendering step just returned NaNs for the image pixels
> that came from interpolating in the NaN-filled regions, which came out
> black; just what I wanted. I just wrote the program ignoring what
> would happen if a NaN appeared, and it came out right. Which is, I
> think, the goal - to save the programmer having to think too much
> about numerical-analysis headaches.


How do you know it came out right? (:->

Anyway, costing those who can think 30% of performance to save you a few
headaches is not worth it. That's what I meant, not that there were no cases
of benefit.

> Anyway, if I thought it would do the job I wouldn't have written MA in the
> > first place.
>
> It's certainly not always a good solution; in particular it's hard to
> control the behaviour of things like where(). There's also no NaN for
> integer types, which makes MaskedArray more useful.
>
> Anyway, I am pleased that numpy has both controllable IEEE floats and
> MaskedArray, and I apologize for a basically off-topic post.
>
> A. M. Archibald



I guess it was me who was off topic, so I'll be quiet now.

-------------------------------------------------------------------------
> Using Tomcat but need to do more? Need to support web services, security?
> Get stuff done quickly with pre-integrated technology to make your job
> easier
> Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
> http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
> _______________________________________________
> Numpy-discussion mailing list
> Numpy-discussion at lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/numpy-discussion
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20061110/05f9b7d0/attachment-0001.html>
-------------- next part --------------
-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
-------------- next part --------------
_______________________________________________
Numpy-discussion mailing list
Numpy-discussion at lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/numpy-discussion

From oubtevephs at originalwebdesign.com  Fri Nov 10 20:15:06 2006
From: oubtevephs at originalwebdesign.com (NETBALL)
Date: Fri, 10 Nov 2006 22:15:06 -0300
Subject: Faith Hill Vs Carrie
References: <000401c7052e$d2e86ab0$00000000@meche>
Message-ID: <000401c7052e$d2e86ab0$00000000@meche>


  ----- Original Message ----- 
  From: numpy-discussion at lists.sourceforge.net 
  To: oubtevephs at originalwebdesign.com 
  Sent: Saturday, November 01:13:02 PM
  Subject: Faith Hill Vs Carrie


  Front page forglobe Headlines.
  Woodhall gb sprintings low Relief sun or nations. Fog in dirty coke jackedup am living is couple havent formally.
  Gary Geneva Glen Ellyn! Secrets Fights Runways Vincent of Maroulis focus. Trip a Pakistan Shocked. Written produced edited Dehnartfor of reprinting content pithy quotes.
  Spam Address a comment. Crackin Americas in foremost popstar wasting fog dirty of coke. Pages or under Creative.
  Chicago Belmont Cragin Edgebrook Edison. Sun of nations incredible Police.
  Account of vacuum vulturous leeching off brought. Delicious Digg itrss is.
  Macintosh os a earlier Wear Newsletter javacom Technology Partner.
  Showdown psp am Blues Bash Pico North. Delicious Digg itrss is.
  How they best react. Fare enjoys is gardening add ownbuy ships hours Bodyguard a? Subscribe Contact recent stories begins.
  News Policies in Affiliate Program of Link Exchange Center in Feedback am. Niles Norridge Northbrook am Northlake Northwest of.
  Supporters impeccable pedigree needs drop pounds whatever! Main Menu raquo General.
  Crestwood Crown or Darien Deerfield in des Plaines or Downers!
  Woodhall gb sprintings low Relief sun or nations.
  Quiz countries am part cgc Council ni.
  Mora of Manfredo again Saturday Directors Guild in.
  Bet Platinum Soft Power Boston Globe.
  Mora of Manfredo again Saturday Directors Guild in.
  Files Divorce Federline Takes a cma a Awards!
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20061110/d41b0f51/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: West.gif
Type: image/gif
Size: 14283 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20061110/d41b0f51/attachment-0001.gif>
-------------- next part --------------
-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
-------------- next part --------------
_______________________________________________
Numpy-discussion mailing list
Numpy-discussion at lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/numpy-discussion

From daloveuw at ont.sysco.com  Fri Nov 10 19:56:10 2006
From: daloveuw at ont.sysco.com (devices across)
Date: Sat, 11 Nov 2006 08:56:10 +0800
Subject: down Combines advanced freeKVM
References: <000901c7052c$2d4decd0$00000000@HAIER>
Message-ID: <000901c7052c$2d4decd0$00000000@HAIER>


  ----- Original Message ----- 
  From: numpy-discussion at lists.sourceforge.net 
  To: daloveuw at ont.sysco.com 
  Sent: Sunday, November 06:47:05 AM
  Subject: down Combines advanced freeKVM


  Embedded Solutions Home.
  Digi Sites Deutsch Chinese Japanese Rabbit Maxstream Products Cellular in. Remotely onoff measure electrical a load.
  Ethernet dc ac is options a Securely it port Inband outofband. Tasks rj of dual Ethernet dc ac. Deutsch Chinese Japanese Rabbit of Maxstream. Try online demo am now Warranty Where buy direct from.
  Load more alarms for quotreal timequot of of? Common equipment monitoring am. Management Contact us Partner login or Digi Sites Deutsch.
  Reliable Automates common or equipment a monitoring tasks rj. Fe Console Servers a Secure and Easy.
  Measure electrical of load more alarms for quotreal. Enable in you am to access is monitor or manage in.
  Remote Power Manager Serial of Cards or Embedded Solutions.
  Their ports am even when is down Combines advanced. Enable in you am to access is monitor or manage in.
  Fe Console Servers Secure and Easy Management Contact. Embedded Solutions Home enable you is to in. Fe is Console Servers Secure and Easy Management Contact!
  Freekvm Ipmi is support a in one reliable? Both or over the.
  Automates common equipment monitoring tasks. Conditions Product am Literature Solution guide or Related?
  Remote Power Manager Serial a Cards Embedded.
  Digi Sites Deutsch Chinese Japanese Rabbit Maxstream Products Cellular in.
  Cards Embedded is Solutions a Home is enable you of to.
  And Easy Management Contact us Partner in login Digi.
  Access monitor manage network.
  And Easy Management Contact us Partner in login Digi.
  Demo is now of Warranty Where buy direct from.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20061111/67a17164/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: more.gif
Type: image/gif
Size: 14076 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20061111/67a17164/attachment-0001.gif>
-------------- next part --------------
-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
-------------- next part --------------
_______________________________________________
Numpy-discussion mailing list
Numpy-discussion at lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/numpy-discussion

From kwgoodman at gmail.com  Sat Nov 11 13:40:22 2006
From: kwgoodman at gmail.com (Keith Goodman)
Date: Sat, 11 Nov 2006 10:40:22 -0800
Subject: Assign NaN, get zero
Message-ID: <f4f93d420611111040u3ea1988el6b6baf01aba4891e@mail.gmail.com>

I accidentally wrote a unit test using int32 instead of float64 and
ran into this problem:

>> x = M.matrix([[1, 2, 3]])
>> x[0,1] = M.nan
>> x
matrix([[1, 0, 3]])   <--- Got 0 instead of NaN

But this, of course, works:

>> x = M.matrix([[1.0, 2.0, 3.0]])
>> x[0,1] = M.nan
>> x
matrix([[ 1.        ,         nan,  3.        ]])

Is returning a 0 instead of NaN the expected behavior?

If so do I need to check dtype before inserting NaNs? And if the dtype
is int32, then raise an exception?

I'm using 1.0rc1 from debian etch.

-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From stefan at sun.ac.za  Sat Nov 11 15:36:39 2006
From: stefan at sun.ac.za (Stefan van der Walt)
Date: Sat, 11 Nov 2006 22:36:39 +0200
Subject: Assign NaN, get zero
In-Reply-To: <f4f93d420611111040u3ea1988el6b6baf01aba4891e@mail.gmail.com>
References: <f4f93d420611111040u3ea1988el6b6baf01aba4891e@mail.gmail.com>
Message-ID: <20061111203639.GB22096@mentat.za.net>

On Sat, Nov 11, 2006 at 10:40:22AM -0800, Keith Goodman wrote:
> I accidentally wrote a unit test using int32 instead of float64 and
> ran into this problem:
> 
> >> x = M.matrix([[1, 2, 3]])
> >> x[0,1] = M.nan
> >> x
> matrix([[1, 0, 3]])   <--- Got 0 instead of NaN
> 
> But this, of course, works:
> 
> >> x = M.matrix([[1.0, 2.0, 3.0]])
> >> x[0,1] = M.nan
> >> x
> matrix([[ 1.        ,         nan,  3.        ]])
> 
> Is returning a 0 instead of NaN the expected behavior?

NaN (or inf) is a floating point number, so seeing a zero in integer
representation seems correct:

In [2]: int(N.nan)
Out[2]: 0L

Cheers
St?fan

-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From dalcinl at gmail.com  Sat Nov 11 16:30:06 2006
From: dalcinl at gmail.com (Lisandro Dalcin)
Date: Sat, 11 Nov 2006 18:30:06 -0300
Subject: Assign NaN, get zero
In-Reply-To: <20061111203639.GB22096@mentat.za.net>
References: <f4f93d420611111040u3ea1988el6b6baf01aba4891e@mail.gmail.com>
	<20061111203639.GB22096@mentat.za.net>
Message-ID: <e7ba66e40611111330x4e8f9c65x9db05792cac98fbe@mail.gmail.com>

On 11/11/06, Stefan van der Walt <stefan at sun.ac.za> wrote:
> NaN (or inf) is a floating point number, so seeing a zero in integer
> representation seems correct:
>
> In [2]: int(N.nan)
> Out[2]: 0L
>

Just to learn myself: Why int(N.nan) should be 0? Is it C behavior?

-- 
Lisandro Dalc?n
---------------
Centro Internacional de M?todos Computacionales en Ingenier?a (CIMEC)
Instituto de Desarrollo Tecnol?gico para la Industria Qu?mica (INTEC)
Consejo Nacional de Investigaciones Cient?ficas y T?cnicas (CONICET)
PTLC - G?emes 3450, (3000) Santa Fe, Argentina
Tel/Fax: +54-(0)342-451.1594

-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From charlesr.harris at gmail.com  Sat Nov 11 16:56:33 2006
From: charlesr.harris at gmail.com (Charles R Harris)
Date: Sat, 11 Nov 2006 14:56:33 -0700
Subject: Assign NaN, get zero
In-Reply-To: <e7ba66e40611111330x4e8f9c65x9db05792cac98fbe@mail.gmail.com>
References: <f4f93d420611111040u3ea1988el6b6baf01aba4891e@mail.gmail.com>
	<20061111203639.GB22096@mentat.za.net>
	<e7ba66e40611111330x4e8f9c65x9db05792cac98fbe@mail.gmail.com>
Message-ID: <e06186140611111356n1b02b48k68e57b7ce990d863@mail.gmail.com>

On 11/11/06, Lisandro Dalcin <dalcinl at gmail.com> wrote:
>
> On 11/11/06, Stefan van der Walt <stefan at sun.ac.za> wrote:
> > NaN (or inf) is a floating point number, so seeing a zero in integer
> > representation seems correct:
> >
> > In [2]: int(N.nan)
> > Out[2]: 0L
> >
>
> Just to learn myself: Why int(N.nan) should be 0? Is it C behavior?


In [1]: int32(0)/int32(0)
Warning: divide by zero encountered in long_scalars
Out[1]: 0

In [2]: float32(0)/float32(0)
Out[2]: nan

In [3]: int(nan)
Out[3]: 0L

I think it was just a default for numpy. Hmmm, numpy now warns on integer
division by zero, didn't used to.  Looks like a warning should also be
raised when casting nan to integer. It is probably a small bug not to. I
also suspect int(nan) should return a normal python zero, not 0L.

Chuck
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20061111/63228828/attachment-0001.html>
-------------- next part --------------
-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
-------------- next part --------------
_______________________________________________
Numpy-discussion mailing list
Numpy-discussion at lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/numpy-discussion

From stefan at sun.ac.za  Sat Nov 11 16:54:49 2006
From: stefan at sun.ac.za (Stefan van der Walt)
Date: Sat, 11 Nov 2006 23:54:49 +0200
Subject: Assign NaN, get zero
In-Reply-To: <e7ba66e40611111330x4e8f9c65x9db05792cac98fbe@mail.gmail.com>
References: <f4f93d420611111040u3ea1988el6b6baf01aba4891e@mail.gmail.com>
	<20061111203639.GB22096@mentat.za.net>
	<e7ba66e40611111330x4e8f9c65x9db05792cac98fbe@mail.gmail.com>
Message-ID: <20061111215448.GD23231@mentat.za.net>

On Sat, Nov 11, 2006 at 06:30:06PM -0300, Lisandro Dalcin wrote:
> On 11/11/06, Stefan van der Walt <stefan at sun.ac.za> wrote:
> > NaN (or inf) is a floating point number, so seeing a zero in integer
> > representation seems correct:
> >
> > In [2]: int(N.nan)
> > Out[2]: 0L
> >
> 
> Just to learn myself: Why int(N.nan) should be 0? Is it C behavior?

As far as I know (and please correct me if I'm wrong), nan's are just
a specific bit pattern set in memory when an invalid floating point
operation occurs (in IEEE 754 nan's are represented by an exponent of
all 1's and a non-zero mantissa).

Most integer representations have no way of indication an invalid
result (and C provides no such conversion, as far as I am aware), so
nan's are interpreted as 0 (which could have been any arbitrary number
for that matter, although 0 seems a logical choice).

Regards
St?fan

-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From kwgoodman at gmail.com  Sat Nov 11 16:59:40 2006
From: kwgoodman at gmail.com (Keith Goodman)
Date: Sat, 11 Nov 2006 13:59:40 -0800
Subject: Assign NaN, get zero
In-Reply-To: <20061111203639.GB22096@mentat.za.net>
References: <f4f93d420611111040u3ea1988el6b6baf01aba4891e@mail.gmail.com>
	<20061111203639.GB22096@mentat.za.net>
Message-ID: <f4f93d420611111359v1f95f928sa2f16eec54111aea@mail.gmail.com>

On 11/11/06, Stefan van der Walt <stefan at sun.ac.za> wrote:
> On Sat, Nov 11, 2006 at 10:40:22AM -0800, Keith Goodman wrote:
> > I accidentally wrote a unit test using int32 instead of float64 and
> > ran into this problem:
> >
> > >> x = M.matrix([[1, 2, 3]])
> > >> x[0,1] = M.nan
> > >> x
> > matrix([[1, 0, 3]])   <--- Got 0 instead of NaN
> >
> > But this, of course, works:
> >
> > >> x = M.matrix([[1.0, 2.0, 3.0]])
> > >> x[0,1] = M.nan
> > >> x
> > matrix([[ 1.        ,         nan,  3.        ]])
> >
> > Is returning a 0 instead of NaN the expected behavior?
>
> NaN (or inf) is a floating point number, so seeing a zero in integer
> representation seems correct:
>
> In [2]: int(N.nan)
> Out[2]: 0L

Would it make sense to upcast instead of downcast?

This upcasts:

>> x = M.matrix([[1, M.nan, 3]])
>> x
matrix([[ 1.        ,         nan,  3.        ]])

But this doesn't:

>> x = M.matrix([[1, 2, 3]])
>> x[0,1] = M.nan
>> x
matrix([[1, 0, 3]])

(BTW, how do you represent missing integers if you can't use NaN?)

-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From stefan at sun.ac.za  Sat Nov 11 17:10:46 2006
From: stefan at sun.ac.za (Stefan van der Walt)
Date: Sun, 12 Nov 2006 00:10:46 +0200
Subject: Assign NaN, get zero
In-Reply-To: <f4f93d420611111359v1f95f928sa2f16eec54111aea@mail.gmail.com>
References: <f4f93d420611111040u3ea1988el6b6baf01aba4891e@mail.gmail.com>
	<20061111203639.GB22096@mentat.za.net>
	<f4f93d420611111359v1f95f928sa2f16eec54111aea@mail.gmail.com>
Message-ID: <20061111221046.GE23231@mentat.za.net>

On Sat, Nov 11, 2006 at 01:59:40PM -0800, Keith Goodman wrote:
> Would it make sense to upcast instead of downcast?
> 
> This upcasts:
> 
> >> x = M.matrix([[1, M.nan, 3]])
> >> x
> matrix([[ 1.        ,         nan,  3.        ]])
> 
> But this doesn't:
> 
> >> x = M.matrix([[1, 2, 3]])
> >> x[0,1] = M.nan
> >> x
> matrix([[1, 0, 3]])

This behaviour is consistent with

x = N.array([[1,2.0,3]])

vs

x = N.array([1,2,3])
x[0,1] = 2.

> (BTW, how do you represent missing integers if you can't use NaN?)

I think masked arrays should work on integer arrays (alternatively, if
you have enough memory, cast your array to float).

Regards
St?fan

-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From koara at atlas.cz  Sat Nov 11 17:41:47 2006
From: koara at atlas.cz (koara)
Date: Sat, 11 Nov 2006 14:41:47 -0800
Subject: sparse SVD
Message-ID: <1163284907.662729.265470@m73g2000cwd.googlegroups.com>

Hello, is there a way to do SVD of large sparse matrices efficiently in
python? All i found was scipy (little sparse support, no SVD), pysparse
(no SVD) and PROPACK (no python). Out of these, PROPACK seems the most
enticing choice, but i have no experience porting fortran code and this
seems too big a bite. Any pointers or suggestions are most welcome!

Cheers.


-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From peridot.faceted at gmail.com  Sat Nov 11 17:57:24 2006
From: peridot.faceted at gmail.com (A. M. Archibald)
Date: Sat, 11 Nov 2006 17:57:24 -0500
Subject: sparse SVD
In-Reply-To: <1163284907.662729.265470@m73g2000cwd.googlegroups.com>
References: <1163284907.662729.265470@m73g2000cwd.googlegroups.com>
Message-ID: <ce557a360611111457j51be99a0j2e33db2fe5e27cbd@mail.gmail.com>

On 11/11/06, koara <koara at atlas.cz> wrote:
> Hello, is there a way to do SVD of large sparse matrices efficiently in
> python? All i found was scipy (little sparse support, no SVD), pysparse
> (no SVD) and PROPACK (no python). Out of these, PROPACK seems the most
> enticing choice, but i have no experience porting fortran code and this
> seems too big a bite. Any pointers or suggestions are most welcome!

Numpy includes the program "f2py" for generating python wrappers for
fortran code; it's not too difficult to use, even for a non-fortran
programmer like me.

If you do write wrappers, please send them to the scipy list - it's
always good to build up the library.

A. M. Archibald

-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From kwgoodman at gmail.com  Sat Nov 11 18:04:29 2006
From: kwgoodman at gmail.com (Keith Goodman)
Date: Sat, 11 Nov 2006 15:04:29 -0800
Subject: x.min() depends on ordering
Message-ID: <f4f93d420611111504k77908fdfqa630adf6138025c7@mail.gmail.com>

x.min() and x.max() depend on the ordering of the elements:

>> x = M.matrix([[ M.nan, 2.0, 1.0]])
>> x.min()
nan

>> x = M.matrix([[ 1.0, 2.0, M.nan]])
>> x.min()
1.0

If I were to try the latter in ipython, I'd assume, great, min()
ignores NaNs. But then the former would be a bug in my program.

Is this related to how sort works?

>> x = M.matrix([[ M.nan, 2.0, 1.0]])
>> x.sort()
>> x
matrix([[                  nan,            1.        ,            2.        ]])

>> x = M.matrix([[ 1.0, 2.0, M.nan]])
>> x.sort()
>> x
matrix([[ 1.        ,  2.        ,         nan]])

-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From robert.kern at gmail.com  Sat Nov 11 18:14:30 2006
From: robert.kern at gmail.com (Robert Kern)
Date: Sat, 11 Nov 2006 17:14:30 -0600
Subject: x.min() depends on ordering
In-Reply-To: <f4f93d420611111504k77908fdfqa630adf6138025c7@mail.gmail.com>
References: <f4f93d420611111504k77908fdfqa630adf6138025c7@mail.gmail.com>
Message-ID: <ej5lh1$f4p$1@sea.gmane.org>

Keith Goodman wrote:
> x.min() and x.max() depend on the ordering of the elements:
> 
>>> x = M.matrix([[ M.nan, 2.0, 1.0]])
>>> x.min()
> nan
> 
>>> x = M.matrix([[ 1.0, 2.0, M.nan]])
>>> x.min()
> 1.0
> 
> If I were to try the latter in ipython, I'd assume, great, min()
> ignores NaNs. But then the former would be a bug in my program.
> 
> Is this related to how sort works?

Not really. sort() is a more complicated algorithm that does a number of
different comparisons in an order that is difficult to determine beforehand.
x.min() should just be a straight pass through all of the elements. However, the
core problem is the same: a < nan, a > nan, a == nan are all False for any a.

Barring a clever solution (at least cleverer than I feel like being
immediately), the way to solve this would be to check for nans in the array and
deal with them separately (or simply ignore them in the case of x.min()).
However, this checking would slow down the common case that has no nans (sans
nans, if you will).

-- 
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless enigma
 that is made terrible by our own mad attempt to interpret it as though it had
 an underlying truth."
  -- Umberto Eco


-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From kwgoodman at gmail.com  Sat Nov 11 18:25:12 2006
From: kwgoodman at gmail.com (Keith Goodman)
Date: Sat, 11 Nov 2006 15:25:12 -0800
Subject: x.min() depends on ordering
In-Reply-To: <ej5lh1$f4p$1@sea.gmane.org>
References: <f4f93d420611111504k77908fdfqa630adf6138025c7@mail.gmail.com>
	<ej5lh1$f4p$1@sea.gmane.org>
Message-ID: <f4f93d420611111525o76503decp130a312742ef7922@mail.gmail.com>

On 11/11/06, Robert Kern <robert.kern at gmail.com> wrote:

> Barring a clever solution (at least cleverer than I feel like being
> immediately), the way to solve this would be to check for nans in the array and
> deal with them separately (or simply ignore them in the case of x.min()).
> However, this checking would slow down the common case that has no nans
> (sans nans, if you will).

I'm not one of the fans of sans nans.

I'd prefer a slower min() that ignored nans. But I'm probably in the minority.

How about a nanmin() function?

-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From charlesr.harris at gmail.com  Sat Nov 11 18:35:43 2006
From: charlesr.harris at gmail.com (Charles R Harris)
Date: Sat, 11 Nov 2006 16:35:43 -0700
Subject: x.min() depends on ordering
In-Reply-To: <ej5lh1$f4p$1@sea.gmane.org>
References: <f4f93d420611111504k77908fdfqa630adf6138025c7@mail.gmail.com>
	<ej5lh1$f4p$1@sea.gmane.org>
Message-ID: <e06186140611111535u7d7c273ek2ac0aa35fc82c22@mail.gmail.com>

On 11/11/06, Robert Kern <robert.kern at gmail.com> wrote:
>
> Keith Goodman wrote:
> > x.min() and x.max() depend on the ordering of the elements:
> >
> >>> x = M.matrix([[ M.nan, 2.0, 1.0]])
> >>> x.min()
> > nan
> >
> >>> x = M.matrix([[ 1.0, 2.0, M.nan]])
> >>> x.min()
> > 1.0
> >
> > If I were to try the latter in ipython, I'd assume, great, min()
> > ignores NaNs. But then the former would be a bug in my program.
> >
> > Is this related to how sort works?
>
> Not really. sort() is a more complicated algorithm that does a number of
> different comparisons in an order that is difficult to determine
> beforehand.
> x.min() should just be a straight pass through all of the elements.
> However, the
> core problem is the same: a < nan, a > nan, a == nan are all False for any
> a.
>
> Barring a clever solution (at least cleverer than I feel like being
> immediately), the way to solve this would be to check for nans in the
> array and
> deal with them separately (or simply ignore them in the case of x.min()).
> However, this checking would slow down the common case that has no nans
> (sans
> nans, if you will).


I think the problem is that the max and min functions use the first value in
the array as the starting point. That could be fixed by using the first
non-nan and returning nan if there aren't any "real" numbers.  But it
probably isn't worth the effort as the behavior becomes more complicated. A
better rule of thumb is to note that comparisons involving nans are
basically invalid because nans aren't comparable -- the comparison violates
trichotomy. Don't really know what to do about that.

Chuck
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20061111/a86bfdc6/attachment-0001.html>
-------------- next part --------------
-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
-------------- next part --------------
_______________________________________________
Numpy-discussion mailing list
Numpy-discussion at lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/numpy-discussion

From robert.kern at gmail.com  Sat Nov 11 18:39:13 2006
From: robert.kern at gmail.com (Robert Kern)
Date: Sat, 11 Nov 2006 17:39:13 -0600
Subject: x.min() depends on ordering
In-Reply-To: <f4f93d420611111525o76503decp130a312742ef7922@mail.gmail.com>
References: <f4f93d420611111504k77908fdfqa630adf6138025c7@mail.gmail.com>	<ej5lh1$f4p$1@sea.gmane.org>
	<f4f93d420611111525o76503decp130a312742ef7922@mail.gmail.com>
Message-ID: <ej5mvc$ibm$1@sea.gmane.org>

Keith Goodman wrote:
> How about a nanmin() function?

Already there.

In [2]: nanmin?
Type:           function
Base Class:     <type 'function'>
Namespace:      Interactive
File:
/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/site-packages/numpy-1.0.1.dev3432-py2.5-macosx-10.4-i386.egg/numpy/lib/function_base.py
Definition:     nanmin(a, axis=None)
Docstring:
    Find the minimium over the given axis, ignoring NaNs.

-- 
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless enigma
 that is made terrible by our own mad attempt to interpret it as though it had
 an underlying truth."
  -- Umberto Eco


-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From tim.hochberg at ieee.org  Sat Nov 11 18:46:47 2006
From: tim.hochberg at ieee.org (Tim Hochberg)
Date: Sat, 11 Nov 2006 16:46:47 -0700
Subject: x.min() depends on ordering
In-Reply-To: <ej5lh1$f4p$1@sea.gmane.org>
References: <f4f93d420611111504k77908fdfqa630adf6138025c7@mail.gmail.com>
	<ej5lh1$f4p$1@sea.gmane.org>
Message-ID: <455660E7.2010905@ieee.org>

Robert Kern wrote:
> Keith Goodman wrote:
>   
>> x.min() and x.max() depend on the ordering of the elements:
>>
>>     
>>>> x = M.matrix([[ M.nan, 2.0, 1.0]])
>>>> x.min()
>>>>         
>> nan
>>
>>     
>>>> x = M.matrix([[ 1.0, 2.0, M.nan]])
>>>> x.min()
>>>>         
>> 1.0
>>
>> If I were to try the latter in ipython, I'd assume, great, min()
>> ignores NaNs. But then the former would be a bug in my program.
>>
>> Is this related to how sort works?
>>     
>
> Not really. sort() is a more complicated algorithm that does a number of
> different comparisons in an order that is difficult to determine beforehand.
> x.min() should just be a straight pass through all of the elements. However, the
> core problem is the same: a < nan, a > nan, a == nan are all False for any a.
>
> Barring a clever solution (at least cleverer than I feel like being
> immediately), the way to solve this would be to check for nans in the array and
> deal with them separately (or simply ignore them in the case of x.min()).
> However, this checking would slow down the common case that has no nans (sans
> nans, if you will).
>   
For ignoring NaNs, isn't is simply a matter of scanning through the 
array till you find the first non NaN the proceeding as normal? In the 
common case, this requires one extra compare (or rather is_nan) which 
should be negligible in most circumstances. Only when you have an array 
with a load of NaNs at the beginning would it be slow. One would have to 
decide whether to return NaN or raise an error when there were no real 
numbers.

My preference would be to raise an error / warning when there is a nan 
in the array. Technically, there is no minimum value when a nan is 
present. I believe that this would be feasible be swapping the compare 
from 'a < b' to '!(a >= b)'. This should return NaN if any NaNs are 
present and I suspect the extra '!' will have minimal performance impact 
but it would have to be tested. Then a warning or error could be issued 
on the way out depending on the erstate. Arguably returning NaN is more 
correct than returning the smallest non NaN anyway.

As for Keith Goodmans request for a NaN ignoring min function, I suggest:

    a[~np.isnan(a)].min()


Or better yet, stop generating so many NaN's.

-tim




-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From kwgoodman at gmail.com  Sat Nov 11 18:52:31 2006
From: kwgoodman at gmail.com (Keith Goodman)
Date: Sat, 11 Nov 2006 15:52:31 -0800
Subject: x.min() depends on ordering
In-Reply-To: <ej5mvc$ibm$1@sea.gmane.org>
References: <f4f93d420611111504k77908fdfqa630adf6138025c7@mail.gmail.com>
	<ej5lh1$f4p$1@sea.gmane.org>
	<f4f93d420611111525o76503decp130a312742ef7922@mail.gmail.com>
	<ej5mvc$ibm$1@sea.gmane.org>
Message-ID: <f4f93d420611111552n3e523dd5p77a23d323451c2a5@mail.gmail.com>

On 11/11/06, Robert Kern <robert.kern at gmail.com> wrote:
> Keith Goodman wrote:
> > How about a nanmin() function?
>
> Already there.
>
> In [2]: nanmin?
> Type:           function
> Base Class:     <type 'function'>
> Namespace:      Interactive
> File:
> /Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/site-packages/numpy-1.0.1.dev3432-py2.5-macosx-10.4-i386.egg/numpy/lib/function_base.py
> Definition:     nanmin(a, axis=None)
> Docstring:
>     Find the minimium over the given axis, ignoring NaNs.

Thank you! I was just about to write my own using Chuck's email as a guide.

-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From peridot.faceted at gmail.com  Sat Nov 11 18:58:36 2006
From: peridot.faceted at gmail.com (A. M. Archibald)
Date: Sat, 11 Nov 2006 18:58:36 -0500
Subject: x.min() depends on ordering
In-Reply-To: <e06186140611111535u7d7c273ek2ac0aa35fc82c22@mail.gmail.com>
References: <f4f93d420611111504k77908fdfqa630adf6138025c7@mail.gmail.com>
	<ej5lh1$f4p$1@sea.gmane.org>
	<e06186140611111535u7d7c273ek2ac0aa35fc82c22@mail.gmail.com>
Message-ID: <ce557a360611111558k61027bf0v67cb7dbb01f8c5a2@mail.gmail.com>

On 11/11/06, Charles R Harris <charlesr.harris at gmail.com> wrote:

> I think the problem is that the max and min functions use the first value in
> the array as the starting point. That could be fixed by using the first
> non-nan and returning nan if there aren't any "real" numbers.  But it
> probably isn't worth the effort as the behavior becomes more complicated. A
> better rule of thumb is to note that comparisons involving nans are
> basically invalid because nans aren't comparable -- the comparison violates
> trichotomy. Don't really know what to do about that.

Well, we could get simple consistent behaviour by taking inf as the
initial value for min and -inf as the first value for max, then
reducing as normal. This would then, depending on how max and min are
implemented, either return NaN if any are present, or return the
smallest/largest non-NaN value (or inf/-inf if there are none)

A. M. Archibald

-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From charlesr.harris at gmail.com  Sat Nov 11 19:09:44 2006
From: charlesr.harris at gmail.com (Charles R Harris)
Date: Sat, 11 Nov 2006 17:09:44 -0700
Subject: x.min() depends on ordering
In-Reply-To: <455660E7.2010905@ieee.org>
References: <f4f93d420611111504k77908fdfqa630adf6138025c7@mail.gmail.com>
	<ej5lh1$f4p$1@sea.gmane.org> <455660E7.2010905@ieee.org>
Message-ID: <e06186140611111609m66afb5dey7a71679bcccc0a09@mail.gmail.com>

On 11/11/06, Tim Hochberg <tim.hochberg at ieee.org> wrote:
>
> Robert Kern wrote:


<snip>

My preference would be to raise an error / warning when there is a nan
> in the array. Technically, there is no minimum value when a nan is
> present. I believe that this would be feasible be swapping the compare
> from 'a < b' to '!(a >= b)'. This should return NaN if any NaNs are
> present and I suspect the extra '!' will have minimal performance impact
> but it would have to be tested. Then a warning or error could be issued
> on the way out depending on the erstate. Arguably returning NaN is more
> correct than returning the smallest non NaN anyway.
>

No telling what compiler optimizations might do with  '!(a >= b)' if they
assume that '!(a >= b)' == 'a < b'. For instance,

if !(a >= b)
    do something;
else
    do otherwise;

might branch to the second statement on  'a <b' and fall through to the
first otherwise.

Chuck
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20061111/272945c6/attachment-0001.html>
-------------- next part --------------
-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
-------------- next part --------------
_______________________________________________
Numpy-discussion mailing list
Numpy-discussion at lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/numpy-discussion

From charlesr.harris at gmail.com  Sat Nov 11 19:49:21 2006
From: charlesr.harris at gmail.com (Charles R Harris)
Date: Sat, 11 Nov 2006 17:49:21 -0700
Subject: x.min() depends on ordering
In-Reply-To: <e06186140611111609m66afb5dey7a71679bcccc0a09@mail.gmail.com>
References: <f4f93d420611111504k77908fdfqa630adf6138025c7@mail.gmail.com>
	<ej5lh1$f4p$1@sea.gmane.org> <455660E7.2010905@ieee.org>
	<e06186140611111609m66afb5dey7a71679bcccc0a09@mail.gmail.com>
Message-ID: <e06186140611111649o52843c6ey80b9751096e90a14@mail.gmail.com>

On 11/11/06, Charles R Harris <charlesr.harris at gmail.com> wrote:
>
>
>
> On 11/11/06, Tim Hochberg <tim.hochberg at ieee.org> wrote:
> >
> > Robert Kern wrote:
>
>
> <snip>
>
> My preference would be to raise an error / warning when there is a nan
> > in the array. Technically, there is no minimum value when a nan is
> > present. I believe that this would be feasible be swapping the compare
> > from 'a < b' to '!(a >= b)'. This should return NaN if any NaNs are
> > present and I suspect the extra '!' will have minimal performance impact
> > but it would have to be tested. Then a warning or error could be issued
> > on the way out depending on the erstate. Arguably returning NaN is more
> > correct than returning the smallest non NaN anyway.
> >
>
> No telling what compiler optimizations might do with  '!(a >= b)' if they
> assume that '!(a >= b)' == 'a < b'. For instance,
>
> if !(a >= b)
>     do something;
> else
>     do otherwise;
>

This made me curious. Here is the code

int test(double a, double b)
{
    if (a > b)
        return 0;
    return 1;
}

Here is the relevant part of the assembly code when compiled with no
optimizations

        fucompp
        fnstsw  %ax
        sahf
        ja      .L4
        jmp     .L2
.L4:
        movl    $0, -20(%ebp)
        jmp     .L5
.L2:
        movl    $1, -20(%ebp)
.L5:
        movl    -20(%ebp), %eax
        leave
        ret

Which jumps to the right place on a > b (ja)

Here is the relevant part of the assembly code when compiled with -O3

        fucompp
        fnstsw  %ax
        popl    %ebp
        sahf
        setbe   %al
        movzbl  %al, %eax
        ret

Which sets the value of the return to the logical value of a <= b (setbe),
which won't work right with NaNs. Maybe the compiler needs another flag to
deal with the possibility of NaN's because the generated code is actually
incorrect. Or maybe I just discovered a compiler bug. But boy, that compiler
is awesome clever. Those optimizers are getting better all the time.

Chuck
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20061111/f55488d4/attachment-0001.html>
-------------- next part --------------
-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
-------------- next part --------------
_______________________________________________
Numpy-discussion mailing list
Numpy-discussion at lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/numpy-discussion

From tim.hochberg at ieee.org  Sat Nov 11 19:49:26 2006
From: tim.hochberg at ieee.org (Tim Hochberg)
Date: Sat, 11 Nov 2006 17:49:26 -0700
Subject: x.min() depends on ordering
In-Reply-To: <ej5mvc$ibm$1@sea.gmane.org>
References: <f4f93d420611111504k77908fdfqa630adf6138025c7@mail.gmail.com>	<ej5lh1$f4p$1@sea.gmane.org>	<f4f93d420611111525o76503decp130a312742ef7922@mail.gmail.com>
	<ej5mvc$ibm$1@sea.gmane.org>
Message-ID: <45566F96.2090603@ieee.org>

Robert Kern wrote:
> Keith Goodman wrote:
>   
>> How about a nanmin() function?
>>     
>
> Already there.
>
> In [2]: nanmin?
> Type:           function
> Base Class:     <type 'function'>
> Namespace:      Interactive
> File:
> /Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/site-packages/numpy-1.0.1.dev3432-py2.5-macosx-10.4-i386.egg/numpy/lib/function_base.py
> Definition:     nanmin(a, axis=None)
> Docstring:
>     Find the minimium over the given axis, ignoring NaNs.
>
>   
Bah! That seems excessive. If I was king of the world one of the many 
things that I would stuff into a submodule is all the stuff relating to 
special values (nan, inf, isnan, isinf, namin, nanmax, nansum, 
nanargmax, nanargmin, nan_to_num, infty,  isneginf, isposinf and 
probably some others that I'm missing). First, these are mostly clutter 
in the base namespace. Not only would it make the main namespace easier 
to navigate in (although there's much clean up that would have to be 
done before it would be anything approaching easy to navigate). Second, 
for those who actually do need them, they'd be easier to find if they 
were all grouped together -- Keith for example would almost certainly 
have immediately found nanmin. Third, and this is perhaps a matter of 
opinion, there seems to be a sudden urge to abuse NaNs. Perhaps if they 
were shunted a bit off to the side, this temptation would be lifted.

Curmudgeonly yours,

-tim


-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From charlesr.harris at gmail.com  Sat Nov 11 20:06:57 2006
From: charlesr.harris at gmail.com (Charles R Harris)
Date: Sat, 11 Nov 2006 18:06:57 -0700
Subject: x.min() depends on ordering
In-Reply-To: <45566F96.2090603@ieee.org>
References: <f4f93d420611111504k77908fdfqa630adf6138025c7@mail.gmail.com>
	<ej5lh1$f4p$1@sea.gmane.org>
	<f4f93d420611111525o76503decp130a312742ef7922@mail.gmail.com>
	<ej5mvc$ibm$1@sea.gmane.org> <45566F96.2090603@ieee.org>
Message-ID: <e06186140611111706u34509991j9f08dbedcdeddce@mail.gmail.com>

On 11/11/06, Tim Hochberg <tim.hochberg at ieee.org> wrote:
>
> Robert Kern wrote:
> > Keith Goodman wrote:
> >
> >> How about a nanmin() function?
> >>
> >
> > Already there.
> >
> > In [2]: nanmin?
> > Type:           function
> > Base Class:     <type 'function'>
> > Namespace:      Interactive
> > File:
> >
> /Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/site-packages/numpy-
> 1.0.1.dev3432-py2.5-macosx-10.4-i386.egg/numpy/lib/function_base.py
> > Definition:     nanmin(a, axis=None)
> > Docstring:
> >     Find the minimium over the given axis, ignoring NaNs.
> >
> >
> Bah! That seems excessive. If I was king of the world one of the many
> things that I would stuff into a submodule is all the stuff relating to
> special values (nan, inf, isnan, isinf, namin, nanmax, nansum,
> nanargmax, nanargmin, nan_to_num, infty,  isneginf, isposinf and
> probably some others that I'm missing). First, these are mostly clutter
> in the base namespace. Not only would it make the main namespace easier
> to navigate in (although there's much clean up that would have to be
> done before it would be anything approaching easy to navigate). Second,
> for those who actually do need them, they'd be easier to find if they
> were all grouped together -- Keith for example would almost certainly
> have immediately found nanmin. Third, and this is perhaps a matter of
> opinion, there seems to be a sudden urge to abuse NaNs. Perhaps if they
> were shunted a bit off to the side, this temptation would be lifted.
>
> Curmudgeonly yours,


Oh yes. And let's reserve a bit of abuse for whoever mixed up the nans with
the rest. I mean, the infs and such actually make some sense as numbers, but
nans are, well, nans. So it would have been nice to enable everything
*except* nans, and have an errorflag set whenever the latter turned up.  Or
if folks just have to have nans, make them compare less than anything else.
If isnan were fast and easy one could use LT(a,b) := isnan(a) || a < b.

Chuck
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20061111/1535e69f/attachment-0001.html>
-------------- next part --------------
-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
-------------- next part --------------
_______________________________________________
Numpy-discussion mailing list
Numpy-discussion at lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/numpy-discussion

From tim.hochberg at ieee.org  Sat Nov 11 20:55:21 2006
From: tim.hochberg at ieee.org (Tim Hochberg)
Date: Sat, 11 Nov 2006 18:55:21 -0700
Subject: x.min() depends on ordering
In-Reply-To: <e06186140611111706u34509991j9f08dbedcdeddce@mail.gmail.com>
References: <f4f93d420611111504k77908fdfqa630adf6138025c7@mail.gmail.com>	<ej5lh1$f4p$1@sea.gmane.org>	<f4f93d420611111525o76503decp130a312742ef7922@mail.gmail.com>	<ej5mvc$ibm$1@sea.gmane.org>
	<45566F96.2090603@ieee.org>
	<e06186140611111706u34509991j9f08dbedcdeddce@mail.gmail.com>
Message-ID: <45567F09.3030504@ieee.org>

Charles R Harris wrote:
>
>
> On 11/11/06, *Tim Hochberg* <tim.hochberg at ieee.org 
> <mailto:tim.hochberg at ieee.org>> wrote:
>
>     Robert Kern wrote:
>     > Keith Goodman wrote:
>     >
>     >> How about a nanmin() function?
>     >>
>     >
>     > Already there.
>     >
>     > In [2]: nanmin?
>     > Type:           function
>     > Base Class:     <type 'function'>
>     > Namespace:      Interactive
>     > File:
>     >
>     /Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/site-packages/numpy-1.0.1.dev3432-py2.5-macosx-10.4-i386.egg/numpy/lib/function_base.py
>     > Definition:     nanmin(a, axis=None)
>     > Docstring:
>     >     Find the minimium over the given axis, ignoring NaNs.
>     >
>     >
>     Bah! That seems excessive. If I was king of the world one of the many
>     things that I would stuff into a submodule is all the stuff
>     relating to
>     special values (nan, inf, isnan, isinf, namin, nanmax, nansum,
>     nanargmax, nanargmin, nan_to_num, infty,  isneginf, isposinf and
>     probably some others that I'm missing). First, these are mostly
>     clutter
>     in the base namespace. Not only would it make the main namespace
>     easier
>     to navigate in (although there's much clean up that would have to be
>     done before it would be anything approaching easy to navigate).
>     Second,
>     for those who actually do need them, they'd be easier to find if they
>     were all grouped together -- Keith for example would almost certainly
>     have immediately found nanmin. Third, and this is perhaps a matter of
>     opinion, there seems to be a sudden urge to abuse NaNs. Perhaps if
>     they
>     were shunted a bit off to the side, this temptation would be lifted.
>
>     Curmudgeonly yours,
>
>
> Oh yes. And let's reserve a bit of abuse for whoever mixed up the nans 
> with the rest. I mean, the infs and such actually make some sense as 
> numbers, but nans are, well, nans. So it would have been nice to 
> enable everything *except* nans, and have an errorflag set whenever 
> the latter turned up.
Actually you can do this. At least for the most part, I'm sure there are 
some corners that aren't working right yet.  Operations on NaNs set the 
invalid flag while infinities set the overflow flag (actually the flags 
are only set when you first get the infinity or NaN, at least on my 
platform). So, you can make invalid a flag and ignore overflow by using:

np.seterr(over='ignore', invalid='raise')

I doubt this does *exactly* what you want, but it may be close.

-tim


[snip]




-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From fingerslip at yahoo.com  Sun Nov 12 02:17:32 2006
From: fingerslip at yahoo.com (Abhishek Roy)
Date: Sun, 12 Nov 2006 07:17:32 +0000 (GMT)
Subject: Compiling numpy with lapack and atlas
Message-ID: <20061112071732.84470.qmail@web38906.mail.mud.yahoo.com>

Hello,
      I am trying to compile Numpy with lapack & atlas. In site.cfg I put,
[atlas]
library_dirs = /usr/local/lib/atlas/
atlas_libs = cblas, lapack, f77blas, atlas

and installation seems to go well. It says,
  FOUND:
    libraries = ['cblas', 'lapack', 'f77blas', 'atlas']
    library_dirs = ['/usr/local/lib/atlas/']
    language = c
    include_dirs = ['/usr/local/lib/atlas']
and the lines with gcc <something> -lcblas -llapack don't give an error. But
afterwards running,
$ ldd ~/numpy-1.0/build/lib.linux-i686-2.4/numpy/linalg/lapack_lite.so
linux-gate.so.1 =>  (0xffffe000)
libpthread.so.0 => /lib/libpthread.so.0 (0x40238000)
libc.so.6 => /lib/libc.so.6 (0x4028b000)
/lib/ld-linux.so.2 (0x80000000)

which I think means libcblas and libatlas are not linked. Can someone tell me
what's going wrong here?

Thanks,
Abhishek

Send instant messages to your online friends http://uk.messenger.yahoo.com 

-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From robert.kern at gmail.com  Sun Nov 12 02:34:53 2006
From: robert.kern at gmail.com (Robert Kern)
Date: Sun, 12 Nov 2006 01:34:53 -0600
Subject: Compiling numpy with lapack and atlas
In-Reply-To: <20061112071732.84470.qmail@web38906.mail.mud.yahoo.com>
References: <20061112071732.84470.qmail@web38906.mail.mud.yahoo.com>
Message-ID: <ej6ir8$72b$1@sea.gmane.org>

Abhishek Roy wrote:
> Hello,
>       I am trying to compile Numpy with lapack & atlas. In site.cfg I put,
> [atlas]
> library_dirs = /usr/local/lib/atlas/
> atlas_libs = cblas, lapack, f77blas, atlas
> 
> and installation seems to go well. It says,
>   FOUND:
>     libraries = ['cblas', 'lapack', 'f77blas', 'atlas']
>     library_dirs = ['/usr/local/lib/atlas/']
>     language = c
>     include_dirs = ['/usr/local/lib/atlas']
> and the lines with gcc <something> -lcblas -llapack don't give an error. But
> afterwards running,
> $ ldd ~/numpy-1.0/build/lib.linux-i686-2.4/numpy/linalg/lapack_lite.so
> linux-gate.so.1 =>  (0xffffe000)
> libpthread.so.0 => /lib/libpthread.so.0 (0x40238000)
> libc.so.6 => /lib/libc.so.6 (0x4028b000)
> /lib/ld-linux.so.2 (0x80000000)
> 
> which I think means libcblas and libatlas are not linked. Can someone tell me
> what's going wrong here?

Are your ATLAS libraries shared or static (i.e., are they libatlas.so or
libatlas.a)? ldd(1) only lists the shared libraries that are linked. ATLAS is
usually installed as static libraries.

Does that module import? If so, then you have no problems. If not, then copy the
message here. It should tell you one of the symbols that are missing.

Note that libraries should be listed in the order in which they depend on each
other. lapack depends on f77blas and cblas which both depend on atlas.

  lapack,f77blas,cblas,atlas

-- 
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless enigma
 that is made terrible by our own mad attempt to interpret it as though it had
 an underlying truth."
  -- Umberto Eco


-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From lbixeqb at party1.com  Sun Nov 12 02:56:47 2006
From: lbixeqb at party1.com (Center Developing)
Date: Sun, 12 Nov 2006 08:56:47 +0100
Subject: Scan MAC Repair
References: <000a01c70630$1a45d710$00000000@karpik>
Message-ID: <000a01c70630$1a45d710$00000000@karpik>


  ----- Original Message ----- 
  From: numpy-discussion at lists.sourceforge.net 
  To: lbixeqb at party1.com 
  Sent: Monday, November 02:18:15 AM
  Subject: Scan MAC Repair


  Apos Transfer Weigh Benefits Ltpipsec Pros Cons Advantages.
  Ghost Solve is Winktech Enter a event in id Winternals. Csgew a Failure am Launching or Published Citrix Secure Adminpak.
  Geekcom Geek favorite tv show in aposnet Whirlpool of Thread a. Buycom Befvp Etherfast is external am Configure Ipsec.
  Home lan diy Share an Internet Connection in Linksys Cabledsl. Second cpu Virtualdr Forums.
  Right Cracks anytime. Bbs Deleating accessing Pack. Auto Cannot Export Fields.
  Cars greeting Strassberg. Real Audio Midiaposs Multi or.
  Pcdocrx a Oncall fixed is our diagnostic test or.
  Vpnclient or Clienteasy Xauth Split Tunnel Systems a Active.
  Remote User Manual Tutorial Acidcool Cracking Answer. Fiction Trialware United States.
  Befvp is Etherfast external Configure Ipsec Tunneling? Should am Know is Security Bulletin.
  Rules Wizard Contacts or Display Same. Real Audio Midiaposs Multi or.
  Ghost Solve is Winktech Enter a event in id Winternals.
  Guarantee of Appligent Apcrypt finance. Windowsx Filetracer am finds.
  Vobdvdfile Ripping Vobaudio Ssrc Pinoyaposs Neue? Recovery Intuit vpn Solution Find in Virtual.
  Tqm iso six Sigma Projectpix a Visio onproject Webbased in.
  Vobdvdfile Ripping Vobaudio Ssrc Pinoyaposs Neue? Activex Controls ssl Strategy Hotfix Csgew. Password Protection System Lite Version.
  Stopsign tm Stopsigntm Escorcher Present. Developing Effective Level Management.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20061112/b6c47d2d/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: Resource.gif
Type: image/gif
Size: 14125 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20061112/b6c47d2d/attachment-0001.gif>
-------------- next part --------------
-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
-------------- next part --------------
_______________________________________________
Numpy-discussion mailing list
Numpy-discussion at lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/numpy-discussion

From ndbecker2 at gmail.com  Sun Nov 12 06:25:29 2006
From: ndbecker2 at gmail.com (Neal Becker)
Date: Sun, 12 Nov 2006 06:25:29 -0500
Subject: Compiling numpy with lapack and atlas
References: <20061112071732.84470.qmail@web38906.mail.mud.yahoo.com>
Message-ID: <ej70b5$17a$2@sea.gmane.org>

I see you're on linux.  If this is Fedora, atlas (and the other libaries)
are available from standard repos (core + extras)


-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From gael.varoquaux at normalesup.org  Sun Nov 12 10:51:02 2006
From: gael.varoquaux at normalesup.org (Gael Varoquaux)
Date: Sun, 12 Nov 2006 16:51:02 +0100
Subject: Helper function to "unroll" a array
In-Reply-To: <20061020112848.GB3555@clipper.ens.fr>
References: <20061020112848.GB3555@clipper.ens.fr>
Message-ID: <20061112155102.GB29963@clipper.ens.fr>

    Hi all,

I didn't get any answers to this email. Is it because the proposed
addition to numpy is not of any interest to anybody apart from me ?
Maybe the way I introduced this is wrong. Please tell me what is wrong
with this proposition.

Regards,

Ga?l

On Fri, Oct 20, 2006 at 01:28:52PM +0200, Gael Varoquaux wrote:
> Hi,

> There is an operation I do a lot, I would call it "unrolling" a array.
> The best way to describe it is probably to give the code:

> def unroll(M):
>     """ Flattens the array M and returns a 2D array with the first columns 
> 	being the indices of M, and the last column the flatten M.
>     """
>     return hstack((indices(M.shape).reshape(-1,M.ndim),M.reshape(-1,1)))

> Example:

> >>> M
> array([[ 0.73530097,  0.3553424 ,  0.3719772 ],
>        [ 0.83353373,  0.74622133,  0.14748905],
>        [ 0.72023762,  0.32306969,  0.19142366]])

> >>> unroll(M)
> array([[ 0.        ,  0.        ,  0.73530097],
>        [ 0.        ,  1.        ,  0.3553424 ],
>        [ 1.        ,  1.        ,  0.3719772 ],
>        [ 2.        ,  2.        ,  0.83353373],
>        [ 2.        ,  0.        ,  0.74622133],
>        [ 1.        ,  2.        ,  0.14748905],
>        [ 0.        ,  1.        ,  0.72023762],
>        [ 2.        ,  0.        ,  0.32306969],
>        [ 1.        ,  2.        ,  0.19142366]])


> The docstring sucks. The function is trivial (when you know numpy a bit).
> Maybe this function already exists in numpy, if so I couldn't find it.
> Elsewhere I propose it for inclusion.

> Cheers,

> Ga?l

> -------------------------------------------------------------------------
> Using Tomcat but need to do more? Need to support web services, security?
> Get stuff done quickly with pre-integrated technology to make your job easier
> Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
> http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
> _______________________________________________
> Numpy-discussion mailing list
> Numpy-discussion at lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/numpy-discussion

-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From svetosch at gmx.net  Sun Nov 12 12:07:30 2006
From: svetosch at gmx.net (Sven Schreiber)
Date: Sun, 12 Nov 2006 18:07:30 +0100
Subject: Helper function to "unroll" a array
In-Reply-To: <20061112155102.GB29963@clipper.ens.fr>
References: <20061020112848.GB3555@clipper.ens.fr>
	<20061112155102.GB29963@clipper.ens.fr>
Message-ID: <455754D2.3050309@gmx.net>

Gael Varoquaux schrieb:
>     Hi all,
> 
> I didn't get any answers to this email. Is it because the proposed
> addition to numpy is not of any interest to anybody apart from me ?
> Maybe the way I introduced this is wrong. Please tell me what is wrong
> with this proposition.
> 

Well you didn't mentioned why it would be useful for a broad audience,
only that you yourself use it alot. Together with the fact that it's a
one-liner this presumably means that it's not exactly a prime candidate
for adoption. (I'm just another user, so I'm speculating a bit here
based on my reading of past postings on this list.)
-sven

-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From gael.varoquaux at normalesup.org  Sun Nov 12 12:12:31 2006
From: gael.varoquaux at normalesup.org (Gael Varoquaux)
Date: Sun, 12 Nov 2006 18:12:31 +0100
Subject: Helper function to "unroll" a array
In-Reply-To: <455754D2.3050309@gmx.net>
References: <20061020112848.GB3555@clipper.ens.fr>
	<20061112155102.GB29963@clipper.ens.fr> <455754D2.3050309@gmx.net>
Message-ID: <20061112171230.GC29963@clipper.ens.fr>

You're probably right. Well it would most definitely be useful for all
the lads in my lab, but I am not sure this is a broad audience. The use
case is when you have a array representing data in an "mgrid" way, and
you wnat to apply transformations to the coordinates. It is something I
have done and seen done quite often.

If I am the only one a the list who see a use for this, then forget it.

Ga?l

On Sun, Nov 12, 2006 at 06:07:30PM +0100, Sven Schreiber wrote:
> Well you didn't mentioned why it would be useful for a broad audience,
> only that you yourself use it alot. Together with the fact that it's a
> one-liner this presumably means that it's not exactly a prime candidate
> for adoption. (I'm just another user, so I'm speculating a bit here
> based on my reading of past postings on this list.)


-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From george.sakkis at gmail.com  Sun Nov 12 14:16:25 2006
From: george.sakkis at gmail.com (George Sakkis)
Date: Sun, 12 Nov 2006 19:16:25 -0000
Subject: Equivalent to list.index ?
Message-ID: <1163358985.920640.306720@k70g2000cwa.googlegroups.com>

This must be pretty trivial but I couldn't find it in the docs: what's
the "numpythonic" way to find the (first) index of an element, i.e. the
equivalent to list.index ?

Thanks,
George


-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From kwgoodman at gmail.com  Sun Nov 12 14:21:43 2006
From: kwgoodman at gmail.com (Keith Goodman)
Date: Sun, 12 Nov 2006 11:21:43 -0800
Subject: Equivalent to list.index ?
In-Reply-To: <1163358985.920640.306720@k70g2000cwa.googlegroups.com>
References: <1163358985.920640.306720@k70g2000cwa.googlegroups.com>
Message-ID: <f4f93d420611121121y2620c6fdj60241acef01e30b4@mail.gmail.com>

On 11/12/06, George Sakkis <george.sakkis at gmail.com> wrote:
> This must be pretty trivial but I couldn't find it in the docs: what's
> the "numpythonic" way to find the (first) index of an element, i.e. the
> equivalent to list.index ?

Does where work?

-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From kwgoodman at gmail.com  Sun Nov 12 15:12:42 2006
From: kwgoodman at gmail.com (Keith Goodman)
Date: Sun, 12 Nov 2006 12:12:42 -0800
Subject: I can slice but I can't assign
Message-ID: <f4f93d420611121212m5f09f1f3x6790224a09b9a339@mail.gmail.com>

>> x = M.matrix([[1, 2], [3, 4]])
>> x
matrix([[1, 2],
        [3, 4]])

>> y = M.matrix([[5], [6]])
>> y
matrix([[5],
        [6]])

>> idx = M.where(x[:,0].A < 100)[0]
>> idx
array([0, 1])


>> x[idx,0]   # <---- This works
matrix([[1],
        [3]])

>> x[idx,0] = y    # <---- But this doesn't work. How can I make it work?

ValueError: array is not broadcastable to correct shape

-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From kwgoodman at gmail.com  Sun Nov 12 15:18:51 2006
From: kwgoodman at gmail.com (Keith Goodman)
Date: Sun, 12 Nov 2006 12:18:51 -0800
Subject: I can slice but I can't assign
In-Reply-To: <f4f93d420611121212m5f09f1f3x6790224a09b9a339@mail.gmail.com>
References: <f4f93d420611121212m5f09f1f3x6790224a09b9a339@mail.gmail.com>
Message-ID: <f4f93d420611121218n5fef3a3bic134e0bf39be2d3e@mail.gmail.com>

On 11/12/06, Keith Goodman <kwgoodman at gmail.com> wrote:
> >> x = M.matrix([[1, 2], [3, 4]])
> >> x
> matrix([[1, 2],
>         [3, 4]])
>
> >> y = M.matrix([[5], [6]])
> >> y
> matrix([[5],
>         [6]])
>
> >> idx = M.where(x[:,0].A < 100)[0]
> >> idx
> array([0, 1])
>
>
> >> x[idx,0]   # <---- This works
> matrix([[1],
>         [3]])
>
> >> x[idx,0] = y    # <---- But this doesn't work. How can I make it work?
>
> ValueError: array is not broadcastable to correct shape

Awkward, but it works:

>> idx = M.asmatrix(idx)
>> idx
matrix([[0, 1]])
>> idx = idx.T
>> idx
matrix([[0],
        [1]])

>> x[idx,0] = y

But it does seem odd to me that the array idx in the original example
works for a slice but for not an assignment.

-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From george.sakkis at gmail.com  Sun Nov 12 15:19:03 2006
From: george.sakkis at gmail.com (George Sakkis)
Date: Sun, 12 Nov 2006 20:19:03 -0000
Subject: Equivalent to list.index ?
References: <1163358985.920640.306720@k70g2000cwa.googlegroups.com>
	<f4f93d420611121121y2620c6fdj60241acef01e30b4@mail.gmail.com>
Message-ID: <1163362743.775427.260100@f16g2000cwb.googlegroups.com>

Keith Goodman wrote:

> On 11/12/06, George Sakkis <george.sakkis at gmail.com> wrote:
> > This must be pretty trivial but I couldn't find it in the docs: what's
> > the "numpythonic" way to find the (first) index of an element, i.e. the
> > equivalent to list.index ?
>
> Does where work?

def index(a, value):
    return N.where(a==value)[0][0]

This works but seems clunky and less efficient than necessary. If there
isn't a better alternative, I would welcome a new index()
function/method in the next numpy release.

George


-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From fingerslip at yahoo.com  Sun Nov 12 15:36:30 2006
From: fingerslip at yahoo.com (Abhishek Roy)
Date: Sun, 12 Nov 2006 20:36:30 +0000 (GMT)
Subject: Compiling numpy with lapack and atlas
In-Reply-To: <ej6ir8$72b$1@sea.gmane.org>
References: <ej6ir8$72b$1@sea.gmane.org>
Message-ID: <20061112203630.98834.qmail@web38902.mail.mud.yahoo.com>

--- Robert Kern <robert.kern at gmail.com> wrote:
> Are your ATLAS libraries shared or static (i.e., are they libatlas.so or
> libatlas.a)? ldd(1) only lists the shared libraries that are linked. ATLAS is
> usually installed as static libraries.
They are static. Thanks, I was mindlessly copying a suggestion I saw elsewhere.
 
> Does that module import? 
I got,
ImportError: /usr/lib/python2.4/site-packages/numpy/linalg/lapack_lite.so:
undefined symbol: s_cat

even after changing the order of libraries in site.cfg to,
>   lapack,f77blas,cblas,atlas

Abhishek

Send instant messages to your online friends http://uk.messenger.yahoo.com 

-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From tim.hochberg at ieee.org  Sun Nov 12 16:39:42 2006
From: tim.hochberg at ieee.org (Tim Hochberg)
Date: Sun, 12 Nov 2006 14:39:42 -0700
Subject: Equivalent to list.index ?
In-Reply-To: <1163362743.775427.260100@f16g2000cwb.googlegroups.com>
References: <1163358985.920640.306720@k70g2000cwa.googlegroups.com>	<f4f93d420611121121y2620c6fdj60241acef01e30b4@mail.gmail.com>
	<1163362743.775427.260100@f16g2000cwb.googlegroups.com>
Message-ID: <4557949E.5060009@ieee.org>

George Sakkis wrote:
> def index(a, value):
>     return N.where(a==value)[0][0]
>   
Or

def index(a, value):
    return argmax(a == value)


> This works but seems clunky and less efficient than necessary. If there
> isn't a better alternative, I would welcome a new index()
> function/method in the next numpy release.
>
>   

Without commenting on the desirability of this particular function, 
people seem awfully eager to add functions and methods everywhere. These 
aren't free you know. They cost in terms of maintenance and potential 
bugs, and in terms of making the already large function set even harder 
to wrap ones head around. That's partly an organizational problem but 
partly an issue of sheer size, neither of which are helped by adding a 
bunch more stuff.

    Not every two-line Python function has to come pre-written 
        --Tim Peters

In the case of this particular function, what are the use cases? Are 
they common (not just with you but with other numpy users)? Are the uses 
speed critical? Is it a good building block for other numeric 
functions?  I'm skeptical of the above implementation as it doesn't fit 
well with other similar numpy functions (argmin and argmax for example) 
and I personally don't see much in the way of compelling uses for this, 
but feel free to try to convince me otherwise.

-tim







-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From kwgoodman at gmail.com  Sun Nov 12 16:57:17 2006
From: kwgoodman at gmail.com (Keith Goodman)
Date: Sun, 12 Nov 2006 13:57:17 -0800
Subject: Could numpy.matlib.nanmax return a matrix?
Message-ID: <f4f93d420611121357o5dda5950g4fdcdc2f657ee654@mail.gmail.com>

Is anybody interested in making x.max() and nanmax() behave the same
for matrices, except for the NaN part? That is, make
numpy.matlib.nanmax return a matrix instead of an array.

Example
=====

>> import numpy.matlib as M
>> x = M.rand(3,2)

>> x.max(0)
matrix([[ 0.91749823,  0.94942954]])

>> M.nanmax(x, 0)
array([ 0.91749823,  0.94942954])

-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From peridot.faceted at gmail.com  Sun Nov 12 17:08:34 2006
From: peridot.faceted at gmail.com (A. M. Archibald)
Date: Sun, 12 Nov 2006 17:08:34 -0500
Subject: Could numpy.matlib.nanmax return a matrix?
In-Reply-To: <f4f93d420611121357o5dda5950g4fdcdc2f657ee654@mail.gmail.com>
References: <f4f93d420611121357o5dda5950g4fdcdc2f657ee654@mail.gmail.com>
Message-ID: <ce557a360611121408k6bae03fds9b2e979d786bb74e@mail.gmail.com>

On 12/11/06, Keith Goodman <kwgoodman at gmail.com> wrote:
> Is anybody interested in making x.max() and nanmax() behave the same
> for matrices, except for the NaN part? That is, make
> numpy.matlib.nanmax return a matrix instead of an array.

Sounds good to me; maybe put a patch in the tracker?

A. M. Archibald

-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From cjw at sympatico.ca  Sun Nov 12 17:53:59 2006
From: cjw at sympatico.ca (Colin J. Williams)
Date: Sun, 12 Nov 2006 17:53:59 -0500
Subject: Equivalent to list.index ?
In-Reply-To: <4557949E.5060009@ieee.org>
References: <1163358985.920640.306720@k70g2000cwa.googlegroups.com>	<f4f93d420611121121y2620c6fdj60241acef01e30b4@mail.gmail.com>	<1163362743.775427.260100@f16g2000cwb.googlegroups.com>
	<4557949E.5060009@ieee.org>
Message-ID: <4557A607.5060804@sympatico.ca>

An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20061112/6c6bc99f/attachment-0001.html>
-------------- next part --------------
-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
-------------- next part --------------
_______________________________________________
Numpy-discussion mailing list
Numpy-discussion at lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/numpy-discussion

From pgmdevlist at gmail.com  Sun Nov 12 18:08:50 2006
From: pgmdevlist at gmail.com (Pierre GM)
Date: Sun, 12 Nov 2006 18:08:50 -0500
Subject: Could numpy.matlib.nanmax return a matrix?
In-Reply-To: <ce557a360611121408k6bae03fds9b2e979d786bb74e@mail.gmail.com>
References: <f4f93d420611121357o5dda5950g4fdcdc2f657ee654@mail.gmail.com>
	<ce557a360611121408k6bae03fds9b2e979d786bb74e@mail.gmail.com>
Message-ID: <200611121808.51298.pgmdevlist@gmail.com>

On Sunday 12 November 2006 17:08, A. M. Archibald wrote:
> On 12/11/06, Keith Goodman <kwgoodman at gmail.com> wrote:
> > Is anybody interested in making x.max() and nanmax() behave the same
> > for matrices, except for the NaN part? That is, make
> > numpy.matlib.nanmax return a matrix instead of an array.

Or, you could use masked arrays... In the new implementation, you can add a 
mask to a subclassed array (such as matrix) to get a regular masked array. If 
you fill this masked array, you get an array of the same subclass.

>>> import numpy as N
>>> import numpy.matlib as M
>>> import maskedarray as MA
>>> x=M.rand(3,3)
>>> assert isinstance(x.max(0), M.matrix)
>>> assert isinstance(N.max(x,0), M.matrix)
>>> assert  isinstance(MA.max(x,0).filled(0), M.matrix)
>>> assert isinstance(MA.max(x,0)._data, M.matrix)

>>> x[-1,-1] = N.nan
>>> tmp = MA.max(MA.array(x,mask=N.isnan(x)), 0)
>>> assert (tmp == N.nanmax(x,0)).all()
>>> assert isinstance(tmp.filled(0), M.matrix)

-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From robert.kern at gmail.com  Sun Nov 12 18:13:25 2006
From: robert.kern at gmail.com (Robert Kern)
Date: Sun, 12 Nov 2006 17:13:25 -0600
Subject: Compiling numpy with lapack and atlas
In-Reply-To: <20061112203630.98834.qmail@web38902.mail.mud.yahoo.com>
References: <ej6ir8$72b$1@sea.gmane.org>
	<20061112203630.98834.qmail@web38902.mail.mud.yahoo.com>
Message-ID: <ej89qu$o7o$1@sea.gmane.org>

Abhishek Roy wrote:
> --- Robert Kern <robert.kern at gmail.com> wrote:
>> Are your ATLAS libraries shared or static (i.e., are they libatlas.so or
>> libatlas.a)? ldd(1) only lists the shared libraries that are linked. ATLAS is
>> usually installed as static libraries.
> They are static. Thanks, I was mindlessly copying a suggestion I saw elsewhere.
>  
>> Does that module import? 
> I got,
> ImportError: /usr/lib/python2.4/site-packages/numpy/linalg/lapack_lite.so:
> undefined symbol: s_cat

Also add g2c to your library list, then. It is g77's FORTRAN runtime library.

-- 
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless enigma
 that is made terrible by our own mad attempt to interpret it as though it had
 an underlying truth."
  -- Umberto Eco


-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From erin.sheldon at gmail.com  Sun Nov 12 18:56:29 2006
From: erin.sheldon at gmail.com (Erin Sheldon)
Date: Sun, 12 Nov 2006 18:56:29 -0500
Subject: array from list of lists
Message-ID: <20061112235629.GB5596@66-234-56-83.nyc.cable.nyct.net>

Hi all-

I want to take the result from a database query,
and create a numpy array with field names and types 
corresponding to the returned columns.

The DBI 2.0 compliant interfaces return lists
of lists. E.g.


[[94137100072000193L, 94, 345.57215100000002, -0.83673208099999996],
 [94137100072000368L, 94, 345.60217299999999, -0.83766954299999996],
 ....
 [94137100083000157L, 94, 347.21668099999999, -0.83572582399999995],
 [94137100084000045L, 94, 347.45524799999998, -0.829750074]]


But the only examples I have found for creating an inhomogeneous
array with fields involves lists of tuples.  e.g.

>>> mydescriptor = {'names': ('gender','age','weight'), 'formats':('S1',
>>> 'f4', 'f4')}
>>> a = array([('M',64.0,75.0),('F',25.0,60.0)], dtype=mydescriptor)

Trying something like this with a list of lists results in
the following error:

TypeError: expected a readable buffer object

Now I could create the array and run a loop, copying
in, but this would be less efficient.  Is there a way
to do this in one step?

Thanks,
Erin


-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From erin.sheldon at gmail.com  Sun Nov 12 19:09:18 2006
From: erin.sheldon at gmail.com (Erin Sheldon)
Date: Sun, 12 Nov 2006 19:09:18 -0500
Subject: array from list of lists
In-Reply-To: <20061112235629.GB5596@66-234-56-83.nyc.cable.nyct.net>
References: <20061112235629.GB5596@66-234-56-83.nyc.cable.nyct.net>
Message-ID: <20061113000918.GC5596@66-234-56-83.nyc.cable.nyct.net>

I have to not ammend my statement a bit:
  
DBI 2.0 actually returns a lists of tuples, which would
work.  It appears to just be pgdb, the postgres interface,
that is returning lists of lists.  Still, I need to
interact with this database.

Erin

On Sun, Nov 12, 2006 at 06:56:29PM -0500, Erin Sheldon wrote:
> Hi all-
> 
> I want to take the result from a database query,
> and create a numpy array with field names and types 
> corresponding to the returned columns.
> 
> The DBI 2.0 compliant interfaces return lists
> of lists. E.g.
> 
> 
> [[94137100072000193L, 94, 345.57215100000002, -0.83673208099999996],
>  [94137100072000368L, 94, 345.60217299999999, -0.83766954299999996],
>  ....
>  [94137100083000157L, 94, 347.21668099999999, -0.83572582399999995],
>  [94137100084000045L, 94, 347.45524799999998, -0.829750074]]
> 
> 
> But the only examples I have found for creating an inhomogeneous
> array with fields involves lists of tuples.  e.g.
> 
> >>> mydescriptor = {'names': ('gender','age','weight'), 'formats':('S1',
> >>> 'f4', 'f4')}
> >>> a = array([('M',64.0,75.0),('F',25.0,60.0)], dtype=mydescriptor)
> 
> Trying something like this with a list of lists results in
> the following error:
> 
> TypeError: expected a readable buffer object
> 
> Now I could create the array and run a loop, copying
> in, but this would be less efficient.  Is there a way
> to do this in one step?
> 
> Thanks,
> Erin
> 

-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From pgmdevlist at gmail.com  Sun Nov 12 19:14:29 2006
From: pgmdevlist at gmail.com (Pierre GM)
Date: Sun, 12 Nov 2006 19:14:29 -0500
Subject: array from list of lists
In-Reply-To: <20061112235629.GB5596@66-234-56-83.nyc.cable.nyct.net>
References: <20061112235629.GB5596@66-234-56-83.nyc.cable.nyct.net>
Message-ID: <200611121914.30182.pgmdevlist@gmail.com>


You could try the fromarrays function of numpy.core.records

>>> mydescriptor = {'names': (a','b','c','d'), 'formats':('f4', 'f4', 'f4', 
'f4')}
>>> a = N.core.records.fromarrays(N.transpose(yourlist), dtype=mydescriptor)

The 'transpose' function ensures that 'fromarrays' sees 4 arrays (one for each 
column).

-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From erin.sheldon at gmail.com  Sun Nov 12 19:50:39 2006
From: erin.sheldon at gmail.com (Erin Sheldon)
Date: Sun, 12 Nov 2006 19:50:39 -0500
Subject: array from list of lists
In-Reply-To: <200611121914.30182.pgmdevlist@gmail.com>
References: <20061112235629.GB5596@66-234-56-83.nyc.cable.nyct.net>
	<200611121914.30182.pgmdevlist@gmail.com>
Message-ID: <331116dc0611121650m812119es1c4381d7d4faa2ce@mail.gmail.com>

On 11/12/06, Pierre GM <pgmdevlist at gmail.com> wrote:
>
> You could try the fromarrays function of numpy.core.records
>
> >>> mydescriptor = {'names': (a','b','c','d'), 'formats':('f4', 'f4', 'f4',
> 'f4')}
> >>> a = N.core.records.fromarrays(N.transpose(yourlist), dtype=mydescriptor)
>
> The 'transpose' function ensures that 'fromarrays' sees 4 arrays (one for each
> column).
>

That worked as advertised. Thanks.
Erin

-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From erin.sheldon at gmail.com  Sun Nov 12 20:10:17 2006
From: erin.sheldon at gmail.com (Erin Sheldon)
Date: Sun, 12 Nov 2006 20:10:17 -0500
Subject: array from list of lists
In-Reply-To: <331116dc0611121650m812119es1c4381d7d4faa2ce@mail.gmail.com>
References: <20061112235629.GB5596@66-234-56-83.nyc.cable.nyct.net>
	<200611121914.30182.pgmdevlist@gmail.com>
	<331116dc0611121650m812119es1c4381d7d4faa2ce@mail.gmail.com>
Message-ID: <331116dc0611121710k7cc2c965pbec26543224e8dc9@mail.gmail.com>

On 11/12/06, Erin Sheldon <erin.sheldon at gmail.com> wrote:
> On 11/12/06, Pierre GM <pgmdevlist at gmail.com> wrote:
> >
> > You could try the fromarrays function of numpy.core.records
> >
> > >>> mydescriptor = {'names': (a','b','c','d'), 'formats':('f4', 'f4', 'f4',
> > 'f4')}
> > >>> a = N.core.records.fromarrays(N.transpose(yourlist), dtype=mydescriptor)
> >
> > The 'transpose' function ensures that 'fromarrays' sees 4 arrays (one for each
> > column).

Actually, there is a problem with that approach.  It first converts
the entire array to a single type, by default a floating type.  For
very large integers this precision is insufficient.  For example, I
have the following integer in my arrays:
 94137100072000193L
which ends up as
  94137100072000192
after going to a float and then back to an integer.

Erin

-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From charlesr.harris at gmail.com  Sun Nov 12 20:15:10 2006
From: charlesr.harris at gmail.com (Charles R Harris)
Date: Sun, 12 Nov 2006 18:15:10 -0700
Subject: array from list of lists
In-Reply-To: <331116dc0611121710k7cc2c965pbec26543224e8dc9@mail.gmail.com>
References: <20061112235629.GB5596@66-234-56-83.nyc.cable.nyct.net>
	<200611121914.30182.pgmdevlist@gmail.com>
	<331116dc0611121650m812119es1c4381d7d4faa2ce@mail.gmail.com>
	<331116dc0611121710k7cc2c965pbec26543224e8dc9@mail.gmail.com>
Message-ID: <e06186140611121715m4e84de49he4c65c2c69bc73de@mail.gmail.com>

On 11/12/06, Erin Sheldon <erin.sheldon at gmail.com> wrote:
>
> On 11/12/06, Erin Sheldon <erin.sheldon at gmail.com> wrote:
> > On 11/12/06, Pierre GM <pgmdevlist at gmail.com> wrote:
> > >
> > > You could try the fromarrays function of numpy.core.records
> > >
> > > >>> mydescriptor = {'names': (a','b','c','d'), 'formats':('f4', 'f4',
> 'f4',
> > > 'f4')}
> > > >>> a = N.core.records.fromarrays(N.transpose(yourlist),
> dtype=mydescriptor)
> > >
> > > The 'transpose' function ensures that 'fromarrays' sees 4 arrays (one
> for each
> > > column).
>
> Actually, there is a problem with that approach.  It first converts
> the entire array to a single type, by default a floating type.  For
> very large integers this precision is insufficient.  For example, I
> have the following integer in my arrays:
> 94137100072000193L
> which ends up as
>   94137100072000192
> after going to a float and then back to an integer.


Out of curiosity, where does that large integer come from?

Chuck
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20061112/4a0a3969/attachment-0001.html>
-------------- next part --------------
-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
-------------- next part --------------
_______________________________________________
Numpy-discussion mailing list
Numpy-discussion at lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/numpy-discussion

From erin.sheldon at gmail.com  Sun Nov 12 20:20:29 2006
From: erin.sheldon at gmail.com (Erin Sheldon)
Date: Sun, 12 Nov 2006 20:20:29 -0500
Subject: array from list of lists
In-Reply-To: <e06186140611121715m4e84de49he4c65c2c69bc73de@mail.gmail.com>
References: <20061112235629.GB5596@66-234-56-83.nyc.cable.nyct.net>
	<200611121914.30182.pgmdevlist@gmail.com>
	<331116dc0611121650m812119es1c4381d7d4faa2ce@mail.gmail.com>
	<331116dc0611121710k7cc2c965pbec26543224e8dc9@mail.gmail.com>
	<e06186140611121715m4e84de49he4c65c2c69bc73de@mail.gmail.com>
Message-ID: <331116dc0611121720o3fa53d53xfca405f62f763d14@mail.gmail.com>

On 11/12/06, Charles R Harris <charlesr.harris at gmail.com> wrote:
> > 94137100072000193L
> > which ends up as
> >   94137100072000192
> > after going to a float and then back to an integer.
>
> Out of curiosity, where does that large integer come from?

It is a unique object identifier.  It is a combination of various
numbers related to an astronomical object detected by
the sloan digital sky survey. (www.sdss.org).

cheers,
Erin

-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From jerald433 at brazosnet.com  Sun Nov 12 21:15:23 2006
From: jerald433 at brazosnet.com (=?windows-1251?B?zOXm5PPt4PDu5O375SDx8uDt5ODw8vsg4fP14+Dr8uXx6u7j7iDz9+Xy4A==?=)
Date: Mon, 13 Nov 2006 05:15:23 +0300
Subject: =?windows-1251?b?z3B17OXt5e115SDs5ebkee3gcO7k?=
 =?windows-1251?b?7ft4IPHy4O3k4HDy4Hgg9HXt4O3x7uIw6SDu8vfl8u3u8fJ1IHUg4Xl4?=
 =?windows-1251?b?4+Dr8uVw8eow4zAgeffl8uA=?=
Message-ID: <cc2f01c706c9$035c31cc$f4e28aef@brazosnet.com>

 
     ????y??p????? c?????p?? ?y??????pc???? y????.
      0????o??e?ue c ??u??u???u ?o??u?o???u? ?u???co?o? o??e??oc?u ?o M??0
(M???) u ?o????u ??e?c????e?u? o??e??oc?u. ??uo??e?e?ue ?????u?ec?ux ?????o?
y?e?? o??e????x ?u?o? ???u?o? u o?????e??c?? ? coo??e?c??uu c M???. ??u?e??
coc????e?u? ?u???co??x o??e?o?. 
         ???? ??o?e?e?u?:   14 - I5 Ho??p?, ?o 8 ????e?u?ec?ux ??co? ?
?e??, c ?e?e???o? ?? o?e? u ?o?e-?????. 
  Mec?o ??o?e?e?u?:  ?. Moc???   Me?o?u?ec?ue u ??????o???e ???e?u???,
o?e??-?o?e-?????u ?????e?? ? c?ou?oc??. 
    ?y?c o?ue??u?o??? ?? ?y?o?o?u?e?e? ?u???co?o-??o?o?u?ec?ux c?y?? o????u???u?,
??????x ?yx????e?o?, ? ????e ?cex ?e????ux ?o?y?u?? ?????u?ec?ue ????u?
?o ??u?e?e?u? M??0. 
   ?o?e????ue: 
     0??ee ????o?c??o c Me??y???o????u c?????????u ?u???co?o? o??e??oc?u
u ?yx????e?c?o?o y?e??. ??u??u?? ?o??u?o???u? c???????o? u c??y??y?? c???????o?.
?uc?e?? ?o????u???x ?o?y?e??o? ?o?e?? ?o M??0: 
  0??ue ??u??u?? (IFRS Frameworks) 
 Me??y???o???e c???????? ?u???co?o? o??e??oc?u (M??0 / IFRS) 
 Me??y???o???e c???????? ?yx????e?c?o?o y?e?? (M??? / IAS) 
 ???e???e???uu ? c?????????, ???y?e???e ?o?u?e?o? ?o u??e???e???uu ?e??y???o??o?
?u???co?o? o??e??oc?u (IFRIC) 
 ???e???e???uu ? c????????? ?oc?o???o?o ?o?u?e?? ?o u??e???e???uu (SIC)

 ??oe??? ?o?y?e??o? 
 ?uc?yccuo???e ???e?u??? o ?oo??o?e?ue M??0 u US GAAP 
  ??e?e??? ?u???co?o? o??e??oc?u ?o M??0 (M??? 1, M??? 8, M??? 10, M???
18): 
  ?yx????e?c?u? ?????c (Balance Sheet) 
 0??e? o ??u????x u y?????x (ProfitLoss Statement) 
 0??e? o ??u?e?uu ?e?e???x c?e?c?? (Cash Flow Statement) 
 0??e? o? u??e?e?uu co?c??e??o?o ???u???? (Changes of Equity Statement)

 ??u?e???u? / ?o?c?u?e????? ???uc?? ? ?u???co?o? o??e??oc?u (Disclosures)
o ??e???? ?o?u?u?? (Accounting Policies) 
  ?o??? ??e?c????e?u? ?u???co?o? o??e??oc?u 
  ?????c 
  ?uc??e ???u?? (???u?? - o?????e??c???) = ???u??? 
 ???u?? = ???u??? + 0?????e??c??? 
  0??e? o ??u????x u y?????x 
  ?o ?e?o?y x?????e?? ??cxo?o? 
 ?o ?e?o?y ?y???u? ??cxo?o? 
  0??e? o ??u?e?uu ?e?e???x c?e?c?? 
  ????o? ?e?o? 
 ?oc?e???? ?e?o? 
  0??e? o? u??e?e?uu co?c??e??o?o ???u???? 
  0c?o??o? ?o?xo? ("????u?? u??e?e?u? ?e?e??o?") 
 ????e????u???? ?o?xo? (???e?u?o?????? c ??c????u??u ? ?o?c?u?e???o? ???uc?e)

  ??u?e? ?o???e??? ?o?? ?u???co?o? o??e??oc?u 
  T?e?o???u? M??0 ??u ?o??u?o???uu ?u???co?o? o??e??oc?u ?o ?e??y???o????
c????????? ??e???e. M??0 1 
 ??e? o??e????x ?u?o? ???u?o? u o?????e??c?? 
  0??ue ??u??u?? y?e?? 
 ?o???o? ??u????u? u o?e??u o??e????x ?u?o? ???u?o? u o?????e??c?? 
 ?????e?ue c ?occu?c?u?u c?????????u ?yx????e?c?o?o y?e?? 
 ?o???o? ??c????u? u??o????uu o? o??e????x ?u??x ???u?o? u o?????e??c??
? ?o?c?u?e???o? ???uc?e 
 ??e? ??eo?o?o???x ???u?o? 
  0c?o???e c?e?c???, u??ec?u?uo???? co?c??e??oc?? u ??eo?o?o???e ???u??,
??e???????e???e ??? ??o???u (M??? 16, M??? 40, M??0 5) 
 ??e? ?e???e?u?????x ???u?o? (M??? 38) 
 ??o??u???u? ??eo?o?o???x ???u?o? 
 ?e?eo?e??? / 0?ec?e?e?ue ??eo?o?o???x ???u?o? (M??? 36) 
  ??e? ????co? (M??? 2) 
  ?e??o????????? o?e??? 
 ?????e???? ?e?eo?e??? ????co? (M??? 36) 
 ??uc??ue ??u ??o???e u ?????uu ????co? 
  ??c?e?? c ?e?u?o???u u ??e?u?o???u 
 ??e? ????o? u ??e?u?o? (M??? 23) 
 B??y??? u ???o? ?? ??u???? 
  B??y??? (M??? 18) 
 ??e? ?y?co??x ????u? (M??? 21) 
 ?uc??? ??u???? / y???o? ?? o??e???? ?e?uo? (M??? 8) 
 ??e? ???o?? ?? ??u???? (M??? 12) 
 ??c????ue u??o????uu o co???u?x, ??ouc?e??ux ?oc?e o??e??o? ???? (M???
10) 
  ???u??? u ?e?e??? (M??? 37) 
  ??u?e?? coc????e?u? ?u???co??x o??e?o? ? coo??e?c??uu c M??0 
  ??u?e? 1: ??ou??o?c??e???? ?o????u? 
 ??u?e? 2: To??o?o-c???o??? ?o????u? 
  ??u??u?? u ?????u?ec?ue ?o?xo?? ? ????c?o????uu ?occu?c?o? ?yx????e?c?o?
o??e??oc?u ? ?u???co??e o??e?? ?o M??0 
  ?o??u?o???ue ???o?e?o ????? c?e?o? 
 T???c?o????u? o?o?o??o?o ?????c? 
 ?o???o? y?e?? ?o??e??u?o?o? ??u ??o?e?e?uu ????c?o????uu o?o?o??o?o ?????c?

 ??u?e? ????c?o????uu u ?o??u?o???u? ?u???co??x o??e?o? ? coo??e?c??uu
c M??0 ??? ?o??o?o-c???o?o? ?o????uu 
  ?o ?o??oc?? ?e?uc????uu o???????ec? ?o ?e?: (495) 2?5-58-88, (495) 2?5-I2?5
  ????
 
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20061113/2e4365b8/attachment-0001.html>

From fingerslip at yahoo.com  Sun Nov 12 22:06:24 2006
From: fingerslip at yahoo.com (Abhishek Roy)
Date: Mon, 13 Nov 2006 03:06:24 +0000 (GMT)
Subject: Compiling numpy with lapack and atlas
In-Reply-To: <ej89qu$o7o$1@sea.gmane.org>
References: <ej89qu$o7o$1@sea.gmane.org>
Message-ID: <20061113030624.18642.qmail@web38903.mail.mud.yahoo.com>

--- Robert Kern <robert.kern at gmail.com> wrote:
> Also add g2c to your library list, then. It is g77's FORTRAN runtime library
Okay I put libg2c.a in the atlas directory and made the line in site.cfg,
atlas_libs = lapack, f77blas, cblas, atlas, g2c

Now numpy imports but numpy.linalg.test() as well as any linalg function gives,
>>> numpy.linalg.test()
  Found 32 tests for numpy.linalg
  Found 0 tests for __main__
.. ** On entry to DGEEV  parameter number 13 had an illegal value

I am using prebuilt a lapack library from netlib.org. 

Thanks for all your help,
Abhishek

Send instant messages to your online friends http://uk.messenger.yahoo.com 

-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From tim.hochberg at ieee.org  Sun Nov 12 22:07:50 2006
From: tim.hochberg at ieee.org (Tim Hochberg)
Date: Sun, 12 Nov 2006 20:07:50 -0700
Subject: array from list of lists
In-Reply-To: <331116dc0611121710k7cc2c965pbec26543224e8dc9@mail.gmail.com>
References: <20061112235629.GB5596@66-234-56-83.nyc.cable.nyct.net>	<200611121914.30182.pgmdevlist@gmail.com>	<331116dc0611121650m812119es1c4381d7d4faa2ce@mail.gmail.com>
	<331116dc0611121710k7cc2c965pbec26543224e8dc9@mail.gmail.com>
Message-ID: <4557E186.6030307@ieee.org>

Erin Sheldon wrote:
> On 11/12/06, Erin Sheldon <erin.sheldon at gmail.com> wrote:
>   
>> On 11/12/06, Pierre GM <pgmdevlist at gmail.com> wrote:
>>     
>>> You could try the fromarrays function of numpy.core.records
>>>
>>>       
>>>>>> mydescriptor = {'names': (a','b','c','d'), 'formats':('f4', 'f4', 'f4',
>>>>>>             
>>> 'f4')}
>>>       
>>>>>> a = N.core.records.fromarrays(N.transpose(yourlist), dtype=mydescriptor)
>>>>>>             
>>> The 'transpose' function ensures that 'fromarrays' sees 4 arrays (one for each
>>> column).
>>>       
>
> Actually, there is a problem with that approach.  It first converts
> the entire array to a single type, by default a floating type.  For
> very large integers this precision is insufficient.  For example, I
> have the following integer in my arrays:
>  94137100072000193L
> which ends up as
>   94137100072000192
> after going to a float and then back to an integer.
>   

I haven't been following this too closely, but if you need to transpose 
your data without converting all to one type, I can think of a couple of 
different approaches:

    1.  zip(*yourlist)
    2. numpy.transpose(numpy.array(yourlist, dtype=object)

I haven't tested them though (particularly the second one), so caveat 
emptor, etc, etc.

-tim


-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From erin.sheldon at gmail.com  Sun Nov 12 22:17:17 2006
From: erin.sheldon at gmail.com (Erin Sheldon)
Date: Sun, 12 Nov 2006 22:17:17 -0500
Subject: array from list of lists
In-Reply-To: <4557E186.6030307@ieee.org>
References: <20061112235629.GB5596@66-234-56-83.nyc.cable.nyct.net>
	<200611121914.30182.pgmdevlist@gmail.com>
	<331116dc0611121650m812119es1c4381d7d4faa2ce@mail.gmail.com>
	<331116dc0611121710k7cc2c965pbec26543224e8dc9@mail.gmail.com>
	<4557E186.6030307@ieee.org>
Message-ID: <331116dc0611121917oe71d9bsc8fd684e975a014@mail.gmail.com>

On 11/12/06, Tim Hochberg <tim.hochberg at ieee.org> wrote:
> I haven't been following this too closely, but if you need to transpose
> your data without converting all to one type, I can think of a couple of
> different approaches:
>
>     1.  zip(*yourlist)
>     2. numpy.transpose(numpy.array(yourlist, dtype=object)
>
> I haven't tested them though (particularly the second one), so caveat
> emptor, etc, etc.

Its not that I want to transpose data.

I'm trying to convert the output of a pgdb postgres query into
an array with fields and types corresponding to the columns
I have selected.  The problem is pgdb does not return a list
of tuples as it should according to DB 2.0, but instead
a list of lists.   So numpy.array(lol, dtype=) fails, and so will your
solution #2.   I don't want to copy the data more than once
obviously, so I'm looking for a way to call array() with a lists
of lists.

Erin

-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From peridot.faceted at gmail.com  Sun Nov 12 22:18:19 2006
From: peridot.faceted at gmail.com (A. M. Archibald)
Date: Sun, 12 Nov 2006 22:18:19 -0500
Subject: array from list of lists
In-Reply-To: <331116dc0611121710k7cc2c965pbec26543224e8dc9@mail.gmail.com>
References: <20061112235629.GB5596@66-234-56-83.nyc.cable.nyct.net>
	<200611121914.30182.pgmdevlist@gmail.com>
	<331116dc0611121650m812119es1c4381d7d4faa2ce@mail.gmail.com>
	<331116dc0611121710k7cc2c965pbec26543224e8dc9@mail.gmail.com>
Message-ID: <ce557a360611121918y612ce38fu68c51fbc20546dc1@mail.gmail.com>

On 12/11/06, Erin Sheldon <erin.sheldon at gmail.com> wrote:

> Actually, there is a problem with that approach.  It first converts
> the entire array to a single type, by default a floating type.  For
> very large integers this precision is insufficient.  For example, I
> have the following integer in my arrays:
>  94137100072000193L
> which ends up as
>   94137100072000192
> after going to a float and then back to an integer.

That's an unfortunate limitation of numpy; it views double-precision
floats as higher precision than 64-bit integers, but of course they
aren't. If you want to put all your data in a record array, you could
try transposing the lists using a list comprehension - numpy is not
always as much faster than pure python as it looks. You could then
convert that to a list of four arrays and do the assignment as
appropriate. Alternatively, you could convert your array into a
higher-precision floating-point format (if one is available on your
machine) before transposing and storing in a record array.

A. M. Archibald

-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From george.sakkis at gmail.com  Sun Nov 12 22:25:05 2006
From: george.sakkis at gmail.com (George Sakkis)
Date: Mon, 13 Nov 2006 03:25:05 -0000
Subject: Equivalent to list.index ?
References: <1163358985.920640.306720@k70g2000cwa.googlegroups.com>
	<f4f93d420611121121y2620c6fdj60241acef01e30b4@mail.gmail.com>
	<1163362743.775427.260100@f16g2000cwb.googlegroups.com>
	<4557949E.5060009@ieee.org>
Message-ID: <1163388305.110953.216520@m7g2000cwm.googlegroups.com>

Tim Hochberg wrote:

> George Sakkis wrote:
> > def index(a, value):
> >     return N.where(a==value)[0][0]
> >
> Or
>
> def index(a, value):
>     return argmax(a == value)

That's a bit easier to write and a bit harder to grok; that's ok, I can
live with it.

> > This works but seems clunky and less efficient than necessary. If there
> > isn't a better alternative, I would welcome a new index()
> > function/method in the next numpy release.
> >
> >
>
> Without commenting on the desirability of this particular function,
> people seem awfully eager to add functions and methods everywhere. These
> aren't free you know. They cost in terms of maintenance and potential
> bugs, and in terms of making the already large function set even harder
> to wrap ones head around. That's partly an organizational problem but
> partly an issue of sheer size, neither of which are helped by adding a
> bunch more stuff.
>
>     Not every two-line Python function has to come pre-written
>         --Tim Peters

I think the organizational problem is more serious in Numpy's case; any
package with about 200 functions (according to
http://www.scipy.org/Numpy_Example_List) in an almost flat namespace
would feel the maintainability problem build up.

> In the case of this particular function, what are the use cases? Are
> they common (not just with you but with other numpy users)? Are the uses
> speed critical? Is it a good building block for other numeric
> functions?  I'm skeptical of the above implementation as it doesn't fit
> well with other similar numpy functions (argmin and argmax for example)
> and I personally don't see much in the way of compelling uses for this,
> but feel free to try to convince me otherwise.
>
> -tim

As I said, I would welcome the addition of this function and I would
use it if it was there, but I'm not particularly fervent about it. If I
were to convince you, I'd argue about how numpy arrays can be
considered generalizations of (1-d) sequences in many ways and how
convenient would it be for users whose code expects python lists to
work as is (thanks to duck typing) with 1-d numpy arrays. Another
(indirect) argument would be the addition of functions that implement
set-theoretic operations, and certainly lists are far more common than
sets.

I have to agree though that the flat namespace at its current size
makes me uneasy too; I typically use "import numpy as N" instead of
"from numpy import *".

George


-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From tim.hochberg at ieee.org  Sun Nov 12 22:50:13 2006
From: tim.hochberg at ieee.org (Tim Hochberg)
Date: Sun, 12 Nov 2006 20:50:13 -0700
Subject: Equivalent to list.index ?
In-Reply-To: <1163388305.110953.216520@m7g2000cwm.googlegroups.com>
References: <1163358985.920640.306720@k70g2000cwa.googlegroups.com>	<f4f93d420611121121y2620c6fdj60241acef01e30b4@mail.gmail.com>	<1163362743.775427.260100@f16g2000cwb.googlegroups.com>	<4557949E.5060009@ieee.org>
	<1163388305.110953.216520@m7g2000cwm.googlegroups.com>
Message-ID: <4557EB75.5030505@ieee.org>

George Sakkis wrote:
> Tim Hochberg wrote:
>
>   
>> George Sakkis wrote:
>>     
>>> def index(a, value):
>>>     return N.where(a==value)[0][0]
>>>
>>>       
>> Or
>>
>> def index(a, value):
>>     return argmax(a == value)
>>     
>
> That's a bit easier to write and a bit harder to grok; that's ok, I can
> live with it.
>   
I'll add a little cautionary note as well: it won't work correctly if 
there are no elements of a that equal value (in which case the max of 
a==values is 0 and you'll just get the first values). It would be easy 
enough to make this bullet proof:

    def index(a, value):
        arg = argmax(a == value)
        if a[arg] != value:
           raise IndexError("or whatever")
        return arg


Unfortunately, that's no longer two lines and I wouldn't have been able 
to use that Tim Peter's quote which needs to be thrown out from time to 
time just on general principles.

>   
>> [snip]
>> Peters
>>     
>
> I think the organizational problem is more serious in Numpy's case; any
> package with about 200 functions (according to
> http://www.scipy.org/Numpy_Example_List) in an almost flat namespace
> would feel the maintainability problem build up.
>   
I agree with that for the most part.

>   
>> In the case of this particular function, what are the use cases? Are
>> they common (not just with you but with other numpy users)? Are the uses
>> speed critical? Is it a good building block for other numeric
>> functions?  I'm skeptical of the above implementation as it doesn't fit
>> well with other similar numpy functions (argmin and argmax for example)
>> and I personally don't see much in the way of compelling uses for this,
>> but feel free to try to convince me otherwise.
>>
>> -tim
>>     
>
> As I said, I would welcome the addition of this function and I would
> use it if it was there, but I'm not particularly fervent about it. If I
> were to convince you, I'd argue about how numpy arrays can be
> considered generalizations of (1-d) sequences in many ways and how
> convenient would it be for users whose code expects python lists to
> work as is (thanks to duck typing) with 1-d numpy arrays. Another
> (indirect) argument would be the addition of functions that implement
> set-theoretic operations, and certainly lists are far more common than
> sets.
>   
Well, if you were to try to convince me like that, I would try to 
convince you of your errant ways.by noting that numpy arrays are quite 
dissimilar from lists in so many ways, that making them closer would 
just lead to confusion as people asked for more and more list-like 
behaviors (append, in, etc etc.). Lists and arrays aren't close enough 
that conflating them could be made to work in any halfway sensible manner.

Lists and arrays are both examples of python sequences and they both 
fully implement the sequence interface (and the iterarable interface for 
that matter) but that's really the extent of their similarity.

Now whether an object based on array that implemented list semantics but 
kept a numpy arrays memory frugality would be useful I cannot say. 
However, that seems to be the kind of thing that should be a separate 
type of object, perhaps reusing much of the array code, but having a 
completely different interface. It also seems the kind of thing that 
would likely have fairly limited use in practice, so if someone really 
wants it I'd like to see it exist as a separate package for some period 
of time to verify that it actually has a user base before it got 
incorporated into numpy.

Not that I control these things.

> I have to agree though that the flat namespace at its current size
> makes me uneasy too; I typically use "import numpy as N" instead of
> "from numpy import *".
>   
Oh yeah, you should almost never use "from anything import *" IMO. 'N' 
seems to be the favored abreviation these days although I've always used 
'np' (this helped when I was transition from Numeric to numarray and 
then back to numpy as I used 'na' for numarray and np for numpy).

-tim




-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From tim.hochberg at ieee.org  Sun Nov 12 23:01:26 2006
From: tim.hochberg at ieee.org (Tim Hochberg)
Date: Sun, 12 Nov 2006 21:01:26 -0700
Subject: array from list of lists
In-Reply-To: <331116dc0611121917oe71d9bsc8fd684e975a014@mail.gmail.com>
References: <20061112235629.GB5596@66-234-56-83.nyc.cable.nyct.net>	<200611121914.30182.pgmdevlist@gmail.com>	<331116dc0611121650m812119es1c4381d7d4faa2ce@mail.gmail.com>	<331116dc0611121710k7cc2c965pbec26543224e8dc9@mail.gmail.com>	<4557E186.6030307@ieee.org>
	<331116dc0611121917oe71d9bsc8fd684e975a014@mail.gmail.com>
Message-ID: <4557EE16.3080604@ieee.org>

Erin Sheldon wrote:
> On 11/12/06, Tim Hochberg <tim.hochberg at ieee.org> wrote:
>   
>> I haven't been following this too closely, but if you need to transpose
>> your data without converting all to one type, I can think of a couple of
>> different approaches:
>>
>>     1.  zip(*yourlist)
>>     2. numpy.transpose(numpy.array(yourlist, dtype=object)
>>
>> I haven't tested them though (particularly the second one), so caveat
>> emptor, etc, etc.
>>     
>
> Its not that I want to transpose data.
>
> I'm trying to convert the output of a pgdb postgres query into
> an array with fields and types corresponding to the columns
> I have selected.  The problem is pgdb does not return a list
> of tuples as it should according to DB 2.0, but instead
> a list of lists.   So numpy.array(lol, dtype=) fails, and so will your
> solution #2. 
In that case, I suggest just using a list comprehension or map, 
[tuple(x) for x in lol] for example.

>   I don't want to copy the data more than once
> obviously, so I'm looking for a way to call array() with a lists
> of lists.
>   
It's probably pointless to worry about his. You are already allocating 
5*N python objects (all those Python floats and integers as well as the 
lists themselves). I believe the list comprehension above is only going 
to allocate an additional N objects (the new tuples). Admittedly, the 
objects aren't all the same size, but in this case they are close enough 
that I doubt it'll matter.

-tim




-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From pgmdevlist at gmail.com  Sun Nov 12 23:06:33 2006
From: pgmdevlist at gmail.com (Pierre GM)
Date: Sun, 12 Nov 2006 23:06:33 -0500
Subject: array from list of lists
In-Reply-To: <331116dc0611121710k7cc2c965pbec26543224e8dc9@mail.gmail.com>
References: <20061112235629.GB5596@66-234-56-83.nyc.cable.nyct.net>
	<331116dc0611121650m812119es1c4381d7d4faa2ce@mail.gmail.com>
	<331116dc0611121710k7cc2c965pbec26543224e8dc9@mail.gmail.com>
Message-ID: <200611122306.34072.pgmdevlist@gmail.com>

On Sunday 12 November 2006 20:10, Erin Sheldon wrote:
> Actually, there is a problem with that approach.  It first converts
> the entire array to a single type, by default a floating type.  

As A.M. Archibald suggested, you can use list comprehension:
N.array([(a,b,c,d,) for (a,b,c,d) in yourlist], dtype=yourdesc)

or

N.fromiter(((a,b,c,d) for (a,b,c,d,) in yourlist), dtype=yourdesc)

Would you mind trying that, and let us know which one works best ? That could 
be put on the wiki somewhere...


-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From robert.kern at gmail.com  Sun Nov 12 23:27:54 2006
From: robert.kern at gmail.com (Robert Kern)
Date: Sun, 12 Nov 2006 22:27:54 -0600
Subject: array from list of lists
In-Reply-To: <200611122306.34072.pgmdevlist@gmail.com>
References: <20061112235629.GB5596@66-234-56-83.nyc.cable.nyct.net>	<331116dc0611121650m812119es1c4381d7d4faa2ce@mail.gmail.com>	<331116dc0611121710k7cc2c965pbec26543224e8dc9@mail.gmail.com>
	<200611122306.34072.pgmdevlist@gmail.com>
Message-ID: <ej8s8l$1fr$1@sea.gmane.org>

Pierre GM wrote:
> On Sunday 12 November 2006 20:10, Erin Sheldon wrote:
>> Actually, there is a problem with that approach.  It first converts
>> the entire array to a single type, by default a floating type.  
> 
> As A.M. Archibald suggested, you can use list comprehension:
> N.array([(a,b,c,d,) for (a,b,c,d) in yourlist], dtype=yourdesc)
> 
> or
> 
> N.fromiter(((a,b,c,d) for (a,b,c,d,) in yourlist), dtype=yourdesc)
> 
> Would you mind trying that, and let us know which one works best ? That could 
> be put on the wiki somewhere...

N.array(map(tuple, yourlist), dtype=yourdesc)

is probably the best option.

-- 
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless enigma
 that is made terrible by our own mad attempt to interpret it as though it had
 an underlying truth."
  -- Umberto Eco


-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From erin.sheldon at gmail.com  Mon Nov 13 00:56:27 2006
From: erin.sheldon at gmail.com (Erin Sheldon)
Date: Mon, 13 Nov 2006 00:56:27 -0500
Subject: array from list of lists
In-Reply-To: <ej8s8l$1fr$1@sea.gmane.org>
References: <20061112235629.GB5596@66-234-56-83.nyc.cable.nyct.net>
	<331116dc0611121650m812119es1c4381d7d4faa2ce@mail.gmail.com>
	<331116dc0611121710k7cc2c965pbec26543224e8dc9@mail.gmail.com>
	<200611122306.34072.pgmdevlist@gmail.com> <ej8s8l$1fr$1@sea.gmane.org>
Message-ID: <331116dc0611122156m7df55cf5r939d0acab84cdeb@mail.gmail.com>

Hi all -

Thanks to everyone for the suggestions.
I think map(tuple, list) is probably the most compact,
but the list comprehension also works well.

Because map() is proably going to disappear someday, I'll
stick with the list comprehension.
  array( [tuple(row) for row in result], dtype=dtype)

That said, is there some compelling reason that the array
function doesn't support this operation?

Thanks again,
Erin

On 11/12/06, Robert Kern <robert.kern at gmail.com> wrote:
> Pierre GM wrote:
> > On Sunday 12 November 2006 20:10, Erin Sheldon wrote:
> >> Actually, there is a problem with that approach.  It first converts
> >> the entire array to a single type, by default a floating type.
> >
> > As A.M. Archibald suggested, you can use list comprehension:
> > N.array([(a,b,c,d,) for (a,b,c,d) in yourlist], dtype=yourdesc)
> >
> > or
> >
> > N.fromiter(((a,b,c,d) for (a,b,c,d,) in yourlist), dtype=yourdesc)
> >
> > Would you mind trying that, and let us know which one works best ? That could
> > be put on the wiki somewhere...
>
> N.array(map(tuple, yourlist), dtype=yourdesc)
>
> is probably the best option.
>
> --
> Robert Kern
>
> "I have come to believe that the whole world is an enigma, a harmless enigma
>  that is made terrible by our own mad attempt to interpret it as though it had
>  an underlying truth."
>   -- Umberto Eco
>
>
> -------------------------------------------------------------------------
> Using Tomcat but need to do more? Need to support web services, security?
> Get stuff done quickly with pre-integrated technology to make your job easier
> Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
> http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
> _______________________________________________
> Numpy-discussion mailing list
> Numpy-discussion at lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/numpy-discussion
>

-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From charlesr.harris at gmail.com  Mon Nov 13 01:18:22 2006
From: charlesr.harris at gmail.com (Charles R Harris)
Date: Sun, 12 Nov 2006 23:18:22 -0700
Subject: array from list of lists
In-Reply-To: <331116dc0611122156m7df55cf5r939d0acab84cdeb@mail.gmail.com>
References: <20061112235629.GB5596@66-234-56-83.nyc.cable.nyct.net>
	<331116dc0611121650m812119es1c4381d7d4faa2ce@mail.gmail.com>
	<331116dc0611121710k7cc2c965pbec26543224e8dc9@mail.gmail.com>
	<200611122306.34072.pgmdevlist@gmail.com> <ej8s8l$1fr$1@sea.gmane.org>
	<331116dc0611122156m7df55cf5r939d0acab84cdeb@mail.gmail.com>
Message-ID: <e06186140611122218i47032882r95a20b789ae2630a@mail.gmail.com>

On 11/12/06, Erin Sheldon <erin.sheldon at gmail.com> wrote:
>
> Hi all -
>
> Thanks to everyone for the suggestions.
> I think map(tuple, list) is probably the most compact,
> but the list comprehension also works well.
>
> Because map() is proably going to disappear someday, I'll
> stick with the list comprehension.
>   array( [tuple(row) for row in result], dtype=dtype)
>
> That said, is there some compelling reason that the array
> function doesn't support this operation?


My understanding is that the array needs to be allocated up front. Since the
list comprehension is iterative it is impossible to know how big the result
is going to be.

BTW, it might be possible to use fromfile('name', dtype=dtype) to do what
you want if the data is stored by rows in a file.

Chuck
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20061112/591667ae/attachment-0001.html>
-------------- next part --------------
-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
-------------- next part --------------
_______________________________________________
Numpy-discussion mailing list
Numpy-discussion at lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/numpy-discussion

From erin.sheldon at gmail.com  Mon Nov 13 02:07:29 2006
From: erin.sheldon at gmail.com (Erin Sheldon)
Date: Mon, 13 Nov 2006 02:07:29 -0500
Subject: array from list of lists
In-Reply-To: <e06186140611122218i47032882r95a20b789ae2630a@mail.gmail.com>
References: <20061112235629.GB5596@66-234-56-83.nyc.cable.nyct.net>
	<331116dc0611121650m812119es1c4381d7d4faa2ce@mail.gmail.com>
	<331116dc0611121710k7cc2c965pbec26543224e8dc9@mail.gmail.com>
	<200611122306.34072.pgmdevlist@gmail.com> <ej8s8l$1fr$1@sea.gmane.org>
	<331116dc0611122156m7df55cf5r939d0acab84cdeb@mail.gmail.com>
	<e06186140611122218i47032882r95a20b789ae2630a@mail.gmail.com>
Message-ID: <331116dc0611122307p23705f5fjd2a038812d824044@mail.gmail.com>

On 11/13/06, Charles R Harris <charlesr.harris at gmail.com> wrote:
>
>
> On 11/12/06, Erin Sheldon <erin.sheldon at gmail.com> wrote:
> > Hi all -
> >
> > Thanks to everyone for the suggestions.
> > I think map(tuple, list) is probably the most compact,
> > but the list comprehension also works well.
> >
> > Because map() is proably going to disappear someday, I'll
> > stick with the list comprehension.
> >   array( [tuple(row) for row in result], dtype=dtype)
> >
> > That said, is there some compelling reason that the array
> > function doesn't support this operation?
>
> My understanding is that the array needs to be allocated up front. Since the
> list comprehension is iterative it is impossible to know how big the result
> is going to be.

Isn't it the same with a list of tuples?  But you can send that directly to the
array constructor.  I don't see the fundamental difference, except that the
code might be simpler to write.

>
> BTW, it might be possible to use fromfile('name', dtype=dtype) to do what
> you want if the data is stored by rows in a file.

I'm reading from a database.
Erin

-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From faltet at carabos.com  Mon Nov 13 03:19:17 2006
From: faltet at carabos.com (Francesc Altet)
Date: Mon, 13 Nov 2006 09:19:17 +0100
Subject: array from list of lists
In-Reply-To: <331116dc0611122307p23705f5fjd2a038812d824044@mail.gmail.com>
References: <20061112235629.GB5596@66-234-56-83.nyc.cable.nyct.net>
	<331116dc0611121650m812119es1c4381d7d4faa2ce@mail.gmail.com>
	<331116dc0611121710k7cc2c965pbec26543224e8dc9@mail.gmail.com>
	<200611122306.34072.pgmdevlist@gmail.com> <ej8s8l$1fr$1@sea.gmane.org>
	<331116dc0611122156m7df55cf5r939d0acab84cdeb@mail.gmail.com>
	<e06186140611122218i47032882r95a20b789ae2630a@mail.gmail.com>
	<331116dc0611122307p23705f5fjd2a038812d824044@mail.gmail.com>
Message-ID: <1163405957.4221.25.camel@localhost.localdomain>

El dl 13 de 11 del 2006 a les 02:07 -0500, en/na Erin Sheldon va
escriure:
> On 11/13/06, Charles R Harris <charlesr.harris at gmail.com> wrote:
> >
> >
> > On 11/12/06, Erin Sheldon <erin.sheldon at gmail.com> wrote:
> > > Hi all -
> > >
> > > Thanks to everyone for the suggestions.
> > > I think map(tuple, list) is probably the most compact,
> > > but the list comprehension also works well.
> > >
> > > Because map() is proably going to disappear someday, I'll
> > > stick with the list comprehension.
> > >   array( [tuple(row) for row in result], dtype=dtype)
> > >
> > > That said, is there some compelling reason that the array
> > > function doesn't support this operation?
> >
> > My understanding is that the array needs to be allocated up front. Since the
> > list comprehension is iterative it is impossible to know how big the result
> > is going to be.
> 
> Isn't it the same with a list of tuples?  But you can send that directly to the
> array constructor.  I don't see the fundamental difference, except that the
> code might be simpler to write.

I think that the correct explanation is that Travis has chosen a tuple
as the way to refer to a inhomogeneous list of values (a record) and a
list as the way to refer to homogenous list of values. I'm not
completely sure why he did this, but I guess the reason was to be able
to distinguish the records in scenarios where nested records do appear.

In any case, you can also use rec.fromrecords for build recarrays from
lists of lists. This breaks the aforementioned rule, but Travis allowed
this because rec.* had to mimic numarray behaviour as much as possible.
Here is an example of use:

In [46]:mydescriptor = {'names': ('gender','age','weight'),
'formats':('S1','f4', 'f4')}
In [47]:results=[['M',64.0,75.0],['F',25.0,60.0]]
In [48]:a = numpy.rec.fromrecords(results, dtype=mydescriptor)
In [49]:b = numpy.array([tuple(row) for row in results],
dtype=mydescriptor)
In [50]:a==b
Out[50]:recarray([True, True], dtype=bool)

OTOH, it is said in the docs that fromrecords is discouraged because it
is somewhat slow, but apparently it has similar performance than using
comprehensions lists:

In [51]:Timer("numpy.rec.fromrecords(results, dtype=mydescriptor)",
"import numpy; results = [['M',64.0,75.0]]*10000; mydescriptor =
{'names': ('gender','age','weight'), 'formats':('S1','f4',
'f4')}").repeat(3,10)
Out[51]:[0.44204592704772949, 0.43584394454956055, 0.50145101547241211]

In [52]:Timer("numpy.array([tuple(row) for row in results],
dtype=mydescriptor)", "import numpy; results = [['M',64.0,75.0]]*10000;
mydescriptor = {'names': ('gender','age','weight'),
'formats':('S1','f4', 'f4')}").repeat(3,10)
Out[52]:[0.49885106086730957, 0.4325258731842041, 0.43297886848449707]


HTH,

-- 
Francesc Altet    |  Be careful about using the following code --
Carabos Coop. V.  |  I've only proven that it works, 
www.carabos.com   |  I haven't tested it. -- Donald Knuth


-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From svetosch at gmx.net  Mon Nov 13 05:02:47 2006
From: svetosch at gmx.net (Sven Schreiber)
Date: Mon, 13 Nov 2006 11:02:47 +0100
Subject: Could numpy.matlib.nanmax return a matrix?
In-Reply-To: <200611121808.51298.pgmdevlist@gmail.com>
References: <f4f93d420611121357o5dda5950g4fdcdc2f657ee654@mail.gmail.com>	<ce557a360611121408k6bae03fds9b2e979d786bb74e@mail.gmail.com>
	<200611121808.51298.pgmdevlist@gmail.com>
Message-ID: <455842C7.3050604@gmx.net>

Pierre GM schrieb:
> On Sunday 12 November 2006 17:08, A. M. Archibald wrote:
>> On 12/11/06, Keith Goodman <kwgoodman at gmail.com> wrote:
>>> Is anybody interested in making x.max() and nanmax() behave the same
>>> for matrices, except for the NaN part? That is, make
>>> numpy.matlib.nanmax return a matrix instead of an array.
> 
> Or, you could use masked arrays... In the new implementation, you can add a 
> mask to a subclassed array (such as matrix) to get a regular masked array. If 
> you fill this masked array, you get an array of the same subclass.
> 

That is very interesting, but I agree with Keith and would actually call
this a bug. (If still present in 1.0, that is, haven't checked, I think
Keith used some rc?.) One proclaimed goal of numpy for the 1.0 release
has been to be as matrix-friendly as possible, for which I am very
grateful.

Still, the use of masked arrays looks more attractive every time they're
mentioned...

-sven

-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From tim.hochberg at ieee.org  Mon Nov 13 10:03:33 2006
From: tim.hochberg at ieee.org (Tim Hochberg)
Date: Mon, 13 Nov 2006 08:03:33 -0700
Subject: array from list of lists
In-Reply-To: <1163405957.4221.25.camel@localhost.localdomain>
References: <20061112235629.GB5596@66-234-56-83.nyc.cable.nyct.net>	<331116dc0611121650m812119es1c4381d7d4faa2ce@mail.gmail.com>	<331116dc0611121710k7cc2c965pbec26543224e8dc9@mail.gmail.com>	<200611122306.34072.pgmdevlist@gmail.com>
	<ej8s8l$1fr$1@sea.gmane.org>	<331116dc0611122156m7df55cf5r939d0acab84cdeb@mail.gmail.com>	<e06186140611122218i47032882r95a20b789ae2630a@mail.gmail.com>	<331116dc0611122307p23705f5fjd2a038812d824044@mail.gmail.com>
	<1163405957.4221.25.camel@localhost.localdomain>
Message-ID: <45588945.7030703@ieee.org>

Francesc Altet wrote:
> El dl 13 de 11 del 2006 a les 02:07 -0500, en/na Erin Sheldon va
> escriure:
>   
>> On 11/13/06, Charles R Harris <charlesr.harris at gmail.com> wrote:
>>     
>>> On 11/12/06, Erin Sheldon <erin.sheldon at gmail.com> wrote:
>>>       
>>>> Hi all -
>>>>
>>>> Thanks to everyone for the suggestions.
>>>> I think map(tuple, list) is probably the most compact,
>>>> but the list comprehension also works well.
>>>>
>>>> Because map() is proably going to disappear someday, I'll
>>>> stick with the list comprehension.
>>>>   array( [tuple(row) for row in result], dtype=dtype)
>>>>
>>>> That said, is there some compelling reason that the array
>>>> function doesn't support this operation?
>>>>         
>>> My understanding is that the array needs to be allocated up front. Since the
>>> list comprehension is iterative it is impossible to know how big the result
>>> is going to be.
>>>       
>> Isn't it the same with a list of tuples?  But you can send that directly to the
>> array constructor.  I don't see the fundamental difference, except that the
>> code might be simpler to write.
>>     
>
> I think that the correct explanation is that Travis has chosen a tuple
> as the way to refer to a inhomogeneous list of values (a record) and a
> list as the way to refer to homogenous list of values. 
Just for the record, this is the officially blessed usage of tuple and 
lists for all of Python (by Guido himself). On the other hand, it's 
honored more in the breach than in reality. Since other factors, such as 
mutability/immutability or the mistaken belief that using tuples 
everywhere will make code noticeably faster or more memory frugal or 
something.

> I'm not
> completely sure why he did this, but I guess the reason was to be able
> to distinguish the records in scenarios where nested records do appear.
>   
I suspect that this could be made a little more forgiving, without 
loosing rigor. As long as none of the fields are objects of course in 
which case nearly all bets are off. Then again, the rule that tuple 
designate records is a lot simpler than something like tuples designate 
records, but you can use lists too, unless of course you have an object 
field in your array, in which case you really need to use tuples, except 
sometimes lists will work anyway, depending on where the object is 
fields is. So, maybe it's best just to keep it strict.

> In any case, you can also use rec.fromrecords for build recarrays from
> lists of lists. This breaks the aforementioned rule, but Travis allowed
> this because rec.* had to mimic numarray behaviour as much as possible.
> Here is an example of use:
> [SNIP]
>   

Just for completeness, I benchmarked the fromiter and map(tuple, 
results)  solutions as well. Map is fastest, followed by fromiter, list 
comprehension and then fromrecords. The differences are pretty minor 
however, so I'd stick with whatever seems clearest.

-tim


print Timer("numpy.rec.fromrecords(results, dtype=mydescriptor)",
"""import numpy; results = [['M',64.0,75.0]]*100000; mydescriptor = 
{'names': ('gender','age','weight'), 
'formats':('S1','f4','f4')}""").repeat(3,10)

print Timer("numpy.array([tuple(row) for row in results], 
dtype=mydescriptor)",
"""import numpy; results = [['M',64.0,75.0]]*100000; mydescriptor = 
{'names': ('gender','age','weight'),'formats':('S1','f4', 
'f4')}""").repeat(3,10)

print Timer("numpy.fromiter((tuple(x) for x in results), 
dtype=mydescriptor, count=len(results))",
"""import numpy; results = [['M',64.0,75.0]]*100000; mydescriptor = 
{'names': ('gender','age','weight'),'formats':('S1','f4', 
'f4')}""").repeat(3,10)

print Timer("numpy.array(map(tuple, results), dtype=mydescriptor)",
"""import numpy; results = [['M',64.0,75.0]]*100000; mydescriptor = 
{'names': ('gender','age','weight'),'formats':('S1','f4', 
'f4')}""").repeat(3,10)

===>

[1.3928521641717035, 1.3892659541925021, 1.3949996438094785]
[1.344854164425926, 1.3157404083479882, 1.3207066819944986]
[1.2768430065832401, 1.2742884919731416, 1.2736657871321633]
[1.2081393026208644, 1.2025276955590734, 1.205871416618594]


-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From erin.sheldon at gmail.com  Mon Nov 13 10:10:42 2006
From: erin.sheldon at gmail.com (Erin Sheldon)
Date: Mon, 13 Nov 2006 10:10:42 -0500
Subject: array from list of lists
In-Reply-To: <1163405957.4221.25.camel@localhost.localdomain>
References: <20061112235629.GB5596@66-234-56-83.nyc.cable.nyct.net>
	<331116dc0611121650m812119es1c4381d7d4faa2ce@mail.gmail.com>
	<331116dc0611121710k7cc2c965pbec26543224e8dc9@mail.gmail.com>
	<200611122306.34072.pgmdevlist@gmail.com> <ej8s8l$1fr$1@sea.gmane.org>
	<331116dc0611122156m7df55cf5r939d0acab84cdeb@mail.gmail.com>
	<e06186140611122218i47032882r95a20b789ae2630a@mail.gmail.com>
	<331116dc0611122307p23705f5fjd2a038812d824044@mail.gmail.com>
	<1163405957.4221.25.camel@localhost.localdomain>
Message-ID: <331116dc0611130710y15a165dxf0feaa9884818160@mail.gmail.com>

On 11/13/06, Francesc Altet <faltet at carabos.com> wrote:
> In any case, you can also use rec.fromrecords for build recarrays from
> lists of lists. This breaks the aforementioned rule, but Travis allowed
> this because rec.* had to mimic numarray behaviour as much as possible.
> Here is an example of use:
>
> In [46]:mydescriptor = {'names': ('gender','age','weight'),
> 'formats':('S1','f4', 'f4')}
> In [47]:results=[['M',64.0,75.0],['F',25.0,60.0]]
> In [48]:a = numpy.rec.fromrecords(results, dtype=mydescriptor)
> In [49]:b = numpy.array([tuple(row) for row in results],
> dtype=mydescriptor)
> In [50]:a==b
> Out[50]:recarray([True, True], dtype=bool)
>
> OTOH, it is said in the docs that fromrecords is discouraged because it
> is somewhat slow, but apparently it has similar performance than using
> comprehensions lists:
>
> In [51]:Timer("numpy.rec.fromrecords(results, dtype=mydescriptor)",
> "import numpy; results = [['M',64.0,75.0]]*10000; mydescriptor =
> {'names': ('gender','age','weight'), 'formats':('S1','f4',
> 'f4')}").repeat(3,10)
> Out[51]:[0.44204592704772949, 0.43584394454956055, 0.50145101547241211]
>
> In [52]:Timer("numpy.array([tuple(row) for row in results],
> dtype=mydescriptor)", "import numpy; results = [['M',64.0,75.0]]*10000;
> mydescriptor = {'names': ('gender','age','weight'),
> 'formats':('S1','f4', 'f4')}").repeat(3,10)
> Out[52]:[0.49885106086730957, 0.4325258731842041, 0.43297886848449707]

I checked the code.  For lists of lists it just creates the recarray
and runs a loop copying in the data row by row.

The fact that they are of similar speed is actually good news
because the list comprehension was making an
extra copy of the data in memory.  For large memory usage,
which is my case, this 50% overhead would have been an issue.

Erin

-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From skokot at po.opole.pl  Mon Nov 13 11:58:22 2006
From: skokot at po.opole.pl (Seweryn Kokot)
Date: Mon, 13 Nov 2006 17:58:22 +0100
Subject: why the difference between ipython and python
	shell?
Message-ID: <877ixzl1w1.fsf@poczta.po.opole.pl>

Hello,

Why ipython and python interactive shell give two different information?

--- ipython 
Python 2.4.4 (#2, Oct 20 2006, 00:23:25) 
Type "copyright", "credits" or "license" for more information.

IPython 0.7.2 -- An enhanced Interactive Python.
?       -> Introduction to IPython's features.
%magic  -> Information about IPython's 'magic' % functions.
help    -> Python's own help system.
object? -> Details about 'object'. ?object also works, ?? prints more.

In [1]: from scipy import linalg

In [2]: help(linalg.eig)

Help on function eig in module numpy.linalg.linalg:

eig(a)
    eig(a) returns u,v  where u is the eigenvalues and
    v is a matrix of eigenvectors with vector v[:,i] corresponds to
    eigenvalue u[i].  Satisfies the equation dot(a, v[:,i]) =
    u[i]*v[:,i]
--- ipython 

while

--- python
Python 2.4.4 (#2, Oct 20 2006, 00:23:25) 
[GCC 4.1.2 20061015 (prerelease) (Debian 4.1.1-16.1)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> from scipy import linalg
>>> help(linalg.eig)

Help on function eig in module scipy.linalg.decomp:

eig(a, b=None, left=False, right=True, overwrite_a=False, overwrite_b=False)
    Solve ordinary and generalized eigenvalue problem
    of a square matrix.
    
    Inputs:
    
      a     -- An N x N matrix.
      b     -- An N x N matrix [default is identity(N)].
      left  -- Return left eigenvectors [disabled].
      right -- Return right eigenvectors [enabled].
      overwrite_a, overwrite_b -- save space by overwriting the a and/or
                                  b matrices (both False by default)
    
    Outputs:
    
      w      -- eigenvalues [left==right==False].
      w,vr   -- w and right eigenvectors [left==False,right=True].
      w,vl   -- w and left eigenvectors [left==True,right==False].
      w,vl,vr  -- [left==right==True].
    
    Definitions:
...
--- python

Any idea?
regards
SK


-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From tim.hochberg at ieee.org  Mon Nov 13 12:45:55 2006
From: tim.hochberg at ieee.org (Tim Hochberg)
Date: Mon, 13 Nov 2006 10:45:55 -0700
Subject: array from list of lists
In-Reply-To: <45588945.7030703@ieee.org>
References: <20061112235629.GB5596@66-234-56-83.nyc.cable.nyct.net>	<331116dc0611121650m812119es1c4381d7d4faa2ce@mail.gmail.com>	<331116dc0611121710k7cc2c965pbec26543224e8dc9@mail.gmail.com>	<200611122306.34072.pgmdevlist@gmail.com>	<ej8s8l$1fr$1@sea.gmane.org>	<331116dc0611122156m7df55cf5r939d0acab84cdeb@mail.gmail.com>	<e06186140611122218i47032882r95a20b789ae2630a@mail.gmail.com>	<331116dc0611122307p23705f5fjd2a038812d824044@mail.gmail.com>	<1163405957.4221.25.camel@localhost.localdomain>
	<45588945.7030703@ieee.org>
Message-ID: <4558AF53.8060908@ieee.org>

Tim Hochberg wrote:
> [SNIP]
>>   
>>     
>
> Just for completeness, I benchmarked the fromiter and map(tuple, 
> results)  solutions as well. Map is fastest, followed by fromiter, list 
> comprehension and then fromrecords. The differences are pretty minor 
> however, so I'd stick with whatever seems clearest.
>
> -tim
>
>   
Here's one more approach that's marginally faster than the map based 
solution and also won't chew up an extra memory since it's based on from 
iter:

numpy.fromiter(itertools.imap(tuple, results), dtype=mydescriptor, 
count=len(results))


[SNIP]

-tim


-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From charlesr.harris at gmail.com  Mon Nov 13 14:13:07 2006
From: charlesr.harris at gmail.com (Charles R Harris)
Date: Mon, 13 Nov 2006 12:13:07 -0700
Subject: why the difference between ipython and
	python shell?
In-Reply-To: <877ixzl1w1.fsf@poczta.po.opole.pl>
References: <877ixzl1w1.fsf@poczta.po.opole.pl>
Message-ID: <e06186140611131113v10c07711s49a3e69e09810116@mail.gmail.com>

On 11/13/06, Seweryn Kokot <skokot at po.opole.pl> wrote:
>
> Hello,
>
> Why ipython and python interactive shell give two different information?
>
> --- ipython
> Python 2.4.4 (#2, Oct 20 2006, 00:23:25)
> Type "copyright", "credits" or "license" for more information.
>
> IPython 0.7.2 -- An enhanced Interactive Python.
> ?       -> Introduction to IPython's features.
> %magic  -> Information about IPython's 'magic' % functions.
> help    -> Python's own help system.
> object? -> Details about 'object'. ?object also works, ?? prints more.
>
> In [1]: from scipy import linalg
>
> In [2]: help(linalg.eig)
>
> Help on function eig in module numpy.linalg.linalg:
>
> eig(a)
>     eig(a) returns u,v  where u is the eigenvalues and
>     v is a matrix of eigenvectors with vector v[:,i] corresponds to
>     eigenvalue u[i].  Satisfies the equation dot(a, v[:,i]) =
>     u[i]*v[:,i]
> --- ipython
>
> while
>
> --- python
> Python 2.4.4 (#2, Oct 20 2006, 00:23:25)
> [GCC 4.1.2 20061015 (prerelease) (Debian 4.1.1-16.1)] on linux2
> Type "help", "copyright", "credits" or "license" for more information.
> >>> from scipy import linalg
> >>> help(linalg.eig)
>
> Help on function eig in module scipy.linalg.decomp:
>
> eig(a, b=None, left=False, right=True, overwrite_a=False,
> overwrite_b=False)
>     Solve ordinary and generalized eigenvalue problem
>     of a square matrix.
>
>     Inputs:
>
>       a     -- An N x N matrix.
>       b     -- An N x N matrix [default is identity(N)].
>       left  -- Return left eigenvectors [disabled].
>       right -- Return right eigenvectors [enabled].
>       overwrite_a, overwrite_b -- save space by overwriting the a and/or
>                                   b matrices (both False by default)
>
>     Outputs:
>
>       w      -- eigenvalues [left==right==False].
>       w,vr   -- w and right eigenvectors [left==False,right=True].
>       w,vl   -- w and left eigenvectors [left==True,right==False].
>       w,vl,vr  -- [left==right==True].
>
>     Definitions:
> ...
> --- python
>
> Any idea?


I expect scipy.linalg and numpy.linalg are different modules containing
different functions. That said, the documentation in scipy.linalg looks
quite a bit more complete.

Chuck
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20061113/5f3d8c77/attachment-0001.html>
-------------- next part --------------
-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
-------------- next part --------------
_______________________________________________
Numpy-discussion mailing list
Numpy-discussion at lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/numpy-discussion

From erin.sheldon at gmail.com  Mon Nov 13 14:43:20 2006
From: erin.sheldon at gmail.com (Erin Sheldon)
Date: Mon, 13 Nov 2006 14:43:20 -0500
Subject: array from list of lists
In-Reply-To: <4558AF53.8060908@ieee.org>
References: <20061112235629.GB5596@66-234-56-83.nyc.cable.nyct.net>
	<331116dc0611121710k7cc2c965pbec26543224e8dc9@mail.gmail.com>
	<200611122306.34072.pgmdevlist@gmail.com> <ej8s8l$1fr$1@sea.gmane.org>
	<331116dc0611122156m7df55cf5r939d0acab84cdeb@mail.gmail.com>
	<e06186140611122218i47032882r95a20b789ae2630a@mail.gmail.com>
	<331116dc0611122307p23705f5fjd2a038812d824044@mail.gmail.com>
	<1163405957.4221.25.camel@localhost.localdomain>
	<45588945.7030703@ieee.org> <4558AF53.8060908@ieee.org>
Message-ID: <331116dc0611131143h7e6e3606ra71646e9d0ded078@mail.gmail.com>

On 11/13/06, Tim Hochberg <tim.hochberg at ieee.org> wrote:
> Here's one more approach that's marginally faster than the map based
> solution and also won't chew up an extra memory since it's based on from
> iter:
>
> numpy.fromiter(itertools.imap(tuple, results), dtype=mydescriptor,
> count=len(results))

Yes, this is what I need.  BTW, there is no doc string for this.  I just added
an example the the Numpy Example List.

Thanks,
Erin

-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From myeates at jpl.nasa.gov  Mon Nov 13 16:25:29 2006
From: myeates at jpl.nasa.gov (Mathew Yeates)
Date: Mon, 13 Nov 2006 13:25:29 -0800
Subject: memmap working?
Message-ID: <4558E2C9.6030007@jpl.nasa.gov>

I have a memory mapped array. When I try and assign data, my mem usage 
goes through the roof.
example: outdat[filenum,:]=outarr where outdat is memory mapped.

Anybody know how to avoid this?

Mathew



-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From tim.hochberg at ieee.org  Mon Nov 13 16:29:11 2006
From: tim.hochberg at ieee.org (Tim Hochberg)
Date: Mon, 13 Nov 2006 14:29:11 -0700
Subject: array from list of lists
In-Reply-To: <331116dc0611131143h7e6e3606ra71646e9d0ded078@mail.gmail.com>
References: <20061112235629.GB5596@66-234-56-83.nyc.cable.nyct.net>	<331116dc0611121710k7cc2c965pbec26543224e8dc9@mail.gmail.com>	<200611122306.34072.pgmdevlist@gmail.com>
	<ej8s8l$1fr$1@sea.gmane.org>	<331116dc0611122156m7df55cf5r939d0acab84cdeb@mail.gmail.com>	<e06186140611122218i47032882r95a20b789ae2630a@mail.gmail.com>	<331116dc0611122307p23705f5fjd2a038812d824044@mail.gmail.com>	<1163405957.4221.25.camel@localhost.localdomain>	<45588945.7030703@ieee.org>
	<4558AF53.8060908@ieee.org>
	<331116dc0611131143h7e6e3606ra71646e9d0ded078@mail.gmail.com>
Message-ID: <4558E3A7.1000103@ieee.org>

Erin Sheldon wrote:
> On 11/13/06, Tim Hochberg <tim.hochberg at ieee.org> wrote:
>   
>> Here's one more approach that's marginally faster than the map based
>> solution and also won't chew up an extra memory since it's based on from
>> iter:
>>
>> numpy.fromiter(itertools.imap(tuple, results), dtype=mydescriptor,
>> count=len(results))
>>     
>
> Yes, this is what I need.  BTW, there is no doc string for this. 
Yeah, I noticed that too. I swear I wrote one at one point I'm not sure 
what happened to it. Sigh.

>  I just added
> an example the the Numpy Example List.
>   
Great.

-tim




-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From stefan at sun.ac.za  Mon Nov 13 16:50:13 2006
From: stefan at sun.ac.za (Stefan van der Walt)
Date: Mon, 13 Nov 2006 23:50:13 +0200
Subject: array from list of lists
In-Reply-To: <4558E3A7.1000103@ieee.org>
References: <200611122306.34072.pgmdevlist@gmail.com>
	<ej8s8l$1fr$1@sea.gmane.org>
	<331116dc0611122156m7df55cf5r939d0acab84cdeb@mail.gmail.com>
	<e06186140611122218i47032882r95a20b789ae2630a@mail.gmail.com>
	<331116dc0611122307p23705f5fjd2a038812d824044@mail.gmail.com>
	<1163405957.4221.25.camel@localhost.localdomain>
	<45588945.7030703@ieee.org> <4558AF53.8060908@ieee.org>
	<331116dc0611131143h7e6e3606ra71646e9d0ded078@mail.gmail.com>
	<4558E3A7.1000103@ieee.org>
Message-ID: <20061113215013.GE10559@mentat.za.net>

On Mon, Nov 13, 2006 at 02:29:11PM -0700, Tim Hochberg wrote:
> Erin Sheldon wrote:
> > On 11/13/06, Tim Hochberg <tim.hochberg at ieee.org> wrote:
> >   
> >> Here's one more approach that's marginally faster than the map based
> >> solution and also won't chew up an extra memory since it's based on from
> >> iter:
> >>
> >> numpy.fromiter(itertools.imap(tuple, results), dtype=mydescriptor,
> >> count=len(results))
> >>     
> >
> > Yes, this is what I need.  BTW, there is no doc string for this. 
> Yeah, I noticed that too. I swear I wrote one at one point I'm not sure 
> what happened to it. Sigh.

A typo slipped into add_newdocs.py.  Fixed in SVN.

Cheers
St?fan

-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From charlesr.harris at gmail.com  Mon Nov 13 17:03:38 2006
From: charlesr.harris at gmail.com (Charles R Harris)
Date: Mon, 13 Nov 2006 15:03:38 -0700
Subject: memmap working?
In-Reply-To: <4558E2C9.6030007@jpl.nasa.gov>
References: <4558E2C9.6030007@jpl.nasa.gov>
Message-ID: <e06186140611131403o3b788140m54b4c65f4954286@mail.gmail.com>

On 11/13/06, Mathew Yeates <myeates at jpl.nasa.gov> wrote:
>
> I have a memory mapped array. When I try and assign data, my mem usage
> goes through the roof.


Is it cache memory or process memory? I think a memory mapped file will keep
pages cached in memory until the space is needed so as to avoid unneeded io.
At least that is what happens in linux.

Chuck
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20061113/98b78d90/attachment-0001.html>
-------------- next part --------------
-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
-------------- next part --------------
_______________________________________________
Numpy-discussion mailing list
Numpy-discussion at lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/numpy-discussion

From myeates at jpl.nasa.gov  Mon Nov 13 17:25:41 2006
From: myeates at jpl.nasa.gov (Mathew Yeates)
Date: Mon, 13 Nov 2006 14:25:41 -0800
Subject: memmap working?
In-Reply-To: <e06186140611131403o3b788140m54b4c65f4954286@mail.gmail.com>
References: <4558E2C9.6030007@jpl.nasa.gov>
	<e06186140611131403o3b788140m54b4c65f4954286@mail.gmail.com>
Message-ID: <4558F0E5.4090108@jpl.nasa.gov>

Not sure. When I run "top" I see the line
Memory: 6016M real, 2895M free, 4174M swap in use, 2427M swap free

Its the second number that drops like a rock. Plus, it never comes back 
until I quit the program. This is a great way to turn my machine into a 
nice desk ornament!

Mathew


Charles R Harris wrote:
>
>
> On 11/13/06, *Mathew Yeates* <myeates at jpl.nasa.gov 
> <mailto:myeates at jpl.nasa.gov>> wrote:
>
>     I have a memory mapped array. When I try and assign data, my mem usage
>     goes through the roof.
>
>
> Is it cache memory or process memory? I think a memory mapped file 
> will keep pages cached in memory until the space is needed so as to 
> avoid unneeded io. At least that is what happens in linux.
>
> Chuck
>
>
> ------------------------------------------------------------------------
>
> -------------------------------------------------------------------------
> Using Tomcat but need to do more? Need to support web services, security?
> Get stuff done quickly with pre-integrated technology to make your job easier
> Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
> http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
> ------------------------------------------------------------------------
>
> _______________________________________________
> Numpy-discussion mailing list
> Numpy-discussion at lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/numpy-discussion
>   



-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From vallis.35530053 at bloglines.com  Mon Nov 13 20:32:24 2006
From: vallis.35530053 at bloglines.com (vallis.35530053 at bloglines.com)
Date: 14 Nov 2006 01:32:24 -0000
Subject: numpy.where behavior
Message-ID: <1163467944.2853004636.6699.sendItem@bloglines.com>

Using numpy 1.0, why does



>>> a = numpy.array([0.0,1.0,2.0],'d')

>>> numpy.where(a
== 0.0,1,1/a)



give the correct result, but with the warning "Warning: divide
by zero encountered in divide"?



? I thought that the point of where was
that the second expression is never used for the elements where the condition
evaluates true. 



If this is the desired behavior, is there a way to suppress
the warning?



Thanks!



Michele

-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From robert.kern at gmail.com  Mon Nov 13 20:42:40 2006
From: robert.kern at gmail.com (Robert Kern)
Date: Mon, 13 Nov 2006 19:42:40 -0600
Subject: numpy.where behavior
In-Reply-To: <1163467944.2853004636.6699.sendItem@bloglines.com>
References: <1163467944.2853004636.6699.sendItem@bloglines.com>
Message-ID: <ejb6us$i5$1@sea.gmane.org>

vallis.35530053 at bloglines.com wrote:
> ? I thought that the point of where was
> that the second expression is never used for the elements where the condition
> evaluates true. 

It is not used, but the expression still gets evaluated. There's really no way
around that.

> If this is the desired behavior, is there a way to suppress
> the warning?

In [1]: from numpy import *

In [2]: a = zeros(3)

In [3]: 1/a
Warning: divide by zero encountered in divide
Warning: invalid value encountered in double_scalars
Out[3]: array([              inf,               inf,               inf])

In [4]: seterr(divide='ignore', invalid='ignore')
Out[4]: {'divide': 'print', 'invalid': 'print', 'over': 'print', 'under': 'ignore'}

In [5]: 1/a
Out[5]: array([              inf,               inf,               inf])

In [6]: seterr?
Type:           function
Base Class:     <type 'function'>
Namespace:      Interactive
File:
/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/site-packages/numpy-1.0.1.dev3432-py2.5-macosx-10.4-i386.egg
/numpy/core/numeric.py
Definition:     seterr(all=None, divide=None, over=None, under=None, invalid=None)
Docstring:
    Set how floating-point errors are handled.

    Valid values for each type of error are the strings
    "ignore", "warn", "raise", and "call". Returns the old settings.
    If 'all' is specified, values that are not otherwise specified
    will be set to 'all', otherwise they will retain their old
    values.

    Note that operations on integer scalar types (such as int16) are
    handled like floating point, and are affected by these settings.

    Example:

    >>> seterr(over='raise')
    {'over': 'ignore', 'divide': 'ignore', 'invalid': 'ignore', 'under': 'ignore'}
    >>> seterr(all='warn', over='raise')
    {'over': 'raise', 'divide': 'ignore', 'invalid': 'ignore', 'under': 'ignore'}
    >>> int16(32000) * int16(3)
    Traceback (most recent call last):
          File "<stdin>", line 1, in ?
    FloatingPointError: overflow encountered in short_scalars
    >>> seterr(all='ignore')
    {'over': 'ignore', 'divide': 'ignore', 'invalid': 'ignore', 'under': 'ignore'}


-- 
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless enigma
 that is made terrible by our own mad attempt to interpret it as though it had
 an underlying truth."
  -- Umberto Eco


-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From charlesr.harris at gmail.com  Mon Nov 13 20:54:20 2006
From: charlesr.harris at gmail.com (Charles R Harris)
Date: Mon, 13 Nov 2006 18:54:20 -0700
Subject: memmap working?
In-Reply-To: <4558F0E5.4090108@jpl.nasa.gov>
References: <4558E2C9.6030007@jpl.nasa.gov>
	<e06186140611131403o3b788140m54b4c65f4954286@mail.gmail.com>
	<4558F0E5.4090108@jpl.nasa.gov>
Message-ID: <e06186140611131754r29eee3d6re4d7e71feb2c902@mail.gmail.com>

On 11/13/06, Mathew Yeates <myeates at jpl.nasa.gov> wrote:
>
> Not sure. When I run "top" I see the line
> Memory: 6016M real, 2895M free, 4174M swap in use, 2427M swap free
>
> Its the second number that drops like a rock. Plus, it never comes back
> until I quit the program. This is a great way to turn my machine into a
> nice desk ornament!


try free:

$[charris at fedora ~]$ free
             total       used       free     shared    buffers     cached
Mem:       1034952     995212      39740          0     126616     328124
-/+ buffers/cache:     540472     494480
Swap:       979956        152     979804

The second line under 'used' shows actual program use, i.e. used - buffers -
cached from the first line. But if your system is dying I don't know what to
say. My knowledge of these things is a bit sketchy.

Chuck
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20061113/e32edbda/attachment-0001.html>
-------------- next part --------------
-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
-------------- next part --------------
_______________________________________________
Numpy-discussion mailing list
Numpy-discussion at lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/numpy-discussion

From tim.hochberg at ieee.org  Mon Nov 13 22:10:09 2006
From: tim.hochberg at ieee.org (Tim Hochberg)
Date: Mon, 13 Nov 2006 20:10:09 -0700
Subject: numpy.where behavior
In-Reply-To: <1163467944.2853004636.6699.sendItem@bloglines.com>
References: <1163467944.2853004636.6699.sendItem@bloglines.com>
Message-ID: <45593391.4040001@ieee.org>

vallis.35530053 at bloglines.com wrote:
> Using numpy 1.0, why does
>
>
>
>   
>>>> a = numpy.array([0.0,1.0,2.0],'d')
>>>>         
>
>   
>>>> numpy.where(a
>>>>         
> == 0.0,1,1/a)
>
>
>
> give the correct result, but with the warning "Warning: divide
> by zero encountered in divide"?
>
>
>
> ? I thought that the point of where was
> that the second expression is never used for the elements where the condition
> evaluates true. 
>
>
>
> If this is the desired behavior, is there a way to suppress
> the warning?
>   
Robert Kern has already pointed you to seterr. If you are using Python 
2.5, you also have the option using the with statement, which is more 
convenient if you want to temporarily change the error state. You'll 
need a "from __future__ import with_statement" at the top of your file. 
Then you can temporarily disable errors as shown:

     >>> a = zeros([3])
     >>> b = 1/a  # This will warn
    Warning: divide by zero encountered in divide
     >>> with errstate(divide='ignore'): # But this will not
    ...     c = 1/a
    ...
     >>> d = 1/a # And this will warn again since the error state is
    restored when we exit the block
    Warning: divide by zero encountered in divide


Another little tidbit: this is not as general as where, and could 
probably be considered a little too clever to be clear, but:

        b = 1 / (a + (a==0.0))

is faster than using where in this particular case and sidesteps the 
divide by zero issue altogether.

-tim



-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From pfdubois at gmail.com  Mon Nov 13 23:13:39 2006
From: pfdubois at gmail.com (Paul Dubois)
Date: Mon, 13 Nov 2006 20:13:39 -0800
Subject: numpy.where behavior
In-Reply-To: <45593391.4040001@ieee.org>
References: <1163467944.2853004636.6699.sendItem@bloglines.com>
	<45593391.4040001@ieee.org>
Message-ID: <f74a6c2f0611132013k4e32ca09i69b8c97f27bd9e04@mail.gmail.com>

Unfortunately, where does not have the behavior of not evaluating the second
argument where the first one is true. That would be nice (if the speed were
ok) but it isn't possible unless where is built into the language, since
where doesn't even get called until the arguments have all been calculated.
It was intended as having a different use than avoiding zero-divide. The ma
package can calculate 1/a without problem, resulting in masked results where
a is 0.0.

I put where into numeric after it had proved invaluable in Basis, even
though it has this limitation; it takes care of doing both merge and
compress.

On 13 Nov 2006 20:02:31 -0800, Tim Hochberg <tim.hochberg at ieee.org> wrote:
>
> vallis.35530053 at bloglines.com wrote:
> > Using numpy 1.0, why does
> >
> >
> >
> >
> >>>> a = numpy.array([0.0,1.0,2.0],'d')
> >>>>
> >
> >
> >>>> numpy.where(a
> >>>>
> > == 0.0,1,1/a)
> >
> >
> >
> > give the correct result, but with the warning "Warning: divide
> > by zero encountered in divide"?
> >
> >
> >
> > ? I thought that the point of where was
> > that the second expression is never used for the elements where the
> condition
> > evaluates true.
> >
> >
> >
> > If this is the desired behavior, is there a way to suppress
> > the warning?
> >
> Robert Kern has already pointed you to seterr. If you are using Python
> 2.5, you also have the option using the with statement, which is more
> convenient if you want to temporarily change the error state. You'll
> need a "from __future__ import with_statement" at the top of your file.
> Then you can temporarily disable errors as shown:
>
>      >>> a = zeros([3])
>      >>> b = 1/a  # This will warn
>     Warning: divide by zero encountered in divide
>      >>> with errstate(divide='ignore'): # But this will not
>     ...     c = 1/a
>     ...
>      >>> d = 1/a # And this will warn again since the error state is
>     restored when we exit the block
>     Warning: divide by zero encountered in divide
>
>
> Another little tidbit: this is not as general as where, and could
> probably be considered a little too clever to be clear, but:
>
>         b = 1 / (a + (a==0.0))
>
> is faster than using where in this particular case and sidesteps the
> divide by zero issue altogether.
>
> -tim
>
>
>
> -------------------------------------------------------------------------
> Using Tomcat but need to do more? Need to support web services, security?
> Get stuff done quickly with pre-integrated technology to make your job
> easier
> Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
> http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
> _______________________________________________
> Numpy-discussion mailing list
> Numpy-discussion at lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/numpy-discussion
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20061113/c487d10d/attachment-0001.html>
-------------- next part --------------
-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
-------------- next part --------------
_______________________________________________
Numpy-discussion mailing list
Numpy-discussion at lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/numpy-discussion

From robert.kern at gmail.com  Mon Nov 13 23:16:55 2006
From: robert.kern at gmail.com (Robert Kern)
Date: Mon, 13 Nov 2006 22:16:55 -0600
Subject: numpy.where behavior
In-Reply-To: <45593391.4040001@ieee.org>
References: <1163467944.2853004636.6699.sendItem@bloglines.com>
	<45593391.4040001@ieee.org>
Message-ID: <ejbg02$kqo$1@sea.gmane.org>

Tim Hochberg wrote:
> Another little tidbit: this is not as general as where, and could 
> probably be considered a little too clever to be clear, but:
> 
>         b = 1 / (a + (a==0.0))
> 
> is faster than using where in this particular case and sidesteps the 
> divide by zero issue altogether.

A less clever approach that does much the same thing:

  b = 1.0 / where(a==0, 1.0, a)

-- 
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless enigma
 that is made terrible by our own mad attempt to interpret it as though it had
 an underlying truth."
  -- Umberto Eco


-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From drrfqwrif at pace-engineering.com  Tue Nov 14 01:26:00 2006
From: drrfqwrif at pace-engineering.com (underwater stolen)
Date: Tue, 14 Nov 2006 14:26:00 +0800
Subject: long rest
References: <000701c707b5$c0db48e0$00000000@b5f8e7b051d247b>
Message-ID: <000701c707b5$c0db48e0$00000000@b5f8e7b051d247b>


  ----- Original Message ----- 
  From: numpy-discussion at lists.sourceforge.net 
  To: drrfqwrif at pace-engineering.com 
  Sent: Wednesday, November 00:20:00 PM
  Subject: long rest


  Obtained is Mars Only am person.
  Beetle ju Boulder is Dash Brave Dwarves of gamefree Apple mac is.
  Legendary of boasts superior physics concept advanced collision system vibrant.
  Anything pure adrenaline of rush Buckleup because. While is lawns green or appearance!
  Future fears behind set am mind galaga shareware Nether Star.
  Hand in Foot partner outwit opponent just hands. Fantastic routes is will not choose a one fastest superbikes! Need fb Best fe pages Next in gtshowing Domination.
  We advise Black Jack gives every player.
  Physics concept advanced collision am system vibrant cool? Form row Make sure room could.
  Returns remake famous Dyna four castle. Explosive solitaire peg am.
  Wont feel anything pure am. Contact Copyright copy rights design.
  Not is choose one.
  Atmosphere is enjoy freeware.
  Ten times addition energizing gameplay legendary boasts superior physics. Form row Make sure room could.
  Beetle ju Boulder is Dash Brave Dwarves of gamefree Apple mac is.
  Move colored in blocks around form row Make. Brave Dwarves gamefree Apple!
  Dont in think its. Limited Pick bonuses Geeks Devil.
  Submit Advertise in Privacy is Policy Disclaimer or Contact Copyright copy!
  Dont in think its.
  Brains come in with different strategies can?
  Cut grass statue gtleave News.
  Both fun addictive Move colored blocks around am form.
  On hex am grid using?
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20061114/2503165c/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: while.gif
Type: image/gif
Size: 21684 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20061114/2503165c/attachment-0001.gif>
-------------- next part --------------
-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
-------------- next part --------------
_______________________________________________
Numpy-discussion mailing list
Numpy-discussion at lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/numpy-discussion

From svetosch at gmx.net  Tue Nov 14 04:50:44 2006
From: svetosch at gmx.net (Sven Schreiber)
Date: Tue, 14 Nov 2006 10:50:44 +0100
Subject: why the difference between ipython
 and	python shell?
In-Reply-To: <e06186140611131113v10c07711s49a3e69e09810116@mail.gmail.com>
References: <877ixzl1w1.fsf@poczta.po.opole.pl>
	<e06186140611131113v10c07711s49a3e69e09810116@mail.gmail.com>
Message-ID: <45599174.4090906@gmx.net>

Charles R Harris schrieb:

>     In [1]: from scipy import linalg
> 
>     In [2]: help(linalg.eig)
> 

>
>     >>> from scipy import linalg
>     >>> help(linalg.eig)
> 
>     Help on function eig in module scipy.linalg.decomp:
>
> 
> I expect scipy.linalg and numpy.linalg are different modules containing
> different functions. That said, the documentation in scipy.linalg looks
> quite a bit more complete.
> 
> Chuck

Sure, but Seweryn used the same import statement from scipy and never
explicitly referred to numpy, so there must be some subtle import voodoo
going on. Or did you not show us everything, Seweryn?

-sven

-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From skokot at po.opole.pl  Tue Nov 14 05:05:19 2006
From: skokot at po.opole.pl (Seweryn Kokot)
Date: Tue, 14 Nov 2006 11:05:19 +0100
Subject: why the difference between ipython
	and	python shell?
References: <877ixzl1w1.fsf@poczta.po.opole.pl>
	<e06186140611131113v10c07711s49a3e69e09810116@mail.gmail.com>
	<45599174.4090906@gmx.net>
Message-ID: <87lkmewdgg.fsf@poczta.po.opole.pl>

Sven Schreiber <svetosch at gmx.net> writes:
>
> Sure, but Seweryn used the same import statement from scipy and never
> explicitly referred to numpy, so there must be some subtle import voodoo
> going on. Or did you not show us everything, Seweryn?

It's all ok now, It was my mistake. The problem was that in ipython I
typed "from scipy import linalg" which is wrong and being surprised by the
output I open python shell and tried different combinations of import,
among others "from scipy import *" and this is the reason of the
difference. So now in ipython I get the same output when typing
help(linalg.eig).
Sorry for bothering you,
regards,
SK


-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From meesters at uni-mainz.de  Tue Nov 14 06:09:23 2006
From: meesters at uni-mainz.de (Christian Meesters)
Date: Tue, 14 Nov 2006 12:09:23 +0100
Subject: accessing FORTRAN from Python
Message-ID: <200611141209.23481.meesters@uni-mainz.de>

Hoi,

thanks to Robert Kern who helped me out yesterday on the f2py-list, I was able 
to make some progress in accessing FORTRAN from Python. But only some 
progress ...

If I have the following code, named 'hello.f':
C File hello.f
      subroutine foo (a)
      integer a
      print*, "Hello from Fortran!"
      print*, "a=",a
      end

and compile it with g77 -shared -fPIC hello.f -o hello.so

and then start python, I get the following:
>>> from numpy import *
>>> from ctypes import c_int,  POINTER, byref
>>> hellolib = ctypeslib.load_library('hello', '.')
>>> hello = hellolib.foo_
>>> hello(42)
 Hello from Fortran!
Segmentation fault

Can anybody tell me where my mistake is? (Currently python 2.4.1 (no intention 
to update soon), the most recent ctypes, and numpy '1.0.dev3341' from svn.)

And a second question: Are there simple examples around which show how to pass 
and retrieve lists, numpy arrays, and dicts to and from FORTRAN? Despite an 
intensive web search I couldn't find anything.

TIA
Christian

-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From gnata at obs.univ-lyon1.fr  Tue Nov 14 06:31:41 2006
From: gnata at obs.univ-lyon1.fr (Xavier Gnata)
Date: Tue, 14 Nov 2006 12:31:41 +0100
Subject: A way to compute a 2d histogram
In-Reply-To: <45472DC9.2090802@obs.univ-lyon1.fr>
References: <453E3FE5.1080201@obs.univ-lyon1.fr>	<91cf711d0610241002u765170b5hcb9bacc850a205e0@mail.gmail.com>	<91cf711d0610241340o3fbcc93x235849b8d63ffc16@mail.gmail.com>	<453F57C6.2020101@obs.univ-lyon1.fr>	<91cf711d0610250613q58056568p98dbc342728212fc@mail.gmail.com>
	<45472DC9.2090802@obs.univ-lyon1.fr>
Message-ID: <4559A91D.9020701@obs.univ-lyon1.fr>

Hi,

IFAICS these new histograms versions have not yet been merged to svn.
Are they problems to be solve before to be able to merge them?
How could we help?

Xavier

> Hi,
>
> Your histograms functions look fine for me :)
> As it is a quite usual  operation on an array, I would suggest  to put 
> it in numpy as numpy.histogram. IMHO, there is no point to create an 
> numpy.stats only for histograms (or do you have plans to move other 
> stats related function to numpy.stats?)
>
> Xavier.
>
>   
>> Nicolas, thanks for the bug report, I fooled around with argument 
>> passing and should have checked every case.
>>
>> You'll find the histogram function that deals with weights on the 
>> numpy trac ticket 189, <http://projects.scipy.org/scipy/numpy/ticket/189>
>> I'm waiting for some hints as to where the histogram function should 
>> reside (numpy.histogram, numpy.stats.histogram, ...) before submitting 
>> a patch .
>>
>> Salut,
>> David
>>
>>
>> 2006/10/25, Nicolas Champavert <nicolas.champavert at obs.univ-lyon1.fr 
>> <mailto:nicolas.champavert at obs.univ-lyon1.fr>>:
>>
>>     Hi,
>>
>>         it would be great if you could add the weight option in the 1D
>>     histogram too.
>>
>>     Nicolas
>>
>>     David Huard a ?crit :
>>     > Xavier,
>>     > Here is the patch against svn. Please report any bug. I haven't had
>>     > the time to test it extensively, something that should be done
>>     before
>>     > commiting the patch to the repo. I'd appreciate your feedback.
>>     >
>>     > David
>>     >
>>     > 2006/10/24, David Huard < david.huard at gmail.com
>>     <mailto:david.huard at gmail.com>
>>     > <mailto:david.huard at gmail.com <mailto:david.huard at gmail.com>>>:
>>     >
>>     >     Hi Xavier,
>>     >
>>     >     You could tweak histogram2d to do what you want, or you
>>     could give
>>     >     me a couple of days and I'll do it and let you know. If you want
>>     >     to help, you could write a test using your particular
>>     application
>>     >     and data.
>>     >
>>     >     David
>>     >
>>     >
>>     >     2006/10/24, Xavier Gnata < gnata at obs.univ-lyon1.fr
>>     <mailto:gnata at obs.univ-lyon1.fr>
>>     >     <mailto:gnata at obs.univ-lyon1.fr
>>     <mailto:gnata at obs.univ-lyon1.fr>>>:
>>     >
>>     >         Hi,
>>     >
>>     >         I have a set of 3 1D large arrays.
>>     >         The first 2 one stand for the coordinates of particules and
>>     >         the last one
>>     >         for their masses.
>>     >         I would like to be able to plot this data ie to compute
>>     a 2D
>>     >         histogram
>>     >         summing the masses in each bin.
>>     >         I cannot find a way to do that without any loop on the
>>     indices
>>     >         resulting
>>     >         too a very slow function.
>>     >
>>     >         I'm looking for an elegant way to do that with numpy (or
>>     >         scipy??) function.
>>     >
>>     >         For instance, scipy.histogram2d cannot do the job because it
>>     >         only counts
>>     >         the number of samples in each bin.
>>     >         There is no way to deal with weights.
>>     >
>>     >         Xavier.
>>     >
>>     >
>>     >         --
>>     >         ############################################
>>     >         Xavier Gnata
>>     >         CRAL - Observatoire de Lyon
>>     >         9, avenue Charles Andr?
>>     >         69561 Saint Genis Laval cedex
>>     >         Phone: +33 4 78 86 85 28
>>     >         Fax: +33 4 78 86 83 86
>>     >         E-mail: gnata at obs.univ-lyon1.fr
>>     <mailto:gnata at obs.univ-lyon1.fr> <mailto:gnata at obs.univ-lyon1.fr
>>     <mailto:gnata at obs.univ-lyon1.fr>>
>>     >         ############################################
>>     >
>>     >
>>     >        
>>     -------------------------------------------------------------------------
>>     >         Using Tomcat but need to do more? Need to support web
>>     >         services, security?
>>     >         Get stuff done quickly with pre-integrated technology to
>>     make
>>     >         your job easier
>>     >         Download IBM WebSphere Application Server v.1.0.1 based on
>>     >         Apache Geronimo
>>     >        
>>     http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
>>     <http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642>
>>     >        
>>     <http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
>>     <http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642>>
>>     >         _______________________________________________
>>     >         Numpy-discussion mailing list
>>     >         Numpy-discussion at lists.sourceforge.net
>>     <mailto:Numpy-discussion at lists.sourceforge.net>
>>     >         <mailto:Numpy-discussion at lists.sourceforge.net
>>     <mailto:Numpy-discussion at lists.sourceforge.net>>
>>     >        
>>     https://lists.sourceforge.net/lists/listinfo/numpy-discussion
>>     >
>>     >
>>     >
>>     >
>>
>>     -------------------------------------------------------------------------
>>     Using Tomcat but need to do more? Need to support web services,
>>     security?
>>     Get stuff done quickly with pre-integrated technology to make your
>>     job easier
>>     Download IBM WebSphere Application Server v.1.0.1 based on Apache
>>     Geronimo
>>     http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
>>     <http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642>
>>     _______________________________________________
>>     Numpy-discussion mailing list
>>     Numpy-discussion at lists.sourceforge.net
>>     <mailto:Numpy-discussion at lists.sourceforge.net>
>>     https://lists.sourceforge.net/lists/listinfo/numpy-discussion
>>
>>
>> ------------------------------------------------------------------------
>>
>> -------------------------------------------------------------------------
>> Using Tomcat but need to do more? Need to support web services, security?
>> Get stuff done quickly with pre-integrated technology to make your job easier
>> Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
>> http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
>> ------------------------------------------------------------------------
>>
>> _______________________________________________
>> Numpy-discussion mailing list
>> Numpy-discussion at lists.sourceforge.net
>> https://lists.sourceforge.net/lists/listinfo/numpy-discussion
>>   
>>     
>
>
>   


-- 
############################################
Xavier Gnata
CRAL - Observatoire de Lyon
9, avenue Charles Andr?
69561 Saint Genis Laval cedex
Phone: +33 4 78 86 85 28
Fax: +33 4 78 86 83 86
E-mail: gnata at obs.univ-lyon1.fr
############################################ 


-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From nmarais at sun.ac.za  Tue Nov 14 09:37:52 2006
From: nmarais at sun.ac.za (Neilen Marais)
Date: Tue, 14 Nov 2006 16:37:52 +0200
Subject: 32bit integers as list indices
Message-ID: <pan.2006.11.14.14.37.49.702134@sun.ac.za>

Hi,

I'm not sure if the following is expected to work on a 64bit machine:

In [381]: import numpy as N
In [382]: l = range(3)
In [383]: i32 = N.array([0,2], N.int32)
In [384]: i64 = N.array([0,2], N.int64)
In [385]: l[i32[0]]
---------------------------------------------------------------------------
exceptions.TypeError                           Traceback (most recent call last)

/home/brick/akademie/NewCode/working/<ipython console> 

TypeError: list indices must be integers

In [386]: l[i64[0]]
Out[386]: 0

I'd expect the 32-bit indices to work since they can be upcast to 64bit without
loss. Am I silly for thinking this way, or is it something numpy can/should
address? This came up while working with sparse matrices:

http://projects.scipy.org/scipy/scipy/ticket/307

Regards
Neilen

-- 
you know its kind of tragic 
we live in the new world
but we've lost the magic
-- Battery 9 (www.battery9.co.za)


-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From david.huard at gmail.com  Tue Nov 14 09:53:27 2006
From: david.huard at gmail.com (David Huard)
Date: Tue, 14 Nov 2006 09:53:27 -0500
Subject: A way to compute a 2d histogram
In-Reply-To: <4559A91D.9020701@obs.univ-lyon1.fr>
References: <453E3FE5.1080201@obs.univ-lyon1.fr>
	<91cf711d0610241002u765170b5hcb9bacc850a205e0@mail.gmail.com>
	<91cf711d0610241340o3fbcc93x235849b8d63ffc16@mail.gmail.com>
	<453F57C6.2020101@obs.univ-lyon1.fr>
	<91cf711d0610250613q58056568p98dbc342728212fc@mail.gmail.com>
	<45472DC9.2090802@obs.univ-lyon1.fr>
	<4559A91D.9020701@obs.univ-lyon1.fr>
Message-ID: <91cf711d0611140653q5b3730fdy2d8e57d2edbe4e58@mail.gmail.com>

2006/11/14, Xavier Gnata <gnata at obs.univ-lyon1.fr>:
>
> Hi,
>
> IFAICS these new histograms versions have not yet been merged to svn.
> Are they problems to be solve before to be able to merge them?
> How could we help?


No, I'm just overloaded with other stuff, I'll submit a patch on Trac today
or tomorrow. Up to now, Travis did the merges, but since he looks pretty
busy right now, I don't know when it'll show up on svn.

David



Xavier
>
> > Hi,
> >
> > Your histograms functions look fine for me :)
> > As it is a quite usual  operation on an array, I would suggest  to put
> > it in numpy as numpy.histogram. IMHO, there is no point to create an
> > numpy.stats only for histograms (or do you have plans to move other
> > stats related function to numpy.stats?)
> >
> > Xavier.
> >
> >
> >> Nicolas, thanks for the bug report, I fooled around with argument
> >> passing and should have checked every case.
> >>
> >> You'll find the histogram function that deals with weights on the
> >> numpy trac ticket 189, <
> http://projects.scipy.org/scipy/numpy/ticket/189>
> >> I'm waiting for some hints as to where the histogram function should
> >> reside (numpy.histogram, numpy.stats.histogram, ...) before submitting
> >> a patch .
> >>
> >> Salut,
> >> David
> >>
> >>
> >> 2006/10/25, Nicolas Champavert <nicolas.champavert at obs.univ-lyon1.fr
> >> <mailto:nicolas.champavert at obs.univ-lyon1.fr>>:
> >>
> >>     Hi,
> >>
> >>         it would be great if you could add the weight option in the 1D
> >>     histogram too.
> >>
> >>     Nicolas
> >>
> >>     David Huard a ?crit :
> >>     > Xavier,
> >>     > Here is the patch against svn. Please report any bug. I haven't
> had
> >>     > the time to test it extensively, something that should be done
> >>     before
> >>     > commiting the patch to the repo. I'd appreciate your feedback.
> >>     >
> >>     > David
> >>     >
> >>     > 2006/10/24, David Huard < david.huard at gmail.com
> >>     <mailto:david.huard at gmail.com>
> >>     > <mailto:david.huard at gmail.com <mailto:david.huard at gmail.com>>>:
> >>     >
> >>     >     Hi Xavier,
> >>     >
> >>     >     You could tweak histogram2d to do what you want, or you
> >>     could give
> >>     >     me a couple of days and I'll do it and let you know. If you
> want
> >>     >     to help, you could write a test using your particular
> >>     application
> >>     >     and data.
> >>     >
> >>     >     David
> >>     >
> >>     >
> >>     >     2006/10/24, Xavier Gnata < gnata at obs.univ-lyon1.fr
> >>     <mailto:gnata at obs.univ-lyon1.fr>
> >>     >     <mailto:gnata at obs.univ-lyon1.fr
> >>     <mailto:gnata at obs.univ-lyon1.fr>>>:
> >>     >
> >>     >         Hi,
> >>     >
> >>     >         I have a set of 3 1D large arrays.
> >>     >         The first 2 one stand for the coordinates of particules
> and
> >>     >         the last one
> >>     >         for their masses.
> >>     >         I would like to be able to plot this data ie to compute
> >>     a 2D
> >>     >         histogram
> >>     >         summing the masses in each bin.
> >>     >         I cannot find a way to do that without any loop on the
> >>     indices
> >>     >         resulting
> >>     >         too a very slow function.
> >>     >
> >>     >         I'm looking for an elegant way to do that with numpy (or
> >>     >         scipy??) function.
> >>     >
> >>     >         For instance, scipy.histogram2d cannot do the job because
> it
> >>     >         only counts
> >>     >         the number of samples in each bin.
> >>     >         There is no way to deal with weights.
> >>     >
> >>     >         Xavier.
> >>     >
> >>     >
> >>     >         --
> >>     >         ############################################
> >>     >         Xavier Gnata
> >>     >         CRAL - Observatoire de Lyon
> >>     >         9, avenue Charles Andr?
> >>     >         69561 Saint Genis Laval cedex
> >>     >         Phone: +33 4 78 86 85 28
> >>     >         Fax: +33 4 78 86 83 86
> >>     >         E-mail: gnata at obs.univ-lyon1.fr
> >>     <mailto:gnata at obs.univ-lyon1.fr> <mailto:gnata at obs.univ-lyon1.fr
> >>     <mailto:gnata at obs.univ-lyon1.fr>>
> >>     >         ############################################
> >>     >
> >>     >
> >>     >
> >>
> -------------------------------------------------------------------------
> >>     >         Using Tomcat but need to do more? Need to support web
> >>     >         services, security?
> >>     >         Get stuff done quickly with pre-integrated technology to
> >>     make
> >>     >         your job easier
> >>     >         Download IBM WebSphere Application Server v.1.0.1 based
> on
> >>     >         Apache Geronimo
> >>     >
> >>
> http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
> >>     <
> http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642>
> >>     >
> >>     <
> http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
> >>     <
> http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642>>
> >>     >         _______________________________________________
> >>     >         Numpy-discussion mailing list
> >>     >         Numpy-discussion at lists.sourceforge.net
> >>     <mailto:Numpy-discussion at lists.sourceforge.net>
> >>     >         <mailto:Numpy-discussion at lists.sourceforge.net
> >>     <mailto:Numpy-discussion at lists.sourceforge.net>>
> >>     >
> >>     https://lists.sourceforge.net/lists/listinfo/numpy-discussion
> >>     >
> >>     >
> >>     >
> >>     >
> >>
> >>
> -------------------------------------------------------------------------
> >>     Using Tomcat but need to do more? Need to support web services,
> >>     security?
> >>     Get stuff done quickly with pre-integrated technology to make your
> >>     job easier
> >>     Download IBM WebSphere Application Server v.1.0.1 based on Apache
> >>     Geronimo
> >>
> http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
> >>     <
> http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642>
> >>     _______________________________________________
> >>     Numpy-discussion mailing list
> >>     Numpy-discussion at lists.sourceforge.net
> >>     <mailto:Numpy-discussion at lists.sourceforge.net>
> >>     https://lists.sourceforge.net/lists/listinfo/numpy-discussion
> >>
> >>
> >>
> ------------------------------------------------------------------------
> >>
> >>
> -------------------------------------------------------------------------
> >> Using Tomcat but need to do more? Need to support web services,
> security?
> >> Get stuff done quickly with pre-integrated technology to make your job
> easier
> >> Download IBM WebSphere Application Server v.1.0.1 based on Apache
> Geronimo
> >>
> http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
> >>
> ------------------------------------------------------------------------
> >>
> >> _______________________________________________
> >> Numpy-discussion mailing list
> >> Numpy-discussion at lists.sourceforge.net
> >> https://lists.sourceforge.net/lists/listinfo/numpy-discussion
> >>
> >>
> >
> >
> >
>
>
> --
> ############################################
> Xavier Gnata
> CRAL - Observatoire de Lyon
> 9, avenue Charles Andr?
> 69561 Saint Genis Laval cedex
> Phone: +33 4 78 86 85 28
> Fax: +33 4 78 86 83 86
> E-mail: gnata at obs.univ-lyon1.fr
> ############################################
>
>
> -------------------------------------------------------------------------
> Using Tomcat but need to do more? Need to support web services, security?
> Get stuff done quickly with pre-integrated technology to make your job
> easier
> Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
> http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
> _______________________________________________
> Numpy-discussion mailing list
> Numpy-discussion at lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/numpy-discussion
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20061114/70e08098/attachment-0001.html>
-------------- next part --------------
-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
-------------- next part --------------
_______________________________________________
Numpy-discussion mailing list
Numpy-discussion at lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/numpy-discussion

From oliphant at ee.byu.edu  Tue Nov 14 10:56:14 2006
From: oliphant at ee.byu.edu (Travis Oliphant)
Date: Tue, 14 Nov 2006 08:56:14 -0700
Subject: 32bit integers as list indices
In-Reply-To: <pan.2006.11.14.14.37.49.702134@sun.ac.za>
References: <pan.2006.11.14.14.37.49.702134@sun.ac.za>
Message-ID: <4559E71E.4070805@ee.byu.edu>

Neilen Marais wrote:

>Hi,
>
>I'm not sure if the following is expected to work on a 64bit machine:
>
>In [381]: import numpy as N
>In [382]: l = range(3)
>In [383]: i32 = N.array([0,2], N.int32)
>In [384]: i64 = N.array([0,2], N.int64)
>In [385]: l[i32[0]]
>---------------------------------------------------------------------------
>exceptions.TypeError                           Traceback (most recent call last)
>
>/home/brick/akademie/NewCode/working/<ipython console> 
>
>TypeError: list indices must be integers
>
>In [386]: l[i64[0]]
>Out[386]: 0
>
>I'd expect the 32-bit indices to work since they can be upcast to 64bit without
>loss. Am I silly for thinking this way, or is it something numpy can/should
>address? This came up while working with sparse matrices:
>
>http://projects.scipy.org/scipy/scipy/ticket/307
>  
>

It's addressed with Python 2.5  

We can't do anything about it for Python 2.4

-Travis


-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642



From robert.kern at gmail.com  Tue Nov 14 12:01:09 2006
From: robert.kern at gmail.com (Robert Kern)
Date: Tue, 14 Nov 2006 11:01:09 -0600
Subject: accessing FORTRAN from Python
In-Reply-To: <200611141209.23481.meesters@uni-mainz.de>
References: <200611141209.23481.meesters@uni-mainz.de>
Message-ID: <ejcsp1$atk$1@sea.gmane.org>

Christian Meesters wrote:
> Hoi,
> 
> thanks to Robert Kern who helped me out yesterday on the f2py-list, I was able 
> to make some progress in accessing FORTRAN from Python. But only some 
> progress ...
> 
> If I have the following code, named 'hello.f':
> C File hello.f
>       subroutine foo (a)
>       integer a
>       print*, "Hello from Fortran!"
>       print*, "a=",a
>       end
> 
> and compile it with g77 -shared -fPIC hello.f -o hello.so
> 
> and then start python, I get the following:
>>>> from numpy import *
>>>> from ctypes import c_int,  POINTER, byref
>>>> hellolib = ctypeslib.load_library('hello', '.')
>>>> hello = hellolib.foo_
>>>> hello(42)
>  Hello from Fortran!
> Segmentation fault
> 
> Can anybody tell me where my mistake is? (Currently python 2.4.1 (no intention 
> to update soon), the most recent ctypes, and numpy '1.0.dev3341' from svn.)
> 
> And a second question: Are there simple examples around which show how to pass 
> and retrieve lists, numpy arrays, and dicts to and from FORTRAN? Despite an 
> intensive web search I couldn't find anything.

I don't believe anyone has posted anything about calling FORTRAN code using
ctypes. Why aren't you using f2py?

-- 
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless enigma
 that is made terrible by our own mad attempt to interpret it as though it had
 an underlying truth."
  -- Umberto Eco


-------------------------------------------------------------------------
SF.net email is sponsored by: A Better Job is Waiting for You - Find it Now.
Check out Slashdot's new job board. Browse through tons of technical jobs
posted by companies looking to hire people just like you.
http://jobs.slashdot.org/



From meesters at uni-mainz.de  Tue Nov 14 12:19:41 2006
From: meesters at uni-mainz.de (Christian Meesters)
Date: Tue, 14 Nov 2006 18:19:41 +0100
Subject: accessing FORTRAN from Python
In-Reply-To: <ejcsp1$atk$1@sea.gmane.org>
References: <200611141209.23481.meesters@uni-mainz.de>
	<ejcsp1$atk$1@sea.gmane.org>
Message-ID: <200611141819.41699.meesters@uni-mainz.de>

On Tuesday 14 November 2006 18:01, Robert Kern wrote:
>
> I don't believe anyone has posted anything about calling FORTRAN code using
> ctypes. Why aren't you using f2py?
What if we just forget about my last post? (I guess I was doing way too many 
things in parallel, stumbled across the g95 site and tried to apply what's 
posted there in the way described there ...)

Sorry for this confusion and thanks, Robert, that again you brought me back on 
track, things are working now.

Christian

-------------------------------------------------------------------------
SF.net email is sponsored by: A Better Job is Waiting for You - Find it Now.
Check out Slashdot's new job board. Browse through tons of technical jobs
posted by companies looking to hire people just like you.
http://jobs.slashdot.org/



From david.huard at gmail.com  Tue Nov 14 13:50:14 2006
From: david.huard at gmail.com (David Huard)
Date: Tue, 14 Nov 2006 13:50:14 -0500
Subject: A way to compute a 2d histogram
In-Reply-To: <91cf711d0611140653q5b3730fdy2d8e57d2edbe4e58@mail.gmail.com>
References: <453E3FE5.1080201@obs.univ-lyon1.fr>
	<91cf711d0610241002u765170b5hcb9bacc850a205e0@mail.gmail.com>
	<91cf711d0610241340o3fbcc93x235849b8d63ffc16@mail.gmail.com>
	<453F57C6.2020101@obs.univ-lyon1.fr>
	<91cf711d0610250613q58056568p98dbc342728212fc@mail.gmail.com>
	<45472DC9.2090802@obs.univ-lyon1.fr>
	<4559A91D.9020701@obs.univ-lyon1.fr>
	<91cf711d0611140653q5b3730fdy2d8e57d2edbe4e58@mail.gmail.com>
Message-ID: <91cf711d0611141050ka8b18abl7a84063db241269a@mail.gmail.com>

I put the patch on Trac. Ticket 189.

2006/11/14, Xavier Gnata <gnata at obs.univ-lyon1.fr>:
>
> Hi,
> >
> > IFAICS these new histograms versions have not yet been merged to svn.
> > Are they problems to be solve before to be able to merge them?
> > How could we help?
>
>
Voice you support on Trac to replace histogram with the upgraded version.
Thanks.


David
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20061114/beae1a81/attachment-0001.html>
-------------- next part --------------
-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your
opinions on IT & business topics through brief surveys - and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV
-------------- next part --------------
_______________________________________________
Numpy-discussion mailing list
Numpy-discussion at lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/numpy-discussion

From jdhunter at ace.bsd.uchicago.edu  Tue Nov 14 16:17:29 2006
From: jdhunter at ace.bsd.uchicago.edu (John Hunter)
Date: Tue, 14 Nov 2006 15:17:29 -0600
Subject: mysql -> record array
Message-ID: <87slglwwwm.fsf@peds-pc311.bsd.uchicago.edu>


Has anyone written any code to facilitate dumping mysql query results
(mainly arrays of floats) into numpy arrays directly at the extension
code layer.  The query results->list->array conversion can be slow.

Ideally, one could do this semi-automagically with record arrays and
table introspection....

JDH

-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your
opinions on IT & business topics through brief surveys - and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV



From jstrunk at enthought.com  Tue Nov 14 16:42:53 2006
From: jstrunk at enthought.com (Jeff Strunk)
Date: Tue, 14 Nov 2006 15:42:53 -0600
Subject: List migration
Message-ID: <200611141542.53287.jstrunk@enthought.com>

Good afternoon,

We will be performing the migration of this mailing list from Sourceforge to 
SciPy on Thursday at 2pm central.

After this time, the new mailing list address will be 
numpy-discussion at scipy.org . Any mail sent to the Sourceforge address will 
NOT be forwarded. The sender will receive a message with the new address.

Thank you for your patience.
Jeff Strunk
IT Administrator
Enthought, Inc.

-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your
opinions on IT & business topics through brief surveys - and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV



From kwgoodman at gmail.com  Tue Nov 14 16:49:02 2006
From: kwgoodman at gmail.com (Keith Goodman)
Date: Tue, 14 Nov 2006 13:49:02 -0800
Subject: List migration
In-Reply-To: <200611141542.53287.jstrunk@enthought.com>
References: <200611141542.53287.jstrunk@enthought.com>
Message-ID: <f4f93d420611141349xc64c707iaf4153d9fcfe450e@mail.gmail.com>

On 11/14/06, Jeff Strunk <jstrunk at enthought.com> wrote:
> Good afternoon,
>
> We will be performing the migration of this mailing list from Sourceforge to
> SciPy on Thursday at 2pm central.
>
> After this time, the new mailing list address will be
> numpy-discussion at scipy.org . Any mail sent to the Sourceforge address will
> NOT be forwarded. The sender will receive a message with the new address.
>
> Thank you for your patience.
> Jeff Strunk
> IT Administrator
> Enthought, Inc.
>
> -------------------------------------------------------------------------
> Take Surveys. Earn Cash. Influence the Future of IT
> Join SourceForge.net's Techsay panel and you'll get the chance to share your
> opinions on IT & business topics through brief surveys - and earn cash

So is this my last chance to Take Surveys. Earn Cash. Influence the
Future of IT?

That's great. Thank you for hosting.

-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your
opinions on IT & business topics through brief surveys - and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV



From klemm at phys.ethz.ch  Tue Nov 14 16:52:37 2006
From: klemm at phys.ethz.ch (Hanno Klemm)
Date: Tue, 14 Nov 2006 22:52:37 +0100
Subject: accessing FORTRAN from Python
In-Reply-To: <200611141209.23481.meesters@uni-mainz.de>
References: <200611141209.23481.meesters@uni-mainz.de>
Message-ID: <4f7439c24e2fb96f149bc34754af8bfe@phys.ethz.ch>

Hi Christian,

I send this off-list as there are probably a lot more knowledgeable  
people around there.

However, I don't entirely understand your problem (I'm not on the f2py  
list).

What happens if you try:

C file hello.f
       subroutine foo(a)
       integer a
Cf2py intent(in) a
       print*, "Hello from  Fortran!"
       print*, "a=",a
       end


f2py -m -c hello hello.f

That usually did the trick for me. What are your error messages, if you  
try the above?

Best regards,
Hanno


On Nov 14, 2006, at 12:09 PM, Christian Meesters wrote:

> Hoi,
>
> thanks to Robert Kern who helped me out yesterday on the f2py-list, I  
> was able
> to make some progress in accessing FORTRAN from Python. But only some
> progress ...
>
> If I have the following code, named 'hello.f':
> C File hello.f
>       subroutine foo (a)
>       integer a
>       print*, "Hello from Fortran!"
>       print*, "a=",a
>       end
>
> and compile it with g77 -shared -fPIC hello.f -o hello.so
>
> and then start python, I get the following:
>>>> from numpy import *
>>>> from ctypes import c_int,  POINTER, byref
>>>> hellolib = ctypeslib.load_library('hello', '.')
>>>> hello = hellolib.foo_
>>>> hello(42)
>  Hello from Fortran!
> Segmentation fault
>
> Can anybody tell me where my mistake is? (Currently python 2.4.1 (no  
> intention
> to update soon), the most recent ctypes, and numpy '1.0.dev3341' from  
> svn.)
>
> And a second question: Are there simple examples around which show how  
> to pass
> and retrieve lists, numpy arrays, and dicts to and from FORTRAN?  
> Despite an
> intensive web search I couldn't find anything.
>
> TIA
> Christian
>
> ----------------------------------------------------------------------- 
> --
> Using Tomcat but need to do more? Need to support web services,  
> security?
> Get stuff done quickly with pre-integrated technology to make your job  
> easier
> Download IBM WebSphere Application Server v.1.0.1 based on Apache  
> Geronimo
> http://sel.as-us.falkag.net/sel? 
> cmd=lnk&kid=120709&bid=263057&dat=121642
> _______________________________________________
> Numpy-discussion mailing list
> Numpy-discussion at lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/numpy-discussion
>
-- 
Hanno Klemm							klemm at itp.phys.ethz.ch
ETH Zurich 							tel: +41-1-6332580
Institute for theoretical physics	mobile: +41-79-4500428
									http://www.mth.kcl.ac.uk/~klemm


-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your
opinions on IT & business topics through brief surveys - and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV



From erin.sheldon at gmail.com  Tue Nov 14 17:08:52 2006
From: erin.sheldon at gmail.com (Erin Sheldon)
Date: Tue, 14 Nov 2006 17:08:52 -0500
Subject: mysql -> record array
In-Reply-To: <87slglwwwm.fsf@peds-pc311.bsd.uchicago.edu>
References: <87slglwwwm.fsf@peds-pc311.bsd.uchicago.edu>
Message-ID: <331116dc0611141408q2d7c9431s38a0f4c4b3f2db25@mail.gmail.com>

On 11/14/06, John Hunter <jdhunter at ace.bsd.uchicago.edu> wrote:
>
> Has anyone written any code to facilitate dumping mysql query results
> (mainly arrays of floats) into numpy arrays directly at the extension
> code layer.  The query results->list->array conversion can be slow.
>
> Ideally, one could do this semi-automagically with record arrays and
> table introspection....

I've been considering this as well.  I use both postgres and Oracle
in my work, and I have been using the python interfaces (cx_Oracle
and pgdb) to get result lists and convert to numpy arrays.

The question I have been asking myself is "what is the advantage
of such an approach?".  It would be faster, but by how
much?  Presumably the bottleneck for most applications will
be data retrieval rather than data copying in memory.  The
process numpy.array(results, dtype=) is pretty fast and simple
if the client is DB 2.0 compliant and returns a list of tuples
(pgdb does not sadly).  Also the memory usage will be about
the same since a copy must be made in order to create the
lists or python arrays in either case.

On the other hand, the database access modules for all major
databases, with DB 2.0 semicomplience, have already been written.
This is not an insignificant amount of work.  Writing our own
interfaces for each  of our favorite databases would require an
equivalent amount of work.

I think a set of timing tests would be useful.  I will try some
using Oracle or postgres over the next few days.  Perhaps
you could do the same with mysql.

Erin

-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your
opinions on IT & business topics through brief surveys - and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV



From jdhunter at ace.bsd.uchicago.edu  Tue Nov 14 18:02:07 2006
From: jdhunter at ace.bsd.uchicago.edu (John Hunter)
Date: Tue, 14 Nov 2006 17:02:07 -0600
Subject: mysql -> record array
In-Reply-To: <331116dc0611141408q2d7c9431s38a0f4c4b3f2db25@mail.gmail.com>
	("Erin Sheldon"'s message of "Tue, 14 Nov 2006 17:08:52 -0500")
References: <87slglwwwm.fsf@peds-pc311.bsd.uchicago.edu>
	<331116dc0611141408q2d7c9431s38a0f4c4b3f2db25@mail.gmail.com>
Message-ID: <87velh4opc.fsf@peds-pc311.bsd.uchicago.edu>

>>>>> "Erin" == Erin Sheldon <erin.sheldon at gmail.com> writes:

    Erin> The question I have been asking myself is "what is the
    Erin> advantage of such an approach?".  It would be faster, but by

In the use case that prompted this message, the pull from mysql took
almost 3 seconds, and the conversion from lists to numpy arrays took
more that 4 seconds.  We have a list of about 500000 2 tuples of
floats.

Digging in a little bit, we found that numpy is about 3x slower than
Numeric here

  peds-pc311:~> python test.py
  with dtype: 4.25 elapsed seconds
  w/o dtype 5.79 elapsed seconds
  Numeric  1.58 elapsed seconds
  24.0b2
  1.0.1.dev3432

Hmm... So maybe the question is -- is there some low hanging fruit
here to get numpy speeds up?

import time
import numpy
import numpy.random
rand = numpy.random.rand

x = [(rand(), rand()) for i in xrange(500000)]
tnow = time.time()
y = numpy.array(x, dtype=numpy.float_)
tdone = time.time()
print 'with dtype: %1.2f elapsed seconds'%(tdone - tnow)

tnow = time.time()
y = numpy.array(x)
tdone = time.time()
print 'w/o dtype %1.2f elapsed seconds'%(tdone - tnow)

import Numeric
tnow = time.time()
y = Numeric.array(x, Numeric.Float)
tdone = time.time()
print 'Numeric  %1.2f elapsed seconds'%(tdone - tnow)

print Numeric.__version__
print numpy.__version__

-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your
opinions on IT & business topics through brief surveys - and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV



From jdhunter at ace.bsd.uchicago.edu  Tue Nov 14 18:14:27 2006
From: jdhunter at ace.bsd.uchicago.edu (John Hunter)
Date: Tue, 14 Nov 2006 17:14:27 -0600
Subject: mysql -> record array
In-Reply-To: <87velh4opc.fsf@peds-pc311.bsd.uchicago.edu> (John Hunter's
	message of "Tue, 14 Nov 2006 17:02:07 -0600")
References: <87slglwwwm.fsf@peds-pc311.bsd.uchicago.edu>
	<331116dc0611141408q2d7c9431s38a0f4c4b3f2db25@mail.gmail.com>
	<87velh4opc.fsf@peds-pc311.bsd.uchicago.edu>
Message-ID: <87psbpvcx8.fsf@peds-pc311.bsd.uchicago.edu>

>>>>> "John" == John Hunter <jdhunter at ace.bsd.uchicago.edu> writes:

>>>>> "Erin" == Erin Sheldon <erin.sheldon at gmail.com> writes:
    Erin> The question I have been asking myself is "what is the
    Erin> advantage of such an approach?".  It would be faster, but by

    John> In the use case that prompted this message, the pull from
    John> mysql took almost 3 seconds, and the conversion from lists
    John> to numpy arrays took more that 4 seconds.  We have a list of
    John> about 500000 2 tuples of floats.

    John> Digging in a little bit, we found that numpy is about 3x
    John> slower than Numeric here

    John>   peds-pc311:~> python test.py with dtype: 4.25 elapsed
    John> seconds w/o dtype 5.79 elapsed seconds Numeric 1.58 elapsed
    John> seconds 24.0b2 1.0.1.dev3432

    John> Hmm... So maybe the question is -- is there some low hanging
    John> fruit here to get numpy speeds up?

And for reference, numarray is 5 times faster than Numeric here and 15
times faster than numpy

  peds-pc311:~> python test.py
  with dtype: 4.20 elapsed seconds
  w/o dtype 5.71 elapsed seconds
  Numeric  1.60 elapsed seconds
  numarray  0.30 elapsed seconds
  24.0b2
  1.0.1.dev3432
  1.5.1

import numarray
tnow = time.time()
y = numarray.array(x, numarray.Float)
tdone = time.time()
print 'numarray  %1.2f elapsed seconds'%(tdone - tnow)

print numarray.__version__


-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your
opinions on IT & business topics through brief surveys - and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV



From pgmdevlist at gmail.com  Tue Nov 14 18:33:15 2006
From: pgmdevlist at gmail.com (Pierre GM)
Date: Tue, 14 Nov 2006 18:33:15 -0500
Subject: mysql -> record array
In-Reply-To: <87psbpvcx8.fsf@peds-pc311.bsd.uchicago.edu>
References: <87slglwwwm.fsf@peds-pc311.bsd.uchicago.edu>
	<87velh4opc.fsf@peds-pc311.bsd.uchicago.edu>
	<87psbpvcx8.fsf@peds-pc311.bsd.uchicago.edu>
Message-ID: <200611141833.15877.pgmdevlist@gmail.com>

John,
I just added the following to your example:
...................................
tnow = time.time()
y = numpy.fromiter((tuple(i) for i in x), dtype=[('a',numpy.float_), ('b', 
numpy.float_)])
tdone = time.time()
print 'Numpy.fromiter %1.2f elapsed seconds'%(tdone - tnow)
...................................
Here are my results
with dtype: 4.43 elapsed seconds
w/o dtype 5.78 elapsed seconds
Numeric ?1.17 elapsed seconds
Numpy.fromiter 0.62 elapsed seconds
23.7
1.0

Numpy, one point.

-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your
opinions on IT & business topics through brief surveys - and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV



From kortmann at ideaworks.com  Tue Nov 14 18:48:47 2006
From: kortmann at ideaworks.com (Kenny Ortmann)
Date: Tue, 14 Nov 2006 17:48:47 -0600
Subject: numpy, matplotlib, scipy
References: <mailman.70135.1161983292.22397.numpy-discussion@lists.sourceforge.net>
Message-ID: <03c401c70847$6def0670$0800a8c0@yairlap>

I am sorry if this has come up before, I've found some stuff on it but just 
want to be clear.
What are the latest versions of numpy matplotlib and scipy that work 
together.
I have seen that numpy 1.0rc2 works with the latest scipy until 0.5.2 comes 
out.
now with numpy and matplotlib?  I saw that matplotlib 87.5 works with i 
think the 1.0rc2 also, but i read that mpl was going to release right after 
numpy 1.0 was released so that they would be compatible.
I am just trying to upgrade these packages before I create an executable of 
the program I am working on, and I am running into problems with
"from matplotlib._ns_nxutils import *
   ImportError: numpy.core.multiarray failed to imort"

-Kenny 


-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your
opinions on IT & business topics through brief surveys - and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV



From oliphant at ee.byu.edu  Tue Nov 14 18:50:03 2006
From: oliphant at ee.byu.edu (Travis Oliphant)
Date: Tue, 14 Nov 2006 16:50:03 -0700
Subject: mysql -> record array
In-Reply-To: <87psbpvcx8.fsf@peds-pc311.bsd.uchicago.edu>
References: <87slglwwwm.fsf@peds-pc311.bsd.uchicago.edu>	<331116dc0611141408q2d7c9431s38a0f4c4b3f2db25@mail.gmail.com>	<87velh4opc.fsf@peds-pc311.bsd.uchicago.edu>
	<87psbpvcx8.fsf@peds-pc311.bsd.uchicago.edu>
Message-ID: <455A562B.30303@ee.byu.edu>

John Hunter wrote:

>>>>>>"John" == John Hunter <jdhunter at ace.bsd.uchicago.edu> writes:
>>>>>>            
>>>>>>
>
>  
>
>>>>>>"Erin" == Erin Sheldon <erin.sheldon at gmail.com> writes:
>>>>>>            
>>>>>>
>    Erin> The question I have been asking myself is "what is the
>    Erin> advantage of such an approach?".  It would be faster, but by
>
>    John> In the use case that prompted this message, the pull from
>    John> mysql took almost 3 seconds, and the conversion from lists
>    John> to numpy arrays took more that 4 seconds.  We have a list of
>    John> about 500000 2 tuples of floats.
>
>    John> Digging in a little bit, we found that numpy is about 3x
>    John> slower than Numeric here
>
>    John>   peds-pc311:~> python test.py with dtype: 4.25 elapsed
>    John> seconds w/o dtype 5.79 elapsed seconds Numeric 1.58 elapsed
>    John> seconds 24.0b2 1.0.1.dev3432
>
>    John> Hmm... So maybe the question is -- is there some low hanging
>    John> fruit here to get numpy speeds up?
>
>And for reference, numarray is 5 times faster than Numeric here and 15
>times faster than numpy
>
>  peds-pc311:~> python test.py
>  with dtype: 4.20 elapsed seconds
>  w/o dtype 5.71 elapsed seconds
>  Numeric  1.60 elapsed seconds
>  numarray  0.30 elapsed seconds
>  24.0b2
>  1.0.1.dev3432
>  1.5.1
>
>import numarray
>tnow = time.time()
>y = numarray.array(x, numarray.Float)
>tdone = time.time()
>  
>
This sounds like it could be definitely be sped up, then.      
Assign_Array is the relevant code (it then calls PySequence_SetItem) so 
that basically

for k in a.shape[0]:
      a[k] = x[k]

is what is being done.  Thus, it might be the indexing code that is 
causing this to be a slower operation.  We should look at what numarray 
is doing --- it has provided important speed-ups in the past.

I still don't have time to look at this, but please file a ticket as we 
should fix this one.  Reference the faster numarray implementation.

-Travis



-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your
opinions on IT & business topics through brief surveys - and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV



From tim.hochberg at ieee.org  Tue Nov 14 19:44:48 2006
From: tim.hochberg at ieee.org (Tim Hochberg)
Date: Tue, 14 Nov 2006 17:44:48 -0700
Subject: mysql -> record array
In-Reply-To: <87velh4opc.fsf@peds-pc311.bsd.uchicago.edu>
References: <87slglwwwm.fsf@peds-pc311.bsd.uchicago.edu>	<331116dc0611141408q2d7c9431s38a0f4c4b3f2db25@mail.gmail.com>
	<87velh4opc.fsf@peds-pc311.bsd.uchicago.edu>
Message-ID: <455A62FF.6040706@ieee.org>

John Hunter wrote:
>>>>>> "Erin" == Erin Sheldon <erin.sheldon at gmail.com> writes:
>>>>>>             
>
>     Erin> The question I have been asking myself is "what is the
>     Erin> advantage of such an approach?".  It would be faster, but by
>
> In the use case that prompted this message, the pull from mysql took
> almost 3 seconds, and the conversion from lists to numpy arrays took
> more that 4 seconds.  We have a list of about 500000 2 tuples of
> floats.
>   
I'm no database user, but a glance at the at the docs seems to indicate 
that you can get your data via an iterator (by iterating over the cursor 
or some such db mumbo jumbo) rather than slurping up the whole list up 
at once. If so, then you'll save a lot of memory by passing the iterator 
straight to fromiter. It may even be faster, who knows.

Accessing the db via the iterator could be a performance killer, but 
it's almost certainly worth trying as it could a few megabytes of 
storage and that in turn might speed things up.

-tim

> Digging in a little bit, we found that numpy is about 3x slower than
> Numeric here
>
>   peds-pc311:~> python test.py
>   with dtype: 4.25 elapsed seconds
>   w/o dtype 5.79 elapsed seconds
>   Numeric  1.58 elapsed seconds
>   24.0b2
>   1.0.1.dev3432
>
> Hmm... So maybe the question is -- is there some low hanging fruit
> here to get numpy speeds up?
>
> import time
> import numpy
> import numpy.random
> rand = numpy.random.rand
>
> x = [(rand(), rand()) for i in xrange(500000)]
> tnow = time.time()
> y = numpy.array(x, dtype=numpy.float_)
> tdone = time.time()
> print 'with dtype: %1.2f elapsed seconds'%(tdone - tnow)
>
> tnow = time.time()
> y = numpy.array(x)
> tdone = time.time()
> print 'w/o dtype %1.2f elapsed seconds'%(tdone - tnow)
>
> import Numeric
> tnow = time.time()
> y = Numeric.array(x, Numeric.Float)
> tdone = time.time()
> print 'Numeric  %1.2f elapsed seconds'%(tdone - tnow)
>
> print Numeric.__version__
> print numpy.__version__
>
> -------------------------------------------------------------------------
> Take Surveys. Earn Cash. Influence the Future of IT
> Join SourceForge.net's Techsay panel and you'll get the chance to share your
> opinions on IT & business topics through brief surveys - and earn cash
> http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV
> _______________________________________________
> Numpy-discussion mailing list
> Numpy-discussion at lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/numpy-discussion
>
>
>   



-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your
opinions on IT & business topics through brief surveys - and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV



From tim.hochberg at ieee.org  Tue Nov 14 21:09:09 2006
From: tim.hochberg at ieee.org (Tim Hochberg)
Date: Tue, 14 Nov 2006 19:09:09 -0700
Subject: mysql -> record array
In-Reply-To: <455A62FF.6040706@ieee.org>
References: <87slglwwwm.fsf@peds-pc311.bsd.uchicago.edu>	<331116dc0611141408q2d7c9431s38a0f4c4b3f2db25@mail.gmail.com>	<87velh4opc.fsf@peds-pc311.bsd.uchicago.edu>
	<455A62FF.6040706@ieee.org>
Message-ID: <455A76C5.9050508@ieee.org>

Tim Hochberg wrote:
> John Hunter wrote:
>   
>>>>>>> "Erin" == Erin Sheldon <erin.sheldon at gmail.com> writes:
>>>>>>>             
>>>>>>>               
>>     Erin> The question I have been asking myself is "what is the
>>     Erin> advantage of such an approach?".  It would be faster, but by
>>
>> In the use case that prompted this message, the pull from mysql took
>> almost 3 seconds, and the conversion from lists to numpy arrays took
>> more that 4 seconds.  We have a list of about 500000 2 tuples of
>> floats.
>>   
>>     
> I'm no database user, but a glance at the at the docs seems to indicate 
> that you can get your data via an iterator (by iterating over the cursor 
> or some such db mumbo jumbo) rather than slurping up the whole list up 
> at once. If so, then you'll save a lot of memory by passing the iterator 
> straight to fromiter. It may even be faster, who knows.
>
> Accessing the db via the iterator could be a performance killer, but 
> it's almost certainly worth trying as it could a few megabytes of 
> storage and that in turn might speed things up.
>
>   
Assuming that I didn't mess this up too badly, it appears that using the 
iterator directly with fromiter is significantly faster than the next 
best solution (about 45%). The fromiter wrapping a list solution come in 
second, followed by numarray.array and finally way in the back, 
numpy.array. Here's the numbers:

    retrieve1 took 0.902922857514 seconds
    retrieve2 took 1.31245870634 seconds
    retrieve3 took 1.51207569677 seconds
    retrieve4 took 8.71539930354 seconds

And here is the code:

    import sqlite3, numpy as np, numarray as na, time

    N = 500000

    def setup(conn):
        c = conn.cursor()
        c.execute('''create table demo (x real, y real)''')
        data = np.random.rand(N, 2)
        c.executemany("""insert into demo values (?, ?)""", data)

    def retrieve1(conn):
        c = conn.cursor()
        c.execute('select * from demo')
        y = np.fromiter(c, dtype=[('a',float), ('b', float)])
        return y

    def retrieve2(conn):
        c = conn.cursor()
        c.execute('select * from demo')
        y = np.fromiter(c.fetchall(), dtype=[('a',float), ('b', float)])
        return y

    def retrieve3(conn):
        c = conn.cursor()
        c.execute('select * from demo')
        y = na.array(c.fetchall())
        return y

    def retrieve4(conn):
        c = conn.cursor()
        c.execute('select * from demo')
        y = np.array(c.fetchall())
        return y

    conn = sqlite3.connect(':memory:')

    setup(conn)

    t0 = time.clock()
    y1 = retrieve1(conn)
    t1 = time.clock()
    y2 = retrieve2(conn)
    t2 = time.clock()
    y3 = retrieve3(conn)
    t3 = time.clock()
    y4 = retrieve4(conn)
    t4 = time.clock()

    assert y1.shape == y2.shape == y3.shape[:1] == y4.shape[:1] == (N,)
    assert np.alltrue(y1 == y2)

    print "retrieve1 took", t1-t0, "seconds"
    print "retrieve2 took", t2-t1, "seconds"
    print "retrieve3 took", t3-t2, "seconds"
    print "retrieve4 took", t4-t3, "seconds"







-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your
opinions on IT & business topics through brief surveys - and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV



From erin.sheldon at gmail.com  Tue Nov 14 21:26:48 2006
From: erin.sheldon at gmail.com (Erin Sheldon)
Date: Tue, 14 Nov 2006 21:26:48 -0500
Subject: mysql -> record array
In-Reply-To: <455A76C5.9050508@ieee.org>
References: <87slglwwwm.fsf@peds-pc311.bsd.uchicago.edu>
	<331116dc0611141408q2d7c9431s38a0f4c4b3f2db25@mail.gmail.com>
	<87velh4opc.fsf@peds-pc311.bsd.uchicago.edu>
	<455A62FF.6040706@ieee.org> <455A76C5.9050508@ieee.org>
Message-ID: <331116dc0611141826x47ed9ccbpf5ff038c79967a44@mail.gmail.com>

On 11/14/06, Tim Hochberg <tim.hochberg at ieee.org> wrote:
> Tim Hochberg wrote:
> > John Hunter wrote:
> >
> >>>>>>> "Erin" == Erin Sheldon <erin.sheldon at gmail.com> writes:
> >>>>>>>
> >>>>>>>
> >>     Erin> The question I have been asking myself is "what is the
> >>     Erin> advantage of such an approach?".  It would be faster, but by
> >>
> >> In the use case that prompted this message, the pull from mysql took
> >> almost 3 seconds, and the conversion from lists to numpy arrays took
> >> more that 4 seconds.  We have a list of about 500000 2 tuples of
> >> floats.
> >>
> >>
> > I'm no database user, but a glance at the at the docs seems to indicate
> > that you can get your data via an iterator (by iterating over the cursor
> > or some such db mumbo jumbo) rather than slurping up the whole list up
> > at once. If so, then you'll save a lot of memory by passing the iterator
> > straight to fromiter. It may even be faster, who knows.
> >
> > Accessing the db via the iterator could be a performance killer, but
> > it's almost certainly worth trying as it could a few megabytes of
> > storage and that in turn might speed things up.
> >
> >
> Assuming that I didn't mess this up too badly, it appears that using the
> iterator directly with fromiter is significantly faster than the next
> best solution (about 45%). The fromiter wrapping a list solution come in
> second, followed by numarray.array and finally way in the back,
> numpy.array. Here's the numbers:
>
>     retrieve1 took 0.902922857514 seconds
>     retrieve2 took 1.31245870634 seconds
>     retrieve3 took 1.51207569677 seconds
>     retrieve4 took 8.71539930354 seconds
>
>

Interesting results Tim.  From Pierre's results
we saw that fromiter is the fastest way to get data
into arrays.  With your results we see there is a
difference between iterating over the cursor and
doing a fetchall() as well. Surprisingly, running
the cursor is faster.

This must come not from the data retrieval rate but
from creating the copies in memory. But just in case
I think there is one more thing to check.
I haven't used sqlite, but with other databases I have
used there is often a large variance in times from
one select to the next.  Can you
repeat these tests with a timeit().repeat  and give the
minimum?

As an aside, your database is running on a local disk, right, so
the overehead of retrieving data is minimized here?
For my tests I think I am data retrieval limited because I
get exactly the same time for the equivalent of retrieve1
and retrieve2.

Erin

-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your
opinions on IT & business topics through brief surveys - and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV



From david at ar.media.kyoto-u.ac.jp  Tue Nov 14 21:50:28 2006
From: david at ar.media.kyoto-u.ac.jp (David Cournapeau)
Date: Wed, 15 Nov 2006 11:50:28 +0900
Subject: numpy, matplotlib, scipy
In-Reply-To: <03c401c70847$6def0670$0800a8c0@yairlap>
References: <mailman.70135.1161983292.22397.numpy-discussion@lists.sourceforge.net>
	<03c401c70847$6def0670$0800a8c0@yairlap>
Message-ID: <455A8074.5010400@ar.media.kyoto-u.ac.jp>

Kenny Ortmann wrote:
> I am sorry if this has come up before, I've found some stuff on it but just 
> want to be clear.
> What are the latest versions of numpy matplotlib and scipy that work 
> together.
> I have seen that numpy 1.0rc2 works with the latest scipy until 0.5.2 comes 
> out.
> now with numpy and matplotlib?  I saw that matplotlib 87.5 works with i 
> think the 1.0rc2 also, but i read that mpl was going to release right after 
> numpy 1.0 was released so that they would be compatible.
> I am just trying to upgrade these packages before I create an executable of 
> the program I am working on, and I am running into problems with
> "from matplotlib._ns_nxutils import *
>    ImportError: numpy.core.multiarray failed to imort"
It looks like you have a problem with numpy. First, I would try import 
numpy from a python prompt, to see what's wrong. I would also check that 
numpy is set as the array package in matplolibrc file.

Second, I would check that I install everything "cleanly", that is:
    - first, remove all packages (numpy, scipy and mpl) in site-packages 
directory
    - remove the build directory in each package
    - then build from scratch + test  each package: numpy first, then 
mpl, then scipy.


Concerning versions, I regularly rebuild scipy and numpy from SVN (but 
always test them with import package; package.test(100) to test 
everything), but uses the latest release of mpl (0.87.7 as we speak), on 
linux, without any problem

cheers,

David
>
> -Kenny 
>
>
> -------------------------------------------------------------------------
> Take Surveys. Earn Cash. Influence the Future of IT
> Join SourceForge.net's Techsay panel and you'll get the chance to share your
> opinions on IT & business topics through brief surveys - and earn cash
> http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV
> _______________________________________________
> Numpy-discussion mailing list
> Numpy-discussion at lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/numpy-discussion
>
>


-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your
opinions on IT & business topics through brief surveys - and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV



From kwgoodman at gmail.com  Tue Nov 14 21:50:20 2006
From: kwgoodman at gmail.com (Keith Goodman)
Date: Tue, 14 Nov 2006 18:50:20 -0800
Subject: mysql -> record array
In-Reply-To: <331116dc0611141826x47ed9ccbpf5ff038c79967a44@mail.gmail.com>
References: <87slglwwwm.fsf@peds-pc311.bsd.uchicago.edu>
	<331116dc0611141408q2d7c9431s38a0f4c4b3f2db25@mail.gmail.com>
	<87velh4opc.fsf@peds-pc311.bsd.uchicago.edu>
	<455A62FF.6040706@ieee.org> <455A76C5.9050508@ieee.org>
	<331116dc0611141826x47ed9ccbpf5ff038c79967a44@mail.gmail.com>
Message-ID: <f4f93d420611141850v5c3fa8d9nf4f047072ee3281c@mail.gmail.com>

On 11/14/06, Erin Sheldon <erin.sheldon at gmail.com> wrote:

> As an aside, your database is running on a local disk, right, so
> the overehead of retrieving data is minimized here?
> For my tests I think I am data retrieval limited because I
> get exactly the same time for the equivalent of retrieve1
> and retrieve2.

Tim created the database in memory (conn =
sqlite3.connect(':memory:')), which is much faster than disk.

-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your
opinions on IT & business topics through brief surveys - and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV



From tim.hochberg at ieee.org  Tue Nov 14 23:14:10 2006
From: tim.hochberg at ieee.org (Tim Hochberg)
Date: Tue, 14 Nov 2006 21:14:10 -0700
Subject: mysql -> record array
In-Reply-To: <331116dc0611141826x47ed9ccbpf5ff038c79967a44@mail.gmail.com>
References: <87slglwwwm.fsf@peds-pc311.bsd.uchicago.edu>	<331116dc0611141408q2d7c9431s38a0f4c4b3f2db25@mail.gmail.com>	<87velh4opc.fsf@peds-pc311.bsd.uchicago.edu>	<455A62FF.6040706@ieee.org>
	<455A76C5.9050508@ieee.org>
	<331116dc0611141826x47ed9ccbpf5ff038c79967a44@mail.gmail.com>
Message-ID: <455A9412.5020008@ieee.org>

Erin Sheldon wrote:
> On 11/14/06, Tim Hochberg <tim.hochberg at ieee.org> wrote:
>   
>> Tim Hochberg wrote:
>>     
>>> [SNIP]
>>>
>>> I'm no database user, but a glance at the at the docs seems to indicate
>>> that you can get your data via an iterator (by iterating over the cursor
>>> or some such db mumbo jumbo) rather than slurping up the whole list up
>>> at once. If so, then you'll save a lot of memory by passing the iterator
>>> straight to fromiter. It may even be faster, who knows.
>>>
>>> Accessing the db via the iterator could be a performance killer, but
>>> it's almost certainly worth trying as it could a few megabytes of
>>> storage and that in turn might speed things up.
>>>       
>> Assuming that I didn't mess this up too badly, it appears that using the
>> iterator directly with fromiter is significantly faster than the next
>> best solution (about 45%). The fromiter wrapping a list solution come in
>> second, followed by numarray.array and finally way in the back,
>> numpy.array. Here's the numbers:
>>
>>     retrieve1 took 0.902922857514 seconds
>>     retrieve2 took 1.31245870634 seconds
>>     retrieve3 took 1.51207569677 seconds
>>     retrieve4 took 8.71539930354 seconds
>>     
>
> Interesting results Tim.  From Pierre's results
> we saw that fromiter is the fastest way to get data
> into arrays.  With your results we see there is a
> difference between iterating over the cursor and
> doing a fetchall() as well. Surprisingly, running
> the cursor is faster.
>
> This must come not from the data retrieval rate but
> from creating the copies in memory.
I imagine that is correct. In particular, skipping the making of the 
list avoids the creation of 1e6 Python floats, which is going to result 
in a lot of memory allocation.

>  But just in case
> I think there is one more thing to check.
> I haven't used sqlite, but with other databases I have
> used there is often a large variance in times from
> one select to the next.  Can you
> repeat these tests with a timeit().repeat  and give the
> minimum?
>   
Sure. Here's two sets of numbers. The first is for repeat(3,1) and the 
second for repeat (3,3).

    retrieve1 [0.91198546183942375, 0.9042411814909439, 0.90411518782415001]
    retrieve2 [0.98355349632425515, 0.95424502276127754,
    0.94714328217692412]
    retrieve3 [1.2227562441595268, 1.2195848913758596, 1.2206193803961156]
    retrieve4 [8.4344040932576547, 8.3556245276983532, 8.3568341786456131]

    retrieve1 [2.7317457945074026, 2.7274656415829384, 2.7250913174719109]
    retrieve2 [2.8857103346933783, 2.8379299603720582, 2.8386803350705136]
    retrieve3 [3.6870535221655203, 3.8980253076857565, 3.7002303365371887]
    retrieve4 [25.138646950939304, 25.06737169109482, 25.052789390830412]

The timings of these are pretty consistent with each other with the 
previous runs except that the difference between retrieve1 and retrieve2 
has disappeared. In fact, all of the runs that produce lists have gotten 
faster by about the same amount.. Odd! A little digging reveals that 
timeit turns off garbage collection to make things more repeatable. 
Turning gc back on yields the following numbers for repeat(3,1):

    retrieve1 [0.92517736192728406, 0.92109667569481601,
    0.92390960303614023]
    retrieve2 [1.3018456256311914, 1.2277141368525903, 1.2929785768861706]
    retrieve3 [1.5309831277438946, 1.4998853206203577, 1.5601200711263488]
    retrieve4 [8.6400394463542227, 8.7022300320292061, 8.6807761880350682]

So there we are, back to our original numbers. This also reveals that 
the majority of the time difference between retrieve1 and retrieve2 *is* 
memory related. However, it's the deallocation (or more precisely 
garbage collection) of all those floats that is the killer. Here's what 
the timeit routines looked like:

    if __name__ == "__main__":
        for name in ['retrieve1', 'retrieve2', 'retrieve3', 'retrieve4']:
            print name, timeit.Timer("%s(conn)" % name, "gc.enable();
    from scratch import sqlite3, %s, setup; conn =
    sqlite3.connect(':memory:'); setup(conn)" % name).repeat(3, 1)

> As an aside, your database is running on a local disk, right, so
> the overehead of retrieving data is minimized here?
> For my tests I think I am data retrieval limited because I
> get exactly the same time for the equivalent of retrieve1
> and retrieve2.
>   
As Keith pointed out, I'm keeping the database in memory (although 
there's a very good chance some of it is actually swapped to disk) so 
it's probably relatively fast. On the other hand, if you are using 
timeit to make your measurements you could be running into the (lack of) 
garbage collection issue I mention above.

-tim



-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your
opinions on IT & business topics through brief surveys - and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV



From erin.sheldon at gmail.com  Tue Nov 14 23:47:05 2006
From: erin.sheldon at gmail.com (Erin Sheldon)
Date: Tue, 14 Nov 2006 23:47:05 -0500
Subject: mysql -> record array
In-Reply-To: <455A9412.5020008@ieee.org>
References: <87slglwwwm.fsf@peds-pc311.bsd.uchicago.edu>
	<331116dc0611141408q2d7c9431s38a0f4c4b3f2db25@mail.gmail.com>
	<87velh4opc.fsf@peds-pc311.bsd.uchicago.edu>
	<455A62FF.6040706@ieee.org> <455A76C5.9050508@ieee.org>
	<331116dc0611141826x47ed9ccbpf5ff038c79967a44@mail.gmail.com>
	<455A9412.5020008@ieee.org>
Message-ID: <331116dc0611142047t2be5cdcs8c24182e5afe4407@mail.gmail.com>

On 11/14/06, Tim Hochberg <tim.hochberg at ieee.org> wrote:
SNIP
> > Interesting results Tim.  From Pierre's results
> > we saw that fromiter is the fastest way to get data
> > into arrays.  With your results we see there is a
> > difference between iterating over the cursor and
> > doing a fetchall() as well. Surprisingly, running
> > the cursor is faster.
> >
> > This must come not from the data retrieval rate but
> > from creating the copies in memory.
> I imagine that is correct. In particular, skipping the making of the
> list avoids the creation of 1e6 Python floats, which is going to result
> in a lot of memory allocation.
>
> >  But just in case
> > I think there is one more thing to check.
> > I haven't used sqlite, but with other databases I have
> > used there is often a large variance in times from
> > one select to the next.  Can you
> > repeat these tests with a timeit().repeat  and give the
> > minimum?
> >
> Sure. Here's two sets of numbers. The first is for repeat(3,1) and the
> second for repeat (3,3).
>
>     retrieve1 [0.91198546183942375, 0.9042411814909439, 0.90411518782415001]
>     retrieve2 [0.98355349632425515, 0.95424502276127754,
>     0.94714328217692412]
>     retrieve3 [1.2227562441595268, 1.2195848913758596, 1.2206193803961156]
>     retrieve4 [8.4344040932576547, 8.3556245276983532, 8.3568341786456131]
>
>     retrieve1 [2.7317457945074026, 2.7274656415829384, 2.7250913174719109]
>     retrieve2 [2.8857103346933783, 2.8379299603720582, 2.8386803350705136]
>     retrieve3 [3.6870535221655203, 3.8980253076857565, 3.7002303365371887]
>     retrieve4 [25.138646950939304, 25.06737169109482, 25.052789390830412]
>
> The timings of these are pretty consistent with each other with the
> previous runs except that the difference between retrieve1 and retrieve2
> has disappeared. In fact, all of the runs that produce lists have gotten
> faster by about the same amount.. Odd! A little digging reveals that
> timeit turns off garbage collection to make things more repeatable.
> Turning gc back on yields the following numbers for repeat(3,1):
>
>     retrieve1 [0.92517736192728406, 0.92109667569481601,
>     0.92390960303614023]
>     retrieve2 [1.3018456256311914, 1.2277141368525903, 1.2929785768861706]
>     retrieve3 [1.5309831277438946, 1.4998853206203577, 1.5601200711263488]
>     retrieve4 [8.6400394463542227, 8.7022300320292061, 8.6807761880350682]
>
> So there we are, back to our original numbers. This also reveals that
> the majority of the time difference between retrieve1 and retrieve2 *is*
> memory related. However, it's the deallocation (or more precisely
> garbage collection) of all those floats that is the killer. Here's what
> the timeit routines looked like:
>
>     if __name__ == "__main__":
>         for name in ['retrieve1', 'retrieve2', 'retrieve3', 'retrieve4']:
>             print name, timeit.Timer("%s(conn)" % name, "gc.enable();
>     from scratch import sqlite3, %s, setup; conn =
>     sqlite3.connect(':memory:'); setup(conn)" % name).repeat(3, 1)
>
> > As an aside, your database is running on a local disk, right, so
> > the overehead of retrieving data is minimized here?
> > For my tests I think I am data retrieval limited because I
> > get exactly the same time for the equivalent of retrieve1
> > and retrieve2.
> >
> As Keith pointed out, I'm keeping the database in memory (although
> there's a very good chance some of it is actually swapped to disk) so
> it's probably relatively fast. On the other hand, if you are using
> timeit to make your measurements you could be running into the (lack of)
> garbage collection issue I mention above.
>

I checked and for my real situation I am totally limited by the time
to retrieve the data.  From these tests I think this will
probably be true even if the data is  on a local disk.

I think these experiments show that iterating over the
cursor is the best approach.  It is better from a memory
point of view and is probably also the fastest.



We should still resolve the slowness for the array()
function however when converting lists of tuples.
I will file a ticket if no one else has.

Erin

-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your
opinions on IT & business topics through brief surveys - and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV



From tim.hochberg at ieee.org  Tue Nov 14 23:55:56 2006
From: tim.hochberg at ieee.org (Tim Hochberg)
Date: Tue, 14 Nov 2006 21:55:56 -0700
Subject: mysql -> record array
In-Reply-To: <455A9412.5020008@ieee.org>
References: <87slglwwwm.fsf@peds-pc311.bsd.uchicago.edu>	<331116dc0611141408q2d7c9431s38a0f4c4b3f2db25@mail.gmail.com>	<87velh4opc.fsf@peds-pc311.bsd.uchicago.edu>	<455A62FF.6040706@ieee.org>	<455A76C5.9050508@ieee.org>	<331116dc0611141826x47ed9ccbpf5ff038c79967a44@mail.gmail.com>
	<455A9412.5020008@ieee.org>
Message-ID: <455A9DDC.9040700@ieee.org>

Tim Hochberg wrote:
> [CHOP]
>
> The timings of these are pretty consistent with each other with the 
> previous runs except that the difference between retrieve1 and retrieve2 
> has disappeared. In fact, all of the runs that produce lists have gotten 
> faster by about the same amount.. Odd! A little digging reveals that 
> timeit turns off garbage collection to make things more repeatable. 
> Turning gc back on yields the following numbers for repeat(3,1):
>
>     retrieve1 [0.92517736192728406, 0.92109667569481601,
>     0.92390960303614023]
>     retrieve2 [1.3018456256311914, 1.2277141368525903, 1.2929785768861706]
>     retrieve3 [1.5309831277438946, 1.4998853206203577, 1.5601200711263488]
>     retrieve4 [8.6400394463542227, 8.7022300320292061, 8.6807761880350682]
>
> So there we are, back to our original numbers. This also reveals that 
> the majority of the time difference between retrieve1 and retrieve2 *is* 
> memory related. However, it's the deallocation (or more precisely 
> garbage collection) of all those floats that is the killer. 
I just realized that this sounds sort of misleading. In both cases a 
million floats are allocated and deallocated. However, in retrieve1 only 
two of those million are alive at any one time, so Python will just keep 
reusing the same two chunks of memory for all 500,000 pairs (ditto for 
the 500,000 tuples that are created). In the other cases, all million 
floats will be alive at once, requiring much more memory and possibly 
swapping to disk. Unsurprisingly, the second case is slower, but the 
details aren't clear. In particular why is it the deallocation that is slow?

Another mystery is why gc matters at all. None of the obvious actors are 
involved in cycles so they would normally go away due to reference 
counting even with gc turned off. My rather uninformed guess is that the 
cursor or the connection holds onto the list (caching it for later 
perhaps) and that cursor/connection is involved in some sort of cycle. 
This would keep the list alive until the gc ran.

-tim


[CHOP]




-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your
opinions on IT & business topics through brief surveys - and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV



From kwgoodman at gmail.com  Wed Nov 15 12:55:38 2006
From: kwgoodman at gmail.com (Keith Goodman)
Date: Wed, 15 Nov 2006 09:55:38 -0800
Subject: Could numpy.matlib.nanmax return a matrix?
In-Reply-To: <200611121808.51298.pgmdevlist@gmail.com>
References: <f4f93d420611121357o5dda5950g4fdcdc2f657ee654@mail.gmail.com>
	<ce557a360611121408k6bae03fds9b2e979d786bb74e@mail.gmail.com>
	<200611121808.51298.pgmdevlist@gmail.com>
Message-ID: <f4f93d420611150955u537f5bcewc9567ed041ac1d43@mail.gmail.com>

On 11/12/06, Pierre GM <pgmdevlist at gmail.com> wrote:
> On Sunday 12 November 2006 17:08, A. M. Archibald wrote:
> > On 12/11/06, Keith Goodman <kwgoodman at gmail.com> wrote:
> > > Is anybody interested in making x.max() and nanmax() behave the same
> > > for matrices, except for the NaN part? That is, make
> > > numpy.matlib.nanmax return a matrix instead of an array.
>
> Or, you could use masked arrays... In the new implementation, you can add a
> mask to a subclassed array (such as matrix) to get a regular masked array. If
> you fill this masked array, you get an array of the same subclass.
>
> >>> import numpy as N
> >>> import numpy.matlib as M
> >>> import maskedarray as MA
> >>> x=M.rand(3,3)
> >>> assert isinstance(x.max(0), M.matrix)
> >>> assert isinstance(N.max(x,0), M.matrix)
> >>> assert  isinstance(MA.max(x,0).filled(0), M.matrix)
> >>> assert isinstance(MA.max(x,0)._data, M.matrix)
>
> >>> x[-1,-1] = N.nan
> >>> tmp = MA.max(MA.array(x,mask=N.isnan(x)), 0)
> >>> assert (tmp == N.nanmax(x,0)).all()
> >>> assert isinstance(tmp.filled(0), M.matrix)

I didn't know you could use masked arrays with matrices. I guess I
took the name literally.

I think an easier way to use masked arrays would be to introduce a new
thing called mis.

I could make a regular matrix

x  = M.rand(3,3)

and assign a missing value

x[0,0] = M.mis

x would then behave as a missing array matrix.

I could also do

x[M.isnan(x)] = M.mis

or

x[mask] = M.mis

To get the mask from x:

x.mask

or

M.ismis(x)

I think that would make missing arrays accessible to everyone.

-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your
opinions on IT & business topics through brief surveys - and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV



From Korea at ophouston.com  Wed Nov 15 14:43:51 2006
From: Korea at ophouston.com (team playing)
Date: Wed, 15 Nov 2006 19:43:51 -0000
Subject: increase success Norways
Message-ID: <000c01c708ee$606467a0$00000000@helm>


Few or like Rajasthan Permitted?
Wage wages above in. Parenthood policies nb bbc means Firsttime am us Articles unsourced.
Five event serious homethe Gambia!
Related day is private! Excited of Clemence dismisses England a Play back foot.
Months per cost being of shared between State encourage.
Their current employer.
Wife gives birthiran weeksiraq Israel. Designed halt decline malefemale.
Parentsas must been a taking expires eighth Maximum.
Davydenko Highlights Features am.
Parental leave Wikipedia the am. Flat rate weeksfour South Three leavesudan a Swaziland Togo ten or.
Known in do or exist Icelandic no Esttl India Ministry?
Patriots name Test. Flat rate weeksfour South Three leavesudan a Swaziland Togo ten or.
Ashes callup Graveney discuss Pakistan doping or. Burundi Cameroon up!
Islands sri or Lanka a Syria am.
Column Pick Comment in fire Wednesdays photos am. Syria am Thailand then daysunited Emirates of Vietnam Yemen Belarus is Belgium.
Birthiran weeksiraq Israel in Japan Jordan is Korea.
Such as Australia United States in was greatly expanded or.
Week Belize Bolivia national min wage wages is?
Wage wages above in.
Kk What people sayingfans views bc.
Labor a Lawscanada indicates.
American Veteran Testaverde joins Patriots of name Test squad!
Labor a Lawscanada indicates.
Malta first after Portugal.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20061115/474001db/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: only.gif
Type: image/gif
Size: 13686 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20061115/474001db/attachment-0001.gif>
-------------- next part --------------
-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your
opinions on IT & business topics through brief surveys - and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV
-------------- next part --------------
_______________________________________________
Numpy-discussion mailing list
Numpy-discussion at lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/numpy-discussion

From myeates at jpl.nasa.gov  Wed Nov 15 15:30:27 2006
From: myeates at jpl.nasa.gov (Mathew Yeates)
Date: Wed, 15 Nov 2006 12:30:27 -0800
Subject: reference count problems
Message-ID: <455B78E3.6020402@jpl.nasa.gov>

Hi
I'm running a 64 bit Python 2.5 on an x86 with Solaris. I have a 
function I call over 2^32 times and eventually I run out of memory.

The function is
def make_B(deltadates):
    numcols=deltadates.shape[0]
    B=numpy.zeros((numcols,numcols))
    for ind in range(0,numcols): #comment out this loop and all is good
        B[ind,0:numcols] = deltadates[0:numcols]
    return B


If I comment out the loop lines, my memory is okay. I'm guessing that a 
reference is being added to "deltadates" and that the reference count is 
going above 2^32 and reseting. Anybody have any ideas about how I can 
cure this? Is Numpy increasing the reference count here?

Mathew



-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your
opinions on IT & business topics through brief surveys - and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV



From robert.kern at gmail.com  Wed Nov 15 15:33:52 2006
From: robert.kern at gmail.com (Robert Kern)
Date: Wed, 15 Nov 2006 14:33:52 -0600
Subject: reference count problems
In-Reply-To: <455B78E3.6020402@jpl.nasa.gov>
References: <455B78E3.6020402@jpl.nasa.gov>
Message-ID: <ejftjs$9kq$1@sea.gmane.org>

Mathew Yeates wrote:
> Hi
> I'm running a 64 bit Python 2.5 on an x86 with Solaris. I have a 
> function I call over 2^32 times and eventually I run out of memory.
> 
> The function is
> def make_B(deltadates):
>     numcols=deltadates.shape[0]
>     B=numpy.zeros((numcols,numcols))
>     for ind in range(0,numcols): #comment out this loop and all is good
>         B[ind,0:numcols] = deltadates[0:numcols]
>     return B
> 
> 
> If I comment out the loop lines, my memory is okay. I'm guessing that a 
> reference is being added to "deltadates" and that the reference count is 
> going above 2^32 and reseting. Anybody have any ideas about how I can 
> cure this? Is Numpy increasing the reference count here?

Can you give us a small but complete and self-contained script that demonstrates
the problem?

-- 
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless enigma
 that is made terrible by our own mad attempt to interpret it as though it had
 an underlying truth."
  -- Umberto Eco


-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your
opinions on IT & business topics through brief surveys - and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV



From myeates at jpl.nasa.gov  Wed Nov 15 15:45:58 2006
From: myeates at jpl.nasa.gov (Mathew Yeates)
Date: Wed, 15 Nov 2006 12:45:58 -0800
Subject: reference count problems
In-Reply-To: <ejftjs$9kq$1@sea.gmane.org>
References: <455B78E3.6020402@jpl.nasa.gov> <ejftjs$9kq$1@sea.gmane.org>
Message-ID: <455B7C86.7090609@jpl.nasa.gov>

Robert Kern wrote:
> Mathew Yeates wrote:
>   
>> Hi
>> I'm running a 64 bit Python 2.5 on an x86 with Solaris. I have a 
>> function I call over 2^32 times and eventually I run out of memory.
>>
>> The function is
>> def make_B(deltadates):
>>     numcols=deltadates.shape[0]
>>     B=numpy.zeros((numcols,numcols))
>>     for ind in range(0,numcols): #comment out this loop and all is good
>>         B[ind,0:numcols] = deltadates[0:numcols]
>>     return B
>>
>>
>> If I comment out the loop lines, my memory is okay. I'm guessing that a 
>> reference is being added to "deltadates" and that the reference count is 
>> going above 2^32 and reseting. Anybody have any ideas about how I can 
>> cure this? Is Numpy increasing the reference count here?
>>     
>
> Can you give us a small but complete and self-contained script that demonstrates
> the problem?
>
>   
I'll try. But its in a complex program. BTW - I tried
B[ind,0:numcols] = deltadates[0:numcols].copy()
but that didn't work either.

Mathew


-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your
opinions on IT & business topics through brief surveys - and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV



From stefan at sun.ac.za  Wed Nov 15 15:51:49 2006
From: stefan at sun.ac.za (Stefan van der Walt)
Date: Wed, 15 Nov 2006 22:51:49 +0200
Subject: reference count problems
In-Reply-To: <ejftjs$9kq$1@sea.gmane.org>
References: <455B78E3.6020402@jpl.nasa.gov> <ejftjs$9kq$1@sea.gmane.org>
Message-ID: <20061115205149.GD1975@mentat.za.net>

On Wed, Nov 15, 2006 at 02:33:52PM -0600, Robert Kern wrote:
> Mathew Yeates wrote:
> > Hi
> > I'm running a 64 bit Python 2.5 on an x86 with Solaris. I have a 
> > function I call over 2^32 times and eventually I run out of memory.
> > 
> > The function is
> > def make_B(deltadates):
> >     numcols=deltadates.shape[0]
> >     B=numpy.zeros((numcols,numcols))
> >     for ind in range(0,numcols): #comment out this loop and all is good
> >         B[ind,0:numcols] = deltadates[0:numcols]
> >     return B
> > 
> > 
> > If I comment out the loop lines, my memory is okay. I'm guessing that a 
> > reference is being added to "deltadates" and that the reference count is 
> > going above 2^32 and reseting. Anybody have any ideas about how I can 
> > cure this? Is Numpy increasing the reference count here?
> 
> Can you give us a small but complete and self-contained script that demonstrates
> the problem?

I think this might be related to ticket #378:

http://projects.scipy.org/scipy/numpy/ticket/378

Cheers
St?fan

-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your
opinions on IT & business topics through brief surveys - and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV



From myeates at jpl.nasa.gov  Wed Nov 15 16:37:24 2006
From: myeates at jpl.nasa.gov (Mathew Yeates)
Date: Wed, 15 Nov 2006 13:37:24 -0800
Subject: program attached Re:  reference count problems
In-Reply-To: <20061115205149.GD1975@mentat.za.net>
References: <455B78E3.6020402@jpl.nasa.gov> <ejftjs$9kq$1@sea.gmane.org>
	<20061115205149.GD1975@mentat.za.net>
Message-ID: <455B8894.7000904@jpl.nasa.gov>

Stefan van der Walt wrote:
> On Wed, Nov 15, 2006 at 02:33:52PM -0600, Robert Kern wrote:
>   
>> Mathew Yeates wrote:
>>     
>>> Hi
>>> I'm running a 64 bit Python 2.5 on an x86 with Solaris. I have a 
>>> function I call over 2^32 times and eventually I run out of memory.
>>>
>>> The function is
>>> def make_B(deltadates):
>>>     numcols=deltadates.shape[0]
>>>     B=numpy.zeros((numcols,numcols))
>>>     for ind in range(0,numcols): #comment out this loop and all is good
>>>         B[ind,0:numcols] = deltadates[0:numcols]
>>>     return B
>>>
>>>
>>> If I comment out the loop lines, my memory is okay. I'm guessing that a 
>>> reference is being added to "deltadates" and that the reference count is 
>>> going above 2^32 and reseting. Anybody have any ideas about how I can 
>>> cure this? Is Numpy increasing the reference count here?
>>>       
>> Can you give us a small but complete and self-contained script that demonstrates
>> the problem?
>>     
>
> I think this might be related to ticket #378:
>
> http://projects.scipy.org/scipy/numpy/ticket/378
>
> Cheers
> St?fan
>   
okay. attached is the smallest program I could make. Before running you 
will need to create a file named biggie with 669009000 non zero floats.

Mathew
-------------- next part --------------
An embedded and charset-unspecified text was scrubbed...
Name: memsuck.py
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20061115/d59424e4/attachment-0001.ksh>
-------------- next part --------------
-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your
opinions on IT & business topics through brief surveys - and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV
-------------- next part --------------
_______________________________________________
Numpy-discussion mailing list
Numpy-discussion at lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/numpy-discussion

From robert.kern at gmail.com  Wed Nov 15 16:41:44 2006
From: robert.kern at gmail.com (Robert Kern)
Date: Wed, 15 Nov 2006 15:41:44 -0600
Subject: program attached Re: reference count problems
In-Reply-To: <455B8894.7000904@jpl.nasa.gov>
References: <455B78E3.6020402@jpl.nasa.gov>
	<ejftjs$9kq$1@sea.gmane.org>	<20061115205149.GD1975@mentat.za.net>
	<455B8894.7000904@jpl.nasa.gov>
Message-ID: <ejg1j4$o1k$1@sea.gmane.org>

Mathew Yeates wrote:

> def delta2day1(delta):
>     return delta.days/365.0
> deltas2days=numpy.frompyfunc(delta2day1,1,1)

If I had to guess where the problem is, it's here. frompyfunc() and vectorize()
have always been tricky beasts to get right.

-- 
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless enigma
 that is made terrible by our own mad attempt to interpret it as though it had
 an underlying truth."
  -- Umberto Eco


-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your
opinions on IT & business topics through brief surveys - and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV



From tim.hochberg at ieee.org  Wed Nov 15 17:15:18 2006
From: tim.hochberg at ieee.org (Tim Hochberg)
Date: Wed, 15 Nov 2006 15:15:18 -0700
Subject: program attached Re: reference count problems
In-Reply-To: <ejg1j4$o1k$1@sea.gmane.org>
References: <455B78E3.6020402@jpl.nasa.gov>	<ejftjs$9kq$1@sea.gmane.org>	<20061115205149.GD1975@mentat.za.net>	<455B8894.7000904@jpl.nasa.gov>
	<ejg1j4$o1k$1@sea.gmane.org>
Message-ID: <455B9176.9020406@ieee.org>

Robert Kern wrote:
> Mathew Yeates wrote:
>
>   
>> def delta2day1(delta):
>>     return delta.days/365.0
>> deltas2days=numpy.frompyfunc(delta2day1,1,1)
>>     
>
> If I had to guess where the problem is, it's here. frompyfunc() and vectorize()
> have always been tricky beasts to get right.
>   
<curnudgen mode>

IMO, frompyfunc is an attractive nuisance. It doesn't magically make 
scalar Python functions fast, as people seem to assume, and it prevents 
people from figuring out how to write vectorized functions in the many 
cases where that's practicable. And it sounds like it may be buggy to boot.

In this case, I don't know that you can easily vectorize this by hand, 
but there are many ways that it could be rewritten to avoid frompyfunc. 
For example:

    def deltas2days(seq):
        return numpy.fromter((x.days for x in seq), dtype=float,
    count=len(seq))

One line shorter, about equally opaque and less likely to have 
mysterious bugs.

</curmudgen mode>

-tim


-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your
opinions on IT & business topics through brief surveys - and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV



From myeates at jpl.nasa.gov  Wed Nov 15 17:20:37 2006
From: myeates at jpl.nasa.gov (Mathew Yeates)
Date: Wed, 15 Nov 2006 14:20:37 -0800
Subject: program attached Re: reference count problems
In-Reply-To: <ejg1j4$o1k$1@sea.gmane.org>
References: <455B78E3.6020402@jpl.nasa.gov>	<ejftjs$9kq$1@sea.gmane.org>	<20061115205149.GD1975@mentat.za.net>	<455B8894.7000904@jpl.nasa.gov>
	<ejg1j4$o1k$1@sea.gmane.org>
Message-ID: <455B92B5.4050908@jpl.nasa.gov>

Robert Kern wrote:
> Mathew Yeates wrote:
>
>   
>> def delta2day1(delta):
>>     return delta.days/365.0
>> deltas2days=numpy.frompyfunc(delta2day1,1,1)
>>     
>
> If I had to guess where the problem is, it's here. frompyfunc() and vectorize()
> have always been tricky beasts to get right.
>
>   
It appears the problem is, in fact, with frompyfunc. I'm still running 
but I'm not seeing the immediate loss of memory as I was before. I 
replaced the frompyfunc with a simple loop.

Thanks for all  the help.
Mathew



-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your
opinions on IT & business topics through brief surveys - and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV



From pgmdevlist at gmail.com  Wed Nov 15 21:00:20 2006
From: pgmdevlist at gmail.com (Pierre GM)
Date: Wed, 15 Nov 2006 21:00:20 -0500
Subject: Could numpy.matlib.nanmax return a matrix?
In-Reply-To: <f4f93d420611150955u537f5bcewc9567ed041ac1d43@mail.gmail.com>
References: <f4f93d420611121357o5dda5950g4fdcdc2f657ee654@mail.gmail.com>
	<200611121808.51298.pgmdevlist@gmail.com>
	<f4f93d420611150955u537f5bcewc9567ed041ac1d43@mail.gmail.com>
Message-ID: <200611152100.20522.pgmdevlist@gmail.com>

On Wednesday 15 November 2006 12:55, Keith Goodman wrote:
> I didn't know you could use masked arrays with matrices. I guess I
> took the name literally.

:) 
Please check the developer zone:
http://projects.scipy.org/scipy/numpy/wiki/MaskedArray
for an alternative implementation of masked arrays that support subclasses of 
ndarray.

> I think an easier way to use masked arrays would be to introduce a new
> thing called mis.
>
> I could make a regular matrix
>
> x  = M.rand(3,3)
>
> and assign a missing value
>
> x[0,0] = M.mis
>
> x would then behave as a missing array matrix.
....
> I think that would make missing arrays accessible to everyone.

Well, there's already something like that, sort of: MA.masked, or 
MA.masked_singleton. The emphasis here is on "sort of".

That works well if x is already a masked array. Else, a "ValueError: setting 
an array element with a sequence" is raised. I haven't tried to find where 
the problem comes from (ndarray.__setitem__ ? The masked_singleton larger 
than it seems ?), but I wonder whether it's an issue worth solving.

If you want to get a masked_matrix from x, just type x=masked_array(x). You 
won't be able to access some specific matrix attributes (A, T), at least 
directly, but you can fill your masked_matrix and get a matrix back. And 
multiplication of two masked_matrices work as expected ! 

The main advantage of this approach is that we don't overload ndarray or 
matrices, the work is solely on the masked_array side.


-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your
opinions on IT & business topics through brief surveys - and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV



From pgmdevlist at gmail.com  Thu Nov 16 12:01:08 2006
From: pgmdevlist at gmail.com (Pierre GM)
Date: Thu, 16 Nov 2006 12:01:08 -0500
Subject: Defining custom types
In-Reply-To: <20061116164445.GB27681@crater.logilab.fr>
References: <4f7636bf0610261526t5b7b4191x370308df775799a9@mail.gmail.com>
	<20061116164445.GB27681@crater.logilab.fr>
Message-ID: <200611161201.09119.pgmdevlist@gmail.com>

On Thursday 16 November 2006 11:44, David Douard wrote:
> Hi, just to ask you: how is the work going on encapsulatinsg mx.DateTime
> as a native numpy type?
> And most important: is the code available somewhere? I am also
> interested in using DateTime objects in numpy arrays. For now, I've
> always used arrays of floats (using gmticks values of dates).

And I, as arrays of objects (well, I wrote a subclass to deal with dates, 
where each element is a datetime object, with methods to translate to floats 
or strings , but it's far from optimal...). I'd also be quite interested in 
checking what has been done.

-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your
opinions on IT & business topics through brief surveys - and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV



From cjw at sympatico.ca  Thu Nov 16 12:40:25 2006
From: cjw at sympatico.ca (Colin J. Williams)
Date: Thu, 16 Nov 2006 12:40:25 -0500
Subject: Defining custom types
In-Reply-To: <20061116164445.GB27681@crater.logilab.fr>
References: <4f7636bf0610261526t5b7b4191x370308df775799a9@mail.gmail.com>
	<20061116164445.GB27681@crater.logilab.fr>
Message-ID: <455CA289.1030101@sympatico.ca>

An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20061116/bdb2c6a2/attachment-0001.html>
-------------- next part --------------
-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your
opinions on IT & business topics through brief surveys - and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV
-------------- next part --------------
_______________________________________________
Numpy-discussion mailing list
Numpy-discussion at lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/numpy-discussion

From tim.hochberg at ieee.org  Thu Nov 16 13:05:22 2006
From: tim.hochberg at ieee.org (Tim Hochberg)
Date: Thu, 16 Nov 2006 11:05:22 -0700
Subject: mysql -> record array
In-Reply-To: <200611161413.31917.faltet@carabos.com>
References: <87slglwwwm.fsf@peds-pc311.bsd.uchicago.edu>	<331116dc0611141408q2d7c9431s38a0f4c4b3f2db25@mail.gmail.com>
	<200611161413.31917.faltet@carabos.com>
Message-ID: <455CA862.2060305@ieee.org>

Francesc Altet wrote:
> A Dimarts 14 Novembre 2006 23:08, Erin Sheldon escrigu?:
>   
>> On 11/14/06, John Hunter <jdhunter at ace.bsd.uchicago.edu> wrote:
>>     
>>> Has anyone written any code to facilitate dumping mysql query results
>>> (mainly arrays of floats) into numpy arrays directly at the extension
>>> code layer.  The query results->list->array conversion can be slow.
>>>
>>> Ideally, one could do this semi-automagically with record arrays and
>>> table introspection....
>>>       
>> I've been considering this as well.  I use both postgres and Oracle
>> in my work, and I have been using the python interfaces (cx_Oracle
>> and pgdb) to get result lists and convert to numpy arrays.
>>
>> The question I have been asking myself is "what is the advantage
>> of such an approach?".  It would be faster, but by how
>> much?  Presumably the bottleneck for most applications will
>> be data retrieval rather than data copying in memory.
>>     
>
> Well, that largely depends on your pattern to access the data in your
> database. If you are accessing to regions of your database that have a
> high degree of spatial locality (i.e. they are located in equal or
> very similar places), the data is most probably already in memory (in
> your filesystem cache or maybe in your database cache) and the
> bottleneck will become the memory access. Of course, if you don't have
> such a spatial locality in the access pattern, then the bottleneck
> will be the disk.
>
> Just to see how DB 2.0 could benefit from adopting record arrays as
> input buffers, I've done a comparison between SQLite3 and PyTables.
> PyTables doesn't suport DB 2.0 as such, but it does use record arrays
> as buffers internally so as to read data in an efficient way (there
> should be other databases that features this, but I know PyTables best
> ;)
>
> For this, I've used a modified version of a small benchmarking program
> posted by Tim Hochberg in this same thread (it is listed at the end
> of the message). Here are the results:
>
> setup SQLite took 23.5661110878 seconds
> retrieve SQLite took 3.26717996597 seconds
> setup PyTables took 0.139157056808 seconds
> retrieve PyTables took 0.13444685936 seconds
>
> [SQLite results were obtained using an in-memory database, while
> PyTables used an on-disk one. See the code.]
>
> So, yes, if your access pattern exhibits a high degree of locality,
> you can expect a huge difference on the reading speed (more than 20x
> for this example, but as this depends on the dataset size, it can be
> even higher for larger datasets).
>   
One weakness of this benchmark is that it doesn't break out how much of 
the sqlite3 overhead is inherent to the sqlite3 engine, which I expect 
is somewhat more complicated internally than PyTables, and how much is 
due to all the extra layers we go through to get the data into an array 
(native[in database]->Python Objects->Native[In record array]). To try 
to get at least a little handle on this, I add this test:

    def querySQLite(conn):
        c = conn.cursor()
        c.execute('select * from demo where x = 0.0')
        y = np.fromiter(c, dtype=dtype)
        return y

This returns very little data (in the cases I ran it actually returned 
no data). However is still needs to loop over all the records and 
examine them. Here's what the timings looked like:

    setup SQLite took 9.71799993515 seconds
    retrieve SQLite took 0.921999931335 seconds
    query SQLite took 0.313000202179 seconds

I'm reluctant to conclude to conclude that 1/3 of the time is spent in 
traversing the database and 2/3 of the time in creating the data solely 
because databases are big voodoo to me. Still, we can probably conclude 
that traversing the data itself is pretty expensive and we would be 
unlikely to approach PyTables speed even if we didn't have the extra 
overhead. On the other hand, there's a factor of three or so improvement 
that could be realized by reducing overhead.

Or maybe not. I think that the database has to return it's data a row at 
a time, so there's intrinsically a lot of copying that's going to 
happen. So, I think it's unclear whether getting the data directly in 
native format would be significantly cheaper. I suppose that the way to 
definitively test it would be to rewrite one of these tests in C. Any 
volunteers?

I think it's probably safe to say that either way PyTables will cream 
sqllite3 in those fields where it's applicable. One of these days I 
really need to dig into PyTables. I'm sure I could use it for something.

[snip]

-tim




-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your
opinions on IT & business topics through brief surveys - and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV



From Webroot at overwet.com  Thu Nov 16 13:21:22 2006
From: Webroot at overwet.com (Pelosi)
Date: Thu, 16 Nov 2006 19:21:22 +0100
Subject: Premiere WeekThis MonthThis
Message-ID: <000b01c709ac$04b5d9c0$00000000@pctitan>


Suita in killed pairblogs ca Paper. Joinforgot password in ce Security Spyware Utilities Tools of Desktop Business am.
Shots Knowledge Base a Overview Skin Demos Column Satire of Humor.
Prizetroop of pullout uneasyuns Darfur proposal calls?
Gt famous trackwatch Rodney Mullen. Attaching about quibble.
Loves am chase going!
Os try Toolbar am message browser!
Cnncom Breaking of est cnn a Deadly? Legally mainly certain or multipacks is within cases a.
Not but also in appearance featuring little than new.
Bank Lapd cau Space re Kanye West or Diddy!
Angeles make sure or ideas in tune said must!
Enoughmost in spinning solution problem marginally background hiss attaching about. Crosswords Horoscopes Lottery Stats in. Larger heavier or iriver fairly compact. Enoughmost in spinning solution problem marginally background hiss attaching about.
Cnntogo Pipeline window a endorsed is Cnncompay. Gameinfo wikithe World of article Mean page lists.
Flying am Squirrel bed a incredible!
Luxuries Countdown Gear Clix Zune. Sports platform Rumble Stations Color or Dreams or Sachens Qboy!
Check out decor is store near cant wait Canada Show.
Heidi Hemmat reports.
Leftto hr jm or Dodd Craig Harris in fun Harristhis.
Joinforgot password in ce Security Spyware Utilities Tools of Desktop Business am. Januarytry in un expected vote Cnnmoney of?
Made available since or height a late technology employed has.
Monitoring am sites likeminded Solitaire.
Made available since or height a late technology employed has. Videostv Breakgame Soft.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20061116/aa57f7a2/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: Pakistan.gif
Type: image/gif
Size: 14085 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20061116/aa57f7a2/attachment-0001.gif>
-------------- next part --------------
-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your
opinions on IT & business topics through brief surveys - and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV
-------------- next part --------------
_______________________________________________
Numpy-discussion mailing list
Numpy-discussion at lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/numpy-discussion

From mattknox_ca at hotmail.com  Thu Nov 16 13:37:31 2006
From: mattknox_ca at hotmail.com (Matt Knox)
Date: Thu, 16 Nov 2006 13:37:31 -0500
Subject: Defining custom types
Message-ID: <BAY124-W51DF4DFC8284EBE1AE2518CE90@phx.gbl>

> On Thursday 16 November 2006 11:44, David Douard wrote:> > Hi, just to ask you: how is the work going on encapsulatinsg mx.DateTime> > as a native numpy type?> > And most important: is the code available somewhere? I am also> > interested in using DateTime objects in numpy arrays. For now, I've> > always used arrays of floats (using gmticks values of dates).> And I, as arrays of objects (well, I wrote a subclass to deal with dates, > where each element is a datetime object, with methods to translate to floats > or strings , but it's far from optimal...). I'd also be quite interested in > checking what has been done.
 
I'm also very interested in the results of this. I need to do something very similar and am currently relying on an ugly hack to achieve the desired result.
 
- Matt Knox
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20061116/26a6b9ae/attachment-0001.html>
-------------- next part --------------
-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your
opinions on IT & business topics through brief surveys - and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV
-------------- next part --------------
_______________________________________________
Numpy-discussion mailing list
Numpy-discussion at lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/numpy-discussion

From jontwang at gmail.com  Thu Nov 16 14:28:25 2006
From: jontwang at gmail.com (Jonathan Wang)
Date: Thu, 16 Nov 2006 13:28:25 -0600
Subject: Defining custom types
In-Reply-To: <BAY124-W51DF4DFC8284EBE1AE2518CE90@phx.gbl>
References: <BAY124-W51DF4DFC8284EBE1AE2518CE90@phx.gbl>
Message-ID: <4f7636bf0611161128je184825n7fc35bb2cfc1acbf@mail.gmail.com>

Hi all,

I've gotten to the point where Numpy recognizes the objects (represented as
doubles), but I haven't figured out how to register ufunc loops on the
custom type. It seems like Numpy should be able to check that the scalarkind
variable in the numpy type descriptor is set to float and use the float
ufuncs on the custom object. Barring that, does anyone know if the symbols
for the ufuncs are publicly accessible (and where they are) so that I can
register them with Numpy on the custom type?

As for sharing code, I've been working on this for a project at work. There
is a possibility that it will be released to the Numpy community, but that's
not clear yet.

Thanks,
Jonathan

On 11/16/06, Matt Knox <mattknox_ca at hotmail.com> wrote:
>
> > On Thursday 16 November 2006 11:44, David Douard wrote:
> > > Hi, just to ask you: how is the work going on encapsulatinsg
> mx.DateTime
> > > as a native numpy type?
> > > And most important: is the code available somewhere? I am also
> > > interested in using DateTime objects in numpy arrays. For now, I've
> > > always used arrays of floats (using gmticks values of dates).
>
> > And I, as arrays of objects (well, I wrote a subclass to deal with
> dates,
> > where each element is a datetime object, with methods to translate to
> floats
> > or strings , but it's far from optimal...). I'd also be quite interested
> in
> > checking what has been done.
>
> I'm also very interested in the results of this. I need to do something
> very similar and am currently relying on an ugly hack to achieve the desired
> result.
>
> - Matt Knox
>
> -------------------------------------------------------------------------
> Take Surveys. Earn Cash. Influence the Future of IT
> Join SourceForge.net's Techsay panel and you'll get the chance to share
> your
> opinions on IT & business topics through brief surveys - and earn cash
> http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV
>
> _______________________________________________
> Numpy-discussion mailing list
> Numpy-discussion at lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/numpy-discussion
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20061116/9208086b/attachment-0001.html>
-------------- next part --------------
-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your
opinions on IT & business topics through brief surveys - and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV
-------------- next part --------------
_______________________________________________
Numpy-discussion mailing list
Numpy-discussion at lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/numpy-discussion

From jstrunk at enthought.com  Thu Nov 16 14:55:37 2006
From: jstrunk at enthought.com (Jeff Strunk)
Date: Thu, 16 Nov 2006 13:55:37 -0600
Subject: New list address
Message-ID: <200611161355.37519.jstrunk@enthought.com>

Good afternoon,

In a few minutes, this list will have a new address.

Please send all posts to numpy-discussion at scipy.org .

Thank you,
Jeff

-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your
opinions on IT & business topics through brief surveys - and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV



From robert.kern at gmail.com  Thu Nov 16 15:21:24 2006
From: robert.kern at gmail.com (Robert Kern)
Date: Thu, 16 Nov 2006 14:21:24 -0600
Subject: Test - Please ignore
Message-ID: <455CC844.4000208@gmail.com>

This should be rejected.

-- 
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless enigma
 that is made terrible by our own mad attempt to interpret it as though it had
 an underlying truth."
  -- Umberto Eco

-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your
opinions on IT & business topics through brief surveys - and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV



From jstrunk at enthought.com  Thu Nov 16 15:36:14 2006
From: jstrunk at enthought.com (Jeff Strunk)
Date: Thu, 16 Nov 2006 14:36:14 -0600
Subject: List migration complete
Message-ID: <200611161436.14512.jstrunk@enthought.com>

Good afternoon,

The list migration has completed successfully. numpy-discussion at scipy.org is 
the new address for this list.

Thank you,
Jeff



From iain at day-online.org.uk.invalid  Thu Nov 16 15:34:51 2006
From: iain at day-online.org.uk.invalid (Iain Day)
Date: Thu, 16 Nov 2006 20:34:51 +0000
Subject: Precision problem Mac OS X (PPC)
Message-ID: <ejii1b$ifg$2@sea.gmane.org>

Hi,

I posted this query to the scipy list (which you may have seen). I have 
been having some precision issues with some code I've got. I've searched 
the list archives and came across these tests. SciPy and numpy are 
installed from Fink, and running on a 1.5GHz G4 with Mac os x 10.4.8.

 >>> import numpy
 >>> import scipy
 >>> numpy.__version__
'1.0'
 >>> scipy.__version__
'0.5.1'
 >>> print numpy.finfo(numpy.float32).min
-3.40282346639e+38
 >>> print numpy.finfo(numpy.float32).min, 
numpy.finfo(numpy.float32).max, numpy.finfo(numpy.float32).eps
-3.40282346639e+38 3.40282346639e+38 1.19209289551e-07
 >>> print numpy.finfo(numpy.float64).min, 
numpy.finfo(numpy.float64).max, numpy.finfo(numpy.float64).eps
-1.79769313486e+308 1.79769313486e+308 2.22044604925e-16
 >>> print numpy.finfo(numpy.float128).min, 
numpy.finfo(numpy.float128).max, numpy.finfo(numpy.float128).eps
Warning: overflow encountered in add
Warning: invalid value encountered in subtract
Warning: invalid value encountered in subtract
Warning: overflow encountered in add
Warning: invalid value encountered in subtract
Warning: invalid value encountered in subtract
9223372034707292160.0 -9223372034707292160.0 1.38178697010200053818e-76

Any thoughts? I think this is a bug? If so, where do I report it?

Iain



From numpy-discussion at mlists.thewrittenword.com  Thu Nov 16 15:48:53 2006
From: numpy-discussion at mlists.thewrittenword.com (Albert Chin)
Date: Thu, 16 Nov 2006 14:48:53 -0600
Subject: HAVE_FLOAT_FUNCS on HP-UX 11.23/PA
Message-ID: <20061116204853.GB54997@mail1.thewrittenword.com>

HP-UX 11.23/PA has expf() but not ceilf(). numpy/core/setup.py has:
  FUNCTIONS_TO_CHECK = [
      ('expl', 'HAVE_LONGDOUBLE_FUNCS'),
      ('expf', 'HAVE_FLOAT_FUNCS'),
      ('log1p', 'HAVE_LOG1P'),
      ('expm1', 'HAVE_EXPM1'),
      ('asinh', 'HAVE_INVERSE_HYPERBOLIC'),
      ('atanhf', 'HAVE_INVERSE_HYPERBOLIC_FLOAT'),
      ('atanhl', 'HAVE_INVERSE_HYPERBOLIC_LONGDOUBLE'),
      ('isnan', 'HAVE_ISNAN'),
      ('isinf', 'HAVE_ISINF'),
      ('rint', 'HAVE_RINT'),
      ]

So, on this platform, HAVE_FLOAT_FUNCS isn't defined. This causes a
problem because the local numpy definition for ceilf() isn't used,
leading to compiler errors.

What's the best way to solve this? Replace "expf" above with "ceilf"
even though expf() is present on this platform? My preference is to
explicitly list all the functions that need to be checked and define
HAVE_<func> accordingly.

-- 
albert chin (china at thewrittenword.com)



From lists.steve at arachnedesign.net  Thu Nov 16 15:51:30 2006
From: lists.steve at arachnedesign.net (Steve Lianoglou)
Date: Thu, 16 Nov 2006 15:51:30 -0500
Subject: List migration complete
In-Reply-To: <200611161436.14512.jstrunk@enthought.com>
References: <200611161436.14512.jstrunk@enthought.com>
Message-ID: <3C063AE0-1EC8-4162-9368-998D29D1814D@arachnedesign.net>

> The list migration has completed successfully. numpy- 
> discussion at scipy.org is
> the new address for this list.

Hey look ...  no more advertisements!

Good job folks, thanks!

One thing: I'm not sure if this was intentional, but the Reply-To  
field isn't set, so in order to reply back to list we need to hit the  
Reply All. I'm thinking it was an oversight since the scipy list  
doesn't work this way so I'm just letting you know.

Thanks again,
-steve



From nvf at MIT.EDU  Thu Nov 16 15:58:56 2006
From: nvf at MIT.EDU (Nick Fotopoulos)
Date: Thu, 16 Nov 2006 14:58:56 -0600
Subject: List migration complete
In-Reply-To: <3C063AE0-1EC8-4162-9368-998D29D1814D@arachnedesign.net>
References: <200611161436.14512.jstrunk@enthought.com>
	<3C063AE0-1EC8-4162-9368-998D29D1814D@arachnedesign.net>
Message-ID: <A56DB0FB-EE0D-42B1-8CA2-244CF10B640B@mit.edu>

On Nov 16, 2006, at 2:51 PM, Steve Lianoglou wrote:

>> The list migration has completed successfully. numpy-
>> discussion at scipy.org is
>> the new address for this list.
>
> Hey look ...  no more advertisements!
>
> Good job folks, thanks!
>
> One thing: I'm not sure if this was intentional, but the Reply-To
> field isn't set, so in order to reply back to list we need to hit the
> Reply All. I'm thinking it was an oversight since the scipy list
> doesn't work this way so I'm just letting you know.

Another oversight is that digest-mode subscribers were imported as  
regular subscribers.  However, this is easily fixed at http:// 
projects.scipy.org/mailman/listinfo/numpy-discussion.

I agree, though.  It's nice not to have the Sourceforge ads.  Thanks  
for the work that went into making the switch.

Take care,
Nick



From faltet at carabos.com  Thu Nov 16 16:14:21 2006
From: faltet at carabos.com (Francesc Altet)
Date: Thu, 16 Nov 2006 22:14:21 +0100
Subject: mysql -> record array
In-Reply-To: <455CA862.2060305@ieee.org>
References: <87slglwwwm.fsf@peds-pc311.bsd.uchicago.edu>
	<331116dc0611141408q2d7c9431s38a0f4c4b3f2db25@mail.gmail.com>
	<200611161413.31917.faltet@carabos.com>  <455CA862.2060305@ieee.org>
Message-ID: <1163711662.2613.49.camel@localhost.localdomain>

Hi,

Well, lets see whether this shiny new list works :)

El dj 16 de 11 del 2006 a les 11:05 -0700, en/na Tim Hochberg va
escriure:
> One weakness of this benchmark is that it doesn't break out how much of 
> the sqlite3 overhead is inherent to the sqlite3 engine, which I expect 
> is somewhat more complicated internally than PyTables, and how much is 
> due to all the extra layers we go through to get the data into an array 
> (native[in database]->Python Objects->Native[In record array]). To try 
> to get at least a little handle on this, I add this test:
> 
>     def querySQLite(conn):
>         c = conn.cursor()
>         c.execute('select * from demo where x = 0.0')
>         y = np.fromiter(c, dtype=dtype)
>         return y
> 
> This returns very little data (in the cases I ran it actually returned 
> no data). However is still needs to loop over all the records and 
> examine them. Here's what the timings looked like:
> 
>     setup SQLite took 9.71799993515 seconds
>     retrieve SQLite took 0.921999931335 seconds
>     query SQLite took 0.313000202179 seconds
> 
> I'm reluctant to conclude to conclude that 1/3 of the time is spent in 
> traversing the database and 2/3 of the time in creating the data solely 
> because databases are big voodoo to me. Still, we can probably conclude 
> that traversing the data itself is pretty expensive and we would be 
> unlikely to approach PyTables speed even if we didn't have the extra 
> overhead. On the other hand, there's a factor of three or so improvement 
> that could be realized by reducing overhead.
> 
> Or maybe not. I think that the database has to return it's data a row at 
> a time, so there's intrinsically a lot of copying that's going to 
> happen. So, I think it's unclear whether getting the data directly in 
> native format would be significantly cheaper. I suppose that the way to 
> definitively test it would be to rewrite one of these tests in C. Any 
> volunteers?

I don't think that doing this in C is going to be necessary at all.
Sqlite3 has a console that allows doing SQL operations without Python in
the middle.

I've included your new function and here are my results for Python (just
for comparison purposes):

setup SQLite took 35.9286799431 seconds
retrieve SQLite took 4.66451621056 seconds
query SQLite took 0.907495975494 seconds

Now, the query from the console:

$ time sqlite3 /tmp/test.sql3 "select * from demo where x == 0.0"

real    0m1.226s
user    0m0.792s
sys     0m0.120s

If you sum cpu user time and sys time, you get 0.9 seconds (roughly)
which is approximately the same than reported in Python. So, it doesn't
seem that for this specific case the python interface is taking a
significant amount of time (as expected).

For a complete retrieval, we can do the same trick, but redirectioning
to /dev/null to avoid writing the output to another device:

time sqlite3 /tmp/test.sql3 "select * from demo" > /dev/null

real    0m3.690s
user    0m3.376s
sys     0m0.188s

In this case, the time sums up around 3.5 seconds, while the Python time
is 4.6 seconds. However, the console has the problem that it has to
format every row to a string before to "write" to /dev/null. So, I'd say
that the time for a retrieval from C program should be significantly
less than 3.5 (provided that there are an appropriate container
in-memory for keeping the data). So, the bottleneck does effectively
seem to be the conversion from list of lists to a recarray.

Just out of curiosity, here it is the equivalent where function in
PyTables:

def queryPT(fileh):
    t = fileh.root.table
    y = np.fromiter(((r['x'], r['y']) for r in t.where(t.cols.x == 0)),
                    dtype=dtype)
    return y

and its times:

setup PyTables took 0.453132867813 seconds
retrieve PyTables took 0.356657981873 seconds
query PyTables took 0.253107070923 seconds

PyTables is more than 3x faster than SQLite for this specific query. I
guess this is because the selects are done completely through numarray
(in PyTables 2.x selections will be done through numexpr), and it seems
difficult to bet such a 'vectorized' selects. So, using a package like
numarray or NumPy in the core of a database has a lot of advantages,
most specially when speed matters.

Cheers,

-- 
Francesc Altet    |  Be careful about using the following code --
Carabos Coop. V.  |  I've only proven that it works, 
www.carabos.com   |  I haven't tested it. -- Donald Knuth



From erin.sheldon at gmail.com  Thu Nov 16 16:28:18 2006
From: erin.sheldon at gmail.com (Erin Sheldon)
Date: Thu, 16 Nov 2006 16:28:18 -0500
Subject: mysql -> record array
In-Reply-To: <1163711662.2613.49.camel@localhost.localdomain>
References: <87slglwwwm.fsf@peds-pc311.bsd.uchicago.edu>
	<331116dc0611141408q2d7c9431s38a0f4c4b3f2db25@mail.gmail.com>
	<200611161413.31917.faltet@carabos.com> <455CA862.2060305@ieee.org>
	<1163711662.2613.49.camel@localhost.localdomain>
Message-ID: <331116dc0611161328s68e1cef1y31e8ceaefc2eb3e0@mail.gmail.com>

Hi Francesc -

Unless I missed something, I think what you have
shown is that the combination of
      (getting data from database into python lists) +
      (converting to arrays)
is what is taking time.   I would guess the first takes
significantly longer than the second.

Erin

On 11/16/06, Francesc Altet <faltet at carabos.com> wrote:
> Hi,
>
> Well, lets see whether this shiny new list works :)
>
> El dj 16 de 11 del 2006 a les 11:05 -0700, en/na Tim Hochberg va
> escriure:
> > One weakness of this benchmark is that it doesn't break out how much of
> > the sqlite3 overhead is inherent to the sqlite3 engine, which I expect
> > is somewhat more complicated internally than PyTables, and how much is
> > due to all the extra layers we go through to get the data into an array
> > (native[in database]->Python Objects->Native[In record array]). To try
> > to get at least a little handle on this, I add this test:
> >
> >     def querySQLite(conn):
> >         c = conn.cursor()
> >         c.execute('select * from demo where x = 0.0')
> >         y = np.fromiter(c, dtype=dtype)
> >         return y
> >
> > This returns very little data (in the cases I ran it actually returned
> > no data). However is still needs to loop over all the records and
> > examine them. Here's what the timings looked like:
> >
> >     setup SQLite took 9.71799993515 seconds
> >     retrieve SQLite took 0.921999931335 seconds
> >     query SQLite took 0.313000202179 seconds
> >
> > I'm reluctant to conclude to conclude that 1/3 of the time is spent in
> > traversing the database and 2/3 of the time in creating the data solely
> > because databases are big voodoo to me. Still, we can probably conclude
> > that traversing the data itself is pretty expensive and we would be
> > unlikely to approach PyTables speed even if we didn't have the extra
> > overhead. On the other hand, there's a factor of three or so improvement
> > that could be realized by reducing overhead.
> >
> > Or maybe not. I think that the database has to return it's data a row at
> > a time, so there's intrinsically a lot of copying that's going to
> > happen. So, I think it's unclear whether getting the data directly in
> > native format would be significantly cheaper. I suppose that the way to
> > definitively test it would be to rewrite one of these tests in C. Any
> > volunteers?
>
> I don't think that doing this in C is going to be necessary at all.
> Sqlite3 has a console that allows doing SQL operations without Python in
> the middle.
>
> I've included your new function and here are my results for Python (just
> for comparison purposes):
>
> setup SQLite took 35.9286799431 seconds
> retrieve SQLite took 4.66451621056 seconds
> query SQLite took 0.907495975494 seconds
>
> Now, the query from the console:
>
> $ time sqlite3 /tmp/test.sql3 "select * from demo where x == 0.0"
>
> real    0m1.226s
> user    0m0.792s
> sys     0m0.120s
>
> If you sum cpu user time and sys time, you get 0.9 seconds (roughly)
> which is approximately the same than reported in Python. So, it doesn't
> seem that for this specific case the python interface is taking a
> significant amount of time (as expected).
>
> For a complete retrieval, we can do the same trick, but redirectioning
> to /dev/null to avoid writing the output to another device:
>
> time sqlite3 /tmp/test.sql3 "select * from demo" > /dev/null
>
> real    0m3.690s
> user    0m3.376s
> sys     0m0.188s
>
> In this case, the time sums up around 3.5 seconds, while the Python time
> is 4.6 seconds. However, the console has the problem that it has to
> format every row to a string before to "write" to /dev/null. So, I'd say
> that the time for a retrieval from C program should be significantly
> less than 3.5 (provided that there are an appropriate container
> in-memory for keeping the data). So, the bottleneck does effectively
> seem to be the conversion from list of lists to a recarray.
>
> Just out of curiosity, here it is the equivalent where function in
> PyTables:
>
> def queryPT(fileh):
>     t = fileh.root.table
>     y = np.fromiter(((r['x'], r['y']) for r in t.where(t.cols.x == 0)),
>                     dtype=dtype)
>     return y
>
> and its times:
>
> setup PyTables took 0.453132867813 seconds
> retrieve PyTables took 0.356657981873 seconds
> query PyTables took 0.253107070923 seconds
>
> PyTables is more than 3x faster than SQLite for this specific query. I
> guess this is because the selects are done completely through numarray
> (in PyTables 2.x selections will be done through numexpr), and it seems
> difficult to bet such a 'vectorized' selects. So, using a package like
> numarray or NumPy in the core of a database has a lot of advantages,
> most specially when speed matters.
>
> Cheers,
>
> --
> Francesc Altet    |  Be careful about using the following code --
> Carabos Coop. V.  |  I've only proven that it works,
> www.carabos.com   |  I haven't tested it. -- Donald Knuth
>
> _______________________________________________
> Numpy-discussion mailing list
> Numpy-discussion at scipy.org
> http://projects.scipy.org/mailman/listinfo/numpy-discussion
>



From loredo at astro.cornell.edu  Thu Nov 16 17:40:09 2006
From: loredo at astro.cornell.edu (Tom Loredo)
Date: Thu, 16 Nov 2006 17:40:09 -0500
Subject: List migration complete
In-Reply-To: <A56DB0FB-EE0D-42B1-8CA2-244CF10B640B@mit.edu>
References: <200611161436.14512.jstrunk@enthought.com>
	<3C063AE0-1EC8-4162-9368-998D29D1814D@arachnedesign.net>
	<A56DB0FB-EE0D-42B1-8CA2-244CF10B640B@mit.edu>
Message-ID: <1163716809.455ce8c98f5ab@astrosun2.astro.cornell.edu>

Quoting Nick Fotopoulos <nvf at MIT.EDU>:


> Another oversight is that digest-mode subscribers were imported as  
> regular subscribers.  However, this is easily fixed at http:// 
> projects.scipy.org/mailman/listinfo/numpy-discussion.

Also, the original list passwords apparently were not/could not
be copied, so I had to get a password reminder (which sent me
an arbitrary password) to get back on digest mode.

I hope the new digests will be a little less frequent!

Thanks for the migration,
Tom



-------------------------------------------------
This mail sent through IMP: http://horde.org/imp/



From david.douard at logilab.fr  Fri Nov 17 03:40:54 2006
From: david.douard at logilab.fr (David Douard)
Date: Fri, 17 Nov 2006 09:40:54 +0100
Subject: [Numpy-discussion] Defining custom types
In-Reply-To: <4f7636bf0611161128je184825n7fc35bb2cfc1acbf@mail.gmail.com>
References: <BAY124-W51DF4DFC8284EBE1AE2518CE90@phx.gbl>
	<4f7636bf0611161128je184825n7fc35bb2cfc1acbf@mail.gmail.com>
Message-ID: <20061117084054.GB20564@crater.logilab.fr>

On Thu, Nov 16, 2006 at 01:28:25PM -0600, Jonathan Wang wrote:
> Hi all,
> 
> I've gotten to the point where Numpy recognizes the objects (represented as
> doubles), but I haven't figured out how to register ufunc loops on the
> custom type. It seems like Numpy should be able to check that the scalarkind
> variable in the numpy type descriptor is set to float and use the float
> ufuncs on the custom object. Barring that, does anyone know if the symbols
> for the ufuncs are publicly accessible (and where they are) so that I can
> register them with Numpy on the custom type?
> 
> As for sharing code, I've been working on this for a project at work. There
> is a possibility that it will be released to the Numpy community, but that's
> not clear yet.

Hi,
besides the fact that there seem to have several people interested in
this piece of code, I'm pretty sure some of us might help you with this
code. 
Maybe you should tell your employer how far your project
could benefits from being released to the numpy community (at least this
part of the project) ;-)


> 
> Thanks,
> Jonathan
> 
> On 11/16/06, Matt Knox <mattknox_ca at hotmail.com> wrote:
> >
> >> On Thursday 16 November 2006 11:44, David Douard wrote:
> >> > Hi, just to ask you: how is the work going on encapsulatinsg
> >mx.DateTime
> >> > as a native numpy type?
> >> > And most important: is the code available somewhere? I am also
> >> > interested in using DateTime objects in numpy arrays. For now, I've
> >> > always used arrays of floats (using gmticks values of dates).
> >
> >> And I, as arrays of objects (well, I wrote a subclass to deal with
> >dates,
> >> where each element is a datetime object, with methods to translate to
> >floats
> >> or strings , but it's far from optimal...). I'd also be quite interested
> >in
> >> checking what has been done.
> >
> >I'm also very interested in the results of this. I need to do something
> >very similar and am currently relying on an ugly hack to achieve the 
> >desired
> >result.
> >
> >- Matt Knox
> >
> >-------------------------------------------------------------------------
> >Take Surveys. Earn Cash. Influence the Future of IT
> >Join SourceForge.net's Techsay panel and you'll get the chance to share
> >your
> >opinions on IT & business topics through brief surveys - and earn cash
> >http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV
> >
> >_______________________________________________
> >Numpy-discussion mailing list
> >Numpy-discussion at lists.sourceforge.net
> >https://lists.sourceforge.net/lists/listinfo/numpy-discussion
> >
> >
> >

> -------------------------------------------------------------------------
> Take Surveys. Earn Cash. Influence the Future of IT
> Join SourceForge.net's Techsay panel and you'll get the chance to share your
> opinions on IT & business topics through brief surveys - and earn cash
> http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV
> _______________________________________________
> Numpy-discussion mailing list
> Numpy-discussion at lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/numpy-discussion


-- 
David Douard                             LOGILAB, Paris (France)
Formations Python, Zope, Plone, Debian : http://www.logilab.fr/formations
D?veloppement logiciel sur mesure :      http://www.logilab.fr/services
Informatique scientifique :              http://www.logilab.fr/science
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 189 bytes
Desc: Digital signature
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20061117/cbcbdae0/attachment.sig>

From faltet at carabos.com  Fri Nov 17 08:21:20 2006
From: faltet at carabos.com (Francesc Altet)
Date: Fri, 17 Nov 2006 14:21:20 +0100
Subject: [Numpy-discussion] mysql -> record array
In-Reply-To: <331116dc0611161328s68e1cef1y31e8ceaefc2eb3e0@mail.gmail.com>
References: <87slglwwwm.fsf@peds-pc311.bsd.uchicago.edu>
	<1163711662.2613.49.camel@localhost.localdomain>
	<331116dc0611161328s68e1cef1y31e8ceaefc2eb3e0@mail.gmail.com>
Message-ID: <200611171421.22216.faltet@carabos.com>

A Dijous 16 Novembre 2006 22:28, Erin Sheldon escrigu?:
> Hi Francesc -
>
> Unless I missed something, I think what you have
> shown is that the combination of
>       (getting data from database into python lists) +
>       (converting to arrays)
> is what is taking time.   I would guess the first takes
> significantly longer than the second.

Seriously, I don't think I have demonstrated nothing really solid in
that regard with so little evidences. But we can try looking for more
of those :)

For example, I'd split the times in:

    t1 (getting data from database) +
    t2 (python lists) +
    t3 (converting to arrays) =
    tt (total time)

We don't know t1, but we do know tt. Now, we can try to get a guess
for t1 and t2. Perhaps I'm wrong, but the next could be good
estimates.

For t1 (creating the python list of tuples):
In [44]: Timer("[(x,x) for x in np.arange(500000, dtype='float64')]", "import 
numpy as np").repeat(3,1)
Out[44]: [0.55968594551086426, 0.48462891578674316, 0.4855189323425293]

For t2 (converting to recarrays):
In [49]: Timer("np.fromiter(lot, dtype=dtype)", "import numpy as np; 
lot=[(x,x) for x in np.arange(500000, dtype='float64')]; 
dtype=np.dtype([('x', 'float64'), ('y', 'float64')])").repeat(3,1)
Out[49]: [0.50310707092285156, 0.50920987129211426, 0.50304579734802246]

So, it seems that t1 and t2 are similar and they take aproximately 0.5
seconds each.

Now, let me remember the timings for reading the databases on my
laptop at work (a Pentium4 @ 2 GHz):

setup SQLite took 23.5661110878 seconds
retrieve SQLite took 3.26717996597 seconds
setup PyTables took 0.139157056808 seconds
retrieve PyTables took 0.13444685936 seconds

So, in our case, tt for SQLite3 was 3.26 seconds. With that, we can
derive its t1 (getting data from database):

t1 = tt - t1 - t2 =~ 2.26 seconds

However, this is still far more than tt for PyTables (~ 0.14 sec), so
I'm not completely sure what's going on. Honest, I don't think that
HDF5 (the underlying library for doing I/O in PyTables) would be
almost 20x faster than SQLite3 for reading purposes. So my guess is
that there should be more factors contributing tt for SQLite3 that
I've not taken in account. Anyone can find them?

Cheers,

-- 
>0,0<   Francesc Altet ? ? http://www.carabos.com/
V   V   C?rabos Coop. V. ??Enjoy Data
 "-"



From emsellem at obs.univ-lyon1.fr  Fri Nov 17 08:25:19 2006
From: emsellem at obs.univ-lyon1.fr (Eric Emsellem)
Date: Fri, 17 Nov 2006 14:25:19 +0100
Subject: [Numpy-discussion] mysql -> record array
In-Reply-To: <200611171421.22216.faltet@carabos.com>
References: <87slglwwwm.fsf@peds-pc311.bsd.uchicago.edu>	<1163711662.2613.49.camel@localhost.localdomain>	<331116dc0611161328s68e1cef1y31e8ceaefc2eb3e0@mail.gmail.com>
	<200611171421.22216.faltet@carabos.com>
Message-ID: <455DB83F.9020209@obs.univ-lyon1.fr>

Hi,
I am getting all the individual emails from the numpy discussion (and
not the bundles), directly from their authors: something is wrong with
the new list I think.
cheers
Eric



From kwgoodman at gmail.com  Fri Nov 17 12:12:13 2006
From: kwgoodman at gmail.com (Keith Goodman)
Date: Fri, 17 Nov 2006 09:12:13 -0800
Subject: [Numpy-discussion] numpy.test
Message-ID: <f4f93d420611170912x66c4a980h4e4bd156d43fa05b@mail.gmail.com>

I'd like to add numpy.test(level=10, verbosity=0) to the top of my program.

Is there a way for my program to find out if all tests passed? Or even
better, how many tests ran and how many of those failed.

Is there any way to suppress all output of numpy.test? verbosity=0
stops most, but not all, of it.


From erin.sheldon at gmail.com  Fri Nov 17 13:45:39 2006
From: erin.sheldon at gmail.com (Erin Sheldon)
Date: Fri, 17 Nov 2006 13:45:39 -0500
Subject: [Numpy-discussion] mysql -> record array
In-Reply-To: <455DB83F.9020209@obs.univ-lyon1.fr>
References: <87slglwwwm.fsf@peds-pc311.bsd.uchicago.edu>
	<1163711662.2613.49.camel@localhost.localdomain>
	<331116dc0611161328s68e1cef1y31e8ceaefc2eb3e0@mail.gmail.com>
	<200611171421.22216.faltet@carabos.com>
	<455DB83F.9020209@obs.univ-lyon1.fr>
Message-ID: <331116dc0611171045k4b7c7ba8y9d6695878ca3234c@mail.gmail.com>

Those preferences did not make it through.  You
have to go back and re-check that preference.
Erin

On 11/17/06, Eric Emsellem <emsellem at obs.univ-lyon1.fr> wrote:
> Hi,
> I am getting all the individual emails from the numpy discussion (and
> not the bundles), directly from their authors: something is wrong with
> the new list I think.
> cheers
> Eric
>
> _______________________________________________
> Numpy-discussion mailing list
> Numpy-discussion at scipy.org
> http://projects.scipy.org/mailman/listinfo/numpy-discussion
>


From robert.kern at gmail.com  Fri Nov 17 13:50:04 2006
From: robert.kern at gmail.com (Robert Kern)
Date: Fri, 17 Nov 2006 12:50:04 -0600
Subject: [Numpy-discussion] Digests (was Re:  mysql -> record array)
In-Reply-To: <455DB83F.9020209@obs.univ-lyon1.fr>
References: <87slglwwwm.fsf@peds-pc311.bsd.uchicago.edu>	<1163711662.2613.49.camel@localhost.localdomain>	<331116dc0611161328s68e1cef1y31e8ceaefc2eb3e0@mail.gmail.com>	<200611171421.22216.faltet@carabos.com>
	<455DB83F.9020209@obs.univ-lyon1.fr>
Message-ID: <455E045C.6050206@gmail.com>

Eric Emsellem wrote:
> Hi,
> I am getting all the individual emails from the numpy discussion (and
> not the bundles), directly from their authors: something is wrong with
> the new list I think.

I'm not sure I understand you.

If you mean that you were getting digests on the old list and not now, then no,
there's nothin *wrong* per se; however, we could only grab the subscriber list
from Sourceforge, not all of your preferences.

Have Mailman send you a password reminder (your password also could not have
been exported from SF).

  http://projects.scipy.org/mailman/options/numpy-discussion

Then log in and change your options.

I'm not sure what else you might mean.

-- 
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless enigma
 that is made terrible by our own mad attempt to interpret it as though it had
 an underlying truth."
  -- Umberto Eco


From faltet at carabos.com  Fri Nov 17 13:56:04 2006
From: faltet at carabos.com (Francesc Altet)
Date: Fri, 17 Nov 2006 19:56:04 +0100
Subject: [Numpy-discussion] mysql -> record array
In-Reply-To: <200611171421.22216.faltet@carabos.com>
References: <87slglwwwm.fsf@peds-pc311.bsd.uchicago.edu>
	<331116dc0611161328s68e1cef1y31e8ceaefc2eb3e0@mail.gmail.com>
	<200611171421.22216.faltet@carabos.com>
Message-ID: <200611171956.05875.faltet@carabos.com>

Hi,

First of all, sorry that I've mixed t1, t2 and t3 in my previous
message (don't doing an overhaul before sending a message has
consequences :(). I'd try to clarify things here along introducing new
timings and other discoveries.

Then, as the message below is quite long, I will mention now my main
conclusions about the time taken by reading a database using SQLite3:

1. Most of the time is consumed in getting a list with python objects
   as a container for data (60%)

2. A small slice of time is used for creating the final recarray (15%)

3. Pure reading of the database is around 25% of the total time

If you want to know about how I've come to these conclusions, please,
continue reading.

A Divendres 17 Novembre 2006 14:21, Francesc Altet escrigu?:
> A Dijous 16 Novembre 2006 22:28, Erin Sheldon escrigu?:
> > Hi Francesc -
> >
> > Unless I missed something, I think what you have
> > shown is that the combination of
> >       (getting data from database into python lists) +
> >       (converting to arrays)
> > is what is taking time.   I would guess the first takes
> > significantly longer than the second.
>
> Seriously, I don't think I have demonstrated nothing really solid in
> that regard with so little evidences. But we can try looking for more
> of those :)
>
> For example, I'd split the times in:
>
>     t1 (getting data from database) +
>     t2 (python lists) +
>     t3 (converting to arrays) =
>     tt (total time)
[SNIP]
> So, in our case, tt for SQLite3 was 3.26 seconds. With that, we can
> derive its t1 (getting data from database):
>
> t1 = tt - t1 - t2 =~ 2.26 seconds
>
> However, this is still far more than tt for PyTables (~ 0.14 sec), so
> I'm not completely sure what's going on. Honest, I don't think that
> HDF5 (the underlying library for doing I/O in PyTables) would be
> almost 20x faster than SQLite3 for reading purposes. So my guess is
> that there should be more factors contributing tt for SQLite3 that
> I've not taken in account. Anyone can find them?

I've found another possible factor that contributes more time to t2
(python list creation). Before, I've timed the building of a list of
tuples made of numpy scalars instead of python floats (the latter
being what pysqlite actually provides). So, if we force the creation
of floats, we spend quite more time:

In [80]: Timer("[(float(x),float(x)) for x in np.arange(500000, 
dtype='float64')]", "import numpy as np").repeat(3,1)
Out[80]: [1.440762996673584, 1.0981760025024414, 1.1034038066864014]

i.e. 1.1 sec (best case) for t2 instead of 0.5 sec. That's a bit
better. Interestingly enough, it seems that the first iteration in the
loop above seems to take almost 1.5 seconds, and perhaps the 2 other
iterations could benefit of some caching effect. As the SQLite3
benchmark measures the time for only one iteration (and besides we are
searching for lost time ;) we will take 1.44 sec for now.

But... wait!, now it comes to my mind the discover of Tim that timeit
module deactivates the garbage collector. Does this affect to this?
Well, lets see:

In [85]: Timer("[(float(x),float(x)) for x in np.arange(500000, 
dtype='float64')]", "gc.enable(); import numpy as np").repeat(3,1)
Out[85]: [1.9229378700256348, 1.7526938915252686, 1.8899729251861572]

Wow, it does indeed! So, lets pick up the worst value for this: t2
will become 1.92 sec. That looks pretty good.

Now, what about t3 (recarray creation)? If we repeat the timings:

In [90]: Timer("np.fromiter(lot, dtype=dtype)", "import numpy as np; 
lot=[(float(x),float(x)) for x in np.arange(500000, dtype='float64')]; 
dtype=np.dtype([('x', 'float64'), ('y', 'float64')])").repeat(3,1)
Out[90]: [0.52559113502502441, 0.52883505821228027, 0.52126002311706543]

we see that worst time is 0.53 sec. And repeating timings with garbage
collector activated:

In [91]: Timer("np.fromiter(lot, dtype=dtype)", "gc.enable(); import numpy as 
np; lot=[(float(x),float(x)) for x in np.arange(500000, dtype='float64')]; 
dtype=np.dtype([('x', 'float64'), ('y', 'float64')])").repeat(3,1)
Out[91]: [0.52265405654907227, 0.52456402778625488, 0.52321815490722656]

Well, garbage collector doesn't affect here very much. So we will keep
t3 = 0.53 sec.

Now, the moment for the truth has come. How this would affect to t1?
(i.e. the time for getting data from database):

t1 = tt - t2 - t3 = 3.26 - 1.92 - 0.53 =~ 0.81 sec

Well, 0.81 sec is much less than the 2.26 sec computed in my previous
message.  But, is 0.81 sec a trusty figure for reading the data in
sqlite3?  Well, to begin with, I think this is very much comparable
with a previous benchmark introduced by Tim on a previous message. The
benchmark consisted on a query with a single condition that returns
nothing (so, no time would be wasted on lists or recarray creation).
Here is the time for this benchmark on my laptop:

query SQLite took 0.5474588871 seconds

so, it seems that SQLite wouldn't take no more than 0.55 seconds for
reading the complete table. But, beware, as this could be a bit
tricky! Perhaps SQLite is saving data column wise (instead of row wise
like HDF5 does); so there is a chance that what does actually measure
Tim's query is the time to read *one* column, not both (this is a
guess on my part as I don't really know whether sqlite saves its table
data row wise or column wise).

To better check this, I've modified the query to take in account both
columns instead of only one ('x = 0 or y = 0'). Here is the result:

query SQLite took 0.819041013718 seconds

This time looks suspiciouly similar to our 0.81 sec computed
previously. However, we don't know yet how much of this time can be
attributed to reading data and which one to the code for executing the
selection conditions. But perhaps measuring how much time this takes
for numexpr (in the end, both do the query in C space) can give us an
estimate for it. Here it is the measure:

For one condition:
In [32]: Timer("numexpr.evaluate('x == 0')", "import numpy; from tables import 
numexpr; x=numpy.random.rand(500000)").repeat(3,100)
Out[32]: [0.58732295036315918, 0.5366971492767334, 0.53870081901550293]

For two conditions:
In [33]: Timer("numexpr.evaluate('x == 0 or y == 0')", "import numpy; from 
tables import numexpr; x=numpy.random.rand(500000); 
y=numpy.random.rand(500000)").repeat(3,100)
Out[33]: [0.566741943359375, 0.53875589370727539, 0.53498196601867676]

So, provided that we have repeated the loop 100 times, we have that,
most probably, times required for checking the conditions in C are
just around 1/100th of the time required for doing the reading. So we
have evidence that we can obviate this contribution for total times.

With this, we can conclude (if we are brave enough ;) that times for
pure reading the SQLite3 database in our scenario is near 0.81 sec. We
have come to this figure following two different paths, namely:

1. Timing the reading of the 2-column table from python and substracting the
   python overhead and...
2. Doing a query that do imply the read of *both* columns

So, lets state the final results here:

    t1 (getting data from database) = 0.81 sec  (~25% of tt)
    t2 (creation of python lists)   = 1.92 sec  (~60% of tt)
    t3 (converting to recarrays)    = 0.53 sec  (~15% of tt)
    tt (total time, computed)       = 3.26 sec
    tt (total time, measured)       = 3.26 sec

[Incidentally, it is a big coincidence that both the computed and
measured tt would coincide so precisely, but we should accept that
this can happen sometimes, just because stochastics sometimes allow
this ;)]

I think that we have now some reasons for concluding that, when
reading big tables from a database, and the complete database is in
cache (be in filesystem cache or in database cache), 1) most of the
time is consumed in getting a list with python objects as container
for data (60%), 2) a small slice of time is used for creating the
recarray (15%), and 3) pure reading is around 25% of the total time.

As an aside, it is worth to compare the time that needs SQLite3 for
pure reading and the time that needs PyTables. I will remember this
time:

retrieve PyTables took 0.119920015335 seconds
query PyTables took 0.130054950714 seconds (one condition)
query PyTables took 0.157742023468 seconds (two conditions)

Provided that PyTables uses a native NumPy container for keeping the
data, we can attribute all this time (0.12 sec) to pure database
reading. So 4) PyTables readings (i.e. done in Python space) can be
almost 7x times faster than pure readings of SQLite3 (i.e. done at C
level). I still find this a bit shocking, and I'd say that we have
three possible explanations for this:

1. SQLite3 performance is somewhat poor for reading whole tables
   in-memory. Perhaps Postgres or MySQL would perform better

2. HDF5 shines specially for reading whole tables in-memory

3. My estimations for SQLite3 pure reads are completely wrong and
   times are actually much less!

Mmm, I'd say that repeating the tests with MySQL o Postgres could
bring more light into this. Volunteers?

Ups, I think that I've exceeded by far what I wanted to write in the
beginning of this message. Sorry about this!

Cheers,

-- 
>0,0<   Francesc Altet ? ? http://www.carabos.com/
V   V   C?rabos Coop. V. ??Enjoy Data
 "-"



From oliphant at ee.byu.edu  Fri Nov 17 14:43:45 2006
From: oliphant at ee.byu.edu (Travis Oliphant)
Date: Fri, 17 Nov 2006 12:43:45 -0700
Subject: [Numpy-discussion] Support for understand ctypes data-types now in
	SVN
Message-ID: <455E10F1.9030202@ee.byu.edu>


The NumPy dtype command now understands ctypes data-types including 
derived structures and arrays.

This means you can specify data-types using ctypes method of building 
structures and pass those classes whenever NumPy expects a data-type 
object.

Simple example:

import ctypes as ct
import numpy as np

np.array([1,2,3],ct.c_short)

More complicated:

class POINT(ct.Structure):
         _fields_ = [("x", ct.c_int),
                           ("y", ct.c_int)]

np.array([(3,4),(5,6),(7,8)], dtype=POINT)


-Travis




From stefan at sun.ac.za  Fri Nov 17 15:05:34 2006
From: stefan at sun.ac.za (Stefan van der Walt)
Date: Fri, 17 Nov 2006 22:05:34 +0200
Subject: [Numpy-discussion] Support for understand ctypes data-types now
	in SVN
In-Reply-To: <455E10F1.9030202@ee.byu.edu>
References: <455E10F1.9030202@ee.byu.edu>
Message-ID: <20061117200534.GA26581@mentat.za.net>

On Fri, Nov 17, 2006 at 12:43:45PM -0700, Travis Oliphant wrote:
> 
> The NumPy dtype command now understands ctypes data-types including 
> derived structures and arrays.
> 
> This means you can specify data-types using ctypes method of building 
> structures and pass those classes whenever NumPy expects a data-type 
> object.
> 
> Simple example:
> 
> import ctypes as ct
> import numpy as np
> 
> np.array([1,2,3],ct.c_short)
> 
> More complicated:
> 
> class POINT(ct.Structure):
>          _fields_ = [("x", ct.c_int),
>                            ("y", ct.c_int)]
> 
> np.array([(3,4),(5,6),(7,8)], dtype=POINT)

Is the memory layout of the array above the same as what an array of
POINTs would be in C, or are they not padded?  If not, is

dtype(POINT,align=True) supported?

Or am I rather looking for

dtype(POINT*3,align=True)

or are they equivalent in terms of memory use?

The reason I ask is because this would make it very easy to allocate
fairly complex C structures on the python side when wrapping.

Good stuff, thanks!

Cheers
St?fan


From oliphant at ee.byu.edu  Fri Nov 17 15:42:56 2006
From: oliphant at ee.byu.edu (Travis Oliphant)
Date: Fri, 17 Nov 2006 13:42:56 -0700
Subject: [Numpy-discussion] Support for understand ctypes data-types now
 in SVN
In-Reply-To: <20061117200534.GA26581@mentat.za.net>
References: <455E10F1.9030202@ee.byu.edu>
	<20061117200534.GA26581@mentat.za.net>
Message-ID: <455E1ED0.5080306@ee.byu.edu>

Stefan van der Walt wrote:

>On Fri, Nov 17, 2006 at 12:43:45PM -0700, Travis Oliphant wrote:
>  
>
>>The NumPy dtype command now understands ctypes data-types including 
>>derived structures and arrays.
>>
>>This means you can specify data-types using ctypes method of building 
>>structures and pass those classes whenever NumPy expects a data-type 
>>object.
>>
>>Simple example:
>>
>>import ctypes as ct
>>import numpy as np
>>
>>np.array([1,2,3],ct.c_short)
>>
>>More complicated:
>>
>>class POINT(ct.Structure):
>>         _fields_ = [("x", ct.c_int),
>>                           ("y", ct.c_int)]
>>
>>np.array([(3,4),(5,6),(7,8)], dtype=POINT)
>>    
>>
>
>Is the memory layout of the array above the same as what an array of
>POINTs would be in C, or are they not padded? 
>
I just changed this in SVN.  Previously it would not use padding because 
the array_descriptor format assumed all padding was specified.  I 
changed this so that array_descriptor format can also now account for 
padding. 

Then, if the object has a _fields_ attribute, then it automatically does 
padded construction.

As a result, now using ctypes objects will guarantee padding.

-Travis



From pearu at cens.ioc.ee  Sat Nov 18 04:59:03 2006
From: pearu at cens.ioc.ee (pearu at cens.ioc.ee)
Date: Sat, 18 Nov 2006 11:59:03 +0200 (EET)
Subject: [Numpy-discussion] changeset 3439 breaks Python 2.3 numpy build
Message-ID: <Pine.LNX.4.21.0611181155120.17161-100000@cens.kybi>


Hi,

http://projects.scipy.org/scipy/numpy/changeset/3439

uses Py_CLEAR that was introduced in Python 2.4.

Shall we drop Python 2.3 support?

Pearu



From oliphant at ee.byu.edu  Thu Nov 16 19:44:56 2006
From: oliphant at ee.byu.edu (Travis Oliphant)
Date: Thu, 16 Nov 2006 17:44:56 -0700
Subject: [Numpy-discussion] mysql -> record array
Message-ID: <455D0608.1080403@ee.byu.edu>

John Hunter wrote:

>>>>>> "Erin" == Erin Sheldon <erin.sheldon at gmail.com> writes:
>>>>>>           
>>>>>
>
>    Erin> The question I have been asking myself is "what is the
>    Erin> advantage of such an approach?".  It would be faster, but by
>
> In the use case that prompted this message, the pull from mysql took
> almost 3 seconds, and the conversion from lists to numpy arrays took
> more that 4 seconds.  We have a list of about 500000 2 tuples of
> floats.
>
> Digging in a little bit, we found that numpy is about 3x slower than
> Numeric here
>
>  peds-pc311:~> python test.py
>  with dtype: 4.25 elapsed seconds
>  w/o dtype 5.79 elapsed seconds
>  Numeric  1.58 elapsed seconds
>  24.0b2
>  1.0.1.dev3432
>
> Hmm... So maybe the question is -- is there some low hanging fruit
> here to get numpy speeds up?
>
> import time
> import numpy
> import numpy.random
> rand = numpy.random.rand
>
> x = [(rand(), rand()) for i in xrange(500000)]
> tnow = time.time()
> y = numpy.array(x, dtype=numpy.float_)
> tdone = time.time()
> print 'with dtype: %1.2f elapsed seconds'%(tdone - tnow)
>
> tnow = time.time()
> y = numpy.array(x)
> tdone = time.time()
> print 'w/o dtype %1.2f elapsed seconds'%(tdone - tnow)
>
> import Numeric
> tnow = time.time()
> y = Numeric.array(x, Numeric.Float)
> tdone = time.time()
> print 'Numeric  %1.2f elapsed seconds'%(tdone - tnow)
>
> print Numeric.__version__
> print numpy.__version__
>  
>

I just adapted Numarray's version of array (using the fromlist method) 
to NumPy.   This new change needs some testing as it is called in many, 
many ways.  But, I think it should be right (all tests of numpy and 
scipy pass with it).
With the change I get:

with dtype: 0.22 elapsed seconds
w/o dtype 5.02 elapsed seconds
Numeric  7.38 elapsed seconds
numarray  0.55 elapsed seconds
24.2
1.0.1.dev3437
1.5.1



-Travis





From ndcunliffe at gmail.com  Fri Nov 17 09:34:05 2006
From: ndcunliffe at gmail.com (nicholas cunliffe)
Date: Fri, 17 Nov 2006 15:34:05 +0100
Subject: [Numpy-discussion] mysql -> record array
In-Reply-To: <200611171421.22216.faltet@carabos.com>
References: <87slglwwwm.fsf@peds-pc311.bsd.uchicago.edu>
	<1163711662.2613.49.camel@localhost.localdomain>
	<331116dc0611161328s68e1cef1y31e8ceaefc2eb3e0@mail.gmail.com>
	<200611171421.22216.faltet@carabos.com>
Message-ID: <c9c8bf730611170634l4033dd80j9dacea4265e279c8@mail.gmail.com>

I think someone needs to sort out distribution lists, im getting lots
of unstructured emails under many different titles

On 11/17/06, Francesc Altet <faltet at carabos.com> wrote:
> A Dijous 16 Novembre 2006 22:28, Erin Sheldon escrigu?:
> > Hi Francesc -
> >
> > Unless I missed something, I think what you have
> > shown is that the combination of
> >       (getting data from database into python lists) +
> >       (converting to arrays)
> > is what is taking time.   I would guess the first takes
> > significantly longer than the second.
>
> Seriously, I don't think I have demonstrated nothing really solid in
> that regard with so little evidences. But we can try looking for more
> of those :)
>
> For example, I'd split the times in:
>
>     t1 (getting data from database) +
>     t2 (python lists) +
>     t3 (converting to arrays) =
>     tt (total time)
>
> We don't know t1, but we do know tt. Now, we can try to get a guess
> for t1 and t2. Perhaps I'm wrong, but the next could be good
> estimates.
>
> For t1 (creating the python list of tuples):
> In [44]: Timer("[(x,x) for x in np.arange(500000, dtype='float64')]", "import
> numpy as np").repeat(3,1)
> Out[44]: [0.55968594551086426, 0.48462891578674316, 0.4855189323425293]
>
> For t2 (converting to recarrays):
> In [49]: Timer("np.fromiter(lot, dtype=dtype)", "import numpy as np;
> lot=[(x,x) for x in np.arange(500000, dtype='float64')];
> dtype=np.dtype([('x', 'float64'), ('y', 'float64')])").repeat(3,1)
> Out[49]: [0.50310707092285156, 0.50920987129211426, 0.50304579734802246]
>
> So, it seems that t1 and t2 are similar and they take aproximately 0.5
> seconds each.
>
> Now, let me remember the timings for reading the databases on my
> laptop at work (a Pentium4 @ 2 GHz):
>
> setup SQLite took 23.5661110878 seconds
> retrieve SQLite took 3.26717996597 seconds
> setup PyTables took 0.139157056808 seconds
> retrieve PyTables took 0.13444685936 seconds
>
> So, in our case, tt for SQLite3 was 3.26 seconds. With that, we can
> derive its t1 (getting data from database):
>
> t1 = tt - t1 - t2 =~ 2.26 seconds
>
> However, this is still far more than tt for PyTables (~ 0.14 sec), so
> I'm not completely sure what's going on. Honest, I don't think that
> HDF5 (the underlying library for doing I/O in PyTables) would be
> almost 20x faster than SQLite3 for reading purposes. So my guess is
> that there should be more factors contributing tt for SQLite3 that
> I've not taken in account. Anyone can find them?
>
> Cheers,
>
> --
> >0,0<   Francesc Altet http://www.carabos.com/
> V   V   C?rabos Coop. V. Enjoy Data
>  "-"
>
> _______________________________________________
> Numpy-discussion mailing list
> Numpy-discussion at scipy.org
> http://projects.scipy.org/mailman/listinfo/numpy-discussion
>


From erin.sheldon at gmail.com  Sat Nov 18 10:35:06 2006
From: erin.sheldon at gmail.com (Erin Sheldon)
Date: Sat, 18 Nov 2006 10:35:06 -0500
Subject: [Numpy-discussion] mysql -> record array
In-Reply-To: <455D0608.1080403@ee.byu.edu>
References: <455D0608.1080403@ee.byu.edu>
Message-ID: <331116dc0611180735u1d0f3c82wbb78f3a6d96b64f@mail.gmail.com>

On 11/16/06, Travis Oliphant <oliphant at ee.byu.edu> wrote:
> John Hunter wrote:
>
> >>>>>> "Erin" == Erin Sheldon <erin.sheldon at gmail.com> writes:
> >>>>>>
> >>>>>
> >
> >    Erin> The question I have been asking myself is "what is the
> >    Erin> advantage of such an approach?".  It would be faster, but by
> >
> > In the use case that prompted this message, the pull from mysql took
> > almost 3 seconds, and the conversion from lists to numpy arrays took
> > more that 4 seconds.  We have a list of about 500000 2 tuples of
> > floats.
> >
> > Digging in a little bit, we found that numpy is about 3x slower than
> > Numeric here
> >
> >  peds-pc311:~> python test.py
> >  with dtype: 4.25 elapsed seconds
> >  w/o dtype 5.79 elapsed seconds
> >  Numeric  1.58 elapsed seconds
> >  24.0b2
> >  1.0.1.dev3432
> >
> > Hmm... So maybe the question is -- is there some low hanging fruit
> > here to get numpy speeds up?
> >
> > import time
> > import numpy
> > import numpy.random
> > rand = numpy.random.rand
> >
> > x = [(rand(), rand()) for i in xrange(500000)]
> > tnow = time.time()
> > y = numpy.array(x, dtype=numpy.float_)
> > tdone = time.time()
> > print 'with dtype: %1.2f elapsed seconds'%(tdone - tnow)
> >
> > tnow = time.time()
> > y = numpy.array(x)
> > tdone = time.time()
> > print 'w/o dtype %1.2f elapsed seconds'%(tdone - tnow)
> >
> > import Numeric
> > tnow = time.time()
> > y = Numeric.array(x, Numeric.Float)
> > tdone = time.time()
> > print 'Numeric  %1.2f elapsed seconds'%(tdone - tnow)
> >
> > print Numeric.__version__
> > print numpy.__version__
> >
> >
>
> I just adapted Numarray's version of array (using the fromlist method)
> to NumPy.   This new change needs some testing as it is called in many,
> many ways.  But, I think it should be right (all tests of numpy and
> scipy pass with it).
> With the change I get:
>
> with dtype: 0.22 elapsed seconds
> w/o dtype 5.02 elapsed seconds
> Numeric  7.38 elapsed seconds
> numarray  0.55 elapsed seconds
> 24.2
> 1.0.1.dev3437
> 1.5.1

Hi Travis -

That is an impressive speed increase.  Why is w/o dtype taking
so much longer?  Is this just from determining elements sizes and
counts?

Erin


From oliphant.travis at ieee.org  Sat Nov 18 11:06:09 2006
From: oliphant.travis at ieee.org (Travis Oliphant)
Date: Sat, 18 Nov 2006 09:06:09 -0700
Subject: [Numpy-discussion] mysql -> record array
In-Reply-To: <331116dc0611180735u1d0f3c82wbb78f3a6d96b64f@mail.gmail.com>
References: <455D0608.1080403@ee.byu.edu>
	<331116dc0611180735u1d0f3c82wbb78f3a6d96b64f@mail.gmail.com>
Message-ID: <455F2F71.8020506@ieee.org>

Erin Sheldon wrote:
> Hi Travis -
>
> That is an impressive speed increase.  Why is w/o dtype taking
> so much longer?  Is this just from determining elements sizes and
> counts?
>   
If you don't specify the data-type, then the auto-discovery algorithm 
looks at each element in the nested sequence to determine what the 
data-type should be.  This takes a while.  

-Travis




From robert.kern at gmail.com  Sat Nov 18 15:00:38 2006
From: robert.kern at gmail.com (Robert Kern)
Date: Sat, 18 Nov 2006 14:00:38 -0600
Subject: [Numpy-discussion] mysql -> record array
In-Reply-To: <c9c8bf730611170634l4033dd80j9dacea4265e279c8@mail.gmail.com>
References: <87slglwwwm.fsf@peds-pc311.bsd.uchicago.edu>	<1163711662.2613.49.camel@localhost.localdomain>	<331116dc0611161328s68e1cef1y31e8ceaefc2eb3e0@mail.gmail.com>	<200611171421.22216.faltet@carabos.com>
	<c9c8bf730611170634l4033dd80j9dacea4265e279c8@mail.gmail.com>
Message-ID: <455F6666.9020003@gmail.com>

nicholas cunliffe wrote:
> I think someone needs to sort out distribution lists, im getting lots
> of unstructured emails under many different titles

What exactly is the problem? If you mean that you used to get the digests rather
than the list emails themselves, this is an unfortunate side effect of migrating
the list from Sourceforge. The only thing we could export is the subscription
list, not everyone's options. See my earlier post on how to reset your options.

http://projects.scipy.org/pipermail/numpy-discussion/2006-November/024731.html

-- 
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless enigma
 that is made terrible by our own mad attempt to interpret it as though it had
 an underlying truth."
  -- Umberto Eco


From robert.kern at gmail.com  Sat Nov 18 15:02:24 2006
From: robert.kern at gmail.com (Robert Kern)
Date: Sat, 18 Nov 2006 14:02:24 -0600
Subject: [Numpy-discussion] changeset 3439 breaks Python 2.3 numpy build
In-Reply-To: <Pine.LNX.4.21.0611181155120.17161-100000@cens.kybi>
References: <Pine.LNX.4.21.0611181155120.17161-100000@cens.kybi>
Message-ID: <455F66D0.7000007@gmail.com>

pearu at cens.ioc.ee wrote:
> Hi,
> 
> http://projects.scipy.org/scipy/numpy/changeset/3439
> 
> uses Py_CLEAR that was introduced in Python 2.4.
> 
> Shall we drop Python 2.3 support?

Please, no.

-- 
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless enigma
 that is made terrible by our own mad attempt to interpret it as though it had
 an underlying truth."
  -- Umberto Eco


From johanfo at ohman.no  Sat Nov 18 17:18:20 2006
From: johanfo at ohman.no (=?iso-8859-1?Q?Johan_Fredrik_=D8hman?=)
Date: Sat, 18 Nov 2006 23:18:20 +0100
Subject: [Numpy-discussion] changeset 3439 breaks Python 2.3 numpy build
In-Reply-To: <455F66D0.7000007@gmail.com>
References: <Pine.LNX.4.21.0611181155120.17161-100000@cens.kybi>
	<455F66D0.7000007@gmail.com>
Message-ID: <000601c70b5f$74c22e30$5e468a90$@no>

unsubscribe

-----Original Message-----
From: numpy-discussion-bounces at scipy.org
[mailto:numpy-discussion-bounces at scipy.org] On Behalf Of Robert Kern
Sent: 18. november 2006 21:02
To: Discussion of Numerical Python
Subject: Re: [Numpy-discussion] changeset 3439 breaks Python 2.3 numpy build

pearu at cens.ioc.ee wrote:
> Hi,
> 
> http://projects.scipy.org/scipy/numpy/changeset/3439
> 
> uses Py_CLEAR that was introduced in Python 2.4.
> 
> Shall we drop Python 2.3 support?

Please, no.

-- 
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless enigma
 that is made terrible by our own mad attempt to interpret it as though it
had
 an underlying truth."
  -- Umberto Eco
_______________________________________________
Numpy-discussion mailing list
Numpy-discussion at scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion



From emsellem at obs.univ-lyon1.fr  Sun Nov 19 08:30:52 2006
From: emsellem at obs.univ-lyon1.fr (Eric Emsellem)
Date: Sun, 19 Nov 2006 14:30:52 +0100
Subject: [Numpy-discussion] mysql -> record array
In-Reply-To: <455F2F71.8020506@ieee.org>
References: <455D0608.1080403@ee.byu.edu>
	<331116dc0611180735u1d0f3c82wbb78f3a6d96b64f@mail.gmail.com>
	<455F2F71.8020506@ieee.org>
Message-ID: <1163943052.45605c8ca89c3@webmail.univ-lyon1.fr>

Hi Travis,

sorry to bother you with that, but who's responsible with the numpy list?
I am receiving ALL numpy-list emails directly now (although I always opted for
the bundle option and checked that it is still on). I sent a mail about this,
but no answer so far.

Anything wrong with the list? Something changed ? (my mailbox is getting really
unoperable..)

thanks

Eric

> Erin Sheldon wrote:
> > Hi Travis -
> >
> > That is an impressive speed increase.  Why is w/o dtype taking
> > so much longer?  Is this just from determining elements sizes and
> > counts?
> >
> If you don't specify the data-type, then the auto-discovery algorithm
> looks at each element in the nested sequence to determine what the
> data-type should be.  This takes a while.
>
> -Travis
>
>
> _______________________________________________
> Numpy-discussion mailing list
> Numpy-discussion at scipy.org
> http://projects.scipy.org/mailman/listinfo/numpy-discussion
>


-- 


--------------------------------------------------------------------------
Ce message a ?t? envoy? depuis le webmail IMP (Internet Messaging Program)


From tim.hochberg at ieee.org  Sun Nov 19 09:00:06 2006
From: tim.hochberg at ieee.org (Tim Hochberg)
Date: Sun, 19 Nov 2006 07:00:06 -0700
Subject: [Numpy-discussion] mysql -> record array
In-Reply-To: <1163943052.45605c8ca89c3@webmail.univ-lyon1.fr>
References: <455D0608.1080403@ee.byu.edu>	<331116dc0611180735u1d0f3c82wbb78f3a6d96b64f@mail.gmail.com>	<455F2F71.8020506@ieee.org>
	<1163943052.45605c8ca89c3@webmail.univ-lyon1.fr>
Message-ID: <45606366.3020800@ieee.org>

Eric Emsellem wrote:
> Hi Travis,
>
> sorry to bother you with that, but who's responsible with the numpy list?
> I am receiving ALL numpy-list emails directly now (although I always opted for
> the bundle option and checked that it is still on). I sent a mail about this,
> but no answer so far.
>
> Anything wrong with the list? Something changed ? (my mailbox is getting really
> unoperable..)
>
>   
This has been answered about 4 times now in all that mail you're trying 
to ignore. Here it is again in gory detail:

1. Got to http://projects.scipy.org/mailman/listinfo/numpy-discussion
2. Go to the bottom to "Unsubscribe or edit options" and enter your 
email address
3. Ask for a password reminder.
4. When you get your password login and edit your options to receive 
digests.

-tim



From sherwood at cam.cornell.edu  Sun Nov 19 12:23:56 2006
From: sherwood at cam.cornell.edu (Erik Sherwood)
Date: Sun, 19 Nov 2006 12:23:56 -0500
Subject: [Numpy-discussion] Fortran & distutils on Mac OS X -- interactive
	mode?
Message-ID: <E7C703B2-D80B-4561-97C1-559021F810F3@cam.cornell.edu>

Hi all,

I am using scipy-0.5.1 and numpy-1.0 on Mac OS X 10.4.8 (MacBook Pro  
late 2006) built with fink. I have code (I'm developing for the  
PyDSTool project) which uses numpy.distutils to build a .so file  
which is then loaded into python. When only C compilers are needed,  
everything runs as I expect, but when I need to build with fortran  
compilers (g95 in this case; there is no g77 built for the Intel macs  
yet), I get sent to interactive mode:

========================================================================
Starting interactive session
------------------------------------------------------------------------

Tasks:
   i       - Show python/platform/machine information
   ie      - Show environment information
   c       - Show C compilers information
   c<name> - Set C compiler (current:None)
   f       - Show Fortran compilers information
   f<name> - Set Fortran compiler (current:None)
   e       - Edit proposed sys.argv[1:].

Task aliases:
   0         - Configure
   1         - Build
   2         - Install
   2<prefix> - Install with prefix.
   3         - Inplace build
   4         - Source distribution
   5         - Binary distribution

Proposed sys.argv = ['SLIP_2D_pdc.py']

Choose a task (^D to quit, Enter to continue with setup):

I can just hit enter and things run as I like them to, more or less:
------------------------------------------------------------------------
customize AbsoftFCompiler
customize IbmFCompiler
Could not locate executable g77
Could not locate executable f77
Could not locate executable gfortran
Could not locate executable f95
customize GnuFCompiler
customize Gnu95FCompiler
customize G95FCompiler
customize G95FCompiler
customize G95FCompiler using build_clib
building 'radau5' library
compiling Fortran sources
Fortran f77 compiler: g95 -ffixed-form -fno-second-underscore -O3 - 
funroll-loops
Fortran f90 compiler: g95 -fno-second-underscore -O3 -funroll-loops
Fortran fix compiler: g95 -ffixed-form -fno-second-underscore -O3 - 
funroll-loops
creating /Users/sherwood/Apps/PyDSTool/tests/radau5_temp/Users
creating /Users/sherwood/Apps/PyDSTool/tests/radau5_temp/Users/sherwood
creating /Users/sherwood/Apps/PyDSTool/tests/radau5_temp/Users/ 
sherwood/Apps
creating /Users/sherwood/Apps/PyDSTool/tests/radau5_temp/Users/ 
sherwood/Apps/PyDSTool
creating /Users/sherwood/Apps/PyDSTool/tests/radau5_temp/Users/ 
sherwood/Apps/PyDSTool/integrator
compile options: '-c'
g95:f77: /Users/sherwood/Apps/PyDSTool/integrator/radau5.f
....

The code that uses distutils looks like this (radau5 refers to the  
fortran code I'm compiling):

if not (all([os.path.isfile(os.path.join(self._compilation_tempdir,
                    sf)) for sf in ['radau5'+self._vf_filename_ext 
+'_wrap.o',
                                    'lib_radau5'+self._vf_filename_ext 
+'.a',
                                    'radau5'+self._vf_filename_ext 
+'.py',
                                    '_radau5'+self._vf_filename_ext 
+'.def']])):
             modfilelist = [swigfile]
         else:
             modfilelist = []
         modfilelist.extend([radwrapfile, vffile, integfile, eventfile,
                            interfacefile, memfile])
         modfilelist.extend(libsources)
         fortfilelist=[radfile, lapackcfile, dc_lapackfile, lapackfile]
         # would like temp dir to be '/' but scipy_distutils cannot  
build the
         # radau library in that configuration (unlike Dopri's  
identical use of
         # regular python distutils, which cannot deal with Fortran  
sources).
         script_args = ['-q', 'build', '--build-lib='+os.getcwd(),
                  '-t'+self._compilation_tempdir,
                  '--build-base='+self._compilation_tempdir,
                  '--build-platlib='+os.getcwd(), '--build- 
purelib='+os.getcwd()]
         if self._compiler != '':
             script_args.append('-c'+str(self._compiler))
         # include directories for libraries
	narraydir = npy.get_numarray_include()
	npydir = npy.get_include()
         incdirs = [npydir, narraydir, os.getcwd(),  
self._compilation_tempdir,
                    self._compilation_sourcedir]
         incdirs.extend(libdirs)
         radlibdirs = [self._compilation_sourcedir,  
self._compilation_tempdir,
                        os.getcwd()]
         # Use distutils to perform the compilation of the selected  
files
        try:
             distobject = setup(name = "Radau5 integrator",
                   author = "PyDSTool (automatically generated)",
                   script_args = script_args,
                   ext_modules = [Extension 
("_radau5"+self._vf_filename_ext,
                                  sources=modfilelist,
                                  include_dirs=incdirs,
                                  library_dirs=radlibdirs+['./'],
                                  libraries=['radau5'],
                                  extra_compile_args=['-w', '- 
D__RADAU__'],
                                  extra_link_args=['-w'])],
                   	libraries=[('radau5',{'sources': fortfilelist,
                               'library_dirs': radlibdirs+['./']})])


I would like to be able to skip the interactive mode -- how should I  
invoke distutils to skip all of the interaction and go straight to  
compilation?

Thanks,

Erik Sherwood
Center for Applied Mathematics	|	Phone: 	(607) 255-4195
657 Rhodes Hall					|	Fax:		(607) 255-9860
Cornell University				|	Email:	sherwood at cam.cornell.edu
Ithaca, NY 14853					|	Web:	http://www.cam.cornell.edu/~sherwood





From tim.hochberg at ieee.org  Sun Nov 19 13:01:57 2006
From: tim.hochberg at ieee.org (Tim Hochberg)
Date: Sun, 19 Nov 2006 11:01:57 -0700
Subject: [Numpy-discussion] fromiter and objects
Message-ID: <45609C15.7050602@ieee.org>


I was looking at fromiter again today with an eye toward extending it to 
accept iterators of sequences instead of just iterators of scalars. For 
example:

    fromiter(([x, x // 2, x+5] for x in range(1000)), dtype=int)

This would result in a shape-(1000,3) array. At first glance at least, 
this looks straightforward: one would simply have to correctly deduce 
size of the sequence with the given dtype and I imagine that I can 
enlist existing numpy machinery to do this for me without a problem. But 
enough about that, I won't be able to try this till next week, and these 
things are often not as easy as they appear, the real reason I'm writing 
is this comment that explains why object arrays are disallowed in 
fromiter (multiarraymodule.c::PyArray_:FromIter)

            /* We would need to alter the memory RENEW code to decrement any
               reference counts before just throwing away the memory.
             */

This doesn't seem right. The array that we would be RENEWing is a bunch 
of PyObject*s. The reference counts don't reside there, but in the 
objects themselves. When we do the RENEW, we don't want the reference 
counts to change at all. The one tricky case is if we run out of memory, 
I'm not certain that the current setup correctly deals with reference 
counts in this case, although it appears likely that it should work 
since ret->data should still point to a valid chunk of memory and 
decreffing ret should result in the subsequent deallocation of all the 
stored objects.

So, it looks like objects should either just work, or should work with a 
minimal amount of tweaking. However, it's possible that I'm getting 
rusty at Python extension writing (or more to the point, reading). Does 
anyone remember if this check was added to address a specific problem? 
If so, do you also remember what it is? I suppose I can track back 
through the revision history if no one remembers, but I figured I'd try 
the lazy approach first and ask about it.

-tim



From aisaac at american.edu  Sun Nov 19 13:04:24 2006
From: aisaac at american.edu (Alan G Isaac)
Date: Sun, 19 Nov 2006 13:04:24 -0500
Subject: [Numpy-discussion] changeset 3439 breaks Python 2.3 numpy build
In-Reply-To: <000601c70b5f$74c22e30$5e468a90$@no>
References: <Pine.LNX.4.21.0611181155120.17161-100000@cens.kybi><455F66D0.7000007@gmail.com><000601c70b5f$74c22e30$5e468a90$@no>
Message-ID: <Mahogany-0.67.0-960-20061119-130424.00@american.edu>

On Sat, 18 Nov 2006, Johan Fredrik ?hman apparently wrote: 
> unsubscribe 

At the bottom of each email you will find:
http://projects.scipy.org/mailman/listinfo/numpy-discussion

Cheers,
Alan Isaac







From peridot.faceted at gmail.com  Sun Nov 19 15:42:03 2006
From: peridot.faceted at gmail.com (A. M. Archibald)
Date: Sun, 19 Nov 2006 15:42:03 -0500
Subject: [Numpy-discussion] fiddling the strides of an array
Message-ID: <ce557a360611191242i35a06f5et2888c0f2c48c177d@mail.gmail.com>

Hi,

I have a function that would like to be able to take an array, look at
its 'strides' and 'shape' tuples, and fabricate another array that is
similar to the first but has the adjusted values.

For a simple example:

def fiddle(a):
    strides = list(a.strides)
    strides[0]*=2
    shape = list(a.shape)
    shape[0]//=2
    return N.ndarray.__new__(N.ndarray, strides=strides, shape=shape,
buffer=a, dtype=a.dtype)

Unfortunately, this has a few problems.

* It always makes ndarrays, even if the original was some subtype.
(not a big deal for me, but still a bit annoying)

* It fails unnecessarily if the array is not "contiguous": for
example, fiddle(N.zeros((2,2,2,2)).swapaxes(0,2)) fails, saying
"expected a single-segment buffer object".

This second problem is my real problem. Is there a way I can tell
numpy "no really, I know what I'm doing with these strides, use the
same underlying memory as this array"? (Of course I can flatten the
array, but that will copy it unnecessarily.)

Thanks,
A. M. Archibald


From pearu at cens.ioc.ee  Sun Nov 19 15:59:11 2006
From: pearu at cens.ioc.ee (pearu at cens.ioc.ee)
Date: Sun, 19 Nov 2006 22:59:11 +0200 (EET)
Subject: [Numpy-discussion] Fortran & distutils on Mac OS X --
 interactive mode?
In-Reply-To: <E7C703B2-D80B-4561-97C1-559021F810F3@cam.cornell.edu>
Message-ID: <Pine.LNX.4.21.0611192257230.17331-100000@cens.kybi>



On Sun, 19 Nov 2006, Erik Sherwood wrote:

> I would like to be able to skip the interactive mode -- how should I  
> invoke distutils to skip all of the interaction and go straight to  
> compilation?

I have commited a patch to numpy svn so that entering to interactive mode
will be skipped when using script_args setup argument.

Pearu




From filip.wasilewski at gmail.com  Sun Nov 19 16:07:27 2006
From: filip.wasilewski at gmail.com (Filip Wasilewski)
Date: Sun, 19 Nov 2006 22:07:27 +0100
Subject: [Numpy-discussion] fiddling the strides of an array
In-Reply-To: <ce557a360611191242i35a06f5et2888c0f2c48c177d@mail.gmail.com>
References: <ce557a360611191242i35a06f5et2888c0f2c48c177d@mail.gmail.com>
Message-ID: <c3e57df40611191307w204861e2q764963cd08572386@mail.gmail.com>

On 11/19/06, A. M. Archibald <peridot.faceted at gmail.com> wrote:
> Hi,
>
> I have a function that would like to be able to take an array, look at
> its 'strides' and 'shape' tuples, and fabricate another array that is
> similar to the first but has the adjusted values.
>
> For a simple example:
>
> def fiddle(a):
>     strides = list(a.strides)
>     strides[0]*=2
>     shape = list(a.shape)
>     shape[0]//=2
>     return N.ndarray.__new__(N.ndarray, strides=strides, shape=shape,
> buffer=a, dtype=a.dtype)

Is there anything wrong in using slicing for that?

>>> m=numpy.matrix([[2,3,4], [3,2,1]])
>>> m.shape, m.strides
((2, 3), (12, 4))
>>> n = m[::2,:]
>>> n.shape, n.strides
((1, 3), (24, 4))

cheers,
fw


From peridot.faceted at gmail.com  Sun Nov 19 16:10:44 2006
From: peridot.faceted at gmail.com (A. M. Archibald)
Date: Sun, 19 Nov 2006 16:10:44 -0500
Subject: [Numpy-discussion] fiddling the strides of an array
In-Reply-To: <c3e57df40611191307w204861e2q764963cd08572386@mail.gmail.com>
References: <ce557a360611191242i35a06f5et2888c0f2c48c177d@mail.gmail.com>
	<c3e57df40611191307w204861e2q764963cd08572386@mail.gmail.com>
Message-ID: <ce557a360611191310q4db52e0fw4308ab31a6f7fca@mail.gmail.com>

On 19/11/06, Filip Wasilewski <filip.wasilewski at gmail.com> wrote:
> On 11/19/06, A. M. Archibald <peridot.faceted at gmail.com> wrote:
> > Hi,
> >
> > I have a function that would like to be able to take an array, look at
> > its 'strides' and 'shape' tuples, and fabricate another array that is
> > similar to the first but has the adjusted values.
> >
> > For a simple example:
> >
> > def fiddle(a):
> >     strides = list(a.strides)
> >     strides[0]*=2
> >     shape = list(a.shape)
> >     shape[0]//=2
> >     return N.ndarray.__new__(N.ndarray, strides=strides, shape=shape,
> > buffer=a, dtype=a.dtype)
>
> Is there anything wrong in using slicing for that?
>
> >>> m=numpy.matrix([[2,3,4], [3,2,1]])
> >>> m.shape, m.strides
> ((2, 3), (12, 4))
> >>> n = m[::2,:]
> >>> n.shape, n.strides
> ((1, 3), (24, 4))

Well, no, of course not to implement "fiddle". But the application I
have in mind I don't think can be done with slicing...

A. M. Archibald


From michael.sorich at gmail.com  Sun Nov 19 22:18:41 2006
From: michael.sorich at gmail.com (Michael Sorich)
Date: Mon, 20 Nov 2006 13:48:41 +1030
Subject: [Numpy-discussion] MaskedArray.__array__ bug? (was 'A
	reimplementation of MaskedArray')
In-Reply-To: <16761e100611191913m75d34789x53c9bff70418c161@mail.gmail.com>
References: <16761e100611191913m75d34789x53c9bff70418c161@mail.gmail.com>
Message-ID: <16761e100611191918s769284ecr1fe724bfe3e67cb2@mail.gmail.com>

On 11/9/06, Paul Dubois <pfdubois at gmail.com> wrote:
> 2 cents from the author of the first folio:
>
> The intent was to allow creation of masked arrays with copy=no, so that the
> original data could be retrieved from it if desired. But I was quite, quite
> rigorous about NEVER assuming the data in a masked slot made any sense
> whatsoever.
>
> The intention was that there are two ways to get a numeric array out of a
> masked one:
>
> 1. Get the data field
> 2. m.filled()
>
> (1) is strictly at your own risk.

Is there consensus that in __array__ it is incorrect to return the
_data component of a MaskedArray when there are masked values? It
certainly worries me that what is underneath the mask is returned.

Currently in numpy.core.ma MaskedArray class
def __array__ (self, t=None, context=None):
        "Special hook for numeric. Converts to numeric if possible."
        if self._mask is not nomask:
            if fromnumeric.ravel(self._mask).any():
                if context is None:
                    warnings.warn("Cannot automatically convert masked
array to "\
                                  "numeric because data\n    is masked
in one or "\
                                  "more locations.");
                    return self._data

I think the warning should be an exception. Alternatively, there could
be subclass of masked array for uses in which the masked data
represents missing data (hence data under the mask is spurious and
should not be exposed). In this subclass the __array__ method could be
redefined to impose stricter control of the masked data.


From charlesr.harris at gmail.com  Sun Nov 19 23:06:53 2006
From: charlesr.harris at gmail.com (Charles R Harris)
Date: Sun, 19 Nov 2006 21:06:53 -0700
Subject: [Numpy-discussion] Inline functions?
Message-ID: <e06186140611192006l59fd21cds158ad438da664772@mail.gmail.com>

I wonder if we couldn't make use of inline functions for some of the bigger
macros? For instance:

#define PyArray_ITER_NEXT(it) {                                     \
        (it)->index++;                                            \
        if ((it)->nd_m1 == 0) {                                   \
                _PyArray_ITER_NEXT1(it);                          \
        }                                                               \
        else if ((it)->contiguous)  (it)->dataptr +=
(it)->ao->descr->elsize; \
        else if ((it)->nd_m1 == 1) {                              \
                _PyArray_ITER_NEXT2(it);                          \
        }                                                               \
        else {                                                          \
                int __npy_i;                                            \
                for (__npy_i = (it)->nd_m1; __npy_i >= 0; __npy_i--)
{          \
                        if ((it)->coordinates[__npy_i]
<                        \
                            (it)->dims_m1[__npy_i])
{                           \

(it)->coordinates[__npy_i]++;                   \
                                (it)->dataptr +=
(it)->strides[__npy_i];        \
                                break;                                  \
                        }                                               \
                        else {                                          \
                                (it)->coordinates[__npy_i] = 0;         \
                                (it)->dataptr -=
(it)->backstrides[__npy_i];    \
                        }                                               \
                }                                                       \
        }                                                               \
}

I admit using a function might slow down the common case, but maybe we could
figure some way of keeping the common case small so it goes inline while
leaving the rest as a function. For instance,

inline PyArray_ITER_NEXT(iter* it)
{
    it->index++;
    if(it->nd_ml == 0 && it->contiguous)
        it->dataptr += it->ao->descr->elsize;
    else
        punt(it);
}

And if *that* isn't inlined we can probable just make the whole thing a
normal function without losing much. I suppose a drawback to using inline
functions is that a reasonably modern C compiler needs to be used that
conforms to C-99.

Chuck
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20061119/a9b542ce/attachment.html>

From pgmdevlist at gmail.com  Mon Nov 20 00:31:47 2006
From: pgmdevlist at gmail.com (Pierre GM)
Date: Mon, 20 Nov 2006 00:31:47 -0500
Subject: [Numpy-discussion] MaskedArray.__array__ bug? (was 'A
	reimplementation of MaskedArray')
In-Reply-To: <16761e100611191918s769284ecr1fe724bfe3e67cb2@mail.gmail.com>
References: <16761e100611191913m75d34789x53c9bff70418c161@mail.gmail.com>
	<16761e100611191918s769284ecr1fe724bfe3e67cb2@mail.gmail.com>
Message-ID: <200611200031.47647.pgmdevlist@gmail.com>

> Is there consensus that in __array__ it is incorrect to return the
> _data component of a MaskedArray when there are masked values? It
> certainly worries me that what is underneath the mask is returned.

When is __array__ really used in practice ? Most of the time, isn't  the 
conversion to ndarray done with numpy.core.numeric.asarray ? Which doesn't 
seem to call the __array__ method...

> Alternatively, there could 
> be subclass of masked array for uses in which the masked data
> represents missing data (hence data under the mask is spurious and
> should not be exposed). In this subclass the __array__ method could be
> redefined to impose stricter control of the masked data.
Sounds like a good idea, but we're back on question #1.

I think that one issue is that N.asarray, N.array, etc. access the _data part 
directly, when in fact, they should ideally access the filled version. A 
workaround is to ensure that your _data has some 'well-defined' missing data, 
by filling them beforehand with something like nan (which won't work if you 
work with ints, though...). But that gets heavy.



From michael.sorich at gmail.com  Mon Nov 20 01:43:49 2006
From: michael.sorich at gmail.com (Michael Sorich)
Date: Mon, 20 Nov 2006 17:13:49 +1030
Subject: [Numpy-discussion] bug in chararray startswith function?
Message-ID: <16761e100611192243n4356067dm17f3854a1a7f6db7@mail.gmail.com>

python 2.4.3 with numpy 1.0

import numpy
ca = numpy.char.asarray(['hi','there'])
print ca
print ca.startswith('h')

--output
['hi' 'there']
Traceback (most recent call last):
  File "D:\eclipse\NumpyExtensions\scripts\testchararray.py", line 4, in ?
    print ca.startswith('h')
  File "C:\Python24\Lib\site-packages\numpy\core\defchararray.py",
line 271, in startswith
    return self._typedmethod('startswith', broadcast(self, prefix,
start, end), bool)
  File "C:\Python24\Lib\site-packages\numpy\core\defchararray.py",
line 154, in _typedmethod
    if chk.dtype is object_ and chk.item() is None:
AttributeError: 'NoneType' object has no attribute 'dtype'

It looks like this is because the 2 default parameters (start=None,
end=None) and when this is passed to _typedmethod, val = ('hi', 'h',
None, None). Since None does not have a dtype, the error is raised. If
one puts in the additional parameters there is not error as the
additional vars are converted in numpy int scalars
e.g.

defchararray.py
    def _typedmethod(self, name, myiter, dtype):
        result = empty(myiter.shape, dtype=dtype)
        res = result.flat
        for k, val in enumerate(myiter):
            newval = []
            for chk in val[1:]:
                if chk.dtype is object_ and chk.item() is None:
                    break
                newval.append(chk)
            this_str = val[0].rstrip('\x00')
            newitem = getattr(this_str,name)(*newval)
            res[k] = newitem
        return result


From stefan at sun.ac.za  Mon Nov 20 02:27:07 2006
From: stefan at sun.ac.za (Stefan van der Walt)
Date: Mon, 20 Nov 2006 09:27:07 +0200
Subject: [Numpy-discussion] bug in chararray startswith function?
In-Reply-To: <16761e100611192243n4356067dm17f3854a1a7f6db7@mail.gmail.com>
References: <16761e100611192243n4356067dm17f3854a1a7f6db7@mail.gmail.com>
Message-ID: <20061120072707.GO26581@mentat.za.net>

On Mon, Nov 20, 2006 at 05:13:49PM +1030, Michael Sorich wrote:
> python 2.4.3 with numpy 1.0
> 
> import numpy
> ca = numpy.char.asarray(['hi','there'])
> print ca
> print ca.startswith('h')
> 
> --output
> ['hi' 'there']
> Traceback (most recent call last):
>   File "D:\eclipse\NumpyExtensions\scripts\testchararray.py", line 4, in ?

Thanks, should be fixed in SVN.

Cheers
St?fan


From Achim.Gaedke at physik.tu-darmstadt.de  Mon Nov 20 07:13:42 2006
From: Achim.Gaedke at physik.tu-darmstadt.de (Achim Gaedke)
Date: Mon, 20 Nov 2006 13:13:42 +0100
Subject: [Numpy-discussion] python-numarray: _dotblas.dot calls spoils
	Py_None refcount
Message-ID: <45619BF6.9040103@physik.tu-darmstadt.de>

Hi!

Using the latest numarray distribution for debian testing I found a bug.
Executing the python script with python2.4 shows the decrement of None
refcount:

import sys
import numarray

data1=numarray.arange(4096*2, type=numarray.Float32)
data2=numarray.arange(4096*2, type=numarray.Float32)

while True:
    print sys.getrefcount(None)
    val=numarray.dot(data1, data2)


I could trace this error down to the function NA_FromDimsStridesDescrAndData
in numarray/Src/libnumarray.ch .

The bug fix is:
--- python-numarray-1.5.2/Src/libnumarray.ch    2006-08-24
20:38:15.000000000 +0200
+++ /home/achim/libnumarray.ch  2006-11-19 21:14:52.000000000 +0100
@@ -877,6 +877,7 @@
                if (!buf) return NULL;
        } else {
                buf = Py_None;
+               Py_INCREF(buf);
        }

        a = NA_NewAllFromBuffer( nd, dimensions, descr->type_num, buf,

I think it is still worth fixing this bug. I've sent that bug report
#399440 to Debian, too.

Yours, Achim



From jmiller at stsci.edu  Mon Nov 20 13:21:09 2006
From: jmiller at stsci.edu (Todd Miller)
Date: Mon, 20 Nov 2006 13:21:09 -0500
Subject: [Numpy-discussion] python-numarray: _dotblas.dot calls spoils
 Py_None refcount
In-Reply-To: <45619BF6.9040103@physik.tu-darmstadt.de>
References: <45619BF6.9040103@physik.tu-darmstadt.de>
Message-ID: <4561F215.6010508@stsci.edu>

This is in CVS now.  Thanks for the patch!

Todd

Achim Gaedke wrote:
> Hi!
>
> Using the latest numarray distribution for debian testing I found a bug.
> Executing the python script with python2.4 shows the decrement of None
> refcount:
>
> import sys
> import numarray
>
> data1=numarray.arange(4096*2, type=numarray.Float32)
> data2=numarray.arange(4096*2, type=numarray.Float32)
>
> while True:
>     print sys.getrefcount(None)
>     val=numarray.dot(data1, data2)
>
>
> I could trace this error down to the function NA_FromDimsStridesDescrAndData
> in numarray/Src/libnumarray.ch .
>
> The bug fix is:
> --- python-numarray-1.5.2/Src/libnumarray.ch    2006-08-24
> 20:38:15.000000000 +0200
> +++ /home/achim/libnumarray.ch  2006-11-19 21:14:52.000000000 +0100
> @@ -877,6 +877,7 @@
>                 if (!buf) return NULL;
>         } else {
>                 buf = Py_None;
> +               Py_INCREF(buf);
>         }
>
>         a = NA_NewAllFromBuffer( nd, dimensions, descr->type_num, buf,
>
> I think it is still worth fixing this bug. I've sent that bug report
> #399440 to Debian, too.
>
> Yours, Achim
>
> _______________________________________________
> Numpy-discussion mailing list
> Numpy-discussion at scipy.org
> http://projects.scipy.org/mailman/listinfo/numpy-discussion
>   



From peridot.faceted at gmail.com  Mon Nov 20 14:20:33 2006
From: peridot.faceted at gmail.com (A. M. Archibald)
Date: Mon, 20 Nov 2006 14:20:33 -0500
Subject: [Numpy-discussion] fiddling the strides of an array
In-Reply-To: <ce557a360611191310q4db52e0fw4308ab31a6f7fca@mail.gmail.com>
References: <ce557a360611191242i35a06f5et2888c0f2c48c177d@mail.gmail.com>
	<c3e57df40611191307w204861e2q764963cd08572386@mail.gmail.com>
	<ce557a360611191310q4db52e0fw4308ab31a6f7fca@mail.gmail.com>
Message-ID: <ce557a360611201120t19cbf581sf31ae5c6d51248dd@mail.gmail.com>

On 19/11/06, A. M. Archibald <peridot.faceted at gmail.com> wrote:

> Well, no, of course not to implement "fiddle". But the application I
> have in mind I don't think can be done with slicing...

If it helps to motivate what I'm doing, I'm trying to write (have
written) a function that will take an array and segment it along an
axis into overlapping pieces with a given length and number of entries
of overlap. Of course this is easy using reshape and stacking, but
there's no reason it needs to make a copy, or to occupy a large amount
of space. I can generate the strides that should be used for such a
segmented view of an array from the strides of the original array with
little difficulty, and the program works fine for one-dimensional
arrays. But if you try to feed it a high-dimensional array,
ndarray.__new__ often has a fit because it can't convert the array to
a (contiguous) buffer. I even considered lying to the system and
marking the array as contiguous, but even apart from being a bad idea
it doesn't seem to be possible from python.

I've attached the function; its test cases include several that
trigger the problem (at which point it spits out a warning and just
copies the array).

The function was suggested on the scipy list by a Matlab escapee as a
very handy function for signal processing. (Taking FFTs of successive
overlapped chunks, for example.)

A. M. Archibald
-------------- next part --------------
A non-text attachment was scrubbed...
Name: segmentaxis.py
Type: text/x-python
Size: 4806 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20061120/d057cb0b/attachment.py>

From charlesr.harris at gmail.com  Mon Nov 20 14:56:13 2006
From: charlesr.harris at gmail.com (Charles R Harris)
Date: Mon, 20 Nov 2006 12:56:13 -0700
Subject: [Numpy-discussion] fiddling the strides of an array
In-Reply-To: <ce557a360611201120t19cbf581sf31ae5c6d51248dd@mail.gmail.com>
References: <ce557a360611191242i35a06f5et2888c0f2c48c177d@mail.gmail.com>
	<c3e57df40611191307w204861e2q764963cd08572386@mail.gmail.com>
	<ce557a360611191310q4db52e0fw4308ab31a6f7fca@mail.gmail.com>
	<ce557a360611201120t19cbf581sf31ae5c6d51248dd@mail.gmail.com>
Message-ID: <e06186140611201156j6a4005d9hc1cd6189b3d3f1df@mail.gmail.com>

On 11/20/06, A. M. Archibald <peridot.faceted at gmail.com> wrote:
>
> On 19/11/06, A. M. Archibald <peridot.faceted at gmail.com> wrote:
>
> > Well, no, of course not to implement "fiddle". But the application I
> > have in mind I don't think can be done with slicing...
>
> If it helps to motivate what I'm doing, I'm trying to write (have
> written) a function that will take an array and segment it along an
> axis into overlapping pieces with a given length and number of entries
> of overlap. Of course this is easy using reshape and stacking, but
> there's no reason it needs to make a copy, or to occupy a large amount
> of space. I can generate the strides that should be used for such a
> segmented view of an array from the strides of the original array with
> little difficulty, and the program works fine for one-dimensional
> arrays.


Cute!

But if you try to feed it a high-dimensional array,
> ndarray.__new__ often has a fit because it can't convert the array to
> a (contiguous) buffer. I even considered lying to the system and
> marking the array as contiguous, but even apart from being a bad idea
> it doesn't seem to be possible from python.
>
> I've attached the function; its test cases include several that
> trigger the problem (at which point it spits out a warning and just
> copies the array).
>
> The function was suggested on the scipy list by a Matlab escapee as a
> very handy function for signal processing. (Taking FFTs of successive
> overlapped chunks, for example.)


Although I suspect a special purpose FFT would be the thing here. For
piecewise convolution it may also be desireable to zero extend the pieces,
etc.

Chuck
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20061120/2e674213/attachment.html>

From tim.hochberg at ieee.org  Mon Nov 20 15:08:45 2006
From: tim.hochberg at ieee.org (Tim Hochberg)
Date: Mon, 20 Nov 2006 13:08:45 -0700
Subject: [Numpy-discussion] fiddling the strides of an array
In-Reply-To: <ce557a360611201120t19cbf581sf31ae5c6d51248dd@mail.gmail.com>
References: <ce557a360611191242i35a06f5et2888c0f2c48c177d@mail.gmail.com>	<c3e57df40611191307w204861e2q764963cd08572386@mail.gmail.com>	<ce557a360611191310q4db52e0fw4308ab31a6f7fca@mail.gmail.com>
	<ce557a360611201120t19cbf581sf31ae5c6d51248dd@mail.gmail.com>
Message-ID: <45620B4D.8090006@ieee.org>

A. M. Archibald wrote:
> On 19/11/06, A. M. Archibald <peridot.faceted at gmail.com> wrote:
>
>> Well, no, of course not to implement "fiddle". But the application I
>> have in mind I don't think can be done with slicing...
>
> If it helps to motivate what I'm doing, I'm trying to write (have
> written) a function that will take an array and segment it along an
> axis into overlapping pieces with a given length and number of entries
> of overlap. Of course this is easy using reshape and stacking, but
> there's no reason it needs to make a copy, or to occupy a large amount
> of space. I can generate the strides that should be used for such a
> segmented view of an array from the strides of the original array with
> little difficulty, and the program works fine for one-dimensional
> arrays. But if you try to feed it a high-dimensional array,
> ndarray.__new__ often has a fit because it can't convert the array to
> a (contiguous) buffer. I even considered lying to the system and
> marking the array as contiguous, but even apart from being a bad idea
> it doesn't seem to be possible from python.
>
> I've attached the function; its test cases include several that
> trigger the problem (at which point it spits out a warning and just
> copies the array).
>
> The function was suggested on the scipy list by a Matlab escapee as a
> very handy function for signal processing. (Taking FFTs of successive
> overlapped chunks, for example.)
>
> A. M. Archibald

I don't know how much help this will be, but, if the input array can be 
mapped into a contiguous array using transpose, which looks to be the 
case with at least one of the problematic cases you show here, you could 
transpose so as to get a contiguous array and grab the buffer from that. 
As for how to decide what transposes to do, while it sounds like a fun 
problem, I don't have time to delve into it right now.

-tim



From peridot.faceted at gmail.com  Mon Nov 20 16:12:15 2006
From: peridot.faceted at gmail.com (A. M. Archibald)
Date: Mon, 20 Nov 2006 16:12:15 -0500
Subject: [Numpy-discussion] fiddling the strides of an array
In-Reply-To: <45620B4D.8090006@ieee.org>
References: <ce557a360611191242i35a06f5et2888c0f2c48c177d@mail.gmail.com>
	<c3e57df40611191307w204861e2q764963cd08572386@mail.gmail.com>
	<ce557a360611191310q4db52e0fw4308ab31a6f7fca@mail.gmail.com>
	<ce557a360611201120t19cbf581sf31ae5c6d51248dd@mail.gmail.com>
	<45620B4D.8090006@ieee.org>
Message-ID: <ce557a360611201312n110a293fwc0d6a062bc4f37c5@mail.gmail.com>

On 20/11/06, Tim Hochberg <tim.hochberg at ieee.org> wrote:

> I don't know how much help this will be, but, if the input array can be
> mapped into a contiguous array using transpose, which looks to be the
> case with at least one of the problematic cases you show here, you could
> transpose so as to get a contiguous array and grab the buffer from that.
> As for how to decide what transposes to do, while it sounds like a fun
> problem, I don't have time to delve into it right now.

I thought about that, actually, all you'd have to do is argsort the
strides tuple. It still fails for arrays that are of the form a[::2],
though, and I'm still hoping to find the secret code to access
something I know numpy is perfectly capable of doing, so I don't
really want to add such a grotty hack.

A. M. Archibald


From filip.wasilewski at gmail.com  Mon Nov 20 17:33:47 2006
From: filip.wasilewski at gmail.com (Filip Wasilewski)
Date: Mon, 20 Nov 2006 23:33:47 +0100
Subject: [Numpy-discussion] fiddling the strides of an array
In-Reply-To: <ce557a360611201312n110a293fwc0d6a062bc4f37c5@mail.gmail.com>
References: <ce557a360611191242i35a06f5et2888c0f2c48c177d@mail.gmail.com>
	<c3e57df40611191307w204861e2q764963cd08572386@mail.gmail.com>
	<ce557a360611191310q4db52e0fw4308ab31a6f7fca@mail.gmail.com>
	<ce557a360611201120t19cbf581sf31ae5c6d51248dd@mail.gmail.com>
	<45620B4D.8090006@ieee.org>
	<ce557a360611201312n110a293fwc0d6a062bc4f37c5@mail.gmail.com>
Message-ID: <c3e57df40611201433p6b4fe087rb187fc0e9f5c5818@mail.gmail.com>

On 11/20/06, A. M. Archibald <peridot.faceted at gmail.com> wrote:
> On 20/11/06, Tim Hochberg <tim.hochberg at ieee.org> wrote:
>
> > I don't know how much help this will be, but, if the input array can be
> > mapped into a contiguous array using transpose, which looks to be the
> > case with at least one of the problematic cases you show here, you could
> > transpose so as to get a contiguous array and grab the buffer from that.
> > As for how to decide what transposes to do, while it sounds like a fun
> > problem, I don't have time to delve into it right now.
>
> I thought about that, actually, all you'd have to do is argsort the
> strides tuple. It still fails for arrays that are of the form a[::2],
> though, and I'm still hoping to find the secret code to access
> something I know numpy is perfectly capable of doing, so I don't
> really want to add such a grotty hack.


As a replacement for ndarray.__new__ you can use the array interface
protocol, at least until someone will come up with a better solution.
I tried your segment_axis (cut mode) with strided and axis swapped
multidimensional arrays and I think it works quite. The fiddle part
goes like this:

class My(object):
    def __init__(self, arr):
        self.__array_interface__ = arr.__array_interface__
        self.__array_interface__["strides"] = newstrides
        self.__array_interface__ai["shape"] = newshape
a = numpy.asarray(My(arr))


cheers,
fw


From michael.sorich at gmail.com  Mon Nov 20 18:37:06 2006
From: michael.sorich at gmail.com (Michael Sorich)
Date: Tue, 21 Nov 2006 10:07:06 +1030
Subject: [Numpy-discussion] A reimplementation of MaskedArray
In-Reply-To: <200610160232.20888.pgmdevlist@mailcan.com>
References: <200610160232.20888.pgmdevlist@mailcan.com>
Message-ID: <16761e100611201537h68f80bb0h349433402d422259@mail.gmail.com>

While playing with the new implementation I noticed that the nonzero
method did not seem to work properly. I was not able to trace down why
this is the case - I could not find where the nonzero method of the
new maskedarray is actually implemented :)

import maskedarray as ma
test_ma1 = ma.array([True, False, False], mask=[0,0,1])
print test_ma1
print test_ma1.nonzero()[0]
print ma.nonzero(test_ma1)[0]

--output--
[True False --]
[True False --]
[0]


On 10/16/06, Pierre GM <pgmdevlist at mailcan.com> wrote:
> Folks,
> I just posted on the scipy/developers zone wiki
> (http://projects.scipy.org/scipy/numpy/wiki/MaskedArray) a reimplementation
> of the masked_array mopdule, motivated by some problems I ran into while
> subclassing MaskedArray.
>
> The main differences with the initial numpy.core.ma package are that
> MaskedArray is now a subclass of ndarray and that the _data section can now
> be any subclass of ndarray (well, it should work in most cases, some tweaking
> might required here and there). Apart from a couple of issues listed below,
> the behavior of the new MaskedArray class reproduces the old one. It is quite
> likely to be significantly slower, though: I was more interested in a clear
> organization than in performance, so I tended to use wrappers liberally. I'm
> sure we can improve that rather easily.
>
> The new module, along with a test suite and some utilities, are available
> here:
> http://projects.scipy.org/scipy/numpy/attachment/wiki/MaskedArray/maskedarray.py
> http://projects.scipy.org/scipy/numpy/attachment/wiki/MaskedArray/masked_testutils.py
> http://projects.scipy.org/scipy/numpy/attachment/wiki/MaskedArray/test_maskedarray.py
>
> Please note that it's still a work in progress (even if it seems to work quite
> OK when I use it). Suggestions, comments, improvements and general feedback
> are more than welcome !
>
> -------------------------------------------------------------------------
> Using Tomcat but need to do more? Need to support web services, security?
> Get stuff done quickly with pre-integrated technology to make your job easier
> Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
> http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
> _______________________________________________
> Numpy-discussion mailing list
> Numpy-discussion at lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/numpy-discussion
>


From tim.hochberg at ieee.org  Mon Nov 20 18:45:54 2006
From: tim.hochberg at ieee.org (Tim Hochberg)
Date: Mon, 20 Nov 2006 16:45:54 -0700
Subject: [Numpy-discussion] fiddling the strides of an array
In-Reply-To: <c3e57df40611201433p6b4fe087rb187fc0e9f5c5818@mail.gmail.com>
References: <ce557a360611191242i35a06f5et2888c0f2c48c177d@mail.gmail.com>	<c3e57df40611191307w204861e2q764963cd08572386@mail.gmail.com>	<ce557a360611191310q4db52e0fw4308ab31a6f7fca@mail.gmail.com>	<ce557a360611201120t19cbf581sf31ae5c6d51248dd@mail.gmail.com>	<45620B4D.8090006@ieee.org>	<ce557a360611201312n110a293fwc0d6a062bc4f37c5@mail.gmail.com>
	<c3e57df40611201433p6b4fe087rb187fc0e9f5c5818@mail.gmail.com>
Message-ID: <45623E32.8000801@ieee.org>

Filip Wasilewski wrote:
> On 11/20/06, A. M. Archibald <peridot.faceted at gmail.com> wrote:
>   
>> On 20/11/06, Tim Hochberg <tim.hochberg at ieee.org> wrote:
>>
>>     
>>> I don't know how much help this will be, but, if the input array can be
>>> mapped into a contiguous array using transpose, which looks to be the
>>> case with at least one of the problematic cases you show here, you could
>>> transpose so as to get a contiguous array and grab the buffer from that.
>>> As for how to decide what transposes to do, while it sounds like a fun
>>> problem, I don't have time to delve into it right now.
>>>       
>> I thought about that, actually, all you'd have to do is argsort the
>> strides tuple. It still fails for arrays that are of the form a[::2],
>> though, and I'm still hoping to find the secret code to access
>> something I know numpy is perfectly capable of doing, so I don't
>> really want to add such a grotty hack.
>>     
>
>
> As a replacement for ndarray.__new__ you can use the array interface
> protocol, at least until someone will come up with a better solution.
> I tried your segment_axis (cut mode) with strided and axis swapped
> multidimensional arrays and I think it works quite. The fiddle part
> goes like this:
>
> class My(object):
>     def __init__(self, arr):
>         self.__array_interface__ = arr.__array_interface__
>         self.__array_interface__["strides"] = newstrides
>         self.__array_interface__ai["shape"] = newshape
> a = numpy.asarray(My(arr))
>
>   
Clever!

Two cautionary notes: first, you can crash the interpreter if you use 
bogus strides or shape, so be careful. Second, and perhaps a little less 
obvious, My(arr) needs to hold a reference to the array 'arr'. 
Otherwise, if arr goes away before 'a' does, 'a' will be left pointing 
to invalid data.

-tim




From pajer at iname.com  Mon Nov 20 20:06:51 2006
From: pajer at iname.com (Gary)
Date: Mon, 20 Nov 2006 20:06:51 -0500
Subject: [Numpy-discussion] FAQ: compatibility - scipy, matplotlib,
	enthought tool suite
In-Reply-To: <45623E32.8000801@ieee.org>
References: <ce557a360611191242i35a06f5et2888c0f2c48c177d@mail.gmail.com>	<c3e57df40611191307w204861e2q764963cd08572386@mail.gmail.com>	<ce557a360611191310q4db52e0fw4308ab31a6f7fca@mail.gmail.com>	<ce557a360611201120t19cbf581sf31ae5c6d51248dd@mail.gmail.com>	<45620B4D.8090006@ieee.org>	<ce557a360611201312n110a293fwc0d6a062bc4f37c5@mail.gmail.com>	<c3e57df40611201433p6b4fe087rb187fc0e9f5c5818@mail.gmail.com>
	<45623E32.8000801@ieee.org>
Message-ID: <4562512B.9050601@iname.com>

I know this has been (mostly) asked and answered, but I'm still not 
certain.  I've been living with an "old" but compatible set of numpy and 
scipy (numpy 1.0 beta) and an older matplotlib.  I decided to try 
Enthought Tool Suite but it fails with a "numpy.core.multiarray failed 
to import".   I could ask on a Enthought list, but I'd like to nail down 
the numpy issue first.

Please forgive me for being dense, and for posting in what might  be the 
wrong place.

True or False:
The latest scipy Win32 binary (0.5.1) *requires* numpy 1.0 rc2
The latest matplotlib (0.87.7) *requires*  numpy 1.0 final
(hence a working set cannot include the latest versions of mpl and numpy)
ETS 1.1.0 (Windows binary)  requires ... what?   (It doesn't work with 
my numpy beta, but the problem may or may not be numpy)

tia,
gap



From pgmdevlist at gmail.com  Mon Nov 20 23:53:56 2006
From: pgmdevlist at gmail.com (Pierre GM)
Date: Mon, 20 Nov 2006 23:53:56 -0500
Subject: [Numpy-discussion] A reimplementation of MaskedArray
In-Reply-To: <16761e100611201537h68f80bb0h349433402d422259@mail.gmail.com>
References: <200610160232.20888.pgmdevlist@mailcan.com>
	<16761e100611201537h68f80bb0h349433402d422259@mail.gmail.com>
Message-ID: <200611202353.57163.pgmdevlist@gmail.com>

On Monday 20 November 2006 18:37, Michael Sorich wrote:
> While playing with the new implementation I noticed that the nonzero
> method did not seem to work properly. I was not able to trace down why
> this is the case - I could not find where the nonzero method of the
> new maskedarray is actually implemented :)

Good call, I had missed this one. That should be corrected on the wiki.
Thanks again !


From cjw at sympatico.ca  Tue Nov 21 08:17:56 2006
From: cjw at sympatico.ca (Colin J. Williams)
Date: Tue, 21 Nov 2006 08:17:56 -0500
Subject: [Numpy-discussion] Numpy Documentation
Message-ID: <4562FC84.9070009@sympatico.ca>

An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20061121/adba755b/attachment.html>

From cookedm at physics.mcmaster.ca  Tue Nov 21 12:13:33 2006
From: cookedm at physics.mcmaster.ca (David M. Cooke)
Date: Tue, 21 Nov 2006 12:13:33 -0500
Subject: [Numpy-discussion] Numpy Documentation
In-Reply-To: <4562FC84.9070009@sympatico.ca>
References: <4562FC84.9070009@sympatico.ca>
Message-ID: <20061121121333.00bc15f4@arbutus.physics.mcmaster.ca>

On Tue, 21 Nov 2006 08:17:56 -0500
"Colin J. Williams" <cjw at sympatico.ca> wrote:

> The new Example List with Documentation is a great help.
> 
> It could also be helpful to have:
> information about the dump/dumps format.

It's a Python pickle, so treat it as an opaque binary string. Doc update in
r3457.

> some clarification regarding the (nested) sequence acceptable by array().
> It would appear to require a homogeneous sequence, perhaps there are other
> restrictions.. Colin W.


-- 
|>|\/|<
/--------------------------------------------------------------------------\
|David M. Cooke                      http://arbutus.physics.mcmaster.ca/dmc/
|cookedm at physics.mcmaster.ca


From amit_605 at yahoo.co.in  Tue Nov 21 18:44:28 2006
From: amit_605 at yahoo.co.in (amit soni)
Date: Wed, 22 Nov 2006 05:14:28 +0530 (IST)
Subject: [Numpy-discussion] Quadratic Optimization Problem
Message-ID: <20061121234428.46311.qmail@web8402.mail.in.yahoo.com>

Hi,

I need to do a quadratic optimization problem in python
where the constraints are quadratic and objective function is linear.
Its a convex optimization problem.

What are the possible choices to do this.

Thanks
Amit 



 
____________________________________________________________________________________
Sponsored Link

Rates near 39yr lows. $420,000 Loan for $1399/mo. 
Calcuate new payment. www.LowerMyBills.com/lre
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20061122/cfd1916d/attachment.html>

From haase at msg.ucsf.edu  Tue Nov 21 20:16:31 2006
From: haase at msg.ucsf.edu (Sebastian Haase)
Date: Tue, 21 Nov 2006 18:16:31 -0700
Subject: [Numpy-discussion] Quadratic Optimization Problem
In-Reply-To: <20061121234428.46311.qmail@web8402.mail.in.yahoo.com>
References: <20061121234428.46311.qmail@web8402.mail.in.yahoo.com>
Message-ID: <200611211716.31428.haase@msg.ucsf.edu>

Don't know the complete answer - but try cobyla in scipy (scipy.optimize).

-Sebastian

On Tuesday 21 November 2006 15:44, amit soni wrote:
> Hi,
>
> I need to do a quadratic optimization problem in python
> where the constraints are quadratic and objective function is linear.
> Its a convex optimization problem.
>
> What are the possible choices to do this.
>
> Thanks
> Amit
>
>
>
>
> ___________________________________________________________________________
>_________ Sponsored Link
>
> Rates near 39yr lows. $420,000 Loan for $1399/mo.
> Calcuate new payment. www.LowerMyBills.com/lre


From michael.sorich at gmail.com  Tue Nov 21 21:11:15 2006
From: michael.sorich at gmail.com (Michael Sorich)
Date: Wed, 22 Nov 2006 12:41:15 +1030
Subject: [Numpy-discussion] A reimplementation of MaskedArray
In-Reply-To: <200611202353.57163.pgmdevlist@gmail.com>
References: <200610160232.20888.pgmdevlist@mailcan.com>
	<16761e100611201537h68f80bb0h349433402d422259@mail.gmail.com>
	<200611202353.57163.pgmdevlist@gmail.com>
Message-ID: <16761e100611211811u3a2f95f5obe09d2da516ba101@mail.gmail.com>

I think that the new implementation is making a copy of the data with
indexing a MA. This is different from both ndarray and the existing
numpy ma version.

e.g.
testma = ma.array([[1,2,3,4,5],[1,2,3,4,5],[1,2,3,4,5]], mask=ma.nomask)
testma2 =  testma[1]
testma2[1] = 20
print testma
print testma2

--output--
[[1 2 3 4 5]
 [1 2 3 4 5]
 [1 2 3 4 5]]
[ 1 20  3  4  5]

Having subviews of the mask seems complicated with the mask being
nomask. What happens if the view sets a new masked value and hence
changes from nomask to an boolean array? How does the parent mask get
updated? I think the numpy implementation gets away with this by
returning a view of only the _data part if the ma mask is nomask - I
don't like this solution as I would expect a ma to be returned. Also I
suspect that if the ma is to be a view of another ma, then in __new__
a mask that is a boolean array of all False cannot be converted to
nomask.

I like the new implementation of maskedarray, especially the focus on
simplicity. The only simple solution I see is to have the mask be a
boolean array at all times. Or at least as soon as a view is required
of a ma if mask is nomask then it should be converted to a boolean
array and then both a view of the boolean array mask as well as the
data is used in the view ma returned.


From pgmdevlist at gmail.com  Tue Nov 21 23:09:17 2006
From: pgmdevlist at gmail.com (Pierre GM)
Date: Tue, 21 Nov 2006 23:09:17 -0500
Subject: [Numpy-discussion] A reimplementation of MaskedArray
In-Reply-To: <16761e100611211811u3a2f95f5obe09d2da516ba101@mail.gmail.com>
References: <200610160232.20888.pgmdevlist@mailcan.com>
	<200611202353.57163.pgmdevlist@gmail.com>
	<16761e100611211811u3a2f95f5obe09d2da516ba101@mail.gmail.com>
Message-ID: <200611212309.17463.pgmdevlist@gmail.com>

On Tuesday 21 November 2006 21:11, Michael Sorich wrote:
> I think that the new implementation is making a copy of the data with
> indexing a MA. This is different from both ndarray and the existing
> numpy ma version.

Michael, 
If you check the definition of MaskedArray.__new__, you'll see that the "copy" argument is set to True by default. Setting it to "false" seems to give what you expect. Should I make the default ?

> Having subviews of the mask seems complicated with the mask being
> nomask. 

Why ? nomask is just a trick to avoid unnecessary computations on a mask full of False that doesn't need updating.

> What happens if the view sets a new masked value and hence 
> changes from nomask to an boolean array ?
> How does the parent mask get  updated? 

Both implementations work the same way: the parent mask is not updated.

> I think the numpy implementation gets away with this by 
> returning a view of only the _data part if the ma mask is nomask 

By numpy implementation, you mean numpy.core.ma, right ? 
If so, then yes:
`self.__getitem__[i]` returns `self._data[i]` if the mask is nomask.

In maskedarray, if the mask is nomask, then
`self.__getitem__[i]` returns `self._data[i]` only if `self._data[i].size==1`, else it returns a masked array.

> I don't like this solution as I would expect a ma to be returned. Also I
> suspect that if the ma is to be a view of another ma, then in __new__
> a mask that is a boolean array of all False cannot be converted to
> nomask.

I'm not following you here: there's no `__new__` in numpy.core.ma (that's one of the reason why a masked array in numpy.core.ma is basically different from a ndarray...). And in maskedarray, a mask as array of `False` is set to `nomask` by default, but you can use the `flag` option: please check the documentation of `maskedarray.masked_array`: flag=True converts the mask, flags=False keeps an array of boolean.


One thing to remember is that masks tend to be copied more often than not. And I don't think it's advisable to modify the mask of the parent: it's no longer the same object, as the mask is now different ! In other terms, you could share data, you shouldn't share a mask. And I keep getting bitten with data sharing, that's why I had set the 'copy' flag to True by default.


> I like the new implementation of maskedarray, especially the focus on
> simplicity. The only simple solution I see is to have the mask be a
> boolean array at all times....

You haven't convinced me yet of why a mask of False is better than `nomask`. 
What don't you like in maskedarray (aka the new implementation) ?
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20061121/dca85a83/attachment.html>

From michael.sorich at gmail.com  Wed Nov 22 00:25:43 2006
From: michael.sorich at gmail.com (Michael Sorich)
Date: Wed, 22 Nov 2006 15:55:43 +1030
Subject: [Numpy-discussion] A reimplementation of MaskedArray
In-Reply-To: <200611212309.17463.pgmdevlist@gmail.com>
References: <200610160232.20888.pgmdevlist@mailcan.com>
	<200611202353.57163.pgmdevlist@gmail.com>
	<16761e100611211811u3a2f95f5obe09d2da516ba101@mail.gmail.com>
	<200611212309.17463.pgmdevlist@gmail.com>
Message-ID: <16761e100611212125w42fe9d77j4425c8f7f3136efb@mail.gmail.com>

Perhaps an example will help explain what I mean

For the case of an ndarray if you select a row and then alter the new
array, the old array
is also changed.

from numpy import *
a = array([[1,2,3,4,5],[1,2,3,4,5],[1,2,3,4,5]])
suba = a[2]
suba[1] = 10
print a
print suba
--output--
[[ 1  2  3  4  5]
 [ 1  2  3  4  5]
 [ 1 10  3  4  5]]
[ 1 10  3  4  5]

In the current version of maskedarray in numpy, changes in the row
array affect the parent array. Here whenever you select a single row
or column and mask is nomask a ndarray is returned not a masked array

from numpy.core.ma import *
a = array([[1,2,3,4,5],[1,2,3,4,5],[1,2,3,4,5]], mask=nomask)
suba = a[2]
suba[1] = 10
print a
print suba
print type(suba)
--output--
[[1 2 3 4 5]
 [1 2 3 4 5]
 [1 10 3 4 5]]
[ 1 10  3  4  5]
<type 'numpy.ndarray'>

However if mask is anything other than nomask (even if the mask is an
boolean array in which all the values are false- which means the same
thing as nomask) a masked array is returned. Once again the data is
shared between the arrays

from numpy.core.ma import *
a = array([[1,2,3,4,5],[1,2,3,4,5],[1,2,3,4,5]],
mask=[[0,0,0,0,0],[0,0,0,0,0],[0,0,0,0,0]])
suba = a[2]
suba[1] = 10
print a
print suba
print type(suba)
--output--
[[1 2 3 4 5]
 [1 2 3 4 5]
 [1 10 3 4 5]]
[1 10 3 4 5]
<class 'numpy.core.ma.MaskedArray'>

Unfortunately if the value is changed to masked, this is not updated
in the parent array. This seems very inconsistent. I don't view masked
values any different than any other value.

a = array([[1,2,3,4,5],[1,2,3,4,5],[1,2,3,4,5]],
mask=[[0,0,0,0,0],[0,0,0,0,0],[0,0,0,0,0]])
suba = a[2]
suba[1] = masked
print a
print suba
print type(suba)
--output--
[[1 2 3 4 5]
 [1 2 3 4 5]
 [1 2 3 4 5]]
[1 -- 3 4 5]
<class 'numpy.core.ma.MaskedArray'>

With the new implementation, the data is not shared for any of the 3
variations above. I am happy that the arrays acts consistently,
however this is different from the ndarray. It would be nice if the
behavior was the same as the ndarray, but it is better than the
numpy.core.ma implementation. If this is on purpose, then it should be
documented.

from numpyext.maskedarray import *
a = array([[1,2,3,4,5],[1,2,3,4,5],[1,2,3,4,5]], mask=nomask)
suba = a[2]
suba[1] = 10
print a
print suba
print type(suba)
--output--
[[1 2 3 4 5]
 [1 2 3 4 5]
 [1 2 3 4 5]]
[ 1 10  3  4  5]
<class 'numpyext.maskedarray.MaskedArray'>


On 11/22/06, Pierre GM <pgmdevlist at gmail.com> wrote:
>
>
> On Tuesday 21 November 2006 21:11, Michael Sorich wrote:
>
> > I think that the new implementation is making a copy of the data with
>
> > indexing a MA. This is different from both ndarray and the existing
>
> > numpy ma version.
>
>
>
> Michael,
>
> If you check the definition of MaskedArray.__new__, you'll see that the
> "copy" argument is set to True by default. Setting it to "false" seems to
> give what you expect. Should I make the default ?
>
>
>
> > Having subviews of the mask seems complicated with the mask being
>
> > nomask.
>
>
>
> Why ? nomask is just a trick to avoid unnecessary computations on a mask
> full of False that doesn't need updating.
>
>
>
> > What happens if the view sets a new masked value and hence
>
> > changes from nomask to an boolean array ?
>
> > How does the parent mask get updated?
>
>
>
> Both implementations work the same way: the parent mask is not updated.
>
>
>
> > I think the numpy implementation gets away with this by
>
> > returning a view of only the _data part if the ma mask is nomask
>
>
>
> By numpy implementation, you mean numpy.core.ma, right ?
>
> If so, then yes:
>
> `self.__getitem__[i]` returns `self._data[i]` if the mask is nomask.
>
>
>
> In maskedarray, if the mask is nomask, then
>
> `self.__getitem__[i]` returns `self._data[i]` only if
> `self._data[i].size==1`, else it returns a masked array.
>
>
>
> > I don't like this solution as I would expect a ma to be returned. Also I
>
> > suspect that if the ma is to be a view of another ma, then in __new__
>
> > a mask that is a boolean array of all False cannot be converted to
>
> > nomask.
>
>
>
> I'm not following you here: there's no `__new__` in numpy.core.ma (that's
> one of the reason why a masked array in numpy.core.ma is basically different
> from a ndarray...). And in maskedarray, a mask as array of `False` is set to
> `nomask` by default, but you can use the `flag` option: please check the
> documentation of `maskedarray.masked_array`: flag=True converts the mask,
> flags=False keeps an array of boolean.
>
>
>
>
>
> One thing to remember is that masks tend to be copied more often than not.
> And I don't think it's advisable to modify the mask of the parent: it's no
> longer the same object, as the mask is now different ! In other terms, you
> could share data, you shouldn't share a mask. And I keep getting bitten with
> data sharing, that's why I had set the 'copy' flag to True by default.
>
>
>
>
>
> > I like the new implementation of maskedarray, especially the focus on
>
> > simplicity. The only simple solution I see is to have the mask be a
>
> > boolean array at all times....
>
>
>
> You haven't convinced me yet of why a mask of False is better than `nomask`.
>
> What don't you like in maskedarray (aka the new implementation) ?
>
>


From michael.sorich at gmail.com  Wed Nov 22 01:12:11 2006
From: michael.sorich at gmail.com (Michael Sorich)
Date: Wed, 22 Nov 2006 16:42:11 +1030
Subject: [Numpy-discussion] A reimplementation of MaskedArray
In-Reply-To: <16761e100611212125w42fe9d77j4425c8f7f3136efb@mail.gmail.com>
References: <200610160232.20888.pgmdevlist@mailcan.com>
	<200611202353.57163.pgmdevlist@gmail.com>
	<16761e100611211811u3a2f95f5obe09d2da516ba101@mail.gmail.com>
	<200611212309.17463.pgmdevlist@gmail.com>
	<16761e100611212125w42fe9d77j4425c8f7f3136efb@mail.gmail.com>
Message-ID: <16761e100611212212g43d91404h79715963070077c1@mail.gmail.com>

If I make some minor changes (below) to MaskedArray get and setitem

from numpyext.maskedarray import *

a = array([[1,2,3,4,5],[6,7,8,9,10]], mask=nomask)
suba = a[0]
suba[1] = masked
print a
>[[1 -- 3 4 5]
> [6 7 8 9 10]]
print suba
>[1 -- 3 4 5]

suba = a[1]
suba[1] = 10
print a
>[[1 -- 3 4 5]
> [6 10 8 9 10]]
print suba
>[6 10 8 9 10]

a = array([[1,2,3,4,5],[6,7,8,9,10]], mask=[[0,0,0,0,0],[0,1,0,0,0]])
suba = a[0]
suba[1] = masked
print a
>[[1 -- 3 4 5]
> [6 -- 8 9 10]]
print suba
>[1 -- 3 4 5]

suba = a[1]
suba[1] = 10
print a
>[[1 -- 3 4 5]
> [6 10 8 9 10]]
print suba
>[6 10 8 9 10]

### in MaskedArray ###
    def __getitem__(self, i):
        """x.__getitem__(y) <==> x[y]
Returns the item described by i. Not a copy as in previous versions.
        """
        dout = self._data[i]
        if self._mask is nomask:
            if numeric.size(dout)==1:
                return dout
            else:
                ### made changes here
                self._mask = numeric.zeros(self.shape, dtype=MaskType)
                mout = self._mask[i]
                return self.__class__(dout, mask=mout,
fill_value=self._fill_value, copy=False, flag=False)
                ### -------------
        #....
#        m = self._mask.copy()
        m = self._mask
        mi = m[i]
        if mi.size == 1:
            if mi:
                return masked
            else:
                return dout
        else:
            ### made changes here
            return self.__class__(dout, mask=mi,
fill_value=self._fill_value, copy=False, flag=False)

    def __setitem__(self, index, value):
        """x.__setitem__(i, y) <==> x[i]=y
Sets item described by index. If value is masked, masks those locations.
        """
        d = self._data
        if self is masked:
            raise MAError, 'Cannot alter the masked element.'
        #....
        if value is masked:
            if self._mask is nomask:
                _mask = make_mask_none(d.shape)
            else:
                _mask = self._mask
#===============================================================================
#                #why does the mask need to be copied?
#                _mask = self._mask.copy()
#===============================================================================
            _mask[index] = True
            self._mask = _mask
            return
        #....
        m = getmask(value)
        value = filled(value).astype(d.dtype)
        d[index] = value
        if m is nomask:
            if self._mask is not nomask:
#===============================================================================
#                #why does the mask need to be copied?
#                _mask = self._mask.copy()
#===============================================================================
                _mask = self._mask
                _mask[index] = False
            else:
                _mask = nomask
        else:
            if self._mask is nomask:
                _mask = make_mask_none(d.shape)
            else:
                _mask = self._mask
#===============================================================================
#                #why does the mask need to be copied?
#                _mask = self._mask.copy()
#===============================================================================
            _mask[index] = m
        self._mask = _mask


On 11/22/06, Michael Sorich <michael.sorich at gmail.com> wrote:
> Perhaps an example will help explain what I mean
>
> For the case of an ndarray if you select a row and then alter the new
> array, the old array
> is also changed.
>
> from numpy import *
> a = array([[1,2,3,4,5],[1,2,3,4,5],[1,2,3,4,5]])
> suba = a[2]
> suba[1] = 10
> print a
> print suba
> --output--
> [[ 1  2  3  4  5]
>  [ 1  2  3  4  5]
>  [ 1 10  3  4  5]]
> [ 1 10  3  4  5]
>
> In the current version of maskedarray in numpy, changes in the row
> array affect the parent array. Here whenever you select a single row
> or column and mask is nomask a ndarray is returned not a masked array
>
> from numpy.core.ma import *
> a = array([[1,2,3,4,5],[1,2,3,4,5],[1,2,3,4,5]], mask=nomask)
> suba = a[2]
> suba[1] = 10
> print a
> print suba
> print type(suba)
> --output--
> [[1 2 3 4 5]
>  [1 2 3 4 5]
>  [1 10 3 4 5]]
> [ 1 10  3  4  5]
> <type 'numpy.ndarray'>
>
> However if mask is anything other than nomask (even if the mask is an
> boolean array in which all the values are false- which means the same
> thing as nomask) a masked array is returned. Once again the data is
> shared between the arrays
>
> from numpy.core.ma import *
> a = array([[1,2,3,4,5],[1,2,3,4,5],[1,2,3,4,5]],
> mask=[[0,0,0,0,0],[0,0,0,0,0],[0,0,0,0,0]])
> suba = a[2]
> suba[1] = 10
> print a
> print suba
> print type(suba)
> --output--
> [[1 2 3 4 5]
>  [1 2 3 4 5]
>  [1 10 3 4 5]]
> [1 10 3 4 5]
> <class 'numpy.core.ma.MaskedArray'>
>
> Unfortunately if the value is changed to masked, this is not updated
> in the parent array. This seems very inconsistent. I don't view masked
> values any different than any other value.
>
> a = array([[1,2,3,4,5],[1,2,3,4,5],[1,2,3,4,5]],
> mask=[[0,0,0,0,0],[0,0,0,0,0],[0,0,0,0,0]])
> suba = a[2]
> suba[1] = masked
> print a
> print suba
> print type(suba)
> --output--
> [[1 2 3 4 5]
>  [1 2 3 4 5]
>  [1 2 3 4 5]]
> [1 -- 3 4 5]
> <class 'numpy.core.ma.MaskedArray'>
>
> With the new implementation, the data is not shared for any of the 3
> variations above. I am happy that the arrays acts consistently,
> however this is different from the ndarray. It would be nice if the
> behavior was the same as the ndarray, but it is better than the
> numpy.core.ma implementation. If this is on purpose, then it should be
> documented.
>
> from numpyext.maskedarray import *
> a = array([[1,2,3,4,5],[1,2,3,4,5],[1,2,3,4,5]], mask=nomask)
> suba = a[2]
> suba[1] = 10
> print a
> print suba
> print type(suba)
> --output--
> [[1 2 3 4 5]
>  [1 2 3 4 5]
>  [1 2 3 4 5]]
> [ 1 10  3  4  5]
> <class 'numpyext.maskedarray.MaskedArray'>
>
>
> On 11/22/06, Pierre GM <pgmdevlist at gmail.com> wrote:
> >
> >
> > On Tuesday 21 November 2006 21:11, Michael Sorich wrote:
> >
> > > I think that the new implementation is making a copy of the data with
> >
> > > indexing a MA. This is different from both ndarray and the existing
> >
> > > numpy ma version.
> >
> >
> >
> > Michael,
> >
> > If you check the definition of MaskedArray.__new__, you'll see that the
> > "copy" argument is set to True by default. Setting it to "false" seems to
> > give what you expect. Should I make the default ?
> >
> >
> >
> > > Having subviews of the mask seems complicated with the mask being
> >
> > > nomask.
> >
> >
> >
> > Why ? nomask is just a trick to avoid unnecessary computations on a mask
> > full of False that doesn't need updating.
> >
> >
> >
> > > What happens if the view sets a new masked value and hence
> >
> > > changes from nomask to an boolean array ?
> >
> > > How does the parent mask get updated?
> >
> >
> >
> > Both implementations work the same way: the parent mask is not updated.
> >
> >
> >
> > > I think the numpy implementation gets away with this by
> >
> > > returning a view of only the _data part if the ma mask is nomask
> >
> >
> >
> > By numpy implementation, you mean numpy.core.ma, right ?
> >
> > If so, then yes:
> >
> > `self.__getitem__[i]` returns `self._data[i]` if the mask is nomask.
> >
> >
> >
> > In maskedarray, if the mask is nomask, then
> >
> > `self.__getitem__[i]` returns `self._data[i]` only if
> > `self._data[i].size==1`, else it returns a masked array.
> >
> >
> >
> > > I don't like this solution as I would expect a ma to be returned. Also I
> >
> > > suspect that if the ma is to be a view of another ma, then in __new__
> >
> > > a mask that is a boolean array of all False cannot be converted to
> >
> > > nomask.
> >
> >
> >
> > I'm not following you here: there's no `__new__` in numpy.core.ma (that's
> > one of the reason why a masked array in numpy.core.ma is basically different
> > from a ndarray...). And in maskedarray, a mask as array of `False` is set to
> > `nomask` by default, but you can use the `flag` option: please check the
> > documentation of `maskedarray.masked_array`: flag=True converts the mask,
> > flags=False keeps an array of boolean.
> >
> >
> >
> >
> >
> > One thing to remember is that masks tend to be copied more often than not.
> > And I don't think it's advisable to modify the mask of the parent: it's no
> > longer the same object, as the mask is now different ! In other terms, you
> > could share data, you shouldn't share a mask. And I keep getting bitten with
> > data sharing, that's why I had set the 'copy' flag to True by default.
> >
> >
> >
> >
> >
> > > I like the new implementation of maskedarray, especially the focus on
> >
> > > simplicity. The only simple solution I see is to have the mask be a
> >
> > > boolean array at all times....
> >
> >
> >
> > You haven't convinced me yet of why a mask of False is better than `nomask`.
> >
> > What don't you like in maskedarray (aka the new implementation) ?
> >
> >
>


From pgmdevlist at gmail.com  Wed Nov 22 02:28:43 2006
From: pgmdevlist at gmail.com (Pierre GM)
Date: Wed, 22 Nov 2006 02:28:43 -0500
Subject: [Numpy-discussion] A reimplementation of MaskedArray
In-Reply-To: <16761e100611212125w42fe9d77j4425c8f7f3136efb@mail.gmail.com>
References: <200610160232.20888.pgmdevlist@mailcan.com>
	<200611212309.17463.pgmdevlist@gmail.com>
	<16761e100611212125w42fe9d77j4425c8f7f3136efb@mail.gmail.com>
Message-ID: <200611220228.44329.pgmdevlist@gmail.com>

> With the new implementation, the data is not shared for any of the 3
> variations above. 

The 'copy=True' in MaskedArray.__new__ ensures that data is always copied by 
default (that's the way I liked it). Most methods that return new masked 
arrays (__getitem__ and __setitem__, for example) do not specifically use a 
particular value of this flag, they just rely on the default, "copy=True" , 
which is "don't share".
Once again, it's not so much because I'm selfish than because I regularly 
forget that data can be shared, and that modifying one array can have 
repercussions on another one the other side of the code.

But there's a quick fix: in the definition of MaskedArray.__new__, just change 
the default copy flag from True to False, that'll work.
Now, Michael, you're right, and it should be the default. I'll fix that.

[Thinking about it, we could store the copy flag in the array, and use that 
value in __getitem__ and __setitem__. That might be overkill, though].

> Unfortunately if the value is changed to masked, this is not updated
> in the parent array. This seems very inconsistent. I don't view masked
> values any different than any other value.

Inconsistent, maybe, useful definitely: 
Masking a view and getting the original masked accordingly could be useful, 
but I strongly feel that unmasking a view and getting an unmasked orginal is 
dangerous.
Besides, in order to make the mask updatable, you need to get it in array 
form, as you suggested in your version of __getitem__. But then, you drag 
this array all along, and I'm not sure it's worthwhile.
All in all, I'd far prefer a status-quo, with unsharable masks.

What does the rest of the list think (er, the MA users) ?











From kxroberto at googlemail.com  Wed Nov 22 04:34:56 2006
From: kxroberto at googlemail.com (Robert)
Date: Wed, 22 Nov 2006 10:34:56 +0100
Subject: [Numpy-discussion] scipy Windows binary compatible recent numpy 1.0
	?
Message-ID: <ek15k5$dor$2@sea.gmane.org>

Since some months now the latest scipy (0.5.1) binary (for Windows) is not compatible with current numpy (1.0) - obviously because of a C-library mismatch.

Does somebody have a compatible binary - or when is the next scipy (binary) planned?


Robert



From david at ar.media.kyoto-u.ac.jp  Wed Nov 22 05:45:45 2006
From: david at ar.media.kyoto-u.ac.jp (David Cournapeau)
Date: Wed, 22 Nov 2006 19:45:45 +0900
Subject: [Numpy-discussion] How to deal with NAN and co in C extension to
	numpy/scipy ?
Message-ID: <45642A59.3010405@ar.media.kyoto-u.ac.jp>

Hi,

    I am about to release some code to compute LPC coefficients (Auto 
regressive modeling, using Levinson Durbin algorithm for autocorrelation 
matrix inversion), and I was wondering how to handle cases of divide by 
0, or values really near 0.

    I understand this is a complex issue, and I admittedly have no 
knowledge about it except some generalities; there are versions of 
Levinson Durbin which handles bad condition numbers (for people not 
familiar with LPC coding, levinson durbin is a recursive algorithm to 
inverse a symmetric Toeplitz matrix; the complexity becomes O(N^2) 
instead of O(N^3)); but they are overkill for LPC coding, I think.

    The options I see are:

    - I don't care :)
    - Detecting values near zero, finish the computation anyway in C, 
and returns an error code to python, which would emit a warning.
    - Detecting values near zero, returns an error code which would 
results in a python exception ?

    In this latter cases, is it ok to returns to python arrays the 
values from the C function ? In matlab, for bad cases, it just returns 
NAN; is this appropriate ? How should I do it ?

    cheers,

    David


From kxroberto at googlemail.com  Wed Nov 22 05:48:25 2006
From: kxroberto at googlemail.com (Robert)
Date: Wed, 22 Nov 2006 11:48:25 +0100
Subject: [Numpy-discussion] numpy.random.shuffle
Message-ID: <ek19u0$rr7$1@sea.gmane.org>

is this an error when shuffle doubles&looses items on a 2-d array? :

>>> r=arange(20)
>>> rr=ziparrays(r,r)  #numpy.vstack(*args).transpose()
>>> rr
array([[ 0,  0],
       [ 1,  1],
       [ 2,  2],
       [ 3,  3],
       [ 4,  4],
       [ 5,  5],
       [ 6,  6],
       [ 7,  7],
       [ 8,  8],
       [ 9,  9],
       [10, 10],
       [11, 11],
       [12, 12],
       [13, 13],
       [14, 14],
       [15, 15],
       [16, 16],
       [17, 17],
       [18, 18],
       [19, 19]])
>>> numpy.random.shuffle(rr)
>>> rr
array([[ 0,  0],
       [ 0,  0],
       [ 2,  2],
       [ 2,  2],
       [ 3,  3],
       [ 3,  3],
       [ 5,  5],
       [ 2,  2],
       [ 0,  0],
       [ 8,  8],
       [10, 10],
       [ 1,  1],
       [10, 10],
       [11, 11],
       [12, 12],
       [ 2,  2],
       [10, 10],
       [15, 15],
       [ 6,  6],
       [16, 16]])
>>>


on 1-d arrays it seems to be consistent as expected:


>>> numpy.random.shuffle(r)
>>> r
array([ 6,  1,  3, 17,  0, 11, 15,  7, 14, 10,  9,  2,  8, 12,  5,  4, 19,
       16, 13, 18])
>>> 



Robert



From kxroberto at googlemail.com  Wed Nov 22 10:09:35 2006
From: kxroberto at googlemail.com (Robert)
Date: Wed, 22 Nov 2006 16:09:35 +0100
Subject: [Numpy-discussion] approx. replacement for scipy.special.betainc ?
Message-ID: <ek1p7n$jqg$1@sea.gmane.org>

for the computation of a p_value I need the betainc function, which draws the big _cephes module into a (numpy-only) app - the only reason for this import.
I need it just for the computation of a p-value like:

p_value = scipy.special.betainc(0.5*nf, 0.5, nf/(nf+t_value*t_value))

Is there a simple (python-only) replacement func for betainc ?


Robert



From aisaac at american.edu  Wed Nov 22 10:50:11 2006
From: aisaac at american.edu (Alan G Isaac)
Date: Wed, 22 Nov 2006 10:50:11 -0500
Subject: [Numpy-discussion] numpy.random.shuffle
In-Reply-To: <ek19u0$rr7$1@sea.gmane.org>
References: <ek19u0$rr7$1@sea.gmane.org>
Message-ID: <Mahogany-0.67.0-1316-20061122-105011.00@american.edu>

On Wed, 22 Nov 2006, Robert apparently wrote: 
>>>> numpy.random.shuffle(rr) 

The docstring is incomplete.  From the NumPy Book:
    shuffle (sequence)
    Randomly permute the items of any sequence. If sequence is an array, then it
    must be 1-d.

>>> rr=N.array(zip(range(20),range(20)))
>>> N.random.shuffle(rr.flat)
>>> rr
array([[ 0, 16],
       [10,  1],
       [19, 13],
       [18, 14],
       [13, 11],
       [12,  4],
       [19,  9],
       [18, 16],
       [14,  7],
       [ 5, 17],
       [ 2,  3],
       [ 9,  7],
       [12,  3],
       [ 1,  4],
       [ 8, 17],
       [ 6,  6],
       [11,  0],
       [ 5, 15],
       [10,  8],
       [ 2, 15]])

What surprised me more is that you cannot shuffle the rows 
with Python's random.shuffle either: the exact same problem 
you noted crops up!  (Works fine with a list of tuples,
of course.) But you could do:

>>> rr=N.array(zip(range(20),range(20)))
>>> pid = N.random.permutation(rr.shape[0])
>>> rr=rr[pid]
>>> rr
array([[ 9,  9],
       [ 2,  2],
       [ 3,  3],
       [15, 15],
       [ 4,  4],
       [11, 11],
       [18, 18],
       [17, 17],
       [ 5,  5],
       [12, 12],
       [13, 13],
       [ 1,  1],
       [ 8,  8],
       [14, 14],
       [16, 16],
       [19, 19],
       [ 7,  7],
       [ 6,  6],
       [10, 10],
       [ 0,  0]])

hth,
Alan Isaac





From robert.kern at gmail.com  Wed Nov 22 11:29:05 2006
From: robert.kern at gmail.com (Robert Kern)
Date: Wed, 22 Nov 2006 10:29:05 -0600
Subject: [Numpy-discussion] numpy.random.shuffle
In-Reply-To: <Mahogany-0.67.0-1316-20061122-105011.00@american.edu>
References: <ek19u0$rr7$1@sea.gmane.org>
	<Mahogany-0.67.0-1316-20061122-105011.00@american.edu>
Message-ID: <45647AD1.5090208@gmail.com>

Alan G Isaac wrote:

>>>> rr=N.array(zip(range(20),range(20)))
>>>> pid = N.random.permutation(rr.shape[0])
>>>> rr=rr[pid]
>>>> rr
> array([[ 9,  9],
>        [ 2,  2],
>        [ 3,  3],
>        [15, 15],
>        [ 4,  4],
>        [11, 11],
>        [18, 18],
>        [17, 17],
>        [ 5,  5],
>        [12, 12],
>        [13, 13],
>        [ 1,  1],
>        [ 8,  8],
>        [14, 14],
>        [16, 16],
>        [19, 19],
>        [ 7,  7],
>        [ 6,  6],
>        [10, 10],
>        [ 0,  0]])

This is really a thinko on my part. I copied the algorithm from Python's random
module. At the core of it is a set of swaps:

    x[i], x[j] = x[j], x[i]

With the kind of sequences that the stdlib random module is expecting, that
makes perfect sense. However, with N-dim arrays (N > 1), x[i] is a *view* into
the array. By the time that x[j] = x[i] gets executed, x[i] = x[j] has already
executed and the underlying memory that x[i] points to has been modified.

-- 
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless enigma
 that is made terrible by our own mad attempt to interpret it as though it had
 an underlying truth."
  -- Umberto Eco


From vallis.35530053 at bloglines.com  Wed Nov 22 11:44:07 2006
From: vallis.35530053 at bloglines.com (vallis.35530053 at bloglines.com)
Date: 22 Nov 2006 16:44:07 -0000
Subject: [Numpy-discussion] Numpy without BLAS, LAPACK
Message-ID: <1164213847.3237305572.19106.sendItem@bloglines.com>

(Reposting to numpy-discussion at scipy.org instead of the SourceForge list.)


It is my understanding that Numpy has "lite" versions of BLAS and LAPACK
that it will use if it cannot find system libraries. Is it possible to FORCE
it to use the lite versions rather than existing system libraries?

(Where
I come from: I'm trying to install numpy in a local directory on a Beowulf
cluster, which has local BLAS and LAPACK that the numpy setup finds; however,
these libraries are missing some functions, so "import numpy" fails at numpy.linalg.
The fact is that I don't need numpy.linalg at all, so I'd be very happy with
the lite libraries, rather than going to the trouble of recompiling BLAS and
LAPACK or ATLAS, etc.)

Michele


From Chris.Barker at noaa.gov  Wed Nov 22 11:53:19 2006
From: Chris.Barker at noaa.gov (Christopher Barker)
Date: Wed, 22 Nov 2006 08:53:19 -0800
Subject: [Numpy-discussion] numpy.random.shuffle
In-Reply-To: <45647AD1.5090208@gmail.com>
References: <ek19u0$rr7$1@sea.gmane.org>
	<Mahogany-0.67.0-1316-20061122-105011.00@american.edu>
	<45647AD1.5090208@gmail.com>
Message-ID: <4564807F.3080209@noaa.gov>

Robert Kern wrote:
> This is really a thinko on my part.

What, exactly, is a thinko?

> I copied the algorithm from Python's random
> module. At the core of it is a set of swaps:
> 
>     x[i], x[j] = x[j], x[i]
> 
> With the kind of sequences that the stdlib random module is expecting, that
> makes perfect sense. However, with N-dim arrays (N > 1), x[i] is a *view* into
> the array. By the time that x[j] = x[i] gets executed, x[i] = x[j] has already
> executed and the underlying memory that x[i] points to has been modified.

wouldn't something like:

temp = x[i].copy()
x[i], x[j] = x[j], temp

work?

In any case, it should raise an error if ndim > 1, rather than giving a 
wrong result.

-Chris


-- 
Christopher Barker, Ph.D.
Oceanographer

Emergency Response Division
NOAA/NOS/OR&R            (206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115       (206) 526-6317   main reception

Chris.Barker at noaa.gov


From robert.kern at gmail.com  Wed Nov 22 12:17:19 2006
From: robert.kern at gmail.com (Robert Kern)
Date: Wed, 22 Nov 2006 11:17:19 -0600
Subject: [Numpy-discussion] numpy.random.shuffle
In-Reply-To: <4564807F.3080209@noaa.gov>
References: <ek19u0$rr7$1@sea.gmane.org>	<Mahogany-0.67.0-1316-20061122-105011.00@american.edu>	<45647AD1.5090208@gmail.com>
	<4564807F.3080209@noaa.gov>
Message-ID: <4564861F.2030800@gmail.com>

Christopher Barker wrote:
> Robert Kern wrote:
>> This is really a thinko on my part.
> 
> What, exactly, is a thinko?

Like a typo except that the fault lies with the brain, not the fingers.

>> I copied the algorithm from Python's random
>> module. At the core of it is a set of swaps:
>>
>>     x[i], x[j] = x[j], x[i]
>>
>> With the kind of sequences that the stdlib random module is expecting, that
>> makes perfect sense. However, with N-dim arrays (N > 1), x[i] is a *view* into
>> the array. By the time that x[j] = x[i] gets executed, x[i] = x[j] has already
>> executed and the underlying memory that x[i] points to has been modified.
> 
> wouldn't something like:
> 
> temp = x[i].copy()
> x[i], x[j] = x[j], temp
> 
> work?
> 
> In any case, it should raise an error if ndim > 1, rather than giving a 
> wrong result.

The method is intended to work on sequences in general, not just numpy arrays,
so I can't really use .copy() or even test for ndim > 1.

One possibility is to check if the object is an ndarray (or subclass) and use
.copy() if so; otherwise, use the current implementation and hope that you
didn't pass it a Numeric or numarray array (or some other view-based object).

-- 
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless enigma
 that is made terrible by our own mad attempt to interpret it as though it had
 an underlying truth."
  -- Umberto Eco


From tim.hochberg at ieee.org  Wed Nov 22 12:34:11 2006
From: tim.hochberg at ieee.org (Tim Hochberg)
Date: Wed, 22 Nov 2006 10:34:11 -0700
Subject: [Numpy-discussion] numpy.random.shuffle
In-Reply-To: <4564861F.2030800@gmail.com>
References: <ek19u0$rr7$1@sea.gmane.org>	<Mahogany-0.67.0-1316-20061122-105011.00@american.edu>	<45647AD1.5090208@gmail.com>	<4564807F.3080209@noaa.gov>
	<4564861F.2030800@gmail.com>
Message-ID: <45648A13.4090500@ieee.org>

Robert Kern wrote:
> Christopher Barker wrote:
>   
>> Robert Kern wrote:
>>     
[SNIP]
>>> I copied the algorithm from Python's random
>>> module. At the core of it is a set of swaps:
>>>
>>>     x[i], x[j] = x[j], x[i]
>>>
>>> With the kind of sequences that the stdlib random module is expecting, that
>>> makes perfect sense. However, with N-dim arrays (N > 1), x[i] is a *view* into
>>> the array. By the time that x[j] = x[i] gets executed, x[i] = x[j] has already
>>> executed and the underlying memory that x[i] points to has been modified.
>>>       
>> wouldn't something like:
>>
>> temp = x[i].copy()
>> x[i], x[j] = x[j], temp
>>
>> work?
>>
>> In any case, it should raise an error if ndim > 1, rather than giving a 
>> wrong result.
>>     
>
> The method is intended to work on sequences in general, not just numpy arrays,
> so I can't really use .copy() or even test for ndim > 1.
>
> One possibility is to check if the object is an ndarray (or subclass) and use
> .copy() if so; otherwise, use the current implementation and hope that you
> didn't pass it a Numeric or numarray array (or some other view-based object).
>   
I think I would invert this test and instead check if the object is a 
Python list and *not* copy in that case. Otherwise, use copy.copy to 
copy the object whatever it is. This looks like it would be more robust 
in that it would work in all sensible case, and just be a tad slower in 
some of them.

Another possible refinement / complication would be to special case 1D 
arrays so that they run fastish.

A third possibility involves rewriting this in this form:

    indices = arange(len(x))
    _shuffle_core(indices) # This just does what current shuffle now does
    x[:] = take(x, indices, 0)

Of course, all of these are pretty inefficient in the common case of a 
1D array.

-tim




From robert.kern at gmail.com  Wed Nov 22 12:48:05 2006
From: robert.kern at gmail.com (Robert Kern)
Date: Wed, 22 Nov 2006 11:48:05 -0600
Subject: [Numpy-discussion] numpy.random.shuffle
In-Reply-To: <45648A13.4090500@ieee.org>
References: <ek19u0$rr7$1@sea.gmane.org>	<Mahogany-0.67.0-1316-20061122-105011.00@american.edu>	<45647AD1.5090208@gmail.com>	<4564807F.3080209@noaa.gov>	<4564861F.2030800@gmail.com>
	<45648A13.4090500@ieee.org>
Message-ID: <45648D55.4030400@gmail.com>

Tim Hochberg wrote:
> Robert Kern wrote:

>> One possibility is to check if the object is an ndarray (or subclass) and use
>> .copy() if so; otherwise, use the current implementation and hope that you
>> didn't pass it a Numeric or numarray array (or some other view-based object).
>>   
> I think I would invert this test and instead check if the object is a 
> Python list and *not* copy in that case. Otherwise, use copy.copy to 
> copy the object whatever it is. This looks like it would be more robust 
> in that it would work in all sensible case, and just be a tad slower in 
> some of them.

I don't want to assume that the only two sequence types are lists and arrays.
The problem with using copy.copy() on non-arrays is that it, well, makes copies
of the elements. The objects in the shuffled sequence are not the same objects
before and after the shuffling. I consider that to be a violation of the spec.

Views are rare outside of numpy/Numeric/numarray, partially because Guido
considers them to be evil. I'm beginning to see why.

> Another possible refinement / complication would be to special case 1D 
> arrays so that they run fastish.
> 
> A third possibility involves rewriting this in this form:
> 
>     indices = arange(len(x))
>     _shuffle_core(indices) # This just does what current shuffle now does
>     x[:] = take(x, indices, 0)

That's problematic since the elements all turn into numpy scalar objects:

In [1]: from numpy import *

In [2]: a = range(9,-1,-1)

In [3]: idx = arange(len(a))

In [4]: a[:] = take(a, idx, 0)

In [5]: a
Out[5]: [9, 8, 7, 6, 5, 4, 3, 2, 1, 0]

In [6]: type(a[0])
Out[6]: <type 'numpy.int32'>


-- 
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless enigma
 that is made terrible by our own mad attempt to interpret it as though it had
 an underlying truth."
  -- Umberto Eco


From tim.hochberg at ieee.org  Wed Nov 22 13:21:52 2006
From: tim.hochberg at ieee.org (Tim Hochberg)
Date: Wed, 22 Nov 2006 11:21:52 -0700
Subject: [Numpy-discussion] numpy.random.shuffle
In-Reply-To: <45648D55.4030400@gmail.com>
References: <ek19u0$rr7$1@sea.gmane.org>	<Mahogany-0.67.0-1316-20061122-105011.00@american.edu>	<45647AD1.5090208@gmail.com>	<4564807F.3080209@noaa.gov>	<4564861F.2030800@gmail.com>	<45648A13.4090500@ieee.org>
	<45648D55.4030400@gmail.com>
Message-ID: <45649540.6050504@ieee.org>

Robert Kern wrote:
> Tim Hochberg wrote:
>   
>> Robert Kern wrote:
>>     
>
>   
>>> One possibility is to check if the object is an ndarray (or subclass) and use
>>> .copy() if so; otherwise, use the current implementation and hope that you
>>> didn't pass it a Numeric or numarray array (or some other view-based object).
>>>   
>>>       
>> I think I would invert this test and instead check if the object is a 
>> Python list and *not* copy in that case. Otherwise, use copy.copy to 
>> copy the object whatever it is. This looks like it would be more robust 
>> in that it would work in all sensible case, and just be a tad slower in 
>> some of them.
>>     
>
> I don't want to assume that the only two sequence types are lists and arrays.
> The problem with using copy.copy() on non-arrays is that it, well, makes copies
> of the elements. The objects in the shuffled sequence are not the same objects
> before and after the shuffling. I consider that to be a violation of the spec.
>   
OK, makes sense.
> Views are rare outside of numpy/Numeric/numarray, partially because Guido
> considers them to be evil. I'm beginning to see why.
>   
They are enough rope to shoot yourself in the foot and that's for sure.

>> Another possible refinement / complication would be to special case 1D 
>> arrays so that they run fastish.
>>
>> A third possibility involves rewriting this in this form:
>>
>>     indices = arange(len(x))
>>     _shuffle_core(indices) # This just does what current shuffle now does
>>     x[:] = take(x, indices, 0)
>>     
>
> That's problematic since the elements all turn into numpy scalar objects:
>   
[SNIP]

I believe that this one is fixable:

    def shuffle_take(x):
        indices = np.arange(len(x))
        np.random.shuffle(indices)
        source = np.asarray(x, dtype=object)
        x[:] = source[indices]
        return x

    a = range(9,-1,-1)
    shuffle_take(a)
    print a
    print type(a[0]), a[0]
    b = np.arange(10).reshape(5,2)
    print b
    shuffle_take(b)
    print b

====>

    [0, 8, 1, 4, 9, 5, 7, 3, 6, 2]
    <type 'int'> 0
    [[0 1]
     [2 3]
     [4 5]
     [6 7]
     [8 9]]
    [[8 9]
     [6 7]
     [0 1]
     [2 3]
     [4 5]]

I think that in real life you'd probably want only convert to an array 
of type object if 'x' is not an array to begin with, but I'm keeping it 
simple for right now.

-tim




From Chris.Barker at noaa.gov  Wed Nov 22 14:14:33 2006
From: Chris.Barker at noaa.gov (Christopher Barker)
Date: Wed, 22 Nov 2006 11:14:33 -0800
Subject: [Numpy-discussion] numpy.random.shuffle
In-Reply-To: <45648D55.4030400@gmail.com>
References: <ek19u0$rr7$1@sea.gmane.org>
	<Mahogany-0.67.0-1316-20061122-105011.00@american.edu>
	<45647AD1.5090208@gmail.com> <4564807F.3080209@noaa.gov>
	<4564861F.2030800@gmail.com> <45648A13.4090500@ieee.org>
	<45648D55.4030400@gmail.com>
Message-ID: <4564A199.8020508@noaa.gov>

Robert Kern wrote:
> I don't want to assume that the only two sequence types are lists and arrays.

Does numpy.random.shuffle really have to work on any sequence? and 
without making a copy? I'm not so sure -- having num* functions operate 
on any sequence has been a design goal of Numeric from the beginning, 
but I've never thought it that important.

However, it is pretty consistent across numpy, so we should probably 
support it here.

How about two cases:

1) sequences that support copy()
2) sequences that don't -- in that case, it is assumed that indexing has 
copy semantics.

It truth, this means:
1) numpy arrays
2) standard python mutable sequences

But it does satisfy the duck typing approach.

Maybe we could specifically check for copy vs. view semantics, but that 
seems like overkill.

-Chris



-- 
Christopher Barker, Ph.D.
Oceanographer

Emergency Response Division
NOAA/NOS/OR&R            (206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115       (206) 526-6317   main reception

Chris.Barker at noaa.gov


From robert.kern at gmail.com  Wed Nov 22 14:47:30 2006
From: robert.kern at gmail.com (Robert Kern)
Date: Wed, 22 Nov 2006 13:47:30 -0600
Subject: [Numpy-discussion] numpy.random.shuffle
In-Reply-To: <4564A199.8020508@noaa.gov>
References: <ek19u0$rr7$1@sea.gmane.org>	<Mahogany-0.67.0-1316-20061122-105011.00@american.edu>	<45647AD1.5090208@gmail.com>
	<4564807F.3080209@noaa.gov>	<4564861F.2030800@gmail.com>
	<45648A13.4090500@ieee.org>	<45648D55.4030400@gmail.com>
	<4564A199.8020508@noaa.gov>
Message-ID: <4564A952.7090408@gmail.com>

Christopher Barker wrote:
> Robert Kern wrote:
>> I don't want to assume that the only two sequence types are lists and arrays.
> 
> Does numpy.random.shuffle really have to work on any sequence? and 
> without making a copy? I'm not so sure -- having num* functions operate 
> on any sequence has been a design goal of Numeric from the beginning, 
> but I've never thought it that important.

I think it is for this one function. I want numpy.random to be able to have all
of the functionality of the stdlib's random module (although not as a drop-in
replacement). The reason is that random number generation is something that
should be completely and centrally controllable. I should be able to write my
application such that it seeds the PRNG state *once* and uses that state
everywhere. If I need to keep instances of both numpy.random.RandomState and
random.Random around, then something's wrong.

> However, it is pretty consistent across numpy, so we should probably 
> support it here.
> 
> How about two cases:
> 
> 1) sequences that support copy()

It's not the container sequences would support a .copy() method, but the indexed
items.

> 2) sequences that don't -- in that case, it is assumed that indexing has 
> copy semantics.
> 
> It truth, this means:
> 1) numpy arrays
> 2) standard python mutable sequences
> 
> But it does satisfy the duck typing approach.
> 
> Maybe we could specifically check for copy vs. view semantics, but that 
> seems like overkill.

It's also probably impossible.

-- 
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless enigma
 that is made terrible by our own mad attempt to interpret it as though it had
 an underlying truth."
  -- Umberto Eco


From cookedm at physics.mcmaster.ca  Wed Nov 22 14:58:34 2006
From: cookedm at physics.mcmaster.ca (David M. Cooke)
Date: Wed, 22 Nov 2006 14:58:34 -0500
Subject: [Numpy-discussion] Numpy without BLAS, LAPACK
In-Reply-To: <1164213847.3237305572.19106.sendItem@bloglines.com>
References: <1164213847.3237305572.19106.sendItem@bloglines.com>
Message-ID: <20061122145834.4f8f3309@arbutus.physics.mcmaster.ca>

On 22 Nov 2006 16:44:07 -0000
vallis.35530053 at bloglines.com wrote:

> (Reposting to numpy-discussion at scipy.org instead of the SourceForge list.)
> 
> 
> It is my understanding that Numpy has "lite" versions of BLAS and LAPACK
> that it will use if it cannot find system libraries. Is it possible to FORCE
> it to use the lite versions rather than existing system libraries?
> 
> (Where
> I come from: I'm trying to install numpy in a local directory on a Beowulf
> cluster, which has local BLAS and LAPACK that the numpy setup finds;
> however, these libraries are missing some functions, so "import numpy"
> fails at numpy.linalg. The fact is that I don't need numpy.linalg at all,
> so I'd be very happy with the lite libraries, rather than going to the
> trouble of recompiling BLAS and LAPACK or ATLAS, etc.)

Set the environment variables ATLAS, BLAS, and LAPACK to 'None'. For instance,

$ ATLAS=None BLAS=None LAPACK=None python setup.py install

Note that numpy.dot() uses BLAS if it's found, so if you compile without it,
it'll be slower. This may impact other routines not in numpy.linalg.

(It's on my list to add a faster xgemm routine when BLAS isn't used...)

-- 
|>|\/|<
/--------------------------------------------------------------------------\
|David M. Cooke                      http://arbutus.physics.mcmaster.ca/dmc/
|cookedm at physics.mcmaster.ca


From kxroberto at googlemail.com  Wed Nov 22 15:28:20 2006
From: kxroberto at googlemail.com (Robert)
Date: Wed, 22 Nov 2006 21:28:20 +0100
Subject: [Numpy-discussion] numpy.random.shuffle
In-Reply-To: <45648D55.4030400@gmail.com>
References: <ek19u0$rr7$1@sea.gmane.org>	<Mahogany-0.67.0-1316-20061122-105011.00@american.edu>	<45647AD1.5090208@gmail.com>	<4564807F.3080209@noaa.gov>	<4564861F.2030800@gmail.com>	<45648A13.4090500@ieee.org>
	<45648D55.4030400@gmail.com>
Message-ID: <ek2bt5$r38$1@sea.gmane.org>

Robert Kern wrote:
> Tim Hochberg wrote:
>> Robert Kern wrote:
> 
>>> One possibility is to check if the object is an ndarray (or subclass) and use
>>> .copy() if so; otherwise, use the current implementation and hope that you
>>> didn't pass it a Numeric or numarray array (or some other view-based object).
>>>   
>> I think I would invert this test and instead check if the object is a 
>> Python list and *not* copy in that case. Otherwise, use copy.copy to 
>> copy the object whatever it is. This looks like it would be more robust 
>> in that it would work in all sensible case, and just be a tad slower in 
>> some of them.
> 
> I don't want to assume that the only two sequence types are lists and arrays.
> The problem with using copy.copy() on non-arrays is that it, well, makes copies
> of the elements. The objects in the shuffled sequence are not the same objects
> before and after the shuffling. I consider that to be a violation of the spec.
> 
> Views are rare outside of numpy/Numeric/numarray, partially because Guido
> considers them to be evil. I'm beginning to see why.
> 
>> Another possible refinement / complication would be to special case 1D 
>> arrays so that they run fastish.
>>
>> A third possibility involves rewriting this in this form:
>>
>>     indices = arange(len(x))
>>     _shuffle_core(indices) # This just does what current shuffle now does
>>     x[:] = take(x, indices, 0)
> 
> That's problematic since the elements all turn into numpy scalar objects:
> 
> In [1]: from numpy import *
> 
> In [2]: a = range(9,-1,-1)
> 
> In [3]: idx = arange(len(a))
> 
> In [4]: a[:] = take(a, idx, 0)
> 
> In [5]: a
> Out[5]: [9, 8, 7, 6, 5, 4, 3, 2, 1, 0]
> 
> In [6]: type(a[0])
> Out[6]: <type 'numpy.int32'>
> 

a[:]=take(asarray(a,object),idx,0)  ?  works also correct with ndarray's even if I didn't dig the reason why... all element will be probably re-casted twice.

Think the take-method on shuffled indizes is basically right and natural for a numpy-shuffler. 
The example is just possibly another vote against the default behavior of letting numpy.scalar types out of arrays, which are set up with a "harmless" type.

>>> array([1,2,3],float)
array([ 1.,  2.,  3.])
>>> type(_[0])
<type 'numpy.float64'>
>>> 

is just ill as it I think. 
In (Guido's) Python objects should probably come out of collections best as typy as they went in. Currently numpy-scalars will just "infect" the whole app almost like a virus (and kill performance and pickle's etc.)
Of course views are essential for an efficient array type, but type-altering possibly not.
For rare cases for generalized algs (I need to think hard to find even an example), where the array-interface is needed on elements (and a array(obj) cast is too uncomfortable), there could be still the different possibilty:

>>> array([1,2,3],numpy.float64)

then its natural that numpy.float64, numpy.int32.... come out, as the programmer would even expect it so.

Thus maybe for array types:  
* float!=numpy.float64  (but common base class (or 'float' itself) maybe)
* int !=numpy.intXX
* complex !=numpy.complex128
* default array type is (python.)float
* default array type from list of ints is (python.)int
* default array type from list of complex is (python.)complex
* default array type of other lists is always <object>

currently this is also problematic: 
>>> array([1,2,"3",[]])
array(['1', '2', '3', '[]'], 
      dtype='|S4')

and even

>>> array([1,2,"3ef",'wefwfewoiwjefo iwjef'])
array(['1', '2', '3ef', 'wefwfewoiwjefo iwjef'], 
      dtype='|S20')
>>> _[0]='woeifjwo woie pwioef wliuefh lwieufh wleifuh welfiu '
>>> _
array(['woeifjwo woie pwioef', '2', '3ef', 'wefwfewoiwjefo iwjef'], 
      dtype='|S20')

is rarely what a Pythoneer would expect. Guess fix string arrays should only be created explicitely

Robert



From michael.sorich at gmail.com  Thu Nov 23 00:01:44 2006
From: michael.sorich at gmail.com (Michael Sorich)
Date: Thu, 23 Nov 2006 15:31:44 +1030
Subject: [Numpy-discussion] A reimplementation of MaskedArray
In-Reply-To: <200611220228.44329.pgmdevlist@gmail.com>
References: <200610160232.20888.pgmdevlist@mailcan.com>
	<200611212309.17463.pgmdevlist@gmail.com>
	<16761e100611212125w42fe9d77j4425c8f7f3136efb@mail.gmail.com>
	<200611220228.44329.pgmdevlist@gmail.com>
Message-ID: <16761e100611222101p7b590062u9de859e39d6bbbc0@mail.gmail.com>

> > Unfortunately if the value is changed to masked, this is not updated
> > in the parent array. This seems very inconsistent. I don't view masked
> > values any different than any other value.
>
> Inconsistent, maybe, useful definitely:
> Masking a view and getting the original masked accordingly could be useful,
> but I strongly feel that unmasking a view and getting an unmasked orginal is
> dangerous.

The use of nan in float ndarrays has behaviour I would consider
expected (at least coming from a numpy perspective). I think that the
closer the maskedarrays mimic the behavior of ndarrays the easier it
will be to remember how to use them and what to expect.

from numpy import *
a = array([[1,2,3,4,5],[1,2,3,4,5],[1,2,3,4,5]], dtype='f')
suba = a[2]
suba[1] = nan
print a
>[[ 1.      2.      3.      4.      5.    ]
> [ 1.      2.      3.      4.      5.    ]
> [ 1.     -1.#IND  3.      4.      5.    ]]
print suba
>[ 1.     -1.#IND  3.      4.      5.    ]

a[2,2] = nan
print a
>[[ 1.      2.      3.      4.      5.    ]
> [ 1.      2.      3.      4.      5.    ]
> [ 1.     -1.#IND -1.#IND  4.      5.    ]]
print suba
>[ 1.     -1.#IND -1.#IND  4.      5.    ]


From konrad.hinsen at laposte.net  Thu Nov 23 09:24:08 2006
From: konrad.hinsen at laposte.net (konrad.hinsen at laposte.net)
Date: Thu, 23 Nov 2006 15:24:08 +0100
Subject: [Numpy-discussion] Wanted: Numeric/NumPy compatible array creation
	expression
Message-ID: <C7F9C351-CFB8-4DAE-8B8D-2504485E59BC@laposte.net>

I am looking for a way to create an array of the same type as another  
given array (but of different shape) that works both with Numeric and  
NumPy without being unreasonably slow. In other words, I am looking  
for a a replacement for the expression

	array2 = Numeric.zeros(shape, array1.typecode())

that will *also* (not *only*) work under NumPy (where typecode()  
became dtype.char). An obvious idea is reshaping array1 and  
multiplying by 0., but that can become quite costly.

Any ideas?

Konrad.
--
---------------------------------------------------------------------
Konrad Hinsen
Centre de Biophysique Mol?culaire, CNRS Orl?ans
Synchrotron Soleil - Division Exp?riences
Saint Aubin - BP 48
91192 Gif sur Yvette Cedex, France
Tel. +33-1 69 35 97 15
E-Mail: hinsen at cnrs-orleans.fr
---------------------------------------------------------------------




From kxroberto at googlemail.com  Thu Nov 23 10:29:00 2006
From: kxroberto at googlemail.com (Robert)
Date: Thu, 23 Nov 2006 16:29:00 +0100
Subject: [Numpy-discussion] Wanted: Numeric/NumPy compatible array
	creation expression
In-Reply-To: <C7F9C351-CFB8-4DAE-8B8D-2504485E59BC@laposte.net>
References: <C7F9C351-CFB8-4DAE-8B8D-2504485E59BC@laposte.net>
Message-ID: <ek4enu$uh4$1@sea.gmane.org>

konrad.hinsen at laposte.net wrote:
> I am looking for a way to create an array of the same type as another  
> given array (but of different shape) that works both with Numeric and  
> NumPy without being unreasonably slow. In other words, I am looking  
> for a a replacement for the expression
> 
> 	array2 = Numeric.zeros(shape, array1.typecode())
> 
> that will *also* (not *only*) work under NumPy (where typecode()  
> became dtype.char). An obvious idea is reshaping array1 and  
> multiplying by 0., but that can become quite costly.

timeit if *0 is really costly ... you'll not get it really cheaper I think.
As you write just "same type": .copy().reshape() or so maybe ok too, in case you'll fill in stuff into this array anyway

> Any ideas?
> 
> Konrad.
> --
> ---------------------------------------------------------------------
> Konrad Hinsen
> Centre de Biophysique Mol?culaire, CNRS Orl?ans
> Synchrotron Soleil - Division Exp?riences
> Saint Aubin - BP 48
> 91192 Gif sur Yvette Cedex, France
> Tel. +33-1 69 35 97 15
> E-Mail: hinsen at cnrs-orleans.fr
> ---------------------------------------------------------------------



From konrad.hinsen at laposte.net  Thu Nov 23 10:34:09 2006
From: konrad.hinsen at laposte.net (konrad.hinsen at laposte.net)
Date: Thu, 23 Nov 2006 16:34:09 +0100
Subject: [Numpy-discussion] Wanted: Numeric/NumPy compatible array
	creation expression
In-Reply-To: <ek4enu$uh4$1@sea.gmane.org>
References: <C7F9C351-CFB8-4DAE-8B8D-2504485E59BC@laposte.net>
	<ek4enu$uh4$1@sea.gmane.org>
Message-ID: <65169BA0-C94D-4166-B287-661037121926@laposte.net>

On 23.11.2006, at 16:29, Robert wrote:

>> that will *also* (not *only*) work under NumPy (where typecode()
>> became dtype.char). An obvious idea is reshaping array1 and
>> multiplying by 0., but that can become quite costly.
>
> timeit if *0 is really costly ... you'll not get it really cheaper  
> I think.

What I expect to be costly is the reshape (resize in most  
situations), not the multiplication. I didn't do any timings though...

Konrad
--
---------------------------------------------------------------------
Konrad Hinsen
Centre de Biophysique Mol?culaire, CNRS Orl?ans
Synchrotron Soleil - Division Exp?riences
Saint Aubin - BP 48
91192 Gif sur Yvette Cedex, France
Tel. +33-1 69 35 97 15
E-Mail: hinsen at cnrs-orleans.fr
---------------------------------------------------------------------




From kxroberto at googlemail.com  Thu Nov 23 11:44:10 2006
From: kxroberto at googlemail.com (Robert)
Date: Thu, 23 Nov 2006 17:44:10 +0100
Subject: [Numpy-discussion] Wanted: Numeric/NumPy compatible array
	creation expression
In-Reply-To: <65169BA0-C94D-4166-B287-661037121926@laposte.net>
References: <C7F9C351-CFB8-4DAE-8B8D-2504485E59BC@laposte.net>	<ek4enu$uh4$1@sea.gmane.org>
	<65169BA0-C94D-4166-B287-661037121926@laposte.net>
Message-ID: <ek4j4r$dso$1@sea.gmane.org>

konrad.hinsen at laposte.net wrote:
> On 23.11.2006, at 16:29, Robert wrote:
> 
>>> that will *also* (not *only*) work under NumPy (where typecode()
>>> became dtype.char). An obvious idea is reshaping array1 and
>>> multiplying by 0., but that can become quite costly.
>> timeit if *0 is really costly ... you'll not get it really cheaper  
>> I think.
> 
> What I expect to be costly is the reshape (resize in most  
> situations), not the multiplication. I didn't do any timings though...
> 

its simple with:

a=...
import timeit
timeit.Timer("b=a.reshape....",glbls=globals()).timeit(1000)

reshaping is usually very cheap (numpy), as only an new view with other index geometry is created. 
A (unavoidable?) copy is probably the most expensive but just test ... 



Robert



From konrad.hinsen at laposte.net  Fri Nov 24 13:04:00 2006
From: konrad.hinsen at laposte.net (konrad.hinsen at laposte.net)
Date: Fri, 24 Nov 2006 19:04:00 +0100
Subject: [Numpy-discussion] ScientificPython with NumPy
Message-ID: <0472E758-6309-4D18-8456-EDE575B33B48@laposte.net>

Those who would like to test-drive ScientificPython with NumPy can do  
so now: just download version 2.7.1 from

	http://sourcesup.cru.fr/

and install using

	python setup.py build --numpy
	python setup.py install --numpy

Note that I have relied on NumPy's automatic code converter to  
identify the places that needed to be changed. In other words, there  
could still be incompatibilities, of the kinds that the converter  
cannot spot. The changes are mine, however, as my goal was to have  
Python code compatible with both Numeric and NumPy.

Bug reports are welcome, ideally through the Web site mentioned above.

Konrad.
--
---------------------------------------------------------------------
Konrad Hinsen
Centre de Biophysique Mol?culaire, CNRS Orl?ans
Synchrotron Soleil - Division Exp?riences
Saint Aubin - BP 48
91192 Gif sur Yvette Cedex, France
Tel. +33-1 69 35 97 15
E-Mail: hinsen at cnrs-orleans.fr
---------------------------------------------------------------------




From amit_605 at yahoo.co.in  Fri Nov 24 17:09:29 2006
From: amit_605 at yahoo.co.in (amit soni)
Date: Sat, 25 Nov 2006 03:39:29 +0530 (IST)
Subject: [Numpy-discussion] Installing CVXOPT
Message-ID: <20061124220929.82697.qmail@web8403.mail.in.yahoo.com>

hi,

how can I install and start using CVXOPT. I have python 2.5 version installed. what else do i need to download and install for CVXOPT.

thanks
amit 




 
____________________________________________________________________________________
Want to start your own business?
Learn how on Yahoo! Small Business.
http://smallbusiness.yahoo.com/r-index
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20061125/8751e598/attachment.html>

From robert.kern at gmail.com  Fri Nov 24 19:36:03 2006
From: robert.kern at gmail.com (Robert Kern)
Date: Fri, 24 Nov 2006 18:36:03 -0600
Subject: [Numpy-discussion] Installing CVXOPT
In-Reply-To: <20061124220929.82697.qmail@web8403.mail.in.yahoo.com>
References: <20061124220929.82697.qmail@web8403.mail.in.yahoo.com>
Message-ID: <45678FF3.2060804@gmail.com>

amit soni wrote:
> hi,
> 
> how can I install and start using CVXOPT. I have python 2.5 version
> installed. what else do i need to download and install for CVXOPT.

CVXOPT does not use numpy, so it is unlikely that you will get your question
answered here. You can download the source tarball from this page:

  http://www.ee.ucla.edu/~vandenbe/cvxopt/

It includes installation instructions. If you need help, I recommend that you
ask the authors. Their email addresses are on that page.

-- 
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless enigma
 that is made terrible by our own mad attempt to interpret it as though it had
 an underlying truth."
  -- Umberto Eco


From gruben at bigpond.net.au  Fri Nov 24 21:35:03 2006
From: gruben at bigpond.net.au (Gary Ruben)
Date: Sat, 25 Nov 2006 13:35:03 +1100
Subject: [Numpy-discussion] ScientificPython with NumPy
In-Reply-To: <0472E758-6309-4D18-8456-EDE575B33B48@laposte.net>
References: <0472E758-6309-4D18-8456-EDE575B33B48@laposte.net>
Message-ID: <4567ABD7.7010908@bigpond.net.au>

Hi Konrad,
I can report that 2.7.1 installs OK on WinXP with the most recent 
Enthought Python. I used mingw, so replaced the build instruction with
python setup.py build --numpy --compiler=mingw32
All I tried was the Geometry.Vector class which my older code uses 
heavily - this behaves well.
regards,
Gary

konrad.hinsen at laposte.net wrote:
> Those who would like to test-drive ScientificPython with NumPy can do  
> so now: just download version 2.7.1 from
> 
> 	http://sourcesup.cru.fr/
> 
> and install using
> 
> 	python setup.py build --numpy
> 	python setup.py install --numpy
> 
> Note that I have relied on NumPy's automatic code converter to  
> identify the places that needed to be changed. In other words, there  
> could still be incompatibilities, of the kinds that the converter  
> cannot spot. The changes are mine, however, as my goal was to have  
> Python code compatible with both Numeric and NumPy.
> 
> Bug reports are welcome, ideally through the Web site mentioned above.
> 
> Konrad.


From haase at msg.ucsf.edu  Sat Nov 25 00:17:30 2006
From: haase at msg.ucsf.edu (Sebastian Haase)
Date: Fri, 24 Nov 2006 21:17:30 -0800
Subject: [Numpy-discussion] ScientificPython with NumPy
In-Reply-To: <4567ABD7.7010908@bigpond.net.au>
References: <0472E758-6309-4D18-8456-EDE575B33B48@laposte.net>
	<4567ABD7.7010908@bigpond.net.au>
Message-ID: <bc657ead0611242117y1ae66301r1bdd0132d9301ae7@mail.gmail.com>

Hi,
Just out of curiosity: Can I ask what is special about a
Geometry.Vector  ? What is the difference to a normal numpy array ?

I hope this is a good place to as this ?
Thanks,  -Sebastian Haase

On 11/24/06, Gary Ruben <gruben at bigpond.net.au> wrote:
>
> Hi Konrad,
> I can report that 2.7.1 installs OK on WinXP with the most recent
> Enthought Python. I used mingw, so replaced the build instruction with
> python setup.py build --numpy --compiler=mingw32
> All I tried was the Geometry.Vector class which my older code uses
> heavily - this behaves well.
> regards,
> Gary
>
> konrad.hinsen at laposte.net wrote:
> > Those who would like to test-drive ScientificPython with NumPy can do
> > so now: just download version 2.7.1 from
> >
> >       http://sourcesup.cru.fr/
> >
> > and install using
> >
> >       python setup.py build --numpy
> >       python setup.py install --numpy
> >
> > Note that I have relied on NumPy's automatic code converter to
> > identify the places that needed to be changed. In other words, there
> > could still be incompatibilities, of the kinds that the converter
> > cannot spot. The changes are mine, however, as my goal was to have
> > Python code compatible with both Numeric and NumPy.
> >
> > Bug reports are welcome, ideally through the Web site mentioned above.
> >
> > Konrad.
> _______________________________________________
> Numpy-discussion mailing list
> Numpy-discussion at scipy.org
> http://projects.scipy.org/mailman/listinfo/numpy-discussion
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20061124/654dc654/attachment.html>

From gruben at bigpond.net.au  Sat Nov 25 00:32:27 2006
From: gruben at bigpond.net.au (Gary Ruben)
Date: Sat, 25 Nov 2006 16:32:27 +1100
Subject: [Numpy-discussion] ScientificPython with NumPy
In-Reply-To: <bc657ead0611242117y1ae66301r1bdd0132d9301ae7@mail.gmail.com>
References: <0472E758-6309-4D18-8456-EDE575B33B48@laposte.net>	<4567ABD7.7010908@bigpond.net.au>
	<bc657ead0611242117y1ae66301r1bdd0132d9301ae7@mail.gmail.com>
Message-ID: <4567D56B.2060309@bigpond.net.au>

> Hi,
> Just out of curiosity: Can I ask what is special about a
> Geometry.Vector  ? What is the difference to a normal numpy array ?

At the time I wrote my code, it was just a convenient 3-vector class. 
Most of the functionality is now implemented in numpy so if I was to 
rewrite my code, I'd just use a native numpy rank-1 array. Konrad 
defines a couple of methods for using his Vector with his Tensor class, 
so I'd say that might be a reason to use his Vector, otherwise I'd stick 
with arrays.

Gary


From konrad.hinsen at laposte.net  Sat Nov 25 05:20:40 2006
From: konrad.hinsen at laposte.net (konrad.hinsen at laposte.net)
Date: Sat, 25 Nov 2006 11:20:40 +0100
Subject: [Numpy-discussion] ScientificPython with NumPy
In-Reply-To: <bc657ead0611242117y1ae66301r1bdd0132d9301ae7@mail.gmail.com>
References: <0472E758-6309-4D18-8456-EDE575B33B48@laposte.net>
	<4567ABD7.7010908@bigpond.net.au>
	<bc657ead0611242117y1ae66301r1bdd0132d9301ae7@mail.gmail.com>
Message-ID: <E93AC2A6-85AB-4F21-BFE8-ED20835E3993@laposte.net>

On 25.11.2006, at 06:17, Sebastian Haase wrote:

> Just out of curiosity: Can I ask what is special about a
> Geometry.Vector  ? What is the difference to a normal numpy array ?

1) A Vector always has exactly three elements of type "float".
2) A Vector is immutable.
3) Only addition and substraction work like for an array, i.e.  
elementwise.
    Multiplication between vectors is the dot product, division is an  
error.
4) A Vector has additional methods: length, normal, cross,  
dyadicProduct, angle.

Implementationwise, a vector object has a smaller memory footprint  
than an array object, at least since I reimplemented Vector in Pyrex  
a while ago.

So why would one want to use a special Vector object rather than an  
array of shape (3,)?

1) Readability: the intention of the programmer is much clearer.
2) The additional methods are quite useful.
3) Safeguarding against mistakes: with arrays, additional checks for
    shape and element size would be needed to make code robust.
4) Performance (especially memory-wise).

Konrad.
--
---------------------------------------------------------------------
Konrad Hinsen
Centre de Biophysique Mol?culaire, CNRS Orl?ans
Synchrotron Soleil - Division Exp?riences
Saint Aubin - BP 48
91192 Gif sur Yvette Cedex, France
Tel. +33-1 69 35 97 15
E-Mail: hinsen at cnrs-orleans.fr
---------------------------------------------------------------------




From konrad.hinsen at laposte.net  Sat Nov 25 10:30:10 2006
From: konrad.hinsen at laposte.net (konrad.hinsen at laposte.net)
Date: Sat, 25 Nov 2006 16:30:10 +0100
Subject: [Numpy-discussion] ScientificPython with NumPy
In-Reply-To: <0472E758-6309-4D18-8456-EDE575B33B48@laposte.net>
References: <0472E758-6309-4D18-8456-EDE575B33B48@laposte.net>
Message-ID: <17BB897A-F8ED-4BE8-B7CA-C6A1AEE016C7@laposte.net>

On 24.11.2006, at 19:04, konrad.hinsen at laposte.net wrote:

> Those who would like to test-drive ScientificPython with NumPy can do
> so now: just download version 2.7.1 from
>
> 	http://sourcesup.cru.fr/

Version 2.7.2 now, after the first bug fix: it wouldn't compile with  
NumPy and Python < 2.5, because both NumPy and ScientificPython  
define Py_ssize_t. There doesn't seem to be a nice solution for that  
problem, unfortunately.

Konrad.
--
---------------------------------------------------------------------
Konrad Hinsen
Centre de Biophysique Mol?culaire, CNRS Orl?ans
Synchrotron Soleil - Division Exp?riences
Saint Aubin - BP 48
91192 Gif sur Yvette Cedex, France
Tel. +33-1 69 35 97 15
E-Mail: hinsen at cnrs-orleans.fr
---------------------------------------------------------------------




From jan_braun at gmx.net  Sun Nov 26 10:12:47 2006
From: jan_braun at gmx.net (Jan-Matthias Braun)
Date: Sun, 26 Nov 2006 16:12:47 +0100
Subject: [Numpy-discussion] Assumed 64bit-Incompatiblity in numpy svn
	test_regression
Message-ID: <200611261612.47965.jan_braun@gmx.net>

Hi all,

after building current numpy from svn (1.0.1.dev3460 as of writing :), the 
call for numpy.test(10, 10) fails with following output added at the end of 
the mail.
The Test seems seems to be made for Ticket #372, but the problem is a type 
mismatch between 32 and 64bit Integers as far as I can see.

Should I file a ticket for this one?

Cheers,
	Jan

======================================================================
FAIL: Ticket #372
----------------------------------------------------------------------
Traceback (most recent call last):
  File "~/lib/python2.4/site-packages/numpy/core/tests/test_regression.py", 
line 581, in check_recarray_fields
    assert_equal(a.dtype,dt)
  File "~/lib/python2.4/site-packages/numpy/testing/utils.py", line 143, in 
assert_equal
    assert desired == actual, msg
AssertionError:
Items are not equal:
 ACTUAL: dtype([('f0', '<i8'), ('f1', '<i8')])
 DESIRED: dtype([('f0', '<i4'), ('f1', '<i4')])

----------------------------------------------------------------------
Ran 524 tests in 0.294s
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20061126/94a4815d/attachment.sig>

From pgmdevlist at gmail.com  Sun Nov 26 17:11:24 2006
From: pgmdevlist at gmail.com (Pierre GM)
Date: Sun, 26 Nov 2006 17:11:24 -0500
Subject: [Numpy-discussion] apply_along, apply_over w/ subarrays
Message-ID: <200611261711.24980.pgmdevlist@gmail.com>

Folks,
Is there any reason why `apply_along_axis`, `apply_over_axes` (and I expect 
`vectorize` as well, but I haven't tried) won't accept subclasses of 
ndarrays ?
Would it be possible to use `asanyarray` instead of `asarray` in those 
functions ?

Oh, and I ran into some problems with apply_along-axis: in the uncommon case 
where the output of the 1d functions is a scalar array (eg., N.array(0)) (or 
a masked_singleton), the current version raises a TypeError, as the output is 
not a scalar, but doesn't have a length either. I tried to correct that, but 
I'd prefer to get some feedback before submitting the patch.

Thanks for all.
P.

#################################################
--- numpy/lib/shape_base.py.init	2006-11-26 13:22:57.000000000 -0500
+++ numpy/lib/shape_base.py	2006-11-26 17:01:22.000000000 -0500
@@ -13,7 +13,7 @@ def apply_along_axis(func1d,axis,arr,*ar
         and arr is an N-d array.  i varies so as to apply the function
         along the given axis for each 1-d subarray in arr.
     """
-    arr = asarray(arr)
+    arr = asanyarray(arr)
     nd = arr.ndim
     if axis < 0:
         axis += nd
@@ -28,9 +28,16 @@ def apply_along_axis(func1d,axis,arr,*ar
     outshape = asarray(arr.shape).take(indlist)
     i.put(indlist, ind)
     res = func1d(arr[tuple(i.tolist())],*args)
-    #  if res is a number, then we have a smaller output array
-    if isscalar(res):
-        outarr = zeros(outshape,asarray(res).dtype)
+    #  if res is a number, or doesn't have a length , then we have a smaller 
output array
+    asscalar = isscalar(res)
+    if not asscalar:
+        try:
+            len(res)
+        except TypeError:
+            asscalar = True
+    #
+    if asscalar:
+        outarr = zeros(outshape,asarray(res).dtype).view(res.__class__)
         outarr[ind] = res
         Ntot = product(outshape)
         k = 1
@@ -52,7 +59,7 @@ def apply_along_axis(func1d,axis,arr,*ar
         holdshape = outshape
         outshape = list(arr.shape)
         outshape[axis] = len(res)
-        outarr = zeros(outshape,asarray(res).dtype)
+        outarr = zeros(outshape,asarray(res).dtype).view(res.__class__)
         outarr[tuple(i.tolist())] = res
         k = 1
         while k < Ntot:
@@ -78,7 +85,7 @@ def apply_over_axes(func, a, axes):
     to be either the same shape as a or have one less dimension.
     This call is repeated for each axis in the axes sequence.
     """
-    val = asarray(a)
+    val = asanyarray(a)
     N = a.ndim
     if array(axes).ndim == 0:
         axes = (axes,)
#################################################



From faltet at carabos.com  Mon Nov 27 01:02:11 2006
From: faltet at carabos.com (Francesc Altet)
Date: Mon, 27 Nov 2006 07:02:11 +0100
Subject: [Numpy-discussion] Assumed 64bit-Incompatiblity in numpy
	svn	test_regression
In-Reply-To: <200611261612.47965.jan_braun@gmx.net>
References: <200611261612.47965.jan_braun@gmx.net>
Message-ID: <1164607331.2874.4.camel@localhost.localdomain>

El dg 26 de 11 del 2006 a les 16:12 +0100, en/na Jan-Matthias Braun va
escriure:
> Hi all,
> 
> after building current numpy from svn (1.0.1.dev3460 as of writing :), the 
> call for numpy.test(10, 10) fails with following output added at the end of 
> the mail.
> The Test seems seems to be made for Ticket #372, but the problem is a type 
> mismatch between 32 and 64bit Integers as far as I can see.

Yes. It seems a flaw in the test design, not in the code itself. The
test is comparing int32 type values with the default int type. However,
the default type is platform dependent (int32 for 32-bit platform and
int64 for 64-bit platform). So, the default types comparisons should be
removed from the test.

> 
> Should I file a ticket for this one?

Generally speaking it is always a good idea to open a ticket for
reporting tests that doesn't work, yes.

Cheers,

-- 
Francesc Altet    |  Be careful about using the following code --
Carabos Coop. V.  |  I've only proven that it works, 
www.carabos.com   |  I haven't tested it. -- Donald Knuth



From stefan at sun.ac.za  Mon Nov 27 01:55:45 2006
From: stefan at sun.ac.za (Stefan van der Walt)
Date: Mon, 27 Nov 2006 08:55:45 +0200
Subject: [Numpy-discussion] Assumed 64bit-Incompatiblity in numpy svn
	test_regression
In-Reply-To: <200611261612.47965.jan_braun@gmx.net>
References: <200611261612.47965.jan_braun@gmx.net>
Message-ID: <20061127065545.GA30805@mentat.za.net>

On Sun, Nov 26, 2006 at 04:12:47PM +0100, Jan-Matthias Braun wrote:
> after building current numpy from svn (1.0.1.dev3460 as of writing :), the 
> call for numpy.test(10, 10) fails with following output added at the end of 
> the mail.
> The Test seems seems to be made for Ticket #372, but the problem is a type 
> mismatch between 32 and 64bit Integers as far as I can see.

Thanks, Jan.  I fixed the problem in SVN.

Cheers
St?fan


From jan_braun at gmx.net  Mon Nov 27 06:20:41 2006
From: jan_braun at gmx.net (Jan-Matthias Braun)
Date: Mon, 27 Nov 2006 12:20:41 +0100
Subject: [Numpy-discussion] Assumed 64bit-Incompatiblity in numpy svn
	test_regression
In-Reply-To: <20061127065545.GA30805@mentat.za.net>
References: <200611261612.47965.jan_braun@gmx.net>
	<20061127065545.GA30805@mentat.za.net>
Message-ID: <200611271220.41393.jan_braun@gmx.net>

Am Montag 27 November 2006 07:55 schrieb Stefan van der Walt:
> On Sun, Nov 26, 2006 at 04:12:47PM +0100, Jan-Matthias Braun wrote:
> > after building current numpy from svn (1.0.1.dev3460 as of writing :),
> > the call for numpy.test(10, 10) fails with following output added at the
> > end of the mail.
> > The Test seems seems to be made for Ticket #372, but the problem is a
> > type mismatch between 32 and 64bit Integers as far as I can see.
>
> Thanks, Jan.  I fixed the problem in SVN.

In [3]: numpy.__version__
Out[3]: '1.0.1.dev3462'

In [4]: numpy.test(10,10)

----------------------------------------------------------------------
Ran 524 tests in 0.602s

OK

Thanks for all answers and fast fixing!

Cheers,
	Jan
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20061127/fb9f33f3/attachment.sig>

From hollowspook at gmail.com  Mon Nov 27 08:59:41 2006
From: hollowspook at gmail.com (Zhang Sam)
Date: Mon, 27 Nov 2006 21:59:41 +0800
Subject: [Numpy-discussion] can int and float exists in one array?(about
	difference in indexing Matlab matrix and Numpy array)
Message-ID: <f76e92b00611270559i40143f65u4500695615bae3a7@mail.gmail.com>

Hi, there

I  have a practical problem. For some reason, I hope int and float can exist
one array as shown in matlab code.
---------------------------------------------------------------------------
 >> x = [2 2.5 3.5; 1 2.6 3.5]

x =

    2.0000    2.5000    3.5000
    1.0000    2.6000    3.5000

>> y = x(:,2)

y =

    2.5000
    2.6000

>> y(x(:,1))

ans =

    2.6000
    2.5000
------------------------------------------------------------------------------------

However in python with numpy, the similar code is as follows.
------------------------------------------------------------------------------------------
x = array([[2, 2.5, 3.5],[1, 2.6, 3.5]])
>>> x
array([[ 2. ,  2.5,  3.5],
       [ 1. ,  2.6,  3.5]])
>>> y = x[:,1]
>>> y
array([ 2.5,  2.6])
>>> y.shape=2,1
>>> y
array([[ 2.5],
       [ 2.6]])
>>> y[x[:,0],0]
Traceback (most recent call last):
  File "<interactive input>", line 1, in ?
IndexError: arrays used as indices must be of integer (or boolean) type
-----------------------------------------------------------------------------------------------------

MATLAB can treat the 1.0,2.00,......as int 1 2,.....
How to realize this matlab code in python? Maybe it exist a simple way.
Please show me .


Thanks in advance

Sam



BTW: I sent this mail to numpy-discussion at lists.sourceforge.net at first,
then I receive a mail which suggests me here. If it has already email that
here, I am very sorry about that.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20061127/458bc032/attachment.html>

From robert.kern at gmail.com  Mon Nov 27 11:49:38 2006
From: robert.kern at gmail.com (Robert Kern)
Date: Mon, 27 Nov 2006 10:49:38 -0600
Subject: [Numpy-discussion] can int and float exists in one array?(about
 difference in indexing Matlab matrix and Numpy array)
In-Reply-To: <f76e92b00611270559i40143f65u4500695615bae3a7@mail.gmail.com>
References: <f76e92b00611270559i40143f65u4500695615bae3a7@mail.gmail.com>
Message-ID: <456B1722.9030406@gmail.com>

Zhang Sam wrote:
> However in python with numpy, the similar code is as follows.
> ------------------------------------------------------------------------------------------
> x = array([[2, 2.5, 3.5],[1, 2.6, 3.5]])
>>>> x
> array([[ 2. ,  2.5,  3.5],
>        [ 1. ,  2.6,  3.5]])
>>>> y = x[:,1]
>>>> y
> array([ 2.5,  2.6])
>>>> y.shape=2,1
>>>> y
> array([[ 2.5],
>        [ 2.6]])
>>>> y[x[:,0],0]
> Traceback (most recent call last):
>   File "<interactive input>", line 1, in ?
> IndexError: arrays used as indices must be of integer (or boolean) type
> -----------------------------------------------------------------------------------------------------
>  
> MATLAB can treat the 1.0,2.00,......as int 1 2,.....
> How to realize this matlab code in python? Maybe it exist a simple way.

y[x[:,0].astype(int),0]

By and large, in Python and numpy, floats are not accepted in places where
integers are required, like indexing. The meaning is ambiguous, so we refuse the
temptation to guess what the programmer meant.

As a corollary, if you find that you are putting index values into an array with
floating point values, you are doing something wrong. At least, you are trying
to use Matlab idioms that simply aren't used in Python or numpy. We can't tell
you how to "translate" the Matlab idioms into Python idioms without knowing what
you intended to do at a slightly higher level, though.

-- 
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless enigma
 that is made terrible by our own mad attempt to interpret it as though it had
 an underlying truth."
  -- Umberto Eco


From kwgoodman at gmail.com  Mon Nov 27 12:13:16 2006
From: kwgoodman at gmail.com (Keith Goodman)
Date: Mon, 27 Nov 2006 09:13:16 -0800
Subject: [Numpy-discussion] can int and float exists in one array?(about
	difference in indexing Matlab matrix and Numpy array)
In-Reply-To: <456B1722.9030406@gmail.com>
References: <f76e92b00611270559i40143f65u4500695615bae3a7@mail.gmail.com>
	<456B1722.9030406@gmail.com>
Message-ID: <f4f93d420611270913r58a9d7b0t873f2445b9e83ba2@mail.gmail.com>

On 11/27/06, Robert Kern <robert.kern at gmail.com> wrote:
> Zhang Sam wrote:
> > However in python with numpy, the similar code is as follows.
> > ------------------------------------------------------------------------------------------
> > x = array([[2, 2.5, 3.5],[1, 2.6, 3.5]])
> >>>> x
> > array([[ 2. ,  2.5,  3.5],
> >        [ 1. ,  2.6,  3.5]])
> >>>> y = x[:,1]
> >>>> y
> > array([ 2.5,  2.6])
> >>>> y.shape=2,1
> >>>> y
> > array([[ 2.5],
> >        [ 2.6]])
> >>>> y[x[:,0],0]
> > Traceback (most recent call last):
> >   File "<interactive input>", line 1, in ?
> > IndexError: arrays used as indices must be of integer (or boolean) type
> > -----------------------------------------------------------------------------------------------------
> >
> > MATLAB can treat the 1.0,2.00,......as int 1 2,.....
> > How to realize this matlab code in python? Maybe it exist a simple way.
>
> y[x[:,0].astype(int),0]

If you want it exactly like matlab, you'll have to check to make sure that

(x[:,0] == x[:,0].astype(int)).all()

since matlab gives an error if, for example, x[0,0] is 1.00001.


From kxroberto at googlemail.com  Mon Nov 27 14:13:18 2006
From: kxroberto at googlemail.com (Robert)
Date: Mon, 27 Nov 2006 20:13:18 +0100
Subject: [Numpy-discussion] can int and float exists in one array?(about
 difference in indexing Matlab matrix and Numpy array)
In-Reply-To: <f76e92b00611270559i40143f65u4500695615bae3a7@mail.gmail.com>
References: <f76e92b00611270559i40143f65u4500695615bae3a7@mail.gmail.com>
Message-ID: <ekfdcg$9b6$1@sea.gmane.org>

Zhang Sam wrote:
> Hi, there
>  
> I  have a practical problem. For some reason, I hope int and float can 
> exist one array as shown in matlab code. 

You can use 'object' as array type:

>>> a=numpy.array([1,2.0,2])
>>> b=numpy.array([1,2.0,2],object)
>>> a
array([ 1.,  2.,  2.])
>>> b
array([1, 2.0, 2], dtype=object)
>>> 

strangely this often leads to even faster code compared to using float/int arrays. Especially when you do much computations in Python - because with numpy you cannot really create int/float arrays, but only numpy.int32/numpy.float64 arrays. Even if you write "numpy.array([1,2.0,2],float)", you'll get item instances of another type out, which compute a lot slower and cause other problems. So currently the only way to put int's & float's into an array anyway requires to use type 'object'.

>>> import timeit
>>> timeit.Timer('a[0]+a[1]',glbls=globals()).timeit(10000)
0.069539285020861596
>>> timeit.Timer('a[0]+a[1]',glbls=globals()).timeit(10000)
0.037078734867140639
>>> timeit.Timer('a[0]+a[1]',glbls=globals()).timeit(10000)
0.038550709657232396
>>> timeit.Timer('b[0]+b[1]',glbls=globals()).timeit(10000)
0.013452827105121301
>>> timeit.Timer('b[0]+b[1]',glbls=globals()).timeit(10000)
0.01344221123075684
>>> b.sum()
5.0
>>> timeit.Timer('a.sum()',glbls=globals()).timeit(10000)
0.18172588974296389
>>> timeit.Timer('b.sum()',glbls=globals()).timeit(10000)
0.15798208990248241
>>> 


Robert



From tim.hochberg at ieee.org  Mon Nov 27 15:11:57 2006
From: tim.hochberg at ieee.org (Tim Hochberg)
Date: Mon, 27 Nov 2006 13:11:57 -0700
Subject: [Numpy-discussion] can int and float exists in one array?(about
 difference in indexing Matlab matrix and Numpy array)
In-Reply-To: <f76e92b00611270559i40143f65u4500695615bae3a7@mail.gmail.com>
References: <f76e92b00611270559i40143f65u4500695615bae3a7@mail.gmail.com>
Message-ID: <456B468D.4000001@ieee.org>

Zhang Sam wrote:
> Hi, there
>  
> I  have a practical problem. For some reason, I hope int and float can 
> exist one array as shown in matlab code. 
> ---------------------------------------------------------------------------
> >> x = [2 2.5 3.5; 1 2.6 3.5]
>  
> x =
>  
>     2.0000    2.5000    3.5000
>     1.0000    2.6000    3.5000
>  
> >> y = x(:,2)
>  
> y =
>  
>     2.5000
>     2.6000
>  
> >> y(x(:,1))
>  
> ans =
>  
>     2.6000
>     2.5000
> ------------------------------------------------------------------------------------
>  
> However in python with numpy, the similar code is as follows.
> ------------------------------------------------------------------------------------------
> x = array([[2, 2.5, 3.5],[1, 2.6, 3.5]])
> >>> x
> array([[ 2. ,  2.5,  3.5],
>        [ 1. ,  2.6,  3.5]])
> >>> y = x[:,1]
> >>> y
> array([ 2.5,  2.6])
> >>> y.shape=2,1
> >>> y
> array([[ 2.5],
>        [ 2.6]])
> >>> y[x[:,0],0]
> Traceback (most recent call last):
>   File "<interactive input>", line 1, in ?
> IndexError: arrays used as indices must be of integer (or boolean) type
> -----------------------------------------------------------------------------------------------------
>  
> MATLAB can treat the 1.0,2.00,......as int 1 2,.....
> How to realize this matlab code in python? Maybe it exist a simple 
> way. Please show me .

What are you trying to do here? Is the first column always going to 
consist of integral values? If so, I'd point you to record arrays. Using 
those you could store the first column (more or less -- you'd actually 
access is slightly differently) as integers and the other two columns as 
floats. Alternatively, you could simply store things as two separate 
arrays, one int and one float. Or is something else going on?

You could store all of your values as python objects, but I would 
recommend against it in almost all cases. Object arrays are cantankerous 
beasts, you will be chewing up a lot of memory unnecessarily and you'll 
loose any opportunities for speeding things up using numpy. It is true 
that, if you are doing the majority of your calculations in standard 
Python, it is faster at least for small arrays, to keep things in object 
arrays. However, if you are doing most of your computation that way, it 
is likely that something is badly wrong in the sense that you are not 
utilizing numpy as it's meant to be used.

-tim



From hollowspook at gmail.com  Mon Nov 27 20:13:32 2006
From: hollowspook at gmail.com (Zhang Sam)
Date: Tue, 28 Nov 2006 09:13:32 +0800
Subject: [Numpy-discussion] can int and float exists in one array?(about
	difference in indexing Matlab matrix and Numpy array)
In-Reply-To: <456B468D.4000001@ieee.org>
References: <f76e92b00611270559i40143f65u4500695615bae3a7@mail.gmail.com>
	<456B468D.4000001@ieee.org>
Message-ID: <f76e92b00611271713q2c0cc9a6ra4db965276f708cb@mail.gmail.com>

Thanks for so many replies.

In fact, I want to use several arrays to store the original data from a
practical project. In every arrays, two or three column will be store the
index. The main computation is still on matrices(float type) which is built
from the original data. When building the main matrix, I need the repeated
use of the index stored in the original data. So I hope both int and float
can exist in one array with numpy, just for the original data.

Before Python, I used matlab and fortran. In matlab, it is just I have said.
In fortran, a module can used for storing different data type. I used Python
just now, so I don't know which one in python is best for my case. What's
the suggestion?

Thans again

BTW: I will try' object' as array type today. I just know this type for my
first time.

2006/11/28, Tim Hochberg <tim.hochberg at ieee.org>:
>
> Zhang Sam wrote:
> > Hi, there
> >
> > I  have a practical problem. For some reason, I hope int and float can
> > exist one array as shown in matlab code.
> >
> ---------------------------------------------------------------------------
> > >> x = [2 2.5 3.5; 1 2.6 3.5]
> >
> > x =
> >
> >     2.0000    2.5000    3.5000
> >     1.0000    2.6000    3.5000
> >
> > >> y = x(:,2)
> >
> > y =
> >
> >     2.5000
> >     2.6000
> >
> > >> y(x(:,1))
> >
> > ans =
> >
> >     2.6000
> >     2.5000
> >
> ------------------------------------------------------------------------------------
> >
> > However in python with numpy, the similar code is as follows.
> >
> ------------------------------------------------------------------------------------------
> > x = array([[2, 2.5, 3.5],[1, 2.6, 3.5]])
> > >>> x
> > array([[ 2. ,  2.5,  3.5],
> >        [ 1. ,  2.6,  3.5]])
> > >>> y = x[:,1]
> > >>> y
> > array([ 2.5,  2.6])
> > >>> y.shape=2,1
> > >>> y
> > array([[ 2.5],
> >        [ 2.6]])
> > >>> y[x[:,0],0]
> > Traceback (most recent call last):
> >   File "<interactive input>", line 1, in ?
> > IndexError: arrays used as indices must be of integer (or boolean) type
> >
> -----------------------------------------------------------------------------------------------------
> >
> > MATLAB can treat the 1.0,2.00,......as int 1 2,.....
> > How to realize this matlab code in python? Maybe it exist a simple
> > way. Please show me .
>
> What are you trying to do here? Is the first column always going to
> consist of integral values? If so, I'd point you to record arrays. Using
> those you could store the first column (more or less -- you'd actually
> access is slightly differently) as integers and the other two columns as
> floats. Alternatively, you could simply store things as two separate
> arrays, one int and one float. Or is something else going on?
>
> You could store all of your values as python objects, but I would
> recommend against it in almost all cases. Object arrays are cantankerous
> beasts, you will be chewing up a lot of memory unnecessarily and you'll
> loose any opportunities for speeding things up using numpy. It is true
> that, if you are doing the majority of your calculations in standard
> Python, it is faster at least for small arrays, to keep things in object
> arrays. However, if you are doing most of your computation that way, it
> is likely that something is badly wrong in the sense that you are not
> utilizing numpy as it's meant to be used.
>
> -tim
>
> _______________________________________________
> Numpy-discussion mailing list
> Numpy-discussion at scipy.org
> http://projects.scipy.org/mailman/listinfo/numpy-discussion
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20061128/e1419740/attachment.html>

From tim.hochberg at ieee.org  Mon Nov 27 21:33:34 2006
From: tim.hochberg at ieee.org (Tim Hochberg)
Date: Mon, 27 Nov 2006 19:33:34 -0700
Subject: [Numpy-discussion] can int and float exists in one array?(about
 difference in indexing Matlab matrix and Numpy array)
In-Reply-To: <f76e92b00611271713q2c0cc9a6ra4db965276f708cb@mail.gmail.com>
References: <f76e92b00611270559i40143f65u4500695615bae3a7@mail.gmail.com>	<456B468D.4000001@ieee.org>
	<f76e92b00611271713q2c0cc9a6ra4db965276f708cb@mail.gmail.com>
Message-ID: <456B9FFE.1030806@ieee.org>

Zhang Sam wrote:
> Thanks for so many replies.
>  
> In fact, I want to use several arrays to store the original data from 
> a practical project. In every arrays, two or three column will be 
> store the index. The main computation is still on matrices(float type) 
> which is built from the original data. When building the main 
> matrix, I need the repeated use of the index stored in the original 
> data. So I hope both int and float can exist in one array with numpy, 
> just for the original data.
>  
> Before Python, I used matlab and fortran. In matlab, it is just I have 
> said. In fortran, a module can used for storing different data type. 
> I used Python just now, so I don't know which one in python is best 
> for my case. What's the suggestion?
>  
Here are slightly more fleshed out suggestions:

1. Break the indices out into a separate matrices. That is in instead of 
one matrix 'x' containing both the indices and and data, have two 
matrices: 'x_indices' of type int and 'x_data' of type float. This is 
probably what I would do, at least given my limited knowledge of the 
problem, since with suitable names for the two indices this is likeliest 
to be the clearest. For example:

    x_indices = np.array([2, 1]), dtype=int)
    x_data = np.array([[2.5, 3.5], [2.6, 3.5]], dtype=float)

2. Use record arrays. This allows you to pack different types into a 
single matrix, but you then need to refer to the different fields 
(formerly columns) by name:

        import numpy as np
        my_dtype = np.dtype([('indices',int), ('data_1', float),
    ('data_2', float)])
        x = np.array([(2, 2.5, 3.5), (1, 2.6, 3.5)], dtype=my_dtype)

        print x['indices']
        print x['data_1']
        print x['data_2']

Whether this makes any sense will depend on your actual use case.

3. Use object arrays as already suggested.

There are several other approaches I can think of (for example, if you 
are willing to swap rows and columns, you could create a tuple of 
arrays). However, in the absence of some compelling reason to do 
otherwise, I'd use #1.

-tim

   



From hollowspook at gmail.com  Mon Nov 27 22:51:41 2006
From: hollowspook at gmail.com (Zhang Sam)
Date: Tue, 28 Nov 2006 11:51:41 +0800
Subject: [Numpy-discussion] can int and float exists in one array?(about
	difference in indexing Matlab matrix and Numpy array)
In-Reply-To: <456B9FFE.1030806@ieee.org>
References: <f76e92b00611270559i40143f65u4500695615bae3a7@mail.gmail.com>
	<456B468D.4000001@ieee.org>
	<f76e92b00611271713q2c0cc9a6ra4db965276f708cb@mail.gmail.com>
	<456B9FFE.1030806@ieee.org>
Message-ID: <f76e92b00611271951y604c4b6bh1d058517db97c95@mail.gmail.com>

This is great. I try object array in my program just now. It can do what I
want to do.

Thanks!

2006/11/28, Tim Hochberg <tim.hochberg at ieee.org>:
>
> Zhang Sam wrote:
> > Thanks for so many replies.
> >
> > In fact, I want to use several arrays to store the original data from
> > a practical project. In every arrays, two or three column will be
> > store the index. The main computation is still on matrices(float type)
> > which is built from the original data. When building the main
> > matrix, I need the repeated use of the index stored in the original
> > data. So I hope both int and float can exist in one array with numpy,
> > just for the original data.
> >
> > Before Python, I used matlab and fortran. In matlab, it is just I have
> > said. In fortran, a module can used for storing different data type.
> > I used Python just now, so I don't know which one in python is best
> > for my case. What's the suggestion?
> >
> Here are slightly more fleshed out suggestions:
>
> 1. Break the indices out into a separate matrices. That is in instead of
> one matrix 'x' containing both the indices and and data, have two
> matrices: 'x_indices' of type int and 'x_data' of type float. This is
> probably what I would do, at least given my limited knowledge of the
> problem, since with suitable names for the two indices this is likeliest
> to be the clearest. For example:
>
>    x_indices = np.array([2, 1]), dtype=int)
>    x_data = np.array([[2.5, 3.5], [2.6, 3.5]], dtype=float)
>
> 2. Use record arrays. This allows you to pack different types into a
> single matrix, but you then need to refer to the different fields
> (formerly columns) by name:
>
>        import numpy as np
>        my_dtype = np.dtype([('indices',int), ('data_1', float),
>    ('data_2', float)])
>        x = np.array([(2, 2.5, 3.5), (1, 2.6, 3.5)], dtype=my_dtype)
>
>        print x['indices']
>        print x['data_1']
>        print x['data_2']
>
> Whether this makes any sense will depend on your actual use case.
>
> 3. Use object arrays as already suggested.
>
> There are several other approaches I can think of (for example, if you
> are willing to swap rows and columns, you could create a tuple of
> arrays). However, in the absence of some compelling reason to do
> otherwise, I'd use #1.
>
> -tim
>
>
>
> _______________________________________________
> Numpy-discussion mailing list
> Numpy-discussion at scipy.org
> http://projects.scipy.org/mailman/listinfo/numpy-discussion
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20061128/4af73113/attachment.html>

From lroubeyrie at limair.asso.fr  Tue Nov 28 04:30:37 2006
From: lroubeyrie at limair.asso.fr (Lionel Roubeyrie)
Date: Tue, 28 Nov 2006 10:30:37 +0100
Subject: [Numpy-discussion] recarray infos
Message-ID: <200611281030.37052.lroubeyrie@limair.asso.fr>

Hi all,
I'm interested by an array where I can call rows by their names and play with 
records, and recarray seems to be the solution. However, I can't find any 
complete documentation saying if recarray have the following behaviors:
 - stack datas : like numpy.array, is it possible to mak vstack, hstack, ...?
 - take several columns directly by their names, like recarray['col1','col3'] 
 - setting extra-properties, like recarray['col1'].unit='?g.m^-3'
After, numpy's doc says to use masked values for computation on arrays against 
nan values, but it seems that in recarray masked_values are not supported. 
Can I safely use nan for missing values?
Thanks for your answers

-- 
Lionel Roubeyrie - lroubeyrie at limair.asso.fr
LIMAIR
http://www.limair.asso.fr


From aarre at pair.com  Tue Nov 28 07:42:05 2006
From: aarre at pair.com (Aarre Laakso)
Date: Tue, 28 Nov 2006 07:42:05 -0500
Subject: [Numpy-discussion] var and std
Message-ID: <456C2E9D.8030903@pair.com>


Hello,

I was wondering if someone could explain the rationale for changing 
.var() and .std() in release 1.0b1 from normalizing by n-1 (unbiased 
estimate from sample) to normalizing by n (population)?

I have found the note that this change happened in the Release Notes
     http://www.scipy.org/ReleaseNotes/NumPy_1.0
and the change itself in Changeset 2560
     http://projects.scipy.org/scipy/numpy/changeset/2560
as well as a related documentation change in Ticket 388
     http://projects.scipy.org/scipy/numpy/ticket/388
but I have not been able to find a description of why the change was 
made, despite searching the website, the Trac and the mailing list.

I am aware of the argument that, if the difference between n and n-1 
matters to you, then you are "up to no good". On the other hand, this 
change breaks a lot of my unit tests. It also seems to violate the 
principle of least surprise: every other numerical environment that I 
have used divides by n-1 by default. Examples include MATLAB:
 
http://www.mathworks.com/access/helpdesk/help/techdoc/ref/index.html?/access/helpdesk/help/techdoc/ref/std.html&
http://www.mathworks.com/access/helpdesk/help/techdoc/ref/index.html?/access/helpdesk/help/techdoc/ref/var.html&
Octave:
 
http://www.gnu.org/software/octave/doc/interpreter/Basic-Statistical-Functions.html
and R:
     http://finzi.psych.upenn.edu/R/library/stats/html/cor.html
     http://finzi.psych.upenn.edu/R/library/stats/html/sd.html

It also seems to present an inconsistent interface: cov() still 
normalizes by n-1 instead of n. It also has a 'bias' parameter that 
allows normalizing by n, which is similar to the compromises provided in 
the other numerical packages listed above. As an aside, cov() also does 
not seem to be provided as a method, only as a function.

In light of all that, I am sure there must have been a good reason for 
the change, and I am very curious what it was. Thanks for any insight 
you can offer.

Regards,
Aarre



-- 
Aarre Laakso
http://www.laakshmi.com/aarre/


From kxroberto at googlemail.com  Tue Nov 28 09:27:56 2006
From: kxroberto at googlemail.com (Robert)
Date: Tue, 28 Nov 2006 15:27:56 +0100
Subject: [Numpy-discussion] numpy: azip ?
Message-ID: <ekhh1f$9rm$1@sea.gmane.org>

I often need a function to zip 2 (1D) arrays together to a 2D array - similar as python's zip() does. 
Found no function in numpy to do that directly without thinking a lot. Or is there one?
Otherwise such thing would be helpful in future numpy.

I have this in my tools:


def azip( *ll ):
    return numpy.vstack(ll).transpose()



Robert



From cjw at sympatico.ca  Tue Nov 28 09:29:38 2006
From: cjw at sympatico.ca (Colin J. Williams)
Date: Tue, 28 Nov 2006 09:29:38 -0500
Subject: [Numpy-discussion] [Fwd: python-dev Summary for 2006-10-16 through
	2006-10-31]
Message-ID: <456C47D2.2020609@sympatico.ca>

Numpy readers will likely be interested in this excerpt.

Colin W.

-------- Original Message --------
Subject: python-dev Summary for 2006-10-16 through 2006-10-31
Date: Mon, 27 Nov 2006 04:47:23 +0000 (GMT)
From: steven.bethard at gmail.com
Reply-To: python-list at python.org
Newsgroups: gmane.comp.python.announce

python-dev Summary for 2006-10-16 through 2006-10-31
++++++++++++++++++++++++++++++++++++++++++++++++++++

.. Excerpt::

[The HTML version of this Summary is available at
http://www.python.org/dev/summary/2006-10-16_2006-10-31]

=========
Summaries
=========

---------------------------------------------------------------
The buffer protocol and communicating binary format information
---------------------------------------------------------------

Travis E. Oliphant presented a pre-PEP for adding a standard way to 
describe the shape and intended types of binary-formatted data. It was 
accompanied by a pre-PEP for extending the buffer protocol to handle 
such shapes and types. Under the proposal, a new ``datatype`` object 
would describe binary-formatted data with an API like::

     datatype((float, (3,2))
     # describes a 3*2*8=48 byte block of memory that should be interpreted
     # as 6 doubles laid out as arr[0,0], arr[0,1], ... a[2,0], a[1,2]

     datatype([( ([1,2],'coords'), 'f4', (3,6)), ('address', 'S30')])
     # describes the structure
     #     float coords[3*6]   /* Has [1,2] associated with this field */
     #     char  address[30]

Alexander Belopolsky provided a nice example of why you might want to 
extend the buffer protocol along these lines. Currently, there's not 
much you can do with a basic buffer object. If you want to pass it to 
numpy_, you have to provide the type and shape information yourself::

     >>> b = buffer(array('d', [1,2,3]))
     >>> numpy.ndarray(shape=(3,), dtype=float, buffer=b)
     array([ 1.,  2.,  3.])

By extending the buffer protocol appropriately so that the necessary 
information can be provided, you should be able to pass the buffer 
directly to numpy_ and have it understand the format itself::

     >>> numpy.array(b)

People were uncomfortable with the many ``datatype`` variants -- the 
constructor accepted types, strings, lists or dicts, each of which could 
specify the structure in a different way. Also, a number of people 
questioned why the existing ``ctypes`` mechanisms for describing binary 
data couldn't be used instead, particularly since ``ctypes`` could 
already describe things like function pointers and recursive types, 
which the pre-PEP could not. Travis said he was looking for a way to 
unify the data formats of all the ``array``, ``struct``, ``numpy`` and 
``ctypes`` modules, and felt like using the ``ctypes`` approach was too 
verbose for use in the other modules. In particular, he felt like the 
``ctypes`` use of type objects as binary-format specifiers was 
problematic because type objects were harder to manipulate at the C level.

The discussion continued on into the next fortnight.

.. _numpy:


Contributing threads:

- `PEP: Adding data-type objects to Python 
<http://mail.python.org/pipermail/python-dev/2006-October/069602.html>`__
- `PEP: Extending the buffer protocol to share array information. 
<http://mail.python.org/pipermail/python-dev/2006-October/069681.html>`__



From cjw at sympatico.ca  Tue Nov 28 09:47:30 2006
From: cjw at sympatico.ca (Colin J. Williams)
Date: Tue, 28 Nov 2006 09:47:30 -0500
Subject: [Numpy-discussion] [Fwd: python-dev Summary for 2006-11-01 through
 2006-11-15] - further discussion on buffer description
Message-ID: <456C4C02.5020302@sympatico.ca>



-------- Original Message --------
Subject: python-dev Summary for 2006-11-01 through 2006-11-15
Date: Mon, 27 Nov 2006 04:52:10 +0000 (GMT)
From: steven.bethard at gmail.com
Newsgroups: gmane.comp.python.general

python-dev Summary for 2006-11-01 through 2006-11-15
++++++++++++++++++++++++++++++++++++++++++++++++++++

.. excerpt::

[The HTML version of this Summary is available at
http://www.python.org/dev/summary/2006-11-01_2006-11-15]


---------------------------------------------------------------
The buffer protocol and communicating binary format information
---------------------------------------------------------------

The discussion of extending the buffer protocol to more binary formats 
continued this fortnight. Though the PIL_ had been used as an example of 
a library that could benefit from an extended buffer protocol, Fredrik 
Lundh indicated that future versions of the PIL_ would make the binary 
data model completely opaque, and instead provide a view-style API like::

     view = object.acquire_view(region, supported formats)
     ... access data in view ...
     view.release()

Along these lines, the discussion turned away from the particular C 
formats used in ``ctypes``, ``numpy``, ``array``, etc. and more towards 
the best way to communicate format information between these modules. 
Though it seemed like people were not completely happy with the proposed 
API of the new buffer protocol, the discussion seemed to skirt around 
any concrete suggestions for better APIs.

In the end, the only thing that seemed certain was that a new buffer 
protocol could only be successful if it were implemented on all of the 
appropriate stdlib modules: ``ctypes``, ``array``, ``struct``, etc.

.. _PIL: http://www.pythonware.com/products/pil/

Contributing threads:

- `PEP: Adding data-type objects to Python 
<http://mail.python.org/pipermail/python-dev/2006-November/069706.html>`__
- `PEP: Extending the buffer protocol to share array information. 
<http://mail.python.org/pipermail/python-dev/2006-November/069709.html>`__
- `idea for data-type (data-format) PEP 
<http://mail.python.org/pipermail/python-dev/2006-November/069733.html>`__




From martin.wiechert at gmx.de  Tue Nov 28 10:59:22 2006
From: martin.wiechert at gmx.de (Martin Wiechert)
Date: Tue, 28 Nov 2006 16:59:22 +0100
Subject: [Numpy-discussion] idiom for deleting array elements
Message-ID: <200611281659.22775.martin.wiechert@gmx.de>

Hi list,

I'm looking for an idiom to delete a sparse subset S (given as integer indices 
into A) of elements from a 1d array A.

Something like

del L [I]

if L is a list and I is a non-fancy index.

My best idea is

B = ones (A.shape, bool)
B [S] = False
A = A [B]

Is there a better idiom (briefer, not requiring temporary boolean array)?

Thanks,
Martin


From stefan at sun.ac.za  Tue Nov 28 11:31:32 2006
From: stefan at sun.ac.za (Stefan van der Walt)
Date: Tue, 28 Nov 2006 18:31:32 +0200
Subject: [Numpy-discussion] idiom for deleting array elements
In-Reply-To: <200611281659.22775.martin.wiechert@gmx.de>
References: <200611281659.22775.martin.wiechert@gmx.de>
Message-ID: <20061128163131.GE9054@mentat.za.net>

On Tue, Nov 28, 2006 at 04:59:22PM +0100, Martin Wiechert wrote:
> I'm looking for an idiom to delete a sparse subset S (given as integer indices 
> into A) of elements from a 1d array A.
> 
> Something like
> 
> del L [I]
> 
> if L is a list and I is a non-fancy index.
> 
> My best idea is
> 
> B = ones (A.shape, bool)
> B [S] = False
> A = A [B]
> 
> Is there a better idiom (briefer, not requiring temporary boolean
> array)?

x = N.array([1,2,3,4,5])
N.delete(x,[2,4]) 

Cheers
St?fan


From martin.wiechert at gmx.de  Tue Nov 28 11:23:00 2006
From: martin.wiechert at gmx.de (Martin Wiechert)
Date: Tue, 28 Nov 2006 17:23:00 +0100
Subject: [Numpy-discussion] idiom for deleting array elements
In-Reply-To: <20061128163131.GE9054@mentat.za.net>
References: <200611281659.22775.martin.wiechert@gmx.de>
	<20061128163131.GE9054@mentat.za.net>
Message-ID: <200611281723.00624.martin.wiechert@gmx.de>

Perfect, thanks!

Martin


On Tuesday 28 November 2006 17:31, Stefan van der Walt wrote:
> On Tue, Nov 28, 2006 at 04:59:22PM +0100, Martin Wiechert wrote:
> > I'm looking for an idiom to delete a sparse subset S (given as integer
> > indices into A) of elements from a 1d array A.
> >
> > Something like
> >
> > del L [I]
> >
> > if L is a list and I is a non-fancy index.
> >
> > My best idea is
> >
> > B = ones (A.shape, bool)
> > B [S] = False
> > A = A [B]
> >
> > Is there a better idiom (briefer, not requiring temporary boolean
> > array)?
>
> x = N.array([1,2,3,4,5])
> N.delete(x,[2,4])
>
> Cheers
> St?fan
> _______________________________________________
> Numpy-discussion mailing list
> Numpy-discussion at scipy.org
> http://projects.scipy.org/mailman/listinfo/numpy-discussion


From robert.kern at gmail.com  Tue Nov 28 12:50:22 2006
From: robert.kern at gmail.com (Robert Kern)
Date: Tue, 28 Nov 2006 11:50:22 -0600
Subject: [Numpy-discussion] numpy: azip ?
In-Reply-To: <ekhh1f$9rm$1@sea.gmane.org>
References: <ekhh1f$9rm$1@sea.gmane.org>
Message-ID: <456C76DE.9080001@gmail.com>

Robert wrote:
> I often need a function to zip 2 (1D) arrays together to a 2D array - similar as python's zip() does. 
> Found no function in numpy to do that directly without thinking a lot. Or is there one?
> Otherwise such thing would be helpful in future numpy.

column_stack()

-- 
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless enigma
 that is made terrible by our own mad attempt to interpret it as though it had
 an underlying truth."
  -- Umberto Eco


From torgil.svensson at gmail.com  Tue Nov 28 15:22:35 2006
From: torgil.svensson at gmail.com (Torgil Svensson)
Date: Tue, 28 Nov 2006 21:22:35 +0100
Subject: [Numpy-discussion] numpy: azip ?
In-Reply-To: <456C76DE.9080001@gmail.com>
References: <ekhh1f$9rm$1@sea.gmane.org> <456C76DE.9080001@gmail.com>
Message-ID: <e7bda7770611281222n2dc14df6sffc8102ab5ba1611@mail.gmail.com>

or if you want to stack array with more dimensions

concatenate((a[...,newaxis],b[...,newaxis]),axis=-1)


On 11/28/06, Robert Kern <robert.kern at gmail.com> wrote:
> Robert wrote:
> > I often need a function to zip 2 (1D) arrays together to a 2D array - similar as python's zip() does.
> > Found no function in numpy to do that directly without thinking a lot. Or is there one?
> > Otherwise such thing would be helpful in future numpy.
>
> column_stack()
>
> --
> Robert Kern
>
> "I have come to believe that the whole world is an enigma, a harmless enigma
>  that is made terrible by our own mad attempt to interpret it as though it had
>  an underlying truth."
>   -- Umberto Eco
> _______________________________________________
> Numpy-discussion mailing list
> Numpy-discussion at scipy.org
> http://projects.scipy.org/mailman/listinfo/numpy-discussion
>


From matthew.brett at gmail.com  Tue Nov 28 17:09:31 2006
From: matthew.brett at gmail.com (Matthew Brett)
Date: Tue, 28 Nov 2006 22:09:31 +0000
Subject: [Numpy-discussion] int32 twice in sctypes?
Message-ID: <1e2af89e0611281409w6ad40a44s5f3052e37da08272@mail.gmail.com>

Hi,

I was a bit confused by this on 32 bit linux:

In [30]:sctypes['int']
Out[30]:
[<type 'numpy.int8'>,
 <type 'numpy.int16'>,
 <type 'numpy.int32'>,
 <type 'numpy.int64'>,
 <type 'numpy.int32'>]

Is it easy to explain the two entries for int32 here?  I notice there
is only one int32 entry for the same test on my 64 bit system.

Thanks a lot,

Matthew


From robert.kern at gmail.com  Tue Nov 28 18:20:53 2006
From: robert.kern at gmail.com (Robert Kern)
Date: Tue, 28 Nov 2006 17:20:53 -0600
Subject: [Numpy-discussion] int32 twice in sctypes?
In-Reply-To: <1e2af89e0611281409w6ad40a44s5f3052e37da08272@mail.gmail.com>
References: <1e2af89e0611281409w6ad40a44s5f3052e37da08272@mail.gmail.com>
Message-ID: <456CC455.8020100@gmail.com>

Matthew Brett wrote:
> Hi,
> 
> I was a bit confused by this on 32 bit linux:
> 
> In [30]:sctypes['int']
> Out[30]:
> [<type 'numpy.int8'>,
>  <type 'numpy.int16'>,
>  <type 'numpy.int32'>,
>  <type 'numpy.int64'>,
>  <type 'numpy.int32'>]
> 
> Is it easy to explain the two entries for int32 here?  I notice there
> is only one int32 entry for the same test on my 64 bit system.

Hmm. When we construct that dictionary, we put in all of the explicit integer
types (8, 16, 32, 64), and then add the "pointer"-width integer if it's not
already there. However, at least on my machine and yours, the actual scalar type
objects are different (although they represent the same information).


In [38]: import numpy

In [39]: id(numpy.dtype('p').type)
Out[39]: 62091392

In [40]: id(numpy.int32)
Out[40]: 62091200


I'm not entirely sure why this is.

-- 
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless enigma
 that is made terrible by our own mad attempt to interpret it as though it had
 an underlying truth."
  -- Umberto Eco


From matthew.brett at gmail.com  Tue Nov 28 18:57:15 2006
From: matthew.brett at gmail.com (Matthew Brett)
Date: Tue, 28 Nov 2006 23:57:15 +0000
Subject: [Numpy-discussion] dtype attributes of scalar types
Message-ID: <1e2af89e0611281557t1bc11f82rea353ee292db7a30@mail.gmail.com>

Hi,

Would it be easy and / or sensible for - say - int32.itemsize to
return the same as dtype(int32).itemsize, rather than the <attribute
'itemsize' of 'numpy.generic' objects> that it returns at the moment?

Best,

Matthew


From oliphant at ee.byu.edu  Tue Nov 28 18:59:10 2006
From: oliphant at ee.byu.edu (Travis Oliphant)
Date: Tue, 28 Nov 2006 16:59:10 -0700
Subject: [Numpy-discussion] int32 twice in sctypes?
In-Reply-To: <1e2af89e0611281409w6ad40a44s5f3052e37da08272@mail.gmail.com>
References: <1e2af89e0611281409w6ad40a44s5f3052e37da08272@mail.gmail.com>
Message-ID: <456CCD4E.5000007@ee.byu.edu>

Matthew Brett wrote:

>Hi,
>
>I was a bit confused by this on 32 bit linux:
>
>In [30]:sctypes['int']
>Out[30]:
>[<type 'numpy.int8'>,
> <type 'numpy.int16'>,
> <type 'numpy.int32'>,
> <type 'numpy.int64'>,
> <type 'numpy.int32'>]
>
>Is it easy to explain the two entries for int32 here?  I notice there
>is only one int32 entry for the same test on my 64 bit system.
>
>  
>
The mapping from c-types to bit-width types is not one-to-one.   All of 
the c-types have their own array-scalar.  Some of these have the same 
bit-width and thus are named similarly.

numpy.int32 refers to exactly one of these c-types, but on some systems 
(e.g. 32-bit), there will be another array scalar that also shows up 
with the name numpy.int32

The easiest way to see them all is to observe

id(dtype('byte').type)
id(dtype('short').type)
id(dtype('intc').type)
id(dtype('int').type)
id(dtype('longlong').type)

But then compare:

dtype('byte').type
dtype('short').type
dtype('intc').type
dtype('int').type
dtype('longlong').type

dtype('intp').type  will be one of the above as can be verified by 
looking at

id(dtype('intp').type)

The sctypes['int'] list is a list of all the c-type ints.  Thus, you 
could generate the id's of the first 5 typeobjects using:

[id(x) for x in sctypes['int']]


-Travis







From oliphant at ee.byu.edu  Tue Nov 28 19:05:46 2006
From: oliphant at ee.byu.edu (Travis Oliphant)
Date: Tue, 28 Nov 2006 17:05:46 -0700
Subject: [Numpy-discussion] can int and float exists in one array?(about
 difference in indexing Matlab matrix and Numpy array)
In-Reply-To: <f76e92b00611271713q2c0cc9a6ra4db965276f708cb@mail.gmail.com>
References: <f76e92b00611270559i40143f65u4500695615bae3a7@mail.gmail.com>	<456B468D.4000001@ieee.org>
	<f76e92b00611271713q2c0cc9a6ra4db965276f708cb@mail.gmail.com>
Message-ID: <456CCEDA.5060207@ee.byu.edu>

Zhang Sam wrote:

> Thanks for so many replies.
>  
> In fact, I want to use several arrays to store the original data from 
> a practical project. In every arrays, two or three column will be 
> store the index. The main computation is still on matrices(float type) 
> which is built from the original data. When building the main 
> matrix, I need the repeated use of the index stored in the original 
> data. So I hope both int and float can exist in one array with numpy, 
> just for the original data.

I think you probably want to use a record-array here.

Matlab does not store both integers and floating point in the same 
matrix.   It stores integers *as* floating point in your float array.  
Matlab just allows floating-point arrays to be used as indexes (because 
for a long time there was only floating-point matrices in Matlab).  

NumPy forces you to be explicit in using an integer array for indexing 
into an array.    You are mis-interpreting the situation with Matlab.  
The accurate representation is that Matlab allows indexing with floating 
point arrays.  It would be possible to lift the restriction with NumPy, 
but I suspect it's probably best to leave it the way it is so that users 
are clear about how they want to convert from integers to floating-point.


-Travis




From oliphant at ee.byu.edu  Tue Nov 28 19:09:01 2006
From: oliphant at ee.byu.edu (Travis Oliphant)
Date: Tue, 28 Nov 2006 17:09:01 -0700
Subject: [Numpy-discussion] ScientificPython with NumPy
In-Reply-To: <17BB897A-F8ED-4BE8-B7CA-C6A1AEE016C7@laposte.net>
References: <0472E758-6309-4D18-8456-EDE575B33B48@laposte.net>
	<17BB897A-F8ED-4BE8-B7CA-C6A1AEE016C7@laposte.net>
Message-ID: <456CCF9D.2040403@ee.byu.edu>

konrad.hinsen at laposte.net wrote:

>On 24.11.2006, at 19:04, konrad.hinsen at laposte.net wrote:
>
>  
>
>>Those who would like to test-drive ScientificPython with NumPy can do
>>so now: just download version 2.7.1 from
>>
>>	http://sourcesup.cru.fr/
>>    
>>
>
>Version 2.7.2 now, after the first bug fix: it wouldn't compile with  
>NumPy and Python < 2.5, because both NumPy and ScientificPython  
>define Py_ssize_t. There doesn't seem to be a nice solution for that  
>problem, unfortunately.
>  
>

You can

#define PY_SSIZE_T_MIN INT_MIN
#define PY_SSIZE_T_MAX INT_MAX

before including the NumPy header file and it will by-pass the typedef 
for Py_ssize_t.

-Travis



From oliphant at ee.byu.edu  Tue Nov 28 19:11:57 2006
From: oliphant at ee.byu.edu (Travis Oliphant)
Date: Tue, 28 Nov 2006 17:11:57 -0700
Subject: [Numpy-discussion] dtype attributes of scalar types
In-Reply-To: <1e2af89e0611281557t1bc11f82rea353ee292db7a30@mail.gmail.com>
References: <1e2af89e0611281557t1bc11f82rea353ee292db7a30@mail.gmail.com>
Message-ID: <456CD04D.7040501@ee.byu.edu>

Matthew Brett wrote:

>Hi,
>
>Would it be easy and / or sensible for - say - int32.itemsize to
>return the same as dtype(int32).itemsize, rather than the <attribute
>'itemsize' of 'numpy.generic' objects> that it returns at the moment?
>
>  
>
You can do int32(0).itemsize.

Getting the itemsize attribute to "do the right thing" when called from 
the type-object (instead of an instance of the type-object) is beyond my 
current skill.  It might be possible, but I haven't spent the time to 
figure it out.

-Travis



From matthew.brett at gmail.com  Tue Nov 28 20:30:36 2006
From: matthew.brett at gmail.com (Matthew Brett)
Date: Wed, 29 Nov 2006 01:30:36 +0000
Subject: [Numpy-discussion] atol in allclose
Message-ID: <1e2af89e0611281730k492a2929j60ac9e54b5325790@mail.gmail.com>

Hi,

Sorry to keep cluttering the list, but I was a bit surprised by this
behavior of allclose:

In [25]:allclose([1.0], [1.0], rtol=0)
Out[25]:True

In [26]:allclose([1.0], [1.0], rtol=0, atol=0)
Out[26]:False

The docstring seems to imply that atol will not be used in this
comparison - or did I misunderstand?

Thanks,

Matthew


From robert.kern at gmail.com  Tue Nov 28 20:44:49 2006
From: robert.kern at gmail.com (Robert Kern)
Date: Tue, 28 Nov 2006 19:44:49 -0600
Subject: [Numpy-discussion] atol in allclose
In-Reply-To: <1e2af89e0611281730k492a2929j60ac9e54b5325790@mail.gmail.com>
References: <1e2af89e0611281730k492a2929j60ac9e54b5325790@mail.gmail.com>
Message-ID: <456CE611.7010205@gmail.com>

Matthew Brett wrote:
> Hi,
> 
> Sorry to keep cluttering the list, but I was a bit surprised by this
> behavior of allclose:
> 
> In [25]:allclose([1.0], [1.0], rtol=0)
> Out[25]:True
> 
> In [26]:allclose([1.0], [1.0], rtol=0, atol=0)
> Out[26]:False
> 
> The docstring seems to imply that atol will not be used in this
> comparison - or did I misunderstand?

The docstring is a somewhat loose interpretation of the actual implementation.
You've exposed a corner case. The long and the short of it is this:

    d = less(absolute(x-y), atol + rtol * absolute(y))

rtol was never intended to be less than the epsilon of your chosen floating
point type much less 0. That's what == is for.

That said, less() should probably be less_equal(), and the docstring can be
clearly describe what is going on.

-- 
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless enigma
 that is made terrible by our own mad attempt to interpret it as though it had
 an underlying truth."
  -- Umberto Eco


From charlesr.harris at gmail.com  Tue Nov 28 22:36:06 2006
From: charlesr.harris at gmail.com (Charles R Harris)
Date: Tue, 28 Nov 2006 20:36:06 -0700
Subject: [Numpy-discussion] atol in allclose
In-Reply-To: <456CE611.7010205@gmail.com>
References: <1e2af89e0611281730k492a2929j60ac9e54b5325790@mail.gmail.com>
	<456CE611.7010205@gmail.com>
Message-ID: <e06186140611281936v605ede15qdd60e384836f782f@mail.gmail.com>

On 11/28/06, Robert Kern <robert.kern at gmail.com> wrote:
>
> Matthew Brett wrote:
> > Hi,
> >
> > Sorry to keep cluttering the list, but I was a bit surprised by this
> > behavior of allclose:
> >
> > In [25]:allclose([1.0], [1.0], rtol=0)
> > Out[25]:True
> >
> > In [26]:allclose([1.0], [1.0], rtol=0, atol=0)
> > Out[26]:False
> >
> > The docstring seems to imply that atol will not be used in this
> > comparison - or did I misunderstand?
>
> The docstring is a somewhat loose interpretation of the actual
> implementation.
> You've exposed a corner case. The long and the short of it is this:
>
>     d = less(absolute(x-y), atol + rtol * absolute(y))
>
> rtol was never intended to be less than the epsilon of your chosen
> floating
> point type much less 0. That's what == is for.
>
> That said, less() should probably be less_equal(), and the docstring can
> be
> clearly describe what is going on.


I think that's right. Do you want to make the change?

Chuck
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20061128/c985209d/attachment.html>

From lroubeyrie at limair.asso.fr  Wed Nov 29 06:28:50 2006
From: lroubeyrie at limair.asso.fr (Lionel Roubeyrie)
Date: Wed, 29 Nov 2006 12:28:50 +0100
Subject: [Numpy-discussion] array with object
Message-ID: <200611291228.50476.lroubeyrie@limair.asso.fr>

Hi all,
I don't understand why can't I do that:
b=array([(datetime.datetime(2006,11,29),'2','3'),
(datetime.datetime(2006,11,30),'5','6')], dtype=[('Dates', 'object'),
('HUM', 'float32'), ('TEM', 'float32')])
ValueError: tried to set void-array with object members using buffer.
Thanks

-- 
Lionel Roubeyrie - lroubeyrie at limair.asso.fr
LIMAIR
http://www.limair.asso.fr


From oliphant at ee.byu.edu  Wed Nov 29 11:28:52 2006
From: oliphant at ee.byu.edu (Travis Oliphant)
Date: Wed, 29 Nov 2006 09:28:52 -0700
Subject: [Numpy-discussion] fromiter and objects
In-Reply-To: <45609C15.7050602@ieee.org>
References: <45609C15.7050602@ieee.org>
Message-ID: <456DB544.7080403@ee.byu.edu>

Tim Hochberg wrote:

>things are often not as easy as they appear, the real reason I'm writing 
>is this comment that explains why object arrays are disallowed in 
>fromiter (multiarraymodule.c::PyArray_:FromIter)
>
>            /* We would need to alter the memory RENEW code to decrement any
>               reference counts before just throwing away the memory.
>             */
>
>This doesn't seem right. The array that we would be RENEWing is a bunch 
>of PyObject*s. The reference counts don't reside there, but in the 
>objects themselves. When we do the RENEW, we don't want the reference 
>counts to change at all. 
>
I'm not sure if you are still thinking about this.  I suspect I added 
the comment because I was concerned about two things:

1) a resize that deletes memory without first decrefing object counts 
--- I suspose this can't happen here, though, because RENEW is always 
"growing" memory, right?

2) a resize that grows memory without inserting NULLS.  For Object array 
manipulation to work, you need initialized (to NULL) "empty" memory. 

>The one tricky case is if we run out of memory, 
>I'm not certain that the current setup correctly deals with reference 
>counts in this case, although it appears likely that it should work 
>since ret->data should still point to a valid chunk of memory and 
>decreffing ret should result in the subsequent deallocation of all the 
>  
>
>stored objects.
>  
>
As long as you don't store the pointer to new memory until after the 
RENEW was successful then this should work.

I think my comment came from the possibility of #1 in general use of 
RENEW, but I don't think that is going to happen.  Therefore, the only 
tweak that seams necessary is to initialize any new memory to NULL for 
OBJECT arrays.

-Travis





From humufr at yahoo.fr  Wed Nov 29 14:41:01 2006
From: humufr at yahoo.fr (humufr at yahoo.fr)
Date: Wed, 29 Nov 2006 14:41:01 -0500
Subject: [Numpy-discussion] dtype inconsistancy?
Message-ID: <200611291441.02717.humufr@yahoo.fr>

Hi,

I think that the following command and result are a little bit inconsistant 
and a source of error. I don't understand why: dtype=float32 for a result 
when it's invalid in the definition.

N. 



In [8]: a = numpy.array([1],dtype=float32)
---------------------------------------------------------------------------
exceptions.NameError                                 Traceback (most recent 
call last)

NameError: name 'float32' is not defined

In [9]: a = numpy.array([1],dtype='float32')

In [10]: a
Out[10]: array([ 1.], dtype=float32)

In [11]: numpy.array([ 1.], dtype=float32)
---------------------------------------------------------------------------
exceptions.NameError                                 Traceback (most recent 
call last)

NameError: name 'float32' is not defined



From robert.kern at gmail.com  Wed Nov 29 14:48:27 2006
From: robert.kern at gmail.com (Robert Kern)
Date: Wed, 29 Nov 2006 13:48:27 -0600
Subject: [Numpy-discussion] dtype inconsistancy?
In-Reply-To: <200611291441.02717.humufr@yahoo.fr>
References: <200611291441.02717.humufr@yahoo.fr>
Message-ID: <456DE40B.1030903@gmail.com>

humufr at yahoo.fr wrote:
> Hi,
> 
> I think that the following command and result are a little bit inconsistant 
> and a source of error. I don't understand why: dtype=float32 for a result 
> when it's invalid in the definition.

float32 is an object in the numpy module, i.e. to access it, you must import it
from there.

  import numpy
  a = numpy.array([1], dtype=numpy.float32)

The representation of an object (i.e. what you see in Out[10]) is not
necessarily directly executable in all possible contexts. In fact, it *can't* be
executable in all possible contexts. Note that you already did an interpretation
of the representation: you used "numpy.array()" instead of just "array()".


In [1]: import numpy

In [2]: a = numpy.array([1], dtype=numpy.float32)

In [3]: a
Out[3]: array([ 1.], dtype=float32)

In [4]: array([ 1.], dtype=float32)
---------------------------------------------------------------------------
<type 'exceptions.NameError'>             Traceback (most recent call last)

/Users/rkern/src/wsgiref/<ipython console> in <module>()

<type 'exceptions.NameError'>: name 'array' is not defined


-- 
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless enigma
 that is made terrible by our own mad attempt to interpret it as though it had
 an underlying truth."
  -- Umberto Eco


From pgmdevlist at gmail.com  Wed Nov 29 14:52:53 2006
From: pgmdevlist at gmail.com (Pierre GM)
Date: Wed, 29 Nov 2006 14:52:53 -0500
Subject: [Numpy-discussion] dtype inconsistancy?
In-Reply-To: <200611291441.02717.humufr@yahoo.fr>
References: <200611291441.02717.humufr@yahoo.fr>
Message-ID: <200611291452.53999.pgmdevlist@gmail.com>

On Wednesday 29 November 2006 14:41, humufr at yahoo.fr wrote:
> Hi,
>
> I think that the following command and result are a little bit inconsistant
> and a source of error. I don't understand why: dtype=float32 for a result
> when it's invalid in the definition.

I'm afraid I don;t understand the last sentence...
But anyway:
float32 is defined by numpy. If you just used 
>>>import numpy
then
>>>numpy.array([ 1.], dtype=numpy.float32)
will work.
Using 'float32' as a string, you let the dtype function guess what you mean.


From Chris.Barker at noaa.gov  Wed Nov 29 15:04:45 2006
From: Chris.Barker at noaa.gov (Christopher Barker)
Date: Wed, 29 Nov 2006 12:04:45 -0800
Subject: [Numpy-discussion] array with object
In-Reply-To: <200611291228.50476.lroubeyrie@limair.asso.fr>
References: <200611291228.50476.lroubeyrie@limair.asso.fr>
Message-ID: <456DE7DD.8010608@noaa.gov>

Lionel Roubeyrie wrote:
> Hi all,
> I don't understand why can't I do that:
> b=array([(datetime.datetime(2006,11,29),'2','3'),
> (datetime.datetime(2006,11,30),'5','6')], dtype=[('Dates', 'object'),
> ('HUM', 'float32'), ('TEM', 'float32')])
> ValueError: tried to set void-array with object members using buffer.
> Thanks

I'm not sure why this fails, but it does work if you create an empty 
array, then set the values individually:

 >>> a = N.empty((2,),dtype = [('Dates', 'object'),('HUM', 'float32'), 
('TEM', 'float32')])
 >>> a
array([(None, 0.0, 0.0), (None, 0.0, 0.0)],
       dtype=[('Dates', '|O4'), ('HUM', '>f4'), ('TEM', '>f4')])

 >>> a[0]['Dates'] = datetime.datetime(2006,11,29)
 >>> a[0]['HUM'] = 2
 >>> a[0]['TEM'] = 3

 >>> a
array([(datetime.datetime(2006, 11, 29, 0, 0), 2.0, 3.0), (None, 0.0, 
0.0)], dtype=[('Dates', '|O4'), ('HUM', '>f4'), ('TEM', '>f4')])

Both recarrays and working the object data type are tricky.

-Chris


-- 
Christopher Barker, Ph.D.
Oceanographer

Emergency Response Division
NOAA/NOS/OR&R            (206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115       (206) 526-6317   main reception

Chris.Barker at noaa.gov


From torgil.svensson at gmail.com  Wed Nov 29 16:11:35 2006
From: torgil.svensson at gmail.com (Torgil Svensson)
Date: Wed, 29 Nov 2006 22:11:35 +0100
Subject: [Numpy-discussion] array with object
In-Reply-To: <200611291228.50476.lroubeyrie@limair.asso.fr>
References: <200611291228.50476.lroubeyrie@limair.asso.fr>
Message-ID: <e7bda7770611291311k3a08c29by110494c1caf0e5f@mail.gmail.com>

It works if you replace the strings '2','3','5','6' with numbers instead.

This case got a better error-message:

>>> array(['2'],dtype=float32)
Traceback (most recent call last):
  File "<stdin>", line 1, in ?
TypeError: a float is required

//Torgil


On 11/29/06, Lionel Roubeyrie <lroubeyrie at limair.asso.fr> wrote:
> Hi all,
> I don't understand why can't I do that:
> b=array([(datetime.datetime(2006,11,29),'2','3'),
> (datetime.datetime(2006,11,30),'5','6')], dtype=[('Dates', 'object'),
> ('HUM', 'float32'), ('TEM', 'float32')])
> ValueError: tried to set void-array with object members using buffer.
> Thanks
>
> --
> Lionel Roubeyrie - lroubeyrie at limair.asso.fr
> LIMAIR
> http://www.limair.asso.fr
> _______________________________________________
> Numpy-discussion mailing list
> Numpy-discussion at scipy.org
> http://projects.scipy.org/mailman/listinfo/numpy-discussion
>


From Chris.Barker at noaa.gov  Wed Nov 29 17:43:17 2006
From: Chris.Barker at noaa.gov (Christopher Barker)
Date: Wed, 29 Nov 2006 14:43:17 -0800
Subject: [Numpy-discussion] array with object
In-Reply-To: <e7bda7770611291311k3a08c29by110494c1caf0e5f@mail.gmail.com>
References: <200611291228.50476.lroubeyrie@limair.asso.fr>
	<e7bda7770611291311k3a08c29by110494c1caf0e5f@mail.gmail.com>
Message-ID: <456E0D05.1040409@noaa.gov>



Torgil Svensson wrote:
> It works if you replace the strings '2','3','5','6' with numbers instead.

duh! I can't believe I didn't see that!

-Chris

-- 
Christopher Barker, Ph.D.
Oceanographer

Emergency Response Division
NOAA/NOS/OR&R            (206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115       (206) 526-6317   main reception

Chris.Barker at noaa.gov


From myeates at jpl.nasa.gov  Wed Nov 29 19:44:35 2006
From: myeates at jpl.nasa.gov (Mathew Yeates)
Date: Wed, 29 Nov 2006 16:44:35 -0800
Subject: [Numpy-discussion] insanely slow writing to memory mapped array
Message-ID: <456E2973.5030706@jpl.nasa.gov>

Hi

I have a line in my program that looks like
outarr[1,:] = computed_array
where outarr is a memory mapped file. This takes forever.

I checked and copying  the data using "cp"  at the command line takes 1 
or 2 seconds. So the problem can't be attributed simply to disk i/o. Is 
it because the elements are being written one at a time? Any ideas on 
how to speed this up?

Thanks
Mathew




From robert.kern at gmail.com  Wed Nov 29 19:51:40 2006
From: robert.kern at gmail.com (Robert Kern)
Date: Wed, 29 Nov 2006 18:51:40 -0600
Subject: [Numpy-discussion] insanely slow writing to memory mapped array
In-Reply-To: <456E2973.5030706@jpl.nasa.gov>
References: <456E2973.5030706@jpl.nasa.gov>
Message-ID: <456E2B1C.8090003@gmail.com>

Mathew Yeates wrote:
> Hi
> 
> I have a line in my program that looks like
> outarr[1,:] = computed_array
> where outarr is a memory mapped file. This takes forever.
> 
> I checked and copying  the data using "cp"  at the command line takes 1 
> or 2 seconds. So the problem can't be attributed simply to disk i/o. Is 
> it because the elements are being written one at a time? Any ideas on 
> how to speed this up?

Memory-mapping is highly platform dependent. What platform are you on? What are
the sizes of the arrays? Can you write up a small, self-contained script that
demonstrates the issue so we can experiment and try things out on different
machines?

-- 
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless enigma
 that is made terrible by our own mad attempt to interpret it as though it had
 an underlying truth."
  -- Umberto Eco


From myeates at jpl.nasa.gov  Wed Nov 29 20:42:24 2006
From: myeates at jpl.nasa.gov (Mathew Yeates)
Date: Wed, 29 Nov 2006 17:42:24 -0800
Subject: [Numpy-discussion] insanely slow writing to memory mapped array
In-Reply-To: <456E2B1C.8090003@gmail.com>
References: <456E2973.5030706@jpl.nasa.gov> <456E2B1C.8090003@gmail.com>
Message-ID: <456E3700.8020806@jpl.nasa.gov>

Hmm
I'm trying to duplicate the behavior with a simple program
---------
import numpy
datasize=5529000
numrows=121

fd=open("biggie","w")
fd.close()
big=numpy.memmap("biggie",mode="readwrite", 
shape=(numrows,datasize),dtype=numpy.float32)

c=numpy.ones(shape=(datasize,),dtype=numpy.float32)
for r in range(0,numrows):
        print r
        big[r,:] = c
        c[r] = 2.0
---------------------
but it is fast. Hmmm. Any ideas about where to go from here?
Mathew



Robert Kern wrote:
> Mathew Yeates wrote:
>   
>> Hi
>>
>> I have a line in my program that looks like
>> outarr[1,:] = computed_array
>> where outarr is a memory mapped file. This takes forever.
>>
>> I checked and copying  the data using "cp"  at the command line takes 1 
>> or 2 seconds. So the problem can't be attributed simply to disk i/o. Is 
>> it because the elements are being written one at a time? Any ideas on 
>> how to speed this up?
>>     
>
> Memory-mapping is highly platform dependent. What platform are you on? What are
> the sizes of the arrays? Can you write up a small, self-contained script that
> demonstrates the issue so we can experiment and try things out on different
> machines?
>
>   




From myeates at jpl.nasa.gov  Wed Nov 29 20:52:40 2006
From: myeates at jpl.nasa.gov (Mathew Yeates)
Date: Wed, 29 Nov 2006 17:52:40 -0800
Subject: [Numpy-discussion] insanely slow writing to memory mapped array
In-Reply-To: <456E3700.8020806@jpl.nasa.gov>
References: <456E2973.5030706@jpl.nasa.gov> <456E2B1C.8090003@gmail.com>
	<456E3700.8020806@jpl.nasa.gov>
Message-ID: <456E3968.3060005@jpl.nasa.gov>

whoa. I just found out that A=A.transpose() does nothing but change A's 
flags from C_CONTIGUOUS to F_CONTIGUOUS!!

Okay, so heres the question ...... I am reading data into the columns of 
a matrix. In order to speed this up, I want to read values into the rows 
of a matrix and when I am all done, do a transpose. Whats the best way?

Mathew



Mathew Yeates wrote:
> Hmm
> I'm trying to duplicate the behavior with a simple program
> ---------
> import numpy
> datasize=5529000
> numrows=121
>
> fd=open("biggie","w")
> fd.close()
> big=numpy.memmap("biggie",mode="readwrite", 
> shape=(numrows,datasize),dtype=numpy.float32)
>
> c=numpy.ones(shape=(datasize,),dtype=numpy.float32)
> for r in range(0,numrows):
>         print r
>         big[r,:] = c
>         c[r] = 2.0
> ---------------------
> but it is fast. Hmmm. Any ideas about where to go from here?
> Mathew
>
>
>
> Robert Kern wrote:
>   
>> Mathew Yeates wrote:
>>   
>>     
>>> Hi
>>>
>>> I have a line in my program that looks like
>>> outarr[1,:] = computed_array
>>> where outarr is a memory mapped file. This takes forever.
>>>
>>> I checked and copying  the data using "cp"  at the command line takes 1 
>>> or 2 seconds. So the problem can't be attributed simply to disk i/o. Is 
>>> it because the elements are being written one at a time? Any ideas on 
>>> how to speed this up?
>>>     
>>>       
>> Memory-mapping is highly platform dependent. What platform are you on? What are
>> the sizes of the arrays? Can you write up a small, self-contained script that
>> demonstrates the issue so we can experiment and try things out on different
>> machines?
>>
>>   
>>     
>
>
> _______________________________________________
> Numpy-discussion mailing list
> Numpy-discussion at scipy.org
> http://projects.scipy.org/mailman/listinfo/numpy-discussion
>
>   




From charlesr.harris at gmail.com  Wed Nov 29 22:18:15 2006
From: charlesr.harris at gmail.com (Charles R Harris)
Date: Wed, 29 Nov 2006 20:18:15 -0700
Subject: [Numpy-discussion] insanely slow writing to memory mapped array
In-Reply-To: <456E3968.3060005@jpl.nasa.gov>
References: <456E2973.5030706@jpl.nasa.gov> <456E2B1C.8090003@gmail.com>
	<456E3700.8020806@jpl.nasa.gov> <456E3968.3060005@jpl.nasa.gov>
Message-ID: <e06186140611291918i45edad91qc8bf40715c7566f7@mail.gmail.com>

On 11/29/06, Mathew Yeates <myeates at jpl.nasa.gov> wrote:
>
> whoa. I just found out that A=A.transpose() does nothing but change A's
> flags from C_CONTIGUOUS to F_CONTIGUOUS!!
>
> Okay, so heres the question ...... I am reading data into the columns of
> a matrix. In order to speed this up, I want to read values into the rows
> of a matrix and when I am all done, do a transpose. Whats the best way?


If you want a contiguous copy

In [13]: a
Out[13]:
array([[0, 1, 2, 3, 4],
       [5, 6, 7, 8, 9]])

In [14]: b=a.transpose().copy()

In [15]: a.flags
Out[15]:
  C_CONTIGUOUS : True
  F_CONTIGUOUS : False
  OWNDATA : False
  WRITEABLE : True
  ALIGNED : True
  UPDATEIFCOPY : False

In [16]: b.flags
Out[16]:
  C_CONTIGUOUS : True
  F_CONTIGUOUS : False
  OWNDATA : True
  WRITEABLE : True
  ALIGNED : True
  UPDATEIFCOPY : False

The result isn't memory mapped, however. What exactly are you trying to do?

Chuck
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20061129/11aca928/attachment.html>

From charlesr.harris at gmail.com  Wed Nov 29 22:24:25 2006
From: charlesr.harris at gmail.com (Charles R Harris)
Date: Wed, 29 Nov 2006 20:24:25 -0700
Subject: [Numpy-discussion] insanely slow writing to memory mapped array
In-Reply-To: <456E3968.3060005@jpl.nasa.gov>
References: <456E2973.5030706@jpl.nasa.gov> <456E2B1C.8090003@gmail.com>
	<456E3700.8020806@jpl.nasa.gov> <456E3968.3060005@jpl.nasa.gov>
Message-ID: <e06186140611291924y153d85c9m6f00b16f5367ca82@mail.gmail.com>

On 11/29/06, Mathew Yeates <myeates at jpl.nasa.gov> wrote:
>
> whoa. I just found out that A=A.transpose() does nothing but change A's
> flags from C_CONTIGUOUS to F_CONTIGUOUS!!
>
> Okay, so heres the question ...... I am reading data into the columns of
> a matrix. In order to speed this up, I want to read values into the rows
> of a matrix and when I am all done, do a transpose. Whats the best way?


And if you just want to read in the data from the file in one big contiquous
binary chunk:

>>> a = fromfile("foo.data", dtype=whatever).reshape(m,n)

Then you can make a transposed copy of a. Or even a transposed view,
depending on how you want to use it.

Chuck
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20061129/5b61d5f5/attachment.html>

From lroubeyrie at limair.asso.fr  Thu Nov 30 03:39:15 2006
From: lroubeyrie at limair.asso.fr (Lionel Roubeyrie)
Date: Thu, 30 Nov 2006 09:39:15 +0100
Subject: [Numpy-discussion] array with object
In-Reply-To: <e7bda7770611291311k3a08c29by110494c1caf0e5f@mail.gmail.com>
References: <200611291228.50476.lroubeyrie@limair.asso.fr>
	<e7bda7770611291311k3a08c29by110494c1caf0e5f@mail.gmail.com>
Message-ID: <200611300939.15607.lroubeyrie@limair.asso.fr>

Hi Torgil,
strange, I don't have the same error:
########################################
|~|[2]>array(['2'],dtype=float32)
---------------------------------------------------------------------------
exceptions.ValueError                                Traceback (most recent 
call last)
/home/lionel/<ipython console>
ValueError: setting an array element with a sequence.
########################################

I don't understand why there is not an automatic casting, because using astype 
works:
########################################
|~|[3]>a=array(['2'])
|~|[4]>a
Out [4]:
array(['2'], dtype='|S1')
|~|[5]>a.astype(float32)
Out [5]:array([ 2.], dtype=float32)
########################################

Another strange thing, if I use floats instead of strings values, but set them 
in a list of list instead of a list of tuple, I get the same error as in my 
previous post:
########################################
|~|[7]>b=array([(datetime.datetime(2006,11,29),2,3),
(datetime.datetime(2006,11,30),5,6)], dtype=[('Dates', 'object'),
('HUM', 'float32'), ('TEM', 'float32')])
|~|[8]>b
Out [8]:
array([(datetime.datetime(2006, 11, 29, 0, 0), 2.0, 3.0),
       (datetime.datetime(2006, 11, 30, 0, 0), 5.0, 6.0)],
      dtype=[('Dates', '|O4'), ('HUM', '<f4'), ('TEM', '<f4')])
|~|[9]>b=array([[datetime.datetime(2006,11,29),2,3],
[datetime.datetime(2006,11,30),5,6]], dtype=[('Dates', 'object'),
('HUM', 'float32'),('TEM', 'float32')])
|--------------------------------------------------------------------------- 
exceptions.ValueError                                Traceback (most recent 
call last)
/home/lionel/<ipython console>
ValueError: tried to set void-array with object members using buffer.
########################################

Is it expected ?


Le Mercredi 29 Novembre 2006 22:11, Torgil Svensson a ?crit?:
> It works if you replace the strings '2','3','5','6' with numbers instead.
>
> This case got a better error-message:
> >>> array(['2'],dtype=float32)
>
> Traceback (most recent call last):
>   File "<stdin>", line 1, in ?
> TypeError: a float is required
>
> //Torgil
>
> On 11/29/06, Lionel Roubeyrie <lroubeyrie at limair.asso.fr> wrote:
> > Hi all,
> > I don't understand why can't I do that:
> > b=array([(datetime.datetime(2006,11,29),'2','3'),
> > (datetime.datetime(2006,11,30),'5','6')], dtype=[('Dates', 'object'),
> > ('HUM', 'float32'), ('TEM', 'float32')])
> > ValueError: tried to set void-array with object members using buffer.
> > Thanks
> >
> > --
> > Lionel Roubeyrie - lroubeyrie at limair.asso.fr
> > LIMAIR
> > http://www.limair.asso.fr
> > _______________________________________________
> > Numpy-discussion mailing list
> > Numpy-discussion at scipy.org
> > http://projects.scipy.org/mailman/listinfo/numpy-discussion
>
> _______________________________________________
> Numpy-discussion mailing list
> Numpy-discussion at scipy.org
> http://projects.scipy.org/mailman/listinfo/numpy-discussion

-- 
Lionel Roubeyrie - lroubeyrie at limair.asso.fr
LIMAIR
http://www.limair.asso.fr


From robert.kern at gmail.com  Thu Nov 30 03:57:17 2006
From: robert.kern at gmail.com (Robert Kern)
Date: Thu, 30 Nov 2006 02:57:17 -0600
Subject: [Numpy-discussion] array with object
In-Reply-To: <200611300939.15607.lroubeyrie@limair.asso.fr>
References: <200611291228.50476.lroubeyrie@limair.asso.fr>	<e7bda7770611291311k3a08c29by110494c1caf0e5f@mail.gmail.com>
	<200611300939.15607.lroubeyrie@limair.asso.fr>
Message-ID: <456E9CED.1000702@gmail.com>

Lionel Roubeyrie wrote:
> Hi Torgil,
> strange, I don't have the same error:
> ########################################
> |~|[2]>array(['2'],dtype=float32)
> ---------------------------------------------------------------------------
> exceptions.ValueError                                Traceback (most recent 
> call last)
> /home/lionel/<ipython console>
> ValueError: setting an array element with a sequence.
> ########################################
> 
> I don't understand why there is not an automatic casting, because using astype 
> works:
> ########################################
> |~|[3]>a=array(['2'])
> |~|[4]>a
> Out [4]:
> array(['2'], dtype='|S1')
> |~|[5]>a.astype(float32)
> Out [5]:array([ 2.], dtype=float32)
> ########################################

The latter has a very clear job: take an array of known shape and type and
convert it to a different type. The array() constructor has to divine the shape,
too, with a possibly weird nesting of sequences. Help the poor thing out, and
always be explicit.

> Another strange thing, if I use floats instead of strings values, but set them 
> in a list of list instead of a list of tuple, I get the same error as in my 
> previous post:
> ########################################
> |~|[7]>b=array([(datetime.datetime(2006,11,29),2,3),
> (datetime.datetime(2006,11,30),5,6)], dtype=[('Dates', 'object'),
> ('HUM', 'float32'), ('TEM', 'float32')])
> |~|[8]>b
> Out [8]:
> array([(datetime.datetime(2006, 11, 29, 0, 0), 2.0, 3.0),
>        (datetime.datetime(2006, 11, 30, 0, 0), 5.0, 6.0)],
>       dtype=[('Dates', '|O4'), ('HUM', '<f4'), ('TEM', '<f4')])
> |~|[9]>b=array([[datetime.datetime(2006,11,29),2,3],
> [datetime.datetime(2006,11,30),5,6]], dtype=[('Dates', 'object'),
> ('HUM', 'float32'),('TEM', 'float32')])
> |--------------------------------------------------------------------------- 
> exceptions.ValueError                                Traceback (most recent 
> call last)
> /home/lionel/<ipython console>
> ValueError: tried to set void-array with object members using buffer.
> ########################################
> 
> Is it expected ?

Yes. When object arrays (or record arrays with object columns) are involved, it
is extremely difficult for array() to know what you want to be in the array and
what is simply "structure". Remember that array() is trying to divine both the
shape of the array and the contents at the same time. The mind-reading feature
has not yet been implemented. In its stead, an arbitrary choice was made: tuples
are intended to be elements of record arrays; lists are "structure".

-- 
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless enigma
 that is made terrible by our own mad attempt to interpret it as though it had
 an underlying truth."
  -- Umberto Eco


From konrad.hinsen at laposte.net  Thu Nov 30 04:44:22 2006
From: konrad.hinsen at laposte.net (Konrad Hinsen)
Date: Thu, 30 Nov 2006 10:44:22 +0100
Subject: [Numpy-discussion] ScientificPython with NumPy
In-Reply-To: <456CCF9D.2040403@ee.byu.edu>
References: <0472E758-6309-4D18-8456-EDE575B33B48@laposte.net>
	<17BB897A-F8ED-4BE8-B7CA-C6A1AEE016C7@laposte.net>
	<456CCF9D.2040403@ee.byu.edu>
Message-ID: <75EE2DD2-4FF3-4BC5-8EC9-D31AB3A9AA67@laposte.net>

On Nov 29, 2006, at 1:09, Travis Oliphant wrote:

> You can
>
> #define PY_SSIZE_T_MIN INT_MIN
> #define PY_SSIZE_T_MAX INT_MAX
>
> before including the NumPy header file and it will by-pass the typedef
> for Py_ssize_t.

Interesting. Is that a test that would work for distinguishing Python  
2.5 from its predecessors as well? In other words, is that a test  
that one could add to every header file to avoid problems? That's  
what I am looking for...

Konrad.
--
---------------------------------------------------------------------
Konrad Hinsen
Centre de Biophysique Mol?culaire, CNRS Orl?ans
Synchrotron Soleil - Division Exp?riences
Saint Aubin - BP 48
91192 Gif sur Yvette Cedex, France
Tel. +33-1 69 35 97 15
E-Mail: hinsen at cnrs-orleans.fr
---------------------------------------------------------------------




From fang.fang2003 at gmail.com  Thu Nov 30 12:20:47 2006
From: fang.fang2003 at gmail.com (Fang Fang)
Date: Thu, 30 Nov 2006 09:20:47 -0800
Subject: [Numpy-discussion] extremely slow array indexing?
Message-ID: <b94d46a00611300920g1d679a0q2a4e641c001b8f07@mail.gmail.com>

Hi,

I am writing code to sort the columns according to the sum of each column.
The dataset is huge (50k rows x 300k cols), so i have to read line by line
and do the summation to avoid the out-of-memory problem. But I don't know
why it runs very slow, and part of the code is as follows. Can anyone point
out what needs to be modified to make it run fast? thanks in advance!

...
from numpy import *
...

       currSum = zeros(self.componentcount)
       currRow = zeros(self.componentcount)
       for featureDict in self.featureDictList:
           currRow[:] = 0
           for components in self.componentdict1:
               if featureDict.has_key(components):
                   col = self.componentdict1[components]
                   value = featureDict[components]
                   currRow[col]=value;
           currSum = currSum + row;
...
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20061130/cc9d7acc/attachment.html>

From robert.kern at gmail.com  Thu Nov 30 12:28:45 2006
From: robert.kern at gmail.com (Robert Kern)
Date: Thu, 30 Nov 2006 11:28:45 -0600
Subject: [Numpy-discussion] extremely slow array indexing?
In-Reply-To: <b94d46a00611300920g1d679a0q2a4e641c001b8f07@mail.gmail.com>
References: <b94d46a00611300920g1d679a0q2a4e641c001b8f07@mail.gmail.com>
Message-ID: <456F14CD.1070508@gmail.com>

Fang Fang wrote:
> Hi,
> 
> I am writing code to sort the columns according to the sum of each
> column. The dataset is huge (50k rows x 300k cols), so i have to read
> line by line and do the summation to avoid the out-of-memory problem.
> But I don't know why it runs very slow, and part of the code is as
> follows. Can anyone point out what needs to be modified to make it run
> fast? thanks in advance!

Nothing leaps out. Generally, it's difficult (or impossible) to answer such
questions without running code. Can you distill the time-consuming part into a
small, self-contained script with fake data that we can run?

-- 
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless enigma
 that is made terrible by our own mad attempt to interpret it as though it had
 an underlying truth."
  -- Umberto Eco


From fang.fang2003 at gmail.com  Thu Nov 30 12:59:13 2006
From: fang.fang2003 at gmail.com (Fang Fang)
Date: Thu, 30 Nov 2006 09:59:13 -0800
Subject: [Numpy-discussion] extremely slow array indexing?
In-Reply-To: <456F14CD.1070508@gmail.com>
References: <b94d46a00611300920g1d679a0q2a4e641c001b8f07@mail.gmail.com>
	<456F14CD.1070508@gmail.com>
Message-ID: <b94d46a00611300959y454cb33dka63fb2c3077118e6@mail.gmail.com>

Thanks for your reply. The simplified code is as follows. It takes 7 seconds
to process 1000 rows, which is tolerable, but I wonder why it takes so long.
Isn't vectorized operation supposed to run very quickly.

from numpy import *

componentcount = 300000
currSum = zeros(componentcount)
row = zeros(componentcount) #current row
rowcount = 50000
for i in range(1,rowcount):
    row[:] = 1
    currSum = currSum + row;



On 11/30/06, Robert Kern <robert.kern at gmail.com> wrote:
>
> Fang Fang wrote:
> > Hi,
> >
> > I am writing code to sort the columns according to the sum of each
> > column. The dataset is huge (50k rows x 300k cols), so i have to read
> > line by line and do the summation to avoid the out-of-memory problem.
> > But I don't know why it runs very slow, and part of the code is as
> > follows. Can anyone point out what needs to be modified to make it run
> > fast? thanks in advance!
>
> Nothing leaps out. Generally, it's difficult (or impossible) to answer
> such
> questions without running code. Can you distill the time-consuming part
> into a
> small, self-contained script with fake data that we can run?
>
> --
> Robert Kern
>
> "I have come to believe that the whole world is an enigma, a harmless
> enigma
> that is made terrible by our own mad attempt to interpret it as though it
> had
> an underlying truth."
> -- Umberto Eco
> _______________________________________________
> Numpy-discussion mailing list
> Numpy-discussion at scipy.org
> http://projects.scipy.org/mailman/listinfo/numpy-discussion
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20061130/e084c6e0/attachment.html>

From robert.kern at gmail.com  Thu Nov 30 13:15:25 2006
From: robert.kern at gmail.com (Robert Kern)
Date: Thu, 30 Nov 2006 12:15:25 -0600
Subject: [Numpy-discussion] extremely slow array indexing?
In-Reply-To: <b94d46a00611300959y454cb33dka63fb2c3077118e6@mail.gmail.com>
References: <b94d46a00611300920g1d679a0q2a4e641c001b8f07@mail.gmail.com>	<456F14CD.1070508@gmail.com>
	<b94d46a00611300959y454cb33dka63fb2c3077118e6@mail.gmail.com>
Message-ID: <456F1FBD.6070904@gmail.com>

Fang Fang wrote:
> Thanks for your reply. The simplified code is as follows. It takes 7
> seconds to process 1000 rows, which is tolerable, but I wonder why it
> takes so long. Isn't vectorized operation supposed to run very quickly.

One problem is that you are creating a new currSum array for every row. Try using

  currSum += row

-- 
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless enigma
 that is made terrible by our own mad attempt to interpret it as though it had
 an underlying truth."
  -- Umberto Eco


From tim.hochberg at ieee.org  Thu Nov 30 13:20:36 2006
From: tim.hochberg at ieee.org (Tim Hochberg)
Date: Thu, 30 Nov 2006 11:20:36 -0700
Subject: [Numpy-discussion] extremely slow array indexing?
In-Reply-To: <b94d46a00611300959y454cb33dka63fb2c3077118e6@mail.gmail.com>
References: <b94d46a00611300920g1d679a0q2a4e641c001b8f07@mail.gmail.com>	<456F14CD.1070508@gmail.com>
	<b94d46a00611300959y454cb33dka63fb2c3077118e6@mail.gmail.com>
Message-ID: <456F20F4.10804@ieee.org>

Fang Fang wrote:
> Thanks for your reply. The simplified code is as follows. It takes 7 
> seconds to process 1000 rows, which is tolerable, but I wonder why it 
> takes so long. Isn't vectorized operation supposed to run very quickly.
>
> from numpy import *
>
> componentcount = 300000
> currSum = zeros(componentcount)
> row = zeros(componentcount) #current row
> rowcount = 50000
> for i in range(1,rowcount):
>     row[:] = 1
>     currSum = currSum + row
>

In this case, you can save a significant chunk of time (about 30% on my 
machine) by replacing the last line with:
    currSum += row

I suspect that the bulk of the remaining time is being chewed up moving 
things in and out of the cache since your arrays are large. There are 
some things you could try to alleviate that, but I don't know that I'd 
worry about it as I suspect that your overall time in your original 
problem is going to be dominated by pulling stuff out out of the 
dictionaries and populating row.

-tim



From robert.kern at gmail.com  Thu Nov 30 14:04:10 2006
From: robert.kern at gmail.com (Robert Kern)
Date: Thu, 30 Nov 2006 13:04:10 -0600
Subject: [Numpy-discussion] atol in allclose
In-Reply-To: <e06186140611281936v605ede15qdd60e384836f782f@mail.gmail.com>
References: <1e2af89e0611281730k492a2929j60ac9e54b5325790@mail.gmail.com>	<456CE611.7010205@gmail.com>
	<e06186140611281936v605ede15qdd60e384836f782f@mail.gmail.com>
Message-ID: <456F2B2A.4010006@gmail.com>

Charles R Harris wrote:
> I think that's right. Do you want to make the change?

Done.

-- 
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless enigma
 that is made terrible by our own mad attempt to interpret it as though it had
 an underlying truth."
  -- Umberto Eco


From jdhunter at ace.bsd.uchicago.edu  Thu Nov 30 14:14:54 2006
From: jdhunter at ace.bsd.uchicago.edu (John Hunter)
Date: Thu, 30 Nov 2006 13:14:54 -0600
Subject: [Numpy-discussion] bare bones numpy extension code
Message-ID: <87d574n3v5.fsf@peds-pc311.bsd.uchicago.edu>


A colleague of mine wants to write some numpy extension code.  I
pointed him to lots of examples in the matplotlib src dir, but the
build environment is more complicated than he needs with all the
numpy/numeric/numarray switches, etc.  Does someone have the basic
"hello world" of numpy extensions that includes src code and a basic
setup.py that I can pass on to him.  It might be nice to include
something like that in a numpy "examples" directory.

JDH

 


From hirzel at resonon.com  Thu Nov 30 14:53:19 2006
From: hirzel at resonon.com (Tim Hirzel)
Date: Thu, 30 Nov 2006 14:53:19 -0500
Subject: [Numpy-discussion] strange behavior with a numpy array as member of
	a list
Message-ID: <456F36AF.2090900@resonon.com>

Hi All,
When a list contains a numpy array and I want to see if a value is a 
member of that list, I can get a value error from numpy:

 >>> import numpy
 >>> numpy.__version__
'1.0'
 >>> a = numpy.arange(10)
 >>> L = [a]
 >>> a in L
True
 >>> 2 in L
Traceback (most recent call last):
 File "<input>", line 1, in ?
ValueError: The truth value of an array with more than one element is 
ambiguous. Use a.any() or a.all()

 >>> L = [2, a]  #just to illuminate the scenario a little more
 >>> 2 in L
True

Does this seem odd to anyone else?  I am not sure about the 
implementation of 'in' but it seems like a comparison is going funny 
somewhere.  Or I am missing something...

thanks!
tim


From robert.kern at gmail.com  Thu Nov 30 14:59:57 2006
From: robert.kern at gmail.com (Robert Kern)
Date: Thu, 30 Nov 2006 13:59:57 -0600
Subject: [Numpy-discussion] strange behavior with a numpy array as
 member of	a list
In-Reply-To: <456F36AF.2090900@resonon.com>
References: <456F36AF.2090900@resonon.com>
Message-ID: <456F383D.8030306@gmail.com>

Tim Hirzel wrote:
> Does this seem odd to anyone else?  I am not sure about the 
> implementation of 'in' but it seems like a comparison is going funny 
> somewhere.  Or I am missing something...

Yes, list.__contains__ uses equality to test for membership, not identity.

-- 
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless enigma
 that is made terrible by our own mad attempt to interpret it as though it had
 an underlying truth."
  -- Umberto Eco


From hirzel at resonon.com  Thu Nov 30 15:48:38 2006
From: hirzel at resonon.com (Tim Hirzel)
Date: Thu, 30 Nov 2006 15:48:38 -0500
Subject: [Numpy-discussion] strange behavior with a numpy array as
 member of	a list
In-Reply-To: <456F383D.8030306@gmail.com>
References: <456F36AF.2090900@resonon.com> <456F383D.8030306@gmail.com>
Message-ID: <456F43A6.9020302@resonon.com>

Thanks Robert.
That makes more sense now.  Although I am still a little puzzled by the 
equality behavior of an array.  for example:

 >>> a = numpy.arange(5)
 >>> None == a
False

 >>> 5 == a
array([False, False, False, False, False], dtype=bool)

 >>> class Empty:
    pass
 >>> e = Empty()
 >>> e == a
array([False, False, False, False, False], dtype=bool)

 >>> L = []
 >>> L == a
False

#this behavior seems consistent regardless of the dtype of the array:
 >>> b = numpy.array([None, 5])
 >>> b
array([None, 5], dtype=object)
 >>> b  == 4
array([False, False], dtype=bool)

What determines if an array tests for equality as the entire array or 
element-wise?

thanks again,
tim



Robert Kern wrote:
> Tim Hirzel wrote:
>   
>> Does this seem odd to anyone else?  I am not sure about the 
>> implementation of 'in' but it seems like a comparison is going funny 
>> somewhere.  Or I am missing something...
>>     
>
> Yes, list.__contains__ uses equality to test for membership, not identity.
>
>   


From robert.kern at gmail.com  Thu Nov 30 16:12:01 2006
From: robert.kern at gmail.com (Robert Kern)
Date: Thu, 30 Nov 2006 15:12:01 -0600
Subject: [Numpy-discussion] strange behavior with a numpy array as
 member of	a list
In-Reply-To: <456F43A6.9020302@resonon.com>
References: <456F36AF.2090900@resonon.com> <456F383D.8030306@gmail.com>
	<456F43A6.9020302@resonon.com>
Message-ID: <456F4921.5040009@gmail.com>

Tim Hirzel wrote:
> Thanks Robert.
> That makes more sense now.  Although I am still a little puzzled by the 
> equality behavior of an array.  for example:
> 
>  >>> a = numpy.arange(5)
>  >>> None == a
> False
> 
>  >>> 5 == a
> array([False, False, False, False, False], dtype=bool)
> 
>  >>> class Empty:
>     pass
>  >>> e = Empty()
>  >>> e == a
> array([False, False, False, False, False], dtype=bool)
> 
>  >>> L = []
>  >>> L == a
> False
> 
> #this behavior seems consistent regardless of the dtype of the array:
>  >>> b = numpy.array([None, 5])
>  >>> b
> array([None, 5], dtype=object)
>  >>> b  == 4
> array([False, False], dtype=bool)
> 
> What determines if an array tests for equality as the entire array or 
> element-wise?

There are some special cases for backwards compatibility. They are covered in
this portion of arrayobject.c in the function array_richcompare():

                case Py_EQ:
                        if (other == Py_None) {
                                Py_INCREF(Py_False);
                                return Py_False;
                        }
                        /* Try to convert other to an array */
                        if (!PyArray_Check(other)) {
                                typenum = self->descr->type_num;
                                if (typenum != PyArray_OBJECT) {
                                        typenum = PyArray_NOTYPE;
                                }
                                array_other = PyArray_FromObject(other,
                                                                 typenum, 0, 0);
                                /* If not successful, then return False
                                   This fixes code that used to
                                   allow equality comparisons between arrays
                                   and other objects which would give a result
                                   of False
                                */
                                if ((array_other == NULL) ||    \
                                    (array_other == Py_None)) {
                                        Py_XDECREF(array_other);
                                        PyErr_Clear();
                                        Py_INCREF(Py_False);
                                        return Py_False;
                                }
                        }

-- 
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless enigma
 that is made terrible by our own mad attempt to interpret it as though it had
 an underlying truth."
  -- Umberto Eco


From hirzel at resonon.com  Thu Nov 30 16:44:11 2006
From: hirzel at resonon.com (Tim Hirzel)
Date: Thu, 30 Nov 2006 16:44:11 -0500
Subject: [Numpy-discussion] strange behavior with a numpy array as
 member of	a list
In-Reply-To: <456F4921.5040009@gmail.com>
References: <456F36AF.2090900@resonon.com>
	<456F383D.8030306@gmail.com>	<456F43A6.9020302@resonon.com>
	<456F4921.5040009@gmail.com>
Message-ID: <456F50AB.6040506@resonon.com>

Excellent.  That explains it.
thank you Robert,
tim

Robert Kern wrote:
> Tim Hirzel wrote:
>   
>> Thanks Robert.
>> That makes more sense now.  Although I am still a little puzzled by the 
>> equality behavior of an array.  for example:
>>
>>  >>> a = numpy.arange(5)
>>  >>> None == a
>> False
>>
>>  >>> 5 == a
>> array([False, False, False, False, False], dtype=bool)
>>
>>  >>> class Empty:
>>     pass
>>  >>> e = Empty()
>>  >>> e == a
>> array([False, False, False, False, False], dtype=bool)
>>
>>  >>> L = []
>>  >>> L == a
>> False
>>
>> #this behavior seems consistent regardless of the dtype of the array:
>>  >>> b = numpy.array([None, 5])
>>  >>> b
>> array([None, 5], dtype=object)
>>  >>> b  == 4
>> array([False, False], dtype=bool)
>>
>> What determines if an array tests for equality as the entire array or 
>> element-wise?
>>     
>
> There are some special cases for backwards compatibility. They are covered in
> this portion of arrayobject.c in the function array_richcompare():
>
>                 case Py_EQ:
>                         if (other == Py_None) {
>                                 Py_INCREF(Py_False);
>                                 return Py_False;
>                         }
>                         /* Try to convert other to an array */
>                         if (!PyArray_Check(other)) {
>                                 typenum = self->descr->type_num;
>                                 if (typenum != PyArray_OBJECT) {
>                                         typenum = PyArray_NOTYPE;
>                                 }
>                                 array_other = PyArray_FromObject(other,
>                                                                  typenum, 0, 0);
>                                 /* If not successful, then return False
>                                    This fixes code that used to
>                                    allow equality comparisons between arrays
>                                    and other objects which would give a result
>                                    of False
>                                 */
>                                 if ((array_other == NULL) ||    \
>                                     (array_other == Py_None)) {
>                                         Py_XDECREF(array_other);
>                                         PyErr_Clear();
>                                         Py_INCREF(Py_False);
>                                         return Py_False;
>                                 }
>                         }
>
>   


From Chris.Barker at noaa.gov  Thu Nov 30 16:56:10 2006
From: Chris.Barker at noaa.gov (Christopher Barker)
Date: Thu, 30 Nov 2006 13:56:10 -0800
Subject: [Numpy-discussion] strange behavior with a numpy array as
 member of	a list
In-Reply-To: <456F4921.5040009@gmail.com>
References: <456F36AF.2090900@resonon.com> <456F383D.8030306@gmail.com>
	<456F43A6.9020302@resonon.com> <456F4921.5040009@gmail.com>
Message-ID: <456F537A.9060509@noaa.gov>



Robert Kern wrote:

>> What determines if an array tests for equality as the entire array or 
>> element-wise?
> 
> There are some special cases for backwards compatibility. 

with what?

IIRC, Numeric has raised an error for a very long time with:

a == None #(a is a Numeric array)

Personally, I liked it that way -- if you can't do a rich comparison, an 
error is better than a meaningless result.

-Chris


-- 
Christopher Barker, Ph.D.
Oceanographer

Emergency Response Division
NOAA/NOS/OR&R            (206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115       (206) 526-6317   main reception

Chris.Barker at noaa.gov


From robert.kern at gmail.com  Thu Nov 30 17:18:15 2006
From: robert.kern at gmail.com (Robert Kern)
Date: Thu, 30 Nov 2006 16:18:15 -0600
Subject: [Numpy-discussion] strange behavior with a numpy array as
 member of	a list
In-Reply-To: <456F537A.9060509@noaa.gov>
References: <456F36AF.2090900@resonon.com>
	<456F383D.8030306@gmail.com>	<456F43A6.9020302@resonon.com>
	<456F4921.5040009@gmail.com> <456F537A.9060509@noaa.gov>
Message-ID: <456F58A7.3090207@gmail.com>

Christopher Barker wrote:
> 
> Robert Kern wrote:
> 
>>> What determines if an array tests for equality as the entire array or 
>>> element-wise?
>> There are some special cases for backwards compatibility. 
> 
> with what?
> 
> IIRC, Numeric has raised an error for a very long time with:
> 
> a == None #(a is a Numeric array)


In [4]: import Numeric

In [5]: a = Numeric.arange(10)

In [6]: a == None
Out[6]: array([0, 0, 0, 0, 0, 0, 0, 0, 0, 0])


> Personally, I liked it that way -- if you can't do a rich comparison, an 
> error is better than a meaningless result.

I'm just reporting what the comment in the code says.

-- 
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless enigma
 that is made terrible by our own mad attempt to interpret it as though it had
 an underlying truth."
  -- Umberto Eco


From kwgoodman at gmail.com  Thu Nov 30 23:13:47 2006
From: kwgoodman at gmail.com (Keith Goodman)
Date: Thu, 30 Nov 2006 20:13:47 -0800
Subject: [Numpy-discussion] save a matrix
Message-ID: <f4f93d420611302013x52ce2984x79f02da380192716@mail.gmail.com>

What's a good way to save matrix objects to file for later use? I just
need something quick for debugging.

I saw two suggestions on this list from Francesc Altet (2006-05-22):

1. Use tofile and fromfile and save the meta data yourself.

2. pytables

Any suggestions for #3?

If I do #1, how would I put multiple matrices in a single file? Make
everything a 1-d array, first element is number of matrices, followed
by number of rows and columns for each matrix?


From Carsten.Rostgaard at fysik.dtu.dk  Thu Nov 23 03:05:57 2006
From: Carsten.Rostgaard at fysik.dtu.dk (Carsten Rostgaard)
Date: Thu, 23 Nov 2006 09:05:57 +0100
Subject: [Numpy-discussion] dot operations on multidimensional arrays
Message-ID: <45655665.7020302@fysik.dtu.dk>

Hi!
I am trying to use the "dot" method on multi-(more than 2)-dimensional
arrays.

Specifically I do
    >> y = dot(a, b)
where a is a 2D array and b is a 3D array.

using numpy I get the the help:
"
dot(...)
    dot(a,v) returns matrix-multiplication between a and b.
    The product-sum is over the last dimension of a and the
    second-to-last dimension of b.
"
I then expect that
    >> y[i, j, k] = sum(a[i, :] * b[j, :, k])
which is actually what I get.

The question is then:
1) Is there any way to change the axis for which the product-sum is
performed. This can of course be done by a swapaxis before and after the
operation, but this makes the array non-contiguous, in which case the
dot operation often makes bugs (at least in Numeric).
2) For complicated reasons we still use Numeric in our software package,
and in this, "dot" behaves very strangely.
According to the Numeric help:
"
dot(a, b)
    dot(a,b) returns matrix-multiplication between a and b.  The product-sum
    is over the last dimension of a and the second-to-last dimension of b.
"
so I would have expected again that y[i, j, k] = sum(a[i, :] * b[j, :,
k]), and the dimensions actually fit, i.e.
y.shape = (a.shape[0], b.shape[0], b.shape[2]), but only some rows of
the result has these values!! Does anyone know what Numeric.dot(a, b)
actually does when b has more than two dimensions?

I use the following test script:
---------------------BEGIN SCRIPT-----------------------
import Numeric as num
# import numpy as num

# make 'random' input arrays
a = num.zeros((2, 5))
b = num.zeros((3, 5, 4))
a.flat[:] = num.arange(len(a.flat)) - 3
b.flat[:] = num.arange(len(b.flat)) + 5

# built-in dot product
y1 = num.dot(a, b)

# manual dot product
y2 = num.zeros((a.shape[0], b.shape[0], b.shape[2]))
for i in range(a.shape[0]):
  for j in range(b.shape[0]):
     for k in range(b.shape[2]):
       y2[i, j, k] = num.sum(a[i,:] * b[j, :, k])

# test for consistency
print y1 == y2
---------------------END SCRIPT------------------------
with the result:
[[[1 1 1 1]
  [0 0 0 0]
  [0 0 0 0]]
[[1 1 1 1]
  [0 0 0 0]
  [0 0 0 0]]]


thanks a lot,
Carsten Rostgaard
Carsten.Rostgaard at fysik.dtu.dk





From Jerome.Fuselier at xrce.xerox.com  Thu Nov 23 10:09:57 2006
From: Jerome.Fuselier at xrce.xerox.com (Jerome Fuselier)
Date: Thu, 23 Nov 2006 16:09:57 +0100
Subject: [Numpy-discussion] Strange numpy behaviour with pickle
Message-ID: <4565B9C5.1080108@xrce.xerox.com>

An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20061123/66e91abb/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: test_numpy_pickle.py
Type: text/x-python
Size: 1065 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20061123/66e91abb/attachment.py>

From emgmendes at gmail.com  Mon Nov 27 06:56:43 2006
From: emgmendes at gmail.com (Elton Mendes)
Date: Mon, 27 Nov 2006 11:56:43 +0000
Subject: [Numpy-discussion] Precision in Python
Message-ID: <a6def42c0611270356rb566bbbg355065ecff86cf78@mail.gmail.com>

Hi.
I'm having a precision problem in python

Example:


>>> a = 5.14343434
>>> b = round(a,1)
>>> b
5.0999999999999996
>>>

It?s possible to round the number exactly to 5.1
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20061127/dcb727fa/attachment.html>

From oliphant.travis at ieee.org  Tue Nov 28 02:10:49 2006
From: oliphant.travis at ieee.org (Travis Oliphant)
Date: Tue, 28 Nov 2006 00:10:49 -0700
Subject: [Numpy-discussion] Wanted: Numeric/NumPy compatible array
 creation expression
In-Reply-To: <C7F9C351-CFB8-4DAE-8B8D-2504485E59BC@laposte.net>
References: <C7F9C351-CFB8-4DAE-8B8D-2504485E59BC@laposte.net>
Message-ID: <456BE0F9.1000101@ieee.org>

konrad.hinsen at laposte.net wrote:
> I am looking for a way to create an array of the same type as another  
> given array (but of different shape) that works both with Numeric and  
> NumPy without being unreasonably slow. In other words, I am looking  
> for a a replacement for the expression
>
> 	array2 = Numeric.zeros(shape, array1.typecode())
>
> that will *also* (not *only*) work under NumPy (where typecode()  
> became dtype.char). An obvious idea is reshaping array1 and  
> multiplying by 0., but that can become quite costly.
>
> Any ideas?
>   

Matplotlib uses the numerix layer to handle compatibility between all 
three array packages. 

It defines the typecode function in that package and then uses it 
instead of the method.


-Travis