From nadavh at visionsense.com  Thu Apr  1 06:38:10 2004
From: nadavh at visionsense.com (Nadav Horesh)
Date: Thu Apr  1 06:38:10 2004
Subject: [Numpy-discussion] type coercion question
Message-ID: <07C6A61102C94148B8104D42DE95F7E86DECCD@exchange2k.envision.co.il>

(3) seems to be reasonable since Int32-precision > Float32-precision

  Nadav.


-----Original Message-----
From:	Perry Greenfield [mailto:perry at stsci.edu]
Sent:	Wed 31-Mar-04 17:39
To:	David M. Cooke; Todd Miller
Cc:	Gary Ruben; numpy-discussion
Subject:	[Numpy-discussion] type coercion question
David Cooke wrote:
> On Mon, Mar 29, 2004 at 06:55:57AM -0500, Todd Miller wrote:
> > On Sun, 2004-03-28 at 22:46, Gary Ruben wrote:
> > > > > >>> cos(1)
> > > > > 0.54030227661132813
> > > > >
> > > > > gives a different result to cos(1.)
> > > <snip>
> > > > >  Do others think precision is being lost unnecessarily?
> > > >
> > > > No.  Do you have any suggestions?
> > >
> > > Yes. My problem is that numarray is not replicating the behaviour of
> > > the Python math module, which presumably coerces immediately to the
> > > float precision of the underlying platform. This is probably a
> > > float64 in Windows. If you don't explicitly specify that float32 is
> > > to be used, shouldn't the default be for a rank-0 value to
> > > immediately coerce ints to the same precision as the native Python
> > > float type on the underlying platform, since you know it will be
> > > coerced to that later and in the meantime you've lost precision
> > > because you've applied a function to a value of lower precision?
> > >
> >
> > I see your point and I'll talk it over with Perry unless he chimes in on
> > his own.  I'm ambivalent:  while it would be nice for the numarray
> > ufuncs to act as full equivalents to the Python math functions in scalar
> > cases,  numarray has to strike a balance between managing storage space
> > and maintaining precision and these two goals are in conflict.  I think
> > using an array package you have to be at least a little more aware of
> > storage concerns.
>
> I'll throw in a vote for 'cos(1)' coercing 1 to a (equivalent-to-)
> Python float. The reason is that I have 'from numarray import *' for my
> interactive interpreter, as I use numarray a lot in that.
> I would rather not have to remember to always cast my args to ufuncs to
> arrays. This isn't a storage space problem, per se., the question is
> whether cos(1) and cos(1.0) should return the same answer.
>
> Having cos(1) cast 1 to a Float32 makes numarray much less useful as a
> calculator -- mistakes are more likely. I'll probably switch my
> interpreter back to Numeric so as not to make mistakes, or I'll write a
> wrapper (which I'll post if I do).
>
> Whether cos([1,1,1]) and cos([1.0,1.0,1.0]) are the same I'd say is
> different, as there I've taken the conscious decision not to cast to an
> array. (And the fact that cos([1,1,1]) will return an array printed out
> with 'type=Float32' is a clue to what I've done.)
>
Let me say that I see the last issue (whether cos([1,1,1]) and
cos([1.0,1.0,1.0]) are the same) is important. If cos([1,1,1])[0] !=
cos(1) I think that is just as bad. So the question is whether
ints scalars and arrays should automatically be promoted to
double floats when given to unary scientific functions (sqrt,
sin, cos, log, etc.). By the way, if an Int32 is so promoted,
I don't see why it isn't the same for Int8, Int16, etc.
But it appears that Numeric promotes Int32 to Float64 for cos(),
but promotes In16 to Float32 for cos().

I believe making this change to numarray is fairly easy (we'd have
to check). But is what most people want? Should we retain complete
compatibility with Numeric (which, as pointed out, treats different
kinds of integers differently)

Just to summarize what I see as acceptable choices:

1) ints promote to Float32 (current behavior)
2) ints promote to Float64
3) Int32 promotes to Float64, lesser ints promote to Float32 (Numeric
behavior)

Perry






-------------------------------------------------------
This SF.Net email is sponsored by: IBM Linux Tutorials
Free Linux tutorial presented by Daniel Robbins, President and CEO of
GenToo technologies. Learn everything from fundamentals to system
administration.http://ads.osdn.com/?ad_id=1470&alloc_id=3638&op=click
_______________________________________________
Numpy-discussion mailing list
Numpy-discussion at lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/numpy-discussion






From perry at stsci.edu  Thu Apr  1 14:38:21 2004
From: perry at stsci.edu (Perry Greenfield)
Date: Thu Apr  1 14:38:21 2004
Subject: [Numpy-discussion] question about nonzero behavior in numarray
In-Reply-To: <slrnc6gv95.3v3.faheem@Chrestomanci.home.earth>
Message-ID: <CEELJPECNGEGKFNDLHFFOEALCDAA.perry@stsci.edu>

Todd pointed out to me that he's gotten some complaints about
the difference in how numarray nonzero behavior works.
In version 0.9 the use of an array for a truth value was
changed from a deprecation warning to an exception.

We deprecated it because the meaning of an array as a truth
value was ambiguous (or potentially confusing at best)
and we felt it was better to force users to be explicit
about what they wanted. Nevertheless, Numeric does allow that
and removing the capability to use it in that context breaks
some code. 

We can give some thought to reversing that decision, but since
we only hear from those that don't like the new behavior, we'd
like to get some sense if there are many that do like the new
behavior.

To clarify, what happens when:

x = zeros(2)
if x:
    print "true!"

I see 3 possibilities:

1) Keep it the way it is, no use of an array is permitted; an 
   exception is raised.
2) Print a warning message when it is used that way, and always
   keep it at that, i.e., warning message only.
3) Replicate Numeric behavior.

Personally, I prefer 1) since I think there is too much confusion
between whether false means an empty (len=0) array (like lists),
or an array that contains all zeros, and that it is worth breaking
backward compatibility on this (functions should be used to 
distinguish which meaning is desired). 

Perry

[Option 2 on the previous question wins hands down]






From Fernando.Perez at colorado.edu  Thu Apr  1 14:52:09 2004
From: Fernando.Perez at colorado.edu (Fernando Perez)
Date: Thu Apr  1 14:52:09 2004
Subject: [Numpy-discussion] question about nonzero behavior in numarray
In-Reply-To: <CEELJPECNGEGKFNDLHFFOEALCDAA.perry@stsci.edu>
References: <CEELJPECNGEGKFNDLHFFOEALCDAA.perry@stsci.edu>
Message-ID: <406C9CEB.8050707@colorado.edu>

Perry Greenfield wrote:

> Personally, I prefer 1) since I think there is too much confusion
> between whether false means an empty (len=0) array (like lists),
> or an array that contains all zeros, and that it is worth breaking
> backward compatibility on this (functions should be used to 
> distinguish which meaning is desired). 

I tend to prefer keeping compatibility with Python list behavior.  I think 
that in general, numarray arrays should behave different from lists only when 
truly justified (for example, at least in numeric all ufuncs can be applied to 
a list of floats as much as to an array, which is nice).

Since anyone who knows python already knows that lists are true based on 
whether they contain _anything_:

In [9]: a=[]

In [10]: b=[0]

In [11]: if a:
    ....:     print 1
    ....:

In [12]: if b:
    ....:     print 1
    ....:
1


then I think it's a fair, clean extension to make NumArray behave similarly.

I think a good guiding principle is to keep the number of special cases for 
arrays at a strict minimum (there are obviously cases where it does make sense 
to have special cases).  Besides, this is an O(1) check, so it doesn't have 
the dangers of checking for all elements being zero.   Such a potentially 
expensive operation should _definitely_ happen only if explicitly requested, IMHO.

Regards,

f




From cjw at sympatico.ca  Thu Apr  1 15:00:00 2004
From: cjw at sympatico.ca (Colin J. Williams)
Date: Thu Apr  1 15:00:00 2004
Subject: [Numpy-discussion] question about nonzero behavior in numarray
In-Reply-To: <CEELJPECNGEGKFNDLHFFOEALCDAA.perry@stsci.edu>
References: <CEELJPECNGEGKFNDLHFFOEALCDAA.perry@stsci.edu>
Message-ID: <406C9EC3.8080504@sympatico.ca>


Perry Greenfield wrote:

>Todd pointed out to me that he's gotten some complaints about
>the difference in how numarray nonzero behavior works.
>In version 0.9 the use of an array for a truth value was
>changed from a deprecation warning to an exception.
>
>We deprecated it because the meaning of an array as a truth
>value was ambiguous (or potentially confusing at best)
>and we felt it was better to force users to be explicit
>about what they wanted. Nevertheless, Numeric does allow that
>and removing the capability to use it in that context breaks
>some code. 
>
>We can give some thought to reversing that decision, but since
>we only hear from those that don't like the new behavior, we'd
>like to get some sense if there are many that do like the new
>behavior.
>
>To clarify, what happens when:
>
>x = zeros(2)
>if x:
>    print "true!"
>
>I see 3 possibilities:
>
>1) Keep it the way it is, no use of an array is permitted; an 
>   exception is raised.
>2) Print a warning message when it is used that way, and always
>   keep it at that, i.e., warning message only.
>3) Replicate Numeric behavior.
>
>Personally, I prefer 1) since I think there is too much confusion
>between whether false means an empty (len=0) array (like lists),
>or an array that contains all zeros, and that it is worth breaking
>backward compatibility on this (functions should be used to 
>distinguish which meaning is desired). 
>
>Perry
>
>[Option 2 on the previous question wins hands down]
>  
>
I favour option 1, otherwise there is a need to define just what it means.

Does it return a Boolean array indicating those cells which are nonzero, or
a single Boolean value indicating that some cell is nonzero, or a single
Boolean value indicating that all cells are nonzero?

Colin W.

>
>
>
>
>-------------------------------------------------------
>This SF.Net email is sponsored by: IBM Linux Tutorials
>Free Linux tutorial presented by Daniel Robbins, President and CEO of
>GenToo technologies. Learn everything from fundamentals to system
>administration.http://ads.osdn.com/?ad_id=1470&alloc_id=3638&op=click
>_______________________________________________
>Numpy-discussion mailing list
>Numpy-discussion at lists.sourceforge.net
>https://lists.sourceforge.net/lists/listinfo/numpy-discussion
>
>  
>




From list at jsaul.de  Thu Apr  1 15:40:04 2004
From: list at jsaul.de (Joachim Saul)
Date: Thu Apr  1 15:40:04 2004
Subject: [Numpy-discussion] question about nonzero behavior in numarray
In-Reply-To: <406C9CEB.8050707@colorado.edu>
References: <CEELJPECNGEGKFNDLHFFOEALCDAA.perry@stsci.edu> <406C9CEB.8050707@colorado.edu>
Message-ID: <20040401233805.GA684@jsaul.de>

* Fernando Perez [2004-04-02 00:51]:
> Perry Greenfield wrote:
>
> >Personally, I prefer 1) since I think there is too much confusion
> >between whether false means an empty (len=0) array (like lists),
> >or an array that contains all zeros, and that it is worth breaking
> >backward compatibility on this (functions should be used to
> >distinguish which meaning is desired).
>
> I tend to prefer keeping compatibility with Python list behavior.  I think
> that in general, numarray arrays should behave different from lists only
> when truly justified (for example, at least in numeric all ufuncs can be
> applied to a list of floats as much as to an array, which is nice).

How often does it happen that you have an empty array, which you
need to test for? Certainly not as frequently as in case of a
list.

I think that if

  >>> a=ones(3)
  >>> a==1
  array([1, 1, 1])
  >>> a==0
  array([0, 0, 0])

then I would follow that

  >>> if a==0:
  ...
  >>> if array([0, 0, 0]):
  ...

should also _both_ evaluate as "false".

> Since anyone who knows python already knows that lists are true based on
> whether they contain _anything_:
>
> In [9]: a=[]
>
> In [10]: b=[0]
>
> In [11]: if a:
>    ....:     print 1
>    ....:
>
> In [12]: if b:
>    ....:     print 1
>    ....:
> 1
>
>
> then I think it's a fair, clean extension to make NumArray behave similarly.
>
> I think a good guiding principle is to keep the number of special cases for
> arrays at a strict minimum (there are obviously cases where it does make
> sense to have special cases).  Besides, this is an O(1) check, so it
> doesn't have the dangers of checking for all elements being zero.   Such a
> potentially expensive operation should _definitely_ happen only if
> explicitly requested, IMHO.

But if the behaviour is unambiguously defined, I see no problem
here, because I can as well check the length of a list using
len().

BTW, How about

    >>> a = [[]]
    >>> if a:
    ...

?

Cheers,
Joachim



From Fernando.Perez at colorado.edu  Thu Apr  1 16:16:16 2004
From: Fernando.Perez at colorado.edu (Fernando Perez)
Date: Thu Apr  1 16:16:16 2004
Subject: [Numpy-discussion] question about nonzero behavior in numarray
In-Reply-To: <20040401233805.GA684@jsaul.de>
References: <CEELJPECNGEGKFNDLHFFOEALCDAA.perry@stsci.edu> <406C9CEB.8050707@colorado.edu> <20040401233805.GA684@jsaul.de>
Message-ID: <406CB0A0.5030509@colorado.edu>

Joachim Saul wrote:

> How often does it happen that you have an empty array, which you
> need to test for? Certainly not as frequently as in case of a
> list.

Well, just these days I'm working with code where I'm precisely using that 
kind of check, relying on Numeric's behavior.  But I know that with 
potentially nested structures, it's probably best to be explicit and do all 
checks via things like:

if len(a)

and

if a==0

etc.  So I could also be +1 on Perry's wish for turning it into an extension, 
at the expense of my current code breaking.

Cheers,

f.




From owen at astro.washington.edu  Fri Apr  2 09:59:06 2004
From: owen at astro.washington.edu (Russell E Owen)
Date: Fri Apr  2 09:59:06 2004
Subject: [Numpy-discussion] Questions about records
Message-ID: <p06020405bc934e349109@[128.95.99.44]>

I've been trying to generate a 2-dimensional numarray.records array 
and am rather puzzled about some failures:

I tried a pair of 2x2 arrays for the buffer (one per field). I 
thought the record would get its shape from those, but instead of a 
2x2 record I get a 2-length record, each of whose elements is a pair 
of 2-length arrays.

>>>  arr1 = num.arange(4, shape=(2,2), type=num.Float64)
>>>  arr2 = num.arange(4, shape=(2,2), type=num.Float64) + 10
>>>  a = rec.array([arr1, arr2], names="a,b")
>>>  a
array(
[(array([ 0.,  1.]), array([ 10.,  11.])),
(array([ 2.,  3.]), array([ 12.,  13.]))],
formats=['(2,)Float64', '(2,)Float64'],
shape=2,
names=['a', 'b'])

So I tried passing in 4-length arrays while specifying the shape 
explicitly, but it failed. Is this a bug?:

>>>  arr1 = num.arange(4, type=num.Float64)
>>>  arr2 = num.arange(4, type=num.Float64) + 10
>>>  a = rec.array([arr1, arr2], shape=(2,2), names="a, b")
Traceback (most recent call last):
   File "<stdin>", line 1, in ?
   File "/usr/local/lib/python2.3/site-packages/numarray/records.py", 
line 384, in array
     byteorder=byteorder, aligned=aligned)
   File "/usr/local/lib/python2.3/site-packages/numarray/records.py", 
line 177, in fromarrays
     raise ValueError, "array has different lengths"
ValueError: array has different lengths

Generating a 4-length record and reshaping it does seem to work, 
though there seems to be a bug in __str__ which I'll report:

>>>  arr1 = num.arange(4, type=num.Float64)
>>>  arr2 = num.arange(4, type=num.Float64) + 10
>>>  a = rec.array([arr1, arr2], names="a, b")
>>>  a.setshape((2,2))
>>>  a
Traceback (most recent call last):
   File "<stdin>", line 1, in ?
   File "/usr/local/lib/python2.3/site-packages/numarray/records.py", 
line 718, in __repr__
     outlist.append(Record.__str__(i))
TypeError: unbound method __str__() must be called with Record 
instance as first argument (got RecArray instance instead)
>>>  a[1,1]
<numarray.records.Record instance at 0x15d2ad0>
>>>  str(a[1,1])
'(1.0, 11.0)'

I see the str problem again if I don't specify any buffer:

>>>  import numarray as num
>>>  import numarray.records as rec
>>>  a = rec.array(formats="Float64,Float64", names="a,b", shape=(2,2))
>>>  a
Traceback (most recent call last):
   File "<stdin>", line 1, in ?
   File "/usr/local/lib/python2.3/site-packages/numarray/records.py", 
line 718, in __repr__
     outlist.append(Record.__str__(i))
TypeError: unbound method __str__() must be called with Record 
instance as first argument (got RecArray instance instead)

So...comments? Should I report the shape issues as a bug?

-- Russell



From hsu at stsci.edu  Fri Apr  2 11:13:06 2004
From: hsu at stsci.edu (Jin-chung Hsu)
Date: Fri Apr  2 11:13:06 2004
Subject: FW: [Numpy-discussion] Questions about records
Message-ID: <200404021912.AKB80704@donner.stsci.edu>

> I've been trying to generate a 2-dimensional numarray.records array 
> and am rather puzzled about some failures:
> 
> I tried a pair of 2x2 arrays for the buffer (one per field). I 
> thought the record would get its shape from those, but instead of a 
> 2x2 record I get a 2-length record, each of whose elements is a pair 
> of 2-length arrays.
> 
> >>>  arr1 = num.arange(4, shape=(2,2), type=num.Float64)
> >>>  arr2 = num.arange(4, shape=(2,2), type=num.Float64) + 10
> >>>  a = rec.array([arr1, arr2], names="a,b")
> >>>  a
> array(
> [(array([ 0.,  1.]), array([ 10.,  11.])),
> (array([ 2.,  3.]), array([ 12.,  13.]))],
> formats=['(2,)Float64', '(2,)Float64'],
> shape=2,
> names=['a', 'b'])
> 
> So I tried passing in 4-length arrays while specifying the shape 
> explicitly, but it failed. Is this a bug?:
> 
> >>>  arr1 = num.arange(4, type=num.Float64)
> >>>  arr2 = num.arange(4, type=num.Float64) + 10
> >>>  a = rec.array([arr1, arr2], shape=(2,2), names="a, b")
> Traceback (most recent call last):
>    File "<stdin>", line 1, in ?
>    File "/usr/local/lib/python2.3/site-packages/numarray/records.py", 
> line 384, in array
>      byteorder=byteorder, aligned=aligned)
>    File "/usr/local/lib/python2.3/site-packages/numarray/records.py", 
> line 177, in fromarrays
>      raise ValueError, "array has different lengths"
> ValueError: array has different lengths
> 
> Generating a 4-length record and reshaping it does seem to work, 
> though there seems to be a bug in __str__ which I'll report:
> 
> >>>  arr1 = num.arange(4, type=num.Float64)
> >>>  arr2 = num.arange(4, type=num.Float64) + 10
> >>>  a = rec.array([arr1, arr2], names="a, b")
> >>>  a.setshape((2,2))
> >>>  a
> Traceback (most recent call last):
>    File "<stdin>", line 1, in ?
>    File "/usr/local/lib/python2.3/site-packages/numarray/records.py", 
> line 718, in __repr__
>      outlist.append(Record.__str__(i))
> TypeError: unbound method __str__() must be called with Record 
> instance as first argument (got RecArray instance instead)
> >>>  a[1,1]
> <numarray.records.Record instance at 0x15d2ad0>
> >>>  str(a[1,1])
> '(1.0, 11.0)'
> 
> I see the str problem again if I don't specify any buffer:
> 
> >>>  import numarray as num
> >>>  import numarray.records as rec
> >>>  a = rec.array(formats="Float64,Float64", names="a,b", shape=(2,2))
> >>>  a
> Traceback (most recent call last):
>    File "<stdin>", line 1, in ?
>    File "/usr/local/lib/python2.3/site-packages/numarray/records.py", 
> line 718, in __repr__
>      outlist.append(Record.__str__(i))
> TypeError: unbound method __str__() must be called with Record 
> instance as first argument (got RecArray instance instead)
> 
> So...comments? Should I report the shape issues as a bug?
> 
> -- Russell

First of all, the records module was developed mainly having the 1-D table in 
mind.  Even though it can have higher than one dimension, it is not thoroughly 
tested, as you have found out.  However, I'd argue that in many cases that the 
need to use a 2-D (or high) table can be substituted by having an array in each 
cell(element).  In your example, instead of creating a 2x2 table with each cell 
just having one number, you may be able to use a table with just one row and 
each cell is a 2x2 array.  You can create such a record like this:

--> arr1 = num.arange(4, shape=(1,2,2), type=num.Float64)
--> arr2 = num.arange(4, shape=(1,2,2), type=num.Float64)+10
--> a = rec.array([arr1, arr2], names="a,b")

I'd be interested in your application as to why a 2x2 table is necessary.

The __str__ method in RecArray is rather primitive but usually works for 1-D 
tables.  Eventually, we'll need a C-implementation of this method for both speed 
and flexibility.

When you use a list for the buffer in the array function, it is using a 
relatively intuitive (simplistic?) approach.  It tries to figure out the (one 
dimensional) length of each item in the list and use that as the record shape.

JC Hsu




From owen at astro.washington.edu  Fri Apr  2 12:09:04 2004
From: owen at astro.washington.edu (Russell E Owen)
Date: Fri Apr  2 12:09:04 2004
Subject: FW: [Numpy-discussion] Questions about records
In-Reply-To: <200404021912.AKB80704@donner.stsci.edu>
References: <200404021912.AKB80704@donner.stsci.edu>
Message-ID: <p06020406bc9372470570@[128.95.99.44]>

At 2:12 PM -0500 2004-04-02, Jin-chung Hsu wrote:
>First of all, the records module was developed mainly having the 1-D table in
>mind.  Even though it can have higher than one dimension, it is not thoroughly
>tested, as you have found out.  However, I'd argue that in many cases that the
>need to use a 2-D (or high) table can be substituted by having an 
>array in each
>cell(element).  In your example, instead of creating a 2x2 table 
>with each cell
>just having one number, you may be able to use a table with just one row and
>each cell is a 2x2 array.  You can create such a record like this:
>
>--> arr1 = num.arange(4, shape=(1,2,2), type=num.Float64)
>--> arr2 = num.arange(4, shape=(1,2,2), type=num.Float64)+10
>--> a = rec.array([arr1, arr2], names="a,b")

But is there any advantage to that compared to just using named 
arrays of the desired shape:
a = num.arange(4, shape=(2,2), type=num.Float64)
b = num.arange(4, shape=(2,2), type=num.Float64)+10

>I'd be interested in your application as to why a 2x2 table is necessary.

Here are two different uses I've come up with (both related to image 
processing). Both are beautifully served by a 2-d records array:

1) Find the centroid of a star. The algorithm I'm using (invented by 
Jim Gunn, I believe) is to walk across the image, looking for the 
point of maximum symmetry. At each point total pixels and a measure 
of asymmetry are measured in a 3x3 grid centered at that point. The 
minimum asymmetry in that 3x3 array is then used to determine where 
to walk next. (At the end a parabolic fit is done to the 3x3 
asymmetry data to find the true centroid; up until then it's only 
know to the nearest pixel).

In any case...right now I maintain two separate 3x3 arrays (total 
pixels and asymmetry). Whenever I take a step I shift the both arrays 
and then compute new data for the points which are missing data.

It would be cleaner and nicer to maintain one 3x3 records array with 
fields "totPix" and "asymm". Then the related data sticks together 
and I only have to shift the data once. (I meant to code it that way 
from the start, but my early attempts to use numeric.records were a 
disaster. I have a somewhat better handle on it now and may update my 
code).

2) Find all stars on an image. The algorithm I'm using (invented by 
Jeff Morgan, I believe) is to break an image up into blocks of, say, 
5x5 pixels. I then compute information about each "super pixel", such 
as center of mass, total counts, etc. My C++ code has 12 items of 
information for each super pixel (including 7 boolean flags) and is 
written to use a 2-dimensional array each element of which is a data 
structure with the appropriate fields. The obvious python equivalent 
is a numarray.records array. It sure sounds better than trying to 
keep track of 12 separate arrays!

-- Russell



From hsu at stsci.edu  Fri Apr  2 13:30:05 2004
From: hsu at stsci.edu (Jin-chung Hsu)
Date: Fri Apr  2 13:30:05 2004
Subject: FW: [Numpy-discussion] Questions about records
Message-ID: <200404022129.AKB90076@donner.stsci.edu>

> >--> arr1 = num.arange(4, shape=(1,2,2), type=num.Float64)
> >--> arr2 = num.arange(4, shape=(1,2,2), type=num.Float64)+10
> >--> a = rec.array([arr1, arr2], names="a,b")
> 
> But is there any advantage to that compared to just using named 
> arrays of the desired shape:
> a = num.arange(4, shape=(2,2), type=num.Float64)
> b = num.arange(4, shape=(2,2), type=num.Float64)+10
>

Not really, in this particular example.
 
> >I'd be interested in your application as to why a 2x2 table is necessary.
> 
> Here are two different uses I've come up with (both related to image 
> processing). 
[snip]

You need it because you need to pass it to the C-structure, I think.  In any 
case, you have found a way to get around the problem by using setshape.  I'll 
take a look of the module to get an idea of how much effort is needed to make 
the 2-D (and higher) record arrays to work more smoothly.

JC Hsu




From list at jsaul.de  Fri Apr  2 15:32:08 2004
From: list at jsaul.de (Joachim Saul)
Date: Fri Apr  2 15:32:08 2004
Subject: [Numpy-discussion] question about nonzero behavior in numarray
In-Reply-To: <406CB0A0.5030509@colorado.edu>
References: <CEELJPECNGEGKFNDLHFFOEALCDAA.perry@stsci.edu> <406C9CEB.8050707@colorado.edu> <20040401233805.GA684@jsaul.de> <406CB0A0.5030509@colorado.edu>
Message-ID: <20040402233135.GA688@jsaul.de>

* Fernando Perez [2004-04-02 02:15]:
> Joachim Saul wrote:
>
> >How often does it happen that you have an empty array, which you
> >need to test for? Certainly not as frequently as in case of a
> >list.
>
> Well, just these days I'm working with code where I'm precisely using that
> kind of check, relying on Numeric's behavior.  But I know that with
> potentially nested structures, it's probably best to be explicit and do all
> checks via things like:
>
> if len(a)
>
> and
>
> if a==0

But what would a==0 return? A boolean? An array of booleans? An
array if integers? The way it is evaluated now in Numeric is
pretty clear, though there could be other evaluations.  For
instance,

    if a==1:

might be understood as "if a is everywhere one" or "if the length
of vector a is one", etc. The current meaning is "if a is anywhere
one", which may not be intuitive under all circumstances, but as
long as it is the well defined behaviour, I simply don't see the
need to change or forbid that.

Nested structures are an important argument against keeping
compatibility with Python list behaviour. For instance

    >>> if []:   print "True"
    >>> if [[]]: print "True"
    True

Which perfectly makes sense in Python, but an equivalent behaviour
of a numeric array would be complete humbug. Thus, I don't think
that it would make any sense to emulate Python list behaviour,
rather I would support keeping the *existing* Numeric behaviour.

Cheers,
Joachim



From list at jsaul.de  Fri Apr  2 22:57:22 2004
From: list at jsaul.de (Joachim Saul)
Date: Fri Apr  2 22:57:22 2004
Subject: [Numpy-discussion] question about nonzero behavior in numarray
In-Reply-To: <20040402233135.GA688@jsaul.de>
References: <CEELJPECNGEGKFNDLHFFOEALCDAA.perry@stsci.edu> <406C9CEB.8050707@colorado.edu> <20040401233805.GA684@jsaul.de> <406CB0A0.5030509@colorado.edu> <20040402233135.GA688@jsaul.de>
Message-ID: <20040403064835.GA681@jsaul.de>

* I claimed:
>     if a==1:
>
> might be understood as "if a is everywhere one" or "if the length
> of vector a is one", etc. The current meaning is "if a is anywhere
> one", [...]

This is of course wrong and would probably not be misunderstood as
that. What I meant to say is that

    >>> if a:
    ...

would be evaluated as "true" if a is anywhere non-zero. It was too
late in the night it seems...

Sorry for the confusion.
Joachim



From noori at yellow.plala.or.jp  Sat Apr  3 12:02:11 2004
From: noori at yellow.plala.or.jp (Noori)
Date: Sat Apr  3 12:02:11 2004
Subject: [Numpy-discussion] Numpy-discussion Wanted to try Cialis but thought it was way too expensive for you? numpy-discussion@lists.sourceforge.net Numpy-discussion
In-Reply-To: <IBIAH045864GD21K@lists.sourceforge.net>
References: <IBIAH045864GD21K@lists.sourceforge.net>
Message-ID: <80BD72GFEF0GJ1LG@yellow.plala.or.jp>

You can do that now. Cialiis at only $2.00 p/d.
Your best deals are here
http://smart-pharmacy.com/15/
You have enjoyed Viaggra and wish for a longer effect? Ciaaliss can make it happen.

UNBELIEVABLE. CIIALISS AT ONLY $2.00 P/D

Say NO. to planning seex for hours in advance and your disappointment after "a missed dose". You don't have to be content with a four hour action of an regular pill. Get CIALIS now to ensure the whole day of joyful sexual intimacy for you and your partner.
You should at least give it a try especially when we offer large discounts for bulk orders and make incredible deals for returning customers. NOW
http://smart-pharmacy.com/15/




From Mailscan at eagle.org  Mon Apr  5 00:29:04 2004
From: Mailscan at eagle.org (Content Filter)
Date: Mon Apr  5 00:29:04 2004
Subject: [Numpy-discussion] Undeliverable message returned to sender.
Message-ID: <20040405072817.7E2EA2CCB53@mail24-red-R.bigfish.com>

This message was created automatically by mail delivery software.

Delivery failed for the following recipients(s):
 	fm at eqe.fr

The email message referenced below contained an executable attachment.  Since such attachments can contain dangerous or malicious code, the message and it?s attachment has been blocked. In the future please use non-executable file attachments whenever possible to avoid delays. See below for instructions on how to send this file to your ABS contact.
If the message also contained a virus, the sender should also receive a separate notification.  The name(s) of the blocked file(s) follow:
	application.pif

To send this file, please place it in a compressed archive using WinZip (.zip file extension) [http://www.winzip.com] or the archive software of your choice, and resend the message.

Thank you.

MailScanner Administrator: mailscan at eagle.org
ABS Customer Service Center: CSC at eagle.org

----- Original Message Header -----
Received: by mail24-red (MessageSwitch) id 1081150097256744_26344; Mon,  5 Apr 2004 07:28:17 +0000 (UCT)
Received: from eqe.fr (AVelizy-112-1-3-152.w193-253.abo.wanadoo.fr [193.253.228.152])
	by mail24-red.bigfish.com (Postfix) with ESMTP id 2818F2CCBC3
	for <fm at eqe.fr>; Mon,  5 Apr 2004 07:28:13 +0000 (UCT)
From: numpy-discussion at lists.sourceforge.net
To: fm at eqe.fr
Subject: Re: Your software
Date: Mon, 5 Apr 2004 09:28:26 +0200
MIME-Version: 1.0
Content-Type: multipart/mixed;
	boundary="----=_NextPart_000_0000_00005C9C.00003D49"
X-Priority: 3
X-MSMail-Priority: Normal
Message-Id: <20040405072813.2818F2CCBC3 at mail24-red.bigfish.com>



From faheem at email.unc.edu  Wed Apr  7 20:03:25 2004
From: faheem at email.unc.edu (Faheem Mitha)
Date: Wed Apr  7 20:03:25 2004
Subject: [Numpy-discussion] problems with compiling and loading C++ extension
Message-ID: <Pine.LNX.4.58.0404072259470.8108@Chrestomanci>

Dear People,

I have been having an odd problem with compiling and loading a simple
C++ extension to python (as a .so file in Linux). Unfortunately, I'll
need to include my files in here so I apologize in advance for the
length of this message.

You can download also the files below from
http://www.stat.unc.edu/students/faheem/python/python.tar.gz in case
that is more convenient.

This needs blitz++ and python's distutils and numarray to compile.

When I try to load arrmod.so into python I get

In [1]: import arrmod
---------------------------------------------------------------------------
ImportError                               Traceback (most recent call
last)

/tmp/wc/corrmodel/ex/<console>

ImportError: ./arrmod.so: undefined symbol:
_Z11py_to_blitzIdLi2EEN5blitz5ArrayIT_XT0_EEEP13PyArrayObject

If I move the py_to_blitz routine from conv.cc back into arrmod.cc,
the error disappears, arrmod loads in python, and everything works
correctly. I'm not sure what is going on here. Can someone enlighten
me? At first I thought it might have to do with the need for C
linkage, but I tried that, and it does not appear to be the case.

I also got the following compilation warning, which I don't
understand. I don't know if that is relevant.

---------------------------------------------------------------------
gcc -pthread -fno-strict-aliasing -DNDEBUG -g -O3 -Wall
-Wstrict-prototypes -fPIC -I/usr/include/python2.3 -c conv.cc -o
/tmp/temp.linux-i686-2.3/conv.o -Wall
/usr/include/python2.3/Numeric/arrayobject.h:286: warning:
`void**PyArray_API'
   defined but not used
---------------------------------------------------------------------

Thanks in advance for any enlightenment.

                                                 Faheem.

**********************************************************************
Makefile
**********************************************************************
arrmod.so: arrmod.cc common.hh conv.cc
	python setup.py build --build-base=/tmp --build-lib=.
# may not do the right thing for everyone
clean:
	rm arrmod.so
	rm -r /tmp/temp.linux-i686-2.3
**********************************************************************

**********************************************************************
setup.py
**********************************************************************
from distutils.core import setup, Extension

module4 = Extension('arrmod',
                    sources = ['arrmod.cc','conv.cc'],libraries=["blitz","m"],
                    extra_compile_args = ['-Wall'])

setup (name = 'arrmod',
       version = '1.0',
       description = 'This module performs different operations on arrays',
       ext_modules = [module4]
       )
**********************************************************************

**********************************************************************
common.hh
**********************************************************************
#include "Python.h"
#include "Numeric/arrayobject.h"
#include <blitz/array.h>

using namespace std;
using namespace blitz;

template<class T, int N>
static Array<T,N> py_to_blitz(PyArrayObject* arr_obj);
**********************************************************************

**********************************************************************
conv.cc
**********************************************************************
#include "common.hh"

// Convert a Numpy array to a blitz one, using the original's data (no
// copy)
template<class T, int N>
static Array<T,N> py_to_blitz(PyArrayObject* arr_obj)
{
  int T_size = sizeof(T);
  TinyVector<int,N> shape(0);
  TinyVector<int,N> strides(0);
  int *arr_dimensions = arr_obj->dimensions;
  int *arr_strides = arr_obj->strides;

  for (int i=0;i<N;++i)
    {
      shape[i]   = arr_dimensions[i];
      strides[i] = arr_strides[i]/T_size;
    }
  return Array<T,N>((T*) arr_obj->data,shape,strides,neverDeleteData);
}
************************************************************************

************************************************************************
arrmod.cc
************************************************************************
#include "common.hh"

template<class T, int N>
static Array<T,N> py_to_blitz(PyArrayObject* arr_obj);

static PyObject * arrmod_elem(PyObject *self, PyObject *args);

static PyMethodDef arrmod_methods[] =
  {
    {"elem", (PyCFunction)arrmod_elem, METH_VARARGS, "Returns the trace of a two-dimensional array.\n"},
    {NULL, NULL, 0, NULL}
  };

PyMODINIT_FUNC initarrmod(void)
{
  (void) Py_InitModule3("arrmod", arrmod_methods, "Returns the Trace of a two-dimensional array.\n");
  import_array();
}

static PyObject * arrmod_elem(PyObject *self, PyObject *args)
{
  PyObject *input, *result;
  PyArrayObject *array;
  double el;
  int i, j;

  if (!PyArg_ParseTuple(args, "Oii", &input, &i, &j))
    return NULL;
  array = (PyArrayObject *)
    PyArray_ContiguousFromObject(input, PyArray_DOUBLE, 2, 2);
  if (array == NULL)
    return NULL;

  Array<double,2> arr = py_to_blitz<double,2>(array);

  el = arr(i,j);
  result = Py_BuildValue("d",el);
  return result;
}
********************************************************************************



From fperez at colorado.edu  Wed Apr  7 21:26:10 2004
From: fperez at colorado.edu (Fernando Perez)
Date: Wed Apr  7 21:26:10 2004
Subject: [Numpy-discussion] problems with compiling and loading C++ extension
In-Reply-To: <Pine.LNX.4.58.0404072259470.8108@Chrestomanci>
References: <Pine.LNX.4.58.0404072259470.8108@Chrestomanci>
Message-ID: <4074D44D.3010806@colorado.edu>

Faheem Mitha wrote:
> Dear People,
> 
> I have been having an odd problem with compiling and loading a simple
> C++ extension to python (as a .so file in Linux). Unfortunately, I'll
> need to include my files in here so I apologize in advance for the
> length of this message.

Are you sure you have all the needed 'extern C' declarations in place? 
Remember, you'll be compiling this thing with a C++ compiler, but it will be 
called by Python's C runtime.  Hence, _all_ python-visible functions will need 
to be wrapped in extern C calls, including your module initializer (you often 
don't think of this one, because it's implicitly

Here's a typical snippet for a python-visible function in one of my C++ 
blitz-based extensions:

// mts_contract
extern "C" {
     static PyObject* mts_contract(PyObject *self, PyObject *args)
     {

         PyArrayObject *M_array, *T_array;
...

and here's what the module initializer looks like:


/* Module initializer -- called on first import.
    It *must* be called initMODULENAME */
extern "C" void initutils_()
{
     import_array();

     // Initialize the dimension dispatch tables
     init_mts_contract_funcs();
     init_mtt_contract2_funcs();
     init_mseqtt_contract2_funcs();

     // Create the module and add the functions
     PyObject* mod = Py_InitModule4("utils_",
                                    compiled_methods,
                                    module_doc,
                                    (PyObject*)NULL,
                                    PYTHON_API_VERSION);

     // Add some symbolic constants to the module
     PyObject* mod_dict = PyModule_GetDict(mod);

     // Make a module-level exception
     // The first argument is what the exception class will appear to be
     // in python. The return value is a class object.
     utils_Error = PyErr_NewException("utils_.utils_Error",
                                      NULL, NULL);

     // Add it to the module's dictionary. Second argument is the key
     PyDict_SetItemString(mod_dict, "utils_Error", utils_Error);

     // Add other constants here
     PyDict_SetItemString(mod_dict, "__all__",
                          Py_BuildValue("sss",
                                        "mts_contract2",
                                        "mtt_contract2",
                                        "mseqtt_contract2"
                                        ));

     // Check for errors
     if (PyErr_Occurred())
         Py_FatalError("can't initialize module utils_");
}


I hope this helps you.  Mail me if you want the full files (this is old code I 
don't use anymore, so it may not quite compile now anymore, but you can still 
look a it).

Cheers,

f




From faheem at email.unc.edu  Thu Apr  8 00:31:02 2004
From: faheem at email.unc.edu (Faheem Mitha)
Date: Thu Apr  8 00:31:02 2004
Subject: [Numpy-discussion] problems with compiling and loading C++
 extension
In-Reply-To: <4074D44D.3010806@colorado.edu>
References: <Pine.LNX.4.58.0404072259470.8108@Chrestomanci>
 <4074D44D.3010806@colorado.edu>
Message-ID: <Pine.LNX.4.58.0404080305570.8108@Chrestomanci>


On Wed, 7 Apr 2004, Fernando Perez wrote:

> I hope this helps you.  Mail me if you want the full files (this is old code I
> don't use anymore, so it may not quite compile now anymore, but you can still
> look a it).

Yes, please do send me your files (preferably as a tar.gz). Thanks for the
info. I expect you are right. I'll take a more careful look. One problem
is that I don't understand the logic behind C linkage very well. For
example, does it suffice to wrap the declaration of the function or do I
need to wrap the definition too?

Can you suggest a good reference, preferably online? My C++ books are not
much help with this.

Thanks.

                                                                Faheem.



From owen at astro.washington.edu  Fri Apr  9 10:26:09 2004
From: owen at astro.washington.edu (Russell E Owen)
Date: Fri Apr  9 10:26:09 2004
Subject: [Numpy-discussion] Questions about C API
Message-ID: <p06100502bc9c894aa9a5@[128.95.99.44]>

I'm trying to write a median function (surely one is there already, 
but if so I've not found it). The fastest technique seems to involve 
rearranging the data in place. So I'd like to offer two calls:
- A version that rearranges the input array if no local copy needed 
to be made (e.g. the input data was already contiguous and of the 
expected type). I know how to write that one.
- A safer, slower version that always makes a copy of the input data. 
I'm stuck on that. The high-level interface has flags for when to 
make a local copy, but none of them seems to mean "always make a 
local copy". Surely I am missing something there? I guess I could get 
the data, see if a local copy was made and if not, make a copy of my 
own, but that seems messy.

Also, is there any simple sample code running around for handling 
arrays of any type (e.g. with C++ templates)? Right now I'm writing 
to a specific type (since it's easy and I know what type to expect 
for my application), but I'd rather make my code more general and 
numarray-like. I figured the numarray code itself would be a good 
example, but, well, I didn't understand the parts I looked at (maybe 
I'm clueless, maybe I didn't find the right bits to examine).

-- Russell



From jdhunter at ace.bsd.uchicago.edu  Fri Apr  9 10:44:01 2004
From: jdhunter at ace.bsd.uchicago.edu (John Hunter)
Date: Fri Apr  9 10:44:01 2004
Subject: [Numpy-discussion] Questions about C API
In-Reply-To: <p06100502bc9c894aa9a5@[128.95.99.44]> (Russell E Owen's
 message of "Fri, 9 Apr 2004 10:25:10 -0700")
References: <p06100502bc9c894aa9a5@[128.95.99.44]>
Message-ID: <m2u0ztkqz6.fsf@mother.paradise.lost>

>>>>> "Russell" == Russell E Owen <owen at astro.washington.edu> writes:

    Russell> I'm trying to write a median function (surely one is
    Russell> there already, but if so I've not found it). 

from MLab import median

JDH



From jmiller at stsci.edu  Fri Apr  9 11:33:06 2004
From: jmiller at stsci.edu (Todd Miller)
Date: Fri Apr  9 11:33:06 2004
Subject: [Numpy-discussion] Questions about C API
In-Reply-To: <p06100502bc9c894aa9a5@[128.95.99.44]>
References: <p06100502bc9c894aa9a5@[128.95.99.44]>
Message-ID: <1081535554.13508.57.camel@halloween.stsci.edu>

On Fri, 2004-04-09 at 13:25, Russell E Owen wrote:
> I'm trying to write a median function (surely one is there already, 
> but if so I've not found it). The fastest technique seems to involve 
> rearranging the data in place. 

You might want to try out numarray.image.median (or
numarray.image.combine.median for versions < 0.9).

> So I'd like to offer two calls:
> - A version that rearranges the input array if no local copy needed 
> to be made (e.g. the input data was already contiguous and of the 
> expected type). I know how to write that one.
> - A safer, slower version that always makes a copy of the input data. 
> I'm stuck on that. The high-level interface has flags for when to 
> make a local copy, but none of them seems to mean "always make a 
> local copy". Surely I am missing something there? I guess I could get 
> the data, see if a local copy was made and if not, make a copy of my 
> own, but that seems messy.

The high level API is set up to do the least amount of work to guarantee
a set of constraints.  You can independently constrain numarray
properties (contiguousness, byteswap, alignment) and the array type to
be what you know your code can process.  The high level API will create
or convert arrays as necessary, and then release them when the function
is done processing (provided you manage your ref counts correctly as
shown in the manual).  So, the gist of the high level API is that it
takes some mysterious object (a poorly behaved array, an array of the
wrong type, nested lists, etc.) and converts it into a nicely formed
proxy array on which you do your computing.  In the case where the
mysterious object actually was a well behaved properly typed array,
almost no work has to be done and the proxy is the original array.

If you need a copy because your code is going to destroy the input
array,  use NA_copy() or just do a .copy() method call yourself.  Making
this kind of copy could indeed be folded into the existing API and it
would result in simpler application code, but it isn't there now.

> 
> Also, is there any simple sample code running around for handling 
> arrays of any type (e.g. with C++ templates)? 

I think Peter Verveer did some code like this in the nd_image package. 
The general idea is to constrain the arrays with type tAny in the high
level API and then switch off of a->descr->type_num to select type
specific code.

Regards,
Todd

> -- 
> Todd Miller <jmiller at stsci.edu>




From owen at astro.washington.edu  Fri Apr  9 11:49:44 2004
From: owen at astro.washington.edu (Russell E Owen)
Date: Fri Apr  9 11:49:44 2004
Subject: [Numpy-discussion] Questions about C API
In-Reply-To: <1081535554.13508.57.camel@halloween.stsci.edu>
References: <p06100502bc9c894aa9a5@[128.95.99.44]>
 <1081535554.13508.57.camel@halloween.stsci.edu>
Message-ID: <p06100505bc9c9f13c4f6@[128.95.99.44]>

At 2:32 PM -0400 2004-04-09, Todd Miller wrote:
>On Fri, 2004-04-09 at 13:25, Russell E Owen wrote:
>>  I'm trying to write a median function (surely one is there already,
>>  but if so I've not found it). The fastest technique seems to involve
>>  rearranging the data in place.
>
>You might want to try out numarray.image.median (or
>numarray.image.combine.median for versions < 0.9).

Thanks. Based on the source, it looks like that prefers a stack of 
identical images (instead of a single array, though perhaps one could 
make that case work).

John Hunter also kindly pointed out MLab (which it turns out is 
buried under linear_algebra in numarray: 
numarray.linear_algebra.mlab).

I had been working on a fast median instead of one that sorted the 
whole array, but after timing mlab's median I can probably get away 
with the simple implementation (and skip the C extension).

>>  Also, is there any simple sample code running around for handling
>>  arrays of any type (e.g. with C++ templates)?
>
>I think Peter Verveer did some code like this in the nd_image package.
>The general idea is to constrain the arrays with type tAny in the high
>level API and then switch off of a->descr->type_num to select type
>specific code.

Thank you very much for the explanation of the high level interface 
and the pointer to this sample code.

-- Russell



From Nicolas.Chauvat at logilab.fr  Sat Apr 10 14:00:10 2004
From: Nicolas.Chauvat at logilab.fr (Nicolas Chauvat)
Date: Sat Apr 10 14:00:10 2004
Subject: [Numpy-discussion] Europython: Registration open - Talk submission deadline is Apr 15th
Message-ID: <20040410205917.GA10699@logilab.fr>

Dear Scientific Pythonistas,

I'm forwarding this update about the EuroPython conference. Please
note that the talk submission deadline is April 15th but will probably
be extended a bit. I suppose most of you will be interested in the
Science Track at least.

For previous years' program, please refer to :

	http://www.europython.org/2002/sessions/talks
	http://www.europython.org/2003/sessions/talks

---------------------------------------------------------------------
Subject: [EuroPython] Europython Update: Registration open

Europython Update
=================

- Registration is now open. We apologise for the delay, but we have
had some technical problems.

- Due to this, we have decided to keep the submission of abstracts
for the refereed track open for one more day. Last submission time is
now on Sunday 11 April at 23.59 CET.

- We have a limited number of beds available in very affordable
accomodation near the conference venue. Book early before it runs out.

- We are still receiving submissions for regular talks and tutorials.
Closing date is 15 April.

- There is now a wiki at the Europython website for sprint organising. 
Start planning!

About the conference
====================

EuroPython 2004 will be held 7-9 June in G?teborg, Sweden.

The EuroPython conference will have tracks for Science, Business,
Education, Applications, Frameworks, Zope and the Python language
itself. Lightning talks, Open Space and BOF sessions are also planned.
There will be tutorials as well, both for newcomers to Python and
Python users interested in special subjects. In the days before and
after the conference, programming sprints will be arranged.


Important dates
===============

Refereed paper proposals: until 11 April.
Submission of talks: 1 March - 15 April.
Early Bird registration: 9 April - 1 May.
Accomodation booking: 9 April - 1 May (or until space runs out)

More information at http://www.europython.org.

---------------------------------------------------------------------

Hope to see you there.


-- 
Nicolas Chauvat

logilab.fr - services en informatique avanc?e et gestion de connaissances  



From jiiyso84sjoj at webnexus.com  Sat Apr 10 17:22:12 2004
From: jiiyso84sjoj at webnexus.com (Audrey Shanice)
Date: Sat Apr 10 17:22:12 2004
Subject: [Numpy-discussion] sa.ve on software?? this is for u orchard pterygium
Message-ID: <jqeeu86-cpk8-q4jiiyso84sjoj@webnexus.com>

commando arthralgic precipitin endenizen 
vicksburg seawater pretence 
choler sickbay brassiere xerobates 
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20040410/db039215/attachment.html>

From faheem at email.unc.edu  Wed Apr 14 22:14:06 2004
From: faheem at email.unc.edu (Faheem Mitha)
Date: Wed Apr 14 22:14:06 2004
Subject: [Numpy-discussion] take for character arrays?
Message-ID: <slrnc7s6fp.1gd.faheem@Chrestomanci.home.earth>

Hi,

I'm trying to extract subarrays of a character array using take(), but
getting mysterious errors, for example:

ValueError: Invalid destination array: partial indices require
contiguous non-byteswapped destination

I am thinking that perhaps this operation is not supported for
character arrays, and I should convert my character array to a numeric
array instead (the strings are all of length 1).

Advice, suggestions, enlightenment appreciated.

                                                          Faheem.




From faheem at email.unc.edu  Thu Apr 15 10:07:15 2004
From: faheem at email.unc.edu (Faheem Mitha)
Date: Thu Apr 15 10:07:15 2004
Subject: [Numpy-discussion] Re: take for character arrays?
References: <slrnc7s6fp.1gd.faheem@Chrestomanci.home.earth>
Message-ID: <slrnc7tg8d.1gd.faheem@Chrestomanci.home.earth>

On Thu, 15 Apr 2004 05:13:29 +0000 (UTC), Faheem Mitha
<faheem at email.unc.edu> wrote:

> Hi,
>
> I'm trying to extract subarrays of a character array using take(), but
> getting mysterious errors, for example:
>
> ValueError: Invalid destination array: partial indices require
> contiguous non-byteswapped destination
>
> I am thinking that perhaps this operation is not supported for
> character arrays, and I should convert my character array to a numeric
> array instead (the strings are all of length 1).

I read this again and realised this does not give any specific
information, and is not very coherent. Got to avoid those late night
postings. So here are the details. If I set axis = 0 there is no
problem. I also don't have this problem with numerical arrays.

I am sure there is a simple reason for this, and hopefully a simple
workaround.

Thanks in advance.

                                                          Faheem.

**************************************************************************
In [22]: import numarray.strings as str

In [23]: import numarray

In [24]: s = str.array(['a', 'b', 'c', 'd', 'e', 'f', 'g', 'h',
'i'],shape=(3,3))

In [25]: numarray.take(s,(1,2),axis=1)
---------------------------------------------------------------------------
ValueError                                Traceback (most recent call
last)

/home/faheem/wc/corrmodel/boost/<console>

/usr/lib/python2.3/site-packages/numarray/ufunc.py in take(array,
indices, axis, outarr, clipmode)
   1779             raise ValueError("indices must be a sequence")
   1780         work = _gen.swapaxes(array, 0, axis)
-> 1781         work = work._take((indices,), outarr=outarr,
clipmode=clipmode)
   1782         return _gen.swapaxes(work, 0, axis)
   1783     else:

/usr/lib/python2.3/site-packages/numarray/generic.py in _take(self,
indices, **keywds)
    795     def _take(self, indices, **keywds):
    796         indices = list(indices)
--> 797         impliedShape, N = _takeShape(self, indices)
    798         result = self._clone(shape=impliedShape)
    799         indices = self._fix_pt_indices(indices)

/usr/lib/python2.3/site-packages/numarray/generic.py in
_takeShape(scattered, indexArrays)
    133         nShape = scattered._shape[-leftOver:]
    134         if scattered.isbyteswapped() or not
scattered.iscontiguous():
--> 135             raise ValueError("Invalid destination array:
partial indices require contiguous non-byteswapped destination")
    136     else:
    137         N = scattered._itemsize

ValueError: Invalid destination array: partial indices require
contiguous non-byteswapped destination





From jmiller at stsci.edu  Thu Apr 15 12:56:01 2004
From: jmiller at stsci.edu (Todd Miller)
Date: Thu Apr 15 12:56:01 2004
Subject: [Numpy-discussion] Re: take for character arrays?
In-Reply-To: <slrnc7tg8d.1gd.faheem@Chrestomanci.home.earth>
References: <slrnc7s6fp.1gd.faheem@Chrestomanci.home.earth>
	 <slrnc7tg8d.1gd.faheem@Chrestomanci.home.earth>
Message-ID: <1082058888.18786.53.camel@halloween.stsci.edu>

On Thu, 2004-04-15 at 13:06, Faheem Mitha wrote:
> On Thu, 15 Apr 2004 05:13:29 +0000 (UTC), Faheem Mitha
> <faheem at email.unc.edu> wrote:
> 
> > Hi,
> >
> > I'm trying to extract subarrays of a character array using take(), but
> > getting mysterious errors, for example:
> >
> > ValueError: Invalid destination array: partial indices require
> > contiguous non-byteswapped destination
> >
> > I am thinking that perhaps this operation is not supported for
> > character arrays, and I should convert my character array to a numeric
> > array instead (the strings are all of length 1).

This is a bug.  I logged it on source forge.

> 
> I read this again and realised this does not give any specific
> information, and is not very coherent. Got to avoid those late night
> postings. So here are the details. If I set axis = 0 there is no
> problem. I also don't have this problem with numerical arrays.
> 
> I am sure there is a simple reason for this, and hopefully a simple
> workaround.

Try this for a workaround:

s.swapaxes(0,1)
s = s.copy()
r = numarray.take(s, (1,2))
r.swapaxes(0,1)

Regards,
Todd

-- 
Todd Miller <jmiller at stsci.edu>




From faheem at email.unc.edu  Thu Apr 15 13:21:11 2004
From: faheem at email.unc.edu (Faheem Mitha)
Date: Thu Apr 15 13:21:11 2004
Subject: [Numpy-discussion] Re: take for character arrays?
In-Reply-To: <1082058888.18786.53.camel@halloween.stsci.edu>
References: <slrnc7s6fp.1gd.faheem@Chrestomanci.home.earth> 
 <slrnc7tg8d.1gd.faheem@Chrestomanci.home.earth> <1082058888.18786.53.camel@halloween.stsci.edu>
Message-ID: <Pine.LNX.4.58.0404151609240.19516@Chrestomanci>


On Thu, 15 Apr 2004, Todd Miller wrote:

> Try this for a workaround:
>
> s.swapaxes(0,1)
> s = s.copy()
> r = numarray.take(s, (1,2))
> r.swapaxes(0,1)

I was trying

In [50]: ts = numarray.transpose(s)

In [51]: numarray.take(ts,(1,2),axis=0)
...
ValueError: Invalid destination array: partial indices require
contiguous non-byteswapped destination

In [52]: tscopy = numarray.transpose(s).copy()

In [53]: numarray.take(tscopy,(1,2),axis=0)
Out[53]:
CharArray([['b', 'e', 'h'],
           ['c', 'f', 'i']])

Is this the same or a related bug? Thanks for the reply and info.

                                                        Faheem.



From jmiller at stsci.edu  Thu Apr 15 13:29:11 2004
From: jmiller at stsci.edu (Todd Miller)
Date: Thu Apr 15 13:29:11 2004
Subject: [Numpy-discussion] Re: take for character arrays?
In-Reply-To: <Pine.LNX.4.58.0404151609240.19516@Chrestomanci>
References: <slrnc7s6fp.1gd.faheem@Chrestomanci.home.earth>
	 <slrnc7tg8d.1gd.faheem@Chrestomanci.home.earth>
	 <1082058888.18786.53.camel@halloween.stsci.edu>
	 <Pine.LNX.4.58.0404151609240.19516@Chrestomanci>
Message-ID: <1082060853.18786.56.camel@halloween.stsci.edu>

On Thu, 2004-04-15 at 16:17, Faheem Mitha wrote:
> On Thu, 15 Apr 2004, Todd Miller wrote:
> 
> > Try this for a workaround:
> >
> > s.swapaxes(0,1)
> > s = s.copy()
> > r = numarray.take(s, (1,2))
> > r.swapaxes(0,1)
> 
> I was trying
> 
> In [50]: ts = numarray.transpose(s)
> 
> In [51]: numarray.take(ts,(1,2),axis=0)
> ...
> ValueError: Invalid destination array: partial indices require
> contiguous non-byteswapped destination
> 
> In [52]: tscopy = numarray.transpose(s).copy()
> 
> In [53]: numarray.take(tscopy,(1,2),axis=0)
> Out[53]:
> CharArray([['b', 'e', 'h'],
>            ['c', 'f', 'i']])
> 
> Is this the same or a related bug? 

Same bug.  Making a copy of the transposed array ensures that it (the
copy) is contiguous.

Regards,
Todd

-- 
Todd Miller <jmiller at stsci.edu>




From faheem at email.unc.edu  Thu Apr 15 14:03:23 2004
From: faheem at email.unc.edu (Faheem Mitha)
Date: Thu Apr 15 14:03:23 2004
Subject: [Numpy-discussion] Re: take for character arrays?
In-Reply-To: <1082060853.18786.56.camel@halloween.stsci.edu>
References: <slrnc7s6fp.1gd.faheem@Chrestomanci.home.earth> 
 <slrnc7tg8d.1gd.faheem@Chrestomanci.home.earth>  <1082058888.18786.53.camel@halloween.stsci.edu>
  <Pine.LNX.4.58.0404151609240.19516@Chrestomanci> <1082060853.18786.56.camel@halloween.stsci.edu>
Message-ID: <Pine.LNX.4.58.0404151639070.19516@Chrestomanci>


On Thu, 15 Apr 2004, Todd Miller wrote:

> On Thu, 2004-04-15 at 16:17, Faheem Mitha wrote:
> > On Thu, 15 Apr 2004, Todd Miller wrote:
> >
> > > Try this for a workaround:
> > >
> > > s.swapaxes(0,1)
> > > s = s.copy()
> > > r = numarray.take(s, (1,2))
> > > r.swapaxes(0,1)
> >
> > I was trying
> >
> > In [50]: ts = numarray.transpose(s)
> >
> > In [51]: numarray.take(ts,(1,2),axis=0)
> > ...
> > ValueError: Invalid destination array: partial indices require
> > contiguous non-byteswapped destination
> >
> > In [52]: tscopy = numarray.transpose(s).copy()
> >
> > In [53]: numarray.take(tscopy,(1,2),axis=0)
> > Out[53]:
> > CharArray([['b', 'e', 'h'],
> >            ['c', 'f', 'i']])
> >
> > Is this the same or a related bug?
>
> Same bug.  Making a copy of the transposed array ensures that it (the
> copy) is contiguous.

I am wondering if this bug affects boost-python as well. I got the same
behaviour when using boost.python's version of take.

Can you point to where in the source code the bug is? Thanks.

                                                           Faheem.



From jmiller at stsci.edu  Thu Apr 15 14:44:20 2004
From: jmiller at stsci.edu (Todd Miller)
Date: Thu Apr 15 14:44:20 2004
Subject: [Numpy-discussion] Re: take for character arrays?
In-Reply-To: <1082064038.18786.85.camel@halloween.stsci.edu>
References: <slrnc7s6fp.1gd.faheem@Chrestomanci.home.earth>
	 <slrnc7tg8d.1gd.faheem@Chrestomanci.home.earth>
	 <1082058888.18786.53.camel@halloween.stsci.edu>
	 <Pine.LNX.4.58.0404151609240.19516@Chrestomanci>
	 <1082060853.18786.56.camel@halloween.stsci.edu>
	 <Pine.LNX.4.58.0404151639070.19516@Chrestomanci>
	 <1082064038.18786.85.camel@halloween.stsci.edu>
Message-ID: <1082065422.18786.94.camel@halloween.stsci.edu>

This problem is now fixed in CVS.

Regards,
Todd
-- 
Todd Miller <jmiller at stsci.edu>




From jmiller at stsci.edu  Thu Apr 15 14:58:25 2004
From: jmiller at stsci.edu (Todd Miller)
Date: Thu Apr 15 14:58:25 2004
Subject: [Numpy-discussion] Re: take for character arrays?
In-Reply-To: <Pine.LNX.4.58.0404151639070.19516@Chrestomanci>
References: <slrnc7s6fp.1gd.faheem@Chrestomanci.home.earth>
	 <slrnc7tg8d.1gd.faheem@Chrestomanci.home.earth>
	 <1082058888.18786.53.camel@halloween.stsci.edu>
	 <Pine.LNX.4.58.0404151609240.19516@Chrestomanci>
	 <1082060853.18786.56.camel@halloween.stsci.edu>
	 <Pine.LNX.4.58.0404151639070.19516@Chrestomanci>
Message-ID: <1082064038.18786.85.camel@halloween.stsci.edu>

On Thu, 2004-04-15 at 16:40, Faheem Mitha wrote:
> On Thu, 15 Apr 2004, Todd Miller wrote:
> 
> > On Thu, 2004-04-15 at 16:17, Faheem Mitha wrote:
> > > On Thu, 15 Apr 2004, Todd Miller wrote:
> > >
> > > > Try this for a workaround:
> > > >
> > > > s.swapaxes(0,1)
> > > > s = s.copy()
> > > > r = numarray.take(s, (1,2))
> > > > r.swapaxes(0,1)
> > >
> > > I was trying
> > >
> > > In [50]: ts = numarray.transpose(s)
> > >
> > > In [51]: numarray.take(ts,(1,2),axis=0)
> > > ...
> > > ValueError: Invalid destination array: partial indices require
> > > contiguous non-byteswapped destination
> > >
> > > In [52]: tscopy = numarray.transpose(s).copy()
> > >
> > > In [53]: numarray.take(tscopy,(1,2),axis=0)
> > > Out[53]:
> > > CharArray([['b', 'e', 'h'],
> > >            ['c', 'f', 'i']])
> > >
> > > Is this the same or a related bug?
> >
> > Same bug.  Making a copy of the transposed array ensures that it (the
> > copy) is contiguous.
> 
> I am wondering if this bug affects boost-python as well. I got the same
> behaviour when using boost.python's version of take.

The bug shouldn't affect numerical arrays because there are different
underlying implementations of take.

> Can you point to where in the source code the bug is? Thanks.

I think the relevant modules are numarray.generic and numarray.ufunc. 
There's a support function which computes the shape of the take() result
in generic.py, _takeShape().  It's testing for a property,
contiguousness, which is not required by the generic take code but is
required by the numerical code.  

The fix may be to factor the exception out of _takeShape and put it into
the (numerical) ufunc.put() setup code somewhere. 

Regards,
Todd

-- 
Todd Miller <jmiller at stsci.edu>




From MAILER-DAEMON at lackingtalent.com  Thu Apr 15 21:02:07 2004
From: MAILER-DAEMON at lackingtalent.com (MAILER-DAEMON at lackingtalent.com)
Date: Thu Apr 15 21:02:07 2004
Subject: [Numpy-discussion] failure notice
Message-ID: <E1BEKXV-0005zG-1U@sc8-sf-mx1.sourceforge.net>

Hi. This is the qmail-send program at lackingtalent.com.
I'm afraid I wasn't able to deliver your message to the following addresses.
This is a permanent error; I've given up. Sorry it didn't work out.

<slrnbqbpqt.vf5.ramen at lackingtalent.com>:
Sorry, no mailbox here by that name. vpopmail (#5.1.1)

--- Below this line is a copy of the message.


From numpy-discussion at lists.sourceforge.net  Fri Apr 16 09:01:44 2004
From: numpy-discussion at lists.sourceforge.net (numpy-discussion at lists.sourceforge.net)
Date: Fri, 16 Apr 2004 06:01:44 -0700
Subject: Delivery Bot (slrnbqbpqt.vf5.ramen@lackingtalent.com)
Message-ID: <mailman.14.1490332306.18468.numpy-discussion@python.org>

An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20040416/2436fe96/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: message.pif
Type: audio/x-wav
Size: 28008 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20040416/2436fe96/attachment.wav>

From owen at astro.washington.edu  Fri Apr 16 13:41:01 2004
From: owen at astro.washington.edu (Russell E Owen)
Date: Fri Apr 16 13:41:01 2004
Subject: [Numpy-discussion] announcing a ds9 module and a question about indices
Message-ID: <p0610050abca5f1500f17@[128.95.99.44]>

I wrote a simple module that allows one to display numarray arrays in 
ds9. It is available as part of the RO package (RO.DS9) 
<http://www.astro.washington.edu/owen/ROPython.html>, though it is 
presently self-contained. (I may well end up expanding it.) It sends 
a copy of the data, rather than trying to share a buffer.

It uses the command-line XPA interface instead of trying to use the 
underlying C XPA API. I'm curious if anybody has gotten a python 
interface to the C XPA API working. It looks a bit messy, but popen2 
messy in other ways (detecting errors without hanging on read, for 
example).



I have what is probably a dumb question, but...if I load an array 
using PyFits and display it using ds9, I find that the array indices 
appear to be transposed relative to what I expect, i.e. numarray 
seems to use array[yind, xind].

Is this normal? Any hints on handling notation in code (I suspect 
there's not much I can do except document it)?

-- Russell



From perry at stsci.edu  Fri Apr 16 15:03:01 2004
From: perry at stsci.edu (Perry Greenfield)
Date: Fri Apr 16 15:03:01 2004
Subject: [Numpy-discussion] announcing a ds9 module and a question about indices
In-Reply-To: <p0610050abca5f1500f17@[128.95.99.44]>
Message-ID: <CEELJPECNGEGKFNDLHFFGECICDAA.perry@stsci.edu>

Russel Owen wrote:
> I wrote a simple module that allows one to display numarray arrays in 
> ds9. It is available as part of the RO package (RO.DS9) 
> <http://www.astro.washington.edu/owen/ROPython.html>, though it is 
> presently self-contained. (I may well end up expanding it.) It sends 
> a copy of the data, rather than trying to share a buffer.
> 
> It uses the command-line XPA interface instead of trying to use the 
> underlying C XPA API. I'm curious if anybody has gotten a python 
> interface to the C XPA API working. It looks a bit messy, but popen2 
> messy in other ways (detecting errors without hanging on read, for 
> example).
> 
We distributed a simple module with PyRAF that could display
to DS9 (and other similar display tools, SAOIMAGE, ximtool, etc)
using the underlying protocol used by them (offhand, I don't know
if that was the XPA interface; I seem to recall it was the
iis protocol). Unfortunately, outside of the PyRAF announcement,
we never really publicized it. We'll take a look at yours.
It could be that it has more features.
> 
> I have what is probably a dumb question, but...if I load an array 
> using PyFits and display it using ds9, I find that the array indices 
> appear to be transposed relative to what I expect, i.e. numarray 
> seems to use array[yind, xind].
> 
> Is this normal? Any hints on handling notation in code (I suspect 
> there's not much I can do except document it)?
>
Yep, it's normal. The indexing convention in Python is the same
as C with regard to multidimensional arrays and opposite to that
of Fortran and IDL. There are many reasons for that and it would
cause more trouble than it is worth trying to emulate Fortran on 
this. I know it is probably the single biggest annoyance to those
not used to it. 

Perry Greenfield 



From faheem at email.unc.edu  Sat Apr 17 16:07:07 2004
From: faheem at email.unc.edu (Faheem Mitha)
Date: Sat Apr 17 16:07:07 2004
Subject: [Numpy-discussion] add.reduce for character arrays
Message-ID: <slrnc83e38.1gd.faheem@Chrestomanci.home.earth>

Hi,

The following does not work. I am trying to do something similar to
add.reduce, except that I want strings concatenated across the
appropriate axes. I was hoping that Python would "magically" recognise
that addition in strings translated to concatenation.

Incidentally, the error message is not terribly imformative.

Is there any way to make this work, or any other method people can
suggest, or do I need to write my own function? 

Thanks.
                                                     Regards, Faheem.

**********************************************************************
In [43]: s
Out[43]:
CharArray([['a', 'b', 'c'],
           ['d', 'e', 'f'],
           ['g', 'h', 'i']])

In [44]: numarray.add.reduce(s,1)
---------------------------------------------------------------------------
error                                     Traceback (most recent call
last)

/home/faheem/wc/corrmodel/boost/<console>

error: getShape: sequence object nested more than MAXDIM




From jmiller at stsci.edu  Mon Apr 19 07:06:07 2004
From: jmiller at stsci.edu (Todd Miller)
Date: Mon Apr 19 07:06:07 2004
Subject: [Numpy-discussion] add.reduce for character arrays
In-Reply-To: <slrnc83e38.1gd.faheem@Chrestomanci.home.earth>
References: <slrnc83e38.1gd.faheem@Chrestomanci.home.earth>
Message-ID: <1082383540.19596.72.camel@halloween.stsci.edu>

On Sat, 2004-04-17 at 19:06, Faheem Mitha wrote:
> Hi,
> 
> The following does not work. I am trying to do something similar to
> add.reduce, except that I want strings concatenated across the
> appropriate axes. I was hoping that Python would "magically" recognise
> that addition in strings translated to concatenation.
> 

Use object arrays (numarray.objects) instead.  While CharArrays do have
an overloaded add operator they don't support reduction. Object arrays
have more general ufuncs which apply object oriented operators and also
support reduction.  In general, character arrays (numarray.strings) do
not support the numerical ufuncs found in numarray

> Incidentally, the error message is not terribly imformative.

True enough.  The numerical getShape code should recognize strings and
raise an exception pointing out that strings can't be passed into
numerical code.  This is fixed in CVS.

> Is there any way to make this work, or any other method people can
> suggest, or do I need to write my own function? 

Again, use numarray.objects.

> 
> Thanks.
>                                                      Regards, Faheem.
> 
> **********************************************************************
> In [43]: s
> Out[43]:
> CharArray([['a', 'b', 'c'],
>            ['d', 'e', 'f'],
>            ['g', 'h', 'i']])
> 
> In [44]: numarray.add.reduce(s,1)
> ---------------------------------------------------------------------------
> error                                     Traceback (most recent call
> last)
> 
> /home/faheem/wc/corrmodel/boost/<console>
> 
> error: getShape: sequence object nested more than MAXDIM
> 
> 
> 
> -------------------------------------------------------
> This SF.Net email is sponsored by: IBM Linux Tutorials
> Free Linux tutorial presented by Daniel Robbins, President and CEO of
> GenToo technologies. Learn everything from fundamentals to system
> administration.http://ads.osdn.com/?ad_id=1470&alloc_id=3638&op=click
> _______________________________________________
> Numpy-discussion mailing list
> Numpy-discussion at lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/numpy-discussion
-- 
Todd Miller <jmiller at stsci.edu>




From KateGillis at mails.ch  Mon Apr 19 12:28:22 2004
From: KateGillis at mails.ch (Monroe Bean)
Date: Mon Apr 19 12:28:22 2004
Subject: [Numpy-discussion] Sale on OEM software XP, office, Adobe, all under 25-99$ per CD from Bowen's SoftShop
Message-ID: <LPOVYCAAEKEQZFVYGEMCTJ@Xoom.de>

An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20040419/e37509c7/attachment.html>

From faheem at email.unc.edu  Mon Apr 19 20:21:25 2004
From: faheem at email.unc.edu (Faheem Mitha)
Date: Mon Apr 19 20:21:25 2004
Subject: [Numpy-discussion] add.reduce for character arrays
In-Reply-To: <1082383540.19596.72.camel@halloween.stsci.edu>
References: <slrnc83e38.1gd.faheem@Chrestomanci.home.earth>
 <1082383540.19596.72.camel@halloween.stsci.edu>
Message-ID: <Pine.LNX.4.58.0404192315010.25609@Chrestomanci>


On Mon, 19 Apr 2004, Todd Miller wrote:

> On Sat, 2004-04-17 at 19:06, Faheem Mitha wrote:
> > Hi,
> >
> > The following does not work. I am trying to do something similar to
> > add.reduce, except that I want strings concatenated across the
> > appropriate axes. I was hoping that Python would "magically" recognise
> > that addition in strings translated to concatenation.
> >
>
> Use object arrays (numarray.objects) instead.  While CharArrays do have
> an overloaded add operator they don't support reduction. Object arrays
> have more general ufuncs which apply object oriented operators and also
> support reduction.  In general, character arrays (numarray.strings) do
> not support the numerical ufuncs found in numarray
>
> > Incidentally, the error message is not terribly imformative.
>
> True enough.  The numerical getShape code should recognize strings and
> raise an exception pointing out that strings can't be passed into
> numerical code.  This is fixed in CVS.
>
> > Is there any way to make this work, or any other method people can
> > suggest, or do I need to write my own function?
>
> Again, use numarray.objects.

Yes, that works. Thanks.

In [13]: numarray.objects.add.reduce(s,dim=1)
Out[13]: ObjectArray(['abc', 'def', 'ghi'])

BTW, that makes me wonder what the rules are for the order in which this
reduction proceeds for arrays bigger than dimension two. Since there is no
total order in this case, the answer is not obvious. For string
concatentation (since it is not commutative) at least this matters. I
didn't see this documented in the manual but I may have missed it.

                                                              Faheem.



From jmiller at stsci.edu  Tue Apr 20 09:54:01 2004
From: jmiller at stsci.edu (Todd Miller)
Date: Tue Apr 20 09:54:01 2004
Subject: [Numpy-discussion] add.reduce for character arrays
In-Reply-To: <Pine.LNX.4.58.0404192315010.25609@Chrestomanci>
References: <slrnc83e38.1gd.faheem@Chrestomanci.home.earth>
	 <1082383540.19596.72.camel@halloween.stsci.edu>
	 <Pine.LNX.4.58.0404192315010.25609@Chrestomanci>
Message-ID: <1082480003.3843.330.camel@localhost.localdomain>

On Mon, 2004-04-19 at 23:20, Faheem Mitha wrote:
> 
> Yes, that works. Thanks.
> 
> In [13]: numarray.objects.add.reduce(s,dim=1)
> Out[13]: ObjectArray(['abc', 'def', 'ghi'])
> 
> BTW, that makes me wonder what the rules are for the order in which this
> reduction proceeds for arrays bigger than dimension two. Since there is no
> total order in this case, the answer is not obvious. 

(I think) the answer is that for numarray, reductions ultimately occur
along the innermost axis.  Off-axis reductions are performed by a
swapaxis between the off-axis and the innermost axis, followed by the
reduction, followed by a swapaxis back between the new innermost axis
and the off-axis.  I'm not sure there's an especially good reason for
the swap back (since the original swapped axis no longer exists),  but
it seems to be what Numeric does and not doing it in numarray broke
extension (linear_algebra?) self-tests.

> For string
> concatentation (since it is not commutative) at least this matters. I
> didn't see this documented in the manual but I may have missed it.

Reductions always occur in order of increasing indexes with the
"reduction-so-far" as the left operand and the next index as the right
operand.  Reductions occur in a sort of depth-first manner with the
outermost dimensions varying most slowly.

Regards,
Todd
-- 
Todd Miller <jmiller at stsci.edu>




From travis at enthought.com  Wed Apr 21 14:41:19 2004
From: travis at enthought.com (Travis N. Vaught)
Date: Wed Apr 21 14:41:19 2004
Subject: [Numpy-discussion] ANN: SciPy 0.3 Released
Message-ID: <4086EA48.4080105@enthought.com>

Greetings,

SciPy 0.3 has been released and binaries are available from the 
scipy.org site.

http://www.scipy.org

Changes since the 0.1 version (0.1 enjoyed a wide release, there was a 
version 0.2 that had limited exposure) include the following:

 - general improvements: Added utility functions for constructing arrays 
by concatenation, intended mainly for command-line use. Added bmat 
constructor for easy creation of block matrices. Added mat constructor 
for constructing matrices (where * is matrix multiplication). Added many 
PIL utility functions so that if the PIL is installed, image loading, 
saving, and other operations are available. Added scipy.info, which 
handles dynamic docstrings and class help better than python's help command.

 - documentation: much improved

 - sparse: superLU upgraded to 3.0

 - optimize: Added non-linear conjugate gradient algorithm. Improvements 
to BFGS algorithm and Wolfe-condition line-search. Added two CONSTRAINED 
optimization techniques. Added simulated annealing and brute-force 
optimization strategies and Powell's method. Added many very good 1-D 
root-finding routines.

 - stats: Added many statistical distributions. Many continuous and 
discrete random variables are available with a simple mechanism for 
adding new ones. Added several statistical tests.

 - special: Added MANY new special functions. |general_function| renamed 
to |vectorize| and moved to scipy_base. Improved the way domain errors 
are handled (user specified display of problems). More tests on special 
functions added.

 - fftpack: replaced with fftpack2--can optionally be configured to 
support djbfft

 - io: Reading of MATLAB .mat files (version 4 and 5) supported. Writing 
version 4 .mat files supported. Added very flexible and general_purpose 
text file reader. Added shelving save operation to save variables into 
an importable module. Routines for reading and writing fortran-records.

 - linalg: Linear algebra is greatly improved over SciPy 0.1. ATLAS is 
now optional (but encouraged). Most lapack functions have simplified 
interfaces (all lapack and blas available). Matrix exponentials and 
other matrix functions added.

 - signal: Added support for filter design and LTI system analysis.

 - xplt: Updated xplt to use newer pygist so that Windows platform is 
supported. Addition of several plotting types.

 - integrate: added another ODE integrator. Added romberg integration 
and several other integration approaches.


The complete release notes can be found here:

http://www.scipy.org/download/scipy_release_notes_0.3.html


You'll also notice that scipy.org is now a spanking new Plone portal 
(http://www.plone.org -- keep up the good work, plone folks).

This is the first binary release in a long time and we hope to increase 
the frequency to every 6 months or so.

If you'd like to follow or join the community, you can subscribe to the 
mailing lists here:

http://www.scipy.org/mailinglists/

Best Regards,

Travis


BTW SciPy is:
-------------
SciPy is an open source library of scientific tools for Python. SciPy 
supplements the popular Numeric module, gathering a variety of high 
level science and engineering modules together as a single package.

SciPy includes modules for graphics and plotting, optimization, 
integration, special functions, signal and image processing, genetic 
algorithms, ODE solvers, and others.




From ville at e.math.helsinki.fi  Thu Apr 22 05:56:17 2004
From: ville at e.math.helsinki.fi (Ville Hakulinen)
Date: Thu Apr 22 05:56:17 2004
Subject: [Numpy-discussion] Numarrays from C arrays without copying
Message-ID: <20040422125131.GA7003@e.math.helsinki.fi>

Dear all,

I would like to turn a large C array allocated by a library into a
Numarray object without copying the contents. In other words I have
a given array of double precision floats and I want to get a Numarray
object, where the data pointer points to this array and no additional
memory is allocated. Naturally when the reference count of the Numarray
object gets back to zero, the object is freed, but the array itself
is left as a nuisance for the C code.

Is there a standard way of doing this?

Thanks in advance,
-- 
Ville



From jmiller at stsci.edu  Thu Apr 22 06:11:02 2004
From: jmiller at stsci.edu (Todd Miller)
Date: Thu Apr 22 06:11:02 2004
Subject: [Numpy-discussion] Numarrays from C arrays without copying
In-Reply-To: <20040422125131.GA7003@e.math.helsinki.fi>
References: <20040422125131.GA7003@e.math.helsinki.fi>
Message-ID: <1082639406.3272.10.camel@halloween.stsci.edu>

On Thu, 2004-04-22 at 08:51, Ville Hakulinen wrote:
> Dear all,
> 
> I would like to turn a large C array allocated by a library into a
> Numarray object without copying the contents. In other words I have
> a given array of double precision floats and I want to get a Numarray
> object, where the data pointer points to this array and no additional
> memory is allocated. Naturally when the reference count of the Numarray
> object gets back to zero, the object is freed, but the array itself
> is left as a nuisance for the C code.
> 
> Is there a standard way of doing this?

No, not yet.  You're not the first person to ask for this but I'd
appreciate it if you'd explain why you need it.  So far,  not having
that feature is a conscious omission to keep complexity down.

Regards,
Todd

-- 
Todd Miller <jmiller at stsci.edu>




From crasmussen at lanl.gov  Thu Apr 22 07:29:02 2004
From: crasmussen at lanl.gov (Craig Rasmussen)
Date: Thu Apr 22 07:29:02 2004
Subject: [Numpy-discussion] Numarrays from C arrays without copying
In-Reply-To: <1082639406.3272.10.camel@halloween.stsci.edu>
References: <20040422125131.GA7003@e.math.helsinki.fi> <1082639406.3272.10.camel@halloween.stsci.edu>
Message-ID: <560EC36A-9469-11D8-AAE7-000A957CA856@lanl.gov>

On Apr 22, 2004, at 7:10 AM, Todd Miller wrote:

>> I would like to turn a large C array allocated by a library into a
>> Numarray object without copying the contents. In other words I have
>> a given array of double precision floats and I want to get a Numarray
>> object, where the data pointer points to this array and no additional
>> memory is allocated. Naturally when the reference count of the 
>> Numarray
>> object gets back to zero, the object is freed, but the array itself
>> is left as a nuisance for the C code.
>>
>> Is there a standard way of doing this?
>
> No, not yet.  You're not the first person to ask for this but I'd
> appreciate it if you'd explain why you need it.  So far,  not having
> that feature is a conscious omission to keep complexity down.

The reason I need this is two fold:

1. I want to export to Python existing Fortran arrays.  If you want
Fortran and Python to share array memory, you can't always expect
Fortran programmers to use memory allocated in Python.  In fact,
since there is no standards conforming way to associate a Fortran
array pointer with Python memory, you have to do the opposite,
associate a Python array with Fortran memory.

2. In the CoArray Python module, we MUST use special memory
allocators that are parallel aware.

Because of these requirements, our only choice is to use Numeric.

Regards,
Craig




From haase at msg.ucsf.edu  Thu Apr 22 09:47:00 2004
From: haase at msg.ucsf.edu (Sebastian Haase)
Date: Thu Apr 22 09:47:00 2004
Subject: [Numpy-discussion] Numarrays from C arrays without copying
In-Reply-To: <560EC36A-9469-11D8-AAE7-000A957CA856@lanl.gov>
References: <20040422125131.GA7003@e.math.helsinki.fi> <1082639406.3272.10.camel@halloween.stsci.edu> <560EC36A-9469-11D8-AAE7-000A957CA856@lanl.gov>
Message-ID: <200404220946.01190.haase@msg.ucsf.edu>

Hi Craig,
I am sharing arrays between C and Python numarray (w/o any copying) for more 
than a year by now - please check my postings and (mostly) Todd's replys on 
this list. The code looks mostly like this:
<code sniplet>
 int shape[4]= { n,nCams,pic_ny,pic_nx };
   PyObject *obj, *obj2;

   obj = PyBuffer_FromReadWriteMemory(theRing.bufferPtr(), 
theRing.bufferSize());
   obj2 =(PyObject*) NA_NewAllFromBuffer( 4, shape, tUInt16, obj, 0, 0, 
NA_ByteOrder(), 1, 1/*writable*/) ;
   wxASSERT(obj != NULL);
   wxASSERT(obj2 != NULL);
   PyDict_SetItemString(globals, "ring", obj2);
   Py_XDECREF(obj);
   Py_XDECREF(obj2);
</code sniplet>
It even works with records arrays!
Luckily my arrays stay (mostly) around until the program quits. 
So I also don't have a good answer for 'who should free the memory?'

As far as numarray/Python concerns - once it's reference counter goes to zero 
- a 'delete[]' would be needed if the array was allocated with a C++ 'new' 
but a 'free()' is required if the memory got "malloc()'ed" in C.
Our internal discussion here found that the only real solution to this might 
be to extend numarray's object structure by a functions pointer 
('callToKill') and also an  argument pointer (the array should be delete'd or 
free'd).
But then we felt this might be to much to ask (to much of a change / overhead) 
that this proposal would not get accepted - so we choose to not bring it up.
Please comment,
Regards
Sebastian Haase





On Thursday 22 April 2004 07:28 am, Craig Rasmussen wrote:
> On Apr 22, 2004, at 7:10 AM, Todd Miller wrote:
> >> I would like to turn a large C array allocated by a library into a
> >> Numarray object without copying the contents. In other words I have
> >> a given array of double precision floats and I want to get a Numarray
> >> object, where the data pointer points to this array and no additional
> >> memory is allocated. Naturally when the reference count of the
> >> Numarray
> >> object gets back to zero, the object is freed, but the array itself
> >> is left as a nuisance for the C code.
> >>
> >> Is there a standard way of doing this?
> >
> > No, not yet.  You're not the first person to ask for this but I'd
> > appreciate it if you'd explain why you need it.  So far,  not having
> > that feature is a conscious omission to keep complexity down.
>
> The reason I need this is two fold:
>
> 1. I want to export to Python existing Fortran arrays.  If you want
> Fortran and Python to share array memory, you can't always expect
> Fortran programmers to use memory allocated in Python.  In fact,
> since there is no standards conforming way to associate a Fortran
> array pointer with Python memory, you have to do the opposite,
> associate a Python array with Fortran memory.
>
> 2. In the CoArray Python module, we MUST use special memory
> allocators that are parallel aware.
>
> Because of these requirements, our only choice is to use Numeric.
>
> Regards,
> Craig



From jmiller at stsci.edu  Thu Apr 22 09:58:18 2004
From: jmiller at stsci.edu (Todd Miller)
Date: Thu Apr 22 09:58:18 2004
Subject: [Numpy-discussion] Numarrays from C arrays without copying
In-Reply-To: <4087E8FB.8020801@pfdubois.com>
References: <20040422125131.GA7003@e.math.helsinki.fi>
	 <1082639406.3272.10.camel@halloween.stsci.edu>
	 <4087E8FB.8020801@pfdubois.com>
Message-ID: <1082653045.3272.308.camel@halloween.stsci.edu>

On Thu, 2004-04-22 at 11:47, Paul F. Dubois wrote:
> This is what PyArray_FromDimsAndData was for, and I believe it is fairly 
> heavily used.

That's good to know.  I was just discussing with Perry how to expand the
C-API to support this stuff.  Perhaps we don't have to.

> Todd Miller wrote:
> > On Thu, 2004-04-22 at 08:51, Ville Hakulinen wrote:
> > 
> >>Dear all,
> >>
> >>I would like to turn a large C array allocated by a library into a
> >>Numarray object without copying the contents. In other words I have
> >>a given array of double precision floats and I want to get a Numarray
> >>object, where the data pointer points to this array and no additional
> >>memory is allocated. Naturally when the reference count of the Numarray
> >>object gets back to zero, the object is freed, but the array itself
> >>is left as a nuisance for the C code.
> >>
> >>Is there a standard way of doing this?
> > 
> > 
> > No, not yet.  You're not the first person to ask for this but I'd
> > appreciate it if you'd explain why you need it.  So far,  not having
> > that feature is a conscious omission to keep complexity down.
> > 
> > Regards,
> > Todd
> > 
-- 
Todd Miller <jmiller at stsci.edu>




From ville at e.math.helsinki.fi  Thu Apr 22 11:24:10 2004
From: ville at e.math.helsinki.fi (Ville Hakulinen)
Date: Thu Apr 22 11:24:10 2004
Subject: [Numpy-discussion] Numarrays from C arrays without copying
In-Reply-To: <1082639406.3272.10.camel@halloween.stsci.edu>
References: <20040422125131.GA7003@e.math.helsinki.fi> <1082639406.3272.10.camel@halloween.stsci.edu>
Message-ID: <20040422182332.GA8864@e.math.helsinki.fi>

On Thu, Apr 22, 2004 at 09:10:07AM -0400, Todd Miller wrote:
> No, not yet.  You're not the first person to ask for this but I'd
> appreciate it if you'd explain why you need it.  So far,  not having
> that feature is a conscious omission to keep complexity down.

First I managed not to reply to numpy-discussion, so here we go again:

We are building a Python interface to PETSc 2.2
(http://www-unix.mcs.anl.gov/petsc/petsc-2/) using SWIG. It would
be very convenient to access the part of a PETSc vector that is
local to a process(or) using Numarray.

Other options would be to just write functions to access the part
of a vector independently of Numarray or to figure out whether one
could tell PETSc to use the memory area allocated by Numarray.
-- 
Ville



From crasmussen at lanl.gov  Thu Apr 22 11:33:01 2004
From: crasmussen at lanl.gov (Craig Rasmussen)
Date: Thu Apr 22 11:33:01 2004
Subject: [Numpy-discussion] Numarrays from C arrays without copying
In-Reply-To: <200404220946.01190.haase@msg.ucsf.edu>
References: <20040422125131.GA7003@e.math.helsinki.fi> <1082639406.3272.10.camel@halloween.stsci.edu> <560EC36A-9469-11D8-AAE7-000A957CA856@lanl.gov> <200404220946.01190.haase@msg.ucsf.edu>
Message-ID: <6BC5B671-948B-11D8-AAE7-000A957CA856@lanl.gov>

On Apr 22, 2004, at 10:46 AM, Sebastian Haase wrote:

>    obj = PyBuffer_FromReadWriteMemory(theRing.bufferPtr(),
> theRing.bufferSize());
>    obj2 =(PyObject*) NA_NewAllFromBuffer( 4, shape, tUInt16, obj, 0, 0,
> NA_ByteOrder(), 1, 1/*writable*/) ;

It looks like this will work.

I thought Todd had been saying that this wasn't possible.  Perhaps I
did not phrase my question very well when I asked earlier.

Thanks,
Craig




From crasmussen at lanl.gov  Thu Apr 22 11:35:12 2004
From: crasmussen at lanl.gov (Craig Rasmussen)
Date: Thu Apr 22 11:35:12 2004
Subject: [Numpy-discussion] Numarrays from C arrays without copying
In-Reply-To: <1082653045.3272.308.camel@halloween.stsci.edu>
References: <20040422125131.GA7003@e.math.helsinki.fi> <1082639406.3272.10.camel@halloween.stsci.edu> <4087E8FB.8020801@pfdubois.com> <1082653045.3272.308.camel@halloween.stsci.edu>
Message-ID: <B9B0CCD2-948B-11D8-AAE7-000A957CA856@lanl.gov>

On Apr 22, 2004, at 10:57 AM, Todd Miller wrote:

>> This is what PyArray_FromDimsAndData was for, and I believe it is 
>> fairly
>> heavily used.
>
> That's good to know.  I was just discussing with Perry how to expand 
> the
> C-API to support this stuff.  Perhaps we don't have to.

So is there an equivalent NA_FromDimsAndData?  I have been using
PyArray_FromDimsAnd Data.

Craig




From jmiller at stsci.edu  Thu Apr 22 12:09:09 2004
From: jmiller at stsci.edu (Todd Miller)
Date: Thu Apr 22 12:09:09 2004
Subject: [Numpy-discussion] Numarrays from C arrays without copying
In-Reply-To: <6BC5B671-948B-11D8-AAE7-000A957CA856@lanl.gov>
References: <20040422125131.GA7003@e.math.helsinki.fi>
	 <1082639406.3272.10.camel@halloween.stsci.edu>
	 <560EC36A-9469-11D8-AAE7-000A957CA856@lanl.gov>
	 <200404220946.01190.haase@msg.ucsf.edu>
	 <6BC5B671-948B-11D8-AAE7-000A957CA856@lanl.gov>
Message-ID: <1082660922.3272.504.camel@halloween.stsci.edu>

On Thu, 2004-04-22 at 14:32, Craig Rasmussen wrote:
> On Apr 22, 2004, at 10:46 AM, Sebastian Haase wrote:
> 
> >    obj = PyBuffer_FromReadWriteMemory(theRing.bufferPtr(),
> > theRing.bufferSize());
> >    obj2 =(PyObject*) NA_NewAllFromBuffer( 4, shape, tUInt16, obj, 0, 0,
> > NA_ByteOrder(), 1, 1/*writable*/) ;
> 
> It looks like this will work.
> 

It'll definitely work.  I'd forgotten about it.

> I thought Todd had been saying that this wasn't possible.  Perhaps I
> did not phrase my question very well when I asked earlier.

Nope,  your question was fine.

I'm hopeful that PyArray_FromDimsAndData can be improved 
so that this becomes easier for numarray-1.0.

Regards,
Todd
-- 
Todd Miller <jmiller at stsci.edu>




From jmiller at stsci.edu  Thu Apr 22 13:07:01 2004
From: jmiller at stsci.edu (Todd Miller)
Date: Thu Apr 22 13:07:01 2004
Subject: [Numpy-discussion] Numarrays from C arrays without copying
In-Reply-To: <B9B0CCD2-948B-11D8-AAE7-000A957CA856@lanl.gov>
References: <20040422125131.GA7003@e.math.helsinki.fi>
	 <1082639406.3272.10.camel@halloween.stsci.edu>
	 <4087E8FB.8020801@pfdubois.com>
	 <1082653045.3272.308.camel@halloween.stsci.edu>
	 <B9B0CCD2-948B-11D8-AAE7-000A957CA856@lanl.gov>
Message-ID: <1082664345.3272.592.camel@halloween.stsci.edu>

On Thu, 2004-04-22 at 14:34, Craig Rasmussen wrote:
> On Apr 22, 2004, at 10:57 AM, Todd Miller wrote:
> 
> >> This is what PyArray_FromDimsAndData was for, and I believe it is 
> >> fairly
> >> heavily used.
> >
> > That's good to know.  I was just discussing with Perry how to expand 
> > the
> > C-API to support this stuff.  Perhaps we don't have to.
> 
> So is there an equivalent NA_FromDimsAndData?

No.  I was making the mistake of thinking there needed to be.

>   I have been using
> PyArray_FromDimsAndData.

That's fine and should basically work.  

Between numarray and Numeric there is currently the not-so-subtle
difference that numarray copies the external data buffer and Numeric
just refers to it.

This is a documented difference,  but my vote is for changing in favor
of better functionality and Numeric compatibility.  I'll probably just
quietly change it unless there is a deluge of dissenters...

-- 
Todd Miller <jmiller at stsci.edu>




From info at deutsches-panel.com  Tue Apr 27 07:50:02 2004
From: info at deutsches-panel.com (Das deutsche Panel)
Date: Tue Apr 27 07:50:02 2004
Subject: [Numpy-discussion] Einladung
Message-ID: <SUJXHWVOQZNRSHCOQEFULLL@yahoo.com>

K?nnen Sie uns Ihre Meinung mitteilen ?

Das deutsche Panel ist eine Marktforschungswebsite, die mit dem Ziel errichtet worden ist, die Bev?lkerung Deutschlands in Bezug auf verschiedene Bereiche, wie z.B. neue Fernsehwerbung zu befragen. Wenn Sie daran interessiert sind, Ihre Meinungen zu ver?ffentlichen, f?llen Sie bitte unseren Fragebogen aus, bei :

http://www.deutsches-panel.com/rec/index.php?id=numpy-discussion at lists.sourceforge.net

Wenn Sie unser Panel besuchen und an unseren Studien teilnehmen, werden Sie automatisch bei einer Verlosung teilnehmen. Je nach Umfang der Studie, k?nnen Sie verschiedene Objekte im Wert zwischen 200 und 500 Euro gewinnen. So haben Sie die M?glichkeit verschiedene Objekte wie Playstations, Handies oder computerverbundene Ger?te zu gewinnen.

Falls Sie sich entscheiden den Wert des Objektes einer Hilfeorganisation zu spenden, werden wir Ihnen eine Liste mit unseren Partnerorganisationen zur Verf?gung stellen. Dadurch erm?glichen wir Ihnen die Wahl einer Organisation, der Sie spenden wollen.
Best?tigungen dieser Spenden werden auf Anfrage gew?hrt.

Falls Sie zuk?nftig keine weiteren Einladungen von uns erhalten wollen, klicken Sie bitte auf folgende Link um Ihre Einschreibung abzubestellen :
http://www.deutsches-panel.com/rec/unsubscribe.php?id=numpy-discussion at lists.sourceforge.net

Die Ergebnisse der Verlosungen und Spenden werden Ihnen online auf dem ?Das deutsche Panel Site? zur Verf?gung stehen.

Mit freundlichen Gr??en ,
Anna Klaus
Das deutsche Panel





From scipy-user-bounces at scipy.net  Wed Apr 28 12:21:00 2004
From: scipy-user-bounces at scipy.net (scipy-user-bounces at scipy.net)
Date: Wed Apr 28 12:21:00 2004
Subject: [Numpy-discussion] Your message to SciPy-user awaits moderator approval
Message-ID: <mailman.1774.1083180633.1305.scipy-user@scipy.net>

Your mail to 'SciPy-user' with the subject

    RE: Incoming Msg

Is being held until the list moderator can review it for approval.

The reason it is being held:

    Post by non-member to a members-only list

Either the message will get posted to the list, or you will receive
notification of the moderator's decision.  If you would like to cancel
this posting, please visit the following URL:

    http://www.scipy.net/mailman/confirm/scipy-user/8c8e42fd74bfe658b790f95e8d6accd96369bf95




From Marc.Poinot at onera.fr  Thu Apr 29 00:52:04 2004
From: Marc.Poinot at onera.fr (Marc Poinot)
Date: Thu Apr 29 00:52:04 2004
Subject: [Numpy-discussion] numarray on 64 bits
Message-ID: <4090B3FA.F8D4665A@onera.fr>

Did anybody use numarray on a 64 bits platform ? (e.g. SGI/Irix)
The package works on our 64 bits platforms... using 32 bits mode,
but not using 64 mode. Big crash in _ufunc... Is there a flag to
set somewhere, trick, hint, noway ?

-MP- 
-----------------------------------------------------------------------
 Marc POINOT             Alias: marcvs        Email: poinot at onera.fr
 ONERA -MFE/DSNA/ELSA    Tel: 01.46.73.42.84  Info: elsa-info at onera.fr
 29, Div. Leclerc        Fax: 01.46.73.41.66  Site:     
 92322 Chatillon FRANCE  Project: elsA        Web: http://www.onera.fr



From jmiller at stsci.edu  Thu Apr 29 04:17:06 2004
From: jmiller at stsci.edu (Todd Miller)
Date: Thu Apr 29 04:17:06 2004
Subject: [Numpy-discussion] numarray on 64 bits
In-Reply-To: <4090B3FA.F8D4665A@onera.fr>
References: <4090B3FA.F8D4665A@onera.fr>
Message-ID: <1083237365.5730.41.camel@localhost.localdomain>

On Thu, 2004-04-29 at 03:51, Marc Poinot wrote:
> Did anybody use numarray on a 64 bits platform ? 

Numarray has been used on Compaq Alpha/Tru64 and also SGI Altix,  but
neither is an everyday platform for us at STScI.

> (e.g. SGI/Irix)
> The package works on our 64 bits platforms... using 32 bits mode,
> but not using 64 mode. Big crash in _ufunc... 

This is probably a bug which has was created since we dropped Tru64 at
STScI.  A gdb traceback would be useful here to help understand where
the crash occurred.  Generally, crashes result from the improper use of
ints/longs in the numarray C-code.  

> Is there a flag to set somewhere, trick, hint, noway ?

There's really only one flag that matters for 64-bit operation on UNIX:
LP64.  It's controlled/propagated in the file generate.py.  LP64 asserts
that longs and pointers are 64-bit, while ints remain 32-bit. I imagine
you've already set this.

Regards, 
Todd

-- 
Todd Miller <jmiller at stsci.edu>




From rowen at u.washington.edu  Thu Apr 29 10:19:06 2004
From: rowen at u.washington.edu (Russell E Owen)
Date: Thu Apr 29 10:19:06 2004
Subject: [Numpy-discussion] rationalizing functions that return number of elements
Message-ID: <p06100505bcb6dae36e66@[128.95.99.44]>

I'm hoping we might rationalize the names of functions that return 
the number of elements in an array and fix the associated help 
strings. At present we have:

numarray function size(array, axis=None)
   Seems to return # of elements in array, or in a particular axis,
   but its help says something else:
   "size  returns the size in bytes of an array."

numarray.ma has function size(a, axis=None)
   Seems the same as numarray.size but the help is wrong,
   and in a different way:
   "returns shape tuple for 'a'.  MaskedArray version."

nummarray.ma arrays have method "size(seolf, axis=None):
   The same as the others and correctly documented:
   "Number of elements in array, or in a particular axis."

numarray arrays have method nelements()
   The same as "size" above, but has no axis argument:
   "nelements() -> total number of elements in array."

I strongly encourage you to settle on one name (and one
argument list, i.e. with "axis"). The current situation
forces one to treat ma arrays differently than normal arrays
in an area where there's no need for such a difference.
It is also confusing.

"size" is a perfectly good name and is already used
in 3 out of 4 cases, so adopting it will break the
least existing code and deprecate only one odd method.

On the other hand, "nelements" has the advantage
that it is less ambiguous (based on the incorrect help
strings). So if you really think it's worth the pain
you could switch to that.

My personal leaning is towards "size", but it's not
my decision.

By the way, "itemsize" already exists as an
array method, so "size" is NOT needed for that
(and would be a wretched choice--ambigious and
the potential to mysteriously break existing code).

-- Russell

P.S. I reported "nelements" as PR 934514
<http://sourceforge.net/tracker/index.php?func=detail&aid=934514&group_id=1369&atid=450446>
and the incorrect help strings as PR 944678
<http://sourceforge.net/tracker/index.php?func=detail&aid=944678&group_id=1369&atid=450446>

P.P.S. here's a log showing the tests that convinced me size works as 
I said, not as the help strings say:
>>>  a = num.array([[1.1, 2.2, 3.3], [4.4, 5.5, 6.6]])
>>>  a
array([[ 1.1,  2.2,  3.3],
        [ 4.4,  5.5,  6.6]])
>>>  numarray.size(a)
6
>>>  numarray.size(a, 1)
3
>>>  aa = numarray.ma.array(a)
>>>  numarray.ma.size(aa)
6
>>>  numarray.ma.size(aa, 1)
3
>>>  aa.size()
6
>>>  aa.size(1)
3
>>>  a.nelements()
6



From nadavh at visionsense.com  Thu Apr  1 06:38:10 2004
From: nadavh at visionsense.com (Nadav Horesh)
Date: Thu Apr  1 06:38:10 2004
Subject: [Numpy-discussion] type coercion question
Message-ID: <07C6A61102C94148B8104D42DE95F7E86DECCD@exchange2k.envision.co.il>

(3) seems to be reasonable since Int32-precision > Float32-precision

  Nadav.


-----Original Message-----
From:	Perry Greenfield [mailto:perry at stsci.edu]
Sent:	Wed 31-Mar-04 17:39
To:	David M. Cooke; Todd Miller
Cc:	Gary Ruben; numpy-discussion
Subject:	[Numpy-discussion] type coercion question
David Cooke wrote:
> On Mon, Mar 29, 2004 at 06:55:57AM -0500, Todd Miller wrote:
> > On Sun, 2004-03-28 at 22:46, Gary Ruben wrote:
> > > > > >>> cos(1)
> > > > > 0.54030227661132813
> > > > >
> > > > > gives a different result to cos(1.)
> > > <snip>
> > > > >  Do others think precision is being lost unnecessarily?
> > > >
> > > > No.  Do you have any suggestions?
> > >
> > > Yes. My problem is that numarray is not replicating the behaviour of
> > > the Python math module, which presumably coerces immediately to the
> > > float precision of the underlying platform. This is probably a
> > > float64 in Windows. If you don't explicitly specify that float32 is
> > > to be used, shouldn't the default be for a rank-0 value to
> > > immediately coerce ints to the same precision as the native Python
> > > float type on the underlying platform, since you know it will be
> > > coerced to that later and in the meantime you've lost precision
> > > because you've applied a function to a value of lower precision?
> > >
> >
> > I see your point and I'll talk it over with Perry unless he chimes in on
> > his own.  I'm ambivalent:  while it would be nice for the numarray
> > ufuncs to act as full equivalents to the Python math functions in scalar
> > cases,  numarray has to strike a balance between managing storage space
> > and maintaining precision and these two goals are in conflict.  I think
> > using an array package you have to be at least a little more aware of
> > storage concerns.
>
> I'll throw in a vote for 'cos(1)' coercing 1 to a (equivalent-to-)
> Python float. The reason is that I have 'from numarray import *' for my
> interactive interpreter, as I use numarray a lot in that.
> I would rather not have to remember to always cast my args to ufuncs to
> arrays. This isn't a storage space problem, per se., the question is
> whether cos(1) and cos(1.0) should return the same answer.
>
> Having cos(1) cast 1 to a Float32 makes numarray much less useful as a
> calculator -- mistakes are more likely. I'll probably switch my
> interpreter back to Numeric so as not to make mistakes, or I'll write a
> wrapper (which I'll post if I do).
>
> Whether cos([1,1,1]) and cos([1.0,1.0,1.0]) are the same I'd say is
> different, as there I've taken the conscious decision not to cast to an
> array. (And the fact that cos([1,1,1]) will return an array printed out
> with 'type=Float32' is a clue to what I've done.)
>
Let me say that I see the last issue (whether cos([1,1,1]) and
cos([1.0,1.0,1.0]) are the same) is important. If cos([1,1,1])[0] !=
cos(1) I think that is just as bad. So the question is whether
ints scalars and arrays should automatically be promoted to
double floats when given to unary scientific functions (sqrt,
sin, cos, log, etc.). By the way, if an Int32 is so promoted,
I don't see why it isn't the same for Int8, Int16, etc.
But it appears that Numeric promotes Int32 to Float64 for cos(),
but promotes In16 to Float32 for cos().

I believe making this change to numarray is fairly easy (we'd have
to check). But is what most people want? Should we retain complete
compatibility with Numeric (which, as pointed out, treats different
kinds of integers differently)

Just to summarize what I see as acceptable choices:

1) ints promote to Float32 (current behavior)
2) ints promote to Float64
3) Int32 promotes to Float64, lesser ints promote to Float32 (Numeric
behavior)

Perry






-------------------------------------------------------
This SF.Net email is sponsored by: IBM Linux Tutorials
Free Linux tutorial presented by Daniel Robbins, President and CEO of
GenToo technologies. Learn everything from fundamentals to system
administration.http://ads.osdn.com/?ad_id=1470&alloc_id=3638&op=click
_______________________________________________
Numpy-discussion mailing list
Numpy-discussion at lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/numpy-discussion






From perry at stsci.edu  Thu Apr  1 14:38:21 2004
From: perry at stsci.edu (Perry Greenfield)
Date: Thu Apr  1 14:38:21 2004
Subject: [Numpy-discussion] question about nonzero behavior in numarray
In-Reply-To: <slrnc6gv95.3v3.faheem@Chrestomanci.home.earth>
Message-ID: <CEELJPECNGEGKFNDLHFFOEALCDAA.perry@stsci.edu>

Todd pointed out to me that he's gotten some complaints about
the difference in how numarray nonzero behavior works.
In version 0.9 the use of an array for a truth value was
changed from a deprecation warning to an exception.

We deprecated it because the meaning of an array as a truth
value was ambiguous (or potentially confusing at best)
and we felt it was better to force users to be explicit
about what they wanted. Nevertheless, Numeric does allow that
and removing the capability to use it in that context breaks
some code. 

We can give some thought to reversing that decision, but since
we only hear from those that don't like the new behavior, we'd
like to get some sense if there are many that do like the new
behavior.

To clarify, what happens when:

x = zeros(2)
if x:
    print "true!"

I see 3 possibilities:

1) Keep it the way it is, no use of an array is permitted; an 
   exception is raised.
2) Print a warning message when it is used that way, and always
   keep it at that, i.e., warning message only.
3) Replicate Numeric behavior.

Personally, I prefer 1) since I think there is too much confusion
between whether false means an empty (len=0) array (like lists),
or an array that contains all zeros, and that it is worth breaking
backward compatibility on this (functions should be used to 
distinguish which meaning is desired). 

Perry

[Option 2 on the previous question wins hands down]






From Fernando.Perez at colorado.edu  Thu Apr  1 14:52:09 2004
From: Fernando.Perez at colorado.edu (Fernando Perez)
Date: Thu Apr  1 14:52:09 2004
Subject: [Numpy-discussion] question about nonzero behavior in numarray
In-Reply-To: <CEELJPECNGEGKFNDLHFFOEALCDAA.perry@stsci.edu>
References: <CEELJPECNGEGKFNDLHFFOEALCDAA.perry@stsci.edu>
Message-ID: <406C9CEB.8050707@colorado.edu>

Perry Greenfield wrote:

> Personally, I prefer 1) since I think there is too much confusion
> between whether false means an empty (len=0) array (like lists),
> or an array that contains all zeros, and that it is worth breaking
> backward compatibility on this (functions should be used to 
> distinguish which meaning is desired). 

I tend to prefer keeping compatibility with Python list behavior.  I think 
that in general, numarray arrays should behave different from lists only when 
truly justified (for example, at least in numeric all ufuncs can be applied to 
a list of floats as much as to an array, which is nice).

Since anyone who knows python already knows that lists are true based on 
whether they contain _anything_:

In [9]: a=[]

In [10]: b=[0]

In [11]: if a:
    ....:     print 1
    ....:

In [12]: if b:
    ....:     print 1
    ....:
1


then I think it's a fair, clean extension to make NumArray behave similarly.

I think a good guiding principle is to keep the number of special cases for 
arrays at a strict minimum (there are obviously cases where it does make sense 
to have special cases).  Besides, this is an O(1) check, so it doesn't have 
the dangers of checking for all elements being zero.   Such a potentially 
expensive operation should _definitely_ happen only if explicitly requested, IMHO.

Regards,

f




From cjw at sympatico.ca  Thu Apr  1 15:00:00 2004
From: cjw at sympatico.ca (Colin J. Williams)
Date: Thu Apr  1 15:00:00 2004
Subject: [Numpy-discussion] question about nonzero behavior in numarray
In-Reply-To: <CEELJPECNGEGKFNDLHFFOEALCDAA.perry@stsci.edu>
References: <CEELJPECNGEGKFNDLHFFOEALCDAA.perry@stsci.edu>
Message-ID: <406C9EC3.8080504@sympatico.ca>


Perry Greenfield wrote:

>Todd pointed out to me that he's gotten some complaints about
>the difference in how numarray nonzero behavior works.
>In version 0.9 the use of an array for a truth value was
>changed from a deprecation warning to an exception.
>
>We deprecated it because the meaning of an array as a truth
>value was ambiguous (or potentially confusing at best)
>and we felt it was better to force users to be explicit
>about what they wanted. Nevertheless, Numeric does allow that
>and removing the capability to use it in that context breaks
>some code. 
>
>We can give some thought to reversing that decision, but since
>we only hear from those that don't like the new behavior, we'd
>like to get some sense if there are many that do like the new
>behavior.
>
>To clarify, what happens when:
>
>x = zeros(2)
>if x:
>    print "true!"
>
>I see 3 possibilities:
>
>1) Keep it the way it is, no use of an array is permitted; an 
>   exception is raised.
>2) Print a warning message when it is used that way, and always
>   keep it at that, i.e., warning message only.
>3) Replicate Numeric behavior.
>
>Personally, I prefer 1) since I think there is too much confusion
>between whether false means an empty (len=0) array (like lists),
>or an array that contains all zeros, and that it is worth breaking
>backward compatibility on this (functions should be used to 
>distinguish which meaning is desired). 
>
>Perry
>
>[Option 2 on the previous question wins hands down]
>  
>
I favour option 1, otherwise there is a need to define just what it means.

Does it return a Boolean array indicating those cells which are nonzero, or
a single Boolean value indicating that some cell is nonzero, or a single
Boolean value indicating that all cells are nonzero?

Colin W.

>
>
>
>
>-------------------------------------------------------
>This SF.Net email is sponsored by: IBM Linux Tutorials
>Free Linux tutorial presented by Daniel Robbins, President and CEO of
>GenToo technologies. Learn everything from fundamentals to system
>administration.http://ads.osdn.com/?ad_id=1470&alloc_id=3638&op=click
>_______________________________________________
>Numpy-discussion mailing list
>Numpy-discussion at lists.sourceforge.net
>https://lists.sourceforge.net/lists/listinfo/numpy-discussion
>
>  
>




From list at jsaul.de  Thu Apr  1 15:40:04 2004
From: list at jsaul.de (Joachim Saul)
Date: Thu Apr  1 15:40:04 2004
Subject: [Numpy-discussion] question about nonzero behavior in numarray
In-Reply-To: <406C9CEB.8050707@colorado.edu>
References: <CEELJPECNGEGKFNDLHFFOEALCDAA.perry@stsci.edu> <406C9CEB.8050707@colorado.edu>
Message-ID: <20040401233805.GA684@jsaul.de>

* Fernando Perez [2004-04-02 00:51]:
> Perry Greenfield wrote:
>
> >Personally, I prefer 1) since I think there is too much confusion
> >between whether false means an empty (len=0) array (like lists),
> >or an array that contains all zeros, and that it is worth breaking
> >backward compatibility on this (functions should be used to
> >distinguish which meaning is desired).
>
> I tend to prefer keeping compatibility with Python list behavior.  I think
> that in general, numarray arrays should behave different from lists only
> when truly justified (for example, at least in numeric all ufuncs can be
> applied to a list of floats as much as to an array, which is nice).

How often does it happen that you have an empty array, which you
need to test for? Certainly not as frequently as in case of a
list.

I think that if

  >>> a=ones(3)
  >>> a==1
  array([1, 1, 1])
  >>> a==0
  array([0, 0, 0])

then I would follow that

  >>> if a==0:
  ...
  >>> if array([0, 0, 0]):
  ...

should also _both_ evaluate as "false".

> Since anyone who knows python already knows that lists are true based on
> whether they contain _anything_:
>
> In [9]: a=[]
>
> In [10]: b=[0]
>
> In [11]: if a:
>    ....:     print 1
>    ....:
>
> In [12]: if b:
>    ....:     print 1
>    ....:
> 1
>
>
> then I think it's a fair, clean extension to make NumArray behave similarly.
>
> I think a good guiding principle is to keep the number of special cases for
> arrays at a strict minimum (there are obviously cases where it does make
> sense to have special cases).  Besides, this is an O(1) check, so it
> doesn't have the dangers of checking for all elements being zero.   Such a
> potentially expensive operation should _definitely_ happen only if
> explicitly requested, IMHO.

But if the behaviour is unambiguously defined, I see no problem
here, because I can as well check the length of a list using
len().

BTW, How about

    >>> a = [[]]
    >>> if a:
    ...

?

Cheers,
Joachim



From Fernando.Perez at colorado.edu  Thu Apr  1 16:16:16 2004
From: Fernando.Perez at colorado.edu (Fernando Perez)
Date: Thu Apr  1 16:16:16 2004
Subject: [Numpy-discussion] question about nonzero behavior in numarray
In-Reply-To: <20040401233805.GA684@jsaul.de>
References: <CEELJPECNGEGKFNDLHFFOEALCDAA.perry@stsci.edu> <406C9CEB.8050707@colorado.edu> <20040401233805.GA684@jsaul.de>
Message-ID: <406CB0A0.5030509@colorado.edu>

Joachim Saul wrote:

> How often does it happen that you have an empty array, which you
> need to test for? Certainly not as frequently as in case of a
> list.

Well, just these days I'm working with code where I'm precisely using that 
kind of check, relying on Numeric's behavior.  But I know that with 
potentially nested structures, it's probably best to be explicit and do all 
checks via things like:

if len(a)

and

if a==0

etc.  So I could also be +1 on Perry's wish for turning it into an extension, 
at the expense of my current code breaking.

Cheers,

f.




From owen at astro.washington.edu  Fri Apr  2 09:59:06 2004
From: owen at astro.washington.edu (Russell E Owen)
Date: Fri Apr  2 09:59:06 2004
Subject: [Numpy-discussion] Questions about records
Message-ID: <p06020405bc934e349109@[128.95.99.44]>

I've been trying to generate a 2-dimensional numarray.records array 
and am rather puzzled about some failures:

I tried a pair of 2x2 arrays for the buffer (one per field). I 
thought the record would get its shape from those, but instead of a 
2x2 record I get a 2-length record, each of whose elements is a pair 
of 2-length arrays.

>>>  arr1 = num.arange(4, shape=(2,2), type=num.Float64)
>>>  arr2 = num.arange(4, shape=(2,2), type=num.Float64) + 10
>>>  a = rec.array([arr1, arr2], names="a,b")
>>>  a
array(
[(array([ 0.,  1.]), array([ 10.,  11.])),
(array([ 2.,  3.]), array([ 12.,  13.]))],
formats=['(2,)Float64', '(2,)Float64'],
shape=2,
names=['a', 'b'])

So I tried passing in 4-length arrays while specifying the shape 
explicitly, but it failed. Is this a bug?:

>>>  arr1 = num.arange(4, type=num.Float64)
>>>  arr2 = num.arange(4, type=num.Float64) + 10
>>>  a = rec.array([arr1, arr2], shape=(2,2), names="a, b")
Traceback (most recent call last):
   File "<stdin>", line 1, in ?
   File "/usr/local/lib/python2.3/site-packages/numarray/records.py", 
line 384, in array
     byteorder=byteorder, aligned=aligned)
   File "/usr/local/lib/python2.3/site-packages/numarray/records.py", 
line 177, in fromarrays
     raise ValueError, "array has different lengths"
ValueError: array has different lengths

Generating a 4-length record and reshaping it does seem to work, 
though there seems to be a bug in __str__ which I'll report:

>>>  arr1 = num.arange(4, type=num.Float64)
>>>  arr2 = num.arange(4, type=num.Float64) + 10
>>>  a = rec.array([arr1, arr2], names="a, b")
>>>  a.setshape((2,2))
>>>  a
Traceback (most recent call last):
   File "<stdin>", line 1, in ?
   File "/usr/local/lib/python2.3/site-packages/numarray/records.py", 
line 718, in __repr__
     outlist.append(Record.__str__(i))
TypeError: unbound method __str__() must be called with Record 
instance as first argument (got RecArray instance instead)
>>>  a[1,1]
<numarray.records.Record instance at 0x15d2ad0>
>>>  str(a[1,1])
'(1.0, 11.0)'

I see the str problem again if I don't specify any buffer:

>>>  import numarray as num
>>>  import numarray.records as rec
>>>  a = rec.array(formats="Float64,Float64", names="a,b", shape=(2,2))
>>>  a
Traceback (most recent call last):
   File "<stdin>", line 1, in ?
   File "/usr/local/lib/python2.3/site-packages/numarray/records.py", 
line 718, in __repr__
     outlist.append(Record.__str__(i))
TypeError: unbound method __str__() must be called with Record 
instance as first argument (got RecArray instance instead)

So...comments? Should I report the shape issues as a bug?

-- Russell



From hsu at stsci.edu  Fri Apr  2 11:13:06 2004
From: hsu at stsci.edu (Jin-chung Hsu)
Date: Fri Apr  2 11:13:06 2004
Subject: FW: [Numpy-discussion] Questions about records
Message-ID: <200404021912.AKB80704@donner.stsci.edu>

> I've been trying to generate a 2-dimensional numarray.records array 
> and am rather puzzled about some failures:
> 
> I tried a pair of 2x2 arrays for the buffer (one per field). I 
> thought the record would get its shape from those, but instead of a 
> 2x2 record I get a 2-length record, each of whose elements is a pair 
> of 2-length arrays.
> 
> >>>  arr1 = num.arange(4, shape=(2,2), type=num.Float64)
> >>>  arr2 = num.arange(4, shape=(2,2), type=num.Float64) + 10
> >>>  a = rec.array([arr1, arr2], names="a,b")
> >>>  a
> array(
> [(array([ 0.,  1.]), array([ 10.,  11.])),
> (array([ 2.,  3.]), array([ 12.,  13.]))],
> formats=['(2,)Float64', '(2,)Float64'],
> shape=2,
> names=['a', 'b'])
> 
> So I tried passing in 4-length arrays while specifying the shape 
> explicitly, but it failed. Is this a bug?:
> 
> >>>  arr1 = num.arange(4, type=num.Float64)
> >>>  arr2 = num.arange(4, type=num.Float64) + 10
> >>>  a = rec.array([arr1, arr2], shape=(2,2), names="a, b")
> Traceback (most recent call last):
>    File "<stdin>", line 1, in ?
>    File "/usr/local/lib/python2.3/site-packages/numarray/records.py", 
> line 384, in array
>      byteorder=byteorder, aligned=aligned)
>    File "/usr/local/lib/python2.3/site-packages/numarray/records.py", 
> line 177, in fromarrays
>      raise ValueError, "array has different lengths"
> ValueError: array has different lengths
> 
> Generating a 4-length record and reshaping it does seem to work, 
> though there seems to be a bug in __str__ which I'll report:
> 
> >>>  arr1 = num.arange(4, type=num.Float64)
> >>>  arr2 = num.arange(4, type=num.Float64) + 10
> >>>  a = rec.array([arr1, arr2], names="a, b")
> >>>  a.setshape((2,2))
> >>>  a
> Traceback (most recent call last):
>    File "<stdin>", line 1, in ?
>    File "/usr/local/lib/python2.3/site-packages/numarray/records.py", 
> line 718, in __repr__
>      outlist.append(Record.__str__(i))
> TypeError: unbound method __str__() must be called with Record 
> instance as first argument (got RecArray instance instead)
> >>>  a[1,1]
> <numarray.records.Record instance at 0x15d2ad0>
> >>>  str(a[1,1])
> '(1.0, 11.0)'
> 
> I see the str problem again if I don't specify any buffer:
> 
> >>>  import numarray as num
> >>>  import numarray.records as rec
> >>>  a = rec.array(formats="Float64,Float64", names="a,b", shape=(2,2))
> >>>  a
> Traceback (most recent call last):
>    File "<stdin>", line 1, in ?
>    File "/usr/local/lib/python2.3/site-packages/numarray/records.py", 
> line 718, in __repr__
>      outlist.append(Record.__str__(i))
> TypeError: unbound method __str__() must be called with Record 
> instance as first argument (got RecArray instance instead)
> 
> So...comments? Should I report the shape issues as a bug?
> 
> -- Russell

First of all, the records module was developed mainly having the 1-D table in 
mind.  Even though it can have higher than one dimension, it is not thoroughly 
tested, as you have found out.  However, I'd argue that in many cases that the 
need to use a 2-D (or high) table can be substituted by having an array in each 
cell(element).  In your example, instead of creating a 2x2 table with each cell 
just having one number, you may be able to use a table with just one row and 
each cell is a 2x2 array.  You can create such a record like this:

--> arr1 = num.arange(4, shape=(1,2,2), type=num.Float64)
--> arr2 = num.arange(4, shape=(1,2,2), type=num.Float64)+10
--> a = rec.array([arr1, arr2], names="a,b")

I'd be interested in your application as to why a 2x2 table is necessary.

The __str__ method in RecArray is rather primitive but usually works for 1-D 
tables.  Eventually, we'll need a C-implementation of this method for both speed 
and flexibility.

When you use a list for the buffer in the array function, it is using a 
relatively intuitive (simplistic?) approach.  It tries to figure out the (one 
dimensional) length of each item in the list and use that as the record shape.

JC Hsu




From owen at astro.washington.edu  Fri Apr  2 12:09:04 2004
From: owen at astro.washington.edu (Russell E Owen)
Date: Fri Apr  2 12:09:04 2004
Subject: FW: [Numpy-discussion] Questions about records
In-Reply-To: <200404021912.AKB80704@donner.stsci.edu>
References: <200404021912.AKB80704@donner.stsci.edu>
Message-ID: <p06020406bc9372470570@[128.95.99.44]>

At 2:12 PM -0500 2004-04-02, Jin-chung Hsu wrote:
>First of all, the records module was developed mainly having the 1-D table in
>mind.  Even though it can have higher than one dimension, it is not thoroughly
>tested, as you have found out.  However, I'd argue that in many cases that the
>need to use a 2-D (or high) table can be substituted by having an 
>array in each
>cell(element).  In your example, instead of creating a 2x2 table 
>with each cell
>just having one number, you may be able to use a table with just one row and
>each cell is a 2x2 array.  You can create such a record like this:
>
>--> arr1 = num.arange(4, shape=(1,2,2), type=num.Float64)
>--> arr2 = num.arange(4, shape=(1,2,2), type=num.Float64)+10
>--> a = rec.array([arr1, arr2], names="a,b")

But is there any advantage to that compared to just using named 
arrays of the desired shape:
a = num.arange(4, shape=(2,2), type=num.Float64)
b = num.arange(4, shape=(2,2), type=num.Float64)+10

>I'd be interested in your application as to why a 2x2 table is necessary.

Here are two different uses I've come up with (both related to image 
processing). Both are beautifully served by a 2-d records array:

1) Find the centroid of a star. The algorithm I'm using (invented by 
Jim Gunn, I believe) is to walk across the image, looking for the 
point of maximum symmetry. At each point total pixels and a measure 
of asymmetry are measured in a 3x3 grid centered at that point. The 
minimum asymmetry in that 3x3 array is then used to determine where 
to walk next. (At the end a parabolic fit is done to the 3x3 
asymmetry data to find the true centroid; up until then it's only 
know to the nearest pixel).

In any case...right now I maintain two separate 3x3 arrays (total 
pixels and asymmetry). Whenever I take a step I shift the both arrays 
and then compute new data for the points which are missing data.

It would be cleaner and nicer to maintain one 3x3 records array with 
fields "totPix" and "asymm". Then the related data sticks together 
and I only have to shift the data once. (I meant to code it that way 
from the start, but my early attempts to use numeric.records were a 
disaster. I have a somewhat better handle on it now and may update my 
code).

2) Find all stars on an image. The algorithm I'm using (invented by 
Jeff Morgan, I believe) is to break an image up into blocks of, say, 
5x5 pixels. I then compute information about each "super pixel", such 
as center of mass, total counts, etc. My C++ code has 12 items of 
information for each super pixel (including 7 boolean flags) and is 
written to use a 2-dimensional array each element of which is a data 
structure with the appropriate fields. The obvious python equivalent 
is a numarray.records array. It sure sounds better than trying to 
keep track of 12 separate arrays!

-- Russell



From hsu at stsci.edu  Fri Apr  2 13:30:05 2004
From: hsu at stsci.edu (Jin-chung Hsu)
Date: Fri Apr  2 13:30:05 2004
Subject: FW: [Numpy-discussion] Questions about records
Message-ID: <200404022129.AKB90076@donner.stsci.edu>

> >--> arr1 = num.arange(4, shape=(1,2,2), type=num.Float64)
> >--> arr2 = num.arange(4, shape=(1,2,2), type=num.Float64)+10
> >--> a = rec.array([arr1, arr2], names="a,b")
> 
> But is there any advantage to that compared to just using named 
> arrays of the desired shape:
> a = num.arange(4, shape=(2,2), type=num.Float64)
> b = num.arange(4, shape=(2,2), type=num.Float64)+10
>

Not really, in this particular example.
 
> >I'd be interested in your application as to why a 2x2 table is necessary.
> 
> Here are two different uses I've come up with (both related to image 
> processing). 
[snip]

You need it because you need to pass it to the C-structure, I think.  In any 
case, you have found a way to get around the problem by using setshape.  I'll 
take a look of the module to get an idea of how much effort is needed to make 
the 2-D (and higher) record arrays to work more smoothly.

JC Hsu




From list at jsaul.de  Fri Apr  2 15:32:08 2004
From: list at jsaul.de (Joachim Saul)
Date: Fri Apr  2 15:32:08 2004
Subject: [Numpy-discussion] question about nonzero behavior in numarray
In-Reply-To: <406CB0A0.5030509@colorado.edu>
References: <CEELJPECNGEGKFNDLHFFOEALCDAA.perry@stsci.edu> <406C9CEB.8050707@colorado.edu> <20040401233805.GA684@jsaul.de> <406CB0A0.5030509@colorado.edu>
Message-ID: <20040402233135.GA688@jsaul.de>

* Fernando Perez [2004-04-02 02:15]:
> Joachim Saul wrote:
>
> >How often does it happen that you have an empty array, which you
> >need to test for? Certainly not as frequently as in case of a
> >list.
>
> Well, just these days I'm working with code where I'm precisely using that
> kind of check, relying on Numeric's behavior.  But I know that with
> potentially nested structures, it's probably best to be explicit and do all
> checks via things like:
>
> if len(a)
>
> and
>
> if a==0

But what would a==0 return? A boolean? An array of booleans? An
array if integers? The way it is evaluated now in Numeric is
pretty clear, though there could be other evaluations.  For
instance,

    if a==1:

might be understood as "if a is everywhere one" or "if the length
of vector a is one", etc. The current meaning is "if a is anywhere
one", which may not be intuitive under all circumstances, but as
long as it is the well defined behaviour, I simply don't see the
need to change or forbid that.

Nested structures are an important argument against keeping
compatibility with Python list behaviour. For instance

    >>> if []:   print "True"
    >>> if [[]]: print "True"
    True

Which perfectly makes sense in Python, but an equivalent behaviour
of a numeric array would be complete humbug. Thus, I don't think
that it would make any sense to emulate Python list behaviour,
rather I would support keeping the *existing* Numeric behaviour.

Cheers,
Joachim



From list at jsaul.de  Fri Apr  2 22:57:22 2004
From: list at jsaul.de (Joachim Saul)
Date: Fri Apr  2 22:57:22 2004
Subject: [Numpy-discussion] question about nonzero behavior in numarray
In-Reply-To: <20040402233135.GA688@jsaul.de>
References: <CEELJPECNGEGKFNDLHFFOEALCDAA.perry@stsci.edu> <406C9CEB.8050707@colorado.edu> <20040401233805.GA684@jsaul.de> <406CB0A0.5030509@colorado.edu> <20040402233135.GA688@jsaul.de>
Message-ID: <20040403064835.GA681@jsaul.de>

* I claimed:
>     if a==1:
>
> might be understood as "if a is everywhere one" or "if the length
> of vector a is one", etc. The current meaning is "if a is anywhere
> one", [...]

This is of course wrong and would probably not be misunderstood as
that. What I meant to say is that

    >>> if a:
    ...

would be evaluated as "true" if a is anywhere non-zero. It was too
late in the night it seems...

Sorry for the confusion.
Joachim



From noori at yellow.plala.or.jp  Sat Apr  3 12:02:11 2004
From: noori at yellow.plala.or.jp (Noori)
Date: Sat Apr  3 12:02:11 2004
Subject: [Numpy-discussion] Numpy-discussion Wanted to try Cialis but thought it was way too expensive for you? numpy-discussion@lists.sourceforge.net Numpy-discussion
In-Reply-To: <IBIAH045864GD21K@lists.sourceforge.net>
References: <IBIAH045864GD21K@lists.sourceforge.net>
Message-ID: <80BD72GFEF0GJ1LG@yellow.plala.or.jp>

You can do that now. Cialiis at only $2.00 p/d.
Your best deals are here
http://smart-pharmacy.com/15/
You have enjoyed Viaggra and wish for a longer effect? Ciaaliss can make it happen.

UNBELIEVABLE. CIIALISS AT ONLY $2.00 P/D

Say NO. to planning seex for hours in advance and your disappointment after "a missed dose". You don't have to be content with a four hour action of an regular pill. Get CIALIS now to ensure the whole day of joyful sexual intimacy for you and your partner.
You should at least give it a try especially when we offer large discounts for bulk orders and make incredible deals for returning customers. NOW
http://smart-pharmacy.com/15/




From Mailscan at eagle.org  Mon Apr  5 00:29:04 2004
From: Mailscan at eagle.org (Content Filter)
Date: Mon Apr  5 00:29:04 2004
Subject: [Numpy-discussion] Undeliverable message returned to sender.
Message-ID: <20040405072817.7E2EA2CCB53@mail24-red-R.bigfish.com>

This message was created automatically by mail delivery software.

Delivery failed for the following recipients(s):
 	fm at eqe.fr

The email message referenced below contained an executable attachment.  Since such attachments can contain dangerous or malicious code, the message and it?s attachment has been blocked. In the future please use non-executable file attachments whenever possible to avoid delays. See below for instructions on how to send this file to your ABS contact.
If the message also contained a virus, the sender should also receive a separate notification.  The name(s) of the blocked file(s) follow:
	application.pif

To send this file, please place it in a compressed archive using WinZip (.zip file extension) [http://www.winzip.com] or the archive software of your choice, and resend the message.

Thank you.

MailScanner Administrator: mailscan at eagle.org
ABS Customer Service Center: CSC at eagle.org

----- Original Message Header -----
Received: by mail24-red (MessageSwitch) id 1081150097256744_26344; Mon,  5 Apr 2004 07:28:17 +0000 (UCT)
Received: from eqe.fr (AVelizy-112-1-3-152.w193-253.abo.wanadoo.fr [193.253.228.152])
	by mail24-red.bigfish.com (Postfix) with ESMTP id 2818F2CCBC3
	for <fm at eqe.fr>; Mon,  5 Apr 2004 07:28:13 +0000 (UCT)
From: numpy-discussion at lists.sourceforge.net
To: fm at eqe.fr
Subject: Re: Your software
Date: Mon, 5 Apr 2004 09:28:26 +0200
MIME-Version: 1.0
Content-Type: multipart/mixed;
	boundary="----=_NextPart_000_0000_00005C9C.00003D49"
X-Priority: 3
X-MSMail-Priority: Normal
Message-Id: <20040405072813.2818F2CCBC3 at mail24-red.bigfish.com>



From faheem at email.unc.edu  Wed Apr  7 20:03:25 2004
From: faheem at email.unc.edu (Faheem Mitha)
Date: Wed Apr  7 20:03:25 2004
Subject: [Numpy-discussion] problems with compiling and loading C++ extension
Message-ID: <Pine.LNX.4.58.0404072259470.8108@Chrestomanci>

Dear People,

I have been having an odd problem with compiling and loading a simple
C++ extension to python (as a .so file in Linux). Unfortunately, I'll
need to include my files in here so I apologize in advance for the
length of this message.

You can download also the files below from
http://www.stat.unc.edu/students/faheem/python/python.tar.gz in case
that is more convenient.

This needs blitz++ and python's distutils and numarray to compile.

When I try to load arrmod.so into python I get

In [1]: import arrmod
---------------------------------------------------------------------------
ImportError                               Traceback (most recent call
last)

/tmp/wc/corrmodel/ex/<console>

ImportError: ./arrmod.so: undefined symbol:
_Z11py_to_blitzIdLi2EEN5blitz5ArrayIT_XT0_EEEP13PyArrayObject

If I move the py_to_blitz routine from conv.cc back into arrmod.cc,
the error disappears, arrmod loads in python, and everything works
correctly. I'm not sure what is going on here. Can someone enlighten
me? At first I thought it might have to do with the need for C
linkage, but I tried that, and it does not appear to be the case.

I also got the following compilation warning, which I don't
understand. I don't know if that is relevant.

---------------------------------------------------------------------
gcc -pthread -fno-strict-aliasing -DNDEBUG -g -O3 -Wall
-Wstrict-prototypes -fPIC -I/usr/include/python2.3 -c conv.cc -o
/tmp/temp.linux-i686-2.3/conv.o -Wall
/usr/include/python2.3/Numeric/arrayobject.h:286: warning:
`void**PyArray_API'
   defined but not used
---------------------------------------------------------------------

Thanks in advance for any enlightenment.

                                                 Faheem.

**********************************************************************
Makefile
**********************************************************************
arrmod.so: arrmod.cc common.hh conv.cc
	python setup.py build --build-base=/tmp --build-lib=.
# may not do the right thing for everyone
clean:
	rm arrmod.so
	rm -r /tmp/temp.linux-i686-2.3
**********************************************************************

**********************************************************************
setup.py
**********************************************************************
from distutils.core import setup, Extension

module4 = Extension('arrmod',
                    sources = ['arrmod.cc','conv.cc'],libraries=["blitz","m"],
                    extra_compile_args = ['-Wall'])

setup (name = 'arrmod',
       version = '1.0',
       description = 'This module performs different operations on arrays',
       ext_modules = [module4]
       )
**********************************************************************

**********************************************************************
common.hh
**********************************************************************
#include "Python.h"
#include "Numeric/arrayobject.h"
#include <blitz/array.h>

using namespace std;
using namespace blitz;

template<class T, int N>
static Array<T,N> py_to_blitz(PyArrayObject* arr_obj);
**********************************************************************

**********************************************************************
conv.cc
**********************************************************************
#include "common.hh"

// Convert a Numpy array to a blitz one, using the original's data (no
// copy)
template<class T, int N>
static Array<T,N> py_to_blitz(PyArrayObject* arr_obj)
{
  int T_size = sizeof(T);
  TinyVector<int,N> shape(0);
  TinyVector<int,N> strides(0);
  int *arr_dimensions = arr_obj->dimensions;
  int *arr_strides = arr_obj->strides;

  for (int i=0;i<N;++i)
    {
      shape[i]   = arr_dimensions[i];
      strides[i] = arr_strides[i]/T_size;
    }
  return Array<T,N>((T*) arr_obj->data,shape,strides,neverDeleteData);
}
************************************************************************

************************************************************************
arrmod.cc
************************************************************************
#include "common.hh"

template<class T, int N>
static Array<T,N> py_to_blitz(PyArrayObject* arr_obj);

static PyObject * arrmod_elem(PyObject *self, PyObject *args);

static PyMethodDef arrmod_methods[] =
  {
    {"elem", (PyCFunction)arrmod_elem, METH_VARARGS, "Returns the trace of a two-dimensional array.\n"},
    {NULL, NULL, 0, NULL}
  };

PyMODINIT_FUNC initarrmod(void)
{
  (void) Py_InitModule3("arrmod", arrmod_methods, "Returns the Trace of a two-dimensional array.\n");
  import_array();
}

static PyObject * arrmod_elem(PyObject *self, PyObject *args)
{
  PyObject *input, *result;
  PyArrayObject *array;
  double el;
  int i, j;

  if (!PyArg_ParseTuple(args, "Oii", &input, &i, &j))
    return NULL;
  array = (PyArrayObject *)
    PyArray_ContiguousFromObject(input, PyArray_DOUBLE, 2, 2);
  if (array == NULL)
    return NULL;

  Array<double,2> arr = py_to_blitz<double,2>(array);

  el = arr(i,j);
  result = Py_BuildValue("d",el);
  return result;
}
********************************************************************************



From fperez at colorado.edu  Wed Apr  7 21:26:10 2004
From: fperez at colorado.edu (Fernando Perez)
Date: Wed Apr  7 21:26:10 2004
Subject: [Numpy-discussion] problems with compiling and loading C++ extension
In-Reply-To: <Pine.LNX.4.58.0404072259470.8108@Chrestomanci>
References: <Pine.LNX.4.58.0404072259470.8108@Chrestomanci>
Message-ID: <4074D44D.3010806@colorado.edu>

Faheem Mitha wrote:
> Dear People,
> 
> I have been having an odd problem with compiling and loading a simple
> C++ extension to python (as a .so file in Linux). Unfortunately, I'll
> need to include my files in here so I apologize in advance for the
> length of this message.

Are you sure you have all the needed 'extern C' declarations in place? 
Remember, you'll be compiling this thing with a C++ compiler, but it will be 
called by Python's C runtime.  Hence, _all_ python-visible functions will need 
to be wrapped in extern C calls, including your module initializer (you often 
don't think of this one, because it's implicitly

Here's a typical snippet for a python-visible function in one of my C++ 
blitz-based extensions:

// mts_contract
extern "C" {
     static PyObject* mts_contract(PyObject *self, PyObject *args)
     {

         PyArrayObject *M_array, *T_array;
...

and here's what the module initializer looks like:


/* Module initializer -- called on first import.
    It *must* be called initMODULENAME */
extern "C" void initutils_()
{
     import_array();

     // Initialize the dimension dispatch tables
     init_mts_contract_funcs();
     init_mtt_contract2_funcs();
     init_mseqtt_contract2_funcs();

     // Create the module and add the functions
     PyObject* mod = Py_InitModule4("utils_",
                                    compiled_methods,
                                    module_doc,
                                    (PyObject*)NULL,
                                    PYTHON_API_VERSION);

     // Add some symbolic constants to the module
     PyObject* mod_dict = PyModule_GetDict(mod);

     // Make a module-level exception
     // The first argument is what the exception class will appear to be
     // in python. The return value is a class object.
     utils_Error = PyErr_NewException("utils_.utils_Error",
                                      NULL, NULL);

     // Add it to the module's dictionary. Second argument is the key
     PyDict_SetItemString(mod_dict, "utils_Error", utils_Error);

     // Add other constants here
     PyDict_SetItemString(mod_dict, "__all__",
                          Py_BuildValue("sss",
                                        "mts_contract2",
                                        "mtt_contract2",
                                        "mseqtt_contract2"
                                        ));

     // Check for errors
     if (PyErr_Occurred())
         Py_FatalError("can't initialize module utils_");
}


I hope this helps you.  Mail me if you want the full files (this is old code I 
don't use anymore, so it may not quite compile now anymore, but you can still 
look a it).

Cheers,

f




From faheem at email.unc.edu  Thu Apr  8 00:31:02 2004
From: faheem at email.unc.edu (Faheem Mitha)
Date: Thu Apr  8 00:31:02 2004
Subject: [Numpy-discussion] problems with compiling and loading C++
 extension
In-Reply-To: <4074D44D.3010806@colorado.edu>
References: <Pine.LNX.4.58.0404072259470.8108@Chrestomanci>
 <4074D44D.3010806@colorado.edu>
Message-ID: <Pine.LNX.4.58.0404080305570.8108@Chrestomanci>


On Wed, 7 Apr 2004, Fernando Perez wrote:

> I hope this helps you.  Mail me if you want the full files (this is old code I
> don't use anymore, so it may not quite compile now anymore, but you can still
> look a it).

Yes, please do send me your files (preferably as a tar.gz). Thanks for the
info. I expect you are right. I'll take a more careful look. One problem
is that I don't understand the logic behind C linkage very well. For
example, does it suffice to wrap the declaration of the function or do I
need to wrap the definition too?

Can you suggest a good reference, preferably online? My C++ books are not
much help with this.

Thanks.

                                                                Faheem.



From owen at astro.washington.edu  Fri Apr  9 10:26:09 2004
From: owen at astro.washington.edu (Russell E Owen)
Date: Fri Apr  9 10:26:09 2004
Subject: [Numpy-discussion] Questions about C API
Message-ID: <p06100502bc9c894aa9a5@[128.95.99.44]>

I'm trying to write a median function (surely one is there already, 
but if so I've not found it). The fastest technique seems to involve 
rearranging the data in place. So I'd like to offer two calls:
- A version that rearranges the input array if no local copy needed 
to be made (e.g. the input data was already contiguous and of the 
expected type). I know how to write that one.
- A safer, slower version that always makes a copy of the input data. 
I'm stuck on that. The high-level interface has flags for when to 
make a local copy, but none of them seems to mean "always make a 
local copy". Surely I am missing something there? I guess I could get 
the data, see if a local copy was made and if not, make a copy of my 
own, but that seems messy.

Also, is there any simple sample code running around for handling 
arrays of any type (e.g. with C++ templates)? Right now I'm writing 
to a specific type (since it's easy and I know what type to expect 
for my application), but I'd rather make my code more general and 
numarray-like. I figured the numarray code itself would be a good 
example, but, well, I didn't understand the parts I looked at (maybe 
I'm clueless, maybe I didn't find the right bits to examine).

-- Russell



From jdhunter at ace.bsd.uchicago.edu  Fri Apr  9 10:44:01 2004
From: jdhunter at ace.bsd.uchicago.edu (John Hunter)
Date: Fri Apr  9 10:44:01 2004
Subject: [Numpy-discussion] Questions about C API
In-Reply-To: <p06100502bc9c894aa9a5@[128.95.99.44]> (Russell E Owen's
 message of "Fri, 9 Apr 2004 10:25:10 -0700")
References: <p06100502bc9c894aa9a5@[128.95.99.44]>
Message-ID: <m2u0ztkqz6.fsf@mother.paradise.lost>

>>>>> "Russell" == Russell E Owen <owen at astro.washington.edu> writes:

    Russell> I'm trying to write a median function (surely one is
    Russell> there already, but if so I've not found it). 

from MLab import median

JDH



From jmiller at stsci.edu  Fri Apr  9 11:33:06 2004
From: jmiller at stsci.edu (Todd Miller)
Date: Fri Apr  9 11:33:06 2004
Subject: [Numpy-discussion] Questions about C API
In-Reply-To: <p06100502bc9c894aa9a5@[128.95.99.44]>
References: <p06100502bc9c894aa9a5@[128.95.99.44]>
Message-ID: <1081535554.13508.57.camel@halloween.stsci.edu>

On Fri, 2004-04-09 at 13:25, Russell E Owen wrote:
> I'm trying to write a median function (surely one is there already, 
> but if so I've not found it). The fastest technique seems to involve 
> rearranging the data in place. 

You might want to try out numarray.image.median (or
numarray.image.combine.median for versions < 0.9).

> So I'd like to offer two calls:
> - A version that rearranges the input array if no local copy needed 
> to be made (e.g. the input data was already contiguous and of the 
> expected type). I know how to write that one.
> - A safer, slower version that always makes a copy of the input data. 
> I'm stuck on that. The high-level interface has flags for when to 
> make a local copy, but none of them seems to mean "always make a 
> local copy". Surely I am missing something there? I guess I could get 
> the data, see if a local copy was made and if not, make a copy of my 
> own, but that seems messy.

The high level API is set up to do the least amount of work to guarantee
a set of constraints.  You can independently constrain numarray
properties (contiguousness, byteswap, alignment) and the array type to
be what you know your code can process.  The high level API will create
or convert arrays as necessary, and then release them when the function
is done processing (provided you manage your ref counts correctly as
shown in the manual).  So, the gist of the high level API is that it
takes some mysterious object (a poorly behaved array, an array of the
wrong type, nested lists, etc.) and converts it into a nicely formed
proxy array on which you do your computing.  In the case where the
mysterious object actually was a well behaved properly typed array,
almost no work has to be done and the proxy is the original array.

If you need a copy because your code is going to destroy the input
array,  use NA_copy() or just do a .copy() method call yourself.  Making
this kind of copy could indeed be folded into the existing API and it
would result in simpler application code, but it isn't there now.

> 
> Also, is there any simple sample code running around for handling 
> arrays of any type (e.g. with C++ templates)? 

I think Peter Verveer did some code like this in the nd_image package. 
The general idea is to constrain the arrays with type tAny in the high
level API and then switch off of a->descr->type_num to select type
specific code.

Regards,
Todd

> -- 
> Todd Miller <jmiller at stsci.edu>




From owen at astro.washington.edu  Fri Apr  9 11:49:44 2004
From: owen at astro.washington.edu (Russell E Owen)
Date: Fri Apr  9 11:49:44 2004
Subject: [Numpy-discussion] Questions about C API
In-Reply-To: <1081535554.13508.57.camel@halloween.stsci.edu>
References: <p06100502bc9c894aa9a5@[128.95.99.44]>
 <1081535554.13508.57.camel@halloween.stsci.edu>
Message-ID: <p06100505bc9c9f13c4f6@[128.95.99.44]>

At 2:32 PM -0400 2004-04-09, Todd Miller wrote:
>On Fri, 2004-04-09 at 13:25, Russell E Owen wrote:
>>  I'm trying to write a median function (surely one is there already,
>>  but if so I've not found it). The fastest technique seems to involve
>>  rearranging the data in place.
>
>You might want to try out numarray.image.median (or
>numarray.image.combine.median for versions < 0.9).

Thanks. Based on the source, it looks like that prefers a stack of 
identical images (instead of a single array, though perhaps one could 
make that case work).

John Hunter also kindly pointed out MLab (which it turns out is 
buried under linear_algebra in numarray: 
numarray.linear_algebra.mlab).

I had been working on a fast median instead of one that sorted the 
whole array, but after timing mlab's median I can probably get away 
with the simple implementation (and skip the C extension).

>>  Also, is there any simple sample code running around for handling
>>  arrays of any type (e.g. with C++ templates)?
>
>I think Peter Verveer did some code like this in the nd_image package.
>The general idea is to constrain the arrays with type tAny in the high
>level API and then switch off of a->descr->type_num to select type
>specific code.

Thank you very much for the explanation of the high level interface 
and the pointer to this sample code.

-- Russell



From Nicolas.Chauvat at logilab.fr  Sat Apr 10 14:00:10 2004
From: Nicolas.Chauvat at logilab.fr (Nicolas Chauvat)
Date: Sat Apr 10 14:00:10 2004
Subject: [Numpy-discussion] Europython: Registration open - Talk submission deadline is Apr 15th
Message-ID: <20040410205917.GA10699@logilab.fr>

Dear Scientific Pythonistas,

I'm forwarding this update about the EuroPython conference. Please
note that the talk submission deadline is April 15th but will probably
be extended a bit. I suppose most of you will be interested in the
Science Track at least.

For previous years' program, please refer to :

	http://www.europython.org/2002/sessions/talks
	http://www.europython.org/2003/sessions/talks

---------------------------------------------------------------------
Subject: [EuroPython] Europython Update: Registration open

Europython Update
=================

- Registration is now open. We apologise for the delay, but we have
had some technical problems.

- Due to this, we have decided to keep the submission of abstracts
for the refereed track open for one more day. Last submission time is
now on Sunday 11 April at 23.59 CET.

- We have a limited number of beds available in very affordable
accomodation near the conference venue. Book early before it runs out.

- We are still receiving submissions for regular talks and tutorials.
Closing date is 15 April.

- There is now a wiki at the Europython website for sprint organising. 
Start planning!

About the conference
====================

EuroPython 2004 will be held 7-9 June in G?teborg, Sweden.

The EuroPython conference will have tracks for Science, Business,
Education, Applications, Frameworks, Zope and the Python language
itself. Lightning talks, Open Space and BOF sessions are also planned.
There will be tutorials as well, both for newcomers to Python and
Python users interested in special subjects. In the days before and
after the conference, programming sprints will be arranged.


Important dates
===============

Refereed paper proposals: until 11 April.
Submission of talks: 1 March - 15 April.
Early Bird registration: 9 April - 1 May.
Accomodation booking: 9 April - 1 May (or until space runs out)

More information at http://www.europython.org.

---------------------------------------------------------------------

Hope to see you there.


-- 
Nicolas Chauvat

logilab.fr - services en informatique avanc?e et gestion de connaissances  



From jiiyso84sjoj at webnexus.com  Sat Apr 10 17:22:12 2004
From: jiiyso84sjoj at webnexus.com (Audrey Shanice)
Date: Sat Apr 10 17:22:12 2004
Subject: [Numpy-discussion] sa.ve on software?? this is for u orchard pterygium
Message-ID: <jqeeu86-cpk8-q4jiiyso84sjoj@webnexus.com>

commando arthralgic precipitin endenizen 
vicksburg seawater pretence 
choler sickbay brassiere xerobates 
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20040410/db039215/attachment-0001.html>

From faheem at email.unc.edu  Wed Apr 14 22:14:06 2004
From: faheem at email.unc.edu (Faheem Mitha)
Date: Wed Apr 14 22:14:06 2004
Subject: [Numpy-discussion] take for character arrays?
Message-ID: <slrnc7s6fp.1gd.faheem@Chrestomanci.home.earth>

Hi,

I'm trying to extract subarrays of a character array using take(), but
getting mysterious errors, for example:

ValueError: Invalid destination array: partial indices require
contiguous non-byteswapped destination

I am thinking that perhaps this operation is not supported for
character arrays, and I should convert my character array to a numeric
array instead (the strings are all of length 1).

Advice, suggestions, enlightenment appreciated.

                                                          Faheem.




From faheem at email.unc.edu  Thu Apr 15 10:07:15 2004
From: faheem at email.unc.edu (Faheem Mitha)
Date: Thu Apr 15 10:07:15 2004
Subject: [Numpy-discussion] Re: take for character arrays?
References: <slrnc7s6fp.1gd.faheem@Chrestomanci.home.earth>
Message-ID: <slrnc7tg8d.1gd.faheem@Chrestomanci.home.earth>

On Thu, 15 Apr 2004 05:13:29 +0000 (UTC), Faheem Mitha
<faheem at email.unc.edu> wrote:

> Hi,
>
> I'm trying to extract subarrays of a character array using take(), but
> getting mysterious errors, for example:
>
> ValueError: Invalid destination array: partial indices require
> contiguous non-byteswapped destination
>
> I am thinking that perhaps this operation is not supported for
> character arrays, and I should convert my character array to a numeric
> array instead (the strings are all of length 1).

I read this again and realised this does not give any specific
information, and is not very coherent. Got to avoid those late night
postings. So here are the details. If I set axis = 0 there is no
problem. I also don't have this problem with numerical arrays.

I am sure there is a simple reason for this, and hopefully a simple
workaround.

Thanks in advance.

                                                          Faheem.

**************************************************************************
In [22]: import numarray.strings as str

In [23]: import numarray

In [24]: s = str.array(['a', 'b', 'c', 'd', 'e', 'f', 'g', 'h',
'i'],shape=(3,3))

In [25]: numarray.take(s,(1,2),axis=1)
---------------------------------------------------------------------------
ValueError                                Traceback (most recent call
last)

/home/faheem/wc/corrmodel/boost/<console>

/usr/lib/python2.3/site-packages/numarray/ufunc.py in take(array,
indices, axis, outarr, clipmode)
   1779             raise ValueError("indices must be a sequence")
   1780         work = _gen.swapaxes(array, 0, axis)
-> 1781         work = work._take((indices,), outarr=outarr,
clipmode=clipmode)
   1782         return _gen.swapaxes(work, 0, axis)
   1783     else:

/usr/lib/python2.3/site-packages/numarray/generic.py in _take(self,
indices, **keywds)
    795     def _take(self, indices, **keywds):
    796         indices = list(indices)
--> 797         impliedShape, N = _takeShape(self, indices)
    798         result = self._clone(shape=impliedShape)
    799         indices = self._fix_pt_indices(indices)

/usr/lib/python2.3/site-packages/numarray/generic.py in
_takeShape(scattered, indexArrays)
    133         nShape = scattered._shape[-leftOver:]
    134         if scattered.isbyteswapped() or not
scattered.iscontiguous():
--> 135             raise ValueError("Invalid destination array:
partial indices require contiguous non-byteswapped destination")
    136     else:
    137         N = scattered._itemsize

ValueError: Invalid destination array: partial indices require
contiguous non-byteswapped destination





From jmiller at stsci.edu  Thu Apr 15 12:56:01 2004
From: jmiller at stsci.edu (Todd Miller)
Date: Thu Apr 15 12:56:01 2004
Subject: [Numpy-discussion] Re: take for character arrays?
In-Reply-To: <slrnc7tg8d.1gd.faheem@Chrestomanci.home.earth>
References: <slrnc7s6fp.1gd.faheem@Chrestomanci.home.earth>
	 <slrnc7tg8d.1gd.faheem@Chrestomanci.home.earth>
Message-ID: <1082058888.18786.53.camel@halloween.stsci.edu>

On Thu, 2004-04-15 at 13:06, Faheem Mitha wrote:
> On Thu, 15 Apr 2004 05:13:29 +0000 (UTC), Faheem Mitha
> <faheem at email.unc.edu> wrote:
> 
> > Hi,
> >
> > I'm trying to extract subarrays of a character array using take(), but
> > getting mysterious errors, for example:
> >
> > ValueError: Invalid destination array: partial indices require
> > contiguous non-byteswapped destination
> >
> > I am thinking that perhaps this operation is not supported for
> > character arrays, and I should convert my character array to a numeric
> > array instead (the strings are all of length 1).

This is a bug.  I logged it on source forge.

> 
> I read this again and realised this does not give any specific
> information, and is not very coherent. Got to avoid those late night
> postings. So here are the details. If I set axis = 0 there is no
> problem. I also don't have this problem with numerical arrays.
> 
> I am sure there is a simple reason for this, and hopefully a simple
> workaround.

Try this for a workaround:

s.swapaxes(0,1)
s = s.copy()
r = numarray.take(s, (1,2))
r.swapaxes(0,1)

Regards,
Todd

-- 
Todd Miller <jmiller at stsci.edu>




From faheem at email.unc.edu  Thu Apr 15 13:21:11 2004
From: faheem at email.unc.edu (Faheem Mitha)
Date: Thu Apr 15 13:21:11 2004
Subject: [Numpy-discussion] Re: take for character arrays?
In-Reply-To: <1082058888.18786.53.camel@halloween.stsci.edu>
References: <slrnc7s6fp.1gd.faheem@Chrestomanci.home.earth> 
 <slrnc7tg8d.1gd.faheem@Chrestomanci.home.earth> <1082058888.18786.53.camel@halloween.stsci.edu>
Message-ID: <Pine.LNX.4.58.0404151609240.19516@Chrestomanci>


On Thu, 15 Apr 2004, Todd Miller wrote:

> Try this for a workaround:
>
> s.swapaxes(0,1)
> s = s.copy()
> r = numarray.take(s, (1,2))
> r.swapaxes(0,1)

I was trying

In [50]: ts = numarray.transpose(s)

In [51]: numarray.take(ts,(1,2),axis=0)
...
ValueError: Invalid destination array: partial indices require
contiguous non-byteswapped destination

In [52]: tscopy = numarray.transpose(s).copy()

In [53]: numarray.take(tscopy,(1,2),axis=0)
Out[53]:
CharArray([['b', 'e', 'h'],
           ['c', 'f', 'i']])

Is this the same or a related bug? Thanks for the reply and info.

                                                        Faheem.



From jmiller at stsci.edu  Thu Apr 15 13:29:11 2004
From: jmiller at stsci.edu (Todd Miller)
Date: Thu Apr 15 13:29:11 2004
Subject: [Numpy-discussion] Re: take for character arrays?
In-Reply-To: <Pine.LNX.4.58.0404151609240.19516@Chrestomanci>
References: <slrnc7s6fp.1gd.faheem@Chrestomanci.home.earth>
	 <slrnc7tg8d.1gd.faheem@Chrestomanci.home.earth>
	 <1082058888.18786.53.camel@halloween.stsci.edu>
	 <Pine.LNX.4.58.0404151609240.19516@Chrestomanci>
Message-ID: <1082060853.18786.56.camel@halloween.stsci.edu>

On Thu, 2004-04-15 at 16:17, Faheem Mitha wrote:
> On Thu, 15 Apr 2004, Todd Miller wrote:
> 
> > Try this for a workaround:
> >
> > s.swapaxes(0,1)
> > s = s.copy()
> > r = numarray.take(s, (1,2))
> > r.swapaxes(0,1)
> 
> I was trying
> 
> In [50]: ts = numarray.transpose(s)
> 
> In [51]: numarray.take(ts,(1,2),axis=0)
> ...
> ValueError: Invalid destination array: partial indices require
> contiguous non-byteswapped destination
> 
> In [52]: tscopy = numarray.transpose(s).copy()
> 
> In [53]: numarray.take(tscopy,(1,2),axis=0)
> Out[53]:
> CharArray([['b', 'e', 'h'],
>            ['c', 'f', 'i']])
> 
> Is this the same or a related bug? 

Same bug.  Making a copy of the transposed array ensures that it (the
copy) is contiguous.

Regards,
Todd

-- 
Todd Miller <jmiller at stsci.edu>




From faheem at email.unc.edu  Thu Apr 15 14:03:23 2004
From: faheem at email.unc.edu (Faheem Mitha)
Date: Thu Apr 15 14:03:23 2004
Subject: [Numpy-discussion] Re: take for character arrays?
In-Reply-To: <1082060853.18786.56.camel@halloween.stsci.edu>
References: <slrnc7s6fp.1gd.faheem@Chrestomanci.home.earth> 
 <slrnc7tg8d.1gd.faheem@Chrestomanci.home.earth>  <1082058888.18786.53.camel@halloween.stsci.edu>
  <Pine.LNX.4.58.0404151609240.19516@Chrestomanci> <1082060853.18786.56.camel@halloween.stsci.edu>
Message-ID: <Pine.LNX.4.58.0404151639070.19516@Chrestomanci>


On Thu, 15 Apr 2004, Todd Miller wrote:

> On Thu, 2004-04-15 at 16:17, Faheem Mitha wrote:
> > On Thu, 15 Apr 2004, Todd Miller wrote:
> >
> > > Try this for a workaround:
> > >
> > > s.swapaxes(0,1)
> > > s = s.copy()
> > > r = numarray.take(s, (1,2))
> > > r.swapaxes(0,1)
> >
> > I was trying
> >
> > In [50]: ts = numarray.transpose(s)
> >
> > In [51]: numarray.take(ts,(1,2),axis=0)
> > ...
> > ValueError: Invalid destination array: partial indices require
> > contiguous non-byteswapped destination
> >
> > In [52]: tscopy = numarray.transpose(s).copy()
> >
> > In [53]: numarray.take(tscopy,(1,2),axis=0)
> > Out[53]:
> > CharArray([['b', 'e', 'h'],
> >            ['c', 'f', 'i']])
> >
> > Is this the same or a related bug?
>
> Same bug.  Making a copy of the transposed array ensures that it (the
> copy) is contiguous.

I am wondering if this bug affects boost-python as well. I got the same
behaviour when using boost.python's version of take.

Can you point to where in the source code the bug is? Thanks.

                                                           Faheem.



From jmiller at stsci.edu  Thu Apr 15 14:44:20 2004
From: jmiller at stsci.edu (Todd Miller)
Date: Thu Apr 15 14:44:20 2004
Subject: [Numpy-discussion] Re: take for character arrays?
In-Reply-To: <1082064038.18786.85.camel@halloween.stsci.edu>
References: <slrnc7s6fp.1gd.faheem@Chrestomanci.home.earth>
	 <slrnc7tg8d.1gd.faheem@Chrestomanci.home.earth>
	 <1082058888.18786.53.camel@halloween.stsci.edu>
	 <Pine.LNX.4.58.0404151609240.19516@Chrestomanci>
	 <1082060853.18786.56.camel@halloween.stsci.edu>
	 <Pine.LNX.4.58.0404151639070.19516@Chrestomanci>
	 <1082064038.18786.85.camel@halloween.stsci.edu>
Message-ID: <1082065422.18786.94.camel@halloween.stsci.edu>

This problem is now fixed in CVS.

Regards,
Todd
-- 
Todd Miller <jmiller at stsci.edu>




From jmiller at stsci.edu  Thu Apr 15 14:58:25 2004
From: jmiller at stsci.edu (Todd Miller)
Date: Thu Apr 15 14:58:25 2004
Subject: [Numpy-discussion] Re: take for character arrays?
In-Reply-To: <Pine.LNX.4.58.0404151639070.19516@Chrestomanci>
References: <slrnc7s6fp.1gd.faheem@Chrestomanci.home.earth>
	 <slrnc7tg8d.1gd.faheem@Chrestomanci.home.earth>
	 <1082058888.18786.53.camel@halloween.stsci.edu>
	 <Pine.LNX.4.58.0404151609240.19516@Chrestomanci>
	 <1082060853.18786.56.camel@halloween.stsci.edu>
	 <Pine.LNX.4.58.0404151639070.19516@Chrestomanci>
Message-ID: <1082064038.18786.85.camel@halloween.stsci.edu>

On Thu, 2004-04-15 at 16:40, Faheem Mitha wrote:
> On Thu, 15 Apr 2004, Todd Miller wrote:
> 
> > On Thu, 2004-04-15 at 16:17, Faheem Mitha wrote:
> > > On Thu, 15 Apr 2004, Todd Miller wrote:
> > >
> > > > Try this for a workaround:
> > > >
> > > > s.swapaxes(0,1)
> > > > s = s.copy()
> > > > r = numarray.take(s, (1,2))
> > > > r.swapaxes(0,1)
> > >
> > > I was trying
> > >
> > > In [50]: ts = numarray.transpose(s)
> > >
> > > In [51]: numarray.take(ts,(1,2),axis=0)
> > > ...
> > > ValueError: Invalid destination array: partial indices require
> > > contiguous non-byteswapped destination
> > >
> > > In [52]: tscopy = numarray.transpose(s).copy()
> > >
> > > In [53]: numarray.take(tscopy,(1,2),axis=0)
> > > Out[53]:
> > > CharArray([['b', 'e', 'h'],
> > >            ['c', 'f', 'i']])
> > >
> > > Is this the same or a related bug?
> >
> > Same bug.  Making a copy of the transposed array ensures that it (the
> > copy) is contiguous.
> 
> I am wondering if this bug affects boost-python as well. I got the same
> behaviour when using boost.python's version of take.

The bug shouldn't affect numerical arrays because there are different
underlying implementations of take.

> Can you point to where in the source code the bug is? Thanks.

I think the relevant modules are numarray.generic and numarray.ufunc. 
There's a support function which computes the shape of the take() result
in generic.py, _takeShape().  It's testing for a property,
contiguousness, which is not required by the generic take code but is
required by the numerical code.  

The fix may be to factor the exception out of _takeShape and put it into
the (numerical) ufunc.put() setup code somewhere. 

Regards,
Todd

-- 
Todd Miller <jmiller at stsci.edu>




From MAILER-DAEMON at lackingtalent.com  Thu Apr 15 21:02:07 2004
From: MAILER-DAEMON at lackingtalent.com (MAILER-DAEMON at lackingtalent.com)
Date: Thu Apr 15 21:02:07 2004
Subject: [Numpy-discussion] failure notice
Message-ID: <E1BEKXV-0005zG-1U@sc8-sf-mx1.sourceforge.net>

Hi. This is the qmail-send program at lackingtalent.com.
I'm afraid I wasn't able to deliver your message to the following addresses.
This is a permanent error; I've given up. Sorry it didn't work out.

<slrnbqbpqt.vf5.ramen at lackingtalent.com>:
Sorry, no mailbox here by that name. vpopmail (#5.1.1)

--- Below this line is a copy of the message.


From numpy-discussion at lists.sourceforge.net  Fri Apr 16 09:01:44 2004
From: numpy-discussion at lists.sourceforge.net (numpy-discussion at lists.sourceforge.net)
Date: Fri, 16 Apr 2004 06:01:44 -0700
Subject: Delivery Bot (slrnbqbpqt.vf5.ramen@lackingtalent.com)
Message-ID: <mailman.38.1490332404.18468.numpy-discussion@python.org>

An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20040416/71af9f6b/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: message.pif
Type: audio/x-wav
Size: 28008 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20040416/71af9f6b/attachment.wav>

From owen at astro.washington.edu  Fri Apr 16 13:41:01 2004
From: owen at astro.washington.edu (Russell E Owen)
Date: Fri Apr 16 13:41:01 2004
Subject: [Numpy-discussion] announcing a ds9 module and a question about indices
Message-ID: <p0610050abca5f1500f17@[128.95.99.44]>

I wrote a simple module that allows one to display numarray arrays in 
ds9. It is available as part of the RO package (RO.DS9) 
<http://www.astro.washington.edu/owen/ROPython.html>, though it is 
presently self-contained. (I may well end up expanding it.) It sends 
a copy of the data, rather than trying to share a buffer.

It uses the command-line XPA interface instead of trying to use the 
underlying C XPA API. I'm curious if anybody has gotten a python 
interface to the C XPA API working. It looks a bit messy, but popen2 
messy in other ways (detecting errors without hanging on read, for 
example).



I have what is probably a dumb question, but...if I load an array 
using PyFits and display it using ds9, I find that the array indices 
appear to be transposed relative to what I expect, i.e. numarray 
seems to use array[yind, xind].

Is this normal? Any hints on handling notation in code (I suspect 
there's not much I can do except document it)?

-- Russell



From perry at stsci.edu  Fri Apr 16 15:03:01 2004
From: perry at stsci.edu (Perry Greenfield)
Date: Fri Apr 16 15:03:01 2004
Subject: [Numpy-discussion] announcing a ds9 module and a question about indices
In-Reply-To: <p0610050abca5f1500f17@[128.95.99.44]>
Message-ID: <CEELJPECNGEGKFNDLHFFGECICDAA.perry@stsci.edu>

Russel Owen wrote:
> I wrote a simple module that allows one to display numarray arrays in 
> ds9. It is available as part of the RO package (RO.DS9) 
> <http://www.astro.washington.edu/owen/ROPython.html>, though it is 
> presently self-contained. (I may well end up expanding it.) It sends 
> a copy of the data, rather than trying to share a buffer.
> 
> It uses the command-line XPA interface instead of trying to use the 
> underlying C XPA API. I'm curious if anybody has gotten a python 
> interface to the C XPA API working. It looks a bit messy, but popen2 
> messy in other ways (detecting errors without hanging on read, for 
> example).
> 
We distributed a simple module with PyRAF that could display
to DS9 (and other similar display tools, SAOIMAGE, ximtool, etc)
using the underlying protocol used by them (offhand, I don't know
if that was the XPA interface; I seem to recall it was the
iis protocol). Unfortunately, outside of the PyRAF announcement,
we never really publicized it. We'll take a look at yours.
It could be that it has more features.
> 
> I have what is probably a dumb question, but...if I load an array 
> using PyFits and display it using ds9, I find that the array indices 
> appear to be transposed relative to what I expect, i.e. numarray 
> seems to use array[yind, xind].
> 
> Is this normal? Any hints on handling notation in code (I suspect 
> there's not much I can do except document it)?
>
Yep, it's normal. The indexing convention in Python is the same
as C with regard to multidimensional arrays and opposite to that
of Fortran and IDL. There are many reasons for that and it would
cause more trouble than it is worth trying to emulate Fortran on 
this. I know it is probably the single biggest annoyance to those
not used to it. 

Perry Greenfield 



From faheem at email.unc.edu  Sat Apr 17 16:07:07 2004
From: faheem at email.unc.edu (Faheem Mitha)
Date: Sat Apr 17 16:07:07 2004
Subject: [Numpy-discussion] add.reduce for character arrays
Message-ID: <slrnc83e38.1gd.faheem@Chrestomanci.home.earth>

Hi,

The following does not work. I am trying to do something similar to
add.reduce, except that I want strings concatenated across the
appropriate axes. I was hoping that Python would "magically" recognise
that addition in strings translated to concatenation.

Incidentally, the error message is not terribly imformative.

Is there any way to make this work, or any other method people can
suggest, or do I need to write my own function? 

Thanks.
                                                     Regards, Faheem.

**********************************************************************
In [43]: s
Out[43]:
CharArray([['a', 'b', 'c'],
           ['d', 'e', 'f'],
           ['g', 'h', 'i']])

In [44]: numarray.add.reduce(s,1)
---------------------------------------------------------------------------
error                                     Traceback (most recent call
last)

/home/faheem/wc/corrmodel/boost/<console>

error: getShape: sequence object nested more than MAXDIM




From jmiller at stsci.edu  Mon Apr 19 07:06:07 2004
From: jmiller at stsci.edu (Todd Miller)
Date: Mon Apr 19 07:06:07 2004
Subject: [Numpy-discussion] add.reduce for character arrays
In-Reply-To: <slrnc83e38.1gd.faheem@Chrestomanci.home.earth>
References: <slrnc83e38.1gd.faheem@Chrestomanci.home.earth>
Message-ID: <1082383540.19596.72.camel@halloween.stsci.edu>

On Sat, 2004-04-17 at 19:06, Faheem Mitha wrote:
> Hi,
> 
> The following does not work. I am trying to do something similar to
> add.reduce, except that I want strings concatenated across the
> appropriate axes. I was hoping that Python would "magically" recognise
> that addition in strings translated to concatenation.
> 

Use object arrays (numarray.objects) instead.  While CharArrays do have
an overloaded add operator they don't support reduction. Object arrays
have more general ufuncs which apply object oriented operators and also
support reduction.  In general, character arrays (numarray.strings) do
not support the numerical ufuncs found in numarray

> Incidentally, the error message is not terribly imformative.

True enough.  The numerical getShape code should recognize strings and
raise an exception pointing out that strings can't be passed into
numerical code.  This is fixed in CVS.

> Is there any way to make this work, or any other method people can
> suggest, or do I need to write my own function? 

Again, use numarray.objects.

> 
> Thanks.
>                                                      Regards, Faheem.
> 
> **********************************************************************
> In [43]: s
> Out[43]:
> CharArray([['a', 'b', 'c'],
>            ['d', 'e', 'f'],
>            ['g', 'h', 'i']])
> 
> In [44]: numarray.add.reduce(s,1)
> ---------------------------------------------------------------------------
> error                                     Traceback (most recent call
> last)
> 
> /home/faheem/wc/corrmodel/boost/<console>
> 
> error: getShape: sequence object nested more than MAXDIM
> 
> 
> 
> -------------------------------------------------------
> This SF.Net email is sponsored by: IBM Linux Tutorials
> Free Linux tutorial presented by Daniel Robbins, President and CEO of
> GenToo technologies. Learn everything from fundamentals to system
> administration.http://ads.osdn.com/?ad_id=1470&alloc_id=3638&op=click
> _______________________________________________
> Numpy-discussion mailing list
> Numpy-discussion at lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/numpy-discussion
-- 
Todd Miller <jmiller at stsci.edu>




From KateGillis at mails.ch  Mon Apr 19 12:28:22 2004
From: KateGillis at mails.ch (Monroe Bean)
Date: Mon Apr 19 12:28:22 2004
Subject: [Numpy-discussion] Sale on OEM software XP, office, Adobe, all under 25-99$ per CD from Bowen's SoftShop
Message-ID: <LPOVYCAAEKEQZFVYGEMCTJ@Xoom.de>

An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20040419/e37509c7/attachment-0001.html>

From faheem at email.unc.edu  Mon Apr 19 20:21:25 2004
From: faheem at email.unc.edu (Faheem Mitha)
Date: Mon Apr 19 20:21:25 2004
Subject: [Numpy-discussion] add.reduce for character arrays
In-Reply-To: <1082383540.19596.72.camel@halloween.stsci.edu>
References: <slrnc83e38.1gd.faheem@Chrestomanci.home.earth>
 <1082383540.19596.72.camel@halloween.stsci.edu>
Message-ID: <Pine.LNX.4.58.0404192315010.25609@Chrestomanci>


On Mon, 19 Apr 2004, Todd Miller wrote:

> On Sat, 2004-04-17 at 19:06, Faheem Mitha wrote:
> > Hi,
> >
> > The following does not work. I am trying to do something similar to
> > add.reduce, except that I want strings concatenated across the
> > appropriate axes. I was hoping that Python would "magically" recognise
> > that addition in strings translated to concatenation.
> >
>
> Use object arrays (numarray.objects) instead.  While CharArrays do have
> an overloaded add operator they don't support reduction. Object arrays
> have more general ufuncs which apply object oriented operators and also
> support reduction.  In general, character arrays (numarray.strings) do
> not support the numerical ufuncs found in numarray
>
> > Incidentally, the error message is not terribly imformative.
>
> True enough.  The numerical getShape code should recognize strings and
> raise an exception pointing out that strings can't be passed into
> numerical code.  This is fixed in CVS.
>
> > Is there any way to make this work, or any other method people can
> > suggest, or do I need to write my own function?
>
> Again, use numarray.objects.

Yes, that works. Thanks.

In [13]: numarray.objects.add.reduce(s,dim=1)
Out[13]: ObjectArray(['abc', 'def', 'ghi'])

BTW, that makes me wonder what the rules are for the order in which this
reduction proceeds for arrays bigger than dimension two. Since there is no
total order in this case, the answer is not obvious. For string
concatentation (since it is not commutative) at least this matters. I
didn't see this documented in the manual but I may have missed it.

                                                              Faheem.



From jmiller at stsci.edu  Tue Apr 20 09:54:01 2004
From: jmiller at stsci.edu (Todd Miller)
Date: Tue Apr 20 09:54:01 2004
Subject: [Numpy-discussion] add.reduce for character arrays
In-Reply-To: <Pine.LNX.4.58.0404192315010.25609@Chrestomanci>
References: <slrnc83e38.1gd.faheem@Chrestomanci.home.earth>
	 <1082383540.19596.72.camel@halloween.stsci.edu>
	 <Pine.LNX.4.58.0404192315010.25609@Chrestomanci>
Message-ID: <1082480003.3843.330.camel@localhost.localdomain>

On Mon, 2004-04-19 at 23:20, Faheem Mitha wrote:
> 
> Yes, that works. Thanks.
> 
> In [13]: numarray.objects.add.reduce(s,dim=1)
> Out[13]: ObjectArray(['abc', 'def', 'ghi'])
> 
> BTW, that makes me wonder what the rules are for the order in which this
> reduction proceeds for arrays bigger than dimension two. Since there is no
> total order in this case, the answer is not obvious. 

(I think) the answer is that for numarray, reductions ultimately occur
along the innermost axis.  Off-axis reductions are performed by a
swapaxis between the off-axis and the innermost axis, followed by the
reduction, followed by a swapaxis back between the new innermost axis
and the off-axis.  I'm not sure there's an especially good reason for
the swap back (since the original swapped axis no longer exists),  but
it seems to be what Numeric does and not doing it in numarray broke
extension (linear_algebra?) self-tests.

> For string
> concatentation (since it is not commutative) at least this matters. I
> didn't see this documented in the manual but I may have missed it.

Reductions always occur in order of increasing indexes with the
"reduction-so-far" as the left operand and the next index as the right
operand.  Reductions occur in a sort of depth-first manner with the
outermost dimensions varying most slowly.

Regards,
Todd
-- 
Todd Miller <jmiller at stsci.edu>




From travis at enthought.com  Wed Apr 21 14:41:19 2004
From: travis at enthought.com (Travis N. Vaught)
Date: Wed Apr 21 14:41:19 2004
Subject: [Numpy-discussion] ANN: SciPy 0.3 Released
Message-ID: <4086EA48.4080105@enthought.com>

Greetings,

SciPy 0.3 has been released and binaries are available from the 
scipy.org site.

http://www.scipy.org

Changes since the 0.1 version (0.1 enjoyed a wide release, there was a 
version 0.2 that had limited exposure) include the following:

 - general improvements: Added utility functions for constructing arrays 
by concatenation, intended mainly for command-line use. Added bmat 
constructor for easy creation of block matrices. Added mat constructor 
for constructing matrices (where * is matrix multiplication). Added many 
PIL utility functions so that if the PIL is installed, image loading, 
saving, and other operations are available. Added scipy.info, which 
handles dynamic docstrings and class help better than python's help command.

 - documentation: much improved

 - sparse: superLU upgraded to 3.0

 - optimize: Added non-linear conjugate gradient algorithm. Improvements 
to BFGS algorithm and Wolfe-condition line-search. Added two CONSTRAINED 
optimization techniques. Added simulated annealing and brute-force 
optimization strategies and Powell's method. Added many very good 1-D 
root-finding routines.

 - stats: Added many statistical distributions. Many continuous and 
discrete random variables are available with a simple mechanism for 
adding new ones. Added several statistical tests.

 - special: Added MANY new special functions. |general_function| renamed 
to |vectorize| and moved to scipy_base. Improved the way domain errors 
are handled (user specified display of problems). More tests on special 
functions added.

 - fftpack: replaced with fftpack2--can optionally be configured to 
support djbfft

 - io: Reading of MATLAB .mat files (version 4 and 5) supported. Writing 
version 4 .mat files supported. Added very flexible and general_purpose 
text file reader. Added shelving save operation to save variables into 
an importable module. Routines for reading and writing fortran-records.

 - linalg: Linear algebra is greatly improved over SciPy 0.1. ATLAS is 
now optional (but encouraged). Most lapack functions have simplified 
interfaces (all lapack and blas available). Matrix exponentials and 
other matrix functions added.

 - signal: Added support for filter design and LTI system analysis.

 - xplt: Updated xplt to use newer pygist so that Windows platform is 
supported. Addition of several plotting types.

 - integrate: added another ODE integrator. Added romberg integration 
and several other integration approaches.


The complete release notes can be found here:

http://www.scipy.org/download/scipy_release_notes_0.3.html


You'll also notice that scipy.org is now a spanking new Plone portal 
(http://www.plone.org -- keep up the good work, plone folks).

This is the first binary release in a long time and we hope to increase 
the frequency to every 6 months or so.

If you'd like to follow or join the community, you can subscribe to the 
mailing lists here:

http://www.scipy.org/mailinglists/

Best Regards,

Travis


BTW SciPy is:
-------------
SciPy is an open source library of scientific tools for Python. SciPy 
supplements the popular Numeric module, gathering a variety of high 
level science and engineering modules together as a single package.

SciPy includes modules for graphics and plotting, optimization, 
integration, special functions, signal and image processing, genetic 
algorithms, ODE solvers, and others.




From ville at e.math.helsinki.fi  Thu Apr 22 05:56:17 2004
From: ville at e.math.helsinki.fi (Ville Hakulinen)
Date: Thu Apr 22 05:56:17 2004
Subject: [Numpy-discussion] Numarrays from C arrays without copying
Message-ID: <20040422125131.GA7003@e.math.helsinki.fi>

Dear all,

I would like to turn a large C array allocated by a library into a
Numarray object without copying the contents. In other words I have
a given array of double precision floats and I want to get a Numarray
object, where the data pointer points to this array and no additional
memory is allocated. Naturally when the reference count of the Numarray
object gets back to zero, the object is freed, but the array itself
is left as a nuisance for the C code.

Is there a standard way of doing this?

Thanks in advance,
-- 
Ville



From jmiller at stsci.edu  Thu Apr 22 06:11:02 2004
From: jmiller at stsci.edu (Todd Miller)
Date: Thu Apr 22 06:11:02 2004
Subject: [Numpy-discussion] Numarrays from C arrays without copying
In-Reply-To: <20040422125131.GA7003@e.math.helsinki.fi>
References: <20040422125131.GA7003@e.math.helsinki.fi>
Message-ID: <1082639406.3272.10.camel@halloween.stsci.edu>

On Thu, 2004-04-22 at 08:51, Ville Hakulinen wrote:
> Dear all,
> 
> I would like to turn a large C array allocated by a library into a
> Numarray object without copying the contents. In other words I have
> a given array of double precision floats and I want to get a Numarray
> object, where the data pointer points to this array and no additional
> memory is allocated. Naturally when the reference count of the Numarray
> object gets back to zero, the object is freed, but the array itself
> is left as a nuisance for the C code.
> 
> Is there a standard way of doing this?

No, not yet.  You're not the first person to ask for this but I'd
appreciate it if you'd explain why you need it.  So far,  not having
that feature is a conscious omission to keep complexity down.

Regards,
Todd

-- 
Todd Miller <jmiller at stsci.edu>




From crasmussen at lanl.gov  Thu Apr 22 07:29:02 2004
From: crasmussen at lanl.gov (Craig Rasmussen)
Date: Thu Apr 22 07:29:02 2004
Subject: [Numpy-discussion] Numarrays from C arrays without copying
In-Reply-To: <1082639406.3272.10.camel@halloween.stsci.edu>
References: <20040422125131.GA7003@e.math.helsinki.fi> <1082639406.3272.10.camel@halloween.stsci.edu>
Message-ID: <560EC36A-9469-11D8-AAE7-000A957CA856@lanl.gov>

On Apr 22, 2004, at 7:10 AM, Todd Miller wrote:

>> I would like to turn a large C array allocated by a library into a
>> Numarray object without copying the contents. In other words I have
>> a given array of double precision floats and I want to get a Numarray
>> object, where the data pointer points to this array and no additional
>> memory is allocated. Naturally when the reference count of the 
>> Numarray
>> object gets back to zero, the object is freed, but the array itself
>> is left as a nuisance for the C code.
>>
>> Is there a standard way of doing this?
>
> No, not yet.  You're not the first person to ask for this but I'd
> appreciate it if you'd explain why you need it.  So far,  not having
> that feature is a conscious omission to keep complexity down.

The reason I need this is two fold:

1. I want to export to Python existing Fortran arrays.  If you want
Fortran and Python to share array memory, you can't always expect
Fortran programmers to use memory allocated in Python.  In fact,
since there is no standards conforming way to associate a Fortran
array pointer with Python memory, you have to do the opposite,
associate a Python array with Fortran memory.

2. In the CoArray Python module, we MUST use special memory
allocators that are parallel aware.

Because of these requirements, our only choice is to use Numeric.

Regards,
Craig




From haase at msg.ucsf.edu  Thu Apr 22 09:47:00 2004
From: haase at msg.ucsf.edu (Sebastian Haase)
Date: Thu Apr 22 09:47:00 2004
Subject: [Numpy-discussion] Numarrays from C arrays without copying
In-Reply-To: <560EC36A-9469-11D8-AAE7-000A957CA856@lanl.gov>
References: <20040422125131.GA7003@e.math.helsinki.fi> <1082639406.3272.10.camel@halloween.stsci.edu> <560EC36A-9469-11D8-AAE7-000A957CA856@lanl.gov>
Message-ID: <200404220946.01190.haase@msg.ucsf.edu>

Hi Craig,
I am sharing arrays between C and Python numarray (w/o any copying) for more 
than a year by now - please check my postings and (mostly) Todd's replys on 
this list. The code looks mostly like this:
<code sniplet>
 int shape[4]= { n,nCams,pic_ny,pic_nx };
   PyObject *obj, *obj2;

   obj = PyBuffer_FromReadWriteMemory(theRing.bufferPtr(), 
theRing.bufferSize());
   obj2 =(PyObject*) NA_NewAllFromBuffer( 4, shape, tUInt16, obj, 0, 0, 
NA_ByteOrder(), 1, 1/*writable*/) ;
   wxASSERT(obj != NULL);
   wxASSERT(obj2 != NULL);
   PyDict_SetItemString(globals, "ring", obj2);
   Py_XDECREF(obj);
   Py_XDECREF(obj2);
</code sniplet>
It even works with records arrays!
Luckily my arrays stay (mostly) around until the program quits. 
So I also don't have a good answer for 'who should free the memory?'

As far as numarray/Python concerns - once it's reference counter goes to zero 
- a 'delete[]' would be needed if the array was allocated with a C++ 'new' 
but a 'free()' is required if the memory got "malloc()'ed" in C.
Our internal discussion here found that the only real solution to this might 
be to extend numarray's object structure by a functions pointer 
('callToKill') and also an  argument pointer (the array should be delete'd or 
free'd).
But then we felt this might be to much to ask (to much of a change / overhead) 
that this proposal would not get accepted - so we choose to not bring it up.
Please comment,
Regards
Sebastian Haase





On Thursday 22 April 2004 07:28 am, Craig Rasmussen wrote:
> On Apr 22, 2004, at 7:10 AM, Todd Miller wrote:
> >> I would like to turn a large C array allocated by a library into a
> >> Numarray object without copying the contents. In other words I have
> >> a given array of double precision floats and I want to get a Numarray
> >> object, where the data pointer points to this array and no additional
> >> memory is allocated. Naturally when the reference count of the
> >> Numarray
> >> object gets back to zero, the object is freed, but the array itself
> >> is left as a nuisance for the C code.
> >>
> >> Is there a standard way of doing this?
> >
> > No, not yet.  You're not the first person to ask for this but I'd
> > appreciate it if you'd explain why you need it.  So far,  not having
> > that feature is a conscious omission to keep complexity down.
>
> The reason I need this is two fold:
>
> 1. I want to export to Python existing Fortran arrays.  If you want
> Fortran and Python to share array memory, you can't always expect
> Fortran programmers to use memory allocated in Python.  In fact,
> since there is no standards conforming way to associate a Fortran
> array pointer with Python memory, you have to do the opposite,
> associate a Python array with Fortran memory.
>
> 2. In the CoArray Python module, we MUST use special memory
> allocators that are parallel aware.
>
> Because of these requirements, our only choice is to use Numeric.
>
> Regards,
> Craig



From jmiller at stsci.edu  Thu Apr 22 09:58:18 2004
From: jmiller at stsci.edu (Todd Miller)
Date: Thu Apr 22 09:58:18 2004
Subject: [Numpy-discussion] Numarrays from C arrays without copying
In-Reply-To: <4087E8FB.8020801@pfdubois.com>
References: <20040422125131.GA7003@e.math.helsinki.fi>
	 <1082639406.3272.10.camel@halloween.stsci.edu>
	 <4087E8FB.8020801@pfdubois.com>
Message-ID: <1082653045.3272.308.camel@halloween.stsci.edu>

On Thu, 2004-04-22 at 11:47, Paul F. Dubois wrote:
> This is what PyArray_FromDimsAndData was for, and I believe it is fairly 
> heavily used.

That's good to know.  I was just discussing with Perry how to expand the
C-API to support this stuff.  Perhaps we don't have to.

> Todd Miller wrote:
> > On Thu, 2004-04-22 at 08:51, Ville Hakulinen wrote:
> > 
> >>Dear all,
> >>
> >>I would like to turn a large C array allocated by a library into a
> >>Numarray object without copying the contents. In other words I have
> >>a given array of double precision floats and I want to get a Numarray
> >>object, where the data pointer points to this array and no additional
> >>memory is allocated. Naturally when the reference count of the Numarray
> >>object gets back to zero, the object is freed, but the array itself
> >>is left as a nuisance for the C code.
> >>
> >>Is there a standard way of doing this?
> > 
> > 
> > No, not yet.  You're not the first person to ask for this but I'd
> > appreciate it if you'd explain why you need it.  So far,  not having
> > that feature is a conscious omission to keep complexity down.
> > 
> > Regards,
> > Todd
> > 
-- 
Todd Miller <jmiller at stsci.edu>




From ville at e.math.helsinki.fi  Thu Apr 22 11:24:10 2004
From: ville at e.math.helsinki.fi (Ville Hakulinen)
Date: Thu Apr 22 11:24:10 2004
Subject: [Numpy-discussion] Numarrays from C arrays without copying
In-Reply-To: <1082639406.3272.10.camel@halloween.stsci.edu>
References: <20040422125131.GA7003@e.math.helsinki.fi> <1082639406.3272.10.camel@halloween.stsci.edu>
Message-ID: <20040422182332.GA8864@e.math.helsinki.fi>

On Thu, Apr 22, 2004 at 09:10:07AM -0400, Todd Miller wrote:
> No, not yet.  You're not the first person to ask for this but I'd
> appreciate it if you'd explain why you need it.  So far,  not having
> that feature is a conscious omission to keep complexity down.

First I managed not to reply to numpy-discussion, so here we go again:

We are building a Python interface to PETSc 2.2
(http://www-unix.mcs.anl.gov/petsc/petsc-2/) using SWIG. It would
be very convenient to access the part of a PETSc vector that is
local to a process(or) using Numarray.

Other options would be to just write functions to access the part
of a vector independently of Numarray or to figure out whether one
could tell PETSc to use the memory area allocated by Numarray.
-- 
Ville



From crasmussen at lanl.gov  Thu Apr 22 11:33:01 2004
From: crasmussen at lanl.gov (Craig Rasmussen)
Date: Thu Apr 22 11:33:01 2004
Subject: [Numpy-discussion] Numarrays from C arrays without copying
In-Reply-To: <200404220946.01190.haase@msg.ucsf.edu>
References: <20040422125131.GA7003@e.math.helsinki.fi> <1082639406.3272.10.camel@halloween.stsci.edu> <560EC36A-9469-11D8-AAE7-000A957CA856@lanl.gov> <200404220946.01190.haase@msg.ucsf.edu>
Message-ID: <6BC5B671-948B-11D8-AAE7-000A957CA856@lanl.gov>

On Apr 22, 2004, at 10:46 AM, Sebastian Haase wrote:

>    obj = PyBuffer_FromReadWriteMemory(theRing.bufferPtr(),
> theRing.bufferSize());
>    obj2 =(PyObject*) NA_NewAllFromBuffer( 4, shape, tUInt16, obj, 0, 0,
> NA_ByteOrder(), 1, 1/*writable*/) ;

It looks like this will work.

I thought Todd had been saying that this wasn't possible.  Perhaps I
did not phrase my question very well when I asked earlier.

Thanks,
Craig




From crasmussen at lanl.gov  Thu Apr 22 11:35:12 2004
From: crasmussen at lanl.gov (Craig Rasmussen)
Date: Thu Apr 22 11:35:12 2004
Subject: [Numpy-discussion] Numarrays from C arrays without copying
In-Reply-To: <1082653045.3272.308.camel@halloween.stsci.edu>
References: <20040422125131.GA7003@e.math.helsinki.fi> <1082639406.3272.10.camel@halloween.stsci.edu> <4087E8FB.8020801@pfdubois.com> <1082653045.3272.308.camel@halloween.stsci.edu>
Message-ID: <B9B0CCD2-948B-11D8-AAE7-000A957CA856@lanl.gov>

On Apr 22, 2004, at 10:57 AM, Todd Miller wrote:

>> This is what PyArray_FromDimsAndData was for, and I believe it is 
>> fairly
>> heavily used.
>
> That's good to know.  I was just discussing with Perry how to expand 
> the
> C-API to support this stuff.  Perhaps we don't have to.

So is there an equivalent NA_FromDimsAndData?  I have been using
PyArray_FromDimsAnd Data.

Craig




From jmiller at stsci.edu  Thu Apr 22 12:09:09 2004
From: jmiller at stsci.edu (Todd Miller)
Date: Thu Apr 22 12:09:09 2004
Subject: [Numpy-discussion] Numarrays from C arrays without copying
In-Reply-To: <6BC5B671-948B-11D8-AAE7-000A957CA856@lanl.gov>
References: <20040422125131.GA7003@e.math.helsinki.fi>
	 <1082639406.3272.10.camel@halloween.stsci.edu>
	 <560EC36A-9469-11D8-AAE7-000A957CA856@lanl.gov>
	 <200404220946.01190.haase@msg.ucsf.edu>
	 <6BC5B671-948B-11D8-AAE7-000A957CA856@lanl.gov>
Message-ID: <1082660922.3272.504.camel@halloween.stsci.edu>

On Thu, 2004-04-22 at 14:32, Craig Rasmussen wrote:
> On Apr 22, 2004, at 10:46 AM, Sebastian Haase wrote:
> 
> >    obj = PyBuffer_FromReadWriteMemory(theRing.bufferPtr(),
> > theRing.bufferSize());
> >    obj2 =(PyObject*) NA_NewAllFromBuffer( 4, shape, tUInt16, obj, 0, 0,
> > NA_ByteOrder(), 1, 1/*writable*/) ;
> 
> It looks like this will work.
> 

It'll definitely work.  I'd forgotten about it.

> I thought Todd had been saying that this wasn't possible.  Perhaps I
> did not phrase my question very well when I asked earlier.

Nope,  your question was fine.

I'm hopeful that PyArray_FromDimsAndData can be improved 
so that this becomes easier for numarray-1.0.

Regards,
Todd
-- 
Todd Miller <jmiller at stsci.edu>




From jmiller at stsci.edu  Thu Apr 22 13:07:01 2004
From: jmiller at stsci.edu (Todd Miller)
Date: Thu Apr 22 13:07:01 2004
Subject: [Numpy-discussion] Numarrays from C arrays without copying
In-Reply-To: <B9B0CCD2-948B-11D8-AAE7-000A957CA856@lanl.gov>
References: <20040422125131.GA7003@e.math.helsinki.fi>
	 <1082639406.3272.10.camel@halloween.stsci.edu>
	 <4087E8FB.8020801@pfdubois.com>
	 <1082653045.3272.308.camel@halloween.stsci.edu>
	 <B9B0CCD2-948B-11D8-AAE7-000A957CA856@lanl.gov>
Message-ID: <1082664345.3272.592.camel@halloween.stsci.edu>

On Thu, 2004-04-22 at 14:34, Craig Rasmussen wrote:
> On Apr 22, 2004, at 10:57 AM, Todd Miller wrote:
> 
> >> This is what PyArray_FromDimsAndData was for, and I believe it is 
> >> fairly
> >> heavily used.
> >
> > That's good to know.  I was just discussing with Perry how to expand 
> > the
> > C-API to support this stuff.  Perhaps we don't have to.
> 
> So is there an equivalent NA_FromDimsAndData?

No.  I was making the mistake of thinking there needed to be.

>   I have been using
> PyArray_FromDimsAndData.

That's fine and should basically work.  

Between numarray and Numeric there is currently the not-so-subtle
difference that numarray copies the external data buffer and Numeric
just refers to it.

This is a documented difference,  but my vote is for changing in favor
of better functionality and Numeric compatibility.  I'll probably just
quietly change it unless there is a deluge of dissenters...

-- 
Todd Miller <jmiller at stsci.edu>




From info at deutsches-panel.com  Tue Apr 27 07:50:02 2004
From: info at deutsches-panel.com (Das deutsche Panel)
Date: Tue Apr 27 07:50:02 2004
Subject: [Numpy-discussion] Einladung
Message-ID: <SUJXHWVOQZNRSHCOQEFULLL@yahoo.com>

K?nnen Sie uns Ihre Meinung mitteilen ?

Das deutsche Panel ist eine Marktforschungswebsite, die mit dem Ziel errichtet worden ist, die Bev?lkerung Deutschlands in Bezug auf verschiedene Bereiche, wie z.B. neue Fernsehwerbung zu befragen. Wenn Sie daran interessiert sind, Ihre Meinungen zu ver?ffentlichen, f?llen Sie bitte unseren Fragebogen aus, bei :

http://www.deutsches-panel.com/rec/index.php?id=numpy-discussion at lists.sourceforge.net

Wenn Sie unser Panel besuchen und an unseren Studien teilnehmen, werden Sie automatisch bei einer Verlosung teilnehmen. Je nach Umfang der Studie, k?nnen Sie verschiedene Objekte im Wert zwischen 200 und 500 Euro gewinnen. So haben Sie die M?glichkeit verschiedene Objekte wie Playstations, Handies oder computerverbundene Ger?te zu gewinnen.

Falls Sie sich entscheiden den Wert des Objektes einer Hilfeorganisation zu spenden, werden wir Ihnen eine Liste mit unseren Partnerorganisationen zur Verf?gung stellen. Dadurch erm?glichen wir Ihnen die Wahl einer Organisation, der Sie spenden wollen.
Best?tigungen dieser Spenden werden auf Anfrage gew?hrt.

Falls Sie zuk?nftig keine weiteren Einladungen von uns erhalten wollen, klicken Sie bitte auf folgende Link um Ihre Einschreibung abzubestellen :
http://www.deutsches-panel.com/rec/unsubscribe.php?id=numpy-discussion at lists.sourceforge.net

Die Ergebnisse der Verlosungen und Spenden werden Ihnen online auf dem ?Das deutsche Panel Site? zur Verf?gung stehen.

Mit freundlichen Gr??en ,
Anna Klaus
Das deutsche Panel





From scipy-user-bounces at scipy.net  Wed Apr 28 12:21:00 2004
From: scipy-user-bounces at scipy.net (scipy-user-bounces at scipy.net)
Date: Wed Apr 28 12:21:00 2004
Subject: [Numpy-discussion] Your message to SciPy-user awaits moderator approval
Message-ID: <mailman.1774.1083180633.1305.scipy-user@scipy.net>

Your mail to 'SciPy-user' with the subject

    RE: Incoming Msg

Is being held until the list moderator can review it for approval.

The reason it is being held:

    Post by non-member to a members-only list

Either the message will get posted to the list, or you will receive
notification of the moderator's decision.  If you would like to cancel
this posting, please visit the following URL:

    http://www.scipy.net/mailman/confirm/scipy-user/8c8e42fd74bfe658b790f95e8d6accd96369bf95




From Marc.Poinot at onera.fr  Thu Apr 29 00:52:04 2004
From: Marc.Poinot at onera.fr (Marc Poinot)
Date: Thu Apr 29 00:52:04 2004
Subject: [Numpy-discussion] numarray on 64 bits
Message-ID: <4090B3FA.F8D4665A@onera.fr>

Did anybody use numarray on a 64 bits platform ? (e.g. SGI/Irix)
The package works on our 64 bits platforms... using 32 bits mode,
but not using 64 mode. Big crash in _ufunc... Is there a flag to
set somewhere, trick, hint, noway ?

-MP- 
-----------------------------------------------------------------------
 Marc POINOT             Alias: marcvs        Email: poinot at onera.fr
 ONERA -MFE/DSNA/ELSA    Tel: 01.46.73.42.84  Info: elsa-info at onera.fr
 29, Div. Leclerc        Fax: 01.46.73.41.66  Site:     
 92322 Chatillon FRANCE  Project: elsA        Web: http://www.onera.fr



From jmiller at stsci.edu  Thu Apr 29 04:17:06 2004
From: jmiller at stsci.edu (Todd Miller)
Date: Thu Apr 29 04:17:06 2004
Subject: [Numpy-discussion] numarray on 64 bits
In-Reply-To: <4090B3FA.F8D4665A@onera.fr>
References: <4090B3FA.F8D4665A@onera.fr>
Message-ID: <1083237365.5730.41.camel@localhost.localdomain>

On Thu, 2004-04-29 at 03:51, Marc Poinot wrote:
> Did anybody use numarray on a 64 bits platform ? 

Numarray has been used on Compaq Alpha/Tru64 and also SGI Altix,  but
neither is an everyday platform for us at STScI.

> (e.g. SGI/Irix)
> The package works on our 64 bits platforms... using 32 bits mode,
> but not using 64 mode. Big crash in _ufunc... 

This is probably a bug which has was created since we dropped Tru64 at
STScI.  A gdb traceback would be useful here to help understand where
the crash occurred.  Generally, crashes result from the improper use of
ints/longs in the numarray C-code.  

> Is there a flag to set somewhere, trick, hint, noway ?

There's really only one flag that matters for 64-bit operation on UNIX:
LP64.  It's controlled/propagated in the file generate.py.  LP64 asserts
that longs and pointers are 64-bit, while ints remain 32-bit. I imagine
you've already set this.

Regards, 
Todd

-- 
Todd Miller <jmiller at stsci.edu>




From rowen at u.washington.edu  Thu Apr 29 10:19:06 2004
From: rowen at u.washington.edu (Russell E Owen)
Date: Thu Apr 29 10:19:06 2004
Subject: [Numpy-discussion] rationalizing functions that return number of elements
Message-ID: <p06100505bcb6dae36e66@[128.95.99.44]>

I'm hoping we might rationalize the names of functions that return 
the number of elements in an array and fix the associated help 
strings. At present we have:

numarray function size(array, axis=None)
   Seems to return # of elements in array, or in a particular axis,
   but its help says something else:
   "size  returns the size in bytes of an array."

numarray.ma has function size(a, axis=None)
   Seems the same as numarray.size but the help is wrong,
   and in a different way:
   "returns shape tuple for 'a'.  MaskedArray version."

nummarray.ma arrays have method "size(seolf, axis=None):
   The same as the others and correctly documented:
   "Number of elements in array, or in a particular axis."

numarray arrays have method nelements()
   The same as "size" above, but has no axis argument:
   "nelements() -> total number of elements in array."

I strongly encourage you to settle on one name (and one
argument list, i.e. with "axis"). The current situation
forces one to treat ma arrays differently than normal arrays
in an area where there's no need for such a difference.
It is also confusing.

"size" is a perfectly good name and is already used
in 3 out of 4 cases, so adopting it will break the
least existing code and deprecate only one odd method.

On the other hand, "nelements" has the advantage
that it is less ambiguous (based on the incorrect help
strings). So if you really think it's worth the pain
you could switch to that.

My personal leaning is towards "size", but it's not
my decision.

By the way, "itemsize" already exists as an
array method, so "size" is NOT needed for that
(and would be a wretched choice--ambigious and
the potential to mysteriously break existing code).

-- Russell

P.S. I reported "nelements" as PR 934514
<http://sourceforge.net/tracker/index.php?func=detail&aid=934514&group_id=1369&atid=450446>
and the incorrect help strings as PR 944678
<http://sourceforge.net/tracker/index.php?func=detail&aid=944678&group_id=1369&atid=450446>

P.P.S. here's a log showing the tests that convinced me size works as 
I said, not as the help strings say:
>>>  a = num.array([[1.1, 2.2, 3.3], [4.4, 5.5, 6.6]])
>>>  a
array([[ 1.1,  2.2,  3.3],
        [ 4.4,  5.5,  6.6]])
>>>  numarray.size(a)
6
>>>  numarray.size(a, 1)
3
>>>  aa = numarray.ma.array(a)
>>>  numarray.ma.size(aa)
6
>>>  numarray.ma.size(aa, 1)
3
>>>  aa.size()
6
>>>  aa.size(1)
3
>>>  a.nelements()
6