From verveer at embl-heidelberg.de  Mon Sep  1 01:44:03 2003
From: verveer at embl-heidelberg.de (Peter Verveer)
Date: Mon Sep  1 01:44:03 2003
Subject: [Numpy-discussion] type info
Message-ID: <200309011043.30784.verveer@embl-heidelberg.de>

Hi all,

Is there some way in numarray to find out what the maximum and minimum values 
are that a type can hold? For instance, is there a convenient way to find out 
what the maximal possible value of the Float32 type is?

Cheers, Peter

-- 
Dr. Peter J. Verveer
Cell Biology and Cell Biophysics Programme
EMBL
Meyerhofstrasse 1
D-69117 Heidelberg
Germany
Tel. : +49 6221 387245
Fax  : +49 6221 387242
Email: Peter.Verveer at embl-heidelberg.de





From verveer at embl-heidelberg.de  Mon Sep  1 02:35:03 2003
From: verveer at embl-heidelberg.de (Peter Verveer)
Date: Mon Sep  1 02:35:03 2003
Subject: [Numpy-discussion] sum and mean methods behaviour
Message-ID: <200309011134.39281.verveer@embl-heidelberg.de>

Hi All,

I noticed that the sum() and mean() methods of numarrays use the precision of 
the given array in their calculations. That leads to resuls like this:

>>> array([255, 255], Int8).sum()
-2
>>> array([255, 255], Int8).mean()
-1.0

Would it not be better to use double precision internally and return the 
correct result?

Cheers, Peter

-- 
Dr. Peter J. Verveer
Cell Biology and Cell Biophysics Programme
EMBL
Meyerhofstrasse 1
D-69117 Heidelberg
Germany
Tel. : +49 6221 387245
Fax  : +49 6221 387242
Email: Peter.Verveer at embl-heidelberg.de





From jmiller at stsci.edu  Mon Sep  1 04:57:04 2003
From: jmiller at stsci.edu (Todd Miller)
Date: Mon Sep  1 04:57:04 2003
Subject: [Numpy-discussion] type info
In-Reply-To: <200309011043.30784.verveer@embl-heidelberg.de>
References: <200309011043.30784.verveer@embl-heidelberg.de>
Message-ID: <1062417380.3459.63.camel@localhost.localdomain>

On Mon, 2003-09-01 at 04:43, Peter Verveer wrote:
> Hi all,
> 
> Is there some way in numarray to find out what the maximum and minimum values 
> are that a type can hold? 

I think the answer is "no".  I added a feature request on Source Forge.

> For instance, is there a convenient way to find out 
> what the maximal possible value of the Float32 type is?

Nope.

> Cheers, Peter
-- 
Todd Miller <jmiller at stsci.edu>




From pearu at cens.ioc.ee  Mon Sep  1 05:57:03 2003
From: pearu at cens.ioc.ee (Pearu Peterson)
Date: Mon Sep  1 05:57:03 2003
Subject: [Numpy-discussion] type info
In-Reply-To: <200309011043.30784.verveer@embl-heidelberg.de>
Message-ID: <Pine.LNX.4.21.0309011551310.9326-100000@cens.kybi>

On Mon, 1 Sep 2003, Peter Verveer wrote:

> Is there some way in numarray to find out what the maximum and minimum values 
> are that a type can hold? For instance, is there a convenient way to find out 
> what the maximal possible value of the Float32 type is?

I am not completely sure what is behind of the following way but

>>> numarray.fromstring('\xff\xff\x7f\x7f','f')
array([  3.40282347e+38], type=Float32)

seems to give the maximum value for Float32.

Pearu





From verveer at embl-heidelberg.de  Mon Sep  1 06:11:02 2003
From: verveer at embl-heidelberg.de (Peter Verveer)
Date: Mon Sep  1 06:11:02 2003
Subject: [Numpy-discussion] type info
In-Reply-To: <Pine.LNX.4.21.0309011551310.9326-100000@cens.kybi>
References: <Pine.LNX.4.21.0309011551310.9326-100000@cens.kybi>
Message-ID: <200309011510.02692.verveer@embl-heidelberg.de>

On Monday 01 September 2003 14:55, Pearu Peterson wrote:
> On Mon, 1 Sep 2003, Peter Verveer wrote:
> > Is there some way in numarray to find out what the maximum and minimum
> > values are that a type can hold? For instance, is there a convenient way
> > to find out what the maximal possible value of the Float32 type is?
>
> I am not completely sure what is behind of the following way but
>
> >>> numarray.fromstring('\xff\xff\x7f\x7f','f')
>
> array([  3.40282347e+38], type=Float32)
>
> seems to give the maximum value for Float32.
>
> Pearu

Sure, it is not too difficult to find the minimum and maximum values of each 
type, given the bitsize (although there may be portability issues). But it 
would be convenient to have such properties directly accessible in numarray, 
I think.

Peter




From jmiller at stsci.edu  Tue Sep  2 11:34:04 2003
From: jmiller at stsci.edu (Todd Miller)
Date: Tue Sep  2 11:34:04 2003
Subject: [Numpy-discussion] sum and mean methods behaviour
In-Reply-To: <200309011134.39281.verveer@embl-heidelberg.de>
References: <200309011134.39281.verveer@embl-heidelberg.de>
Message-ID: <1062527563.16365.283.camel@halloween.stsci.edu>

On Mon, 2003-09-01 at 05:34, Peter Verveer wrote:
> Hi All,
> 
> I noticed that the sum() and mean() methods of numarrays use the precision of 
> the given array in their calculations. That leads to resuls like this:
> 
> >>> array([255, 255], Int8).sum()
> -2
> >>> array([255, 255], Int8).mean()
> -1.0
> 
> Would it not be better to use double precision internally and return the 
> correct result?
> 
> Cheers, Peter
> 

Hi Peter,

I thought about this a lot yesterday and today talked it over with
Perry.   There are several ways to fix the problem with mean() and
sum(), and I'm hoping that you and the rest of the community will help
sort them out.

(1) The first "solution" is to require users to do their own up-casting
prior to calling mean() or sum().  This gives the end user fine control
over storage cost but leaves the C-like pitfall/bug you discovered.   I
mention this because this is how the numarray/Numeric reductions are
designed.  Is there a reason why the numarray/Numeric reductions don't
implicitly up-cast? 

(2) The second way is what you proposed:  use double precision within
mean and sum.  This has great simplicity but gives no control over
storage usage, and as implemented, the storage would be much higher than
one might think, potentially 8x.

(3) Lastly, Perry suggested a more radical approach:  rather than
changing the mean and sum methods themselves,  we could alter the
universal function accumulate and reduce methods to implicitly use
additional precision.  Perry's idea was to make all accumulations and
reductions up-cast their results to the largest type of the current
family, either Bool, Int64, Float64, or Complex64.   By doing this, we
can improve the utility of the reductions and accumulations as well as
fixing the problem with sum and mean.

-- 
Todd Miller 			jmiller at stsci.edu
STSCI / ESS / SSB




From jmiller at stsci.edu  Tue Sep  2 12:06:12 2003
From: jmiller at stsci.edu (Todd Miller)
Date: Tue Sep  2 12:06:12 2003
Subject: [Numpy-discussion] Possible bug
In-Reply-To: <3F40D29E.2090308@sympatico.ca>
References: <200308181401.17137.verveer@embl-heidelberg.de> 
	<3F40D29E.2090308@sympatico.ca>
Message-ID: <1062529529.16822.318.camel@halloween.stsci.edu>

These problems are (partially) addressed in numarray-0.7 with the
addition of a check_overflow parameter to numarray.fromlist().  The
checked fromlist is now used in the implementation of ufuncs (e.g.
multiply), preventing the silent truncation of scalar values to the type
of the array.  

Problem c:  solved -- 1001 scalar was silently truncated to Int8 before
multiplication.

Problem e:  solved -- 3000 was silently truncated to Int8 during
fromlist.  Use fromlist() rather than array() and add
check_overflow=1 parameter.

Problem f: rejected  -- working as designed, won't change.  Floating
point numbers passed into array() are silently truncated if the type is
not a floating point type.  Operating on an integer array and a floating
point scalar returns a floating point array by design;  likewise,
operating on an integer array with an integer scalar returns an integer
array.  This is all expected behavior.

Problem g: unaddressed -- working as designed, open to discussion.  The
silent truncation of g to Int16 is documented in the description of
astype().  It's possible to add a check_overflow flag to astype()

Thanks for the feedback,
Todd

On Mon, 2003-08-18 at 09:20, Colin J. Williams wrote:
> # ta.py to check Int8, Int16
> import numarray as _n
> import numarray.numerictypes as _nt
> b= _n.arange(4, type= _nt.Int8)
> print 'b, b._type:', b, b._type
> c= b*1001                                        # Grief here - type not 
> updated
> print 'c, c._type:', c, c._type    
> e= _n.array([1, -2, 3000, 4.6], type= _nt.Int8)  # undocumented feature 
> fraction discarded
>                                                  # and lowest eight bits 
> retained as a signed
>                                                  # integer.
> print 'e, e._type:', e, e._type
> f= _n.array([1, 2, 3, 4.6], type= _nt.Int8) * 9.6
> print 'f, f._type:', f, f._type
> g= (f.copy()*2000).astype(_nt.Int16)             # undocumented - see e 
> above
> print 'g, g._type:', g, g._type
> 
> 
> --------------------------------------------------------------------
> PythonWin 2.3 (#46, Jul 29 2003, 18:54:32) [MSC v.1200 32 bit (Intel)] 
> on win32.
> Portions Copyright 1994-2001 Mark Hammond (mhammond at skippinet.com.au) - 
> see 'Help/About PythonWin' for further copyright information.
>  >>> b, b._type: [0 1 2 3] Int8
> c, c._type: [  0 -23 -46 -69] Int8
> e, e._type: [  1  -2 -72   4] Int8
> f, f._type: [  9.6  19.2  28.8  38.4] Float64
> g, g._type: [ 19200 -27136  -7937  11264] Int16
> 
> Colin W.
> 
> 
> 
> -------------------------------------------------------
> This SF.Net email sponsored by: Free pre-built ASP.NET sites including
> Data Reports, E-commerce, Portals, and Forums are available now.
> Download today and enter to win an XBOX or Visual Studio .NET.
> http://aspnet.click-url.com/go/psa00100003ave/direct;at.aspnet_072303_01/01
> _______________________________________________
> Numpy-discussion mailing list
> Numpy-discussion at lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/numpy-discussion
-- 
Todd Miller 			jmiller at stsci.edu
STSCI / ESS / SSB




From fperez at colorado.edu  Tue Sep  2 12:21:07 2003
From: fperez at colorado.edu (Fernando Perez)
Date: Tue Sep  2 12:21:07 2003
Subject: [Numpy-discussion] sum and mean methods behaviour
In-Reply-To: <1062527563.16365.283.camel@halloween.stsci.edu>
References: <200309011134.39281.verveer@embl-heidelberg.de> <1062527563.16365.283.camel@halloween.stsci.edu>
Message-ID: <3F54ED71.4070301@colorado.edu>

Todd Miller wrote:

> I thought about this a lot yesterday and today talked it over with
> Perry.   There are several ways to fix the problem with mean() and
> sum(), and I'm hoping that you and the rest of the community will help
> sort them out.

[snip]

Just a thought: why not make the upcasting an optional parameter?

I've found that python's arguments with default values provide a very 
convenient way of giving the user fine control with minimal conceptual overhead.

I'd rather write:

arr = array([255, 255], Int8)
... later
arr.sum(use_double=1)  # or some similar way of tuning sum()


than

arr = array([255, 255], Int8)
... later
array(arr,typecode='d').sum()

Numarray/numpy are trying to tackle an inherently hard problem: matching the 
high-level comfort of python with the low-level performance of C.  This 
situation is an excellent example of what I've seen described as the 'law of 
leaky abstractions':  in most cases where you encapsulate low level details in 
a high level abstraction, there end up being situations where the details poke 
through the abstraction and cause you grief.  This is an inherently tricky 
problem, with no easy, universal solution (that I'm aware of).

Cheers,

f.




From verveer at embl.de  Tue Sep  2 13:33:08 2003
From: verveer at embl.de (verveer at embl.de)
Date: Tue Sep  2 13:33:08 2003
Subject: [Numpy-discussion] sum and mean methods behaviour
In-Reply-To: <1062527563.16365.283.camel@halloween.stsci.edu>
References: <200309011134.39281.verveer@embl-heidelberg.de> <1062527563.16365.283.camel@halloween.stsci.edu>
Message-ID: <1062534753.3f54fe610ed4d@webmail.embl.de>

Hi Todd, 
 
> I thought about this a lot yesterday and today talked it over with 
> Perry.   There are several ways to fix the problem with mean() and 
> sum(), and I'm hoping that you and the rest of the community will help 
> sort them out. 
 
It was just an innocent question, I did not think it would have such 
ramifications :-)  
 
Here are my thoughts: 
 
If I understand you well, the sum() and mean() array methods are based on the 
reduce method of the universal functions. And these do their calculations in 
the precision of the array, is that correct? 
 
I also gave this some thought, and I would like to make a distinction between 
a reduction and the calculation of a statistical value such as the mean or the 
sum: 
 
To me, a reduction means the projection of an multi-dimensional array to an 
array with a rank that is one less than the input. The result is still an 
array, and often I want the result to have the same precision as the input.  
 
A statistical calculation like a sum or a mean is different: the result should 
be correct and the same irrespective of the type of the input, and that 
mandates using sufficient precision in the calculation. Note however, that 
such a statistic is a scalar result and does not require temporary storage at 
high precision for the whole array. 
 
So keeping this in mind, my comments to your solutions are: 
 
> (1) The first "solution" is to require users to do their own up-casting 
> prior to calling mean() or sum().  This gives the end user fine control 
> over storage cost but leaves the C-like pitfall/bug you discovered.   I 
> mention this because this is how the numarray/Numeric reductions are 
> designed.  Is there a reason why the numarray/Numeric reductions don't 
> implicitly up-cast? 
 
For reductions this behaviour suits me, precisely because it allows control 
over storage, which is one of the strengths of numarray. 
 
For calculating the mean or the sum of an array this is however an expensive 
solution for a very common operation. I do use this solution, but sometimes I 
prefer an optimized C routine instead. 
 
> (2) The second way is what you proposed:  use double precision within 
> mean and sum.  This has great simplicity but gives no control over 
> storage usage, and as implemented, the storage would be much higher than 
> one might think, potentially 8x. 
 
I did not want to suggest to store a casted version of the array before 
calculation of the mean or the sum. That can be done in double precision 
without converting the whole array in memory. I think we can all agree that 
this option would not be a good idea. 
 
> (3) Lastly, Perry suggested a more radical approach:  rather than 
> changing the mean and sum methods themselves,  we could alter the 
> universal function accumulate and reduce methods to implicitly use 
> additional precision.  Perry's idea was to make all accumulations and 
> reductions up-cast their results to the largest type of the current 
> family, either Bool, Int64, Float64, or Complex64.   By doing this, we 
> can improve the utility of the reductions and accumulations as well as 
> fixing the problem with sum and mean. 
 
I think that is a great idea in principle, but I think you should consider 
this carefully: 
 
First of all control of the storage cost is lost when you do a reduction. I 
would not find that always desirable. Thus, I would then like the old 
behaviour for reduction to be accesible either as a different method or by a 
setting an optional argument. 
 
Additionally, it would not work well for some operations. For instance precise 
calculation of the mean requires floating point precision. Maybe this can be 
solved, but would it require different casting behaviour for different 
operations. That might be too much trouble... 
 
I would like to propose a fourth option: 
 
(4) provide separate implementations for array methods like sum() and mean() 
that only calculate the scalar result. No additional storage would be 
necessary and the calculation can be done in double precision. 
 
I guess that the disadvantage is that one cannot leverage the existing code in 
the ufuncs so easily.  I also assume that it would not be such a general 
solution as changing the reduce method is. 
 
I do have some experience in writing these sorts of code for multidimensional 
arrays in C and would be happy to contribute code. However, I am not too 
familiar with the internals of numarray library and I don't know how well my 
code fits in there (although I interface all my code to numarray). But I am 
happy to help if I can, numarray is great stuff, it has become the main tool 
for my numerical work. 
 
Peter 
 
-- 
Dr. Peter J. Verveer 
Cell Biology and Cell Biophysics Programme 
EMBL 
Meyerhofstrasse 1 
D-69117 Heidelberg 
Germany 
Tel. : +49 6221 387245 
Fax  : +49 6221 387242 
 
 



From kern at caltech.edu  Tue Sep  2 14:32:03 2003
From: kern at caltech.edu (Robert Kern)
Date: Tue Sep  2 14:32:03 2003
Subject: [Numpy-discussion] sum and mean methods behaviour
In-Reply-To: <3F54ED71.4070301@colorado.edu>
References: <200309011134.39281.verveer@embl-heidelberg.de> <1062527563.16365.283.camel@halloween.stsci.edu> <3F54ED71.4070301@colorado.edu>
Message-ID: <20030902213010.GA27423@taliesen.caltech.edu>

On Tue, Sep 02, 2003 at 01:20:17PM -0600, Fernando Perez wrote:
> Todd Miller wrote:
> 
> >I thought about this a lot yesterday and today talked it over with
> >Perry.   There are several ways to fix the problem with mean() and
> >sum(), and I'm hoping that you and the rest of the community will help
> >sort them out.
> 
> [snip]
> 
> Just a thought: why not make the upcasting an optional parameter?
> 
> I've found that python's arguments with default values provide a very 
> convenient way of giving the user fine control with minimal conceptual 
> overhead.
> 
> I'd rather write:
> 
> arr = array([255, 255], Int8)
> ... later
> arr.sum(use_double=1)  # or some similar way of tuning sum()

+1, but

arr.sum(typecode=Float64)

would be my choice of spelling. Not sure what the default typecode
should be, though. Probably Perry's suggestion: the largest type of the
family.

-- 
Robert Kern
kern at caltech.edu

"In the fields of hell where the grass grows high
 Are the graves of dreams allowed to die."
  -- Richard Harter



From jmiller at stsci.edu  Wed Sep  3 05:44:11 2003
From: jmiller at stsci.edu (Todd Miller)
Date: Wed Sep  3 05:44:11 2003
Subject: [Numpy-discussion] sum and mean methods behaviour
In-Reply-To: <3F54ED71.4070301@colorado.edu>
References: <200309011134.39281.verveer@embl-heidelberg.de>
	 <1062527563.16365.283.camel@halloween.stsci.edu>
	 <3F54ED71.4070301@colorado.edu>
Message-ID: <1062592991.3460.89.camel@localhost.localdomain>

On Tue, 2003-09-02 at 15:20, Fernando Perez wrote:
> Todd Miller wrote:
> 
> > I thought about this a lot yesterday and today talked it over with
> > Perry.   There are several ways to fix the problem with mean() and
> > sum(), and I'm hoping that you and the rest of the community will help
> > sort them out.
> 
> [snip]
> 
> Just a thought: why not make the upcasting an optional parameter?
> 
<snip>
That sounds like a great idea.  Simple, but doesn't throw out all
storage control.

> in most cases where you encapsulate low level details in 
> a high level abstraction, there end up being situations where the details poke 
> through the abstraction and cause you grief.  

Thanks for these kind words.  

-- 
Todd Miller <jmiller at stsci.edu>




From verveer at embl-heidelberg.de  Wed Sep  3 08:40:20 2003
From: verveer at embl-heidelberg.de (Peter Verveer)
Date: Wed Sep  3 08:40:20 2003
Subject: [Numpy-discussion] sum and mean methods behaviour
In-Reply-To: <3F560505.7090209@pfdubois.com>
References: <200309011134.39281.verveer@embl-heidelberg.de> <1062527563.16365.283.camel@halloween.stsci.edu> <3F560505.7090209@pfdubois.com>
Message-ID: <200309031739.57327.verveer@embl-heidelberg.de>

I also believe that the current behavior for numarray/Numeric reduce method 
(not to cast) is the right one. It is fine to leave the user with the 
responsibility to be careful in the case of the reduce operation.

But to correctly calculate a mean or a sum by the array methods that are 
provided you have to convert the array first to a more precise type, and then 
do the calculation. That wastes space and is slow, and seems not very elegant 
considering that these are very common statistical operations.

A separate implementation for the mean() and sum() methods that uses double 
precision in the calculation without first converting the array would be 
straightforward. Since calculating a mean or a sum of a complete array is 
such a common case I think this would be useful.

That leaves the same problem for the reduce method which in some cases would 
require first a conversion, but this is much less of a problem (at least for 
me). Having to convert before the operation can be wasteful though.

I do like the idea that was also proposed on the list to supply an optional 
argument to specify the output type. Then the user has full control of the 
output type (nice if you want high precision in the result without converting 
the input), and the code can easily be used to implement the mean() and sum() 
methods. The default behavior of the reduce method can then remain unchanged, 
so this would not be an obtrusive change. But, I imagine that this may 
complicate the implementation.

Cheers, Peter

On Wednesday 03 September 2003 17:13, Paul Dubois wrote:
> So after you get the result in a higher precision, then what?
> a. Cast it down blindly?
> b. Test every element and throw an exception if casting would lose
> precision?
> c. Test every element and return the smallest kind that "holds" the answer?
> d. Always return the highest precision?
>
> a. is close to equivalent to the present behavior
> b. and c. are expensive.
> c. makes the type of the result unpredictable, which has its own problems.
> d. uses space
>
> It was the originally design of Numeric to be fast rather than careful,
> user beware. There is now a another considerable portion of the
> community that is for very careful, and another that is for keeping it
> small. You can't satisfy all those goals at once.
>
> If you make it slow or big in order to be careful, it will always be
> slow or big, while the opposite is not true. If you make it fast, the
> user can be careful.
>
> Todd Miller wrote:
> > On Mon, 2003-09-01 at 05:34, Peter Verveer wrote:
> >>Hi All,
> >>
> >>I noticed that the sum() and mean() methods of numarrays use the
> >> precision of
> >>
> >>the given array in their calculations. That leads to resuls like this:
> >>>>>array([255, 255], Int8).sum()
> >>
> >>-2
> >>
> >>>>>array([255, 255], Int8).mean()
> >>
> >>-1.0
> >>
> >>Would it not be better to use double precision internally and return the
> >>correct result?
> >>
> >>Cheers, Peter
> >
> > Hi Peter,
> >
> > I thought about this a lot yesterday and today talked it over with
> > Perry.   There are several ways to fix the problem with mean() and
> > sum(), and I'm hoping that you and the rest of the community will help
> > sort them out.
> >
> > (1) The first "solution" is to require users to do their own up-casting
> > prior to calling mean() or sum().  This gives the end user fine control
> > over storage cost but leaves the C-like pitfall/bug you discovered.   I
> > mention this because this is how the numarray/Numeric reductions are
> > designed.  Is there a reason why the numarray/Numeric reductions don't
> > implicitly up-cast?
> >
> > (2) The second way is what you proposed:  use double precision within
> > mean and sum.  This has great simplicity but gives no control over
> > storage usage, and as implemented, the storage would be much higher than
> > one might think, potentially 8x.
> >
> > (3) Lastly, Perry suggested a more radical approach:  rather than
> > changing the mean and sum methods themselves,  we could alter the
> > universal function accumulate and reduce methods to implicitly use
> > additional precision.  Perry's idea was to make all accumulations and
> > reductions up-cast their results to the largest type of the current
> > family, either Bool, Int64, Float64, or Complex64.   By doing this, we
> > can improve the utility of the reductions and accumulations as well as
> > fixing the problem with sum and mean.




From jmiller at stsci.edu  Wed Sep  3 14:47:04 2003
From: jmiller at stsci.edu (Todd Miller)
Date: Wed Sep  3 14:47:04 2003
Subject: [Numpy-discussion] sum and mean methods behaviour
In-Reply-To: <3F560505.7090209@pfdubois.com>
References: <200309011134.39281.verveer@embl-heidelberg.de>
	 <1062527563.16365.283.camel@halloween.stsci.edu>
	 <3F560505.7090209@pfdubois.com>
Message-ID: <1062625547.3459.707.camel@localhost.localdomain>

I want to thank everyone who participated in this discussion:  Peter,
Fernando, Robert, Paul, Perry, and Tim.

Tim's post has IMO a completely synthesized solution:  

1. Add a type parameter to sum which defaults to widest type.

2. Add a type parameter to reductions (and fix output type handling). 
Default is same-type as it is now.  No major changes to C-code.

3. Add a WidestType(array) function:

	Bool                        --> Bool
	Int8,Int16,Int32,Int64      --> Int64
	UInt8, UInt16,UInt32,UInt64 --> UInt64 (Int64 on win32)
	Float32, Float64            --> Float64
	Complex32, Complex64        --> Complex64

The only thing this really leaves out, is a higher performance
implementation of sum/mean which Peter referred to a few times.  Peter, 
if you want to write a specialized module, we'd be happy to put it in
the add-ons package.
	
Thanks again everybody,
Todd

-- 
Todd Miller <jmiller at stsci.edu>




From verveer at embl-heidelberg.de  Thu Sep  4 01:55:03 2003
From: verveer at embl-heidelberg.de (Peter Verveer)
Date: Thu Sep  4 01:55:03 2003
Subject: [Numpy-discussion] Re: Numpy-discussion digest, Vol 1 #693 - 3 msgs
In-Reply-To: <E19ukg2-0006wj-00@sc8-sf-list2.sourceforge.net>
References: <E19ukg2-0006wj-00@sc8-sf-list2.sourceforge.net>
Message-ID: <200309041054.09224.verveer@embl-heidelberg.de>

> 1. Add a type parameter to sum which defaults to widest type.
>
> 2. Add a type parameter to reductions (and fix output type handling).
> Default is same-type as it is now.  No major changes to C-code.
>
> 3. Add a WidestType(array) function:
>
>         Bool                        --> Bool
>         Int8,Int16,Int32,Int64      --> Int64
>         UInt8, UInt16,UInt32,UInt64 --> UInt64 (Int64 on win32)
>         Float32, Float64            --> Float64
>         Complex32, Complex64        --> Complex64

This sounds like a good solution.

> The only thing this really leaves out, is a higher performance
> implementation of sum/mean which Peter referred to a few times. 

Is this really the case? It depends on how you plan to implement the output 
conversion. If you will do this by allocating a temporary converted copy of 
the complete input before the calculations then this potentially requires a 
lot of temporary storage. But it is certainly possible to come up with an 
implementation that avoids this. Have you given this some thought?

> Peter,
> if you want to write a specialized module, we'd be happy to put it in
> the add-ons package.

I hope that the reduce methods can be made sufficiently efficient so that this 
will not be necessary. 

Cheers, Peter




From alnesbit at students.cs.mu.OZ.AU  Thu Sep  4 02:59:05 2003
From: alnesbit at students.cs.mu.OZ.AU (Andrew Nesbit)
Date: Thu Sep  4 02:59:05 2003
Subject: [Numpy-discussion] Extracting column vectors from matrix.
Message-ID: <Pine.GSO.4.56.0309041946040.904@munyoo.cs.mu.OZ.AU>

Say that I have a rank-2 array (matrix) of shape (m,n).  Call it A.

When extracting column vectors from A, I often want to keep the result
as a rank-2 column vector (for subsequent matrix multiplications,
etc), so I usually end up writing something ugly like this:

column_vector = reshape(A[:,col],(m,1))

I've got a function to wrap this, but is there a builtin (or cleaner)
way to do this sort of operation?

TIA.

Andrew.




From verveer at embl-heidelberg.de  Thu Sep  4 04:26:04 2003
From: verveer at embl-heidelberg.de (Peter Verveer)
Date: Thu Sep  4 04:26:04 2003
Subject: [Numpy-discussion] Re: sum and mean methods behaviour
Message-ID: <200309041325.13324.verveer@embl-heidelberg.de>

Hi all,

I was thinking a bit more about the changes to reduce() that Todd proposed, 
and have some questions:

The problem that the output may not be able to hold the result of an operation 
is not unique to the reduce() method. For instance adding two arrays of type 
UInt can also give you the wrong answer: 

>>> array(255, UInt8) + array(255, UInt8)
254

So, if this is a general problem, why should only the reduce method be 
enhanced to avoid this? If you implement this, should this capability not be 
supported more broadly than only by reduce(), for instance by universal 
functons such as 'add'? Would it not be unexpected for users that only 
reduce() provides such added functionality?

However, as Paul Dubois pointed out earlier, the original design philosphy of 
Numeric/numarray was to let the user deal with such problems himself and keep 
the package small and fast. This seems actually a sound decision, so would it 
not be better to avoid complicating numarray with these type of changes and 
also leave reduce as it is?

Personally I don't have a need for the proposed changes to the reduce 
function. My original complaint that started the whole discussion was that 
the mean() and sum() array methods did not give the correct result in some 
cases. I still think they should return a correct double precision value, 
even if the universal functions may not. That could be achieved by a separate  
implementation that does not use the universal functions. I would be prepared 
to provide that implementation either to replace the mean and sum methods, or 
as a separate add-on.

Cheers, Peter

> 1. Add a type parameter to sum which defaults to widest type.
>
> 2. Add a type parameter to reductions (and fix output type handling).
> Default is same-type as it is now.  No major changes to C-code.
>
> 3. Add a WidestType(array) function:
>
>         Bool                        --> Bool
>         Int8,Int16,Int32,Int64      --> Int64
>         UInt8, UInt16,UInt32,UInt64 --> UInt64 (Int64 on win32)
>         Float32, Float64            --> Float64
>         Complex32, Complex64        --> Complex64
>
> The only thing this really leaves out, is a higher performance
> implementation of sum/mean which Peter referred to a few times. 
> Peter, if you want to write a specialized module, we'd be happy 
> to put it in the add-ons package.
 



From hinsen at cnrs-orleans.fr  Thu Sep  4 05:37:05 2003
From: hinsen at cnrs-orleans.fr (Konrad Hinsen)
Date: Thu Sep  4 05:37:05 2003
Subject: [Numpy-discussion] Extracting column vectors from matrix.
In-Reply-To: <Pine.GSO.4.56.0309041946040.904@munyoo.cs.mu.OZ.AU>
References: <Pine.GSO.4.56.0309041946040.904@munyoo.cs.mu.OZ.AU>
Message-ID: <200309041433.47180.hinsen@cnrs-orleans.fr>

On Thursday 04 September 2003 11:58, Andrew Nesbit wrote:
> Say that I have a rank-2 array (matrix) of shape (m,n).  Call it A.
>
> When extracting column vectors from A, I often want to keep the result
> as a rank-2 column vector (for subsequent matrix multiplications,
> etc), so I usually end up writing something ugly like this:
>
> column_vector = reshape(A[:,col],(m,1))
>
> I've got a function to wrap this, but is there a builtin (or cleaner)
> way to do this sort of operation?

1) A[:, col:col+1]

or perhaps

2) A[:, col][:, NewAxis]

Konrad.
-- 
-------------------------------------------------------------------------------
Konrad Hinsen                            | E-Mail: hinsen at cnrs-orleans.fr
Centre de Biophysique Moleculaire (CNRS) | Tel.: +33-2.38.25.56.24
Rue Charles Sadron                       | Fax:  +33-2.38.63.15.17
45071 Orleans Cedex 2                    | Deutsch/Esperanto/English/
France                                   | Nederlands/Francais
-------------------------------------------------------------------------------




From perry at stsci.edu  Thu Sep  4 06:12:04 2003
From: perry at stsci.edu (Perry Greenfield)
Date: Thu Sep  4 06:12:04 2003
Subject: [Numpy-discussion] Re: sum and mean methods behaviour
In-Reply-To: <200309041054.09224.verveer@embl-heidelberg.de>
Message-ID: <CEELJPECNGEGKFNDLHFFEEJJCAAA.perry@stsci.edu>

Peter Verveer writes:
> Todd Miller writes:
> > 1. Add a type parameter to sum which defaults to widest type.
> >
> > 2. Add a type parameter to reductions (and fix output type handling).
> > Default is same-type as it is now.  No major changes to C-code.
> >
> > 3. Add a WidestType(array) function:
> >
> >         Bool                        --> Bool
> >         Int8,Int16,Int32,Int64      --> Int64
> >         UInt8, UInt16,UInt32,UInt64 --> UInt64 (Int64 on win32)
> >         Float32, Float64            --> Float64
> >         Complex32, Complex64        --> Complex64
>
> This sounds like a good solution.
>
> > The only thing this really leaves out, is a higher performance
> > implementation of sum/mean which Peter referred to a few times.
>
> Is this really the case? It depends on how you plan to implement
> the output
> conversion. If you will do this by allocating a temporary
> converted copy of
> the complete input before the calculations then this potentially
> requires a
> lot of temporary storage. But it is certainly possible to come up with an
> implementation that avoids this. Have you given this some thought?
>
The way numarray ufuncs currently work is that for larger arrays,
the conversion of types is only done in blocks. No temporary copy is
made of the whole input array. So the temporary storage issue is not
a problem.




From alnesbit at students.cs.mu.OZ.AU  Thu Sep  4 06:34:19 2003
From: alnesbit at students.cs.mu.OZ.AU (Andrew Nesbit)
Date: Thu Sep  4 06:34:19 2003
Subject: [Numpy-discussion] Extracting column vectors from matrix.
In-Reply-To: <200309041433.47180.hinsen@cnrs-orleans.fr> (Konrad Hinsen's
 message of "Thu, 4 Sep 2003 14:33:47 +0200")
References: <Pine.GSO.4.56.0309041946040.904@munyoo.cs.mu.OZ.AU>
	<200309041433.47180.hinsen@cnrs-orleans.fr>
Message-ID: <m2r82wptjk.fsf@naima.lensflare.org>

Konrad Hinsen <hinsen at cnrs-orleans.fr> writes:

> 1) A[:, col:col+1]
>
> or perhaps
>
> 2) A[:, col][:, NewAxis]

Yep, those are the sort of tricks I was looking for.  Thankyou.

Andrew.




From perry at stsci.edu  Thu Sep  4 06:35:03 2003
From: perry at stsci.edu (Perry Greenfield)
Date: Thu Sep  4 06:35:03 2003
Subject: [Numpy-discussion] Re: sum and mean methods behaviour
In-Reply-To: <200309041325.13324.verveer@embl-heidelberg.de>
Message-ID: <CEELJPECNGEGKFNDLHFFMEJJCAAA.perry@stsci.edu>

Peter Verveer writes

> Hi all,
>
> I was thinking a bit more about the changes to reduce() that Todd
> proposed,
> and have some questions:
>
> The problem that the output may not be able to hold the result of
> an operation
> is not unique to the reduce() method. For instance adding two
> arrays of type
> UInt can also give you the wrong answer:
>
> >>> array(255, UInt8) + array(255, UInt8)
> 254
>
> So, if this is a general problem, why should only the reduce method be
> enhanced to avoid this? If you implement this, should this
> capability not be
> supported more broadly than only by reduce(), for instance by universal
> functons such as 'add'? Would it not be unexpected for users that only
> reduce() provides such added functionality?
>
Certainly true (and much more likely a problem for integer multiplication
than addition). On the other hand, it is more likely to be only an
occasional problem for binary operations. With reductions, the risk is
severe that overflows will happen. For example, for addition it is
the difference between a+a for the normal operation and len(a)*a for
the reduction. Arguably reductions on Int8 and Int16 arrays are likely
to run into a problem than not.

> However, as Paul Dubois pointed out earlier, the original design
> philosphy of
> Numeric/numarray was to let the user deal with such problems
> himself and keep
> the package small and fast. This seems actually a sound decision,
> so would it
> not be better to avoid complicating numarray with these type of
> changes and
> also leave reduce as it is?
>
No, I'm inclined to change reductions because of the high potential
for problems, particularly with ints. I don't think ufunc type handling
needs to change though. Todd believes that changing reduction behavior
would not be difficult (though we will try to finish other work first before
doing that). Changing reduction behavior is probably the easiest way
of implementing the improved sum and mean functions. The only thing we
need to determine is what the default behavior is (Todd proposes
the defaults remain the same, I'm not so sure.)

> Personally I don't have a need for the proposed changes to the reduce
> function. My original complaint that started the whole discussion
> was that
> the mean() and sum() array methods did not give the correct
> result in some
> cases. I still think they should return a correct double precision value,
> even if the universal functions may not. That could be achieved
> by a separate
> implementation that does not use the universal functions. I would
> be prepared
> to provide that implementation either to replace the mean and sum
> methods, or
> as a separate add-on.
>
>




From verveer at embl-heidelberg.de  Thu Sep  4 06:53:05 2003
From: verveer at embl-heidelberg.de (Peter Verveer)
Date: Thu Sep  4 06:53:05 2003
Subject: [Numpy-discussion] Re: sum and mean methods behaviour
In-Reply-To: <CEELJPECNGEGKFNDLHFFMEJJCAAA.perry@stsci.edu>
References: <CEELJPECNGEGKFNDLHFFMEJJCAAA.perry@stsci.edu>
Message-ID: <200309041552.34197.verveer@embl-heidelberg.de>

On Thursday 04 September 2003 15:33, Perry Greenfield wrote:
> > So, if this is a general problem, why should only the reduce method be
> > enhanced to avoid this? If you implement this, should this
> > capability not be
> > supported more broadly than only by reduce(), for instance by universal
> > functons such as 'add'? Would it not be unexpected for users that only
> > reduce() provides such added functionality?
>
> Certainly true (and much more likely a problem for integer multiplication
> than addition). On the other hand, it is more likely to be only an
> occasional problem for binary operations. With reductions, the risk is
> severe that overflows will happen. For example, for addition it is
> the difference between a+a for the normal operation and len(a)*a for
> the reduction. Arguably reductions on Int8 and Int16 arrays are likely
> to run into a problem than not.

That true, but this argument really only holds for the integer types. For 
32-bit floating point or complex types it will usually not be necessary to 
convert to 64-bit to prevent overflow. In that case it may often not be 
desirable to change the array type. I am not saying that the convert option 
would not useful for the case of floats, but it is maybe an argument to keep 
the default behaviour, at least for Float32 and Complex32 types.

Generally I do agree that there is no need to change the ufuncs, I did not 
want to suggest that this actually be implemented...

> > However, as Paul Dubois pointed out earlier, the original design
> > philosphy of
> > Numeric/numarray was to let the user deal with such problems
> > himself and keep
> > the package small and fast. This seems actually a sound decision,
> > so would it
> > not be better to avoid complicating numarray with these type of
> > changes and
> > also leave reduce as it is?
>
> No, I'm inclined to change reductions because of the high potential
> for problems, particularly with ints. I don't think ufunc type handling
> needs to change though. Todd believes that changing reduction behavior
> would not be difficult (though we will try to finish other work first
> before doing that). Changing reduction behavior is probably the easiest way
> of implementing the improved sum and mean functions. The only thing we need
> to determine is what the default behavior is (Todd proposes
> the defaults remain the same, I'm not so sure.)

This would solve my problem with mean() and sum(). I think these should 
certainly return the result in the optimal precision. These may then not be 
the most optimal in terms of speed, but certainly 'good enough'. I would  
like to second Todds preference to keep the default behaviour of reductions 
to be the same as it is now. For reductions, I mostly want the result to be 
in the same type, because I chose that type in the first place for storage 
reasons.

Cheers, Peter



From alnesbit at students.cs.mu.oz.au  Thu Sep  4 07:46:55 2003
From: alnesbit at students.cs.mu.oz.au (Andrew Nesbit)
Date: Thu Sep  4 07:46:55 2003
Subject: [Numpy-discussion] Matrix square root
Message-ID: <m2ekywpqd8.fsf@naima.lensflare.org>

I need a function equivalent to Matlab's sqrtm, i.e., a square root
for matrices.

I constructed the following definition:

def sqrtm(M):
    (U,S,VT) = LinearAlgebra.singular_value_decomposition(M)
    D = MLab.diag(sqrt(S))
    return matrixmultiply(matrixmultiply(U,D),VT)

but this technique only works for Hermitian, positive definite
matrices.  The matrices I operate on don't necessarily satisfy that
condition.

Does anybody know of a more general technique, preferably with an
accompanying Python implementation?

Thanks.

Andrew.




From hinsen at cnrs-orleans.fr  Thu Sep  4 08:48:44 2003
From: hinsen at cnrs-orleans.fr (Konrad Hinsen)
Date: Thu Sep  4 08:48:44 2003
Subject: [Numpy-discussion] Matrix square root
In-Reply-To: <m2ekywpqd8.fsf@naima.lensflare.org>
References: <m2ekywpqd8.fsf@naima.lensflare.org>
Message-ID: <200309041707.34519.hinsen@cnrs-orleans.fr>

On Thursday 04 September 2003 16:42, Andrew Nesbit wrote:
> I need a function equivalent to Matlab's sqrtm, i.e., a square root
> for matrices.
>
> I constructed the following definition:
>
> def sqrtm(M):
>     (U,S,VT) = LinearAlgebra.singular_value_decomposition(M)
>     D = MLab.diag(sqrt(S))
>     return matrixmultiply(matrixmultiply(U,D),VT)
>
> but this technique only works for Hermitian, positive definite
> matrices.  The matrices I operate on don't necessarily satisfy that
> condition.

I'd use an eigenvalue decomposition, then take the square root of the 
eigenvalues, and then apply the diagonlization matrix in reverse. If you 
convert to eigenvalues to complex before taking the square root, this will 
work for non-positive-definite matrices, yielding a complex result.

Konrad.
-- 
-------------------------------------------------------------------------------
Konrad Hinsen                            | E-Mail: hinsen at cnrs-orleans.fr
Centre de Biophysique Moleculaire (CNRS) | Tel.: +33-2.38.25.56.24
Rue Charles Sadron                       | Fax:  +33-2.38.63.15.17
45071 Orleans Cedex 2                    | Deutsch/Esperanto/English/
France                                   | Nederlands/Francais
-------------------------------------------------------------------------------




From alnesbit at students.cs.mu.OZ.AU  Thu Sep  4 09:05:18 2003
From: alnesbit at students.cs.mu.OZ.AU (Andrew Nesbit)
Date: Thu Sep  4 09:05:18 2003
Subject: [Numpy-discussion] Matrix square root
In-Reply-To: <200309041707.34519.hinsen@cnrs-orleans.fr> (Konrad Hinsen's
 message of "Thu, 4 Sep 2003 17:07:34 +0200")
References: <m2ekywpqd8.fsf@naima.lensflare.org>
	<200309041707.34519.hinsen@cnrs-orleans.fr>
Message-ID: <m265k8o8q0.fsf@naima.lensflare.org>

Konrad Hinsen <hinsen at cnrs-orleans.fr> writes:

> On Thursday 04 September 2003 16:42, Andrew Nesbit wrote:
>> I need a function equivalent to Matlab's sqrtm, i.e., a square root
>> for matrices.

[snip]

> I'd use an eigenvalue decomposition, then take the square root of the 
> eigenvalues, and then apply the diagonlization matrix in reverse. If you 
> convert to eigenvalues to complex before taking the square root, this will 
> work for non-positive-definite matrices, yielding a complex result.

Thankyou for the advice.  This gives me a good starting point.

Andrew.




From verveer at embl.de  Fri Sep  5 11:47:02 2003
From: verveer at embl.de (verveer at embl.de)
Date: Fri Sep  5 11:47:02 2003
Subject: [Numpy-discussion] bug?
Message-ID: <1062787560.3f58d9e903356@webmail.embl.de>

possible bug? 
 
>>> a = array([[[1, 1], [0, 1]]]) 
>>> print logical_or.reduce(a, 0) 
[[1 0] 
 [1 1]] 
 
should be: 
 
>>> print a[0, ...] 
[[1 1] 
 [0 1]] 
 
Cheers, Peter 
-- 
Dr. Peter J. Verveer 
Cell Biology and Cell Biophysics Programme 
EMBL 
Meyerhofstrasse 1 
D-69117 Heidelberg 
Germany 
Tel. : +49 6221 387245 
Fax  : +49 6221 387242 
 



From cjw at sympatico.ca  Fri Sep  5 12:04:02 2003
From: cjw at sympatico.ca (Colin J. Williams)
Date: Fri Sep  5 12:04:02 2003
Subject: [Numpy-discussion] Attempting to sub-class NumArray
Message-ID: <3F58DDE5.5090608@sympatico.ca>

When ravel() is used on an instance of NumArray, the _strides attribute 
is recalculated correctly.
When ravel() is used on an instance of special, a sub-class of NumArray, 
the _strides attribute keeps its old value.  This appears to cause the 
display problem which is reported.

The essentials of the code used are below.

To investigate this problem, two print statements were added to the 
method setshape in generic.py:

            if newnelements == nelements:
                self._shape = tuple(shape)
                print 'In setShape self._shape:', self._shape
                self._strides = self._stridesFromShape()
                print 'In setShape self._strides:', self._strides
            else:
                raise ValueError("New shape is not consistent with the
    old shape")

It seems that only the first of these print statements is executed with 
special, but both are executed with NumArray.  In the second case, the 
output is not in the expected sequence.

I would appreciate any suggestion as to a workaround.

Colin W,

Grief when attempting to sub-class NumArray

Code used:

# Try to sub-class
class special(N.NumArray):
  def __init__(self, data= None, shape= None, eType= None):
##                         eType= _nt.Any or AnyType not acceptable %%
    arr= N.array(sequence= data, shape= shape, type= eType)
    for attr in dir(arr):
##      This is a longwinded way of setting up special
##      There must be a better way                     %%
##      Perhaps we should use generic._view
##      It's not documented                 TRY IT LATER
##      There is no need to transfer methods!
      if attr[0] == '_' and attr[1] != '_':
        print attr
        exec 'self.' + attr + '= ' 'arr.' + attr
        exec 'print self.' + attr
    pass

a= special(data= [1, 2, 3, 4], shape= (2, 2))
a= N.array([1, 2, 3, 4], shape= (2, 2))
print 'In ts a._strides:', a._strides
b= _gen.ravel(a)
print 'In ts b._strides:', b._strides       #  <<< Unchanged with 
special, OK with NumArray
print 'b:', b



USING THE SUBCLASS special
 >C:\Progra~1\Python23\pythonw -u ts.py
In ts a._strides: (8, 4)
In setShape self._shape: (4,)
In ts b._strides: (8, 4)
b: In setShape self._shape: (4,)
Traceback (most recent call last):
  File "ts.py", line 37, in ?
    print 'b:', b
  File 
"C:\PROGRA~1\Python23\lib\site-packages\numarray\numarraycore.py", line 
618, in __str__
    MAX_LINE_WIDTH, PRECISION, SUPPRESS_SMALL, ' ', "")
  File "C:\PROGRA~1\Python23\lib\site-packages\numarray\arrayprint.py", 
line 176, in array2string
    separator, prefix)
  File "C:\PROGRA~1\Python23\lib\site-packages\numarray\arrayprint.py", 
line 138, in _array2string
    max_str_len = max(len(str(max_reduce(data))),
libnumarray.error: maximum_Int32_reduce: access beyond buffer. offset=27 
buffersize=16
 >Exit code: 1
__________________________________________________________

# USING NumArray

In setShape self._shape: (2, 2)
In ts a._strides: (8, 4)
In setShape self._shape: (4,)
In ts b._strides: (4,)
b: In setShape self._shape: (4,)
[1 2 3 4]
 >Exit code: 0





From jmiller at stsci.edu  Fri Sep  5 15:04:06 2003
From: jmiller at stsci.edu (Todd Miller)
Date: Fri Sep  5 15:04:06 2003
Subject: [Numpy-discussion] Attempting to sub-class NumArray
In-Reply-To: <3F58DDE5.5090608@sympatico.ca>
References: <3F58DDE5.5090608@sympatico.ca>
Message-ID: <1062799403.5615.25.camel@halloween.stsci.edu>

Hi Colin,

I haven't looked at this in detail,  but special.__init__() is making me
queasy.  Why doesn't it call NumArray.__init__()?

Todd

On Fri, 2003-09-05 at 15:03, Colin J. Williams wrote:
> When ravel() is used on an instance of NumArray, the _strides attribute 
> is recalculated correctly.
> When ravel() is used on an instance of special, a sub-class of NumArray, 
> the _strides attribute keeps its old value.  This appears to cause the 
> display problem which is reported.
> 
> The essentials of the code used are below.
> 
> To investigate this problem, two print statements were added to the 
> method setshape in generic.py:
> 
>             if newnelements == nelements:
>                 self._shape = tuple(shape)
>                 print 'In setShape self._shape:', self._shape
>                 self._strides = self._stridesFromShape()
>                 print 'In setShape self._strides:', self._strides
>             else:
>                 raise ValueError("New shape is not consistent with the
>     old shape")
> 
> It seems that only the first of these print statements is executed with 
> special, but both are executed with NumArray.  In the second case, the 
> output is not in the expected sequence.
> 
> I would appreciate any suggestion as to a workaround.
> 
> Colin W,
> 
> Grief when attempting to sub-class NumArray
> 
> Code used:
> 
> # Try to sub-class
> class special(N.NumArray):
>   def __init__(self, data= None, shape= None, eType= None):
> ##                         eType= _nt.Any or AnyType not acceptable %%
>     arr= N.array(sequence= data, shape= shape, type= eType)
>     for attr in dir(arr):
> ##      This is a longwinded way of setting up special
> ##      There must be a better way                     %%
> ##      Perhaps we should use generic._view
> ##      It's not documented                 TRY IT LATER
> ##      There is no need to transfer methods!
>       if attr[0] == '_' and attr[1] != '_':
>         print attr
>         exec 'self.' + attr + '= ' 'arr.' + attr
>         exec 'print self.' + attr
>     pass
> 
> a= special(data= [1, 2, 3, 4], shape= (2, 2))
> a= N.array([1, 2, 3, 4], shape= (2, 2))
> print 'In ts a._strides:', a._strides
> b= _gen.ravel(a)
> print 'In ts b._strides:', b._strides       #  <<< Unchanged with 
> special, OK with NumArray
> print 'b:', b
> 
> 
> 
> USING THE SUBCLASS special
>  >C:\Progra~1\Python23\pythonw -u ts.py
> In ts a._strides: (8, 4)
> In setShape self._shape: (4,)
> In ts b._strides: (8, 4)
> b: In setShape self._shape: (4,)
> Traceback (most recent call last):
>   File "ts.py", line 37, in ?
>     print 'b:', b
>   File 
> "C:\PROGRA~1\Python23\lib\site-packages\numarray\numarraycore.py", line 
> 618, in __str__
>     MAX_LINE_WIDTH, PRECISION, SUPPRESS_SMALL, ' ', "")
>   File "C:\PROGRA~1\Python23\lib\site-packages\numarray\arrayprint.py", 
> line 176, in array2string
>     separator, prefix)
>   File "C:\PROGRA~1\Python23\lib\site-packages\numarray\arrayprint.py", 
> line 138, in _array2string
>     max_str_len = max(len(str(max_reduce(data))),
> libnumarray.error: maximum_Int32_reduce: access beyond buffer. offset=27 
> buffersize=16
>  >Exit code: 1
> __________________________________________________________
> 
> # USING NumArray
> 
> In setShape self._shape: (2, 2)
> In ts a._strides: (8, 4)
> In setShape self._shape: (4,)
> In ts b._strides: (4,)
> b: In setShape self._shape: (4,)
> [1 2 3 4]
>  >Exit code: 0
> 
> 
> 
> 
> -------------------------------------------------------
> This sf.net email is sponsored by:ThinkGeek
> Welcome to geek heaven.
> http://thinkgeek.com/sf
> _______________________________________________
> Numpy-discussion mailing list
> Numpy-discussion at lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/numpy-discussion
-- 
Todd Miller 			jmiller at stsci.edu
STSCI / ESS / SSB




From frankpit at erols.com  Sat Sep  6 07:28:02 2003
From: frankpit at erols.com (Bernard Frankpitt)
Date: Sat Sep  6 07:28:02 2003
Subject: [Numpy-discussion] Matrix square root
References: <m2ekywpqd8.fsf@naima.lensflare.org>	<200309041707.34519.hinsen@cnrs-orleans.fr> <m265k8o8q0.fsf@naima.lensflare.org>
Message-ID: <3F59F1D8.5020900@erols.com>

Andrew Nesbit wrote:

>Konrad Hinsen <hinsen at cnrs-orleans.fr> writes:
>
>>On Thursday 04 September 2003 16:42, Andrew Nesbit wrote:
>>    
>>
>>>I need a function equivalent to Matlab's sqrtm, i.e., a square root
>>>for matrices
>>>
>[snip]
>
>>I'd use an eigenvalue decomposition, then take the square root of the 
>>eigenvalues, and then apply the diagonlization matrix in reverse. If you 
>>convert to eigenvalues to complex before taking the square root, this will 
>>work for non-positive-definite matrices, yielding a complex result
>>
>
>Thankyou for the advice.  This gives me a good starting point.
>
>Andrew.
>
>  
>
The only problem with this approach is that you have to find all the 
eigenvalues and eigenvectors to get the spectral decomposition.  For 
large, poorly conditioned matrices this can be a source of significant 
errors, and a lot of work. An alternative, and often faster and more 
accurate approach is to use a series expansion for the square root. 
Evaluating the series expansion only requires multiplication.

Bernie






From cjw at sympatico.ca  Sat Sep  6 08:26:04 2003
From: cjw at sympatico.ca (Colin J. Williams)
Date: Sat Sep  6 08:26:04 2003
Subject: [Numpy-discussion] Attempting to sub-class NumArray
In-Reply-To: <1062848229.3459.21.camel@localhost.localdomain>
References: <3F58DDE5.5090608@sympatico.ca>	 <1062799403.5615.25.camel@halloween.stsci.edu>	 <3F592B3A.5060505@sympatico.ca> <1062848229.3459.21.camel@localhost.localdomain>
Message-ID: <3F59FC78.3060307@sympatico.ca>

Todd Miller wrote:

>I think the problem you are having is caused by the assumption that
>instance initialization is equivalent to an unordered assignment of all
>instance attributes.  That's not true because the private attributes are
>interrelated;  assigning "shape" affects "_shape" and "_strides",
>_itemsize must be known before _strides can be computed from shape, and
>so on. 
>
It would be helpful if you could let us know what assignment ordering 
would be appropriate, or, if there were information about the 
interactions, perhaps we could work these things out for ourselves.

As we've discussed before, there is little information available on 
private objects and their roles.  Even to have the source code of some 
of the C code might be useful.  I wasn't able to spot the relevant C 
code among the CVS directories.

However, the assumption would appear to apply to the crude 
initialization of 'special'.  The problem which is being reported seems 
to be connected with the function setshape in generic.py.  This is far 
from the unordered assignment, although I recognize that ther might be 
some connection.

Colin W.

>
>On Fri, 2003-09-05 at 20:32, Colin J. Williams wrote:
>  
>
>>Todd,
>>
>>Thanks again for your prompt response.
>>Todd Miller wrote:
>>    
>>
>>>Hi Colin,
>>>
>>>I haven't looked at this in detail,  but special.__init__() is making me
>>>queasy.  
>>>      
>>>
>>We don't call __init__ directly, but only when we construct the
>>instance, a,  of the class special.
>>    
>>
>>>Why doesn't it call NumArray.__init__()?
>>>      
>>>
>>Because we have data to install in the newly created a.  We go off to
>>the factory function, numarray.array, to create the array, which we
>>then specialize.
>>
>>I don't see how NumArray.__init__ would help.  This cumbersome
>>procedure seems to work OK in most respects, but fails to display
>>properly.
>>
>>I don't follow the code sufficiently well to be sure, but my hunch is
>>that the problem lies with setShape.  It fails to update _strides.
>>
>>Colin W.
>>    
>>
>>>Todd
>>>
>>>On Fri, 2003-09-05 at 15:03, Colin J. Williams wrote:
>>>  
>>>      
>>>
>>>>When ravel() is used on an instance of NumArray, the _strides attribute 
>>>>is recalculated correctly.
>>>>When ravel() is used on an instance of special, a sub-class of NumArray, 
>>>>the _strides attribute keeps its old value.  This appears to cause the 
>>>>display problem which is reported.
>>>>
>>>>The essentials of the code used are below.
>>>>
>>>>To investigate this problem, two print statements were added to the 
>>>>method setshape in generic.py:
>>>>
>>>>            if newnelements == nelements:
>>>>                self._shape = tuple(shape)
>>>>                print 'In setShape self._shape:', self._shape
>>>>                self._strides = self._stridesFromShape()
>>>>                print 'In setShape self._strides:', self._strides
>>>>            else:
>>>>                raise ValueError("New shape is not consistent with the
>>>>    old shape")
>>>>
>>>>It seems that only the first of these print statements is executed with 
>>>>special, but both are executed with NumArray.  In the second case, the 
>>>>output is not in the expected sequence.
>>>>
>>>>I would appreciate any suggestion as to a workaround.
>>>>
>>>>Colin W,
>>>>
>>>>Grief when attempting to sub-class NumArray
>>>>
>>>>Code used:
>>>>
>>>># Try to sub-class
>>>>class special(N.NumArray):
>>>>  def __init__(self, data= None, shape= None, eType= None):
>>>>##                         eType= _nt.Any or AnyType not acceptable %%
>>>>    arr= N.array(sequence= data, shape= shape, type= eType)
>>>>    for attr in dir(arr):
>>>>##      This is a longwinded way of setting up special
>>>>##      There must be a better way                     %%
>>>>##      Perhaps we should use generic._view
>>>>##      It's not documented                 TRY IT LATER
>>>>##      There is no need to transfer methods!
>>>>      if attr[0] == '_' and attr[1] != '_':
>>>>        print attr
>>>>        exec 'self.' + attr + '= ' 'arr.' + attr
>>>>        exec 'print self.' + attr
>>>>    pass
>>>>
>>>>a= special(data= [1, 2, 3, 4], shape= (2, 2))
>>>>a= N.array([1, 2, 3, 4], shape= (2, 2))
>>>>print 'In ts a._strides:', a._strides
>>>>b= _gen.ravel(a)
>>>>print 'In ts b._strides:', b._strides       #  <<< Unchanged with 
>>>>special, OK with NumArray
>>>>print 'b:', b
>>>>
>>>>
>>>>
>>>>USING THE SUBCLASS special
>>>> >C:\Progra~1\Python23\pythonw -u ts.py
>>>>In ts a._strides: (8, 4)
>>>>In setShape self._shape: (4,)
>>>>In ts b._strides: (8, 4)
>>>>b: In setShape self._shape: (4,)
>>>>Traceback (most recent call last):
>>>>  File "ts.py", line 37, in ?
>>>>    print 'b:', b
>>>>  File 
>>>>"C:\PROGRA~1\Python23\lib\site-packages\numarray\numarraycore.py", line 
>>>>618, in __str__
>>>>    MAX_LINE_WIDTH, PRECISION, SUPPRESS_SMALL, ' ', "")
>>>>  File "C:\PROGRA~1\Python23\lib\site-packages\numarray\arrayprint.py", 
>>>>line 176, in array2string
>>>>    separator, prefix)
>>>>  File "C:\PROGRA~1\Python23\lib\site-packages\numarray\arrayprint.py", 
>>>>line 138, in _array2string
>>>>    max_str_len = max(len(str(max_reduce(data))),
>>>>libnumarray.error: maximum_Int32_reduce: access beyond buffer. offset=27 
>>>>buffersize=16
>>>> >Exit code: 1
>>>>__________________________________________________________
>>>>
>>>># USING NumArray
>>>>
>>>>In setShape self._shape: (2, 2)
>>>>In ts a._strides: (8, 4)
>>>>In setShape self._shape: (4,)
>>>>In ts b._strides: (4,)
>>>>b: In setShape self._shape: (4,)
>>>>[1 2 3 4]
>>>> >Exit code: 0
>>>>
>>>>
>>>>
>>>>
>>>>-------------------------------------------------------
>>>>This sf.net email is sponsored by:ThinkGeek
>>>>Welcome to geek heaven.
>>>>http://thinkgeek.com/sf
>>>>_______________________________________________
>>>>Numpy-discussion mailing list
>>>>Numpy-discussion at lists.sourceforge.net
>>>>https://lists.sourceforge.net/lists/listinfo/numpy-discussion
>>>>    
>>>>        
>>>>





From alnesbit at students.cs.mu.oz.au  Sun Sep  7 03:50:07 2003
From: alnesbit at students.cs.mu.oz.au (Andrew Nesbit)
Date: Sun Sep  7 03:50:07 2003
Subject: [Numpy-discussion] Matrix square root
In-Reply-To: <3F59F1D8.5020900@erols.com> (Bernard Frankpitt's message of
 "Sat, 06 Sep 2003 10:40:24 -0400")
References: <m2ekywpqd8.fsf@naima.lensflare.org>
	<200309041707.34519.hinsen@cnrs-orleans.fr>
	<m265k8o8q0.fsf@naima.lensflare.org> <3F59F1D8.5020900@erols.com>
Message-ID: <m2iso4na92.fsf@naima.lensflare.org>

Bernard Frankpitt <frankpit at erols.com> writes:

[snip... finding matrix square root through eigenvalue decomposition.]

> The only problem with this approach is that you have to find all the
> eigenvalues and eigenvectors to get the spectral decomposition.  For
> large, poorly conditioned matrices this can be a source of significant
> errors, and a lot of work. An alternative, and often faster and more
> accurate approach is to use a series expansion for the square
> root. Evaluating the series expansion only requires multiplication.

This technique sounds like it's definitely worth checking out.

Thankyou for the tip.

Andrew.




From jmiller at stsci.edu  Sun Sep  7 04:09:02 2003
From: jmiller at stsci.edu (Todd Miller)
Date: Sun Sep  7 04:09:02 2003
Subject: [Numpy-discussion] Attempting to sub-class NumArray
In-Reply-To: <3F59FC78.3060307@sympatico.ca>
References: <3F58DDE5.5090608@sympatico.ca>
	 <1062799403.5615.25.camel@halloween.stsci.edu>
	 <3F592B3A.5060505@sympatico.ca>
	 <1062848229.3459.21.camel@localhost.localdomain>
	 <3F59FC78.3060307@sympatico.ca>
Message-ID: <1062932892.15057.105.camel@localhost.localdomain>

On Sat, 2003-09-06 at 11:25, Colin J. Williams wrote:
> Todd Miller wrote:
> 
> >I think the problem you are having is caused by the assumption that
> >instance initialization is equivalent to an unordered assignment of all
> >instance attributes.  That's not true because the private attributes are
> >interrelated;  assigning "shape" affects "_shape" and "_strides",
> >_itemsize must be known before _strides can be computed from shape, and
> >so on. 
> >
> It would be helpful if you could let us know what assignment ordering 
> would be appropriate, or, if there were information about the 
> interactions, perhaps we could work these things out for ourselves.

In truth,  I don't think the order is the issue, so let's drop that. 
All I can say is that somehow special.__init__() is creating
inconsistent _shape and _strides.  I also noticed it creating and
assigning bound methods, which you probably don't want to do.

> As we've discussed before, there is little information available on 
> private objects and their roles.  Even to have the source code of some 
> of the C code might be useful.  I wasn't able to spot the relevant C 
> code among the CVS directories.

The source code is all there.

Look in Include/numarray/numarray.h, Src/_ndarraymodule.c, and
Src/_numarraymodule.c, and maybe Src/libnumarraymodule.c.

> However, the assumption would appear to apply to the crude 
> initialization of 'special'.  The problem which is being reported seems 
> to be connected with the function setshape in generic.py.  This is far 
> from the unordered assignment, although I recognize that ther might be 
> some connection.

I'm not sure where and how, but I think the problem is in your dir()
loop.   So, instead, try this:

self.__setstate__(arr.__getstate__())

That ensures that you're only copying the data attributes and appears to
work.

Todd

> Colin W.
> 
> >
> >On Fri, 2003-09-05 at 20:32, Colin J. Williams wrote:
> >  
> >
> >>Todd,
> >>
> >>Thanks again for your prompt response.
> >>Todd Miller wrote:
> >>    
> >>
> >>>Hi Colin,
> >>>
> >>>I haven't looked at this in detail,  but special.__init__() is making me
> >>>queasy.  
> >>>      
> >>>
> >>We don't call __init__ directly, but only when we construct the
> >>instance, a,  of the class special.
> >>    
> >>
> >>>Why doesn't it call NumArray.__init__()?
> >>>      
> >>>
> >>Because we have data to install in the newly created a.  We go off to
> >>the factory function, numarray.array, to create the array, which we
> >>then specialize.
> >>
> >>I don't see how NumArray.__init__ would help.  This cumbersome
> >>procedure seems to work OK in most respects, but fails to display
> >>properly.
> >>
> >>I don't follow the code sufficiently well to be sure, but my hunch is
> >>that the problem lies with setShape.  It fails to update _strides.
> >>
> >>Colin W.
> >>    
> >>
> >>>Todd
> >>>
> >>>On Fri, 2003-09-05 at 15:03, Colin J. Williams wrote:
> >>>  
> >>>      
> >>>
> >>>>When ravel() is used on an instance of NumArray, the _strides attribute 
> >>>>is recalculated correctly.
> >>>>When ravel() is used on an instance of special, a sub-class of NumArray, 
> >>>>the _strides attribute keeps its old value.  This appears to cause the 
> >>>>display problem which is reported.
> >>>>
> >>>>The essentials of the code used are below.
> >>>>
> >>>>To investigate this problem, two print statements were added to the 
> >>>>method setshape in generic.py:
> >>>>
> >>>>            if newnelements == nelements:
> >>>>                self._shape = tuple(shape)
> >>>>                print 'In setShape self._shape:', self._shape
> >>>>                self._strides = self._stridesFromShape()
> >>>>                print 'In setShape self._strides:', self._strides
> >>>>            else:
> >>>>                raise ValueError("New shape is not consistent with the
> >>>>    old shape")
> >>>>
> >>>>It seems that only the first of these print statements is executed with 
> >>>>special, but both are executed with NumArray.  In the second case, the 
> >>>>output is not in the expected sequence.
> >>>>
> >>>>I would appreciate any suggestion as to a workaround.
> >>>>
> >>>>Colin W,
> >>>>
> >>>>Grief when attempting to sub-class NumArray
> >>>>
> >>>>Code used:
> >>>>
> >>>># Try to sub-class
> >>>>class special(N.NumArray):
> >>>>  def __init__(self, data= None, shape= None, eType= None):
> >>>>##                         eType= _nt.Any or AnyType not acceptable %%
> >>>>    arr= N.array(sequence= data, shape= shape, type= eType)
> >>>>    for attr in dir(arr):
> >>>>##      This is a longwinded way of setting up special
> >>>>##      There must be a better way                     %%
> >>>>##      Perhaps we should use generic._view
> >>>>##      It's not documented                 TRY IT LATER
> >>>>##      There is no need to transfer methods!
> >>>>      if attr[0] == '_' and attr[1] != '_':
> >>>>        print attr
> >>>>        exec 'self.' + attr + '= ' 'arr.' + attr
> >>>>        exec 'print self.' + attr
> >>>>    pass
> >>>>
> >>>>a= special(data= [1, 2, 3, 4], shape= (2, 2))
> >>>>a= N.array([1, 2, 3, 4], shape= (2, 2))
> >>>>print 'In ts a._strides:', a._strides
> >>>>b= _gen.ravel(a)
> >>>>print 'In ts b._strides:', b._strides       #  <<< Unchanged with 
> >>>>special, OK with NumArray
> >>>>print 'b:', b
> >>>>
> >>>>
> >>>>
> >>>>USING THE SUBCLASS special
> >>>> >C:\Progra~1\Python23\pythonw -u ts.py
> >>>>In ts a._strides: (8, 4)
> >>>>In setShape self._shape: (4,)
> >>>>In ts b._strides: (8, 4)
> >>>>b: In setShape self._shape: (4,)
> >>>>Traceback (most recent call last):
> >>>>  File "ts.py", line 37, in ?
> >>>>    print 'b:', b
> >>>>  File 
> >>>>"C:\PROGRA~1\Python23\lib\site-packages\numarray\numarraycore.py", line 
> >>>>618, in __str__
> >>>>    MAX_LINE_WIDTH, PRECISION, SUPPRESS_SMALL, ' ', "")
> >>>>  File "C:\PROGRA~1\Python23\lib\site-packages\numarray\arrayprint.py", 
> >>>>line 176, in array2string
> >>>>    separator, prefix)
> >>>>  File "C:\PROGRA~1\Python23\lib\site-packages\numarray\arrayprint.py", 
> >>>>line 138, in _array2string
> >>>>    max_str_len = max(len(str(max_reduce(data))),
> >>>>libnumarray.error: maximum_Int32_reduce: access beyond buffer. offset=27 
> >>>>buffersize=16
> >>>> >Exit code: 1
> >>>>__________________________________________________________
> >>>>
> >>>># USING NumArray
> >>>>
> >>>>In setShape self._shape: (2, 2)
> >>>>In ts a._strides: (8, 4)
> >>>>In setShape self._shape: (4,)
> >>>>In ts b._strides: (4,)
> >>>>b: In setShape self._shape: (4,)
> >>>>[1 2 3 4]
> >>>> >Exit code: 0
> >>>>
> >>>>
> >>>>
> >>>>
> >>>>-------------------------------------------------------
> >>>>This sf.net email is sponsored by:ThinkGeek
> >>>>Welcome to geek heaven.
> >>>>http://thinkgeek.com/sf
> >>>>_______________________________________________
> >>>>Numpy-discussion mailing list
> >>>>Numpy-discussion at lists.sourceforge.net
> >>>>https://lists.sourceforge.net/lists/listinfo/numpy-discussion
> >>>>    
> >>>>        
> >>>>
-- 
Todd Miller <jmiller at stsci.edu>




From cjw at sympatico.ca  Sun Sep  7 16:55:01 2003
From: cjw at sympatico.ca (Colin J. Williams)
Date: Sun Sep  7 16:55:01 2003
Subject: [Numpy-discussion] Attempting to sub-class NumArray
In-Reply-To: <1062932892.15057.105.camel@localhost.localdomain>
References: <3F58DDE5.5090608@sympatico.ca>	 <1062799403.5615.25.camel@halloween.stsci.edu>	 <3F592B3A.5060505@sympatico.ca>	 <1062848229.3459.21.camel@localhost.localdomain>	 <3F59FC78.3060307@sympatico.ca> <1062932892.15057.105.camel@localhost.localdomain>
Message-ID: <3F5BC513.6090400@sympatico.ca>

Todd Miller wrote:

>On Sat, 2003-09-06 at 11:25, Colin J. Williams wrote:
>  
>
>>Todd Miller wrote:
>>
>>    
>>
>>>I think the problem you are having is caused by the assumption that
>>>instance initialization is equivalent to an unordered assignment of all
>>>instance attributes.  That's not true because the private attributes are
>>>interrelated;  assigning "shape" affects "_shape" and "_strides",
>>>_itemsize must be known before _strides can be computed from shape, and
>>>so on. 
>>>
>>>      
>>>
>>It would be helpful if you could let us know what assignment ordering 
>>would be appropriate, or, if there were information about the 
>>interactions, perhaps we could work these things out for ourselves.
>>    
>>
>
>In truth,  I don't think the order is the issue, so let's drop that. 
>All I can say is that somehow special.__init__() is creating
>inconsistent _shape and _strides.  I also noticed it creating and
>assigning bound methods, which you probably don't want to do.
>
Yes, I had spotted that and limited the updating to the variables.  This 
seemed to work, but your suggestion below is the neater way to go.

>>As we've discussed before, there is little information available on 
>>private objects and their roles.  Even to have the source code of some 
>>of the C code might be useful.  I wasn't able to spot the relevant C 
>>code among the CVS directories.
>>    
>>
>
>The source code is all there.
>
>Look in Include/numarray/numarray.h, Src/_ndarraymodule.c, and
>Src/_numarraymodule.c, and maybe Src/libnumarraymodule.c.
>
Thanks, I'll poke around there.

>
>  
>
>>However, the assumption would appear to apply to the crude 
>>initialization of 'special'.  The problem which is being reported seems 
>>to be connected with the function setshape in generic.py.  This is far 
>>from the unordered assignment, although I recognize that ther might be 
>>some connection.
>>    
>>
>
>I'm not sure where and how, but I think the problem is in your dir()
>loop.   So, instead, try this:
>
>self.__setstate__(arr.__getstate__())
>
This seems to be a better approach than mine.

>
>That ensures that you're only copying the data attributes and appears to
>work.
>
Thanks, over that hurdle and on to the next

Colin W.

>
>Todd
>
>  
>
>>Colin W.
>>
>>    
>>
>>>On Fri, 2003-09-05 at 20:32, Colin J. Williams wrote:
>>> 
>>>
>>>      
>>>
>>>>Todd,
>>>>
>>>>Thanks again for your prompt response.
>>>>Todd Miller wrote:
>>>>   
>>>>
>>>>        
>>>>
>>>>>Hi Colin,
>>>>>
>>>>>I haven't looked at this in detail,  but special.__init__() is making me
>>>>>queasy.  
>>>>>     
>>>>>
>>>>>          
>>>>>
>>>>We don't call __init__ directly, but only when we construct the
>>>>instance, a,  of the class special.
>>>>   
>>>>
>>>>        
>>>>
>>>>>Why doesn't it call NumArray.__init__()?
>>>>>     
>>>>>
>>>>>          
>>>>>
>>>>Because we have data to install in the newly created a.  We go off to
>>>>the factory function, numarray.array, to create the array, which we
>>>>then specialize.
>>>>
>>>>I don't see how NumArray.__init__ would help.  This cumbersome
>>>>procedure seems to work OK in most respects, but fails to display
>>>>properly.
>>>>
>>>>I don't follow the code sufficiently well to be sure, but my hunch is
>>>>that the problem lies with setShape.  It fails to update _strides.
>>>>
>>>>Colin W.
>>>>   
>>>>
>>>>        
>>>>
>>>>>Todd
>>>>>
>>>>>On Fri, 2003-09-05 at 15:03, Colin J. Williams wrote:
>>>>> 
>>>>>     
>>>>>
>>>>>          
>>>>>
>>>>>>When ravel() is used on an instance of NumArray, the _strides attribute 
>>>>>>is recalculated correctly.
>>>>>>When ravel() is used on an instance of special, a sub-class of NumArray, 
>>>>>>the _strides attribute keeps its old value.  This appears to cause the 
>>>>>>display problem which is reported.
>>>>>>
>>>>>>The essentials of the code used are below.
>>>>>>
>>>>>>To investigate this problem, two print statements were added to the 
>>>>>>method setshape in generic.py:
>>>>>>
>>>>>>           if newnelements == nelements:
>>>>>>               self._shape = tuple(shape)
>>>>>>               print 'In setShape self._shape:', self._shape
>>>>>>               self._strides = self._stridesFromShape()
>>>>>>               print 'In setShape self._strides:', self._strides
>>>>>>           else:
>>>>>>               raise ValueError("New shape is not consistent with the
>>>>>>   old shape")
>>>>>>
>>>>>>It seems that only the first of these print statements is executed with 
>>>>>>special, but both are executed with NumArray.  In the second case, the 
>>>>>>output is not in the expected sequence.
>>>>>>
>>>>>>I would appreciate any suggestion as to a workaround.
>>>>>>
>>>>>>Colin W,
>>>>>>
>>>>>>Grief when attempting to sub-class NumArray
>>>>>>
>>>>>>Code used:
>>>>>>
>>>>>># Try to sub-class
>>>>>>class special(N.NumArray):
>>>>>> def __init__(self, data= None, shape= None, eType= None):
>>>>>>##                         eType= _nt.Any or AnyType not acceptable %%
>>>>>>   arr= N.array(sequence= data, shape= shape, type= eType)
>>>>>>   for attr in dir(arr):
>>>>>>##      This is a longwinded way of setting up special
>>>>>>##      There must be a better way                     %%
>>>>>>##      Perhaps we should use generic._view
>>>>>>##      It's not documented                 TRY IT LATER
>>>>>>##      There is no need to transfer methods!
>>>>>>     if attr[0] == '_' and attr[1] != '_':
>>>>>>       print attr
>>>>>>       exec 'self.' + attr + '= ' 'arr.' + attr
>>>>>>       exec 'print self.' + attr
>>>>>>   pass
>>>>>>
>>>>>>a= special(data= [1, 2, 3, 4], shape= (2, 2))
>>>>>>a= N.array([1, 2, 3, 4], shape= (2, 2))
>>>>>>print 'In ts a._strides:', a._strides
>>>>>>b= _gen.ravel(a)
>>>>>>print 'In ts b._strides:', b._strides       #  <<< Unchanged with 
>>>>>>special, OK with NumArray
>>>>>>print 'b:', b
>>>>>>
>>>>>>
>>>>>>
>>>>>>USING THE SUBCLASS special
>>>>>>            
>>>>>>
>>>>>>>C:\Progra~1\Python23\pythonw -u ts.py
>>>>>>>              
>>>>>>>
>>>>>>In ts a._strides: (8, 4)
>>>>>>In setShape self._shape: (4,)
>>>>>>In ts b._strides: (8, 4)
>>>>>>b: In setShape self._shape: (4,)
>>>>>>Traceback (most recent call last):
>>>>>> File "ts.py", line 37, in ?
>>>>>>   print 'b:', b
>>>>>> File 
>>>>>>"C:\PROGRA~1\Python23\lib\site-packages\numarray\numarraycore.py", line 
>>>>>>618, in __str__
>>>>>>   MAX_LINE_WIDTH, PRECISION, SUPPRESS_SMALL, ' ', "")
>>>>>> File "C:\PROGRA~1\Python23\lib\site-packages\numarray\arrayprint.py", 
>>>>>>line 176, in array2string
>>>>>>   separator, prefix)
>>>>>> File "C:\PROGRA~1\Python23\lib\site-packages\numarray\arrayprint.py", 
>>>>>>line 138, in _array2string
>>>>>>   max_str_len = max(len(str(max_reduce(data))),
>>>>>>libnumarray.error: maximum_Int32_reduce: access beyond buffer. offset=27 
>>>>>>buffersize=16
>>>>>>            
>>>>>>
>>>>>>>Exit code: 1
>>>>>>>              
>>>>>>>
>>>>>>__________________________________________________________
>>>>>>
>>>>>># USING NumArray
>>>>>>
>>>>>>In setShape self._shape: (2, 2)
>>>>>>In ts a._strides: (8, 4)
>>>>>>In setShape self._shape: (4,)
>>>>>>In ts b._strides: (4,)
>>>>>>b: In setShape self._shape: (4,)
>>>>>>[1 2 3 4]
>>>>>>            
>>>>>>
>>>>>>>Exit code: 0
>>>>>>>              
>>>>>>>
>>>>>>
>>>>>>
>>>>>>-------------------------------------------------------
>>>>>>This sf.net email is sponsored by:ThinkGeek
>>>>>>Welcome to geek heaven.
>>>>>>http://thinkgeek.com/sf
>>>>>>_______________________________________________
>>>>>>Numpy-discussion mailing list
>>>>>>Numpy-discussion at lists.sourceforge.net
>>>>>>https://lists.sourceforge.net/lists/listinfo/numpy-discussion
>>>>>>   
>>>>>>       
>>>>>>
>>>>>>            
>>>>>>





From verveer at embl-heidelberg.de  Mon Sep  8 21:30:11 2003
From: verveer at embl-heidelberg.de (Peter Verveer)
Date: Mon Sep  8 21:30:11 2003
Subject: [Numpy-discussion] NA_ReturnOutput question
Message-ID: <200309081734.12819.verveer@embl-heidelberg.de>

The documentation states that NA_ReturnOutput(numarray, shadow) should be used 
with the value returned by NA_OptionalOutputArray() for the shadow parameter. 
Is it also possible to use a shadow array allocated otherwise, e.g. by 
NA_NewArray()?

Cheers, Peter

-- 
Dr. Peter J. Verveer
Cell Biology and Cell Biophysics Programme
EMBL
Meyerhofstrasse 1
D-69117 Heidelberg
Germany
Tel. : +49 6221 387245
Fax  : +49 6221 387242
Email: Peter.Verveer at embl-heidelberg.de





From jmiller at stsci.edu  Tue Sep  9 06:04:10 2003
From: jmiller at stsci.edu (Todd Miller)
Date: Tue Sep  9 06:04:10 2003
Subject: [Numpy-discussion] NA_ReturnOutput question
In-Reply-To: <200309081734.12819.verveer@embl-heidelberg.de>
References: <200309081734.12819.verveer@embl-heidelberg.de>
Message-ID: <1063112612.1633.64.camel@halloween.stsci.edu>

On Mon, 2003-09-08 at 11:34, Peter Verveer wrote:
> The documentation states that NA_ReturnOutput(numarray, shadow) should be used 
> with the value returned by NA_OptionalOutputArray() for the shadow parameter. 

The purpose of NA_OptionalOutputArray() and NA_ReturnOutput() is to
support an optional output parameter and return value with a couple
specific behaviors:

1. If no output array is specified to the function (it is left defaulted
to None), then the function makes up an output array by cloning it
(shape and type) from some 'master' array.  The output array is returned
as the result of the function.

2. If an output array is specified, then it may still need to be
"shadowed" to correctly match the type, byteswap, and alignment
requirements specified for the function.  When a shadow output is
created,  it is used in all of the real calculations and then copied
back onto the original array and deallocated at function exit.  The
function returns None.

> Is it also possible to use a shadow array allocated otherwise, e.g. by 
> NA_NewArray()?

I don't think so.  It is however possible to return an array allocated
by NA_NewArray() as-is.  

> 
> Cheers, Peter
> 
> -- 
> Dr. Peter J. Verveer
> Cell Biology and Cell Biophysics Programme
> EMBL
> Meyerhofstrasse 1
> D-69117 Heidelberg
> Germany
> Tel. : +49 6221 387245
> Fax  : +49 6221 387242
> Email: Peter.Verveer at embl-heidelberg.de
> 
> 
> 
> 
> -------------------------------------------------------
> This sf.net email is sponsored by:ThinkGeek
> Welcome to geek heaven.
> http://thinkgeek.com/sf
> _______________________________________________
> Numpy-discussion mailing list
> Numpy-discussion at lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/numpy-discussion
-- 
Todd Miller 			jmiller at stsci.edu
STSCI / ESS / SSB




From jmiller at stsci.edu  Tue Sep  9 13:25:59 2003
From: jmiller at stsci.edu (Todd Miller)
Date: Tue Sep  9 13:25:59 2003
Subject: [Numpy-discussion] Boolean array formatting poll
Message-ID: <1063138536.1704.181.camel@halloween.stsci.edu>

Now that Python has a bool type, I have received a patch for numarray's
arrayprint module which alters the formatting of Boolean arrays from:

>>>> a=numarray.arange(6, type=numarray.Bool); a[::2] = 0; a
array([0, 1, 0, 1, 0, 1], type=Bool)

to:

>>>> a=numarray.arange(6, type=numarray.Bool); a[::2] = 0; a
array([False, True, False, True, False, True], type=Bool)

Which format do you prefer?   Please vote for or against changing the
format:  +1 (for)  or -1 (against).

-- 
Todd Miller 			jmiller at stsci.edu
STSCI / ESS / SSB




From Chris.Barker at noaa.gov  Tue Sep  9 16:15:21 2003
From: Chris.Barker at noaa.gov (Chris Barker)
Date: Tue Sep  9 16:15:21 2003
Subject: [Numpy-discussion] Boolean array formatting poll
References: <1063138536.1704.181.camel@halloween.stsci.edu>
Message-ID: <3F5E3D2D.48EA9D54@noaa.gov>

Todd Miller wrote:

> >>>> a=numarray.arange(6, type=numarray.Bool); a[::2] = 0; a
> array([False, True, False, True, False, True], type=Bool)
> 
> Which format do you prefer?   Please vote for or against changing the
> format:  +1 (for)  or -1 (against).

-1

It just makes it harder to read, particularly for a large array

-Chris


-- 
Christopher Barker, Ph.D.
Oceanographer
                                    		
NOAA/OR&R/HAZMAT         (206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115       (206) 526-6317   main reception

Chris.Barker at noaa.gov



From cjw at sympatico.ca  Tue Sep  9 16:31:14 2003
From: cjw at sympatico.ca (Colin J. Williams)
Date: Tue Sep  9 16:31:14 2003
Subject: [Numpy-discussion] Boolean array formatting poll
In-Reply-To: <1063138536.1704.181.camel@halloween.stsci.edu>
References: <1063138536.1704.181.camel@halloween.stsci.edu>
Message-ID: <3F5E4950.9070801@sympatico.ca>

Vote +1
Todd Miller wrote:

>Now that Python has a bool type, I have received a patch for numarray's
>arrayprint module which alters the formatting of Boolean arrays from:
>
>  
>
>>>>>a=numarray.arange(6, type=numarray.Bool); a[::2] = 0; a
>>>>>          
>>>>>
>array([0, 1, 0, 1, 0, 1], type=Bool)
>
>to:
>
>  
>
>>>>>a=numarray.arange(6, type=numarray.Bool); a[::2] = 0; a
>>>>>          
>>>>>
>array([False, True, False, True, False, True], type=Bool)
>
>Which format do you prefer?   Please vote for or against changing the
>format:  +1 (for)  or -1 (against).
>
>  
>





From gvermeul at grenoble.cnrs.fr  Wed Sep 10 00:29:07 2003
From: gvermeul at grenoble.cnrs.fr (Gerard Vermeulen)
Date: Wed Sep 10 00:29:07 2003
Subject: [Numpy-discussion] Boolean array formatting poll
In-Reply-To: <1063138536.1704.181.camel@halloween.stsci.edu>
References: <1063138536.1704.181.camel@halloween.stsci.edu>
Message-ID: <20030910092709.018a0d78.gvermeul@grenoble.cnrs.fr>

-1, because of readability

Gerard

On 09 Sep 2003 16:15:36 -0400
Todd Miller <jmiller at stsci.edu> wrote:

> Now that Python has a bool type, I have received a patch for numarray's
> arrayprint module which alters the formatting of Boolean arrays from:
> 
> >>>> a=numarray.arange(6, type=numarray.Bool); a[::2] = 0; a
> array([0, 1, 0, 1, 0, 1], type=Bool)
> 
> to:
> 
> >>>> a=numarray.arange(6, type=numarray.Bool); a[::2] = 0; a
> array([False, True, False, True, False, True], type=Bool)
> 
> Which format do you prefer?   Please vote for or against changing the
> format:  +1 (for)  or -1 (against).
> 
> -- 
> Todd Miller 			jmiller at stsci.edu
> STSCI / ESS / SSB
> 
> 
> 
> -------------------------------------------------------
> This sf.net email is sponsored by:ThinkGeek
> Welcome to geek heaven.
> http://thinkgeek.com/sf
> _______________________________________________
> Numpy-discussion mailing list
> Numpy-discussion at lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/numpy-discussion
> 



From falted at openlc.org  Wed Sep 10 00:41:03 2003
From: falted at openlc.org (Francesc Alted)
Date: Wed Sep 10 00:41:03 2003
Subject: [Numpy-discussion] Boolean array formatting poll
In-Reply-To: <1063138536.1704.181.camel@halloween.stsci.edu>
References: <1063138536.1704.181.camel@halloween.stsci.edu>
Message-ID: <200309100939.47150.falted@openlc.org>

+ 1, because of consistency

-- 
Francesc Alted




From falted at openlc.org  Wed Sep 10 02:25:29 2003
From: falted at openlc.org (Francesc Alted)
Date: Wed Sep 10 02:25:29 2003
Subject: [Numpy-discussion] Why stripping spaces in strings?
Message-ID: <200309101117.15534.falted@openlc.org>

Hi,

A user of pytables recently asked me why it truncates the trailing spaces on
strings. I have to explain him that the information is there, but the
strings module of the numarray package deliberately strips off the trailing
spaces when printing:

>>> from numarray import strings
>>> a=strings.array("hello world  ")
>>> print a
['hello world']
>>> print "<%s>" % a
<['hello world']>
>>> len(a[0])
11

however:

>>> print "<%s>" % a._data
<hello world  >
>>> a._byteView()
array([[104, 101, 108, 108, 111,  32, 119, 111, 114, 108, 100,  32,  32]], 
type=UInt8)
>>> a._itemsize
13

In the beginning I thought that this was a nice feature, but when people is
using numarray.strings objects as containers (for example for Pickled
objects), that can be rather confusing, and I would prefer to see all the
trailing blanks.

But as I am not completely sure, I just would like to discuss the pro's and
con's of the current printing apprach.

Cheers,

-- 
Francesc Alted




From pearu at cens.ioc.ee  Wed Sep 10 03:19:03 2003
From: pearu at cens.ioc.ee (Pearu Peterson)
Date: Wed Sep 10 03:19:03 2003
Subject: [Numpy-discussion] Boolean array formatting poll
In-Reply-To: <1063138536.1704.181.camel@halloween.stsci.edu>
Message-ID: <Pine.LNX.4.21.0309101313200.8554-100000@cens.kybi>

Considering the number of `-1`s, how about

>>> a=numarray.arange(6, type=numarray.Bool); a[::2] = 0; a
 array([F, T, F, T, F, T], type=Bool)

which is short and readable.

Pearu




From cjw at sympatico.ca  Wed Sep 10 04:06:22 2003
From: cjw at sympatico.ca (Colin J. Williams)
Date: Wed Sep 10 04:06:22 2003
Subject: [Numpy-discussion] Boolean array formatting poll
In-Reply-To: <3F5E3D2D.48EA9D54@noaa.gov>
References: <1063138536.1704.181.camel@halloween.stsci.edu> <3F5E3D2D.48EA9D54@noaa.gov>
Message-ID: <3F5F058A.2010204@sympatico.ca>

Chris Barker wrote:

>Todd Miller wrote:
>
>  
>
>>>>>>a=numarray.arange(6, type=numarray.Bool); a[::2] = 0; a
>>>>>>            
>>>>>>
>>array([False, True, False, True, False, True], type=Bool)
>>
>>Which format do you prefer?   Please vote for or against changing the
>>format:  +1 (for)  or -1 (against).
>>    
>>
>
>-1
>
>It just makes it harder to read, particularly for a large array
>
>-Chris
>
Perhaps the important thing is to facilitate visual comparison between 
one row and another or between one array and another.  Maybe 'False' and 
'True ' (with a trailing space) could be used as representations of the 
Boolean values.

If a compressed representation is desired then 'F' and 'T' might be used.

This could be specified in a site constant which, for the distribution, 
would default to one of ('0, '1'), ('False', 'True'), ('False', 'True ') 
or ('F', 'T').

My feeling is that, since it can easily be done, it is desirable to 
distinguish between Integer and Boolean values.

Colin W.

>
>
>  
>





From barrett at stsci.edu  Wed Sep 10 05:21:03 2003
From: barrett at stsci.edu (Paul Barrett)
Date: Wed Sep 10 05:21:03 2003
Subject: [Numpy-discussion] Boolean array formatting poll
In-Reply-To: <1063138536.1704.181.camel@halloween.stsci.edu>
References: <1063138536.1704.181.camel@halloween.stsci.edu>
Message-ID: <3F5F1691.4080906@stsci.edu>

+1 for consistency and clarity.

Todd Miller wrote:
> Now that Python has a bool type, I have received a patch for numarray's
> arrayprint module which alters the formatting of Boolean arrays from:
> 
> 
>>>>>a=numarray.arange(6, type=numarray.Bool); a[::2] = 0; a
> 
> array([0, 1, 0, 1, 0, 1], type=Bool)
> 
> to:
> 
> 
>>>>>a=numarray.arange(6, type=numarray.Bool); a[::2] = 0; a
> 
> array([False, True, False, True, False, True], type=Bool)
> 
> Which format do you prefer?   Please vote for or against changing the
> format:  +1 (for)  or -1 (against).
> 


-- 
Paul Barrett, PhD      Space Telescope Science Institute
Phone: 410-338-4475    ESS/Science Software Group
FAX:   410-338-4767    Baltimore, MD 21218




From jmiller at stsci.edu  Wed Sep 10 05:51:11 2003
From: jmiller at stsci.edu (Todd Miller)
Date: Wed Sep 10 05:51:11 2003
Subject: [Numpy-discussion] Why stripping spaces in strings?
In-Reply-To: <200309101117.15534.falted@openlc.org>
References: <200309101117.15534.falted@openlc.org>
Message-ID: <1063197624.3869.3.camel@halloween.stsci.edu>

Take a look at RawCharArray instead of CharArray.  CharArray is a
RawCharArray with specific stripping and padding behaviors and some
extra methods.  RawCharArray should be the starting point for this kind
of discussion;  it may do what you want as-is, or you can tell me where
it's falling short of what you need.

Todd

On Wed, 2003-09-10 at 05:17, Francesc Alted wrote:
> Hi,
> 
> A user of pytables recently asked me why it truncates the trailing spaces on
> strings. I have to explain him that the information is there, but the
> strings module of the numarray package deliberately strips off the trailing
> spaces when printing:
> 
> >>> from numarray import strings
> >>> a=strings.array("hello world  ")
> >>> print a
> ['hello world']
> >>> print "<%s>" % a
> <['hello world']>
> >>> len(a[0])
> 11
> 
> however:
> 
> >>> print "<%s>" % a._data
> <hello world  >
> >>> a._byteView()
> array([[104, 101, 108, 108, 111,  32, 119, 111, 114, 108, 100,  32,  32]], 
> type=UInt8)
> >>> a._itemsize
> 13
> 
> In the beginning I thought that this was a nice feature, but when people is
> using numarray.strings objects as containers (for example for Pickled
> objects), that can be rather confusing, and I would prefer to see all the
> trailing blanks.
> 
> But as I am not completely sure, I just would like to discuss the pro's and
> con's of the current printing apprach.
> 
> Cheers,
> 
> -- 
> Francesc Alted
> 
> 
> 
> -------------------------------------------------------
> This sf.net email is sponsored by:ThinkGeek
> Welcome to geek heaven.
> http://thinkgeek.com/sf
> _______________________________________________
> Numpy-discussion mailing list
> Numpy-discussion at lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/numpy-discussion
-- 
Todd Miller 			jmiller at stsci.edu
STSCI / ESS / SSB




From mdehoon at ims.u-tokyo.ac.jp  Wed Sep 10 10:28:12 2003
From: mdehoon at ims.u-tokyo.ac.jp (Michiel Jan Laurens de Hoon)
Date: Wed Sep 10 10:28:12 2003
Subject: [Numpy-discussion] Boolean array formatting poll
In-Reply-To: <Pine.LNX.4.21.0309101313200.8554-100000@cens.kybi>
References: <Pine.LNX.4.21.0309101313200.8554-100000@cens.kybi>
Message-ID: <3F5F3F99.8010105@ims.u-tokyo.ac.jp>

-1, for readability.

1's and 0's can be distinguished more easily visually than T's and F's.

Michiel, U Tokyo.

Pearu Peterson wrote:
> Considering the number of `-1`s, how about
> 
> 
>>>>a=numarray.arange(6, type=numarray.Bool); a[::2] = 0; a
> 
>  array([F, T, F, T, F, T], type=Bool)
> 
> which is short and readable.
> 
> Pearu
> 
> 
> 
> -------------------------------------------------------
> This sf.net email is sponsored by:ThinkGeek
> Welcome to geek heaven.
> http://thinkgeek.com/sf
> _______________________________________________
> Numpy-discussion mailing list
> Numpy-discussion at lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/numpy-discussion
> 
> 

-- 
Michiel de Hoon
Assistant Professor
University of Tokyo, Institute of Medical Science
Human Genome Center
4-6-1 Shirokane-dai, Minato-ku
Tokyo 108-8639
Japan
http://bonsai.ims.u-tokyo.ac.jp/~mdehoon




From Chris.Barker at noaa.gov  Wed Sep 10 13:11:46 2003
From: Chris.Barker at noaa.gov (Chris Barker)
Date: Wed Sep 10 13:11:46 2003
Subject: [Numpy-discussion] Boolean array formatting poll
References: <Pine.LNX.4.21.0309101313200.8554-100000@cens.kybi>
Message-ID: <3F5F4DA7.2D640794@noaa.gov>

Pearu Peterson wrote:

> >>> a=numarray.arange(6, type=numarray.Bool); a[::2] = 0; a
>  array([F, T, F, T, F, T], type=Bool)

That's better. However, I"m still inclined to prefer [1, 0, 0, 1, ....

Someone mentioned making clear the distiction between bool and int
arrays. HOw distict are they? In Python, the bool type is really just a
pretty wrapper around an Int anway. I like it, but if in NumArray, a
bool is really and int in disguise, I'd be just as happy to ahve it be
0, 1, 0, .... If however, a bool really is a totally different beast (or
will be in a future version), then maybe it's worth making the clear
distiction.

-Chris

For what I mean, note in Python 2.3:

>>> a = True
>>> a
True
>>> a * 5
5


Can you do this in NumArray?




-- 
Christopher Barker, Ph.D.
Oceanographer
                                    		
NOAA/OR&R/HAZMAT         (206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115       (206) 526-6317   main reception

Chris.Barker at noaa.gov



From jmiller at stsci.edu  Wed Sep 10 13:37:03 2003
From: jmiller at stsci.edu (Todd Miller)
Date: Wed Sep 10 13:37:03 2003
Subject: [Numpy-discussion] Boolean array formatting poll
In-Reply-To: <3F5F4DA7.2D640794@noaa.gov>
References: <Pine.LNX.4.21.0309101313200.8554-100000@cens.kybi> 
	<3F5F4DA7.2D640794@noaa.gov>
Message-ID: <1063225004.3869.175.camel@halloween.stsci.edu>

<snip>

> 
> For what I mean, note in Python 2.3:
> 
> >>> a = True
> >>> a
> True
> >>> a * 5
> 5
> 
> 
> Can you do this in NumArray?

The numarray analog is:

>>> numarray.ones((10,), type=numarray.Bool)*5
array([5, 5, 5, 5, 5, 5, 5, 5, 5, 5])

So, yes.


-- 
Todd Miller 			jmiller at stsci.edu
STSCI / ESS / SSB




From fperez at colorado.edu  Wed Sep 10 17:19:38 2003
From: fperez at colorado.edu (Fernando Perez)
Date: Wed Sep 10 17:19:38 2003
Subject: [Numpy-discussion] Boolean array formatting poll
In-Reply-To: <Pine.LNX.4.21.0309101313200.8554-100000@cens.kybi>
References: <Pine.LNX.4.21.0309101313200.8554-100000@cens.kybi>
Message-ID: <3F5F52D6.1030106@colorado.edu>

Pearu Peterson wrote:
> Considering the number of `-1`s, how about
> 
> 
>>>>a=numarray.arange(6, type=numarray.Bool); a[::2] = 0; a
> 
>  array([F, T, F, T, F, T], type=Bool)

Somehow I seem to parse faster 0/1 arrays than F/T (the symbols are visually 
more different).

-0.5, but better than the change to True/False.

Cheers,

f




From falted at openlc.org  Thu Sep 11 03:58:58 2003
From: falted at openlc.org (Francesc Alted)
Date: Thu Sep 11 03:58:58 2003
Subject: [Numpy-discussion] Why stripping spaces in strings?
In-Reply-To: <1063197624.3869.3.camel@halloween.stsci.edu>
References: <200309101117.15534.falted@openlc.org> <1063197624.3869.3.camel@halloween.stsci.edu>
Message-ID: <200309111244.08209.falted@openlc.org>

Yups, RawCharrArray do the trick:

>>> strings.array([["hello world  ","pp\0"]], kind=strings.CharArray)
CharArray([['hello world', 'pp']])
>>> strings.array([["hello world  ","pp\0"]], kind=strings.RawCharArray)
RawCharArray([['hello world  ', 'pp\x00          ']])

Thanks for the hint,

A Dimecres 10 Setembre 2003 14:40, Todd Miller va escriure:
> Take a look at RawCharArray instead of CharArray.  CharArray is a
> RawCharArray with specific stripping and padding behaviors and some
> extra methods.  RawCharArray should be the starting point for this kind
> of discussion;  it may do what you want as-is, or you can tell me where
> it's falling short of what you need.
>
> Todd

-- 
Francesc Alted




From Chris.Barker at noaa.gov  Thu Sep 11 10:16:07 2003
From: Chris.Barker at noaa.gov (Chris Barker)
Date: Thu Sep 11 10:16:07 2003
Subject: [Numpy-discussion] Boolean array formatting poll
References: <Pine.LNX.4.21.0309101313200.8554-100000@cens.kybi> 
		<3F5F4DA7.2D640794@noaa.gov> <1063225004.3869.175.camel@halloween.stsci.edu>
Message-ID: <3F60A0F9.B9ACC707@noaa.gov>

Todd Miller wrote:

> >>> numarray.ones((10,), type=numarray.Bool)*5
> array([5, 5, 5, 5, 5, 5, 5, 5, 5, 5])
> 
> So, yes.

In that case, definately -1: if the data really are integers, that might
as well be clear from the display

-Chris

-- 
Christopher Barker, Ph.D.
Oceanographer
                                    		
NOAA/OR&R/HAZMAT         (206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115       (206) 526-6317   main reception

Chris.Barker at noaa.gov



From kern at caltech.edu  Thu Sep 11 17:55:29 2003
From: kern at caltech.edu (Robert Kern)
Date: Thu Sep 11 17:55:29 2003
Subject: [Numpy-discussion] Boolean array formatting poll
In-Reply-To: <1063138536.1704.181.camel@halloween.stsci.edu>
References: <1063138536.1704.181.camel@halloween.stsci.edu>
Message-ID: <20030912005204.GA16405@taliesen.caltech.edu>

On Tue, Sep 09, 2003 at 04:15:36PM -0400, Todd Miller wrote:
> Now that Python has a bool type, I have received a patch for numarray's
> arrayprint module which alters the formatting of Boolean arrays from:
> 
> >>>> a=numarray.arange(6, type=numarray.Bool); a[::2] = 0; a
> array([0, 1, 0, 1, 0, 1], type=Bool)
> 
> to:
> 
> >>>> a=numarray.arange(6, type=numarray.Bool); a[::2] = 0; a
> array([False, True, False, True, False, True], type=Bool)
> 
> Which format do you prefer?   Please vote for or against changing the
> format:  +1 (for)  or -1 (against).

-1

If I'm using those kinds of values in arrays, I'm more likely to be
thinking in terms of 1's and 0's not True's and False's anyways.

E.g.

frequency = float(sum(a)) / len(a)

-- 
Robert Kern
kern at caltech.edu

"In the fields of hell where the grass grows high
 Are the graves of dreams allowed to die."
  -- Richard Harter



From Jack.Jansen at cwi.nl  Mon Sep 15 01:36:05 2003
From: Jack.Jansen at cwi.nl (Jack Jansen)
Date: Mon Sep 15 01:36:05 2003
Subject: [Numpy-discussion] Re: MacPython and Numeric
In-Reply-To: <8F0C35DF-E6EA-11D7-BC87-000393C5B4C6@attglobal.net>
Message-ID: <AE63F153-E757-11D7-9BE1-0030655234CE@cwi.nl>

Folks,
I got this report on Numeric as included with MacPython (through the 
package manager) and it looks
promising. But: I'm building Numeric simply by running setup.py, so in 
order to get the added performance
Numeric's setup.py would need to be taught about veclib.

Is anyone willing to look into this?

On Sunday, September 14, 2003, at 09:35 PM, Alun Carr wrote:

> Jack,
>
> I would like to thank you for all your hard work in producing 
> MacPython 2.3 for OS X. It's superb.
>
> One point to note: I think that you may have compiled the Numeric 
> stuff from scratch. Apple supply hand-tuned BLAS, LAPACK, and FFT code 
> in their vecLib framework. Despite the name, according to Apple's 
> documentation this framework is suitable for both G3 and G4 
> processors, and it will automatically use vectorized code if it 
> detects a G4.
>
> From <http://developer.apple.com/hardware/ve/quickstart.html>:
>
>> If you are a developer who would like to make more use of AltiVec in 
>> your application, you should first explore the functionality that is 
>> available in Apple's vecLib.framework . This framework can be found 
>> in /System/Library/Frameworks on any MacOS X installation. There you 
>> will find many highly optimized routines for a diversity of tasks. 
>> There are single and double precision FFT's. Some of these are up to 
>> five times faster than FFT-W . There are also convolutions, 1D and 2D 
>> matrix operations and much, much more. MacOS X.2 (Jaguar) rounds this 
>> out with a full BLAS and LAPACK implementation in both single and 
>> double precision. These functions also work on G3 and earlier 
>> processors, meaning that you can call them safely regardless of what 
>> machine you are running on.
>
> For future releases of Numeric for MacPython, it might be worth using 
> vecLib, to (i) reduce the effort on your part, (ii) reduce the size of 
> the distribution, and (iii) to use code which has been optimized by 
> Apple for maximum performance.
>
> Anyway, thanks again for your efforts, and please keep up the good 
> work.
>
> Best regards,
>
> Alun
>
> ==
>
> Dr Alun J. Carr                Phone: +353-(0)1-716 1989 (Office)
> Mechanical Engineering Dept           +353-(0)1-716 1752 (BRC)
> University College Dublin      Fax:   +353-(0)1-283 0534
> Belfield                       Email: alun.j.carr at ucd.ie (UCD)
> Dublin 4                              carraj at attglobal.net (Home)
> Ireland
>
--
Jack Jansen, <Jack.Jansen at cwi.nl>, http://www.cwi.nl/~jack
If I can't dance I don't want to be part of your revolution -- Emma 
Goldman




From ariciputi at pito.com  Mon Sep 15 03:24:03 2003
From: ariciputi at pito.com (Andrea Riciputi)
Date: Mon Sep 15 03:24:03 2003
Subject: [Numpy-discussion] Re: MacPython and Numeric
In-Reply-To: <AE63F153-E757-11D7-9BE1-0030655234CE@cwi.nl>
Message-ID: <B3749F40-E766-11D7-B4A6-000393933E4E@pito.com>

Fink project (see <http://fink.sourceforge.net/>) offers Numeric  
already linked against Apple's vecLib. Here is the output about that  
package:

> [ibook:~/Installs/IPython] andrea% fink describe numeric-py22
> Information about 2973 packages read in 5 seconds.
>
>
> numeric-py22-23.0-1: Multidim'l numerical arrays for Python, using  
> Apple vecLib
>   Links against optimized Apple vecLib BLAS/LAPACK, instead of
>   using slower, built-in blas and lapack routines.
>   Should now be just as fast as numeric-atlas.
>  .
>  Web site: http://www.pfdubois.com/numpy
>  .
>  Maintainer: Jeffrey Whitaker <jsw at cdc.noaa.gov>

I think you can ask the maintainer for more information or look at the  
Fink's installation script if you know how debs system works. Here it  
is a link to the scripts:

<http://cvs.sourceforge.net/cgi-bin/viewcvs.cgi/fink/dists/10.2/ 
unstable/main/finkinfo/sci/numeric-py22-23.0-1.info>
<http://cvs.sourceforge.net/cgi-bin/viewcvs.cgi/fink/dists/10.2/ 
unstable/main/finkinfo/sci/numeric-py22-23.0-1.patch>

Hope this help.

Cheers,
	Andrea.

On Monday, Sep 15, 2003, at 10:36 Europe/Rome, Jack Jansen wrote:

> Folks,
> I got this report on Numeric as included with MacPython (through the  
> package manager) and it looks
> promising. But: I'm building Numeric simply by running setup.py, so in  
> order to get the added performance
> Numeric's setup.py would need to be taught about veclib.
>
> Is anyone willing to look into this?
>
---
Andrea Riciputi

"Science is like sex: sometimes something useful comes out,
   but that is not the reason we are doing it" -- (Richard Feynman)




From nadavh at VisionSense.com  Wed Sep 17 03:05:08 2003
From: nadavh at VisionSense.com (Nadav Horesh)
Date: Wed Sep 17 03:05:08 2003
Subject: [Numpy-discussion] numarray latest version
Message-ID: <1063796708.8864.32.camel@Nadav.Envision.co.il>

How comes that numarray 0.6.2 succeeds 0.7? Are there 2 active branches?

  Nadav.




From jmiller at stsci.edu  Wed Sep 17 05:54:09 2003
From: jmiller at stsci.edu (Todd Miller)
Date: Wed Sep 17 05:54:09 2003
Subject: [Numpy-discussion] numarray latest version
In-Reply-To: <1063796708.8864.32.camel@Nadav.Envision.co.il>
References: <1063796708.8864.32.camel@Nadav.Envision.co.il>
Message-ID: <1063803173.21121.15.camel@halloween.stsci.edu>

On Wed, 2003-09-17 at 07:05, Nadav Horesh wrote:
> How comes that numarray 0.6.2 succeeds 0.7? 

numarray-0.6.2 is numarray-0.6.1 which makes the licensing clearer at
runtime by including it in __init__.py for numarray.  That's it.  No new
(real) code, just licensing,  which is basically the modified BSD
license.    It's not really intended for use outside the astronomical
community,  so I didn't announce it.

numarray-0.6.2 is being used by STSCI in a larger system, and it has
been tested in that context.  

numarray-0.7 is newer, better, and with several bug fixes,  but not the
same level of testing.

It is unfortunate that numarray-0.6.2 is listed first on Source Forge, 
but as far as I know I can't control that.

> Are there 2 active branches?

Not for long.

Todd

>   Nadav.
> 
> 
> 
> -------------------------------------------------------
> This sf.net email is sponsored by:ThinkGeek
> Welcome to geek heaven.
> http://thinkgeek.com/sf
> _______________________________________________
> Numpy-discussion mailing list
> Numpy-discussion at lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/numpy-discussion
-- 
Todd Miller 			jmiller at stsci.edu
STSCI / ESS / SSB




From anewgene at hotpop.com  Wed Sep 17 13:03:03 2003
From: anewgene at hotpop.com (CL WU)
Date: Wed Sep 17 13:03:03 2003
Subject: [Numpy-discussion] array sort questions
Message-ID: <3F68BE05.1010104@hotpop.com>

Hi, group,
          I am new to numpy. I have 2 questions for array sort.

1. How to sort an array by its one column or one row?
    I know python build-in sort() can do it for list by passing own cmp 
function. but array function sort() will sort each column or row 
seperately,as I know. I don't want to convert array to list to sort and 
then convert back to array.

2. How to get the rank of a rank-0 array? The first "rank" means the 
order of each element after sorting, instead of the "dimension" meaning 
in numpy.     Just like "rank()" function in splus.

Thank you

Chunlei





From tim.hochberg at ieee.org  Wed Sep 17 13:19:21 2003
From: tim.hochberg at ieee.org (Tim Hochberg)
Date: Wed Sep 17 13:19:21 2003
Subject: [Numpy-discussion] array sort questions
In-Reply-To: <3F68BE05.1010104@hotpop.com>
References: <3F68BE05.1010104@hotpop.com>
Message-ID: <3F68C190.1080808@ieee.org>

CL WU wrote:

> Hi, group,
>          I am new to numpy. I have 2 questions for array sort.
>
> 1. How to sort an array by its one column or one row?
>    I know python build-in sort() can do it for list by passing own cmp 
> function. but array function sort() will sort each column or row 
> seperately,as I know. I don't want to convert array to list to sort 
> and then convert back to array.

I think you want argsort plus take. For example, the following sorts on 
the second column of a:

     a = array([[4,5,6], [1,2,3], [7,8,9]])
     arg = argsort(a[:,1])
     take(a, arg, 0)

> 2. How to get the rank of a rank-0 array? The first "rank" means the 
> order of each element after sorting, instead of the "dimension" 
> meaning in numpy.     Just like "rank()" function in splus.

If  I understand you correctly, you want argsort as mentioned above.

Regards,

-tim


>
> Thank you
>
> Chunlei
>
>
>
>
> -------------------------------------------------------
> This sf.net email is sponsored by:ThinkGeek
> Welcome to geek heaven.
> http://thinkgeek.com/sf
> _______________________________________________
> Numpy-discussion mailing list
> Numpy-discussion at lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/numpy-discussion
>






From anewgene at hotpop.com  Wed Sep 17 13:55:13 2003
From: anewgene at hotpop.com (CL WU)
Date: Wed Sep 17 13:55:13 2003
Subject: [Numpy-discussion] array sort questions
In-Reply-To: <3F68C190.1080808@ieee.org>
References: <3F68BE05.1010104@hotpop.com> <3F68C190.1080808@ieee.org>
Message-ID: <3F68CA44.4080408@hotpop.com>

Thank you, Tim.  argsort() and take() does provide a easy way to sort an 
array based on any col or row. But for the second question, it doesn't 
return the result I want.
As below, softrank or softrank1  are functions I am currently using for 
get the rank of a vector(first is more efficient). It returns the index 
of each value from original array/list in sorted array/list.
I hope there is an efficient function in array level to do the same work.
 >>> from Numeric import *
 >>> a=array([5,2,3])
 >>> argsort(a)
array([1, 2, 0])
 >>> def sortrank(list):
...     n=len(list)
...     li_a=[(i,list[i]) for i in range(n)]
...     li_a.sort(lambda a,b:cmp(a[1],b[1]))
...     li_b=[(i,li_a[i]) for i in range(n)]
...     li_b.sort(lambda a,b:cmp(a[1][0],b[1][0]))
...     return [x[0] for x in li_b]
...    
 >>> sortrank(a)
[2, 0, 1]
 >>> def sortrank2(li):
...     li_sorted=li[:]
...     li_sorted.sort()
...     return [li_sorted.index(x) for x in li]
 >>> sortrank1(list(a))
[2, 0, 1]


Thanks again.

Chunlei

Tim Hochberg wrote:

> CL WU wrote:
>
>> Hi, group,
>>          I am new to numpy. I have 2 questions for array sort.
>>
>> 1. How to sort an array by its one column or one row?
>>    I know python build-in sort() can do it for list by passing own 
>> cmp function. but array function sort() will sort each column or row 
>> seperately,as I know. I don't want to convert array to list to sort 
>> and then convert back to array.
>
>
> I think you want argsort plus take. For example, the following sorts 
> on the second column of a:
>
>     a = array([[4,5,6], [1,2,3], [7,8,9]])
>     arg = argsort(a[:,1])
>     take(a, arg, 0)
>
>> 2. How to get the rank of a rank-0 array? The first "rank" means the 
>> order of each element after sorting, instead of the "dimension" 
>> meaning in numpy.     Just like "rank()" function in splus.
>
>
> If  I understand you correctly, you want argsort as mentioned above.
>
> Regards,
>
> -tim
>
>
>>
>> Thank you
>>
>> Chunlei
>>
>>
>>
>>
>> -------------------------------------------------------
>> This sf.net email is sponsored by:ThinkGeek
>> Welcome to geek heaven.
>> http://thinkgeek.com/sf
>> _______________________________________________
>> Numpy-discussion mailing list
>> Numpy-discussion at lists.sourceforge.net
>> https://lists.sourceforge.net/lists/listinfo/numpy-discussion
>>
>
>
>
>
>
> -------------------------------------------------------
> This sf.net email is sponsored by:ThinkGeek
> Welcome to geek heaven.
> http://thinkgeek.com/sf
> _______________________________________________
> Numpy-discussion mailing list
> Numpy-discussion at lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/numpy-discussion
>





From anewgene at hotpop.com  Wed Sep 17 15:28:06 2003
From: anewgene at hotpop.com (CL WU)
Date: Wed Sep 17 15:28:06 2003
Subject: [Numpy-discussion] array sort questions
In-Reply-To: <3F68CFA5.4080206@ieee.org>
References: <3F68BE05.1010104@hotpop.com> <3F68C190.1080808@ieee.org> <3F68CA44.4080408@hotpop.com> <3F68CFA5.4080206@ieee.org>
Message-ID: <3F68E025.90106@hotpop.com>

Great! It works much more efficiently. Thank you so much.

Best,

Chunlei

Tim Hochberg wrote:

> CL WU wrote:
>
>> Thank you, Tim.  argsort() and take() does provide a easy way to sort 
>> an array based on any col or row. But for the second question, it 
>> doesn't return the result I want.
>> As below, softrank or softrank1  are functions I am currently using 
>> for get the rank of a vector(first is more efficient). It returns the 
>> index of each value from original array/list in sorted array/list.
>
>
> Hmmm. It seems that argsort and sortrank are inverses of a sort, so it 
> should be possible to do what you want efficiently, but I'm not sure how.
>
> <think>
>
> Ah, it appears to be quite simple. I believe:
>
> argsort(argsort(a))
>
> is equivalent to your sortrank and should be much faster.
>
> regards,
>
> -tim
>
>
>> I hope there is an efficient function in array level to do the same 
>> work.
>> >>> from Numeric import *
>> >>> a=array([5,2,3])
>> >>> argsort(a)
>> array([1, 2, 0])
>> >>> def sortrank(list):
>> ...     n=len(list)
>> ...     li_a=[(i,list[i]) for i in range(n)]
>> ...     li_a.sort(lambda a,b:cmp(a[1],b[1]))
>> ...     li_b=[(i,li_a[i]) for i in range(n)]
>> ...     li_b.sort(lambda a,b:cmp(a[1][0],b[1][0]))
>> ...     return [x[0] for x in li_b]
>> ...    >>> sortrank(a)
>> [2, 0, 1]
>> >>> def sortrank2(li):
>> ...     li_sorted=li[:]
>> ...     li_sorted.sort()
>> ...     return [li_sorted.index(x) for x in li]
>> >>> sortrank1(list(a))
>> [2, 0, 1]
>>
>>
>> Thanks again.
>>
>> Chunlei
>>
>> Tim Hochberg wrote:
>>
>>> CL WU wrote:
>>>
>>>> Hi, group,
>>>>          I am new to numpy. I have 2 questions for array sort.
>>>>
>>>> 1. How to sort an array by its one column or one row?
>>>>    I know python build-in sort() can do it for list by passing own 
>>>> cmp function. but array function sort() will sort each column or 
>>>> row seperately,as I know. I don't want to convert array to list to 
>>>> sort and then convert back to array.
>>>
>>>
>>>
>>>
>>> I think you want argsort plus take. For example, the following sorts 
>>> on the second column of a:
>>>
>>>     a = array([[4,5,6], [1,2,3], [7,8,9]])
>>>     arg = argsort(a[:,1])
>>>     take(a, arg, 0)
>>>
>>>> 2. How to get the rank of a rank-0 array? The first "rank" means 
>>>> the order of each element after sorting, instead of the "dimension" 
>>>> meaning in numpy.     Just like "rank()" function in splus.
>>>
>>>
>>>
>>>
>>> If  I understand you correctly, you want argsort as mentioned above.
>>>
>>> Regards,
>>>
>>> -tim
>>>
>>>
>>>>
>>>> Thank you
>>>>
>>>> Chunlei
>>>>
>>>>
>>>>
>>>>
>>>> -------------------------------------------------------
>>>> This sf.net email is sponsored by:ThinkGeek
>>>> Welcome to geek heaven.
>>>> http://thinkgeek.com/sf
>>>> _______________________________________________
>>>> Numpy-discussion mailing list
>>>> Numpy-discussion at lists.sourceforge.net
>>>> https://lists.sourceforge.net/lists/listinfo/numpy-discussion
>>>>
>>>
>>>
>>>
>>>
>>>
>>> -------------------------------------------------------
>>> This sf.net email is sponsored by:ThinkGeek
>>> Welcome to geek heaven.
>>> http://thinkgeek.com/sf
>>> _______________________________________________
>>> Numpy-discussion mailing list
>>> Numpy-discussion at lists.sourceforge.net
>>> https://lists.sourceforge.net/lists/listinfo/numpy-discussion
>>>
>>
>>
>>
>>
>> -------------------------------------------------------
>> This sf.net email is sponsored by:ThinkGeek
>> Welcome to geek heaven.
>> http://thinkgeek.com/sf
>> _______________________________________________
>> Numpy-discussion mailing list
>> Numpy-discussion at lists.sourceforge.net
>> https://lists.sourceforge.net/lists/listinfo/numpy-discussion
>>
>
>
>
>





From tim.hochberg at ieee.org  Thu Sep 18 10:54:18 2003
From: tim.hochberg at ieee.org (Tim Hochberg)
Date: Thu Sep 18 10:54:18 2003
Subject: [Numpy-discussion] Numarray - Numeric incompatibility
In-Reply-To: <3F68E025.90106@hotpop.com>
References: <3F68BE05.1010104@hotpop.com> <3F68C190.1080808@ieee.org> <3F68CA44.4080408@hotpop.com> <3F68CFA5.4080206@ieee.org> <3F68E025.90106@hotpop.com>
Message-ID: <3F69F0FC.50404@ieee.org>

I'm just starting to move some of my code over to numarray and I was 
dismayed to find that basic operation between Numeric and numarray 
arrays fail.

 >>> import Numeric as np
 >>> import numarray as na
 >>> a = na.arange(5)
 >>> p = np.arange(5)
 >>> a + p
['vector', 'vector']
Traceback (most recent call last):
  File "<stdin>", line 1, in ?
  File "C:\Python23\Lib\site-packages\numarray\numarraycore.py", line 
648, in __add__
    def __add__(self, operand): return ufunc.add(self, operand)
  File "C:\Python23\lib\site-packages\numarray\ufunc.py", line 818, in 
_cache_miss2
    key = (_digest(n1), _digest(n2), _digest(out), safethread.get_ident())
KeyError: '_digest force cache miss'

I suspect (hope!) that this is just a bug and not something inherent in 
numarray. I dug around in unfunc.py a bit and it appears that the bug is 
shallow and can be fixed simply by replacing::

         if not (_sequence(n1) or _sequence(n2)):
            key = (_digest(n1), _digest(n2), _digest(out), 
safethread.get_ident())
            self._cache[ key ] = cached

with::

        try:
            key = (_digest(n1), _digest(n2), _digest(out), 
safethread.get_ident())
        except KeyError:
            pass
        else:
            self._cache[ key ] = cached

in _cache_miss2 and _cache_miss1. If this were done, _sequence could 
probably be deleted as well.

I'm not very familiar with the numarray code yet, so it's quite possible 
I'm missing something, but I'm willing to do more digging to fix this if 
this turns out to not be sufficient.

Regards,

-tim





From jmiller at stsci.edu  Thu Sep 18 11:09:39 2003
From: jmiller at stsci.edu (Todd Miller)
Date: Thu Sep 18 11:09:39 2003
Subject: [Numpy-discussion] Numarray - Numeric incompatibility
In-Reply-To: <3F69F0FC.50404@ieee.org>
References: <3F68BE05.1010104@hotpop.com> <3F68C190.1080808@ieee.org>
	<3F68CA44.4080408@hotpop.com> <3F68CFA5.4080206@ieee.org>
	<3F68E025.90106@hotpop.com>  <3F69F0FC.50404@ieee.org>
Message-ID: <1063908412.28142.10.camel@halloween.stsci.edu>

On Thu, 2003-09-18 at 13:53, Tim Hochberg wrote:
> 
> I'm just starting to move some of my code over to numarray and I was 
> dismayed to find that basic operation between Numeric and numarray 
> arrays fail.
> 
>  >>> import Numeric as np
>  >>> import numarray as na
>  >>> a = na.arange(5)
>  >>> p = np.arange(5)
>  >>> a + p
> ['vector', 'vector']
> Traceback (most recent call last):
>   File "<stdin>", line 1, in ?
>   File "C:\Python23\Lib\site-packages\numarray\numarraycore.py", line 
> 648, in __add__
>     def __add__(self, operand): return ufunc.add(self, operand)
>   File "C:\Python23\lib\site-packages\numarray\ufunc.py", line 818, in 
> _cache_miss2
>     key = (_digest(n1), _digest(n2), _digest(out), safethread.get_ident())
> KeyError: '_digest force cache miss'
> 
> I suspect (hope!) that this is just a bug and not something inherent in 
> numarray. 

It's an interoperability issue.  Please let us know if you find others.

> I dug around in unfunc.py a bit and it appears that the bug is 
> shallow and can be fixed simply by replacing::
> 
>          if not (_sequence(n1) or _sequence(n2)):
>             key = (_digest(n1), _digest(n2), _digest(out), 
> safethread.get_ident())
>             self._cache[ key ] = cached
> 
> with::
> 
>         try:
>             key = (_digest(n1), _digest(n2), _digest(out), 
> safethread.get_ident())
>         except KeyError:
>             pass
>         else:
>             self._cache[ key ] = cached
> 
> in _cache_miss2 and _cache_miss1. If this were done, _sequence could 
> probably be deleted as well.
> 
> I'm not very familiar with the numarray code yet, so it's quite possible 
> I'm missing something, but I'm willing to do more digging to fix this if 
> this turns out to not be sufficient.
> 

I ran into the same problem trying to port MA to numarray, and came up
with an identical work around.  

A fix like this will be part of numarray-0.8.

Todd
-- 
Todd Miller 			jmiller at stsci.edu
STSCI / ESS / SSB




From tim.hochberg at ieee.org  Thu Sep 18 11:35:10 2003
From: tim.hochberg at ieee.org (Tim Hochberg)
Date: Thu Sep 18 11:35:10 2003
Subject: [Numpy-discussion] array sort questions
In-Reply-To: <3F68E025.90106@hotpop.com>
References: <3F68BE05.1010104@hotpop.com> <3F68C190.1080808@ieee.org> <3F68CA44.4080408@hotpop.com> <3F68CFA5.4080206@ieee.org> <3F68E025.90106@hotpop.com>
Message-ID: <3F69FA4C.1020504@ieee.org>

Hi Chunlei,

I just realized one other thing that you should probably be aware of. 
You could write a much faster version of sortrank in pure python by 
doing your sorts differently. Python's built in sort is very fast, but 
as soon as you start passing in comparison functions it slows down 
dramatically. The trick is to arange the data you need to sort so that 
you don't need an auxilliary function  (know asDecorate-Sort-Undecorate 
or the Schwartzian transform). Thus, the following is almost certainly a 
lot faster than your original sortrank, although probably still slower 
than the argsort solution.

def sortrank(list):
    index = range(len(list))
    li_a = zip(list, index)
    li_a.sort()
    li_b = [(li_a[i][1], i) for i in index]
    li_b.sort()
    return [x[1] for x in li_b]

Regards,

-tim






CL WU wrote:

>>> I hope there is an efficient function in array level to do the same 
>>> work.
>>> >>> from Numeric import *
>>> >>> a=array([5,2,3])
>>> >>> argsort(a)
>>> array([1, 2, 0])
>>> >>> def sortrank(list):
>>> ...     n=len(list)
>>> ...     li_a=[(i,list[i]) for i in range(n)]
>>> ...     li_a.sort(lambda a,b:cmp(a[1],b[1]))
>>> ...     li_b=[(i,li_a[i]) for i in range(n)]
>>> ...     li_b.sort(lambda a,b:cmp(a[1][0],b[1][0]))
>>> ...     return [x[0] for x in li_b]
>>> ...    >>> sortrank(a)
>>> [2, 0, 1]
>>> >>> def sortrank2(li):
>>> ...     li_sorted=li[:]
>>> ...     li_sorted.sort()
>>> ...     return [li_sorted.index(x) for x in li]
>>> >>> sortrank1(list(a))
>>> [2, 0, 1]
>>>
>>>
>>> Thanks again.
>>>
>>> Chunlei
>>>
>>> Tim Hochberg wrote:
>>>
>>>> CL WU wrote:
>>>>
>>>>> Hi, group,
>>>>>          I am new to numpy. I have 2 questions for array sort.
>>>>>
>>>>> 1. How to sort an array by its one column or one row?
>>>>>    I know python build-in sort() can do it for list by passing own 
>>>>> cmp function. but array function sort() will sort each column or 
>>>>> row seperately,as I know. I don't want to convert array to list to 
>>>>> sort and then convert back to array.
>>>>
>>>>
>>>>
>>>>
>>>>
>>>> I think you want argsort plus take. For example, the following 
>>>> sorts on the second column of a:
>>>>
>>>>     a = array([[4,5,6], [1,2,3], [7,8,9]])
>>>>     arg = argsort(a[:,1])
>>>>     take(a, arg, 0)
>>>>
>>>>> 2. How to get the rank of a rank-0 array? The first "rank" means 
>>>>> the order of each element after sorting, instead of the 
>>>>> "dimension" meaning in numpy.     Just like "rank()" function in 
>>>>> splus.
>>>>
>>>>
>>>>
>>>>
>>>>
>>>> If  I understand you correctly, you want argsort as mentioned above.
>>>>
>>>> Regards,
>>>>
>>>> -tim
>>>>
>>>>
>>>>>
>>>>> Thank you
>>>>>
>>>>> Chunlei
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> -------------------------------------------------------
>>>>> This sf.net email is sponsored by:ThinkGeek
>>>>> Welcome to geek heaven.
>>>>> http://thinkgeek.com/sf
>>>>> _______________________________________________
>>>>> Numpy-discussion mailing list
>>>>> Numpy-discussion at lists.sourceforge.net
>>>>> https://lists.sourceforge.net/lists/listinfo/numpy-discussion
>>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>> -------------------------------------------------------
>>>> This sf.net email is sponsored by:ThinkGeek
>>>> Welcome to geek heaven.
>>>> http://thinkgeek.com/sf
>>>> _______________________________________________
>>>> Numpy-discussion mailing list
>>>> Numpy-discussion at lists.sourceforge.net
>>>> https://lists.sourceforge.net/lists/listinfo/numpy-discussion
>>>>
>>>
>>>
>>>
>>>
>>> -------------------------------------------------------
>>> This sf.net email is sponsored by:ThinkGeek
>>> Welcome to geek heaven.
>>> http://thinkgeek.com/sf
>>> _______________________________________________
>>> Numpy-discussion mailing list
>>> Numpy-discussion at lists.sourceforge.net
>>> https://lists.sourceforge.net/lists/listinfo/numpy-discussion
>>>
>>
>>
>>
>>
>
>
>






From anewgene at hotpop.com  Thu Sep 18 14:04:44 2003
From: anewgene at hotpop.com (CL WU)
Date: Thu Sep 18 14:04:44 2003
Subject: [Numpy-discussion] array sort questions
In-Reply-To: <3F69FA4C.1020504@ieee.org>
References: <3F68BE05.1010104@hotpop.com> <3F68C190.1080808@ieee.org> <3F68CA44.4080408@hotpop.com> <3F68CFA5.4080206@ieee.org> <3F68E025.90106@hotpop.com> <3F69FA4C.1020504@ieee.org>
Message-ID: <3F6A1D38.1050809@hotpop.com>

Thanks again, Tim. It a wonderful example to show how efficient python 
can run if it is well written.

Best,

Chunlei

Tim Hochberg wrote:

> Hi Chunlei,
>
> I just realized one other thing that you should probably be aware of. 
> You could write a much faster version of sortrank in pure python by 
> doing your sorts differently. Python's built in sort is very fast, but 
> as soon as you start passing in comparison functions it slows down 
> dramatically. The trick is to arange the data you need to sort so that 
> you don't need an auxilliary function  (know 
> asDecorate-Sort-Undecorate or the Schwartzian transform). Thus, the 
> following is almost certainly a lot faster than your original 
> sortrank, although probably still slower than the argsort solution.
>
> def sortrank(list):
>    index = range(len(list))
>    li_a = zip(list, index)
>    li_a.sort()
>    li_b = [(li_a[i][1], i) for i in index]
>    li_b.sort()
>    return [x[1] for x in li_b]
>
> Regards,
>
> -tim
>
>
>
>
>
>
> CL WU wrote:
>
>>>> I hope there is an efficient function in array level to do the same 
>>>> work.
>>>> >>> from Numeric import *
>>>> >>> a=array([5,2,3])
>>>> >>> argsort(a)
>>>> array([1, 2, 0])
>>>> >>> def sortrank(list):
>>>> ...     n=len(list)
>>>> ...     li_a=[(i,list[i]) for i in range(n)]
>>>> ...     li_a.sort(lambda a,b:cmp(a[1],b[1]))
>>>> ...     li_b=[(i,li_a[i]) for i in range(n)]
>>>> ...     li_b.sort(lambda a,b:cmp(a[1][0],b[1][0]))
>>>> ...     return [x[0] for x in li_b]
>>>> ...    >>> sortrank(a)
>>>> [2, 0, 1]
>>>> >>> def sortrank2(li):
>>>> ...     li_sorted=li[:]
>>>> ...     li_sorted.sort()
>>>> ...     return [li_sorted.index(x) for x in li]
>>>> >>> sortrank1(list(a))
>>>> [2, 0, 1]
>>>>
>>>>
>>>> Thanks again.
>>>>
>>>> Chunlei
>>>>
>>>> Tim Hochberg wrote:
>>>>
>>>>> CL WU wrote:
>>>>>
>>>>>> Hi, group,
>>>>>>          I am new to numpy. I have 2 questions for array sort.
>>>>>>
>>>>>> 1. How to sort an array by its one column or one row?
>>>>>>    I know python build-in sort() can do it for list by passing 
>>>>>> own cmp function. but array function sort() will sort each column 
>>>>>> or row seperately,as I know. I don't want to convert array to 
>>>>>> list to sort and then convert back to array.
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> I think you want argsort plus take. For example, the following 
>>>>> sorts on the second column of a:
>>>>>
>>>>>     a = array([[4,5,6], [1,2,3], [7,8,9]])
>>>>>     arg = argsort(a[:,1])
>>>>>     take(a, arg, 0)
>>>>>
>>>>>> 2. How to get the rank of a rank-0 array? The first "rank" means 
>>>>>> the order of each element after sorting, instead of the 
>>>>>> "dimension" meaning in numpy.     Just like "rank()" function in 
>>>>>> splus.
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> If  I understand you correctly, you want argsort as mentioned above.
>>>>>
>>>>> Regards,
>>>>>
>>>>> -tim
>>>>>
>>>>>
>>>>>>
>>>>>> Thank you
>>>>>>
>>>>>> Chunlei
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> -------------------------------------------------------
>>>>>> This sf.net email is sponsored by:ThinkGeek
>>>>>> Welcome to geek heaven.
>>>>>> http://thinkgeek.com/sf
>>>>>> _______________________________________________
>>>>>> Numpy-discussion mailing list
>>>>>> Numpy-discussion at lists.sourceforge.net
>>>>>> https://lists.sourceforge.net/lists/listinfo/numpy-discussion
>>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> -------------------------------------------------------
>>>>> This sf.net email is sponsored by:ThinkGeek
>>>>> Welcome to geek heaven.
>>>>> http://thinkgeek.com/sf
>>>>> _______________________________________________
>>>>> Numpy-discussion mailing list
>>>>> Numpy-discussion at lists.sourceforge.net
>>>>> https://lists.sourceforge.net/lists/listinfo/numpy-discussion
>>>>>
>>>>
>>>>
>>>>
>>>>
>>>> -------------------------------------------------------
>>>> This sf.net email is sponsored by:ThinkGeek
>>>> Welcome to geek heaven.
>>>> http://thinkgeek.com/sf
>>>> _______________________________________________
>>>> Numpy-discussion mailing list
>>>> Numpy-discussion at lists.sourceforge.net
>>>> https://lists.sourceforge.net/lists/listinfo/numpy-discussion
>>>>
>>>
>>>
>>>
>>>
>>
>>
>>
>
>
>
>





From tim.hochberg at ieee.org  Thu Sep 18 16:20:02 2003
From: tim.hochberg at ieee.org (Tim Hochberg)
Date: Thu Sep 18 16:20:02 2003
Subject: [Numpy-discussion] numarray unfriendly to user defined types
Message-ID: <3F6A3D4C.3070703@ieee.org>


I've run into another issue in my attempt to transition over to 
numarray: it's less friendly to user defined types than Numeric. I think 
this is mainly accidental friednliness on Numeric's part, but it's handy 
nonetheless. The attached file illustrates the issue. Given some object, 
in this case *zero*, that a numarray array does not now how to handle, 
it ends up raising an exception instead of giving the object a chance to 
try. Thus ``zero + numarray.arange(5)`` works while ``numarray.arange(5) 
+ zero`` fails, since in the first case Zero.__add__ is called first, 
while in the second NumArray.__add__ is called first.

This should probably be fixed, but it's not clear what the best way is. 
My first thought was to always give the other object a chance to use 
__rop__ first. This is simple and easy to explain, but fails miserably 
in the common case of multiplying a list and an array. Just to be 
concrete, __mul__ would be replaced by::

    def __mul__(self, operand):
        try:
            return operand.__rmul__(self)
        except:
            return ufunc.multiply(self, operand)

Next thought is to catch exceptions when they occur in numarray and then 
give the other operand a chance::

    def __mul__(self, operand):
        try:
           return ufunc.multiply(self, operand)
        except:
            return operand.__rmul__(self)

This appears like it would fix my particular problem, but still is not 
ideal. Since numarray is the base of libraries that it will know nothing 
about, it should defer to classes it doesn't know about whenever 
possible. Otherewise it's not possible (or maybe just hard)  to create  
new classes that try to be clever, but still interact with numarray in a 
reasonable way. In my case I'm thinking of proxy objects that don't do 
some computations till they are actually required. So my current 
thinking is that __mul__ and friends would be best implemented as::

    def __mul__(self, operand):
        if not isinstance(operand, knownTypes):
            try:
                  return ufunc.multiply(self, operand)
            except:
                pass
        return operand.__rmul__(self)

Where knownTypes is something like (int, long, float, complex, tuple, list).

Anyway, that's my excessively long two cents on this.

-tim




-------------- next part --------------
An embedded and charset-unspecified text was scrubbed...
Name: unfriendly.py
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20030918/b12aebe9/attachment.ksh>

From ssilrich at yahoo.com  Fri Sep 19 12:37:42 2003
From: ssilrich at yahoo.com (ELECTRICIAN)
Date: Fri Sep 19 12:37:42 2003
Subject: [Numpy-discussion] R E S U M E
Message-ID: <E1A0CuK-0005li-V7@sc8-sf-mx1.sourceforge.net>

Resume from: 	Rich		for   Job    or    Service   " no job too small "

E L E C T R I C I A N


Tel. (408) 482-2102           rysio3 at yahoo.com    


WIRING & INSTALLATION

Hands on electrical installations perform fitting, mounting, laying cables on Commercial, Industrial, residential new & existing buildings.

Electrical Power Supply for Lights, Plugs, Receptacles, Panels, & Fuse boxes, Emergency Generators wiring and testing, Transformers, Power Lines & conduit layout, bending and mounting, parking lighting, lamps, switches, SOLAR PROJECTS, posts and underground installations. 

Shopping Centers; grocery stories, hardware stories, restaurants & residential - housing areas, computer business & fast food units installation & buildings; Solar Panels, Sun Tracking, Flywheel Storage & electric cars systems modify, Natural Energy in Remote areas install. 

LOW VOLTAGE           Office    Home   Yard   Patio   Parking

12 / 24 Volt audio & video equipment, Computer & data network wiring, data backup and UPS; Monitoring Video Control & backup tapes set up and mounting, electro-optical assemblies & subsystems. DC Power Supply, Switch & Motion sensors Alarm. Fire & safety systems install.

Fiber Optics systems, PLC setup, Master Control Center, cable modems & cable TV install. Network, UPS Battery Backup mounting and charging systems; Power supply testing, troubleshooting, and analyzing to a components level. Electric Vehicles Design, Assembly & Installations.

CC TV & Cameras, Security Systems & Sensors for Safety, Fire sprinklers and traffic Monitoring & Door Control. Telephones / Net move & install.

TECHNICIAN

Use lab & shop equipment, mechanical, electrical & electronic tools, measurement & testing equipment,  video cameras & microscopes. Support scientists & electronic engineers. Mechanical & Electro-Mech. Design.

OFFICE, ELECTRICAL AND MECHANICAL PROJECTS

Electrical & Network Sketches, one line diagrams, and "as is" drawings update. Customizing Electronic and Electrical Components & Parts, Layouts electronic and electrical schematic, connectors and mechanical detailing. Quotes, supply, bids and job estimating. 

Customers contact, inspection, project mgmt & supervision of electricians & material handling; Use CAD, Windows and applications;

ELECTRICAL & MAINTENANCE SERVICE   US Citizen; open for travel
.



From Boostering at 2911.net  Sat Sep 20 10:09:01 2003
From: Boostering at 2911.net (Betty)
Date: Sat Sep 20 10:09:01 2003
Subject: [Numpy-discussion] Promoting Your Business
Message-ID: <E1A0lDI-0000YY-BT@sc8-sf-mx1.sourceforge.net>

Email Marketing is one of the most effective and inexpensive ways to
promote your products and services.

We offer a complete Email Marketing solution with quality service and
the lowest prices. The result is that you will enjoy more success.

1. Targeted Email Addresses 

We can supply targeted email addresses according to your requirements,
which are compiled only on your order. We will customize your customer
email addresses.

* We have millions of email addresses in a wide variety of categories.

2. Targeted Mailing  

If you are worried about any complications or consequences with sending 
out targeted emails, or want to avoid the work of sending out targeted 
emails. We will do it for you!  We can send your email message to your 
targeted customers.

* We can Bullet-Proof your Web Site.

We also offer a wide variety of marketing software. For more details,
you can refer to: www.marketingboosting.com

Our services will help you get more business opportunities.


Regards!

Betty Jones
Customer Support
www.marketingboosting.com
Sales at marketingboosting.com

************************************************************************
Receiving this email because you registered to receive special 
offers from one of our partners. If you would prefer not to receive
future email, click: Http://optin.garyshawkey.com/optin/r.php3
************************************************************************



From falted at openlc.org  Mon Sep 22 10:39:16 2003
From: falted at openlc.org (Francesc Alted)
Date: Mon Sep 22 10:39:16 2003
Subject: [Numpy-discussion] [ANN] PyTables 0.7.2 released
Message-ID: <200309221938.13790.falted@openlc.org>

Announcing PyTables 0.7.2
-------------------------

PyTables is a hierarchical database package designed to efficently
manage very large amounts of data. PyTables is built on top of the
HDF5 library and the numarray package. It features an object-oriented
interface that, combined with natural naming and C-code generated from
Pyrex sources, makes it a fast, yet extremely easy to use tool for
interactively save and retrieve large amounts of data. Besides, it
provides flexible indexed access on disk to anywhere in the data you
want to go.
 
On this release you will not find any exciting new features. It is
mainly a maintenance release where the next issues has been addressed:
       - a memory leak was fixed
       - memory consumption is being addressed and lowered
       - much faster opening of files
       - Some important index patterns cases in table reads has been
         optimized

More in detail:

What's new
-----------

        - Fixed a nasty memory leak located on the C libraries (it was
          happening during HDF5 attribute writes). After that, the
          memory consumption when using large object trees has dropped
          quite a bit. However, there remains some small leaks that
          has been tracked down to the underlying numarray
          library. These leaks has been reported, and hopefully they
          should be fixed more sooner than later.

        - Table buffers are built dinamically now, so if Tables are
	  not accessed for reading or writing this memory will not be
	  booked. This will help to reduce the memory consumption.

	- The opening of files with lots of nodes has been accelerated
	  between a factor 2 or 3. For example, a file with 10 groups
	  and 3000 tables that takes 9.3 seconds to open in 0.7.1, now
	  takes only 2.8 seconds.

	- The Table.read() method has been refactored and optimized
	  and some parts of its code has been moved to Pyrex. In
	  particular, in the special case of step=1, up to a factor 5
	  of speedup (reaching 160 MB/s on a Pentium4 @ 2 GHz) when
	  reading table contents can be achieved now.

	- Done some cosmetic changes in the user manual, but, as no
          new features has been added, you won't need to read the
          manual again :-)

What is a table?
----------------

A table is defined as a collection of records whose values are stored
in fixed-length fields. All records have the same structure and all
values in each field have the same data type.  The terms
"fixed-length" and "strict data types" seems to be quite a strange
requirement for an language like Python, that supports dynamic data
types, but they serve a useful function if the goal is to save very
large quantities of data (such as is generated by many scientific
applications, for example) in an efficient manner that reduces demand
on CPU time and I/O resources.

What is HDF5?
-------------

For those people who know nothing about HDF5, it is is a general
purpose library and file format for storing scientific data made at
NCSA. HDF5 can store two primary objects: datasets and groups. A
dataset is essentially a multidimensional array of data elements, and
a group is a structure for organizing objects in an HDF5 file. Using
these two basic constructs, one can create and store almost any kind of
scientific data structure, such as images, arrays of vectors, and
structured and unstructured grids. You can also mix and match them in
HDF5 files according to your needs.

Platforms
---------

I'm using Linux as the main development platform, but PyTables should
be easy to compile/install on other UNIX machines. This package has
also passed all the tests on a UltraSparc platform with Solaris 7 and
Solaris 8. It also compiles and passes all the tests on a SGI
Origin2000 with MIPS R12000 processors and running IRIX 6.5.

Regarding Windows platforms, PyTables has been tested with Windows
2000 and Windows XP, but it should also work with other flavors.

An example?
-----------

For online code examples, have a look at

http://pytables.sourceforge.net/tut/tutorial1-1.html

and 

http://pytables.sourceforge.net/tut/tutorial1-2.html

Web site
--------

Go to the PyTables web site for more details:

http://pytables.sourceforge.net/

Share your experience
---------------------

Let me know of any bugs, suggestions, gripes, kudos, etc. you may
have.

Have fun!

-- Francesc Alted




From jmiller at stsci.edu  Mon Sep 22 14:33:03 2003
From: jmiller at stsci.edu (Todd Miller)
Date: Mon Sep 22 14:33:03 2003
Subject: [Numpy-discussion] numarray unfriendly to user defined types
In-Reply-To: <3F6A3D4C.3070703@ieee.org>
References: <3F6A3D4C.3070703@ieee.org>
Message-ID: <1064266268.9234.351.camel@halloween.stsci.edu>

On Thu, 2003-09-18 at 19:18, Tim Hochberg wrote:
> 
Sorry for the delay;  I was pretty much shut down by hurricane Isabelle
from the end of last week.

I hit and bypassed this issue trying to port MA.

> I've run into another issue in my attempt to transition over to 
> numarray: it's less friendly to user defined types than Numeric. I think 
> this is mainly accidental friednliness on Numeric's part, but it's handy 
> nonetheless. The attached file illustrates the issue. Given some object, 
> in this case *zero*, that a numarray array does not now how to handle, 
> it ends up raising an exception instead of giving the object a chance to 
> try. Thus ``zero + numarray.arange(5)`` works while ``numarray.arange(5) 
> + zero`` fails, since in the first case Zero.__add__ is called first, 
> while in the second NumArray.__add__ is called first.
> 
> This should probably be fixed, but it's not clear what the best way is. 
> My first thought was to always give the other object a chance to use 
> __rop__ first. This is simple and easy to explain, but fails miserably 
> in the common case of multiplying a list and an array. Just to be 
> concrete, __mul__ would be replaced by::
> 
>     def __mul__(self, operand):
>         try:
>             return operand.__rmul__(self)
>         except:
>             return ufunc.multiply(self, operand)
> 
> Next thought is to catch exceptions when they occur in numarray and then 
> give the other operand a chance::
> 
>     def __mul__(self, operand):
>         try:
>            return ufunc.multiply(self, operand)
>         except:
>             return operand.__rmul__(self)
> 
> This appears like it would fix my particular problem, but still is not 
> ideal. Since numarray is the base of libraries that it will know nothing 
> about, it should defer to classes it doesn't know about whenever 
> possible. 

That does sound like the right heuristic and is echoed (somewhat) in the
core language. 

> Otherewise it's not possible (or maybe just hard)  to create  
> new classes that try to be clever, but still interact with numarray in a 
> reasonable way. 

Looking in Python's Objects/abstract.c, the function binop1 shows one
way to squirm around this issue:  subclass from NumArray.   It would be
interesting to know how Numeric does it.  

> In my case I'm thinking of proxy objects that don't do 
> some computations till they are actually required. So my current 
> thinking is that __mul__ and friends would be best implemented as::
> 
>     def __mul__(self, operand):
>         if not isinstance(operand, knownTypes):

If the "not" belongs there, I'm lost.

>             try:
>                   return ufunc.multiply(self, operand)
>             except:
>                 pass
>         return operand.__rmul__(self)
> 
> Where knownTypes is something like (int, long, float, complex, tuple, list).
> 
> Anyway, that's my excessively long two cents on this.
> 
I agree with your heuristic,  but I am leery of putting an unqualified
try/except in front of the numarray ufunc code.  On the other hand,
qualifying it seems too complicated.  

What about adding a tuple of types to be deferred to?

	def __mul__(self, operand):
		if isinstance(operand, _numarray_deferred_types):
			operand.__rmul__(self)
		else:
			self.__mul__(operand)

Then other libraries could register classes with something like:

numarray.defer_to(my_class)

which is slightly painful, but avoids masking the ufunc exceptions and
only needs to be done once for each library base class.  How does that
sound?

Todd

> -tim
> 
> 
> 
> 
> ----
> 

> import numarray as na
> import Numeric as np
> 
> class Zero:
>     def __add__(self, other):
>         return other
>     __radd__ = __add__
> zero = Zero()
>     
> #~ print zero + np.arange(5)
> #~ print np.arange(5) + zero
> #~ print zero + na.arange(5) 
> #~ print na.arange(5) + zero
> 
> a = na.arange(5)
> 
> import copy
> copy.deepcopy(a)
-- 
Todd Miller 			jmiller at stsci.edu
STSCI / ESS / SSB




From tim.hochberg at ieee.org  Mon Sep 22 15:28:11 2003
From: tim.hochberg at ieee.org (Tim Hochberg)
Date: Mon Sep 22 15:28:11 2003
Subject: [Numpy-discussion] numarray unfriendly to user defined types
In-Reply-To: <1064266268.9234.351.camel@halloween.stsci.edu>
References: <3F6A3D4C.3070703@ieee.org> <1064266268.9234.351.camel@halloween.stsci.edu>
Message-ID: <3F6F7744.80908@ieee.org>

Hi Todd,

Glad to have you back. I hope you suffered no permanant damage from the 
hurricane. Or the worm for that matter.

Todd Miller wrote:

>On Thu, 2003-09-18 at 19:18, Tim Hochberg wrote:
>  
>
>Sorry for the delay;  I was pretty much shut down by hurricane Isabelle
>from the end of last week.
>
>I hit and bypassed this issue trying to port MA.
>
>  
>
[SNIP]

>Looking in Python's Objects/abstract.c, the function binop1 shows one
>way to squirm around this issue:  subclass from NumArray.   It would be
>interesting to know how Numeric does it.  
>  
>
I think that since Numeric arrays are C classes, they always defer to 
any Python class although I'm not sure of the details. In any event, I'm 
pretty sure it's an accident.

>>In my case I'm thinking of proxy objects that don't do 
>>some computations till they are actually required. So my current 
>>thinking is that __mul__ and friends would be best implemented as::
>>
>>    def __mul__(self, operand):
>>        if not isinstance(operand, knownTypes):
>>    
>>
>
>If the "not" belongs there, I'm lost.
>  
>
It does belong there. KnownTypes (perhaps _numarray_nondeferred_types) 
would be a better term) are types that numarray knows that it should 
handle itself. Let me reverse the logic -- maybe it'll be clearer.

    def __mul__(self, operand):
        if isinstance(operand, _numarray_nondeferred_types):
           return operand.__rmul__(self)
        else:
	   try:
              return ufunc.multiply(self, operand)
           except:
              return operand.__rmul__(self)

>I agree with your heuristic,  but I am leery of putting an unqualified
>try/except in front of the numarray ufunc code.  On the other hand,
>qualifying it seems too complicated.  
>  
>
I agree on all counts.

>What about adding a tuple of types to be deferred to?
>
>	def __mul__(self, operand):
>		if isinstance(operand, _numarray_deferred_types):
>			operand.__rmul__(self)
>		else:
>			self.__mul__(operand)
>
>Then other libraries could register classes with something like:
>
>numarray.defer_to(my_class)
>
>which is slightly painful, but avoids masking the ufunc exceptions and
>only needs to be done once for each library base class.  How does that
>sound?
>  
>
I agree that using try/except is probably not worth the danger. However, 
I think I'd prefer the inverse of what you propose. That is:

	def __mul__(self, operand):
		if isinstance(operand, _numarray_nondeferred_types):
                        self.__mul__(operand)
		else:
			operand.__rmul__(self)


and of course a dont_defer_to registration function. Hopefully with a 
better name. The choice comes down to whether we defer by default or 
handle by default. I'm marginally on the side of deferring, but don't 
feel too strongly either way. Either one would be a big improvement.

Perhaps someone else out there has some profound thoughts.

-tim




>Todd
>
>  
>
>>-tim
>>
>>
>>
>>
>>----
>>
>>    
>>
>
>  
>
>>import numarray as na
>>import Numeric as np
>>
>>class Zero:
>>    def __add__(self, other):
>>        return other
>>    __radd__ = __add__
>>zero = Zero()
>>    
>>#~ print zero + np.arange(5)
>>#~ print np.arange(5) + zero
>>#~ print zero + na.arange(5) 
>>#~ print na.arange(5) + zero
>>
>>a = na.arange(5)
>>
>>import copy
>>copy.deepcopy(a)
>>    
>>






From fperez at colorado.edu  Mon Sep 22 15:46:13 2003
From: fperez at colorado.edu (Fernando Perez)
Date: Mon Sep 22 15:46:13 2003
Subject: [Numpy-discussion] numarray unfriendly to user defined types
In-Reply-To: <3F6F7744.80908@ieee.org>
References: <3F6A3D4C.3070703@ieee.org> <1064266268.9234.351.camel@halloween.stsci.edu> <3F6F7744.80908@ieee.org>
Message-ID: <3F6F7B83.3040305@colorado.edu>

Tim Hochberg wrote:

> I agree that using try/except is probably not worth the danger. However, 
> I think I'd prefer the inverse of what you propose. That is:
> 
> 	def __mul__(self, operand):
> 		if isinstance(operand, _numarray_nondeferred_types):
>                         self.__mul__(operand)
> 		else:
> 			operand.__rmul__(self)
> 
> 
> and of course a dont_defer_to registration function. Hopefully with a 
> better name. The choice comes down to whether we defer by default or 
> handle by default. I'm marginally on the side of deferring, but don't 
> feel too strongly either way. Either one would be a big improvement.
> 
> Perhaps someone else out there has some profound thoughts.

Well, certainly not profound, but here's my proverbial $0.02.  I tend to 
really dislike separate registration functions, since they reek too much of 
'programming by side-effect'.  Why not put the necessary information into a 
keyword argument to the constructor?  Once a sensible default is chosen, then 
it can be overridden with 'defers=True' (if the default was False) at __init__ 
time, for example.  Since this is a class-wide issue, perhaps it may make more 
sense to handle it via a meta-class mechanism.  But I'm not an expert on 
metaclasses, so I'll shut up there :)

I know that the above is rather vague, partly because I haven't followed the 
whole thread in detail.  But I hope that my intent, as a design idea, is clear 
(and perhaps even useful :)  While the actual implementation details 
(constructor keyword, metaclass, class-wide global like self.__defers, etc.) 
will have to be settled by the experts, I hope that an approach which doesn't 
rely on separate registration functions can be achieved.

Best regards,

Fernando.




From tim.hochberg at ieee.org  Mon Sep 22 16:28:04 2003
From: tim.hochberg at ieee.org (Tim Hochberg)
Date: Mon Sep 22 16:28:04 2003
Subject: [Numpy-discussion] numarray unfriendly to user defined types
In-Reply-To: <3F6F7B83.3040305@colorado.edu>
References: <3F6A3D4C.3070703@ieee.org> <1064266268.9234.351.camel@halloween.stsci.edu> <3F6F7744.80908@ieee.org> <3F6F7B83.3040305@colorado.edu>
Message-ID: <3F6F8546.9020500@ieee.org>

Fernando Perez wrote:

> Tim Hochberg wrote:
>
>> I agree that using try/except is probably not worth the danger. 
>> However, I think I'd prefer the inverse of what you propose. That is:
>>
>>     def __mul__(self, operand):
>>         if isinstance(operand, _numarray_nondeferred_types):
>>                         self.__mul__(operand)
>>         else:
>>             operand.__rmul__(self)
>>
>>
>> and of course a dont_defer_to registration function. Hopefully with a 
>> better name. The choice comes down to whether we defer by default or 
>> handle by default. I'm marginally on the side of deferring, but don't 
>> feel too strongly either way. Either one would be a big improvement.
>>
>> Perhaps someone else out there has some profound thoughts.
>
>
> Well, certainly not profound, but here's my proverbial $0.02.  I tend 
> to really dislike separate registration functions, since they reek too 
> much of 'programming by side-effect'.  Why not put the necessary 
> information into a keyword argument to the constructor?  Once a 
> sensible default is chosen, then it can be overridden with 
> 'defers=True' (if the default was False) at __init__ time, for 
> example.  Since this is a class-wide issue, perhaps it may make more 
> sense to handle it via a meta-class mechanism.  But I'm not an expert 
> on metaclasses, so I'll shut up there :)

I actually have no idea how you plan to make keyword arguments work 
here, perhaps you could explain that in more detail. Metaclasses are 
overkill, but a mixin, marker class could be used. That is, when 
designing a class for use with numarray, one would derive a class from a 
marker class in numarray::

   class MyArrayLikeClass(numarray.DeferToMe):
       ....

Hmmm. That's not too bad. Todd, what do you think about using this logic::

    def __mul__(self, operand):
        if isinstance(operand, DeferToMe):
            operand.__rmul__(self)
        else:
            self.__mul__(operand)


The only case where I see a potential problem is an old-style 
C-extenstion that can't be subclassed. I think that might be a case of 
YAGNI though. Avoiding registration is appealing.

-tim






From fperez at colorado.edu  Mon Sep 22 16:42:02 2003
From: fperez at colorado.edu (Fernando Perez)
Date: Mon Sep 22 16:42:02 2003
Subject: [Numpy-discussion] numarray unfriendly to user defined types
In-Reply-To: <3F6F8546.9020500@ieee.org>
References: <3F6A3D4C.3070703@ieee.org> <1064266268.9234.351.camel@halloween.stsci.edu> <3F6F7744.80908@ieee.org> <3F6F7B83.3040305@colorado.edu> <3F6F8546.9020500@ieee.org>
Message-ID: <3F6F8897.3080902@colorado.edu>

Tim Hochberg wrote:
> I actually have no idea how you plan to make keyword arguments work 
> here, perhaps you could explain that in more detail. Metaclasses are 
> overkill, but a mixin, marker class could be used. That is, when 
> designing a class for use with numarray, one would derive a class from a 
> marker class in numarray::

Well, the joys of being vague :)  As I said, I haven't followed in enough 
detail to be too specific, so perhaps my idea plain doesn't work.  But what I 
had in mind was something along the lines of:

class ArrayLike(numarray):
	def __init__(self,...):
		...
		numarray.__init__(defers=1)

This is ultimately equivalent to a registration process, but since it is done 
in the constructor, it feels less like a separate dangling side-effect.  On 
the other hand, it makes your new constructor more expensive, and it feels 
ugly to be doing at the instance level things which truly belong at the class 
level.  Hence my comment about this perhaps being better done via metaclasses.

As I said, I knew I was being vague.  But hopefully these comments at least 
suggest a way out of a separate registration step.

Regards,

Fernando.




From jmiller at stsci.edu  Tue Sep 23 03:35:01 2003
From: jmiller at stsci.edu (Todd Miller)
Date: Tue Sep 23 03:35:01 2003
Subject: [Numpy-discussion] numarray unfriendly to user defined types
In-Reply-To: <3F6F8546.9020500@ieee.org>
References: <3F6A3D4C.3070703@ieee.org>
	 <1064266268.9234.351.camel@halloween.stsci.edu> <3F6F7744.80908@ieee.org>
	 <3F6F7B83.3040305@colorado.edu>  <3F6F8546.9020500@ieee.org>
Message-ID: <1064313211.3460.160.camel@localhost.localdomain>

On Mon, 2003-09-22 at 19:27, Tim Hochberg wrote:

> I actually have no idea how you plan to make keyword arguments work 
> here, perhaps you could explain that in more detail. Metaclasses are 
> overkill, but a mixin, marker class could be used. That is, when 
> designing a class for use with numarray, one would derive a class from a 
> marker class in numarray::
> 
>    class MyArrayLikeClass(numarray.DeferToMe):
>        ....
> 
> Hmmm. That's not too bad. Todd, what do you think about using this logic::
> 
>     def __mul__(self, operand):
>         if isinstance(operand, DeferToMe):
>             operand.__rmul__(self)
>         else:
>             self.__mul__(operand)
> 
> 

I like the core idea a lot.  My only doubt is whether forcing the use of
inheritance is appropriate / a good thing.  We might also consider
spelling it like:

class MyArrayLikeClass:
	_numarray_defer_to_me = True

class NumArray:
	def __mul__(self, operand):
		if hasattr(operand, "_numarray_defer_to_me"):
			return operand.__rmul__(self)
		else:	
			return ufunc.multiply(self, operand)

> The only case where I see a potential problem is an old-style 
> C-extenstion that can't be subclassed. I think that might be a case of 
> YAGNI though. 

Sounds YAGNI to me.

> Avoiding registration is appealing.

Good to have this seconded.

> -tim
> 

All in all, great ideas!

Todd

-- 
Todd Miller <jmiller at stsci.edu>




From jmiller at stsci.edu  Tue Sep 23 04:01:10 2003
From: jmiller at stsci.edu (Todd Miller)
Date: Tue Sep 23 04:01:10 2003
Subject: [Numpy-discussion] numarray unfriendly to user defined types
In-Reply-To: <3F6F8897.3080902@colorado.edu>
References: <3F6A3D4C.3070703@ieee.org>
	 <1064266268.9234.351.camel@halloween.stsci.edu> <3F6F7744.80908@ieee.org>
	 <3F6F7B83.3040305@colorado.edu> <3F6F8546.9020500@ieee.org>
	 <3F6F8897.3080902@colorado.edu>
Message-ID: <1064313214.14570.162.camel@localhost.localdomain>

On Mon, 2003-09-22 at 19:41, Fernando Perez wrote:
> Tim Hochberg wrote:
> > I actually have no idea how you plan to make keyword arguments work 
> > here, perhaps you could explain that in more detail. Metaclasses are 
> > overkill, but a mixin, marker class could be used. That is, when 
> > designing a class for use with numarray, one would derive a class from a 
> > marker class in numarray::
> 
> Well, the joys of being vague :)  As I said, I haven't followed in enough 
> detail to be too specific, so perhaps my idea plain doesn't work.  But what I 
> had in mind was something along the lines of:
> 
> class ArrayLike(numarray):
> 	def __init__(self,...):
> 		...
> 		numarray.__init__(defers=1)
> 

I think this behavior is already granted by Python at the abstract
object level;  that is, if you subclass, the r-method of the subclass is
given preference over the l-method of the superclass.  That's my
impression anyway.

Thus, I think we already get "deference" for free with subclasses, but
that is too weak in general because often (like MA for instance) people
won't want to subclass from NumArray.

> This is ultimately equivalent to a registration process, but since it is done 
> in the constructor, it feels less like a separate dangling side-effect.  On 
> the other hand, it makes your new constructor more expensive, and it feels 
> ugly to be doing at the instance level things which truly belong at the class 
> level.  Hence my comment about this perhaps being better done via metaclasses.
> 
> As I said, I knew I was being vague.  But hopefully these comments at least 
> suggest a way out of a separate registration step.

They certainly appear to have done so.  Thanks for commenting.


Todd

-- 
Todd Miller <jmiller at stsci.edu>




From jmiller at stsci.edu  Tue Sep 23 08:11:10 2003
From: jmiller at stsci.edu (Todd Miller)
Date: Tue Sep 23 08:11:10 2003
Subject: [Numpy-discussion] numarray boolean array formatting poll results
Message-ID: <1064329792.11003.55.camel@halloween.stsci.edu>

The thread on what to do with boolean array formatting in numarray has
died out,  so I thought I should explain (what I saw) as the results of
the poll:

(1) There was a clear majority of numarray users who actually prefer the
status quo of 0,1 for representing arrays of boolean values.  

(2) There was a sizable minority that would like to see consistency with
Python-2.3 by using False, True for representing arrays of boolean
values.

(3) There was a (good) suggestion of using F, T for representing boolean
values.  However, on reflection and reviewing the poll comments,  F, T
is not really as good as 0, 1 visually, nor is it likely to satisfy
those people interested in consistency with Python, so it is unlikely to
make most numarray users any happier.

(4) Also suggested was supporting configurable boolean formats.  This
approach is reasonable to implement,  but I have a reservation that
adding support for multiple formats may cause as much confusion as it
gives satisfaction.  Is it *really* worth it?  (more later)

Given 1-4,  I think the best thing to do for now is keep 0,1 as the only
format for boolean array values.   

If someone wants False, True bad enough to submit a complete patch for
supporting multiple formats (and defaulting to 0,1),  we will still
accept it.  If anyone is still interested in doing this, the patch
should include modified TeX documentation and pass all the self tests in
setupall.py (all of core numarray and all current add-ons).  The patch
should also run cleanly on both Python-2.2 and Python-2.3.

Thanks everyone for responding,
Todd

-- 
Todd Miller 			jmiller at stsci.edu
STSCI / ESS / SSB




From Chris.Barker at noaa.gov  Tue Sep 23 13:49:23 2003
From: Chris.Barker at noaa.gov (Chris Barker)
Date: Tue Sep 23 13:49:23 2003
Subject: [Numpy-discussion] numarray and NaN ?
References: <010701c362c0$3b54ec50$a47ba8c0@ericlaptop>
Message-ID: <3F70AF4D.5DFC0734@noaa.gov>

Hi all,

There was a lot of discussion a long while back about how Numeric treats
erros in the middle of an array operation. The way it works is if one
operation in an array has a problem (overflow, division by zero, etc),
an exception is returned. You have no idea which element caused the
problem, and teh whole operation grinds to a halt. Personally, I would
MUCH prefer MATLAB's way of handling this: the problematic operation
results in a -Inf, Inf, or NaN result for that element, and the rest of
the array operation can continue. MATLAB also provides utility functions
along the line of isnan, isinf, etc, so you can check for the
problematic areas later.

There was a fair bit of disagreement over whether Numeric should act
similarly, as well as the problem that not all compilers have the
required IEEE support to make this work.

Anyway, how does numarray handle this issue? I'd love to have the MATLAB
behaviour...I'm writing this because I'm about to have to write a bunch
of code to keep making sure I don't pass a 0 into log(), when I'd be
much happier if I could just replace the NaNs with what I want later.

-Chris

-- 
Christopher Barker, Ph.D.
Oceanographer
                                    		
NOAA/OR&R/HAZMAT         (206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115       (206) 526-6317   main reception

Chris.Barker at noaa.gov



From perry at stsci.edu  Tue Sep 23 14:27:10 2003
From: perry at stsci.edu (Perry Greenfield)
Date: Tue Sep 23 14:27:10 2003
Subject: [Numpy-discussion] numarray and NaN ?
In-Reply-To: <3F70AF4D.5DFC0734@noaa.gov>
Message-ID: <CEELJPECNGEGKFNDLHFFMEMMCAAA.perry@stsci.edu>

Chris Barker writes:
>
> There was a lot of discussion a long while back about how Numeric treats
> erros in the middle of an array operation. The way it works is if one
> operation in an array has a problem (overflow, division by zero, etc),
> an exception is returned. You have no idea which element caused the
> problem, and teh whole operation grinds to a halt. Personally, I would
> MUCH prefer MATLAB's way of handling this: the problematic operation
> results in a -Inf, Inf, or NaN result for that element, and the rest of
> the array operation can continue. MATLAB also provides utility functions
> along the line of isnan, isinf, etc, so you can check for the
> problematic areas later.
>
> There was a fair bit of disagreement over whether Numeric should act
> similarly, as well as the problem that not all compilers have the
> required IEEE support to make this work.
>
I don't recall if there was disagreement about this being useful, but
perhaps there was disagreement that it should always do this.

> Anyway, how does numarray handle this issue? I'd love to have the MATLAB
> behaviour...I'm writing this because I'm about to have to write a bunch
> of code to keep making sure I don't pass a 0 into log(), when I'd be
> much happier if I could just replace the NaNs with what I want later.
>
numarray allows one to customize how errors are handled. You can
choose:

1) to silently ignore all errors.
2) print a warning message (default)
3) raise an exception.

One may separately set one of these three behaviors for each of
the 4 ieee categories of floating point errors, namely

1) invalid results (i.e., NaNs)
2) divide by zeros (Infs)
3) overflows
4) underflows

for the first two modes the computation on floats just inserts the
appropriate
ieee value in the array (Infs and NaNs). There are portable constants and
functions to test and set these values (see examples below).
These modes also apply to integer calculations.

I've pasted some examples I presented from some slides shown at the scipy
conference to illustrate how it works and what's available.

Customize how errors are handled

>>> numeric.Error.setMode(all="warn")
>>> numeric.Error.pushMode(dividebyzero="warn",
..  Invalid="raise",overflow="warn",underflow="ignore")
>>> numeric.Error.popMode()
>>> a = 1/arange(10.)
Warning: Encountered divide by zero(s) in divide

Manipulate IEEE-754 special values

>>> import numarray.ieeespecial as ieee
>>> ieee.getinf(a)  # return indices of Infs
(array([0],)

>>> a = arange(10.)/arange(10.)
>>> ieee.setnan(a, 999) # set all NaNs to 999
>>> a
array([999.,1.,1.,1.,1.,1.,1.,1.,1.,1.])
>>> a[ieee.getinf(a)] = 999 # equivalent

Use defined constants

>>> a[5] = ieee.nan
>>> a[6] = ieee.plus_inf
>>> a[7] = ieee.minus_inf
>>> a[8] = ieee.inf
>>> a
array([999.,1.,1.,1.,1.,nan,inf,-inf,inf,1.]

Perry




From Chris.Barker at noaa.gov  Tue Sep 23 15:04:06 2003
From: Chris.Barker at noaa.gov (Chris Barker)
Date: Tue Sep 23 15:04:06 2003
Subject: [Numpy-discussion] numarray and NaN ?
References: <CEELJPECNGEGKFNDLHFFMEMMCAAA.perry@stsci.edu>
Message-ID: <3F70C0D0.F28E157A@noaa.gov>

Perry Greenfield wrote:

> I don't recall if there was disagreement about this being useful,

probably not

> but perhaps there was disagreement that it should always do this.

absolutely, and I think there was a wide range of opinions about how
usefull (and or critical!) this is.

> numarray allows one to customize how errors are handled. You can
> choose:
> 
> 1) to silently ignore all errors.
> 2) print a warning message (default)
> 3) raise an exception.
> 
> One may separately set one of these three behaviors for each of
> the 4 ieee categories of floating point errors, namely
> 
> 1) invalid results (i.e., NaNs)
> 2) divide by zeros (Infs)
> 3) overflows
> 4) underflows

Fabulous!!! You folks are wonderfull! I guess it's time to make the
switch to numarray...I've been waiting for a compelling reason!

-Chris



-- 
Christopher Barker, Ph.D.
Oceanographer
                                    		
NOAA/OR&R/HAZMAT         (206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115       (206) 526-6317   main reception

Chris.Barker at noaa.gov



From nadavh at VisionSense.com  Wed Sep 24 00:15:05 2003
From: nadavh at VisionSense.com (Nadav Horesh)
Date: Wed Sep 24 00:15:05 2003
Subject: [Numpy-discussion] Subclassing numarray arrays.
Message-ID: <1064391268.1645.38.camel@Nadav.Envision.co.il>

Recently I had an idea to subclass numarray's array in order to
implement a different scheme of array indexing via the __call__ method
(ugly indeed, by my experience shows that it can be useful). It occurred
to me that the array class is not in the spirit of Python 2.2+
classes/types unification:
      * The class name is not "trivial": numarraycore.NumArray
      * The class constructor is not built to be used by the commons.
Since arrays are normally generated by functions (zeros, array,
fromfunction etc.) and not by the constructor, subclassing
numarraycore.NumArray is not very useful. If, for instance, "array" (or
"Array") would be the class name, where the constructor would optionally
have the same parameters as the array function (with the option to act
as a copy constructor as the array function does), then it would be easy
to subclass it:

class MyArray(array):
  ...


new_array = MyArray(zeros((10,5), type=Int64))


  Any comments?

   Nadav.




From cjw at sympatico.ca  Wed Sep 24 04:32:01 2003
From: cjw at sympatico.ca (Colin J. Williams)
Date: Wed Sep 24 04:32:01 2003
Subject: [Numpy-discussion] Subclassing numarray arrays.
In-Reply-To: <1064391268.1645.38.camel@Nadav.Envision.co.il>
References: <1064391268.1645.38.camel@Nadav.Envision.co.il>
Message-ID: <3F718076.3050002@sympatico.ca>

Nadav Horesh wrote:

>Recently I had an idea to subclass numarray's array in order to
>implement a different scheme of array indexing via the __call__ method
>(ugly indeed, by my experience shows that it can be useful). It occurred
>to me that the array class is not in the spirit of Python 2.2+
>classes/types unification:
>
I agree.

>      * The class name is not "trivial": numarraycore.NumArray
>      * The class constructor is not built to be used by the commons.
>
I'm not sure what you have in mind here by "trivial" and "commons".

>Since arrays are normally generated by functions (zeros, array,
>fromfunction etc.) and not by the constructor, subclassing
>numarraycore.NumArray is not very useful. If, for instance, "array" (or
>"Array") would be the class name, where the constructor would optionally
>have the same parameters as the array function (with the option to act
>as a copy constructor as the array function does), then it would be easy
>to subclass it:
>
>class MyArray(array):
>  ...
>
>
>new_array = MyArray(zeros((10,5), type=Int64))
>
>  
>
This is desirable, with the present design, one has to go through a two 
step process:

new_array = MyArray(type=Int64, shape= (10, 5))
load_some_data_into_array(new_array, zeros(10, 5))
  

Colin W.

>  Any comments?
>
>   Nadav.
>
>
>
>-------------------------------------------------------
>This sf.net email is sponsored by:ThinkGeek
>Welcome to geek heaven.
>http://thinkgeek.com/sf
>_______________________________________________
>Numpy-discussion mailing list
>Numpy-discussion at lists.sourceforge.net
>https://lists.sourceforge.net/lists/listinfo/numpy-discussion
>
>  
>





From tim.hochberg at ieee.org  Wed Sep 24 15:28:32 2003
From: tim.hochberg at ieee.org (Tim Hochberg)
Date: Wed Sep 24 15:28:32 2003
Subject: [Numpy-discussion] numarray unfriendly to user defined types
In-Reply-To: <1064313211.3460.160.camel@localhost.localdomain>
References: <3F6A3D4C.3070703@ieee.org>	 <1064266268.9234.351.camel@halloween.stsci.edu> <3F6F7744.80908@ieee.org>	 <3F6F7B83.3040305@colorado.edu>  <3F6F8546.9020500@ieee.org> <1064313211.3460.160.camel@localhost.localdomain>
Message-ID: <3F721A4C.7080300@ieee.org>

Hi Todd,

There are three ways to spell "defer to me" on the table (the precise 
details of
each spelling are, of course, still open for debate):

   1.  numarray.defer_to(my_class)

   2. class ArrayLike(numarray.DeferTo):
         # ...

   3. class ArrayLike:
         _numarray_defer_to = True
        # ...

I'd prefer a non-registration solution since those are both 
aesthetically displeasing and leave you open to the situation where a 
class in module A gets registered by module B, but module C expects it 
not to be registered and everything breaks. Not all that likely, I 
admit, but let's avoid the registration version if we can.

The other two solutions are almost equivalent. The one case where 3 has 
an edge over 2 is if I have an object (not a class), I could potentially 
set a _numarray_defer_to on the object before passing it to numarray 
without having to mess with the class
of the object at all. YAGNI, though.

The advantage of 2 in my view is that it *does* force you to subclass. 
With 3, there will be the temptation to poke into some other  module and 
set _numarray_defer_to on some poor unsuspecting class. This has the 
same disadvantage as 1, that it could confuse some other poor 
unsuspecting module. The correct way to do get a deferred class from a 
third party module is to import and subclass. This works with either 2 
or 3::

  import A

  class Klass2(a.Klass, numarray.DeferTo):   #2
     #...

  class Klass3(a.Klass): #3 the good way
      _numarray_defer_to = True
     # ...

   A.Klass._numarray_defer_to = True #3 the evil way.

 Version 2 is cleaner and encourages you to do the right thing, so I'd 
prefer that solution.


regards,

-tim




Todd Miller wrote:

>On Mon, 2003-09-22 at 19:27, Tim Hochberg wrote:
>
>  
>
>>I actually have no idea how you plan to make keyword arguments work 
>>here, perhaps you could explain that in more detail. Metaclasses are 
>>overkill, but a mixin, marker class could be used. That is, when 
>>designing a class for use with numarray, one would derive a class from a 
>>marker class in numarray::
>>
>>   class MyArrayLikeClass(numarray.DeferToMe):
>>       ....
>>
>>Hmmm. That's not too bad. Todd, what do you think about using this logic::
>>
>>    def __mul__(self, operand):
>>        if isinstance(operand, DeferToMe):
>>            operand.__rmul__(self)
>>        else:
>>            self.__mul__(operand)
>>
>>
>>    
>>
>
>I like the core idea a lot.  My only doubt is whether forcing the use of
>inheritance is appropriate / a good thing.  We might also consider
>spelling it like:
>
>class MyArrayLikeClass:
>	_numarray_defer_to_me = True
>
>class NumArray:
>	def __mul__(self, operand):
>		if hasattr(operand, "_numarray_defer_to_me"):
>			return operand.__rmul__(self)
>		else:	
>			return ufunc.multiply(self, operand)
>
>  
>
>>The only case where I see a potential problem is an old-style 
>>C-extenstion that can't be subclassed. I think that might be a case of 
>>YAGNI though. 
>>    
>>
>
>Sounds YAGNI to me.
>
>  
>
>>Avoiding registration is appealing.
>>    
>>
>
>Good to have this seconded.
>
>  
>
>>-tim
>>
>>    
>>
>
>All in all, great ideas!
>
>Todd
>
>  
>






From jmiller at stsci.edu  Wed Sep 24 16:01:05 2003
From: jmiller at stsci.edu (Todd Miller)
Date: Wed Sep 24 16:01:05 2003
Subject: [Numpy-discussion] numarray unfriendly to user defined types
In-Reply-To: <3F721A4C.7080300@ieee.org>
References: <3F6A3D4C.3070703@ieee.org>
	 <1064266268.9234.351.camel@halloween.stsci.edu> <3F6F7744.80908@ieee.org>
	 <3F6F7B83.3040305@colorado.edu>  <3F6F8546.9020500@ieee.org>
	 <1064313211.3460.160.camel@localhost.localdomain>
	 <3F721A4C.7080300@ieee.org>
Message-ID: <1064444385.23878.14.camel@localhost.localdomain>

On Wed, 2003-09-24 at 18:27, Tim Hochberg wrote:
> Hi Todd,
> 
> There are three ways to spell "defer to me" on the table (the precise 
> details of
> each spelling are, of course, still open for debate):
> 
>    1.  numarray.defer_to(my_class)
> 
>    2. class ArrayLike(numarray.DeferTo):
>          # ...
> 
>    3. class ArrayLike:
>          _numarray_defer_to = True
>         # ...
> 
> I'd prefer a non-registration solution since those are both 
> aesthetically displeasing and leave you open to the situation where a 
> class in module A gets registered by module B, but module C expects it 
> not to be registered and everything breaks. Not all that likely, I 
> admit, but let's avoid the registration version if we can.

I was picturing this as module A registering it's own classes only. 
Nevertheless, inverting the problem and distributing the registration as
you suggested is better.

> 
> The other two solutions are almost equivalent. The one case where 3 has 
> an edge over 2 is if I have an object (not a class), I could potentially 
> set a _numarray_defer_to on the object before passing it to numarray 
> without having to mess with the class
> of the object at all. YAGNI, though.

I was more concerned about the potential impact of lots of multiple
inheritance, but that's probably just my own personal blend of FUD.

> The advantage of 2 in my view is that it *does* force you to subclass. 
> With 3, there will be the temptation to poke into some other  module and 
> set _numarray_defer_to on some poor unsuspecting class. This has the 
> same disadvantage as 1, that it could confuse some other poor 
> unsuspecting module. The correct way to do get a deferred class from a 
> third party module is to import and subclass. This works with either 2 
> or 3::
> 
>   import A
> 
>   class Klass2(a.Klass, numarray.DeferTo):   #2
>      #...
> 
>   class Klass3(a.Klass): #3 the good way
>       _numarray_defer_to = True
>      # ...
> 
>    A.Klass._numarray_defer_to = True #3 the evil way.
> 
>  Version 2 is cleaner and encourages you to do the right thing, so I'd 
> prefer that solution.
> 

Good enough for me.  If no one else has any comments, then
numarray.DeferTo is where I'll start implementing.  Tomorrow.

> 
> regards,
> 
> -tim

Thanks again,
Todd







From tim.hochberg at ieee.org  Wed Sep 24 16:57:05 2003
From: tim.hochberg at ieee.org (Tim Hochberg)
Date: Wed Sep 24 16:57:05 2003
Subject: [Numpy-discussion] numarray unfriendly to user defined types
In-Reply-To: <1064444385.23878.14.camel@localhost.localdomain>
References: <3F6A3D4C.3070703@ieee.org>	 <1064266268.9234.351.camel@halloween.stsci.edu> <3F6F7744.80908@ieee.org>	 <3F6F7B83.3040305@colorado.edu>  <3F6F8546.9020500@ieee.org>	 <1064313211.3460.160.camel@localhost.localdomain>	 <3F721A4C.7080300@ieee.org> <1064444385.23878.14.camel@localhost.localdomain>
Message-ID: <3F722F0B.7070609@ieee.org>

Todd Miller wrote:

>On Wed, 2003-09-24 at 18:27, Tim Hochberg wrote:
>  
>
>>Hi Todd,
>>
>>There are three ways to spell "defer to me" on the table (the precise 
>>details of
>>each spelling are, of course, still open for debate):
>>
>>   1.  numarray.defer_to(my_class)
>>
>>   2. class ArrayLike(numarray.DeferTo):
>>         # ...
>>
>>   3. class ArrayLike:
>>         _numarray_defer_to = True
>>        # ...
>>
>>I'd prefer a non-registration solution since those are both 
>>aesthetically displeasing and leave you open to the situation where a 
>>class in module A gets registered by module B, but module C expects it 
>>not to be registered and everything breaks. Not all that likely, I 
>>admit, but let's avoid the registration version if we can.
>>    
>>
>
>I was picturing this as module A registering it's own classes only. 
>Nevertheless, inverting the problem and distributing the registration as
>you suggested is better.
>  
>

The case you describe probably will describe the majority of actual use 
cases, and in fact describes mine. I'm trying to think ahead a bit to 
cases may encounter as start using NumArray more extensively. Let's hope 
this solution still looks good in six months!

>>The other two solutions are almost equivalent. The one case where 3 has 
>>an edge over 2 is if I have an object (not a class), I could potentially 
>>set a _numarray_defer_to on the object before passing it to numarray 
>>without having to mess with the class
>>of the object at all. YAGNI, though.
>>    
>>
>
>I was more concerned about the potential impact of lots of multiple
>inheritance, but that's probably just my own personal blend of FUD.
>
>  
>
>>The advantage of 2 in my view is that it *does* force you to subclass. 
>>With 3, there will be the temptation to poke into some other  module and 
>>set _numarray_defer_to on some poor unsuspecting class. This has the 
>>same disadvantage as 1, that it could confuse some other poor 
>>unsuspecting module. The correct way to do get a deferred class from a 
>>third party module is to import and subclass. This works with either 2 
>>or 3::
>>
>>  import A
>>
>>  class Klass2(a.Klass, numarray.DeferTo):   #2
>>     #...
>>
>>  class Klass3(a.Klass): #3 the good way
>>      _numarray_defer_to = True
>>     # ...
>>
>>   A.Klass._numarray_defer_to = True #3 the evil way.
>>
>> Version 2 is cleaner and encourages you to do the right thing, so I'd 
>>prefer that solution.
>>
>>    
>>
>
>Good enough for me.  If no one else has any comments, then
>numarray.DeferTo is where I'll start implementing.  Tomorrow.
>  
>

One more minor thing. I'm not sure tha DeferTo is ideal as the mix-in 
class name. It was perfect for the registration function name, but I'm 
not sure it's so clear whether the class or numarray is being deferred 
to when you say numarray.DeferTo. DeferToMe is more descriptive, but 
seems sort of slangy. DeferredTo is better than DeferTo, but still not 
as descriptive as DeferToMe. numarray.DefersTo reads perfect as long as 
numarray is included but is a disaster if you read it on your own.  
Below I've put down all the ideas I could come up with

class CustomArray(numarray.DeferTo)
class CustomArray(numarray.DefersTo)
class CustomArray(numarray.DeferredTo)
class CustomArray(numarray.DeferToMe)
class CustomArray(numarray.DeferredToByNumarray)
class CustomArray(DeferTo)
class CustomArray(DefersTo)
class CustomArray(DeferredTo)
class CustomArray(DeferToMe)
class CustomArray(DeferredToByNumarray)

For me it's a toss up between DefferedTo, DeferToMe and 
DeferredToByNumarray. The first is a little lacking in descriptive 
power, the second is slangy and the third is wordy.

-tim

[not that this matters much....]


>>regards,
>>
>>-tim
>>    
>>
>
>Thanks again,
>Todd
>
>
>
>
>
>  
>






From nadavh at VisionSense.com  Thu Sep 25 00:26:05 2003
From: nadavh at VisionSense.com (Nadav Horesh)
Date: Thu Sep 25 00:26:05 2003
Subject: [Numpy-discussion] Subclassing numarray arrays
Message-ID: <1064478298.10525.24.camel@Nadav.Envision.co.il>

On Wed, 2003-09-24 at 14:31, Colin J. Williams wrote:
> Nadav Horesh wrote:
>
> >Recently I had an idea to subclass numarray's array in order to
> >implement a different scheme of array indexing via the __call__
method
> >(ugly indeed, by my experience shows that it can be useful). It
occurred
> >to me that the array class is not in the spirit of Python 2.2+
> >classes/types unification:
> >
> I agree.
>
> >      * The class name is not "trivial": numarraycore.NumArray
> >      * The class constructor is not built to be used by the commons.
> >
> I'm not sure what you have in mind here by "trivial" and "commons".

commons == End users whose most scripts life-time is few days.
trivial == A name one would expect, like "Array" or similar.

>
> >Since arrays are normally generated by functions (zeros, array,
> >fromfunction etc.) and not by the constructor, subclassing
> >numarraycore.NumArray is not very useful. If, for instance, "array"
(or
> >"Array") would be the class name, where the constructor would
optionally
> >have the same parameters as the array function (with the option to
act
> >as a copy constructor as the array function does), then it would be
easy
> >to subclass it:
> >
> >class MyArray(array):
> >  ...
> >
> >
> >new_array = MyArray(zeros((10,5), type=Int64))
> >
> > 
> >
> This is desirable, with the present design, one has to go through a
two
> step process:
>
> new_array = MyArray(type=Int64, shape= (10, 5))
> load_some_data_into_array(new_array, zeros(10, 5))
>  
> Colin W.

As I recall, one of the targets of the numarray project is to be
included as a standard python library --- shouldn't it eventually comply
with the new-classes standard?

>
> >  Any comments?
> >
> >   Nadav.
> >
> >
> >
> >-------------------------------------------------------
> >This sf.net email is sponsored by:ThinkGeek
> >Welcome to geek heaven.
> >http://thinkgeek.com/sf
> >_______________________________________________
> >Numpy-discussion mailing list
> >Numpy-discussion at lists.sourceforge.net
> >https://lists.sourceforge.net/lists/listinfo/numpy-discussion
> >
> > 
> >
>
>




From cjw at sympatico.ca  Thu Sep 25 05:16:08 2003
From: cjw at sympatico.ca (Colin J. Williams)
Date: Thu Sep 25 05:16:08 2003
Subject: [Numpy-discussion] Subclassing numarray arrays
In-Reply-To: <1064478298.10525.24.camel@Nadav.Envision.co.il>
References: <1064478298.10525.24.camel@Nadav.Envision.co.il>
Message-ID: <3F72DC68.6030403@sympatico.ca>

Nadav Horesh wrote:
[snip]

>As I recall, one of the targets of the numarray project is to be
>included as a standard python library --- shouldn't it eventually comply
>with the new-classes standard?
>  
>
I hope that more consideration will be given to this as the numarray 
package evolves.

 From time to time, I am developing a matrix sub-class, it's a bit more 
convoluted than I would like it to be.

It would also be helpful if a current assessment of the speed of 
numarray could be published.  I gather that some enhancements arrived 
with version 0.7.

Colin W.




From jmiller at stsci.edu  Thu Sep 25 07:01:07 2003
From: jmiller at stsci.edu (Todd Miller)
Date: Thu Sep 25 07:01:07 2003
Subject: [Numpy-discussion] numarray unfriendly to user defined types
In-Reply-To: <3F722F0B.7070609@ieee.org>
References: <3F6A3D4C.3070703@ieee.org>	
	<1064266268.9234.351.camel@halloween.stsci.edu> <3F6F7744.80908@ieee.org>	
	<3F6F7B83.3040305@colorado.edu>  <3F6F8546.9020500@ieee.org>	
	<1064313211.3460.160.camel@localhost.localdomain>	
	<3F721A4C.7080300@ieee.org>
	<1064444385.23878.14.camel@localhost.localdomain> 
	<3F722F0B.7070609@ieee.org>
Message-ID: <1064498375.13806.84.camel@halloween.stsci.edu>

On Wed, 2003-09-24 at 19:55, Tim Hochberg wrote:
> One more minor thing. I'm not sure tha DeferTo is ideal as the mix-in 
> class name. It was perfect for the registration function name, but I'm 
> not sure it's so clear whether the class or numarray is being deferred 
> to when you say numarray.DeferTo. DeferToMe is more descriptive, but 
> seems sort of slangy. DeferredTo is better than DeferTo, but still not 
> as descriptive as DeferToMe. numarray.DefersTo reads perfect as long as 
> numarray is included but is a disaster if you read it on your own.  
> Below I've put down all the ideas I could come up with
> 
> class CustomArray(numarray.DeferTo)
> class CustomArray(numarray.DefersTo)
> class CustomArray(numarray.DeferredTo)
> class CustomArray(numarray.DeferToMe)
> class CustomArray(numarray.DeferredToByNumarray)
> class CustomArray(DeferTo)
> class CustomArray(DefersTo)
> class CustomArray(DeferredTo)
> class CustomArray(DeferToMe)
> class CustomArray(DeferredToByNumarray)
> 
> For me it's a toss up between DefferedTo, DeferToMe and 
> DeferredToByNumarray. The first is a little lacking in descriptive 
> power, the second is slangy and the third is wordy.

For something as esoteric as this, I think wordy is probably best. 
Barring a Perrenial edict,  DeferredToByNumArray it is.

I'm psyched.  The MA port should inter-operate with numarray... today.

Thanks for all the lucid, constructive suggestions,

Todd

-- 
Todd Miller 			jmiller at stsci.edu
STSCI / ESS / SSB




From jmiller at stsci.edu  Thu Sep 25 07:26:11 2003
From: jmiller at stsci.edu (Todd Miller)
Date: Thu Sep 25 07:26:11 2003
Subject: [Numpy-discussion] Subclassing numarray arrays
In-Reply-To: <3F72DC68.6030403@sympatico.ca>
References: <1064478298.10525.24.camel@Nadav.Envision.co.il> 
	<3F72DC68.6030403@sympatico.ca>
Message-ID: <1064499880.13807.111.camel@halloween.stsci.edu>

On Thu, 2003-09-25 at 08:15, Colin J. Williams wrote:
> Nadav Horesh wrote:
> [snip]
> 
> >As I recall, one of the targets of the numarray project is to be
> >included as a standard python library --- shouldn't it eventually comply
> >with the new-classes standard?

Can someone please explain what the "new-classes standard" is? 
(Obviously meeting it is not one of our current goals)

> I hope that more consideration will be given to this as the numarray 
> package evolves.

Of course.  We don't need sub-classing internally, however, so it's
difficult to predict when we'll do better.

>  From time to time, I am developing a matrix sub-class, it's a bit more 
> convoluted than I would like it to be.

Posting it here would be good.  Sending things directly to me tends to
dumb things down... to me.
 
> It would also be helpful if a current assessment of the speed of 
> numarray could be published.  

You asked... and...  (sorry if this is less than succinct)

The numarray source distribution has a module called bench.py in the
Examples directory.  Without any analysis,  I've attached the output
from bench.py on my 1.7 GHz Pentium IV running under i386 linux.  Each
row shows the timing numbers for numarray and Numeric for a particular
function call and parameter set;  the important part is the ratio ( < 1
means numarray is faster.  In general, numarray is slower;  this is due
to the fact that numarray supports in-place operation on memory mapped
non-native data formats).

Suspect numbers are... suspect.   Please investigate!  

Caveats:

1) The benchmarks were run on a multi-tasking kernel with no specific
load.  The benchmarks also vary from platform to platform, processor to
processor, compiler to compiler, etc.  These are gcc version 3.2
20020903 (Red Hat Linux 8.0 3.2-7) on a 1.7 GHz P-IV with 384M of RAM.

2) The absolute measured units (microseconds) haven't really been
validated and shouldn't be taken too seriously.  The relative time
comparisons should be valid.

3) a,c,d are 1D arrays.  b is a 2D array.

4) the 10**x column shows the logarithmic array size. 
   2 --> 100 elements, 6 --> 1 million elements

5) The version of bench.py which made these measurements is in CVS, not
the numarray-0.7 tar-ball.

Wish-list:

1) The right way to eliminate the effects of host load on the
benchmarks.  

2) A way to create a regression test which includes performance.

> I gather that some enhancements arrived with version 0.7.

I don't recall any performance enhancements.  numarray-0.7 was mostly
about object array support and bug fixes.  

bench.py results from numarray-0.7 09-25-03:

benchmark                10**x   numarray (usec)    Numeric (usec)  numarray:Numeric
take(a,c)                    2              1477                25              56.8
                             3              1592               108              14.7
                             4              2920               989               2.9
                             5             15568              9814               1.6
                             6            136960             96253               1.4
put(a,c,d)                   2              1673                64              26.1
                             3              1798               195               9.2
                             4              3270              1587               2.1
                             5             16742             15334               1.1
                             6            147768            151932               1.0
transpose(b)                 2               154                22               6.8
                             3               157                24               6.3
                             4               190                39               4.8
                             5               262                74               3.5
                             6               257                67               3.8
putmask(a,c%2,d)             2              1415               139              10.1
                             3              1504               271               5.6
                             4              2439              1735               1.4
                             5             11036             17337               0.6
                             6             93927            157848               0.6
repeat(a,2)                  2              1533               138              11.1
                             3             10165               579              17.5
                             4            103064              4701              21.9
                             5           1028850             58812              17.5
                             6          10010859            493157              20.3
choose(a%2,(a,c))            2              1312                73              18.0
                             3              1389               190               7.3
                             4              2479              1500               1.7
                             5             12516             16219               0.8
                             6            108841            142280               0.8
ravel(b)                     2               164                38               4.3
                             3               169                41               4.1
                             4               196                58               3.4
                             5               279                93               3.0
                             6               275                90               3.0
nonzero(a)                   2              1044               143               7.3
                             3              1133               285               4.0
                             4              2108              1929               1.1
                             5             12002             17974               0.7
                             6            111598            176598               0.6
where(a%2,a,c)               2              1273               111              11.4
                             3              1380               242               5.7
                             4              2587              1752               1.5
                             5             13368             16785               0.8
                             6            117731            164187               0.7
compress(a%2,a)              2              2489               198              12.6
                             3              2661               375               7.1
                             4              4575              2426               1.9
                             5             22933             22621               1.0
                             6            200138            219215               0.9
diagonal(b)                  2              2420             16883               0.1
                             3              2425             16896               0.1
                             4              2591             16932               0.2
                             5              2588             16875               0.2
                             6              2579             16861               0.2
trace(b)                     2              2611             16988               0.2
                             3              2625             16990               0.2
                             4              2750             17004               0.2
                             5              2758             17048               0.2
                             6              2766             16942               0.2
searchsorted(a,a)            2               506                55               9.2
                             3               708               490               1.4
                             4              3054              6507               0.5
                             5             32101             77762               0.4
                             6            378341            940912               0.4
sort(a)                      2               554                85               6.4
                             3               940               365               2.6
                             4              5394              3533               1.5
                             5             55238             36049               1.5
                             6            607845            420334               1.4
argsort(a)                   2              1541                90              16.9
                             3              2061               467               4.4
                             4              8113              5130               1.6
                             5             76924             49497               1.6
                             6            853114            588399               1.4
argmax(b)                    2           1112081              8692             127.9
                             3           1110896              9320             119.2
                             4           1108330              8706             127.3
                             5           1113449              8720             127.7
                             6           1112556              8743             127.3
array(s,typecode='i')        2               229               146               1.6
                             3               274              1103               0.2
                             4               764             10900               0.1
                             5              5450            105774               0.1
                             6             51500           1253819               0.0
dot(b,b)                     2           8825828          31402493               0.3
                             3           8804230          32136602               0.3
                             4           8748239          31123564               0.3
                             5           8811900          32910953               0.3
                             6           8827314          31784649               0.3
clip(a,50,100)               2              2920               177              16.4
                             3              3089               370               8.3
                             4              5138              2471               2.1
                             5             24340             24737               1.0
                             6            211277            245540               0.9
indices((100,100))           2              5253              2696               1.9
                             3              5230              2634               2.0
                             4              5290              2687               2.0
                             5              5383              2732               2.0
                             6              5558              2733               2.0
swapaxes(b,0,-1)             2                77                79               1.0
                             3                77                82               1.0
                             4                94               106               0.9
                             5               154               142               1.1
                             6               146               144               1.0
concatenate((a,a,a))         2              1021                35              29.1
                             3              1082                48              22.1
                             4              1868               431               4.3
                             5              8843              4945               1.8
                             6             75181             47495               1.6
innerproduct(b,b)            2           8719124           8989714               1.0
                             3           8662431           8840564               1.0
                             4           8734306           9033604               1.0
                             5           8694994           8935343               1.0
                             6           8680599           8869750               1.0
outerproduct(b[0],b[0])      2             42320             27598               1.5
                             3             42299             27802               1.5
                             4             42842             27935               1.5
                             5             42346             27822               1.5
                             6             42257             27709               1.5
resize(a,(2*10**i,))         2              1708               160              10.6
                             3              1769               171              10.3
                             4              2648               468               5.6
                             5              9778              3515               2.8
                             6             75513             31778               2.4
a[0]                         2                10                10               1.0
                             3                10                10               1.0
                             4                13                12               1.1
                             5                25                20               1.3
                             6                25                22               1.1
b[0]                         2                46                15               3.1
                             3                48                17               2.8
                             4                56                18               3.0
                             5               106                33               3.1
                             6               100                35               2.9
a[1000:2000]                 2               303                16              18.0
                             3               298                15              18.7
                             4               275                20              13.8
                             5               354                33              10.4
                             6               344                34               9.8
b[400:600,400:600]           2               320                25              12.8
                             3               317                25              12.2
                             4               348                30              11.3
                             5               432                46               9.2
                             6               418                47               8.7
arange(10.**i)               2               411                37              11.1
                             3               448                74               6.0
                             4               923               570               1.6
                             5              5630              5592               1.0
                             6             50735             54222               0.9
identity(i+2)                2              3147               226              13.9
                             3              3158               227              13.9
                             4              3214               258              12.4
                             5              3311               294              11.3
                             6              3316               291              11.4
add(a,c)                     2                76                27               2.7
                             3                90                40               2.2
                             4               313               225               1.4
                             5              2902              2128               1.4
                             6             28106             19972               1.4
add(a,c,d)                   2                40                22               1.9
                             3                46                33               1.4
                             4               144               132               1.1
                             5              1128              1165               1.0
                             6             10316             10827               1.0
a+c                          2                91                31               2.9
                             3               105                45               2.4
                             4               335               231               1.5
                             5              2945              2130               1.4
                             6             28102             19744               1.4
cos(a)                       2                84                53               1.6
                             3               212               235               0.9
                             4              1546              2351               0.7
                             5             15237             23074               0.7
                             6            151604            227898               0.7
add.reduce(a)                2                97                31               3.0
                             3               109                41               2.7
                             4               207               123               1.7
                             5              1062               949               1.1
                             6              8974              8844               1.0
add.reduce(b,0)              2             32527              9218               3.5
                             3             31652              9238               3.4
                             4             31376              9171               3.4
                             5             31412              9271               3.4
                             6             30443              9263               3.3
add.reduce(b,-1)             2              9065              9389               1.0
                             3              9075              9360               1.0
                             4              9102              9271               1.0
                             5              9168              9389               1.0
                             6              9148              9394               1.0
add.accumulate(a)            2               100                28               3.5
                             3               120                41               2.9
                             4               416               254               1.6
                             5              3584              2696               1.3
                             6             34837             25608               1.4
add.accumulate(b,0)          2             63947             25673               2.5
                             3             65875             25691               2.6
                             4             65540             25708               2.5
                             5             65161             25776               2.5
                             6             66539             25646               2.6
add.accumulate(b,1)          2             35048             25756               1.4
                             3             35022             25738               1.4
                             4             35048             25688               1.4
                             5             35092             25761               1.4
                             6             35110             25686               1.4





From cjw at sympatico.ca  Thu Sep 25 09:15:13 2003
From: cjw at sympatico.ca (Colin J. Williams)
Date: Thu Sep 25 09:15:13 2003
Subject: [Numpy-discussion] Subclassing numarray arrays
In-Reply-To: <1064499880.13807.111.camel@halloween.stsci.edu>
References: <1064478298.10525.24.camel@Nadav.Envision.co.il> 	<3F72DC68.6030403@sympatico.ca> <1064499880.13807.111.camel@halloween.stsci.edu>
Message-ID: <3F731467.1070402@sympatico.ca>

Todd,

Thanks for the benchmark data.

I intend to post the matrix stuff.  Performance is not a problem for me 
at this stage.

Currently, I'm plodding along, making things work.  Later, I'll post it 
with some rudimentary documentation.

Regarding the "new-classes standard", I can't find much in either the 
Python docs or Alex Martelli's "Python in a Nutshell".  One of the new 
features is a __new__ constructor.

The Python array module doesn't formally have a class and thus it can't 
be sub-classed.  It uses a factory function to create instance objects.

Martelli, in PIAN p74, writes of factory functions being useful when one 
wishes to create instances of different classes depending on some 
condition or for the reuse of an existing instance.

In the Python Cookbook p173, he proposes the use of a factory function 
to get around the limitations of Python 2.1 and earlier.  On page 179, 
he refers to the use of a factory method, again to work around 
limitations before Python 2.2.

My rough and ready definition would include the ability to create an 
instance and initialize
the data in one step.

Perhaps it would be good to post your exchange with Nadev to the 
newsgroup comp.lang.python.  There are many contributors to the group 
who could give a much more complete answer.

Colin W.


Todd Miller wrote:

>On Thu, 2003-09-25 at 08:15, Colin J. Williams wrote:
>  
>
>>Nadav Horesh wrote:
>>[snip]
>>
>>    
>>
>>>As I recall, one of the targets of the numarray project is to be
>>>included as a standard python library --- shouldn't it eventually comply
>>>with the new-classes standard?
>>>      
>>>
>
>Can someone please explain what the "new-classes standard" is? 
>(Obviously meeting it is not one of our current goals)
>
>  
>
>>I hope that more consideration will be given to this as the numarray 
>>package evolves.
>>    
>>
>
>Of course.  We don't need sub-classing internally, however, so it's
>difficult to predict when we'll do better.
>
>  
>
>> From time to time, I am developing a matrix sub-class, it's a bit more 
>>convoluted than I would like it to be.
>>    
>>
>
>Posting it here would be good.  Sending things directly to me tends to
>dumb things down... to me.
> 
>  
>
>>It would also be helpful if a current assessment of the speed of 
>>numarray could be published.  
>>    
>>
>
>You asked... and...  (sorry if this is less than succinct)
>
>The numarray source distribution has a module called bench.py in the
>Examples directory.  Without any analysis,  I've attached the output
>from bench.py on my 1.7 GHz Pentium IV running under i386 linux.  Each
>row shows the timing numbers for numarray and Numeric for a particular
>function call and parameter set;  the important part is the ratio ( < 1
>means numarray is faster.  In general, numarray is slower;  this is due
>to the fact that numarray supports in-place operation on memory mapped
>non-native data formats).
>
>Suspect numbers are... suspect.   Please investigate!  
>
>Caveats:
>
>1) The benchmarks were run on a multi-tasking kernel with no specific
>load.  The benchmarks also vary from platform to platform, processor to
>processor, compiler to compiler, etc.  These are gcc version 3.2
>20020903 (Red Hat Linux 8.0 3.2-7) on a 1.7 GHz P-IV with 384M of RAM.
>
>2) The absolute measured units (microseconds) haven't really been
>validated and shouldn't be taken too seriously.  The relative time
>comparisons should be valid.
>
>3) a,c,d are 1D arrays.  b is a 2D array.
>
>4) the 10**x column shows the logarithmic array size. 
>   2 --> 100 elements, 6 --> 1 million elements
>
>5) The version of bench.py which made these measurements is in CVS, not
>the numarray-0.7 tar-ball.
>
>Wish-list:
>
>1) The right way to eliminate the effects of host load on the
>benchmarks.  
>
>2) A way to create a regression test which includes performance.
>
>  
>
>>I gather that some enhancements arrived with version 0.7.
>>    
>>
>
>I don't recall any performance enhancements.  numarray-0.7 was mostly
>about object array support and bug fixes.  
>
>bench.py results from numarray-0.7 09-25-03:
>
>benchmark                10**x   numarray (usec)    Numeric (usec)  numarray:Numeric
>take(a,c)                    2              1477                25              56.8
>                             3              1592               108              14.7
>                             4              2920               989               2.9
>                             5             15568              9814               1.6
>                             6            136960             96253               1.4
>put(a,c,d)                   2              1673                64              26.1
>                             3              1798               195               9.2
>                             4              3270              1587               2.1
>                             5             16742             15334               1.1
>                             6            147768            151932               1.0
>transpose(b)                 2               154                22               6.8
>                             3               157                24               6.3
>                             4               190                39               4.8
>                             5               262                74               3.5
>                             6               257                67               3.8
>putmask(a,c%2,d)             2              1415               139              10.1
>                             3              1504               271               5.6
>                             4              2439              1735               1.4
>                             5             11036             17337               0.6
>                             6             93927            157848               0.6
>repeat(a,2)                  2              1533               138              11.1
>                             3             10165               579              17.5
>                             4            103064              4701              21.9
>                             5           1028850             58812              17.5
>                             6          10010859            493157              20.3
>choose(a%2,(a,c))            2              1312                73              18.0
>                             3              1389               190               7.3
>                             4              2479              1500               1.7
>                             5             12516             16219               0.8
>                             6            108841            142280               0.8
>ravel(b)                     2               164                38               4.3
>                             3               169                41               4.1
>                             4               196                58               3.4
>                             5               279                93               3.0
>                             6               275                90               3.0
>nonzero(a)                   2              1044               143               7.3
>                             3              1133               285               4.0
>                             4              2108              1929               1.1
>                             5             12002             17974               0.7
>                             6            111598            176598               0.6
>where(a%2,a,c)               2              1273               111              11.4
>                             3              1380               242               5.7
>                             4              2587              1752               1.5
>                             5             13368             16785               0.8
>                             6            117731            164187               0.7
>compress(a%2,a)              2              2489               198              12.6
>                             3              2661               375               7.1
>                             4              4575              2426               1.9
>                             5             22933             22621               1.0
>                             6            200138            219215               0.9
>diagonal(b)                  2              2420             16883               0.1
>                             3              2425             16896               0.1
>                             4              2591             16932               0.2
>                             5              2588             16875               0.2
>                             6              2579             16861               0.2
>trace(b)                     2              2611             16988               0.2
>                             3              2625             16990               0.2
>                             4              2750             17004               0.2
>                             5              2758             17048               0.2
>                             6              2766             16942               0.2
>searchsorted(a,a)            2               506                55               9.2
>                             3               708               490               1.4
>                             4              3054              6507               0.5
>                             5             32101             77762               0.4
>                             6            378341            940912               0.4
>sort(a)                      2               554                85               6.4
>                             3               940               365               2.6
>                             4              5394              3533               1.5
>                             5             55238             36049               1.5
>                             6            607845            420334               1.4
>argsort(a)                   2              1541                90              16.9
>                             3              2061               467               4.4
>                             4              8113              5130               1.6
>                             5             76924             49497               1.6
>                             6            853114            588399               1.4
>argmax(b)                    2           1112081              8692             127.9
>                             3           1110896              9320             119.2
>                             4           1108330              8706             127.3
>                             5           1113449              8720             127.7
>                             6           1112556              8743             127.3
>array(s,typecode='i')        2               229               146               1.6
>                             3               274              1103               0.2
>                             4               764             10900               0.1
>                             5              5450            105774               0.1
>                             6             51500           1253819               0.0
>dot(b,b)                     2           8825828          31402493               0.3
>                             3           8804230          32136602               0.3
>                             4           8748239          31123564               0.3
>                             5           8811900          32910953               0.3
>                             6           8827314          31784649               0.3
>clip(a,50,100)               2              2920               177              16.4
>                             3              3089               370               8.3
>                             4              5138              2471               2.1
>                             5             24340             24737               1.0
>                             6            211277            245540               0.9
>indices((100,100))           2              5253              2696               1.9
>                             3              5230              2634               2.0
>                             4              5290              2687               2.0
>                             5              5383              2732               2.0
>                             6              5558              2733               2.0
>swapaxes(b,0,-1)             2                77                79               1.0
>                             3                77                82               1.0
>                             4                94               106               0.9
>                             5               154               142               1.1
>                             6               146               144               1.0
>concatenate((a,a,a))         2              1021                35              29.1
>                             3              1082                48              22.1
>                             4              1868               431               4.3
>                             5              8843              4945               1.8
>                             6             75181             47495               1.6
>innerproduct(b,b)            2           8719124           8989714               1.0
>                             3           8662431           8840564               1.0
>                             4           8734306           9033604               1.0
>                             5           8694994           8935343               1.0
>                             6           8680599           8869750               1.0
>outerproduct(b[0],b[0])      2             42320             27598               1.5
>                             3             42299             27802               1.5
>                             4             42842             27935               1.5
>                             5             42346             27822               1.5
>                             6             42257             27709               1.5
>resize(a,(2*10**i,))         2              1708               160              10.6
>                             3              1769               171              10.3
>                             4              2648               468               5.6
>                             5              9778              3515               2.8
>                             6             75513             31778               2.4
>a[0]                         2                10                10               1.0
>                             3                10                10               1.0
>                             4                13                12               1.1
>                             5                25                20               1.3
>                             6                25                22               1.1
>b[0]                         2                46                15               3.1
>                             3                48                17               2.8
>                             4                56                18               3.0
>                             5               106                33               3.1
>                             6               100                35               2.9
>a[1000:2000]                 2               303                16              18.0
>                             3               298                15              18.7
>                             4               275                20              13.8
>                             5               354                33              10.4
>                             6               344                34               9.8
>b[400:600,400:600]           2               320                25              12.8
>                             3               317                25              12.2
>                             4               348                30              11.3
>                             5               432                46               9.2
>                             6               418                47               8.7
>arange(10.**i)               2               411                37              11.1
>                             3               448                74               6.0
>                             4               923               570               1.6
>                             5              5630              5592               1.0
>                             6             50735             54222               0.9
>identity(i+2)                2              3147               226              13.9
>                             3              3158               227              13.9
>                             4              3214               258              12.4
>                             5              3311               294              11.3
>                             6              3316               291              11.4
>add(a,c)                     2                76                27               2.7
>                             3                90                40               2.2
>                             4               313               225               1.4
>                             5              2902              2128               1.4
>                             6             28106             19972               1.4
>add(a,c,d)                   2                40                22               1.9
>                             3                46                33               1.4
>                             4               144               132               1.1
>                             5              1128              1165               1.0
>                             6             10316             10827               1.0
>a+c                          2                91                31               2.9
>                             3               105                45               2.4
>                             4               335               231               1.5
>                             5              2945              2130               1.4
>                             6             28102             19744               1.4
>cos(a)                       2                84                53               1.6
>                             3               212               235               0.9
>                             4              1546              2351               0.7
>                             5             15237             23074               0.7
>                             6            151604            227898               0.7
>add.reduce(a)                2                97                31               3.0
>                             3               109                41               2.7
>                             4               207               123               1.7
>                             5              1062               949               1.1
>                             6              8974              8844               1.0
>add.reduce(b,0)              2             32527              9218               3.5
>                             3             31652              9238               3.4
>                             4             31376              9171               3.4
>                             5             31412              9271               3.4
>                             6             30443              9263               3.3
>add.reduce(b,-1)             2              9065              9389               1.0
>                             3              9075              9360               1.0
>                             4              9102              9271               1.0
>                             5              9168              9389               1.0
>                             6              9148              9394               1.0
>add.accumulate(a)            2               100                28               3.5
>                             3               120                41               2.9
>                             4               416               254               1.6
>                             5              3584              2696               1.3
>                             6             34837             25608               1.4
>add.accumulate(b,0)          2             63947             25673               2.5
>                             3             65875             25691               2.6
>                             4             65540             25708               2.5
>                             5             65161             25776               2.5
>                             6             66539             25646               2.6
>add.accumulate(b,1)          2             35048             25756               1.4
>                             3             35022             25738               1.4
>                             4             35048             25688               1.4
>                             5             35092             25761               1.4
>                             6             35110             25686               1.4
>
>
>
>
>-------------------------------------------------------
>This sf.net email is sponsored by:ThinkGeek
>Welcome to geek heaven.
>http://thinkgeek.com/sf
>_______________________________________________
>Numpy-discussion mailing list
>Numpy-discussion at lists.sourceforge.net
>https://lists.sourceforge.net/lists/listinfo/numpy-discussion
>
>  
>





From jmiller at stsci.edu  Thu Sep 25 12:06:02 2003
From: jmiller at stsci.edu (Todd Miller)
Date: Thu Sep 25 12:06:02 2003
Subject: [Numpy-discussion] Subclassing numarray arrays
In-Reply-To: <3F7317A4.2040107@pfdubois.com>
References: <1064478298.10525.24.camel@Nadav.Envision.co.il> 
	<3F7317A4.2040107@pfdubois.com>
Message-ID: <1064516557.13807.157.camel@halloween.stsci.edu>

On Thu, 2003-09-25 at 12:28, Paul Dubois wrote:
> I have not followed this thread in any detail but I continue to be 
> puzzled by the idea of subclassing numarray arrays. Any subclass worth 
> its salt has an unpredictable constructor argument list, so that 
> virtually every ufunc and every array method have to be overridden in 
> order to return the subclass object rather than a numarray. 

I think there is a need to effectively subclass numarray where the
parameter lists don't change.  Doing a Matrix subclass sounds reasonable
to me and we should explore the OOP necessary to make it convenient. 
STSCI hasn't asked for it,  but a couple different numarray contributors
have.

> I gather 
> that the defers-to-me stuff allows the subclass to get control of a 
> mixed operation but that was possible with the radd, etc. operators 
> already as in the old MA.

radd does not work for numarray + MA.  Instead, the expression is
resolved as numarray.__add__(MA).  Were MA a numarray subclass, it would
work, given the facilities provided by Python alone.

The defers-to-me-stuff is a simple work around to avoid having to
reverse engineer this one Numeric behavior.

> This reasoning led me to believe that subclassing had virtually no value 
> and to the design of MA that has-a numeric array rather than is-a.

Since we want to port MA as-is to numarray, we're not changing that.

> Is there some big breakthrough here I missed?

No,  but there are two issues:  (1) effectively supporting subclasses
(2) effectively supporting interoperability with other classes.

(1) needs more discussion.
(2) is already solved.

-- 
Todd Miller 			jmiller at stsci.edu
STSCI / ESS / SSB




From cjw at sympatico.ca  Thu Sep 25 12:09:03 2003
From: cjw at sympatico.ca (Colin J. Williams)
Date: Thu Sep 25 12:09:03 2003
Subject: [Numpy-discussion] Crash with numarray testall.py
Message-ID: <3F733D14.5060200@sympatico.ca>

C:\Program Files\Python23\Lib\site-packages\numarray>python
Python 2.3.1 (#47, Sep 23 2003, 23:47:32) [MSC v.1200 32 bit (Intel)] on 
win32
Type "help", "copyright", "credits" or "license" for more information.
 >>> import testall

The 21K file created by Windows to contain information about the crash 
is not readable.

Windows XP and numarray 0.7 used.

Colin W.




From jmiller at stsci.edu  Thu Sep 25 13:20:05 2003
From: jmiller at stsci.edu (Todd Miller)
Date: Thu Sep 25 13:20:05 2003
Subject: [Numpy-discussion] Subclassing numarray arrays
In-Reply-To: <3F731467.1070402@sympatico.ca>
References: <1064478298.10525.24.camel@Nadav.Envision.co.il>
		<3F72DC68.6030403@sympatico.ca>
	<1064499880.13807.111.camel@halloween.stsci.edu> 
	<3F731467.1070402@sympatico.ca>
Message-ID: <1064521138.13806.229.camel@halloween.stsci.edu>

On Thu, 2003-09-25 at 12:14, Colin J. Williams wrote:
> Todd,
> 
> Thanks for the benchmark data.

Thanks for the references.  They suggest that the classmethod approach
we've already tried for folding in the numarray factory functions is
reasonable OOP.
 
> I intend to post the matrix stuff.  Performance is not a problem for me 
> at this stage.
> 
> Currently, I'm plodding along, making things work.  Later, I'll post it 
> with some rudimentary documentation.
> 
> Regarding the "new-classes standard", I can't find much in either the 
> Python docs or Alex Martelli's "Python in a Nutshell".  One of the new 
> features is a __new__ constructor.

Well,  we've certainly got those (__new__ methods).  They don't match
the __init__ signature though,  nor was I aware that they should.

> The Python array module doesn't formally have a class and thus it can't 
> be sub-classed.  It uses a factory function to create instance objects.
> 
> Martelli, in PIAN p74, writes of factory functions being useful when one 
> wishes to create instances of different classes depending on some 
> condition or for the reuse of an existing instance.
> 
> In the Python Cookbook p173, he proposes the use of a factory function 
> to get around the limitations of Python 2.1 and earlier.  On page 179, 
> he refers to the use of a factory method, again to work around 
> limitations before Python 2.2.
> 
> My rough and ready definition would include the ability to create an 
> instance and initialize
> the data in one step.
> 

One thing to keep in mind is that NumArray was designed to run on
Python-2.0.  As Python-2.2 became widely available and stable, we
realized we could exploit it for better C-API compatibility with
Numeric, and for better speed.   

I think what you and later Nadav have pointed out is that we can exploit
Python-2.2 for better sub-classing as well.  We just haven't done it
yet.  Concrete proposals are welcome.

Regards,
Todd





From jmiller at stsci.edu  Fri Sep 26 06:41:12 2003
From: jmiller at stsci.edu (Todd Miller)
Date: Fri Sep 26 06:41:12 2003
Subject: [Numpy-discussion] numarray unfriendly to user defined types
In-Reply-To: <3F722F0B.7070609@ieee.org>
References: <3F6A3D4C.3070703@ieee.org>	
	<1064266268.9234.351.camel@halloween.stsci.edu> <3F6F7744.80908@ieee.org>	
	<3F6F7B83.3040305@colorado.edu>  <3F6F8546.9020500@ieee.org>	
	<1064313211.3460.160.camel@localhost.localdomain>	
	<3F721A4C.7080300@ieee.org>
	<1064444385.23878.14.camel@localhost.localdomain> 
	<3F722F0B.7070609@ieee.org>
Message-ID: <1064583557.13805.293.camel@halloween.stsci.edu>

I tried out DeferredToByNumArray yesterday morning and was able to get
it to work as planned with numarray and MA.  So yay!

Yesterday afternoon,  I stopped by Perry's office for a final check, and
not surprisingly,  there are some aspects of this solution that we've
overlooked.  So not yay!

Perry pointed out two ways that the DeferredToByNumArray scheme may fall
apart as our inheritance hierarchy becomes richer:

1)  Imagine not one, but two independent subclasses of NumArray, each of
which want NumArray to defer to them.  Now imagine them trying to
inter-operate with each other.  In this case, the meaning of the
expression is determined by the order of the operands, so A+B and B+A
will either return an A or a B depending on the expression order.

2)  Imagine again two subclasses of numarray, but this time imagine B as
a subclass of A.  Here again, both might want to defer to numarray, and
again, A+B and B+A return different types driven by the type order of
the expression.

I don't have a solution yet, but am hopeful that more free candy will
fall from the sky... or YAGNI.  Perry pointed out that similar problems
exist for *any* Python class hierarchy, so we're not alone, and perhaps
should forget about this until it matters.  If someone sees an easy
fix,  now would be better than later.

Todd
On Wed, 2003-09-24 at 19:55, Tim Hochberg wrote:
> Todd Miller wrote:
> 
> >On Wed, 2003-09-24 at 18:27, Tim Hochberg wrote:
> >  
> >
> >>Hi Todd,
> >>
> >>There are three ways to spell "defer to me" on the table (the precise 
> >>details of
> >>each spelling are, of course, still open for debate):
> >>
> >>   1.  numarray.defer_to(my_class)
> >>
> >>   2. class ArrayLike(numarray.DeferTo):
> >>         # ...
> >>
> >>   3. class ArrayLike:
> >>         _numarray_defer_to = True
> >>        # ...
> >>
> >>I'd prefer a non-registration solution since those are both 
> >>aesthetically displeasing and leave you open to the situation where a 
> >>class in module A gets registered by module B, but module C expects it 
> >>not to be registered and everything breaks. Not all that likely, I 
> >>admit, but let's avoid the registration version if we can.
> >>    
> >>
> >
> >I was picturing this as module A registering it's own classes only. 
> >Nevertheless, inverting the problem and distributing the registration as
> >you suggested is better.
> >  
> >
> 
> The case you describe probably will describe the majority of actual use 
> cases, and in fact describes mine. I'm trying to think ahead a bit to 
> cases may encounter as start using NumArray more extensively. Let's hope 
> this solution still looks good in six months!
> 
> >>The other two solutions are almost equivalent. The one case where 3 has 
> >>an edge over 2 is if I have an object (not a class), I could potentially 
> >>set a _numarray_defer_to on the object before passing it to numarray 
> >>without having to mess with the class
> >>of the object at all. YAGNI, though.
> >>    
> >>
> >
> >I was more concerned about the potential impact of lots of multiple
> >inheritance, but that's probably just my own personal blend of FUD.
> >
> >  
> >
> >>The advantage of 2 in my view is that it *does* force you to subclass. 
> >>With 3, there will be the temptation to poke into some other  module and 
> >>set _numarray_defer_to on some poor unsuspecting class. This has the 
> >>same disadvantage as 1, that it could confuse some other poor 
> >>unsuspecting module. The correct way to do get a deferred class from a 
> >>third party module is to import and subclass. This works with either 2 
> >>or 3::
> >>
> >>  import A
> >>
> >>  class Klass2(a.Klass, numarray.DeferTo):   #2
> >>     #...
> >>
> >>  class Klass3(a.Klass): #3 the good way
> >>      _numarray_defer_to = True
> >>     # ...
> >>
> >>   A.Klass._numarray_defer_to = True #3 the evil way.
> >>
> >> Version 2 is cleaner and encourages you to do the right thing, so I'd 
> >>prefer that solution.
> >>
> >>    
> >>
> >
> >Good enough for me.  If no one else has any comments, then
> >numarray.DeferTo is where I'll start implementing.  Tomorrow.
> >  
> >
> 
> One more minor thing. I'm not sure tha DeferTo is ideal as the mix-in 
> class name. It was perfect for the registration function name, but I'm 
> not sure it's so clear whether the class or numarray is being deferred 
> to when you say numarray.DeferTo. DeferToMe is more descriptive, but 
> seems sort of slangy. DeferredTo is better than DeferTo, but still not 
> as descriptive as DeferToMe. numarray.DefersTo reads perfect as long as 
> numarray is included but is a disaster if you read it on your own.  
> Below I've put down all the ideas I could come up with
> 
> class CustomArray(numarray.DeferTo)
> class CustomArray(numarray.DefersTo)
> class CustomArray(numarray.DeferredTo)
> class CustomArray(numarray.DeferToMe)
> class CustomArray(numarray.DeferredToByNumarray)
> class CustomArray(DeferTo)
> class CustomArray(DefersTo)
> class CustomArray(DeferredTo)
> class CustomArray(DeferToMe)
> class CustomArray(DeferredToByNumarray)
> 
> For me it's a toss up between DefferedTo, DeferToMe and 
> DeferredToByNumarray. The first is a little lacking in descriptive 
> power, the second is slangy and the third is wordy.
> 
> -tim
> 
> [not that this matters much....]
> 
> 
> >>regards,
> >>
> >>-tim
> >>    
> >>
> >
> >Thanks again,
> >Todd
> >
> >
> >
> >
> >
> >  
> >
> 
> 
> 
> 
> 
> -------------------------------------------------------
> This sf.net email is sponsored by:ThinkGeek
> Welcome to geek heaven.
> http://thinkgeek.com/sf
> _______________________________________________
> Numpy-discussion mailing list
> Numpy-discussion at lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/numpy-discussion
-- 
Todd Miller 			jmiller at stsci.edu
STSCI / ESS / SSB




From tim.hochberg at ieee.org  Fri Sep 26 10:14:07 2003
From: tim.hochberg at ieee.org (Tim Hochberg)
Date: Fri Sep 26 10:14:07 2003
Subject: [Numpy-discussion] numarray unfriendly to user defined types
In-Reply-To: <1064583557.13805.293.camel@halloween.stsci.edu>
References: <3F6A3D4C.3070703@ieee.org>		<1064266268.9234.351.camel@halloween.stsci.edu> <3F6F7744.80908@ieee.org>		<3F6F7B83.3040305@colorado.edu>  <3F6F8546.9020500@ieee.org>		<1064313211.3460.160.camel@localhost.localdomain>		<3F721A4C.7080300@ieee.org>	<1064444385.23878.14.camel@localhost.localdomain> 	<3F722F0B.7070609@ieee.org> <1064583557.13805.293.camel@halloween.stsci.edu>
Message-ID: <3F746764.2070403@ieee.org>

Todd Miller wrote:

>I tried out DeferredToByNumArray yesterday morning and was able to get
>it to work as planned with numarray and MA.  So yay!
>  
>
Excellent.

>Yesterday afternoon,  I stopped by Perry's office for a final check, and
>not surprisingly,  there are some aspects of this solution that we've
>overlooked.  So not yay!
>
>Perry pointed out two ways that the DeferredToByNumArray scheme may fall
>apart as our inheritance hierarchy becomes richer:
>  
>
Not so excellent.

>1)  Imagine not one, but two independent subclasses of NumArray, each of
>which want NumArray to defer to them.  Now imagine them trying to
>inter-operate with each other.  In this case, the meaning of the
>expression is determined by the order of the operands, so A+B and B+A
>will either return an A or a B depending on the expression order.
>  
>
I think that this is YAGNI right now, but may well be an issue is 
"proper" subclassing get's implemented. So we may as well think about it 
a bit. I call it YAGNI now because these subclasses either use the 
default __op__ methods, which presently have the same behaviour 
regardless of the subclasses (they always return NumArrays) or they 
define there own __op__ methods in which case they can worry about it ;> 
If NumArray starts to become more friendly to subclassing, this will 
probably change though.

>2)  Imagine again two subclasses of numarray, but this time imagine B as
>a subclass of A.  Here again, both might want to defer to numarray, and
>again, A+B and B+A return different types driven by the type order of
>the expression.
>  
>

I'd make the same comment for this case as for the above case except 
that all else being equal, subclasses should prevail over superclasses.

>I don't have a solution yet, but am hopeful that more free candy will
>fall from the sky... or YAGNI.  Perry pointed out that similar problems
>exist for *any* Python class hierarchy, so we're not alone, and perhaps
>should forget about this until it matters.  If someone sees an easy
>fix,  now would be better than later.
>  
>
Here's some quick thoughts. In order for this to work out sensibly you 
want the order between classes to be transistive and you probably also 
always want to be able to insert a class into the order between any two 
classes  This is equivalent to deciding the op priority based on a 
comparison based on some real number associated with each class. This 
leads to something like:

   
class NumArrayFamily:
    op_priority = 0.0
   
# This makes it compatible with the current proposal , but isn't 
strictly necessary.
class DeferredToByNumarray(NumArrayFamily):
    op_priority = 1000.0
   
class NumArrary(..., NumArrayFamily):
    # ...
    def __add__(self, object):
        if isinstance(object, NumArrayFamily):
            if ((object.op_priority > self.object.op_priority) or
                (object.op_priority == self.object.op_priority) and 
issubclass(object, self.__class__)):
                    return other.__radd__(self)
        return ufunc.add(self, other)


class MySubclassOfNumarray(NumArray):
    op_priority = 10.0 # deffered to by NumArray, but defers to 
DeferredToByNumarray classes.



Of course the user still has to be careful to make sure the priority 
order makes sense and you could have big problems when combining 
packages with disprate ideas about priority levels, but I'm not sure how 
you can get away from that.

-tim



>Todd
>On Wed, 2003-09-24 at 19:55, Tim Hochberg wrote:
>  
>
>>Todd Miller wrote:
>>
>>    
>>
>>>On Wed, 2003-09-24 at 18:27, Tim Hochberg wrote:
>>> 
>>>
>>>      
>>>
>>>>Hi Todd,
>>>>
>>>>There are three ways to spell "defer to me" on the table (the precise 
>>>>details of
>>>>each spelling are, of course, still open for debate):
>>>>
>>>>  1.  numarray.defer_to(my_class)
>>>>
>>>>  2. class ArrayLike(numarray.DeferTo):
>>>>        # ...
>>>>
>>>>  3. class ArrayLike:
>>>>        _numarray_defer_to = True
>>>>       # ...
>>>>
>>>>I'd prefer a non-registration solution since those are both 
>>>>aesthetically displeasing and leave you open to the situation where a 
>>>>class in module A gets registered by module B, but module C expects it 
>>>>not to be registered and everything breaks. Not all that likely, I 
>>>>admit, but let's avoid the registration version if we can.
>>>>   
>>>>
>>>>        
>>>>
>>>I was picturing this as module A registering it's own classes only. 
>>>Nevertheless, inverting the problem and distributing the registration as
>>>you suggested is better.
>>> 
>>>
>>>      
>>>
>>The case you describe probably will describe the majority of actual use 
>>cases, and in fact describes mine. I'm trying to think ahead a bit to 
>>cases may encounter as start using NumArray more extensively. Let's hope 
>>this solution still looks good in six months!
>>
>>    
>>
>>>>The other two solutions are almost equivalent. The one case where 3 has 
>>>>an edge over 2 is if I have an object (not a class), I could potentially 
>>>>set a _numarray_defer_to on the object before passing it to numarray 
>>>>without having to mess with the class
>>>>of the object at all. YAGNI, though.
>>>>   
>>>>
>>>>        
>>>>
>>>I was more concerned about the potential impact of lots of multiple
>>>inheritance, but that's probably just my own personal blend of FUD.
>>>
>>> 
>>>
>>>      
>>>
>>>>The advantage of 2 in my view is that it *does* force you to subclass. 
>>>>With 3, there will be the temptation to poke into some other  module and 
>>>>set _numarray_defer_to on some poor unsuspecting class. This has the 
>>>>same disadvantage as 1, that it could confuse some other poor 
>>>>unsuspecting module. The correct way to do get a deferred class from a 
>>>>third party module is to import and subclass. This works with either 2 
>>>>or 3::
>>>>
>>>> import A
>>>>
>>>> class Klass2(a.Klass, numarray.DeferTo):   #2
>>>>    #...
>>>>
>>>> class Klass3(a.Klass): #3 the good way
>>>>     _numarray_defer_to = True
>>>>    # ...
>>>>
>>>>  A.Klass._numarray_defer_to = True #3 the evil way.
>>>>
>>>>Version 2 is cleaner and encourages you to do the right thing, so I'd 
>>>>prefer that solution.
>>>>
>>>>   
>>>>
>>>>        
>>>>
>>>Good enough for me.  If no one else has any comments, then
>>>numarray.DeferTo is where I'll start implementing.  Tomorrow.
>>> 
>>>
>>>      
>>>
>>One more minor thing. I'm not sure tha DeferTo is ideal as the mix-in 
>>class name. It was perfect for the registration function name, but I'm 
>>not sure it's so clear whether the class or numarray is being deferred 
>>to when you say numarray.DeferTo. DeferToMe is more descriptive, but 
>>seems sort of slangy. DeferredTo is better than DeferTo, but still not 
>>as descriptive as DeferToMe. numarray.DefersTo reads perfect as long as 
>>numarray is included but is a disaster if you read it on your own.  
>>Below I've put down all the ideas I could come up with
>>
>>class CustomArray(numarray.DeferTo)
>>class CustomArray(numarray.DefersTo)
>>class CustomArray(numarray.DeferredTo)
>>class CustomArray(numarray.DeferToMe)
>>class CustomArray(numarray.DeferredToByNumarray)
>>class CustomArray(DeferTo)
>>class CustomArray(DefersTo)
>>class CustomArray(DeferredTo)
>>class CustomArray(DeferToMe)
>>class CustomArray(DeferredToByNumarray)
>>
>>For me it's a toss up between DefferedTo, DeferToMe and 
>>DeferredToByNumarray. The first is a little lacking in descriptive 
>>power, the second is slangy and the third is wordy.
>>
>>-tim
>>
>>[not that this matters much....]
>>
>>
>>    
>>
>>>>regards,
>>>>
>>>>-tim
>>>>   
>>>>
>>>>        
>>>>
>>>Thanks again,
>>>Todd
>>>
>>>
>>>
>>>
>>>
>>> 
>>>
>>>      
>>>
>>
>>
>>
>>-------------------------------------------------------
>>This sf.net email is sponsored by:ThinkGeek
>>Welcome to geek heaven.
>>http://thinkgeek.com/sf
>>_______________________________________________
>>Numpy-discussion mailing list
>>Numpy-discussion at lists.sourceforge.net
>>https://lists.sourceforge.net/lists/listinfo/numpy-discussion
>>    
>>






From tim.hochberg at ieee.org  Fri Sep 26 10:14:08 2003
From: tim.hochberg at ieee.org (Tim Hochberg)
Date: Fri Sep 26 10:14:08 2003
Subject: [Numpy-discussion] numarray incompatibility: searchsorted
Message-ID: <3F747208.6020401@ieee.org>

There are some compatibility and doc issues and perhaps a bug in 
numarray.searchsorted. The compatibility issue is that 
Numeric.searchsorted(a, v) accepts either a sequence or scalar value for 
v. Numarray.searchsorted accepts only sequence values.

Second, the doc issue. The docstring for numarray.searchsorted states::

  searchsorted(bins, values)
    searchsort(bins, values) returns the array of greatest indices 'i'
    such that each values[i] <= bins[i].

I assume that should really read something more like::

  searchsorted(bins, values)
    searchsort(bins, values) returns the array A[j] of greatest indices 'i'
    such that each values[j] <= bins[i].

Third, the possible bug:

# na = numarray, np = NumPy
 >>> na.searchsorted([1,2,3,4], [2.5, 3.5])
array([1, 2])
 >>> np.searchsorted([1,2,3,4], [2.5, 3.5])
array([2, 3])

Hmmm. It appears that numarray result does match the numarray docs, (at 
least as I read them), but I like the Numeric behaviour better.  The 
Numeric behaviour also matches the behaviour of the bisect module, which 
is described as::

    bisect = bisect_right(a, x, lo=0, hi=None)
        Return the index where to insert item x in list a, assuming a is 
sorted.

        The return value i is such that all e in a[:i] have e <= x, and 
all e in
        a[i:] have e > x.  So if x already appears in the list, i points 
just
        beyond the rightmost x already there.

        Optional args lo (default 0) and hi (default len(a)) bound the
        slice of a to be searched.

I'd recomend matching the behaviour of the two existing modules (bisect 
and Numeric).

-tim








From jmiller at stsci.edu  Fri Sep 26 11:08:01 2003
From: jmiller at stsci.edu (Todd Miller)
Date: Fri Sep 26 11:08:01 2003
Subject: [Numpy-discussion] numarray incompatibility: searchsorted
In-Reply-To: <3F747208.6020401@ieee.org>
References: <3F747208.6020401@ieee.org>
Message-ID: <1064599467.1021.2.camel@halloween.stsci.edu>

I logged this on Source Forge.  Thanks for the comments.

Todd

On Fri, 2003-09-26 at 13:06, Tim Hochberg wrote:
> 
> There are some compatibility and doc issues and perhaps a bug in 
> numarray.searchsorted. The compatibility issue is that 
> Numeric.searchsorted(a, v) accepts either a sequence or scalar value for 
> v. Numarray.searchsorted accepts only sequence values.
> 
> Second, the doc issue. The docstring for numarray.searchsorted states::
> 
>   searchsorted(bins, values)
>     searchsort(bins, values) returns the array of greatest indices 'i'
>     such that each values[i] <= bins[i].
> 
> I assume that should really read something more like::
> 
>   searchsorted(bins, values)
>     searchsort(bins, values) returns the array A[j] of greatest indices 'i'
>     such that each values[j] <= bins[i].
> 
> Third, the possible bug:
> 
> # na = numarray, np = NumPy
>  >>> na.searchsorted([1,2,3,4], [2.5, 3.5])
> array([1, 2])
>  >>> np.searchsorted([1,2,3,4], [2.5, 3.5])
> array([2, 3])
> 
> Hmmm. It appears that numarray result does match the numarray docs, (at 
> least as I read them), but I like the Numeric behaviour better.  The 
> Numeric behaviour also matches the behaviour of the bisect module, which 
> is described as::
> 
>     bisect = bisect_right(a, x, lo=0, hi=None)
>         Return the index where to insert item x in list a, assuming a is 
> sorted.
> 
>         The return value i is such that all e in a[:i] have e <= x, and 
> all e in
>         a[i:] have e > x.  So if x already appears in the list, i points 
> just
>         beyond the rightmost x already there.
> 
>         Optional args lo (default 0) and hi (default len(a)) bound the
>         slice of a to be searched.
> 
> I'd recomend matching the behaviour of the two existing modules (bisect 
> and Numeric).
> 
> -tim
> 
> 
> 
> 
> 
> 
> 
> -------------------------------------------------------
> This sf.net email is sponsored by:ThinkGeek
> Welcome to geek heaven.
> http://thinkgeek.com/sf
> _______________________________________________
> Numpy-discussion mailing list
> Numpy-discussion at lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/numpy-discussion
-- 
Todd Miller 			jmiller at stsci.edu
STSCI / ESS / SSB




From jmiller at stsci.edu  Fri Sep 26 11:24:09 2003
From: jmiller at stsci.edu (Todd Miller)
Date: Fri Sep 26 11:24:09 2003
Subject: [Numpy-discussion] numarray unfriendly to user defined types
In-Reply-To: <3F74780D.3070701@sympatico.ca>
References: 	<3F6A3D4C.3070703@ieee.org>		<1064266268.9234.351.camel@halloween.stsci.edu>
	 <3F6F7744.80908@ieee.org>		<3F6F7B83.3040305@colorado.edu> 
	<3F6F8546.9020500@ieee.org>		<1064313211.3460.160.camel@localhost.localdomai
	n>		<3F721A4C.7080300@ieee.org>	<1064444385.23878.14.camel@localhost.localdo
	main> 	<3F722F0B.7070609@ieee.org>
	<1064583557.13805.293.camel@halloween.stsci.edu> 
	<3F74780D.3070701@sympatico.ca>
Message-ID: <1064600606.1021.17.camel@halloween.stsci.edu>

On Fri, 2003-09-26 at 13:31, Colin J. Williams wrote:
> Todd,
> 
> This scheme seems rather complex.  The purpose isn't clear to me.
> 
> What is the problem which this arrangement is intended to solve?

Bottom line, currently:  
	
numarray + another_class --> exception

It would be nice if:  

numarray + another_class --> another_class.__radd__(numarray)

Also, look backward at Tim's original post on this subject.

> 
> Could I suggest that some scheme along the lines of the Python PEP's be 
> used to set out the Why's and Wherefore's?
> 
Sounds good for next time.

> It might also be useful to copy this discussion to comp.lang.python as 
> that newsgroup has people experienced with classes.  

OK,  I'll stop being a chicken.  Everyone, get ready for the sound of a
pin dropping on c.l.py.

> The problem appears 
> to be Python related rather than a problem of numerical analysis.

Yes indeed.

> 
> Colin W.
> 
> Todd Miller wrote:
> 
> >I tried out DeferredToByNumArray yesterday morning and was able to get
> >it to work as planned with numarray and MA.  So yay!
> >
> >Yesterday afternoon,  I stopped by Perry's office for a final check, and
> >not surprisingly,  there are some aspects of this solution that we've
> >overlooked.  So not yay!
> >
> >Perry pointed out two ways that the DeferredToByNumArray scheme may fall
> >apart as our inheritance hierarchy becomes richer:
> >
> >1)  Imagine not one, but two independent subclasses of NumArray, each of
> >which want NumArray to defer to them.  Now imagine them trying to
> >inter-operate with each other.  In this case, the meaning of the
> >expression is determined by the order of the operands, so A+B and B+A
> >will either return an A or a B depending on the expression order.
> >
> >2)  Imagine again two subclasses of numarray, but this time imagine B as
> >a subclass of A.  Here again, both might want to defer to numarray, and
> >again, A+B and B+A return different types driven by the type order of
> >the expression.
> >
> >I don't have a solution yet, but am hopeful that more free candy will
> >fall from the sky... or YAGNI.  Perry pointed out that similar problems
> >exist for *any* Python class hierarchy, so we're not alone, and perhaps
> >should forget about this until it matters.  If someone sees an easy
> >fix,  now would be better than later.
> >
> >Todd
> >On Wed, 2003-09-24 at 19:55, Tim Hochberg wrote:
> >  
> >
> >>Todd Miller wrote:
> >>
> >>    
> >>
> >>>On Wed, 2003-09-24 at 18:27, Tim Hochberg wrote:
> >>> 
> >>>
> >>>      
> >>>
> >>>>Hi Todd,
> >>>>
> >>>>There are three ways to spell "defer to me" on the table (the precise 
> >>>>details of
> >>>>each spelling are, of course, still open for debate):
> >>>>
> >>>>  1.  numarray.defer_to(my_class)
> >>>>
> >>>>  2. class ArrayLike(numarray.DeferTo):
> >>>>        # ...
> >>>>
> >>>>  3. class ArrayLike:
> >>>>        _numarray_defer_to = True
> >>>>       # ...
> >>>>
> >>>>I'd prefer a non-registration solution since those are both 
> >>>>aesthetically displeasing and leave you open to the situation where a 
> >>>>class in module A gets registered by module B, but module C expects it 
> >>>>not to be registered and everything breaks. Not all that likely, I 
> >>>>admit, but let's avoid the registration version if we can.
> >>>>   
> >>>>
> >>>>        
> >>>>
> >>>I was picturing this as module A registering it's own classes only. 
> >>>Nevertheless, inverting the problem and distributing the registration as
> >>>you suggested is better.
> >>> 
> >>>
> >>>      
> >>>
> >>The case you describe probably will describe the majority of actual use 
> >>cases, and in fact describes mine. I'm trying to think ahead a bit to 
> >>cases may encounter as start using NumArray more extensively. Let's hope 
> >>this solution still looks good in six months!
> >>
> >>    
> >>
> >>>>The other two solutions are almost equivalent. The one case where 3 has 
> >>>>an edge over 2 is if I have an object (not a class), I could potentially 
> >>>>set a _numarray_defer_to on the object before passing it to numarray 
> >>>>without having to mess with the class
> >>>>of the object at all. YAGNI, though.
> >>>>   
> >>>>
> >>>>        
> >>>>
> >>>I was more concerned about the potential impact of lots of multiple
> >>>inheritance, but that's probably just my own personal blend of FUD.
> >>>
> >>> 
> >>>
> >>>      
> >>>
> >>>>The advantage of 2 in my view is that it *does* force you to subclass. 
> >>>>With 3, there will be the temptation to poke into some other  module and 
> >>>>set _numarray_defer_to on some poor unsuspecting class. This has the 
> >>>>same disadvantage as 1, that it could confuse some other poor 
> >>>>unsuspecting module. The correct way to do get a deferred class from a 
> >>>>third party module is to import and subclass. This works with either 2 
> >>>>or 3::
> >>>>
> >>>> import A
> >>>>
> >>>> class Klass2(a.Klass, numarray.DeferTo):   #2
> >>>>    #...
> >>>>
> >>>> class Klass3(a.Klass): #3 the good way
> >>>>     _numarray_defer_to = True
> >>>>    # ...
> >>>>
> >>>>  A.Klass._numarray_defer_to = True #3 the evil way.
> >>>>
> >>>>Version 2 is cleaner and encourages you to do the right thing, so I'd 
> >>>>prefer that solution.
> >>>>
> >>>>   
> >>>>
> >>>>        
> >>>>
> >>>Good enough for me.  If no one else has any comments, then
> >>>numarray.DeferTo is where I'll start implementing.  Tomorrow.
> >>> 
> >>>
> >>>      
> >>>
> >>One more minor thing. I'm not sure tha DeferTo is ideal as the mix-in 
> >>class name. It was perfect for the registration function name, but I'm 
> >>not sure it's so clear whether the class or numarray is being deferred 
> >>to when you say numarray.DeferTo. DeferToMe is more descriptive, but 
> >>seems sort of slangy. DeferredTo is better than DeferTo, but still not 
> >>as descriptive as DeferToMe. numarray.DefersTo reads perfect as long as 
> >>numarray is included but is a disaster if you read it on your own.  
> >>Below I've put down all the ideas I could come up with
> >>
> >>class CustomArray(numarray.DeferTo)
> >>class CustomArray(numarray.DefersTo)
> >>class CustomArray(numarray.DeferredTo)
> >>class CustomArray(numarray.DeferToMe)
> >>class CustomArray(numarray.DeferredToByNumarray)
> >>class CustomArray(DeferTo)
> >>class CustomArray(DefersTo)
> >>class CustomArray(DeferredTo)
> >>class CustomArray(DeferToMe)
> >>class CustomArray(DeferredToByNumarray)
> >>
> >>For me it's a toss up between DefferedTo, DeferToMe and 
> >>DeferredToByNumarray. The first is a little lacking in descriptive 
> >>power, the second is slangy and the third is wordy.
> >>
> >>-tim
> >>
> >>[not that this matters much....]
> >>
> >>
> >>    
> >>
> >>>>regards,
> >>>>
> >>>>-tim
> >>>>   
> >>>>
> >>>>        
> >>>>
> >>>Thanks again,
> >>>Todd
> >>>
> >>>
> >>>
> >>>
> >>>
> >>> 
> >>>
> >>>      
> >>>
> >>
> >>
> >>
> >>-------------------------------------------------------
> >>This sf.net email is sponsored by:ThinkGeek
> >>Welcome to geek heaven.
> >>http://thinkgeek.com/sf
> >>_______________________________________________
> >>Numpy-discussion mailing list
> >>Numpy-discussion at lists.sourceforge.net
> >>https://lists.sourceforge.net/lists/listinfo/numpy-discussion
> >>    
> >>
> 
> 
-- 
Todd Miller 			jmiller at stsci.edu
STSCI / ESS / SSB




From jochen at jochen-kuepper.de  Fri Sep 26 14:24:03 2003
From: jochen at jochen-kuepper.de (=?iso-8859-1?q?Jochen_K=FCpper?=)
Date: Fri Sep 26 14:24:03 2003
Subject: [Numpy-discussion] Cygwin compilation
In-Reply-To: <3F4525A1.3040701@colorado.edu> (Fernando Perez's message of
 "Thu, 21 Aug 2003 14:03:45 -0600")
References: <3F4525A1.3040701@colorado.edu>
Message-ID: <868yob9rfm.fsf@doze.rijnh.nl>

I just compiled/installed numarray-0.7 on (latest) Cygwin.

_ufunc.dll needs to be linked against /usr/lib/libmingwex.a. Doing
that manually seems to work fine. Adding it to the setup files is not
as simple, as libnumarray.dll does not link any more as some wrong
functions seem to be picked up from the MINGW library as well (and the
pow cannot be resolved. Explicitely adding -lm does not help).

How is this planned to work. Where needs the sequence
"-L/usr/lib/mingw -lmingwex" be put?

Is it save to use this mingw library in "normal" Cygwin programs?

Same happens with cvs HEAD.


Moreover I have the follwoing test error:
,----
| Failure in example: log10([1+1j])
| from line #2267 of numarray.numtest
| Expected: array([ 0.150515+0.34109409j])
| Got: array([ 0.150515+5.21807245j])
`----

Greetings,
Jochen
-- 
Einigkeit und Recht und Freiheit                http://www.Jochen-Kuepper.de
    Libert?, ?galit?, Fraternit?                GnuPG key: CC1B0B4D
        (Part 3 you find in my messages before fall 2003.)




From cjw at sympatico.ca  Fri Sep 26 19:01:02 2003
From: cjw at sympatico.ca (Colin J. Williams)
Date: Fri Sep 26 19:01:02 2003
Subject: [Numpy-discussion] numarray unfriendly to user defined types
In-Reply-To: <1064583557.13805.293.camel@halloween.stsci.edu>
References: <3F6A3D4C.3070703@ieee.org>		<1064266268.9234.351.camel@halloween.stsci.edu> <3F6F7744.80908@ieee.org>		<3F6F7B83.3040305@colorado.edu>  <3F6F8546.9020500@ieee.org>		<1064313211.3460.160.camel@localhost.localdomain>		<3F721A4C.7080300@ieee.org>	<1064444385.23878.14.camel@localhost.localdomain> 	<3F722F0B.7070609@ieee.org> <1064583557.13805.293.camel@halloween.stsci.edu>
Message-ID: <3F74780D.3070701@sympatico.ca>

Todd,

This scheme seems rather complex.  The purpose isn't clear to me.

What is the problem which this arrangement is intended to solve?

Could I suggest that some scheme along the lines of the Python PEP's be 
used to set out the Why's and Wherefore's?

It might also be useful to copy this discussion to comp.lang.python as 
that newsgroup has people experienced with classes.  The problem appears 
to be Python related rather than a problem of numerical analysis.

Colin W.

Todd Miller wrote:

>I tried out DeferredToByNumArray yesterday morning and was able to get
>it to work as planned with numarray and MA.  So yay!
>
>Yesterday afternoon,  I stopped by Perry's office for a final check, and
>not surprisingly,  there are some aspects of this solution that we've
>overlooked.  So not yay!
>
>Perry pointed out two ways that the DeferredToByNumArray scheme may fall
>apart as our inheritance hierarchy becomes richer:
>
>1)  Imagine not one, but two independent subclasses of NumArray, each of
>which want NumArray to defer to them.  Now imagine them trying to
>inter-operate with each other.  In this case, the meaning of the
>expression is determined by the order of the operands, so A+B and B+A
>will either return an A or a B depending on the expression order.
>
>2)  Imagine again two subclasses of numarray, but this time imagine B as
>a subclass of A.  Here again, both might want to defer to numarray, and
>again, A+B and B+A return different types driven by the type order of
>the expression.
>
>I don't have a solution yet, but am hopeful that more free candy will
>fall from the sky... or YAGNI.  Perry pointed out that similar problems
>exist for *any* Python class hierarchy, so we're not alone, and perhaps
>should forget about this until it matters.  If someone sees an easy
>fix,  now would be better than later.
>
>Todd
>On Wed, 2003-09-24 at 19:55, Tim Hochberg wrote:
>  
>
>>Todd Miller wrote:
>>
>>    
>>
>>>On Wed, 2003-09-24 at 18:27, Tim Hochberg wrote:
>>> 
>>>
>>>      
>>>
>>>>Hi Todd,
>>>>
>>>>There are three ways to spell "defer to me" on the table (the precise 
>>>>details of
>>>>each spelling are, of course, still open for debate):
>>>>
>>>>  1.  numarray.defer_to(my_class)
>>>>
>>>>  2. class ArrayLike(numarray.DeferTo):
>>>>        # ...
>>>>
>>>>  3. class ArrayLike:
>>>>        _numarray_defer_to = True
>>>>       # ...
>>>>
>>>>I'd prefer a non-registration solution since those are both 
>>>>aesthetically displeasing and leave you open to the situation where a 
>>>>class in module A gets registered by module B, but module C expects it 
>>>>not to be registered and everything breaks. Not all that likely, I 
>>>>admit, but let's avoid the registration version if we can.
>>>>   
>>>>
>>>>        
>>>>
>>>I was picturing this as module A registering it's own classes only. 
>>>Nevertheless, inverting the problem and distributing the registration as
>>>you suggested is better.
>>> 
>>>
>>>      
>>>
>>The case you describe probably will describe the majority of actual use 
>>cases, and in fact describes mine. I'm trying to think ahead a bit to 
>>cases may encounter as start using NumArray more extensively. Let's hope 
>>this solution still looks good in six months!
>>
>>    
>>
>>>>The other two solutions are almost equivalent. The one case where 3 has 
>>>>an edge over 2 is if I have an object (not a class), I could potentially 
>>>>set a _numarray_defer_to on the object before passing it to numarray 
>>>>without having to mess with the class
>>>>of the object at all. YAGNI, though.
>>>>   
>>>>
>>>>        
>>>>
>>>I was more concerned about the potential impact of lots of multiple
>>>inheritance, but that's probably just my own personal blend of FUD.
>>>
>>> 
>>>
>>>      
>>>
>>>>The advantage of 2 in my view is that it *does* force you to subclass. 
>>>>With 3, there will be the temptation to poke into some other  module and 
>>>>set _numarray_defer_to on some poor unsuspecting class. This has the 
>>>>same disadvantage as 1, that it could confuse some other poor 
>>>>unsuspecting module. The correct way to do get a deferred class from a 
>>>>third party module is to import and subclass. This works with either 2 
>>>>or 3::
>>>>
>>>> import A
>>>>
>>>> class Klass2(a.Klass, numarray.DeferTo):   #2
>>>>    #...
>>>>
>>>> class Klass3(a.Klass): #3 the good way
>>>>     _numarray_defer_to = True
>>>>    # ...
>>>>
>>>>  A.Klass._numarray_defer_to = True #3 the evil way.
>>>>
>>>>Version 2 is cleaner and encourages you to do the right thing, so I'd 
>>>>prefer that solution.
>>>>
>>>>   
>>>>
>>>>        
>>>>
>>>Good enough for me.  If no one else has any comments, then
>>>numarray.DeferTo is where I'll start implementing.  Tomorrow.
>>> 
>>>
>>>      
>>>
>>One more minor thing. I'm not sure tha DeferTo is ideal as the mix-in 
>>class name. It was perfect for the registration function name, but I'm 
>>not sure it's so clear whether the class or numarray is being deferred 
>>to when you say numarray.DeferTo. DeferToMe is more descriptive, but 
>>seems sort of slangy. DeferredTo is better than DeferTo, but still not 
>>as descriptive as DeferToMe. numarray.DefersTo reads perfect as long as 
>>numarray is included but is a disaster if you read it on your own.  
>>Below I've put down all the ideas I could come up with
>>
>>class CustomArray(numarray.DeferTo)
>>class CustomArray(numarray.DefersTo)
>>class CustomArray(numarray.DeferredTo)
>>class CustomArray(numarray.DeferToMe)
>>class CustomArray(numarray.DeferredToByNumarray)
>>class CustomArray(DeferTo)
>>class CustomArray(DefersTo)
>>class CustomArray(DeferredTo)
>>class CustomArray(DeferToMe)
>>class CustomArray(DeferredToByNumarray)
>>
>>For me it's a toss up between DefferedTo, DeferToMe and 
>>DeferredToByNumarray. The first is a little lacking in descriptive 
>>power, the second is slangy and the third is wordy.
>>
>>-tim
>>
>>[not that this matters much....]
>>
>>
>>    
>>
>>>>regards,
>>>>
>>>>-tim
>>>>   
>>>>
>>>>        
>>>>
>>>Thanks again,
>>>Todd
>>>
>>>
>>>
>>>
>>>
>>> 
>>>
>>>      
>>>
>>
>>
>>
>>-------------------------------------------------------
>>This sf.net email is sponsored by:ThinkGeek
>>Welcome to geek heaven.
>>http://thinkgeek.com/sf
>>_______________________________________________
>>Numpy-discussion mailing list
>>Numpy-discussion at lists.sourceforge.net
>>https://lists.sourceforge.net/lists/listinfo/numpy-discussion
>>    
>>





From cjw at sympatico.ca  Sat Sep 27 13:39:02 2003
From: cjw at sympatico.ca (Colin J. Williams)
Date: Sat Sep 27 13:39:02 2003
Subject: [Numpy-discussion] numarray unfriendly to user defined types
In-Reply-To: <1064600606.1021.17.camel@halloween.stsci.edu>
References: <3F6A3D4C.3070703@ieee.org>		<1064266268.9234.351.camel@halloween.stsci.edu>	 <3F6F7744.80908@ieee.org>		<3F6F7B83.3040305@colorado.edu> 	<3F6F8546.9020500@ieee.org>		<1064313211.3460.160.camel@localhost.localdomai	n>		<3F721A4C.7080300@ieee.org>	<1064444385.23878.14.camel@localhost.localdo	main> 	<3F722F0B.7070609@ieee.org>	<1064583557.13805.293.camel@halloween.stsci.edu> 	<3F74780D.3070701@sympatico.ca> <1064600606.1021.17.camel@halloween.stsci.edu>
Message-ID: <3F75EAD0.3030909@sympatico.ca>

Tim,

Todd refers to your original post.  The only thing I've found is a 
reference to a pre and post addition of zero, the latter was said not to 
work.  Both work for me, see below:

 >>> zero= 0
 >>> zero + _num.arange(5)
array([0, 1, 2, 3, 4])
 >>> _num.arange(5) + zero
array([0, 1, 2, 3, 4])

If I'm barking up the wrong tree, could you point me to your original 
posting in the archive please?

Colin W.
 >>>
Todd Miller wrote:

>On Fri, 2003-09-26 at 13:31, Colin J. Williams wrote:
>  
>
>>Todd,
>>
>>This scheme seems rather complex.  The purpose isn't clear to me.
>>
>>What is the problem which this arrangement is intended to solve?
>>    
>>
>
>Bottom line, currently:  
>	
>numarray + another_class --> exception
>
>It would be nice if:  
>
>numarray + another_class --> another_class.__radd__(numarray)
>
>Also, look backward at Tim's original post on this subject.
>
>  
>
>>Could I suggest that some scheme along the lines of the Python PEP's be 
>>used to set out the Why's and Wherefore's?
>>
>>    
>>
>Sounds good for next time.
>
>  
>
>>It might also be useful to copy this discussion to comp.lang.python as 
>>that newsgroup has people experienced with classes.  
>>    
>>
>
>OK,  I'll stop being a chicken.  Everyone, get ready for the sound of a
>pin dropping on c.l.py.
>
>  
>
>>The problem appears 
>>to be Python related rather than a problem of numerical analysis.
>>    
>>
>
>Yes indeed.
>
>  
>
>>Colin W.
>>
>>Todd Miller wrote:
>>
>>    
>>
>>>I tried out DeferredToByNumArray yesterday morning and was able to get
>>>it to work as planned with numarray and MA.  So yay!
>>>
>>>Yesterday afternoon,  I stopped by Perry's office for a final check, and
>>>not surprisingly,  there are some aspects of this solution that we've
>>>overlooked.  So not yay!
>>>
>>>Perry pointed out two ways that the DeferredToByNumArray scheme may fall
>>>apart as our inheritance hierarchy becomes richer:
>>>
>>>1)  Imagine not one, but two independent subclasses of NumArray, each of
>>>which want NumArray to defer to them.  Now imagine them trying to
>>>inter-operate with each other.  In this case, the meaning of the
>>>expression is determined by the order of the operands, so A+B and B+A
>>>will either return an A or a B depending on the expression order.
>>>
>>>2)  Imagine again two subclasses of numarray, but this time imagine B as
>>>a subclass of A.  Here again, both might want to defer to numarray, and
>>>again, A+B and B+A return different types driven by the type order of
>>>the expression.
>>>
>>>I don't have a solution yet, but am hopeful that more free candy will
>>>fall from the sky... or YAGNI.  Perry pointed out that similar problems
>>>exist for *any* Python class hierarchy, so we're not alone, and perhaps
>>>should forget about this until it matters.  If someone sees an easy
>>>fix,  now would be better than later.
>>>
>>>Todd
>>>On Wed, 2003-09-24 at 19:55, Tim Hochberg wrote:
>>> 
>>>
>>>      
>>>
>>>>Todd Miller wrote:
>>>>
>>>>   
>>>>
>>>>        
>>>>
>>>>>On Wed, 2003-09-24 at 18:27, Tim Hochberg wrote:
>>>>>
>>>>>
>>>>>     
>>>>>
>>>>>          
>>>>>
>>>>>>Hi Todd,
>>>>>>
>>>>>>There are three ways to spell "defer to me" on the table (the precise 
>>>>>>details of
>>>>>>each spelling are, of course, still open for debate):
>>>>>>
>>>>>> 1.  numarray.defer_to(my_class)
>>>>>>
>>>>>> 2. class ArrayLike(numarray.DeferTo):
>>>>>>       # ...
>>>>>>
>>>>>> 3. class ArrayLike:
>>>>>>       _numarray_defer_to = True
>>>>>>      # ...
>>>>>>
>>>>>>I'd prefer a non-registration solution since those are both 
>>>>>>aesthetically displeasing and leave you open to the situation where a 
>>>>>>class in module A gets registered by module B, but module C expects it 
>>>>>>not to be registered and everything breaks. Not all that likely, I 
>>>>>>admit, but let's avoid the registration version if we can.
>>>>>>  
>>>>>>
>>>>>>       
>>>>>>
>>>>>>            
>>>>>>
>>>>>I was picturing this as module A registering it's own classes only. 
>>>>>Nevertheless, inverting the problem and distributing the registration as
>>>>>you suggested is better.
>>>>>
>>>>>
>>>>>     
>>>>>
>>>>>          
>>>>>
>>>>The case you describe probably will describe the majority of actual use 
>>>>cases, and in fact describes mine. I'm trying to think ahead a bit to 
>>>>cases may encounter as start using NumArray more extensively. Let's hope 
>>>>this solution still looks good in six months!
>>>>
>>>>   
>>>>
>>>>        
>>>>
>>>>>>The other two solutions are almost equivalent. The one case where 3 has 
>>>>>>an edge over 2 is if I have an object (not a class), I could potentially 
>>>>>>set a _numarray_defer_to on the object before passing it to numarray 
>>>>>>without having to mess with the class
>>>>>>of the object at all. YAGNI, though.
>>>>>>  
>>>>>>
>>>>>>       
>>>>>>
>>>>>>            
>>>>>>
>>>>>I was more concerned about the potential impact of lots of multiple
>>>>>inheritance, but that's probably just my own personal blend of FUD.
>>>>>
>>>>>
>>>>>
>>>>>     
>>>>>
>>>>>          
>>>>>
>>>>>>The advantage of 2 in my view is that it *does* force you to subclass. 
>>>>>>With 3, there will be the temptation to poke into some other  module and 
>>>>>>set _numarray_defer_to on some poor unsuspecting class. This has the 
>>>>>>same disadvantage as 1, that it could confuse some other poor 
>>>>>>unsuspecting module. The correct way to do get a deferred class from a 
>>>>>>third party module is to import and subclass. This works with either 2 
>>>>>>or 3::
>>>>>>
>>>>>>import A
>>>>>>
>>>>>>class Klass2(a.Klass, numarray.DeferTo):   #2
>>>>>>   #...
>>>>>>
>>>>>>class Klass3(a.Klass): #3 the good way
>>>>>>    _numarray_defer_to = True
>>>>>>   # ...
>>>>>>
>>>>>> A.Klass._numarray_defer_to = True #3 the evil way.
>>>>>>
>>>>>>Version 2 is cleaner and encourages you to do the right thing, so I'd 
>>>>>>prefer that solution.
>>>>>>
>>>>>>  
>>>>>>
>>>>>>       
>>>>>>
>>>>>>            
>>>>>>
>>>>>Good enough for me.  If no one else has any comments, then
>>>>>numarray.DeferTo is where I'll start implementing.  Tomorrow.
>>>>>
>>>>>
>>>>>     
>>>>>
>>>>>          
>>>>>
>>>>One more minor thing. I'm not sure tha DeferTo is ideal as the mix-in 
>>>>class name. It was perfect for the registration function name, but I'm 
>>>>not sure it's so clear whether the class or numarray is being deferred 
>>>>to when you say numarray.DeferTo. DeferToMe is more descriptive, but 
>>>>seems sort of slangy. DeferredTo is better than DeferTo, but still not 
>>>>as descriptive as DeferToMe. numarray.DefersTo reads perfect as long as 
>>>>numarray is included but is a disaster if you read it on your own.  
>>>>Below I've put down all the ideas I could come up with
>>>>
>>>>class CustomArray(numarray.DeferTo)
>>>>class CustomArray(numarray.DefersTo)
>>>>class CustomArray(numarray.DeferredTo)
>>>>class CustomArray(numarray.DeferToMe)
>>>>class CustomArray(numarray.DeferredToByNumarray)
>>>>class CustomArray(DeferTo)
>>>>class CustomArray(DefersTo)
>>>>class CustomArray(DeferredTo)
>>>>class CustomArray(DeferToMe)
>>>>class CustomArray(DeferredToByNumarray)
>>>>
>>>>For me it's a toss up between DefferedTo, DeferToMe and 
>>>>DeferredToByNumarray. The first is a little lacking in descriptive 
>>>>power, the second is slangy and the third is wordy.
>>>>
>>>>-tim
>>>>
>>>>[not that this matters much....]
>>>>
>>>>
>>>>   
>>>>
>>>>        
>>>>
>>>>>>regards,
>>>>>>
>>>>>>-tim
>>>>>>  
>>>>>>
>>>>>>       
>>>>>>
>>>>>>            
>>>>>>
>>>>>Thanks again,
>>>>>Todd
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>     
>>>>>
>>>>>          
>>>>>
>>>>
>>>>-------------------------------------------------------
>>>>This sf.net email is sponsored by:ThinkGeek
>>>>Welcome to geek heaven.
>>>>http://thinkgeek.com/sf
>>>>_______________________________________________
>>>>Numpy-discussion mailing list
>>>>Numpy-discussion at lists.sourceforge.net
>>>>https://lists.sourceforge.net/lists/listinfo/numpy-discussion
>>>>   
>>>>
>>>>        
>>>>
>>    
>>





From cjw at sympatico.ca  Sun Sep 28 11:32:04 2003
From: cjw at sympatico.ca (Colin J. Williams)
Date: Sun Sep 28 11:32:04 2003
Subject: [Numpy-discussion] numarray unfriendly to user defined types
In-Reply-To: <1064600606.1021.17.camel@halloween.stsci.edu>
References: <3F6A3D4C.3070703@ieee.org>		<1064266268.9234.351.camel@halloween.stsci.edu>	 <3F6F7744.80908@ieee.org>		<3F6F7B83.3040305@colorado.edu> 	<3F6F8546.9020500@ieee.org>		<1064313211.3460.160.camel@localhost.localdomai	n>		<3F721A4C.7080300@ieee.org>	<1064444385.23878.14.camel@localhost.localdo	main> 	<3F722F0B.7070609@ieee.org>	<1064583557.13805.293.camel@halloween.stsci.edu> 	<3F74780D.3070701@sympatico.ca> <1064600606.1021.17.camel@halloween.stsci.edu>
Message-ID: <3F75EAD0.3030909@sympatico.ca>

Tim,

Todd refers to your original post.  The only thing I've found is a 
reference to a pre and post addition of zero, the latter was said not to 
work.  Both work for me, see below:

 >>> zero= 0
 >>> zero + _num.arange(5)
array([0, 1, 2, 3, 4])
 >>> _num.arange(5) + zero
array([0, 1, 2, 3, 4])

If I'm barking up the wrong tree, could you point me to your original 
posting in the archive please?

Colin W.
 >>>
Todd Miller wrote:

>On Fri, 2003-09-26 at 13:31, Colin J. Williams wrote:
>  
>
>>Todd,
>>
>>This scheme seems rather complex.  The purpose isn't clear to me.
>>
>>What is the problem which this arrangement is intended to solve?
>>    
>>
>
>Bottom line, currently:  
>	
>numarray + another_class --> exception
>
>It would be nice if:  
>
>numarray + another_class --> another_class.__radd__(numarray)
>
>Also, look backward at Tim's original post on this subject.
>
>  
>
>>Could I suggest that some scheme along the lines of the Python PEP's be 
>>used to set out the Why's and Wherefore's?
>>
>>    
>>
>Sounds good for next time.
>
>  
>
>>It might also be useful to copy this discussion to comp.lang.python as 
>>that newsgroup has people experienced with classes.  
>>    
>>
>
>OK,  I'll stop being a chicken.  Everyone, get ready for the sound of a
>pin dropping on c.l.py.
>
>  
>
>>The problem appears 
>>to be Python related rather than a problem of numerical analysis.
>>    
>>
>
>Yes indeed.
>
>  
>
>>Colin W.
>>
>>Todd Miller wrote:
>>
>>    
>>
>>>I tried out DeferredToByNumArray yesterday morning and was able to get
>>>it to work as planned with numarray and MA.  So yay!
>>>
>>>Yesterday afternoon,  I stopped by Perry's office for a final check, and
>>>not surprisingly,  there are some aspects of this solution that we've
>>>overlooked.  So not yay!
>>>
>>>Perry pointed out two ways that the DeferredToByNumArray scheme may fall
>>>apart as our inheritance hierarchy becomes richer:
>>>
>>>1)  Imagine not one, but two independent subclasses of NumArray, each of
>>>which want NumArray to defer to them.  Now imagine them trying to
>>>inter-operate with each other.  In this case, the meaning of the
>>>expression is determined by the order of the operands, so A+B and B+A
>>>will either return an A or a B depending on the expression order.
>>>
>>>2)  Imagine again two subclasses of numarray, but this time imagine B as
>>>a subclass of A.  Here again, both might want to defer to numarray, and
>>>again, A+B and B+A return different types driven by the type order of
>>>the expression.
>>>
>>>I don't have a solution yet, but am hopeful that more free candy will
>>>fall from the sky... or YAGNI.  Perry pointed out that similar problems
>>>exist for *any* Python class hierarchy, so we're not alone, and perhaps
>>>should forget about this until it matters.  If someone sees an easy
>>>fix,  now would be better than later.
>>>
>>>Todd
>>>On Wed, 2003-09-24 at 19:55, Tim Hochberg wrote:
>>> 
>>>
>>>      
>>>
>>>>Todd Miller wrote:
>>>>
>>>>   
>>>>
>>>>        
>>>>
>>>>>On Wed, 2003-09-24 at 18:27, Tim Hochberg wrote:
>>>>>
>>>>>
>>>>>     
>>>>>
>>>>>          
>>>>>
>>>>>>Hi Todd,
>>>>>>
>>>>>>There are three ways to spell "defer to me" on the table (the precise 
>>>>>>details of
>>>>>>each spelling are, of course, still open for debate):
>>>>>>
>>>>>> 1.  numarray.defer_to(my_class)
>>>>>>
>>>>>> 2. class ArrayLike(numarray.DeferTo):
>>>>>>       # ...
>>>>>>
>>>>>> 3. class ArrayLike:
>>>>>>       _numarray_defer_to = True
>>>>>>      # ...
>>>>>>
>>>>>>I'd prefer a non-registration solution since those are both 
>>>>>>aesthetically displeasing and leave you open to the situation where a 
>>>>>>class in module A gets registered by module B, but module C expects it 
>>>>>>not to be registered and everything breaks. Not all that likely, I 
>>>>>>admit, but let's avoid the registration version if we can.
>>>>>>  
>>>>>>
>>>>>>       
>>>>>>
>>>>>>            
>>>>>>
>>>>>I was picturing this as module A registering it's own classes only. 
>>>>>Nevertheless, inverting the problem and distributing the registration as
>>>>>you suggested is better.
>>>>>
>>>>>
>>>>>     
>>>>>
>>>>>          
>>>>>
>>>>The case you describe probably will describe the majority of actual use 
>>>>cases, and in fact describes mine. I'm trying to think ahead a bit to 
>>>>cases may encounter as start using NumArray more extensively. Let's hope 
>>>>this solution still looks good in six months!
>>>>
>>>>   
>>>>
>>>>        
>>>>
>>>>>>The other two solutions are almost equivalent. The one case where 3 has 
>>>>>>an edge over 2 is if I have an object (not a class), I could potentially 
>>>>>>set a _numarray_defer_to on the object before passing it to numarray 
>>>>>>without having to mess with the class
>>>>>>of the object at all. YAGNI, though.
>>>>>>  
>>>>>>
>>>>>>       
>>>>>>
>>>>>>            
>>>>>>
>>>>>I was more concerned about the potential impact of lots of multiple
>>>>>inheritance, but that's probably just my own personal blend of FUD.
>>>>>
>>>>>
>>>>>
>>>>>     
>>>>>
>>>>>          
>>>>>
>>>>>>The advantage of 2 in my view is that it *does* force you to subclass. 
>>>>>>With 3, there will be the temptation to poke into some other  module and 
>>>>>>set _numarray_defer_to on some poor unsuspecting class. This has the 
>>>>>>same disadvantage as 1, that it could confuse some other poor 
>>>>>>unsuspecting module. The correct way to do get a deferred class from a 
>>>>>>third party module is to import and subclass. This works with either 2 
>>>>>>or 3::
>>>>>>
>>>>>>import A
>>>>>>
>>>>>>class Klass2(a.Klass, numarray.DeferTo):   #2
>>>>>>   #...
>>>>>>
>>>>>>class Klass3(a.Klass): #3 the good way
>>>>>>    _numarray_defer_to = True
>>>>>>   # ...
>>>>>>
>>>>>> A.Klass._numarray_defer_to = True #3 the evil way.
>>>>>>
>>>>>>Version 2 is cleaner and encourages you to do the right thing, so I'd 
>>>>>>prefer that solution.
>>>>>>
>>>>>>  
>>>>>>
>>>>>>       
>>>>>>
>>>>>>            
>>>>>>
>>>>>Good enough for me.  If no one else has any comments, then
>>>>>numarray.DeferTo is where I'll start implementing.  Tomorrow.
>>>>>
>>>>>
>>>>>     
>>>>>
>>>>>          
>>>>>
>>>>One more minor thing. I'm not sure tha DeferTo is ideal as the mix-in 
>>>>class name. It was perfect for the registration function name, but I'm 
>>>>not sure it's so clear whether the class or numarray is being deferred 
>>>>to when you say numarray.DeferTo. DeferToMe is more descriptive, but 
>>>>seems sort of slangy. DeferredTo is better than DeferTo, but still not 
>>>>as descriptive as DeferToMe. numarray.DefersTo reads perfect as long as 
>>>>numarray is included but is a disaster if you read it on your own.  
>>>>Below I've put down all the ideas I could come up with
>>>>
>>>>class CustomArray(numarray.DeferTo)
>>>>class CustomArray(numarray.DefersTo)
>>>>class CustomArray(numarray.DeferredTo)
>>>>class CustomArray(numarray.DeferToMe)
>>>>class CustomArray(numarray.DeferredToByNumarray)
>>>>class CustomArray(DeferTo)
>>>>class CustomArray(DefersTo)
>>>>class CustomArray(DeferredTo)
>>>>class CustomArray(DeferToMe)
>>>>class CustomArray(DeferredToByNumarray)
>>>>
>>>>For me it's a toss up between DefferedTo, DeferToMe and 
>>>>DeferredToByNumarray. The first is a little lacking in descriptive 
>>>>power, the second is slangy and the third is wordy.
>>>>
>>>>-tim
>>>>
>>>>[not that this matters much....]
>>>>
>>>>
>>>>   
>>>>
>>>>        
>>>>
>>>>>>regards,
>>>>>>
>>>>>>-tim
>>>>>>  
>>>>>>
>>>>>>       
>>>>>>
>>>>>>            
>>>>>>
>>>>>Thanks again,
>>>>>Todd
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>     
>>>>>
>>>>>          
>>>>>
>>>>
>>>>-------------------------------------------------------
>>>>This sf.net email is sponsored by:ThinkGeek
>>>>Welcome to geek heaven.
>>>>http://thinkgeek.com/sf
>>>>_______________________________________________
>>>>Numpy-discussion mailing list
>>>>Numpy-discussion at lists.sourceforge.net
>>>>https://lists.sourceforge.net/lists/listinfo/numpy-discussion
>>>>   
>>>>
>>>>        
>>>>
>>    
>>





From jmiller at stsci.edu  Mon Sep 29 06:09:02 2003
From: jmiller at stsci.edu (Todd Miller)
Date: Mon Sep 29 06:09:02 2003
Subject: [Numpy-discussion] numarray unfriendly to user defined types
In-Reply-To: <3F75EAD0.3030909@sympatico.ca>
References: 	<3F6A3D4C.3070703@ieee.org>		<1064266268.9234.351.camel@halloween.stsci.edu>
		 <3F6F7744.80908@ieee.org>		<3F6F7B83.3040305@colorado.edu>
		<3F6F8546.9020500@ieee.org>		<1064313211.3460.160.camel@localhost.localdoma
	i	n>		<3F721A4C.7080300@ieee.org>	<1064444385.23878.14.camel@localhost.local
	do	main>
		<3F722F0B.7070609@ieee.org>	<1064583557.13805.293.camel@halloween.stsci.edu
	> 	<3F74780D.3070701@sympatico.ca>
	<1064600606.1021.17.camel@halloween.stsci.edu> 
	<3F75EAD0.3030909@sympatico.ca>
Message-ID: <1064840600.2297.66.camel@halloween.stsci.edu>

On Sat, 2003-09-27 at 15:53, Colin J. Williams wrote:
> Tim,
> 
> Todd refers to your original post.  The only thing I've found is a 
> reference to a pre and post addition of zero, the latter was said not to 
> work.  Both work for me, see below:
> 
>  >>> zero= 0
>  >>> zero + _num.arange(5)
> array([0, 1, 2, 3, 4])
>  >>> _num.arange(5) + zero
> array([0, 1, 2, 3, 4])
> 
> If I'm barking up the wrong tree, could you point me to your original 
> posting in the archive please?
> 

It's not the wrong tree,  but there's more than one case:  objects which
look like numeric sequence work but always return numarrays,  objects
which do not look like a numeric sequence raise an exception.

Here's a link to the numpy-discussion-list archives on Source Forge:

http://sourceforge.net/mailarchive/forum.php?thread_id=3154046&forum_id=4890


> Colin W.
>  >>>
> Todd Miller wrote:
> 
> >On Fri, 2003-09-26 at 13:31, Colin J. Williams wrote:
> >  
> >
> >>Todd,
> >>
> >>This scheme seems rather complex.  The purpose isn't clear to me.
> >>
> >>What is the problem which this arrangement is intended to solve?
> >>    
> >>
> >
> >Bottom line, currently:  
> >	
> >numarray + another_class --> exception
> >
> >It would be nice if:  
> >
> >numarray + another_class --> another_class.__radd__(numarray)
> >
> >Also, look backward at Tim's original post on this subject.
> >
> >  
> >
> >>Could I suggest that some scheme along the lines of the Python PEP's be 
> >>used to set out the Why's and Wherefore's?
> >>
> >>    
> >>
> >Sounds good for next time.
> >
> >  
> >
> >>It might also be useful to copy this discussion to comp.lang.python as 
> >>that newsgroup has people experienced with classes.  
> >>    
> >>
> >
> >OK,  I'll stop being a chicken.  Everyone, get ready for the sound of a
> >pin dropping on c.l.py.
> >
> >  
> >
> >>The problem appears 
> >>to be Python related rather than a problem of numerical analysis.
> >>    
> >>
> >
> >Yes indeed.
> >
> >  
> >
> >>Colin W.
> >>
> >>Todd Miller wrote:
> >>
> >>    
> >>
> >>>I tried out DeferredToByNumArray yesterday morning and was able to get
> >>>it to work as planned with numarray and MA.  So yay!
> >>>
> >>>Yesterday afternoon,  I stopped by Perry's office for a final check, and
> >>>not surprisingly,  there are some aspects of this solution that we've
> >>>overlooked.  So not yay!
> >>>
> >>>Perry pointed out two ways that the DeferredToByNumArray scheme may fall
> >>>apart as our inheritance hierarchy becomes richer:
> >>>
> >>>1)  Imagine not one, but two independent subclasses of NumArray, each of
> >>>which want NumArray to defer to them.  Now imagine them trying to
> >>>inter-operate with each other.  In this case, the meaning of the
> >>>expression is determined by the order of the operands, so A+B and B+A
> >>>will either return an A or a B depending on the expression order.
> >>>
> >>>2)  Imagine again two subclasses of numarray, but this time imagine B as
> >>>a subclass of A.  Here again, both might want to defer to numarray, and
> >>>again, A+B and B+A return different types driven by the type order of
> >>>the expression.
> >>>
> >>>I don't have a solution yet, but am hopeful that more free candy will
> >>>fall from the sky... or YAGNI.  Perry pointed out that similar problems
> >>>exist for *any* Python class hierarchy, so we're not alone, and perhaps
> >>>should forget about this until it matters.  If someone sees an easy
> >>>fix,  now would be better than later.
> >>>
> >>>Todd
> >>>On Wed, 2003-09-24 at 19:55, Tim Hochberg wrote:
> >>> 
> >>>
> >>>      
> >>>
> >>>>Todd Miller wrote:
> >>>>
> >>>>   
> >>>>
> >>>>        
> >>>>
> >>>>>On Wed, 2003-09-24 at 18:27, Tim Hochberg wrote:
> >>>>>
> >>>>>
> >>>>>     
> >>>>>
> >>>>>          
> >>>>>
> >>>>>>Hi Todd,
> >>>>>>
> >>>>>>There are three ways to spell "defer to me" on the table (the precise 
> >>>>>>details of
> >>>>>>each spelling are, of course, still open for debate):
> >>>>>>
> >>>>>> 1.  numarray.defer_to(my_class)
> >>>>>>
> >>>>>> 2. class ArrayLike(numarray.DeferTo):
> >>>>>>       # ...
> >>>>>>
> >>>>>> 3. class ArrayLike:
> >>>>>>       _numarray_defer_to = True
> >>>>>>      # ...
> >>>>>>
> >>>>>>I'd prefer a non-registration solution since those are both 
> >>>>>>aesthetically displeasing and leave you open to the situation where a 
> >>>>>>class in module A gets registered by module B, but module C expects it 
> >>>>>>not to be registered and everything breaks. Not all that likely, I 
> >>>>>>admit, but let's avoid the registration version if we can.
> >>>>>>  
> >>>>>>
> >>>>>>       
> >>>>>>
> >>>>>>            
> >>>>>>
> >>>>>I was picturing this as module A registering it's own classes only. 
> >>>>>Nevertheless, inverting the problem and distributing the registration as
> >>>>>you suggested is better.
> >>>>>
> >>>>>
> >>>>>     
> >>>>>
> >>>>>          
> >>>>>
> >>>>The case you describe probably will describe the majority of actual use 
> >>>>cases, and in fact describes mine. I'm trying to think ahead a bit to 
> >>>>cases may encounter as start using NumArray more extensively. Let's hope 
> >>>>this solution still looks good in six months!
> >>>>
> >>>>   
> >>>>
> >>>>        
> >>>>
> >>>>>>The other two solutions are almost equivalent. The one case where 3 has 
> >>>>>>an edge over 2 is if I have an object (not a class), I could potentially 
> >>>>>>set a _numarray_defer_to on the object before passing it to numarray 
> >>>>>>without having to mess with the class
> >>>>>>of the object at all. YAGNI, though.
> >>>>>>  
> >>>>>>
> >>>>>>       
> >>>>>>
> >>>>>>            
> >>>>>>
> >>>>>I was more concerned about the potential impact of lots of multiple
> >>>>>inheritance, but that's probably just my own personal blend of FUD.
> >>>>>
> >>>>>
> >>>>>
> >>>>>     
> >>>>>
> >>>>>          
> >>>>>
> >>>>>>The advantage of 2 in my view is that it *does* force you to subclass. 
> >>>>>>With 3, there will be the temptation to poke into some other  module and 
> >>>>>>set _numarray_defer_to on some poor unsuspecting class. This has the 
> >>>>>>same disadvantage as 1, that it could confuse some other poor 
> >>>>>>unsuspecting module. The correct way to do get a deferred class from a 
> >>>>>>third party module is to import and subclass. This works with either 2 
> >>>>>>or 3::
> >>>>>>
> >>>>>>import A
> >>>>>>
> >>>>>>class Klass2(a.Klass, numarray.DeferTo):   #2
> >>>>>>   #...
> >>>>>>
> >>>>>>class Klass3(a.Klass): #3 the good way
> >>>>>>    _numarray_defer_to = True
> >>>>>>   # ...
> >>>>>>
> >>>>>> A.Klass._numarray_defer_to = True #3 the evil way.
> >>>>>>
> >>>>>>Version 2 is cleaner and encourages you to do the right thing, so I'd 
> >>>>>>prefer that solution.
> >>>>>>
> >>>>>>  
> >>>>>>
> >>>>>>       
> >>>>>>
> >>>>>>            
> >>>>>>
> >>>>>Good enough for me.  If no one else has any comments, then
> >>>>>numarray.DeferTo is where I'll start implementing.  Tomorrow.
> >>>>>
> >>>>>
> >>>>>     
> >>>>>
> >>>>>          
> >>>>>
> >>>>One more minor thing. I'm not sure tha DeferTo is ideal as the mix-in 
> >>>>class name. It was perfect for the registration function name, but I'm 
> >>>>not sure it's so clear whether the class or numarray is being deferred 
> >>>>to when you say numarray.DeferTo. DeferToMe is more descriptive, but 
> >>>>seems sort of slangy. DeferredTo is better than DeferTo, but still not 
> >>>>as descriptive as DeferToMe. numarray.DefersTo reads perfect as long as 
> >>>>numarray is included but is a disaster if you read it on your own.  
> >>>>Below I've put down all the ideas I could come up with
> >>>>
> >>>>class CustomArray(numarray.DeferTo)
> >>>>class CustomArray(numarray.DefersTo)
> >>>>class CustomArray(numarray.DeferredTo)
> >>>>class CustomArray(numarray.DeferToMe)
> >>>>class CustomArray(numarray.DeferredToByNumarray)
> >>>>class CustomArray(DeferTo)
> >>>>class CustomArray(DefersTo)
> >>>>class CustomArray(DeferredTo)
> >>>>class CustomArray(DeferToMe)
> >>>>class CustomArray(DeferredToByNumarray)
> >>>>
> >>>>For me it's a toss up between DefferedTo, DeferToMe and 
> >>>>DeferredToByNumarray. The first is a little lacking in descriptive 
> >>>>power, the second is slangy and the third is wordy.
> >>>>
> >>>>-tim
> >>>>
> >>>>[not that this matters much....]
> >>>>
> >>>>
> >>>>   
> >>>>
> >>>>        
> >>>>
> >>>>>>regards,
> >>>>>>
> >>>>>>-tim
> >>>>>>  
> >>>>>>
> >>>>>>       
> >>>>>>
> >>>>>>            
> >>>>>>
> >>>>>Thanks again,
> >>>>>Todd
> >>>>>
> >>>>>
> >>>>>
> >>>>>
> >>>>>
> >>>>>
> >>>>>
> >>>>>     
> >>>>>
> >>>>>          
> >>>>>
> >>>>
> >>>>-------------------------------------------------------
> >>>>This sf.net email is sponsored by:ThinkGeek
> >>>>Welcome to geek heaven.
> >>>>http://thinkgeek.com/sf
> >>>>_______________________________________________
> >>>>Numpy-discussion mailing list
> >>>>Numpy-discussion at lists.sourceforge.net
> >>>>https://lists.sourceforge.net/lists/listinfo/numpy-discussion
> >>>>   
> >>>>
> >>>>        
> >>>>
> >>    
> >>
> 
> 
> 
> 
> -------------------------------------------------------
> This sf.net email is sponsored by:ThinkGeek
> Welcome to geek heaven.
> http://thinkgeek.com/sf
> _______________________________________________
> Numpy-discussion mailing list
> Numpy-discussion at lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/numpy-discussion
-- 
Todd Miller 			jmiller at stsci.edu
STSCI / ESS / SSB




From jmiller at stsci.edu  Mon Sep 29 07:02:47 2003
From: jmiller at stsci.edu (Todd Miller)
Date: Mon Sep 29 07:02:47 2003
Subject: [Numpy-discussion] Crash with numarray testall.py
In-Reply-To: <3F733D14.5060200@sympatico.ca>
References: <3F733D14.5060200@sympatico.ca>
Message-ID: <1064844032.2296.262.camel@halloween.stsci.edu>

I confirmed this as a problem under Windows and Linux, and also for 
Python-2.2.3 as well as Python-2.3.1.  

It should be pointed out that this is not a problem related to
Python-2.3.1, nor does the problem occur unless you try to use numarray
from it's own directory. 

For now, self-test/use numarray like this: 

> cd C:\somewhere_else 
> python 
>>> import numarray.testall as testall 
>>> testall.test() 
.. 
>>> import numarray 
.. 

I filed it as a bug on Source Forge. 

Todd 

On Thu, 2003-09-25 at 15:08, Colin J. Williams wrote: 
> 
> C:\Program Files\Python23\Lib\site-packages\numarray>python
> Python 2.3.1 (#47, Sep 23 2003, 23:47:32) [MSC v.1200 32 bit (Intel)] on 
> win32
> Type "help", "copyright", "credits" or "license" for more information.
>  >>> import testall
> 
> The 21K file created by Windows to contain information about the crash 
> is not readable.
> 
> Windows XP and numarray 0.7 used.
> 
> Colin W.
> 
> 
> 
> -------------------------------------------------------
> This sf.net email is sponsored by:ThinkGeek
> Welcome to geek heaven.
> http://thinkgeek.com/sf
> _______________________________________________
> Numpy-discussion mailing list
> Numpy-discussion at lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/numpy-discussion
-- 
Todd Miller 			jmiller at stsci.edu
STSCI / ESS / SSB




From jochen at jochen-kuepper.de  Tue Sep 30 01:46:03 2003
From: jochen at jochen-kuepper.de (=?iso-8859-1?q?Jochen_K=FCpper?=)
Date: Tue Sep 30 01:46:03 2003
Subject: [Numpy-discussion] Re: Cygwin compilation
In-Reply-To: <1064869509.2297.320.camel@halloween.stsci.edu> (Todd Miller's
 message of "29 Sep 2003 17:05:09 -0400")
References: <3F4525A1.3040701@colorado.edu> <868yob9rfm.fsf@doze.rijnh.nl>
	<1064869509.2297.320.camel@halloween.stsci.edu>
Message-ID: <86ad8mwtqk.fsf_-_@doze.rijnh.nl>

On 29 Sep 2003 17:05:09 -0400 Todd Miller wrote:

>> _ufunc.dll needs to be linked against /usr/lib/libmingwex.a. Doing
>> that manually seems to work fine.

Todd> In the file generate.py, a clause for sys.platform == "cygwin?"
Todd> probably needs to be added. Then, define EXTRA_LINK_ARGS for
Todd> cygwin.

Ok, did that.
,----[cvs diff -u generate.py]
| Warning: No xauth data; using fake authentication data for X11 forwarding.
| Index: generate.py
| ===================================================================
| RCS file: /cvsroot/numpy/numarray/generate.py,v
| retrieving revision 1.14
| diff -u -u -r1.14 generate.py
| --- generate.py 16 Sep 2003 20:02:41 -0000      1.14
| +++ generate.py 30 Sep 2003 08:33:38 -0000
| @@ -51,6 +51,9 @@
|      LP64, HAS_UINT64, HAS_FLOAT128 = 0, 1, 0
|  elif sys.platform == "win32":
|      LP64, HAS_UINT64, HAS_FLOAT128 = 0, 0, 0
| +elif sys.platform == "cygwin":
| +    LP64, HAS_UINT64, HAS_FLOAT128 = 0, 0, 0
| +    EXTRA_LINK_ARGS += ["-L/lib/mingw -lmingwex"]
|  elif sys.platform == "darwin":
|      LP64, HAS_UINT64, HAS_FLOAT128 = 0, 1, 0
|      EXTRA_COMPILE_ARGS.extend(["-Ddarwin"])
`----

Should work, but I get the following strange behavior:
,----
| > python setup.py build
| Using EXTRA_COMPILE_ARGS = []
| running build
| running build_py
| copying Lib/numinclude.py -> build/lib.cygwin-1.5.5-i686-2.3/numarray
| running build_ext
| building 'numarray._ufunc' extension
| gcc -shared -Wl,--enable-auto-image-base -march=pentium3 -mcpu=pentium4 -O2 -Wall -pipe build/temp.cygwin-1.5.5-i686-2.3/Src/_ufuncmodule.o -L/usr/lib/python2.3/config -lpython2.3 -o build/lib.cygwin-1.5.5-i686-2.3/numarray/_ufunc.dll -L/lib/mingw -lmingwex
| build/temp.cygwin-1.5.5-i686-2.3/Src/_ufuncmodule.o(.text+0x14f): In function `_CheckFPErrors':
| /home/software/programming/numeric/numarray/Src/_ufuncmodule.c:136: undefined reference to `_fetestexcept'
| build/temp.cygwin-1.5.5-i686-2.3/Src/_ufuncmodule.o(.text+0x17b):/home/software/programming/numeric/numarray/Src/_ufuncmodule.c:142: undefined reference to `_feclearexcept'
| collect2: ld returned 1 exit status
| error: command 'gcc' failed with exit status 1
| 
| > gcc -shared -Wl,--enable-auto-image-base -march=pentium3 -mcpu=pentium4 -O2 -Wall -pipe build/temp.cygwin-1.5.5-i686-2.3/Src/_ufuncmodule.o -L/usr/lib/python2.3/config -lpython2.3 -o build/lib.cygwin-1.5.5-i686-2.3/numarray/_ufunc.dll -L/lib/mingw -lmingwex
`----

That is, it does not work when called from setup.py. When I just copy
'n' paste the link statement to the command-line, it just works.(#$%)

Any ideas??


>> Is it save to use this mingw library in "normal" Cygwin programs?

Todd> I don't know. I didn't get any farther than this, perhaps not as
Todd> far. I got the impression that mingw was a minimal library
Todd> intended for porting gcc (but it has IEEE stuff), while the
Todd> cygwin libc is intended for real programs (but has no IEEE stuff
Todd> because it is a lightweight knock off of glibc used to bypass
Todd> the GPL).

Well, as I see it Cygwin is a Unix emulation layer for Windows,
whereas MINGW is a port of gcc and some other GNU tools to Windows
(not using Cygwin at all).

But looking around using it this way should be ok. Maybe someone who
knows cygwin better can speak up here? I would carbon-copy this email
to the cygwin list, but that is subscribers only nowadays:(



>> Moreover I have the follwoing test error:
>> ,----
>> | Failure in example: log10([1+1j])
>> | from line #2267 of numarray.numtest
>> | Expected: array([ 0.150515+0.34109409j])
>> | Got: array([ 0.150515+5.21807245j])
>> `----

Todd> One of our guys found a bug in the complex log10 function; both
Todd> the expected result and the implementation were changed.

Seems to be resolved.

Greetings,
Jochen
-- 
Einigkeit und Recht und Freiheit                http://www.Jochen-Kuepper.de
    Libert?, ?galit?, Fraternit?                GnuPG key: CC1B0B4D
        (Part 3 you find in my messages before fall 2003.)




From jmiller at stsci.edu  Tue Sep 30 04:31:03 2003
From: jmiller at stsci.edu (Todd Miller)
Date: Tue Sep 30 04:31:03 2003
Subject: [Numpy-discussion] Re: Cygwin compilation
In-Reply-To: <86ad8mwtqk.fsf_-_@doze.rijnh.nl>
References: <3F4525A1.3040701@colorado.edu> <868yob9rfm.fsf@doze.rijnh.nl>
	<1064869509.2297.320.camel@halloween.stsci.edu> 
	<86ad8mwtqk.fsf_-_@doze.rijnh.nl>
Message-ID: <1064921346.16063.55.camel@halloween.stsci.edu>

On Tue, 2003-09-30 at 04:44, Jochen K?pper wrote:

> Should work, but I get the following strange behavior:
> ,----
> | > python setup.py build
> | Using EXTRA_COMPILE_ARGS = []
> | running build
> | running build_py
> | copying Lib/numinclude.py -> build/lib.cygwin-1.5.5-i686-2.3/numarray
> | running build_ext
> | building 'numarray._ufunc' extension
> | gcc -shared -Wl,--enable-auto-image-base -march=pentium3 -mcpu=pentium4 -O2 -Wall -pipe build/temp.cygwin-1.5.5-i686-2.3/Src/_ufuncmodule.o -L/usr/lib/python2.3/config -lpython2.3 -o build/lib.cygwin-1.5.5-i686-2.3/numarray/_ufunc.dll -L/lib/mingw -lmingwex
> | build/temp.cygwin-1.5.5-i686-2.3/Src/_ufuncmodule.o(.text+0x14f): In function `_CheckFPErrors':
> | /home/software/programming/numeric/numarray/Src/_ufuncmodule.c:136: undefined reference to `_fetestexcept'
> | build/temp.cygwin-1.5.5-i686-2.3/Src/_ufuncmodule.o(.text+0x17b):/home/software/programming/numeric/numarray/Src/_ufuncmodule.c:142: undefined reference to `_feclearexcept'
> | collect2: ld returned 1 exit status
> | error: command 'gcc' failed with exit status 1
> | 
> | > gcc -shared -Wl,--enable-auto-image-base -march=pentium3 -mcpu=pentium4 -O2 -Wall -pipe build/temp.cygwin-1.5.5-i686-2.3/Src/_ufuncmodule.o -L/usr/lib/python2.3/config -lpython2.3 -o build/lib.cygwin-1.5.5-i686-2.3/numarray/_ufunc.dll -L/lib/mingw -lmingwex
> `----
> 
> That is, it does not work when called from setup.py. When I just copy
> 'n' paste the link statement to the command-line, it just works.(#$%)
> 
> Any ideas??

You could try replacing gcc in order to dump the environment or perhaps
add a switch to run gcc in a more verbose mode and see if any
differences pop up.  distutils-sig?

Have you checked the #ifdef clause in _CheckFPErrors in _ufuncmodule.c
for cygwin/mingw?  Apparently I added it, and I wasn't sure if it was
valid to compile for cygwin and link against mingw.

One other thing:  HAS_UINT64 should work.

Regards,
Todd
 
-- 
Todd Miller 			jmiller at stsci.edu
STSCI / ESS / SSB




From jochen at jochen-kuepper.de  Tue Sep 30 07:00:04 2003
From: jochen at jochen-kuepper.de (=?iso-8859-1?q?Jochen_K=FCpper?=)
Date: Tue Sep 30 07:00:04 2003
Subject: [Numpy-discussion] Re: Cygwin compilation
In-Reply-To: <1064921346.16063.55.camel@halloween.stsci.edu> (Todd Miller's
 message of "30 Sep 2003 07:29:06 -0400")
References: <3F4525A1.3040701@colorado.edu> <868yob9rfm.fsf@doze.rijnh.nl>
	<1064869509.2297.320.camel@halloween.stsci.edu>
	<86ad8mwtqk.fsf_-_@doze.rijnh.nl>
	<1064921346.16063.55.camel@halloween.stsci.edu>
Message-ID: <86smmetm2t.fsf@doze.rijnh.nl>

With the current patch agianst cvs HEAD numarray compiles and passes
all tests on current Cygwin:

cvs diff -u generate.py ChangeLog
Warning: No xauth data; using fake authentication data for X11 forwarding.
Index: generate.py
===================================================================
RCS file: /cvsroot/numpy/numarray/generate.py,v
retrieving revision 1.14
diff -u -u -r1.14 generate.py
--- generate.py 16 Sep 2003 20:02:41 -0000      1.14
+++ generate.py 30 Sep 2003 13:56:25 -0000
@@ -51,6 +51,9 @@
     LP64, HAS_UINT64, HAS_FLOAT128 = 0, 1, 0
 elif sys.platform == "win32":
     LP64, HAS_UINT64, HAS_FLOAT128 = 0, 0, 0
+elif sys.platform == "cygwin":
+    LP64, HAS_UINT64, HAS_FLOAT128 = 0, 1, 0
+    EXTRA_LINK_ARGS += ["-L/lib", "-lm", "-lc", "-lgcc", "-L/lib/mingw", "-lmingwex"]
 elif sys.platform == "darwin":
     LP64, HAS_UINT64, HAS_FLOAT128 = 0, 1, 0
     EXTRA_COMPILE_ARGS.extend(["-Ddarwin"])
Index: ChangeLog
===================================================================
RCS file: /cvsroot/numpy/numarray/ChangeLog,v
retrieving revision 1.1
diff -u -u -r1.1 ChangeLog
--- ChangeLog   20 Apr 2002 16:13:38 -0000      1.1
+++ ChangeLog   30 Sep 2003 13:56:25 -0000
@@ -1,3 +1,13 @@
+2003-09-30  Jochen K?pper  <jochen at jochen-kuepper.de>
+
+       * generate.py (codeargs): Add cygwin case to link against libmingwex.
+       This is a little tricky as _ufinc.dll needs to be linked against
+       libmingwex, but libnumarray.dll normally picks up wrong symbols from
+       this library. Therefore we explicitely add libm, libc, and libgcc before
+       the mingw libraries to avoid the latter and still have them available to
+       resolve the FE symbols. We also explicitely have to add -L/lib before
+       -L/lib/mingw to check for normal cygwin libraries first.
+ 
 2002-04-20  Jochen K?pper  <jochen at jochen-kuepper.de>
 
        * MANIFEST.in: Updated.  
@@ -9,5 +19,5 @@
 
        * Local Variables:
        * mode: auto-fill
-       * fill-column: 79
+       * fill-column: 80
        * End:

Greetings,
Jochen
-- 
Einigkeit und Recht und Freiheit                http://www.Jochen-Kuepper.de
    Libert?, ?galit?, Fraternit?                GnuPG key: CC1B0B4D
        (Part 3 you find in my messages before fall 2003.)
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 185 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20030930/52dccffd/attachment.sig>

From jmiller at stsci.edu  Tue Sep 30 07:13:04 2003
From: jmiller at stsci.edu (Todd Miller)
Date: Tue Sep 30 07:13:04 2003
Subject: [Numpy-discussion] Re: Cygwin compilation
In-Reply-To: <86smmetm2t.fsf@doze.rijnh.nl>
References: <3F4525A1.3040701@colorado.edu> <868yob9rfm.fsf@doze.rijnh.nl>
	 <1064869509.2297.320.camel@halloween.stsci.edu>
	 <86ad8mwtqk.fsf_-_@doze.rijnh.nl>
	 <1064921346.16063.55.camel@halloween.stsci.edu>
	 <86smmetm2t.fsf@doze.rijnh.nl>
Message-ID: <1064931107.14680.107.camel@localhost.localdomain>

Beautiful.  Thanks Jochen!


Todd

On Tue, 2003-09-30 at 09:58, Jochen K?pper wrote:
> With the current patch agianst cvs HEAD numarray compiles and passes
> all tests on current Cygwin:
> 
> cvs diff -u generate.py ChangeLog
> Warning: No xauth data; using fake authentication data for X11 forwarding.
> Index: generate.py
> ===================================================================
> RCS file: /cvsroot/numpy/numarray/generate.py,v
> retrieving revision 1.14
> diff -u -u -r1.14 generate.py
> --- generate.py 16 Sep 2003 20:02:41 -0000      1.14
> +++ generate.py 30 Sep 2003 13:56:25 -0000
> @@ -51,6 +51,9 @@
>      LP64, HAS_UINT64, HAS_FLOAT128 = 0, 1, 0
>  elif sys.platform == "win32":
>      LP64, HAS_UINT64, HAS_FLOAT128 = 0, 0, 0
> +elif sys.platform == "cygwin":
> +    LP64, HAS_UINT64, HAS_FLOAT128 = 0, 1, 0
> +    EXTRA_LINK_ARGS += ["-L/lib", "-lm", "-lc", "-lgcc", "-L/lib/mingw", "-lmingwex"]
>  elif sys.platform == "darwin":
>      LP64, HAS_UINT64, HAS_FLOAT128 = 0, 1, 0
>      EXTRA_COMPILE_ARGS.extend(["-Ddarwin"])
> Index: ChangeLog
> ===================================================================
> RCS file: /cvsroot/numpy/numarray/ChangeLog,v
> retrieving revision 1.1
> diff -u -u -r1.1 ChangeLog
> --- ChangeLog   20 Apr 2002 16:13:38 -0000      1.1
> +++ ChangeLog   30 Sep 2003 13:56:25 -0000
> @@ -1,3 +1,13 @@
> +2003-09-30  Jochen K?pper  <jochen at jochen-kuepper.de>
> +
> +       * generate.py (codeargs): Add cygwin case to link against libmingwex.
> +       This is a little tricky as _ufinc.dll needs to be linked against
> +       libmingwex, but libnumarray.dll normally picks up wrong symbols from
> +       this library. Therefore we explicitely add libm, libc, and libgcc before
> +       the mingw libraries to avoid the latter and still have them available to
> +       resolve the FE symbols. We also explicitely have to add -L/lib before
> +       -L/lib/mingw to check for normal cygwin libraries first.
> + 
>  2002-04-20  Jochen K?pper  <jochen at jochen-kuepper.de>
>  
>         * MANIFEST.in: Updated.  
> @@ -9,5 +19,5 @@
>  
>         * Local Variables:
>         * mode: auto-fill
> -       * fill-column: 79
> +       * fill-column: 80
>         * End:
> 
> Greetings,
> Jochen
-- 
Todd Miller <jmiller at stsci.edu>




From jmiller at stsci.edu  Tue Sep 30 07:55:02 2003
From: jmiller at stsci.edu (Todd Miller)
Date: Tue Sep 30 07:55:02 2003
Subject: [Numpy-discussion] numarray unfriendly to user defined types
In-Reply-To: <3F746764.2070403@ieee.org>
References: <3F6A3D4C.3070703@ieee.org>
	 <1064266268.9234.351.camel@halloween.stsci.edu> <3F6F7744.80908@ieee.org>
	 <3F6F7B83.3040305@colorado.edu>  <3F6F8546.9020500@ieee.org>
	 <1064313211.3460.160.camel@localhost.localdomain>
	 <3F721A4C.7080300@ieee.org>
	 <1064444385.23878.14.camel@localhost.localdomain>
	 <3F722F0B.7070609@ieee.org>
	 <1064583557.13805.293.camel@halloween.stsci.edu>
	 <3F746764.2070403@ieee.org>
Message-ID: <1064933606.14679.157.camel@localhost.localdomain>

On Fri, 2003-09-26 at 12:20, Tim Hochberg wrote:
> Todd Miller wrote:
> I'd make the same comment for this case as for the above case except 
> that all else being equal, subclasses should prevail over superclasses.
> 
> >I don't have a solution yet, but am hopeful that more free candy will
> >fall from the sky... or YAGNI.  Perry pointed out that similar problems
> >exist for *any* Python class hierarchy, so we're not alone, and perhaps
> >should forget about this until it matters.  If someone sees an easy
> >fix,  now would be better than later.
> >  
> >
> Here's some quick thoughts. In order for this to work out sensibly you 
> want the order between classes to be transistive and you probably also 
> always want to be able to insert a class into the order between any two 
> classes  This is equivalent to deciding the op priority based on a 
> comparison based on some real number associated with each class. This 
> leads to something like:
> 

Once again I think your ideas solve the problem, so thanks.  I have a
few things I wanted to bounce off you.

>    
> class NumArrayFamily:
>     op_priority = 0.0
>    
> # This makes it compatible with the current proposal , but isn't 
> strictly necessary.
> class DeferredToByNumarray(NumArrayFamily):
>     op_priority = 1000.0
>    
> class NumArrary(..., NumArrayFamily):
>     # ...
>     def __add__(self, object):
>         if isinstance(object, NumArrayFamily):
>             if ((object.op_priority > self.object.op_priority) or
>                 (object.op_priority == self.object.op_priority) and 
> issubclass(object, self.__class__)):
>                     return other.__radd__(self)
>         return ufunc.add(self, other)
> 
> 
> class MySubclassOfNumarray(NumArray):
>     op_priority = 10.0 # deffered to by NumArray, but defers to 
> DeferredToByNumarray classes.


As I understand it, 2.2 new style object subclass r-operators
automatically get priority (subclass.__radd__ is called in preference to
superclass.__add__).  I think this has two implications:

1) the ==/subclass term won't work, because __add__ will never get
called.

2) we have to redefine __radd__ as well to implement the priority
scheme.   I'm not sure the isinstance is necessary for __radd__,  but I
like the symmetry.

How does this look:

class DeferredToByNumArray: # name seems a little off now, but whatever
	pass

class NumArray(..., DeferredToByNumArray):
	op_priority = 0

	def __add__(self, object):
		if (isinstance(object, DeferredToByNumArray) and 
		    object.op_priority > self.op_priority):
			return object.__radd__(self)
		return ufunc.add(self, object)

	def __radd__(self, object):
		if (isinstance(object, DeferredToByNumArray) and
		    object.op_priority > self.op_priority):
			return object.__add__(self)
		return ufunc.add(object, self)

class MyForeignClass(DeferredToNumArray):
	op_priority = 1.0

class MySubclassOfNumArray(NumArray):
	op_priority = -1.0

> Of course the user still has to be careful to make sure the priority 
> order makes sense and you could have big problems when combining 
> packages with disprate ideas about priority levels, but I'm not sure how 
> you can get away from that.

As long as there's a scheme in place to resolve potential conflicts with
a little discussion,  I think your solution is already ahead of the
demand.

I told Colin last week I'd raise this issue on c.l.py.  I have decided
not to (although someone else could obviously do it if they are still
concerned about it),  because (a) the problem is solved,  (b) explaining
the problem well enough to get usable input is difficult and I've got
other stuff to do.

Thanks again for these ideas,

Todd

-- 
Todd Miller <jmiller at stsci.edu>




From Marc.Poinot at onera.fr  Tue Sep 30 09:22:01 2003
From: Marc.Poinot at onera.fr (Marc Poinot)
Date: Tue Sep 30 09:22:01 2003
Subject: [Numpy-discussion] Migrating Char arrays
Message-ID: <3F79AD6D.12732A71@onera.fr>

Hi all,

I'm migrating my pyCGNS package to numarray. I think that numarray is
better suited for the in-memory share of array I'd like to achieve.

Now, with Numeric, I use to create arrays with integers, floats, but 
also char type.

In CGNS (http://www.cgns.org) we have types I4, I8, R4 and R8, but
also C1 which is an array of chars. It looks like, in numarray, we should
consider chars as unsigned ints. This is not false from a system point
of view, but from an application point of view I have to make a difference
between *real* char arrays, and unsigned int arrays. In other words, when
I have a C1 array, I have to translate it as a String, when I have an I4
or even I8, I have to translate it as Integer array. Now I only have
integers arrays, and I have to find out a way to know that my array of int
is a string. The CGNS->numarray translation is OK, but now I'm loosing
information when I'm going back numarray->CGNS.

You can say: It's up to your application to understand that this list of ints
is a string.

I'll reply: So why don't you consider a list of bytes, it's up to the application
to know that you're 32 or 64 bits...

Is there something in numarray that could replace the Numeric interface
for this string type? Is it difficult to add a 'Character' type?
Is the (undocumented) CharArrayType the solution for this?

Marcvs [alias Not using unicode...]

-----------------------------------------------------------------------
 Marc POINOT             Alias: marcvs        Email: poinot at onera.fr
 ONERA -MFE/DSNA/ELSA    Tel: 01.46.73.42.84  Info: elsa-info at onera.fr
 29, Div. Leclerc        Fax: 01.46.73.41.66  Site:     
 92322 Chatillon FRANCE  Project: elsA        Web: http://www.onera.fr



From falted at openlc.org  Tue Sep 30 09:42:05 2003
From: falted at openlc.org (Francesc Alted)
Date: Tue Sep 30 09:42:05 2003
Subject: [Numpy-discussion] Migrating Char arrays
In-Reply-To: <3F79AD6D.12732A71@onera.fr>
References: <3F79AD6D.12732A71@onera.fr>
Message-ID: <200309301840.41624.falted@openlc.org>

Hi Marc,

A Dimarts 30 Setembre 2003 18:21, Marc Poinot va escriure:
>
> Is there something in numarray that could replace the Numeric interface
> for this string type? Is it difficult to add a 'Character' type?
> Is the (undocumented) CharArrayType the solution for this?

Yes, it is. See:

>>> from numarray import strings
>>> strings.array(["asd", "aa"])
CharArray(['asd', 'aa'])
>>> a=strings.array(["asd", "aa"])
>>> a[1]
'aa'
>>> a.raw()[1]   # If you want to get *all* spaces and nulls in strings
'aa '

This is undocumented in the manual, but it's quite well described in the doc
strings:

>>> help(strings.array)

Cheers,

-- 
Francesc Alted




From verveer at embl-heidelberg.de  Mon Sep  1 01:44:03 2003
From: verveer at embl-heidelberg.de (Peter Verveer)
Date: Mon Sep  1 01:44:03 2003
Subject: [Numpy-discussion] type info
Message-ID: <200309011043.30784.verveer@embl-heidelberg.de>

Hi all,

Is there some way in numarray to find out what the maximum and minimum values 
are that a type can hold? For instance, is there a convenient way to find out 
what the maximal possible value of the Float32 type is?

Cheers, Peter

-- 
Dr. Peter J. Verveer
Cell Biology and Cell Biophysics Programme
EMBL
Meyerhofstrasse 1
D-69117 Heidelberg
Germany
Tel. : +49 6221 387245
Fax  : +49 6221 387242
Email: Peter.Verveer at embl-heidelberg.de





From verveer at embl-heidelberg.de  Mon Sep  1 02:35:03 2003
From: verveer at embl-heidelberg.de (Peter Verveer)
Date: Mon Sep  1 02:35:03 2003
Subject: [Numpy-discussion] sum and mean methods behaviour
Message-ID: <200309011134.39281.verveer@embl-heidelberg.de>

Hi All,

I noticed that the sum() and mean() methods of numarrays use the precision of 
the given array in their calculations. That leads to resuls like this:

>>> array([255, 255], Int8).sum()
-2
>>> array([255, 255], Int8).mean()
-1.0

Would it not be better to use double precision internally and return the 
correct result?

Cheers, Peter

-- 
Dr. Peter J. Verveer
Cell Biology and Cell Biophysics Programme
EMBL
Meyerhofstrasse 1
D-69117 Heidelberg
Germany
Tel. : +49 6221 387245
Fax  : +49 6221 387242
Email: Peter.Verveer at embl-heidelberg.de





From jmiller at stsci.edu  Mon Sep  1 04:57:04 2003
From: jmiller at stsci.edu (Todd Miller)
Date: Mon Sep  1 04:57:04 2003
Subject: [Numpy-discussion] type info
In-Reply-To: <200309011043.30784.verveer@embl-heidelberg.de>
References: <200309011043.30784.verveer@embl-heidelberg.de>
Message-ID: <1062417380.3459.63.camel@localhost.localdomain>

On Mon, 2003-09-01 at 04:43, Peter Verveer wrote:
> Hi all,
> 
> Is there some way in numarray to find out what the maximum and minimum values 
> are that a type can hold? 

I think the answer is "no".  I added a feature request on Source Forge.

> For instance, is there a convenient way to find out 
> what the maximal possible value of the Float32 type is?

Nope.

> Cheers, Peter
-- 
Todd Miller <jmiller at stsci.edu>




From pearu at cens.ioc.ee  Mon Sep  1 05:57:03 2003
From: pearu at cens.ioc.ee (Pearu Peterson)
Date: Mon Sep  1 05:57:03 2003
Subject: [Numpy-discussion] type info
In-Reply-To: <200309011043.30784.verveer@embl-heidelberg.de>
Message-ID: <Pine.LNX.4.21.0309011551310.9326-100000@cens.kybi>

On Mon, 1 Sep 2003, Peter Verveer wrote:

> Is there some way in numarray to find out what the maximum and minimum values 
> are that a type can hold? For instance, is there a convenient way to find out 
> what the maximal possible value of the Float32 type is?

I am not completely sure what is behind of the following way but

>>> numarray.fromstring('\xff\xff\x7f\x7f','f')
array([  3.40282347e+38], type=Float32)

seems to give the maximum value for Float32.

Pearu





From verveer at embl-heidelberg.de  Mon Sep  1 06:11:02 2003
From: verveer at embl-heidelberg.de (Peter Verveer)
Date: Mon Sep  1 06:11:02 2003
Subject: [Numpy-discussion] type info
In-Reply-To: <Pine.LNX.4.21.0309011551310.9326-100000@cens.kybi>
References: <Pine.LNX.4.21.0309011551310.9326-100000@cens.kybi>
Message-ID: <200309011510.02692.verveer@embl-heidelberg.de>

On Monday 01 September 2003 14:55, Pearu Peterson wrote:
> On Mon, 1 Sep 2003, Peter Verveer wrote:
> > Is there some way in numarray to find out what the maximum and minimum
> > values are that a type can hold? For instance, is there a convenient way
> > to find out what the maximal possible value of the Float32 type is?
>
> I am not completely sure what is behind of the following way but
>
> >>> numarray.fromstring('\xff\xff\x7f\x7f','f')
>
> array([  3.40282347e+38], type=Float32)
>
> seems to give the maximum value for Float32.
>
> Pearu

Sure, it is not too difficult to find the minimum and maximum values of each 
type, given the bitsize (although there may be portability issues). But it 
would be convenient to have such properties directly accessible in numarray, 
I think.

Peter




From jmiller at stsci.edu  Tue Sep  2 11:34:04 2003
From: jmiller at stsci.edu (Todd Miller)
Date: Tue Sep  2 11:34:04 2003
Subject: [Numpy-discussion] sum and mean methods behaviour
In-Reply-To: <200309011134.39281.verveer@embl-heidelberg.de>
References: <200309011134.39281.verveer@embl-heidelberg.de>
Message-ID: <1062527563.16365.283.camel@halloween.stsci.edu>

On Mon, 2003-09-01 at 05:34, Peter Verveer wrote:
> Hi All,
> 
> I noticed that the sum() and mean() methods of numarrays use the precision of 
> the given array in their calculations. That leads to resuls like this:
> 
> >>> array([255, 255], Int8).sum()
> -2
> >>> array([255, 255], Int8).mean()
> -1.0
> 
> Would it not be better to use double precision internally and return the 
> correct result?
> 
> Cheers, Peter
> 

Hi Peter,

I thought about this a lot yesterday and today talked it over with
Perry.   There are several ways to fix the problem with mean() and
sum(), and I'm hoping that you and the rest of the community will help
sort them out.

(1) The first "solution" is to require users to do their own up-casting
prior to calling mean() or sum().  This gives the end user fine control
over storage cost but leaves the C-like pitfall/bug you discovered.   I
mention this because this is how the numarray/Numeric reductions are
designed.  Is there a reason why the numarray/Numeric reductions don't
implicitly up-cast? 

(2) The second way is what you proposed:  use double precision within
mean and sum.  This has great simplicity but gives no control over
storage usage, and as implemented, the storage would be much higher than
one might think, potentially 8x.

(3) Lastly, Perry suggested a more radical approach:  rather than
changing the mean and sum methods themselves,  we could alter the
universal function accumulate and reduce methods to implicitly use
additional precision.  Perry's idea was to make all accumulations and
reductions up-cast their results to the largest type of the current
family, either Bool, Int64, Float64, or Complex64.   By doing this, we
can improve the utility of the reductions and accumulations as well as
fixing the problem with sum and mean.

-- 
Todd Miller 			jmiller at stsci.edu
STSCI / ESS / SSB




From jmiller at stsci.edu  Tue Sep  2 12:06:12 2003
From: jmiller at stsci.edu (Todd Miller)
Date: Tue Sep  2 12:06:12 2003
Subject: [Numpy-discussion] Possible bug
In-Reply-To: <3F40D29E.2090308@sympatico.ca>
References: <200308181401.17137.verveer@embl-heidelberg.de> 
	<3F40D29E.2090308@sympatico.ca>
Message-ID: <1062529529.16822.318.camel@halloween.stsci.edu>

These problems are (partially) addressed in numarray-0.7 with the
addition of a check_overflow parameter to numarray.fromlist().  The
checked fromlist is now used in the implementation of ufuncs (e.g.
multiply), preventing the silent truncation of scalar values to the type
of the array.  

Problem c:  solved -- 1001 scalar was silently truncated to Int8 before
multiplication.

Problem e:  solved -- 3000 was silently truncated to Int8 during
fromlist.  Use fromlist() rather than array() and add
check_overflow=1 parameter.

Problem f: rejected  -- working as designed, won't change.  Floating
point numbers passed into array() are silently truncated if the type is
not a floating point type.  Operating on an integer array and a floating
point scalar returns a floating point array by design;  likewise,
operating on an integer array with an integer scalar returns an integer
array.  This is all expected behavior.

Problem g: unaddressed -- working as designed, open to discussion.  The
silent truncation of g to Int16 is documented in the description of
astype().  It's possible to add a check_overflow flag to astype()

Thanks for the feedback,
Todd

On Mon, 2003-08-18 at 09:20, Colin J. Williams wrote:
> # ta.py to check Int8, Int16
> import numarray as _n
> import numarray.numerictypes as _nt
> b= _n.arange(4, type= _nt.Int8)
> print 'b, b._type:', b, b._type
> c= b*1001                                        # Grief here - type not 
> updated
> print 'c, c._type:', c, c._type    
> e= _n.array([1, -2, 3000, 4.6], type= _nt.Int8)  # undocumented feature 
> fraction discarded
>                                                  # and lowest eight bits 
> retained as a signed
>                                                  # integer.
> print 'e, e._type:', e, e._type
> f= _n.array([1, 2, 3, 4.6], type= _nt.Int8) * 9.6
> print 'f, f._type:', f, f._type
> g= (f.copy()*2000).astype(_nt.Int16)             # undocumented - see e 
> above
> print 'g, g._type:', g, g._type
> 
> 
> --------------------------------------------------------------------
> PythonWin 2.3 (#46, Jul 29 2003, 18:54:32) [MSC v.1200 32 bit (Intel)] 
> on win32.
> Portions Copyright 1994-2001 Mark Hammond (mhammond at skippinet.com.au) - 
> see 'Help/About PythonWin' for further copyright information.
>  >>> b, b._type: [0 1 2 3] Int8
> c, c._type: [  0 -23 -46 -69] Int8
> e, e._type: [  1  -2 -72   4] Int8
> f, f._type: [  9.6  19.2  28.8  38.4] Float64
> g, g._type: [ 19200 -27136  -7937  11264] Int16
> 
> Colin W.
> 
> 
> 
> -------------------------------------------------------
> This SF.Net email sponsored by: Free pre-built ASP.NET sites including
> Data Reports, E-commerce, Portals, and Forums are available now.
> Download today and enter to win an XBOX or Visual Studio .NET.
> http://aspnet.click-url.com/go/psa00100003ave/direct;at.aspnet_072303_01/01
> _______________________________________________
> Numpy-discussion mailing list
> Numpy-discussion at lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/numpy-discussion
-- 
Todd Miller 			jmiller at stsci.edu
STSCI / ESS / SSB




From fperez at colorado.edu  Tue Sep  2 12:21:07 2003
From: fperez at colorado.edu (Fernando Perez)
Date: Tue Sep  2 12:21:07 2003
Subject: [Numpy-discussion] sum and mean methods behaviour
In-Reply-To: <1062527563.16365.283.camel@halloween.stsci.edu>
References: <200309011134.39281.verveer@embl-heidelberg.de> <1062527563.16365.283.camel@halloween.stsci.edu>
Message-ID: <3F54ED71.4070301@colorado.edu>

Todd Miller wrote:

> I thought about this a lot yesterday and today talked it over with
> Perry.   There are several ways to fix the problem with mean() and
> sum(), and I'm hoping that you and the rest of the community will help
> sort them out.

[snip]

Just a thought: why not make the upcasting an optional parameter?

I've found that python's arguments with default values provide a very 
convenient way of giving the user fine control with minimal conceptual overhead.

I'd rather write:

arr = array([255, 255], Int8)
... later
arr.sum(use_double=1)  # or some similar way of tuning sum()


than

arr = array([255, 255], Int8)
... later
array(arr,typecode='d').sum()

Numarray/numpy are trying to tackle an inherently hard problem: matching the 
high-level comfort of python with the low-level performance of C.  This 
situation is an excellent example of what I've seen described as the 'law of 
leaky abstractions':  in most cases where you encapsulate low level details in 
a high level abstraction, there end up being situations where the details poke 
through the abstraction and cause you grief.  This is an inherently tricky 
problem, with no easy, universal solution (that I'm aware of).

Cheers,

f.




From verveer at embl.de  Tue Sep  2 13:33:08 2003
From: verveer at embl.de (verveer at embl.de)
Date: Tue Sep  2 13:33:08 2003
Subject: [Numpy-discussion] sum and mean methods behaviour
In-Reply-To: <1062527563.16365.283.camel@halloween.stsci.edu>
References: <200309011134.39281.verveer@embl-heidelberg.de> <1062527563.16365.283.camel@halloween.stsci.edu>
Message-ID: <1062534753.3f54fe610ed4d@webmail.embl.de>

Hi Todd, 
 
> I thought about this a lot yesterday and today talked it over with 
> Perry.   There are several ways to fix the problem with mean() and 
> sum(), and I'm hoping that you and the rest of the community will help 
> sort them out. 
 
It was just an innocent question, I did not think it would have such 
ramifications :-)  
 
Here are my thoughts: 
 
If I understand you well, the sum() and mean() array methods are based on the 
reduce method of the universal functions. And these do their calculations in 
the precision of the array, is that correct? 
 
I also gave this some thought, and I would like to make a distinction between 
a reduction and the calculation of a statistical value such as the mean or the 
sum: 
 
To me, a reduction means the projection of an multi-dimensional array to an 
array with a rank that is one less than the input. The result is still an 
array, and often I want the result to have the same precision as the input.  
 
A statistical calculation like a sum or a mean is different: the result should 
be correct and the same irrespective of the type of the input, and that 
mandates using sufficient precision in the calculation. Note however, that 
such a statistic is a scalar result and does not require temporary storage at 
high precision for the whole array. 
 
So keeping this in mind, my comments to your solutions are: 
 
> (1) The first "solution" is to require users to do their own up-casting 
> prior to calling mean() or sum().  This gives the end user fine control 
> over storage cost but leaves the C-like pitfall/bug you discovered.   I 
> mention this because this is how the numarray/Numeric reductions are 
> designed.  Is there a reason why the numarray/Numeric reductions don't 
> implicitly up-cast? 
 
For reductions this behaviour suits me, precisely because it allows control 
over storage, which is one of the strengths of numarray. 
 
For calculating the mean or the sum of an array this is however an expensive 
solution for a very common operation. I do use this solution, but sometimes I 
prefer an optimized C routine instead. 
 
> (2) The second way is what you proposed:  use double precision within 
> mean and sum.  This has great simplicity but gives no control over 
> storage usage, and as implemented, the storage would be much higher than 
> one might think, potentially 8x. 
 
I did not want to suggest to store a casted version of the array before 
calculation of the mean or the sum. That can be done in double precision 
without converting the whole array in memory. I think we can all agree that 
this option would not be a good idea. 
 
> (3) Lastly, Perry suggested a more radical approach:  rather than 
> changing the mean and sum methods themselves,  we could alter the 
> universal function accumulate and reduce methods to implicitly use 
> additional precision.  Perry's idea was to make all accumulations and 
> reductions up-cast their results to the largest type of the current 
> family, either Bool, Int64, Float64, or Complex64.   By doing this, we 
> can improve the utility of the reductions and accumulations as well as 
> fixing the problem with sum and mean. 
 
I think that is a great idea in principle, but I think you should consider 
this carefully: 
 
First of all control of the storage cost is lost when you do a reduction. I 
would not find that always desirable. Thus, I would then like the old 
behaviour for reduction to be accesible either as a different method or by a 
setting an optional argument. 
 
Additionally, it would not work well for some operations. For instance precise 
calculation of the mean requires floating point precision. Maybe this can be 
solved, but would it require different casting behaviour for different 
operations. That might be too much trouble... 
 
I would like to propose a fourth option: 
 
(4) provide separate implementations for array methods like sum() and mean() 
that only calculate the scalar result. No additional storage would be 
necessary and the calculation can be done in double precision. 
 
I guess that the disadvantage is that one cannot leverage the existing code in 
the ufuncs so easily.  I also assume that it would not be such a general 
solution as changing the reduce method is. 
 
I do have some experience in writing these sorts of code for multidimensional 
arrays in C and would be happy to contribute code. However, I am not too 
familiar with the internals of numarray library and I don't know how well my 
code fits in there (although I interface all my code to numarray). But I am 
happy to help if I can, numarray is great stuff, it has become the main tool 
for my numerical work. 
 
Peter 
 
-- 
Dr. Peter J. Verveer 
Cell Biology and Cell Biophysics Programme 
EMBL 
Meyerhofstrasse 1 
D-69117 Heidelberg 
Germany 
Tel. : +49 6221 387245 
Fax  : +49 6221 387242 
 
 



From kern at caltech.edu  Tue Sep  2 14:32:03 2003
From: kern at caltech.edu (Robert Kern)
Date: Tue Sep  2 14:32:03 2003
Subject: [Numpy-discussion] sum and mean methods behaviour
In-Reply-To: <3F54ED71.4070301@colorado.edu>
References: <200309011134.39281.verveer@embl-heidelberg.de> <1062527563.16365.283.camel@halloween.stsci.edu> <3F54ED71.4070301@colorado.edu>
Message-ID: <20030902213010.GA27423@taliesen.caltech.edu>

On Tue, Sep 02, 2003 at 01:20:17PM -0600, Fernando Perez wrote:
> Todd Miller wrote:
> 
> >I thought about this a lot yesterday and today talked it over with
> >Perry.   There are several ways to fix the problem with mean() and
> >sum(), and I'm hoping that you and the rest of the community will help
> >sort them out.
> 
> [snip]
> 
> Just a thought: why not make the upcasting an optional parameter?
> 
> I've found that python's arguments with default values provide a very 
> convenient way of giving the user fine control with minimal conceptual 
> overhead.
> 
> I'd rather write:
> 
> arr = array([255, 255], Int8)
> ... later
> arr.sum(use_double=1)  # or some similar way of tuning sum()

+1, but

arr.sum(typecode=Float64)

would be my choice of spelling. Not sure what the default typecode
should be, though. Probably Perry's suggestion: the largest type of the
family.

-- 
Robert Kern
kern at caltech.edu

"In the fields of hell where the grass grows high
 Are the graves of dreams allowed to die."
  -- Richard Harter



From jmiller at stsci.edu  Wed Sep  3 05:44:11 2003
From: jmiller at stsci.edu (Todd Miller)
Date: Wed Sep  3 05:44:11 2003
Subject: [Numpy-discussion] sum and mean methods behaviour
In-Reply-To: <3F54ED71.4070301@colorado.edu>
References: <200309011134.39281.verveer@embl-heidelberg.de>
	 <1062527563.16365.283.camel@halloween.stsci.edu>
	 <3F54ED71.4070301@colorado.edu>
Message-ID: <1062592991.3460.89.camel@localhost.localdomain>

On Tue, 2003-09-02 at 15:20, Fernando Perez wrote:
> Todd Miller wrote:
> 
> > I thought about this a lot yesterday and today talked it over with
> > Perry.   There are several ways to fix the problem with mean() and
> > sum(), and I'm hoping that you and the rest of the community will help
> > sort them out.
> 
> [snip]
> 
> Just a thought: why not make the upcasting an optional parameter?
> 
<snip>
That sounds like a great idea.  Simple, but doesn't throw out all
storage control.

> in most cases where you encapsulate low level details in 
> a high level abstraction, there end up being situations where the details poke 
> through the abstraction and cause you grief.  

Thanks for these kind words.  

-- 
Todd Miller <jmiller at stsci.edu>




From verveer at embl-heidelberg.de  Wed Sep  3 08:40:20 2003
From: verveer at embl-heidelberg.de (Peter Verveer)
Date: Wed Sep  3 08:40:20 2003
Subject: [Numpy-discussion] sum and mean methods behaviour
In-Reply-To: <3F560505.7090209@pfdubois.com>
References: <200309011134.39281.verveer@embl-heidelberg.de> <1062527563.16365.283.camel@halloween.stsci.edu> <3F560505.7090209@pfdubois.com>
Message-ID: <200309031739.57327.verveer@embl-heidelberg.de>

I also believe that the current behavior for numarray/Numeric reduce method 
(not to cast) is the right one. It is fine to leave the user with the 
responsibility to be careful in the case of the reduce operation.

But to correctly calculate a mean or a sum by the array methods that are 
provided you have to convert the array first to a more precise type, and then 
do the calculation. That wastes space and is slow, and seems not very elegant 
considering that these are very common statistical operations.

A separate implementation for the mean() and sum() methods that uses double 
precision in the calculation without first converting the array would be 
straightforward. Since calculating a mean or a sum of a complete array is 
such a common case I think this would be useful.

That leaves the same problem for the reduce method which in some cases would 
require first a conversion, but this is much less of a problem (at least for 
me). Having to convert before the operation can be wasteful though.

I do like the idea that was also proposed on the list to supply an optional 
argument to specify the output type. Then the user has full control of the 
output type (nice if you want high precision in the result without converting 
the input), and the code can easily be used to implement the mean() and sum() 
methods. The default behavior of the reduce method can then remain unchanged, 
so this would not be an obtrusive change. But, I imagine that this may 
complicate the implementation.

Cheers, Peter

On Wednesday 03 September 2003 17:13, Paul Dubois wrote:
> So after you get the result in a higher precision, then what?
> a. Cast it down blindly?
> b. Test every element and throw an exception if casting would lose
> precision?
> c. Test every element and return the smallest kind that "holds" the answer?
> d. Always return the highest precision?
>
> a. is close to equivalent to the present behavior
> b. and c. are expensive.
> c. makes the type of the result unpredictable, which has its own problems.
> d. uses space
>
> It was the originally design of Numeric to be fast rather than careful,
> user beware. There is now a another considerable portion of the
> community that is for very careful, and another that is for keeping it
> small. You can't satisfy all those goals at once.
>
> If you make it slow or big in order to be careful, it will always be
> slow or big, while the opposite is not true. If you make it fast, the
> user can be careful.
>
> Todd Miller wrote:
> > On Mon, 2003-09-01 at 05:34, Peter Verveer wrote:
> >>Hi All,
> >>
> >>I noticed that the sum() and mean() methods of numarrays use the
> >> precision of
> >>
> >>the given array in their calculations. That leads to resuls like this:
> >>>>>array([255, 255], Int8).sum()
> >>
> >>-2
> >>
> >>>>>array([255, 255], Int8).mean()
> >>
> >>-1.0
> >>
> >>Would it not be better to use double precision internally and return the
> >>correct result?
> >>
> >>Cheers, Peter
> >
> > Hi Peter,
> >
> > I thought about this a lot yesterday and today talked it over with
> > Perry.   There are several ways to fix the problem with mean() and
> > sum(), and I'm hoping that you and the rest of the community will help
> > sort them out.
> >
> > (1) The first "solution" is to require users to do their own up-casting
> > prior to calling mean() or sum().  This gives the end user fine control
> > over storage cost but leaves the C-like pitfall/bug you discovered.   I
> > mention this because this is how the numarray/Numeric reductions are
> > designed.  Is there a reason why the numarray/Numeric reductions don't
> > implicitly up-cast?
> >
> > (2) The second way is what you proposed:  use double precision within
> > mean and sum.  This has great simplicity but gives no control over
> > storage usage, and as implemented, the storage would be much higher than
> > one might think, potentially 8x.
> >
> > (3) Lastly, Perry suggested a more radical approach:  rather than
> > changing the mean and sum methods themselves,  we could alter the
> > universal function accumulate and reduce methods to implicitly use
> > additional precision.  Perry's idea was to make all accumulations and
> > reductions up-cast their results to the largest type of the current
> > family, either Bool, Int64, Float64, or Complex64.   By doing this, we
> > can improve the utility of the reductions and accumulations as well as
> > fixing the problem with sum and mean.




From jmiller at stsci.edu  Wed Sep  3 14:47:04 2003
From: jmiller at stsci.edu (Todd Miller)
Date: Wed Sep  3 14:47:04 2003
Subject: [Numpy-discussion] sum and mean methods behaviour
In-Reply-To: <3F560505.7090209@pfdubois.com>
References: <200309011134.39281.verveer@embl-heidelberg.de>
	 <1062527563.16365.283.camel@halloween.stsci.edu>
	 <3F560505.7090209@pfdubois.com>
Message-ID: <1062625547.3459.707.camel@localhost.localdomain>

I want to thank everyone who participated in this discussion:  Peter,
Fernando, Robert, Paul, Perry, and Tim.

Tim's post has IMO a completely synthesized solution:  

1. Add a type parameter to sum which defaults to widest type.

2. Add a type parameter to reductions (and fix output type handling). 
Default is same-type as it is now.  No major changes to C-code.

3. Add a WidestType(array) function:

	Bool                        --> Bool
	Int8,Int16,Int32,Int64      --> Int64
	UInt8, UInt16,UInt32,UInt64 --> UInt64 (Int64 on win32)
	Float32, Float64            --> Float64
	Complex32, Complex64        --> Complex64

The only thing this really leaves out, is a higher performance
implementation of sum/mean which Peter referred to a few times.  Peter, 
if you want to write a specialized module, we'd be happy to put it in
the add-ons package.
	
Thanks again everybody,
Todd

-- 
Todd Miller <jmiller at stsci.edu>




From verveer at embl-heidelberg.de  Thu Sep  4 01:55:03 2003
From: verveer at embl-heidelberg.de (Peter Verveer)
Date: Thu Sep  4 01:55:03 2003
Subject: [Numpy-discussion] Re: Numpy-discussion digest, Vol 1 #693 - 3 msgs
In-Reply-To: <E19ukg2-0006wj-00@sc8-sf-list2.sourceforge.net>
References: <E19ukg2-0006wj-00@sc8-sf-list2.sourceforge.net>
Message-ID: <200309041054.09224.verveer@embl-heidelberg.de>

> 1. Add a type parameter to sum which defaults to widest type.
>
> 2. Add a type parameter to reductions (and fix output type handling).
> Default is same-type as it is now.  No major changes to C-code.
>
> 3. Add a WidestType(array) function:
>
>         Bool                        --> Bool
>         Int8,Int16,Int32,Int64      --> Int64
>         UInt8, UInt16,UInt32,UInt64 --> UInt64 (Int64 on win32)
>         Float32, Float64            --> Float64
>         Complex32, Complex64        --> Complex64

This sounds like a good solution.

> The only thing this really leaves out, is a higher performance
> implementation of sum/mean which Peter referred to a few times. 

Is this really the case? It depends on how you plan to implement the output 
conversion. If you will do this by allocating a temporary converted copy of 
the complete input before the calculations then this potentially requires a 
lot of temporary storage. But it is certainly possible to come up with an 
implementation that avoids this. Have you given this some thought?

> Peter,
> if you want to write a specialized module, we'd be happy to put it in
> the add-ons package.

I hope that the reduce methods can be made sufficiently efficient so that this 
will not be necessary. 

Cheers, Peter




From alnesbit at students.cs.mu.OZ.AU  Thu Sep  4 02:59:05 2003
From: alnesbit at students.cs.mu.OZ.AU (Andrew Nesbit)
Date: Thu Sep  4 02:59:05 2003
Subject: [Numpy-discussion] Extracting column vectors from matrix.
Message-ID: <Pine.GSO.4.56.0309041946040.904@munyoo.cs.mu.OZ.AU>

Say that I have a rank-2 array (matrix) of shape (m,n).  Call it A.

When extracting column vectors from A, I often want to keep the result
as a rank-2 column vector (for subsequent matrix multiplications,
etc), so I usually end up writing something ugly like this:

column_vector = reshape(A[:,col],(m,1))

I've got a function to wrap this, but is there a builtin (or cleaner)
way to do this sort of operation?

TIA.

Andrew.




From verveer at embl-heidelberg.de  Thu Sep  4 04:26:04 2003
From: verveer at embl-heidelberg.de (Peter Verveer)
Date: Thu Sep  4 04:26:04 2003
Subject: [Numpy-discussion] Re: sum and mean methods behaviour
Message-ID: <200309041325.13324.verveer@embl-heidelberg.de>

Hi all,

I was thinking a bit more about the changes to reduce() that Todd proposed, 
and have some questions:

The problem that the output may not be able to hold the result of an operation 
is not unique to the reduce() method. For instance adding two arrays of type 
UInt can also give you the wrong answer: 

>>> array(255, UInt8) + array(255, UInt8)
254

So, if this is a general problem, why should only the reduce method be 
enhanced to avoid this? If you implement this, should this capability not be 
supported more broadly than only by reduce(), for instance by universal 
functons such as 'add'? Would it not be unexpected for users that only 
reduce() provides such added functionality?

However, as Paul Dubois pointed out earlier, the original design philosphy of 
Numeric/numarray was to let the user deal with such problems himself and keep 
the package small and fast. This seems actually a sound decision, so would it 
not be better to avoid complicating numarray with these type of changes and 
also leave reduce as it is?

Personally I don't have a need for the proposed changes to the reduce 
function. My original complaint that started the whole discussion was that 
the mean() and sum() array methods did not give the correct result in some 
cases. I still think they should return a correct double precision value, 
even if the universal functions may not. That could be achieved by a separate  
implementation that does not use the universal functions. I would be prepared 
to provide that implementation either to replace the mean and sum methods, or 
as a separate add-on.

Cheers, Peter

> 1. Add a type parameter to sum which defaults to widest type.
>
> 2. Add a type parameter to reductions (and fix output type handling).
> Default is same-type as it is now.  No major changes to C-code.
>
> 3. Add a WidestType(array) function:
>
>         Bool                        --> Bool
>         Int8,Int16,Int32,Int64      --> Int64
>         UInt8, UInt16,UInt32,UInt64 --> UInt64 (Int64 on win32)
>         Float32, Float64            --> Float64
>         Complex32, Complex64        --> Complex64
>
> The only thing this really leaves out, is a higher performance
> implementation of sum/mean which Peter referred to a few times. 
> Peter, if you want to write a specialized module, we'd be happy 
> to put it in the add-ons package.
 



From hinsen at cnrs-orleans.fr  Thu Sep  4 05:37:05 2003
From: hinsen at cnrs-orleans.fr (Konrad Hinsen)
Date: Thu Sep  4 05:37:05 2003
Subject: [Numpy-discussion] Extracting column vectors from matrix.
In-Reply-To: <Pine.GSO.4.56.0309041946040.904@munyoo.cs.mu.OZ.AU>
References: <Pine.GSO.4.56.0309041946040.904@munyoo.cs.mu.OZ.AU>
Message-ID: <200309041433.47180.hinsen@cnrs-orleans.fr>

On Thursday 04 September 2003 11:58, Andrew Nesbit wrote:
> Say that I have a rank-2 array (matrix) of shape (m,n).  Call it A.
>
> When extracting column vectors from A, I often want to keep the result
> as a rank-2 column vector (for subsequent matrix multiplications,
> etc), so I usually end up writing something ugly like this:
>
> column_vector = reshape(A[:,col],(m,1))
>
> I've got a function to wrap this, but is there a builtin (or cleaner)
> way to do this sort of operation?

1) A[:, col:col+1]

or perhaps

2) A[:, col][:, NewAxis]

Konrad.
-- 
-------------------------------------------------------------------------------
Konrad Hinsen                            | E-Mail: hinsen at cnrs-orleans.fr
Centre de Biophysique Moleculaire (CNRS) | Tel.: +33-2.38.25.56.24
Rue Charles Sadron                       | Fax:  +33-2.38.63.15.17
45071 Orleans Cedex 2                    | Deutsch/Esperanto/English/
France                                   | Nederlands/Francais
-------------------------------------------------------------------------------




From perry at stsci.edu  Thu Sep  4 06:12:04 2003
From: perry at stsci.edu (Perry Greenfield)
Date: Thu Sep  4 06:12:04 2003
Subject: [Numpy-discussion] Re: sum and mean methods behaviour
In-Reply-To: <200309041054.09224.verveer@embl-heidelberg.de>
Message-ID: <CEELJPECNGEGKFNDLHFFEEJJCAAA.perry@stsci.edu>

Peter Verveer writes:
> Todd Miller writes:
> > 1. Add a type parameter to sum which defaults to widest type.
> >
> > 2. Add a type parameter to reductions (and fix output type handling).
> > Default is same-type as it is now.  No major changes to C-code.
> >
> > 3. Add a WidestType(array) function:
> >
> >         Bool                        --> Bool
> >         Int8,Int16,Int32,Int64      --> Int64
> >         UInt8, UInt16,UInt32,UInt64 --> UInt64 (Int64 on win32)
> >         Float32, Float64            --> Float64
> >         Complex32, Complex64        --> Complex64
>
> This sounds like a good solution.
>
> > The only thing this really leaves out, is a higher performance
> > implementation of sum/mean which Peter referred to a few times.
>
> Is this really the case? It depends on how you plan to implement
> the output
> conversion. If you will do this by allocating a temporary
> converted copy of
> the complete input before the calculations then this potentially
> requires a
> lot of temporary storage. But it is certainly possible to come up with an
> implementation that avoids this. Have you given this some thought?
>
The way numarray ufuncs currently work is that for larger arrays,
the conversion of types is only done in blocks. No temporary copy is
made of the whole input array. So the temporary storage issue is not
a problem.




From alnesbit at students.cs.mu.OZ.AU  Thu Sep  4 06:34:19 2003
From: alnesbit at students.cs.mu.OZ.AU (Andrew Nesbit)
Date: Thu Sep  4 06:34:19 2003
Subject: [Numpy-discussion] Extracting column vectors from matrix.
In-Reply-To: <200309041433.47180.hinsen@cnrs-orleans.fr> (Konrad Hinsen's
 message of "Thu, 4 Sep 2003 14:33:47 +0200")
References: <Pine.GSO.4.56.0309041946040.904@munyoo.cs.mu.OZ.AU>
	<200309041433.47180.hinsen@cnrs-orleans.fr>
Message-ID: <m2r82wptjk.fsf@naima.lensflare.org>

Konrad Hinsen <hinsen at cnrs-orleans.fr> writes:

> 1) A[:, col:col+1]
>
> or perhaps
>
> 2) A[:, col][:, NewAxis]

Yep, those are the sort of tricks I was looking for.  Thankyou.

Andrew.




From perry at stsci.edu  Thu Sep  4 06:35:03 2003
From: perry at stsci.edu (Perry Greenfield)
Date: Thu Sep  4 06:35:03 2003
Subject: [Numpy-discussion] Re: sum and mean methods behaviour
In-Reply-To: <200309041325.13324.verveer@embl-heidelberg.de>
Message-ID: <CEELJPECNGEGKFNDLHFFMEJJCAAA.perry@stsci.edu>

Peter Verveer writes

> Hi all,
>
> I was thinking a bit more about the changes to reduce() that Todd
> proposed,
> and have some questions:
>
> The problem that the output may not be able to hold the result of
> an operation
> is not unique to the reduce() method. For instance adding two
> arrays of type
> UInt can also give you the wrong answer:
>
> >>> array(255, UInt8) + array(255, UInt8)
> 254
>
> So, if this is a general problem, why should only the reduce method be
> enhanced to avoid this? If you implement this, should this
> capability not be
> supported more broadly than only by reduce(), for instance by universal
> functons such as 'add'? Would it not be unexpected for users that only
> reduce() provides such added functionality?
>
Certainly true (and much more likely a problem for integer multiplication
than addition). On the other hand, it is more likely to be only an
occasional problem for binary operations. With reductions, the risk is
severe that overflows will happen. For example, for addition it is
the difference between a+a for the normal operation and len(a)*a for
the reduction. Arguably reductions on Int8 and Int16 arrays are likely
to run into a problem than not.

> However, as Paul Dubois pointed out earlier, the original design
> philosphy of
> Numeric/numarray was to let the user deal with such problems
> himself and keep
> the package small and fast. This seems actually a sound decision,
> so would it
> not be better to avoid complicating numarray with these type of
> changes and
> also leave reduce as it is?
>
No, I'm inclined to change reductions because of the high potential
for problems, particularly with ints. I don't think ufunc type handling
needs to change though. Todd believes that changing reduction behavior
would not be difficult (though we will try to finish other work first before
doing that). Changing reduction behavior is probably the easiest way
of implementing the improved sum and mean functions. The only thing we
need to determine is what the default behavior is (Todd proposes
the defaults remain the same, I'm not so sure.)

> Personally I don't have a need for the proposed changes to the reduce
> function. My original complaint that started the whole discussion
> was that
> the mean() and sum() array methods did not give the correct
> result in some
> cases. I still think they should return a correct double precision value,
> even if the universal functions may not. That could be achieved
> by a separate
> implementation that does not use the universal functions. I would
> be prepared
> to provide that implementation either to replace the mean and sum
> methods, or
> as a separate add-on.
>
>




From verveer at embl-heidelberg.de  Thu Sep  4 06:53:05 2003
From: verveer at embl-heidelberg.de (Peter Verveer)
Date: Thu Sep  4 06:53:05 2003
Subject: [Numpy-discussion] Re: sum and mean methods behaviour
In-Reply-To: <CEELJPECNGEGKFNDLHFFMEJJCAAA.perry@stsci.edu>
References: <CEELJPECNGEGKFNDLHFFMEJJCAAA.perry@stsci.edu>
Message-ID: <200309041552.34197.verveer@embl-heidelberg.de>

On Thursday 04 September 2003 15:33, Perry Greenfield wrote:
> > So, if this is a general problem, why should only the reduce method be
> > enhanced to avoid this? If you implement this, should this
> > capability not be
> > supported more broadly than only by reduce(), for instance by universal
> > functons such as 'add'? Would it not be unexpected for users that only
> > reduce() provides such added functionality?
>
> Certainly true (and much more likely a problem for integer multiplication
> than addition). On the other hand, it is more likely to be only an
> occasional problem for binary operations. With reductions, the risk is
> severe that overflows will happen. For example, for addition it is
> the difference between a+a for the normal operation and len(a)*a for
> the reduction. Arguably reductions on Int8 and Int16 arrays are likely
> to run into a problem than not.

That true, but this argument really only holds for the integer types. For 
32-bit floating point or complex types it will usually not be necessary to 
convert to 64-bit to prevent overflow. In that case it may often not be 
desirable to change the array type. I am not saying that the convert option 
would not useful for the case of floats, but it is maybe an argument to keep 
the default behaviour, at least for Float32 and Complex32 types.

Generally I do agree that there is no need to change the ufuncs, I did not 
want to suggest that this actually be implemented...

> > However, as Paul Dubois pointed out earlier, the original design
> > philosphy of
> > Numeric/numarray was to let the user deal with such problems
> > himself and keep
> > the package small and fast. This seems actually a sound decision,
> > so would it
> > not be better to avoid complicating numarray with these type of
> > changes and
> > also leave reduce as it is?
>
> No, I'm inclined to change reductions because of the high potential
> for problems, particularly with ints. I don't think ufunc type handling
> needs to change though. Todd believes that changing reduction behavior
> would not be difficult (though we will try to finish other work first
> before doing that). Changing reduction behavior is probably the easiest way
> of implementing the improved sum and mean functions. The only thing we need
> to determine is what the default behavior is (Todd proposes
> the defaults remain the same, I'm not so sure.)

This would solve my problem with mean() and sum(). I think these should 
certainly return the result in the optimal precision. These may then not be 
the most optimal in terms of speed, but certainly 'good enough'. I would  
like to second Todds preference to keep the default behaviour of reductions 
to be the same as it is now. For reductions, I mostly want the result to be 
in the same type, because I chose that type in the first place for storage 
reasons.

Cheers, Peter



From alnesbit at students.cs.mu.oz.au  Thu Sep  4 07:46:55 2003
From: alnesbit at students.cs.mu.oz.au (Andrew Nesbit)
Date: Thu Sep  4 07:46:55 2003
Subject: [Numpy-discussion] Matrix square root
Message-ID: <m2ekywpqd8.fsf@naima.lensflare.org>

I need a function equivalent to Matlab's sqrtm, i.e., a square root
for matrices.

I constructed the following definition:

def sqrtm(M):
    (U,S,VT) = LinearAlgebra.singular_value_decomposition(M)
    D = MLab.diag(sqrt(S))
    return matrixmultiply(matrixmultiply(U,D),VT)

but this technique only works for Hermitian, positive definite
matrices.  The matrices I operate on don't necessarily satisfy that
condition.

Does anybody know of a more general technique, preferably with an
accompanying Python implementation?

Thanks.

Andrew.




From hinsen at cnrs-orleans.fr  Thu Sep  4 08:48:44 2003
From: hinsen at cnrs-orleans.fr (Konrad Hinsen)
Date: Thu Sep  4 08:48:44 2003
Subject: [Numpy-discussion] Matrix square root
In-Reply-To: <m2ekywpqd8.fsf@naima.lensflare.org>
References: <m2ekywpqd8.fsf@naima.lensflare.org>
Message-ID: <200309041707.34519.hinsen@cnrs-orleans.fr>

On Thursday 04 September 2003 16:42, Andrew Nesbit wrote:
> I need a function equivalent to Matlab's sqrtm, i.e., a square root
> for matrices.
>
> I constructed the following definition:
>
> def sqrtm(M):
>     (U,S,VT) = LinearAlgebra.singular_value_decomposition(M)
>     D = MLab.diag(sqrt(S))
>     return matrixmultiply(matrixmultiply(U,D),VT)
>
> but this technique only works for Hermitian, positive definite
> matrices.  The matrices I operate on don't necessarily satisfy that
> condition.

I'd use an eigenvalue decomposition, then take the square root of the 
eigenvalues, and then apply the diagonlization matrix in reverse. If you 
convert to eigenvalues to complex before taking the square root, this will 
work for non-positive-definite matrices, yielding a complex result.

Konrad.
-- 
-------------------------------------------------------------------------------
Konrad Hinsen                            | E-Mail: hinsen at cnrs-orleans.fr
Centre de Biophysique Moleculaire (CNRS) | Tel.: +33-2.38.25.56.24
Rue Charles Sadron                       | Fax:  +33-2.38.63.15.17
45071 Orleans Cedex 2                    | Deutsch/Esperanto/English/
France                                   | Nederlands/Francais
-------------------------------------------------------------------------------




From alnesbit at students.cs.mu.OZ.AU  Thu Sep  4 09:05:18 2003
From: alnesbit at students.cs.mu.OZ.AU (Andrew Nesbit)
Date: Thu Sep  4 09:05:18 2003
Subject: [Numpy-discussion] Matrix square root
In-Reply-To: <200309041707.34519.hinsen@cnrs-orleans.fr> (Konrad Hinsen's
 message of "Thu, 4 Sep 2003 17:07:34 +0200")
References: <m2ekywpqd8.fsf@naima.lensflare.org>
	<200309041707.34519.hinsen@cnrs-orleans.fr>
Message-ID: <m265k8o8q0.fsf@naima.lensflare.org>

Konrad Hinsen <hinsen at cnrs-orleans.fr> writes:

> On Thursday 04 September 2003 16:42, Andrew Nesbit wrote:
>> I need a function equivalent to Matlab's sqrtm, i.e., a square root
>> for matrices.

[snip]

> I'd use an eigenvalue decomposition, then take the square root of the 
> eigenvalues, and then apply the diagonlization matrix in reverse. If you 
> convert to eigenvalues to complex before taking the square root, this will 
> work for non-positive-definite matrices, yielding a complex result.

Thankyou for the advice.  This gives me a good starting point.

Andrew.




From verveer at embl.de  Fri Sep  5 11:47:02 2003
From: verveer at embl.de (verveer at embl.de)
Date: Fri Sep  5 11:47:02 2003
Subject: [Numpy-discussion] bug?
Message-ID: <1062787560.3f58d9e903356@webmail.embl.de>

possible bug? 
 
>>> a = array([[[1, 1], [0, 1]]]) 
>>> print logical_or.reduce(a, 0) 
[[1 0] 
 [1 1]] 
 
should be: 
 
>>> print a[0, ...] 
[[1 1] 
 [0 1]] 
 
Cheers, Peter 
-- 
Dr. Peter J. Verveer 
Cell Biology and Cell Biophysics Programme 
EMBL 
Meyerhofstrasse 1 
D-69117 Heidelberg 
Germany 
Tel. : +49 6221 387245 
Fax  : +49 6221 387242 
 



From cjw at sympatico.ca  Fri Sep  5 12:04:02 2003
From: cjw at sympatico.ca (Colin J. Williams)
Date: Fri Sep  5 12:04:02 2003
Subject: [Numpy-discussion] Attempting to sub-class NumArray
Message-ID: <3F58DDE5.5090608@sympatico.ca>

When ravel() is used on an instance of NumArray, the _strides attribute 
is recalculated correctly.
When ravel() is used on an instance of special, a sub-class of NumArray, 
the _strides attribute keeps its old value.  This appears to cause the 
display problem which is reported.

The essentials of the code used are below.

To investigate this problem, two print statements were added to the 
method setshape in generic.py:

            if newnelements == nelements:
                self._shape = tuple(shape)
                print 'In setShape self._shape:', self._shape
                self._strides = self._stridesFromShape()
                print 'In setShape self._strides:', self._strides
            else:
                raise ValueError("New shape is not consistent with the
    old shape")

It seems that only the first of these print statements is executed with 
special, but both are executed with NumArray.  In the second case, the 
output is not in the expected sequence.

I would appreciate any suggestion as to a workaround.

Colin W,

Grief when attempting to sub-class NumArray

Code used:

# Try to sub-class
class special(N.NumArray):
  def __init__(self, data= None, shape= None, eType= None):
##                         eType= _nt.Any or AnyType not acceptable %%
    arr= N.array(sequence= data, shape= shape, type= eType)
    for attr in dir(arr):
##      This is a longwinded way of setting up special
##      There must be a better way                     %%
##      Perhaps we should use generic._view
##      It's not documented                 TRY IT LATER
##      There is no need to transfer methods!
      if attr[0] == '_' and attr[1] != '_':
        print attr
        exec 'self.' + attr + '= ' 'arr.' + attr
        exec 'print self.' + attr
    pass

a= special(data= [1, 2, 3, 4], shape= (2, 2))
a= N.array([1, 2, 3, 4], shape= (2, 2))
print 'In ts a._strides:', a._strides
b= _gen.ravel(a)
print 'In ts b._strides:', b._strides       #  <<< Unchanged with 
special, OK with NumArray
print 'b:', b



USING THE SUBCLASS special
 >C:\Progra~1\Python23\pythonw -u ts.py
In ts a._strides: (8, 4)
In setShape self._shape: (4,)
In ts b._strides: (8, 4)
b: In setShape self._shape: (4,)
Traceback (most recent call last):
  File "ts.py", line 37, in ?
    print 'b:', b
  File 
"C:\PROGRA~1\Python23\lib\site-packages\numarray\numarraycore.py", line 
618, in __str__
    MAX_LINE_WIDTH, PRECISION, SUPPRESS_SMALL, ' ', "")
  File "C:\PROGRA~1\Python23\lib\site-packages\numarray\arrayprint.py", 
line 176, in array2string
    separator, prefix)
  File "C:\PROGRA~1\Python23\lib\site-packages\numarray\arrayprint.py", 
line 138, in _array2string
    max_str_len = max(len(str(max_reduce(data))),
libnumarray.error: maximum_Int32_reduce: access beyond buffer. offset=27 
buffersize=16
 >Exit code: 1
__________________________________________________________

# USING NumArray

In setShape self._shape: (2, 2)
In ts a._strides: (8, 4)
In setShape self._shape: (4,)
In ts b._strides: (4,)
b: In setShape self._shape: (4,)
[1 2 3 4]
 >Exit code: 0





From jmiller at stsci.edu  Fri Sep  5 15:04:06 2003
From: jmiller at stsci.edu (Todd Miller)
Date: Fri Sep  5 15:04:06 2003
Subject: [Numpy-discussion] Attempting to sub-class NumArray
In-Reply-To: <3F58DDE5.5090608@sympatico.ca>
References: <3F58DDE5.5090608@sympatico.ca>
Message-ID: <1062799403.5615.25.camel@halloween.stsci.edu>

Hi Colin,

I haven't looked at this in detail,  but special.__init__() is making me
queasy.  Why doesn't it call NumArray.__init__()?

Todd

On Fri, 2003-09-05 at 15:03, Colin J. Williams wrote:
> When ravel() is used on an instance of NumArray, the _strides attribute 
> is recalculated correctly.
> When ravel() is used on an instance of special, a sub-class of NumArray, 
> the _strides attribute keeps its old value.  This appears to cause the 
> display problem which is reported.
> 
> The essentials of the code used are below.
> 
> To investigate this problem, two print statements were added to the 
> method setshape in generic.py:
> 
>             if newnelements == nelements:
>                 self._shape = tuple(shape)
>                 print 'In setShape self._shape:', self._shape
>                 self._strides = self._stridesFromShape()
>                 print 'In setShape self._strides:', self._strides
>             else:
>                 raise ValueError("New shape is not consistent with the
>     old shape")
> 
> It seems that only the first of these print statements is executed with 
> special, but both are executed with NumArray.  In the second case, the 
> output is not in the expected sequence.
> 
> I would appreciate any suggestion as to a workaround.
> 
> Colin W,
> 
> Grief when attempting to sub-class NumArray
> 
> Code used:
> 
> # Try to sub-class
> class special(N.NumArray):
>   def __init__(self, data= None, shape= None, eType= None):
> ##                         eType= _nt.Any or AnyType not acceptable %%
>     arr= N.array(sequence= data, shape= shape, type= eType)
>     for attr in dir(arr):
> ##      This is a longwinded way of setting up special
> ##      There must be a better way                     %%
> ##      Perhaps we should use generic._view
> ##      It's not documented                 TRY IT LATER
> ##      There is no need to transfer methods!
>       if attr[0] == '_' and attr[1] != '_':
>         print attr
>         exec 'self.' + attr + '= ' 'arr.' + attr
>         exec 'print self.' + attr
>     pass
> 
> a= special(data= [1, 2, 3, 4], shape= (2, 2))
> a= N.array([1, 2, 3, 4], shape= (2, 2))
> print 'In ts a._strides:', a._strides
> b= _gen.ravel(a)
> print 'In ts b._strides:', b._strides       #  <<< Unchanged with 
> special, OK with NumArray
> print 'b:', b
> 
> 
> 
> USING THE SUBCLASS special
>  >C:\Progra~1\Python23\pythonw -u ts.py
> In ts a._strides: (8, 4)
> In setShape self._shape: (4,)
> In ts b._strides: (8, 4)
> b: In setShape self._shape: (4,)
> Traceback (most recent call last):
>   File "ts.py", line 37, in ?
>     print 'b:', b
>   File 
> "C:\PROGRA~1\Python23\lib\site-packages\numarray\numarraycore.py", line 
> 618, in __str__
>     MAX_LINE_WIDTH, PRECISION, SUPPRESS_SMALL, ' ', "")
>   File "C:\PROGRA~1\Python23\lib\site-packages\numarray\arrayprint.py", 
> line 176, in array2string
>     separator, prefix)
>   File "C:\PROGRA~1\Python23\lib\site-packages\numarray\arrayprint.py", 
> line 138, in _array2string
>     max_str_len = max(len(str(max_reduce(data))),
> libnumarray.error: maximum_Int32_reduce: access beyond buffer. offset=27 
> buffersize=16
>  >Exit code: 1
> __________________________________________________________
> 
> # USING NumArray
> 
> In setShape self._shape: (2, 2)
> In ts a._strides: (8, 4)
> In setShape self._shape: (4,)
> In ts b._strides: (4,)
> b: In setShape self._shape: (4,)
> [1 2 3 4]
>  >Exit code: 0
> 
> 
> 
> 
> -------------------------------------------------------
> This sf.net email is sponsored by:ThinkGeek
> Welcome to geek heaven.
> http://thinkgeek.com/sf
> _______________________________________________
> Numpy-discussion mailing list
> Numpy-discussion at lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/numpy-discussion
-- 
Todd Miller 			jmiller at stsci.edu
STSCI / ESS / SSB




From frankpit at erols.com  Sat Sep  6 07:28:02 2003
From: frankpit at erols.com (Bernard Frankpitt)
Date: Sat Sep  6 07:28:02 2003
Subject: [Numpy-discussion] Matrix square root
References: <m2ekywpqd8.fsf@naima.lensflare.org>	<200309041707.34519.hinsen@cnrs-orleans.fr> <m265k8o8q0.fsf@naima.lensflare.org>
Message-ID: <3F59F1D8.5020900@erols.com>

Andrew Nesbit wrote:

>Konrad Hinsen <hinsen at cnrs-orleans.fr> writes:
>
>>On Thursday 04 September 2003 16:42, Andrew Nesbit wrote:
>>    
>>
>>>I need a function equivalent to Matlab's sqrtm, i.e., a square root
>>>for matrices
>>>
>[snip]
>
>>I'd use an eigenvalue decomposition, then take the square root of the 
>>eigenvalues, and then apply the diagonlization matrix in reverse. If you 
>>convert to eigenvalues to complex before taking the square root, this will 
>>work for non-positive-definite matrices, yielding a complex result
>>
>
>Thankyou for the advice.  This gives me a good starting point.
>
>Andrew.
>
>  
>
The only problem with this approach is that you have to find all the 
eigenvalues and eigenvectors to get the spectral decomposition.  For 
large, poorly conditioned matrices this can be a source of significant 
errors, and a lot of work. An alternative, and often faster and more 
accurate approach is to use a series expansion for the square root. 
Evaluating the series expansion only requires multiplication.

Bernie






From cjw at sympatico.ca  Sat Sep  6 08:26:04 2003
From: cjw at sympatico.ca (Colin J. Williams)
Date: Sat Sep  6 08:26:04 2003
Subject: [Numpy-discussion] Attempting to sub-class NumArray
In-Reply-To: <1062848229.3459.21.camel@localhost.localdomain>
References: <3F58DDE5.5090608@sympatico.ca>	 <1062799403.5615.25.camel@halloween.stsci.edu>	 <3F592B3A.5060505@sympatico.ca> <1062848229.3459.21.camel@localhost.localdomain>
Message-ID: <3F59FC78.3060307@sympatico.ca>

Todd Miller wrote:

>I think the problem you are having is caused by the assumption that
>instance initialization is equivalent to an unordered assignment of all
>instance attributes.  That's not true because the private attributes are
>interrelated;  assigning "shape" affects "_shape" and "_strides",
>_itemsize must be known before _strides can be computed from shape, and
>so on. 
>
It would be helpful if you could let us know what assignment ordering 
would be appropriate, or, if there were information about the 
interactions, perhaps we could work these things out for ourselves.

As we've discussed before, there is little information available on 
private objects and their roles.  Even to have the source code of some 
of the C code might be useful.  I wasn't able to spot the relevant C 
code among the CVS directories.

However, the assumption would appear to apply to the crude 
initialization of 'special'.  The problem which is being reported seems 
to be connected with the function setshape in generic.py.  This is far 
from the unordered assignment, although I recognize that ther might be 
some connection.

Colin W.

>
>On Fri, 2003-09-05 at 20:32, Colin J. Williams wrote:
>  
>
>>Todd,
>>
>>Thanks again for your prompt response.
>>Todd Miller wrote:
>>    
>>
>>>Hi Colin,
>>>
>>>I haven't looked at this in detail,  but special.__init__() is making me
>>>queasy.  
>>>      
>>>
>>We don't call __init__ directly, but only when we construct the
>>instance, a,  of the class special.
>>    
>>
>>>Why doesn't it call NumArray.__init__()?
>>>      
>>>
>>Because we have data to install in the newly created a.  We go off to
>>the factory function, numarray.array, to create the array, which we
>>then specialize.
>>
>>I don't see how NumArray.__init__ would help.  This cumbersome
>>procedure seems to work OK in most respects, but fails to display
>>properly.
>>
>>I don't follow the code sufficiently well to be sure, but my hunch is
>>that the problem lies with setShape.  It fails to update _strides.
>>
>>Colin W.
>>    
>>
>>>Todd
>>>
>>>On Fri, 2003-09-05 at 15:03, Colin J. Williams wrote:
>>>  
>>>      
>>>
>>>>When ravel() is used on an instance of NumArray, the _strides attribute 
>>>>is recalculated correctly.
>>>>When ravel() is used on an instance of special, a sub-class of NumArray, 
>>>>the _strides attribute keeps its old value.  This appears to cause the 
>>>>display problem which is reported.
>>>>
>>>>The essentials of the code used are below.
>>>>
>>>>To investigate this problem, two print statements were added to the 
>>>>method setshape in generic.py:
>>>>
>>>>            if newnelements == nelements:
>>>>                self._shape = tuple(shape)
>>>>                print 'In setShape self._shape:', self._shape
>>>>                self._strides = self._stridesFromShape()
>>>>                print 'In setShape self._strides:', self._strides
>>>>            else:
>>>>                raise ValueError("New shape is not consistent with the
>>>>    old shape")
>>>>
>>>>It seems that only the first of these print statements is executed with 
>>>>special, but both are executed with NumArray.  In the second case, the 
>>>>output is not in the expected sequence.
>>>>
>>>>I would appreciate any suggestion as to a workaround.
>>>>
>>>>Colin W,
>>>>
>>>>Grief when attempting to sub-class NumArray
>>>>
>>>>Code used:
>>>>
>>>># Try to sub-class
>>>>class special(N.NumArray):
>>>>  def __init__(self, data= None, shape= None, eType= None):
>>>>##                         eType= _nt.Any or AnyType not acceptable %%
>>>>    arr= N.array(sequence= data, shape= shape, type= eType)
>>>>    for attr in dir(arr):
>>>>##      This is a longwinded way of setting up special
>>>>##      There must be a better way                     %%
>>>>##      Perhaps we should use generic._view
>>>>##      It's not documented                 TRY IT LATER
>>>>##      There is no need to transfer methods!
>>>>      if attr[0] == '_' and attr[1] != '_':
>>>>        print attr
>>>>        exec 'self.' + attr + '= ' 'arr.' + attr
>>>>        exec 'print self.' + attr
>>>>    pass
>>>>
>>>>a= special(data= [1, 2, 3, 4], shape= (2, 2))
>>>>a= N.array([1, 2, 3, 4], shape= (2, 2))
>>>>print 'In ts a._strides:', a._strides
>>>>b= _gen.ravel(a)
>>>>print 'In ts b._strides:', b._strides       #  <<< Unchanged with 
>>>>special, OK with NumArray
>>>>print 'b:', b
>>>>
>>>>
>>>>
>>>>USING THE SUBCLASS special
>>>> >C:\Progra~1\Python23\pythonw -u ts.py
>>>>In ts a._strides: (8, 4)
>>>>In setShape self._shape: (4,)
>>>>In ts b._strides: (8, 4)
>>>>b: In setShape self._shape: (4,)
>>>>Traceback (most recent call last):
>>>>  File "ts.py", line 37, in ?
>>>>    print 'b:', b
>>>>  File 
>>>>"C:\PROGRA~1\Python23\lib\site-packages\numarray\numarraycore.py", line 
>>>>618, in __str__
>>>>    MAX_LINE_WIDTH, PRECISION, SUPPRESS_SMALL, ' ', "")
>>>>  File "C:\PROGRA~1\Python23\lib\site-packages\numarray\arrayprint.py", 
>>>>line 176, in array2string
>>>>    separator, prefix)
>>>>  File "C:\PROGRA~1\Python23\lib\site-packages\numarray\arrayprint.py", 
>>>>line 138, in _array2string
>>>>    max_str_len = max(len(str(max_reduce(data))),
>>>>libnumarray.error: maximum_Int32_reduce: access beyond buffer. offset=27 
>>>>buffersize=16
>>>> >Exit code: 1
>>>>__________________________________________________________
>>>>
>>>># USING NumArray
>>>>
>>>>In setShape self._shape: (2, 2)
>>>>In ts a._strides: (8, 4)
>>>>In setShape self._shape: (4,)
>>>>In ts b._strides: (4,)
>>>>b: In setShape self._shape: (4,)
>>>>[1 2 3 4]
>>>> >Exit code: 0
>>>>
>>>>
>>>>
>>>>
>>>>-------------------------------------------------------
>>>>This sf.net email is sponsored by:ThinkGeek
>>>>Welcome to geek heaven.
>>>>http://thinkgeek.com/sf
>>>>_______________________________________________
>>>>Numpy-discussion mailing list
>>>>Numpy-discussion at lists.sourceforge.net
>>>>https://lists.sourceforge.net/lists/listinfo/numpy-discussion
>>>>    
>>>>        
>>>>





From alnesbit at students.cs.mu.oz.au  Sun Sep  7 03:50:07 2003
From: alnesbit at students.cs.mu.oz.au (Andrew Nesbit)
Date: Sun Sep  7 03:50:07 2003
Subject: [Numpy-discussion] Matrix square root
In-Reply-To: <3F59F1D8.5020900@erols.com> (Bernard Frankpitt's message of
 "Sat, 06 Sep 2003 10:40:24 -0400")
References: <m2ekywpqd8.fsf@naima.lensflare.org>
	<200309041707.34519.hinsen@cnrs-orleans.fr>
	<m265k8o8q0.fsf@naima.lensflare.org> <3F59F1D8.5020900@erols.com>
Message-ID: <m2iso4na92.fsf@naima.lensflare.org>

Bernard Frankpitt <frankpit at erols.com> writes:

[snip... finding matrix square root through eigenvalue decomposition.]

> The only problem with this approach is that you have to find all the
> eigenvalues and eigenvectors to get the spectral decomposition.  For
> large, poorly conditioned matrices this can be a source of significant
> errors, and a lot of work. An alternative, and often faster and more
> accurate approach is to use a series expansion for the square
> root. Evaluating the series expansion only requires multiplication.

This technique sounds like it's definitely worth checking out.

Thankyou for the tip.

Andrew.




From jmiller at stsci.edu  Sun Sep  7 04:09:02 2003
From: jmiller at stsci.edu (Todd Miller)
Date: Sun Sep  7 04:09:02 2003
Subject: [Numpy-discussion] Attempting to sub-class NumArray
In-Reply-To: <3F59FC78.3060307@sympatico.ca>
References: <3F58DDE5.5090608@sympatico.ca>
	 <1062799403.5615.25.camel@halloween.stsci.edu>
	 <3F592B3A.5060505@sympatico.ca>
	 <1062848229.3459.21.camel@localhost.localdomain>
	 <3F59FC78.3060307@sympatico.ca>
Message-ID: <1062932892.15057.105.camel@localhost.localdomain>

On Sat, 2003-09-06 at 11:25, Colin J. Williams wrote:
> Todd Miller wrote:
> 
> >I think the problem you are having is caused by the assumption that
> >instance initialization is equivalent to an unordered assignment of all
> >instance attributes.  That's not true because the private attributes are
> >interrelated;  assigning "shape" affects "_shape" and "_strides",
> >_itemsize must be known before _strides can be computed from shape, and
> >so on. 
> >
> It would be helpful if you could let us know what assignment ordering 
> would be appropriate, or, if there were information about the 
> interactions, perhaps we could work these things out for ourselves.

In truth,  I don't think the order is the issue, so let's drop that. 
All I can say is that somehow special.__init__() is creating
inconsistent _shape and _strides.  I also noticed it creating and
assigning bound methods, which you probably don't want to do.

> As we've discussed before, there is little information available on 
> private objects and their roles.  Even to have the source code of some 
> of the C code might be useful.  I wasn't able to spot the relevant C 
> code among the CVS directories.

The source code is all there.

Look in Include/numarray/numarray.h, Src/_ndarraymodule.c, and
Src/_numarraymodule.c, and maybe Src/libnumarraymodule.c.

> However, the assumption would appear to apply to the crude 
> initialization of 'special'.  The problem which is being reported seems 
> to be connected with the function setshape in generic.py.  This is far 
> from the unordered assignment, although I recognize that ther might be 
> some connection.

I'm not sure where and how, but I think the problem is in your dir()
loop.   So, instead, try this:

self.__setstate__(arr.__getstate__())

That ensures that you're only copying the data attributes and appears to
work.

Todd

> Colin W.
> 
> >
> >On Fri, 2003-09-05 at 20:32, Colin J. Williams wrote:
> >  
> >
> >>Todd,
> >>
> >>Thanks again for your prompt response.
> >>Todd Miller wrote:
> >>    
> >>
> >>>Hi Colin,
> >>>
> >>>I haven't looked at this in detail,  but special.__init__() is making me
> >>>queasy.  
> >>>      
> >>>
> >>We don't call __init__ directly, but only when we construct the
> >>instance, a,  of the class special.
> >>    
> >>
> >>>Why doesn't it call NumArray.__init__()?
> >>>      
> >>>
> >>Because we have data to install in the newly created a.  We go off to
> >>the factory function, numarray.array, to create the array, which we
> >>then specialize.
> >>
> >>I don't see how NumArray.__init__ would help.  This cumbersome
> >>procedure seems to work OK in most respects, but fails to display
> >>properly.
> >>
> >>I don't follow the code sufficiently well to be sure, but my hunch is
> >>that the problem lies with setShape.  It fails to update _strides.
> >>
> >>Colin W.
> >>    
> >>
> >>>Todd
> >>>
> >>>On Fri, 2003-09-05 at 15:03, Colin J. Williams wrote:
> >>>  
> >>>      
> >>>
> >>>>When ravel() is used on an instance of NumArray, the _strides attribute 
> >>>>is recalculated correctly.
> >>>>When ravel() is used on an instance of special, a sub-class of NumArray, 
> >>>>the _strides attribute keeps its old value.  This appears to cause the 
> >>>>display problem which is reported.
> >>>>
> >>>>The essentials of the code used are below.
> >>>>
> >>>>To investigate this problem, two print statements were added to the 
> >>>>method setshape in generic.py:
> >>>>
> >>>>            if newnelements == nelements:
> >>>>                self._shape = tuple(shape)
> >>>>                print 'In setShape self._shape:', self._shape
> >>>>                self._strides = self._stridesFromShape()
> >>>>                print 'In setShape self._strides:', self._strides
> >>>>            else:
> >>>>                raise ValueError("New shape is not consistent with the
> >>>>    old shape")
> >>>>
> >>>>It seems that only the first of these print statements is executed with 
> >>>>special, but both are executed with NumArray.  In the second case, the 
> >>>>output is not in the expected sequence.
> >>>>
> >>>>I would appreciate any suggestion as to a workaround.
> >>>>
> >>>>Colin W,
> >>>>
> >>>>Grief when attempting to sub-class NumArray
> >>>>
> >>>>Code used:
> >>>>
> >>>># Try to sub-class
> >>>>class special(N.NumArray):
> >>>>  def __init__(self, data= None, shape= None, eType= None):
> >>>>##                         eType= _nt.Any or AnyType not acceptable %%
> >>>>    arr= N.array(sequence= data, shape= shape, type= eType)
> >>>>    for attr in dir(arr):
> >>>>##      This is a longwinded way of setting up special
> >>>>##      There must be a better way                     %%
> >>>>##      Perhaps we should use generic._view
> >>>>##      It's not documented                 TRY IT LATER
> >>>>##      There is no need to transfer methods!
> >>>>      if attr[0] == '_' and attr[1] != '_':
> >>>>        print attr
> >>>>        exec 'self.' + attr + '= ' 'arr.' + attr
> >>>>        exec 'print self.' + attr
> >>>>    pass
> >>>>
> >>>>a= special(data= [1, 2, 3, 4], shape= (2, 2))
> >>>>a= N.array([1, 2, 3, 4], shape= (2, 2))
> >>>>print 'In ts a._strides:', a._strides
> >>>>b= _gen.ravel(a)
> >>>>print 'In ts b._strides:', b._strides       #  <<< Unchanged with 
> >>>>special, OK with NumArray
> >>>>print 'b:', b
> >>>>
> >>>>
> >>>>
> >>>>USING THE SUBCLASS special
> >>>> >C:\Progra~1\Python23\pythonw -u ts.py
> >>>>In ts a._strides: (8, 4)
> >>>>In setShape self._shape: (4,)
> >>>>In ts b._strides: (8, 4)
> >>>>b: In setShape self._shape: (4,)
> >>>>Traceback (most recent call last):
> >>>>  File "ts.py", line 37, in ?
> >>>>    print 'b:', b
> >>>>  File 
> >>>>"C:\PROGRA~1\Python23\lib\site-packages\numarray\numarraycore.py", line 
> >>>>618, in __str__
> >>>>    MAX_LINE_WIDTH, PRECISION, SUPPRESS_SMALL, ' ', "")
> >>>>  File "C:\PROGRA~1\Python23\lib\site-packages\numarray\arrayprint.py", 
> >>>>line 176, in array2string
> >>>>    separator, prefix)
> >>>>  File "C:\PROGRA~1\Python23\lib\site-packages\numarray\arrayprint.py", 
> >>>>line 138, in _array2string
> >>>>    max_str_len = max(len(str(max_reduce(data))),
> >>>>libnumarray.error: maximum_Int32_reduce: access beyond buffer. offset=27 
> >>>>buffersize=16
> >>>> >Exit code: 1
> >>>>__________________________________________________________
> >>>>
> >>>># USING NumArray
> >>>>
> >>>>In setShape self._shape: (2, 2)
> >>>>In ts a._strides: (8, 4)
> >>>>In setShape self._shape: (4,)
> >>>>In ts b._strides: (4,)
> >>>>b: In setShape self._shape: (4,)
> >>>>[1 2 3 4]
> >>>> >Exit code: 0
> >>>>
> >>>>
> >>>>
> >>>>
> >>>>-------------------------------------------------------
> >>>>This sf.net email is sponsored by:ThinkGeek
> >>>>Welcome to geek heaven.
> >>>>http://thinkgeek.com/sf
> >>>>_______________________________________________
> >>>>Numpy-discussion mailing list
> >>>>Numpy-discussion at lists.sourceforge.net
> >>>>https://lists.sourceforge.net/lists/listinfo/numpy-discussion
> >>>>    
> >>>>        
> >>>>
-- 
Todd Miller <jmiller at stsci.edu>




From cjw at sympatico.ca  Sun Sep  7 16:55:01 2003
From: cjw at sympatico.ca (Colin J. Williams)
Date: Sun Sep  7 16:55:01 2003
Subject: [Numpy-discussion] Attempting to sub-class NumArray
In-Reply-To: <1062932892.15057.105.camel@localhost.localdomain>
References: <3F58DDE5.5090608@sympatico.ca>	 <1062799403.5615.25.camel@halloween.stsci.edu>	 <3F592B3A.5060505@sympatico.ca>	 <1062848229.3459.21.camel@localhost.localdomain>	 <3F59FC78.3060307@sympatico.ca> <1062932892.15057.105.camel@localhost.localdomain>
Message-ID: <3F5BC513.6090400@sympatico.ca>

Todd Miller wrote:

>On Sat, 2003-09-06 at 11:25, Colin J. Williams wrote:
>  
>
>>Todd Miller wrote:
>>
>>    
>>
>>>I think the problem you are having is caused by the assumption that
>>>instance initialization is equivalent to an unordered assignment of all
>>>instance attributes.  That's not true because the private attributes are
>>>interrelated;  assigning "shape" affects "_shape" and "_strides",
>>>_itemsize must be known before _strides can be computed from shape, and
>>>so on. 
>>>
>>>      
>>>
>>It would be helpful if you could let us know what assignment ordering 
>>would be appropriate, or, if there were information about the 
>>interactions, perhaps we could work these things out for ourselves.
>>    
>>
>
>In truth,  I don't think the order is the issue, so let's drop that. 
>All I can say is that somehow special.__init__() is creating
>inconsistent _shape and _strides.  I also noticed it creating and
>assigning bound methods, which you probably don't want to do.
>
Yes, I had spotted that and limited the updating to the variables.  This 
seemed to work, but your suggestion below is the neater way to go.

>>As we've discussed before, there is little information available on 
>>private objects and their roles.  Even to have the source code of some 
>>of the C code might be useful.  I wasn't able to spot the relevant C 
>>code among the CVS directories.
>>    
>>
>
>The source code is all there.
>
>Look in Include/numarray/numarray.h, Src/_ndarraymodule.c, and
>Src/_numarraymodule.c, and maybe Src/libnumarraymodule.c.
>
Thanks, I'll poke around there.

>
>  
>
>>However, the assumption would appear to apply to the crude 
>>initialization of 'special'.  The problem which is being reported seems 
>>to be connected with the function setshape in generic.py.  This is far 
>>from the unordered assignment, although I recognize that ther might be 
>>some connection.
>>    
>>
>
>I'm not sure where and how, but I think the problem is in your dir()
>loop.   So, instead, try this:
>
>self.__setstate__(arr.__getstate__())
>
This seems to be a better approach than mine.

>
>That ensures that you're only copying the data attributes and appears to
>work.
>
Thanks, over that hurdle and on to the next

Colin W.

>
>Todd
>
>  
>
>>Colin W.
>>
>>    
>>
>>>On Fri, 2003-09-05 at 20:32, Colin J. Williams wrote:
>>> 
>>>
>>>      
>>>
>>>>Todd,
>>>>
>>>>Thanks again for your prompt response.
>>>>Todd Miller wrote:
>>>>   
>>>>
>>>>        
>>>>
>>>>>Hi Colin,
>>>>>
>>>>>I haven't looked at this in detail,  but special.__init__() is making me
>>>>>queasy.  
>>>>>     
>>>>>
>>>>>          
>>>>>
>>>>We don't call __init__ directly, but only when we construct the
>>>>instance, a,  of the class special.
>>>>   
>>>>
>>>>        
>>>>
>>>>>Why doesn't it call NumArray.__init__()?
>>>>>     
>>>>>
>>>>>          
>>>>>
>>>>Because we have data to install in the newly created a.  We go off to
>>>>the factory function, numarray.array, to create the array, which we
>>>>then specialize.
>>>>
>>>>I don't see how NumArray.__init__ would help.  This cumbersome
>>>>procedure seems to work OK in most respects, but fails to display
>>>>properly.
>>>>
>>>>I don't follow the code sufficiently well to be sure, but my hunch is
>>>>that the problem lies with setShape.  It fails to update _strides.
>>>>
>>>>Colin W.
>>>>   
>>>>
>>>>        
>>>>
>>>>>Todd
>>>>>
>>>>>On Fri, 2003-09-05 at 15:03, Colin J. Williams wrote:
>>>>> 
>>>>>     
>>>>>
>>>>>          
>>>>>
>>>>>>When ravel() is used on an instance of NumArray, the _strides attribute 
>>>>>>is recalculated correctly.
>>>>>>When ravel() is used on an instance of special, a sub-class of NumArray, 
>>>>>>the _strides attribute keeps its old value.  This appears to cause the 
>>>>>>display problem which is reported.
>>>>>>
>>>>>>The essentials of the code used are below.
>>>>>>
>>>>>>To investigate this problem, two print statements were added to the 
>>>>>>method setshape in generic.py:
>>>>>>
>>>>>>           if newnelements == nelements:
>>>>>>               self._shape = tuple(shape)
>>>>>>               print 'In setShape self._shape:', self._shape
>>>>>>               self._strides = self._stridesFromShape()
>>>>>>               print 'In setShape self._strides:', self._strides
>>>>>>           else:
>>>>>>               raise ValueError("New shape is not consistent with the
>>>>>>   old shape")
>>>>>>
>>>>>>It seems that only the first of these print statements is executed with 
>>>>>>special, but both are executed with NumArray.  In the second case, the 
>>>>>>output is not in the expected sequence.
>>>>>>
>>>>>>I would appreciate any suggestion as to a workaround.
>>>>>>
>>>>>>Colin W,
>>>>>>
>>>>>>Grief when attempting to sub-class NumArray
>>>>>>
>>>>>>Code used:
>>>>>>
>>>>>># Try to sub-class
>>>>>>class special(N.NumArray):
>>>>>> def __init__(self, data= None, shape= None, eType= None):
>>>>>>##                         eType= _nt.Any or AnyType not acceptable %%
>>>>>>   arr= N.array(sequence= data, shape= shape, type= eType)
>>>>>>   for attr in dir(arr):
>>>>>>##      This is a longwinded way of setting up special
>>>>>>##      There must be a better way                     %%
>>>>>>##      Perhaps we should use generic._view
>>>>>>##      It's not documented                 TRY IT LATER
>>>>>>##      There is no need to transfer methods!
>>>>>>     if attr[0] == '_' and attr[1] != '_':
>>>>>>       print attr
>>>>>>       exec 'self.' + attr + '= ' 'arr.' + attr
>>>>>>       exec 'print self.' + attr
>>>>>>   pass
>>>>>>
>>>>>>a= special(data= [1, 2, 3, 4], shape= (2, 2))
>>>>>>a= N.array([1, 2, 3, 4], shape= (2, 2))
>>>>>>print 'In ts a._strides:', a._strides
>>>>>>b= _gen.ravel(a)
>>>>>>print 'In ts b._strides:', b._strides       #  <<< Unchanged with 
>>>>>>special, OK with NumArray
>>>>>>print 'b:', b
>>>>>>
>>>>>>
>>>>>>
>>>>>>USING THE SUBCLASS special
>>>>>>            
>>>>>>
>>>>>>>C:\Progra~1\Python23\pythonw -u ts.py
>>>>>>>              
>>>>>>>
>>>>>>In ts a._strides: (8, 4)
>>>>>>In setShape self._shape: (4,)
>>>>>>In ts b._strides: (8, 4)
>>>>>>b: In setShape self._shape: (4,)
>>>>>>Traceback (most recent call last):
>>>>>> File "ts.py", line 37, in ?
>>>>>>   print 'b:', b
>>>>>> File 
>>>>>>"C:\PROGRA~1\Python23\lib\site-packages\numarray\numarraycore.py", line 
>>>>>>618, in __str__
>>>>>>   MAX_LINE_WIDTH, PRECISION, SUPPRESS_SMALL, ' ', "")
>>>>>> File "C:\PROGRA~1\Python23\lib\site-packages\numarray\arrayprint.py", 
>>>>>>line 176, in array2string
>>>>>>   separator, prefix)
>>>>>> File "C:\PROGRA~1\Python23\lib\site-packages\numarray\arrayprint.py", 
>>>>>>line 138, in _array2string
>>>>>>   max_str_len = max(len(str(max_reduce(data))),
>>>>>>libnumarray.error: maximum_Int32_reduce: access beyond buffer. offset=27 
>>>>>>buffersize=16
>>>>>>            
>>>>>>
>>>>>>>Exit code: 1
>>>>>>>              
>>>>>>>
>>>>>>__________________________________________________________
>>>>>>
>>>>>># USING NumArray
>>>>>>
>>>>>>In setShape self._shape: (2, 2)
>>>>>>In ts a._strides: (8, 4)
>>>>>>In setShape self._shape: (4,)
>>>>>>In ts b._strides: (4,)
>>>>>>b: In setShape self._shape: (4,)
>>>>>>[1 2 3 4]
>>>>>>            
>>>>>>
>>>>>>>Exit code: 0
>>>>>>>              
>>>>>>>
>>>>>>
>>>>>>
>>>>>>-------------------------------------------------------
>>>>>>This sf.net email is sponsored by:ThinkGeek
>>>>>>Welcome to geek heaven.
>>>>>>http://thinkgeek.com/sf
>>>>>>_______________________________________________
>>>>>>Numpy-discussion mailing list
>>>>>>Numpy-discussion at lists.sourceforge.net
>>>>>>https://lists.sourceforge.net/lists/listinfo/numpy-discussion
>>>>>>   
>>>>>>       
>>>>>>
>>>>>>            
>>>>>>





From verveer at embl-heidelberg.de  Mon Sep  8 21:30:11 2003
From: verveer at embl-heidelberg.de (Peter Verveer)
Date: Mon Sep  8 21:30:11 2003
Subject: [Numpy-discussion] NA_ReturnOutput question
Message-ID: <200309081734.12819.verveer@embl-heidelberg.de>

The documentation states that NA_ReturnOutput(numarray, shadow) should be used 
with the value returned by NA_OptionalOutputArray() for the shadow parameter. 
Is it also possible to use a shadow array allocated otherwise, e.g. by 
NA_NewArray()?

Cheers, Peter

-- 
Dr. Peter J. Verveer
Cell Biology and Cell Biophysics Programme
EMBL
Meyerhofstrasse 1
D-69117 Heidelberg
Germany
Tel. : +49 6221 387245
Fax  : +49 6221 387242
Email: Peter.Verveer at embl-heidelberg.de





From jmiller at stsci.edu  Tue Sep  9 06:04:10 2003
From: jmiller at stsci.edu (Todd Miller)
Date: Tue Sep  9 06:04:10 2003
Subject: [Numpy-discussion] NA_ReturnOutput question
In-Reply-To: <200309081734.12819.verveer@embl-heidelberg.de>
References: <200309081734.12819.verveer@embl-heidelberg.de>
Message-ID: <1063112612.1633.64.camel@halloween.stsci.edu>

On Mon, 2003-09-08 at 11:34, Peter Verveer wrote:
> The documentation states that NA_ReturnOutput(numarray, shadow) should be used 
> with the value returned by NA_OptionalOutputArray() for the shadow parameter. 

The purpose of NA_OptionalOutputArray() and NA_ReturnOutput() is to
support an optional output parameter and return value with a couple
specific behaviors:

1. If no output array is specified to the function (it is left defaulted
to None), then the function makes up an output array by cloning it
(shape and type) from some 'master' array.  The output array is returned
as the result of the function.

2. If an output array is specified, then it may still need to be
"shadowed" to correctly match the type, byteswap, and alignment
requirements specified for the function.  When a shadow output is
created,  it is used in all of the real calculations and then copied
back onto the original array and deallocated at function exit.  The
function returns None.

> Is it also possible to use a shadow array allocated otherwise, e.g. by 
> NA_NewArray()?

I don't think so.  It is however possible to return an array allocated
by NA_NewArray() as-is.  

> 
> Cheers, Peter
> 
> -- 
> Dr. Peter J. Verveer
> Cell Biology and Cell Biophysics Programme
> EMBL
> Meyerhofstrasse 1
> D-69117 Heidelberg
> Germany
> Tel. : +49 6221 387245
> Fax  : +49 6221 387242
> Email: Peter.Verveer at embl-heidelberg.de
> 
> 
> 
> 
> -------------------------------------------------------
> This sf.net email is sponsored by:ThinkGeek
> Welcome to geek heaven.
> http://thinkgeek.com/sf
> _______________________________________________
> Numpy-discussion mailing list
> Numpy-discussion at lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/numpy-discussion
-- 
Todd Miller 			jmiller at stsci.edu
STSCI / ESS / SSB




From jmiller at stsci.edu  Tue Sep  9 13:25:59 2003
From: jmiller at stsci.edu (Todd Miller)
Date: Tue Sep  9 13:25:59 2003
Subject: [Numpy-discussion] Boolean array formatting poll
Message-ID: <1063138536.1704.181.camel@halloween.stsci.edu>

Now that Python has a bool type, I have received a patch for numarray's
arrayprint module which alters the formatting of Boolean arrays from:

>>>> a=numarray.arange(6, type=numarray.Bool); a[::2] = 0; a
array([0, 1, 0, 1, 0, 1], type=Bool)

to:

>>>> a=numarray.arange(6, type=numarray.Bool); a[::2] = 0; a
array([False, True, False, True, False, True], type=Bool)

Which format do you prefer?   Please vote for or against changing the
format:  +1 (for)  or -1 (against).

-- 
Todd Miller 			jmiller at stsci.edu
STSCI / ESS / SSB




From Chris.Barker at noaa.gov  Tue Sep  9 16:15:21 2003
From: Chris.Barker at noaa.gov (Chris Barker)
Date: Tue Sep  9 16:15:21 2003
Subject: [Numpy-discussion] Boolean array formatting poll
References: <1063138536.1704.181.camel@halloween.stsci.edu>
Message-ID: <3F5E3D2D.48EA9D54@noaa.gov>

Todd Miller wrote:

> >>>> a=numarray.arange(6, type=numarray.Bool); a[::2] = 0; a
> array([False, True, False, True, False, True], type=Bool)
> 
> Which format do you prefer?   Please vote for or against changing the
> format:  +1 (for)  or -1 (against).

-1

It just makes it harder to read, particularly for a large array

-Chris


-- 
Christopher Barker, Ph.D.
Oceanographer
                                    		
NOAA/OR&R/HAZMAT         (206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115       (206) 526-6317   main reception

Chris.Barker at noaa.gov



From cjw at sympatico.ca  Tue Sep  9 16:31:14 2003
From: cjw at sympatico.ca (Colin J. Williams)
Date: Tue Sep  9 16:31:14 2003
Subject: [Numpy-discussion] Boolean array formatting poll
In-Reply-To: <1063138536.1704.181.camel@halloween.stsci.edu>
References: <1063138536.1704.181.camel@halloween.stsci.edu>
Message-ID: <3F5E4950.9070801@sympatico.ca>

Vote +1
Todd Miller wrote:

>Now that Python has a bool type, I have received a patch for numarray's
>arrayprint module which alters the formatting of Boolean arrays from:
>
>  
>
>>>>>a=numarray.arange(6, type=numarray.Bool); a[::2] = 0; a
>>>>>          
>>>>>
>array([0, 1, 0, 1, 0, 1], type=Bool)
>
>to:
>
>  
>
>>>>>a=numarray.arange(6, type=numarray.Bool); a[::2] = 0; a
>>>>>          
>>>>>
>array([False, True, False, True, False, True], type=Bool)
>
>Which format do you prefer?   Please vote for or against changing the
>format:  +1 (for)  or -1 (against).
>
>  
>





From gvermeul at grenoble.cnrs.fr  Wed Sep 10 00:29:07 2003
From: gvermeul at grenoble.cnrs.fr (Gerard Vermeulen)
Date: Wed Sep 10 00:29:07 2003
Subject: [Numpy-discussion] Boolean array formatting poll
In-Reply-To: <1063138536.1704.181.camel@halloween.stsci.edu>
References: <1063138536.1704.181.camel@halloween.stsci.edu>
Message-ID: <20030910092709.018a0d78.gvermeul@grenoble.cnrs.fr>

-1, because of readability

Gerard

On 09 Sep 2003 16:15:36 -0400
Todd Miller <jmiller at stsci.edu> wrote:

> Now that Python has a bool type, I have received a patch for numarray's
> arrayprint module which alters the formatting of Boolean arrays from:
> 
> >>>> a=numarray.arange(6, type=numarray.Bool); a[::2] = 0; a
> array([0, 1, 0, 1, 0, 1], type=Bool)
> 
> to:
> 
> >>>> a=numarray.arange(6, type=numarray.Bool); a[::2] = 0; a
> array([False, True, False, True, False, True], type=Bool)
> 
> Which format do you prefer?   Please vote for or against changing the
> format:  +1 (for)  or -1 (against).
> 
> -- 
> Todd Miller 			jmiller at stsci.edu
> STSCI / ESS / SSB
> 
> 
> 
> -------------------------------------------------------
> This sf.net email is sponsored by:ThinkGeek
> Welcome to geek heaven.
> http://thinkgeek.com/sf
> _______________________________________________
> Numpy-discussion mailing list
> Numpy-discussion at lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/numpy-discussion
> 



From falted at openlc.org  Wed Sep 10 00:41:03 2003
From: falted at openlc.org (Francesc Alted)
Date: Wed Sep 10 00:41:03 2003
Subject: [Numpy-discussion] Boolean array formatting poll
In-Reply-To: <1063138536.1704.181.camel@halloween.stsci.edu>
References: <1063138536.1704.181.camel@halloween.stsci.edu>
Message-ID: <200309100939.47150.falted@openlc.org>

+ 1, because of consistency

-- 
Francesc Alted




From falted at openlc.org  Wed Sep 10 02:25:29 2003
From: falted at openlc.org (Francesc Alted)
Date: Wed Sep 10 02:25:29 2003
Subject: [Numpy-discussion] Why stripping spaces in strings?
Message-ID: <200309101117.15534.falted@openlc.org>

Hi,

A user of pytables recently asked me why it truncates the trailing spaces on
strings. I have to explain him that the information is there, but the
strings module of the numarray package deliberately strips off the trailing
spaces when printing:

>>> from numarray import strings
>>> a=strings.array("hello world  ")
>>> print a
['hello world']
>>> print "<%s>" % a
<['hello world']>
>>> len(a[0])
11

however:

>>> print "<%s>" % a._data
<hello world  >
>>> a._byteView()
array([[104, 101, 108, 108, 111,  32, 119, 111, 114, 108, 100,  32,  32]], 
type=UInt8)
>>> a._itemsize
13

In the beginning I thought that this was a nice feature, but when people is
using numarray.strings objects as containers (for example for Pickled
objects), that can be rather confusing, and I would prefer to see all the
trailing blanks.

But as I am not completely sure, I just would like to discuss the pro's and
con's of the current printing apprach.

Cheers,

-- 
Francesc Alted




From pearu at cens.ioc.ee  Wed Sep 10 03:19:03 2003
From: pearu at cens.ioc.ee (Pearu Peterson)
Date: Wed Sep 10 03:19:03 2003
Subject: [Numpy-discussion] Boolean array formatting poll
In-Reply-To: <1063138536.1704.181.camel@halloween.stsci.edu>
Message-ID: <Pine.LNX.4.21.0309101313200.8554-100000@cens.kybi>

Considering the number of `-1`s, how about

>>> a=numarray.arange(6, type=numarray.Bool); a[::2] = 0; a
 array([F, T, F, T, F, T], type=Bool)

which is short and readable.

Pearu




From cjw at sympatico.ca  Wed Sep 10 04:06:22 2003
From: cjw at sympatico.ca (Colin J. Williams)
Date: Wed Sep 10 04:06:22 2003
Subject: [Numpy-discussion] Boolean array formatting poll
In-Reply-To: <3F5E3D2D.48EA9D54@noaa.gov>
References: <1063138536.1704.181.camel@halloween.stsci.edu> <3F5E3D2D.48EA9D54@noaa.gov>
Message-ID: <3F5F058A.2010204@sympatico.ca>

Chris Barker wrote:

>Todd Miller wrote:
>
>  
>
>>>>>>a=numarray.arange(6, type=numarray.Bool); a[::2] = 0; a
>>>>>>            
>>>>>>
>>array([False, True, False, True, False, True], type=Bool)
>>
>>Which format do you prefer?   Please vote for or against changing the
>>format:  +1 (for)  or -1 (against).
>>    
>>
>
>-1
>
>It just makes it harder to read, particularly for a large array
>
>-Chris
>
Perhaps the important thing is to facilitate visual comparison between 
one row and another or between one array and another.  Maybe 'False' and 
'True ' (with a trailing space) could be used as representations of the 
Boolean values.

If a compressed representation is desired then 'F' and 'T' might be used.

This could be specified in a site constant which, for the distribution, 
would default to one of ('0, '1'), ('False', 'True'), ('False', 'True ') 
or ('F', 'T').

My feeling is that, since it can easily be done, it is desirable to 
distinguish between Integer and Boolean values.

Colin W.

>
>
>  
>





From barrett at stsci.edu  Wed Sep 10 05:21:03 2003
From: barrett at stsci.edu (Paul Barrett)
Date: Wed Sep 10 05:21:03 2003
Subject: [Numpy-discussion] Boolean array formatting poll
In-Reply-To: <1063138536.1704.181.camel@halloween.stsci.edu>
References: <1063138536.1704.181.camel@halloween.stsci.edu>
Message-ID: <3F5F1691.4080906@stsci.edu>

+1 for consistency and clarity.

Todd Miller wrote:
> Now that Python has a bool type, I have received a patch for numarray's
> arrayprint module which alters the formatting of Boolean arrays from:
> 
> 
>>>>>a=numarray.arange(6, type=numarray.Bool); a[::2] = 0; a
> 
> array([0, 1, 0, 1, 0, 1], type=Bool)
> 
> to:
> 
> 
>>>>>a=numarray.arange(6, type=numarray.Bool); a[::2] = 0; a
> 
> array([False, True, False, True, False, True], type=Bool)
> 
> Which format do you prefer?   Please vote for or against changing the
> format:  +1 (for)  or -1 (against).
> 


-- 
Paul Barrett, PhD      Space Telescope Science Institute
Phone: 410-338-4475    ESS/Science Software Group
FAX:   410-338-4767    Baltimore, MD 21218




From jmiller at stsci.edu  Wed Sep 10 05:51:11 2003
From: jmiller at stsci.edu (Todd Miller)
Date: Wed Sep 10 05:51:11 2003
Subject: [Numpy-discussion] Why stripping spaces in strings?
In-Reply-To: <200309101117.15534.falted@openlc.org>
References: <200309101117.15534.falted@openlc.org>
Message-ID: <1063197624.3869.3.camel@halloween.stsci.edu>

Take a look at RawCharArray instead of CharArray.  CharArray is a
RawCharArray with specific stripping and padding behaviors and some
extra methods.  RawCharArray should be the starting point for this kind
of discussion;  it may do what you want as-is, or you can tell me where
it's falling short of what you need.

Todd

On Wed, 2003-09-10 at 05:17, Francesc Alted wrote:
> Hi,
> 
> A user of pytables recently asked me why it truncates the trailing spaces on
> strings. I have to explain him that the information is there, but the
> strings module of the numarray package deliberately strips off the trailing
> spaces when printing:
> 
> >>> from numarray import strings
> >>> a=strings.array("hello world  ")
> >>> print a
> ['hello world']
> >>> print "<%s>" % a
> <['hello world']>
> >>> len(a[0])
> 11
> 
> however:
> 
> >>> print "<%s>" % a._data
> <hello world  >
> >>> a._byteView()
> array([[104, 101, 108, 108, 111,  32, 119, 111, 114, 108, 100,  32,  32]], 
> type=UInt8)
> >>> a._itemsize
> 13
> 
> In the beginning I thought that this was a nice feature, but when people is
> using numarray.strings objects as containers (for example for Pickled
> objects), that can be rather confusing, and I would prefer to see all the
> trailing blanks.
> 
> But as I am not completely sure, I just would like to discuss the pro's and
> con's of the current printing apprach.
> 
> Cheers,
> 
> -- 
> Francesc Alted
> 
> 
> 
> -------------------------------------------------------
> This sf.net email is sponsored by:ThinkGeek
> Welcome to geek heaven.
> http://thinkgeek.com/sf
> _______________________________________________
> Numpy-discussion mailing list
> Numpy-discussion at lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/numpy-discussion
-- 
Todd Miller 			jmiller at stsci.edu
STSCI / ESS / SSB




From mdehoon at ims.u-tokyo.ac.jp  Wed Sep 10 10:28:12 2003
From: mdehoon at ims.u-tokyo.ac.jp (Michiel Jan Laurens de Hoon)
Date: Wed Sep 10 10:28:12 2003
Subject: [Numpy-discussion] Boolean array formatting poll
In-Reply-To: <Pine.LNX.4.21.0309101313200.8554-100000@cens.kybi>
References: <Pine.LNX.4.21.0309101313200.8554-100000@cens.kybi>
Message-ID: <3F5F3F99.8010105@ims.u-tokyo.ac.jp>

-1, for readability.

1's and 0's can be distinguished more easily visually than T's and F's.

Michiel, U Tokyo.

Pearu Peterson wrote:
> Considering the number of `-1`s, how about
> 
> 
>>>>a=numarray.arange(6, type=numarray.Bool); a[::2] = 0; a
> 
>  array([F, T, F, T, F, T], type=Bool)
> 
> which is short and readable.
> 
> Pearu
> 
> 
> 
> -------------------------------------------------------
> This sf.net email is sponsored by:ThinkGeek
> Welcome to geek heaven.
> http://thinkgeek.com/sf
> _______________________________________________
> Numpy-discussion mailing list
> Numpy-discussion at lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/numpy-discussion
> 
> 

-- 
Michiel de Hoon
Assistant Professor
University of Tokyo, Institute of Medical Science
Human Genome Center
4-6-1 Shirokane-dai, Minato-ku
Tokyo 108-8639
Japan
http://bonsai.ims.u-tokyo.ac.jp/~mdehoon




From Chris.Barker at noaa.gov  Wed Sep 10 13:11:46 2003
From: Chris.Barker at noaa.gov (Chris Barker)
Date: Wed Sep 10 13:11:46 2003
Subject: [Numpy-discussion] Boolean array formatting poll
References: <Pine.LNX.4.21.0309101313200.8554-100000@cens.kybi>
Message-ID: <3F5F4DA7.2D640794@noaa.gov>

Pearu Peterson wrote:

> >>> a=numarray.arange(6, type=numarray.Bool); a[::2] = 0; a
>  array([F, T, F, T, F, T], type=Bool)

That's better. However, I"m still inclined to prefer [1, 0, 0, 1, ....

Someone mentioned making clear the distiction between bool and int
arrays. HOw distict are they? In Python, the bool type is really just a
pretty wrapper around an Int anway. I like it, but if in NumArray, a
bool is really and int in disguise, I'd be just as happy to ahve it be
0, 1, 0, .... If however, a bool really is a totally different beast (or
will be in a future version), then maybe it's worth making the clear
distiction.

-Chris

For what I mean, note in Python 2.3:

>>> a = True
>>> a
True
>>> a * 5
5


Can you do this in NumArray?




-- 
Christopher Barker, Ph.D.
Oceanographer
                                    		
NOAA/OR&R/HAZMAT         (206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115       (206) 526-6317   main reception

Chris.Barker at noaa.gov



From jmiller at stsci.edu  Wed Sep 10 13:37:03 2003
From: jmiller at stsci.edu (Todd Miller)
Date: Wed Sep 10 13:37:03 2003
Subject: [Numpy-discussion] Boolean array formatting poll
In-Reply-To: <3F5F4DA7.2D640794@noaa.gov>
References: <Pine.LNX.4.21.0309101313200.8554-100000@cens.kybi> 
	<3F5F4DA7.2D640794@noaa.gov>
Message-ID: <1063225004.3869.175.camel@halloween.stsci.edu>

<snip>

> 
> For what I mean, note in Python 2.3:
> 
> >>> a = True
> >>> a
> True
> >>> a * 5
> 5
> 
> 
> Can you do this in NumArray?

The numarray analog is:

>>> numarray.ones((10,), type=numarray.Bool)*5
array([5, 5, 5, 5, 5, 5, 5, 5, 5, 5])

So, yes.


-- 
Todd Miller 			jmiller at stsci.edu
STSCI / ESS / SSB




From fperez at colorado.edu  Wed Sep 10 17:19:38 2003
From: fperez at colorado.edu (Fernando Perez)
Date: Wed Sep 10 17:19:38 2003
Subject: [Numpy-discussion] Boolean array formatting poll
In-Reply-To: <Pine.LNX.4.21.0309101313200.8554-100000@cens.kybi>
References: <Pine.LNX.4.21.0309101313200.8554-100000@cens.kybi>
Message-ID: <3F5F52D6.1030106@colorado.edu>

Pearu Peterson wrote:
> Considering the number of `-1`s, how about
> 
> 
>>>>a=numarray.arange(6, type=numarray.Bool); a[::2] = 0; a
> 
>  array([F, T, F, T, F, T], type=Bool)

Somehow I seem to parse faster 0/1 arrays than F/T (the symbols are visually 
more different).

-0.5, but better than the change to True/False.

Cheers,

f




From falted at openlc.org  Thu Sep 11 03:58:58 2003
From: falted at openlc.org (Francesc Alted)
Date: Thu Sep 11 03:58:58 2003
Subject: [Numpy-discussion] Why stripping spaces in strings?
In-Reply-To: <1063197624.3869.3.camel@halloween.stsci.edu>
References: <200309101117.15534.falted@openlc.org> <1063197624.3869.3.camel@halloween.stsci.edu>
Message-ID: <200309111244.08209.falted@openlc.org>

Yups, RawCharrArray do the trick:

>>> strings.array([["hello world  ","pp\0"]], kind=strings.CharArray)
CharArray([['hello world', 'pp']])
>>> strings.array([["hello world  ","pp\0"]], kind=strings.RawCharArray)
RawCharArray([['hello world  ', 'pp\x00          ']])

Thanks for the hint,

A Dimecres 10 Setembre 2003 14:40, Todd Miller va escriure:
> Take a look at RawCharArray instead of CharArray.  CharArray is a
> RawCharArray with specific stripping and padding behaviors and some
> extra methods.  RawCharArray should be the starting point for this kind
> of discussion;  it may do what you want as-is, or you can tell me where
> it's falling short of what you need.
>
> Todd

-- 
Francesc Alted




From Chris.Barker at noaa.gov  Thu Sep 11 10:16:07 2003
From: Chris.Barker at noaa.gov (Chris Barker)
Date: Thu Sep 11 10:16:07 2003
Subject: [Numpy-discussion] Boolean array formatting poll
References: <Pine.LNX.4.21.0309101313200.8554-100000@cens.kybi> 
		<3F5F4DA7.2D640794@noaa.gov> <1063225004.3869.175.camel@halloween.stsci.edu>
Message-ID: <3F60A0F9.B9ACC707@noaa.gov>

Todd Miller wrote:

> >>> numarray.ones((10,), type=numarray.Bool)*5
> array([5, 5, 5, 5, 5, 5, 5, 5, 5, 5])
> 
> So, yes.

In that case, definately -1: if the data really are integers, that might
as well be clear from the display

-Chris

-- 
Christopher Barker, Ph.D.
Oceanographer
                                    		
NOAA/OR&R/HAZMAT         (206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115       (206) 526-6317   main reception

Chris.Barker at noaa.gov



From kern at caltech.edu  Thu Sep 11 17:55:29 2003
From: kern at caltech.edu (Robert Kern)
Date: Thu Sep 11 17:55:29 2003
Subject: [Numpy-discussion] Boolean array formatting poll
In-Reply-To: <1063138536.1704.181.camel@halloween.stsci.edu>
References: <1063138536.1704.181.camel@halloween.stsci.edu>
Message-ID: <20030912005204.GA16405@taliesen.caltech.edu>

On Tue, Sep 09, 2003 at 04:15:36PM -0400, Todd Miller wrote:
> Now that Python has a bool type, I have received a patch for numarray's
> arrayprint module which alters the formatting of Boolean arrays from:
> 
> >>>> a=numarray.arange(6, type=numarray.Bool); a[::2] = 0; a
> array([0, 1, 0, 1, 0, 1], type=Bool)
> 
> to:
> 
> >>>> a=numarray.arange(6, type=numarray.Bool); a[::2] = 0; a
> array([False, True, False, True, False, True], type=Bool)
> 
> Which format do you prefer?   Please vote for or against changing the
> format:  +1 (for)  or -1 (against).

-1

If I'm using those kinds of values in arrays, I'm more likely to be
thinking in terms of 1's and 0's not True's and False's anyways.

E.g.

frequency = float(sum(a)) / len(a)

-- 
Robert Kern
kern at caltech.edu

"In the fields of hell where the grass grows high
 Are the graves of dreams allowed to die."
  -- Richard Harter



From Jack.Jansen at cwi.nl  Mon Sep 15 01:36:05 2003
From: Jack.Jansen at cwi.nl (Jack Jansen)
Date: Mon Sep 15 01:36:05 2003
Subject: [Numpy-discussion] Re: MacPython and Numeric
In-Reply-To: <8F0C35DF-E6EA-11D7-BC87-000393C5B4C6@attglobal.net>
Message-ID: <AE63F153-E757-11D7-9BE1-0030655234CE@cwi.nl>

Folks,
I got this report on Numeric as included with MacPython (through the 
package manager) and it looks
promising. But: I'm building Numeric simply by running setup.py, so in 
order to get the added performance
Numeric's setup.py would need to be taught about veclib.

Is anyone willing to look into this?

On Sunday, September 14, 2003, at 09:35 PM, Alun Carr wrote:

> Jack,
>
> I would like to thank you for all your hard work in producing 
> MacPython 2.3 for OS X. It's superb.
>
> One point to note: I think that you may have compiled the Numeric 
> stuff from scratch. Apple supply hand-tuned BLAS, LAPACK, and FFT code 
> in their vecLib framework. Despite the name, according to Apple's 
> documentation this framework is suitable for both G3 and G4 
> processors, and it will automatically use vectorized code if it 
> detects a G4.
>
> From <http://developer.apple.com/hardware/ve/quickstart.html>:
>
>> If you are a developer who would like to make more use of AltiVec in 
>> your application, you should first explore the functionality that is 
>> available in Apple's vecLib.framework . This framework can be found 
>> in /System/Library/Frameworks on any MacOS X installation. There you 
>> will find many highly optimized routines for a diversity of tasks. 
>> There are single and double precision FFT's. Some of these are up to 
>> five times faster than FFT-W . There are also convolutions, 1D and 2D 
>> matrix operations and much, much more. MacOS X.2 (Jaguar) rounds this 
>> out with a full BLAS and LAPACK implementation in both single and 
>> double precision. These functions also work on G3 and earlier 
>> processors, meaning that you can call them safely regardless of what 
>> machine you are running on.
>
> For future releases of Numeric for MacPython, it might be worth using 
> vecLib, to (i) reduce the effort on your part, (ii) reduce the size of 
> the distribution, and (iii) to use code which has been optimized by 
> Apple for maximum performance.
>
> Anyway, thanks again for your efforts, and please keep up the good 
> work.
>
> Best regards,
>
> Alun
>
> ==
>
> Dr Alun J. Carr                Phone: +353-(0)1-716 1989 (Office)
> Mechanical Engineering Dept           +353-(0)1-716 1752 (BRC)
> University College Dublin      Fax:   +353-(0)1-283 0534
> Belfield                       Email: alun.j.carr at ucd.ie (UCD)
> Dublin 4                              carraj at attglobal.net (Home)
> Ireland
>
--
Jack Jansen, <Jack.Jansen at cwi.nl>, http://www.cwi.nl/~jack
If I can't dance I don't want to be part of your revolution -- Emma 
Goldman




From ariciputi at pito.com  Mon Sep 15 03:24:03 2003
From: ariciputi at pito.com (Andrea Riciputi)
Date: Mon Sep 15 03:24:03 2003
Subject: [Numpy-discussion] Re: MacPython and Numeric
In-Reply-To: <AE63F153-E757-11D7-9BE1-0030655234CE@cwi.nl>
Message-ID: <B3749F40-E766-11D7-B4A6-000393933E4E@pito.com>

Fink project (see <http://fink.sourceforge.net/>) offers Numeric  
already linked against Apple's vecLib. Here is the output about that  
package:

> [ibook:~/Installs/IPython] andrea% fink describe numeric-py22
> Information about 2973 packages read in 5 seconds.
>
>
> numeric-py22-23.0-1: Multidim'l numerical arrays for Python, using  
> Apple vecLib
>   Links against optimized Apple vecLib BLAS/LAPACK, instead of
>   using slower, built-in blas and lapack routines.
>   Should now be just as fast as numeric-atlas.
>  .
>  Web site: http://www.pfdubois.com/numpy
>  .
>  Maintainer: Jeffrey Whitaker <jsw at cdc.noaa.gov>

I think you can ask the maintainer for more information or look at the  
Fink's installation script if you know how debs system works. Here it  
is a link to the scripts:

<http://cvs.sourceforge.net/cgi-bin/viewcvs.cgi/fink/dists/10.2/ 
unstable/main/finkinfo/sci/numeric-py22-23.0-1.info>
<http://cvs.sourceforge.net/cgi-bin/viewcvs.cgi/fink/dists/10.2/ 
unstable/main/finkinfo/sci/numeric-py22-23.0-1.patch>

Hope this help.

Cheers,
	Andrea.

On Monday, Sep 15, 2003, at 10:36 Europe/Rome, Jack Jansen wrote:

> Folks,
> I got this report on Numeric as included with MacPython (through the  
> package manager) and it looks
> promising. But: I'm building Numeric simply by running setup.py, so in  
> order to get the added performance
> Numeric's setup.py would need to be taught about veclib.
>
> Is anyone willing to look into this?
>
---
Andrea Riciputi

"Science is like sex: sometimes something useful comes out,
   but that is not the reason we are doing it" -- (Richard Feynman)




From nadavh at VisionSense.com  Wed Sep 17 03:05:08 2003
From: nadavh at VisionSense.com (Nadav Horesh)
Date: Wed Sep 17 03:05:08 2003
Subject: [Numpy-discussion] numarray latest version
Message-ID: <1063796708.8864.32.camel@Nadav.Envision.co.il>

How comes that numarray 0.6.2 succeeds 0.7? Are there 2 active branches?

  Nadav.




From jmiller at stsci.edu  Wed Sep 17 05:54:09 2003
From: jmiller at stsci.edu (Todd Miller)
Date: Wed Sep 17 05:54:09 2003
Subject: [Numpy-discussion] numarray latest version
In-Reply-To: <1063796708.8864.32.camel@Nadav.Envision.co.il>
References: <1063796708.8864.32.camel@Nadav.Envision.co.il>
Message-ID: <1063803173.21121.15.camel@halloween.stsci.edu>

On Wed, 2003-09-17 at 07:05, Nadav Horesh wrote:
> How comes that numarray 0.6.2 succeeds 0.7? 

numarray-0.6.2 is numarray-0.6.1 which makes the licensing clearer at
runtime by including it in __init__.py for numarray.  That's it.  No new
(real) code, just licensing,  which is basically the modified BSD
license.    It's not really intended for use outside the astronomical
community,  so I didn't announce it.

numarray-0.6.2 is being used by STSCI in a larger system, and it has
been tested in that context.  

numarray-0.7 is newer, better, and with several bug fixes,  but not the
same level of testing.

It is unfortunate that numarray-0.6.2 is listed first on Source Forge, 
but as far as I know I can't control that.

> Are there 2 active branches?

Not for long.

Todd

>   Nadav.
> 
> 
> 
> -------------------------------------------------------
> This sf.net email is sponsored by:ThinkGeek
> Welcome to geek heaven.
> http://thinkgeek.com/sf
> _______________________________________________
> Numpy-discussion mailing list
> Numpy-discussion at lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/numpy-discussion
-- 
Todd Miller 			jmiller at stsci.edu
STSCI / ESS / SSB




From anewgene at hotpop.com  Wed Sep 17 13:03:03 2003
From: anewgene at hotpop.com (CL WU)
Date: Wed Sep 17 13:03:03 2003
Subject: [Numpy-discussion] array sort questions
Message-ID: <3F68BE05.1010104@hotpop.com>

Hi, group,
          I am new to numpy. I have 2 questions for array sort.

1. How to sort an array by its one column or one row?
    I know python build-in sort() can do it for list by passing own cmp 
function. but array function sort() will sort each column or row 
seperately,as I know. I don't want to convert array to list to sort and 
then convert back to array.

2. How to get the rank of a rank-0 array? The first "rank" means the 
order of each element after sorting, instead of the "dimension" meaning 
in numpy.     Just like "rank()" function in splus.

Thank you

Chunlei





From tim.hochberg at ieee.org  Wed Sep 17 13:19:21 2003
From: tim.hochberg at ieee.org (Tim Hochberg)
Date: Wed Sep 17 13:19:21 2003
Subject: [Numpy-discussion] array sort questions
In-Reply-To: <3F68BE05.1010104@hotpop.com>
References: <3F68BE05.1010104@hotpop.com>
Message-ID: <3F68C190.1080808@ieee.org>

CL WU wrote:

> Hi, group,
>          I am new to numpy. I have 2 questions for array sort.
>
> 1. How to sort an array by its one column or one row?
>    I know python build-in sort() can do it for list by passing own cmp 
> function. but array function sort() will sort each column or row 
> seperately,as I know. I don't want to convert array to list to sort 
> and then convert back to array.

I think you want argsort plus take. For example, the following sorts on 
the second column of a:

     a = array([[4,5,6], [1,2,3], [7,8,9]])
     arg = argsort(a[:,1])
     take(a, arg, 0)

> 2. How to get the rank of a rank-0 array? The first "rank" means the 
> order of each element after sorting, instead of the "dimension" 
> meaning in numpy.     Just like "rank()" function in splus.

If  I understand you correctly, you want argsort as mentioned above.

Regards,

-tim


>
> Thank you
>
> Chunlei
>
>
>
>
> -------------------------------------------------------
> This sf.net email is sponsored by:ThinkGeek
> Welcome to geek heaven.
> http://thinkgeek.com/sf
> _______________________________________________
> Numpy-discussion mailing list
> Numpy-discussion at lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/numpy-discussion
>






From anewgene at hotpop.com  Wed Sep 17 13:55:13 2003
From: anewgene at hotpop.com (CL WU)
Date: Wed Sep 17 13:55:13 2003
Subject: [Numpy-discussion] array sort questions
In-Reply-To: <3F68C190.1080808@ieee.org>
References: <3F68BE05.1010104@hotpop.com> <3F68C190.1080808@ieee.org>
Message-ID: <3F68CA44.4080408@hotpop.com>

Thank you, Tim.  argsort() and take() does provide a easy way to sort an 
array based on any col or row. But for the second question, it doesn't 
return the result I want.
As below, softrank or softrank1  are functions I am currently using for 
get the rank of a vector(first is more efficient). It returns the index 
of each value from original array/list in sorted array/list.
I hope there is an efficient function in array level to do the same work.
 >>> from Numeric import *
 >>> a=array([5,2,3])
 >>> argsort(a)
array([1, 2, 0])
 >>> def sortrank(list):
...     n=len(list)
...     li_a=[(i,list[i]) for i in range(n)]
...     li_a.sort(lambda a,b:cmp(a[1],b[1]))
...     li_b=[(i,li_a[i]) for i in range(n)]
...     li_b.sort(lambda a,b:cmp(a[1][0],b[1][0]))
...     return [x[0] for x in li_b]
...    
 >>> sortrank(a)
[2, 0, 1]
 >>> def sortrank2(li):
...     li_sorted=li[:]
...     li_sorted.sort()
...     return [li_sorted.index(x) for x in li]
 >>> sortrank1(list(a))
[2, 0, 1]


Thanks again.

Chunlei

Tim Hochberg wrote:

> CL WU wrote:
>
>> Hi, group,
>>          I am new to numpy. I have 2 questions for array sort.
>>
>> 1. How to sort an array by its one column or one row?
>>    I know python build-in sort() can do it for list by passing own 
>> cmp function. but array function sort() will sort each column or row 
>> seperately,as I know. I don't want to convert array to list to sort 
>> and then convert back to array.
>
>
> I think you want argsort plus take. For example, the following sorts 
> on the second column of a:
>
>     a = array([[4,5,6], [1,2,3], [7,8,9]])
>     arg = argsort(a[:,1])
>     take(a, arg, 0)
>
>> 2. How to get the rank of a rank-0 array? The first "rank" means the 
>> order of each element after sorting, instead of the "dimension" 
>> meaning in numpy.     Just like "rank()" function in splus.
>
>
> If  I understand you correctly, you want argsort as mentioned above.
>
> Regards,
>
> -tim
>
>
>>
>> Thank you
>>
>> Chunlei
>>
>>
>>
>>
>> -------------------------------------------------------
>> This sf.net email is sponsored by:ThinkGeek
>> Welcome to geek heaven.
>> http://thinkgeek.com/sf
>> _______________________________________________
>> Numpy-discussion mailing list
>> Numpy-discussion at lists.sourceforge.net
>> https://lists.sourceforge.net/lists/listinfo/numpy-discussion
>>
>
>
>
>
>
> -------------------------------------------------------
> This sf.net email is sponsored by:ThinkGeek
> Welcome to geek heaven.
> http://thinkgeek.com/sf
> _______________________________________________
> Numpy-discussion mailing list
> Numpy-discussion at lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/numpy-discussion
>





From anewgene at hotpop.com  Wed Sep 17 15:28:06 2003
From: anewgene at hotpop.com (CL WU)
Date: Wed Sep 17 15:28:06 2003
Subject: [Numpy-discussion] array sort questions
In-Reply-To: <3F68CFA5.4080206@ieee.org>
References: <3F68BE05.1010104@hotpop.com> <3F68C190.1080808@ieee.org> <3F68CA44.4080408@hotpop.com> <3F68CFA5.4080206@ieee.org>
Message-ID: <3F68E025.90106@hotpop.com>

Great! It works much more efficiently. Thank you so much.

Best,

Chunlei

Tim Hochberg wrote:

> CL WU wrote:
>
>> Thank you, Tim.  argsort() and take() does provide a easy way to sort 
>> an array based on any col or row. But for the second question, it 
>> doesn't return the result I want.
>> As below, softrank or softrank1  are functions I am currently using 
>> for get the rank of a vector(first is more efficient). It returns the 
>> index of each value from original array/list in sorted array/list.
>
>
> Hmmm. It seems that argsort and sortrank are inverses of a sort, so it 
> should be possible to do what you want efficiently, but I'm not sure how.
>
> <think>
>
> Ah, it appears to be quite simple. I believe:
>
> argsort(argsort(a))
>
> is equivalent to your sortrank and should be much faster.
>
> regards,
>
> -tim
>
>
>> I hope there is an efficient function in array level to do the same 
>> work.
>> >>> from Numeric import *
>> >>> a=array([5,2,3])
>> >>> argsort(a)
>> array([1, 2, 0])
>> >>> def sortrank(list):
>> ...     n=len(list)
>> ...     li_a=[(i,list[i]) for i in range(n)]
>> ...     li_a.sort(lambda a,b:cmp(a[1],b[1]))
>> ...     li_b=[(i,li_a[i]) for i in range(n)]
>> ...     li_b.sort(lambda a,b:cmp(a[1][0],b[1][0]))
>> ...     return [x[0] for x in li_b]
>> ...    >>> sortrank(a)
>> [2, 0, 1]
>> >>> def sortrank2(li):
>> ...     li_sorted=li[:]
>> ...     li_sorted.sort()
>> ...     return [li_sorted.index(x) for x in li]
>> >>> sortrank1(list(a))
>> [2, 0, 1]
>>
>>
>> Thanks again.
>>
>> Chunlei
>>
>> Tim Hochberg wrote:
>>
>>> CL WU wrote:
>>>
>>>> Hi, group,
>>>>          I am new to numpy. I have 2 questions for array sort.
>>>>
>>>> 1. How to sort an array by its one column or one row?
>>>>    I know python build-in sort() can do it for list by passing own 
>>>> cmp function. but array function sort() will sort each column or 
>>>> row seperately,as I know. I don't want to convert array to list to 
>>>> sort and then convert back to array.
>>>
>>>
>>>
>>>
>>> I think you want argsort plus take. For example, the following sorts 
>>> on the second column of a:
>>>
>>>     a = array([[4,5,6], [1,2,3], [7,8,9]])
>>>     arg = argsort(a[:,1])
>>>     take(a, arg, 0)
>>>
>>>> 2. How to get the rank of a rank-0 array? The first "rank" means 
>>>> the order of each element after sorting, instead of the "dimension" 
>>>> meaning in numpy.     Just like "rank()" function in splus.
>>>
>>>
>>>
>>>
>>> If  I understand you correctly, you want argsort as mentioned above.
>>>
>>> Regards,
>>>
>>> -tim
>>>
>>>
>>>>
>>>> Thank you
>>>>
>>>> Chunlei
>>>>
>>>>
>>>>
>>>>
>>>> -------------------------------------------------------
>>>> This sf.net email is sponsored by:ThinkGeek
>>>> Welcome to geek heaven.
>>>> http://thinkgeek.com/sf
>>>> _______________________________________________
>>>> Numpy-discussion mailing list
>>>> Numpy-discussion at lists.sourceforge.net
>>>> https://lists.sourceforge.net/lists/listinfo/numpy-discussion
>>>>
>>>
>>>
>>>
>>>
>>>
>>> -------------------------------------------------------
>>> This sf.net email is sponsored by:ThinkGeek
>>> Welcome to geek heaven.
>>> http://thinkgeek.com/sf
>>> _______________________________________________
>>> Numpy-discussion mailing list
>>> Numpy-discussion at lists.sourceforge.net
>>> https://lists.sourceforge.net/lists/listinfo/numpy-discussion
>>>
>>
>>
>>
>>
>> -------------------------------------------------------
>> This sf.net email is sponsored by:ThinkGeek
>> Welcome to geek heaven.
>> http://thinkgeek.com/sf
>> _______________________________________________
>> Numpy-discussion mailing list
>> Numpy-discussion at lists.sourceforge.net
>> https://lists.sourceforge.net/lists/listinfo/numpy-discussion
>>
>
>
>
>





From tim.hochberg at ieee.org  Thu Sep 18 10:54:18 2003
From: tim.hochberg at ieee.org (Tim Hochberg)
Date: Thu Sep 18 10:54:18 2003
Subject: [Numpy-discussion] Numarray - Numeric incompatibility
In-Reply-To: <3F68E025.90106@hotpop.com>
References: <3F68BE05.1010104@hotpop.com> <3F68C190.1080808@ieee.org> <3F68CA44.4080408@hotpop.com> <3F68CFA5.4080206@ieee.org> <3F68E025.90106@hotpop.com>
Message-ID: <3F69F0FC.50404@ieee.org>

I'm just starting to move some of my code over to numarray and I was 
dismayed to find that basic operation between Numeric and numarray 
arrays fail.

 >>> import Numeric as np
 >>> import numarray as na
 >>> a = na.arange(5)
 >>> p = np.arange(5)
 >>> a + p
['vector', 'vector']
Traceback (most recent call last):
  File "<stdin>", line 1, in ?
  File "C:\Python23\Lib\site-packages\numarray\numarraycore.py", line 
648, in __add__
    def __add__(self, operand): return ufunc.add(self, operand)
  File "C:\Python23\lib\site-packages\numarray\ufunc.py", line 818, in 
_cache_miss2
    key = (_digest(n1), _digest(n2), _digest(out), safethread.get_ident())
KeyError: '_digest force cache miss'

I suspect (hope!) that this is just a bug and not something inherent in 
numarray. I dug around in unfunc.py a bit and it appears that the bug is 
shallow and can be fixed simply by replacing::

         if not (_sequence(n1) or _sequence(n2)):
            key = (_digest(n1), _digest(n2), _digest(out), 
safethread.get_ident())
            self._cache[ key ] = cached

with::

        try:
            key = (_digest(n1), _digest(n2), _digest(out), 
safethread.get_ident())
        except KeyError:
            pass
        else:
            self._cache[ key ] = cached

in _cache_miss2 and _cache_miss1. If this were done, _sequence could 
probably be deleted as well.

I'm not very familiar with the numarray code yet, so it's quite possible 
I'm missing something, but I'm willing to do more digging to fix this if 
this turns out to not be sufficient.

Regards,

-tim





From jmiller at stsci.edu  Thu Sep 18 11:09:39 2003
From: jmiller at stsci.edu (Todd Miller)
Date: Thu Sep 18 11:09:39 2003
Subject: [Numpy-discussion] Numarray - Numeric incompatibility
In-Reply-To: <3F69F0FC.50404@ieee.org>
References: <3F68BE05.1010104@hotpop.com> <3F68C190.1080808@ieee.org>
	<3F68CA44.4080408@hotpop.com> <3F68CFA5.4080206@ieee.org>
	<3F68E025.90106@hotpop.com>  <3F69F0FC.50404@ieee.org>
Message-ID: <1063908412.28142.10.camel@halloween.stsci.edu>

On Thu, 2003-09-18 at 13:53, Tim Hochberg wrote:
> 
> I'm just starting to move some of my code over to numarray and I was 
> dismayed to find that basic operation between Numeric and numarray 
> arrays fail.
> 
>  >>> import Numeric as np
>  >>> import numarray as na
>  >>> a = na.arange(5)
>  >>> p = np.arange(5)
>  >>> a + p
> ['vector', 'vector']
> Traceback (most recent call last):
>   File "<stdin>", line 1, in ?
>   File "C:\Python23\Lib\site-packages\numarray\numarraycore.py", line 
> 648, in __add__
>     def __add__(self, operand): return ufunc.add(self, operand)
>   File "C:\Python23\lib\site-packages\numarray\ufunc.py", line 818, in 
> _cache_miss2
>     key = (_digest(n1), _digest(n2), _digest(out), safethread.get_ident())
> KeyError: '_digest force cache miss'
> 
> I suspect (hope!) that this is just a bug and not something inherent in 
> numarray. 

It's an interoperability issue.  Please let us know if you find others.

> I dug around in unfunc.py a bit and it appears that the bug is 
> shallow and can be fixed simply by replacing::
> 
>          if not (_sequence(n1) or _sequence(n2)):
>             key = (_digest(n1), _digest(n2), _digest(out), 
> safethread.get_ident())
>             self._cache[ key ] = cached
> 
> with::
> 
>         try:
>             key = (_digest(n1), _digest(n2), _digest(out), 
> safethread.get_ident())
>         except KeyError:
>             pass
>         else:
>             self._cache[ key ] = cached
> 
> in _cache_miss2 and _cache_miss1. If this were done, _sequence could 
> probably be deleted as well.
> 
> I'm not very familiar with the numarray code yet, so it's quite possible 
> I'm missing something, but I'm willing to do more digging to fix this if 
> this turns out to not be sufficient.
> 

I ran into the same problem trying to port MA to numarray, and came up
with an identical work around.  

A fix like this will be part of numarray-0.8.

Todd
-- 
Todd Miller 			jmiller at stsci.edu
STSCI / ESS / SSB




From tim.hochberg at ieee.org  Thu Sep 18 11:35:10 2003
From: tim.hochberg at ieee.org (Tim Hochberg)
Date: Thu Sep 18 11:35:10 2003
Subject: [Numpy-discussion] array sort questions
In-Reply-To: <3F68E025.90106@hotpop.com>
References: <3F68BE05.1010104@hotpop.com> <3F68C190.1080808@ieee.org> <3F68CA44.4080408@hotpop.com> <3F68CFA5.4080206@ieee.org> <3F68E025.90106@hotpop.com>
Message-ID: <3F69FA4C.1020504@ieee.org>

Hi Chunlei,

I just realized one other thing that you should probably be aware of. 
You could write a much faster version of sortrank in pure python by 
doing your sorts differently. Python's built in sort is very fast, but 
as soon as you start passing in comparison functions it slows down 
dramatically. The trick is to arange the data you need to sort so that 
you don't need an auxilliary function  (know asDecorate-Sort-Undecorate 
or the Schwartzian transform). Thus, the following is almost certainly a 
lot faster than your original sortrank, although probably still slower 
than the argsort solution.

def sortrank(list):
    index = range(len(list))
    li_a = zip(list, index)
    li_a.sort()
    li_b = [(li_a[i][1], i) for i in index]
    li_b.sort()
    return [x[1] for x in li_b]

Regards,

-tim






CL WU wrote:

>>> I hope there is an efficient function in array level to do the same 
>>> work.
>>> >>> from Numeric import *
>>> >>> a=array([5,2,3])
>>> >>> argsort(a)
>>> array([1, 2, 0])
>>> >>> def sortrank(list):
>>> ...     n=len(list)
>>> ...     li_a=[(i,list[i]) for i in range(n)]
>>> ...     li_a.sort(lambda a,b:cmp(a[1],b[1]))
>>> ...     li_b=[(i,li_a[i]) for i in range(n)]
>>> ...     li_b.sort(lambda a,b:cmp(a[1][0],b[1][0]))
>>> ...     return [x[0] for x in li_b]
>>> ...    >>> sortrank(a)
>>> [2, 0, 1]
>>> >>> def sortrank2(li):
>>> ...     li_sorted=li[:]
>>> ...     li_sorted.sort()
>>> ...     return [li_sorted.index(x) for x in li]
>>> >>> sortrank1(list(a))
>>> [2, 0, 1]
>>>
>>>
>>> Thanks again.
>>>
>>> Chunlei
>>>
>>> Tim Hochberg wrote:
>>>
>>>> CL WU wrote:
>>>>
>>>>> Hi, group,
>>>>>          I am new to numpy. I have 2 questions for array sort.
>>>>>
>>>>> 1. How to sort an array by its one column or one row?
>>>>>    I know python build-in sort() can do it for list by passing own 
>>>>> cmp function. but array function sort() will sort each column or 
>>>>> row seperately,as I know. I don't want to convert array to list to 
>>>>> sort and then convert back to array.
>>>>
>>>>
>>>>
>>>>
>>>>
>>>> I think you want argsort plus take. For example, the following 
>>>> sorts on the second column of a:
>>>>
>>>>     a = array([[4,5,6], [1,2,3], [7,8,9]])
>>>>     arg = argsort(a[:,1])
>>>>     take(a, arg, 0)
>>>>
>>>>> 2. How to get the rank of a rank-0 array? The first "rank" means 
>>>>> the order of each element after sorting, instead of the 
>>>>> "dimension" meaning in numpy.     Just like "rank()" function in 
>>>>> splus.
>>>>
>>>>
>>>>
>>>>
>>>>
>>>> If  I understand you correctly, you want argsort as mentioned above.
>>>>
>>>> Regards,
>>>>
>>>> -tim
>>>>
>>>>
>>>>>
>>>>> Thank you
>>>>>
>>>>> Chunlei
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> -------------------------------------------------------
>>>>> This sf.net email is sponsored by:ThinkGeek
>>>>> Welcome to geek heaven.
>>>>> http://thinkgeek.com/sf
>>>>> _______________________________________________
>>>>> Numpy-discussion mailing list
>>>>> Numpy-discussion at lists.sourceforge.net
>>>>> https://lists.sourceforge.net/lists/listinfo/numpy-discussion
>>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>> -------------------------------------------------------
>>>> This sf.net email is sponsored by:ThinkGeek
>>>> Welcome to geek heaven.
>>>> http://thinkgeek.com/sf
>>>> _______________________________________________
>>>> Numpy-discussion mailing list
>>>> Numpy-discussion at lists.sourceforge.net
>>>> https://lists.sourceforge.net/lists/listinfo/numpy-discussion
>>>>
>>>
>>>
>>>
>>>
>>> -------------------------------------------------------
>>> This sf.net email is sponsored by:ThinkGeek
>>> Welcome to geek heaven.
>>> http://thinkgeek.com/sf
>>> _______________________________________________
>>> Numpy-discussion mailing list
>>> Numpy-discussion at lists.sourceforge.net
>>> https://lists.sourceforge.net/lists/listinfo/numpy-discussion
>>>
>>
>>
>>
>>
>
>
>






From anewgene at hotpop.com  Thu Sep 18 14:04:44 2003
From: anewgene at hotpop.com (CL WU)
Date: Thu Sep 18 14:04:44 2003
Subject: [Numpy-discussion] array sort questions
In-Reply-To: <3F69FA4C.1020504@ieee.org>
References: <3F68BE05.1010104@hotpop.com> <3F68C190.1080808@ieee.org> <3F68CA44.4080408@hotpop.com> <3F68CFA5.4080206@ieee.org> <3F68E025.90106@hotpop.com> <3F69FA4C.1020504@ieee.org>
Message-ID: <3F6A1D38.1050809@hotpop.com>

Thanks again, Tim. It a wonderful example to show how efficient python 
can run if it is well written.

Best,

Chunlei

Tim Hochberg wrote:

> Hi Chunlei,
>
> I just realized one other thing that you should probably be aware of. 
> You could write a much faster version of sortrank in pure python by 
> doing your sorts differently. Python's built in sort is very fast, but 
> as soon as you start passing in comparison functions it slows down 
> dramatically. The trick is to arange the data you need to sort so that 
> you don't need an auxilliary function  (know 
> asDecorate-Sort-Undecorate or the Schwartzian transform). Thus, the 
> following is almost certainly a lot faster than your original 
> sortrank, although probably still slower than the argsort solution.
>
> def sortrank(list):
>    index = range(len(list))
>    li_a = zip(list, index)
>    li_a.sort()
>    li_b = [(li_a[i][1], i) for i in index]
>    li_b.sort()
>    return [x[1] for x in li_b]
>
> Regards,
>
> -tim
>
>
>
>
>
>
> CL WU wrote:
>
>>>> I hope there is an efficient function in array level to do the same 
>>>> work.
>>>> >>> from Numeric import *
>>>> >>> a=array([5,2,3])
>>>> >>> argsort(a)
>>>> array([1, 2, 0])
>>>> >>> def sortrank(list):
>>>> ...     n=len(list)
>>>> ...     li_a=[(i,list[i]) for i in range(n)]
>>>> ...     li_a.sort(lambda a,b:cmp(a[1],b[1]))
>>>> ...     li_b=[(i,li_a[i]) for i in range(n)]
>>>> ...     li_b.sort(lambda a,b:cmp(a[1][0],b[1][0]))
>>>> ...     return [x[0] for x in li_b]
>>>> ...    >>> sortrank(a)
>>>> [2, 0, 1]
>>>> >>> def sortrank2(li):
>>>> ...     li_sorted=li[:]
>>>> ...     li_sorted.sort()
>>>> ...     return [li_sorted.index(x) for x in li]
>>>> >>> sortrank1(list(a))
>>>> [2, 0, 1]
>>>>
>>>>
>>>> Thanks again.
>>>>
>>>> Chunlei
>>>>
>>>> Tim Hochberg wrote:
>>>>
>>>>> CL WU wrote:
>>>>>
>>>>>> Hi, group,
>>>>>>          I am new to numpy. I have 2 questions for array sort.
>>>>>>
>>>>>> 1. How to sort an array by its one column or one row?
>>>>>>    I know python build-in sort() can do it for list by passing 
>>>>>> own cmp function. but array function sort() will sort each column 
>>>>>> or row seperately,as I know. I don't want to convert array to 
>>>>>> list to sort and then convert back to array.
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> I think you want argsort plus take. For example, the following 
>>>>> sorts on the second column of a:
>>>>>
>>>>>     a = array([[4,5,6], [1,2,3], [7,8,9]])
>>>>>     arg = argsort(a[:,1])
>>>>>     take(a, arg, 0)
>>>>>
>>>>>> 2. How to get the rank of a rank-0 array? The first "rank" means 
>>>>>> the order of each element after sorting, instead of the 
>>>>>> "dimension" meaning in numpy.     Just like "rank()" function in 
>>>>>> splus.
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> If  I understand you correctly, you want argsort as mentioned above.
>>>>>
>>>>> Regards,
>>>>>
>>>>> -tim
>>>>>
>>>>>
>>>>>>
>>>>>> Thank you
>>>>>>
>>>>>> Chunlei
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> -------------------------------------------------------
>>>>>> This sf.net email is sponsored by:ThinkGeek
>>>>>> Welcome to geek heaven.
>>>>>> http://thinkgeek.com/sf
>>>>>> _______________________________________________
>>>>>> Numpy-discussion mailing list
>>>>>> Numpy-discussion at lists.sourceforge.net
>>>>>> https://lists.sourceforge.net/lists/listinfo/numpy-discussion
>>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> -------------------------------------------------------
>>>>> This sf.net email is sponsored by:ThinkGeek
>>>>> Welcome to geek heaven.
>>>>> http://thinkgeek.com/sf
>>>>> _______________________________________________
>>>>> Numpy-discussion mailing list
>>>>> Numpy-discussion at lists.sourceforge.net
>>>>> https://lists.sourceforge.net/lists/listinfo/numpy-discussion
>>>>>
>>>>
>>>>
>>>>
>>>>
>>>> -------------------------------------------------------
>>>> This sf.net email is sponsored by:ThinkGeek
>>>> Welcome to geek heaven.
>>>> http://thinkgeek.com/sf
>>>> _______________________________________________
>>>> Numpy-discussion mailing list
>>>> Numpy-discussion at lists.sourceforge.net
>>>> https://lists.sourceforge.net/lists/listinfo/numpy-discussion
>>>>
>>>
>>>
>>>
>>>
>>
>>
>>
>
>
>
>





From tim.hochberg at ieee.org  Thu Sep 18 16:20:02 2003
From: tim.hochberg at ieee.org (Tim Hochberg)
Date: Thu Sep 18 16:20:02 2003
Subject: [Numpy-discussion] numarray unfriendly to user defined types
Message-ID: <3F6A3D4C.3070703@ieee.org>


I've run into another issue in my attempt to transition over to 
numarray: it's less friendly to user defined types than Numeric. I think 
this is mainly accidental friednliness on Numeric's part, but it's handy 
nonetheless. The attached file illustrates the issue. Given some object, 
in this case *zero*, that a numarray array does not now how to handle, 
it ends up raising an exception instead of giving the object a chance to 
try. Thus ``zero + numarray.arange(5)`` works while ``numarray.arange(5) 
+ zero`` fails, since in the first case Zero.__add__ is called first, 
while in the second NumArray.__add__ is called first.

This should probably be fixed, but it's not clear what the best way is. 
My first thought was to always give the other object a chance to use 
__rop__ first. This is simple and easy to explain, but fails miserably 
in the common case of multiplying a list and an array. Just to be 
concrete, __mul__ would be replaced by::

    def __mul__(self, operand):
        try:
            return operand.__rmul__(self)
        except:
            return ufunc.multiply(self, operand)

Next thought is to catch exceptions when they occur in numarray and then 
give the other operand a chance::

    def __mul__(self, operand):
        try:
           return ufunc.multiply(self, operand)
        except:
            return operand.__rmul__(self)

This appears like it would fix my particular problem, but still is not 
ideal. Since numarray is the base of libraries that it will know nothing 
about, it should defer to classes it doesn't know about whenever 
possible. Otherewise it's not possible (or maybe just hard)  to create  
new classes that try to be clever, but still interact with numarray in a 
reasonable way. In my case I'm thinking of proxy objects that don't do 
some computations till they are actually required. So my current 
thinking is that __mul__ and friends would be best implemented as::

    def __mul__(self, operand):
        if not isinstance(operand, knownTypes):
            try:
                  return ufunc.multiply(self, operand)
            except:
                pass
        return operand.__rmul__(self)

Where knownTypes is something like (int, long, float, complex, tuple, list).

Anyway, that's my excessively long two cents on this.

-tim




-------------- next part --------------
An embedded and charset-unspecified text was scrubbed...
Name: unfriendly.py
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20030918/b12aebe9/attachment-0001.ksh>

From ssilrich at yahoo.com  Fri Sep 19 12:37:42 2003
From: ssilrich at yahoo.com (ELECTRICIAN)
Date: Fri Sep 19 12:37:42 2003
Subject: [Numpy-discussion] R E S U M E
Message-ID: <E1A0CuK-0005li-V7@sc8-sf-mx1.sourceforge.net>

Resume from: 	Rich		for   Job    or    Service   " no job too small "

E L E C T R I C I A N


Tel. (408) 482-2102           rysio3 at yahoo.com    


WIRING & INSTALLATION

Hands on electrical installations perform fitting, mounting, laying cables on Commercial, Industrial, residential new & existing buildings.

Electrical Power Supply for Lights, Plugs, Receptacles, Panels, & Fuse boxes, Emergency Generators wiring and testing, Transformers, Power Lines & conduit layout, bending and mounting, parking lighting, lamps, switches, SOLAR PROJECTS, posts and underground installations. 

Shopping Centers; grocery stories, hardware stories, restaurants & residential - housing areas, computer business & fast food units installation & buildings; Solar Panels, Sun Tracking, Flywheel Storage & electric cars systems modify, Natural Energy in Remote areas install. 

LOW VOLTAGE           Office    Home   Yard   Patio   Parking

12 / 24 Volt audio & video equipment, Computer & data network wiring, data backup and UPS; Monitoring Video Control & backup tapes set up and mounting, electro-optical assemblies & subsystems. DC Power Supply, Switch & Motion sensors Alarm. Fire & safety systems install.

Fiber Optics systems, PLC setup, Master Control Center, cable modems & cable TV install. Network, UPS Battery Backup mounting and charging systems; Power supply testing, troubleshooting, and analyzing to a components level. Electric Vehicles Design, Assembly & Installations.

CC TV & Cameras, Security Systems & Sensors for Safety, Fire sprinklers and traffic Monitoring & Door Control. Telephones / Net move & install.

TECHNICIAN

Use lab & shop equipment, mechanical, electrical & electronic tools, measurement & testing equipment,  video cameras & microscopes. Support scientists & electronic engineers. Mechanical & Electro-Mech. Design.

OFFICE, ELECTRICAL AND MECHANICAL PROJECTS

Electrical & Network Sketches, one line diagrams, and "as is" drawings update. Customizing Electronic and Electrical Components & Parts, Layouts electronic and electrical schematic, connectors and mechanical detailing. Quotes, supply, bids and job estimating. 

Customers contact, inspection, project mgmt & supervision of electricians & material handling; Use CAD, Windows and applications;

ELECTRICAL & MAINTENANCE SERVICE   US Citizen; open for travel
.



From Boostering at 2911.net  Sat Sep 20 10:09:01 2003
From: Boostering at 2911.net (Betty)
Date: Sat Sep 20 10:09:01 2003
Subject: [Numpy-discussion] Promoting Your Business
Message-ID: <E1A0lDI-0000YY-BT@sc8-sf-mx1.sourceforge.net>

Email Marketing is one of the most effective and inexpensive ways to
promote your products and services.

We offer a complete Email Marketing solution with quality service and
the lowest prices. The result is that you will enjoy more success.

1. Targeted Email Addresses 

We can supply targeted email addresses according to your requirements,
which are compiled only on your order. We will customize your customer
email addresses.

* We have millions of email addresses in a wide variety of categories.

2. Targeted Mailing  

If you are worried about any complications or consequences with sending 
out targeted emails, or want to avoid the work of sending out targeted 
emails. We will do it for you!  We can send your email message to your 
targeted customers.

* We can Bullet-Proof your Web Site.

We also offer a wide variety of marketing software. For more details,
you can refer to: www.marketingboosting.com

Our services will help you get more business opportunities.


Regards!

Betty Jones
Customer Support
www.marketingboosting.com
Sales at marketingboosting.com

************************************************************************
Receiving this email because you registered to receive special 
offers from one of our partners. If you would prefer not to receive
future email, click: Http://optin.garyshawkey.com/optin/r.php3
************************************************************************



From falted at openlc.org  Mon Sep 22 10:39:16 2003
From: falted at openlc.org (Francesc Alted)
Date: Mon Sep 22 10:39:16 2003
Subject: [Numpy-discussion] [ANN] PyTables 0.7.2 released
Message-ID: <200309221938.13790.falted@openlc.org>

Announcing PyTables 0.7.2
-------------------------

PyTables is a hierarchical database package designed to efficently
manage very large amounts of data. PyTables is built on top of the
HDF5 library and the numarray package. It features an object-oriented
interface that, combined with natural naming and C-code generated from
Pyrex sources, makes it a fast, yet extremely easy to use tool for
interactively save and retrieve large amounts of data. Besides, it
provides flexible indexed access on disk to anywhere in the data you
want to go.
 
On this release you will not find any exciting new features. It is
mainly a maintenance release where the next issues has been addressed:
       - a memory leak was fixed
       - memory consumption is being addressed and lowered
       - much faster opening of files
       - Some important index patterns cases in table reads has been
         optimized

More in detail:

What's new
-----------

        - Fixed a nasty memory leak located on the C libraries (it was
          happening during HDF5 attribute writes). After that, the
          memory consumption when using large object trees has dropped
          quite a bit. However, there remains some small leaks that
          has been tracked down to the underlying numarray
          library. These leaks has been reported, and hopefully they
          should be fixed more sooner than later.

        - Table buffers are built dinamically now, so if Tables are
	  not accessed for reading or writing this memory will not be
	  booked. This will help to reduce the memory consumption.

	- The opening of files with lots of nodes has been accelerated
	  between a factor 2 or 3. For example, a file with 10 groups
	  and 3000 tables that takes 9.3 seconds to open in 0.7.1, now
	  takes only 2.8 seconds.

	- The Table.read() method has been refactored and optimized
	  and some parts of its code has been moved to Pyrex. In
	  particular, in the special case of step=1, up to a factor 5
	  of speedup (reaching 160 MB/s on a Pentium4 @ 2 GHz) when
	  reading table contents can be achieved now.

	- Done some cosmetic changes in the user manual, but, as no
          new features has been added, you won't need to read the
          manual again :-)

What is a table?
----------------

A table is defined as a collection of records whose values are stored
in fixed-length fields. All records have the same structure and all
values in each field have the same data type.  The terms
"fixed-length" and "strict data types" seems to be quite a strange
requirement for an language like Python, that supports dynamic data
types, but they serve a useful function if the goal is to save very
large quantities of data (such as is generated by many scientific
applications, for example) in an efficient manner that reduces demand
on CPU time and I/O resources.

What is HDF5?
-------------

For those people who know nothing about HDF5, it is is a general
purpose library and file format for storing scientific data made at
NCSA. HDF5 can store two primary objects: datasets and groups. A
dataset is essentially a multidimensional array of data elements, and
a group is a structure for organizing objects in an HDF5 file. Using
these two basic constructs, one can create and store almost any kind of
scientific data structure, such as images, arrays of vectors, and
structured and unstructured grids. You can also mix and match them in
HDF5 files according to your needs.

Platforms
---------

I'm using Linux as the main development platform, but PyTables should
be easy to compile/install on other UNIX machines. This package has
also passed all the tests on a UltraSparc platform with Solaris 7 and
Solaris 8. It also compiles and passes all the tests on a SGI
Origin2000 with MIPS R12000 processors and running IRIX 6.5.

Regarding Windows platforms, PyTables has been tested with Windows
2000 and Windows XP, but it should also work with other flavors.

An example?
-----------

For online code examples, have a look at

http://pytables.sourceforge.net/tut/tutorial1-1.html

and 

http://pytables.sourceforge.net/tut/tutorial1-2.html

Web site
--------

Go to the PyTables web site for more details:

http://pytables.sourceforge.net/

Share your experience
---------------------

Let me know of any bugs, suggestions, gripes, kudos, etc. you may
have.

Have fun!

-- Francesc Alted




From jmiller at stsci.edu  Mon Sep 22 14:33:03 2003
From: jmiller at stsci.edu (Todd Miller)
Date: Mon Sep 22 14:33:03 2003
Subject: [Numpy-discussion] numarray unfriendly to user defined types
In-Reply-To: <3F6A3D4C.3070703@ieee.org>
References: <3F6A3D4C.3070703@ieee.org>
Message-ID: <1064266268.9234.351.camel@halloween.stsci.edu>

On Thu, 2003-09-18 at 19:18, Tim Hochberg wrote:
> 
Sorry for the delay;  I was pretty much shut down by hurricane Isabelle
from the end of last week.

I hit and bypassed this issue trying to port MA.

> I've run into another issue in my attempt to transition over to 
> numarray: it's less friendly to user defined types than Numeric. I think 
> this is mainly accidental friednliness on Numeric's part, but it's handy 
> nonetheless. The attached file illustrates the issue. Given some object, 
> in this case *zero*, that a numarray array does not now how to handle, 
> it ends up raising an exception instead of giving the object a chance to 
> try. Thus ``zero + numarray.arange(5)`` works while ``numarray.arange(5) 
> + zero`` fails, since in the first case Zero.__add__ is called first, 
> while in the second NumArray.__add__ is called first.
> 
> This should probably be fixed, but it's not clear what the best way is. 
> My first thought was to always give the other object a chance to use 
> __rop__ first. This is simple and easy to explain, but fails miserably 
> in the common case of multiplying a list and an array. Just to be 
> concrete, __mul__ would be replaced by::
> 
>     def __mul__(self, operand):
>         try:
>             return operand.__rmul__(self)
>         except:
>             return ufunc.multiply(self, operand)
> 
> Next thought is to catch exceptions when they occur in numarray and then 
> give the other operand a chance::
> 
>     def __mul__(self, operand):
>         try:
>            return ufunc.multiply(self, operand)
>         except:
>             return operand.__rmul__(self)
> 
> This appears like it would fix my particular problem, but still is not 
> ideal. Since numarray is the base of libraries that it will know nothing 
> about, it should defer to classes it doesn't know about whenever 
> possible. 

That does sound like the right heuristic and is echoed (somewhat) in the
core language. 

> Otherewise it's not possible (or maybe just hard)  to create  
> new classes that try to be clever, but still interact with numarray in a 
> reasonable way. 

Looking in Python's Objects/abstract.c, the function binop1 shows one
way to squirm around this issue:  subclass from NumArray.   It would be
interesting to know how Numeric does it.  

> In my case I'm thinking of proxy objects that don't do 
> some computations till they are actually required. So my current 
> thinking is that __mul__ and friends would be best implemented as::
> 
>     def __mul__(self, operand):
>         if not isinstance(operand, knownTypes):

If the "not" belongs there, I'm lost.

>             try:
>                   return ufunc.multiply(self, operand)
>             except:
>                 pass
>         return operand.__rmul__(self)
> 
> Where knownTypes is something like (int, long, float, complex, tuple, list).
> 
> Anyway, that's my excessively long two cents on this.
> 
I agree with your heuristic,  but I am leery of putting an unqualified
try/except in front of the numarray ufunc code.  On the other hand,
qualifying it seems too complicated.  

What about adding a tuple of types to be deferred to?

	def __mul__(self, operand):
		if isinstance(operand, _numarray_deferred_types):
			operand.__rmul__(self)
		else:
			self.__mul__(operand)

Then other libraries could register classes with something like:

numarray.defer_to(my_class)

which is slightly painful, but avoids masking the ufunc exceptions and
only needs to be done once for each library base class.  How does that
sound?

Todd

> -tim
> 
> 
> 
> 
> ----
> 

> import numarray as na
> import Numeric as np
> 
> class Zero:
>     def __add__(self, other):
>         return other
>     __radd__ = __add__
> zero = Zero()
>     
> #~ print zero + np.arange(5)
> #~ print np.arange(5) + zero
> #~ print zero + na.arange(5) 
> #~ print na.arange(5) + zero
> 
> a = na.arange(5)
> 
> import copy
> copy.deepcopy(a)
-- 
Todd Miller 			jmiller at stsci.edu
STSCI / ESS / SSB




From tim.hochberg at ieee.org  Mon Sep 22 15:28:11 2003
From: tim.hochberg at ieee.org (Tim Hochberg)
Date: Mon Sep 22 15:28:11 2003
Subject: [Numpy-discussion] numarray unfriendly to user defined types
In-Reply-To: <1064266268.9234.351.camel@halloween.stsci.edu>
References: <3F6A3D4C.3070703@ieee.org> <1064266268.9234.351.camel@halloween.stsci.edu>
Message-ID: <3F6F7744.80908@ieee.org>

Hi Todd,

Glad to have you back. I hope you suffered no permanant damage from the 
hurricane. Or the worm for that matter.

Todd Miller wrote:

>On Thu, 2003-09-18 at 19:18, Tim Hochberg wrote:
>  
>
>Sorry for the delay;  I was pretty much shut down by hurricane Isabelle
>from the end of last week.
>
>I hit and bypassed this issue trying to port MA.
>
>  
>
[SNIP]

>Looking in Python's Objects/abstract.c, the function binop1 shows one
>way to squirm around this issue:  subclass from NumArray.   It would be
>interesting to know how Numeric does it.  
>  
>
I think that since Numeric arrays are C classes, they always defer to 
any Python class although I'm not sure of the details. In any event, I'm 
pretty sure it's an accident.

>>In my case I'm thinking of proxy objects that don't do 
>>some computations till they are actually required. So my current 
>>thinking is that __mul__ and friends would be best implemented as::
>>
>>    def __mul__(self, operand):
>>        if not isinstance(operand, knownTypes):
>>    
>>
>
>If the "not" belongs there, I'm lost.
>  
>
It does belong there. KnownTypes (perhaps _numarray_nondeferred_types) 
would be a better term) are types that numarray knows that it should 
handle itself. Let me reverse the logic -- maybe it'll be clearer.

    def __mul__(self, operand):
        if isinstance(operand, _numarray_nondeferred_types):
           return operand.__rmul__(self)
        else:
	   try:
              return ufunc.multiply(self, operand)
           except:
              return operand.__rmul__(self)

>I agree with your heuristic,  but I am leery of putting an unqualified
>try/except in front of the numarray ufunc code.  On the other hand,
>qualifying it seems too complicated.  
>  
>
I agree on all counts.

>What about adding a tuple of types to be deferred to?
>
>	def __mul__(self, operand):
>		if isinstance(operand, _numarray_deferred_types):
>			operand.__rmul__(self)
>		else:
>			self.__mul__(operand)
>
>Then other libraries could register classes with something like:
>
>numarray.defer_to(my_class)
>
>which is slightly painful, but avoids masking the ufunc exceptions and
>only needs to be done once for each library base class.  How does that
>sound?
>  
>
I agree that using try/except is probably not worth the danger. However, 
I think I'd prefer the inverse of what you propose. That is:

	def __mul__(self, operand):
		if isinstance(operand, _numarray_nondeferred_types):
                        self.__mul__(operand)
		else:
			operand.__rmul__(self)


and of course a dont_defer_to registration function. Hopefully with a 
better name. The choice comes down to whether we defer by default or 
handle by default. I'm marginally on the side of deferring, but don't 
feel too strongly either way. Either one would be a big improvement.

Perhaps someone else out there has some profound thoughts.

-tim




>Todd
>
>  
>
>>-tim
>>
>>
>>
>>
>>----
>>
>>    
>>
>
>  
>
>>import numarray as na
>>import Numeric as np
>>
>>class Zero:
>>    def __add__(self, other):
>>        return other
>>    __radd__ = __add__
>>zero = Zero()
>>    
>>#~ print zero + np.arange(5)
>>#~ print np.arange(5) + zero
>>#~ print zero + na.arange(5) 
>>#~ print na.arange(5) + zero
>>
>>a = na.arange(5)
>>
>>import copy
>>copy.deepcopy(a)
>>    
>>






From fperez at colorado.edu  Mon Sep 22 15:46:13 2003
From: fperez at colorado.edu (Fernando Perez)
Date: Mon Sep 22 15:46:13 2003
Subject: [Numpy-discussion] numarray unfriendly to user defined types
In-Reply-To: <3F6F7744.80908@ieee.org>
References: <3F6A3D4C.3070703@ieee.org> <1064266268.9234.351.camel@halloween.stsci.edu> <3F6F7744.80908@ieee.org>
Message-ID: <3F6F7B83.3040305@colorado.edu>

Tim Hochberg wrote:

> I agree that using try/except is probably not worth the danger. However, 
> I think I'd prefer the inverse of what you propose. That is:
> 
> 	def __mul__(self, operand):
> 		if isinstance(operand, _numarray_nondeferred_types):
>                         self.__mul__(operand)
> 		else:
> 			operand.__rmul__(self)
> 
> 
> and of course a dont_defer_to registration function. Hopefully with a 
> better name. The choice comes down to whether we defer by default or 
> handle by default. I'm marginally on the side of deferring, but don't 
> feel too strongly either way. Either one would be a big improvement.
> 
> Perhaps someone else out there has some profound thoughts.

Well, certainly not profound, but here's my proverbial $0.02.  I tend to 
really dislike separate registration functions, since they reek too much of 
'programming by side-effect'.  Why not put the necessary information into a 
keyword argument to the constructor?  Once a sensible default is chosen, then 
it can be overridden with 'defers=True' (if the default was False) at __init__ 
time, for example.  Since this is a class-wide issue, perhaps it may make more 
sense to handle it via a meta-class mechanism.  But I'm not an expert on 
metaclasses, so I'll shut up there :)

I know that the above is rather vague, partly because I haven't followed the 
whole thread in detail.  But I hope that my intent, as a design idea, is clear 
(and perhaps even useful :)  While the actual implementation details 
(constructor keyword, metaclass, class-wide global like self.__defers, etc.) 
will have to be settled by the experts, I hope that an approach which doesn't 
rely on separate registration functions can be achieved.

Best regards,

Fernando.




From tim.hochberg at ieee.org  Mon Sep 22 16:28:04 2003
From: tim.hochberg at ieee.org (Tim Hochberg)
Date: Mon Sep 22 16:28:04 2003
Subject: [Numpy-discussion] numarray unfriendly to user defined types
In-Reply-To: <3F6F7B83.3040305@colorado.edu>
References: <3F6A3D4C.3070703@ieee.org> <1064266268.9234.351.camel@halloween.stsci.edu> <3F6F7744.80908@ieee.org> <3F6F7B83.3040305@colorado.edu>
Message-ID: <3F6F8546.9020500@ieee.org>

Fernando Perez wrote:

> Tim Hochberg wrote:
>
>> I agree that using try/except is probably not worth the danger. 
>> However, I think I'd prefer the inverse of what you propose. That is:
>>
>>     def __mul__(self, operand):
>>         if isinstance(operand, _numarray_nondeferred_types):
>>                         self.__mul__(operand)
>>         else:
>>             operand.__rmul__(self)
>>
>>
>> and of course a dont_defer_to registration function. Hopefully with a 
>> better name. The choice comes down to whether we defer by default or 
>> handle by default. I'm marginally on the side of deferring, but don't 
>> feel too strongly either way. Either one would be a big improvement.
>>
>> Perhaps someone else out there has some profound thoughts.
>
>
> Well, certainly not profound, but here's my proverbial $0.02.  I tend 
> to really dislike separate registration functions, since they reek too 
> much of 'programming by side-effect'.  Why not put the necessary 
> information into a keyword argument to the constructor?  Once a 
> sensible default is chosen, then it can be overridden with 
> 'defers=True' (if the default was False) at __init__ time, for 
> example.  Since this is a class-wide issue, perhaps it may make more 
> sense to handle it via a meta-class mechanism.  But I'm not an expert 
> on metaclasses, so I'll shut up there :)

I actually have no idea how you plan to make keyword arguments work 
here, perhaps you could explain that in more detail. Metaclasses are 
overkill, but a mixin, marker class could be used. That is, when 
designing a class for use with numarray, one would derive a class from a 
marker class in numarray::

   class MyArrayLikeClass(numarray.DeferToMe):
       ....

Hmmm. That's not too bad. Todd, what do you think about using this logic::

    def __mul__(self, operand):
        if isinstance(operand, DeferToMe):
            operand.__rmul__(self)
        else:
            self.__mul__(operand)


The only case where I see a potential problem is an old-style 
C-extenstion that can't be subclassed. I think that might be a case of 
YAGNI though. Avoiding registration is appealing.

-tim






From fperez at colorado.edu  Mon Sep 22 16:42:02 2003
From: fperez at colorado.edu (Fernando Perez)
Date: Mon Sep 22 16:42:02 2003
Subject: [Numpy-discussion] numarray unfriendly to user defined types
In-Reply-To: <3F6F8546.9020500@ieee.org>
References: <3F6A3D4C.3070703@ieee.org> <1064266268.9234.351.camel@halloween.stsci.edu> <3F6F7744.80908@ieee.org> <3F6F7B83.3040305@colorado.edu> <3F6F8546.9020500@ieee.org>
Message-ID: <3F6F8897.3080902@colorado.edu>

Tim Hochberg wrote:
> I actually have no idea how you plan to make keyword arguments work 
> here, perhaps you could explain that in more detail. Metaclasses are 
> overkill, but a mixin, marker class could be used. That is, when 
> designing a class for use with numarray, one would derive a class from a 
> marker class in numarray::

Well, the joys of being vague :)  As I said, I haven't followed in enough 
detail to be too specific, so perhaps my idea plain doesn't work.  But what I 
had in mind was something along the lines of:

class ArrayLike(numarray):
	def __init__(self,...):
		...
		numarray.__init__(defers=1)

This is ultimately equivalent to a registration process, but since it is done 
in the constructor, it feels less like a separate dangling side-effect.  On 
the other hand, it makes your new constructor more expensive, and it feels 
ugly to be doing at the instance level things which truly belong at the class 
level.  Hence my comment about this perhaps being better done via metaclasses.

As I said, I knew I was being vague.  But hopefully these comments at least 
suggest a way out of a separate registration step.

Regards,

Fernando.




From jmiller at stsci.edu  Tue Sep 23 03:35:01 2003
From: jmiller at stsci.edu (Todd Miller)
Date: Tue Sep 23 03:35:01 2003
Subject: [Numpy-discussion] numarray unfriendly to user defined types
In-Reply-To: <3F6F8546.9020500@ieee.org>
References: <3F6A3D4C.3070703@ieee.org>
	 <1064266268.9234.351.camel@halloween.stsci.edu> <3F6F7744.80908@ieee.org>
	 <3F6F7B83.3040305@colorado.edu>  <3F6F8546.9020500@ieee.org>
Message-ID: <1064313211.3460.160.camel@localhost.localdomain>

On Mon, 2003-09-22 at 19:27, Tim Hochberg wrote:

> I actually have no idea how you plan to make keyword arguments work 
> here, perhaps you could explain that in more detail. Metaclasses are 
> overkill, but a mixin, marker class could be used. That is, when 
> designing a class for use with numarray, one would derive a class from a 
> marker class in numarray::
> 
>    class MyArrayLikeClass(numarray.DeferToMe):
>        ....
> 
> Hmmm. That's not too bad. Todd, what do you think about using this logic::
> 
>     def __mul__(self, operand):
>         if isinstance(operand, DeferToMe):
>             operand.__rmul__(self)
>         else:
>             self.__mul__(operand)
> 
> 

I like the core idea a lot.  My only doubt is whether forcing the use of
inheritance is appropriate / a good thing.  We might also consider
spelling it like:

class MyArrayLikeClass:
	_numarray_defer_to_me = True

class NumArray:
	def __mul__(self, operand):
		if hasattr(operand, "_numarray_defer_to_me"):
			return operand.__rmul__(self)
		else:	
			return ufunc.multiply(self, operand)

> The only case where I see a potential problem is an old-style 
> C-extenstion that can't be subclassed. I think that might be a case of 
> YAGNI though. 

Sounds YAGNI to me.

> Avoiding registration is appealing.

Good to have this seconded.

> -tim
> 

All in all, great ideas!

Todd

-- 
Todd Miller <jmiller at stsci.edu>




From jmiller at stsci.edu  Tue Sep 23 04:01:10 2003
From: jmiller at stsci.edu (Todd Miller)
Date: Tue Sep 23 04:01:10 2003
Subject: [Numpy-discussion] numarray unfriendly to user defined types
In-Reply-To: <3F6F8897.3080902@colorado.edu>
References: <3F6A3D4C.3070703@ieee.org>
	 <1064266268.9234.351.camel@halloween.stsci.edu> <3F6F7744.80908@ieee.org>
	 <3F6F7B83.3040305@colorado.edu> <3F6F8546.9020500@ieee.org>
	 <3F6F8897.3080902@colorado.edu>
Message-ID: <1064313214.14570.162.camel@localhost.localdomain>

On Mon, 2003-09-22 at 19:41, Fernando Perez wrote:
> Tim Hochberg wrote:
> > I actually have no idea how you plan to make keyword arguments work 
> > here, perhaps you could explain that in more detail. Metaclasses are 
> > overkill, but a mixin, marker class could be used. That is, when 
> > designing a class for use with numarray, one would derive a class from a 
> > marker class in numarray::
> 
> Well, the joys of being vague :)  As I said, I haven't followed in enough 
> detail to be too specific, so perhaps my idea plain doesn't work.  But what I 
> had in mind was something along the lines of:
> 
> class ArrayLike(numarray):
> 	def __init__(self,...):
> 		...
> 		numarray.__init__(defers=1)
> 

I think this behavior is already granted by Python at the abstract
object level;  that is, if you subclass, the r-method of the subclass is
given preference over the l-method of the superclass.  That's my
impression anyway.

Thus, I think we already get "deference" for free with subclasses, but
that is too weak in general because often (like MA for instance) people
won't want to subclass from NumArray.

> This is ultimately equivalent to a registration process, but since it is done 
> in the constructor, it feels less like a separate dangling side-effect.  On 
> the other hand, it makes your new constructor more expensive, and it feels 
> ugly to be doing at the instance level things which truly belong at the class 
> level.  Hence my comment about this perhaps being better done via metaclasses.
> 
> As I said, I knew I was being vague.  But hopefully these comments at least 
> suggest a way out of a separate registration step.

They certainly appear to have done so.  Thanks for commenting.


Todd

-- 
Todd Miller <jmiller at stsci.edu>




From jmiller at stsci.edu  Tue Sep 23 08:11:10 2003
From: jmiller at stsci.edu (Todd Miller)
Date: Tue Sep 23 08:11:10 2003
Subject: [Numpy-discussion] numarray boolean array formatting poll results
Message-ID: <1064329792.11003.55.camel@halloween.stsci.edu>

The thread on what to do with boolean array formatting in numarray has
died out,  so I thought I should explain (what I saw) as the results of
the poll:

(1) There was a clear majority of numarray users who actually prefer the
status quo of 0,1 for representing arrays of boolean values.  

(2) There was a sizable minority that would like to see consistency with
Python-2.3 by using False, True for representing arrays of boolean
values.

(3) There was a (good) suggestion of using F, T for representing boolean
values.  However, on reflection and reviewing the poll comments,  F, T
is not really as good as 0, 1 visually, nor is it likely to satisfy
those people interested in consistency with Python, so it is unlikely to
make most numarray users any happier.

(4) Also suggested was supporting configurable boolean formats.  This
approach is reasonable to implement,  but I have a reservation that
adding support for multiple formats may cause as much confusion as it
gives satisfaction.  Is it *really* worth it?  (more later)

Given 1-4,  I think the best thing to do for now is keep 0,1 as the only
format for boolean array values.   

If someone wants False, True bad enough to submit a complete patch for
supporting multiple formats (and defaulting to 0,1),  we will still
accept it.  If anyone is still interested in doing this, the patch
should include modified TeX documentation and pass all the self tests in
setupall.py (all of core numarray and all current add-ons).  The patch
should also run cleanly on both Python-2.2 and Python-2.3.

Thanks everyone for responding,
Todd

-- 
Todd Miller 			jmiller at stsci.edu
STSCI / ESS / SSB




From Chris.Barker at noaa.gov  Tue Sep 23 13:49:23 2003
From: Chris.Barker at noaa.gov (Chris Barker)
Date: Tue Sep 23 13:49:23 2003
Subject: [Numpy-discussion] numarray and NaN ?
References: <010701c362c0$3b54ec50$a47ba8c0@ericlaptop>
Message-ID: <3F70AF4D.5DFC0734@noaa.gov>

Hi all,

There was a lot of discussion a long while back about how Numeric treats
erros in the middle of an array operation. The way it works is if one
operation in an array has a problem (overflow, division by zero, etc),
an exception is returned. You have no idea which element caused the
problem, and teh whole operation grinds to a halt. Personally, I would
MUCH prefer MATLAB's way of handling this: the problematic operation
results in a -Inf, Inf, or NaN result for that element, and the rest of
the array operation can continue. MATLAB also provides utility functions
along the line of isnan, isinf, etc, so you can check for the
problematic areas later.

There was a fair bit of disagreement over whether Numeric should act
similarly, as well as the problem that not all compilers have the
required IEEE support to make this work.

Anyway, how does numarray handle this issue? I'd love to have the MATLAB
behaviour...I'm writing this because I'm about to have to write a bunch
of code to keep making sure I don't pass a 0 into log(), when I'd be
much happier if I could just replace the NaNs with what I want later.

-Chris

-- 
Christopher Barker, Ph.D.
Oceanographer
                                    		
NOAA/OR&R/HAZMAT         (206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115       (206) 526-6317   main reception

Chris.Barker at noaa.gov



From perry at stsci.edu  Tue Sep 23 14:27:10 2003
From: perry at stsci.edu (Perry Greenfield)
Date: Tue Sep 23 14:27:10 2003
Subject: [Numpy-discussion] numarray and NaN ?
In-Reply-To: <3F70AF4D.5DFC0734@noaa.gov>
Message-ID: <CEELJPECNGEGKFNDLHFFMEMMCAAA.perry@stsci.edu>

Chris Barker writes:
>
> There was a lot of discussion a long while back about how Numeric treats
> erros in the middle of an array operation. The way it works is if one
> operation in an array has a problem (overflow, division by zero, etc),
> an exception is returned. You have no idea which element caused the
> problem, and teh whole operation grinds to a halt. Personally, I would
> MUCH prefer MATLAB's way of handling this: the problematic operation
> results in a -Inf, Inf, or NaN result for that element, and the rest of
> the array operation can continue. MATLAB also provides utility functions
> along the line of isnan, isinf, etc, so you can check for the
> problematic areas later.
>
> There was a fair bit of disagreement over whether Numeric should act
> similarly, as well as the problem that not all compilers have the
> required IEEE support to make this work.
>
I don't recall if there was disagreement about this being useful, but
perhaps there was disagreement that it should always do this.

> Anyway, how does numarray handle this issue? I'd love to have the MATLAB
> behaviour...I'm writing this because I'm about to have to write a bunch
> of code to keep making sure I don't pass a 0 into log(), when I'd be
> much happier if I could just replace the NaNs with what I want later.
>
numarray allows one to customize how errors are handled. You can
choose:

1) to silently ignore all errors.
2) print a warning message (default)
3) raise an exception.

One may separately set one of these three behaviors for each of
the 4 ieee categories of floating point errors, namely

1) invalid results (i.e., NaNs)
2) divide by zeros (Infs)
3) overflows
4) underflows

for the first two modes the computation on floats just inserts the
appropriate
ieee value in the array (Infs and NaNs). There are portable constants and
functions to test and set these values (see examples below).
These modes also apply to integer calculations.

I've pasted some examples I presented from some slides shown at the scipy
conference to illustrate how it works and what's available.

Customize how errors are handled

>>> numeric.Error.setMode(all="warn")
>>> numeric.Error.pushMode(dividebyzero="warn",
..  Invalid="raise",overflow="warn",underflow="ignore")
>>> numeric.Error.popMode()
>>> a = 1/arange(10.)
Warning: Encountered divide by zero(s) in divide

Manipulate IEEE-754 special values

>>> import numarray.ieeespecial as ieee
>>> ieee.getinf(a)  # return indices of Infs
(array([0],)

>>> a = arange(10.)/arange(10.)
>>> ieee.setnan(a, 999) # set all NaNs to 999
>>> a
array([999.,1.,1.,1.,1.,1.,1.,1.,1.,1.])
>>> a[ieee.getinf(a)] = 999 # equivalent

Use defined constants

>>> a[5] = ieee.nan
>>> a[6] = ieee.plus_inf
>>> a[7] = ieee.minus_inf
>>> a[8] = ieee.inf
>>> a
array([999.,1.,1.,1.,1.,nan,inf,-inf,inf,1.]

Perry




From Chris.Barker at noaa.gov  Tue Sep 23 15:04:06 2003
From: Chris.Barker at noaa.gov (Chris Barker)
Date: Tue Sep 23 15:04:06 2003
Subject: [Numpy-discussion] numarray and NaN ?
References: <CEELJPECNGEGKFNDLHFFMEMMCAAA.perry@stsci.edu>
Message-ID: <3F70C0D0.F28E157A@noaa.gov>

Perry Greenfield wrote:

> I don't recall if there was disagreement about this being useful,

probably not

> but perhaps there was disagreement that it should always do this.

absolutely, and I think there was a wide range of opinions about how
usefull (and or critical!) this is.

> numarray allows one to customize how errors are handled. You can
> choose:
> 
> 1) to silently ignore all errors.
> 2) print a warning message (default)
> 3) raise an exception.
> 
> One may separately set one of these three behaviors for each of
> the 4 ieee categories of floating point errors, namely
> 
> 1) invalid results (i.e., NaNs)
> 2) divide by zeros (Infs)
> 3) overflows
> 4) underflows

Fabulous!!! You folks are wonderfull! I guess it's time to make the
switch to numarray...I've been waiting for a compelling reason!

-Chris



-- 
Christopher Barker, Ph.D.
Oceanographer
                                    		
NOAA/OR&R/HAZMAT         (206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115       (206) 526-6317   main reception

Chris.Barker at noaa.gov



From nadavh at VisionSense.com  Wed Sep 24 00:15:05 2003
From: nadavh at VisionSense.com (Nadav Horesh)
Date: Wed Sep 24 00:15:05 2003
Subject: [Numpy-discussion] Subclassing numarray arrays.
Message-ID: <1064391268.1645.38.camel@Nadav.Envision.co.il>

Recently I had an idea to subclass numarray's array in order to
implement a different scheme of array indexing via the __call__ method
(ugly indeed, by my experience shows that it can be useful). It occurred
to me that the array class is not in the spirit of Python 2.2+
classes/types unification:
      * The class name is not "trivial": numarraycore.NumArray
      * The class constructor is not built to be used by the commons.
Since arrays are normally generated by functions (zeros, array,
fromfunction etc.) and not by the constructor, subclassing
numarraycore.NumArray is not very useful. If, for instance, "array" (or
"Array") would be the class name, where the constructor would optionally
have the same parameters as the array function (with the option to act
as a copy constructor as the array function does), then it would be easy
to subclass it:

class MyArray(array):
  ...


new_array = MyArray(zeros((10,5), type=Int64))


  Any comments?

   Nadav.




From cjw at sympatico.ca  Wed Sep 24 04:32:01 2003
From: cjw at sympatico.ca (Colin J. Williams)
Date: Wed Sep 24 04:32:01 2003
Subject: [Numpy-discussion] Subclassing numarray arrays.
In-Reply-To: <1064391268.1645.38.camel@Nadav.Envision.co.il>
References: <1064391268.1645.38.camel@Nadav.Envision.co.il>
Message-ID: <3F718076.3050002@sympatico.ca>

Nadav Horesh wrote:

>Recently I had an idea to subclass numarray's array in order to
>implement a different scheme of array indexing via the __call__ method
>(ugly indeed, by my experience shows that it can be useful). It occurred
>to me that the array class is not in the spirit of Python 2.2+
>classes/types unification:
>
I agree.

>      * The class name is not "trivial": numarraycore.NumArray
>      * The class constructor is not built to be used by the commons.
>
I'm not sure what you have in mind here by "trivial" and "commons".

>Since arrays are normally generated by functions (zeros, array,
>fromfunction etc.) and not by the constructor, subclassing
>numarraycore.NumArray is not very useful. If, for instance, "array" (or
>"Array") would be the class name, where the constructor would optionally
>have the same parameters as the array function (with the option to act
>as a copy constructor as the array function does), then it would be easy
>to subclass it:
>
>class MyArray(array):
>  ...
>
>
>new_array = MyArray(zeros((10,5), type=Int64))
>
>  
>
This is desirable, with the present design, one has to go through a two 
step process:

new_array = MyArray(type=Int64, shape= (10, 5))
load_some_data_into_array(new_array, zeros(10, 5))
  

Colin W.

>  Any comments?
>
>   Nadav.
>
>
>
>-------------------------------------------------------
>This sf.net email is sponsored by:ThinkGeek
>Welcome to geek heaven.
>http://thinkgeek.com/sf
>_______________________________________________
>Numpy-discussion mailing list
>Numpy-discussion at lists.sourceforge.net
>https://lists.sourceforge.net/lists/listinfo/numpy-discussion
>
>  
>





From tim.hochberg at ieee.org  Wed Sep 24 15:28:32 2003
From: tim.hochberg at ieee.org (Tim Hochberg)
Date: Wed Sep 24 15:28:32 2003
Subject: [Numpy-discussion] numarray unfriendly to user defined types
In-Reply-To: <1064313211.3460.160.camel@localhost.localdomain>
References: <3F6A3D4C.3070703@ieee.org>	 <1064266268.9234.351.camel@halloween.stsci.edu> <3F6F7744.80908@ieee.org>	 <3F6F7B83.3040305@colorado.edu>  <3F6F8546.9020500@ieee.org> <1064313211.3460.160.camel@localhost.localdomain>
Message-ID: <3F721A4C.7080300@ieee.org>

Hi Todd,

There are three ways to spell "defer to me" on the table (the precise 
details of
each spelling are, of course, still open for debate):

   1.  numarray.defer_to(my_class)

   2. class ArrayLike(numarray.DeferTo):
         # ...

   3. class ArrayLike:
         _numarray_defer_to = True
        # ...

I'd prefer a non-registration solution since those are both 
aesthetically displeasing and leave you open to the situation where a 
class in module A gets registered by module B, but module C expects it 
not to be registered and everything breaks. Not all that likely, I 
admit, but let's avoid the registration version if we can.

The other two solutions are almost equivalent. The one case where 3 has 
an edge over 2 is if I have an object (not a class), I could potentially 
set a _numarray_defer_to on the object before passing it to numarray 
without having to mess with the class
of the object at all. YAGNI, though.

The advantage of 2 in my view is that it *does* force you to subclass. 
With 3, there will be the temptation to poke into some other  module and 
set _numarray_defer_to on some poor unsuspecting class. This has the 
same disadvantage as 1, that it could confuse some other poor 
unsuspecting module. The correct way to do get a deferred class from a 
third party module is to import and subclass. This works with either 2 
or 3::

  import A

  class Klass2(a.Klass, numarray.DeferTo):   #2
     #...

  class Klass3(a.Klass): #3 the good way
      _numarray_defer_to = True
     # ...

   A.Klass._numarray_defer_to = True #3 the evil way.

 Version 2 is cleaner and encourages you to do the right thing, so I'd 
prefer that solution.


regards,

-tim




Todd Miller wrote:

>On Mon, 2003-09-22 at 19:27, Tim Hochberg wrote:
>
>  
>
>>I actually have no idea how you plan to make keyword arguments work 
>>here, perhaps you could explain that in more detail. Metaclasses are 
>>overkill, but a mixin, marker class could be used. That is, when 
>>designing a class for use with numarray, one would derive a class from a 
>>marker class in numarray::
>>
>>   class MyArrayLikeClass(numarray.DeferToMe):
>>       ....
>>
>>Hmmm. That's not too bad. Todd, what do you think about using this logic::
>>
>>    def __mul__(self, operand):
>>        if isinstance(operand, DeferToMe):
>>            operand.__rmul__(self)
>>        else:
>>            self.__mul__(operand)
>>
>>
>>    
>>
>
>I like the core idea a lot.  My only doubt is whether forcing the use of
>inheritance is appropriate / a good thing.  We might also consider
>spelling it like:
>
>class MyArrayLikeClass:
>	_numarray_defer_to_me = True
>
>class NumArray:
>	def __mul__(self, operand):
>		if hasattr(operand, "_numarray_defer_to_me"):
>			return operand.__rmul__(self)
>		else:	
>			return ufunc.multiply(self, operand)
>
>  
>
>>The only case where I see a potential problem is an old-style 
>>C-extenstion that can't be subclassed. I think that might be a case of 
>>YAGNI though. 
>>    
>>
>
>Sounds YAGNI to me.
>
>  
>
>>Avoiding registration is appealing.
>>    
>>
>
>Good to have this seconded.
>
>  
>
>>-tim
>>
>>    
>>
>
>All in all, great ideas!
>
>Todd
>
>  
>






From jmiller at stsci.edu  Wed Sep 24 16:01:05 2003
From: jmiller at stsci.edu (Todd Miller)
Date: Wed Sep 24 16:01:05 2003
Subject: [Numpy-discussion] numarray unfriendly to user defined types
In-Reply-To: <3F721A4C.7080300@ieee.org>
References: <3F6A3D4C.3070703@ieee.org>
	 <1064266268.9234.351.camel@halloween.stsci.edu> <3F6F7744.80908@ieee.org>
	 <3F6F7B83.3040305@colorado.edu>  <3F6F8546.9020500@ieee.org>
	 <1064313211.3460.160.camel@localhost.localdomain>
	 <3F721A4C.7080300@ieee.org>
Message-ID: <1064444385.23878.14.camel@localhost.localdomain>

On Wed, 2003-09-24 at 18:27, Tim Hochberg wrote:
> Hi Todd,
> 
> There are three ways to spell "defer to me" on the table (the precise 
> details of
> each spelling are, of course, still open for debate):
> 
>    1.  numarray.defer_to(my_class)
> 
>    2. class ArrayLike(numarray.DeferTo):
>          # ...
> 
>    3. class ArrayLike:
>          _numarray_defer_to = True
>         # ...
> 
> I'd prefer a non-registration solution since those are both 
> aesthetically displeasing and leave you open to the situation where a 
> class in module A gets registered by module B, but module C expects it 
> not to be registered and everything breaks. Not all that likely, I 
> admit, but let's avoid the registration version if we can.

I was picturing this as module A registering it's own classes only. 
Nevertheless, inverting the problem and distributing the registration as
you suggested is better.

> 
> The other two solutions are almost equivalent. The one case where 3 has 
> an edge over 2 is if I have an object (not a class), I could potentially 
> set a _numarray_defer_to on the object before passing it to numarray 
> without having to mess with the class
> of the object at all. YAGNI, though.

I was more concerned about the potential impact of lots of multiple
inheritance, but that's probably just my own personal blend of FUD.

> The advantage of 2 in my view is that it *does* force you to subclass. 
> With 3, there will be the temptation to poke into some other  module and 
> set _numarray_defer_to on some poor unsuspecting class. This has the 
> same disadvantage as 1, that it could confuse some other poor 
> unsuspecting module. The correct way to do get a deferred class from a 
> third party module is to import and subclass. This works with either 2 
> or 3::
> 
>   import A
> 
>   class Klass2(a.Klass, numarray.DeferTo):   #2
>      #...
> 
>   class Klass3(a.Klass): #3 the good way
>       _numarray_defer_to = True
>      # ...
> 
>    A.Klass._numarray_defer_to = True #3 the evil way.
> 
>  Version 2 is cleaner and encourages you to do the right thing, so I'd 
> prefer that solution.
> 

Good enough for me.  If no one else has any comments, then
numarray.DeferTo is where I'll start implementing.  Tomorrow.

> 
> regards,
> 
> -tim

Thanks again,
Todd







From tim.hochberg at ieee.org  Wed Sep 24 16:57:05 2003
From: tim.hochberg at ieee.org (Tim Hochberg)
Date: Wed Sep 24 16:57:05 2003
Subject: [Numpy-discussion] numarray unfriendly to user defined types
In-Reply-To: <1064444385.23878.14.camel@localhost.localdomain>
References: <3F6A3D4C.3070703@ieee.org>	 <1064266268.9234.351.camel@halloween.stsci.edu> <3F6F7744.80908@ieee.org>	 <3F6F7B83.3040305@colorado.edu>  <3F6F8546.9020500@ieee.org>	 <1064313211.3460.160.camel@localhost.localdomain>	 <3F721A4C.7080300@ieee.org> <1064444385.23878.14.camel@localhost.localdomain>
Message-ID: <3F722F0B.7070609@ieee.org>

Todd Miller wrote:

>On Wed, 2003-09-24 at 18:27, Tim Hochberg wrote:
>  
>
>>Hi Todd,
>>
>>There are three ways to spell "defer to me" on the table (the precise 
>>details of
>>each spelling are, of course, still open for debate):
>>
>>   1.  numarray.defer_to(my_class)
>>
>>   2. class ArrayLike(numarray.DeferTo):
>>         # ...
>>
>>   3. class ArrayLike:
>>         _numarray_defer_to = True
>>        # ...
>>
>>I'd prefer a non-registration solution since those are both 
>>aesthetically displeasing and leave you open to the situation where a 
>>class in module A gets registered by module B, but module C expects it 
>>not to be registered and everything breaks. Not all that likely, I 
>>admit, but let's avoid the registration version if we can.
>>    
>>
>
>I was picturing this as module A registering it's own classes only. 
>Nevertheless, inverting the problem and distributing the registration as
>you suggested is better.
>  
>

The case you describe probably will describe the majority of actual use 
cases, and in fact describes mine. I'm trying to think ahead a bit to 
cases may encounter as start using NumArray more extensively. Let's hope 
this solution still looks good in six months!

>>The other two solutions are almost equivalent. The one case where 3 has 
>>an edge over 2 is if I have an object (not a class), I could potentially 
>>set a _numarray_defer_to on the object before passing it to numarray 
>>without having to mess with the class
>>of the object at all. YAGNI, though.
>>    
>>
>
>I was more concerned about the potential impact of lots of multiple
>inheritance, but that's probably just my own personal blend of FUD.
>
>  
>
>>The advantage of 2 in my view is that it *does* force you to subclass. 
>>With 3, there will be the temptation to poke into some other  module and 
>>set _numarray_defer_to on some poor unsuspecting class. This has the 
>>same disadvantage as 1, that it could confuse some other poor 
>>unsuspecting module. The correct way to do get a deferred class from a 
>>third party module is to import and subclass. This works with either 2 
>>or 3::
>>
>>  import A
>>
>>  class Klass2(a.Klass, numarray.DeferTo):   #2
>>     #...
>>
>>  class Klass3(a.Klass): #3 the good way
>>      _numarray_defer_to = True
>>     # ...
>>
>>   A.Klass._numarray_defer_to = True #3 the evil way.
>>
>> Version 2 is cleaner and encourages you to do the right thing, so I'd 
>>prefer that solution.
>>
>>    
>>
>
>Good enough for me.  If no one else has any comments, then
>numarray.DeferTo is where I'll start implementing.  Tomorrow.
>  
>

One more minor thing. I'm not sure tha DeferTo is ideal as the mix-in 
class name. It was perfect for the registration function name, but I'm 
not sure it's so clear whether the class or numarray is being deferred 
to when you say numarray.DeferTo. DeferToMe is more descriptive, but 
seems sort of slangy. DeferredTo is better than DeferTo, but still not 
as descriptive as DeferToMe. numarray.DefersTo reads perfect as long as 
numarray is included but is a disaster if you read it on your own.  
Below I've put down all the ideas I could come up with

class CustomArray(numarray.DeferTo)
class CustomArray(numarray.DefersTo)
class CustomArray(numarray.DeferredTo)
class CustomArray(numarray.DeferToMe)
class CustomArray(numarray.DeferredToByNumarray)
class CustomArray(DeferTo)
class CustomArray(DefersTo)
class CustomArray(DeferredTo)
class CustomArray(DeferToMe)
class CustomArray(DeferredToByNumarray)

For me it's a toss up between DefferedTo, DeferToMe and 
DeferredToByNumarray. The first is a little lacking in descriptive 
power, the second is slangy and the third is wordy.

-tim

[not that this matters much....]


>>regards,
>>
>>-tim
>>    
>>
>
>Thanks again,
>Todd
>
>
>
>
>
>  
>






From nadavh at VisionSense.com  Thu Sep 25 00:26:05 2003
From: nadavh at VisionSense.com (Nadav Horesh)
Date: Thu Sep 25 00:26:05 2003
Subject: [Numpy-discussion] Subclassing numarray arrays
Message-ID: <1064478298.10525.24.camel@Nadav.Envision.co.il>

On Wed, 2003-09-24 at 14:31, Colin J. Williams wrote:
> Nadav Horesh wrote:
>
> >Recently I had an idea to subclass numarray's array in order to
> >implement a different scheme of array indexing via the __call__
method
> >(ugly indeed, by my experience shows that it can be useful). It
occurred
> >to me that the array class is not in the spirit of Python 2.2+
> >classes/types unification:
> >
> I agree.
>
> >      * The class name is not "trivial": numarraycore.NumArray
> >      * The class constructor is not built to be used by the commons.
> >
> I'm not sure what you have in mind here by "trivial" and "commons".

commons == End users whose most scripts life-time is few days.
trivial == A name one would expect, like "Array" or similar.

>
> >Since arrays are normally generated by functions (zeros, array,
> >fromfunction etc.) and not by the constructor, subclassing
> >numarraycore.NumArray is not very useful. If, for instance, "array"
(or
> >"Array") would be the class name, where the constructor would
optionally
> >have the same parameters as the array function (with the option to
act
> >as a copy constructor as the array function does), then it would be
easy
> >to subclass it:
> >
> >class MyArray(array):
> >  ...
> >
> >
> >new_array = MyArray(zeros((10,5), type=Int64))
> >
> > 
> >
> This is desirable, with the present design, one has to go through a
two
> step process:
>
> new_array = MyArray(type=Int64, shape= (10, 5))
> load_some_data_into_array(new_array, zeros(10, 5))
>  
> Colin W.

As I recall, one of the targets of the numarray project is to be
included as a standard python library --- shouldn't it eventually comply
with the new-classes standard?

>
> >  Any comments?
> >
> >   Nadav.
> >
> >
> >
> >-------------------------------------------------------
> >This sf.net email is sponsored by:ThinkGeek
> >Welcome to geek heaven.
> >http://thinkgeek.com/sf
> >_______________________________________________
> >Numpy-discussion mailing list
> >Numpy-discussion at lists.sourceforge.net
> >https://lists.sourceforge.net/lists/listinfo/numpy-discussion
> >
> > 
> >
>
>




From cjw at sympatico.ca  Thu Sep 25 05:16:08 2003
From: cjw at sympatico.ca (Colin J. Williams)
Date: Thu Sep 25 05:16:08 2003
Subject: [Numpy-discussion] Subclassing numarray arrays
In-Reply-To: <1064478298.10525.24.camel@Nadav.Envision.co.il>
References: <1064478298.10525.24.camel@Nadav.Envision.co.il>
Message-ID: <3F72DC68.6030403@sympatico.ca>

Nadav Horesh wrote:
[snip]

>As I recall, one of the targets of the numarray project is to be
>included as a standard python library --- shouldn't it eventually comply
>with the new-classes standard?
>  
>
I hope that more consideration will be given to this as the numarray 
package evolves.

 From time to time, I am developing a matrix sub-class, it's a bit more 
convoluted than I would like it to be.

It would also be helpful if a current assessment of the speed of 
numarray could be published.  I gather that some enhancements arrived 
with version 0.7.

Colin W.




From jmiller at stsci.edu  Thu Sep 25 07:01:07 2003
From: jmiller at stsci.edu (Todd Miller)
Date: Thu Sep 25 07:01:07 2003
Subject: [Numpy-discussion] numarray unfriendly to user defined types
In-Reply-To: <3F722F0B.7070609@ieee.org>
References: <3F6A3D4C.3070703@ieee.org>	
	<1064266268.9234.351.camel@halloween.stsci.edu> <3F6F7744.80908@ieee.org>	
	<3F6F7B83.3040305@colorado.edu>  <3F6F8546.9020500@ieee.org>	
	<1064313211.3460.160.camel@localhost.localdomain>	
	<3F721A4C.7080300@ieee.org>
	<1064444385.23878.14.camel@localhost.localdomain> 
	<3F722F0B.7070609@ieee.org>
Message-ID: <1064498375.13806.84.camel@halloween.stsci.edu>

On Wed, 2003-09-24 at 19:55, Tim Hochberg wrote:
> One more minor thing. I'm not sure tha DeferTo is ideal as the mix-in 
> class name. It was perfect for the registration function name, but I'm 
> not sure it's so clear whether the class or numarray is being deferred 
> to when you say numarray.DeferTo. DeferToMe is more descriptive, but 
> seems sort of slangy. DeferredTo is better than DeferTo, but still not 
> as descriptive as DeferToMe. numarray.DefersTo reads perfect as long as 
> numarray is included but is a disaster if you read it on your own.  
> Below I've put down all the ideas I could come up with
> 
> class CustomArray(numarray.DeferTo)
> class CustomArray(numarray.DefersTo)
> class CustomArray(numarray.DeferredTo)
> class CustomArray(numarray.DeferToMe)
> class CustomArray(numarray.DeferredToByNumarray)
> class CustomArray(DeferTo)
> class CustomArray(DefersTo)
> class CustomArray(DeferredTo)
> class CustomArray(DeferToMe)
> class CustomArray(DeferredToByNumarray)
> 
> For me it's a toss up between DefferedTo, DeferToMe and 
> DeferredToByNumarray. The first is a little lacking in descriptive 
> power, the second is slangy and the third is wordy.

For something as esoteric as this, I think wordy is probably best. 
Barring a Perrenial edict,  DeferredToByNumArray it is.

I'm psyched.  The MA port should inter-operate with numarray... today.

Thanks for all the lucid, constructive suggestions,

Todd

-- 
Todd Miller 			jmiller at stsci.edu
STSCI / ESS / SSB




From jmiller at stsci.edu  Thu Sep 25 07:26:11 2003
From: jmiller at stsci.edu (Todd Miller)
Date: Thu Sep 25 07:26:11 2003
Subject: [Numpy-discussion] Subclassing numarray arrays
In-Reply-To: <3F72DC68.6030403@sympatico.ca>
References: <1064478298.10525.24.camel@Nadav.Envision.co.il> 
	<3F72DC68.6030403@sympatico.ca>
Message-ID: <1064499880.13807.111.camel@halloween.stsci.edu>

On Thu, 2003-09-25 at 08:15, Colin J. Williams wrote:
> Nadav Horesh wrote:
> [snip]
> 
> >As I recall, one of the targets of the numarray project is to be
> >included as a standard python library --- shouldn't it eventually comply
> >with the new-classes standard?

Can someone please explain what the "new-classes standard" is? 
(Obviously meeting it is not one of our current goals)

> I hope that more consideration will be given to this as the numarray 
> package evolves.

Of course.  We don't need sub-classing internally, however, so it's
difficult to predict when we'll do better.

>  From time to time, I am developing a matrix sub-class, it's a bit more 
> convoluted than I would like it to be.

Posting it here would be good.  Sending things directly to me tends to
dumb things down... to me.
 
> It would also be helpful if a current assessment of the speed of 
> numarray could be published.  

You asked... and...  (sorry if this is less than succinct)

The numarray source distribution has a module called bench.py in the
Examples directory.  Without any analysis,  I've attached the output
from bench.py on my 1.7 GHz Pentium IV running under i386 linux.  Each
row shows the timing numbers for numarray and Numeric for a particular
function call and parameter set;  the important part is the ratio ( < 1
means numarray is faster.  In general, numarray is slower;  this is due
to the fact that numarray supports in-place operation on memory mapped
non-native data formats).

Suspect numbers are... suspect.   Please investigate!  

Caveats:

1) The benchmarks were run on a multi-tasking kernel with no specific
load.  The benchmarks also vary from platform to platform, processor to
processor, compiler to compiler, etc.  These are gcc version 3.2
20020903 (Red Hat Linux 8.0 3.2-7) on a 1.7 GHz P-IV with 384M of RAM.

2) The absolute measured units (microseconds) haven't really been
validated and shouldn't be taken too seriously.  The relative time
comparisons should be valid.

3) a,c,d are 1D arrays.  b is a 2D array.

4) the 10**x column shows the logarithmic array size. 
   2 --> 100 elements, 6 --> 1 million elements

5) The version of bench.py which made these measurements is in CVS, not
the numarray-0.7 tar-ball.

Wish-list:

1) The right way to eliminate the effects of host load on the
benchmarks.  

2) A way to create a regression test which includes performance.

> I gather that some enhancements arrived with version 0.7.

I don't recall any performance enhancements.  numarray-0.7 was mostly
about object array support and bug fixes.  

bench.py results from numarray-0.7 09-25-03:

benchmark                10**x   numarray (usec)    Numeric (usec)  numarray:Numeric
take(a,c)                    2              1477                25              56.8
                             3              1592               108              14.7
                             4              2920               989               2.9
                             5             15568              9814               1.6
                             6            136960             96253               1.4
put(a,c,d)                   2              1673                64              26.1
                             3              1798               195               9.2
                             4              3270              1587               2.1
                             5             16742             15334               1.1
                             6            147768            151932               1.0
transpose(b)                 2               154                22               6.8
                             3               157                24               6.3
                             4               190                39               4.8
                             5               262                74               3.5
                             6               257                67               3.8
putmask(a,c%2,d)             2              1415               139              10.1
                             3              1504               271               5.6
                             4              2439              1735               1.4
                             5             11036             17337               0.6
                             6             93927            157848               0.6
repeat(a,2)                  2              1533               138              11.1
                             3             10165               579              17.5
                             4            103064              4701              21.9
                             5           1028850             58812              17.5
                             6          10010859            493157              20.3
choose(a%2,(a,c))            2              1312                73              18.0
                             3              1389               190               7.3
                             4              2479              1500               1.7
                             5             12516             16219               0.8
                             6            108841            142280               0.8
ravel(b)                     2               164                38               4.3
                             3               169                41               4.1
                             4               196                58               3.4
                             5               279                93               3.0
                             6               275                90               3.0
nonzero(a)                   2              1044               143               7.3
                             3              1133               285               4.0
                             4              2108              1929               1.1
                             5             12002             17974               0.7
                             6            111598            176598               0.6
where(a%2,a,c)               2              1273               111              11.4
                             3              1380               242               5.7
                             4              2587              1752               1.5
                             5             13368             16785               0.8
                             6            117731            164187               0.7
compress(a%2,a)              2              2489               198              12.6
                             3              2661               375               7.1
                             4              4575              2426               1.9
                             5             22933             22621               1.0
                             6            200138            219215               0.9
diagonal(b)                  2              2420             16883               0.1
                             3              2425             16896               0.1
                             4              2591             16932               0.2
                             5              2588             16875               0.2
                             6              2579             16861               0.2
trace(b)                     2              2611             16988               0.2
                             3              2625             16990               0.2
                             4              2750             17004               0.2
                             5              2758             17048               0.2
                             6              2766             16942               0.2
searchsorted(a,a)            2               506                55               9.2
                             3               708               490               1.4
                             4              3054              6507               0.5
                             5             32101             77762               0.4
                             6            378341            940912               0.4
sort(a)                      2               554                85               6.4
                             3               940               365               2.6
                             4              5394              3533               1.5
                             5             55238             36049               1.5
                             6            607845            420334               1.4
argsort(a)                   2              1541                90              16.9
                             3              2061               467               4.4
                             4              8113              5130               1.6
                             5             76924             49497               1.6
                             6            853114            588399               1.4
argmax(b)                    2           1112081              8692             127.9
                             3           1110896              9320             119.2
                             4           1108330              8706             127.3
                             5           1113449              8720             127.7
                             6           1112556              8743             127.3
array(s,typecode='i')        2               229               146               1.6
                             3               274              1103               0.2
                             4               764             10900               0.1
                             5              5450            105774               0.1
                             6             51500           1253819               0.0
dot(b,b)                     2           8825828          31402493               0.3
                             3           8804230          32136602               0.3
                             4           8748239          31123564               0.3
                             5           8811900          32910953               0.3
                             6           8827314          31784649               0.3
clip(a,50,100)               2              2920               177              16.4
                             3              3089               370               8.3
                             4              5138              2471               2.1
                             5             24340             24737               1.0
                             6            211277            245540               0.9
indices((100,100))           2              5253              2696               1.9
                             3              5230              2634               2.0
                             4              5290              2687               2.0
                             5              5383              2732               2.0
                             6              5558              2733               2.0
swapaxes(b,0,-1)             2                77                79               1.0
                             3                77                82               1.0
                             4                94               106               0.9
                             5               154               142               1.1
                             6               146               144               1.0
concatenate((a,a,a))         2              1021                35              29.1
                             3              1082                48              22.1
                             4              1868               431               4.3
                             5              8843              4945               1.8
                             6             75181             47495               1.6
innerproduct(b,b)            2           8719124           8989714               1.0
                             3           8662431           8840564               1.0
                             4           8734306           9033604               1.0
                             5           8694994           8935343               1.0
                             6           8680599           8869750               1.0
outerproduct(b[0],b[0])      2             42320             27598               1.5
                             3             42299             27802               1.5
                             4             42842             27935               1.5
                             5             42346             27822               1.5
                             6             42257             27709               1.5
resize(a,(2*10**i,))         2              1708               160              10.6
                             3              1769               171              10.3
                             4              2648               468               5.6
                             5              9778              3515               2.8
                             6             75513             31778               2.4
a[0]                         2                10                10               1.0
                             3                10                10               1.0
                             4                13                12               1.1
                             5                25                20               1.3
                             6                25                22               1.1
b[0]                         2                46                15               3.1
                             3                48                17               2.8
                             4                56                18               3.0
                             5               106                33               3.1
                             6               100                35               2.9
a[1000:2000]                 2               303                16              18.0
                             3               298                15              18.7
                             4               275                20              13.8
                             5               354                33              10.4
                             6               344                34               9.8
b[400:600,400:600]           2               320                25              12.8
                             3               317                25              12.2
                             4               348                30              11.3
                             5               432                46               9.2
                             6               418                47               8.7
arange(10.**i)               2               411                37              11.1
                             3               448                74               6.0
                             4               923               570               1.6
                             5              5630              5592               1.0
                             6             50735             54222               0.9
identity(i+2)                2              3147               226              13.9
                             3              3158               227              13.9
                             4              3214               258              12.4
                             5              3311               294              11.3
                             6              3316               291              11.4
add(a,c)                     2                76                27               2.7
                             3                90                40               2.2
                             4               313               225               1.4
                             5              2902              2128               1.4
                             6             28106             19972               1.4
add(a,c,d)                   2                40                22               1.9
                             3                46                33               1.4
                             4               144               132               1.1
                             5              1128              1165               1.0
                             6             10316             10827               1.0
a+c                          2                91                31               2.9
                             3               105                45               2.4
                             4               335               231               1.5
                             5              2945              2130               1.4
                             6             28102             19744               1.4
cos(a)                       2                84                53               1.6
                             3               212               235               0.9
                             4              1546              2351               0.7
                             5             15237             23074               0.7
                             6            151604            227898               0.7
add.reduce(a)                2                97                31               3.0
                             3               109                41               2.7
                             4               207               123               1.7
                             5              1062               949               1.1
                             6              8974              8844               1.0
add.reduce(b,0)              2             32527              9218               3.5
                             3             31652              9238               3.4
                             4             31376              9171               3.4
                             5             31412              9271               3.4
                             6             30443              9263               3.3
add.reduce(b,-1)             2              9065              9389               1.0
                             3              9075              9360               1.0
                             4              9102              9271               1.0
                             5              9168              9389               1.0
                             6              9148              9394               1.0
add.accumulate(a)            2               100                28               3.5
                             3               120                41               2.9
                             4               416               254               1.6
                             5              3584              2696               1.3
                             6             34837             25608               1.4
add.accumulate(b,0)          2             63947             25673               2.5
                             3             65875             25691               2.6
                             4             65540             25708               2.5
                             5             65161             25776               2.5
                             6             66539             25646               2.6
add.accumulate(b,1)          2             35048             25756               1.4
                             3             35022             25738               1.4
                             4             35048             25688               1.4
                             5             35092             25761               1.4
                             6             35110             25686               1.4





From cjw at sympatico.ca  Thu Sep 25 09:15:13 2003
From: cjw at sympatico.ca (Colin J. Williams)
Date: Thu Sep 25 09:15:13 2003
Subject: [Numpy-discussion] Subclassing numarray arrays
In-Reply-To: <1064499880.13807.111.camel@halloween.stsci.edu>
References: <1064478298.10525.24.camel@Nadav.Envision.co.il> 	<3F72DC68.6030403@sympatico.ca> <1064499880.13807.111.camel@halloween.stsci.edu>
Message-ID: <3F731467.1070402@sympatico.ca>

Todd,

Thanks for the benchmark data.

I intend to post the matrix stuff.  Performance is not a problem for me 
at this stage.

Currently, I'm plodding along, making things work.  Later, I'll post it 
with some rudimentary documentation.

Regarding the "new-classes standard", I can't find much in either the 
Python docs or Alex Martelli's "Python in a Nutshell".  One of the new 
features is a __new__ constructor.

The Python array module doesn't formally have a class and thus it can't 
be sub-classed.  It uses a factory function to create instance objects.

Martelli, in PIAN p74, writes of factory functions being useful when one 
wishes to create instances of different classes depending on some 
condition or for the reuse of an existing instance.

In the Python Cookbook p173, he proposes the use of a factory function 
to get around the limitations of Python 2.1 and earlier.  On page 179, 
he refers to the use of a factory method, again to work around 
limitations before Python 2.2.

My rough and ready definition would include the ability to create an 
instance and initialize
the data in one step.

Perhaps it would be good to post your exchange with Nadev to the 
newsgroup comp.lang.python.  There are many contributors to the group 
who could give a much more complete answer.

Colin W.


Todd Miller wrote:

>On Thu, 2003-09-25 at 08:15, Colin J. Williams wrote:
>  
>
>>Nadav Horesh wrote:
>>[snip]
>>
>>    
>>
>>>As I recall, one of the targets of the numarray project is to be
>>>included as a standard python library --- shouldn't it eventually comply
>>>with the new-classes standard?
>>>      
>>>
>
>Can someone please explain what the "new-classes standard" is? 
>(Obviously meeting it is not one of our current goals)
>
>  
>
>>I hope that more consideration will be given to this as the numarray 
>>package evolves.
>>    
>>
>
>Of course.  We don't need sub-classing internally, however, so it's
>difficult to predict when we'll do better.
>
>  
>
>> From time to time, I am developing a matrix sub-class, it's a bit more 
>>convoluted than I would like it to be.
>>    
>>
>
>Posting it here would be good.  Sending things directly to me tends to
>dumb things down... to me.
> 
>  
>
>>It would also be helpful if a current assessment of the speed of 
>>numarray could be published.  
>>    
>>
>
>You asked... and...  (sorry if this is less than succinct)
>
>The numarray source distribution has a module called bench.py in the
>Examples directory.  Without any analysis,  I've attached the output
>from bench.py on my 1.7 GHz Pentium IV running under i386 linux.  Each
>row shows the timing numbers for numarray and Numeric for a particular
>function call and parameter set;  the important part is the ratio ( < 1
>means numarray is faster.  In general, numarray is slower;  this is due
>to the fact that numarray supports in-place operation on memory mapped
>non-native data formats).
>
>Suspect numbers are... suspect.   Please investigate!  
>
>Caveats:
>
>1) The benchmarks were run on a multi-tasking kernel with no specific
>load.  The benchmarks also vary from platform to platform, processor to
>processor, compiler to compiler, etc.  These are gcc version 3.2
>20020903 (Red Hat Linux 8.0 3.2-7) on a 1.7 GHz P-IV with 384M of RAM.
>
>2) The absolute measured units (microseconds) haven't really been
>validated and shouldn't be taken too seriously.  The relative time
>comparisons should be valid.
>
>3) a,c,d are 1D arrays.  b is a 2D array.
>
>4) the 10**x column shows the logarithmic array size. 
>   2 --> 100 elements, 6 --> 1 million elements
>
>5) The version of bench.py which made these measurements is in CVS, not
>the numarray-0.7 tar-ball.
>
>Wish-list:
>
>1) The right way to eliminate the effects of host load on the
>benchmarks.  
>
>2) A way to create a regression test which includes performance.
>
>  
>
>>I gather that some enhancements arrived with version 0.7.
>>    
>>
>
>I don't recall any performance enhancements.  numarray-0.7 was mostly
>about object array support and bug fixes.  
>
>bench.py results from numarray-0.7 09-25-03:
>
>benchmark                10**x   numarray (usec)    Numeric (usec)  numarray:Numeric
>take(a,c)                    2              1477                25              56.8
>                             3              1592               108              14.7
>                             4              2920               989               2.9
>                             5             15568              9814               1.6
>                             6            136960             96253               1.4
>put(a,c,d)                   2              1673                64              26.1
>                             3              1798               195               9.2
>                             4              3270              1587               2.1
>                             5             16742             15334               1.1
>                             6            147768            151932               1.0
>transpose(b)                 2               154                22               6.8
>                             3               157                24               6.3
>                             4               190                39               4.8
>                             5               262                74               3.5
>                             6               257                67               3.8
>putmask(a,c%2,d)             2              1415               139              10.1
>                             3              1504               271               5.6
>                             4              2439              1735               1.4
>                             5             11036             17337               0.6
>                             6             93927            157848               0.6
>repeat(a,2)                  2              1533               138              11.1
>                             3             10165               579              17.5
>                             4            103064              4701              21.9
>                             5           1028850             58812              17.5
>                             6          10010859            493157              20.3
>choose(a%2,(a,c))            2              1312                73              18.0
>                             3              1389               190               7.3
>                             4              2479              1500               1.7
>                             5             12516             16219               0.8
>                             6            108841            142280               0.8
>ravel(b)                     2               164                38               4.3
>                             3               169                41               4.1
>                             4               196                58               3.4
>                             5               279                93               3.0
>                             6               275                90               3.0
>nonzero(a)                   2              1044               143               7.3
>                             3              1133               285               4.0
>                             4              2108              1929               1.1
>                             5             12002             17974               0.7
>                             6            111598            176598               0.6
>where(a%2,a,c)               2              1273               111              11.4
>                             3              1380               242               5.7
>                             4              2587              1752               1.5
>                             5             13368             16785               0.8
>                             6            117731            164187               0.7
>compress(a%2,a)              2              2489               198              12.6
>                             3              2661               375               7.1
>                             4              4575              2426               1.9
>                             5             22933             22621               1.0
>                             6            200138            219215               0.9
>diagonal(b)                  2              2420             16883               0.1
>                             3              2425             16896               0.1
>                             4              2591             16932               0.2
>                             5              2588             16875               0.2
>                             6              2579             16861               0.2
>trace(b)                     2              2611             16988               0.2
>                             3              2625             16990               0.2
>                             4              2750             17004               0.2
>                             5              2758             17048               0.2
>                             6              2766             16942               0.2
>searchsorted(a,a)            2               506                55               9.2
>                             3               708               490               1.4
>                             4              3054              6507               0.5
>                             5             32101             77762               0.4
>                             6            378341            940912               0.4
>sort(a)                      2               554                85               6.4
>                             3               940               365               2.6
>                             4              5394              3533               1.5
>                             5             55238             36049               1.5
>                             6            607845            420334               1.4
>argsort(a)                   2              1541                90              16.9
>                             3              2061               467               4.4
>                             4              8113              5130               1.6
>                             5             76924             49497               1.6
>                             6            853114            588399               1.4
>argmax(b)                    2           1112081              8692             127.9
>                             3           1110896              9320             119.2
>                             4           1108330              8706             127.3
>                             5           1113449              8720             127.7
>                             6           1112556              8743             127.3
>array(s,typecode='i')        2               229               146               1.6
>                             3               274              1103               0.2
>                             4               764             10900               0.1
>                             5              5450            105774               0.1
>                             6             51500           1253819               0.0
>dot(b,b)                     2           8825828          31402493               0.3
>                             3           8804230          32136602               0.3
>                             4           8748239          31123564               0.3
>                             5           8811900          32910953               0.3
>                             6           8827314          31784649               0.3
>clip(a,50,100)               2              2920               177              16.4
>                             3              3089               370               8.3
>                             4              5138              2471               2.1
>                             5             24340             24737               1.0
>                             6            211277            245540               0.9
>indices((100,100))           2              5253              2696               1.9
>                             3              5230              2634               2.0
>                             4              5290              2687               2.0
>                             5              5383              2732               2.0
>                             6              5558              2733               2.0
>swapaxes(b,0,-1)             2                77                79               1.0
>                             3                77                82               1.0
>                             4                94               106               0.9
>                             5               154               142               1.1
>                             6               146               144               1.0
>concatenate((a,a,a))         2              1021                35              29.1
>                             3              1082                48              22.1
>                             4              1868               431               4.3
>                             5              8843              4945               1.8
>                             6             75181             47495               1.6
>innerproduct(b,b)            2           8719124           8989714               1.0
>                             3           8662431           8840564               1.0
>                             4           8734306           9033604               1.0
>                             5           8694994           8935343               1.0
>                             6           8680599           8869750               1.0
>outerproduct(b[0],b[0])      2             42320             27598               1.5
>                             3             42299             27802               1.5
>                             4             42842             27935               1.5
>                             5             42346             27822               1.5
>                             6             42257             27709               1.5
>resize(a,(2*10**i,))         2              1708               160              10.6
>                             3              1769               171              10.3
>                             4              2648               468               5.6
>                             5              9778              3515               2.8
>                             6             75513             31778               2.4
>a[0]                         2                10                10               1.0
>                             3                10                10               1.0
>                             4                13                12               1.1
>                             5                25                20               1.3
>                             6                25                22               1.1
>b[0]                         2                46                15               3.1
>                             3                48                17               2.8
>                             4                56                18               3.0
>                             5               106                33               3.1
>                             6               100                35               2.9
>a[1000:2000]                 2               303                16              18.0
>                             3               298                15              18.7
>                             4               275                20              13.8
>                             5               354                33              10.4
>                             6               344                34               9.8
>b[400:600,400:600]           2               320                25              12.8
>                             3               317                25              12.2
>                             4               348                30              11.3
>                             5               432                46               9.2
>                             6               418                47               8.7
>arange(10.**i)               2               411                37              11.1
>                             3               448                74               6.0
>                             4               923               570               1.6
>                             5              5630              5592               1.0
>                             6             50735             54222               0.9
>identity(i+2)                2              3147               226              13.9
>                             3              3158               227              13.9
>                             4              3214               258              12.4
>                             5              3311               294              11.3
>                             6              3316               291              11.4
>add(a,c)                     2                76                27               2.7
>                             3                90                40               2.2
>                             4               313               225               1.4
>                             5              2902              2128               1.4
>                             6             28106             19972               1.4
>add(a,c,d)                   2                40                22               1.9
>                             3                46                33               1.4
>                             4               144               132               1.1
>                             5              1128              1165               1.0
>                             6             10316             10827               1.0
>a+c                          2                91                31               2.9
>                             3               105                45               2.4
>                             4               335               231               1.5
>                             5              2945              2130               1.4
>                             6             28102             19744               1.4
>cos(a)                       2                84                53               1.6
>                             3               212               235               0.9
>                             4              1546              2351               0.7
>                             5             15237             23074               0.7
>                             6            151604            227898               0.7
>add.reduce(a)                2                97                31               3.0
>                             3               109                41               2.7
>                             4               207               123               1.7
>                             5              1062               949               1.1
>                             6              8974              8844               1.0
>add.reduce(b,0)              2             32527              9218               3.5
>                             3             31652              9238               3.4
>                             4             31376              9171               3.4
>                             5             31412              9271               3.4
>                             6             30443              9263               3.3
>add.reduce(b,-1)             2              9065              9389               1.0
>                             3              9075              9360               1.0
>                             4              9102              9271               1.0
>                             5              9168              9389               1.0
>                             6              9148              9394               1.0
>add.accumulate(a)            2               100                28               3.5
>                             3               120                41               2.9
>                             4               416               254               1.6
>                             5              3584              2696               1.3
>                             6             34837             25608               1.4
>add.accumulate(b,0)          2             63947             25673               2.5
>                             3             65875             25691               2.6
>                             4             65540             25708               2.5
>                             5             65161             25776               2.5
>                             6             66539             25646               2.6
>add.accumulate(b,1)          2             35048             25756               1.4
>                             3             35022             25738               1.4
>                             4             35048             25688               1.4
>                             5             35092             25761               1.4
>                             6             35110             25686               1.4
>
>
>
>
>-------------------------------------------------------
>This sf.net email is sponsored by:ThinkGeek
>Welcome to geek heaven.
>http://thinkgeek.com/sf
>_______________________________________________
>Numpy-discussion mailing list
>Numpy-discussion at lists.sourceforge.net
>https://lists.sourceforge.net/lists/listinfo/numpy-discussion
>
>  
>





From jmiller at stsci.edu  Thu Sep 25 12:06:02 2003
From: jmiller at stsci.edu (Todd Miller)
Date: Thu Sep 25 12:06:02 2003
Subject: [Numpy-discussion] Subclassing numarray arrays
In-Reply-To: <3F7317A4.2040107@pfdubois.com>
References: <1064478298.10525.24.camel@Nadav.Envision.co.il> 
	<3F7317A4.2040107@pfdubois.com>
Message-ID: <1064516557.13807.157.camel@halloween.stsci.edu>

On Thu, 2003-09-25 at 12:28, Paul Dubois wrote:
> I have not followed this thread in any detail but I continue to be 
> puzzled by the idea of subclassing numarray arrays. Any subclass worth 
> its salt has an unpredictable constructor argument list, so that 
> virtually every ufunc and every array method have to be overridden in 
> order to return the subclass object rather than a numarray. 

I think there is a need to effectively subclass numarray where the
parameter lists don't change.  Doing a Matrix subclass sounds reasonable
to me and we should explore the OOP necessary to make it convenient. 
STSCI hasn't asked for it,  but a couple different numarray contributors
have.

> I gather 
> that the defers-to-me stuff allows the subclass to get control of a 
> mixed operation but that was possible with the radd, etc. operators 
> already as in the old MA.

radd does not work for numarray + MA.  Instead, the expression is
resolved as numarray.__add__(MA).  Were MA a numarray subclass, it would
work, given the facilities provided by Python alone.

The defers-to-me-stuff is a simple work around to avoid having to
reverse engineer this one Numeric behavior.

> This reasoning led me to believe that subclassing had virtually no value 
> and to the design of MA that has-a numeric array rather than is-a.

Since we want to port MA as-is to numarray, we're not changing that.

> Is there some big breakthrough here I missed?

No,  but there are two issues:  (1) effectively supporting subclasses
(2) effectively supporting interoperability with other classes.

(1) needs more discussion.
(2) is already solved.

-- 
Todd Miller 			jmiller at stsci.edu
STSCI / ESS / SSB




From cjw at sympatico.ca  Thu Sep 25 12:09:03 2003
From: cjw at sympatico.ca (Colin J. Williams)
Date: Thu Sep 25 12:09:03 2003
Subject: [Numpy-discussion] Crash with numarray testall.py
Message-ID: <3F733D14.5060200@sympatico.ca>

C:\Program Files\Python23\Lib\site-packages\numarray>python
Python 2.3.1 (#47, Sep 23 2003, 23:47:32) [MSC v.1200 32 bit (Intel)] on 
win32
Type "help", "copyright", "credits" or "license" for more information.
 >>> import testall

The 21K file created by Windows to contain information about the crash 
is not readable.

Windows XP and numarray 0.7 used.

Colin W.




From jmiller at stsci.edu  Thu Sep 25 13:20:05 2003
From: jmiller at stsci.edu (Todd Miller)
Date: Thu Sep 25 13:20:05 2003
Subject: [Numpy-discussion] Subclassing numarray arrays
In-Reply-To: <3F731467.1070402@sympatico.ca>
References: <1064478298.10525.24.camel@Nadav.Envision.co.il>
		<3F72DC68.6030403@sympatico.ca>
	<1064499880.13807.111.camel@halloween.stsci.edu> 
	<3F731467.1070402@sympatico.ca>
Message-ID: <1064521138.13806.229.camel@halloween.stsci.edu>

On Thu, 2003-09-25 at 12:14, Colin J. Williams wrote:
> Todd,
> 
> Thanks for the benchmark data.

Thanks for the references.  They suggest that the classmethod approach
we've already tried for folding in the numarray factory functions is
reasonable OOP.
 
> I intend to post the matrix stuff.  Performance is not a problem for me 
> at this stage.
> 
> Currently, I'm plodding along, making things work.  Later, I'll post it 
> with some rudimentary documentation.
> 
> Regarding the "new-classes standard", I can't find much in either the 
> Python docs or Alex Martelli's "Python in a Nutshell".  One of the new 
> features is a __new__ constructor.

Well,  we've certainly got those (__new__ methods).  They don't match
the __init__ signature though,  nor was I aware that they should.

> The Python array module doesn't formally have a class and thus it can't 
> be sub-classed.  It uses a factory function to create instance objects.
> 
> Martelli, in PIAN p74, writes of factory functions being useful when one 
> wishes to create instances of different classes depending on some 
> condition or for the reuse of an existing instance.
> 
> In the Python Cookbook p173, he proposes the use of a factory function 
> to get around the limitations of Python 2.1 and earlier.  On page 179, 
> he refers to the use of a factory method, again to work around 
> limitations before Python 2.2.
> 
> My rough and ready definition would include the ability to create an 
> instance and initialize
> the data in one step.
> 

One thing to keep in mind is that NumArray was designed to run on
Python-2.0.  As Python-2.2 became widely available and stable, we
realized we could exploit it for better C-API compatibility with
Numeric, and for better speed.   

I think what you and later Nadav have pointed out is that we can exploit
Python-2.2 for better sub-classing as well.  We just haven't done it
yet.  Concrete proposals are welcome.

Regards,
Todd





From jmiller at stsci.edu  Fri Sep 26 06:41:12 2003
From: jmiller at stsci.edu (Todd Miller)
Date: Fri Sep 26 06:41:12 2003
Subject: [Numpy-discussion] numarray unfriendly to user defined types
In-Reply-To: <3F722F0B.7070609@ieee.org>
References: <3F6A3D4C.3070703@ieee.org>	
	<1064266268.9234.351.camel@halloween.stsci.edu> <3F6F7744.80908@ieee.org>	
	<3F6F7B83.3040305@colorado.edu>  <3F6F8546.9020500@ieee.org>	
	<1064313211.3460.160.camel@localhost.localdomain>	
	<3F721A4C.7080300@ieee.org>
	<1064444385.23878.14.camel@localhost.localdomain> 
	<3F722F0B.7070609@ieee.org>
Message-ID: <1064583557.13805.293.camel@halloween.stsci.edu>

I tried out DeferredToByNumArray yesterday morning and was able to get
it to work as planned with numarray and MA.  So yay!

Yesterday afternoon,  I stopped by Perry's office for a final check, and
not surprisingly,  there are some aspects of this solution that we've
overlooked.  So not yay!

Perry pointed out two ways that the DeferredToByNumArray scheme may fall
apart as our inheritance hierarchy becomes richer:

1)  Imagine not one, but two independent subclasses of NumArray, each of
which want NumArray to defer to them.  Now imagine them trying to
inter-operate with each other.  In this case, the meaning of the
expression is determined by the order of the operands, so A+B and B+A
will either return an A or a B depending on the expression order.

2)  Imagine again two subclasses of numarray, but this time imagine B as
a subclass of A.  Here again, both might want to defer to numarray, and
again, A+B and B+A return different types driven by the type order of
the expression.

I don't have a solution yet, but am hopeful that more free candy will
fall from the sky... or YAGNI.  Perry pointed out that similar problems
exist for *any* Python class hierarchy, so we're not alone, and perhaps
should forget about this until it matters.  If someone sees an easy
fix,  now would be better than later.

Todd
On Wed, 2003-09-24 at 19:55, Tim Hochberg wrote:
> Todd Miller wrote:
> 
> >On Wed, 2003-09-24 at 18:27, Tim Hochberg wrote:
> >  
> >
> >>Hi Todd,
> >>
> >>There are three ways to spell "defer to me" on the table (the precise 
> >>details of
> >>each spelling are, of course, still open for debate):
> >>
> >>   1.  numarray.defer_to(my_class)
> >>
> >>   2. class ArrayLike(numarray.DeferTo):
> >>         # ...
> >>
> >>   3. class ArrayLike:
> >>         _numarray_defer_to = True
> >>        # ...
> >>
> >>I'd prefer a non-registration solution since those are both 
> >>aesthetically displeasing and leave you open to the situation where a 
> >>class in module A gets registered by module B, but module C expects it 
> >>not to be registered and everything breaks. Not all that likely, I 
> >>admit, but let's avoid the registration version if we can.
> >>    
> >>
> >
> >I was picturing this as module A registering it's own classes only. 
> >Nevertheless, inverting the problem and distributing the registration as
> >you suggested is better.
> >  
> >
> 
> The case you describe probably will describe the majority of actual use 
> cases, and in fact describes mine. I'm trying to think ahead a bit to 
> cases may encounter as start using NumArray more extensively. Let's hope 
> this solution still looks good in six months!
> 
> >>The other two solutions are almost equivalent. The one case where 3 has 
> >>an edge over 2 is if I have an object (not a class), I could potentially 
> >>set a _numarray_defer_to on the object before passing it to numarray 
> >>without having to mess with the class
> >>of the object at all. YAGNI, though.
> >>    
> >>
> >
> >I was more concerned about the potential impact of lots of multiple
> >inheritance, but that's probably just my own personal blend of FUD.
> >
> >  
> >
> >>The advantage of 2 in my view is that it *does* force you to subclass. 
> >>With 3, there will be the temptation to poke into some other  module and 
> >>set _numarray_defer_to on some poor unsuspecting class. This has the 
> >>same disadvantage as 1, that it could confuse some other poor 
> >>unsuspecting module. The correct way to do get a deferred class from a 
> >>third party module is to import and subclass. This works with either 2 
> >>or 3::
> >>
> >>  import A
> >>
> >>  class Klass2(a.Klass, numarray.DeferTo):   #2
> >>     #...
> >>
> >>  class Klass3(a.Klass): #3 the good way
> >>      _numarray_defer_to = True
> >>     # ...
> >>
> >>   A.Klass._numarray_defer_to = True #3 the evil way.
> >>
> >> Version 2 is cleaner and encourages you to do the right thing, so I'd 
> >>prefer that solution.
> >>
> >>    
> >>
> >
> >Good enough for me.  If no one else has any comments, then
> >numarray.DeferTo is where I'll start implementing.  Tomorrow.
> >  
> >
> 
> One more minor thing. I'm not sure tha DeferTo is ideal as the mix-in 
> class name. It was perfect for the registration function name, but I'm 
> not sure it's so clear whether the class or numarray is being deferred 
> to when you say numarray.DeferTo. DeferToMe is more descriptive, but 
> seems sort of slangy. DeferredTo is better than DeferTo, but still not 
> as descriptive as DeferToMe. numarray.DefersTo reads perfect as long as 
> numarray is included but is a disaster if you read it on your own.  
> Below I've put down all the ideas I could come up with
> 
> class CustomArray(numarray.DeferTo)
> class CustomArray(numarray.DefersTo)
> class CustomArray(numarray.DeferredTo)
> class CustomArray(numarray.DeferToMe)
> class CustomArray(numarray.DeferredToByNumarray)
> class CustomArray(DeferTo)
> class CustomArray(DefersTo)
> class CustomArray(DeferredTo)
> class CustomArray(DeferToMe)
> class CustomArray(DeferredToByNumarray)
> 
> For me it's a toss up between DefferedTo, DeferToMe and 
> DeferredToByNumarray. The first is a little lacking in descriptive 
> power, the second is slangy and the third is wordy.
> 
> -tim
> 
> [not that this matters much....]
> 
> 
> >>regards,
> >>
> >>-tim
> >>    
> >>
> >
> >Thanks again,
> >Todd
> >
> >
> >
> >
> >
> >  
> >
> 
> 
> 
> 
> 
> -------------------------------------------------------
> This sf.net email is sponsored by:ThinkGeek
> Welcome to geek heaven.
> http://thinkgeek.com/sf
> _______________________________________________
> Numpy-discussion mailing list
> Numpy-discussion at lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/numpy-discussion
-- 
Todd Miller 			jmiller at stsci.edu
STSCI / ESS / SSB




From tim.hochberg at ieee.org  Fri Sep 26 10:14:07 2003
From: tim.hochberg at ieee.org (Tim Hochberg)
Date: Fri Sep 26 10:14:07 2003
Subject: [Numpy-discussion] numarray unfriendly to user defined types
In-Reply-To: <1064583557.13805.293.camel@halloween.stsci.edu>
References: <3F6A3D4C.3070703@ieee.org>		<1064266268.9234.351.camel@halloween.stsci.edu> <3F6F7744.80908@ieee.org>		<3F6F7B83.3040305@colorado.edu>  <3F6F8546.9020500@ieee.org>		<1064313211.3460.160.camel@localhost.localdomain>		<3F721A4C.7080300@ieee.org>	<1064444385.23878.14.camel@localhost.localdomain> 	<3F722F0B.7070609@ieee.org> <1064583557.13805.293.camel@halloween.stsci.edu>
Message-ID: <3F746764.2070403@ieee.org>

Todd Miller wrote:

>I tried out DeferredToByNumArray yesterday morning and was able to get
>it to work as planned with numarray and MA.  So yay!
>  
>
Excellent.

>Yesterday afternoon,  I stopped by Perry's office for a final check, and
>not surprisingly,  there are some aspects of this solution that we've
>overlooked.  So not yay!
>
>Perry pointed out two ways that the DeferredToByNumArray scheme may fall
>apart as our inheritance hierarchy becomes richer:
>  
>
Not so excellent.

>1)  Imagine not one, but two independent subclasses of NumArray, each of
>which want NumArray to defer to them.  Now imagine them trying to
>inter-operate with each other.  In this case, the meaning of the
>expression is determined by the order of the operands, so A+B and B+A
>will either return an A or a B depending on the expression order.
>  
>
I think that this is YAGNI right now, but may well be an issue is 
"proper" subclassing get's implemented. So we may as well think about it 
a bit. I call it YAGNI now because these subclasses either use the 
default __op__ methods, which presently have the same behaviour 
regardless of the subclasses (they always return NumArrays) or they 
define there own __op__ methods in which case they can worry about it ;> 
If NumArray starts to become more friendly to subclassing, this will 
probably change though.

>2)  Imagine again two subclasses of numarray, but this time imagine B as
>a subclass of A.  Here again, both might want to defer to numarray, and
>again, A+B and B+A return different types driven by the type order of
>the expression.
>  
>

I'd make the same comment for this case as for the above case except 
that all else being equal, subclasses should prevail over superclasses.

>I don't have a solution yet, but am hopeful that more free candy will
>fall from the sky... or YAGNI.  Perry pointed out that similar problems
>exist for *any* Python class hierarchy, so we're not alone, and perhaps
>should forget about this until it matters.  If someone sees an easy
>fix,  now would be better than later.
>  
>
Here's some quick thoughts. In order for this to work out sensibly you 
want the order between classes to be transistive and you probably also 
always want to be able to insert a class into the order between any two 
classes  This is equivalent to deciding the op priority based on a 
comparison based on some real number associated with each class. This 
leads to something like:

   
class NumArrayFamily:
    op_priority = 0.0
   
# This makes it compatible with the current proposal , but isn't 
strictly necessary.
class DeferredToByNumarray(NumArrayFamily):
    op_priority = 1000.0
   
class NumArrary(..., NumArrayFamily):
    # ...
    def __add__(self, object):
        if isinstance(object, NumArrayFamily):
            if ((object.op_priority > self.object.op_priority) or
                (object.op_priority == self.object.op_priority) and 
issubclass(object, self.__class__)):
                    return other.__radd__(self)
        return ufunc.add(self, other)


class MySubclassOfNumarray(NumArray):
    op_priority = 10.0 # deffered to by NumArray, but defers to 
DeferredToByNumarray classes.



Of course the user still has to be careful to make sure the priority 
order makes sense and you could have big problems when combining 
packages with disprate ideas about priority levels, but I'm not sure how 
you can get away from that.

-tim



>Todd
>On Wed, 2003-09-24 at 19:55, Tim Hochberg wrote:
>  
>
>>Todd Miller wrote:
>>
>>    
>>
>>>On Wed, 2003-09-24 at 18:27, Tim Hochberg wrote:
>>> 
>>>
>>>      
>>>
>>>>Hi Todd,
>>>>
>>>>There are three ways to spell "defer to me" on the table (the precise 
>>>>details of
>>>>each spelling are, of course, still open for debate):
>>>>
>>>>  1.  numarray.defer_to(my_class)
>>>>
>>>>  2. class ArrayLike(numarray.DeferTo):
>>>>        # ...
>>>>
>>>>  3. class ArrayLike:
>>>>        _numarray_defer_to = True
>>>>       # ...
>>>>
>>>>I'd prefer a non-registration solution since those are both 
>>>>aesthetically displeasing and leave you open to the situation where a 
>>>>class in module A gets registered by module B, but module C expects it 
>>>>not to be registered and everything breaks. Not all that likely, I 
>>>>admit, but let's avoid the registration version if we can.
>>>>   
>>>>
>>>>        
>>>>
>>>I was picturing this as module A registering it's own classes only. 
>>>Nevertheless, inverting the problem and distributing the registration as
>>>you suggested is better.
>>> 
>>>
>>>      
>>>
>>The case you describe probably will describe the majority of actual use 
>>cases, and in fact describes mine. I'm trying to think ahead a bit to 
>>cases may encounter as start using NumArray more extensively. Let's hope 
>>this solution still looks good in six months!
>>
>>    
>>
>>>>The other two solutions are almost equivalent. The one case where 3 has 
>>>>an edge over 2 is if I have an object (not a class), I could potentially 
>>>>set a _numarray_defer_to on the object before passing it to numarray 
>>>>without having to mess with the class
>>>>of the object at all. YAGNI, though.
>>>>   
>>>>
>>>>        
>>>>
>>>I was more concerned about the potential impact of lots of multiple
>>>inheritance, but that's probably just my own personal blend of FUD.
>>>
>>> 
>>>
>>>      
>>>
>>>>The advantage of 2 in my view is that it *does* force you to subclass. 
>>>>With 3, there will be the temptation to poke into some other  module and 
>>>>set _numarray_defer_to on some poor unsuspecting class. This has the 
>>>>same disadvantage as 1, that it could confuse some other poor 
>>>>unsuspecting module. The correct way to do get a deferred class from a 
>>>>third party module is to import and subclass. This works with either 2 
>>>>or 3::
>>>>
>>>> import A
>>>>
>>>> class Klass2(a.Klass, numarray.DeferTo):   #2
>>>>    #...
>>>>
>>>> class Klass3(a.Klass): #3 the good way
>>>>     _numarray_defer_to = True
>>>>    # ...
>>>>
>>>>  A.Klass._numarray_defer_to = True #3 the evil way.
>>>>
>>>>Version 2 is cleaner and encourages you to do the right thing, so I'd 
>>>>prefer that solution.
>>>>
>>>>   
>>>>
>>>>        
>>>>
>>>Good enough for me.  If no one else has any comments, then
>>>numarray.DeferTo is where I'll start implementing.  Tomorrow.
>>> 
>>>
>>>      
>>>
>>One more minor thing. I'm not sure tha DeferTo is ideal as the mix-in 
>>class name. It was perfect for the registration function name, but I'm 
>>not sure it's so clear whether the class or numarray is being deferred 
>>to when you say numarray.DeferTo. DeferToMe is more descriptive, but 
>>seems sort of slangy. DeferredTo is better than DeferTo, but still not 
>>as descriptive as DeferToMe. numarray.DefersTo reads perfect as long as 
>>numarray is included but is a disaster if you read it on your own.  
>>Below I've put down all the ideas I could come up with
>>
>>class CustomArray(numarray.DeferTo)
>>class CustomArray(numarray.DefersTo)
>>class CustomArray(numarray.DeferredTo)
>>class CustomArray(numarray.DeferToMe)
>>class CustomArray(numarray.DeferredToByNumarray)
>>class CustomArray(DeferTo)
>>class CustomArray(DefersTo)
>>class CustomArray(DeferredTo)
>>class CustomArray(DeferToMe)
>>class CustomArray(DeferredToByNumarray)
>>
>>For me it's a toss up between DefferedTo, DeferToMe and 
>>DeferredToByNumarray. The first is a little lacking in descriptive 
>>power, the second is slangy and the third is wordy.
>>
>>-tim
>>
>>[not that this matters much....]
>>
>>
>>    
>>
>>>>regards,
>>>>
>>>>-tim
>>>>   
>>>>
>>>>        
>>>>
>>>Thanks again,
>>>Todd
>>>
>>>
>>>
>>>
>>>
>>> 
>>>
>>>      
>>>
>>
>>
>>
>>-------------------------------------------------------
>>This sf.net email is sponsored by:ThinkGeek
>>Welcome to geek heaven.
>>http://thinkgeek.com/sf
>>_______________________________________________
>>Numpy-discussion mailing list
>>Numpy-discussion at lists.sourceforge.net
>>https://lists.sourceforge.net/lists/listinfo/numpy-discussion
>>    
>>






From tim.hochberg at ieee.org  Fri Sep 26 10:14:08 2003
From: tim.hochberg at ieee.org (Tim Hochberg)
Date: Fri Sep 26 10:14:08 2003
Subject: [Numpy-discussion] numarray incompatibility: searchsorted
Message-ID: <3F747208.6020401@ieee.org>

There are some compatibility and doc issues and perhaps a bug in 
numarray.searchsorted. The compatibility issue is that 
Numeric.searchsorted(a, v) accepts either a sequence or scalar value for 
v. Numarray.searchsorted accepts only sequence values.

Second, the doc issue. The docstring for numarray.searchsorted states::

  searchsorted(bins, values)
    searchsort(bins, values) returns the array of greatest indices 'i'
    such that each values[i] <= bins[i].

I assume that should really read something more like::

  searchsorted(bins, values)
    searchsort(bins, values) returns the array A[j] of greatest indices 'i'
    such that each values[j] <= bins[i].

Third, the possible bug:

# na = numarray, np = NumPy
 >>> na.searchsorted([1,2,3,4], [2.5, 3.5])
array([1, 2])
 >>> np.searchsorted([1,2,3,4], [2.5, 3.5])
array([2, 3])

Hmmm. It appears that numarray result does match the numarray docs, (at 
least as I read them), but I like the Numeric behaviour better.  The 
Numeric behaviour also matches the behaviour of the bisect module, which 
is described as::

    bisect = bisect_right(a, x, lo=0, hi=None)
        Return the index where to insert item x in list a, assuming a is 
sorted.

        The return value i is such that all e in a[:i] have e <= x, and 
all e in
        a[i:] have e > x.  So if x already appears in the list, i points 
just
        beyond the rightmost x already there.

        Optional args lo (default 0) and hi (default len(a)) bound the
        slice of a to be searched.

I'd recomend matching the behaviour of the two existing modules (bisect 
and Numeric).

-tim








From jmiller at stsci.edu  Fri Sep 26 11:08:01 2003
From: jmiller at stsci.edu (Todd Miller)
Date: Fri Sep 26 11:08:01 2003
Subject: [Numpy-discussion] numarray incompatibility: searchsorted
In-Reply-To: <3F747208.6020401@ieee.org>
References: <3F747208.6020401@ieee.org>
Message-ID: <1064599467.1021.2.camel@halloween.stsci.edu>

I logged this on Source Forge.  Thanks for the comments.

Todd

On Fri, 2003-09-26 at 13:06, Tim Hochberg wrote:
> 
> There are some compatibility and doc issues and perhaps a bug in 
> numarray.searchsorted. The compatibility issue is that 
> Numeric.searchsorted(a, v) accepts either a sequence or scalar value for 
> v. Numarray.searchsorted accepts only sequence values.
> 
> Second, the doc issue. The docstring for numarray.searchsorted states::
> 
>   searchsorted(bins, values)
>     searchsort(bins, values) returns the array of greatest indices 'i'
>     such that each values[i] <= bins[i].
> 
> I assume that should really read something more like::
> 
>   searchsorted(bins, values)
>     searchsort(bins, values) returns the array A[j] of greatest indices 'i'
>     such that each values[j] <= bins[i].
> 
> Third, the possible bug:
> 
> # na = numarray, np = NumPy
>  >>> na.searchsorted([1,2,3,4], [2.5, 3.5])
> array([1, 2])
>  >>> np.searchsorted([1,2,3,4], [2.5, 3.5])
> array([2, 3])
> 
> Hmmm. It appears that numarray result does match the numarray docs, (at 
> least as I read them), but I like the Numeric behaviour better.  The 
> Numeric behaviour also matches the behaviour of the bisect module, which 
> is described as::
> 
>     bisect = bisect_right(a, x, lo=0, hi=None)
>         Return the index where to insert item x in list a, assuming a is 
> sorted.
> 
>         The return value i is such that all e in a[:i] have e <= x, and 
> all e in
>         a[i:] have e > x.  So if x already appears in the list, i points 
> just
>         beyond the rightmost x already there.
> 
>         Optional args lo (default 0) and hi (default len(a)) bound the
>         slice of a to be searched.
> 
> I'd recomend matching the behaviour of the two existing modules (bisect 
> and Numeric).
> 
> -tim
> 
> 
> 
> 
> 
> 
> 
> -------------------------------------------------------
> This sf.net email is sponsored by:ThinkGeek
> Welcome to geek heaven.
> http://thinkgeek.com/sf
> _______________________________________________
> Numpy-discussion mailing list
> Numpy-discussion at lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/numpy-discussion
-- 
Todd Miller 			jmiller at stsci.edu
STSCI / ESS / SSB




From jmiller at stsci.edu  Fri Sep 26 11:24:09 2003
From: jmiller at stsci.edu (Todd Miller)
Date: Fri Sep 26 11:24:09 2003
Subject: [Numpy-discussion] numarray unfriendly to user defined types
In-Reply-To: <3F74780D.3070701@sympatico.ca>
References: 	<3F6A3D4C.3070703@ieee.org>		<1064266268.9234.351.camel@halloween.stsci.edu>
	 <3F6F7744.80908@ieee.org>		<3F6F7B83.3040305@colorado.edu> 
	<3F6F8546.9020500@ieee.org>		<1064313211.3460.160.camel@localhost.localdomai
	n>		<3F721A4C.7080300@ieee.org>	<1064444385.23878.14.camel@localhost.localdo
	main> 	<3F722F0B.7070609@ieee.org>
	<1064583557.13805.293.camel@halloween.stsci.edu> 
	<3F74780D.3070701@sympatico.ca>
Message-ID: <1064600606.1021.17.camel@halloween.stsci.edu>

On Fri, 2003-09-26 at 13:31, Colin J. Williams wrote:
> Todd,
> 
> This scheme seems rather complex.  The purpose isn't clear to me.
> 
> What is the problem which this arrangement is intended to solve?

Bottom line, currently:  
	
numarray + another_class --> exception

It would be nice if:  

numarray + another_class --> another_class.__radd__(numarray)

Also, look backward at Tim's original post on this subject.

> 
> Could I suggest that some scheme along the lines of the Python PEP's be 
> used to set out the Why's and Wherefore's?
> 
Sounds good for next time.

> It might also be useful to copy this discussion to comp.lang.python as 
> that newsgroup has people experienced with classes.  

OK,  I'll stop being a chicken.  Everyone, get ready for the sound of a
pin dropping on c.l.py.

> The problem appears 
> to be Python related rather than a problem of numerical analysis.

Yes indeed.

> 
> Colin W.
> 
> Todd Miller wrote:
> 
> >I tried out DeferredToByNumArray yesterday morning and was able to get
> >it to work as planned with numarray and MA.  So yay!
> >
> >Yesterday afternoon,  I stopped by Perry's office for a final check, and
> >not surprisingly,  there are some aspects of this solution that we've
> >overlooked.  So not yay!
> >
> >Perry pointed out two ways that the DeferredToByNumArray scheme may fall
> >apart as our inheritance hierarchy becomes richer:
> >
> >1)  Imagine not one, but two independent subclasses of NumArray, each of
> >which want NumArray to defer to them.  Now imagine them trying to
> >inter-operate with each other.  In this case, the meaning of the
> >expression is determined by the order of the operands, so A+B and B+A
> >will either return an A or a B depending on the expression order.
> >
> >2)  Imagine again two subclasses of numarray, but this time imagine B as
> >a subclass of A.  Here again, both might want to defer to numarray, and
> >again, A+B and B+A return different types driven by the type order of
> >the expression.
> >
> >I don't have a solution yet, but am hopeful that more free candy will
> >fall from the sky... or YAGNI.  Perry pointed out that similar problems
> >exist for *any* Python class hierarchy, so we're not alone, and perhaps
> >should forget about this until it matters.  If someone sees an easy
> >fix,  now would be better than later.
> >
> >Todd
> >On Wed, 2003-09-24 at 19:55, Tim Hochberg wrote:
> >  
> >
> >>Todd Miller wrote:
> >>
> >>    
> >>
> >>>On Wed, 2003-09-24 at 18:27, Tim Hochberg wrote:
> >>> 
> >>>
> >>>      
> >>>
> >>>>Hi Todd,
> >>>>
> >>>>There are three ways to spell "defer to me" on the table (the precise 
> >>>>details of
> >>>>each spelling are, of course, still open for debate):
> >>>>
> >>>>  1.  numarray.defer_to(my_class)
> >>>>
> >>>>  2. class ArrayLike(numarray.DeferTo):
> >>>>        # ...
> >>>>
> >>>>  3. class ArrayLike:
> >>>>        _numarray_defer_to = True
> >>>>       # ...
> >>>>
> >>>>I'd prefer a non-registration solution since those are both 
> >>>>aesthetically displeasing and leave you open to the situation where a 
> >>>>class in module A gets registered by module B, but module C expects it 
> >>>>not to be registered and everything breaks. Not all that likely, I 
> >>>>admit, but let's avoid the registration version if we can.
> >>>>   
> >>>>
> >>>>        
> >>>>
> >>>I was picturing this as module A registering it's own classes only. 
> >>>Nevertheless, inverting the problem and distributing the registration as
> >>>you suggested is better.
> >>> 
> >>>
> >>>      
> >>>
> >>The case you describe probably will describe the majority of actual use 
> >>cases, and in fact describes mine. I'm trying to think ahead a bit to 
> >>cases may encounter as start using NumArray more extensively. Let's hope 
> >>this solution still looks good in six months!
> >>
> >>    
> >>
> >>>>The other two solutions are almost equivalent. The one case where 3 has 
> >>>>an edge over 2 is if I have an object (not a class), I could potentially 
> >>>>set a _numarray_defer_to on the object before passing it to numarray 
> >>>>without having to mess with the class
> >>>>of the object at all. YAGNI, though.
> >>>>   
> >>>>
> >>>>        
> >>>>
> >>>I was more concerned about the potential impact of lots of multiple
> >>>inheritance, but that's probably just my own personal blend of FUD.
> >>>
> >>> 
> >>>
> >>>      
> >>>
> >>>>The advantage of 2 in my view is that it *does* force you to subclass. 
> >>>>With 3, there will be the temptation to poke into some other  module and 
> >>>>set _numarray_defer_to on some poor unsuspecting class. This has the 
> >>>>same disadvantage as 1, that it could confuse some other poor 
> >>>>unsuspecting module. The correct way to do get a deferred class from a 
> >>>>third party module is to import and subclass. This works with either 2 
> >>>>or 3::
> >>>>
> >>>> import A
> >>>>
> >>>> class Klass2(a.Klass, numarray.DeferTo):   #2
> >>>>    #...
> >>>>
> >>>> class Klass3(a.Klass): #3 the good way
> >>>>     _numarray_defer_to = True
> >>>>    # ...
> >>>>
> >>>>  A.Klass._numarray_defer_to = True #3 the evil way.
> >>>>
> >>>>Version 2 is cleaner and encourages you to do the right thing, so I'd 
> >>>>prefer that solution.
> >>>>
> >>>>   
> >>>>
> >>>>        
> >>>>
> >>>Good enough for me.  If no one else has any comments, then
> >>>numarray.DeferTo is where I'll start implementing.  Tomorrow.
> >>> 
> >>>
> >>>      
> >>>
> >>One more minor thing. I'm not sure tha DeferTo is ideal as the mix-in 
> >>class name. It was perfect for the registration function name, but I'm 
> >>not sure it's so clear whether the class or numarray is being deferred 
> >>to when you say numarray.DeferTo. DeferToMe is more descriptive, but 
> >>seems sort of slangy. DeferredTo is better than DeferTo, but still not 
> >>as descriptive as DeferToMe. numarray.DefersTo reads perfect as long as 
> >>numarray is included but is a disaster if you read it on your own.  
> >>Below I've put down all the ideas I could come up with
> >>
> >>class CustomArray(numarray.DeferTo)
> >>class CustomArray(numarray.DefersTo)
> >>class CustomArray(numarray.DeferredTo)
> >>class CustomArray(numarray.DeferToMe)
> >>class CustomArray(numarray.DeferredToByNumarray)
> >>class CustomArray(DeferTo)
> >>class CustomArray(DefersTo)
> >>class CustomArray(DeferredTo)
> >>class CustomArray(DeferToMe)
> >>class CustomArray(DeferredToByNumarray)
> >>
> >>For me it's a toss up between DefferedTo, DeferToMe and 
> >>DeferredToByNumarray. The first is a little lacking in descriptive 
> >>power, the second is slangy and the third is wordy.
> >>
> >>-tim
> >>
> >>[not that this matters much....]
> >>
> >>
> >>    
> >>
> >>>>regards,
> >>>>
> >>>>-tim
> >>>>   
> >>>>
> >>>>        
> >>>>
> >>>Thanks again,
> >>>Todd
> >>>
> >>>
> >>>
> >>>
> >>>
> >>> 
> >>>
> >>>      
> >>>
> >>
> >>
> >>
> >>-------------------------------------------------------
> >>This sf.net email is sponsored by:ThinkGeek
> >>Welcome to geek heaven.
> >>http://thinkgeek.com/sf
> >>_______________________________________________
> >>Numpy-discussion mailing list
> >>Numpy-discussion at lists.sourceforge.net
> >>https://lists.sourceforge.net/lists/listinfo/numpy-discussion
> >>    
> >>
> 
> 
-- 
Todd Miller 			jmiller at stsci.edu
STSCI / ESS / SSB




From jochen at jochen-kuepper.de  Fri Sep 26 14:24:03 2003
From: jochen at jochen-kuepper.de (=?iso-8859-1?q?Jochen_K=FCpper?=)
Date: Fri Sep 26 14:24:03 2003
Subject: [Numpy-discussion] Cygwin compilation
In-Reply-To: <3F4525A1.3040701@colorado.edu> (Fernando Perez's message of
 "Thu, 21 Aug 2003 14:03:45 -0600")
References: <3F4525A1.3040701@colorado.edu>
Message-ID: <868yob9rfm.fsf@doze.rijnh.nl>

I just compiled/installed numarray-0.7 on (latest) Cygwin.

_ufunc.dll needs to be linked against /usr/lib/libmingwex.a. Doing
that manually seems to work fine. Adding it to the setup files is not
as simple, as libnumarray.dll does not link any more as some wrong
functions seem to be picked up from the MINGW library as well (and the
pow cannot be resolved. Explicitely adding -lm does not help).

How is this planned to work. Where needs the sequence
"-L/usr/lib/mingw -lmingwex" be put?

Is it save to use this mingw library in "normal" Cygwin programs?

Same happens with cvs HEAD.


Moreover I have the follwoing test error:
,----
| Failure in example: log10([1+1j])
| from line #2267 of numarray.numtest
| Expected: array([ 0.150515+0.34109409j])
| Got: array([ 0.150515+5.21807245j])
`----

Greetings,
Jochen
-- 
Einigkeit und Recht und Freiheit                http://www.Jochen-Kuepper.de
    Libert?, ?galit?, Fraternit?                GnuPG key: CC1B0B4D
        (Part 3 you find in my messages before fall 2003.)




From cjw at sympatico.ca  Fri Sep 26 19:01:02 2003
From: cjw at sympatico.ca (Colin J. Williams)
Date: Fri Sep 26 19:01:02 2003
Subject: [Numpy-discussion] numarray unfriendly to user defined types
In-Reply-To: <1064583557.13805.293.camel@halloween.stsci.edu>
References: <3F6A3D4C.3070703@ieee.org>		<1064266268.9234.351.camel@halloween.stsci.edu> <3F6F7744.80908@ieee.org>		<3F6F7B83.3040305@colorado.edu>  <3F6F8546.9020500@ieee.org>		<1064313211.3460.160.camel@localhost.localdomain>		<3F721A4C.7080300@ieee.org>	<1064444385.23878.14.camel@localhost.localdomain> 	<3F722F0B.7070609@ieee.org> <1064583557.13805.293.camel@halloween.stsci.edu>
Message-ID: <3F74780D.3070701@sympatico.ca>

Todd,

This scheme seems rather complex.  The purpose isn't clear to me.

What is the problem which this arrangement is intended to solve?

Could I suggest that some scheme along the lines of the Python PEP's be 
used to set out the Why's and Wherefore's?

It might also be useful to copy this discussion to comp.lang.python as 
that newsgroup has people experienced with classes.  The problem appears 
to be Python related rather than a problem of numerical analysis.

Colin W.

Todd Miller wrote:

>I tried out DeferredToByNumArray yesterday morning and was able to get
>it to work as planned with numarray and MA.  So yay!
>
>Yesterday afternoon,  I stopped by Perry's office for a final check, and
>not surprisingly,  there are some aspects of this solution that we've
>overlooked.  So not yay!
>
>Perry pointed out two ways that the DeferredToByNumArray scheme may fall
>apart as our inheritance hierarchy becomes richer:
>
>1)  Imagine not one, but two independent subclasses of NumArray, each of
>which want NumArray to defer to them.  Now imagine them trying to
>inter-operate with each other.  In this case, the meaning of the
>expression is determined by the order of the operands, so A+B and B+A
>will either return an A or a B depending on the expression order.
>
>2)  Imagine again two subclasses of numarray, but this time imagine B as
>a subclass of A.  Here again, both might want to defer to numarray, and
>again, A+B and B+A return different types driven by the type order of
>the expression.
>
>I don't have a solution yet, but am hopeful that more free candy will
>fall from the sky... or YAGNI.  Perry pointed out that similar problems
>exist for *any* Python class hierarchy, so we're not alone, and perhaps
>should forget about this until it matters.  If someone sees an easy
>fix,  now would be better than later.
>
>Todd
>On Wed, 2003-09-24 at 19:55, Tim Hochberg wrote:
>  
>
>>Todd Miller wrote:
>>
>>    
>>
>>>On Wed, 2003-09-24 at 18:27, Tim Hochberg wrote:
>>> 
>>>
>>>      
>>>
>>>>Hi Todd,
>>>>
>>>>There are three ways to spell "defer to me" on the table (the precise 
>>>>details of
>>>>each spelling are, of course, still open for debate):
>>>>
>>>>  1.  numarray.defer_to(my_class)
>>>>
>>>>  2. class ArrayLike(numarray.DeferTo):
>>>>        # ...
>>>>
>>>>  3. class ArrayLike:
>>>>        _numarray_defer_to = True
>>>>       # ...
>>>>
>>>>I'd prefer a non-registration solution since those are both 
>>>>aesthetically displeasing and leave you open to the situation where a 
>>>>class in module A gets registered by module B, but module C expects it 
>>>>not to be registered and everything breaks. Not all that likely, I 
>>>>admit, but let's avoid the registration version if we can.
>>>>   
>>>>
>>>>        
>>>>
>>>I was picturing this as module A registering it's own classes only. 
>>>Nevertheless, inverting the problem and distributing the registration as
>>>you suggested is better.
>>> 
>>>
>>>      
>>>
>>The case you describe probably will describe the majority of actual use 
>>cases, and in fact describes mine. I'm trying to think ahead a bit to 
>>cases may encounter as start using NumArray more extensively. Let's hope 
>>this solution still looks good in six months!
>>
>>    
>>
>>>>The other two solutions are almost equivalent. The one case where 3 has 
>>>>an edge over 2 is if I have an object (not a class), I could potentially 
>>>>set a _numarray_defer_to on the object before passing it to numarray 
>>>>without having to mess with the class
>>>>of the object at all. YAGNI, though.
>>>>   
>>>>
>>>>        
>>>>
>>>I was more concerned about the potential impact of lots of multiple
>>>inheritance, but that's probably just my own personal blend of FUD.
>>>
>>> 
>>>
>>>      
>>>
>>>>The advantage of 2 in my view is that it *does* force you to subclass. 
>>>>With 3, there will be the temptation to poke into some other  module and 
>>>>set _numarray_defer_to on some poor unsuspecting class. This has the 
>>>>same disadvantage as 1, that it could confuse some other poor 
>>>>unsuspecting module. The correct way to do get a deferred class from a 
>>>>third party module is to import and subclass. This works with either 2 
>>>>or 3::
>>>>
>>>> import A
>>>>
>>>> class Klass2(a.Klass, numarray.DeferTo):   #2
>>>>    #...
>>>>
>>>> class Klass3(a.Klass): #3 the good way
>>>>     _numarray_defer_to = True
>>>>    # ...
>>>>
>>>>  A.Klass._numarray_defer_to = True #3 the evil way.
>>>>
>>>>Version 2 is cleaner and encourages you to do the right thing, so I'd 
>>>>prefer that solution.
>>>>
>>>>   
>>>>
>>>>        
>>>>
>>>Good enough for me.  If no one else has any comments, then
>>>numarray.DeferTo is where I'll start implementing.  Tomorrow.
>>> 
>>>
>>>      
>>>
>>One more minor thing. I'm not sure tha DeferTo is ideal as the mix-in 
>>class name. It was perfect for the registration function name, but I'm 
>>not sure it's so clear whether the class or numarray is being deferred 
>>to when you say numarray.DeferTo. DeferToMe is more descriptive, but 
>>seems sort of slangy. DeferredTo is better than DeferTo, but still not 
>>as descriptive as DeferToMe. numarray.DefersTo reads perfect as long as 
>>numarray is included but is a disaster if you read it on your own.  
>>Below I've put down all the ideas I could come up with
>>
>>class CustomArray(numarray.DeferTo)
>>class CustomArray(numarray.DefersTo)
>>class CustomArray(numarray.DeferredTo)
>>class CustomArray(numarray.DeferToMe)
>>class CustomArray(numarray.DeferredToByNumarray)
>>class CustomArray(DeferTo)
>>class CustomArray(DefersTo)
>>class CustomArray(DeferredTo)
>>class CustomArray(DeferToMe)
>>class CustomArray(DeferredToByNumarray)
>>
>>For me it's a toss up between DefferedTo, DeferToMe and 
>>DeferredToByNumarray. The first is a little lacking in descriptive 
>>power, the second is slangy and the third is wordy.
>>
>>-tim
>>
>>[not that this matters much....]
>>
>>
>>    
>>
>>>>regards,
>>>>
>>>>-tim
>>>>   
>>>>
>>>>        
>>>>
>>>Thanks again,
>>>Todd
>>>
>>>
>>>
>>>
>>>
>>> 
>>>
>>>      
>>>
>>
>>
>>
>>-------------------------------------------------------
>>This sf.net email is sponsored by:ThinkGeek
>>Welcome to geek heaven.
>>http://thinkgeek.com/sf
>>_______________________________________________
>>Numpy-discussion mailing list
>>Numpy-discussion at lists.sourceforge.net
>>https://lists.sourceforge.net/lists/listinfo/numpy-discussion
>>    
>>





From cjw at sympatico.ca  Sat Sep 27 13:39:02 2003
From: cjw at sympatico.ca (Colin J. Williams)
Date: Sat Sep 27 13:39:02 2003
Subject: [Numpy-discussion] numarray unfriendly to user defined types
In-Reply-To: <1064600606.1021.17.camel@halloween.stsci.edu>
References: <3F6A3D4C.3070703@ieee.org>		<1064266268.9234.351.camel@halloween.stsci.edu>	 <3F6F7744.80908@ieee.org>		<3F6F7B83.3040305@colorado.edu> 	<3F6F8546.9020500@ieee.org>		<1064313211.3460.160.camel@localhost.localdomai	n>		<3F721A4C.7080300@ieee.org>	<1064444385.23878.14.camel@localhost.localdo	main> 	<3F722F0B.7070609@ieee.org>	<1064583557.13805.293.camel@halloween.stsci.edu> 	<3F74780D.3070701@sympatico.ca> <1064600606.1021.17.camel@halloween.stsci.edu>
Message-ID: <3F75EAD0.3030909@sympatico.ca>

Tim,

Todd refers to your original post.  The only thing I've found is a 
reference to a pre and post addition of zero, the latter was said not to 
work.  Both work for me, see below:

 >>> zero= 0
 >>> zero + _num.arange(5)
array([0, 1, 2, 3, 4])
 >>> _num.arange(5) + zero
array([0, 1, 2, 3, 4])

If I'm barking up the wrong tree, could you point me to your original 
posting in the archive please?

Colin W.
 >>>
Todd Miller wrote:

>On Fri, 2003-09-26 at 13:31, Colin J. Williams wrote:
>  
>
>>Todd,
>>
>>This scheme seems rather complex.  The purpose isn't clear to me.
>>
>>What is the problem which this arrangement is intended to solve?
>>    
>>
>
>Bottom line, currently:  
>	
>numarray + another_class --> exception
>
>It would be nice if:  
>
>numarray + another_class --> another_class.__radd__(numarray)
>
>Also, look backward at Tim's original post on this subject.
>
>  
>
>>Could I suggest that some scheme along the lines of the Python PEP's be 
>>used to set out the Why's and Wherefore's?
>>
>>    
>>
>Sounds good for next time.
>
>  
>
>>It might also be useful to copy this discussion to comp.lang.python as 
>>that newsgroup has people experienced with classes.  
>>    
>>
>
>OK,  I'll stop being a chicken.  Everyone, get ready for the sound of a
>pin dropping on c.l.py.
>
>  
>
>>The problem appears 
>>to be Python related rather than a problem of numerical analysis.
>>    
>>
>
>Yes indeed.
>
>  
>
>>Colin W.
>>
>>Todd Miller wrote:
>>
>>    
>>
>>>I tried out DeferredToByNumArray yesterday morning and was able to get
>>>it to work as planned with numarray and MA.  So yay!
>>>
>>>Yesterday afternoon,  I stopped by Perry's office for a final check, and
>>>not surprisingly,  there are some aspects of this solution that we've
>>>overlooked.  So not yay!
>>>
>>>Perry pointed out two ways that the DeferredToByNumArray scheme may fall
>>>apart as our inheritance hierarchy becomes richer:
>>>
>>>1)  Imagine not one, but two independent subclasses of NumArray, each of
>>>which want NumArray to defer to them.  Now imagine them trying to
>>>inter-operate with each other.  In this case, the meaning of the
>>>expression is determined by the order of the operands, so A+B and B+A
>>>will either return an A or a B depending on the expression order.
>>>
>>>2)  Imagine again two subclasses of numarray, but this time imagine B as
>>>a subclass of A.  Here again, both might want to defer to numarray, and
>>>again, A+B and B+A return different types driven by the type order of
>>>the expression.
>>>
>>>I don't have a solution yet, but am hopeful that more free candy will
>>>fall from the sky... or YAGNI.  Perry pointed out that similar problems
>>>exist for *any* Python class hierarchy, so we're not alone, and perhaps
>>>should forget about this until it matters.  If someone sees an easy
>>>fix,  now would be better than later.
>>>
>>>Todd
>>>On Wed, 2003-09-24 at 19:55, Tim Hochberg wrote:
>>> 
>>>
>>>      
>>>
>>>>Todd Miller wrote:
>>>>
>>>>   
>>>>
>>>>        
>>>>
>>>>>On Wed, 2003-09-24 at 18:27, Tim Hochberg wrote:
>>>>>
>>>>>
>>>>>     
>>>>>
>>>>>          
>>>>>
>>>>>>Hi Todd,
>>>>>>
>>>>>>There are three ways to spell "defer to me" on the table (the precise 
>>>>>>details of
>>>>>>each spelling are, of course, still open for debate):
>>>>>>
>>>>>> 1.  numarray.defer_to(my_class)
>>>>>>
>>>>>> 2. class ArrayLike(numarray.DeferTo):
>>>>>>       # ...
>>>>>>
>>>>>> 3. class ArrayLike:
>>>>>>       _numarray_defer_to = True
>>>>>>      # ...
>>>>>>
>>>>>>I'd prefer a non-registration solution since those are both 
>>>>>>aesthetically displeasing and leave you open to the situation where a 
>>>>>>class in module A gets registered by module B, but module C expects it 
>>>>>>not to be registered and everything breaks. Not all that likely, I 
>>>>>>admit, but let's avoid the registration version if we can.
>>>>>>  
>>>>>>
>>>>>>       
>>>>>>
>>>>>>            
>>>>>>
>>>>>I was picturing this as module A registering it's own classes only. 
>>>>>Nevertheless, inverting the problem and distributing the registration as
>>>>>you suggested is better.
>>>>>
>>>>>
>>>>>     
>>>>>
>>>>>          
>>>>>
>>>>The case you describe probably will describe the majority of actual use 
>>>>cases, and in fact describes mine. I'm trying to think ahead a bit to 
>>>>cases may encounter as start using NumArray more extensively. Let's hope 
>>>>this solution still looks good in six months!
>>>>
>>>>   
>>>>
>>>>        
>>>>
>>>>>>The other two solutions are almost equivalent. The one case where 3 has 
>>>>>>an edge over 2 is if I have an object (not a class), I could potentially 
>>>>>>set a _numarray_defer_to on the object before passing it to numarray 
>>>>>>without having to mess with the class
>>>>>>of the object at all. YAGNI, though.
>>>>>>  
>>>>>>
>>>>>>       
>>>>>>
>>>>>>            
>>>>>>
>>>>>I was more concerned about the potential impact of lots of multiple
>>>>>inheritance, but that's probably just my own personal blend of FUD.
>>>>>
>>>>>
>>>>>
>>>>>     
>>>>>
>>>>>          
>>>>>
>>>>>>The advantage of 2 in my view is that it *does* force you to subclass. 
>>>>>>With 3, there will be the temptation to poke into some other  module and 
>>>>>>set _numarray_defer_to on some poor unsuspecting class. This has the 
>>>>>>same disadvantage as 1, that it could confuse some other poor 
>>>>>>unsuspecting module. The correct way to do get a deferred class from a 
>>>>>>third party module is to import and subclass. This works with either 2 
>>>>>>or 3::
>>>>>>
>>>>>>import A
>>>>>>
>>>>>>class Klass2(a.Klass, numarray.DeferTo):   #2
>>>>>>   #...
>>>>>>
>>>>>>class Klass3(a.Klass): #3 the good way
>>>>>>    _numarray_defer_to = True
>>>>>>   # ...
>>>>>>
>>>>>> A.Klass._numarray_defer_to = True #3 the evil way.
>>>>>>
>>>>>>Version 2 is cleaner and encourages you to do the right thing, so I'd 
>>>>>>prefer that solution.
>>>>>>
>>>>>>  
>>>>>>
>>>>>>       
>>>>>>
>>>>>>            
>>>>>>
>>>>>Good enough for me.  If no one else has any comments, then
>>>>>numarray.DeferTo is where I'll start implementing.  Tomorrow.
>>>>>
>>>>>
>>>>>     
>>>>>
>>>>>          
>>>>>
>>>>One more minor thing. I'm not sure tha DeferTo is ideal as the mix-in 
>>>>class name. It was perfect for the registration function name, but I'm 
>>>>not sure it's so clear whether the class or numarray is being deferred 
>>>>to when you say numarray.DeferTo. DeferToMe is more descriptive, but 
>>>>seems sort of slangy. DeferredTo is better than DeferTo, but still not 
>>>>as descriptive as DeferToMe. numarray.DefersTo reads perfect as long as 
>>>>numarray is included but is a disaster if you read it on your own.  
>>>>Below I've put down all the ideas I could come up with
>>>>
>>>>class CustomArray(numarray.DeferTo)
>>>>class CustomArray(numarray.DefersTo)
>>>>class CustomArray(numarray.DeferredTo)
>>>>class CustomArray(numarray.DeferToMe)
>>>>class CustomArray(numarray.DeferredToByNumarray)
>>>>class CustomArray(DeferTo)
>>>>class CustomArray(DefersTo)
>>>>class CustomArray(DeferredTo)
>>>>class CustomArray(DeferToMe)
>>>>class CustomArray(DeferredToByNumarray)
>>>>
>>>>For me it's a toss up between DefferedTo, DeferToMe and 
>>>>DeferredToByNumarray. The first is a little lacking in descriptive 
>>>>power, the second is slangy and the third is wordy.
>>>>
>>>>-tim
>>>>
>>>>[not that this matters much....]
>>>>
>>>>
>>>>   
>>>>
>>>>        
>>>>
>>>>>>regards,
>>>>>>
>>>>>>-tim
>>>>>>  
>>>>>>
>>>>>>       
>>>>>>
>>>>>>            
>>>>>>
>>>>>Thanks again,
>>>>>Todd
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>     
>>>>>
>>>>>          
>>>>>
>>>>
>>>>-------------------------------------------------------
>>>>This sf.net email is sponsored by:ThinkGeek
>>>>Welcome to geek heaven.
>>>>http://thinkgeek.com/sf
>>>>_______________________________________________
>>>>Numpy-discussion mailing list
>>>>Numpy-discussion at lists.sourceforge.net
>>>>https://lists.sourceforge.net/lists/listinfo/numpy-discussion
>>>>   
>>>>
>>>>        
>>>>
>>    
>>





From cjw at sympatico.ca  Sun Sep 28 11:32:04 2003
From: cjw at sympatico.ca (Colin J. Williams)
Date: Sun Sep 28 11:32:04 2003
Subject: [Numpy-discussion] numarray unfriendly to user defined types
In-Reply-To: <1064600606.1021.17.camel@halloween.stsci.edu>
References: <3F6A3D4C.3070703@ieee.org>		<1064266268.9234.351.camel@halloween.stsci.edu>	 <3F6F7744.80908@ieee.org>		<3F6F7B83.3040305@colorado.edu> 	<3F6F8546.9020500@ieee.org>		<1064313211.3460.160.camel@localhost.localdomai	n>		<3F721A4C.7080300@ieee.org>	<1064444385.23878.14.camel@localhost.localdo	main> 	<3F722F0B.7070609@ieee.org>	<1064583557.13805.293.camel@halloween.stsci.edu> 	<3F74780D.3070701@sympatico.ca> <1064600606.1021.17.camel@halloween.stsci.edu>
Message-ID: <3F75EAD0.3030909@sympatico.ca>

Tim,

Todd refers to your original post.  The only thing I've found is a 
reference to a pre and post addition of zero, the latter was said not to 
work.  Both work for me, see below:

 >>> zero= 0
 >>> zero + _num.arange(5)
array([0, 1, 2, 3, 4])
 >>> _num.arange(5) + zero
array([0, 1, 2, 3, 4])

If I'm barking up the wrong tree, could you point me to your original 
posting in the archive please?

Colin W.
 >>>
Todd Miller wrote:

>On Fri, 2003-09-26 at 13:31, Colin J. Williams wrote:
>  
>
>>Todd,
>>
>>This scheme seems rather complex.  The purpose isn't clear to me.
>>
>>What is the problem which this arrangement is intended to solve?
>>    
>>
>
>Bottom line, currently:  
>	
>numarray + another_class --> exception
>
>It would be nice if:  
>
>numarray + another_class --> another_class.__radd__(numarray)
>
>Also, look backward at Tim's original post on this subject.
>
>  
>
>>Could I suggest that some scheme along the lines of the Python PEP's be 
>>used to set out the Why's and Wherefore's?
>>
>>    
>>
>Sounds good for next time.
>
>  
>
>>It might also be useful to copy this discussion to comp.lang.python as 
>>that newsgroup has people experienced with classes.  
>>    
>>
>
>OK,  I'll stop being a chicken.  Everyone, get ready for the sound of a
>pin dropping on c.l.py.
>
>  
>
>>The problem appears 
>>to be Python related rather than a problem of numerical analysis.
>>    
>>
>
>Yes indeed.
>
>  
>
>>Colin W.
>>
>>Todd Miller wrote:
>>
>>    
>>
>>>I tried out DeferredToByNumArray yesterday morning and was able to get
>>>it to work as planned with numarray and MA.  So yay!
>>>
>>>Yesterday afternoon,  I stopped by Perry's office for a final check, and
>>>not surprisingly,  there are some aspects of this solution that we've
>>>overlooked.  So not yay!
>>>
>>>Perry pointed out two ways that the DeferredToByNumArray scheme may fall
>>>apart as our inheritance hierarchy becomes richer:
>>>
>>>1)  Imagine not one, but two independent subclasses of NumArray, each of
>>>which want NumArray to defer to them.  Now imagine them trying to
>>>inter-operate with each other.  In this case, the meaning of the
>>>expression is determined by the order of the operands, so A+B and B+A
>>>will either return an A or a B depending on the expression order.
>>>
>>>2)  Imagine again two subclasses of numarray, but this time imagine B as
>>>a subclass of A.  Here again, both might want to defer to numarray, and
>>>again, A+B and B+A return different types driven by the type order of
>>>the expression.
>>>
>>>I don't have a solution yet, but am hopeful that more free candy will
>>>fall from the sky... or YAGNI.  Perry pointed out that similar problems
>>>exist for *any* Python class hierarchy, so we're not alone, and perhaps
>>>should forget about this until it matters.  If someone sees an easy
>>>fix,  now would be better than later.
>>>
>>>Todd
>>>On Wed, 2003-09-24 at 19:55, Tim Hochberg wrote:
>>> 
>>>
>>>      
>>>
>>>>Todd Miller wrote:
>>>>
>>>>   
>>>>
>>>>        
>>>>
>>>>>On Wed, 2003-09-24 at 18:27, Tim Hochberg wrote:
>>>>>
>>>>>
>>>>>     
>>>>>
>>>>>          
>>>>>
>>>>>>Hi Todd,
>>>>>>
>>>>>>There are three ways to spell "defer to me" on the table (the precise 
>>>>>>details of
>>>>>>each spelling are, of course, still open for debate):
>>>>>>
>>>>>> 1.  numarray.defer_to(my_class)
>>>>>>
>>>>>> 2. class ArrayLike(numarray.DeferTo):
>>>>>>       # ...
>>>>>>
>>>>>> 3. class ArrayLike:
>>>>>>       _numarray_defer_to = True
>>>>>>      # ...
>>>>>>
>>>>>>I'd prefer a non-registration solution since those are both 
>>>>>>aesthetically displeasing and leave you open to the situation where a 
>>>>>>class in module A gets registered by module B, but module C expects it 
>>>>>>not to be registered and everything breaks. Not all that likely, I 
>>>>>>admit, but let's avoid the registration version if we can.
>>>>>>  
>>>>>>
>>>>>>       
>>>>>>
>>>>>>            
>>>>>>
>>>>>I was picturing this as module A registering it's own classes only. 
>>>>>Nevertheless, inverting the problem and distributing the registration as
>>>>>you suggested is better.
>>>>>
>>>>>
>>>>>     
>>>>>
>>>>>          
>>>>>
>>>>The case you describe probably will describe the majority of actual use 
>>>>cases, and in fact describes mine. I'm trying to think ahead a bit to 
>>>>cases may encounter as start using NumArray more extensively. Let's hope 
>>>>this solution still looks good in six months!
>>>>
>>>>   
>>>>
>>>>        
>>>>
>>>>>>The other two solutions are almost equivalent. The one case where 3 has 
>>>>>>an edge over 2 is if I have an object (not a class), I could potentially 
>>>>>>set a _numarray_defer_to on the object before passing it to numarray 
>>>>>>without having to mess with the class
>>>>>>of the object at all. YAGNI, though.
>>>>>>  
>>>>>>
>>>>>>       
>>>>>>
>>>>>>            
>>>>>>
>>>>>I was more concerned about the potential impact of lots of multiple
>>>>>inheritance, but that's probably just my own personal blend of FUD.
>>>>>
>>>>>
>>>>>
>>>>>     
>>>>>
>>>>>          
>>>>>
>>>>>>The advantage of 2 in my view is that it *does* force you to subclass. 
>>>>>>With 3, there will be the temptation to poke into some other  module and 
>>>>>>set _numarray_defer_to on some poor unsuspecting class. This has the 
>>>>>>same disadvantage as 1, that it could confuse some other poor 
>>>>>>unsuspecting module. The correct way to do get a deferred class from a 
>>>>>>third party module is to import and subclass. This works with either 2 
>>>>>>or 3::
>>>>>>
>>>>>>import A
>>>>>>
>>>>>>class Klass2(a.Klass, numarray.DeferTo):   #2
>>>>>>   #...
>>>>>>
>>>>>>class Klass3(a.Klass): #3 the good way
>>>>>>    _numarray_defer_to = True
>>>>>>   # ...
>>>>>>
>>>>>> A.Klass._numarray_defer_to = True #3 the evil way.
>>>>>>
>>>>>>Version 2 is cleaner and encourages you to do the right thing, so I'd 
>>>>>>prefer that solution.
>>>>>>
>>>>>>  
>>>>>>
>>>>>>       
>>>>>>
>>>>>>            
>>>>>>
>>>>>Good enough for me.  If no one else has any comments, then
>>>>>numarray.DeferTo is where I'll start implementing.  Tomorrow.
>>>>>
>>>>>
>>>>>     
>>>>>
>>>>>          
>>>>>
>>>>One more minor thing. I'm not sure tha DeferTo is ideal as the mix-in 
>>>>class name. It was perfect for the registration function name, but I'm 
>>>>not sure it's so clear whether the class or numarray is being deferred 
>>>>to when you say numarray.DeferTo. DeferToMe is more descriptive, but 
>>>>seems sort of slangy. DeferredTo is better than DeferTo, but still not 
>>>>as descriptive as DeferToMe. numarray.DefersTo reads perfect as long as 
>>>>numarray is included but is a disaster if you read it on your own.  
>>>>Below I've put down all the ideas I could come up with
>>>>
>>>>class CustomArray(numarray.DeferTo)
>>>>class CustomArray(numarray.DefersTo)
>>>>class CustomArray(numarray.DeferredTo)
>>>>class CustomArray(numarray.DeferToMe)
>>>>class CustomArray(numarray.DeferredToByNumarray)
>>>>class CustomArray(DeferTo)
>>>>class CustomArray(DefersTo)
>>>>class CustomArray(DeferredTo)
>>>>class CustomArray(DeferToMe)
>>>>class CustomArray(DeferredToByNumarray)
>>>>
>>>>For me it's a toss up between DefferedTo, DeferToMe and 
>>>>DeferredToByNumarray. The first is a little lacking in descriptive 
>>>>power, the second is slangy and the third is wordy.
>>>>
>>>>-tim
>>>>
>>>>[not that this matters much....]
>>>>
>>>>
>>>>   
>>>>
>>>>        
>>>>
>>>>>>regards,
>>>>>>
>>>>>>-tim
>>>>>>  
>>>>>>
>>>>>>       
>>>>>>
>>>>>>            
>>>>>>
>>>>>Thanks again,
>>>>>Todd
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>     
>>>>>
>>>>>          
>>>>>
>>>>
>>>>-------------------------------------------------------
>>>>This sf.net email is sponsored by:ThinkGeek
>>>>Welcome to geek heaven.
>>>>http://thinkgeek.com/sf
>>>>_______________________________________________
>>>>Numpy-discussion mailing list
>>>>Numpy-discussion at lists.sourceforge.net
>>>>https://lists.sourceforge.net/lists/listinfo/numpy-discussion
>>>>   
>>>>
>>>>        
>>>>
>>    
>>





From jmiller at stsci.edu  Mon Sep 29 06:09:02 2003
From: jmiller at stsci.edu (Todd Miller)
Date: Mon Sep 29 06:09:02 2003
Subject: [Numpy-discussion] numarray unfriendly to user defined types
In-Reply-To: <3F75EAD0.3030909@sympatico.ca>
References: 	<3F6A3D4C.3070703@ieee.org>		<1064266268.9234.351.camel@halloween.stsci.edu>
		 <3F6F7744.80908@ieee.org>		<3F6F7B83.3040305@colorado.edu>
		<3F6F8546.9020500@ieee.org>		<1064313211.3460.160.camel@localhost.localdoma
	i	n>		<3F721A4C.7080300@ieee.org>	<1064444385.23878.14.camel@localhost.local
	do	main>
		<3F722F0B.7070609@ieee.org>	<1064583557.13805.293.camel@halloween.stsci.edu
	> 	<3F74780D.3070701@sympatico.ca>
	<1064600606.1021.17.camel@halloween.stsci.edu> 
	<3F75EAD0.3030909@sympatico.ca>
Message-ID: <1064840600.2297.66.camel@halloween.stsci.edu>

On Sat, 2003-09-27 at 15:53, Colin J. Williams wrote:
> Tim,
> 
> Todd refers to your original post.  The only thing I've found is a 
> reference to a pre and post addition of zero, the latter was said not to 
> work.  Both work for me, see below:
> 
>  >>> zero= 0
>  >>> zero + _num.arange(5)
> array([0, 1, 2, 3, 4])
>  >>> _num.arange(5) + zero
> array([0, 1, 2, 3, 4])
> 
> If I'm barking up the wrong tree, could you point me to your original 
> posting in the archive please?
> 

It's not the wrong tree,  but there's more than one case:  objects which
look like numeric sequence work but always return numarrays,  objects
which do not look like a numeric sequence raise an exception.

Here's a link to the numpy-discussion-list archives on Source Forge:

http://sourceforge.net/mailarchive/forum.php?thread_id=3154046&forum_id=4890


> Colin W.
>  >>>
> Todd Miller wrote:
> 
> >On Fri, 2003-09-26 at 13:31, Colin J. Williams wrote:
> >  
> >
> >>Todd,
> >>
> >>This scheme seems rather complex.  The purpose isn't clear to me.
> >>
> >>What is the problem which this arrangement is intended to solve?
> >>    
> >>
> >
> >Bottom line, currently:  
> >	
> >numarray + another_class --> exception
> >
> >It would be nice if:  
> >
> >numarray + another_class --> another_class.__radd__(numarray)
> >
> >Also, look backward at Tim's original post on this subject.
> >
> >  
> >
> >>Could I suggest that some scheme along the lines of the Python PEP's be 
> >>used to set out the Why's and Wherefore's?
> >>
> >>    
> >>
> >Sounds good for next time.
> >
> >  
> >
> >>It might also be useful to copy this discussion to comp.lang.python as 
> >>that newsgroup has people experienced with classes.  
> >>    
> >>
> >
> >OK,  I'll stop being a chicken.  Everyone, get ready for the sound of a
> >pin dropping on c.l.py.
> >
> >  
> >
> >>The problem appears 
> >>to be Python related rather than a problem of numerical analysis.
> >>    
> >>
> >
> >Yes indeed.
> >
> >  
> >
> >>Colin W.
> >>
> >>Todd Miller wrote:
> >>
> >>    
> >>
> >>>I tried out DeferredToByNumArray yesterday morning and was able to get
> >>>it to work as planned with numarray and MA.  So yay!
> >>>
> >>>Yesterday afternoon,  I stopped by Perry's office for a final check, and
> >>>not surprisingly,  there are some aspects of this solution that we've
> >>>overlooked.  So not yay!
> >>>
> >>>Perry pointed out two ways that the DeferredToByNumArray scheme may fall
> >>>apart as our inheritance hierarchy becomes richer:
> >>>
> >>>1)  Imagine not one, but two independent subclasses of NumArray, each of
> >>>which want NumArray to defer to them.  Now imagine them trying to
> >>>inter-operate with each other.  In this case, the meaning of the
> >>>expression is determined by the order of the operands, so A+B and B+A
> >>>will either return an A or a B depending on the expression order.
> >>>
> >>>2)  Imagine again two subclasses of numarray, but this time imagine B as
> >>>a subclass of A.  Here again, both might want to defer to numarray, and
> >>>again, A+B and B+A return different types driven by the type order of
> >>>the expression.
> >>>
> >>>I don't have a solution yet, but am hopeful that more free candy will
> >>>fall from the sky... or YAGNI.  Perry pointed out that similar problems
> >>>exist for *any* Python class hierarchy, so we're not alone, and perhaps
> >>>should forget about this until it matters.  If someone sees an easy
> >>>fix,  now would be better than later.
> >>>
> >>>Todd
> >>>On Wed, 2003-09-24 at 19:55, Tim Hochberg wrote:
> >>> 
> >>>
> >>>      
> >>>
> >>>>Todd Miller wrote:
> >>>>
> >>>>   
> >>>>
> >>>>        
> >>>>
> >>>>>On Wed, 2003-09-24 at 18:27, Tim Hochberg wrote:
> >>>>>
> >>>>>
> >>>>>     
> >>>>>
> >>>>>          
> >>>>>
> >>>>>>Hi Todd,
> >>>>>>
> >>>>>>There are three ways to spell "defer to me" on the table (the precise 
> >>>>>>details of
> >>>>>>each spelling are, of course, still open for debate):
> >>>>>>
> >>>>>> 1.  numarray.defer_to(my_class)
> >>>>>>
> >>>>>> 2. class ArrayLike(numarray.DeferTo):
> >>>>>>       # ...
> >>>>>>
> >>>>>> 3. class ArrayLike:
> >>>>>>       _numarray_defer_to = True
> >>>>>>      # ...
> >>>>>>
> >>>>>>I'd prefer a non-registration solution since those are both 
> >>>>>>aesthetically displeasing and leave you open to the situation where a 
> >>>>>>class in module A gets registered by module B, but module C expects it 
> >>>>>>not to be registered and everything breaks. Not all that likely, I 
> >>>>>>admit, but let's avoid the registration version if we can.
> >>>>>>  
> >>>>>>
> >>>>>>       
> >>>>>>
> >>>>>>            
> >>>>>>
> >>>>>I was picturing this as module A registering it's own classes only. 
> >>>>>Nevertheless, inverting the problem and distributing the registration as
> >>>>>you suggested is better.
> >>>>>
> >>>>>
> >>>>>     
> >>>>>
> >>>>>          
> >>>>>
> >>>>The case you describe probably will describe the majority of actual use 
> >>>>cases, and in fact describes mine. I'm trying to think ahead a bit to 
> >>>>cases may encounter as start using NumArray more extensively. Let's hope 
> >>>>this solution still looks good in six months!
> >>>>
> >>>>   
> >>>>
> >>>>        
> >>>>
> >>>>>>The other two solutions are almost equivalent. The one case where 3 has 
> >>>>>>an edge over 2 is if I have an object (not a class), I could potentially 
> >>>>>>set a _numarray_defer_to on the object before passing it to numarray 
> >>>>>>without having to mess with the class
> >>>>>>of the object at all. YAGNI, though.
> >>>>>>  
> >>>>>>
> >>>>>>       
> >>>>>>
> >>>>>>            
> >>>>>>
> >>>>>I was more concerned about the potential impact of lots of multiple
> >>>>>inheritance, but that's probably just my own personal blend of FUD.
> >>>>>
> >>>>>
> >>>>>
> >>>>>     
> >>>>>
> >>>>>          
> >>>>>
> >>>>>>The advantage of 2 in my view is that it *does* force you to subclass. 
> >>>>>>With 3, there will be the temptation to poke into some other  module and 
> >>>>>>set _numarray_defer_to on some poor unsuspecting class. This has the 
> >>>>>>same disadvantage as 1, that it could confuse some other poor 
> >>>>>>unsuspecting module. The correct way to do get a deferred class from a 
> >>>>>>third party module is to import and subclass. This works with either 2 
> >>>>>>or 3::
> >>>>>>
> >>>>>>import A
> >>>>>>
> >>>>>>class Klass2(a.Klass, numarray.DeferTo):   #2
> >>>>>>   #...
> >>>>>>
> >>>>>>class Klass3(a.Klass): #3 the good way
> >>>>>>    _numarray_defer_to = True
> >>>>>>   # ...
> >>>>>>
> >>>>>> A.Klass._numarray_defer_to = True #3 the evil way.
> >>>>>>
> >>>>>>Version 2 is cleaner and encourages you to do the right thing, so I'd 
> >>>>>>prefer that solution.
> >>>>>>
> >>>>>>  
> >>>>>>
> >>>>>>       
> >>>>>>
> >>>>>>            
> >>>>>>
> >>>>>Good enough for me.  If no one else has any comments, then
> >>>>>numarray.DeferTo is where I'll start implementing.  Tomorrow.
> >>>>>
> >>>>>
> >>>>>     
> >>>>>
> >>>>>          
> >>>>>
> >>>>One more minor thing. I'm not sure tha DeferTo is ideal as the mix-in 
> >>>>class name. It was perfect for the registration function name, but I'm 
> >>>>not sure it's so clear whether the class or numarray is being deferred 
> >>>>to when you say numarray.DeferTo. DeferToMe is more descriptive, but 
> >>>>seems sort of slangy. DeferredTo is better than DeferTo, but still not 
> >>>>as descriptive as DeferToMe. numarray.DefersTo reads perfect as long as 
> >>>>numarray is included but is a disaster if you read it on your own.  
> >>>>Below I've put down all the ideas I could come up with
> >>>>
> >>>>class CustomArray(numarray.DeferTo)
> >>>>class CustomArray(numarray.DefersTo)
> >>>>class CustomArray(numarray.DeferredTo)
> >>>>class CustomArray(numarray.DeferToMe)
> >>>>class CustomArray(numarray.DeferredToByNumarray)
> >>>>class CustomArray(DeferTo)
> >>>>class CustomArray(DefersTo)
> >>>>class CustomArray(DeferredTo)
> >>>>class CustomArray(DeferToMe)
> >>>>class CustomArray(DeferredToByNumarray)
> >>>>
> >>>>For me it's a toss up between DefferedTo, DeferToMe and 
> >>>>DeferredToByNumarray. The first is a little lacking in descriptive 
> >>>>power, the second is slangy and the third is wordy.
> >>>>
> >>>>-tim
> >>>>
> >>>>[not that this matters much....]
> >>>>
> >>>>
> >>>>   
> >>>>
> >>>>        
> >>>>
> >>>>>>regards,
> >>>>>>
> >>>>>>-tim
> >>>>>>  
> >>>>>>
> >>>>>>       
> >>>>>>
> >>>>>>            
> >>>>>>
> >>>>>Thanks again,
> >>>>>Todd
> >>>>>
> >>>>>
> >>>>>
> >>>>>
> >>>>>
> >>>>>
> >>>>>
> >>>>>     
> >>>>>
> >>>>>          
> >>>>>
> >>>>
> >>>>-------------------------------------------------------
> >>>>This sf.net email is sponsored by:ThinkGeek
> >>>>Welcome to geek heaven.
> >>>>http://thinkgeek.com/sf
> >>>>_______________________________________________
> >>>>Numpy-discussion mailing list
> >>>>Numpy-discussion at lists.sourceforge.net
> >>>>https://lists.sourceforge.net/lists/listinfo/numpy-discussion
> >>>>   
> >>>>
> >>>>        
> >>>>
> >>    
> >>
> 
> 
> 
> 
> -------------------------------------------------------
> This sf.net email is sponsored by:ThinkGeek
> Welcome to geek heaven.
> http://thinkgeek.com/sf
> _______________________________________________
> Numpy-discussion mailing list
> Numpy-discussion at lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/numpy-discussion
-- 
Todd Miller 			jmiller at stsci.edu
STSCI / ESS / SSB




From jmiller at stsci.edu  Mon Sep 29 07:02:47 2003
From: jmiller at stsci.edu (Todd Miller)
Date: Mon Sep 29 07:02:47 2003
Subject: [Numpy-discussion] Crash with numarray testall.py
In-Reply-To: <3F733D14.5060200@sympatico.ca>
References: <3F733D14.5060200@sympatico.ca>
Message-ID: <1064844032.2296.262.camel@halloween.stsci.edu>

I confirmed this as a problem under Windows and Linux, and also for 
Python-2.2.3 as well as Python-2.3.1.  

It should be pointed out that this is not a problem related to
Python-2.3.1, nor does the problem occur unless you try to use numarray
from it's own directory. 

For now, self-test/use numarray like this: 

> cd C:\somewhere_else 
> python 
>>> import numarray.testall as testall 
>>> testall.test() 
.. 
>>> import numarray 
.. 

I filed it as a bug on Source Forge. 

Todd 

On Thu, 2003-09-25 at 15:08, Colin J. Williams wrote: 
> 
> C:\Program Files\Python23\Lib\site-packages\numarray>python
> Python 2.3.1 (#47, Sep 23 2003, 23:47:32) [MSC v.1200 32 bit (Intel)] on 
> win32
> Type "help", "copyright", "credits" or "license" for more information.
>  >>> import testall
> 
> The 21K file created by Windows to contain information about the crash 
> is not readable.
> 
> Windows XP and numarray 0.7 used.
> 
> Colin W.
> 
> 
> 
> -------------------------------------------------------
> This sf.net email is sponsored by:ThinkGeek
> Welcome to geek heaven.
> http://thinkgeek.com/sf
> _______________________________________________
> Numpy-discussion mailing list
> Numpy-discussion at lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/numpy-discussion
-- 
Todd Miller 			jmiller at stsci.edu
STSCI / ESS / SSB




From jochen at jochen-kuepper.de  Tue Sep 30 01:46:03 2003
From: jochen at jochen-kuepper.de (=?iso-8859-1?q?Jochen_K=FCpper?=)
Date: Tue Sep 30 01:46:03 2003
Subject: [Numpy-discussion] Re: Cygwin compilation
In-Reply-To: <1064869509.2297.320.camel@halloween.stsci.edu> (Todd Miller's
 message of "29 Sep 2003 17:05:09 -0400")
References: <3F4525A1.3040701@colorado.edu> <868yob9rfm.fsf@doze.rijnh.nl>
	<1064869509.2297.320.camel@halloween.stsci.edu>
Message-ID: <86ad8mwtqk.fsf_-_@doze.rijnh.nl>

On 29 Sep 2003 17:05:09 -0400 Todd Miller wrote:

>> _ufunc.dll needs to be linked against /usr/lib/libmingwex.a. Doing
>> that manually seems to work fine.

Todd> In the file generate.py, a clause for sys.platform == "cygwin?"
Todd> probably needs to be added. Then, define EXTRA_LINK_ARGS for
Todd> cygwin.

Ok, did that.
,----[cvs diff -u generate.py]
| Warning: No xauth data; using fake authentication data for X11 forwarding.
| Index: generate.py
| ===================================================================
| RCS file: /cvsroot/numpy/numarray/generate.py,v
| retrieving revision 1.14
| diff -u -u -r1.14 generate.py
| --- generate.py 16 Sep 2003 20:02:41 -0000      1.14
| +++ generate.py 30 Sep 2003 08:33:38 -0000
| @@ -51,6 +51,9 @@
|      LP64, HAS_UINT64, HAS_FLOAT128 = 0, 1, 0
|  elif sys.platform == "win32":
|      LP64, HAS_UINT64, HAS_FLOAT128 = 0, 0, 0
| +elif sys.platform == "cygwin":
| +    LP64, HAS_UINT64, HAS_FLOAT128 = 0, 0, 0
| +    EXTRA_LINK_ARGS += ["-L/lib/mingw -lmingwex"]
|  elif sys.platform == "darwin":
|      LP64, HAS_UINT64, HAS_FLOAT128 = 0, 1, 0
|      EXTRA_COMPILE_ARGS.extend(["-Ddarwin"])
`----

Should work, but I get the following strange behavior:
,----
| > python setup.py build
| Using EXTRA_COMPILE_ARGS = []
| running build
| running build_py
| copying Lib/numinclude.py -> build/lib.cygwin-1.5.5-i686-2.3/numarray
| running build_ext
| building 'numarray._ufunc' extension
| gcc -shared -Wl,--enable-auto-image-base -march=pentium3 -mcpu=pentium4 -O2 -Wall -pipe build/temp.cygwin-1.5.5-i686-2.3/Src/_ufuncmodule.o -L/usr/lib/python2.3/config -lpython2.3 -o build/lib.cygwin-1.5.5-i686-2.3/numarray/_ufunc.dll -L/lib/mingw -lmingwex
| build/temp.cygwin-1.5.5-i686-2.3/Src/_ufuncmodule.o(.text+0x14f): In function `_CheckFPErrors':
| /home/software/programming/numeric/numarray/Src/_ufuncmodule.c:136: undefined reference to `_fetestexcept'
| build/temp.cygwin-1.5.5-i686-2.3/Src/_ufuncmodule.o(.text+0x17b):/home/software/programming/numeric/numarray/Src/_ufuncmodule.c:142: undefined reference to `_feclearexcept'
| collect2: ld returned 1 exit status
| error: command 'gcc' failed with exit status 1
| 
| > gcc -shared -Wl,--enable-auto-image-base -march=pentium3 -mcpu=pentium4 -O2 -Wall -pipe build/temp.cygwin-1.5.5-i686-2.3/Src/_ufuncmodule.o -L/usr/lib/python2.3/config -lpython2.3 -o build/lib.cygwin-1.5.5-i686-2.3/numarray/_ufunc.dll -L/lib/mingw -lmingwex
`----

That is, it does not work when called from setup.py. When I just copy
'n' paste the link statement to the command-line, it just works.(#$%)

Any ideas??


>> Is it save to use this mingw library in "normal" Cygwin programs?

Todd> I don't know. I didn't get any farther than this, perhaps not as
Todd> far. I got the impression that mingw was a minimal library
Todd> intended for porting gcc (but it has IEEE stuff), while the
Todd> cygwin libc is intended for real programs (but has no IEEE stuff
Todd> because it is a lightweight knock off of glibc used to bypass
Todd> the GPL).

Well, as I see it Cygwin is a Unix emulation layer for Windows,
whereas MINGW is a port of gcc and some other GNU tools to Windows
(not using Cygwin at all).

But looking around using it this way should be ok. Maybe someone who
knows cygwin better can speak up here? I would carbon-copy this email
to the cygwin list, but that is subscribers only nowadays:(



>> Moreover I have the follwoing test error:
>> ,----
>> | Failure in example: log10([1+1j])
>> | from line #2267 of numarray.numtest
>> | Expected: array([ 0.150515+0.34109409j])
>> | Got: array([ 0.150515+5.21807245j])
>> `----

Todd> One of our guys found a bug in the complex log10 function; both
Todd> the expected result and the implementation were changed.

Seems to be resolved.

Greetings,
Jochen
-- 
Einigkeit und Recht und Freiheit                http://www.Jochen-Kuepper.de
    Libert?, ?galit?, Fraternit?                GnuPG key: CC1B0B4D
        (Part 3 you find in my messages before fall 2003.)




From jmiller at stsci.edu  Tue Sep 30 04:31:03 2003
From: jmiller at stsci.edu (Todd Miller)
Date: Tue Sep 30 04:31:03 2003
Subject: [Numpy-discussion] Re: Cygwin compilation
In-Reply-To: <86ad8mwtqk.fsf_-_@doze.rijnh.nl>
References: <3F4525A1.3040701@colorado.edu> <868yob9rfm.fsf@doze.rijnh.nl>
	<1064869509.2297.320.camel@halloween.stsci.edu> 
	<86ad8mwtqk.fsf_-_@doze.rijnh.nl>
Message-ID: <1064921346.16063.55.camel@halloween.stsci.edu>

On Tue, 2003-09-30 at 04:44, Jochen K?pper wrote:

> Should work, but I get the following strange behavior:
> ,----
> | > python setup.py build
> | Using EXTRA_COMPILE_ARGS = []
> | running build
> | running build_py
> | copying Lib/numinclude.py -> build/lib.cygwin-1.5.5-i686-2.3/numarray
> | running build_ext
> | building 'numarray._ufunc' extension
> | gcc -shared -Wl,--enable-auto-image-base -march=pentium3 -mcpu=pentium4 -O2 -Wall -pipe build/temp.cygwin-1.5.5-i686-2.3/Src/_ufuncmodule.o -L/usr/lib/python2.3/config -lpython2.3 -o build/lib.cygwin-1.5.5-i686-2.3/numarray/_ufunc.dll -L/lib/mingw -lmingwex
> | build/temp.cygwin-1.5.5-i686-2.3/Src/_ufuncmodule.o(.text+0x14f): In function `_CheckFPErrors':
> | /home/software/programming/numeric/numarray/Src/_ufuncmodule.c:136: undefined reference to `_fetestexcept'
> | build/temp.cygwin-1.5.5-i686-2.3/Src/_ufuncmodule.o(.text+0x17b):/home/software/programming/numeric/numarray/Src/_ufuncmodule.c:142: undefined reference to `_feclearexcept'
> | collect2: ld returned 1 exit status
> | error: command 'gcc' failed with exit status 1
> | 
> | > gcc -shared -Wl,--enable-auto-image-base -march=pentium3 -mcpu=pentium4 -O2 -Wall -pipe build/temp.cygwin-1.5.5-i686-2.3/Src/_ufuncmodule.o -L/usr/lib/python2.3/config -lpython2.3 -o build/lib.cygwin-1.5.5-i686-2.3/numarray/_ufunc.dll -L/lib/mingw -lmingwex
> `----
> 
> That is, it does not work when called from setup.py. When I just copy
> 'n' paste the link statement to the command-line, it just works.(#$%)
> 
> Any ideas??

You could try replacing gcc in order to dump the environment or perhaps
add a switch to run gcc in a more verbose mode and see if any
differences pop up.  distutils-sig?

Have you checked the #ifdef clause in _CheckFPErrors in _ufuncmodule.c
for cygwin/mingw?  Apparently I added it, and I wasn't sure if it was
valid to compile for cygwin and link against mingw.

One other thing:  HAS_UINT64 should work.

Regards,
Todd
 
-- 
Todd Miller 			jmiller at stsci.edu
STSCI / ESS / SSB




From jochen at jochen-kuepper.de  Tue Sep 30 07:00:04 2003
From: jochen at jochen-kuepper.de (=?iso-8859-1?q?Jochen_K=FCpper?=)
Date: Tue Sep 30 07:00:04 2003
Subject: [Numpy-discussion] Re: Cygwin compilation
In-Reply-To: <1064921346.16063.55.camel@halloween.stsci.edu> (Todd Miller's
 message of "30 Sep 2003 07:29:06 -0400")
References: <3F4525A1.3040701@colorado.edu> <868yob9rfm.fsf@doze.rijnh.nl>
	<1064869509.2297.320.camel@halloween.stsci.edu>
	<86ad8mwtqk.fsf_-_@doze.rijnh.nl>
	<1064921346.16063.55.camel@halloween.stsci.edu>
Message-ID: <86smmetm2t.fsf@doze.rijnh.nl>

With the current patch agianst cvs HEAD numarray compiles and passes
all tests on current Cygwin:

cvs diff -u generate.py ChangeLog
Warning: No xauth data; using fake authentication data for X11 forwarding.
Index: generate.py
===================================================================
RCS file: /cvsroot/numpy/numarray/generate.py,v
retrieving revision 1.14
diff -u -u -r1.14 generate.py
--- generate.py 16 Sep 2003 20:02:41 -0000      1.14
+++ generate.py 30 Sep 2003 13:56:25 -0000
@@ -51,6 +51,9 @@
     LP64, HAS_UINT64, HAS_FLOAT128 = 0, 1, 0
 elif sys.platform == "win32":
     LP64, HAS_UINT64, HAS_FLOAT128 = 0, 0, 0
+elif sys.platform == "cygwin":
+    LP64, HAS_UINT64, HAS_FLOAT128 = 0, 1, 0
+    EXTRA_LINK_ARGS += ["-L/lib", "-lm", "-lc", "-lgcc", "-L/lib/mingw", "-lmingwex"]
 elif sys.platform == "darwin":
     LP64, HAS_UINT64, HAS_FLOAT128 = 0, 1, 0
     EXTRA_COMPILE_ARGS.extend(["-Ddarwin"])
Index: ChangeLog
===================================================================
RCS file: /cvsroot/numpy/numarray/ChangeLog,v
retrieving revision 1.1
diff -u -u -r1.1 ChangeLog
--- ChangeLog   20 Apr 2002 16:13:38 -0000      1.1
+++ ChangeLog   30 Sep 2003 13:56:25 -0000
@@ -1,3 +1,13 @@
+2003-09-30  Jochen K?pper  <jochen at jochen-kuepper.de>
+
+       * generate.py (codeargs): Add cygwin case to link against libmingwex.
+       This is a little tricky as _ufinc.dll needs to be linked against
+       libmingwex, but libnumarray.dll normally picks up wrong symbols from
+       this library. Therefore we explicitely add libm, libc, and libgcc before
+       the mingw libraries to avoid the latter and still have them available to
+       resolve the FE symbols. We also explicitely have to add -L/lib before
+       -L/lib/mingw to check for normal cygwin libraries first.
+ 
 2002-04-20  Jochen K?pper  <jochen at jochen-kuepper.de>
 
        * MANIFEST.in: Updated.  
@@ -9,5 +19,5 @@
 
        * Local Variables:
        * mode: auto-fill
-       * fill-column: 79
+       * fill-column: 80
        * End:

Greetings,
Jochen
-- 
Einigkeit und Recht und Freiheit                http://www.Jochen-Kuepper.de
    Libert?, ?galit?, Fraternit?                GnuPG key: CC1B0B4D
        (Part 3 you find in my messages before fall 2003.)
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 185 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20030930/52dccffd/attachment-0001.sig>

From jmiller at stsci.edu  Tue Sep 30 07:13:04 2003
From: jmiller at stsci.edu (Todd Miller)
Date: Tue Sep 30 07:13:04 2003
Subject: [Numpy-discussion] Re: Cygwin compilation
In-Reply-To: <86smmetm2t.fsf@doze.rijnh.nl>
References: <3F4525A1.3040701@colorado.edu> <868yob9rfm.fsf@doze.rijnh.nl>
	 <1064869509.2297.320.camel@halloween.stsci.edu>
	 <86ad8mwtqk.fsf_-_@doze.rijnh.nl>
	 <1064921346.16063.55.camel@halloween.stsci.edu>
	 <86smmetm2t.fsf@doze.rijnh.nl>
Message-ID: <1064931107.14680.107.camel@localhost.localdomain>

Beautiful.  Thanks Jochen!


Todd

On Tue, 2003-09-30 at 09:58, Jochen K?pper wrote:
> With the current patch agianst cvs HEAD numarray compiles and passes
> all tests on current Cygwin:
> 
> cvs diff -u generate.py ChangeLog
> Warning: No xauth data; using fake authentication data for X11 forwarding.
> Index: generate.py
> ===================================================================
> RCS file: /cvsroot/numpy/numarray/generate.py,v
> retrieving revision 1.14
> diff -u -u -r1.14 generate.py
> --- generate.py 16 Sep 2003 20:02:41 -0000      1.14
> +++ generate.py 30 Sep 2003 13:56:25 -0000
> @@ -51,6 +51,9 @@
>      LP64, HAS_UINT64, HAS_FLOAT128 = 0, 1, 0
>  elif sys.platform == "win32":
>      LP64, HAS_UINT64, HAS_FLOAT128 = 0, 0, 0
> +elif sys.platform == "cygwin":
> +    LP64, HAS_UINT64, HAS_FLOAT128 = 0, 1, 0
> +    EXTRA_LINK_ARGS += ["-L/lib", "-lm", "-lc", "-lgcc", "-L/lib/mingw", "-lmingwex"]
>  elif sys.platform == "darwin":
>      LP64, HAS_UINT64, HAS_FLOAT128 = 0, 1, 0
>      EXTRA_COMPILE_ARGS.extend(["-Ddarwin"])
> Index: ChangeLog
> ===================================================================
> RCS file: /cvsroot/numpy/numarray/ChangeLog,v
> retrieving revision 1.1
> diff -u -u -r1.1 ChangeLog
> --- ChangeLog   20 Apr 2002 16:13:38 -0000      1.1
> +++ ChangeLog   30 Sep 2003 13:56:25 -0000
> @@ -1,3 +1,13 @@
> +2003-09-30  Jochen K?pper  <jochen at jochen-kuepper.de>
> +
> +       * generate.py (codeargs): Add cygwin case to link against libmingwex.
> +       This is a little tricky as _ufinc.dll needs to be linked against
> +       libmingwex, but libnumarray.dll normally picks up wrong symbols from
> +       this library. Therefore we explicitely add libm, libc, and libgcc before
> +       the mingw libraries to avoid the latter and still have them available to
> +       resolve the FE symbols. We also explicitely have to add -L/lib before
> +       -L/lib/mingw to check for normal cygwin libraries first.
> + 
>  2002-04-20  Jochen K?pper  <jochen at jochen-kuepper.de>
>  
>         * MANIFEST.in: Updated.  
> @@ -9,5 +19,5 @@
>  
>         * Local Variables:
>         * mode: auto-fill
> -       * fill-column: 79
> +       * fill-column: 80
>         * End:
> 
> Greetings,
> Jochen
-- 
Todd Miller <jmiller at stsci.edu>




From jmiller at stsci.edu  Tue Sep 30 07:55:02 2003
From: jmiller at stsci.edu (Todd Miller)
Date: Tue Sep 30 07:55:02 2003
Subject: [Numpy-discussion] numarray unfriendly to user defined types
In-Reply-To: <3F746764.2070403@ieee.org>
References: <3F6A3D4C.3070703@ieee.org>
	 <1064266268.9234.351.camel@halloween.stsci.edu> <3F6F7744.80908@ieee.org>
	 <3F6F7B83.3040305@colorado.edu>  <3F6F8546.9020500@ieee.org>
	 <1064313211.3460.160.camel@localhost.localdomain>
	 <3F721A4C.7080300@ieee.org>
	 <1064444385.23878.14.camel@localhost.localdomain>
	 <3F722F0B.7070609@ieee.org>
	 <1064583557.13805.293.camel@halloween.stsci.edu>
	 <3F746764.2070403@ieee.org>
Message-ID: <1064933606.14679.157.camel@localhost.localdomain>

On Fri, 2003-09-26 at 12:20, Tim Hochberg wrote:
> Todd Miller wrote:
> I'd make the same comment for this case as for the above case except 
> that all else being equal, subclasses should prevail over superclasses.
> 
> >I don't have a solution yet, but am hopeful that more free candy will
> >fall from the sky... or YAGNI.  Perry pointed out that similar problems
> >exist for *any* Python class hierarchy, so we're not alone, and perhaps
> >should forget about this until it matters.  If someone sees an easy
> >fix,  now would be better than later.
> >  
> >
> Here's some quick thoughts. In order for this to work out sensibly you 
> want the order between classes to be transistive and you probably also 
> always want to be able to insert a class into the order between any two 
> classes  This is equivalent to deciding the op priority based on a 
> comparison based on some real number associated with each class. This 
> leads to something like:
> 

Once again I think your ideas solve the problem, so thanks.  I have a
few things I wanted to bounce off you.

>    
> class NumArrayFamily:
>     op_priority = 0.0
>    
> # This makes it compatible with the current proposal , but isn't 
> strictly necessary.
> class DeferredToByNumarray(NumArrayFamily):
>     op_priority = 1000.0
>    
> class NumArrary(..., NumArrayFamily):
>     # ...
>     def __add__(self, object):
>         if isinstance(object, NumArrayFamily):
>             if ((object.op_priority > self.object.op_priority) or
>                 (object.op_priority == self.object.op_priority) and 
> issubclass(object, self.__class__)):
>                     return other.__radd__(self)
>         return ufunc.add(self, other)
> 
> 
> class MySubclassOfNumarray(NumArray):
>     op_priority = 10.0 # deffered to by NumArray, but defers to 
> DeferredToByNumarray classes.


As I understand it, 2.2 new style object subclass r-operators
automatically get priority (subclass.__radd__ is called in preference to
superclass.__add__).  I think this has two implications:

1) the ==/subclass term won't work, because __add__ will never get
called.

2) we have to redefine __radd__ as well to implement the priority
scheme.   I'm not sure the isinstance is necessary for __radd__,  but I
like the symmetry.

How does this look:

class DeferredToByNumArray: # name seems a little off now, but whatever
	pass

class NumArray(..., DeferredToByNumArray):
	op_priority = 0

	def __add__(self, object):
		if (isinstance(object, DeferredToByNumArray) and 
		    object.op_priority > self.op_priority):
			return object.__radd__(self)
		return ufunc.add(self, object)

	def __radd__(self, object):
		if (isinstance(object, DeferredToByNumArray) and
		    object.op_priority > self.op_priority):
			return object.__add__(self)
		return ufunc.add(object, self)

class MyForeignClass(DeferredToNumArray):
	op_priority = 1.0

class MySubclassOfNumArray(NumArray):
	op_priority = -1.0

> Of course the user still has to be careful to make sure the priority 
> order makes sense and you could have big problems when combining 
> packages with disprate ideas about priority levels, but I'm not sure how 
> you can get away from that.

As long as there's a scheme in place to resolve potential conflicts with
a little discussion,  I think your solution is already ahead of the
demand.

I told Colin last week I'd raise this issue on c.l.py.  I have decided
not to (although someone else could obviously do it if they are still
concerned about it),  because (a) the problem is solved,  (b) explaining
the problem well enough to get usable input is difficult and I've got
other stuff to do.

Thanks again for these ideas,

Todd

-- 
Todd Miller <jmiller at stsci.edu>




From Marc.Poinot at onera.fr  Tue Sep 30 09:22:01 2003
From: Marc.Poinot at onera.fr (Marc Poinot)
Date: Tue Sep 30 09:22:01 2003
Subject: [Numpy-discussion] Migrating Char arrays
Message-ID: <3F79AD6D.12732A71@onera.fr>

Hi all,

I'm migrating my pyCGNS package to numarray. I think that numarray is
better suited for the in-memory share of array I'd like to achieve.

Now, with Numeric, I use to create arrays with integers, floats, but 
also char type.

In CGNS (http://www.cgns.org) we have types I4, I8, R4 and R8, but
also C1 which is an array of chars. It looks like, in numarray, we should
consider chars as unsigned ints. This is not false from a system point
of view, but from an application point of view I have to make a difference
between *real* char arrays, and unsigned int arrays. In other words, when
I have a C1 array, I have to translate it as a String, when I have an I4
or even I8, I have to translate it as Integer array. Now I only have
integers arrays, and I have to find out a way to know that my array of int
is a string. The CGNS->numarray translation is OK, but now I'm loosing
information when I'm going back numarray->CGNS.

You can say: It's up to your application to understand that this list of ints
is a string.

I'll reply: So why don't you consider a list of bytes, it's up to the application
to know that you're 32 or 64 bits...

Is there something in numarray that could replace the Numeric interface
for this string type? Is it difficult to add a 'Character' type?
Is the (undocumented) CharArrayType the solution for this?

Marcvs [alias Not using unicode...]

-----------------------------------------------------------------------
 Marc POINOT             Alias: marcvs        Email: poinot at onera.fr
 ONERA -MFE/DSNA/ELSA    Tel: 01.46.73.42.84  Info: elsa-info at onera.fr
 29, Div. Leclerc        Fax: 01.46.73.41.66  Site:     
 92322 Chatillon FRANCE  Project: elsA        Web: http://www.onera.fr



From falted at openlc.org  Tue Sep 30 09:42:05 2003
From: falted at openlc.org (Francesc Alted)
Date: Tue Sep 30 09:42:05 2003
Subject: [Numpy-discussion] Migrating Char arrays
In-Reply-To: <3F79AD6D.12732A71@onera.fr>
References: <3F79AD6D.12732A71@onera.fr>
Message-ID: <200309301840.41624.falted@openlc.org>

Hi Marc,

A Dimarts 30 Setembre 2003 18:21, Marc Poinot va escriure:
>
> Is there something in numarray that could replace the Numeric interface
> for this string type? Is it difficult to add a 'Character' type?
> Is the (undocumented) CharArrayType the solution for this?

Yes, it is. See:

>>> from numarray import strings
>>> strings.array(["asd", "aa"])
CharArray(['asd', 'aa'])
>>> a=strings.array(["asd", "aa"])
>>> a[1]
'aa'
>>> a.raw()[1]   # If you want to get *all* spaces and nulls in strings
'aa '

This is undocumented in the manual, but it's quite well described in the doc
strings:

>>> help(strings.array)

Cheers,

-- 
Francesc Alted