From list at jsaul.de  Mon Dec  2 03:02:17 2002
From: list at jsaul.de (Joachim Saul)
Date: Mon Dec  2 03:02:17 2002
Subject: [Numpy-discussion] Accessing array members
Message-ID: <20021202105922.GA870@jsaul.de>

Hi there,

given the following (simplified) scenario:

typedef struct {
        PyObject_HEAD
        float bar[10];
} FooObject;

I want to be able to set and retrieve the elements of bar from
Python using e.g.

>>> foo = Foo()
>>> foo.bar[4] = 1.23
>>> x = foo.bar[4]

I have chosen an approach using 'PyArray_FromDimsAndData'. In fact
I programmed it after studying 'arrayobject.c', namely the part in
'array_getattr' where the 'flat' attribute is accessed.

foo_getattr(FooObject *self, char *name)
{
        if (!strcmp(name, "bar")) {
                int n=10;
                PyObject *bar = PyArray_FromDimsAndData(1, &n,
                                    PyArray_FLOAT, (char*)self->bar);
                if (bar == NULL) return NULL;
                return bar;
        }

        return Py_FindMethod(foo_methods, (PyObject*)self, name);
}

And it works! :-)

BUT how about refcounts here? 'PyArray_FromDimsAndData' will
return an array which only contains a reference to foo's original
bar array; that's why I can both set and access the latter the way
described. And no memory leak is created.

But what if I create a reference to foo.bar, and later delete foo,
i.e.

>>> b = foo.bar
>>> del foo

Now the data pointer in b refers to freed data! In the mentioned
'array_getattr' this apeears to be solved by increasing the
refcount; in the above example this would mean 'Py_INCREF(self)'
before returning 'bar'. Then if deleting 'foo', its memory is not
freed because the refcount is not zero. But AFAICS in this case
(as well as in the Numeric code) the INCREF prevents the object
from EVER being freed. Who would DECREF the object?

Or am I misunderstanding something here?

In my actual code I can perfectly live with the above solution
because I only need to access foo's data using 'foo.bar[i]' and
probably never need to create a reference to 'bar' which might
survive the actual 'foo' object. However, I want to program it the
'clean' way; any hints on how to do it properly would therefore be
highly welcome.

Cheers,
Joachim



From hinsen at cnrs-orleans.fr  Mon Dec  2 03:42:12 2002
From: hinsen at cnrs-orleans.fr (Konrad Hinsen)
Date: Mon Dec  2 03:42:12 2002
Subject: [Numpy-discussion] Accessing array members
In-Reply-To: <20021202105922.GA870@jsaul.de>
References: <20021202105922.GA870@jsaul.de>
Message-ID: <m3fztgps7m.fsf@chinon.cnrs-orleans.fr>

Joachim Saul <list at jsaul.de> writes:

> But what if I create a reference to foo.bar, and later delete foo,
> i.e.
> 
> >>> b = foo.bar
> >>> del foo
> 
> Now the data pointer in b refers to freed data! In the mentioned

And that is why the condition for using PyArray_FromDimsAndData is
that the data space passed is not freed before the end of the process.

> survive the actual 'foo' object. However, I want to program it the
> 'clean' way; any hints on how to do it properly would therefore be
> highly welcome.

I see only one clean solution: implement your own array-like object
that represents foo.bar. This object would keep a reference to foo
and release it when it is itself destroyed.

Konrad.
-- 
-------------------------------------------------------------------------------
Konrad Hinsen                            | E-Mail: hinsen at cnrs-orleans.fr
Centre de Biophysique Moleculaire (CNRS) | Tel.: +33-2.38.25.56.24
Rue Charles Sadron                       | Fax:  +33-2.38.63.15.17
45071 Orleans Cedex 2                    | Deutsch/Esperanto/English/
France                                   | Nederlands/Francais
-------------------------------------------------------------------------------



From verveer at embl-heidelberg.de  Mon Dec  2 15:28:03 2002
From: verveer at embl-heidelberg.de (verveer at embl-heidelberg.de)
Date: Mon Dec  2 15:28:03 2002
Subject: [Numpy-discussion] coercion of arrays
Message-ID: <1038871565.3debec0d23849@webmail.EMBL-Heidelberg.DE>

Hi,

I was looking at the coercion of arrays in the new numarray. The following is 
not intuitive to me:

>>> a = array(1, Int8)
>>> b = array(1, UInt16)
>>> c = a + b
>>> c.type()
UInt16

Should the result not be a signed integer type?

Also the following result is strange to me:

>>> a = array(1, Float64)
>>> b = array(1, Complex32)
>>> c = a + b
>>> c.type()
Complex32

Complex64 would seem to be the more apropiate type here for the result.

Could somebody comment if these are bugs or not?

Cheers, Peter

--
Dr. Peter J. Verveer
Cell Biology and Cell Biophysics Programme
European Molecular Biology Laboratory
Meyerhofstrasse 1
D-69117 Heidelberg
Germany
Tel. : +49 6221 387245
Fax  : +49 6221 387242
Email: Peter.Verveer at embl-heidelberg.de



 



From oliphant at ee.byu.edu  Mon Dec  2 15:33:01 2002
From: oliphant at ee.byu.edu (Travis Oliphant)
Date: Mon Dec  2 15:33:01 2002
Subject: [Numpy-discussion] Accessing array members
In-Reply-To: <m3fztgps7m.fsf@chinon.cnrs-orleans.fr>
Message-ID: <Pine.LNX.4.33L2.0212021627410.21184-100000@oliphant.ee.byu.edu>

On 2 Dec 2002, Konrad Hinsen wrote:

> Joachim Saul <list at jsaul.de> writes:
>
> > But what if I create a reference to foo.bar, and later delete foo,
> > i.e.
> >
> > >>> b = foo.bar
> > >>> del foo

> >
> > Now the data pointer in b refers to freed data! In the mentioned

Forgive me for jumping in.  But why should the data be deleted when you do
this?  Shouldn't del foo merely decrease the reference count of foo.bar?
Because there are still outstanding references to foo.bar (i.e. b)  then
the data itself shouldn't be freed.

Perhaps I don't understand the question well enough.

-Travis





From perry at stsci.edu  Mon Dec  2 17:21:13 2002
From: perry at stsci.edu (Perry Greenfield)
Date: Mon Dec  2 17:21:13 2002
Subject: [Numpy-discussion] coercion of arrays
In-Reply-To: <1038871565.3debec0d23849@webmail.EMBL-Heidelberg.DE>
Message-ID: <NEBBIJKBMLDBLNCEEFOCOEOHCOAA.perry@stsci.edu>

You are right on both counts. These are indeed coercion
bugs and we will fix them. The first should end up being an
Int32 actually.

Thanks for pointing this out.

Perry Greenfield
> 
> I was looking at the coercion of arrays in the new numarray. The 
> following is 
> not intuitive to me:
> 
> >>> a = array(1, Int8)
> >>> b = array(1, UInt16)
> >>> c = a + b
> >>> c.type()
> UInt16
> 
> Should the result not be a signed integer type?
> 
> Also the following result is strange to me:
> 
> >>> a = array(1, Float64)
> >>> b = array(1, Complex32)
> >>> c = a + b
> >>> c.type()
> Complex32
> 
> Complex64 would seem to be the more apropiate type here for the result.
> 
> Could somebody comment if these are bugs or not?
> 
> Cheers, Peter
> 
> --
> Dr. Peter J. Verveer
> Cell Biology and Cell Biophysics Programme
> European Molecular Biology Laboratory
> Meyerhofstrasse 1
> D-69117 Heidelberg
> Germany
> Tel. : +49 6221 387245
> Fax  : +49 6221 387242
> Email: Peter.Verveer at embl-heidelberg.de
> 




From cookedm at arbutus.physics.mcmaster.ca  Mon Dec  2 17:47:01 2002
From: cookedm at arbutus.physics.mcmaster.ca (David M. Cooke)
Date: Mon Dec  2 17:47:01 2002
Subject: [Numpy-discussion] Accessing array members
In-Reply-To: <20021202105922.GA870@jsaul.de>
References: <20021202105922.GA870@jsaul.de>
Message-ID: <20021203014206.GA3009@arbutus.physics.mcmaster.ca>

On Mon, Dec 02, 2002 at 11:59:22AM +0100, Joachim Saul wrote:
> Hi there,
> 
> given the following (simplified) scenario:
> 
> typedef struct {
>         PyObject_HEAD
>         float bar[10];
> } FooObject;
> 
> I want to be able to set and retrieve the elements of bar from
> Python using e.g.
> 
> >>> foo = Foo()
> >>> foo.bar[4] = 1.23
> >>> x = foo.bar[4]
> 
> I have chosen an approach using 'PyArray_FromDimsAndData'. In fact
> I programmed it after studying 'arrayobject.c', namely the part in
> 'array_getattr' where the 'flat' attribute is accessed.
> 
> foo_getattr(FooObject *self, char *name)
> {
>         if (!strcmp(name, "bar")) {
>                 int n=10;
>                 PyObject *bar = PyArray_FromDimsAndData(1, &n,
>                                     PyArray_FLOAT, (char*)self->bar);
>                 if (bar == NULL) return NULL;
>                 return bar;
>         }
> 
>         return Py_FindMethod(foo_methods, (PyObject*)self, name);
> }
> 
> And it works! :-)
> 
> BUT how about refcounts here? 'PyArray_FromDimsAndData' will
> return an array which only contains a reference to foo's original
> bar array; that's why I can both set and access the latter the way
> described. And no memory leak is created.
> 
> But what if I create a reference to foo.bar, and later delete foo,
> i.e.
> 
> >>> b = foo.bar
> >>> del foo
> 
> Now the data pointer in b refers to freed data! In the mentioned
> 'array_getattr' this apeears to be solved by increasing the
> refcount; in the above example this would mean 'Py_INCREF(self)'
> before returning 'bar'. Then if deleting 'foo', its memory is not
> freed because the refcount is not zero. But AFAICS in this case
> (as well as in the Numeric code) the INCREF prevents the object
> from EVER being freed. Who would DECREF the object?

Something similiar came up a few weeks ago: how do you pass data owned
by something else as a Numeric array, while keeping track of when to
delete the data?

It's so simple I almost kicked myself when I saw it, from the code at
http://pobox.com/~kragen/sw/arrayfrombuffer/
which allows you to use memory-mapped files as arrays.

The idea is that a PyArrayObject has a member 'base', which is DECREF'd
when the array is deallocated. The idea is for when arrays are slices of
other arrays, deallocating the slice will decrease the reference count
of the original. However, we can subvert this by using our own base,
that knows how to deallocate our data.

In your case, the DECREF'ing is all you need, so you could use

foo_getattr(FooObject *self, char *name)
{
        if (!strcmp(name, "bar")) {
                int n=10;
                PyObject *bar = PyArray_FromDimsAndData(1, &n,
                                    PyArray_FLOAT, (char*)self->bar);
                if (bar == NULL) return NULL;
                /***** new stuff here *******/
                Py_INCREF(self);
                ((PyArrayObject *)bar)->base = self;
                /***********/
                return bar;
        }

        return Py_FindMethod(foo_methods, (PyObject*)self, name);
}

So, now with
>>> b = foo.bar
>>> del foo
b will still reference the original foo object. Now, do
>>> del b
and foo's data will be DECREF'd, freeing it if b had the only reference.

This can be extended: say you've allocated memory from some memory pool
that has to be freed with, say, 'my_pool_free'. You can create a Numeric
array from this without copying by

PyArrayObject *A = (PyArrayObject *)PyArray_FromDimsAndData(1, dims,
                                      PyArray_DOUBLE, (char *)data);
A->base = PyCObject_FromVoidPtr(data, my_pool_free);

Then A will be a PyArrayObject, that, when the last reference is
deleted, will DECREF A->base, which will free the memory.

Easy, huh?

-- 
|>|\/|<
/--------------------------------------------------------------------------\
|David M. Cooke                 http://arbutus.physics.mcmaster.ca/cookedm/
|cookedm at physics.mcmaster.ca



From hinsen at cnrs-orleans.fr  Tue Dec  3 02:07:06 2002
From: hinsen at cnrs-orleans.fr (Konrad Hinsen)
Date: Tue Dec  3 02:07:06 2002
Subject: [Numpy-discussion] Accessing array members
In-Reply-To: <20021203014206.GA3009@arbutus.physics.mcmaster.ca>
References: <20021202105922.GA870@jsaul.de>
	<20021203014206.GA3009@arbutus.physics.mcmaster.ca>
Message-ID: <m3isyba07r.fsf@chinon.cnrs-orleans.fr>

cookedm at arbutus.physics.mcmaster.ca (David M. Cooke) writes:

> The idea is that a PyArrayObject has a member 'base', which is DECREF'd
> when the array is deallocated. The idea is for when arrays are slices of

Indeed, but this is an undocumented implementation feature. Use at your
own risk.

Konrad.
-- 
-------------------------------------------------------------------------------
Konrad Hinsen                            | E-Mail: hinsen at cnrs-orleans.fr
Centre de Biophysique Moleculaire (CNRS) | Tel.: +33-2.38.25.56.24
Rue Charles Sadron                       | Fax:  +33-2.38.63.15.17
45071 Orleans Cedex 2                    | Deutsch/Esperanto/English/
France                                   | Nederlands/Francais
-------------------------------------------------------------------------------



From haase at msg.ucsf.edu  Tue Dec  3 14:35:02 2002
From: haase at msg.ucsf.edu (Sebastian Haase)
Date: Tue Dec  3 14:35:02 2002
Subject: [Numpy-discussion] howto build numarray 0.4 ? file 'Src/_convmodule.c' does not exist
Message-ID: <030301c29b1c$16db8530$3b45da80@rodan>

Hi all,
I downloaded numarray 0.4 about 5 minutes after I got the announce but 
my naive 'python2.2 ./setup.py build'   gives this

haase at baboon:~/numarray-0.4: python2.2 ./setup.py build
running build
running build_py
not copying Lib/ndarray.py (output up-to-date)
not copying Lib/numtest.py (output up-to-date)
not copying Lib/codegenerator.py (output up-to-date)
not copying Lib/ufunc.py (output up-to-date)
not copying Lib/testdata.py (output up-to-date)
not copying Lib/numarray.py (output up-to-date)
not copying Lib/ieeespecial.py (output up-to-date)
not copying Lib/recarray.py (output up-to-date)
not copying Lib/template.py (output up-to-date)
not copying Lib/arrayprint.py (output up-to-date)
not copying Lib/typeconv.py (output up-to-date)
not copying Lib/numinclude.py (output up-to-date)
not copying Lib/numarrayext.py (output up-to-date)
not copying Lib/_ufunc.py (output up-to-date)
not copying Lib/chararray.py (output up-to-date)
not copying Lib/numtestall.py (output up-to-date)
not copying Lib/numerictypes.py (output up-to-date)
not copying Lib/memmap.py (output up-to-date)
running build_ext
building '_conv' extension
error: file 'Src/_convmodule.c' does not exist

What am I missing? I'm running Linux (debian woody) on i386.

Thanks,
Sebastian





From cookedm at arbutus.physics.mcmaster.ca  Tue Dec  3 14:40:02 2002
From: cookedm at arbutus.physics.mcmaster.ca (David M. Cooke)
Date: Tue Dec  3 14:40:02 2002
Subject: [Numpy-discussion] Accessing array members
In-Reply-To: <m3isyba07r.fsf@chinon.cnrs-orleans.fr>
References: <20021202105922.GA870@jsaul.de> <20021203014206.GA3009@arbutus.physics.mcmaster.ca> <m3isyba07r.fsf@chinon.cnrs-orleans.fr>
Message-ID: <20021203223516.GA22325@arbutus.physics.mcmaster.ca>

On Tue, Dec 03, 2002 at 11:03:20AM +0100, Konrad Hinsen wrote:
> cookedm at arbutus.physics.mcmaster.ca (David M. Cooke) writes:
> 
> > The idea is that a PyArrayObject has a member 'base', which is DECREF'd
> > when the array is deallocated. The idea is for when arrays are slices of
> 
> Indeed, but this is an undocumented implementation feature. Use at your
> own risk.

Nope, documented implementation feature. From the C API documentation,

  PyObject  * base

  Used internally in arrays that are created as slices of other arrays.
  Since the new array shares its data area with the old one, the original
  array's reference count is incremented. When the subarray is garbage
  collected, the base array's reference count is decremented.

Looking through Numeric's code, nothing requires base to be an array
object.  Besides, Numeric isn't going to change substantially before
Numarray replaces it (although I don't know the analogue of this trick
in Numarray).

The usefulness of this trick (IMHO) outweighs the small chance of the
interface changing.

-- 
|>|\/|<
/--------------------------------------------------------------------------\
|David M. Cooke                 http://arbutus.physics.mcmaster.ca/cookedm/
|cookedm at physics.mcmaster.ca



From cookedm at arbutus.physics.mcmaster.ca  Tue Dec  3 14:43:05 2002
From: cookedm at arbutus.physics.mcmaster.ca (David M. Cooke)
Date: Tue Dec  3 14:43:05 2002
Subject: [Numpy-discussion] howto build numarray 0.4 ? file 'Src/_convmodule.c' does not exist
In-Reply-To: <030301c29b1c$16db8530$3b45da80@rodan>
References: <030301c29b1c$16db8530$3b45da80@rodan>
Message-ID: <20021203223837.GB22325@arbutus.physics.mcmaster.ca>

On Tue, Dec 03, 2002 at 02:34:06PM -0800, Sebastian Haase wrote:
> Hi all,
> I downloaded numarray 0.4 about 5 minutes after I got the announce but 
> my naive 'python2.2 ./setup.py build'   gives this
> 
> haase at baboon:~/numarray-0.4: python2.2 ./setup.py build
> running build
> running build_py
> not copying Lib/ndarray.py (output up-to-date)
[...]
> not copying Lib/memmap.py (output up-to-date)
> running build_ext
> building '_conv' extension
> error: file 'Src/_convmodule.c' does not exist
> 
> What am I missing? I'm running Linux (debian woody) on i386.

Looks like you have to run 'python2.2 ./setup.py install' instead. Looking at
setup.py, something special is done when the target is 'install'.

[I think this is a bad idea, as I like to build stuff as my user, and
install as root. This requires me to build it as root.]

-- 
|>|\/|<
/--------------------------------------------------------------------------\
|David M. Cooke                 http://arbutus.physics.mcmaster.ca/cookedm/
|cookedm at physics.mcmaster.ca



From haase at msg.ucsf.edu  Tue Dec  3 15:09:04 2002
From: haase at msg.ucsf.edu (Sebastian Haase)
Date: Tue Dec  3 15:09:04 2002
Subject: [Numpy-discussion] howto build numarray 0.4 ? file 'Src/_convmodule.c' does not exist
References: <030301c29b1c$16db8530$3b45da80@rodan> <20021203223837.GB22325@arbutus.physics.mcmaster.ca>
Message-ID: <032c01c29b20$d6b79750$3b45da80@rodan>

> On Tue, Dec 03, 2002 at 02:34:06PM -0800, Sebastian Haase wrote:
> > Hi all,
> > I downloaded numarray 0.4 about 5 minutes after I got the announce but
> > my naive 'python2.2 ./setup.py build'   gives this
> >
> > haase at baboon:~/numarray-0.4: python2.2 ./setup.py build
> > running build
> > running build_py
> > not copying Lib/ndarray.py (output up-to-date)
> [...]
> > not copying Lib/memmap.py (output up-to-date)
> > running build_ext
> > building '_conv' extension
> > error: file 'Src/_convmodule.c' does not exist
> >
> > What am I missing? I'm running Linux (debian woody) on i386.
>
> Looks like you have to run 'python2.2 ./setup.py install' instead. Looking
at
> setup.py, something special is done when the target is 'install'.
>
> [I think this is a bad idea, as I like to build stuff as my user, and
> install as root. This requires me to build it as root.]
>

Thanks for the hint - I'll try that.
But I would consider that a bug, right?

Sebastian





From haase at msg.ucsf.edu  Tue Dec  3 15:14:08 2002
From: haase at msg.ucsf.edu (Sebastian Haase)
Date: Tue Dec  3 15:14:08 2002
Subject: [Numpy-discussion] howto build numarray 0.4 ? file 'Src/_convmodule.c' does not exist
References: <030301c29b1c$16db8530$3b45da80@rodan> <20021203223837.GB22325@arbutus.physics.mcmaster.ca>
Message-ID: <033f01c29b21$9b610cd0$3b45da80@rodan>

> On Tue, Dec 03, 2002 at 02:34:06PM -0800, Sebastian Haase wrote:
> > Hi all,
> > I downloaded numarray 0.4 about 5 minutes after I got the announce but
> > my naive 'python2.2 ./setup.py build'   gives this
> >
> > haase at baboon:~/numarray-0.4: python2.2 ./setup.py build
> > running build
> > running build_py
> > not copying Lib/ndarray.py (output up-to-date)
> [...]
> > not copying Lib/memmap.py (output up-to-date)
> > running build_ext
> > building '_conv' extension
> > error: file 'Src/_convmodule.c' does not exist
> >
> > What am I missing? I'm running Linux (debian woody) on i386.
>
> Looks like you have to run 'python2.2 ./setup.py install' instead. Looking
at
> setup.py, something special is done when the target is 'install'.
>
> [I think this is a bad idea, as I like to build stuff as my user, and
> install as root. This requires me to build it as root.]

I just tried it (actually as user not root!!) and it runs through up to a
"cannot create directory..." error.
That's probably fine with me (for now) and I'm just setting PYTHONPATH ...

Thanks again, Sebastian.






From paul at pfdubois.com  Thu Dec  5 17:03:18 2002
From: paul at pfdubois.com (Paul F Dubois)
Date: Thu Dec  5 17:03:18 2002
Subject: [Numpy-discussion] PEP 242 Numeric kinds
Message-ID: <000001c29cc3$281ed420$6501a8c0@NICKLEBY>

We had put off consideration of PEP 242 Numeric kinds until June 2002
when a meeting of some interested parties was to occur but the meeting
didn't occur. I have a draft implementation and users of it but my
feeling is that although correct and useful the PEP is not useful enough
to put it in the standard library. Since it really only interests
scientific programmers I propose simply making it a separate deliverable
on the Numeric site and withdrawing the PEP.




From j_r_fonseca at yahoo.co.uk  Thu Dec  5 18:38:09 2002
From: j_r_fonseca at yahoo.co.uk (=?iso-8859-15?Q?Jos=E9?= Fonseca)
Date: Thu Dec  5 18:38:09 2002
Subject: [Numpy-discussion] PEP 242 Numeric kinds
In-Reply-To: <000001c29cc3$281ed420$6501a8c0@NICKLEBY>
References: <000001c29cc3$281ed420$6501a8c0@NICKLEBY>
Message-ID: <20021206023728.GA24182@localhost.localdomain>

On Thu, Dec 05, 2002 at 05:02:26PM -0800, Paul F Dubois wrote:
> Since it really only interests
> scientific programmers I propose simply making it a separate deliverable
> on the Numeric site and withdrawing the PEP.

I'm not fully aware about the advantages/implications of puting Numeric
in Python standard library, but to say that the manipulation of numeric
arrays is only of interest to scientific programmers is the same of when
in the early computing days engineers saing that computers would only be
good for crunching numbers, and that the concept of personal computers
was just non-sense...

For a non-scientic usage of Numeric see the examples in
http://www.pygame.org/pcr/repository.php, but I can imagine the
usefullness of Numeric in many more non-scientific applications:
imaging, sound visualization plugins, 3D graphics, and probably much
more.

The use of Numeric can speed up alot of algorithms which would be
otherwise very slow in pure Python, and therefore forcing one to write C
extensions. That's why IMHO something with the Numeric functionality
should exist in the Python standard library.

Note that this is not the same that saying that Numeric should be
included as is - perhaps it's better to have it seperately to let it
mature more, perhaps not - but still, there is much more than a niche
interest around it.

Jos? Fonseca
__________________________________________________
Do You Yahoo!?
Everything you'll ever need on one web page
from News and Sport to Email and Music Charts
http://uk.my.yahoo.com



From python at rcn.com  Thu Dec  5 22:57:03 2002
From: python at rcn.com (Raymond Hettinger)
Date: Thu Dec  5 22:57:03 2002
Subject: [Numpy-discussion] Re: [Python-Dev] PEP 242 Numeric kinds
References: <000001c29cc3$281ed420$6501a8c0@NICKLEBY>
Message-ID: <002701c29cf4$98296c40$770ca044@oemcomputer>

> I have a draft implementation and users of it but my
> feeling is that although correct and useful the PEP is not useful enough
> to put it in the standard library. Since it really only interests
> scientific programmers I propose simply making it a separate deliverable
> on the Numeric site and withdrawing the PEP.

As the PEP author, your feeling is probably the most accurate.
And, it is true that Kinds will, in general, appeal to the same crowd 
as Numeric.

OTOH, "import kinds" is entirely unobtrusive and general purpose.
The interface is simple; and may help some code become less platform
sensitive (in the way the Metafont writes its own portable routines to 
create consistent lettering); and it has some educational value (making
it easy to demonstrate the effects of truncation and floating point
round-off).  

Also, I think accounting applications can benefit from having some
positive assurance that multi-billion dollar calculations don't lose 
their pennies and that the books will still balance.

So, the only downside is having an additional block of code to 
maintain.  My vote is for adding it to the standard library.  Control
over precision is more of a core capability than it is an extension,
but, as the PEP author and principal user, you probably know best.

'nuff said,


Raymond Hettinger


#################################################################
#################################################################
#################################################################
#####
#####
#####
#################################################################
#################################################################
#################################################################



From bac at OCF.Berkeley.EDU  Thu Dec  5 23:13:04 2002
From: bac at OCF.Berkeley.EDU (Brett Cannon)
Date: Thu Dec  5 23:13:04 2002
Subject: [Numpy-discussion] Re: [Python-Dev] PEP 242 Numeric kinds
In-Reply-To: <000001c29cc3$281ed420$6501a8c0@NICKLEBY>
References: <000001c29cc3$281ed420$6501a8c0@NICKLEBY>
Message-ID: <Pine.SOL.4.50.0212052304510.28199-100000@death.OCF.Berkeley.EDU>

[Paul F Dubois]

> We had put off consideration of PEP 242 Numeric kinds until June 2002
> when a meeting of some interested parties was to occur but the meeting
> didn't occur. I have a draft implementation and users of it but my
> feeling is that although correct and useful the PEP is not useful enough
> to put it in the standard library. Since it really only interests
> scientific programmers I propose simply making it a separate deliverable
> on the Numeric site and withdrawing the PEP.
>

I say add it.  If there are already users out there then that demonstrates
as least some form of a demand.

Besides, if rationals can get into the library (that module was finally
accepted to be added to the library, right?) then a module to help do
consistent decimal math should at least be included at least until the
Python core can move to C99 (if I am remembering a comment from Tim saying
that C99 adds more explicit numeric types, e.g., exactly 2 byte float).

I just want to be able to add up a bunch of money values and not have a
few pennies disappear in the end; usually I am the one who has to eat the
loss and I am an poor college graduate who can't afford the rounding.  =)

-Brett



From mclay at nist.gov  Fri Dec  6 08:40:01 2002
From: mclay at nist.gov (Michael McLay)
Date: Fri Dec  6 08:40:01 2002
Subject: [Numpy-discussion] Re: [Python-Dev] PEP 242 Numeric kinds
In-Reply-To: <Pine.SOL.4.50.0212052304510.28199-100000@death.OCF.Berkeley.EDU>
References: <000001c29cc3$281ed420$6501a8c0@NICKLEBY> <Pine.SOL.4.50.0212052304510.28199-100000@death.OCF.Berkeley.EDU>
Message-ID: <200212061138.30065.mclay@nist.gov>

On Friday 06 December 2002 02:12 am, Brett Cannon wrote:
> [Paul F Dubois]
>
> > We had put off consideration of PEP 242 Numeric kinds until June 2002
> > when a meeting of some interested parties was to occur but the meeting
> > didn't occur. 
[...] 
> I just want to be able to add up a bunch of money values and not have a
> few pennies disappear in the end; usually I am the one who has to eat the
> loss and I am an poor college graduate who can't afford the rounding.  =)

Does the kinds implementation work with binary floats and decimal numbers? 
>From a quick scan of the PEP it looks like it is only binary floats. If it 
only applies to binary numbers then the kinds capability will not eliminate 
decimal number rounding errors. That problem will require extending Python to 
include a decimal number type. 

The abstract mentions a clarification of decimal literals, but I suspect Paul 
was not suggesting that the numbers defined by kinds will use decimal 
arithmetic instead of binary arithmetic.



From Chris.Barker at noaa.gov  Fri Dec  6 10:10:04 2002
From: Chris.Barker at noaa.gov (Chris Barker)
Date: Fri Dec  6 10:10:04 2002
Subject: [Numpy-discussion] PEP 242 Numeric kinds
References: <000001c29cc3$281ed420$6501a8c0@NICKLEBY> <20021206023728.GA24182@localhost.localdomain>
Message-ID: <3DF0DF06.835F72A3@noaa.gov>

Jos? Fonseca wrote:
> to say that the manipulation of numeric
> arrays is only of interest to scientific programmers is the same of when
> in the early computing days engineers saing that computers would only be
> good for crunching numbers, and that the concept of personal computers
> was just non-sense...

I absolutely concur.
 
> For a non-scientic usage of Numeric see the examples in
> http://www.pygame.org/pcr/repository.php, but I can imagine the
> usefullness of Numeric in many more non-scientific applications:
> imaging, sound visualization plugins, 3D graphics, and probably much
> more.

It goes MUCH farther than this. All these examples are what I would call
serious number crunching. Perhaps not strictly scientific, but certainly
the realm of numeric programming, and all places where the "kinds"
concept would be useful.

However, there is a great deal of usefulness in Numeric for applications
that are not doing a lot of number crunching. Performance is only one
reason to use Numeric. The other reason is much cleaner and easier
syntax for manipulating arrays of numbers, especially ones of more than
one dimension. Being able to slice across multiple dimensions, array
oriented arithmetic, overloaded comparisons, etc. I use Numeric for
virtually every program I write, whether performance is an issue or not.
Not having to write all those ugly, confusing and error prone loops is a
godsend.

All that being said, the "kinds" concept is probably mostly of concern
to the Number crunching community. However, it is also sort of a low
level concept, and having it part of core language makes sense, just
like it makes sense to have complex number and long integers in the core
language.

-Chris



-- 
Christopher Barker, Ph.D.
Oceanographer
                                    		
NOAA/OR&R/HAZMAT         (206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115       (206) 526-6317   main reception

Chris.Barker at noaa.gov



From rob at pythonemproject.com  Fri Dec  6 15:15:02 2002
From: rob at pythonemproject.com (Rob)
Date: Fri Dec  6 15:15:02 2002
Subject: [Numpy-discussion] Numeric Python EM Project supposed to be in Dec IEEE A & P Magazine
Message-ID: <3DF12E52.5E952B17@pythonemproject.com>

It should be in the educational corner.  I've beefed up the intro to the
site.  Will be interesting to see if I get many new hits.  Rob.
-- 
-----------------------------
The Numeric Python EM Project

www.pythonemproject.com



From rob at pythonemproject.com  Sat Dec  7 08:40:04 2002
From: rob at pythonemproject.com (Rob)
Date: Sat Dec  7 08:40:04 2002
Subject: [Numpy-discussion] Numeric Python EM Project supposed to be in Dec 
 IEEE A & P Magazine
References: <3DF12E52.5E952B17@pythonemproject.com>
Message-ID: <3DF22343.2776D978@pythonemproject.com>

Rob wrote:
> 
> It should be in the educational corner.  I've beefed up the intro to the
> site.  Will be interesting to see if I get many new hits.  Rob.
> --
> -----------------------------
> The Numeric Python EM Project
> 
> www.pythonemproject.com
> 
> -------------------------------------------------------
> This sf.net email is sponsored by:ThinkGeek
> Welcome to geek heaven.
> http://thinkgeek.com/sf
> _______________________________________________
> Numpy-discussion mailing list
> Numpy-discussion at lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/numpy-discussion

It was supposed to be in A & P magazine all the way back in June, but
the editors goofed up, and profusely apologized :)  Rob.

-- 
-----------------------------
The Numeric Python EM Project

www.pythonemproject.com



From pearu at cens.ioc.ee  Sun Dec  8 09:24:01 2002
From: pearu at cens.ioc.ee (Pearu Peterson)
Date: Sun Dec  8 09:24:01 2002
Subject: [Numpy-discussion] ANN: F2PY - Fortran to Python Interface Generator - 6th Release
Message-ID: <Pine.LNX.4.21.0212081921200.20312-100000@cens.kybi>

F2PY - Fortran to Python Interface Generator, Version 2.32.225
---------------------------------------------------------------

I am pleased to announce the sixth public release of F2PY. The purpose of
the F2PY project is to provide a connection between Python and Fortran
programming languages. For more information, see

  http://cens.ioc.ee/projects/f2py2e/

Download source:

  http://cens.ioc.ee/projects/f2py2e/2.x/F2PY-2-latest.tar.gz

What's new?
------------

It has been almost a year from the previous release of F2PY. This release
comes with a number of improvements, most important ones are listed as
follows:

- The issue with a Fortran and C multi-dimensional array storage ordering
  is finally resolved. F2PY generated wrappers automatically carry out all
  necessary transformations, trying hard to minimize any performance hits.
  As a result, the end users of F2PY generated wrappers need not to worry
  about this issue anymore, multi-dimensional arrays in Python and Fortran
  have now exactly the same signatures.

- Improved wrapping C functions with F2PY.

- F2PY Users Guide has been throughly revised to describe and illustrate
  all latest F2PY features such as wrapping Fortran 77 COMMON blocks,
  Fortran 90 module data, including Fortran 90 module ALLOCATABLE arrays,
  etc. The F2PY Users Guide is composed using the Python Docutils tool and
  is available here:

    http://cens.ioc.ee/projects/f2py2e/usersguide/

- F2PY has new site for unit testing.

- F2PY uses scipy_distutils from SciPy (www.scipy.org) project for
  compiling Fortran sources and building extension modules.
  Currently, the following Fortran 77/90 compilers are described by
  scipy_distutils:

    Absoft Sun SGI Intel Itanium NAG Compaq Digital Gnu VAST

- Finally, F2PY has been extensively used/tested for wrapping large
  Fortran/C libraries, such as, LAPACK, BLAS, ATLAS, FFTW, FFTPACK,
  etc. (see SciPy project for more information). This experience has been
  a very important source for ideas how to make binding Fortran/C codes to
  Python easier and more robust. 

Enjoy,
	Pearu Peterson

---------------
<P><A HREF="http://cens.ioc.ee/projects/f2py2e/">F2PY 2.32.225</A> - The
Fortran to Python Interface Generator (08-Dec-02)





From perry at stsci.edu  Mon Dec  9 07:40:04 2002
From: perry at stsci.edu (Perry Greenfield)
Date: Mon Dec  9 07:40:04 2002
Subject: [Numpy-discussion] numarray equality testing
Message-ID: <NEBBIJKBMLDBLNCEEFOCIEPLCOAA.perry@stsci.edu>

This is a Numeric/numarray compatibility question.

Numeric currently allows the equals ufunc to compare 
arrays of unequal sizes and returns a scalar 0 in such
cases. When arrays have consistent shapes, an array of
ints is returned. We argue that this is inconsistent with
normal ufunc behavior and that it should generate an
exception as do all non-equality ufuncs. (numarray currently
does not allow comparison of shape-inconsistent arrays
including for equality). 

Instead we propose a function whose purpose is to determine
if two arrays are shape consistent (i.e., can be broadcast
against each other) and have all values equal to each other.

>>> array_equal(arange(2), arange(4))
0
>>> array_equal(array([1,2,3]), array([1,2,0]))
0
>>> array_equal( arange(2), None )
0
>>> array_equal( arange(2), not_an_ndarray_instance)
0
>>> array_equal(array([[1,2,3],[1,2,3]]), array([1,2,3]))
1

Comments?

Perry Greenfield




From perry at stsci.edu  Mon Dec  9 07:41:02 2002
From: perry at stsci.edu (Perry Greenfield)
Date: Mon Dec  9 07:41:02 2002
Subject: [Numpy-discussion] numarray complex comparisons
Message-ID: <NEBBIJKBMLDBLNCEEFOCMEPLCOAA.perry@stsci.edu>

The questions just keep coming...

We need to decide whether or not complex comparisons work.
They do not work for Python scalars. Consistency would argue
for them not working for numarray arrays. However some argue:

1) not allowing them defeats more generic programming. We agreed
until we found that IDL doesn't support them either, and we never
noticed. We are skeptical of this claim and would like to see
real-life examples.

2) it is useful to allow comparisons since that would result in
repeatable, sorting of values (e.g., to find duplicate values)
for ordering purposes. Cannot this just be handled by the sort
routines themselves? Why must this result in comparison operators
working?

In the absence of good examples for 1) or good arguments for 2)
we propose to make complex comparisons generate exceptions.

Perry Greenfield




From perry at stsci.edu  Mon Dec  9 07:41:03 2002
From: perry at stsci.edu (Perry Greenfield)
Date: Mon Dec  9 07:41:03 2002
Subject: [Numpy-discussion] numarray rank-0 array issues
Message-ID: <NEBBIJKBMLDBLNCEEFOCAEPMCOAA.perry@stsci.edu>

A little while ago we tackled the issue of whether indexing numarray
arrays should return Python scalars or rank-0 arrays. For reasons
gone into at great length previously on this list we decided to
always return Python scalars when no precision is lost by conversion to
Python scalars (Float128 may be an exception to this rule, but we'll
deal with that later). We consider this issue closed. But there
are some loose ends.

There was an opinion that reduction operations (e.g., add.reduce)
should always return arrays. However, some desired that the rank-0
arrays returned when reducing a rank-1 array should be indexable
and have a length. In particular:

>>> x = arange(10)
>>> v = add.reduce(x)
>>> print v
55
>>> v
array(55)
>>> len(v)
1
>>> v[0]
55

The great majority objected to len(rank-0) being allowed
as well as rank-0[0] working. We agree.

We propose that at least one version of the reduction operations
always return arrays, but that the array "end-point" always
be a rank-1 len-1 array instead of a rank-0 array. This is because
a rank-1 len-1 array 

1) has a len() = 1
2) can be indexed.
3) because of broadcasting, behaves like a scalar in expressions
   with arrays (i.e., as was intended for rank-0 arrays)

Thus, if one repeatedly reduces an N-dimensional array, one
eventually gets a rank-1 len-1 array, and reducing a rank-1
len-1 array  simply generates a new rank-1 len-1 array with the
same value.

>>> x = arange(10)
>>> v = add.areduce(x)
>>> v
array([55])
>>> add.areduce(v)
array([55])

Is there any reason why this behavior would not serve the purposes
of those that wanted rank-0 arrays returned from reduction operations?

Likewise, we could provide a function to wrap scalars as rank-1 len-1
arrays (much like array(5) produces a rank-0 array) for the purposes
of more generic functions and routines so that checks for scalars
do not need to be made. This function would convert scalars to arrays,
but simply return the arrays themselves if passed as arguments.

One possibility is that we eliminate rank-0 arrays and use rank-1 len-1
arrays in their place (e.g., array(5) returns a rank-1 len-1 array).
The problem is that there may be some who already dependon rank-0
properties and this probably will break existing code. Any opinions?

Finally, the default reduction operation (i.e., add.reduce) could 
be given the behavior described above. We are inclined to leave it
as is, i.e., to return scalars when reducing 1-d arrays and provide
a different operator method (areduce) to always return arrays. Any
disagreements?

Perry Greenfield



From pearu at cens.ioc.ee  Mon Dec  9 08:13:04 2002
From: pearu at cens.ioc.ee (Pearu Peterson)
Date: Mon Dec  9 08:13:04 2002
Subject: [Numpy-discussion] Re: [SciPy-user] numarray equality testing
In-Reply-To: <NEBBIJKBMLDBLNCEEFOCIEPLCOAA.perry@stsci.edu>
Message-ID: <Pine.LNX.4.21.0212091801410.6643-100000@cens.kybi>

On Mon, 9 Dec 2002, Perry Greenfield wrote:

> >>> array_equal(array([[1,2,3],[1,2,3]]), array([1,2,3]))
> 1

Hmm, I would have expected 0. 
What's the rationale for 1?

May be you meant

>>> array_equal(array([[1,2,3]]), array([1,2,3]))
1

which I would agree. Also

>>> array_equal(array([[1],[2],[3]]), array([1,2,3]))
1

but I am not sure about

>>> array_equal(array([[1,2,3],[1,2,3]]), array([1,2,3,1,2,3]))
1

Pearu





From tim.hochberg at ieee.org  Mon Dec  9 08:15:05 2002
From: tim.hochberg at ieee.org (Tim Hochberg)
Date: Mon Dec  9 08:15:05 2002
Subject: [Numpy-discussion] Re: [SciPy-user] numarray complex comparisons
References: <NEBBIJKBMLDBLNCEEFOCMEPLCOAA.perry@stsci.edu>
Message-ID: <005101c29f9d$ca8dfce0$391e6244@cx781526b>


> The questions just keep coming...
>
> We need to decide whether or not complex comparisons work.
> They do not work for Python scalars. Consistency would argue
> for them not working for numarray arrays. However some argue:
>
> 1) not allowing them defeats more generic programming. We agreed
> until we found that IDL doesn't support them either, and we never
> noticed. We are skeptical of this claim and would like to see
> real-life examples.
>
> 2) it is useful to allow comparisons since that would result in
> repeatable, sorting of values (e.g., to find duplicate values)
> for ordering purposes. Cannot this just be handled by the sort
> routines themselves? Why must this result in comparison operators
> working?
>
> In the absence of good examples for 1) or good arguments for 2)
> we propose to make complex comparisons generate exceptions.

What is the argument against allowing __eq__ to continue to work, while
disallowing __lt__ and friends. The former is well defined. I know comparing
floating point number for equality is a bit of a suckers game, but were
allowing it for floats and it's sometimes useful.

On second thought, maybe I'm just misinterpreting you here since __eq__
works fine for complex scalars.

-tim





From perry at stsci.edu  Mon Dec  9 08:25:01 2002
From: perry at stsci.edu (Perry Greenfield)
Date: Mon Dec  9 08:25:01 2002
Subject: [Numpy-discussion] RE: numarray equality testing
In-Reply-To: <Pine.LNX.4.21.0212091801410.6643-100000@cens.kybi>
Message-ID: <NEBBIJKBMLDBLNCEEFOCEEPMCOAA.perry@stsci.edu>

Pearu Peterson write:

> On Mon, 9 Dec 2002, Perry Greenfield wrote:
> 
> > >>> array_equal(array([[1,2,3],[1,2,3]]), array([1,2,3]))
> > 1
> 
> Hmm, I would have expected 0. 
> What's the rationale for 1?
> 
Because the arrays can be broadcast into a consistent 
array. In other words

equals(array([[1,2,3],[1,2,3]]), array([1,2,3])

returns

array([[1,1,1],[1,1,1]])

But I take your meaning. There may be those that wish
to ensure that two arrays are really identical in shape
and have all equal values. Should these be two different
functions? One function with different options.
By the way, I'm open to better function names as Tim Hochberg
suggests.

> May be you meant
> 
> >>> array_equal(array([[1,2,3]]), array([1,2,3]))
> 1
> 
> which I would agree. Also
> 
> >>> array_equal(array([[1],[2],[3]]), array([1,2,3]))
> 1
> 
> but I am not sure about
> 
> >>> array_equal(array([[1,2,3],[1,2,3]]), array([1,2,3,1,2,3]))
> 1
> 
> Pearu

Perry 



From perry at stsci.edu  Mon Dec  9 08:30:02 2002
From: perry at stsci.edu (Perry Greenfield)
Date: Mon Dec  9 08:30:02 2002
Subject: [Numpy-discussion] Re: [SciPy-user] numarray complex comparisons
In-Reply-To: <005101c29f9d$ca8dfce0$391e6244@cx781526b>
Message-ID: <NEBBIJKBMLDBLNCEEFOCMEPMCOAA.perry@stsci.edu>

Tim Hochberg write:

> 
> What is the argument against allowing __eq__ to continue to work, while
> disallowing __lt__ and friends. The former is well defined. I 
> know comparing
> floating point number for equality is a bit of a suckers game, but were
> allowing it for floats and it's sometimes useful.
> 
> On second thought, maybe I'm just misinterpreting you here since __eq__
> works fine for complex scalars.
> 
> -tim

I wasn't clear on this. What I was arguing was that less, less_equal,
greater, greater_equal would not work, but that equal and not_equal

Primarily because that is what Python does. It does allow one
to test for equality (or non equality), but doesn't allow
for >,<,>=,<=, which is sensible in my view.

Perry



From hinsen at cnrs-orleans.fr  Mon Dec  9 09:07:02 2002
From: hinsen at cnrs-orleans.fr (Konrad Hinsen)
Date: Mon Dec  9 09:07:02 2002
Subject: [Numpy-discussion] numarray equality testing
In-Reply-To: <NEBBIJKBMLDBLNCEEFOCIEPLCOAA.perry@stsci.edu>
References: <NEBBIJKBMLDBLNCEEFOCIEPLCOAA.perry@stsci.edu>
Message-ID: <m3lm2z9lcc.fsf@chinon.cnrs-orleans.fr>

"Perry Greenfield" <perry at stsci.edu> writes:

> Instead we propose a function whose purpose is to determine
> if two arrays are shape consistent (i.e., can be broadcast
> against each other) and have all values equal to each other.

So there would be that operation, plus one that checks identity
of shape and all elements? That should cover all needs I can think of.

Konrad.



From hinsen at cnrs-orleans.fr  Mon Dec  9 09:09:09 2002
From: hinsen at cnrs-orleans.fr (Konrad Hinsen)
Date: Mon Dec  9 09:09:09 2002
Subject: [Numpy-discussion] numarray rank-0 array issues
In-Reply-To: <NEBBIJKBMLDBLNCEEFOCAEPMCOAA.perry@stsci.edu>
References: <NEBBIJKBMLDBLNCEEFOCAEPMCOAA.perry@stsci.edu>
Message-ID: <m3fzt79l7x.fsf@chinon.cnrs-orleans.fr>

"Perry Greenfield" <perry at stsci.edu> writes:

> Finally, the default reduction operation (i.e., add.reduce) could 
> be given the behavior described above. We are inclined to leave it
> as is, i.e., to return scalars when reducing 1-d arrays and provide

Please do! There is no point in breaking code without any gain in return.

Konrad.
-- 
-------------------------------------------------------------------------------
Konrad Hinsen                            | E-Mail: hinsen at cnrs-orleans.fr
Centre de Biophysique Moleculaire (CNRS) | Tel.: +33-2.38.25.56.24
Rue Charles Sadron                       | Fax:  +33-2.38.63.15.17
45071 Orleans Cedex 2                    | Deutsch/Esperanto/English/
France                                   | Nederlands/Francais
-------------------------------------------------------------------------------



From bazell at comcast.net  Mon Dec  9 11:10:04 2002
From: bazell at comcast.net (Dave Bazell)
Date: Mon Dec  9 11:10:04 2002
Subject: [Numpy-discussion] installation warning on RH 7.3
Message-ID: <001f01c29fb6$1a6e0f80$6401a8c0@DB>

I just installed numpy on my RH 7.3 system under python 2.2.2.  I get the
following messages when I import Numeric.

Could someone enlighten me as to what this means and what I should do about
it?

Thanks,

Dave Bazell


Python 2.2.2 (#1, Oct 21 2002, 12:22:55)
[GCC 2.96 20000731 (Red Hat Linux 7.3 2.96-110)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import Numeric
Traceback (most recent call last):
  File "<stdin>", line 1, in ?
  File "/usr/local/lib/python2.2/site-packages/Numeric/Numeric.py", line 93,
in ?
    from Precision import *
  File "/usr/local/lib/python2.2/site-packages/Numeric/Precision.py", line
26, in ?
    _code_table = _fill_table(typecodes)
  File "/usr/local/lib/python2.2/site-packages/Numeric/Precision.py", line
23, in _fill_table
    table[key] = _get_precisions(value)
  File "/usr/local/lib/python2.2/site-packages/Numeric/Precision.py", line
18, in _get_precisions
    lst.append( (zeros( (1,), t ).itemsize()*8, t) )
ValueError: Invalid type for array
>>> import Numeric
>>>






From kern at caltech.edu  Mon Dec  9 16:48:03 2002
From: kern at caltech.edu (Robert Kern)
Date: Mon Dec  9 16:48:03 2002
Subject: [Numpy-discussion] Re: [SciPy-user] numarray equality testing
In-Reply-To: <004101c29f9c$f7d5e560$391e6244@cx781526b>
References: <NEBBIJKBMLDBLNCEEFOCIEPLCOAA.perry@stsci.edu> <004101c29f9c$f7d5e560$391e6244@cx781526b>
Message-ID: <20021210004645.GA3033@taliesen.caltech.edu>

On Mon, Dec 09, 2002 at 09:06:43AM -0700, Tim Hochberg wrote:

[snip]

> This seems like a good plan. I'm not enthralled by the name though:
> array_equal seems equally likely to describethe  behaviour of the unfunc
> equal as it does the behaviour above. Maybe array_identical or array_same or
> equal_arrays or some such (although none of those are great).

I'd put in my vote for array_equiv, equiv, or equivalent since I think
"equivalence" best expresses the relationship tested here, not
"equality."

> -tim

-- 
Robert Kern
Ruddock House President
kern at caltech.edu

"In the fields of hell where the grass grows high
 Are the graves of dreams allowed to die."
  -- Richard Harter



From ariciputi at pito.com  Wed Dec 11 02:10:02 2002
From: ariciputi at pito.com (Andrea Riciputi)
Date: Wed Dec 11 02:10:02 2002
Subject: [Numpy-discussion] Detecting Numeric.
Message-ID: <5F29487C-0CF0-11D7-883B-000393933E4E@pito.com>

Hi there,
I'm playing with Numeric and even if I've not so much experience with 
it I'm trying to write a C extention to Python that can use Numeric 
arrays. The basic idea consists in handling C (dynamically allocated) 
array as Numeric array (if Numeric is installed) or as simple Python 
lists if not.

How can I detect at compile time if Numeric is installed or not?? I've 
thought something like:

#ifdef NUMERIC_IS_HERE
#include "Numeric/arrayobject.h"
#endif

But I've not found any hints in Numeric manual. Can anyone suggest a 
solution to me?

Thanks in advance,
Andrea.

---
Andrea Riciputi        <mailto:andrea.riciputi at libero.it>

"Science is like sex: sometimes something useful comes out,
   but that is not the reason we are doing it" -- (Richard Feynman) 
  




From hinsen at cnrs-orleans.fr  Wed Dec 11 02:38:04 2002
From: hinsen at cnrs-orleans.fr (Konrad Hinsen)
Date: Wed Dec 11 02:38:04 2002
Subject: [Numpy-discussion] Detecting Numeric.
In-Reply-To: <5F29487C-0CF0-11D7-883B-000393933E4E@pito.com>
References: <5F29487C-0CF0-11D7-883B-000393933E4E@pito.com>
Message-ID: <m37kegkfoa.fsf@chinon.cnrs-orleans.fr>

Andrea Riciputi <ariciputi at pito.com> writes:

> How can I detect at compile time if Numeric is installed or not?? I've
> thought something like:
> 
> #ifdef NUMERIC_IS_HERE
> #include "Numeric/arrayobject.h"
> #endif

Supposing that you will use distutils for compilation and installation
of your extension module, the solution is easy: your distutils script
can test for Numeric and then add the compiler flag. For example:

   defines = []
   try:
       import Numeric
       defines.append('NUMERIC_IS_HERE')
   except ImportError:
       pass

   ...

   Extension("foo", ["foo.c"], define_macros = defines)

Konrad.
-- 
-------------------------------------------------------------------------------
Konrad Hinsen                            | E-Mail: hinsen at cnrs-orleans.fr
Centre de Biophysique Moleculaire (CNRS) | Tel.: +33-2.38.25.56.24
Rue Charles Sadron                       | Fax:  +33-2.38.63.15.17
45071 Orleans Cedex 2                    | Deutsch/Esperanto/English/
France                                   | Nederlands/Francais
-------------------------------------------------------------------------------



From pearu at cens.ioc.ee  Wed Dec 11 02:49:02 2002
From: pearu at cens.ioc.ee (Pearu Peterson)
Date: Wed Dec 11 02:49:02 2002
Subject: [Numpy-discussion] Detecting Numeric.
In-Reply-To: <5F29487C-0CF0-11D7-883B-000393933E4E@pito.com>
Message-ID: <Pine.LNX.4.21.0212111233520.20968-100000@cens.kybi>

On Wed, 11 Dec 2002, Andrea Riciputi wrote:

> Hi there,
> I'm playing with Numeric and even if I've not so much experience with 
> it I'm trying to write a C extention to Python that can use Numeric 
> arrays. The basic idea consists in handling C (dynamically allocated) 
> array as Numeric array (if Numeric is installed) or as simple Python 
> lists if not.
> 
> How can I detect at compile time if Numeric is installed or not?? I've 
> thought something like:
> 
> #ifdef NUMERIC_IS_HERE
> #include "Numeric/arrayobject.h"
> #endif
> 
> But I've not found any hints in Numeric manual. Can anyone suggest a 
> solution to me?

If you are using distutils for building extension modules then you can
test if Numeric is available in setup.py file and define, say, HAVE_NUMPY
macro, if Numeric is available. For example,

#-------

have_numpy = 0
try:
    import Numeric
    have_numpy = 1
except ImportError:
    pass

define_macros = []
if have_numpy:
    define_macros.append(('HAVE_NUMPY', None))

if __name__ == "__main__":

    from distutils.core import setup
    setup(name = ...,
          ext_modules = [...],
          define_macros = define_macros
          )

#---------

and in extension source files use

#ifdef HAVE_NUMPY
...
#endif

HTH,
	Pearu




From ariciputi at pito.com  Wed Dec 11 03:03:06 2002
From: ariciputi at pito.com (Andrea Riciputi)
Date: Wed Dec 11 03:03:06 2002
Subject: [Numpy-discussion] Some questions about PyArray_As2D.
Message-ID: <15153D8E-0CF8-11D7-883B-000393933E4E@pito.com>

Hi,
I've two question about PyArray_As2D function:

1) Looking at the source code I realized that the prototype given in 
the manual is different from that of the source:

PyArray_As2D(PyObject *op, char **ptr, int *m, int *n, int type) (from 
manual)

PyArray_As2D(PyObject **op, char ***ptr, int *d1, int *d2, int 
typecode) (from source)

As far as I can understand the source version is correct while the 
manual's one is not. Am I wrong?

2) Next in the source code I've found the following memory allocation:

data = (char **)malloc(n*sizeof(char *));

without checking if malloc return NULL or not. As far as I know it's 
not safe, even if it's very unlikely that this malloc would fail.
Anyway in that case the following:

...
data[i] = ap->data + i*ap->strides[0];
...

would cause the function to abort. Again am I wrong?

Thanks in advance,
Andrea.

---
Andrea Riciputi        <mailto:andrea.riciputi at libero.it>

"Science is like sex: sometimes something useful comes out,
   but that is not the reason we are doing it" -- (Richard Feynman) 
  




From hinsen at cnrs-orleans.fr  Wed Dec 11 05:37:02 2002
From: hinsen at cnrs-orleans.fr (Konrad Hinsen)
Date: Wed Dec 11 05:37:02 2002
Subject: [Numpy-discussion] Some questions about PyArray_As2D.
In-Reply-To: <15153D8E-0CF8-11D7-883B-000393933E4E@pito.com>
References: <15153D8E-0CF8-11D7-883B-000393933E4E@pito.com>
Message-ID: <m33cp4k7e8.fsf@chinon.cnrs-orleans.fr>

Andrea Riciputi <ariciputi at pito.com> writes:

> As far as I can understand the source version is correct while the
> manual's one is not. Am I wrong?

The source code always wins the argument :-)

> 2) Next in the source code I've found the following memory allocation:
> 
> data = (char **)malloc(n*sizeof(char *));
> 
> without checking if malloc return NULL or not. As far as I know it's
> not safe, even if it's very unlikely that this malloc would fail.

Right. Did you submit a bug report?

Konrad.
-- 
-------------------------------------------------------------------------------
Konrad Hinsen                            | E-Mail: hinsen at cnrs-orleans.fr
Centre de Biophysique Moleculaire (CNRS) | Tel.: +33-2.38.25.56.24
Rue Charles Sadron                       | Fax:  +33-2.38.63.15.17
45071 Orleans Cedex 2                    | Deutsch/Esperanto/English/
France                                   | Nederlands/Francais
-------------------------------------------------------------------------------



From ariciputi at pito.com  Wed Dec 11 07:04:02 2002
From: ariciputi at pito.com (Andrea Riciputi)
Date: Wed Dec 11 07:04:02 2002
Subject: [Numpy-discussion] Some questions about PyArray_As2D.
In-Reply-To: <m33cp4k7e8.fsf@chinon.cnrs-orleans.fr>
Message-ID: <9EAAE954-0D19-11D7-883B-000393933E4E@pito.com>

On Wednesday, Dec 11, 2002, at 14:33 Europe/Rome, Konrad Hinsen wrote:

>> 2) Next in the source code I've found the following memory allocation:
>>
>> data = (char **)malloc(n*sizeof(char *));
>>
>> without checking if malloc return NULL or not. As far as I know it's
>> not safe, even if it's very unlikely that this malloc would fail.
>
> Right. Did you submit a bug report?

Just done. By the way reading the code again and again I got another 
question. Here is the complete code fragment:

extern int PyArray_As2D(PyObject **op, char ***ptr, int *d1, int *d2, 
int typecode) {
     PyArrayObject *ap;
     int i, n;
     char **data;

     if ((ap = (PyArrayObject *)PyArray_ContiguousFromObject(*op, 
typecode, 2, 2)) == NULL)
         return -1;

     n = ap->dimensions[0];
     data = (char **)malloc(n*sizeof(char *));
     for(i=0; i<n; i++) {
         data[i] = ap->data + i*ap->strides[0];
     }
     *op = (PyObject *)ap;  <=== It doesn't sound good to me!!!
     *ptr = data;
     *d1 = ap->dimensions[0];
     *d2 = ap->dimensions[1];
     return 0;
}

Looking at the marked line I started wondering about the fate of the 
object originally pointed by op. Without explicitly deallocating it you 
lost any chance to reach it. It turns out in a memory leakage. 
Obviously the same problem happend in PyArray_As1D function.

I'm very interested in this topic because I'm writing some Python 
extensions and I'd like to understand how I have to handle all these 
objects correctly. So how "long" does a Python object live? How can I 
release correctly the allocated memory?

Thanks,
Andrea.
---
Andrea Riciputi        <mailto:andrea.riciputi at libero.it>

"Science is like sex: sometimes something useful comes out,
   but that is not the reason we are doing it" -- (Richard Feynman)




From hinsen at cnrs-orleans.fr  Wed Dec 11 10:46:03 2002
From: hinsen at cnrs-orleans.fr (Konrad Hinsen)
Date: Wed Dec 11 10:46:03 2002
Subject: [Numpy-discussion] Some questions about PyArray_As2D.
In-Reply-To: <9EAAE954-0D19-11D7-883B-000393933E4E@pito.com> (message from
	Andrea Riciputi on Wed, 11 Dec 2002 16:02:51 +0100)
References: <9EAAE954-0D19-11D7-883B-000393933E4E@pito.com>
Message-ID: <200212111843.gBBIhSl22020@chinon.cnrs-orleans.fr>

> Just done. By the way reading the code again and again I got another 
> question. Here is the complete code fragment:
> 
> extern int PyArray_As2D(PyObject **op, char ***ptr, int *d1, int *d2, 
> int typecode) {
>      PyArrayObject *ap;
>      int i, n;
>      char **data;
> 
>      if ((ap = (PyArrayObject *)PyArray_ContiguousFromObject(*op, 
> typecode, 2, 2)) == NULL)
>          return -1;
> 
>      n = ap->dimensions[0];
>      data = (char **)malloc(n*sizeof(char *));
>      for(i=0; i<n; i++) {
>          data[i] = ap->data + i*ap->strides[0];
>      }
>      *op = (PyObject *)ap;  <=== It doesn't sound good to me!!!
>      *ptr = data;
>      *d1 = ap->dimensions[0];
>      *d2 = ap->dimensions[1];
>      return 0;
> }
> 
> Looking at the marked line I started wondering about the fate of the 
> object originally pointed by op. Without explicitly deallocating it you 
> lost any chance to reach it. It turns out in a memory leakage. 

No. op is an input parameter and thus a "borrowed" reference. It might
not be the best coding style to reuse that variable name for something
unrelated later on, but it doesn't cause a memory leak.

> I'm very interested in this topic because I'm writing some Python 
> extensions and I'd like to understand how I have to handle all these 
> objects correctly. So how "long" does a Python object live? How can I 
> release correctly the allocated memory?

There is an explanation of this topic in chapter 1.10 of the Python
"Extending and Embedding" manual. Basically, you have to increase an
object's reference counter if you want to keep a reference to it
beyond the end of the currently running function, and to decrease the
counter if you want to release that reference. Whenever the reference
count goes to zero, the object is deleted. With very few exceptions,
functions that take an object as input to work on (but not store)
don't increase the reference counter. They don't have to, because
nothing can happen to the object until the function terminates.
Special precautions need to be taken for multithreading, but these go
much beyond object reference counting.

As a rule of thumb, you don't have to worry about reference counting
at all as long as you only write C functions that act on existing
objects. It becomes an issue when you define your own extension types
in C.

Konrad.
-- 
-------------------------------------------------------------------------------
Konrad Hinsen                            | E-Mail: hinsen at cnrs-orleans.fr
Centre de Biophysique Moleculaire (CNRS) | Tel.: +33-2.38.25.56.24
Rue Charles Sadron                       | Fax:  +33-2.38.63.15.17
45071 Orleans Cedex 2                    | Deutsch/Esperanto/English/
France                                   | Nederlands/Francais
-------------------------------------------------------------------------------



From edcjones at erols.com  Wed Dec 11 18:48:02 2002
From: edcjones at erols.com (Edward C. Jones)
Date: Wed Dec 11 18:48:02 2002
Subject: [Numpy-discussion] 3 comments on numarray documentation
Message-ID: <3DF80012.4070008@erols.com>

To access UInt8, etc,

     from numerictypes import *

Maybe mention this in 4.2.1.

-------

In 4.7

      Only one "..." is expanded in an index expression, so if one has a
      rank-5 array C, then C[...,0,...] is the same thing as
      C[:,:,:,0,:].

So an unexpanded "..." is treated as a ':'?

----------

In 5.1.1,

      >>> a = arange(5, type=Float64)
      >>> print a[::-1] * 1.2
      [ 4.8  3.6  2.4  1.2  0. ]
      >>> multiply(a[::-1], 1.2, a)
      >>> a
      array([ 4.8 ,  3.6 ,  2.4 ,  1.2,  0. ])

doesn't make the desired point. Try:

      >>> a = arange(5, type=Int32)
      >>> a
      [0 1 2 3 4]
      >>> print a[::-1] * 1.2
      [ 4.8  3.6  2.4  1.2  0. ]
      >>> multiply(a[::-1], 1.2, a)
      >>> a
      array([4 3 2 1 0])

Why does Python documentation always use the interpreter? I doubt if it 
is used much. Why not:

      a = arange(5, type=Int32)
      print a.type(), a
      b = a[::-1] * 1.2
      print b.type(), b
      numarray.multiply(a[::-1], 1.2, a)
      print a.type(), a





From Chris.Barker at noaa.gov  Thu Dec 12 12:34:03 2002
From: Chris.Barker at noaa.gov (Chris Barker)
Date: Thu Dec 12 12:34:03 2002
Subject: [Numpy-discussion] 3 comments on numarray documentation
References: <3DF80012.4070008@erols.com>
Message-ID: <3DF8DC98.28232AC4@noaa.gov>

"Edward C. Jones" wrote:

> Why does Python documentation always use the interpreter? I doubt if it
> is used much. 

I, for one, use it all the time to explore the syntax and function of a
simple statement, just like the examples.

-Chris


-- 
Christopher Barker, Ph.D.
Oceanographer
                                    		
NOAA/OR&R/HAZMAT         (206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115       (206) 526-6317   main reception

Chris.Barker at noaa.gov



From oliphant.travis at ieee.org  Thu Dec 12 12:40:03 2002
From: oliphant.travis at ieee.org (Travis Oliphant)
Date: Thu Dec 12 12:40:03 2002
Subject: [Numpy-discussion] real/imag for non complex numbers [was numarray complex comparisons]
In-Reply-To: <003101c2a16c$7901dba0$391e6244@cx781526b>
References: <Pine.LNX.4.21.0212112257250.19308-100000@cens.kybi> 
	<003101c2a16c$7901dba0$391e6244@cx781526b>
Message-ID: <1039725475.2227.13.camel@travis.local.net>

In investigating, why Numeric does not allow a.real for non-complex
arrays which many have argued that it should, I think I uncovered an old
bug in Numeric.

In looking at the code, it appears that the intention was to have a.real
just return a new array (pointing to the same data as a but having a new
header) whenever a was not a complex array.  Unfortunately, the code had
a bug and did not actually return the newly created array.  As a result,
the default behavior of raising an "Attribute not found" exception
occurred.  But, before this happened, the array the reference count on
the array occurred, thereby springing a memory leak. 

Try this out.

a = ones((10,10))

print sys.getrefcount(a)

a.real   # will generate an exception

print sys.getrefcount(a)

a.real  

print sys.getrefcount(a)

Notice that each time a.real is called the reference count for a goes up
by one even though no object is returned.


I have fixed the bug in Numeric CVS by returning the created object for
a.real.

The end result is that a.real works for all arrays (not just
complex-valued ones). 

-Travis O.



On Wed, 2002-12-11 at 16:24, Tim Hochberg wrote:
> 
> Here's my latest thinking on this. I'll put this in the form of a proposal
> so that it's easier to comment on. Please comment on part I and part II
> separately.
> 
> I. Attributes real and imag/imaginary[1]
> 
> All numeric array types (i.e., excluding character and object arrays) would
> have both a real and imag/imaginary attribute. For complex types ('F', 'D')
> this would behave just as it does now. For non-complex types ('i', 'f', 'd',
> etc.), the real attribute would return the object itself, while the
> imaginary attribute would return a ReadOnlyZeroArray [2] of the correct
> shape and typecode. I think that this would provide behaviour consistent
> with the current real and imag attributes on complex arrays without
> introducing and spooky corner cases (that statement is asking for trouble!).
> Examples:
> 
> >>> a = array([1,2,3])
> >>> a.real[0] = 99
> >>> a.real
> array([99, 2, 3])
> >>> a.imag
> array([0,0,0])
> >>> a.imag[0] = 99
> TypeError: object is read only.
> 
> II. Functions real and imag
> 
> Two functions real and imag (or imaginary or both) would be provided. The
> would behave as if they were defined as shown below, even if item I. is not
> adopted.
> 
> def real(a):
>     return asarray(a).real
> 
> def imag(a):
>     return asarray(a).imag
> 
> 
> Comments?
> 
> I'm currently +0 on I. and +1 on II.
> 
> -tim
> 
> 
> 
> [1] Numeric arrays haveboth imag and imaginary attributes, but they are
> identical. I'm not sure what Numarray's plans are in this regard.
> 
> [2] A ReadOnlyZeroArray(shape, typecode) would behave like an array of zeros
> with the given shape and typecode for all non mutating operations. For
> mutation operations (a[x] = b, a += c, etc.) it would raise an exception.
> This object does not actually allocate any space, so creation is fast and
> memory efficient. Implementing this might be the hardest part of the
> proposal.
> 
> 
> _______________________________________________
> SciPy-user mailing list
> SciPy-user at scipy.net
> http://www.scipy.net/mailman/listinfo/scipy-user





From ariciputi at pito.com  Fri Dec 13 02:46:00 2002
From: ariciputi at pito.com (Andrea Riciputi)
Date: Fri Dec 13 02:46:00 2002
Subject: [Numpy-discussion] Some questions about PyArray_As2D.
In-Reply-To: <200212111843.gBBIhSl22020@chinon.cnrs-orleans.fr>
Message-ID: <8550842A-0E87-11D7-883B-000393933E4E@pito.com>

On Wednesday, Dec 11, 2002, at 19:43 Europe/Rome, Konrad Hinsen wrote:

> No. op is an input parameter and thus a "borrowed" reference. It might
> not be the best coding style to reuse that variable name for something
> unrelated later on, but it doesn't cause a memory leak.

I'm sorry but I don't agree. I've read your answer many times and 
re-read the code but I still think to be right.

The function prototype says:

extern int PyArray_As2D(PyObject **op, char ***ptr, int *d1, int *d2, 
int typecode)

and a call to this function looks like this:

PyObject *input;
double **result;
int nrows, ncols;

PyArray_As2D((&input, (char ***) &(result), &(nrows), &(ncols), 
PyArray_DOUBLE)

Now when you call the function in this way op is a pointer to the 
pointer that points to your original ArrayObject. It allows you to 
change the memory address which is originally pointed by input. And it 
is exactly what you do with the instruction:

*op = (PyObject *)ap;

So you create a new PyArrayObject (allocating another memory area) by 
means of PyArray_ContiguousFromObject and names it ap, then you modify 
the memory address which op points to with the above instruction. Now 
*op (that is input) points to another memory region, but you haven't 
deallocated the previous pointed memory and it _is_ a memory leak!

These are my two cents, comments are welcome.

Cheers,
Andrea.


---
Andrea Riciputi        <mailto:andrea.riciputi at libero.it>

"Science is like sex: sometimes something useful comes out,
   but that is not the reason we are doing it" -- (Richard Feynman)




From hinsen at cnrs-orleans.fr  Fri Dec 13 03:12:02 2002
From: hinsen at cnrs-orleans.fr (Konrad Hinsen)
Date: Fri Dec 13 03:12:02 2002
Subject: [Numpy-discussion] Some questions about PyArray_As2D.
In-Reply-To: <8550842A-0E87-11D7-883B-000393933E4E@pito.com> (message from
	Andrea Riciputi on Fri, 13 Dec 2002 11:42:05 +0100)
References: <8550842A-0E87-11D7-883B-000393933E4E@pito.com>
Message-ID: <200212131109.gBDB9Mk27388@chinon.cnrs-orleans.fr>

> The function prototype says:
> 
> extern int PyArray_As2D(PyObject **op, char ***ptr, int *d1, int *d2, 
> int typecode)

You are right that my first comment doesn't quite apply, I hadn't
noticed the two stars...

But the code is still OK, it is the calling routine that is responsible
for avoiding memory leaks.

> is exactly what you do with the instruction:
> 
> *op = (PyObject *)ap;
> 
> So you create a new PyArrayObject (allocating another memory area) by 
> means of PyArray_ContiguousFromObject and names it ap, then you modify 
> the memory address which op points to with the above instruction. Now 

Right. This routine cannot know if that reference contains an "owned"
or a "borrowed" reference. In the first case, the reference counter
of the original array must be decreased, in the second case not.

Assume for example that the calling routine got an array passed in
as a borrowed reference:

void foo(PyObject *array)
{
   /* I want a 2D version! */
   PyArray_As2D(&array, ...)
}

In that case, decreasing the reference count in PyArray_As2D would
be disastrous.

In the case of an owned reference, the calling routine must keep a
copy of the pointer, call PyArray_As2D, and then decrease the
reference counter on the original pointer. This ought to be easier.
In fact, the interface of PyArray_As2D is pretty badly designed.
It should not overwrite the original pointer, but return a new
one. I also don't see the point in passing all the array information
in additional arguments - they are easy to obtain from the new
array object.

Konrad.
-- 
-------------------------------------------------------------------------------
Konrad Hinsen                            | E-Mail: hinsen at cnrs-orleans.fr
Centre de Biophysique Moleculaire (CNRS) | Tel.: +33-2.38.25.56.24
Rue Charles Sadron                       | Fax:  +33-2.38.63.15.17
45071 Orleans Cedex 2                    | Deutsch/Esperanto/English/
France                                   | Nederlands/Francais
-------------------------------------------------------------------------------



From perry at stsci.edu  Fri Dec 13 08:30:05 2002
From: perry at stsci.edu (Perry Greenfield)
Date: Fri Dec 13 08:30:05 2002
Subject: [Numpy-discussion] 3 comments on numarray documentation
In-Reply-To: <3DF80012.4070008@erols.com>
Message-ID: <JFEGLNDJEDNOMPPHDEJFGELDEBAA.perry@stsci.edu>

Sorry it took so long to respond. We appreciate this feedback.

Edward C. Jones writes:
 
> To access UInt8, etc,
> 
>      from numerictypes import *
> 
> Maybe mention this in 4.2.1.
> 
Are you referring to text in another part of the manual or are you
suggesting that this be added to 4.2.1? If added I would reword it
somewhat since these type names are in the numarray namespace. If 
one wants to do:

import numarray

and have the types as part of you namespace it would make sense to

import numarray
from numerictypes import *

(though if we go to a package system, this may become
from numarray.numerictypes import *)

We also need to add the fact that there are now 
UInt32, UInt64 (not on windows), Int64 types.

> -------
> 
> In 4.7
> 
>       Only one "..." is expanded in an index expression, so if one has a
>       rank-5 array C, then C[...,0,...] is the same thing as
>       C[:,:,:,0,:].
> 
> So an unexpanded "..." is treated as a ':'?
> 
yes

> ----------
> 
> In 5.1.1,
> 
>       >>> a = arange(5, type=Float64)
>       >>> print a[::-1] * 1.2
>       [ 4.8  3.6  2.4  1.2  0. ]
>       >>> multiply(a[::-1], 1.2, a)
>       >>> a
>       array([ 4.8 ,  3.6 ,  2.4 ,  1.2,  0. ])
> 
> doesn't make the desired point. Try:
> 
>       >>> a = arange(5, type=Int32)
>       >>> a
>       [0 1 2 3 4]
>       >>> print a[::-1] * 1.2
>       [ 4.8  3.6  2.4  1.2  0. ]
>       >>> multiply(a[::-1], 1.2, a)
>       >>> a
>       array([4 3 2 1 0])
> 
Yes, we will make this change

> Why does Python documentation always use the interpreter? I doubt if it 
> is used much. Why not:
> 
>       a = arange(5, type=Int32)
>       print a.type(), a
>       b = a[::-1] * 1.2
>       print b.type(), b
>       numarray.multiply(a[::-1], 1.2, a)
>       print a.type(), a
> 
Actually many do use it in interpreter mode, at least here. But I think
you miss the main point which is to show the result of each command
for the purposes of instruction. Even if you never plan to use the
interpreter (which I think would be a mistake since it is a wonderful
way of verifying that things work the way you thought they did),
it serves to show examples in a very clear and concise way. 

Perry



From edcjones at erols.com  Fri Dec 13 08:34:02 2002
From: edcjones at erols.com (Edward C. Jones)
Date: Fri Dec 13 08:34:02 2002
Subject: [Numpy-discussion] Comments on Documentation - 2
Message-ID: <3DFA1344.6030003@erols.com>

I am currently learning about numarray. Here are further comments on the 
documentation.

----------------------

"libnumarray.h" and "numarray.h" are never mentioned in the docs.

----------------------

10.2.3 Is "free(result)" needed here?

         double *result;
         int m, n;
         .
         .
         .
         result = func(...);
         if(NULL == result)
             return NULL;
         return NA_New((void *)result, tFloat64, m, n, 2);

----------------------

10.6 "NA_New", "NA_Empty", and "NA_NewA11" return a "PyArrayObject*", 
not a "PyObject*".




From edcjones at erols.com  Fri Dec 13 17:09:04 2002
From: edcjones at erols.com (Edward C. Jones)
Date: Fri Dec 13 17:09:04 2002
Subject: [Numpy-discussion] Comments on numarray - 3
Message-ID: <3DFA8BEB.2040209@erols.com>

NA_OFFSETDATA is mentioned during the example in 10.2.3. It should be 
documented in 10.2.1. Should some of the other macros in "nummacro.h" be 
documented? Also the macros in "arrayobject.h" and "numcomplex.h".

-------------------

The "libnumarray_UNIQUE_SYMBOL" trick discussed in "libnumarray.h" has 
been useful with Numeric and will be useful with numarray. It needs to 
be in the main documentation. I think it is mentioned somewhere in the 
Python documentation. I found it very confusing the first time I used 
it. A detailed example is needed.

-------------------

"NA_longsFromIntTuple" works on both tuples and lists. A better name 
might be "NA_longsFromIntSequence". Functions of this type have proven 
_very_ useful for me in writing large SWIG wrappers. I generate them 
using SWIG macros. Here are some of them. On the C side, auxillary 
information about the lengths of things needs to be dealt with. "list" 
means "list or tuple".

Python                     C
list of ints          <--> array of ints
list of floats        <--> array of doubles
list of floats        <--> array of floats
...
list of strings       <--> array of arrays of char
list of lists of ints <--> array of arrays of int
...
list                  <--> array of PyObject*

Thanks,
Ed Jones





From magnus at hetland.org  Sun Dec 15 13:17:03 2002
From: magnus at hetland.org (Magnus Lie Hetland)
Date: Sun Dec 15 13:17:03 2002
Subject: [Numpy-discussion] Index array confusion...
Message-ID: <20021215211615.GA18385@idi.ntnu.no>

I can see why this happens, to some degree, but it is a bit confusing
still:

>>> a = zeros([5,5])
>>> a[[1,2,3]]
array([[0, 0, 0, 0, 0],
       [0, 0, 0, 0, 0],
       [0, 0, 0, 0, 0]])
>>> a[[1,2]]
0
>>> a[[1]]
array([[0, 0, 0, 0, 0]])

I was planning to use index arrays to select a subset of the rows in a
two-dimensional array, but because of the special-casing for
one-dimensional arrays of the same length as the dimensionality of the
array being indexed, it seems I can't do that (without using
put/take, which may be quite OK, I guess)... Am I right? (I see why
this behaviour is needed for tuples and slice objects; and I guess
type checking would be sort of "ugly"...)

Of course a[[1,2],] works just as expected.

This may not be important, but perhaps worth a sentence or example in
the manual? I.e. if you are using an index array idx and you don't
want to risk having it cast as a single index lookup, you should do

  a[idx,]

Wouldn't that be sound advice in general?

-- 
Magnus Lie Hetland        Practical Python          The Anygui Project
http://hetland.org        http://ppython.com        http://anygui.org



From jmiller at stsci.edu  Sun Dec 15 13:34:02 2002
From: jmiller at stsci.edu (Todd Miller)
Date: Sun Dec 15 13:34:02 2002
Subject: [Numpy-discussion] Index array confusion...
References: <20021215211615.GA18385@idi.ntnu.no>
Message-ID: <3DFCF883.5000806@stsci.edu>

Magnus Lie Hetland wrote:

>I can see why this happens, to some degree, but it is a bit confusing
>still:
>
>  
>
>>>>a = zeros([5,5])
>>>>a[[1,2,3]]
>>>>        
>>>>
>array([[0, 0, 0, 0, 0],
>       [0, 0, 0, 0, 0],
>       [0, 0, 0, 0, 0]])
>  
>
>>>>a[[1,2]]
>>>>        
>>>>
>0
>  
>
>>>>a[[1]]
>>>>        
>>>>
>array([[0, 0, 0, 0, 0]])
>
>I was planning to use index arrays to select a subset of the rows in a
>two-dimensional array, but because of the special-casing for
>one-dimensional arrays of the same length as the dimensionality of the
>array being indexed, it seems I can't do that (without using
>put/take, which may be quite OK, I guess)... Am I right? (I see why
>this behaviour is needed for tuples and slice objects; and I guess
>type checking would be sort of "ugly"...)
>
>Of course a[[1,2],] works just as expected.
>
>This may not be important, but perhaps worth a sentence or example in
>the manual? I.e. if you are using an index array idx and you don't
>want to risk having it cast as a single index lookup, you should do
>
>  a[idx,]
>
>Wouldn't that be sound advice in general?
>
>  
>
Assuming you're talking about numarray here,  I regard this as a 0.4 bug 
which you just found.  Thanks!

Todd





From magnus at hetland.org  Sun Dec 15 13:36:02 2002
From: magnus at hetland.org (Magnus Lie Hetland)
Date: Sun Dec 15 13:36:02 2002
Subject: [Numpy-discussion] Index array confusion...
In-Reply-To: <3DFCF883.5000806@stsci.edu>
References: <20021215211615.GA18385@idi.ntnu.no> <3DFCF883.5000806@stsci.edu>
Message-ID: <20021215213541.GA20046@idi.ntnu.no>

Todd Miller <jmiller at stsci.edu>:
[snip]
> Assuming you're talking about numarray here,

Indeed.

> I regard this as a 0.4 bug which you just found.  Thanks!

No problem :)

So, I guess you're saying that in the future you will typecheck for
tuples and slice objects vs. other objects (such as arrays and lists)?

> Todd

-- 
Magnus Lie Hetland        Practical Python          The Anygui Project
http://hetland.org        http://ppython.com        http://anygui.org



From magnus at hetland.org  Sun Dec 15 14:03:03 2002
From: magnus at hetland.org (Magnus Lie Hetland)
Date: Sun Dec 15 14:03:03 2002
Subject: [Numpy-discussion] Index array confusion...
In-Reply-To: <3DFCFA3B.7090109@stsci.edu>
References: <20021215211615.GA18385@idi.ntnu.no> <3DFCF883.5000806@stsci.edu> <20021215213541.GA20046@idi.ntnu.no> <3DFCFA3B.7090109@stsci.edu>
Message-ID: <20021215220230.GA20949@idi.ntnu.no>

Todd Miller <jmiller at stsci.edu>:
>
[snip]
> What I'm saying is that in the future it will work correctly.  :)

OK :)

> Most likely as you suggest,  but I haven't really looked at the code, 
> just the rather sickening results.

Well, what I was asking about was really what you meant by "correct".
My interpretation of "correct" here was that only tuples and slices
would be allowed as indexes.

BTW: I found something else that I think is rather odd:

>>> x = [0, 0, 0]
>>> y = [1, 1, 1]
>>> p = 1
>>> x[p:], y[p:] = y[p:], x[p:]
>>> x, y
([0, 1, 1], [1, 0, 0])
>>> x = array([0, 0, 0])
>>> y = array([1, 1, 1])
>>> x[p:], y[p:] = y[p:], x[p:]
>>> x, y
(array([0, 1, 1]), array([1, 1, 1]))

This seems like a bug to me. The assignment ought to swap the tails of
the sequences, as is the case with lists, but with numeric arrays,
some weird form of overwriting occurs. I guess this may be an
optimization (i.e. to avoid making copies), but it is rather
confusing. What do you think?

> Todd

-- 
Magnus Lie Hetland        Practical Python          The Anygui Project
http://hetland.org        http://ppython.com        http://anygui.org



From magnus at hetland.org  Sun Dec 15 14:10:01 2002
From: magnus at hetland.org (Magnus Lie Hetland)
Date: Sun Dec 15 14:10:01 2002
Subject: [Numpy-discussion] Index array confusion...
In-Reply-To: <20021215220230.GA20949@idi.ntnu.no>
References: <20021215211615.GA18385@idi.ntnu.no> <3DFCF883.5000806@stsci.edu> <20021215213541.GA20046@idi.ntnu.no> <3DFCFA3B.7090109@stsci.edu> <20021215220230.GA20949@idi.ntnu.no>
Message-ID: <20021215220911.GA21137@idi.ntnu.no>

Magnus Lie Hetland <magnus at hetland.org>:
[snip]
> This seems like a bug to me. The assignment ought to swap the tails of
> the sequences, as is the case with lists, but with numeric arrays,
> some weird form of overwriting occurs. I guess this may be an
> optimization (i.e. to avoid making copies), but it is rather
> confusing. What do you think?

Even stranger:

>>> a = zeros([5])
>>> b = ones([5])
>>> p = 2
>>> tail_a = a[p:]
>>> tail_b = b[p:]
>>> a[p:] = tail_b
>>> b[p:] = tail_a
>>> a, b
(array([0, 0, 1, 1, 1]), array([1, 1, 1, 1, 1]))

I suppose this is due to sharing of slices somehow? I.e:

>>> a = ones([5])
>>> b = a[:]
>>> b[2] = 0
>>> a
array([1, 1, 0, 1, 1])

I have to use the copy method here, I guess?

> > Todd

-- 
Magnus Lie Hetland        Practical Python          The Anygui Project
http://hetland.org        http://ppython.com        http://anygui.org



From magnus at hetland.org  Sun Dec 15 14:31:07 2002
From: magnus at hetland.org (Magnus Lie Hetland)
Date: Sun Dec 15 14:31:07 2002
Subject: [Numpy-discussion] "not"
Message-ID: <20021215222955.GA21992@idi.ntnu.no>

Just a quick note: Why does the operator 'not' appear in parentheses
beside logical_not in the manual? Isn't that a bit misleading (since
it doesn't work the same way, as opposed to 'and' and 'or')?

-- 
Magnus Lie Hetland        Practical Python          The Anygui Project
http://hetland.org        http://ppython.com        http://anygui.org



From jmiller at stsci.edu  Mon Dec 16 06:57:01 2002
From: jmiller at stsci.edu (Todd Miller)
Date: Mon Dec 16 06:57:01 2002
Subject: [Numpy-discussion] Index array confusion...
References: <20021215211615.GA18385@idi.ntnu.no> <3DFCF883.5000806@stsci.edu> <20021215213541.GA20046@idi.ntnu.no> <3DFCFA3B.7090109@stsci.edu> <20021215220230.GA20949@idi.ntnu.no>
Message-ID: <3DFDEA77.10204@stsci.edu>

Magnus Lie Hetland wrote:

>Todd Miller <jmiller at stsci.edu>:
>  
>
>[snip]
>  
>
>>What I'm saying is that in the future it will work correctly.  :)
>>    
>>
>
>OK :)
>
>  
>
>>Most likely as you suggest,  but I haven't really looked at the code, 
>>just the rather sickening results.
>>    
>>
>
>Well, what I was asking about was really what you meant by "correct".
>My interpretation of "correct" here was that only tuples and slices
>would be allowed as indexes.
>
This actually worked as you expected in numarray-0.3.6.  The current 
behavior is a casualty of optimization to C.

>
>BTW: I found something else that I think is rather odd:
>
>  
>
>>>>x = [0, 0, 0]
>>>>y = [1, 1, 1]
>>>>p = 1
>>>>x[p:], y[p:] = y[p:], x[p:]
>>>>x, y
>>>>        
>>>>
>([0, 1, 1], [1, 0, 0])
>  
>
>>>>x = array([0, 0, 0])
>>>>y = array([1, 1, 1])
>>>>x[p:], y[p:] = y[p:], x[p:]
>>>>x, y
>>>>        
>>>>
>(array([0, 1, 1]), array([1, 1, 1]))
>
>This seems like a bug to me. The assignment ought to swap the tails of
>
Numeric does this as well,  so I would not call it a bug,  but  I agree 
that it is unfortunate.

>the sequences, as is the case with lists, but with numeric arrays,
>some weird form of overwriting occurs. I guess this may be an
>optimization (i.e. to avoid making copies), but it is rather
>
I think you're correct here.   This behavior is a consequence of the 
fact that array slices are views and not copies.  To fix it,  just say:

x[p:], y[p:] = y[p:].copy(), x[p:].copy()

>confusing. What do you think?
>
The manual should probably document this as a gotcha, and we might want 
to consider adding an exchange ufunc which can do the swap without a 
temporary.   I doubt the cost of exchange is worth it though.

Thanks for the feedback,
Todd




From jmiller at stsci.edu  Mon Dec 16 07:01:01 2002
From: jmiller at stsci.edu (Todd Miller)
Date: Mon Dec 16 07:01:01 2002
Subject: [Numpy-discussion] Index array confusion...
References: <20021215211615.GA18385@idi.ntnu.no> <3DFCF883.5000806@stsci.edu> <20021215213541.GA20046@idi.ntnu.no> <3DFCFA3B.7090109@stsci.edu> <20021215220230.GA20949@idi.ntnu.no> <20021215220911.GA21137@idi.ntnu.no>
Message-ID: <3DFDEB60.7090604@stsci.edu>

Magnus Lie Hetland wrote:

>Magnus Lie Hetland <magnus at hetland.org>:
>[snip]
>  
>
>>This seems like a bug to me. The assignment ought to swap the tails of
>>the sequences, as is the case with lists, but with numeric arrays,
>>some weird form of overwriting occurs. I guess this may be an
>>optimization (i.e. to avoid making copies), but it is rather
>>confusing. What do you think?
>>    
>>
>
>Even stranger:
>
>  
>
>>>>a = zeros([5])
>>>>b = ones([5])
>>>>p = 2
>>>>tail_a = a[p:]
>>>>tail_b = b[p:]
>>>>a[p:] = tail_b
>>>>b[p:] = tail_a
>>>>a, b
>>>>        
>>>>
>(array([0, 0, 1, 1, 1]), array([1, 1, 1, 1, 1]))
>
>I suppose this is due to sharing of slices somehow? I.e:
>  
>
This looks to me like the same problem,  just performing the effects of 
the tuple copy one step at a time.  The key is that the "tails" are 
views and not copies.

>  
>
>>>>a = ones([5])
>>>>b = a[:]
>>>>b[2] = 0
>>>>a
>>>>        
>>>>
>array([1, 1, 0, 1, 1])
>
>I have to use the copy method here, I guess?
>  
>
Yes.   array[ slice ] --> a view of a subarray.

>  
>
>>>Todd
>>>      
>>>
>
>  
>






From perry at stsci.edu  Mon Dec 16 11:13:06 2002
From: perry at stsci.edu (Perry Greenfield)
Date: Mon Dec 16 11:13:06 2002
Subject: [Numpy-discussion] Index array confusion...
In-Reply-To: <3DFDEA77.10204@stsci.edu>
Message-ID: <JFEGLNDJEDNOMPPHDEJFIELMEBAA.perry@stsci.edu>

Magnus Lie Hetland wrote:

> >Well, what I was asking about was really what you meant by "correct".
> >My interpretation of "correct" here was that only tuples and slices
> >would be allowed as indexes.
> >
To clarify, tuples should be interpreted differently than lists or 
arrays as arguments, as you suggested earlier so that
x[1,2,3] is not interpreted the same as x[[1,2,3]] or x[array([1,2,3])].
Granted, that will be confusing for those that try x[(1,2,3)]
expecting it to be equivalent ot x[[1,2,3]] but we decided that
the benefits of array indices well outweigh that confusing case.

As far as far as slices resulting in views rather than copies
(was with lists), this is a topic that has been talked to death.
The original Numeric has view behavior. When we started on numarray
we were intending to make it more consistent with lists, but there
were many that argued that view semantics were more sensible and
we were persuaded that they were right (besides, the backward
compatibility issue was very important as well).

Perry



From magnus at hetland.org  Mon Dec 16 11:23:02 2002
From: magnus at hetland.org (Magnus Lie Hetland)
Date: Mon Dec 16 11:23:02 2002
Subject: [Numpy-discussion] Index array confusion...
In-Reply-To: <JFEGLNDJEDNOMPPHDEJFIELMEBAA.perry@stsci.edu>
References: <3DFDEA77.10204@stsci.edu> <JFEGLNDJEDNOMPPHDEJFIELMEBAA.perry@stsci.edu>
Message-ID: <20021216192207.GA15908@idi.ntnu.no>

Perry Greenfield <perry at stsci.edu>:
>
> Magnus Lie Hetland wrote:
> 
> > >Well, what I was asking about was really what you meant by "correct".
> > >My interpretation of "correct" here was that only tuples and slices
> > >would be allowed as indexes.
> > >
> To clarify, tuples should be interpreted differently than lists or 
> arrays as arguments, as you suggested earlier so that
> x[1,2,3] is not interpreted the same as x[[1,2,3]] or x[array([1,2,3])].
> Granted, that will be confusing for those that try x[(1,2,3)]
> expecting it to be equivalent ot x[[1,2,3]] but we decided that
> the benefits of array indices well outweigh that confusing case.

Exactly. I agree completely with this (intended) behaviour. I was just
a bit baffled to find that numarray (0.4), in fact, behaved
differently. Since this is indeed a bug, the issue seems resolved to
me :)

(Except that I might have to add some version control to avoid running
with 0.4 :/)

> As far as far as slices resulting in views rather than copies
> (was with lists), this is a topic that has been talked to death.

Yes -- sorry about that.

> The original Numeric has view behavior.

I know. Just a mind-bug on my part :)

> When we started on numarray
> we were intending to make it more consistent with lists, but there
> were many that argued that view semantics were more sensible and
> we were persuaded that they were right (besides, the backward
> compatibility issue was very important as well).

And performance, perhaps?

And; from Todd's comment about a hypothetical built-in for the
purpose, I assume there is no way of copying the contents of one slice
to another without going via a temporary array?

> Perry

-- 
Magnus Lie Hetland        Practical Python          The Anygui Project
http://hetland.org        http://ppython.com        http://anygui.org



From jmiller at stsci.edu  Mon Dec 16 12:05:58 2002
From: jmiller at stsci.edu (Todd Miller)
Date: Mon Dec 16 12:05:58 2002
Subject: [Numpy-discussion] Index array confusion...
References: <3DFDEA77.10204@stsci.edu> <JFEGLNDJEDNOMPPHDEJFIELMEBAA.perry@stsci.edu> <20021216192207.GA15908@idi.ntnu.no>
Message-ID: <3DFE32C5.4070506@stsci.edu>

Magnus Lie Hetland wrote:

>  
>
>And; from Todd's comment about a hypothetical built-in for the
>purpose, I assume there is no way of copying the contents of one slice
>to another without going via a temporary array?
>  
>
The problem there is with overlapping slices.

>Perry
>  
>






From jmiller at stsci.edu  Mon Dec 16 14:22:03 2002
From: jmiller at stsci.edu (Todd Miller)
Date: Mon Dec 16 14:22:03 2002
Subject: [Numpy-discussion] Re: numarray questions
References: <Pine.LNX.4.40.0212161225530.1299-200000@sylvia>
Message-ID: <3DFE52E0.6020003@stsci.edu>

Mika Illouz wrote:

>Hi Todd,
>
>  
>
>>Thanks for the encouragement!   numarray-0.4 is coming out today...
>>    
>>
>
>Finally got around to playing with version 0.4 and the new NA_New (code
>attached).  Wnen I ran the test, I got the following trace:
>
>  
>
>>>>import _test_numarray
>>>>z = _test_numarray.t1()
>>>>        
>>>>
>Traceback (most recent call last):
>  File "<stdin>", line 1, in ?
>  File "/usr/lib/python2.2/site-packages/numarray/numarray.py", line 432, in __init__
>    bytestride=bytestride)
>  File "/usr/lib/python2.2/site-packages/numarray/ndarray.py", line 173, in __init__
>    self._data = memory.new_memory(size)
>OverflowError: long int too large to convert to int
>
>What am I doing wrong?
>
The parameter order in the call in t1() should be:

NA_New(a, tInt32, 1, length)

Because it is:

NA_New(a, tInt32, length, 1)

numarray thinks you're declaring a 5 dimensional array with some bizarre 
dimensions from "undefined values" on the stack.  One of these 
dimensions appears to be > MAX_INT.

>Thanks again,
>Mika
>  
>
Regards,
Todd

>  
>
>------------------------------------------------------------------------
>
>#include <Python.h>
>#include <libnumarray.h>
>
>static PyObject* t1( PyObject* self, PyObject* args )
>{
>  int length = 5;
>  int* a = new int[ length ];
>  for ( int i = 0 ; i < length ; i++ ) {
>    a[i] = i;
>  }
>
>  PyArrayObject* vec = NA_New( a, tInt32, length, 1 );
>  return (PyObject*)vec;
>  // return NA_ReturnOutput( NULL, vec );
>
>}
>
>static PyMethodDef TestNumarrayMethods[] = {
>  {"t1",  
>   t1, 
>   METH_VARARGS, 
>   "first test"
>  },
>  { NULL, NULL, 0, NULL}
>};
>
>extern "C" void init_test_numarray()
>{
>  Py_InitModule( "_test_numarray", TestNumarrayMethods );
>  import_libnumarray();
>}
>  
>






From perry at stsci.edu  Mon Dec 16 14:25:01 2002
From: perry at stsci.edu (Perry Greenfield)
Date: Mon Dec 16 14:25:01 2002
Subject: [Numpy-discussion] Index array confusion...
In-Reply-To: <20021216192207.GA15908@idi.ntnu.no>
Message-ID: <JFEGLNDJEDNOMPPHDEJFCELNEBAA.perry@stsci.edu>

> And performance, perhaps?
> 
Certainly. That was one big factor (more with regard to
memory demands than CPU demands).

> And; from Todd's comment about a hypothetical built-in for the
> purpose, I assume there is no way of copying the contents of one slice
> to another without going via a temporary array?
> 
If you mean:

Without overwriting the slice being copied to with overlapping slices
(whether it's a problem depends on how they overlap and whether one is
reversed or not, and one other effect described below) on the same array,
that's generally true.

In numarray, there are cases where the slice is copied internally in
blocks. It's possible that you could get away with

x[:] = x[-1::-1]

if x is smaller than the blocksize being used. You would be crazy
to depend on this though :-) For large enough arrays (try 10000 elements),
it definitely won't work.

Perry



From edcjones at erols.com  Tue Dec 17 19:37:03 2002
From: edcjones at erols.com (Edward C. Jones)
Date: Tue Dec 17 19:37:03 2002
Subject: [Numpy-discussion] Comments on numarray - 4
Message-ID: <3DFFF4D6.3070005@erols.com>

The optional shape argument to fromstring is not documented.

-----------------

I suggest a documentation section on NumericTypes. "Int8.bytes" and
need documenting.

----------------

      Table of Automatic Coercions (found by experiment)

          B   I8   U8  I16  U16  I32  U32  I64  U64  F32  F64  C32  C64
      ------------------------------------------------------------------
    B |  *I8   I8   U8  I16  U16  I32  U32  I64  U64  F32  F64  C32  C64
   I8 |        I8 *I16  I16  U16  I32  U32  I64  U64  F32  F64  C32  C64
   U8 |             U8  I16  U16  I32  U32  I64  U64  F32  F64  C32  C64
  I16 |                 I16 *I32  I32  U32  I64  U64  F32  F64  C32  C64
  U16 |                      U16  I32  U32  I64  U64  F32  F64  C32  C64
  I32 |                           I32 *I64  I64  U64  F32  F64  C32  C64
  U32 |                                U32  I64  U64  F32  F64  C32  C64
  I64 |                                     I64 *I64 ?F32  F64  C32  C64
  U64 |                                          U64 ?F32  F64  C32  C64
  F32 |                                               F32  F64  C32  C64
  F64 |                                                    F64 ?C32  C64
  C32 |                                                         C32  C64
  C64 |                                                              C64

The names are listed in the order used in "numerictypes.py". The "*" 
cases are exceptions treated there. Are the "?" conversions correct? See 
"genericPromotionExclusions" in "numerictypes.py".




From jmiller at stsci.edu  Wed Dec 18 07:04:04 2002
From: jmiller at stsci.edu (Todd Miller)
Date: Wed Dec 18 07:04:04 2002
Subject: [Numpy-discussion] Comments on numarray - 4
References: <3DFFF4D6.3070005@erols.com>
Message-ID: <3E008F65.90802@stsci.edu>

Edward C. Jones wrote:

> The optional shape argument to fromstring is not documented.

Noted.

>
> -----------------
>
> I suggest a documentation section on NumericTypes. "Int8.bytes" and
> need documenting.

Noted.

>
> ----------------
>
>      Table of Automatic Coercions (found by experiment)
>
>          B   I8   U8  I16  U16  I32  U32  I64  U64  F32  F64  C32  C64
>      ------------------------------------------------------------------
>    B |  *I8   I8   U8  I16  U16  I32  U32  I64  U64  F32  F64  C32  C64
>   I8 |        I8 *I16  I16  U16  I32  U32  I64  U64  F32  F64  C32  C64
>   U8 |             U8  I16  U16  I32  U32  I64  U64  F32  F64  C32  C64
>  I16 |                 I16 *I32  I32  U32  I64  U64  F32  F64  C32  C64
>  U16 |                      U16  I32  U32  I64  U64  F32  F64  C32  C64
>  I32 |                           I32 *I64  I64  U64  F32  F64  C32  C64
>  U32 |                                U32  I64  U64  F32  F64  C32  C64
>  I64 |                                     I64 *I64 ?F32  F64  C32  C64
>  U64 |                                          U64 ?F32  F64  C32  C64
>  F32 |                                               F32  F64  C32  C64
>  F64 |                                                    F64 ?C32  C64
>  C32 |                                                         C32  C64
>  C64 |                                                              C64

Nice table!

>
> The names are listed in the order used in "numerictypes.py". The "*" 
> cases are exceptions treated there. Are the "?" conversions correct? See

No.  The Complex32/Float64 conversion has already been fixed in CVS.   
The Int64/Float32 coercion will be changed from Float32 to Float64.

> "genericPromotionExclusions" in "numerictypes.py". 

Thanks for the excellent feedback!

Todd




From edcjones at erols.com  Wed Dec 18 14:27:03 2002
From: edcjones at erols.com (Edward C. Jones)
Date: Wed Dec 18 14:27:03 2002
Subject: [Numpy-discussion] Two bugs in numerictypes.py
Message-ID: <3E00FDB5.2090804@erols.com>

In "numerictypes.py' it says "This module is designed so 'from 
numerictypes import *' is safe." When I "import numerictypes", and run 
dir(), I get:

['Any', 'AnyType', 'Bool', 'BooleanType', 'Byte', 'Complex', 
'Complex32', 'Complex64', 'ComplexType', 'Double', 'Float', 'Float32', 
'Float64', 'FloatingType', 'Int', 'Int16', 'Int32', 'Int64', 'Int8', 
'IntegralType', 'Long', 'MAX_ALIGN', 'NumericType', 'Short', 
'SignedIntegralType', 'SignedType', 'UInt16', 'UInt32', 'UInt64', 
'UInt8', 'UnsignedIntegralType', 'UnsignedType', '__builtins__', 
'__doc__', '__name__', 'genericCoercions', 'genericPromotionExclusions', 
'genericTypeRank', 'inttype1', 'inttype2', 'kind', 'mapto', 'maptype1', 
'maptype2', 'nt1', 'nt2', 'ntypesize1', 'ntypesize2', 'numinclude', 
'outtype', 'pythonTypeMap', 'pythonTypeRank', 'rank1', 'rank2', 
'scalarTypeMap', 'signedtype1', 'signedtype2', 'typeDict', 'typecode', 
'typecodes']

A bunch of leading "_" are needed.

----------------

My code is:

#! /usr/bin/env python
from numerictypes import UInt8
print UInt8 == "ABC"

The message is:

Traceback (most recent call last):
   File "./silly02.py", line 5, in ?
     print UInt8 == "ABC"
   File "/usr/lib/python2.2/site-packages/numarray/numerictypes.py",
line 101, in __cmp__
     other = typeDict[other]
KeyError: ABC

I would expect that the only object == to UInt8 be itself. Maybe add a 
function comparetypecodes(x, y) which returns True iff x and y are 
either NumericType's or strings which represent the same type.




From haase at msg.ucsf.edu  Wed Dec 18 17:23:05 2002
From: haase at msg.ucsf.edu (Sebastian Haase)
Date: Wed Dec 18 17:23:05 2002
Subject: [Numpy-discussion] Have a problem: what is attribute 'compress'
References: <3E00FDB5.2090804@erols.com>
Message-ID: <004b01c2a6fd$195c95f0$3b45da80@rodan>

Hi!
Somehow I have a problem with numarray. Please take a look at this:
>>>import numarray as na
>>>na.array([0, 0])
array([0, 0])
>>>na.array([0.0, 0.0])
Traceback (most recent call last):
  File "<input>", line 1, in ?
  File "C:\Python22\Lib\site-packages\numarray\numarray.py", line 581, in
__repr__
    MAX_LINE_WIDTH, PRECISION, SUPPRESS_SMALL, ', ', 1)
  File "C:\Python22\Lib\site-packages\numarray\arrayprint.py", line 163, in
array2string
    separator, array_output)
  File "C:\Python22\Lib\site-packages\numarray\arrayprint.py", line 125, in
_array2string
    format, item_length = _floatFormat(data, precision, suppress_small)
  File "C:\Python22\Lib\site-packages\numarray\arrayprint.py", line 246, in
_floatFormat
    non_zero = numarray.abs(numarray.compress(numarray.not_equal(data, 0),
data))
AttributeError: 'module' object has no attribute 'compress'

The same workes fine with Numeric. But I would prefer numarray because I'm
writing C++-extensions and I need "unsigned shorts".

What is this error about?

Thanks,
Sebastian





From jmiller at stsci.edu  Thu Dec 19 05:54:03 2002
From: jmiller at stsci.edu (Todd Miller)
Date: Thu Dec 19 05:54:03 2002
Subject: [Numpy-discussion] Have a problem: what is attribute 'compress'
References: <3E00FDB5.2090804@erols.com> <004b01c2a6fd$195c95f0$3b45da80@rodan>
Message-ID: <3E01D07B.3070009@stsci.edu>

Sebastian Haase wrote:

>Hi!
>Somehow I have a problem with numarray. Please take a look at this:
>
Hi Sebastian,

I've don't recall seeing anything like this,  nor can I reproduce it 
now.   If you've been following numarray for a while now,  I can say 
that it is important to remove the old version of numarray before 
installing the new version.   I recommend deleting your current 
installation and reinstalling numarray.

compress() is a ufunc,  much like add() or put().  It is defined in 
ndarray.py,  right after the import of the modules ufunc and _ufunc.   
_ufunc in particular is a problematic module,  because it has followed 
the atypical development path of moving from C-code to Python code. 
 Because of this, and the fact that a .so or .dll overrides a .py, 
 older installations interfere with newer ones.  The atypical path was 
required because the original _ufuncmodule.c was so large that it could 
not be compiled on some systems;  as a result,  I split _ufuncmodule.c 
into pieces by data type and now use _ufunc.py to glue the pieces together.

Good luck!    Please let me know if reinstalling doesn't clear up the 
problem.

Todd

>  
>
>>>>import numarray as na
>>>>na.array([0, 0])
>>>>        
>>>>
>array([0, 0])
>  
>
>>>>na.array([0.0, 0.0])
>>>>        
>>>>
>Traceback (most recent call last):
>  File "<input>", line 1, in ?
>  File "C:\Python22\Lib\site-packages\numarray\numarray.py", line 581, in
>__repr__
>    MAX_LINE_WIDTH, PRECISION, SUPPRESS_SMALL, ', ', 1)
>  File "C:\Python22\Lib\site-packages\numarray\arrayprint.py", line 163, in
>array2string
>    separator, array_output)
>  File "C:\Python22\Lib\site-packages\numarray\arrayprint.py", line 125, in
>_array2string
>    format, item_length = _floatFormat(data, precision, suppress_small)
>  File "C:\Python22\Lib\site-packages\numarray\arrayprint.py", line 246, in
>_floatFormat
>    non_zero = numarray.abs(numarray.compress(numarray.not_equal(data, 0),
>data))
>AttributeError: 'module' object has no attribute 'compress'
>
>The same workes fine with Numeric. But I would prefer numarray because I'm
>writing C++-extensions and I need "unsigned shorts".
>
>What is this error about?
>
>Thanks,
>Sebastian
>
>
>
>
>-------------------------------------------------------
>This SF.NET email is sponsored by: Order your Holiday Geek Presents Now!
>Green Lasers, Hip Geek T-Shirts, Remote Control Tanks, Caffeinated Soap,
>MP3 Players,  XBox Games,  Flying Saucers,  WebCams,  Smart Putty.
>T H I N K G E E K . C O M       http://www.thinkgeek.com/sf/
>_______________________________________________
>Numpy-discussion mailing list
>Numpy-discussion at lists.sourceforge.net
>https://lists.sourceforge.net/lists/listinfo/numpy-discussion
>  
>






From jmiller at stsci.edu  Thu Dec 19 06:37:11 2002
From: jmiller at stsci.edu (Todd Miller)
Date: Thu Dec 19 06:37:11 2002
Subject: [Numpy-discussion] howto build numarray 0.4 ? file 'Src/_convmodule.c'
 does not exist
References: <030301c29b1c$16db8530$3b45da80@rodan>
Message-ID: <3E01DA83.90101@stsci.edu>

Sebastian Haase wrote:

>Hi all,
>I downloaded numarray 0.4 about 5 minutes after I got the announce but 
>my naive 'python2.2 ./setup.py build'   gives this
>  
>
numarray-0.4 doesn't currently support a two step build process because 
code generation,  which is required,  doesn't occur unless the install 
command is present.   Unfortunately,  code generation *always* occurs 
when the install command is present,  which defeats the purpose of a 
seperate build.   This problem has been fixed in CVS by making code 
generation the result of an explicit setup.py switch (--gencode) which 
makes it possible to generate code during a build command.

One work-around is the following:

1. First generate code (as a non-priviledged user) using "python 
setup.py install".  Control-C as soon as the code starts to build.

2. Do  "python setup.py build"

3. Edit setup.py,  commenting out the 2 calls to os.system() in the 
function prepare().  This prevents codegeneration during the "real" install.

4. Do "python setup.py install" as root,  noting that no new code is 
generated, and everything is already built.

Todd

>haase at baboon:~/numarray-0.4: python2.2 ./setup.py build
>running build
>running build_py
>not copying Lib/ndarray.py (output up-to-date)
>not copying Lib/numtest.py (output up-to-date)
>not copying Lib/codegenerator.py (output up-to-date)
>not copying Lib/ufunc.py (output up-to-date)
>not copying Lib/testdata.py (output up-to-date)
>not copying Lib/numarray.py (output up-to-date)
>not copying Lib/ieeespecial.py (output up-to-date)
>not copying Lib/recarray.py (output up-to-date)
>not copying Lib/template.py (output up-to-date)
>not copying Lib/arrayprint.py (output up-to-date)
>not copying Lib/typeconv.py (output up-to-date)
>not copying Lib/numinclude.py (output up-to-date)
>not copying Lib/numarrayext.py (output up-to-date)
>not copying Lib/_ufunc.py (output up-to-date)
>not copying Lib/chararray.py (output up-to-date)
>not copying Lib/numtestall.py (output up-to-date)
>not copying Lib/numerictypes.py (output up-to-date)
>not copying Lib/memmap.py (output up-to-date)
>running build_ext
>building '_conv' extension
>error: file 'Src/_convmodule.c' does not exist
>
>What am I missing? I'm running Linux (debian woody) on i386.
>
>Thanks,
>Sebastian
>
>-------------------------------------------------------
>This SF.net email is sponsored by: Microsoft Visual Studio.NET 
>comprehensive development tool, built to increase your 
>productivity. Try a free online hosted session at:
>http://ads.sourceforge.net/cgi-bin/redirect.pl?micr0003en
>_______________________________________________
>Numpy-discussion mailing list
>Numpy-discussion at lists.sourceforge.net
>https://lists.sourceforge.net/lists/listinfo/numpy-discussion
>  
>






From Chris.Barker at noaa.gov  Fri Dec 20 11:41:07 2002
From: Chris.Barker at noaa.gov (Chris Barker)
Date: Fri Dec 20 11:41:07 2002
Subject: [Numpy-discussion] Another nice write-up about floating point issues.
References: <000001c29cc3$281ed420$6501a8c0@NICKLEBY> <Pine.SOL.4.50.0212052304510.28199-100000@death.OCF.Berkeley.EDU> <200212061138.30065.mclay@nist.gov>
Message-ID: <3E036FD9.61BC6CA@noaa.gov>

Hi all,

I just stumbled upon this paper by William Kahan, a Professor at
Berkeley that is well known for his work on the first Intel math
co-processor, and the development of IEE 754 (plus a lot of other
stuff). I took a course with him at Berkeley, and the man is brilliant.
So brilliant that is is very hard to follow him, as a student, if you
are not so sharp. I'm not, and I just squeeked by and missed a great
deal of what he tried to teach. I did gain a good appreciation of the
complexity of floating point arithmetic, however. Here is the paper as a
PDF:

http://www.cs.berkeley.edu/%7ewkahan/MgtkMath.pdf

There's a lot of other good stuff on his web page at:

http://www.cs.berkeley.edu/%7ewkahan/

THe paper is entitled "Marketing versus Mathematics". In particular,
take a look at the section on Quattro Pro on Page 13, for a nice
discussion of binary arithmetic displayed as decimal, which is an often
brought up subject in the Python newsgroups. He discusses how you'd
really have to display more than the 15 digits that Quattro displayed,
to avoid confusion.

"""
But no such cure can be liberated from little annoyances:
0.8  entered would display as 0.80000000000000004 to 17 sig. dec.;
"""
At the Python prompt:
>>> 0.8
0.80000000000000004

More evidence that the current Python way is "best", if confusing to
newbies. This would be another good paper to point people towards that
ask about floating point on the newsgroups.

-Chris

-- 
Christopher Barker, Ph.D.
Oceanographer
                                    		
NOAA/OR&R/HAZMAT         (206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115       (206) 526-6317   main reception

Chris.Barker at noaa.gov



From vo.chi.cong at is.titech.ac.jp  Sun Dec 22 00:34:04 2002
From: vo.chi.cong at is.titech.ac.jp (Cong)
Date: Sun Dec 22 00:34:04 2002
Subject: [Numpy-discussion] bug in Numeric, LinearAlgebra
Message-ID: <20021222.173433.59464351.vo.chi.cong@is.titech.ac.jp>

I've just upgraded from Numeric-21.0 to Numeric-22.0 
( python-2.2.2 ) and found a bug. Do anyone use LinearAlgebra too ?

Now, I downed it to Numeric 21.0 . This version works very well.

Cong

$ python
Python 2.2.2 (#1, Dec 14 2002, 05:45:37) 
[GCC 3.2.1] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import LinearAlgebra
Traceback (most recent call last):
  File "<stdin>", line 1, in ?
  File "/usr/lib/python2.2/site-packages/Numeric/LinearAlgebra.py", line 10, in ?
    import MLab
  File "/usr/lib/python2.2/site-packages/Numeric/MLab.py", line 20, in ?
    import RandomArray
  File "/usr/lib/python2.2/site-packages/Numeric/RandomArray.py", line 30, in ?
    seed()
  File "/usr/lib/python2.2/site-packages/Numeric/RandomArray.py", line 24, in seed
    ndigits = int(math.log10(t))
OverflowError: float too large to convert




From vo.chi.cong at is.titech.ac.jp  Sun Dec 22 00:52:04 2002
From: vo.chi.cong at is.titech.ac.jp (Cong)
Date: Sun Dec 22 00:52:04 2002
Subject: [Numpy-discussion] Re: bug in Numeric, LinearAlgebra
In-Reply-To: <20021222.173433.59464351.vo.chi.cong@is.titech.ac.jp>
References: <20021222.173433.59464351.vo.chi.cong@is.titech.ac.jp>
Message-ID: <20021222.175219.74752851.vo.chi.cong@is.titech.ac.jp>

Cong> I've just upgraded from Numeric-21.0 to Numeric-22.0 
Cong> ( python-2.2.2 ) and found a bug. Do anyone use LinearAlgebra too ?
Cong> 
Cong> Now, I downed it to Numeric 21.0 . This version works very well.

I found that now this version doesnot work too. So I delete Numeric
21.0 complettely then install Numeric 22.0 again. After that the error
message changed to:

$ python
Python 2.2.2 (#1, Dec 22 2002, 17:44:21) 
[GCC 3.2.1] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import LinearAlgebra
Traceback (most recent call last):
  File "<stdin>", line 1, in ?
  File "/usr/lib/python2.2/site-packages/Numeric/LinearAlgebra.py", line 10, in ?
    import MLab
  File "/usr/lib/python2.2/site-packages/Numeric/MLab.py", line 20, in ?
    import RandomArray
  File "/usr/lib/python2.2/site-packages/Numeric/RandomArray.py", line 30, in ?
    seed()
  File "/usr/lib/python2.2/site-packages/Numeric/RandomArray.py", line 26, in seed
    x = int(t/base)
OverflowError: float too large to conver

I can not figure out what is happenning. Temporarily, I import
LinearAlgebra this way:

try:
	import LinearAlgebra
except:
	import LinearAlgebra

And this worked, for Numeric 22.0 on python 2.2.2
as well as for Numeric 21.0 on python 2.2.2 .

// Cong

Cong> 
Cong> Cong
Cong> 
Cong> $ python
Cong> Python 2.2.2 (#1, Dec 14 2002, 05:45:37) 
Cong> [GCC 3.2.1] on linux2
Cong> Type "help", "copyright", "credits" or "license" for more information.
Cong> >>> import LinearAlgebra
Cong> Traceback (most recent call last):
Cong>   File "<stdin>", line 1, in ?
Cong>   File "/usr/lib/python2.2/site-packages/Numeric/LinearAlgebra.py", line 10, in ?
Cong>     import MLab
Cong>   File "/usr/lib/python2.2/site-packages/Numeric/MLab.py", line 20, in ?
Cong>     import RandomArray
Cong>   File "/usr/lib/python2.2/site-packages/Numeric/RandomArray.py", line 30, in ?
Cong>     seed()
Cong>   File "/usr/lib/python2.2/site-packages/Numeric/RandomArray.py", line 24, in seed
Cong>     ndigits = int(math.log10(t))
Cong> OverflowError: float too large to convert
Cong> 



From edcjones at erols.com  Sun Dec 22 18:34:04 2002
From: edcjones at erols.com (Edward C. Jones)
Date: Sun Dec 22 18:34:04 2002
Subject: [Numpy-discussion] "<<", ">>" undocumented
Message-ID: <3E067DD5.6020609@erols.com>

The operators "<<" and ">>" seem to be undocumented. Is there a reason 
for this?

Ed Jones




From magnus at hetland.org  Fri Dec 27 13:30:03 2002
From: magnus at hetland.org (Magnus Lie Hetland)
Date: Fri Dec 27 13:30:03 2002
Subject: [Numpy-discussion] Non-numerical info associated with sub-arrays
Message-ID: <20021227212942.GA11897@idi.ntnu.no>

I'm working on some two-dimensional tables of data, where some data
are numerical, while other aren't. I'd like to use numarray's
numerical capabilities with the numerical parts (columns) while
keeping the data in each row together. (I'm sure this generalizes to
more dimensions, and to sub-arrays in general, not just rows.)

It's not a hard problem, really, but the obvious solution--to keep
the other rows in separate arrays/lists and just juggle things
around--seems a bit clunky. I was just wondering if anyone had other
ideas (would it be practical to include all the data in a single array
somehow--I seem to recall that Numeric could have arbitrary object
arrays, but I'm not sure whether numarray supports this?) or perhaps
some hints on how to organize code around this? I wrote a small class
that wraps things up and works a bit lik R/S-plus's data frames; is
there some other more standard code out there for this sort of thing?
(It's a problem that crops up often in data processing of various
kinds...)

Thanks,

Magnus

-- 
Magnus Lie Hetland
http://hetland.org



From tchur at optushome.com.au  Fri Dec 27 14:13:02 2002
From: tchur at optushome.com.au (Tim Churches)
Date: Fri Dec 27 14:13:02 2002
Subject: [Numpy-discussion] Non-numerical info associated with sub-arrays
In-Reply-To: <20021227212942.GA11897@idi.ntnu.no>
References: <20021227212942.GA11897@idi.ntnu.no>
Message-ID: <1041102817.1264.142.camel@localhost.localdomain>

On Fri, 2002-12-27 at 11:29, Magnus Lie Hetland wrote:
> I'm working on some two-dimensional tables of data, where some data
> are numerical, while other aren't. I'd like to use numarray's
> numerical capabilities with the numerical parts (columns) while
> keeping the data in each row together. (I'm sure this generalizes to
> more dimensions, and to sub-arrays in general, not just rows.)
> 
> It's not a hard problem, really, but the obvious solution--to keep
> the other rows in separate arrays/lists and just juggle things
> around--seems a bit clunky. I was just wondering if anyone had other
> ideas (would it be practical to include all the data in a single array
> somehow--I seem to recall that Numeric could have arbitrary object
> arrays, but I'm not sure whether numarray supports this?) or perhaps
> some hints on how to organize code around this? I wrote a small class
> that wraps things up and works a bit lik R/S-plus's data frames; is
> there some other more standard code out there for this sort of thing?
> (It's a problem that crops up often in data processing of various
> kinds...)

Have a look at the discussion on RecordArrays in this overview of
Numarray: http://stsdas.stsci.edu/numarray/DesignOverview.html

However, in the meantime, as you note, its not too hard to write a class
which emulates R/S-Plus data frames. Just store each column in its own
Numeric array of the appropriate type (which might be the PyObject
types, which can hold any Python object type), and have the wrapper
class implement __getitem__ etc to collect the relevant "rows" from each
column and return them as a complete row as a dict or a sequence. Not
that fast, but not slow either. You can implement a generator to allow
cursor-like traversal of the all the rows if you like. Happy to
collaborate on furthering this idea. By memory-mapping disc-based
versions of the  Numeric arrays, and using the BsdDb3 record number
database format for the string columns, you can even make a disc-based
"record array" which can be larger than available RAM+swap. I hope to
release code written under contract by Dave Cole (see
http://www.object-craft.com.au ) which illustrates this idea in the next
month or so (but I've been saying that to myself for a year or more...).

Tim C
 





From magnus at hetland.org  Fri Dec 27 14:57:01 2002
From: magnus at hetland.org (Magnus Lie Hetland)
Date: Fri Dec 27 14:57:01 2002
Subject: [Numpy-discussion] Non-numerical info associated with sub-arrays
In-Reply-To: <1041102817.1264.142.camel@localhost.localdomain>
References: <20021227212942.GA11897@idi.ntnu.no> <1041102817.1264.142.camel@localhost.localdomain>
Message-ID: <20021227225549.GA21921@idi.ntnu.no>

Tim Churches <tchur at optushome.com.au>:
[snip]
> Have a look at the discussion on RecordArrays in this overview of
> Numarray: http://stsdas.stsci.edu/numarray/DesignOverview.html

Sounds interesting.

> However, in the meantime, as you note, its not too hard to write a class
> which emulates R/S-Plus data frames. Just store each column in its own
> Numeric array of the appropriate type

Yeah -- it's just that I'd like to keep a set of columns collected as
a two-dimensional array, to allow horizontal summing and the like.
(Not much more complicated, but an extra issue to address.)

> (which might be the PyObject
> types, which can hold any Python object type),

Hm. Yes. I can't seem to find these anymore. I seem to recall using
type='o' or something in Numeric, but I can't find the right type
objects now... (Guess I'm just reading the docs and dir(numeric)
poorly...) It would be nice if array(['foo']) just worked. Oh, well.

[snip]
> Happy to
> collaborate on furthering this idea.

That would be great (even though I don't really have any time to use
for this -- it's just a really tiny part of a small project I'm
working on :)

> By memory-mapping disc-based
> versions of the  Numeric arrays, and using the BsdDb3 record number
> database format for the string columns, you can even make a disc-based
> "record array" which can be larger than available RAM+swap.

Sounds quite useful, although quite similar to MetaKit. (I suppose I
could use some functions from numarray on columns in MetaKit... But
that might just be too weird -- and it would still just be a
collection of columns :])

[snip]

Thanks for your input.

-- 
Magnus Lie Hetland
http://hetland.org



From tchur at optushome.com.au  Fri Dec 27 15:50:02 2002
From: tchur at optushome.com.au (Tim Churches)
Date: Fri Dec 27 15:50:02 2002
Subject: [Numpy-discussion] Non-numerical info associated with sub-arrays
In-Reply-To: <20021227225549.GA21921@idi.ntnu.no>
References: <20021227212942.GA11897@idi.ntnu.no>
	<1041102817.1264.142.camel@localhost.localdomain> 
	<20021227225549.GA21921@idi.ntnu.no>
Message-ID: <1041108637.1260.153.camel@localhost.localdomain>

On Fri, 2002-12-27 at 12:55, Magnus Lie Hetland wrote:
> Tim Churches <tchur at optushome.com.au>:
> [snip]
> > Have a look at the discussion on RecordArrays in this overview of
> > Numarray: http://stsdas.stsci.edu/numarray/DesignOverview.html
> 
> Sounds interesting.
> 
> > However, in the meantime, as you note, its not too hard to write a class
> > which emulates R/S-Plus data frames. Just store each column in its own
> > Numeric array of the appropriate type
> 
> Yeah -- it's just that I'd like to keep a set of columns collected as
> a two-dimensional array, to allow horizontal summing and the like.
> (Not much more complicated, but an extra issue to address.)
> 
> > (which might be the PyObject
> > types, which can hold any Python object type),
> 
> Hm. Yes. I can't seem to find these anymore. I seem to recall using
> type='o' or something in Numeric, but I can't find the right type
> objects now... (Guess I'm just reading the docs and dir(numeric)
> poorly...) It would be nice if array(['foo']) just worked. Oh, well.

Just like this:

>>> import Numeric
>>> a = Numeric.array(['a','b','c'],typecode=Numeric.PyObject)
>>> a
array([a , b , c ],'O')
>>>

> 
> > By memory-mapping disc-based
> > versions of the  Numeric arrays, and using the BsdDb3 record number
> > database format for the string columns, you can even make a disc-based
> > "record array" which can be larger than available RAM+swap.
> 
> Sounds quite useful, although quite similar to MetaKit. (I suppose I
> could use some functions from numarray on columns in MetaKit... But
> that might just be too weird -- and it would still just be a
> collection of columns :])

I really like MetaKit's column-based storage, but it just doesn't scale
well (on the author's admission, and verified empirically) - beyond a
few 10**5 records, it bogs down terribly, whereas memory-mapped NumPy
plus BsdDb3 recno databse for strings scales well to many tens of
millions of records (or more, but thats as far as I have tested).

Tim C





From magnus at hetland.org  Fri Dec 27 16:07:03 2002
From: magnus at hetland.org (Magnus Lie Hetland)
Date: Fri Dec 27 16:07:03 2002
Subject: [Numpy-discussion] Non-numerical info associated with sub-arrays
In-Reply-To: <1041108637.1260.153.camel@localhost.localdomain>
References: <20021227212942.GA11897@idi.ntnu.no> <1041102817.1264.142.camel@localhost.localdomain> <20021227225549.GA21921@idi.ntnu.no> <1041108637.1260.153.camel@localhost.localdomain>
Message-ID: <20021228000628.GA23661@idi.ntnu.no>

Tim Churches <tchur at optushome.com.au>:
[snip]
> Just like this:
> 
> >>> import Numeric
> >>> a = Numeric.array(['a','b','c'],typecode=Numeric.PyObject)
> >>> a
> array([a , b , c ],'O')
> >>>

As you may have noticed from my previous descriptions, I'm using
numarray, not Numeric. I've used this in Numeric before -- I just
can't find the equivalent functionality in numarray :)

[snip]
> I really like MetaKit's column-based storage,

Me too.

> but it just doesn't scale
> well (on the author's admission, and verified empirically)

Yes, you're right.

> - beyond a
> few 10**5 records, it bogs down terribly, whereas memory-mapped NumPy
> plus BsdDb3 recno databse for strings scales well to many tens of
> millions of records (or more, but thats as far as I have tested).

Impressive!

Now this *does* sound interesting... The project I originally posted
about only has a few hundred records, so I'm only considering numarray
for expressiveness/readability there -- performance is not an issue.
But using bsddb and numarray (or Numeric) together like this seems
useful in many applications.

> Tim C

-- 
Magnus Lie Hetland
http://hetland.org



From perry at stsci.edu  Fri Dec 27 17:23:02 2002
From: perry at stsci.edu (Perry Greenfield)
Date: Fri Dec 27 17:23:02 2002
Subject: [Numpy-discussion] Non-numerical info associated with sub-arrays
In-Reply-To: <20021228000628.GA23661@idi.ntnu.no>
Message-ID: <NEBBIJKBMLDBLNCEEFOCCECHCPAA.perry@stsci.edu>

Magnus Lie Hetland writes:
> Tim Churches <tchur at optushome.com.au>:
> [snip]
> > Just like this:
> > 
> > >>> import Numeric
> > >>> a = Numeric.array(['a','b','c'],typecode=Numeric.PyObject)
> > >>> a
> > array([a , b , c ],'O')
> > >>>
> 
> As you may have noticed from my previous descriptions, I'm using
> numarray, not Numeric. I've used this in Numeric before -- I just
> can't find the equivalent functionality in numarray :)
> 
At the moment, PyObject arrays are not supported (mainly because
it hasn't been a priority for our needs yet. But if all one needs
is such an array to hold PyObjects and nothing more (for example, 
we envisioned more sophisticated uses such as apply object methods
to the array and returning arrays of the results) than associative
purposes (and being able to set and get array values), it should
be quite easy to add this capability. In fact one could subclass
NDArray and just define the _get and _setitem methods (I forget
the exact names) and probably customize the __init__ and have
the functionality that you need. I can take a look at it next
week (or if you feel bold, look at NDArray yourself). As with
Numeric, speed is sacrificed when using such arrays. The presumption 
is that one is using Numeric or numarray on such things mainly for
the convenience of the array manipulations, not the kind of 
efficiency that bulk numerical operations provide.

Combining that with RecordArrays may be a bit trickier in the
sense that RecordArrays presume that records use the same buffer
for all data. If one doesn't mind storing PyObject pointers in
that data array, it probably is also fairly simple to extend
it (but I frankly haven't thought this through so I may be wrong
about how easy it is). Doing this may require some thought about
how to pickle such arrays. Of course, one may have a set of arrays
as Tim suggests which also acts like a record array where there is
no single data buffer. Our RecordArrays were intended to map
to data files closely, but other variants are certainly possible.

Perry Greenfield 




From falted at openlc.org  Sat Dec 28 06:42:01 2002
From: falted at openlc.org (Francesc Alted)
Date: Sat Dec 28 06:42:01 2002
Subject: [Numpy-discussion] Non-numerical info associated with sub-arrays
In-Reply-To: <20021227212942.GA11897@idi.ntnu.no>
References: <20021227212942.GA11897@idi.ntnu.no>
Message-ID: <1041085706.3e0db50a682fb@webmail.imk.es>

Mensaje citado por: Magnus Lie Hetland <magnus at hetland.org>:

> I'm working on some two-dimensional tables of data, where some data
> are numerical, while other aren't. I'd like to use numarray's
> numerical capabilities with the numerical parts (columns) while
> keeping the data in each row together. (I'm sure this generalizes to
> more dimensions, and to sub-arrays in general, not just rows.)

You may want to have a look at PyTables (http://pytables.sourceforge.net). 
It's designed to be used in scenarios similar to that you are exposing.

It supports Numeric objects and although columns in tables are not 
automatically converted to Numeric o numarray objects, you can build them on 
the flight easily using its powerful selection capabilities.

It uses HDF5 (http://hdf.ncsa.uiuc.edu/HDF5/) format to save its data, so you 
can read PyTables files in a variety of languages and platforms.

Cheers,

Francesc Alted



From falted at openlc.org  Sat Dec 28 07:11:01 2002
From: falted at openlc.org (Francesc Alted)
Date: Sat Dec 28 07:11:01 2002
Subject: [Numpy-discussion] Non-numerical info associated with sub-arrays
In-Reply-To: <NEBBIJKBMLDBLNCEEFOCCECHCPAA.perry@stsci.edu>
References: <NEBBIJKBMLDBLNCEEFOCCECHCPAA.perry@stsci.edu>
Message-ID: <1041087459.3e0dbbe376d61@webmail.imk.es>

Mensaje citado por: Perry Greenfield <perry at stsci.edu>:
> Our RecordArrays were intended to map
> to data files closely, but other variants are certainly possible.

In fact, I'm thinking of adopting numarray for my pytables project, but I 
don't like the fact that data is not natively aligned inside recarrays, i.e. 
there is not a gap between the different fields even if datatypes doesn't 
match the "native" architecture alignement. IMO this can affect very much to 
the read/write efficency when one wants to work with data rows or columns of 
recarrays objects.

Are there any plans to support this "natural" alignment in addition of the non-
alignment schema present right now?.

Francesc Alted



From magnus at hetland.org  Sat Dec 28 08:14:03 2002
From: magnus at hetland.org (Magnus Lie Hetland)
Date: Sat Dec 28 08:14:03 2002
Subject: [Numpy-discussion] Non-numerical info associated with sub-arrays
In-Reply-To: <1041085706.3e0db50a682fb@webmail.imk.es>
References: <20021227212942.GA11897@idi.ntnu.no> <1041085706.3e0db50a682fb@webmail.imk.es>
Message-ID: <20021228161244.GB11568@idi.ntnu.no>

Francesc Alted <falted at openlc.org>:
[snip]
> You may want to have a look at PyTables (http://pytables.sourceforge.net). 
> It's designed to be used in scenarios similar to that you are exposing.
[snip]

Sounds interesting. I'll look into it.

-- 
Magnus Lie Hetland
http://hetland.org



From edcjones at erols.com  Sat Dec 28 09:52:23 2002
From: edcjones at erols.com (Edward C. Jones)
Date: Sat Dec 28 09:52:23 2002
Subject: [Numpy-discussion] Further comments
Message-ID: <3E0DEC9C.8080103@erols.com>

Should the variable "type" be used in numarray? It is an important 
function in Python.

---------------------------------------

There needs to be a function or method that returns the number of 
elements in an array.

def Elements(array):
     """Number of elements in an array.

     This version is slow.
     """
     return numarray.multiply.reduce(array.shape)

---------------------------------------

I write code using both PIL and numarray. PIL uses strings for modes and
numarray uses (optionally) strings as typecodes. This causes problems.
One fix is to emit a DeprecationWarning when string typecodes are used. 
Two functions are needed: StringTypeWarningOn and StringTypeWarningOff. 
The default should be to ignore this warning.

In my code I use the following workarounds:

def SameType(x, y):
     """Are the two input the same object of NumericType?"""
     if isinstance(x, NumericType) and isinstance(y, NumericType) \
            and x == y:
         return True
     return False

def IsTypeInList(typecode, seq):
     """Is a NumericType object in a list of NumericType objects?"""
     if not isinstance(typecode, NumericType):
         return False
     for item in seq:
         if isinstance(item, NumericType) and typecode == item:
             return True
     return False

---------------------------------------

The following function is useful for downsizing arrays. I suggest that
it should be a ufunc method. This is how I have used reduceat in Numeric.

def blockreduce(array, blocksizes, ufunc):
     """Apply ufunc.reduce to blocks in an array."""
     dims = len(array.shape)
     if type(blocksizes) is IntType:
         blocksizes = dims * [blocksizes]
     if len(blocksizes) != dims:
         raise TypeError, 'blocksizes must be same length as shape'
     for i in range(dims):
         if array.shape[i] % blocksizes[i] != 0:
             raise ValueError, 'blocksizes must exactly divide ' \
                'the corresponding array dimension'
     for i in range(dims):
         array = array.copy()
         newshape = (array.shape[0] / blocksizes[i], blocksizes[i]) + \
                       array.shape[1:]
         array.shape = newshape
         array = ufunc.reduce(array, 1)
         dims = len(array.shape)
         # (0,1,2,3) --> (1,2,3,0)
         perm =  tuple(range(1, dims)) + (0,)
         array = numarray.transpose(array, perm)
     return array




From perry at stsci.edu  Sat Dec 28 11:03:35 2002
From: perry at stsci.edu (Perry Greenfield)
Date: Sat Dec 28 11:03:35 2002
Subject: [Numpy-discussion] Non-numerical info associated with sub-arrays
In-Reply-To: <1041087459.3e0dbbe376d61@webmail.imk.es>
Message-ID: <NEBBIJKBMLDBLNCEEFOCOECHCPAA.perry@stsci.edu>

> In fact, I'm thinking of adopting numarray for my pytables project, but I
> don't like the fact that data is not natively aligned inside
> recarrays, i.e.
> there is not a gap between the different fields even if datatypes doesn't
> match the "native" architecture alignement. IMO this can affect
> very much to
> the read/write efficency when one wants to work with data rows or
> columns of
> recarrays objects.
>
> Are there any plans to support this "natural" alignment in
> addition of the non-
> alignment schema present right now?.
>
Are you asking for an option to create record arrays with
aligned fields (in the sense that the addresses of all values
are consistent with their type)? Or are you arguing that
non-aligned columns must be prohibited? The former is certainly
possible (not not very difficult to implement; basically it requires
that record sizes must be a multiple of the largest numerical type,
and that padding is placed within records to ensure that all fields
have offsets that are a multiple of their size). We cannot accept
the latter since we need to access data that are stored in such
a non-aligned manner in data files.

> Francesc Alted

Thanks, Perry Greenfield





From magnus at hetland.org  Sat Dec 28 13:30:03 2002
From: magnus at hetland.org (Magnus Lie Hetland)
Date: Sat Dec 28 13:30:03 2002
Subject: [Numpy-discussion] from RandomArray2 import *?
Message-ID: <20021228212928.GA19852@idi.ntnu.no>

I just tried a starred import from RandomArray2, but it seems that it
didn't work properly -- e.g. the uniform function wasn't imported. And
I can't see any __all__ attribute; what's going on?

-- 
Magnus Lie Hetland
http://hetland.org



From perry at stsci.edu  Sat Dec 28 13:54:02 2002
From: perry at stsci.edu (Perry Greenfield)
Date: Sat Dec 28 13:54:02 2002
Subject: [Numpy-discussion] Further comments
In-Reply-To: <3E0DEC9C.8080103@erols.com>
Message-ID: <NEBBIJKBMLDBLNCEEFOCIECICPAA.perry@stsci.edu>

Edward Jones writes: 
> 
> Should the variable "type" be used in numarray? It is an important 
> function in Python.
> 
We pondered this for a while. Yes, it conflicts with the built 
in function (and so that function is aliased within the numarray
code to a different name). Do you see this causing problems for 
you? Normally there is no conflict in using it as a keyword name
for function or method arguments, but if you can show practical
cases where it is a problem, we may reconsider. It seemed to us
that type was the best name for the keyword, particularly since
we were discouraging people from thinging of typecodes.

> ---------------------------------------
> 
> There needs to be a function or method that returns the number of 
> elements in an array.
> 
> def Elements(array):
>      """Number of elements in an array.
> 
>      This version is slow.
>      """
>      return numarray.multiply.reduce(array.shape)
> 
> ---------------------------------------
> 
I don't disagree. It would be nice to have a consensus for what the
function name (and perhaps an array attribute?) should be. I believe
IDL uses nelements (which I find more suggestive than elements), but
perhaps others have better names.

> I write code using both PIL and numarray. PIL uses strings for modes and
> numarray uses (optionally) strings as typecodes. This causes problems.
> One fix is to emit a DeprecationWarning when string typecodes are used. 
> Two functions are needed: StringTypeWarningOn and StringTypeWarningOff. 
> The default should be to ignore this warning.
> 
I'm not sure I understand. Can you give me an example of problem
code or usage? It sounds like you are trying to test the types
of  PIL and numarray objects in a generic sense. But I'd understand
better if you could show an example.

> 
> The following function is useful for downsizing arrays. I suggest that
> it should be a ufunc method. This is how I have used reduceat in Numeric.
> 
> def blockreduce(array, blocksizes, ufunc):
>      """Apply ufunc.reduce to blocks in an array."""
>      dims = len(array.shape)
>      if type(blocksizes) is IntType:
>          blocksizes = dims * [blocksizes]
>      if len(blocksizes) != dims:
>          raise TypeError, 'blocksizes must be same length as shape'
>      for i in range(dims):
>          if array.shape[i] % blocksizes[i] != 0:
>              raise ValueError, 'blocksizes must exactly divide ' \
>                 'the corresponding array dimension'
>      for i in range(dims):
>          array = array.copy()
>          newshape = (array.shape[0] / blocksizes[i], blocksizes[i]) + \
>                        array.shape[1:]
>          array.shape = newshape
>          array = ufunc.reduce(array, 1)
>          dims = len(array.shape)
>          # (0,1,2,3) --> (1,2,3,0)
>          perm =  tuple(range(1, dims)) + (0,)
>          array = numarray.transpose(array, perm)
>      return array
> 
We certainly have frequent need for a binning function (i.e.,
the equivalent of a blockreduce for add). Do others see this
as a generally useful extension for all binary ufuncs?
(As a side comment, the requirement that the dimensions be
evenly divisible is often a pain; one question is to ask whether
this must be a requirement or not, or whether there is a way to
disable this requirement, i.e., permit the last block to be smaller
than usual?)

Thanks, Perry Greenfield 




From perry at stsci.edu  Sat Dec 28 14:29:04 2002
From: perry at stsci.edu (Perry Greenfield)
Date: Sat Dec 28 14:29:04 2002
Subject: [Numpy-discussion] from RandomArray2 import *?
In-Reply-To: <20021228212928.GA19852@idi.ntnu.no>
Message-ID: <NEBBIJKBMLDBLNCEEFOCMECJCPAA.perry@stsci.edu>

Hmmm, I see what you mean. Looks like we have something to
fix. Thanks for pointing this out.

Perry

> -----Original Message-----
> From: numpy-discussion-admin at lists.sourceforge.net
> [mailto:numpy-discussion-admin at lists.sourceforge.net]On Behalf Of Magnus
> Lie Hetland
> Sent: Saturday, December 28, 2002 4:29 PM
> To: Numpy-discussion
> Subject: [Numpy-discussion] from RandomArray2 import *?
> 
> 
> I just tried a starred import from RandomArray2, but it seems that it
> didn't work properly -- e.g. the uniform function wasn't imported. And
> I can't see any __all__ attribute; what's going on?
> 
> -- 
> Magnus Lie Hetland
> http://hetland.org
> 
> 
> -------------------------------------------------------
> This sf.net email is sponsored by:ThinkGeek
> Welcome to geek heaven.
> http://thinkgeek.com/sf
> _______________________________________________
> Numpy-discussion mailing list
> Numpy-discussion at lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/numpy-discussion
> 




From magnus at hetland.org  Sat Dec 28 14:29:04 2002
From: magnus at hetland.org (Magnus Lie Hetland)
Date: Sat Dec 28 14:29:04 2002
Subject: [Numpy-discussion] Further comments
In-Reply-To: <NEBBIJKBMLDBLNCEEFOCIECICPAA.perry@stsci.edu>
References: <3E0DEC9C.8080103@erols.com> <NEBBIJKBMLDBLNCEEFOCIECICPAA.perry@stsci.edu>
Message-ID: <20021228222836.GA21002@idi.ntnu.no>

Perry Greenfield <perry at stsci.edu>:
>
> > There needs to be a function or method that returns the number of 
> > elements in an array.
> > 
> > def Elements(array):
> >      """Number of elements in an array.
> > 
> >      This version is slow.
> >      """
> >      return numarray.multiply.reduce(array.shape)

len(ravel(array)) may be a faster in some cases, it seems. (Or
ravel(array).shape[0], for that matter).

BTW: Using the name "array" here seems just as bad as the use of
"type" in numarray... Just a thought :]

[snip]
> > The following function is useful for downsizing arrays. I suggest that
> > it should be a ufunc method. This is how I have used reduceat in Numeric.
> > 
> > def blockreduce(array, blocksizes, ufunc):
> >      """Apply ufunc.reduce to blocks in an array."""
[snip]
> We certainly have frequent need for a binning function (i.e.,
> the equivalent of a blockreduce for add). Do others see this
> as a generally useful extension for all binary ufuncs?

Not sure if I fully understand this -- but in my work on sequences and
time series, I certainly need to do stuff like sums and averages over
fixed-sized blocks of one-dimensional arrays... (Discretization.) But
there are easier ways of doing that, I suppose (with reshape or
indices, for example).

-- 
Magnus Lie Hetland
http://hetland.org



From magnus at hetland.org  Sat Dec 28 14:35:02 2002
From: magnus at hetland.org (Magnus Lie Hetland)
Date: Sat Dec 28 14:35:02 2002
Subject: [Numpy-discussion] Increased polymorphism?
Message-ID: <20021228223435.GA21226@idi.ntnu.no>

I know PyObject arrays aren't supported in numarray yet -- but how
about making the functions more polymorphic? Yes, they work on other
sequence types such as lists -- but there seems to be quite a bit of
type checking going on. I suppose that is necessary for performance
reasons, but how about some polymorphic behaviour in the cases where
an unknown type is encountered?

It would be nice, for example, if sum() worked on general iterators
and generators, and if argsort() worked on, e.g., lists of strings...
This would make it easier to integrate the purely numeric stuff with
other parts of Python programs...

It may be that I'm going beyond the scope of numarray here, but this
sort of agnostic signature-based polymorphism is very Pythonic.

-- 
Magnus Lie Hetland
http://hetland.org



From falted at openlc.org  Sat Dec 28 17:00:02 2002
From: falted at openlc.org (Francesc Alted)
Date: Sat Dec 28 17:00:02 2002
Subject: [Numpy-discussion] Non-numerical info associated with sub-arrays
In-Reply-To: <NEBBIJKBMLDBLNCEEFOCOECHCPAA.perry@stsci.edu>
References: <NEBBIJKBMLDBLNCEEFOCOECHCPAA.perry@stsci.edu>
Message-ID: <1041122789.3e0e45e50b668@webmail.imk.es>

Mensaje citado por: Perry Greenfield <perry at stsci.edu>:

> Are you asking for an option to create record arrays with
> aligned fields (in the sense that the addresses of all values
> are consistent with their type)?

Yes, I'm advocating for that

> Or are you arguing that
> non-aligned columns must be prohibited? The former is certainly
> possible (not not very difficult to implement; basically it requires
> that record sizes must be a multiple of the largest numerical type,
> and that padding is placed within records to ensure that all fields
> have offsets that are a multiple of their size).

Well, for the sake of keeping the size of dataset to a minimum, I think it's 
not necessary to adjust all the record field sizes to the largest data type 
because depending on the type of the field, the padding can be shorter or 
larger.

For example, short ints only needs to be aligned in two-byte basis, while 
doubles need 4 bytes (or 8, I don't remember well). In any case, this depends 
on the architecture. But it is still possible to figure out safely what is the 
required minimum alignments for the different types. Look at Python's struct 
module for a good example on how you can reduce the padding to a minimum, 
without sacrificing performance.

Francesc Alted



From magnus at hetland.org  Sun Dec 29 13:37:02 2002
From: magnus at hetland.org (Magnus Lie Hetland)
Date: Sun Dec 29 13:37:02 2002
Subject: [Numpy-discussion] average()...?
Message-ID: <20021229213556.GA7942@idi.ntnu.no>

Just a quick question (which I believe I have asked before sometime,
but I don't recall the outcome ;) -- why is the average() function
from MaskedArray not available in numarray? It seems like a very
useful function?

It seems that looking through the list of functions in MaskedArray
might be useful in finding candidates for numarray, in so far as they
differ. (Wouldn't be logical if they supported the same functions,
more or less?) For example, MaskedArray has the function count(),
which counts the number of elements in an array -- exactly what was
asked for yesterday, IIRC. (The use may be more obvious for masked
arrays, where it counts non-masked elements, but it works on plain
arrays too.)

If nothing else, perhaps it would be possible to put

from MA import count, average

in numarray.py? (And similarly for other functions that may be missing
in numarray -- I didn't check that closely.) Of course this wouldn't
work if MA isn't available... :/

-- 
Magnus Lie Hetland
http://hetland.org



From magnus at hetland.org  Sun Dec 29 13:54:02 2002
From: magnus at hetland.org (Magnus Lie Hetland)
Date: Sun Dec 29 13:54:02 2002
Subject: [Numpy-discussion] indices with list arg?
Message-ID: <20021229215310.GA8757@idi.ntnu.no>

Why isn't indices([42]) (or foo.setshape([42]), for that matter)
allowed? Overzealous type checking, or a necessity of the C code?

-- 
Magnus Lie Hetland
http://hetland.org



From jmiller at stsci.edu  Mon Dec 30 06:27:06 2002
From: jmiller at stsci.edu (Todd Miller)
Date: Mon Dec 30 06:27:06 2002
Subject: [Numpy-discussion] indices with list arg?
References: <20021229215310.GA8757@idi.ntnu.no>
Message-ID: <3E10593E.8010802@stsci.edu>

Magnus Lie Hetland wrote:

>Why isn't indices([42]) (or foo.setshape([42]), for that matter)
>allowed? Overzealous type checking, or a necessity of the C code?
>
>  
>
Neither.   It  looks like a limitation of the indices function.   
Inserting "shape = tuple(shape)" into indices() removed the limitation.

Thanks,
Todd




From magnus at hetland.org  Mon Dec 30 07:45:05 2002
From: magnus at hetland.org (Magnus Lie Hetland)
Date: Mon Dec 30 07:45:05 2002
Subject: [Numpy-discussion] indices with list arg?
In-Reply-To: <3E10593E.8010802@stsci.edu>
References: <20021229215310.GA8757@idi.ntnu.no> <3E10593E.8010802@stsci.edu>
Message-ID: <20021230154359.GC25685@idi.ntnu.no>

Todd Miller <jmiller at stsci.edu>:
>
[snip]
> Neither.   It  looks like a limitation of the indices function.   
> Inserting "shape = tuple(shape)" into indices() removed the limitation.

Ah.

But it's still impossible to use foo.setshape([bar]), then, I guess,
even though it works with reshape()?

> Thanks,
> Todd

-- 
Magnus Lie Hetland
http://hetland.org



From magnus at hetland.org  Mon Dec 30 08:46:01 2002
From: magnus at hetland.org (Magnus Lie Hetland)
Date: Mon Dec 30 08:46:01 2002
Subject: [Numpy-discussion] Compiling from CVS?
Message-ID: <20021230164459.GA7115@idi.ntnu.no>

I guess I'm just missing something obvious here: How do I compile
numarray from the cvs dist? It seems like files such as numconfig.h
etc. are missing -- it seems to have been generated by setup.py in the
normal distributions, but I'm not sure how to do that... I just
expected "python setup.py build" to work :]

-- 
Magnus Lie Hetland
http://hetland.org



From jmiller at stsci.edu  Mon Dec 30 09:01:07 2002
From: jmiller at stsci.edu (Todd Miller)
Date: Mon Dec 30 09:01:07 2002
Subject: [Numpy-discussion] Compiling from CVS?
References: <20021230164459.GA7115@idi.ntnu.no>
Message-ID: <3E107D52.2070502@stsci.edu>

Magnus Lie Hetland wrote:

>I guess I'm just missing something obvious here: How do I compile
>numarray from the cvs dist? It seems like files such as numconfig.h
>
Not really.  People have asked for modifications to setup.py to support 
a two step build/install process.  CVS has the first cut,  which works, 
but is harder to use than it has to be.

>etc. are missing -- it seems to have been generated by setup.py in the
>normal distributions, but I'm not sure how to do that... I just
>expected "python setup.py build" to work :]
>  
>
Try:

python setup.py build --gencode
python setup.py install

for now.   I plan to revisit this later and eliminate the need for the 
switch.

Todd




From magnus at hetland.org  Mon Dec 30 10:02:02 2002
From: magnus at hetland.org (Magnus Lie Hetland)
Date: Mon Dec 30 10:02:02 2002
Subject: [Numpy-discussion] Yet another bug
Message-ID: <20021230180111.GA8587@idi.ntnu.no>

Hi, there... I just found another bug :]

from numarray import arange
from __future__ import division

arange(10) / 10

... This will give an error.

I really think this "real" division should be supported for arrays...

-- 
Magnus Lie Hetland
http://hetland.org



From jmiller at stsci.edu  Mon Dec 30 10:41:15 2002
From: jmiller at stsci.edu (Todd Miller)
Date: Mon Dec 30 10:41:15 2002
Subject: [Numpy-discussion] Yet another bug
References: <20021230180111.GA8587@idi.ntnu.no>
Message-ID: <3E1094E9.3020002@stsci.edu>

Magnus Lie Hetland wrote:

>Hi, there... I just found another bug :]
>
>  
>
numarray doesn't support "true division" yet.

>from numarray import arange
>from __future__ import division
>
>arange(10) / 10
>
>... This will give an error.
>
>I really think this "real" division should be supported for arrays...
>  
>
It will be.  It just hasn't been implemented yet.

>  
>






From magnus at hetland.org  Mon Dec 30 15:58:02 2002
From: magnus at hetland.org (Magnus Lie Hetland)
Date: Mon Dec 30 15:58:02 2002
Subject: [Numpy-discussion] Bin selection
Message-ID: <20021230235736.GA15420@idi.ntnu.no>

I have a set of limits, e.g. array([0, 35, 45, 55, 75]) and I want to
use these to "classify" a set of numbers (another one-dimensional
array). The "class" is the number of the first limit that is lower
than or equal to the number I want to classify. E.g. I'd classify 17
as 0 and 42 as 1.

My current approach is:

  sum(nums[:,NewAxis] >= lims, dim=-1)

It seems a bit unnecessary to compare each number with all the limits
when O(log(n)) would suffice (the limits are ordered); or even with
O(n) running time, a smarter implementation could get an average of
n/2 comparisons...

Suggestions?

-- 
Magnus Lie Hetland
http://hetland.org



From Chris.Barker at noaa.gov  Mon Dec 30 16:15:02 2002
From: Chris.Barker at noaa.gov (Chris Barker)
Date: Mon Dec 30 16:15:02 2002
Subject: [Numpy-discussion] Possible NetCDF bug...
References: <000001c29cc3$281ed420$6501a8c0@NICKLEBY> <Pine.SOL.4.50.0212052304510.28199-100000@death.OCF.Berkeley.EDU> <200212061138.30065.mclay@nist.gov> <3E036FD9.61BC6CA@noaa.gov>
Message-ID: <3E10DD14.9B2550AA@noaa.gov>

Hi all,

Does anyone here use Konrad Hinsen's NetCDF module? I'm having a problem
with it.


I opened an existing NetCDF file and examine one of it's variables:

file = NetCDF.NetCDFFile("test.cdf")

>>> file.variables['water_u'].typecode()
'f'

The variable appears to contain float data (which I think it does).
However, when I try to get a slice of that data:

array([3199453293L, 3198428385L], 'u')

Numeric appears to think it's an unsigned 32 bit integer variable. I get
the same result if I use file.getValue() as well.

Could this be a Numeric version mismatch? I couldn't find a recommended
version of Numeric on the Scientific site.

RedHat Linux 7.2
Python2.2.1
Numeric 22.0

I've enclosed my test.cdf file, if you want to test it yourself.

Anyone have any ideas?

-Chris






-- 
Christopher Barker, Ph.D.
Oceanographer
                                    		
NOAA/OR&R/HAZMAT         (206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115       (206) 526-6317   main reception

Chris.Barker at noaa.gov



From jmiller at stsci.edu  Tue Dec 31 05:25:04 2002
From: jmiller at stsci.edu (Todd Miller)
Date: Tue Dec 31 05:25:04 2002
Subject: [Numpy-discussion] Bin selection
References: <20021230235736.GA15420@idi.ntnu.no>
Message-ID: <3E119E0E.2010403@stsci.edu>

Magnus Lie Hetland wrote:

>I have a set of limits, e.g. array([0, 35, 45, 55, 75]) and I want to
>use these to "classify" a set of numbers (another one-dimensional
>array). The "class" is the number of the first limit that is lower
>than or equal to the number I want to classify. E.g. I'd classify 17
>as 0 and 42 as 1.
>
>My current approach is:
>
>  sum(nums[:,NewAxis] >= lims, dim=-1)
>
>It seems a bit unnecessary to compare each number with all the limits
>when O(log(n)) would suffice (the limits are ordered); or even with
>O(n) running time, a smarter implementation could get an average of
>n/2 comparisons...
>
>Suggestions?
>
>  
>
Try searchsorted().  Searchsorted returns the index of the first bin >= 
the number being classified and has O(log(n)) running time.

 >>> a=numarray.array([0, 35, 45, 55, 75])
 >>> numarray.searchsorted(a, [1,42,35])
array([1, 2, 1])

Todd




From magnus at hetland.org  Tue Dec 31 06:48:01 2002
From: magnus at hetland.org (Magnus Lie Hetland)
Date: Tue Dec 31 06:48:01 2002
Subject: [Numpy-discussion] Bin selection
In-Reply-To: <3E119E0E.2010403@stsci.edu>
References: <20021230235736.GA15420@idi.ntnu.no> <3E119E0E.2010403@stsci.edu>
Message-ID: <20021231144654.GA987@idi.ntnu.no>

Todd Miller <jmiller at stsci.edu>:
[snip]
> Try searchsorted().

Ah! <slaps forehead> Thanks! That's exactly what I was looking for; I
think I've seen it before, but somehow I missed this when looking for
it...

Thanks again,

Magnus

-- 
Magnus Lie Hetland
http://hetland.org



From magnus at hetland.org  Tue Dec 31 07:29:02 2002
From: magnus at hetland.org (Magnus Lie Hetland)
Date: Tue Dec 31 07:29:02 2002
Subject: [Numpy-discussion] argsort on CharArrays?
Message-ID: <20021231152800.GA1929@idi.ntnu.no>

Is there any way to make argsort work on CharArrays?

-- 
Magnus Lie Hetland
http://hetland.org



From jmiller at stsci.edu  Tue Dec 31 07:44:02 2002
From: jmiller at stsci.edu (Todd Miller)
Date: Tue Dec 31 07:44:02 2002
Subject: [Numpy-discussion] argsort on CharArrays?
References: <20021231152800.GA1929@idi.ntnu.no>
Message-ID: <3E11BEB3.2010602@stsci.edu>

Magnus Lie Hetland wrote:

>Is there any way to make argsort work on CharArrays?
>
>  
>
Not yet.

Todd





From magnus at hetland.org  Tue Dec 31 07:49:03 2002
From: magnus at hetland.org (Magnus Lie Hetland)
Date: Tue Dec 31 07:49:03 2002
Subject: [Numpy-discussion] Lists in fromfile shape?
Message-ID: <20021231154816.GA2308@idi.ntnu.no>

I just noticed that the shape argument in fromfile cannot be an
unhashable sequence. I assume tuples are the norm, but still -- it
would be nice to accept other sequences?

And... I also saw an explicit type check in fromfile, comparing
_type(file) with _type("") -- what if I supplied some other
string-like object? Perhaps a try/except with file+"" would be
appropriate, to check for "string-like-ness" instead of an explicit
type check? (I've just grown suspicious of typechecking in general...)

Sorry if I'm being a pest, nagging about all these details, but I
expect it may be useful to get them resolved? :]

-- 
Magnus Lie Hetland
http://hetland.org



From magnus at hetland.org  Tue Dec 31 07:53:02 2002
From: magnus at hetland.org (Magnus Lie Hetland)
Date: Tue Dec 31 07:53:02 2002
Subject: [Numpy-discussion] Lists in fromfile shape?
In-Reply-To: <20021231154816.GA2308@idi.ntnu.no>
References: <20021231154816.GA2308@idi.ntnu.no>
Message-ID: <20021231155203.GA2478@idi.ntnu.no>

Magnus Lie Hetland <magnus at hetland.org>:
>
> I just noticed that the shape argument in fromfile cannot be an
> unhashable sequence. I assume tuples are the norm, but still -- it
> would be nice to accept other sequences?

Sorry about that -- it worked after all. My mistake.

-- 
Magnus Lie Hetland
http://hetland.org



From jmiller at stsci.edu  Tue Dec 31 10:55:01 2002
From: jmiller at stsci.edu (Todd Miller)
Date: Tue Dec 31 10:55:01 2002
Subject: [Numpy-discussion] argsort on CharArrays?
References: <20021231152800.GA1929@idi.ntnu.no> <3E11BEB3.2010602@stsci.edu> <20021231154901.GB2308@idi.ntnu.no>
Message-ID: <3E11EB49.50106@stsci.edu>

Magnus Lie Hetland wrote:

>Todd Miller <jmiller at stsci.edu>:
>  
>
>>Magnus Lie Hetland wrote:
>>
>>    
>>
>>>Is there any way to make argsort work on CharArrays?
>>>
>>>      
>>>
>>Not yet.
>>    
>>
>
>OK, thanks.
>
>(I don't mean to nag -- I'm just eager about numarray :)
>  
>
Don't worry about nagging.    All this feedback is good.  Results, 
however,  may take a little time.

>>Todd
>>    
>>
Todd





From list at jsaul.de  Mon Dec  2 03:02:17 2002
From: list at jsaul.de (Joachim Saul)
Date: Mon Dec  2 03:02:17 2002
Subject: [Numpy-discussion] Accessing array members
Message-ID: <20021202105922.GA870@jsaul.de>

Hi there,

given the following (simplified) scenario:

typedef struct {
        PyObject_HEAD
        float bar[10];
} FooObject;

I want to be able to set and retrieve the elements of bar from
Python using e.g.

>>> foo = Foo()
>>> foo.bar[4] = 1.23
>>> x = foo.bar[4]

I have chosen an approach using 'PyArray_FromDimsAndData'. In fact
I programmed it after studying 'arrayobject.c', namely the part in
'array_getattr' where the 'flat' attribute is accessed.

foo_getattr(FooObject *self, char *name)
{
        if (!strcmp(name, "bar")) {
                int n=10;
                PyObject *bar = PyArray_FromDimsAndData(1, &n,
                                    PyArray_FLOAT, (char*)self->bar);
                if (bar == NULL) return NULL;
                return bar;
        }

        return Py_FindMethod(foo_methods, (PyObject*)self, name);
}

And it works! :-)

BUT how about refcounts here? 'PyArray_FromDimsAndData' will
return an array which only contains a reference to foo's original
bar array; that's why I can both set and access the latter the way
described. And no memory leak is created.

But what if I create a reference to foo.bar, and later delete foo,
i.e.

>>> b = foo.bar
>>> del foo

Now the data pointer in b refers to freed data! In the mentioned
'array_getattr' this apeears to be solved by increasing the
refcount; in the above example this would mean 'Py_INCREF(self)'
before returning 'bar'. Then if deleting 'foo', its memory is not
freed because the refcount is not zero. But AFAICS in this case
(as well as in the Numeric code) the INCREF prevents the object
from EVER being freed. Who would DECREF the object?

Or am I misunderstanding something here?

In my actual code I can perfectly live with the above solution
because I only need to access foo's data using 'foo.bar[i]' and
probably never need to create a reference to 'bar' which might
survive the actual 'foo' object. However, I want to program it the
'clean' way; any hints on how to do it properly would therefore be
highly welcome.

Cheers,
Joachim



From hinsen at cnrs-orleans.fr  Mon Dec  2 03:42:12 2002
From: hinsen at cnrs-orleans.fr (Konrad Hinsen)
Date: Mon Dec  2 03:42:12 2002
Subject: [Numpy-discussion] Accessing array members
In-Reply-To: <20021202105922.GA870@jsaul.de>
References: <20021202105922.GA870@jsaul.de>
Message-ID: <m3fztgps7m.fsf@chinon.cnrs-orleans.fr>

Joachim Saul <list at jsaul.de> writes:

> But what if I create a reference to foo.bar, and later delete foo,
> i.e.
> 
> >>> b = foo.bar
> >>> del foo
> 
> Now the data pointer in b refers to freed data! In the mentioned

And that is why the condition for using PyArray_FromDimsAndData is
that the data space passed is not freed before the end of the process.

> survive the actual 'foo' object. However, I want to program it the
> 'clean' way; any hints on how to do it properly would therefore be
> highly welcome.

I see only one clean solution: implement your own array-like object
that represents foo.bar. This object would keep a reference to foo
and release it when it is itself destroyed.

Konrad.
-- 
-------------------------------------------------------------------------------
Konrad Hinsen                            | E-Mail: hinsen at cnrs-orleans.fr
Centre de Biophysique Moleculaire (CNRS) | Tel.: +33-2.38.25.56.24
Rue Charles Sadron                       | Fax:  +33-2.38.63.15.17
45071 Orleans Cedex 2                    | Deutsch/Esperanto/English/
France                                   | Nederlands/Francais
-------------------------------------------------------------------------------



From verveer at embl-heidelberg.de  Mon Dec  2 15:28:03 2002
From: verveer at embl-heidelberg.de (verveer at embl-heidelberg.de)
Date: Mon Dec  2 15:28:03 2002
Subject: [Numpy-discussion] coercion of arrays
Message-ID: <1038871565.3debec0d23849@webmail.EMBL-Heidelberg.DE>

Hi,

I was looking at the coercion of arrays in the new numarray. The following is 
not intuitive to me:

>>> a = array(1, Int8)
>>> b = array(1, UInt16)
>>> c = a + b
>>> c.type()
UInt16

Should the result not be a signed integer type?

Also the following result is strange to me:

>>> a = array(1, Float64)
>>> b = array(1, Complex32)
>>> c = a + b
>>> c.type()
Complex32

Complex64 would seem to be the more apropiate type here for the result.

Could somebody comment if these are bugs or not?

Cheers, Peter

--
Dr. Peter J. Verveer
Cell Biology and Cell Biophysics Programme
European Molecular Biology Laboratory
Meyerhofstrasse 1
D-69117 Heidelberg
Germany
Tel. : +49 6221 387245
Fax  : +49 6221 387242
Email: Peter.Verveer at embl-heidelberg.de



 



From oliphant at ee.byu.edu  Mon Dec  2 15:33:01 2002
From: oliphant at ee.byu.edu (Travis Oliphant)
Date: Mon Dec  2 15:33:01 2002
Subject: [Numpy-discussion] Accessing array members
In-Reply-To: <m3fztgps7m.fsf@chinon.cnrs-orleans.fr>
Message-ID: <Pine.LNX.4.33L2.0212021627410.21184-100000@oliphant.ee.byu.edu>

On 2 Dec 2002, Konrad Hinsen wrote:

> Joachim Saul <list at jsaul.de> writes:
>
> > But what if I create a reference to foo.bar, and later delete foo,
> > i.e.
> >
> > >>> b = foo.bar
> > >>> del foo

> >
> > Now the data pointer in b refers to freed data! In the mentioned

Forgive me for jumping in.  But why should the data be deleted when you do
this?  Shouldn't del foo merely decrease the reference count of foo.bar?
Because there are still outstanding references to foo.bar (i.e. b)  then
the data itself shouldn't be freed.

Perhaps I don't understand the question well enough.

-Travis





From perry at stsci.edu  Mon Dec  2 17:21:13 2002
From: perry at stsci.edu (Perry Greenfield)
Date: Mon Dec  2 17:21:13 2002
Subject: [Numpy-discussion] coercion of arrays
In-Reply-To: <1038871565.3debec0d23849@webmail.EMBL-Heidelberg.DE>
Message-ID: <NEBBIJKBMLDBLNCEEFOCOEOHCOAA.perry@stsci.edu>

You are right on both counts. These are indeed coercion
bugs and we will fix them. The first should end up being an
Int32 actually.

Thanks for pointing this out.

Perry Greenfield
> 
> I was looking at the coercion of arrays in the new numarray. The 
> following is 
> not intuitive to me:
> 
> >>> a = array(1, Int8)
> >>> b = array(1, UInt16)
> >>> c = a + b
> >>> c.type()
> UInt16
> 
> Should the result not be a signed integer type?
> 
> Also the following result is strange to me:
> 
> >>> a = array(1, Float64)
> >>> b = array(1, Complex32)
> >>> c = a + b
> >>> c.type()
> Complex32
> 
> Complex64 would seem to be the more apropiate type here for the result.
> 
> Could somebody comment if these are bugs or not?
> 
> Cheers, Peter
> 
> --
> Dr. Peter J. Verveer
> Cell Biology and Cell Biophysics Programme
> European Molecular Biology Laboratory
> Meyerhofstrasse 1
> D-69117 Heidelberg
> Germany
> Tel. : +49 6221 387245
> Fax  : +49 6221 387242
> Email: Peter.Verveer at embl-heidelberg.de
> 




From cookedm at arbutus.physics.mcmaster.ca  Mon Dec  2 17:47:01 2002
From: cookedm at arbutus.physics.mcmaster.ca (David M. Cooke)
Date: Mon Dec  2 17:47:01 2002
Subject: [Numpy-discussion] Accessing array members
In-Reply-To: <20021202105922.GA870@jsaul.de>
References: <20021202105922.GA870@jsaul.de>
Message-ID: <20021203014206.GA3009@arbutus.physics.mcmaster.ca>

On Mon, Dec 02, 2002 at 11:59:22AM +0100, Joachim Saul wrote:
> Hi there,
> 
> given the following (simplified) scenario:
> 
> typedef struct {
>         PyObject_HEAD
>         float bar[10];
> } FooObject;
> 
> I want to be able to set and retrieve the elements of bar from
> Python using e.g.
> 
> >>> foo = Foo()
> >>> foo.bar[4] = 1.23
> >>> x = foo.bar[4]
> 
> I have chosen an approach using 'PyArray_FromDimsAndData'. In fact
> I programmed it after studying 'arrayobject.c', namely the part in
> 'array_getattr' where the 'flat' attribute is accessed.
> 
> foo_getattr(FooObject *self, char *name)
> {
>         if (!strcmp(name, "bar")) {
>                 int n=10;
>                 PyObject *bar = PyArray_FromDimsAndData(1, &n,
>                                     PyArray_FLOAT, (char*)self->bar);
>                 if (bar == NULL) return NULL;
>                 return bar;
>         }
> 
>         return Py_FindMethod(foo_methods, (PyObject*)self, name);
> }
> 
> And it works! :-)
> 
> BUT how about refcounts here? 'PyArray_FromDimsAndData' will
> return an array which only contains a reference to foo's original
> bar array; that's why I can both set and access the latter the way
> described. And no memory leak is created.
> 
> But what if I create a reference to foo.bar, and later delete foo,
> i.e.
> 
> >>> b = foo.bar
> >>> del foo
> 
> Now the data pointer in b refers to freed data! In the mentioned
> 'array_getattr' this apeears to be solved by increasing the
> refcount; in the above example this would mean 'Py_INCREF(self)'
> before returning 'bar'. Then if deleting 'foo', its memory is not
> freed because the refcount is not zero. But AFAICS in this case
> (as well as in the Numeric code) the INCREF prevents the object
> from EVER being freed. Who would DECREF the object?

Something similiar came up a few weeks ago: how do you pass data owned
by something else as a Numeric array, while keeping track of when to
delete the data?

It's so simple I almost kicked myself when I saw it, from the code at
http://pobox.com/~kragen/sw/arrayfrombuffer/
which allows you to use memory-mapped files as arrays.

The idea is that a PyArrayObject has a member 'base', which is DECREF'd
when the array is deallocated. The idea is for when arrays are slices of
other arrays, deallocating the slice will decrease the reference count
of the original. However, we can subvert this by using our own base,
that knows how to deallocate our data.

In your case, the DECREF'ing is all you need, so you could use

foo_getattr(FooObject *self, char *name)
{
        if (!strcmp(name, "bar")) {
                int n=10;
                PyObject *bar = PyArray_FromDimsAndData(1, &n,
                                    PyArray_FLOAT, (char*)self->bar);
                if (bar == NULL) return NULL;
                /***** new stuff here *******/
                Py_INCREF(self);
                ((PyArrayObject *)bar)->base = self;
                /***********/
                return bar;
        }

        return Py_FindMethod(foo_methods, (PyObject*)self, name);
}

So, now with
>>> b = foo.bar
>>> del foo
b will still reference the original foo object. Now, do
>>> del b
and foo's data will be DECREF'd, freeing it if b had the only reference.

This can be extended: say you've allocated memory from some memory pool
that has to be freed with, say, 'my_pool_free'. You can create a Numeric
array from this without copying by

PyArrayObject *A = (PyArrayObject *)PyArray_FromDimsAndData(1, dims,
                                      PyArray_DOUBLE, (char *)data);
A->base = PyCObject_FromVoidPtr(data, my_pool_free);

Then A will be a PyArrayObject, that, when the last reference is
deleted, will DECREF A->base, which will free the memory.

Easy, huh?

-- 
|>|\/|<
/--------------------------------------------------------------------------\
|David M. Cooke                 http://arbutus.physics.mcmaster.ca/cookedm/
|cookedm at physics.mcmaster.ca



From hinsen at cnrs-orleans.fr  Tue Dec  3 02:07:06 2002
From: hinsen at cnrs-orleans.fr (Konrad Hinsen)
Date: Tue Dec  3 02:07:06 2002
Subject: [Numpy-discussion] Accessing array members
In-Reply-To: <20021203014206.GA3009@arbutus.physics.mcmaster.ca>
References: <20021202105922.GA870@jsaul.de>
	<20021203014206.GA3009@arbutus.physics.mcmaster.ca>
Message-ID: <m3isyba07r.fsf@chinon.cnrs-orleans.fr>

cookedm at arbutus.physics.mcmaster.ca (David M. Cooke) writes:

> The idea is that a PyArrayObject has a member 'base', which is DECREF'd
> when the array is deallocated. The idea is for when arrays are slices of

Indeed, but this is an undocumented implementation feature. Use at your
own risk.

Konrad.
-- 
-------------------------------------------------------------------------------
Konrad Hinsen                            | E-Mail: hinsen at cnrs-orleans.fr
Centre de Biophysique Moleculaire (CNRS) | Tel.: +33-2.38.25.56.24
Rue Charles Sadron                       | Fax:  +33-2.38.63.15.17
45071 Orleans Cedex 2                    | Deutsch/Esperanto/English/
France                                   | Nederlands/Francais
-------------------------------------------------------------------------------



From haase at msg.ucsf.edu  Tue Dec  3 14:35:02 2002
From: haase at msg.ucsf.edu (Sebastian Haase)
Date: Tue Dec  3 14:35:02 2002
Subject: [Numpy-discussion] howto build numarray 0.4 ? file 'Src/_convmodule.c' does not exist
Message-ID: <030301c29b1c$16db8530$3b45da80@rodan>

Hi all,
I downloaded numarray 0.4 about 5 minutes after I got the announce but 
my naive 'python2.2 ./setup.py build'   gives this

haase at baboon:~/numarray-0.4: python2.2 ./setup.py build
running build
running build_py
not copying Lib/ndarray.py (output up-to-date)
not copying Lib/numtest.py (output up-to-date)
not copying Lib/codegenerator.py (output up-to-date)
not copying Lib/ufunc.py (output up-to-date)
not copying Lib/testdata.py (output up-to-date)
not copying Lib/numarray.py (output up-to-date)
not copying Lib/ieeespecial.py (output up-to-date)
not copying Lib/recarray.py (output up-to-date)
not copying Lib/template.py (output up-to-date)
not copying Lib/arrayprint.py (output up-to-date)
not copying Lib/typeconv.py (output up-to-date)
not copying Lib/numinclude.py (output up-to-date)
not copying Lib/numarrayext.py (output up-to-date)
not copying Lib/_ufunc.py (output up-to-date)
not copying Lib/chararray.py (output up-to-date)
not copying Lib/numtestall.py (output up-to-date)
not copying Lib/numerictypes.py (output up-to-date)
not copying Lib/memmap.py (output up-to-date)
running build_ext
building '_conv' extension
error: file 'Src/_convmodule.c' does not exist

What am I missing? I'm running Linux (debian woody) on i386.

Thanks,
Sebastian





From cookedm at arbutus.physics.mcmaster.ca  Tue Dec  3 14:40:02 2002
From: cookedm at arbutus.physics.mcmaster.ca (David M. Cooke)
Date: Tue Dec  3 14:40:02 2002
Subject: [Numpy-discussion] Accessing array members
In-Reply-To: <m3isyba07r.fsf@chinon.cnrs-orleans.fr>
References: <20021202105922.GA870@jsaul.de> <20021203014206.GA3009@arbutus.physics.mcmaster.ca> <m3isyba07r.fsf@chinon.cnrs-orleans.fr>
Message-ID: <20021203223516.GA22325@arbutus.physics.mcmaster.ca>

On Tue, Dec 03, 2002 at 11:03:20AM +0100, Konrad Hinsen wrote:
> cookedm at arbutus.physics.mcmaster.ca (David M. Cooke) writes:
> 
> > The idea is that a PyArrayObject has a member 'base', which is DECREF'd
> > when the array is deallocated. The idea is for when arrays are slices of
> 
> Indeed, but this is an undocumented implementation feature. Use at your
> own risk.

Nope, documented implementation feature. From the C API documentation,

  PyObject  * base

  Used internally in arrays that are created as slices of other arrays.
  Since the new array shares its data area with the old one, the original
  array's reference count is incremented. When the subarray is garbage
  collected, the base array's reference count is decremented.

Looking through Numeric's code, nothing requires base to be an array
object.  Besides, Numeric isn't going to change substantially before
Numarray replaces it (although I don't know the analogue of this trick
in Numarray).

The usefulness of this trick (IMHO) outweighs the small chance of the
interface changing.

-- 
|>|\/|<
/--------------------------------------------------------------------------\
|David M. Cooke                 http://arbutus.physics.mcmaster.ca/cookedm/
|cookedm at physics.mcmaster.ca



From cookedm at arbutus.physics.mcmaster.ca  Tue Dec  3 14:43:05 2002
From: cookedm at arbutus.physics.mcmaster.ca (David M. Cooke)
Date: Tue Dec  3 14:43:05 2002
Subject: [Numpy-discussion] howto build numarray 0.4 ? file 'Src/_convmodule.c' does not exist
In-Reply-To: <030301c29b1c$16db8530$3b45da80@rodan>
References: <030301c29b1c$16db8530$3b45da80@rodan>
Message-ID: <20021203223837.GB22325@arbutus.physics.mcmaster.ca>

On Tue, Dec 03, 2002 at 02:34:06PM -0800, Sebastian Haase wrote:
> Hi all,
> I downloaded numarray 0.4 about 5 minutes after I got the announce but 
> my naive 'python2.2 ./setup.py build'   gives this
> 
> haase at baboon:~/numarray-0.4: python2.2 ./setup.py build
> running build
> running build_py
> not copying Lib/ndarray.py (output up-to-date)
[...]
> not copying Lib/memmap.py (output up-to-date)
> running build_ext
> building '_conv' extension
> error: file 'Src/_convmodule.c' does not exist
> 
> What am I missing? I'm running Linux (debian woody) on i386.

Looks like you have to run 'python2.2 ./setup.py install' instead. Looking at
setup.py, something special is done when the target is 'install'.

[I think this is a bad idea, as I like to build stuff as my user, and
install as root. This requires me to build it as root.]

-- 
|>|\/|<
/--------------------------------------------------------------------------\
|David M. Cooke                 http://arbutus.physics.mcmaster.ca/cookedm/
|cookedm at physics.mcmaster.ca



From haase at msg.ucsf.edu  Tue Dec  3 15:09:04 2002
From: haase at msg.ucsf.edu (Sebastian Haase)
Date: Tue Dec  3 15:09:04 2002
Subject: [Numpy-discussion] howto build numarray 0.4 ? file 'Src/_convmodule.c' does not exist
References: <030301c29b1c$16db8530$3b45da80@rodan> <20021203223837.GB22325@arbutus.physics.mcmaster.ca>
Message-ID: <032c01c29b20$d6b79750$3b45da80@rodan>

> On Tue, Dec 03, 2002 at 02:34:06PM -0800, Sebastian Haase wrote:
> > Hi all,
> > I downloaded numarray 0.4 about 5 minutes after I got the announce but
> > my naive 'python2.2 ./setup.py build'   gives this
> >
> > haase at baboon:~/numarray-0.4: python2.2 ./setup.py build
> > running build
> > running build_py
> > not copying Lib/ndarray.py (output up-to-date)
> [...]
> > not copying Lib/memmap.py (output up-to-date)
> > running build_ext
> > building '_conv' extension
> > error: file 'Src/_convmodule.c' does not exist
> >
> > What am I missing? I'm running Linux (debian woody) on i386.
>
> Looks like you have to run 'python2.2 ./setup.py install' instead. Looking
at
> setup.py, something special is done when the target is 'install'.
>
> [I think this is a bad idea, as I like to build stuff as my user, and
> install as root. This requires me to build it as root.]
>

Thanks for the hint - I'll try that.
But I would consider that a bug, right?

Sebastian





From haase at msg.ucsf.edu  Tue Dec  3 15:14:08 2002
From: haase at msg.ucsf.edu (Sebastian Haase)
Date: Tue Dec  3 15:14:08 2002
Subject: [Numpy-discussion] howto build numarray 0.4 ? file 'Src/_convmodule.c' does not exist
References: <030301c29b1c$16db8530$3b45da80@rodan> <20021203223837.GB22325@arbutus.physics.mcmaster.ca>
Message-ID: <033f01c29b21$9b610cd0$3b45da80@rodan>

> On Tue, Dec 03, 2002 at 02:34:06PM -0800, Sebastian Haase wrote:
> > Hi all,
> > I downloaded numarray 0.4 about 5 minutes after I got the announce but
> > my naive 'python2.2 ./setup.py build'   gives this
> >
> > haase at baboon:~/numarray-0.4: python2.2 ./setup.py build
> > running build
> > running build_py
> > not copying Lib/ndarray.py (output up-to-date)
> [...]
> > not copying Lib/memmap.py (output up-to-date)
> > running build_ext
> > building '_conv' extension
> > error: file 'Src/_convmodule.c' does not exist
> >
> > What am I missing? I'm running Linux (debian woody) on i386.
>
> Looks like you have to run 'python2.2 ./setup.py install' instead. Looking
at
> setup.py, something special is done when the target is 'install'.
>
> [I think this is a bad idea, as I like to build stuff as my user, and
> install as root. This requires me to build it as root.]

I just tried it (actually as user not root!!) and it runs through up to a
"cannot create directory..." error.
That's probably fine with me (for now) and I'm just setting PYTHONPATH ...

Thanks again, Sebastian.






From paul at pfdubois.com  Thu Dec  5 17:03:18 2002
From: paul at pfdubois.com (Paul F Dubois)
Date: Thu Dec  5 17:03:18 2002
Subject: [Numpy-discussion] PEP 242 Numeric kinds
Message-ID: <000001c29cc3$281ed420$6501a8c0@NICKLEBY>

We had put off consideration of PEP 242 Numeric kinds until June 2002
when a meeting of some interested parties was to occur but the meeting
didn't occur. I have a draft implementation and users of it but my
feeling is that although correct and useful the PEP is not useful enough
to put it in the standard library. Since it really only interests
scientific programmers I propose simply making it a separate deliverable
on the Numeric site and withdrawing the PEP.




From j_r_fonseca at yahoo.co.uk  Thu Dec  5 18:38:09 2002
From: j_r_fonseca at yahoo.co.uk (=?iso-8859-15?Q?Jos=E9?= Fonseca)
Date: Thu Dec  5 18:38:09 2002
Subject: [Numpy-discussion] PEP 242 Numeric kinds
In-Reply-To: <000001c29cc3$281ed420$6501a8c0@NICKLEBY>
References: <000001c29cc3$281ed420$6501a8c0@NICKLEBY>
Message-ID: <20021206023728.GA24182@localhost.localdomain>

On Thu, Dec 05, 2002 at 05:02:26PM -0800, Paul F Dubois wrote:
> Since it really only interests
> scientific programmers I propose simply making it a separate deliverable
> on the Numeric site and withdrawing the PEP.

I'm not fully aware about the advantages/implications of puting Numeric
in Python standard library, but to say that the manipulation of numeric
arrays is only of interest to scientific programmers is the same of when
in the early computing days engineers saing that computers would only be
good for crunching numbers, and that the concept of personal computers
was just non-sense...

For a non-scientic usage of Numeric see the examples in
http://www.pygame.org/pcr/repository.php, but I can imagine the
usefullness of Numeric in many more non-scientific applications:
imaging, sound visualization plugins, 3D graphics, and probably much
more.

The use of Numeric can speed up alot of algorithms which would be
otherwise very slow in pure Python, and therefore forcing one to write C
extensions. That's why IMHO something with the Numeric functionality
should exist in the Python standard library.

Note that this is not the same that saying that Numeric should be
included as is - perhaps it's better to have it seperately to let it
mature more, perhaps not - but still, there is much more than a niche
interest around it.

Jos? Fonseca
__________________________________________________
Do You Yahoo!?
Everything you'll ever need on one web page
from News and Sport to Email and Music Charts
http://uk.my.yahoo.com



From python at rcn.com  Thu Dec  5 22:57:03 2002
From: python at rcn.com (Raymond Hettinger)
Date: Thu Dec  5 22:57:03 2002
Subject: [Numpy-discussion] Re: [Python-Dev] PEP 242 Numeric kinds
References: <000001c29cc3$281ed420$6501a8c0@NICKLEBY>
Message-ID: <002701c29cf4$98296c40$770ca044@oemcomputer>

> I have a draft implementation and users of it but my
> feeling is that although correct and useful the PEP is not useful enough
> to put it in the standard library. Since it really only interests
> scientific programmers I propose simply making it a separate deliverable
> on the Numeric site and withdrawing the PEP.

As the PEP author, your feeling is probably the most accurate.
And, it is true that Kinds will, in general, appeal to the same crowd 
as Numeric.

OTOH, "import kinds" is entirely unobtrusive and general purpose.
The interface is simple; and may help some code become less platform
sensitive (in the way the Metafont writes its own portable routines to 
create consistent lettering); and it has some educational value (making
it easy to demonstrate the effects of truncation and floating point
round-off).  

Also, I think accounting applications can benefit from having some
positive assurance that multi-billion dollar calculations don't lose 
their pennies and that the books will still balance.

So, the only downside is having an additional block of code to 
maintain.  My vote is for adding it to the standard library.  Control
over precision is more of a core capability than it is an extension,
but, as the PEP author and principal user, you probably know best.

'nuff said,


Raymond Hettinger


#################################################################
#################################################################
#################################################################
#####
#####
#####
#################################################################
#################################################################
#################################################################



From bac at OCF.Berkeley.EDU  Thu Dec  5 23:13:04 2002
From: bac at OCF.Berkeley.EDU (Brett Cannon)
Date: Thu Dec  5 23:13:04 2002
Subject: [Numpy-discussion] Re: [Python-Dev] PEP 242 Numeric kinds
In-Reply-To: <000001c29cc3$281ed420$6501a8c0@NICKLEBY>
References: <000001c29cc3$281ed420$6501a8c0@NICKLEBY>
Message-ID: <Pine.SOL.4.50.0212052304510.28199-100000@death.OCF.Berkeley.EDU>

[Paul F Dubois]

> We had put off consideration of PEP 242 Numeric kinds until June 2002
> when a meeting of some interested parties was to occur but the meeting
> didn't occur. I have a draft implementation and users of it but my
> feeling is that although correct and useful the PEP is not useful enough
> to put it in the standard library. Since it really only interests
> scientific programmers I propose simply making it a separate deliverable
> on the Numeric site and withdrawing the PEP.
>

I say add it.  If there are already users out there then that demonstrates
as least some form of a demand.

Besides, if rationals can get into the library (that module was finally
accepted to be added to the library, right?) then a module to help do
consistent decimal math should at least be included at least until the
Python core can move to C99 (if I am remembering a comment from Tim saying
that C99 adds more explicit numeric types, e.g., exactly 2 byte float).

I just want to be able to add up a bunch of money values and not have a
few pennies disappear in the end; usually I am the one who has to eat the
loss and I am an poor college graduate who can't afford the rounding.  =)

-Brett



From mclay at nist.gov  Fri Dec  6 08:40:01 2002
From: mclay at nist.gov (Michael McLay)
Date: Fri Dec  6 08:40:01 2002
Subject: [Numpy-discussion] Re: [Python-Dev] PEP 242 Numeric kinds
In-Reply-To: <Pine.SOL.4.50.0212052304510.28199-100000@death.OCF.Berkeley.EDU>
References: <000001c29cc3$281ed420$6501a8c0@NICKLEBY> <Pine.SOL.4.50.0212052304510.28199-100000@death.OCF.Berkeley.EDU>
Message-ID: <200212061138.30065.mclay@nist.gov>

On Friday 06 December 2002 02:12 am, Brett Cannon wrote:
> [Paul F Dubois]
>
> > We had put off consideration of PEP 242 Numeric kinds until June 2002
> > when a meeting of some interested parties was to occur but the meeting
> > didn't occur. 
[...] 
> I just want to be able to add up a bunch of money values and not have a
> few pennies disappear in the end; usually I am the one who has to eat the
> loss and I am an poor college graduate who can't afford the rounding.  =)

Does the kinds implementation work with binary floats and decimal numbers? 
>From a quick scan of the PEP it looks like it is only binary floats. If it 
only applies to binary numbers then the kinds capability will not eliminate 
decimal number rounding errors. That problem will require extending Python to 
include a decimal number type. 

The abstract mentions a clarification of decimal literals, but I suspect Paul 
was not suggesting that the numbers defined by kinds will use decimal 
arithmetic instead of binary arithmetic.



From Chris.Barker at noaa.gov  Fri Dec  6 10:10:04 2002
From: Chris.Barker at noaa.gov (Chris Barker)
Date: Fri Dec  6 10:10:04 2002
Subject: [Numpy-discussion] PEP 242 Numeric kinds
References: <000001c29cc3$281ed420$6501a8c0@NICKLEBY> <20021206023728.GA24182@localhost.localdomain>
Message-ID: <3DF0DF06.835F72A3@noaa.gov>

Jos? Fonseca wrote:
> to say that the manipulation of numeric
> arrays is only of interest to scientific programmers is the same of when
> in the early computing days engineers saing that computers would only be
> good for crunching numbers, and that the concept of personal computers
> was just non-sense...

I absolutely concur.
 
> For a non-scientic usage of Numeric see the examples in
> http://www.pygame.org/pcr/repository.php, but I can imagine the
> usefullness of Numeric in many more non-scientific applications:
> imaging, sound visualization plugins, 3D graphics, and probably much
> more.

It goes MUCH farther than this. All these examples are what I would call
serious number crunching. Perhaps not strictly scientific, but certainly
the realm of numeric programming, and all places where the "kinds"
concept would be useful.

However, there is a great deal of usefulness in Numeric for applications
that are not doing a lot of number crunching. Performance is only one
reason to use Numeric. The other reason is much cleaner and easier
syntax for manipulating arrays of numbers, especially ones of more than
one dimension. Being able to slice across multiple dimensions, array
oriented arithmetic, overloaded comparisons, etc. I use Numeric for
virtually every program I write, whether performance is an issue or not.
Not having to write all those ugly, confusing and error prone loops is a
godsend.

All that being said, the "kinds" concept is probably mostly of concern
to the Number crunching community. However, it is also sort of a low
level concept, and having it part of core language makes sense, just
like it makes sense to have complex number and long integers in the core
language.

-Chris



-- 
Christopher Barker, Ph.D.
Oceanographer
                                    		
NOAA/OR&R/HAZMAT         (206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115       (206) 526-6317   main reception

Chris.Barker at noaa.gov



From rob at pythonemproject.com  Fri Dec  6 15:15:02 2002
From: rob at pythonemproject.com (Rob)
Date: Fri Dec  6 15:15:02 2002
Subject: [Numpy-discussion] Numeric Python EM Project supposed to be in Dec IEEE A & P Magazine
Message-ID: <3DF12E52.5E952B17@pythonemproject.com>

It should be in the educational corner.  I've beefed up the intro to the
site.  Will be interesting to see if I get many new hits.  Rob.
-- 
-----------------------------
The Numeric Python EM Project

www.pythonemproject.com



From rob at pythonemproject.com  Sat Dec  7 08:40:04 2002
From: rob at pythonemproject.com (Rob)
Date: Sat Dec  7 08:40:04 2002
Subject: [Numpy-discussion] Numeric Python EM Project supposed to be in Dec 
 IEEE A & P Magazine
References: <3DF12E52.5E952B17@pythonemproject.com>
Message-ID: <3DF22343.2776D978@pythonemproject.com>

Rob wrote:
> 
> It should be in the educational corner.  I've beefed up the intro to the
> site.  Will be interesting to see if I get many new hits.  Rob.
> --
> -----------------------------
> The Numeric Python EM Project
> 
> www.pythonemproject.com
> 
> -------------------------------------------------------
> This sf.net email is sponsored by:ThinkGeek
> Welcome to geek heaven.
> http://thinkgeek.com/sf
> _______________________________________________
> Numpy-discussion mailing list
> Numpy-discussion at lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/numpy-discussion

It was supposed to be in A & P magazine all the way back in June, but
the editors goofed up, and profusely apologized :)  Rob.

-- 
-----------------------------
The Numeric Python EM Project

www.pythonemproject.com



From pearu at cens.ioc.ee  Sun Dec  8 09:24:01 2002
From: pearu at cens.ioc.ee (Pearu Peterson)
Date: Sun Dec  8 09:24:01 2002
Subject: [Numpy-discussion] ANN: F2PY - Fortran to Python Interface Generator - 6th Release
Message-ID: <Pine.LNX.4.21.0212081921200.20312-100000@cens.kybi>

F2PY - Fortran to Python Interface Generator, Version 2.32.225
---------------------------------------------------------------

I am pleased to announce the sixth public release of F2PY. The purpose of
the F2PY project is to provide a connection between Python and Fortran
programming languages. For more information, see

  http://cens.ioc.ee/projects/f2py2e/

Download source:

  http://cens.ioc.ee/projects/f2py2e/2.x/F2PY-2-latest.tar.gz

What's new?
------------

It has been almost a year from the previous release of F2PY. This release
comes with a number of improvements, most important ones are listed as
follows:

- The issue with a Fortran and C multi-dimensional array storage ordering
  is finally resolved. F2PY generated wrappers automatically carry out all
  necessary transformations, trying hard to minimize any performance hits.
  As a result, the end users of F2PY generated wrappers need not to worry
  about this issue anymore, multi-dimensional arrays in Python and Fortran
  have now exactly the same signatures.

- Improved wrapping C functions with F2PY.

- F2PY Users Guide has been throughly revised to describe and illustrate
  all latest F2PY features such as wrapping Fortran 77 COMMON blocks,
  Fortran 90 module data, including Fortran 90 module ALLOCATABLE arrays,
  etc. The F2PY Users Guide is composed using the Python Docutils tool and
  is available here:

    http://cens.ioc.ee/projects/f2py2e/usersguide/

- F2PY has new site for unit testing.

- F2PY uses scipy_distutils from SciPy (www.scipy.org) project for
  compiling Fortran sources and building extension modules.
  Currently, the following Fortran 77/90 compilers are described by
  scipy_distutils:

    Absoft Sun SGI Intel Itanium NAG Compaq Digital Gnu VAST

- Finally, F2PY has been extensively used/tested for wrapping large
  Fortran/C libraries, such as, LAPACK, BLAS, ATLAS, FFTW, FFTPACK,
  etc. (see SciPy project for more information). This experience has been
  a very important source for ideas how to make binding Fortran/C codes to
  Python easier and more robust. 

Enjoy,
	Pearu Peterson

---------------
<P><A HREF="http://cens.ioc.ee/projects/f2py2e/">F2PY 2.32.225</A> - The
Fortran to Python Interface Generator (08-Dec-02)





From perry at stsci.edu  Mon Dec  9 07:40:04 2002
From: perry at stsci.edu (Perry Greenfield)
Date: Mon Dec  9 07:40:04 2002
Subject: [Numpy-discussion] numarray equality testing
Message-ID: <NEBBIJKBMLDBLNCEEFOCIEPLCOAA.perry@stsci.edu>

This is a Numeric/numarray compatibility question.

Numeric currently allows the equals ufunc to compare 
arrays of unequal sizes and returns a scalar 0 in such
cases. When arrays have consistent shapes, an array of
ints is returned. We argue that this is inconsistent with
normal ufunc behavior and that it should generate an
exception as do all non-equality ufuncs. (numarray currently
does not allow comparison of shape-inconsistent arrays
including for equality). 

Instead we propose a function whose purpose is to determine
if two arrays are shape consistent (i.e., can be broadcast
against each other) and have all values equal to each other.

>>> array_equal(arange(2), arange(4))
0
>>> array_equal(array([1,2,3]), array([1,2,0]))
0
>>> array_equal( arange(2), None )
0
>>> array_equal( arange(2), not_an_ndarray_instance)
0
>>> array_equal(array([[1,2,3],[1,2,3]]), array([1,2,3]))
1

Comments?

Perry Greenfield




From perry at stsci.edu  Mon Dec  9 07:41:02 2002
From: perry at stsci.edu (Perry Greenfield)
Date: Mon Dec  9 07:41:02 2002
Subject: [Numpy-discussion] numarray complex comparisons
Message-ID: <NEBBIJKBMLDBLNCEEFOCMEPLCOAA.perry@stsci.edu>

The questions just keep coming...

We need to decide whether or not complex comparisons work.
They do not work for Python scalars. Consistency would argue
for them not working for numarray arrays. However some argue:

1) not allowing them defeats more generic programming. We agreed
until we found that IDL doesn't support them either, and we never
noticed. We are skeptical of this claim and would like to see
real-life examples.

2) it is useful to allow comparisons since that would result in
repeatable, sorting of values (e.g., to find duplicate values)
for ordering purposes. Cannot this just be handled by the sort
routines themselves? Why must this result in comparison operators
working?

In the absence of good examples for 1) or good arguments for 2)
we propose to make complex comparisons generate exceptions.

Perry Greenfield




From perry at stsci.edu  Mon Dec  9 07:41:03 2002
From: perry at stsci.edu (Perry Greenfield)
Date: Mon Dec  9 07:41:03 2002
Subject: [Numpy-discussion] numarray rank-0 array issues
Message-ID: <NEBBIJKBMLDBLNCEEFOCAEPMCOAA.perry@stsci.edu>

A little while ago we tackled the issue of whether indexing numarray
arrays should return Python scalars or rank-0 arrays. For reasons
gone into at great length previously on this list we decided to
always return Python scalars when no precision is lost by conversion to
Python scalars (Float128 may be an exception to this rule, but we'll
deal with that later). We consider this issue closed. But there
are some loose ends.

There was an opinion that reduction operations (e.g., add.reduce)
should always return arrays. However, some desired that the rank-0
arrays returned when reducing a rank-1 array should be indexable
and have a length. In particular:

>>> x = arange(10)
>>> v = add.reduce(x)
>>> print v
55
>>> v
array(55)
>>> len(v)
1
>>> v[0]
55

The great majority objected to len(rank-0) being allowed
as well as rank-0[0] working. We agree.

We propose that at least one version of the reduction operations
always return arrays, but that the array "end-point" always
be a rank-1 len-1 array instead of a rank-0 array. This is because
a rank-1 len-1 array 

1) has a len() = 1
2) can be indexed.
3) because of broadcasting, behaves like a scalar in expressions
   with arrays (i.e., as was intended for rank-0 arrays)

Thus, if one repeatedly reduces an N-dimensional array, one
eventually gets a rank-1 len-1 array, and reducing a rank-1
len-1 array  simply generates a new rank-1 len-1 array with the
same value.

>>> x = arange(10)
>>> v = add.areduce(x)
>>> v
array([55])
>>> add.areduce(v)
array([55])

Is there any reason why this behavior would not serve the purposes
of those that wanted rank-0 arrays returned from reduction operations?

Likewise, we could provide a function to wrap scalars as rank-1 len-1
arrays (much like array(5) produces a rank-0 array) for the purposes
of more generic functions and routines so that checks for scalars
do not need to be made. This function would convert scalars to arrays,
but simply return the arrays themselves if passed as arguments.

One possibility is that we eliminate rank-0 arrays and use rank-1 len-1
arrays in their place (e.g., array(5) returns a rank-1 len-1 array).
The problem is that there may be some who already dependon rank-0
properties and this probably will break existing code. Any opinions?

Finally, the default reduction operation (i.e., add.reduce) could 
be given the behavior described above. We are inclined to leave it
as is, i.e., to return scalars when reducing 1-d arrays and provide
a different operator method (areduce) to always return arrays. Any
disagreements?

Perry Greenfield



From pearu at cens.ioc.ee  Mon Dec  9 08:13:04 2002
From: pearu at cens.ioc.ee (Pearu Peterson)
Date: Mon Dec  9 08:13:04 2002
Subject: [Numpy-discussion] Re: [SciPy-user] numarray equality testing
In-Reply-To: <NEBBIJKBMLDBLNCEEFOCIEPLCOAA.perry@stsci.edu>
Message-ID: <Pine.LNX.4.21.0212091801410.6643-100000@cens.kybi>

On Mon, 9 Dec 2002, Perry Greenfield wrote:

> >>> array_equal(array([[1,2,3],[1,2,3]]), array([1,2,3]))
> 1

Hmm, I would have expected 0. 
What's the rationale for 1?

May be you meant

>>> array_equal(array([[1,2,3]]), array([1,2,3]))
1

which I would agree. Also

>>> array_equal(array([[1],[2],[3]]), array([1,2,3]))
1

but I am not sure about

>>> array_equal(array([[1,2,3],[1,2,3]]), array([1,2,3,1,2,3]))
1

Pearu





From tim.hochberg at ieee.org  Mon Dec  9 08:15:05 2002
From: tim.hochberg at ieee.org (Tim Hochberg)
Date: Mon Dec  9 08:15:05 2002
Subject: [Numpy-discussion] Re: [SciPy-user] numarray complex comparisons
References: <NEBBIJKBMLDBLNCEEFOCMEPLCOAA.perry@stsci.edu>
Message-ID: <005101c29f9d$ca8dfce0$391e6244@cx781526b>


> The questions just keep coming...
>
> We need to decide whether or not complex comparisons work.
> They do not work for Python scalars. Consistency would argue
> for them not working for numarray arrays. However some argue:
>
> 1) not allowing them defeats more generic programming. We agreed
> until we found that IDL doesn't support them either, and we never
> noticed. We are skeptical of this claim and would like to see
> real-life examples.
>
> 2) it is useful to allow comparisons since that would result in
> repeatable, sorting of values (e.g., to find duplicate values)
> for ordering purposes. Cannot this just be handled by the sort
> routines themselves? Why must this result in comparison operators
> working?
>
> In the absence of good examples for 1) or good arguments for 2)
> we propose to make complex comparisons generate exceptions.

What is the argument against allowing __eq__ to continue to work, while
disallowing __lt__ and friends. The former is well defined. I know comparing
floating point number for equality is a bit of a suckers game, but were
allowing it for floats and it's sometimes useful.

On second thought, maybe I'm just misinterpreting you here since __eq__
works fine for complex scalars.

-tim





From perry at stsci.edu  Mon Dec  9 08:25:01 2002
From: perry at stsci.edu (Perry Greenfield)
Date: Mon Dec  9 08:25:01 2002
Subject: [Numpy-discussion] RE: numarray equality testing
In-Reply-To: <Pine.LNX.4.21.0212091801410.6643-100000@cens.kybi>
Message-ID: <NEBBIJKBMLDBLNCEEFOCEEPMCOAA.perry@stsci.edu>

Pearu Peterson write:

> On Mon, 9 Dec 2002, Perry Greenfield wrote:
> 
> > >>> array_equal(array([[1,2,3],[1,2,3]]), array([1,2,3]))
> > 1
> 
> Hmm, I would have expected 0. 
> What's the rationale for 1?
> 
Because the arrays can be broadcast into a consistent 
array. In other words

equals(array([[1,2,3],[1,2,3]]), array([1,2,3])

returns

array([[1,1,1],[1,1,1]])

But I take your meaning. There may be those that wish
to ensure that two arrays are really identical in shape
and have all equal values. Should these be two different
functions? One function with different options.
By the way, I'm open to better function names as Tim Hochberg
suggests.

> May be you meant
> 
> >>> array_equal(array([[1,2,3]]), array([1,2,3]))
> 1
> 
> which I would agree. Also
> 
> >>> array_equal(array([[1],[2],[3]]), array([1,2,3]))
> 1
> 
> but I am not sure about
> 
> >>> array_equal(array([[1,2,3],[1,2,3]]), array([1,2,3,1,2,3]))
> 1
> 
> Pearu

Perry 



From perry at stsci.edu  Mon Dec  9 08:30:02 2002
From: perry at stsci.edu (Perry Greenfield)
Date: Mon Dec  9 08:30:02 2002
Subject: [Numpy-discussion] Re: [SciPy-user] numarray complex comparisons
In-Reply-To: <005101c29f9d$ca8dfce0$391e6244@cx781526b>
Message-ID: <NEBBIJKBMLDBLNCEEFOCMEPMCOAA.perry@stsci.edu>

Tim Hochberg write:

> 
> What is the argument against allowing __eq__ to continue to work, while
> disallowing __lt__ and friends. The former is well defined. I 
> know comparing
> floating point number for equality is a bit of a suckers game, but were
> allowing it for floats and it's sometimes useful.
> 
> On second thought, maybe I'm just misinterpreting you here since __eq__
> works fine for complex scalars.
> 
> -tim

I wasn't clear on this. What I was arguing was that less, less_equal,
greater, greater_equal would not work, but that equal and not_equal

Primarily because that is what Python does. It does allow one
to test for equality (or non equality), but doesn't allow
for >,<,>=,<=, which is sensible in my view.

Perry



From hinsen at cnrs-orleans.fr  Mon Dec  9 09:07:02 2002
From: hinsen at cnrs-orleans.fr (Konrad Hinsen)
Date: Mon Dec  9 09:07:02 2002
Subject: [Numpy-discussion] numarray equality testing
In-Reply-To: <NEBBIJKBMLDBLNCEEFOCIEPLCOAA.perry@stsci.edu>
References: <NEBBIJKBMLDBLNCEEFOCIEPLCOAA.perry@stsci.edu>
Message-ID: <m3lm2z9lcc.fsf@chinon.cnrs-orleans.fr>

"Perry Greenfield" <perry at stsci.edu> writes:

> Instead we propose a function whose purpose is to determine
> if two arrays are shape consistent (i.e., can be broadcast
> against each other) and have all values equal to each other.

So there would be that operation, plus one that checks identity
of shape and all elements? That should cover all needs I can think of.

Konrad.



From hinsen at cnrs-orleans.fr  Mon Dec  9 09:09:09 2002
From: hinsen at cnrs-orleans.fr (Konrad Hinsen)
Date: Mon Dec  9 09:09:09 2002
Subject: [Numpy-discussion] numarray rank-0 array issues
In-Reply-To: <NEBBIJKBMLDBLNCEEFOCAEPMCOAA.perry@stsci.edu>
References: <NEBBIJKBMLDBLNCEEFOCAEPMCOAA.perry@stsci.edu>
Message-ID: <m3fzt79l7x.fsf@chinon.cnrs-orleans.fr>

"Perry Greenfield" <perry at stsci.edu> writes:

> Finally, the default reduction operation (i.e., add.reduce) could 
> be given the behavior described above. We are inclined to leave it
> as is, i.e., to return scalars when reducing 1-d arrays and provide

Please do! There is no point in breaking code without any gain in return.

Konrad.
-- 
-------------------------------------------------------------------------------
Konrad Hinsen                            | E-Mail: hinsen at cnrs-orleans.fr
Centre de Biophysique Moleculaire (CNRS) | Tel.: +33-2.38.25.56.24
Rue Charles Sadron                       | Fax:  +33-2.38.63.15.17
45071 Orleans Cedex 2                    | Deutsch/Esperanto/English/
France                                   | Nederlands/Francais
-------------------------------------------------------------------------------



From bazell at comcast.net  Mon Dec  9 11:10:04 2002
From: bazell at comcast.net (Dave Bazell)
Date: Mon Dec  9 11:10:04 2002
Subject: [Numpy-discussion] installation warning on RH 7.3
Message-ID: <001f01c29fb6$1a6e0f80$6401a8c0@DB>

I just installed numpy on my RH 7.3 system under python 2.2.2.  I get the
following messages when I import Numeric.

Could someone enlighten me as to what this means and what I should do about
it?

Thanks,

Dave Bazell


Python 2.2.2 (#1, Oct 21 2002, 12:22:55)
[GCC 2.96 20000731 (Red Hat Linux 7.3 2.96-110)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import Numeric
Traceback (most recent call last):
  File "<stdin>", line 1, in ?
  File "/usr/local/lib/python2.2/site-packages/Numeric/Numeric.py", line 93,
in ?
    from Precision import *
  File "/usr/local/lib/python2.2/site-packages/Numeric/Precision.py", line
26, in ?
    _code_table = _fill_table(typecodes)
  File "/usr/local/lib/python2.2/site-packages/Numeric/Precision.py", line
23, in _fill_table
    table[key] = _get_precisions(value)
  File "/usr/local/lib/python2.2/site-packages/Numeric/Precision.py", line
18, in _get_precisions
    lst.append( (zeros( (1,), t ).itemsize()*8, t) )
ValueError: Invalid type for array
>>> import Numeric
>>>






From kern at caltech.edu  Mon Dec  9 16:48:03 2002
From: kern at caltech.edu (Robert Kern)
Date: Mon Dec  9 16:48:03 2002
Subject: [Numpy-discussion] Re: [SciPy-user] numarray equality testing
In-Reply-To: <004101c29f9c$f7d5e560$391e6244@cx781526b>
References: <NEBBIJKBMLDBLNCEEFOCIEPLCOAA.perry@stsci.edu> <004101c29f9c$f7d5e560$391e6244@cx781526b>
Message-ID: <20021210004645.GA3033@taliesen.caltech.edu>

On Mon, Dec 09, 2002 at 09:06:43AM -0700, Tim Hochberg wrote:

[snip]

> This seems like a good plan. I'm not enthralled by the name though:
> array_equal seems equally likely to describethe  behaviour of the unfunc
> equal as it does the behaviour above. Maybe array_identical or array_same or
> equal_arrays or some such (although none of those are great).

I'd put in my vote for array_equiv, equiv, or equivalent since I think
"equivalence" best expresses the relationship tested here, not
"equality."

> -tim

-- 
Robert Kern
Ruddock House President
kern at caltech.edu

"In the fields of hell where the grass grows high
 Are the graves of dreams allowed to die."
  -- Richard Harter



From ariciputi at pito.com  Wed Dec 11 02:10:02 2002
From: ariciputi at pito.com (Andrea Riciputi)
Date: Wed Dec 11 02:10:02 2002
Subject: [Numpy-discussion] Detecting Numeric.
Message-ID: <5F29487C-0CF0-11D7-883B-000393933E4E@pito.com>

Hi there,
I'm playing with Numeric and even if I've not so much experience with 
it I'm trying to write a C extention to Python that can use Numeric 
arrays. The basic idea consists in handling C (dynamically allocated) 
array as Numeric array (if Numeric is installed) or as simple Python 
lists if not.

How can I detect at compile time if Numeric is installed or not?? I've 
thought something like:

#ifdef NUMERIC_IS_HERE
#include "Numeric/arrayobject.h"
#endif

But I've not found any hints in Numeric manual. Can anyone suggest a 
solution to me?

Thanks in advance,
Andrea.

---
Andrea Riciputi        <mailto:andrea.riciputi at libero.it>

"Science is like sex: sometimes something useful comes out,
   but that is not the reason we are doing it" -- (Richard Feynman) 
  




From hinsen at cnrs-orleans.fr  Wed Dec 11 02:38:04 2002
From: hinsen at cnrs-orleans.fr (Konrad Hinsen)
Date: Wed Dec 11 02:38:04 2002
Subject: [Numpy-discussion] Detecting Numeric.
In-Reply-To: <5F29487C-0CF0-11D7-883B-000393933E4E@pito.com>
References: <5F29487C-0CF0-11D7-883B-000393933E4E@pito.com>
Message-ID: <m37kegkfoa.fsf@chinon.cnrs-orleans.fr>

Andrea Riciputi <ariciputi at pito.com> writes:

> How can I detect at compile time if Numeric is installed or not?? I've
> thought something like:
> 
> #ifdef NUMERIC_IS_HERE
> #include "Numeric/arrayobject.h"
> #endif

Supposing that you will use distutils for compilation and installation
of your extension module, the solution is easy: your distutils script
can test for Numeric and then add the compiler flag. For example:

   defines = []
   try:
       import Numeric
       defines.append('NUMERIC_IS_HERE')
   except ImportError:
       pass

   ...

   Extension("foo", ["foo.c"], define_macros = defines)

Konrad.
-- 
-------------------------------------------------------------------------------
Konrad Hinsen                            | E-Mail: hinsen at cnrs-orleans.fr
Centre de Biophysique Moleculaire (CNRS) | Tel.: +33-2.38.25.56.24
Rue Charles Sadron                       | Fax:  +33-2.38.63.15.17
45071 Orleans Cedex 2                    | Deutsch/Esperanto/English/
France                                   | Nederlands/Francais
-------------------------------------------------------------------------------



From pearu at cens.ioc.ee  Wed Dec 11 02:49:02 2002
From: pearu at cens.ioc.ee (Pearu Peterson)
Date: Wed Dec 11 02:49:02 2002
Subject: [Numpy-discussion] Detecting Numeric.
In-Reply-To: <5F29487C-0CF0-11D7-883B-000393933E4E@pito.com>
Message-ID: <Pine.LNX.4.21.0212111233520.20968-100000@cens.kybi>

On Wed, 11 Dec 2002, Andrea Riciputi wrote:

> Hi there,
> I'm playing with Numeric and even if I've not so much experience with 
> it I'm trying to write a C extention to Python that can use Numeric 
> arrays. The basic idea consists in handling C (dynamically allocated) 
> array as Numeric array (if Numeric is installed) or as simple Python 
> lists if not.
> 
> How can I detect at compile time if Numeric is installed or not?? I've 
> thought something like:
> 
> #ifdef NUMERIC_IS_HERE
> #include "Numeric/arrayobject.h"
> #endif
> 
> But I've not found any hints in Numeric manual. Can anyone suggest a 
> solution to me?

If you are using distutils for building extension modules then you can
test if Numeric is available in setup.py file and define, say, HAVE_NUMPY
macro, if Numeric is available. For example,

#-------

have_numpy = 0
try:
    import Numeric
    have_numpy = 1
except ImportError:
    pass

define_macros = []
if have_numpy:
    define_macros.append(('HAVE_NUMPY', None))

if __name__ == "__main__":

    from distutils.core import setup
    setup(name = ...,
          ext_modules = [...],
          define_macros = define_macros
          )

#---------

and in extension source files use

#ifdef HAVE_NUMPY
...
#endif

HTH,
	Pearu




From ariciputi at pito.com  Wed Dec 11 03:03:06 2002
From: ariciputi at pito.com (Andrea Riciputi)
Date: Wed Dec 11 03:03:06 2002
Subject: [Numpy-discussion] Some questions about PyArray_As2D.
Message-ID: <15153D8E-0CF8-11D7-883B-000393933E4E@pito.com>

Hi,
I've two question about PyArray_As2D function:

1) Looking at the source code I realized that the prototype given in 
the manual is different from that of the source:

PyArray_As2D(PyObject *op, char **ptr, int *m, int *n, int type) (from 
manual)

PyArray_As2D(PyObject **op, char ***ptr, int *d1, int *d2, int 
typecode) (from source)

As far as I can understand the source version is correct while the 
manual's one is not. Am I wrong?

2) Next in the source code I've found the following memory allocation:

data = (char **)malloc(n*sizeof(char *));

without checking if malloc return NULL or not. As far as I know it's 
not safe, even if it's very unlikely that this malloc would fail.
Anyway in that case the following:

...
data[i] = ap->data + i*ap->strides[0];
...

would cause the function to abort. Again am I wrong?

Thanks in advance,
Andrea.

---
Andrea Riciputi        <mailto:andrea.riciputi at libero.it>

"Science is like sex: sometimes something useful comes out,
   but that is not the reason we are doing it" -- (Richard Feynman) 
  




From hinsen at cnrs-orleans.fr  Wed Dec 11 05:37:02 2002
From: hinsen at cnrs-orleans.fr (Konrad Hinsen)
Date: Wed Dec 11 05:37:02 2002
Subject: [Numpy-discussion] Some questions about PyArray_As2D.
In-Reply-To: <15153D8E-0CF8-11D7-883B-000393933E4E@pito.com>
References: <15153D8E-0CF8-11D7-883B-000393933E4E@pito.com>
Message-ID: <m33cp4k7e8.fsf@chinon.cnrs-orleans.fr>

Andrea Riciputi <ariciputi at pito.com> writes:

> As far as I can understand the source version is correct while the
> manual's one is not. Am I wrong?

The source code always wins the argument :-)

> 2) Next in the source code I've found the following memory allocation:
> 
> data = (char **)malloc(n*sizeof(char *));
> 
> without checking if malloc return NULL or not. As far as I know it's
> not safe, even if it's very unlikely that this malloc would fail.

Right. Did you submit a bug report?

Konrad.
-- 
-------------------------------------------------------------------------------
Konrad Hinsen                            | E-Mail: hinsen at cnrs-orleans.fr
Centre de Biophysique Moleculaire (CNRS) | Tel.: +33-2.38.25.56.24
Rue Charles Sadron                       | Fax:  +33-2.38.63.15.17
45071 Orleans Cedex 2                    | Deutsch/Esperanto/English/
France                                   | Nederlands/Francais
-------------------------------------------------------------------------------



From ariciputi at pito.com  Wed Dec 11 07:04:02 2002
From: ariciputi at pito.com (Andrea Riciputi)
Date: Wed Dec 11 07:04:02 2002
Subject: [Numpy-discussion] Some questions about PyArray_As2D.
In-Reply-To: <m33cp4k7e8.fsf@chinon.cnrs-orleans.fr>
Message-ID: <9EAAE954-0D19-11D7-883B-000393933E4E@pito.com>

On Wednesday, Dec 11, 2002, at 14:33 Europe/Rome, Konrad Hinsen wrote:

>> 2) Next in the source code I've found the following memory allocation:
>>
>> data = (char **)malloc(n*sizeof(char *));
>>
>> without checking if malloc return NULL or not. As far as I know it's
>> not safe, even if it's very unlikely that this malloc would fail.
>
> Right. Did you submit a bug report?

Just done. By the way reading the code again and again I got another 
question. Here is the complete code fragment:

extern int PyArray_As2D(PyObject **op, char ***ptr, int *d1, int *d2, 
int typecode) {
     PyArrayObject *ap;
     int i, n;
     char **data;

     if ((ap = (PyArrayObject *)PyArray_ContiguousFromObject(*op, 
typecode, 2, 2)) == NULL)
         return -1;

     n = ap->dimensions[0];
     data = (char **)malloc(n*sizeof(char *));
     for(i=0; i<n; i++) {
         data[i] = ap->data + i*ap->strides[0];
     }
     *op = (PyObject *)ap;  <=== It doesn't sound good to me!!!
     *ptr = data;
     *d1 = ap->dimensions[0];
     *d2 = ap->dimensions[1];
     return 0;
}

Looking at the marked line I started wondering about the fate of the 
object originally pointed by op. Without explicitly deallocating it you 
lost any chance to reach it. It turns out in a memory leakage. 
Obviously the same problem happend in PyArray_As1D function.

I'm very interested in this topic because I'm writing some Python 
extensions and I'd like to understand how I have to handle all these 
objects correctly. So how "long" does a Python object live? How can I 
release correctly the allocated memory?

Thanks,
Andrea.
---
Andrea Riciputi        <mailto:andrea.riciputi at libero.it>

"Science is like sex: sometimes something useful comes out,
   but that is not the reason we are doing it" -- (Richard Feynman)




From hinsen at cnrs-orleans.fr  Wed Dec 11 10:46:03 2002
From: hinsen at cnrs-orleans.fr (Konrad Hinsen)
Date: Wed Dec 11 10:46:03 2002
Subject: [Numpy-discussion] Some questions about PyArray_As2D.
In-Reply-To: <9EAAE954-0D19-11D7-883B-000393933E4E@pito.com> (message from
	Andrea Riciputi on Wed, 11 Dec 2002 16:02:51 +0100)
References: <9EAAE954-0D19-11D7-883B-000393933E4E@pito.com>
Message-ID: <200212111843.gBBIhSl22020@chinon.cnrs-orleans.fr>

> Just done. By the way reading the code again and again I got another 
> question. Here is the complete code fragment:
> 
> extern int PyArray_As2D(PyObject **op, char ***ptr, int *d1, int *d2, 
> int typecode) {
>      PyArrayObject *ap;
>      int i, n;
>      char **data;
> 
>      if ((ap = (PyArrayObject *)PyArray_ContiguousFromObject(*op, 
> typecode, 2, 2)) == NULL)
>          return -1;
> 
>      n = ap->dimensions[0];
>      data = (char **)malloc(n*sizeof(char *));
>      for(i=0; i<n; i++) {
>          data[i] = ap->data + i*ap->strides[0];
>      }
>      *op = (PyObject *)ap;  <=== It doesn't sound good to me!!!
>      *ptr = data;
>      *d1 = ap->dimensions[0];
>      *d2 = ap->dimensions[1];
>      return 0;
> }
> 
> Looking at the marked line I started wondering about the fate of the 
> object originally pointed by op. Without explicitly deallocating it you 
> lost any chance to reach it. It turns out in a memory leakage. 

No. op is an input parameter and thus a "borrowed" reference. It might
not be the best coding style to reuse that variable name for something
unrelated later on, but it doesn't cause a memory leak.

> I'm very interested in this topic because I'm writing some Python 
> extensions and I'd like to understand how I have to handle all these 
> objects correctly. So how "long" does a Python object live? How can I 
> release correctly the allocated memory?

There is an explanation of this topic in chapter 1.10 of the Python
"Extending and Embedding" manual. Basically, you have to increase an
object's reference counter if you want to keep a reference to it
beyond the end of the currently running function, and to decrease the
counter if you want to release that reference. Whenever the reference
count goes to zero, the object is deleted. With very few exceptions,
functions that take an object as input to work on (but not store)
don't increase the reference counter. They don't have to, because
nothing can happen to the object until the function terminates.
Special precautions need to be taken for multithreading, but these go
much beyond object reference counting.

As a rule of thumb, you don't have to worry about reference counting
at all as long as you only write C functions that act on existing
objects. It becomes an issue when you define your own extension types
in C.

Konrad.
-- 
-------------------------------------------------------------------------------
Konrad Hinsen                            | E-Mail: hinsen at cnrs-orleans.fr
Centre de Biophysique Moleculaire (CNRS) | Tel.: +33-2.38.25.56.24
Rue Charles Sadron                       | Fax:  +33-2.38.63.15.17
45071 Orleans Cedex 2                    | Deutsch/Esperanto/English/
France                                   | Nederlands/Francais
-------------------------------------------------------------------------------



From edcjones at erols.com  Wed Dec 11 18:48:02 2002
From: edcjones at erols.com (Edward C. Jones)
Date: Wed Dec 11 18:48:02 2002
Subject: [Numpy-discussion] 3 comments on numarray documentation
Message-ID: <3DF80012.4070008@erols.com>

To access UInt8, etc,

     from numerictypes import *

Maybe mention this in 4.2.1.

-------

In 4.7

      Only one "..." is expanded in an index expression, so if one has a
      rank-5 array C, then C[...,0,...] is the same thing as
      C[:,:,:,0,:].

So an unexpanded "..." is treated as a ':'?

----------

In 5.1.1,

      >>> a = arange(5, type=Float64)
      >>> print a[::-1] * 1.2
      [ 4.8  3.6  2.4  1.2  0. ]
      >>> multiply(a[::-1], 1.2, a)
      >>> a
      array([ 4.8 ,  3.6 ,  2.4 ,  1.2,  0. ])

doesn't make the desired point. Try:

      >>> a = arange(5, type=Int32)
      >>> a
      [0 1 2 3 4]
      >>> print a[::-1] * 1.2
      [ 4.8  3.6  2.4  1.2  0. ]
      >>> multiply(a[::-1], 1.2, a)
      >>> a
      array([4 3 2 1 0])

Why does Python documentation always use the interpreter? I doubt if it 
is used much. Why not:

      a = arange(5, type=Int32)
      print a.type(), a
      b = a[::-1] * 1.2
      print b.type(), b
      numarray.multiply(a[::-1], 1.2, a)
      print a.type(), a





From Chris.Barker at noaa.gov  Thu Dec 12 12:34:03 2002
From: Chris.Barker at noaa.gov (Chris Barker)
Date: Thu Dec 12 12:34:03 2002
Subject: [Numpy-discussion] 3 comments on numarray documentation
References: <3DF80012.4070008@erols.com>
Message-ID: <3DF8DC98.28232AC4@noaa.gov>

"Edward C. Jones" wrote:

> Why does Python documentation always use the interpreter? I doubt if it
> is used much. 

I, for one, use it all the time to explore the syntax and function of a
simple statement, just like the examples.

-Chris


-- 
Christopher Barker, Ph.D.
Oceanographer
                                    		
NOAA/OR&R/HAZMAT         (206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115       (206) 526-6317   main reception

Chris.Barker at noaa.gov



From oliphant.travis at ieee.org  Thu Dec 12 12:40:03 2002
From: oliphant.travis at ieee.org (Travis Oliphant)
Date: Thu Dec 12 12:40:03 2002
Subject: [Numpy-discussion] real/imag for non complex numbers [was numarray complex comparisons]
In-Reply-To: <003101c2a16c$7901dba0$391e6244@cx781526b>
References: <Pine.LNX.4.21.0212112257250.19308-100000@cens.kybi> 
	<003101c2a16c$7901dba0$391e6244@cx781526b>
Message-ID: <1039725475.2227.13.camel@travis.local.net>

In investigating, why Numeric does not allow a.real for non-complex
arrays which many have argued that it should, I think I uncovered an old
bug in Numeric.

In looking at the code, it appears that the intention was to have a.real
just return a new array (pointing to the same data as a but having a new
header) whenever a was not a complex array.  Unfortunately, the code had
a bug and did not actually return the newly created array.  As a result,
the default behavior of raising an "Attribute not found" exception
occurred.  But, before this happened, the array the reference count on
the array occurred, thereby springing a memory leak. 

Try this out.

a = ones((10,10))

print sys.getrefcount(a)

a.real   # will generate an exception

print sys.getrefcount(a)

a.real  

print sys.getrefcount(a)

Notice that each time a.real is called the reference count for a goes up
by one even though no object is returned.


I have fixed the bug in Numeric CVS by returning the created object for
a.real.

The end result is that a.real works for all arrays (not just
complex-valued ones). 

-Travis O.



On Wed, 2002-12-11 at 16:24, Tim Hochberg wrote:
> 
> Here's my latest thinking on this. I'll put this in the form of a proposal
> so that it's easier to comment on. Please comment on part I and part II
> separately.
> 
> I. Attributes real and imag/imaginary[1]
> 
> All numeric array types (i.e., excluding character and object arrays) would
> have both a real and imag/imaginary attribute. For complex types ('F', 'D')
> this would behave just as it does now. For non-complex types ('i', 'f', 'd',
> etc.), the real attribute would return the object itself, while the
> imaginary attribute would return a ReadOnlyZeroArray [2] of the correct
> shape and typecode. I think that this would provide behaviour consistent
> with the current real and imag attributes on complex arrays without
> introducing and spooky corner cases (that statement is asking for trouble!).
> Examples:
> 
> >>> a = array([1,2,3])
> >>> a.real[0] = 99
> >>> a.real
> array([99, 2, 3])
> >>> a.imag
> array([0,0,0])
> >>> a.imag[0] = 99
> TypeError: object is read only.
> 
> II. Functions real and imag
> 
> Two functions real and imag (or imaginary or both) would be provided. The
> would behave as if they were defined as shown below, even if item I. is not
> adopted.
> 
> def real(a):
>     return asarray(a).real
> 
> def imag(a):
>     return asarray(a).imag
> 
> 
> Comments?
> 
> I'm currently +0 on I. and +1 on II.
> 
> -tim
> 
> 
> 
> [1] Numeric arrays haveboth imag and imaginary attributes, but they are
> identical. I'm not sure what Numarray's plans are in this regard.
> 
> [2] A ReadOnlyZeroArray(shape, typecode) would behave like an array of zeros
> with the given shape and typecode for all non mutating operations. For
> mutation operations (a[x] = b, a += c, etc.) it would raise an exception.
> This object does not actually allocate any space, so creation is fast and
> memory efficient. Implementing this might be the hardest part of the
> proposal.
> 
> 
> _______________________________________________
> SciPy-user mailing list
> SciPy-user at scipy.net
> http://www.scipy.net/mailman/listinfo/scipy-user





From ariciputi at pito.com  Fri Dec 13 02:46:00 2002
From: ariciputi at pito.com (Andrea Riciputi)
Date: Fri Dec 13 02:46:00 2002
Subject: [Numpy-discussion] Some questions about PyArray_As2D.
In-Reply-To: <200212111843.gBBIhSl22020@chinon.cnrs-orleans.fr>
Message-ID: <8550842A-0E87-11D7-883B-000393933E4E@pito.com>

On Wednesday, Dec 11, 2002, at 19:43 Europe/Rome, Konrad Hinsen wrote:

> No. op is an input parameter and thus a "borrowed" reference. It might
> not be the best coding style to reuse that variable name for something
> unrelated later on, but it doesn't cause a memory leak.

I'm sorry but I don't agree. I've read your answer many times and 
re-read the code but I still think to be right.

The function prototype says:

extern int PyArray_As2D(PyObject **op, char ***ptr, int *d1, int *d2, 
int typecode)

and a call to this function looks like this:

PyObject *input;
double **result;
int nrows, ncols;

PyArray_As2D((&input, (char ***) &(result), &(nrows), &(ncols), 
PyArray_DOUBLE)

Now when you call the function in this way op is a pointer to the 
pointer that points to your original ArrayObject. It allows you to 
change the memory address which is originally pointed by input. And it 
is exactly what you do with the instruction:

*op = (PyObject *)ap;

So you create a new PyArrayObject (allocating another memory area) by 
means of PyArray_ContiguousFromObject and names it ap, then you modify 
the memory address which op points to with the above instruction. Now 
*op (that is input) points to another memory region, but you haven't 
deallocated the previous pointed memory and it _is_ a memory leak!

These are my two cents, comments are welcome.

Cheers,
Andrea.


---
Andrea Riciputi        <mailto:andrea.riciputi at libero.it>

"Science is like sex: sometimes something useful comes out,
   but that is not the reason we are doing it" -- (Richard Feynman)




From hinsen at cnrs-orleans.fr  Fri Dec 13 03:12:02 2002
From: hinsen at cnrs-orleans.fr (Konrad Hinsen)
Date: Fri Dec 13 03:12:02 2002
Subject: [Numpy-discussion] Some questions about PyArray_As2D.
In-Reply-To: <8550842A-0E87-11D7-883B-000393933E4E@pito.com> (message from
	Andrea Riciputi on Fri, 13 Dec 2002 11:42:05 +0100)
References: <8550842A-0E87-11D7-883B-000393933E4E@pito.com>
Message-ID: <200212131109.gBDB9Mk27388@chinon.cnrs-orleans.fr>

> The function prototype says:
> 
> extern int PyArray_As2D(PyObject **op, char ***ptr, int *d1, int *d2, 
> int typecode)

You are right that my first comment doesn't quite apply, I hadn't
noticed the two stars...

But the code is still OK, it is the calling routine that is responsible
for avoiding memory leaks.

> is exactly what you do with the instruction:
> 
> *op = (PyObject *)ap;
> 
> So you create a new PyArrayObject (allocating another memory area) by 
> means of PyArray_ContiguousFromObject and names it ap, then you modify 
> the memory address which op points to with the above instruction. Now 

Right. This routine cannot know if that reference contains an "owned"
or a "borrowed" reference. In the first case, the reference counter
of the original array must be decreased, in the second case not.

Assume for example that the calling routine got an array passed in
as a borrowed reference:

void foo(PyObject *array)
{
   /* I want a 2D version! */
   PyArray_As2D(&array, ...)
}

In that case, decreasing the reference count in PyArray_As2D would
be disastrous.

In the case of an owned reference, the calling routine must keep a
copy of the pointer, call PyArray_As2D, and then decrease the
reference counter on the original pointer. This ought to be easier.
In fact, the interface of PyArray_As2D is pretty badly designed.
It should not overwrite the original pointer, but return a new
one. I also don't see the point in passing all the array information
in additional arguments - they are easy to obtain from the new
array object.

Konrad.
-- 
-------------------------------------------------------------------------------
Konrad Hinsen                            | E-Mail: hinsen at cnrs-orleans.fr
Centre de Biophysique Moleculaire (CNRS) | Tel.: +33-2.38.25.56.24
Rue Charles Sadron                       | Fax:  +33-2.38.63.15.17
45071 Orleans Cedex 2                    | Deutsch/Esperanto/English/
France                                   | Nederlands/Francais
-------------------------------------------------------------------------------



From perry at stsci.edu  Fri Dec 13 08:30:05 2002
From: perry at stsci.edu (Perry Greenfield)
Date: Fri Dec 13 08:30:05 2002
Subject: [Numpy-discussion] 3 comments on numarray documentation
In-Reply-To: <3DF80012.4070008@erols.com>
Message-ID: <JFEGLNDJEDNOMPPHDEJFGELDEBAA.perry@stsci.edu>

Sorry it took so long to respond. We appreciate this feedback.

Edward C. Jones writes:
 
> To access UInt8, etc,
> 
>      from numerictypes import *
> 
> Maybe mention this in 4.2.1.
> 
Are you referring to text in another part of the manual or are you
suggesting that this be added to 4.2.1? If added I would reword it
somewhat since these type names are in the numarray namespace. If 
one wants to do:

import numarray

and have the types as part of you namespace it would make sense to

import numarray
from numerictypes import *

(though if we go to a package system, this may become
from numarray.numerictypes import *)

We also need to add the fact that there are now 
UInt32, UInt64 (not on windows), Int64 types.

> -------
> 
> In 4.7
> 
>       Only one "..." is expanded in an index expression, so if one has a
>       rank-5 array C, then C[...,0,...] is the same thing as
>       C[:,:,:,0,:].
> 
> So an unexpanded "..." is treated as a ':'?
> 
yes

> ----------
> 
> In 5.1.1,
> 
>       >>> a = arange(5, type=Float64)
>       >>> print a[::-1] * 1.2
>       [ 4.8  3.6  2.4  1.2  0. ]
>       >>> multiply(a[::-1], 1.2, a)
>       >>> a
>       array([ 4.8 ,  3.6 ,  2.4 ,  1.2,  0. ])
> 
> doesn't make the desired point. Try:
> 
>       >>> a = arange(5, type=Int32)
>       >>> a
>       [0 1 2 3 4]
>       >>> print a[::-1] * 1.2
>       [ 4.8  3.6  2.4  1.2  0. ]
>       >>> multiply(a[::-1], 1.2, a)
>       >>> a
>       array([4 3 2 1 0])
> 
Yes, we will make this change

> Why does Python documentation always use the interpreter? I doubt if it 
> is used much. Why not:
> 
>       a = arange(5, type=Int32)
>       print a.type(), a
>       b = a[::-1] * 1.2
>       print b.type(), b
>       numarray.multiply(a[::-1], 1.2, a)
>       print a.type(), a
> 
Actually many do use it in interpreter mode, at least here. But I think
you miss the main point which is to show the result of each command
for the purposes of instruction. Even if you never plan to use the
interpreter (which I think would be a mistake since it is a wonderful
way of verifying that things work the way you thought they did),
it serves to show examples in a very clear and concise way. 

Perry



From edcjones at erols.com  Fri Dec 13 08:34:02 2002
From: edcjones at erols.com (Edward C. Jones)
Date: Fri Dec 13 08:34:02 2002
Subject: [Numpy-discussion] Comments on Documentation - 2
Message-ID: <3DFA1344.6030003@erols.com>

I am currently learning about numarray. Here are further comments on the 
documentation.

----------------------

"libnumarray.h" and "numarray.h" are never mentioned in the docs.

----------------------

10.2.3 Is "free(result)" needed here?

         double *result;
         int m, n;
         .
         .
         .
         result = func(...);
         if(NULL == result)
             return NULL;
         return NA_New((void *)result, tFloat64, m, n, 2);

----------------------

10.6 "NA_New", "NA_Empty", and "NA_NewA11" return a "PyArrayObject*", 
not a "PyObject*".




From edcjones at erols.com  Fri Dec 13 17:09:04 2002
From: edcjones at erols.com (Edward C. Jones)
Date: Fri Dec 13 17:09:04 2002
Subject: [Numpy-discussion] Comments on numarray - 3
Message-ID: <3DFA8BEB.2040209@erols.com>

NA_OFFSETDATA is mentioned during the example in 10.2.3. It should be 
documented in 10.2.1. Should some of the other macros in "nummacro.h" be 
documented? Also the macros in "arrayobject.h" and "numcomplex.h".

-------------------

The "libnumarray_UNIQUE_SYMBOL" trick discussed in "libnumarray.h" has 
been useful with Numeric and will be useful with numarray. It needs to 
be in the main documentation. I think it is mentioned somewhere in the 
Python documentation. I found it very confusing the first time I used 
it. A detailed example is needed.

-------------------

"NA_longsFromIntTuple" works on both tuples and lists. A better name 
might be "NA_longsFromIntSequence". Functions of this type have proven 
_very_ useful for me in writing large SWIG wrappers. I generate them 
using SWIG macros. Here are some of them. On the C side, auxillary 
information about the lengths of things needs to be dealt with. "list" 
means "list or tuple".

Python                     C
list of ints          <--> array of ints
list of floats        <--> array of doubles
list of floats        <--> array of floats
...
list of strings       <--> array of arrays of char
list of lists of ints <--> array of arrays of int
...
list                  <--> array of PyObject*

Thanks,
Ed Jones





From magnus at hetland.org  Sun Dec 15 13:17:03 2002
From: magnus at hetland.org (Magnus Lie Hetland)
Date: Sun Dec 15 13:17:03 2002
Subject: [Numpy-discussion] Index array confusion...
Message-ID: <20021215211615.GA18385@idi.ntnu.no>

I can see why this happens, to some degree, but it is a bit confusing
still:

>>> a = zeros([5,5])
>>> a[[1,2,3]]
array([[0, 0, 0, 0, 0],
       [0, 0, 0, 0, 0],
       [0, 0, 0, 0, 0]])
>>> a[[1,2]]
0
>>> a[[1]]
array([[0, 0, 0, 0, 0]])

I was planning to use index arrays to select a subset of the rows in a
two-dimensional array, but because of the special-casing for
one-dimensional arrays of the same length as the dimensionality of the
array being indexed, it seems I can't do that (without using
put/take, which may be quite OK, I guess)... Am I right? (I see why
this behaviour is needed for tuples and slice objects; and I guess
type checking would be sort of "ugly"...)

Of course a[[1,2],] works just as expected.

This may not be important, but perhaps worth a sentence or example in
the manual? I.e. if you are using an index array idx and you don't
want to risk having it cast as a single index lookup, you should do

  a[idx,]

Wouldn't that be sound advice in general?

-- 
Magnus Lie Hetland        Practical Python          The Anygui Project
http://hetland.org        http://ppython.com        http://anygui.org



From jmiller at stsci.edu  Sun Dec 15 13:34:02 2002
From: jmiller at stsci.edu (Todd Miller)
Date: Sun Dec 15 13:34:02 2002
Subject: [Numpy-discussion] Index array confusion...
References: <20021215211615.GA18385@idi.ntnu.no>
Message-ID: <3DFCF883.5000806@stsci.edu>

Magnus Lie Hetland wrote:

>I can see why this happens, to some degree, but it is a bit confusing
>still:
>
>  
>
>>>>a = zeros([5,5])
>>>>a[[1,2,3]]
>>>>        
>>>>
>array([[0, 0, 0, 0, 0],
>       [0, 0, 0, 0, 0],
>       [0, 0, 0, 0, 0]])
>  
>
>>>>a[[1,2]]
>>>>        
>>>>
>0
>  
>
>>>>a[[1]]
>>>>        
>>>>
>array([[0, 0, 0, 0, 0]])
>
>I was planning to use index arrays to select a subset of the rows in a
>two-dimensional array, but because of the special-casing for
>one-dimensional arrays of the same length as the dimensionality of the
>array being indexed, it seems I can't do that (without using
>put/take, which may be quite OK, I guess)... Am I right? (I see why
>this behaviour is needed for tuples and slice objects; and I guess
>type checking would be sort of "ugly"...)
>
>Of course a[[1,2],] works just as expected.
>
>This may not be important, but perhaps worth a sentence or example in
>the manual? I.e. if you are using an index array idx and you don't
>want to risk having it cast as a single index lookup, you should do
>
>  a[idx,]
>
>Wouldn't that be sound advice in general?
>
>  
>
Assuming you're talking about numarray here,  I regard this as a 0.4 bug 
which you just found.  Thanks!

Todd





From magnus at hetland.org  Sun Dec 15 13:36:02 2002
From: magnus at hetland.org (Magnus Lie Hetland)
Date: Sun Dec 15 13:36:02 2002
Subject: [Numpy-discussion] Index array confusion...
In-Reply-To: <3DFCF883.5000806@stsci.edu>
References: <20021215211615.GA18385@idi.ntnu.no> <3DFCF883.5000806@stsci.edu>
Message-ID: <20021215213541.GA20046@idi.ntnu.no>

Todd Miller <jmiller at stsci.edu>:
[snip]
> Assuming you're talking about numarray here,

Indeed.

> I regard this as a 0.4 bug which you just found.  Thanks!

No problem :)

So, I guess you're saying that in the future you will typecheck for
tuples and slice objects vs. other objects (such as arrays and lists)?

> Todd

-- 
Magnus Lie Hetland        Practical Python          The Anygui Project
http://hetland.org        http://ppython.com        http://anygui.org



From magnus at hetland.org  Sun Dec 15 14:03:03 2002
From: magnus at hetland.org (Magnus Lie Hetland)
Date: Sun Dec 15 14:03:03 2002
Subject: [Numpy-discussion] Index array confusion...
In-Reply-To: <3DFCFA3B.7090109@stsci.edu>
References: <20021215211615.GA18385@idi.ntnu.no> <3DFCF883.5000806@stsci.edu> <20021215213541.GA20046@idi.ntnu.no> <3DFCFA3B.7090109@stsci.edu>
Message-ID: <20021215220230.GA20949@idi.ntnu.no>

Todd Miller <jmiller at stsci.edu>:
>
[snip]
> What I'm saying is that in the future it will work correctly.  :)

OK :)

> Most likely as you suggest,  but I haven't really looked at the code, 
> just the rather sickening results.

Well, what I was asking about was really what you meant by "correct".
My interpretation of "correct" here was that only tuples and slices
would be allowed as indexes.

BTW: I found something else that I think is rather odd:

>>> x = [0, 0, 0]
>>> y = [1, 1, 1]
>>> p = 1
>>> x[p:], y[p:] = y[p:], x[p:]
>>> x, y
([0, 1, 1], [1, 0, 0])
>>> x = array([0, 0, 0])
>>> y = array([1, 1, 1])
>>> x[p:], y[p:] = y[p:], x[p:]
>>> x, y
(array([0, 1, 1]), array([1, 1, 1]))

This seems like a bug to me. The assignment ought to swap the tails of
the sequences, as is the case with lists, but with numeric arrays,
some weird form of overwriting occurs. I guess this may be an
optimization (i.e. to avoid making copies), but it is rather
confusing. What do you think?

> Todd

-- 
Magnus Lie Hetland        Practical Python          The Anygui Project
http://hetland.org        http://ppython.com        http://anygui.org



From magnus at hetland.org  Sun Dec 15 14:10:01 2002
From: magnus at hetland.org (Magnus Lie Hetland)
Date: Sun Dec 15 14:10:01 2002
Subject: [Numpy-discussion] Index array confusion...
In-Reply-To: <20021215220230.GA20949@idi.ntnu.no>
References: <20021215211615.GA18385@idi.ntnu.no> <3DFCF883.5000806@stsci.edu> <20021215213541.GA20046@idi.ntnu.no> <3DFCFA3B.7090109@stsci.edu> <20021215220230.GA20949@idi.ntnu.no>
Message-ID: <20021215220911.GA21137@idi.ntnu.no>

Magnus Lie Hetland <magnus at hetland.org>:
[snip]
> This seems like a bug to me. The assignment ought to swap the tails of
> the sequences, as is the case with lists, but with numeric arrays,
> some weird form of overwriting occurs. I guess this may be an
> optimization (i.e. to avoid making copies), but it is rather
> confusing. What do you think?

Even stranger:

>>> a = zeros([5])
>>> b = ones([5])
>>> p = 2
>>> tail_a = a[p:]
>>> tail_b = b[p:]
>>> a[p:] = tail_b
>>> b[p:] = tail_a
>>> a, b
(array([0, 0, 1, 1, 1]), array([1, 1, 1, 1, 1]))

I suppose this is due to sharing of slices somehow? I.e:

>>> a = ones([5])
>>> b = a[:]
>>> b[2] = 0
>>> a
array([1, 1, 0, 1, 1])

I have to use the copy method here, I guess?

> > Todd

-- 
Magnus Lie Hetland        Practical Python          The Anygui Project
http://hetland.org        http://ppython.com        http://anygui.org



From magnus at hetland.org  Sun Dec 15 14:31:07 2002
From: magnus at hetland.org (Magnus Lie Hetland)
Date: Sun Dec 15 14:31:07 2002
Subject: [Numpy-discussion] "not"
Message-ID: <20021215222955.GA21992@idi.ntnu.no>

Just a quick note: Why does the operator 'not' appear in parentheses
beside logical_not in the manual? Isn't that a bit misleading (since
it doesn't work the same way, as opposed to 'and' and 'or')?

-- 
Magnus Lie Hetland        Practical Python          The Anygui Project
http://hetland.org        http://ppython.com        http://anygui.org



From jmiller at stsci.edu  Mon Dec 16 06:57:01 2002
From: jmiller at stsci.edu (Todd Miller)
Date: Mon Dec 16 06:57:01 2002
Subject: [Numpy-discussion] Index array confusion...
References: <20021215211615.GA18385@idi.ntnu.no> <3DFCF883.5000806@stsci.edu> <20021215213541.GA20046@idi.ntnu.no> <3DFCFA3B.7090109@stsci.edu> <20021215220230.GA20949@idi.ntnu.no>
Message-ID: <3DFDEA77.10204@stsci.edu>

Magnus Lie Hetland wrote:

>Todd Miller <jmiller at stsci.edu>:
>  
>
>[snip]
>  
>
>>What I'm saying is that in the future it will work correctly.  :)
>>    
>>
>
>OK :)
>
>  
>
>>Most likely as you suggest,  but I haven't really looked at the code, 
>>just the rather sickening results.
>>    
>>
>
>Well, what I was asking about was really what you meant by "correct".
>My interpretation of "correct" here was that only tuples and slices
>would be allowed as indexes.
>
This actually worked as you expected in numarray-0.3.6.  The current 
behavior is a casualty of optimization to C.

>
>BTW: I found something else that I think is rather odd:
>
>  
>
>>>>x = [0, 0, 0]
>>>>y = [1, 1, 1]
>>>>p = 1
>>>>x[p:], y[p:] = y[p:], x[p:]
>>>>x, y
>>>>        
>>>>
>([0, 1, 1], [1, 0, 0])
>  
>
>>>>x = array([0, 0, 0])
>>>>y = array([1, 1, 1])
>>>>x[p:], y[p:] = y[p:], x[p:]
>>>>x, y
>>>>        
>>>>
>(array([0, 1, 1]), array([1, 1, 1]))
>
>This seems like a bug to me. The assignment ought to swap the tails of
>
Numeric does this as well,  so I would not call it a bug,  but  I agree 
that it is unfortunate.

>the sequences, as is the case with lists, but with numeric arrays,
>some weird form of overwriting occurs. I guess this may be an
>optimization (i.e. to avoid making copies), but it is rather
>
I think you're correct here.   This behavior is a consequence of the 
fact that array slices are views and not copies.  To fix it,  just say:

x[p:], y[p:] = y[p:].copy(), x[p:].copy()

>confusing. What do you think?
>
The manual should probably document this as a gotcha, and we might want 
to consider adding an exchange ufunc which can do the swap without a 
temporary.   I doubt the cost of exchange is worth it though.

Thanks for the feedback,
Todd




From jmiller at stsci.edu  Mon Dec 16 07:01:01 2002
From: jmiller at stsci.edu (Todd Miller)
Date: Mon Dec 16 07:01:01 2002
Subject: [Numpy-discussion] Index array confusion...
References: <20021215211615.GA18385@idi.ntnu.no> <3DFCF883.5000806@stsci.edu> <20021215213541.GA20046@idi.ntnu.no> <3DFCFA3B.7090109@stsci.edu> <20021215220230.GA20949@idi.ntnu.no> <20021215220911.GA21137@idi.ntnu.no>
Message-ID: <3DFDEB60.7090604@stsci.edu>

Magnus Lie Hetland wrote:

>Magnus Lie Hetland <magnus at hetland.org>:
>[snip]
>  
>
>>This seems like a bug to me. The assignment ought to swap the tails of
>>the sequences, as is the case with lists, but with numeric arrays,
>>some weird form of overwriting occurs. I guess this may be an
>>optimization (i.e. to avoid making copies), but it is rather
>>confusing. What do you think?
>>    
>>
>
>Even stranger:
>
>  
>
>>>>a = zeros([5])
>>>>b = ones([5])
>>>>p = 2
>>>>tail_a = a[p:]
>>>>tail_b = b[p:]
>>>>a[p:] = tail_b
>>>>b[p:] = tail_a
>>>>a, b
>>>>        
>>>>
>(array([0, 0, 1, 1, 1]), array([1, 1, 1, 1, 1]))
>
>I suppose this is due to sharing of slices somehow? I.e:
>  
>
This looks to me like the same problem,  just performing the effects of 
the tuple copy one step at a time.  The key is that the "tails" are 
views and not copies.

>  
>
>>>>a = ones([5])
>>>>b = a[:]
>>>>b[2] = 0
>>>>a
>>>>        
>>>>
>array([1, 1, 0, 1, 1])
>
>I have to use the copy method here, I guess?
>  
>
Yes.   array[ slice ] --> a view of a subarray.

>  
>
>>>Todd
>>>      
>>>
>
>  
>






From perry at stsci.edu  Mon Dec 16 11:13:06 2002
From: perry at stsci.edu (Perry Greenfield)
Date: Mon Dec 16 11:13:06 2002
Subject: [Numpy-discussion] Index array confusion...
In-Reply-To: <3DFDEA77.10204@stsci.edu>
Message-ID: <JFEGLNDJEDNOMPPHDEJFIELMEBAA.perry@stsci.edu>

Magnus Lie Hetland wrote:

> >Well, what I was asking about was really what you meant by "correct".
> >My interpretation of "correct" here was that only tuples and slices
> >would be allowed as indexes.
> >
To clarify, tuples should be interpreted differently than lists or 
arrays as arguments, as you suggested earlier so that
x[1,2,3] is not interpreted the same as x[[1,2,3]] or x[array([1,2,3])].
Granted, that will be confusing for those that try x[(1,2,3)]
expecting it to be equivalent ot x[[1,2,3]] but we decided that
the benefits of array indices well outweigh that confusing case.

As far as far as slices resulting in views rather than copies
(was with lists), this is a topic that has been talked to death.
The original Numeric has view behavior. When we started on numarray
we were intending to make it more consistent with lists, but there
were many that argued that view semantics were more sensible and
we were persuaded that they were right (besides, the backward
compatibility issue was very important as well).

Perry



From magnus at hetland.org  Mon Dec 16 11:23:02 2002
From: magnus at hetland.org (Magnus Lie Hetland)
Date: Mon Dec 16 11:23:02 2002
Subject: [Numpy-discussion] Index array confusion...
In-Reply-To: <JFEGLNDJEDNOMPPHDEJFIELMEBAA.perry@stsci.edu>
References: <3DFDEA77.10204@stsci.edu> <JFEGLNDJEDNOMPPHDEJFIELMEBAA.perry@stsci.edu>
Message-ID: <20021216192207.GA15908@idi.ntnu.no>

Perry Greenfield <perry at stsci.edu>:
>
> Magnus Lie Hetland wrote:
> 
> > >Well, what I was asking about was really what you meant by "correct".
> > >My interpretation of "correct" here was that only tuples and slices
> > >would be allowed as indexes.
> > >
> To clarify, tuples should be interpreted differently than lists or 
> arrays as arguments, as you suggested earlier so that
> x[1,2,3] is not interpreted the same as x[[1,2,3]] or x[array([1,2,3])].
> Granted, that will be confusing for those that try x[(1,2,3)]
> expecting it to be equivalent ot x[[1,2,3]] but we decided that
> the benefits of array indices well outweigh that confusing case.

Exactly. I agree completely with this (intended) behaviour. I was just
a bit baffled to find that numarray (0.4), in fact, behaved
differently. Since this is indeed a bug, the issue seems resolved to
me :)

(Except that I might have to add some version control to avoid running
with 0.4 :/)

> As far as far as slices resulting in views rather than copies
> (was with lists), this is a topic that has been talked to death.

Yes -- sorry about that.

> The original Numeric has view behavior.

I know. Just a mind-bug on my part :)

> When we started on numarray
> we were intending to make it more consistent with lists, but there
> were many that argued that view semantics were more sensible and
> we were persuaded that they were right (besides, the backward
> compatibility issue was very important as well).

And performance, perhaps?

And; from Todd's comment about a hypothetical built-in for the
purpose, I assume there is no way of copying the contents of one slice
to another without going via a temporary array?

> Perry

-- 
Magnus Lie Hetland        Practical Python          The Anygui Project
http://hetland.org        http://ppython.com        http://anygui.org



From jmiller at stsci.edu  Mon Dec 16 12:05:58 2002
From: jmiller at stsci.edu (Todd Miller)
Date: Mon Dec 16 12:05:58 2002
Subject: [Numpy-discussion] Index array confusion...
References: <3DFDEA77.10204@stsci.edu> <JFEGLNDJEDNOMPPHDEJFIELMEBAA.perry@stsci.edu> <20021216192207.GA15908@idi.ntnu.no>
Message-ID: <3DFE32C5.4070506@stsci.edu>

Magnus Lie Hetland wrote:

>  
>
>And; from Todd's comment about a hypothetical built-in for the
>purpose, I assume there is no way of copying the contents of one slice
>to another without going via a temporary array?
>  
>
The problem there is with overlapping slices.

>Perry
>  
>






From jmiller at stsci.edu  Mon Dec 16 14:22:03 2002
From: jmiller at stsci.edu (Todd Miller)
Date: Mon Dec 16 14:22:03 2002
Subject: [Numpy-discussion] Re: numarray questions
References: <Pine.LNX.4.40.0212161225530.1299-200000@sylvia>
Message-ID: <3DFE52E0.6020003@stsci.edu>

Mika Illouz wrote:

>Hi Todd,
>
>  
>
>>Thanks for the encouragement!   numarray-0.4 is coming out today...
>>    
>>
>
>Finally got around to playing with version 0.4 and the new NA_New (code
>attached).  Wnen I ran the test, I got the following trace:
>
>  
>
>>>>import _test_numarray
>>>>z = _test_numarray.t1()
>>>>        
>>>>
>Traceback (most recent call last):
>  File "<stdin>", line 1, in ?
>  File "/usr/lib/python2.2/site-packages/numarray/numarray.py", line 432, in __init__
>    bytestride=bytestride)
>  File "/usr/lib/python2.2/site-packages/numarray/ndarray.py", line 173, in __init__
>    self._data = memory.new_memory(size)
>OverflowError: long int too large to convert to int
>
>What am I doing wrong?
>
The parameter order in the call in t1() should be:

NA_New(a, tInt32, 1, length)

Because it is:

NA_New(a, tInt32, length, 1)

numarray thinks you're declaring a 5 dimensional array with some bizarre 
dimensions from "undefined values" on the stack.  One of these 
dimensions appears to be > MAX_INT.

>Thanks again,
>Mika
>  
>
Regards,
Todd

>  
>
>------------------------------------------------------------------------
>
>#include <Python.h>
>#include <libnumarray.h>
>
>static PyObject* t1( PyObject* self, PyObject* args )
>{
>  int length = 5;
>  int* a = new int[ length ];
>  for ( int i = 0 ; i < length ; i++ ) {
>    a[i] = i;
>  }
>
>  PyArrayObject* vec = NA_New( a, tInt32, length, 1 );
>  return (PyObject*)vec;
>  // return NA_ReturnOutput( NULL, vec );
>
>}
>
>static PyMethodDef TestNumarrayMethods[] = {
>  {"t1",  
>   t1, 
>   METH_VARARGS, 
>   "first test"
>  },
>  { NULL, NULL, 0, NULL}
>};
>
>extern "C" void init_test_numarray()
>{
>  Py_InitModule( "_test_numarray", TestNumarrayMethods );
>  import_libnumarray();
>}
>  
>






From perry at stsci.edu  Mon Dec 16 14:25:01 2002
From: perry at stsci.edu (Perry Greenfield)
Date: Mon Dec 16 14:25:01 2002
Subject: [Numpy-discussion] Index array confusion...
In-Reply-To: <20021216192207.GA15908@idi.ntnu.no>
Message-ID: <JFEGLNDJEDNOMPPHDEJFCELNEBAA.perry@stsci.edu>

> And performance, perhaps?
> 
Certainly. That was one big factor (more with regard to
memory demands than CPU demands).

> And; from Todd's comment about a hypothetical built-in for the
> purpose, I assume there is no way of copying the contents of one slice
> to another without going via a temporary array?
> 
If you mean:

Without overwriting the slice being copied to with overlapping slices
(whether it's a problem depends on how they overlap and whether one is
reversed or not, and one other effect described below) on the same array,
that's generally true.

In numarray, there are cases where the slice is copied internally in
blocks. It's possible that you could get away with

x[:] = x[-1::-1]

if x is smaller than the blocksize being used. You would be crazy
to depend on this though :-) For large enough arrays (try 10000 elements),
it definitely won't work.

Perry



From edcjones at erols.com  Tue Dec 17 19:37:03 2002
From: edcjones at erols.com (Edward C. Jones)
Date: Tue Dec 17 19:37:03 2002
Subject: [Numpy-discussion] Comments on numarray - 4
Message-ID: <3DFFF4D6.3070005@erols.com>

The optional shape argument to fromstring is not documented.

-----------------

I suggest a documentation section on NumericTypes. "Int8.bytes" and
need documenting.

----------------

      Table of Automatic Coercions (found by experiment)

          B   I8   U8  I16  U16  I32  U32  I64  U64  F32  F64  C32  C64
      ------------------------------------------------------------------
    B |  *I8   I8   U8  I16  U16  I32  U32  I64  U64  F32  F64  C32  C64
   I8 |        I8 *I16  I16  U16  I32  U32  I64  U64  F32  F64  C32  C64
   U8 |             U8  I16  U16  I32  U32  I64  U64  F32  F64  C32  C64
  I16 |                 I16 *I32  I32  U32  I64  U64  F32  F64  C32  C64
  U16 |                      U16  I32  U32  I64  U64  F32  F64  C32  C64
  I32 |                           I32 *I64  I64  U64  F32  F64  C32  C64
  U32 |                                U32  I64  U64  F32  F64  C32  C64
  I64 |                                     I64 *I64 ?F32  F64  C32  C64
  U64 |                                          U64 ?F32  F64  C32  C64
  F32 |                                               F32  F64  C32  C64
  F64 |                                                    F64 ?C32  C64
  C32 |                                                         C32  C64
  C64 |                                                              C64

The names are listed in the order used in "numerictypes.py". The "*" 
cases are exceptions treated there. Are the "?" conversions correct? See 
"genericPromotionExclusions" in "numerictypes.py".




From jmiller at stsci.edu  Wed Dec 18 07:04:04 2002
From: jmiller at stsci.edu (Todd Miller)
Date: Wed Dec 18 07:04:04 2002
Subject: [Numpy-discussion] Comments on numarray - 4
References: <3DFFF4D6.3070005@erols.com>
Message-ID: <3E008F65.90802@stsci.edu>

Edward C. Jones wrote:

> The optional shape argument to fromstring is not documented.

Noted.

>
> -----------------
>
> I suggest a documentation section on NumericTypes. "Int8.bytes" and
> need documenting.

Noted.

>
> ----------------
>
>      Table of Automatic Coercions (found by experiment)
>
>          B   I8   U8  I16  U16  I32  U32  I64  U64  F32  F64  C32  C64
>      ------------------------------------------------------------------
>    B |  *I8   I8   U8  I16  U16  I32  U32  I64  U64  F32  F64  C32  C64
>   I8 |        I8 *I16  I16  U16  I32  U32  I64  U64  F32  F64  C32  C64
>   U8 |             U8  I16  U16  I32  U32  I64  U64  F32  F64  C32  C64
>  I16 |                 I16 *I32  I32  U32  I64  U64  F32  F64  C32  C64
>  U16 |                      U16  I32  U32  I64  U64  F32  F64  C32  C64
>  I32 |                           I32 *I64  I64  U64  F32  F64  C32  C64
>  U32 |                                U32  I64  U64  F32  F64  C32  C64
>  I64 |                                     I64 *I64 ?F32  F64  C32  C64
>  U64 |                                          U64 ?F32  F64  C32  C64
>  F32 |                                               F32  F64  C32  C64
>  F64 |                                                    F64 ?C32  C64
>  C32 |                                                         C32  C64
>  C64 |                                                              C64

Nice table!

>
> The names are listed in the order used in "numerictypes.py". The "*" 
> cases are exceptions treated there. Are the "?" conversions correct? See

No.  The Complex32/Float64 conversion has already been fixed in CVS.   
The Int64/Float32 coercion will be changed from Float32 to Float64.

> "genericPromotionExclusions" in "numerictypes.py". 

Thanks for the excellent feedback!

Todd




From edcjones at erols.com  Wed Dec 18 14:27:03 2002
From: edcjones at erols.com (Edward C. Jones)
Date: Wed Dec 18 14:27:03 2002
Subject: [Numpy-discussion] Two bugs in numerictypes.py
Message-ID: <3E00FDB5.2090804@erols.com>

In "numerictypes.py' it says "This module is designed so 'from 
numerictypes import *' is safe." When I "import numerictypes", and run 
dir(), I get:

['Any', 'AnyType', 'Bool', 'BooleanType', 'Byte', 'Complex', 
'Complex32', 'Complex64', 'ComplexType', 'Double', 'Float', 'Float32', 
'Float64', 'FloatingType', 'Int', 'Int16', 'Int32', 'Int64', 'Int8', 
'IntegralType', 'Long', 'MAX_ALIGN', 'NumericType', 'Short', 
'SignedIntegralType', 'SignedType', 'UInt16', 'UInt32', 'UInt64', 
'UInt8', 'UnsignedIntegralType', 'UnsignedType', '__builtins__', 
'__doc__', '__name__', 'genericCoercions', 'genericPromotionExclusions', 
'genericTypeRank', 'inttype1', 'inttype2', 'kind', 'mapto', 'maptype1', 
'maptype2', 'nt1', 'nt2', 'ntypesize1', 'ntypesize2', 'numinclude', 
'outtype', 'pythonTypeMap', 'pythonTypeRank', 'rank1', 'rank2', 
'scalarTypeMap', 'signedtype1', 'signedtype2', 'typeDict', 'typecode', 
'typecodes']

A bunch of leading "_" are needed.

----------------

My code is:

#! /usr/bin/env python
from numerictypes import UInt8
print UInt8 == "ABC"

The message is:

Traceback (most recent call last):
   File "./silly02.py", line 5, in ?
     print UInt8 == "ABC"
   File "/usr/lib/python2.2/site-packages/numarray/numerictypes.py",
line 101, in __cmp__
     other = typeDict[other]
KeyError: ABC

I would expect that the only object == to UInt8 be itself. Maybe add a 
function comparetypecodes(x, y) which returns True iff x and y are 
either NumericType's or strings which represent the same type.




From haase at msg.ucsf.edu  Wed Dec 18 17:23:05 2002
From: haase at msg.ucsf.edu (Sebastian Haase)
Date: Wed Dec 18 17:23:05 2002
Subject: [Numpy-discussion] Have a problem: what is attribute 'compress'
References: <3E00FDB5.2090804@erols.com>
Message-ID: <004b01c2a6fd$195c95f0$3b45da80@rodan>

Hi!
Somehow I have a problem with numarray. Please take a look at this:
>>>import numarray as na
>>>na.array([0, 0])
array([0, 0])
>>>na.array([0.0, 0.0])
Traceback (most recent call last):
  File "<input>", line 1, in ?
  File "C:\Python22\Lib\site-packages\numarray\numarray.py", line 581, in
__repr__
    MAX_LINE_WIDTH, PRECISION, SUPPRESS_SMALL, ', ', 1)
  File "C:\Python22\Lib\site-packages\numarray\arrayprint.py", line 163, in
array2string
    separator, array_output)
  File "C:\Python22\Lib\site-packages\numarray\arrayprint.py", line 125, in
_array2string
    format, item_length = _floatFormat(data, precision, suppress_small)
  File "C:\Python22\Lib\site-packages\numarray\arrayprint.py", line 246, in
_floatFormat
    non_zero = numarray.abs(numarray.compress(numarray.not_equal(data, 0),
data))
AttributeError: 'module' object has no attribute 'compress'

The same workes fine with Numeric. But I would prefer numarray because I'm
writing C++-extensions and I need "unsigned shorts".

What is this error about?

Thanks,
Sebastian





From jmiller at stsci.edu  Thu Dec 19 05:54:03 2002
From: jmiller at stsci.edu (Todd Miller)
Date: Thu Dec 19 05:54:03 2002
Subject: [Numpy-discussion] Have a problem: what is attribute 'compress'
References: <3E00FDB5.2090804@erols.com> <004b01c2a6fd$195c95f0$3b45da80@rodan>
Message-ID: <3E01D07B.3070009@stsci.edu>

Sebastian Haase wrote:

>Hi!
>Somehow I have a problem with numarray. Please take a look at this:
>
Hi Sebastian,

I've don't recall seeing anything like this,  nor can I reproduce it 
now.   If you've been following numarray for a while now,  I can say 
that it is important to remove the old version of numarray before 
installing the new version.   I recommend deleting your current 
installation and reinstalling numarray.

compress() is a ufunc,  much like add() or put().  It is defined in 
ndarray.py,  right after the import of the modules ufunc and _ufunc.   
_ufunc in particular is a problematic module,  because it has followed 
the atypical development path of moving from C-code to Python code. 
 Because of this, and the fact that a .so or .dll overrides a .py, 
 older installations interfere with newer ones.  The atypical path was 
required because the original _ufuncmodule.c was so large that it could 
not be compiled on some systems;  as a result,  I split _ufuncmodule.c 
into pieces by data type and now use _ufunc.py to glue the pieces together.

Good luck!    Please let me know if reinstalling doesn't clear up the 
problem.

Todd

>  
>
>>>>import numarray as na
>>>>na.array([0, 0])
>>>>        
>>>>
>array([0, 0])
>  
>
>>>>na.array([0.0, 0.0])
>>>>        
>>>>
>Traceback (most recent call last):
>  File "<input>", line 1, in ?
>  File "C:\Python22\Lib\site-packages\numarray\numarray.py", line 581, in
>__repr__
>    MAX_LINE_WIDTH, PRECISION, SUPPRESS_SMALL, ', ', 1)
>  File "C:\Python22\Lib\site-packages\numarray\arrayprint.py", line 163, in
>array2string
>    separator, array_output)
>  File "C:\Python22\Lib\site-packages\numarray\arrayprint.py", line 125, in
>_array2string
>    format, item_length = _floatFormat(data, precision, suppress_small)
>  File "C:\Python22\Lib\site-packages\numarray\arrayprint.py", line 246, in
>_floatFormat
>    non_zero = numarray.abs(numarray.compress(numarray.not_equal(data, 0),
>data))
>AttributeError: 'module' object has no attribute 'compress'
>
>The same workes fine with Numeric. But I would prefer numarray because I'm
>writing C++-extensions and I need "unsigned shorts".
>
>What is this error about?
>
>Thanks,
>Sebastian
>
>
>
>
>-------------------------------------------------------
>This SF.NET email is sponsored by: Order your Holiday Geek Presents Now!
>Green Lasers, Hip Geek T-Shirts, Remote Control Tanks, Caffeinated Soap,
>MP3 Players,  XBox Games,  Flying Saucers,  WebCams,  Smart Putty.
>T H I N K G E E K . C O M       http://www.thinkgeek.com/sf/
>_______________________________________________
>Numpy-discussion mailing list
>Numpy-discussion at lists.sourceforge.net
>https://lists.sourceforge.net/lists/listinfo/numpy-discussion
>  
>






From jmiller at stsci.edu  Thu Dec 19 06:37:11 2002
From: jmiller at stsci.edu (Todd Miller)
Date: Thu Dec 19 06:37:11 2002
Subject: [Numpy-discussion] howto build numarray 0.4 ? file 'Src/_convmodule.c'
 does not exist
References: <030301c29b1c$16db8530$3b45da80@rodan>
Message-ID: <3E01DA83.90101@stsci.edu>

Sebastian Haase wrote:

>Hi all,
>I downloaded numarray 0.4 about 5 minutes after I got the announce but 
>my naive 'python2.2 ./setup.py build'   gives this
>  
>
numarray-0.4 doesn't currently support a two step build process because 
code generation,  which is required,  doesn't occur unless the install 
command is present.   Unfortunately,  code generation *always* occurs 
when the install command is present,  which defeats the purpose of a 
seperate build.   This problem has been fixed in CVS by making code 
generation the result of an explicit setup.py switch (--gencode) which 
makes it possible to generate code during a build command.

One work-around is the following:

1. First generate code (as a non-priviledged user) using "python 
setup.py install".  Control-C as soon as the code starts to build.

2. Do  "python setup.py build"

3. Edit setup.py,  commenting out the 2 calls to os.system() in the 
function prepare().  This prevents codegeneration during the "real" install.

4. Do "python setup.py install" as root,  noting that no new code is 
generated, and everything is already built.

Todd

>haase at baboon:~/numarray-0.4: python2.2 ./setup.py build
>running build
>running build_py
>not copying Lib/ndarray.py (output up-to-date)
>not copying Lib/numtest.py (output up-to-date)
>not copying Lib/codegenerator.py (output up-to-date)
>not copying Lib/ufunc.py (output up-to-date)
>not copying Lib/testdata.py (output up-to-date)
>not copying Lib/numarray.py (output up-to-date)
>not copying Lib/ieeespecial.py (output up-to-date)
>not copying Lib/recarray.py (output up-to-date)
>not copying Lib/template.py (output up-to-date)
>not copying Lib/arrayprint.py (output up-to-date)
>not copying Lib/typeconv.py (output up-to-date)
>not copying Lib/numinclude.py (output up-to-date)
>not copying Lib/numarrayext.py (output up-to-date)
>not copying Lib/_ufunc.py (output up-to-date)
>not copying Lib/chararray.py (output up-to-date)
>not copying Lib/numtestall.py (output up-to-date)
>not copying Lib/numerictypes.py (output up-to-date)
>not copying Lib/memmap.py (output up-to-date)
>running build_ext
>building '_conv' extension
>error: file 'Src/_convmodule.c' does not exist
>
>What am I missing? I'm running Linux (debian woody) on i386.
>
>Thanks,
>Sebastian
>
>-------------------------------------------------------
>This SF.net email is sponsored by: Microsoft Visual Studio.NET 
>comprehensive development tool, built to increase your 
>productivity. Try a free online hosted session at:
>http://ads.sourceforge.net/cgi-bin/redirect.pl?micr0003en
>_______________________________________________
>Numpy-discussion mailing list
>Numpy-discussion at lists.sourceforge.net
>https://lists.sourceforge.net/lists/listinfo/numpy-discussion
>  
>






From Chris.Barker at noaa.gov  Fri Dec 20 11:41:07 2002
From: Chris.Barker at noaa.gov (Chris Barker)
Date: Fri Dec 20 11:41:07 2002
Subject: [Numpy-discussion] Another nice write-up about floating point issues.
References: <000001c29cc3$281ed420$6501a8c0@NICKLEBY> <Pine.SOL.4.50.0212052304510.28199-100000@death.OCF.Berkeley.EDU> <200212061138.30065.mclay@nist.gov>
Message-ID: <3E036FD9.61BC6CA@noaa.gov>

Hi all,

I just stumbled upon this paper by William Kahan, a Professor at
Berkeley that is well known for his work on the first Intel math
co-processor, and the development of IEE 754 (plus a lot of other
stuff). I took a course with him at Berkeley, and the man is brilliant.
So brilliant that is is very hard to follow him, as a student, if you
are not so sharp. I'm not, and I just squeeked by and missed a great
deal of what he tried to teach. I did gain a good appreciation of the
complexity of floating point arithmetic, however. Here is the paper as a
PDF:

http://www.cs.berkeley.edu/%7ewkahan/MgtkMath.pdf

There's a lot of other good stuff on his web page at:

http://www.cs.berkeley.edu/%7ewkahan/

THe paper is entitled "Marketing versus Mathematics". In particular,
take a look at the section on Quattro Pro on Page 13, for a nice
discussion of binary arithmetic displayed as decimal, which is an often
brought up subject in the Python newsgroups. He discusses how you'd
really have to display more than the 15 digits that Quattro displayed,
to avoid confusion.

"""
But no such cure can be liberated from little annoyances:
0.8  entered would display as 0.80000000000000004 to 17 sig. dec.;
"""
At the Python prompt:
>>> 0.8
0.80000000000000004

More evidence that the current Python way is "best", if confusing to
newbies. This would be another good paper to point people towards that
ask about floating point on the newsgroups.

-Chris

-- 
Christopher Barker, Ph.D.
Oceanographer
                                    		
NOAA/OR&R/HAZMAT         (206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115       (206) 526-6317   main reception

Chris.Barker at noaa.gov



From vo.chi.cong at is.titech.ac.jp  Sun Dec 22 00:34:04 2002
From: vo.chi.cong at is.titech.ac.jp (Cong)
Date: Sun Dec 22 00:34:04 2002
Subject: [Numpy-discussion] bug in Numeric, LinearAlgebra
Message-ID: <20021222.173433.59464351.vo.chi.cong@is.titech.ac.jp>

I've just upgraded from Numeric-21.0 to Numeric-22.0 
( python-2.2.2 ) and found a bug. Do anyone use LinearAlgebra too ?

Now, I downed it to Numeric 21.0 . This version works very well.

Cong

$ python
Python 2.2.2 (#1, Dec 14 2002, 05:45:37) 
[GCC 3.2.1] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import LinearAlgebra
Traceback (most recent call last):
  File "<stdin>", line 1, in ?
  File "/usr/lib/python2.2/site-packages/Numeric/LinearAlgebra.py", line 10, in ?
    import MLab
  File "/usr/lib/python2.2/site-packages/Numeric/MLab.py", line 20, in ?
    import RandomArray
  File "/usr/lib/python2.2/site-packages/Numeric/RandomArray.py", line 30, in ?
    seed()
  File "/usr/lib/python2.2/site-packages/Numeric/RandomArray.py", line 24, in seed
    ndigits = int(math.log10(t))
OverflowError: float too large to convert




From vo.chi.cong at is.titech.ac.jp  Sun Dec 22 00:52:04 2002
From: vo.chi.cong at is.titech.ac.jp (Cong)
Date: Sun Dec 22 00:52:04 2002
Subject: [Numpy-discussion] Re: bug in Numeric, LinearAlgebra
In-Reply-To: <20021222.173433.59464351.vo.chi.cong@is.titech.ac.jp>
References: <20021222.173433.59464351.vo.chi.cong@is.titech.ac.jp>
Message-ID: <20021222.175219.74752851.vo.chi.cong@is.titech.ac.jp>

Cong> I've just upgraded from Numeric-21.0 to Numeric-22.0 
Cong> ( python-2.2.2 ) and found a bug. Do anyone use LinearAlgebra too ?
Cong> 
Cong> Now, I downed it to Numeric 21.0 . This version works very well.

I found that now this version doesnot work too. So I delete Numeric
21.0 complettely then install Numeric 22.0 again. After that the error
message changed to:

$ python
Python 2.2.2 (#1, Dec 22 2002, 17:44:21) 
[GCC 3.2.1] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import LinearAlgebra
Traceback (most recent call last):
  File "<stdin>", line 1, in ?
  File "/usr/lib/python2.2/site-packages/Numeric/LinearAlgebra.py", line 10, in ?
    import MLab
  File "/usr/lib/python2.2/site-packages/Numeric/MLab.py", line 20, in ?
    import RandomArray
  File "/usr/lib/python2.2/site-packages/Numeric/RandomArray.py", line 30, in ?
    seed()
  File "/usr/lib/python2.2/site-packages/Numeric/RandomArray.py", line 26, in seed
    x = int(t/base)
OverflowError: float too large to conver

I can not figure out what is happenning. Temporarily, I import
LinearAlgebra this way:

try:
	import LinearAlgebra
except:
	import LinearAlgebra

And this worked, for Numeric 22.0 on python 2.2.2
as well as for Numeric 21.0 on python 2.2.2 .

// Cong

Cong> 
Cong> Cong
Cong> 
Cong> $ python
Cong> Python 2.2.2 (#1, Dec 14 2002, 05:45:37) 
Cong> [GCC 3.2.1] on linux2
Cong> Type "help", "copyright", "credits" or "license" for more information.
Cong> >>> import LinearAlgebra
Cong> Traceback (most recent call last):
Cong>   File "<stdin>", line 1, in ?
Cong>   File "/usr/lib/python2.2/site-packages/Numeric/LinearAlgebra.py", line 10, in ?
Cong>     import MLab
Cong>   File "/usr/lib/python2.2/site-packages/Numeric/MLab.py", line 20, in ?
Cong>     import RandomArray
Cong>   File "/usr/lib/python2.2/site-packages/Numeric/RandomArray.py", line 30, in ?
Cong>     seed()
Cong>   File "/usr/lib/python2.2/site-packages/Numeric/RandomArray.py", line 24, in seed
Cong>     ndigits = int(math.log10(t))
Cong> OverflowError: float too large to convert
Cong> 



From edcjones at erols.com  Sun Dec 22 18:34:04 2002
From: edcjones at erols.com (Edward C. Jones)
Date: Sun Dec 22 18:34:04 2002
Subject: [Numpy-discussion] "<<", ">>" undocumented
Message-ID: <3E067DD5.6020609@erols.com>

The operators "<<" and ">>" seem to be undocumented. Is there a reason 
for this?

Ed Jones




From magnus at hetland.org  Fri Dec 27 13:30:03 2002
From: magnus at hetland.org (Magnus Lie Hetland)
Date: Fri Dec 27 13:30:03 2002
Subject: [Numpy-discussion] Non-numerical info associated with sub-arrays
Message-ID: <20021227212942.GA11897@idi.ntnu.no>

I'm working on some two-dimensional tables of data, where some data
are numerical, while other aren't. I'd like to use numarray's
numerical capabilities with the numerical parts (columns) while
keeping the data in each row together. (I'm sure this generalizes to
more dimensions, and to sub-arrays in general, not just rows.)

It's not a hard problem, really, but the obvious solution--to keep
the other rows in separate arrays/lists and just juggle things
around--seems a bit clunky. I was just wondering if anyone had other
ideas (would it be practical to include all the data in a single array
somehow--I seem to recall that Numeric could have arbitrary object
arrays, but I'm not sure whether numarray supports this?) or perhaps
some hints on how to organize code around this? I wrote a small class
that wraps things up and works a bit lik R/S-plus's data frames; is
there some other more standard code out there for this sort of thing?
(It's a problem that crops up often in data processing of various
kinds...)

Thanks,

Magnus

-- 
Magnus Lie Hetland
http://hetland.org



From tchur at optushome.com.au  Fri Dec 27 14:13:02 2002
From: tchur at optushome.com.au (Tim Churches)
Date: Fri Dec 27 14:13:02 2002
Subject: [Numpy-discussion] Non-numerical info associated with sub-arrays
In-Reply-To: <20021227212942.GA11897@idi.ntnu.no>
References: <20021227212942.GA11897@idi.ntnu.no>
Message-ID: <1041102817.1264.142.camel@localhost.localdomain>

On Fri, 2002-12-27 at 11:29, Magnus Lie Hetland wrote:
> I'm working on some two-dimensional tables of data, where some data
> are numerical, while other aren't. I'd like to use numarray's
> numerical capabilities with the numerical parts (columns) while
> keeping the data in each row together. (I'm sure this generalizes to
> more dimensions, and to sub-arrays in general, not just rows.)
> 
> It's not a hard problem, really, but the obvious solution--to keep
> the other rows in separate arrays/lists and just juggle things
> around--seems a bit clunky. I was just wondering if anyone had other
> ideas (would it be practical to include all the data in a single array
> somehow--I seem to recall that Numeric could have arbitrary object
> arrays, but I'm not sure whether numarray supports this?) or perhaps
> some hints on how to organize code around this? I wrote a small class
> that wraps things up and works a bit lik R/S-plus's data frames; is
> there some other more standard code out there for this sort of thing?
> (It's a problem that crops up often in data processing of various
> kinds...)

Have a look at the discussion on RecordArrays in this overview of
Numarray: http://stsdas.stsci.edu/numarray/DesignOverview.html

However, in the meantime, as you note, its not too hard to write a class
which emulates R/S-Plus data frames. Just store each column in its own
Numeric array of the appropriate type (which might be the PyObject
types, which can hold any Python object type), and have the wrapper
class implement __getitem__ etc to collect the relevant "rows" from each
column and return them as a complete row as a dict or a sequence. Not
that fast, but not slow either. You can implement a generator to allow
cursor-like traversal of the all the rows if you like. Happy to
collaborate on furthering this idea. By memory-mapping disc-based
versions of the  Numeric arrays, and using the BsdDb3 record number
database format for the string columns, you can even make a disc-based
"record array" which can be larger than available RAM+swap. I hope to
release code written under contract by Dave Cole (see
http://www.object-craft.com.au ) which illustrates this idea in the next
month or so (but I've been saying that to myself for a year or more...).

Tim C
 





From magnus at hetland.org  Fri Dec 27 14:57:01 2002
From: magnus at hetland.org (Magnus Lie Hetland)
Date: Fri Dec 27 14:57:01 2002
Subject: [Numpy-discussion] Non-numerical info associated with sub-arrays
In-Reply-To: <1041102817.1264.142.camel@localhost.localdomain>
References: <20021227212942.GA11897@idi.ntnu.no> <1041102817.1264.142.camel@localhost.localdomain>
Message-ID: <20021227225549.GA21921@idi.ntnu.no>

Tim Churches <tchur at optushome.com.au>:
[snip]
> Have a look at the discussion on RecordArrays in this overview of
> Numarray: http://stsdas.stsci.edu/numarray/DesignOverview.html

Sounds interesting.

> However, in the meantime, as you note, its not too hard to write a class
> which emulates R/S-Plus data frames. Just store each column in its own
> Numeric array of the appropriate type

Yeah -- it's just that I'd like to keep a set of columns collected as
a two-dimensional array, to allow horizontal summing and the like.
(Not much more complicated, but an extra issue to address.)

> (which might be the PyObject
> types, which can hold any Python object type),

Hm. Yes. I can't seem to find these anymore. I seem to recall using
type='o' or something in Numeric, but I can't find the right type
objects now... (Guess I'm just reading the docs and dir(numeric)
poorly...) It would be nice if array(['foo']) just worked. Oh, well.

[snip]
> Happy to
> collaborate on furthering this idea.

That would be great (even though I don't really have any time to use
for this -- it's just a really tiny part of a small project I'm
working on :)

> By memory-mapping disc-based
> versions of the  Numeric arrays, and using the BsdDb3 record number
> database format for the string columns, you can even make a disc-based
> "record array" which can be larger than available RAM+swap.

Sounds quite useful, although quite similar to MetaKit. (I suppose I
could use some functions from numarray on columns in MetaKit... But
that might just be too weird -- and it would still just be a
collection of columns :])

[snip]

Thanks for your input.

-- 
Magnus Lie Hetland
http://hetland.org



From tchur at optushome.com.au  Fri Dec 27 15:50:02 2002
From: tchur at optushome.com.au (Tim Churches)
Date: Fri Dec 27 15:50:02 2002
Subject: [Numpy-discussion] Non-numerical info associated with sub-arrays
In-Reply-To: <20021227225549.GA21921@idi.ntnu.no>
References: <20021227212942.GA11897@idi.ntnu.no>
	<1041102817.1264.142.camel@localhost.localdomain> 
	<20021227225549.GA21921@idi.ntnu.no>
Message-ID: <1041108637.1260.153.camel@localhost.localdomain>

On Fri, 2002-12-27 at 12:55, Magnus Lie Hetland wrote:
> Tim Churches <tchur at optushome.com.au>:
> [snip]
> > Have a look at the discussion on RecordArrays in this overview of
> > Numarray: http://stsdas.stsci.edu/numarray/DesignOverview.html
> 
> Sounds interesting.
> 
> > However, in the meantime, as you note, its not too hard to write a class
> > which emulates R/S-Plus data frames. Just store each column in its own
> > Numeric array of the appropriate type
> 
> Yeah -- it's just that I'd like to keep a set of columns collected as
> a two-dimensional array, to allow horizontal summing and the like.
> (Not much more complicated, but an extra issue to address.)
> 
> > (which might be the PyObject
> > types, which can hold any Python object type),
> 
> Hm. Yes. I can't seem to find these anymore. I seem to recall using
> type='o' or something in Numeric, but I can't find the right type
> objects now... (Guess I'm just reading the docs and dir(numeric)
> poorly...) It would be nice if array(['foo']) just worked. Oh, well.

Just like this:

>>> import Numeric
>>> a = Numeric.array(['a','b','c'],typecode=Numeric.PyObject)
>>> a
array([a , b , c ],'O')
>>>

> 
> > By memory-mapping disc-based
> > versions of the  Numeric arrays, and using the BsdDb3 record number
> > database format for the string columns, you can even make a disc-based
> > "record array" which can be larger than available RAM+swap.
> 
> Sounds quite useful, although quite similar to MetaKit. (I suppose I
> could use some functions from numarray on columns in MetaKit... But
> that might just be too weird -- and it would still just be a
> collection of columns :])

I really like MetaKit's column-based storage, but it just doesn't scale
well (on the author's admission, and verified empirically) - beyond a
few 10**5 records, it bogs down terribly, whereas memory-mapped NumPy
plus BsdDb3 recno databse for strings scales well to many tens of
millions of records (or more, but thats as far as I have tested).

Tim C





From magnus at hetland.org  Fri Dec 27 16:07:03 2002
From: magnus at hetland.org (Magnus Lie Hetland)
Date: Fri Dec 27 16:07:03 2002
Subject: [Numpy-discussion] Non-numerical info associated with sub-arrays
In-Reply-To: <1041108637.1260.153.camel@localhost.localdomain>
References: <20021227212942.GA11897@idi.ntnu.no> <1041102817.1264.142.camel@localhost.localdomain> <20021227225549.GA21921@idi.ntnu.no> <1041108637.1260.153.camel@localhost.localdomain>
Message-ID: <20021228000628.GA23661@idi.ntnu.no>

Tim Churches <tchur at optushome.com.au>:
[snip]
> Just like this:
> 
> >>> import Numeric
> >>> a = Numeric.array(['a','b','c'],typecode=Numeric.PyObject)
> >>> a
> array([a , b , c ],'O')
> >>>

As you may have noticed from my previous descriptions, I'm using
numarray, not Numeric. I've used this in Numeric before -- I just
can't find the equivalent functionality in numarray :)

[snip]
> I really like MetaKit's column-based storage,

Me too.

> but it just doesn't scale
> well (on the author's admission, and verified empirically)

Yes, you're right.

> - beyond a
> few 10**5 records, it bogs down terribly, whereas memory-mapped NumPy
> plus BsdDb3 recno databse for strings scales well to many tens of
> millions of records (or more, but thats as far as I have tested).

Impressive!

Now this *does* sound interesting... The project I originally posted
about only has a few hundred records, so I'm only considering numarray
for expressiveness/readability there -- performance is not an issue.
But using bsddb and numarray (or Numeric) together like this seems
useful in many applications.

> Tim C

-- 
Magnus Lie Hetland
http://hetland.org



From perry at stsci.edu  Fri Dec 27 17:23:02 2002
From: perry at stsci.edu (Perry Greenfield)
Date: Fri Dec 27 17:23:02 2002
Subject: [Numpy-discussion] Non-numerical info associated with sub-arrays
In-Reply-To: <20021228000628.GA23661@idi.ntnu.no>
Message-ID: <NEBBIJKBMLDBLNCEEFOCCECHCPAA.perry@stsci.edu>

Magnus Lie Hetland writes:
> Tim Churches <tchur at optushome.com.au>:
> [snip]
> > Just like this:
> > 
> > >>> import Numeric
> > >>> a = Numeric.array(['a','b','c'],typecode=Numeric.PyObject)
> > >>> a
> > array([a , b , c ],'O')
> > >>>
> 
> As you may have noticed from my previous descriptions, I'm using
> numarray, not Numeric. I've used this in Numeric before -- I just
> can't find the equivalent functionality in numarray :)
> 
At the moment, PyObject arrays are not supported (mainly because
it hasn't been a priority for our needs yet. But if all one needs
is such an array to hold PyObjects and nothing more (for example, 
we envisioned more sophisticated uses such as apply object methods
to the array and returning arrays of the results) than associative
purposes (and being able to set and get array values), it should
be quite easy to add this capability. In fact one could subclass
NDArray and just define the _get and _setitem methods (I forget
the exact names) and probably customize the __init__ and have
the functionality that you need. I can take a look at it next
week (or if you feel bold, look at NDArray yourself). As with
Numeric, speed is sacrificed when using such arrays. The presumption 
is that one is using Numeric or numarray on such things mainly for
the convenience of the array manipulations, not the kind of 
efficiency that bulk numerical operations provide.

Combining that with RecordArrays may be a bit trickier in the
sense that RecordArrays presume that records use the same buffer
for all data. If one doesn't mind storing PyObject pointers in
that data array, it probably is also fairly simple to extend
it (but I frankly haven't thought this through so I may be wrong
about how easy it is). Doing this may require some thought about
how to pickle such arrays. Of course, one may have a set of arrays
as Tim suggests which also acts like a record array where there is
no single data buffer. Our RecordArrays were intended to map
to data files closely, but other variants are certainly possible.

Perry Greenfield 




From falted at openlc.org  Sat Dec 28 06:42:01 2002
From: falted at openlc.org (Francesc Alted)
Date: Sat Dec 28 06:42:01 2002
Subject: [Numpy-discussion] Non-numerical info associated with sub-arrays
In-Reply-To: <20021227212942.GA11897@idi.ntnu.no>
References: <20021227212942.GA11897@idi.ntnu.no>
Message-ID: <1041085706.3e0db50a682fb@webmail.imk.es>

Mensaje citado por: Magnus Lie Hetland <magnus at hetland.org>:

> I'm working on some two-dimensional tables of data, where some data
> are numerical, while other aren't. I'd like to use numarray's
> numerical capabilities with the numerical parts (columns) while
> keeping the data in each row together. (I'm sure this generalizes to
> more dimensions, and to sub-arrays in general, not just rows.)

You may want to have a look at PyTables (http://pytables.sourceforge.net). 
It's designed to be used in scenarios similar to that you are exposing.

It supports Numeric objects and although columns in tables are not 
automatically converted to Numeric o numarray objects, you can build them on 
the flight easily using its powerful selection capabilities.

It uses HDF5 (http://hdf.ncsa.uiuc.edu/HDF5/) format to save its data, so you 
can read PyTables files in a variety of languages and platforms.

Cheers,

Francesc Alted



From falted at openlc.org  Sat Dec 28 07:11:01 2002
From: falted at openlc.org (Francesc Alted)
Date: Sat Dec 28 07:11:01 2002
Subject: [Numpy-discussion] Non-numerical info associated with sub-arrays
In-Reply-To: <NEBBIJKBMLDBLNCEEFOCCECHCPAA.perry@stsci.edu>
References: <NEBBIJKBMLDBLNCEEFOCCECHCPAA.perry@stsci.edu>
Message-ID: <1041087459.3e0dbbe376d61@webmail.imk.es>

Mensaje citado por: Perry Greenfield <perry at stsci.edu>:
> Our RecordArrays were intended to map
> to data files closely, but other variants are certainly possible.

In fact, I'm thinking of adopting numarray for my pytables project, but I 
don't like the fact that data is not natively aligned inside recarrays, i.e. 
there is not a gap between the different fields even if datatypes doesn't 
match the "native" architecture alignement. IMO this can affect very much to 
the read/write efficency when one wants to work with data rows or columns of 
recarrays objects.

Are there any plans to support this "natural" alignment in addition of the non-
alignment schema present right now?.

Francesc Alted



From magnus at hetland.org  Sat Dec 28 08:14:03 2002
From: magnus at hetland.org (Magnus Lie Hetland)
Date: Sat Dec 28 08:14:03 2002
Subject: [Numpy-discussion] Non-numerical info associated with sub-arrays
In-Reply-To: <1041085706.3e0db50a682fb@webmail.imk.es>
References: <20021227212942.GA11897@idi.ntnu.no> <1041085706.3e0db50a682fb@webmail.imk.es>
Message-ID: <20021228161244.GB11568@idi.ntnu.no>

Francesc Alted <falted at openlc.org>:
[snip]
> You may want to have a look at PyTables (http://pytables.sourceforge.net). 
> It's designed to be used in scenarios similar to that you are exposing.
[snip]

Sounds interesting. I'll look into it.

-- 
Magnus Lie Hetland
http://hetland.org



From edcjones at erols.com  Sat Dec 28 09:52:23 2002
From: edcjones at erols.com (Edward C. Jones)
Date: Sat Dec 28 09:52:23 2002
Subject: [Numpy-discussion] Further comments
Message-ID: <3E0DEC9C.8080103@erols.com>

Should the variable "type" be used in numarray? It is an important 
function in Python.

---------------------------------------

There needs to be a function or method that returns the number of 
elements in an array.

def Elements(array):
     """Number of elements in an array.

     This version is slow.
     """
     return numarray.multiply.reduce(array.shape)

---------------------------------------

I write code using both PIL and numarray. PIL uses strings for modes and
numarray uses (optionally) strings as typecodes. This causes problems.
One fix is to emit a DeprecationWarning when string typecodes are used. 
Two functions are needed: StringTypeWarningOn and StringTypeWarningOff. 
The default should be to ignore this warning.

In my code I use the following workarounds:

def SameType(x, y):
     """Are the two input the same object of NumericType?"""
     if isinstance(x, NumericType) and isinstance(y, NumericType) \
            and x == y:
         return True
     return False

def IsTypeInList(typecode, seq):
     """Is a NumericType object in a list of NumericType objects?"""
     if not isinstance(typecode, NumericType):
         return False
     for item in seq:
         if isinstance(item, NumericType) and typecode == item:
             return True
     return False

---------------------------------------

The following function is useful for downsizing arrays. I suggest that
it should be a ufunc method. This is how I have used reduceat in Numeric.

def blockreduce(array, blocksizes, ufunc):
     """Apply ufunc.reduce to blocks in an array."""
     dims = len(array.shape)
     if type(blocksizes) is IntType:
         blocksizes = dims * [blocksizes]
     if len(blocksizes) != dims:
         raise TypeError, 'blocksizes must be same length as shape'
     for i in range(dims):
         if array.shape[i] % blocksizes[i] != 0:
             raise ValueError, 'blocksizes must exactly divide ' \
                'the corresponding array dimension'
     for i in range(dims):
         array = array.copy()
         newshape = (array.shape[0] / blocksizes[i], blocksizes[i]) + \
                       array.shape[1:]
         array.shape = newshape
         array = ufunc.reduce(array, 1)
         dims = len(array.shape)
         # (0,1,2,3) --> (1,2,3,0)
         perm =  tuple(range(1, dims)) + (0,)
         array = numarray.transpose(array, perm)
     return array




From perry at stsci.edu  Sat Dec 28 11:03:35 2002
From: perry at stsci.edu (Perry Greenfield)
Date: Sat Dec 28 11:03:35 2002
Subject: [Numpy-discussion] Non-numerical info associated with sub-arrays
In-Reply-To: <1041087459.3e0dbbe376d61@webmail.imk.es>
Message-ID: <NEBBIJKBMLDBLNCEEFOCOECHCPAA.perry@stsci.edu>

> In fact, I'm thinking of adopting numarray for my pytables project, but I
> don't like the fact that data is not natively aligned inside
> recarrays, i.e.
> there is not a gap between the different fields even if datatypes doesn't
> match the "native" architecture alignement. IMO this can affect
> very much to
> the read/write efficency when one wants to work with data rows or
> columns of
> recarrays objects.
>
> Are there any plans to support this "natural" alignment in
> addition of the non-
> alignment schema present right now?.
>
Are you asking for an option to create record arrays with
aligned fields (in the sense that the addresses of all values
are consistent with their type)? Or are you arguing that
non-aligned columns must be prohibited? The former is certainly
possible (not not very difficult to implement; basically it requires
that record sizes must be a multiple of the largest numerical type,
and that padding is placed within records to ensure that all fields
have offsets that are a multiple of their size). We cannot accept
the latter since we need to access data that are stored in such
a non-aligned manner in data files.

> Francesc Alted

Thanks, Perry Greenfield





From magnus at hetland.org  Sat Dec 28 13:30:03 2002
From: magnus at hetland.org (Magnus Lie Hetland)
Date: Sat Dec 28 13:30:03 2002
Subject: [Numpy-discussion] from RandomArray2 import *?
Message-ID: <20021228212928.GA19852@idi.ntnu.no>

I just tried a starred import from RandomArray2, but it seems that it
didn't work properly -- e.g. the uniform function wasn't imported. And
I can't see any __all__ attribute; what's going on?

-- 
Magnus Lie Hetland
http://hetland.org



From perry at stsci.edu  Sat Dec 28 13:54:02 2002
From: perry at stsci.edu (Perry Greenfield)
Date: Sat Dec 28 13:54:02 2002
Subject: [Numpy-discussion] Further comments
In-Reply-To: <3E0DEC9C.8080103@erols.com>
Message-ID: <NEBBIJKBMLDBLNCEEFOCIECICPAA.perry@stsci.edu>

Edward Jones writes: 
> 
> Should the variable "type" be used in numarray? It is an important 
> function in Python.
> 
We pondered this for a while. Yes, it conflicts with the built 
in function (and so that function is aliased within the numarray
code to a different name). Do you see this causing problems for 
you? Normally there is no conflict in using it as a keyword name
for function or method arguments, but if you can show practical
cases where it is a problem, we may reconsider. It seemed to us
that type was the best name for the keyword, particularly since
we were discouraging people from thinging of typecodes.

> ---------------------------------------
> 
> There needs to be a function or method that returns the number of 
> elements in an array.
> 
> def Elements(array):
>      """Number of elements in an array.
> 
>      This version is slow.
>      """
>      return numarray.multiply.reduce(array.shape)
> 
> ---------------------------------------
> 
I don't disagree. It would be nice to have a consensus for what the
function name (and perhaps an array attribute?) should be. I believe
IDL uses nelements (which I find more suggestive than elements), but
perhaps others have better names.

> I write code using both PIL and numarray. PIL uses strings for modes and
> numarray uses (optionally) strings as typecodes. This causes problems.
> One fix is to emit a DeprecationWarning when string typecodes are used. 
> Two functions are needed: StringTypeWarningOn and StringTypeWarningOff. 
> The default should be to ignore this warning.
> 
I'm not sure I understand. Can you give me an example of problem
code or usage? It sounds like you are trying to test the types
of  PIL and numarray objects in a generic sense. But I'd understand
better if you could show an example.

> 
> The following function is useful for downsizing arrays. I suggest that
> it should be a ufunc method. This is how I have used reduceat in Numeric.
> 
> def blockreduce(array, blocksizes, ufunc):
>      """Apply ufunc.reduce to blocks in an array."""
>      dims = len(array.shape)
>      if type(blocksizes) is IntType:
>          blocksizes = dims * [blocksizes]
>      if len(blocksizes) != dims:
>          raise TypeError, 'blocksizes must be same length as shape'
>      for i in range(dims):
>          if array.shape[i] % blocksizes[i] != 0:
>              raise ValueError, 'blocksizes must exactly divide ' \
>                 'the corresponding array dimension'
>      for i in range(dims):
>          array = array.copy()
>          newshape = (array.shape[0] / blocksizes[i], blocksizes[i]) + \
>                        array.shape[1:]
>          array.shape = newshape
>          array = ufunc.reduce(array, 1)
>          dims = len(array.shape)
>          # (0,1,2,3) --> (1,2,3,0)
>          perm =  tuple(range(1, dims)) + (0,)
>          array = numarray.transpose(array, perm)
>      return array
> 
We certainly have frequent need for a binning function (i.e.,
the equivalent of a blockreduce for add). Do others see this
as a generally useful extension for all binary ufuncs?
(As a side comment, the requirement that the dimensions be
evenly divisible is often a pain; one question is to ask whether
this must be a requirement or not, or whether there is a way to
disable this requirement, i.e., permit the last block to be smaller
than usual?)

Thanks, Perry Greenfield 




From perry at stsci.edu  Sat Dec 28 14:29:04 2002
From: perry at stsci.edu (Perry Greenfield)
Date: Sat Dec 28 14:29:04 2002
Subject: [Numpy-discussion] from RandomArray2 import *?
In-Reply-To: <20021228212928.GA19852@idi.ntnu.no>
Message-ID: <NEBBIJKBMLDBLNCEEFOCMECJCPAA.perry@stsci.edu>

Hmmm, I see what you mean. Looks like we have something to
fix. Thanks for pointing this out.

Perry

> -----Original Message-----
> From: numpy-discussion-admin at lists.sourceforge.net
> [mailto:numpy-discussion-admin at lists.sourceforge.net]On Behalf Of Magnus
> Lie Hetland
> Sent: Saturday, December 28, 2002 4:29 PM
> To: Numpy-discussion
> Subject: [Numpy-discussion] from RandomArray2 import *?
> 
> 
> I just tried a starred import from RandomArray2, but it seems that it
> didn't work properly -- e.g. the uniform function wasn't imported. And
> I can't see any __all__ attribute; what's going on?
> 
> -- 
> Magnus Lie Hetland
> http://hetland.org
> 
> 
> -------------------------------------------------------
> This sf.net email is sponsored by:ThinkGeek
> Welcome to geek heaven.
> http://thinkgeek.com/sf
> _______________________________________________
> Numpy-discussion mailing list
> Numpy-discussion at lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/numpy-discussion
> 




From magnus at hetland.org  Sat Dec 28 14:29:04 2002
From: magnus at hetland.org (Magnus Lie Hetland)
Date: Sat Dec 28 14:29:04 2002
Subject: [Numpy-discussion] Further comments
In-Reply-To: <NEBBIJKBMLDBLNCEEFOCIECICPAA.perry@stsci.edu>
References: <3E0DEC9C.8080103@erols.com> <NEBBIJKBMLDBLNCEEFOCIECICPAA.perry@stsci.edu>
Message-ID: <20021228222836.GA21002@idi.ntnu.no>

Perry Greenfield <perry at stsci.edu>:
>
> > There needs to be a function or method that returns the number of 
> > elements in an array.
> > 
> > def Elements(array):
> >      """Number of elements in an array.
> > 
> >      This version is slow.
> >      """
> >      return numarray.multiply.reduce(array.shape)

len(ravel(array)) may be a faster in some cases, it seems. (Or
ravel(array).shape[0], for that matter).

BTW: Using the name "array" here seems just as bad as the use of
"type" in numarray... Just a thought :]

[snip]
> > The following function is useful for downsizing arrays. I suggest that
> > it should be a ufunc method. This is how I have used reduceat in Numeric.
> > 
> > def blockreduce(array, blocksizes, ufunc):
> >      """Apply ufunc.reduce to blocks in an array."""
[snip]
> We certainly have frequent need for a binning function (i.e.,
> the equivalent of a blockreduce for add). Do others see this
> as a generally useful extension for all binary ufuncs?

Not sure if I fully understand this -- but in my work on sequences and
time series, I certainly need to do stuff like sums and averages over
fixed-sized blocks of one-dimensional arrays... (Discretization.) But
there are easier ways of doing that, I suppose (with reshape or
indices, for example).

-- 
Magnus Lie Hetland
http://hetland.org



From magnus at hetland.org  Sat Dec 28 14:35:02 2002
From: magnus at hetland.org (Magnus Lie Hetland)
Date: Sat Dec 28 14:35:02 2002
Subject: [Numpy-discussion] Increased polymorphism?
Message-ID: <20021228223435.GA21226@idi.ntnu.no>

I know PyObject arrays aren't supported in numarray yet -- but how
about making the functions more polymorphic? Yes, they work on other
sequence types such as lists -- but there seems to be quite a bit of
type checking going on. I suppose that is necessary for performance
reasons, but how about some polymorphic behaviour in the cases where
an unknown type is encountered?

It would be nice, for example, if sum() worked on general iterators
and generators, and if argsort() worked on, e.g., lists of strings...
This would make it easier to integrate the purely numeric stuff with
other parts of Python programs...

It may be that I'm going beyond the scope of numarray here, but this
sort of agnostic signature-based polymorphism is very Pythonic.

-- 
Magnus Lie Hetland
http://hetland.org



From falted at openlc.org  Sat Dec 28 17:00:02 2002
From: falted at openlc.org (Francesc Alted)
Date: Sat Dec 28 17:00:02 2002
Subject: [Numpy-discussion] Non-numerical info associated with sub-arrays
In-Reply-To: <NEBBIJKBMLDBLNCEEFOCOECHCPAA.perry@stsci.edu>
References: <NEBBIJKBMLDBLNCEEFOCOECHCPAA.perry@stsci.edu>
Message-ID: <1041122789.3e0e45e50b668@webmail.imk.es>

Mensaje citado por: Perry Greenfield <perry at stsci.edu>:

> Are you asking for an option to create record arrays with
> aligned fields (in the sense that the addresses of all values
> are consistent with their type)?

Yes, I'm advocating for that

> Or are you arguing that
> non-aligned columns must be prohibited? The former is certainly
> possible (not not very difficult to implement; basically it requires
> that record sizes must be a multiple of the largest numerical type,
> and that padding is placed within records to ensure that all fields
> have offsets that are a multiple of their size).

Well, for the sake of keeping the size of dataset to a minimum, I think it's 
not necessary to adjust all the record field sizes to the largest data type 
because depending on the type of the field, the padding can be shorter or 
larger.

For example, short ints only needs to be aligned in two-byte basis, while 
doubles need 4 bytes (or 8, I don't remember well). In any case, this depends 
on the architecture. But it is still possible to figure out safely what is the 
required minimum alignments for the different types. Look at Python's struct 
module for a good example on how you can reduce the padding to a minimum, 
without sacrificing performance.

Francesc Alted



From magnus at hetland.org  Sun Dec 29 13:37:02 2002
From: magnus at hetland.org (Magnus Lie Hetland)
Date: Sun Dec 29 13:37:02 2002
Subject: [Numpy-discussion] average()...?
Message-ID: <20021229213556.GA7942@idi.ntnu.no>

Just a quick question (which I believe I have asked before sometime,
but I don't recall the outcome ;) -- why is the average() function
from MaskedArray not available in numarray? It seems like a very
useful function?

It seems that looking through the list of functions in MaskedArray
might be useful in finding candidates for numarray, in so far as they
differ. (Wouldn't be logical if they supported the same functions,
more or less?) For example, MaskedArray has the function count(),
which counts the number of elements in an array -- exactly what was
asked for yesterday, IIRC. (The use may be more obvious for masked
arrays, where it counts non-masked elements, but it works on plain
arrays too.)

If nothing else, perhaps it would be possible to put

from MA import count, average

in numarray.py? (And similarly for other functions that may be missing
in numarray -- I didn't check that closely.) Of course this wouldn't
work if MA isn't available... :/

-- 
Magnus Lie Hetland
http://hetland.org



From magnus at hetland.org  Sun Dec 29 13:54:02 2002
From: magnus at hetland.org (Magnus Lie Hetland)
Date: Sun Dec 29 13:54:02 2002
Subject: [Numpy-discussion] indices with list arg?
Message-ID: <20021229215310.GA8757@idi.ntnu.no>

Why isn't indices([42]) (or foo.setshape([42]), for that matter)
allowed? Overzealous type checking, or a necessity of the C code?

-- 
Magnus Lie Hetland
http://hetland.org



From jmiller at stsci.edu  Mon Dec 30 06:27:06 2002
From: jmiller at stsci.edu (Todd Miller)
Date: Mon Dec 30 06:27:06 2002
Subject: [Numpy-discussion] indices with list arg?
References: <20021229215310.GA8757@idi.ntnu.no>
Message-ID: <3E10593E.8010802@stsci.edu>

Magnus Lie Hetland wrote:

>Why isn't indices([42]) (or foo.setshape([42]), for that matter)
>allowed? Overzealous type checking, or a necessity of the C code?
>
>  
>
Neither.   It  looks like a limitation of the indices function.   
Inserting "shape = tuple(shape)" into indices() removed the limitation.

Thanks,
Todd




From magnus at hetland.org  Mon Dec 30 07:45:05 2002
From: magnus at hetland.org (Magnus Lie Hetland)
Date: Mon Dec 30 07:45:05 2002
Subject: [Numpy-discussion] indices with list arg?
In-Reply-To: <3E10593E.8010802@stsci.edu>
References: <20021229215310.GA8757@idi.ntnu.no> <3E10593E.8010802@stsci.edu>
Message-ID: <20021230154359.GC25685@idi.ntnu.no>

Todd Miller <jmiller at stsci.edu>:
>
[snip]
> Neither.   It  looks like a limitation of the indices function.   
> Inserting "shape = tuple(shape)" into indices() removed the limitation.

Ah.

But it's still impossible to use foo.setshape([bar]), then, I guess,
even though it works with reshape()?

> Thanks,
> Todd

-- 
Magnus Lie Hetland
http://hetland.org



From magnus at hetland.org  Mon Dec 30 08:46:01 2002
From: magnus at hetland.org (Magnus Lie Hetland)
Date: Mon Dec 30 08:46:01 2002
Subject: [Numpy-discussion] Compiling from CVS?
Message-ID: <20021230164459.GA7115@idi.ntnu.no>

I guess I'm just missing something obvious here: How do I compile
numarray from the cvs dist? It seems like files such as numconfig.h
etc. are missing -- it seems to have been generated by setup.py in the
normal distributions, but I'm not sure how to do that... I just
expected "python setup.py build" to work :]

-- 
Magnus Lie Hetland
http://hetland.org



From jmiller at stsci.edu  Mon Dec 30 09:01:07 2002
From: jmiller at stsci.edu (Todd Miller)
Date: Mon Dec 30 09:01:07 2002
Subject: [Numpy-discussion] Compiling from CVS?
References: <20021230164459.GA7115@idi.ntnu.no>
Message-ID: <3E107D52.2070502@stsci.edu>

Magnus Lie Hetland wrote:

>I guess I'm just missing something obvious here: How do I compile
>numarray from the cvs dist? It seems like files such as numconfig.h
>
Not really.  People have asked for modifications to setup.py to support 
a two step build/install process.  CVS has the first cut,  which works, 
but is harder to use than it has to be.

>etc. are missing -- it seems to have been generated by setup.py in the
>normal distributions, but I'm not sure how to do that... I just
>expected "python setup.py build" to work :]
>  
>
Try:

python setup.py build --gencode
python setup.py install

for now.   I plan to revisit this later and eliminate the need for the 
switch.

Todd




From magnus at hetland.org  Mon Dec 30 10:02:02 2002
From: magnus at hetland.org (Magnus Lie Hetland)
Date: Mon Dec 30 10:02:02 2002
Subject: [Numpy-discussion] Yet another bug
Message-ID: <20021230180111.GA8587@idi.ntnu.no>

Hi, there... I just found another bug :]

from numarray import arange
from __future__ import division

arange(10) / 10

... This will give an error.

I really think this "real" division should be supported for arrays...

-- 
Magnus Lie Hetland
http://hetland.org



From jmiller at stsci.edu  Mon Dec 30 10:41:15 2002
From: jmiller at stsci.edu (Todd Miller)
Date: Mon Dec 30 10:41:15 2002
Subject: [Numpy-discussion] Yet another bug
References: <20021230180111.GA8587@idi.ntnu.no>
Message-ID: <3E1094E9.3020002@stsci.edu>

Magnus Lie Hetland wrote:

>Hi, there... I just found another bug :]
>
>  
>
numarray doesn't support "true division" yet.

>from numarray import arange
>from __future__ import division
>
>arange(10) / 10
>
>... This will give an error.
>
>I really think this "real" division should be supported for arrays...
>  
>
It will be.  It just hasn't been implemented yet.

>  
>






From magnus at hetland.org  Mon Dec 30 15:58:02 2002
From: magnus at hetland.org (Magnus Lie Hetland)
Date: Mon Dec 30 15:58:02 2002
Subject: [Numpy-discussion] Bin selection
Message-ID: <20021230235736.GA15420@idi.ntnu.no>

I have a set of limits, e.g. array([0, 35, 45, 55, 75]) and I want to
use these to "classify" a set of numbers (another one-dimensional
array). The "class" is the number of the first limit that is lower
than or equal to the number I want to classify. E.g. I'd classify 17
as 0 and 42 as 1.

My current approach is:

  sum(nums[:,NewAxis] >= lims, dim=-1)

It seems a bit unnecessary to compare each number with all the limits
when O(log(n)) would suffice (the limits are ordered); or even with
O(n) running time, a smarter implementation could get an average of
n/2 comparisons...

Suggestions?

-- 
Magnus Lie Hetland
http://hetland.org



From Chris.Barker at noaa.gov  Mon Dec 30 16:15:02 2002
From: Chris.Barker at noaa.gov (Chris Barker)
Date: Mon Dec 30 16:15:02 2002
Subject: [Numpy-discussion] Possible NetCDF bug...
References: <000001c29cc3$281ed420$6501a8c0@NICKLEBY> <Pine.SOL.4.50.0212052304510.28199-100000@death.OCF.Berkeley.EDU> <200212061138.30065.mclay@nist.gov> <3E036FD9.61BC6CA@noaa.gov>
Message-ID: <3E10DD14.9B2550AA@noaa.gov>

Hi all,

Does anyone here use Konrad Hinsen's NetCDF module? I'm having a problem
with it.


I opened an existing NetCDF file and examine one of it's variables:

file = NetCDF.NetCDFFile("test.cdf")

>>> file.variables['water_u'].typecode()
'f'

The variable appears to contain float data (which I think it does).
However, when I try to get a slice of that data:

array([3199453293L, 3198428385L], 'u')

Numeric appears to think it's an unsigned 32 bit integer variable. I get
the same result if I use file.getValue() as well.

Could this be a Numeric version mismatch? I couldn't find a recommended
version of Numeric on the Scientific site.

RedHat Linux 7.2
Python2.2.1
Numeric 22.0

I've enclosed my test.cdf file, if you want to test it yourself.

Anyone have any ideas?

-Chris






-- 
Christopher Barker, Ph.D.
Oceanographer
                                    		
NOAA/OR&R/HAZMAT         (206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115       (206) 526-6317   main reception

Chris.Barker at noaa.gov



From jmiller at stsci.edu  Tue Dec 31 05:25:04 2002
From: jmiller at stsci.edu (Todd Miller)
Date: Tue Dec 31 05:25:04 2002
Subject: [Numpy-discussion] Bin selection
References: <20021230235736.GA15420@idi.ntnu.no>
Message-ID: <3E119E0E.2010403@stsci.edu>

Magnus Lie Hetland wrote:

>I have a set of limits, e.g. array([0, 35, 45, 55, 75]) and I want to
>use these to "classify" a set of numbers (another one-dimensional
>array). The "class" is the number of the first limit that is lower
>than or equal to the number I want to classify. E.g. I'd classify 17
>as 0 and 42 as 1.
>
>My current approach is:
>
>  sum(nums[:,NewAxis] >= lims, dim=-1)
>
>It seems a bit unnecessary to compare each number with all the limits
>when O(log(n)) would suffice (the limits are ordered); or even with
>O(n) running time, a smarter implementation could get an average of
>n/2 comparisons...
>
>Suggestions?
>
>  
>
Try searchsorted().  Searchsorted returns the index of the first bin >= 
the number being classified and has O(log(n)) running time.

 >>> a=numarray.array([0, 35, 45, 55, 75])
 >>> numarray.searchsorted(a, [1,42,35])
array([1, 2, 1])

Todd




From magnus at hetland.org  Tue Dec 31 06:48:01 2002
From: magnus at hetland.org (Magnus Lie Hetland)
Date: Tue Dec 31 06:48:01 2002
Subject: [Numpy-discussion] Bin selection
In-Reply-To: <3E119E0E.2010403@stsci.edu>
References: <20021230235736.GA15420@idi.ntnu.no> <3E119E0E.2010403@stsci.edu>
Message-ID: <20021231144654.GA987@idi.ntnu.no>

Todd Miller <jmiller at stsci.edu>:
[snip]
> Try searchsorted().

Ah! <slaps forehead> Thanks! That's exactly what I was looking for; I
think I've seen it before, but somehow I missed this when looking for
it...

Thanks again,

Magnus

-- 
Magnus Lie Hetland
http://hetland.org



From magnus at hetland.org  Tue Dec 31 07:29:02 2002
From: magnus at hetland.org (Magnus Lie Hetland)
Date: Tue Dec 31 07:29:02 2002
Subject: [Numpy-discussion] argsort on CharArrays?
Message-ID: <20021231152800.GA1929@idi.ntnu.no>

Is there any way to make argsort work on CharArrays?

-- 
Magnus Lie Hetland
http://hetland.org



From jmiller at stsci.edu  Tue Dec 31 07:44:02 2002
From: jmiller at stsci.edu (Todd Miller)
Date: Tue Dec 31 07:44:02 2002
Subject: [Numpy-discussion] argsort on CharArrays?
References: <20021231152800.GA1929@idi.ntnu.no>
Message-ID: <3E11BEB3.2010602@stsci.edu>

Magnus Lie Hetland wrote:

>Is there any way to make argsort work on CharArrays?
>
>  
>
Not yet.

Todd





From magnus at hetland.org  Tue Dec 31 07:49:03 2002
From: magnus at hetland.org (Magnus Lie Hetland)
Date: Tue Dec 31 07:49:03 2002
Subject: [Numpy-discussion] Lists in fromfile shape?
Message-ID: <20021231154816.GA2308@idi.ntnu.no>

I just noticed that the shape argument in fromfile cannot be an
unhashable sequence. I assume tuples are the norm, but still -- it
would be nice to accept other sequences?

And... I also saw an explicit type check in fromfile, comparing
_type(file) with _type("") -- what if I supplied some other
string-like object? Perhaps a try/except with file+"" would be
appropriate, to check for "string-like-ness" instead of an explicit
type check? (I've just grown suspicious of typechecking in general...)

Sorry if I'm being a pest, nagging about all these details, but I
expect it may be useful to get them resolved? :]

-- 
Magnus Lie Hetland
http://hetland.org



From magnus at hetland.org  Tue Dec 31 07:53:02 2002
From: magnus at hetland.org (Magnus Lie Hetland)
Date: Tue Dec 31 07:53:02 2002
Subject: [Numpy-discussion] Lists in fromfile shape?
In-Reply-To: <20021231154816.GA2308@idi.ntnu.no>
References: <20021231154816.GA2308@idi.ntnu.no>
Message-ID: <20021231155203.GA2478@idi.ntnu.no>

Magnus Lie Hetland <magnus at hetland.org>:
>
> I just noticed that the shape argument in fromfile cannot be an
> unhashable sequence. I assume tuples are the norm, but still -- it
> would be nice to accept other sequences?

Sorry about that -- it worked after all. My mistake.

-- 
Magnus Lie Hetland
http://hetland.org



From jmiller at stsci.edu  Tue Dec 31 10:55:01 2002
From: jmiller at stsci.edu (Todd Miller)
Date: Tue Dec 31 10:55:01 2002
Subject: [Numpy-discussion] argsort on CharArrays?
References: <20021231152800.GA1929@idi.ntnu.no> <3E11BEB3.2010602@stsci.edu> <20021231154901.GB2308@idi.ntnu.no>
Message-ID: <3E11EB49.50106@stsci.edu>

Magnus Lie Hetland wrote:

>Todd Miller <jmiller at stsci.edu>:
>  
>
>>Magnus Lie Hetland wrote:
>>
>>    
>>
>>>Is there any way to make argsort work on CharArrays?
>>>
>>>      
>>>
>>Not yet.
>>    
>>
>
>OK, thanks.
>
>(I don't mean to nag -- I'm just eager about numarray :)
>  
>
Don't worry about nagging.    All this feedback is good.  Results, 
however,  may take a little time.

>>Todd
>>    
>>
Todd