From ehagemann at home.com Tue May 1 20:26:26 2001 From: ehagemann at home.com (Eric Hagemann) Date: Tue, 1 May 2001 20:26:26 -0400 Subject: [Numpy-discussion] arg matrix max question Message-ID: <002c01c0d29e$865b9af0$cfefb418@cc911046a> I need to find the location of the maximum value in a two dimensional Numeric array The best I have come up with uses a couple of calls to argmax and then cross checking the results I know there must be a better way and that I am not the only one who needs to do such a thing Anybody got a code snippet to share ? -------------- next part -------------- An HTML attachment was scrubbed... URL: From ehagemann at home.com Thu May 3 20:53:22 2001 From: ehagemann at home.com (Eric Hagemann) Date: Thu, 3 May 2001 20:53:22 -0400 Subject: [Numpy-discussion] curious FFFT timing Message-ID: <006801c0d434$9e7d3ed0$cfefb418@cc911046a> I have an application that needs to perform close 1/2 million FFT's (actually inverse FFTs) I was stymied by the time it was taking so I created a simple benchmark program to measure raw fft speed - hoping to extrapolate and see if the FFT algorithm was the time consuming portion of the algorithm. The result I obtained surprised me Inverse FFT ( 4096) Number of seconds 5.608000 Time per FFT 0.005608 Forward FFT ( 4096) Number of seconds 3.164000 Time per FFT 0.003164 Inverse FFT ( 4095) Number of seconds 8.222000 Time per FFT 0.008222 Forward FFT ( 4095) Number of seconds 5.468000 Time per FFT 0.005468 Inverse FFT ( 8192) Number of seconds 12.578000 Time per FFT 0.012578 Forward FFT ( 8192) Number of seconds 7.551000 Time per FFT 0.007551 Inverse FFTs were taking almost twice as long as forward FFTs ! Bottom line I found that in the inverse code the multiplication of 1/N at the end was the culprit. When I removed the /n from the line return _raw_fft(a, n, axis, fftpack.cffti, fftpack.cfftb, _fft_cache)/n the code timing was more along the line of what was expected. Any one want to speculate on the timing difference for the 4095 vice 4096 long transforms ? Since 4095 is not a power of two I would have expected a greater time difference (DFT vice FFT) N*N / N*log2(N) = N/log2(N) = 4096/12 = 341.33 which is >> than the 1.72 seen above Cheers Eric ------------------------------------ Code Snippet --------------------------------------------- import time from Numeric import * from FFT import * def f_fft(number,length): a = arange(float(length)) start_time = time.time() for i in xrange(number): b = fft(a) durat = time.time() - start_time durat_per = durat / number print "Forward FFT (%10d) Number of seconds %12.6f Time per FFT %12.6f" % (length,durat,durat_per) def i_fft(number,length): a = arange(float(length)) start_time = time.time() for i in xrange(number): b = inverse_fft(a) durat = time.time() - start_time durat_per = durat / number print "Inverse FFT (%10d) Number of seconds %12.6f Time per FFT %12.6f" % (length,durat,durat_per) i_fft(1000,4096) f_fft(1000,4096) i_fft(1000,4095) f_fft(1000,4095) i_fft(1000,8192) f_fft(1000,8192) -------------- next part -------------- An HTML attachment was scrubbed... URL: From ehagemann at home.com Thu May 3 21:11:26 2001 From: ehagemann at home.com (Eric Hagemann) Date: Thu, 3 May 2001 21:11:26 -0400 Subject: [Numpy-discussion] curious FFFT timing References: <006801c0d434$9e7d3ed0$cfefb418@cc911046a> <3ACA7349.421BAD50@cfa.harvard.edu> Message-ID: <007601c0d437$249ef1a0$cfefb418@cc911046a> Scott, Right you are. My brain fried -- I forgot about the other "small prime" stuff Inverse FFT ( 4096) Number of seconds 5.508000 Time per FFT 0.005508 Forward FFT ( 4096) Number of seconds 3.225000 Time per FFT 0.003225 Inverse FFT ( 4097) Number of seconds 29.723000 Time per FFT 0.029723 Forward FFT ( 4097) Number of seconds 26.788000 Time per FFT 0.026788 Cheers Eric ----- Original Message ----- From: "Scott Ransom" To: "Eric Hagemann" Cc: Sent: Tuesday, April 03, 2001 9:05 PM Subject: Re: [Numpy-discussion] curious FFFT timing > Hi Eric, > > > Any one want to speculate on the timing difference for the 4095 vice > > 4096 long transforms ? > > Since 4095 is not a power of two I would have expected a greater time > > difference (DFT vice FFT) > > You are correct in that 4095 is not a power of two, but is is the > product of only small prime factors: > 3 * 3 * 5 * 7 * 13 = 4095 > > Since the FFT code implements a N*log_2(N) algorithm for numbers that > contain only small prime factors, the difference in times is rather > small. FFTs that have lengths that are powers-of-two tend to be more > efficient in general (since the decimation routines are cleaner for this > case). > > If you test with 4097 (17 * 241) it will be quite a bit slower I'd > guess... > > Scott > > > -- > Scott M. Ransom Address: Harvard-Smithsonian CfA > Phone: (617) 495-4142 60 Garden St. MS 10 > email: ransom at cfa.harvard.edu Cambridge, MA 02138 > GPG Fingerprint: 06A9 9553 78BE 16DB 407B FFCA 9BFA B6FF FFD3 2989 > > _______________________________________________ > Numpy-discussion mailing list > Numpy-discussion at lists.sourceforge.net > http://lists.sourceforge.net/lists/listinfo/numpy-discussion > From gvermeul at labs.polycnrs-gre.fr Tue May 1 08:37:00 2001 From: gvermeul at labs.polycnrs-gre.fr (Gerard Vermeulen) Date: Tue, 01 May 2001 14:37:00 +0200 Subject: [Numpy-discussion] 20.0.0b2 - OS X fix breaks building a shared library on Linux Message-ID: <3AEEADEC.8B586A7@labs.polycnrs-gre.fr> Hi, Configuration: Python-2.1 + Numeric-20.0.0b2 on Linux-Mandrake-7.2. I have a Python interface to Numeric and the Qwt-C++ plotting library. When building this interface (a shared library) for Numeric-20.0.0b2, I noticed that the additional static in front of the importing API pointer in arrayobject.h (your OS X fix) leads to an unresolved symbol PyArray_API when building a shared library. My fix is to remove the keyword static from arrayobject.h. Has somebody else encountered this problem? I suppose that this problem should show up with any shared library using import_array() (version 20.0.0b2) on Linux. If the additional static really solves the OS X problems, maybe an #ifdef OSX ... #else ... #endif is necessary. Best regards -- Gerard From paul at pfdubois.com Wed May 2 09:56:42 2001 From: paul at pfdubois.com (Paul F. Dubois) Date: Wed, 2 May 2001 06:56:42 -0700 Subject: [Numpy-discussion] static pointer Message-ID: Mr. Verveer wrote: "I recently installed this version and it unfortunately broke my own extension C code. I found that my code did not link properly anymore because of the addition of the static declaration tot the API pointer (as noted in changes.txt). Since my extensions consist of several source files, this led to problems when importing my extensions. If I understand correctly, the addition of the static keyword means that the import_array() function can now only be used by extensions that are defined in a single C source file. I can easily work around this problem, by defining my own api pointer, and using a modified import_array() in my code, but I thought I should draw your attention to this problem in any case. Maybe another solution can be found that does not need this pointer to be declared static?" -- I tried this change to see whether it had any bad consequences. It did not appear to have bad consequences in that Numpy itself and all its packages built ok, and we were formerly not following the instructions in the Python documentation and now we are. And, OS-X now worked. So far so good. If I understand you correctly, this does not work properly if the extension is built in pieces. I think I see why. (Frankly, linking and dynamic loading are not something I know much about; as a user I just flounder around.) This all started because of the need to have access to the C API of one dynamic extension from another. Is there a way of institutionalizing your work-around so that others can use it? Frankly, I'm out of ideas. The community is going to have to rescue me on this one. Paul P.S. Wait. I have an idea. Konrad was the one who put all this in. He should be punished for his good deed...(:-> From hinsen at cnrs-orleans.fr Wed May 2 13:39:11 2001 From: hinsen at cnrs-orleans.fr (Konrad Hinsen) Date: Wed, 2 May 2001 19:39:11 +0200 Subject: [Numpy-discussion] static pointer In-Reply-To: References: Message-ID: <200105021739.TAA15221@chinon.cnrs-orleans.fr> > If I understand correctly, the addition of the static keyword means that the > import_array() function can now only be used by extensions that are defined > in a single C source file. I can easily work around this problem, by True. That's exactly what I pointed out in a previous message in this list. > defining my own api pointer, and using a modified import_array() in my code, That seems to me the preferred solution. > built ok, and we were formerly not following the instructions in the Python > documentation and now we are. And, OS-X now worked. So far so good. If I And those instructions mention the static keyword for a good reason: without it there are problems not only with portability, but also with static linking of extension modules. > Is there a way of institutionalizing your work-around so that others can use > it? I thought about providing a general solution right in the header files. The problem is the generation of a unique symbol by preprocessor tricks, perhaps somehow based on the module name. I don't see how this can be done. On the other hand, if we accept that the extension module programmer has to supply a unique symbol, then it is even easy. This would only be necessary for multi-file extension modules. Does that seem like a good solution? > Wait. I have an idea. Konrad was the one who put all this in. He should be > punished for his good deed...(:-> I am even thinking of punishing myself by imposing extra work on me! ;-) Konrad. -- ------------------------------------------------------------------------------- Konrad Hinsen | E-Mail: hinsen at cnrs-orleans.fr Centre de Biophysique Moleculaire (CNRS) | Tel.: +33-2.38.25.56.24 Rue Charles Sadron | Fax: +33-2.38.63.15.17 45071 Orleans Cedex 2 | Deutsch/Esperanto/English/ France | Nederlands/Francais ------------------------------------------------------------------------------- From dubois1 at llnl.gov Wed May 2 18:06:40 2001 From: dubois1 at llnl.gov (Paul F. Dubois) Date: Wed, 2 May 2001 15:06:40 -0700 Subject: [Numpy-discussion] MA pickle Message-ID: <01050215114800.31638@almanac> I have a version of MA that pickles. I am so frightened of it that I don't even want to chuck it into CVS yet . I just wanted those who expressed concern about this to know there is hope. If someone is desperate please email me and I'll let you try it privately. From Peter.Verveer at embl-heidelberg.de Thu May 3 04:32:44 2001 From: Peter.Verveer at embl-heidelberg.de (Peter Verveer) Date: Thu, 3 May 2001 10:32:44 +0200 Subject: [Numpy-discussion] static pointer In-Reply-To: <200105021739.TAA15221@chinon.cnrs-orleans.fr> References: <200105021739.TAA15221@chinon.cnrs-orleans.fr> Message-ID: <01050310324400.06846@marvin.embl-heidelberg.de> On Wednesday 02 May 2001 19:39, Konrad Hinsen wrote: > > If I understand correctly, the addition of the static keyword means that > > the import_array() function can now only be used by extensions that are > > defined in a single C source file. I can easily work around this problem, > > by > > True. That's exactly what I pointed out in a previous message in this > list. > > > defining my own api pointer, and using a modified import_array() in my > > code, > > That seems to me the preferred solution. But this will still fail on OSX, I guess. > > built ok, and we were formerly not following the instructions in the > > Python documentation and now we are. And, OS-X now worked. So far so > > good. If I > > And those instructions mention the static keyword for a good reason: > without it there are problems not only with portability, but also with > static linking of extension modules. > > > Is there a way of institutionalizing your work-around so that others can > > use it? > > I thought about providing a general solution right in the header > files. The problem is the generation of a unique symbol by > preprocessor tricks, perhaps somehow based on the module name. > I don't see how this can be done. > > On the other hand, if we accept that the extension module programmer > has to supply a unique symbol, then it is even easy. This would only > be necessary for multi-file extension modules. Does that seem like a > good solution? That sounds very good. I don't mind providing a symbol name for the pointer myself. > > Wait. I have an idea. Konrad was the one who put all this in. He should > > be punished for his good deed...(:-> > > I am even thinking of punishing myself by imposing extra work on me! ;-) > > Konrad. -- Dr. Peter J. Verveer Bastiaens Group Cell Biology and Cell Biophysics Programme EMBL Meyerhofstrasse 1 D-69117 Heidelberg Germany Tel. : +49 6221 387245 Fax : +49 6221 387242 Email: Peter.Verveer at embl-heidelberg.de From hinsen at cnrs-orleans.fr Thu May 3 04:57:14 2001 From: hinsen at cnrs-orleans.fr (Konrad Hinsen) Date: Thu, 3 May 2001 10:57:14 +0200 Subject: [Numpy-discussion] static pointer In-Reply-To: <01050310324400.06846@marvin.embl-heidelberg.de> (message from Peter Verveer on Thu, 3 May 2001 10:32:44 +0200) References: <200105021739.TAA15221@chinon.cnrs-orleans.fr> <01050310324400.06846@marvin.embl-heidelberg.de> Message-ID: <200105030857.KAA15830@chinon.cnrs-orleans.fr> > > That seems to me the preferred solution. > > But this will still fail on OSX, I guess. Not necessarily. The API pointer just has to be passed between the source code modules in a global variable whose name is highly unlikely to be used somewhere else. > That sounds very good. I don't mind providing a symbol name for the pointer > myself. Good to hear. Is there anyone who would oppose this? Konrad. -- ------------------------------------------------------------------------------- Konrad Hinsen | E-Mail: hinsen at cnrs-orleans.fr Centre de Biophysique Moleculaire (CNRS) | Tel.: +33-2.38.25.56.24 Rue Charles Sadron | Fax: +33-2.38.63.15.17 45071 Orleans Cedex 2 | Deutsch/Esperanto/English/ France | Nederlands/Francais ------------------------------------------------------------------------------- From chrishbarker at home.net Thu May 3 17:47:02 2001 From: chrishbarker at home.net (Chris Barker) Date: Thu, 03 May 2001 14:47:02 -0700 Subject: [Numpy-discussion] Just what is an "Int32" ? References: Message-ID: <3AF1D1D6.A9B99A4E@home.net> Hi all, I'm writing an extension that uses a NumPy array (among other things). To make sure that I have passed in what I'm looking for, I do a: if (Array->descr->type_num != PyArray_LONG) ... PyErr_SetString (PyExc_ValueError,"Array is the wrong type"); return NULL; } (I want Array to be an array of longs) When I call my extension with an array built using array([...],Int32), this error results (this is on both Windows and Linux on Intel hardware). If I build the array with "Int" it works fine. This isn't really a problem, but I am curious, what is an Int32 on Intel hardware?, and why wouldn't it be a long? -Chris -- Christopher Barker, Ph.D. ChrisHBarker at home.net --- --- --- http://members.home.net/barkerlohmann ---@@ -----@@ -----@@ ------@@@ ------@@@ ------@@@ Oil Spill Modeling ------ @ ------ @ ------ @ Water Resources Engineering ------- --------- -------- Coastal and Fluvial Hydrodynamics -------------------------------------- ------------------------------------------------------------------------ From chrishbarker at home.net Fri May 4 14:52:01 2001 From: chrishbarker at home.net (Chris Barker) Date: Fri, 04 May 2001 11:52:01 -0700 Subject: [Numpy-discussion] Can this be done efficiently with array functions? References: <008601c03575$5036fa80$52f9a218@nc.rr.com> <20001014183233.28900.qmail@lisboa.ifm.uni-kiel.de> Message-ID: <3AF2FA51.9B0AC68F@home.net> I have a problem that I can't figure otu a non-kludgey way to solve: Given a (MXN) array, a, I want to compute the minimum value in each row, excluding the values that are zero. example: for this array, a: [[10, 0, 5, 0,] [ 0, 0, 0, 0,] [ 0, 5,15, 0,] [ 0, 0, 0, 0,] [ 0, 3, 1,25,] [ 0, 0, 0, 0,] [ 4, 7, 2,12,] [ 0, 0, 0, 0,]] I want: [[5,] [0,] [5,] [0,] [1,] [0,] [2,] [0,]] what I want is minimum.reduce(a,1), except that the zeros would be ignored. At first glace, Masked arrays seemed the perfect option (as that's what the zero means, no real data), but there doesn't seem to be a minimum.reduce() in MA. By the way, what is the downside to Masked Arrays? they seem to be a really powerful option. Is there any performance hit to using a masked array over a regular onw if there is no mask? If not, would it make sense to move toward just using Masked arrays for everything? thanks, -Chris -- Christopher Barker, Ph.D. ChrisHBarker at home.net --- --- --- http://members.home.net/barkerlohmann ---@@ -----@@ -----@@ ------@@@ ------@@@ ------@@@ Oil Spill Modeling ------ @ ------ @ ------ @ Water Resources Engineering ------- --------- -------- Coastal and Fluvial Hydrodynamics -------------------------------------- ------------------------------------------------------------------------ From paul at pfdubois.com Fri May 4 15:07:44 2001 From: paul at pfdubois.com (Paul F. Dubois) Date: Fri, 4 May 2001 12:07:44 -0700 Subject: [Numpy-discussion] Can this be done efficiently with array functions? In-Reply-To: <3AF2FA51.9B0AC68F@home.net> Message-ID: -----Original Message----- ...snip what I want is minimum.reduce(a,1), except that the zeros would be ignored. At first glace, Masked arrays seemed the perfect option (as that's what the zero means, no real data), but there doesn't seem to be a minimum.reduce() in MA. --> I'll add this to my list. Meanwhile you could do something like this, where x is your array: y = MA.masked_equal(x,0,copy=0) Numeric.minimum.reduce(y.filled(100000), 1) If x is floating point use masked_value rather than masked_equal. --- By the way, what is the downside to Masked Arrays? they seem to be a really powerful option. Is there any performance hit to using a masked array over a regular onw if there is no mask? If not, would it make sense to move toward just using Masked arrays for everything? thanks, -Chris --> There are two problems. There is a hit, even in the case of no mask. It is hard to quantify because it depends on what operations you do. Much of that could be eliminated if MA was in C but it is Python. It would have driven me mad to write it in C, however. More seriously, there are a few semantic issues where it isn't clear what you mean if a mask is present. From chrishbarker at home.net Fri May 4 17:42:34 2001 From: chrishbarker at home.net (Chris Barker) Date: Fri, 04 May 2001 14:42:34 -0700 Subject: [Numpy-discussion] Can this be done efficiently with array functions? References: <008601c03575$5036fa80$52f9a218@nc.rr.com> <20001014183233.28900.qmail@lisboa.ifm.uni-kiel.de> <3AF2FA51.9B0AC68F@home.net> <20010504193541.29870.qmail@lisboa.ifm.uni-kiel.de> Message-ID: <3AF3224A.ED902632@home.net> Janko and Paul, Thanks for your suggestions. I had come up with something like Janko's solution, but not as cleanly. It seems that Paul's suggestion is essentially the same thing. Maybe I'll do some tests to see which is faster. By the way, I tried Janko's code, and got an error on "a == 0" (I'm running Python 2.0). I looked at it again, and then looked at the release notes for Python 2.1, and saw that rich comparisons are working..Glory be! I'll be upgrading real soon, just for that! thanks again, -Chris PS: anyone have a response for my Int32 question? Why isn't it a "long" on Linux or windows on Intel? Janko Hauser wrote: > truely_max=100 > > a=array([[10, 0, 5, 0,], > [ 0, 0, 0, 0,], > [ 0, 5,15, 0,], > [ 0, 0, 0, 0,], > [ 0, 3, 1,25,], > [ 0, 0, 0, 0,], > [ 4, 7, 2,12,], > [ 0, 0, 0, 0,]]) > > am=minimum.reduce(where(a==0,truely_max,a),1) > am=where(am==truely_max,0,am) > print am > [5 0 5 0 1 0 2 0] > "Paul F. Dubois" wrote: > but there doesn't seem to be > a minimum.reduce() in MA. > > --> I'll add this to my list. That will be great if you get a chance. > Meanwhile you could do something like this, > where x is your array: > y = MA.masked_equal(x,0,copy=0) > Numeric.minimum.reduce(y.filled(100000), 1) > If x is floating point use masked_value rather than masked_equal. > --- > --> There are two problems. There is a hit, even in the case of no mask. > It is hard to quantify because it depends on what operations you do. > Much of that could be eliminated if MA was in C but it is Python. > It would have driven me mad to write it in C, however. WOW, I'm impressed, I had assumed that some C was involved. Nice job. > More seriously, there are a few semantic issues where it isn't clear > what you mean if a mask is present. I'm sure these could be resolved. It would be nice if we could have in mind to move in that direction. I'm not going ot be able to do any of the coding, so that's the last you'll here from me about it. -Chris -- Christopher Barker, Ph.D. ChrisHBarker at home.net --- --- --- http://members.home.net/barkerlohmann ---@@ -----@@ -----@@ ------@@@ ------@@@ ------@@@ Oil Spill Modeling ------ @ ------ @ ------ @ Water Resources Engineering ------- --------- -------- Coastal and Fluvial Hydrodynamics -------------------------------------- ------------------------------------------------------------------------ From Karl.Bellve at umassmed.edu Wed May 9 12:06:39 2001 From: Karl.Bellve at umassmed.edu (Karl Bellve) Date: Wed, 09 May 2001 12:06:39 -0400 Subject: [Numpy-discussion] UNREF Message-ID: <3AF96B0F.642F365D@umassmed.edu> Occasionally, if I reinitialize Python and rerun a script, I get the following error: FATAL Python error: UNREF invalid object from my code I do the following: /* alot of other crap and then */ PyRun_SimpleString("From Numeric import *\n"); eventually in multiarraymodule.c array_set_string_function() Py_INCREF(Py_None) which eventually calls (according to my stack trace) where the error occurs PyArray_SetStringFunction() Py_XDECREF(PyArray_ReprFunction) Is this a Numpy problem, Python problem, or my problem? I can run the same python script several times, and then I get the error. Something isn't being cleaned up properly around Py_Initialize() Py_Finalize() pair. I am using Python 2.1b1 and Numeric 18_1 with Boost -- Cheers, Karl Bellve From hinsen at cnrs-orleans.fr Wed May 9 12:12:25 2001 From: hinsen at cnrs-orleans.fr (Konrad Hinsen) Date: Wed, 9 May 2001 18:12:25 +0200 Subject: [Numpy-discussion] Modified header files Message-ID: <200105091612.SAA24523@chinon.cnrs-orleans.fr> Recently we had a discussion about how to use NumPy arrays from extension modules with multiple source files, on various platforms. The modified header files that are attached to this message provide a (hopefully!) general solution. In fact, I propose to make them part of the official distribution, unless there are objections. If used like before, these header files give exactly the same result as the ones in NumPy 20.0.0. However, they permit to define the name of the C API pointer array and make it globally visible. Under the condition that the chosen name is unique, this should not create problems under any platform, no matter if static or dynamic linking is used. To use NumPy features in multiple source file extension modules, you have to write #define PY_ARRAY_UNIQUE_SYMBOL PyArrayXXX #include "Numeric/arrayobject.h" in the main source file (the one that contains the init function) and #define PY_ARRAY_UNIQUE_SYMBOL PyArrayXXX #define NO_IMPORT_ARRAY #include "Numeric/arrayobject.h" in all the others. The symbol you choose instead of PyArrayXXX should contain both the name of the imported module (array) and the name of the importing module (whatever your module is called) in order to be unique with a reasonably high probability. The same applies to the Ufunc module, just replace "array" by "ufunc" in the example. I have also applied the "static" correction to the Ufunc header file, there is no reason not to do it. Konrad. -- ------------------------------------------------------------------------------- Konrad Hinsen | E-Mail: hinsen at cnrs-orleans.fr Centre de Biophysique Moleculaire (CNRS) | Tel.: +33-2.38.25.56.24 Rue Charles Sadron | Fax: +33-2.38.63.15.17 45071 Orleans Cedex 2 | Deutsch/Esperanto/English/ France | Nederlands/Francais ------------------------------------------------------------------------------- -------------- next part -------------- A non-text attachment was scrubbed... Name: arrayobject.h Type: application/octet-stream Size: 13881 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: ufuncobject.h Type: application/octet-stream Size: 6701 bytes Desc: not available URL: From Karl.Bellve at umassmed.edu Wed May 9 12:22:11 2001 From: Karl.Bellve at umassmed.edu (Karl Bellve) Date: Wed, 09 May 2001 12:22:11 -0400 Subject: [Numpy-discussion] UNREF References: <3AF96B0F.642F365D@umassmed.edu> Message-ID: <3AF96EB3.93A2B06@umassmed.edu> More info: I can reproduce this error with the following loop: for (int x = 0; x < 100; x++) { Py_Initialize(); import_array(); PyRun_SimpleString("from Numeric import *\n"); TRACE("%d\n",x); Py_Finalize(); } Here is the output: 0 1 Fatal Python error: UNREF invalid object -- Cheers, Karl Bellve From mhagger at alum.mit.edu Wed May 9 19:20:37 2001 From: mhagger at alum.mit.edu (Michael Haggerty) Date: 09 May 2001 19:20:37 -0400 Subject: [Numpy-discussion] Thoughts on masked arrays Message-ID: Hi, I've spent several days using the masked arrays that have been added to NumPy recently. They're a great feature and they were just what I needed for the little project I was working on (aside from a few bugs that I found). However, there were a few things about MA that I found inconvenient and/or counterintuitive, so I thought I'd post them to the list while they're fresh in my mind. I'm using Numeric-20.0.0b2. 1. I couldn't find a simple way to tell if all of the cells of a masked array are unmasked. There are times when you fill an array incrementally and you want to convert it to a Numeric array but first make sure that all of the elements have been set. "m.filled()" is a bit dangerous (in my opinion) because it silently fills. The shortest idiom I could think of is >>> assert not logical_or.reduce(ravel(MA.getmaskarray(m))) which isn't very short :-) and is also awkward because it creates a mask array even if m.mask() is None. How about a m.is_unmasked() method, or even giving a special meaning to "m.filled(masked)", namely that it raises an exception if any cells are still masked. (As an optimization, this method could set m.__mask = None to speed up future checks.) 2. I can't reproduce this problem now, but I could swear that the MaskedArray.__str__() method sometimes printed "typecode='O'" if masked.enabled() is true. This would be a byproduct of using Numeric's __str__() method to print the array, at least under the unknown circumstances in which Numeric.__str__() prints the typecode. This confused me for a while. 3. I found the semantics of MA.compress(condition,a,axis=0) to be inconvenient and inconsistent with those of Numeric.compress. MA.compress() squeezes out not only those elements for which condition is false, but also those elements that are masked. This differs from the behavior of Numeric.compress, which always returns an array with the "axis" dimension equal to the number of nonzero elements of "condition". The real problem, though, is that MA.compress can't be used on a multidimensional array with a nontrivial mask, because squeezing out the masked values is highly unlikely to result in a rectangular matrix. It is nice to be able to squeeze masked values out of a 1-d array, but not at the price of not being able to use compress on a multidimensional array. I suggest giving MA.compress() semantics closer to those of Numeric.compress(), and adding an optional argument or a separate method to cause masked elements to be omitted. Thanks for a great package! Yours, Michael -- Michael Haggerty mhagger at alum.mit.edu From paul at pfdubois.com Wed May 9 22:56:16 2001 From: paul at pfdubois.com (Paul F. Dubois) Date: Wed, 9 May 2001 19:56:16 -0700 Subject: [Numpy-discussion] Thoughts on masked arrays In-Reply-To: Message-ID: -----Original Message----- From: numpy-discussion-admin at lists.sourceforge.net [mailto:numpy-discussion-admin at lists.sourceforge.net]On Behalf Of Michael Haggerty wrote 1. I couldn't find a simple way to tell if all of the cells of a masked array are unmasked. There are times when you fill an array incrementally and you want to convert it to a Numeric array but first make sure that all of the elements have been set. "m.filled()" is a bit dangerous (in my opinion) because it silently fills. The shortest idiom I could think of is >>> assert not logical_or.reduce(ravel(MA.getmaskarray(m))) which isn't very short :-) and is also awkward because it creates a mask array even if m.mask() is None. How about a m.is_unmasked() method, or even giving a special meaning to "m.filled(masked)", namely that it raises an exception if any cells are still masked. (As an optimization, this method could set m.__mask = None to speed up future checks.) ====== >>> from MA import * >>> x=array([[1,2],[3,4]],mask=[[0,0],[0,0]]) >>> count(x) 4 >>> product(x.shape) 4 >>> So your test could be if count(x) < product(x.shape): error... make_mask(m, flag=1) will make a mask and have it be None if possible. It also accepts an argument of None correctly. So your test could be if make_mask(m.mask(),flag=1) is not None: error... You could also consider if not Numeric.allclose(m.filled(0), m.filled(1)) or m.mask() is not None and not Numeric.alltrue(Numeric.ravel(m.mask())): Is that enough ways to do it? (TM) (:-> I don't recommend using assert if the test is data-driven, since it won't get executed with python -O. Instead use if...: raise .... I'm not against is_unmasked but I'm not sure how much it would get used and I don't like the name. I hate query methods with side effects (if you use them in an assert you change the program). A method that replaces the mask with None if possible might make sense. m.unmask()? m.demask()? m.debride() ? ============= 2. I can't reproduce this problem now, but I could swear that the MaskedArray.__str__() method sometimes printed "typecode='O'" if masked.enabled() is true. This would be a byproduct of using Numeric's __str__() method to print the array, at least under the unknown circumstances in which Numeric.__str__() prints the typecode. This confused me for a while. ========= Short of writing my own print routine, I basically have to create something filled with '--', which is of course of type object. That's why you can disable it. (:-> ========= 3. I found the semantics of MA.compress(condition,a,axis=0) to be inconvenient and inconsistent with those of Numeric.compress. MA.compress() squeezes out not only those elements for which condition is false, but also those elements that are masked. This differs from the behavior of Numeric.compress, which always returns an array with the "axis" dimension equal to the number of nonzero elements of "condition". The real problem, though, is that MA.compress can't be used on a multidimensional array with a nontrivial mask, because squeezing out the masked values is highly unlikely to result in a rectangular matrix. It is nice to be able to squeeze masked values out of a 1-d array, but not at the price of not being able to use compress on a multidimensional array. I suggest giving MA.compress() semantics closer to those of Numeric.compress(), and adding an optional argument or a separate method to cause masked elements to be omitted. ====== It has been an interesting project in that there are hundreds of these individual little design questions. Can you propose the semantics you would like in a precise way? Include the case where the condition has masked values. ====== Thanks for a great package! Yours, Michael === I appreciate the encouragement. -- Paul From paul at pfdubois.com Wed May 9 23:12:01 2001 From: paul at pfdubois.com (Paul F. Dubois) Date: Wed, 9 May 2001 20:12:01 -0700 Subject: [Numpy-discussion] Modified header files In-Reply-To: <200105091612.SAA24523@chinon.cnrs-orleans.fr> Message-ID: I have added these files to CVS. Thanks! --Paul -----Original Message----- From: numpy-discussion-admin at lists.sourceforge.net [mailto:numpy-discussion-admin at lists.sourceforge.net]On Behalf Of Konrad Hinsen Sent: Wednesday, May 09, 2001 9:12 AM To: numpy-discussion at lists.sourceforge.net Subject: [Numpy-discussion] Modified header files Recently we had a discussion about how to use NumPy arrays from extension modules with multiple source files, on various platforms. The modified header files that are attached to this message provide a (hopefully!) general solution. In fact, I propose to make them part of the official distribution, unless there are objections. If used like before, these header files give exactly the same result as the ones in NumPy 20.0.0. However, they permit to define the name of the C API pointer array and make it globally visible. Under the condition that the chosen name is unique, this should not create problems under any platform, no matter if static or dynamic linking is used. To use NumPy features in multiple source file extension modules, you have to write #define PY_ARRAY_UNIQUE_SYMBOL PyArrayXXX #include "Numeric/arrayobject.h" in the main source file (the one that contains the init function) and #define PY_ARRAY_UNIQUE_SYMBOL PyArrayXXX #define NO_IMPORT_ARRAY #include "Numeric/arrayobject.h" in all the others. The symbol you choose instead of PyArrayXXX should contain both the name of the imported module (array) and the name of the importing module (whatever your module is called) in order to be unique with a reasonably high probability. The same applies to the Ufunc module, just replace "array" by "ufunc" in the example. I have also applied the "static" correction to the Ufunc header file, there is no reason not to do it. Konrad. -- ---------------------------------------------------------------------------- --- Konrad Hinsen | E-Mail: hinsen at cnrs-orleans.fr Centre de Biophysique Moleculaire (CNRS) | Tel.: +33-2.38.25.56.24 Rue Charles Sadron | Fax: +33-2.38.63.15.17 45071 Orleans Cedex 2 | Deutsch/Esperanto/English/ France | Nederlands/Francais ---------------------------------------------------------------------------- --- From Karl.Bellve at umassmed.edu Thu May 10 08:57:24 2001 From: Karl.Bellve at umassmed.edu (Karl Bellve) Date: Thu, 10 May 2001 08:57:24 -0400 Subject: [Numpy-discussion] Numpy bug? Message-ID: <3AFA9034.F581DCA9@umassmed.edu> Since I didn't get a response, I will post again with a more appropiate subject heading. If you run the following code, you get an error. It appears that Numpy doesn't like to be restarted. If you take out the lines "import_array();" and "PyRun_SimpleString("from Numeric import *\n");" the code loops fine. for (int x = 0; x < 100; x++) { Py_Initialize(); import_array(); PyRun_SimpleString("from Numeric import *\n"); TRACE("%d\n",x); Py_Finalize(); } Here is the output: 0 1 Fatal Python error: UNREF invalid object -- Cheers, Karl Bellve From paul at pfdubois.com Thu May 10 13:19:20 2001 From: paul at pfdubois.com (Paul F. Dubois) Date: Thu, 10 May 2001 10:19:20 -0700 Subject: [Numpy-discussion] Numpy bug? In-Reply-To: <3AFA9034.F581DCA9@umassmed.edu> Message-ID: The import_array is not proper, but I have no idea if that is the problem or not. import array only goes in the initialization routine of modules that need the Numpy C-API. Somebody convinced me to decref something in one of the numpy C module initializers a while back, so maybe that was an error. (?) I just do what they tell me... -----Original Message----- From: numpy-discussion-admin at lists.sourceforge.net [mailto:numpy-discussion-admin at lists.sourceforge.net]On Behalf Of Karl Bellve Sent: Thursday, May 10, 2001 5:57 AM To: Numpy Subject: [Numpy-discussion] Numpy bug? Since I didn't get a response, I will post again with a more appropiate subject heading. If you run the following code, you get an error. It appears that Numpy doesn't like to be restarted. If you take out the lines "import_array();" and "PyRun_SimpleString("from Numeric import *\n");" the code loops fine. for (int x = 0; x < 100; x++) { Py_Initialize(); import_array(); PyRun_SimpleString("from Numeric import *\n"); TRACE("%d\n",x); Py_Finalize(); } Here is the output: 0 1 Fatal Python error: UNREF invalid object -- Cheers, Karl Bellve _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net http://lists.sourceforge.net/lists/listinfo/numpy-discussion From paul at pfdubois.com Thu May 10 15:20:52 2001 From: paul at pfdubois.com (Paul F. Dubois) Date: Thu, 10 May 2001 12:20:52 -0700 Subject: [Numpy-discussion] 20.1.0a3 -- for those who like pickles Message-ID: I put a source tarball up for 20.1.0a3, also in CVS as of now. This contains: DL_EXPORT changes Konrad's fix to import_array A new MA that pickles A new package "Properties", that replaces MA.activeattr. Properties allows you to have properties such as MA/Numeric's shape attribute, that look like attributes but actually invoke functions on get, set, and/or delete. It can also be used to make attributes unwriteable or undeleteable. It doesn't have anything to do with Numeric but I use it in MA and kinds and don't have much else of an idea where to put it. From mhagger at alum.mit.edu Thu May 10 18:11:37 2001 From: mhagger at alum.mit.edu (Michael Haggerty) Date: 10 May 2001 18:11:37 -0400 Subject: [Numpy-discussion] Thoughts on masked arrays In-Reply-To: "Paul F. Dubois"'s message of "Wed, 9 May 2001 19:56:16 -0700" References: Message-ID: "Paul F. Dubois" writes: > -----Original Message----- > Michael Haggerty wrote > 1. I couldn't find a simple way to tell if all of the cells of a > masked array are unmasked. > ====== > So your test could be if count(x) < product(x.shape): error... > > So your test could be > if make_mask(m.mask(),flag=1) is not None: > error... > > You could also consider if not Numeric.allclose(m.filled(0), m.filled(1)) > or > m.mask() is not None and not Numeric.alltrue(Numeric.ravel(m.mask())): Shouldn't that be m.mask() is not None and Numeric.sometrue(Numeric.ravel(m.mask())) ? ("Proof" that these expressions are nonintuitive.) > Is that enough ways to do it? (TM) (:-> Frankly, it's too many ways to do it, none of them obvious to the writer or the reader. This is a simple and useful concept and it should have one obvious implementation. > I'm not against is_unmasked but I'm not sure how much it would get > used and I don't like the name. I hate query methods with side > effects (if you use them in an assert you change the program). In this case the side effect is to change the internal representation of the object without changing its semantics, so I don't find it too objectionable. But omit this optimization if you prefer; the query method would be just as useful even without the side effect. Because of the relationship with filled(), maybe this query function should be called m.isfull(). There should probably also be an isfull(m) function for the same reason that there is a mask(m) function. > A method that replaces the mask with None if possible might make > sense. m.unmask()? m.demask()? m.debride() ? Of these names, I like m.unmask() the best. I assume that it would set m.__mask=None if possible and throw an exception if not. On the other hand, while it would be desirable to have a function equivalent (i.e., unmask(m)), this would be awkward because a function should usually not change its argument. Therefore, I suggest adding a safe analogue of raw_data() that throws an exception if the array has a nontrivial mask and otherwise returns self.__data. E.g. [untested]: class MaskedArray: [...] def data(self): """If no values are masked, return self.__data(). Otherwise raise an exception. """ d = self.__data m = self.__mask if m is not None and Numeric.sometrue(Numeric.ravel(m)): raise MAError, "MaskedArray cannot be converted to array" elif d.iscontiguous(): return d else: return Numeric.array(d, typecode=d.typecode(), copy=1, savespace = d.spacesaver()) def data(a): if isinstance(a, MaskedArray): return m.data() elif isinstance(a, Numeric.ArrayType) and a.iscontiguous(): return a else: return Numeric.array(a) A more obscure name should be chosen since you seem to encourage "from MA import *". > 3. I found the semantics of MA.compress(condition,a,axis=0) to be > inconvenient and inconsistent with those of Numeric.compress. > ====== > It has been an interesting project in that there are hundreds of these > individual little design questions. > Can you propose the semantics you would like in a precise way? Include the > case where the condition has masked values. > ====== In the simple case where the condition has no masked values, I think compress() should simply pick slices out according to condition, without regard to which cells of x are masked. When condition is masked, I don't think that there is a sensible interpretation for compress() because a "masked" value in condition means you don't know whether that slice of x should be included or not. Since you can't have an output array of indeterminate shape, I would throw an exception if condition is masked. Here is my attempt [untested]: def compress(condition, x, dimension=-1): # data function is defined above (throws exception if condition is masked): c = data(condition) if mask(x) is None: mask = None else: mask=Numeric.compress(condition, mask(x), dimension) return array(Numeric.compress(condition, filled(x), dimension), mask=mask) Yours, Michael -- Michael Haggerty mhagger at alum.mit.edu From paul at pfdubois.com Fri May 11 12:02:54 2001 From: paul at pfdubois.com (Paul F. Dubois) Date: Fri, 11 May 2001 09:02:54 -0700 Subject: [Numpy-discussion] Thoughts on masked arrays In-Reply-To: Message-ID: Thank you for provoking me to think about these issues in MA. Here is the conclusion I have reached. Please let me know what you think of it. Background: Michael wanted a way to use a masked array as a Numeric array but with assurance that in fact no element was masked, without obscure tests such as count(x) == product(x.shape). The method __array__(self, typecode=None) is a special (existing) hook for conversion to a Numeric array. Many operations in Numeric, when presented with an object x to be operated upon, such as Numeric.sqrt(x), will call x.__array__ as a final act of desperation in an attempt to convert their argument to a Numeric array. Heretofore it was essentially returning x.filled(). This bothered me, because it was a silent conversion that replaced masked values with the fill value. Solution: a. Add a method 'unmask()' which will replace the mask by None if possible. It will not fail. b. Change MaskedArray.__array__ to work as follows: a. self.unmask(), and then b. Return the raw data if the mask is now None. Otherwise, throw an MAError. Example usage: >>> from MA import * >>> x=arange(10) >>> Numeric.array(x) [0,1,2,3,4,5,6,7,8,9,] >>> x[3]=masked >>> Numeric.array(x) Traceback (most recent call last): File "", line 1, in ? File "/pcmdi/dubois/linux/lib/python2.1/site-packages/MA/MA.py", line 578, in __array__ raise MAError, \ MA.MA.MAError: Cannot convert masked array to Numeric because data is masked in one or more locations. Merits of this solution: a. It reads like what it is -- there is no doubt you are converting to a Numeric array when you see Numeric.array. b. It gives you the full range of options in Numeric.array, such as messing with the typecode. c. It allows Numeric operations for speed on masked arrays that you know to be masked in name only. No copy of data occurs here unless the typecode needs to be changed. d. It removes the possibility of a 'dishonest' conversion. e. No new method or function is required, other than the otherwise-useful unmask(). f. Successive conversions are optimized because once the mask is None, unmask is cheap. Deficiency: __array__ becomes a query with an internal, albeit safe, side-effect. Mitigating this is that __array__ is not a "public" method and would not normally be used in assertions. From Karl.Bellve at umassmed.edu Fri May 11 12:12:18 2001 From: Karl.Bellve at umassmed.edu (Karl Bellve) Date: Fri, 11 May 2001 12:12:18 -0400 Subject: [Numpy-discussion] Numpy bug? References: Message-ID: <3AFC0F62.25968CC8@umassmed.edu> How can we correct this error? Otherwise, Numpy can only be initialized once per application/thread? Here is what I do, a user loads an image into my application, then runs a Python script. I do a Py_Initialize(), initialize Numpy, initialize my module, run the script and finally Py_Finalize(). Each image can do this. My module only knows about its own image and can run a python script multiple times. So, if I have to make it more global, I would have to run Py_Intialize/Py_Finalize once per application. Redesign my module so that a user must pick the specific image to work on. Redesign the scripts so that a user must transverse all the images to the correct image. Hey, I would rather fix numpy so it can be initialized multiple times. Do you remember why the person wanted to do that? Or, if you let me know where you did that, I can fix it in my local source instead. "Paul F. Dubois" wrote: > > The import_array is not proper, but I have no idea if that is the problem or > not. > import array only goes in the initialization routine of modules that need > the Numpy C-API. > > Somebody convinced me to decref something in one of the numpy C module > initializers a while back, so maybe that was an error. (?) > I just do what they tell me... > > -----Original Message----- > From: numpy-discussion-admin at lists.sourceforge.net > [mailto:numpy-discussion-admin at lists.sourceforge.net]On Behalf Of Karl > Bellve > Sent: Thursday, May 10, 2001 5:57 AM > To: Numpy > Subject: [Numpy-discussion] Numpy bug? > > Since I didn't get a response, I will post again with a more appropiate > subject heading. If you run the following code, you get an error. It > appears that Numpy doesn't like to be restarted. If you take out the > lines "import_array();" and "PyRun_SimpleString("from Numeric import > *\n");" the code loops fine. > > for (int x = 0; x < 100; x++) > { > Py_Initialize(); > import_array(); > PyRun_SimpleString("from Numeric import *\n"); > TRACE("%d\n",x); > Py_Finalize(); > } > > Here is the output: > 0 > 1 > Fatal Python error: UNREF invalid object > > -- > Cheers, > > Karl Bellve > -- Cheers, Karl Bellve From mhagger at alum.mit.edu Fri May 11 13:59:14 2001 From: mhagger at alum.mit.edu (Michael Haggerty) Date: Fri, 11 May 2001 13:59:14 -0400 (EDT) Subject: [Numpy-discussion] Re: New version of compress for you In-Reply-To: References: Message-ID: <15100.10354.203517.711104@freak.kaiserty.com> Paul F. Dubois writes: > def compress (condition, x, dimension=-1): > """Select those parts of x for which condition is true. > Masked values in condition are considered false. > """ > c = filled(condition, 0) > m = getmask(x) > if m is not None: > m=Numeric.compress(c, m, dimension) > d = Numeric.compress(c, filled(x), dimension) > return masked_array(d, m) > > > I did want a treatment of masked conditions. Consider: > > compress( x > 10, x) I see your point, but this doesn't generalize to x having more than one dimension. And with my semantics, for your example (assuming 1-d), you could substitute compress((x > 10).filled(0), x) which isn't very obscure. Moreover, sometimes it is interesting to compress an array in such a way that masked values are carried along as masked values; this cannot be done with the existing compress(). Michael -- Michael Haggerty mhagger at alum.mit.edu From mhagger at alum.mit.edu Fri May 11 14:31:58 2001 From: mhagger at alum.mit.edu (Michael Haggerty) Date: 11 May 2001 14:31:58 -0400 Subject: [Numpy-discussion] Thoughts on masked arrays In-Reply-To: "Paul F. Dubois"'s message of "Fri, 11 May 2001 09:02:54 -0700" References: Message-ID: "Paul F. Dubois" writes: > The method __array__(self, typecode=None) is a special (existing) > hook for conversion to a Numeric array. Many operations in Numeric, > when presented with an object x to be operated upon, such as > Numeric.sqrt(x), will call x.__array__ as a final act of desperation > in an attempt to convert their argument to a Numeric > array. Heretofore it was essentially returning x.filled(). This > bothered me, because it was a silent conversion that replaced masked > values with the fill value. This bothered me too. > Solution: > > a. Add a method 'unmask()' which will replace the mask by None if > possible. It will not fail. > > b. Change MaskedArray.__array__ to work as follows: > a. self.unmask(), and then > b. Return the raw data if the mask is now None. > Otherwise, throw an MAError. Perfect. Great solution! Michael -- Michael Haggerty mhagger at alum.mit.edu From kiran.karkera at alopa.com Thu May 17 10:06:36 2001 From: kiran.karkera at alopa.com (kk) Date: Thu, 17 May 2001 19:36:36 +0530 Subject: [Numpy-discussion] creation of array of objects Message-ID: <3B03DAEC.4060609@alopa.com> hi all.. I want to create an array of class objects of size say mXn. What would be the easiest way to do this? eg. class test: pass i want 5x10 array of test() instances. Then only way i know of doing this ,is by creating a=array(test()) resize(a,[5,10]) but this creates only references to the initial objects.Is there some way of creating fresh objects ? Also, if the __init__() call of the class takes some parameters, i should be able to enter values.. thanx in anticipation ciao From chrishbarker at home.net Thu May 17 18:33:54 2001 From: chrishbarker at home.net (Chris Barker) Date: Thu, 17 May 2001 15:33:54 -0700 Subject: [Numpy-discussion] transpose and putmask questions. References: Message-ID: <3B0451D2.98EFA9FD@home.net> Hi all, I recently tried to take a slice out of an array, transpose it, and then change it with the putmask function. It didn't work, because it ends up not being contiguous. This brings up two points: 1) why does the array passed as the first argument to putmask have to be contiguous? this can be a substantial limitation, I end up making copies when I have no other reason to. 2) the docs say that transpose "returns a new array..." This is confusing. I had assumed that "new array" meant a copy, when, in fact it is another array, but the data is a reference, much like a slice. We might want to make it a little more clear in the docs what that means. Another question I have is about how to match array shapes. I find I am often getting error when I try to do something (like putmask) with two arrays, one of shape (n,1) and one of shape (n,). It seems to me that what I want to do is unambiguous, so I should be able to do it. Is the reason I can't do this: a) The NumPy interface is designed to be unambiguous, and so it is important that the code never silently translate a rank one array into a rank two array where one of the dimensions is one (and vice-versa) or b) No one has gotten around to making the code smart enough to do this when it is unambigous.? -Thanks, -Chris -- Christopher Barker, Ph.D. ChrisHBarker at home.net --- --- --- http://members.home.net/barkerlohmann ---@@ -----@@ -----@@ ------@@@ ------@@@ ------@@@ Oil Spill Modeling ------ @ ------ @ ------ @ Water Resources Engineering ------- --------- -------- Coastal and Fluvial Hydrodynamics -------------------------------------- ------------------------------------------------------------------------ From chrishbarker at home.net Fri May 18 13:09:36 2001 From: chrishbarker at home.net (Chris Barker) Date: Fri, 18 May 2001 10:09:36 -0700 Subject: [Numpy-discussion] transpose and putmask questions. References: Message-ID: <3B055750.C40AFB04@home.net> "Paul F. Dubois" wrote: >> 1) why does the array passed as the first argument to putmask have to be >> contiguous? this can be a substantial limitation, I end up making copies >> when I have no other reason to. > -----> because nobody has implemented the general case; feel free to > volunteer. Fair enough. I have yet to open up the NumPy code myself, but I have written some custom extensions with it, and have a number of small improvements in mind that I would like to make. Hopefully I will get to this some day. > b) No one has gotten around to making the code smart enough to do this > when it is unambigous.? > ------> Numpy's attitude toward extra dimensions of length 1 is somewhat > schizophrenic. > If you know you want one-d you can put a ravel() around it. > > In general the answer to your question is (b) but it is very > complicated so that is why people don't work on it. I suspected as much. I'm starting to get used to it. I think my problem is that I come from MATLAB, where everything is a 2-d array. Thanks for your comments. -Chris -- Christopher Barker, Ph.D. ChrisHBarker at home.net --- --- --- http://members.home.net/barkerlohmann ---@@ -----@@ -----@@ ------@@@ ------@@@ ------@@@ Oil Spill Modeling ------ @ ------ @ ------ @ Water Resources Engineering ------- --------- -------- Coastal and Fluvial Hydrodynamics -------------------------------------- ------------------------------------------------------------------------ From focke at SLAC.Stanford.EDU Fri May 18 18:35:20 2001 From: focke at SLAC.Stanford.EDU (Warren Focke) Date: Fri, 18 May 2001 15:35:20 -0700 (PDT) Subject: [Numpy-discussion] comparison style Message-ID: Howdy all, We've got these spiffy comparisons now (thanks Travis et al.!): ] >>> 2 <= arange(8) ] array([0, 0, 1, 1, 1, 1, 1, 1]) but, as noted in the rich comparison PEP, they don't chain: ] >>> 2 <= arange(8) < 6 ] array([1, 1, 1, 1, 1, 1, 0, 0]) since x <= y < z is equivalent to x <= y and y < z and we don't get to overload "and" :-(. But we did overload "&" long ago, and bitwise_and is equivalent to logical_and if the arguments are only 1 or 0, which is what our comparisons return (I think?). So (x <= y) & (y < z) seems like it might be a semi-reasonable substitute, and is shorter and appeals more to my preferring-operators-over-functions eyes, than Numeric.logical_and(x <= y, y < z) (which still beats the ^%$#&^(*&*^ out of Numeric.logical_and(Numeric.less_equal(x, y), Numeric.less(y, z)) ). But I'm a bit squeamish about it. Cons that I see are: - Do our comparisons really and truly ALWAYS return 1 or 0, even with Infs and Nans and whatnot? - It's potentially misleading for readers of the code. - It just seems kinda wrong, even if it does look better. So, what think you? bitwise, bytefoolish-ly y'rs, Warren Focke From chrishbarker at home.net Mon May 21 12:55:13 2001 From: chrishbarker at home.net (Chris Barker) Date: Mon, 21 May 2001 09:55:13 -0700 Subject: [Numpy-discussion] comparison style References: Message-ID: <3B094871.9B86742D@home.net> Warren Focke wrote: > ] >>> 2 <= arange(8) > ] array([0, 0, 1, 1, 1, 1, 1, 1]) > > but, as noted in the rich comparison PEP, they don't chain: > x <= y and y < z > > and we don't get to overload "and" :-(. Clearly that would be nice. Why wasn't "and" allowed to be overloaded when the rest of rich comparisons was allowed? > (x <= y) & (y < z) > > seems like it might be a semi-reasonable substitute, and is shorter and > appeals more to my preferring-operators-over-functions eyes, than > > Numeric.logical_and(x <= y, y < z) It looks fine to me as well. > - Do our comparisons really and truly ALWAYS return 1 or 0, even with Infs > and Nans and whatnot? Good question. There was a lot of discussion here recently about what level of support NumPy should give to NaNs an Inf. Personally, I'd like to see a well integrated support for it, which would include comparisons (though bitwise could still be a problem). Of course, I have neither the time nor the knowledge to do it, so I'll just have to wait... There has also been talk of a "logical" array type. Is that going to happen? If so, it would be the obvious choice to be returned from a comparison and we could make sure the bitwise & and | work with it. I notice that currently (I'm running 2.0, so ignore me if this has changed) comparisons return an array of python Integers which seems a massive waste of space. > - It's potentially misleading for readers of the code. well, yes, unless we really do have a 1 bit logical array type. -Chris -- Christopher Barker, Ph.D. ChrisHBarker at home.net --- --- --- http://members.home.net/barkerlohmann ---@@ -----@@ -----@@ ------@@@ ------@@@ ------@@@ Oil Spill Modeling ------ @ ------ @ ------ @ Water Resources Engineering ------- --------- -------- Coastal and Fluvial Hydrodynamics -------------------------------------- ------------------------------------------------------------------------ From paul at pfdubois.com Mon May 21 16:54:23 2001 From: paul at pfdubois.com (Paul F. Dubois) Date: Mon, 21 May 2001 13:54:23 -0700 Subject: [Numpy-discussion] Climate Data Analysis Tools Version 3.0.0 Message-ID: The Program for Climate Model Diagnosis and Intercomparison is pleased to announce the release of Version 3.0.0 of Climate Data Analysis Tools, an open-source set of Python modules and tools for accessing, manipulating, and plotting scientific data in general and climate data sets in particular. The tools make extensive use of Numerical Python and its optional Masked Arrays (MA) package and are designed to interoperate with them. The CDAT website is http://cdat.sf.net, and the SourceForge project page is http://sf.net/projects/cdat. Version 3.0.0 is available for Linux and Solaris. Binary distributions are available for RedHat 6.2, RedHat 7.1, SuSe 7.1, and Solaris 5.6. The user must have NetCDF and Tcl/Tk installed before using CDAT. -- Paul Dubois PCMDI Lawrence Livermore National Laboratory From grumpycrocodile at yahoo.com Tue May 22 13:11:35 2001 From: grumpycrocodile at yahoo.com (Grant Callis) Date: Tue, 22 May 2001 10:11:35 -0700 (PDT) Subject: [Numpy-discussion] Win32 binary of ScientificPython Message-ID: <20010522171135.17109.qmail@web14104.mail.yahoo.com> Does have an updated Win32 binary of Konrad Hinsen's ScientificPython? I've been using Robert Kern's version, but it's only good up to Python 1.5.2. Now that the main modules I use have been compiled for Python 2.1 (dislin & mxTools) I have upgraded. The only other binary extension I use often is ScientificPython, especially the netcdf portion. I'm not really a computer person so compiling myself is a scary option. Any help? Thank you, Grant Callis __________________________________________________ Do You Yahoo!? Yahoo! Auctions - buy the things you want at great prices http://auctions.yahoo.com/ From schaffer at optonline.net Tue May 22 15:34:51 2001 From: schaffer at optonline.net (Les Schaffer) Date: Tue, 22 May 2001 15:34:51 -0400 Subject: [Numpy-discussion] Win32 binary of ScientificPython In-Reply-To: <20010522171135.17109.qmail@web14104.mail.yahoo.com> References: <20010522171135.17109.qmail@web14104.mail.yahoo.com> Message-ID: <15114.48987.390000.219072@gargle.gargle.HOWL> > The only other binary extension I use often is ScientificPython, > especially the netcdf portion. I'm not really a computer person so > compiling myself is a scary option. Any help? i have the scientific_netcdf module compiled for python 2.1. i can email to you or can put a small zip file up on my web site. you need the netcdf dll. i built that too, if needed. havent tried compiling the MPI stuff yet. les schaffer From grumpycrocodile at yahoo.com Tue May 22 18:53:57 2001 From: grumpycrocodile at yahoo.com (Grant Callis) Date: Tue, 22 May 2001 15:53:57 -0700 (PDT) Subject: [Numpy-discussion] Win32 binary of ScientificPython In-Reply-To: <15114.48987.390000.219072@gargle.gargle.HOWL> Message-ID: <20010522225357.81298.qmail@web14104.mail.yahoo.com> > i have the scientific_netcdf module compiled for > python 2.1. i can > email to you or can put a small zip file up on my > web site. > > you need the netcdf dll. i built that too, if > needed. Excellent news! E-mail is fine but I'm sure that others are interested in the netcdf module and the netcdf .dll as well. Perhaps posting to your web site would be better? Regardless, thank you very much. Grant Callis __________________________________________________ Do You Yahoo!? Yahoo! Auctions - buy the things you want at great prices http://auctions.yahoo.com/ From prem at engr.uky.edu Wed May 23 11:32:41 2001 From: prem at engr.uky.edu (Prem Rachakonda) Date: Wed, 23 May 2001 11:32:41 -0400 (EDT) Subject: [Numpy-discussion] Help on installilng NumPY Message-ID: Hi, I am trying to install NumPY on my Windows 2000 machine. Could you please list out the procedure I have to follow to complete the installation. Prem. ____________________________________________________________ Prem Rachakonda ____________________________________________________________ Mailing Address: Work Address: 700 Woodland Ave., #E4 Teradyne Connection Systems, Lexington, Kentucky-40508 44 Simon St., Mail Stop#006, PH:859-323-2880 Nashua, NewHampshire - 03060 Work Phone No : 603-879-3442 From schaffer at optonline.net Wed May 23 16:44:38 2001 From: schaffer at optonline.net (Les Schaffer) Date: Wed, 23 May 2001 16:44:38 -0400 Subject: [Numpy-discussion] Win32 binary of ScientificPython In-Reply-To: <20010522225357.81298.qmail@web14104.mail.yahoo.com> References: <15114.48987.390000.219072@gargle.gargle.HOWL> <20010522225357.81298.qmail@web14104.mail.yahoo.com> Message-ID: <15116.8502.953000.768869@gargle.gargle.HOWL> > Excellent news! E-mail is fine but I'm sure that others are > interested in the netcdf module and the netcdf .dll as well. Perhaps > posting to your web site would be better? i will put a package up on my site by twm. the URL will be: http://folks.astrian.net/godzilla there's no isue with me distributing the betcdf dll is there??? les schaffer From roitblat at hawaii.edu Wed May 23 16:57:56 2001 From: roitblat at hawaii.edu (Herbert L. Roitblat) Date: Wed, 23 May 2001 10:57:56 -1000 Subject: [Numpy-discussion] Seeking hypergeometric 2F1 algorithm Message-ID: <04d901c0e3cb$0aec52e0$8fd6afcf@pixi.com> Does anyone have an agorithm for computing the Gaussian hypergeometric function 2F1 ? I understand that in principle it should be easy (sum of a series), but I think that there is a more efficient way to calculate it. My algorithmic skills are not up to it. Thanks, Herb From markespl at yahoo.com Thu May 24 09:41:45 2001 From: markespl at yahoo.com (Mark Esplin) Date: Thu, 24 May 2001 09:41:45 -0400 Subject: [Numpy-discussion] Seeking hypergeometric 2F1 algorithm Message-ID: <01052409414500.00638@marke> The Gaussian hypergeometric function 2F1 is included with other special functions in the cephes-1.3 package. See http://pylab.sourceforge.net I haven't used the hypergeometric function, but the Bessel functions work well with Numerical Python. -Mark Esplin ---------- Forwarded Message ---------- Subject: [Numpy-discussion] Seeking hypergeometric 2F1 algorithm Date: Wed, 23 May 2001 10:57:56 -1000 From: "Herbert L. Roitblat" To: Does anyone have an agorithm for computing the Gaussian hypergeometric function 2F1 ? I understand that in principle it should be easy (sum of a series), but I think that there is a more efficient way to calculate it. My algorithmic skills are not up to it. Thanks, Herb _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net http://lists.sourceforge.net/lists/listinfo/numpy-discussion ------------------------------------------------------- _________________________________________________________ Do You Yahoo!? Get your free @yahoo.com address at http://mail.yahoo.com From schaffer at optonline.net Fri May 25 09:35:42 2001 From: schaffer at optonline.net (Les Schaffer) Date: Fri, 25 May 2001 09:35:42 -0400 Subject: [Numpy-discussion] Windows DLLs for NumPyish things In-Reply-To: <15116.8502.953000.768869@gargle.gargle.HOWL> References: <15114.48987.390000.219072@gargle.gargle.HOWL> <20010522225357.81298.qmail@web14104.mail.yahoo.com> <15116.8502.953000.768869@gargle.gargle.HOWL> Message-ID: <15118.24494.140000.639372@gargle.gargle.HOWL> Scientific_netcdf: ----------------- I have placed a link to a package of DLLs for Scientific module on my web site: http://folks.astrian.net/godzilla included are the netcdf DLL and lib file, a pointer to the Unidata site where MS VC++ Project files are available (i used them to build this DLL), and the scientific_netcdf.pyd. I tested the package using netcdf_demo.py from the Scientific Example's directory. The project files on the Unidata site are for version 3.5, wheras mine are for beta8 of the same version. if the latest one's on their site cause you grief, i'll update mine for their latest version and post to the site. Visual Python: ------------- For people interested in using Visual Python under Windows and Python 2.1, i have also placed the cvisual.dll file on my page. PIL: --- Sometime this weekend i should have the PIL packages on my page updated as well. I have it working for Python 2.1, just need to bundle it up. les schaffer From markespl at yahoo.com Fri May 25 12:09:08 2001 From: markespl at yahoo.com (Mark Esplin) Date: Fri, 25 May 2001 12:09:08 -0400 Subject: [Numpy-discussion] The future of Numpy Message-ID: <01052512090800.01375@marke> What is the plan for the future development of Numerical Python? For example, will additional special functions be included in numpy, or is NumPy to be a building block for other applications? I guess the big question is "Will Numerical Python become a complete computing environment like Matlab or IDL?" There is an interesting series of article on Numerical Python by Eric Hagemann on O'Reilly network (www.oreillynet.com) where Eric Hagemann is using NumPy with other python modules as a computing environment more than as a software development framework. I think NumPy could become a lot like the development of linux. Linux strictly speaking is only the operating system kernel. However, a linux system has come to includes a whole set of packages that work together to make up a whole system. -Mark Esplin _________________________________________________________ Do You Yahoo!? Get your free @yahoo.com address at http://mail.yahoo.com From Barrett at stsci.edu Fri May 25 15:15:30 2001 From: Barrett at stsci.edu (Paul Barrett) Date: Fri, 25 May 2001 15:15:30 -0400 Subject: [Numpy-discussion] The future of Numpy References: <01052512090800.01375@marke> Message-ID: <3B0EAF52.97846E89@STScI.Edu> Mark Esplin wrote: > > What is the plan for the future development of Numerical Python? For > example, will additional special functions be included in numpy, or is > NumPy to be a building block for other applications? I guess the big > question is "Will Numerical Python become a complete computing > environment like Matlab or IDL?" > > There is an interesting series of article on Numerical Python by Eric > Hagemann on O'Reilly network (www.oreillynet.com) where Eric Hagemann > is using NumPy with other python modules as a computing environment > more than as a software development framework. > > I think NumPy could become a lot like the development of linux. Linux > strictly speaking is only the operating system kernel. However, a > linux system has come to includes a whole set of packages that work > together to make up a whole system. See PEP 209: Multi-dimensional Arrays http://python.sourceforge.net/peps/pep-0209.html -- Paul Barrett, PhD Space Telescope Science Institute Phone: 410-338-4475 ESS/Science Software Group FAX: 410-338-4767 Baltimore, MD 21218 From acapnotic at users.sourceforge.net Mon May 28 00:28:08 2001 From: acapnotic at users.sourceforge.net (Kevin Turner) Date: Sun, 27 May 2001 21:28:08 -0700 Subject: [Numpy-discussion] fromfunction documentation Message-ID: <20010527212808.A2822@troglodyte.menefee> Quoting from numdoc.pdf, dated March 31, 2001, in section "Creating an array from a function: fromfunction()" > The implementation of fromfunction consists of: > > def fromfunction(function, dimensions): > return apply(function, tuple(indices(dimensions))) > > which means that the function function is called for each element in > the sequence indices(dimensions) [...] This is wholly incorrect. apply() does not call a function for each element in the sequence, apply calls a function exactly *once*, using the elements of the sequence for arguments. Also, that same section of the documentation refers to "the first example below [...] works" and "the second array [...] fails", but there is only one example, which apparently is the failing one (as it's named "buggy"). Having a *working* example would be nice. =) Perhaps this is a formatting glitch in the documentation processor, but I couldn't find the documentation's source format. and then... there's always the possibility that I'm barking up the wrong tree. What I'm trying to achieve is this: I've got a function which, given a vector describing a point in n-dimensional space, returns a value for that point. I want a matrix of the shape I specify, filled these values, with the array indicies as coordinates for the points. Is that too much to ask? ;) Thanks, - Kevin Turner [using Python version 2.0, PyNum 19.0.0] -- Kevin Turner | OpenPGP encryption welcome here http://www.purl.org/wiki/python/KevinTurner From acapnotic at users.sourceforge.net Mon May 28 00:42:43 2001 From: acapnotic at users.sourceforge.net (Kevin Turner) Date: Sun, 27 May 2001 21:42:43 -0700 Subject: [Numpy-discussion] fromfunction documentation In-Reply-To: <20010527212808.A2822@troglodyte.menefee>; from acapnotic@users.sourceforge.net on Sun, May 27, 2001 at 09:28:08PM -0700 References: <20010527212808.A2822@troglodyte.menefee> Message-ID: <20010527214243.A2914@troglodyte.menefee> On Sun, May 27, 2001 at 09:28:08PM -0700, Kevin Turner wrote: > and then... there's always the possibility that I'm barking up the > wrong tree. What I'm trying to achieve is this: I've got a function > which, given a vector describing a point in n-dimensional space, returns > a value for that point. I want a matrix of the shape I specify, filled > these values, with the array indicies as coordinates for the points. err, I meant "multiarray", not "matrix" > > Is that too much to ask? ;) > > Thanks, From paul at pfdubois.com Mon May 28 15:31:44 2001 From: paul at pfdubois.com (Paul F. Dubois) Date: Mon, 28 May 2001 12:31:44 -0700 Subject: [Numpy-discussion] fromfunction documentation In-Reply-To: <20010527212808.A2822@troglodyte.menefee> Message-ID: A previous report of this doc error had caused me to fix it in my copy but it hadn't made it up to the web yet. The section has been redone to make it clearer, I hope. I am in the process of doing the update now. The documentation source is not available to you. Conventional use of fromfunction is like this: >>> from Numeric import * >>> def f (x, y): return x**2 + y**2 >>> a=fromfunction(f, (3,2)) >>> a array([[0, 1], [1, 2], [4, 5]]) For your use, you need a "helper" function. Suppose g is your function. >>> def g(x): return x[0] + x[1]**2 >>> def helper (x, y): return g([x,y]) >>> from Numeric import * >>> a = fromfunction(helper, (3,3)) >>> a array([[0, 1, 4], [1, 2, 5], [2, 3, 6]]) Of course, you can use a lambda to avoid explicity creating the helper. >>> fromfunction(lambda x,y:g([x,y]), (3,3)) array([[0, 1, 4], [1, 2, 5], [2, 3, 6]]) >>> -----Original Message----- From: numpy-discussion-admin at lists.sourceforge.net [mailto:numpy-discussion-admin at lists.sourceforge.net]On Behalf Of Kevin Turner Sent: Sunday, May 27, 2001 9:28 PM To: numpy-discussion at lists.sourceforge.net Subject: [Numpy-discussion] fromfunction documentation Quoting from numdoc.pdf, dated March 31, 2001, in section "Creating an array from a function: fromfunction()" > The implementation of fromfunction consists of: > > def fromfunction(function, dimensions): > return apply(function, tuple(indices(dimensions))) > > which means that the function function is called for each element in > the sequence indices(dimensions) [...] This is wholly incorrect. apply() does not call a function for each element in the sequence, apply calls a function exactly *once*, using the elements of the sequence for arguments. Also, that same section of the documentation refers to "the first example below [...] works" and "the second array [...] fails", but there is only one example, which apparently is the failing one (as it's named "buggy"). Having a *working* example would be nice. =) Perhaps this is a formatting glitch in the documentation processor, but I couldn't find the documentation's source format. and then... there's always the possibility that I'm barking up the wrong tree. What I'm trying to achieve is this: I've got a function which, given a vector describing a point in n-dimensional space, returns a value for that point. I want a matrix of the shape I specify, filled these values, with the array indicies as coordinates for the points. Is that too much to ask? ;) Thanks, - Kevin Turner [using Python version 2.0, PyNum 19.0.0] -- Kevin Turner | OpenPGP encryption welcome here http://www.purl.org/wiki/python/KevinTurner _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net http://lists.sourceforge.net/lists/listinfo/numpy-discussion From perry at stsci.edu Tue May 29 11:14:33 2001 From: perry at stsci.edu (Perry Greenfield) Date: Tue, 29 May 2001 11:14:33 -0400 Subject: [Numpy-discussion] Re: The future of Numpy In-Reply-To: Message-ID: > Mark Esplin wrote: > > > > What is the plan for the future development of Numerical Python? For > > example, will additional special functions be included in numpy, or is > > NumPy to be a building block for other applications? I guess the big > > question is "Will Numerical Python become a complete computing > > environment like Matlab or IDL?" > > > > There is an interesting series of article on Numerical Python by Eric > > Hagemann on O'Reilly network (www.oreillynet.com) where Eric Hagemann > > is using NumPy with other python modules as a computing environment > > more than as a software development framework. > > > > I think NumPy could become a lot like the development of linux. Linux > > strictly speaking is only the operating system kernel. However, a > > linux system has come to includes a whole set of packages that work > > together to make up a whole system. > > See PEP 209: Multi-dimensional Arrays > > http://python.sourceforge.net/peps/pep-0209.html > > -- > Paul Barrett, PhD Space Telescope Science Institute > We at STScI have been working on a new implementation of Numeric. We are far enough along to release an incomplete version for evaluation by the Numeric (the current version) developers in about 2 weeks. The existing PEP (209) doesn't address all the details of the implementation and there are some differences from the PEP, but it does give a general idea of what our goals are. We have been more concerned at this point in implementation than keeping the PEP up to date. The PEP will be revised when we release a preliminary version. The preliminary version will be 0) Called something else. Probably numarray. 1) Not completely backwards compatible with the existing Numeric. (I don't have time to detail the differences, but they will be listed when we make it availble). 2) Incomplete. It will be missing functionality, especially with regard to structural operations (choose, where, nonzero, put, take, compress...). It will have most types (excepting complex) and most standard Ufuncs. We do not expect this version will be usuable for most projects because of this. We are looking for feedback on the interface and design approach rather than advertising it as a usuable version. We also are interested in collaborators in implementing the rest of the functionality. It probably will be moved to sourceforge soon after. 3) Slower for smaller arrays. We expect that performance for large arrays (>100,000 elements) should be fairly respectable, but since most of the original implementation is in Python, the overhead for smaller arrays is substantially increased over the existing Numeric. While we did benchmarking for determining what design approaches were efficient enough, we have eschewed benchmarks until we have completed all the important functionality (soon). At that point we will begin to work on optimization to improve the performance. I'll stop here, but you should be hearing more about it in the near future. Perry Greenfield Space Telescope Science Institute Science Software Group From karshi.hasanov at utoronto.ca Thu May 31 02:11:36 2001 From: karshi.hasanov at utoronto.ca (karshi.hasanov at utoronto.ca) Date: Thu, 31 May 2001 01:11:36 -0500 Subject: [Numpy-discussion] numpyio Message-ID: <3B15E098.4FBC0FDA@utoronto.ca> I am having a problem to download the numpyio.py file. Does any one knows whereand how to get it? Thanks. From edcjones at erols.com Thu May 31 13:46:07 2001 From: edcjones at erols.com (Edward C. Jones) Date: Thu, 31 May 2001 13:46:07 -0400 Subject: [Numpy-discussion] Array values sometimes have "" Message-ID: <3B16835F.7BE382CE@erols.com> I use Numeric 20.0.0, Python 2.1, and RedHat 7.1 on a PC. Here is a small Python program: import Numeric a = Numeric.zeros((2,2), 'b') print a[0,0], type(a[0,0]) print a[0][0], type(a[0][0]) a = Numeric.zeros((2,2), 'i') print a[0,0], type(a[0,0]) print a[0][0], type(a[0][0]) a = Numeric.zeros((2,2), 'f') print a[0,0], type(a[0,0]) print a[0][0], type(a[0][0]) The output is: 0 0 0 0 0.0 0.0 This can't be correct. "a[0,0]" and "a[0][0]" should have the same type. A quick check of my old Python code shows that this problem may have appeared since Numeric version 17. Thanks, Ed Jones From jbaddor at physics.mcgill.ca Thu May 31 18:59:54 2001 From: jbaddor at physics.mcgill.ca (Jean-Bernard Addor) Date: Thu, 31 May 2001 18:59:54 -0400 (EDT) Subject: [Numpy-discussion] underflow array([2.9e-131])**3 Message-ID: Hey Numpy people! Do anyone know how to disable underflow exception errors in Numeric? I have a lot of these in my code. It is now a very important problem in my calculations. The only solution I see is to make a for loop and make the arithmetic in python instead of Numeric. Thanks for your help, Jean-Bernard Python 2.0 (#9, Feb 2 2001, 12:17:02) [GCC 2.95.2 19991024 (release)] on linux2 Type "copyright", "credits" or "license" for more information. Hello from .pythonrc.py >>> import Numeric >>> Numeric.__version__ '17.2.0' >>> Numeric.array([2.9e-131])**3 Traceback (most recent call last): File "", line 1, in ? OverflowError: math range error >>> 2.9e-131**3 0.0 From jbaddor at physics.mcgill.ca Thu May 31 19:09:03 2001 From: jbaddor at physics.mcgill.ca (Jean-Bernard Addor) Date: Thu, 31 May 2001 19:09:03 -0400 (EDT) Subject: [Numpy-discussion] PS: underflow array([2.9e-131+0j])**3 (why complex give a different result? bug?) Message-ID: Hey Numpy people! This a PS from my previous message. I just tried the underflow with complex : >>> Numeric.array([2.9e-131])**3 Traceback (most recent call last): File "", line 1, in ? OverflowError: math range error >>> 2.9e-131**3 0.0 >>> Numeric.array([2.9e-131+0j])**3 array([ 0.+0.j]) Now I have a quick solution for my problem, but I have the impression that this is a bug, at least I don't understand the underlying logic. If somebody can explain it to me? Jean-Bernard From ehagemann at home.com Tue May 1 20:26:26 2001 From: ehagemann at home.com (Eric Hagemann) Date: Tue, 1 May 2001 20:26:26 -0400 Subject: [Numpy-discussion] arg matrix max question Message-ID: <002c01c0d29e$865b9af0$cfefb418@cc911046a> I need to find the location of the maximum value in a two dimensional Numeric array The best I have come up with uses a couple of calls to argmax and then cross checking the results I know there must be a better way and that I am not the only one who needs to do such a thing Anybody got a code snippet to share ? -------------- next part -------------- An HTML attachment was scrubbed... URL: From ehagemann at home.com Thu May 3 20:53:22 2001 From: ehagemann at home.com (Eric Hagemann) Date: Thu, 3 May 2001 20:53:22 -0400 Subject: [Numpy-discussion] curious FFFT timing Message-ID: <006801c0d434$9e7d3ed0$cfefb418@cc911046a> I have an application that needs to perform close 1/2 million FFT's (actually inverse FFTs) I was stymied by the time it was taking so I created a simple benchmark program to measure raw fft speed - hoping to extrapolate and see if the FFT algorithm was the time consuming portion of the algorithm. The result I obtained surprised me Inverse FFT ( 4096) Number of seconds 5.608000 Time per FFT 0.005608 Forward FFT ( 4096) Number of seconds 3.164000 Time per FFT 0.003164 Inverse FFT ( 4095) Number of seconds 8.222000 Time per FFT 0.008222 Forward FFT ( 4095) Number of seconds 5.468000 Time per FFT 0.005468 Inverse FFT ( 8192) Number of seconds 12.578000 Time per FFT 0.012578 Forward FFT ( 8192) Number of seconds 7.551000 Time per FFT 0.007551 Inverse FFTs were taking almost twice as long as forward FFTs ! Bottom line I found that in the inverse code the multiplication of 1/N at the end was the culprit. When I removed the /n from the line return _raw_fft(a, n, axis, fftpack.cffti, fftpack.cfftb, _fft_cache)/n the code timing was more along the line of what was expected. Any one want to speculate on the timing difference for the 4095 vice 4096 long transforms ? Since 4095 is not a power of two I would have expected a greater time difference (DFT vice FFT) N*N / N*log2(N) = N/log2(N) = 4096/12 = 341.33 which is >> than the 1.72 seen above Cheers Eric ------------------------------------ Code Snippet --------------------------------------------- import time from Numeric import * from FFT import * def f_fft(number,length): a = arange(float(length)) start_time = time.time() for i in xrange(number): b = fft(a) durat = time.time() - start_time durat_per = durat / number print "Forward FFT (%10d) Number of seconds %12.6f Time per FFT %12.6f" % (length,durat,durat_per) def i_fft(number,length): a = arange(float(length)) start_time = time.time() for i in xrange(number): b = inverse_fft(a) durat = time.time() - start_time durat_per = durat / number print "Inverse FFT (%10d) Number of seconds %12.6f Time per FFT %12.6f" % (length,durat,durat_per) i_fft(1000,4096) f_fft(1000,4096) i_fft(1000,4095) f_fft(1000,4095) i_fft(1000,8192) f_fft(1000,8192) -------------- next part -------------- An HTML attachment was scrubbed... URL: From ehagemann at home.com Thu May 3 21:11:26 2001 From: ehagemann at home.com (Eric Hagemann) Date: Thu, 3 May 2001 21:11:26 -0400 Subject: [Numpy-discussion] curious FFFT timing References: <006801c0d434$9e7d3ed0$cfefb418@cc911046a> <3ACA7349.421BAD50@cfa.harvard.edu> Message-ID: <007601c0d437$249ef1a0$cfefb418@cc911046a> Scott, Right you are. My brain fried -- I forgot about the other "small prime" stuff Inverse FFT ( 4096) Number of seconds 5.508000 Time per FFT 0.005508 Forward FFT ( 4096) Number of seconds 3.225000 Time per FFT 0.003225 Inverse FFT ( 4097) Number of seconds 29.723000 Time per FFT 0.029723 Forward FFT ( 4097) Number of seconds 26.788000 Time per FFT 0.026788 Cheers Eric ----- Original Message ----- From: "Scott Ransom" To: "Eric Hagemann" Cc: Sent: Tuesday, April 03, 2001 9:05 PM Subject: Re: [Numpy-discussion] curious FFFT timing > Hi Eric, > > > Any one want to speculate on the timing difference for the 4095 vice > > 4096 long transforms ? > > Since 4095 is not a power of two I would have expected a greater time > > difference (DFT vice FFT) > > You are correct in that 4095 is not a power of two, but is is the > product of only small prime factors: > 3 * 3 * 5 * 7 * 13 = 4095 > > Since the FFT code implements a N*log_2(N) algorithm for numbers that > contain only small prime factors, the difference in times is rather > small. FFTs that have lengths that are powers-of-two tend to be more > efficient in general (since the decimation routines are cleaner for this > case). > > If you test with 4097 (17 * 241) it will be quite a bit slower I'd > guess... > > Scott > > > -- > Scott M. Ransom Address: Harvard-Smithsonian CfA > Phone: (617) 495-4142 60 Garden St. MS 10 > email: ransom at cfa.harvard.edu Cambridge, MA 02138 > GPG Fingerprint: 06A9 9553 78BE 16DB 407B FFCA 9BFA B6FF FFD3 2989 > > _______________________________________________ > Numpy-discussion mailing list > Numpy-discussion at lists.sourceforge.net > http://lists.sourceforge.net/lists/listinfo/numpy-discussion > From gvermeul at labs.polycnrs-gre.fr Tue May 1 08:37:00 2001 From: gvermeul at labs.polycnrs-gre.fr (Gerard Vermeulen) Date: Tue, 01 May 2001 14:37:00 +0200 Subject: [Numpy-discussion] 20.0.0b2 - OS X fix breaks building a shared library on Linux Message-ID: <3AEEADEC.8B586A7@labs.polycnrs-gre.fr> Hi, Configuration: Python-2.1 + Numeric-20.0.0b2 on Linux-Mandrake-7.2. I have a Python interface to Numeric and the Qwt-C++ plotting library. When building this interface (a shared library) for Numeric-20.0.0b2, I noticed that the additional static in front of the importing API pointer in arrayobject.h (your OS X fix) leads to an unresolved symbol PyArray_API when building a shared library. My fix is to remove the keyword static from arrayobject.h. Has somebody else encountered this problem? I suppose that this problem should show up with any shared library using import_array() (version 20.0.0b2) on Linux. If the additional static really solves the OS X problems, maybe an #ifdef OSX ... #else ... #endif is necessary. Best regards -- Gerard From paul at pfdubois.com Wed May 2 09:56:42 2001 From: paul at pfdubois.com (Paul F. Dubois) Date: Wed, 2 May 2001 06:56:42 -0700 Subject: [Numpy-discussion] static pointer Message-ID: Mr. Verveer wrote: "I recently installed this version and it unfortunately broke my own extension C code. I found that my code did not link properly anymore because of the addition of the static declaration tot the API pointer (as noted in changes.txt). Since my extensions consist of several source files, this led to problems when importing my extensions. If I understand correctly, the addition of the static keyword means that the import_array() function can now only be used by extensions that are defined in a single C source file. I can easily work around this problem, by defining my own api pointer, and using a modified import_array() in my code, but I thought I should draw your attention to this problem in any case. Maybe another solution can be found that does not need this pointer to be declared static?" -- I tried this change to see whether it had any bad consequences. It did not appear to have bad consequences in that Numpy itself and all its packages built ok, and we were formerly not following the instructions in the Python documentation and now we are. And, OS-X now worked. So far so good. If I understand you correctly, this does not work properly if the extension is built in pieces. I think I see why. (Frankly, linking and dynamic loading are not something I know much about; as a user I just flounder around.) This all started because of the need to have access to the C API of one dynamic extension from another. Is there a way of institutionalizing your work-around so that others can use it? Frankly, I'm out of ideas. The community is going to have to rescue me on this one. Paul P.S. Wait. I have an idea. Konrad was the one who put all this in. He should be punished for his good deed...(:-> From hinsen at cnrs-orleans.fr Wed May 2 13:39:11 2001 From: hinsen at cnrs-orleans.fr (Konrad Hinsen) Date: Wed, 2 May 2001 19:39:11 +0200 Subject: [Numpy-discussion] static pointer In-Reply-To: References: Message-ID: <200105021739.TAA15221@chinon.cnrs-orleans.fr> > If I understand correctly, the addition of the static keyword means that the > import_array() function can now only be used by extensions that are defined > in a single C source file. I can easily work around this problem, by True. That's exactly what I pointed out in a previous message in this list. > defining my own api pointer, and using a modified import_array() in my code, That seems to me the preferred solution. > built ok, and we were formerly not following the instructions in the Python > documentation and now we are. And, OS-X now worked. So far so good. If I And those instructions mention the static keyword for a good reason: without it there are problems not only with portability, but also with static linking of extension modules. > Is there a way of institutionalizing your work-around so that others can use > it? I thought about providing a general solution right in the header files. The problem is the generation of a unique symbol by preprocessor tricks, perhaps somehow based on the module name. I don't see how this can be done. On the other hand, if we accept that the extension module programmer has to supply a unique symbol, then it is even easy. This would only be necessary for multi-file extension modules. Does that seem like a good solution? > Wait. I have an idea. Konrad was the one who put all this in. He should be > punished for his good deed...(:-> I am even thinking of punishing myself by imposing extra work on me! ;-) Konrad. -- ------------------------------------------------------------------------------- Konrad Hinsen | E-Mail: hinsen at cnrs-orleans.fr Centre de Biophysique Moleculaire (CNRS) | Tel.: +33-2.38.25.56.24 Rue Charles Sadron | Fax: +33-2.38.63.15.17 45071 Orleans Cedex 2 | Deutsch/Esperanto/English/ France | Nederlands/Francais ------------------------------------------------------------------------------- From dubois1 at llnl.gov Wed May 2 18:06:40 2001 From: dubois1 at llnl.gov (Paul F. Dubois) Date: Wed, 2 May 2001 15:06:40 -0700 Subject: [Numpy-discussion] MA pickle Message-ID: <01050215114800.31638@almanac> I have a version of MA that pickles. I am so frightened of it that I don't even want to chuck it into CVS yet . I just wanted those who expressed concern about this to know there is hope. If someone is desperate please email me and I'll let you try it privately. From Peter.Verveer at embl-heidelberg.de Thu May 3 04:32:44 2001 From: Peter.Verveer at embl-heidelberg.de (Peter Verveer) Date: Thu, 3 May 2001 10:32:44 +0200 Subject: [Numpy-discussion] static pointer In-Reply-To: <200105021739.TAA15221@chinon.cnrs-orleans.fr> References: <200105021739.TAA15221@chinon.cnrs-orleans.fr> Message-ID: <01050310324400.06846@marvin.embl-heidelberg.de> On Wednesday 02 May 2001 19:39, Konrad Hinsen wrote: > > If I understand correctly, the addition of the static keyword means that > > the import_array() function can now only be used by extensions that are > > defined in a single C source file. I can easily work around this problem, > > by > > True. That's exactly what I pointed out in a previous message in this > list. > > > defining my own api pointer, and using a modified import_array() in my > > code, > > That seems to me the preferred solution. But this will still fail on OSX, I guess. > > built ok, and we were formerly not following the instructions in the > > Python documentation and now we are. And, OS-X now worked. So far so > > good. If I > > And those instructions mention the static keyword for a good reason: > without it there are problems not only with portability, but also with > static linking of extension modules. > > > Is there a way of institutionalizing your work-around so that others can > > use it? > > I thought about providing a general solution right in the header > files. The problem is the generation of a unique symbol by > preprocessor tricks, perhaps somehow based on the module name. > I don't see how this can be done. > > On the other hand, if we accept that the extension module programmer > has to supply a unique symbol, then it is even easy. This would only > be necessary for multi-file extension modules. Does that seem like a > good solution? That sounds very good. I don't mind providing a symbol name for the pointer myself. > > Wait. I have an idea. Konrad was the one who put all this in. He should > > be punished for his good deed...(:-> > > I am even thinking of punishing myself by imposing extra work on me! ;-) > > Konrad. -- Dr. Peter J. Verveer Bastiaens Group Cell Biology and Cell Biophysics Programme EMBL Meyerhofstrasse 1 D-69117 Heidelberg Germany Tel. : +49 6221 387245 Fax : +49 6221 387242 Email: Peter.Verveer at embl-heidelberg.de From hinsen at cnrs-orleans.fr Thu May 3 04:57:14 2001 From: hinsen at cnrs-orleans.fr (Konrad Hinsen) Date: Thu, 3 May 2001 10:57:14 +0200 Subject: [Numpy-discussion] static pointer In-Reply-To: <01050310324400.06846@marvin.embl-heidelberg.de> (message from Peter Verveer on Thu, 3 May 2001 10:32:44 +0200) References: <200105021739.TAA15221@chinon.cnrs-orleans.fr> <01050310324400.06846@marvin.embl-heidelberg.de> Message-ID: <200105030857.KAA15830@chinon.cnrs-orleans.fr> > > That seems to me the preferred solution. > > But this will still fail on OSX, I guess. Not necessarily. The API pointer just has to be passed between the source code modules in a global variable whose name is highly unlikely to be used somewhere else. > That sounds very good. I don't mind providing a symbol name for the pointer > myself. Good to hear. Is there anyone who would oppose this? Konrad. -- ------------------------------------------------------------------------------- Konrad Hinsen | E-Mail: hinsen at cnrs-orleans.fr Centre de Biophysique Moleculaire (CNRS) | Tel.: +33-2.38.25.56.24 Rue Charles Sadron | Fax: +33-2.38.63.15.17 45071 Orleans Cedex 2 | Deutsch/Esperanto/English/ France | Nederlands/Francais ------------------------------------------------------------------------------- From chrishbarker at home.net Thu May 3 17:47:02 2001 From: chrishbarker at home.net (Chris Barker) Date: Thu, 03 May 2001 14:47:02 -0700 Subject: [Numpy-discussion] Just what is an "Int32" ? References: Message-ID: <3AF1D1D6.A9B99A4E@home.net> Hi all, I'm writing an extension that uses a NumPy array (among other things). To make sure that I have passed in what I'm looking for, I do a: if (Array->descr->type_num != PyArray_LONG) ... PyErr_SetString (PyExc_ValueError,"Array is the wrong type"); return NULL; } (I want Array to be an array of longs) When I call my extension with an array built using array([...],Int32), this error results (this is on both Windows and Linux on Intel hardware). If I build the array with "Int" it works fine. This isn't really a problem, but I am curious, what is an Int32 on Intel hardware?, and why wouldn't it be a long? -Chris -- Christopher Barker, Ph.D. ChrisHBarker at home.net --- --- --- http://members.home.net/barkerlohmann ---@@ -----@@ -----@@ ------@@@ ------@@@ ------@@@ Oil Spill Modeling ------ @ ------ @ ------ @ Water Resources Engineering ------- --------- -------- Coastal and Fluvial Hydrodynamics -------------------------------------- ------------------------------------------------------------------------ From chrishbarker at home.net Fri May 4 14:52:01 2001 From: chrishbarker at home.net (Chris Barker) Date: Fri, 04 May 2001 11:52:01 -0700 Subject: [Numpy-discussion] Can this be done efficiently with array functions? References: <008601c03575$5036fa80$52f9a218@nc.rr.com> <20001014183233.28900.qmail@lisboa.ifm.uni-kiel.de> Message-ID: <3AF2FA51.9B0AC68F@home.net> I have a problem that I can't figure otu a non-kludgey way to solve: Given a (MXN) array, a, I want to compute the minimum value in each row, excluding the values that are zero. example: for this array, a: [[10, 0, 5, 0,] [ 0, 0, 0, 0,] [ 0, 5,15, 0,] [ 0, 0, 0, 0,] [ 0, 3, 1,25,] [ 0, 0, 0, 0,] [ 4, 7, 2,12,] [ 0, 0, 0, 0,]] I want: [[5,] [0,] [5,] [0,] [1,] [0,] [2,] [0,]] what I want is minimum.reduce(a,1), except that the zeros would be ignored. At first glace, Masked arrays seemed the perfect option (as that's what the zero means, no real data), but there doesn't seem to be a minimum.reduce() in MA. By the way, what is the downside to Masked Arrays? they seem to be a really powerful option. Is there any performance hit to using a masked array over a regular onw if there is no mask? If not, would it make sense to move toward just using Masked arrays for everything? thanks, -Chris -- Christopher Barker, Ph.D. ChrisHBarker at home.net --- --- --- http://members.home.net/barkerlohmann ---@@ -----@@ -----@@ ------@@@ ------@@@ ------@@@ Oil Spill Modeling ------ @ ------ @ ------ @ Water Resources Engineering ------- --------- -------- Coastal and Fluvial Hydrodynamics -------------------------------------- ------------------------------------------------------------------------ From paul at pfdubois.com Fri May 4 15:07:44 2001 From: paul at pfdubois.com (Paul F. Dubois) Date: Fri, 4 May 2001 12:07:44 -0700 Subject: [Numpy-discussion] Can this be done efficiently with array functions? In-Reply-To: <3AF2FA51.9B0AC68F@home.net> Message-ID: -----Original Message----- ...snip what I want is minimum.reduce(a,1), except that the zeros would be ignored. At first glace, Masked arrays seemed the perfect option (as that's what the zero means, no real data), but there doesn't seem to be a minimum.reduce() in MA. --> I'll add this to my list. Meanwhile you could do something like this, where x is your array: y = MA.masked_equal(x,0,copy=0) Numeric.minimum.reduce(y.filled(100000), 1) If x is floating point use masked_value rather than masked_equal. --- By the way, what is the downside to Masked Arrays? they seem to be a really powerful option. Is there any performance hit to using a masked array over a regular onw if there is no mask? If not, would it make sense to move toward just using Masked arrays for everything? thanks, -Chris --> There are two problems. There is a hit, even in the case of no mask. It is hard to quantify because it depends on what operations you do. Much of that could be eliminated if MA was in C but it is Python. It would have driven me mad to write it in C, however. More seriously, there are a few semantic issues where it isn't clear what you mean if a mask is present. From chrishbarker at home.net Fri May 4 17:42:34 2001 From: chrishbarker at home.net (Chris Barker) Date: Fri, 04 May 2001 14:42:34 -0700 Subject: [Numpy-discussion] Can this be done efficiently with array functions? References: <008601c03575$5036fa80$52f9a218@nc.rr.com> <20001014183233.28900.qmail@lisboa.ifm.uni-kiel.de> <3AF2FA51.9B0AC68F@home.net> <20010504193541.29870.qmail@lisboa.ifm.uni-kiel.de> Message-ID: <3AF3224A.ED902632@home.net> Janko and Paul, Thanks for your suggestions. I had come up with something like Janko's solution, but not as cleanly. It seems that Paul's suggestion is essentially the same thing. Maybe I'll do some tests to see which is faster. By the way, I tried Janko's code, and got an error on "a == 0" (I'm running Python 2.0). I looked at it again, and then looked at the release notes for Python 2.1, and saw that rich comparisons are working..Glory be! I'll be upgrading real soon, just for that! thanks again, -Chris PS: anyone have a response for my Int32 question? Why isn't it a "long" on Linux or windows on Intel? Janko Hauser wrote: > truely_max=100 > > a=array([[10, 0, 5, 0,], > [ 0, 0, 0, 0,], > [ 0, 5,15, 0,], > [ 0, 0, 0, 0,], > [ 0, 3, 1,25,], > [ 0, 0, 0, 0,], > [ 4, 7, 2,12,], > [ 0, 0, 0, 0,]]) > > am=minimum.reduce(where(a==0,truely_max,a),1) > am=where(am==truely_max,0,am) > print am > [5 0 5 0 1 0 2 0] > "Paul F. Dubois" wrote: > but there doesn't seem to be > a minimum.reduce() in MA. > > --> I'll add this to my list. That will be great if you get a chance. > Meanwhile you could do something like this, > where x is your array: > y = MA.masked_equal(x,0,copy=0) > Numeric.minimum.reduce(y.filled(100000), 1) > If x is floating point use masked_value rather than masked_equal. > --- > --> There are two problems. There is a hit, even in the case of no mask. > It is hard to quantify because it depends on what operations you do. > Much of that could be eliminated if MA was in C but it is Python. > It would have driven me mad to write it in C, however. WOW, I'm impressed, I had assumed that some C was involved. Nice job. > More seriously, there are a few semantic issues where it isn't clear > what you mean if a mask is present. I'm sure these could be resolved. It would be nice if we could have in mind to move in that direction. I'm not going ot be able to do any of the coding, so that's the last you'll here from me about it. -Chris -- Christopher Barker, Ph.D. ChrisHBarker at home.net --- --- --- http://members.home.net/barkerlohmann ---@@ -----@@ -----@@ ------@@@ ------@@@ ------@@@ Oil Spill Modeling ------ @ ------ @ ------ @ Water Resources Engineering ------- --------- -------- Coastal and Fluvial Hydrodynamics -------------------------------------- ------------------------------------------------------------------------ From Karl.Bellve at umassmed.edu Wed May 9 12:06:39 2001 From: Karl.Bellve at umassmed.edu (Karl Bellve) Date: Wed, 09 May 2001 12:06:39 -0400 Subject: [Numpy-discussion] UNREF Message-ID: <3AF96B0F.642F365D@umassmed.edu> Occasionally, if I reinitialize Python and rerun a script, I get the following error: FATAL Python error: UNREF invalid object from my code I do the following: /* alot of other crap and then */ PyRun_SimpleString("From Numeric import *\n"); eventually in multiarraymodule.c array_set_string_function() Py_INCREF(Py_None) which eventually calls (according to my stack trace) where the error occurs PyArray_SetStringFunction() Py_XDECREF(PyArray_ReprFunction) Is this a Numpy problem, Python problem, or my problem? I can run the same python script several times, and then I get the error. Something isn't being cleaned up properly around Py_Initialize() Py_Finalize() pair. I am using Python 2.1b1 and Numeric 18_1 with Boost -- Cheers, Karl Bellve From hinsen at cnrs-orleans.fr Wed May 9 12:12:25 2001 From: hinsen at cnrs-orleans.fr (Konrad Hinsen) Date: Wed, 9 May 2001 18:12:25 +0200 Subject: [Numpy-discussion] Modified header files Message-ID: <200105091612.SAA24523@chinon.cnrs-orleans.fr> Recently we had a discussion about how to use NumPy arrays from extension modules with multiple source files, on various platforms. The modified header files that are attached to this message provide a (hopefully!) general solution. In fact, I propose to make them part of the official distribution, unless there are objections. If used like before, these header files give exactly the same result as the ones in NumPy 20.0.0. However, they permit to define the name of the C API pointer array and make it globally visible. Under the condition that the chosen name is unique, this should not create problems under any platform, no matter if static or dynamic linking is used. To use NumPy features in multiple source file extension modules, you have to write #define PY_ARRAY_UNIQUE_SYMBOL PyArrayXXX #include "Numeric/arrayobject.h" in the main source file (the one that contains the init function) and #define PY_ARRAY_UNIQUE_SYMBOL PyArrayXXX #define NO_IMPORT_ARRAY #include "Numeric/arrayobject.h" in all the others. The symbol you choose instead of PyArrayXXX should contain both the name of the imported module (array) and the name of the importing module (whatever your module is called) in order to be unique with a reasonably high probability. The same applies to the Ufunc module, just replace "array" by "ufunc" in the example. I have also applied the "static" correction to the Ufunc header file, there is no reason not to do it. Konrad. -- ------------------------------------------------------------------------------- Konrad Hinsen | E-Mail: hinsen at cnrs-orleans.fr Centre de Biophysique Moleculaire (CNRS) | Tel.: +33-2.38.25.56.24 Rue Charles Sadron | Fax: +33-2.38.63.15.17 45071 Orleans Cedex 2 | Deutsch/Esperanto/English/ France | Nederlands/Francais ------------------------------------------------------------------------------- -------------- next part -------------- A non-text attachment was scrubbed... Name: arrayobject.h Type: application/octet-stream Size: 13881 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: ufuncobject.h Type: application/octet-stream Size: 6701 bytes Desc: not available URL: From Karl.Bellve at umassmed.edu Wed May 9 12:22:11 2001 From: Karl.Bellve at umassmed.edu (Karl Bellve) Date: Wed, 09 May 2001 12:22:11 -0400 Subject: [Numpy-discussion] UNREF References: <3AF96B0F.642F365D@umassmed.edu> Message-ID: <3AF96EB3.93A2B06@umassmed.edu> More info: I can reproduce this error with the following loop: for (int x = 0; x < 100; x++) { Py_Initialize(); import_array(); PyRun_SimpleString("from Numeric import *\n"); TRACE("%d\n",x); Py_Finalize(); } Here is the output: 0 1 Fatal Python error: UNREF invalid object -- Cheers, Karl Bellve From mhagger at alum.mit.edu Wed May 9 19:20:37 2001 From: mhagger at alum.mit.edu (Michael Haggerty) Date: 09 May 2001 19:20:37 -0400 Subject: [Numpy-discussion] Thoughts on masked arrays Message-ID: Hi, I've spent several days using the masked arrays that have been added to NumPy recently. They're a great feature and they were just what I needed for the little project I was working on (aside from a few bugs that I found). However, there were a few things about MA that I found inconvenient and/or counterintuitive, so I thought I'd post them to the list while they're fresh in my mind. I'm using Numeric-20.0.0b2. 1. I couldn't find a simple way to tell if all of the cells of a masked array are unmasked. There are times when you fill an array incrementally and you want to convert it to a Numeric array but first make sure that all of the elements have been set. "m.filled()" is a bit dangerous (in my opinion) because it silently fills. The shortest idiom I could think of is >>> assert not logical_or.reduce(ravel(MA.getmaskarray(m))) which isn't very short :-) and is also awkward because it creates a mask array even if m.mask() is None. How about a m.is_unmasked() method, or even giving a special meaning to "m.filled(masked)", namely that it raises an exception if any cells are still masked. (As an optimization, this method could set m.__mask = None to speed up future checks.) 2. I can't reproduce this problem now, but I could swear that the MaskedArray.__str__() method sometimes printed "typecode='O'" if masked.enabled() is true. This would be a byproduct of using Numeric's __str__() method to print the array, at least under the unknown circumstances in which Numeric.__str__() prints the typecode. This confused me for a while. 3. I found the semantics of MA.compress(condition,a,axis=0) to be inconvenient and inconsistent with those of Numeric.compress. MA.compress() squeezes out not only those elements for which condition is false, but also those elements that are masked. This differs from the behavior of Numeric.compress, which always returns an array with the "axis" dimension equal to the number of nonzero elements of "condition". The real problem, though, is that MA.compress can't be used on a multidimensional array with a nontrivial mask, because squeezing out the masked values is highly unlikely to result in a rectangular matrix. It is nice to be able to squeeze masked values out of a 1-d array, but not at the price of not being able to use compress on a multidimensional array. I suggest giving MA.compress() semantics closer to those of Numeric.compress(), and adding an optional argument or a separate method to cause masked elements to be omitted. Thanks for a great package! Yours, Michael -- Michael Haggerty mhagger at alum.mit.edu From paul at pfdubois.com Wed May 9 22:56:16 2001 From: paul at pfdubois.com (Paul F. Dubois) Date: Wed, 9 May 2001 19:56:16 -0700 Subject: [Numpy-discussion] Thoughts on masked arrays In-Reply-To: Message-ID: -----Original Message----- From: numpy-discussion-admin at lists.sourceforge.net [mailto:numpy-discussion-admin at lists.sourceforge.net]On Behalf Of Michael Haggerty wrote 1. I couldn't find a simple way to tell if all of the cells of a masked array are unmasked. There are times when you fill an array incrementally and you want to convert it to a Numeric array but first make sure that all of the elements have been set. "m.filled()" is a bit dangerous (in my opinion) because it silently fills. The shortest idiom I could think of is >>> assert not logical_or.reduce(ravel(MA.getmaskarray(m))) which isn't very short :-) and is also awkward because it creates a mask array even if m.mask() is None. How about a m.is_unmasked() method, or even giving a special meaning to "m.filled(masked)", namely that it raises an exception if any cells are still masked. (As an optimization, this method could set m.__mask = None to speed up future checks.) ====== >>> from MA import * >>> x=array([[1,2],[3,4]],mask=[[0,0],[0,0]]) >>> count(x) 4 >>> product(x.shape) 4 >>> So your test could be if count(x) < product(x.shape): error... make_mask(m, flag=1) will make a mask and have it be None if possible. It also accepts an argument of None correctly. So your test could be if make_mask(m.mask(),flag=1) is not None: error... You could also consider if not Numeric.allclose(m.filled(0), m.filled(1)) or m.mask() is not None and not Numeric.alltrue(Numeric.ravel(m.mask())): Is that enough ways to do it? (TM) (:-> I don't recommend using assert if the test is data-driven, since it won't get executed with python -O. Instead use if...: raise .... I'm not against is_unmasked but I'm not sure how much it would get used and I don't like the name. I hate query methods with side effects (if you use them in an assert you change the program). A method that replaces the mask with None if possible might make sense. m.unmask()? m.demask()? m.debride() ? ============= 2. I can't reproduce this problem now, but I could swear that the MaskedArray.__str__() method sometimes printed "typecode='O'" if masked.enabled() is true. This would be a byproduct of using Numeric's __str__() method to print the array, at least under the unknown circumstances in which Numeric.__str__() prints the typecode. This confused me for a while. ========= Short of writing my own print routine, I basically have to create something filled with '--', which is of course of type object. That's why you can disable it. (:-> ========= 3. I found the semantics of MA.compress(condition,a,axis=0) to be inconvenient and inconsistent with those of Numeric.compress. MA.compress() squeezes out not only those elements for which condition is false, but also those elements that are masked. This differs from the behavior of Numeric.compress, which always returns an array with the "axis" dimension equal to the number of nonzero elements of "condition". The real problem, though, is that MA.compress can't be used on a multidimensional array with a nontrivial mask, because squeezing out the masked values is highly unlikely to result in a rectangular matrix. It is nice to be able to squeeze masked values out of a 1-d array, but not at the price of not being able to use compress on a multidimensional array. I suggest giving MA.compress() semantics closer to those of Numeric.compress(), and adding an optional argument or a separate method to cause masked elements to be omitted. ====== It has been an interesting project in that there are hundreds of these individual little design questions. Can you propose the semantics you would like in a precise way? Include the case where the condition has masked values. ====== Thanks for a great package! Yours, Michael === I appreciate the encouragement. -- Paul From paul at pfdubois.com Wed May 9 23:12:01 2001 From: paul at pfdubois.com (Paul F. Dubois) Date: Wed, 9 May 2001 20:12:01 -0700 Subject: [Numpy-discussion] Modified header files In-Reply-To: <200105091612.SAA24523@chinon.cnrs-orleans.fr> Message-ID: I have added these files to CVS. Thanks! --Paul -----Original Message----- From: numpy-discussion-admin at lists.sourceforge.net [mailto:numpy-discussion-admin at lists.sourceforge.net]On Behalf Of Konrad Hinsen Sent: Wednesday, May 09, 2001 9:12 AM To: numpy-discussion at lists.sourceforge.net Subject: [Numpy-discussion] Modified header files Recently we had a discussion about how to use NumPy arrays from extension modules with multiple source files, on various platforms. The modified header files that are attached to this message provide a (hopefully!) general solution. In fact, I propose to make them part of the official distribution, unless there are objections. If used like before, these header files give exactly the same result as the ones in NumPy 20.0.0. However, they permit to define the name of the C API pointer array and make it globally visible. Under the condition that the chosen name is unique, this should not create problems under any platform, no matter if static or dynamic linking is used. To use NumPy features in multiple source file extension modules, you have to write #define PY_ARRAY_UNIQUE_SYMBOL PyArrayXXX #include "Numeric/arrayobject.h" in the main source file (the one that contains the init function) and #define PY_ARRAY_UNIQUE_SYMBOL PyArrayXXX #define NO_IMPORT_ARRAY #include "Numeric/arrayobject.h" in all the others. The symbol you choose instead of PyArrayXXX should contain both the name of the imported module (array) and the name of the importing module (whatever your module is called) in order to be unique with a reasonably high probability. The same applies to the Ufunc module, just replace "array" by "ufunc" in the example. I have also applied the "static" correction to the Ufunc header file, there is no reason not to do it. Konrad. -- ---------------------------------------------------------------------------- --- Konrad Hinsen | E-Mail: hinsen at cnrs-orleans.fr Centre de Biophysique Moleculaire (CNRS) | Tel.: +33-2.38.25.56.24 Rue Charles Sadron | Fax: +33-2.38.63.15.17 45071 Orleans Cedex 2 | Deutsch/Esperanto/English/ France | Nederlands/Francais ---------------------------------------------------------------------------- --- From Karl.Bellve at umassmed.edu Thu May 10 08:57:24 2001 From: Karl.Bellve at umassmed.edu (Karl Bellve) Date: Thu, 10 May 2001 08:57:24 -0400 Subject: [Numpy-discussion] Numpy bug? Message-ID: <3AFA9034.F581DCA9@umassmed.edu> Since I didn't get a response, I will post again with a more appropiate subject heading. If you run the following code, you get an error. It appears that Numpy doesn't like to be restarted. If you take out the lines "import_array();" and "PyRun_SimpleString("from Numeric import *\n");" the code loops fine. for (int x = 0; x < 100; x++) { Py_Initialize(); import_array(); PyRun_SimpleString("from Numeric import *\n"); TRACE("%d\n",x); Py_Finalize(); } Here is the output: 0 1 Fatal Python error: UNREF invalid object -- Cheers, Karl Bellve From paul at pfdubois.com Thu May 10 13:19:20 2001 From: paul at pfdubois.com (Paul F. Dubois) Date: Thu, 10 May 2001 10:19:20 -0700 Subject: [Numpy-discussion] Numpy bug? In-Reply-To: <3AFA9034.F581DCA9@umassmed.edu> Message-ID: The import_array is not proper, but I have no idea if that is the problem or not. import array only goes in the initialization routine of modules that need the Numpy C-API. Somebody convinced me to decref something in one of the numpy C module initializers a while back, so maybe that was an error. (?) I just do what they tell me... -----Original Message----- From: numpy-discussion-admin at lists.sourceforge.net [mailto:numpy-discussion-admin at lists.sourceforge.net]On Behalf Of Karl Bellve Sent: Thursday, May 10, 2001 5:57 AM To: Numpy Subject: [Numpy-discussion] Numpy bug? Since I didn't get a response, I will post again with a more appropiate subject heading. If you run the following code, you get an error. It appears that Numpy doesn't like to be restarted. If you take out the lines "import_array();" and "PyRun_SimpleString("from Numeric import *\n");" the code loops fine. for (int x = 0; x < 100; x++) { Py_Initialize(); import_array(); PyRun_SimpleString("from Numeric import *\n"); TRACE("%d\n",x); Py_Finalize(); } Here is the output: 0 1 Fatal Python error: UNREF invalid object -- Cheers, Karl Bellve _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net http://lists.sourceforge.net/lists/listinfo/numpy-discussion From paul at pfdubois.com Thu May 10 15:20:52 2001 From: paul at pfdubois.com (Paul F. Dubois) Date: Thu, 10 May 2001 12:20:52 -0700 Subject: [Numpy-discussion] 20.1.0a3 -- for those who like pickles Message-ID: I put a source tarball up for 20.1.0a3, also in CVS as of now. This contains: DL_EXPORT changes Konrad's fix to import_array A new MA that pickles A new package "Properties", that replaces MA.activeattr. Properties allows you to have properties such as MA/Numeric's shape attribute, that look like attributes but actually invoke functions on get, set, and/or delete. It can also be used to make attributes unwriteable or undeleteable. It doesn't have anything to do with Numeric but I use it in MA and kinds and don't have much else of an idea where to put it. From mhagger at alum.mit.edu Thu May 10 18:11:37 2001 From: mhagger at alum.mit.edu (Michael Haggerty) Date: 10 May 2001 18:11:37 -0400 Subject: [Numpy-discussion] Thoughts on masked arrays In-Reply-To: "Paul F. Dubois"'s message of "Wed, 9 May 2001 19:56:16 -0700" References: Message-ID: "Paul F. Dubois" writes: > -----Original Message----- > Michael Haggerty wrote > 1. I couldn't find a simple way to tell if all of the cells of a > masked array are unmasked. > ====== > So your test could be if count(x) < product(x.shape): error... > > So your test could be > if make_mask(m.mask(),flag=1) is not None: > error... > > You could also consider if not Numeric.allclose(m.filled(0), m.filled(1)) > or > m.mask() is not None and not Numeric.alltrue(Numeric.ravel(m.mask())): Shouldn't that be m.mask() is not None and Numeric.sometrue(Numeric.ravel(m.mask())) ? ("Proof" that these expressions are nonintuitive.) > Is that enough ways to do it? (TM) (:-> Frankly, it's too many ways to do it, none of them obvious to the writer or the reader. This is a simple and useful concept and it should have one obvious implementation. > I'm not against is_unmasked but I'm not sure how much it would get > used and I don't like the name. I hate query methods with side > effects (if you use them in an assert you change the program). In this case the side effect is to change the internal representation of the object without changing its semantics, so I don't find it too objectionable. But omit this optimization if you prefer; the query method would be just as useful even without the side effect. Because of the relationship with filled(), maybe this query function should be called m.isfull(). There should probably also be an isfull(m) function for the same reason that there is a mask(m) function. > A method that replaces the mask with None if possible might make > sense. m.unmask()? m.demask()? m.debride() ? Of these names, I like m.unmask() the best. I assume that it would set m.__mask=None if possible and throw an exception if not. On the other hand, while it would be desirable to have a function equivalent (i.e., unmask(m)), this would be awkward because a function should usually not change its argument. Therefore, I suggest adding a safe analogue of raw_data() that throws an exception if the array has a nontrivial mask and otherwise returns self.__data. E.g. [untested]: class MaskedArray: [...] def data(self): """If no values are masked, return self.__data(). Otherwise raise an exception. """ d = self.__data m = self.__mask if m is not None and Numeric.sometrue(Numeric.ravel(m)): raise MAError, "MaskedArray cannot be converted to array" elif d.iscontiguous(): return d else: return Numeric.array(d, typecode=d.typecode(), copy=1, savespace = d.spacesaver()) def data(a): if isinstance(a, MaskedArray): return m.data() elif isinstance(a, Numeric.ArrayType) and a.iscontiguous(): return a else: return Numeric.array(a) A more obscure name should be chosen since you seem to encourage "from MA import *". > 3. I found the semantics of MA.compress(condition,a,axis=0) to be > inconvenient and inconsistent with those of Numeric.compress. > ====== > It has been an interesting project in that there are hundreds of these > individual little design questions. > Can you propose the semantics you would like in a precise way? Include the > case where the condition has masked values. > ====== In the simple case where the condition has no masked values, I think compress() should simply pick slices out according to condition, without regard to which cells of x are masked. When condition is masked, I don't think that there is a sensible interpretation for compress() because a "masked" value in condition means you don't know whether that slice of x should be included or not. Since you can't have an output array of indeterminate shape, I would throw an exception if condition is masked. Here is my attempt [untested]: def compress(condition, x, dimension=-1): # data function is defined above (throws exception if condition is masked): c = data(condition) if mask(x) is None: mask = None else: mask=Numeric.compress(condition, mask(x), dimension) return array(Numeric.compress(condition, filled(x), dimension), mask=mask) Yours, Michael -- Michael Haggerty mhagger at alum.mit.edu From paul at pfdubois.com Fri May 11 12:02:54 2001 From: paul at pfdubois.com (Paul F. Dubois) Date: Fri, 11 May 2001 09:02:54 -0700 Subject: [Numpy-discussion] Thoughts on masked arrays In-Reply-To: Message-ID: Thank you for provoking me to think about these issues in MA. Here is the conclusion I have reached. Please let me know what you think of it. Background: Michael wanted a way to use a masked array as a Numeric array but with assurance that in fact no element was masked, without obscure tests such as count(x) == product(x.shape). The method __array__(self, typecode=None) is a special (existing) hook for conversion to a Numeric array. Many operations in Numeric, when presented with an object x to be operated upon, such as Numeric.sqrt(x), will call x.__array__ as a final act of desperation in an attempt to convert their argument to a Numeric array. Heretofore it was essentially returning x.filled(). This bothered me, because it was a silent conversion that replaced masked values with the fill value. Solution: a. Add a method 'unmask()' which will replace the mask by None if possible. It will not fail. b. Change MaskedArray.__array__ to work as follows: a. self.unmask(), and then b. Return the raw data if the mask is now None. Otherwise, throw an MAError. Example usage: >>> from MA import * >>> x=arange(10) >>> Numeric.array(x) [0,1,2,3,4,5,6,7,8,9,] >>> x[3]=masked >>> Numeric.array(x) Traceback (most recent call last): File "", line 1, in ? File "/pcmdi/dubois/linux/lib/python2.1/site-packages/MA/MA.py", line 578, in __array__ raise MAError, \ MA.MA.MAError: Cannot convert masked array to Numeric because data is masked in one or more locations. Merits of this solution: a. It reads like what it is -- there is no doubt you are converting to a Numeric array when you see Numeric.array. b. It gives you the full range of options in Numeric.array, such as messing with the typecode. c. It allows Numeric operations for speed on masked arrays that you know to be masked in name only. No copy of data occurs here unless the typecode needs to be changed. d. It removes the possibility of a 'dishonest' conversion. e. No new method or function is required, other than the otherwise-useful unmask(). f. Successive conversions are optimized because once the mask is None, unmask is cheap. Deficiency: __array__ becomes a query with an internal, albeit safe, side-effect. Mitigating this is that __array__ is not a "public" method and would not normally be used in assertions. From Karl.Bellve at umassmed.edu Fri May 11 12:12:18 2001 From: Karl.Bellve at umassmed.edu (Karl Bellve) Date: Fri, 11 May 2001 12:12:18 -0400 Subject: [Numpy-discussion] Numpy bug? References: Message-ID: <3AFC0F62.25968CC8@umassmed.edu> How can we correct this error? Otherwise, Numpy can only be initialized once per application/thread? Here is what I do, a user loads an image into my application, then runs a Python script. I do a Py_Initialize(), initialize Numpy, initialize my module, run the script and finally Py_Finalize(). Each image can do this. My module only knows about its own image and can run a python script multiple times. So, if I have to make it more global, I would have to run Py_Intialize/Py_Finalize once per application. Redesign my module so that a user must pick the specific image to work on. Redesign the scripts so that a user must transverse all the images to the correct image. Hey, I would rather fix numpy so it can be initialized multiple times. Do you remember why the person wanted to do that? Or, if you let me know where you did that, I can fix it in my local source instead. "Paul F. Dubois" wrote: > > The import_array is not proper, but I have no idea if that is the problem or > not. > import array only goes in the initialization routine of modules that need > the Numpy C-API. > > Somebody convinced me to decref something in one of the numpy C module > initializers a while back, so maybe that was an error. (?) > I just do what they tell me... > > -----Original Message----- > From: numpy-discussion-admin at lists.sourceforge.net > [mailto:numpy-discussion-admin at lists.sourceforge.net]On Behalf Of Karl > Bellve > Sent: Thursday, May 10, 2001 5:57 AM > To: Numpy > Subject: [Numpy-discussion] Numpy bug? > > Since I didn't get a response, I will post again with a more appropiate > subject heading. If you run the following code, you get an error. It > appears that Numpy doesn't like to be restarted. If you take out the > lines "import_array();" and "PyRun_SimpleString("from Numeric import > *\n");" the code loops fine. > > for (int x = 0; x < 100; x++) > { > Py_Initialize(); > import_array(); > PyRun_SimpleString("from Numeric import *\n"); > TRACE("%d\n",x); > Py_Finalize(); > } > > Here is the output: > 0 > 1 > Fatal Python error: UNREF invalid object > > -- > Cheers, > > Karl Bellve > -- Cheers, Karl Bellve From mhagger at alum.mit.edu Fri May 11 13:59:14 2001 From: mhagger at alum.mit.edu (Michael Haggerty) Date: Fri, 11 May 2001 13:59:14 -0400 (EDT) Subject: [Numpy-discussion] Re: New version of compress for you In-Reply-To: References: Message-ID: <15100.10354.203517.711104@freak.kaiserty.com> Paul F. Dubois writes: > def compress (condition, x, dimension=-1): > """Select those parts of x for which condition is true. > Masked values in condition are considered false. > """ > c = filled(condition, 0) > m = getmask(x) > if m is not None: > m=Numeric.compress(c, m, dimension) > d = Numeric.compress(c, filled(x), dimension) > return masked_array(d, m) > > > I did want a treatment of masked conditions. Consider: > > compress( x > 10, x) I see your point, but this doesn't generalize to x having more than one dimension. And with my semantics, for your example (assuming 1-d), you could substitute compress((x > 10).filled(0), x) which isn't very obscure. Moreover, sometimes it is interesting to compress an array in such a way that masked values are carried along as masked values; this cannot be done with the existing compress(). Michael -- Michael Haggerty mhagger at alum.mit.edu From mhagger at alum.mit.edu Fri May 11 14:31:58 2001 From: mhagger at alum.mit.edu (Michael Haggerty) Date: 11 May 2001 14:31:58 -0400 Subject: [Numpy-discussion] Thoughts on masked arrays In-Reply-To: "Paul F. Dubois"'s message of "Fri, 11 May 2001 09:02:54 -0700" References: Message-ID: "Paul F. Dubois" writes: > The method __array__(self, typecode=None) is a special (existing) > hook for conversion to a Numeric array. Many operations in Numeric, > when presented with an object x to be operated upon, such as > Numeric.sqrt(x), will call x.__array__ as a final act of desperation > in an attempt to convert their argument to a Numeric > array. Heretofore it was essentially returning x.filled(). This > bothered me, because it was a silent conversion that replaced masked > values with the fill value. This bothered me too. > Solution: > > a. Add a method 'unmask()' which will replace the mask by None if > possible. It will not fail. > > b. Change MaskedArray.__array__ to work as follows: > a. self.unmask(), and then > b. Return the raw data if the mask is now None. > Otherwise, throw an MAError. Perfect. Great solution! Michael -- Michael Haggerty mhagger at alum.mit.edu From kiran.karkera at alopa.com Thu May 17 10:06:36 2001 From: kiran.karkera at alopa.com (kk) Date: Thu, 17 May 2001 19:36:36 +0530 Subject: [Numpy-discussion] creation of array of objects Message-ID: <3B03DAEC.4060609@alopa.com> hi all.. I want to create an array of class objects of size say mXn. What would be the easiest way to do this? eg. class test: pass i want 5x10 array of test() instances. Then only way i know of doing this ,is by creating a=array(test()) resize(a,[5,10]) but this creates only references to the initial objects.Is there some way of creating fresh objects ? Also, if the __init__() call of the class takes some parameters, i should be able to enter values.. thanx in anticipation ciao From chrishbarker at home.net Thu May 17 18:33:54 2001 From: chrishbarker at home.net (Chris Barker) Date: Thu, 17 May 2001 15:33:54 -0700 Subject: [Numpy-discussion] transpose and putmask questions. References: Message-ID: <3B0451D2.98EFA9FD@home.net> Hi all, I recently tried to take a slice out of an array, transpose it, and then change it with the putmask function. It didn't work, because it ends up not being contiguous. This brings up two points: 1) why does the array passed as the first argument to putmask have to be contiguous? this can be a substantial limitation, I end up making copies when I have no other reason to. 2) the docs say that transpose "returns a new array..." This is confusing. I had assumed that "new array" meant a copy, when, in fact it is another array, but the data is a reference, much like a slice. We might want to make it a little more clear in the docs what that means. Another question I have is about how to match array shapes. I find I am often getting error when I try to do something (like putmask) with two arrays, one of shape (n,1) and one of shape (n,). It seems to me that what I want to do is unambiguous, so I should be able to do it. Is the reason I can't do this: a) The NumPy interface is designed to be unambiguous, and so it is important that the code never silently translate a rank one array into a rank two array where one of the dimensions is one (and vice-versa) or b) No one has gotten around to making the code smart enough to do this when it is unambigous.? -Thanks, -Chris -- Christopher Barker, Ph.D. ChrisHBarker at home.net --- --- --- http://members.home.net/barkerlohmann ---@@ -----@@ -----@@ ------@@@ ------@@@ ------@@@ Oil Spill Modeling ------ @ ------ @ ------ @ Water Resources Engineering ------- --------- -------- Coastal and Fluvial Hydrodynamics -------------------------------------- ------------------------------------------------------------------------ From chrishbarker at home.net Fri May 18 13:09:36 2001 From: chrishbarker at home.net (Chris Barker) Date: Fri, 18 May 2001 10:09:36 -0700 Subject: [Numpy-discussion] transpose and putmask questions. References: Message-ID: <3B055750.C40AFB04@home.net> "Paul F. Dubois" wrote: >> 1) why does the array passed as the first argument to putmask have to be >> contiguous? this can be a substantial limitation, I end up making copies >> when I have no other reason to. > -----> because nobody has implemented the general case; feel free to > volunteer. Fair enough. I have yet to open up the NumPy code myself, but I have written some custom extensions with it, and have a number of small improvements in mind that I would like to make. Hopefully I will get to this some day. > b) No one has gotten around to making the code smart enough to do this > when it is unambigous.? > ------> Numpy's attitude toward extra dimensions of length 1 is somewhat > schizophrenic. > If you know you want one-d you can put a ravel() around it. > > In general the answer to your question is (b) but it is very > complicated so that is why people don't work on it. I suspected as much. I'm starting to get used to it. I think my problem is that I come from MATLAB, where everything is a 2-d array. Thanks for your comments. -Chris -- Christopher Barker, Ph.D. ChrisHBarker at home.net --- --- --- http://members.home.net/barkerlohmann ---@@ -----@@ -----@@ ------@@@ ------@@@ ------@@@ Oil Spill Modeling ------ @ ------ @ ------ @ Water Resources Engineering ------- --------- -------- Coastal and Fluvial Hydrodynamics -------------------------------------- ------------------------------------------------------------------------ From focke at SLAC.Stanford.EDU Fri May 18 18:35:20 2001 From: focke at SLAC.Stanford.EDU (Warren Focke) Date: Fri, 18 May 2001 15:35:20 -0700 (PDT) Subject: [Numpy-discussion] comparison style Message-ID: Howdy all, We've got these spiffy comparisons now (thanks Travis et al.!): ] >>> 2 <= arange(8) ] array([0, 0, 1, 1, 1, 1, 1, 1]) but, as noted in the rich comparison PEP, they don't chain: ] >>> 2 <= arange(8) < 6 ] array([1, 1, 1, 1, 1, 1, 0, 0]) since x <= y < z is equivalent to x <= y and y < z and we don't get to overload "and" :-(. But we did overload "&" long ago, and bitwise_and is equivalent to logical_and if the arguments are only 1 or 0, which is what our comparisons return (I think?). So (x <= y) & (y < z) seems like it might be a semi-reasonable substitute, and is shorter and appeals more to my preferring-operators-over-functions eyes, than Numeric.logical_and(x <= y, y < z) (which still beats the ^%$#&^(*&*^ out of Numeric.logical_and(Numeric.less_equal(x, y), Numeric.less(y, z)) ). But I'm a bit squeamish about it. Cons that I see are: - Do our comparisons really and truly ALWAYS return 1 or 0, even with Infs and Nans and whatnot? - It's potentially misleading for readers of the code. - It just seems kinda wrong, even if it does look better. So, what think you? bitwise, bytefoolish-ly y'rs, Warren Focke From chrishbarker at home.net Mon May 21 12:55:13 2001 From: chrishbarker at home.net (Chris Barker) Date: Mon, 21 May 2001 09:55:13 -0700 Subject: [Numpy-discussion] comparison style References: Message-ID: <3B094871.9B86742D@home.net> Warren Focke wrote: > ] >>> 2 <= arange(8) > ] array([0, 0, 1, 1, 1, 1, 1, 1]) > > but, as noted in the rich comparison PEP, they don't chain: > x <= y and y < z > > and we don't get to overload "and" :-(. Clearly that would be nice. Why wasn't "and" allowed to be overloaded when the rest of rich comparisons was allowed? > (x <= y) & (y < z) > > seems like it might be a semi-reasonable substitute, and is shorter and > appeals more to my preferring-operators-over-functions eyes, than > > Numeric.logical_and(x <= y, y < z) It looks fine to me as well. > - Do our comparisons really and truly ALWAYS return 1 or 0, even with Infs > and Nans and whatnot? Good question. There was a lot of discussion here recently about what level of support NumPy should give to NaNs an Inf. Personally, I'd like to see a well integrated support for it, which would include comparisons (though bitwise could still be a problem). Of course, I have neither the time nor the knowledge to do it, so I'll just have to wait... There has also been talk of a "logical" array type. Is that going to happen? If so, it would be the obvious choice to be returned from a comparison and we could make sure the bitwise & and | work with it. I notice that currently (I'm running 2.0, so ignore me if this has changed) comparisons return an array of python Integers which seems a massive waste of space. > - It's potentially misleading for readers of the code. well, yes, unless we really do have a 1 bit logical array type. -Chris -- Christopher Barker, Ph.D. ChrisHBarker at home.net --- --- --- http://members.home.net/barkerlohmann ---@@ -----@@ -----@@ ------@@@ ------@@@ ------@@@ Oil Spill Modeling ------ @ ------ @ ------ @ Water Resources Engineering ------- --------- -------- Coastal and Fluvial Hydrodynamics -------------------------------------- ------------------------------------------------------------------------ From paul at pfdubois.com Mon May 21 16:54:23 2001 From: paul at pfdubois.com (Paul F. Dubois) Date: Mon, 21 May 2001 13:54:23 -0700 Subject: [Numpy-discussion] Climate Data Analysis Tools Version 3.0.0 Message-ID: The Program for Climate Model Diagnosis and Intercomparison is pleased to announce the release of Version 3.0.0 of Climate Data Analysis Tools, an open-source set of Python modules and tools for accessing, manipulating, and plotting scientific data in general and climate data sets in particular. The tools make extensive use of Numerical Python and its optional Masked Arrays (MA) package and are designed to interoperate with them. The CDAT website is http://cdat.sf.net, and the SourceForge project page is http://sf.net/projects/cdat. Version 3.0.0 is available for Linux and Solaris. Binary distributions are available for RedHat 6.2, RedHat 7.1, SuSe 7.1, and Solaris 5.6. The user must have NetCDF and Tcl/Tk installed before using CDAT. -- Paul Dubois PCMDI Lawrence Livermore National Laboratory From grumpycrocodile at yahoo.com Tue May 22 13:11:35 2001 From: grumpycrocodile at yahoo.com (Grant Callis) Date: Tue, 22 May 2001 10:11:35 -0700 (PDT) Subject: [Numpy-discussion] Win32 binary of ScientificPython Message-ID: <20010522171135.17109.qmail@web14104.mail.yahoo.com> Does have an updated Win32 binary of Konrad Hinsen's ScientificPython? I've been using Robert Kern's version, but it's only good up to Python 1.5.2. Now that the main modules I use have been compiled for Python 2.1 (dislin & mxTools) I have upgraded. The only other binary extension I use often is ScientificPython, especially the netcdf portion. I'm not really a computer person so compiling myself is a scary option. Any help? Thank you, Grant Callis __________________________________________________ Do You Yahoo!? Yahoo! Auctions - buy the things you want at great prices http://auctions.yahoo.com/ From schaffer at optonline.net Tue May 22 15:34:51 2001 From: schaffer at optonline.net (Les Schaffer) Date: Tue, 22 May 2001 15:34:51 -0400 Subject: [Numpy-discussion] Win32 binary of ScientificPython In-Reply-To: <20010522171135.17109.qmail@web14104.mail.yahoo.com> References: <20010522171135.17109.qmail@web14104.mail.yahoo.com> Message-ID: <15114.48987.390000.219072@gargle.gargle.HOWL> > The only other binary extension I use often is ScientificPython, > especially the netcdf portion. I'm not really a computer person so > compiling myself is a scary option. Any help? i have the scientific_netcdf module compiled for python 2.1. i can email to you or can put a small zip file up on my web site. you need the netcdf dll. i built that too, if needed. havent tried compiling the MPI stuff yet. les schaffer From grumpycrocodile at yahoo.com Tue May 22 18:53:57 2001 From: grumpycrocodile at yahoo.com (Grant Callis) Date: Tue, 22 May 2001 15:53:57 -0700 (PDT) Subject: [Numpy-discussion] Win32 binary of ScientificPython In-Reply-To: <15114.48987.390000.219072@gargle.gargle.HOWL> Message-ID: <20010522225357.81298.qmail@web14104.mail.yahoo.com> > i have the scientific_netcdf module compiled for > python 2.1. i can > email to you or can put a small zip file up on my > web site. > > you need the netcdf dll. i built that too, if > needed. Excellent news! E-mail is fine but I'm sure that others are interested in the netcdf module and the netcdf .dll as well. Perhaps posting to your web site would be better? Regardless, thank you very much. Grant Callis __________________________________________________ Do You Yahoo!? Yahoo! Auctions - buy the things you want at great prices http://auctions.yahoo.com/ From prem at engr.uky.edu Wed May 23 11:32:41 2001 From: prem at engr.uky.edu (Prem Rachakonda) Date: Wed, 23 May 2001 11:32:41 -0400 (EDT) Subject: [Numpy-discussion] Help on installilng NumPY Message-ID: Hi, I am trying to install NumPY on my Windows 2000 machine. Could you please list out the procedure I have to follow to complete the installation. Prem. ____________________________________________________________ Prem Rachakonda ____________________________________________________________ Mailing Address: Work Address: 700 Woodland Ave., #E4 Teradyne Connection Systems, Lexington, Kentucky-40508 44 Simon St., Mail Stop#006, PH:859-323-2880 Nashua, NewHampshire - 03060 Work Phone No : 603-879-3442 From schaffer at optonline.net Wed May 23 16:44:38 2001 From: schaffer at optonline.net (Les Schaffer) Date: Wed, 23 May 2001 16:44:38 -0400 Subject: [Numpy-discussion] Win32 binary of ScientificPython In-Reply-To: <20010522225357.81298.qmail@web14104.mail.yahoo.com> References: <15114.48987.390000.219072@gargle.gargle.HOWL> <20010522225357.81298.qmail@web14104.mail.yahoo.com> Message-ID: <15116.8502.953000.768869@gargle.gargle.HOWL> > Excellent news! E-mail is fine but I'm sure that others are > interested in the netcdf module and the netcdf .dll as well. Perhaps > posting to your web site would be better? i will put a package up on my site by twm. the URL will be: http://folks.astrian.net/godzilla there's no isue with me distributing the betcdf dll is there??? les schaffer From roitblat at hawaii.edu Wed May 23 16:57:56 2001 From: roitblat at hawaii.edu (Herbert L. Roitblat) Date: Wed, 23 May 2001 10:57:56 -1000 Subject: [Numpy-discussion] Seeking hypergeometric 2F1 algorithm Message-ID: <04d901c0e3cb$0aec52e0$8fd6afcf@pixi.com> Does anyone have an agorithm for computing the Gaussian hypergeometric function 2F1 ? I understand that in principle it should be easy (sum of a series), but I think that there is a more efficient way to calculate it. My algorithmic skills are not up to it. Thanks, Herb From markespl at yahoo.com Thu May 24 09:41:45 2001 From: markespl at yahoo.com (Mark Esplin) Date: Thu, 24 May 2001 09:41:45 -0400 Subject: [Numpy-discussion] Seeking hypergeometric 2F1 algorithm Message-ID: <01052409414500.00638@marke> The Gaussian hypergeometric function 2F1 is included with other special functions in the cephes-1.3 package. See http://pylab.sourceforge.net I haven't used the hypergeometric function, but the Bessel functions work well with Numerical Python. -Mark Esplin ---------- Forwarded Message ---------- Subject: [Numpy-discussion] Seeking hypergeometric 2F1 algorithm Date: Wed, 23 May 2001 10:57:56 -1000 From: "Herbert L. Roitblat" To: Does anyone have an agorithm for computing the Gaussian hypergeometric function 2F1 ? I understand that in principle it should be easy (sum of a series), but I think that there is a more efficient way to calculate it. My algorithmic skills are not up to it. Thanks, Herb _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net http://lists.sourceforge.net/lists/listinfo/numpy-discussion ------------------------------------------------------- _________________________________________________________ Do You Yahoo!? Get your free @yahoo.com address at http://mail.yahoo.com From schaffer at optonline.net Fri May 25 09:35:42 2001 From: schaffer at optonline.net (Les Schaffer) Date: Fri, 25 May 2001 09:35:42 -0400 Subject: [Numpy-discussion] Windows DLLs for NumPyish things In-Reply-To: <15116.8502.953000.768869@gargle.gargle.HOWL> References: <15114.48987.390000.219072@gargle.gargle.HOWL> <20010522225357.81298.qmail@web14104.mail.yahoo.com> <15116.8502.953000.768869@gargle.gargle.HOWL> Message-ID: <15118.24494.140000.639372@gargle.gargle.HOWL> Scientific_netcdf: ----------------- I have placed a link to a package of DLLs for Scientific module on my web site: http://folks.astrian.net/godzilla included are the netcdf DLL and lib file, a pointer to the Unidata site where MS VC++ Project files are available (i used them to build this DLL), and the scientific_netcdf.pyd. I tested the package using netcdf_demo.py from the Scientific Example's directory. The project files on the Unidata site are for version 3.5, wheras mine are for beta8 of the same version. if the latest one's on their site cause you grief, i'll update mine for their latest version and post to the site. Visual Python: ------------- For people interested in using Visual Python under Windows and Python 2.1, i have also placed the cvisual.dll file on my page. PIL: --- Sometime this weekend i should have the PIL packages on my page updated as well. I have it working for Python 2.1, just need to bundle it up. les schaffer From markespl at yahoo.com Fri May 25 12:09:08 2001 From: markespl at yahoo.com (Mark Esplin) Date: Fri, 25 May 2001 12:09:08 -0400 Subject: [Numpy-discussion] The future of Numpy Message-ID: <01052512090800.01375@marke> What is the plan for the future development of Numerical Python? For example, will additional special functions be included in numpy, or is NumPy to be a building block for other applications? I guess the big question is "Will Numerical Python become a complete computing environment like Matlab or IDL?" There is an interesting series of article on Numerical Python by Eric Hagemann on O'Reilly network (www.oreillynet.com) where Eric Hagemann is using NumPy with other python modules as a computing environment more than as a software development framework. I think NumPy could become a lot like the development of linux. Linux strictly speaking is only the operating system kernel. However, a linux system has come to includes a whole set of packages that work together to make up a whole system. -Mark Esplin _________________________________________________________ Do You Yahoo!? Get your free @yahoo.com address at http://mail.yahoo.com From Barrett at stsci.edu Fri May 25 15:15:30 2001 From: Barrett at stsci.edu (Paul Barrett) Date: Fri, 25 May 2001 15:15:30 -0400 Subject: [Numpy-discussion] The future of Numpy References: <01052512090800.01375@marke> Message-ID: <3B0EAF52.97846E89@STScI.Edu> Mark Esplin wrote: > > What is the plan for the future development of Numerical Python? For > example, will additional special functions be included in numpy, or is > NumPy to be a building block for other applications? I guess the big > question is "Will Numerical Python become a complete computing > environment like Matlab or IDL?" > > There is an interesting series of article on Numerical Python by Eric > Hagemann on O'Reilly network (www.oreillynet.com) where Eric Hagemann > is using NumPy with other python modules as a computing environment > more than as a software development framework. > > I think NumPy could become a lot like the development of linux. Linux > strictly speaking is only the operating system kernel. However, a > linux system has come to includes a whole set of packages that work > together to make up a whole system. See PEP 209: Multi-dimensional Arrays http://python.sourceforge.net/peps/pep-0209.html -- Paul Barrett, PhD Space Telescope Science Institute Phone: 410-338-4475 ESS/Science Software Group FAX: 410-338-4767 Baltimore, MD 21218 From acapnotic at users.sourceforge.net Mon May 28 00:28:08 2001 From: acapnotic at users.sourceforge.net (Kevin Turner) Date: Sun, 27 May 2001 21:28:08 -0700 Subject: [Numpy-discussion] fromfunction documentation Message-ID: <20010527212808.A2822@troglodyte.menefee> Quoting from numdoc.pdf, dated March 31, 2001, in section "Creating an array from a function: fromfunction()" > The implementation of fromfunction consists of: > > def fromfunction(function, dimensions): > return apply(function, tuple(indices(dimensions))) > > which means that the function function is called for each element in > the sequence indices(dimensions) [...] This is wholly incorrect. apply() does not call a function for each element in the sequence, apply calls a function exactly *once*, using the elements of the sequence for arguments. Also, that same section of the documentation refers to "the first example below [...] works" and "the second array [...] fails", but there is only one example, which apparently is the failing one (as it's named "buggy"). Having a *working* example would be nice. =) Perhaps this is a formatting glitch in the documentation processor, but I couldn't find the documentation's source format. and then... there's always the possibility that I'm barking up the wrong tree. What I'm trying to achieve is this: I've got a function which, given a vector describing a point in n-dimensional space, returns a value for that point. I want a matrix of the shape I specify, filled these values, with the array indicies as coordinates for the points. Is that too much to ask? ;) Thanks, - Kevin Turner [using Python version 2.0, PyNum 19.0.0] -- Kevin Turner | OpenPGP encryption welcome here http://www.purl.org/wiki/python/KevinTurner From acapnotic at users.sourceforge.net Mon May 28 00:42:43 2001 From: acapnotic at users.sourceforge.net (Kevin Turner) Date: Sun, 27 May 2001 21:42:43 -0700 Subject: [Numpy-discussion] fromfunction documentation In-Reply-To: <20010527212808.A2822@troglodyte.menefee>; from acapnotic@users.sourceforge.net on Sun, May 27, 2001 at 09:28:08PM -0700 References: <20010527212808.A2822@troglodyte.menefee> Message-ID: <20010527214243.A2914@troglodyte.menefee> On Sun, May 27, 2001 at 09:28:08PM -0700, Kevin Turner wrote: > and then... there's always the possibility that I'm barking up the > wrong tree. What I'm trying to achieve is this: I've got a function > which, given a vector describing a point in n-dimensional space, returns > a value for that point. I want a matrix of the shape I specify, filled > these values, with the array indicies as coordinates for the points. err, I meant "multiarray", not "matrix" > > Is that too much to ask? ;) > > Thanks, From paul at pfdubois.com Mon May 28 15:31:44 2001 From: paul at pfdubois.com (Paul F. Dubois) Date: Mon, 28 May 2001 12:31:44 -0700 Subject: [Numpy-discussion] fromfunction documentation In-Reply-To: <20010527212808.A2822@troglodyte.menefee> Message-ID: A previous report of this doc error had caused me to fix it in my copy but it hadn't made it up to the web yet. The section has been redone to make it clearer, I hope. I am in the process of doing the update now. The documentation source is not available to you. Conventional use of fromfunction is like this: >>> from Numeric import * >>> def f (x, y): return x**2 + y**2 >>> a=fromfunction(f, (3,2)) >>> a array([[0, 1], [1, 2], [4, 5]]) For your use, you need a "helper" function. Suppose g is your function. >>> def g(x): return x[0] + x[1]**2 >>> def helper (x, y): return g([x,y]) >>> from Numeric import * >>> a = fromfunction(helper, (3,3)) >>> a array([[0, 1, 4], [1, 2, 5], [2, 3, 6]]) Of course, you can use a lambda to avoid explicity creating the helper. >>> fromfunction(lambda x,y:g([x,y]), (3,3)) array([[0, 1, 4], [1, 2, 5], [2, 3, 6]]) >>> -----Original Message----- From: numpy-discussion-admin at lists.sourceforge.net [mailto:numpy-discussion-admin at lists.sourceforge.net]On Behalf Of Kevin Turner Sent: Sunday, May 27, 2001 9:28 PM To: numpy-discussion at lists.sourceforge.net Subject: [Numpy-discussion] fromfunction documentation Quoting from numdoc.pdf, dated March 31, 2001, in section "Creating an array from a function: fromfunction()" > The implementation of fromfunction consists of: > > def fromfunction(function, dimensions): > return apply(function, tuple(indices(dimensions))) > > which means that the function function is called for each element in > the sequence indices(dimensions) [...] This is wholly incorrect. apply() does not call a function for each element in the sequence, apply calls a function exactly *once*, using the elements of the sequence for arguments. Also, that same section of the documentation refers to "the first example below [...] works" and "the second array [...] fails", but there is only one example, which apparently is the failing one (as it's named "buggy"). Having a *working* example would be nice. =) Perhaps this is a formatting glitch in the documentation processor, but I couldn't find the documentation's source format. and then... there's always the possibility that I'm barking up the wrong tree. What I'm trying to achieve is this: I've got a function which, given a vector describing a point in n-dimensional space, returns a value for that point. I want a matrix of the shape I specify, filled these values, with the array indicies as coordinates for the points. Is that too much to ask? ;) Thanks, - Kevin Turner [using Python version 2.0, PyNum 19.0.0] -- Kevin Turner | OpenPGP encryption welcome here http://www.purl.org/wiki/python/KevinTurner _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net http://lists.sourceforge.net/lists/listinfo/numpy-discussion From perry at stsci.edu Tue May 29 11:14:33 2001 From: perry at stsci.edu (Perry Greenfield) Date: Tue, 29 May 2001 11:14:33 -0400 Subject: [Numpy-discussion] Re: The future of Numpy In-Reply-To: Message-ID: > Mark Esplin wrote: > > > > What is the plan for the future development of Numerical Python? For > > example, will additional special functions be included in numpy, or is > > NumPy to be a building block for other applications? I guess the big > > question is "Will Numerical Python become a complete computing > > environment like Matlab or IDL?" > > > > There is an interesting series of article on Numerical Python by Eric > > Hagemann on O'Reilly network (www.oreillynet.com) where Eric Hagemann > > is using NumPy with other python modules as a computing environment > > more than as a software development framework. > > > > I think NumPy could become a lot like the development of linux. Linux > > strictly speaking is only the operating system kernel. However, a > > linux system has come to includes a whole set of packages that work > > together to make up a whole system. > > See PEP 209: Multi-dimensional Arrays > > http://python.sourceforge.net/peps/pep-0209.html > > -- > Paul Barrett, PhD Space Telescope Science Institute > We at STScI have been working on a new implementation of Numeric. We are far enough along to release an incomplete version for evaluation by the Numeric (the current version) developers in about 2 weeks. The existing PEP (209) doesn't address all the details of the implementation and there are some differences from the PEP, but it does give a general idea of what our goals are. We have been more concerned at this point in implementation than keeping the PEP up to date. The PEP will be revised when we release a preliminary version. The preliminary version will be 0) Called something else. Probably numarray. 1) Not completely backwards compatible with the existing Numeric. (I don't have time to detail the differences, but they will be listed when we make it availble). 2) Incomplete. It will be missing functionality, especially with regard to structural operations (choose, where, nonzero, put, take, compress...). It will have most types (excepting complex) and most standard Ufuncs. We do not expect this version will be usuable for most projects because of this. We are looking for feedback on the interface and design approach rather than advertising it as a usuable version. We also are interested in collaborators in implementing the rest of the functionality. It probably will be moved to sourceforge soon after. 3) Slower for smaller arrays. We expect that performance for large arrays (>100,000 elements) should be fairly respectable, but since most of the original implementation is in Python, the overhead for smaller arrays is substantially increased over the existing Numeric. While we did benchmarking for determining what design approaches were efficient enough, we have eschewed benchmarks until we have completed all the important functionality (soon). At that point we will begin to work on optimization to improve the performance. I'll stop here, but you should be hearing more about it in the near future. Perry Greenfield Space Telescope Science Institute Science Software Group From karshi.hasanov at utoronto.ca Thu May 31 02:11:36 2001 From: karshi.hasanov at utoronto.ca (karshi.hasanov at utoronto.ca) Date: Thu, 31 May 2001 01:11:36 -0500 Subject: [Numpy-discussion] numpyio Message-ID: <3B15E098.4FBC0FDA@utoronto.ca> I am having a problem to download the numpyio.py file. Does any one knows whereand how to get it? Thanks. From edcjones at erols.com Thu May 31 13:46:07 2001 From: edcjones at erols.com (Edward C. Jones) Date: Thu, 31 May 2001 13:46:07 -0400 Subject: [Numpy-discussion] Array values sometimes have "" Message-ID: <3B16835F.7BE382CE@erols.com> I use Numeric 20.0.0, Python 2.1, and RedHat 7.1 on a PC. Here is a small Python program: import Numeric a = Numeric.zeros((2,2), 'b') print a[0,0], type(a[0,0]) print a[0][0], type(a[0][0]) a = Numeric.zeros((2,2), 'i') print a[0,0], type(a[0,0]) print a[0][0], type(a[0][0]) a = Numeric.zeros((2,2), 'f') print a[0,0], type(a[0,0]) print a[0][0], type(a[0][0]) The output is: 0 0 0 0 0.0 0.0 This can't be correct. "a[0,0]" and "a[0][0]" should have the same type. A quick check of my old Python code shows that this problem may have appeared since Numeric version 17. Thanks, Ed Jones From jbaddor at physics.mcgill.ca Thu May 31 18:59:54 2001 From: jbaddor at physics.mcgill.ca (Jean-Bernard Addor) Date: Thu, 31 May 2001 18:59:54 -0400 (EDT) Subject: [Numpy-discussion] underflow array([2.9e-131])**3 Message-ID: Hey Numpy people! Do anyone know how to disable underflow exception errors in Numeric? I have a lot of these in my code. It is now a very important problem in my calculations. The only solution I see is to make a for loop and make the arithmetic in python instead of Numeric. Thanks for your help, Jean-Bernard Python 2.0 (#9, Feb 2 2001, 12:17:02) [GCC 2.95.2 19991024 (release)] on linux2 Type "copyright", "credits" or "license" for more information. Hello from .pythonrc.py >>> import Numeric >>> Numeric.__version__ '17.2.0' >>> Numeric.array([2.9e-131])**3 Traceback (most recent call last): File "", line 1, in ? OverflowError: math range error >>> 2.9e-131**3 0.0 From jbaddor at physics.mcgill.ca Thu May 31 19:09:03 2001 From: jbaddor at physics.mcgill.ca (Jean-Bernard Addor) Date: Thu, 31 May 2001 19:09:03 -0400 (EDT) Subject: [Numpy-discussion] PS: underflow array([2.9e-131+0j])**3 (why complex give a different result? bug?) Message-ID: Hey Numpy people! This a PS from my previous message. I just tried the underflow with complex : >>> Numeric.array([2.9e-131])**3 Traceback (most recent call last): File "", line 1, in ? OverflowError: math range error >>> 2.9e-131**3 0.0 >>> Numeric.array([2.9e-131+0j])**3 array([ 0.+0.j]) Now I have a quick solution for my problem, but I have the impression that this is a bug, at least I don't understand the underlying logic. If somebody can explain it to me? Jean-Bernard