From james at hello.com Fri May 2 04:07:02 2003 From: james at hello.com (james at hello.com) Date: Fri May 2 04:07:02 2003 Subject: [Numpy-discussion] Italian-crafted Rolex - only $65 - $140!! Free SHIPPING ! ! Message-ID: An HTML attachment was scrubbed... URL: From mtsbra003 at mail.uct.ac.za Fri May 2 08:25:08 2003 From: mtsbra003 at mail.uct.ac.za (mtsbra003 at mail.uct.ac.za) Date: Fri May 2 08:25:08 2003 Subject: [Numpy-discussion] numpy problem In-Reply-To: References: Message-ID: <1051888620.3eb28beccd666@intranet.uct.ac.za> Hello I have installed python,numpy(latest),and numtut. So when testing the package could see the image that was supposed to come out. Per instruction I had to do the following: 1. from NumTut import* , 2. view(greece). But view brought nothing. Thanking you in advance Thats Bramley ------------------------------------------------- This mail sent through IMP: http://horde.org/imp/ From jmiller at stsci.edu Mon May 5 05:12:07 2003 From: jmiller at stsci.edu (Todd Miller) Date: Mon May 5 05:12:07 2003 Subject: [Numpy-discussion] Numarray-0.5 released Message-ID: <1052136718.23013.4.camel@halloween.stsci.edu> Release Notes for numarray-0.5 Numarray is an array processing package designed to efficiently manipulate large multi-dimensional arrays. Numarray is modelled after Numeric and features c-code generated from python template scripts, the capacity to operate directly on arrays in files, and improved type promotions. I. ENHANCEMENTS 1. Universal Function Overhead Reduction The constant time overhead for most universal functions has been reduced by a factor of 10-20. This results in better performance for small arrays. 2. FFT mode and IRAF boundary modes added to Convolve.convolve2d There's now an FFT switch for the 2d convolution function in the Convolve package; when set to 1, convolution is performed via the FFT rather than using the naiive algorithm. In addition, convolve2d now supports boundary modes which are identical to IRAF's convolution function. 3. Numarray is now supported by f2py Numarray numerical arrays can now be used with f2py wrappers for Fortran code. To compile f2py wrapped extensions for numarray, use the switch -DNUMARRAY on the f2py command line. Support is currently limited to f77 and numerical arrays. II. BUGS FIXED 650926 exotic type coercions 653424 Convolve.boxcar boundary bug 653429 python setup.py build 654669 array index by list 655942 logical operator reductions 657058 inverse real fft bug 677796 byteswap not working 709956 error summing over large boolean arrays 710480 MLab.median makes unnecessary msort call 714537 conjugate function changes its argument See http://sourceforge.net/tracker/?atid=450446&group_id=1369&func=browse for more details. III. CAUTIONS 1. Due to some module renamings, numarray-0.5 will not install correctly on top of an existing numarray installation. Before installing numarray-0.5 remove your old version of numarray. 2. Due to reorganization of the C-API, numarray extensions must be recompiled. 3. For numarray-0.5 and up, the byteswap related methods have been redefined: a.byteswap() swaps but leaves byteorder alone a.togglebyteorder() Does "big" <-> "little" a._byteswap() now an alias for byteswap a._togglebyteorder() deleted 4. round() has been deprecated. Use around() instead. 5. Installing from source, the first time you run setup.py, you must specify --gencode, e.g.: python setup.py install --gencode WHERE ----------- Numarray-0.5 windows executable installers, source code, and manual is here: http://sourceforge.net/project/showfiles.php?group_id=1369 Numarray is hosted by Source Forge in the same project which hosts Numeric: http://sourceforge.net/projects/numpy/ The web page for Numarray information is at: http://stsdas.stsci.edu/numarray/index.html Trackers for Numarray Bugs, Feature Requests, Support, and Patches are at the Source Forge project for NumPy at: http://sourceforge.net/tracker/?group_id=1369 REQUIREMENTS ------------------------------ numarray-0.5 requires Python 2.2.2 or greater. AUTHORS, LICENSE ------------------------------ Numarray was written by Perry Greenfield, Rick White, Todd Miller, JC Hsu, Paul Barrett, Phil Hodge at the Space Telescope Science Institute. Thanks go to Jochen Kupper of the University of North Carolina for his work on Numarray and for porting the Numarray manual to TeX format. Thanks also to Edward C. Jones, Francesc Alted, Paul Dubois, Eric Jones, Travis Oliphant, Pearu Peterson and everyone who has contributed with comments and feedback. Numarray is made available under a BSD-style License. See LICENSE.txt in the source distribution for details. -- Todd Miller jmiller at stsci.edu STSCI / ESS / SSB From haase at msg.ucsf.edu Mon May 5 09:36:06 2003 From: haase at msg.ucsf.edu (Sebastian Haase) Date: Mon May 5 09:36:06 2003 Subject: [Numpy-discussion] http://stsdas.stsci.edu/numarray outdated References: <1052136718.23013.4.camel@halloween.stsci.edu> Message-ID: <018701c31324$7c7293e0$421ee6a9@rodan> Hi all, I'm happy to hear about the new release ! Could someone put this information on the web ? http://stsdas.stsci.edu/numarray/ still dates to 2002 Aug (which is before 0.4 !!) Or is this page the wrong page anyway -- I just end up there because google points into that direction when you ask for 'numarray' Thanks for the great work. Sebastian ----- Original Message ----- From: "Todd Miller" To: Sent: Monday, May 05, 2003 5:11 AM Subject: [Numpy-discussion] Numarray-0.5 released > Release Notes for numarray-0.5 > > Numarray is an array processing package designed to efficiently > manipulate large multi-dimensional arrays. Numarray is modelled after > Numeric and features c-code generated from python template scripts, > the capacity to operate directly on arrays in files, and improved type > promotions. > > I. ENHANCEMENTS > > 1. Universal Function Overhead Reduction > > The constant time overhead for most universal functions has been reduced > by a factor of 10-20. This results in better performance for small > arrays. > > 2. FFT mode and IRAF boundary modes added to Convolve.convolve2d > > There's now an FFT switch for the 2d convolution function in the > Convolve package; when set to 1, convolution is performed via the FFT > rather than using the naiive algorithm. In addition, convolve2d now > supports boundary modes which are identical to IRAF's convolution > function. > > 3. Numarray is now supported by f2py > > Numarray numerical arrays can now be used with f2py wrappers for > Fortran code. To compile f2py wrapped extensions for numarray, > use the switch -DNUMARRAY on the f2py command line. Support > is currently limited to f77 and numerical arrays. > > > II. BUGS FIXED > > 650926 exotic type coercions > 653424 Convolve.boxcar boundary bug > 653429 python setup.py build > 654669 array index by list > 655942 logical operator reductions > 657058 inverse real fft bug > 677796 byteswap not working > 709956 error summing over large boolean arrays > 710480 MLab.median makes unnecessary msort call > 714537 conjugate function changes its argument > > See > http://sourceforge.net/tracker/?atid=450446&group_id=1369&func=browse > for more details. > > III. CAUTIONS > > 1. Due to some module renamings, numarray-0.5 will not install > correctly on top of an existing numarray installation. Before > installing numarray-0.5 remove your old version of numarray. > > 2. Due to reorganization of the C-API, numarray extensions must be > recompiled. > > 3. For numarray-0.5 and up, the byteswap related methods have been > redefined: > > a.byteswap() swaps but leaves byteorder alone > a.togglebyteorder() Does "big" <-> "little" > a._byteswap() now an alias for byteswap > a._togglebyteorder() deleted > > 4. round() has been deprecated. Use around() instead. > > 5. Installing from source, the first time you run setup.py, you must > specify --gencode, e.g.: python setup.py install --gencode > > WHERE > ----------- > > Numarray-0.5 windows executable installers, source code, and manual is > here: > > http://sourceforge.net/project/showfiles.php?group_id=1369 > > Numarray is hosted by Source Forge in the same project which hosts > Numeric: > > http://sourceforge.net/projects/numpy/ > > The web page for Numarray information is at: > > http://stsdas.stsci.edu/numarray/index.html > > Trackers for Numarray Bugs, Feature Requests, Support, and Patches are > at > the Source Forge project for NumPy at: > > http://sourceforge.net/tracker/?group_id=1369 > > REQUIREMENTS > ------------------------------ > > numarray-0.5 requires Python 2.2.2 or greater. > > > AUTHORS, LICENSE > ------------------------------ > > Numarray was written by Perry Greenfield, Rick White, Todd Miller, JC > Hsu, Paul Barrett, Phil Hodge at the Space Telescope Science > Institute. Thanks go to Jochen Kupper of the University of North > Carolina for his work on Numarray and for porting the Numarray manual > to TeX format. Thanks also to Edward C. Jones, Francesc Alted, > Paul Dubois, Eric Jones, Travis Oliphant, Pearu Peterson and everyone > who has contributed with comments and feedback. > > Numarray is made available under a BSD-style License. See > LICENSE.txt in the source distribution for details. > > -- > Todd Miller jmiller at stsci.edu > STSCI / ESS / SSB > > > > ------------------------------------------------------- > This sf.net email is sponsored by:ThinkGeek > Welcome to geek heaven. > http://thinkgeek.com/sf > _______________________________________________ > Numpy-discussion mailing list > Numpy-discussion at lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/numpy-discussion > From perry at stsci.edu Mon May 5 10:23:12 2003 From: perry at stsci.edu (Perry Greenfield) Date: Mon May 5 10:23:12 2003 Subject: [Numpy-discussion] [numarray issue] nonzero? Comments on new name solicited In-Reply-To: <018701c31324$7c7293e0$421ee6a9@rodan> Message-ID: [In the next week or so I am going to be raising some issues regarding the next numarray release, 0.6. By the way, we keep the version number less than 1.0 because we still hold out the possiblity of changing some interface issues (generally it has been in the direction of making numarray more compatible with Numeric). The following raises one of these interface issues. numarray has a more general approach to the nonzero function than does Numeric. It is able to return multiple index arrays for multidimensional arrays (where as nonzero only works for 1-d arrays). In the interest of giving a more generic interface for the function, numarray always returns the index array(s) as part of a tuple, even if the argument to nonzero is 1-D and there is only one resulting index array. However, this is a problem for Numeric old code that uses nonzero and expects an array instead of a tuple. The sensible thing to do is to retain compatibilty with the Numeric nonzero (and likewise only support 1-d arrays). So we need a new name for our new nonzero function. To tell you the truth, I always felt that nonzero was a terrible name anyway and much preferred the name that IDL used for that function, namely "where". Alas, "where" is already used for another purpose in Numeric. But to illustrate, which is clearer? x[nonzero(x > 0.)] = 0. or x[where(x > 0.)] = 0 So, we considered some other alternatives, in particular nz wheretrue but our favorite is: "at". For example: x[at(x > 0.)] = 0. This has the benefit of being short and having a fairly clear meaning. Short is important since it will keep expressions of arrays using index arrays short and more readable. True, this is more likely to conflict with existing user variable or function names, but if one is simply trying to use existing files, then there is no problem since they won't need the "at" function since its functionality was not previously available. Any objections to using "at"? Any better suggestions. Of course, these changes will break existing numarray code that use the nonzero function. Perry Greenfield From pearu at cens.ioc.ee Mon May 5 13:27:07 2003 From: pearu at cens.ioc.ee (Pearu Peterson) Date: Mon May 5 13:27:07 2003 Subject: [Numpy-discussion] Re: [SciPy-user] [numarray issue] nonzero? Comments on new name solicited In-Reply-To: Message-ID: On Mon, 5 May 2003, Perry Greenfield wrote: > Any objections to using "at"? Any better suggestions. For me 'wheretrue' sounds most accurate but I agree, it's longish. Its short form 'where' is less accurate (because of its general meaning) but not too confusing. At first, 'at' confused me a little... until I figured out its long version 'trueat'. Hmm, I have always assumed that 'nonzero(x)' means elementwise 'not not x' (like Python __nonzero__ is equivalent to 'not not') and therefore never felt the need to use it. But now I see I was wrong! So, 'nonzero' sounded and still sounds misleading for me. For 'nz' I am -1 because 'nz' can often be a suitable variable name. In summary, I would like 'trueat' the most and 'nonzero' the least. 'where' and 'at' (in that order) are somewhere in between. Pearu From haase at msg.ucsf.edu Mon May 5 18:25:02 2003 From: haase at msg.ucsf.edu (Sebastian Haase) Date: Mon May 5 18:25:02 2003 Subject: [Numpy-discussion] numarray 0.5 -> python setup.py build fails References: <1052136718.23013.4.camel@halloween.stsci.edu> Message-ID: <003101c3136e$3d7d2d90$421ee6a9@rodan> Hi, I just tried this : haase at baboon:~/numarray-0.5: python2.2 setup.py build --gencode creating /usr/lib/python2.2/site-packages/numarray error: could not create '/usr/lib/python2.2/site-packages/numarray': Permission denied I guess this is the same bug I ran into a couple month ago - I already now a workaround ... - Sebastian ----- Original Message ----- From: "Todd Miller" To: Sent: Monday, May 05, 2003 5:11 AM Subject: [Numpy-discussion] Numarray-0.5 released > Release Notes for numarray-0.5 > > Numarray is an array processing package designed to efficiently > manipulate large multi-dimensional arrays. Numarray is modelled after > Numeric and features c-code generated from python template scripts, > the capacity to operate directly on arrays in files, and improved type > promotions. > > I. ENHANCEMENTS > > 1. Universal Function Overhead Reduction > > The constant time overhead for most universal functions has been reduced > by a factor of 10-20. This results in better performance for small > arrays. > > 2. FFT mode and IRAF boundary modes added to Convolve.convolve2d > > There's now an FFT switch for the 2d convolution function in the > Convolve package; when set to 1, convolution is performed via the FFT > rather than using the naiive algorithm. In addition, convolve2d now > supports boundary modes which are identical to IRAF's convolution > function. > > 3. Numarray is now supported by f2py > > Numarray numerical arrays can now be used with f2py wrappers for > Fortran code. To compile f2py wrapped extensions for numarray, > use the switch -DNUMARRAY on the f2py command line. Support > is currently limited to f77 and numerical arrays. > > > II. BUGS FIXED > > 650926 exotic type coercions > 653424 Convolve.boxcar boundary bug > 653429 python setup.py build > 654669 array index by list > 655942 logical operator reductions > 657058 inverse real fft bug > 677796 byteswap not working > 709956 error summing over large boolean arrays > 710480 MLab.median makes unnecessary msort call > 714537 conjugate function changes its argument > > See > http://sourceforge.net/tracker/?atid=450446&group_id=1369&func=browse > for more details. > > III. CAUTIONS > > 1. Due to some module renamings, numarray-0.5 will not install > correctly on top of an existing numarray installation. Before > installing numarray-0.5 remove your old version of numarray. > > 2. Due to reorganization of the C-API, numarray extensions must be > recompiled. > > 3. For numarray-0.5 and up, the byteswap related methods have been > redefined: > > a.byteswap() swaps but leaves byteorder alone > a.togglebyteorder() Does "big" <-> "little" > a._byteswap() now an alias for byteswap > a._togglebyteorder() deleted > > 4. round() has been deprecated. Use around() instead. > > 5. Installing from source, the first time you run setup.py, you must > specify --gencode, e.g.: python setup.py install --gencode > > WHERE > ----------- > > Numarray-0.5 windows executable installers, source code, and manual is > here: > > http://sourceforge.net/project/showfiles.php?group_id=1369 > > Numarray is hosted by Source Forge in the same project which hosts > Numeric: > > http://sourceforge.net/projects/numpy/ > > The web page for Numarray information is at: > > http://stsdas.stsci.edu/numarray/index.html > > Trackers for Numarray Bugs, Feature Requests, Support, and Patches are > at > the Source Forge project for NumPy at: > > http://sourceforge.net/tracker/?group_id=1369 > > REQUIREMENTS > ------------------------------ > > numarray-0.5 requires Python 2.2.2 or greater. > > > AUTHORS, LICENSE > ------------------------------ > > Numarray was written by Perry Greenfield, Rick White, Todd Miller, JC > Hsu, Paul Barrett, Phil Hodge at the Space Telescope Science > Institute. Thanks go to Jochen Kupper of the University of North > Carolina for his work on Numarray and for porting the Numarray manual > to TeX format. Thanks also to Edward C. Jones, Francesc Alted, > Paul Dubois, Eric Jones, Travis Oliphant, Pearu Peterson and everyone > who has contributed with comments and feedback. > > Numarray is made available under a BSD-style License. See > LICENSE.txt in the source distribution for details. > > -- > Todd Miller jmiller at stsci.edu > STSCI / ESS / SSB > > > > ------------------------------------------------------- > This sf.net email is sponsored by:ThinkGeek > Welcome to geek heaven. > http://thinkgeek.com/sf > _______________________________________________ > Numpy-discussion mailing list > Numpy-discussion at lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/numpy-discussion > From hinsen at cnrs-orleans.fr Tue May 6 03:27:04 2003 From: hinsen at cnrs-orleans.fr (Konrad Hinsen) Date: Tue May 6 03:27:04 2003 Subject: [Numpy-discussion] Python global interpreter lock ... In-Reply-To: References: Message-ID: <200305061227.13910.hinsen@cnrs-orleans.fr> On Wednesday 30 April 2003 08:47, Andrew P. Lentvorski, Jr. wrote: > 1) Does numpy (or numarray) release the Python GIL? No. > 2) If not, why not? Is this simply something I could fix by making the > changes myself, or are there larger reasons for not releasing the GIL? There is one good reason, which is that the GIL release code doesn't nest. If NumPy released the GIL during its operations, then any C code that uses NumPy functions would have to reacquire the GIL before calling NumPy, then re-release the GIL afterwards. In most real-life situations, it is simpler and presumably more efficient to release the GIL once for a block of numerical computations, which would include many calls to NumPy functions. One could of course release the GIL not within the functions that are part of the NumPy C API, but in a thin wrapper around it that is directly called from Python. The only other problem that I see is the general object type arrays, which make calls back into the Python interpreter. Konrad. -- ------------------------------------------------------------------------------- Konrad Hinsen | E-Mail: hinsen at cnrs-orleans.fr Centre de Biophysique Moleculaire (CNRS) | Tel.: +33-2.38.25.56.24 Rue Charles Sadron | Fax: +33-2.38.63.15.17 45071 Orleans Cedex 2 | Deutsch/Esperanto/English/ France | Nederlands/Francais ------------------------------------------------------------------------------- From verveer at embl-heidelberg.de Tue May 6 05:25:07 2003 From: verveer at embl-heidelberg.de (Peter Verveer) Date: Tue May 6 05:25:07 2003 Subject: [Numpy-discussion] Numarray 0.5 In-Reply-To: References: Message-ID: <200305061424.26714.verveer@embl-heidelberg.de> Hi All, I found the following problems after testing my software with numarray 0.5: 1) Following works fine if both a and b are arrays: >>> a = array([2]) >>> b = array([1, 2]) >>> print a + b [3 4] However, if b is an python sequence: >>> a = array([2]) >>> b = [1, 2] >>> print a + b [3] Apparently broadcasting does not work with python sequeces anymore. This used to work fine in version 0.4. Is this a bug? 2) It is not possible to compare an array type object to the 'Any' object: >>> Float64 == Any Traceback (most recent call last): File "", line 1, in ? File "/usr/local/lib/python2.2/site-packages/numarray/numerictypes.py", line 112, in __cmp__ return (genericTypeRank.index(self.name) - ValueError: list.index(x): x not in list I am not sure if this is a bug, or intended behaviour, but the possibilty to compare an array type object to 'Any' would be very useful for me. 3) The NA_typeNoToTypeObject() function fails if it is called before any arrays are created. It looks to me as if the pNumType array in libnumarraymodule.c is not initialized until an array is created. I don't know if any other functions are affected in the same way. Could this be fixed? Cheers, Peter -- Dr. Peter J. Verveer Cell Biology and Cell Biophysics Programme EMBL Meyerhofstrasse 1 D-69117 Heidelberg Germany Tel. : +49 6221 387245 Fax : +49 6221 387242 Email: Peter.Verveer at embl-heidelberg.de From verveer at embl-heidelberg.de Tue May 6 05:52:13 2003 From: verveer at embl-heidelberg.de (Peter Verveer) Date: Tue May 6 05:52:13 2003 Subject: [Numpy-discussion] Question about numarray Bool type Message-ID: <200305061451.45361.verveer@embl-heidelberg.de> Hi, In the numarray source code Bool is defined as a typedef to char. Does the Bool array type correspond to the Int8, or the UInt8 type? If I recall correctly, the char type may be signed or unsigned. Can I make any assumptions about the C type used to implement Bool at all? I guess at the Python level it does not matter what actual C type is used to implement bool arrays as it only represents boolean values. However, at the level of C extensions I would like to deal with Bool arrays by calling existing functions written for the appropiate C type. That would be somewhat simpler if Bool is always garantueed to be either a unsigned or a signed char type. Cheers, Peter. -- Dr. Peter J. Verveer Cell Biology and Cell Biophysics Programme EMBL Meyerhofstrasse 1 D-69117 Heidelberg Germany Tel. : +49 6221 387245 Fax : +49 6221 387242 Email: Peter.Verveer at embl-heidelberg.de From falted at openlc.org Tue May 6 07:21:10 2003 From: falted at openlc.org (Francesc Alted) Date: Tue May 6 07:21:10 2003 Subject: [Numpy-discussion] numarray 0.5 -> python setup.py build fails In-Reply-To: <003101c3136e$3d7d2d90$421ee6a9@rodan> References: <1052136718.23013.4.camel@halloween.stsci.edu> <003101c3136e$3d7d2d90$421ee6a9@rodan> Message-ID: <200305061619.54534.falted@openlc.org> I guess this is a problem with permissions. It seems like if setup.py is trying to install numarray on the Python site-packages directory. It should not, but meanwhile, run this command as root and it should works. Ah!, do not forget to delete (or better, move) the old site-packages/numarray in order to do a problem-free fresh installation. Remember: > > 1. Due to some module renamings, numarray-0.5 will not install > > correctly on top of an existing numarray installation. Before > > installing numarray-0.5 remove your old version of numarray. Cheers, A Dimarts 06 Maig 2003 03:24, Sebastian Haase va escriure: > Hi, > I just tried this : > haase at baboon:~/numarray-0.5: python2.2 setup.py build --gencode > > > creating /usr/lib/python2.2/site-packages/numarray > error: could not create '/usr/lib/python2.2/site-packages/numarray': > Permission denied > > I guess this is the same bug I ran into a couple month ago - I already now > a workaround ... > > - Sebastian > -- Francesc Alted From jmiller at stsci.edu Tue May 6 08:49:03 2003 From: jmiller at stsci.edu (Todd Miller) Date: Tue May 6 08:49:03 2003 Subject: [Numpy-discussion] Question about numarray Bool type References: <200305061451.45361.verveer@embl-heidelberg.de> Message-ID: <3EB7D96B.6080201@stsci.edu> Peter Verveer wrote: >Hi, > >In the numarray source code Bool is defined as a typedef to char. Does the >Bool array type correspond to the Int8, or the UInt8 type? If I recall >correctly, the char type may be signed or unsigned. Can I make any >assumptions about the C type used to implement Bool at all? > Yes! It's named Bool. :) >I guess at the Python level it does not matter what actual C type is used to >implement bool arrays as it only represents boolean values. However, at the >level of C extensions I would like to deal with Bool arrays by calling >existing functions written for the appropiate C type. > The easiest way now is to just declare your variables with type "Bool". The only thing I can think of which that doesn't cover is printf/scanf. Are there others? >That would be somewhat >simpler if Bool is always garantueed to be either a unsigned or a signed char >type. > I think perhaps we should just re-define Bool as signed char. Any other opinions? > >Cheers, Peter. > > > Todd From haase at msg.ucsf.edu Tue May 6 09:06:09 2003 From: haase at msg.ucsf.edu (Sebastian Haase) Date: Tue May 6 09:06:09 2003 Subject: [Numpy-discussion] numarray 0.5 -> python setup.py build fails References: <1052136718.23013.4.camel@halloween.stsci.edu> <003101c3136e$3d7d2d90$421ee6a9@rodan> <200305061619.54534.falted@openlc.org> Message-ID: <004d01c313e9$65d65980$421ee6a9@rodan> The reason that I always try "setup.py BUILD" first is that I don't have root permissions on that machine - so this is a quite important "test" for me. Thanks, Sebastian ----- Original Message ----- From: "Francesc Alted" To: "Sebastian Haase" ; Sent: Tuesday, May 06, 2003 7:19 AM Subject: Re: [Numpy-discussion] numarray 0.5 -> python setup.py build fails I guess this is a problem with permissions. It seems like if setup.py is trying to install numarray on the Python site-packages directory. It should not, but meanwhile, run this command as root and it should works. Ah!, do not forget to delete (or better, move) the old site-packages/numarray in order to do a problem-free fresh installation. Remember: > > 1. Due to some module renamings, numarray-0.5 will not install > > correctly on top of an existing numarray installation. Before > > installing numarray-0.5 remove your old version of numarray. Cheers, A Dimarts 06 Maig 2003 03:24, Sebastian Haase va escriure: > Hi, > I just tried this : > haase at baboon:~/numarray-0.5: python2.2 setup.py build --gencode > > > creating /usr/lib/python2.2/site-packages/numarray > error: could not create '/usr/lib/python2.2/site-packages/numarray': > Permission denied > > I guess this is the same bug I ran into a couple month ago - I already now > a workaround ... > > - Sebastian > -- Francesc Alted ------------------------------------------------------- Enterprise Linux Forum Conference & Expo, June 4-6, 2003, Santa Clara The only event dedicated to issues related to Linux enterprise solutions www.enterpriselinuxforum.com _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From verveer at embl-heidelberg.de Tue May 6 09:18:02 2003 From: verveer at embl-heidelberg.de (Peter Verveer) Date: Tue May 6 09:18:02 2003 Subject: [Numpy-discussion] Question about numarray Bool type In-Reply-To: <3EB7D96B.6080201@stsci.edu> References: <200305061451.45361.verveer@embl-heidelberg.de> <3EB7D96B.6080201@stsci.edu> Message-ID: <200305061800.18923.verveer@embl-heidelberg.de> On Tuesday 06 May 2003 17:48, Todd Miller wrote: > Peter Verveer wrote: > >Hi, > > > >In the numarray source code Bool is defined as a typedef to char. Does the > >Bool array type correspond to the Int8, or the UInt8 type? If I recall > >correctly, the char type may be signed or unsigned. Can I make any > >assumptions about the C type used to implement Bool at all? > > Yes! It's named Bool. :) Obviously, but then I need to include the numarray include file to make sure I get the right definition, and I do not always want to do that. See below. > >I guess at the Python level it does not matter what actual C type is used > > to implement bool arrays as it only represents boolean values. However, > > at the level of C extensions I would like to deal with Bool arrays by > > calling existing functions written for the appropiate C type. > > The easiest way now is to just declare your variables with type "Bool". > The only thing I can think of which that doesn't cover is printf/scanf. > Are there others? This does not work if you want to call a routine that was not written using the numarray typedefs. For instance, I like to write my array processing routines such that I can use them in a standalone C program without linking to numarray. These routines know nothing about numarray, but as long as the layout and data types of the arrays are correct I can still use them in numarray C routines. Then you need to know of course exactly what the C data types of the numarray routines are. That is easy for most types since they are exactly defined by bit-size and sign. But this is currently not true for the Bool type. > >That would be somewhat > >simpler if Bool is always garantueed to be either a unsigned or a signed > > char type. > > I think perhaps we should just re-define Bool as signed char. Any other > opinions? That would solve the problem. Peter From falted at openlc.org Tue May 6 09:42:18 2003 From: falted at openlc.org (Francesc Alted) Date: Tue May 6 09:42:18 2003 Subject: [Numpy-discussion] Question about numarray Bool type In-Reply-To: <200305061800.18923.verveer@embl-heidelberg.de> References: <200305061451.45361.verveer@embl-heidelberg.de> <3EB7D96B.6080201@stsci.edu> <200305061800.18923.verveer@embl-heidelberg.de> Message-ID: <1052237809.3eb7dff188366@webmail.imk.es> Mensaje citado por: Peter Verveer : > > >That would be somewhat > > >simpler if Bool is always garantueed to be either a unsigned or a > signed > > > char type. > > > > I think perhaps we should just re-define Bool as signed char. Any > other > > opinions? > > That would solve the problem. I'm in the same position than Peter. I would definitively prefer a standard C type for booleans. Francesc Alted From jmiller at stsci.edu Tue May 6 09:46:15 2003 From: jmiller at stsci.edu (Todd Miller) Date: Tue May 6 09:46:15 2003 Subject: [Numpy-discussion] numarray 0.5 -> python setup.py build fails References: <1052136718.23013.4.camel@halloween.stsci.edu> <003101c3136e$3d7d2d90$421ee6a9@rodan> Message-ID: <3EB7E6E9.5090800@stsci.edu> Sebastian Haase wrote: >Hi, >I just tried this : >haase at baboon:~/numarray-0.5: python2.2 setup.py build --gencode > > >creating /usr/lib/python2.2/site-packages/numarray >error: could not create '/usr/lib/python2.2/site-packages/numarray': >Permission denied > >I guess this is the same bug I ran into a couple month ago - I already now a >workaround ... > >- Sebastian > > Here's a couple work arounds I use: 1) When trying to install to /usr/lib, I first "su root". 2) When I don't like to or can't "su root", I first install Python in my home directory using: "./configure --prefix=$HOME". With a local Python, site-packages is also local and writeable by default. That said, setup.py has some kludges in it that need to be eliminated or factored out. Todd From jmiller at stsci.edu Tue May 6 11:26:07 2003 From: jmiller at stsci.edu (Todd Miller) Date: Tue May 6 11:26:07 2003 Subject: [Numpy-discussion] numarray 0.5 -> python setup.py build fails References: <1052136718.23013.4.camel@halloween.stsci.edu> <003101c3136e$3d7d2d90$421ee6a9@rodan> <3EB7E6E9.5090800@stsci.edu> Message-ID: <3EB7FE4E.4080209@stsci.edu> Todd Miller wrote: > Sebastian Haase wrote: > >> Hi, >> I just tried this : >> haase at baboon:~/numarray-0.5: python2.2 setup.py build --gencode >> >> >> creating /usr/lib/python2.2/site-packages/numarray >> error: could not create '/usr/lib/python2.2/site-packages/numarray': >> Permission denied >> >> I guess this is the same bug I ran into a couple month ago - I >> already now a >> workaround ... >> >> - Sebastian >> >> > Here's a couple work arounds I use: > > 1) When trying to install to /usr/lib, I first "su root". > > 2) When I don't like to or can't "su root", I first install Python in > my home directory using: "./configure --prefix=$HOME". With a local > Python, site-packages is also local and writeable by default. > > That said, setup.py has some kludges in it that need to be eliminated > or factored out. > > Todd > A third method, a little less onerous than the first two, is: 3) mkdir $HOME/numarray; python setup.py build --gencode --local=$HOME/numarray This works OK even if you're using a Python which does not have a writeable site-packages directory. > > > > ------------------------------------------------------- > Enterprise Linux Forum Conference & Expo, June 4-6, 2003, Santa Clara > The only event dedicated to issues related to Linux enterprise solutions > www.enterpriselinuxforum.com > > _______________________________________________ > Numpy-discussion mailing list > Numpy-discussion at lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/numpy-discussion From fperez at colorado.edu Tue May 6 11:34:03 2003 From: fperez at colorado.edu (Fernando Perez) Date: Tue May 6 11:34:03 2003 Subject: [Numpy-discussion] numarray 0.5 -> python setup.py build fails In-Reply-To: <3EB7FE4E.4080209@stsci.edu> References: <1052136718.23013.4.camel@halloween.stsci.edu> <003101c3136e$3d7d2d90$421ee6a9@rodan> <3EB7E6E9.5090800@stsci.edu> <3EB7FE4E.4080209@stsci.edu> Message-ID: <3EB7FFDD.20002@colorado.edu> Todd Miller wrote: >>Here's a couple work arounds I use: >> >>1) When trying to install to /usr/lib, I first "su root". >> >>2) When I don't like to or can't "su root", I first install Python in >>my home directory using: "./configure --prefix=$HOME". With a local >>Python, site-packages is also local and writeable by default. >> >>That said, setup.py has some kludges in it that need to be eliminated >>or factored out. >> >>Todd >> > > A third method, a little less onerous than the first two, is: > > 3) mkdir $HOME/numarray; python setup.py build --gencode > --local=$HOME/numarray > > This works OK even if you're using a Python which does not have a > writeable site-packages directory. My apologies if I'm off-base, but I'm pretty sure that a properly written setup.py should not try to write anywhere but in the build/ subdirectory when invoked with the 'build' option. It should only write in site-packages when called with 'install'. I rely on this fact all the time, by first building packages and then calling 'setup.py install' separately. And I do this as a non-root user always. I could be wrong, but if setup.py is trying to write at build (not install) time into /usr/lib/..., I think that should be considered a bug. Best, f. From fperez at colorado.edu Tue May 6 11:40:19 2003 From: fperez at colorado.edu (Fernando Perez) Date: Tue May 6 11:40:19 2003 Subject: [Numpy-discussion] numarray 0.5 -> python setup.py build fails In-Reply-To: <3EB7FFDD.20002@colorado.edu> References: <1052136718.23013.4.camel@halloween.stsci.edu> <003101c3136e$3d7d2d90$421ee6a9@rodan> <3EB7E6E9.5090800@stsci.edu> <3EB7FE4E.4080209@stsci.edu> <3EB7FFDD.20002@colorado.edu> Message-ID: <3EB8015E.2010402@colorado.edu> Fernando Perez wrote: > I rely on this fact all the time, by first building packages and then calling > 'setup.py install' separately. And I do this as a non-root user always. Clarification: I call 'build' as non-root, then 'install' as root. Same as doing: make su make install in python it's setup.py build su setup.py install Cheers, f. From jmiller at stsci.edu Tue May 6 14:21:54 2003 From: jmiller at stsci.edu (Todd Miller) Date: Tue May 6 14:21:54 2003 Subject: [Numpy-discussion] Question about numarray Bool type In-Reply-To: <1052237809.3eb7dff188366@webmail.imk.es> References: <200305061451.45361.verveer@embl-heidelberg.de> <3EB7D96B.6080201@stsci.edu> <200305061800.18923.verveer@embl-heidelberg.de> <1052237809.3eb7dff188366@webmail.imk.es> Message-ID: <1052256083.27219.16.camel@halloween.stsci.edu> On Tue, 2003-05-06 at 12:16, Francesc Alted wrote: > Mensaje citado por: Peter Verveer : > > > >That would be somewhat > > > >simpler if Bool is always garantueed to be either a unsigned or a > > signed > > > > char type. > > > > > > I think perhaps we should just re-define Bool as signed char. Any > > other > > > opinions? > > > > That would solve the problem. > > I'm in the same position than Peter. I would definitively prefer a standard C > type for booleans. > > Francesc Alted > I re-defined Bool as signed char in CVS. Without any real expertise in Boolology , I am unqualified to comment further. Let me know if you would like something different. Todd > > ------------------------------------------------------- > Enterprise Linux Forum Conference & Expo, June 4-6, 2003, Santa Clara > The only event dedicated to issues related to Linux enterprise solutions > www.enterpriselinuxforum.com > > _______________________________________________ > Numpy-discussion mailing list > Numpy-discussion at lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/numpy-discussion -- Todd Miller jmiller at stsci.edu STSCI / ESS / SSB From jmiller at stsci.edu Tue May 6 15:02:10 2003 From: jmiller at stsci.edu (Todd Miller) Date: Tue May 6 15:02:10 2003 Subject: [Numpy-discussion] Numarray 0.5 In-Reply-To: <200305061424.26714.verveer@embl-heidelberg.de> References: <200305061424.26714.verveer@embl-heidelberg.de> Message-ID: <1052258543.27223.55.camel@halloween.stsci.edu> On Tue, 2003-05-06 at 08:24, Peter Verveer wrote: > Hi All, > > I found the following problems after testing my software with numarray 0.5: > > 1) Following works fine if both a and b are arrays: > > >>> a = array([2]) > >>> b = array([1, 2]) > >>> print a + b > [3 4] > > However, if b is an python sequence: > > >>> a = array([2]) > >>> b = [1, 2] > >>> print a + b > [3] > > Apparently broadcasting does not work with python sequeces anymore. This used > to work fine in version 0.4. Is this a bug? Yes, unfortunately. Thanks for reporting it! > > 2) It is not possible to compare an array type object to the 'Any' object: > > >>> Float64 == Any > Traceback (most recent call last): > File "", line 1, in ? > File "/usr/local/lib/python2.2/site-packages/numarray/numerictypes.py", line > 112, in __cmp__ > return (genericTypeRank.index(self.name) - > ValueError: list.index(x): x not in list > > I am not sure if this is a bug, or intended behaviour, but the possibilty to > compare an array type object to 'Any' would be very useful for me. > 'Any' grew up from the C-API, rather than down from the Python design, so it's not very well thought out. Right now, it is a placeholder used to mark arrays with undefined types and to indicate "no type constraint" in C API calls. In normal contexts, you can't make an array of type 'Any'. I think there are two reasonable behaviors for comparisons with 'Any', both used in C. The first behavior is literal comparison; here comparison to Any would generally return "not equal". The second behavior is wild-card matching; here, comparison to Any would generally return "equal". Which makes sense to you? How do you want to use this? > 3) The NA_typeNoToTypeObject() function fails if it is called before any > arrays are created. It looks to me as if the pNumType array in > libnumarraymodule.c is not initialized until an array is created. I don't > know if any other functions are affected in the same way. Could this be > fixed? Yes. This is fixed now in CVS. Thanks again! > > Cheers, Peter > > -- > Dr. Peter J. Verveer > Cell Biology and Cell Biophysics Programme > EMBL > Meyerhofstrasse 1 > D-69117 Heidelberg > Germany > Tel. : +49 6221 387245 > Fax : +49 6221 387242 > Email: Peter.Verveer at embl-heidelberg.de > > > > > ------------------------------------------------------- > This sf.net email is sponsored by:ThinkGeek > Welcome to geek heaven. > http://thinkgeek.com/sf > _______________________________________________ > Numpy-discussion mailing list > Numpy-discussion at lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/numpy-discussion -- Todd Miller jmiller at stsci.edu STSCI / ESS / SSB From stan at stanheckman.com Tue May 6 17:08:30 2003 From: stan at stanheckman.com (Stan Heckman) Date: Tue May 6 17:08:30 2003 Subject: [Numpy-discussion] Re: [SciPy-user] [numarray issue] nonzero? Comments on new name solicited In-Reply-To: References: Message-ID: <87n0hzy604.fsf@schedar.com> "Perry Greenfield" writes: > Any objections to using "at"? no, "at" reads reasonably well. > Any better suggestions. "suchthat"? -- Stan From alobo at ija.csic.es Wed May 7 00:41:10 2003 From: alobo at ija.csic.es (Agustin Lobo) Date: Wed May 7 00:41:10 2003 Subject: [Numpy-discussion] Re: [SciPy-user] [numarray issue] nonzero? Comments on new name solicited In-Reply-To: <87n0hzy604.fsf@schedar.com> Message-ID: suchdat would make the programs much easier to read, very good idea. Agus Dr. Agustin Lobo Instituto de Ciencias de la Tierra (CSIC) Lluis Sole Sabaris s/n 08028 Barcelona SPAIN tel 34 93409 5410 fax 34 93411 0012 alobo at ija.csic.es On 6 May 2003, Stan Heckman wrote: > "Perry Greenfield" writes: > > > Any objections to using "at"? > > no, "at" reads reasonably well. > > > Any better suggestions. > > "suchthat"? > > -- > Stan > _______________________________________________ > SciPy-user mailing list > SciPy-user at scipy.net > http://www.scipy.net/mailman/listinfo/scipy-user > From alobo at ija.csic.es Wed May 7 02:02:09 2003 From: alobo at ija.csic.es (Agustin Lobo) Date: Wed May 7 02:02:09 2003 Subject: [Numpy-discussion] Re: [SciPy-user] [numarray issue] nonzero? Comments on new name solicited In-Reply-To: Message-ID: I meant "suchthat" (not "suchdat"), of course. The "spanglish" spelling would be confusing for others! Agus On Wed, 7 May 2003, Agustin Lobo wrote: > suchdat would make the programs much easier > to read, very good idea. > > Agus > > Dr. Agustin Lobo > Instituto de Ciencias de la Tierra (CSIC) > Lluis Sole Sabaris s/n > 08028 Barcelona SPAIN > tel 34 93409 5410 > fax 34 93411 0012 > alobo at ija.csic.es > > > On 6 May 2003, Stan Heckman wrote: > > > "Perry Greenfield" writes: > > > > > Any objections to using "at"? > > > > no, "at" reads reasonably well. > > > > > Any better suggestions. > > > > "suchthat"? > > > > -- > > Stan > > _______________________________________________ > > SciPy-user mailing list > > SciPy-user at scipy.net > > http://www.scipy.net/mailman/listinfo/scipy-user > > > _______________________________________________ > SciPy-user mailing list > SciPy-user at scipy.net > http://www.scipy.net/mailman/listinfo/scipy-user > From verveer at embl-heidelberg.de Wed May 7 02:17:12 2003 From: verveer at embl-heidelberg.de (Peter Verveer) Date: Wed May 7 02:17:12 2003 Subject: [Numpy-discussion] Numarray 0.5 In-Reply-To: <1052258543.27223.55.camel@halloween.stsci.edu> References: <200305061424.26714.verveer@embl-heidelberg.de> <1052258543.27223.55.camel@halloween.stsci.edu> Message-ID: <200305071116.53135.verveer@embl-heidelberg.de> > > I am not sure if this is a bug, or intended behaviour, but the possibilty > > to compare an array type object to 'Any' would be very useful for me. > > 'Any' grew up from the C-API, rather than down from the Python design, > so it's not very well thought out. Right now, it is a placeholder used > to mark arrays with undefined types and to indicate "no type constraint" > in C API calls. In normal contexts, you can't make an array of type > 'Any'. I think there are two reasonable behaviors for comparisons with > 'Any', both used in C. The first behavior is literal comparison; here > comparison to Any would generally return "not equal". The second > behavior is wild-card matching; here, comparison to Any would generally > return "equal". Which makes sense to you? How do you want to use > this? I use 'Any' right now in C functions to indicate 'no type constraint'. You could use both literal comparison and wild-card matching behaviour for that, if you want to do this in Python, I think. But maybe one should not do such a thing in Python by comparison to 'Any' at all. Literal comparison is what I generally expect from Python objects, so anything else may just be confusing (at least to me). However, the name 'Any' does suggest the wild-card matching behaviour. Is there specific reason why you exposed 'Any' in python? Maybe it would be better not to expose such a type object since it seems to be a 'special case'. I am starting to think that my use for it at the Python level is not appropiate. For instance, I could easily use 'None' instead, and I think I will do that in the future. You mentioned that 'Any' is really a C-API thing. Unless somebody has a good use for it I would now actually say that it maybe should not be exposed at the Python API level at all... Cheers, Peter -- Dr. Peter J. Verveer Cell Biology and Cell Biophysics Programme EMBL Meyerhofstrasse 1 D-69117 Heidelberg Germany Tel. : +49 6221 387245 Fax : +49 6221 387242 Email: Peter.Verveer at embl-heidelberg.de From verveer at embl-heidelberg.de Wed May 7 04:19:03 2003 From: verveer at embl-heidelberg.de (Peter Verveer) Date: Wed May 7 04:19:03 2003 Subject: [Numpy-discussion] numarray 0.5 bug Message-ID: <200305071318.10950.verveer@embl-heidelberg.de> In numarray 0.5, the PyArray_ContiguousFromObject() does not appear to work correctly with Python scalar objects, if the requested type is not equal to the type of the object. (I believe that this function should convert the input to an array of the requested type?) For instance, PyArray_ContiguousFromObject(object, tInt32, 0, 1))) works correctly if object is a python integer, but not if it is a float number. Cheers, Peter -- Dr. Peter J. Verveer Cell Biology and Cell Biophysics Programme EMBL Meyerhofstrasse 1 D-69117 Heidelberg Germany Tel. : +49 6221 387245 Fax : +49 6221 387242 Email: Peter.Verveer at embl-heidelberg.de From jmiller at stsci.edu Wed May 7 05:35:20 2003 From: jmiller at stsci.edu (Todd Miller) Date: Wed May 7 05:35:20 2003 Subject: [Numpy-discussion] Numarray 0.5 In-Reply-To: <200305071116.53135.verveer@embl-heidelberg.de> References: <200305061424.26714.verveer@embl-heidelberg.de> <1052258543.27223.55.camel@halloween.stsci.edu> <200305071116.53135.verveer@embl-heidelberg.de> Message-ID: <1052310950.29151.26.camel@halloween.stsci.edu> On Wed, 2003-05-07 at 05:16, Peter Verveer wrote: > > > I am not sure if this is a bug, or intended behaviour, but the possibilty > > > to compare an array type object to 'Any' would be very useful for me. > > > > 'Any' grew up from the C-API, rather than down from the Python design, > > so it's not very well thought out. Right now, it is a placeholder used > > to mark arrays with undefined types and to indicate "no type constraint" > > in C API calls. In normal contexts, you can't make an array of type > > 'Any'. I think there are two reasonable behaviors for comparisons with > > 'Any', both used in C. The first behavior is literal comparison; here > > comparison to Any would generally return "not equal". The second > > behavior is wild-card matching; here, comparison to Any would generally > > return "equal". Which makes sense to you? How do you want to use > > this? > > I use 'Any' right now in C functions to indicate 'no type constraint'. You > could use both literal comparison and wild-card matching behaviour for that, > if you want to do this in Python, I think. But maybe one should not do such a > thing in Python by comparison to 'Any' at all. > > Literal comparison is what I generally expect from Python objects, so anything > else may just be confusing (at least to me). However, the name 'Any' does > suggest the wild-card matching behaviour. > > Is there specific reason why you exposed 'Any' in python? Initially it was not exposed. As the API type functions evolved, it made Any less of a special case to expose it. > Maybe it would be > better not to expose such a type object since it seems to be a 'special > case'. I am starting to think that my use for it at the Python level is not > appropiate. > For instance, I could easily use 'None' instead, and I think I > will do that in the future. Yes. That's probably what I should have done in C as well: tNone rather than tAny. > > You mentioned that 'Any' is really a C-API thing. Unless somebody has a good > use for it I would now actually say that it maybe should not be exposed at > the Python API level at all... I agree with this. It might be as simple as renaming it to _Any in Python, but I still hesitate to change this until I have a little more time to think about it. Too often, things go from "bad" to "not so good"... Thanks for the input, Todd From perry at stsci.edu Wed May 7 13:20:20 2003 From: perry at stsci.edu (Perry Greenfield) Date: Wed May 7 13:20:20 2003 Subject: [Numpy-discussion] RE: [SciPy-user] [numarray issue] nonzero? Comments on new name solicited In-Reply-To: <87n0hzy604.fsf@schedar.com> Message-ID: Some one suggested that numarray ditch compatibility in this area and "do what's right". Namely he suggested that nonzero --> where where --> select I have to agree that these are far better names than the existing ones. But compatibilty is important. So what to do? Here's one suggestion; I'm curious what people think about this solution. Nothing prevents us from having a module that people would import for Numeric compatibilty and a different one that has somewhat different behavior. Before people start throwing rocks (when you finish reading you can throw them ;-) and say "didn't you say that customized versions of Numeric or numarray were a bad idea?" Yes I did. But I think some customizations are bad and perhaps some are acceptable. What I objected to was having array objects with different behavior Since array objects may be passed between modules that assume different behaviors, that will cause all sorts of problems (e.g., module A was written assuming array slices are views but is passed an array that produces a copy when sliced). But is there any harm when all that is different are module functions? These are unlikely to be passed from one module to another and thus the customization is localized to the user modules. Suppose that the Numeric-compatible version defines "where" and "nonzero" to have their current meanings and numarray uses "select", and "where" respectively. There is one case that could arise that might cause problems. About the only time a problem would occur is if someone did from numarray import * from B import * And in B.py from Numericcompatible import * in which case the numarray "where" is replaced by the Numeric "where" I don't know if this is a strong enough reason to support two different flavors, but what do people think of taking this approach? Are there any other module functions that would benefit from such changes in name or behavior (For example, the axis order issue)? Perry Greenfield From perry at stsci.edu Wed May 7 13:36:09 2003 From: perry at stsci.edu (Perry Greenfield) Date: Wed May 7 13:36:09 2003 Subject: [Numpy-discussion] [numarray] packaging and names In-Reply-To: Message-ID: There was general consensus that numarray should be organized as a package. We are about to start doing so. It does raise some issues (and opportunities) however. 1) What should the package be called? numarray is a possiblity, but we have an opportunity to take a different approach. For example if we could name the package "array" (unfortunately, we can't) then the following becomes possible (these are just examples, we haven't settled on any naming scheme yet). from array.numeric import * # instead of "from numarray import *" import array.fft import array.records # currently recarray Since array is not avaible (there is already an array module in the Python Standard Library) what are the alternatives? Here are some: num numarray arr arrays ndarray I've polled people locally and arrays and ndarray were the most popular, but there doesn't seem to be a clear winner. I tend toward arr somewhat because of the brevity but do realize it isn't terribly descriptive. The drawback of "arrays" is that it may be visually confused with array. 2) What to call the package components? Locally the favorites were numarray --> numeric recarray --> records chararray--> strings (but since we plan to implement PyObject arrays there is a possibilty of real Python string arrays) ndarray --> generic (or base) 3) Where to place 3rd party array extension modules (by that I mean all those not distributed with numarray)? Should they go in the package directory? I'd prefer that they go into a separate directory from the things that are part of the standard numarray distribution (a site-packages equivalent in the package?) 4) We would likely also include in the package numarray.py, recarray.py, chararray.py, ndarray.py so that if the package was added to the path, the current imports would continue to work. This is intended as a short-term measure (i.e., they are deprecated and will disappear at version 1.0, or earlier). We may also keep these files at the current nummarray interface so that some of the software we currently distribute does not need to change immediately to match the changes we have been talking about. We realize that these are big changes, but if we move to a package structure, we ought to think about what makes the most sense (and if it is any comfort, we suffer the consequences of change more than anyone else). Better now than later. Screams? Perry From falted at openlc.org Sat May 10 04:40:01 2003 From: falted at openlc.org (Francesc Alted) Date: Sat May 10 04:40:01 2003 Subject: [Numpy-discussion] PyTables 0.5 released Message-ID: <200305101338.25291.falted@openlc.org> Announcing PyTables 0.5 ----------------------- This is the second public beta release. On this release you will find a 20% of I/O speed improvement over the previous one (0.4), some bugs has been fixed and support for a couple of compression (LZO and UCL) libraries has been added, and... a long awaited Windows version is finally available!. More in detail: What's new ----------- - As a consequence of some twiking the write/read performance has been improved by a 20% overall. One particular case were performance has largely increased (0.5 is up to 6 times faster than 0.4) is when column elements are unidimensional arrays. This impressive speed-up is mainly because of the recent improvements in numarray 0.5 performance (good work, folks!). With that, the reading speed is reaching its theoretical maximum (at least when using the current data access schema). - When reading a Table object, and the user wants to fetch column elements which are unidimensional arrays, a copy of the array from the I/O buffer is delivered automatically to him, so that there is no need to make a call to .copy() method of the numarray arrays anymore. It think this is more comfortable for the user. - The compression was enabled by default in version 0.4, despite of what was stated in the documentation. Now, this has been corrected and compression is *disabled* by default. - Support for two new compression libraries: LZO and UCL (http://www.oberhumer.com/opensource/). These libraries are made by Markus F.X.J. Oberhumer, and they stand for allowing *very* fast decompression. Now, if your data is compressible, you can obtain better reading speed than if not using compression at all!. The improvement is still more noticeable if your are dealing with extremely large (and compressible) data sets. Read the online documentation for more info about that: http://pytables.sourceforge.net/html-doc/usersguide-html3.html#subsection3.4.1 - A couple of memory leaks has been isolated and fixed (it was hard, but I finally did it!). - A bug with column ordering of tables that happens in some special situations has been fixed (thanks to Stan Heckman for reporting this and suggesting the patch). - File class has now an 'isopen' attribute in order to check if a file is open or not. - Updated documentation, specially for giving advice about the use of the new compression libraries. See "Compression issues" subsection, (also on the web: http://pytables.sourceforge.net/html-doc/usersguide-html.html) - Added more unit tests (up to 218 now!) - PyTables has been tested against newest numarray 0.5 and it works just fine. It even works well with Python 2.3b1. - And last, but not least, a Windows version is available!. Thanks to Alan McIntyre for its porting!. There is even a binary ready for click and install. What it is ---------- In short, PyTables provides a powerful and very Pythonic interface to process and organize your table and array data on disk. Its goal is to enable the end user to manipulate easily scientific data tables and Numerical and numarray Python objects in a persistent hierarchical structure. The foundation of the underlying hierarchical data organization is the excellent HDF5 library (http://hdf.ncsa.uiuc.edu/HDF5). A table is defined as a collection of records whose values are stored in fixed-length fields. All records have the same structure and all values in each field have the same data type. The terms "fixed-length" and strict "data types" seems to be quite a strange requirement for an interpreted language like Python, but they serve a useful function if the goal is to save very large quantities of data (such as is generated by many scientific applications, for example) in an efficient manner that reduces demand on CPU time and I/O resources. Quite a bit effort has been invested to make browsing the hierarchical data structure a pleasant experience. PyTables implements just two (orthogonal) easy-to-use methods for browsing. What is HDF5? ------------- For those people who know nothing about HDF5, it is is a general purpose library and file format for storing scientific data made at NCSA. HDF5 can store two primary objects: datasets and groups. A dataset is essentially a multidimensional array of data elements, and a group is a structure for organizing objects in an HDF5 file. Using these two basic constructs, one can create and store almost any kind of scientific data structure, such as images, arrays of vectors, and structured and unstructured grids. You can also mix and match them in HDF5 files according to your needs. Platforms --------- I'm using Linux as the main development platform, but PyTables should be easy to compile/install on other UNIX machines. This package has also passed all the tests on a UltraSparc platform with Solaris 7 and Solaris 8. It also compiles and passes all the tests on a SGI Origin2000 with MIPS R12000 processors and running IRIX 6.5. With Windows, PyTables has been tested with Windows 2000 Professional SP1 and Windows XP, but it should also work with other flavors. An example? ----------- For online code examples, have a look at http://pytables.sourceforge.net/tut/tutorial1-1.html and http://pytables.sourceforge.net/tut/tutorial1-2.html Web site -------- Go to the PyTables web site for more details: http://pytables.sourceforge.net/ Share your experience --------------------- Let me know of any bugs, suggestions, gripes, kudos, etc. you may have. Have fun! -- Francesc Alted From mdehoon at ims.u-tokyo.ac.jp Mon May 12 02:49:09 2003 From: mdehoon at ims.u-tokyo.ac.jp (Michiel Jan Laurens de Hoon) Date: Mon May 12 02:49:09 2003 Subject: [Numpy-discussion] Linking to ranlib Message-ID: <3EBF6EF4.8020305@ims.u-tokyo.ac.jp> Hi, I have a C extension module that makes use of the random number functions in ranlib. Since numpy's RandomArray also relies on ranlib, there is a ranlib.so in site-packages/Numeric, and I found the ranlib.h header file in the include/python2.3/Numeric directory, containing the prototypes for the ranlib routines that I am trying to use. So I was hoping to #include this header file and link to numpy's ranlib.so instead of including another copy of ranlib as part of my C extension module. However, I haven't been able to get this to work -- on Linux and Mac OS X, gcc does not recognize ranlib.so as a library it can link to, whereas on Cygwin gcc cannot find the routines I am looking for in ranlib.dll. Does anybody know if it is possible to link to ranlib.so? If not, what is the function of the ranlib.h include file? Unfortunately the calls to the ranlib routines are way down in my C extension module, I wouldn't be able to call those routines at the Python level. Many thanks, --Michiel. -- Michiel de Hoon, Assistant Professor University of Tokyo, Institute of Medical Science Human Genome Center 4-6-1 Shirokane-dai, Minato-ku Tokyo 108-8639 Japan http://bonsai.ims.u-tokyo.ac.jp/~mdehoon From falted at openlc.org Mon May 12 11:13:15 2003 From: falted at openlc.org (Francesc Alted) Date: Mon May 12 11:13:15 2003 Subject: [Numpy-discussion] PyTables 0.5.1 Message-ID: <200305122012.00407.falted@openlc.org> PyTables 0.5.1 -------------- This is a maintenance release of PyTables. Due to a problem with an specific optimization in PyTables 0.5, it does not work with numarray 0.4 (although it works just fine with numarray 0.5). Thanks to Marc Gehling for reporting that. Todd Miller has already warned me that this optimization was not safe, so I am *disabling* it in 0.5.1. The consequence is that the 20% of improvement during reading tables has almost evaporated to a rather small 4%, but that's life!. If you already have installed PyTables 0.5, I strongly suggest you to upgrade to 0.5.1, even if you are already using numarray 0.5. I will try to further investigate the problem, and, if a good solution is found, I will enable again the optimization in a future release. Another new thing you can find in 0.5.1 is that the use of "UInt64" data types has been removed (it has been replaced by the "Int64" type) of the tutorial chapters in User's Manual. I've done that because the numarray Windows version does not support such a type (due to MSVC compiler limitations). Now, the tutorial section should run fine in all the supported platforms (even Windows). My apologies for being a sinner and trying to optimize too soon ;-) Web site -------- Go to the PyTables web site for more details: http://pytables.sourceforge.net/ Share your experience --------------------- Let me know of any bugs, suggestions, gripes, kudos, etc. you may have. Have fun! -- Francesc Alted From gvermeul at grenoble.cnrs.fr Mon May 12 13:08:02 2003 From: gvermeul at grenoble.cnrs.fr (Gerard Vermeulen) Date: Mon May 12 13:08:02 2003 Subject: [Numpy-discussion] mixing NUMERIC and NUMARRAY Message-ID: <20030512220653.63dc08b9.gvermeul@grenoble.cnrs.fr> Hi, It was remarkable easy to make my plot package PyQwt for Numeric, also compatible with numarray :-) ( PyQwt is a wrapper for Qwt, a Qt-based library with widgets useful for science and engineering -- http://gerard.vermeulen.free.fr ) I discovered by accident that PyQwt compiled with #include plots Numeric-arrays and that PyQwt compiled with #include plots numarray-arrays. The only interface between PyQwt and Numeric (or numarray) passes through PyArray_ContiguousFromObject to cast sequences into arrays from which data is copied into Qwt's data structures. PyQwt never returns Numeric- or numarray-arrays. Is this feature platform dependent? Or is it safe to release PyQwt, claiming that a Numeric-compiled PyQwt is compatible with numarray and vice-versa? Gerard From jmiller at stsci.edu Mon May 12 13:52:02 2003 From: jmiller at stsci.edu (Todd Miller) Date: Mon May 12 13:52:02 2003 Subject: [Numpy-discussion] mixing NUMERIC and NUMARRAY In-Reply-To: <20030512220653.63dc08b9.gvermeul@grenoble.cnrs.fr> References: <20030512220653.63dc08b9.gvermeul@grenoble.cnrs.fr> Message-ID: <1052772708.13040.60.camel@halloween.stsci.edu> On Mon, 2003-05-12 at 16:06, Gerard Vermeulen wrote: > Hi, > > It was remarkable easy to make my plot package PyQwt for Numeric, also > compatible with numarray :-) > Great! That's they way things are *supposed* to work, and usually do, but don't always. > ( PyQwt is a wrapper for Qwt, a Qt-based library with widgets useful for > science and engineering -- http://gerard.vermeulen.free.fr ) > > I discovered by accident that PyQwt compiled with > #include > plots Numeric-arrays and that PyQwt compiled with > #include > plots numarray-arrays. > This part sounds a little backward. Presumably this was a typo. > The only interface between PyQwt and Numeric (or numarray) passes through > PyArray_ContiguousFromObject to cast sequences into arrays from which data > is copied into Qwt's data structures. > > PyQwt never returns Numeric- or numarray-arrays. > > Is this feature platform dependent? Yes, if I understand your question correctly. > Or is it safe to release PyQwt, > claiming that a Numeric-compiled PyQwt is compatible with numarray > and vice-versa? No dice. numarray is currently trying for source level compatibility only, so an extension compiled for Numeric creates slightly different object code than one compiled for numarray; slightly different means inoperable. Still, it's good to know that PyQwt is very close to workable for numarray. > Gerard > _______________________________________________ > Numpy-discussion mailing list > Numpy-discussion at lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/numpy-discussion -- Todd Miller jmiller at stsci.edu STSCI / ESS / SSB From gvermeul at grenoble.cnrs.fr Mon May 12 15:53:04 2003 From: gvermeul at grenoble.cnrs.fr (gvermeul at grenoble.cnrs.fr) Date: Mon May 12 15:53:04 2003 Subject: [Numpy-discussion] no typo: mixing NUMERIC and NUMARRAY Message-ID: <200305122252.h4CMq9dk024899@grenoble.cnrs.fr> > > > ( PyQwt is a wrapper for Qwt, a Qt-based library with widgets useful for > > science and engineering -- http://gerard.vermeulen.free.fr ) > > > > I discovered by accident that PyQwt compiled with > > #include > > plots Numeric-arrays and that PyQwt compiled with > > #include > > plots numarray-arrays. > > > > This part sounds a little backward. Presumably this was a typo. > No Todd, this is no typo. I launched a test program from the wrong directory and was shocked to see that it worked (have checked with python -vv which modules were loaded). The other way around worked, too. Of course, the normal cases work, too. (verified that multiplying a Numeric array with numarray does not work) > > > The only interface between PyQwt and Numeric (or numarray) passes through > > PyArray_ContiguousFromObject to cast sequences into arrays from which data > > is copied into Qwt's data structures. > > > > PyQwt never returns Numeric- or numarray-arrays. > > > > Is this feature platform dependent? > > Yes, if I understand your question correctly. > > > Or is it safe to release PyQwt, > > claiming that a Numeric-compiled PyQwt is compatible with numarray > > and vice-versa? > > No dice. numarray is currently trying for source level compatibility > only, so an extension compiled for Numeric creates slightly different > object code than one compiled for numarray; slightly different means > inoperable. Still, it's good to know that PyQwt is very close to > workable for numarray. > Well, my experience shows that on x86-Linux the layout of the data in a numarray-array and Numeric-array is sufficiently close that both PyArray_ContiguousFromObject() functions also interprete the arrays of the 'cousin' correctly (at least in 1D). I realize that this observation is very hackish, but it may help to speed up the acceptance of numarray. Gerard ------------------------------------------------------------- This message was sent using HTTPS service from CNRS Grenoble. ---> https://grenoble.cnrs.fr <--- From alnesbit at students.cs.mu.OZ.AU Mon May 12 23:01:03 2003 From: alnesbit at students.cs.mu.OZ.AU (Andrew Nesbit) Date: Mon May 12 23:01:03 2003 Subject: [Numpy-discussion] Converting string to array Message-ID: I have a string representing a sequence of 16-bit audio samples, read in from an AIFF file like this: f = aifc.open("filename.aiff", "r") data = f.readframes(f.getnframes()) I can convert this to a tuple of signed 16-bit integers using the Python standard library "array" module: a = array.array("h", data) This gives samples in the range [-32768, 32767] and everything is fine. However, instead of using module array I want to use Numeric's array function in an equivalent way. I've been trying to do it like this: a = Numeric.array(data, Numeric.Int16) Now, a.itemsize() outputs 2, which seems fine, but when I iterate over the resulting array (using standard a[i] notation), the results indicate that the array is being treated as a sequence of 8-bit, rather than 16-bit, integers. What am I doing wrong here? Thanks in advance. Andrew. From eric at enthought.com Mon May 12 23:11:07 2003 From: eric at enthought.com (eric jones) Date: Mon May 12 23:11:07 2003 Subject: [Numpy-discussion] Converting string to array In-Reply-To: Message-ID: <001c01c31916$5b4fb0b0$8901a8c0@ERICDESKTOP> Try fromstring. I think it should do what you want: >>> from Numeric import * >>> print fromstring.__doc__ fromstring(string, typecode='l', count=-1) returns a new 1d array initialized from the raw binary data in string. If count is positive, the new array will have count elements, otherwise it's size is determined by the size of string. eric ---------------------------------------------- eric jones 515 Congress Ave www.enthought.com Suite 1614 512 536-1057 Austin, Tx 78701 > -----Original Message----- > From: numpy-discussion-admin at lists.sourceforge.net [mailto:numpy- > discussion-admin at lists.sourceforge.net] On Behalf Of Andrew Nesbit > Sent: Monday, May 12, 2003 11:57 PM > To: numpy-discussion at lists.sourceforge.net > Subject: [Numpy-discussion] Converting string to array > > I have a string representing a sequence of 16-bit audio samples, read > in from an AIFF file like this: > > f = aifc.open("filename.aiff", "r") > data = f.readframes(f.getnframes()) > > I can convert this to a tuple of signed 16-bit integers using the > Python standard library "array" module: > > a = array.array("h", data) > > This gives samples in the range [-32768, 32767] and everything is > fine. > > However, instead of using module array I want to use Numeric's array > function in an equivalent way. I've been trying to do it like this: > > a = Numeric.array(data, Numeric.Int16) > > Now, a.itemsize() outputs 2, which seems fine, but when I iterate over > the resulting array (using standard a[i] notation), the results > indicate that the array is being treated as a sequence of 8-bit, > rather than 16-bit, integers. > > What am I doing wrong here? > > Thanks in advance. > > Andrew. > > > > > ------------------------------------------------------- > Enterprise Linux Forum Conference & Expo, June 4-6, 2003, Santa Clara > The only event dedicated to issues related to Linux enterprise solutions > www.enterpriselinuxforum.com > > _______________________________________________ > Numpy-discussion mailing list > Numpy-discussion at lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/numpy-discussion From alnesbit at students.cs.mu.OZ.AU Mon May 12 23:26:03 2003 From: alnesbit at students.cs.mu.OZ.AU (Andrew Nesbit) Date: Mon May 12 23:26:03 2003 Subject: [Numpy-discussion] Converting string to array In-Reply-To: <001c01c31916$5b4fb0b0$8901a8c0@ERICDESKTOP> Message-ID: On Tue, 13 May 2003, eric jones wrote: > Try fromstring. I think it should do what you want: > > >>> from Numeric import * print fromstring.__doc__ > > fromstring(string, typecode='l', count=-1) returns a new 1d array > initialized from the raw binary data in string. If count is > positive, the new array will have count elements, otherwise it's > size is determined by the size of string. Thanks, Eric. That did the trick. The manual said that Numeric.array() takes a sequence type as one of its arguments, and considering that a string is a sequence type, I thought that, given an appropriate typecode, it would do the conversion automatically in the same way that array.array does. Is this difference an intended feature of numpy? Andrew. From jmiller at stsci.edu Tue May 13 14:50:11 2003 From: jmiller at stsci.edu (Todd Miller) Date: Tue May 13 14:50:11 2003 Subject: [Numpy-discussion] numarray repackaging one more time... Message-ID: <3EC1688F.5070606@stsci.edu> As has been mentioned before, we're planning to repackage numarray as a package rather than as a collection of modules. We're soliciting comments now because we only want to do this once. Here are the planned renamings for the numarray modules as we transition to a package: numarray --> numarray.numeric recarray --> numarray.records chararray --> numarray.strings ndarray --> numarray.generic * --> numarray.* Note that class and function names will remain unchanged, so recarray.RecArray will become numarray.records.RecArray. Here are the planned renamings for the current and planned add-on packages consolidated into a single add-ons distribution: LinearAlgrebra2 --> numarray.LinearAlgebra FFT2 --> numarray.FFT RandomArray2 --> numarray.RandomArray Convolve --> numarray.Convolve Convolve.Image --> numarray.Image MA2 --> numarray.MA Future 3rd party packages should locate themselves under the numarray.addons sub-package: * --> numarray.addons.* The package name (shown here as numarray) is still undecided. By including stub modules which emulate the "import behavior" of numarray-0.5, we'll make it possible to ignore / work around the effects of the re-packaging for the time being. The stub modules will be activated either through the creation of a .pth file or via additions to PYTHONPATH to make them visible. Comments? From cookedm at physics.mcmaster.ca Tue May 13 15:16:15 2003 From: cookedm at physics.mcmaster.ca (David M. Cooke) Date: Tue May 13 15:16:15 2003 Subject: [Numpy-discussion] numarray repackaging one more time... In-Reply-To: <3EC1688F.5070606@stsci.edu> References: <3EC1688F.5070606@stsci.edu> Message-ID: <20030513221331.GA14575@arbutus.physics.mcmaster.ca> On Tue, May 13, 2003 at 05:50:07PM -0400, Todd Miller wrote: > As has been mentioned before, we're planning to repackage numarray as > a package rather than as a collection of modules. We're soliciting > comments now because we only want to do this once. ... > Future 3rd party packages should locate themselves under the > numarray.addons sub-package: > > * --> numarray.addons.* What's the rationale for having a separate place for third-party packages? Shouldn't they just install themselves as their own packages or modules? (i.e., in site-packages/) -- |>|\/|< /--------------------------------------------------------------------------\ |David M. Cooke http://arbutus.physics.mcmaster.ca/dmc/ |cookedm at physics.mcmaster.ca From haase at msg.ucsf.edu Tue May 13 19:14:05 2003 From: haase at msg.ucsf.edu (Sebastian Haase) Date: Tue May 13 19:14:05 2003 Subject: [Numpy-discussion] make C array accessible to python without copy Message-ID: <00ca01c319be$6f25e160$421ee6a9@rodan> Hi - I hope it's OK to just resent my posting from Feb 22. I didn't get any response at the time. Also - looking into the code again - I am wondering what the best way to handle the reference counting would be ? Here come my original posting: > Short follow up: > 1) Is it planned to support this more directly? > 2) How much does it cost to create a buffer object if it > uses my already allocated memory ? > 3) Can I change the pointer so that it points to a > different memory space WITHOUT having to recreate any > python objects? Or would that "confuse" the buffer or > numarray? (We are hoping to aquire 30 images per second > - the images should get written into a circular buffer so > that the data can be written to disk in larger chunks - > but the python array should always refer to the current > image ) > > Thanks for all the nice toys (tools) ;-) > Sebastian Haase > > > > On Fri, 17 Jan 2003 18:16:01 -0500 > Todd Miller wrote: > >Sebastian Haase wrote: > > > >>Hi, > >>What is the C API to make an array that got allocated, > >>let's say, by a = new short[512*512], > >>accessible to python as numarray. > >> > >What you want to do is not currently supported well in C. > > The way to do what you want is: > > > >1. Create a buffer object from your C++ array. The > >buffer object can be built such that it refers to the > >original copy of the data. > > > >2. Call back into Python (numarray.NumArray) with your > >buffer object as the buffer parameter. > > > >You can scavenge the code in NA_newAll (Src/newarray.ch) > >for most of the callback. > > > >>I tried NA_New - but that seems to make a copy. > >>I would need it to use the original memory space > >>so that I can "observe" the array from Python WHILE > >>the underlying C array changes (it's actually a camera > >>image) > >> > >That sounds cool! > > > >> > >>Thanks, > >>Sebastian Haase From gvermeul at grenoble.cnrs.fr Wed May 14 00:25:10 2003 From: gvermeul at grenoble.cnrs.fr (Gerard Vermeulen) Date: Wed May 14 00:25:10 2003 Subject: [Numpy-discussion] numarray repackaging one more time... In-Reply-To: <20030513221331.GA14575@arbutus.physics.mcmaster.ca> References: <3EC1688F.5070606@stsci.edu> <20030513221331.GA14575@arbutus.physics.mcmaster.ca> Message-ID: <20030514092419.5b3cdd10.gvermeul@grenoble.cnrs.fr> On Tue, 13 May 2003 18:13:31 -0400 "David M. Cooke" wrote: > On Tue, May 13, 2003 at 05:50:07PM -0400, Todd Miller wrote: > > As has been mentioned before, we're planning to repackage numarray as > > a package rather than as a collection of modules. We're soliciting > > comments now because we only want to do this once. > ... > > Future 3rd party packages should locate themselves under the > > numarray.addons sub-package: > > > > * --> numarray.addons.* > > What's the rationale for having a separate place for third-party > packages? Shouldn't they just install themselves as their own packages > or modules? (i.e., in site-packages/) > I like the idea, because it would mean that I can release my PyQwt plot package so that it can be build for Numeric and numarray on the same system. It reduces the risk of calling 'PyQwt for numarray' functions with 'Numeric arrays' as arguments and vice-versa (for me it seems to work, but it scares me to death). Gerard From hinsen at cnrs-orleans.fr Wed May 14 02:27:03 2003 From: hinsen at cnrs-orleans.fr (Konrad Hinsen) Date: Wed May 14 02:27:03 2003 Subject: [Numpy-discussion] numarray repackaging one more time... In-Reply-To: <20030514092419.5b3cdd10.gvermeul@grenoble.cnrs.fr> References: <3EC1688F.5070606@stsci.edu> <20030513221331.GA14575@arbutus.physics.mcmaster.ca> <20030514092419.5b3cdd10.gvermeul@grenoble.cnrs.fr> Message-ID: <200305141123.45560.hinsen@cnrs-orleans.fr> > I like the idea, because it would mean that I can release my PyQwt > plot package so that it can be build for Numeric and numarray > on the same system. It reduces the risk of calling 'PyQwt for numarray' I'd like to check that this doesn't cause any technical problems with package managers, distutils, etc. Otherwise it will prevent people from switching to numarray. Konrad. -- ------------------------------------------------------------------------------- Konrad Hinsen | E-Mail: hinsen at cnrs-orleans.fr Centre de Biophysique Moleculaire (CNRS) | Tel.: +33-2.38.25.56.24 Rue Charles Sadron | Fax: +33-2.38.63.15.17 45071 Orleans Cedex 2 | Deutsch/Esperanto/English/ France | Nederlands/Francais ------------------------------------------------------------------------------- From hinsen at cnrs-orleans.fr Wed May 14 02:29:03 2003 From: hinsen at cnrs-orleans.fr (Konrad Hinsen) Date: Wed May 14 02:29:03 2003 Subject: [Numpy-discussion] numarray repackaging one more time... In-Reply-To: <3EC1688F.5070606@stsci.edu> References: <3EC1688F.5070606@stsci.edu> Message-ID: <200305141126.07426.hinsen@cnrs-orleans.fr> On Tuesday 13 May 2003 23:50, Todd Miller wrote: > Here are the planned renamings for the numarray modules as we transition > to a package: > > numarray --> numarray.numeric > recarray --> numarray.records > chararray --> numarray.strings > ndarray --> numarray.generic > * --> numarray.* > > Note that class and function names will remain unchanged, so > recarray.RecArray will become numarray.records.RecArray. > > Here are the planned renamings for the current and planned add-on packages > consolidated into a single add-ons distribution: > > LinearAlgrebra2 --> numarray.LinearAlgebra > FFT2 --> numarray.FFT > RandomArray2 --> numarray.RandomArray > Convolve --> numarray.Convolve > Convolve.Image --> numarray.Image > MA2 --> numarray.MA Please use a uniform capitalization scheme, no matter which one. Either numarray.image or Numarray.Image (I prefer the latter, but both are better than the mixed one). Konrad. -- ------------------------------------------------------------------------------- Konrad Hinsen | E-Mail: hinsen at cnrs-orleans.fr Centre de Biophysique Moleculaire (CNRS) | Tel.: +33-2.38.25.56.24 Rue Charles Sadron | Fax: +33-2.38.63.15.17 45071 Orleans Cedex 2 | Deutsch/Esperanto/English/ France | Nederlands/Francais ------------------------------------------------------------------------------- From falted at openlc.org Wed May 14 02:46:03 2003 From: falted at openlc.org (Francesc Alted) Date: Wed May 14 02:46:03 2003 Subject: [Numpy-discussion] numarray repackaging one more time... In-Reply-To: <20030514092419.5b3cdd10.gvermeul@grenoble.cnrs.fr> References: <3EC1688F.5070606@stsci.edu> <20030513221331.GA14575@arbutus.physics.mcmaster.ca> <20030514092419.5b3cdd10.gvermeul@grenoble.cnrs.fr> Message-ID: <200305141145.07726.falted@openlc.org> A Dimecres 14 Maig 2003 09:24, Gerard Vermeulen va escriure: > > I like the idea, because it would mean that I can release my PyQwt > plot package so that it can be build for Numeric and numarray > on the same system. It reduces the risk of calling 'PyQwt for numarray' > functions with 'Numeric arrays' as arguments and vice-versa (for me it > seems to work, but it scares me to death). I think this is not necessary as you can always distinguish if the user is passing a numarray or Numeric object, like in: In [3]: a=Numeric.array([1,2]) In [4]: b=numarray.array([1,2]) In [5]: type(a) Out[5]: In [6]: type(b) Out[6]: It's just a check before accessing the object. Cheers, -- Francesc Alted From Kasper.Souren at ircam.fr Wed May 14 03:03:08 2003 From: Kasper.Souren at ircam.fr (Kasper Souren) Date: Wed May 14 03:03:08 2003 Subject: [Numpy-discussion] numarray repackaging one more time... In-Reply-To: <20030513221331.GA14575@arbutus.physics.mcmaster.ca> References: <3EC1688F.5070606@stsci.edu> <20030513221331.GA14575@arbutus.physics.mcmaster.ca> Message-ID: <200305141334.58603.Kasper.Souren@ircam.fr> > > Future 3rd party packages should locate themselves under the > > numarray.addons sub-package: > > > > * --> numarray.addons.* > > What's the rationale for having a separate place for third-party > packages? Shouldn't they just install themselves as their own packages > or modules? (i.e., in site-packages/) Maybe it's nice to have both. Leave it to the third-party to decide whether it's a good idea to install in numarray.addons or in site-packages/. For stuff that's not too dependent on numpy (PyGame for instance) it would be better to install in site-packages/. For other things (like PyQwt) it might be better to locate them in numarray.addons. bye, Kasper From jmiller at stsci.edu Wed May 14 04:08:08 2003 From: jmiller at stsci.edu (Todd Miller) Date: Wed May 14 04:08:08 2003 Subject: [Numpy-discussion] numarray repackaging one more time... References: <3EC1688F.5070606@stsci.edu> <20030513221331.GA14575@arbutus.physics.mcmaster.ca> Message-ID: <3EC223D2.2030901@stsci.edu> David M. Cooke wrote: >On Tue, May 13, 2003 at 05:50:07PM -0400, Todd Miller wrote: > > >>As has been mentioned before, we're planning to repackage numarray as >>a package rather than as a collection of modules. We're soliciting >>comments now because we only want to do this once. >> >> >... > > >>Future 3rd party packages should locate themselves under the >>numarray.addons sub-package: >> >>* --> numarray.addons.* >> >> > >What's the rationale for having a separate place for third-party >packages? Shouldn't they just install themselves as their own packages >or modules? (i.e., in site-packages/) > > > I think there are two goals which are served by numarray.addons: 1. It becomes clear what is in the numarray core, and what isn't. This is a maintenance issue. 2. It becomes clear that a particular extension was built explicitly for numarray. This might be valuable in a transition phase from Numeric to numarray. Todd From jmiller at stsci.edu Wed May 14 04:31:02 2003 From: jmiller at stsci.edu (Todd Miller) Date: Wed May 14 04:31:02 2003 Subject: [Numpy-discussion] numarray repackaging one more time... References: <3EC1688F.5070606@stsci.edu> <200305141126.07426.hinsen@cnrs-orleans.fr> Message-ID: <3EC22950.8070107@stsci.edu> Konrad Hinsen wrote: >On Tuesday 13 May 2003 23:50, Todd Miller wrote: > > > >>Here are the planned renamings for the numarray modules as we transition >>to a package: >> >>numarray --> numarray.numeric >>recarray --> numarray.records >>chararray --> numarray.strings >>ndarray --> numarray.generic >>* --> numarray.* >> >>Note that class and function names will remain unchanged, so >>recarray.RecArray will become numarray.records.RecArray. >> >>Here are the planned renamings for the current and planned add-on packages >>consolidated into a single add-ons distribution: >> >>LinearAlgrebra2 --> numarray.LinearAlgebra >>FFT2 --> numarray.FFT >>RandomArray2 --> numarray.RandomArray >>Convolve --> numarray.Convolve >>Convolve.Image --> numarray.Image >>MA2 --> numarray.MA >> >> > >Please use a uniform capitalization scheme, no matter which one. Either >numarray.image or Numarray.Image (I prefer the latter, but both are better >than the mixed one). > >Konrad. > > Point taken. I guess that changes the addon mappings to: LinearAlgrebra2 --> numarray.linear_algebra FFT2 --> numarray.fft RandomArray2 --> numarray.random_array Convolve --> numarray.convolve Convolve.Image --> numarray.image MA2 --> numarray.ma or changes everything to: numarray --> Numarray.Numeric recarray --> Numarray.Records chararray --> Numarray.Strings ndarray --> Numarray.Generic * --> Numarray.* LinearAlgrebra2 --> Numarray.LinearAlgebra FFT2 --> Numarray.FFT RandomArray2 --> Numarray.RandomArray Convolve --> Numarray.Convolve Convolve.Image --> Numarray.Image MA2 --> Numarray.MA I have no preference myself (obviously!). Lower case and underscores may be more in line with our group's style. Todd From Kasper.Souren at ircam.fr Wed May 14 04:41:21 2003 From: Kasper.Souren at ircam.fr (Kasper Souren) Date: Wed May 14 04:41:21 2003 Subject: [Numpy-discussion] numarray repackaging one more time... Message-ID: <200305141337.30397.Kasper.Souren@ircam.fr> > I have no preference myself (obviously!). Lower case and underscores > may be more in line with our group's style. The Python Style Guide [1] doesn't give a definitive answer, but there seems to be a preference for the lowercase in this situation. bye, Kasper [1] http://www.python.org/doc/essays/styleguide.html P.S. A simple reply on this list just replies to the owner. I would prefer that it just gives me Numpy-discussion at lists.sourceforge.net instead. -- Hi! I'm a .signature virus! copy me into your .signature file to help me spread! Thanks to DRM, you know that something has been built in environment of unspecified degree of security, from source you cannot check, written by programmers you don't know, released after passing QA of unknown quality and which is released under a license which disclaims any responsibility... From jmiller at stsci.edu Wed May 14 04:55:09 2003 From: jmiller at stsci.edu (Todd Miller) Date: Wed May 14 04:55:09 2003 Subject: [Numpy-discussion] numarray repackaging one more time... References: <3EC1688F.5070606@stsci.edu> <20030513221331.GA14575@arbutus.physics.mcmaster.ca> <20030514092419.5b3cdd10.gvermeul@grenoble.cnrs.fr> <200305141145.07726.falted@openlc.org> Message-ID: <3EC22EC0.90309@stsci.edu> Francesc Alted wrote: >A Dimecres 14 Maig 2003 09:24, Gerard Vermeulen va escriure: > > >>I like the idea, because it would mean that I can release my PyQwt >>plot package so that it can be build for Numeric and numarray >>on the same system. It reduces the risk of calling 'PyQwt for numarray' >>functions with 'Numeric arrays' as arguments and vice-versa (for me it >>seems to work, but it scares me to death). >> >> > >I think this is not necessary as you can always distinguish if the user is >passing a numarray or Numeric object, like in: > >In [3]: a=Numeric.array([1,2]) > >In [4]: b=numarray.array([1,2]) > >In [5]: type(a) >Out[5]: > >In [6]: type(b) >Out[6]: > > >It's just a check before accessing the object. > >Cheers, > > > I think installing in numarray.addons (or not) is a package-by-package decision. Some packages might prefer to be independent. That's fine. Others might prefer (or require) dedicated versions, particularly those packages which use compiled extensions. For those instances where dedicated versions are necessary, numarray.addons is a recommendation to keep clear seperation between core numarray (maintained and distributed by one group) and software which has been layered on it (maintained and distributed by other groups). Todd From bsder at allcaps.org Wed May 14 13:10:16 2003 From: bsder at allcaps.org (Andrew P. Lentvorski, Jr.) Date: Wed May 14 13:10:16 2003 Subject: [Numpy-discussion] numarray repackaging one more time... In-Reply-To: <3EC22950.8070107@stsci.edu> Message-ID: On Wed, 14 May 2003, Todd Miller wrote: > Point taken. I guess that changes the addon mappings to: > > LinearAlgrebra2 --> numarray.linear_algebra > FFT2 --> numarray.fft > RandomArray2 --> numarray.random_array > Convolve --> numarray.convolve > Convolve.Image --> numarray.image > MA2 --> numarray.ma I prefer this naming scheme as it helps visually delineate between modules and classes. ie: numarray.linear_algebra.LUMatrix is likely to be a class numarray.linear_algebra.lumatrix is likely to be a module It seems that most people use StudlyCaps for class names in Python. While there aren't a lot of times that this matters, I have bumped into this more often than I expect. But I may just be doing odd things with modules. -a From bsder at allcaps.org Wed May 14 13:19:04 2003 From: bsder at allcaps.org (Andrew P. Lentvorski, Jr.) Date: Wed May 14 13:19:04 2003 Subject: [Numpy-discussion] Zeroing an existing array Message-ID: What is the official way to zero out an array in numarray/Numeric? While I can create a new array of all zeros and then assign it to the old variable, this is extremely wasteful of memory. Currently, I am just using: >>> import numarray >>> a = numarray.arange(6) >>> a array([0, 1, 2, 3, 4, 5]) >>> a[:] = 0.0 >>> a array([0, 0, 0, 0, 0, 0]) I looked through the manual for a function or array mathod which would accomplish the same thing, but I didn't find an obvious one. Did I miss something obvious? Thanks, -a From jmiller at stsci.edu Wed May 14 13:27:04 2003 From: jmiller at stsci.edu (Todd Miller) Date: Wed May 14 13:27:04 2003 Subject: [Numpy-discussion] Zeroing an existing array In-Reply-To: References: Message-ID: <1052944153.2700.23.camel@localhost.localdomain> On Wed, 2003-05-14 at 16:21, Andrew P. Lentvorski, Jr. wrote: > What is the official way to zero out an array in numarray/Numeric? > I'm don't think there is one in numarray; which is to say, I'd do what you did below. > While I can create a new array of all zeros and then assign it to the old > variable, this is extremely wasteful of memory. > > Currently, I am just using: > > >>> import numarray > >>> a = numarray.arange(6) > >>> a > array([0, 1, 2, 3, 4, 5]) > >>> a[:] = 0.0 > >>> a > array([0, 0, 0, 0, 0, 0]) > > I looked through the manual for a function or array mathod which would > accomplish the same thing, but I didn't find an obvious one. Did I miss > something obvious? I don't think so. Todd From perry at stsci.edu Wed May 14 13:39:08 2003 From: perry at stsci.edu (Perry Greenfield) Date: Wed May 14 13:39:08 2003 Subject: [Numpy-discussion] Zeroing an existing array In-Reply-To: Message-ID: > Currently, I am just using: > > >>> import numarray > >>> a = numarray.arange(6) > >>> a > array([0, 1, 2, 3, 4, 5]) > >>> a[:] = 0.0 > >>> a > array([0, 0, 0, 0, 0, 0]) > Is there any reason for anything else if this does the job? This is how I would recommend the action be performed. Perry Greenfield From verveer at embl.de Wed May 14 15:36:03 2003 From: verveer at embl.de (verveer at embl.de) Date: Wed May 14 15:36:03 2003 Subject: [Numpy-discussion] Re: Numpy-discussion digest, Vol 1 #656 - 11 msgs In-Reply-To: References: Message-ID: <1052951705.3ec2c499c4e08@webmail.embl.de> > As has been mentioned before, we're planning to repackage numarray as > a package rather than as a collection of modules. We're soliciting > comments now because we only want to do this once. [some stuff deleted] > Convolve.Image --> numarray.Image Should this not become "numarray.Convolve.Image" instead of "numarray.Image"? Peter From turner at lanl.gov Wed May 14 16:49:09 2003 From: turner at lanl.gov (John A. Turner) Date: Wed May 14 16:49:09 2003 Subject: [Numpy-discussion] using CXML on Alpha Message-ID: <3EC2D626.4090601@lanl.gov> is anyone using the Compaq Extended Math Libraries (CXML) on a Alpha running Tru64Unix? although Numeric-23.0 builds out-of-the-box and runs fine with its own BLAS/LAPACK, I wanted to try making it use the (significantly faster) CXML implementations I modified this bit of setup.py: # delete all but the first one in this list if using your own LAPACK/BLAS # sourcelist = [os.path.join('Src', 'lapack_litemodule.c'), # os.path.join('Src', 'blas_lite.c'), # os.path.join('Src', 'f2c_lite.c'), # os.path.join('Src', 'zlapack_lite.c'), # os.path.join('Src', 'dlapack_lite.c') # ] sourcelist = [os.path.join('Src', 'lapack_litemodule.c')] # set these to use your own BLAS; library_dirs_list = ['/usr/opt/XMDLIB6510'] libraries_list = ['cxml_ev6'] # if you also set `use_dotblas` (see below), you'll need: # ['lapack', 'cblas', 'f77blas', 'atlas', 'g2c'] (I had to do some hunting around to find where the libs actually live - when using CXML with Fortran all you do is put -lcxml on the command line and it links with the appropriate lib) but this doesn't seem to have done it - after I build and install I see: qsc4% python cg.py Traceback (most recent call last): File "cg.py", line 98, in ? from RandomArray import * File "/users/turner/Tools/T64U/lib/python2.2/site-packages/Numeric/RandomArray.py", line 3, in ? import LinearAlgebra File "/users/turner/Tools/T64U/lib/python2.2/site-packages/Numeric/LinearAlgebra.py", line 8, in ? import lapack_lite ImportError: dlopen: /users/turner/Tools/T64U/lib/python2.2/site-packages/Numeric/lapack_lite.so: symbol "zgelsd_" unresolved I'm pretty sure I'm missing something important... thanks in advance... -John Turner Los Alamos Natl. Lab. Adv. Sci. Sim. (CCS-2) From bsder at allcaps.org Wed May 14 18:08:06 2003 From: bsder at allcaps.org (Andrew P. Lentvorski, Jr.) Date: Wed May 14 18:08:06 2003 Subject: [Numpy-discussion] Zeroing an existing array In-Reply-To: Message-ID: On Wed, 14 May 2003, Perry Greenfield wrote: > > >>> import numarray > > >>> a = numarray.arange(6) > > >>> a > > array([0, 1, 2, 3, 4, 5]) > > >>> a[:] = 0.0 > > >>> a > > array([0, 0, 0, 0, 0, 0]) > > > Is there any reason for anything else if this does the job? > This is how I would recommend the action be performed. It depends. How does the slicing code work? If it iterates across the matrix indicies (requiring multiply and add operations to index specific memory locations), this will be much slower than doing a contiguous memory fill (1 add and 1 multiply per index vs. increment). This is not a small hit, we're talking factors of 2x, 4x, 6x. If the slicing code special cases "array[:] = n" to be a memory fill, then probably not. -a From jensj at fysik.dtu.dk Wed May 14 23:30:08 2003 From: jensj at fysik.dtu.dk (Jens Jorgen Mortensen) Date: Wed May 14 23:30:08 2003 Subject: [Numpy-discussion] using CXML on Alpha In-Reply-To: <3EC2D626.4090601@lanl.gov> References: <3EC2D626.4090601@lanl.gov> Message-ID: <200305150829.19897.jensj@bose.fysik.dtu.dk> On Torsdag den 15. maj 2003 01:49, John A. Turner wrote: > ImportError: dlopen: > /users/turner/Tools/T64U/lib/python2.2/site-packages/Numeric/lapack_lite.so >: symbol "zgelsd_" unresolved Perhaps libcxml does not does not have the zgelsd_ function? Try nm /usr/lib/libcxml_ev6.a | grep zgelds to see if it is there. Jens J?rgen From perry at stsci.edu Thu May 15 06:16:10 2003 From: perry at stsci.edu (Perry Greenfield) Date: Thu May 15 06:16:10 2003 Subject: [Numpy-discussion] Zeroing an existing array In-Reply-To: Message-ID: Andrew P. Lentvorski, Jr. wrote: > On Wed, 14 May 2003, Perry Greenfield wrote: > > > > >>> import numarray > > > >>> a = numarray.arange(6) > > > >>> a > > > array([0, 1, 2, 3, 4, 5]) > > > >>> a[:] = 0.0 > > > >>> a > > > array([0, 0, 0, 0, 0, 0]) > > > > > Is there any reason for anything else if this does the job? > > This is how I would recommend the action be performed. > > It depends. How does the slicing code work? > > If it iterates across the matrix indicies (requiring multiply and add > operations to index specific memory locations), this will be much slower > than doing a contiguous memory fill (1 add and 1 multiply per index vs. > increment). This is not a small hit, we're talking factors of 2x, 4x, 6x. > > If the slicing code special cases "array[:] = n" to be a memory fill, then > probably not. > Well, there are two issues the question relates to: 1) what should the user interface be for filling arrays with a constant value. I'd argue that a slice assignment is both sufficent and idiomatic. 2) is it efficient. As to 2, the current implementation handles scalar assignment by broadcasting a one-element array across the array being assigned to (that's the simplest way to code it). That isn't the fastest implementation when the array being assigned to is congtiguous. It could be made faster. But frankly, it isn't very high on our list of priorities. And I'm a little skeptical that this performance improvement is an important one. Is your program perfromance dominated by the time it takes to zero an array? I have trouble thinking of many realistic problems where that would be true (not to say that there aren't any). If someone is willing to write special code to optimize this, that would be fine with us. Perry From jmiller at stsci.edu Thu May 15 06:44:06 2003 From: jmiller at stsci.edu (Todd Miller) Date: Thu May 15 06:44:06 2003 Subject: [Numpy-discussion] Re: Numpy-discussion digest, Vol 1 #656 - 11 msgs In-Reply-To: <1052951705.3ec2c499c4e08@webmail.embl.de> References: <1052951705.3ec2c499c4e08@webmail.embl.de> Message-ID: <1053006256.19531.226.camel@halloween.stsci.edu> On Wed, 2003-05-14 at 18:35, verveer at embl.de wrote: > > As has been mentioned before, we're planning to repackage numarray as > > a package rather than as a collection of modules. We're soliciting > > comments now because we only want to do this once. > > [some stuff deleted] > > > Convolve.Image --> numarray.Image > > Should this not become "numarray.Convolve.Image" instead of "numarray.Image"? Currently there's one function in Image: translate. I was thinking that there are probably going to be more Image operations for work at STSCI, not necessarily all built on convolution (e.g. rotate). Since we're renaming stuff, I thought we should probably change this now. Todd > > Peter > > > > ------------------------------------------------------- > Enterprise Linux Forum Conference & Expo, June 4-6, 2003, Santa Clara > The only event dedicated to issues related to Linux enterprise solutions > www.enterpriselinuxforum.com > > _______________________________________________ > Numpy-discussion mailing list > Numpy-discussion at lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/numpy-discussion -- Todd Miller jmiller at stsci.edu STSCI / ESS / SSB From jmiller at stsci.edu Thu May 15 12:38:04 2003 From: jmiller at stsci.edu (Todd Miller) Date: Thu May 15 12:38:04 2003 Subject: [Numpy-discussion] numarray repackaging, last call... Message-ID: <1053027509.19531.334.camel@halloween.stsci.edu> Based on the comments I have received, the numarray repackaging is shaping up as shown below: Here are the planned renamings for the numarray modules as we transition to a package: numarray --> numarray.numeric recarray --> numarray.records chararray --> numarray.strings ndarray --> numarray.generic * --> numarray.* Note that class and function names will remain unchanged, so recarray.RecArray will become numarray.records.RecArray. Here are the renamings for the current and planned add-on packages consolidated into a single add-ons distribution: LinearAlgrebra2 --> numarray.linear_algebra FFT2 --> numarray.fft RandomArray2 --> numarray.random_array Convolve --> numarray.convolve Convolve.Image --> numarray.image MA2 --> numarray.ma Future 3rd party packages that require numarray specific C extensions should locate themselves directly in the numarray package. Doing so makes it easy to figure out what to delete when extensions need to be rebuilt. If you don't want to, that's OK too. Barring a sudden burst of interest or superior name choice, I think the top level package name should just remain "numarray". Stub modules will be included in the first few repackaged releases to make it possible to import the primary modules (numarray, recarray, chararray, and ndarray) as you do with numarray-0.5. This will relax the requirement for synchronous change of numarray and software which uses numarray. The stub modules will be activated either through the creation of a .pth file or via additions to PYTHONPATH to make them visible. This is a backwards compatability mode only, not recommended usage following the repackaging. Thanks to everyone who responded. Final comments? -- Todd Miller jmiller at stsci.edu STSCI / ESS / SSB From jmiller at stsci.edu Fri May 16 11:37:09 2003 From: jmiller at stsci.edu (Todd Miller) Date: Fri May 16 11:37:09 2003 Subject: [Numpy-discussion] Re: change in semantics of arange? In-Reply-To: <000001c31bcd$24ef1600$6801a8c0@NICKLEBY> References: <000001c31bcd$24ef1600$6801a8c0@NICKLEBY> Message-ID: <1053110274.993.105.camel@halloween.stsci.edu> On Fri, 2003-05-16 at 13:03, Paul Dubois wrote: > C:\numpy\numarray\Packages\MA2>python > Python 2.3b1 (#40, Apr 25 2003, 19:06:24) [MSC v.1200 32 bit (Intel)] on > win32 > Type "help", "copyright", "credits" or "license" for more information. > >>> import numarray > >>> x=numarray.arange(5,3,2) > Traceback (most recent call last): > File "", line 1, in ? > File "c:\python23\Lib\site-packages\numarray\numarray.py", line 924, in > arange > r = _fillarray(size, start, stride, type) > File "c:\python23\Lib\site-packages\numarray\numarray.py", line 120, in > _fillarray > outarr = NumArray((size,), outtype) > libnumarray.error: NA_updateDataPtr: error getting read buffer data ptr > >>> import Numeric > >>> Numeric.arange(5,3,2) > zeros((0,), 'l') > >>> > > Is this change intentional? No. It's bugs in arange and also the memory object which are creating a negative length buffer. The buffer API then reports a negative length which is fortuitously interpreted as an error. Ya gotta smile... It's fixed in CVS. I modified the memory module so that: >>> import memory >>> memory.new_memory(-10) Traceback (most recent call last): File "", line 1, in ? ValueError: new_memory: invalid region size: -10. And I modified arange so that negative sizes are clipped to 0. So now: >>> import numarray >>> numarray.arange(5,3,2) array([]) > -- Todd Miller jmiller at stsci.edu STSCI / ESS / SSB From haase at msg.ucsf.edu Tue May 20 09:19:04 2003 From: haase at msg.ucsf.edu (Sebastian Haase) Date: Tue May 20 09:19:04 2003 Subject: [Numpy-discussion] Numarray 0.5 References: <200305061424.26714.verveer@embl-heidelberg.de> <1052258543.27223.55.camel@halloween.stsci.edu> Message-ID: <009101c31eeb$90235b90$421ee6a9@rodan> Hi All, After I read this thread I thought I would wait a bit before upgrading my numarray (from 0.4) - Are the mentioned fixes available somewhere ? Or actually: Is the CVS version publicly readable and - if so - would you recommend using that ? Thanks, Sebastian Haase ----- Original Message ----- From: "Todd Miller" To: "Peter Verveer" Cc: Sent: Tuesday, May 06, 2003 3:02 PM Subject: Re: [Numpy-discussion] Numarray 0.5 > On Tue, 2003-05-06 at 08:24, Peter Verveer wrote: > > Hi All, > > > > I found the following problems after testing my software with numarray 0.5: > > > > 1) Following works fine if both a and b are arrays: > > > > >>> a = array([2]) > > >>> b = array([1, 2]) > > >>> print a + b > > [3 4] > > > > However, if b is an python sequence: > > > > >>> a = array([2]) > > >>> b = [1, 2] > > >>> print a + b > > [3] > > > > Apparently broadcasting does not work with python sequeces anymore. This used > > to work fine in version 0.4. Is this a bug? > > Yes, unfortunately. Thanks for reporting it! > > > > > 2) It is not possible to compare an array type object to the 'Any' object: > > > > >>> Float64 == Any > > Traceback (most recent call last): > > File "", line 1, in ? > > File "/usr/local/lib/python2.2/site-packages/numarray/numerictypes.py", line > > 112, in __cmp__ > > return (genericTypeRank.index(self.name) - > > ValueError: list.index(x): x not in list > > > > I am not sure if this is a bug, or intended behaviour, but the possibilty to > > compare an array type object to 'Any' would be very useful for me. > > > > 'Any' grew up from the C-API, rather than down from the Python design, > so it's not very well thought out. Right now, it is a placeholder used > to mark arrays with undefined types and to indicate "no type constraint" > in C API calls. In normal contexts, you can't make an array of type > 'Any'. I think there are two reasonable behaviors for comparisons with > 'Any', both used in C. The first behavior is literal comparison; here > comparison to Any would generally return "not equal". The second > behavior is wild-card matching; here, comparison to Any would generally > return "equal". Which makes sense to you? How do you want to use > this? > > > 3) The NA_typeNoToTypeObject() function fails if it is called before any > > arrays are created. It looks to me as if the pNumType array in > > libnumarraymodule.c is not initialized until an array is created. I don't > > know if any other functions are affected in the same way. Could this be > > fixed? > > Yes. This is fixed now in CVS. Thanks again! > > > > > Cheers, Peter > > > > -- > > Dr. Peter J. Verveer > > Cell Biology and Cell Biophysics Programme > > EMBL > > Meyerhofstrasse 1 > > D-69117 Heidelberg > > Germany > > Tel. : +49 6221 387245 > > Fax : +49 6221 387242 > > Email: Peter.Verveer at embl-heidelberg.de > > > > > > > > > > ------------------------------------------------------- > > This sf.net email is sponsored by:ThinkGeek > > Welcome to geek heaven. > > http://thinkgeek.com/sf > > _______________________________________________ > > Numpy-discussion mailing list > > Numpy-discussion at lists.sourceforge.net > > https://lists.sourceforge.net/lists/listinfo/numpy-discussion > -- > Todd Miller jmiller at stsci.edu > STSCI / ESS / SSB > > > > ------------------------------------------------------- > Enterprise Linux Forum Conference & Expo, June 4-6, 2003, Santa Clara > The only event dedicated to issues related to Linux enterprise solutions > www.enterpriselinuxforum.com > > _______________________________________________ > Numpy-discussion mailing list > Numpy-discussion at lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/numpy-discussion > From jmiller at stsci.edu Tue May 20 10:08:12 2003 From: jmiller at stsci.edu (Todd Miller) Date: Tue May 20 10:08:12 2003 Subject: [Numpy-discussion] Numarray 0.5 In-Reply-To: <009101c31eeb$90235b90$421ee6a9@rodan> References: <200305061424.26714.verveer@embl-heidelberg.de> <1052258543.27223.55.camel@halloween.stsci.edu> <009101c31eeb$90235b90$421ee6a9@rodan> Message-ID: <1053450454.25500.13.camel@halloween.stsci.edu> On Tue, 2003-05-20 at 12:19, Sebastian Haase wrote: > Hi All, > After I read this thread I thought I would wait a bit before upgrading my > numarray (from 0.4) - numarray-0.6 is probably not far off; if you're on the fence, you might want to sit this out. However, there is a new API function which I added in response to your last post: NA_NewAllFromBuffer() which enables you to create arrays in C from existing buffer objects rather than just C arrays. Also, I modified setup.py to support independent build and install. Both of these new features are under tested. Let me know how it works out. > Are the mentioned fixes available somewhere ? All of the problems Peter found have been fixed as of yesterday. The fixes are checked into CVS on Source Forge in the numarray component of the numpy project. Instructions for doing an anonymous numarray cvs checkout are here: http://sourceforge.net/cvs/?group_id=1369 These basically say to: % cvs -d:pserver:anonymous at cvs.sourceforge.net:/cvsroot/numpy login % cvs -z3 -d:pserver:anonymous at cvs.sourceforge.net:/cvsroot/numpy co numarray > Or actually: Is the CVS version publicly readable and - if so - would you > recommend using that ? If you check out today, you'll be OK. I tagged "now" as v0_5_2 which will never be "officially released" but which just passed all self-tests under i386-Linux. Todd -- Todd Miller jmiller at stsci.edu STSCI / ESS / SSB From camartin at snet.net Wed May 21 08:09:02 2003 From: camartin at snet.net (Cliff Martin) Date: Wed May 21 08:09:02 2003 Subject: [Numpy-discussion] Finding indices and setting values Message-ID: <3ECB9685.6060809@snet.net> Hi, I have a problem where I want to set values in a 2D array based on conditions I?ve established. The variables are the NA of the system, its wavelength and the correct units on the device I?m modeling. Using these variables I define a variable N. I set up a 512 by 512 array [x,y]. Then I set up a radius , r = x^2 +y^2 and ask it to give me all r?s(to the closest integer) <= N(the variable I?ve defined above). This gives all the index values in that radius and then I set all those locations to a value of 1. After this I do some FFT ?s, etc. So how do I do this in Numerical Python without having to index through i,j steps which would be incredibly tedious. This works fairly well in MatLab but I want to port my program to Python(for lots of reasons). If you?d rather I write the MatLab code snippet I can do that. Thanks. Cliff Martin From perry at stsci.edu Wed May 21 08:42:07 2003 From: perry at stsci.edu (Perry Greenfield) Date: Wed May 21 08:42:07 2003 Subject: [Numpy-discussion] Finding indices and setting values In-Reply-To: <3ECB9685.6060809@snet.net> Message-ID: > > I have a problem where I want to set values in a 2D array based on > conditions I?ve established. The variables are the NA of the system, its > wavelength and the correct units on the device I?m modeling. Using these > variables I define a variable N. I set up a 512 by 512 array [x,y]. Then > I set up a radius , r = x^2 +y^2 and ask it to give me all r?s(to the > closest integer) <= N(the variable I?ve defined above). This gives all > the index values in that radius and then I set all those locations to a > value of 1. After this I do some FFT ?s, etc. So how do I do this in > Numerical Python without having to index through i,j steps which would > be incredibly tedious. This works fairly well in MatLab but I want to > port my program to Python(for lots of reasons). If you?d rather I write > the MatLab code snippet I can do that. Thanks. > > Cliff Martin > One way should be something like this (I haven't tested this) This only works in numarray. If data is the 2-d array where you want to set values within a radius of N (of x0, y0 I presume) y, x = indices(data.shape) data[nonzero(((x-x0)**2+(y-y0)**2) < N**2)] = 1 (in a future version of numarray this will also work and the meaning should be clearer: data[where(((x-x0)**2+(y-y0)**2) < N**2)] = 1 It's a little more involved for Numeric but still possible to do without resorting to loops (I'll leave it to someone else to show that). Does this do what you wanted? Perry Greenfield. From jmiller at stsci.edu Wed May 21 09:01:14 2003 From: jmiller at stsci.edu (Todd Miller) Date: Wed May 21 09:01:14 2003 Subject: [Numpy-discussion] Finding indices and setting values In-Reply-To: <3ECB9685.6060809@snet.net> References: <3ECB9685.6060809@snet.net> Message-ID: <1053531800.1072.47.camel@localhost.localdomain> On Wed, 2003-05-21 at 11:08, Cliff Martin wrote: > Hi, > > I have a problem where I want to set values in a 2D array based on > conditions I?ve established. The variables are the NA of the system, its > wavelength and the correct units on the device I?m modeling. Using these > variables I define a variable N. I set up a 512 by 512 array [x,y]. Then > I set up a radius , r = x^2 +y^2 and ask it to give me all r?s(to the > closest integer) <= N(the variable I?ve defined above). This gives all > the index values in that radius and then I set all those locations to a > value of 1. After this I do some FFT ?s, etc. So how do I do this in > Numerical Python without having to index through i,j steps which would > be incredibly tedious. This works fairly well in MatLab but I want to > port my program to Python(for lots of reasons). If you?d rather I write > the MatLab code snippet I can do that. Thanks. > > Cliff Martin I think part of your code looks like this: x,y = Numeric.indices((512,512)) r = x**2 + y**2 c = r < N In numarray you can then say: a[ c ] = 1 In Numeric, I think you say: a = a.flat c = c.flat Numeric.put(a, Numeric.nonzero(c), 1) a.shape = (512,512) Todd From magnus at hetland.org Wed May 21 11:53:04 2003 From: magnus at hetland.org (Magnus Lie Hetland) Date: Wed May 21 11:53:04 2003 Subject: [Numpy-discussion] Finding indices and setting values In-Reply-To: References: <3ECB9685.6060809@snet.net> Message-ID: <20030521185206.GC10762@idi.ntnu.no> Perry Greenfield : [snip] > (in a future version of numarray this will also work and the meaning > should be clearer: data[where(((x-x0)**2+(y-y0)**2) < N**2)] = 1 Cool... Do you know which version? -- Magnus Lie Hetland "In this house we obey the laws of http://hetland.org thermodynamics!" Homer Simpson From Kasper.Souren at ircam.fr Wed May 21 12:56:05 2003 From: Kasper.Souren at ircam.fr (Kasper Souren) Date: Wed May 21 12:56:05 2003 Subject: [Numpy-discussion] faster way for calculating distance matrix... Message-ID: <200305211943.17529.Kasper.Souren@ircam.fr> Hi, I was happy to see solutions coming up for Cliff's problems and it made me think: maybe I can mention my problem here as well. From an array of feature vectors I want to calculate it's distance matrix. Something like [1]. Currently it can take quite a while to calculate the stuff for a long array. Some questions: 1) Is there a smart speed up possible? Like, a way to avoid the double loop? It's no problem if this would lead to less generality (like the choice for a distance function). I know there is a little speed up to be gained by leaving out the array(f) thing, but that's not what I'm looking for. 2) Is it possible (in Numeric or numarray) to define a class DiagonalMatrix that at least saves half of the memory? 3) If 1) is not possible, what would be the way to go for speeding it up by writing it in C? weave because of its availability in scipy or would pyrex be more interesting, or are there even more options..? bye, Kasper [1] Example program: import Numeric def euclidean_dist(a, b): diff = a - b return Numeric.dot(diff, diff) def calc_dist_matrix(f, distance_function): W = Numeric.array(f) length = W.shape[0] S = Numeric.zeros((length, length)) * 1.0 for i in range(length): for j in range(i): S[j, i] = S[i, j] = distance_function(W[i], W[j]) return S print calc_dist_matrix(Numeric.sin(Numeric.arange(30)), euclidean_dist) From perry at stsci.edu Wed May 21 14:10:06 2003 From: perry at stsci.edu (Perry Greenfield) Date: Wed May 21 14:10:06 2003 Subject: [Numpy-discussion] Finding indices and setting values In-Reply-To: <1053531800.1072.47.camel@localhost.localdomain> Message-ID: > > x,y = Numeric.indices((512,512)) > r = x**2 + y**2 > c = r < N > > In numarray you can then say: > > a[ c ] = 1 > > In Numeric, I think you say: > > a = a.flat > c = c.flat > Numeric.put(a, Numeric.nonzero(c), 1) > a.shape = (512,512) > > > Todd > Yeah, that's what I would do in Numeric :-) I was thinking that the solution I gave was wasteful of memory if N was generally much smaller than the dimensions of data. Something like this (also untested) would be better in that regard (again for numarray) y, x = indices((2*N+1, 2*N+1)) yind, xind = nonzero(((x-N)**2+(y-N)**2) < N**2) data[yind+y0-N, xind+x0-N] = 1 Note that this doesn't check to see if x0 and y0 are at least N away from the boundaries of data, and also note my convention for x and y is different than what Todd used (it depends on how you interpret data in arrays, the convention I use is more common for image data if you think of x corresponding to the most rapidly varying index). The advantage of this approach is that the x and y arrays are small if N is small, and computing xind, yind take much less time than for very large arrays. Doing this with Numeric would be much messier I believe (but a clever person could prove me wrong). Perry Greenfield From jcollins at boulder.net Wed May 21 18:03:02 2003 From: jcollins at boulder.net (Jeffery D. Collins) Date: Wed May 21 18:03:02 2003 Subject: [Numpy-discussion] faster way for calculating distance matrix... In-Reply-To: <200305211943.17529.Kasper.Souren@ircam.fr> References: <200305211943.17529.Kasper.Souren@ircam.fr> Message-ID: <3ECC220E.3090700@boulder.net> Kasper Souren wrote: >Hi, > >I was happy to see solutions coming up for Cliff's problems and it made me >think: maybe I can mention my problem here as well. > >>From an array of feature vectors I want to calculate it's distance matrix. >Something like [1]. Currently it can take quite a while to calculate the >stuff for a long array. > >Some questions: > >1) Is there a smart speed up possible? Like, a way to avoid the double loop? >It's no problem if this would lead to less generality (like the choice for a >distance function). I know there is a little speed up to be gained by leaving >out the array(f) thing, but that's not what I'm looking for. > >2) Is it possible (in Numeric or numarray) to define a class DiagonalMatrix >that at least saves half of the memory? > >3) If 1) is not possible, what would be the way to go for speeding it up by >writing it in C? weave because of its availability in scipy or would pyrex be >more interesting, or are there even more options..? > >bye, >Kasper > > > >[1] Example program: > >import Numeric > >def euclidean_dist(a, b): > diff = a - b > return Numeric.dot(diff, diff) > >def calc_dist_matrix(f, distance_function): > W = Numeric.array(f) > length = W.shape[0] > S = Numeric.zeros((length, length)) * 1.0 > > for i in range(length): > for j in range(i): > S[j, i] = S[i, j] = distance_function(W[i], W[j]) > return S > >print calc_dist_matrix(Numeric.sin(Numeric.arange(30)), euclidean_dist) > > The following is a bit faster (at the cost of some memory): def euclidean_dist(a, b): diff = a - b return diff**2 def calc_dist_matrix2(f, distance_function): W = Numeric.array(f) i,j = indices((len(W),)*2) S = distance_function(take(W,i), take(W,j)) return S -- Jeffery Collins (http://www.boulder.net/~jcollins) From christopheranderson at shaw.ca Tue May 27 08:55:15 2003 From: christopheranderson at shaw.ca (christopheranderson at shaw.ca) Date: Tue May 27 08:55:15 2003 Subject: [Numpy-discussion] Numeric -> numarray gives MemoryError Message-ID: Hi, I've started to move some of my Numeric code to numarray 0.5 and have run into a problem. This is a relatively large optimization problem that requires finite difference estimation of gradients. The implementation uses either Numeric/LinearAlgebra/RandomArray or numarray/LinearAlgebra2/RandomArray2; these are the only extensions used. The code itself is fairly straightforward and ports from Numeric to numarray with no modifications. Typical run times are on the order of hours with Numeric. When using Numeric, I am able to repeat multiple optimization runs (> 10) with no problems. When the same code is run with numarray instead, I get this error part of the way through the first run: Traceback (most recent call last): File "runFD2.py", line 81, in ? W1, W2 = estimFD2(inputs, outputs, Nh, Ne, alpha) File "optimFD2.py", line 171, in estimFD2 gradW1, gradW2 = dEdW(inputs, outputs, MSE, alpha, W1, W2, f1, f2) File "optimFD2.py", line 83, in dEdW y_plus = evalFD2(inputs, W1_plus, W2, f1, f2) File "optimFD2.py", line 63, in evalFD2 h1 = dot(inputs, W1) File "F:\Python22\Lib\site-packages\numarray\numarray.py", line 940, in dot return innerproduct(a, swapaxes(inputarray(b), -1, -2)) File "F:\Python22\Lib\site-packages\numarray\ufunc.py", line 1866, in innerproduct a = a.astype(rtype) File "F:\Python22\Lib\site-packages\numarray\numarray.py", line 478, in astype return self.copy() File "F:\Python22\Lib\site-packages\numarray\numarray.py", line 553, in copy c = ndarray.NDArray.copy(self) File "F:\Python22\Lib\site-packages\numarray\ndarray.py", line 571, in copy arr._data = memory.new_memory(arr._itemsize * arr.nelements()) MemoryError >>> This is a little frustrating, because numarray is clearly much faster. Checking the results along the way shows that numarray gives the same outputs as Numeric, so it doesn't appear to be a porting issue. The datasets involved are large (~10000x20). Initial memory usage with Numeric and numarray is similar, and is well below the machine's limit. The code is running on a Windows 2000 machine using the binary release of numarray. What might be causing this problem? Should numarray be less tolerant than Numeric in a situation like this? Thanks. Chris From jmiller at stsci.edu Tue May 27 09:48:09 2003 From: jmiller at stsci.edu (Todd Miller) Date: Tue May 27 09:48:09 2003 Subject: [Numpy-discussion] Numeric -> numarray gives MemoryError In-Reply-To: References: Message-ID: <1054054030.28955.27.camel@halloween.stsci.edu> On Tue, 2003-05-27 at 11:52, christopheranderson at shaw.ca wrote: > Hi, > > I've started to move some of my Numeric code to numarray 0.5 and have run into a problem. This is a relatively large optimization problem that requires finite difference estimation of gradients. > > The implementation uses either Numeric/LinearAlgebra/RandomArray or numarray/LinearAlgebra2/RandomArray2; these are the only extensions used. The code itself is fairly straightforward and ports from Numeric to numarray with no modifications. Typical run times are on the order of hours with Numeric. > > When using Numeric, I am able to repeat multiple optimization runs (> 10) with no problems. When the same code is run with numarray instead, I get this error part of the way through the first run: > > Traceback (most recent call last): > File "runFD2.py", line 81, in ? > W1, W2 = estimFD2(inputs, outputs, Nh, Ne, alpha) > File "optimFD2.py", line 171, in estimFD2 > gradW1, gradW2 = dEdW(inputs, outputs, MSE, alpha, W1, W2, f1, f2) > > File "optimFD2.py", line 83, in dEdW > y_plus = evalFD2(inputs, W1_plus, W2, f1, f2) > File "optimFD2.py", line 63, in evalFD2 > h1 = dot(inputs, W1) > File "F:\Python22\Lib\site-packages\numarray\numarray.py", line 940, in dot > return innerproduct(a, swapaxes(inputarray(b), -1, -2)) > File "F:\Python22\Lib\site-packages\numarray\ufunc.py", line 1866, in innerproduct > a = a.astype(rtype) > File "F:\Python22\Lib\site-packages\numarray\numarray.py", line 478, in astype > > return self.copy() > File "F:\Python22\Lib\site-packages\numarray\numarray.py", line 553, in copy > c = ndarray.NDArray.copy(self) > File "F:\Python22\Lib\site-packages\numarray\ndarray.py", line 571, in copy > arr._data = memory.new_memory(arr._itemsize * arr.nelements()) > MemoryError > >>> > > This is a little frustrating, because numarray is clearly much faster. Checking the results along the way shows that numarray gives the same outputs as Numeric, so it doesn't appear to be a porting issue. The datasets involved are large (~10000x20). Initial memory usage with Numeric and numarray is similar, and is well below the machine's limit. The code is running on a Windows 2000 machine using the binary release of numarray. > Well... the faster part is good. :) > What might be causing this problem? It sounds to me like a memory leak (reference counting error) in numarray. Did you run the same code with older versions of numarray, or did you start with numarray-0.5? How long does it take to fail? Can you mail me your code (or better, the smallest derivative of it which reproduces the problem)? > Should numarray be less tolerant than Numeric in a situation like this? No, it's just newer and less mature. > > Thanks. You're welcome! Todd > > Chris > > > > > ------------------------------------------------------- > This SF.net email is sponsored by: ObjectStore. > If flattening out C++ or Java code to make your application fit in a > relational database is painful, don't do it! Check out ObjectStore. > Now part of Progress Software. http://www.objectstore.net/sourceforge > _______________________________________________ > Numpy-discussion mailing list > Numpy-discussion at lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/numpy-discussion -- Todd Miller jmiller at stsci.edu STSCI / ESS / SSB From christopheranderson at shaw.ca Tue May 27 13:24:04 2003 From: christopheranderson at shaw.ca (christopheranderson at shaw.ca) Date: Tue May 27 13:24:04 2003 Subject: [Numpy-discussion] Re: Numeric -> numarray gives MemoryError Message-ID: <5c8ec60da7.60da75c8ec@shaw.ca> Todd, thanks very much for the quick reply. > >Well... the faster part is good. :) Yes indeed. > > > What might be causing this problem? > >It sounds to me like a memory leak (reference counting error) in >numarray. > >Did you run the same code with older versions of numarray, or did you >start with numarray-0.5? > >How long does it take to fail? This was my first stab at using numarray, so I haven't tried with prior versions. Not that this means anything, but it usually dies around 500 or so iterations through the optimization routine. For comparison, Numeric will cheerfully run for several thousands of iterations, multiple times, without complaint. >Can you mail me your code (or better, the smallest derivative of it >which reproduces the problem)? This might be a little complicated. I'll try stripping things down to isolate the problem. Chris From camartin at snet.net Thu May 29 13:17:04 2003 From: camartin at snet.net (Cliff Martin) Date: Thu May 29 13:17:04 2003 Subject: [Numpy-discussion] fftshift Message-ID: <3ED66AB5.40209@snet.net> All, First I'd like to thank Paul Dubois, Todd Miller and Perry Greenfield for the help you gave me on setting defined values based on index values in an array. Your fixes worked and let me get done what was needed. This leads me to my second problem. In Matlab there is a function called fftshift that lets one shift the fft values for more easy visualization. Numeric doesn't seem to have a function like that explicitly although I suppose one could use the flip function in the MatPy module. Has anyone implemented something like the fftshift function or could you suggest the best way to do this? Thanks for your help. Oh I'm doing this on a 2D array. Cliff Martin From pearu at cens.ioc.ee Thu May 29 13:48:02 2003 From: pearu at cens.ioc.ee (Pearu Peterson) Date: Thu May 29 13:48:02 2003 Subject: [Numpy-discussion] fftshift In-Reply-To: <3ED66AB5.40209@snet.net> Message-ID: On Thu, 29 May 2003, Cliff Martin wrote: > This leads me to my second problem. In Matlab there is a function > called fftshift that lets one shift the fft values for more easy > visualization. Numeric doesn't seem to have a function like that > explicitly although I suppose one could use the flip function in the > MatPy module. Has anyone implemented something like the fftshift > function or could you suggest the best way to do this? Thanks for your > help. Oh I'm doing this on a 2D array. scipy.fftpack has fftshift. If you don't care building scipy yourself then the implementation of fftshift can be viewed via CVSview at www.scipy.org, look at the file scipy/Lib/fftpack/helper.py HTH, Pearu From james at hello.com Fri May 2 04:07:02 2003 From: james at hello.com (james at hello.com) Date: Fri May 2 04:07:02 2003 Subject: [Numpy-discussion] Italian-crafted Rolex - only $65 - $140!! Free SHIPPING ! ! Message-ID: An HTML attachment was scrubbed... URL: From mtsbra003 at mail.uct.ac.za Fri May 2 08:25:08 2003 From: mtsbra003 at mail.uct.ac.za (mtsbra003 at mail.uct.ac.za) Date: Fri May 2 08:25:08 2003 Subject: [Numpy-discussion] numpy problem In-Reply-To: References: Message-ID: <1051888620.3eb28beccd666@intranet.uct.ac.za> Hello I have installed python,numpy(latest),and numtut. So when testing the package could see the image that was supposed to come out. Per instruction I had to do the following: 1. from NumTut import* , 2. view(greece). But view brought nothing. Thanking you in advance Thats Bramley ------------------------------------------------- This mail sent through IMP: http://horde.org/imp/ From jmiller at stsci.edu Mon May 5 05:12:07 2003 From: jmiller at stsci.edu (Todd Miller) Date: Mon May 5 05:12:07 2003 Subject: [Numpy-discussion] Numarray-0.5 released Message-ID: <1052136718.23013.4.camel@halloween.stsci.edu> Release Notes for numarray-0.5 Numarray is an array processing package designed to efficiently manipulate large multi-dimensional arrays. Numarray is modelled after Numeric and features c-code generated from python template scripts, the capacity to operate directly on arrays in files, and improved type promotions. I. ENHANCEMENTS 1. Universal Function Overhead Reduction The constant time overhead for most universal functions has been reduced by a factor of 10-20. This results in better performance for small arrays. 2. FFT mode and IRAF boundary modes added to Convolve.convolve2d There's now an FFT switch for the 2d convolution function in the Convolve package; when set to 1, convolution is performed via the FFT rather than using the naiive algorithm. In addition, convolve2d now supports boundary modes which are identical to IRAF's convolution function. 3. Numarray is now supported by f2py Numarray numerical arrays can now be used with f2py wrappers for Fortran code. To compile f2py wrapped extensions for numarray, use the switch -DNUMARRAY on the f2py command line. Support is currently limited to f77 and numerical arrays. II. BUGS FIXED 650926 exotic type coercions 653424 Convolve.boxcar boundary bug 653429 python setup.py build 654669 array index by list 655942 logical operator reductions 657058 inverse real fft bug 677796 byteswap not working 709956 error summing over large boolean arrays 710480 MLab.median makes unnecessary msort call 714537 conjugate function changes its argument See http://sourceforge.net/tracker/?atid=450446&group_id=1369&func=browse for more details. III. CAUTIONS 1. Due to some module renamings, numarray-0.5 will not install correctly on top of an existing numarray installation. Before installing numarray-0.5 remove your old version of numarray. 2. Due to reorganization of the C-API, numarray extensions must be recompiled. 3. For numarray-0.5 and up, the byteswap related methods have been redefined: a.byteswap() swaps but leaves byteorder alone a.togglebyteorder() Does "big" <-> "little" a._byteswap() now an alias for byteswap a._togglebyteorder() deleted 4. round() has been deprecated. Use around() instead. 5. Installing from source, the first time you run setup.py, you must specify --gencode, e.g.: python setup.py install --gencode WHERE ----------- Numarray-0.5 windows executable installers, source code, and manual is here: http://sourceforge.net/project/showfiles.php?group_id=1369 Numarray is hosted by Source Forge in the same project which hosts Numeric: http://sourceforge.net/projects/numpy/ The web page for Numarray information is at: http://stsdas.stsci.edu/numarray/index.html Trackers for Numarray Bugs, Feature Requests, Support, and Patches are at the Source Forge project for NumPy at: http://sourceforge.net/tracker/?group_id=1369 REQUIREMENTS ------------------------------ numarray-0.5 requires Python 2.2.2 or greater. AUTHORS, LICENSE ------------------------------ Numarray was written by Perry Greenfield, Rick White, Todd Miller, JC Hsu, Paul Barrett, Phil Hodge at the Space Telescope Science Institute. Thanks go to Jochen Kupper of the University of North Carolina for his work on Numarray and for porting the Numarray manual to TeX format. Thanks also to Edward C. Jones, Francesc Alted, Paul Dubois, Eric Jones, Travis Oliphant, Pearu Peterson and everyone who has contributed with comments and feedback. Numarray is made available under a BSD-style License. See LICENSE.txt in the source distribution for details. -- Todd Miller jmiller at stsci.edu STSCI / ESS / SSB From haase at msg.ucsf.edu Mon May 5 09:36:06 2003 From: haase at msg.ucsf.edu (Sebastian Haase) Date: Mon May 5 09:36:06 2003 Subject: [Numpy-discussion] http://stsdas.stsci.edu/numarray outdated References: <1052136718.23013.4.camel@halloween.stsci.edu> Message-ID: <018701c31324$7c7293e0$421ee6a9@rodan> Hi all, I'm happy to hear about the new release ! Could someone put this information on the web ? http://stsdas.stsci.edu/numarray/ still dates to 2002 Aug (which is before 0.4 !!) Or is this page the wrong page anyway -- I just end up there because google points into that direction when you ask for 'numarray' Thanks for the great work. Sebastian ----- Original Message ----- From: "Todd Miller" To: Sent: Monday, May 05, 2003 5:11 AM Subject: [Numpy-discussion] Numarray-0.5 released > Release Notes for numarray-0.5 > > Numarray is an array processing package designed to efficiently > manipulate large multi-dimensional arrays. Numarray is modelled after > Numeric and features c-code generated from python template scripts, > the capacity to operate directly on arrays in files, and improved type > promotions. > > I. ENHANCEMENTS > > 1. Universal Function Overhead Reduction > > The constant time overhead for most universal functions has been reduced > by a factor of 10-20. This results in better performance for small > arrays. > > 2. FFT mode and IRAF boundary modes added to Convolve.convolve2d > > There's now an FFT switch for the 2d convolution function in the > Convolve package; when set to 1, convolution is performed via the FFT > rather than using the naiive algorithm. In addition, convolve2d now > supports boundary modes which are identical to IRAF's convolution > function. > > 3. Numarray is now supported by f2py > > Numarray numerical arrays can now be used with f2py wrappers for > Fortran code. To compile f2py wrapped extensions for numarray, > use the switch -DNUMARRAY on the f2py command line. Support > is currently limited to f77 and numerical arrays. > > > II. BUGS FIXED > > 650926 exotic type coercions > 653424 Convolve.boxcar boundary bug > 653429 python setup.py build > 654669 array index by list > 655942 logical operator reductions > 657058 inverse real fft bug > 677796 byteswap not working > 709956 error summing over large boolean arrays > 710480 MLab.median makes unnecessary msort call > 714537 conjugate function changes its argument > > See > http://sourceforge.net/tracker/?atid=450446&group_id=1369&func=browse > for more details. > > III. CAUTIONS > > 1. Due to some module renamings, numarray-0.5 will not install > correctly on top of an existing numarray installation. Before > installing numarray-0.5 remove your old version of numarray. > > 2. Due to reorganization of the C-API, numarray extensions must be > recompiled. > > 3. For numarray-0.5 and up, the byteswap related methods have been > redefined: > > a.byteswap() swaps but leaves byteorder alone > a.togglebyteorder() Does "big" <-> "little" > a._byteswap() now an alias for byteswap > a._togglebyteorder() deleted > > 4. round() has been deprecated. Use around() instead. > > 5. Installing from source, the first time you run setup.py, you must > specify --gencode, e.g.: python setup.py install --gencode > > WHERE > ----------- > > Numarray-0.5 windows executable installers, source code, and manual is > here: > > http://sourceforge.net/project/showfiles.php?group_id=1369 > > Numarray is hosted by Source Forge in the same project which hosts > Numeric: > > http://sourceforge.net/projects/numpy/ > > The web page for Numarray information is at: > > http://stsdas.stsci.edu/numarray/index.html > > Trackers for Numarray Bugs, Feature Requests, Support, and Patches are > at > the Source Forge project for NumPy at: > > http://sourceforge.net/tracker/?group_id=1369 > > REQUIREMENTS > ------------------------------ > > numarray-0.5 requires Python 2.2.2 or greater. > > > AUTHORS, LICENSE > ------------------------------ > > Numarray was written by Perry Greenfield, Rick White, Todd Miller, JC > Hsu, Paul Barrett, Phil Hodge at the Space Telescope Science > Institute. Thanks go to Jochen Kupper of the University of North > Carolina for his work on Numarray and for porting the Numarray manual > to TeX format. Thanks also to Edward C. Jones, Francesc Alted, > Paul Dubois, Eric Jones, Travis Oliphant, Pearu Peterson and everyone > who has contributed with comments and feedback. > > Numarray is made available under a BSD-style License. See > LICENSE.txt in the source distribution for details. > > -- > Todd Miller jmiller at stsci.edu > STSCI / ESS / SSB > > > > ------------------------------------------------------- > This sf.net email is sponsored by:ThinkGeek > Welcome to geek heaven. > http://thinkgeek.com/sf > _______________________________________________ > Numpy-discussion mailing list > Numpy-discussion at lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/numpy-discussion > From perry at stsci.edu Mon May 5 10:23:12 2003 From: perry at stsci.edu (Perry Greenfield) Date: Mon May 5 10:23:12 2003 Subject: [Numpy-discussion] [numarray issue] nonzero? Comments on new name solicited In-Reply-To: <018701c31324$7c7293e0$421ee6a9@rodan> Message-ID: [In the next week or so I am going to be raising some issues regarding the next numarray release, 0.6. By the way, we keep the version number less than 1.0 because we still hold out the possiblity of changing some interface issues (generally it has been in the direction of making numarray more compatible with Numeric). The following raises one of these interface issues. numarray has a more general approach to the nonzero function than does Numeric. It is able to return multiple index arrays for multidimensional arrays (where as nonzero only works for 1-d arrays). In the interest of giving a more generic interface for the function, numarray always returns the index array(s) as part of a tuple, even if the argument to nonzero is 1-D and there is only one resulting index array. However, this is a problem for Numeric old code that uses nonzero and expects an array instead of a tuple. The sensible thing to do is to retain compatibilty with the Numeric nonzero (and likewise only support 1-d arrays). So we need a new name for our new nonzero function. To tell you the truth, I always felt that nonzero was a terrible name anyway and much preferred the name that IDL used for that function, namely "where". Alas, "where" is already used for another purpose in Numeric. But to illustrate, which is clearer? x[nonzero(x > 0.)] = 0. or x[where(x > 0.)] = 0 So, we considered some other alternatives, in particular nz wheretrue but our favorite is: "at". For example: x[at(x > 0.)] = 0. This has the benefit of being short and having a fairly clear meaning. Short is important since it will keep expressions of arrays using index arrays short and more readable. True, this is more likely to conflict with existing user variable or function names, but if one is simply trying to use existing files, then there is no problem since they won't need the "at" function since its functionality was not previously available. Any objections to using "at"? Any better suggestions. Of course, these changes will break existing numarray code that use the nonzero function. Perry Greenfield From pearu at cens.ioc.ee Mon May 5 13:27:07 2003 From: pearu at cens.ioc.ee (Pearu Peterson) Date: Mon May 5 13:27:07 2003 Subject: [Numpy-discussion] Re: [SciPy-user] [numarray issue] nonzero? Comments on new name solicited In-Reply-To: Message-ID: On Mon, 5 May 2003, Perry Greenfield wrote: > Any objections to using "at"? Any better suggestions. For me 'wheretrue' sounds most accurate but I agree, it's longish. Its short form 'where' is less accurate (because of its general meaning) but not too confusing. At first, 'at' confused me a little... until I figured out its long version 'trueat'. Hmm, I have always assumed that 'nonzero(x)' means elementwise 'not not x' (like Python __nonzero__ is equivalent to 'not not') and therefore never felt the need to use it. But now I see I was wrong! So, 'nonzero' sounded and still sounds misleading for me. For 'nz' I am -1 because 'nz' can often be a suitable variable name. In summary, I would like 'trueat' the most and 'nonzero' the least. 'where' and 'at' (in that order) are somewhere in between. Pearu From haase at msg.ucsf.edu Mon May 5 18:25:02 2003 From: haase at msg.ucsf.edu (Sebastian Haase) Date: Mon May 5 18:25:02 2003 Subject: [Numpy-discussion] numarray 0.5 -> python setup.py build fails References: <1052136718.23013.4.camel@halloween.stsci.edu> Message-ID: <003101c3136e$3d7d2d90$421ee6a9@rodan> Hi, I just tried this : haase at baboon:~/numarray-0.5: python2.2 setup.py build --gencode creating /usr/lib/python2.2/site-packages/numarray error: could not create '/usr/lib/python2.2/site-packages/numarray': Permission denied I guess this is the same bug I ran into a couple month ago - I already now a workaround ... - Sebastian ----- Original Message ----- From: "Todd Miller" To: Sent: Monday, May 05, 2003 5:11 AM Subject: [Numpy-discussion] Numarray-0.5 released > Release Notes for numarray-0.5 > > Numarray is an array processing package designed to efficiently > manipulate large multi-dimensional arrays. Numarray is modelled after > Numeric and features c-code generated from python template scripts, > the capacity to operate directly on arrays in files, and improved type > promotions. > > I. ENHANCEMENTS > > 1. Universal Function Overhead Reduction > > The constant time overhead for most universal functions has been reduced > by a factor of 10-20. This results in better performance for small > arrays. > > 2. FFT mode and IRAF boundary modes added to Convolve.convolve2d > > There's now an FFT switch for the 2d convolution function in the > Convolve package; when set to 1, convolution is performed via the FFT > rather than using the naiive algorithm. In addition, convolve2d now > supports boundary modes which are identical to IRAF's convolution > function. > > 3. Numarray is now supported by f2py > > Numarray numerical arrays can now be used with f2py wrappers for > Fortran code. To compile f2py wrapped extensions for numarray, > use the switch -DNUMARRAY on the f2py command line. Support > is currently limited to f77 and numerical arrays. > > > II. BUGS FIXED > > 650926 exotic type coercions > 653424 Convolve.boxcar boundary bug > 653429 python setup.py build > 654669 array index by list > 655942 logical operator reductions > 657058 inverse real fft bug > 677796 byteswap not working > 709956 error summing over large boolean arrays > 710480 MLab.median makes unnecessary msort call > 714537 conjugate function changes its argument > > See > http://sourceforge.net/tracker/?atid=450446&group_id=1369&func=browse > for more details. > > III. CAUTIONS > > 1. Due to some module renamings, numarray-0.5 will not install > correctly on top of an existing numarray installation. Before > installing numarray-0.5 remove your old version of numarray. > > 2. Due to reorganization of the C-API, numarray extensions must be > recompiled. > > 3. For numarray-0.5 and up, the byteswap related methods have been > redefined: > > a.byteswap() swaps but leaves byteorder alone > a.togglebyteorder() Does "big" <-> "little" > a._byteswap() now an alias for byteswap > a._togglebyteorder() deleted > > 4. round() has been deprecated. Use around() instead. > > 5. Installing from source, the first time you run setup.py, you must > specify --gencode, e.g.: python setup.py install --gencode > > WHERE > ----------- > > Numarray-0.5 windows executable installers, source code, and manual is > here: > > http://sourceforge.net/project/showfiles.php?group_id=1369 > > Numarray is hosted by Source Forge in the same project which hosts > Numeric: > > http://sourceforge.net/projects/numpy/ > > The web page for Numarray information is at: > > http://stsdas.stsci.edu/numarray/index.html > > Trackers for Numarray Bugs, Feature Requests, Support, and Patches are > at > the Source Forge project for NumPy at: > > http://sourceforge.net/tracker/?group_id=1369 > > REQUIREMENTS > ------------------------------ > > numarray-0.5 requires Python 2.2.2 or greater. > > > AUTHORS, LICENSE > ------------------------------ > > Numarray was written by Perry Greenfield, Rick White, Todd Miller, JC > Hsu, Paul Barrett, Phil Hodge at the Space Telescope Science > Institute. Thanks go to Jochen Kupper of the University of North > Carolina for his work on Numarray and for porting the Numarray manual > to TeX format. Thanks also to Edward C. Jones, Francesc Alted, > Paul Dubois, Eric Jones, Travis Oliphant, Pearu Peterson and everyone > who has contributed with comments and feedback. > > Numarray is made available under a BSD-style License. See > LICENSE.txt in the source distribution for details. > > -- > Todd Miller jmiller at stsci.edu > STSCI / ESS / SSB > > > > ------------------------------------------------------- > This sf.net email is sponsored by:ThinkGeek > Welcome to geek heaven. > http://thinkgeek.com/sf > _______________________________________________ > Numpy-discussion mailing list > Numpy-discussion at lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/numpy-discussion > From hinsen at cnrs-orleans.fr Tue May 6 03:27:04 2003 From: hinsen at cnrs-orleans.fr (Konrad Hinsen) Date: Tue May 6 03:27:04 2003 Subject: [Numpy-discussion] Python global interpreter lock ... In-Reply-To: References: Message-ID: <200305061227.13910.hinsen@cnrs-orleans.fr> On Wednesday 30 April 2003 08:47, Andrew P. Lentvorski, Jr. wrote: > 1) Does numpy (or numarray) release the Python GIL? No. > 2) If not, why not? Is this simply something I could fix by making the > changes myself, or are there larger reasons for not releasing the GIL? There is one good reason, which is that the GIL release code doesn't nest. If NumPy released the GIL during its operations, then any C code that uses NumPy functions would have to reacquire the GIL before calling NumPy, then re-release the GIL afterwards. In most real-life situations, it is simpler and presumably more efficient to release the GIL once for a block of numerical computations, which would include many calls to NumPy functions. One could of course release the GIL not within the functions that are part of the NumPy C API, but in a thin wrapper around it that is directly called from Python. The only other problem that I see is the general object type arrays, which make calls back into the Python interpreter. Konrad. -- ------------------------------------------------------------------------------- Konrad Hinsen | E-Mail: hinsen at cnrs-orleans.fr Centre de Biophysique Moleculaire (CNRS) | Tel.: +33-2.38.25.56.24 Rue Charles Sadron | Fax: +33-2.38.63.15.17 45071 Orleans Cedex 2 | Deutsch/Esperanto/English/ France | Nederlands/Francais ------------------------------------------------------------------------------- From verveer at embl-heidelberg.de Tue May 6 05:25:07 2003 From: verveer at embl-heidelberg.de (Peter Verveer) Date: Tue May 6 05:25:07 2003 Subject: [Numpy-discussion] Numarray 0.5 In-Reply-To: References: Message-ID: <200305061424.26714.verveer@embl-heidelberg.de> Hi All, I found the following problems after testing my software with numarray 0.5: 1) Following works fine if both a and b are arrays: >>> a = array([2]) >>> b = array([1, 2]) >>> print a + b [3 4] However, if b is an python sequence: >>> a = array([2]) >>> b = [1, 2] >>> print a + b [3] Apparently broadcasting does not work with python sequeces anymore. This used to work fine in version 0.4. Is this a bug? 2) It is not possible to compare an array type object to the 'Any' object: >>> Float64 == Any Traceback (most recent call last): File "", line 1, in ? File "/usr/local/lib/python2.2/site-packages/numarray/numerictypes.py", line 112, in __cmp__ return (genericTypeRank.index(self.name) - ValueError: list.index(x): x not in list I am not sure if this is a bug, or intended behaviour, but the possibilty to compare an array type object to 'Any' would be very useful for me. 3) The NA_typeNoToTypeObject() function fails if it is called before any arrays are created. It looks to me as if the pNumType array in libnumarraymodule.c is not initialized until an array is created. I don't know if any other functions are affected in the same way. Could this be fixed? Cheers, Peter -- Dr. Peter J. Verveer Cell Biology and Cell Biophysics Programme EMBL Meyerhofstrasse 1 D-69117 Heidelberg Germany Tel. : +49 6221 387245 Fax : +49 6221 387242 Email: Peter.Verveer at embl-heidelberg.de From verveer at embl-heidelberg.de Tue May 6 05:52:13 2003 From: verveer at embl-heidelberg.de (Peter Verveer) Date: Tue May 6 05:52:13 2003 Subject: [Numpy-discussion] Question about numarray Bool type Message-ID: <200305061451.45361.verveer@embl-heidelberg.de> Hi, In the numarray source code Bool is defined as a typedef to char. Does the Bool array type correspond to the Int8, or the UInt8 type? If I recall correctly, the char type may be signed or unsigned. Can I make any assumptions about the C type used to implement Bool at all? I guess at the Python level it does not matter what actual C type is used to implement bool arrays as it only represents boolean values. However, at the level of C extensions I would like to deal with Bool arrays by calling existing functions written for the appropiate C type. That would be somewhat simpler if Bool is always garantueed to be either a unsigned or a signed char type. Cheers, Peter. -- Dr. Peter J. Verveer Cell Biology and Cell Biophysics Programme EMBL Meyerhofstrasse 1 D-69117 Heidelberg Germany Tel. : +49 6221 387245 Fax : +49 6221 387242 Email: Peter.Verveer at embl-heidelberg.de From falted at openlc.org Tue May 6 07:21:10 2003 From: falted at openlc.org (Francesc Alted) Date: Tue May 6 07:21:10 2003 Subject: [Numpy-discussion] numarray 0.5 -> python setup.py build fails In-Reply-To: <003101c3136e$3d7d2d90$421ee6a9@rodan> References: <1052136718.23013.4.camel@halloween.stsci.edu> <003101c3136e$3d7d2d90$421ee6a9@rodan> Message-ID: <200305061619.54534.falted@openlc.org> I guess this is a problem with permissions. It seems like if setup.py is trying to install numarray on the Python site-packages directory. It should not, but meanwhile, run this command as root and it should works. Ah!, do not forget to delete (or better, move) the old site-packages/numarray in order to do a problem-free fresh installation. Remember: > > 1. Due to some module renamings, numarray-0.5 will not install > > correctly on top of an existing numarray installation. Before > > installing numarray-0.5 remove your old version of numarray. Cheers, A Dimarts 06 Maig 2003 03:24, Sebastian Haase va escriure: > Hi, > I just tried this : > haase at baboon:~/numarray-0.5: python2.2 setup.py build --gencode > > > creating /usr/lib/python2.2/site-packages/numarray > error: could not create '/usr/lib/python2.2/site-packages/numarray': > Permission denied > > I guess this is the same bug I ran into a couple month ago - I already now > a workaround ... > > - Sebastian > -- Francesc Alted From jmiller at stsci.edu Tue May 6 08:49:03 2003 From: jmiller at stsci.edu (Todd Miller) Date: Tue May 6 08:49:03 2003 Subject: [Numpy-discussion] Question about numarray Bool type References: <200305061451.45361.verveer@embl-heidelberg.de> Message-ID: <3EB7D96B.6080201@stsci.edu> Peter Verveer wrote: >Hi, > >In the numarray source code Bool is defined as a typedef to char. Does the >Bool array type correspond to the Int8, or the UInt8 type? If I recall >correctly, the char type may be signed or unsigned. Can I make any >assumptions about the C type used to implement Bool at all? > Yes! It's named Bool. :) >I guess at the Python level it does not matter what actual C type is used to >implement bool arrays as it only represents boolean values. However, at the >level of C extensions I would like to deal with Bool arrays by calling >existing functions written for the appropiate C type. > The easiest way now is to just declare your variables with type "Bool". The only thing I can think of which that doesn't cover is printf/scanf. Are there others? >That would be somewhat >simpler if Bool is always garantueed to be either a unsigned or a signed char >type. > I think perhaps we should just re-define Bool as signed char. Any other opinions? > >Cheers, Peter. > > > Todd From haase at msg.ucsf.edu Tue May 6 09:06:09 2003 From: haase at msg.ucsf.edu (Sebastian Haase) Date: Tue May 6 09:06:09 2003 Subject: [Numpy-discussion] numarray 0.5 -> python setup.py build fails References: <1052136718.23013.4.camel@halloween.stsci.edu> <003101c3136e$3d7d2d90$421ee6a9@rodan> <200305061619.54534.falted@openlc.org> Message-ID: <004d01c313e9$65d65980$421ee6a9@rodan> The reason that I always try "setup.py BUILD" first is that I don't have root permissions on that machine - so this is a quite important "test" for me. Thanks, Sebastian ----- Original Message ----- From: "Francesc Alted" To: "Sebastian Haase" ; Sent: Tuesday, May 06, 2003 7:19 AM Subject: Re: [Numpy-discussion] numarray 0.5 -> python setup.py build fails I guess this is a problem with permissions. It seems like if setup.py is trying to install numarray on the Python site-packages directory. It should not, but meanwhile, run this command as root and it should works. Ah!, do not forget to delete (or better, move) the old site-packages/numarray in order to do a problem-free fresh installation. Remember: > > 1. Due to some module renamings, numarray-0.5 will not install > > correctly on top of an existing numarray installation. Before > > installing numarray-0.5 remove your old version of numarray. Cheers, A Dimarts 06 Maig 2003 03:24, Sebastian Haase va escriure: > Hi, > I just tried this : > haase at baboon:~/numarray-0.5: python2.2 setup.py build --gencode > > > creating /usr/lib/python2.2/site-packages/numarray > error: could not create '/usr/lib/python2.2/site-packages/numarray': > Permission denied > > I guess this is the same bug I ran into a couple month ago - I already now > a workaround ... > > - Sebastian > -- Francesc Alted ------------------------------------------------------- Enterprise Linux Forum Conference & Expo, June 4-6, 2003, Santa Clara The only event dedicated to issues related to Linux enterprise solutions www.enterpriselinuxforum.com _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From verveer at embl-heidelberg.de Tue May 6 09:18:02 2003 From: verveer at embl-heidelberg.de (Peter Verveer) Date: Tue May 6 09:18:02 2003 Subject: [Numpy-discussion] Question about numarray Bool type In-Reply-To: <3EB7D96B.6080201@stsci.edu> References: <200305061451.45361.verveer@embl-heidelberg.de> <3EB7D96B.6080201@stsci.edu> Message-ID: <200305061800.18923.verveer@embl-heidelberg.de> On Tuesday 06 May 2003 17:48, Todd Miller wrote: > Peter Verveer wrote: > >Hi, > > > >In the numarray source code Bool is defined as a typedef to char. Does the > >Bool array type correspond to the Int8, or the UInt8 type? If I recall > >correctly, the char type may be signed or unsigned. Can I make any > >assumptions about the C type used to implement Bool at all? > > Yes! It's named Bool. :) Obviously, but then I need to include the numarray include file to make sure I get the right definition, and I do not always want to do that. See below. > >I guess at the Python level it does not matter what actual C type is used > > to implement bool arrays as it only represents boolean values. However, > > at the level of C extensions I would like to deal with Bool arrays by > > calling existing functions written for the appropiate C type. > > The easiest way now is to just declare your variables with type "Bool". > The only thing I can think of which that doesn't cover is printf/scanf. > Are there others? This does not work if you want to call a routine that was not written using the numarray typedefs. For instance, I like to write my array processing routines such that I can use them in a standalone C program without linking to numarray. These routines know nothing about numarray, but as long as the layout and data types of the arrays are correct I can still use them in numarray C routines. Then you need to know of course exactly what the C data types of the numarray routines are. That is easy for most types since they are exactly defined by bit-size and sign. But this is currently not true for the Bool type. > >That would be somewhat > >simpler if Bool is always garantueed to be either a unsigned or a signed > > char type. > > I think perhaps we should just re-define Bool as signed char. Any other > opinions? That would solve the problem. Peter From falted at openlc.org Tue May 6 09:42:18 2003 From: falted at openlc.org (Francesc Alted) Date: Tue May 6 09:42:18 2003 Subject: [Numpy-discussion] Question about numarray Bool type In-Reply-To: <200305061800.18923.verveer@embl-heidelberg.de> References: <200305061451.45361.verveer@embl-heidelberg.de> <3EB7D96B.6080201@stsci.edu> <200305061800.18923.verveer@embl-heidelberg.de> Message-ID: <1052237809.3eb7dff188366@webmail.imk.es> Mensaje citado por: Peter Verveer : > > >That would be somewhat > > >simpler if Bool is always garantueed to be either a unsigned or a > signed > > > char type. > > > > I think perhaps we should just re-define Bool as signed char. Any > other > > opinions? > > That would solve the problem. I'm in the same position than Peter. I would definitively prefer a standard C type for booleans. Francesc Alted From jmiller at stsci.edu Tue May 6 09:46:15 2003 From: jmiller at stsci.edu (Todd Miller) Date: Tue May 6 09:46:15 2003 Subject: [Numpy-discussion] numarray 0.5 -> python setup.py build fails References: <1052136718.23013.4.camel@halloween.stsci.edu> <003101c3136e$3d7d2d90$421ee6a9@rodan> Message-ID: <3EB7E6E9.5090800@stsci.edu> Sebastian Haase wrote: >Hi, >I just tried this : >haase at baboon:~/numarray-0.5: python2.2 setup.py build --gencode > > >creating /usr/lib/python2.2/site-packages/numarray >error: could not create '/usr/lib/python2.2/site-packages/numarray': >Permission denied > >I guess this is the same bug I ran into a couple month ago - I already now a >workaround ... > >- Sebastian > > Here's a couple work arounds I use: 1) When trying to install to /usr/lib, I first "su root". 2) When I don't like to or can't "su root", I first install Python in my home directory using: "./configure --prefix=$HOME". With a local Python, site-packages is also local and writeable by default. That said, setup.py has some kludges in it that need to be eliminated or factored out. Todd From jmiller at stsci.edu Tue May 6 11:26:07 2003 From: jmiller at stsci.edu (Todd Miller) Date: Tue May 6 11:26:07 2003 Subject: [Numpy-discussion] numarray 0.5 -> python setup.py build fails References: <1052136718.23013.4.camel@halloween.stsci.edu> <003101c3136e$3d7d2d90$421ee6a9@rodan> <3EB7E6E9.5090800@stsci.edu> Message-ID: <3EB7FE4E.4080209@stsci.edu> Todd Miller wrote: > Sebastian Haase wrote: > >> Hi, >> I just tried this : >> haase at baboon:~/numarray-0.5: python2.2 setup.py build --gencode >> >> >> creating /usr/lib/python2.2/site-packages/numarray >> error: could not create '/usr/lib/python2.2/site-packages/numarray': >> Permission denied >> >> I guess this is the same bug I ran into a couple month ago - I >> already now a >> workaround ... >> >> - Sebastian >> >> > Here's a couple work arounds I use: > > 1) When trying to install to /usr/lib, I first "su root". > > 2) When I don't like to or can't "su root", I first install Python in > my home directory using: "./configure --prefix=$HOME". With a local > Python, site-packages is also local and writeable by default. > > That said, setup.py has some kludges in it that need to be eliminated > or factored out. > > Todd > A third method, a little less onerous than the first two, is: 3) mkdir $HOME/numarray; python setup.py build --gencode --local=$HOME/numarray This works OK even if you're using a Python which does not have a writeable site-packages directory. > > > > ------------------------------------------------------- > Enterprise Linux Forum Conference & Expo, June 4-6, 2003, Santa Clara > The only event dedicated to issues related to Linux enterprise solutions > www.enterpriselinuxforum.com > > _______________________________________________ > Numpy-discussion mailing list > Numpy-discussion at lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/numpy-discussion From fperez at colorado.edu Tue May 6 11:34:03 2003 From: fperez at colorado.edu (Fernando Perez) Date: Tue May 6 11:34:03 2003 Subject: [Numpy-discussion] numarray 0.5 -> python setup.py build fails In-Reply-To: <3EB7FE4E.4080209@stsci.edu> References: <1052136718.23013.4.camel@halloween.stsci.edu> <003101c3136e$3d7d2d90$421ee6a9@rodan> <3EB7E6E9.5090800@stsci.edu> <3EB7FE4E.4080209@stsci.edu> Message-ID: <3EB7FFDD.20002@colorado.edu> Todd Miller wrote: >>Here's a couple work arounds I use: >> >>1) When trying to install to /usr/lib, I first "su root". >> >>2) When I don't like to or can't "su root", I first install Python in >>my home directory using: "./configure --prefix=$HOME". With a local >>Python, site-packages is also local and writeable by default. >> >>That said, setup.py has some kludges in it that need to be eliminated >>or factored out. >> >>Todd >> > > A third method, a little less onerous than the first two, is: > > 3) mkdir $HOME/numarray; python setup.py build --gencode > --local=$HOME/numarray > > This works OK even if you're using a Python which does not have a > writeable site-packages directory. My apologies if I'm off-base, but I'm pretty sure that a properly written setup.py should not try to write anywhere but in the build/ subdirectory when invoked with the 'build' option. It should only write in site-packages when called with 'install'. I rely on this fact all the time, by first building packages and then calling 'setup.py install' separately. And I do this as a non-root user always. I could be wrong, but if setup.py is trying to write at build (not install) time into /usr/lib/..., I think that should be considered a bug. Best, f. From fperez at colorado.edu Tue May 6 11:40:19 2003 From: fperez at colorado.edu (Fernando Perez) Date: Tue May 6 11:40:19 2003 Subject: [Numpy-discussion] numarray 0.5 -> python setup.py build fails In-Reply-To: <3EB7FFDD.20002@colorado.edu> References: <1052136718.23013.4.camel@halloween.stsci.edu> <003101c3136e$3d7d2d90$421ee6a9@rodan> <3EB7E6E9.5090800@stsci.edu> <3EB7FE4E.4080209@stsci.edu> <3EB7FFDD.20002@colorado.edu> Message-ID: <3EB8015E.2010402@colorado.edu> Fernando Perez wrote: > I rely on this fact all the time, by first building packages and then calling > 'setup.py install' separately. And I do this as a non-root user always. Clarification: I call 'build' as non-root, then 'install' as root. Same as doing: make su make install in python it's setup.py build su setup.py install Cheers, f. From jmiller at stsci.edu Tue May 6 14:21:54 2003 From: jmiller at stsci.edu (Todd Miller) Date: Tue May 6 14:21:54 2003 Subject: [Numpy-discussion] Question about numarray Bool type In-Reply-To: <1052237809.3eb7dff188366@webmail.imk.es> References: <200305061451.45361.verveer@embl-heidelberg.de> <3EB7D96B.6080201@stsci.edu> <200305061800.18923.verveer@embl-heidelberg.de> <1052237809.3eb7dff188366@webmail.imk.es> Message-ID: <1052256083.27219.16.camel@halloween.stsci.edu> On Tue, 2003-05-06 at 12:16, Francesc Alted wrote: > Mensaje citado por: Peter Verveer : > > > >That would be somewhat > > > >simpler if Bool is always garantueed to be either a unsigned or a > > signed > > > > char type. > > > > > > I think perhaps we should just re-define Bool as signed char. Any > > other > > > opinions? > > > > That would solve the problem. > > I'm in the same position than Peter. I would definitively prefer a standard C > type for booleans. > > Francesc Alted > I re-defined Bool as signed char in CVS. Without any real expertise in Boolology , I am unqualified to comment further. Let me know if you would like something different. Todd > > ------------------------------------------------------- > Enterprise Linux Forum Conference & Expo, June 4-6, 2003, Santa Clara > The only event dedicated to issues related to Linux enterprise solutions > www.enterpriselinuxforum.com > > _______________________________________________ > Numpy-discussion mailing list > Numpy-discussion at lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/numpy-discussion -- Todd Miller jmiller at stsci.edu STSCI / ESS / SSB From jmiller at stsci.edu Tue May 6 15:02:10 2003 From: jmiller at stsci.edu (Todd Miller) Date: Tue May 6 15:02:10 2003 Subject: [Numpy-discussion] Numarray 0.5 In-Reply-To: <200305061424.26714.verveer@embl-heidelberg.de> References: <200305061424.26714.verveer@embl-heidelberg.de> Message-ID: <1052258543.27223.55.camel@halloween.stsci.edu> On Tue, 2003-05-06 at 08:24, Peter Verveer wrote: > Hi All, > > I found the following problems after testing my software with numarray 0.5: > > 1) Following works fine if both a and b are arrays: > > >>> a = array([2]) > >>> b = array([1, 2]) > >>> print a + b > [3 4] > > However, if b is an python sequence: > > >>> a = array([2]) > >>> b = [1, 2] > >>> print a + b > [3] > > Apparently broadcasting does not work with python sequeces anymore. This used > to work fine in version 0.4. Is this a bug? Yes, unfortunately. Thanks for reporting it! > > 2) It is not possible to compare an array type object to the 'Any' object: > > >>> Float64 == Any > Traceback (most recent call last): > File "", line 1, in ? > File "/usr/local/lib/python2.2/site-packages/numarray/numerictypes.py", line > 112, in __cmp__ > return (genericTypeRank.index(self.name) - > ValueError: list.index(x): x not in list > > I am not sure if this is a bug, or intended behaviour, but the possibilty to > compare an array type object to 'Any' would be very useful for me. > 'Any' grew up from the C-API, rather than down from the Python design, so it's not very well thought out. Right now, it is a placeholder used to mark arrays with undefined types and to indicate "no type constraint" in C API calls. In normal contexts, you can't make an array of type 'Any'. I think there are two reasonable behaviors for comparisons with 'Any', both used in C. The first behavior is literal comparison; here comparison to Any would generally return "not equal". The second behavior is wild-card matching; here, comparison to Any would generally return "equal". Which makes sense to you? How do you want to use this? > 3) The NA_typeNoToTypeObject() function fails if it is called before any > arrays are created. It looks to me as if the pNumType array in > libnumarraymodule.c is not initialized until an array is created. I don't > know if any other functions are affected in the same way. Could this be > fixed? Yes. This is fixed now in CVS. Thanks again! > > Cheers, Peter > > -- > Dr. Peter J. Verveer > Cell Biology and Cell Biophysics Programme > EMBL > Meyerhofstrasse 1 > D-69117 Heidelberg > Germany > Tel. : +49 6221 387245 > Fax : +49 6221 387242 > Email: Peter.Verveer at embl-heidelberg.de > > > > > ------------------------------------------------------- > This sf.net email is sponsored by:ThinkGeek > Welcome to geek heaven. > http://thinkgeek.com/sf > _______________________________________________ > Numpy-discussion mailing list > Numpy-discussion at lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/numpy-discussion -- Todd Miller jmiller at stsci.edu STSCI / ESS / SSB From stan at stanheckman.com Tue May 6 17:08:30 2003 From: stan at stanheckman.com (Stan Heckman) Date: Tue May 6 17:08:30 2003 Subject: [Numpy-discussion] Re: [SciPy-user] [numarray issue] nonzero? Comments on new name solicited In-Reply-To: References: Message-ID: <87n0hzy604.fsf@schedar.com> "Perry Greenfield" writes: > Any objections to using "at"? no, "at" reads reasonably well. > Any better suggestions. "suchthat"? -- Stan From alobo at ija.csic.es Wed May 7 00:41:10 2003 From: alobo at ija.csic.es (Agustin Lobo) Date: Wed May 7 00:41:10 2003 Subject: [Numpy-discussion] Re: [SciPy-user] [numarray issue] nonzero? Comments on new name solicited In-Reply-To: <87n0hzy604.fsf@schedar.com> Message-ID: suchdat would make the programs much easier to read, very good idea. Agus Dr. Agustin Lobo Instituto de Ciencias de la Tierra (CSIC) Lluis Sole Sabaris s/n 08028 Barcelona SPAIN tel 34 93409 5410 fax 34 93411 0012 alobo at ija.csic.es On 6 May 2003, Stan Heckman wrote: > "Perry Greenfield" writes: > > > Any objections to using "at"? > > no, "at" reads reasonably well. > > > Any better suggestions. > > "suchthat"? > > -- > Stan > _______________________________________________ > SciPy-user mailing list > SciPy-user at scipy.net > http://www.scipy.net/mailman/listinfo/scipy-user > From alobo at ija.csic.es Wed May 7 02:02:09 2003 From: alobo at ija.csic.es (Agustin Lobo) Date: Wed May 7 02:02:09 2003 Subject: [Numpy-discussion] Re: [SciPy-user] [numarray issue] nonzero? Comments on new name solicited In-Reply-To: Message-ID: I meant "suchthat" (not "suchdat"), of course. The "spanglish" spelling would be confusing for others! Agus On Wed, 7 May 2003, Agustin Lobo wrote: > suchdat would make the programs much easier > to read, very good idea. > > Agus > > Dr. Agustin Lobo > Instituto de Ciencias de la Tierra (CSIC) > Lluis Sole Sabaris s/n > 08028 Barcelona SPAIN > tel 34 93409 5410 > fax 34 93411 0012 > alobo at ija.csic.es > > > On 6 May 2003, Stan Heckman wrote: > > > "Perry Greenfield" writes: > > > > > Any objections to using "at"? > > > > no, "at" reads reasonably well. > > > > > Any better suggestions. > > > > "suchthat"? > > > > -- > > Stan > > _______________________________________________ > > SciPy-user mailing list > > SciPy-user at scipy.net > > http://www.scipy.net/mailman/listinfo/scipy-user > > > _______________________________________________ > SciPy-user mailing list > SciPy-user at scipy.net > http://www.scipy.net/mailman/listinfo/scipy-user > From verveer at embl-heidelberg.de Wed May 7 02:17:12 2003 From: verveer at embl-heidelberg.de (Peter Verveer) Date: Wed May 7 02:17:12 2003 Subject: [Numpy-discussion] Numarray 0.5 In-Reply-To: <1052258543.27223.55.camel@halloween.stsci.edu> References: <200305061424.26714.verveer@embl-heidelberg.de> <1052258543.27223.55.camel@halloween.stsci.edu> Message-ID: <200305071116.53135.verveer@embl-heidelberg.de> > > I am not sure if this is a bug, or intended behaviour, but the possibilty > > to compare an array type object to 'Any' would be very useful for me. > > 'Any' grew up from the C-API, rather than down from the Python design, > so it's not very well thought out. Right now, it is a placeholder used > to mark arrays with undefined types and to indicate "no type constraint" > in C API calls. In normal contexts, you can't make an array of type > 'Any'. I think there are two reasonable behaviors for comparisons with > 'Any', both used in C. The first behavior is literal comparison; here > comparison to Any would generally return "not equal". The second > behavior is wild-card matching; here, comparison to Any would generally > return "equal". Which makes sense to you? How do you want to use > this? I use 'Any' right now in C functions to indicate 'no type constraint'. You could use both literal comparison and wild-card matching behaviour for that, if you want to do this in Python, I think. But maybe one should not do such a thing in Python by comparison to 'Any' at all. Literal comparison is what I generally expect from Python objects, so anything else may just be confusing (at least to me). However, the name 'Any' does suggest the wild-card matching behaviour. Is there specific reason why you exposed 'Any' in python? Maybe it would be better not to expose such a type object since it seems to be a 'special case'. I am starting to think that my use for it at the Python level is not appropiate. For instance, I could easily use 'None' instead, and I think I will do that in the future. You mentioned that 'Any' is really a C-API thing. Unless somebody has a good use for it I would now actually say that it maybe should not be exposed at the Python API level at all... Cheers, Peter -- Dr. Peter J. Verveer Cell Biology and Cell Biophysics Programme EMBL Meyerhofstrasse 1 D-69117 Heidelberg Germany Tel. : +49 6221 387245 Fax : +49 6221 387242 Email: Peter.Verveer at embl-heidelberg.de From verveer at embl-heidelberg.de Wed May 7 04:19:03 2003 From: verveer at embl-heidelberg.de (Peter Verveer) Date: Wed May 7 04:19:03 2003 Subject: [Numpy-discussion] numarray 0.5 bug Message-ID: <200305071318.10950.verveer@embl-heidelberg.de> In numarray 0.5, the PyArray_ContiguousFromObject() does not appear to work correctly with Python scalar objects, if the requested type is not equal to the type of the object. (I believe that this function should convert the input to an array of the requested type?) For instance, PyArray_ContiguousFromObject(object, tInt32, 0, 1))) works correctly if object is a python integer, but not if it is a float number. Cheers, Peter -- Dr. Peter J. Verveer Cell Biology and Cell Biophysics Programme EMBL Meyerhofstrasse 1 D-69117 Heidelberg Germany Tel. : +49 6221 387245 Fax : +49 6221 387242 Email: Peter.Verveer at embl-heidelberg.de From jmiller at stsci.edu Wed May 7 05:35:20 2003 From: jmiller at stsci.edu (Todd Miller) Date: Wed May 7 05:35:20 2003 Subject: [Numpy-discussion] Numarray 0.5 In-Reply-To: <200305071116.53135.verveer@embl-heidelberg.de> References: <200305061424.26714.verveer@embl-heidelberg.de> <1052258543.27223.55.camel@halloween.stsci.edu> <200305071116.53135.verveer@embl-heidelberg.de> Message-ID: <1052310950.29151.26.camel@halloween.stsci.edu> On Wed, 2003-05-07 at 05:16, Peter Verveer wrote: > > > I am not sure if this is a bug, or intended behaviour, but the possibilty > > > to compare an array type object to 'Any' would be very useful for me. > > > > 'Any' grew up from the C-API, rather than down from the Python design, > > so it's not very well thought out. Right now, it is a placeholder used > > to mark arrays with undefined types and to indicate "no type constraint" > > in C API calls. In normal contexts, you can't make an array of type > > 'Any'. I think there are two reasonable behaviors for comparisons with > > 'Any', both used in C. The first behavior is literal comparison; here > > comparison to Any would generally return "not equal". The second > > behavior is wild-card matching; here, comparison to Any would generally > > return "equal". Which makes sense to you? How do you want to use > > this? > > I use 'Any' right now in C functions to indicate 'no type constraint'. You > could use both literal comparison and wild-card matching behaviour for that, > if you want to do this in Python, I think. But maybe one should not do such a > thing in Python by comparison to 'Any' at all. > > Literal comparison is what I generally expect from Python objects, so anything > else may just be confusing (at least to me). However, the name 'Any' does > suggest the wild-card matching behaviour. > > Is there specific reason why you exposed 'Any' in python? Initially it was not exposed. As the API type functions evolved, it made Any less of a special case to expose it. > Maybe it would be > better not to expose such a type object since it seems to be a 'special > case'. I am starting to think that my use for it at the Python level is not > appropiate. > For instance, I could easily use 'None' instead, and I think I > will do that in the future. Yes. That's probably what I should have done in C as well: tNone rather than tAny. > > You mentioned that 'Any' is really a C-API thing. Unless somebody has a good > use for it I would now actually say that it maybe should not be exposed at > the Python API level at all... I agree with this. It might be as simple as renaming it to _Any in Python, but I still hesitate to change this until I have a little more time to think about it. Too often, things go from "bad" to "not so good"... Thanks for the input, Todd From perry at stsci.edu Wed May 7 13:20:20 2003 From: perry at stsci.edu (Perry Greenfield) Date: Wed May 7 13:20:20 2003 Subject: [Numpy-discussion] RE: [SciPy-user] [numarray issue] nonzero? Comments on new name solicited In-Reply-To: <87n0hzy604.fsf@schedar.com> Message-ID: Some one suggested that numarray ditch compatibility in this area and "do what's right". Namely he suggested that nonzero --> where where --> select I have to agree that these are far better names than the existing ones. But compatibilty is important. So what to do? Here's one suggestion; I'm curious what people think about this solution. Nothing prevents us from having a module that people would import for Numeric compatibilty and a different one that has somewhat different behavior. Before people start throwing rocks (when you finish reading you can throw them ;-) and say "didn't you say that customized versions of Numeric or numarray were a bad idea?" Yes I did. But I think some customizations are bad and perhaps some are acceptable. What I objected to was having array objects with different behavior Since array objects may be passed between modules that assume different behaviors, that will cause all sorts of problems (e.g., module A was written assuming array slices are views but is passed an array that produces a copy when sliced). But is there any harm when all that is different are module functions? These are unlikely to be passed from one module to another and thus the customization is localized to the user modules. Suppose that the Numeric-compatible version defines "where" and "nonzero" to have their current meanings and numarray uses "select", and "where" respectively. There is one case that could arise that might cause problems. About the only time a problem would occur is if someone did from numarray import * from B import * And in B.py from Numericcompatible import * in which case the numarray "where" is replaced by the Numeric "where" I don't know if this is a strong enough reason to support two different flavors, but what do people think of taking this approach? Are there any other module functions that would benefit from such changes in name or behavior (For example, the axis order issue)? Perry Greenfield From perry at stsci.edu Wed May 7 13:36:09 2003 From: perry at stsci.edu (Perry Greenfield) Date: Wed May 7 13:36:09 2003 Subject: [Numpy-discussion] [numarray] packaging and names In-Reply-To: Message-ID: There was general consensus that numarray should be organized as a package. We are about to start doing so. It does raise some issues (and opportunities) however. 1) What should the package be called? numarray is a possiblity, but we have an opportunity to take a different approach. For example if we could name the package "array" (unfortunately, we can't) then the following becomes possible (these are just examples, we haven't settled on any naming scheme yet). from array.numeric import * # instead of "from numarray import *" import array.fft import array.records # currently recarray Since array is not avaible (there is already an array module in the Python Standard Library) what are the alternatives? Here are some: num numarray arr arrays ndarray I've polled people locally and arrays and ndarray were the most popular, but there doesn't seem to be a clear winner. I tend toward arr somewhat because of the brevity but do realize it isn't terribly descriptive. The drawback of "arrays" is that it may be visually confused with array. 2) What to call the package components? Locally the favorites were numarray --> numeric recarray --> records chararray--> strings (but since we plan to implement PyObject arrays there is a possibilty of real Python string arrays) ndarray --> generic (or base) 3) Where to place 3rd party array extension modules (by that I mean all those not distributed with numarray)? Should they go in the package directory? I'd prefer that they go into a separate directory from the things that are part of the standard numarray distribution (a site-packages equivalent in the package?) 4) We would likely also include in the package numarray.py, recarray.py, chararray.py, ndarray.py so that if the package was added to the path, the current imports would continue to work. This is intended as a short-term measure (i.e., they are deprecated and will disappear at version 1.0, or earlier). We may also keep these files at the current nummarray interface so that some of the software we currently distribute does not need to change immediately to match the changes we have been talking about. We realize that these are big changes, but if we move to a package structure, we ought to think about what makes the most sense (and if it is any comfort, we suffer the consequences of change more than anyone else). Better now than later. Screams? Perry From falted at openlc.org Sat May 10 04:40:01 2003 From: falted at openlc.org (Francesc Alted) Date: Sat May 10 04:40:01 2003 Subject: [Numpy-discussion] PyTables 0.5 released Message-ID: <200305101338.25291.falted@openlc.org> Announcing PyTables 0.5 ----------------------- This is the second public beta release. On this release you will find a 20% of I/O speed improvement over the previous one (0.4), some bugs has been fixed and support for a couple of compression (LZO and UCL) libraries has been added, and... a long awaited Windows version is finally available!. More in detail: What's new ----------- - As a consequence of some twiking the write/read performance has been improved by a 20% overall. One particular case were performance has largely increased (0.5 is up to 6 times faster than 0.4) is when column elements are unidimensional arrays. This impressive speed-up is mainly because of the recent improvements in numarray 0.5 performance (good work, folks!). With that, the reading speed is reaching its theoretical maximum (at least when using the current data access schema). - When reading a Table object, and the user wants to fetch column elements which are unidimensional arrays, a copy of the array from the I/O buffer is delivered automatically to him, so that there is no need to make a call to .copy() method of the numarray arrays anymore. It think this is more comfortable for the user. - The compression was enabled by default in version 0.4, despite of what was stated in the documentation. Now, this has been corrected and compression is *disabled* by default. - Support for two new compression libraries: LZO and UCL (http://www.oberhumer.com/opensource/). These libraries are made by Markus F.X.J. Oberhumer, and they stand for allowing *very* fast decompression. Now, if your data is compressible, you can obtain better reading speed than if not using compression at all!. The improvement is still more noticeable if your are dealing with extremely large (and compressible) data sets. Read the online documentation for more info about that: http://pytables.sourceforge.net/html-doc/usersguide-html3.html#subsection3.4.1 - A couple of memory leaks has been isolated and fixed (it was hard, but I finally did it!). - A bug with column ordering of tables that happens in some special situations has been fixed (thanks to Stan Heckman for reporting this and suggesting the patch). - File class has now an 'isopen' attribute in order to check if a file is open or not. - Updated documentation, specially for giving advice about the use of the new compression libraries. See "Compression issues" subsection, (also on the web: http://pytables.sourceforge.net/html-doc/usersguide-html.html) - Added more unit tests (up to 218 now!) - PyTables has been tested against newest numarray 0.5 and it works just fine. It even works well with Python 2.3b1. - And last, but not least, a Windows version is available!. Thanks to Alan McIntyre for its porting!. There is even a binary ready for click and install. What it is ---------- In short, PyTables provides a powerful and very Pythonic interface to process and organize your table and array data on disk. Its goal is to enable the end user to manipulate easily scientific data tables and Numerical and numarray Python objects in a persistent hierarchical structure. The foundation of the underlying hierarchical data organization is the excellent HDF5 library (http://hdf.ncsa.uiuc.edu/HDF5). A table is defined as a collection of records whose values are stored in fixed-length fields. All records have the same structure and all values in each field have the same data type. The terms "fixed-length" and strict "data types" seems to be quite a strange requirement for an interpreted language like Python, but they serve a useful function if the goal is to save very large quantities of data (such as is generated by many scientific applications, for example) in an efficient manner that reduces demand on CPU time and I/O resources. Quite a bit effort has been invested to make browsing the hierarchical data structure a pleasant experience. PyTables implements just two (orthogonal) easy-to-use methods for browsing. What is HDF5? ------------- For those people who know nothing about HDF5, it is is a general purpose library and file format for storing scientific data made at NCSA. HDF5 can store two primary objects: datasets and groups. A dataset is essentially a multidimensional array of data elements, and a group is a structure for organizing objects in an HDF5 file. Using these two basic constructs, one can create and store almost any kind of scientific data structure, such as images, arrays of vectors, and structured and unstructured grids. You can also mix and match them in HDF5 files according to your needs. Platforms --------- I'm using Linux as the main development platform, but PyTables should be easy to compile/install on other UNIX machines. This package has also passed all the tests on a UltraSparc platform with Solaris 7 and Solaris 8. It also compiles and passes all the tests on a SGI Origin2000 with MIPS R12000 processors and running IRIX 6.5. With Windows, PyTables has been tested with Windows 2000 Professional SP1 and Windows XP, but it should also work with other flavors. An example? ----------- For online code examples, have a look at http://pytables.sourceforge.net/tut/tutorial1-1.html and http://pytables.sourceforge.net/tut/tutorial1-2.html Web site -------- Go to the PyTables web site for more details: http://pytables.sourceforge.net/ Share your experience --------------------- Let me know of any bugs, suggestions, gripes, kudos, etc. you may have. Have fun! -- Francesc Alted From mdehoon at ims.u-tokyo.ac.jp Mon May 12 02:49:09 2003 From: mdehoon at ims.u-tokyo.ac.jp (Michiel Jan Laurens de Hoon) Date: Mon May 12 02:49:09 2003 Subject: [Numpy-discussion] Linking to ranlib Message-ID: <3EBF6EF4.8020305@ims.u-tokyo.ac.jp> Hi, I have a C extension module that makes use of the random number functions in ranlib. Since numpy's RandomArray also relies on ranlib, there is a ranlib.so in site-packages/Numeric, and I found the ranlib.h header file in the include/python2.3/Numeric directory, containing the prototypes for the ranlib routines that I am trying to use. So I was hoping to #include this header file and link to numpy's ranlib.so instead of including another copy of ranlib as part of my C extension module. However, I haven't been able to get this to work -- on Linux and Mac OS X, gcc does not recognize ranlib.so as a library it can link to, whereas on Cygwin gcc cannot find the routines I am looking for in ranlib.dll. Does anybody know if it is possible to link to ranlib.so? If not, what is the function of the ranlib.h include file? Unfortunately the calls to the ranlib routines are way down in my C extension module, I wouldn't be able to call those routines at the Python level. Many thanks, --Michiel. -- Michiel de Hoon, Assistant Professor University of Tokyo, Institute of Medical Science Human Genome Center 4-6-1 Shirokane-dai, Minato-ku Tokyo 108-8639 Japan http://bonsai.ims.u-tokyo.ac.jp/~mdehoon From falted at openlc.org Mon May 12 11:13:15 2003 From: falted at openlc.org (Francesc Alted) Date: Mon May 12 11:13:15 2003 Subject: [Numpy-discussion] PyTables 0.5.1 Message-ID: <200305122012.00407.falted@openlc.org> PyTables 0.5.1 -------------- This is a maintenance release of PyTables. Due to a problem with an specific optimization in PyTables 0.5, it does not work with numarray 0.4 (although it works just fine with numarray 0.5). Thanks to Marc Gehling for reporting that. Todd Miller has already warned me that this optimization was not safe, so I am *disabling* it in 0.5.1. The consequence is that the 20% of improvement during reading tables has almost evaporated to a rather small 4%, but that's life!. If you already have installed PyTables 0.5, I strongly suggest you to upgrade to 0.5.1, even if you are already using numarray 0.5. I will try to further investigate the problem, and, if a good solution is found, I will enable again the optimization in a future release. Another new thing you can find in 0.5.1 is that the use of "UInt64" data types has been removed (it has been replaced by the "Int64" type) of the tutorial chapters in User's Manual. I've done that because the numarray Windows version does not support such a type (due to MSVC compiler limitations). Now, the tutorial section should run fine in all the supported platforms (even Windows). My apologies for being a sinner and trying to optimize too soon ;-) Web site -------- Go to the PyTables web site for more details: http://pytables.sourceforge.net/ Share your experience --------------------- Let me know of any bugs, suggestions, gripes, kudos, etc. you may have. Have fun! -- Francesc Alted From gvermeul at grenoble.cnrs.fr Mon May 12 13:08:02 2003 From: gvermeul at grenoble.cnrs.fr (Gerard Vermeulen) Date: Mon May 12 13:08:02 2003 Subject: [Numpy-discussion] mixing NUMERIC and NUMARRAY Message-ID: <20030512220653.63dc08b9.gvermeul@grenoble.cnrs.fr> Hi, It was remarkable easy to make my plot package PyQwt for Numeric, also compatible with numarray :-) ( PyQwt is a wrapper for Qwt, a Qt-based library with widgets useful for science and engineering -- http://gerard.vermeulen.free.fr ) I discovered by accident that PyQwt compiled with #include plots Numeric-arrays and that PyQwt compiled with #include plots numarray-arrays. The only interface between PyQwt and Numeric (or numarray) passes through PyArray_ContiguousFromObject to cast sequences into arrays from which data is copied into Qwt's data structures. PyQwt never returns Numeric- or numarray-arrays. Is this feature platform dependent? Or is it safe to release PyQwt, claiming that a Numeric-compiled PyQwt is compatible with numarray and vice-versa? Gerard From jmiller at stsci.edu Mon May 12 13:52:02 2003 From: jmiller at stsci.edu (Todd Miller) Date: Mon May 12 13:52:02 2003 Subject: [Numpy-discussion] mixing NUMERIC and NUMARRAY In-Reply-To: <20030512220653.63dc08b9.gvermeul@grenoble.cnrs.fr> References: <20030512220653.63dc08b9.gvermeul@grenoble.cnrs.fr> Message-ID: <1052772708.13040.60.camel@halloween.stsci.edu> On Mon, 2003-05-12 at 16:06, Gerard Vermeulen wrote: > Hi, > > It was remarkable easy to make my plot package PyQwt for Numeric, also > compatible with numarray :-) > Great! That's they way things are *supposed* to work, and usually do, but don't always. > ( PyQwt is a wrapper for Qwt, a Qt-based library with widgets useful for > science and engineering -- http://gerard.vermeulen.free.fr ) > > I discovered by accident that PyQwt compiled with > #include > plots Numeric-arrays and that PyQwt compiled with > #include > plots numarray-arrays. > This part sounds a little backward. Presumably this was a typo. > The only interface between PyQwt and Numeric (or numarray) passes through > PyArray_ContiguousFromObject to cast sequences into arrays from which data > is copied into Qwt's data structures. > > PyQwt never returns Numeric- or numarray-arrays. > > Is this feature platform dependent? Yes, if I understand your question correctly. > Or is it safe to release PyQwt, > claiming that a Numeric-compiled PyQwt is compatible with numarray > and vice-versa? No dice. numarray is currently trying for source level compatibility only, so an extension compiled for Numeric creates slightly different object code than one compiled for numarray; slightly different means inoperable. Still, it's good to know that PyQwt is very close to workable for numarray. > Gerard > _______________________________________________ > Numpy-discussion mailing list > Numpy-discussion at lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/numpy-discussion -- Todd Miller jmiller at stsci.edu STSCI / ESS / SSB From gvermeul at grenoble.cnrs.fr Mon May 12 15:53:04 2003 From: gvermeul at grenoble.cnrs.fr (gvermeul at grenoble.cnrs.fr) Date: Mon May 12 15:53:04 2003 Subject: [Numpy-discussion] no typo: mixing NUMERIC and NUMARRAY Message-ID: <200305122252.h4CMq9dk024899@grenoble.cnrs.fr> > > > ( PyQwt is a wrapper for Qwt, a Qt-based library with widgets useful for > > science and engineering -- http://gerard.vermeulen.free.fr ) > > > > I discovered by accident that PyQwt compiled with > > #include > > plots Numeric-arrays and that PyQwt compiled with > > #include > > plots numarray-arrays. > > > > This part sounds a little backward. Presumably this was a typo. > No Todd, this is no typo. I launched a test program from the wrong directory and was shocked to see that it worked (have checked with python -vv which modules were loaded). The other way around worked, too. Of course, the normal cases work, too. (verified that multiplying a Numeric array with numarray does not work) > > > The only interface between PyQwt and Numeric (or numarray) passes through > > PyArray_ContiguousFromObject to cast sequences into arrays from which data > > is copied into Qwt's data structures. > > > > PyQwt never returns Numeric- or numarray-arrays. > > > > Is this feature platform dependent? > > Yes, if I understand your question correctly. > > > Or is it safe to release PyQwt, > > claiming that a Numeric-compiled PyQwt is compatible with numarray > > and vice-versa? > > No dice. numarray is currently trying for source level compatibility > only, so an extension compiled for Numeric creates slightly different > object code than one compiled for numarray; slightly different means > inoperable. Still, it's good to know that PyQwt is very close to > workable for numarray. > Well, my experience shows that on x86-Linux the layout of the data in a numarray-array and Numeric-array is sufficiently close that both PyArray_ContiguousFromObject() functions also interprete the arrays of the 'cousin' correctly (at least in 1D). I realize that this observation is very hackish, but it may help to speed up the acceptance of numarray. Gerard ------------------------------------------------------------- This message was sent using HTTPS service from CNRS Grenoble. ---> https://grenoble.cnrs.fr <--- From alnesbit at students.cs.mu.OZ.AU Mon May 12 23:01:03 2003 From: alnesbit at students.cs.mu.OZ.AU (Andrew Nesbit) Date: Mon May 12 23:01:03 2003 Subject: [Numpy-discussion] Converting string to array Message-ID: I have a string representing a sequence of 16-bit audio samples, read in from an AIFF file like this: f = aifc.open("filename.aiff", "r") data = f.readframes(f.getnframes()) I can convert this to a tuple of signed 16-bit integers using the Python standard library "array" module: a = array.array("h", data) This gives samples in the range [-32768, 32767] and everything is fine. However, instead of using module array I want to use Numeric's array function in an equivalent way. I've been trying to do it like this: a = Numeric.array(data, Numeric.Int16) Now, a.itemsize() outputs 2, which seems fine, but when I iterate over the resulting array (using standard a[i] notation), the results indicate that the array is being treated as a sequence of 8-bit, rather than 16-bit, integers. What am I doing wrong here? Thanks in advance. Andrew. From eric at enthought.com Mon May 12 23:11:07 2003 From: eric at enthought.com (eric jones) Date: Mon May 12 23:11:07 2003 Subject: [Numpy-discussion] Converting string to array In-Reply-To: Message-ID: <001c01c31916$5b4fb0b0$8901a8c0@ERICDESKTOP> Try fromstring. I think it should do what you want: >>> from Numeric import * >>> print fromstring.__doc__ fromstring(string, typecode='l', count=-1) returns a new 1d array initialized from the raw binary data in string. If count is positive, the new array will have count elements, otherwise it's size is determined by the size of string. eric ---------------------------------------------- eric jones 515 Congress Ave www.enthought.com Suite 1614 512 536-1057 Austin, Tx 78701 > -----Original Message----- > From: numpy-discussion-admin at lists.sourceforge.net [mailto:numpy- > discussion-admin at lists.sourceforge.net] On Behalf Of Andrew Nesbit > Sent: Monday, May 12, 2003 11:57 PM > To: numpy-discussion at lists.sourceforge.net > Subject: [Numpy-discussion] Converting string to array > > I have a string representing a sequence of 16-bit audio samples, read > in from an AIFF file like this: > > f = aifc.open("filename.aiff", "r") > data = f.readframes(f.getnframes()) > > I can convert this to a tuple of signed 16-bit integers using the > Python standard library "array" module: > > a = array.array("h", data) > > This gives samples in the range [-32768, 32767] and everything is > fine. > > However, instead of using module array I want to use Numeric's array > function in an equivalent way. I've been trying to do it like this: > > a = Numeric.array(data, Numeric.Int16) > > Now, a.itemsize() outputs 2, which seems fine, but when I iterate over > the resulting array (using standard a[i] notation), the results > indicate that the array is being treated as a sequence of 8-bit, > rather than 16-bit, integers. > > What am I doing wrong here? > > Thanks in advance. > > Andrew. > > > > > ------------------------------------------------------- > Enterprise Linux Forum Conference & Expo, June 4-6, 2003, Santa Clara > The only event dedicated to issues related to Linux enterprise solutions > www.enterpriselinuxforum.com > > _______________________________________________ > Numpy-discussion mailing list > Numpy-discussion at lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/numpy-discussion From alnesbit at students.cs.mu.OZ.AU Mon May 12 23:26:03 2003 From: alnesbit at students.cs.mu.OZ.AU (Andrew Nesbit) Date: Mon May 12 23:26:03 2003 Subject: [Numpy-discussion] Converting string to array In-Reply-To: <001c01c31916$5b4fb0b0$8901a8c0@ERICDESKTOP> Message-ID: On Tue, 13 May 2003, eric jones wrote: > Try fromstring. I think it should do what you want: > > >>> from Numeric import * print fromstring.__doc__ > > fromstring(string, typecode='l', count=-1) returns a new 1d array > initialized from the raw binary data in string. If count is > positive, the new array will have count elements, otherwise it's > size is determined by the size of string. Thanks, Eric. That did the trick. The manual said that Numeric.array() takes a sequence type as one of its arguments, and considering that a string is a sequence type, I thought that, given an appropriate typecode, it would do the conversion automatically in the same way that array.array does. Is this difference an intended feature of numpy? Andrew. From jmiller at stsci.edu Tue May 13 14:50:11 2003 From: jmiller at stsci.edu (Todd Miller) Date: Tue May 13 14:50:11 2003 Subject: [Numpy-discussion] numarray repackaging one more time... Message-ID: <3EC1688F.5070606@stsci.edu> As has been mentioned before, we're planning to repackage numarray as a package rather than as a collection of modules. We're soliciting comments now because we only want to do this once. Here are the planned renamings for the numarray modules as we transition to a package: numarray --> numarray.numeric recarray --> numarray.records chararray --> numarray.strings ndarray --> numarray.generic * --> numarray.* Note that class and function names will remain unchanged, so recarray.RecArray will become numarray.records.RecArray. Here are the planned renamings for the current and planned add-on packages consolidated into a single add-ons distribution: LinearAlgrebra2 --> numarray.LinearAlgebra FFT2 --> numarray.FFT RandomArray2 --> numarray.RandomArray Convolve --> numarray.Convolve Convolve.Image --> numarray.Image MA2 --> numarray.MA Future 3rd party packages should locate themselves under the numarray.addons sub-package: * --> numarray.addons.* The package name (shown here as numarray) is still undecided. By including stub modules which emulate the "import behavior" of numarray-0.5, we'll make it possible to ignore / work around the effects of the re-packaging for the time being. The stub modules will be activated either through the creation of a .pth file or via additions to PYTHONPATH to make them visible. Comments? From cookedm at physics.mcmaster.ca Tue May 13 15:16:15 2003 From: cookedm at physics.mcmaster.ca (David M. Cooke) Date: Tue May 13 15:16:15 2003 Subject: [Numpy-discussion] numarray repackaging one more time... In-Reply-To: <3EC1688F.5070606@stsci.edu> References: <3EC1688F.5070606@stsci.edu> Message-ID: <20030513221331.GA14575@arbutus.physics.mcmaster.ca> On Tue, May 13, 2003 at 05:50:07PM -0400, Todd Miller wrote: > As has been mentioned before, we're planning to repackage numarray as > a package rather than as a collection of modules. We're soliciting > comments now because we only want to do this once. ... > Future 3rd party packages should locate themselves under the > numarray.addons sub-package: > > * --> numarray.addons.* What's the rationale for having a separate place for third-party packages? Shouldn't they just install themselves as their own packages or modules? (i.e., in site-packages/) -- |>|\/|< /--------------------------------------------------------------------------\ |David M. Cooke http://arbutus.physics.mcmaster.ca/dmc/ |cookedm at physics.mcmaster.ca From haase at msg.ucsf.edu Tue May 13 19:14:05 2003 From: haase at msg.ucsf.edu (Sebastian Haase) Date: Tue May 13 19:14:05 2003 Subject: [Numpy-discussion] make C array accessible to python without copy Message-ID: <00ca01c319be$6f25e160$421ee6a9@rodan> Hi - I hope it's OK to just resent my posting from Feb 22. I didn't get any response at the time. Also - looking into the code again - I am wondering what the best way to handle the reference counting would be ? Here come my original posting: > Short follow up: > 1) Is it planned to support this more directly? > 2) How much does it cost to create a buffer object if it > uses my already allocated memory ? > 3) Can I change the pointer so that it points to a > different memory space WITHOUT having to recreate any > python objects? Or would that "confuse" the buffer or > numarray? (We are hoping to aquire 30 images per second > - the images should get written into a circular buffer so > that the data can be written to disk in larger chunks - > but the python array should always refer to the current > image ) > > Thanks for all the nice toys (tools) ;-) > Sebastian Haase > > > > On Fri, 17 Jan 2003 18:16:01 -0500 > Todd Miller wrote: > >Sebastian Haase wrote: > > > >>Hi, > >>What is the C API to make an array that got allocated, > >>let's say, by a = new short[512*512], > >>accessible to python as numarray. > >> > >What you want to do is not currently supported well in C. > > The way to do what you want is: > > > >1. Create a buffer object from your C++ array. The > >buffer object can be built such that it refers to the > >original copy of the data. > > > >2. Call back into Python (numarray.NumArray) with your > >buffer object as the buffer parameter. > > > >You can scavenge the code in NA_newAll (Src/newarray.ch) > >for most of the callback. > > > >>I tried NA_New - but that seems to make a copy. > >>I would need it to use the original memory space > >>so that I can "observe" the array from Python WHILE > >>the underlying C array changes (it's actually a camera > >>image) > >> > >That sounds cool! > > > >> > >>Thanks, > >>Sebastian Haase From gvermeul at grenoble.cnrs.fr Wed May 14 00:25:10 2003 From: gvermeul at grenoble.cnrs.fr (Gerard Vermeulen) Date: Wed May 14 00:25:10 2003 Subject: [Numpy-discussion] numarray repackaging one more time... In-Reply-To: <20030513221331.GA14575@arbutus.physics.mcmaster.ca> References: <3EC1688F.5070606@stsci.edu> <20030513221331.GA14575@arbutus.physics.mcmaster.ca> Message-ID: <20030514092419.5b3cdd10.gvermeul@grenoble.cnrs.fr> On Tue, 13 May 2003 18:13:31 -0400 "David M. Cooke" wrote: > On Tue, May 13, 2003 at 05:50:07PM -0400, Todd Miller wrote: > > As has been mentioned before, we're planning to repackage numarray as > > a package rather than as a collection of modules. We're soliciting > > comments now because we only want to do this once. > ... > > Future 3rd party packages should locate themselves under the > > numarray.addons sub-package: > > > > * --> numarray.addons.* > > What's the rationale for having a separate place for third-party > packages? Shouldn't they just install themselves as their own packages > or modules? (i.e., in site-packages/) > I like the idea, because it would mean that I can release my PyQwt plot package so that it can be build for Numeric and numarray on the same system. It reduces the risk of calling 'PyQwt for numarray' functions with 'Numeric arrays' as arguments and vice-versa (for me it seems to work, but it scares me to death). Gerard From hinsen at cnrs-orleans.fr Wed May 14 02:27:03 2003 From: hinsen at cnrs-orleans.fr (Konrad Hinsen) Date: Wed May 14 02:27:03 2003 Subject: [Numpy-discussion] numarray repackaging one more time... In-Reply-To: <20030514092419.5b3cdd10.gvermeul@grenoble.cnrs.fr> References: <3EC1688F.5070606@stsci.edu> <20030513221331.GA14575@arbutus.physics.mcmaster.ca> <20030514092419.5b3cdd10.gvermeul@grenoble.cnrs.fr> Message-ID: <200305141123.45560.hinsen@cnrs-orleans.fr> > I like the idea, because it would mean that I can release my PyQwt > plot package so that it can be build for Numeric and numarray > on the same system. It reduces the risk of calling 'PyQwt for numarray' I'd like to check that this doesn't cause any technical problems with package managers, distutils, etc. Otherwise it will prevent people from switching to numarray. Konrad. -- ------------------------------------------------------------------------------- Konrad Hinsen | E-Mail: hinsen at cnrs-orleans.fr Centre de Biophysique Moleculaire (CNRS) | Tel.: +33-2.38.25.56.24 Rue Charles Sadron | Fax: +33-2.38.63.15.17 45071 Orleans Cedex 2 | Deutsch/Esperanto/English/ France | Nederlands/Francais ------------------------------------------------------------------------------- From hinsen at cnrs-orleans.fr Wed May 14 02:29:03 2003 From: hinsen at cnrs-orleans.fr (Konrad Hinsen) Date: Wed May 14 02:29:03 2003 Subject: [Numpy-discussion] numarray repackaging one more time... In-Reply-To: <3EC1688F.5070606@stsci.edu> References: <3EC1688F.5070606@stsci.edu> Message-ID: <200305141126.07426.hinsen@cnrs-orleans.fr> On Tuesday 13 May 2003 23:50, Todd Miller wrote: > Here are the planned renamings for the numarray modules as we transition > to a package: > > numarray --> numarray.numeric > recarray --> numarray.records > chararray --> numarray.strings > ndarray --> numarray.generic > * --> numarray.* > > Note that class and function names will remain unchanged, so > recarray.RecArray will become numarray.records.RecArray. > > Here are the planned renamings for the current and planned add-on packages > consolidated into a single add-ons distribution: > > LinearAlgrebra2 --> numarray.LinearAlgebra > FFT2 --> numarray.FFT > RandomArray2 --> numarray.RandomArray > Convolve --> numarray.Convolve > Convolve.Image --> numarray.Image > MA2 --> numarray.MA Please use a uniform capitalization scheme, no matter which one. Either numarray.image or Numarray.Image (I prefer the latter, but both are better than the mixed one). Konrad. -- ------------------------------------------------------------------------------- Konrad Hinsen | E-Mail: hinsen at cnrs-orleans.fr Centre de Biophysique Moleculaire (CNRS) | Tel.: +33-2.38.25.56.24 Rue Charles Sadron | Fax: +33-2.38.63.15.17 45071 Orleans Cedex 2 | Deutsch/Esperanto/English/ France | Nederlands/Francais ------------------------------------------------------------------------------- From falted at openlc.org Wed May 14 02:46:03 2003 From: falted at openlc.org (Francesc Alted) Date: Wed May 14 02:46:03 2003 Subject: [Numpy-discussion] numarray repackaging one more time... In-Reply-To: <20030514092419.5b3cdd10.gvermeul@grenoble.cnrs.fr> References: <3EC1688F.5070606@stsci.edu> <20030513221331.GA14575@arbutus.physics.mcmaster.ca> <20030514092419.5b3cdd10.gvermeul@grenoble.cnrs.fr> Message-ID: <200305141145.07726.falted@openlc.org> A Dimecres 14 Maig 2003 09:24, Gerard Vermeulen va escriure: > > I like the idea, because it would mean that I can release my PyQwt > plot package so that it can be build for Numeric and numarray > on the same system. It reduces the risk of calling 'PyQwt for numarray' > functions with 'Numeric arrays' as arguments and vice-versa (for me it > seems to work, but it scares me to death). I think this is not necessary as you can always distinguish if the user is passing a numarray or Numeric object, like in: In [3]: a=Numeric.array([1,2]) In [4]: b=numarray.array([1,2]) In [5]: type(a) Out[5]: In [6]: type(b) Out[6]: It's just a check before accessing the object. Cheers, -- Francesc Alted From Kasper.Souren at ircam.fr Wed May 14 03:03:08 2003 From: Kasper.Souren at ircam.fr (Kasper Souren) Date: Wed May 14 03:03:08 2003 Subject: [Numpy-discussion] numarray repackaging one more time... In-Reply-To: <20030513221331.GA14575@arbutus.physics.mcmaster.ca> References: <3EC1688F.5070606@stsci.edu> <20030513221331.GA14575@arbutus.physics.mcmaster.ca> Message-ID: <200305141334.58603.Kasper.Souren@ircam.fr> > > Future 3rd party packages should locate themselves under the > > numarray.addons sub-package: > > > > * --> numarray.addons.* > > What's the rationale for having a separate place for third-party > packages? Shouldn't they just install themselves as their own packages > or modules? (i.e., in site-packages/) Maybe it's nice to have both. Leave it to the third-party to decide whether it's a good idea to install in numarray.addons or in site-packages/. For stuff that's not too dependent on numpy (PyGame for instance) it would be better to install in site-packages/. For other things (like PyQwt) it might be better to locate them in numarray.addons. bye, Kasper From jmiller at stsci.edu Wed May 14 04:08:08 2003 From: jmiller at stsci.edu (Todd Miller) Date: Wed May 14 04:08:08 2003 Subject: [Numpy-discussion] numarray repackaging one more time... References: <3EC1688F.5070606@stsci.edu> <20030513221331.GA14575@arbutus.physics.mcmaster.ca> Message-ID: <3EC223D2.2030901@stsci.edu> David M. Cooke wrote: >On Tue, May 13, 2003 at 05:50:07PM -0400, Todd Miller wrote: > > >>As has been mentioned before, we're planning to repackage numarray as >>a package rather than as a collection of modules. We're soliciting >>comments now because we only want to do this once. >> >> >... > > >>Future 3rd party packages should locate themselves under the >>numarray.addons sub-package: >> >>* --> numarray.addons.* >> >> > >What's the rationale for having a separate place for third-party >packages? Shouldn't they just install themselves as their own packages >or modules? (i.e., in site-packages/) > > > I think there are two goals which are served by numarray.addons: 1. It becomes clear what is in the numarray core, and what isn't. This is a maintenance issue. 2. It becomes clear that a particular extension was built explicitly for numarray. This might be valuable in a transition phase from Numeric to numarray. Todd From jmiller at stsci.edu Wed May 14 04:31:02 2003 From: jmiller at stsci.edu (Todd Miller) Date: Wed May 14 04:31:02 2003 Subject: [Numpy-discussion] numarray repackaging one more time... References: <3EC1688F.5070606@stsci.edu> <200305141126.07426.hinsen@cnrs-orleans.fr> Message-ID: <3EC22950.8070107@stsci.edu> Konrad Hinsen wrote: >On Tuesday 13 May 2003 23:50, Todd Miller wrote: > > > >>Here are the planned renamings for the numarray modules as we transition >>to a package: >> >>numarray --> numarray.numeric >>recarray --> numarray.records >>chararray --> numarray.strings >>ndarray --> numarray.generic >>* --> numarray.* >> >>Note that class and function names will remain unchanged, so >>recarray.RecArray will become numarray.records.RecArray. >> >>Here are the planned renamings for the current and planned add-on packages >>consolidated into a single add-ons distribution: >> >>LinearAlgrebra2 --> numarray.LinearAlgebra >>FFT2 --> numarray.FFT >>RandomArray2 --> numarray.RandomArray >>Convolve --> numarray.Convolve >>Convolve.Image --> numarray.Image >>MA2 --> numarray.MA >> >> > >Please use a uniform capitalization scheme, no matter which one. Either >numarray.image or Numarray.Image (I prefer the latter, but both are better >than the mixed one). > >Konrad. > > Point taken. I guess that changes the addon mappings to: LinearAlgrebra2 --> numarray.linear_algebra FFT2 --> numarray.fft RandomArray2 --> numarray.random_array Convolve --> numarray.convolve Convolve.Image --> numarray.image MA2 --> numarray.ma or changes everything to: numarray --> Numarray.Numeric recarray --> Numarray.Records chararray --> Numarray.Strings ndarray --> Numarray.Generic * --> Numarray.* LinearAlgrebra2 --> Numarray.LinearAlgebra FFT2 --> Numarray.FFT RandomArray2 --> Numarray.RandomArray Convolve --> Numarray.Convolve Convolve.Image --> Numarray.Image MA2 --> Numarray.MA I have no preference myself (obviously!). Lower case and underscores may be more in line with our group's style. Todd From Kasper.Souren at ircam.fr Wed May 14 04:41:21 2003 From: Kasper.Souren at ircam.fr (Kasper Souren) Date: Wed May 14 04:41:21 2003 Subject: [Numpy-discussion] numarray repackaging one more time... Message-ID: <200305141337.30397.Kasper.Souren@ircam.fr> > I have no preference myself (obviously!). Lower case and underscores > may be more in line with our group's style. The Python Style Guide [1] doesn't give a definitive answer, but there seems to be a preference for the lowercase in this situation. bye, Kasper [1] http://www.python.org/doc/essays/styleguide.html P.S. A simple reply on this list just replies to the owner. I would prefer that it just gives me Numpy-discussion at lists.sourceforge.net instead. -- Hi! I'm a .signature virus! copy me into your .signature file to help me spread! Thanks to DRM, you know that something has been built in environment of unspecified degree of security, from source you cannot check, written by programmers you don't know, released after passing QA of unknown quality and which is released under a license which disclaims any responsibility... From jmiller at stsci.edu Wed May 14 04:55:09 2003 From: jmiller at stsci.edu (Todd Miller) Date: Wed May 14 04:55:09 2003 Subject: [Numpy-discussion] numarray repackaging one more time... References: <3EC1688F.5070606@stsci.edu> <20030513221331.GA14575@arbutus.physics.mcmaster.ca> <20030514092419.5b3cdd10.gvermeul@grenoble.cnrs.fr> <200305141145.07726.falted@openlc.org> Message-ID: <3EC22EC0.90309@stsci.edu> Francesc Alted wrote: >A Dimecres 14 Maig 2003 09:24, Gerard Vermeulen va escriure: > > >>I like the idea, because it would mean that I can release my PyQwt >>plot package so that it can be build for Numeric and numarray >>on the same system. It reduces the risk of calling 'PyQwt for numarray' >>functions with 'Numeric arrays' as arguments and vice-versa (for me it >>seems to work, but it scares me to death). >> >> > >I think this is not necessary as you can always distinguish if the user is >passing a numarray or Numeric object, like in: > >In [3]: a=Numeric.array([1,2]) > >In [4]: b=numarray.array([1,2]) > >In [5]: type(a) >Out[5]: > >In [6]: type(b) >Out[6]: > > >It's just a check before accessing the object. > >Cheers, > > > I think installing in numarray.addons (or not) is a package-by-package decision. Some packages might prefer to be independent. That's fine. Others might prefer (or require) dedicated versions, particularly those packages which use compiled extensions. For those instances where dedicated versions are necessary, numarray.addons is a recommendation to keep clear seperation between core numarray (maintained and distributed by one group) and software which has been layered on it (maintained and distributed by other groups). Todd From bsder at allcaps.org Wed May 14 13:10:16 2003 From: bsder at allcaps.org (Andrew P. Lentvorski, Jr.) Date: Wed May 14 13:10:16 2003 Subject: [Numpy-discussion] numarray repackaging one more time... In-Reply-To: <3EC22950.8070107@stsci.edu> Message-ID: On Wed, 14 May 2003, Todd Miller wrote: > Point taken. I guess that changes the addon mappings to: > > LinearAlgrebra2 --> numarray.linear_algebra > FFT2 --> numarray.fft > RandomArray2 --> numarray.random_array > Convolve --> numarray.convolve > Convolve.Image --> numarray.image > MA2 --> numarray.ma I prefer this naming scheme as it helps visually delineate between modules and classes. ie: numarray.linear_algebra.LUMatrix is likely to be a class numarray.linear_algebra.lumatrix is likely to be a module It seems that most people use StudlyCaps for class names in Python. While there aren't a lot of times that this matters, I have bumped into this more often than I expect. But I may just be doing odd things with modules. -a From bsder at allcaps.org Wed May 14 13:19:04 2003 From: bsder at allcaps.org (Andrew P. Lentvorski, Jr.) Date: Wed May 14 13:19:04 2003 Subject: [Numpy-discussion] Zeroing an existing array Message-ID: What is the official way to zero out an array in numarray/Numeric? While I can create a new array of all zeros and then assign it to the old variable, this is extremely wasteful of memory. Currently, I am just using: >>> import numarray >>> a = numarray.arange(6) >>> a array([0, 1, 2, 3, 4, 5]) >>> a[:] = 0.0 >>> a array([0, 0, 0, 0, 0, 0]) I looked through the manual for a function or array mathod which would accomplish the same thing, but I didn't find an obvious one. Did I miss something obvious? Thanks, -a From jmiller at stsci.edu Wed May 14 13:27:04 2003 From: jmiller at stsci.edu (Todd Miller) Date: Wed May 14 13:27:04 2003 Subject: [Numpy-discussion] Zeroing an existing array In-Reply-To: References: Message-ID: <1052944153.2700.23.camel@localhost.localdomain> On Wed, 2003-05-14 at 16:21, Andrew P. Lentvorski, Jr. wrote: > What is the official way to zero out an array in numarray/Numeric? > I'm don't think there is one in numarray; which is to say, I'd do what you did below. > While I can create a new array of all zeros and then assign it to the old > variable, this is extremely wasteful of memory. > > Currently, I am just using: > > >>> import numarray > >>> a = numarray.arange(6) > >>> a > array([0, 1, 2, 3, 4, 5]) > >>> a[:] = 0.0 > >>> a > array([0, 0, 0, 0, 0, 0]) > > I looked through the manual for a function or array mathod which would > accomplish the same thing, but I didn't find an obvious one. Did I miss > something obvious? I don't think so. Todd From perry at stsci.edu Wed May 14 13:39:08 2003 From: perry at stsci.edu (Perry Greenfield) Date: Wed May 14 13:39:08 2003 Subject: [Numpy-discussion] Zeroing an existing array In-Reply-To: Message-ID: > Currently, I am just using: > > >>> import numarray > >>> a = numarray.arange(6) > >>> a > array([0, 1, 2, 3, 4, 5]) > >>> a[:] = 0.0 > >>> a > array([0, 0, 0, 0, 0, 0]) > Is there any reason for anything else if this does the job? This is how I would recommend the action be performed. Perry Greenfield From verveer at embl.de Wed May 14 15:36:03 2003 From: verveer at embl.de (verveer at embl.de) Date: Wed May 14 15:36:03 2003 Subject: [Numpy-discussion] Re: Numpy-discussion digest, Vol 1 #656 - 11 msgs In-Reply-To: References: Message-ID: <1052951705.3ec2c499c4e08@webmail.embl.de> > As has been mentioned before, we're planning to repackage numarray as > a package rather than as a collection of modules. We're soliciting > comments now because we only want to do this once. [some stuff deleted] > Convolve.Image --> numarray.Image Should this not become "numarray.Convolve.Image" instead of "numarray.Image"? Peter From turner at lanl.gov Wed May 14 16:49:09 2003 From: turner at lanl.gov (John A. Turner) Date: Wed May 14 16:49:09 2003 Subject: [Numpy-discussion] using CXML on Alpha Message-ID: <3EC2D626.4090601@lanl.gov> is anyone using the Compaq Extended Math Libraries (CXML) on a Alpha running Tru64Unix? although Numeric-23.0 builds out-of-the-box and runs fine with its own BLAS/LAPACK, I wanted to try making it use the (significantly faster) CXML implementations I modified this bit of setup.py: # delete all but the first one in this list if using your own LAPACK/BLAS # sourcelist = [os.path.join('Src', 'lapack_litemodule.c'), # os.path.join('Src', 'blas_lite.c'), # os.path.join('Src', 'f2c_lite.c'), # os.path.join('Src', 'zlapack_lite.c'), # os.path.join('Src', 'dlapack_lite.c') # ] sourcelist = [os.path.join('Src', 'lapack_litemodule.c')] # set these to use your own BLAS; library_dirs_list = ['/usr/opt/XMDLIB6510'] libraries_list = ['cxml_ev6'] # if you also set `use_dotblas` (see below), you'll need: # ['lapack', 'cblas', 'f77blas', 'atlas', 'g2c'] (I had to do some hunting around to find where the libs actually live - when using CXML with Fortran all you do is put -lcxml on the command line and it links with the appropriate lib) but this doesn't seem to have done it - after I build and install I see: qsc4% python cg.py Traceback (most recent call last): File "cg.py", line 98, in ? from RandomArray import * File "/users/turner/Tools/T64U/lib/python2.2/site-packages/Numeric/RandomArray.py", line 3, in ? import LinearAlgebra File "/users/turner/Tools/T64U/lib/python2.2/site-packages/Numeric/LinearAlgebra.py", line 8, in ? import lapack_lite ImportError: dlopen: /users/turner/Tools/T64U/lib/python2.2/site-packages/Numeric/lapack_lite.so: symbol "zgelsd_" unresolved I'm pretty sure I'm missing something important... thanks in advance... -John Turner Los Alamos Natl. Lab. Adv. Sci. Sim. (CCS-2) From bsder at allcaps.org Wed May 14 18:08:06 2003 From: bsder at allcaps.org (Andrew P. Lentvorski, Jr.) Date: Wed May 14 18:08:06 2003 Subject: [Numpy-discussion] Zeroing an existing array In-Reply-To: Message-ID: On Wed, 14 May 2003, Perry Greenfield wrote: > > >>> import numarray > > >>> a = numarray.arange(6) > > >>> a > > array([0, 1, 2, 3, 4, 5]) > > >>> a[:] = 0.0 > > >>> a > > array([0, 0, 0, 0, 0, 0]) > > > Is there any reason for anything else if this does the job? > This is how I would recommend the action be performed. It depends. How does the slicing code work? If it iterates across the matrix indicies (requiring multiply and add operations to index specific memory locations), this will be much slower than doing a contiguous memory fill (1 add and 1 multiply per index vs. increment). This is not a small hit, we're talking factors of 2x, 4x, 6x. If the slicing code special cases "array[:] = n" to be a memory fill, then probably not. -a From jensj at fysik.dtu.dk Wed May 14 23:30:08 2003 From: jensj at fysik.dtu.dk (Jens Jorgen Mortensen) Date: Wed May 14 23:30:08 2003 Subject: [Numpy-discussion] using CXML on Alpha In-Reply-To: <3EC2D626.4090601@lanl.gov> References: <3EC2D626.4090601@lanl.gov> Message-ID: <200305150829.19897.jensj@bose.fysik.dtu.dk> On Torsdag den 15. maj 2003 01:49, John A. Turner wrote: > ImportError: dlopen: > /users/turner/Tools/T64U/lib/python2.2/site-packages/Numeric/lapack_lite.so >: symbol "zgelsd_" unresolved Perhaps libcxml does not does not have the zgelsd_ function? Try nm /usr/lib/libcxml_ev6.a | grep zgelds to see if it is there. Jens J?rgen From perry at stsci.edu Thu May 15 06:16:10 2003 From: perry at stsci.edu (Perry Greenfield) Date: Thu May 15 06:16:10 2003 Subject: [Numpy-discussion] Zeroing an existing array In-Reply-To: Message-ID: Andrew P. Lentvorski, Jr. wrote: > On Wed, 14 May 2003, Perry Greenfield wrote: > > > > >>> import numarray > > > >>> a = numarray.arange(6) > > > >>> a > > > array([0, 1, 2, 3, 4, 5]) > > > >>> a[:] = 0.0 > > > >>> a > > > array([0, 0, 0, 0, 0, 0]) > > > > > Is there any reason for anything else if this does the job? > > This is how I would recommend the action be performed. > > It depends. How does the slicing code work? > > If it iterates across the matrix indicies (requiring multiply and add > operations to index specific memory locations), this will be much slower > than doing a contiguous memory fill (1 add and 1 multiply per index vs. > increment). This is not a small hit, we're talking factors of 2x, 4x, 6x. > > If the slicing code special cases "array[:] = n" to be a memory fill, then > probably not. > Well, there are two issues the question relates to: 1) what should the user interface be for filling arrays with a constant value. I'd argue that a slice assignment is both sufficent and idiomatic. 2) is it efficient. As to 2, the current implementation handles scalar assignment by broadcasting a one-element array across the array being assigned to (that's the simplest way to code it). That isn't the fastest implementation when the array being assigned to is congtiguous. It could be made faster. But frankly, it isn't very high on our list of priorities. And I'm a little skeptical that this performance improvement is an important one. Is your program perfromance dominated by the time it takes to zero an array? I have trouble thinking of many realistic problems where that would be true (not to say that there aren't any). If someone is willing to write special code to optimize this, that would be fine with us. Perry From jmiller at stsci.edu Thu May 15 06:44:06 2003 From: jmiller at stsci.edu (Todd Miller) Date: Thu May 15 06:44:06 2003 Subject: [Numpy-discussion] Re: Numpy-discussion digest, Vol 1 #656 - 11 msgs In-Reply-To: <1052951705.3ec2c499c4e08@webmail.embl.de> References: <1052951705.3ec2c499c4e08@webmail.embl.de> Message-ID: <1053006256.19531.226.camel@halloween.stsci.edu> On Wed, 2003-05-14 at 18:35, verveer at embl.de wrote: > > As has been mentioned before, we're planning to repackage numarray as > > a package rather than as a collection of modules. We're soliciting > > comments now because we only want to do this once. > > [some stuff deleted] > > > Convolve.Image --> numarray.Image > > Should this not become "numarray.Convolve.Image" instead of "numarray.Image"? Currently there's one function in Image: translate. I was thinking that there are probably going to be more Image operations for work at STSCI, not necessarily all built on convolution (e.g. rotate). Since we're renaming stuff, I thought we should probably change this now. Todd > > Peter > > > > ------------------------------------------------------- > Enterprise Linux Forum Conference & Expo, June 4-6, 2003, Santa Clara > The only event dedicated to issues related to Linux enterprise solutions > www.enterpriselinuxforum.com > > _______________________________________________ > Numpy-discussion mailing list > Numpy-discussion at lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/numpy-discussion -- Todd Miller jmiller at stsci.edu STSCI / ESS / SSB From jmiller at stsci.edu Thu May 15 12:38:04 2003 From: jmiller at stsci.edu (Todd Miller) Date: Thu May 15 12:38:04 2003 Subject: [Numpy-discussion] numarray repackaging, last call... Message-ID: <1053027509.19531.334.camel@halloween.stsci.edu> Based on the comments I have received, the numarray repackaging is shaping up as shown below: Here are the planned renamings for the numarray modules as we transition to a package: numarray --> numarray.numeric recarray --> numarray.records chararray --> numarray.strings ndarray --> numarray.generic * --> numarray.* Note that class and function names will remain unchanged, so recarray.RecArray will become numarray.records.RecArray. Here are the renamings for the current and planned add-on packages consolidated into a single add-ons distribution: LinearAlgrebra2 --> numarray.linear_algebra FFT2 --> numarray.fft RandomArray2 --> numarray.random_array Convolve --> numarray.convolve Convolve.Image --> numarray.image MA2 --> numarray.ma Future 3rd party packages that require numarray specific C extensions should locate themselves directly in the numarray package. Doing so makes it easy to figure out what to delete when extensions need to be rebuilt. If you don't want to, that's OK too. Barring a sudden burst of interest or superior name choice, I think the top level package name should just remain "numarray". Stub modules will be included in the first few repackaged releases to make it possible to import the primary modules (numarray, recarray, chararray, and ndarray) as you do with numarray-0.5. This will relax the requirement for synchronous change of numarray and software which uses numarray. The stub modules will be activated either through the creation of a .pth file or via additions to PYTHONPATH to make them visible. This is a backwards compatability mode only, not recommended usage following the repackaging. Thanks to everyone who responded. Final comments? -- Todd Miller jmiller at stsci.edu STSCI / ESS / SSB From jmiller at stsci.edu Fri May 16 11:37:09 2003 From: jmiller at stsci.edu (Todd Miller) Date: Fri May 16 11:37:09 2003 Subject: [Numpy-discussion] Re: change in semantics of arange? In-Reply-To: <000001c31bcd$24ef1600$6801a8c0@NICKLEBY> References: <000001c31bcd$24ef1600$6801a8c0@NICKLEBY> Message-ID: <1053110274.993.105.camel@halloween.stsci.edu> On Fri, 2003-05-16 at 13:03, Paul Dubois wrote: > C:\numpy\numarray\Packages\MA2>python > Python 2.3b1 (#40, Apr 25 2003, 19:06:24) [MSC v.1200 32 bit (Intel)] on > win32 > Type "help", "copyright", "credits" or "license" for more information. > >>> import numarray > >>> x=numarray.arange(5,3,2) > Traceback (most recent call last): > File "", line 1, in ? > File "c:\python23\Lib\site-packages\numarray\numarray.py", line 924, in > arange > r = _fillarray(size, start, stride, type) > File "c:\python23\Lib\site-packages\numarray\numarray.py", line 120, in > _fillarray > outarr = NumArray((size,), outtype) > libnumarray.error: NA_updateDataPtr: error getting read buffer data ptr > >>> import Numeric > >>> Numeric.arange(5,3,2) > zeros((0,), 'l') > >>> > > Is this change intentional? No. It's bugs in arange and also the memory object which are creating a negative length buffer. The buffer API then reports a negative length which is fortuitously interpreted as an error. Ya gotta smile... It's fixed in CVS. I modified the memory module so that: >>> import memory >>> memory.new_memory(-10) Traceback (most recent call last): File "", line 1, in ? ValueError: new_memory: invalid region size: -10. And I modified arange so that negative sizes are clipped to 0. So now: >>> import numarray >>> numarray.arange(5,3,2) array([]) > -- Todd Miller jmiller at stsci.edu STSCI / ESS / SSB From haase at msg.ucsf.edu Tue May 20 09:19:04 2003 From: haase at msg.ucsf.edu (Sebastian Haase) Date: Tue May 20 09:19:04 2003 Subject: [Numpy-discussion] Numarray 0.5 References: <200305061424.26714.verveer@embl-heidelberg.de> <1052258543.27223.55.camel@halloween.stsci.edu> Message-ID: <009101c31eeb$90235b90$421ee6a9@rodan> Hi All, After I read this thread I thought I would wait a bit before upgrading my numarray (from 0.4) - Are the mentioned fixes available somewhere ? Or actually: Is the CVS version publicly readable and - if so - would you recommend using that ? Thanks, Sebastian Haase ----- Original Message ----- From: "Todd Miller" To: "Peter Verveer" Cc: Sent: Tuesday, May 06, 2003 3:02 PM Subject: Re: [Numpy-discussion] Numarray 0.5 > On Tue, 2003-05-06 at 08:24, Peter Verveer wrote: > > Hi All, > > > > I found the following problems after testing my software with numarray 0.5: > > > > 1) Following works fine if both a and b are arrays: > > > > >>> a = array([2]) > > >>> b = array([1, 2]) > > >>> print a + b > > [3 4] > > > > However, if b is an python sequence: > > > > >>> a = array([2]) > > >>> b = [1, 2] > > >>> print a + b > > [3] > > > > Apparently broadcasting does not work with python sequeces anymore. This used > > to work fine in version 0.4. Is this a bug? > > Yes, unfortunately. Thanks for reporting it! > > > > > 2) It is not possible to compare an array type object to the 'Any' object: > > > > >>> Float64 == Any > > Traceback (most recent call last): > > File "", line 1, in ? > > File "/usr/local/lib/python2.2/site-packages/numarray/numerictypes.py", line > > 112, in __cmp__ > > return (genericTypeRank.index(self.name) - > > ValueError: list.index(x): x not in list > > > > I am not sure if this is a bug, or intended behaviour, but the possibilty to > > compare an array type object to 'Any' would be very useful for me. > > > > 'Any' grew up from the C-API, rather than down from the Python design, > so it's not very well thought out. Right now, it is a placeholder used > to mark arrays with undefined types and to indicate "no type constraint" > in C API calls. In normal contexts, you can't make an array of type > 'Any'. I think there are two reasonable behaviors for comparisons with > 'Any', both used in C. The first behavior is literal comparison; here > comparison to Any would generally return "not equal". The second > behavior is wild-card matching; here, comparison to Any would generally > return "equal". Which makes sense to you? How do you want to use > this? > > > 3) The NA_typeNoToTypeObject() function fails if it is called before any > > arrays are created. It looks to me as if the pNumType array in > > libnumarraymodule.c is not initialized until an array is created. I don't > > know if any other functions are affected in the same way. Could this be > > fixed? > > Yes. This is fixed now in CVS. Thanks again! > > > > > Cheers, Peter > > > > -- > > Dr. Peter J. Verveer > > Cell Biology and Cell Biophysics Programme > > EMBL > > Meyerhofstrasse 1 > > D-69117 Heidelberg > > Germany > > Tel. : +49 6221 387245 > > Fax : +49 6221 387242 > > Email: Peter.Verveer at embl-heidelberg.de > > > > > > > > > > ------------------------------------------------------- > > This sf.net email is sponsored by:ThinkGeek > > Welcome to geek heaven. > > http://thinkgeek.com/sf > > _______________________________________________ > > Numpy-discussion mailing list > > Numpy-discussion at lists.sourceforge.net > > https://lists.sourceforge.net/lists/listinfo/numpy-discussion > -- > Todd Miller jmiller at stsci.edu > STSCI / ESS / SSB > > > > ------------------------------------------------------- > Enterprise Linux Forum Conference & Expo, June 4-6, 2003, Santa Clara > The only event dedicated to issues related to Linux enterprise solutions > www.enterpriselinuxforum.com > > _______________________________________________ > Numpy-discussion mailing list > Numpy-discussion at lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/numpy-discussion > From jmiller at stsci.edu Tue May 20 10:08:12 2003 From: jmiller at stsci.edu (Todd Miller) Date: Tue May 20 10:08:12 2003 Subject: [Numpy-discussion] Numarray 0.5 In-Reply-To: <009101c31eeb$90235b90$421ee6a9@rodan> References: <200305061424.26714.verveer@embl-heidelberg.de> <1052258543.27223.55.camel@halloween.stsci.edu> <009101c31eeb$90235b90$421ee6a9@rodan> Message-ID: <1053450454.25500.13.camel@halloween.stsci.edu> On Tue, 2003-05-20 at 12:19, Sebastian Haase wrote: > Hi All, > After I read this thread I thought I would wait a bit before upgrading my > numarray (from 0.4) - numarray-0.6 is probably not far off; if you're on the fence, you might want to sit this out. However, there is a new API function which I added in response to your last post: NA_NewAllFromBuffer() which enables you to create arrays in C from existing buffer objects rather than just C arrays. Also, I modified setup.py to support independent build and install. Both of these new features are under tested. Let me know how it works out. > Are the mentioned fixes available somewhere ? All of the problems Peter found have been fixed as of yesterday. The fixes are checked into CVS on Source Forge in the numarray component of the numpy project. Instructions for doing an anonymous numarray cvs checkout are here: http://sourceforge.net/cvs/?group_id=1369 These basically say to: % cvs -d:pserver:anonymous at cvs.sourceforge.net:/cvsroot/numpy login % cvs -z3 -d:pserver:anonymous at cvs.sourceforge.net:/cvsroot/numpy co numarray > Or actually: Is the CVS version publicly readable and - if so - would you > recommend using that ? If you check out today, you'll be OK. I tagged "now" as v0_5_2 which will never be "officially released" but which just passed all self-tests under i386-Linux. Todd -- Todd Miller jmiller at stsci.edu STSCI / ESS / SSB From camartin at snet.net Wed May 21 08:09:02 2003 From: camartin at snet.net (Cliff Martin) Date: Wed May 21 08:09:02 2003 Subject: [Numpy-discussion] Finding indices and setting values Message-ID: <3ECB9685.6060809@snet.net> Hi, I have a problem where I want to set values in a 2D array based on conditions I?ve established. The variables are the NA of the system, its wavelength and the correct units on the device I?m modeling. Using these variables I define a variable N. I set up a 512 by 512 array [x,y]. Then I set up a radius , r = x^2 +y^2 and ask it to give me all r?s(to the closest integer) <= N(the variable I?ve defined above). This gives all the index values in that radius and then I set all those locations to a value of 1. After this I do some FFT ?s, etc. So how do I do this in Numerical Python without having to index through i,j steps which would be incredibly tedious. This works fairly well in MatLab but I want to port my program to Python(for lots of reasons). If you?d rather I write the MatLab code snippet I can do that. Thanks. Cliff Martin From perry at stsci.edu Wed May 21 08:42:07 2003 From: perry at stsci.edu (Perry Greenfield) Date: Wed May 21 08:42:07 2003 Subject: [Numpy-discussion] Finding indices and setting values In-Reply-To: <3ECB9685.6060809@snet.net> Message-ID: > > I have a problem where I want to set values in a 2D array based on > conditions I?ve established. The variables are the NA of the system, its > wavelength and the correct units on the device I?m modeling. Using these > variables I define a variable N. I set up a 512 by 512 array [x,y]. Then > I set up a radius , r = x^2 +y^2 and ask it to give me all r?s(to the > closest integer) <= N(the variable I?ve defined above). This gives all > the index values in that radius and then I set all those locations to a > value of 1. After this I do some FFT ?s, etc. So how do I do this in > Numerical Python without having to index through i,j steps which would > be incredibly tedious. This works fairly well in MatLab but I want to > port my program to Python(for lots of reasons). If you?d rather I write > the MatLab code snippet I can do that. Thanks. > > Cliff Martin > One way should be something like this (I haven't tested this) This only works in numarray. If data is the 2-d array where you want to set values within a radius of N (of x0, y0 I presume) y, x = indices(data.shape) data[nonzero(((x-x0)**2+(y-y0)**2) < N**2)] = 1 (in a future version of numarray this will also work and the meaning should be clearer: data[where(((x-x0)**2+(y-y0)**2) < N**2)] = 1 It's a little more involved for Numeric but still possible to do without resorting to loops (I'll leave it to someone else to show that). Does this do what you wanted? Perry Greenfield. From jmiller at stsci.edu Wed May 21 09:01:14 2003 From: jmiller at stsci.edu (Todd Miller) Date: Wed May 21 09:01:14 2003 Subject: [Numpy-discussion] Finding indices and setting values In-Reply-To: <3ECB9685.6060809@snet.net> References: <3ECB9685.6060809@snet.net> Message-ID: <1053531800.1072.47.camel@localhost.localdomain> On Wed, 2003-05-21 at 11:08, Cliff Martin wrote: > Hi, > > I have a problem where I want to set values in a 2D array based on > conditions I?ve established. The variables are the NA of the system, its > wavelength and the correct units on the device I?m modeling. Using these > variables I define a variable N. I set up a 512 by 512 array [x,y]. Then > I set up a radius , r = x^2 +y^2 and ask it to give me all r?s(to the > closest integer) <= N(the variable I?ve defined above). This gives all > the index values in that radius and then I set all those locations to a > value of 1. After this I do some FFT ?s, etc. So how do I do this in > Numerical Python without having to index through i,j steps which would > be incredibly tedious. This works fairly well in MatLab but I want to > port my program to Python(for lots of reasons). If you?d rather I write > the MatLab code snippet I can do that. Thanks. > > Cliff Martin I think part of your code looks like this: x,y = Numeric.indices((512,512)) r = x**2 + y**2 c = r < N In numarray you can then say: a[ c ] = 1 In Numeric, I think you say: a = a.flat c = c.flat Numeric.put(a, Numeric.nonzero(c), 1) a.shape = (512,512) Todd From magnus at hetland.org Wed May 21 11:53:04 2003 From: magnus at hetland.org (Magnus Lie Hetland) Date: Wed May 21 11:53:04 2003 Subject: [Numpy-discussion] Finding indices and setting values In-Reply-To: References: <3ECB9685.6060809@snet.net> Message-ID: <20030521185206.GC10762@idi.ntnu.no> Perry Greenfield : [snip] > (in a future version of numarray this will also work and the meaning > should be clearer: data[where(((x-x0)**2+(y-y0)**2) < N**2)] = 1 Cool... Do you know which version? -- Magnus Lie Hetland "In this house we obey the laws of http://hetland.org thermodynamics!" Homer Simpson From Kasper.Souren at ircam.fr Wed May 21 12:56:05 2003 From: Kasper.Souren at ircam.fr (Kasper Souren) Date: Wed May 21 12:56:05 2003 Subject: [Numpy-discussion] faster way for calculating distance matrix... Message-ID: <200305211943.17529.Kasper.Souren@ircam.fr> Hi, I was happy to see solutions coming up for Cliff's problems and it made me think: maybe I can mention my problem here as well. From an array of feature vectors I want to calculate it's distance matrix. Something like [1]. Currently it can take quite a while to calculate the stuff for a long array. Some questions: 1) Is there a smart speed up possible? Like, a way to avoid the double loop? It's no problem if this would lead to less generality (like the choice for a distance function). I know there is a little speed up to be gained by leaving out the array(f) thing, but that's not what I'm looking for. 2) Is it possible (in Numeric or numarray) to define a class DiagonalMatrix that at least saves half of the memory? 3) If 1) is not possible, what would be the way to go for speeding it up by writing it in C? weave because of its availability in scipy or would pyrex be more interesting, or are there even more options..? bye, Kasper [1] Example program: import Numeric def euclidean_dist(a, b): diff = a - b return Numeric.dot(diff, diff) def calc_dist_matrix(f, distance_function): W = Numeric.array(f) length = W.shape[0] S = Numeric.zeros((length, length)) * 1.0 for i in range(length): for j in range(i): S[j, i] = S[i, j] = distance_function(W[i], W[j]) return S print calc_dist_matrix(Numeric.sin(Numeric.arange(30)), euclidean_dist) From perry at stsci.edu Wed May 21 14:10:06 2003 From: perry at stsci.edu (Perry Greenfield) Date: Wed May 21 14:10:06 2003 Subject: [Numpy-discussion] Finding indices and setting values In-Reply-To: <1053531800.1072.47.camel@localhost.localdomain> Message-ID: > > x,y = Numeric.indices((512,512)) > r = x**2 + y**2 > c = r < N > > In numarray you can then say: > > a[ c ] = 1 > > In Numeric, I think you say: > > a = a.flat > c = c.flat > Numeric.put(a, Numeric.nonzero(c), 1) > a.shape = (512,512) > > > Todd > Yeah, that's what I would do in Numeric :-) I was thinking that the solution I gave was wasteful of memory if N was generally much smaller than the dimensions of data. Something like this (also untested) would be better in that regard (again for numarray) y, x = indices((2*N+1, 2*N+1)) yind, xind = nonzero(((x-N)**2+(y-N)**2) < N**2) data[yind+y0-N, xind+x0-N] = 1 Note that this doesn't check to see if x0 and y0 are at least N away from the boundaries of data, and also note my convention for x and y is different than what Todd used (it depends on how you interpret data in arrays, the convention I use is more common for image data if you think of x corresponding to the most rapidly varying index). The advantage of this approach is that the x and y arrays are small if N is small, and computing xind, yind take much less time than for very large arrays. Doing this with Numeric would be much messier I believe (but a clever person could prove me wrong). Perry Greenfield From jcollins at boulder.net Wed May 21 18:03:02 2003 From: jcollins at boulder.net (Jeffery D. Collins) Date: Wed May 21 18:03:02 2003 Subject: [Numpy-discussion] faster way for calculating distance matrix... In-Reply-To: <200305211943.17529.Kasper.Souren@ircam.fr> References: <200305211943.17529.Kasper.Souren@ircam.fr> Message-ID: <3ECC220E.3090700@boulder.net> Kasper Souren wrote: >Hi, > >I was happy to see solutions coming up for Cliff's problems and it made me >think: maybe I can mention my problem here as well. > >>From an array of feature vectors I want to calculate it's distance matrix. >Something like [1]. Currently it can take quite a while to calculate the >stuff for a long array. > >Some questions: > >1) Is there a smart speed up possible? Like, a way to avoid the double loop? >It's no problem if this would lead to less generality (like the choice for a >distance function). I know there is a little speed up to be gained by leaving >out the array(f) thing, but that's not what I'm looking for. > >2) Is it possible (in Numeric or numarray) to define a class DiagonalMatrix >that at least saves half of the memory? > >3) If 1) is not possible, what would be the way to go for speeding it up by >writing it in C? weave because of its availability in scipy or would pyrex be >more interesting, or are there even more options..? > >bye, >Kasper > > > >[1] Example program: > >import Numeric > >def euclidean_dist(a, b): > diff = a - b > return Numeric.dot(diff, diff) > >def calc_dist_matrix(f, distance_function): > W = Numeric.array(f) > length = W.shape[0] > S = Numeric.zeros((length, length)) * 1.0 > > for i in range(length): > for j in range(i): > S[j, i] = S[i, j] = distance_function(W[i], W[j]) > return S > >print calc_dist_matrix(Numeric.sin(Numeric.arange(30)), euclidean_dist) > > The following is a bit faster (at the cost of some memory): def euclidean_dist(a, b): diff = a - b return diff**2 def calc_dist_matrix2(f, distance_function): W = Numeric.array(f) i,j = indices((len(W),)*2) S = distance_function(take(W,i), take(W,j)) return S -- Jeffery Collins (http://www.boulder.net/~jcollins) From christopheranderson at shaw.ca Tue May 27 08:55:15 2003 From: christopheranderson at shaw.ca (christopheranderson at shaw.ca) Date: Tue May 27 08:55:15 2003 Subject: [Numpy-discussion] Numeric -> numarray gives MemoryError Message-ID: Hi, I've started to move some of my Numeric code to numarray 0.5 and have run into a problem. This is a relatively large optimization problem that requires finite difference estimation of gradients. The implementation uses either Numeric/LinearAlgebra/RandomArray or numarray/LinearAlgebra2/RandomArray2; these are the only extensions used. The code itself is fairly straightforward and ports from Numeric to numarray with no modifications. Typical run times are on the order of hours with Numeric. When using Numeric, I am able to repeat multiple optimization runs (> 10) with no problems. When the same code is run with numarray instead, I get this error part of the way through the first run: Traceback (most recent call last): File "runFD2.py", line 81, in ? W1, W2 = estimFD2(inputs, outputs, Nh, Ne, alpha) File "optimFD2.py", line 171, in estimFD2 gradW1, gradW2 = dEdW(inputs, outputs, MSE, alpha, W1, W2, f1, f2) File "optimFD2.py", line 83, in dEdW y_plus = evalFD2(inputs, W1_plus, W2, f1, f2) File "optimFD2.py", line 63, in evalFD2 h1 = dot(inputs, W1) File "F:\Python22\Lib\site-packages\numarray\numarray.py", line 940, in dot return innerproduct(a, swapaxes(inputarray(b), -1, -2)) File "F:\Python22\Lib\site-packages\numarray\ufunc.py", line 1866, in innerproduct a = a.astype(rtype) File "F:\Python22\Lib\site-packages\numarray\numarray.py", line 478, in astype return self.copy() File "F:\Python22\Lib\site-packages\numarray\numarray.py", line 553, in copy c = ndarray.NDArray.copy(self) File "F:\Python22\Lib\site-packages\numarray\ndarray.py", line 571, in copy arr._data = memory.new_memory(arr._itemsize * arr.nelements()) MemoryError >>> This is a little frustrating, because numarray is clearly much faster. Checking the results along the way shows that numarray gives the same outputs as Numeric, so it doesn't appear to be a porting issue. The datasets involved are large (~10000x20). Initial memory usage with Numeric and numarray is similar, and is well below the machine's limit. The code is running on a Windows 2000 machine using the binary release of numarray. What might be causing this problem? Should numarray be less tolerant than Numeric in a situation like this? Thanks. Chris From jmiller at stsci.edu Tue May 27 09:48:09 2003 From: jmiller at stsci.edu (Todd Miller) Date: Tue May 27 09:48:09 2003 Subject: [Numpy-discussion] Numeric -> numarray gives MemoryError In-Reply-To: References: Message-ID: <1054054030.28955.27.camel@halloween.stsci.edu> On Tue, 2003-05-27 at 11:52, christopheranderson at shaw.ca wrote: > Hi, > > I've started to move some of my Numeric code to numarray 0.5 and have run into a problem. This is a relatively large optimization problem that requires finite difference estimation of gradients. > > The implementation uses either Numeric/LinearAlgebra/RandomArray or numarray/LinearAlgebra2/RandomArray2; these are the only extensions used. The code itself is fairly straightforward and ports from Numeric to numarray with no modifications. Typical run times are on the order of hours with Numeric. > > When using Numeric, I am able to repeat multiple optimization runs (> 10) with no problems. When the same code is run with numarray instead, I get this error part of the way through the first run: > > Traceback (most recent call last): > File "runFD2.py", line 81, in ? > W1, W2 = estimFD2(inputs, outputs, Nh, Ne, alpha) > File "optimFD2.py", line 171, in estimFD2 > gradW1, gradW2 = dEdW(inputs, outputs, MSE, alpha, W1, W2, f1, f2) > > File "optimFD2.py", line 83, in dEdW > y_plus = evalFD2(inputs, W1_plus, W2, f1, f2) > File "optimFD2.py", line 63, in evalFD2 > h1 = dot(inputs, W1) > File "F:\Python22\Lib\site-packages\numarray\numarray.py", line 940, in dot > return innerproduct(a, swapaxes(inputarray(b), -1, -2)) > File "F:\Python22\Lib\site-packages\numarray\ufunc.py", line 1866, in innerproduct > a = a.astype(rtype) > File "F:\Python22\Lib\site-packages\numarray\numarray.py", line 478, in astype > > return self.copy() > File "F:\Python22\Lib\site-packages\numarray\numarray.py", line 553, in copy > c = ndarray.NDArray.copy(self) > File "F:\Python22\Lib\site-packages\numarray\ndarray.py", line 571, in copy > arr._data = memory.new_memory(arr._itemsize * arr.nelements()) > MemoryError > >>> > > This is a little frustrating, because numarray is clearly much faster. Checking the results along the way shows that numarray gives the same outputs as Numeric, so it doesn't appear to be a porting issue. The datasets involved are large (~10000x20). Initial memory usage with Numeric and numarray is similar, and is well below the machine's limit. The code is running on a Windows 2000 machine using the binary release of numarray. > Well... the faster part is good. :) > What might be causing this problem? It sounds to me like a memory leak (reference counting error) in numarray. Did you run the same code with older versions of numarray, or did you start with numarray-0.5? How long does it take to fail? Can you mail me your code (or better, the smallest derivative of it which reproduces the problem)? > Should numarray be less tolerant than Numeric in a situation like this? No, it's just newer and less mature. > > Thanks. You're welcome! Todd > > Chris > > > > > ------------------------------------------------------- > This SF.net email is sponsored by: ObjectStore. > If flattening out C++ or Java code to make your application fit in a > relational database is painful, don't do it! Check out ObjectStore. > Now part of Progress Software. http://www.objectstore.net/sourceforge > _______________________________________________ > Numpy-discussion mailing list > Numpy-discussion at lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/numpy-discussion -- Todd Miller jmiller at stsci.edu STSCI / ESS / SSB From christopheranderson at shaw.ca Tue May 27 13:24:04 2003 From: christopheranderson at shaw.ca (christopheranderson at shaw.ca) Date: Tue May 27 13:24:04 2003 Subject: [Numpy-discussion] Re: Numeric -> numarray gives MemoryError Message-ID: <5c8ec60da7.60da75c8ec@shaw.ca> Todd, thanks very much for the quick reply. > >Well... the faster part is good. :) Yes indeed. > > > What might be causing this problem? > >It sounds to me like a memory leak (reference counting error) in >numarray. > >Did you run the same code with older versions of numarray, or did you >start with numarray-0.5? > >How long does it take to fail? This was my first stab at using numarray, so I haven't tried with prior versions. Not that this means anything, but it usually dies around 500 or so iterations through the optimization routine. For comparison, Numeric will cheerfully run for several thousands of iterations, multiple times, without complaint. >Can you mail me your code (or better, the smallest derivative of it >which reproduces the problem)? This might be a little complicated. I'll try stripping things down to isolate the problem. Chris From camartin at snet.net Thu May 29 13:17:04 2003 From: camartin at snet.net (Cliff Martin) Date: Thu May 29 13:17:04 2003 Subject: [Numpy-discussion] fftshift Message-ID: <3ED66AB5.40209@snet.net> All, First I'd like to thank Paul Dubois, Todd Miller and Perry Greenfield for the help you gave me on setting defined values based on index values in an array. Your fixes worked and let me get done what was needed. This leads me to my second problem. In Matlab there is a function called fftshift that lets one shift the fft values for more easy visualization. Numeric doesn't seem to have a function like that explicitly although I suppose one could use the flip function in the MatPy module. Has anyone implemented something like the fftshift function or could you suggest the best way to do this? Thanks for your help. Oh I'm doing this on a 2D array. Cliff Martin From pearu at cens.ioc.ee Thu May 29 13:48:02 2003 From: pearu at cens.ioc.ee (Pearu Peterson) Date: Thu May 29 13:48:02 2003 Subject: [Numpy-discussion] fftshift In-Reply-To: <3ED66AB5.40209@snet.net> Message-ID: On Thu, 29 May 2003, Cliff Martin wrote: > This leads me to my second problem. In Matlab there is a function > called fftshift that lets one shift the fft values for more easy > visualization. Numeric doesn't seem to have a function like that > explicitly although I suppose one could use the flip function in the > MatPy module. Has anyone implemented something like the fftshift > function or could you suggest the best way to do this? Thanks for your > help. Oh I'm doing this on a 2D array. scipy.fftpack has fftshift. If you don't care building scipy yourself then the implementation of fftshift can be viewed via CVSview at www.scipy.org, look at the file scipy/Lib/fftpack/helper.py HTH, Pearu