From gb at cs.unc.edu Wed Oct 3 12:08:08 2001 From: gb at cs.unc.edu (Gary Bishop) Date: Wed Oct 3 12:08:08 2001 Subject: [Numpy-discussion] Changes to improve performance on small matricies Message-ID: <200110031907.PAA02478@wren.cs.unc.edu> We are porting some code from Matlab that does many thousands of operations on small matrices. We have found that some small changes to LinearAlgebra.py, Numeric.py, and multiarraymodule.c make our code run approximately twice as fast as with the unmodified Numeric-20.2 code. Our changes specifically replace LinearAlgebra._castCopyAndTranspose with a version we call _fastCopyAndTranspose that does most of the work in C instead of calling Numeric.transpose, which calls arange, which is quite expensive. We also optimized Numeric.dot to call a new function multiarray.matrixproduct which does the axis swap on the fly instead of calling swapaxes (which calls arange) and then calling innerproduct (which as the very first step copies the transposed matrix to make it contiguous). Formerly our code spent much of its time in arange (which we never explicitly call), dot, and _castCopyAndTranspose. The changes described above eliminate this overhead. I'm writing to ask if these changes might make a worthy patch to NumPy? We have tested them with on Windows2k (both Native and under Cygwin) and on Linux. Soon, we'll have a test on Mac OS X.1. If anyone is interested, I will figure out how to generate a patch file to submit. Thanks gb From vanandel at atd.ucar.edu Wed Oct 3 12:55:03 2001 From: vanandel at atd.ucar.edu (Joe Van Andel) Date: Wed Oct 3 12:55:03 2001 Subject: [Numpy-discussion] pgcc (Portland Group C compiler) performance improvements for Numeric applications? Message-ID: <3BBB6CFE.94A6730E@atd.ucar.edu> Has anyone benchmarked their Numeric applications using 'pgcc'? I'm interested in improving the performance of my Numeric Python code, and I'm wondering what speedup I might achieve, just by re-compiling Python, Numeric, and my extensions with pgcc. (We currently use gcc 2.95.3) Also, has anyone used the multiprocessor (OpenMP) or CDK (Cluster Development Kit) tools from Portland Group? -- Joe VanAndel National Center for Atmospheric Research http://www.atd.ucar.edu/~vanandel/ Internet: vanandel at ucar.edu From chrishbarker at home.net Thu Oct 4 16:45:02 2001 From: chrishbarker at home.net (Chris Barker) Date: Thu Oct 4 16:45:02 2001 Subject: [Numpy-discussion] Changes to improve performance on small matricies References: <200110031907.PAA02478@wren.cs.unc.edu> Message-ID: <3BBCF875.B274B2BB@home.net> Gary Bishop wrote: > I'm writing to ask if these changes might make a worthy patch to NumPy? > We have tested them with on Windows2k (both Native and under Cygwin) > and on Linux. Soon, we'll have a test on Mac OS X.1. I'd say yes, but we do have to be satisfied with the robustness of the code, and I have no idea how to go about doing that. This brings up a good point. In Numeric.Py, the is the following comment: #The following functions are considered builtin, they all might be #in C some day I'd love to see that happen. As evidenced by this poster's example, Numeric could be made a lot faster by optimizing a number of build-in functions. What would it take to get a "Numeric Optimization project" going?? A few ideas: unit tests: If we had unit tests designed for all the built-in functions, we would have a good way to know if a newly coded C version is working right. How-To_Docs: I have tried to write some Numeric extensions myself, and it is very difficult. Writing one that does something unusual for my particular application is not so hard with the current docs. In that case it usually only has to work for one type, I know what shape the array should be etc. Writing a universal extension to Numeric that would work on any Numeric Array is a whole lot harder!! I am currently working on two of these, and the hard parts for me are: Doing something to all the data in a perhaps discontiguous array of arbitrary shape. I have soemthing working now, but it's pretty slow. Having a function work for all types of Arrays: dealing this this kin dof dynamicism on C is very difficult for me. The best place to look is clearly the source itself, but I have tried, and it's pretty darn dense. A How-To would be pretty darn helpful. I don't know who's going to write these things, but I think you could enlarge the pool of NumPy developers quite a bit. If anyone is interested, I have written "fastclip()" which does what clip() does, but it does it in place, and only works on Floats (see above) It is very fast for contiguous arrays, and disappointingly slow on discontiguous arrays (also see above). Utill I get it working on all types, there isn't much point in submitting it for inclusion in NumPy. The other two I have written I use for dealing with binary data coming in from files: byteswap() does a bytswap in place, whioch is faster an more memory efficient that the existing bytswapped() method. changetype() changes the type of the array, using the same data: changetype(m,typecode) = fromstring(m.tostring(),typecode) It is a lot faster and memory efficient (the memory was the issue for me, I'm dealing with up to 10MB arrays), but kind of an obscure need to, so this may not be a candidate for inclusion in NumPy either. I still have some questions about my code for that one: I'll post a separate message about that. -Chris -- Christopher Barker, Ph.D. ChrisHBarker at home.net --- --- --- http://members.home.net/barkerlohmann ---@@ -----@@ -----@@ ------@@@ ------@@@ ------@@@ Oil Spill Modeling ------ @ ------ @ ------ @ Water Resources Engineering ------- --------- -------- Coastal and Fluvial Hydrodynamics -------------------------------------- ------------------------------------------------------------------------ From chrishbarker at home.net Thu Oct 4 16:58:02 2001 From: chrishbarker at home.net (Chris Barker) Date: Thu Oct 4 16:58:02 2001 Subject: [Numpy-discussion] Coding questions for a type change function References: <200110031907.PAA02478@wren.cs.unc.edu> Message-ID: <3BBCFB85.886E8405@home.net> HI all, I've written a function (in C) that changes the type of an array, in place, using the same data block. The reason I need this is because I am reading in a large block of data from a file that is essentially a set of records, each one a C struct that has a couple of longs and a char in it. I read it in as a big array, slice iot to separate the fields, and then need to change the type. I had been using: A = fromstring(A.tostring(),Int32) This works fine, but needs a lot of memory, and it kind of slow (althought enough faster than the file read that it matters little) The array can be on order of 10MB however, so the memory issue is a problem. Anyway, I wrote a function that changes the type of data in an array, without making any copies of the data. It seems to work but I have a few questions (code to follow): 1) Is there an easier way to do this? I just notices that PyArray_As1D takes a type as an input... would it do what I want?? 2) Is there a way , at the C levbel to reshape an array. PyArray_Reshape takes a PyObject, which is kind of awkward, I'd love to pass in a few integers, or maybe a nd, and a pointer to a dimensions array. 3) What I did do is use PyArray_DescrFromType() to create a new desciption, and then assign the description pointer of the array to the new one. This seems to work, but I expect that the old description memory never gets freed. How do I free it (can you tell I'm a C newbie?). Note that I have called this function is a loop thousands of times, and not seen memory use go up, but it's a pretty small structure. 4) If I call PyArray_DescrFromType() with an invalid typcode, I get a crash. How can I check if the typecode is valid?? The code follows, I'd love any feedback anyone can offer. -Chris static PyObject * NumericExtras_changetype(PyObject *self, PyObject *args) { PyArrayObject *Array; //int *new_strides; //int *new_dimensions; PyArray_Descr *NewDescription; int TotalBytes; int NewElementSize; char typecode; if (!PyArg_ParseTuple(args, "O!c", &PyArray_Type, &Array, &typecode)) { return NULL; } if (!PyArray_ISCONTIGUOUS(Array)){ PyErr_SetString (PyExc_ValueError, "m must be contiguous"); return NULL; } TotalBytes = PyArray_NBYTES(Array); // This will crash if an invalid typecode is passed in. NewDescription = PyArray_DescrFromType(typecode); NewElementSize = NewDescription->elsize; if (TotalBytes % NewElementSize > 0){ PyErr_SetString (PyExc_ValueError, "The input array is the wrong size for the requested type"); return NULL; } Array->descr = NewDescription; // does the old descr need to be freed?? how?? Array->nd = 1; Array->strides[0] = NewElementSize; Array->dimensions[0] = TotalBytes / NewElementSize; return Py_BuildValue(""); } -- Christopher Barker, Ph.D. ChrisHBarker at home.net --- --- --- http://members.home.net/barkerlohmann ---@@ -----@@ -----@@ ------@@@ ------@@@ ------@@@ Oil Spill Modeling ------ @ ------ @ ------ @ Water Resources Engineering ------- --------- -------- Coastal and Fluvial Hydrodynamics -------------------------------------- ------------------------------------------------------------------------ From lindy at scripps.edu Fri Oct 5 10:11:05 2001 From: lindy at scripps.edu (Lindy Lindstrom) Date: Fri Oct 5 10:11:05 2001 Subject: [Numpy-discussion] Numeric.put bug References: Message-ID: <3BBDE98C.55CED05A@scripps.edu> Has anyone yet noted this? The following code works in python1.5.2 (Numeric.__version__ == '11'). However, in python2.0 (with Numeric.__version__ == '17.1.1') and python2.1 (with Numeric.__version__ == '20.0.0'), it doesn't work correctly. coords[3][3] is mis-assigned. It happens at least on SGI and Sun. I've not seen anything in the change lists for 20.1 or 20.2 about such a bug. # # bug.py # import Numeric coords = Numeric.resize(Numeric.array(range(17*4)), (17,4)) print "before coords[3]: ", coords[3] list = [3, 15] takecoords = Numeric.take(coords, list) print "takecoords:" print takecoords print "coords.iscontiguous: ", coords.iscontiguous() Numeric.put(coords, list, takecoords) print "after coords[3]: ", coords[3] ********************************** William M. Lindstrom, Jr., Ph.D. Molecular Graphics Laboratory Department of Molecular Biology, MB-5 The Scripps Research Institute 10550 North Torrey Pines Road La Jolla, CA 92037-1000 USA Tel: (858) 784-2055 Fax: (858) 784-2860 E-mail: lindy at scripps.edu From oliphant.travis at ieee.org Fri Oct 5 13:18:04 2001 From: oliphant.travis at ieee.org (Travis Oliphant) Date: Fri Oct 5 13:18:04 2001 Subject: [Numpy-discussion] Numeric.put bug In-Reply-To: <3BBDE98C.55CED05A@scripps.edu> References: <3BBDE98C.55CED05A@scripps.edu> Message-ID: On Friday 05 October 2001 11:10 am, you wrote: > Has anyone yet noted this? > > The following code works in python1.5.2 (Numeric.__version__ == '11'). > However, in python2.0 (with Numeric.__version__ == '17.1.1') and > python2.1 (with Numeric.__version__ == '20.0.0'), it doesn't work > correctly. > coords[3][3] is mis-assigned. It happens at least on SGI and Sun. I've > not > seen anything in the change lists for 20.1 or 20.2 about such a bug. > I'm not sure why this worked in Numeric version 11, but it appears you are using the put function incorrectly. The documentation might need to be fixed to emphasize that the index list is interpreted as a one-dimensional index into a flattened representation of the array. The put function does the equivalent of the operation: a.flat[n] = v.flat[n] In other words, if a is a 2-d array, then the index list does NOT represent indices into the rows of the array. It represents indicies into the flattened array. In your example, the function is working correctly. It is placing a 12 into the 3 position in the array (i.e. row 0 column 3) and placing a 13 into the 15th position in the array (i.e. row 3 column 3). Like I said, I'm not sure why this worked in Numeric 11 --- it shouldn't have. -Travis From jochen at unc.edu Sun Oct 7 15:34:02 2001 From: jochen at unc.edu (Jochen =?iso-8859-1?q?K=FCpper?=) Date: Sun Oct 7 15:34:02 2001 Subject: [Numpy-discussion] demo Message-ID: <86ofnj5bdu.fsf@bock.chem.unc.edu> I know this is absolute highest priority:)), but it annoys me every time... Could someone please apply this patch to display the Mandelbrot demo correctly on wide terminals? Index: Demo/mandelbrot.py =================================================================== RCS file: /cvsroot/numpy/Numerical/Demo/mandelbrot.py,v retrieving revision 1.1.1.1 diff -u -r1.1.1.1 mandelbrot.py --- Demo/mandelbrot.py 2000/01/13 21:23:02 1.1.1.1 +++ Demo/mandelbrot.py 2001/10/07 22:27:19 @@ -22,6 +22,7 @@ if __name__ == "__main__": - print draw(-2.1, 0.7, -1.2, 1.2) - + data = draw(-2.1, 0.7, -1.2, 1.2) + for i in range(0, len(data), 81): + print data[i:i+81] Greetings, Jochen -- University of North Carolina phone: +1-919-962-4403 Department of Chemistry phone: +1-919-962-1579 Venable Hall CB#3290 (Kenan C148) fax: +1-919-843-6041 Chapel Hill, NC 27599, USA GnuPG key: 44BCCD8E From nde at comp.leeds.ac.uk Mon Oct 8 05:44:02 2001 From: nde at comp.leeds.ac.uk (Nick Efford) Date: Mon Oct 8 05:44:02 2001 Subject: [Numpy-discussion] problem with FFT module from Numeric 20.2.0 Message-ID: I posted on c.l.py about this and was pointed to the NumPy list... I'm compiling Numeric 20.2.0 from source using gcc 3.0, on a Linux system running Python 2.1 (also built with gcc 3.0). Numeric itself works, but problems occur with FFT: >>> import FFT Fatal Python error: can't initialize module fftpack Abort Kinda looks to me like fttpack.so wasn't built correctly. Any ideas on what is wrong / how to fix? Cheers, Nick -- Nick Efford | nde at comp.leeds.ac.uk School of Computing | tel: +44 113 233 6809 Univ. of Leeds, Leeds, UK | WWW: http://www.comp.leeds.ac.uk/nde/ From gvermeul at labs.polycnrs-gre.fr Mon Oct 8 10:00:03 2001 From: gvermeul at labs.polycnrs-gre.fr (Gerard Vermeulen) Date: Mon Oct 8 10:00:03 2001 Subject: [Numpy-discussion] Warning: be carefull with the use of Fortran Lapack/Blas libraries Message-ID: <01100818591200.16437@taco.polycnrs-gre.fr> When I build Numeric, I allways link the lapack_lite module to the Fortran Lapack and Blas libraries on my system. This time (Numeric-20.2.1), when calling the function eigenvalues(), I got a message saying that parameter 13 of dgeev had a wrong value. The reason is that the LinearAlgebra.py module assumes that you have a very recent version of dgeev, that you can use to calculate the workspace requirements for the real call. On http://www.netlib.org/lapack/release_notes.html you'll find a table "The following bug fixes have NOT yet been incorporated in a release/update of LAPACK." I quote "LAPACK/SRC/dgeev.f -- Corrected error with LQUERY and setting of WORK(1)". LinearAlgebra.py assumes that this bug has been corrected. So, if you want to use LinearAlgebra.py with Fortran Lapack/Blas, you'd better apply all those patches listed on http://www.netlib.org/lapack/release_notes.html Of course, it is not a bug in LinearAlgebra.py, but I don't know if it is wise to program LinearAlgebra.py in such a way that you run into this problem on probably more than 90 % of the Lapack installations. (getting the latest lapack.tgz is not sufficient) Gerard From altis at semi-retired.com Mon Oct 8 10:38:13 2001 From: altis at semi-retired.com (Kevin Altis) Date: Mon Oct 8 10:38:13 2001 Subject: [Numpy-discussion] drawing with NumPy and PythonCard Message-ID: Hi, I'm the coordinator for the PythonCard project, which is at: http://pythoncard.sourceforge.net/ PythonCard provides a GUI framework for building Python applications. The project is relatively new, but there are a large number of sample applications, including a number of drawing and visualization samples. PythonCard already supports loading and drawing of common formats like XBM, GIF, JPEG, PNG, etc. from disk files. Last month I started a BitmapCanvas for the framework to simplify interactive drawing. Rather than duplicate all of the functionality of the Python Imaging Library (PIL) or the various plotting packages already available, I want to make it easy to display results created with other drawing packages and numPy in PythonCard. To that end, the Bitmap and BitmapCanvas class already support PIL. You can convert a PIL RGB image to/from a PythonCard bitmap and you can draw a PIL RGB image in a BitmapCanvas at a given x, y location. If you're already converting from numPy to PIL, you should be able to show your results in PythonCard today. I will be making a related post about this on the image-sig. Anyway, what I would like to do is provide similar methods to make it easy to convert numPy arrays to images or a bitmap to numPy without having to go to PIL first. I don't currently use numPy since I don't do any scientific computing (and my math skills are pretty pathetic) and most of my programs are text or database-oriented, so I'm relying on others to spec out the features they need or want. Eventually, I would like to add some visual effects to PythonCard such as wipes and fades which would be a perfect use of arrays. I want to make sure that PythonCard provides a good GUI framework for the scientific community. I would appreciate any suggestions, pointers to related resources, etc. PythonCard currently sits on top of wxPython, so if you are already working with numPy and wxPython I would be very interested to hear about it. If you would like to get more actively involved, please join the PythonCard mailing list, which is probably a better place for longer-term discussions than the numPy list. http://lists.sourceforge.net/lists/listinfo/pythoncard-users Thanks, ka --- Kevin Altis altis at semi-retired.com From karshi.hasanov at utoronto.ca Mon Oct 8 13:32:25 2001 From: karshi.hasanov at utoronto.ca (Karshi Hasanov) Date: Mon Oct 8 13:32:25 2001 Subject: [Numpy-discussion] cephes_libs Message-ID: <20011008203127Z235124-15181+2@bureau8.utcc.utoronto.ca> Hi all, I've tried to install cephes libs from Travis Oliphan's website, but wasn't able to do it. Is there anybody knows how to install these libs. I don't know C much, so didn't understand the instruction for installation. Thanks From paul at pfdubois.com Mon Oct 8 15:58:20 2001 From: paul at pfdubois.com (Paul F. Dubois) Date: Mon Oct 8 15:58:20 2001 Subject: [Numpy-discussion] Warning: be carefull with the use of Fortran Lapack/Blas libraries In-Reply-To: <01100818591200.16437@taco.polycnrs-gre.fr> Message-ID: <000801c1504c$65f9f0a0$3d01a8c0@plstn1.sfba.home.com> No good deed goes unpunished. The contributor who put in these changes wanted to improve Numeric, and I accepted the changes because they seemed to work. I regret that I didn't investigate this in detail but the truth is I have to depend on strangers. I have no idea what to do about it. -----Original Message----- From: numpy-discussion-admin at lists.sourceforge.net [mailto:numpy-discussion-admin at lists.sourceforge.net] On Behalf Of Gerard Vermeulen Sent: Monday, October 08, 2001 9:59 AM To: numpy-discussion at lists.sourceforge.net Subject: [Numpy-discussion] Warning: be carefull with the use of Fortran Lapack/Blas libraries When I build Numeric, I allways link the lapack_lite module to the Fortran Lapack and Blas libraries on my system. This time (Numeric-20.2.1), when calling the function eigenvalues(), I got a message saying that parameter 13 of dgeev had a wrong value. The reason is that the LinearAlgebra.py module assumes that you have a very recent version of dgeev, that you can use to calculate the workspace requirements for the real call. On http://www.netlib.org/lapack/release_notes.html you'll find a table "The following bug fixes have NOT yet been incorporated in a release/update of LAPACK." I quote "LAPACK/SRC/dgeev.f -- Corrected error with LQUERY and setting of WORK(1)". LinearAlgebra.py assumes that this bug has been corrected. So, if you want to use LinearAlgebra.py with Fortran Lapack/Blas, you'd better apply all those patches listed on http://www.netlib.org/lapack/release_notes.html Of course, it is not a bug in LinearAlgebra.py, but I don't know if it is wise to program LinearAlgebra.py in such a way that you run into this problem on probably more than 90 % of the Lapack installations. (getting the latest lapack.tgz is not sufficient) Gerard _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From markespl at yahoo.com Tue Oct 9 06:29:13 2001 From: markespl at yahoo.com (Mark Esplin) Date: Tue Oct 9 06:29:13 2001 Subject: [Numpy-discussion] cephes_libs In-Reply-To: <20011008203127Z235124-15181+2@bureau8.utcc.utoronto.ca> References: <20011008203127Z235124-15181+2@bureau8.utcc.utoronto.ca> Message-ID: When I installed the cephes libs, I had to comment out three lines in the file cephes/protos.h (by putting "//" at the front of each line) to get it to compile. They are line numbers 67-69 //extern int signbit ( double x ); //extern int isnan ( double x ); //extern int isfinite ( double x ); It has seemed to run O.K, I assume those varibles are defined somewhere else. I am running on SuSE Linux 7.1, Python 2.0, Numeric-19.0.0 On Monday 08 October 2001 04:31 pm, Karshi Hasanov wrote: > Hi all, > I've tried to install cephes libs from Travis Oliphan's website, but > wasn't able to do it. Is there anybody knows how to install these libs. > I don't know C much, so didn't understand the instruction for > installation. Thanks _________________________________________________________ Do You Yahoo!? Get your free @yahoo.com address at http://mail.yahoo.com From gvermeul at labs.polycnrs-gre.fr Tue Oct 9 07:06:09 2001 From: gvermeul at labs.polycnrs-gre.fr (Gerard Vermeulen) Date: Tue Oct 9 07:06:09 2001 Subject: [Numpy-discussion] cephes_libs In-Reply-To: References: <20011008203127Z235124-15181+2@bureau8.utcc.utoronto.ca> Message-ID: <01100916030901.18812@taco.polycnrs-gre.fr> Try SciPy from www.scipy.org It contains a cephes module and you'll get a lot of other usefull stuff too Gerard On Tuesday 09 October 2001 15:30, Mark Esplin wrote: > When I installed the cephes libs, I had to comment out three lines in the > file cephes/protos.h (by putting "//" at the front of each line) to get it > to compile. They are line numbers 67-69 > > //extern int signbit ( double x ); > //extern int isnan ( double x ); > //extern int isfinite ( double x ); > > It has seemed to run O.K, I assume those varibles are defined somewhere > else. I am running on SuSE Linux 7.1, Python 2.0, Numeric-19.0.0 > > On Monday 08 October 2001 04:31 pm, Karshi Hasanov wrote: > > Hi all, > > I've tried to install cephes libs from Travis Oliphan's website, but > > wasn't able to do it. Is there anybody knows how to install these libs. > > I don't know C much, so didn't understand the instruction for > > installation. Thanks _______________________________________________ > Numpy-discussion mailing list > Numpy-discussion at lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/numpy-discussion From jsw at cdc.noaa.gov Tue Oct 9 14:15:12 2001 From: jsw at cdc.noaa.gov (Jeff Whitaker) Date: Tue Oct 9 14:15:12 2001 Subject: [Numpy-discussion] Warning: be carefull with the use of Fortran Lapack/Blas libraries In-Reply-To: <000401c15102$a7dc4660$3d01a8c0@plstn1.sfba.home.com> Message-ID: I have been able to reproduce this error either with the lapack_lite included with numeric, or with the lapack3.0 libs on my system. Gerard: are you sure the libs you are linking on your system are not lapack 2.0? If not, could you send me some code that triggers the error? -Jeff -- Jeffrey S. Whitaker Phone : (303)497-6313 Meteorologist FAX : (303)497-6449 NOAA/OAR/CDC R/CDC1 Email : jsw at cdc.noaa.gov 325 Broadway Web : www.cdc.noaa.gov/~jsw Boulder, CO, USA 80303-3328 Office : Skaggs Research Cntr 1D-124 From europax at home.com Wed Oct 10 06:58:13 2001 From: europax at home.com (Rob) Date: Wed Oct 10 06:58:13 2001 Subject: [Numpy-discussion] slightly OT: Animabob lives with Cygwin! Message-ID: <3BC4536F.C46506EA@home.com> I'm proud to have made my first GNU-tools port and my first Cygwin port Animabob-Cygwin-1.0. Its the visualization package that mates to all of my Numpy FDTD code. Get it on my website. Interestly on my laptop with Win2K it runs almost as fast as the native FreeBSD code. Yet on my machine at work its very slow- I think it has to do with color depth and screen resolutions. I'll find out today. I pretty much gave up on all the other visualization packages for Windows as they were either giant elephants or couldn't render 3d movies of array dumps. OpenDX would have been nice but it dumps core on every platform I've tried it on. I'm told this is fixed in CVS. Rob. -- The Numeric Python EM Project www.members.home.net/europax From gvermeul at labs.polycnrs-gre.fr Wed Oct 10 07:01:11 2001 From: gvermeul at labs.polycnrs-gre.fr (Gerard Vermeulen) Date: Wed Oct 10 07:01:11 2001 Subject: [Numpy-discussion] Warning: be carefull with the use of Fortran Lapack/Blas libraries In-Reply-To: References: Message-ID: <01101016003501.22480@taco.polycnrs-gre.fr> On Tuesday 09 October 2001 23:14, Jeff Whitaker wrote: > I have been able to reproduce this error either with the lapack_lite > included with numeric, or with the lapack3.0 libs on my system. > > Gerard: are you sure the libs you are linking on your system are not > lapack 2.0? If not, could you send me some code that triggers the error? > > > -Jeff After discussion with Jeff, I installed the latest lapack-3.0 (bugfix 2) and the problem went away. (I suppose I had lapack-3.0-bugfix 1 installed). Gerard From Aureli.Soria_Frisch at ipk.fhg.de Wed Oct 10 07:52:05 2001 From: Aureli.Soria_Frisch at ipk.fhg.de (Aureli Soria Frisch) Date: Wed Oct 10 07:52:05 2001 Subject: [Numpy-discussion] NumPy compatible GTK-based framework for color images Message-ID: Hi folks, A framework for manipulation of images with explicit support for NumPy arrays has been implemented: > >hello, >i would like to share a programming effort with you concerning the >display and manipulation of color images using GTK. >my idea is to have a programming tool which gives easy access to the >image memory space (using C matrix notation) while allowing the user >confortable interactive display features like pan/zoom/selection. >you may have a look at it under: > >http://vision.fhg.de/~daniel/pixelpark > >greetings, daniel. > >Daniel Kottow >daniel.kottow at ipk.fhg.de >Security and Testing Technology >http://vision.fhg.de/~daniel >Production Technology Centre Pascalstr. 8 - 9 >Fraunhofer-Institut IPK Berlin D - 10587 Berlin Best regards, Aureli ################################# Aureli Soria Frisch Fraunhofer IPK Dept. Pattern Recognition post: Pascalstr. 8-9, 10587 Berlin, Germany e-mail:aureli at ipk.fhg.de fon: +49 30 39006-150 fax: +49 30 3917517 web: http://vision.fhg.de/~aureli/web-aureli_en.html ################################# From altis at semi-retired.com Wed Oct 10 11:11:11 2001 From: altis at semi-retired.com (Kevin Altis) Date: Wed Oct 10 11:11:11 2001 Subject: [Numpy-discussion] drawing with NumPy and PythonCard - SciPy test In-Reply-To: Message-ID: A suggestion was made that I should check out SciPy. I downloaded and verified that the plt module works fine from within a PythonCard app using the built-in PythonCard shell. My test was the plt tutorial at http://www.scipy.org/site_content/tutorials/plot_tutorial I've emailed some of the SciPy developers, but there seems to be some mail gateway issues (the mail bounced) on their server right now, so I haven't heard back from anyone yet. Given that plt works with PythonCard today, people might be interested in being able to create dynamic buttons, fields, and methods from the interactive prompt using SciPy with PythonCard. The follwoing message shows dynamic method binding. http://aspn.activestate.com/ASPN/Mail/Message/770015 Adding other widgets such as a field is just as easy as adding a button, it is the same syntax as used in the resource files, so the following is valid: >>> comp['field2'] = {'type':'TextField', 'name':'field2', 'position':(0, 30), 'size':(150, -1), 'text':'this is field2'} For people that want to mostly do interactive sessions, but still be able to use buttons and fields to speed up data entry... this will probably be useful in the context of something like SciPy. Let me know if there are other questions or suggestions. ka > -----Original Message----- > From: numpy-discussion-admin at lists.sourceforge.net > [mailto:numpy-discussion-admin at lists.sourceforge.net]On Behalf Of Kevin > Altis > Sent: Monday, October 08, 2001 10:40 AM > To: numpy-discussion at lists.sourceforge.net > Subject: [Numpy-discussion] drawing with NumPy and PythonCard > > > Hi, > I'm the coordinator for the PythonCard project, which is at: > http://pythoncard.sourceforge.net/ > > PythonCard provides a GUI framework for building Python applications. The > project is relatively new, but there are a large number of sample > applications, including a number of drawing and visualization samples. > PythonCard already supports loading and drawing of common formats > like XBM, > GIF, JPEG, PNG, etc. from disk files. > > Last month I started a BitmapCanvas for the framework to simplify > interactive drawing. Rather than duplicate all of the functionality of the > Python Imaging Library (PIL) or the various plotting packages already > available, I want to make it easy to display results created with other > drawing packages and numPy in PythonCard. To that end, the Bitmap and > BitmapCanvas class already support PIL. You can convert a PIL RGB image > to/from a PythonCard bitmap and you can draw a PIL RGB image in a > BitmapCanvas at a given x, y location. If you're already converting from > numPy to PIL, you should be able to show your results in > PythonCard today. I > will be making a related post about this on the image-sig. > > Anyway, what I would like to do is provide similar methods to make it easy > to convert numPy arrays to images or a bitmap to numPy without > having to go > to PIL first. I don't currently use numPy since I don't do any scientific > computing (and my math skills are pretty pathetic) and most of my programs > are text or database-oriented, so I'm relying on others to spec out the > features they need or want. Eventually, I would like to add some visual > effects to PythonCard such as wipes and fades which would be a perfect use > of arrays. I want to make sure that PythonCard provides a good > GUI framework > for the scientific community. I would appreciate any suggestions, pointers > to related resources, etc. > > PythonCard currently sits on top of wxPython, so if you are > already working > with numPy and wxPython I would be very interested to hear about > it. If you > would like to get more actively involved, please join the > PythonCard mailing > list, which is probably a better place for longer-term > discussions than the > numPy list. > http://lists.sourceforge.net/lists/listinfo/pythoncard-users > > Thanks, > > ka > --- > Kevin Altis > altis at semi-retired.com > > > _______________________________________________ > Numpy-discussion mailing list > Numpy-discussion at lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/numpy-discussion > From nodwell at physics.ubc.ca Fri Oct 12 10:40:13 2001 From: nodwell at physics.ubc.ca (Eric Nodwell) Date: Fri Oct 12 10:40:13 2001 Subject: [Numpy-discussion] sign of modulo Message-ID: <20011012103933.A3659@holler.physics.ubc.ca> The Numpy version of modulo uses a different sign convention for negative arguments than does Python itself: >>> -5 % 3 1 >>> array((-5,-5)) % 3 array([-2, -2]) (Numpy version 20.2.0, Python version 1.5.2 and version 2.2a4.) Is there a reason for this (hard to imagine), or is this a bug? Eric From focke at SLAC.Stanford.EDU Fri Oct 12 11:09:11 2001 From: focke at SLAC.Stanford.EDU (Warren Focke) Date: Fri Oct 12 11:09:11 2001 Subject: [Numpy-discussion] sign of modulo In-Reply-To: <20011012103933.A3659@holler.physics.ubc.ca> Message-ID: On Fri, 12 Oct 2001, Eric Nodwell wrote: > The Numpy version of modulo uses a different sign convention for > negative arguments than does Python itself: ... > Is there a reason for this (hard to imagine), or is this a bug? Numpy delegates to the C platform's / and % operators, Python does it "right". Maybe this should go on the FAQ. Waren Focke From nodwell at physics.ubc.ca Fri Oct 12 11:22:05 2001 From: nodwell at physics.ubc.ca (Eric Nodwell) Date: Fri Oct 12 11:22:05 2001 Subject: [Numpy-discussion] sign of modulo In-Reply-To: References: <20011012103933.A3659@holler.physics.ubc.ca> Message-ID: <20011012112109.A3835@holler.physics.ubc.ca> > Numpy delegates to the C platform's / and % operators, Python does it > "right". > > Maybe this should go on the FAQ. Wouldn't it be preferable to change Numpy so it is also "right"? From focke at SLAC.Stanford.EDU Fri Oct 12 18:03:10 2001 From: focke at SLAC.Stanford.EDU (Warren Focke) Date: Fri Oct 12 18:03:10 2001 Subject: [Numpy-discussion] sign of modulo In-Reply-To: <20011012112109.A3835@holler.physics.ubc.ca> Message-ID: On Fri, 12 Oct 2001, Eric Nodwell wrote: > > Numpy delegates to the C platform's / and % operators, Python does it > > "right". > Wouldn't it be preferable to change Numpy so it is also "right"? Speed. In Python, there's so much overhead from symbol resolution, looking up __mod__s and __rmod__s and whatnot that the extra work is lost in the noise. Numpy would take a bigger hit when working with large arrays. One could always write python_compatible_div and python_compatible_mod functions, if one wanted. Could even submit a patch to add them, if truly motivated. Warren Focke From sag at hydrosphere.com Mon Oct 15 12:41:12 2001 From: sag at hydrosphere.com (Sue Giller) Date: Mon Oct 15 12:41:12 2001 Subject: [Numpy-discussion] Problems with pickling an array with type Object Message-ID: <20011015194003406.AAA229@mail.climatedata.com@spare> Greetings, I am trying to pickle an array of objects with a typecode of Object. Ultimately this array will be in a class I create based on UserArray, but the following illustrates a problem I am having. See code at the end of the message. I am working in Windows 2000, Service Pack 1, using PythonWin, win32all build 140. I am using the following versions of python stuff Python: 2.1.1 (#20, Jul 20 2001, 01:19:29) [MSC 32 bit (Intel)] Numeric: 20.2.0 cPickle: 1.71 I create a simple array with a type of Object, pickle it to a file using cPickle. I then run this script in PythonWin. I can unpickle this file from PythonWin, but not from DOS window. Alternatively, I can pickle the array from DOS, and unpickle it from DOS, but not from PythonWin. If I save both versions of the file and compare them, they are the same size, but not the same bytes. This does not happen with other types for the array. When unpickling, I get a GPF, either from PythonWin.exe or Python.exe. Any ideas? My ultimate goal is to be able to store simple class objects in the array and be able to pickle and unpickle them. ----- CODE ---- # simple test for pickling problem import Numeric import cPickle import sys basePath = "c:\\python21\\scripts\\pickle\\" print "++++++" # create a simple array of object, populated by integer data ds1 = Numeric.array([1,2,3,4], 'O') print ds1, ds1.typecode() print "Python: ", sys.version print "Numeric: " , Numeric.__version__ print "cPickle: ", cPickle.__version__ pickle = 1 # change to 0 to skip the pickling step for ii in (range(1,2)): if pickle: fileName = "%s%s%d%s" % (basePath, "simple", ii, ".pck") print "pickling ", fileName fp = open(fileName, 'wb') cPickle.dump(ds1, fp, 1) fp.close() for ii in range(1,2): fileName = "%s%s%d%s" % (basePath, "simple", ii, ".pck") print "unpickling ", fileName fp = open(fileName, 'rb') obj = cPickle.load(fp) fp.close() print obj, obj.typecode() From paul at pfdubois.com Tue Oct 16 08:55:14 2001 From: paul at pfdubois.com (Paul F. Dubois) Date: Tue Oct 16 08:55:14 2001 Subject: [Numpy-discussion] RE: activity on NumPy In-Reply-To: Message-ID: <000301c1565a$a75e3920$3d01a8c0@plstn1.sfba.home.com> No, I just have a major release coming up of cdat (cdat.sf.net) so I've been busy. -----Original Message----- From: nobody [mailto:nobody at sourceforge.net] On Behalf Of Alessandro MIRONE Sent: Tuesday, October 16, 2001 2:22 AM To: paul at pfdubois.com Subject: activity on NumPy Hi, I have noticed that that developer activity has almost stopped ( bugs and patches are no more processed) after 11/9. Did you get some restriction at your lab? Best regards Alessandro Mirone From sag at hydrosphere.com Thu Oct 25 09:35:09 2001 From: sag at hydrosphere.com (Sue Giller) Date: Thu Oct 25 09:35:09 2001 Subject: [Numpy-discussion] Problem with Numeric? Message-ID: <20011025163432046.AAA233@mail.climatedata.com@spare> I am using Numeric 20.2.0. I have found the following inconsistency. I am not a python guru and have no idea how I would fix it myself. The following code creates a series of arrays of data, and then applies maximum.reduce to the array to get the largest value. It then prints the type of returned max value. Most of the returns are typed as an array with a shape of (). If the array is created as a type of 'l' (long) or 'd' (double), the return type is not array. This seems wrong. Can someone tell me if this is expected, and if not, where would I look to change/fix it. --- CODE --- from Numeric import * print "++++" for tc in ('b', 'i', 'l', 's', 'f', 'd'): smallArray = array([1,2,3,4], tc) amax = maximum.reduce(smallArray) print smallArray.typecode(), smallArray, "Max:", amax, type(amax) if type(amax) == type(smallArray): print amax.shape --- OUTPUT --- >>>> ++++ b [1 2 3 4] Max: 4 () i [1 2 3 4] Max: 4 () l [1 2 3 4] Max: 4 s [1 2 3 4] Max: 4 () f [ 1. 2. 3. 4.] Max: 4.0 () d [ 1. 2. 3. 4.] Max: 4.0 From jbaddor at physics.mcgill.ca Fri Oct 26 07:47:03 2001 From: jbaddor at physics.mcgill.ca (Jean-Bernard Addor) Date: Fri Oct 26 07:47:03 2001 Subject: [Numpy-discussion] (real)**(2x2 general real matrix) Message-ID: Hey numpy people! Do you know the better way to compute with numpy: r**M where r is a positive real and M a 2x2 (rank 2) matrix, symmetric or not, with real or not eigenvalues. Help or comment are appreciated, Jean-Bernard From nwagner at mecha.uni-stuttgart.de Tue Oct 30 00:22:02 2001 From: nwagner at mecha.uni-stuttgart.de (Nils Wagner) Date: Tue Oct 30 00:22:02 2001 Subject: [Numpy-discussion] Rank deficient matrices Message-ID: <3BDE63B2.EA84FA27@mecha.uni-stuttgart.de> Hi, Let us assume that r = rank(C) < N (1) where C is a symmetric NxN matrix. This implies that the number of non-zero eigenvalues of C is r. Because C is a symmetric matrix there exists an orthogonal matrix U whose columns are the eigenvectors of C such that U^\top C U = [ d , O O , O]. (2) In the above equation d \in rxr is a diagonal matrix consisting of only the non-zero eigenvalues of C. For convenience, partition U as U = [U_1 | U_2] (3) where the columns of U_1 (Nxr) are the eigenvectors corresponding to the non-zero block d and the columns of U_2 are the eigenvectors corresponding to the rest (N-r) number of zero eigenvalues. Defining a rectangular transformation matrix R = U_1 (4) it is easy to show that R^\top C R = d. (5) Therefore, the matrix R in equation (4) transforms the originally rank deficient matrix C to a full-rank (diagonal) matrix of rank r. I am looking for an efficient Numpy implementation of this transformation. Thanks in advance Nils Wagner From paul at pfdubois.com Tue Oct 30 07:21:07 2001 From: paul at pfdubois.com (Paul F. Dubois) Date: Tue Oct 30 07:21:07 2001 Subject: [Numpy-discussion] Rank deficient matrices In-Reply-To: <3BDE63B2.EA84FA27@mecha.uni-stuttgart.de> Message-ID: <001a01c16156$3827d1e0$3d01a8c0@plstn1.sfba.home.com> Use LinearAlgebra.singular_value_decomposition -----Original Message----- From: numpy-discussion-admin at lists.sourceforge.net [mailto:numpy-discussion-admin at lists.sourceforge.net] On Behalf Of Nils Wagner Sent: Tuesday, October 30, 2001 12:24 AM To: numpy-discussion at lists.sourceforge.net Subject: [Numpy-discussion] Rank deficient matrices Hi, Let us assume that r = rank(C) < N (1) where C is a symmetric NxN matrix. This implies that the number of non-zero eigenvalues of C is r. Because C is a symmetric matrix there exists an orthogonal matrix U whose columns are the eigenvectors of C such that U^\top C U = [ d , O O , O]. (2) In the above equation d \in rxr is a diagonal matrix consisting of only the non-zero eigenvalues of C. For convenience, partition U as U = [U_1 | U_2] (3) where the columns of U_1 (Nxr) are the eigenvectors corresponding to the non-zero block d and the columns of U_2 are the eigenvectors corresponding to the rest (N-r) number of zero eigenvalues. Defining a rectangular transformation matrix R = U_1 (4) it is easy to show that R^\top C R = d. (5) Therefore, the matrix R in equation (4) transforms the originally rank deficient matrix C to a full-rank (diagonal) matrix of rank r. I am looking for an efficient Numpy implementation of this transformation. Thanks in advance Nils Wagner _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From edcjones at erols.com Tue Oct 30 12:00:02 2001 From: edcjones at erols.com (Edward C. Jones) Date: Tue Oct 30 12:00:02 2001 Subject: [Numpy-discussion] Numeric2 Message-ID: <3BDF066E.3050200@erols.com> What is the current status of Numeric2? From perry at stsci.edu Tue Oct 30 12:40:03 2001 From: perry at stsci.edu (Perry Greenfield) Date: Tue Oct 30 12:40:03 2001 Subject: [Numpy-discussion] RE: Numeric2 In-Reply-To: Message-ID: > Date: Tue, 30 Oct 2001 14:58:38 -0500 > From: "Edward C. Jones" > To: numpy-discussion at lists.sourceforge.net > Subject: [Numpy-discussion] Numeric2 > > What is the current status of Numeric2? > We are in the process of putting it up on sourceforge now. While you can actually download it from there now, I would strongly recommend waiting until we put some more current and up-to-date documentation about it there as well. We will announce it on this mailing list when it is ready (within a week?). But it isn't called Numeric2 any longer. We've called it numarray and it will be found under "python arrays" on Sourceforge. Perry Greenfield From edcjones at erols.com Wed Oct 31 07:51:03 2001 From: edcjones at erols.com (Edward C. Jones) Date: Wed Oct 31 07:51:03 2001 Subject: [Numpy-discussion] Undefined symbol in lapack_lite.so Message-ID: <3BE01DC2.8090209@erols.com> The following Python program: import Numeric, LinearAlgebra a = Numeric.identity(5) v, s, wt = LinearAlgebra.singular_value_decomposition(a) gave the following error: Traceback (most recent call last): File "./silly.py", line 3, in ? import Numeric, LinearAlgebra File "/usr/lib/python2.2/site-packages/Numeric/LinearAlgebra.py", line 8, in ? import lapack_lite ImportError: /usr/lib/python2.2/site-packages/Numeric/lapack_lite.so: undefined symbol: dgesdd_ When I installed Numeric-20.2.0, I changed Setup.py to: # delete all but the first one in this list if using your own LAPACK/BLAS sourcelist = ['Src/lapack_litemodule.c',] # set these to use your own BLAS library_dirs_list = ['usr/lib'] libraries_list = ['/usr/lib/libblas.so', '/usr/lib/liblapack.so'] What is the problem? Thanks, Ed Jones From gvermeul at labs.polycnrs-gre.fr Wed Oct 31 08:41:04 2001 From: gvermeul at labs.polycnrs-gre.fr (Gerard Vermeulen) Date: Wed Oct 31 08:41:04 2001 Subject: [Numpy-discussion] Undefined symbol in lapack_lite.so In-Reply-To: <3BE01DC2.8090209@erols.com> References: <3BE01DC2.8090209@erols.com> Message-ID: <01103117403200.00960@taco.polycnrs-gre.fr> Do you have lapack-3.0? Numeric-20.2.0 (better get 20.2.1) is interfaced to new faster lapack routines. You could check it by [packer at taco lib]$ strings liblapack.so.3 | grep dgesdd dgesdd_ [packer at taco lib]$ best regards -- Gerard On Wednesday 31 October 2001 16:50, Edward C. Jones wrote: > The following Python program: > > import Numeric, LinearAlgebra > > a = Numeric.identity(5) > v, s, wt = LinearAlgebra.singular_value_decomposition(a) > > gave the following error: > > Traceback (most recent call last): > File "./silly.py", line 3, in ? > import Numeric, LinearAlgebra > File "/usr/lib/python2.2/site-packages/Numeric/LinearAlgebra.py", line > 8, in ? import lapack_lite > ImportError: /usr/lib/python2.2/site-packages/Numeric/lapack_lite.so: > undefined symbol: dgesdd_ > > When I installed Numeric-20.2.0, I changed Setup.py to: > > # delete all but the first one in this list if using your own LAPACK/BLAS > sourcelist = ['Src/lapack_litemodule.c',] > # set these to use your own BLAS > library_dirs_list = ['usr/lib'] > libraries_list = ['/usr/lib/libblas.so', '/usr/lib/liblapack.so'] > > What is the problem? > > Thanks, > Ed Jones > > > > _______________________________________________ > Numpy-discussion mailing list > Numpy-discussion at lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/numpy-discussion From gb at cs.unc.edu Wed Oct 3 12:08:08 2001 From: gb at cs.unc.edu (Gary Bishop) Date: Wed Oct 3 12:08:08 2001 Subject: [Numpy-discussion] Changes to improve performance on small matricies Message-ID: <200110031907.PAA02478@wren.cs.unc.edu> We are porting some code from Matlab that does many thousands of operations on small matrices. We have found that some small changes to LinearAlgebra.py, Numeric.py, and multiarraymodule.c make our code run approximately twice as fast as with the unmodified Numeric-20.2 code. Our changes specifically replace LinearAlgebra._castCopyAndTranspose with a version we call _fastCopyAndTranspose that does most of the work in C instead of calling Numeric.transpose, which calls arange, which is quite expensive. We also optimized Numeric.dot to call a new function multiarray.matrixproduct which does the axis swap on the fly instead of calling swapaxes (which calls arange) and then calling innerproduct (which as the very first step copies the transposed matrix to make it contiguous). Formerly our code spent much of its time in arange (which we never explicitly call), dot, and _castCopyAndTranspose. The changes described above eliminate this overhead. I'm writing to ask if these changes might make a worthy patch to NumPy? We have tested them with on Windows2k (both Native and under Cygwin) and on Linux. Soon, we'll have a test on Mac OS X.1. If anyone is interested, I will figure out how to generate a patch file to submit. Thanks gb From vanandel at atd.ucar.edu Wed Oct 3 12:55:03 2001 From: vanandel at atd.ucar.edu (Joe Van Andel) Date: Wed Oct 3 12:55:03 2001 Subject: [Numpy-discussion] pgcc (Portland Group C compiler) performance improvements for Numeric applications? Message-ID: <3BBB6CFE.94A6730E@atd.ucar.edu> Has anyone benchmarked their Numeric applications using 'pgcc'? I'm interested in improving the performance of my Numeric Python code, and I'm wondering what speedup I might achieve, just by re-compiling Python, Numeric, and my extensions with pgcc. (We currently use gcc 2.95.3) Also, has anyone used the multiprocessor (OpenMP) or CDK (Cluster Development Kit) tools from Portland Group? -- Joe VanAndel National Center for Atmospheric Research http://www.atd.ucar.edu/~vanandel/ Internet: vanandel at ucar.edu From chrishbarker at home.net Thu Oct 4 16:45:02 2001 From: chrishbarker at home.net (Chris Barker) Date: Thu Oct 4 16:45:02 2001 Subject: [Numpy-discussion] Changes to improve performance on small matricies References: <200110031907.PAA02478@wren.cs.unc.edu> Message-ID: <3BBCF875.B274B2BB@home.net> Gary Bishop wrote: > I'm writing to ask if these changes might make a worthy patch to NumPy? > We have tested them with on Windows2k (both Native and under Cygwin) > and on Linux. Soon, we'll have a test on Mac OS X.1. I'd say yes, but we do have to be satisfied with the robustness of the code, and I have no idea how to go about doing that. This brings up a good point. In Numeric.Py, the is the following comment: #The following functions are considered builtin, they all might be #in C some day I'd love to see that happen. As evidenced by this poster's example, Numeric could be made a lot faster by optimizing a number of build-in functions. What would it take to get a "Numeric Optimization project" going?? A few ideas: unit tests: If we had unit tests designed for all the built-in functions, we would have a good way to know if a newly coded C version is working right. How-To_Docs: I have tried to write some Numeric extensions myself, and it is very difficult. Writing one that does something unusual for my particular application is not so hard with the current docs. In that case it usually only has to work for one type, I know what shape the array should be etc. Writing a universal extension to Numeric that would work on any Numeric Array is a whole lot harder!! I am currently working on two of these, and the hard parts for me are: Doing something to all the data in a perhaps discontiguous array of arbitrary shape. I have soemthing working now, but it's pretty slow. Having a function work for all types of Arrays: dealing this this kin dof dynamicism on C is very difficult for me. The best place to look is clearly the source itself, but I have tried, and it's pretty darn dense. A How-To would be pretty darn helpful. I don't know who's going to write these things, but I think you could enlarge the pool of NumPy developers quite a bit. If anyone is interested, I have written "fastclip()" which does what clip() does, but it does it in place, and only works on Floats (see above) It is very fast for contiguous arrays, and disappointingly slow on discontiguous arrays (also see above). Utill I get it working on all types, there isn't much point in submitting it for inclusion in NumPy. The other two I have written I use for dealing with binary data coming in from files: byteswap() does a bytswap in place, whioch is faster an more memory efficient that the existing bytswapped() method. changetype() changes the type of the array, using the same data: changetype(m,typecode) = fromstring(m.tostring(),typecode) It is a lot faster and memory efficient (the memory was the issue for me, I'm dealing with up to 10MB arrays), but kind of an obscure need to, so this may not be a candidate for inclusion in NumPy either. I still have some questions about my code for that one: I'll post a separate message about that. -Chris -- Christopher Barker, Ph.D. ChrisHBarker at home.net --- --- --- http://members.home.net/barkerlohmann ---@@ -----@@ -----@@ ------@@@ ------@@@ ------@@@ Oil Spill Modeling ------ @ ------ @ ------ @ Water Resources Engineering ------- --------- -------- Coastal and Fluvial Hydrodynamics -------------------------------------- ------------------------------------------------------------------------ From chrishbarker at home.net Thu Oct 4 16:58:02 2001 From: chrishbarker at home.net (Chris Barker) Date: Thu Oct 4 16:58:02 2001 Subject: [Numpy-discussion] Coding questions for a type change function References: <200110031907.PAA02478@wren.cs.unc.edu> Message-ID: <3BBCFB85.886E8405@home.net> HI all, I've written a function (in C) that changes the type of an array, in place, using the same data block. The reason I need this is because I am reading in a large block of data from a file that is essentially a set of records, each one a C struct that has a couple of longs and a char in it. I read it in as a big array, slice iot to separate the fields, and then need to change the type. I had been using: A = fromstring(A.tostring(),Int32) This works fine, but needs a lot of memory, and it kind of slow (althought enough faster than the file read that it matters little) The array can be on order of 10MB however, so the memory issue is a problem. Anyway, I wrote a function that changes the type of data in an array, without making any copies of the data. It seems to work but I have a few questions (code to follow): 1) Is there an easier way to do this? I just notices that PyArray_As1D takes a type as an input... would it do what I want?? 2) Is there a way , at the C levbel to reshape an array. PyArray_Reshape takes a PyObject, which is kind of awkward, I'd love to pass in a few integers, or maybe a nd, and a pointer to a dimensions array. 3) What I did do is use PyArray_DescrFromType() to create a new desciption, and then assign the description pointer of the array to the new one. This seems to work, but I expect that the old description memory never gets freed. How do I free it (can you tell I'm a C newbie?). Note that I have called this function is a loop thousands of times, and not seen memory use go up, but it's a pretty small structure. 4) If I call PyArray_DescrFromType() with an invalid typcode, I get a crash. How can I check if the typecode is valid?? The code follows, I'd love any feedback anyone can offer. -Chris static PyObject * NumericExtras_changetype(PyObject *self, PyObject *args) { PyArrayObject *Array; //int *new_strides; //int *new_dimensions; PyArray_Descr *NewDescription; int TotalBytes; int NewElementSize; char typecode; if (!PyArg_ParseTuple(args, "O!c", &PyArray_Type, &Array, &typecode)) { return NULL; } if (!PyArray_ISCONTIGUOUS(Array)){ PyErr_SetString (PyExc_ValueError, "m must be contiguous"); return NULL; } TotalBytes = PyArray_NBYTES(Array); // This will crash if an invalid typecode is passed in. NewDescription = PyArray_DescrFromType(typecode); NewElementSize = NewDescription->elsize; if (TotalBytes % NewElementSize > 0){ PyErr_SetString (PyExc_ValueError, "The input array is the wrong size for the requested type"); return NULL; } Array->descr = NewDescription; // does the old descr need to be freed?? how?? Array->nd = 1; Array->strides[0] = NewElementSize; Array->dimensions[0] = TotalBytes / NewElementSize; return Py_BuildValue(""); } -- Christopher Barker, Ph.D. ChrisHBarker at home.net --- --- --- http://members.home.net/barkerlohmann ---@@ -----@@ -----@@ ------@@@ ------@@@ ------@@@ Oil Spill Modeling ------ @ ------ @ ------ @ Water Resources Engineering ------- --------- -------- Coastal and Fluvial Hydrodynamics -------------------------------------- ------------------------------------------------------------------------ From lindy at scripps.edu Fri Oct 5 10:11:05 2001 From: lindy at scripps.edu (Lindy Lindstrom) Date: Fri Oct 5 10:11:05 2001 Subject: [Numpy-discussion] Numeric.put bug References: Message-ID: <3BBDE98C.55CED05A@scripps.edu> Has anyone yet noted this? The following code works in python1.5.2 (Numeric.__version__ == '11'). However, in python2.0 (with Numeric.__version__ == '17.1.1') and python2.1 (with Numeric.__version__ == '20.0.0'), it doesn't work correctly. coords[3][3] is mis-assigned. It happens at least on SGI and Sun. I've not seen anything in the change lists for 20.1 or 20.2 about such a bug. # # bug.py # import Numeric coords = Numeric.resize(Numeric.array(range(17*4)), (17,4)) print "before coords[3]: ", coords[3] list = [3, 15] takecoords = Numeric.take(coords, list) print "takecoords:" print takecoords print "coords.iscontiguous: ", coords.iscontiguous() Numeric.put(coords, list, takecoords) print "after coords[3]: ", coords[3] ********************************** William M. Lindstrom, Jr., Ph.D. Molecular Graphics Laboratory Department of Molecular Biology, MB-5 The Scripps Research Institute 10550 North Torrey Pines Road La Jolla, CA 92037-1000 USA Tel: (858) 784-2055 Fax: (858) 784-2860 E-mail: lindy at scripps.edu From oliphant.travis at ieee.org Fri Oct 5 13:18:04 2001 From: oliphant.travis at ieee.org (Travis Oliphant) Date: Fri Oct 5 13:18:04 2001 Subject: [Numpy-discussion] Numeric.put bug In-Reply-To: <3BBDE98C.55CED05A@scripps.edu> References: <3BBDE98C.55CED05A@scripps.edu> Message-ID: On Friday 05 October 2001 11:10 am, you wrote: > Has anyone yet noted this? > > The following code works in python1.5.2 (Numeric.__version__ == '11'). > However, in python2.0 (with Numeric.__version__ == '17.1.1') and > python2.1 (with Numeric.__version__ == '20.0.0'), it doesn't work > correctly. > coords[3][3] is mis-assigned. It happens at least on SGI and Sun. I've > not > seen anything in the change lists for 20.1 or 20.2 about such a bug. > I'm not sure why this worked in Numeric version 11, but it appears you are using the put function incorrectly. The documentation might need to be fixed to emphasize that the index list is interpreted as a one-dimensional index into a flattened representation of the array. The put function does the equivalent of the operation: a.flat[n] = v.flat[n] In other words, if a is a 2-d array, then the index list does NOT represent indices into the rows of the array. It represents indicies into the flattened array. In your example, the function is working correctly. It is placing a 12 into the 3 position in the array (i.e. row 0 column 3) and placing a 13 into the 15th position in the array (i.e. row 3 column 3). Like I said, I'm not sure why this worked in Numeric 11 --- it shouldn't have. -Travis From jochen at unc.edu Sun Oct 7 15:34:02 2001 From: jochen at unc.edu (Jochen =?iso-8859-1?q?K=FCpper?=) Date: Sun Oct 7 15:34:02 2001 Subject: [Numpy-discussion] demo Message-ID: <86ofnj5bdu.fsf@bock.chem.unc.edu> I know this is absolute highest priority:)), but it annoys me every time... Could someone please apply this patch to display the Mandelbrot demo correctly on wide terminals? Index: Demo/mandelbrot.py =================================================================== RCS file: /cvsroot/numpy/Numerical/Demo/mandelbrot.py,v retrieving revision 1.1.1.1 diff -u -r1.1.1.1 mandelbrot.py --- Demo/mandelbrot.py 2000/01/13 21:23:02 1.1.1.1 +++ Demo/mandelbrot.py 2001/10/07 22:27:19 @@ -22,6 +22,7 @@ if __name__ == "__main__": - print draw(-2.1, 0.7, -1.2, 1.2) - + data = draw(-2.1, 0.7, -1.2, 1.2) + for i in range(0, len(data), 81): + print data[i:i+81] Greetings, Jochen -- University of North Carolina phone: +1-919-962-4403 Department of Chemistry phone: +1-919-962-1579 Venable Hall CB#3290 (Kenan C148) fax: +1-919-843-6041 Chapel Hill, NC 27599, USA GnuPG key: 44BCCD8E From nde at comp.leeds.ac.uk Mon Oct 8 05:44:02 2001 From: nde at comp.leeds.ac.uk (Nick Efford) Date: Mon Oct 8 05:44:02 2001 Subject: [Numpy-discussion] problem with FFT module from Numeric 20.2.0 Message-ID: I posted on c.l.py about this and was pointed to the NumPy list... I'm compiling Numeric 20.2.0 from source using gcc 3.0, on a Linux system running Python 2.1 (also built with gcc 3.0). Numeric itself works, but problems occur with FFT: >>> import FFT Fatal Python error: can't initialize module fftpack Abort Kinda looks to me like fttpack.so wasn't built correctly. Any ideas on what is wrong / how to fix? Cheers, Nick -- Nick Efford | nde at comp.leeds.ac.uk School of Computing | tel: +44 113 233 6809 Univ. of Leeds, Leeds, UK | WWW: http://www.comp.leeds.ac.uk/nde/ From gvermeul at labs.polycnrs-gre.fr Mon Oct 8 10:00:03 2001 From: gvermeul at labs.polycnrs-gre.fr (Gerard Vermeulen) Date: Mon Oct 8 10:00:03 2001 Subject: [Numpy-discussion] Warning: be carefull with the use of Fortran Lapack/Blas libraries Message-ID: <01100818591200.16437@taco.polycnrs-gre.fr> When I build Numeric, I allways link the lapack_lite module to the Fortran Lapack and Blas libraries on my system. This time (Numeric-20.2.1), when calling the function eigenvalues(), I got a message saying that parameter 13 of dgeev had a wrong value. The reason is that the LinearAlgebra.py module assumes that you have a very recent version of dgeev, that you can use to calculate the workspace requirements for the real call. On http://www.netlib.org/lapack/release_notes.html you'll find a table "The following bug fixes have NOT yet been incorporated in a release/update of LAPACK." I quote "LAPACK/SRC/dgeev.f -- Corrected error with LQUERY and setting of WORK(1)". LinearAlgebra.py assumes that this bug has been corrected. So, if you want to use LinearAlgebra.py with Fortran Lapack/Blas, you'd better apply all those patches listed on http://www.netlib.org/lapack/release_notes.html Of course, it is not a bug in LinearAlgebra.py, but I don't know if it is wise to program LinearAlgebra.py in such a way that you run into this problem on probably more than 90 % of the Lapack installations. (getting the latest lapack.tgz is not sufficient) Gerard From altis at semi-retired.com Mon Oct 8 10:38:13 2001 From: altis at semi-retired.com (Kevin Altis) Date: Mon Oct 8 10:38:13 2001 Subject: [Numpy-discussion] drawing with NumPy and PythonCard Message-ID: Hi, I'm the coordinator for the PythonCard project, which is at: http://pythoncard.sourceforge.net/ PythonCard provides a GUI framework for building Python applications. The project is relatively new, but there are a large number of sample applications, including a number of drawing and visualization samples. PythonCard already supports loading and drawing of common formats like XBM, GIF, JPEG, PNG, etc. from disk files. Last month I started a BitmapCanvas for the framework to simplify interactive drawing. Rather than duplicate all of the functionality of the Python Imaging Library (PIL) or the various plotting packages already available, I want to make it easy to display results created with other drawing packages and numPy in PythonCard. To that end, the Bitmap and BitmapCanvas class already support PIL. You can convert a PIL RGB image to/from a PythonCard bitmap and you can draw a PIL RGB image in a BitmapCanvas at a given x, y location. If you're already converting from numPy to PIL, you should be able to show your results in PythonCard today. I will be making a related post about this on the image-sig. Anyway, what I would like to do is provide similar methods to make it easy to convert numPy arrays to images or a bitmap to numPy without having to go to PIL first. I don't currently use numPy since I don't do any scientific computing (and my math skills are pretty pathetic) and most of my programs are text or database-oriented, so I'm relying on others to spec out the features they need or want. Eventually, I would like to add some visual effects to PythonCard such as wipes and fades which would be a perfect use of arrays. I want to make sure that PythonCard provides a good GUI framework for the scientific community. I would appreciate any suggestions, pointers to related resources, etc. PythonCard currently sits on top of wxPython, so if you are already working with numPy and wxPython I would be very interested to hear about it. If you would like to get more actively involved, please join the PythonCard mailing list, which is probably a better place for longer-term discussions than the numPy list. http://lists.sourceforge.net/lists/listinfo/pythoncard-users Thanks, ka --- Kevin Altis altis at semi-retired.com From karshi.hasanov at utoronto.ca Mon Oct 8 13:32:25 2001 From: karshi.hasanov at utoronto.ca (Karshi Hasanov) Date: Mon Oct 8 13:32:25 2001 Subject: [Numpy-discussion] cephes_libs Message-ID: <20011008203127Z235124-15181+2@bureau8.utcc.utoronto.ca> Hi all, I've tried to install cephes libs from Travis Oliphan's website, but wasn't able to do it. Is there anybody knows how to install these libs. I don't know C much, so didn't understand the instruction for installation. Thanks From paul at pfdubois.com Mon Oct 8 15:58:20 2001 From: paul at pfdubois.com (Paul F. Dubois) Date: Mon Oct 8 15:58:20 2001 Subject: [Numpy-discussion] Warning: be carefull with the use of Fortran Lapack/Blas libraries In-Reply-To: <01100818591200.16437@taco.polycnrs-gre.fr> Message-ID: <000801c1504c$65f9f0a0$3d01a8c0@plstn1.sfba.home.com> No good deed goes unpunished. The contributor who put in these changes wanted to improve Numeric, and I accepted the changes because they seemed to work. I regret that I didn't investigate this in detail but the truth is I have to depend on strangers. I have no idea what to do about it. -----Original Message----- From: numpy-discussion-admin at lists.sourceforge.net [mailto:numpy-discussion-admin at lists.sourceforge.net] On Behalf Of Gerard Vermeulen Sent: Monday, October 08, 2001 9:59 AM To: numpy-discussion at lists.sourceforge.net Subject: [Numpy-discussion] Warning: be carefull with the use of Fortran Lapack/Blas libraries When I build Numeric, I allways link the lapack_lite module to the Fortran Lapack and Blas libraries on my system. This time (Numeric-20.2.1), when calling the function eigenvalues(), I got a message saying that parameter 13 of dgeev had a wrong value. The reason is that the LinearAlgebra.py module assumes that you have a very recent version of dgeev, that you can use to calculate the workspace requirements for the real call. On http://www.netlib.org/lapack/release_notes.html you'll find a table "The following bug fixes have NOT yet been incorporated in a release/update of LAPACK." I quote "LAPACK/SRC/dgeev.f -- Corrected error with LQUERY and setting of WORK(1)". LinearAlgebra.py assumes that this bug has been corrected. So, if you want to use LinearAlgebra.py with Fortran Lapack/Blas, you'd better apply all those patches listed on http://www.netlib.org/lapack/release_notes.html Of course, it is not a bug in LinearAlgebra.py, but I don't know if it is wise to program LinearAlgebra.py in such a way that you run into this problem on probably more than 90 % of the Lapack installations. (getting the latest lapack.tgz is not sufficient) Gerard _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From markespl at yahoo.com Tue Oct 9 06:29:13 2001 From: markespl at yahoo.com (Mark Esplin) Date: Tue Oct 9 06:29:13 2001 Subject: [Numpy-discussion] cephes_libs In-Reply-To: <20011008203127Z235124-15181+2@bureau8.utcc.utoronto.ca> References: <20011008203127Z235124-15181+2@bureau8.utcc.utoronto.ca> Message-ID: When I installed the cephes libs, I had to comment out three lines in the file cephes/protos.h (by putting "//" at the front of each line) to get it to compile. They are line numbers 67-69 //extern int signbit ( double x ); //extern int isnan ( double x ); //extern int isfinite ( double x ); It has seemed to run O.K, I assume those varibles are defined somewhere else. I am running on SuSE Linux 7.1, Python 2.0, Numeric-19.0.0 On Monday 08 October 2001 04:31 pm, Karshi Hasanov wrote: > Hi all, > I've tried to install cephes libs from Travis Oliphan's website, but > wasn't able to do it. Is there anybody knows how to install these libs. > I don't know C much, so didn't understand the instruction for > installation. Thanks _________________________________________________________ Do You Yahoo!? Get your free @yahoo.com address at http://mail.yahoo.com From gvermeul at labs.polycnrs-gre.fr Tue Oct 9 07:06:09 2001 From: gvermeul at labs.polycnrs-gre.fr (Gerard Vermeulen) Date: Tue Oct 9 07:06:09 2001 Subject: [Numpy-discussion] cephes_libs In-Reply-To: References: <20011008203127Z235124-15181+2@bureau8.utcc.utoronto.ca> Message-ID: <01100916030901.18812@taco.polycnrs-gre.fr> Try SciPy from www.scipy.org It contains a cephes module and you'll get a lot of other usefull stuff too Gerard On Tuesday 09 October 2001 15:30, Mark Esplin wrote: > When I installed the cephes libs, I had to comment out three lines in the > file cephes/protos.h (by putting "//" at the front of each line) to get it > to compile. They are line numbers 67-69 > > //extern int signbit ( double x ); > //extern int isnan ( double x ); > //extern int isfinite ( double x ); > > It has seemed to run O.K, I assume those varibles are defined somewhere > else. I am running on SuSE Linux 7.1, Python 2.0, Numeric-19.0.0 > > On Monday 08 October 2001 04:31 pm, Karshi Hasanov wrote: > > Hi all, > > I've tried to install cephes libs from Travis Oliphan's website, but > > wasn't able to do it. Is there anybody knows how to install these libs. > > I don't know C much, so didn't understand the instruction for > > installation. Thanks _______________________________________________ > Numpy-discussion mailing list > Numpy-discussion at lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/numpy-discussion From jsw at cdc.noaa.gov Tue Oct 9 14:15:12 2001 From: jsw at cdc.noaa.gov (Jeff Whitaker) Date: Tue Oct 9 14:15:12 2001 Subject: [Numpy-discussion] Warning: be carefull with the use of Fortran Lapack/Blas libraries In-Reply-To: <000401c15102$a7dc4660$3d01a8c0@plstn1.sfba.home.com> Message-ID: I have been able to reproduce this error either with the lapack_lite included with numeric, or with the lapack3.0 libs on my system. Gerard: are you sure the libs you are linking on your system are not lapack 2.0? If not, could you send me some code that triggers the error? -Jeff -- Jeffrey S. Whitaker Phone : (303)497-6313 Meteorologist FAX : (303)497-6449 NOAA/OAR/CDC R/CDC1 Email : jsw at cdc.noaa.gov 325 Broadway Web : www.cdc.noaa.gov/~jsw Boulder, CO, USA 80303-3328 Office : Skaggs Research Cntr 1D-124 From europax at home.com Wed Oct 10 06:58:13 2001 From: europax at home.com (Rob) Date: Wed Oct 10 06:58:13 2001 Subject: [Numpy-discussion] slightly OT: Animabob lives with Cygwin! Message-ID: <3BC4536F.C46506EA@home.com> I'm proud to have made my first GNU-tools port and my first Cygwin port Animabob-Cygwin-1.0. Its the visualization package that mates to all of my Numpy FDTD code. Get it on my website. Interestly on my laptop with Win2K it runs almost as fast as the native FreeBSD code. Yet on my machine at work its very slow- I think it has to do with color depth and screen resolutions. I'll find out today. I pretty much gave up on all the other visualization packages for Windows as they were either giant elephants or couldn't render 3d movies of array dumps. OpenDX would have been nice but it dumps core on every platform I've tried it on. I'm told this is fixed in CVS. Rob. -- The Numeric Python EM Project www.members.home.net/europax From gvermeul at labs.polycnrs-gre.fr Wed Oct 10 07:01:11 2001 From: gvermeul at labs.polycnrs-gre.fr (Gerard Vermeulen) Date: Wed Oct 10 07:01:11 2001 Subject: [Numpy-discussion] Warning: be carefull with the use of Fortran Lapack/Blas libraries In-Reply-To: References: Message-ID: <01101016003501.22480@taco.polycnrs-gre.fr> On Tuesday 09 October 2001 23:14, Jeff Whitaker wrote: > I have been able to reproduce this error either with the lapack_lite > included with numeric, or with the lapack3.0 libs on my system. > > Gerard: are you sure the libs you are linking on your system are not > lapack 2.0? If not, could you send me some code that triggers the error? > > > -Jeff After discussion with Jeff, I installed the latest lapack-3.0 (bugfix 2) and the problem went away. (I suppose I had lapack-3.0-bugfix 1 installed). Gerard From Aureli.Soria_Frisch at ipk.fhg.de Wed Oct 10 07:52:05 2001 From: Aureli.Soria_Frisch at ipk.fhg.de (Aureli Soria Frisch) Date: Wed Oct 10 07:52:05 2001 Subject: [Numpy-discussion] NumPy compatible GTK-based framework for color images Message-ID: Hi folks, A framework for manipulation of images with explicit support for NumPy arrays has been implemented: > >hello, >i would like to share a programming effort with you concerning the >display and manipulation of color images using GTK. >my idea is to have a programming tool which gives easy access to the >image memory space (using C matrix notation) while allowing the user >confortable interactive display features like pan/zoom/selection. >you may have a look at it under: > >http://vision.fhg.de/~daniel/pixelpark > >greetings, daniel. > >Daniel Kottow >daniel.kottow at ipk.fhg.de >Security and Testing Technology >http://vision.fhg.de/~daniel >Production Technology Centre Pascalstr. 8 - 9 >Fraunhofer-Institut IPK Berlin D - 10587 Berlin Best regards, Aureli ################################# Aureli Soria Frisch Fraunhofer IPK Dept. Pattern Recognition post: Pascalstr. 8-9, 10587 Berlin, Germany e-mail:aureli at ipk.fhg.de fon: +49 30 39006-150 fax: +49 30 3917517 web: http://vision.fhg.de/~aureli/web-aureli_en.html ################################# From altis at semi-retired.com Wed Oct 10 11:11:11 2001 From: altis at semi-retired.com (Kevin Altis) Date: Wed Oct 10 11:11:11 2001 Subject: [Numpy-discussion] drawing with NumPy and PythonCard - SciPy test In-Reply-To: Message-ID: A suggestion was made that I should check out SciPy. I downloaded and verified that the plt module works fine from within a PythonCard app using the built-in PythonCard shell. My test was the plt tutorial at http://www.scipy.org/site_content/tutorials/plot_tutorial I've emailed some of the SciPy developers, but there seems to be some mail gateway issues (the mail bounced) on their server right now, so I haven't heard back from anyone yet. Given that plt works with PythonCard today, people might be interested in being able to create dynamic buttons, fields, and methods from the interactive prompt using SciPy with PythonCard. The follwoing message shows dynamic method binding. http://aspn.activestate.com/ASPN/Mail/Message/770015 Adding other widgets such as a field is just as easy as adding a button, it is the same syntax as used in the resource files, so the following is valid: >>> comp['field2'] = {'type':'TextField', 'name':'field2', 'position':(0, 30), 'size':(150, -1), 'text':'this is field2'} For people that want to mostly do interactive sessions, but still be able to use buttons and fields to speed up data entry... this will probably be useful in the context of something like SciPy. Let me know if there are other questions or suggestions. ka > -----Original Message----- > From: numpy-discussion-admin at lists.sourceforge.net > [mailto:numpy-discussion-admin at lists.sourceforge.net]On Behalf Of Kevin > Altis > Sent: Monday, October 08, 2001 10:40 AM > To: numpy-discussion at lists.sourceforge.net > Subject: [Numpy-discussion] drawing with NumPy and PythonCard > > > Hi, > I'm the coordinator for the PythonCard project, which is at: > http://pythoncard.sourceforge.net/ > > PythonCard provides a GUI framework for building Python applications. The > project is relatively new, but there are a large number of sample > applications, including a number of drawing and visualization samples. > PythonCard already supports loading and drawing of common formats > like XBM, > GIF, JPEG, PNG, etc. from disk files. > > Last month I started a BitmapCanvas for the framework to simplify > interactive drawing. Rather than duplicate all of the functionality of the > Python Imaging Library (PIL) or the various plotting packages already > available, I want to make it easy to display results created with other > drawing packages and numPy in PythonCard. To that end, the Bitmap and > BitmapCanvas class already support PIL. You can convert a PIL RGB image > to/from a PythonCard bitmap and you can draw a PIL RGB image in a > BitmapCanvas at a given x, y location. If you're already converting from > numPy to PIL, you should be able to show your results in > PythonCard today. I > will be making a related post about this on the image-sig. > > Anyway, what I would like to do is provide similar methods to make it easy > to convert numPy arrays to images or a bitmap to numPy without > having to go > to PIL first. I don't currently use numPy since I don't do any scientific > computing (and my math skills are pretty pathetic) and most of my programs > are text or database-oriented, so I'm relying on others to spec out the > features they need or want. Eventually, I would like to add some visual > effects to PythonCard such as wipes and fades which would be a perfect use > of arrays. I want to make sure that PythonCard provides a good > GUI framework > for the scientific community. I would appreciate any suggestions, pointers > to related resources, etc. > > PythonCard currently sits on top of wxPython, so if you are > already working > with numPy and wxPython I would be very interested to hear about > it. If you > would like to get more actively involved, please join the > PythonCard mailing > list, which is probably a better place for longer-term > discussions than the > numPy list. > http://lists.sourceforge.net/lists/listinfo/pythoncard-users > > Thanks, > > ka > --- > Kevin Altis > altis at semi-retired.com > > > _______________________________________________ > Numpy-discussion mailing list > Numpy-discussion at lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/numpy-discussion > From nodwell at physics.ubc.ca Fri Oct 12 10:40:13 2001 From: nodwell at physics.ubc.ca (Eric Nodwell) Date: Fri Oct 12 10:40:13 2001 Subject: [Numpy-discussion] sign of modulo Message-ID: <20011012103933.A3659@holler.physics.ubc.ca> The Numpy version of modulo uses a different sign convention for negative arguments than does Python itself: >>> -5 % 3 1 >>> array((-5,-5)) % 3 array([-2, -2]) (Numpy version 20.2.0, Python version 1.5.2 and version 2.2a4.) Is there a reason for this (hard to imagine), or is this a bug? Eric From focke at SLAC.Stanford.EDU Fri Oct 12 11:09:11 2001 From: focke at SLAC.Stanford.EDU (Warren Focke) Date: Fri Oct 12 11:09:11 2001 Subject: [Numpy-discussion] sign of modulo In-Reply-To: <20011012103933.A3659@holler.physics.ubc.ca> Message-ID: On Fri, 12 Oct 2001, Eric Nodwell wrote: > The Numpy version of modulo uses a different sign convention for > negative arguments than does Python itself: ... > Is there a reason for this (hard to imagine), or is this a bug? Numpy delegates to the C platform's / and % operators, Python does it "right". Maybe this should go on the FAQ. Waren Focke From nodwell at physics.ubc.ca Fri Oct 12 11:22:05 2001 From: nodwell at physics.ubc.ca (Eric Nodwell) Date: Fri Oct 12 11:22:05 2001 Subject: [Numpy-discussion] sign of modulo In-Reply-To: References: <20011012103933.A3659@holler.physics.ubc.ca> Message-ID: <20011012112109.A3835@holler.physics.ubc.ca> > Numpy delegates to the C platform's / and % operators, Python does it > "right". > > Maybe this should go on the FAQ. Wouldn't it be preferable to change Numpy so it is also "right"? From focke at SLAC.Stanford.EDU Fri Oct 12 18:03:10 2001 From: focke at SLAC.Stanford.EDU (Warren Focke) Date: Fri Oct 12 18:03:10 2001 Subject: [Numpy-discussion] sign of modulo In-Reply-To: <20011012112109.A3835@holler.physics.ubc.ca> Message-ID: On Fri, 12 Oct 2001, Eric Nodwell wrote: > > Numpy delegates to the C platform's / and % operators, Python does it > > "right". > Wouldn't it be preferable to change Numpy so it is also "right"? Speed. In Python, there's so much overhead from symbol resolution, looking up __mod__s and __rmod__s and whatnot that the extra work is lost in the noise. Numpy would take a bigger hit when working with large arrays. One could always write python_compatible_div and python_compatible_mod functions, if one wanted. Could even submit a patch to add them, if truly motivated. Warren Focke From sag at hydrosphere.com Mon Oct 15 12:41:12 2001 From: sag at hydrosphere.com (Sue Giller) Date: Mon Oct 15 12:41:12 2001 Subject: [Numpy-discussion] Problems with pickling an array with type Object Message-ID: <20011015194003406.AAA229@mail.climatedata.com@spare> Greetings, I am trying to pickle an array of objects with a typecode of Object. Ultimately this array will be in a class I create based on UserArray, but the following illustrates a problem I am having. See code at the end of the message. I am working in Windows 2000, Service Pack 1, using PythonWin, win32all build 140. I am using the following versions of python stuff Python: 2.1.1 (#20, Jul 20 2001, 01:19:29) [MSC 32 bit (Intel)] Numeric: 20.2.0 cPickle: 1.71 I create a simple array with a type of Object, pickle it to a file using cPickle. I then run this script in PythonWin. I can unpickle this file from PythonWin, but not from DOS window. Alternatively, I can pickle the array from DOS, and unpickle it from DOS, but not from PythonWin. If I save both versions of the file and compare them, they are the same size, but not the same bytes. This does not happen with other types for the array. When unpickling, I get a GPF, either from PythonWin.exe or Python.exe. Any ideas? My ultimate goal is to be able to store simple class objects in the array and be able to pickle and unpickle them. ----- CODE ---- # simple test for pickling problem import Numeric import cPickle import sys basePath = "c:\\python21\\scripts\\pickle\\" print "++++++" # create a simple array of object, populated by integer data ds1 = Numeric.array([1,2,3,4], 'O') print ds1, ds1.typecode() print "Python: ", sys.version print "Numeric: " , Numeric.__version__ print "cPickle: ", cPickle.__version__ pickle = 1 # change to 0 to skip the pickling step for ii in (range(1,2)): if pickle: fileName = "%s%s%d%s" % (basePath, "simple", ii, ".pck") print "pickling ", fileName fp = open(fileName, 'wb') cPickle.dump(ds1, fp, 1) fp.close() for ii in range(1,2): fileName = "%s%s%d%s" % (basePath, "simple", ii, ".pck") print "unpickling ", fileName fp = open(fileName, 'rb') obj = cPickle.load(fp) fp.close() print obj, obj.typecode() From paul at pfdubois.com Tue Oct 16 08:55:14 2001 From: paul at pfdubois.com (Paul F. Dubois) Date: Tue Oct 16 08:55:14 2001 Subject: [Numpy-discussion] RE: activity on NumPy In-Reply-To: Message-ID: <000301c1565a$a75e3920$3d01a8c0@plstn1.sfba.home.com> No, I just have a major release coming up of cdat (cdat.sf.net) so I've been busy. -----Original Message----- From: nobody [mailto:nobody at sourceforge.net] On Behalf Of Alessandro MIRONE Sent: Tuesday, October 16, 2001 2:22 AM To: paul at pfdubois.com Subject: activity on NumPy Hi, I have noticed that that developer activity has almost stopped ( bugs and patches are no more processed) after 11/9. Did you get some restriction at your lab? Best regards Alessandro Mirone From sag at hydrosphere.com Thu Oct 25 09:35:09 2001 From: sag at hydrosphere.com (Sue Giller) Date: Thu Oct 25 09:35:09 2001 Subject: [Numpy-discussion] Problem with Numeric? Message-ID: <20011025163432046.AAA233@mail.climatedata.com@spare> I am using Numeric 20.2.0. I have found the following inconsistency. I am not a python guru and have no idea how I would fix it myself. The following code creates a series of arrays of data, and then applies maximum.reduce to the array to get the largest value. It then prints the type of returned max value. Most of the returns are typed as an array with a shape of (). If the array is created as a type of 'l' (long) or 'd' (double), the return type is not array. This seems wrong. Can someone tell me if this is expected, and if not, where would I look to change/fix it. --- CODE --- from Numeric import * print "++++" for tc in ('b', 'i', 'l', 's', 'f', 'd'): smallArray = array([1,2,3,4], tc) amax = maximum.reduce(smallArray) print smallArray.typecode(), smallArray, "Max:", amax, type(amax) if type(amax) == type(smallArray): print amax.shape --- OUTPUT --- >>>> ++++ b [1 2 3 4] Max: 4 () i [1 2 3 4] Max: 4 () l [1 2 3 4] Max: 4 s [1 2 3 4] Max: 4 () f [ 1. 2. 3. 4.] Max: 4.0 () d [ 1. 2. 3. 4.] Max: 4.0 From jbaddor at physics.mcgill.ca Fri Oct 26 07:47:03 2001 From: jbaddor at physics.mcgill.ca (Jean-Bernard Addor) Date: Fri Oct 26 07:47:03 2001 Subject: [Numpy-discussion] (real)**(2x2 general real matrix) Message-ID: Hey numpy people! Do you know the better way to compute with numpy: r**M where r is a positive real and M a 2x2 (rank 2) matrix, symmetric or not, with real or not eigenvalues. Help or comment are appreciated, Jean-Bernard From nwagner at mecha.uni-stuttgart.de Tue Oct 30 00:22:02 2001 From: nwagner at mecha.uni-stuttgart.de (Nils Wagner) Date: Tue Oct 30 00:22:02 2001 Subject: [Numpy-discussion] Rank deficient matrices Message-ID: <3BDE63B2.EA84FA27@mecha.uni-stuttgart.de> Hi, Let us assume that r = rank(C) < N (1) where C is a symmetric NxN matrix. This implies that the number of non-zero eigenvalues of C is r. Because C is a symmetric matrix there exists an orthogonal matrix U whose columns are the eigenvectors of C such that U^\top C U = [ d , O O , O]. (2) In the above equation d \in rxr is a diagonal matrix consisting of only the non-zero eigenvalues of C. For convenience, partition U as U = [U_1 | U_2] (3) where the columns of U_1 (Nxr) are the eigenvectors corresponding to the non-zero block d and the columns of U_2 are the eigenvectors corresponding to the rest (N-r) number of zero eigenvalues. Defining a rectangular transformation matrix R = U_1 (4) it is easy to show that R^\top C R = d. (5) Therefore, the matrix R in equation (4) transforms the originally rank deficient matrix C to a full-rank (diagonal) matrix of rank r. I am looking for an efficient Numpy implementation of this transformation. Thanks in advance Nils Wagner From paul at pfdubois.com Tue Oct 30 07:21:07 2001 From: paul at pfdubois.com (Paul F. Dubois) Date: Tue Oct 30 07:21:07 2001 Subject: [Numpy-discussion] Rank deficient matrices In-Reply-To: <3BDE63B2.EA84FA27@mecha.uni-stuttgart.de> Message-ID: <001a01c16156$3827d1e0$3d01a8c0@plstn1.sfba.home.com> Use LinearAlgebra.singular_value_decomposition -----Original Message----- From: numpy-discussion-admin at lists.sourceforge.net [mailto:numpy-discussion-admin at lists.sourceforge.net] On Behalf Of Nils Wagner Sent: Tuesday, October 30, 2001 12:24 AM To: numpy-discussion at lists.sourceforge.net Subject: [Numpy-discussion] Rank deficient matrices Hi, Let us assume that r = rank(C) < N (1) where C is a symmetric NxN matrix. This implies that the number of non-zero eigenvalues of C is r. Because C is a symmetric matrix there exists an orthogonal matrix U whose columns are the eigenvectors of C such that U^\top C U = [ d , O O , O]. (2) In the above equation d \in rxr is a diagonal matrix consisting of only the non-zero eigenvalues of C. For convenience, partition U as U = [U_1 | U_2] (3) where the columns of U_1 (Nxr) are the eigenvectors corresponding to the non-zero block d and the columns of U_2 are the eigenvectors corresponding to the rest (N-r) number of zero eigenvalues. Defining a rectangular transformation matrix R = U_1 (4) it is easy to show that R^\top C R = d. (5) Therefore, the matrix R in equation (4) transforms the originally rank deficient matrix C to a full-rank (diagonal) matrix of rank r. I am looking for an efficient Numpy implementation of this transformation. Thanks in advance Nils Wagner _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From edcjones at erols.com Tue Oct 30 12:00:02 2001 From: edcjones at erols.com (Edward C. Jones) Date: Tue Oct 30 12:00:02 2001 Subject: [Numpy-discussion] Numeric2 Message-ID: <3BDF066E.3050200@erols.com> What is the current status of Numeric2? From perry at stsci.edu Tue Oct 30 12:40:03 2001 From: perry at stsci.edu (Perry Greenfield) Date: Tue Oct 30 12:40:03 2001 Subject: [Numpy-discussion] RE: Numeric2 In-Reply-To: Message-ID: > Date: Tue, 30 Oct 2001 14:58:38 -0500 > From: "Edward C. Jones" > To: numpy-discussion at lists.sourceforge.net > Subject: [Numpy-discussion] Numeric2 > > What is the current status of Numeric2? > We are in the process of putting it up on sourceforge now. While you can actually download it from there now, I would strongly recommend waiting until we put some more current and up-to-date documentation about it there as well. We will announce it on this mailing list when it is ready (within a week?). But it isn't called Numeric2 any longer. We've called it numarray and it will be found under "python arrays" on Sourceforge. Perry Greenfield From edcjones at erols.com Wed Oct 31 07:51:03 2001 From: edcjones at erols.com (Edward C. Jones) Date: Wed Oct 31 07:51:03 2001 Subject: [Numpy-discussion] Undefined symbol in lapack_lite.so Message-ID: <3BE01DC2.8090209@erols.com> The following Python program: import Numeric, LinearAlgebra a = Numeric.identity(5) v, s, wt = LinearAlgebra.singular_value_decomposition(a) gave the following error: Traceback (most recent call last): File "./silly.py", line 3, in ? import Numeric, LinearAlgebra File "/usr/lib/python2.2/site-packages/Numeric/LinearAlgebra.py", line 8, in ? import lapack_lite ImportError: /usr/lib/python2.2/site-packages/Numeric/lapack_lite.so: undefined symbol: dgesdd_ When I installed Numeric-20.2.0, I changed Setup.py to: # delete all but the first one in this list if using your own LAPACK/BLAS sourcelist = ['Src/lapack_litemodule.c',] # set these to use your own BLAS library_dirs_list = ['usr/lib'] libraries_list = ['/usr/lib/libblas.so', '/usr/lib/liblapack.so'] What is the problem? Thanks, Ed Jones From gvermeul at labs.polycnrs-gre.fr Wed Oct 31 08:41:04 2001 From: gvermeul at labs.polycnrs-gre.fr (Gerard Vermeulen) Date: Wed Oct 31 08:41:04 2001 Subject: [Numpy-discussion] Undefined symbol in lapack_lite.so In-Reply-To: <3BE01DC2.8090209@erols.com> References: <3BE01DC2.8090209@erols.com> Message-ID: <01103117403200.00960@taco.polycnrs-gre.fr> Do you have lapack-3.0? Numeric-20.2.0 (better get 20.2.1) is interfaced to new faster lapack routines. You could check it by [packer at taco lib]$ strings liblapack.so.3 | grep dgesdd dgesdd_ [packer at taco lib]$ best regards -- Gerard On Wednesday 31 October 2001 16:50, Edward C. Jones wrote: > The following Python program: > > import Numeric, LinearAlgebra > > a = Numeric.identity(5) > v, s, wt = LinearAlgebra.singular_value_decomposition(a) > > gave the following error: > > Traceback (most recent call last): > File "./silly.py", line 3, in ? > import Numeric, LinearAlgebra > File "/usr/lib/python2.2/site-packages/Numeric/LinearAlgebra.py", line > 8, in ? import lapack_lite > ImportError: /usr/lib/python2.2/site-packages/Numeric/lapack_lite.so: > undefined symbol: dgesdd_ > > When I installed Numeric-20.2.0, I changed Setup.py to: > > # delete all but the first one in this list if using your own LAPACK/BLAS > sourcelist = ['Src/lapack_litemodule.c',] > # set these to use your own BLAS > library_dirs_list = ['usr/lib'] > libraries_list = ['/usr/lib/libblas.so', '/usr/lib/liblapack.so'] > > What is the problem? > > Thanks, > Ed Jones > > > > _______________________________________________ > Numpy-discussion mailing list > Numpy-discussion at lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/numpy-discussion