From alawhead@vcn.bc.ca Fri Aug 7 06:19:28 1998 From: alawhead@vcn.bc.ca (Alexander Lawhead) Date: Thu, 6 Aug 1998 22:19:28 -0700 (PDT) Subject: [Matrix-SIG] 1-d smoother? Message-ID: Has anyone implemented any form of 1-d smoother using NumPy (loess, smoothing spline, kernel smoother etc)? I have need of such a beast and would rather be lazy and use someone else's code than do it myself! :) Thanks in advance! Alexander From janne@avocado.pc.helsinki.fi Sat Aug 8 22:22:09 1998 From: janne@avocado.pc.helsinki.fi (Janne Sinkkonen) Date: 09 Aug 1998 00:22:09 +0300 Subject: [Matrix-SIG] Numerical Python in Alpha Linux, is it ok? Message-ID: Just to make sure I will not have problems with the most important tool: is the combination of NumPy, Gist and AXP Linux working flawlessly, or should I expect some kind of problems? -- Janne From ffjhl@aurora.uaf.edu Sat Aug 8 22:42:03 1998 From: ffjhl@aurora.uaf.edu (Jonah Lee) Date: Sat, 8 Aug 1998 13:42:03 -0800 (AKDT) Subject: [Matrix-SIG] Numerical Python in Alpha Linux, is it ok? In-Reply-To: Message-ID: On 9 Aug 1998, Janne Sinkkonen wrote: > > Just to make sure I will not have problems with the most important tool: > is the combination of NumPy, Gist and AXP Linux working flawlessly, or > should I expect some kind of problems? I have NumPy compiled on AXP Linux just fine. I have not installed Gist on that system but there should not be any problems. > > -- > Janne > > _______________________________________________ > Matrix-SIG maillist - Matrix-SIG@python.org > http://www.python.org/mailman/listinfo/matrix-sig > > Regards, Jonah From mhagger@blizzard.harvard.edu Tue Aug 11 01:02:49 1998 From: mhagger@blizzard.harvard.edu (Michael Haggerty) Date: Mon, 10 Aug 1998 20:02:49 -0400 Subject: [Matrix-SIG] repeated transposes Message-ID: <9808110002.AA22223@tsunami.harvard.edu> Hi, There is a feature of the Numeric module that is somewhat inconvenient, and can adversely affect performance. Namely, that Numeric doesn't realize that a matrix can always be transposed without copying data, and that a matrix that has been transposed twice is back in `contiguous' storage format. >>> M = array([[1,2],[3,4]]) >>> Numeric.transpose(Numeric.transpose(M)).iscontiguous() 0 Why is this case important? Often Numeric is used to interface to large C or Fortran matrices. But Fortran stores matrices column-wise, whereas Numeric stores them (by default) row-wise. No problem: whenever you receive a matrix from Fortran, you transpose() it; and before you pass any matrix back to Fortran, you transpose() it again. But since Numeric isn't fastidious about keeping track of when a matrix is stored contiguously, the double transpose causes the matrix to be copied two extra times (once on entry to the second transpose operation, which copies it into the *wrong* order; a second time when putting it back into Fortran order. But there is no need for transpose ever to copy data--transposes can always be done by adjusting dimensions and strides--and in fact, the current algorithm is general enough to work with non-contiguous matrices. The only trick is to get it to realize when it has just put a matrix *back* into contiguous order and set the CONTIGUOUS bit back on. The following patch to multiarraymodule.c does that. With this patch, >>> m = array([[1,2,3],[4,5,6]]) >>> m array([[1, 2, 3], [4, 5, 6]]) >>> n = transpose(m) >>> n.iscontiguous() 0 >>> o = transpose(n) >>> o array([[1, 2, 3], [4, 5, 6]]) >>> o.iscontiguous() 1 >>> o[1,0] = 0 >>> o array([[1, 2, 3], [0, 5, 6]]) >>> n array([[1, 0], [2, 5], [3, 6]]) >>> m array([[1, 2, 3], [0, 5, 6]]) So you can see that now all three arrays refer to the same memory, and that o is recognized to be contiguous. There might be other Numeric algorithms that can be implemented in a way that they can deal with noncontiguous data. This way, the copying of large matrices can be avoided as much as possible. The function provided in the patch below, `array_really_contiguous()', might be useful for that purpose. See patch below... Yours, Michael -- Michael Haggerty mhagger@blizzard.harvard.edu ========================= Patch ============================== *** ../../../LLNLPython4/Numerical/Src/multiarraymodule.c Fri Jun 19 11:27:18 1998 --- multiarraymodule.c Mon Aug 10 19:10:04 1998 *************** *** 115,120 **** --- 115,136 ---- return NULL; } + /* Check whether the given array is stored contiguously (row-wise) in + memory. */ + static int array_really_contiguous(PyArrayObject *ap) { + int sd; + int i; + + sd = ap->descr->elsize; + for (i = ap->nd-1; i >= 0; --i) { + if (ap->dimensions[i] == 0) return 1; /* contiguous by definition */ + if (ap->strides[i] != sd) return 0; + sd *= ap->dimensions[i]; + } + return 1; + } + + /* Changed to be able to deal with non-contiguous arrays. */ extern PyObject *PyArray_Transpose(PyArrayObject *ap, PyObject *op) { long *axes, axis; int i, n; *************** *** 135,145 **** --- 151,164 ---- permutation[i] = axis; } + /* this allocates memory for dimensions and strides (but fills them + incorrectly), sets up descr, and points data at ap->data. */ ret = (PyArrayObject *)PyArray_FromDimsAndData(n, permutation, ap->descr->type_num, ap->data); if (ret == NULL) goto fail; + /* point at true owner of memory: */ ret->base = (PyObject *)ap; Py_INCREF(ap); *************** *** 147,154 **** ret->dimensions[i] = ap->dimensions[permutation[i]]; ret->strides[i] = ap->strides[permutation[i]]; } ! ret->flags &= ~CONTIGUOUS; ! PyArray_Free(op, (char *)axes); free(permutation); return (PyObject *)ret; --- 166,174 ---- ret->dimensions[i] = ap->dimensions[permutation[i]]; ret->strides[i] = ap->strides[permutation[i]]; } ! if (array_really_contiguous(ret)) ret->flags |= CONTIGUOUS; ! else ret->flags &= ~CONTIGUOUS; ! PyArray_Free(op, (char *)axes); free(permutation); return (PyObject *)ret; *************** *** 969,977 **** PyArrayObject *a; if (PyArg_ParseTuple(args, "OO", &a0, &shape) == NULL) return NULL; ! if ((a = (PyArrayObject *)PyArray_ContiguousFromObject(a0, ! PyArray_NOTYPE,0,0)) ! ==NULL) return NULL; ret = PyArray_Transpose(a, shape); Py_DECREF(a); --- 989,997 ---- PyArrayObject *a; if (PyArg_ParseTuple(args, "OO", &a0, &shape) == NULL) return NULL; ! if ((a = (PyArrayObject *)PyArray_FromObject(a0, ! PyArray_NOTYPE,0,0)) ! ==NULL) return NULL; ret = PyArray_Transpose(a, shape); Py_DECREF(a); From kate@cptc.wisc.edu Wed Aug 12 06:55:51 1998 From: kate@cptc.wisc.edu (Kate Comer) Date: Wed, 12 Aug 1998 00:55:51 -0500 (CDT) Subject: [Matrix-SIG] returning multiple arrays from C Message-ID: I'm wrapping C around a set of FORTRAN spline subroutines that I want to access from Python. Before I pull out the rest of my hair in frustration, could someone please reveal the secret to returning more than one array from a C extension back to python, and having the returned arrays actually be different? This is what I'm doing, with ~8 arrays instead of just y, dy, and yy double *c_y, *c_yy, *c_dy; PyObject *num_y, *num_yy, *num_dy; PyArrayObject *arr_y, arr_yy, *num_dy; DECLARE_INTEGER(m); \* these DECLARES *\ DECLARE_REAL_ARRAY(y, m); \* are for interface *\ DECLARE_REAL_ARRAY(yy,m); \* to FORTRAN spline *\ DECLARE_REAL_ARRAY(dy, m); \* subroutine *\ arr_y = (PyArrayObject *)PyArray_ContiguousFromObject( num_y, PyArray_DOUBLE, 0, 0); c_y = (double *)arr_y->data; \* ...and the same thing two more times for the yy variables and dy variables ...*\ for( i=0; i (message from Kate Comer on Wed, 12 Aug 1998 00:55:51 -0500 (CDT)) References: Message-ID: <199808121041.MAA28690@dirac.cnrs-orleans.fr> > I'm wrapping C around a set of FORTRAN spline subroutines that I want to > access from Python. Before I pull out the rest of my hair in frustration, > could someone please reveal the secret to returning more than one array > from a C extension back to python, and having the returned arrays actually > be different? No secret, it works as advertised. I suppose there is some subtle bug in your code. There are certainly plenty of problems in the example below, it shouldn't even compile! > This is what I'm doing, with ~8 arrays instead of just y, dy, and yy > > double *c_y, *c_yy, *c_dy; > PyObject *num_y, *num_yy, *num_dy; > PyArrayObject *arr_y, arr_yy, *num_dy; That should be PyArrayObject *arr_y, *arr_yy, *arr_dy; Right? > DECLARE_INTEGER(m); \* these DECLARES *\ > DECLARE_REAL_ARRAY(y, m); \* are for interface *\ > DECLARE_REAL_ARRAY(yy,m); \* to FORTRAN spline *\ > DECLARE_REAL_ARRAY(dy, m); \* subroutine *\ No idea what that does... > arr_y = (PyArrayObject *)PyArray_ContiguousFromObject( > num_y, PyArray_DOUBLE, 0, 0); > c_y = (double *)arr_y->data; > > \* ...and the same thing two more times for the yy variables and dy > variables ...*\ Make sure that this section is correct; a typo anywhere could well cause the problem you describe. > for( i=0; i y[i] = *(c_x++); \* not real data, so they aren't in loop *\ > } (That's c_y instead of c_x, right?) Why do you have to copy the data? You could just pass c_y directly to the Fortran routine. And again, any typo... > return Py_BuildValue("OO", arr_yy, arr_dy); Looks OK. > Within Python I have > > y = array(whatever - it's 1d) > m = len(y) > dy = zeros(m, Float) > yy = dy There it is! You set yy equal to dy, which means that they refer to the same array. In the C routine, you call PyArray_ContiguousFromObject, but this routine does not create a new array if it finds that the original argument is already contiguous. So the two arrays stay equal at all times. The obvious quick fix is dy = zeros(m, Float) yy = zeros(m, Float) in Python. A more elegant way (in my opinion) is to create the arrays dy and yy in the C code instead of passing them in from Python. And you don't even have to pass in m, since it's just the length of the array y. -- ------------------------------------------------------------------------------- Konrad Hinsen | E-Mail: hinsen@cnrs-orleans.fr Centre de Biophysique Moleculaire (CNRS) | Tel.: +33-2.38.25.55.69 Rue Charles Sadron | Fax: +33-2.38.63.15.17 45071 Orleans Cedex 2 | Deutsch/Esperanto/English/ France | Nederlands/Francais ------------------------------------------------------------------------------- From johann@leporello.Berkeley.EDU Fri Aug 21 00:06:42 1998 From: johann@leporello.Berkeley.EDU (Johann Hibschman) Date: Thu, 20 Aug 1998 16:06:42 -0700 (PDT) Subject: [Matrix-SIG] roll-my-own ufuncs Message-ID: Hi all, Is there any additional documentation available for ufuncs? I have a few functions which I regularly call, and it would be convenient if there were expressed in python as ufuncs, but at first glance the ufuncmodule code is rather impenetrable. In general, is there a recommended method of passing a function to a NumPy-aware function? I want to write a curve-fitting routine which would work either on generic python functions or on wrapped C functions. I'd like wrapped C functions to be used at C speeds, if possible. Any hints? - Johann From hauser@engr.uky.edu Wed Aug 26 20:18:31 1998 From: hauser@engr.uky.edu (Thomas Hauser) Date: Wed, 26 Aug 1998 15:18:31 -0400 Subject: [Matrix-SIG] Missing EB.py for PyPDB Message-ID: <199808261918.PAA03224@tomslinux.hauser.home> Hi, I'm trying to use the PyPDB package. The PR.py imports EB where is EB.py Thanks Thomas Hauser From just@letterror.com Wed Aug 26 20:42:53 1998 From: just@letterror.com (Just van Rossum) Date: Wed, 26 Aug 1998 21:42:53 +0200 Subject: [Matrix-SIG] Conway's Life with NumPy Message-ID: Today it occurred to me that it should be fairly trivial to implement Conway's Life with NumPy. It was. It's a bit of nostalgia, since it was one of the first programs I ever typed over from a book on my first computer: a Sinclair ZX81... Anyway, I thought I'd contribute this as a demo program, since it's fairly trivial, and shows some basic NumPy features. It also shows why Rich Comparisons are a Good Thing... It would be nice to have a Tk based version of this, to make it more attractive, but my Tk skills are still non-existant. I still feel like a total NumPy newbie, so if anyone has suggestions on how to do things differently, I'd be very interested. Just --- Life.py --- """A Python implementation of Conway's Life Michael Wohlgenannt writes this about Life: "For a single cell there are two distinct states, dead and alive. A living cell dies of boredom if there are less than two living neighbours (all in all there are eight). A dead cell gets back to life again if there are exactly three living neighbours. The last rule is that a living cell dies if there are more than three living neighbours. A lot of cell configurations that can be constructed show a peculiar and amusing behaviour. Some stay as they are, dead or alive, some oscillate, some even propagate." """ # 26 august 1998, Just van Rossum from Numeric import * import sys import string def life(cells): # convert cells to bytes for speed cells = cells.astype(Int8) # calculate how many neibors each cell has neighbors = shift_left(shift_up(cells)) neighbors = neighbors + shift_up(cells) neighbors = neighbors + shift_right(shift_up(cells)) neighbors = neighbors + shift_left(cells) neighbors = neighbors + shift_right(cells) neighbors = neighbors + shift_left(shift_down(cells)) neighbors = neighbors + shift_down(cells) neighbors = neighbors + shift_right(shift_down(cells)) # apply the "Life" rules (see module doc string) newcells = logical_or( logical_and( cells, logical_and( less_equal(2, neighbors), less_equal(neighbors, 3) ) ), equal(neighbors, 3) ) # If I understood it correctly, with "rich comparisons" # the above could look like this: # # newcells = cell and (2 <= neighbors <= 3) or neighbors == 3 # # Now, wouldn't that be nice... return newcells def shift_up(cells): return concatenate((cells[1:], cells[:1])) def shift_down(cells): return concatenate((cells[-1:], cells[:-1])) def shift_left(cells): return transpose(shift_up(transpose(cells))) def shift_right(cells): return transpose(shift_down(transpose(cells))) def randomcells(width, height, chance = 5): from whrandom import randint cells = zeros((height, width), Int8) _range = range # fill with noise for y in _range(height): for x in _range(width): cells[y][x] = randint(0, chance) == 0 return cells def printcells(cells): x, y = cells.shape thing = "+" + y * "-" + "+" lines = [thing] for i in range(x): list = map(lambda x: " O"[x], cells[i]) lines.append("|"+string.join(list, "")+"|") lines.append(thing) print string.join(lines, "\n") if __name__ == "__main__": import time width = 20 height = 10 cells = randomcells(width, height) while 1: printcells(cells) time.sleep(0.1) cells = life(cells) From Paul F. Dubois" Your program is now LLNLDistribution/Numerical/Demo/life.py. Look for it in our next release, and thanks. A version that uses Tk for display, as suggested, would be welcome. Anyone? From amullhau@ix.netcom.com Wed Aug 26 21:13:10 1998 From: amullhau@ix.netcom.com (Andrew P. Mullhaupt) Date: Wed, 26 Aug 1998 16:13:10 -0400 Subject: [Matrix-SIG] Conway's Life with NumPy Message-ID: <04d201bdd12d$f3303c00$f2615ecf@amullhau.ix.netcom.com> -----Original Message----- From: Just van Rossum To: matrix-sig@python.org Date: Wednesday, August 26, 1998 3:44 PM Subject: [Matrix-SIG] Conway's Life with NumPy >Today it occurred to me that it should be fairly trivial to implement >Conway's Life with NumPy. It was.> >I still feel like a total NumPy newbie, so if anyone has suggestions >on how to do things differently, I'd be very interested. You might find that the expressive power of Numerical Python makes it easy to implement Gosper's recursive version which is orders of magnitude more efficient in time and space for sparse cases, which is where you end up with many interesting life patterns. Later, Andrew Mullhaupt From aaron@cs.rutgers.edu Thu Aug 27 13:53:25 1998 From: aaron@cs.rutgers.edu (Aaron Watters) Date: Thu, 27 Aug 1998 08:53:25 -0400 Subject: [Matrix-SIG] Conway's Life with NumPy References: <04d201bdd12d$f3303c00$f2615ecf@amullhau.ix.netcom.com> Message-ID: <35E556C5.EF75B4B1@cs.rutgers.edu> Please expand. It seems to me that you need missing "rich indexing" functionality to implement sparse life, but perhaps I'm missing something important (very likely). Please inform. -- Aaron Watters Andrew P. Mullhaupt wrote: > -----Original Message----- > From: Just van Rossum > To: matrix-sig@python.org > Date: Wednesday, August 26, 1998 3:44 PM > Subject: [Matrix-SIG] Conway's Life with NumPy > > >Today it occurred to me that it should be fairly trivial to implement > >Conway's Life with NumPy. It was.> > >I still feel like a total NumPy newbie, so if anyone has suggestions > >on how to do things differently, I'd be very interested. > > You might find that the expressive power of Numerical Python makes it easy > to implement Gosper's recursive version which is orders of magnitude more > efficient in time and space for sparse cases, which is where you end up with > many interesting life patterns. > > Later, > Andrew Mullhaupt > > _______________________________________________ > Matrix-SIG maillist - Matrix-SIG@python.org > http://www.python.org/mailman/listinfo/matrix-sig From Paul F. Dubois" Everybody, Prompted by a message from Aaron, I think it is time to revisit this question. What do you think I should do about the N announcements of numerical-related Python packages? N is getting large. Clearly I can't maintain and test all this stuff. Even maintaining a list of links is fraught with peril. I wonder if some of this FAQ technology that Guido has might be a better tool to let people keep their own announcements/software up to date? I know some people have talked about me creating some big Linnean classification for a library (or we could, for example, use the NAG one, except that most people don't know what it is). But I'm sure I can't manage that level of effort. We are extremely short-handed at the moment. I'd appreciate any comments on what to do. Or rather, how I can accomplish something useful without doing much. Paul -----Original Message----- From: Aaron Watters To: Paul F. Dubois Date: Thursday, August 27, 1998 5:57 AM Subject: Re: [Matrix-SIG] Conway's Life with NumPy >I would be delighted if you'd put a link to the pysimplex >stuff somewhere in there too. (a copy might not be a good >idea, since i may get around to improving it at any time. > >http://www.pythonpros.com/arw/pysimpex > > -- Aaron Watters > >Paul F. Dubois wrote: > >> Your program is now LLNLDistribution/Numerical/Demo/life.py. Look for it in >> our next release, and thanks. A version that uses Tk for display, as >> suggested, would be welcome. Anyone? >> >> _______________________________________________ >> Matrix-SIG maillist - Matrix-SIG@python.org >> http://www.python.org/mailman/listinfo/matrix-sig > > > > From hinsen@cnrs-orleans.fr Mon Aug 31 15:20:14 1998 From: hinsen@cnrs-orleans.fr (Konrad Hinsen) Date: Mon, 31 Aug 1998 16:20:14 +0200 Subject: [Matrix-SIG] Conway's Life with NumPy In-Reply-To: <001001bdd48e$57d89c20$52120f80@dubois1-1.llnl.gov> References: <001001bdd48e$57d89c20$52120f80@dubois1-1.llnl.gov> Message-ID: <199808311420.QAA23048@dirac.cnrs-orleans.fr> > Clearly I can't maintain and test all this stuff. Even maintaining a list of > links is fraught with peril. I wonder if some of this FAQ technology that > Guido has might be a better tool to let people keep their own > announcements/software up to date? I know some people have talked about me I think that's worth a try. Maybe we can even convince one of the FAQ Wizard wizards to set up one for NumPy. Then you'd just have to add a pointer to your Web page and let others do the work. Konrad. -- ------------------------------------------------------------------------------- Konrad Hinsen | E-Mail: hinsen@cnrs-orleans.fr Centre de Biophysique Moleculaire (CNRS) | Tel.: +33-2.38.25.55.69 Rue Charles Sadron | Fax: +33-2.38.63.15.17 45071 Orleans Cedex 2 | Deutsch/Esperanto/English/ France | Nederlands/Francais ------------------------------------------------------------------------------- From da@skivs.ski.org Mon Aug 31 17:30:39 1998 From: da@skivs.ski.org (David Ascher) Date: Mon, 31 Aug 1998 09:30:39 -0700 (PDT) Subject: [Matrix-SIG] Conway's Life with NumPy In-Reply-To: <199808311420.QAA23048@dirac.cnrs-orleans.fr> Message-ID: > I think that's worth a try. Maybe we can even convince one of the > FAQ Wizard wizards to set up one for NumPy. Then you'd just > have to add a pointer to your Web page and let others do the work. It's in the works -- I'll announce the URL as soon as Ken and I finish getting the scientific computing alpha topic guide (which will happen once I get back stateside and clean it up a little). For management of numpy software, IMHO the right solution is to piggy back on the Grove system -- alas, I don't think it's quite ready yet. I'll ask Eric Raymond for his feeling on the ETA of that project and let y'all know. --david From Paul F. Dubois" I just rebuilt Python, Numerical, and C++ with MS Visual C++ 6.0 on an NT box and everything seemed to go well. They automatically convert the project files that David's tool puts out to a new format. I didn't do extensive testing. The compile seemed much faster.