From da@skivs.ski.org Wed Apr 1 22:11:30 1998 From: da@skivs.ski.org (David Ascher) Date: Wed, 1 Apr 1998 14:11:30 -0800 (PST) Subject: [Matrix-SIG] convolution... In-Reply-To: Message-ID: yarraghghhg.... The reason my linear convolution "wasn't working" is that the display code was buggy! I *hate* it when that happens. Ah, well, at least it's Tim appreciation day. --david From da@skivs.ski.org Fri Apr 3 00:31:58 1998 From: da@skivs.ski.org (David Ascher) Date: Thu, 2 Apr 1998 16:31:58 -0800 (PST) Subject: [Matrix-SIG] functions for numbers & arrays Message-ID: I'm getting annoyed at the fact that, as mentioned previously on this list (I believe it was Konrad, eons ago), it's hard to write functions which work for both arrays and lists. For example, I have functions which I want to evaluate at grids of parameter values (x = arange(2,4,.2), y=arange(1,2,.01)) and specific points (x = 2, y = 3). I haven't come up with a good way to do this that does the right thing in most circumstances. [for example, I'd like such a function to return an array in the first case, but a scalar in the second] What do other array languages do to address this problem? Anything we can steal? --david From dars@soton.ac.uk Fri Apr 3 10:03:49 1998 From: dars@soton.ac.uk (Dave Stinchcombe) Date: Fri, 3 Apr 1998 11:03:49 +0100 (BST) Subject: [Matrix-SIG] precision of numbers Message-ID: <199804031003.LAA02579@oak.sucs.soton.ac.uk> Hello again, I've been doing some more work (shock horror), and this time I want to be able to increase the precision of calculation. This is because I can't seem to make a parameter move in small enough steps to pick up a solution where I know one exists. At the moment I recursively increase precision in traditional bisection fashion, until a zero is found, except I run out of precision. Zero is obviously a very small number and not an actual zero, I chose |1e-10|. Is there another number type, or is there a precision parameter in Numpy, or can I fool it with a simple trick(I can think of one for my particular problem, but howabout in general)??? All answers gratefully received. Yours Dave From Paul F. Dubois" For many functions an actual zero does not exist on a computer. One correct way to do bisection is to pick an epsilon e and stop when (upper-lower)/(upper+lower) < e/2. At that point f(upper)*f(lower) <= 0.0 and you can choose upper, lower, or (upper+lower)/2.0 as your "root". -----Original Message----- From: Dave Stinchcombe To: matrix-sig@python.org Date: Friday, April 03, 1998 2:19 AM Subject: [Matrix-SIG] precision of numbers >Hello again, > >I've been doing some more work (shock horror), and this time I want to be >able to increase the precision of calculation. This is because I can't seem >to make a parameter move in small enough steps to pick up a solution where >I know one exists. At the moment I recursively increase precision in >traditional bisection fashion, until a zero is found, except I run out of >precision. Zero is obviously a very small number and not an actual zero, >I chose |1e-10|. > >Is there another number type, or is there a precision parameter in Numpy, >or can I fool it with a simple trick(I can think of one for my particular >problem, but howabout in general)??? All answers gratefully received. > >Yours >Dave > > > >------------------------------------------------------ >Matrix-SIG maillist - Matrix-SIG@python.org >http://www.python.org/mailman/listinfo/matrix-sig > From Paul F. Dubois" Line 466 of abstract.c should be: if (PyFloat_Check(v) && PyFloat_Check(w) && PyFloat_AsDouble(v) < 0.0) { This was the reason a numeric array to a real power was not working. From guido@CNRI.Reston.Va.US Fri Apr 3 23:42:34 1998 From: guido@CNRI.Reston.Va.US (Guido van Rossum) Date: Fri, 03 Apr 1998 18:42:34 -0500 Subject: [Matrix-SIG] Re: Bug fix In-Reply-To: Your message of "Fri, 03 Apr 1998 15:24:46 PST." <000301bd5f57$b1d28800$998a7380@pduboispc.llnl.gov> References: <000301bd5f57$b1d28800$998a7380@pduboispc.llnl.gov> Message-ID: <199804032342.SAA19298@eric.CNRI.Reston.Va.US> Thanks, Paul! I've added it to the 1.5 patch page at http://www.python.org/1.5/errors.html Note that I've also added an "all.txt" file to the 1.5 directory (not to its patches subdirectory) that contains all patches -- for those who would like to apply all patches to a clean Python 1.5 source tree. --Guido van Rossum (home page: http://www.python.org/~guido/) From da@skivs.ski.org Mon Apr 6 20:09:43 1998 From: da@skivs.ski.org (David Ascher) Date: Mon, 6 Apr 1998 12:09:43 -0700 (PDT) Subject: [Matrix-SIG] Proposal: adding rich comparisons to Python Message-ID: Note: the long-term address of this protocol is: http://starship.skyport.net/~da/rich/proposal.html That page also has links to the patches, for those who care. I'm CC'ing the Matrix-SIG so that those folks can be made aware of the issue, but I feel the discussion should really take place on the main list, since it's a Python-wide change for the sake of NumPy (and others). I await your constructive criticism, --david ----------------------------------------------------------------- Proposal for adding of a new "rich comparison" protocol to Python David Ascher (da@skivs.ski.org) version 0.1 April 6, 1998 Abstract A new mechanism allowing comparisons of Python objects to return values other than -1, 0, or 1 (or raise exceptions) is proposed. This mechanism is entirely backwards compatible, and can be controlled at the level of the C PyObject type or of the Python class definition. There are three cooperating parts to the proposed mechanism: * the use of the last slot in the type object structure to store a pointer to a rich comparison * the addition of special methods for classes * the addition of an optional argument to the builtin 'cmp()' function. Motivation The current comparison protocol for Python objects assumes that any two Python objects can be compared (as of Python 1.5, object comparisons can raise exceptions), and that the return value for any comparison should be -1, 0 or 1. -1 indicates that the first argument to the comparison function is less than the right one, +1 indicating the contrapositive, and 0 indicating that the two objects are equal. While this mechanism allows the establishment of a order relationship (e.g. for use by the sort() method of list objects), it has proven to be limited in the context of Numeric Python (NumPy). Specifically, NumPy allows the creation of multidimensional arrays, which support most of the numeric operators. Thus:: x = array((1,2,3,4)) y = array((2,2,4,4)) are two NumPy arrays. While they can be added elementwise,:: z = x + y # z == array((3,4,7,8)) they cannot be compared in the current framework - the released version of NumPy compares the *pointers*, which was the only solution before the recent addition of the ability (in 1.5) to raise exceptions in comparison functions. Even with the ability to raise exceptions, the current protocol makes array comparisons useless. To deal with this fact, NumPy includes several functions which perform the comparisons: 'less()', 'less_equal()', 'greater()', 'greater_equal()', 'equal()', 'not_equal()'. These functions return arrays with the same shape as their arguments (modulo broadcasting), filled with 0's and 1's depending on whether the comparison is true or not for each element pair. Thus, for example, using the arrays 'x' and 'y' defined above: 'less(x,y)' would be an array containing the numbers '(1,0,0,0)'. The current proposal is to modify the Python object interface to allow the NumPy package to make it so that x < y returns the same thing as 'less(x,y)'. The exact return value is up to the NumPy package -- what this proposal *really asks for* is changing the Python core so that extension objects have *the ability* to return something other than -1, 0, 1, should their authors choose to do so. Current State of Affairs The current protocol is, at the C level, that each object type defines a tp_compare slot, which is a pointer to a function which takes two 'PyObject*' references and returns -1, 0, or 1. This function is called by the 'PyObject_Compare()' function defined in the C API. 'PyObject_Compare()' is also called by the builtin function 'cmp()' which takes two arguments. Proposed Mechanism 1 Changes to the C structure for type objects The last availabel slot in the PyTypeObject, reserved up to now for future expansion, is used to optionally store a pointer to a new comparison function, of type 'richcmpfunc' defined by:: typedef PyObject *(*richcmpfunc) Py_PROTO((PyObject *, PyObject *, int)); This function takes three arguments. The first two are the objects to be compared, and the third is an integer corresponding to an opcode (one of LT, LE, EQ, NE, GT, GE). If this slot is left NULL, then rich comparison for that object type is not supported (except for class instances whose class provide the special methods described below). The above opcodes need to be added to the published Python/C API (probably under the names 'Py_LT', 'Py_LE', etc.) 2 Additions of special methods for classes Classes wishing to support the rich comparison mechanisms must add one or more of the following new special methods:: def __less__(self, other): ... def __less_equal__(self, other): ... def __greater__(self, other): ... def __greater__equal(self, other): ... def __equal__(self, other): ... def __not_equal__(self, other): ... Each of these is called when the class instance is the on the left-hand-side of the corresponding operators ('<', '<=', '>', '>=', '==', and '!=' or '<>'). The argument 'other' is set to the object on the right side of the operator. The return value of these methods is up to the class implementor (after all, that's the entire point of the proposal). If the object on the left side of the operator does not define an appropriate rich comparison operator (either at the C level or with one of the special methods, then the comparison is reversed, and the right hand operator is called with the *opposite* operator, and the two objects are swapped. This assumes that 'a < b' and 'b >= a' are equivalent, as are 'a > b' and 'b <= a', and that '==' and '!=' are commutative. For example, if 'obj1' is an object which supports the rich comparison protocol and x and y are objects which do not support the rich comparison protocol, then 'obj1 < x' will call the '__less__' method of 'obj1' with x as the second argument. 'x < obj1' will call obj1's '__greater_equal__' method with x as a second argument, and 'x < y' will just use the existing (non-rich) comparison mechanism. The above mechanism is such that classes can get away with not implementing either '__less__' and '__less_equal' or '__greater__' and '__greater_equal__'. Further smarts could have been added to the comparison mechanism, but this limited set of allowed "swaps" was chosen because it doesn't require the infrastructure to do any processing (negation) of return values. The choice of six special methods was made over a single (e.g. '__richcmp__') method to allow the dispatching on the opcode to be performed at the level of the C implementation rather than the user-defined method. 3 Addition of an optional argument to the builtin 'cmp()' The builtin cmp() is still used for simple comparisons. For rich comparisons, it is called with a third argument, one of "<", "<=", ">", ">=", "==", "!=", "<>" (the last two have the same meaning). When called with one of these strings as the third argument, cmp() can return any Python object. Otherwise, it can only return -1, 0 or 1 as before. Consequence Regarding Chained Comparisons Problem Objects for which the comparison returns something other than -1, 0, or 1 may be used in chained comparisons, such as:: x < y > z This is interpreted by Python as:: if x < y: if y > z: return y > z # 1 for plain comparisons else: return y > z # 0 for plain comparisons else: return x < y # 0 for plain comparisons Note that this requires testing the truth value of the result of comparisons, with potential "shortcutting" of the right-side comparison testings. In other words, the truth-value of the result of a comparison determines the result of a chained operation. This is problematic in the case of arrays, since if 'x', 'y' and 'z' are three arrays, then the user expects:: x < y < z to be an array of 0's and 1's where 1's are in the locations corresponding to the elements of 'y' which are between the corresponding elements in 'x' and 'z'. In other words, the right-hand side must be evaluated regardless of the result of 'x < y', which is incompatible with the mechanism currently in use by the parser. Solution If comparisons return objects which raise an exception when their truth condition is being queried, then no user code will yield surprising results. The use of helper functions (e.g. 'chain_cmp(x, "<", y, ">", z)') is needed to fulfill the required functionality. Patches I have patches against Python 1.5 which implement this proposal (modulo bugs). The following files were modified: * Include/object.h * Objects/object.c * Python/bltinmodule.c * Python/ceval.c I also have patches which allow the use of this feature in NumPy, but I'll delay discussion of those until this issue is settled (I have bigger plans for NumPy =). Note that these patches are not what I propose for the final version -- for one there's at least one place where I'm not sure what the right behavior is, and for another I don't publish the opcodes in the Python API. That's trivial and left up to Guido to decide on the names. From David Ascher Mon Apr 6 23:42:37 1998 From: David Ascher (David Ascher) Date: Mon, 6 Apr 1998 15:42:37 -0700 (PDT) Subject: [Matrix-SIG] placing points in a grid... Message-ID: I have a function evaluated at a set of points ((p1x, p1y), (p2x, p2y), ...)). The pNx's are in an array px, the pNy's are in an array py. I wish to 'place' these in a zero-filled grid I for display. The grid is a PxQ grid where the X axis corresponds to an array of x values xs, and the Y axis to an array of y value ys. I can do this by: datapoints = func(px, py[:,NewAxis]) for i in range(len(px)): x = px[i] y = py[i] xi = searchsorted(x, xs) yi = searchsorted(y, ys) I[xy,yi] = datapoints From bsd@scripps.edu Tue Apr 7 00:23:14 1998 From: bsd@scripps.edu (Bruce Duncan) Date: Mon, 6 Apr 1998 16:23:14 -0700 (PDT) Subject: [Matrix-SIG] Numeric arrays / assertions / truth testing Message-ID: Greetings, Is it documented how Numeric treats assertions, e.g., assert myarray The behavior, although consistent, is somewhat counter-intuitive: Numeric checks for identity to zero. I suggest an exposition of this in the next round of documentation. Python 1.5 (#63, Mar 11 1998, 14:53:33) [C] on irix5 Copyright 1991-1995 Stichting Mathematisch Centrum, Amsterdam >>> import Numeric >>> a = Numeric.array([1,2,3.0]) >>> assert a >>> b = Numeric.array([0.0,0.0]) >>> assert b Traceback (innermost last): File "", line 1, in ? AssertionError >>> len(a) 3 >>> len(b) 2 I was using assert myarray when I meant assert myarray != None -bsd- Bruce Duncan The Scripps Research Institute bsd "is located at" scripps "in the domain" edu # anti spam... From da@skivs.ski.org Mon Apr 6 23:42:55 1998 From: da@skivs.ski.org (David Ascher) Date: Mon, 6 Apr 1998 15:42:55 -0700 (PDT) Subject: [Matrix-SIG] argh.. Message-ID: ignore that last message -- it was sent by mistake. From da@skivs.ski.org Tue Apr 7 03:59:25 1998 From: da@skivs.ski.org (David Ascher) Date: Mon, 6 Apr 1998 19:59:25 -0700 (PDT) Subject: [Matrix-SIG] Numeric arrays / assertions / truth testing In-Reply-To: Message-ID: On Mon, 6 Apr 1998, Bruce Duncan wrote: > Is it documented how Numeric treats assertions, e.g., > > assert myarray > > The behavior, although consistent, is somewhat counter-intuitive: > Numeric checks for identity to zero. I suggest an exposition of > this in the next round of documentation. Actually, one consequence of my proposal on comparison is that rich arrays should not be truth-testable (and should raise exceptions), if one uses the most basic plan of returning arrays on comparisons. There are other, more complex possibilities. Does anyone currently rely on the truth of an array corresponding to it having one or more non-zero elements? --david From miller5@uiuc.edu Thu Apr 9 20:52:27 1998 From: miller5@uiuc.edu (Mike Miller) Date: 09 Apr 1998 14:52:27 -0500 Subject: [Matrix-SIG] TableIO - it's I/O for tables Message-ID: A month or so ago, I sent a message to matrix-sig with a thing I wrote called arrayio. I've improved it (I think) by removing the limits on row and column number and made it somewhat more flexible in that the base routines deal with lists rather than arrays. Plus I renamed it TableIO this time around - since I use it most often to read and write rectangular data tables I thought that was more descriptive. If you're interested, you can find it at . I am interested in hearing what people think about expanding on this to make a more general table module. At some level this would be a duplication of the work that has already gone into various data-base and data-format modules. On the other hand, this simple module has been very useful. With fast list/array reading I've moved from fiddling with Python now and then to using it frequently for serious data analysis. In fact, I've also put together a collection of histograming and graphing modules that, while not exactly polished, are sufficiently useful that I'm thinking of taking on the challenge made on the linux-hep list a while ago - building a complete analysis tool with Python. If any of you are interested in seeing such a thing, please let me know. Regards, Mike -- Michael A. Miller miller5@uiuc.edu Department of Physics, University of Illinois, Urbana-Champaign PGP public key available on request From miller5@uiuc.edu Fri Apr 10 15:18:50 1998 From: miller5@uiuc.edu (Mike Miller) Date: 10 Apr 1998 09:18:50 -0500 Subject: [Matrix-SIG] TableIO - it's I/O for tables In-Reply-To: David Ascher's message of Thu, 9 Apr 1998 17:10:53 -0700 (PDT) References: Message-ID: > Python is currently case insensitive on Windows. That's what I thought you meant :) If I change tableio.c to _tableio.c, I'd need to change the routine names to _tableio_blah as well. Is that right? Also, does the underscore affect the accessibility of the module? (other than by avoiding case sensitivity problems I mean) Mike From miller5@uiuc.edu Fri Apr 10 15:19:57 1998 From: miller5@uiuc.edu (Mike Miller) Date: 10 Apr 1998 09:19:57 -0500 Subject: [Matrix-SIG] TableIO - it's I/O for tables In-Reply-To: Harald Singer's message of Fri, 10 Apr 1998 11:03:18 +0900 References: <9804100203.AA16660@atrh16.itl.atr.co.jp> Message-ID: I'm not familiar with HTMLgen. Would you send a pointer? From miller5@uiuc.edu Fri Apr 10 15:24:56 1998 From: miller5@uiuc.edu (Mike Miller) Date: 10 Apr 1998 09:24:56 -0500 Subject: [Matrix-SIG] TableIO - it's I/O for tables In-Reply-To: Janko Hauser's message of Fri, 10 Apr 1998 11:34:47 +0200 (CEST) References: Message-ID: Thanks for the fixes, Janko. I'll take a look at the rows key and see if I can sort out what's happening with the trailing empty lines. By empty lines, do you mean lines that are only whitespace and a newline? That's something that I hadn't considered yet. The problem may go away if you put a comment character on those lines. Also, thanks very much for the Setup-file. I'm away from the machine where I have the code today, but I'll work that in over the weekend and send a message when it's done. Mike -- Michael A. Miller miller5@uiuc.edu Department of Physics, University of Illinois, Urbana-Champaign PGP public key available on request From miller5@uiuc.edu Mon Apr 13 19:51:16 1998 From: miller5@uiuc.edu (Mike Miller) Date: 13 Apr 1998 13:51:16 -0500 Subject: [Matrix-SIG] "data" class? Message-ID: More and more frequently I've been coming up against problems that make me wish I had a general class to hold a collection of data pairs, represented by two NumPpy arrays, along with associated information such as names, descriptions and units. This would be something like a netCDF file or an HDF vdata or a curve in Gist. I'd like to have such a creature so that I can easily plot a set of data using default labels and ranges and so I can do calculations that take units into account. I can even imagine interfaces to netCDF and HDF that are subclasses of such a class. It could even be used as an object for uniform access to various plotting libraries and databases. Has anyone else thought about such a thing? Or does any one know of refs that might help me reach a sensible design? Mike -- Michael A. Miller miller5@uiuc.edu Department of Physics, University of Illinois, Urbana-Champaign PGP public key available on request From just@letterror.com Wed Apr 15 18:38:32 1998 From: just@letterror.com (Just van Rossum) Date: Wed, 15 Apr 1998 19:38:32 +0200 Subject: [Matrix-SIG] matrix transformations on vector graphics Message-ID: Hello, I am very new to NumPy, and I would like some tips on how to efficiently do transformations for vector graphics (I mean, NumPy just screams "use me!" to me for this kind of stuff ;-). Say, I have an array of 2D coordinates: v = Numeric.array([(0.0, 0.0), (10.0, 0.0), (100.0, 75.0), etc...]) and a simple transformation matrix (this one should flip the y coordinate): tm = Numeric.array([[1.0, 0, 0], [0, -1.0, 0], [0, 0, 1.0]]) Is there a convenient way to apply the matrix to the vector array? If I turn the x and y values into separate arrays, I could probably do something like this: [x, y] = Numeric.transpose(v) xnew = tm[0][0] * x + tm[0][1] * y + tm[2][0] ynew = tm[1][0] * x + tm[1][1] * y + tm[2][1] But if I in general would prefer xy pairs, I would have to do Numeric.transpose() before and after I do this. Or is transpose() relatively cheap? Did I just answer my own question? What more introduction material is there besides David Ascher's wonderful pages and the stuff that comes with NumPy? Thanks in advance for any pointers or tips. Just From da@skivs.ski.org Wed Apr 15 18:48:51 1998 From: da@skivs.ski.org (David Ascher) Date: Wed, 15 Apr 1998 10:48:51 -0700 (PDT) Subject: [Matrix-SIG] matrix transformations on vector graphics In-Reply-To: Message-ID: On Wed, 15 Apr 1998, Just van Rossum wrote: > I am very new to NumPy, and I would like some tips on how to efficiently do > transformations for vector graphics (I mean, NumPy just screams "use me!" > to me for this kind of stuff ;-). Indeed -- there was a discussion on similar topics a few months back -- check the findmail archives re: use of complex numbers for graphics. > Say, I have an array of 2D coordinates: > > v = Numeric.array([(0.0, 0.0), (10.0, 0.0), (100.0, 75.0), etc...]) > > and a simple transformation matrix (this one should flip the y coordinate): > > tm = Numeric.array([[1.0, 0, 0], [0, -1.0, 0], [0, 0, 1.0]]) > > Is there a convenient way to apply the matrix to the vector array? Why do you define a 3x3 array if you're just using 2d coordinates? I'd say tm = Numeric.array([[1,0],[0,-1]], Float) is closer to what you want. And then, you can just use the 'dot()' function: >>> Numeric.dot(v, tm) array([[ 0., 0.], [ 10., 0.], [ 100., -75.]]) > But if I in general would prefer xy pairs, I would have to do > Numeric.transpose() before and after I do this. Or is transpose() > relatively cheap? Did I just answer my own question? transpose is very cheap, since it doesn't move any of the data, just the description of the data. > What more introduction material is there besides David Ascher's wonderful > pages and the stuff that comes with NumPy? Nothing yet -- actually, that's not true -- I have an HTML version of my SPAM-6 tutorial, which I keep meaning to put on the web. It's not great (after all, you don't get my wonderful presentation, just the supporting material), but it is more up to date than the tutorial. I'll try and do that sometime soon... --david From just@letterror.com Wed Apr 15 19:04:20 1998 From: just@letterror.com (Just van Rossum) Date: Wed, 15 Apr 1998 20:04:20 +0200 Subject: [Matrix-SIG] matrix transformations on vector graphics In-Reply-To: References: Message-ID: At 10:48 -0700 4/15/98, David Ascher wrote: >Indeed -- there was a discussion on similar topics a few months back -- >check the findmail archives re: use of complex numbers for graphics. I'll do that, thanks. >Why do you define a 3x3 array if you're just using 2d coordinates? > >I'd say > tm = Numeric.array([[1,0],[0,-1]], Float) > >is closer to what you want. For this particular one, yes. But I'd like to to PostScript-like transformations, which can also translate. A 2x3 would suffice, but then it isn't a matrix anymore, and I'd like to use matrixmultiply... >And then, you can just use the 'dot()' function: > > >>> Numeric.dot(v, tm) > array([[ 0., 0.], > [ 10., 0.], > [ 100., -75.]]) That's cool. I see that perhaps I'd better stick with 2x2 and do the translation separate: that should easy enough. Thanks again, Just From just@letterror.com Wed Apr 15 19:22:19 1998 From: just@letterror.com (Just van Rossum) Date: Wed, 15 Apr 1998 20:22:19 +0200 Subject: [Matrix-SIG] matrix transformations on vector graphics In-Reply-To: References: Message-ID: >Indeed -- there was a discussion on similar topics a few months back -- >check the findmail archives re: use of complex numbers for graphics. Hmm, the sig-search engine doesn't return anything for this string :-(. I found one innocent query about NumPy and graphics from 96, which got one response from Jim Hugunin. Could it have been under a slightly misleading subject line? I've browsed the archives back to 96: no luck... Just From David Ascher Wed Apr 15 19:33:48 1998 From: David Ascher (David Ascher) Date: Wed, 15 Apr 1998 11:33:48 -0700 (PDT) Subject: [Matrix-SIG] matrix transformations on vector graphics In-Reply-To: Message-ID: On Wed, 15 Apr 1998, Just van Rossum wrote: > >Indeed -- there was a discussion on similar topics a few months back -- > >check the findmail archives re: use of complex numbers for graphics. > > Hmm, the sig-search engine doesn't return anything for this string :-(. > I found one innocent query about NumPy and graphics from 96, which got > one response from Jim Hugunin. Could it have been under a slightly > misleading subject line? I've browsed the archives back to 96: no luck... That's because it was a discussion on the main list, not on the sig (they all blur): See the thread starting at: http://x10.dejanews.com/getdoc.xp?AN=289883206&search=thread&threaded=1&CONTEXT=892665211.898957355&HIT_CONTEXT=892665188.892600837&HIT_NUM=1&hitnum=3&NTL=1 [I hope this URL is permanent -- otherwise, search for "quaternions" and follow the "Numpy novice questions" thread back up to Harri's original post]. --david From Warren Focke Wed Apr 15 20:59:22 1998 From: Warren Focke (Warren Focke) Date: Wed, 15 Apr 1998 15:59:22 -0400 (EDT) Subject: [Matrix-SIG] matrix transformations on vector graphics In-Reply-To: Message-ID: On Wed, 15 Apr 1998, David Ascher wrote: > On Wed, 15 Apr 1998, Just van Rossum wrote: > > > > Is there a convenient way to apply the matrix to the vector array? > ... > And then, you can just use the 'dot()' function: > > >>> Numeric.dot(v, tm) > array([[ 0., 0.], > [ 10., 0.], > [ 100., -75.]]) > ... > > If I turn the x and y values into separate arrays, I could probably do > > something like this: > > > > [x, y] = Numeric.transpose(v) > > xnew = tm[0][0] * x + tm[0][1] * y + tm[2][0] > > ynew = tm[1][0] * x + tm[1][1] * y + tm[2][1] > > > > But if I in general would prefer xy pairs, I would have to do > > Numeric.transpose() before and after I do this. Or is transpose() > > relatively cheap? Did I just answer my own question? > > transpose is very cheap, since it doesn't move any of the data, just the > description of the data. > But PyArray_ContiguousFromObject eventually gets called on both arguments to Numeric.dot. Does this not copy the data of transposed arrays, negating the ``cheapness'' in this case (on the input side, at least)? Warren Focke From cgw@pgt.com Wed Apr 15 21:01:39 1998 From: cgw@pgt.com (Charles G Waldman) Date: Wed, 15 Apr 1998 16:01:39 -0400 (EDT) Subject: [Matrix-SIG] matrix transformations on vector graphics In-Reply-To: References: Message-ID: <13621.3892.552730.986077@janus.pgt.com> > > >And then, you can just use the 'dot()' function: > > > > >>> Numeric.dot(v, tm) > > array([[ 0., 0.], > > [ 10., 0.], > > [ 100., -75.]]) > > That's cool. I see that perhaps I'd better stick with 2x2 and do the > translation separate: that should easy enough. > There's no reason that the translations have to be handled separately. You just encode the point (x,y) as (x,y,1) and use matrices of the form a b e c d f 0 0 1 The a,b,c,d part is a normal 2x2 matrix and e,f is the offset. This is an old trick called "projective coordinates". You can still use the "dot" function as David suggests. This is superior because you can use matrix multiplication to generate matrices representing composite transformations - this way all of your manipulations are matrix multiplies, rather than multiplies and adds. If you do this, make sure to use the matrix_multiply, rather than the default "elementwise" multiplication of arrays, (a NumPy pitfall that's too easy to fall into). From da@skivs.ski.org Wed Apr 15 21:07:26 1998 From: da@skivs.ski.org (David Ascher) Date: Wed, 15 Apr 1998 13:07:26 -0700 (PDT) Subject: [Matrix-SIG] matrix transformations on vector graphics In-Reply-To: Message-ID: On Wed, 15 Apr 1998, Warren Focke wrote: > On Wed, 15 Apr 1998, David Ascher wrote: > > > On Wed, 15 Apr 1998, Just van Rossum wrote: > > > > > > Is there a convenient way to apply the matrix to the vector array? > > > ... > > And then, you can just use the 'dot()' function: > > > > >>> Numeric.dot(v, tm) > > array([[ 0., 0.], > > [ 10., 0.], > > [ 100., -75.]]) > > > ... > > > If I turn the x and y values into separate arrays, I could probably do > > > something like this: > > > > > > [x, y] = Numeric.transpose(v) > > > xnew = tm[0][0] * x + tm[0][1] * y + tm[2][0] > > > ynew = tm[1][0] * x + tm[1][1] * y + tm[2][1] > > > > > > But if I in general would prefer xy pairs, I would have to do > > > Numeric.transpose() before and after I do this. Or is transpose() > > > relatively cheap? Did I just answer my own question? > > > > transpose is very cheap, since it doesn't move any of the data, just the > > description of the data. > > But PyArray_ContiguousFromObject eventually gets called on both arguments > to Numeric.dot. Does this not copy the data of transposed arrays, > negating the ``cheapness'' in this case (on the input side, at least)? Good point, but note that I didn't say 'dot()' was cheap, I said 'transpose()' was cheap. Whenever you're doing an 'en masse' computation, I believe that before the operation (be it innerproduct, elementwise multiplication, addition, etc.), a contiguous version of the array is obtained -- the argument being that the cost of the eventual memory relocation is offset by the speedup of the inner loop accessing sequential memory locations. A quick glance at the code seems to confirm that this happens. I'd guess that Jim did some testing to verify that it is efficient. --david 'the group is alive! it's alive!' ascher From hinsen@ibs.ibs.fr Thu Apr 16 16:19:08 1998 From: hinsen@ibs.ibs.fr (Konrad Hinsen) Date: Thu, 16 Apr 1998 17:19:08 +0200 Subject: [Matrix-SIG] precision of numbers In-Reply-To: <199804031003.LAA02579@oak.sucs.soton.ac.uk> (dars@soton.ac.uk) Message-ID: <199804161519.RAA10166@lmspc1.ibs.fr> > Is there another number type, or is there a precision parameter in Numpy, > or can I fool it with a simple trick(I can think of one for my particular > problem, but howabout in general)??? All answers gratefully received. NumPy arrays by default use the C type "double". There is no guarantee that a larger float type exists on any machine, so NumPy doesn't offer "long double" arrays. So if you really need more precision (i.e. if you can't solve your problem with another algorithm), you'd have to use an arbitrary-precision data type. There is at least one implementation available at ftp.python.org, and you can use such a type in a general-object array to get array behaviour. But be warned that this will be *extremely* slow. -- ------------------------------------------------------------------------------- Konrad Hinsen | E-Mail: hinsen@ibs.ibs.fr Laboratoire de Dynamique Moleculaire | Tel.: +33-4.76.88.99.28 Institut de Biologie Structurale | Fax: +33-4.76.88.54.94 41, av. des Martyrs | Deutsch/Esperanto/English/ 38027 Grenoble Cedex 1, France | Nederlands/Francais ------------------------------------------------------------------------------- From hinsen@ibs.ibs.fr Thu Apr 16 16:26:25 1998 From: hinsen@ibs.ibs.fr (Konrad Hinsen) Date: Thu, 16 Apr 1998 17:26:25 +0200 Subject: [Matrix-SIG] functions for numbers & arrays In-Reply-To: (message from David Ascher on Thu, 2 Apr 1998 16:31:58 -0800 (PST)) Message-ID: <199804161526.RAA10188@lmspc1.ibs.fr> > work for both arrays and lists. For example, I have functions which I > want to evaluate at grids of parameter values (x = arange(2,4,.2), > y=arange(1,2,.01)) and specific points (x = 2, y = 3). > > I haven't come up with a good way to do this that does the right thing in > most circumstances. [for example, I'd like such a function to return an > array in the first case, but a scalar in the second] There are currently two cases. If your function can be written entirely in terms of umath functions, everything will work as expected automatically. Otherwise, you'll need explicit type checks, unfortunately. > What do other array languages do to address this problem? Anything we can > steal? Other array languages treat scalars as rank-0 arrays, not as a separate data type. We don't have that option for compatibility reasons. What could be done is a careful modification of both scalar types and NumPy arrays to make them all behave as similarly as possible. For example, Python scalars could respond to array methods such as shape(), typecode(), astype(), etc. However, even with "real" array languages such as APL, designing a function such that it will do the right thing for both scalar and array arguments is not always trivial. In many cases it requires adapting the shapes of input arguments and results. But at least it can all be done without explicit type checks. -- ------------------------------------------------------------------------------- Konrad Hinsen | E-Mail: hinsen@ibs.ibs.fr Laboratoire de Dynamique Moleculaire | Tel.: +33-4.76.88.99.28 Institut de Biologie Structurale | Fax: +33-4.76.88.54.94 41, av. des Martyrs | Deutsch/Esperanto/English/ 38027 Grenoble Cedex 1, France | Nederlands/Francais ------------------------------------------------------------------------------- From hinsen@ibs.ibs.fr Thu Apr 16 16:29:57 1998 From: hinsen@ibs.ibs.fr (Konrad Hinsen) Date: Thu, 16 Apr 1998 17:29:57 +0200 Subject: [Matrix-SIG] "data" class? In-Reply-To: (message from Mike Miller on 13 Apr 1998 13:51:16 -0500) Message-ID: <199804161529.RAA10255@lmspc1.ibs.fr> > More and more frequently I've been coming up against problems > that make me wish I had a general class to hold a collection of > data pairs, represented by two NumPpy arrays, along with > associated information such as names, descriptions and units. I guess a more general and thus more useful class would be a "table" of data. Ideally it's functionality would be that of a spreadsheet. That's overkill, of course, but I think spreadsheets could serve as a design guideline anyway. The big problem is defining how operations on such an object should be performed. For example, multiplication by a scalar could mean multiplying everything, or just specific rows or columns, depending on the interpretation of the data. Maybe different classes would be necessary. This is definitely a worthy project, but not a trivial one... -- ------------------------------------------------------------------------------- Konrad Hinsen | E-Mail: hinsen@ibs.ibs.fr Laboratoire de Dynamique Moleculaire | Tel.: +33-4.76.88.99.28 Institut de Biologie Structurale | Fax: +33-4.76.88.54.94 41, av. des Martyrs | Deutsch/Esperanto/English/ 38027 Grenoble Cedex 1, France | Nederlands/Francais ------------------------------------------------------------------------------- From hinsen@ibs.ibs.fr Thu Apr 16 16:26:25 1998 From: hinsen@ibs.ibs.fr (Konrad Hinsen) Date: Thu, 16 Apr 1998 17:26:25 +0200 Subject: [Matrix-SIG] functions for numbers & arrays In-Reply-To: (message from David Ascher on Thu, 2 Apr 1998 16:31:58 -0800 (PST)) Message-ID: <199804161526.RAA10188@lmspc1.ibs.fr> > work for both arrays and lists. For example, I have functions which I > want to evaluate at grids of parameter values (x = arange(2,4,.2), > y=arange(1,2,.01)) and specific points (x = 2, y = 3). > > I haven't come up with a good way to do this that does the right thing in > most circumstances. [for example, I'd like such a function to return an > array in the first case, but a scalar in the second] There are currently two cases. If your function can be written entirely in terms of umath functions, everything will work as expected automatically. Otherwise, you'll need explicit type checks, unfortunately. > What do other array languages do to address this problem? Anything we can > steal? Other array languages treat scalars as rank-0 arrays, not as a separate data type. We don't have that option for compatibility reasons. What could be done is a careful modification of both scalar types and NumPy arrays to make them all behave as similarly as possible. For example, Python scalars could respond to array methods such as shape(), typecode(), astype(), etc. However, even with "real" array languages such as APL, designing a function such that it will do the right thing for both scalar and array arguments is not always trivial. In many cases it requires adapting the shapes of input arguments and results. But at least it can all be done without explicit type checks. -- ------------------------------------------------------------------------------- Konrad Hinsen | E-Mail: hinsen@ibs.ibs.fr Laboratoire de Dynamique Moleculaire | Tel.: +33-4.76.88.99.28 Institut de Biologie Structurale | Fax: +33-4.76.88.54.94 41, av. des Martyrs | Deutsch/Esperanto/English/ 38027 Grenoble Cedex 1, France | Nederlands/Francais ------------------------------------------------------------------------------- From da@skivs.ski.org Mon Apr 20 22:42:00 1998 From: da@skivs.ski.org (David Ascher) Date: Mon, 20 Apr 1998 14:42:00 -0700 (PDT) Subject: [Matrix-SIG] rich comparisons, version 0.2 Message-ID: Since I got a little bit of feedback, I'm putting up a revised version of the proposal which would allow you to do: data[data > 255] = 255 instead of data = where(umath.greater(data, 255), 255, data) up on my newly redesigned website: http://starship.skyport.net/~da/proposals/richcmp.html Changes: - folks prefered the fortran/perl names (__le__ instead of __less_equal__) by an overwhelming margin ((guido, tim_one) > da) - more discussion of how to save chained comparisons. Followup-To: main list. From just@letterror.com Thu Apr 23 13:59:09 1998 From: just@letterror.com (Just van Rossum) Date: Thu, 23 Apr 1998 14:59:09 +0200 Subject: [Matrix-SIG] Yet another newbie question Message-ID: Hullo, First of all, thank you for your previous help, especially David. I am now successfully transforming vector arrays at an incredible speed, with an incredible ease! I am in awe. Anyway, I have one more question. I want to quickly find the indices of array elements that are true for a certain expression/condition. More specifically: I have an array containing two dimensional coordinates, and I'd like to find the indices of those coordinates that are within a specified rectangle. I have the feeling that Numeric.where() might do something I could use, but I don't understand the explanations in either the official doc nor in David's tutorial :-(. ... a while later... I think I understand it slightly better now. where() that is. Still, what I've just coded up feels clumsy: from Numeric import * # some coordinates a = array([[ 163., -28.], [ 152., -28.], [ 117., -23.], [ 88., 0.], [ 34., 43.], [ 16., 123.], [ 16., 180.], [ 16., 201.], [ 18., 217.], [ 21., 227.]],'f') # the rectangle I'd like to match lefttop = [30, 40] rightbottom = [40, 50] condition = logical_and(greater(a, lefttop), less(a, rightbottom)) w = where(condition, 1, 0) print w Now w is an array containing [0, 0] pairs for mismatches, and [1, 1] for matches. But I still have to iterate to get to the indices... Is there something more convenient/efficient? Thanks in advance for any tips. Just From hinsen@ibs.ibs.fr Thu Apr 23 18:25:52 1998 From: hinsen@ibs.ibs.fr (Konrad Hinsen) Date: Thu, 23 Apr 1998 19:25:52 +0200 Subject: [Matrix-SIG] Yet another newbie question In-Reply-To: (message from Just van Rossum on Thu, 23 Apr 1998 14:59:09 +0200) Message-ID: <199804231725.TAA15792@lmspc1.ibs.fr> > ... a while later... I think I understand it slightly better now. > where() that is. Still, what I've just coded up feels clumsy: > > from Numeric import * > # some coordinates > a = array([[ 163., -28.], > [ 152., -28.], > [ 117., -23.], > [ 88., 0.], > [ 34., 43.], > [ 16., 123.], > [ 16., 180.], > [ 16., 201.], > [ 18., 217.], > [ 21., 227.]],'f') > # the rectangle I'd like to match > lefttop = [30, 40] > rightbottom = [40, 50] > > condition = logical_and(greater(a, lefttop), less(a, rightbottom)) > w = where(condition, 1, 0) > print w > > Now w is an array containing [0, 0] pairs for mismatches, and [1, 1] for > matches. But I still have to iterate to get to the indices... > Is there something more convenient/efficient? First of all you must reduce your condition to a 1D array. In your case, both the x and the y comparison must be true for the point to be in the box, so use: condition = logical_and(greater(a, lefttop), less(a, rightbottom)) condition = logical_and.reduce(condition, -1) To get the indices, use repeat() with an arange() list as the first argument: indices = repeat(arange(len(condition)), condition) That's it, no explicit loops! Konrad. -- ------------------------------------------------------------------------------- Konrad Hinsen | E-Mail: hinsen@ibs.ibs.fr Laboratoire de Dynamique Moleculaire | Tel.: +33-4.76.88.99.28 Institut de Biologie Structurale | Fax: +33-4.76.88.54.94 41, av. des Martyrs | Deutsch/Esperanto/English/ 38027 Grenoble Cedex 1, France | Nederlands/Francais ------------------------------------------------------------------------------- From matrix-sig-request@python.org Thu Apr 23 19:09:55 1998 From: matrix-sig-request@python.org (matrix-sig-request@python.org) Date: Thu, 23 Apr 1998 14:09:55 -0400 (EDT) Subject: [Matrix-SIG] Subscribing to Matrix-SIG Message-ID: <199804231809.OAA06880@python.org> A request for subscription of your address to the Matrix-SIG@python.org mailing list has been received via the web, from oobleck.tn.cornell.edu. This is a confirmation request, to prevent anyone from subscribing you against your wishes. In order to complete this subscription you must send a confirming email, to matrix-sig-request@python.org, by repling to this mail, and including just the line: subscribe jh-matsig-passwd in the body or as the subject line. If you do not actually wish to subscribe you need not do anything. Upon subscribing you will receive a message welcoming you to the list and describing how to tailor your account. Questions or comments? Send them to mailman-owner@python.org From klm@python.org Thu Apr 23 19:34:59 1998 From: klm@python.org (Ken Manheimer) Date: Thu, 23 Apr 1998 14:34:59 -0400 (EDT) Subject: [Matrix-SIG] Re: Hmmm... (Subscribing to Matrix-SIG) In-Reply-To: Message-ID: You may have noticed a kind of odd matrix-sig subscription attempt notice show up in your mailboxes - not to worry. Someone accidentally put the matrix-sig as their subscription address when trying a web-based subscribe, so mailman diligently sent subscription instructions to the list. There's some relatively simple things we could do to prevent such a thing in the future, but it hasn't happened particularly frequently, thus far, so i'm going to let it be. BUT THERE'S NO CAUSE FOR CONCERN! DON'T WORRY! BE CALM! WHY AM I SHOUTING?!-) Ken Manheimer klm@python.org 703 620-8990 x268 (orporation for National Research |nitiatives # If you appreciate Python, consider joining the PSA! # # . # From managan@llnl.gov Thu Apr 30 20:55:33 1998 From: managan@llnl.gov (Rob Managan) Date: Thu, 30 Apr 1998 12:55:33 -0700 Subject: [Matrix-SIG] plplot on the Mac Message-ID: I am trying to get a final version of an updated macintosh plplot module to work with Python 1.5. Let me know if you are interested in getting a copy. I have a question for Mac users. The plplot library stores fonts and maps in a 'lib' folder and on unix boxes an environment variable is set to tell the library where to find the files. My thinking is to use the Preferences folder in some way. Would it be better to have a preference file that stores the environment variables (similar to what pgplot does) or just move the 'lib' file into the Preferences folder and look for them there? One problem is keeping the file updated if the user moves the 'lib' file to another location on the disk. Either approach requires you to put something into the Preferences folder. This is not as user friendly as I would like but is better than losing the text in the plots! *-*-*-*-*-*-*-*-*-*-**-*-*-*-*-*-*-*-*-*-*- Rob Managan mailto://managan@llnl.gov LLNL ph: 925-423-0903 P.O. Box 808, L-183 FAX: 925-423-5804 Livermore, CA 94551-0808 From just@letterror.com Thu Apr 30 21:11:47 1998 From: just@letterror.com (Just van Rossum) Date: Thu, 30 Apr 1998 22:11:47 +0200 Subject: [Matrix-SIG] Re: [Pythonmac-SIG] plplot on the Mac In-Reply-To: Message-ID: At 12:55 -0700 4/30/98, Rob Managan wrote: >I am trying to get a final version of an updated macintosh plplot module to >work with Python 1.5. Let me know if you are interested in getting a copy. > >I have a question for Mac users. The plplot library stores fonts and maps >in a 'lib' folder and on unix boxes an environment variable is set to tell >the library where to find the files. My thinking is to use the Preferences >folder in some way. Would it be better to have a preference file that >stores the environment variables (similar to what pgplot does) or just move >the 'lib' file into the Preferences folder and look for them there? One >problem is keeping the file updated if the user moves the 'lib' file to >another location on the disk. > >Either approach requires you to put something into the Preferences folder. >This is not as user friendly as I would like but is better than losing the >text in the plots! I'd say, use the Python approach, which is to put a file containing an alias pointing to the lib into the preferences folder. That way, if the user moves the lib folder to a different location on the same volume, the alias will still point to the correct place. See the Python sources and EditPythonPrefs.py for examples on how to do that. Just