From jjh@mama-bear.lcs.mit.edu Fri Dec 1 23:39:41 1995 From: jjh@mama-bear.lcs.mit.edu (James Hugunin) Date: Fri, 1 Dec 95 18:39:41 -0500 Subject: [PYTHON MATRIX-SIG] Release 0.20 of matrix object is available Message-ID: <9512012339.AA04760@ling-ling.lcs.mit.edu.LCS.MIT.EDU> I can't completely believe I finally finished this latest release. I'm heading off for the python workshop on Monday, and I wanted to get this out before I left. The whole package can be found at the same site as before, (if you don't know the site you need to send me mail saying that you agree to test and review the object and I'll give you the site). The NEWS for this release follows. The two major incompatible changes at the very beginning are why this release took so long (that and the fact that I have a few other demands on my time). Please let me know what you think of these changes! I finally feel good about the feature set and performance of the object. If nobody has any major complaints (or REALLY good suggestions like Yorick pseudo and rubber indices, curse them) then I don't expect the basic structure of the object to change much in the future. As before, don't hesitate to send me your gripes, your bugs, and your praise (if ever merited). However, I'll be gone for most of next week at the python workshop, so I might be slow in replying. -Jim ********** Release 0.20 of the matrix object Major incompatible changes (I expect that these might elict some interesting discussion): 1) Matrix indexing now returns by reference to the original data, not by value. As a side effect of this change, arbitrary sequences are not allowed in multidimensional indices, but only single indices, slices, RubberIndex and None. Note: there is a new method, take, which will allow you to index a matrix with an arbitrary sequence as before. This change was motivated both by the possible speed increases (about 40% for some code) and more importantly by issues of clarity of expression. I couldn't come up with any other way to make the following hold: m[0][1] is an efficient way to index a multi-dimensional array. m[0,1] == m[0][1] m[0,] == m[0] 2) No more ranks of operators, instead Yorick style pseudo indices are used. As a side effect of this, outer product is now a convenience function rather than a method on ofuncs. For functions of unbounded rank (currently the only kind of function supported by my omathmodule) Yorick pseudo indices support a superset of what was possible to express using ranks and outer products. There is no fundamental reason not to have both pseudo indices and ranks, I just think that it's conceptually cleaner not to mix the two up. Here's a fun thing you can do with pseudo indices that is a lot uglier with ranks and outer products: x=mrange(-1, 1, 2.0/m)[All,None] y=mrange(-1, 1, 2.0/n) r=sqrt(x**2+y**2) z=exp(-r**2)*cos(2*pi*r) This is modified from a piece of Tom Schwaller's code for producing a 2d surface plot. Notice that this is much more efficient than generating x and y as full 2d matrices. Major improvements: 1) This new release is about 40% faster than 0.11 (on a benchmark code I was given by the folks at LBL). 2) No longer need to say a[[1,2,3]] for multidimensional indexing, now a[1,2,3] works just fine (thanks Konrad!) 3) Type coercion is performed more or less according to the same rules that python uses. So I can add a matrix of ints to one of floats, and I wind up with a matrix of floats at the end. 4) Finally there's a matrixMultiply method for matrices that behaves reasonably for all combinations of matrix, vector, and scalar arguments. Minor improvements: 1) pickling of matrices works (and uses a binary format with endianness labeled for both efficiency and portability). 2) The omath module supports trig functions as object methods (if m has method sin, then I can take sin(m) and get the sine of m. This also works on matrices of objects. omathmodule can be used now a a replacement for cmathmodule and mathmodule. 3) transpose now does the appropriate arbitrary permutation of dimensions. 4) More matrix methods, and better implementations of existing ones. Notes 1) The code hasn't actually grown to more than double it's previous size. This is just the effect of including all of Konrad's patches to the python Grammar as complete files rather than diffs. Hopefully, this will be able to be cleanly seperated from the matrix object, but I just don't have the time to deal with that right now. Things I plan to fix ASAP 1) Integrate Chris Chase's grammar patches to replace Slice(None,None,2) with ::2. 2) Make matrixMultiply work on matrices larger than 2d (this requires a notion of rank). 3) Fix the memory leaks 4) Make some real documentation (I already have doc strings for all the matrix methods, I'm hoping to steal some code to convert this to "real" documentation at the workshop). ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From hinsenk@ere.umontreal.ca Sun Dec 3 16:59:56 1995 From: hinsenk@ere.umontreal.ca (Hinsen Konrad) Date: Sun, 3 Dec 1995 11:59:56 -0500 Subject: [PYTHON MATRIX-SIG] Release 0.20 of matrix object is available In-Reply-To: <9512012339.AA04760@ling-ling.lcs.mit.edu.LCS.MIT.EDU> (message from James Hugunin on Fri, 1 Dec 95 18:39:41 -0500) Message-ID: <199512031659.LAA21925@cyclone.ERE.UMontreal.CA> Major incompatible changes (I expect that these might elict some interesting discussion): OK, let's start ;-) 1) Matrix indexing now returns by reference to the original data, not by value. As a side effect of this change, arbitrary sequences are not allowed in multidimensional indices, but only single indices, slices, RubberIndex and None. I don't see any problem with this. Of course the documentation (to be written...) should be clear so that everyone knows the implications of this, but since Python variables in general hold references, it shouldn't be a surprise for anyone. This change was motivated both by the possible speed increases (about 40% for some code) and more importantly by issues of clarity of expression. I couldn't come up with any other way to make the following hold: m[0][1] is an efficient way to index a multi-dimensional array. m[0,1] == m[0][1] m[0,] == m[0] I don't see why these are important features, but of course they don't do any harm. 2) No more ranks of operators, instead Yorick style pseudo indices are used. As a side effect of this, outer product is now a convenience function rather than a method on ofuncs. As another side effect, we have lost some important functionality. For functions of unbounded rank (currently the only kind of function supported by my omathmodule) Yorick pseudo indices support a superset of what was possible to express using ranks and outer products. There is no fundamental reason not to have both pseudo indices and ranks, I just think that it's conceptually cleaner not to mix the two up. The pseudo indices are not a replacement for ranks, but just a convenient notation for some structural functions. I like this notation very much, but we still need ranks. And as you said, there is no reason not to have both. Apart from the problem of functions with finite rank (which we will certainly need e.g. for matrix inversion), it is not true that pseudo indiced support a superset of what is possible with ranks and the associated structural functions. In fact, it is just the opposite, as I will demonstrate. First let me explain how the new pseudo indices fit into the J-style array system. J doesn't have the concept of "indexing"; instead there are a few structural functions that can be applied with different ranks to construct all kinds of rearrangements of array elements. All the (pseudo) indices we currently have can be mapped onto these structural functions as follows: - Explicit numerical indices, lists, slices and the rubber index are equivalent to the function take() (see my Array.py) with various rank. - The pseudo-index "None" is equivalent to the J-function "itemize" that adds an axis of length one to its argument. I didn't implement this in Array.py, but it is a trivial addition. - The contracting rubber index (* in Yorick, I haven't found it yet in Python)is equivalent to the function ravel() with appropriate rank. As an example, the outer product mentioned above could be (inefficiently) implemented in my Array.py as: def outer(f, a, b): return f(reshape[0](a, shape(b)), b) So every construction with (pseudo) indices can be emulated with three structural functions of suitable rank. On the other hand, there are constructions that cannot be expressed with pseudo indices, for example all cases where a function rank is not a constant but a variable. In fact these can be handled in Python -- as opposed to Yorick -- by remembering that a[x,y,z] is really just a shorthand for a[(x,y,z)] and that the index tuple can be constructed from an expression, but that seems more like an implementation accident than a feature to me. More importantly, pseudo indices don't help in the least when it comes to reductions and general inner products that require the specification of an axis. Yorick does this by putting the function into the index list -- a clever idea, but I'd prefer to keep a clear distinction between functions and arguments. The current matrix module allows a second parameter to reduce() that indicates the axis, which is exactly how APL handles the problem. But both these methods are restrictive. They are sufficient only for reductions with functions of unbounded rank. I can give an example of an actual application where this is not enough: I had a sequence of n rotation matrices, stored in a rank-3 array D of shape (n, 3, 3). Then the net rotation of this sequence could be obtained by matrixMultiply.reduce(D). Since matrix multiplication is a rank-2 function, I couldn't express this with the current matrix module. The bottom line: pseudo indices are convenient shorthands for the most common structural functions, but they are no replacement for the general function rank scheme we had in version 0.11. And now for something completely different ------------------------------------------ 1) Names This may seem to be a minor issue, but we should tackle this before we all get used to the current ones. I strongly dislike the type-specific constructors (also used for output). They should be IntegerMatrix, FloatMatrix, ComplexMatrix, CharacterMatrix, and GeneralMatrix. Anyone is free to define shorter names for efficient typing, if desired. I even propose a more radical renaming. Many people associate "matrix" with the 2D-matrices from linear algebra. So it would be better to call our general objects "arrays", and leave the name "matrix" for linear-algebra type objects that are restricted to rank 2 and use * for matrix multiplication. They could be implemented in Python based on arrays. The pseudo-index 'None' is very confusing. An index "2" picks item number two from an axis, an index "All" picks all items from an axis. Consequently "None" should pick no item from an axis, which is course is a pointless operation. What it really does is create a new axis, so it should be named "New". "RubberIndex" is not confusing, but a bit strange, so it's worth thinking about alternatives. How about "Skip", in the sense "skip all following axes for explicit consideration"? For the "*" in Yorick I propose "Contract". 2) Constructors Currently there are two constructors, matrix() and Matrix(), with slightly different behaviour: matrix() takes a single argument that is a (possibly nested) Python sequence object, e.g. a list. Matrix() takes an arbitrary number of arguments that are made into a list before being passed to matrix(). So Matrix(1,2,3) is equivalent to matrix([1,2,3]). I find it confusing to have to constructors with almost the same name and almost the same function. I propose to have only one, of whatever name, which behaves like matrix(). The reason is that many matrix functions accept nested lists or tuples instead of matrix arguments, e.g. matrix([2,3]) + [5,4] works. So does matrix([2,3]) + matrix([5,4]), which is equivalent. But Matrix([2,3]) + [5,4] and Matrix([2,3]) + Matrix([5,4]) are not equivalent (well, they are in this simple exaple due to broadcasting, but not in general). So matrix() can be considered a conversion function from lists and tuples to arrays, which is needed anyway. I realize that the calling style of Matrix() has the advantage of saving one pair of brackets, but I don't consider this important enough to create the current confusion. I guess that's enough for today ;-) I wish the lucky participants of the workshop much fun and hope that they don't have to eat spam for lunch ;-) ------------------------------------------------------------------------------- Konrad Hinsen | E-Mail: hinsenk@ere.umontreal.ca Departement de chimie | Tel.: +1-514-343-6111 ext. 3953 Universite de Montreal | Fax: +1-514-343-7586 C.P. 6128, succ. Centre-Ville | Deutsch/Esperanto/English/Nederlands/ Montreal (QC) H3C 3J7 | Francais (phase experimentale) ------------------------------------------------------------------------------- ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From forrest@rose.rsoc.rockwell.com Mon Dec 4 19:25:11 1995 From: forrest@rose.rsoc.rockwell.com (Dave Forrest) Date: Mon, 4 Dec 1995 13:25:11 -0600 Subject: [PYTHON MATRIX-SIG] Renaming and Constructors Message-ID: <9512041925.AA00688@feynman.rsoc.rockwell.com> > From owner-matrix-sig@python.org Mon Dec 4 12:06 CST 1995 > Date: Sun, 3 Dec 1995 11:59:56 -0500 > From: hinsenk@ERE.UMontreal.CA (Hinsen Konrad) [snip] > > I strongly dislike the type-specific constructors (also used for > output). They should be IntegerMatrix, FloatMatrix, ComplexMatrix, > CharacterMatrix, and GeneralMatrix. Anyone is free to define shorter > names for efficient typing, if desired. > > I even propose a more radical renaming. Many people associate "matrix" > with the 2D-matrices from linear algebra. So it would be better to Yes!, Yes we do! (Some of us even thought that that was the reason for having a Matrix SIG) > call our general objects "arrays", and leave the name "matrix" for > linear-algebra type objects that are restricted to rank 2 and use * > for matrix multiplication. They could be implemented in Python based > on arrays. I thought that we had decided a while back to call it "Tensor" because "Array" was a more general, computer-science concept. [snip] > 2) Constructors > [snip] > I find it confusing to have to constructors with almost the same name > and almost the same function. I propose to have only one, of whatever > name, which behaves like matrix(). The reason is that many matrix > functions accept nested lists or tuples instead of matrix arguments, > e.g. matrix([2,3]) + [5,4] works. So does matrix([2,3]) + > matrix([5,4]), which is equivalent. But Matrix([2,3]) + [5,4] and > Matrix([2,3]) + Matrix([5,4]) are not equivalent (well, they are in > this simple exaple due to broadcasting, but not in general). So > matrix() can be considered a conversion function from lists and tuples > to arrays, which is needed anyway. I agree. David Forrest ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From hinsenk@ere.umontreal.ca Mon Dec 4 22:12:32 1995 From: hinsenk@ere.umontreal.ca (Hinsen Konrad) Date: Mon, 4 Dec 1995 17:12:32 -0500 Subject: [PYTHON MATRIX-SIG] Renaming and Constructors In-Reply-To: <9512041925.AA00688@feynman.rsoc.rockwell.com> (forrest@rose.rsoc.rockwell.com) Message-ID: <199512042212.RAA14614@cyclone.ERE.UMontreal.CA> I thought that we had decided a while back to call it "Tensor" because "Array" was a more general, computer-science concept. But what we have now corresponds to that concept. In contrary, a tensor is something completely different: a mathematical object with certain transformation behaviour under coordinate system transforms. ------------------------------------------------------------------------------- Konrad Hinsen | E-Mail: hinsenk@ere.umontreal.ca Departement de chimie | Tel.: +1-514-343-6111 ext. 3953 Universite de Montreal | Fax: +1-514-343-7586 C.P. 6128, succ. Centre-Ville | Deutsch/Esperanto/English/Nederlands/ Montreal (QC) H3C 3J7 | Francais (phase experimentale) ------------------------------------------------------------------------------- ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From stoll@atr-sw.atr.co.jp Tue Dec 5 08:23:21 1995 From: stoll@atr-sw.atr.co.jp (Perry A. Stoll) Date: Tue, 05 Dec 1995 17:23:21 +0900 Subject: [PYTHON MATRIX-SIG] A bug fix for release 0.20 Message-ID: <199512050823.RAA27076@ciris21.atr-sw.atr.co.jp> While Jim Hugunin is gone at the Python conference, I thought I'd share a bug fix with the list. Finding this bug was accidental (trying to cast a matrix of ints to types.ListType), but unavoidable (it core dumped). Here is a brief example which demonstrates this: import Matrix, types Matrix.mrange(1,5).cast(types.ListType) The fix is simple once you've found it ;-) It is only one line in the file Objects/matrixtypes.c. Here are the diffs: 564c566 < sizeof(PyObject), 'O', PyMatrix_OBJECT}; --- > sizeof(PyObject *), 'O', PyMatrix_OBJECT}; i.e. the size of entries in an ObjectMatrix should be the size of a pointer to a PyObject struct, not the size of a PyObject struct. And now for more poking around in the guts of the matrix lib... -Perry Stoll Research Associate, CSRL Advanced Telecommunications Research 2-2 Hikaridai, Seika-cho Soraku-gun, Kyoto 619-02 JAPAN VOICE: +81-774-95-1217 FINGER: stoll@atrwide.atr.co.jp FAX : +81-774 95-1208 EMAIL: stoll@atr-sw.atr.co.jp PGP 2.6 Key fingerprint = AF 56 5C D8 5F 78 BA FD 21 6E 2A 68 C4 55 9E B0 ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From mclay@eeel.nist.gov Tue Dec 5 13:28:54 1995 From: mclay@eeel.nist.gov (Michael McLay) Date: Tue, 5 Dec 95 08:28:54 EST Subject: [PYTHON MATRIX-SIG] Re: Python vs Perl: benchmarking for speed? In-Reply-To: References: <496if1$8hg@mrnews.mro.dec.com> <49m6j5$m7v@mrnews.mro.dec.com> <49of3l$86g@csnews.cs.colorado.edu> Message-ID: <9512051328.AA03465@acdc.eeel.nist.gov> Hinsen Konrad (hinsenk@cyclone.ERE.UMontreal.CAwrites: > It seems to me that the argument you replied to was a purely > hypothetical one, to illustrate the usefulness of tabulated > benchmarks. Noone suggests that Perl takes ten times more time > for a given problem than Python. In fact, I would be very much > surprised if the difference between Perl and Python would ever > reach a factor of ten in any direction for any problem. What an invitation. How about doing that benchmark comparison of Python matrix math vs. Perl matrix math that the world has been waiting for. I can't wait to see the results:-) Michael ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From forrest@rose.rsoc.rockwell.com Tue Dec 5 13:41:53 1995 From: forrest@rose.rsoc.rockwell.com (Dave Forrest) Date: Tue, 5 Dec 1995 07:41:53 -0600 Subject: [PYTHON MATRIX-SIG] Re: Python vs Perl: benchmarking for speed? Message-ID: <9512051341.AA03099@feynman.rsoc.rockwell.com> > From owner-matrix-sig@python.org Tue Dec 5 07:35 CST 1995 [snip] > > Hinsen Konrad (hinsenk@cyclone.ERE.UMontreal.CAwrites: > > It seems to me that the argument you replied to was a purely > > hypothetical one, to illustrate the usefulness of tabulated > > benchmarks. Noone suggests that Perl takes ten times more time > > for a given problem than Python. In fact, I would be very much > > surprised if the difference between Perl and Python would ever > > reach a factor of ten in any direction for any problem. > > What an invitation. How about doing that benchmark comparison of > Python matrix math vs. Perl matrix math that the world has been > waiting for. I can't wait to see the results:-) IMHO, this "dialogue" is not relevant to the Matrix discussion. Please take it elsewhere. David Forrest ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From jjh@mama-bear.lcs.mit.edu Thu Dec 7 17:47:03 1995 From: jjh@mama-bear.lcs.mit.edu (James Hugunin) Date: Thu, 7 Dec 95 12:47:03 -0500 Subject: [PYTHON MATRIX-SIG] Ranks and pseudo indices References: <199512031659.LAA21925@cyclone.ERE.UMontreal.CA> Message-ID: <9512071747.AA07309@ling-ling.lcs.mit.edu.LCS.MIT.EDU> Konrad writes: > 2) No more ranks of operators, instead Yorick style pseudo indices are > used. As a side effect of this, outer product is now a convenience function > rather than a method on ofuncs. > > As another side effect, we have lost some important functionality. > > For functions of unbounded rank (currently the only kind of function > supported by my omathmodule) Yorick pseudo indices support a superset of > what was possible to express using ranks and outer products. There is no > fundamental reason not to have both pseudo indices and ranks, I just think > that it's conceptually cleaner not to mix the two up. > > The pseudo indices are not a replacement for ranks, but just a convenient > notation for some structural functions. I like this notation very much, > but we still need ranks. And as you said, there is no reason not to have both. > I'll look at your specific comments, but I don't yet find them convincing. > So every construction with (pseudo) indices can be emulated with three > structural functions of suitable rank. On the other hand, there are > constructions that cannot be expressed with pseudo indices, for > example all cases where a function rank is not a constant but a > variable. In fact these can be handled in Python -- as opposed to > Yorick -- by remembering that a[x,y,z] is really just a shorthand for > a[(x,y,z)] and that the index tuple can be constructed from an > expression, but that seems more like an implementation accident than a > feature to me. Actually, I do think that this is a feature, but I'd be curious for a good example of a case of this. > More importantly, pseudo indices don't help in the least when it comes > to reductions and general inner products that require the specification > of an axis. Yorick does this by putting the function into the index > list -- a clever idea, but I'd prefer to keep a clear distinction > between functions and arguments. The current matrix module allows > a second parameter to reduce() that indicates the axis, which is > exactly how APL handles the problem. I don't think that you're actually complaining about my current solution to this problem are you? > But both these methods are restrictive. They are sufficient only > for reductions with functions of unbounded rank. I can give an > example of an actual application where this is not enough: I had > a sequence of n rotation matrices, stored in a rank-3 array D of > shape (n, 3, 3). Then the net rotation of this sequence could > be obtained by matrixMultiply.reduce(D). Since matrix > multiplication is a rank-2 function, I couldn't express this > with the current matrix module. This is unfortunate, but unless I hear of many more of these problems, I don't expect this to go away any time soon. It's REALLY hard to add in function of finite rank to the Optimized FUNCtion scheme I'm currently using. I spent a week and a half trying to make this happen and wound up with an ugly useless mess. Since I can imagine very few cases where this sort of behavior is important, I'm very tempted to leave ofuncs as they are right now, only operating on unbounded functions. > The bottom line: pseudo indices are convenient shorthands for > the most common structural functions, but they are no replacement > for the general function rank scheme we had in version 0.11. You haven't convinced me of this (except in the cases of functions with finite rank) and I have a large number of reasons to ignore them. -Jim ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From jjh@mama-bear.lcs.mit.edu Thu Dec 7 18:04:03 1995 From: jjh@mama-bear.lcs.mit.edu (James Hugunin) Date: Thu, 7 Dec 95 13:04:03 -0500 Subject: [PYTHON MATRIX-SIG] Renaming and Constructors References: <9512041925.AA00688@feynman.rsoc.rockwell.com> Message-ID: <9512071804.AA07317@ling-ling.lcs.mit.edu.LCS.MIT.EDU> > 1) Names > > This may seem to be a minor issue, but we should tackle this before > we all get used to the current ones. I agree completely! > I strongly dislike the type-specific constructors (also used for > output). They should be IntegerMatrix, FloatMatrix, ComplexMatrix, > CharacterMatrix, and GeneralMatrix. Anyone is free to define shorter > names for efficient typing, if desired. Does a FloatMatrix contain C floats or doubles (both are possible in matrices)? Please define a complete naming scheme that can be compared to the current (admittedly cryptic) typecodes. Note: The current output is only being used because nobody has yet given me a good replacement for the PrintMatrix function. I deliberately pulled this function out into python code so that other people could do it right for me. > I even propose a more radical renaming. Many people associate "matrix" > with the 2D-matrices from linear algebra. So it would be better to > call our general objects "arrays", and leave the name "matrix" for > linear-algebra type objects that are restricted to rank 2 and use * > for matrix multiplication. They could be implemented in Python based > on arrays. The only problem that I have with array is that python already has arrays with a known semantics. The fundamental rule in python is to never break existing code. I know this is annoying, but the only other option would be to make the current matrices completely backward compatible with the existing arrays. (Which is possibly possible...) Any other naming suggestions? (I agree that tensor is as bad as matrix) > The pseudo-index 'None' is very confusing. An index "2" picks item > number two from an axis, an index "All" picks all items from an > axis. Consequently "None" should pick no item from an axis, which is > course is a pointless operation. What it really does is create a new > axis, so it should be named "New". "RubberIndex" is not confusing, > but a bit strange, so it's worth thinking about alternatives. How > about "Skip", in the sense "skip all following axes for explicit > consideration"? For the "*" in Yorick I propose "Contract". I have no good opinions on this issue. The obvious reason that I chose None is that it is a built-in python object, so it is always in the current namespace. This namespace issue is an important one. I really don't think that we are going to be adding any of the three of these to the python core any time soon. One solution would be to use "new", "ellipses", and "contract" and to have Matrix.py map these strings to objects. Note: I picked RubberIndex in the hopes of choosing something sufficiently long and hard to type that nobody would assume I meant it to be the final solution. I had hoped that this might be turned into the syntax ".." one day, but after the python workshop I doubt that this will happen any time soon. > 2) Constructors > > Currently there are two constructors, matrix() and Matrix(), with > slightly different behaviour: matrix() takes a single argument that is > a (possibly nested) Python sequence object, e.g. a list. Matrix() > takes an arbitrary number of arguments that are made into a list > before being passed to matrix(). So Matrix(1,2,3) is equivalent > to matrix([1,2,3]). > > I find it confusing to have to constructors with almost the same name > and almost the same function. I propose to have only one, of whatever > name, which behaves like matrix(). The reason is that many matrix > functions accept nested lists or tuples instead of matrix arguments, > e.g. matrix([2,3]) + [5,4] works. So does matrix([2,3]) + > matrix([5,4]), which is equivalent. But Matrix([2,3]) + [5,4] and > Matrix([2,3]) + Matrix([5,4]) are not equivalent (well, they are in > this simple exaple due to broadcasting, but not in general). So > matrix() can be considered a conversion function from lists and tuples > to arrays, which is needed anyway. > > I realize that the calling style of Matrix() has the advantage of > saving one pair of brackets, but I don't consider this important > enough to create the current confusion. This is a real problem. I'll repeat my reasons for creating Matrix, but otherwise, I'm willing to go along with the consensus of the list on this one. I was looking for the cleanest and easiest to read notation for a matrix in the unfortunate world where I couldn't add a new bracket type to python (okay, there aren't even any brackets to add so what am I complaining about). I noticed that by using the variable number of arguments notation, I got something that looked a lot more like just a new kind of brackets. Let me know who else agrees with Konrad here. > I guess that's enough for today ;-) I wish the lucky participants > of the workshop much fun and hope that they don't have to eat > spam for lunch ;-) Had much fun, never ate SPAM - Jim ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From hinsenk@ere.umontreal.ca Thu Dec 7 19:34:11 1995 From: hinsenk@ere.umontreal.ca (Hinsen Konrad) Date: Thu, 7 Dec 1995 14:34:11 -0500 Subject: [PYTHON MATRIX-SIG] Ranks and pseudo indices In-Reply-To: <9512071747.AA07309@ling-ling.lcs.mit.edu.LCS.MIT.EDU> (message from James Hugunin on Thu, 7 Dec 95 12:47:03 -0500) Message-ID: <199512071934.OAA10658@cyclone.ERE.UMontreal.CA> > variable. In fact these can be handled in Python -- as opposed to > Yorick -- by remembering that a[x,y,z] is really just a shorthand for > a[(x,y,z)] and that the index tuple can be constructed from an > expression, but that seems more like an implementation accident than a > feature to me. Actually, I do think that this is a feature, but I'd be curious for a good example of a case of this. OK, let's make it a feature. I can't remember an example right now, but I know that I used variables to specify ranks in J a few times. Besides, my philosophy is the opposite: there should be good arguments for restrictions, not for generality. Sooner or later someone will reach the limits of everything, so you should not impose narrow limits arbitrarily just because you can't think of any problem affected by them. > between functions and arguments. The current matrix module allows > a second parameter to reduce() that indicates the axis, which is > exactly how APL handles the problem. I don't think that you're actually complaining about my current solution to this problem are you? In the sense of being restrictive, yes. Many APL users have found this handling of reduction restrictive, which is why J's rank scheme is generally appreciated, even by those who are put off by J's other features. This is unfortunate, but unless I hear of many more of these problems, I don't expect this to go away any time soon. It's REALLY hard to add in function of finite rank to the Optimized FUNCtion scheme I'm currently using. I spent a week and a half trying to make this happen and wound up I confess that I have never looked at their implementation, but I don't see where the problem is. It is even possible to implement the mapping from finite-rank functions to unbounded-rank functions in Python relatively easily, but that means for-loops and the associated loss of efficiency. I'll see if I can figure out how the OFUNCs work. with an ugly useless mess. Since I can imagine very few cases where this sort of behavior is important, I'm very tempted to leave ofuncs as they are right now, only operating on unbounded functions. If you want more cases, try to locate an archive of comp.lang.apl of the time when J came out. There were long discussions about the merits of the rank system back then. ------------------------------------------------------------------------------- Konrad Hinsen | E-Mail: hinsenk@ere.umontreal.ca Departement de chimie | Tel.: +1-514-343-6111 ext. 3953 Universite de Montreal | Fax: +1-514-343-7586 C.P. 6128, succ. Centre-Ville | Deutsch/Esperanto/English/Nederlands/ Montreal (QC) H3C 3J7 | Francais (phase experimentale) ------------------------------------------------------------------------------- ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From hinsenk@ere.umontreal.ca Thu Dec 7 20:14:31 1995 From: hinsenk@ere.umontreal.ca (Hinsen Konrad) Date: Thu, 7 Dec 1995 15:14:31 -0500 Subject: [PYTHON MATRIX-SIG] Renaming and Constructors In-Reply-To: <9512071804.AA07317@ling-ling.lcs.mit.edu.LCS.MIT.EDU> (message from James Hugunin on Thu, 7 Dec 95 13:04:03 -0500) Message-ID: <199512072014.PAA13383@cyclone.ERE.UMontreal.CA> Does a FloatMatrix contain C floats or doubles (both are possible in matrices)? Please define a complete naming scheme that can be compared to Really? I didn't know that. Anyway, "float" should mean what it means in normal Python, i.e. C double. C floats could be called "ShortFloat", for example. As for the other typecodes, I am not sure what they all mean (is there a list somewhere?). But let's start: 'c' Character 'u' Unsigned (is that really useful?) '1' ??? (maybe Boolean?) 's' String 'i' Integer 'l' LongInteger 'f' ShortFloat 'd' Float 'F' ??? (maybe ShortComplex?) 'D' Complex 'O' General The only problem that I have with array is that python already has arrays with a known semantics. The fundamental rule in python is to never break existing code. I know this is annoying, but the only other option would be to make the current matrices completely backward compatible with the existing arrays. (Which is possibly possible...) So that means we can not call any module "array", but we can still use the name "array" within the modules. A suggestions: - "arrays_and_matrices" for the C module that nobody imports directly anyway. - "Array" for the Python wrapper with array semantics; "Array" in the names of the constructors. - "Matrix" for the Python wrapper with 2D-matrix semantics; "Matrix" in the names of the constructors. That leaves the possibility of confusion between "array" and "Array", but I can live with that. I doubt the current Python arrays will be attractive to anyone who has the new module. namespace. This namespace issue is an important one. I really don't think that we are going to be adding any of the three of these to the python core any time soon. They don't have to be. They just have to be in the namespace of the modules "Array" and "Matrix". > I guess that's enough for today ;-) I wish the lucky participants > of the workshop much fun and hope that they don't have to eat > spam for lunch ;-) Had much fun, never ate SPAM - Jim That's cheating ;-) ------------------------------------------------------------------------------- Konrad Hinsen | E-Mail: hinsenk@ere.umontreal.ca Departement de chimie | Tel.: +1-514-343-6111 ext. 3953 Universite de Montreal | Fax: +1-514-343-7586 C.P. 6128, succ. Centre-Ville | Deutsch/Esperanto/English/Nederlands/ Montreal (QC) H3C 3J7 | Francais (phase experimentale) ------------------------------------------------------------------------------- ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From chris.chase@jhuapl.edu Fri Dec 8 20:59:02 1995 From: chris.chase@jhuapl.edu (Chris Chase S1A) Date: Fri, 8 Dec 1995 15:59:02 -0500 Subject: [PYTHON MATRIX-SIG] Release 0.20 of matrix object is available In-Reply-To: <9512012339.AA04760@ling-ling.lcs.mit.edu.LCS.MIT.EDU> References: <9512012339.AA04760@ling-ling.lcs.mit.edu.LCS.MIT.EDU> Message-ID: <199512082050.PAA00755@python.org> James> The NEWS for this release follows. The two major incompatible James> changes at the very beginning are why this release took so long James> (that and the fact that I have a few other demands on my time). James> Please let me know what you think of these changes! James> As before, don't hesitate to send me your gripes, your bugs, James> and your praise (if ever merited). However, I'll be gone for James> most of next week at the python workshop, so I might be slow in James> replying. James> -Jim James> Major incompatible changes (I expect that these might elict some James> interesting discussion): James> 1) Matrix indexing now returns by reference to the original James> data, not by value. As a side effect of this change, James> arbitrary sequences are not allowed in multidimensional James> indices, but only single indices, slices, RubberIndex and James> None. I personally prefer general product indexing that allow multidimensional indexes. These could be added to the reference version of indexing but they would require extra baggage by retaining a copy of the index vector. This would not be very efficient but is not a reason to eliminate it since it would not affect the efficiency of the other types of indexes. Are references better than copying for indexing? I can see that a speed increase is natural when using only references rather than creating a copy. But are references always efficient? If a multidimensional slice reference is passed as an argument to another routine that accesses the object multiple times then it may end up being less efficient. To avoid this would require that these routines make contiguous copies of these types of arguments (whether or not the copy is needed since the reference nature of the object is transparent to the user). Similarly a copy to contiguous storage would need to be done before passing a reference to an external FORTRAN or C routine. On the other hand, I can see that references can save memory for very large matrix objects. Additionally, a copy can be performed at user discretion. I would think that assignment to references that change the originally indexed object may lead to some surprises for some users used to the implementations of other matrix languages like Matlab and IDL. I am not an expert on this. What are the other pros and cons about references versus copies being returned by indexing? James> Note: there is a new method, take, which will allow you to James> index a matrix with an arbitrary sequence as before. This then requires a sequence of take() plus [] indexing to obtain general product indexing. How does one do assignment using a multi-dimensional index vector? James> This change was motivated both by the possible speed increases James> (about 40% for some code) and more importantly by issues of James> clarity of expression. I couldn't come up with any other way James> to make the following hold: James> m[0][1] is an efficient way to index a multi-dimensional array. James> m[0,1] == m[0][1] James> m[0,] == m[0] It will be difficult to maintain a natural connection between hierarchical index using [][] versus multi-dimensional indexing. For example: m[0:4,1] != m[0:4][1] James> 2) No more ranks of operators, instead Yorick style pseudo James> indices are used. As a side effect of this, outer product James> is now a convenience function rather than a method on James> ofuncs. James> For functions of unbounded rank (currently the only kind of James> function supported by my omathmodule) Yorick pseudo indices James> support a superset of what was possible to express using ranks James> and outer products. There is no fundamental reason not to have James> both pseudo indices and ranks, I just think that it's James> conceptually cleaner not to mix the two up. Function ranks offer more generality than what can be done with pseudo indices. I would prefer that the matrix module keep function ranks rather than limit the full generality of the object. Although I do not what it takes to add it cleanly. I have not had time to look deeply into your code other than browsing the matrixobject.c (the current state of the code made for rather dense reading). James> 1) This new release is about 40% faster than 0.11 (on a James> benchmark code I was given by the folks at LBL). What was the benchmark? >> 1) Names >> >> This may seem to be a minor issue, but we should tackle this before >> we all get used to the current ones. James> I agree completely! >> I strongly dislike the type-specific constructors (also used for >> output). They should be IntegerMatrix, FloatMatrix, ComplexMatrix, >> CharacterMatrix, and GeneralMatrix. Anyone is free to define shorter >> names for efficient typing, if desired. James> Does a FloatMatrix contain C floats or doubles (both are James> possible in matrices)? Please define a complete naming scheme James> that can be compared to the current (admittedly cryptic) James> typecodes. Perhaps a naming scheme similar to FORTRAN's INTEGER*8, REAL*4 REAL*8. The advantage of this is that the precision is known. With ANSI C the precision of floats and doubles is not specified. ANSI C only enforces a minimum precision on an implementation. >> I even propose a more radical renaming. Many people associate "matrix" >> with the 2D-matrices from linear algebra. So it would be better to >> call our general objects "arrays", and leave the name "matrix" for >> linear-algebra type objects that are restricted to rank 2 and use * >> for matrix multiplication. They could be implemented in Python based >> on arrays. I would prefer the term "array" with a specialized "matrix" module providing linear algebra matrix functions and a matrix multiplication operator that take 2D arrays as arguments. But it was pointed out that we can not use "array" as it is already a different module in Python. I don't suppose the module produced by this SIG could completely replace the current array module (without breadking its old behavior)? James> Note: I picked RubberIndex in the hopes of choosing something James> sufficiently long and hard to type that nobody would assume I James> meant it to be the final solution. I had hoped that this might James> be turned into the syntax ".." one day, but after the python James> workshop I doubt that this will happen any time soon. What happened at the workshop to rule out the ".." or "*" syntax for rubber indexes? Chris ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From hinsenk@ere.umontreal.ca Fri Dec 8 23:16:02 1995 From: hinsenk@ere.umontreal.ca (Hinsen Konrad) Date: Fri, 8 Dec 1995 18:16:02 -0500 Subject: [PYTHON MATRIX-SIG] Release 0.20 of matrix object is available In-Reply-To: <199512082050.PAA00755@python.org> (message from Chris Chase S1A on Fri, 8 Dec 1995 15:59:02 -0500) Message-ID: <199512082316.SAA05920@cyclone.ERE.UMontreal.CA> It will be difficult to maintain a natural connection between hierarchical index using [][] versus multi-dimensional indexing. For example: m[0:4,1] != m[0:4][1] I agree, and we shouldn't insist on this as a feature. It might even be better (in terms of error checking) to forbid an index that doesn't cover all dimensions. Right now, a[0] is just a shorthand for a[0, RubberIndex], and I don't think this shorthand is necessary. Perhaps a naming scheme similar to FORTRAN's INTEGER*8, REAL*4 REAL*8. The advantage of this is that the precision is known. With ANSI C the precision of floats and doubles is not specified. ANSI C only enforces a minimum precision on an implementation. And since Python is written in C, we can't promise anything more. Besides, giving a single number still doesn't specify the precision of a floating point number. The memory space could be distributed differently between mantissa and exponent. My major concern is that array names correspond with the standard Python number type names as far as possible. Anything else just generates confusion. operator that take 2D arrays as arguments. But it was pointed out that we can not use "array" as it is already a different module in As I pointed out before, it is only the module name which may not be "array". Even "Array" is OK. ------------------------------------------------------------------------------- Konrad Hinsen | E-Mail: hinsenk@ere.umontreal.ca Departement de chimie | Tel.: +1-514-343-6111 ext. 3953 Universite de Montreal | Fax: +1-514-343-7586 C.P. 6128, succ. Centre-Ville | Deutsch/Esperanto/English/Nederlands/ Montreal (QC) H3C 3J7 | Francais (phase experimentale) ------------------------------------------------------------------------------- ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From dubois1@llnl.gov Sat Dec 9 00:05:54 1995 From: dubois1@llnl.gov (Paul. Dubois) Date: Fri, 08 Dec 1995 16:05:54 -0800 Subject: [PYTHON MATRIX-SIG] Release 0.20 of matrix object is available References: <199512082316.SAA05920@cyclone.ERE.UMontreal.CA> Message-ID: <30C8D2E2.6190@llnl.gov> A followup to the remarks about precision. In fact, Fortran 90 goes to considerable lengths to try to improve this situation but it isn't REAL*8 that is the solution. Fortran 90 uses something called a Kind specifier, and Kind specifiers are calculable from desired characteristics at compile time. Actually knowing that something is a certain number of bits doesn't tell you what you need to know, as there is a tradeoff between precision and dynamic range. Anyway, you end up saying REAL(kind) x, y, z. I believe Jim has made fundamentally the right decision in deciding that the abstraction he is after is "things that store a homogenous bunch of stuff in contiguous memory". This would be a parameterized type in other languages, such as Eiffel's ARRAY[T], where one actually instantiates ARRAY[DOUBLE], ARRAY[INTEGER], ARRAY[CHARACTER], ARRAY[TELEPHONE_BOOK], etc. Since a large use of this abstraction is connection to C and Fortran libraries, one has to be precise about it in a cross-language way and I think the present proposal is correct. The question of what language must be used to express this at the Python level is an interesting one. While I favor names like IntegerMatrix, ..., reserving Matrix to be a "smart" operator that decides on the type from the arguments, there is a place for a version with a calculable type or type code, so that one can get different things on different platforms with the same source by doing tests on the system type. I think Array is a better name than Matrix and if 'array' weren't already taken would have no hesitation in going ahead. If we picked Array as the name that is now Matrix, what would we call that which is now 'matrix'? ArrayFromSequence ? -- Paul F. Dubois, L-472 (510)-422-5426 Lawrence Livermore National Laboratory FAX (510)-423-9969 Livermore, CA 94550 dubois1@llnl.gov Consulting: PaulDubois@aol.com ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From hinsenk@ere.umontreal.ca Sat Dec 9 16:05:43 1995 From: hinsenk@ere.umontreal.ca (Hinsen Konrad) Date: Sat, 9 Dec 1995 11:05:43 -0500 Subject: [PYTHON MATRIX-SIG] Release 0.20 of matrix object is available In-Reply-To: <30C8D2E2.6190@llnl.gov> (dubois1@llnl.gov) Message-ID: <199512091605.LAA12973@cyclone.ERE.UMontreal.CA> interesting one. While I favor names like IntegerMatrix, ..., reserving Matrix to be a "smart" operator that decides on the type from the arguments, there is a place for a version with a calculable type or type code, so that one can get different things on different platforms with the same source by doing tests on the system type. The "smart" version could take an optional argument indicating the type somehow. In addition to the typecodes, it would be nice if it would accept the return values of the builtin function type() . Then you could write something like list = [1,3,2] Array(list, type(list[0])) The typecode will still be needed for the types that have no Python equivalents, like C-floats. I think Array is a better name than Matrix and if 'array' weren't already taken would have no hesitation in going ahead. If we picked Once more, it is only the module name "array" that we can't use. Nothing else is global in Python. Array as the name that is now Matrix, what would we call that which is now 'matrix'? ArrayFromSequence ? I still propose that "Array" should behave like "matrix" does now. That way we need only one constructor. ------------------------------------------------------------------------------- Konrad Hinsen | E-Mail: hinsenk@ere.umontreal.ca Departement de chimie | Tel.: +1-514-343-6111 ext. 3953 Universite de Montreal | Fax: +1-514-343-7586 C.P. 6128, succ. Centre-Ville | Deutsch/Esperanto/English/Nederlands/ Montreal (QC) H3C 3J7 | Francais (phase experimentale) ------------------------------------------------------------------------------- ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From hinsenk@ere.umontreal.ca Sat Dec 9 16:05:43 1995 From: hinsenk@ere.umontreal.ca (Hinsen Konrad) Date: Sat, 9 Dec 1995 11:05:43 -0500 Subject: [PYTHON MATRIX-SIG] Release 0.20 of matrix object is available In-Reply-To: <30C8D2E2.6190@llnl.gov> (dubois1@llnl.gov) Message-ID: <199512091605.LAA12973@cyclone.ERE.UMontreal.CA> interesting one. While I favor names like IntegerMatrix, ..., reserving Matrix to be a "smart" operator that decides on the type from the arguments, there is a place for a version with a calculable type or type code, so that one can get different things on different platforms with the same source by doing tests on the system type. The "smart" version could take an optional argument indicating the type somehow. In addition to the typecodes, it would be nice if it would accept the return values of the builtin function type() . Then you could write something like list = [1,3,2] Array(list, type(list[0])) The typecode will still be needed for the types that have no Python equivalents, like C-floats. I think Array is a better name than Matrix and if 'array' weren't already taken would have no hesitation in going ahead. If we picked Once more, it is only the module name "array" that we can't use. Nothing else is global in Python. Array as the name that is now Matrix, what would we call that which is now 'matrix'? ArrayFromSequence ? I still propose that "Array" should behave like "matrix" does now. That way we need only one constructor. ------------------------------------------------------------------------------- Konrad Hinsen | E-Mail: hinsenk@ere.umontreal.ca Departement de chimie | Tel.: +1-514-343-6111 ext. 3953 Universite de Montreal | Fax: +1-514-343-7586 C.P. 6128, succ. Centre-Ville | Deutsch/Esperanto/English/Nederlands/ Montreal (QC) H3C 3J7 | Francais (phase experimentale) ------------------------------------------------------------------------------- ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From da@maigret.cog.brown.edu Sat Dec 9 21:12:04 1995 From: da@maigret.cog.brown.edu (David Ascher) Date: Sat, 9 Dec 1995 16:12:04 -0500 (EST) Subject: [PYTHON MATRIX-SIG] Display of 1 & 2D matrices? In-Reply-To: <199510021627.MAA14222@cyclone.ERE.UMontreal.CA> from "Hinsen Konrad" at Oct 2, 95 12:27:22 pm Message-ID: <199512092112.QAA05656@maigret> Pardon this digression please: Has anyone started working on display modules for 1 and 2D matrices using any of the gui systems available for Python (Tk-based, X-based, whatever-based)? --david ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From da@maigret.cog.brown.edu Sat Dec 9 21:12:04 1995 From: da@maigret.cog.brown.edu (David Ascher) Date: Sat, 9 Dec 1995 16:12:04 -0500 (EST) Subject: [PYTHON MATRIX-SIG] Display of 1 & 2D matrices? In-Reply-To: <199510021627.MAA14222@cyclone.ERE.UMontreal.CA> from "Hinsen Konrad" at Oct 2, 95 12:27:22 pm Message-ID: <199512092112.QAA05656@maigret> Pardon this digression please: Has anyone started working on display modules for 1 and 2D matrices using any of the gui systems available for Python (Tk-based, X-based, whatever-based)? --david ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From jfulton@usgs.gov Mon Dec 11 13:39:43 1995 From: jfulton@usgs.gov (Jim Fulton, U.S. Geological Survey) Date: Mon, 11 Dec 1995 08:39:43 -0500 Subject: [PYTHON MATRIX-SIG] Release 0.20 of matrix object is available In-Reply-To: "Paul. Dubois" "Re: [PYTHON MATRIX-SIG] Release 0.20 of matrix object is available" (Dec 8, 4:05pm) References: <199512082316.SAA05920@cyclone.ERE.UMontreal.CA> <30C8D2E2.6190@llnl.gov> Message-ID: <9512110839.ZM26537@dsdbqvarsa.er.usgs.gov> One minor note on these naming discussions. It is necessary for names to be unique without regard to case. That is, although Python is case sensitive, some of the operating systems it functions in are not, and since module names are found in the local file systems, the names like array and Array may conflict. Jim On Dec 8, 4:05pm, Paul. Dubois wrote: > Subject: Re: [PYTHON MATRIX-SIG] Release 0.20 of matrix object is availabl > A followup to the remarks about precision. > In fact, Fortran 90 goes to considerable lengths to try to improve this > situation but it isn't REAL*8 that is the solution. Fortran 90 uses > something called a Kind specifier, and Kind specifiers are calculable > from desired characteristics at compile time. Actually knowing that > something is a certain number of bits doesn't tell you what you need to > know, as there is a tradeoff between precision and dynamic range. > Anyway, you end up saying REAL(kind) x, y, z. > > I believe Jim has made fundamentally the right decision in deciding that > the abstraction he is after is "things that store a homogenous bunch of > stuff in contiguous memory". This would be a parameterized type in other > languages, such as Eiffel's ARRAY[T], where one actually instantiates > ARRAY[DOUBLE], ARRAY[INTEGER], ARRAY[CHARACTER], ARRAY[TELEPHONE_BOOK], > etc. Since a large use of this abstraction is connection to C and > Fortran libraries, one has to be precise about it in a cross-language > way and I think the present proposal is correct. The question of what > language must be used to express this at the Python level is an > interesting one. While I favor names like IntegerMatrix, ..., reserving > Matrix to be a "smart" operator that decides on the type from the > arguments, there is a place for a version with a calculable type or type > code, so that one can get different things on different platforms with > the same source by doing tests on the system type. > > I think Array is a better name than Matrix and if 'array' weren't > already taken would have no hesitation in going ahead. If we picked > Array as the name that is now Matrix, what would we call that which is > now 'matrix'? ArrayFromSequence ? > > -- > Paul F. Dubois, L-472 (510)-422-5426 > Lawrence Livermore National Laboratory FAX (510)-423-9969 > Livermore, CA 94550 dubois1@llnl.gov > Consulting: PaulDubois@aol.com > > ================= > MATRIX-SIG - SIG on Matrix Math for Python > > send messages to: matrix-sig@python.org > administrivia to: matrix-sig-request@python.org > ================= >-- End of excerpt from Paul. Dubois ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From hinsenk@ere.umontreal.ca Mon Dec 11 14:32:05 1995 From: hinsenk@ere.umontreal.ca (Hinsen Konrad) Date: Mon, 11 Dec 1995 09:32:05 -0500 Subject: [PYTHON MATRIX-SIG] Release 0.20 of matrix object is available In-Reply-To: <9512110839.ZM26537@dsdbqvarsa.er.usgs.gov> (jfulton@usgs.gov) Message-ID: <199512111432.JAA26952@cyclone.ERE.UMontreal.CA> It is necessary for names to be unique without regard to case. That is, although Python is case sensitive, some of the operating systems it functions in are not, and since module names are found in the local file systems, the names like array and Array may conflict. Does that mean that Python doesn't do any clever filename mapping on sytems that do not support case distinction and/or long file names? I never checked, but I always assumed that it does something like the GNU library that maps Unix filenames to short DOS-style filenames (i.e. provide a mapping that is as unique as possible, even if that produces strange-looking filenames). So, in effect, Python modules names are restricted to 8 lower-case characters? That would create a few problems with the Python code I have produced so far... ------------------------------------------------------------------------------- Konrad Hinsen | E-Mail: hinsenk@ere.umontreal.ca Departement de chimie | Tel.: +1-514-343-6111 ext. 3953 Universite de Montreal | Fax: +1-514-343-7586 C.P. 6128, succ. Centre-Ville | Deutsch/Esperanto/English/Nederlands/ Montreal (QC) H3C 3J7 | Francais (phase experimentale) ------------------------------------------------------------------------------- ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From et@appl-math.tu-muenchen.de Mon Dec 11 14:11:45 1995 From: et@appl-math.tu-muenchen.de (Thomas Schwaller) Date: Mon, 11 Dec 1995 15:11:45 +0100 Subject: [PYTHON MATRIX-SIG] plmodule.c, openglmodule.c Message-ID: <9512111511.ZM4023@piranha.appl-math.tu-muenchen.de> Hi Matrix SIG'ers, 1) First a comment on the Array, array, matrix discussion... What about the name "multiarray" for Array. When I see uppercase letters I always think I can derive from that class, but for Array this would not be the case. Just an idea, not really important. 2) I found a bug in my plmodule.c. When you use just matrices everything is ok but when tuple or list input is transformed to matrices these are dereferenced too early. Fixed that bug. ftp://ftp.appl-math.tu-muenchen.de/pub/et/plmodule.c.0.3.gz Will "try" to put it on our matrix location. Connections are nearly impossible for me here! :-( 3) Completely reworked my OpenGl module. This was quite old, used the old naming convention and the cgen script for producing the glmodule. Now everything works with matrices as input, not just tuples or lists. The module is not completely finished yet, but you can already do a whole bunch of things with it (257 procedures up to now in the module). For GLUT or OpenGL widgets and general information about this openGL module check also the Info collected by David Ascher at http://maigret.cog.brown.edu:80/python/opengl/ If you want to have a look at this preversion check: ftp://ftp.appl-math.tu-muenchen.de/pub/et/openglmodule.c.0.1.gz If there's enough interest in that I will certainly contribute a lot of examples. 4) Heard about the visual toolkit C++ classes and the Tcl binding for it. Coolest thing for OpenGl graphics at the moment (at least for me! :-)) Ken Martin (one of the authors) agreed to to a python binding with me. The Tcl binding works well (I already did some wrappers for Python `a la Tkinter, but we will do a native Python binding) and at the moment he is doing a Java binding. To map the quite complex data hierarchy to python I will certainly need advice from people having done this before. Unfortunately startup time is quite high and comiling the classes and working with them shows the limits of C++ compiler technology (at least the one I have, g++ and SGI CC). Using the code in the classes directly in a Python module would be great but why developping C++ classes you will ask. Well, ... good question ... >>THIS WAS JUST FOR YOUR INFORMATION<< Hope it's not boring you to death! ;_) 5) How was the feedback at the python workshop concernig the matrixmodule. Would be nice to hear a word or two. 6) Read about the sprse matrix stuff in the paper for the workshop. Is this making any progress or is it a long term story not been started yet. Sorry I'm just curious! Tom ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From fredrik_lundh@ivab.se Mon Dec 11 15:14:15 1995 From: fredrik_lundh@ivab.se (Fredrik Lundh) Date: Mon, 11 Dec 1995 16:14:15 +0100 Subject: [PYTHON MATRIX-SIG] Release 0.20 of matrix object is available In-Reply-To: <199512111432.JAA26952@cyclone.ERE.UMontreal.CA> (hinsenk@ere.umontreal.ca) Message-ID: <9512111514.AA28909@kalle.image.ivab.se> > Does that mean that Python doesn't do any clever filename mapping > on sytems that do not support case distinction and/or long file names? As with most other things, Python leaves it to the underlying system to handle this. Would think that MS-DOS and Windows 3.1 is the worst case, where filenames are _silently truncated_ to fit into the 8.3 scheme. So "import ContainerIO" will in look for "containe.py". Since my editor does the same, I have no problems when writing modules, as long as I make sure that the 8 first characters are unique. This is no problem under Windows 95/NT, even on FAT file systems, nor on Macs. Don't know about Amigas. (Generally, I think its a bad idea to let lowercase and capitalized versions of the same name mean different things.) /F ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From jfulton@usgs.gov Mon Dec 11 15:16:20 1995 From: jfulton@usgs.gov (Jim Fulton, U.S. Geological Survey) Date: Mon, 11 Dec 1995 10:16:20 -0500 Subject: [PYTHON MATRIX-SIG] Release 0.20 of matrix object is available In-Reply-To: hinsenk@ERE.UMontreal.CA (Hinsen Konrad) "Re: [PYTHON MATRIX-SIG] Release 0.20 of matrix object is available" (Dec 11, 9:32am) References: <199512111432.JAA26952@cyclone.ERE.UMontreal.CA> Message-ID: <9512111016.ZM27294@dsdbqvarsa.er.usgs.gov> On Dec 11, 9:32am, Hinsen Konrad wrote: > Subject: Re: [PYTHON MATRIX-SIG] Release 0.20 of matrix object is availabl > > It is necessary for names to be unique without regard to case. That is, > although Python is case sensitive, some of the operating systems it functions > in are not, and since module names are found in the local file systems, the > names like array and Array may conflict. > > Does that mean that Python doesn't do any clever filename mapping > on sytems that do not support case distinction and/or long file names? > I never checked, but I always assumed that it does something like > the GNU library that maps Unix filenames to short DOS-style filenames > (i.e. provide a mapping that is as unique as possible, even if that > produces strange-looking filenames). > > So, in effect, Python modules names are restricted to 8 lower-case > characters? That would create a few problems with the Python code I > have produced so far... Actually, the 8-character limit is not as serious, since: 1. For most dos/windows-3.1x programs, including python, long file names are automatically truncated, and 2. Windows NT and Windows-95 do not have the 8-character limitation. Still, Windows-NT and Windows 95 are both case-insensitive, and when Python is told to import "Spam", it will happily import from Spam.py, spam.py. SPAM.PY, ... Jim ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From mclay@eeel.nist.gov Mon Dec 11 17:57:22 1995 From: mclay@eeel.nist.gov (Michael McLay) Date: Mon, 11 Dec 95 12:57:22 EST Subject: [PYTHON MATRIX-SIG] Release 0.20 of matrix object is available In-Reply-To: <199512111432.JAA26952@cyclone.ERE.UMontreal.CA> References: <9512110839.ZM26537@dsdbqvarsa.er.usgs.gov> <199512111432.JAA26952@cyclone.ERE.UMontreal.CA> Message-ID: <9512111757.AA00826@acdc.eeel.nist.gov> Hinsen Konrad writes: > > It is necessary for names to be unique without regard to case. That is, > although Python is case sensitive, some of the operating systems it functions > in are not, and since module names are found in the local file systems, the > names like array and Array may conflict. > > Does that mean that Python doesn't do any clever filename mapping > on sytems that do not support case distinction and/or long file names? > I never checked, but I always assumed that it does something like > the GNU library that maps Unix filenames to short DOS-style filenames > (i.e. provide a mapping that is as unique as possible, even if that > produces strange-looking filenames). > > So, in effect, Python modules names are restricted to 8 lower-case > characters? That would create a few problems with the Python code I > have produced so far... I don't think a universally agreed upon solution has ever been reached on the filename mapping issue. It wouldn't be to hard to implement a platform independent mechanism that allowed case sensitive names. It could all be done by redefining the import statement so that filenames are looked up at run time on some platforms. The filesystem on CD-ROMs uses this technique for allowing "portable" file names. Python might even use the same encoding rules. This will have to be done when the PSA develops a multi-platform CD-ROM distribution of Python. Michael ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From hinsenk@ere.umontreal.ca Mon Dec 11 23:27:45 1995 From: hinsenk@ere.umontreal.ca (Hinsen Konrad) Date: Mon, 11 Dec 1995 18:27:45 -0500 Subject: [PYTHON MATRIX-SIG] Release 0.20 of matrix object is available In-Reply-To: <9512111016.ZM27294@dsdbqvarsa.er.usgs.gov> (jfulton@usgs.gov) Message-ID: <199512112327.SAA05548@cyclone.ERE.UMontreal.CA> Actually, the 8-character limit is not as serious, since: 1. For most dos/windows-3.1x programs, including python, long file names are automatically truncated, and Bad enough. I have lots of module names that do not differ in the first eight characters. Not much of a problem for me, because I am not going to use 8-character systems any more unless forced with a gun, but a problem for Python portability in general. Still, Windows-NT and Windows 95 are both case-insensitive, and when Python is told to import "Spam", it will happily import from Spam.py, spam.py. SPAM.PY, ... Another portability problem. Although in general I wouldn't use upper- and lowercase variants of the same name together, this can happen by accident, and then I have code that works only on some platforms. Since Pyton uses case-sensitive names everywhere, it should try to enforce this as much as possible also for module names. Michael McLay on the same topic: I don't think a universally agreed upon solution has ever been reached on the filename mapping issue. It wouldn't be to hard to implement a platform independent mechanism that allowed case sensitive names. It could all be done by redefining the import statement so that filenames are looked up at run time on some platforms. The filesystem on That would be a good solution. Is there any SIG we can make responsible for this? ;-) ------------------------------------------------------------------------------- Konrad Hinsen | E-Mail: hinsenk@ere.umontreal.ca Departement de chimie | Tel.: +1-514-343-6111 ext. 3953 Universite de Montreal | Fax: +1-514-343-7586 C.P. 6128, succ. Centre-Ville | Deutsch/Esperanto/English/Nederlands/ Montreal (QC) H3C 3J7 | Francais (phase experimentale) ------------------------------------------------------------------------------- ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From jfulton@usgs.gov Tue Dec 12 23:38:35 1995 From: jfulton@usgs.gov (Jim Fulton, U.S. Geological Survey) Date: Tue, 12 Dec 1995 18:38:35 -0500 Subject: [PYTHON MATRIX-SIG] [comp.lang.python] Proposal for sharing C interfaces between extension modules Message-ID: <199512122338.XAA14048@qvarsx.er.usgs.GOV> The following message was posted on the Python list. One of the driving reasons for this proposal is to simplify the configuration of the matrix object implementation and the many extension modules that will use it. My idea is that the matrix object can be included in a module and that the C interface to the matrix object type can be exported as a collection of CObjects or through a single CObject that is an array or struct (or other data structure) of C function pointers. Then extension modules that want to use the C interface to the matrix object can import the matrix module and use PyObject_GetAttrString to get access to the C interface. In this way, the matrix object need not be munged into the standard source tree. -- Jim ------- Forwarded Message From: jfulton@disqvarsa.er.usgs.GOV (Jim Fulton ) Newsgroups: comp.lang.python Subject: Proposal for sharing C interfaces between extension modules Date: 12 Dec 1995 19:36:04 GMT Organization: U.S. Geological Survey NNTP-Posting-Host: disqvarsa.er.usgs.gov Problem: It is sometimes the case that an extension module needs to export a C interface to other extension modules. This is especially true for modules that define useful built-in types. If modules are dynamically linked, sharing C interfaces between modules can get someone complicated, because the extension modules need to get linked together and get linked with Python. How this is done can vary significantly from system to system. Python already provides a dynamic linking facility, via import. An extension module can import other modules, including other extension modules. So, for example, a Matrix module could import a complex number module and use PyObject_GetAttrString to get any objects (classes, functions, etc) exported by the complex module. This mechanism is nice because it doesn't depend in any way how the module being imported was linked. Python handles the linking for you. Unfortunately, this currently only works with Python objects exported by a module. (Fortunately, if only a Python interface to a module is needed, this mechanism allows an extension module to "link" to another module regardless of whether the other module is a Python module or an extension module. Cool. :) I propose adding a very small new type to the core language, called CObject, that is a Python wrapper for C objects. On my system, this new type adds a total of 204 bytes to the text portion of the interpreter. :-) If an extension module wishes to export a C object (such as a C function, or an array of C functions) to other extension modules, the extension module would use the function CObject_FromVoidPtr to create a Python object that wraps a pointer to the C object. The module would then insert this object in the module's dictionary during module initialization. Here is the declaration for CObject_FromVoidPtr: /* Create a CObject from a pointer to a C object and an optional destrutor function. If the second argument is non-null, then it will be called with the first argument if and when the CObject is destroyed. */ extern PyObject * CObject_FromVoidPtr Py_PROTO((void *cobj, void (*destruct)(void*))); An extension module that wishes to use a C object exported by another extension module, would: - Import the other module, - Use PyObject_GetAttrString to get the CObject, using a published name, and - Use CObject_AsVoidPtr to obtain a pointer to the C object, Here is the declaration for CObject_AsVoidPtr: /* Retrieve a pointer to a C object from a CObject. */ extern void * CObject_AsVoidPtr Py_PROTO((PyObject *)); Of course, CObjects exported by a module can be accessed from Python, but they provide no useful behavior that can be accessed directly from Python scripts. They can be assigned to Python variables and passed to extension functions or extension type methods. This small extension to the language should significantly ease the configuration of modules that need or want to interface with each other via C. -- -- Jim Fulton jfulton@usgs.gov (703) 648-5622 U.S. Geological Survey, Reston VA 22092 This message is being posted to obtain or provide technical information relating to my duties at the U.S. Geological Survey. ------- End of Forwarded Message ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From jjh@mama-bear.lcs.mit.edu Wed Dec 13 17:51:32 1995 From: jjh@mama-bear.lcs.mit.edu (James Hugunin) Date: Wed, 13 Dec 95 12:51:32 -0500 Subject: [PYTHON MATRIX-SIG] Re: Version 2.0 References: <9512011859.AA10898@kristen.llnl.gov> <9512011916.AA04586@ling-ling.lcs.mit.edu.LCS.MIT.EDU> <30BF59D4.1E33@llnl.gov> <9512071830.AA07326@ling-ling.lcs.mit.edu.LCS.MIT.EDU> <30C8AAE4.616E@llnl.gov> Message-ID: <9512131751.AA09345@ling-ling.lcs.mit.edu.LCS.MIT.EDU> Paul sent me this as private comments and I'm going to violate all rules of Netiquette by replying to the entire group. I think that there are some important things in here to comment on. > a. It seems like if I don't do the import of omath then 2.*x doesn't > work. After a from omath import * it does. (?) This is true. You have to import omath somehow, or matrices do not have any numeric behavior (notice that if you use import Matrix, or from Matrix import *. this gets done for you). Here are the reasons... 1) This makes the matrixobject code a lot smaller, and therefore likelier to be included in the base python distribution (fingers crossed) 2) This makes add.reduce(m) possible. 3) This makes it possible to have a vectorized_ofuncmodule (or something like that) that defines high performance versions of these operators for a particular machine, and then have them be used when you say "a+b". > b. Matrix(1,2,3) produces a Matrix_l, or at least that is what it > claims it is. Should be a Matrix_i? This will be seen as a BIG issue, and I'll write more about it later (I've a few other demands on my time right now). It should either be Matrix_b (a matrix of bytes, because 1,2, and 3 can all be stored in a single byte) or it should be Matrix_l (a matrix of longs because a python integer technically holds a C long). > c. I preferred the [1.,2.,3.] rather than the Matrix_d(...). A lot. A > whole lot. Provide a query function for those who really want to know > the flavor of matrix. The printing of matrices can be GREATLY improved. I changed the matrix_print function so that it goes out and calls a python routine to do the actual printing. Check out the PrintMatrix function in Matrix.py. Currently this function just returns the string representation of the object (for which Matrix_d is a good choice modulo naming conventions). Anybody who has good ideas about how to print a matrix can go in and modify this python function to get whatever print behavior they like. If you come up with a good print function, send it to me, and I'll include the best one in the next release. > d. I really like the way Matrix decides on a type. Thanks. > e. I like the idea that others have suggested of IntegerMatrix(...) > instead of a constructor that takes a character to decide type. However, > there are times when the latter is useful because you are COMPUTING the > type (like choose a precision based on the system, for example) so I > wouldn't get rid of the character method. > The fact that Matrix() does the right thing and that everything > coerces mitigates most of this to the margins, however. And here I decided to respond to your message in order to stay out of the naming issue. I like both approaches, and I'm waiting to see what things look like after the dust has settled. -Jim ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From hinsenk@ere.umontreal.ca Wed Dec 13 19:04:10 1995 From: hinsenk@ere.umontreal.ca (Hinsen Konrad) Date: Wed, 13 Dec 1995 14:04:10 -0500 Subject: [PYTHON MATRIX-SIG] Re: Version 2.0 In-Reply-To: <9512131751.AA09345@ling-ling.lcs.mit.edu.LCS.MIT.EDU> (message from James Hugunin on Wed, 13 Dec 95 12:51:32 -0500) Message-ID: <199512131904.OAA20843@cyclone.ERE.UMontreal.CA> This is true. You have to import omath somehow, or matrices do not have any numeric behavior (notice that if you use import Matrix, or from Matrix import *. this gets done for you). Here are the reasons... If omath is essential to the module "matrix", then this module should do the import. All it takes is PyImport_ImportModule("omath"); somewhere in the module initialization. > b. Matrix(1,2,3) produces a Matrix_l, or at least that is what it > claims it is. Should be a Matrix_i? This will be seen as a BIG issue, and I'll write more about it later (I've a few other demands on my time right now). It should either be Matrix_b (a matrix of bytes, because 1,2, and 3 can all be stored in a single byte) or it should be Matrix_l (a matrix of longs because a python integer technically holds a C long). I'd vote for "long" ("IntegerMatrix"). The idea is that the existence of all the types that Python normally doesn't have should be hidden as far as possible; those who need the smaller types to save memory should be the only ones who have to know about it. This guarantees that code like x = Matrix(a,b) x[0] = c will always work correctly as long as a, b, and c are of the same Python type. If the "smallest possible" rule were used, then this would show an unexpected behaviour if a=1, b=2 and c=10000. > c. I preferred the [1.,2.,3.] rather than the Matrix_d(...). A lot. A > whole lot. Provide a query function for those who really want to know > the flavor of matrix. I agree! Anybody who has good ideas about how to print a matrix can go in and modify this python function to get whatever print behavior they like. If you come up with a good print function, send it to me, and I'll include the best one in the next release. The first contest in this SIG! ;-) But before everyone starts coding essentially the same stuff, let's first discuss what the output format should look like and then search a volunteer to do the implementation. The two esential problems to be solved are: 1) How to print arrays of rank > 2. 2) How to deal with large arrays that don't fit on the screen. Personally I have a certain bias towards the APL output format, which is based on the following principles: - all elements of an array are printed using the same format, to make them line up nicely - there is no border decoration whatsoever, just elements separated by spaces - one-dimensional arays are printed in a line, two-dimensional arrays as a sequence of lines, N-dimensional arrays as sequences of (N-1)-dimensional arrays separated by (N-2) blank lines - lines longer than the terminal width (an adjustable parameter) are wrapped to the following line with an indentation of six characters This format is space-saving and very readable. The only problem is that there is no visible difference between a scalar and a higher-dimensional array with exactly one element, but this has not turned out to be a problem in practice. And here I decided to respond to your message in order to stay out of the naming issue. I like both approaches, and I'm waiting to see what things look like after the dust has settled. My suggestion for constructors: Array(sequence, typecode=None) transforms the sequence into an array of the given typecode; if the typecode is None, it determines the type by itself. Typecodes can be letters or type objects (as returned by type()). IntegerArray(sequence) transforms the given sequence into an array of integers. ShortIntegerArray(sequence) transforms the given sequence into an array of C-integers. and so on for the other specific types. Output of repr() should be IntegerArray() etc., output of str() should be the "pretty" format to be discussed separately. ------------------------------------------------------------------------------- Konrad Hinsen | E-Mail: hinsenk@ere.umontreal.ca Departement de chimie | Tel.: +1-514-343-6111 ext. 3953 Universite de Montreal | Fax: +1-514-343-7586 C.P. 6128, succ. Centre-Ville | Deutsch/Esperanto/English/Nederlands/ Montreal (QC) H3C 3J7 | Francais (phase experimentale) ------------------------------------------------------------------------------- ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From jjh@mama-bear.lcs.mit.edu Wed Dec 13 19:20:11 1995 From: jjh@mama-bear.lcs.mit.edu (James Hugunin) Date: Wed, 13 Dec 95 14:20:11 -0500 Subject: [PYTHON MATRIX-SIG] plmodule.c, openglmodule.c References: <9512111511.ZM4023@piranha.appl-math.tu-muenchen.de> Message-ID: <9512131920.AA09380@ling-ling.lcs.mit.edu.LCS.MIT.EDU> It's great to see these modules! This object is primarily intended as a vehicle to make it easier to extend python with these sorts of numerically intensive libraries. The fact that writing them has gone well so far is a great sign. > 5) How was the feedback at the python workshop concernig the matrixmodule. > Would be nice to hear a word or two. Everybody seemed very positive about it. They loved the surface plot you sent me. Many people (including Guido) seemed rather excited when I mentioned that you were also doing an openglmodule "right" using matrices as inputs. A few people seemed skeptical about the speed claims, but hopefully those of you who have been using the object believe me. Check out the main list for a discussion of complex numbers that was motivated by the matrix discussion at the meeting. > 6) Read about the sprse matrix stuff in the paper for the workshop. Is this > making any progress or is it a long term story not been started yet. > Sorry I'm just curious! I'm writing a sparse matrix object in python on top of the matrix object. This is partially a proof of concept, but it is also because my speech recognition code desperately needs sparse matrices. Currently it's good enough for my applications. I'll try and get a rough version of this assembled for public consumption by the 0.21 release of the matrix object. -Jim ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From jjh@mama-bear.lcs.mit.edu Wed Dec 13 19:20:11 1995 From: jjh@mama-bear.lcs.mit.edu (James Hugunin) Date: Wed, 13 Dec 95 14:20:11 -0500 Subject: [PYTHON MATRIX-SIG] plmodule.c, openglmodule.c References: <9512111511.ZM4023@piranha.appl-math.tu-muenchen.de> Message-ID: <9512131920.AA09380@ling-ling.lcs.mit.edu.LCS.MIT.EDU> It's great to see these modules! This object is primarily intended as a vehicle to make it easier to extend python with these sorts of numerically intensive libraries. The fact that writing them has gone well so far is a great sign. > 5) How was the feedback at the python workshop concernig the matrixmodule. > Would be nice to hear a word or two. Everybody seemed very positive about it. They loved the surface plot you sent me. Many people (including Guido) seemed rather excited when I mentioned that you were also doing an openglmodule "right" using matrices as inputs. A few people seemed skeptical about the speed claims, but hopefully those of you who have been using the object believe me. Check out the main list for a discussion of complex numbers that was motivated by the matrix discussion at the meeting. > 6) Read about the sprse matrix stuff in the paper for the workshop. Is this > making any progress or is it a long term story not been started yet. > Sorry I'm just curious! I'm writing a sparse matrix object in python on top of the matrix object. This is partially a proof of concept, but it is also because my speech recognition code desperately needs sparse matrices. Currently it's good enough for my applications. I'll try and get a rough version of this assembled for public consumption by the 0.21 release of the matrix object. -Jim ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From hinsenk@ere.umontreal.ca Wed Dec 13 23:23:46 1995 From: hinsenk@ere.umontreal.ca (Hinsen Konrad) Date: Wed, 13 Dec 1995 18:23:46 -0500 Subject: [PYTHON MATRIX-SIG] Another naming issue Message-ID: <199512132323.SAA07693@cyclone.ERE.UMontreal.CA> While we are discussing names, let me point out another one that is not perfect: omath. From the source code I know that "o" is supposed to mean "optimized", which may well be true, but the characterising feature from a user's point of view is generality (compared to "math"). So how about "gmath" (general math) or "umath" (universal math)? As a motivation for a quick decision, I promise to make two of my Python modules that use Xmath (no, *not* Xmas!) available as soon as there is a definite name. One implements 3D-Vectors (in the sense of geometrical objects, not vectors in the sense of linear algebra), the other automatic differentiation of arbitrary functions. ------------------------------------------------------------------------------- Konrad Hinsen | E-Mail: hinsenk@ere.umontreal.ca Departement de chimie | Tel.: +1-514-343-6111 ext. 3953 Universite de Montreal | Fax: +1-514-343-7586 C.P. 6128, succ. Centre-Ville | Deutsch/Esperanto/English/Nederlands/ Montreal (QC) H3C 3J7 | Francais (phase experimentale) ------------------------------------------------------------------------------- ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From jfulton@usgs.gov Wed Dec 13 23:24:31 1995 From: jfulton@usgs.gov (Jim Fulton, U.S. Geological Survey) Date: Wed, 13 Dec 1995 18:24:31 -0500 Subject: [PYTHON MATRIX-SIG] Re: Version 2.0 In-Reply-To: James Hugunin "[PYTHON MATRIX-SIG] Re: Version 2.0" (Dec 13, 12:51pm) References: <9512011859.AA10898@kristen.llnl.gov> <9512011916.AA04586@ling-ling.lcs.mit.edu.LCS.MIT.EDU> <30BF59D4.1E33@llnl.gov> <9512071830.AA07326@ling-ling.lcs.mit.edu.LCS.MIT.EDU> <30C8AAE4.616E@llnl.gov> <9512131751.AA09345@ling-ling.lcs.mit.edu.LCS.MIT.EDU> Message-ID: <9512131824.ZM11092@dsdbqvarsa.er.usgs.gov> On Dec 13, 12:51pm, James Hugunin wrote: > Subject: [PYTHON MATRIX-SIG] Re: Version 2.0 > > Paul sent me this as private comments and I'm going to violate all rules of > Netiquette by replying to the entire group. I think that there are some > important things in here to comment on. > > > a. It seems like if I don't do the import of omath then 2.*x doesn't > > work. After a from omath import * it does. (?) > > This is true. You have to import omath somehow, or matrices do not have > any numeric behavior (notice that if you use import Matrix, or from Matrix > import *. this gets done for you). Here are the reasons... > > 1) This makes the matrixobject code a lot smaller, and therefore likelier > to be included in the base python distribution (fingers crossed) I don't see that this makes any difference. The matrix module can and should be in the standard "distribution", but it need not be linked statically into the interpreter on systems that support dynamic linking, which includes most or all of the interesting ones. If it is not linked statically into the interpreter, but is available as a dynamically linked module, then the size is not so critical. I don't think the space issue justifies the weird side-effect dependencies indicated above. > > 2) This makes add.reduce(m) possible. This doesn't make any sense to me. But I haven't kept up with this list either. How does importing or not importing a module have any impact on the amove expression or it's feasabity? > 3) This makes it possible to have a vectorized_ofuncmodule (or something > like that) that defines high performance versions of these operators for a > particular machine, and then have them be used when you say "a+b". I don't get this either. Could you elaborate. > > b. Matrix(1,2,3) produces a Matrix_l, or at least that is what it > > claims it is. Should be a Matrix_i? > > This will be seen as a BIG issue, and I'll write more about it later (I've > a few other demands on my time right now). > > It should either be Matrix_b (a matrix of bytes, because 1,2, and 3 can all > be stored in a single byte) or it should be Matrix_l (a matrix of longs > because a python integer technically holds a C long). > > > c. I preferred the [1.,2.,3.] rather than the Matrix_d(...). A lot. A > > whole lot. Provide a query function for those who really want to know > > the flavor of matrix. > > The printing of matrices can be GREATLY improved. I changed the > matrix_print function so that it goes out and calls a python routine to do > the actual printing. Check out the PrintMatrix function in Matrix.py. > Currently this function just returns the string representation of the object > (for which Matrix_d is a good choice modulo naming conventions). > > Anybody who has good ideas about how to print a matrix can go in and modify > this python function to get whatever print behavior they like. If you come > up with a good print function, send it to me, and I'll include the best one > in the next release. > > > d. I really like the way Matrix decides on a type. > > Thanks. > > > e. I like the idea that others have suggested of IntegerMatrix(...) > > instead of a constructor that takes a character to decide type. However, > > there are times when the latter is useful because you are COMPUTING the > > type (like choose a precision based on the system, for example) so I > > wouldn't get rid of the character method. I agree. > > The fact that Matrix() does the right thing and that everything > > coerces mitigates most of this to the margins, however. > > And here I decided to respond to your message in order to stay out of the > naming issue. I like both approaches, and I'm waiting to see what things > look like after the dust has settled. I think both approaches should be provided. Jim ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From jjh@mama-bear.lcs.mit.edu Thu Dec 14 18:43:08 1995 From: jjh@mama-bear.lcs.mit.edu (James Hugunin) Date: Thu, 14 Dec 95 13:43:08 -0500 Subject: [PYTHON MATRIX-SIG] A ramble on ofunc's and omath References: <199512132323.SAA07693@cyclone.ERE.UMontreal.CA> Message-ID: <9512141843.AA09889@ling-ling.lcs.mit.edu.LCS.MIT.EDU> This relates to both Konrad's and Jim's recent posts. Also, this is a bit of a rambling set of comments rather than a coherent argument. I just want people to understand where this weird ofunc thing came from, and what it does now. 1) What is an Optimized FUNCtion? An ofunc is a python object that contains C code to execute a function on arrays of raw C numbers (or on raw python objects, but that's another story) at close to compiled speed. Each function takes an arbitrary number of arguments of given types, and produces an arbitrary number of results of given types. For every desired combination of types that the function should operate on, there is another bit of C code to do the actual computation. 2) Why not bundle it into the matrix object? For one thing, I'd like to eventually clean things up and document them well enough that anybody can create an ofunc. They're really useful little critters for doing very fast computations on arrays of numbers. Also, there are things like generalized reductions that need to have something to grab onto as the function to reduce over. Without ofuncs as real python objects, I don't have any good way to express something like: add.reduce(m) (equivalent to but much faster than reduce(lambda x, y: x+y, m) ). There are also many of these creatures that don't really belong with the matrixobject, but which should behave in exactly the same way as "built-in" functions like add. These include things like hypot. 3) Why the current weird dependency on import omath? I wanted to have a module which defined add and subtract as ofuncs so that they could be used in things like add.reduce. I also wanted to be able to play with different implementations of the ofuncs for different purposes. For example, it should be possible to do the usual numeric tricks of special casing the currently very general inner loops to gain speed improvements. Using the current setup, this can be done by creating a new module, and using the ofuncs for things like add and subtract defined in this new module in the matrix object. Also, I noticed that the matrix object was quite useful without any math functions at all, and I thought that it was a nice coincidence that this arrangement made it possible to have a "stripped-down" matrixobject with no numerical capabilites if you so desired. 4) How this turned into a generalized math package I started to notice a certain duplication of code going on. For example, mathmodule now had a function to take the sin of a double, cmathmodule had one to take the sin of a complex number, and omathmodule (for these ofuncs I'd created) had one to take the sin of a matrix of doubles, complex, or even of generic python objects (that had a sin method). This made it a very small jump to allow a single double or complex to be treated as a 0d matrix, and completely replace the code in mathmodule and cmathmodule with omathmodule. This is what turned into the generic math module that now exists. I have no arguments with giving it a name more fitting of its new role. If nobody has any objections, or better ideas, I'll change the name to "umath" (gxxxx always makes me think of gnu code) for the upcoming bug-fix release. -Jim ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From jfulton@usgs.gov Thu Dec 14 20:03:13 1995 From: jfulton@usgs.gov (Jim Fulton, U.S. Geological Survey) Date: Thu, 14 Dec 1995 15:03:13 -0500 Subject: [PYTHON MATRIX-SIG] A ramble on ofunc's and omath In-Reply-To: James Hugunin "[PYTHON MATRIX-SIG] A ramble on ofunc's and omath" (Dec 14, 1:43pm) References: <199512132323.SAA07693@cyclone.ERE.UMontreal.CA> <9512141843.AA09889@ling-ling.lcs.mit.edu.LCS.MIT.EDU> Message-ID: <9512141503.ZM11806@dsdbqvarsa.er.usgs.gov> Hm. This made be reread: > > a. It seems like if I don't do the import of omath then 2.*x doesn't > > work. After a from omath import * it does. (?) > This is true. You have to import omath somehow, or matrices do not have > any numeric behavior (notice that if you use import Matrix, or from Matrix > import *. this gets done for you). Here are the reasons... Let me guess: x is an object returned from a module other than the matrix module that creates matrix objects, right? In the above example, Matrix has not been imported yet? Then the initialization code for Matrix causes the ofunc stuff to get wired into the matrix type? I'm more and more inclined to think that extension types (types not built into the language) should always be hosted and exported by an extension module and accessed from other modules via the import mechanism. If this were the case, then any extension module that created matrices would have to import the Matrix module to get access to a matrix constructor. The Matrix module would then import other needed modules, like omath. (See additional comment below.) I intend to rewrite FIDL so that modules that it creates import Matrix and use PyObject_GetAttrString to obtain matrix constructors. So if you get a matrix from a FIDL-generated module, Matrix and therefore omath, will have been imported and matrices will have numeric behavior. On Dec 14, 1:43pm, James Hugunin wrote: > Subject: [PYTHON MATRIX-SIG] A ramble on ofunc's and omath > > This relates to both Konrad's and Jim's recent posts. Also, this is a bit > of a rambling set of comments rather than a coherent argument. I just want > people to understand where this weird ofunc thing came from, and what it > does now. > > 1) What is an Optimized FUNCtion? > > An ofunc is a python object that contains C code to execute a function on > arrays of raw C numbers (or on raw python objects, but that's another story) > at close to compiled speed. Each function takes an arbitrary number of > arguments of given types, and produces an arbitrary number of results of > given types. For every desired combination of types that the function > should operate on, there is another bit of C code to do the actual > computation. > > > 2) Why not bundle it into the matrix object? > > For one thing, I'd like to eventually clean things up and document them > well enough that anybody can create an ofunc. They're really useful little > critters for doing very fast computations on arrays of numbers. > > Also, there are things like generalized reductions that need to have > something to grab onto as the function to reduce over. Without ofuncs as > real python objects, I don't have any good way to express something like: > > add.reduce(m) > (equivalent to but much faster than reduce(lambda x, y: x+y, m) ). > > There are also many of these creatures that don't really belong with the > matrixobject, but which should behave in exactly the same way as "built-in" > functions like add. These include things like hypot. > > > 3) Why the current weird dependency on import omath? > > I wanted to have a module which defined add and subtract as ofuncs so that > they could be used in things like add.reduce. > > I also wanted to be able to play with different implementations of the > ofuncs for different purposes. For example, it should be possible to do the > usual numeric tricks of special casing the currently very general inner > loops to gain speed improvements. Using the current setup, this can be done > by creating a new module, and using the ofuncs for things like add and > subtract defined in this new module in the matrix object. > > Also, I noticed that the matrix object was quite useful without any math > functions at all, and I thought that it was a nice coincidence that this > arrangement made it possible to have a "stripped-down" matrixobject with no > numerical capabilites if you so desired. But you now have a weird interface to the matrix module. Assuming that on most systems, the matrix module will be dynamically linked anyway, I wouldn't worry about size. Just let the matrix module import omath so that the user who doesn't need omath can still use matrix math without having to be aware of your implementation decision to use omath. Jim ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From jfulton@usgs.gov Thu Dec 14 23:07:57 1995 From: jfulton@usgs.gov (Jim Fulton, U.S. Geological Survey) Date: Thu, 14 Dec 1995 18:07:57 -0500 Subject: [PYTHON MATRIX-SIG] A ramble on ofunc's and omath In-Reply-To: hinsenk@ERE.UMontreal.CA (Hinsen Konrad) "Re: [PYTHON MATRIX-SIG] A ramble on ofunc's and omath" (Dec 14, 5:33pm) References: <199512142233.RAA15710@cyclone.ERE.UMontreal.CA> Message-ID: <9512141807.ZM12630@dsdbqvarsa.er.usgs.gov> On Dec 14, 5:33pm, Hinsen Konrad wrote: > Subject: Re: [PYTHON MATRIX-SIG] A ramble on ofunc's and omath > > I'm more and more inclined to think that extension types (types not > built into the language) should always be hosted and exported by an > extension module and accessed from other modules via the import > mechanism. If this were the case, then any extension module that > > That is certainly preferable. However, I am not sure whether it > is possible in all cases with the current implementation. Clearly > there is not much provision for C modules that depend on each other. > Your proposed CObjects solve one problem, but there may be more. There may be, but I haven't though of any yet. If you come up with any, I'd be happy to try to come up with a solution to them too. :-] Jim ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From hinsenk@ere.umontreal.ca Thu Dec 14 22:33:36 1995 From: hinsenk@ere.umontreal.ca (Hinsen Konrad) Date: Thu, 14 Dec 1995 17:33:36 -0500 Subject: [PYTHON MATRIX-SIG] A ramble on ofunc's and omath In-Reply-To: <9512141503.ZM11806@dsdbqvarsa.er.usgs.gov> (jfulton@usgs.gov) Message-ID: <199512142233.RAA15710@cyclone.ERE.UMontreal.CA> I'm more and more inclined to think that extension types (types not built into the language) should always be hosted and exported by an extension module and accessed from other modules via the import mechanism. If this were the case, then any extension module that That is certainly preferable. However, I am not sure whether it is possible in all cases with the current implementation. Clearly there is not much provision for C modules that depend on each other. Your proposed CObjects solve one problem, but there may be more. But you now have a weird interface to the matrix module. Assuming that on most systems, the matrix module will be dynamically linked anyway, I wouldn't worry about size. Just let the matrix module import omath so that the user who doesn't need omath can still use matrix math without having to be aware of your implementation decision to use omath. I agree. omath should be usable without matrix, but the reverse is not so important. ------------------------------------------------------------------------------- Konrad Hinsen | E-Mail: hinsenk@ere.umontreal.ca Departement de chimie | Tel.: +1-514-343-6111 ext. 3953 Universite de Montreal | Fax: +1-514-343-7586 C.P. 6128, succ. Centre-Ville | Deutsch/Esperanto/English/Nederlands/ Montreal (QC) H3C 3J7 | Francais (phase experimentale) ------------------------------------------------------------------------------- ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From hinsenk@ere.umontreal.ca Fri Dec 15 14:05:15 1995 From: hinsenk@ere.umontreal.ca (Hinsen Konrad) Date: Fri, 15 Dec 1995 09:05:15 -0500 Subject: [PYTHON MATRIX-SIG] A ramble on ofunc's and omath In-Reply-To: <9512141807.ZM12630@dsdbqvarsa.er.usgs.gov> (jfulton@usgs.gov) Message-ID: <199512151405.JAA23256@cyclone.ERE.UMontreal.CA> > That is certainly preferable. However, I am not sure whether it > is possible in all cases with the current implementation. Clearly > there is not much provision for C modules that depend on each other. > Your proposed CObjects solve one problem, but there may be more. There may be, but I haven't though of any yet. If you come up with any, I'd be happy to try to come up with a solution to them too. :-] It seems that you can do everything necessary with these CObjects. After all, you can make all non-static variables and functions known in this way, and then you have published everything that other statically linked C modules have access to. A potential problem is that another module can do whatever it wants with the CObjects, including something wrong (like calling a variable). This can be avoided (except for intentional misuse) by providing a header file that decribes the public features of a module. One could produce a tool (in Python, of course!) that produces such a header file with the prototypes of the original module code. It could be made so easy to use that except for an additional import call at the beginning, another module could use all functions and variables exactly as if they were statically linked. And then one could pack the whole interface for a module into a single CObject, which helps keeping a module's name space clean. ------------------------------------------------------------------------------- Konrad Hinsen | E-Mail: hinsenk@ere.umontreal.ca Departement de chimie | Tel.: +1-514-343-6111 ext. 3953 Universite de Montreal | Fax: +1-514-343-7586 C.P. 6128, succ. Centre-Ville | Deutsch/Esperanto/English/Nederlands/ Montreal (QC) H3C 3J7 | Francais (phase experimentale) ------------------------------------------------------------------------------- ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From jfulton@usgs.gov Fri Dec 15 14:40:16 1995 From: jfulton@usgs.gov (Jim Fulton, U.S. Geological Survey) Date: Fri, 15 Dec 1995 09:40:16 -0500 Subject: [PYTHON MATRIX-SIG] A ramble on ofunc's and omath In-Reply-To: hinsenk@ERE.UMontreal.CA (Hinsen Konrad) "Re: [PYTHON MATRIX-SIG] A ramble on ofunc's and omath" (Dec 15, 9:05am) References: <199512151405.JAA23256@cyclone.ERE.UMontreal.CA> Message-ID: <9512150940.ZM14205@dsdbqvarsa.er.usgs.gov> On Dec 15, 9:05am, Hinsen Konrad wrote: > Subject: Re: [PYTHON MATRIX-SIG] A ramble on ofunc's and omath > > > That is certainly preferable. However, I am not sure whether it > > is possible in all cases with the current implementation. Clearly > > there is not much provision for C modules that depend on each other. > > Your proposed CObjects solve one problem, but there may be more. > > There may be, but I haven't though of any yet. If you come up with any, I'd be > happy to try to come up with a solution to them too. :-] > > It seems that you can do everything necessary with these CObjects. After > all, you can make all non-static variables and functions known in this > way, Actually, with CObjects, all of the exported C objects can be static. So essentially, the only non-static variable in a module is the module initialization function. This is a plus when the module is statically linked, because you don't have to worry about global name colisions. You get the benefits of Python's namespace mechanisms even for your C code. :-) > and then you have published everything that other statically > linked C modules have access to. > > A potential problem is that another module can do whatever it wants > with the CObjects, including something wrong (like calling a variable). This is a problem with C in general. Of course ... > This can be avoided (except for intentional misuse) by providing > a header file that decribes the public features of a module. Absolutely. > One could produce a tool (in Python, of course!) that produces such > a header file with the prototypes of the original module code. > It could be made so easy to use that except for an additional import > call at the beginning, another module could use all functions and > variables exactly as if they were statically linked. And then one > could pack the whole interface for a module into a single CObject, > which helps keeping a module's name space clean. Yup, there are plenty of opportunities to build on and improve the basic mechanism. Alot of this depends on how complex the C interface being exported is. Jim ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From dubois1@llnl.gov Fri Dec 15 19:45:37 1995 From: dubois1@llnl.gov (Paul. Dubois) Date: Fri, 15 Dec 1995 11:45:37 -0800 Subject: [PYTHON MATRIX-SIG] Re: Version 2.0 References: <199512131904.OAA20843@cyclone.ERE.UMontreal.CA> Message-ID: <30D1D061.5045@llnl.gov> In a previous message I asked (stupidly) > b. Matrix(1,2,3) produces a Matrix_l, or at least that is what it > claims it is. Should be a Matrix_i? This question provoked a long response from Jim which taught me that I had simply made a mistake; I didn't know that a Python integer corresponded to a C long. If it does, Matrix_l is correct. Sorry. -- Paul F. Dubois, L-472 (510)-422-5426 Lawrence Livermore National Laboratory FAX (510)-423-9969 Livermore, CA 94550 dubois1@llnl.gov Consulting: PaulDubois@aol.com ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From hinsenk@ere.umontreal.ca Sat Dec 16 16:09:38 1995 From: hinsenk@ere.umontreal.ca (Hinsen Konrad) Date: Sat, 16 Dec 1995 11:09:38 -0500 Subject: [PYTHON MATRIX-SIG] A ramble on ofunc's and omath In-Reply-To: <9512150940.ZM14205@dsdbqvarsa.er.usgs.gov> (jfulton@usgs.gov) Message-ID: <199512161609.LAA02631@cyclone.ERE.UMontreal.CA> Yup, there are plenty of opportunities to build on and improve the basic mechanism. Alot of this depends on how complex the C interface being exported is. Variables and functions are all there is to export; anything else (typedefs, macros etc.) would have to be exported via header files even in the case of statically linked C programs. If something like this is to become an official feature of Python it would be nice to add a minimum of support to CObjects, even if that increases the code size too 300 bytes ;-) The aim should be to make this feature as easy to use as possible and reduce potential problems to a minimum. For example, one could have a standard typedef for a structure that contains the complete C interface to a module. Plus a standard special name for the variable that holds the C interface (perhaps "__C__"), and an access function that looks for this variable and retrieves the pointer. Then add conventions for header files, and the only difference between this mechanism and acess to a statically linked module is two lines of code: one for the import, and one to retrieve the C interface. ------------------------------------------------------------------------------- Konrad Hinsen | E-Mail: hinsenk@ere.umontreal.ca Departement de chimie | Tel.: +1-514-343-6111 ext. 3953 Universite de Montreal | Fax: +1-514-343-7586 C.P. 6128, succ. Centre-Ville | Deutsch/Esperanto/English/Nederlands/ Montreal (QC) H3C 3J7 | Francais (phase experimentale) ------------------------------------------------------------------------------- ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From Emmanuel.Viennet@laforia.ibp.fr Sun Dec 17 22:45:18 1995 From: Emmanuel.Viennet@laforia.ibp.fr (VIENNET Emmanuel 48.06.27.97 Equipe Gallinari) Date: Sun, 17 Dec 1995 23:45:18 +0100 Subject: [PYTHON MATRIX-SIG] bug: assignment from string Message-ID: <199512172245.XAA15748@lpia2.ibp.fr> While playing with some matrices of char: ----------------------------------------------------------------------------- [~/softs/Python-1.3/Modules/]% python Python 1.3 (Dec 17 1995) [GCC 2.5.8] Copyright 1991-1995 Stichting Mathematisch Centrum, Amsterdam >>> import Matrix >>> >>> m1 = Matrix.fromString( '\001\002\003\004', 4, 'c' ) >>> >>> m2 = m1.reshape( 2, 2 ) >>> >>> m2[1,1] '\004' >>> >>> m2[1,1] = 'e' Segmentation fault (core dumped) [~/softs/Python-1.3/Modules/]% gdb ../python core GDB 4.13 (i486-slackware-linux), Copyright 1994 Free Software Foundation, Inc... Core was generated by `python'. #0 0x64fc4 in optimize_slices (dest_strides=0xbffff37c, dest_dimensions=0xbffff378, dest_nd=0xbffff374, src_strides=0xbffff370, src_dimensions=0xbffff36c, src_nd=0xbffff368, elsize=0xbffff364, copies=0xbffff360) at matrixobject.c:127 127 if (((*dest_strides)[*dest_nd-1] == *elsize) && ((*src_strides)[*src_nd-1] == *elsize)) { (gdb) where #0 0x64fc4 in optimize_slices (dest_strides=0xbffff37c, dest_dimensions=0xbffff378, dest_nd=0xbffff374, src_strides=0xbffff370, src_dimensions=0xbffff36c, src_nd=0xbffff368, elsize=0xbffff364, copies=0xbffff360) at matrixobject.c:127 #1 0x68949 in PyMatrix_CopyMatrix (dest=0x164080, src=0x164200) at matrixobject.c:262 #2 0x68a21 in PyMatrix_CopyObject (dest=0x164080, src_object=0x187b20) at matrixobject.c:278 #3 0x6a0a1 in matrix_ass_sub (self=0x164140, index=0x174600, op=0x187b20) at matrixobject.c:673 #4 0x3531f in assign_subscript (w=0x164140, key=0x174600, v=0x187b20) at ceval.c:2600 ----------------------------------------------------------------------------- The problem is with the string in the rhs: m2[1,1] = 'e' (m1[1,1] = 66 works). I suppose the bug is in matrix_ass_sub(), but I can't figure where. Also, I wonder why can't we specify all the dimensions as arguments to fromString(), avoiding to call reshape() ? Emmanuel ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From jjh@mama-bear.lcs.mit.edu Mon Dec 18 20:56:36 1995 From: jjh@mama-bear.lcs.mit.edu (James Hugunin) Date: Mon, 18 Dec 95 15:56:36 -0500 Subject: [PYTHON MATRIX-SIG] bug: assignment from string References: <199512172245.XAA15748@lpia2.ibp.fr> Message-ID: <9512182056.AA12077@ling-ling.lcs.mit.edu.LCS.MIT.EDU> > (Core dump data ommitted) As a small point, these detailed listings of the results of a core dump are probably not at all useful to any member of this list except myself (to whom the are EXTREMELY helpful) so it might be a better idea to just send them directly to me and I'll fix them ASAP. However, I don't want to say anything that will discourage people from sending in bug reports. The other comments are great for the list. > The problem is with the string in the rhs: m2[1,1] = 'e' > (m1[1,1] = 66 works). > > I suppose the bug is in matrix_ass_sub(), but I can't figure where. There's actually something rather strange going on here, and while it shouldn't dump core, my current interpretation of strings says that this should raise an exception. Here's what's going on: 'hello' gets converted to a 5-length vector ['h', 'e', 'l', 'l', 'o'] 'e' gets converted to a 1-length vector ['e'] 66 is a 0-dimensional scalar So, m[1,1] = 'e' is trying to assign a 1-length vector to a 0-dimensional scalar, which is an exception. I can fix this by treating 1-length strings as 0-dimensional scalars if you think this is the appropriate choice. Note: I NEVER use arrays of strings personally, so I'm depending on the other people on this list to tell me how you want them to behave. I can't imagine a use for arrays of strings except for interfacing to existing libraries. > Also, I wonder why can't we specify all the dimensions as arguments to > fromString(), avoiding to call reshape() ? This is a simple oversight on my part. I'm not convinced that the current form of "fromString" is exactly the way it should finally be, and I'm reluctant to churn out C-code for an interface that might change. Nevertheless, I'll add a function to Matrix.py to fix this in the next release. -Jim ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From jfulton@usgs.gov Tue Dec 19 14:50:26 1995 From: jfulton@usgs.gov (Jim Fulton, U.S. Geological Survey) Date: Tue, 19 Dec 1995 09:50:26 -0500 Subject: [PYTHON MATRIX-SIG] bug: assignment from string In-Reply-To: James Hugunin "Re: [PYTHON MATRIX-SIG] bug: assignment from string" (Dec 18, 3:56pm) References: <199512172245.XAA15748@lpia2.ibp.fr> <9512182056.AA12077@ling-ling.lcs.mit.edu.LCS.MIT.EDU> Message-ID: <9512190950.ZM19947@dsdbqvarsa.er.usgs.gov> On Dec 18, 3:56pm, James Hugunin wrote: > Subject: Re: [PYTHON MATRIX-SIG] bug: assignment from string > (snip) > The other comments are great for the list. > > > The problem is with the string in the rhs: m2[1,1] = 'e' > > (m1[1,1] = 66 works). > > > > I suppose the bug is in matrix_ass_sub(), but I can't figure where. > > There's actually something rather strange going on here, and while it > shouldn't dump core, my current interpretation of strings says that this > should raise an exception. >> > Here's what's going on: > > 'hello' gets converted to a 5-length vector ['h', 'e', 'l', 'l', 'o'] > 'e' gets converted to a 1-length vector ['e'] > 66 is a 0-dimensional scalar > > So, m[1,1] = 'e' is trying to assign a 1-length vector to a 0-dimensional > scalar, which is an exception. > > I can fix this by treating 1-length strings as 0-dimensional scalars if you > think this is the appropriate choice. Yes. This is necessary because Python does not have a character type. 1-length strings must be used as "characters". Witness the fact that 'hello'[1] wants to return a character, but returns a string instead. Jim ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From stoll@atr-sw.atr.co.jp Wed Dec 20 08:32:39 1995 From: stoll@atr-sw.atr.co.jp (Perry A. Stoll) Date: Wed, 20 Dec 1995 17:32:39 +0900 Subject: [PYTHON MATRIX-SIG] __doc__ string problems? Message-ID: <199512200832.RAA18484@ciris21.atr-sw.atr.co.jp> Matrix-sig'ers, Sorry to disturb everyone. Is anyone else having a problem using __ doc__ strings since installing the Matrix module? Before rebuilding, I thought I'd check with others on this list. Could it be due to some of the changes in the distribution? As a short example, the following does nothing in my python with the Matrix library but works fine on the Mac: def foo(x): """Some documentation string.""" return x foo.__doc__ Ideas? -Perry Stoll Perry A. Stoll Research Associate, CSRL Advanced Telecommunications Research 2-2 Hikaridai, Seika-cho Soraku-gun, Kyoto 619-02 JAPAN VOICE: +81-774-95-1217 FINGER: stoll@atrwide.atr.co.jp FAX : +81-774 95-1208 EMAIL: stoll@atr-sw.atr.co.jp PGP 2.6 Key fingerprint = AF 56 5C D8 5F 78 BA FD 21 6E 2A 68 C4 55 9E B0 ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From hinsen@physik.rwth-aachen.de Thu Dec 21 15:02:53 1995 From: hinsen@physik.rwth-aachen.de (Konrad Hinsen) Date: Thu, 21 Dec 1995 16:02:53 +0100 Subject: [PYTHON MATRIX-SIG] __doc__ string problems? In-Reply-To: <199512200832.RAA18484@ciris21.atr-sw.atr.co.jp> (stoll@atr-sw.atr.co.jp) Message-ID: <9512211502.AA06882@acds21> Sorry to disturb everyone. Is anyone else having a problem using __ doc__ strings since installing the Matrix module? Before rebuilding, I My fault! It has to do with the power operator patch. I have already sent the fix to Guido, but obviously forgotten that other people on the Matrix SIG might need it. You have to add one line in the function get_docstring() in Python/compile.c; I'll include the full function: static object * get_docstring(n) node *n; { int i; switch (TYPE(n)) { case suite: if (NCH(n) == 1) return get_docstring(CHILD(n, 0)); else { for (i = 0; i < NCH(n); i++) { node *ch = CHILD(n, i); if (TYPE(ch) == stmt) return get_docstring(ch); } } break; case file_input: for (i = 0; i < NCH(n); i++) { node *ch = CHILD(n, i); if (TYPE(ch) == stmt) return get_docstring(ch); } break; case stmt: case simple_stmt: case small_stmt: return get_docstring(CHILD(n, 0)); case expr_stmt: case testlist: case test: case and_test: case not_test: case comparison: case expr: case xor_expr: case and_expr: case shift_expr: case arith_expr: case term: case factor: case power: if (NCH(n) == 1) return get_docstring(CHILD(n, 0)); break; case atom: if (TYPE(CHILD(n, 0)) == STRING) return parsestrplus(n); break; } return NULL; } The new line is "case power:". Briefly, I had to introduce a new node type to give the power operator a higher priority than multiplication, and get_docstring() needs a complete list of node types in its case statements in order to work correctly. -- ------------------------------------------------------------------------------- Konrad Hinsen | E-Mail: hinsenk@ere.umontreal.ca Departement de Chimie | Tel.: +1-514-343-6111 ext. 3953 Universite de Montreal | Fax: +1-514-343-7586 C.P. 6128, succ. A | Deutsch/Esperanto/English/Nederlands/ Montreal (QC) H3C 3J7 | Francais (phase experimentale) ------------------------------------------------------------------------------- ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org =================