From hinsenk@ere.umontreal.ca Mon Oct 2 17:27:22 1995 From: hinsenk@ere.umontreal.ca (Hinsen Konrad) Date: Mon, 2 Oct 1995 12:27:22 -0400 Subject: [PYTHON MATRIX-SIG] Time for recap? In-Reply-To: <199509301807.OAA22282@monty> (message from Guido van Rossum on Sat, 30 Sep 1995 14:07:39 -0400) Message-ID: <199510021627.MAA14222@cyclone.ERE.UMontreal.CA> Ah. I was writing from memory, and forgot this feature. I don't like it. How many times do you have to select a more or less arbitrary group of rows/columns from a matrix? It makes the slice representation Quite often, if you use matrices as tables of numbers. bulkier -- contiguous slices can be stored (in C) as 4 ints per dimension, while random selections will require a variable-length list of indices per dimension. (It is also wasteful to have to generate the full range of numbers when what you mean is a contiguous slice.) Maybe there should be two internal representations, one for contigous blocks (i.e. submatrices) and one for arbitrary slices. A tuple is just one example of a sequence in Python. Lists are another example. In many situations, any sequence is acceptable and the results are the same (e.g. for loops). (And in situations where only lists or only tuples are accepted by the current version of the language, Steve Majewski has often made the point that there is no inherent reason why only one type should be accepted and that this should be fixed. I agree in most cases.) Actually, I have never really understood why Python needs both tuples and lists. Anything you can with tuples you can do with lists, so why have tuples? Instead of supporting a[[2,3,5]] to select elements 2, 3 and 5 from a, I would propose to use filter() or a multi-dimensional extension thereof if you want to access selected subarrays. Or perhaps just a You mean actually having to iterate over the whole array just to pick some element? That sounds a bit wasteful. One problem I am seeing in this discussion is that indexing is being treated separately from other array operations, although it is really only one structural function among many (reshaping, transposing, etc.). We should rather discuss the complete set of structural functions together; they should behave consistently and together allow all array manipulations that might occur, no matter which can be done by "indexing" and which by something with another name. If you look at my Array implementation, you will see that there indexing is just syntactic sugar for a structural function "take" that selects an arbitrary set of items from its argument. It has the useful property that the shape of the result is the combination of the shape of the "index" argument and the shape of the items of the "data" argument. That gives great flexibility in selecting subarrays, and provides an easy-to-understand behaviour even in complicated cases. In my implementation, indexing itself is limited, since you can't specify rank, but I don't really care, since "take" lets me do whatever I want. > It wouldn't be too hard to expand the definition of basic type to > include (PyObject *)'s if you'd like to have the possibility of a > matrix of "real" python objects. Yes, the latter is definitely something that should be possible even if my idea doesn't find the acceptance I hope it will get. You can add support from me ;-) General arrays would be a useful feature without causing much effort. The main problem I see is how to specify whether e.g. a constant array of integers is supposed to be an array of integers or a general array whose initial values happen to be integers. But this can be overcome with explicit conversion, if necessary. ------------------------------------------------------------------------------- Konrad Hinsen | E-Mail: hinsenk@ere.umontreal.ca Departement de chimie | Tel.: +1-514-343-6111 ext. 3953 Universite de Montreal | Fax: +1-514-343-7586 C.P. 6128, succ. A | Deutsch/Esperanto/English/Nederlands/ Montreal (QC) H3C 3J7 | Francais (phase experimentale) ------------------------------------------------------------------------------- ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From hinsenk@ere.umontreal.ca Mon Oct 2 18:08:11 1995 From: hinsenk@ere.umontreal.ca (Hinsen Konrad) Date: Mon, 2 Oct 1995 13:08:11 -0400 Subject: [PYTHON MATRIX-SIG] Time for recap? In-Reply-To: <199509301830.OAA22332@monty> (message from Guido van Rossum on Sat, 30 Sep 1995 14:30:24 -0400) Message-ID: <199510021708.NAA15986@cyclone.ERE.UMontreal.CA> I just meant to end the debate about whether a*b should mean matrix multiplication in the LA sense or elementwise multiplication like APL/J. This only an issue for * and /. For / most people agree that ... OK, on that I agree, of course. I thought you were referring to functions like matrix inversion. I don't see much of a problem with that. Functions/methods that take an array and return a like-shaped array should always copy their argument before modifying it. Methods that are supposed to modify an array in place should not also return a reference to the array. That seems a good way to make the distinction clear. ------------------------------------------------------------------------------- Konrad Hinsen | E-Mail: hinsenk@ere.umontreal.ca Departement de chimie | Tel.: +1-514-343-6111 ext. 3953 Universite de Montreal | Fax: +1-514-343-7586 C.P. 6128, succ. A | Deutsch/Esperanto/English/Nederlands/ Montreal (QC) H3C 3J7 | Francais (phase experimentale) ------------------------------------------------------------------------------- ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From chris.chase@jhuapl.edu Tue Oct 3 17:47:47 1995 From: chris.chase@jhuapl.edu (Chris Chase S1A) Date: Tue, 3 Oct 1995 12:47:47 -0400 Subject: [PYTHON MATRIX-SIG] Mutli-dimensional indexing and other comments Message-ID: <199510031645.MAA06249@python.org> This is a long note. I have tried to go back and read through the various matrix/array proposals from the start of this list. I have some comments on various proposals and opinions. I also would like to make some suggestions about multi-dimensional indexing. ---- Hinsen Konrad's proposal: - Like arrays in J, my arrays are immutable, i.e. there is no provision for changing individual elements. I find that changinng individual elements a necessity in almost all the computations that I do. For example, this kind of operation is pervasive in imaging applications (e.g. segmentation and region-of-interest manipulations) and block-matrix linear algebra computations. For me, setting elements is a must. ----- Array functions: Hinsen Konrad supports calculator sytle array usage, i.e. interactive computation: I am typing things like sqrt(Array("file1"))*Array("file2) which I prefer a lot to Array.sqrt(Array.Array("file1"))*Array.Array("file2) I definitely prefer the first sytle for interactive computations. I also like Konrad's array function prototype that allows rank specification, e.g. product.over[1](b). Guido did not like the rank specification via an index argument. Could the rank be overriden using a keyword argument instead of an index argument, e.g. product.over(b,rank=1)? Since keywords are commonly used to override default arguments this would seem a natural possibility. ------ Heirarchical (list-like) indexing: Jim Fulton's proposal thinks of multi-dimensional arrays as being heirarchical so that a[i] returns a N-1 dimensioned array. This allows him to use list style indexing, e.g. a[i][j]. This approach would seem to rule out flatten indexing (one-dimensional indexing) and multi-dimensional slicing, e.g. would a[2:9][1:5] just end up being equivalent to a[3:7]? Perhaps that is what is wanted. If the heirarchical approach is used then a different mechanism would be necessary for more general multi-dimensional indexing. I could see that heirarchical indexing like list indexing, would be useful. My opinion is that arbitrary multi-dimensional slicing is more useful for the homogeneous arrays we are discussing (they are not lists). Perhaps both could be supported. ---- I am confused about proposals for array slices to be references. In Jim's proposal, assignment is by copy and access is by reference. Then am I correct in understanding that for b=a[i] (a is two-dimensional) changing elements of b will not change a? ----- General Multi-dimensional indexing: In general, I have not been able to keep straight the various proposals. It seems that most issues have been concerned with implementation issues or with limiting extensions so they will not break the current language. Instead of diving into the implementation questions I would like to present my views on multi-dimensional indexing for arrays. Then I would like to know how the current proposals fit into this view. I have worked with or investigated a number of interactive array-oriented computational languages: IDL, Matlab, Tela, scilab, Octave, rlab, Mathematica, APL. I have seen the following indexing concepts all of which are very useful and completely generalize array access. Like Konrad, I will use the term "rank" to refer to the number of dimensions of an array and "shape" to refer to the ordered list of dimension sizes of an array. The arrays are homogeneous and can be viewed as one-dimensional (as in C or Fortran) or multi-dimensional. index vector: a scalar, slice (with optional stride), or array. An array index could be our new array (matrix) object, tuple or list. Let A be a multi-dimensional array (rank is 3 in the examples) to be indexed and let a1, a2, a3 be arbitrary index vectors. one-dimensional indexing: a) flattened indexing. Takes a single index vector and the array is viewed like contiguously-stored arrays in Fortran or C (one or the other, but consistent in that always the first or last dimension changes most rapidly.) The shape of the result is the same as the shape of the index vector. b (not really 1-D?) heirachical indexing (Jim Fulton's list-like indexing): Mentioned above. The index vector is used only for the first dimension. I have never used this type of indexing for homogeneous numeric arrays (because it is a special case of the product indexing), but I see that it could be useful when viewing the array as a list. For a scalar index the result has rank one less than A. If a non-scalar is used as an index vector then the index vector would be treated in flattened form and the result would have have the same rank as A. Multi-dimensional indexing: The number of index vectors is equal to the rank A with one index vector for each dimension. There are two types of indexing that I have seen: a) product indexing: You might call this arbitrary slice indexing. I sometimes call this Cartesian product indexing because all combinations of the index vector elements are used for indexing. All index vectors are used in flattened form. The result has the same rank but the size of each dimension is equal to the length of the corresponding index vector. The elements of the result are taken from all possible ordered combinations of the index vector elements, e.g. the result element at index i,j,k is taken from A at index a1(i),a2(j),a3(k). b) mapped indexing (as in Tela): The index vectors all have the same shape and are used in flattened form. Indexes into array A are generated from ordered elementwise grouping, i.e. the result at 1-D index i is taken from A at index a1(i),a2(i),a3(i). The shape of the result is the same as the shape of the index vectors. Selection indexing (as in Octave or APL): A type of 1-D indexing where the index vector is a {0,1} vector that has the same number of elements as A. The result contains only those elements of A where the corresponding index vector is non-zero. Rather than support this directly, many languages, e.g. IDL and Matlab, support this indirectly with a "where" or "find" function. where(A) returns a rank 1 array containing in increasing order the one-dimensional indexes where A is nonzero. The result of where(A) can then be used for 1-D indexing. This is more powerful then supporting selection directly. Insertion indexing (as in IDL): Used in setting blocks of items. Takes a 1-D or multi-dimensional scalar index that specifies the starting position for the object insertion, overwriting existing items. For a 1-D scalar index the object inserted is used in flattened form. For a multi-dim index the object inserted must have the same rank as A. When the inserted object would extend beyond the dimension bounds of A there are several possible behaviors: signal error, index wrap-around, or truncation of A to fit. In most of the languages that I have used, the syntax A[] is used for both 1-D and multi-dimensional product/slicing indexing. Sometimes, 1-D indexing is used when only a single index vector is given. Konrad suggests that this is just syntactic sugar in place of function calls like "take" and "ravel". But it is extremely useful for writing clear, readable code (even APL acknowledges this by supporting "[]" syntax for indexing). A natural solution syntactically would be: 1. "[]" subscripting to support 1-D indexing and product indexing depending on the number of index vectors given. 2. allowing ":" slice notation for index vectors. Mapped indexing, heirarchical indexing, insertion and where/selection could be access methods. It does not seem possible without major internal Python changes to implement both 1) and 2) because of the ingrained and non-extensible 1-D indexing built-in to Python. Someone (?) suggested that arbitrary (product) indexing did not seem useful and slicing was sufficient. For my personal interests, product indexing is necessary for a huge number of applications. I use mapped indexing less often, but it is similarly necessary in many applications. Of course all indexing can be converted manually into 1-D indexing, but this would make an array extension almost unuseable for interactive use. ------ Question: Do any of the proposals completely support these types of indexing in some form? The closest I have seen to supporting product indexing is James Hugunin proposal of using a mapping type: M[(range(1,3),range(2,4))] -> [[13,14],[23,24]] The idea suggests wrapping the index arguments of [] into a tuple. However, this would not allow both 1-D and multi-dimensional indexing for the same array without possible ambiguity. To avoid the extra parentheses Guido suggests: - Allowing M[i, j] for (multidimensional) sequence types would also mean that D[i, j] would be equivalent to D[(i, j)] for dictionaries. But Jim Fulton: I see no reason to support M[i,j] for arbitrary sequence types. I'd say that if a type wants to support multiple arguments to [], then it should provide mapping behavior and have the mapping implementation sniff for either an integer or a tuple argument and do the right thing. I am *very much* against a language change to support this. One complex solution that preserves the current Python language: Treat M[i,j] as multiple dimension indexing and M[(i,j)] as 1-D indexing. Think of M[i,j] as a function call with 2 arguments and M[(i,j)] as a function call with a single argument. As specified by the current language, M[(i,j)] is a mapping type, but (i,j) is treated as a one-dimensional index vector selecting items i and j. On the other hand, when multiple index arguments are used, e.g. M[i,j], a different multi-dimension index method taking a variable length argument list is called, e.g. __msetitem__ or __mgetitem__. This allows both 1-D and multi-dim indexing for the same object that _preserves_ existing language behavior. This also overcomes the discussion of about bundling the index arguments into a tuple, e.g. a[1,], which I think could be a source of errors and confusion. Of course, this solution requires large changes to Python internals. ------------- Multi-dimensional slicing: If possible I would prefer slice notation over range(). Using range() is certainly not as clean as ":". ":" is more than syntaic sugar since it will use dimension length information for the object that is not available to range(). With range() how do I specify the entire dimension or dimension length for the upper bound? For large array expressions the repetitive range() can generate a lot of clutter. For example: a[1:3,:,5:] == a[range(1:3),range(0,shape(a)[1])),range(5,shape(a)[2])] The left side is much more readable at a glance. Then imagine if this was part of a larger expression. Of course this could have been made a little simpler by storing the shape in an auxillary variable: s = shape(a) then: a[1:3,:,5:] == a[range(1:3),range(0,s[1])),range(5,len(s[2]))] To support slice ":" for multi-dimensions Guido suggests an expression like (2:3). But Jim Fulton replies: Can't be, (2:3) is not a valid expression, so it can't yield a valid element of the tuple. Additionally, allowing (2:3) expressions in place of range() expressions would break old slice usage. A possible complex solution to allow ":" multi-dimensional expressions with backwards compatibility: To make slice expressions valid there would have to be some kind of built-in slice type. For example, it could just be a tuple subclass with the only difference being a type name of "slice". When used for multi-dim indexing the indexing function would have to support scalars, arrays or slices for the indexes by checking the type of the index. For backwards compatibility, when using one-dimensional indexing M[2:3] would call __getslice__(2,3) by unpacking the slice tuple. When using slices for multi-dimensional indexes, supporting negative upper bounds would have to be a built-in dimension length function so that the dimension upper bound would be available, e.g. when computing M[2,3:-1]. If negative upper bounds could be forfeited for multi-dim slices then the dimension length function would not be necessary ("None" could be use for the upper bound in "2:"). If such additions to Python are just too much work, then I think Jim's range() workaround is the next best thing. ------ The solutions above to allow ":" and 1-D plus multi-dimensional indexing are not simple to implement. They would require major internal changes to the compiler in addition to other areas. But they would _preserve_ backwards compatibility with the current language. They are complex because the original language was not designed with extensibility to completely general multi-dimensional indexing in mind. General opinion about possible Python array/matrix extensions: Trying to implement multi-dimensional arrays without any changes to Python internals could end up being cumbersome, inelegant workarounds that may be confusing and discouraging to end users (especially those coming from numeric array languages like APL, Matlab, IDL, Tela, etc). The extensions would most likely not be completely general. Why should we limit the hamstring the capabilities of an array extension just to avoid internal Python changes? Additionally, while they might be useable in scripts, they will be inefficient for extensive interactive use. Chris Chase ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From guido@CNRI.Reston.VA.US Wed Oct 4 02:58:07 1995 From: guido@CNRI.Reston.VA.US (Guido van Rossum) Date: Tue, 03 Oct 1995 21:58:07 -0400 Subject: [PYTHON MATRIX-SIG] Mutli-dimensional indexing and other comments In-Reply-To: Your message of "Tue, 03 Oct 1995 12:47:47 EDT." <199510031645.MAA06249@python.org> References: <199510031645.MAA06249@python.org> Message-ID: <199510040158.VAA05077@monty> > I also like Konrad's array function prototype that allows rank > specification, e.g. product.over[1](b). Guido did not like the rank > specification via an index argument. Could the rank be overriden > using a keyword argument instead of an index argument, > e.g. product.over(b,rank=1)? Since keywords are commonly used to > override default arguments this would seem a natural possibility. Sounds good to me, though I'm still struggling with the concent of the rank of an operator... > Heirarchical (list-like) indexing: > > Jim Fulton's proposal thinks of multi-dimensional arrays as being > heirarchical so that a[i] returns a N-1 dimensioned array. This > allows him to use list style indexing, e.g. a[i][j]. This approach > would seem to rule out flatten indexing (one-dimensional indexing) and > multi-dimensional slicing, e.g. would a[2:9][1:5] just end up being > equivalent to a[3:7]? Yes. > Perhaps that is what is wanted. Not necessarily, but it's the only thing that's possible given the various constraints and the current language implementation. While a[i,j] could easily be added (meaning a[i][j]), a[i:j,k:l] would require major re-engineering of a complicated piece of code. I proposed a.slice((i,j), (k,l)) to express this. > I am confused about proposals for array slices to be references. In > Jim's proposal, assignment is by copy and access is by reference. > Then am I correct in understanding that for b=a[i] (a is > two-dimensional) changing elements of b will not change a? No. Assignment to a simple name (e.g. "b") will always be by reference, this is fundamental in the language. However, *slice* assignment has to copy (e.g. b[i:-1j] = a[i-1:j]), and I have proposed that slice references (a[i:j] used in an expression) of arrays return a reference to the original elements of a. E.g. after b = a[i:j], assignment to elements of b will change the corresponding elements of a. This is different from slice assignments for Python lists, but seems to be the most useful rslice semantics for arrays. > In general, I have not been able to keep straight the various > proposals. It seems that most issues have been concerned with > implementation issues or with limiting extensions so they will not > break the current language. Indeed, such are the practicalities of extending a language that has been used and developed extensively by/for thousands of users over the past five years. > Instead of diving into the implementation questions I would like to > present my views on multi-dimensional indexing for arrays. Then I > would like to know how the current proposals fit into this view. An approach that I like, at least in theory. [What followed was too long for me to digest completely.] > Why should we limit the hamstring the capabilities of an array > extension just to avoid internal Python changes? Because nobody wants to do the work. > Additionally, while they might be useable in scripts, they will be > inefficient for extensive interactive use. Surely you mean "too much to type" and not "inefficient to execute". Even though Python is quite usable interactively I don't think its its real strengths lie there, and I don't think that interactive use should be used as an argument in favor or against certain solutions. --Guido van Rossum URL: ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From hinsenk@ere.umontreal.ca Wed Oct 4 20:03:40 1995 From: hinsenk@ere.umontreal.ca (Hinsen Konrad) Date: Wed, 4 Oct 1995 15:03:40 -0400 Subject: [PYTHON MATRIX-SIG] Mutli-dimensional indexing and other comments In-Reply-To: <199510031645.MAA06249@python.org> (message from Chris Chase S1A on Tue, 3 Oct 1995 12:47:47 -0400) Message-ID: <199510041903.PAA18613@cyclone.ERE.UMontreal.CA> specification via an index argument. Could the rank be overriden using a keyword argument instead of an index argument, e.g. product.over(b,rank=1)? Since keywords are commonly used to override default arguments this would seem a natural possibility. I have never used keyword arguments, but that's no reason not to use them ;-) Basically the idea is not bad; however, it lacks one useful feature of my implementation: it is not possible to create function objects with modified ranks if the rank is only supplied in the function call. Whenever I need a particular function/rank combination often, I just define a new function, e.g. reshape_items = reshape[[-1,1]] reshape_items(range(5),[2,2]) gives the result 0 0 0 0 1 1 1 1 2 2 2 2 3 3 3 3 4 4 4 4 This is useful not just to save typing, but also to give meaningful names to nonobvious array functions, which helps to make the code more readable. ------------------------------------------------------------------------------- Konrad Hinsen | E-Mail: hinsenk@ere.umontreal.ca Departement de chimie | Tel.: +1-514-343-6111 ext. 3953 Universite de Montreal | Fax: +1-514-343-7586 C.P. 6128, succ. Centre-Ville | Deutsch/Esperanto/English/Nederlands/ Montreal (QC) H3C 3J7 | Francais (phase experimentale) ------------------------------------------------------------------------------- ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From dubois@kristen.llnl.gov Thu Oct 12 09:26:42 1995 From: dubois@kristen.llnl.gov (P. Dubois) Date: Thu, 12 Oct 1995 16:26:42 +0800 Subject: [PYTHON MATRIX-SIG] Hello from LLNL Message-ID: <9510122326.AA13225@kristen.llnl.gov> Greetings, I head the computer science effort at Lawrence Livermore National Laboratory in X-Division, where we do plasma physics and the design of targets for laser fusion experiments. We have many large numerical applications in Fortran and are starting newer codes in C++ and Eiffel. For the last ten years we have used a programmable applications shell for Fortran which I wrote. This is reaching the end of its useful lifetime as we move further and further away from the world for which it was designed, the monolithic (one CPU, one process, one language = Fortran) large code. I had been trying to design a replacement system for us but I found Python already is very close to what I was designing (except it is better done than I would have managed). We have decided that building on Python is the way to go. So, we'd like to know what is going on in the Matrix SIG, and other items of interest to those of us who are numerically and graphically intensive. We bring a big pile of experienced and eager manpower to the table and hope we can give a hand to this effort. I designed the EiffelMath numerical library for the Eiffel language, so I have considerable experience in both numerical mathematics and object-oriented technology. And of course having run my own extension language for 10 years I and the members of my team can do things of a compiler sort. We have some special expertise in MPI and things parallel/ vector, too. So: what do you have already? and how can we help? A bunch of us will be there in December to get trained properly... (My Fortran system is at http://www-phys.llnl.gov/X_Div/htdocs/basis.html) Paul F. Dubois, L-472 (510)-422-5426 Lawrence Livermore National Laboratory FAX (510)-423-9969 Livermore, CA 94550 dubois1@llnl.gov Consulting: PaulDubois@aol.com Editor, Scientific Programming Department Computers in Physics ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From hinsenk@ere.umontreal.ca Fri Oct 13 03:04:59 1995 From: hinsenk@ere.umontreal.ca (Hinsen Konrad) Date: Thu, 12 Oct 1995 22:04:59 -0400 Subject: [PYTHON MATRIX-SIG] J-style arrays in Python, second edition Message-ID: <199510130204.WAA14506@cyclone.ERE.UMontreal.CA> In the meantime I have debugged and extended the Python implementation of J-style arrays that I distributed earlier, so it is time for an update. The main new features: file I/O, better output formatting, more functions (cumulative reduction, comparison, and more mathematical functions). And hopefully fewer bugs; reshape() was actually quite destructive in the first version... ------------------------------------------------------------------------------- Konrad Hinsen | E-Mail: hinsenk@ere.umontreal.ca Departement de chimie | Tel.: +1-514-343-6111 ext. 3953 Universite de Montreal | Fax: +1-514-343-7586 C.P. 6128, succ. Centre-Ville | Deutsch/Esperanto/English/Nederlands/ Montreal (QC) H3C 3J7 | Francais (phase experimentale) ------------------------------------------------------------------------------- ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From hinsenk@ere.umontreal.ca Fri Oct 13 03:05:22 1995 From: hinsenk@ere.umontreal.ca (Hinsen Konrad) Date: Thu, 12 Oct 1995 22:05:22 -0400 Subject: [PYTHON MATRIX-SIG] Array.py Message-ID: <199510130205.WAA14547@cyclone.ERE.UMontreal.CA> # J-style array class # Arrays are represented by a scalar or a list, possibly containing # other lists in case of higher-rank arrays. Array rank is limited # only by the user's patience. # Send comments to Konrad Hinsen import types, math, string, regexp, copy ###################################################################### # Error type ArrayError = 'ArrayError' ###################################################################### # Various functions that do the real work. Classes follow. # Construct string representation of array def _output(data, dimension, maxlen): s = '' if dimension == 0: s = s + string.rjust(data,maxlen) elif dimension == 1: for e in data: s = s + string.rjust(e,maxlen) + ' ' else: separator = (dimension-1)*'\n' for e in data: s = s + _output(e,dimension-1,maxlen) + separator i = len(s)-1 while i > 0 and s[i] == '\n': i = i-1 return s[:i+1] # Find the shape of an array and check for consistency def _shape(data): if type(data) == types.ListType: if data and type(data[0]) == types.ListType: shapes = map(lambda x:_shape(x),data) for i in range(1,len(shapes)): if shapes[i] != shapes[0]: raise ArrayError, 'Inconsistent shapes' shape = [len(data)] shape = shape + shapes[0] return shape else: return [len(data)] else: return [] # Copy the data structure of an array def _copy(data, dimension): if (dimension == 0): return data else: c = copy.copy(data) for i in range(len(c)): c[i] = _copy(c[i], dimension-1) return c # Construct a one-dimensional list of all array elements def __ravel(data): if type(data) == types.ListType: if len(data) and type(data[0]) == types.ListType: return reduce(lambda a,b: a+b, map(lambda x: __ravel(x), data)) else: return data else: return [data] def _ravel(array): return Array(__ravel(array._data), [reduce(lambda a,b: a*b, array._shape, 1)]) # Reshape an array def _reshape(array, shape): array = _ravel(array) if len(shape._data) == 0: return take(array,0) else: array = _copy(array._data, len(array._shape)) shape = shape._data n = reduce(lambda a,b: a*b, shape) if n > len(array): nr = (n+len(array)-1)/len(array) array = (nr*array)[:n] elif n < len(array): array = array[:n] for i in range(len(shape)-1, 0, -1): d = shape[i] n = n/d for j in range(n): array[j:j+d] = [array[j:j+d]] return Array(array,shape) # Map a function on the first dimensions of an array def _extract(a, index, dimension): if len(a[1]) < dimension: return a else: return (a[0][index], a[1][1:], a[2]) def _map(function, arglist, max_frame, scalar_flag): result = [] if len(max_frame) == 0: if scalar_flag: result = apply(function,tuple(map(lambda a: a[0], arglist))) else: result = apply(function,tuple(map(lambda a: Array(a[0],a[2]), arglist)))._data else: for index in range(max_frame[0]): result.append(_map(function, map(lambda a,i=index,d=len(max_frame): _extract(a,i,d), arglist), max_frame[1:], scalar_flag)) return result # Reduce an array with a given binary function def _reduce(function, array): function = function[0] array = array[0] if len(array._shape) == 0: return array elif array._shape[0] == 0: return reshape(function._neutral, array._shape[1:]) else: result = Array(array._data[0], array._shape[1:]) for i in range(1,array._shape[0]): result = function(result, Array(array._data[i], array._shape[1:])) return result def _cumulative(function, array): function = function[0] array = array[0] if len(array._shape) == 0: return array elif array._shape[0] == 0: return array else: shape = array._shape last_result = array._data[0] result = [last_result] for i in range(1,array._shape[0]): last_result = function(last_result, Array(array._data[i], array._shape[1:]))._data result.append(last_result) return Array(result, shape) # Find the higher of two ranks def _maxrank(a,b): if a == None or b == None: return None else: return max(a,b) ###################################################################### # Array class definition class Array: def __init__(self, scalar_or_list, shape = None): self._data = scalar_or_list if shape == None: self._shape = _shape(self._data) else: self._shape = shape def __copy__(self): return Array(_copy(self._data, len(self._shape)), copy.copy(self._shape)) def __str__(self): s = tostr(self) maxstrlen = maximum.over(_strlen(_ravel(s)))._data return _output(s._data,len(s._shape),maxstrlen) __repr__ = __str__ def __len__(self): if type(self._data) == types.ListType: return len(self._data) else: return 1 def __getitem__(self, index): return take(self, index) def __getslice__(self, i, j): return take(self, range(i,j)) def __add__(self, other): return sum(self, other) __radd__ = __add__ def __sub__(self, other): return difference(self, other) def __rsub__(self, other): return difference(other, self) def __mul__(self, other): return product(self, other) __rmul__ = __mul__ def __div__(self, other): return quotient(self, other) def __rdiv__(self, other): return quotient(other, self) def __pow__(self,other): return power(self, other) def __rpow__(self,other): return power(other, self) def __neg__(self): return 0-self def writeToFile(self, filename): file = open(filename, 'w') file.write(str(self)+'\n') file.close # Check for arrayness def isArray(x): return hasattr(x,'_shape') # Read array from file _int_pattern = regexp.compile('-?[0-9]+') _float_pattern = regexp.compile('-?[0-9]*\\.[0-9]*([eE][+-]?[0-9]+)*') def _match(pattern,string): r = pattern.match(string) for i in r: if i == (0, len(string)): return 1 return 0 def _convertEntry(s): if _match(_int_pattern,s): return string.atoi(s) elif _match(_float_pattern,s): return string.atof(s) else: return s def readArray(filename): list = a = [] stack = [] blanks = 0 file = open(filename) line = file.readline() while line: if line[0] != '#': elements = map(_convertEntry, string.split(line)) if len(elements): if blanks: while blanks > len(stack): a = [a] stack.append(a) list = copy.copy([]) stack[blanks-1].append(list) for i in range(blanks-2,-1,-1): list.append(copy.copy([])) stack[i] = list list = list[0] list.append(elements) blanks = 0 else: blanks = blanks + 1 line = file.readline() file.close() while type(a) == types.ListType and len(a) == 1: a = a[0] return Array(a) ###################################################################### # Array function class class ArrayFunction: def __init__(self, function, ranks, intrinsic_ranks=None): self._function = function if isArray(ranks): self._ranks = ranks._data elif type(ranks) == types.ListType: self._ranks = ranks else: self._ranks = [ranks] if intrinsic_ranks == None: self._intrinsic_ranks = self._ranks else: self._intrinsic_ranks = intrinsic_ranks if len(self._ranks) == 1: self._ranks = len(self._intrinsic_ranks)*self._ranks def __call__(self, *args): if len(self._ranks) != len(args): raise ArrayError, 'Wrong number of arguments for an array function' arglist = [] framelist = [] shapelist = [] for i in range(len(args)): if isArray(args[i]): arglist.append(args[i]) else: arglist.append(Array(args[i])) shape = arglist[i]._shape rank = self._ranks[i] intrinsic_rank = self._intrinsic_ranks[i] if rank == None: cell = 0 elif rank < 0: cell = min(-rank,len(shape)) else: cell = max(0,len(shape)-rank) if intrinsic_rank != None: cell = max(cell,len(shape)-intrinsic_rank) framelist.append(shape[:cell]) shapelist.append(shape[cell:]) max_frame = [] for frame in framelist: if len(frame) > len(max_frame): max_frame = frame for i in range(len(framelist)): if framelist[i] != max_frame[len(max_frame)-len(framelist[i]):]: raise ArrayError, 'Incompatible arguments' scalar_function = reduce(lambda a,b:_maxrank(a,b), self._intrinsic_ranks) == 0 return Array(_map(self._function, map(lambda a,b,c: (a._data,b,c), arglist, framelist, shapelist), max_frame, scalar_function)) def __getitem__(self, ranks): return ArrayFunction(self._function,ranks,self._intrinsic_ranks) class BinaryArrayFunction(ArrayFunction): def __init__(self, function, neutral_element, ranks, intrinsic_ranks=None): ArrayFunction.__init__(self, function, ranks, intrinsic_ranks) self._neutral = neutral_element self.over = ArrayFunction(ArrayOperator(_reduce, [self]), [None]) self.cumulative = ArrayFunction(ArrayOperator(_cumulative, [self]), [None]) def __getitem__(self, ranks): return BinaryArrayFunction(self._function, self._neutral, ranks, self._intrinsic_ranks) class ArrayOperator: def __init__(self, operator, function_list): self._operator = operator self._functions = function_list def __call__(self, *args): return apply(self._operator, (self._functions, args)) ###################################################################### # Array functions # Functions for internal use _strlen = ArrayFunction(len, [0]) # Structural functions shape = ArrayFunction(lambda a: Array(a._shape,[len(a._shape)]), [None]) reshape = ArrayFunction(_reshape, [None, 1]) ravel = ArrayFunction(_ravel, [None]) take = ArrayFunction(lambda a,i: Array(a._data[i._data], a._shape[1:]), [None, 0]) # Elementwise binary functions _sum = ArrayFunction(lambda a,b: a+b, [0, 0]) _difference = ArrayFunction(lambda a,b: a-b, [0, 0]) _product = ArrayFunction(lambda a,b: a*b, [0, 0]) _quotient = ArrayFunction(lambda a,b: a/b, [0, 0]) _power = ArrayFunction(pow, [0, 0]) _max = ArrayFunction(max, [0, 0]) _min = ArrayFunction(min, [0, 0]) _smaller = ArrayFunction(lambda a,b: ab, [0, 0]) _equal = ArrayFunction(lambda a,b: a==b, [0, 0]) sum = BinaryArrayFunction(_sum, 0, [None, None]) difference = BinaryArrayFunction(_difference, 0, [None, None]) product = BinaryArrayFunction(_product, 1, [None, None]) quotient = BinaryArrayFunction(_quotient, 1, [None, None]) power = BinaryArrayFunction(_power, 1, [None, None]) maximum = BinaryArrayFunction(_max, 0, [None, None]) minimum = BinaryArrayFunction(_min, 0, [None, None]) smaller = BinaryArrayFunction(_smaller, 1, [None, None]) greater = BinaryArrayFunction(_greater, 1, [None, None]) equal = BinaryArrayFunction(_equal, 1, [None, None]) # Scalar functions of one variable tostr = ArrayFunction(str, [0]) sqrt = ArrayFunction(math.sqrt, [0]) exp = ArrayFunction(math.exp, [0]) log = ArrayFunction(math.log, [0]) log10 = ArrayFunction(math.log10, [0]) sin = ArrayFunction(math.sin, [0]) cos = ArrayFunction(math.cos, [0]) tan = ArrayFunction(math.tan, [0]) asin = ArrayFunction(math.asin, [0]) acos = ArrayFunction(math.acos, [0]) atan = ArrayFunction(math.atan, [0]) sinh = ArrayFunction(math.sinh, [0]) cosh = ArrayFunction(math.cosh, [0]) tanh = ArrayFunction(math.tanh, [0]) floor = ArrayFunction(math.floor, [0]) ceil = ArrayFunction(math.ceil, [0]) # Nasty hack to make fix max and min safe to use. # Without this, they would return an array as most users # would expect, but it would not be the correct answer. # I know I shouldn't do this, but it seems the lesser of # two evils. builtin_max = max builtin_min = min def max(*args): return apply(builtin_max,args) def min(*args): return apply(builtin_min,args) # test data x = Array(range(10)) ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From hinsenk@ere.umontreal.ca Fri Oct 13 03:05:48 1995 From: hinsenk@ere.umontreal.ca (Hinsen Konrad) Date: Thu, 12 Oct 1995 22:05:48 -0400 Subject: [PYTHON MATRIX-SIG] ArrayExamples.py Message-ID: <199510130205.WAA14571@cyclone.ERE.UMontreal.CA> # This file illustrates the use of the the Array class. # # Send comments to Konrad Hinsen # from Array import * ###################################################################### # Arrays are constructed from (nested) lists: a = Array(range(10)) b = Array([ [2,3,7], [9,8,2] ]) c = Array([ [ [4,5,6], [0,4,5] ], [ [1,6,5], [8,5,2] ] ]) # Scalars make arrays of rank 0: s = Array(42) # Array elements can be anything: text_array = Array(['Hello', 'world']) # Arrays can be printed: print 'a:\n', a print 'b:\n', b print 'c:\n', c print 's:\n', s # shape() returns an array containing the dimensions of another array: print 'shape(a):\n', shape(a) print 'shape(b):\n', shape(b) print 'shape(c):\n', shape(c) print 'shape(s):\n', shape(s) # Scalar functions act on each element of an array: print 'sqrt(b):\n',sqrt(b) # Binary operators likewise work elementwise: print 'c+c:\n',c+c # But you can also add a scalar: print 'c+s:\n',c+s # To understand the general rule for combining arrays of different # shapes in a function, we need some more technical terms: # The length of the shape vector of an array is called its rank. # The elements of an array along the first axis are called items. # The items of an array of rank N have rank N-1. More generally, # the shape vector is divided into frames and cells. Frames and # cells are not properties of an array, but describe ways of looking # at an array. For example, a rank-3 array can be regarded as # as just that - a single rank-3 array. It can also be regarded # as a rank-1 frame of rank-2 cells, or as a rank-2 frame of # rank-1 cells. Or even as a rank-3 array of rank-0 cells, i.e. # scalar cells. # # When two arrays are to be added (or multiplied, or...), their # shapes need not equal, but the lower-rank array must match # an equal-rank cell of the higher-rank array. The lower-rank # array will then be combined with each corresponding cell, and # the result will have the shape of the higher-rank array. print 'b+c:\n',b+c # The addition of a scalar is just a special case: it has rank 0 # and therefore matches the rank-0 cells (scalar elements) of any array! print 'b+s:\n',b+s # All operators are also available as normal binary function, # e.g. addition can be written as print 'sum(a,s):\n',sum(a,s) # You'll need this form to perform reductions, e.g. a sum # of all items of an array: print 'sum.over(a):\n',sum.over(a) # Let's do it with a higher-rank array: print 'product.over(b):\n',product.over(b) # But how do you get the product along the second axis # of b? Easy: print 'product.over[1](b):\n',product.over[1](b) # The [1] after the function name product.over modifies # the functions rank. Function ranks are related to array # ranks, in that a function of rank N operates on the # N-cells of its argument. If the argument has a higher # rank, the function is applied to each N-cell and the # result is combined with the frame of the argument. # In the last example, product.over will be # called for each of the 1-cells of b, returning a # rank-0 result for each, and the results will be # collected in the 1-frame of b, producing as a net # result an array of rank 1. # # All functions have ranks; if no rank is explicitly # given, the default rank will be used. The default # rank of all reductions is 'unbounded', which means # that the function will operate on the highest-level # cells possible. Many functions have unbounded rank, # for example shape(): print 'shape(c):\n',shape(c) # But of course you can modify the rank of shape(): print 'shape[1](c):\n',shape[1](c) print 'shape[2](c):\n',shape[2](c) # Functions with more than one argument can have a different # rank for each. The rank is applied to each argument, # defining its frames and cells for this purpose. The frames # must match in the same way as indicated above for # addition of two arrays. The function is then applied # to the cells, and if appropriate, the same matching # procedure is applied once again. This may seem confusing # at first, but it is really just the application of a # single principle everywhere! # # For example, let's take a (our rank-1 array) and add # b (our rank-2 array) to each of a's 0-cells: print 'sum[[0,None]](a,b):\n',sum[[0,None]](a,b) # 'None' stands for 'unbounded'. Since the rank of sum is # 0 for its first argument, a is divided into 1-frames # and 0-cells. For b the rank is unbounded, so it is # treated as a 0-frame with 2-cells. b's 0-frame matches # a's 1-frame (a 0-frame matches everything!), and # the result gets the 1-frame. The cells of the result # are sums of a rank-0 array (element of a) and a rank-2 # array (b), i.e. rank-2 arrays by the matching rules # given above. So the net total is a rank-3 array, # as you have seen. # From now on we will specify the default rank of each function. # It should be noted that specifying a higher rank than the # default rank has no effect on the function's behaviour. Only # lower ranks make a difference. # # All the scalar mathematical functions (sqrt, sin, ...) have # rank 0. The binary arithmetic functions (sum, product, ...) # have unbounded rank for both argument. For structural functions # (i.e. those that modify an array's shape rather than its # elements), the rank varies. As we have seen, shape() is # unbounded. The other structural functions are yet to be # introduced: # ravel() produces a rank-1 array containing all elements # of its argument. It has unbounded rank: print 'ravel(c):\n',ravel(c) # reshape() allows you to change the shape of an array. # It first obtains a rank-1 list of elements of its first # argument (like ravel()) and then reassembles the # elements into an array with the shape given by the # second argument: print 'reshape(a,[2,2]):\n',reshape(a,[2,2]) print 'reshape(b,[10]):\n',reshape(b,[10]) # As you have seen in the second case, reshape() reuses # the elements of its arguments all over if they get # exhausted. # You may have noticed that in some examples, a nested list # has been used instead of an array as a function argument. # In general, all array functions will convert its arguments # to arrays if necessary. # Now we need a way to select parts of an array. You can # use standard indexing and slicing to obtain items # (i.e. N-1 cells for a rank-N array): print 'c[1]:\n',c[1] print 'a[3:7]:\n',a[3:7] # You can also specify an array as the index and obtain an # array of the corresponding items: print 'a[[2,6,1]]:\n',a[[2,6,1]] # The function take() does exactly the same: print 'take(c,1):\n',take(c,1) # You will have to use take() if you want to modify its # ranks, which are [None,0] by default. There is little # point in changing the second rank (it wouldn't make # any difference), but changing the first rank lets you # select along other dimensions than the first: print 'take[1](c,0):\n',take[1](c,0) print 'take[2](c,1):\n',take[2](c,1) # Isn't there something wrong here? take() takes # two arguments and therefore needs two ranks. But for # convenience, only one rank must be given if all ranks # are to be the same. # Now the hard part is over. You are supposed to know # how data and function ranks work together. What # remains to be done is to show some more array functions. # First of all, unary functions that act on each element # of a matrix. These are: tostr, sqrt, exp, log, log10, # sin, cos, tan, asin, acos, atan, sinh, cosh, tanh, # floor, and ceil. # # Of course you expect the standard arithmetic operations, # + - * / and pow(). The first four also exist as named functions # which allow you to change rank; the names are sum, difference, # product, and quotient. They work as expected. But there are # some more binary functions that work in the same way. # maximum() and minimum select the larger/smaller one of # their arguments. With .over they find the maximum/minimum # of an arbitrary list: print 'maximum.over(a):\n',maximum.over(a) # Then there are the comparison operations: smaller, greater, # and equal. They exist only as functions of that name, not in # the form of the familiar operators. It is simply not possible # with the current Python implementation to assign such a meaning # to the comparison operators, so you'll have to live with that # for a while. # Sometimes you want not just the sum over some list # of items, but all the intermediate partial sums along # the way. There is a special function for that: print 'sum.cumulative(a):\n',sum.cumulative(a) # Finally, you might want to read arrays from disk files, # or write an array to a file. The second problem is handled # by a method, so you write something like c.writeToFile('just_a_test') # You can read this back using c_from_file = readArray('just_a_test') # and then you should check whether the two are indeed equal: print 'equal(c,c_from_file):\n',equal(c,c_from_file) # And that's the end of this introduction. Stay tuned for # updates of the array package that will provide some # important still missing in this version. In the meantime, # play round with what there is to get a feeling for # how things work. ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From dubois@kristen.llnl.gov Thu Oct 19 03:38:09 1995 From: dubois@kristen.llnl.gov (P. Dubois) Date: Thu, 19 Oct 1995 10:38:09 +0800 Subject: [PYTHON MATRIX-SIG] a few thoughts Message-ID: <9510191738.AA25244@kristen.llnl.gov> I have only begun to peruse the archive but I did see a couple of discussions worth some comments which I would like to make. For ten years I have been in charge of a Fortran extension language called Basis, which has now been used in perhaps 150-200 codes. I designed this language to look like an array Fortran. It has been extremely popular with users. So what follows is a synthesis of a whole lot of experience. Let me begin with a short synopsis of Basis' array rules: Arrays can be of any of the standard Fortran types, including complex. They consist of a single block of storage with auxillary "shape" information. They can be up to seven dimensional (the Fortran limit) but in practice five is the highest I have seen in use. Had I been implementing them in a modern language of course no limit would have been necessary. Operations are element-wise but we have a separate operator for matrix multiply and matrix divide (the latter means, a /! b is that x such that b *! x = a), where /! and *! are the matrix operators). All functions such as sin(x) operate element wise and do the right thing depending on the type of x. It is important to be able to pass the address of the storage area off to compiled code. Scalars broadcast but we have an explicit function for "dup'ing" something to make it match something higher dimensional, rather than do this automagically Now some comments: 1. Usage of the matrix operators is perhaps 1% or less of the usage of the element-wise numbers. This is because when two-dimensional matrices do arise, they usually represent the spatial or other type of discretization, far more often than they represent operators. If I had it to do over again, I would not have special operators, just a function call, since the light usage is not worth the trouble. 2. Speed is crucial. The basic operations must take place in compiled loops without function calls, with as little overhead as possible. In other words, x + y must be lead to a compiled loop doing the corresponding operation on the blocks of storage. Yes, maybe one has broadcast/type coercion/shape checks or operations first, but if both x and y are really double arrays of length n we want to be into a C loop doing xa[i] + ya[i] where xa and ya point to the storage areas. This is because when a code is written as a programmable application with an interpreter over compiled routines, the codes tend to evolve to having more and more parts of the code in the interpreter. Also, the interpreter is used to calculate information that is derivable from the compiled state variables rather than add new compiled code, but sometimes these calculations are quite intensive. 3. I think it is mistaken to try to reduce the implementor's job by doing many types in one like the "array" built-in object does. Having basic double/integer/complex stuff work fast should be the primary consideration, even if it means some tedious and not terribly elegant coding. A completely Python class like the J-array posting is beautiful and suitable for cases where uniformity of representation is of value, but it is a completely different question than having something fast that can talk to C or Fortran. I timed this class doing x+x where x was 100,000 elements long, and it was about 1000 times slower than a simple C extension to Python and even ten times slower than doing a for loop in Python (but boy, I learned a lot about Python from it!). 4. In Basis, sqrt(-1.) is an error not a complex; we actually went through a time when it was complex and found it to be a disadvantage. One must balance the need to avoid/find errors against the need for easy expression. 5. One might want to consider having a very fast, very raw vector class on which to base higher level classes that have concepts like shape, etc. 6. Banging malloc too hard can be a problem in these beasts. You probably couldn't afford to represent a big matrix with independently malloc'ed pointers for each row. Some historical notes: a. I noticed some discussion of representing things always as complex numbers. The original Matlab (when Cleve Moler wrote it to teach students linear algebra) represented everything as a complex number. There was a limit of I think 5000 numbers in a system, period, because it used a fixed array. When I ported it to a Cray I replaced that part of it with a memory manager so that you could have a lot of complex numbers. But of course having all the numbers be secretly complex is completely crazy from an efficiency point of view, not to mention storage. b. Yes, I implemented an extension language in Fortran. It was 1984 and it was the only language available on a Cray. The computer center manager said that I didn't need C, you can do everything in Fortran. c. The documentation for the Basis language is available on line at http://www-phys.llnl.gov/X_Div/htdocs/basis.html. I have decided to base my next generation system on Python. d. Reference for some philosophy: Dubois, P.F., "Making Applications Programmable", Computers in Physics Jan/Feb 1994. Paul F. Dubois, L-472 (510)-422-5426 Lawrence Livermore National Laboratory FAX (510)-423-9969 Livermore, CA 94550 dubois1@llnl.gov Consulting: PaulDubois@aol.com Editor, Scientific Programming Department, Computers in Physics ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From da@maigret.cog.brown.edu Thu Oct 19 19:43:29 1995 From: da@maigret.cog.brown.edu (David Ascher) Date: Thu, 19 Oct 1995 14:43:29 -0400 (EDT) Subject: [PYTHON MATRIX-SIG] a few thoughts In-Reply-To: <9510191738.AA25244@kristen.llnl.gov> from "P. Dubois" at Oct 19, 95 10:38:09 am Message-ID: <199510191843.OAA09461@maigret> Paul Dubois' comments struck a chord with me. While I appreciate the cleanliness of python's generic containers, I am painfully aware of the slowness of python's operations on, say, sound files with thousands of elements, to take a mid-size example. 1. How fervently are people opposed to splitting the array/tensors in terms of element type? One array for any python object, and one for numbers, all of the same type (one type per array). The two array types could have the same interface, but one could use the uniformity of its elements' types for optimization. 2. Am I right that this dichotomy would allow massive optimization? -david ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From jfulton@usgs.gov Thu Oct 19 20:23:40 1995 From: jfulton@usgs.gov (Jim Fulton, U.S. Geological Survey) Date: Thu, 19 Oct 1995 15:23:40 -0400 Subject: [PYTHON MATRIX-SIG] a few thoughts In-Reply-To: <199510191843.OAA09461@maigret> Message-ID: <199510191918.TAA00071@qvarsx.er.usgs.GOV> On Thu, 19 Oct 1995 14:43:29 -0400 (EDT) David Ascher said: > Paul Dubois' comments struck a chord with me. While I appreciate the > cleanliness of python's generic containers, I am painfully aware of the > slowness of python's operations on, say, sound files with thousands of > elements, to take a mid-size example. > > 1. How fervently are people opposed to splitting the array/tensors in > terms of element type? One array for any python object, and one for > numbers, all of the same type (one type per array). The two array > types could have the same interface, but one could use the uniformity > of its elements' types for optimization. > > 2. Am I right that this dichotomy would allow massive optimization? The current proposal is for homogenous matrices. So also supporting heterogenous matrices would not allow any additional optimizaton. Note also that the current proposal also allows homogenous matrices of characters, to aid in interfacing to Fortran routines with character arguments. Jim ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From hinsenk@ere.umontreal.ca Thu Oct 19 23:05:37 1995 From: hinsenk@ere.umontreal.ca (Hinsen Konrad) Date: Thu, 19 Oct 1995 18:05:37 -0400 Subject: [PYTHON MATRIX-SIG] a few thoughts In-Reply-To: <9510191738.AA25244@kristen.llnl.gov> (dubois@kristen.llnl.gov) Message-ID: <199510192205.SAA12015@cyclone.ERE.UMontreal.CA> 1. Usage of the matrix operators is perhaps 1% or less of the usage of the element-wise numbers. This is because when two-dimensional That raises the question of what kinds of applications were typically involved. I can think of several that would make heavy use of arrays as operators (e.g. all kinds of quantum mechanics). Maybe they were just underrepresented among your users. fast that can talk to C or Fortran. I timed this class doing x+x where x was 100,000 elements long, and it was about 1000 times slower than a simple C extension to Python and even ten times slower than doing a for loop in Python (but boy, I learned a lot about Python from it!). I agree that speed is a real problem. In fact, I openly admit that I keep a "stripped-down" version that does only one-dimensional lists for the many cases where I need long one-dimensional arrays. But this is purely an implementation problem, not a problem in principle. Much of the efficiency problems probably comes from the decision to use nested lists as an internal representation, and this decision was mostly based on laziness; it let me use simple recursive functions to handle higher-rank arrays. But of course having all the numbers be secretly complex is completely crazy from an efficiency point of view, not to mention storage. When I made this suggestion, I referred to modern implementations of APL (including J), which in fact have many internal representations for numbers, for efficiency reasons. A typical APL implementation has 1) Bits 2) small integers (i.e. bytes) 3) long integers (4 bytes) 4) real numbers 5) complex numbers But to the user all this looks like a single number type, since all conversions happen automatically. The price to pay is not in efficiency (internal APL operations tend to outperform Fortran), but in a rather complex implementation, which has to decide the optimal data type based on various criteria. For example, the high cost of unpacking bit arrays means that they will be used only for very large objects and/or when memory runs down. The advantage of this is that numbers behave like you would expect from mathematics, e.g. 1/3 equals 1./3., not 0. This prevents many errors. There are actually more user-friendly features like this in APL, e.g. non-zero comparison tolerance. Actually, my dream language would also handle symbolic operations in the style of Mathematica or Maple... b. Yes, I implemented an extension language in Fortran. It was 1984 and it was the only language available on a Cray. The computer center manager said that I didn't need C, you can do everything in Fortran. Isn't it great to have someone who always knows your real needs? ;-) ------------------------------------------------------------------------------- Konrad Hinsen | E-Mail: hinsenk@ere.umontreal.ca Departement de chimie | Tel.: +1-514-343-6111 ext. 3953 Universite de Montreal | Fax: +1-514-343-7586 C.P. 6128, succ. Centre-Ville | Deutsch/Esperanto/English/Nederlands/ Montreal (QC) H3C 3J7 | Francais (phase experimentale) ------------------------------------------------------------------------------- ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From hinsenk@ere.umontreal.ca Thu Oct 19 23:08:31 1995 From: hinsenk@ere.umontreal.ca (Hinsen Konrad) Date: Thu, 19 Oct 1995 18:08:31 -0400 Subject: [PYTHON MATRIX-SIG] a few thoughts In-Reply-To: <199510191843.OAA09461@maigret> (da@maigret.cog.brown.edu) Message-ID: <199510192208.SAA12155@cyclone.ERE.UMontreal.CA> 1. How fervently are people opposed to splitting the array/tensors in terms of element type? One array for any python object, and one for numbers, all of the same type (one type per array). The two array types could have the same interface, but one could use the uniformity of its elements' types for optimization. Isn't that what Guido already proposed? I see nothing that speaks agains that. The implementation of "general" arrays wouldn't even be much different from the others; a general array would simply be an array of object pointers. Only the application of non-structural functions and operators has be handled differently. 2. Am I right that this dichotomy would allow massive optimization? Of what? ------------------------------------------------------------------------------- Konrad Hinsen | E-Mail: hinsenk@ere.umontreal.ca Departement de chimie | Tel.: +1-514-343-6111 ext. 3953 Universite de Montreal | Fax: +1-514-343-7586 C.P. 6128, succ. Centre-Ville | Deutsch/Esperanto/English/Nederlands/ Montreal (QC) H3C 3J7 | Francais (phase experimentale) ------------------------------------------------------------------------------- ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From mclay@eeel.nist.gov Fri Oct 20 17:15:01 1995 From: mclay@eeel.nist.gov (Michael McLay) Date: Fri, 20 Oct 95 12:15:01 EDT Subject: [PYTHON MATRIX-SIG] Comments on 'a few thoughts' In-Reply-To: <9510191738.AA25244@kristen.llnl.gov> References: <9510191738.AA25244@kristen.llnl.gov> Message-ID: <9510201615.AA21381@acdc.eeel.nist.gov> P. Dubois writes: > 1. Usage of the matrix operators is perhaps 1% or less of the usage of > the element-wise numbers. This is because when two-dimensional > matrices do arise, they usually represent the spatial or other type of > discretization, far more often than they represent operators. If I > had it to do over again, I would not have special operators, just a > function call, since the light usage is not worth the trouble. Wouldn't it be appropriate to make them methods instead of functions? I agree that a good design rule would be to not implement symbolic operators for rarely used operations. Unfortunately convincing someone who is creating an operator that it is truly rare may not be easy. The goal of this design rule is to ensure code is readable. Readability is dependent on the vocabulary of the audience. If code is only to be read by a small group who have agreed upon a standardized shorthand then overloading symbols is Ok. However, to reach the broadest audience the use of functions or methods is necessary in order for the code to be unambiguous to the human parsing the source. Granted, it will make the application source code a little wordy, but that is the price to be paid for clarity. Giving programmers the unrestricted ability to overload operators can be dangerous since it tends to lead to the development of obscure dialects in notation. While I'm on the subject of design rules... Perhaps the Mathematica convention of spelling out the full name of everything instead of choosing arbitrary abbreviations should also be adopted. This would be consistent with the Python tradition of making the source code readable to others. A simple example will illustrate. How would you interpret the following expression. speed = m/hr In scanning the text can you be sure if this is miles/hour or meters/hour? I'd also suggest that only SI units be used in setting up libraries. It is simple enough to do unit conversion at the user interface. > 3. I think it is mistaken to try to reduce the implementor's job by > doing many types in one like the "array" built-in object does. > Having basic double/integer/complex stuff work fast should be the > primary consideration, even if it means some tedious and not terribly > elegant coding. > 5. One might want to consider having a very fast, very raw vector class > on which to base higher level classes that have concepts like shape, etc. This proposal suggests adding many specialized numerical types to Python. Each type would be tuned for efficient performance in solving a specific problem. From an implantation viewpoint this should not be difficult to put in place. Each new numeric type would be implemented as a dynamically linked module. This solution may be inevitable. Each application domain will by necessity build an efficient set of types required for the application domain's calculations. This approach to implementation be a pragmatic necessity since at the implementation level the computational requirements will demand that all operations on large data sets run at the speed of compiled code. Dividing the problem into discrete, importable modules compartmentalizes the work. Each numeric type module can independently be implement. All the operations needed for a numeric type would be incorporated into the module. This proposal just solves the easy part of the problem. Hinsen Konrad writes: > When I made this suggestion, I referred to modern implementations of > APL (including J), which in fact have many internal representations > for numbers, for efficiency reasons. A typical APL implementation > has > 1) Bits > 2) small integers (i.e. bytes) > 3) long integers (4 bytes) > 4) real numbers > 5) complex numbers > But to the user all this looks like a single number type, since all > conversions happen automatically. The price to pay is not in efficiency > (internal APL operations tend to outperform Fortran), but in a > rather complex implementation, which has to decide the optimal > data type based on various criteria. For example, the high cost > of unpacking bit arrays means that they will be used only for > very large objects and/or when memory runs down. > > The advantage of this is that numbers behave like you would expect > from mathematics, e.g. 1/3 equals 1./3., not 0. This prevents many > errors. There are actually more user-friendly features like this in > APL, e.g. non-zero comparison tolerance. This is the hard part of the problem to solve. Providing automatic type conversion would be a great feature and would help reduce the number of bugs and the complexity of applications. The only hitch is that it may take a significant effort to create a working implementation. Assuming that the first solution is inevitable. That is, people will write point solutions to solve their problems. Is there something that would prevent the addition of automatic type conversion from being implemented as a layer on top of the numeric type modules that will be created independently? What rules need to be established be ensure that the initially independent numeric types can be integrated into the more elegant solution? > Actually, my dream language would also handle symbolic operations > in the style of Mathematica or Maple... Yes, and one of the features from Mathematica that is missing is the ability to tag objects with symbols that represent units of measure. In Mathematica you can do the following: In[1]: 12 meters The meters symbol tags the integer 12 as its unit of measure. Michael ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From forrest@rose.rsoc.rockwell.com Fri Oct 27 15:38:07 1995 From: forrest@rose.rsoc.rockwell.com (Dave Forrest) Date: Fri, 27 Oct 1995 09:38:07 -0500 Subject: [PYTHON MATRIX-SIG] Comments on 'a few thoughts' Message-ID: <9510271438.AA17917@feynman.rsoc.rockwell.com> > From owner-matrix-sig@python.org Fri Oct 20 11:20 CDT 1995 [snip] > P. Dubois writes: > > > 1. Usage of the matrix operators is perhaps 1% or less of the usage of > > the element-wise numbers. This is because when two-dimensional Not in my office. Here, when we implemented a Matrix class in C++, we didn't even supply elementwise multiplication or _any_ division operator because _we_never_use_those_operations (well, we did implement division by a scalar - but that's it! :-). I made this plea once before but I'm going to make it again - please consider providing two interfaces into the matrix implementation - one that acts like a two-dimensional, linear-systems-type matrix and one that acts like a tensor. Even in C you can have the same code underlying it (reuse is good). I wouldn't try to deny you tensor-types the elementwise behaviour that you obviously need, but we need linear algebra - it's at the core of all our math models - please take this need seriously. > > matrices do arise, they usually represent the spatial or other type of > > discretization, far more often than they represent operators. If I > > had it to do over again, I would not have special operators, just a > > function call, since the light usage is not worth the trouble. > > Wouldn't it be appropriate to make them methods instead of functions? Absolutely - failing making them operators (as above) - methods are definitely the way to go. Actually, I think that that is what everyone meant, it's just a matter of OO semantics. ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From jjh@mama-bear.lcs.mit.edu Mon Oct 30 17:28:37 1995 From: jjh@mama-bear.lcs.mit.edu (James Hugunin) Date: Mon, 30 Oct 95 12:28:37 -0500 Subject: [PYTHON MATRIX-SIG] Alpha Matrix Object in C available - Guinea Pigs wanted Message-ID: <9510301728.AA09927@nineveh.lcs.mit.edu.LCS.MIT.EDU> I finally have an alpha version of a working, reasonably efficient matrix object implemented in C. It compiles sucessfully on a Sparc10 running SUNOS, and on a P5(and P6) running NeXTStep. I think that this object should satisfy many of the features requested by members of this group. 1) Extremely fast basic arithmetic functions To me, this is sine qua non of a useful matrix object. If I can't do arithmetic at close to the speed of hand coded C, then I have no use for the object. As a sanity check that things are reasonably efficient, I performed a very rough comparision of the basic speeds of matlab, octave, python, and C for vector arithmetic. The operation was to multiply a 10000 length vector of doubles with itself 1000 times (10 M floating point multiplies). The tests were all run on an unloaded Sun Sparc 10. tool speed(in MFlops) Array.py in python 0.002 for loop in python 0.03 octave 0.2 matlab 1.6 matrixobject in python w/-O 2.1 C w/-O 2.4 C w/-O4 2.6 So, the python object is within 10% of the speed of the hand-coded C. This is good enough for me. 2) Fancy arithmetic operations borrowing concepts from J. Every operator can be given a rank, and can be used to perform direct, outer, and inner products as well as reductions and accumulations. Much of the style of this part of the code is borrowed from Konrad's Array.py object. Thanks Konrad for the introduction to J! 3) Very general multidimensional slicing operation. This is based on Jim Fulton's proposal for multidimensional slicing, which those of you who have been following this list have seen. This is a superset of most other slicing approaches that I've seen. (Though I think that I need to go over Chris Chase's post on slices a little bit more carefully). 4) reshaping, general transposition, byteswaping, interface to/from strings, ... What's obviously missing: 1) Complex numbers don't work right yet. My basic problem is that I need a good complex number class in C to interface with, and I just haven't felt like writing this myself yet. 2) outer, inner, reduce, and accumulate style arithmetic functions could be optimized by a factor of 2-4 for simple cases. 3) inner product specfication is not completely clear to me when applied to operators with non-zero rank. 4) Documentation and comments in the code are minimal. 5) I'm sure that there must be a couple of memory leaks left hanging around. What remains to be argued about (in my opinion at least) 1) Matrix-style multiplication as an option. I really hate to leave the linear-algebra folks out in the cold. On the other hand, I don't like the idea of setting a flag to determine how the "*" operator will act on any given object. I'm open to suggestions. 2) Return by-value vs. by-reference. I finally made the decision that every matrix slicing function returns by-value, and that indexing by a single value returns by-reference. The by-value decision was based on a couple of bad experiences with the complexity of doing by-reference returns "right", and my observation that even in code that relied heavily on slicing a matrix I couldn't identify a performance difference that was ever greater than 20%. Having a single index return by-reference is necessary to allow typical python style "multi-dimensional" indexing (ie. a[0][0][0]) to remain efficient (this great hack is Jim Fulton's idea). 3) Type-coercion. At the moment, if I try to add a matrix of ints to a matrix of floats, I'll get an exception. I'm not at all sure that it wouldn't be better to just coerce the matrix of ints to floats and then do the add. 4) Default type of a matrix. At the moment, if I say "Matrix(1,2,3)", I'll get back a matrix of doubles. This should probably be setable with a matrix module function, so that if I'm working a lot with complex numbers, I can make this return a complex matrix by default instead, or whatever other type I like to work in. 5) Rank-system. I've really grown to like the J-style rank operators. One thing that they provide is that I can say Matrix(1,2,3)+Matrix([1,2,3],[11,12,13]) -> Matrix([2,4,6], [12,14,16]). There are those who might believe that it would be safer for this to return an error that the two matrices aren't the same size. 6) There are a couple of possible patches to "ceval.c" to modify python to deal a little bit better with an object that is a sequence type, but that doesn't implement the sequence copy method when multiplied by a scalar. These would general-purpose changes and should be completely compatible with all existing python code. At the moment, I've left this patch out. 7) Many other things that I'm sure I'm missing, which leads me to... I'd really like to have people play around with this object a bit and offer me as harsh feedback as you wish. This code is all based on Jim Fulton's implementation of a matrix object for using to interface to FORTRAN libraries, Unfortunately, his employer has a small restriction on the distribution of this code (until it becomes officially released). THIS SOFTWARE HAS *NOT* BEEN APPROVED FOR RELEASE TO THE PUBLIC. THIS SOFTWARE MAY ONLY BE PROVIDED TO INDIVIDUALS WHO ARE CO-DEVELOPERS, REVIEWERS, OR TESTERS OF THE SOFTWARE, OR TO PERSONNEL OF THE U.S. GEOLOGICAL SURVEY. So, anybody who wants to play around with this object, send me a message (at hugunin@mit.edu) saying that you agree to test and review it, I'll then point you to the appropriate FTP site. If I get some positive feedback from people saying that they like the overall design of the object, then I'll go in and write up documentation and comment the code better. 'til then I'm not sure that I wouldn't just be wasting my time. -Jim ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From jjh@mama-bear.lcs.mit.edu Tue Oct 31 17:53:43 1995 From: jjh@mama-bear.lcs.mit.edu (James Hugunin) Date: Tue, 31 Oct 95 12:53:43 -0500 Subject: [PYTHON MATRIX-SIG] Ranks of operators Message-ID: <9510311753.AA11479@nineveh.lcs.mit.edu.LCS.MIT.EDU> Yesterday I announced an alpha version of my C-based matrix object, and I already have gotten significant feedback on it from a number of folks on the list. One topic was brought up by Konrad Hinsen that I think merits discussion by the full list. This is the issue of ranks of functions, and generalizations of functions to outer, inner, reduce, etc. And yes, I know that we've been over this before. Here's what I'm pretty sure of: 1) We need an object that can hold a basic mathematical function in a form that it can be efficiently (nearly as fast as hand-coded C) applied to a matrix of raw basic types (ints and floats). 2) Many of these are binary functions (add, subtract, etc.). These functions should be able to be applied as outer products, reductions and accumulations also efficiently (inner products as well, but these involve two binary functions, so I'll leave them out for now). 3) The basic notion of rank in J is a sound one. A n-dimensional matrix is well viewed as a k-dimensional frame of (n-k)-dimensional cells. Thus a "matrix" can be a 2d cell, or a 1d frame of 1d cells (vector of vectors) or a 2d array of 0d numbers. My new plan is the following. I want to throw this out to the list to see if there are any significant complaints about this approach before I spend a couple of days modifying my code to match this spec. add is a python object of type ofunc. This means that the add object knows how to add matrices together efficiently. add(a,b) <--> a+b Every basic object of type ofunc has the members "reduce", "outer", "accumulate", ("inner"?). These members are themselves ofuncs, however, they don't have the members given above. [This is in contrast to these being methods of the ofunc object, my initial implementation.] ie. add.reduce is an ofunc where reduce(add, m) <-> add.reduce(m) (except that the second form is a couple orders of magnitude faster). Note: add.reduce.reduce is an error Every object of type ofunc can have its rank specified using [] notation. [This is in contrast to using a keyword specifier for rank during the call.] ie. add.reduce[1] is an ofunc with rank 1. Note: add.reduce[1][0] is an ofunc with rank 0 Negative ranks are allowed (as in J). These are similar to negative indices in slices, and specify an offset from the rank of there argument. ie. add[-1]([1,2,3],[[1,2],[4,5]]) <--> add[(0,1)]([1,2,3],[[1,2],[4,5]]) I'm open to suggestions as to how the rank of inner product should be treated. Well, that's it for now. Let me know if this makes sense, or seems like a bad idea. -Jim ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From hinsenk@ere.umontreal.ca Tue Oct 31 20:43:04 1995 From: hinsenk@ere.umontreal.ca (Hinsen Konrad) Date: Tue, 31 Oct 1995 15:43:04 -0500 Subject: [PYTHON MATRIX-SIG] Ranks of operators In-Reply-To: <9510311753.AA11479@nineveh.lcs.mit.edu.LCS.MIT.EDU> (message from James Hugunin on Tue, 31 Oct 95 12:53:43 -0500) Message-ID: <199510312043.PAA00284@cyclone.ERE.UMontreal.CA> 1) We need an object that can hold a basic mathematical function in a form that it can be efficiently (nearly as fast as hand-coded C) applied to a matrix of raw basic types (ints and floats). Also to complex numbers... Every basic object of type ofunc has the members "reduce", "outer", "accumulate", ("inner"?). These members are themselves ofuncs, however, they don't have the members given above. OK. For "inner" I'd prefer a notation of the form add.inner.multiply(a,b) with the usual possibility of specifying ranks. It seems that this requires an explicit list of all possible combinations of binary functions somewhere, but for the built-in functions that list is not too long. Another possibility would be to make the second binary function an argument, i.e. add.inner(multiply,a,b), but I'd rather not have functions as arguments. I'm open to suggestions as to how the rank of inner product should be treated. No different from other operations. The inner product imposes a restriction on its arguments in that the dimension of the rank-1 cells of the first argument must be equal to the dimension of the rank-1 frames of the second argument, but this doesn't affect the general principle of applying ranks. If the arguments do not fit, an exception should be raised. Well, that's it for now. Let me know if this makes sense, or seems like a bad idea. It seems OK to me! ------------------------------------------------------------------------------- Konrad Hinsen | E-Mail: hinsenk@ere.umontreal.ca Departement de chimie | Tel.: +1-514-343-6111 ext. 3953 Universite de Montreal | Fax: +1-514-343-7586 C.P. 6128, succ. Centre-Ville | Deutsch/Esperanto/English/Nederlands/ Montreal (QC) H3C 3J7 | Francais (phase experimentale) ------------------------------------------------------------------------------- ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org =================