From da@maigret.cog.brown.edu Fri Feb 2 00:40:45 1996 From: da@maigret.cog.brown.edu (David Ascher) Date: Thu, 1 Feb 1996 19:40:45 -0500 (EST) Subject: [PYTHON MATRIX-SIG] lists vs. tuples Message-ID: <199602020040.TAA28070@maigret> Just a thought. I believe that most existing python code, when returning collections, return tuples. The matrix code as implemented returns lists (e.g. shape). Shape strikes me as the kind of thing which might be better off being a tuple than a list. That way people doing: a = array(...) s = a.shape() s[0] = 3 would be stopped by the interpreter (I'm assuming this isn't working and isn't likely to work). Thoughts? --david ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From hinsenk@ere.umontreal.ca Fri Feb 2 01:00:29 1996 From: hinsenk@ere.umontreal.ca (Hinsen Konrad) Date: Thu, 1 Feb 1996 20:00:29 -0500 Subject: [PYTHON MATRIX-SIG] lists vs. tuples In-Reply-To: <199602020040.TAA28070@maigret> (da@maigret.cog.brown.edu) Message-ID: <199602020100.UAA08987@cyclone.ERE.UMontreal.CA> I believe that most existing python code, when returning collections, return tuples. I'd say it's mostly lists, but that's just my personal experience. The matrix code as implemented returns lists (e.g. shape). Shape Well, my installation returns tuples for shapes! Are we talking about the same thing? ------------------------------------------------------------------------------- Konrad Hinsen | E-Mail: hinsenk@ere.umontreal.ca Departement de chimie | Tel.: +1-514-343-6111 ext. 3953 Universite de Montreal | Fax: +1-514-343-7586 C.P. 6128, succ. Centre-Ville | Deutsch/Esperanto/English/Nederlands/ Montreal (QC) H3C 3J7 | Francais (phase experimentale) ------------------------------------------------------------------------------- ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From jjh@Goldilocks.LCS.MIT.EDU Fri Feb 2 15:31:26 1996 From: jjh@Goldilocks.LCS.MIT.EDU (James Hugunin) Date: Fri, 2 Feb 96 10:31:26 EST Subject: [PYTHON MATRIX-SIG] lists vs. tuples In-Reply-To: <199602020040.TAA28070@maigret> (da@maigret.cog.brown.edu) Message-ID: <9602021531.AA01424@baribal.LCS.MIT.EDU.LCS.MIT.EDU> From: da@maigret.cog.brown.edu (David Ascher) Just a thought. I believe that most existing python code, when returning collections, return tuples. The matrix code as implemented returns lists (e.g. shape). Shape strikes me as the kind of thing which might be better off being a tuple than a list. That way people doing: a = array(...) s = a.shape() s[0] = 3 would be stopped by the interpreter (I'm assuming this isn't working and isn't likely to work). Your code fragment won't work for two reasons. First, a.shape is a data member, not a function (this is because you can do a.shape = new_shape). And the data member a.shape is a tuple. You are allowed to assign to a.shape from any sequence type, but it can only be accessed as a tuple. Are you thinking of some other problem? -Jim ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From da@maigret.cog.brown.edu Fri Feb 2 15:42:07 1996 From: da@maigret.cog.brown.edu (David Ascher) Date: Fri, 2 Feb 1996 10:42:07 -0500 (EST) Subject: [PYTHON MATRIX-SIG] lists vs. tuples In-Reply-To: <9602021531.AA01424@baribal.LCS.MIT.EDU.LCS.MIT.EDU> from "James Hugunin" at Feb 2, 96 10:31:26 am Message-ID: <199602021542.KAA29265@maigret> > And the data member a.shape is a tuple. You are allowed > to assign to a.shape from any sequence type, but it can only be > accessed as a tuple. > >> The matrix code as implemented returns lists (e.g. shape). Shape > Well, my installation returns tuples for shapes! Are we talking about > the same thing? Hmm. Sorry. I was remembering Jim's description of the pseudoindices etc., where he says things like "array([3,13]) is a one dimensional array with a shape of [2]". Clearly the implemenation is doing the right thing. Sorry for decreasing the signal-to-noise ratio... --david ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From jjh@Goldilocks.LCS.MIT.EDU Fri Feb 2 18:03:31 1996 From: jjh@Goldilocks.LCS.MIT.EDU (James Hugunin) Date: Fri, 2 Feb 96 13:03:31 EST Subject: [PYTHON MATRIX-SIG] Bug-Fix Release 0.33 available Message-ID: <9602021803.AA02697@baribal.LCS.MIT.EDU.LCS.MIT.EDU> Bug-Fix Release 0.33 of the numerical extension to python is available. New in this release: Several small BUG fixes. Hopefully this release will compile "out of the box". Konrad contributed a first cut at a decent print function for arrays An expanded TUTORIAL.NumPy mrange is now arange and honors the typecode (thanks Perry) the function reshape is gone, use the method instead A new function copyAndReshape is available that does what APL folks expect from reshape (I think). A better name would be appreciated. You can try picking up ftp://sls-ftp.lcs.mit.edu/pub/jjh/NumPy.patch-0.32-0.33.gz If you copy this to the root python directory and type patch < patch-0.32-0.33 You might have managed to pick up the latest release. After remaking your system, rerun the test suite, and if the matrices are printed in a pretty fashion, you'll know that the patch worked. This is the first time that I've tried distributing a patch, so please let me know whether or not this works for you. If you're having problems, or don't trust patch, the complete tar file is also available. ftp://sls-ftp.lcs.mit.edu/pub/jjh/NumericalPython-0.33.tar.gz Enjoy - Jim ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From hinsenk@ere.umontreal.ca Fri Feb 2 19:51:08 1996 From: hinsenk@ere.umontreal.ca (Hinsen Konrad) Date: Fri, 2 Feb 1996 14:51:08 -0500 Subject: [PYTHON MATRIX-SIG] Bug-Fix Release 0.33 available In-Reply-To: <9602021803.AA02697@baribal.LCS.MIT.EDU.LCS.MIT.EDU> (jjh@Goldilocks.LCS.MIT.EDU) Message-ID: <199602021951.OAA07396@cyclone.ERE.UMontreal.CA> This is the first time that I've tried distributing a patch, so please let me know whether or not this works for you. No. Patch kept asking me which file it should patch (as if I could know that!), and I preferred downloading the complete tar file to ruining my installation. Otherwise the new version seems to work; at least the test suite runs without problems. The new function copyAndReshape doesn't work at all - it references an undefined name "ArrayType". I replaced that by type(array([0])), but then I got another error later on, telling me that the "dimensions are not legal". That of course should never happen if the function does what it is supposed to do. ------------------------------------------------------------------------------- Konrad Hinsen | E-Mail: hinsenk@ere.umontreal.ca Departement de chimie | Tel.: +1-514-343-6111 ext. 3953 Universite de Montreal | Fax: +1-514-343-7586 C.P. 6128, succ. Centre-Ville | Deutsch/Esperanto/English/Nederlands/ Montreal (QC) H3C 3J7 | Francais (phase experimentale) ------------------------------------------------------------------------------- ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From chris.chase@jhuapl.edu Fri Feb 2 10:54:20 1996 From: chris.chase@jhuapl.edu (Chris Chase S1A) Date: Fri, 2 Feb 1996 15:54:20 +0500 Subject: [PYTHON MATRIX-SIG] Bug-Fix Release 0.33 available In-Reply-To: <199602021951.OAA07396@cyclone.ERE.UMontreal.CA> References: <9602021803.AA02697@baribal.LCS.MIT.EDU.LCS.MIT.EDU> <199602021951.OAA07396@cyclone.ERE.UMontreal.CA> Message-ID: <9602022054.AA04256@hurlbut.> Hinsen> No. Patch kept asking me which file it should patch (as if I could Hinsen> know that!), and I preferred downloading the complete tar file Hinsen> to ruining my installation. I got the patch to work. After patching the first few files, my patch program also started asking what file to patch. I just copied and pasted the file name shown from the patch header, i.e., when patch showed: diff -c -r OldVersion/Modules/ofuncobject.c ./Modules/ofuncobject.c *** OldVersion/Modules/ofuncobject.c Mon Jan 29 10:29:30 1996 --- ./Modules/ofuncobject.c Thu Feb 1 16:57:23 1996 *************** And asked what file to patch I copied and pasted to the prompt ./Modules/ofuncobject.c I don't know why patch needed the file names after being succesful the first few patches. Chris ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From hinsenk@ere.umontreal.ca Fri Feb 2 23:11:57 1996 From: hinsenk@ere.umontreal.ca (Konrad HINSEN) Date: Fri, 2 Feb 1996 18:11:57 -0500 Subject: [PYTHON MATRIX-SIG] New pretty printer Message-ID: <199602022311.SAA23116@cyclone.ERE.UMontreal.CA> My pretty printer has made some progress, and I'll include the latest version below. Just put it into your copy of Numeric.py. Line wrapping is still missing. (BTW, what would be a good default line width? In the VT100 days that would have been 80 characters, but does this still make sense?) I can't say that I am satisfied with float formatting, but anything better than now will be difficult to do and, most of all, slow. The two problems are 1) Trailing zeros in non-exponential format. Right now there are always eight digits after the decimal point. It would be better to replace trailing zeros by spaces (easy) and reduce the width of the output fields to match the longest remaining number. But that requires a scan of the string representations of all the elements in an array, which is an expensive operation. 2) In exponential notation, numbers with three-digit exponents will not line up correctly with others. It would be necessary to have three-digit exponents for all numbers when at least one element requires it, but there is no formatting option to specify the number of exponent digits. The only solution would be cosmetic surgery on the final string, but that again is slow. There are currently some problems with complex number output due to a remaining bug in the array module. Normally it will only lead to a wrong format selection, so it's not fatal. And now the code: def arrayToString(a, max_line_length = 80): if a.contiguous(): data = a.reshape(None) else: data = a.copy().reshape(None) type = a.typecode() items_per_line = a.shape[-1] if type == 'b' or type == '1' or type == 's' or type == 'i' \ or type == 'l': max_str_len = max(len(str(maximum.reduce(data))), len(str(minimum.reduce(data)))) format = '%' + str(max_str_len) + 'd' type = 'l' item_length = max_str_len+1 elif type == 'f' or type == 'd': format, item_length = _floatFormat(data) type = 'd' elif type == 'F' or type == 'D': real_format, real_item_length = _floatFormat(data.real, sign=0) imag_format, imag_item_length = _floatFormat(data.imaginary, sign=1) format = '(' + real_format + imag_format + 'j)' item_length = real_item_length + imag_item_length + 2 type = 'D' elif type == 'c': format = '%s' item_length = 1 else: # nothing for 'O' return str(a) line_length = item_length*items_per_line - (type != 'c') if line_length > max_line_length: pass return _arrayToString(a, type, format, len(a.shape), line_length)[:-1] def _floatFormat(data, sign = 0): exp_format = 0 non_zero = abs(compress(data.notEqual(0), data)) print str(data), str(data.notEqual(0)), str(compress(data.notEqual(0), data)) if len(non_zero) == 0: max_val = 0. min_val = 0. else: max_val = maximum.reduce(non_zero) min_val = minimum.reduce(non_zero) if max_val >= 1.e12 or min_val < 0.0001 or max_val/min_val > 1000.: exp_format = 1 if exp_format: max_str_len = 16 + (0 < min_val < 1e-99 or max_val >= 1e100) if sign: format = '%+' else: format = '%' format = format + str(max_str_len) + '.8e' item_length = max_str_len + 1 else: max_str_len = len(str(int(max_val))) + 10 if sign: format = '%+' else: format = '%' format = format + str(max_str_len) + '.8f' item_length = max_str_len + 1 return (format, item_length) def _arrayToString(a, type, format, rank, line_length): if rank == 0: return str(a[0]) elif rank == 1: s = '' if type == 'c': for i in range(a.shape[0]): s = s + (format % a[i]) s = s + '\n' elif type == 'D': for i in range(a.shape[0]): s = s + (format % (a[i].real, a[i].imag)) + ' ' s = s[:-1] + '\n' else: for i in range(a.shape[0]): s = s + (format % a[i]) + ' ' s = s[:-1] + '\n' else: s = '' for i in range(a.shape[0]-1): s = s + _arrayToString(a[i], type, format, rank-1, line_length) if rank == 3: s = s + '\n' elif rank > 3: s = s + (rank-3)*(line_length*'-'+'\n') s = s + _arrayToString(a[a.shape[0]-1], type, format, rank-1, line_length) return s ------------------------------------------------------------------------------- Konrad Hinsen | E-Mail: hinsenk@ere.umontreal.ca Departement de chimie | Tel.: +1-514-343-6111 ext. 3953 Universite de Montreal | Fax: +1-514-343-7586 C.P. 6128, succ. Centre-Ville | Deutsch/Esperanto/English/Nederlands/ Montreal (QC) H3C 3J7 | Francais (phase experimentale) ------------------------------------------------------------------------------- ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From guido@CNRI.Reston.VA.US Sat Feb 3 00:06:42 1996 From: guido@CNRI.Reston.VA.US (Guido van Rossum) Date: Fri, 02 Feb 1996 19:06:42 -0500 Subject: [PYTHON MATRIX-SIG] New pretty printer In-Reply-To: Your message of "Fri, 02 Feb 1996 18:11:57 EST." <199602022311.SAA23116@cyclone.ERE.UMontreal.CA> References: <199602022311.SAA23116@cyclone.ERE.UMontreal.CA> Message-ID: <199602030006.TAA10516@monty> > My pretty printer has made some progress, and I'll include the latest > version below. Just put it into your copy of Numeric.py. Line wrapping > is still missing. (BTW, what would be a good default line width? In the > VT100 days that would have been 80 characters, but does this still > make sense?) Yes! I'd even say 77, so at least one level of email quoting won't cause it to wrap. I keep my Emacs religiously at 80 columns wide when programming -- this way I can fit two columns of windows plus some icons on my screen. Also, the default wrap columns for printing text seems to be 80 still. > I can't say that I am satisfied with float formatting, > but anything better than now will be difficult to do and, most of > all, slow. The two problems are > > 1) Trailing zeros in non-exponential format. Right now there are > always eight digits after the decimal point. It would be better > to replace trailing zeros by spaces (easy) and reduce the width > of the output fields to match the longest remaining number. > But that requires a scan of the string representations of all > the elements in an array, which is an expensive operation. > > 2) In exponential notation, numbers with three-digit exponents will > not line up correctly with others. It would be necessary to > have three-digit exponents for all numbers when at least > one element requires it, but there is no formatting option > to specify the number of exponent digits. The only solution > would be cosmetic surgery on the final string, but that again > is slow. (For both case:) Surely, if you're printing a reasonably sized array, it will be fast enough, and if it is truly huge, nobody will really look at the numbers... I'd say make it look as good as you can. Also, perhaps a regular expression can help you do the cosmetic surgery efficiently. --Guido van Rossum URL: ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From guido@CNRI.Reston.VA.US Sat Feb 3 00:06:42 1996 From: guido@CNRI.Reston.VA.US (Guido van Rossum) Date: Fri, 02 Feb 1996 19:06:42 -0500 Subject: [PYTHON MATRIX-SIG] New pretty printer In-Reply-To: Your message of "Fri, 02 Feb 1996 18:11:57 EST." <199602022311.SAA23116@cyclone.ERE.UMontreal.CA> References: <199602022311.SAA23116@cyclone.ERE.UMontreal.CA> Message-ID: <199602030006.TAA10516@monty> > My pretty printer has made some progress, and I'll include the latest > version below. Just put it into your copy of Numeric.py. Line wrapping > is still missing. (BTW, what would be a good default line width? In the > VT100 days that would have been 80 characters, but does this still > make sense?) Yes! I'd even say 77, so at least one level of email quoting won't cause it to wrap. I keep my Emacs religiously at 80 columns wide when programming -- this way I can fit two columns of windows plus some icons on my screen. Also, the default wrap columns for printing text seems to be 80 still. > I can't say that I am satisfied with float formatting, > but anything better than now will be difficult to do and, most of > all, slow. The two problems are > > 1) Trailing zeros in non-exponential format. Right now there are > always eight digits after the decimal point. It would be better > to replace trailing zeros by spaces (easy) and reduce the width > of the output fields to match the longest remaining number. > But that requires a scan of the string representations of all > the elements in an array, which is an expensive operation. > > 2) In exponential notation, numbers with three-digit exponents will > not line up correctly with others. It would be necessary to > have three-digit exponents for all numbers when at least > one element requires it, but there is no formatting option > to specify the number of exponent digits. The only solution > would be cosmetic surgery on the final string, but that again > is slow. (For both case:) Surely, if you're printing a reasonably sized array, it will be fast enough, and if it is truly huge, nobody will really look at the numbers... I'd say make it look as good as you can. Also, perhaps a regular expression can help you do the cosmetic surgery efficiently. --Guido van Rossum URL: ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From da@maigret.cog.brown.edu Sun Feb 4 22:54:46 1996 From: da@maigret.cog.brown.edu (David Ascher) Date: Sun, 4 Feb 1996 17:54:46 -0500 (EST) Subject: [PYTHON MATRIX-SIG] slicing rank-1 arrays Message-ID: <199602042254.RAA08528@maigret> Given the following, which seems fine: >>> a 0 1 2 3 4 0 1 2 3 4 0 1 2 3 4 0 1 2 3 4 >>> a[All,All] 0 1 2 3 4 0 1 2 3 4 0 1 2 3 4 0 1 2 3 4 why doesn't the following work? >>> b 0 1 2 3 4 >>> b[All] Traceback (innermost last): File "", line 1, in ? AttributeError: __len__ >>> b[Reverse] Traceback (innermost last): File "", line 1, in ? AttributeError: __len__ [Using 0.33 on an SGI] --david ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From da@maigret.cog.brown.edu Sun Feb 4 23:00:26 1996 From: da@maigret.cog.brown.edu (David Ascher) Date: Sun, 4 Feb 1996 18:00:26 -0500 (EST) Subject: [PYTHON MATRIX-SIG] New Tutorial / Some Questions Message-ID: <199602042300.SAA08594@maigret> Given that I'm starting a course on Python on Tuesday for a bunch of neural net folks, I had to take a good hard look at the Numeric package. Jim H. has gotten more mail from me than he probably wanted. The good thing is that I didn't see myself teaching them about a feature that had no documentation. So, I started a more detailed tutorial than that Jim shipped in 0.33. I aimed it at people who have very little python experience, and no prior experience w/ matlab, basis, yorick or anything like it (i.e. undergraduates =). It's obviously too basic for the folks in this SIG, but I'd appreciate it if you could give it a once-over and make sure I'm not lying outright. It's available at http://maigret.cog.brown.edu/python/arraytut.html for now. It is very much under construction (I haven't explained rubber indices and pseudo indices, for example, mostly because I'm not sure I understand them yet). I came up with a few questions in the process. Some have gone to jim h. and hinsen, but I thought I'd spread the load: 1. I can't seem to find the counterpart to the toFile() method. I've tried just the array constructor w/ a file object, which I thought might work, but I get: >>> a 0 0 1 0 0 0 1 2 3 >>> f = open('myarray', 'w') >>> a.toFile(f) >>> f.close() >>> f = open('myarray', 'r') >>> b = array(f) >>> b Traceback (innermost last): File "", line 1, in ? File "/usr/local/lib/python/numeric/Numeric.py", line 37, in arrayToString items_per_line = a.shape[-1] IndexError: tuple index out of range 2. The choose() and take() methods are a little fuzzy for me. Can someone give me a description of what they do, and more useful even examples of why they're useful? 3. Shouldn't typecodes look more like real types than strings, sort of like exceptions? >>> b.typecode() 'l' vs. >>> b.typecode() LongInteger or whatever. That's all for now. Let me know what you think of the tutorial. --david ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From jjh@Goldilocks.LCS.MIT.EDU Mon Feb 5 16:13:43 1996 From: jjh@Goldilocks.LCS.MIT.EDU (James Hugunin) Date: Mon, 5 Feb 96 11:13:43 EST Subject: [PYTHON MATRIX-SIG] slicing rank-1 arrays In-Reply-To: <199602042254.RAA08528@maigret> (da@maigret.cog.brown.edu) Message-ID: <9602051613.AA26208@baribal.LCS.MIT.EDU.LCS.MIT.EDU> From: da@maigret.cog.brown.edu (David Ascher) Given the following, which seems fine: >>> a 0 1 2 3 4 0 1 2 3 4 0 1 2 3 4 0 1 2 3 4 >>> a[All,All] 0 1 2 3 4 0 1 2 3 4 0 1 2 3 4 0 1 2 3 4 why doesn't the following work? >>> b 0 1 2 3 4 >>> b[All] Traceback (innermost last): File "", line 1, in ? AttributeError: __len__ >>> b[Reverse] Traceback (innermost last): File "", line 1, in ? AttributeError: __len__ [Using 0.33 on an SGI] This is a bug. I'll fix it. -Jim ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From jjh@Goldilocks.LCS.MIT.EDU Mon Feb 5 15:56:03 1996 From: jjh@Goldilocks.LCS.MIT.EDU (James Hugunin) Date: Mon, 5 Feb 96 10:56:03 EST Subject: [PYTHON MATRIX-SIG] New Tutorial / Some Questions In-Reply-To: <199602042300.SAA08594@maigret> (da@maigret.cog.brown.edu) Message-ID: <9602051556.AA26139@baribal.LCS.MIT.EDU.LCS.MIT.EDU> From: da@maigret.cog.brown.edu (David Ascher) Given that I'm starting a course on Python on Tuesday for a bunch of neural net folks, I had to take a good hard look at the Numeric package. Jim H. has gotten more mail from me than he probably wanted. The good Actually, I love the mail. This is the best way to really shake down the package. I came up with a few questions in the process. Some have gone to jim h. and hinsen, but I thought I'd spread the load: Well, since nobody else has answered yet (and since I'd LOVE to see your course succeed). 1. I can't seem to find the counterpart to the toFile() method. There used to be such a function in Numeric.py. The reason it's not there any more is to strongly encourage people to use pickling of matrices and better yet of objects containing matrices instead of the raw toFile and fromFile methods. This is the method I use to store the fairly complicated GaussianClassifier objects that I use for speech recognition. This approach will gain you a certain degree of architecture independence (though 64-bit longs on alpha's give it trouble) and will cost almost nothing in performance terms. Note, you can still read in an array from a file if you need to for compatibility reasons as follows: a = fromString(fp.read()) 2. The choose() and take() methods are a little fuzzy for me. Can someone give me a description of what they do, and more useful even examples of why they're useful? Actually, Konrad or Paul could probably do a better job of this than me, but here are my two favorite examples. static char doc_choose[] = "m.choose(m_0, ..., m_(n-1)). m is a array of integers from 0 to n-1. Each entry in m will be taken from the corresponding entry in m_i, where i is the value of m. Frequently used with exactly two arguments for a boolean valued m, ie. m.choose(m_false, m_true)."; Assume a is an array that you want to "clip" so that no values are greater than 100.0. (a.greater(100.0)).choose(a, 100.0) Everywhere that a.greater(100.0) is false (ie. 0) this will "choose" the corresponding value in a. Everywhere else it will "choose" 100.0. static char doc_take[] = "m.take([i_0, ..., i_n], dimension=0). Selects the items from m along dimension"; Take is the equivalent of "full product form indexing" that for a number of reasons is not available in normal indexing. The most common use I have for take is to implement decoding from mu-law to 16-bit linear sound files. mu-law files map the bytes from 0-255 to a given 16-bit number using a non-linear scale as a form of compression. Assuming you have an array mu_law_array (of length 256) where mu_law_array[value_mu_law] = value_linear linear_encoded_array = mu_law_array.take(mu_law_encoded_array) 3. Shouldn't typecodes look more like real types than strings, sort of like exceptions? >>> b.typecode() 'l' vs. >>> b.typecode() LongInteger or whatever. The only hassle is that doing this would lead to yet another new object type added to python (after already adding array's, ufunc's, and slices (hopefully soon). I'm willing to be convinced that this is the right choice, but for now I opted for the minimal approach. That's all for now. Let me know what you think of the tutorial. Will look at it ASAP. -Jim ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From nick@osg.saic.com Mon Feb 5 14:34:21 1996 From: nick@osg.saic.com (Nick Seidenman) Date: Mon, 5 Feb 1996 09:34:21 -0500 (EST) Subject: [PYTHON MATRIX-SIG] NumPython Message-ID: <9602051434.AA02953@typhoon.osg.saic.com> Just downloaded and built 0.33. No problems so far! Compiled right out of the box on a linux 1.2.13 system rnning on my handy-dandy pentium laptop! Thanks, Jim! nick --------------------------- Nick Seidenman Science Applications International Corporation 1710 Goodridge Drive McLean, VA 22102 Phone: (703) 448-6497 (fax): (703) 821-3576 email: nick@osg.saic.com http://www.osg.saic.com/~nick/ ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From hinsenk@ere.umontreal.ca Mon Feb 5 16:40:38 1996 From: hinsenk@ere.umontreal.ca (Konrad HINSEN) Date: Mon, 5 Feb 1996 11:40:38 -0500 Subject: [PYTHON MATRIX-SIG] New Tutorial / Some Questions In-Reply-To: <199602042300.SAA08594@maigret> (da@maigret.cog.brown.edu) Message-ID: <199602051640.LAA05335@cyclone.ERE.UMontreal.CA> 3. Shouldn't typecodes look more like real types than strings, sort of like exceptions? >>> b.typecode() 'l' vs. >>> b.typecode() LongInteger or whatever. Definitely. Especially now that we have the stuff in Precision.py for specifying types, it should be used consistently wherever possible. ------------------------------------------------------------------------------- Konrad Hinsen | E-Mail: hinsenk@ere.umontreal.ca Departement de chimie | Tel.: +1-514-343-6111 ext. 3953 Universite de Montreal | Fax: +1-514-343-7586 C.P. 6128, succ. Centre-Ville | Deutsch/Esperanto/English/Nederlands/ Montreal (QC) H3C 3J7 | Francais (phase experimentale) ------------------------------------------------------------------------------- ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From hinsenk@ere.umontreal.ca Mon Feb 5 21:23:57 1996 From: hinsenk@ere.umontreal.ca (Konrad HINSEN) Date: Mon, 5 Feb 1996 16:23:57 -0500 Subject: [PYTHON MATRIX-SIG] New Tutorial / Some Questions In-Reply-To: <9602051556.AA26139@baribal.LCS.MIT.EDU.LCS.MIT.EDU> (jjh@Goldilocks.LCS.MIT.EDU) Message-ID: <199602052123.QAA24690@cyclone.ERE.UMontreal.CA> 1. I can't seem to find the counterpart to the toFile() method. There used to be such a function in Numeric.py. The reason it's not there any more is to strongly encourage people to use pickling of matrices and better yet of objects containing matrices instead of the raw toFile and fromFile methods. This is the method I use to store That makes sense, but then toFile() should also disappear. Take is the equivalent of "full product form indexing" that for a number of reasons is not available in normal indexing. The most common use I have for take is to implement decoding from mu-law to 16-bit linear sound files. mu-law files map the bytes from 0-255 to a given 16-bit number using a non-linear scale as a form of compression. Assuming you have an array mu_law_array (of length 256) where mu_law_array[value_mu_law] = value_linear linear_encoded_array = mu_law_array.take(mu_law_encoded_array) Translation tables are one of the important applications of this operation. Another is sorting, ...ehhh... *would* be sorting if it were implemented in a more flexible way. Right now we have the built-in function sort() which works also on arrays. What is missing (and should be added) is a function that returns the indices of the elements in sorted order. As an example, suppose you have an array of measurements of some function f(x) as a rank-2 array a; a[0] being the evaluation points and a[1] the values. You want to sort the array by evaluation points. The answer is a.take(a[0].sortIndex(), 1) BTW, it would be extremely helpful to have a list of all existing functions/methods with a short definition. It is difficult to comment on whether the current set is sufficient and/or appropriately defined and names without knowing what is out there! Do you happen to have something suitable? 3. Shouldn't typecodes look more like real types than strings, sort of like exceptions? >>> b.typecode() 'l' vs. >>> b.typecode() LongInteger or whatever. The only hassle is that doing this would lead to yet another new object type added to python (after already adding array's, ufunc's, and slices (hopefully soon). I'm willing to be convinced that this is the right choice, but for now I opted for the minimal approach. There is no need for a new object type, since the typecode could be of type "type". All that has to be added is type objects for the new types. ------------------------------------------------------------------------------- Konrad Hinsen | E-Mail: hinsenk@ere.umontreal.ca Departement de chimie | Tel.: +1-514-343-6111 ext. 3953 Universite de Montreal | Fax: +1-514-343-7586 C.P. 6128, succ. Centre-Ville | Deutsch/Esperanto/English/Nederlands/ Montreal (QC) H3C 3J7 | Francais (phase experimentale) ------------------------------------------------------------------------------- ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From da@maigret.cog.brown.edu Mon Feb 5 21:49:33 1996 From: da@maigret.cog.brown.edu (David Ascher) Date: Mon, 5 Feb 1996 16:49:33 -0500 (EST) Subject: [PYTHON MATRIX-SIG] New Tutorial / Some Questions In-Reply-To: <199602052123.QAA24690@cyclone.ERE.UMontreal.CA> from "Konrad HINSEN" at Feb 5, 96 04:23:57 pm Message-ID: <199602052149.QAA11829@maigret> > BTW, it would be extremely helpful to have a list of all existing > functions/methods with a short definition. It is difficult to comment > on whether the current set is sufficient and/or appropriately defined > and names without knowing what is out there! Do you happen to have > something suitable? I've tried to list them all in the tutorial. Well, at least they should all be mentioned in the tutorial. A whole bunch of them have no definitions yet, but that will get fixed. I just got them from browsing the source, so I may very well have missed some. --david ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From dubois1@llnl.gov Mon Feb 5 22:16:04 1996 From: dubois1@llnl.gov (Paul. Dubois) Date: Mon, 05 Feb 1996 14:16:04 -0800 Subject: [PYTHON MATRIX-SIG] New pretty printer References: <199602022311.SAA23116@cyclone.ERE.UMontreal.CA> <199602030006.TAA10516@monty> Message-ID: <311681A4.1E79@llnl.gov> I suggest a default size of 79 (or 77 as suggested by Guido). Exactly 80 causes wrap in some display vehicles. I didn't look yet at how you did it but perhaps a user-settable precision could be managed. I have done that in both Basis and EiffelMath. Guido is right about the timing too; if there are enough components that you have to worry about the timing to print it, nobody is going to print it anyway or look at it if they do. One thing I did in Basis was an optional "compression" that compressed identical rows (or strings of identical elements in 1-D). Basis> ones(10000) ones(10000) shape: (10000) 1: 9976 - 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 9977: - 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 Basis> shape([1,2,3]//ones(10000)//[5,6],2001,5) shape([1,2,3]//ones(10000)//[5,6],2001,5) shape: (2001,5) row col = 1 2 3 4 5 1: - 1 1 1 1 1 2: - 2 1 1 1 1 3: - 3 1 1 1 1 4:1999 - 1 1 1 1 1 2000: - 1 1 1 1 5 2001: - 1 1 1 1 6 Basis> This has proven useful. -- Paul F. Dubois, L-472 (510)-422-5426 Lawrence Livermore National Laboratory FAX (510)-423-9969 Livermore, CA 94550 dubois1@llnl.gov Consulting: PaulDubois@aol.com ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From hinsenk@ere.umontreal.ca Tue Feb 6 00:11:50 1996 From: hinsenk@ere.umontreal.ca (Konrad HINSEN) Date: Mon, 5 Feb 1996 19:11:50 -0500 Subject: [PYTHON MATRIX-SIG] New pretty printer In-Reply-To: <311681A4.1E79@llnl.gov> (dubois1@llnl.gov) Message-ID: <199602060011.TAA05144@cyclone.ERE.UMontreal.CA> I didn't look yet at how you did it but perhaps a user-settable precision could be managed. I have done that in both Basis and Just implemented :-) The problem with both this and the line width is that there is no convenient way to set them when simply printing arrays. Right now, you have to call the function arrayToString() explicitly with additional parameters. I could of course define global variables in module Numeric to handle this, but I am not sure whether that is a good idea. Maybe a good compromise is optional parameters that are set from global variables if undefined. Comments? EiffelMath. Guido is right about the timing too; if there are enough components that you have to worry about the timing to print it, nobody is going to print it anyway or look at it if they do. I tend to agree, especially since you can always interrupt the output, as long as the function is written in Python. One thing I did in Basis was an optional "compression" that compressed identical rows (or strings of identical elements in 1-D). That could indeed be useful, and not too complicated to implement. Another option... ------------------------------------------------------------------------------- Konrad Hinsen | E-Mail: hinsenk@ere.umontreal.ca Departement de chimie | Tel.: +1-514-343-6111 ext. 3953 Universite de Montreal | Fax: +1-514-343-7586 C.P. 6128, succ. Centre-Ville | Deutsch/Esperanto/English/Nederlands/ Montreal (QC) H3C 3J7 | Francais (phase experimentale) ------------------------------------------------------------------------------- ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From jjh@Goldilocks.LCS.MIT.EDU Tue Feb 6 14:21:02 1996 From: jjh@Goldilocks.LCS.MIT.EDU (James Hugunin) Date: Tue, 6 Feb 96 09:21:02 EST Subject: [PYTHON MATRIX-SIG] New pretty printer In-Reply-To: <199602060011.TAA05144@cyclone.ERE.UMontreal.CA> (hinsenk@ere.umontreal.ca) Message-ID: <9602061421.AA20009@baribal.LCS.MIT.EDU.LCS.MIT.EDU> From: hinsenk@ere.umontreal.ca (Konrad HINSEN) I didn't look yet at how you did it but perhaps a user-settable precision could be managed. I have done that in both Basis and Just implemented :-) The problem with both this and the line width is that there is no convenient way to set them when simply printing arrays. Right now, you have to call the function arrayToString() explicitly with additional parameters. I could of course define global variables in module Numeric to handle this, but I am not sure whether that is a good idea. Maybe a good compromise is optional parameters that are set from global variables if undefined. Comments? The compromise approach sounds perfect. I think that this is one of those rare cases where there is no potential for problems to be caused by global variables because you are not going to be altering the results of any computation, only the way those results are displayed. Without global variables there is just no reasonable way to alter the default behavior for "print a". -Jim ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From jfulton@usgs.gov Tue Feb 6 14:32:18 1996 From: jfulton@usgs.gov (Jim Fulton, U.S. Geological Survey) Date: Tue, 6 Feb 1996 09:32:18 -0500 Subject: [PYTHON MATRIX-SIG] New pretty printer In-Reply-To: jjh@Goldilocks.LCS.MIT.EDU (James Hugunin) "Re: [PYTHON MATRIX-SIG] New pretty printer" (Feb 6, 9:21am) References: <9602061421.AA20009@baribal.LCS.MIT.EDU.LCS.MIT.EDU> Message-ID: <9602060932.ZM10571@dsdbqvarsa.er.usgs.gov> On Feb 6, 9:21am, James Hugunin wrote: > Subject: Re: [PYTHON MATRIX-SIG] New pretty printer > > From: hinsenk@ere.umontreal.ca (Konrad HINSEN) > > I didn't look yet at how you did it but perhaps a user-settable > precision could be managed. I have done that in both Basis and > > Just implemented :-) The problem with both this and the line width is > that there is no convenient way to set them when simply printing > arrays. Right now, you have to call the function arrayToString() > explicitly with additional parameters. I could of course define global > variables in module Numeric to handle this, but I am not sure whether > that is a good idea. Maybe a good compromise is optional parameters > that are set from global variables if undefined. Comments? > > The compromise approach sounds perfect. I think that this is one of > those rare cases where there is no potential for problems to be caused > by global variables because you are not going to be altering the > results of any computation, only the way those results are displayed. But other modules may have different display needs, so you can still effect a different piece of code. > Without global variables there is just no reasonable way to alter the > default behavior for "print a". You could make this parameter a part of the state of an array. Jim ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From hinsenk@CHIMCN.UMontreal.CA Wed Feb 7 15:11:56 1996 From: hinsenk@CHIMCN.UMontreal.CA (Hinsen Konrad) Date: Wed, 7 Feb 1996 10:11:56 -0500 Subject: [PYTHON MATRIX-SIG] FTP address Message-ID: <199602071511.KAA03680@chims1.CHIMCN.UMontreal.CA> Due to a major server breakdown I am cut off from all my files and my normal e-mail address. Nevertheless I'd like to go on playing with the array package. I have already reinstalled Python, but I have to get the numerics extension again. Could anyone please send me the address of Jim's FTP server? ------------------------------------------------------------------------------- Konrad Hinsen | E-Mail: hinsenk@chims1.chimcn.umontreal.ca Departement de Chimie | Tel.: +1-514-343-6111 ext. 3953 Universite de Montreal | Fax: +1-514-343-7586 C.P. 6128, succ. A | Deutsch/Esperanto/English/Nederlands/ Montreal (QC) H3C 3J7 | Francais (phase experimentale) ------------------------------------------------------------------------------- ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From hinsenk@ere.umontreal.ca Wed Feb 7 16:52:33 1996 From: hinsenk@ere.umontreal.ca (Konrad HINSEN) Date: Wed, 7 Feb 1996 11:52:33 -0500 Subject: [PYTHON MATRIX-SIG] New pretty printer In-Reply-To: <9602060932.ZM10571@dsdbqvarsa.er.usgs.gov> (jfulton@usgs.gov) Message-ID: <199602071652.LAA04701@cyclone.ERE.UMontreal.CA> But other modules may have different display needs, so you can still effect a different piece of code. > Without global variables there is just no reasonable way to alter the > default behavior for "print a". You could make this parameter a part of the state of an array. That implies that precision and line width are properties of an array. For precision this may be justified, but certainly not for line width! Such things really belong into a "system parameters" collection. The closest Python equivalent would be the module "sys". But that doesn't solve the problem of who is allowed to modify them how and when. ------------------------------------------------------------------------------- Konrad Hinsen | E-Mail: hinsenk@ere.umontreal.ca Departement de chimie | Tel.: +1-514-343-6111 ext. 3953 Universite de Montreal | Fax: +1-514-343-7586 C.P. 6128, succ. Centre-Ville | Deutsch/Esperanto/English/Nederlands/ Montreal (QC) H3C 3J7 | Francais (phase experimentale) ------------------------------------------------------------------------------- ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From guido@CNRI.Reston.VA.US Wed Feb 7 17:01:39 1996 From: guido@CNRI.Reston.VA.US (Guido van Rossum) Date: Wed, 07 Feb 1996 12:01:39 -0500 Subject: [PYTHON MATRIX-SIG] New pretty printer In-Reply-To: Your message of "Wed, 07 Feb 1996 11:52:33 EST." <199602071652.LAA04701@cyclone.ERE.UMontreal.CA> References: <199602071652.LAA04701@cyclone.ERE.UMontreal.CA> Message-ID: <199602071701.MAA20712@monty.cnri.reston.va.us> > Such things really belong into a "system parameters" collection. > The closest Python equivalent would be the module "sys". But > that doesn't solve the problem of who is allowed to modify them > how and when. It makes sense for it to be a property of the file object. But since adding properties to those is no fun, I'd suggest to go for a parameter is sys. Whoever feels like they can change sys.stdout can also change the line width if they feel like it. --Guido van Rossum URL: ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From guido@CNRI.Reston.VA.US Wed Feb 7 17:01:39 1996 From: guido@CNRI.Reston.VA.US (Guido van Rossum) Date: Wed, 07 Feb 1996 12:01:39 -0500 Subject: [PYTHON MATRIX-SIG] New pretty printer In-Reply-To: Your message of "Wed, 07 Feb 1996 11:52:33 EST." <199602071652.LAA04701@cyclone.ERE.UMontreal.CA> References: <199602071652.LAA04701@cyclone.ERE.UMontreal.CA> Message-ID: <199602071701.MAA20712@monty.cnri.reston.va.us> > Such things really belong into a "system parameters" collection. > The closest Python equivalent would be the module "sys". But > that doesn't solve the problem of who is allowed to modify them > how and when. It makes sense for it to be a property of the file object. But since adding properties to those is no fun, I'd suggest to go for a parameter is sys. Whoever feels like they can change sys.stdout can also change the line width if they feel like it. --Guido van Rossum URL: ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From hinsenk@ere.umontreal.ca Wed Feb 7 17:11:21 1996 From: hinsenk@ere.umontreal.ca (Konrad HINSEN) Date: Wed, 7 Feb 1996 12:11:21 -0500 Subject: [PYTHON MATRIX-SIG] New pretty printer In-Reply-To: <199602071701.MAA20712@monty.cnri.reston.va.us> (message from Guido van Rossum on Wed, 07 Feb 1996 12:01:39 -0500) Message-ID: <199602071711.MAA07277@cyclone.ERE.UMontreal.CA> It makes sense for it to be a property of the file object. But since adding properties to those is no fun, I'd suggest to go for a parameter is sys. Whoever feels like they can change sys.stdout can also change the line width if they feel like it. OK. Then it makes sense to add another parameter for printing precision and use it also for scalars. ------------------------------------------------------------------------------- Konrad Hinsen | E-Mail: hinsenk@ere.umontreal.ca Departement de chimie | Tel.: +1-514-343-6111 ext. 3953 Universite de Montreal | Fax: +1-514-343-7586 C.P. 6128, succ. Centre-Ville | Deutsch/Esperanto/English/Nederlands/ Montreal (QC) H3C 3J7 | Francais (phase experimentale) ------------------------------------------------------------------------------- ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From jjh@Goldilocks.LCS.MIT.EDU Wed Feb 7 19:48:13 1996 From: jjh@Goldilocks.LCS.MIT.EDU (James Hugunin) Date: Wed, 7 Feb 96 14:48:13 EST Subject: [PYTHON MATRIX-SIG] Steadily approaching BETA release Message-ID: <9602071948.AA17725@baribal.LCS.MIT.EDU.LCS.MIT.EDU> Chris Chase just sent me a bunch of patches to allow a[::-1, 1:-1] style indexing. So the grammar is solidifying. David Ascher's tutorial (and my own TUTORIAL.NumPy) are starting to produce something like an acceptable first pass at documentation. Konrad Hinsen's print function offers acceptable printing of arrays. I've received very few bug reports this week, so the base code is starting to look stable. So, what's left before a general BETA release? Note: I'm going to comment on each of these in a seperate follow-up piece of e-mail in order to try and seperate these two different issues. 1) Type coercion 2) Naming conventions What I've decided is not important to resolve before the BETA release: 1) static vs. dynamic linking (static linking is fine for now) 2) Contents of Numeric.py (I'm going to rule by fiat here) 3) The exact details of Konrad's print function (Unlikley that changes here will break anyone's code) ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From jjh@Goldilocks.LCS.MIT.EDU Wed Feb 7 20:18:00 1996 From: jjh@Goldilocks.LCS.MIT.EDU (James Hugunin) Date: Wed, 7 Feb 96 15:18:00 EST Subject: [PYTHON MATRIX-SIG] Type Coercion Message-ID: <9602072018.AA17817@baribal.LCS.MIT.EDU.LCS.MIT.EDU> I still don't see a great solution to this one. Here's my attempt to boil the issue down to its essence: (While I only talk about floats and doubles here, this same issue comes up with longs and shorts and bytes as well as float and double complex numbers). C (and FORTRAN) has two kinds of floating point numbers, float and double. A double has more precision and range than a float. In C if I multiply a double by a float I get a double. Python has only one floating point type, call this a PyFloat. Internally, these numbers store their data using C doubles. However, the standard C interface to PyFloats allows them to be extracted into either a C float or a C double (PyArg_ParseTuple), so its not clear that a PyFloat should really be considered either a float or a double. Nevertheless, its obvious that PyFloats are more closely related to doubles than floats. An array can be of type float or double. It is reasonable to have the result of multiplying an array of floats by an array of doubles be an array of doubles. What should happen if I multiply an array of floats by a PyFloat? Three things can happen: 1) Return an array of floats (possible undesired loss of precision) 2) Return an array of doubles (possible undesired gain of precision) 3) Raise an exception (very annoying to the user) Currently there is also the convention that when an array of floats or doubles returns a single element, that element will be a PyFloat. Perhaps this behavior should be altered as the following examples of confusing behavior using either approach 1 or approach 2 show. a_float and a_double are 1d arrays of the appropriate type. Using Method 1) a_float[0]*a_float -> a_float (good) a_double[0]*a_float -> a_float (not good) Using Method 2) a_float[0]*a_float -> a_double (not good) a_double[0]*a_float -> a_double (good) This particular problem could be eliminated by having a_float[0] return a 0d array of floats with a single element. Perhaps this is worth considering. The next confusing behavior involves python numeric constants. Again, considering the two non-exception producing methods discussed above: very_precise_constant = 1.2345678910 Using method 1) 0.5*a_float -> a_float (good) very_precise_constant*a_float -> a_float (probably not good) Using method 2) 0.5*a_float -> a_double (not good) very_precise_constant*a_float -> a_double (probably good) Using this method, the only way to write 0.5*a_float and get an array of floats back is to write: array(0.5, Float(32))*a_float This becomes very ugly for any reasonably sized expression. Currently, method 3 is used because it seems to make everybody unhappy, yet it doesn't do anything terribly "wrong". Personally I would prefer to use method 1, combined with the major change to the existing system that scalars are returned as 0d arrays, not as python scalars. This means that python scalars will be considered to have a malleable type (as they in fact seem to) and can be cast to either a float or a double as needed in the particular equation. If you have a high-precision number whose precision you need to hold on to, you will need to say: very_precise_constant = array(1.2345678910, Float(64)) As I said at the beginning, none of these three options really appeals to me, yet I feel that this is a very important issue that really needs to be resolved before the BETA release. Opinions? (like I have to ask on this issue) - Jim ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From hinsenk@ere.umontreal.ca Wed Feb 7 20:51:28 1996 From: hinsenk@ere.umontreal.ca (Konrad HINSEN) Date: Wed, 7 Feb 1996 15:51:28 -0500 Subject: [PYTHON MATRIX-SIG] Steadily approaching BETA release In-Reply-To: <9602071948.AA17725@baribal.LCS.MIT.EDU.LCS.MIT.EDU> (jjh@Goldilocks.LCS.MIT.EDU) Message-ID: <199602072051.PAA21905@cyclone.ERE.UMontreal.CA> So, what's left before a general BETA release? Note: I'm going to comment on each of these in a seperate follow-up piece of e-mail in order to try and seperate these two different issues. 1) Type coercion 2) Naming conventions 3) Practical experience I don't know how much real-life code others have produced with the current release, but I find that even with the relatively modest amount of application code I have written, I keep finding weak spots in the current collection of functions, i.e. operations that are impossible or difficult to code efficiently. It may seem that this is not a major problem, since more functions can always be added later. But that will quickly lead to the same type of function chaos that other "historically evolved" systems suffer from. Sometimes new functionality is best implemented by extending the definition of an existing function, which should however be reflected by a new name - and at that point compatibility considerations make the best solution impossible once the core function set is frozen. To give an example: I need to repeat certain items in an array a specified number of times, e.g. given n = array([1,1,2,3]) x = array([2.,1.,6.,3.5]) I want to obtain something like y = repeat(n,x) yielding array([2.,1.,6.,6.,3.5,3.5,3.5]) I don't see any reasonably efficient way to implement the function repeat() with the current array operations. Supposing that this operation is generally useful (and my APL/J experience definitely indicates that it is), it should be implemented in C in the array module. But a little reflection shows that this is a simple generalization of the already existing function compress(); indeed compress() is identical to repeat() as long as the first argument contains only 0s and 1s. So the most rational design would be to have repeat() and eliminate compress(). This leaves the question of how to find out which operations are necessary and which aren't. Apart from practical experience (which takes a lot of time to accumulate, and much more users than we currently have), it makes sense to look what other array languages have. It has been my intention to check whether common APL/J function can be implemented easily with what we have, but without knowing the existing function set, this is very difficult! What I've decided is not important to resolve before the BETA release: 1) static vs. dynamic linking (static linking is fine for now) 2) Contents of Numeric.py (I'm going to rule by fiat here) 3) The exact details of Konrad's print function (Unlikley that changes here will break anyone's code) I agree. ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From hinsenk@ere.umontreal.ca Wed Feb 7 20:52:09 1996 From: hinsenk@ere.umontreal.ca (Konrad HINSEN) Date: Wed, 7 Feb 1996 15:52:09 -0500 Subject: [PYTHON MATRIX-SIG] Steadily approaching BETA release In-Reply-To: <9602071948.AA17725@baribal.LCS.MIT.EDU.LCS.MIT.EDU> (jjh@Goldilocks.LCS.MIT.EDU) Message-ID: <199602072052.PAA22142@cyclone.ERE.UMontreal.CA> So, what's left before a general BETA release? Note: I'm going to comment on each of these in a seperate follow-up piece of e-mail in order to try and seperate these two different issues. 1) Type coercion 2) Naming conventions 3) Practical experience I don't know how much real-life code others have produced with the current release, but I find that even with the relatively modest amount of application code I have written, I keep finding weak spots in the current collection of functions, i.e. operations that are impossible or difficult to code efficiently. It may seem that this is not a major problem, since more functions can always be added later. But that will quickly lead to the same type of function chaos that other "historically evolved" systems suffer from. Sometimes new functionality is best implemented by extending the definition of an existing function, which should however be reflected by a new name - and at that point compatibility considerations make the best solution impossible once the core function set is frozen. To give an example: I need to repeat certain items in an array a specified number of times, e.g. given n = array([1,1,2,3]) x = array([2.,1.,6.,3.5]) I want to obtain something like y = repeat(n,x) yielding array([2.,1.,6.,6.,3.5,3.5,3.5]) I don't see any reasonably efficient way to implement the function repeat() with the current array operations. Supposing that this operation is generally useful (and my APL/J experience definitely indicates that it is), it should be implemented in C in the array module. But a little reflection shows that this is a simple generalization of the already existing function compress(); indeed compress() is identical to repeat() as long as the first argument contains only 0s and 1s. So the most rational design would be to have repeat() and eliminate compress(). This leaves the question of how to find out which operations are necessary and which aren't. Apart from practical experience (which takes a lot of time to accumulate, and much more users than we currently have), it makes sense to look what other array languages have. It has been my intention to check whether common APL/J function can be implemented easily with what we have, but without knowing the existing function set, this is very difficult! What I've decided is not important to resolve before the BETA release: 1) static vs. dynamic linking (static linking is fine for now) 2) Contents of Numeric.py (I'm going to rule by fiat here) 3) The exact details of Konrad's print function (Unlikley that changes here will break anyone's code) I agree. ------------------------------------------------------------------------------- Konrad Hinsen | E-Mail: hinsenk@ere.umontreal.ca Departement de chimie | Tel.: +1-514-343-6111 ext. 3953 Universite de Montreal | Fax: +1-514-343-7586 C.P. 6128, succ. Centre-Ville | Deutsch/Esperanto/English/Nederlands/ Montreal (QC) H3C 3J7 | Francais (phase experimentale) ------------------------------------------------------------------------------- ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From jjh@Goldilocks.LCS.MIT.EDU Wed Feb 7 21:37:24 1996 From: jjh@Goldilocks.LCS.MIT.EDU (James Hugunin) Date: Wed, 7 Feb 96 16:37:24 EST Subject: [PYTHON MATRIX-SIG] Naming Conventions And Practical Experience Message-ID: <9602072137.AA18068@baribal.LCS.MIT.EDU.LCS.MIT.EDU> Personally, I'm not terribly unhappy with the current naming conventions and feature set. I agree that they could be better, but I lack the time to sit down and do a careful redesign. The current feature set is significantly more powerful than the basic array operations provided by matlab, the most popular array processing language that people buy. Rather than complaining about the lack of a list of the available functions, David Ascher recently sat down and pulled them out of the C and python source code (not too hard to do, since I did include doc strings for everything). Check out his beta tutorial (http://maigret.cog.brown.edu/python/arraytut.html) for the current set of names in use. It's still an early draft with the expected bugs, but its getting close to a fairly complete intro. I'm going to step out of my current role of moderator for this issue. If somebody else out there has the time and energy to design a complete set of functions and methods for the system, go for it. If you write this up nicely and can convince the people in this SIG that your feature set makes sense, I'll be happy to implement it (well, so long as you don't ask me to solve the halting problem). Otherwise, I'm tired of people asking for me to do things like make a better distinction between functions and methods. This is just not useful unless I had more free time than I do. Sorry for the negative tone, but I've been getting a few too many questions like "So how's your speech recognition system coming along?" from my advisor lately and I'm eager to get the array object "out the door". Bug fixes take up very little of my intellectual energy, but these issues of naming conventions and function sets use up way too much. -Jim ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From jfulton@usgs.gov Wed Feb 7 21:39:08 1996 From: jfulton@usgs.gov (Jim Fulton) Date: Wed, 7 Feb 1996 16:39:08 -0500 Subject: [PYTHON MATRIX-SIG] New pretty printer In-Reply-To: hinsenk@ere.umontreal.ca (Konrad HINSEN) "Re: [PYTHON MATRIX-SIG] New pretty printer" (Feb 7, 12:11pm) References: <199602071711.MAA07277@cyclone.ERE.UMontreal.CA> Message-ID: <9602071639.ZM12755@dsdbqvarsa.er.usgs.gov> On Feb 7, 12:11pm, Konrad HINSEN wrote: > Subject: Re: [PYTHON MATRIX-SIG] New pretty printer > > It makes sense for it to be a property of the file object. But since > adding properties to those is no fun, I'd suggest to go for a > parameter is sys. Whoever feels like they can change sys.stdout can > also change the line width if they feel like it. > > OK. Then it makes sense to add another parameter for printing > precision and use it also for scalars. I'd rather see precision handled as an array attribute. Jim ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From dubois1@llnl.gov Wed Feb 7 21:44:09 1996 From: dubois1@llnl.gov (Paul. Dubois) Date: Wed, 07 Feb 1996 13:44:09 -0800 Subject: [PYTHON MATRIX-SIG] Type Coercion References: <9602072018.AA17817@baribal.LCS.MIT.EDU.LCS.MIT.EDU> Message-ID: <31191D29.D42@llnl.gov> I hold these truths to be self-evident, your argument notwithstanding: a. A PyFloat is a C double The fact that some of the Python API lets you convert one to a C float easily doesn't change this. b. It is a GOOD THING that you can count on the precision of a PyFloat. Writing portable code between platforms is easy with Python, since even on 64 bit machines C doubles are implemented as 64 bit floats. (I know that isn't for certain but as a practical truth, it is true). c. It is a GOOD THING not to lose precision. d. It is a GOOD THING not to get an exception. The original design functioned perfectly, as far as I was concerned. The change has been made because of a desire not to GAIN precision. This is only a concern if you have a concern about space, really. (The time penalty on most workstations for double vs. float is nowhere near 2). So this means you must have a humongous amount of data. In which case, Python is a lousy choice anyway, since it will make lots of temporaries in the act of expression evaluation, while a nice Fortran routine won't. In short, I'd argue that it is not a good idea to pervert the matrix class in Python to satisfy a marginal need. But the alternative is unclean. The minute you find yourself proposing that elements of an array are zero-d arrays you should realize the slope has gotten too slippery. I return to my previous horror example: (1./3.) * a_float[i] vs. ((1./3.) * a_float)[i] Can you really stand for those to be different? Vectorize a loop and change the answer? -- Paul F. Dubois, L-472 (510)-422-5426 Lawrence Livermore National Laboratory FAX (510)-423-9969 Livermore, CA 94550 dubois1@llnl.gov Consulting: PaulDubois@aol.com ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From hinsenk@ere.umontreal.ca Wed Feb 7 21:46:08 1996 From: hinsenk@ere.umontreal.ca (Konrad HINSEN) Date: Wed, 7 Feb 1996 16:46:08 -0500 Subject: [PYTHON MATRIX-SIG] New pretty printer In-Reply-To: <9602071639.ZM12755@dsdbqvarsa.er.usgs.gov> (jfulton@usgs.gov) Message-ID: <199602072146.QAA24976@cyclone.ERE.UMontreal.CA> > OK. Then it makes sense to add another parameter for printing > precision and use it also for scalars. I'd rather see precision handled as an array attribute. But it isn't. If precision were an attribute of an object, you would expect arithmetic etc. to respect it. But what we are discussing is just a matter of output formatting. To be consistent, you would also have to require that the radix for integer output be a property of an object, creating separate types for decimal and hex numbers. If precision were an attribute, we would also have to decide what the precision of the result of an array operation is. This just doesn't make sense. ------------------------------------------------------------------------------- Konrad Hinsen | E-Mail: hinsenk@ere.umontreal.ca Departement de chimie | Tel.: +1-514-343-6111 ext. 3953 Universite de Montreal | Fax: +1-514-343-7586 C.P. 6128, succ. Centre-Ville | Deutsch/Esperanto/English/Nederlands/ Montreal (QC) H3C 3J7 | Francais (phase experimentale) ------------------------------------------------------------------------------- ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From jfulton@usgs.gov Wed Feb 7 21:55:36 1996 From: jfulton@usgs.gov (Jim Fulton) Date: Wed, 7 Feb 1996 16:55:36 -0500 Subject: [PYTHON MATRIX-SIG] New pretty printer In-Reply-To: hinsenk@ERE.UMontreal.CA (Konrad HINSEN) "Re: [PYTHON MATRIX-SIG] New pretty printer" (Feb 7, 4:46pm) References: <199602072146.QAA24976@cyclone.ERE.UMontreal.CA> Message-ID: <9602071655.ZM12804@dsdbqvarsa.er.usgs.gov> On Feb 7, 4:46pm, Konrad HINSEN wrote: > Subject: Re: [PYTHON MATRIX-SIG] New pretty printer > > > OK. Then it makes sense to add another parameter for printing > > precision and use it also for scalars. > > I'd rather see precision handled as an array attribute. > > But it isn't. If precision were an attribute of an object, you would > expect arithmetic etc. to respect it. But what we are discussing is > just a matter of output formatting. To be consistent, you would > also have to require that the radix for integer output be a > property of an object, creating separate types for decimal and > hex numbers. > > If precision were an attribute, we would also have to decide what the > precision of the result of an array operation is. This just doesn't > make sense. Perhaps precision is a poor term. "output precision" is fine with me. My opinion is based on two ideas: - The output precision is needed specifically for printing arrays, mainly because you have lots of data to output. - Global variables are bad. A correlary is that changes to core things like sys is bad. I don't mind output width beig added to sys, because it clearly is not array specific. (Well, maybe I do. :) Jim ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From jfulton@usgs.gov Wed Feb 7 21:56:30 1996 From: jfulton@usgs.gov (Jim Fulton) Date: Wed, 7 Feb 1996 16:56:30 -0500 Subject: [PYTHON MATRIX-SIG] Type Coercion In-Reply-To: "Paul. Dubois" "Re: [PYTHON MATRIX-SIG] Type Coercion" (Feb 7, 1:44pm) References: <9602072018.AA17817@baribal.LCS.MIT.EDU.LCS.MIT.EDU> <31191D29.D42@llnl.gov> Message-ID: <9602071656.ZM12810@dsdbqvarsa.er.usgs.gov> I agree. On Feb 7, 1:44pm, Paul. Dubois wrote: > Subject: Re: [PYTHON MATRIX-SIG] Type Coercion > I hold these truths to be self-evident, your argument notwithstanding: > a. A PyFloat is a C double > The fact that some of the Python API lets you convert one to a C > float easily doesn't change this. > > b. It is a GOOD THING that you can count on the precision of a PyFloat. > Writing portable code between platforms is easy with Python, since > even on 64 bit machines C doubles are implemented as 64 bit floats. (I > know that isn't for certain but as a practical truth, it is true). > > c. It is a GOOD THING not to lose precision. > d. It is a GOOD THING not to get an exception. > > The original design functioned perfectly, as far as I was concerned. > > The change has been made because of a desire not to GAIN precision. This > is only a concern if you have a concern about space, really. (The time > penalty on most workstations for double vs. float is nowhere near 2). > So this means you must have a humongous amount of data. In which case, > Python is a lousy choice anyway, since it will make lots of temporaries > in the act of expression evaluation, while a nice Fortran routine won't. > > In short, I'd argue that it is not a good idea to pervert the matrix > class in Python to satisfy a marginal need. > > But the alternative is unclean. The minute you find yourself proposing > that elements of an array are zero-d arrays you should realize the slope > has gotten too slippery. > > I return to my previous horror example: > > (1./3.) * a_float[i] > vs. > ((1./3.) * a_float)[i] > > Can you really stand for those to be different? Vectorize a loop and > change the answer? > -- > Paul F. Dubois, L-472 (510)-422-5426 > Lawrence Livermore National Laboratory FAX (510)-423-9969 > Livermore, CA 94550 dubois1@llnl.gov > Consulting: PaulDubois@aol.com > > ================= > MATRIX-SIG - SIG on Matrix Math for Python > > send messages to: matrix-sig@python.org > administrivia to: matrix-sig-request@python.org > ================= >-- End of excerpt from Paul. Dubois ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From hinsenk@ere.umontreal.ca Wed Feb 7 21:59:44 1996 From: hinsenk@ere.umontreal.ca (Konrad HINSEN) Date: Wed, 7 Feb 1996 16:59:44 -0500 Subject: [PYTHON MATRIX-SIG] Naming Conventions And Practical Experience In-Reply-To: <9602072137.AA18068@baribal.LCS.MIT.EDU.LCS.MIT.EDU> (jjh@Goldilocks.LCS.MIT.EDU) Message-ID: <199602072159.QAA25799@cyclone.ERE.UMontreal.CA> lack the time to sit down and do a careful redesign. The current feature set is significantly more powerful than the basic array operations provided by matlab, the most popular array processing language that people buy. Also the worst one, mostly because it is "historically grown" from a very restrictive basis (LINPACK). Rather than complaining about the lack of a list of the available functions, David Ascher recently sat down and pulled them out of the C and python source code (not too hard to do, since I did include doc strings for everything). Check out his beta tutorial I know, and in fact I have postponed my own attempts waiting for him to be continue the investigation. I wasn't really complaining, but providing an excuse for not having done something until now. If somebody else out there has the time and energy to design a complete set of functions and methods for the system, go for it. If I volunteer to do so, but can't promise any deadline. I am also doing all my work on Python as a side project, so I fully sympathize with your problems of allocating time to it! Sorry for the negative tone, but I've been getting a few too many questions like "So how's your speech recognition system coming along?" from my advisor lately and I'm eager to get the array object "out the door". Bug fixes take up very little of my intellectual energy, but I fully understand that, but I don't see the need to rush. After all, we are doing something close to language design, which has never profited from tight deadlines. Rather than rush out a beta version to the public, I advocate a public alpha release that comes with no promise about stability of function names etc. We need all the test users we can get! Why make any commitments without anybody forcing us? ------------------------------------------------------------------------------- Konrad Hinsen | E-Mail: hinsenk@ere.umontreal.ca Departement de chimie | Tel.: +1-514-343-6111 ext. 3953 Universite de Montreal | Fax: +1-514-343-7586 C.P. 6128, succ. Centre-Ville | Deutsch/Esperanto/English/Nederlands/ Montreal (QC) H3C 3J7 | Francais (phase experimentale) ------------------------------------------------------------------------------- ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From hinsenk@ere.umontreal.ca Wed Feb 7 21:36:58 1996 From: hinsenk@ere.umontreal.ca (Konrad HINSEN) Date: Wed, 7 Feb 1996 16:36:58 -0500 Subject: [PYTHON MATRIX-SIG] Type Coercion In-Reply-To: <9602072018.AA17817@baribal.LCS.MIT.EDU.LCS.MIT.EDU> (jjh@Goldilocks.LCS.MIT.EDU) Message-ID: <199602072136.QAA24352@cyclone.ERE.UMontreal.CA> Python has only one floating point type, call this a PyFloat. Internally, these numbers store their data using C doubles. However, the standard C interface to PyFloats allows them to be extracted into either a C float or a C double (PyArg_ParseTuple), so its not clear that a PyFloat should really be considered either a float or a double. The association of PyFloats with C floats exists only in the C interface, presumably to facilitate interfacing to existing C code. No part of the standard language makes use of it, and to an average Python user PyFloat are definitely C doubles. Three things can happen: 1) Return an array of floats (possible undesired loss of precision) 2) Return an array of doubles (possible undesired gain of precision) 3) Raise an exception (very annoying to the user) The potential problems in 1) and 2) are also very annoying to the user! This particular problem could be eliminated by having a_float[0] return a 0d array of floats with a single element. Perhaps this is worth considering. I don't see any serious problem with that. Most Python code will work just as well with an array of rank 0 as with a scalar. Only code that does explicit type checks will have to take care of the difference. And explicit conversion would always be possible using the standard conversion functions. Personally I would prefer to use method 1, combined with the major change to the existing system that scalars are returned as 0d arrays, not as python scalars. Personally I see this as the least desirable option, my preference being in the order 3) 2) 1). My argument is again the principle of least surprise: if something unexpected is going to happen, I prefer it to be 1) an exception 2) a loss in efficiency 3) a loss in accuracy. in that order. Let me propose another solution, fully knowing that Guido might throw a 16-ton weight at me: Suppose we modify the Python float object by adding a flag "single precision". This flag would be set if either 1) A constant is created that can be represented in a C float (such as 0.5). 2) A constant is created with a special suffix like 'S' (to be used for low-precision constants with no exact binary representation, such as 0.1, if precision doesn't matter). 3) When a PyFloat is generated as a rank-0 float array. 4) Possibly as the result of an explicit conversion to a "low precision scalar", if that turns out to be useful. The default behaviour of PyFloats would not change at all, and so there are no compatibility problems. Any result of a scalar operation would not have the flag set, of course. There is also no significant speed penalty, since only the creation of a constant becomes more involved, and that happens at compile time. The only price to pay would be an increase in the size of a float object. The only place where the new flag would matter is in coercion to an array object. It should take care of most the problems of method 2), which after all is the most common coercion convention in programming languages. Unwanted promotion to double can still occur in situations such as 2.*pi*array([1.,2.,3.], Float(32)), because 2.*pi would be "non-degradable" even if pi was, because it is the result of a scalar operation. But such standard cases can be handled by something like twopi = short_float(2.*pi), where short_float() is the conversion function mentioned above. To help pinning down unwanted coercion to double, one could further offer some option that raises an exception for such operations. This would have the status of a debugging aid rather than a language feature and could, for example, be activated by a command line switch. ------------------------------------------------------------------------------- Konrad Hinsen | E-Mail: hinsenk@ere.umontreal.ca Departement de chimie | Tel.: +1-514-343-6111 ext. 3953 Universite de Montreal | Fax: +1-514-343-7586 C.P. 6128, succ. Centre-Ville | Deutsch/Esperanto/English/Nederlands/ Montreal (QC) H3C 3J7 | Francais (phase experimentale) ------------------------------------------------------------------------------- ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From hinsenk@ere.umontreal.ca Wed Feb 7 22:13:56 1996 From: hinsenk@ere.umontreal.ca (Konrad HINSEN) Date: Wed, 7 Feb 1996 17:13:56 -0500 Subject: [PYTHON MATRIX-SIG] New pretty printer In-Reply-To: <9602071655.ZM12804@dsdbqvarsa.er.usgs.gov> (jfulton@usgs.gov) Message-ID: <199602072213.RAA26801@cyclone.ERE.UMontreal.CA> Perhaps precision is a poor term. "output precision" is fine with me. I guess the name is the least problem... The main question remains: If precision becomes a property, what is the precision of a+b, if a has precision 2 and b has precision 7? And why? If this question doesn't make sense, then output precision should not be an attribute. My opinion is based on two ideas: - The output precision is needed specifically for printing arrays, mainly because you have lots of data to output. Needed, yes. But it makes sense and could be easily implemented for scalars as well. - Global variables are bad. A correlary is that changes to core things like sys is bad. I don't mind output width beig added to sys, because it clearly is not array specific. (Well, maybe I do. :) Line width is just as much array specific, as long as no other module makes use of it. I think we are overly dramatizing the "global variable" problem here. There is very little reason to change this global variable except for interactive use. Any module that want output with a specific format would call the output function Numeric.arrayToString() directly and supply line width and precision as optional arguments. The default values from the global variables would be used *only* when someone writes "print a". They should be set only by the end user, never by any library code. ------------------------------------------------------------------------------- Konrad Hinsen | E-Mail: hinsenk@ere.umontreal.ca Departement de chimie | Tel.: +1-514-343-6111 ext. 3953 Universite de Montreal | Fax: +1-514-343-7586 C.P. 6128, succ. Centre-Ville | Deutsch/Esperanto/English/Nederlands/ Montreal (QC) H3C 3J7 | Francais (phase experimentale) ------------------------------------------------------------------------------- ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From dubois1@llnl.gov Wed Feb 7 22:40:23 1996 From: dubois1@llnl.gov (Paul. Dubois) Date: Wed, 07 Feb 1996 14:40:23 -0800 Subject: [PYTHON MATRIX-SIG] printing arrays References: <199602071711.MAA07277@cyclone.ERE.UMontreal.CA> <9602071639.ZM12755@dsdbqvarsa.er.usgs.gov> Message-ID: <31192A57.6665@llnl.gov> Here is my take on this: >From a purely object-oriented point of view, the width/precision are attributes of the *array formatter*, ie an object that knows how to print arrays. At the moment, this gets confounded with the array itself. The default formatter object could be a class variable (i.e., when an array is created, it has as its default formatter a single shared formatter). This means that changing the precision in the shared formatter would change it across the board. The discussion has really arisen because array isn't really a class and the "object" that does the printing in it is a function that doesn't have any state, so we've started to talk about where to put that state. In general, while I appreciate Jim Fulton's concern about compartmentalizing things, I think in practice the idea of a user-changeable default printing precision is well established in similar products. "How do I increase the number of decimals" is certainly a FAQ for Basis users. They don't ask, "How do I increase the number of decimals for printing THIS array only." -- Paul F. Dubois, L-472 (510)-422-5426 Lawrence Livermore National Laboratory FAX (510)-423-9969 Livermore, CA 94550 dubois1@llnl.gov Consulting: PaulDubois@aol.com ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From jfulton@usgs.gov Wed Feb 7 22:52:47 1996 From: jfulton@usgs.gov (Jim Fulton) Date: Wed, 7 Feb 1996 17:52:47 -0500 Subject: [PYTHON MATRIX-SIG] printing arrays In-Reply-To: "Paul. Dubois" "[PYTHON MATRIX-SIG] printing arrays" (Feb 7, 2:40pm) References: <199602071711.MAA07277@cyclone.ERE.UMontreal.CA> <9602071639.ZM12755@dsdbqvarsa.er.usgs.gov> <31192A57.6665@llnl.gov> Message-ID: <9602071752.ZM12949@dsdbqvarsa.er.usgs.gov> On Feb 7, 2:40pm, Paul. Dubois wrote: > Subject: [PYTHON MATRIX-SIG] printing arrays > Here is my take on this: > > >From a purely object-oriented point of view, the width/precision are > attributes of the *array formatter*, ie an object that knows how to > print arrays. At the moment, this gets confounded with the array itself. > The default formatter object could be a class variable (i.e., when an > array is created, it has as its default formatter a single shared > formatter). This means that changing the precision in the shared > formatter would change it across the board. > > The discussion has really arisen because array isn't really a class and > the "object" that does the printing in it is a function that doesn't > have any state, so we've started to talk about where to put that state. > > In general, while I appreciate Jim Fulton's concern about > compartmentalizing things, I think in practice the idea of a > user-changeable default printing precision is well established in > similar products. "How do I increase the number of decimals" is > certainly a FAQ for Basis users. They don't ask, "How do I increase the > number of decimals for printing THIS array only." OK, I have no problem with a user settable default precision. Why not have a default set in Numeric and let arrays also have attributes that are initialized with the default, and can be set by the user? Actually, my main gripe at this point is that I don't like diddling sys for. I can live with having global variables set in some array-specific module. Jim ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From hinsenk@ere.umontreal.ca Wed Feb 7 22:26:00 1996 From: hinsenk@ere.umontreal.ca (Konrad HINSEN) Date: Wed, 7 Feb 1996 17:26:00 -0500 Subject: [PYTHON MATRIX-SIG] Type Coercion In-Reply-To: <31191D29.D42@llnl.gov> (dubois1@llnl.gov) Message-ID: <199602072226.RAA27524@cyclone.ERE.UMontreal.CA> The change has been made because of a desire not to GAIN precision. This is only a concern if you have a concern about space, really. (The time penalty on most workstations for double vs. float is nowhere near 2). So this means you must have a humongous amount of data. In which case, Python is a lousy choice anyway, since it will make lots of temporaries in the act of expression evaluation, while a nice Fortran routine won't. You might need floats just for interfacing with existing code. Or for a big application in which most of the processing happens in C/Fortran modules. Anyway, if there is a need to have float arrays, then there is also a need to keep them to float size in a reasonable way. ------------------------------------------------------------------------------- Konrad Hinsen | E-Mail: hinsenk@ere.umontreal.ca Departement de chimie | Tel.: +1-514-343-6111 ext. 3953 Universite de Montreal | Fax: +1-514-343-7586 C.P. 6128, succ. Centre-Ville | Deutsch/Esperanto/English/Nederlands/ Montreal (QC) H3C 3J7 | Francais (phase experimentale) ------------------------------------------------------------------------------- ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From da@maigret.cog.brown.edu Wed Feb 7 23:09:29 1996 From: da@maigret.cog.brown.edu (David Ascher) Date: Wed, 7 Feb 1996 18:09:29 -0500 (EST) Subject: [PYTHON MATRIX-SIG] Naming Conventions And Practical Experience In-Reply-To: <199602072159.QAA25799@cyclone.ERE.UMontreal.CA> from "Konrad HINSEN" at Feb 7, 96 04:59:44 pm Message-ID: <199602072309.SAA18847@maigret> > If somebody else out there has the time and energy to design a > complete set of functions and methods for the system, go for it. If > > I volunteer to do so, but can't promise any deadline. I am also doing > all my work on Python as a side project, so I fully sympathize with > your problems of allocating time to it! You know, I think we should setup a support group for people who'se lives are disrupted by this damn computer language. Hell, my SO would qualify under that definition. With a special low membership fee for PhD students, of course. --da ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From dubois1@llnl.gov Wed Feb 7 23:10:40 1996 From: dubois1@llnl.gov (Paul. Dubois) Date: Wed, 07 Feb 1996 15:10:40 -0800 Subject: [PYTHON MATRIX-SIG] Type Coercion References: <199602072226.RAA27524@cyclone.ERE.UMontreal.CA> Message-ID: <31193170.31B3@llnl.gov> Konrad remarks that you have to "keep" yourself at the lower precision so you can interact to C/Fortran. This is not strictly correct. You just have to get yourself *back* first. But: We have been doing just that lately and in fact you have to do a conversion of some sort nearly every time if you want to be safe, anyway. This has to do with the contiguous issue. For example, you have a C plotting array that wants an array of C floats. You might even have an array of C floats but in general you don't know it is contiguous. It might be a slice of something else, or the result of any number of calculations that destroy contiguity (if that is a word!). So even if you have made yourself double, you aren't any worse off as you repack yourself contiguous/float to get to the C routine. -- Paul F. Dubois, L-472 (510)-422-5426 Lawrence Livermore National Laboratory FAX (510)-423-9969 Livermore, CA 94550 dubois1@llnl.gov Consulting: PaulDubois@aol.com ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From hinsenk@ere.umontreal.ca Wed Feb 7 23:10:49 1996 From: hinsenk@ere.umontreal.ca (Konrad HINSEN) Date: Wed, 7 Feb 1996 18:10:49 -0500 Subject: [PYTHON MATRIX-SIG] printing arrays In-Reply-To: <9602071752.ZM12949@dsdbqvarsa.er.usgs.gov> (jfulton@usgs.gov) Message-ID: <199602072310.SAA01109@cyclone.ERE.UMontreal.CA> Actually, my main gripe at this point is that I don't like diddling sys for. I can live with having global variables set in some array-specific module. I don't care too much whether this goes to Numeric or sys. I still consider sys more appropriate just because these definitions are not specific to arrays, but I won't insist. ------------------------------------------------------------------------------- Konrad Hinsen | E-Mail: hinsenk@ere.umontreal.ca Departement de chimie | Tel.: +1-514-343-6111 ext. 3953 Universite de Montreal | Fax: +1-514-343-7586 C.P. 6128, succ. Centre-Ville | Deutsch/Esperanto/English/Nederlands/ Montreal (QC) H3C 3J7 | Francais (phase experimentale) ------------------------------------------------------------------------------- ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From dubois1@llnl.gov Wed Feb 7 23:12:22 1996 From: dubois1@llnl.gov (Paul. Dubois) Date: Wed, 07 Feb 1996 15:12:22 -0800 Subject: [PYTHON MATRIX-SIG] [Fwd: Foreign Function Interface Generator available.] Message-ID: <311931D6.6B1E@llnl.gov> This is a multi-part message in MIME format. --------------521F5C6038EE Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit I thought this audience might find this interesting, considering some of the discussions at Spam III. -- Paul F. Dubois, L-472 (510)-422-5426 Lawrence Livermore National Laboratory FAX (510)-423-9969 Livermore, CA 94550 dubois1@llnl.gov Consulting: PaulDubois@aol.com --------------521F5C6038EE Content-Type: message/rfc822 Content-Transfer-Encoding: 7bit Content-Disposition: inline Return-Path: Received: from icf.llnl.gov by kristen.llnl.gov (5.x/SMI-SVR4) id AA17149; Wed, 7 Feb 1996 12:28:33 -0800 Received: from pierce.llnl.gov by icf.llnl.gov (4.1/LLNL-1.19) id AA00439; Wed, 7 Feb 96 12:28:32 PST Received: by pierce.llnl.gov (8.6.10/LLNL-1.18/llnl.gov-03.95) id MAA20155; Wed, 7 Feb 1996 12:28:31 -0800 Received: from alpha.xerox.com by pierce.llnl.gov (8.6.10/LLNL-1.18/llnl.gov-03.95) id MAA20146; Wed, 7 Feb 1996 12:28:29 -0800 Received: from Homer.Parc.Xerox.xns by alpha.xerox.com via XNS id <15450(15)>; Wed, 7 Feb 1996 12:19:11 PST Received: from skinner.cs.uoregon.edu ([128.223.4.13]) by alpha.xerox.com with SMTP id <15439(2)>; Wed, 7 Feb 1996 12:06:23 PST Received: from blackrabbit.cs.uoregon.edu by skinner.cs.uoregon.edu with SMTP id AA13990 (5.65/IDA-1.4.2 for ilu@parc.xerox.com); Wed, 7 Feb 96 12:05:22 -0800 Received: from blackrabbit.cs.uoregon.edu (localhost [127.0.0.1]) by blackrabbit.cs.uoregon.edu (8.6.12/8.6.12) with ESMTP id MAA20417; Wed, 7 Feb 1996 12:02:27 -0800 X-Ns-Transport-Id: 0800207424C500CC559B Date: Wed, 7 Feb 1996 12:02:26 PST From: Lars Thomas Hansen Subject: Foreign Function Interface Generator available. To: fjh%cs.mu.OZ.AU@uunet.uu.net, will@ccs.neu.edu, jcg@cs.cmu.edu, gurr@snap.med.ge.com, nr@cs.purdue.edu, Geoff.Wyant@east.sun.com, kalsow@sctc.com, nayeri@gte.com, ilu@parc.xerox.com, scheme@mc.lcs.mit.edu Cc: lth@cs.uoregon.edu, malony@cs.uoregon.edu, hersey@cs.uoregon.edu Message-Id: <199602072002.MAA20417@blackrabbit.cs.uoregon.edu> Content-Type: text Content-Length: 901 X-Mozilla-Status: 0001 FFIGEN Release 1 is now available from http://www.cs.uoregon.edu/~lth/ffigen. FFIGEN (Foreign Function Interface GENerator) is a program suite which facilitates the writing of translators from C header files to foreign function interfaces for particular language implementations. It is based on the lcc C compiler and handles nearly all of ANSI C, including the preprocessor. (Missing features are bitfields and general type qualifiers; support for those will be included in a future version). This release includes the entire front-end, a back-end for Chez Scheme version 5, and documents which explain the ideas behind FFIGEN, its use, and how to write of custom back-ends for you language. This is a preliminary release; it works but is neither full-function nor polished. In particular, installation is crude. See the Web page for further details, download information, and so on. --lars --------------521F5C6038EE-- ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From hinsenk@ere.umontreal.ca Wed Feb 7 23:21:19 1996 From: hinsenk@ere.umontreal.ca (Konrad HINSEN) Date: Wed, 7 Feb 1996 18:21:19 -0500 Subject: [PYTHON MATRIX-SIG] Type Coercion In-Reply-To: <31193170.31B3@llnl.gov> (dubois1@llnl.gov) Message-ID: <199602072321.SAA01631@cyclone.ERE.UMontreal.CA> a C plotting array that wants an array of C floats. You might even have an array of C floats but in general you don't know it is contiguous. It might be a slice of something else, or the result of any number of calculations that destroy contiguity (if that is a word!). Believe it or not, it is accepted by "spell" on my system ;-) Contiguity might not be an issue when interfacing to Fortran code, which can often handle non-contiguous arrays. Anyway, the C-API should have the utility functions "copy if not contiguous" and "copy if not sufficiently contiguous to be passed to a Fortran function". There could be versions that include a cast. ------------------------------------------------------------------------------- Konrad Hinsen | E-Mail: hinsenk@ere.umontreal.ca Departement de chimie | Tel.: +1-514-343-6111 ext. 3953 Universite de Montreal | Fax: +1-514-343-7586 C.P. 6128, succ. Centre-Ville | Deutsch/Esperanto/English/Nederlands/ Montreal (QC) H3C 3J7 | Francais (phase experimentale) ------------------------------------------------------------------------------- ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From jjh@Goldilocks.LCS.MIT.EDU Wed Feb 7 23:30:50 1996 From: jjh@Goldilocks.LCS.MIT.EDU (James Hugunin) Date: Wed, 7 Feb 96 18:30:50 EST Subject: [PYTHON MATRIX-SIG] Type Coercion In-Reply-To: <199602072321.SAA01631@cyclone.ERE.UMontreal.CA> (hinsenk@ere.umontreal.ca) Message-ID: <9602072330.AA18989@baribal.LCS.MIT.EDU.LCS.MIT.EDU> From: hinsenk@ere.umontreal.ca (Konrad HINSEN) a C plotting array that wants an array of C floats. You might even have an array of C floats but in general you don't know it is contiguous. It might be a slice of something else, or the result of any number of calculations that destroy contiguity (if that is a word!). Believe it or not, it is accepted by "spell" on my system ;-) Contiguity might not be an issue when interfacing to Fortran code, which can often handle non-contiguous arrays. Anyway, the C-API should have the utility functions "copy if not contiguous" and "copy if not sufficiently contiguous to be passed to a Fortran function". There could be versions that include a cast. Just to clarify the facts on this issue. The C-API has the standard function PyArray_ContiguousFromObject which is the standard method of obtaining an array. If the object is a contiguous PyArray of the same type you desire, no memory copying occurs. Otherwise there is obviously a copying step and possibly a cast required. The two other "standard" ways of obtaining an array from a python object are PyArray_CopyFromObject which guarantees you get a copy and can do with the memory area what you please. PyArray_FromObject which can give you discontiguous arrays if you want to deal with them (this is what the internal ofuncs use). So the facts are that there is in fact a cost proportional to the size of the array when you pass an array of doubles to one expecting floats. However, the cast operation is fairly efficient, so I doubt that this particular case really matters in practice at all. ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From hinsenk@ere.umontreal.ca Thu Feb 8 00:00:54 1996 From: hinsenk@ere.umontreal.ca (Konrad HINSEN) Date: Wed, 7 Feb 1996 19:00:54 -0500 Subject: [PYTHON MATRIX-SIG] Pretty Printer Message-ID: <199602080000.TAA04035@cyclone.ERE.UMontreal.CA> And here's another pretty printer. Float and complex formatting is much improved, but there is still no line wrapping. Precision control works, however. Note that due to the problem in compress() complex arrays will still often be printed with an inappropriate format. It is also still impossible to print a float array whose elements are all zero. This will be possible with one of Jim's upcoming bug fixes. Line width and precision are currently taken from global variables in sys if these exist and no explicit parameters are given. If the general opinion favors Numeric, I'll be happy to change this. For now you should not forget to put an "import sys" in your Numeric.py. Before I forget: Precision is indicated by an integer specifying the number of decimal digits after the decimal point. If there are good arguments for another specification, I'll be happy to change this. I am now quite happy with formatting of floats and complex numbers, although I admit that I haven't done extensive tests. I hope there won't be complaints about speed or lack thereof. I'd appreciate any feedback on how you like the float formatting. I now remove all trailing zeros, although Python scalars get printed with at least one. Would you prefer full compatibility in this case? Or would you prefer to have no decimal point if there are no fractional digits? And now the code: --------------------------------------------------------------------------- import sys def arrayToString(a, max_line_width = None, precision = None): if not max_line_width: try: max_line_width = sys.output_line_width except AttributeError: max_line_width = 77 if not precision: try: precision = sys.float_output_precision except AttributeError: precision = 8 if a.contiguous(): data = a.reshape(None) else: data = a.copy().reshape(None) type = a.typecode() items_per_line = a.shape[-1] if type == 'b' or type == '1' or type == 's' or type == 'i' \ or type == 'l': max_str_len = max(len(str(maximum.reduce(data))), len(str(minimum.reduce(data)))) format = '%' + str(max_str_len) + 'd' item_length = max_str_len+1 format_function = lambda x, f = format: _formatInteger(x, f) elif type == 'f' or type == 'd': format, item_length = _floatFormat(data, precision) format_function = lambda x, f = format: _formatFloat(x, f) elif type == 'F' or type == 'D': real_format, real_item_length = _floatFormat(data.real, precision, sign=0) imag_format, imag_item_length = _floatFormat(data.imaginary, precision, sign=1) item_length = real_item_length + imag_item_length + 2 format_function = lambda x, f1 = real_format, f2 = imag_format: \ _formatComplex(x, f1, f2) elif type == 'c': format = '%s' item_length = 1 format_function = lambda x, f = format: _formatCharacter(x, f) else: # nothing for 'O' return str(a) line_width = item_length*items_per_line - (type != 'c') if line_width > max_line_width: pass return _arrayToString(a, format_function, len(a.shape), line_width)[:-1] def _floatFormat(data, precision, sign = 0): exp_format = 0 non_zero = abs(compress(data.notEqual(0), data)) if len(non_zero) == 0: max_val = 0. min_val = 0. else: max_val = maximum.reduce(non_zero) min_val = minimum.reduce(non_zero) if max_val >= 1.e12 or min_val < 0.0001 or max_val/min_val > 1000.: exp_format = 1 if exp_format: large_exponent = 0 < min_val < 1e-99 or max_val >= 1e100 max_str_len = 8 + precision + large_exponent if sign: format = '%+' else: format = '%' format = format + str(max_str_len) + '.' + str(precision) + 'e' if large_exponent: format = format + '3' item_length = max_str_len + 1 else: format = '%.' + str(precision) + 'f' precision = min(precision, apply(max, tuple(map(lambda x, p=precision, f=format: _digits(x,p,f), data)))) max_str_len = len(str(int(max_val))) + precision + 2 if sign: format = '%#+' else: format = '%#' format = format + str(max_str_len) + '.' + str(precision) + 'f' item_length = max_str_len + 1 return (format, item_length) def _digits(x, precision, format): s = format % x zeros = len(s) while s[zeros-1] == '0': zeros = zeros-1 return precision-len(s)+zeros def _arrayToString(a, format_function, rank, line_width): if rank == 0: return str(a[0]) elif rank == 1: s = '' for i in range(a.shape[0]): s = s + format_function(a[i]) if s[-1] == ' ': s = s[:-1] s = s + '\n' else: s = '' for i in range(a.shape[0]-1): s = s + _arrayToString(a[i], format_function, rank-1, line_width) if rank == 3: s = s + '\n' elif rank > 3: s = s + (rank-3)*(line_width*'-'+'\n') s = s + _arrayToString(a[a.shape[0]-1], format_function, rank-1, line_width) return s def _formatInteger(x, format): return format % x + ' ' def _formatFloat(x, format, strip_zeros = 1): if format[-1] == '3': format = format[:-1] s = format % x third = s[-3] if third == '+' or third == '-': s = s[1:-2] + '0' + s[-2:] elif format[-1] == 'f': s = format % x if strip_zeros: zeros = len(s) while s[zeros-1] == '0': zeros = zeros-1 s = s[:zeros] + (len(s)-zeros)*' ' else: s = format % x return s + ' ' def _formatComplex(x, real_format, imag_format): r = _formatFloat(x.real, real_format)[:-1] i = _formatFloat(x.imag, imag_format, 0)[:-1] spaces = 0 while r[spaces] == ' ': spaces = spaces + 1 r = spaces*' ' + '(' + r[spaces:] if imag_format[-1] == 'f': zeros = len(i) while i[zeros-1] == '0': zeros = zeros-1 i = i[:zeros] + 'j)' + (len(i)-zeros)*' ' else: i = i + 'j)' return r + i + ' ' def _formatCharacter(x, format): return format % x ------------------------------------------------------------------------------- Konrad Hinsen | E-Mail: hinsenk@ere.umontreal.ca Departement de chimie | Tel.: +1-514-343-6111 ext. 3953 Universite de Montreal | Fax: +1-514-343-7586 C.P. 6128, succ. Centre-Ville | Deutsch/Esperanto/English/Nederlands/ Montreal (QC) H3C 3J7 | Francais (phase experimentale) ------------------------------------------------------------------------------- ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From stoll@atr-sw.atr.co.jp Thu Feb 8 03:11:01 1996 From: stoll@atr-sw.atr.co.jp (Perry A. Stoll) Date: Thu, 08 Feb 1996 12:11:01 +0900 Subject: [PYTHON MATRIX-SIG] Pretty Printer In-Reply-To: Your message of "Wed, 07 Feb 1996 19:00:54 JST" References: <199602080000.TAA04035@cyclone.ERE.UMontreal.CA> Message-ID: <199602080311.MAA03526@ciris21.atr-sw.atr.co.jp> As Konrad's pretty printer continues to evolve (please!) it seems to be taking over most of the Numeric module. How about we make it a separate file? I've done the following in my Python: 1) Save his pretty printer as a separate python file ArrayFormatter.py 2) Add "import Numeric" to the top of ArrayFormatter.py file and prepend "Numeric." to the compress, maximum, and minimum calls. 3) Put the following near the end of Numeric.py: from ArrayFormatter import arrayToString # Set the official array printing function set_print_function(arrayToString) Seems like a nice division. -Perry ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From fredrik_lundh@ivab.se Thu Feb 8 08:04:22 1996 From: fredrik_lundh@ivab.se (Fredrik Lundh) Date: Thu, 8 Feb 1996 09:04:22 +0100 Subject: [PYTHON MATRIX-SIG] Naming Conventions And Practical Experience In-Reply-To: <199602072159.QAA25799@cyclone.ERE.UMontreal.CA> (hinsenk@ere.umontreal.ca) Message-ID: <9602080804.AA26470@arnold.image.ivab.se> > Rather than rush out a beta version to the public, I advocate a > public alpha release that comes with no promise about stability of > function names etc. We need all the test users we can get! Why make > any commitments without anybody forcing us? I support this view, if that's any help :-) /F BTW: what about work on standard packages?; I'd very much like some linear algebra stuff, for example to invert matrices (but don't know enough about it to contribute). In addition, I'd would like to use the matrix extension to do some of the serious processing stuff in my forthcoming Python Imaging library, like FFTs and 2D convolutions. Any ideas on how to achieve this? ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From fredrik_lundh@ivab.se Thu Feb 8 08:04:22 1996 From: fredrik_lundh@ivab.se (Fredrik Lundh) Date: Thu, 8 Feb 1996 09:04:22 +0100 Subject: [PYTHON MATRIX-SIG] Naming Conventions And Practical Experience In-Reply-To: <199602072159.QAA25799@cyclone.ERE.UMontreal.CA> (hinsenk@ere.umontreal.ca) Message-ID: <9602080804.AA26470@arnold.image.ivab.se> > Rather than rush out a beta version to the public, I advocate a > public alpha release that comes with no promise about stability of > function names etc. We need all the test users we can get! Why make > any commitments without anybody forcing us? I support this view, if that's any help :-) /F BTW: what about work on standard packages?; I'd very much like some linear algebra stuff, for example to invert matrices (but don't know enough about it to contribute). In addition, I'd would like to use the matrix extension to do some of the serious processing stuff in my forthcoming Python Imaging library, like FFTs and 2D convolutions. Any ideas on how to achieve this? ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From stoll@atr-sw.atr.co.jp Thu Feb 8 09:22:12 1996 From: stoll@atr-sw.atr.co.jp (Perry A. Stoll) Date: Thu, 08 Feb 1996 18:22:12 +0900 Subject: [PYTHON MATRIX-SIG] Naming Conventions And Practical Experience In-Reply-To: Your message of "Thu, 08 Feb 1996 09:04:22 JST" References: <9602080804.AA26470@arnold.image.ivab.se> Message-ID: <199602080922.SAA04171@ciris21.atr-sw.atr.co.jp> >>>>> "FL" == Fredrik Lundh writes: FL> would like to use the matrix extension to do some of the FL> serious processing stuff in my forthcoming Python Imaging FL> library, like FFTs and 2D convolutions. Any ideas on how to FL> achieve this? I'm working on something similar. I've got various FFT packages off the net - I'll try to interface them, but I don't have time right now. If you are interested in doing it, let me know. In the mean time, for those wanting to play take a look at the file attached at the end. It's not the stuff legends are made of, but it should work. -Perry --------------------- cut here ------------------------------------ """ conv2 - support convolution of a 2 dimensional matrix by a 2 dimensional kernel. This routine was inspired by the matlab(c) version of conv2. Sample usage: >>> import conv2 # a sobel edge finding kernel >>> sobel = Numeric.array([[1.0 ,2, 1],[0, 0, 0],[-1, -2, -1]]) # a test array >>> A = Numeric.zeros(10,10) >>> S = Numeric.Slice(2,8) >>> A[(S,S)] = 1 # do the convolution >>> conv2.conv2(A,sobel) """ # Two dimension convolution for the Python Numeric package. # Should be redone in C. # # questions to: # Perry A. Stoll # or import Numeric,umath class AddSlice(Numeric.Slice): def __init__(self, start=0, stop=None, step=1): Numeric.Slice.__init__(self,start,stop, step) def __add__(self,x): newslice = AddSlice(self.start,self.stop,self.step) newslice.start = newslice.start + x if newslice.stop != None: newslice.stop = newslice.stop + x return newslice __radd__ = __add__ class IncSlice(Numeric.Slice): def __init__(self, start=0, stop=None, step=1): Numeric.Slice.__init__(self,start,stop,step) def inc(self): self.start = self.start + 1 if self.stop != None: self.stop = self.stop + 1 def conv2(arg1,arg2,shape='full'): """ C = conv2(A, B) performs the 2-D convolution of matrices A and B. If [ya,xa] == A.shape and [yb,xb] == B.shape, then C.shape == [ya+yb-1,xa+xb-1]. """ if (len(arg1.shape) != 2) or (len(arg2.shape) !=2): raise TypeError, "both arguments must be 2 dimensional" # we always want to use the small matrix as arg2, # so swap args if necessary swapped = 0 Nmr = Numeric.multiply.reduce size1,size2 = Nmr(arg1.shape),Nmr(arg2.shape) if (size2 > size1): swapped = 1 arg1,arg2 = arg2,arg1 # unpack shape tuples (y1,x1),(y2,x2) = arg1.shape,arg2.shape # create output matrix out = Numeric.zeros(y2+y1-1,x2+x1-1) cols,rows = IncSlice(0,x2),IncSlice(0,y2) for j in range(0,y1): rows.start,rows.stop = 0,y2 for i in range(0,x1): w = arg1[j,i] if w != 0: out[cols,rows] = out[cols,rows] + w*arg2 rows.inc() cols.inc() # not sure we need to do this... #if swapped: #arg1,arg2 = arg2,arg1 if (shape == 'full'): pass elif (shape == 'same'): rows = int(umath.floor(x2/2.0)) + AddSlice(0,x1) cols = int(umath.floor(y2/2.0)) + AddSlice(0,y1) out = out[cols,rows] elif (shape == 'valid'): rows = x2 -1 + AddSlice(0,x1-x2 + 1) cols = y2 -1 + AddSlice(0,y1-y2 + 1) out = out[cols,rows] return out ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From mclay@eeel.nist.gov Thu Feb 8 14:30:23 1996 From: mclay@eeel.nist.gov (Michael McLay) Date: Thu, 8 Feb 96 09:30:23 EST Subject: [PYTHON MATRIX-SIG] Type Names In-Reply-To: <31191D29.D42@llnl.gov> Message-ID: <9602081430.AA17418@acdc.eeel.nist.gov> Paul. Dubois writes: > I hold these truths to be self-evident, your argument notwithstanding: > a. A PyFloat is a C double > The fact that some of the Python API lets you convert one to a C > float easily doesn't change this. > > b. It is a GOOD THING that you can count on the precision of a PyFloat. > Writing portable code between platforms is easy with Python, since > even on 64 bit machines C doubles are implemented as 64 bit floats. (I > know that isn't for certain but as a practical truth, it is true). The idea of naming the sizes of numbers "double" and "float" in a programming language is analogous to continuing the use the U.S. units of measure. If you're familiar with the units it's not a problem, but consider the task of introducing the convention to someone new to programming. Why does the naming convention have to be obscure? To get a sense of the nonsense of this naming convention, answer the following question. How many gils are in a gallon? (The first questions you need to ask of course is whether I mean an Imperial gallon or a U.S. gallon.) What do you think of the idea of explicitly naming the numeric types to reflect the actual machine size of the number? This would make the size of an object perfectly clear. It would also eliminate all future problems with portability of numbers between machine architectures. It won't break existing code since the old names can still be used as aliases for the new names. Here's all the names that would be needed. float32 float64 (alias is float) float128 int8 int16 int32 (alias for int) int64 int128 The advantages are: 1) It would make the size of a number obvious thereby eliminating one more source of stupid programming errors. 2) It would eliminate one more thing that must be learned and remembered by rote. 3) It will be easy for new users to grasp and remember the convention. Some of the disadvantages are: 1) It's something new. 2) It may require introducing some new keywords. (This probably isn't necessary.) 3) It isn't an "elegant" solution. 4) For individuals who already know what a float is, and who have a talent for remembering minute details, the change may seem gratuitous. While an "elegant", theoretical solution, such as Ada's ability to declare types for numbers of any size, may be possible, an implementation that allowed this would be slow for any size number other than those supported directly in hardware. (With the exception of float128, the type names listed above are close to being universally supported in modern hardware.) Consequently, these are probably the only numbers that will be of interest for number crunching applications. For problems where speed isn't as important and more control over the calculation is needed then the application could use an arbitrary length numbers class and add size constraints as needed. Isn't it time to drop a convention that was introduced when machine words varied considerably from system to system? Michael ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From jfulton@usgs.gov Thu Feb 8 14:40:39 1996 From: jfulton@usgs.gov (Jim Fulton) Date: Thu, 8 Feb 1996 09:40:39 -0500 Subject: [PYTHON MATRIX-SIG] Naming Conventions And Practical Experience In-Reply-To: Fredrik Lundh "Re: [PYTHON MATRIX-SIG] Naming Conventions And Practical Experience" (Feb 8, 9:04am) References: <9602080804.AA26470@arnold.image.ivab.se> Message-ID: <9602080940.ZM13456@dsdbqvarsa.er.usgs.gov> On Feb 8, 9:04am, Fredrik Lundh wrote: > Subject: Re: [PYTHON MATRIX-SIG] Naming Conventions And Practical Experien > > > Rather than rush out a beta version to the public, I advocate a > > public alpha release that comes with no promise about stability of > > function names etc. We need all the test users we can get! Why make > > any commitments without anybody forcing us? > > I support this view, if that's any help :-) Me too. Jim ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From hinsenk@ere.umontreal.ca Thu Feb 8 14:53:24 1996 From: hinsenk@ere.umontreal.ca (Konrad HINSEN) Date: Thu, 8 Feb 1996 09:53:24 -0500 Subject: [PYTHON MATRIX-SIG] Naming Conventions And Practical Experience In-Reply-To: <9602080804.AA26470@arnold.image.ivab.se> (message from Fredrik Lundh on Thu, 8 Feb 1996 09:04:22 +0100) Message-ID: <199602081453.JAA19532@cyclone.ERE.UMontreal.CA> BTW: what about work on standard packages?; I'd very much like some linear algebra stuff, for example to invert matrices (but don't know enough about it to contribute). In addition, I'd would like to use I agree that this would be useful, both directly and indirectly by providing a real-life example of a C or Fortran extension. Such extensions should also be considered in the big naming debate. A "everything in methods" strategy won't work, because a module can't define methods for objects implemented in another module. So all linear-algebra (or other application-specific) operations have to be implemented as functions. This suggests that built-in functions of the array module should also be functions unless there is some clear semantic difference. A difficult decision for linear algebra is which library to use. My experience is limited to Fortran libraries, of which I had the best results with LAPACK. There is a C version of LAPACK, which however I have never used. There are also a couple of LA libraries directly designed for C, rather than being translations from Fortran. It would certainly be easiest to use a C library, which eliminates all the Fortran interfacing machine dependencies and doesn't force users to have a Fortran compiler. On the other hand, Fortran libraries are better tested and on some systems (such as SGI) the performance difference is enormous. Any comments? Does anyone have experience with a C library for linear algebra? ------------------------------------------------------------------------------- Konrad Hinsen | E-Mail: hinsenk@ere.umontreal.ca Departement de chimie | Tel.: +1-514-343-6111 ext. 3953 Universite de Montreal | Fax: +1-514-343-7586 C.P. 6128, succ. Centre-Ville | Deutsch/Esperanto/English/Nederlands/ Montreal (QC) H3C 3J7 | Francais (phase experimentale) ------------------------------------------------------------------------------- ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From da@maigret.cog.brown.edu Thu Feb 8 15:31:38 1996 From: da@maigret.cog.brown.edu (David Ascher) Date: Thu, 8 Feb 1996 10:31:38 -0500 (EST) Subject: [PYTHON MATRIX-SIG] Naming Conventions And Practical Experience In-Reply-To: <199602080922.SAA04171@ciris21.atr-sw.atr.co.jp> from "Perry A. Stoll" at Feb 8, 96 06:22:12 pm Message-ID: <199602081531.KAA20717@maigret> > I'm working on something similar. I've got various FFT packages off > the net - I'll try to interface them, but I don't have time right > now. If you are interested in doing it, let me know. Maybe we should start a subsig for linalg, one for fft/image processing? > In the mean time, for those wanting to play take a look at the file > attached at the end. It's not the stuff legends are made of, but it > should work. I did something similar in Python, and then redid it in C. The C version was orders of magnitude faster, not surprisingly. I don't know enough about such things to write really efficient code for this sort of stuff, though. --david ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From fredrik_lundh@ivab.se Thu Feb 8 17:02:45 1996 From: fredrik_lundh@ivab.se (Fredrik Lundh) Date: Thu, 8 Feb 1996 18:02:45 +0100 Subject: [PYTHON MATRIX-SIG] Re: [PYTHON IMAGE-SIG???] In-Reply-To: <199602081526.KAA20669@maigret> (da@maigret.cog.brown.edu) Message-ID: <9602081702.AA32049@arnold.image.ivab.se> > Maybe we should start a subsig for linalg, one for fft/image > processing? Why not an Image SIG, targeted for graphics, publishing, GIS and good ole "scientific image processing" users... I've almost completed the first release of an image handling and processing library for Python. This release focuses on file formats and basic operations (see the attached "preannouncement"). The design is to some extent ready for more serious image processing (including support for 32-bit integer and floating point images), and I would very much like to interface (not merge) this with the matrix package to avoid duplicating stuff like FFTs etc. I've also got ok from my bosses to distribute the library for free use, and to spend some time on promoting it :-) What do you think? Regards/F ==================================================================== Fredrik Lundh | mail to fredrik_lundh@ivab.se IV Image Systems AB | or call +46 13 200100 Teknikringen 9 | or fax to +46 13 214897 583 30 LINKOPING SWEDEN | ==================================================================== -------------------------------------------------------- Preannouncement: The Python Imaging Library, Release 0.1 -------------------------------------------------------- The Python Imaging Library adds an image object to your Python interpreter. You can load image objects from a variety of file formats, and apply a rich set of image operations to them. Features: + Identify and extract image characteristics from a large set of image file formats. + Read and write PPM, BMP, GIF, and JPEG files. Supports progressive reading of all formats through ImageIO objects. + BW, Palette, RGB, RGBA and CMYK pixel formats. + Crop, cut/paste, convert between pixel formats. + Image geometry: resize, rotate, transpose, user defined affine transform. Nearest neighbour only. + Image enhancement: blur, contour, detail, edge enhance, emboss, find edges, smooth, sharpen, and user defined 3x3 and 5x5 integer kernels. User defined point transforms. + Image analysis: histogram, global statistics. ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From fredrik_lundh@ivab.se Thu Feb 8 17:02:45 1996 From: fredrik_lundh@ivab.se (Fredrik Lundh) Date: Thu, 8 Feb 1996 18:02:45 +0100 Subject: [PYTHON MATRIX-SIG] Re: [PYTHON IMAGE-SIG???] In-Reply-To: <199602081526.KAA20669@maigret> (da@maigret.cog.brown.edu) Message-ID: <9602081702.AA32049@arnold.image.ivab.se> > Maybe we should start a subsig for linalg, one for fft/image > processing? Why not an Image SIG, targeted for graphics, publishing, GIS and good ole "scientific image processing" users... I've almost completed the first release of an image handling and processing library for Python. This release focuses on file formats and basic operations (see the attached "preannouncement"). The design is to some extent ready for more serious image processing (including support for 32-bit integer and floating point images), and I would very much like to interface (not merge) this with the matrix package to avoid duplicating stuff like FFTs etc. I've also got ok from my bosses to distribute the library for free use, and to spend some time on promoting it :-) What do you think? Regards/F ==================================================================== Fredrik Lundh | mail to fredrik_lundh@ivab.se IV Image Systems AB | or call +46 13 200100 Teknikringen 9 | or fax to +46 13 214897 583 30 LINKOPING SWEDEN | ==================================================================== -------------------------------------------------------- Preannouncement: The Python Imaging Library, Release 0.1 -------------------------------------------------------- The Python Imaging Library adds an image object to your Python interpreter. You can load image objects from a variety of file formats, and apply a rich set of image operations to them. Features: + Identify and extract image characteristics from a large set of image file formats. + Read and write PPM, BMP, GIF, and JPEG files. Supports progressive reading of all formats through ImageIO objects. + BW, Palette, RGB, RGBA and CMYK pixel formats. + Crop, cut/paste, convert between pixel formats. + Image geometry: resize, rotate, transpose, user defined affine transform. Nearest neighbour only. + Image enhancement: blur, contour, detail, edge enhance, emboss, find edges, smooth, sharpen, and user defined 3x3 and 5x5 integer kernels. User defined point transforms. + Image analysis: histogram, global statistics. ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From hinsenk@ere.umontreal.ca Thu Feb 8 17:33:45 1996 From: hinsenk@ere.umontreal.ca (Konrad HINSEN) Date: Thu, 8 Feb 1996 12:33:45 -0500 Subject: [PYTHON MATRIX-SIG] Type Names In-Reply-To: <9602081430.AA17418@acdc.eeel.nist.gov> (message from Michael McLay on Thu, 8 Feb 96 09:30:23 EST) Message-ID: <199602081733.MAA02062@cyclone.ERE.UMontreal.CA> of measure. If you're familiar with the units it's not a problem, but consider the task of introducing the convention to someone new to programming. Why does the naming convention have to be obscure? To Because we want newcomers to appreciate that we are doing something very complicated. Isn't that the origin of most technical terms? following question. How many gils are in a gallon? (The first questions you need to ask of course is whether I mean an Imperial gallon or a U.S. gallon.) Fortunately Canada has changed to the metric system long before I arrived here! What do you think of the idea of explicitly naming the numeric types to reflect the actual machine size of the number? This would make the size of an object perfectly clear. It would also eliminate all future I have nothing against this, but I wonder where these names would actually be used in Python. For the array constructors, I am very happy with the current system of "type constructors" as implemented in Precision.py. It would be nice to have the same for output instead of typecodes, of course, but we are not really adding all these things as new types to Python. While an "elegant", theoretical solution, such as Ada's ability to declare types for numbers of any size, may be possible, an implementation that allowed this would be slow for any size number other than those supported directly in hardware. (With the exception The current scheme in Precision.py of picking always the next larger size available in hardware seems like a good compromise to me. crunching applications. For problems where speed isn't as important and more control over the calculation is needed then the application could use an arbitrary length numbers class and add size constraints as needed. Arbitrary size floats would indeed be a nice addition to Python... ------------------------------------------------------------------------------- Konrad Hinsen | E-Mail: hinsenk@ere.umontreal.ca Departement de chimie | Tel.: +1-514-343-6111 ext. 3953 Universite de Montreal | Fax: +1-514-343-7586 C.P. 6128, succ. Centre-Ville | Deutsch/Esperanto/English/Nederlands/ Montreal (QC) H3C 3J7 | Francais (phase experimentale) ------------------------------------------------------------------------------- ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From dubois1@llnl.gov Thu Feb 8 17:56:50 1996 From: dubois1@llnl.gov (Paul. Dubois) Date: Thu, 08 Feb 1996 09:56:50 -0800 Subject: [PYTHON MATRIX-SIG] Re: ffts References: <9602081702.AA32049@arnold.image.ivab.se> Message-ID: <311A3962.5818@llnl.gov> We have an FFT package in Basis which in the course of time will be converted to a Python package. If anybody wants to jump the gun, get the Basis 11.2 distribution from ftp-icf.llnl.gov/pub/basis, and look in /dist/library/fft. Maybe a little FIDL will do it? -- Paul F. Dubois, L-472 (510)-422-5426 Lawrence Livermore National Laboratory FAX (510)-423-9969 Livermore, CA 94550 dubois1@llnl.gov Consulting: PaulDubois@aol.com ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From fredrik_lundh@ivab.se Thu Feb 8 22:15:43 1996 From: fredrik_lundh@ivab.se (Fredrik Lundh) Date: Thu, 8 Feb 1996 23:15:43 +0100 Subject: [PYTHON MATRIX-SIG] Naming Conventions And Practical Experience In-Reply-To: <199602081453.JAA19532@cyclone.ERE.UMontreal.CA> (hinsenk@ere.umontreal.ca) Message-ID: <9602082215.AA01433@arnold.image.ivab.se> Konrad wrote: > A "everything in methods" strategy won't work, because a module > can't define methods for objects implemented in another module. Try this: class foo: def spam(self): print "spam" bar = foo() def egg(self): print "egg" foo.egg = egg bar.spam() bar.egg() Yes, it's weird :-) /F ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From hinsenk@ere.umontreal.ca Thu Feb 8 22:30:50 1996 From: hinsenk@ere.umontreal.ca (Konrad HINSEN) Date: Thu, 8 Feb 1996 17:30:50 -0500 Subject: [PYTHON MATRIX-SIG] Naming Conventions And Practical Experience In-Reply-To: <9602082215.AA01433@arnold.image.ivab.se> (message from Fredrik Lundh on Thu, 8 Feb 1996 23:15:43 +0100) Message-ID: <199602082230.RAA22293@cyclone.ERE.UMontreal.CA> Konrad wrote: > A "everything in methods" strategy won't work, because a module > can't define methods for objects implemented in another module. Try this: class foo: def spam(self): print "spam" ... You are right, I should have specified "C module". For Python classes this is possible, although not a particularly good idea from the point of view of modularity. Anyway, arrays are implemented in C... A possible solution would be to require extensions to use a Python wrapper like UserArray and subclass it before adding new methods. In addition to the loss of efficiency that this causes, it would also make our system more complicated for the casual user. ------------------------------------------------------------------------------- Konrad Hinsen | E-Mail: hinsenk@ere.umontreal.ca Departement de chimie | Tel.: +1-514-343-6111 ext. 3953 Universite de Montreal | Fax: +1-514-343-7586 C.P. 6128, succ. Centre-Ville | Deutsch/Esperanto/English/Nederlands/ Montreal (QC) H3C 3J7 | Francais (phase experimentale) ------------------------------------------------------------------------------- ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From fredrik_lundh@ivab.se Thu Feb 8 22:41:09 1996 From: fredrik_lundh@ivab.se (Fredrik Lundh) Date: Thu, 8 Feb 1996 23:41:09 +0100 Subject: [PYTHON MATRIX-SIG] Naming Conventions And Practical Experience In-Reply-To: <199602082230.RAA22293@cyclone.ERE.UMontreal.CA> (hinsenk@ERE.UMontreal.CA) Message-ID: <9602082241.AA01587@arnold.image.ivab.se> > You are right, I should have specified "C module". For Python > classes this is possible, although not a particularly good idea from > the point of view of modularity. And I completely agree. Finding the perfect balance between methods and functions can sometimes be tricky, though. /F ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From hinsenk@ere.umontreal.ca Thu Feb 8 23:49:21 1996 From: hinsenk@ere.umontreal.ca (Konrad HINSEN) Date: Thu, 8 Feb 1996 18:49:21 -0500 Subject: [PYTHON MATRIX-SIG] Final (!?) pretty printer Message-ID: <199602082349.SAA27150@cyclone.ERE.UMontreal.CA> Here it is: the fully functional pretty printer for Python arrays. Unless there are bugs to be fixed or complaints or requests for reasonable new features, I'd like to leave it at this stage. Line wrapping is now implemented and respects the maximum line width whenever possible, but it still puts at least one element on each line. Note that continuation lines are indented by at least 6 characters in such a way as to make sure that their items do *not* align with the main lines. This makes visual parsing easier. And now, as you expected, the code: def arrayToString(a, max_line_width = None, precision = None): if not max_line_width: try: max_line_width = sys.output_line_width except AttributeError: max_line_width = 77 if not precision: try: precision = sys.float_output_precision except AttributeError: precision = 8 if a.contiguous(): data = a.reshape(None) else: data = a.copy().reshape(None) type = a.typecode() items_per_line = a.shape[-1] if type == 'b' or type == '1' or type == 's' or type == 'i' \ or type == 'l': max_str_len = max(len(str(maximum.reduce(data))), len(str(minimum.reduce(data)))) format = '%' + str(max_str_len) + 'd' item_length = max_str_len+1 format_function = lambda x, f = format: _formatInteger(x, f) elif type == 'f' or type == 'd': format, item_length = _floatFormat(data, precision) format_function = lambda x, f = format: _formatFloat(x, f) elif type == 'F' or type == 'D': real_format, real_item_length = _floatFormat(data.real, precision, sign=0) imag_format, imag_item_length = _floatFormat(data.imaginary, precision, sign=1) item_length = real_item_length + imag_item_length + 2 format_function = lambda x, f1 = real_format, f2 = imag_format: \ _formatComplex(x, f1, f2) elif type == 'c': format = '%s' item_length = 1 format_function = lambda x, f = format: _formatCharacter(x, f) else: # give up on 'O' return str(a) final_spaces = (type != 'c') line_width = item_length*items_per_line - final_spaces if line_width > max_line_width: indent = 6 if indent == item_length: indent = 8 items_first = (max_line_width+final_spaces)/item_length if items_first < 1: items_first = 1 items_continuation = (max_line_width+final_spaces-indent)/item_length if items_continuation < 1: items_continuation = 1 line_width = max(item_length*items_first, item_length*items_continuation+indent) - final_spaces number_of_lines = 1 + (items_per_line-items_first + items_continuation-1)/items_continuation line_format = (number_of_lines, items_first, items_continuation, indent, line_width) else: line_format = (1, items_per_line, 0, 0, line_width) return _arrayToString(a, format_function, len(a.shape), line_format)[:-1] def _floatFormat(data, precision, sign = 0): exp_format = 0 non_zero = abs(compress(data.notEqual(0), data)) if len(non_zero) == 0: max_val = 0. min_val = 0. else: max_val = maximum.reduce(non_zero) min_val = minimum.reduce(non_zero) if max_val >= 1.e12 or min_val < 0.0001 or max_val/min_val > 1000.: exp_format = 1 if exp_format: large_exponent = 0 < min_val < 1e-99 or max_val >= 1e100 max_str_len = 8 + precision + large_exponent if sign: format = '%+' else: format = '%' format = format + str(max_str_len) + '.' + str(precision) + 'e' if large_exponent: format = format + '3' item_length = max_str_len + 1 else: format = '%.' + str(precision) + 'f' precision = min(precision, apply(max, tuple(map(lambda x, p=precision, f=format: _digits(x,p,f), data)))) max_str_len = len(str(int(max_val))) + precision + 2 if sign: format = '%#+' else: format = '%#' format = format + str(max_str_len) + '.' + str(precision) + 'f' item_length = max_str_len + 1 return (format, item_length) def _digits(x, precision, format): s = format % x zeros = len(s) while s[zeros-1] == '0': zeros = zeros-1 return precision-len(s)+zeros def _arrayToString(a, format_function, rank, line_format): if rank == 0: return str(a[0]) elif rank == 1: s = '' items = line_format[1] indent = 0 index = 0 for j in range(line_format[0]): s = s + indent * ' ' for i in range(items): s = s + format_function(a[index]) index = index + 1 if index == a.shape[0]: break if s[-1] == ' ': s = s[:-1] s = s + '\n' items = line_format[2] indent = line_format[3] else: s = '' for i in range(a.shape[0]-1): s = s + _arrayToString(a[i], format_function, rank-1, line_format) if rank == 3: s = s + '\n' elif rank > 3: s = s + (rank-3)*(line_format[4]*'-'+'\n') s = s + _arrayToString(a[a.shape[0]-1], format_function, rank-1, line_format) return s def _formatInteger(x, format): return format % x + ' ' def _formatFloat(x, format, strip_zeros = 1): if format[-1] == '3': format = format[:-1] s = format % x third = s[-3] if third == '+' or third == '-': s = s[1:-2] + '0' + s[-2:] elif format[-1] == 'f': s = format % x if strip_zeros: zeros = len(s) while s[zeros-1] == '0': zeros = zeros-1 s = s[:zeros] + (len(s)-zeros)*' ' else: s = format % x return s + ' ' def _formatComplex(x, real_format, imag_format): r = _formatFloat(x.real, real_format)[:-1] i = _formatFloat(x.imag, imag_format, 0)[:-1] spaces = 0 while r[spaces] == ' ': spaces = spaces + 1 r = spaces*' ' + '(' + r[spaces:] if imag_format[-1] == 'f': zeros = len(i) while i[zeros-1] == '0': zeros = zeros-1 i = i[:zeros] + 'j)' + (len(i)-zeros)*' ' else: i = i + 'j)' return r + i + ' ' def _formatCharacter(x, format): return format % x ------------------------------------------------------------------------------- Konrad Hinsen | E-Mail: hinsenk@ere.umontreal.ca Departement de chimie | Tel.: +1-514-343-6111 ext. 3953 Universite de Montreal | Fax: +1-514-343-7586 C.P. 6128, succ. Centre-Ville | Deutsch/Esperanto/English/Nederlands/ Montreal (QC) H3C 3J7 | Francais (phase experimentale) ------------------------------------------------------------------------------- ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From et@appl-math.tu-muenchen.de Sat Feb 10 21:49:25 1996 From: et@appl-math.tu-muenchen.de (Thomas Schwaller) Date: Sat, 10 Feb 1996 22:49:25 +0100 Subject: [PYTHON MATRIX-SIG] plmodule, openglmodule and delaunaymodule (Version 0.33) Message-ID: <9602102249.ZM11999@hamster.appl-math.tu-muenchen.de> Hi all, just uploaded plmodule-0.33.c.gz openglmodule-0.33.c.gz and the experimental: delaunaymodule-0.33.c.gz for computing 3D Delaunay Triangulations. Publish or Perish! :-) Tom ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From hinsenk@ere.umontreal.ca Sun Feb 11 15:33:43 1996 From: hinsenk@ere.umontreal.ca (Konrad HINSEN) Date: Sun, 11 Feb 1996 10:33:43 -0500 Subject: [PYTHON MATRIX-SIG] Naming Conventions And Practical Experience In-Reply-To: <9602102037.ZM10034@hamster.appl-math.tu-muenchen.de> (et@appl-math.tu-muenchen.de) Message-ID: <199602111533.KAA15537@cyclone.ERE.UMontreal.CA> The Problem is mostly that all of these packages use their own matrix format Of course; in C it is not at all obvious how to represent a matrix... which is not that efficient. FORTRAN routines are somehow uniform in style for matrix stuff, but I would prefer a new clean module taking the code from there, not just interfacing it. But that means not profiting from the enormous work others have put into existing libraries. I'd much prefer using an existing library which has been tested on many machines and is constantly kept up to date with new machines, new compilers etc. We are much too small a group to do that ourselves, even we were determined to try. How to do a clean exception handling whitout knowing what a procedure does. Sometimes you just get crashes, which are unacceptable, I want an exception (not FORTRAN or other style coding.) I don't see that as a problem. LAPACK, for example, returns an error code indicating what is going wrong. This can be converted into an exception by the interface. Of course libraries that print error messages or even crash on an error should be avoided. The more I think about it, the more I tend to favour LAPACK. It has the advantage of being available in a C version as well as in Fortran, so those who could gain from it (like me ;-) could use the Fortran version, and yet everyone could use the module without needing a Fortran compiler. And, of course, it is a very reliable and complete library. >Why not an Image SIG, targeted for graphics, publishing, GIS and >good ole "scientific image processing" users... I suggest to do that later, at the moment I would like to see us writing code for the matrixmodule in one group. When we have some I agree! ------------------------------------------------------------------------------- Konrad Hinsen | E-Mail: hinsenk@ere.umontreal.ca Departement de chimie | Tel.: +1-514-343-6111 ext. 3953 Universite de Montreal | Fax: +1-514-343-7586 C.P. 6128, succ. Centre-Ville | Deutsch/Esperanto/English/Nederlands/ Montreal (QC) H3C 3J7 | Francais (phase experimentale) ------------------------------------------------------------------------------- ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From chris.chase@jhuapl.edu Mon Feb 12 14:48:00 1996 From: chris.chase@jhuapl.edu (Chris Chase S1A) Date: Mon, 12 Feb 1996 09:48:00 -0500 Subject: [PYTHON MATRIX-SIG] Type Coercion In-Reply-To: <9602071656.ZM12810@dsdbqvarsa.er.usgs.gov> References: <9602072018.AA17817@baribal.LCS.MIT.EDU.LCS.MIT.EDU> <31191D29.D42@llnl.gov> <9602071656.ZM12810@dsdbqvarsa.er.usgs.gov> Message-ID: <199602121438.JAA18583@python.org> Regarding the comments on type coercion. I agree with Paul's comments: it is good not to lose precision and to not get an exception. However, I would want the capability to turn on exceptions for those who want to identify unwanted coercions. How could this be made an option similar to capabilities of numerical packages that can turn on/off traps on floating point exceptions? I also agree with those who have voiced opinions to not to impose deadlines on the module release. I would prefer an alpha release with flexibility rather than a beta with fixed names/feature set. I do not think there is a rush on this. Besides, deadlines for this project can not supercede the main responsibilities that James and most of us have in our day jobs. Chris ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From hinsenk@ere.umontreal.ca Mon Feb 12 19:11:33 1996 From: hinsenk@ere.umontreal.ca (Konrad HINSEN) Date: Mon, 12 Feb 1996 14:11:33 -0500 Subject: [PYTHON MATRIX-SIG] Type Coercion In-Reply-To: <199602121438.JAA18583@python.org> (message from Chris Chase S1A on Mon, 12 Feb 1996 09:48:00 -0500) Message-ID: <199602121911.OAA10227@cyclone.ERE.UMontreal.CA> exception. However, I would want the capability to turn on exceptions for those who want to identify unwanted coercions. How could this be made an option similar to capabilities of numerical packages that can turn on/off traps on floating point exceptions? The problem gets a bit more complicated by the fact that you might want this check to be applied only to part of the code. For example, you might want to do "precision debugging" on your code while using a library that coerces happily between float and double. Such things are best left to a debugger, which already has provisions for doing things on specific code regions and under specific conditions. And Python fortunately has a debugger. For such an approach, the umath module should provide some hook to register a function that is called in case of a float->double coercion. This hook would then be used by an extended debugger. The main question is: who is willing to look at and modify the debugger? ------------------------------------------------------------------------------- Konrad Hinsen | E-Mail: hinsenk@ere.umontreal.ca Departement de chimie | Tel.: +1-514-343-6111 ext. 3953 Universite de Montreal | Fax: +1-514-343-7586 C.P. 6128, succ. Centre-Ville | Deutsch/Esperanto/English/Nederlands/ Montreal (QC) H3C 3J7 | Francais (phase experimentale) ------------------------------------------------------------------------------- ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From jjh@Goldilocks.LCS.MIT.EDU Mon Feb 12 20:08:07 1996 From: jjh@Goldilocks.LCS.MIT.EDU (James Hugunin) Date: Mon, 12 Feb 96 15:08:07 EST Subject: [PYTHON MATRIX-SIG] Release 0.34 now available Message-ID: <9602122008.AA27148@baribal.LCS.MIT.EDU.LCS.MIT.EDU> The major news for this release is that Chris Chase's grammar patches to implement multidimensional slices are included. This means that you can write: a[1:, :-1] and other exciting variations now work. The old keywords All, Reverse, and Slice no longer work in this new version. I should warn people that I rushed out this version because a few people were particularly eager to get there hands on it. I haven't tested this as well as I'd like to, so as usual use at your own risk. I hope to release something more stable by this Friday, so if you're not into the cutting edge... It's reasonably likely that the implementation details of multi-dimensional slices will change before they become an official patch to the python core. Nevertheless, it's very likely that the syntax/semantics of these slices will be exactly as they are in this distribution. Personally I find that my own code looks a lot cleaner without all the Slice(None, -1) stuff in it, thanks Chris! As usual, this release also includes many minor bug fixes. (It also includes a small performance optimization, but its unlikely you'll notice that). Also, as of this release I officially give in to the will of the sig on the type coercion issue. Paul DuBois's comment on the relative performance differences between floats and doubles was probably the straw that broke my camels back. floats are now automatically coerced to doubles, and all the other coercions that you'd expect. I really like Konrad's idea of adding some hooks into the debugger to detect automatic type coercions and I'm looking into that. Also, this release should compile on the Mac (thanks to Steve Spicklemire for making this happen). He's also got a version working under windows, but I didn't have the time to make the necessary changes before this release. Now if I can just get somebody to test this on a BeBox! Patches from 0.33 are in ftp://sls-ftp.lcs.mit.edu/pub/jjh/NumPy.patch-0.33-0.34.gz The whole thing is in ftp://sls-ftp.lcs.mit.edu/pub/jjh/NumericPython-0.34.tar.gz Enjoy - Jim ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From da@maigret.cog.brown.edu Tue Feb 13 03:39:01 1996 From: da@maigret.cog.brown.edu (David Ascher) Date: Mon, 12 Feb 1996 22:39:01 -0500 (EST) Subject: [PYTHON MATRIX-SIG] RubberIndex Message-ID: <199602130339.WAA02625@maigret> I'm still trying to figure out the ... index. If I understand correctly, it stands for "however many ':' I need depending on the rank of the object I'm indexing, so that the indices I *do* specify are at the end of the index list as opposed to the usual beginning. So, if I have a rank-3 array A, then A[...,0] is the same thing as A[:,:,0] but if B is rank-4, then B[...,0] is the same thing as: B[:,:,:,0] This "stretching" only works from the left: In other words, if '...' is used *after* a non-pseudo index, then it is equivalent to ':', and does not stretch. [should ... be allowed after a non-pseudo index?] If C is a rank-5 array, then C[...,0,...] is the same thing as C[...,0,:] and C[:,:,:,0,:] but NOT: C[:,:,0,:,:] This seems to work with my test cases. Am I missing some key functionality? --david ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From hinsenk@ere.umontreal.ca Tue Feb 13 13:47:08 1996 From: hinsenk@ere.umontreal.ca (Konrad HINSEN) Date: Tue, 13 Feb 1996 08:47:08 -0500 Subject: [PYTHON MATRIX-SIG] RubberIndex In-Reply-To: <199602130339.WAA02625@maigret> (da@maigret.cog.brown.edu) Message-ID: <199602131347.IAA19640@cyclone.ERE.UMontreal.CA> This "stretching" only works from the left: In other words, if '...' is used *after* a non-pseudo index, then it is equivalent to ':', and does not stretch. [should ... be allowed after a non-pseudo index?] '...' at the end of an index expression makes no sense, as you could just as well leave it out (a[0, ...] is the same as a[0]). Neither does it make sense to have more than one '...' in an index expression, since it stands for "all remaining axes", which you can use up only once. It would be good if this would lead to an exception. But it does make sense to have '...' between two groups of other indices, i.e. a[1, ..., 0] or a[2, ..., NewAxis]. ------------------------------------------------------------------------------- Konrad Hinsen | E-Mail: hinsenk@ere.umontreal.ca Departement de chimie | Tel.: +1-514-343-6111 ext. 3953 Universite de Montreal | Fax: +1-514-343-7586 C.P. 6128, succ. Centre-Ville | Deutsch/Esperanto/English/Nederlands/ Montreal (QC) H3C 3J7 | Francais (phase experimentale) ------------------------------------------------------------------------------- ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From chris.chase@jhuapl.edu Tue Feb 13 16:36:50 1996 From: chris.chase@jhuapl.edu (Chris Chase S1A) Date: Tue, 13 Feb 1996 11:36:50 -0500 Subject: [PYTHON MATRIX-SIG] RubberIndex In-Reply-To: <199602131347.IAA19640@cyclone.ERE.UMontreal.CA> References: <199602130339.WAA02625@maigret> <199602131347.IAA19640@cyclone.ERE.UMontreal.CA> Message-ID: <199602131627.LAA29926@python.org> The implementation should only allow a single '...'. If multiple rubber indexes are used then an exception should be raised. I could change Grammar so that multiple '...' causes a syntax error at compile time. Think of the rubber index as justifying/aligning the index list within the shape of the array. Indexes preceding '...' are left-justified and succeeding indexes are right-justified. Let P1,P2,..,PM be indexes preceding and S1,S2,..,SL be indexes succeeding '...' in an index list. If the shape is D1,D2,D3,..,DN-1,DN for a rank N array, P1,P2,..,PL are left-justified so they are associated with dimensions D1,D2,..,DL. S1,S2,..,SM are right-justified so SM -> DN, SM-1 -> DN-1, etc. Any dimension Di not associated with one of the actual indexes is assumed to be selected with a ":". The rubber index is analogous to negative limits for slices. Just as a negative slice limit (a[0:-2]) is relative to last element in the sequence without knowing the length apriori, the rubber index '...' allows one to specify indexes relative to the end of the shape without knowing the rank apriori. Any psuedo indexes (specified by NewAxis) are placed relative to the Pi, Si indexes next to it. Even when the psuedo index is next to a rubber index there is no ambiguity as it will be placed relative to the actual indexes to its left or right. Chris ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From drh@oasis.unl.edu Tue Feb 13 16:49:41 1996 From: drh@oasis.unl.edu (Doug Heisterkamp) Date: Tue, 13 Feb 1996 10:49:41 -0600 (CST) Subject: [PYTHON MATRIX-SIG] LAPACK module questions Message-ID: <199602131649.KAA29944@oasis.unl.edu> Hi, I have completed a first pass at a C module to access the LAPACK library. I have a few questions as I move on an try to refine it. But first, what I've done is to write a parser that reads the source code from CLAPACK and writes an access call for each function. The name and parameters are the same as in the library with the exception of using the array object for matrices and vectors. And now the questions. What is the proper way to access the data of an array object? I did not pay attention to the abstract object interface discussion on the main group and am not sure how to do it through that interface. Currently I'm just directly accessing it such as (for a double a from array b) a=(double *)((PyArrayObject *)PyArray_ContiguousFromObject(b,PyArray_DOUBLE, 1,0))->data; How should I package this C module with a python module? Should it become a subclass of Matrix or UserArray? Should it be a module with just a bunch of functions in it? Are there other people working on accessing math libraries? It would be nice if we could set up some conventions on function names, calling parameters, and exceptions for some of the common items. That way code that needs, say eigenvalues, could import whichever math library that is on the machine and still use a standard call to get the eigenvectors and values of a matrix. I'm planning on writing a python wrapper for eigenvectors, determinants, and svd. What other functions are commonly used and would be nice to have a clean interface to (as compared to the fortran interface of the lapack library)? If people are interested in the module, I could place it on an ftp server somewhere. It currently uses libraries built from CLapack which are just the f2c conversion of the original fortran source. Elf binaries of the lapack and blas libraries are available at the linux sites (if you need the directory, email me. I have the location written down elsewhere). The source for lapack, clapack, and f2c are available from netlib.att.com. Doug Heisterkamp drh@oasis.unl.edu ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From hinsenk@ere.umontreal.ca Tue Feb 13 16:48:04 1996 From: hinsenk@ere.umontreal.ca (Konrad HINSEN) Date: Tue, 13 Feb 1996 11:48:04 -0500 Subject: [PYTHON MATRIX-SIG] RubberIndex In-Reply-To: <199602131627.LAA29926@python.org> (message from Chris Chase S1A on Tue, 13 Feb 1996 11:36:50 -0500) Message-ID: <199602131648.LAA28672@cyclone.ERE.UMontreal.CA> The implementation should only allow a single '...'. If multiple rubber indexes are used then an exception should be raised. I could change Grammar so that multiple '...' causes a syntax error at compile time. That is not a good idea, as someone might want to use this syntax with a somewhat different meaning for other sequence objects. It would be better to have an exception raised at runtime. ------------------------------------------------------------------------------- Konrad Hinsen | E-Mail: hinsenk@ere.umontreal.ca Departement de chimie | Tel.: +1-514-343-6111 ext. 3953 Universite de Montreal | Fax: +1-514-343-7586 C.P. 6128, succ. Centre-Ville | Deutsch/Esperanto/English/Nederlands/ Montreal (QC) H3C 3J7 | Francais (phase experimentale) ------------------------------------------------------------------------------- ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From chris.chase@jhuapl.edu Tue Feb 13 17:14:21 1996 From: chris.chase@jhuapl.edu (Chris Chase S1A) Date: Tue, 13 Feb 1996 12:14:21 -0500 Subject: [PYTHON MATRIX-SIG] RubberIndex In-Reply-To: <199602131347.IAA19640@cyclone.ERE.UMontreal.CA> References: <199602130339.WAA02625@maigret> <199602131347.IAA19640@cyclone.ERE.UMontreal.CA> Message-ID: <199602131704.MAA30240@python.org> >>>>> "Konrad" == Konrad HINSEN writes: Konrad> '...' at the end of an index expression makes no sense, as you could Konrad> just as well leave it out (a[0, ...] is the same as a[0]). It makes perfect sense. Just because there is another way to accomplish the same thing does not make it bad. In fact, I would prefer to eliminate a[0] in preference of a[0,...]. I do not like the fact that an error is not generated if I use to few indexes. For example, suppose 'a' has shape (2,4,5,6). I want the first element but either I forget that it is rank 4 or incorrectly enter the index: scalar = a[0,0,0] Instead of a scalar, I get an array with shape (6,). This "feature" requires that I check the rank of every parameter passed into a function for the desired rank. If I really want the last dimension as a vector I could specify vector = a[0,0,0,...] vector = a[0,0,0,:] I would prefer an error be produced by a[0,0,0]. An error already occurs when specifying too many indexes, i.e. a[0,0,0,0,0]. The same should occur for too few indexes unless a rubber index "..." is present (although even with a rubber index too indexes is an error). When using IDL, I have found that an error for too few indexes to be very useful in quickly finding mistakes in code. Without this error, I might not find out about the bug until much later in my code and then I might have a difficult time tracing it back to its origin. The only purpose I see for a[0] to be valid is to make it look like a list for indexing so that a[0][1] is valid. I suppose that if we want an array to be a specialized or subclassed list (so that it works wherever a list occurs) then this has to be supported. But if we think of an array as more of a dictionary than a list it should not support this kind of indexing. I could see that it could lead to confusion when we have multi-dimensional arrays of other types of python objects. Suppse 'a' is a rank 2 array containing a subscriptable python object. a[0,0][1] and a[0][0][1] would be valid. However, if 'a' had a rank different than 2 then we could have it produce an error - alerting the user that the object is not what he thought. Chris ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From jjh@Goldilocks.LCS.MIT.EDU Tue Feb 13 18:32:24 1996 From: jjh@Goldilocks.LCS.MIT.EDU (James Hugunin) Date: Tue, 13 Feb 96 13:32:24 EST Subject: [PYTHON MATRIX-SIG] LAPACK module questions In-Reply-To: <199602131649.KAA29944@oasis.unl.edu> (drh@oasis.unl.edu) Message-ID: <9602131832.AA12640@baribal.LCS.MIT.EDU.LCS.MIT.EDU> I'm probably the only one here who can answer these questions (yes I know that documentation for the C API would be nice). First a question. Are you building this on top of Guido's bgen tool? I really think that this is the right way to go. Hopefully bgen can be eventually turned into a nice generic tool for specifying interfaces to any C/FORTRAN library. It already handles a large collection of interface specifications. It has the very nice modular design of one automatic part that tries to extract the interface definition which produces interface definition files that can be easily edited by the user for the cases where it guesses wrong. Adding array objects to bgen would not be a whole lot of work (so many things I'd like to do if I just had a free 10 hours or so). I think that creating a raw C level binding to these libraries is the right thing to do, and then to write a python module that provides a nice friendly interface on top of the raw functions created in the module. The basic point here is that this should be doable without writing any C code. I'd be interested in seeing a copy of what you've done so far. Arrays are not part of the abstract object interface. The right way to write a function that takes in a single argument and passes it on to a C function expecting a 1d array of doubles and returning a 1d array of doubles is as follows: static PyObject *function(PyObject *ignored, PyObject *args) { PyArrayObject *ap, *apr; PyObject *op; int n; TRY(PyArg_ParseTuple(args, "O", &op)); TRY(ap = (PyArrayObject *)PyArray_ContiguousFromObject(op, PyArray_DOUBLE, 1, 1)); n = ap->dimensions[0]; TRY(apr = (PyArrayObject *)PyArray_FromDims(1, &n, PyArray_DOUBLE)); c_function((double *)ap->data, (double *)apr->data, n) Py_DECREF(ap); /* Needed because PyArray_ContiguousFromObject INCREFs it. */ return (PyObject *)apr; } In fact, if there is an error in allocating the return array this will leak memory, but I figure anytime you're getting exceptions during array allocation, enough things are wrong with your system that a little memory leak won't be a problem. Hopefully from the general complexity of this you can see the value in a bgen style approach. -Jim ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From dubois@kristen.llnl.gov Tue Feb 13 20:15:00 1996 From: dubois@kristen.llnl.gov (P. Dubois) Date: Tue, 13 Feb 1996 12:15:00 -0800 Subject: [PYTHON MATRIX-SIG] Naming in arrayobject.c Message-ID: <9602132015.AA25919@kristen.llnl.gov> SIG'ers, Here are some thoughts about the names in arrayobject.c; I am now leaving town until the 20th so that I don't have to listen to the resulting howls of outrage (:->. My philosophy is close to that used in the ISE Eiffel libraries and as described in the book Reusable Software by Meyer. 1. These are ok as is: transpose (ok, imperative verb for something that modifies self) cast (ok, ditto) typecode (ok, noun = attribute) itemsize (ok, noun = attribute) toString toFile choose (this is so weird a function anyway ...) 2. These I think need changing: Current -> Proposed Reason ------------------------------------ byteswap -> byteswapped (returns result, shouldn't be an imperative verb) reshape -> shaped (returns result, shouldn't be an imperative verb) If worried about typos because shape and shaped too similar, consider shaped_as. But not reshape, which definitely sounds like self is being modified. isnonzero -> nonzero_indices (sounds too much like test of not all zeros) This returns a list of indices where x is not zero. (Note: maybe a new function x.is_zero() would be useful to mean true => all elements are 0) copy -> clone copy is an ambiguous word as to whether one is the copier or the copyee. But clone definitely carries more of a feeling that this is going to produce a copy. In Eiffel, x.copy(y) means set x's attributes from y, for example. concat -> append (philosophically equivalent to list.append) If you don't like append, then change it to concatenate; abbreviations are not worth the trouble they cause. contiguous -> is_contiguous This is a query, not a command to make self contiguous 3. These are not clear to me take -> gather m.take(i_list) returns [m[i_list[0]], m[i_list[1]], ...] Gather is a more traditional name but I won't argue this too hard. conjugate -> conjugated Ambiguous as to whether it modifies self. But, you could make the case "conjugate" is a noun, not a verb in which case it is ok. ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From hinsenk@ere.umontreal.ca Tue Feb 13 20:16:05 1996 From: hinsenk@ere.umontreal.ca (Konrad HINSEN) Date: Tue, 13 Feb 1996 15:16:05 -0500 Subject: [PYTHON MATRIX-SIG] LAPACK module questions In-Reply-To: <199602131649.KAA29944@oasis.unl.edu> (drh@oasis.unl.edu) Message-ID: <199602132016.PAA11799@cyclone.ERE.UMontreal.CA> I have completed a first pass at a C module to access the LAPACK library. I Great! have a few questions as I move on an try to refine it. But first, what I've done is to write a parser that reads the source code from CLAPACK and writes an access call for each function. The name and parameters are the same as in the library with the exception of using the array object for matrices and vectors. Not bad for a low-level access module; some people are familiar with these names and will probably want to use them. I just wonder what you do with the routines for special storage methods (banded, packed symmetric, etc.); they don't make any sense in Python. How should I package this C module with a python module? Should it become a subclass of Matrix or UserArray? Should it be a module with just a bunch of functions in it? I'd say the latter. I have thought a bit about how the current array operations and extensions such as linear algebra can best be combined and used with a consistent access scheme. I have come to the conclusion that the best solution is to have only functions in various modules for all mathematical and structural operations, and use methods only for special stuff like typecode inquiries. This proposal is certainly somewhat surprising for true believers in object-oriented programming, but I'll argue that it is still the best compromise. The alternative in a true object-oriented spirit would be to implement most (if not all) operations as methods on array objects. For extensions like linear algebra, one would define a subclass of arrays with added methods. Someone who wants to use several such extensions, e.g. linear algebra and fft, would then create another class inheriting both from linear-algebra-arrays and fft-arrays. There are three problems with this approach, one specific to Python, and two generic. The Python-specific problem is that one cannot inherit from C types, and that C types cannot inherit from anything. That would make a Python wrapper for all array classes obligatory, with a resulting performance penalty (mostly noticeable for small arrays). One of the generic problems (at least in languages without type checking) is that mixing different array classes causes a lot of headaches. If some library returns an fft-array and someone want to use it as a linear-algebra-array, then it has to be converted first. This would be worst for the standard constructors, like zeros(), whose results could not be used in any derived module. The other generic problem is caused by the relatively many operations that depend symmetrically on more than one object. It doesn't really make sense to consider an addition as an operation on the first number with the second as parameter, and this is basically what causes all the coercion problems for binary operators that Python suffers from. I am not saying that a more sensible approach is impossible (one could for example define addition as an operation on a tuple of numbers), but I am not aware of any language that supports this in a clean way. The best way out in Python is, in my opinion, not to implement any operation as a method. That leads to a clean system without confusion, and Python's module system takes care of name space pollution. It may be old-fashioned, but it works. Are there other people working on accessing math libraries? It would be nice if we could set up some conventions on function names, calling parameters, and exceptions for some of the common items. That way code that needs, say Definitely. Still all I personally need is LAPACK... I'm planning on writing a python wrapper for eigenvectors, determinants, and svd. What other functions are commonly used and would be nice to have a clean interface to (as compared to the fortran interface of the lapack library)? Inverse, generalized inverse, and solution of linear equations. As a second step, generalized eigenvalue problems and linear equations with iterative refinement. If people are interested in the module, I could place it on an ftp server somewhere. It currently uses libraries built from CLapack which are just Yes, please! the f2c conversion of the original fortran source. Elf binaries of the Great, so one should be able to substitute the Fortran library easily. ------------------------------------------------------------------------------- Konrad Hinsen | E-Mail: hinsenk@ere.umontreal.ca Departement de chimie | Tel.: +1-514-343-6111 ext. 3953 Universite de Montreal | Fax: +1-514-343-7586 C.P. 6128, succ. Centre-Ville | Deutsch/Esperanto/English/Nederlands/ Montreal (QC) H3C 3J7 | Francais (phase experimentale) ------------------------------------------------------------------------------- ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From hinsenk@ere.umontreal.ca Tue Feb 13 20:22:43 1996 From: hinsenk@ere.umontreal.ca (Konrad HINSEN) Date: Tue, 13 Feb 1996 15:22:43 -0500 Subject: [PYTHON MATRIX-SIG] RubberIndex In-Reply-To: <199602131702.MAA00480@cyclone.ERE.UMontreal.CA> (message from Chris Chase S1A on Tue, 13 Feb 1996 12:14:21 -0500) Message-ID: <199602132022.PAA12218@cyclone.ERE.UMontreal.CA> In fact, I would prefer to eliminate a[0] in preference of a[0,...]. I do not like the fact that an error is not generated if I use to few indexes. For example, suppose 'a' has shape (2,4,5,6). I want the You certainly have a point, but it also makes sense to keep the usability of arrays as sequence objects. For example, you can now iterate over the first axis of an array with for x in array([[1,2],[3,4]]): print x*x Also, most array functions accept nested lists as well as arrays. I'd like to keep this analogy as far as possible. ------------------------------------------------------------------------------- Konrad Hinsen | E-Mail: hinsenk@ere.umontreal.ca Departement de chimie | Tel.: +1-514-343-6111 ext. 3953 Universite de Montreal | Fax: +1-514-343-7586 C.P. 6128, succ. Centre-Ville | Deutsch/Esperanto/English/Nederlands/ Montreal (QC) H3C 3J7 | Francais (phase experimentale) ------------------------------------------------------------------------------- ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From da@maigret.cog.brown.edu Tue Feb 13 20:54:00 1996 From: da@maigret.cog.brown.edu (David Ascher) Date: Tue, 13 Feb 1996 15:54:00 -0500 (EST) Subject: [PYTHON MATRIX-SIG] Naming in arrayobject.c In-Reply-To: <9602132015.AA25919@kristen.llnl.gov> from "P. Dubois" at Feb 13, 96 12:15:00 pm Message-ID: <199602132054.PAA05052@maigret> Mostly, I think Paul's suggestions are clear and good. > copy -> clone > copy is an ambiguous word as to whether one is the copier or the > copyee. But clone definitely carries more of a feeling that this > is going to produce a copy. > In Eiffel, x.copy(y) means set x's attributes from y, for example. This is a little odd since python uses copy generally, I think. Clone is certainly less ambiguous. > 3. These are not clear to me > > take -> gather > m.take(i_list) returns [m[i_list[0]], m[i_list[1]], ...] > Gather is a more traditional name but I won't argue this too hard. Neither is very intuitive, but since the function isn't either... > conjugate -> conjugated > Ambiguous as to whether it modifies self. > But, you could make the case "conjugate" is a noun, not a verb > in which case it is ok. Not really, since my first read was that it was a verb. It's not what was intended which should matter, but what is understood. I'll ask my students tonight what they think of these, and see if I come up with any interesting feedback. --david ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From drh@oasis.unl.edu Tue Feb 13 21:40:07 1996 From: drh@oasis.unl.edu (Doug Heisterkamp) Date: Tue, 13 Feb 1996 15:40:07 -0600 (CST) Subject: [PYTHON MATRIX-SIG] LAPACK module questions In-Reply-To: <9602131832.AA12640@baribal.LCS.MIT.EDU.LCS.MIT.EDU> from "James Hugunin" at Feb 13, 96 01:32:24 pm Message-ID: <199602132140.PAA00422@oasis.unl.edu> '' > > > I'm probably the only one here who can answer these questions (yes I > know that documentation for the C API would be nice). > > First a question. Are you building this on top of Guido's bgen tool? > I really think that this is the right way to go. Hopefully bgen can > be eventually turned into a nice generic tool for specifying > interfaces to any C/FORTRAN library. It already handles a large > collection of interface specifications. It has the very nice modular > design of one automatic part that tries to extract the interface > definition which produces interface definition files that can be > easily edited by the user for the cases where it guesses wrong. > Adding array objects to bgen would not be a whole lot of work (so many > things I'd like to do if I just had a free 10 hours or so). Unfortunately, I did not use bgen. I'll take a look at to see what it can do. > > I think that creating a raw C level binding to these libraries is the > right thing to do, and then to write a python module that provides a > nice friendly interface on top of the raw functions created in the > module. The basic point here is that this should be doable without > writing any C code. > > I'd be interested in seeing a copy of what you've done so far. I'll place a copy on the ftp server here at unl. I'll make some of the changes from below before placing it there. I'll let the list know when it's placed on the server and it actual location. Doug > > Arrays are not part of the abstract object interface. The right way > to write a function that takes in a single argument and passes it on > to a C function expecting a 1d array of doubles and returning a 1d > array of doubles is as follows: > > static PyObject *function(PyObject *ignored, PyObject *args) { > PyArrayObject *ap, *apr; > PyObject *op; > int n; > > TRY(PyArg_ParseTuple(args, "O", &op)); > TRY(ap = (PyArrayObject *)PyArray_ContiguousFromObject(op, PyArray_DOUBLE, 1, 1)); > > n = ap->dimensions[0]; > > TRY(apr = (PyArrayObject *)PyArray_FromDims(1, &n, PyArray_DOUBLE)); > > c_function((double *)ap->data, (double *)apr->data, n) > > Py_DECREF(ap); /* Needed because PyArray_ContiguousFromObject INCREFs it. */ > > return (PyObject *)apr; > } > > In fact, if there is an error in allocating the return array this will > leak memory, but I figure anytime you're getting exceptions during > array allocation, enough things are wrong with your system that a > little memory leak won't be a problem. > > Hopefully from the general complexity of this you can see the value in > a bgen style approach. > > -Jim > ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From drh@oasis.unl.edu Tue Feb 13 22:13:07 1996 From: drh@oasis.unl.edu (Doug Heisterkamp) Date: Tue, 13 Feb 1996 16:13:07 -0600 (CST) Subject: [PYTHON MATRIX-SIG] LAPACK module questions In-Reply-To: <3120DBFE.4C9E@llnl.gov> from "Paul. Dubois" at Feb 13, 96 10:44:14 am Message-ID: <199602132213.QAA00459@oasis.unl.edu> '' > > We'd be interested in hearing about your parser. We face many tasks like > this. Actually, parser may be a little to generous to describe it. I wrote it as a throwaway program target at f2c version of the Lapack source code. It scans for the first function definition in a file and grabs it. The parameter list is parsed and depending on the parameter type, different information is added to the end of a number of different lists. These lists are then used to generate the C function to access the original function. The first comment after the function definition is placed as the __doc__ attribute of the access function. Of course this is written in python. I made a number of assumption about the parameter list. The main one is that I assume that all parameters which are double or complex are actually pointers to arrays. This was needed since in the C version, both a double and an array of doubles is referred to as double *a, in the function definition. I will have to change the cases where this is the wrong assumption by hand. Until I make the change, the function could still be used, but it would be necessary to place the double in a python array object of a single element and then use this array object in the function call. Similarly, I made the assumption that all integer parameters are not arrays. My goal was to get close and then make changes as needed. Doug > -- > Paul F. Dubois, L-472 (510)-422-5426 > Lawrence Livermore National Laboratory FAX (510)-423-9969 > Livermore, CA 94550 dubois1@llnl.gov > Consulting: PaulDubois@aol.com > ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From drh@oasis.unl.edu Tue Feb 13 22:57:21 1996 From: drh@oasis.unl.edu (Doug Heisterkamp) Date: Tue, 13 Feb 1996 16:57:21 -0600 (CST) Subject: [PYTHON MATRIX-SIG] LAPACK module questions In-Reply-To: <199602132016.PAA11799@cyclone.ERE.UMontreal.CA> from "Konrad HINSEN" at Feb 13, 96 03:16:05 pm Message-ID: <199602132257.QAA00502@oasis.unl.edu> > > Not bad for a low-level access module; some people are familiar with > these names and will probably want to use them. I just wonder what you > do with the routines for special storage methods (banded, packed > symmetric, etc.); they don't make any sense in Python. I assume that the caller knows the structure of the storage methods and packs it into an python array object. I have not used these functions yet (I really just started to use the Lapack library), but they seem to want one or two dimensional arrays with specific information at certain locations. The low-level access module assumes this has already been done for the objects passed to it. A higher level access module would be useful in setting up these structures. > > How should I package this C module with a python module? Should it become > a subclass of Matrix or UserArray? Should it be a module with just a bunch > of functions in it? > > I'd say the latter. I have thought a bit about how the current array > operations and extensions such as linear algebra can best be combined > and used with a consistent access scheme. I have come to the conclusion > that the best solution is to have only functions in various modules > for all mathematical and structural operations, and use methods only > for special stuff like typecode inquiries. > > > The best way out in Python is, in my opinion, not to implement any > operation as a method. That leads to a clean system without confusion, > and Python's module system takes care of name space pollution. It may > be old-fashioned, but it works. I agree. > > Are there other people working on accessing math libraries? It would be nice > if we could set up some conventions on function names, calling parameters, > and exceptions for some of the common items. That way code that needs, say > > Definitely. Still all I personally need is LAPACK... > > I'm planning on writing a python wrapper for eigenvectors, determinants, > and svd. What other functions are commonly used and would be nice to > have a clean interface to (as compared to the fortran interface of the > lapack library)? > > Inverse, generalized inverse, and solution of linear equations. As a > second step, generalized eigenvalue problems and linear equations with > iterative refinement. What is a "generalized inverse"? If I remember correctly, the generalized eigenvalue problem is (A - lambda C)x = 0 where C is a matrix. But I don't remember running across a generalized inverse. > > If people are interested in the module, I could place it on an ftp server > somewhere. It currently uses libraries built from CLapack which are just > > Yes, please! > > the f2c conversion of the original fortran source. Elf binaries of the > > Great, so one should be able to substitute the Fortran library > easily. I hope so as the Fortran libraries should be faster. Doug ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From hinsenk@ere.umontreal.ca Wed Feb 14 13:45:10 1996 From: hinsenk@ere.umontreal.ca (Konrad HINSEN) Date: Wed, 14 Feb 1996 08:45:10 -0500 Subject: [PYTHON MATRIX-SIG] LAPACK module questions In-Reply-To: <199602132257.QAA00502@oasis.unl.edu> (drh@oasis.unl.edu) Message-ID: <199602141345.IAA19287@cyclone.ERE.UMontreal.CA> I assume that the caller knows the structure of the storage methods and packs it into an python array object. I have not used these functions Yes, that's a reasonable assumption. After all, it's low level. A good interface can always be added later by defining a Python class representing e.g. a packed symmetric matrix. What is a "generalized inverse"? If I remember correctly, the generalized eigenvalue problem is (A - lambda C)x = 0 where C is a matrix. But I don't remember running across a generalized inverse. It's also known as a pseudoinverse or Moore-Penrose-inverse. There is a uniquely defined generalized inverse for any matrix, and in the case of a square non-singular matrix it is equal to the ordinary inverse. Generalized inverses can be used to express the solution of over- or underdetermined (or both) systems of linear equations conveniently, and they are very useful in least-squares problems. The preferred way to calculate the generalized inverse uses SVD: take the original matrix, do SVD, invert all the non-zero singular values, multiply the three matrices again, and return the transpose. Trivial to code in Python based on low-level SVD calls. ------------------------------------------------------------------------------- Konrad Hinsen | E-Mail: hinsenk@ere.umontreal.ca Departement de chimie | Tel.: +1-514-343-6111 ext. 3953 Universite de Montreal | Fax: +1-514-343-7586 C.P. 6128, succ. Centre-Ville | Deutsch/Esperanto/English/Nederlands/ Montreal (QC) H3C 3J7 | Francais (phase experimentale) ------------------------------------------------------------------------------- ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From chris.chase@jhuapl.edu Wed Feb 14 16:56:02 1996 From: chris.chase@jhuapl.edu (Chris Chase S1A) Date: Wed, 14 Feb 1996 11:56:02 -0500 Subject: [PYTHON MATRIX-SIG] RubberIndex In-Reply-To: <199602132022.PAA12218@cyclone.ERE.UMontreal.CA> References: <199602131702.MAA00480@cyclone.ERE.UMontreal.CA> <199602132022.PAA12218@cyclone.ERE.UMontreal.CA> Message-ID: <199602141646.LAA05432@python.org> >>>>> "Konrad" == Konrad HINSEN writes: Konrad> In fact, I would prefer to eliminate a[0] in preference of Konrad> a[0,...]. I do not like the fact that an error is not Konrad> generated if I use to few indexes. For example, suppose Konrad> 'a' has shape (2,4,5,6). I want the Konrad> You certainly have a point, but it also makes sense to keep Konrad> the usability of arrays as sequence objects. For example, you Konrad> can now iterate over the first axis of an array with Konrad> for x in array([[1,2],[3,4]]): print x*x It would be better to have an iterator method that would allow one to access regular sub-arrays. At the minimum it would allow you to specify iteration along any chosen dimension (i.e. a.iterator(dim)). I am not sure how to extend it to allow iterating over other sub-arrays (e.g. 2x2 sub-matrices). Konrad> Also, most array functions accept nested lists as well as arrays. Konrad> I'd like to keep this analogy as far as possible. Generating an error when using too few indexes when subscripting an array does not prohibit using nested lists as arguments to array functions. Chris ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From hinsenk@ere.umontreal.ca Wed Feb 14 18:31:31 1996 From: hinsenk@ere.umontreal.ca (Konrad HINSEN) Date: Wed, 14 Feb 1996 13:31:31 -0500 Subject: [PYTHON MATRIX-SIG] RubberIndex In-Reply-To: <199602141643.LAA28371@cyclone.ERE.UMontreal.CA> (message from Chris Chase S1A on Wed, 14 Feb 1996 11:56:02 -0500) Message-ID: <199602141831.NAA04933@cyclone.ERE.UMontreal.CA> It would be better to have an iterator method that would allow one to access regular sub-arrays. At the minimum it would allow you to specify iteration along any chosen dimension (i.e. a.iterator(dim)). I am not sure how to extend it to allow iterating over other sub-arrays (e.g. 2x2 sub-matrices). That should certainly be available, but that was not my point... Konrad> Also, most array functions accept nested lists as well as arrays. Konrad> I'd like to keep this analogy as far as possible. Generating an error when using too few indexes when subscripting an array does not prohibit using nested lists as arguments to array functions. That wasn't my point either. My point is that since arrays and nested lists are treated as interchangeable in many situations, it makes sense to use this analogy as much as possible and allow an indexing scheme for arrays that matches that for nested arrays. WHich means that it should always be allowed to have just one index, independent of the rank of the array. ------------------------------------------------------------------------------- Konrad Hinsen | E-Mail: hinsenk@ere.umontreal.ca Departement de chimie | Tel.: +1-514-343-6111 ext. 3953 Universite de Montreal | Fax: +1-514-343-7586 C.P. 6128, succ. Centre-Ville | Deutsch/Esperanto/English/Nederlands/ Montreal (QC) H3C 3J7 | Francais (phase experimentale) ------------------------------------------------------------------------------- ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From da@maigret.cog.brown.edu Wed Feb 14 22:30:16 1996 From: da@maigret.cog.brown.edu (David Ascher) Date: Wed, 14 Feb 1996 17:30:16 -0500 (EST) Subject: [PYTHON MATRIX-SIG] RubberIndex In-Reply-To: <199602141831.NAA04933@cyclone.ERE.UMontreal.CA> from "Konrad HINSEN" at Feb 14, 96 01:31:31 pm Message-ID: <199602142230.RAA08591@maigret> > That wasn't my point either. My point is that since arrays and nested > lists are treated as interchangeable in many situations, it makes > sense to use this analogy as much as possible and allow an indexing > scheme for arrays that matches that for nested arrays. WHich means > that it should always be allowed to have just one index, independent > of the rank of the array. This is sort of irrelevant, but it has occured to me, now that I understand and really like [::-1], I was sort of hoping that I could do the same with lists. How easy would it be to extend that third argument to list slices? --david ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From et@appl-math.tu-muenchen.de Thu Feb 15 21:25:25 1996 From: et@appl-math.tu-muenchen.de (Thomas Schwaller) Date: Thu, 15 Feb 1996 22:25:25 +0100 Subject: [PYTHON MATRIX-SIG] LAPACK Message-ID: <9602152225.ZM29681@hamster.appl-math.tu-muenchen.de> I hope so as the Fortran libraries should be faster. I have the possiblity to test the FORTRAN libraries with a new commercial Fortran Compiler on Linux. Will see! BTW g77 is told to be faster than f2c on Linux e.g. Tom Eagerly waiting for the Lapack/Python module.... Sounds great. ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From stoll@atr-sw.atr.co.jp Fri Feb 16 07:55:06 1996 From: stoll@atr-sw.atr.co.jp (Perry A. Stoll) Date: Fri, 16 Feb 1996 16:55:06 +0900 Subject: [PYTHON MATRIX-SIG] changes to compress Message-ID: <199602160755.QAA18871@ciris21.atr-sw.atr.co.jp> Howdy All, Question: How is the function Numeric.compress supposed to behave if none of the elements of m meet the condition? def compress(condition, m, dimension=-1): """compress(condition, x, dimension=-1) = those elements of x corresponding to those elements of condition that are "true". condition must be the same size as the given dimension of x.""" Right now it raises an exception. I think it should return some NULL-type object, either None, [] or maybe an empty array object if such a thing exists. The problem stems from array methods not accepting empty lists as arguments. Couldn't it just return an empty Array object, which would act just like any other empty mapping or sequence object? For now, I've changed compress to: def compress(condition, m, dimension=-1): nonzero = condition.nonzero() if (nonzero): return m.take(nonzero, dimension) else: return [] I noticed this while using Konrad's pretty printer - did anyone else notice this? Konrad? -Perry ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From hinsenk@ere.umontreal.ca Fri Feb 16 14:23:01 1996 From: hinsenk@ere.umontreal.ca (Konrad HINSEN) Date: Fri, 16 Feb 1996 09:23:01 -0500 Subject: [PYTHON MATRIX-SIG] changes to compress In-Reply-To: <199602160755.QAA18871@ciris21.atr-sw.atr.co.jp> (stoll@atr-sw.atr.co.jp) Message-ID: <199602161423.JAA17508@cyclone.ERE.UMontreal.CA> Right now it raises an exception. I think it should return some NULL-type object, either None, [] or maybe an empty array object if such a thing exists. It should definitely return an empty array object. And all the array functions should be able to deal with empty arrays without raising exceptions. As any APL programmer will confirm, empty arrays occur very often in array manipulations. It is even necessary to keep the correct shapes for empty arrays, i.e. an array of shape (0,2) must be distinct from an array of shape (3,0,4), even though both contain no elements. Therefore a generic "empty" object like None can't be used as a substitute. The problem stems from array methods not accepting empty lists as arguments. Couldn't it just return an empty Array object, which would act just like any other empty mapping or sequence object? That's what it should do. I noticed this while using Konrad's pretty printer - did anyone else notice this? Konrad? Yes, I have even pointed it out in one of my messages about the pretty printer. ------------------------------------------------------------------------------- Konrad Hinsen | E-Mail: hinsenk@ere.umontreal.ca Departement de chimie | Tel.: +1-514-343-6111 ext. 3953 Universite de Montreal | Fax: +1-514-343-7586 C.P. 6128, succ. Centre-Ville | Deutsch/Esperanto/English/Nederlands/ Montreal (QC) H3C 3J7 | Francais (phase experimentale) ------------------------------------------------------------------------------- ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From drh@oasis.unl.edu Fri Feb 16 23:27:02 1996 From: drh@oasis.unl.edu (Doug Heisterkamp) Date: Fri, 16 Feb 1996 17:27:02 -0600 (CST) Subject: [PYTHON MATRIX-SIG] Python-LAPACK module is available Message-ID: <199602162327.RAA20173@oasis.unl.edu> For those who may wish to try my python-LAPACK module, it is now available (finally). The ftp location is: ftp.cs.unl.edu/pub/drh/python Two versions are available. One with and one without the __doc__ string attributes. The one without the __doc__ is about 1/4 the size of the other one. Let me know if it works for your machine. I have built it on an SGI and an Linux box. This the first version so please send me the bug reports so we can make a useful module out of it. Doug drh@oasis.unl.edu ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From et@appl-math.tu-muenchen.de Sun Feb 18 23:37:40 1996 From: et@appl-math.tu-muenchen.de (Thomas Schwaller) Date: Mon, 19 Feb 1996 00:37:40 +0100 Subject: [PYTHON MATRIX-SIG] trimodule.c Anybody interested? Message-ID: <9602190037.ZM5679@hamster.appl-math.tu-muenchen.de> I've written some triangulation modules for Python and promised some people to release them. Most of them use C++-class libraries and are difficult to handle (some compilable only with special compilers), so to make them public is probably not so wise. During the weekend I found some time to write a new one with some code I found usefull for it. I consists of only one module, adding tri trimodule.c to Setup is enough. It is based on a french FEM package and I started to write some documentation for it. It works in 2 dimensions only, but there it seems to be a fine and fast tool. (Handling of local refinements, holes, cracks, subdomains is possible, to name just a few possibilities) As I got no feedback for my delaunaymodule.c, I'll first ask this time. Is anybody interested in this module? If yes, I'll release it next week after writing some more stuff for the documentation which I wrangled out of the Source-Code (It is mostly in French, so I have to do that job for this module to be useful for others). And I'll have to test it a little bit more (perhaps adding some pl-module stuff displaying it or a Postscript output methods, which is not that difficult) Comments? Tom ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From da@maigret.cog.brown.edu Tue Feb 20 16:34:36 1996 From: da@maigret.cog.brown.edu (David Ascher) Date: Tue, 20 Feb 1996 11:34:36 -0500 (EST) Subject: [PYTHON MATRIX-SIG] returning scalars or rank-0 arrays.. Message-ID: <199602201634.LAA21501@maigret> I'm working on a 'literate' array class, which labels axes and indices. The point is that one can do slicing based on keyword references to the axes, and labels for index values. I'll of course make it available when it's ready for testing. One issue I've come accross is the "what do I return, a rank-0 entity or a scalar?". My understanding of the current scheme is that indexing returns scalars, while slicing returns ranked entities, even if that rank is 0. Is that correct? >>> a = arange(9).reshape(3,3) >>> b = UserArray(a) >>> b UserArray([[0,1,2],[3,4,5],[6,7,8]], 'l') >>> b[0] UserArray([0,1,2], 'l') >>> b[0,2] 2 >>> b[0,2:3] UserArray([2], 'l') If that's correct, then I guess that's what subclasses of UserArray should do, although it's not exactly intuitive. Also, should there be an _as_scalar method in UserArray which returns a scalar in the cases where self.array is rank-0? -david ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From hinsenk@ere.umontreal.ca Tue Feb 20 20:12:08 1996 From: hinsenk@ere.umontreal.ca (Konrad HINSEN) Date: Tue, 20 Feb 1996 15:12:08 -0500 Subject: [PYTHON MATRIX-SIG] Functions and names Message-ID: <199602202012.PAA17939@cyclone.ERE.UMontreal.CA> I have made an attempt to put some order into the array functions. I have made a list of functions that should be available and proposed a name for each function. Until now I have covered constructors and structural functions; the rest will follow later, assuming I am not hit by too many 16-ton weights before ;-) Lines that describe new or modified operations are marked with a '+'. Comments are at the end of the file. As I have suggested before, it is useful to have an additional module defining abbreviations and/or more meaningful names for often-used combinations. These should not be part of the standard module to reduce name space pollution and learning overhead, but they should nevertheless be part of the standard distribution to ensure uniformity. I'll include recommended abbreviations in the respective sections. Comments of any kind are of course welcome. Constructors ============ 1) Construct an array from an arbitrary sequence object or a function: + array(sequenceOrFunction, shape=None, type=None) Returns an array of the given shape and type using the elements from the flattened sequence. If not all elements of the sequence are used, the rest is discarded. If more elements are needed, the sequence is reused from the beginning. If a callable object is passed instead of a sequence, it is called for each element with its indices as parameters. Calling the current constructor oldArray(), the new one behaves like def array(sequenceOrFunction, shape=None, type=None): if isSequence(sequenceOrFunction): a = oldArray(sequence, type) if shape is not None: a = copyAndReshape(a, shape) return a else: return fromFunction(map(shape), sequenceOrFunction) This constructor replaces the current constructors array(), copyAndReshape(), and fromFunction(), and makes special cases like zeros() so easy that they don't have to be standard functions any more. 2) Construct a rank-1 array of equally spaced points: + arrayRange(x1, x2=None, x3=None) Currently: function arange() does exactly the same. Abbreviations: zeros(n1,n2,...) equals array(0,(n1,n2,...)) ones(n1,n2,...) equals array(1,(n1,n2,...)) unitMatrix(n) equals array([1]+n*[0], (n,n)) arange equals arrayRange Structural array operations =========================== 1) Selecting subarrays 1.1) Selecting on a "raster" along each coordinate: Done by indexing with integers and slices 1.2) Selecting arbitrary elements specified by an array of indices i: + take(a, i, axis=0) Conditions: i must be of an integer array type, minimum.reduce(ravel(i)) >= 0, maximum.reduce(ravel(i)) < a.shape[axis] Currently: method a.take(i, axis=0), i can be of any type, but must have rank 1. 1.3) Selecting the diagonal, i.e. those values where the indices along two axes are equal (see comment 1): + diagonal(a, axis1=0, axis2=1) Conditions: axis1 < len(a.shape), axis2 < len(a.shape), a.shape[axis1] == a.shape[axis2] 2) Rearranging array elements 2.1) Changing the shape 2.1.1) General reshaping: + reshape(a,shape) shape can be of any non-nested sequence type Condition: multiply.reduce(shape) == multiply.reduce(a.shape) Currently: method a.reshape(shape), shape must be a tuple 2.1.2) Reshaping to a rank-1 array: ravel(a) equivalent to reshape(a,(multiply.reduce(a.shape),)) 2.1.3) Combining a group of axes into one axis (see comment 2): + a[i1, i2, ......, i3, i4] (see comment 3) The double ellipsis works similar to the single one, but contracts all the axes covered into a single axis. 2.1.4) Adding an axis of length 1: a[..., NewAxis, ...] 2.2) Transposition: + transpose(a, axes) axes is a non-nested sequence of non-negative integers with maximum.reduce(axes) < len(a.shape) Currently: method a.transpose(axes) does the same, but there is a bug that sometimes gives wrong results if an axis is used more than once in the list. It also insists that len(axes) == len(a.shape), although there is no need for this restriction. 3) Replicating and combining array elements 3.1) Replicating elements along an axis: + repeat(a, n, axis=0) n is a non-nested integer sequence with len(n) == a.shape[axis] Each element in a is repeated as often as indicated by the corresponding element in n. The length of the result along the specified axis is add.reduce(n). Currently: function compress(n, a, axis=0) is a special case limited to boolean (0/1) elements in n. 3.2) Concatenation of arrays along an axis: + concatenate((a1,a2,a3,...), axis=0) Condition: all arrays must have the same shape for the remaining axes. Currently: method a.concat(a1,a2,a3,...) works only along first axis. 3.3) Concatenation of arrays along a new axis: Can be done by combining concatenate() and indexing with NewAxis. (see comment 4) Abbreviation: reverse(a) = a[::-1] Comments ======== 1) APL uses a special case of transposition for selecting a diagonal, but this is very confusing. 2) In J this is done by using ravel() with reduced rank, but as long as we don't have functions with bounded ranks, we need a special function. 3) I tried to find a construction that does not require yet another syntax and this is the best I could think of. But I am not particularly attached to it, so feel free to think of something better. 4) APL does this with fractional indices, which is probably the weirdest feature in APL. ------------------------------------------------------------------------------- Konrad Hinsen | E-Mail: hinsenk@ere.umontreal.ca Departement de chimie | Tel.: +1-514-343-6111 ext. 3953 Universite de Montreal | Fax: +1-514-343-7586 C.P. 6128, succ. Centre-Ville | Deutsch/Esperanto/English/Nederlands/ Montreal (QC) H3C 3J7 | Francais (phase experimentale) ------------------------------------------------------------------------------- ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From hinsenk@ere.umontreal.ca Tue Feb 20 20:15:39 1996 From: hinsenk@ere.umontreal.ca (Konrad HINSEN) Date: Tue, 20 Feb 1996 15:15:39 -0500 Subject: [PYTHON MATRIX-SIG] returning scalars or rank-0 arrays.. In-Reply-To: <199602201634.LAA21501@maigret> (da@maigret.cog.brown.edu) Message-ID: <199602202015.PAA18143@cyclone.ERE.UMontreal.CA> One issue I've come accross is the "what do I return, a rank-0 entity or a scalar?". My understanding of the current scheme is that indexing returns scalars, while slicing returns ranked entities, even if that rank is 0. Is that correct? I am not sure what the current implementation does, but the idea was that rank-0 arrays should always automatically be converted to scalars. ------------------------------------------------------------------------------- Konrad Hinsen | E-Mail: hinsenk@ere.umontreal.ca Departement de chimie | Tel.: +1-514-343-6111 ext. 3953 Universite de Montreal | Fax: +1-514-343-7586 C.P. 6128, succ. Centre-Ville | Deutsch/Esperanto/English/Nederlands/ Montreal (QC) H3C 3J7 | Francais (phase experimentale) ------------------------------------------------------------------------------- ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From jjh@Goldilocks.LCS.MIT.EDU Tue Feb 20 20:22:09 1996 From: jjh@Goldilocks.LCS.MIT.EDU (James Hugunin) Date: Tue, 20 Feb 96 15:22:09 EST Subject: [PYTHON MATRIX-SIG] returning scalars or rank-0 arrays.. In-Reply-To: <199602201634.LAA21501@maigret> (da@maigret.cog.brown.edu) Message-ID: <9602202022.AA17263@baribal.LCS.MIT.EDU.LCS.MIT.EDU> From: da@maigret.cog.brown.edu (David Ascher) One issue I've come accross is the "what do I return, a rank-0 entity or a scalar?". My understanding of the current scheme is that indexing returns scalars, while slicing returns ranked entities, even if that rank is 0. Is that correct? >>> a = arange(9).reshape(3,3) >>> b = UserArray(a) >>> b UserArray([[0,1,2],[3,4,5],[6,7,8]], 'l') >>> b[0] UserArray([0,1,2], 'l') >>> b[0,2] 2 >>> b[0,2:3] UserArray([2], 'l') If that's correct, then I guess that's what subclasses of UserArray should do, although it's not exactly intuitive. Also, should there be an _as_scalar method in UserArray which returns a scalar in the cases where self.array is rank-0? Note: UserArray([2], 'l') is a rank 1 array of shape (1,). It is not a rank 0 array. According to the standard python notation, a slice does not reduce the dimensionality of an array, even if it selects only one element. Scalars are ALWAYS returned as python scalars (unless you explictitly ask for array(3.14)). In general, a user should never manage to produce a UserArray or a C array of rank-0, unless they somehow specifically ask for it. Consider the following using standard python lists: >>> b = [1,2,3] >>> b[1:2] [2] >>> b[1] 2 -Jim ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From jjh@Goldilocks.LCS.MIT.EDU Tue Feb 20 20:29:05 1996 From: jjh@Goldilocks.LCS.MIT.EDU (James Hugunin) Date: Tue, 20 Feb 96 15:29:05 EST Subject: [PYTHON MATRIX-SIG] Functions and names In-Reply-To: <199602202012.PAA17939@cyclone.ERE.UMontreal.CA> (hinsenk@ere.umontreal.ca) Message-ID: <9602202029.AA17288@baribal.LCS.MIT.EDU.LCS.MIT.EDU> I've read both Paul's and Konrad's proposal's for new naming/function schemes. I just want to make it clear that I would have no objections to implementing any part of either of these schemes, but I only intend to go to the effort if I can see that some consensus is developing on this list for one particular proposal and I'm personally going to stay out of most of that discussion. -Jim ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From chris.chase@jhuapl.edu Tue Feb 20 22:33:19 1996 From: chris.chase@jhuapl.edu (Chris Chase S1A) Date: Tue, 20 Feb 1996 17:33:19 -0500 Subject: [PYTHON MATRIX-SIG] Functions and names In-Reply-To: <199602202012.PAA17939@cyclone.ERE.UMontreal.CA> References: <199602202012.PAA17939@cyclone.ERE.UMontreal.CA> Message-ID: <199602202221.RAA01153@python.org> >>>>> "Konrad" == Konrad HINSEN writes: Konrad> Comments of any kind are of course welcome. I like Konrad's other suggestions along with those already suggested by Paul. Paul seems to have a well thought-out naming convention. I would like to make some comments and additions below: Konrad> Constructors Konrad> ============ Konrad> 1) Construct an array from an arbitrary sequence object or a function: Konrad> + array(sequenceOrFunction, shape=None, type=None) Konrad> Returns an array of the given shape and type using the Konrad> elements from the flattened sequence. I must admit that for constructing matrices the nested list argument seemed natural. This version does not allow it. Otherwise, I like its additional functionality. Konrad> Structural array operations Konrad> =========================== Konrad> 1) Selecting subarrays Support for mapped indexing (like the Tela language) would be useful. Perhaps it should be an attribute so as to allow assignment: print a.mapped[c1,c2,c3] a.mapped[c1,c2,c3] = b c1,c2,c3 index vectors that must have the same shape when converted to an array. Elements of 'a' are chosen by the indexes c1[i],c2[i],c3[i] and the result has the same shape as the shape of the index vectors. Think of the index vectors as being coordinates for some space (I use this style of indexing for paths in two-dimensional image maps or three-dimensional volume data). For assignment 'b' must have the same shape. Konrad> 1.2) Selecting arbitrary elements specified by an array of indices i: Konrad> + take(a, i, axis=0) Konrad> Conditions: i must be of an integer array type, Konrad> minimum.reduce(ravel(i)) >= 0, Konrad> maximum.reduce(ravel(i)) < a.shape[axis] As Paul suggested, perhaps gather() is a better name than take() (gather is a name used in vector libraries). But it really does not matter very much to me. There needs to be a dual capability for assigning the items specified by an array of indices (a completely general form of product indexing). Perhaps something like: a.index[1,i,j] = b with i and j index sequences applied to the second and third dimensions. Here, the 'index' attribute permits index vectors that are scalars, sequences or slices. There is no equivalent to take() since all dimensions must treated simultaneously. Perhaps a better name would be 'subarray'. In fact, such an 'subarray' attribute could also be used for selection, only it would not return a reference but a copy and always return the same rank as 'a' (i.e. does reduce the rank for scalar indexes). If it was only being used for assignment it could be called "scatter" (the dual of gather). An insertion method for inserting a subarray into a larger array would also be useful. Something like: a.insert(b, c1, c2, c3) or a.insert[c1,c2,c3] = b Perhaps someone can come up with a better idea. The concept is: 'b' is inserted into 'a' starting at index c1,c2,c3. The operation overwrites elements of 'a'. It is more convenient then using slice notation which would require one to compute the upper limit on the slices. It could have different behaviors by using a keyword argument for when 'b' does not fit within the dimensions of 'a' (e.g. truncate, wrap around or signal error.) Konrad> 2) Rearranging array elements Konrad> 2.1.2) Reshaping to a rank-1 array: Konrad> ravel(a) Konrad> equivalent to reshape(a,(multiply.reduce(a.shape),)) flatten() could also be a possible name. Why should this be a function rather than a method or attribute? For example, a.flat == ravel(a) but a.flat[s] would allow subscription/assignment of 'a' in flattened form using a scalar, sequence object, or a slice. Of course a combination of take() and ravel() and subscripting can accomplish this. However, flat() and subarray() as methods could completely replace ravel() and take() functions with the ability to simply support flattened or product indexing subscription or assignment. Konrad> 2.1.3) Combining a group of axes into one axis (see comment 2): Konrad> + a[i1, i2, ......, i3, i4] (see comment 3) Konrad> The double ellipsis works similar to the single one, but Konrad> contracts all the axes covered into a single axis. I do not like this. Perhaps an attribute 'collapse' would work? E.g. a.collapse[i1, i2, ..., i3, i4] collapse would interpret '...' to mean contract the axes. Otherwise it behaves exactly like normal subscripting. This is not necessarily a good idea either. Konrad> 2.1.4) Adding an axis of length 1: Konrad> a[..., NewAxis, ...] Of course the "..." are notational and have nothing to do with ellipses indexes. Otherwise, this does not make sense. Konrad> 2.2) Transposition: Konrad> + transpose(a, axes) Konrad> axes is a non-nested sequence of non-negative integers Konrad> with maximum.reduce(axes) < len(a.shape) Yorick has a nice general form of the transpose() function. It allows a variable length argument list where the arguments are cyclic permutations of the indexes. It is actually a little more general. Rather than explain here, a description is available in the Yorick online docs at: ftp://icf.llnl.gov/pub/Yorick/html/yorick_49.html#SEC49 Chris ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From hinsenk@ere.umontreal.ca Tue Feb 20 22:38:41 1996 From: hinsenk@ere.umontreal.ca (Konrad HINSEN) Date: Tue, 20 Feb 1996 17:38:41 -0500 Subject: [PYTHON MATRIX-SIG] Functions and names In-Reply-To: <199602202220.RAA23965@cyclone.ERE.UMontreal.CA> (message from Chris Chase S1A on Tue, 20 Feb 1996 17:33:19 -0500) Message-ID: <199602202238.RAA24811@cyclone.ERE.UMontreal.CA> I must admit that for constructing matrices the nested list argument seemed natural. This version does not allow it. Otherwise, I like its additional functionality. Of course this is still allowed! That was what I meant with the somewhat cryptic comment that the shape is inferred from the sequence if not explicitly specified. Support for mapped indexing (like the Tela language) would be useful. Perhaps it should be an attribute so as to allow assignment: Could you give an example for an application where this would be useful? Konrad> 1.2) Selecting arbitrary elements specified by an array of indices i: Konrad> + take(a, i, axis=0) Konrad> Conditions: i must be of an integer array type, Konrad> minimum.reduce(ravel(i)) >= 0, Konrad> maximum.reduce(ravel(i)) < a.shape[axis] As Paul suggested, perhaps gather() is a better name than take() (gather is a name used in vector libraries). But it really does not matter very much to me. I explicitly avoided "gather" for two reasons: 1) It is not a particularly clear description of what is happening. 2) What I describe is not the exact equivalent of what you find in most vector libraries, because the index array can have a higher rank than 1. There needs to be a dual capability for assigning the items specified by an array of indices (a completely general form of product indexing). I agree that this would be nice, but if I remember correctly there was some problem with putting this into indexing. An insertion method for inserting a subarray into a larger array would also be useful. Something like: The APL way of doing this is to use what I call repeat() to make room for the insertion and then assign to this part. Since this is not a very frequent operation, I don't think we need a special function for it. flatten() could also be a possible name. Why should this be a function rather than a method or attribute? For example, a.flat == ravel(a) As I pointed out in an earlier message, we should better avoid methods for array operations except for very specialized ones (like byteswap or typecode). There is simply no clean way to decide what should be a method and what a function, and besides all operations coded in Python have to be functions anyway. (This, by the way, is the current distinction: everything implemented in C is a method, everything else is a function. I think we agree that this is the worst possible distinction.) but a.flat[s] would allow subscription/assignment of 'a' in flattened form using a scalar, sequence object, or a slice. Of course a So will a function, if it doesn't copy the array but returns a reference. Konrad> 2.1.3) Combining a group of axes into one axis (see comment 2): Konrad> + a[i1, i2, ......, i3, i4] (see comment 3) Konrad> The double ellipsis works similar to the single one, but Konrad> contracts all the axes covered into a single axis. I do not like this. Perhaps an attribute 'collapse' would work? E.g. a.collapse[i1, i2, ..., i3, i4] If that can be done, fine. So what should the value of "collapse" be before subscripting? I can't see a way to make this work. Konrad> 2.1.4) Adding an axis of length 1: Konrad> a[..., NewAxis, ...] Of course the "..." are notational and have nothing to do with ellipses indexes. Otherwise, this does not make sense. Of course. I hadn't thought about this possible confusion. Yorick has a nice general form of the transpose() function. It allows a variable length argument list where the arguments are cyclic permutations of the indexes. It is actually a little more general. I think this is close to what I wanted to describe. I'll have to look up the Yorick documentation... ------------------------------------------------------------------------------- Konrad Hinsen | E-Mail: hinsenk@ere.umontreal.ca Departement de chimie | Tel.: +1-514-343-6111 ext. 3953 Universite de Montreal | Fax: +1-514-343-7586 C.P. 6128, succ. Centre-Ville | Deutsch/Esperanto/English/Nederlands/ Montreal (QC) H3C 3J7 | Francais (phase experimentale) ------------------------------------------------------------------------------- ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From da@maigret.cog.brown.edu Wed Feb 21 16:12:15 1996 From: da@maigret.cog.brown.edu (David Ascher) Date: Wed, 21 Feb 1996 11:12:15 -0500 (EST) Subject: [PYTHON MATRIX-SIG] Functions and names In-Reply-To: <199602202238.RAA24811@cyclone.ERE.UMontreal.CA> from "Konrad HINSEN" at Feb 20, 96 05:38:41 pm Message-ID: <199602211612.LAA24573@maigret> > An insertion method for inserting a subarray into a larger array would > also be useful. Something like: > > The APL way of doing this is to use what I call repeat() to make > room for the insertion and then assign to this part. Since this > is not a very frequent operation, I don't think we need a special > function for it. Actually, when I saw Chris' description of that I thought "yeah!", since I can see myself using it quite a bit. But I guess I can always write my own version. --da ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From dredish@CS.cmu.edu Wed Feb 21 17:59:27 1996 From: dredish@CS.cmu.edu (David Redish) Date: Wed, 21 Feb 1996 12:59:27 -0500 Subject: [PYTHON MATRIX-SIG] MatLab Message-ID: <3626.824925567@GS151.SP.CS.CMU.EDU> I've just downloaded the 0.34 Numeric extension (looks really good so far -- compiled out of the box for HPUX), but MLab.py does not seem to work. e.g. >>> import MLab >>> MLab.tri(5) Traceback (innermost last): File "", line 1, in ? File "/users/adr/lib/python/numeric/MLab.py", line 55, in tri m = greaterEqual(subtract.outer(mrange(N), mrange(M)), -k) NameError: greaterEqual I tried adding "From Numeric import *" to MLab.py and then I get >>> import MLab >>> MLab.tri(5) Traceback (innermost last): File "", line 1, in ? File "/users/adr/lib/python/numeric/MLab.py", line 56, in tri m = greaterEqual(subtract.outer(mrange(N), mrange(M)), -k) AttributeError: outer Any help would be appreciated. thanx adr ------------------------------------------------------------ David Redish Computer Science Department CMU graduate student Neural Processes in Cognition Training Program Center for the Neural Basis of Cognition (CNBC) http://www.cs.cmu.edu/Web/People/dredish/home.html ------------------------------------------------------------ maintainer, CNBC website: http://www.cs.cmu.edu/Web/Groups/CNBC maintainer, Cog-Neuro Sites on the Internet, courtesy CNBC: http://www.cs.cmu.edu/Web/Groups/CNBC/other maintainer, NIPS*96 website: http://www.cs.cmu.edu/Web/Groups/NIPS ------------------------------------------------------------ ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From da@maigret.cog.brown.edu Wed Feb 21 20:18:31 1996 From: da@maigret.cog.brown.edu (David Ascher) Date: Wed, 21 Feb 1996 15:18:31 -0500 (EST) Subject: [PYTHON MATRIX-SIG] MatLab In-Reply-To: <3626.824925567@GS151.SP.CS.CMU.EDU> from "David Redish" at Feb 21, 96 12:59:27 pm Message-ID: <199602212018.PAA25500@maigret> > I've just downloaded the 0.34 Numeric extension (looks really good so > far -- compiled out of the box for HPUX), but MLab.py does not seem to > work. I've hacked a version of MLab which mostly works -- no guarantees. --da """Matlab(tm) compatibility functions. This will hopefully become a complete set of the basic functions available in matlab. The syntax is kept as close to the matlab syntax as possible. One fundamental change is that the first index in matlab varies the fastest (as in FORTRAN). That means that it will usually perform reductions over columns, whereas with this object the most natural reductions are over rows. It's perfectly possible to make this work the way it does in matlab if that's desired. """ from Numeric import * # Elementary Matrices # zeros is from matrixmodule in C # ones is from Numeric.py ## This should be replaced by Paul Dubois' URNG very soon now. import Ranf def rand(*args): """rand(d1,...,dn, typecode='d') returns a matrix of the given dimensions which is initialized to random number in the range [0,1). """ return Ranf.random_sample(args) def eye(N, M=None, k=0, typecode=None): """eye(N, M=N, k=0, typecode=None) returns a N-by-M matrix where the k-th diagonal is all ones, and everything else is zeros. """ if M == None: M = N if type(M) == type('d'): typecode = M M = N if (typecode == None): m = outer(subtract, arange(N), arange(M)).equal(-k) else: i = arange(N, typecode=typecode) m = outer(subtract, i, range(M)).equal(-k) return m def tri(N, M=None, k=0, typecode=None): if M == None: M = N if type(M) == type('d'): typecode = M M = N if (typecode == None): m = outer(subtract, arange(N), arange(M)).greaterEqual(-k) else: i = arange(N, typecode=typecode) m = outer(subtract, i, arange(M)).greaterEqual(-k) return m # Matrix manipulation def diag(v, k=0): s = v.shape if len(s)==1: n = s[0]+abs(k) if k > 0: v = v.concat(zeros(k, v.typecode)) elif k < 0: v = zeros(-k, v.typecode).concat(v) return multiply[[1,0]](eye(n, k=k), v) elif len(s)==2: v = add.reduce(eye(s[0], s[1], k=k)*v) if k > 0: return v[:-k] elif k < 0: return v[-k:] else: return v def fliplr(m): return m[:, ::-1] def flipud(m): return m[::-1] # reshape(x, m, n) is not used, instead use reshape(x, (m, n)) def rot90(m, k=1): k = k % 4 if k == 0: return m elif k == 1: return m.transpose()[::-1,::-1] elif k == 2: return fliplr(m)[::-1,::-1] elif k == 3: return fliplr(m.transpose()) def tril(m, k=0): return tri(m.shape[0], m.shape[1], k=k, typecode=m.typecode())*m def triu(m, k=0): return (1-tri(m.shape[0], m.shape[1], k-1, m.typecode()))*m # Data analysis # Basic operations def max(m): return maximum.reduce(m) def min(m): return minimum.reduce(m) # Actually from BASIS, but it fits in so naturally here... def ptp(m): return max(m)-min(m) def mean(m): return add.reduce(m)/len(m) # sort is done in C but is done row-wise rather than column-wise def msort(m): return sort(m.transpose()).transpose() def median(m): return msort(m)[m.shape[0]/2] def std(m): mu = mean(m) return sqrt(add.reduce(pow(m-mu,2)))/sqrt(len(m)-1) def sum(m): return add.reduce(m) def cumsum(m): return add.accumulate(m) def prod(m): return multiply.reduce(m) def cumprod(m): return multiply.accumulate(m) def trapz(y, x=None): """Integrate f using the trapezoidal rule, where y is f(x). """ if x == None: d = 1 else: d = diff(x) return sum(d * (y[1:]+y[0:-1])/2) def diff(x, n=1): """Discrete difference approximation to the derivative """ if n > 1: return diff(x[1:]-x[:-1], n-1) else: return x[1:]-x[:-1] def dot(x, y): return add.reduce(x*y) def corrcoef(x, y=None): """The correlation coefficients """ c = cov(x, y) d = diag(c) return c/sqrt(outer(multiply, d,d)) def cov(m,y=None): if y != None: m = array([m,y], m.typecode()) mu = mean(m) sum_cov = 0.0 for v in m: sum_cov = sum_cov+outer(multiply, v,v) return (sum_cov-len(m)*outer(multiply,mu,mu))/(len(m)-1) ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From jjh@Goldilocks.LCS.MIT.EDU Wed Feb 21 20:26:29 1996 From: jjh@Goldilocks.LCS.MIT.EDU (James Hugunin) Date: Wed, 21 Feb 96 15:26:29 EST Subject: [PYTHON MATRIX-SIG] Version 0.35 Message-ID: <9602212026.AA22323@baribal.LCS.MIT.EDU.LCS.MIT.EDU> Well, I warned people that v0.34 would be buggy and people did a great job of finding bugs. Thanks to Konrad Hinsen, Perry Stoll and David Ascher in particular, v0.35 is now a little bit more solid. It's in the usual place ftp://sls-ftp.lcs.mit.edu/pub/jjh/patch-0.34-0.35 or ftp://sls-ftp.lcs.mit.edu/pub/jjh/NumericPython-0.35.tar.gz Warning: I did something to break the TestSuite fairly recently, so don't expect it to work for you. I thought it was more important to get these bug fixes out to people quickly than to dive into the TestSuite right now. Enjoy - Jim ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From et@appl-math.tu-muenchen.de Wed Feb 21 23:47:09 1996 From: et@appl-math.tu-muenchen.de (Thomas Schwaller) Date: Thu, 22 Feb 1996 00:47:09 +0100 Subject: [PYTHON MATRIX-SIG] Announement: trimodule.tgz Message-ID: <9602220047.ZM18768@hamster.appl-math.tu-muenchen.de> Hi all, Here's a first release of trimodule, a standalone module for triangulation of 2-dimensional domains.You can get it at: ftp://ftp.appl-math.tu-muenchen.de/pub/et/trimodule.tgz I also sent it to Jim by mail and it should appear in the usual place at the matrixmodule site. Read trimodule.html (With Netscape it looks nice) to get a first impression, (look especially at the last pictures) Hope you enjoy it. I'm sorry for not responding immediatly to incoming mail, I read them just every second or third day at the moment, so I'm not that reactive! :-) I tried to do my best documenting the stuff I did, but be prepared for crashes when doing your first tests. The Input has to be correct! BTW. The Lapack Module of Doug works very fine. On My Linux box I use it with a static double Lapack library only (also built statically into the Interpreter, not as dynamically loadable module for maximal speed) When using it dynamically the import time is not as good, as usual. The big documentation modules are are a problem. One should put the docstrings in *.py modules and load them when needed, I think that would be the best solution (one *.py documentation module for each function). So we have a small c-module, speed and the documentation) Tom ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From dredish@CS.cmu.edu Thu Feb 22 17:13:40 1996 From: dredish@CS.cmu.edu (David Redish) Date: Thu, 22 Feb 1996 12:13:40 -0500 Subject: [PYTHON MATRIX-SIG] Applying a general function to an array Message-ID: <4504.825009220@GS151.SP.CS.CMU.EDU> Is it possible to apply a general python function (that takes one input and produces one output to an array? For example, I'd like to have a python function (implemented in C), "y = f(x)" and then be able to say Y = f(X) and get Y set to an array where each element in Y is f applied to the corresponding element in X. I guess I could get the shape, ravel the array, apply the function using map, convert the list to an array, and reshape it, but that seems like it would be very slow. In a similar vein, is there documentation for 1. the Array C API 2. ofuncs While, I'm at it, I'd like to complement Paul Dubois and David Ascher for their documentation efforts. It's been incredibly helpful. (And of course to Jim and Konrad for all their work putting this together.) adr ------------------------------------------------------------ David Redish Computer Science Department CMU graduate student Neural Processes in Cognition Training Program Center for the Neural Basis of Cognition (CNBC) http://www.cs.cmu.edu/Web/People/dredish/home.html ------------------------------------------------------------ maintainer, CNBC website: http://www.cs.cmu.edu/Web/Groups/CNBC maintainer, Cog-Neuro Sites on the Internet, courtesy CNBC: http://www.cs.cmu.edu/Web/Groups/CNBC/other maintainer, NIPS*96 website: http://www.cs.cmu.edu/Web/Groups/NIPS ------------------------------------------------------------ ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From jjh@Goldilocks.LCS.MIT.EDU Thu Feb 22 17:35:06 1996 From: jjh@Goldilocks.LCS.MIT.EDU (James Hugunin) Date: Thu, 22 Feb 96 12:35:06 EST Subject: [PYTHON MATRIX-SIG] Applying a general function to an array In-Reply-To: <4504.825009220@GS151.SP.CS.CMU.EDU> (message from David Redish on Thu, 22 Feb 1996 12:13:40 -0500) Message-ID: <9602221735.AA28770@baribal.LCS.MIT.EDU.LCS.MIT.EDU> An example of the sort of function you want to do this with would be very helpful in answering this. Many python functions already work as you describe. ie. (This is from my memory of how RBF NN's work) Say I have a Radial Basis Function Neural Net and I want the user to be able to give a python function the input RBF (actually this would be nice thing to have): The standard RBF function is (from memory, I last worked on these things years ago): def gaussian(x, A, B, C): return A*exp(B*(x-C)**2) This function wil normally just work for any reasonable combination of 1d, 2d arrays and scalars. > In a similar vein, is there documentation for > 1. the Array C API arrayobject.h provides some minimal documentation on this. I'd look at the various modules that people have produced for examples. > 2. ofuncs I doubt that there will be much need for additional ofuncs in the future. They are currently only really useful for binary functions that need to operate over a large variety of array types, and need to be applied in numerous interesting ways (pseudo indices, reduction, etc.) If you have something that you really think should be an ofunc, I'd like to know some more of the details. -Jim ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From dredish@CS.cmu.edu Thu Feb 22 18:02:50 1996 From: dredish@CS.cmu.edu (David Redish) Date: Thu, 22 Feb 1996 13:02:50 -0500 Subject: [PYTHON MATRIX-SIG] Applying a general function to an array In-Reply-To: Your message of "Thu, 22 Feb 1996 12:35:06 EST." <9602221735.AA28770@baribal.LCS.MIT.EDU.LCS.MIT.EDU> Message-ID: <368.825012170@GS151.SP.CS.CMU.EDU> >An example of the sort of function you want to do this with would be >very helpful in answering this. Many python functions already work as >you describe. This is fine for already-coded python functions (like exp), but what I really want to know how to add new C-coded functions. For example, I have a gaussian function coded up in C that takes care of normalization, random-sampling with a gaussian distribution, etc. So I'd like to be able to apply that function to each element of an array. I could recode the function in python, but that would be much slower. Also, it's already nicely coded-up as a C-coded python object and software-reuse is the word here, right? e.g. I can currently do: x = 10 G = Gaussian(wt, mean, variance) y = G(x) A = arange(10) G = Gaussian(wt, mean, variance) Y = G(A) where G is a C-coded python object that defines a __call__ function. thanx adr PS. I am using these for NN's similar to RBFs. (: ------------------------------------------------------------ David Redish Computer Science Department CMU graduate student Neural Processes in Cognition Training Program Center for the Neural Basis of Cognition (CNBC) http://www.cs.cmu.edu/Web/People/dredish/home.html ------------------------------------------------------------ maintainer, CNBC website: http://www.cs.cmu.edu/Web/Groups/CNBC maintainer, Cog-Neuro Sites on the Internet, courtesy CNBC: http://www.cs.cmu.edu/Web/Groups/CNBC/other maintainer, NIPS*96 website: http://www.cs.cmu.edu/Web/Groups/NIPS ------------------------------------------------------------ ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From jjh@Goldilocks.LCS.MIT.EDU Thu Feb 22 18:25:04 1996 From: jjh@Goldilocks.LCS.MIT.EDU (James Hugunin) Date: Thu, 22 Feb 96 13:25:04 EST Subject: [PYTHON MATRIX-SIG] Applying a general function to an array In-Reply-To: <368.825012170@GS151.SP.CS.CMU.EDU> (message from David Redish on Thu, 22 Feb 1996 13:02:50 -0500) Message-ID: <9602221825.AA29237@baribal.LCS.MIT.EDU.LCS.MIT.EDU> I assume that you have a python function of the form: PyObject *gaussian_call(PyObject *self, PyObject *args) { double x, r; PyArg_ParseTuple("d", &x); r = calculate(self, x); return Py_BuildValue("d", r); } if you convert this to the following, it will work on arbitrary arrays PyObject *gaussian_call(PyObject *self, PyObject *args) { int n, i; PyObject *op; PyArrayObject *ap, *rp; PyArg_ParseTuple("O", &op); ap = PyArray_ContiguousFromObject(op, PyArray_DOUBLE, 0, 0); rp = PyArray_FromDims(ap->nd, ap->dimensions, PyArray_DOUBLE); n = PyArray_Size(ap); for(i=0; idata)[i] = calculate(self, ((double *)ap->data)[i]); } return PyArray_Return(rp); } I admit this looks a little bit more complicated, but this new version will work on any size/shape of array as well as on python scalars. Any other solution will involve converting the elements of the array into and out of python objects, and the overhead of doing this would probably totally swap the computation. I don't know if this is what you're looking for, but I hope it's helpful. -Jim ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From hinsenk@ere.umontreal.ca Thu Feb 22 18:24:27 1996 From: hinsenk@ere.umontreal.ca (Konrad HINSEN) Date: Thu, 22 Feb 1996 13:24:27 -0500 Subject: [PYTHON MATRIX-SIG] Applying a general function to an array In-Reply-To: <4504.825009220@GS151.SP.CS.CMU.EDU> (message from David Redish on Thu, 22 Feb 1996 12:13:40 -0500) Message-ID: <199602221824.NAA05399@cyclone.ERE.UMontreal.CA> Is it possible to apply a general python function (that takes one input and produces one output to an array? For example, I'd like to have a python function (implemented in C), "y = f(x)" and then be able to say Y = f(X) and get Y set to an array where each element in Y is f applied to the corresponding element in X. There is no real support for this right now. It has to be added, and I am trying to make such things part of my function set proposal (progressingly slowly due to lots of other work). If your function f is written in terms of functions from umath, then it should work automatically, but in the case of a C function or a more complicated Python function (with tests, loops, etc.), you have no other way than to write explicit for-loops or some messy map() application. ------------------------------------------------------------------------------- Konrad Hinsen | E-Mail: hinsenk@ere.umontreal.ca Departement de chimie | Tel.: +1-514-343-6111 ext. 3953 Universite de Montreal | Fax: +1-514-343-7586 C.P. 6128, succ. Centre-Ville | Deutsch/Esperanto/English/Nederlands/ Montreal (QC) H3C 3J7 | Francais (phase experimentale) ------------------------------------------------------------------------------- ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From dubois@kristen.llnl.gov Thu Feb 22 21:03:58 1996 From: dubois@kristen.llnl.gov (P. Dubois) Date: Thu, 22 Feb 1996 13:03:58 -0800 Subject: [PYTHON MATRIX-SIG] URNG-2.2 Message-ID: <9602222103.AA11572@kristen.llnl.gov> URNG-2.2 is available at ftp-icf.llnl.gov/pub/basis/URNG-2.2.tar.gz. This fixes an error in memory management. This was the first C extension I wrote. I'd like to describe the error to make sure I have it right this time. I have a routine that allocates a new "urngobject", and I was using it like this: static PyObject * URNG_CreateGenerator(self, args) PyObject *self; /* Not used */ PyObject *args; { int seed; PyObject *result; if (!PyArg_ParseTuple(args, "i", &seed)) return NULL; result = (PyObject *) newurngobject(seed); return result; } I was getting a core dump when the program ended if I created such an object. As a newbie, I assumed I had the reference count stuff messed up, so I tried returning Py_BuildValue("O", result) instead of just result. I stopped dumping core and went on. But this is wrong, of course, since I now have a reference count of two on the object and it is never deleted. I had made the skeleton using modulator and had not checked the "dealloc" option. When I add such a routine: static void urng_dealloc(self) urngobject *self; { PyMem_DEL(self); } and put it in the object's table, all is well. So is this right, you *have* to supply a dealloc method when you write an extension object? If so, maybe modulator shouldn't give you a choice. Or, am I still confused? ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From hinsenk@ere.umontreal.ca Thu Feb 22 22:29:40 1996 From: hinsenk@ere.umontreal.ca (Konrad HINSEN) Date: Thu, 22 Feb 1996 17:29:40 -0500 Subject: [PYTHON MATRIX-SIG] URNG-2.2 In-Reply-To: <9602222103.AA11572@kristen.llnl.gov> (dubois@kristen.llnl.gov) Message-ID: <199602222229.RAA27584@cyclone.ERE.UMontreal.CA> So is this right, you *have* to supply a dealloc method when you write an extension object? If so, maybe modulator shouldn't give you a choice. That is also my understanding. It also makes sense, because Python can't know how to deallocate your object. It can only keep track of the reference count and call the dealloc method when it reaches zero. So even if it were allowed not to have a dealloc method, this would mean a permanent memory leak. ------------------------------------------------------------------------------- Konrad Hinsen | E-Mail: hinsenk@ere.umontreal.ca Departement de chimie | Tel.: +1-514-343-6111 ext. 3953 Universite de Montreal | Fax: +1-514-343-7586 C.P. 6128, succ. Centre-Ville | Deutsch/Esperanto/English/Nederlands/ Montreal (QC) H3C 3J7 | Francais (phase experimentale) ------------------------------------------------------------------------------- ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From chris.chase@jhuapl.edu Fri Feb 23 22:34:13 1996 From: chris.chase@jhuapl.edu (Chris Chase S1A) Date: Fri, 23 Feb 1996 17:34:13 -0500 Subject: [PYTHON MATRIX-SIG] Functions and names In-Reply-To: <199602202238.RAA24811@cyclone.ERE.UMontreal.CA> References: <199602202220.RAA23965@cyclone.ERE.UMontreal.CA> <199602202238.RAA24811@cyclone.ERE.UMontreal.CA> Message-ID: <199602232227.RAA20187@python.org> I apologize that this response is delayed. My regular work takes priority over my attention to the SIG mailing list. >>>>> "Konrad" == Konrad HINSEN writes: Chase> Support for mapped indexing (like the Tela language) would Chase> be useful. Perhaps it should be an attribute so as to Chase> allow assignment: Konrad> Could you give an example for an application where this would Konrad> be useful? One application where I use this: I often use map projections (e.g. azimuthal) to represent regularly sampled data on a sphere (latitude/longitude or azimuth/elevation coordinates), i.e. data is a 2D array on a regularly sampled lat/lon grid. One way to map data onto the map projection is to compute the lat/lon coordinate of each pixel in the map. Suppose the map is to be an NxM image for a particular projection and that the NxM arrays 'lat' and 'lon' contain the latitude and longitude coordinates of each pixel in map. 'lat' and 'lon' then are converted to indexes according to the grid sampling. Then mapped indexing produces the desired map image: map = data.mapped(lat,lon) As I mentioned previously, I use this kind of indexing when dealing with coordinates for 2D image maps or 3D volume data. The coordinates are often used as part of a transformation (as above) or a path through the data (i.e. a line-of-sight). For more info, mapped indexing is supported in both IDL and Tela. See http://www.geo.fmi.fi/prog/tela/telahelp-5.html#ss5.18. Chase> There needs to be a dual capability for assigning the items Chase> specified by an array of indices (a completely general form Chase> of product indexing). Konrad> I agree that this would be nice, but if I remember correctly Konrad> there was some problem with putting this into indexing. Originally, James Hugunin had implemented general index vectors. However, he stated that they were removed when he decided to have array indexing return by reference. Index vectors do not fit well into this implementation. Returning by reference would require keeping a copy of the index vector and would require a rewrite of the current array representation. Besides, I think it is more natural to return a copy of the indexed elements rather than a reference when using a general index vector. When the operation is assignment there is no reason to not support index vectors since array references are not being created. Just because general product indexing does not completely fit into the current implementation is not a good reason to not support it. Rather than mess around with the current indexing that returns by reference, I suggested a 'subarray' attribute that would support general product indexing for assignment and selection (returning a copy). Chase> An insertion method for inserting a subarray into a larger Chase> array would also be useful. Something like: Konrad> The APL way of doing this is to use what I call repeat() to make Konrad> room for the insertion and then assign to this part. Since this Konrad> is not a very frequent operation, I don't think we need a special Konrad> function for it. I specifically said that the function I was describing overwrites elements in the target array rather than making room. True it is not as frequent of an operation as some others, but it is one that I use other with matrix linear algebra (manipulating block matrices) and for cut-away's and slices in 3D volume data. It is a general operation supported by other languages and not application specific. Konrad> 2.1.3) Combining a group of axes into one axis (see comment 2): Konrad> + a[i1, i2, ......, i3, i4] (see comment 3) Konrad> The double ellipsis works similar to the single one, but Konrad> contracts all the axes covered into a single axis. Chase> I do not like this. Perhaps an attribute 'collapse' would Chase> work? E.g. Chase> a.collapse[i1, i2, ..., i3, i4] Konrad> If that can be done, fine. So what should the value of "collapse" be Konrad> before subscripting? I can't see a way to make this work. It would have to be an object containing a reference to the array 'a'. It would have the special property that the ellipses index, '...', collapses missing dimensions. Otherwise it would act like a normal array. Since this is a special kind of access to the array object, a 'collapse' method would be better, but then it could not support the indexing syntax for slices and ellipses. I personally think that the best solution would be a syntactical symbol like "*", but that idea was shot down. Chris ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From chris.chase@jhuapl.edu Sat Feb 24 00:06:19 1996 From: chris.chase@jhuapl.edu (Chris Chase S1A) Date: Fri, 23 Feb 1996 19:06:19 -0500 Subject: [PYTHON MATRIX-SIG] RubberIndex In-Reply-To: <199602141831.NAA04933@cyclone.ERE.UMontreal.CA> References: <199602141643.LAA28371@cyclone.ERE.UMontreal.CA> <199602141831.NAA04933@cyclone.ERE.UMontreal.CA> Message-ID: <199602240000.TAA20621@python.org> I made a suggestion about having array indexing generate an error when too few indexes are used. Konrad felt that the analogy to lists was more important: >>>>> "Konrad" == Konrad HINSEN writes: Konrad> That wasn't my point either. My point is that since arrays and nested Konrad> lists are treated as interchangeable in many situations, it makes Konrad> sense to use this analogy as much as possible and allow an indexing Konrad> scheme for arrays that matches that for nested arrays. WHich means Konrad> that it should always be allowed to have just one index, independent Konrad> of the rank of the array. For an array 'a' (rank > 1), the current implementation of a[0] adds no functionality since it is equivalent to a[0,...]. I would like to make another suggestion along the lines my original one that also handles the purpose of ravel() for both assignment and selection. In IDL, Tela and Yorick using a single index or slice defaults to indexing an array in flattened form. Otherwise, the number of actual indexes used must equal the rank. A similar implementation could be supported for the array object. This obviates 'flat' attribute that I suggested earlier. a[i] indexes 'a' in flattened form. 'i' can be a scalar or a slice. Then, a[:] == ravel(a) and assignment to 'a' in flattened form would also be supported: a[i] = 2 which eliminates the need for a "flattened" assignment function or for ravel(). Like other array languages, if more than one index is used without a rubber index and there are fewer indexes than the array rank then an error should result. If 'a' has rank 3, then a[0] = 1 b = a[0,...] b = a[0,0,0] would be valid. But a[0,0] would generate an error. This would be consistent with the fact that a[0,0,0,0] also generates an error. This implementation still allows an array to be used anywhere where a sequence object (e.g. list) is used. Chris ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From hinsenk@ere.umontreal.ca Sat Feb 24 14:38:58 1996 From: hinsenk@ere.umontreal.ca (Konrad HINSEN) Date: Sat, 24 Feb 1996 09:38:58 -0500 Subject: [PYTHON MATRIX-SIG] Functions and names In-Reply-To: <199602232226.RAA24062@cyclone.ERE.UMontreal.CA> (message from Chris Chase S1A on Fri, 23 Feb 1996 17:34:13 -0500) Message-ID: <199602241438.JAA20228@cyclone.ERE.UMontreal.CA> I often use map projections (e.g. azimuthal) to represent regularly sampled data on a sphere (latitude/longitude or azimuth/elevation coordinates), i.e. data is a 2D array on a regularly sampled lat/lon grid. One way to map data onto the map projection is to compute the lat/lon coordinate of each pixel in the map. Suppose the map is to be an NxM image for a particular projection and that the NxM arrays 'lat' and 'lon' contain the latitude and longitude coordinates of each pixel in map. 'lat' and 'lon' then are converted to indexes according to the grid sampling. Then mapped indexing produces the desired map image: map = data.mapped(lat,lon) I think I understand this now. In fact, I was planning to include this feature in my function proposal, but under "mapping functions" (yet to be written) rather than "indexing", because it is just a special case of array mapping with simple indexing as the function to be applied. current array representation. Besides, I think it is more natural to return a copy of the indexed elements rather than a reference when using a general index vector. Indeed, but this creates the unpleasant situation that an expression like a[i,j] returns either a copy or a reference, depending on the values of i and j. Having references at all is confusing enough, but this would certainly cause some headaches. Personally I'd prefer to have standard indexing always return a copy (that is the safer option) and provide an alternative syntax for obtaining a reference. Then standard indexing could again allow general index vectors. I specifically said that the function I was describing overwrites elements in the target array rather than making room. True it is not Then allowing general index vectors in indexed assignments should be all you need, right? Chase> I do not like this. Perhaps an attribute 'collapse' would Chase> work? E.g. Chase> a.collapse[i1, i2, ..., i3, i4] Konrad> If that can be done, fine. So what should the value of "collapse" be Konrad> before subscripting? I can't see a way to make this work. It would have to be an object containing a reference to the array 'a'. It would have the special property that the ellipses index, '...', collapses missing dimensions. Otherwise it would act like a normal array. Since this is a special kind of access to the array object, a So effectively you want to introduce a second array type with a slightly different behaviour. I don't think this is a good idea. There would be no way to stop users from just writing a.collapse and assign the result to variables, return it from library functions etc. Suddenly there are two array types in free circulation that behave almost, but not quite, identically. ------------------------------------------------------------------------------- Konrad Hinsen | E-Mail: hinsenk@ere.umontreal.ca Departement de chimie | Tel.: +1-514-343-6111 ext. 3953 Universite de Montreal | Fax: +1-514-343-7586 C.P. 6128, succ. Centre-Ville | Deutsch/Esperanto/English/Nederlands/ Montreal (QC) H3C 3J7 | Francais (phase experimentale) ------------------------------------------------------------------------------- ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From hinsenk@ere.umontreal.ca Sat Feb 24 14:50:39 1996 From: hinsenk@ere.umontreal.ca (Konrad HINSEN) Date: Sat, 24 Feb 1996 09:50:39 -0500 Subject: [PYTHON MATRIX-SIG] RubberIndex In-Reply-To: <199602232358.SAA29109@cyclone.ERE.UMontreal.CA> (message from Chris Chase S1A on Fri, 23 Feb 1996 19:06:19 -0500) Message-ID: <199602241450.JAA20452@cyclone.ERE.UMontreal.CA> In IDL, Tela and Yorick using a single index or slice defaults to indexing an array in flattened form. Otherwise, the number of actual indexes used must equal the rank. I won't deny that this has some practical advantages, but it violates the principle that an array is first of all a structured collection of data with a definite shape. Changing this structure should always be an explicit operation, not hidden in a special rule for indexing. For me it still makes more sense to consider an array equivalent to a nested list with the same structure. Like other array languages, if more than one index is used without a rubber index and there are fewer indexes than the array rank then an error should result. I certainly agree on that. ------------------------------------------------------------------------------- Konrad Hinsen | E-Mail: hinsenk@ere.umontreal.ca Departement de chimie | Tel.: +1-514-343-6111 ext. 3953 Universite de Montreal | Fax: +1-514-343-7586 C.P. 6128, succ. Centre-Ville | Deutsch/Esperanto/English/Nederlands/ Montreal (QC) H3C 3J7 | Francais (phase experimentale) ------------------------------------------------------------------------------- ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From hinsenk@ere.umontreal.ca Sat Feb 24 18:41:40 1996 From: hinsenk@ere.umontreal.ca (Konrad HINSEN) Date: Sat, 24 Feb 1996 13:41:40 -0500 Subject: [PYTHON MATRIX-SIG] Two weeks' experience Message-ID: <199602241841.NAA28353@cyclone.ERE.UMontreal.CA> During the last two weeks I have been very busy with my real work, but most of this has been done in Python, with lots of arrays. I was helping a colleague to locate a bug in his Fortran code that is very important for both of us. Python has been incredibly useful for this task, but I have also discovered a few serious problems, which I would like to report here. My point is mostly to show how important practical applications are to find weak points in the current design and implementation. My first and most frustrating experience was that it is very difficult to write code that works for both arrays and scalars (or rank-0 arrays). Sure, a one-line expression using operators and mathematical formulas is no problem. But as soon as there are conditionals, loops, etc., the easiest way is often to have two separate functions. All this is caused mostly by conditional expressions. To check whether all elements of a are greater than the corresponding elements of b you can write andLogical.reduce(ravel(a.greater(b))). But if a is a scalar, this has to be andLogical.reduce(ravel(b.less(a))), and if both a and b are scalars it must be a > b. I ended up writing a special comparison function that uses one of these expressions depending on the type of the arguments. This shows that no array operations that make sense for rank-0 arrays and scalars (i.e. most) should be implemented as a method of the array type. I had reached this conclusion before on the basis of other arguments, but this practical frustration was much more convincing! Another problem that cost me two hours to solve was matrix multiplication. I had a three-dimensional wave function evaluated on a grid, and therefore stored in the array psi of rank 3. I wanted to apply the operator for kinetic energy in one direction, which is a rank-2 array, t. My first attempt was t.matrixMultiply(psi), and I expected this to do a dot product using the last axis of tx and the first of psi. But no, all I got was an exception. So I had to write this function myself, which I still considered a minor exercise. More than an hour later, I had arrived at the following code: def dot(a1, a2): r1 = len(a1.shape)-1 r2 = len(a2.shape)-1 axes = [r1] + range(r1) a1 = a1.transpose(axes).copy() f = a1.reshape a1 = apply(f, a1.shape + r2*(1,)) a2 = a2.copy() f = a2.reshape a2 = apply(f, a2.shape[0:1] + r1*(1,) + a2.shape[1:]) return add.reduce(a1*a2) If you think that this ought to be easier, I agree. The basic idea of this function is to reshape both arrays to make the axis to be summed over the first, and to add new axes of length one to make things match. All the trouble is caused by two unpleasant features of the method reshape(): 1) It works only on contiguous arrays, which makes it necessary to copy the arguments first. 2) It wants the new shape in the form of several integer arguments, which causes the weird construction used above to obtain a rank that is calculated rather than a known constant. The final operation I want to complain about is fromFunction(). I had always thought that it calls the supplied function once for each element, supplying integer indices as arguments. But no, it tries to be smarter, by calling the function only once with suitably shaped arrays as arguments. That is of course much more efficient, but it limits the function to a straightforward sequence of ofuncs, in which case there is hardly any need to use the fromFunction() constructor. Most cases I can think of where this constructor makes sense involve functions with conditional expressions, like my kinetic energy operator: def tij(i,j): if i == j: return pi**2/3 else: d = i-j return 2.*(-1.)**d/d**2 So instead of the straightforward and readable expression t = (hbar**2/(2.*mass*delta**2)) * fromFunction([ngrid,ngrid], tij) I had to write the very Fortran-like code t = zeros(ngrid,ngrid) for i in range(ngrid): for j in range(ngrid): t[i,j] = tij(i,j) t = (hbar**2/(2.*mass*delta**2)) * t There were a few more minor problems, but I think I can stop now. The direct consequences of the problems described here will appear in my proposal for reorganising the array functions, but most of all I'd like to urge all of you to use what we have for as many applications you can think of. And we should seriously consider a public alpha release to get more testers. ------------------------------------------------------------------------------- Konrad Hinsen | E-Mail: hinsenk@ere.umontreal.ca Departement de chimie | Tel.: +1-514-343-6111 ext. 3953 Universite de Montreal | Fax: +1-514-343-7586 C.P. 6128, succ. Centre-Ville | Deutsch/Esperanto/English/Nederlands/ Montreal (QC) H3C 3J7 | Francais (phase experimentale) ------------------------------------------------------------------------------- ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From steve@estel.uindy.edu Sat Feb 24 22:40:21 1996 From: steve@estel.uindy.edu (Steve Spicklemire) Date: Sat, 24 Feb 96 17:40:21 -0500 Subject: [PYTHON MATRIX-SIG] Pretty Printer Message-ID: <9602242240.AA26596@estel.uindy.edu> I'm probably doing something wrong but I couldn't get PrettyPrinter to work with shape (1,) float arrays. I made this change to mine... (there's probably a much better way to do this... but at least I'm back up and running...) RCS file: PrettyPrinter.py,v retrieving revision 1.1 retrieving revision 1.2 diff -c5 -r1.1 -r1.2 *** /tmp/,RCSt1026586 Sat Feb 24 17:36:04 1996 --- /tmp/,RCSt2026586 Sat Feb 24 17:36:05 1996 *************** *** 86,96 **** format = format + str(max_str_len) + '.' + str(precision) + 'e' if large_exponent: format = format + '3' item_length = max_str_len + 1 else: format = '%.' + str(precision) + 'f' ! precision = min(precision, apply(max, tuple(map(lambda x, p=precision, f=format: _digits(x,p,f), data)))) max_str_len = len(str(int(max_val))) + precision + 2 if sign: format = '%#+' else: format = '%#' --- 86,99 ---- format = format + str(max_str_len) + '.' + str(precision) + 'e' if large_exponent: format = format + '3' item_length = max_str_len + 1 else: format = '%.' + str(precision) + 'f' ! if len(data.shape) == 1 and data.shape[0] == 1: ! precision = min(precision, _digits(data[0],precision,format)) ! else: ! precision = min(precision, apply(max, tuple(map(lambda x, p=precision, f=format: _digits(x,p,f), data)))) max_str_len = len(str(int(max_val))) + precision + 2 if sign: format = '%#+' else: format = '%#' ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From hinsenk@ere.umontreal.ca Sun Feb 25 14:35:58 1996 From: hinsenk@ere.umontreal.ca (Konrad HINSEN) Date: Sun, 25 Feb 1996 09:35:58 -0500 Subject: [PYTHON MATRIX-SIG] Pretty Printer In-Reply-To: <9602242240.AA26596@estel.uindy.edu> (message from Steve Spicklemire on Sat, 24 Feb 96 17:40:21 -0500) Message-ID: <199602251435.JAA24211@cyclone.ERE.UMontreal.CA> I'm probably doing something wrong but I couldn't get PrettyPrinter to work with shape (1,) float arrays. I made this change to mine... (there's A better patch is precision = min(precision, max(tuple(map(lambda x, p=precision, f=format: _digits(x,p,f), data)))) There seem to be too many versions of the pretty printer around. I had assumed that everyone by now should have the latest one, but appearantly that's not true. I am even more surprised that there are no more serious complaints. When I installed version 0.35 of the numerics package, I would always get error messages when typing "from Numeric import *" due to recursive imports of Numeric and PrettyPrinter. Am I the only one with this problem? Or has everyone silently fixed this? ------------------------------------------------------------------------------- Konrad Hinsen | E-Mail: hinsenk@ere.umontreal.ca Departement de chimie | Tel.: +1-514-343-6111 ext. 3953 Universite de Montreal | Fax: +1-514-343-7586 C.P. 6128, succ. Centre-Ville | Deutsch/Esperanto/English/Nederlands/ Montreal (QC) H3C 3J7 | Francais (phase experimentale) ------------------------------------------------------------------------------- ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From hinsenk@ere.umontreal.ca Sun Feb 25 19:22:17 1996 From: hinsenk@ere.umontreal.ca (Konrad HINSEN) Date: Sun, 25 Feb 1996 14:22:17 -0500 Subject: [PYTHON MATRIX-SIG] Updated function list Message-ID: <199602251922.OAA02089@cyclone.ERE.UMontreal.CA> Thanks to a rainy weekend, I have completed the proposed function list. It now covers all operations that I can think of (which doesn't mean that nothing is missing!). The descriptions of array() and reshape() have changed with respect to the first draft. Enjoy and ciriticize! --------------------------------------------------------------------------- This list is an attempt to put some order into the array functions. I have made a list of functions that should be available and proposed a name for each function. Lines that describe new or modified operations are marked with a '+'. Comments are at the end of the file. As I have suggested before, it is useful to have an additional module defining abbreviations and/or more meaningful names for often-used combinations. These should not be part of the standard module to reduce name space pollution and learning overhead, but they should nevertheless be part of the standard distribution to ensure uniformity. I'll include recommended abbreviations in the respective sections. The following principles have been applied: 1) Every operation that is not closely related to the implementation details of the array object is implemented as a function, not as a method. This is necessary because methods don't work on scalars, which also serve as rank-0 arrays. It is also necessary to allow the implementation of every operation in C or Python without changing the interface. And this decision also ensures consistency with additional modules (linear algebra etc.); such modules can only add functions, not methods. 2) There are no functions with a variable number of array arguments. Such functions are difficult to apply to a number of arrays that is calculated in the code rather than constant. Instead, a sequence of arrays is passed. This has the added advantage that a higher-dimensional array can be passed, implying an iteration over the first axis. Constructors ============ 1) Construct an array from an arbitrary sequence object or a function: + array(object, shape=None, type=None) Creates an array from the given object with a given shape and type. If the shape and/or the type are not specified, they are inferred from the data. The object providing the data can be of the following types: 1) An object with a method "toArray". This method is called, and the resulting array is converted to the given shape and type (if not None). 2) A nested list or tuple, or an array. The shape is inferred from the nesting, the type from the data, unless space and/or type are specified. 3) A file object. The file is read as a rank-2 array. If a shape and/or type is specified, the result is changed to that shape/type. 4) A callable object. This object is called once for each array element with the indices as arguments. In reshaping to an explicitly given shape, the input is reused from the beginning if necessary. It is also allowed not to use all the input data. This constructor replaces the current constructors array(), copyAndReshape(), and fromFunction(), and makes special cases like zeros() so easy that they don't have to be standard functions any more. Note: the current fromFunction() behaves in a subtly different way: it calls the function only once with index arrays as arguments. This is equivalent if the function consists only of operators and ofuncs, and of course much more efficient, but for most interesting functions it doesn't work at all. In this case I prefer generality to efficiency. 2) Construct a rank-1 array of equally spaced points: + arrayRange(x1, x2=None, x3=None) Currently: function arange() does exactly the same. Abbreviations: zeros(n1,n2,...) equals array(0,(n1,n2,...)) ones(n1,n2,...) equals array(1,(n1,n2,...)) unitMatrix(n) equals array([1]+n*[0], (n,n)) arange equals arrayRange Structural array operations =========================== 1) Selecting subarrays 1.1) Selecting on a "raster" along each coordinate: Done by indexing with integers and slices 1.2) Selecting arbitrary elements specified by an array of indices i: + take(a, i, axis=0) Conditions: i must be of an integer array type, minimum.reduce(ravel(i)) >= 0, maximum.reduce(ravel(i)) < a.shape[axis] Currently: method a.take(i, axis=0), i can be of any type, but must have rank 1. 1.3) Selecting the diagonal, i.e. those values where the indices along two axes are equal (see comment 1): + diagonal(a, axis1=0, axis2=1) Conditions: axis1 < len(a.shape), axis2 < len(a.shape), a.shape[axis1] == a.shape[axis2] 2) Rearranging array elements 2.1) Changing the shape 2.1.1) General reshaping: + reshape(a,shape, copy=1) shape can be of any non-nested sequence type Condition: multiply.reduce(shape) == multiply.reduce(a.shape) If copy==1, the returned array is a new copy, otherwise a reference to a. Currently: method a.reshape(length1, length2, ...) This syntax does not provide an easy way to specify a non-constant shape. Also, the current behaviour is like copy=0, which can lead to unpleasant surprises (since otherwise only indexing returns references) and limits reshaping to contiguous arrays. 2.1.2) Reshaping to a rank-1 array: + ravel(a, copy=1) equivalent to reshape(a,(multiply.reduce(a.shape),), copy) 2.1.3) Combining a group of axes into one axis (see comment 2): + a[i1, i2, ......, i3, i4] (see comment 3) The double ellipsis works similar to the single one, but contracts all the axes covered into a single axis. 2.1.4) Adding an axis of length 1: a[..., NewAxis, ...] 2.2) Transposition: + transpose(a, axes) axes is a non-nested sequence of non-negative integers with maximum.reduce(axes) < len(a.shape) Currently: method a.transpose(axes) does the same, but there is a bug that sometimes gives wrong results if an axis is used more than once in the list. It also insists that len(axes) == len(a.shape), although there is no need for this restriction. 3) Replicating and combining array elements 3.1) Replicating elements along an axis: + repeat(a, n, axis=0) n is a non-nested integer sequence with len(n) == a.shape[axis] Each element in a is repeated as often as indicated by the corresponding element in n. The length of the result along the specified axis is add.reduce(n). Currently: function compress(n, a, axis=0) is a special case limited to boolean (0/1) elements in n. 3.2) Concatenation of arrays along an axis: + concatenate((a1,a2,a3,...), axis=0) Condition: all arrays must have the same shape for the remaining axes. Currently: method a.concat(a1,a2,a3,...) works only along first axis and is difficult to apply to a variable number of arguments. 3.3) Concatenation of arrays along a new axis: Can be done by combining concatenate() and indexing with NewAxis. (see comment 4) Abbreviation: reverse(a) = a[::-1] Arithmetic and logical operations ================================= 1) Binary elementwise arithmetic operators + - * / % ** as functions: add, subtract, multiply, divide, remainder, power 2) Standard "mathematical" functions arccos, arccosh, arcsin, arcsinh, arctan, arctanh, cos, cosh, exp, log, log10, sin, sinh, sqrt, tan, tanh 3) Other elementwise arithmetic functions maximum, minimum, clip + conjugate Currently, conjugate() is implemented as a method both on complex objects and array objects. It should be available as a function, because then it can be applied to other number objects as well. There are many algorithms that work for both real and complex numbers providing that conjugates are used here and there. It ought to be possible to implement these algorithms and make them work for float and complex objects as well as array objects. The method we have now doesn't work on real and integer objects. 4) Comparison functions + equal(a,b), notEqual(a,b), greater(a,b), greaterEqual(a,b), + less(a,b), lessEqual(a,b) Currently all these are methods. Apart from introducing an arbitrary asymmetry, this makes it impossible to write code that works both for scalars and arrays. 5) Logical and boolean operators + logicalAnd(a,b), logicalOr(a,b), logicalXor(a,b), logicalNot(a), + booleanAnd(a,b), booleanOr(a,b), booleanXor(a,b), booleanNot(a,b) Currently there are equivalent methods with "inversed" names. I think these names are more intuitive. About the problem of methods, see above. For some strange reason there is currently no logical xor operation. 6) Non-elementwise operations 6.1) Attribute functions of binary operator functions (see 1 and 5) reduce, accumulate, + outer + Note: reduce and accumulate should return the neutral + element of their base operation, reshaped to agree with the + remaining axes, if used along an axis of length zero. + Example: + add.reduce(array(42, (2,0,1)), 1) + should be equal to + array(0, (2,1)) 6.2) Dot/matrix product + dot(a, b, axis_a = -1, axis_b = 0) equivalent to: if axis_a < 0: axis_a = len(a.shape)+axis_a if axis_b >= 0: axis_b = axis_b + len(a.shape) add.reduce(diagonal(multiply.outer(a,b), axis_a, axis_b)) (but of course done more efficiently) Currently: method matrixMultiply, restricted to rank-2 arrays. Redundant method: nonzero I can't see what this operation is good for. It seems to be equivalent to ravel(notEqual(a,0)), which is not an extremely frequent combination. Abbreviations: sum = add.reduce cumsum = add.accumulate product = multiply.reduce cumproduct = multiply.accumulate allTrue = logicalAnd.reduce someTrue = logicalOr.reduce ptp(a) = maximum.reduce(a)-minimum.reduce(a) Looping, mapping, filtering, etc. ================================= 1) Looping over array elements for element in a: ... Looping over anything else than the first axis can be achieved by indexing. 2) Mapping a function on one or more arrays + arrayMap(function, (a1, a2, a3, ..., an), ranks = None) function must be a callable object. If ranks is specified, it must be a sequence of integers of length n; if ranks is None, all ranks are assumed to be one. The frames of all arrays with respect to the rank specification must match. 3) Filtering + arrayFilter(function, a, axis=0) collects all elements along the given axis for which the supplied function returns non-0. See also repeat() in section "structural operations". 4) Conditional selection + choose(a, (b0, b2, b3, ..., bn)) a must be an array of integers between 0 and n. The result has the same shape as a with elements selected from b0...bn according to the value of the corresponding element of a. Currently: method a.choose(b0, ..., bn) Passing the arrays b0...bn as a tuple has the advantage that this tuple can be constructed arbitrarily in the code. 5) Sorting 5.1) Sort an array + sortArray(a, function=lambda x:x, axis=0) Sorts the elements along the specified axis according to the return value of the supplied function. Note: the default function makes sense only for rank-1 arrays. 5.2) Sort index + sortIndex(a, function=lambda x:x, axis=0) returns an index array i such that take(a,i) == sort(a, function, axis) Redundant function: where(condition, x, y) equals choose(condition, (x,y)) Other operations ================ 1) Type casts 1.1) General cast + a.asType(typecode) 1.2) Special cast functions that also work for scalars + integerArray(a) + floatArray(a) + complexArray(a) 2) Implementation dependent functions: 2.1) Obtain the amount of bytes occupied by each element a.itemsize() 2.2) Return a byte-swapped copy + a.byteswapped() 2.3) Return the element type code a.typecode() 2.4) Return the element type + a.elementType() returns the type object of the Python scalar object that results if an element is extracted. Returns None for a general object (typecode 'O'). 2.5) Checks if an array is contiguous + a.isContiguous() 2.1) Return data as a Python string a.toString() Redundant function: a.copy() Copying should be done with copy.copy(), just as for any other object. Comments ======== 1) APL uses a special case of transposition for selecting a diagonal, but this is often confusing. 2) In J this is done by using ravel() with reduced rank, but as long as we don't have functions with bounded ranks, we need a special function. 3) I tried to find a construction that does not require yet another syntax and this is the best I could think of. But I am not particularly attached to it, so feel free to think of something better. 4) APL does this with fractional indices, which is probably the weirdest feature in APL. ------------------------------------------------------------------------------- Konrad Hinsen | E-Mail: hinsenk@ere.umontreal.ca Departement de chimie | Tel.: +1-514-343-6111 ext. 3953 Universite de Montreal | Fax: +1-514-343-7586 C.P. 6128, succ. Centre-Ville | Deutsch/Esperanto/English/Nederlands/ Montreal (QC) H3C 3J7 | Francais (phase experimentale) ------------------------------------------------------------------------------- ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From jjh@Goldilocks.LCS.MIT.EDU Mon Feb 26 16:28:48 1996 From: jjh@Goldilocks.LCS.MIT.EDU (James Hugunin) Date: Mon, 26 Feb 96 11:28:48 EST Subject: [PYTHON MATRIX-SIG] Two weeks' experience In-Reply-To: <199602241841.NAA28353@cyclone.ERE.UMontreal.CA> (hinsenk@ere.umontreal.ca) Message-ID: <9602261628.AA26473@baribal.LCS.MIT.EDU.LCS.MIT.EDU> As the only person here with several months of experience writing pratical code, I'd like to comment on a few of Konrad's points. 1) You can in fact write greater(a, b) already, today, using the current package. In fact, I personally do this all this time. Once again, I wish that people would at least try things before complaining that they can't be done. 2) reshape does clearly need work, no arguments here 3) fromFunction was only written to prove to Tom Schwaller that it could be done. It is in fact very useful in it's current form. I have made every efort to avoid having to loop over python functions in the array code as this is ridiculously slow. If you want these functions they can always be easily written in python. I really feel that once this gets released beyond this list it's going to get difficult to make the kinds of major changes that Konrad is advocating, so I'm reluctant to go forward on a public alpha release until these issues are at least somewhat resolved. -Jim ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From maurice@blueskyprod.com Mon Feb 26 15:11:13 1996 From: maurice@blueskyprod.com (Maurice Van Swaaij) Date: Mon, 26 Feb 1996 10:11:13 -0500 Subject: [PYTHON MATRIX-SIG] openGL Message-ID: <3131CD91.1CFB@blueskyprod.com> I've downloaded the alpha version of openGL from http://maigret.cog.brown.edu/python/opengl/. It says that it needs the python matrix module. Is there an alpha available to test this openGL module with ? -- Maurice Van Swaaij System Architect Blue Sky Productions maurice@blueskyprod.com 1 914 941 5260 ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From jjh@Goldilocks.LCS.MIT.EDU Mon Feb 26 16:38:14 1996 From: jjh@Goldilocks.LCS.MIT.EDU (James Hugunin) Date: Mon, 26 Feb 96 11:38:14 EST Subject: [PYTHON MATRIX-SIG] RubberIndex In-Reply-To: <199602240000.TAA20621@python.org> (message from Chris Chase S1A on Fri, 23 Feb 1996 19:06:19 -0500) Message-ID: <9602261638.AA26505@baribal.LCS.MIT.EDU.LCS.MIT.EDU> Just a quick point: If 'a' has rank 3, then a[0] = 1 This implementation still allows an array to be used anywhere where a sequence object (e.g. list) is used. This is not true. In order to be able to use an array anywhere a list is used, a[0] for a rank 3 array must return a rank 2 array, because this is how it works for multidimensional lists currently. -Jim ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From chris.chase@jhuapl.edu Mon Feb 26 17:41:00 1996 From: chris.chase@jhuapl.edu (Chris Chase S1A) Date: Mon, 26 Feb 1996 12:41:00 -0500 Subject: [PYTHON MATRIX-SIG] RubberIndex In-Reply-To: <9602261638.AA26505@baribal.LCS.MIT.EDU.LCS.MIT.EDU> References: <199602240000.TAA20621@python.org> <9602261638.AA26505@baribal.LCS.MIT.EDU.LCS.MIT.EDU> Message-ID: <199602261735.MAA06305@python.org> >>>>> "James" == James Hugunin writes: James> Just a quick point: James> If 'a' has rank 3, then James> a[0] = 1 James> This implementation still allows an array to be used anywhere where a James> sequence object (e.g. list) is used. James> This is not true. My statement was that it could be used anywhere a sequence object is used, a list being an example of a sequence object. James> In order to be able to use an array anywhere a list is used, You will never be able to use the current array implementation everywhere that a list is used because array has different definitions for operators (e.g. *, +) and array can not support 'del' without having to copy all of its elements. If you want to use an array as a list then use a.toList() (the toList() method would not be needed if arrays were a subclass of lists). James> a[0] for a rank 3 array must return a rank 2 array, because James> this is how it works for multidimensional lists currently. "must"? Why? As far as I can tell the hierarchical indexing for arrays was a convenience for those used to working lists, because it duplicates behavior already provided by a[0,...] indexing. I was suggesting a different behavior for arrays as sequence objects. My suggestion regarded supporting the use of flattened indexing for both selection and assignment. This specific solution for flattened indexing is supported in all the array languages that I have had exposure to (e.g. Matlab, IDL, Tela, Yorick). To me this is more important than the hierarchical indexing scheme so that an array looks like a list (but does not act like a list for other purposes). As an alternative, a 'flat' (or 'flattened') array attribute could be supported. An aside: I know that Konrad Hinsen prefers functions to methods and attributes so that the same code will work for both scalars and arrays. But this is similar to the list analogy above. scalars will never work in all array code because scalars are neither sequences nor dictionary types. The first time one attempts to subscript a scalar an error will be produced. Certainly we don't want subscription to be a function rather than a method. Rather than jump onto the everything is a function bandwagon we may need to rethink the relationship between scalars and arrays. What is the entire scope of code where scalars and arrays need to be interchangeable? Chris ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From chris.chase@jhuapl.edu Mon Feb 26 19:30:43 1996 From: chris.chase@jhuapl.edu (Chris Chase S1A) Date: Mon, 26 Feb 1996 14:30:43 -0500 Subject: [PYTHON MATRIX-SIG] Functions and names In-Reply-To: <199602241438.JAA20228@cyclone.ERE.UMontreal.CA> References: <199602232226.RAA24062@cyclone.ERE.UMontreal.CA> <199602241438.JAA20228@cyclone.ERE.UMontreal.CA> Message-ID: <199602261925.OAA07208@python.org> >>>>> "Konrad" == Konrad HINSEN writes: Chase> current array representation. Besides, I think it is more Chase> natural to return a copy of the indexed elements rather Chase> than a reference when using a general index vector. Konrad> Indeed, but this creates the unpleasant situation that an expression Konrad> like a[i,j] returns either a copy or a reference, depending on the Konrad> values of i and j. Having references at all is confusing enough, but Konrad> this would certainly cause some headaches. This was not my suggestion. My suggestion kept the current behavior. I suggested a different access method or attribute called 'subarray' allowing the use of a general index vector that would return a copy. Konrad> Personally I'd prefer to have standard indexing always return a copy Konrad> (that is the safer option) and provide an alternative syntax for Konrad> obtaining a reference. Then standard indexing could again allow Konrad> general index vectors. Personally, I currently only care that general index vectors be supported for both subscription and assignment. Chase> I specifically said that the function I was describing overwrites Chase> elements in the target array rather than making room. True it is not Konrad> Then allowing general index vectors in indexed assignments should Konrad> be all you need, right? One only needs slices. The insertion functionality I suggested only requires one to specify the starting position. Normally, the user would have to compute the slice ending position and check to see if the input array will fit (if not the input array could be sliced to fit). I was suggesting a insertion function that would handle options of wrap-around versus truncation versus just error signaling. It is very convenient for manipulating block regions of arrays (like placing one image inside another image). Perhaps a better name than "insert" would be "place"? Chase> I do not like this. Perhaps an attribute 'collapse' would Chase> work? E.g. Chase> a.collapse[i1, i2, ..., i3, i4] Konrad> If that can be done, fine. So what should the value of "collapse" be Konrad> before subscripting? I can't see a way to make this work. Chase> It would have to be an object containing a reference to the Chase> array 'a'. It would have the special property that the Chase> ellipses index, '...', collapses missing dimensions. Chase> Otherwise it would act like a normal array. Since this is a Chase> special kind of access to the array object, a Konrad> So effectively you want to introduce a second array type with Konrad> a slightly different behaviour. Konrad> I don't think this is a good idea. There would be no way to Konrad> stop users from just writing a.collapse and assign the result Konrad> to variables, return it from library functions etc. Suddenly Konrad> there are two array types in free circulation that behave Konrad> almost, but not quite, identically. I was not sure that it was a good solution for this reason. I suggested an attribute because attributes can support the indexing syntax whereas functions can not. ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From hinsenk@ere.umontreal.ca Mon Feb 26 21:09:38 1996 From: hinsenk@ere.umontreal.ca (Konrad HINSEN) Date: Mon, 26 Feb 1996 16:09:38 -0500 Subject: [PYTHON MATRIX-SIG] Two weeks' experience In-Reply-To: <9602261628.AA26473@baribal.LCS.MIT.EDU.LCS.MIT.EDU> (jjh@Goldilocks.LCS.MIT.EDU) Message-ID: <199602262109.QAA23356@cyclone.ERE.UMontreal.CA> 3) fromFunction was only written to prove to Tom Schwaller that it could be done. It is in fact very useful in it's current form. I have made every efort to avoid having to loop over python functions in the array code as this is ridiculously slow. If you want these functions they can always be easily written in python. Of course, but something straightforward should be in the standard function set. The reason why I think the current fromFunction() is not so important is that it can easily be replaced: fromFunction([1,2], f) is the same as apply(f, indices(1,2)) I really feel that once this gets released beyond this list it's going to get difficult to make the kinds of major changes that Konrad is advocating, so I'm reluctant to go forward on a public alpha release until these issues are at least somewhat resolved. Of course it would be necessary to warn people that things are going to change. It would even make sense to include a list of proposals and ask for comments. ------------------------------------------------------------------------------- Konrad Hinsen | E-Mail: hinsenk@ere.umontreal.ca Departement de chimie | Tel.: +1-514-343-6111 ext. 3953 Universite de Montreal | Fax: +1-514-343-7586 C.P. 6128, succ. Centre-Ville | Deutsch/Esperanto/English/Nederlands/ Montreal (QC) H3C 3J7 | Francais (phase experimentale) ------------------------------------------------------------------------------- ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From hinsenk@ere.umontreal.ca Mon Feb 26 21:27:05 1996 From: hinsenk@ere.umontreal.ca (Konrad HINSEN) Date: Mon, 26 Feb 1996 16:27:05 -0500 Subject: [PYTHON MATRIX-SIG] RubberIndex In-Reply-To: <199602261735.MAA06305@python.org> (message from Chris Chase S1A on Mon, 26 Feb 1996 12:41:00 -0500) Message-ID: <199602262127.QAA24193@cyclone.ERE.UMontreal.CA> You will never be able to use the current array implementation everywhere that a list is used because array has different definitions for operators (e.g. *, +) and array can not support 'del' without True, but that is no reason to introduce additional incompatibilities. "must"? Why? As far as I can tell the hierarchical indexing for arrays was a convenience for those used to working lists, because it duplicates behavior already provided by a[0,...] indexing. Historically it was the other way round, but all that doesn't matter. The question is what behaviour is the most useful. both selection and assignment. This specific solution for flattened indexing is supported in all the array languages that I have had exposure to (e.g. Matlab, IDL, Tela, Yorick). To me this is more important than the hierarchical indexing scheme so that an array looks like a list (but does not act like a list for other purposes). The array languages I am familiar with (APL and J) do not permit this. I suppose that the languages you mention support flattened access because Fortran programmers are used to it, not because it is important in real applications. I still think that a structural modification like flattening should be explicit, not implied. And let's not forget that our goal is not to write a Matlab/IDL/whatever clone, but to provide arrays for Python. We should stick as close as possible to established Python habits, which is why I insist so much on the analogy with nested lists. As an alternative, a 'flat' (or 'flattened') array attribute could be supported. Why? What's wrong with ravel(a)? You can index its result if you want. An aside: I know that Konrad Hinsen prefers functions to methods and attributes so that the same code will work for both scalars and arrays. But this is similar to the list analogy above. scalars will never work in all array code because scalars are neither sequences nor dictionary types. The first time one attempts to subscript a scalar an error will be produced. Certainly we don't want subscription to be True, but that makes sense. Scalars are considered rank-0 arrays, and indexing rank-0 arrays makes no sense. So the behaviour is consistent. And again, the fact that something can't be done perfectly is no argument for not doing it as good as possible. My point about functions vs. methods is that many algorithms can reasonably applied to arrays of all ranks, including rank 0. Of course such algorithms cannot include indexing. But all the operations that are rank-independent should be implemented in such a way that they accept both arrays and scalars. a function rather than a method. Rather than jump onto the everything is a function bandwagon we may need to rethink the relationship between scalars and arrays. What is the entire scope of code where scalars and arrays need to be interchangeable? Everything except indexing. Some care must be taken with functions that require an axis specification (like add.reduce). Obviously a rank-0 array does not have axes, but add.reduce(x) still makes sense if x is a scalar. Ideally, a call to add.reduce with an explicit axis specification should raise an exception if given a scalar, but do the right thing with the default axis. BTW, there are more arguments not to implement array operations as methods, as I have outlined in my proposal. ------------------------------------------------------------------------------- Konrad Hinsen | E-Mail: hinsenk@ere.umontreal.ca Departement de chimie | Tel.: +1-514-343-6111 ext. 3953 Universite de Montreal | Fax: +1-514-343-7586 C.P. 6128, succ. Centre-Ville | Deutsch/Esperanto/English/Nederlands/ Montreal (QC) H3C 3J7 | Francais (phase experimentale) ------------------------------------------------------------------------------- ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From hinsenk@ere.umontreal.ca Mon Feb 26 21:34:55 1996 From: hinsenk@ere.umontreal.ca (Konrad HINSEN) Date: Mon, 26 Feb 1996 16:34:55 -0500 Subject: [PYTHON MATRIX-SIG] Functions and names In-Reply-To: <199602261922.OAA18677@cyclone.ERE.UMontreal.CA> (message from Chris Chase S1A on Mon, 26 Feb 1996 14:30:43 -0500) Message-ID: <199602262134.QAA24610@cyclone.ERE.UMontreal.CA> One only needs slices. The insertion functionality I suggested only requires one to specify the starting position. Normally, the user would have to compute the slice ending position and check to see if the input array will fit (if not the input array could be sliced to fit). I was suggesting a insertion function that would handle options of wrap-around versus truncation versus just error signaling. It is very convenient for manipulating block regions of arrays (like placing one image inside another image). Perhaps a better name than "insert" would be "place"? I see. That looks indeed useful; I remember having once written an APL function to do more or less the same. It is however not so easy to find a nice syntax for this. Right now all operations that change array elements are assignments, and I'd like to keep it that way. One possibility would be to allow somthing like this: a = array([1,2,3,4,5,6]) b = array([0,0]) a[2] = b with the result a == array([1,2,0,0,5,6]) I just don't like the error checking that gets lost in the process. Maybe the alternative a[2::] = b (assignment of a vector that is too short) would be a lesser evil. ------------------------------------------------------------------------------- Konrad Hinsen | E-Mail: hinsenk@ere.umontreal.ca Departement de chimie | Tel.: +1-514-343-6111 ext. 3953 Universite de Montreal | Fax: +1-514-343-7586 C.P. 6128, succ. Centre-Ville | Deutsch/Esperanto/English/Nederlands/ Montreal (QC) H3C 3J7 | Francais (phase experimentale) ------------------------------------------------------------------------------- ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From da@maigret.cog.brown.edu Wed Feb 28 15:54:59 1996 From: da@maigret.cog.brown.edu (David Ascher) Date: Wed, 28 Feb 1996 10:54:59 -0500 (EST) Subject: [PYTHON MATRIX-SIG] fft's Message-ID: <199602281554.KAA12674@maigret> I've been working on python interface to the FFT routines in the complib.sgimath library (since those are the ones I have optimized binaries for). This has been educational, but it has raised several questions which I can't answer. First I'll talk about a specific one related to 2-d and 3-d FFT's, then ask a few questions about interfacing to fortran in general. The complib.sgimath (based on stuff from netlib) has a set of fft calls, 1, 2 and 3-d. Things are working fine in 1-D, but I have to admit that my grokking of 2-D and 3-D FFT's is rather limited. When doing a 1-d fft, the output array is bigger than the input array by enough to fit an aligned complex. Fine. The 2-d fft calls in the fft library I'm using have the following syntax: int dfft2du( int job, int n1, int n2, double *sequence, int lda, double *workspace) job specifies direction (forward vs. backward) sequence is where the data is stored (both before and after the fft) workspace is the sines & cosines and the factorization of n1 and n2. All that is fine. n1 and n2 are defined as follows: N1 - INTEGER. On entry, N1 specifies the number of elements in the first dimension of the sequence. Unchanged on exit. N2 - INTEGER. On entry, N2 specifies the number of elements in the second dimension of the sequence. Unchanged on exit. and LDA is defined as: LDA - INTEGER. On entry, LDA specifies the first dimension of the array SEQUENCE as declared in the calling (sub)program. LDA must be at least max( 1, 2*((N1+2)/2) ). Unchanged on exit. This is where I get lost. I assumed that if I had a n1 x n2 array to start with, I'd have an array which would be slightly bigger in, say, n1 and n2 (enough to store a complex?). But this LDA parameter is completely confusing me. I have similar problems in 3d of course, where LDA is replaced by ld1 and ld2. I suspect if I grok LDA i'll grok LD1 and LD2. =) I realize that fortran and C/Python index into arrays differently (row vs. column first). What is a good way of dealing with this? This brings to mind a different issue regarding interfaces to existing libraries. The library I'm using returns rank 1 arrays, which are really rank 1, 2, or 3 but designed for antiquated languages. Should I do the conversion to the appropriate rank (e.g. the assignment of shape) in a .py wrapper, so that the library module itself is as compatible w/ the fortran and C interfaces, or should I make the library python-friendly, and have it return appropriately sized arrays? Thanks for reading this far. --david ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From dubois1@llnl.gov Wed Feb 28 17:19:27 1996 From: dubois1@llnl.gov (Paul. Dubois) Date: Wed, 28 Feb 1996 09:19:27 -0800 Subject: [PYTHON MATRIX-SIG] fft's References: <199602281554.KAA12674@maigret> Message-ID: <31348E9F.8EC@llnl.gov> Most routines of this type written in C or Fortran allow for the possibility that an array has been declared of a larger size than it actually is. For example #define N 32 #define M 20 int n, m, c[N]{M]; n = something m = something ...now you want to do something with c and as far as you are concerned it is n by m. In C, the arrays are stored so that in considering c[i][j] and c[i][j+1], they are 1 storage unit apart, but c[i][j] and c[i+1][j] are M apart. For Fortran, it works the other way, and it is c(i,j) and c(i, j+1) that are N apart. When you pass such a beast to a general routine, therefore, it is necessary to tell it N in Fortran or M in C. So in your fft what they want for lda is M. However, when working with an object-oriented language, this kind of partially full matrix just doesn't arise very often, and in most of your work the python array x with shape (n,m) will be passed to the library fft routine with lda = m. Hope this helps. -- Paul F. Dubois, L-472 (510)-422-5426 Lawrence Livermore National Laboratory FAX (510)-423-9969 Livermore, CA 94550 dubois1@llnl.gov Consulting: PaulDubois@aol.com ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From hinsenk@ere.umontreal.ca Wed Feb 28 22:02:55 1996 From: hinsenk@ere.umontreal.ca (Konrad HINSEN) Date: Wed, 28 Feb 1996 17:02:55 -0500 Subject: [PYTHON MATRIX-SIG] fft's In-Reply-To: <199602281554.KAA12674@maigret> (da@maigret.cog.brown.edu) Message-ID: <199602282202.RAA24236@cyclone.ERE.UMontreal.CA> > n1 and n2 are defined as follows: > > N1 - INTEGER. > On entry, N1 specifies the number of elements in the > first dimension of the sequence. > Unchanged on exit. > N2 - INTEGER. > On entry, N2 specifies the number of elements in the > second dimension of the sequence. > Unchanged on exit. So these describe the number of data points... > and LDA is defined as: > > LDA - INTEGER. > On entry, LDA specifies the first dimension of the > array SEQUENCE as declared in the calling (sub)program. > LDA must be at least max( 1, 2*((N1+2)/2) ). > Unchanged on exit. and this the physical size of the grid. The reason for having both is that you might want to dimension some arrays larger than necessary to avoid recompilation for every different value of N1/N2. Remember, we are talking about a library designed for a language without dynamic allocation! In the case of Python, you would most certainly allocate an array of the correct size and LDA would be equal to N1. You will find these "LDA" parameters everywhere in Fortran libraries. > I realize that fortran and C/Python index into arrays differently (row > vs. column first). What is a good way of dealing with this? Your interface should make sense to Python users. They won't see the Fortran source code anyway and probably don't even care whether it's Fortran. > really rank 1, 2, or 3 but designed for antiquated languages. Should I > do the conversion to the appropriate rank (e.g. the assignment of shape) > in a .py wrapper, so that the library module itself is as compatible w/ > the fortran and C interfaces, or should I make the library > python-friendly, and have it return appropriately sized arrays? It matters only that the function that end users call uses correctly shaped arrays. Personally, I'd use the correct shape also for the low-level interface, because there is no possible use for 1d access. But if you prefer otherwise, I don't care. ------------------------------------------------------------------------------- Konrad Hinsen | E-Mail: hinsenk@ere.umontreal.ca Departement de chimie | Tel.: +1-514-343-6111 ext. 3953 Universite de Montreal | Fax: +1-514-343-7586 C.P. 6128, succ. Centre-Ville | Deutsch/Esperanto/English/Nederlands/ Montreal (QC) H3C 3J7 | Francais (phase experimentale) ------------------------------------------------------------------------------- ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From guido@CNRI.Reston.VA.US Wed Feb 28 22:36:03 1996 From: guido@CNRI.Reston.VA.US (Guido van Rossum) Date: Wed, 28 Feb 1996 17:36:03 -0500 Subject: [PYTHON MATRIX-SIG] fft's In-Reply-To: Your message of "Wed, 28 Feb 1996 17:02:55 EST." <199602282202.RAA24236@cyclone.ERE.UMontreal.CA> References: <199602282202.RAA24236@cyclone.ERE.UMontreal.CA> Message-ID: <199602282236.RAA23873@monty> > > LDA - INTEGER. > > On entry, LDA specifies the first dimension of the > > array SEQUENCE as declared in the calling (sub)program. > > LDA must be at least max( 1, 2*((N1+2)/2) ). > > Unchanged on exit. > > and this the physical size of the grid. The reason for having both > is that you might want to dimension some arrays larger than necessary > to avoid recompilation for every different value of N1/N2. This is what I thought, except that the formula for LDA's minimum size seems to imply that it must be at least *1 larger* than N1? Perhaps the library module uses this as temp space? --Guido van Rossum URL: ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From hinsenk@ere.umontreal.ca Wed Feb 28 22:41:18 1996 From: hinsenk@ere.umontreal.ca (Konrad HINSEN) Date: Wed, 28 Feb 1996 17:41:18 -0500 Subject: [PYTHON MATRIX-SIG] fft's In-Reply-To: <199602282236.RAA23873@monty> (message from Guido van Rossum on Wed, 28 Feb 1996 17:36:03 -0500) Message-ID: <199602282241.RAA26182@cyclone.ERE.UMontreal.CA> > > > LDA - INTEGER. > > > On entry, LDA specifies the first dimension of the > > > array SEQUENCE as declared in the calling (sub)program. > > > LDA must be at least max( 1, 2*((N1+2)/2) ). > > > Unchanged on exit. > > > > and this the physical size of the grid. The reason for having both > > is that you might want to dimension some arrays larger than necessary > > to avoid recompilation for every different value of N1/N2. > > This is what I thought, except that the formula for LDA's minimum size > seems to imply that it must be at least *1 larger* than N1? Perhaps > the library module uses this as temp space? The minimum value is nonsense. The smallest reasonable value for N1 is 1, which makes 2*((N1+2)/2) = 2, which is always larger than 1. ------------------------------------------------------------------------------- Konrad Hinsen | E-Mail: hinsenk@ere.umontreal.ca Departement de chimie | Tel.: +1-514-343-6111 ext. 3953 Universite de Montreal | Fax: +1-514-343-7586 C.P. 6128, succ. Centre-Ville | Deutsch/Esperanto/English/Nederlands/ Montreal (QC) H3C 3J7 | Francais (phase experimentale) ------------------------------------------------------------------------------- ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org =================