From boyle@pcmdi.llnl.gov Mon Jun 2 16:47:33 1997 From: boyle@pcmdi.llnl.gov (James Boyle) Date: Mon, 2 Jun 1997 08:47:33 -0700 (PDT) Subject: [PYTHON MATRIX-SIG] handling missing data values in NumPy Message-ID: <199706021547.IAA17052@cobra.llnl.gov> I often deal with time sequences of 2D arrays that have some values missing at random times. I want to obtain an average of these sequences along the time axis taking into account the missing values. I may just be dense but I do not see any slick way to accomplish this task without writing an extension like the add.reduce but checking each element for a missing value. Does anyone have any insight to this problem or is an extension the solution? Thanks for any information. Jim Boyle _______________ MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org _______________ From hugunin@mit.edu Mon Jun 2 18:18:35 1997 From: hugunin@mit.edu (Jim Hugunin) Date: Mon, 2 Jun 1997 13:18:35 -0400 Subject: [PYTHON MATRIX-SIG] handling missing data values in NumPy Message-ID: <9706021719.AA03501@goldilocks> > I often deal with time sequences of 2D arrays that have > some values missing at random times. I want to obtain > an average of these sequences along the time axis > taking into account the missing values. I can tell you how I solve the problem in my own code. No guarantees that this is appropriate for you though... >>> from Numeric import * >>> data = array([1,2,666,4,5]) >>> valid = array([1,1, 0,1,1]) >>> print sum(data*valid)/sum(valid) 3 If you do a lot of this sort of thing, you could encapsulate this way of representing arrays with missing values into a new python object, but that's possibly overkill. -Jim _______________ MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org _______________ From Daniel.Michelson@smhi.se Tue Jun 3 14:37:08 1997 From: Daniel.Michelson@smhi.se (Daniel.Michelson) Date: Tue, 3 Jun 1997 15:37:08 +0200 Subject: [PYTHON MATRIX-SIG] HDF Message-ID: <9706031337.AA20283@ax004.smhi.se> Greetings! I have a vague recollection of someone mentioning that the LLNL was = working=20 on a Python interface to the HDF library. Am I right? If so, what shape = is=20 it in and is it targeted for public release? curiously, -daniel +------------------+--------------------------------------------------+ | | Daniel B. Michelson | | /\ |\ /| | | | | Swedish Meteorological and Hydrological Institute| | \ | \/ | |__| | | S-601 76 Norrk=F6ping, Sweden = | | \ | | | | | | Telephone: +46 11 - 15 84 94 | | \/ | | | | | | Telefax: +46 11 - 17 02 07 | | | Internet: Daniel.Michelson@smhi.se | +------------------+--------------------------------------------------+ _______________ MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org _______________ From dubois1@llnl.gov Tue Jun 3 17:10:21 1997 From: dubois1@llnl.gov (Paul F. Dubois) Date: Tue, 3 Jun 1997 09:10:21 -0700 Subject: [PYTHON MATRIX-SIG] HDF Message-ID: <9706031613.AA23039@icf.llnl.gov.llnl.gov> We have a basic interface to the PDB file format. This is a format that is part of the PACT system developed by Stuart Brown of LLNL. This interface will be released within the next two weeks; we in fact were just doing the paperwork yesterday. Unfortunately, we have not done an interface to HDF as we do not use it. See coral.llnl.gov/pub/users/pact to get PACT. It is free. PDB is a portable, self-defining file. It is significantly fast. It allows "directories" inside the file and can store entire pointered C structures in a morphology-preserving way. There is both a C and Fortran interface, and a nice document (which you have to get separately from the same site). It has had extensive use at LLNL for many years. See S. A. Brown et. al, "Software for Portable Scientific Data Management", Computers in Physics 7(3), p 304 (1993) for an article comparing HDF, netCDF, and PDB, written by people connected with each project. See S.A. Brown, P. F. Dubois, and D. H. Munro, "Creating and Using PDB Files", Computers in Physics 9(2), p 173 (1995) for a tutorial. Our Python interface uses the Numerical Python package to turn data in the files into NumPy arrays. The interface should be considered a beta when we release it. It will be part of an "LLNL" release which will help people create python with NumPy, Gist, PDB, Tk, and C++ support. (The C++ support is not even an alpha, but there is at least an option in configure to use a C++ main program and add modules written in C++). Our work has gone more slowly than we had hoped due to an oversubscribed work force, but it is coming along. Please be patient. Paul Dubois Project Leader, Computer Science X-Division, LLNL ---------- From: Daniel.Michelson To: matrix-sig@python.org Subject: [PYTHON MATRIX-SIG] HDF Date: Tuesday, June 03, 1997 6:37 AM Greetings! I have a vague recollection of someone mentioning that the LLNL was working on a Python interface to the HDF library. Am I right? If so, what shape is it in and is it targeted for public release? curiously, -daniel +------------------+--------------------------------------------------+ | | Daniel B. Michelson | | /\ |\ /| | | | | Swedish Meteorological and Hydrological Institute| | \ | \/ | |__| | | S-601 76 Norrkvping, Sweden | | \ | | | | | | Telephone: +46 11 - 15 84 94 | | \/ | | | | | | Telefax: +46 11 - 17 02 07 | | | Internet: Daniel.Michelson@smhi.se | +------------------+--------------------------------------------------+ _______________ MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org _______________ ---------- _______________ MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org _______________ From furnish@acl.lanl.gov Tue Jun 3 17:23:30 1997 From: furnish@acl.lanl.gov (Geoffrey Furnish) Date: Tue, 3 Jun 1997 10:23:30 -0600 (MDT) Subject: [PYTHON MATRIX-SIG] plotting software In-Reply-To: <199705262001.NAA23212@curlew.geog.ubc.ca> References: <199705211242.IAA08520@jigster> <199705220639.BAA05980@tillamook-sharp.eaton.net> <199705262001.NAA23212@curlew.geog.ubc.ca> Message-ID: <199706031623.KAA03005@steam.acl.lanl.gov> Phil Austin writes: > >>>>> "JWE" == John W Eaton writes: > JWE> A similar library that is distributed under the GPL is > JWE> plplot, but I'm not sure that it is being actively maintained > JWE> these days. > > Lack of a good free plotting package is a sticking point for several > numerically-oriented interpreted languages. There has been some > discussion of this on the Numeric Python mailing list as well, and > perhaps a group effort spanning Octave, Python, Perl data language, > etc. could produce something. We'd like to see the most portable > possible solution, and the Tk driver in Plplot shows promise in that > direction, although the move of Geoff Furnish, the Plplot maintainer, > from Livermore to Los Alamos may have derailed that (there was also > some talk several months about about merging Plplot with gist, a > free Livermore graphics package that runs on X under Yorick or > Python). One person on the Python list is writing plotting software > in Java to run under Python (see http://estel.uindy.edu/PESSci/ph280/) I can only speak for PLplot. It has been mentioned here several times in the last few months. I have twice composed responses and then deleted them instead of sending them. I'll try again now. Bottom line: PLplot's Python support does and will continue to exist, and will even continue to improve. I am using Python and PLplot in my work, past, present, and future, and PLplot will continue to improve as a consequence. I have been extremely reticient to chime in with info about PLplot on the Python Matrix SIG over the last few months specifically because I am absolutely sick to death of Pythonians publicly maligning PLplot for its allegedly poor autoconf support, whilst never bothering to carefully document their problems and provide them to the PLplot mailing list where the PLplot maintainer (me) or other PLplot hotshots (of whom there are several), might have a fighting chance of helping them. The list of the guilty here is long and grievous, and I am really sick of it. I am especially sick of the misplaced condemnation. Writing autoconf support is *hard* and the Python community seems to be somewhat unique among autoconf users in terms of their oblivion to the issues, apparently specifically b/c Python's "auto"-conf doesn't "automatically" configure very much at all (it checks the system OS characteristics in order to configure the supported Python modules which interact with the OS, but it does not do any meaningful search for auxilliary packages a user might have installed). With Python you have to manually configure practically everything, which is /much/ more burdensome on the user than is PLplot. The problem is that autoconf isn't perfect. It is at best an asymptotically converging system. For 99% of all installed computers systems, autoconf does a fantastic job. For 1% of the installed computers, the sysadmins have succeeded in so incomparably screwing things up that autoconf can't sort it out, and the users on these systems invariably blame the package maintainer. For instance, at my previous place of employment, the installation of X on a particular computer that I used a lot, was so perniciously mishandled that no autoconf version from 2.4 through 2.10 could /automatically/ recognize that the system had X. Fortunately autoconf suports user overides, and I was abble to pass configure the --x-includes=/where/they/actually/were flag, to overcome this annoyance. PLplot has drawn an extremely disproportionate amount of criticism from one community--the Python community--for its allegedly insufferable configuration support. I will be the first to admit that PLplot's autoconf support has required a lot of attention. Many many bugs were fixed over the summer/fall '96 timeframe. There were PLplot snapshots to provide many of these improvemtns to external PLplot users. I do still have additional fixes which have not yet been propagated to an externally accessible snapshot. There will be more PLplot snapshots, and when they show up, people will be able to benefit from the improvements which have been made. I cannot schedule these. I am not paid to hack PLplot. I am paid to do science, and I hack PLplot only in so far as it is undeniably justifiable in the pursuit of my mandated job functions. As with all GPL'd software, it is possible to obtain professional support for PLplot. Besides the autoconf issues with PLplot, Pythonians have also complained regularly about uneven support for a dynamically loadable Python extension module for PLplot. Again, it is no trouble to compile PLplot with -fpic or the equivalent for your compiler. There are, however, major issues to be addressed in terms of the total python integration picture. I certainly view the Python module import system as very primitive, and nowhere near as sophisticated as that seen in certain other scripting environments, and have the very strong sensation that this is poorly appreciated in the Python community. Python extensions composed of /both/ Python source /and/ compiled extensions, are not well served by simply "import"-ing a dynloadable module, since it may be necessary to simultaneously update the PYTHONPATH to find the *.py files that go with the .so. Support for this kind of thing is very poor in Python. The PLplot approach to dealing with this is to favor static builds. Moreover, it is also very hard to support dynloading of PLplot in combination with its Tk support, because of the poor design of the tkinter module. I have found it exceedingly hard to build a Python which seemlessly supports dynloadable tkmodule with selective inclusion of Tcl extension modules. The -DWITH_APPINIT thing is woefully inadequate in the face of the sort of freewheeling dynloadable extension style that Python seems to want to encourage. The last time these issues were discussed in this forum, the moderator told us to quit talking about it, even though the discussion followed directly from issues relating to numeric plotting extensions, which are surely germane to this list, and also even though the NumPy extension itself suffers from many of the same issues. I am flatly exasperated by the Python community with respect to the prevailing sentiments about PLplot, and the deafening silence that has followed is a direct reflection of this exasperation (rather than an indication that PLplot is currently unsupported). This message is very much an attempt to bring some closure to the matter in this forum. I do not intend to go out of my way to publicize PLplot's Python support to the Python community in the future. I will publicize PLplot's Python support to the PLplot community. As a final reference to Phil's comments about a multi-script-language plotting library, let me just say the following: PLplot has long aimed to provide the same plotting library across every interesting programming architecture. This started with support for multiple compiled language interfaces (fortran, C and C++), and has proceeded also to cover multiple script languages (Tcl, Scheme, Python). Perl-dl hasn't been done yet, but it surely will someday. Same for Java--it's not on the calendar, but it is surely going to happen one of these days. PLplot is already bound to packages like Ocatve and Rlab, and similar such support from other packages is inevitable. Bottom line, if you are a PLplot user, and you start working in a new environment one day, hopefully you will find that PLplot is already there, or can be taken there easily. This has been an enduring characteristic of PLplot for many years, and will continue irrespective of the prevailing mood of the Python community. Cheers, -- Geoffrey Furnish email: furnish@lanl.gov LANL CIC-19 POOMA/RadTran phone: 505-665-4529 fax: 505-665-7880 "Here are your ball-peen hammers. Now go malleate some heads!" -Jim Morel _______________ MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org _______________ From R.Hooft@EuroMail.com Wed Jun 4 07:58:04 1997 From: R.Hooft@EuroMail.com (Rob Hooft) Date: Wed, 4 Jun 1997 08:58:04 +0200 (MET DST) Subject: [PYTHON MATRIX-SIG] HDF In-Reply-To: <9706031613.AA23039@icf.llnl.gov.llnl.gov> References: <9706031613.AA23039@icf.llnl.gov.llnl.gov> Message-ID: <199706040658.IAA22790@nu> >>>>> "PFD" == Paul F Dubois writes: PFD> We have a basic interface to the PDB file format. That is nice. Did you know that Konrad Hinsen also has an interface to a PDB file format? But that one is a bit more specialized: the _P_rotein _D_ata _B_ank file can only be used to store 3D structures of biological macromolecules... And this database format was already published in 1977! @Article{bernstein, author = "F. C. Bernstein and T. F. Koetzle and G. J. B. Williams and E. F. {Meyer Jr} and M. D. Brice and J. R. Rodgers and O. Kennard and T. Shimanouchi and M. Tasumi", title = "The protein data bank: a computer-based archival file for macro-molecular structures", journal = "J. Mol. Biol.", year = "1977", volume = "112", pages = "535--542", } Regards, -- ===== R.Hooft@EuroMail.com http://www.Sander.EMBL-Heidelberg.DE/rob/ == ==== In need of protein modeling? http://www.Sander.EMBL-Heidelberg.DE/whatif/ Validation of protein structures? http://biotech.EMBL-Heidelberg.DE:8400/ ==== == PGPid 0xFA19277D == Use Linux! Free Software Rules The World! ============= _______________ MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org _______________ From hinsen@ibs.ibs.fr Wed Jun 4 10:10:03 1997 From: hinsen@ibs.ibs.fr (Konrad Hinsen) Date: Wed, 4 Jun 1997 11:10:03 +0200 Subject: [PYTHON MATRIX-SIG] HDF In-Reply-To: <9706031613.AA23039@icf.llnl.gov.llnl.gov> (dubois1@llnl.gov) Message-ID: <199706040910.LAA24716@lmspc1.ibs.fr> > See coral.llnl.gov/pub/users/pact to get PACT. It is free. PDB is a portable, And don't forget to use FTP for that - if you try http, you'll be asked for a password! PACT looks quite interesting. Do you plan to do Python interfaces for the whole package, or just for PDB? BTW, my major complaint about PDB is its name. In the biology community, one of the first acronyms that everybody learns is PDB - the Protein Data Bank (see http://www.pdb.bnl.gov/ for details). The PDB file format is the most heavily used (and abused) file format in computational chemistry, and many people have set up their Web browsers to feed ".pdb" to a PDB viewer. I doubt anyone in my field would want to use another package called PDB; the confusion would be too much. -- ------------------------------------------------------------------------------- Konrad Hinsen | E-Mail: hinsen@ibs.ibs.fr Laboratoire de Dynamique Moleculaire | Tel.: +33-4.76.88.99.28 Institut de Biologie Structurale | Fax: +33-4.76.88.54.94 41, av. des Martyrs | Deutsch/Esperanto/English/ 38027 Grenoble Cedex 1, France | Nederlands/Francais ------------------------------------------------------------------------------- _______________ MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org _______________ From hinsen@ibs.ibs.fr Wed Jun 4 10:20:15 1997 From: hinsen@ibs.ibs.fr (Konrad Hinsen) Date: Wed, 4 Jun 1997 11:20:15 +0200 Subject: [PYTHON MATRIX-SIG] plotting software In-Reply-To: <199706031623.KAA03005@steam.acl.lanl.gov> (message from Geoffrey Furnish on Tue, 3 Jun 1997 10:23:30 -0600 (MDT)) Message-ID: <199706040920.LAA24754@lmspc1.ibs.fr> > Python extensions composed of /both/ Python source /and/ compiled > extensions, are not well served by simply "import"-ing a dynloadable > module, since it may be necessary to simultaneously update the > PYTHONPATH to find the *.py files that go with the .so. Support for > this kind of thing is very poor in Python. The PLplot approach to This looks rather easy - just import sys and append something to sys.path. Easily doable even in a C extension. But I am not at all sure that it's a good idea to change the search path dynamically, because it could cause unpleasant surprises. For example, a module in the new path could have the same name as a module somewhere else in the path. I'd rather have such name clashed static so that I can find a workaround. That aside, I do agree that Python's support for installing and configuring nontrivial extension packages is not perfect. What I'd like to see is a package system like ni to become standard (i.e. built into the interpreter and fast), plus a package installation script to automate whatever has to be done to get the path set correctly. But in general, the Python installation procedure seems OK to me. What it does automatically works, and what it can't do automatically it doesn't try to do. I don't mind a few minutes of manual configuration (mostly editing Modules/Setup), it's much better than trying to figure out why some fancy automatic configuration doesn't work. -- ------------------------------------------------------------------------------- Konrad Hinsen | E-Mail: hinsen@ibs.ibs.fr Laboratoire de Dynamique Moleculaire | Tel.: +33-4.76.88.99.28 Institut de Biologie Structurale | Fax: +33-4.76.88.54.94 41, av. des Martyrs | Deutsch/Esperanto/English/ 38027 Grenoble Cedex 1, France | Nederlands/Francais ------------------------------------------------------------------------------- _______________ MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org _______________ From dars@soton.ac.uk Wed Jun 4 10:45:24 1997 From: dars@soton.ac.uk (Dave Stinchcombe) Date: Wed, 4 Jun 1997 10:45:24 +0100 (BST) Subject: [PYTHON MATRIX-SIG] Singular Values Message-ID: <199706040945.KAA13667@oak.sucs.soton.ac.uk> Dear Folks, can any one tell me the reason why the LinearAlgerba.singular_value_decomposition routine returns U,S,W^T, where the matrix being decomposed is U*S*W^T, when it seems more conventional to return U,S,W ? Yours gratefully Dave Stinchcombe _______________ MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org _______________ From hinsen@ibs.ibs.fr Wed Jun 4 11:35:21 1997 From: hinsen@ibs.ibs.fr (Konrad Hinsen) Date: Wed, 4 Jun 1997 12:35:21 +0200 Subject: [PYTHON MATRIX-SIG] Singular Values In-Reply-To: <199706040945.KAA13667@oak.sucs.soton.ac.uk> (dars@soton.ac.uk) Message-ID: <199706041035.MAA25024@lmspc1.ibs.fr> > can any one tell me the reason why the > LinearAlgerba.singular_value_decomposition routine returns U,S,W^T, where > the matrix being decomposed is U*S*W^T, when it seems more > conventional to return U,S,W ? The convention with respect to SVD is to return U, S, W^T, because the product of these matrices is the original matrix. Matlab, for example, does the same. -- ------------------------------------------------------------------------------- Konrad Hinsen | E-Mail: hinsen@ibs.ibs.fr Laboratoire de Dynamique Moleculaire | Tel.: +33-4.76.88.99.28 Institut de Biologie Structurale | Fax: +33-4.76.88.54.94 41, av. des Martyrs | Deutsch/Esperanto/English/ 38027 Grenoble Cedex 1, France | Nederlands/Francais ------------------------------------------------------------------------------- _______________ MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org _______________ From dubois1@llnl.gov Wed Jun 4 16:17:11 1997 From: dubois1@llnl.gov (Paul F. Dubois) Date: Wed, 4 Jun 1997 08:17:11 -0700 Subject: [PYTHON MATRIX-SIG] HDF Message-ID: <9706041520.AA07877@icf.llnl.gov.llnl.gov> ---------- > From: Konrad Hinsen > To: dubois1@llnl.gov > Cc: matrix-sig@python.org > Subject: Re: [PYTHON MATRIX-SIG] HDF > Date: Wednesday, June 04, 1997 2:10 AM > > > See coral.llnl.gov/pub/users/pact to get PACT. It is free. PDB is a portable, > > And don't forget to use FTP for that - if you try http, you'll be asked > for a password! > > PACT looks quite interesting. Do you plan to do Python interfaces for the > whole package, or just for PDB? > One of the other people at the Lab is working on a Python interface to the the graphics part of PACT. I do not know the status of that. _______________ MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org _______________ From amullhau@ix.netcom.com Thu Jun 5 01:18:07 1997 From: amullhau@ix.netcom.com (Andrew P. Mullhaupt) Date: Wed, 04 Jun 1997 20:18:07 -0400 Subject: [PYTHON MATRIX-SIG] Singular Values In-Reply-To: <199706041035.MAA25024@lmspc1.ibs.fr> References: <199706040945.KAA13667@oak.sucs.soton.ac.uk> Message-ID: <3.0.1.32.19970604201807.006c2424@popd.netcruiser> At 12:35 PM 6/4/97 +0200, Konrad Hinsen wrote: >The convention with respect to SVD is to return U, S, W^T, because >the product of these matrices is the original matrix. Matlab, for >example, does the same. This isn't really that much of a convention as a concession to languages such as C and Fortran which do not easily do the transpose. S, for example returns V, not V^T, as do some APL versions. The U and V matrices are normally formed by accumulating elementary orthogonal transformations, and it is just as easy to form the matrices or their transposes. Later, Andrew Mullhaupt _______________ MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org _______________ From furnish@acl.lanl.gov Thu Jun 5 15:32:13 1997 From: furnish@acl.lanl.gov (Geoffrey Furnish) Date: Thu, 5 Jun 1997 08:32:13 -0600 (MDT) Subject: [PYTHON MATRIX-SIG] handling missing data values in NumPy In-Reply-To: <199706021547.IAA17052@cobra.llnl.gov> References: <199706021547.IAA17052@cobra.llnl.gov> Message-ID: <199706051432.IAA02549@steam.acl.lanl.gov> James Boyle writes: > I often deal with time sequences of 2D arrays that have > some values missing at random times. I want to obtain > an average of these sequences along the time axis > taking into account the missing values. > > I may just be dense but I do not see any slick way to > accomplish this task without writing an extension > like the add.reduce but checking each element for > a missing value. > > Does anyone have any insight to this problem or is an > extension the solution? > > Thanks for any information. I suggest using two arrays. One with abscissa (time) values, and one with ordinate (the real data) values. You could think of missing data on regular intervals as being just another form of data on irregular intervals. -- Geoffrey Furnish email: furnish@lanl.gov LANL CIC-19 POOMA/RadTran phone: 505-665-4529 fax: 505-665-7880 "Here are your ball-peen hammers. Now go malleate some heads!" -Jim Morel _______________ MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org _______________ From dars@soton.ac.uk Thu Jun 5 20:50:22 1997 From: dars@soton.ac.uk (Dave Stinchcombe) Date: Thu, 5 Jun 1997 20:50:22 +0100 (BST) Subject: [PYTHON MATRIX-SIG] Singular Values In-Reply-To: <199706041035.MAA25024@lmspc1.ibs.fr> from "Konrad Hinsen" at Jun 4, 97 12:35:21 pm Message-ID: <199706051950.UAA28284@oak.sucs.soton.ac.uk> > > > can any one tell me the reason why the > > LinearAlgerba.singular_value_decomposition routine returns U,S,W^T, where > > the matrix being decomposed is U*S*W^T, when it seems more > > conventional to return U,S,W ? > > The convention with respect to SVD is to return U, S, W^T, because > the product of these matrices is the original matrix. Matlab, for > example, does the same. I don't wish to sound ungratefull, as at the moment my work is completely dependant on NumPy, and his friends. But Matlab , Maple and Octave all return U,V,W where the matrix is U*V*W^T. The reason I make note is not to request a change in LinearAlgebra, but simply to discover the logic behind the current choice. The reason for wanting U, W returned is for ease of handling by other (in my case C) routines, but there is no great hardship in adding another line of python to form the transpose. Matlab online help: >> help svd SVD Singular value decomposition. [U,S,V] = SVD(X) produces a diagonal matrix S, of the same dimension as X and with nonnegative diagonal elements in decreasing order, and unitary matrices U and V so that X = U*S*V'. Maple online help: Svd(X,U,V) returns the singular values and the left and right singular vectors in U and V respectively. The singular vectors together with the singular values satisfy U'XV = D where U' is the transpose of U and U is n by n, V is p by p, X is n by p, and D is n by p where D[i,i] is/are the singular value/values of X. I hope this letter does not appear rude, it is not meant to be, as I'm very gratefull for all the work done by this group, indeed currently NumPy is easily the most effective language I have to hand and work would be an order of magnitude more painfull without it. I'm just trying to understand something that niggles me. Yours Dave Stinchcombe _______________ MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org _______________ From drh@cse.unl.edu Thu Jun 5 22:03:44 1997 From: drh@cse.unl.edu (Doug Heisterkamp) Date: Thu, 5 Jun 1997 16:03:44 -0500 (CDT) Subject: [PYTHON MATRIX-SIG] Singular Values In-Reply-To: <199706051950.UAA28284@oak.sucs.soton.ac.uk> from "Dave Stinchcombe" at Jun 5, 97 08:50:22 pm Message-ID: <199706052103.QAA07339@cse.unl.edu> > > > > > > can any one tell me the reason why the > > > LinearAlgerba.singular_value_decomposition routine returns U,S,W^T, where > > > the matrix being decomposed is U*S*W^T, when it seems more > > > conventional to return U,S,W ? > > The underlying LAPACK routines return the transpose. Note: if you are using a complex matrix, the conjugate transpose is returned. Doug Heisterkamp drh@cse.unl.edu _______________ MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org _______________ From hinsen@ibs.ibs.fr Fri Jun 6 13:30:12 1997 From: hinsen@ibs.ibs.fr (Konrad Hinsen) Date: Fri, 6 Jun 1997 14:30:12 +0200 Subject: [PYTHON MATRIX-SIG] Singular Values In-Reply-To: <199706051950.UAA28284@oak.sucs.soton.ac.uk> (dars@soton.ac.uk) Message-ID: <199706061230.OAA00387@lmspc1.ibs.fr> > of magnitude more painfull without it. I'm just trying to > understand something that niggles me. Well, the ultimate reason for choosing this arrangement is that LAPACK returns the matrices this way. After all, it's an arbitrary convention anyway; there is little objective motivation for doing it this or that way. -- ------------------------------------------------------------------------------- Konrad Hinsen | E-Mail: hinsen@ibs.ibs.fr Laboratoire de Dynamique Moleculaire | Tel.: +33-4.76.88.99.28 Institut de Biologie Structurale | Fax: +33-4.76.88.54.94 41, av. des Martyrs | Deutsch/Esperanto/English/ 38027 Grenoble Cedex 1, France | Nederlands/Francais ------------------------------------------------------------------------------- _______________ MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org _______________ From da@maigret.cog.brown.edu Sat Jun 7 20:08:23 1997 From: da@maigret.cog.brown.edu (David Ascher) Date: Sat, 7 Jun 1997 15:08:23 -0400 (EDT) Subject: [PYTHON MATRIX-SIG] request for idioms Message-ID: I will be giving a tutorial on Numeric Python at SPAM-6, with Konrad following up with a tutorial on using Python for science or something like that. While I have a pretty good idea about most of what I want to talk about in the three hours, I would appreciate input for one section in particular. I would like to present common 'idioms' in NumPy, and my feeling is that those idioms are very field-specific. I've developed a few doing my own work, but I'm sure y'all have as well. I'm especially interested in: * things you came up with which map well to constructs you were used to from other systems (e.g. Matlab, S, etc.). * things which are very python-specific * things which are very cool -- use your own judgment. Obviously, there's no rush on this. If you can just keep this in mind and when you come up with something you'd like to share, send it my way. I feel that such a collection would be a good add-on for the more permanent online tutorial as well. If all goes well, I'll be swamped with suggestions and will have to select among them. I will also probably adjust them to fit within a 'story' for didactic reasons, but I'll try to acknowledge authorship for the non-obvious tricks. Please try to explain what your code does in English -- remember that I probably have a different background than you do! =) Cheers, --david _______________ MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org _______________ From hinsen@ibs.ibs.fr Mon Jun 9 12:53:43 1997 From: hinsen@ibs.ibs.fr (Konrad Hinsen) Date: Mon, 9 Jun 1997 13:53:43 +0200 Subject: [PYTHON MATRIX-SIG] request for idioms In-Reply-To: (message from David Ascher on Sat, 7 Jun 1997 15:08:23 -0400 (EDT)) Message-ID: <199706091153.NAA11258@lmspc1.ibs.fr> > about in the three hours, I would appreciate input for one section in > particular. I would like to present common 'idioms' in NumPy, and my > feeling is that those idioms are very field-specific. I've developed a I am not sure there are many field-specific idioms. Idioms depend largely on what a given array represents, e.g. "a table of function values" or "a 3d function on a grid" or whatever. The kind of data you want to store in an array is of course somewhat field-specific, but not that much. > * things you came up with which map well to constructs you were used > to from other systems (e.g. Matlab, S, etc.). Probably most, since our arrays were designed with such systems as a model. And there are books with idioms around, so maybe you should use them as a starting point. > Obviously, there's no rush on this. If you can just keep this in mind and > when you come up with something you'd like to share, send it my way. I OK, if I don't forget... -- ------------------------------------------------------------------------------- Konrad Hinsen | E-Mail: hinsen@ibs.ibs.fr Laboratoire de Dynamique Moleculaire | Tel.: +33-4.76.88.99.28 Institut de Biologie Structurale | Fax: +33-4.76.88.54.94 41, av. des Martyrs | Deutsch/Esperanto/English/ 38027 Grenoble Cedex 1, France | Nederlands/Francais ------------------------------------------------------------------------------- _______________ MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org _______________ From hinsen@ibs.ibs.fr Mon Jun 9 14:08:48 1997 From: hinsen@ibs.ibs.fr (Konrad Hinsen) Date: Mon, 9 Jun 1997 15:08:48 +0200 Subject: [PYTHON MATRIX-SIG] FFT question Message-ID: <199706091308.PAA11463@lmspc1.ibs.fr> Could someone explain to me how the functions real_fft and inverse_real_fft in FFT.py are supposed to be used? The first one returns a complex array, which the second one won't accept as input. -- ------------------------------------------------------------------------------- Konrad Hinsen | E-Mail: hinsen@ibs.ibs.fr Laboratoire de Dynamique Moleculaire | Tel.: +33-4.76.88.99.28 Institut de Biologie Structurale | Fax: +33-4.76.88.54.94 41, av. des Martyrs | Deutsch/Esperanto/English/ 38027 Grenoble Cedex 1, France | Nederlands/Francais ------------------------------------------------------------------------------- _______________ MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org _______________ From hugunin@mit.edu Mon Jun 9 17:27:39 1997 From: hugunin@mit.edu (Jim Hugunin) Date: Mon, 9 Jun 1997 12:27:39 -0400 Subject: [PYTHON MATRIX-SIG] FFT question Message-ID: <9706091628.AA29771@goldilocks> > Could someone explain to me how the functions real_fft and > inverse_real_fft in FFT.py are supposed to be used? The first one > returns a complex array, which the second one won't accept as input. These functions are based on the corresponding functionality in FFTPACK. Both functions are applied to an array of real numbers. In both cases, the result of the transform will be symetric and so only one half of this result is be returned. >From the docs: --- real_fft(a, n=None, axis=-1) Will return the n point discrete Fourier transform of the real valued array a. n defaults to the length of a. This is most efficient for n a power of two. The returned array will be one half of the symmetric complex transform of the real array. --- inverse_real_fft isn't in the documentation yet, but if you replace Fourier transform with inverse-Fourier transform you'll have it's docs. Note: These function are both available for efficiency only. There is no reason that you can't use the ordinary fft function on an array of reals. The result you get back will be much easier to deal with. -Jim _______________ MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org _______________ From hinsen@ibs.ibs.fr Tue Jun 10 11:19:11 1997 From: hinsen@ibs.ibs.fr (Konrad Hinsen) Date: Tue, 10 Jun 1997 12:19:11 +0200 Subject: [PYTHON MATRIX-SIG] FFT question In-Reply-To: <9706091628.AA29771@goldilocks> (hugunin@mit.edu) Message-ID: <199706101019.MAA14452@lmspc1.ibs.fr> > inverse_real_fft isn't in the documentation yet, but if you replace Fourier > transform with inverse-Fourier transform you'll have it's docs. So inverse_real_fft should do the inverse of real_fft, right? That's not what it does at the moment. > Note: These function are both available for efficiency only. There is no reason > that you can't use the ordinary fft function on an array of reals. The result you > get back will be much easier to deal with. Of course, and I am not planning to use them. I am just preparing the next part of my Python tutorial, which will cover linear algebra, FFT, etc. I guess I'll just say nothing about real FFTs! Konrad. -- ------------------------------------------------------------------------------- Konrad Hinsen | E-Mail: hinsen@ibs.ibs.fr Laboratoire de Dynamique Moleculaire | Tel.: +33-4.76.88.99.28 Institut de Biologie Structurale | Fax: +33-4.76.88.54.94 41, av. des Martyrs | Deutsch/Esperanto/English/ 38027 Grenoble Cedex 1, France | Nederlands/Francais ------------------------------------------------------------------------------- _______________ MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org _______________ From R.Hooft@EuroMail.com Tue Jun 10 12:11:22 1997 From: R.Hooft@EuroMail.com (Rob Hooft) Date: Tue, 10 Jun 1997 13:11:22 +0200 (MET DST) Subject: [PYTHON MATRIX-SIG] FFT question In-Reply-To: <199706101019.MAA14452@lmspc1.ibs.fr> References: <9706091628.AA29771@goldilocks> <199706101019.MAA14452@lmspc1.ibs.fr> Message-ID: <199706101111.NAA26146@nu> >>>>> "KH" == Konrad Hinsen writes: >> inverse_real_fft isn't in the documentation yet, but if you >> replace Fourier transform with inverse-Fourier transform you'll >> have it's docs. KH> So inverse_real_fft should do the inverse of real_fft, right? KH> That's not what it does at the moment. Nope, that's not how I understand it. real_fft : real input --> complex output, Fourier transform inverse_real_fft: real input --> complex output, inverse Fourier transform Regards, Rob Hooft. -- ===== R.Hooft@EuroMail.com http://www.Sander.EMBL-Heidelberg.DE/rob/ == ==== In need of protein modeling? http://www.Sander.EMBL-Heidelberg.DE/whatif/ Validation of protein structures? http://biotech.EMBL-Heidelberg.DE:8400/ ==== == PGPid 0xFA19277D == Use Linux! Free Software Rules The World! ============= _______________ MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org _______________ From hinsen@ibs.ibs.fr Tue Jun 10 13:42:18 1997 From: hinsen@ibs.ibs.fr (Konrad Hinsen) Date: Tue, 10 Jun 1997 14:42:18 +0200 Subject: [PYTHON MATRIX-SIG] FFT question In-Reply-To: <199706101111.NAA26146@nu> (R.Hooft@EuroMail.com) Message-ID: <199706101242.OAA14848@lmspc1.ibs.fr> > Nope, that's not how I understand it. > > real_fft : real input --> complex output, Fourier transform > inverse_real_fft: real input --> complex output, inverse Fourier transform I'd call the second one "real_inverse_fft", not "inverse_real_fft", which to me implies "inverse of real_fft". Moreover, what's the point of real_fft if there is no straightforward way to get the inverse? -- ------------------------------------------------------------------------------- Konrad Hinsen | E-Mail: hinsen@ibs.ibs.fr Laboratoire de Dynamique Moleculaire | Tel.: +33-4.76.88.99.28 Institut de Biologie Structurale | Fax: +33-4.76.88.54.94 41, av. des Martyrs | Deutsch/Esperanto/English/ 38027 Grenoble Cedex 1, France | Nederlands/Francais ------------------------------------------------------------------------------- _______________ MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org _______________ From jbaddor@phy.ulaval.ca Tue Jun 10 14:35:09 1997 From: jbaddor@phy.ulaval.ca (Jean-Bernard ADDOR) Date: Tue, 10 Jun 1997 09:35:09 -0400 (EDT) Subject: [PYTHON MATRIX-SIG] FFT question In-Reply-To: <199706101242.OAA14848@lmspc1.ibs.fr> Message-ID: Konrad HINSEN: > I'd call the second one "real_inverse_fft", not "inverse_real_fft", which > to me implies "inverse of real_fft". Moreover, what's the point of real_fft > if there is no straightforward way to get the inverse? I have an application where I used real_fft to compute the spectrum of a very long time serie (length: 2**22), without any need to come back in real space. I was just happy not to have to use too much memory with real_fft. From that point of view, I think it could be very useful to have it in the tutorial. Jean-Bernard _______________ MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org _______________ From jac@gandalf.llnl.gov Tue Jun 10 15:40:25 1997 From: jac@gandalf.llnl.gov (Jim Crotinger) Date: Tue, 10 Jun 1997 07:40:25 -0700 Subject: [PYTHON MATRIX-SIG] FFT question In-Reply-To: <199706101111.NAA26146@nu> References: <9706091628.AA29771@goldilocks> <199706101019.MAA14452@lmspc1.ibs.fr> <199706101111.NAA26146@nu> Message-ID: <199706101440.HAA17670@gandalf.llnl.gov> Rob Hooft writes: > > real_fft : real input --> complex output, Fourier transform > inverse_real_fft: real input --> complex output, inverse Fourier transform Surely not. The latter is just the complex conjugate of the former - we don't need a seperate function to compute it. Since "real fft"s usually pack the amplitude of the highest mode (which is phaseless) into the same complex number as the amplitude of the 0 mode, it is useful to have an inverse (I haven't tried the Python one - I'm just assuming that this is the way that it works). Jim _______________ MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org _______________ From phil@geog.ubc.ca Wed Jun 11 00:26:39 1997 From: phil@geog.ubc.ca (Phil Austin) Date: Tue, 10 Jun 1997 16:26:39 -0700 Subject: [MATRIX-SIG] Re: [PYTHON MATRIX-SIG] request for idioms In-Reply-To: References: Message-ID: <199706102326.QAA04851@curlew.geog.ubc.ca> >>>>> "David" == David Ascher writes: David> I will be giving a tutorial on Numeric Python at SPAM-6, ... David> * things you came up with which map well to constructs David> you were used to from other systems (e.g. Matlab, S, David> etc.). things which are very python-specific things which David> are very cool -- use your own judgment. One thing that impressed me was Konrad's rewrite of our NumPy-extended netcdf module. For our initial attempt, we simply implemented the netcdf C++ interface, which set the origin for an array slice using set_cur. Thus to read a 4x3 slice of an array starting a (1,2) you did: a=nc.open('demo.nc', nc.WRITE) b=a.var('temp') b.set_cur(1,2) print b.get(4,3) In contrast, Konrad's module handles it like this: a=nc.open('demo.nc', nc.WRITE) b=a.var('temp') print b[1:5,2:5] I don't know of another language that let's you pass subscripts to a C extension in this way. Regards, Phil _______________ MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org _______________ From phil@geog.ubc.ca Wed Jun 11 00:26:39 1997 From: phil@geog.ubc.ca (Phil Austin) Date: Tue, 10 Jun 1997 16:26:39 -0700 Subject: [MATRIX-SIG] Re: [PYTHON MATRIX-SIG] request for idioms In-Reply-To: References: Message-ID: <199706102326.QAA04851@curlew.geog.ubc.ca> >>>>> "David" == David Ascher writes: David> I will be giving a tutorial on Numeric Python at SPAM-6, ... David> * things you came up with which map well to constructs David> you were used to from other systems (e.g. Matlab, S, David> etc.). things which are very python-specific things which David> are very cool -- use your own judgment. One thing that impressed me was Konrad's rewrite of our NumPy-extended netcdf module. For our initial attempt, we simply implemented the netcdf C++ interface, which set the origin for an array slice using set_cur. Thus to read a 4x3 slice of an array starting a (1,2) you did: a=nc.open('demo.nc', nc.WRITE) b=a.var('temp') b.set_cur(1,2) print b.get(4,3) In contrast, Konrad's module handles it like this: a=nc.open('demo.nc', nc.WRITE) b=a.var('temp') print b[1:5,2:5] I don't know of another language that let's you pass subscripts to a C extension in this way. Regards, Phil _______________ MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org _______________ From amullhau@ix.netcom.com Wed Jun 11 01:37:25 1997 From: amullhau@ix.netcom.com (Andrew P. Mullhaupt) Date: Tue, 10 Jun 1997 20:37:25 -0400 Subject: [MATRIX-SIG] Re: [PYTHON MATRIX-SIG] request for idioms References: <199706102326.QAA04851@curlew.geog.ubc.ca> Message-ID: <339DF345.32CA@ix.netcom.com> Phil Austin wrote: > > In contrast, Konrad's module handles it like this: > > print b[1:5,2:5] > > I don't know of another language that let's you pass subscripts to > a C extension in this way. It's not that uncommon, presumably it's that you haven't used the languages where it is possible/easy. The S language, and some of the post-APL languages do this, and with their more general subscripting as well. Of course, this is not to take anything away from Konrad's module. Later, Andrew Mullhaupt _______________ MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org _______________ From amullhau@ix.netcom.com Wed Jun 11 01:37:25 1997 From: amullhau@ix.netcom.com (Andrew P. Mullhaupt) Date: Tue, 10 Jun 1997 20:37:25 -0400 Subject: [MATRIX-SIG] Re: [PYTHON MATRIX-SIG] request for idioms References: <199706102326.QAA04851@curlew.geog.ubc.ca> Message-ID: <339DF345.32CA@ix.netcom.com> Phil Austin wrote: > > In contrast, Konrad's module handles it like this: > > print b[1:5,2:5] > > I don't know of another language that let's you pass subscripts to > a C extension in this way. It's not that uncommon, presumably it's that you haven't used the languages where it is possible/easy. The S language, and some of the post-APL languages do this, and with their more general subscripting as well. Of course, this is not to take anything away from Konrad's module. Later, Andrew Mullhaupt _______________ MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org _______________ From acura@vcn.bc.ca Sat Jun 14 22:32:10 1997 From: acura@vcn.bc.ca (Kelvin Lee) Date: Sat, 14 Jun 1997 14:32:10 -0700 (PDT) Subject: [MATRIX-SIG] Varimax rotation? Message-ID: I was wondering if anyone out there has created a script to do Varimax rotation? I'm using NumPy to perform PCA and would like to rotate my retained components. Before writing something myself I wanted to check to see if anyone's already done this. Failing that, does anyone have a good text that describes the procedure (the multivariate stats books I have are a little light describing the actual rotation procedure). Thanks, Kelvin _______________ MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org _______________ From arw@dante.mh.lucent.com Mon Jun 16 22:13:16 1997 From: arw@dante.mh.lucent.com (Aaron Watters) Date: Mon, 16 Jun 1997 17:13:16 -0400 Subject: [MATRIX-SIG] cmp? Message-ID: <199706162110.RAA06244@dante.mh.lucent.com> I'm certain there are good reasons why array cmp works the way it does, please educate me. >>> cmp(xx[0],xx[1]) -1 >>> cmp(xx[1],xx[0]) -1 >>> xx array([[1, 1, 1, 0, 0, 0, 2, 2, 2], [1, 2, 3, 1, 2, 3, 1, 2, 3]]) what's the rationale here? I want to sort arrays. If I must write my own cmp function, it'll be really really slow. Also >>> sort(xx) array([[0, 0, 0, 1, 1, 1, 2, 2, 2], [1, 1, 1, 2, 2, 2, 3, 3, 3]]) Each sub array is sorted individually. It seems to me this should be written array(map(sort, xx)) and in the former the subarrays should be sorted lexicographically as Python sorts tuples and lists. Please explain. I'm missing something here. Sorry if I'm being stupid again. Thanks, Aaron Watters ps: the motivation is, for fun, I'm trying to implement some database style functionality with the help of Numeric, heading towards accounting-type applications. _______________ MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org _______________ From hochberg@wwa.com Tue Jun 17 03:16:58 1997 From: hochberg@wwa.com (Timothy A. Hochberg) Date: Mon, 16 Jun 1997 21:16:58 -0500 (CDT) Subject: [MATRIX-SIG] cmp? In-Reply-To: <199706162110.RAA06244@dante.mh.lucent.com> Message-ID: On Mon, 16 Jun 1997, Aaron Watters wrote: > I'm certain there are good reasons > why array cmp works the way it does, > please educate me. > > >>> cmp(xx[0],xx[1]) > -1 > >>> cmp(xx[1],xx[0]) > -1 > >>> xx > array([[1, 1, 1, 0, 0, 0, 2, 2, 2], > [1, 2, 3, 1, 2, 3, 1, 2, 3]]) > > what's the rationale here? I want to sort arrays. > If I must write my own cmp function, it'll be really > really slow. I believe that what we would like to do here is a) return an array that compares cmp element by element, or b) raise an exception. Since these are(were) not possible, Jim H. did c) punt. (I believe I heard that 1.5 is gonna let exceptions be raised in in comparisons, so maybe this will be "fixed".) How do you want to compare your arrays? If you just want to compare them like tuples you could use: > >>> cmp(tuple(xx[0]),tuple(xx[1])) If you want to compare the sums of the elements, you could use: cmp(sum(xx[0]), sum(xx[1])) or the product of the absolute values of the cosines: cmp(product(abs(cos(xx[0])), product(abs(cos(xx[1])))) or if you want to compare element be element look at less, less_equal, etc. > Also > > >>> sort(xx) > array([[0, 0, 0, 1, 1, 1, 2, 2, 2], > [1, 1, 1, 2, 2, 2, 3, 3, 3]]) > > Each sub array is sorted individually. It seems > to me this should be written > > array(map(sort, xx)) > > and in the former the subarrays should be sorted > lexicographically as Python sorts tuples and lists. > > Please explain. I'm missing something here. > Sorry if I'm being stupid again. Hmmm...again, if you want to sort them lexigraphically like tuples and lists, you could convert them to tuples and lists and sort them then convert them back. The numeric functions all operate pretty much the same way - they operate on the smallest chunk of the matrix that they can and they do it over and over again. Since that's amazingly unclear, let me try some examples: cos(xx) # applies cos to every element of the matrix. sort(xx) # sorts every sub vector of xx along axis -1 (=1) sort(xx,0) # sorts every sub vector of xx along axis 0 sum(xx,1) # sum every sub vector along axis 1 - returns array([ 9 18]) determinant(xx) # wouldn't work with your xx, but gets the determinant of # every sub matrix of xx (imagine xx was 2x3x3) Now again how do you want to sort your array. If you want to sort according to some numeric criteria other than lexigraphic ordering, I would a) construct a sort key using sum, abs, +, *, etc, b) use argsort to determine the sort ordering, then use take to get the sorted matrix. So, if I wanted to sort the columns of xx by sum I'd use: take(xx, argsort(sum(xx)), 1) Hope that's more enlightening than confusing.... ____ /im +------------------------------------------------+ |Tim Hochberg Research Assistant | |hochberg wwa.com University of Illinois | | Electrical Engineering | +------------------------------------------------+ _______________ MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org _______________ From arw@dante.mh.lucent.com Tue Jun 17 14:44:49 1997 From: arw@dante.mh.lucent.com (Aaron Watters) Date: Tue, 17 Jun 1997 09:44:49 -0400 Subject: [MATRIX-SIG] cmp? Message-ID: <199706171343.JAA17434@dante.mh.lucent.com> Thanks for the response > How do you want to compare your arrays? If you just want to compare them > like tuples you could use: > > >>> cmp(tuple(xx[0]),tuple(xx[1])) > If you want to compare the sums of the elements, you could use: The only problem with this is that the internal representation for a tuple of floats must be 12x larger than for a numeric array, not to mention the speed penalty for all those mallocs. Of course, for now, this is exactly what I will do. I suppose for my immediate purpose I'd like to compare arrays any way so that I could use the list.sort builtin on a list of arrays and get all equal arrays grouped together. This would require that if cmp(a1, a2) is negative then cmp(a2, a1) is positive nonzero and if cmp(a2, a3) is also negative then cmp(a1, a3) is negative, and a1==a4 iff cmp(a1,a4)==0. These are the "axioms of cmp" that are assumed by list.sort and other Python facilities. Is there any problem with lexicographic ordering? I understand that cmp() may have no reasonable interpretation from a numeric perspective per se, but you'd still like to be able to do "row sorts". >From a database perspective I'd like to implement the analogue of SQL "group by", ie, as in select a,b, avg(d+e) from array1 group by a,b The "natural" way to do this would be to align the rows of array1 in a list of [a,b,d,e] and then sort the list lexicographically to group equal a,b pairs together. At least *I* think it would be natural. -- Aaron Watters ps: alternatively if there were some way to get partitions of indices where a and b were equal this might do just as well in this instance. I can't figure out how to do this efficiently for all pairings of a,b at once (maybe my ignorance). _______________ MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org _______________ From hugunin@javanet.com Tue Jun 17 16:12:31 1997 From: hugunin@javanet.com (Jim Hugunin) Date: Tue, 17 Jun 1997 11:12:31 -0400 Subject: [MATRIX-SIG] cmp? Message-ID: <199706171512.LAA16034@kona.javanet.com> > Is there any problem with lexicographic ordering? I understand > that cmp() may have no reasonable interpretation from a > numeric perspective per se, but you'd still like to be able to do > "row sorts". Tim H. did a good job of explaining why this isn't done. Because it's unlikely that cmp will be able to return an array of booleans anytime soon, what I really want to do is raise an exception here. In order to avoid any problems with backwards incompatibility I've decided to make cmp as useless as possible for arrays until I can have it start raising exceptions. > >From a database perspective I'd like to implement the > analogue of SQL "group by", ie, as in > > select a,b, avg(d+e) > from array1 > group by a,b For those of us with minimal SQL knowledge, could you go into a few more details on what "group by" is supposed to do? A concrete example of where you'd like to do this (on a database of floats I assume?) would be helpful. I have the strong feeling that either you really shouldn't be using NumPy to do this, or you're missing some obvious better approach. Of course, this might just be my general desire not to admit that a lexocographic cmp could possibly be valuable for arrays. Please send more info - Jim _______________ MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org _______________ From hochberg@wwa.com Tue Jun 17 16:13:51 1997 From: hochberg@wwa.com (Timothy A. Hochberg) Date: Tue, 17 Jun 1997 10:13:51 -0500 (CDT) Subject: [MATRIX-SIG] cmp? In-Reply-To: <199706171343.JAA17434@dante.mh.lucent.com> Message-ID: On Tue, 17 Jun 1997, Aaron Watters wrote: > Thanks for the response > [I give unsatisfactory solution] [AW says noone understands him] > I suppose for my immediate purpose I'd like to compare arrays > any way so that I could use the list.sort builtin on a list of arrays > and get all equal arrays grouped together. [...] How about this (barely tested) code: from Numeric import * xx = array([[1, 1, 1, 0, 0, 0, 2, 2, 2], [1, 2, 3, 1, 2, 3, 1, 2, 3], [1, 6, 3, 1, 2, 3, 1, 2, 3], [1, 2, 3, 1, 3, 3, 1, 2, 3]]) def aw_sort(A, axis=0): """Sort a 2d array along axis.""" A = asarray(A) if axis: A = transpose(A) S = shape(A) if len(S) != 2: raise ValueError, "array must be 2D." for i in range(S[1]-1, -1, -1): A = take(A, argsort(A[:,i]), 0) if axis: A = transpose(A) return A if __name__ == '__main__': print aw_sort(xx) print print aw_sort(xx, 1) It should give you lexicographic sorting on 2d arrays. Extending it to arrays of arbitrary dimension is left as an exercise to the reader. How efficient it is, I don't know. > Is there any problem with lexicographic ordering? I understand > that cmp() may have no reasonable interpretation from a > numeric perspective per se, but you'd still like to be able to do > "row sorts". If I recall correctly, there was some concern with doing something that would be regretted later if functionality for __less__, etc. ever gets added to Python. > >From a database perspective I'd like to implement the > analogue of SQL "group by", ie, as in > > select a,b, avg(d+e) > from array1 > group by a,b I think I understand this, but I'm a database uninitiate... > The "natural" way to do this would be to > align the rows of array1 in a list of [a,b,d,e] > and then sort the list lexicographically to > group equal a,b pairs together. At least *I* > think it would be natural. -- Aaron Watters Sounds good to me. > ps: alternatively if there were some way > to get partitions of indices where a and b > were equal this might do just as well in > this instance. I can't figure out how to do > this efficiently for all pairings of a,b at once > (maybe my ignorance). I can't think of one either. Try aw_sort - let me know if it works for you. ____ /im +------------------------------------------------+ |Tim Hochberg Research Assistant | |hochberg wwa.com University of Illinois | | Electrical Engineering | +------------------------------------------------+ _______________ MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org _______________ From arw@dante.mh.lucent.com Tue Jun 17 18:06:22 1997 From: arw@dante.mh.lucent.com (Aaron Watters) Date: Tue, 17 Jun 1997 13:06:22 -0400 Subject: [MATRIX-SIG] cmp? Message-ID: <199706171832.OAA23582@dante.mh.lucent.com> You're probably right that Numeric deserves to have an array as the result of cmp. The desire to generalize cmp has occured to many, including me. FWIW what I'm doing now is using numeric to hold columns of numeric data where each column may be of a different format. Say I have a table with columns product_id, dept_number, sales expenses I want to be able to do the analogue of select product_id, dept_number, sum(sales-expenses) from sales_info group by product_id, dept_number with result columns product_id, dept_number, sum(sales-expenses) for dept&prod This would result in a summary report listing the total sales less expenses for each product id and department number. To do this I need some mechanism to group product_id, dept_number matches together... I can get by translating arrays to tuples if I must. An alternative would be to provide an array lexicographic comparator not in the type structure (implemented in C and specialized to arrays for speed). Then I could write list_of_arrays.sort(lexicographic) or whatever. In sum, I now understand why it works the way it does, and I am content. -- Aaron Watters _______________ MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org _______________ From ta.short@pti-us.com Tue Jun 17 18:41:03 1997 From: ta.short@pti-us.com (Tom Short) Date: Tue, 17 Jun 1997 17:41:03 +0000 Subject: [MATRIX-SIG] cmp? In-Reply-To: <199706171832.OAA23582@dante.mh.lucent.com> Message-ID: <199706172141.RAA08969@hobbes.pti-us.com> > select product_id, dept_number, sum(sales-expenses) > from sales_info > group by product_id, dept_number > > with result columns > > product_id, dept_number, sum(sales-expenses) for dept&prod In Yorick, the "where" function is great for this sort of thing: > a=[1,2,3,4,5]; b=a*10; c=a*100 > idx=where(a==2&b==20) > write, a(idx), b(idx), c(idx) 2 20 200 In Python, you can use the same approach, but it is more complex (primarily because the boolean comparisons have to be done with functions): >>> a=array([1,2,3,4,5]) >>> b=10*a >>> c=100*a >>> idx = nonzero(where(boolean_and(equal(a,2),equal(b,20)),1,0)) >>> print take(a,idx), take(b,idx), take(c,idx) 2 20 200 - Tom -------------------------- See the PTI Lightning Photo Gallery: http://www.pti-us.com/pti/consult/dist/photos.htm _______________ MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org _______________ From hinsen@ibs.ibs.fr Wed Jun 18 09:10:36 1997 From: hinsen@ibs.ibs.fr (Konrad Hinsen) Date: Wed, 18 Jun 1997 10:10:36 +0200 Subject: [MATRIX-SIG] cmp? In-Reply-To: <199706171832.OAA23582@dante.mh.lucent.com> (arw@dante.mh.lucent.com) Message-ID: <199706180810.KAA12706@lmspc1.ibs.fr> > You're probably right that Numeric deserves > to have an array as the result of cmp. The > desire to generalize cmp has occured to many, > including me. Generalizing cmp() seems obvious, but creates other problems. If cmp() returns an array, what should if and while statement do with the result? One answer is to apply the "check for zero", but it is not at all clear how that should be implemented for arrays. -- ------------------------------------------------------------------------------- Konrad Hinsen | E-Mail: hinsen@ibs.ibs.fr Laboratoire de Dynamique Moleculaire | Tel.: +33-4.76.88.99.28 Institut de Biologie Structurale | Fax: +33-4.76.88.54.94 41, av. des Martyrs | Deutsch/Esperanto/English/ 38027 Grenoble Cedex 1, France | Nederlands/Francais ------------------------------------------------------------------------------- _______________ MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org _______________ From hugunin@mit.edu Fri Jun 20 20:45:52 1997 From: hugunin@mit.edu (Jim Hugunin) Date: Fri, 20 Jun 1997 15:45:52 -0400 Subject: [MATRIX-SIG] FYI: My future status (and Numeric Python's) Message-ID: <9706201946.AA13094@goldilocks.LCS.MIT.EDU> As of last week I have officially taken a leave of absence from MIT in order to pursue my ideas relating to Python and Java full-time. This means that I'm likely to have even less time in the future to spend working on NumPy than I have recently. The good news is that I consider the basic project essentially finished. As soon as possible I'll have the last crop of bug reports fixed and I'll be putting out NumPy1.0 final. After that point I'll probably be putting only minimal effort into this project. Obviously I'll keep handling bug reports and putting out updated versions, but I doubt that I'll be pushing forward to add any new features for quite some time (I think the current feature set is reasonably complete). The web pages for this project are moving to Starship Python over this weekend. I'll send out more information when the move is complete to let people know where to find things. Just thought you should know - Jim Hugunin - hugunin@javanet.com PS - I can probably be persuaded to do paid consulting work on Numeric Python if there is something you really need. Contact me to discuss terms and rates. -----------Extra stuff for the truly interested---------------- Much of my motivation to look into Python/Java came from my experiences with creating and distributing Numeric Python. I feel that the three key flaws in NumPy are: 1) Hard to install -solved by portable Java bytecodes 2) No portable graph packages (arguable I know) -all graphics are portable with Java 3) Hard to build python interfaces to existing libraries -trivial to use Java libraries with Python -I expect to see Java bindings for almost every major library in the not too distant future (this one's just a guess). -Java's features (garbage collection, exceptions, classes) make writing python extensions FAR easier I fully intend to move my Numeric Extensions to Java once the rest of the system I'm building falls into place. I don't think the performance will suffer much at all (I've even come up with some fun ideas involving run-time generated bytecodes that could offer some serious performance boosts). I also think that the three most significant flaws that I mentioned above will go away with such a system. I also realize that many people out there (probably on this list more than anywhere else in the Python community) are using Python primarily as a glue between existing custom C and FORTRAN libraries. In this case, the greater difficulty of writing a Java interface for your native code vs. a Python interface is probably a big deal. I also realize that many people on this list run in environments where Java is not available (most supercomputers). So I'm not claiming that Python/Java is a universal solution to anything. Just thought somebody out there might want to know - Jim _______________ MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org _______________ From raptor10@airmail.net Fri Jun 20 23:30:57 1997 From: raptor10@airmail.net (Dominick Pescatore) Date: Fri, 20 Jun 1997 15:30:57 -0700 Subject: [MATRIX-SIG] pickled sausages Message-ID: <33AB04A1.3E77@airmail.net> I am looking for a recipe for pickling sausages. Anyone that could help me, Iwould appreciate it. Thank You. _______________ MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org _______________ From da@maigret.cog.brown.edu Fri Jun 20 22:02:26 1997 From: da@maigret.cog.brown.edu (David Ascher) Date: Fri, 20 Jun 1997 17:02:26 -0400 (EDT) Subject: [MATRIX-SIG] pickled sausages In-Reply-To: <33AB04A1.3E77@airmail.net> Message-ID: On Fri, 20 Jun 1997, Dominick Pescatore wrote: > I am looking for a recipe for pickling sausages. Anyone that could help > me, Iwould appreciate it. 1) Find a glass jar with a lid which makes a good tight seal. 2) Fill it half way with vinegar. 3) Insert the sausage 4) Add more vinegar until it's all the way to the top. 5) Wait 3 years, 2 months, 4 days, 26 hours, 22 minutes and 2 seconds. 6) Open, slice, eat, enjoy. _______________ MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org _______________ From drh@cse.unl.edu Sat Jun 21 01:55:34 1997 From: drh@cse.unl.edu (Doug Heisterkamp) Date: Fri, 20 Jun 1997 19:55:34 -0500 (CDT) Subject: [MATRIX-SIG] FYI: My future status (and Numeric Python's) In-Reply-To: <9706201946.AA13094@goldilocks.LCS.MIT.EDU> from "Jim Hugunin" at Jun 20, 97 03:45:52 pm Message-ID: <199706210055.TAA28257@cse.unl.edu> > > > As of last week I have officially taken a leave of absence from MIT in order to > pursue my ideas relating to Python and Java full-time. This means that I'm likely > to have even less time in the future to spend working on NumPy than I have recently. > Thanks Jim, for all of the time and effort that you have put into NumPy. I find it very useful. I wish you the best of fortune in your Java-Python project. Maybe we should set up some kind of formal council or committee to handle bug report fixes and new feature prototypes. That way you could pass the bug fixing work to different committee members. Doug Heisterkamp drh@cse.unl.edu _______________ MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org _______________ From hinsen@ibs.ibs.fr Mon Jun 23 10:39:09 1997 From: hinsen@ibs.ibs.fr (Konrad Hinsen) Date: Mon, 23 Jun 1997 11:39:09 +0200 Subject: [MATRIX-SIG] FYI: My future status (and Numeric Python's) In-Reply-To: <9706201946.AA13094@goldilocks.LCS.MIT.EDU> (hugunin@mit.edu) Message-ID: <199706230939.LAA31982@lmspc1.ibs.fr> > The good news is that I consider the basic project essentially finished. As soon as > possible I'll have the last crop of bug reports fixed and I'll be putting out > NumPy1.0 final. After that point I'll probably be putting only minimal effort into Sounds good. I agree that the project is essentially finished, as far as any project can ever be called finished. Thanks for all the work you did! > Much of my motivation to look into Python/Java came from my experiences with > creating and distributing Numeric Python. I feel that the three key flaws in NumPy > are: > > 1) Hard to install > -solved by portable Java bytecodes Well, yes, but... I am sure the installation of Python extensions can be improved without reimplementing Python! > 2) No portable graph packages (arguable I know) > -all graphics are portable with Java Within the limits of Java portability! > 3) Hard to build python interfaces to existing libraries > -trivial to use Java libraries with Python > -I expect to see Java bindings for almost every major library > in the not too distant future (this one's just a guess). > -Java's features (garbage collection, exceptions, classes) > make writing python extensions FAR easier This is partly true, but I don't share your optimism about the second point. Besides, Java-C integration is far from standard, and I haven't even heard about attempts at Java-Fortran integration. > I fully intend to move my Numeric Extensions to Java once the rest of the system I'm > building falls into place. I don't think the performance will suffer much at all That will depend a lot on the quality of Java compilers. But I agree that performance may be good enough for many NumPy applications, e.g. for data analysis. On the other hand, one of my major uses of NumPy is interfacing to C modules that burn lots of CPU times, up to weeks per program run. I am not at all convinced that Java performance will be acceptable for such stuff in the near future. Anyway, all that is *your* problem ;-) We just sit back and watch! Konrad. -- ------------------------------------------------------------------------------- Konrad Hinsen | E-Mail: hinsen@ibs.ibs.fr Laboratoire de Dynamique Moleculaire | Tel.: +33-4.76.88.99.28 Institut de Biologie Structurale | Fax: +33-4.76.88.54.94 41, av. des Martyrs | Deutsch/Esperanto/English/ 38027 Grenoble Cedex 1, France | Nederlands/Francais ------------------------------------------------------------------------------- _______________ MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org _______________ From arw@dante.mh.lucent.com Thu Jun 26 21:15:45 1997 From: arw@dante.mh.lucent.com (Aaron Watters) Date: Thu, 26 Jun 1997 16:15:45 -0400 Subject: [MATRIX-SIG] reverse of take? Message-ID: <199706262012.QAA16688@dante.mh.lucent.com> Okay, stupid question time. (I think I just subscribed, assuming the mail address I used made sense... actually I thought I'd subscribed before, but there ya go...) Been fiddling with numeric recently and I can't seem to find or derive an "inverse" to take. Ie, I'd like a builtin that will set a collection of values scattered about inside an array. Ie, I want to draw a circle on a large array of 0 bytes (without making another large array). solution: compute the array of indices for the circle and set all the indices to one. Might look something like this: set_by_indices (big_byte_array, indices, 1) Does the equivalent of for pair in indices: big_byte_array(pair[0], pair[1]) = 1 but without the Python overhead. or whatever. Sorry if this kinda thing is already there. Please educate me. -- Aaron Watters _______________ MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org _______________ From Fredrik Lundh" >Ie, I want to draw a circle on a large array of 0 bytes >(without making another large array). solution: compute >the array of indices for the circle and set all the indices >to one. from ImageDraw import ImageDraw d = ImageDraw(im) d.setfill(1) d.ellipse((x0,y0,x1,y1)) Oops, wrong SIG, sorry for the intrusion ;-) Cheers /F _______________ MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org _______________ From arw@dante.mh.lucent.com Thu Jun 26 22:15:21 1997 From: arw@dante.mh.lucent.com (Aaron Watters) Date: Thu, 26 Jun 1997 17:15:21 -0400 Subject: [MATRIX-SIG] reverse of take? Message-ID: <199706262112.RAA20624@dante.mh.lucent.com> Ok. Just to be perverse, say I want to draw a 1d fractal border on the huge 2d matrix using only the indices. (I'll definitely have to look into your imagedraw stuff tho!) -- Aaron Watters ---------- > From: Fredrik Lundh > To: matrix-sig@python.org; Aaron Watters > Subject: Re: [MATRIX-SIG] reverse of take? > Date: Thursday, June 26, 1997 5:01 PM > > >Ie, I want to draw a circle on a large array of 0 bytes > >(without making another large array). solution: compute > >the array of indices for the circle and set all the indices > >to one. > > from ImageDraw import ImageDraw > > d = ImageDraw(im) > d.setfill(1) > > d.ellipse((x0,y0,x1,y1)) > > Oops, wrong SIG, sorry for the intrusion ;-) > > Cheers /F _______________ MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org _______________ From amullhau@ix.netcom.com Thu Jun 26 23:01:23 1997 From: amullhau@ix.netcom.com (Andrew P. Mullhaupt) Date: Thu, 26 Jun 1997 18:01:23 -0400 Subject: [MATRIX-SIG] reverse of take? In-Reply-To: <199706262012.QAA16688@dante.mh.lucent.com> Message-ID: <3.0.1.32.19970626180123.006ce0ec@popd.netcruiser> At 04:15 PM 6/26/97 -0400, Aaron Watters wrote: >but without the Python overhead. ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ >or whatever. Sorry if this kinda thing >is already there. Please educate me. -- Aaron Watters What you really want is sometimes called scatter/gather indexing, and there has been occasional discussion in comp.lang.python. It's pretty easy to implement well - although there are some clever optimizations deriving from the vector lattice structure of indexing which should not be missed. There is a large experience with this sort of thing in the APL derivative languages. NumPy may be alone among array languages in not supporting this. Note that in NumPy the advantage of this is multiplied over the advantage in 1-dimensional indexing, too. So join the list of people who want more general indexing in Python. Later, Andrew Mullhaupt _______________ MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org _______________ From amullhau@ix.netcom.com Thu Jun 26 23:03:24 1997 From: amullhau@ix.netcom.com (Andrew P. Mullhaupt) Date: Thu, 26 Jun 1997 18:03:24 -0400 Subject: [MATRIX-SIG] reverse of take? In-Reply-To: <9706262054.AA14002@arnold.image.ivab.se> Message-ID: <3.0.1.32.19970626180324.006cf410@popd.netcruiser> At 11:01 PM 6/26/97 +0200, Fredrik Lundh wrote: >>Ie, I want to draw a circle on a large array of 0 bytes >>(without making another large array). solution: compute >>the array of indices for the circle and set all the indices >>to one. > >from ImageDraw import ImageDraw > >d = ImageDraw(im) >d.setfill(1) > >d.ellipse((x0,y0,x1,y1)) > And if it's not a conic section he wants to draw....? Later, Andrew Mullhaupt _______________ MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org _______________ From amullhau@ix.netcom.com Thu Jun 26 23:11:53 1997 From: amullhau@ix.netcom.com (Andrew P. Mullhaupt) Date: Thu, 26 Jun 1997 18:11:53 -0400 Subject: [MATRIX-SIG] reverse of take? In-Reply-To: <199706262112.RAA20624@dante.mh.lucent.com> Message-ID: <3.0.1.32.19970626181153.006ce0ec@popd.netcruiser> At 05:15 PM 6/26/97 -0400, Aaron Watters wrote: >Ok. Just to be perverse, say I want to draw a 1d fractal >border on the huge 2d matrix using only the indices. Just so. Here's how to do it in S, where z is the huge array, and the x values of the indices are in a vector x, similarly for y, and v is the vector of corresponding values to assign. z[cbind(x, y)] <- v What this does is create an Nx2 matrix with columns x and y, and then indexes the array z with them. This kind of scatter is really neat and one of the attractions of the S language. S has by far the nicest syntax for indexing of all the array languages, and it is the language closest to Python in terms of object-orientation and argument syntax. Note that although S is normally very slow compared to Python, the fact that powerful array indexing is implemented at the "C" level makes up for a lot of lost ground. This could easily be added to NumPy (along with a couple of other really useful forms of indexing). The long history of successful use of the highly flexible indexing in S is a 'proof of concept'. Later, Andrew Mullhaupt _______________ MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org _______________ From Fredrik Lundh" >And if it's not a conic section he wants to draw....? d.polygon(xy), perhaps? or d.point(xy) to draw an arbitrary point set, where xy can contain either [x,y,x,y...] or [(x,y),(x,y),...]. (but he did say a circle, didn't he?) Cheers /F _______________ MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org _______________ From phil@geog.ubc.ca Fri Jun 27 00:24:44 1997 From: phil@geog.ubc.ca (Phil Austin) Date: Thu, 26 Jun 1997 16:24:44 -0700 Subject: [MATRIX-SIG] reverse of take? In-Reply-To: <3.0.1.32.19970626181153.006ce0ec@popd.netcruiser> References: <199706262112.RAA20624@dante.mh.lucent.com> <3.0.1.32.19970626181153.006ce0ec@popd.netcruiser> Message-ID: <199706262324.QAA29257@curlew.geog.ubc.ca> >>>>> "AM" == Andrew P Mullhaupt writes: AM> This kind of scatter AM> is really neat and one of the attractions of the S language. S AM> has by far the nicest syntax for indexing of all the array AM> languages, and it is the language closest to Python in terms AM> of object-orientation and argument syntax. Note that although AM> S is normally very slow compared to Python, the fact that AM> powerful array indexing is implemented at the "C" level makes AM> up for a lot of lost ground. AM> This could easily be added to NumPy (along with a couple of AM> other really useful forms of indexing). The long history of AM> successful use of the highly flexible indexing in S is a AM> 'proof of concept'. One thing that would be a helpful teaching tool, and a useful part of a motivating sales pitch for S indexing, would be a concordance giving a variety of index constructs in S, Matlab, Mathematica, etc., and their python equivalents. I would volunteer to collate such a list if people wanted to make submissions. At the moment I'm trying to translate various legacy S constructs into take, repeat, reduce combinations--it feels like I'm taking (and failing) some kind of IQ test. Phil _______________ MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org _______________ From Johann Hibschman Fri Jun 27 02:07:25 1997 From: Johann Hibschman (Johann Hibschman) Date: Thu, 26 Jun 1997 18:07:25 -0700 (PDT) Subject: [MATRIX-SIG] reverse of take? In-Reply-To: <3.0.1.32.19970626181153.006ce0ec@popd.netcruiser> Message-ID: On Thu, 26 Jun 1997, Andrew P. Mullhaupt wrote: > Just so. Here's how to do it in S, where z is the huge array, > and the x values of the indices are in a vector x, similarly > for y, and v is the vector of corresponding values to assign. > > z[cbind(x, y)] <- v [...] > > This could easily be added to NumPy (along with a couple of other really > useful forms of indexing). The long history of successful use of the > highly flexible indexing in S is a 'proof of concept'. Hmm. Would it be acceptable to define (in a separate extension module) a function to do this, or do you feel that the built-in syntax is necessary? I haven't done much with the NumPy internals, but I'd think that it would be almost trivial to write a simple extension to do multi_set(array, indices, values) by implementing a C for-loop over the indices. Is there a hidden subtlety that I'm missing? - Johann --- Johann A. Hibschman | Grad student in Physics, working in Astronomy. johann@physics.berkeley.edu | Probing pulsar pair production processes. _______________ MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org _______________ From sdm7g@Virginia.EDU Fri Jun 27 03:21:38 1997 From: sdm7g@Virginia.EDU (Steven D. Majewski) Date: Thu, 26 Jun 1997 22:21:38 -0400 (EDT) Subject: [MATRIX-SIG] reverse of take? In-Reply-To: <199706262324.QAA29257@curlew.geog.ubc.ca> Message-ID: I've only followed this discussion lightly ( in some, but perhaps not all of it's several incarnations ), but since others have mentioned how S and some other math languages support scatter-gather, I'll offer one more language for comparison: ( Mainly because the "plumbing" is more visible in this example. ) LispStat is a version of Lisp oriented to interactive statistics and statistical graphics. Some of the capabilities may have been inspired by S+ and some other numerical and statistical packages. LispStat extends many Lisp functions to vectorized versions, for example: >( * 10 ( iseq 10 )) (0 10 20 30 40 50 60 70 80 90) >( * ( iseq 10 ) ( iseq 10 )) (0 1 4 9 16 25 36 49 64 81) This includes comparison functions, which yield a list of boolean values: > ( setf x ( normal-rand 10 )) (0.38107432228578 1.96131809922812 2.36068411946823 1.19178363587238 -0.885422972877445 -0.564994188286439 1.04135377723368 -0.595475370701372 0.941913771141252 0.899280862970745) > ( > x 0 ) (T T T T NIL NIL T NIL T T) There is a function 'which', which yields a list of the indexes that are non-nil: > ( which ( > x 0 )) (0 1 2 3 6 8 9) and a select function that takes a sequence and either an index or a list of indexes: ( select x 3 ) 1.19178363587238 > ( select x ( which ( > x 0 ))) (0.38107432228578 1.96131809922812 2.36068411946823 1.19178363587238 1.04135377723368 0.941913771141252 0.899280862970745) And which can be used as an argument to SETF: ( setf ( select x ( which ( < x 0 ))) 0 ) 0 > x (0.38107432228578 1.96131809922812 2.36068411946823 1.19178363587238 0 0 1.04135377723368 0 0.941913771141252 0.899280862970745) The value arg to SETF can be a single value or a list of values, but if they are both lists, the index list and the value list have to match in length. ---| Steven D. Majewski (804-982-0831) |--- ---| Department of Molecular Physiology and Biological Physics |--- ---| University of Virginia Health Sciences Center |--- ---| P.O. Box 10011 Charlottesville, VA 22906-0011 |--- All power corrupts and obsolete power corrupts obsoletely." - Ted Nelson _______________ MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org _______________ From amullhau@ix.netcom.com Fri Jun 27 03:44:59 1997 From: amullhau@ix.netcom.com (Andrew P. Mullhaupt) Date: Thu, 26 Jun 1997 22:44:59 -0400 Subject: [MATRIX-SIG] reverse of take? In-Reply-To: <199706262324.QAA29257@curlew.geog.ubc.ca> References: <3.0.1.32.19970626181153.006ce0ec@popd.netcruiser> <199706262112.RAA20624@dante.mh.lucent.com> <3.0.1.32.19970626181153.006ce0ec@popd.netcruiser> Message-ID: <3.0.1.32.19970626224459.006cd598@popd.netcruiser> At 04:24 PM 6/26/97 -0700, Phil Austin wrote: >>>>>> "AM" == Andrew P Mullhaupt writes: > AM> This kind of scatter > AM> is really neat and one of the attractions of the S language. S > AM> has by far the nicest syntax for indexing of all the array > AM> languages, > At the moment I'm trying to >translate various legacy S constructs into take, repeat, reduce >combinations--it feels like I'm taking (and failing) some kind of IQ >test. Emigrating from S is likely to be a frightening experience unless you have experience with another interpreted language. Although python (with Numerical Extensions) and S are actually very close in most respects, you are likely to get hung up on the limited indexing of Python and the very different semantics of argument processing _despite the identical syntax_. Until something is done about the indexing you really ought to avoid _transliterating_ S into python; rather you should recast the algorithm into a more python-friendly form. Keep in mind that python loops are a _lot_ less punitive than S loops, and although python uses an obsolete approach to memory management, it is a _good_ implementation of that approach, as opposed to S, which uses a combination of bad implementation and demented design decisions to arrive at what may very well be the worst memory behavior of any actually useful program. (Or, as my new Director of Technology put it, "worst memory behavior of any actually useful program not used for testing memory systems....") This means that you don't need to jump through some of the same memory-saving hoops with python as with S, and this means that explicit loops which may contain temporary variables are not catastrophic, as they are in S. So although it is a pain to write all that code where a few index objects used to suffice, python can usually get the job done. You're going to be in a minority - you're coming to python programming from a language which offers you a lot more in the way of comfortable operations than python instead of coming from medieval torture chambers like C or Fortran, which offer so much less. It does seem, though, that several people have mentioned moving from S to python recently, so maybe it is a growing minority. Some of us have thought about doing a data.frame class for python. Would you be interested? Later, Andrew Mullhaupt _______________ MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org _______________ From amullhau@ix.netcom.com Fri Jun 27 04:07:10 1997 From: amullhau@ix.netcom.com (Andrew P. Mullhaupt) Date: Thu, 26 Jun 1997 23:07:10 -0400 Subject: [MATRIX-SIG] reverse of take? In-Reply-To: References: <3.0.1.32.19970626181153.006ce0ec@popd.netcruiser> Message-ID: <3.0.1.32.19970626230710.006d5054@popd.netcruiser> At 06:07 PM 6/26/97 -0700, Johann Hibschman wrote: >On Thu, 26 Jun 1997, Andrew P. Mullhaupt wrote: > >> This could easily be added to NumPy (along with a couple of other really >> useful forms of indexing). The long history of successful use of the >> highly flexible indexing in S is a 'proof of concept'. > >Hmm. Would it be acceptable to define (in a separate extension module) >a function to do this, or do you feel that the built-in syntax is >necessary? > >I haven't done much with the NumPy internals, but I'd think that it would >be almost trivial to write a simple extension to do > > multi_set(array, indices, values) > >by implementing a C for-loop over the indices. Is there a hidden subtlety >that I'm missing? There are a couple points. First, it _looks_ a lot better to anyone who has used an array language to do array[indices] = values instead. It is also a _lot_ easier to remember, and it does the expected thing. A function such as "multi_set" may or may not be suggestive to the programmer. Consider that the "function style" code for z[k] = foo(x[k], y[k]) looks something like multi_set(z, k, foo(multi_get(x, k), multi_get(y, k))) and we haven't even started to deal with anything slippery. Second, you want to put this operation where it will be exposed to inheritance, etc. so you want it to be a _method_. Since it conceptually replaces an existing method, it's better to use that syntax. I think you can get this pretty easily with __setitem__ or __setslice__, so the syntax shouldn't really be an issue. Later, Andrew Mullhaupt _______________ MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org _______________ From hinsen@ibs.ibs.fr Fri Jun 27 10:18:29 1997 From: hinsen@ibs.ibs.fr (Konrad Hinsen) Date: Fri, 27 Jun 1997 11:18:29 +0200 Subject: [MATRIX-SIG] reverse of take? In-Reply-To: <3.0.1.32.19970626230710.006d5054@popd.netcruiser> (amullhau@ix.netcom.com) Message-ID: <199706270918.LAA15972@lmspc1.ibs.fr> > existing method, it's better to use that syntax. I think you can get this > pretty easily with __setitem__ or __setslice__, so the syntax shouldn't > really be an issue. Indeed. A single subscript of any type is passed to __getitem__: x[[1, 3, 5]] --> x.__getitem__([1, 3, 5]) Several subscripts are passed as a tuple: x[[1, 3, 5], [3, 2, 5]] --> x.__getitem__(([1, 3, 5], [3, 2, 5])) Of course __setitem__ works in the same way. For C types it's a bit more complicated since integers are passed via the sequence interface whereas everything else gets passed via the mapping interface, but it's not a real problem either. So the difficult part is not syntax, but implementing the indexing operations in a general and efficient way. -- ------------------------------------------------------------------------------- Konrad Hinsen | E-Mail: hinsen@ibs.ibs.fr Laboratoire de Dynamique Moleculaire | Tel.: +33-4.76.88.99.28 Institut de Biologie Structurale | Fax: +33-4.76.88.54.94 41, av. des Martyrs | Deutsch/Esperanto/English/ 38027 Grenoble Cedex 1, France | Nederlands/Francais ------------------------------------------------------------------------------- _______________ MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org _______________ From motteler@laura.llnl.gov Fri Jun 27 15:37:15 1997 From: motteler@laura.llnl.gov (Zane C. Motteler) Date: Fri, 27 Jun 1997 07:37:15 -0700 (PDT) Subject: [MATRIX-SIG] reverse of take? In-Reply-To: <199706262012.QAA16688@dante.mh.lucent.com> Message-ID: On Thu, 26 Jun 1997, Aaron Watters wrote: >Okay, stupid question time. No such thing. >Been fiddling with numeric recently and I can't seem >to find or derive an "inverse" to take. Ie, I'd like a >builtin that will set a collection of values scattered about >inside an array. You and me both. >or whatever. Sorry if this kinda thing >is already there. Please educate me. -- Aaron Watters You're right, Numeric has no such capability. I needed this almost from the first, and in fact I have written a handy-dandy little C extension called arrayfnsmodule which contains this and a number of other Yorick-like capabilities that I found necessary. Our anonymous ftp site contains an entire python1.4 distribution which contains not only this module, but also graphics modules which connect to Gist and Narcisse graphics engines (among other things). The ftp site is ftp-icf.llnl.gov; look in /ftp/pub/python/busby for python1.4+patches+NumPy.tgz. Just for the heck of it, I have put arrayfnsmodule.c.gz into /ftp/pub/python/motteler. Feel free to use whatever you need. Zane ----------------------------------------------------------------------------- Zane C. Motteler, Ph. D. ___________ALSO:_________________ Computer Scientist | Professor Emeritus of Computer| Lawrence Livermore National Laboratory | Science and Engineering | P O Box 808, L-472 | California Polytechnic State | Livermore, CA 94551-9900 | University, San Luis Obispo | 510/423-2143 --------------------------------- FAX 510/423-9969 zcm@llnl.gov or motteler@laura.llnl.gov or zmottel@calpoly.edu _______________ MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org _______________ From motteler@laura.llnl.gov Fri Jun 27 15:48:50 1997 From: motteler@laura.llnl.gov (Zane C. Motteler) Date: Fri, 27 Jun 1997 07:48:50 -0700 (PDT) Subject: [MATRIX-SIG] reverse of take? In-Reply-To: Message-ID: On Thu, 26 Jun 1997, Johann Hibschman wrote: > >Hmm. Would it be acceptable to define (in a separate extension module) >a function to do this, or do you feel that the built-in syntax is >necessary? > Been there, done that. See my earlier communication to Aaron Watters. Zane ----------------------------------------------------------------------------- Zane C. Motteler, Ph. D. ___________ALSO:_________________ Computer Scientist | Professor Emeritus of Computer| Lawrence Livermore National Laboratory | Science and Engineering | P O Box 808, L-472 | California Polytechnic State | Livermore, CA 94551-9900 | University, San Luis Obispo | 510/423-2143 --------------------------------- FAX 510/423-9969 zcm@llnl.gov or motteler@laura.llnl.gov or zmottel@calpoly.edu _______________ MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org _______________ From hochberg@wwa.com Fri Jun 27 16:02:35 1997 From: hochberg@wwa.com (Timothy A. Hochberg) Date: Fri, 27 Jun 1997 10:02:35 -0500 (CDT) Subject: [MATRIX-SIG] Binaries for NumPy In-Reply-To: <337998C1.43FFF0F5@linux-magazin.de> Message-ID: I just upgraded to linux 2.0 on my linux box, and lo and behold I was able to build NumPy and NumPyLib as clean extensions (something I hadn't been able to get working before for whatever reason....) And it got me thinking that if we could provide pre-built clean extensions for common unix platforms, we could make installing NumPy really simple for people. On Linux the install instructions would be: (A distribution consisting of the .so and .py (and .pyc?) files in a directory called, say, NumPyDist would be provided in a tar.gz file.) 1. Unpack the distribution in some directory we'll call (e.g., /usr/local/src). 2. Add the following two lines to the end of your .bashrc and/or .bash_login files (assuming you use bash, otherwise....) export PYTHONPATH=$PYTHONPATH:/NumPyDist export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/NumPyDist Or something like that... I think this would really lower the investment in time needed to try out NumPy and consequently we'd end up with more users. What do other people think? How many platforms would we be able to round up binaries for? Where would we put it? ____ /im +------------------------------------------------+ |Tim Hochberg Research Assistant | |hochberg wwa.com University of Illinois | | Electrical Engineering | +------------------------------------------------+ _______________ MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org _______________ From hugunin@javanet.com Fri Jun 27 16:14:36 1997 From: hugunin@javanet.com (Jim Hugunin) Date: Fri, 27 Jun 1997 11:14:36 -0400 Subject: [MATRIX-SIG] Binaries for NumPy References: Message-ID: <33B3D8DC.A1DA3DC0@javanet.com> Timothy A. Hochberg wrote: > I think this would really lower the investment in time needed to try out > NumPy and consequently we'd end up with more users. What do other people > think? How many platforms would we be able to round up binaries for? Where > would we put it? This was one of my goals initially, but I got really intimidated when I realized just how many platforms there were out there. Nonetheless, doing this for a few major platforms would probably be a good idea. I'm moving the Numeric Python home pages to the Starship Python web site real soon now, and I think that would also be an appropriate place from which to distribute these binaries. -Jim _______________ MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org _______________ From hinsen@ibs.ibs.fr Fri Jun 27 17:05:52 1997 From: hinsen@ibs.ibs.fr (Konrad Hinsen) Date: Fri, 27 Jun 1997 18:05:52 +0200 Subject: [MATRIX-SIG] reverse of take? In-Reply-To: (motteler@laura.llnl.gov) Message-ID: <199706271605.SAA17557@lmspc1.ibs.fr> > Our anonymous ftp site contains an entire python1.4 distribution which > contains not only this module, but also graphics modules which connect > to Gist and Narcisse graphics engines (among other things). The ftp site Sounds interesting. Could you perhaps provide that stuff separately? The chance that I will ever be able to download a full Python distribution at 100 bytes/second plus interruptions is rather small! Besides, having to search for additions in a full distribution is not very convenient. -- ------------------------------------------------------------------------------- Konrad Hinsen | E-Mail: hinsen@ibs.ibs.fr Laboratoire de Dynamique Moleculaire | Tel.: +33-4.76.88.99.28 Institut de Biologie Structurale | Fax: +33-4.76.88.54.94 41, av. des Martyrs | Deutsch/Esperanto/English/ 38027 Grenoble Cedex 1, France | Nederlands/Francais ------------------------------------------------------------------------------- _______________ MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org _______________ From acura@vcn.bc.ca Sat Jun 28 00:35:18 1997 From: acura@vcn.bc.ca (Kelvin Lee) Date: Fri, 27 Jun 1997 16:35:18 -0700 (PDT) Subject: [MATRIX-SIG] newbie question: convert array of floats to integer In-Reply-To: <199706271605.SAA17557@lmspc1.ibs.fr> Message-ID: Hopefully this doesn't sound like too naive a question... What's the easiest way to change the typecode of an array? Kelvin _______________ MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org _______________ From amullhau@ix.netcom.com Sat Jun 28 01:20:53 1997 From: amullhau@ix.netcom.com (Andrew P. Mullhaupt) Date: Fri, 27 Jun 1997 20:20:53 -0400 Subject: [MATRIX-SIG] reverse of take? In-Reply-To: <199706270918.LAA15972@lmspc1.ibs.fr> References: <3.0.1.32.19970626230710.006d5054@popd.netcruiser> Message-ID: <3.0.1.32.19970627202053.006d96dc@popd.netcruiser> At 11:18 AM 6/27/97 +0200, Konrad Hinsen wrote: >> existing method, it's better to use that syntax. I think you can get this >> pretty easily with __setitem__ or __setslice__, so the syntax shouldn't >> really be an issue. > >Indeed. A single subscript of any type is passed to __getitem__: > > x[[1, 3, 5]] --> x.__getitem__([1, 3, 5]) > >Several subscripts are passed as a tuple: > > x[[1, 3, 5], [3, 2, 5]] --> x.__getitem__(([1, 3, 5], [3, 2, 5])) > >Of course __setitem__ works in the same way. For C types it's a bit >more complicated since integers are passed via the sequence interface >whereas everything else gets passed via the mapping interface, but >it's not a real problem either. Yes. It looks pretty good. The questions that spring to mind are related to problems which might arise in implementing a python class which gives S vector and data.frame functionality. You can provide a vector in S with a vector of character strings and then use the strings to index the original vector. Higher dimensional S arrays inherit this property. The cool thing about S data.frame objects is that they inherit from both S arrays and S lists, (S lists being pretty much like python dictionaries with the column names being the keys and the columns being the values). It is a surprisingly cool object to have around, and so it is something you would want in python. The reservations about expanding python indexing are related to how much of this picture can be achieved with various choices for python indexing. >So the difficult part is not syntax, but implementing the indexing >operations in a general and efficient way. That's not really the hard part at all, since it's been done to death in the APL family of languages. See, for example, Budd's book _An APL Compiler_ for a taste of what's possible. If you really want to get out the hammer and tongs, you can actually implement indexing well enough to beat all but impractically complicated compiled code - the interpreter gives you a very nice way of re-using memory bandwidth strategies which are too difficult for general application in C or Fortran. This is one of the opportunities for interpreters to be faster than some well written compiled code, although this doesn't seem to be generally appreciated outside the post-APL community. Later, Andrew Mullhaupt _______________ MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org _______________ From da@maigret.cog.brown.edu Sat Jun 28 03:34:06 1997 From: da@maigret.cog.brown.edu (David Ascher) Date: Fri, 27 Jun 1997 22:34:06 -0400 (EDT) Subject: [MATRIX-SIG] reverse of take? In-Reply-To: <3.0.1.32.19970627202053.006d96dc@popd.netcruiser> Message-ID: On Fri, 27 Jun 1997, Andrew P. Mullhaupt wrote: > At 11:18 AM 6/27/97 +0200, Konrad Hinsen wrote: > >So the difficult part is not syntax, but implementing the indexing > >operations in a general and efficient way. > > That's not really the hard part at all, since it's been done to death in the > APL family of languages. So what *is* the hard part? --da _______________ MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org _______________ From jbaddor@phy.ulaval.ca Sat Jun 28 14:57:43 1997 From: jbaddor@phy.ulaval.ca (Jean-Bernard ADDOR) Date: Sat, 28 Jun 1997 09:57:43 -0400 (EDT) Subject: [MATRIX-SIG] newbie question: convert array of floats to integer In-Reply-To: Message-ID: try somethink like: new_array = Numeric.array(old_array, Numeric.Integer) A bientot, Jean-Bernard On Fri, 27 Jun 1997, Kelvin Lee wrote: > Hopefully this doesn't sound like too naive a question... What's the > easiest way to change the typecode of an array? > > Kelvin > > > > _______________ > MATRIX-SIG - SIG on Matrix Math for Python > > send messages to: matrix-sig@python.org > administrivia to: matrix-sig-request@python.org > _______________ > _______________ MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org _______________ From hinsen@ibs.ibs.fr Mon Jun 30 10:29:13 1997 From: hinsen@ibs.ibs.fr (Konrad Hinsen) Date: Mon, 30 Jun 1997 11:29:13 +0200 Subject: [MATRIX-SIG] newbie question: convert array of floats to integer In-Reply-To: (message from Jean-Bernard ADDOR on Sat, 28 Jun 1997 09:57:43 -0400 (EDT)) Message-ID: <199706300929.LAA27662@lmspc1.ibs.fr> > try somethink like: > > new_array = Numeric.array(old_array, Numeric.Integer) That will likely produce an error message. Use old_array.astype(Integer): >>> from Numeric import * from Numeric import * >>> a = array([1.,2.]) a = array([1.,2.]) >>> b = array(a, Int) b = array(a, Int) Traceback (innermost last): File "", line 1, in ? TypeError: Array can not be safely cast to required type >>> b = a.astype(Int) b = a.astype(Int) Konrad. -- ------------------------------------------------------------------------------- Konrad Hinsen | E-Mail: hinsen@ibs.ibs.fr Laboratoire de Dynamique Moleculaire | Tel.: +33-4.76.88.99.28 Institut de Biologie Structurale | Fax: +33-4.76.88.54.94 41, av. des Martyrs | Deutsch/Esperanto/English/ 38027 Grenoble Cedex 1, France | Nederlands/Francais ------------------------------------------------------------------------------- _______________ MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org _______________ From hinsen@ibs.ibs.fr Mon Jun 30 11:00:27 1997 From: hinsen@ibs.ibs.fr (Konrad Hinsen) Date: Mon, 30 Jun 1997 12:00:27 +0200 Subject: [MATRIX-SIG] reverse of take? In-Reply-To: (message from David Ascher on Fri, 27 Jun 1997 22:34:06 -0400 (EDT)) Message-ID: <199706301000.MAA27739@lmspc1.ibs.fr> > On Fri, 27 Jun 1997, Andrew P. Mullhaupt wrote: > > > At 11:18 AM 6/27/97 +0200, Konrad Hinsen wrote: > > >So the difficult part is not syntax, but implementing the indexing > > >operations in a general and efficient way. > > > > That's not really the hard part at all, since it's been done to death in the > > APL family of languages. > > So what *is* the hard part? As always: Finding someone to do the actual work! -- ------------------------------------------------------------------------------- Konrad Hinsen | E-Mail: hinsen@ibs.ibs.fr Laboratoire de Dynamique Moleculaire | Tel.: +33-4.76.88.99.28 Institut de Biologie Structurale | Fax: +33-4.76.88.54.94 41, av. des Martyrs | Deutsch/Esperanto/English/ 38027 Grenoble Cedex 1, France | Nederlands/Francais ------------------------------------------------------------------------------- _______________ MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org _______________ From hinsen@ibs.ibs.fr Mon Jun 30 12:51:58 1997 From: hinsen@ibs.ibs.fr (Konrad Hinsen) Date: Mon, 30 Jun 1997 13:51:58 +0200 Subject: [MATRIX-SIG] reverse of take? In-Reply-To: <3.0.1.32.19970627202053.006d96dc@popd.netcruiser> (amullhau@ix.netcom.com) Message-ID: <199706301151.NAA28052@lmspc1.ibs.fr> > Yes. It looks pretty good. The questions that spring to mind are related > to problems which might arise in implementing a python class which gives > S vector and data.frame functionality. You can provide a vector in S with > a vector of character strings and then use the strings to index the original > vector. Higher dimensional S arrays inherit this property. The cool thing For non-integer indexing I usually use a dictionary associated with an array, which stores the mapping from the generalized index to an integer. It wouldn't be hard to wrap this up in a class. Well, what the heck, let's do it: import Numeric class NamedIndexArray: def __init__(self, type, *indices): self.array = Numeric.zeros(tuple(map(len, indices)), type) self.indices = [] for index_list in indices: dict = {} self.indices.append(dict) for i in range(len(index_list)): dict[index_list[i]] = i def __getitem__(self, item): if type(item) != type(()): item = (item, ) index = tuple(map(lambda d, i: d[i], self.indices, item)) return self.array[index] def __setitem__(self, item, value): if type(item) != type(()): item = (item, ) index = tuple(map(lambda d, i: d[i], self.indices, item)) self.array[index] = value days = NamedIndexArray(Numeric.Int, ['Jan', 'Feb', 'Mar']) days['Jan'] = 31 days['Feb'] = 28 days['Mar'] = 31 There's more luxury to add, but that's not difficult either. The only restriction is that keys should not be tuples, since there is no difference between a[(2, 3)] and a[2, 3]. > That's not really the hard part at all, since it's been done to death in the > APL family of languages. See, for example, Budd's book _An APL Compiler_ for Right, but knowledge about the techniques seems to be limited to that community, which is pretty small. It takes some effort to dig it out and write the actual code... -- ------------------------------------------------------------------------------- Konrad Hinsen | E-Mail: hinsen@ibs.ibs.fr Laboratoire de Dynamique Moleculaire | Tel.: +33-4.76.88.99.28 Institut de Biologie Structurale | Fax: +33-4.76.88.54.94 41, av. des Martyrs | Deutsch/Esperanto/English/ 38027 Grenoble Cedex 1, France | Nederlands/Francais ------------------------------------------------------------------------------- _______________ MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org _______________ From arw@dante.mh.lucent.com Mon Jun 30 15:45:21 1997 From: arw@dante.mh.lucent.com (Aaron Watters) Date: Mon, 30 Jun 1997 10:45:21 -0400 Subject: [MATRIX-SIG] reverse of take? Message-ID: <199706301443.KAA23238@dante.mh.lucent.com> > You're right, Numeric has no such capability. I needed this almost from > the first, and in fact I have written a handy-dandy little C extension > called arrayfnsmodule which contains this and a number of other > Yorick-like capabilities that I found necessary. I don't know what issues are here (licensing, whatever) but I vote that arrayfnsmodule be added as to the standard Numeric distribution for this capability alone, ala the Fourier stuff. It's pretty important to my mind. Take should have an inverse, and in my case I am fiddling during spare moments on a win95 box, and I don't have time right now to learn how to build other peoples stuff there. I suspect old S hackers might run into something like this in NumPy and say "huh?" and go back to S. Speaking as an arrogant newbie. -- Aaron Watters _______________ MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org _______________ From hinsen@ibs.ibs.fr Mon Jun 30 16:17:23 1997 From: hinsen@ibs.ibs.fr (Konrad Hinsen) Date: Mon, 30 Jun 1997 17:17:23 +0200 Subject: [MATRIX-SIG] reverse of take? In-Reply-To: <199706301443.KAA23238@dante.mh.lucent.com> (arw@dante.mh.lucent.com) Message-ID: <199706301517.RAA28804@lmspc1.ibs.fr> > I don't know what issues are here (licensing, whatever) but I vote > that arrayfnsmodule be added as to the standard Numeric distribution > for this capability alone, ala the Fourier stuff. It's pretty important to > my mind. Fine, then get and use it. I wouldn't put it into the standard distribution before a definite syntax for list indexing has been agreed on and implemented. -- ------------------------------------------------------------------------------- Konrad Hinsen | E-Mail: hinsen@ibs.ibs.fr Laboratoire de Dynamique Moleculaire | Tel.: +33-4.76.88.99.28 Institut de Biologie Structurale | Fax: +33-4.76.88.54.94 41, av. des Martyrs | Deutsch/Esperanto/English/ 38027 Grenoble Cedex 1, France | Nederlands/Francais ------------------------------------------------------------------------------- _______________ MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org _______________ From arw@dante.mh.lucent.com Mon Jun 30 17:49:52 1997 From: arw@dante.mh.lucent.com (Aaron Watters) Date: Mon, 30 Jun 1997 12:49:52 -0400 Subject: [MATRIX-SIG] reverse of take? Message-ID: <199706301646.MAA01562@dante.mh.lucent.com> As was mentioned, I don't believe syntax is an issue -- once you have the Python builtin functionality you can grow your own syntax using the __getattr__ and __setattr__ hooks at the Python level. What we need is a nice builtin, which I think Zane has already implemented. -- Aaron Watters ---------- > From: Konrad Hinsen > To: arw@mh.lucent.com > Cc: motteler@laura.llnl.gov; matrix-sig@python.org > Subject: Re: [MATRIX-SIG] reverse of take? > Date: Monday, June 30, 1997 11:17 AM > > > I don't know what issues are here (licensing, whatever) but I vote > > that arrayfnsmodule be added as to the standard Numeric distribution > > for this capability alone, ala the Fourier stuff. It's pretty important to > > my mind. > > Fine, then get and use it. I wouldn't put it into the standard > distribution before a definite syntax for list indexing has been > agreed on and implemented. > -- > ------------------------------------------------------------------------------- > Konrad Hinsen | E-Mail: hinsen@ibs.ibs.fr > Laboratoire de Dynamique Moleculaire | Tel.: +33-4.76.88.99.28 > Institut de Biologie Structurale | Fax: +33-4.76.88.54.94 > 41, av. des Martyrs | Deutsch/Esperanto/English/ > 38027 Grenoble Cedex 1, France | Nederlands/Francais > ------------------------------------------------------------------------------- > _______________ MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org _______________ From hinsen@ibs.ibs.fr Mon Jun 30 18:08:35 1997 From: hinsen@ibs.ibs.fr (Konrad Hinsen) Date: Mon, 30 Jun 1997 19:08:35 +0200 Subject: [MATRIX-SIG] reverse of take? In-Reply-To: <199706301646.MAA01562@dante.mh.lucent.com> (arw@dante.mh.lucent.com) Message-ID: <199706301708.TAA29271@lmspc1.ibs.fr> > As was mentioned, I don't believe syntax is > an issue -- once you have the Python builtin > functionality you can grow your own syntax > using the __getattr__ and __setattr__ hooks > at the Python level. What we need is a nice Not for arrays, which are C types! And these arrays are typically used directly, without a Python wrapper. So getting the syntax right means modifying arrayobject.c and putting the new code into the right module etc. Probably not much work, but it has to be done, and should be synchronized with possible changes that Jim has applied to the same module recently. Konrad. -- ------------------------------------------------------------------------------- Konrad Hinsen | E-Mail: hinsen@ibs.ibs.fr Laboratoire de Dynamique Moleculaire | Tel.: +33-4.76.88.99.28 Institut de Biologie Structurale | Fax: +33-4.76.88.54.94 41, av. des Martyrs | Deutsch/Esperanto/English/ 38027 Grenoble Cedex 1, France | Nederlands/Francais ------------------------------------------------------------------------------- _______________ MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org _______________ From arw@dante.mh.lucent.com Mon Jun 30 19:45:53 1997 From: arw@dante.mh.lucent.com (Aaron Watters) Date: Mon, 30 Jun 1997 14:45:53 -0400 Subject: [MATRIX-SIG] reverse of take? Message-ID: <199706301843.OAA03287@dante.mh.lucent.com> > Not for arrays, which are C types! And these arrays are typically used > directly, without a Python wrapper. So getting the syntax right means > modifying arrayobject.c and putting the new code into the right module > etc. Probably not much work, but it has to be done, and should be > synchronized with possible changes that Jim has applied to the same > module recently. okey dokey. (sorry 2 beers at lunch). I propose that arraytype[arraytype] automatically be interpreted as (1) gather as a value (2) scatter as an l-value. I apologize if this is obviously dumb, incompatible with existing usage, or already proposed (I'm new, remember, don't hurt me!) :). -- Aaron Watters _______________ MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org _______________ From motteler@laura.llnl.gov Mon Jun 30 20:32:41 1997 From: motteler@laura.llnl.gov (Zane C. Motteler) Date: Mon, 30 Jun 1997 12:32:41 -0700 (PDT) Subject: [MATRIX-SIG] reverse of take? In-Reply-To: <199706301843.OAA03287@dante.mh.lucent.com> Message-ID: On Mon, 30 Jun 1997, Aaron Watters wrote: >okey dokey. (sorry 2 beers at lunch). >I propose that arraytype[arraytype] automatically be interpreted >as (1) gather as a value (2) scatter as an l-value. I apologize if >this is obviously dumb, incompatible with existing usage, or >already proposed (I'm new, remember, don't hurt me!) :). Aaron, I agree with you and I'm not even new. My "scatter" is not so general, however. With a little work it could be. Now all I need is the time. Zane ----------------------------------------------------------------------------- Zane C. Motteler, Ph. D. ___________ALSO:_________________ Computer Scientist | Professor Emeritus of Computer| Lawrence Livermore National Laboratory | Science and Engineering | P O Box 808, L-472 | California Polytechnic State | Livermore, CA 94551-9900 | University, San Luis Obispo | 510/423-2143 --------------------------------- FAX 510/423-9969 zcm@llnl.gov or motteler@laura.llnl.gov or zmottel@calpoly.edu _______________ MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org _______________