From dubois1@llnl.gov Mon Feb 1 05:04:45 1999 From: dubois1@llnl.gov (Paul F. Dubois) Date: Sun, 31 Jan 1999 21:04:45 -0800 Subject: [Matrix-SIG] Re: Why is there fast_umathmodule.c and umathmodule.c ? Message-ID: <001201be4da0$62d22d40$f4160218@c1004579-c.plstn1.sfba.home.com> The intention was to have the difference mentioned. But it was never implemented. You don't normally control the behavior of the floating point unit that way anyway. Unfortunately there is no standard for doing it. I plan to simply remove fast_umathmodule at the next release unless I get convinced otherwise. -----Original Message----- From: Travis E. Oliphant To: Alan Grossfield ; matrix-sig@python.org Date: Saturday, January 30, 1999 1:56 PM Subject: [Matrix-SIG] Re: Why is there fast_umathmodule.c and umathmodule.c ? >> >> :I have been looking at and using the source from the umathmodules and >> :noticed that fast_umathmodule and umathmodule are nearly identical. >> :Perhaps somebody could tell me why there two diffeerent versions of >> :basically the same code?? >> IIRC, umath raises exception on overflows etc, fast_umath just returns >> NaN. Some speed vs safety. > >Thanks for the quick response. I see the intention, now. I'm a bit >confused at how this is done, though since the latest release of Numeric >shows only the following differences. >[olipt@amylou Src]$ diff fast_umathmodule.c umathmodule.c >6d5 >< >1982c1981 >< void initfast_umath() { >--- >> void initumath() { >1986c1985 >< m = Py_InitModule("fast_umath", methods); >--- >> m = Py_InitModule("umath", methods); > >These seem like only superficial differences. I don't see how the two >modules can be functionally different, here. > >_______________________________________________ >Matrix-SIG maillist - Matrix-SIG@python.org >http://www.python.org/mailman/listinfo/matrix-sig > > From Oliphant.Travis@mayo.edu Mon Feb 1 04:36:28 1999 From: Oliphant.Travis@mayo.edu (Travis E. Oliphant) Date: Sun, 31 Jan 1999 22:36:28 -0600 Subject: [Matrix-SIG] Typo in multiarraymodule.c Message-ID: <36B52F4C.98C69049@mayo.edu> This is a multi-part message in MIME format. --------------C8465BD51365940AFDBF9163 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit I think I found a typo in multiarraymodule.c in the latest 1.9 release that messes up the functionality of convolve. Try convolve([1,2],[1,2,3],2) and convolve([1,2,3],[1,2],2). These should give the same result but they don't in my NumPy. The fix is simple (change a 1 to an i) and a one-line diff is attached. -Travis -- ---------------------------------------------------- Travis Oliphant 200 First St SW Rochester MN 55905 Ultrasound Research Lab (507) 286-5923 Mayo Graduate School Oliphant.Travis@mayo.edu --------------C8465BD51365940AFDBF9163 Content-Type: text/plain; charset=us-ascii; name="multidiff" Content-Transfer-Encoding: 7bit Content-Disposition: inline; filename="multidiff" 692,693c692 < if (n1 < n2) { ret = ap1; ap1 = ap2; ap2 = ret; ret = NULL; i = n1;n1=n2;n2=i;} < --- > if (n1 < n2) { ret = ap1; ap1 = ap2; ap2 = ret; ret = NULL; i = n1;n1=n2;n2=1;} 717a717 > --------------C8465BD51365940AFDBF9163-- From gathmann@scar.utoronto.ca Tue Feb 2 14:32:11 1999 From: gathmann@scar.utoronto.ca (Oliver Gathmann) Date: Tue, 2 Feb 1999 09:32:11 -0500 (EST) Subject: [Matrix-SIG] NumPy arrays with object typecode Message-ID: Hello, I am working on something like an S-Plus frame that would allow efficient access to data of different types (integers, floats, and strings) organized in colums of an ordinary 2d-array. My idea is to use NumPy arrays of typecode 'O'; I know this sacrifices a lot of the performance of genuine numerical arrays, but they still offer their nifty indexing and slicing capabilities. Strangely, NumPy (LLNL v.9) does only seem to allow assigning to slices along the second axis, not along the first: Python 1.5.2b1 (#2, Jan 5 1999, 14:21:49) [GCC 2.7.2.3] on linux2 Copyright 1991-1995 Stichting Mathematisch Centrum, Amsterdam >>> from Numeric import * >>> objArray = zeros((3,2),'O') >>> objArray array([[0 , 0 ], [0 , 0 ], [0 , 0 ]],'O') >>> intArray = reshape(arange(6),(3,2)) >>> intArray array([[0, 1], [2, 3], [4, 5]]) >>> objArray[1,:] = intArray[1,:] >>> objArray array([[0 , 0 ], [2 , 3 ], [0 , 0 ]],'O') >>> objArray[:,1] = intArray[:,1] Segmentation fault (core dumped) Explicitly converting the integer slice into an object slice doesn't help, either. What's going on here? I also find this counter-intuitive: >>> objArrayFromInits = array(['1','12','123'],'O') >>> objArrayFromInits array([[1 , 1 , 1 ], [12 , 12 , 12 ], [123 , 123 , 123 ]],'O') Is there any efficient way to have the array constructor treat the individual strings as objects and not as 1d arrays? Thanks, Oliver F. Oliver Gathmann (gathmann@scar.utoronto.ca) Surface and Groundwater Ecology Research Group University of Toronto phone: (416) - 287 7420 ; fax: (416) - 287 7423 web: http://www.scar.utoronto.ca/~gathmann From hinsen@cnrs-orleans.fr Tue Feb 2 17:41:06 1999 From: hinsen@cnrs-orleans.fr (Konrad Hinsen) Date: Tue, 2 Feb 1999 18:41:06 +0100 Subject: [Matrix-SIG] NumPy arrays with object typecode In-Reply-To: (message from Oliver Gathmann on Tue, 2 Feb 1999 09:32:11 -0500 (EST)) References: Message-ID: <199902021741.SAA13168@dirac.cnrs-orleans.fr> > Strangely, NumPy (LLNL v.9) does only seem to allow assigning to slices > along the second axis, not along the first: That looks like a bug... > I also find this counter-intuitive: > > >>> objArrayFromInits = array(['1','12','123'],'O') > >>> objArrayFromInits > array([[1 , 1 , 1 ], > [12 , 12 , 12 ], > [123 , 123 , 123 ]],'O') > > Is there any efficient way to have the array constructor treat the > individual strings as objects and not as 1d arrays? Not in the current NumPy. It generates one dimension for each nesting level of sequence types, and strings are of course sequence types. The only way out is to create the array with the right shape but some other content (e.g. None) and then assign the string values to the elements individually. Of course the array constructor could be modified, but 1) that creates compatibility problems and 2) there must be a way to construct character arrays from strings. -- ------------------------------------------------------------------------------- Konrad Hinsen | E-Mail: hinsen@cnrs-orleans.fr Centre de Biophysique Moleculaire (CNRS) | Tel.: +33-2.38.25.55.69 Rue Charles Sadron | Fax: +33-2.38.63.15.17 45071 Orleans Cedex 2 | Deutsch/Esperanto/English/ France | Nederlands/Francais ------------------------------------------------------------------------------- From barrett@compass.gsfc.nasa.gov Tue Feb 2 19:29:36 1999 From: barrett@compass.gsfc.nasa.gov (Paul Barrett) Date: Tue, 2 Feb 1999 14:29:36 -0500 (EST) Subject: [Matrix-SIG] NumPy arrays with object typecode In-Reply-To: References: Message-ID: <14007.17399.515521.723117@compass.gsfc.nasa.gov> Oliver Gathmann writes: > Hello, > > I am working on something like an S-Plus frame that would allow efficient > access to data of different types (integers, floats, and strings) > organized in colums of an ordinary 2d-array. My idea is to use NumPy > arrays of typecode 'O'; I know this sacrifices a lot of the performance of > genuine numerical arrays, but they still offer their nifty indexing and > slicing capabilities. > I'm currently working on a C extension class to do just this. I hope to have a useable, though by no means complete, version by the end of the week. Given an array shape and format string (similar to that used by pack and unpack), a recarray object is created which allows access to the data by array syntax. This module is really just a generalization of the struct module, but allows much more efficient access to large arrays of binary data. A paper by Barrett and Bridgman (see http://www.dpt.stsci.edu/dpt_papers/opus_bib.html) presented last November at the Astronomical Data Analysis Software and Systems Conference gives a synopsis of this extension module as it relates to PyFITS, a Python module for reading and writing FITS format files. -- Paul From Oliphant.Travis@mayo.edu Tue Feb 2 23:14:24 1999 From: Oliphant.Travis@mayo.edu (Travis Oliphant) Date: Tue, 2 Feb 1999 17:14:24 -0600 (EST) Subject: [Matrix-SIG] N-D Convolution routine Message-ID: One of the things that I think NumPy could really benefit from is a good set of Signal/Image/Array Processing routines. One fundamental part of that area is a general N-D convolution routine. I have a first version of an N-D convolution routine that I am proposing for inclusion in the multiarraymodule.c source code. I have tested it and it seems to work for all supported types and is useable now. At some time I would like it to be polished off with some appropriate broadcasting rules, so that one could, for example, do 2-D convolution on each plane of a 3-D volume with one statement. If anyone is interested in this routine which I have called convolveND send me email, or post to this list. What would be the method of getting it included in the multiarraymodule proper?? Thanks, Travis Oliphant.Travis@mayo.edu ---------------------------------------------------- Travis Oliphant 200 First St SW Rochester MN 55905 Ultrasound Research Lab (507) 286-5293 Mayo Graduate School Oliphant.Travis@mayo.edu From Oliphant.Travis@mayo.edu Tue Feb 2 23:19:19 1999 From: Oliphant.Travis@mayo.edu (Travis Oliphant) Date: Tue, 2 Feb 1999 17:19:19 -0600 (EST) Subject: [Matrix-SIG] Another buglet in multiarraymodule? Message-ID: This message is in MIME format. The first part should be readable text, while the remaining parts are likely unreadable without MIME-aware tools. Send mail to mime@docserver.cac.washington.edu for more info. ---1333709111-817064630-917997559=:19948 Content-Type: TEXT/PLAIN; charset=US-ASCII While perusing the source to see how to write the N-D convolution routine I noticed that there appears to be a generally insignificant typo in the OBJECT_DotProduct routine but perhaps should be fixed. The diff file attached shows the change. ---------------------------------------------------- Travis Oliphant 200 First St SW Rochester MN 55905 Ultrasound Research Lab (507) 286-5293 Mayo Graduate School Oliphant.Travis@mayo.edu ---1333709111-817064630-917997559=:19948 Content-Type: TEXT/PLAIN; charset=US-ASCII; name=multidiffs Content-Transfer-Encoding: BASE64 Content-ID: Content-Description: Diff file showing buglet and fix. Content-Disposition: attachment; filename=multidiffs KioqIG11bHRpYXJyYXltb2R1bGUuYwlUdWUgRmViICAyIDE2OjQyOjI4IDE5 OTkNCi0tLSBtdWx0aWFycmF5bW9kdWxlLmMuYmFrCVR1ZSBGZWIgIDIgMTc6 MTY6MDEgMTk5OQ0KKioqKioqKioqKioqKioqDQoqKiogNTY2LDU3MiAqKioq DQogIAkJCQkJCQkgIGNoYXIgKm9wLCBpbnQgbikgew0KICAJaW50IGk7DQog IAlQeU9iamVjdCAqdG1wMSwgKnRtcDIsICp0bXA7DQohIAlmb3IoaT0wO2k8 bjtpKyssaXAxKz1pczEsaXAyKz1pczIpIHsgDQogIAkJdG1wMSA9IFB5TnVt YmVyX011bHRpcGx5KCooKFB5T2JqZWN0ICoqKWlwMSksICooKFB5T2JqZWN0 ICoqKWlwMikpOw0KICAJCWlmIChpID09IDApIHsNCiAgCQkJdG1wID0gdG1w MTsNCi0tLSA1NjUsNTcxIC0tLS0NCiAgCQkJCQkJCSAgY2hhciAqb3AsIGlu dCBuKSB7DQogIAlpbnQgaTsNCiAgCVB5T2JqZWN0ICp0bXAxLCAqdG1wMiwg KnRtcDsNCiEgCWZvcihpPTA7aTxuO2krKyxpcDErPWlzMixpcDIrPWlzMSkg eyANCiAgCQl0bXAxID0gUHlOdW1iZXJfTXVsdGlwbHkoKigoUHlPYmplY3Qg KiopaXAxKSwgKigoUHlPYmplY3QgKiopaXAyKSk7DQogIAkJaWYgKGkgPT0g MCkgew0KICAJCQl0bXAgPSB0bXAxOw0KLS0tIDExMzYsMTE0MSAtLS0tDQo= ---1333709111-817064630-917997559=:19948-- From Oliphant.Travis@mayo.edu Tue Feb 2 23:24:23 1999 From: Oliphant.Travis@mayo.edu (Travis Oliphant) Date: Tue, 2 Feb 1999 17:24:23 -0600 (EST) Subject: [Matrix-SIG] Sorry about last attachment Message-ID: Here is the attachment: *** multiarraymodule.c Tue Feb 2 16:42:28 1999 --- multiarraymodule.c.bak Tue Feb 2 17:16:01 1999 *************** *** 566,572 **** char *op, int n) { int i; PyObject *tmp1, *tmp2, *tmp; ! for(i=0;i I'm making available a new and improved version of N-D convolution. It is now much faster (about 5 times faster on large datasets) and has all of the mode options of 1-D convolve (0, 1, or 2) for ('valid', 'same', and 'full'). The routine is defined as the only method in a new module I would like to see called sigtools. I hope this will contain someday all (or most of) the filter routines some of us commonly use in MATLAB, IDL, etc.. The files are available at http://oliphant.netpedia.net/ Any contributions to this toolbox would be greatly appreciated. Anyone hiding any great signal, image, or array processing routines out there that could use a nice home? --Travis From Oliphant.Travis@mayo.edu Sun Feb 7 17:08:12 1999 From: Oliphant.Travis@mayo.edu (Travis E. Oliphant) Date: Sun, 07 Feb 1999 11:08:12 -0600 Subject: [Matrix-SIG] N-D convolution and N-D order filtering with Numeric python Message-ID: <36BDC87C.35770B37@mayo.edu> This is to let interested people who use the Numeric Extension to python know about an update to my sigtools module to version 0.20 In addition to the N-D convolution routine, the module now contains a routine to perform N-D arbitrary order statistic filtering. A median filter is an example of an order statistic filter and can now be implemented in a line of python code provided the data to be filtered is in a Python Sequence Object. It is available at http://oliphant.netpedia.net --Travis From jh@oobleck.tn.cornell.edu Mon Feb 8 21:55:50 1999 From: jh@oobleck.tn.cornell.edu (Joe Harrington) Date: Mon, 8 Feb 1999 16:55:50 -0500 Subject: [Matrix-SIG] Re: a full-blown interactive data analysis environment References: Message-ID: <199902082155.QAA08575@oobleck.tn.cornell.edu> Thanks to Travis Oliphant for urging me on. I've been dealing with yet another (and another after that!) IDL core dump that had stopped my work. Reminds me why I'm doing this! This is in response to the posts of 19Jan99... I appreciate the sentiments of Perry Greenfield and Paul DuBois, but I think they misinterpret what I am saying, or are missing a crucial point. Of course we should write our science analysis code in the interactive languages; that's why we're here in the first place! Nobody is suggesting we go back to C for that. That code is usually either fire-and-forget, or is only run in a few places, or both. It's only when transportability is the key issue that the logic swings the other way, and I'd cite IDL in astronomy as a prime piece of evidence in favor of not repeating that mistake yet again. IDL is a terrible language. The syntax is awful. It costs a thousand dollars for their cheapest license (try teaching a class on that!). The bug fix time is months, and all the other arguments against non-free software apply: too slow response from the developers (see above), you can't fix it yourself if it's broken, if you implement in it only people with a license to their product can run your code, etc. My goal here is to get a core of standard routines that are used in many fields of science into a coherent package with good docs, and to integrate it initially with Python. The point is that then *any* language, now or in the future, can be a "real" science language, just by interfacing to the package. At ADASS IV, we heard from a Sun presenter about how Java would soon rule the world and solve all our problems. When I asked about support for numerics, he said to an audience of several hundred astronomy programmers, "uhhh..use Fortran". With this package and a little work, they could easily extend their language to cover numerics, and do it well. So could Guile, Perl, and a host of others. Without such a package, language implementors look at the job of doing it themselves, and give up. Fortunately, it's not a matter (as Perry and Paul say or imply) of "expecting developers to confine themselves", it's a matter of finding developers who are interested in this approach. We don't have to worry about a "middle layer" of interpreted code from the developers. That's a winning strategy for science analysis projects, but a losing one for anything that is to survive long-term (meaning decades). I'm in full agreement with points a and d of Paul DuBois's posting, and the whole reason I'm trying to start this effort here is that I think it will strengthen NumPy to the point where it can become a usable language for the general scientist, and that it's the best available in terms of language capability. Marble is better than salt for building monuments, though salt is easier to work and tastes better. Such a project is significant. However, gathering and integrating is not more than twice as large a project as writing everything from scratch in the current interactive language. And it only needs to be done once, ever. I'm committed to creating a coherent package of widely-used numerical methods implemented as compiled-language routines and interfaced eventually to multiple languages. If another path is chosen, someone else will have to take on the task of coordinating it. Assuming that enough others are interested in this approach, here's what I see as the next steps: -Determine how and where to organize ourselves, and set those resources up (mailing lists, web sites, etc.). -Figure out how best to lay out the umbrella package, and what a "leaf" package looks like. The key item here is to make it flexible enough that when we have future interactive environments, we can just modify the install programs and have them create the environment around that new interpreter. -Identify the components we need in the first release (numerics: at least FFT, interpolation, simple fitting, a Romberg integrator; graphics: at least plotting image display, color table manipulation, cursor readback, and some form of widget). -Distribute the work of implementing the previous two items and of integration and testing. Looking at the first item, I'd be interested in a very quick list of software efforts we think have been successful that have been organized by volunteers working a few hours a week. Gimp springs to mind, and perhaps some of the window managers like Afterstep. Has anyone been involved in such an effort? Did they do anything unique that helped them out? For the second item, I see the following needs: documentation with images, formulae, and hyperlinks It seems reasonable to make whatever we use generate HTML, since all platforms support it. Do people think TeXinfo will stand the test of time? Its ability to handle formulae is attractive. Windows and Mac support Who has any experience building software here? Can we use make/autoconf with these systems and do they work well? How far away from g++ is C++ on the Mac and PC's various compilers? Is g++ a good thing to standardize upon? Do we need to pick a compiler and implement for it, or has C++ become standard enough not to worry? "leaf" packages Each should have its own docs, code, examples, test scripts, and wrapper generation files. Code and docs get built and installed into a location that might be shared with code, docs, and examples from other packages, for assembly into an intermediate collection of related packages (a "branch"?). This lets us distribute precompiled bite-sized package updates without recompiling or reinstalling the whole thing. There's lots more under this item, of course, but this should be enough to get a discussion started. For the third item, what do people see as *crucial* for a first release? In my view, the first release should define the structure we will be implementing into and provide several useful packages that can get people started with some basic tasks. It should have good docs, and generally carry the aura of something professional, not slapdash. I'd rather it be small and elegant than large and sloppy, as it's easier to grow larger than to fix a large amount of sloppiness, given the tendency of net programmers to jump on the successful bandwagons. --jh-- Joe Harrington 326 Space Sciences Building Cornell University Ithaca, NY 14853-6801 (607) 255-5913 office (607) 255-9002 fax jh@oobleck.tn.cornell.edu jh@alum.mit.edu (permanent) From Oliphant.Travis@mayo.edu Mon Feb 8 22:34:10 1999 From: Oliphant.Travis@mayo.edu (Travis Oliphant) Date: Mon, 8 Feb 1999 16:34:10 -0600 (EST) Subject: [Matrix-SIG] Re: a full-blown interactive data analysis environment In-Reply-To: <199902082155.QAA08575@oobleck.tn.cornell.edu> Message-ID: I am beginning now to see Joe's Point here better after having some experience building different types of modules. We definitely need to talk about what kind of things we expect from the language. For example, MATLAB defines only one datatype but NumPy defines about 10. For lowlevel operations which are coded in C this makes a difference. It seems that we will need to have separate language-independent and language-dependent code developed. I think it is a good idea to establish the expectations of code contributed to the project in terms of having the interpreted-language specific and non interpreted-language specific code separated. I'm beginning to see my released code in this light better. Most of what I've done is interpreted-language specific wrapping around already non-specific code (cephes, fftw). But it is often desirable to use language-specific features in the interface (like treating cephes functions as ufunc objects). I'm not sure that this should be or even can be generalized enough to eliminate the use (if not the need) for specific code that really doesn't contribute to the goal Joe has in mind. If this all sounds confusing, it is because I am a little confused as to how to actually make something like Joe suggests work. Are we talking about describing a generic interface to specialized code that will adapt itself to every future language? Can we guarantee that. Some languages will work, some won't with it, right? I guess what I understand now that I think is doable is writing packages with the idea of general usability in mind. For example, I'm going to try to change my N-D convolution and order-filtering routines so that the core routine assumes little about how the needed data is stored as an object in the interpreted language. This is not necessarily going to be easy (for me) and may not even be possible without sacrificing speed or usability in Python. This is the sort of difficulty I'm talking about in making a generic package that interfaces (well?) with any future scripting language. MATLAB defines their extensibility feature in terms of a GATEWAY procedure and the code. The GATEWAY handles all the MATLAB specific stuff and the procedure itself is portable. I could see doing something like that for this package. We would need a procedure for every supported datatype that way, but it could be done. This may not be possible with every desired addition, however (I'm not sure how this idea would change the cephesmodule I've released --- I guess all I did was write the GATEWAY for that one) I'd love to hear thoughts and ideas from people who have written code and linked it with interpreted languages. To move this forward we definitely need some experienced people to help define generic expectations from the interpreted language (with Python as a model, of course :-) ) and then just publish it somewhere and then start writing code and documentation. Best, Travis Oliphant From wtbridgman@radix.net Tue Feb 9 04:31:51 1999 From: wtbridgman@radix.net (W.T. Bridgman) Date: Mon, 8 Feb 1999 23:31:51 -0500 Subject: [Matrix-SIG] Re: a full-blown interactive data analysis environment In-Reply-To: <199902082155.QAA08575@oobleck.tn.cornell.edu> References: Message-ID: Hello, > > -Determine how and where to organize ourselves, and set those > resources up (mailing lists, web sites, etc.). > I just wanted to mention that I've been working with Paul Barrett on his 'PyFITS' stuff. Mostly I've been making sure that his binary table module will compile and run on a Mac and doing some testing with FITS data. He and I are in the process of bringing an Astronomical Python mailing list, WWW and FTP site online at Goddard. We've got some of the preliminaries out of the way and hope to send out an announcement at the end of this week or early next. As to astronomical analysis with Python, I'm very concerned about the plotting package issue. I've played a lot with Konrad Hinsen's TkPlotCanvas for doing simple plots. I'm now trying to come up-to-speed on Tk to really learn how to do a more flexible plotting package. However, I feel TkPlotCanvas is a good v0.1 plotting package for my current use. I'm using my personal e-mail account since I do most of my Python stuff at home (how much does an IDL license cost for a home system - an additional $500). However, I can also be reached at my lab account: bridgman@lheapop.gsfc.nasa.gov We'll let you know when something comes online. Tom From pfrazao@ualg.pt Tue Feb 9 15:21:26 1999 From: pfrazao@ualg.pt (Pedro Miguel =?iso-8859-1?Q?Fraz=E3o?= F. Ferreira) Date: Tue, 09 Feb 1999 15:21:26 +0000 Subject: [Matrix-SIG] Neural Networks Message-ID: <36C05276.6299952@ualg.pt> Hi All, Just subscribed. Just started looking to python. I am interested in code for Neural Networks (specialy RBF, GRBF). Is there any implementation ? Thanks -- ------------------------------------------------------------------------ Pedro Miguel Frazao Fernandes Ferreira, Universidade do Algarve U.C.E.H., Campus de Gambelas, 8000 - Faro, Portugal pfrazao@ualg.pt Tel.:+351 89 800950 / 872959 Fax: +351 89 818560 http://w3.ualg.pt/~pfrazao From hinsen@cnrs-orleans.fr Tue Feb 9 17:46:48 1999 From: hinsen@cnrs-orleans.fr (Konrad Hinsen) Date: Tue, 9 Feb 1999 18:46:48 +0100 Subject: [Matrix-SIG] Re: a full-blown interactive data analysis environment In-Reply-To: <199902082155.QAA08575@oobleck.tn.cornell.edu> (message from Joe Harrington on Mon, 8 Feb 1999 16:55:50 -0500) References: <199902082155.QAA08575@oobleck.tn.cornell.edu> Message-ID: <199902091746.SAA02706@dirac.cnrs-orleans.fr> > My goal here is to get a core of standard routines that are used in > many fields of science into a coherent package with good docs, and to > integrate it initially with Python. The point is that then *any* Sorry, but I still don't see what you are planning to do... My impression is that generic low-level libraries already exist. How exactly does your idea differ from combining LAPACK, FFTPACK, etc. into one tar file and write wrappers and documentation for all of it? Konrad. -- ------------------------------------------------------------------------------- Konrad Hinsen | E-Mail: hinsen@cnrs-orleans.fr Centre de Biophysique Moleculaire (CNRS) | Tel.: +33-2.38.25.55.69 Rue Charles Sadron | Fax: +33-2.38.63.15.17 45071 Orleans Cedex 2 | Deutsch/Esperanto/English/ France | Nederlands/Francais ------------------------------------------------------------------------------- From ryszard@moldyn.com Tue Feb 9 18:47:11 1999 From: ryszard@moldyn.com (Ryszard Czerminski) Date: Tue, 9 Feb 1999 13:47:11 -0500 (EST) Subject: [Matrix-SIG] Re: a full-blown interactive data analysis environment In-Reply-To: <199902091746.SAA02706@dirac.cnrs-orleans.fr> Message-ID: It would help a lot to start to develop some document which would describe what do we want from this new package (i.e. requirements). This can be a good basis for further discussion. It should probably include as well on some general level analysis and description of existing packages in this field (matlab, idl, root (http://root.cern.ch), scilab - to name the few) and what makes them inadequate enough to justify development of yet-another-data-analysis-package. One basic reason - at least for this group - is lack of Python front-end (:-). Ryszard Ryszard Czerminski phone : (617)354-3124 x 10 Moldyn, Inc. fax : (617)491-4522 955 Massachusetts Avenue e-mail: ryszard@moldyn.com Cambridge MA, 02139-3180 or ryszard@photon.com On Tue, 9 Feb 1999, Konrad Hinsen wrote: > > My goal here is to get a core of standard routines that are used in > > many fields of science into a coherent package with good docs, and to > > integrate it initially with Python. The point is that then *any* > > Sorry, but I still don't see what you are planning to do... > My impression is that generic low-level libraries already exist. > How exactly does your idea differ from combining LAPACK, FFTPACK, etc. > into one tar file and write wrappers and documentation for all of it? > > Konrad. > -- > ------------------------------------------------------------------------------- > Konrad Hinsen | E-Mail: hinsen@cnrs-orleans.fr > Centre de Biophysique Moleculaire (CNRS) | Tel.: +33-2.38.25.55.69 > Rue Charles Sadron | Fax: +33-2.38.63.15.17 > 45071 Orleans Cedex 2 | Deutsch/Esperanto/English/ > France | Nederlands/Francais > ------------------------------------------------------------------------------- > > _______________________________________________ > Matrix-SIG maillist - Matrix-SIG@python.org > http://www.python.org/mailman/listinfo/matrix-sig > From david.buscher@durham.ac.uk Tue Feb 9 19:40:57 1999 From: david.buscher@durham.ac.uk (david.buscher@durham.ac.uk) Date: Tue, 9 Feb 1999 19:40:57 +0000 (GMT) Subject: [Matrix-SIG] Re: a full-blown interactive data analysis environment In-Reply-To: Message-ID: On Tue, 9 Feb 1999, Ryszard Czerminski wrote: > It would help a lot to start to develop some document > which would describe what do we want from this new package > (i.e. requirements). > > This can be a good basis for further discussion. > > It should probably include as well on some general > level analysis and description of existing packages > in this field (matlab, idl, root (http://root.cern.ch), > scilab - to name the few) and what makes them inadequate > enough to justify development of > yet-another-data-analysis-package. > > One basic reason - at least for this group - is lack of > Python front-end (:-). To some extent such documents exist and have been referred to in previous posts (can't immediately lay my hands on the URLs). What these documents do not resolve though is the scope of the current project being proposed, and to some extent we need a way of gauging this major question before proceeding. The options I see are: [1] A python-specific IDL-alike, well documented and integrated, having the major advantages that have been described (basically all the specific advantages of python and general interpreted language and open source advantages). [2] A non-language specific system which can be rapidly ported to the next flavour-of-the-month language (not that I think python is flavour-of-the-month!). My feeling is that the [2] is a much harder project, and we might only have a feeling of how to do it by the end of doing [1]. Obviously it would be a great help if while doing [1] we had [2] in mind. What do others think? cheers, David ----------------------------------------+------------------------------------- David Buscher | Phone +44 191 374 7462 Dept of Physics, University of Durham, | Fax +44 191 374 3709 South Road, Durham DH1 3LE, UK | Email david.buscher@durham.ac.uk ----------------------------------------+------------------------------------- From johann@physics.berkeley.edu Tue Feb 9 20:09:00 1999 From: johann@physics.berkeley.edu (Johann Hibschman) Date: 09 Feb 1999 12:09:00 -0800 Subject: [Matrix-SIG] Re: a full-blown interactive data analysis environment In-Reply-To: david.buscher@durham.ac.uk's message of "Tue, 9 Feb 1999 19:40:57 +0000 (GMT)" References: Message-ID: >>>>> "david" == david buscher writes: david> [1] A python-specific IDL-alike, well documented and david> integrated, having the major advantages that have been david> described (basically all the specific advantages of python david> and general interpreted language and open source david> advantages). david> [2] A non-language specific system which can be rapidly david> ported to the next flavour-of-the-month language (not that david> I think python is flavour-of-the-month!). david> My feeling is that the [2] is a much harder project, and we david> might only have a feeling of how to do it by the end of david> doing [1]. Obviously it would be a great help if while david> doing [1] we had [2] in mind. What do others think? I agree that [2] is much harder. I think that if we are to get anything done, we should just do [1] with [2] in mind, otherwise we'll have a spurt of theory posts which eventually die down without having accomplished anything. [2] seems hard. For a lot of interesting things (numeric integration, ODEs, PDEs), we'd need a decent (and fast) way to pass functions around, whether those functions are native-code, python, or interpolations on discrete data points. I've yet to see one which works well. FFTPACK and friends certainly exist, but using those on many machines requries running them through f2c, and that adds complexity (and extra required libraries). Ideally, in today's world (assuming we're targetting typical workstation users, we'd just use straight C. (C++ still seems dangerous, but that may just be the result of some bad experiences on my part...) Well, I'm rambling. I'd like to see this work, even if all it does is assemble a recommended set of netlib routines with extra documentation. As a numeric novice, I'm often swamped by the sheer volume of choice and lack of information on how to choose when looking at netlib. -- Johann Hibschman johann@physics.berkeley.edu From janne@avocado.pc.helsinki.fi Tue Feb 9 21:59:18 1999 From: janne@avocado.pc.helsinki.fi (Janne Sinkkonen) Date: 09 Feb 1999 23:59:18 +0200 Subject: [Matrix-SIG] Neural Networks In-Reply-To: "Pedro Miguel Frazão F. Ferreira"'s message of "Tue, 09 Feb 1999 15:21:26 +0000" References: <36C05276.6299952@ualg.pt> Message-ID: "Pedro Miguel Frazão F. Ferreira" writes: > Just subscribed. Just started looking to python. I am interested in > code for Neural Networks (specialy RBF, GRBF). Is there any > implementation ? I have one in product use (i.e. tested). Has k-means, adaptive choice of center widths, and regularization built in, and it includes the beginnings of a regression model class hierarchy, but no documentation. Send e-mail if you're interested. -- Janne From ffjhl@uaf.edu Tue Feb 9 22:05:10 1999 From: ffjhl@uaf.edu (Jonah Lee) Date: Tue, 9 Feb 1999 13:05:10 -0900 (AKST) Subject: [Matrix-SIG] Numpy: memory leak in tolist() fixed Message-ID: Hi, I finally got some time to fix this known memory leak in arrayobject.c. The fix is based upon the suggestion from David Ascher. The diff file is 1214,1219c1214 < PyObject *v=array_item((PyArrayObject *)self, i); < PyList_SetItem(lp, i, PyArray_ToList(v)); < if (((PyArrayObject *)self)->nd>1){ < Py_DECREF(v); < } < --- > PyList_SetItem(lp, i, PyArray_ToList(array_item((PyArrayObject *)self, i))); Regards, Jonah From Oliphant.Travis@mayo.edu Tue Feb 9 23:05:31 1999 From: Oliphant.Travis@mayo.edu (Travis Oliphant) Date: Tue, 9 Feb 1999 17:05:31 -0600 (EST) Subject: [Matrix-SIG] Interactive Data Analysis Message-ID: To clarify some of my previous ramblings. It is true that there are far too many integrated data analysis packages all linking basically the same code (especially the free ones) for the core engines. Just look at this page... http://stommel.tamu.edu/~baum/linuxlist/linuxlist/node33.html#numericalenvironments All of these packages are useful to some people but I was not satisfied with either the underlying data object or the lack of extensibility on any of them. It would be great to have a generic package that includes all of the "important" free numerical routines that could be easily plugged-in to any suitable extensible scripting language. That could definitely be a long term vision. But, I agree with David and Johann that this is a hard problem, especially without a model to work from. I also tend to agree that a good way to work towards a generic package is to implement (translate: collect, package, and document the interface) a python-specific collection of good routines while keeping an eye towards reuse in other environments. I think this means either using SWIG as much as possible (without sacrificing real usability with the NumPy defined objects), or making sure that any code written is cleanly separated into Python-specific and generic subroutines. Concentrating on getting lots of useful routines cleanly working with NumPy early will attract more users of Numpy which will improve the quality of any generic design that we might come up with later. Right now there are not enough "toolkit" routines to persuade many to use NumPy. That is unfortunate since it is really quite a simple thing to add these facilities to the python-numpy environment --- It just takes a little personpower. Thanks, Travis From jh@oobleck.tn.cornell.edu Wed Feb 10 01:34:09 1999 From: jh@oobleck.tn.cornell.edu (Joe Harrington) Date: Tue, 9 Feb 1999 20:34:09 -0500 Subject: [Matrix-SIG] Re: a full-blown interactive data analysis environment In-Reply-To: <199902091746.SAA02706@dirac.cnrs-orleans.fr> (message from Konrad Hinsen on Tue, 9 Feb 1999 18:46:48 +0100) References: <199902082155.QAA08575@oobleck.tn.cornell.edu> <199902091746.SAA02706@dirac.cnrs-orleans.fr> Message-ID: <199902100134.UAA16646@oobleck.tn.cornell.edu> > Sorry, but I still don't see what you are planning to do... > My impression is that generic low-level libraries already exist. > How exactly does your idea differ from combining LAPACK, FFTPACK, etc. > into one tar file and write wrappers and documentation for all of it? That is precisely what I want to do. The shocking thing is that nobody has done this yet, at least not in a usable way. Doing it well is not trivial. The difference from what has already been done is to do it coherently, package it well, and provide a place for users and developers to get it, contribute to it, discuss it, etc., following the model of other successful free software efforts. So far, scientific computing has not taken part in the explosion of capability (and drop in price) that other kinds of computing have as a result of the open-source software development model. The rank and file scientists in most disciplines use commercial software or hire programmers to write in C. There are several decent low-level libraries (and many indecent ones), but there is no integration. This project will play the same role as Red Hat or Debian do for Linux. Linux didn't take off until there were coherent distributions that were easy to install and easy for hackers to add to. Prior to then, you had to go out and find and compile everything in /usr/bin. Few did. Now anyone can go grab a CD and install Linux in 15 minutes. When you're done, there's a coherent system whose components interact well. Now, even if you went through the huge effort of collecting lots of numerical stuff, getting it to compile on your platform, and wrapping it, you'd have huge gaps where you couldn't find a package, the packages wouldn't interact cleanly, and you'd be faced with a million pages of really awful docs, with no examples of how to use the stuff together. You could spend several years full-time getting to this point, and still not be able to work as easily as you can in a primitive environment like IDL or even IRAF. The way I see changing the situation is to decide first what we want (rather than listing the packages that are available), then going out and looking for all the free implementations of each component, weighing each, and selecting packages based on how well they fit into the overall scheme. This would include their ease of use, quality of implementation, level of support, documentation, use of data types that most match our standard ones, etc. We'd then pick one to use and write what we needed to get it to interface well. That might be a SWIG wrapper or a piece of C/C++ code that translated the data to our interchange formats. We'd write tutorial docs that described using the routines together, with examples. Where needed, we'd write docs that described packages whose documentation was deficient. Finally, we'd package the whole thing up so that it's easy for a novice to install it. Of course, for novices to install and begin using it, the main package has to include at least one interactive environment. I chose Python because it's the clear winner among the few languages that handle arrays. Such a package would make Python usable for science by anyone who can use IDL, AIPS, IRAF, etc., without spending months struggling against the lack of good numerical routines and graphical display. The assembled package would be free and it would quickly become better than its competition. People in various disciplines (including some on this list, I'm sure) would soon contribute stuff to support work in specific fields. I hope this better explains what I'm getting at. If you have more questions and haven't done so already, please take the time to read our article and visit the Interactive Data Analysis Environments web site, if you haven't: http://oobleck.tn.cornell.edu/jh/ast/papers/idae96.ps http://lheawww.gsfc.nasa.gov/users/barrett/IDAE/table.1.html Thanks, --jh-- Joe Harrington 326 Space Sciences Building Cornell University Ithaca, NY 14853-6801 (607) 255-5913 office (607) 255-9002 fax jh@oobleck.tn.cornell.edu jh@alum.mit.edu (permanent) From jh@oobleck.tn.cornell.edu Wed Feb 10 01:38:00 1999 From: jh@oobleck.tn.cornell.edu (Joe Harrington) Date: Tue, 9 Feb 1999 20:38:00 -0500 Subject: [Matrix-SIG] Re: a full-blown interactive data analysis environment In-Reply-To: (message from Ryszard Czerminski on Tue, 9 Feb 1999 13:47:11 -0500 (EST)) References: Message-ID: <199902100138.UAA16661@oobleck.tn.cornell.edu> > It would help a lot to start to develop some document > which would describe what do we want from this new package > (i.e. requirements). See the 2 URLs at the end of my previous message... --jh-- Joe Harrington 326 Space Sciences Building Cornell University Ithaca, NY 14853-6801 (607) 255-5913 office (607) 255-9002 fax jh@oobleck.tn.cornell.edu jh@alum.mit.edu (permanent) From Oliphant.Travis@mayo.edu Mon Feb 8 07:21:07 1999 From: Oliphant.Travis@mayo.edu (Travis E. Oliphant) Date: Mon, 08 Feb 1999 01:21:07 -0600 Subject: [Matrix-SIG] A full-blown interactive data analysis environment Message-ID: <36BE9063.48140255@mayo.edu> > We'd then pick one to use and > write what we needed to get it to interface well. That might be a > SWIG wrapper or a piece of C/C++ code that translated the data to our > interchange formats. To try and be precise, so that we understand each other, explain what more we would do than what I did in producing the cephesmodule and the fftw module, besides put them together into one package and write better documentaion? If that is the basic idea, just bigger scale and more packages, than let's go do it, I'm in. Perhaps a good way to get this going is to take FFTW or cephes which I've already got into a useable form and discuss what changes to make to make them fit better into the desired scheme. For that matter we could start with LAPACK and see what could be done there. It would help me, if someone could clarify what is wrong (if anything) with the way all of these libraries are currently used with Python. (Is it just that they don't all come in one big tar file and use individual-style make files rather than one big one?) As for packages we need at least these as far as I see (let's get the list rolling): FFT (my vote is for FFTW) LinearAlgebra Random Numbers Ordinary Differential Equations (ODEPACK?) Integration (QUADPACK?) Optimization (I know there's something on netlib) Root finding (Equation solving) Filter package (remez exchange algorithm, low level linear iir or fir filter function, etc.) (I've got something like this in development). Graphics and plotting: This will be the problem I fear for cross-platform. How about we get going and talk about this one once we have the Numerics happening....maybe something will surface elsewhere in the meantime? I vote for help written in LaTeX and translated to TeXinfo and HTML. But again, it is also critical that we have some sort of interactive help system implemented for the package to be useful to the novice. Looking for feedback and someone to blow the whistle :-) --Travis From Oliphant.Travis@mayo.edu Tue Feb 9 16:49:08 1999 From: Oliphant.Travis@mayo.edu (Travis E. Oliphant) Date: Tue, 09 Feb 1999 10:49:08 -0600 Subject: [Matrix-SIG] A full blown interactive data analysis environment. (resent) Message-ID: <36C06704.B50AD87F@mayo.edu> > We'd then pick one to use and > write what we needed to get it to interface well. That might be a > SWIG wrapper or a piece of C/C++ code that translated the data to our > interchange formats. To try and be precise, so that we understand each other, explain what more we would do than what I did in producing the cephesmodule and the fftw module, besides put them together into one package and write better documentaion? If that is the basic idea, just bigger scale and more packages, than let's go do it, I'm in. Perhaps a good way to get this going is to take FFTW or cephes which I've already got into a useable form and discuss what changes to make to make them fit better into the desired scheme. For that matter we could start with LAPACK and see what could be done there. It would help me, if someone could clarify what is wrong (if anything) with the way all of these libraries are currently used with Python. (Is it just that they don't all come in one big tar file and use individual-style make files rather than one big one?) As for packages we need at least these as far as I see (let's get the list rolling): FFT (my vote is for FFTW) LinearAlgebra Random Numbers Ordinary Differential Equations (ODEPACK?) Integration (QUADPACK?) Optimization (I know there's something on netlib) Root finding (Equation solving) Filter package (remez exchange algorithm, low level linear iir or fir filter function, etc.) (I've got something like this in development). Graphics and plotting: This will be the problem I fear for cross-platform. How about we get going and talk about this one once we have the Numerics happening....maybe something will surface elsewhere in the meantime? I vote for help written in LaTeX and translated to TeXinfo and HTML. But again, it is also critical that we have some sort of interactive help system implemented for the package to be useful to the novice. Looking for feedback and someone to blow the whistle :-) --Travis P.S. We really need to set up a CVS server somewhere and get going on the code development. That way we could actually have code to comment to each other on instead of just ideas. I've never set up CVS before. Could we get some space at the Starship and set it up there. Alas, I don't have a PSA membership at the moment... From r.hooft@euromail.net Wed Feb 10 06:41:36 1999 From: r.hooft@euromail.net (Rob Hooft) Date: Wed, 10 Feb 1999 07:41:36 +0100 (MET) Subject: [Matrix-SIG] A full-blown interactive data analysis environment In-Reply-To: <36BE9063.48140255@mayo.edu> References: <36BE9063.48140255@mayo.edu> Message-ID: <14017.10784.433843.450025@octopus.chem.uu.nl> >>>>> "TEO" == Travis E Oliphant writes: TEO> FFT (my vote is for FFTW) If I remember correctly from some old discussion on linking tcl with readline from a number of years ago: any software that is clearly written to be linked to a GPL library is automatically GPL itself. FTTW is GPL (not LGPL), which means that inclusion in any kind of package leaves me out (I'm also writing commercial software in python). Rob -- ===== R.Hooft@EuroMail.net http://www.xs4all.nl/~hooft/rob/ ===== ===== R&D, Nonius BV, Delft http://www.nonius.nl/ ===== ===== PGPid 0xFA19277D ========================== Use Linux! ========= From ryszard@moldyn.com Wed Feb 10 12:24:05 1999 From: ryszard@moldyn.com (Ryszard Czerminski) Date: Wed, 10 Feb 1999 07:24:05 -0500 (EST) Subject: [Matrix-SIG] Re: a full-blown interactive data analysis environment In-Reply-To: <199902100134.UAA16646@oobleck.tn.cornell.edu> Message-ID: On Tue, 9 Feb 1999, Joe Harrington wrote: > I hope this better explains what I'm getting at. If you have more > questions and haven't done so already, please take the time to read > our article and visit the Interactive Data Analysis Environments web > site, if you haven't: > > http://oobleck.tn.cornell.edu/jh/ast/papers/idae96.ps > http://lheawww.gsfc.nasa.gov/users/barrett/IDAE/table.1.html Yes, these two pages directly address my question - thanks. In your comparison table you do not mention ROOT. I have learned about existence of this package few month ago from the article in Linux Journal. I have downloaded it and played with it somewhat and I found it to be very impresive system. It is an open source project and it might provide a lot of functionality we are looking for. It is using C++ interpreter as the the command language. I would be very much interested in opinions of others on this list. Ryszard From: http://root.cern.ch/root/Mission.html Currently the emphasis of ROOT is on the data analysis domain but thanks to the approach of loosely coupled object-oriented frameworks the system can easily be extended to other domains. We believe that ROOT is an ideal environment to introduce physicists quickly to the new world of Objects and C++. ROOT: Executive Summary The ROOT system provides a set of OO frameworks with all the functionality needed to handle and analyse large amounts of data in a very efficient way. Having the data defined as a set of objects, specialised storage methods are used to get direct access to the separate attributes of the selected objects, without having to touch the bulk of the data. Included are histograming methods in 1, 2 and 3 dimensions, curve fitting, function evaluation, minimisation, graphics and visualization classes to allow the easy setup of an analysis system that can query and process the data interactively or in batch mode. Thanks to the builtin CINT C++ interpreter the command language, the scripting, or macro, language and the programming language are all C++. The interpreter allows for fast prototyping of the macros since it removes the time consuming compile/link cycle. It also provides a good environment to learn C++. If more performance is needed the interactively developed macros can be compiled using a C++ compiler. The system has been designed in such a way that it can query its databases in parallel on MPP machines or on clusters of workstations or high-end PC's. ROOT is an open system that can be dynamically extended by linking external libraries. This makes ROOT a premier platform on which to build data acquisition, simulation and data analysis systems. Ryszard Czerminski phone : (617)354-3124 x 10 Moldyn, Inc. fax : (617)491-4522 955 Massachusetts Avenue e-mail: ryszard@moldyn.com Cambridge MA, 02139-3180 or ryszard@photon.com From hinsen@cnrs-orleans.fr Wed Feb 10 17:19:59 1999 From: hinsen@cnrs-orleans.fr (Konrad Hinsen) Date: Wed, 10 Feb 1999 18:19:59 +0100 Subject: [Matrix-SIG] Re: a full-blown interactive data analysis environment In-Reply-To: <199902100134.UAA16646@oobleck.tn.cornell.edu> (message from Joe Harrington on Tue, 9 Feb 1999 20:34:09 -0500) References: <199902082155.QAA08575@oobleck.tn.cornell.edu> <199902091746.SAA02706@dirac.cnrs-orleans.fr> <199902100134.UAA16646@oobleck.tn.cornell.edu> Message-ID: <199902101719.SAA02598@dirac.cnrs-orleans.fr> > > How exactly does your idea differ from combining LAPACK, FFTPACK, etc. > > into one tar file and write wrappers and documentation for all of it? > > That is precisely what I want to do. The shocking thing is that > nobody has done this yet, at least not in a usable way. Doing it well > is not trivial. OK, now I see what you are aiming at - and I agree it's a worthwhile project; the average potential user just can't do these things. > successful free software efforts. So far, scientific computing has > not taken part in the explosion of capability (and drop in price) that > other kinds of computing have as a result of the open-source software > development model. The rank and file scientists in most disciplines > use commercial software or hire programmers to write in C. There are Partly this situation has been caused by the scientists themselves. Much of the commercial software has been written by scientists for their own work, but instead of following the original scientific spirit of sharing the result of scientific work freely with other scientists, they have chosen to go commercial. Tight budgets certainly encourage this, as does the unfortunate fact that providing scientific software is not considered "science". > The way I see changing the situation is to decide first what we want > (rather than listing the packages that are available), then going out > and looking for all the free implementations of each component, > weighing each, and selecting packages based on how well they fit into > the overall scheme. This would include their ease of use, quality of Fine. But we'll have to create separate "task forces"; I suppose none of us knows all relevant fields sufficiently well! -- ------------------------------------------------------------------------------- Konrad Hinsen | E-Mail: hinsen@cnrs-orleans.fr Centre de Biophysique Moleculaire (CNRS) | Tel.: +33-2.38.25.55.69 Rue Charles Sadron | Fax: +33-2.38.63.15.17 45071 Orleans Cedex 2 | Deutsch/Esperanto/English/ France | Nederlands/Francais ------------------------------------------------------------------------------- From hinsen@cnrs-orleans.fr Wed Feb 10 17:25:52 1999 From: hinsen@cnrs-orleans.fr (Konrad Hinsen) Date: Wed, 10 Feb 1999 18:25:52 +0100 Subject: [Matrix-SIG] A full-blown interactive data analysis environment In-Reply-To: <14017.10784.433843.450025@octopus.chem.uu.nl> (r.hooft@euromail.net) References: <36BE9063.48140255@mayo.edu> <14017.10784.433843.450025@octopus.chem.uu.nl> Message-ID: <199902101725.SAA21444@dirac.cnrs-orleans.fr> > FTTW is GPL (not LGPL), which means that inclusion in any kind of > package leaves me out (I'm also writing commercial software in > python). I think licensing could become an important problem in this project. Many scientific packages come with licences that do not allow commercial use (at least not for free), and in some cases the conditions are even unclear. We might have to use a two-level system, one for generally free code and one for non-commercial use only (which would still be useful to many of us). -- ------------------------------------------------------------------------------- Konrad Hinsen | E-Mail: hinsen@cnrs-orleans.fr Centre de Biophysique Moleculaire (CNRS) | Tel.: +33-2.38.25.55.69 Rue Charles Sadron | Fax: +33-2.38.63.15.17 45071 Orleans Cedex 2 | Deutsch/Esperanto/English/ France | Nederlands/Francais ------------------------------------------------------------------------------- From david.buscher@durham.ac.uk Wed Feb 10 18:02:00 1999 From: david.buscher@durham.ac.uk (david.buscher@durham.ac.uk) Date: Wed, 10 Feb 1999 18:02:00 +0000 (GMT) Subject: [Matrix-SIG] FFTW In-Reply-To: <14017.10784.433843.450025@octopus.chem.uu.nl> Message-ID: On Wed, 10 Feb 1999, Rob Hooft wrote: > FTTW is GPL (not LGPL), which means that inclusion in any kind of > package leaves me out (I'm also writing commercial software in > python). I know this is a side issue, but has anyone asked the authors of FFTW about whether they would consider LGPL'ing it? I have been `in the market' for a fast, flexible, open-source C implementation of multi-dimensional FFT's for many years, and FFTW has been the first package that has come along that has hit all of the requirements dead-on. The authors state their intention of it becoming a de-facto standard, so maybe the change of licence would be something they would be amenable to... David ----------------------------------------+------------------------------------- David Buscher | Phone +44 191 374 7462 Dept of Physics, University of Durham, | Fax +44 191 374 3709 South Road, Durham DH1 3LE, UK | Email david.buscher@durham.ac.uk ----------------------------------------+------------------------------------- From asterian@eecs.umich.edu Wed Feb 10 18:20:38 1999 From: asterian@eecs.umich.edu (Andrew Sterian) Date: Wed, 10 Feb 1999 13:20:38 -0500 (EST) Subject: [Matrix-SIG] FFTW In-Reply-To: from "david.buscher@durham.ac.uk" at Feb 10, 1999 06:02:00 PM Message-ID: <199902101820.NAA12221@dip.eecs.umich.edu> david.buscher@durham.ac.uk writes... > > FTTW is GPL (not LGPL), which means that inclusion in any kind of > > package leaves me out (I'm also writing commercial software in > > python). > > I know this is a side issue, but has anyone asked the authors of FFTW > about whether they would consider LGPL'ing it? I have been `in the market' > for a fast, flexible, open-source C implementation of multi-dimensional > FFT's for many years, and FFTW has been the first package that has come > along that has hit all of the requirements dead-on. The authors state > their intention of it becoming a de-facto standard, so maybe the change of > licence would be something they would be amenable to... I quote from the FFTW FAQ: Question 1.3. Is FFTW free software? Starting with version 1.3, FFTW is Free Software in the technical sense defined by the Free Software Foundation (see Categories of Free and Non-Free Software), and is distributed under the terms of the GNU General Public License. Previous versions of FFTW were distributed without fee for noncommercial use, but were not technically ``free.'' Non-free licenses for FFTW are also available that permit different terms of use than the GPL. Question 1.4. What is this about non-free licenses? The non-free licenses are for companies that wish to use FFTW in their products but are unwilling to release their software under the GPL (which would require them to release source code and allow free redistribution). Such users can purchase an unlimited-use license from MIT. Contact us for more details. We could instead have released FFTW under the LGPL, or even disallowed non-Free usage. Suffice it to say, however, that MIT owns the copyright to FFTW and they only let us GPL it because we convinced them that it would neither affect their licensing revenue nor irritate existing licensees. Andrew. asterian@umich.edu | Me: http://www-personal.umich.edu/~asterian ----------------------------+---------------------------------------------- Faith is the willingness to get out of bed in the morning and just show up... -- Richard Thieme From bdmf@email.msn.com Thu Feb 11 19:33:11 1999 From: bdmf@email.msn.com (!Our Publishing Company) Date: 11 Feb 1999 11:33:11 -0800 Subject: [Matrix-SIG] The New Bible of Dating!! (x11) Message-ID: <0d3f00733190b29CPIMSSMTPU02@email.msn.com> Women all over America have said, "Burn This Book!!!" CALL TO ORDER YOUR COPY - (800) 830-2047 [24hrs] How To Juggle Women: Without Getting Killed or Going Broke by Stefan Feller ISBN:0-9658299-4-4 Price: $12.00 This book is an essential guide for the seasoned dating man who needs help organizing the women in his life. Women are constantly grabbing your interest and attention. Do you have to choose on over the other? Of Course not! Stop letting opportunities like these go to waste. Why not date them all? This is a must-read for men who are active on the dating scene. Not only will you learn how to juggle several females at a time, you will also improve your overall chances with women, including where to meet them and how to do the little things to keep them happy. You will learn time-saving techniques that will maximize your availability, and allow you to spend time with several ladies a day. The author details scheduling methods so you never have two dates at the same time. You will also learn money-saving tips so you can date aggressively and still have money left to buy life's other necessities. Forget the saying,"For every man there is a woman out there." Now it will be, "For every man there are at least three women." -------------------------------------------------------- CHAPTER TITLES Chapter One - Getting to Know You Chapter Two - Categories of Women Chapter Three - Where to Find Them Chapter Four - The Rotation Chapter Five - Prior Planning Chapter Six - How to Keep Them Happy and Away From Each Other Chapter Seven - Fiscally Fit Chapter Eight - Personnel Changes -------------------------------------------------------- Juggling Women: Without Getting Killed or Going Broke by Stefan Feller ISBN:0-9658299-4-4 Price: $12.00 $3.00 Shipping individuals - COD Charges are free for a limited time. Wholesale Discounts Available --------------------------------------------------------- Order by Phone 1 (800)830-2047 24 hours a day --------------------------------------------------------- From bridgman@lheapop.gsfc.nasa.gov Fri Feb 12 21:48:08 1999 From: bridgman@lheapop.gsfc.nasa.gov (Dr. William T. Bridgman) Date: Fri, 12 Feb 1999 16:48:08 -0500 Subject: [Matrix-SIG] Announcing the Astronomical Python mailing list Message-ID: Why use Python in astronomy? Many astronomers use scripting and scripting-type languages in astronomical data analysis. Some are very expensive. Others lack portability between different operating system and processor architectures. Still others lack powerful methods for manipulating numerical data. Python is free, runs on a multitude of system architectures, and can handle numerical processing. What Python currently lacks is a large library of fundamental routines needed by the astronomical community. That is the goal of this mailing list. We want to establish a nexus for development and distribution of Python-based routines and other software to address the needs of the astronomical community. You can find out more about what's been done and what is planned at our website (which is still under construction) http://lheawww.gsfc.nasa.gov/~bridgman/AstroPy/ If you are interested in participating in this project, send mail to majordomo@athena.gsfc.nasa.gov with subscribe astropy as the message text. You will be sent a confirmation request to which you must respond before being added to the list. Thanks for your attention. -- Dr. William T."Tom" Bridgman Raytheon STX NASA/Goddard Space Flight Center Code 664 bridgman@lheapop.gsfc.nasa.gov Greenbelt, MD 20771 (301) 286-1346 From ransom@cfa.harvard.edu Sun Feb 14 17:27:47 1999 From: ransom@cfa.harvard.edu (Scott M. Ransom) Date: Sun, 14 Feb 1999 17:27:47 +0000 Subject: [Matrix-SIG] Re: AstroPy ... Re: Data Analysis References: Message-ID: <36C70793.D00A91E0@cfa.harvard.edu> For the Matrix-Sig people out there that have not subscribed to the AstroPy mailing list, this is a reply to a post about creating an Astronomy related data-analysis toolkit with Python. I have cross-posted it because I think it is relevant to the "new" data analysis effort.... W.T. Bridgman wrote: > 1) One of the first tasks for this project is to find out what has already > been done by list members. I have made a couple small packages which might be of use to the Astro/Data Analysis community (and I am more than willing to help out in any way I can...) 1. A Romberg Integrator and a "Safe" Newton-Raphson (both in pure Python) that I wrote, are available as part of Konrad Hinson's Scientific Python package: http://starship.skyport.net/crew/hinsen/scientific.html 2. I have written a Python wrapper for Nick Patavalis' very nice set of PGPLOT wrappers. The Python wrapper lets you make very nice plots (including 2-D images) with a single interactive call (ala IDL or MATLAB). Both the wrapper and Nick's package are available at: ftp://cfa0.harvard.edu/pub/ransom/ 3. Not by me, but very useful nonetheless, Gary Strangman has made a very complete (although I do not know how well it has been tested) set of statistics functions available at: http://www.nmr.mgh.harvard.edu/Neural_Systems_Group/gary/python.html 4. I have interfaced a large pulsar data analysis package that I wrote with Python using SWIG. I therefore have a fairly workable NumPy typemap file (that will probably need to be edited quite extensively for general use) that allows NumPy to use C arrays and vise-versa. If anyone wants to use this as a starting point for a more robust typemap, I have placed it in the above FTP site. A couple other points: I have been considering wrapping the 'C' version of SLALIB for Python. Unfortunately, the author, P.T.Wallace (ptw@star.rl.ac.uk), only makes the 'C' version available by email. If we truly want this wrapped, we need to get his permission and hopefully a method of making the most current 'C' version available to anyone. Does anyone know Mr. Wallace? If not I might try to email him myself about this. I believe the most important thing that needs to be done (and this applies to the Matrix group as well) is to develop a publication qualtiy (fonts especially), portable, 2-D and 3-D surface/mesh plotting routine or library. PGPLOT is great for 2-D work for astronomy, and the Gist stuff is nice as well, but the font support (I believe) is not there (correct me if I'm wrong), but I think we have to go beyond those. The PerlDL people have a good start on a workable OpenGL 3-D plotting package if some Perl guru out there has a mind to start a porting effort. Other options could be a much expanded TkPlotCanvas, or the new Java APIs (including Java3D). We should probably try to keep this tied in as much as possible with the current effort, started by Joe Harrington and others on Matrix-Sig, to create a full-blown data-analysis environment. Cheers, Scott -- Scott M. Ransom Phone: (580) 536-7215 Address: 703 SW Chaucer Cir. email: ransom@cfa.harvard.edu Lawton, OK 73505 PGP Fingerprint: D2 0E D0 10 CD 95 06 DA EF 78 FE 2B CB 3A D3 53 From Oliphant.Travis@mayo.edu Mon Feb 15 06:07:39 1999 From: Oliphant.Travis@mayo.edu (Travis Oliphant) Date: Mon, 15 Feb 1999 00:07:39 -0600 (EST) Subject: [Matrix-SIG] PIL and NumPy Relationship Message-ID: PIL and/or NumPy users: Thanks to F Lundh and all else who have contributed to the PIL. It is a very nice piece of work and a very useful tool for those of us who use python to look at and display images. Since I have only been a python regular for under a year I wasn't around when the PIL was just getting off the ground and I have a couple of questions about the relationship between the PIL and Numerical Python. I come from a background of using other data analysis environments like MATLAB and IDL which treat images as an array of numbers. As a result I'm a little surprised to see two different objects being developed in Python that both serve as images. The Image object in PIL and the multiarray object in NumPy. I wondered if someone could enlighten me as to any advantage of having two separate objects for the same kind of data. Personally, I think this creates an unnecessary dichotomy and division of scarce resources. I know, for example, that people have written and will write "image processing" routines that will work on the Image object and that I would like to apply those same routines to my Array object but will first have to get the data into an Image object and then back into an Array object for further processing. While not difficult this is not really acceptable if one is trying to use Python as an interactive analysis environment. In many cases one deals with an image which is simply a slice through or projection of a larger array data set. With a separate Image object, all of the methods people have and may develop for image processing (including loading and saving of image types) are unavailable unless the translation is made first. What is the possibility of unifying the PIL and NumPy a bit more? I know you can translate between the two objects with fromstring and tostring methods but this is wasteful in both mental effort (in keeping track of the different objects and . It would be better, I think if the PIL were built around the multiarray object already. I guess I'm not sure what kind of work this would involve. Another possibility is to have the underlying image data be both an Image object and Multiarray object at the same time (same data area different headers? in the C-struct) So, I'm basically asking users of the PIL to tell me if and why they feel it is necessary to have two different objects to represent images. I don't mean to sound critical at all. I'm just offering my thoughts. I basically wouldn't care at all except I find the PIL to be such a useful addition to Python and am also a heavy user of Numerical Python. Thanks again to all involved in its development. Thanks, Travis Oliphant ---------------------------------------------------- Travis Oliphant 200 First St SW Rochester MN 55905 Ultrasound Research Lab (507) 286-5293 Mayo Graduate School Oliphant.Travis@mayo.edu From Oliphant.Travis@mayo.edu Mon Feb 15 15:48:35 1999 From: Oliphant.Travis@mayo.edu (Travis Oliphant) Date: Mon, 15 Feb 1999 09:48:35 -0600 (EST) Subject: [Matrix-SIG] Plotting in Python Message-ID: >PGPLOT is great for 2-D work for astronomy, and the Gist stuff is nice as >well, but the font support (I believe) is not there (correct me if I'm >wrong), I've used GIST enough to know that it uses postscript fonts for it's output. I think GIST is quite nice for plotting (the style sheets could use better documentation, though...) I don't know about PGPLOT, yet. From frank@ned.dem.csiro.au Thu Feb 18 04:23:35 1999 From: frank@ned.dem.csiro.au (Frank Horowitz) Date: Thu, 18 Feb 1999 12:23:35 +0800 Subject: [Matrix-SIG] Python Khoros Interface: deceased? Message-ID: <36CB95C6.A35D6088@ned.dem.csiro.au> G'Day David, The graphics page of the PyScience web page site still lists a link to a Khoros Inteface. I've been sporadically trying to follow that link for at least 6 months, to no avail. I've tried e-mailing Suorsa to ask if there is a newer link, but have never received a reply. I believe that the Khoros package referred to was a SWIG job on an early version of Khoros. Since Khoros itself is no longer freely available on the net (I'd love to be proven wrong on that, BTW), and since the last freely available version of Khoros postdates the version the Python wrapper targets (as deduced from some old messages found in deja-news) I'm wondering if there's any point in continuing to claim an existing link between Python and Khoros? Is it time to officially pronounce the Python-Khoros interface dead? Cheers, Frank Horowitz From Oliphant.Travis@mayo.edu Wed Feb 24 00:14:55 1999 From: Oliphant.Travis@mayo.edu (Travis Oliphant) Date: Tue, 23 Feb 1999 18:14:55 -0600 (EST) Subject: [Matrix-SIG] Sigtools 0.40 released In-Reply-To: <199902232251.RAA04613@eric.cnri.reston.va.us> References: <199902232251.RAA04613@eric.cnri.reston.va.us> Message-ID: I'm announcing the release of Sigtools 0.40 for Python with Numerical extensions. This module provides several general-purpose signal processing functions which can operate on Numerical Python arrays. These modules provide core functionality for signal/image/array processing using Numerical Python. The module currently has four methods: (1) an N-D convolution filter (blurring, edge-detection, etc.) (2) an N-D order filter (a median filter is an example of this type) (3) a linear_filter function that operates on N-D (possibly non-contiguous) arrays along an arbitrary axis. The linear_filter implements any linear system with a rational/polynomial transfer function. (It is just like the filter function in MATLAB). (4) a Parks-McClellan algorithm for designing optimal FIR filters: This is just a wrapper around GPL'd code written by Jake Janovetz. Tarfiles (with a Setup file) and RPM's are available at http://oliphant.netpedia.net --Travis Oliphant From Oliphant.Travis@mayo.edu Thu Feb 25 23:15:40 1999 From: Oliphant.Travis@mayo.edu (Travis Oliphant) Date: Thu, 25 Feb 1999 17:15:40 -0600 (EST) Subject: [Matrix-SIG] Wiener filter using sigtools and Python Message-ID: While this routine will go into the next addition of sigtools, I thought I would show interested people, how easy it is to do image (volume) processing with Python. Below find a function to do wiener filtering on an array of data. from sigtools import * from MLab import * def wiener(im,mysize=None,noise=None): if mysize==None: mysize = 3*ones(len(a.shape)) mysize = array(mysize); # Estimate the local mean lMean = convolveND(im,ones(mysize),1) / prod(mysize) # Estimate the local variance lVar = convolveND(im**2,ones(mysize),1) / prod(mysize) - lMean**2 # Estimate the noise power if needed. if noise==None: noise = mean(ravel(lVar)) # Compute result # out = lMean + (maximum(0, lVar - noise) ./ # maximum(lVar, noise)) * (im - lMean) # out = im - lMean im = lVar - noise im = maximum(im,0) lVar = maximum(lVar,noise) out = out / lVar out = out * im out = out + lMean return out Enjoy, --Travis From dubois1@llnl.gov Thu Feb 25 23:29:35 1999 From: dubois1@llnl.gov (Paul F. Dubois) Date: Thu, 25 Feb 1999 15:29:35 -0800 Subject: [Matrix-SIG] Numeric Test on 1.5.2b2 o.k. Message-ID: <000301be6116$b47fc600$f4160218@c1004579-c.plstn1.sfba.home.com> FYI: I made 1.5.2b2 out of the box, and then made the current Numerical by doing python makethis.py and the Numerical test executes correctly. This was on Linux (Redhat, 5.2). Paul