From salsifis at free.fr Mon May 2 06:11:22 2005 From: salsifis at free.fr (Salsifis) Date: Mon, 2 May 2005 12:11:22 +0200 Subject: [SciPy-dev] in-line graphics In-Reply-To: <20050420175347.1A3C93EBCA@www.scipy.com> References: <20050420175347.1A3C93EBCA@www.scipy.com> Message-ID: Dear Scipy developpers, Thanks for the work on scipy and the related effort to bring scientific computing solutions to open-source. I'm in the effort to translate my MATLAB scripts into scipy and the learning curve is steep. I particularly appreciate iPython (so this is perhaps OT on the scipy ML) as a solution to have a researcher-friendly interface. A killer (and that will convince many people around me still sticking to MATLAB) would be to enable inline graphics, as was the case in mathematica. This would be : - a very good way to interact with your code (I love to use hist() to know wehere my variables live), - an easy way to create simple reports of results or to include in paper to allow replication of results (export session to pdf) - an excellent way to produce tutorials for schools (as standalone .py files) Unfortunately, may coding abilities don't even permit me to gauge if this feature is a snap to code or if it involves calling for mysterious extra-human forces... Also, I wish to stress that it's not a call for a "visual scipy" with a lot of avoidable features (no offense to Matlab :) ) but just a simple way to include in the same window a graphic ouput to a command-line request... Cheerts, Laurent exemple (inspired by travis' tutorial) : >> from pylab import * >> plot([1,2,3,4]) -------------- next part -------------- A non-text attachment was scrubbed... Name: pastedGraphic1.tiff Type: image/tiff Size: 9674 bytes Desc: not available URL: -------------- next part -------------- >> plot([1,2,3,4], [1,4,9,16], 'ro') >> axis([0, 6, 0, 20]) -------------- next part -------------- A non-text attachment was scrubbed... Name: pastedGraphic2.tiff Type: image/tiff Size: 6738 bytes Desc: not available URL: -------------- next part -------------- >> savefig('secondfig.png') From brendansimons at yahoo.ca Mon May 2 09:17:20 2005 From: brendansimons at yahoo.ca (Brendan Simons) Date: Mon, 2 May 2005 09:17:20 -0400 (EDT) Subject: [SciPy-dev] Re: in-line graphics Message-ID: <20050502131720.79397.qmail@web31111.mail.mud.yahoo.com> I would immediately pay for such an app. I've fought with MathCAD/Mathematica/Matlab way too often already. Development wise, I can't see how this would be too difficult. You would have to merge iPython with matplotlib and a wysiwyg editor. I think the biggest hurdle would be choosing a gui kit, since no one kit makes everybody happy. -Brendan On 2-May-05, at 6:11 AM, scipy-dev-request at scipy.net wrote: >>A killer >>(and that will convince many people around me still sticking to >>MATLAB) >>would be to enable inline graphics, as was the case in mathematica. ______________________________________________________________________ Post your free ad now! http://personals.yahoo.ca From aisaac at american.edu Sun May 1 12:09:00 2005 From: aisaac at american.edu (Alan G Isaac) Date: Sun, 1 May 2005 12:09:00 -0400 (Eastern Daylight Time) Subject: [SciPy-dev] interesting floating point discussion Message-ID: It's Java focused, but more generally applicable. http://www.cs.berkeley.edu/~wkahan/JAVAhurt.pdf I have no idea how Python has responded to these issues. A pointer to an accessible discussion would be welcome. Thanks, Alan Isaac From boisgera at isia.cma.fr Mon May 2 10:01:59 2005 From: boisgera at isia.cma.fr (=?ISO-8859-1?Q?S=E9bastien_Boisg=E9rault?=) Date: Mon, 02 May 2005 16:01:59 +0200 Subject: [SciPy-dev] interesting floating point discussion In-Reply-To: References: Message-ID: <427632D7.8040108@isia.cma.fr> Alan G Isaac wrote: >It's Java focused, but more generally applicable. >http://www.cs.berkeley.edu/~wkahan/JAVAhurt.pdf >I have no idea how Python has responded >to these issues. A pointer to an accessible >discussion would be welcome. > > The floating-point context in Python is IMHO far worse than in Java, principally because: - the standard "nostop" IEEE 754 policy is not the default : 1.0/0.0 raises an exception and it should not (according to the standard). - any special-value (0+, 0-, NaN, Inf, etc.) handling is *fully* platform dependent. Your platform is determined by your OS, but also by your python version, compilation options, etc. - Try a search on comp.lang.python with the keywords "Tim Peters floating point IEEE 754", you'll end up with some very interesting threads ... Tim Peters is probably one of the best Python experts on the floating-point theme. - The main attempt to "fix" some of theses issues (allow or disallow trap handling, etc) was the fpectl module of the standard library. It is now considered dangerous because it has not been actively maintained. It may even not be installed on your Python platform, given that the python install script had forgotten to mention it ! The omission was dicovered only recently ... - You may also have a look at the fpconst module ... - And recently, try the "marshal/unmarshal" thread on python-dev. - Mmmmm, don't forget the Decimal module. It is pure python and therefore slow, but otherwise far better than the *binary* floating-point stuff. Too bad I am not an accountant ;) Cheers, S?bastien Boisg?rault From jh at oobleck.astro.cornell.edu Mon May 2 10:03:10 2005 From: jh at oobleck.astro.cornell.edu (Joe Harrington) Date: Mon, 2 May 2005 10:03:10 -0400 Subject: [SciPy-dev] implementing IDL, Matlab, etc. functionality Message-ID: <200505021403.j42E3AJu003410@oobleck.astro.cornell.edu> Two of the big barriers to people converting from commercial environments to SciPy are 1) their dependence on specific packages in those languages (e.g., the fuzzy logic package in Matlab that was recently mentioned, or the IDL Wavelet Toolkit), and 2) their dependence on their own code, developed over the course of their career. There are legal issues we need to be aware of in coming up with solutions. There are two problems: first, to use IDL (and, I assume, Matlab), you have to agree to a license that says you will not reverse engineer the product. The packages include a lot of routines in source form. Most users who are likely to write compatible packages in SciPy have looked at these routines. They thus have "tainted brains": the commercial entity can claim that they have seen proprietary information, and that they had to use this information to produce the Python package. Second, Cornell's legal staff has informed me that there are two court precedents that state that if you have *ever* used a product, and you try to make something that works the same way, it's reverse engineering. I know that sounds ludicrous to everyone, it does to me, but you may have to fight at the Federal Appeals level or above to be sure. Again, since users of IDL and Matlab have agreed to licenses that prohibit reverse engineering, this would mean that any code produced by such people would be tainted and we could be sued. "We" means us as individuals, our institutions (who employ us, pay us, and might be construed as the licencees of IDL and Matlab), Enthought for distributing it, etc. Regardless of how crazy this interpretation might sound, I doubt that we as a community have the financial resources to mount an appeal. There is also no shortage of crazy legal interpretations that stick in current computer law, so there is no guarrantee that we'd win. One of the original plaintiffs was Sony, which would surely weigh in with amicus briefs and perhaps financial help to the plaintiff, since their IP would be at stake if their precedent were turned over on our appeal. I think there is a technical way out, however. I have only agreed to an IDL license. Nothing says I can't implement something that works like Matlab. Reverse engineering is legal, at least in the great state of New York, unless you sign that right away. So, it makes some sense to set up some trades: I can write a Matlab -> Python converter if you'll do it for IDL. All that's needed is for each side to provide examples of code in the proprietary language that they have written themselves, along with descriptions of what it does and sample inputs and outputs. That plus commercial books on the languages should provide all the information we need. For specific packages, it makes more sense for people to do them from scratch or by wrapping existing open-source code. For one thing, the efforts of the commercial packages are usually old and almost always procedural. Much is to be gained implementing as a good OO design based on the method descriptions in the literature and the particular strengths of scipy.base. Then, someone else who has not signed a license can do workalike interfaces for the commercial packages. The benefit here is that IDL and Matlab are pretty backward languages, whereas Python is quite modern. Matlab users may benefit from encountering a Matlab-compatible fuzzy logic interface, but IDL users certainly don't. It would be much better for them to learn the OO interface. The reverse is of course true for, say, IDL wavelets. So, before anyone (else) goes diving into implementing something from IDL, Matlab, etc., please give consideration to these issues. --jh-- From prasansamtani at gmail.com Mon May 2 10:35:57 2005 From: prasansamtani at gmail.com (Prasan Samtani) Date: Mon, 2 May 2005 07:35:57 -0700 Subject: [SciPy-dev] implementing IDL, Matlab, etc. functionality In-Reply-To: <200505021403.j42E3AJu003410@oobleck.astro.cornell.edu> References: <200505021403.j42E3AJu003410@oobleck.astro.cornell.edu> Message-ID: <5da497b7050502073515a20a1a@mail.gmail.com> > whereas Python is quite modern. Matlab users may benefit from > encountering a Matlab-compatible fuzzy logic interface, but IDL users > certainly don't. It would be much better for them to learn the OO Point well taken. Over the last week or so, I've found deficencies in the MatLab Fuzzy logic toolkit. Let's say I'll start off with building a FuzzyLogic API that provides the necessary functionality - and then worry about creating an interface that MatLab users would find intuitive (and yet different enough that we don't run into any legal risks). --Prasan On 5/2/05, Joe Harrington wrote: > Two of the big barriers to people converting from commercial > environments to SciPy are 1) their dependence on specific packages in > those languages (e.g., the fuzzy logic package in Matlab that was > recently mentioned, or the IDL Wavelet Toolkit), and 2) their > dependence on their own code, developed over the course of their > career. There are legal issues we need to be aware of in coming up > with solutions. > > There are two problems: first, to use IDL (and, I assume, Matlab), you > have to agree to a license that says you will not reverse engineer the > product. The packages include a lot of routines in source form. Most > users who are likely to write compatible packages in SciPy have looked > at these routines. They thus have "tainted brains": the commercial > entity can claim that they have seen proprietary information, and that > they had to use this information to produce the Python package. > Second, Cornell's legal staff has informed me that there are two court > precedents that state that if you have *ever* used a product, and you > try to make something that works the same way, it's reverse > engineering. I know that sounds ludicrous to everyone, it does to me, > but you may have to fight at the Federal Appeals level or above to be > sure. Again, since users of IDL and Matlab have agreed to licenses > that prohibit reverse engineering, this would mean that any code > produced by such people would be tainted and we could be sued. "We" > means us as individuals, our institutions (who employ us, pay us, and > might be construed as the licencees of IDL and Matlab), Enthought for > distributing it, etc. Regardless of how crazy this interpretation > might sound, I doubt that we as a community have the financial > resources to mount an appeal. There is also no shortage of crazy > legal interpretations that stick in current computer law, so there is > no guarrantee that we'd win. One of the original plaintiffs was Sony, > which would surely weigh in with amicus briefs and perhaps financial > help to the plaintiff, since their IP would be at stake if their > precedent were turned over on our appeal. > > I think there is a technical way out, however. I have only agreed to > an IDL license. Nothing says I can't implement something that works > like Matlab. Reverse engineering is legal, at least in the great > state of New York, unless you sign that right away. So, it makes some > sense to set up some trades: I can write a Matlab -> Python converter > if you'll do it for IDL. All that's needed is for each side to > provide examples of code in the proprietary language that they have > written themselves, along with descriptions of what it does and sample > inputs and outputs. That plus commercial books on the languages > should provide all the information we need. > > For specific packages, it makes more sense for people to do them from > scratch or by wrapping existing open-source code. For one thing, the > efforts of the commercial packages are usually old and almost always > procedural. Much is to be gained implementing as a good OO design > based on the method descriptions in the literature and the particular > strengths of scipy.base. Then, someone else who has not signed a > license can do workalike interfaces for the commercial packages. The > benefit here is that IDL and Matlab are pretty backward languages, > whereas Python is quite modern. Matlab users may benefit from > encountering a Matlab-compatible fuzzy logic interface, but IDL users > certainly don't. It would be much better for them to learn the OO > interface. The reverse is of course true for, say, IDL wavelets. > > So, before anyone (else) goes diving into implementing something from > IDL, Matlab, etc., please give consideration to these issues. > > --jh-- > > _______________________________________________ > Scipy-dev mailing list > Scipy-dev at scipy.net > http://www.scipy.net/mailman/listinfo/scipy-dev > From jdhunter at ace.bsd.uchicago.edu Mon May 2 11:58:09 2005 From: jdhunter at ace.bsd.uchicago.edu (John Hunter) Date: Mon, 02 May 2005 10:58:09 -0500 Subject: [SciPy-dev] implementing IDL, Matlab, etc. functionality In-Reply-To: <200505021403.j42E3AJu003410@oobleck.astro.cornell.edu> (Joe Harrington's message of "Mon, 2 May 2005 10:03:10 -0400") References: <200505021403.j42E3AJu003410@oobleck.astro.cornell.edu> Message-ID: <87mzrdaacu.fsf@peds-pc311.bsd.uchicago.edu> >>>>> "Joe" == Joe Harrington writes: Joe> sign that right away. So, it makes some sense to set up some Joe> trades: I can write a Matlab -> Python converter if you'll do Joe> it for IDL. All that's needed is for each side to provide Joe> examples of code in the proprietary language that they have Joe> written themselves, along with descriptions of what it does Joe> and sample inputs and outputs. That plus commercial books on Joe> the languages should provide all the information we need. Personally, I don't thinks this is a good use of manpower. matlab and IDL are both pretty crappy languages. That's why people don't want to use them. octave already provides a matlab clone (eg can run m-files) but a lot of people don't use it because it never works as well as the original but has all the faults of the original (except cost). Octave is always several versions behind -- an open source project simply can't keep up with the manpower behind matlab or IDL in terms of a feature-by-feature implementation. Users then get frustrated when they try and run their scripts and although a few scripts work, many don't. Look, we don't even have full matlab matfile support, much less the capability to clone matlab or IDL (and good luck with the matlab/IDL API, MEX files etc, on which many matlab extensions depend (eg wavelab). Providing a translator essentially creates false expectations and dissatisfied users. I think development effort should be put into making the python platform as compelling as possible, not in trying to emulate a suboptimal language in python. People can convert -- I used matlab for 8 years and had an enormous code base and just walked away. I never tried to run my mfiles in python, I just implemented the missing functionality in python. Granted, this means some users will never switch from IDL or matlab to python but that's fine, many will, especially if we spend our effort making better tools (eg a wavelet library for scipy) and providing documentation like "python for IDL users" and "python for matlab users". Give them things that they can easily do in python that are painful in matlab/IDL and most of them will bite the bullet. JDH From aisaac at american.edu Mon May 2 10:29:18 2005 From: aisaac at american.edu (Alan G Isaac) Date: Mon, 2 May 2005 10:29:18 -0400 (Eastern Daylight Time) Subject: [SciPy-dev] interesting floating point discussion In-Reply-To: <427632D7.8040108@isia.cma.fr> References: <427632D7.8040108@isia.cma.fr> Message-ID: Thanks! Here is one link with interesting discussion and additional useful links: http://python.fyxm.net/dev/summary/2000-10-1.html Cheers, Alan Isaac From jh at oobleck.astro.cornell.edu Mon May 2 12:40:44 2005 From: jh at oobleck.astro.cornell.edu (Joe Harrington) Date: Mon, 2 May 2005 12:40:44 -0400 Subject: [SciPy-dev] implementing IDL, Matlab, etc. functionality In-Reply-To: <87mzrdaacu.fsf@peds-pc311.bsd.uchicago.edu> (message from John Hunter on Mon, 02 May 2005 10:58:09 -0500) References: <200505021403.j42E3AJu003410@oobleck.astro.cornell.edu> <87mzrdaacu.fsf@peds-pc311.bsd.uchicago.edu> Message-ID: <200505021640.j42Gei4c005816@oobleck.astro.cornell.edu> > Personally, I don't thinks this is a good use of manpower. matlab and > IDL are both pretty crappy languages. That's why people don't want to > use them. I'll appeal to my extensive, if informal, surveys of the astronomy community. A lot of us have a lot more than just 8 years of code we depend on, and simply cannot switch until we can convert our most important codes. For example, several of the packages I now depend on for my research took me and about half a dozen students years of full-time work to write. Most of those students are now gone. I use these packages every day. I'd have to get new grants specifically to hire and supervise more students just to redo them so that I could do my daily work. That isn't going to happen, and I'm far from a unique case. You can tell us to go jump in a lake if you like (which you basically just did). Instead, we're going to take care of our own problem, and hopefully that of a lot of others. My goal is not to make IDL-in-python (gnudl tries, as does octave for matlab, both badly, as you note). It's to facilitate the conversion of code to Python. We can do an 80% solution without a great deal of effort. The idea is to create a code converter that would handle all of the procedural syntax, and which would flag in loud comments the things it couldn't do. Then the user would hand-code the remainder. We would not have a function compatibility library, though we could rig a way of describing the most common functions so that it could convert the calls for them. Nobody could possibly have an expectation that the code produced this way would run unmodified, so I don't think we'll make unsatisfied users. Rather, I think just the opposite: this package will be a great relief to many, as will its Matlab equivalent. My main concern is not for coding labor but rather for numerical accuracy. There are enough subtle differences between Python and IDL that hand-converting large amounts of numerical code would be error-prone. For example, in Python the end of an array slice is one element beyond the end of an IDL slice. An automatic converter would just add one to every slice's ending index. A human wouldn't necessarily remember to do that every time, producing subtle bugs could be very hard to find in some cases. --jh-- From Fernando.Perez at colorado.edu Mon May 2 13:05:19 2005 From: Fernando.Perez at colorado.edu (Fernando Perez) Date: Mon, 02 May 2005 11:05:19 -0600 Subject: [SciPy-dev] Re: in-line graphics In-Reply-To: <20050502131720.79397.qmail@web31111.mail.mud.yahoo.com> References: <20050502131720.79397.qmail@web31111.mail.mud.yahoo.com> Message-ID: <42765DCF.1020107@colorado.edu> Brendan Simons wrote: > I would immediately pay for such an app. I've fought > with MathCAD/Mathematica/Matlab way too often already. > > Development wise, I can't see how this would be too > difficult. You would have to merge iPython with > matplotlib and a wysiwyg editor. I think the biggest > hurdle would be choosing a gui kit, since no one kit > makes everybody happy. It's not fundamentally difficult, but it is a lot of work. However, it is something that I've personally wanted for a very long time (anyone interested will find such references in the ipython lists from way back). One of the main reasons behind the recently mentioned freeze on ipyhton development is precisely this: I need to clean up the ipython internals in a pretty serious way before this kind of integration is possible. The short story is: something like this will probably happen, because John and I (mostly me, in this particular case) are very interested in making it happen. But it will be a slow process; if you (any you, not necessarily the OP) want it faster, roll up your sleeves and come over to the ipython-dev list, where I can give you plenty of work to do :) Best, f From jdhunter at ace.bsd.uchicago.edu Mon May 2 13:24:57 2005 From: jdhunter at ace.bsd.uchicago.edu (John Hunter) Date: Mon, 02 May 2005 12:24:57 -0500 Subject: [SciPy-dev] implementing IDL, Matlab, etc. functionality In-Reply-To: <200505021640.j42Gei4c005816@oobleck.astro.cornell.edu> (Joe Harrington's message of "Mon, 2 May 2005 12:40:44 -0400") References: <200505021403.j42E3AJu003410@oobleck.astro.cornell.edu> <87mzrdaacu.fsf@peds-pc311.bsd.uchicago.edu> <200505021640.j42Gei4c005816@oobleck.astro.cornell.edu> Message-ID: <87vf61tuae.fsf@peds-pc311.bsd.uchicago.edu> >>>>> "Joe" == Joe Harrington writes: Joe> octave for matlab, both badly, as you note). It's to Joe> facilitate the conversion of code to Python. We can do an Joe> 80% solution without a great deal of effort. The idea is to Joe> create a code converter that would handle all of the Joe> procedural syntax, and which would flag in loud comments the Joe> things it couldn't do. Then the user would hand-code the Joe> remainder. We would not have a function compatibility Is it clear that the EULA forbids translating a limited part of a user's IDL file into another syntax? That sounds like something different than reverse engineering. Do you have a copy of the IDL EULA? I've never seen it and couldn't find it via quick searches on google or on the RSI site. JDH From stephen.walton at csun.edu Mon May 2 13:46:31 2005 From: stephen.walton at csun.edu (Stephen Walton) Date: Mon, 02 May 2005 10:46:31 -0700 Subject: [SciPy-dev] in-line graphics In-Reply-To: References: <20050420175347.1A3C93EBCA@www.scipy.com> Message-ID: <42766777.5000300@csun.edu> Salsifis wrote: > I particularly appreciate iPython (so this is perhaps OT on the scipy > ML) as a solution to have a researcher-friendly interface. A killer > (and that will convince many people around me still sticking to > MATLAB) would be to enable inline graphics, as was the case in > mathematica. How far does TeXmacs go in fulfilling this need? I saw it used at SciPy '04, although admittedly mainly as a sort of automatic document update, where scipy code was embedded in a TeX document and its output automatically included therein. ps_out was used to embed a graphic in the document in real time as well. My notes say this stuff is on the LiveZope DVD. From jh at oobleck.astro.cornell.edu Mon May 2 14:06:58 2005 From: jh at oobleck.astro.cornell.edu (Joe Harrington) Date: Mon, 2 May 2005 14:06:58 -0400 Subject: [SciPy-dev] implementing IDL, Matlab, etc. functionality In-Reply-To: <87vf61tuae.fsf@peds-pc311.bsd.uchicago.edu> (message from John Hunter on Mon, 02 May 2005 12:24:57 -0500) References: <200505021403.j42E3AJu003410@oobleck.astro.cornell.edu> <87mzrdaacu.fsf@peds-pc311.bsd.uchicago.edu> <87vf61tuae.fsf@peds-pc311.bsd.uchicago.edu> Message-ID: <200505021806.j42I6w23006088@oobleck.astro.cornell.edu> > Is it clear that the EULA forbids translating a limited part of a > user's IDL file into another syntax? No, a user's code is clearly not RSI's property. However, our lawyers made the point that a code converter could easily be configured as a front-end to python, and for any program it converted 100% successfully, it would be a reimplementation of IDL and thus would be prohibited by the reverse engineering clause. They felt that making even part of a partial reimplementation could be argued as being against the reverse engineering clause of the license, based on these (bogus, in my opinion) court precedents. At least, it wouldn't be grounds for immediate dismissal of a lawsuit. The only relevant statement in the license is: "You may not, however, reverse engineer, decompile or disassemble the Software." The full license may be read here: ftp://ftp.rsinc.com/pub/idl_6.1/info/license.txt Can someone post a pointer to Matlab's license? --jh-- From venkatraghavan at gmail.com Mon May 2 14:31:11 2005 From: venkatraghavan at gmail.com (Venkat Raghavan V.C.) Date: Mon, 2 May 2005 13:31:11 -0500 Subject: [SciPy-dev] implementing IDL, Matlab, etc. functionality In-Reply-To: <20050502164105.63E473EB7B@www.scipy.com> References: <20050502164105.63E473EB7B@www.scipy.com> Message-ID: > There are legal issues we need to be aware of in coming up > with solutions. I was scared that I was missing something. > I think there is a technical way out, however. I have only agreed to > an IDL license. Nothing says I can't implement something that works > like Matlab. Reverse engineering is legal, at least in the great > state of New York, unless you sign that right away. So, it makes some > sense to set up some trades: I can write a Matlab -> Python converter > if you'll do it for IDL. All that's needed is for each side to > provide examples of code in the proprietary language that they have > written themselves, along with descriptions of what it does and sample > inputs and outputs. That plus commercial books on the languages > should provide all the information we need. Sad, if we resort to this! > For specific packages, it makes more sense for people to do them from > scratch or by wrapping existing open-source code. There have been three packages in which I have used the controls "module", 1. scilab, which is open-source, but not free . (Scilab license forbids you to: use a composite or derived version of Scilab for commercial uses without asking INRIA authorization. http://scilabsoft.inria.fr/legal/index_legal.php?page=faq.html ) 2. GNU-Octave, which is free ( GPL) 3. Matlab, which is closed source. I guess I can look to Octave for pointers. But at this > So, before anyone (else) goes diving into implementing something from > IDL, Matlab, etc., please give consideration to these issues. From venkatraghavan at gmail.com Mon May 2 14:38:44 2005 From: venkatraghavan at gmail.com (Venkat Raghavan V.C.) Date: Mon, 2 May 2005 13:38:44 -0500 Subject: Fwd: [SciPy-dev] implementing IDL, Matlab, etc. functionality. In-Reply-To: References: <20050502164105.63E473EB7B@www.scipy.com> Message-ID: > There are legal issues we need to be aware of in coming up > with solutions. I was scared that I was missing something. > I think there is a technical way out, however. I have only agreed to > an IDL license. Nothing says I can't implement something that works > like Matlab. Reverse engineering is legal, at least in the great > state of New York, unless you sign that right away. So, it makes some > sense to set up some trades: I can write a Matlab -> Python converter > if you'll do it for IDL. All that's needed is for each side to > provide examples of code in the proprietary language that they have > written themselves, along with descriptions of what it does and sample > inputs and outputs. That plus commercial books on the languages > should provide all the information we need. Sad, if we resort to this! > For specific packages, it makes more sense for people to do them from > scratch or by wrapping existing open-source code. There have been three packages in which I have used the controls "module", 1. scilab, which is open-source, but not free . (Scilab license forbids you to: use a composite or derived version of Scilab for commercial uses without asking INRIA authorization. http://scilabsoft.inria.fr/legal/index_legal.php?page=faq.html ) 2. GNU-Octave, which is free ( GPL) 3. Matlab, which is closed source. I guess I can look to Octave for pointers. > So, before anyone (else) goes diving into implementing something from > IDL, Matlab, etc., please give consideration to these issues. It's still early days in my implementation. I'm glad someone pointed this out to me this early! P.S.: Sorry about the previous incomplete mail. Guess keyboard shortcuts across multiple OS'es are beginning to hurt me! :) From rkern at ucsd.edu Mon May 2 16:39:53 2005 From: rkern at ucsd.edu (Robert Kern) Date: Mon, 02 May 2005 13:39:53 -0700 Subject: [SciPy-dev] implementing IDL, Matlab, etc. functionality In-Reply-To: <200505021806.j42I6w23006088@oobleck.astro.cornell.edu> References: <200505021403.j42E3AJu003410@oobleck.astro.cornell.edu> <87mzrdaacu.fsf@peds-pc311.bsd.uchicago.edu> <87vf61tuae.fsf@peds-pc311.bsd.uchicago.edu> <200505021806.j42I6w23006088@oobleck.astro.cornell.edu> Message-ID: <42769019.6000109@ucsd.edu> Joe Harrington wrote: >>Is it clear that the EULA forbids translating a limited part of a >>user's IDL file into another syntax? > > > No, a user's code is clearly not RSI's property. However, our lawyers > made the point that a code converter could easily be configured as a > front-end to python, and for any program it converted 100% > successfully, it would be a reimplementation of IDL and thus would be > prohibited by the reverse engineering clause. They felt that making > even part of a partial reimplementation could be argued as being > against the reverse engineering clause of the license, based on these > (bogus, in my opinion) court precedents. At least, it wouldn't be > grounds for immediate dismissal of a lawsuit. Could you ask your lawyers for the actual case citations? In any case, I believe that the usual way that professionals do this (legally, to my knowledge, but IANAL) is to work in two teams. One team actually looks at the software they are trying to emulate. They do not write any code whatsoever; they only write a specification. The other team does not touch the software-to-be-emulated in any way; they only implement from the specification written by the other team. If you're serious about doing these kinds of activities, consider this strategy and talk about it with your lawyers. -- Robert Kern rkern at ucsd.edu "In the fields of hell where the grass grows high Are the graves of dreams allowed to die." -- Richard Harter From jmiller at stsci.edu Mon May 2 17:04:32 2005 From: jmiller at stsci.edu (Todd Miller) Date: Mon, 02 May 2005 17:04:32 -0400 Subject: [SciPy-dev] scipy.special for numarray Message-ID: <1115067871.30278.469.camel@halloween.stsci.edu> scipy.special now passes all its self tests with numarray using the tarball of modifications found here: http://www.stsci.edu/resources/software_hardware/numarray/scipy_special_numarray To try this out, the following works on Linux: 0. Get & unpack the tarball % wget http://www.stsci.edu/resources/software_hardware/numarray/scipy_special_numarray % tar zxvf scipy_special_numarray.tar.gz scipy_special_numarray/ scipy_special_numarray/updates.txt scipy_special_numarray/diffs.txt scipy_special_numarray/na_cephes.py scipy_special_numarray/cephes.py scipy_special_numarray/gen_na_cephes.py scipy_special_numarray/_na_cephesmodule.c scipy_special_numarray/nc_cephesmodule.c 1. Checkout scipy from CVS on scipy.org. 2. Apply the diffs % cd scipy % patch -p0 <../scipy_special_numarray/diffs.txt 3. Save the "added files" into scipy/Lib/special. % cd .. % cp scipy_special_numarray/{*.py,*.c} scipy/Lib/special 4. Build scipy (it's important to have both numarray & Numeric already installed): % cd scipy % python setup.py build_ext --backends=numeric,numarray install --force 5. Numeric is the default array package; make sure still works: % python >>> import scipy >>> scipy.test() ... Ran 676 tests in 3.115s OK 6. Running using numarray requires setting the environment variable NUMERIX to numarray. It should be noted that not all of scipy has been ported to numarray yet: only scipy_core (except for weave) and scipy.special have been ported so only scipy.special is tested here. % setenv NUMERIX numarray % python >>> import scipy.special as spec >>> spec.test() ... Ran 342 tests in 2.279s OK A top level overview of the changes is in updates.txt. Barring any objections I'll commit these changes one week from today. Comments are welcome. Regards, Todd From nwagner at mecha.uni-stuttgart.de Wed May 4 07:11:01 2005 From: nwagner at mecha.uni-stuttgart.de (Nils Wagner) Date: Wed, 04 May 2005 13:11:01 +0200 Subject: [SciPy-dev] linalg.solve and singular systems Message-ID: <4278ADC5.1010605@mecha.uni-stuttgart.de> Hi all, I was surprised about the behaviour of linalg.solve in case of singular/nearly singular coefficient matrices. The user should get at least a warning. The singular values clearly show that the matrix is ill-conditioned. Any comment or suggestion would be appreciated. Thanks in advance Nils -------------- next part -------------- A non-text attachment was scrubbed... Name: strange.py Type: text/x-python Size: 191 bytes Desc: not available URL: From barrett at stsci.edu Thu May 5 05:24:42 2005 From: barrett at stsci.edu (Paul Barrett) Date: Thu, 05 May 2005 05:24:42 -0400 Subject: [SciPy-dev] implementing IDL, Matlab, etc. functionality In-Reply-To: <200505021640.j42Gei4c005816@oobleck.astro.cornell.edu> References: <200505021403.j42E3AJu003410@oobleck.astro.cornell.edu> <87mzrdaacu.fsf@peds-pc311.bsd.uchicago.edu> <200505021640.j42Gei4c005816@oobleck.astro.cornell.edu> Message-ID: <4279E65A.5060206@stsci.edu> Joe Harrington wrote: >My main concern is not for coding labor but rather for numerical >accuracy. There are enough subtle differences between Python and IDL >that hand-converting large amounts of numerical code would be >error-prone. For example, in Python the end of an array slice is one >element beyond the end of an IDL slice. An automatic converter would >just add one to every slice's ending index. A human wouldn't >necessarily remember to do that every time, producing subtle bugs >could be very hard to find in some cases. > > This is my concern too. In the case of IDL, most algorithms are implemented in single precision floating point. The Python implementation by default would use double precision, unless we explicitly direct it to do otherwise. This problem alone can cause much grief, because the IDL version is presumed to be the correct one, until demonstrated otherwise. (I know this from personal experience.) So in addition to the language being crappy, do we also want to propagate crappy (i.e. unstable) algorithms too? -- Paul -- Paul Barrett, PhD Space Telescope Science Institute Phone: 410-338-4475 ESS/Science Software Branch FAX: 410-338-4767 Baltimore, MD 21218 From jh at oobleck.astro.cornell.edu Thu May 5 11:51:47 2005 From: jh at oobleck.astro.cornell.edu (Joe Harrington) Date: Thu, 5 May 2005 11:51:47 -0400 Subject: [SciPy-dev] implementing IDL, Matlab, etc. functionality In-Reply-To: <4279E65A.5060206@stsci.edu> (message from Paul Barrett on Thu, 05 May 2005 05:24:42 -0400) References: <200505021403.j42E3AJu003410@oobleck.astro.cornell.edu> <87mzrdaacu.fsf@peds-pc311.bsd.uchicago.edu> <200505021640.j42Gei4c005816@oobleck.astro.cornell.edu> <4279E65A.5060206@stsci.edu> Message-ID: <200505051551.j45FplS0026832@oobleck.astro.cornell.edu> > In the case of IDL, most algorithms are > implemented in single precision floating point. The Python > implementation by default would use double precision, unless we > explicitly direct it to do otherwise. This problem alone can cause much > grief, because the IDL version is presumed to be the correct one, until > demonstrated otherwise. (I know this from personal experience.) So in > addition to the language being crappy, do we also want to propagate > crappy (i.e. unstable) algorithms too? Paul, that argument involves both guilt by association and an implicit assumption that those scientists who *are* guilty will write better code the second time around. My experience and what I've been told directly by dozens of researchers is that they will not write code a second time, period. Their main reasons to switch are cost and freedom, not any unhappiness with IDL. The balance in their minds tips to IDL if the cost of rewriting is included. It tips back to Python if they have a tool that converts most of their code. As you point out, Python defaults to double, so that may improve things a little for some algorithms. I doubt it will make a big difference for most codes, as astronomical data tends to be uncertain no later than the fourth decimal place, and frequently in the first. As for the guilt by association, writing in IDL does not make an algorithm or its implementation bad, nor does writing in Python automatically make them any better. Quick-and-dirty, procedural scientists will write quick-and-dirty, procedural code in Python as well as in IDL. On the flip side, I have an algorithm for optimal spectrum extraction that I can demonstrate is the best available (paper with the first-ever comparison tests in prep). It's 4300 lines and took well over 1000 billable programmer-hours to write. If I can do an "80%" translation to Python automatically, spend a few weeks filling in the external function calls, and run my verification tests to see where the problems are and fix them, we will have a valuable algorithm that will attract users to Python. I don't have the resources to do it by hand, and I doubt anyone else will do it for me (would STScI?). In preparing our paper, we discovered either algorithmic deficiencies (your "crap") or serious bugs in every other code we tested, including popular ones like Piskunov's REDUCE and IRTF's SpeXtool. So, either we do a converter and you get this and many other tools that will attract scientists to Python, or we don't, and until Python becomes the dominant tool (if ever), people will be home-brewing their own codes for difficult algorithms like optimal spectrum extraction. And then you will have real "crap", as you put it, but only necessarily so in Python. This is just one of many examples. It's great that you at STScI are redoing the astron library from the ground up, but I doubt that you can commit to doing even all of that work (including astron/contrib). You certainly can't do everyone's personal codes, whereas a converter can make a big dent in that. Hopefully, that will be a big enough dent to induce people to switch. --jh-- From stephen.walton at csun.edu Thu May 5 17:54:31 2005 From: stephen.walton at csun.edu (Stephen Walton) Date: Thu, 05 May 2005 14:54:31 -0700 Subject: [SciPy-dev] implementing IDL, Matlab, etc. functionality In-Reply-To: <200505051551.j45FplS0026832@oobleck.astro.cornell.edu> References: <200505021403.j42E3AJu003410@oobleck.astro.cornell.edu> <87mzrdaacu.fsf@peds-pc311.bsd.uchicago.edu> <200505021640.j42Gei4c005816@oobleck.astro.cornell.edu> <4279E65A.5060206@stsci.edu> <200505051551.j45FplS0026832@oobleck.astro.cornell.edu> Message-ID: <427A9617.3060607@csun.edu> Just a few comments. Joe Harrington wrote: >>In the case of IDL, most algorithms are >>implemented in single precision floating point. The Python >>implementation by default would use double precision, unless we >>explicitly direct it to do otherwise. >> I personally vote for single precision algorithms to stay single precision if possible. The Fortran side of the numerical computation community has always felt that C's promotion of everything to double is one of its most ill-considered design decisions. Given that NumPy has single and double arrays, and Numeric 24 (like numarray) doesn't promote a single array to a double when it interacts with a double scalar, I'd think we could keep single precision in many contexts. >I doubt [single vs. double] will make a big difference for >most codes, as astronomical data tends to be uncertain no later than >the fourth decimal place, and frequently in the first. > > pi**2=10 for most astronomers. I second the rest of Joe's comments. (That sounds like an impressive spectrum extraction code, by the way, Joe.) Still, I'd hope that a more feature-ful language would help the conversion as well. No one wrote a Fortran to C translator, at least not one which produced human-readable C, but I think more scientists are writing C than Fortran nowadays, and the reason is probably the big problems in Fortran 77. Fortran 90 fixed most of the problems but came along too late. (We at universities also all found our students were no longer being taught Fortran by our colleagues over in computer science.) So, maybe one effective approach would be to get some IDL and MATLAB users to list their biggest annoyances with the language and then see they don't face same in Scipy. Now if we could just get the lawyers out of our way so we could actually do some work. From stephen.walton at csun.edu Thu May 5 19:40:36 2005 From: stephen.walton at csun.edu (Stephen Walton) Date: Thu, 05 May 2005 16:40:36 -0700 Subject: [SciPy-dev] scipy.test outputs Message-ID: <427AAEF4.9010404@csun.edu> Just for everyone's FYI, I uploaded the results of scipy.test() from version 0.3.3_303.4602 to the scipy.org bug tracker. A rather large number of tests seem to be failing; I'm getting 107 failures on a dual Athlon running Fedora Core 1 and 75 on a P4 running Fedora Core 3, both with Numeric 23.8 and ATLAS 3.6.0. Is anyone else seeing similar numbers? From stephen.walton at csun.edu Fri May 6 00:26:05 2005 From: stephen.walton at csun.edu (Stephen Walton) Date: Thu, 05 May 2005 21:26:05 -0700 Subject: [SciPy-dev] scipy.test outputs In-Reply-To: <427AAEF4.9010404@csun.edu> References: <427AAEF4.9010404@csun.edu> Message-ID: <427AF1DD.1010807@csun.edu> Stephen Walton wrote: > Just for everyone's FYI, I uploaded the results of scipy.test() from > version 0.3.3_303.4602 to the scipy.org bug tracker. A rather large > number of tests seem to be failing; Ignore this, sorry. My CVS checkout was corrupt or something, apparently. A fresh copy of Scipy from CVS seems to have fixed the problems. From nicopernetty at yahoo.fr Fri May 6 09:01:35 2005 From: nicopernetty at yahoo.fr (Nicolas Pernetty) Date: Fri, 6 May 2005 15:01:35 +0200 Subject: [SciPy-dev] Compilation problems (and solutions...) on HP-UX B.11.00 Message-ID: <20050506150135.4a4046f7@linuxcestcomplique.fr> Hello, I've just almost managed to get scipy 0.3.2 compiled, up and running on HP-UX B.11.00 and here are some issues I had to deal with : 1) There should be some way to choose the fortran 77 compilers instead of manually editing the .py files, but maybe there is an environment flag I'm not aware of ? 2) ./Lib/xplt/src/play/unix/fpuset.c had to be modified in order to have it compiled correctly on HP-UX : **************************************************************** #elif defined(FPU_HPUX) /* man pages: fpsetmask * library: -lm */ /* HPUX turns off FP_X_* without this (_INCLUDE_HPUX_SOURCE) */ #ifndef _HPUX_SOURCE #define _HPUX_SOURCE 1 #endif #include void u_fpu_setup(void) { fpsetmask(FP_X_INV | FP_X_DZ | FP_X_OFL); /* 0x1c */ fpsetfastmode(1); /* fast underflows */ } **************************************************************** has to be replaced by **************************************************************** #elif defined(FPU_HPUX) /* man pages: fpsetmask * library: -lm */ /* HPUX turns off FP_X_* without this (_INCLUDE_HPUX_SOURCE) */ #ifndef _HPUX_SOURCE #define _HPUX_SOURCE 1 #endif #include void u_fpu_setup(int when) { if (when <= 0) { fesettrapenable(FE_INVALID | FE_DIVBYZERO | FE_OVERFLOW); } } **************************************************************** 3) In addtion to 2), you have to add math library '-lm' in order to compile fpuset.c. I solved it by editing ./Lib/xplt/config_pygist.py and replace the line 297: elif self.try_link("#define FPU_HPUX\n"+testcode, by elif self.try_link("#define FPU_HPUX\n"+testcode,libraries=[self.mathlib], 4) I had a "Bus Error (core dumped)" when calling the gamma.c function in ./Lib/special/cephes/gamma.c during the test scipy.test(level=1) It seems that the global variable P and Q are incorrectly defined, so I decided to use decimal configuration and add at the top of the file : #undef MIEEE #define UNK It seems to take care of the problem and I had no more errors when calling scipy.test(level=1) 5) When calling : python import scipy scipy.test(level=1, verbosity=2) The computer gets locked, it seems to get stuck in an infinite loop. I have to manually kill the python process But If I type : python -v import scipy scipy.test(level=1, verbosity=2) thus having displayed all the imported modules, everything goes fine ! All tests gets tested and are ok... Don't know if it's a python problem or a scipy problem though... 6) Can you add to the INSTALL.txt file that in order to get scipy find the fftw and ATLAS libraries you have to use an environment var locating both the libraries AND the headers ? I spent more than an hour before understanding that I have to use export ATLAS=/usr/local/lib/atlas:/usr/local/include/atlas instead of just export ATLAS=/usr/local/lib/atlas Hope that all this will help others and this great piece of software ! Regards, From stephen.walton at csun.edu Fri May 6 12:26:52 2005 From: stephen.walton at csun.edu (Stephen Walton) Date: Fri, 06 May 2005 09:26:52 -0700 Subject: [SciPy-dev] Compilation problems (and solutions...) on HP-UX B.11.00 In-Reply-To: <20050506150135.4a4046f7@linuxcestcomplique.fr> References: <20050506150135.4a4046f7@linuxcestcomplique.fr> Message-ID: <427B9ACC.2090009@csun.edu> Nicolas Pernetty wrote: >Hello, > >I've just almost managed to get scipy 0.3.2 compiled, up and running on >HP-UX B.11.00... > > Congratulations! As a long-ago HP-UX user I know how hard it can be to get some freeware working on it. Do you know of the archive at http://hpux.cs.utah.edu/? >1) There should be some way to choose the fortran 77 compilers instead >of manually editing the .py files > > I believe that the INSTALL.txt file which comes with scipy is out of date compared with the scipy.org online documentation. See http://www.scipy.org/documentation/buildscipy.txt. You should be able to simply say python setup.py build --fcompiler=hpux (although the comment at the top says --compiler=hpux). >4) I had a "Bus Error (core dumped)" when calling the gamma.c function >in ./Lib/special/cephes/gamma.c during the test scipy.test(level=1) >It seems that the global variable P and Q are incorrectly defined, so I >decided to use decimal configuration and add at the top of the file : >#undef MIEEE >#define UNK > > This looks like a big-endian vs. little-endian hardware issue to me. >6) Can you add to the INSTALL.txt file that in order to get scipy >find the fftw and ATLAS libraries you have to use an environment var >locating both the libraries AND the headers ? > Hmm...locally I've built scipy with the libraries and includes in either /usr/lib/atlas and /usr/include atlas or /usr/local/lib/atlas and /usr/local/include/atlas. scipy seems to find them when they're in either place without me having to do anything special. From nicopernetty at yahoo.fr Fri May 6 13:10:43 2005 From: nicopernetty at yahoo.fr (Nicolas Pernetty) Date: Fri, 6 May 2005 19:10:43 +0200 Subject: [SciPy-dev] Compilation problems (and solutions...) on HP-UX B.11.00 In-Reply-To: <427B9ACC.2090009@csun.edu> References: <20050506150135.4a4046f7@linuxcestcomplique.fr> <427B9ACC.2090009@csun.edu> Message-ID: <20050506191043.4e295dea@linuxcestcomplique.fr> On Fri, 06 May 2005 09:26:52 -0700, Stephen Walton wrote : > > > >I've just almost managed to get scipy 0.3.2 compiled, up and running > >on HP-UX B.11.00... > > > > > Congratulations! As a long-ago HP-UX user I know how hard it can be > to get some freeware working on it. Do you know of the archive at > http://hpux.cs.utah.edu/? 'hard' ? 'pain-in-the-*beeeep*' would be more appropriate... ;-) Anyway I think that the main problem is the lack of feedback and beta-testers from HP-UX users compared to say Linux users. > > >1) There should be some way to choose the fortran 77 compilers > >instead of manually editing the .py files > > > > > I believe that the INSTALL.txt file which comes with scipy is out of > date compared with the scipy.org online documentation. See > http://www.scipy.org/documentation/buildscipy.txt. You should be able > > to simply say > > python setup.py build --fcompiler=hpux > > (although the comment at the top says --compiler=hpux). My mistake. I haven't seen this page. Thanks ! > > >4) I had a "Bus Error (core dumped)" when calling the gamma.c > >function in ./Lib/special/cephes/gamma.c during the test > >scipy.test(level=1) It seems that the global variable P and Q are > >incorrectly defined, so I decided to use decimal configuration and > >add at the top of the file : #undef MIEEE > >#define UNK > > > > > This looks like a big-endian vs. little-endian hardware issue to me. Yes. Don't know what is going wrong here. I also managed to get it working on Solaris 5.8 and I had _exactly_ the same problem before deciding to use g77 everywhere instead of mixing (forte ?) f77 and g77. But on HP-UX I used g77 everywhere... I will test at work on monday, but could you just try to compile gamma.c with MIEEE enable and see if you can display P and Q properly ? > > >6) Can you add to the INSTALL.txt file that in order to get scipy > >find the fftw and ATLAS libraries you have to use an environment var > >locating both the libraries AND the headers ? > > > Hmm...locally I've built scipy with the libraries and includes in > either /usr/lib/atlas and /usr/include atlas or /usr/local/lib/atlas > and /usr/local/include/atlas. scipy seems to find them when they're > in either place without me having to do anything special. > My mistake, it was just for the example, my ATLAS libraries aren't under /usr/local but instead in another exotic place. At the bottom of http://www.scipy.org/documentation/buildscipy.txt it seems that I'm not the only one to have that problem... From nwagner at mecha.uni-stuttgart.de Wed May 11 05:40:17 2005 From: nwagner at mecha.uni-stuttgart.de (Nils Wagner) Date: Wed, 11 May 2005 11:40:17 +0200 Subject: [SciPy-dev] Problem with building scipy Message-ID: <4281D301.2080007@mecha.uni-stuttgart.de> Hi all, Building scipy from latest cvs failed building 'mach' library compiling Fortran sources Traceback (most recent call last): File "setup.py", line 124, in ? setup_package(ignore_packages) File "setup.py", line 112, in setup_package url = "http://www.scipy.org", File "/var/tmp/cvs/scipy/scipy_core/scipy_distutils/core.py", line 73, in setup return old_setup(**new_attr) File "/usr/local/lib/python2.4/distutils/core.py", line 149, in setup dist.run_commands() File "/usr/local/lib/python2.4/distutils/dist.py", line 946, in run_commands self.run_command(cmd) File "/usr/local/lib/python2.4/distutils/dist.py", line 966, in run_command cmd_obj.run() File "/usr/local/lib/python2.4/distutils/command/build.py", line 112, in run self.run_command(cmd_name) File "/usr/local/lib/python2.4/distutils/cmd.py", line 333, in run_command self.distribution.run_command(command) File "/usr/local/lib/python2.4/distutils/dist.py", line 966, in run_command cmd_obj.run() File "/var/tmp/cvs/scipy/scipy_core/scipy_distutils/command/build_clib.py", line 115, in run self.build_libraries(self.libraries) File "/var/tmp/cvs/scipy/scipy_core/scipy_distutils/command/build_clib.py", line 218, in build_libraries extra_postargs=[]) File "/usr/local/lib/python2.4/site-packages/scipy/distutils/ccompiler.py", line 81, in CCompiler_compile ccomp = self.compiler_so AttributeError: GnuFCompiler instance has no attribute 'compiler_so' Nils From stephen.walton at csun.edu Wed May 11 13:10:30 2005 From: stephen.walton at csun.edu (Stephen Walton) Date: Wed, 11 May 2005 10:10:30 -0700 Subject: [SciPy-dev] Problem with building scipy In-Reply-To: <4281D301.2080007@mecha.uni-stuttgart.de> References: <4281D301.2080007@mecha.uni-stuttgart.de> Message-ID: <42823C86.3070107@csun.edu> Nils Wagner wrote: > Hi all, > > Building scipy from latest cvs failed Worked for me on Fedora Core 3. What platform are you using? Are you using a Fortran compiler other than g77? From nwagner at mecha.uni-stuttgart.de Wed May 11 13:19:06 2005 From: nwagner at mecha.uni-stuttgart.de (Nils Wagner) Date: Wed, 11 May 2005 19:19:06 +0200 Subject: [SciPy-dev] Problem with building scipy In-Reply-To: <42823C86.3070107@csun.edu> References: <4281D301.2080007@mecha.uni-stuttgart.de> <42823C86.3070107@csun.edu> Message-ID: On Wed, 11 May 2005 10:10:30 -0700 Stephen Walton wrote: > Nils Wagner wrote: > >> Hi all, >> >> Building scipy from latest cvs failed > > Worked for me on Fedora Core 3. What platform are you >using? Are you using a Fortran compiler other than g77? > I am on SuSE 9.1 - the standard Fortran compiler is g77. Nils > > _______________________________________________ > Scipy-dev mailing list > Scipy-dev at scipy.net > http://www.scipy.net/mailman/listinfo/scipy-dev From Fernando.Perez at colorado.edu Thu May 12 12:49:56 2005 From: Fernando.Perez at colorado.edu (Fernando Perez) Date: Thu, 12 May 2005 10:49:56 -0600 Subject: [SciPy-dev] On reverse engineering Message-ID: <42838934.6060205@colorado.edu> Hi all, there have been recent discussions on this list on this topic. For those interested, this seems to be a good discussion with a reasonable amount of legal background: http://lwn.net/Articles/134642/ I don't want to turn this into a debate, just to provide useful background info in case the issue comes up again (which unfortunately we all do face one way or another when free software begins to come close to the realm of proprietary stuff, be it IDL, Matlab, or anything else). Cheers, f From prasansamtani at gmail.com Mon May 16 12:55:12 2005 From: prasansamtani at gmail.com (Prasan Samtani) Date: Mon, 16 May 2005 09:55:12 -0700 Subject: [SciPy-dev] Re: Fuzzy logic library for SciPy In-Reply-To: <5da497b705042518035e185dc3@mail.gmail.com> References: <5da497b705042518035e185dc3@mail.gmail.com> Message-ID: <5da497b7050516095533a0c92a@mail.gmail.com> An initial version of the library can be found at: http://www.isi.edu/~samtani/Code/FuzzyLogic.zip Obviously there's a lot of work to be done in making this conform to SciPy standards, hopefully my work/school schedule will allow me to finish this off over the summer. --Prasan On 4/25/05, Prasan Samtani wrote: > Greetings scipy-devers, > > I am currently beginning work on a Fuzzy Logic toolkit in Python > (modeled after the MatLab fuzzy logic toolbox > http://www.mathworks.com/products/fuzzylogic/). Before I get ahead of > myself, I was wondering if someone has already been working on > something like this, I want to make sure I'm not reinventing the > wheel. > > Thanks, > Prasan > From nwagner at mecha.uni-stuttgart.de Tue May 17 04:08:07 2005 From: nwagner at mecha.uni-stuttgart.de (Nils Wagner) Date: Tue, 17 May 2005 10:08:07 +0200 Subject: [SciPy-dev] AttributeError: GnuFCompiler instance has no attribute 'compiler_so' Message-ID: <4289A667.5070100@mecha.uni-stuttgart.de> Hi all, I tried to install scipy from scratch. python setup.py build failed ... compiling Fortran sources Traceback (most recent call last): File "setup.py", line 124, in ? setup_package(ignore_packages) File "setup.py", line 112, in setup_package url = "http://www.scipy.org", File "/var/tmp/cvs/scipy/scipy_core/scipy_distutils/core.py", line 73, in setup return old_setup(**new_attr) File "/usr/local/lib/python2.4/distutils/core.py", line 149, in setup dist.run_commands() File "/usr/local/lib/python2.4/distutils/dist.py", line 946, in run_commands self.run_command(cmd) File "/usr/local/lib/python2.4/distutils/dist.py", line 966, in run_command cmd_obj.run() File "/usr/local/lib/python2.4/distutils/command/build.py", line 112, in run self.run_command(cmd_name) File "/usr/local/lib/python2.4/distutils/cmd.py", line 333, in run_command self.distribution.run_command(command) File "/usr/local/lib/python2.4/distutils/dist.py", line 966, in run_command cmd_obj.run() File "/var/tmp/cvs/scipy/scipy_core/scipy_distutils/command/build_clib.py", line 115, in run self.build_libraries(self.libraries) File "/var/tmp/cvs/scipy/scipy_core/scipy_distutils/command/build_clib.py", line 218, in build_libraries extra_postargs=[]) File "/usr/local/lib/python2.4/site-packages/scipy/distutils/ccompiler.py", line 81, in CCompiler_compile ccomp = self.compiler_so AttributeError: GnuFCompiler instance has no attribute 'compiler_so' Help is needed. Thanks in advance Nils Python 2.4 (#2, May 12 2005, 14:45:33) [GCC 3.3.3 (SuSE Linux)] on linux2 Type "help", "copyright", "credits" or "license" for more information. f2py -v numerix Numeric 24.0b2 2.46.243_1985 g77 -v Reading specs from /usr/lib/gcc-lib/i586-suse-linux/3.3.3/specs Configured with: ../configure --enable-threads=posix --prefix=/usr --with-local-prefix=/usr/local --infodir=/usr/share/info --mandir=/usr/share/man --enable-languages=c,c++,f77,objc,java,ada --disable-checking --libdir=/usr/lib --enable-libgcj --with-gxx-include-dir=/usr/include/g++ --with-slibdir=/lib --with-system-zlib --enable-shared --enable-__cxa_atexit i586-suse-linux Thread model: posix gcc version 3.3.3 (SuSE Linux) From pearu at scipy.org Tue May 17 03:15:33 2005 From: pearu at scipy.org (Pearu Peterson) Date: Tue, 17 May 2005 02:15:33 -0500 (CDT) Subject: [SciPy-dev] AttributeError: GnuFCompiler instance has no attribute 'compiler_so' In-Reply-To: <4289A667.5070100@mecha.uni-stuttgart.de> References: <4289A667.5070100@mecha.uni-stuttgart.de> Message-ID: On Tue, 17 May 2005, Nils Wagner wrote: > Hi all, > > I tried to install scipy from scratch. > > python setup.py build failed > > ... > compiling Fortran sources > Traceback (most recent call last): > File "setup.py", line 124, in ? > setup_package(ignore_packages) > File "setup.py", line 112, in setup_package > url = "http://www.scipy.org", > File "/var/tmp/cvs/scipy/scipy_core/scipy_distutils/core.py", line 73, in > setup > return old_setup(**new_attr) > File "/usr/local/lib/python2.4/distutils/core.py", line 149, in setup > dist.run_commands() > File "/usr/local/lib/python2.4/distutils/dist.py", line 946, in run_commands > self.run_command(cmd) > File "/usr/local/lib/python2.4/distutils/dist.py", line 966, in run_command > cmd_obj.run() > File "/usr/local/lib/python2.4/distutils/command/build.py", line 112, in run > self.run_command(cmd_name) > File "/usr/local/lib/python2.4/distutils/cmd.py", line 333, in run_command > self.distribution.run_command(command) > File "/usr/local/lib/python2.4/distutils/dist.py", line 966, in run_command > cmd_obj.run() > File "/var/tmp/cvs/scipy/scipy_core/scipy_distutils/command/build_clib.py", > line 115, in run > self.build_libraries(self.libraries) > File "/var/tmp/cvs/scipy/scipy_core/scipy_distutils/command/build_clib.py", > line 218, in build_libraries > extra_postargs=[]) > File "/usr/local/lib/python2.4/site-packages/scipy/distutils/ccompiler.py", > line 81, in CCompiler_compile > ccomp = self.compiler_so > AttributeError: GnuFCompiler instance has no attribute 'compiler_so' You seem to have scipy.base (Numeric3) in your system. Removing scipy.base should fix the error, so do rm -rf /usr/local/lib/python2.4/site-packages/scipy before building scipy. Pearu From nwagner at mecha.uni-stuttgart.de Tue May 17 04:56:31 2005 From: nwagner at mecha.uni-stuttgart.de (Nils Wagner) Date: Tue, 17 May 2005 10:56:31 +0200 Subject: [SciPy-dev] AttributeError: GnuFCompiler instance has no attribute 'compiler_so' In-Reply-To: References: <4289A667.5070100@mecha.uni-stuttgart.de> Message-ID: <4289B1BF.1080806@mecha.uni-stuttgart.de> Pearu Peterson wrote: > > > On Tue, 17 May 2005, Nils Wagner wrote: > >> Hi all, >> >> I tried to install scipy from scratch. >> >> python setup.py build failed >> >> ... >> compiling Fortran sources >> Traceback (most recent call last): >> File "setup.py", line 124, in ? >> setup_package(ignore_packages) >> File "setup.py", line 112, in setup_package >> url = "http://www.scipy.org", >> File "/var/tmp/cvs/scipy/scipy_core/scipy_distutils/core.py", line >> 73, in setup >> return old_setup(**new_attr) >> File "/usr/local/lib/python2.4/distutils/core.py", line 149, in setup >> dist.run_commands() >> File "/usr/local/lib/python2.4/distutils/dist.py", line 946, in >> run_commands >> self.run_command(cmd) >> File "/usr/local/lib/python2.4/distutils/dist.py", line 966, in >> run_command >> cmd_obj.run() >> File "/usr/local/lib/python2.4/distutils/command/build.py", line 112, >> in run >> self.run_command(cmd_name) >> File "/usr/local/lib/python2.4/distutils/cmd.py", line 333, in >> run_command >> self.distribution.run_command(command) >> File "/usr/local/lib/python2.4/distutils/dist.py", line 966, in >> run_command >> cmd_obj.run() >> File >> "/var/tmp/cvs/scipy/scipy_core/scipy_distutils/command/build_clib.py", >> line 115, in run >> self.build_libraries(self.libraries) >> File >> "/var/tmp/cvs/scipy/scipy_core/scipy_distutils/command/build_clib.py", >> line 218, in build_libraries >> extra_postargs=[]) >> File >> "/usr/local/lib/python2.4/site-packages/scipy/distutils/ccompiler.py", >> line 81, in CCompiler_compile >> ccomp = self.compiler_so >> AttributeError: GnuFCompiler instance has no attribute 'compiler_so' > > > You seem to have scipy.base (Numeric3) in your system. Removing > scipy.base should fix the error, so do > > rm -rf /usr/local/lib/python2.4/site-packages/scipy > > before building scipy. > > Pearu > > _______________________________________________ > Scipy-dev mailing list > Scipy-dev at scipy.net > http://www.scipy.net/mailman/listinfo/scipy-dev Hi Pearu, Thank you. Now the installation works fine. But scipy.test(1,verbosity=10) yields check_random (scipy.linalg.decomp.test_decomp.test_svd) ** On entry to DGESDD parameter number 12 had an illegal value SUBROUTINE DGESDD( JOBZ, M, N, A, LDA, S, U, LDU, VT, LDVT, WORK, $ LWORK, IWORK, INFO ) Parameter number 12 is LWORK . Any idea ? Nils From nwagner at mecha.uni-stuttgart.de Tue May 17 15:03:28 2005 From: nwagner at mecha.uni-stuttgart.de (Nils Wagner) Date: Tue, 17 May 2005 21:03:28 +0200 Subject: [SciPy-dev] On entry to DGESDD parameter number 12 had an illegal value Message-ID: Hi all, Can someone reproduce the following error ? scipy.test(1,verbosity=10) ... check_random (scipy.linalg.decomp.test_decomp.test_svd) ** On entry to DGESDD parameter number 12 had an illegal value Parameter number 12 is connected with LWORK. Any idea ? Nils From nwagner at mecha.uni-stuttgart.de Wed May 18 04:24:44 2005 From: nwagner at mecha.uni-stuttgart.de (Nils Wagner) Date: Wed, 18 May 2005 10:24:44 +0200 Subject: [SciPy-dev] linalg.flapack strange argument in *gesdd Message-ID: <428AFBCC.9040409@mecha.uni-stuttgart.de> Hi all, A help (linalg.flpack) yields This module 'flapack' is auto-generated with f2py (version:2.46.243_1985) u,s,vt,info = sgesdd(a,compute_uv=1,lwork=(compute_uv?4*minmn*minmn+MAX(m,n)+9*minmn:MAX(14*minmn+4,10*minmn+2+25*(25+8))+MAX(m,n)),overwrite_a=0) u,s,vt,info = dgesdd(a,compute_uv=1,lwork=(compute_uv?4*minmn*minmn+MAX(m,n)+9*minmn:MAX(14*minmn+4,10*minmn+2+25*(25+8))+MAX(m,n)),overwrite_a=0) u,s,vt,info = cgesdd(a,compute_uv=1,lwork=(compute_uv?2*minmn*minmn+MAX(m,n)+2*minmn:2*minmn+MAX(m,n)),overwrite_a=0) u,s,vt,info = zgesdd(a,compute_uv=1,lwork=(compute_uv?2*minmn*minmn+MAX(m,n)+2*minmn:2*minmn+MAX(m,n)),overwrite_a=0) What is the meaning of compute_uv?4*minmn*minmn ? Any idea ? Nils From nwagner at mecha.uni-stuttgart.de Wed May 18 04:31:56 2005 From: nwagner at mecha.uni-stuttgart.de (Nils Wagner) Date: Wed, 18 May 2005 10:31:56 +0200 Subject: [SciPy-dev] QL factorization Message-ID: <428AFD7C.70906@mecha.uni-stuttgart.de> Hi all, How can I compute a QL factorization of a m \times n matrix in scipy ? In LAPACK we have http://www.netlib.org/lapack/double/dgeqlf.f Any pointer would be appreciated. Thanks in advance. Nils Reference: Ghavimi and Laub Numerical methods for nearly singular constrained matrix Sylvester equations. SIAM J. Matrix Anal. Appl. 17/1 (1996) pp. 212-221 From pearu at scipy.org Wed May 18 03:32:44 2005 From: pearu at scipy.org (Pearu Peterson) Date: Wed, 18 May 2005 02:32:44 -0500 (CDT) Subject: [SciPy-dev] linalg.flapack strange argument in *gesdd In-Reply-To: <428AFBCC.9040409@mecha.uni-stuttgart.de> References: <428AFBCC.9040409@mecha.uni-stuttgart.de> Message-ID: On Wed, 18 May 2005, Nils Wagner wrote: > Hi all, > > A help (linalg.flpack) yields > > This module 'flapack' is auto-generated with f2py (version:2.46.243_1985) > > u,s,vt,info = > sgesdd(a,compute_uv=1,lwork=(compute_uv?4*minmn*minmn+MAX(m,n)+9*minmn:MAX(14*minmn+4,10*minmn+2+25*(25+8))+MAX(m,n)),overwrite_a=0) > u,s,vt,info = > dgesdd(a,compute_uv=1,lwork=(compute_uv?4*minmn*minmn+MAX(m,n)+9*minmn:MAX(14*minmn+4,10*minmn+2+25*(25+8))+MAX(m,n)),overwrite_a=0) > u,s,vt,info = > cgesdd(a,compute_uv=1,lwork=(compute_uv?2*minmn*minmn+MAX(m,n)+2*minmn:2*minmn+MAX(m,n)),overwrite_a=0) > u,s,vt,info = > zgesdd(a,compute_uv=1,lwork=(compute_uv?2*minmn*minmn+MAX(m,n)+2*minmn:2*minmn+MAX(m,n)),overwrite_a=0) > > What is the meaning of compute_uv?4*minmn*minmn ? What you see in lwork initial value is given in C syntax. The corresponding complete C statement would be ( ? : ) Pearu From arnd.baecker at web.de Wed May 18 04:46:09 2005 From: arnd.baecker at web.de (Arnd Baecker) Date: Wed, 18 May 2005 10:46:09 +0200 (CEST) Subject: [SciPy-dev] linalg.flapack strange argument in *gesdd In-Reply-To: <428AFBCC.9040409@mecha.uni-stuttgart.de> References: <428AFBCC.9040409@mecha.uni-stuttgart.de> Message-ID: On Wed, 18 May 2005, Nils Wagner wrote: > Hi all, > > A help (linalg.flpack) yields > > This module 'flapack' is auto-generated with f2py (version:2.46.243_1985) > > u,s,vt,info = > sgesdd(a,compute_uv=1,lwork=(compute_uv?4*minmn*minmn+MAX(m,n)+9*minmn:MAX(14*minmn+4,10*minmn+2+25*(25+8))+MAX(m,n)),overwrite_a=0) > u,s,vt,info = > dgesdd(a,compute_uv=1,lwork=(compute_uv?4*minmn*minmn+MAX(m,n)+9*minmn:MAX(14*minmn+4,10*minmn+2+25*(25+8))+MAX(m,n)),overwrite_a=0) > u,s,vt,info = > cgesdd(a,compute_uv=1,lwork=(compute_uv?2*minmn*minmn+MAX(m,n)+2*minmn:2*minmn+MAX(m,n)),overwrite_a=0) > u,s,vt,info = > zgesdd(a,compute_uv=1,lwork=(compute_uv?2*minmn*minmn+MAX(m,n)+2*minmn:2*minmn+MAX(m,n)),overwrite_a=0) > > What is the meaning of compute_uv?4*minmn*minmn ? The "?" is the ternary operator. In this case it means: if compute_uv is 1 (the default), then define lwork as 4*minmn*minmn+MAX(m,n)+9*minmn if not: take the other value written after the ":". If everything works fine you should not bother about setting lwork manually. To really understand all this you will have to look at the documentation for sgesdd via `man sgesdd` (assuming the lapack man-pages are installed properly), or have a look at http://www.netlib.org/lapack/single/sgesdd.f BTW: I think it would be great if the information available at the above link or via `man sgesdd` was accessible from within (I)python via the interactive help. Best, Arnd From nwagner at mecha.uni-stuttgart.de Wed May 18 04:46:17 2005 From: nwagner at mecha.uni-stuttgart.de (Nils Wagner) Date: Wed, 18 May 2005 10:46:17 +0200 Subject: [SciPy-dev] Segmenation fault in flapack.zgesdd Message-ID: <428B00D9.1030509@mecha.uni-stuttgart.de> Hi all, The enclosed test program stops with a segmentation fault. (gdb) run test_zgesdd.py Starting program: /usr/local/bin/python test_zgesdd.py [Thread debugging using libthread_db enabled] [New Thread 1076102528 (LWP 11378)] numerix Numeric 24.0b2 Program received signal SIGSEGV, Segmentation fault. [Switching to Thread 1076102528 (LWP 11378)] 0x4018d208 in _int_free () from /lib/tls/libc.so.6 (gdb) bt #0 0x4018d208 in _int_free () from /lib/tls/libc.so.6 #1 0x4018d5fb in free () from /lib/tls/libc.so.6 #2 0x4030e17b in array_dealloc (self=Variable "self" is not available. ) at Src/arrayobject.c:630 #3 0x407e3b3a in f2py_rout_flapack_zgesdd (capi_self=0x40294638, capi_args=0x403fd1ac, capi_keywds=0x0, f2py_func=0x408a71f0 ) at build/src/build/src/scipy/linalg/flapackmodule.c:5753 #4 0x407ee172 in fortran_call (fp=Variable "fp" is not available. ) at build/src/fortranobject.c:277 #5 0x0805928e in PyObject_Call (func=0x40294638, arg=0x403fd1ac, kw=0x0) at Objects/abstract.c:1746 #6 0x080c408d in PyEval_EvalFrame (f=0x8196d64) at Python/ceval.c:3755 #7 0x080c7a54 in PyEval_EvalCodeEx (co=0x4027bf20, globals=0x40259824, locals=0x40259824, args=0x0, argcount=0, kws=0x0, kwcount=0, defs=0x0, defcount=0, closure=0x0) at Python/ceval.c:2730 #8 0x080c7c85 in PyEval_EvalCode (co=0x4027bf20, globals=0x40259824, locals=0x40259824) at Python/ceval.c:484 #9 0x080f63b8 in PyRun_SimpleFileExFlags (fp=0x815e008, filename=0xbfffec0f "test_zgesdd.py", closeit=1, flags=0xbfffe904) at Python/pythonrun.c:1264 #10 0x08055857 in Py_Main (argc=1, argv=0xbfffe9c4) at Modules/main.c:484 #11 0x08054f07 in main (argc=2, argv=0xbfffe9c4) at Modules/python.c:23 (gdb) Any pointer would be appreciated. Nils -------------- next part -------------- A non-text attachment was scrubbed... Name: test_zgesdd.py Type: text/x-python Size: 100 bytes Desc: not available URL: From rkern at ucsd.edu Wed May 18 04:51:00 2005 From: rkern at ucsd.edu (Robert Kern) Date: Wed, 18 May 2005 01:51:00 -0700 Subject: [SciPy-dev] Segmenation fault in flapack.zgesdd In-Reply-To: <428B00D9.1030509@mecha.uni-stuttgart.de> References: <428B00D9.1030509@mecha.uni-stuttgart.de> Message-ID: <428B01F4.3020900@ucsd.edu> Nils Wagner wrote: > Hi all, > > The enclosed test program stops with a segmentation fault. Works fine for me. OS X 10.4 Python 2.4.1 Scipy CVS Numeric CVS ATLAS 3.7.8 -- Robert Kern rkern at ucsd.edu "In the fields of hell where the grass grows high Are the graves of dreams allowed to die." -- Richard Harter From nwagner at mecha.uni-stuttgart.de Wed May 18 04:55:52 2005 From: nwagner at mecha.uni-stuttgart.de (Nils Wagner) Date: Wed, 18 May 2005 10:55:52 +0200 Subject: [SciPy-dev] Segmenation fault in flapack.zgesdd In-Reply-To: <428B01F4.3020900@ucsd.edu> References: <428B00D9.1030509@mecha.uni-stuttgart.de> <428B01F4.3020900@ucsd.edu> Message-ID: <428B0318.1010900@mecha.uni-stuttgart.de> Robert Kern wrote: > Nils Wagner wrote: > >> Hi all, >> >> The enclosed test program stops with a segmentation fault. > > > Works fine for me. > > OS X 10.4 > Python 2.4.1 > Scipy CVS > Numeric CVS > ATLAS 3.7.8 > And how about f2py -v ? numerix Numeric 24.0b2 2.46.243_1985 >>> scipy.__version__ '0.3.3_305.4604' Python 2.4 (#2, May 12 2005, 14:45:33) [GCC 3.3.3 (SuSE Linux)] on linux2 From rkern at ucsd.edu Wed May 18 04:59:58 2005 From: rkern at ucsd.edu (Robert Kern) Date: Wed, 18 May 2005 01:59:58 -0700 Subject: [SciPy-dev] Segmenation fault in flapack.zgesdd In-Reply-To: <428B0318.1010900@mecha.uni-stuttgart.de> References: <428B00D9.1030509@mecha.uni-stuttgart.de> <428B01F4.3020900@ucsd.edu> <428B0318.1010900@mecha.uni-stuttgart.de> Message-ID: <428B040E.8060603@ucsd.edu> Nils Wagner wrote: > Robert Kern wrote: > >> Nils Wagner wrote: >> >>> Hi all, >>> >>> The enclosed test program stops with a segmentation fault. >> >> >> >> Works fine for me. >> >> OS X 10.4 >> Python 2.4.1 >> Scipy CVS >> Numeric CVS >> ATLAS 3.7.8 >> > And how about f2py -v ? > > numerix Numeric 24.0b2 > 2.46.243_1985 The last real release, I believe. Not CVS. 2.45.241_1926 -- Robert Kern rkern at ucsd.edu "In the fields of hell where the grass grows high Are the graves of dreams allowed to die." -- Richard Harter From mmetz at astro.uni-bonn.de Wed May 18 05:02:12 2005 From: mmetz at astro.uni-bonn.de (Manuel Metz) Date: Wed, 18 May 2005 11:02:12 +0200 Subject: [SciPy-dev] Segmenation fault in flapack.zgesdd In-Reply-To: <428B01F4.3020900@ucsd.edu> References: <428B00D9.1030509@mecha.uni-stuttgart.de> <428B01F4.3020900@ucsd.edu> Message-ID: <428B0494.4030909@astro.uni-bonn.de> Strange behavior for me (Debian Linux (sid), Python 2.3.5, SciPy 0.3.2, Numeric 23.8, Atlas 3.6.0): Result of first call: ************************************************************************ :~/temp> python test_zgesdd.py Exception in thread Thread-1 (most likely raised during interpreter shutdown): Traceback (most recent call last): File "/usr/lib/python2.3/threading.py", line 442, in __bootstrap File "/usr/lib/python2.3/threading.py", line 422, in run File "/usr/lib/python2.3/site-packages/rpy.py", line 119, in r_eventloop exceptions.AttributeError: 'NoneType' object has no attribute 'release' Unhandled exception in thread started by Error in sys.excepthook: Original exception was: ************************************************************************ Second call: ************************************************************************ ~/temp> python test_zgesdd.py Unhandled exception in thread started by Error in sys.excepthook: Original exception was: ************************************************************************ I get the second message several times, however not reproducible. Sometimes the script finished without any error message. The same for python2.4.1. I saw this "threading problem" several times on my box... This seems to be rpy related... Manuel Robert Kern schrieb: > Nils Wagner wrote: > >> Hi all, >> >> The enclosed test program stops with a segmentation fault. > > > Works fine for me. > > OS X 10.4 > Python 2.4.1 > Scipy CVS > Numeric CVS > ATLAS 3.7.8 > -- ------------------------------------- Manuel Metz Sternwarte der Universitaet Bonn Auf dem Huegel 71 (room 3.06) D - 53121 Bonn E-Mail: mmetz at astro.uni-bonn.de Phone: (+49) 228 / 73-3660 Fax: (+49) 228 / 73-3672 ------------------------------------- From oliphant at ee.byu.edu Thu May 19 16:42:46 2005 From: oliphant at ee.byu.edu (Travis Oliphant) Date: Thu, 19 May 2005 14:42:46 -0600 Subject: [SciPy-dev] Re: Numeric 3 status In-Reply-To: <20050519201047.GJ5213@modulation.mit.edu> References: <20050408130007.GH14150@modulation.mit.edu> <4256F21F.7040706@ee.byu.edu> <20050519201047.GJ5213@modulation.mit.edu> Message-ID: <428CFA46.9010406@ee.byu.edu> Eric Jonas wrote: >>I forgot to let you know in my previous email, that I would be very, >>very pleased to receive help on scipy.base. There is a great deal of >>opportunity for people to get involved and assist with the formation of >>a solid scipy.base. I listed a few areas in my previous email, but I'm >>sure I forgot some things. >> >> > >Well, I have an undergraduate working with me with some decent C and >python experience and an interest in numerical computing; we've >downloaded the latest from the SF CVS Numeric3 module and are going to >start looking over things; are there any mailing lists or other >recommended sources of information? As the person in charge of all of >this, is there some area you'd prefer for us to get started on ? We'd >love to help you get this all working this summer... > > Fantastic, Spring term started, and my time for Numeric3 dried up considerably. Fortunately, there is not that much left to do algorithmically but what is left is somewhat difficult. Three things remain for the ufuncs: 1) altered coercion rules to treat scalars differently --- should not be difficult to code (in fact mostly done already SEE scipy.alter_numeric) 2) change the reduce (accumulate), and reduceat functions to handle misaligned and byteswapped data using array iterators and buffers when necessary (i.e. adapt what has been done for general ufuncs to these methods). This is the hard part. I pretty much know how to do it, but it takes a big chunk of time and thought process on my part and I have not been able to get to it for the past month. At the same time, I would like to alter these ufunc methods so that you can specify the argument type they "reduce into" i.e. you should be able to specify that a reduce on a Int8 array returns an output in an Int32 array. 3) add the error handling functionality of numarray. Ufuncs that cause errors will flip the IEEE hardware flag. This needs to be checked if that is the specified error handling mode. This shouldn't take too long. So, #2 is the hard part and is what has me stopped (I know it will take me several hours to days to code it correctly --- and I've not been able to set aside that time). After these three things are accomplished, then what remains is on the critical path is: Updating the scipy numerix module to use scipy.base by default, making sure that scipy.base can ship by itself and have the same basic modules that Numeric provides and a clean path to upgrading to full ATLAS-based linear algebra and/or improved FFT performance. Realistically, this all may not be finished until the scipy conference in September. I'd like it to be done earlier than that of course. The reality is that there are very few actually willing to work on the code base and a great many that will actually want to use it. Those with the C-skill necessary to help (or willing and able to learn some C) are apparently few in number. Any help you provide will be appreciated. You are welcome to tackle anything you think would be useful. -Travis From oliphant at ee.byu.edu Thu May 19 16:50:41 2005 From: oliphant at ee.byu.edu (Travis Oliphant) Date: Thu, 19 May 2005 14:50:41 -0600 Subject: [SciPy-dev] Re: Numeric 3 status In-Reply-To: <20050519201047.GJ5213@modulation.mit.edu> References: <20050408130007.GH14150@modulation.mit.edu> <4256F21F.7040706@ee.byu.edu> <20050519201047.GJ5213@modulation.mit.edu> Message-ID: <428CFC21.4010903@ee.byu.edu> Eric Jonas wrote: >>I forgot to let you know in my previous email, that I would be very, >>very pleased to receive help on scipy.base. There is a great deal of >>opportunity for people to get involved and assist with the formation of >>a solid scipy.base. I listed a few areas in my previous email, but I'm >>sure I forgot some things. >> >> > >Well, I have an undergraduate working with me with some decent C and >python experience and an interest in numerical computing; we've >downloaded the latest from the SF CVS Numeric3 module and are going to >start looking over things; are there any mailing lists or other >recommended sources of information? As the person in charge of all of >this, is there some area you'd prefer for us to get started on ? We'd >love to help you get this all working this summer... > ...Eric > > I forgot to mention that there are a couple of other projects that are more Python-related that would also be important long term and could easily be handled by an undergraduate: 1) Changing Python pickle code so that a buffer object can be pickled directly (without first copying to a string). -- It would be nice to have a better buffer object which has been discussed ad nausem on python-dev, but this simple addition would add a great deal of functionality. -- It might be coupled with a way to get an empty buffer object from Python (i.e.) buffer(1024) would construct an empty writeable buffer of 1024 bytes (you can do this with the Python C-API, but it is not exposed to the Python Interpreter). 2) Adding the __asindex__ method (and equivalent C call) to Python objects so that arbitrary objects can be used to slice Python sequences instead of only integers being allowed as is currently the case. Both of these projects are important to the long term success of Numeric computing on Python and would have positive impact on the entire Python community. It is important that somebody takes these on as soon as possible. Thanks, -Travis From stephen.walton at csun.edu Fri May 20 14:58:03 2005 From: stephen.walton at csun.edu (Stephen Walton) Date: Fri, 20 May 2005 11:58:03 -0700 Subject: [SciPy-dev] On entry to DGESDD parameter number 12 had an illegal value In-Reply-To: References: Message-ID: <428E333B.7030004@csun.edu> Nils Wagner wrote: > Hi all, > > Can someone reproduce the following error ? > > scipy.test(1,verbosity=10) > > ... > check_random (scipy.linalg.decomp.test_decomp.test_svd) ** On entry to > DGESDD parameter number 12 had an illegal value I've tried several times to reproduce this on both FC1 and FC3. Using Python 2.3 on FC1, scipy.test(1) passes all tests. Latest Scipy CVS as of today. From nwagner at mecha.uni-stuttgart.de Fri May 20 15:04:43 2005 From: nwagner at mecha.uni-stuttgart.de (Nils Wagner) Date: Fri, 20 May 2005 21:04:43 +0200 Subject: [SciPy-dev] On entry to DGESDD parameter number 12 had an illegal value In-Reply-To: <428E333B.7030004@csun.edu> References: <428E333B.7030004@csun.edu> Message-ID: On Fri, 20 May 2005 11:58:03 -0700 Stephen Walton wrote: > Nils Wagner wrote: > >> Hi all, >> >> Can someone reproduce the following error ? >> >> scipy.test(1,verbosity=10) >> >> ... >> check_random (scipy.linalg.decomp.test_decomp.test_svd) >>** On entry to >> DGESDD parameter number 12 had an illegal value > > I've tried several times to reproduce this on both FC1 >and FC3. Using Python 2.3 on FC1, scipy.test(1) passes >all tests. Latest Scipy CVS as of today. > Please use l a t e s t f2py (cvs) and run the test again. You can also try from scipy import * n = 20 m = 15 a = rand(n,m)+1j*rand(n,m) u,s,vt,info = linalg.flapack.zgesdd(a) I am curious about your findings. Nils > > _______________________________________________ > Scipy-dev mailing list > Scipy-dev at scipy.net > http://www.scipy.net/mailman/listinfo/scipy-dev From stephen.walton at csun.edu Fri May 20 15:58:54 2005 From: stephen.walton at csun.edu (Stephen Walton) Date: Fri, 20 May 2005 12:58:54 -0700 Subject: [SciPy-dev] On entry to DGESDD parameter number 12 had an illegal value In-Reply-To: References: <428E333B.7030004@csun.edu> Message-ID: <428E417E.4070201@csun.edu> Nils Wagner wrote: > Please use l a t e s t f2py (cvs) and run the test again. Maybe when I return in June. I'm leaving for a meeting in two days and just don't have the time right now. Also, since f2py 2.45.241_1936 works fine here, I'm reluctant to change. "If it ain't broke, don't fix it." From p.berkes at biologie.hu-berlin.de Thu May 26 12:11:24 2005 From: p.berkes at biologie.hu-berlin.de (p.berkes at biologie.hu-berlin.de) Date: Thu, 26 May 2005 18:11:24 +0200 (CEST) Subject: [SciPy-dev] patch for scipy_base.diag Message-ID: Hi! There is a casting problem in the 'diag' function: >>> a = scipy.array([1,2,3], 'f') >>> scipy.diag(a).typecode() >>> 'd' while it should be 'f'. The function can be easily patched (only the calls to eye are modified): In file: /home2/scipy/local/lib/python2.3/site-packages/scipy_base/matrix_base.py def diag(v, k=0): """ returns the k-th diagonal if v is a matrix or returns a matrix with v as the k-th diagonal if v is a vector. """ v = asarray(v) s = v.shape if len(s)==1: n = s[0]+abs(k) if k > 0: v = concatenate((zeros(k, v.typecode()),v)) elif k < 0: v = concatenate((v, zeros(-k, v.typecode()))) return eye(n, k=k, typecode=v.typecode())*v elif len(s)==2: v = add.reduce(eye(s[0], s[1], k=k, typecode=v.typecode())*v) if k > 0: return v[k:] elif k < 0: return v[:k] else: return v else: raise ValueError, "Input must be 1- or 2-D." Best, Pietro. From chris at pseudogreen.org Mon May 30 17:13:09 2005 From: chris at pseudogreen.org (Christopher Stawarz) Date: Mon, 30 May 2005 17:13:09 -0400 Subject: [SciPy-dev] IDL to Python converter available Message-ID: <1117487589.5525.51.camel@localhost> I've written a package, i2py, that converts IDL scripts and programs to Python. It started as a project for the compilers class I took this spring, and in the weeks since I've cleaned it up and improved it enough that I think it's ready for an alpha release. Since there was recently a discussion on this list about this sort of thing, I expect some people will be interested in trying it out. The package is implemented in pure Python and requires Python 2.3 or later. Using PLY (which is included in the distribution), it implements a full-fledged parser for procedural IDL (no OO support yet) and can convert most (but not yet all) IDL language constructs to Python. More information on it, including some notes on its current limitations and a brief introductory tutorial, can be found at http://software.pseudogreen.org/i2py/ Source packages are available for download from ftp://pseudogreen.org/pub/i2py/ If you'd like to try it out, I'd definitely appreciate any feedback. (If there's a lot of interest, I may move the project to SourceForge or somewhere similar, but I'll have to see what happens.) Since the scipy-dev thread I mentioned earlier was mainly about the legal issues surrounding this kind of software, I think I should address those a bit. First, I should make it clear that I've developed i2py entirely on my own time, using my own resources. However, since I am a full-time programmer at an astronomical research institute and have, on a few occasions, used IDL as part of my job, I thought it wise to discuss the potential issues with my employers before releasing the package. The conclusion reached in that discuss was that, although IDL's EULA explicitly forbids reverse engineering, implementing a parser and converter for the language does *not* constitute reverse engineering, because it requires only the published, publicly-available description of the language. (Similarly, if I use Visual C++ and then go buy Stroustrup's book and write my own C++ compiler, my actions don't constitute reverse engineering, because anything I might learn about C++ from using VC++ can also be learned from the book.) Hence, I feel quite confident that my development of i2py has neither violated the IDL EULA nor infringed upon RSI's intellectual property rights in any way. Of course, if you think otherwise, then you shouldn't use the package. Anyway, I hope some of you will find i2py useful, and I look forward to the chance to improve it based on your feedback. Thanks, Chris Stawarz