From tbach at bu.edu Thu Apr 8 13:48:51 2004 From: tbach at bu.edu (Ted Bach) Date: Thu, 8 Apr 2004 13:48:51 -0400 (EDT) Subject: [SciPy-dev] Resolving win 32 problems Message-ID: Dear Scipy Developers, I am considering using scipy as a foundation and helper package for my own SIMP/STEP (http://pm.bu.edu/step) software for cellular automata and lattice gas experiments. The scipy environment would be helpful in providing rendering and viewing capabilities, allowing external compilation of fast routines via weave, as well as providing auxilary libraries for plotting statistics gathered from long simulations. I have been quite impressed with scipy on Linux, however, an important aspect of my choice is Win32 compatability. Unfortunately, I have not been able to get any of the Win32 binaries to work at all. For example, simply importing scipy causes a crash! (On Windows XP with python2.3 and SciPy-0.2.1_260.4162.win32-py2.3.exe and Scipy_core-0.2.2_alpha_108.1756.win32-py2.3.exe). I have seen several reports that the Windows release has not been working (on the Bug Tracker) and have begun to wonder how far the project has drifted from Windows compatability and whether the problems are serious. Are there plans to fix these problems in the near future? Since I am quite keen to use scipy and potentially integrate my software with it down the road, I would be happy to spend some time trying to address these issues myself if someone can point me to the current suspected problems and the strategy (if it exists) for overcoming them. Cheers, Ted Ted Bach tbach at bu.edu |home: Boston University 617/353-0294 |617/776-2926 ECE Dept. fax -6640 |223 Somerville Ave Apt 2 8 St Mary's St. |Somerville, MA 02143 Boston, MA 02215 | From bgoli at sun.ac.za Thu Apr 8 14:33:08 2004 From: bgoli at sun.ac.za (Brett G. Olivier) Date: Thu, 8 Apr 2004 20:33:08 +0200 (SAST) Subject: [SciPy-dev] Resolving win 32 problems In-Reply-To: References: Message-ID: <1195.196.30.180.142.1081449188.squirrel@glue.jjj.sun.ac.za> Hi Ted I'm not a Scipy developer, but currently use Scipy for development on both win2k and linux. Personally I've found the best way is to build everything from scratch (from the SciPy CVS source) using a combination of MinGW and Cygwin. Pearu Peterson has excellent instructions on using Cygwin/MinGW to build scipy at: http://cens.ioc.ee/~pearu/scipy/BUILD_WIN32.html Other possibilities are also to use Cygwin to build Atlas+Lapack (which is a once of thing) and then MinGW (with gcc 3.3.3+) for building Numeric/SciPy etc. My experience has been that recent (cvs) versions of SciPy are all windows/linux friendly, but I'd suggest building from source and then evaluating the situation. Best regards Brett Ted Bach wrote: > I have seen several reports that the Windows release has not been working (on the Bug Tracker) and have begun to wonder how far the project has drifted from Windows compatability and whether the problems are serious. Are there plans to fix these problems in the near future? > > Since I am quite keen to use scipy and potentially integrate my software with it down the road, I would be happy to spend some time trying to address these issues myself if someone can point me to the current suspected problems and the strategy (if it exists) for overcoming them. > Cheers, > > Ted > -- Brett G. Olivier (bgoli at sun dot ac dot za) Biochemistry Dept, Stellenbosch University Tel: +27-21-8085871 Fax: +27-218085863 "Don't think; let the machine do it for you!" -- E. C. Berkeley From travis at enthought.com Thu Apr 8 16:07:25 2004 From: travis at enthought.com (Travis N. Vaught) Date: Thu, 08 Apr 2004 15:07:25 -0500 Subject: [SciPy-dev] Resolving win 32 problems In-Reply-To: References: Message-ID: <4075B0FD.3090800@enthought.com> Ted Bach wrote: >Dear Scipy Developers, > >I am considering using scipy as a foundation and helper package for >my own SIMP/STEP (http://pm.bu.edu/step) software for cellular automata >and lattice gas experiments. The scipy environment would be helpful >in providing rendering and viewing capabilities, allowing external >compilation of fast routines via weave, as well as providing auxilary >libraries for plotting statistics gathered from long simulations. > >I have been quite impressed with scipy on Linux, however, an important >aspect of my choice is Win32 compatability. Unfortunately, I have not >been able to get any of the Win32 binaries to work at all. For example, >simply importing scipy causes a crash! (On Windows XP with python2.3 >and SciPy-0.2.1_260.4162.win32-py2.3.exe and >Scipy_core-0.2.2_alpha_108.1756.win32-py2.3.exe). > > This is the point where I typically recommend using the Python Distribution that Enthought puts out: http://www.enthought.com/python It includes scipy and many other useful pieces, and, most importantly, installs and works out of the box. >I have seen several reports that the Windows release has not been >working (on the Bug Tracker) and have begun to wonder how far the project >has drifted from Windows compatability and whether the problems are >serious. Are there plans to fix these problems in the near future? > > There will soon be another tagged release of scipy (version 0.3) within a week or two and we'll make sure the windows binaries are working. Others may be able to provide more immediate help with your specific problems. >Since I am quite keen to use scipy and potentially integrate my software >with it down the road, I would be happy to spend some time trying to >address these issues myself if someone can point me to the current >suspected problems and the strategy (if it exists) for overcoming them. > > We welcome any help you may be able to provide--I'll try to take a look at the latest build and see what's happening. BTW, we plan to refactor the creaking scipy.org site and announce a new plone-based site soon, so help in the areas of documentation and platform testing would be greatly appreciated. We'll include a release plan/strategy as well, to try to identify and get feedback on areas where future development will concentrate. Thanks for the interest. Travis (Vaught) >Cheers, > >Ted > >Ted Bach tbach at bu.edu |home: >Boston University 617/353-0294 |617/776-2926 >ECE Dept. fax -6640 |223 Somerville Ave Apt 2 >8 St Mary's St. |Somerville, MA 02143 >Boston, MA 02215 | > >_______________________________________________ >Scipy-dev mailing list >Scipy-dev at scipy.net >http://www.scipy.net/mailman/listinfo/scipy-dev > From tbach at bu.edu Thu Apr 8 16:45:56 2004 From: tbach at bu.edu (Ted Bach) Date: Thu, 8 Apr 2004 16:45:56 -0400 (EDT) Subject: [SciPy-dev] Resolving win 32 problems In-Reply-To: <4075B0FD.3090800@enthought.com> Message-ID: Thanks Brett and Travis for your helpful comments. It is heartening to hear that one can compile the code from scratch or just install everything in the Enthought `treasure trove'. (Someone might consider mentioning this on the Win32 scipy installation site!) Now that I have confidence that I will be able to point end users to working scipy distros on all systems, I'll get into the guts of investigating realigning my software to scipy. Of course, I'll share any mutually beneficial fixes, updates, or improvements that may come to mind while I'm exploring and flexing the code. Thanks again! Ted From js at jeannot.org Fri Apr 9 14:06:28 2004 From: js at jeannot.org (Jean-Sebastien Roy) Date: Fri, 09 Apr 2004 20:06:28 +0200 Subject: [SciPy-dev] Nonlinear constrained optimizer in SciPy Message-ID: <4076E624.2010201@jeannot.org> Following an earlier exchange with eric jones, I would like to suggest the addition of a non linear constrained optimizer to SciPy (a functionnality currently lacking). In 2001, a colleague and I did a review on non-linear bound constrained optimizer that only required the gradient of the function to be optimized. We did our review mosty based upon softwares listed in H. Mittlemann optimization guide (http://plato.la.asu.edu/guide.html), the non linear programming faq (http://www-unix.mcs.anl.gov/otc/Guide/faq/) and the NEOS guide (http://www-fp.mcs.anl.gov/otc/Guide/), plus a few other soucres (HARWELL notably). We tested about 12 different optimizers for which the source code was available and the result, while foreseeable, was not very interesting : performance is largely dependent on the problem, and tuning of the solvers parameters can result in huge performance increases. Nevertheless, in this restricted class of optimizers, a few emerged as quite good on a variety of problems. Some of them would probably be good candidates for inclusion in SciPy. (Note: Very few codes come with a license explicitly allowing free redistribution and inclusion in commercial software) L-BFGS-B: The Fortran source can be downloaded here: http://www.ece.northwestern.edu/~nocedal/lbfgsb.html We wrote to Nocedal who did not to oppose inclusion in our software, so maybe we could ask him for inclusion in SciPy ? It was the simplest to use, iterations are fast, and it was quite robust. Probably the first code I would try on any problem. M2QN1: The Fortran source can be extracted from the scilab distribution. See also: http://www-rocq.inria.fr/estime/modulopt/optimization-routines/m2qn1.html An LBGFS code like the previous one, reasonnably simple to use, very fast, very efficient. The use in commercial software if forbiden by scilab's licence, but we may still ask the author, Claude Lemar?chal, who is a very nice person. TNBC: http://iris.gmu.edu/~snash/nash/software/software.html A bound constrained truncated newton code. Stephen Nash did not oppose use in our code, and since this code provided the best performance on our problem (which may not be the case on other problem), I did a C version of it (with some modifications) and recently added a Python interface. It's distributed with a BSD license, so it could be readily integrated in SciPy (but I'm obviously biased). You can download it here (look for TNC): http://www.jeannot.org/~js/code/index.en.html A few more general codes: DONLP2: http://plato.la.asu.edu/donlp2.html SQP code with non linear constraints (including equality constraints). While it was quite difficult to use in our problem, it works well, and is very general. The licence oppose commercial use, but we could ask the author, Peter Spellucci about inclusion in SciPy. TRON: http://www-unix.mcs.anl.gov/~more/tron/ Trust region newton code. Fast iterations. Sparse Hessian estimation using finite differences, so it may be more difficult to use (since the sparsity pattern should be computed). Again, Jorge More must be asked about inclusion. Other interesting codes include: IPOPT: http://www-124.ibm.com/developerworks/opensource/coin/Ipopt/ An interior point code, very general (non linear equality constraints, bounds). We did not review it (its quite recent). It comes with a free license, so integration in SciPy should be possible. (Some options of IPOPT requires part of the HSL library, which cannot be used in commercial software). DFO: http://www-124.ibm.com/developerworks/opensource/coin/faqs.html#DFO A derivative free optimisation code, which, like IPOPT, comes with a free license. But it requires a non-linear gradient based optimizer and some developpement. COBYLA: http://plato.la.asu.edu/topics/problems/nlores.html Another alternative for derivative free optimisation, which seems free. There are probably other codes, better alternatives, which should be considered for inclusion in SciPy. Any suggestions ? Regards, js From eric at enthought.com Fri Apr 9 15:48:50 2004 From: eric at enthought.com (eric jones) Date: Fri, 09 Apr 2004 14:48:50 -0500 Subject: [SciPy-dev] Nonlinear constrained optimizer in SciPy In-Reply-To: <4076E624.2010201@jeannot.org> References: <4076E624.2010201@jeannot.org> Message-ID: <4076FE22.5070204@enthought.com> Just to follow up... Adding one (or multiple) libraries would really plug a big hole in SciPy. Jean-Sebastien has done both a lot of homework and a lot of work here -- thanks a ton. Ideally, we would define a object model that could be wrapped around any of these libraries so that they could be used interchangebly. The "best solver for common cases" would be the "default" solver that people would use when they aren't sure what to do. People with more experience could try the other solvers as they needed. It looks like there is at least one solver here that fits with SciPy's license, so that would seem the place to start. Gathering permission from other authors could proceed in parallel. Jean Sebastien, do you have time to do this? It would good to hear if others have knowledge of other libraries that should be considered. As far as logistics, there is actually going to be an "official" point release within the next couple of weeks (really!). After that, we can start putting this code in the tree. Jean-Sebastien, would you be interested in working on this? If so, we'll get you CVS access. Also, I expect Travis O. will want to be in on this because optimization is near and dear to his heart. :-) Thanks again for all this information and code. I hope to see something of this in SciPy soon. eric Jean-Sebastien Roy wrote: > Following an earlier exchange with eric jones, I would like to suggest > the addition of a non linear constrained optimizer to SciPy (a > functionnality currently lacking). > > In 2001, a colleague and I did a review on non-linear bound > constrained optimizer that only required the gradient of the function > to be optimized. We did our review mosty based upon softwares listed > in H. Mittlemann optimization guide > (http://plato.la.asu.edu/guide.html), the non linear programming faq > (http://www-unix.mcs.anl.gov/otc/Guide/faq/) and the NEOS guide > (http://www-fp.mcs.anl.gov/otc/Guide/), plus a few other soucres > (HARWELL notably). > > We tested about 12 different optimizers for which the source code was > available and the result, while foreseeable, was not very interesting > : performance is largely dependent on the problem, and tuning of the > solvers parameters can result in huge performance increases. > Nevertheless, in this restricted class of optimizers, a few emerged as > quite good on a variety of problems. Some of them would probably be > good candidates for inclusion in SciPy. > > (Note: Very few codes come with a license explicitly allowing free > redistribution and inclusion in commercial software) > > L-BFGS-B: > The Fortran source can be downloaded here: > http://www.ece.northwestern.edu/~nocedal/lbfgsb.html > We wrote to Nocedal who did not to oppose inclusion in our software, > so maybe we could ask him for inclusion in SciPy ? > It was the simplest to use, iterations are fast, and it was quite robust. > Probably the first code I would try on any problem. > > M2QN1: > The Fortran source can be extracted from the scilab distribution. > See also: > http://www-rocq.inria.fr/estime/modulopt/optimization-routines/m2qn1.html > An LBGFS code like the previous one, reasonnably simple to use, very > fast, very efficient. > The use in commercial software if forbiden by scilab's licence, but we > may still ask the author, Claude Lemar?chal, who is a very nice person. > > TNBC: > http://iris.gmu.edu/~snash/nash/software/software.html > A bound constrained truncated newton code. > Stephen Nash did not oppose use in our code, and since this code > provided the best performance on our problem (which may not be the > case on other problem), I did a C version of it (with some > modifications) and recently added a Python interface. It's distributed > with a BSD license, so it could be readily integrated in SciPy (but > I'm obviously biased). You can download it here (look for TNC): > http://www.jeannot.org/~js/code/index.en.html > > A few more general codes: > > DONLP2: > http://plato.la.asu.edu/donlp2.html > SQP code with non linear constraints (including equality constraints). > While it was quite difficult to use in our problem, it works well, and > is very general. > The licence oppose commercial use, but we could ask the author, Peter > Spellucci about inclusion in SciPy. > > TRON: > http://www-unix.mcs.anl.gov/~more/tron/ > Trust region newton code. Fast iterations. Sparse Hessian estimation > using finite differences, so it may be more difficult to use (since > the sparsity pattern should be computed). > Again, Jorge More must be asked about inclusion. > > Other interesting codes include: > > IPOPT: > http://www-124.ibm.com/developerworks/opensource/coin/Ipopt/ > An interior point code, very general (non linear equality constraints, > bounds). > We did not review it (its quite recent). It comes with a free license, > so integration in SciPy should be possible. (Some options of IPOPT > requires part of the HSL library, which cannot be used in commercial > software). > > DFO: > http://www-124.ibm.com/developerworks/opensource/coin/faqs.html#DFO > A derivative free optimisation code, which, like IPOPT, comes with a free > license. But it requires a non-linear gradient based optimizer and > some developpement. > > COBYLA: > http://plato.la.asu.edu/topics/problems/nlores.html > Another alternative for derivative free optimisation, which seems free. > > There are probably other codes, better alternatives, which should be > considered for inclusion in SciPy. Any suggestions ? > > Regards, > > js > > _______________________________________________ > Scipy-dev mailing list > Scipy-dev at scipy.net > http://www.scipy.net/mailman/listinfo/scipy-dev From cookedm at physics.mcmaster.ca Fri Apr 9 16:48:59 2004 From: cookedm at physics.mcmaster.ca (David M. Cooke) Date: Fri, 9 Apr 2004 16:48:59 -0400 Subject: [SciPy-dev] Nonlinear constrained optimizer in SciPy In-Reply-To: <4076E624.2010201@jeannot.org> References: <4076E624.2010201@jeannot.org> Message-ID: <20040409204859.GA27887@arbutus.physics.mcmaster.ca> On Fri, Apr 09, 2004 at 08:06:28PM +0200, Jean-Sebastien Roy wrote: > L-BFGS-B: > The Fortran source can be downloaded here: > http://www.ece.northwestern.edu/~nocedal/lbfgsb.html > We wrote to Nocedal who did not to oppose inclusion in our software, so > maybe we could ask him for inclusion in SciPy ? > It was the simplest to use, iterations are fast, and it was quite robust. > Probably the first code I would try on any problem. I've written a python wrapper (using F2PY) that I've been using for a few months. I just rolled it together and put it on my website at http://arbutus.mcmaster.ca/dmc/software/ The thing to clarify with Nocedal is the 'advertising' clause in it's conditions of use -- this wouldn't be acceptable as a subroutine you might not realize you're using. -- |>|\/|< /--------------------------------------------------------------------------\ |David M. Cooke http://arbutus.physics.mcmaster.ca/dmc/ |cookedm at physics.mcmaster.ca From eric at enthought.com Fri Apr 9 17:19:37 2004 From: eric at enthought.com (eric jones) Date: Fri, 09 Apr 2004 16:19:37 -0500 Subject: [SciPy-dev] Nonlinear constrained optimizer in SciPy In-Reply-To: <20040409204859.GA27887@arbutus.physics.mcmaster.ca> References: <4076E624.2010201@jeannot.org> <20040409204859.GA27887@arbutus.physics.mcmaster.ca> Message-ID: <40771369.2060902@enthought.com> David M. Cooke wrote: >On Fri, Apr 09, 2004 at 08:06:28PM +0200, Jean-Sebastien Roy wrote: > > >>L-BFGS-B: >>The Fortran source can be downloaded here: >>http://www.ece.northwestern.edu/~nocedal/lbfgsb.html >>We wrote to Nocedal who did not to oppose inclusion in our software, so >>maybe we could ask him for inclusion in SciPy ? >>It was the simplest to use, iterations are fast, and it was quite robust. >>Probably the first code I would try on any problem. >> >> > >I've written a python wrapper (using F2PY) that I've been using for a >few months. I just rolled it together and put it on my website at > >http://arbutus.mcmaster.ca/dmc/software/ > >The thing to clarify with Nocedal is the 'advertising' clause in it's >conditions of use -- this wouldn't be acceptable as a subroutine you >might not realize you're using. > > > Could we just place this in the doc string? The doc-string could say something like "If you use this routine in your research or commercial application, then quote this paper." Would that be enough? eric From oliphant at ee.byu.edu Fri Apr 9 21:49:46 2004 From: oliphant at ee.byu.edu (Travis Oliphant) Date: Fri, 09 Apr 2004 20:49:46 -0500 Subject: [SciPy-dev] Nonlinear constrained optimizer in SciPy In-Reply-To: <40771369.2060902@enthought.com> References: <4076E624.2010201@jeannot.org> <20040409204859.GA27887@arbutus.physics.mcmaster.ca> <40771369.2060902@enthought.com> Message-ID: <407752BA.5050607@ee.byu.edu> eric jones wrote: > David M. Cooke wrote: Thanks to David Cooke and the very informative Jean-Sebastien Roy SciPy now has our first multivariable constrained optimization function Just in time for the Scipy 0.3 release. The Zhu, Byrd, Nocedal L-BFGS-B optimizer is named scipy.optimize.fmin_l_bfgs_b It uses the same interface David constructed. I removed dependency on linpack and blas and use instead the lapack returned from scipy_distutils. The docstring contains the information for the papers people should quote if they use the algorithm in their work. I notice that it is slower for unconstrained problems than the other algorithms which I suppose is not surprising. What have other people experienced? I just did a simple test on the rosenbrock function (notice that optimize.rosen and optimize.rosen_der provide the function and gradient for this famous function) both of these are vector-length agnostic so that the initial guess defines the order of the problem. I tested order 3-5 on windows and linux without problems. Thanks again, -Travis Oliphant From oliphant at ee.byu.edu Sat Apr 10 01:25:05 2004 From: oliphant at ee.byu.edu (Travis Oliphant) Date: Sat, 10 Apr 2004 00:25:05 -0500 Subject: [SciPy-dev] TNC constrained optimizer now in SciPy In-Reply-To: <40771369.2060902@enthought.com> References: <4076E624.2010201@jeannot.org> <20040409204859.GA27887@arbutus.physics.mcmaster.ca> <40771369.2060902@enthought.com> Message-ID: <40778531.9080400@ee.byu.edu> Thanks again to Jean-Sebastian for another constrained optimization code. I've added his TNC package to SciPy. I made a couple of changes to the interface to make it consistent with the rest of scipy.optimize. Other than that it's pretty much his code. This code even runs very quickly for the simple unconstrained cases I tried. So, it looks like now we have two decent constrained optimization codes in SciPy. Fantastic!! Play away. -Travis Oliphant From js at jeannot.org Sat Apr 10 06:17:59 2004 From: js at jeannot.org (Jean-Sebastien Roy) Date: Sat, 10 Apr 2004 12:17:59 +0200 Subject: [SciPy-dev] Nonlinear constrained optimizer in SciPy In-Reply-To: <4076FE22.5070204@enthought.com> References: <4076E624.2010201@jeannot.org> <4076FE22.5070204@enthought.com> Message-ID: <4077C9D7.7040500@jeannot.org> eric jones wrote: > Gathering permission from other authors could proceed in parallel. > Jean Sebastien, do you have time to do this? I'll ask Powell about inclusion of COBYLA (which has the great advantage of not requiring the gradient) in SciPy. Regards, js PS: many thanks to David M. Cooke for the LBFGSB interface ! From js at jeannot.org Mon Apr 12 07:07:09 2004 From: js at jeannot.org (Jean-Sebastien Roy) Date: Mon, 12 Apr 2004 13:07:09 +0200 Subject: [SciPy-dev] Nonlinear constrained optimizer in SciPy In-Reply-To: <4077C9D7.7040500@jeannot.org> References: <4076E624.2010201@jeannot.org> <4076FE22.5070204@enthought.com> <4077C9D7.7040500@jeannot.org> Message-ID: <407A785D.4050605@jeannot.org> Jean-Sebastien Roy wrote: > I'll ask Powell about inclusion of COBYLA (which has the great > advantage of not requiring the gradient) in SciPy. (Replying to myself) Michael J.D. Powell agreed to include COBYLA in SciPy (which is very nice of him), so what's left is writting a Python interface to it. I plan to do it in the next days/weeks. Regards, js From js at jeannot.org Tue Apr 13 12:57:38 2004 From: js at jeannot.org (Jean-Sebastien Roy) Date: Tue, 13 Apr 2004 18:57:38 +0200 Subject: [SciPy-dev] Python interface to COBYLA Message-ID: <407C1C02.7060306@jeannot.org> Hi ! Following my previous mail, I did a preliminary Python interface to M.J.D. Powell COBYLA optimization package. I translated the Fortran package to C via f2c, did a few adjustments, and added a Python interface so it could be easily integrated into SciPy. As stated in a previous mail, M.J.D. Powell agreed for the inclusion of COBYLA in SciPy. (Many thanks to him !) COBYLA stands for Constrained Optimization BY Linear Approximation. Two things make COBYLA an interessting algorithm to have in SciPy: - It's derivative free (like Nelder-Mead) ; - It can take non-linear constraints. It has some drawbacks: - It won't probably work if there are too many variables (say > 10) ; - The constraint are not exactly enforced during the minimization ; - It may be quite slow, most notably if there are no constraints. In the last case, a higher performance unconstrained derivative free method like Powell's UOBYQA or NEWUOA should probably be used. (Conn-Scheinberg-Toint DFO is anoter option) I've attached the preliminary Python interface. Suggestions are welcome! Regards, js -------------- next part -------------- A non-text attachment was scrubbed... Name: cobyla2c-1.0BETA.tgz Type: application/x-tar-gz Size: 20984 bytes Desc: not available URL: From pearu at scipy.org Tue Apr 13 13:18:44 2004 From: pearu at scipy.org (Pearu Peterson) Date: Tue, 13 Apr 2004 12:18:44 -0500 (CDT) Subject: [SciPy-dev] Python interface to COBYLA In-Reply-To: <407C1C02.7060306@jeannot.org> Message-ID: On Tue, 13 Apr 2004, Jean-Sebastien Roy wrote: > Hi ! > > Following my previous mail, I did a preliminary Python interface to > M.J.D. Powell COBYLA optimization package. > > I translated the Fortran package to C via f2c, did a few adjustments, > and added a Python interface so it could be easily integrated into SciPy. > As stated in a previous mail, M.J.D. Powell agreed for the inclusion of > COBYLA in SciPy. (Many thanks to him !) Great! Thank you for making it possible to include COBYLA to Scipy! > I've attached the preliminary Python interface. Suggestions are welcome! I have one suggestion: use F2PY to generate Python interfaces to Fortran programs. There are several advantages for doing so. For example, the interface is more efficient and less errorprone and easier to maintain than handwritten wrappers. Let me know if you have any troubles using F2PY. Thanks, Pearu From js at jeannot.org Tue Apr 13 16:04:28 2004 From: js at jeannot.org (Jean-Sebastien Roy) Date: Tue, 13 Apr 2004 22:04:28 +0200 Subject: [SciPy-dev] Python interface to COBYLA In-Reply-To: References: Message-ID: <407C47CC.1040106@jeannot.org> Pearu Peterson wrote: > I have one suggestion: use F2PY to generate Python interfaces to Fortran > programs. There are several advantages for doing so. For example, the > interface is more efficient and less errorprone and easier to maintain > than handwritten wrappers. Let me know if you have any troubles using > F2PY. You're absolutely right. I should have used F2PY. In fact, I tried F2PY first but encountered a few problems. The most basic one was that F2PY didn't work right on my machine (A FreeBSD 4.9 box. The F2PY port didn't recognize any F77 compiler. It's probably specific to my machine). The other problem was that COBYLA expect the function to be minimized to be a Fortran function called CALCFC. Therefore, reentrancy was not easy to achieve without some modifications to the original code (passing a callback at least). So in the end, I ran f2c, modified the code and wrote my own wrapper. A nice side effect is I can now use COBYLA in some of my C programs easily. But I agree it's clearly not the best solution for SciPy. I'll try harder to use F2PY next time (probably for Powell's NEWUOA). In the mean time, feel free to replace my wrapper by a F2PY wrapper to the original Fortran COBYLA2 code: I would be myself very interessted in such wrapper as an example ! Regards, js From oliphant at ee.byu.edu Tue Apr 13 16:22:27 2004 From: oliphant at ee.byu.edu (Travis Oliphant) Date: Tue, 13 Apr 2004 14:22:27 -0600 Subject: [SciPy-dev] Python interface to COBYLA In-Reply-To: References: Message-ID: <407C4C03.40301@ee.byu.edu> Pearu Peterson wrote: >On Tue, 13 Apr 2004, Jean-Sebastien Roy wrote: > > > >>Hi ! >> >>Following my previous mail, I did a preliminary Python interface to >>M.J.D. Powell COBYLA optimization package. >> >>I translated the Fortran package to C via f2c, did a few adjustments, >>and added a Python interface so it could be easily integrated into SciPy. >>As stated in a previous mail, M.J.D. Powell agreed for the inclusion of >>COBYLA in SciPy. (Many thanks to him !) >> >> Many thanks are owed to Jean-Sebastien for his assistance in this area. I would like to echo, however, Pearu's suggestion that there is no need to convert working Fortran code to C for inclusion into scipy. f2py actually makes the process of integrating fortran code into SciPy easier than integrating C code. So, unless there is an important reason to convert to C (dynamic memory allocation for example), I would think it is better to keep Fortran code in Fortran when possible. Thanks again, Jean, -Travis From as8ca at virginia.edu Tue Apr 13 18:27:20 2004 From: as8ca at virginia.edu (Alok Singhal) Date: Tue, 13 Apr 2004 18:27:20 -0400 Subject: [SciPy-dev] import scipy -> SIGSEGV? Message-ID: <20040413222720.GA17928@virginia.edu> Hi all, OK, I have spent two days trying to figure out what the problem migh be, but I give up. I installed scipy on a Redhat 9 system (as a user in my home directory), and when I do: $ python >>> import scipy It says: Segmentation fault The core dump shows SIGSEGV is received in fastumath_unsigned.inc, line 2858: add_functions[11] = PyUFunc_OO_O; The relevant backtrace is: #0 InitOperators (dictionary=0x81b75a4) at fastumath_unsigned.inc:2858 #1 0x40078f9f in initfastumath () at /users/asinghal/SciPy-0.2.0_alpha_200.4161/scipy_core/scipy_base/fastumathmodule.c:92 #2 0x080921ef in _PyImport_LoadDynamicModule () #3 0x08090003 in PyImport_ExecCodeModuleEx () (gdb) p add_functions[10] $2 = 0x4006f180 (gdb) p add_functions[11] $3 = 0 (gdb) p &add_functions[11] $4 = (PyUFuncGenericFunction *) 0x4007bbcc Line 2722 of scipy_core/scipy_base/fastumath_unsigned.inc defines add_functions to be a 12 element array, so an assignment to add_functions[11] should work. I used the SciPy-0.2.0_alpha_200.4161_src.zip code to compile scipy. I am using the following: - Python 2.2.2 (RPM python-2.2.2-26) - Numeric 23.1 - ATLAS 3.6.0 (+LAPACK) - f2py 2.39.235_1693 - fftw 3.0.1 (though it failed to detect fftw during installation of scipy) - gcc (GCC) 3.2.2 20030222 (Red Hat Linux 3.2.2-5) Importing Numeric works fine. I am not sure where the problem is, but if you need any other information, I will be glad to provide it. Alok -- Alok Singhal (as8ca at virginia.edu) * * Graduate Student, dept. of Astronomy * * * University of Virginia http://www.astro.virginia.edu/~as8ca/ * * From pearu at scipy.org Tue Apr 13 18:43:20 2004 From: pearu at scipy.org (Pearu Peterson) Date: Tue, 13 Apr 2004 17:43:20 -0500 (CDT) Subject: [SciPy-dev] import scipy -> SIGSEGV? In-Reply-To: <20040413222720.GA17928@virginia.edu> Message-ID: On Tue, 13 Apr 2004, Alok Singhal wrote: > Hi all, > > OK, I have spent two days trying to figure out what the problem migh > be, but I give up. I installed scipy on a Redhat 9 system (as a user > in my home directory), and when I do: > > $ python > >>> import scipy > > It says: > > Segmentation fault > > The core dump shows SIGSEGV is received in fastumath_unsigned.inc, > line 2858: > > add_functions[11] = PyUFunc_OO_O; > > The relevant backtrace is: > > #0 InitOperators (dictionary=0x81b75a4) at fastumath_unsigned.inc:2858 > #1 0x40078f9f in initfastumath () at /users/asinghal/SciPy-0.2.0_alpha_200.4161/scipy_core/scipy_base/fastumathmodule.c:92 > #2 0x080921ef in _PyImport_LoadDynamicModule () > #3 0x08090003 in PyImport_ExecCodeModuleEx () > > (gdb) p add_functions[10] > $2 = 0x4006f180 > (gdb) p add_functions[11] > $3 = 0 > (gdb) p &add_functions[11] > $4 = (PyUFuncGenericFunction *) 0x4007bbcc > > Line 2722 of scipy_core/scipy_base/fastumath_unsigned.inc defines > add_functions to be a 12 element array, so an assignment to > add_functions[11] should work. > > I used the SciPy-0.2.0_alpha_200.4161_src.zip code to compile scipy. > > I am using the following: > > - Python 2.2.2 (RPM python-2.2.2-26) > - Numeric 23.1 > - ATLAS 3.6.0 (+LAPACK) > - f2py 2.39.235_1693 > - fftw 3.0.1 (though it failed to detect fftw during installation of > scipy) > - gcc (GCC) 3.2.2 20030222 (Red Hat Linux 3.2.2-5) > > Importing Numeric works fine. I am not sure where the problem is, but > if you need any other information, I will be glad to provide it. Thanks for providing a detailed bug report. Above there are two issues: 1) Use Scipy from CVS, that probably solves the segfault problem. 2) Scipy does not support fftw 3 yet, use fftw 2.x instead. HTH, Pearu From pearu at scipy.org Thu Apr 15 01:10:10 2004 From: pearu at scipy.org (Pearu Peterson) Date: Thu, 15 Apr 2004 00:10:10 -0500 (CDT) Subject: [SciPy-dev] Python interface to COBYLA In-Reply-To: <407C47CC.1040106@jeannot.org> Message-ID: Jean, Travis, On Tue, 13 Apr 2004, Jean-Sebastien Roy wrote: > You're absolutely right. I should have used F2PY. > > In fact, I tried F2PY first but encountered a few problems. The most > basic one was that F2PY didn't work right on my machine (A FreeBSD 4.9 > box. The F2PY port didn't recognize any F77 compiler. It's probably > specific to my machine). Please, let us know what are these problems with FreeBSD. I am sure there is a easy fix. > In the mean time, feel free to replace my wrapper by a F2PY wrapper to > the original Fortran COBYLA2 code: I would be myself very interessted in > such wrapper as an example ! The example wrapper to Fortran COBYLA2 code is attached (feel free to suggest changes to the interface). Below follow basic steps how I created this wrapper, the steps are quite typical when creating a new scipy module (for more info, see http://testing.scipy.org/development/developscipy): 1) Modified cobyla2.f: added calcfc as an argument 2) Generated initial .pyf file: f2py cobyla2.f -m _cobyla -h _cobyla.pyf only: cobyla calcfc 3) Modified _cobyla.pyf so that f2py generates a wrapper having the following signature: x = minimize(calcfc,m,x,rhobeg,rhoend, [iprint,maxfun,calcfc_extra_args]) where call-back function calcfc is expected to have the following signature: def calcfc(x,con): # calculate scalar f=f(x) # calculate m-vector con(x) in-place return f 4) Created setup.py file 5) Created tests/test_cobyla.py 6) Build cobyla module: python setup.py build 7) Run tests: $ python tests/test_cobyla.py Found 1 tests for __main__ Result: [-4.80580817 1.44539502] (exact result = -4.898979456, 1) . ---------------------------------------------------------------------- Ran 1 test in 0.004s OK $ 8) Created tar-ball that is attached: python setup.py sdist Bugs: when setting maxfun value higher than 10, the fortran code seem end up into an infinite loop. Not sure what is going wrong. Note that by the end of this week (probably tomorrow) we have Scipy release 0.3 out and at the moment scipy CVS tree is in a freezed state because of that. I think we can add cobyla to Scipy 0.3.1 (the bug fix release) provided that the above bug gets resolved. Regards, Pearu -------------- next part -------------- A non-text attachment was scrubbed... Name: cobyla-0.1.tar.gz Type: application/x-gzip Size: 10993 bytes Desc: URL: From pearu at scipy.org Thu Apr 15 01:14:35 2004 From: pearu at scipy.org (Pearu Peterson) Date: Thu, 15 Apr 2004 00:14:35 -0500 (CDT) Subject: [SciPy-dev] Python interface to COBYLA In-Reply-To: Message-ID: There were few typos in my previous message. On Thu, 15 Apr 2004, Pearu Peterson wrote: > 2) Generated initial .pyf file: > > f2py cobyla2.f -m _cobyla -h _cobyla.pyf only: cobyla calcfc .. should read f2py cobyla2.f -m cobyla -h cobyla.pyf only: cobyla calcfc > 3) Modified _cobyla.pyf so that f2py generates a wrapper.. ^^^^^^^^^^^ - should be cobyla.pyf Pearu From loredo at astro.cornell.edu Thu Apr 15 11:38:33 2004 From: loredo at astro.cornell.edu (Tom Loredo) Date: Thu, 15 Apr 2004 11:38:33 -0400 (EDT) Subject: [SciPy-dev] Python interface to COBYLA Message-ID: <200404151538.i3FFcXO26130@laplace.astro.cornell.edu> > So, unless there is an important reason to convert to C (dynamic memory > llocation for example), I would think it is better to keep Fortran code > in Fortran when possible. One reason is that Mac OS 9 users don't have a free FORTRAN compiler to use. This has thwarted my use of scipy in the past. I've pretty much switched to OS X now. There one can get g77 via the Fink project. It's too bad it's not in the Apple Developer Tools (not sure about the latest version...) since it places one more obstacle in the path of Mac users wanting to use scipy. -Tom From js at jeannot.org Thu Apr 15 14:52:49 2004 From: js at jeannot.org (Jean-Sebastien Roy) Date: Thu, 15 Apr 2004 20:52:49 +0200 Subject: [SciPy-dev] Python interface to COBYLA In-Reply-To: References: Message-ID: <407EDA01.1040500@jeannot.org> Pearu Peterson wrote: > Please, let us know what are these problems with FreeBSD. I am sure there > is a easy fix. I had installed the version in the FreeBSD ports ("2.37.233.1545"). Running: python setup.py build_ext --inplace Returned: running build_ext error: Fortran compiler not available: None I have now installed the latest version (no port is available for it on FreeBSD), an it worked fine besides apparently harmful errors such as: Could not locate executable /usr/home/js/devel/optimisation/cobyla2_f/cobyla-0.1/g77 or: invalid command 'build_src' (no module named 'distutils.command.build_src') (on a side note, is it possible to get f2py messages in black instead of yellow (the color I'm seeing on my machine) which is very difficult to read on white background ?) > The example wrapper to Fortran COBYLA2 code is attached (feel free to > suggest changes to the interface). Many thanks ! It's obviously much simpler than writting a custom C wrapper. About changes to the interface, only a few things may be missing : maxfun is also a value to be returned (it's the number of iterations effectively done). Also, the reason for exiting COBYLA should be returned, as else optimization may fail without warning the caller (beside a printed message). This may require modifications to the Fortran source. Is it normal that the Fortran Wrapper does not complain when None is returned by the python function instead of a float ? > Bugs: when setting maxfun value higher than 10, the fortran code seem end > up into an infinite loop. Not sure what is going wrong. FWIW, I cannot reproduce this bug, neither in my nor in your wrapper, which seems to run fine on my machine. Nevertheless, when trying a nested optimization problem, I encountered an error I cannot explain. Please find below a simple example: import cobyla def f1(x, con): global a f = (x[0]-a)**2 return f def f2(x, con): global a a = x[0] z = cobyla.minimize(f1, m=0, x=[0], iprint = 1,rhobeg=0.5,rhoend=1e-6,maxfun=100) f = z[0]**2+abs(x[1])**3 return f print cobyla.minimize(f2, m=0, x=[1,1], iprint = 1,rhobeg=0.5,rhoend=1e-4,maxfun=1000) Returns: Normal return from subroutine COBYLA NFVALS = 23 F = 1.000000E-12 MAXCV = 0.000000E+00 X = 1.000001E+00 capi_return is NULL Call-back cb_calcfc_in_cobyla__user__routines failed. Traceback (most recent call last): File "tst5.py", line 18, in ? print cobyla.minimize(f2, m=0, x=[1,1], iprint = 1,rhobeg=0.5,rhoend=1e-4,maxfun=1000) File "tst5.py", line 10, in f2 z = cobyla.minimize(f1, m=0, x=[0], iprint = 1,rhobeg=0.5,rhoend=1e-6,maxfun=100) TypeError: minimize argument calcfc must be callable This doesn't seem to happen with the C wrapper. Regards, js From jswhit at fastmail.fm Sat Apr 17 08:32:10 2004 From: jswhit at fastmail.fm (Jeff Whitaker) Date: Sat, 17 Apr 2004 06:32:10 -0600 (MDT) Subject: [SciPy-dev] SPHEREPACK module Message-ID: Hi: I've written a python wrapper for NCAR's SPHEREPACK library for spherical harmonic computations, using f2py. The module is available at ftp://ftp.cdc.noaa.gov/Public/jsw/spharm.tgz, with documentation online at ftp://ftp.cdc.noaa.gov/Public/jsw/spharm/doc/html/index.html. I'd be happy if it could be included in SciPy, but I'm afraid the SPHEREPACK license might prevent that. The SPHEREPACK source code is freely available, but the license appears to forbid redistributing it (http://www.scd.ucar.edu/css/software/spherepack/sp.form.html). f2py never ceases to amaze me - thanks Pearu! -Jeff -- Jeffrey S. Whitaker Phone : (303)497-6313 NOAA/OAR/CDC R/CDC1 FAX : (303)497-6449 325 Broadway Web : http://www.cdc.noaa.gov/~jsw Boulder, CO, USA 80305-3328 Office: Skaggs Research Cntr 1D-124 From travis at enthought.com Sat Apr 17 10:12:38 2004 From: travis at enthought.com (Travis N. Vaught) Date: Sat, 17 Apr 2004 09:12:38 -0500 Subject: [SciPy-dev] Re: [Scipy-cvs] world/scipy/Lib/optimize setup_optimize.py, 1.9, 1.10 In-Reply-To: <20040416235902.543A43EB48@www.scipy.com> References: <20040416235902.543A43EB48@www.scipy.com> Message-ID: <40813B56.7030707@enthought.com> travo at scipy.org wrote: >Update of /home/cvsroot/world/scipy/Lib/optimize >In directory scipy.org:/tmp/cvs-serv11844 > >Modified Files: > setup_optimize.py >Log Message: >Fixed problem with moduleTNC and libfgsb.so not going to scipy > > > Does this need to go into 0.3 (i.e. rebuild the binaries to include it)? It seems our 'tagging' is a bit shaky at this point. Travis From oliphant at ee.byu.edu Sat Apr 17 18:48:58 2004 From: oliphant at ee.byu.edu (Travis Oliphant) Date: Sat, 17 Apr 2004 16:48:58 -0600 Subject: [SciPy-dev] Re: [Scipy-cvs] world/scipy/Lib/optimize setup_optimize.py, 1.9, 1.10 In-Reply-To: <40813B56.7030707@enthought.com> References: <20040416235902.543A43EB48@www.scipy.com> <40813B56.7030707@enthought.com> Message-ID: <4081B45A.6060105@ee.byu.edu> Travis N. Vaught wrote: > travo at scipy.org wrote: > >> Update of /home/cvsroot/world/scipy/Lib/optimize >> In directory scipy.org:/tmp/cvs-serv11844 >> >> Modified Files: >> setup_optimize.py Log Message: >> Fixed problem with moduleTNC and libfgsb.so not going to scipy >> >> >> > > > Does this need to go into 0.3 (i.e. rebuild the binaries to include it)? > > It seems our 'tagging' is a bit shaky at this point. > > Travis > It's not critical. Some shared libraries were just installed into the site-packages directory instead of underneath in scipy. Shouldn't affect any behavior. -Travis O. > _______________________________________________ > Scipy-dev mailing list > Scipy-dev at scipy.net > http://www.scipy.net/mailman/listinfo/scipy-dev From pearu at scipy.org Mon Apr 19 06:44:22 2004 From: pearu at scipy.org (Pearu Peterson) Date: Mon, 19 Apr 2004 05:44:22 -0500 (CDT) Subject: [SciPy-dev] releasing scipy.. In-Reply-To: <4081B45A.6060105@ee.byu.edu> Message-ID: On Sat, 17 Apr 2004, Travis Oliphant wrote: > > Does this need to go into 0.3 (i.e. rebuild the binaries to include it)? > > > > It seems our 'tagging' is a bit shaky at this point. > > > > Travis > > > It's not critical. Some shared libraries were just installed into the > site-packages directory instead of underneath in scipy. Shouldn't > affect any behavior. > > -Travis O. I agree that there should not be any affect on the behaviour, though it introduces mismatch in tagged CVS repository and released binary and source files. I just hope that nobody will notice that when using scipy and upgrading scipy in future (the ../site-packages/moduleTNC.so will be there foreever..). Anyway, I just tagged the CVS repository with tag v0_3_0. I have fixed several links in the new scipy.org (replaced .html->.txt where appropiate) and it looks like we are ready to send the release announcement out. Any takers for composing the announcement? Few issues: 1) Need contents for http://www.scipy.org/development/maintainers.html 2) Top in scipy.org shows: PID USER PRI NI SIZE RSS SHARE STAT %CPU %MEM TIME CPU COMMAND 25930 zope 15 0 814M 218M 1664 S 0.0 34.7 66:42 0 python2.3 that is zope is taking lots of memory. Could this explain why the site is so slow? Maybe restarting zope would free some memory? Regards, Pearu From travis at enthought.com Mon Apr 19 08:48:27 2004 From: travis at enthought.com (Travis N. Vaught) Date: Mon, 19 Apr 2004 07:48:27 -0500 Subject: [SciPy-dev] releasing scipy.. In-Reply-To: References: Message-ID: <4083CA9B.40405@enthought.com> Pearu Peterson wrote: >On Sat, 17 Apr 2004, Travis Oliphant wrote: > > > >>>Does this need to go into 0.3 (i.e. rebuild the binaries to include it)? >>> >>>It seems our 'tagging' is a bit shaky at this point. >>> >>>Travis >>> >>> >>> >>It's not critical. Some shared libraries were just installed into the >>site-packages directory instead of underneath in scipy. Shouldn't >>affect any behavior. >> >>-Travis O. >> >> > >I agree that there should not be any affect on the behaviour, though it >introduces mismatch in tagged CVS repository and released binary and >source files. I just hope that nobody will notice that when using scipy >and upgrading scipy in future (the ../site-packages/moduleTNC.so will be >there foreever..). > > > I'll go ahead and checkout against the tag and rebuild the windows binaries--that's a push-button operation. Joe, what's the level of effort for the RPM's? >Anyway, I just tagged the CVS repository with tag v0_3_0. I have fixed >several links in the new scipy.org (replaced .html->.txt where appropiate) >and it looks like we are ready to send the release announcement out. Any >takers for composing the announcement? > > > I'll give it a shot and possibly run it by you guys for review. >Few issues: >1) Need contents for > http://www.scipy.org/development/maintainers.html >2) Top in scipy.org shows: > > PID USER PRI NI SIZE RSS SHARE STAT %CPU %MEM TIME CPU COMMAND > 25930 zope 15 0 814M 218M 1664 S 0.0 34.7 66:42 0 python2.3 > >that is zope is taking lots of memory. Could this explain why the site is >so slow? Maybe restarting zope would free some memory? > > > Joe, we should make sure we're not in debug mode--not sure what else could be causing it...I did a recent restart of the zope process when I added the LocalFS tool. As long as we're not in swap, the memory issue shouldn't kill performance, should it? >Regards, >Pearu > >_______________________________________________ >Scipy-dev mailing list >Scipy-dev at scipy.net >http://www.scipy.net/mailman/listinfo/scipy-dev > > From joe at enthought.com Mon Apr 19 19:34:36 2004 From: joe at enthought.com (Joe Cooper) Date: Mon, 19 Apr 2004 18:34:36 -0500 Subject: [SciPy-dev] releasing scipy.. In-Reply-To: <4083CA9B.40405@enthought.com> References: <4083CA9B.40405@enthought.com> Message-ID: <4084620C.4040807@enthought.com> Travis N. Vaught wrote: > I'll go ahead and checkout against the tag and rebuild the windows > binaries--that's a push-button operation. Joe, what's the level of > effort for the RPM's? Relatively low. I'm rebuilding for Fedora Core 1 ASAP, anyway, so rerolling the RH9 RPMs is little trouble. > Joe, we should make sure we're not in debug mode--not sure what else > could be causing it...I did a recent restart of the zope process when I > added the LocalFS tool. As long as we're not in swap, the memory issue > shouldn't kill performance, should it? I don't think it is on by default, but I'm restarting Zope with debug-mode turned off explicitly. We'll see if it brings the size down a bit. From joe at enthought.com Mon Apr 19 22:02:44 2004 From: joe at enthought.com (Joe Cooper) Date: Mon, 19 Apr 2004 21:02:44 -0500 Subject: [SciPy-dev] releasing scipy.. In-Reply-To: <4084620C.4040807@enthought.com> References: <4083CA9B.40405@enthought.com> <4084620C.4040807@enthought.com> Message-ID: <408484C4.40406@enthought.com> Joe Cooper wrote: > Travis N. Vaught wrote: >> Joe, we should make sure we're not in debug mode--not sure what else >> could be causing it...I did a recent restart of the zope process when >> I added the LocalFS tool. As long as we're not in swap, the memory >> issue shouldn't kill performance, should it? > > > I don't think it is on by default, but I'm restarting Zope with > debug-mode turned off explicitly. We'll see if it brings the size down > a bit. Looks like that might have done it. It's still growing two hours after a restart, so I might be speaking too soon, but it's only ~90MB right now. From travis at enthought.com Tue Apr 20 06:38:52 2004 From: travis at enthought.com (Travis N. Vaught) Date: Tue, 20 Apr 2004 05:38:52 -0500 Subject: [SciPy-dev] Re: [Scipy-cvs] world/scipy/Lib/optimize cobyla.py, NONE, 1.1 __init__.py, 1.23, 1.24 info_optimize.py, 1.4, 1.5 setup_optimize.py, 1.10, 1.11 In-Reply-To: <20040419213035.231C13EB49@www.scipy.com> References: <20040419213035.231C13EB49@www.scipy.com> Message-ID: <4084FDBC.8070509@enthought.com> travo at scipy.org wrote: >Update of /home/cvsroot/world/scipy/Lib/optimize >In directory scipy.org:/tmp/cvs-serv3551 > >Modified Files: > __init__.py info_optimize.py setup_optimize.py >Added Files: > cobyla.py >Log Message: >Added cobyla interface to optimize. > > >--- NEW FILE: cobyla.py --- ># Interface to Constrained Optimization By Linear Approximation > > > Hey Travis, Are we missing a file that you still need to checkin? The nightly build for windows gives: building extension "scipy.optimize._cobyla" sources Traceback (most recent call last): File "setup.py", line 110, in ? setup_package(ignore_packages) File "setup.py", line 92, in setup_package url = "http://www.scipy.org", File "scipy_core\scipy_distutils\core.py", line 72, in setup return old_setup(**new_attr) File "c:\Python21\lib\distutils\core.py", line 138, in setup dist.run_commands() File "c:\Python21\lib\distutils\dist.py", line 899, in run_commands self.run_command(cmd) File "c:\Python21\lib\distutils\dist.py", line 919, in run_command cmd_obj.run() File "c:\Python21\lib\distutils\command\build.py", line 106, in run self.run_command(cmd_name) File "c:\Python21\lib\distutils\cmd.py", line 328, in run_command self.distribution.run_command(command) File "c:\Python21\lib\distutils\dist.py", line 919, in run_command cmd_obj.run() File "scipy_core\scipy_distutils\command\build_src.py", line 81, in run self.build_sources() File "scipy_core\scipy_distutils\command\build_src.py", line 88, in build_sources self.build_extension_sources(ext) File "scipy_core\scipy_distutils\command\build_src.py", line 126, in build_extension_sources sources = self.f2py_sources(sources, ext) File "scipy_core\scipy_distutils\command\build_src.py", line 213, in f2py_sources assert os.path.isfile(target_file),`target_file`+' missing' AssertionError: 'Lib\\optimize\\cobyla\\_cobylamodule.c' missing The nightly build logs are available here: http://www.scipy.org/development/scoreboard.html From pearu at cens.ioc.ee Tue Apr 20 08:11:02 2004 From: pearu at cens.ioc.ee (Pearu Peterson) Date: Tue, 20 Apr 2004 15:11:02 +0300 (EEST) Subject: [SciPy-dev] Re: [Scipy-cvs] world/scipy/Lib/optimize cobyla.py, setup_optimize.py, 1.10, 1.11 In-Reply-To: <4084FDBC.8070509@enthought.com> Message-ID: On Tue, 20 Apr 2004, Travis N. Vaught wrote: > travo at scipy.org wrote: > > >Log Message: > >Added cobyla interface to optimize. > > Are we missing a file that you still need to checkin? The nightly build > for windows gives: > > building extension "scipy.optimize._cobyla" sources > assert os.path.isfile(target_file),`target_file`+' missing' > AssertionError: 'Lib\\optimize\\cobyla\\_cobylamodule.c' missing > > The nightly build logs are available here: > > http://www.scipy.org/development/scoreboard.html I just tried building scipy from CVS both on Linux and Windows XP boxes, they worked ok. Note that _cobylamodule.c is generated by f2py and it should end up in build\src\optimize\cobyla directory. Maybe you have old _cobylamodule.c in your source tree that confused scipy_distutils somehow? Try fresh cvs checkout. Pearu From travis at enthought.com Wed Apr 21 10:18:22 2004 From: travis at enthought.com (Travis N. Vaught) Date: Wed, 21 Apr 2004 09:18:22 -0500 Subject: [SciPy-dev] Re: [Scipy-cvs] world/scipy_core/scipy_distutils/command build_src.py, 1.6, 1.7 In-Reply-To: <20040421142125.5C8613EB4C@www.scipy.com> References: <20040421142125.5C8613EB4C@www.scipy.com> Message-ID: <408682AE.7060102@enthought.com> pearu at scipy.org wrote: >Update of /home/cvsroot/world/scipy_core/scipy_distutils/command >In directory scipy.org:/tmp/cvs-serv1647/scipy_distutils/command > >Modified Files: > build_src.py >Log Message: >Fixed bug: target from a source file having absolute path did not end up inside build directory. Should also fix failing build in autotest. > > > Thanks Pearu, I'll run autotest now... From travis at enthought.com Wed Apr 21 17:40:24 2004 From: travis at enthought.com (Travis N. Vaught) Date: Wed, 21 Apr 2004 16:40:24 -0500 Subject: [SciPy-dev] ANN: SciPy 0.3 Released Message-ID: <4086EA48.4080105@enthought.com> Greetings, SciPy 0.3 has been released and binaries are available from the scipy.org site. http://www.scipy.org Changes since the 0.1 version (0.1 enjoyed a wide release, there was a version 0.2 that had limited exposure) include the following: - general improvements: Added utility functions for constructing arrays by concatenation, intended mainly for command-line use. Added bmat constructor for easy creation of block matrices. Added mat constructor for constructing matrices (where * is matrix multiplication). Added many PIL utility functions so that if the PIL is installed, image loading, saving, and other operations are available. Added scipy.info, which handles dynamic docstrings and class help better than python's help command. - documentation: much improved - sparse: superLU upgraded to 3.0 - optimize: Added non-linear conjugate gradient algorithm. Improvements to BFGS algorithm and Wolfe-condition line-search. Added two CONSTRAINED optimization techniques. Added simulated annealing and brute-force optimization strategies and Powell's method. Added many very good 1-D root-finding routines. - stats: Added many statistical distributions. Many continuous and discrete random variables are available with a simple mechanism for adding new ones. Added several statistical tests. - special: Added MANY new special functions. |general_function| renamed to |vectorize| and moved to scipy_base. Improved the way domain errors are handled (user specified display of problems). More tests on special functions added. - fftpack: replaced with fftpack2--can optionally be configured to support djbfft - io: Reading of MATLAB .mat files (version 4 and 5) supported. Writing version 4 .mat files supported. Added very flexible and general_purpose text file reader. Added shelving save operation to save variables into an importable module. Routines for reading and writing fortran-records. - linalg: Linear algebra is greatly improved over SciPy 0.1. ATLAS is now optional (but encouraged). Most lapack functions have simplified interfaces (all lapack and blas available). Matrix exponentials and other matrix functions added. - signal: Added support for filter design and LTI system analysis. - xplt: Updated xplt to use newer pygist so that Windows platform is supported. Addition of several plotting types. - integrate: added another ODE integrator. Added romberg integration and several other integration approaches. The complete release notes can be found here: http://www.scipy.org/download/scipy_release_notes_0.3.html You'll also notice that scipy.org is now a spanking new Plone portal (http://www.plone.org -- keep up the good work, plone folks). This is the first binary release in a long time and we hope to increase the frequency to every 6 months or so. If you'd like to follow or join the community, you can subscribe to the mailing lists here: http://www.scipy.org/mailinglists/ Best Regards, Travis BTW SciPy is: ------------- SciPy is an open source library of scientific tools for Python. SciPy supplements the popular Numeric module, gathering a variety of high level science and engineering modules together as a single package. SciPy includes modules for graphics and plotting, optimization, integration, special functions, signal and image processing, genetic algorithms, ODE solvers, and others. From prabhu at aero.iitm.ernet.in Tue Apr 27 14:59:47 2004 From: prabhu at aero.iitm.ernet.in (Prabhu Ramachandran) Date: Wed, 28 Apr 2004 00:29:47 +0530 Subject: [SciPy-dev] Correcting documentation on the site? Message-ID: <16526.44451.266462.320287@monster.linux.in> Hi, For a while, I've been meaning to edit the "Performance Python with Weave" article and make corrections/modifications. http://www.scipy.org/documentation/weave/weaveperformance.html I haven't had the time to do it so far. John Hunter just brought to my notice that these links were broken: http://www.scipy.org/documentation/weave/laplace.html http://www.scipy.org/documentation/weave/laplace.tgz I was wondering if it would be possible for members to edit the pages (or atleast some of the pages) and fix them? Ideally this is the job for a Wiki. However a lot of the content on the site is in the form of regular non-editable articles and it is not possible for a member to modify the article at all. Its probably a lot easier if members could be allowed to edit content. Alternatively it would be nice if some of the members could be allowed to edit certain sections. Something like a group of users who can add/modify content if they belong to the group. The user can email the admin to get privileges and the admin can effectively eliminate the possibility of any vandalism by removing any offending users from the group. Is this possible with Plone? Can something be done about this or is there an alternative? Thanks! regards, prabhu From pearu at scipy.org Tue Apr 27 15:56:51 2004 From: pearu at scipy.org (Pearu Peterson) Date: Tue, 27 Apr 2004 14:56:51 -0500 (CDT) Subject: [SciPy-dev] Correcting documentation on the site? In-Reply-To: <16526.44451.266462.320287@monster.linux.in> Message-ID: Hi Prabhu, On Wed, 28 Apr 2004, Prabhu Ramachandran wrote: > Hi, > > For a while, I've been meaning to edit the "Performance Python with > Weave" article and make corrections/modifications. > > http://www.scipy.org/documentation/weave/weaveperformance.html I have added you to the list of managers (not sure if this is the correct role..) of this page. Try again to edit document. > I was wondering if it would be possible for members to edit the pages > (or atleast some of the pages) and fix them? Ideally this is the job > for a Wiki. However a lot of the content on the site is in the form > of regular non-editable articles and it is not possible for a member > to modify the article at all. Its probably a lot easier if members > could be allowed to edit content. Alternatively it would be nice if > some of the members could be allowed to edit certain sections. > Something like a group of users who can add/modify content if they > belong to the group. The user can email the admin to get privileges > and the admin can effectively eliminate the possibility of any > vandalism by removing any offending users from the group. Is this > possible with Plone? I guess it is if the above works. Pearu From joe at enthought.com Tue Apr 27 15:51:45 2004 From: joe at enthought.com (Joe Cooper) Date: Tue, 27 Apr 2004 14:51:45 -0500 Subject: [SciPy-dev] Correcting documentation on the site? In-Reply-To: References: Message-ID: <408EB9D1.8030308@enthought.com> Pearu Peterson wrote: > Hi Prabhu, > > On Wed, 28 Apr 2004, Prabhu Ramachandran wrote: > > >>Hi, >> >>For a while, I've been meaning to edit the "Performance Python with >>Weave" article and make corrections/modifications. >> >> http://www.scipy.org/documentation/weave/weaveperformance.html > > > I have added you to the list of managers (not sure if this is the correct > role..) of this page. Try again to edit document. I think we're all still trying to figure out how to give members the rights they need to make improvements to the site... One of these days we'll probably get the Plone workflow model figured out. Prabhu, if you have trouble doing what you need to do, let me know, and I'll see if I can get you the permissions you need. From pearu at scipy.org Tue Apr 27 16:06:46 2004 From: pearu at scipy.org (Pearu Peterson) Date: Tue, 27 Apr 2004 15:06:46 -0500 (CDT) Subject: [SciPy-dev] Correcting documentation on the site? In-Reply-To: <16526.44451.266462.320287@monster.linux.in> Message-ID: On Wed, 28 Apr 2004, Prabhu Ramachandran wrote: > Can something be done about this or is there an alternative? Actually I like the way how a similar issue was resolved for "Building SciPy on Mac OSX" document. That is, the non-editable area contains a link to a document in members scipy homepage. As a result, the authors can freely edit their contributions (I'll presume that scipy managers can also edit members documentations, e.g. fixing typos etc) without asking permissions for every small change.. Pearu From joe at enthought.com Tue Apr 27 16:19:20 2004 From: joe at enthought.com (Joe Cooper) Date: Tue, 27 Apr 2004 15:19:20 -0500 Subject: [SciPy-dev] Correcting documentation on the site? In-Reply-To: References: Message-ID: <408EC048.9040608@enthought.com> Pearu Peterson wrote: > > On Wed, 28 Apr 2004, Prabhu Ramachandran wrote: > > >>Can something be done about this or is there an alternative? > > > Actually I like the way how a similar issue was resolved for "Building > SciPy on Mac OSX" document. That is, the non-editable area contains a link > to a document in members scipy homepage. As a result, the authors can > freely edit their contributions (I'll presume that scipy managers can also > edit members documentations, e.g. fixing typos etc) without asking > permissions for every small change.. Yes...and I think that's probably the method the Plone folks were invisioning. Though it does require a bit of work to make sure the new documents get linked to from the main pages, where they will be found by users. From swisher at enthought.com Tue Apr 27 18:40:58 2004 From: swisher at enthought.com (Janet Swisher) Date: Tue, 27 Apr 2004 17:40:58 -0500 Subject: [SciPy-dev] Correcting documentation on the site? In-Reply-To: Message-ID: <004101c42ca8$b8d2d810$ab01a8c0@SWISHER> > Pearu Peterson wrote: > > On Wed, 28 Apr 2004, Prabhu Ramachandran wrote: > > > Can something be done about this or is there an alternative? > > Actually I like the way how a similar issue was resolved for > "Building > SciPy on Mac OSX" document. That is, the non-editable area > contains a link > to a document in members scipy homepage. As a result, the authors can > freely edit their contributions (I'll presume that scipy > managers can also > edit members documentations, e.g. fixing typos etc) without asking > permissions for every small change.. The only problem was that the author of the Mac OSX document didn't have edit access to the document once it was published. We need to find a way to let authors fully control their documents, even after publication. If we can make that change, I think this scheme will work fine. It might eventually get to be a headache to have documents spread all across member's pages, but "too many contributions" would be a good problem to have. Prabhu's document was a special case because it was migrated from the old site, so he wasn't the person who created it on the new site. (And in the migration rush, some pieces got left behind -- sorry about that.) That won't be an issue for new documents going forward. --Janet From prabhu at aero.iitm.ernet.in Wed Apr 28 00:18:21 2004 From: prabhu at aero.iitm.ernet.in (Prabhu Ramachandran) Date: Wed, 28 Apr 2004 09:48:21 +0530 Subject: [SciPy-dev] Correcting documentation on the site? In-Reply-To: <408EB9D1.8030308@enthought.com> References: <408EB9D1.8030308@enthought.com> Message-ID: <16527.12429.117511.671450@monster.linux.in> >>>>> "JC" == Joe Cooper writes: JC> Pearu Peterson wrote: >> On Wed, 28 Apr 2004, Prabhu Ramachandran wrote: >>> For a while, I've been meaning to edit the "Performance Python >>> with Weave" article and make corrections/modifications. >>> >>> http://www.scipy.org/documentation/weave/weaveperformance.html >> >> >> I have added you to the list of managers (not sure if this is >> the correct role..) of this page. Try again to edit document. JC> I think we're all still trying to figure out how to give JC> members the rights they need to make improvements to the JC> site... One of these days we'll probably get the Plone JC> workflow model figured out. Prabhu, if you have trouble doing JC> what you need to do, let me know, and I'll see if I can get JC> you the permissions you need. Great! Thanks everyone for the help. I am able to edit the page. For now I've added new versions of the sources and updated the links on the page. I've tested the examples on my version of weave and it works fine. The only trouble is that weave has changed a bit since I wrote the article and some of the syntax is wrong currently. Editing that is going to be a pain since right now its all HTML. I'll need to re-write it as structured text. I'll get to that when I have some time. For now the links to the examples should work and the source code itself will work correctly with the latest weave. Eric, the Py::new_reference_to(...) is no longer valid. The interface for handling non-blitzed Numeric arrays has also changed. The existing weave docs use it and so do the current examples. I believe this has all changed quite a bit. Any chance you can update the weave docs? If that is not possible can you atleast send in a small update of what has changed and where. I could try to update the docs next month (after submission of the PhD). I also need to mention that VTK support is already there. I have a few questions on that but I'll ask them later. cheers, prabhu From eric at enthought.com Wed Apr 28 01:47:54 2004 From: eric at enthought.com (eric jones) Date: Wed, 28 Apr 2004 00:47:54 -0500 Subject: [SciPy-dev] Correcting documentation on the site? In-Reply-To: <16527.12429.117511.671450@monster.linux.in> References: <408EB9D1.8030308@enthought.com> <16527.12429.117511.671450@monster.linux.in> Message-ID: <408F458A.4090303@enthought.com> Prabhu Ramachandran wrote: >>>>>>"JC" == Joe Cooper writes: >>>>>> >>>>>> > > JC> Pearu Peterson wrote: > >> On Wed, 28 Apr 2004, Prabhu Ramachandran wrote: > >>> For a while, I've been meaning to edit the "Performance Python > >>> with Weave" article and make corrections/modifications. > >>> > >>> http://www.scipy.org/documentation/weave/weaveperformance.html > >> > >> > >> I have added you to the list of managers (not sure if this is > >> the correct role..) of this page. Try again to edit document. > > JC> I think we're all still trying to figure out how to give > JC> members the rights they need to make improvements to the > JC> site... One of these days we'll probably get the Plone > JC> workflow model figured out. Prabhu, if you have trouble doing > JC> what you need to do, let me know, and I'll see if I can get > JC> you the permissions you need. > >Great! Thanks everyone for the help. I am able to edit the page. >For now I've added new versions of the sources and updated the links >on the page. I've tested the examples on my version of weave and it >works fine. > >The only trouble is that weave has changed a bit since I wrote the >article and some of the syntax is wrong currently. Editing that is >going to be a pain since right now its all HTML. I'll need to >re-write it as structured text. I'll get to that when I have some >time. For now the links to the examples should work and the source >code itself will work correctly with the latest weave. > >Eric, the Py::new_reference_to(...) is no longer valid. The interface >for handling non-blitzed Numeric arrays has also changed. The >existing weave docs use it and so do the current examples. I believe >this has all changed quite a bit. Any chance you can update the weave >docs? If that is not possible can you atleast send in a small update >of what has changed and where. I could try to update the docs next >month (after submission of the PhD). I also need to mention that VTK >support is already there. I have a few questions on that but I'll ask >them later. > > > I can't get to this right now. But, we will get it cleaned up as soon as possible. Thanks for the pointers on things that need to be fixed. eric >cheers, >prabhu > >_______________________________________________ >Scipy-dev mailing list >Scipy-dev at scipy.net >http://www.scipy.net/mailman/listinfo/scipy-dev > > From prabhu at aero.iitm.ernet.in Wed Apr 28 14:42:19 2004 From: prabhu at aero.iitm.ernet.in (Prabhu Ramachandran) Date: Thu, 29 Apr 2004 00:12:19 +0530 Subject: [SciPy-dev] Correcting documentation on the site? In-Reply-To: <408F458A.4090303@enthought.com> References: <408EB9D1.8030308@enthought.com> <16527.12429.117511.671450@monster.linux.in> <408F458A.4090303@enthought.com> Message-ID: <16527.64267.511412.353214@monster.linux.in> >>>>> "EJ" == eric jones writes: EJ> Prabhu Ramachandran wrote: >> Eric, the Py::new_reference_to(...) is no longer valid. The >> interface for handling non-blitzed Numeric arrays has also >> changed. The existing weave docs use it and so do the current >> examples. I believe this has all changed quite a bit. Any >> chance you can update the weave docs? If that is not possible [...] EJ> I can't get to this right now. But, we will get it cleaned up EJ> as soon as possible. Thanks for the pointers on things that EJ> need to be fixed. Here is a slightly longer list of things in that case. I'm sure you know better than me but just in case you forget here is a list. :) 0. These links in the TOC are broken: http://www.scipy.org/documentation/weave/weaveusersguide.html#Passing_Variables http://www.scipy.org/documentation/weave/weaveusersguide.html#inline_file_argument_conversion http://www.scipy.org/documentation/weave/weaveusersguide.html#inline_callable_argument_conversion http://www.scipy.org/documentation/weave/weaveusersguide.html#Customizing Conversions 1. The Py:: namespace is gone and anything related to that needs to be updated appropriately. From what I can tell SCXX is used by default now which changes the docs all over the place. 2. Py::new_reference_to is also gone and its not clear what one should do to return a float/int or even a user defined type. 3. Using non-blitz arrays is not documented. From what I can see this is what the user needs to know: # Python a = Numeric.ones(10, 'd') # Inlined C int* Na = a_array->dimensions; // a.shape int* Sa = a_array->strides; // strides int Da = a_array->nd; // number of dimensions double* a (double*) u_array->data; // the actual data. 4. The type_factories keyword arg to inline is gone it is now called type_converters. The keyword list probably needs to be checked as well. The section on "Customizing Conversions" mentions that the default converters are in ext_tools.py. They are actually in converters.py. 5. VTK and swig2 support need to be documented. I would not mind writing these sections when I get some time. I'll let you know when this does happen. 6. Examples and possibly tests need to be updated. cheers, prabhu From prabhu at aero.iitm.ernet.in Wed Apr 28 15:01:07 2004 From: prabhu at aero.iitm.ernet.in (Prabhu Ramachandran) Date: Thu, 29 Apr 2004 00:31:07 +0530 Subject: [SciPy-dev] distutils and persistent system information Message-ID: <16527.65395.559716.2269@monster.linux.in> Hi, I have a question on how do get something optimally working for SciPy. Very specifically, it concerns weave's wxPython/wxWidgets capability and VTK Python<->C++ capabilities. The issues: 1. Both wxPython and VTK (and any other converters that will be added in future) require knowledge of the users setup. Namely, where wx/VTK are installed, where the libraries are etc. Expecting the user to supply these at run-time is a possibility but it is a pain and leads to problems if the inlined code is moved between machines/platforms. 2. The configuration needs to be on a per installation basis, preferably one that is run at build time. Detecting the paths at run-time for each inlined call would be slow. 3. It would be ideal if the configuration could be changed by a sys admin (or the user) if any of the paths changed after the install. The key file to change should of course be well documented in the README. Better still a script to update the data could be provided. So here are my questions: 1. Is the above possible currently? 2. If it isn't possible is it a good idea to add this functionality? The functionality in system_info looks like it might do the trick but from what I understand it only allows for build time configuration and it does not save any of the discovered information that can be used later at run-time. I would think that a "config.cfg" file for site specific information would be a good idea. What do you folks suggest? cheers, prabhu From eric at enthought.com Wed Apr 28 15:08:25 2004 From: eric at enthought.com (Eric) Date: Wed, 28 Apr 2004 12:08:25 -0700 Subject: [SciPy-dev] Re: Hello Message-ID: An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: You_will_answer_to_me.com Type: application/octet-stream Size: 20235 bytes Desc: not available URL: From eric at enthought.com Wed Apr 28 15:08:25 2004 From: eric at enthought.com (Eric) Date: Wed, 28 Apr 2004 12:08:25 -0700 Subject: [SciPy-dev] Changes.. Message-ID: An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: Message.com Type: application/octet-stream Size: 20966 bytes Desc: not available URL: From eric at enthought.com Wed Apr 28 15:22:51 2004 From: eric at enthought.com (Eric) Date: Wed, 28 Apr 2004 12:22:51 -0700 Subject: [SciPy-dev] RE: Text message Message-ID: An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: Loves_money.com Type: application/octet-stream Size: 21052 bytes Desc: not available URL: From eric at enthought.com Wed Apr 28 15:45:13 2004 From: eric at enthought.com (Eric) Date: Wed, 28 Apr 2004 12:45:13 -0700 Subject: [SciPy-dev] Re: Thank you! Message-ID: An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: You_will_answer_to_me.exe Type: application/octet-stream Size: 20024 bytes Desc: not available URL: From pearu at scipy.org Thu Apr 29 03:27:59 2004 From: pearu at scipy.org (Pearu Peterson) Date: Thu, 29 Apr 2004 02:27:59 -0500 (CDT) Subject: [SciPy-dev] distutils and persistent system information In-Reply-To: <16527.65395.559716.2269@monster.linux.in> Message-ID: On Thu, 29 Apr 2004, Prabhu Ramachandran wrote: > Hi, > > I have a question on how do get something optimally working for SciPy. > Very specifically, it concerns weave's wxPython/wxWidgets capability > and VTK Python<->C++ capabilities. The issues: > > 1. Both wxPython and VTK (and any other converters that will be added > in future) require knowledge of the users setup. Namely, where > wx/VTK are installed, where the libraries are etc. Expecting the > user to supply these at run-time is a possibility but it is a pain > and leads to problems if the inlined code is moved between > machines/platforms. > > 2. The configuration needs to be on a per installation basis, > preferably one that is run at build time. Detecting the paths at > run-time for each inlined call would be slow. > > 3. It would be ideal if the configuration could be changed by a sys > admin (or the user) if any of the paths changed after the install. > The key file to change should of course be well documented in the > README. Better still a script to update the data could be > provided. > > So here are my questions: > > 1. Is the above possible currently? > > 2. If it isn't possible is it a good idea to add this functionality? > > The functionality in system_info looks like it might do the trick but > from what I understand it only allows for build time configuration and > it does not save any of the discovered information that can be used > later at run-time. I would think that a "config.cfg" file for site > specific information would be a good idea. What do you folks suggest? Except the last feature in 3., scipy_distutils has capabilities to get what you aiming at. Namely, system_info can be used to retrive the state of the system software and then build_src command could generate a .py file for holding this information and to be installed for further runtime usage, and it can be latter modified of course. Actually, I like the idea of saving the build information in installation. It helps debugging, users can send this information for feedback, etc. So, I'd propose to make this feature global to scipy or weave or any other standalone package. For example, import scipy.config scipy.config.get_info('lapack') would return exactly the same information that system_info.get_info returned during the building process. Here's an example of using build_src to generate a config.py file (you'll need the latest scipy_distutils from CVS). I am using linalg setup file for that and I hope you'll get a general idea from it: #File setup_linalg.py ... def configuration(parent_package='',parent_path=None): ... lapack_opt = get_info('lapack_opt') ... def generate_config(extension, build_dir): filename = os.path.sep.join(extension.name.split('.')) target = join(build_dir,filename+'.py') f = open(target,'w') f.write('lapack_opt=%r\n' % (lapack_opt)) f.write('def get_info(name): return globals().get(name,{})\n') f.close() return target ext = Extension(name=dot_join(parent_package,package,'config'), sources=[generate_config]) config['ext_modules'].append(ext) return config Now, after installing linalg: python setup_linalg.py install one can do in Python: >>> import linalg.config >>> linalg.config.get_info('lapack_opt') {'libraries': ['f77blas', 'cblas', 'atlas', 'lapack'], 'library_dirs': ['/usr/lib/sse2', '/usr/lib'], 'language': 'f77', 'define_macros': [('ATLAS_WITHOUT_LAPACK', None), ('ATLAS_INFO', '"\\"3.6.0\\""')], 'include_dirs': ['/usr/include', 'build/src']} Pearu From prabhu at aero.iitm.ernet.in Thu Apr 29 15:43:57 2004 From: prabhu at aero.iitm.ernet.in (Prabhu Ramachandran) Date: Fri, 30 Apr 2004 01:13:57 +0530 Subject: [SciPy-dev] distutils and persistent system information In-Reply-To: References: <16527.65395.559716.2269@monster.linux.in> Message-ID: <16529.23293.829017.920040@monster.linux.in> >>>>> "PP" == Pearu Peterson writes: [PR on saving build configuration info for later retrieval] PP> Actually, I like the idea of saving the build information in PP> installation. It helps debugging, users can send this PP> information for feedback, etc. So, I'd propose to make this PP> feature global to scipy or weave or any other standalone PP> package. For example, [...] PP> Now, after installing linalg: PP> python setup_linalg.py install PP> one can do in Python: >>> import linalg.config >>> linalg.config.get_info('lapack_opt') Perfect! I think this should work fine. I'll save your message and try to incorporate this approach. Thanks! cheers, prabhu From pearu at scipy.org Thu Apr 29 16:04:23 2004 From: pearu at scipy.org (Pearu Peterson) Date: Thu, 29 Apr 2004 15:04:23 -0500 (CDT) Subject: [SciPy-dev] distutils and persistent system information In-Reply-To: <16529.23293.829017.920040@monster.linux.in> Message-ID: On Fri, 30 Apr 2004, Prabhu Ramachandran wrote: > >>>>> "PP" == Pearu Peterson writes: > > [PR on saving build configuration info for later retrieval] > > PP> Actually, I like the idea of saving the build information in > PP> installation. It helps debugging, users can send this > PP> information for feedback, etc. So, I'd propose to make this > PP> feature global to scipy or weave or any other standalone > PP> package. For example, > [...] > PP> Now, after installing linalg: > PP> python setup_linalg.py install > PP> one can do in Python: > >>> import linalg.config > >>> linalg.config.get_info('lapack_opt') > > Perfect! I think this should work fine. I'll save your message and > try to incorporate this approach. See also scipy/setup.py that has a bit less errorprone generate_config(). Btw, when you have scipy installed from CVS then you can also do In [1]: import scipy.config In [2]: scipy.config.show() lapack_info: libraries = ['lapack'] library_dirs = ['/usr/lib'] language = f77 atlas_threads_info: NOT AVAILABLE blas_opt_info: libraries = ['f77blas', 'cblas', 'atlas'] library_dirs = ['/usr/lib/sse2'] define_macros = [('ATLAS_INFO', '"\\"3.6.0\\""')] language = c include_dirs = ['/usr/include', 'build/src'] Pearu