From eric at scipy.org Mon Feb 11 11:33:10 2002 From: eric at scipy.org (eric) Date: Mon, 11 Feb 2002 11:33:10 -0500 Subject: [SciPy-user] Highlights from the Python10 conference. Message-ID: <01c401c1b319$cb5a94b0$777ba8c0@ericlaptop> Hey Crew, We just got back from the Python conference, and I thought I'd give you an update. Travis Oliphant and I gave a tutorial on Scientific Computing, and, apart from my laptop not working with the projector, it went very well. We'll try to get the slides up on the site as soon as possible. Travis' are very good and will provide nice reference materials for many of the modules. There was also a BoF on Scientific Computing and inling C/C++ in Python where we discussed weave and PyInline. Both were well attended. The Scientific Computing BoF was way to short and we didn't get to all the topics. Scientific Computing: The main issue discussed was whether we should change the default axis on operations like sum, product, etc in both SciPy and the new NumArray module so that all operations default to the same axis. There were multiple views from "leave it like it is", to "force people to include it as an argument (no default)", to "change it and provide a conversion script for people to update old modules". I'll write up in another post to discuss the topic a little further when I've gathered my thoughts on the issue a little more. The second major topic was the discussion of having a NumArray/SciPy workshop in the summer. The details haven't been worked out, but Patrick Miller at LLNL has volunteered to help out with this. Its likely that it will occur in the July timeframe in California at LLNL. We'll post more details as they are available. Hopefully this will get off the ground. Inlining C/C++: This was mainly to let people know about weave and how it worked. Also, I was interested in getting some hints on how to return values from C/C++ back into Python. An off-line conversation with Barry Warsaw gave me some hints, and comments by Skip Montanaro, Fred Drake, and Patrick Miller filled in a few more details. It looks like I can do this by passing in the stack frame object instead of the locals and globals dictionaries to the compiled functions. I don't think it should be to hard. The other exciting thing is that Patrick Miller has a cool package called PyCod that actually translates Python byte code to C. It looks like it will fold into weave quite nicely and has some potentially amazing capabilities. Other things: Pat Miller (again...) got kinda excited about the idea of attempting to run Matlab/Octave scripts using Python's VM -- or maybe a Matlab to Python translator. If this can be made to work in a reliable way, it would be incredibly powerful. Pat Miller (yes, we talked quite a bit :) and I discussed merging ideas from pyMPI and cow.py for a more versatile set of parallel programming capabilities. Pat and his LLNL colleague, Martin Casado, are real experts in this area so there are some good possibilities here. Paul Dubois really urged SciPy to work toward a peer reviewed process by experts in each field to make sure the algorithms used are really the "best of class" in there field. He also feels an strong object oriented interface in SciPy to its underlying functions would be a huge win. He has some papers in this area which I and others hope to read. Janko Hauser offered to work on getting Numeric's documentation and some of the SciPy modules in the same documentation format that Python uses. We had long discussions on plotting with Perry Greenfield, Richard White, Janko Hauser, Travis Oliphant, David Ascher and others (too many to list). I'll summarize this in another posting also. Michel Sanner from the Scripts Institute gave a talk about his AVS work-alike package for visual programming. It is really cool and won the best paper award! (congrats). You should definitely take a look at it: http://www.scripps.edu/pub/olson-web/people/sanner/home.html David Beazley talked about SWIG's new features. Its come a long way baby and now has a full C preprocessor, supports C++ templates, handles multiple argument typemaps, and much more. This ain't your grandmother's C library wrapper. More details at http://www.swig.org There are also some possiblilities of parts of SWIG becoming Python modules which has some interesting implications for weave. I had a short talk with Dave about PLY also. It a Lex/Yacc type tool, and it is apparently getting good reviews from users. You might want to check it out his home page: http://systems.cs.uchicago.edu/ply/ Todd Velantic, from SRI, gave a talk about how they retreive data from *very* remote instrumentation (Radar, etc.) in Greenland, the South Pole, etc. over very slow and periodically available connectivity. There solution uses NNTP in a very clever way to send binary data around. It solved multiple problems for them without reinventing the wheel. Developers day had a session on optimizing the Python virtual machine by Skip Montanaro and Jeremy Hylton. They have alternate approaches to speeding up the thing. The long and the short of it is that there is a little to squeeze out, but it looks like in the 5-30% range, not orders of magnitude. Another approach adopted by the Psycho project actually does JIT of Python byte codes to assembly language (at least this is as near as I can tell what it does...) and can provide huge speed ups from 2-10 times by some accounts. This has some promise, but also concern about how much effort this avenue would require and whether it would truely result in a workable solution. I don't know much about any of this stuff, but it was fun to listen. I missed the talk on the type/class merger ( had to catch a flight). bummer. Fred Drake talked about a new profiler called HotShot that is in the Python CVS now, but still needs some work. It sounds usable though, and is written in C so that it is much faster and less invasive than the current profile module which is written in C. PyChecker by Neil ?? looks like a cool tool for checking for errors in Python code. Guido was very excited about its capabilities, and thinks it may get eventually folded into the compiler somehow (at least I think this is what he said). http://pychecker.sourceforge.net/ Attendance of the conference was slightly up from last year I believe. The scientific community had a very strong presence there. Lots more went on, but at least this hits a few highlights. see ya, eric -- Eric Jones Enthought, Inc. [www.enthought.com and www.scipy.org] (512) 536-1057 From fperez at pizero.colorado.edu Tue Feb 12 03:22:49 2002 From: fperez at pizero.colorado.edu (=?ISO-8859-1?Q?Fernando_P=E9rez?=) Date: Tue, 12 Feb 2002 01:22:49 -0700 (MST) Subject: [SciPy-user] ANN: IPython 0.2.5 In-Reply-To: <01c401c1b319$cb5a94b0$777ba8c0@ericlaptop> Message-ID: Hi everyone, sorry for the cross-posting (this was in c.l.p), but I actually think some of you may find this useful if you don't read c.l.p. If you do, just ignore this message and don't beat me up :) Below is a summary of what IPython is (from the c.l.p post), I'd just like to add here why I think it's relevant for scipy folks: the whole design for IPython was inspired on features from systems like Mathematica, IDL or Matlab. In scientific work having a solid interactive environment in which to explore your data and ideas is critical, and I felt the standard python prompt to be very limited. Hence IPython: I designed it thinking of it being the user front end for interactive scientific computing, while the scipy libraries are the workhorse behind. So my idea is that IPython would be a nice complement to the scipy libraries for those doing day-to-day scientific computing in python. It's low-versioned but I actually use it 100% of the time for my work, and it hasn't crashed in quite a while (famous last words :). If it crashes on you, just mail me the auto-generated post-mortem and I'll try to fix things. Ok, enough self promotion. The summary, and I hope some of you find it useful. Brief summary: IPython, an enhanced interactive Python shell. WHERE: http://www-hep.colorado.edu/~fperez/ipython IPython tries to: 1. Provide an interactive interpreter superior to Python's default. IPython has many features for object introspection, shell access, and its own special command system for adding functionality when working interactively. 2. Serve as an embeddable, ready to use interpreter for your own programs. IPython can be started with a single call from inside another program, providing access to the current namespace. This can be very useful both for debugging purposes and for situations where a blend of batch-processing and interactive exploration are needed. 3. Offer a flexible framework which can be used as the base environment for other systems with Python as the underlying language. Specifically scientific environments like Mathematica, IDL and Mathcad inspired its design, but similar ideas can be useful in many fields. Portability: Linux (and other unices, including Mac OSX), Windows XP. Should run fine on all WinNT, and probably also on Win9x (I can't test that). Python version: requires 2.1 or newer. License: LGPL (a few files from third parties carry MIT licenses). Cheers, Fernando Perez. From karshi.hasanov at utoronto.ca Thu Feb 14 20:53:18 2002 From: karshi.hasanov at utoronto.ca (Karshi) Date: Thu, 14 Feb 2002 20:53:18 -0500 Subject: [SciPy-user] plt.image_help Message-ID: <20020215015408Z234721-27139+3@bureau8.utcc.utoronto.ca> Hi all, I start getting the following messages: ------------------------------------------------ >>> from scipy import plt >>> plt.image(M) Traceback (most recent call last): File "", line 1, in ? File "/usr/local/lib/python2.1/site-packages/scipy/plt/interface.py", line 427, in image axis('equal') File "/usr/local/lib/python2.1/site-packages/scipy/plt/interface.py", line 174, in axis x_ticks = _active.x_axis.ticks AttributeError: axis_window instance has no attribute 'ticks' >>> image=plt.lena() >>> plt.image(image) Traceback (most recent call last): File "", line 1, in ? File "/usr/local/lib/python2.1/site-packages/scipy/plt/interface.py", line 427, in image axis('equal') File "/usr/local/lib/python2.1/site-packages/scipy/plt/interface.py", line 174, in axis x_ticks = _active.x_axis.ticks AttributeError: axis_window instance has no attribute 'ticks' ----------------------------------------------- I don't know what I've done , it was working few minutes ago. From pearu at cens.ioc.ee Fri Feb 15 05:14:51 2002 From: pearu at cens.ioc.ee (Pearu Peterson) Date: Fri, 15 Feb 2002 12:14:51 +0200 (EET) Subject: [SciPy-user] plt.image_help In-Reply-To: <20020215015408Z234721-27139+3@bureau8.utcc.utoronto.ca> Message-ID: On Thu, 14 Feb 2002, Karshi wrote: > ----------------------------------------------- > I don't know what I've done , it was working few minutes ago. > _______________________________________________ I wonder if resetting the computers clock to the past would fix this? ;-) Pearu From eric at scipy.org Sun Feb 17 03:54:15 2002 From: eric at scipy.org (eric) Date: Sun, 17 Feb 2002 03:54:15 -0500 Subject: [SciPy-user] plt.image_help References: <20020215015408Z234721-27139+3@bureau8.utcc.utoronto.ca> Message-ID: <021601c1b790$add14030$6b01a8c0@ericlaptop> Hi Karshi, > > Hi all, > > I start getting the following messages: > ------------------------------------------------ > >>> from scipy import plt > >>> plt.image(M) > Traceback (most recent call last): > File "", line 1, in ? > File "/usr/local/lib/python2.1/site-packages/scipy/plt/interface.py", line > 427, in image > axis('equal') > File "/usr/local/lib/python2.1/site-packages/scipy/plt/interface.py", line > 174, in axis > x_ticks = _active.x_axis.ticks > AttributeError: axis_window instance has no attribute 'ticks' > >>> image=plt.lena() > >>> plt.image(image) > Traceback (most recent call last): > File "", line 1, in ? > File "/usr/local/lib/python2.1/site-packages/scipy/plt/interface.py", line > 427, in image > axis('equal') > File "/usr/local/lib/python2.1/site-packages/scipy/plt/interface.py", line > 174, in axis > x_ticks = _active.x_axis.ticks > AttributeError: axis_window instance has no attribute 'ticks' > > ----------------------------------------------- > I don't know what I've done , it was working few minutes ago. I'm not sure what is happening here either. I vaguely remember seeing this at one point, but don't remember what the problem was. I'm pretty sure it is fixed in the CVS. eric > _______________________________________________ > SciPy-user mailing list > SciPy-user at scipy.net > http://www.scipy.net/mailman/listinfo/scipy-user > From karshi.hasanov at utoronto.ca Sun Feb 17 19:14:47 2002 From: karshi.hasanov at utoronto.ca (Karshi) Date: Sun, 17 Feb 2002 19:14:47 -0500 Subject: [SciPy-user] CVS_installation? Message-ID: <20020218001523Z234729-17762+4@bureau8.utcc.utoronto.ca> Hi all, I am having a problem with installion of CVS- version . First, I had to install ATLAS ( it took about 2-> 3 hours), then the following messages show up when I try " setup.py" . ----------------------------------------------------------------------------- skipping /home/khasanov/Python/scipy/xplt/gist/hlevel.c (build/temp.linux-i686-2.1/hlevel.o up-to-date) skipping /home/khasanov/Python/scipy/xplt/gist/xfont.c (build/temp.linux-i686-2.1/xfont.o up-to-date) skipping build/temp.linux-i686-2.1/libgist.a (up-to-date) running run_f2py Traceback (most recent call last): File "setup.py", line 126, in ? install_package() File "setup.py", line 116, in install_package url = "http://www.scipy.org", File "scipy_distutils/core.py", line 43, in setup return old_setup(**new_attr) File "/usr/local/lib/python2.1/distutils/core.py", line 138, in setup dist.run_commands() File "/usr/local/lib/python2.1/distutils/dist.py", line 899, in run_commands self.run_command(cmd) File "/usr/local/lib/python2.1/distutils/dist.py", line 919, in run_command cmd_obj.run() File "/usr/local/lib/python2.1/distutils/command/build.py", line 106, in run self.run_command(cmd_name) File "/usr/local/lib/python2.1/distutils/cmd.py", line 328, in run_command self.distribution.run_command(command) File "/usr/local/lib/python2.1/distutils/dist.py", line 919, in run_command cmd_obj.run() File "scipy_distutils/command/run_f2py.py", line 65, in run ext.sources = self.f2py_sources(ext.sources,ext) File "scipy_distutils/command/run_f2py.py", line 79, in f2py_sources import f2py2e ImportError: No module named f2py2e ------------------------------------------------------------------------- Thanks From pearu at cens.ioc.ee Sun Feb 17 19:36:07 2002 From: pearu at cens.ioc.ee (Pearu Peterson) Date: Mon, 18 Feb 2002 02:36:07 +0200 (EET) Subject: [SciPy-user] CVS_installation? In-Reply-To: <20020218001523Z234729-17762+4@bureau8.utcc.utoronto.ca> Message-ID: Karshi, On Sun, 17 Feb 2002, Karshi wrote: > I am having a problem with installion of CVS- version . > First, I had to install ATLAS ( it took about 2-> 3 hours), then > the following messages show up when I try " setup.py" . > ----------------------------------------------------------------------------- > import f2py2e > ImportError: No module named f2py2e You'll need also to install f2py: http://cens.ioc.ee/projects/f2py2e/ It is very simple, just run 'setup.py install' in f2py2e directory when you have unpacked the latest f2py. Pearu From karshi.hasanov at utoronto.ca Sun Feb 17 21:59:10 2002 From: karshi.hasanov at utoronto.ca (Karshi) Date: Sun, 17 Feb 2002 21:59:10 -0500 Subject: [SciPy-user] CVS_installation Message-ID: <20020218025947Z234653-22402+3@bureau8.utcc.utoronto.ca> Hi After installing "f2pys" package , am still getting errors: ------------------------------------- >>> import scipy >>> scipy.test() Traceback (most recent call last): File "", line 1, in ? File "/usr/local/lib/python2.1/site-packages/scipy/basic.py", line 333, in test module_test(__name__,__file__) File "/usr/local/lib/python2.1/site-packages/scipy/scipy_test.py", line 156, in module_test exec(test_string) File "", line 1, in ? File "/usr/local/lib/python2.1/site-packages/scipy/tests/test_basic.py", line 534, in test all_tests = test_suite() File "/usr/local/lib/python2.1/site-packages/scipy/tests/test_basic.py", line 523, in test_suite suites.append( unittest.makeSuite(test_trapz,'check_') ) NameError: global name 'test_trapz' is not defined From eric at scipy.org Mon Feb 18 12:42:07 2002 From: eric at scipy.org (eric) Date: Mon, 18 Feb 2002 12:42:07 -0500 Subject: [SciPy-user] CVS_installation References: <20020218025947Z234653-22402+3@bureau8.utcc.utoronto.ca> Message-ID: <04b101c1b8a3$95edbc50$6b01a8c0@ericlaptop> Hey Karshi, This looks like a problem with the test suite. I'll look into it. In the mean time, you might play aorund with scipy and see if everything else seems to work. THere is a reasonable chance that your installation is working fine, and just the tests are broken. eric ----- Original Message ----- From: "Karshi" To: Sent: Sunday, February 17, 2002 9:59 PM Subject: [SciPy-user] CVS_installation > > Hi > > After installing "f2pys" package , am still getting errors: > ------------------------------------- > >>> import scipy > >>> scipy.test() > Traceback (most recent call last): > File "", line 1, in ? > File "/usr/local/lib/python2.1/site-packages/scipy/basic.py", line 333, in > test > module_test(__name__,__file__) > File "/usr/local/lib/python2.1/site-packages/scipy/scipy_test.py", line > 156, in module_test > exec(test_string) > File "", line 1, in ? > File "/usr/local/lib/python2.1/site-packages/scipy/tests/test_basic.py", > line 534, in test > all_tests = test_suite() > File "/usr/local/lib/python2.1/site-packages/scipy/tests/test_basic.py", > line 523, in test_suite > suites.append( unittest.makeSuite(test_trapz,'check_') ) > NameError: global name 'test_trapz' is not defined > _______________________________________________ > SciPy-user mailing list > SciPy-user at scipy.net > http://www.scipy.net/mailman/listinfo/scipy-user > From karshi.hasanov at utoronto.ca Tue Feb 19 13:38:17 2002 From: karshi.hasanov at utoronto.ca (Karshi) Date: Tue, 19 Feb 2002 13:38:17 -0500 Subject: [SciPy-user] installation_problems Message-ID: <20020219183857Z234752-27097+5@bureau8.utcc.utoronto.ca> Hi Eric, It seem to me nothing is working : ---------------------------------------------- khasanov at compton:~> python Python 2.1.1 (#1, Jan 13 2002, 14:58:11) [GCC 2.95.3 20010315 (SuSE)] on linux2 Type "copyright", "credits" or "license" for more information. >>> from scipy import * /usr/local/lib/python2.1/site-packages/scipy/linalg/clapack.so: undefined sy mbol: clapack_sgetrf Warning: FFT package not found. Some names will not be available Traceback (most recent call last): File "", line 1, in ? File "/usr/local/lib/python2.1/site-packages/scipy/__init__.py", line 91, in ? names2all(__all__, _level1a, globals()) File "/usr/local/lib/python2.1/site-packages/scipy/__init__.py", line 41, in names2all exec("import %s" % name, gldict) File "", line 1, in ? File "/usr/local/lib/python2.1/site-packages/scipy/basic1a.py", line 10, i n ? from scipy.linalg import eig ImportError: cannot import name eig >>> -------------------------------------------------------- From pearu at cens.ioc.ee Tue Feb 19 13:52:25 2002 From: pearu at cens.ioc.ee (Pearu Peterson) Date: Tue, 19 Feb 2002 20:52:25 +0200 (EET) Subject: [SciPy-user] installation_problems In-Reply-To: <20020219183857Z234752-27097+5@bureau8.utcc.utoronto.ca> Message-ID: Hi Karshi, I have experienced exactly the same failure several times. And each time I have forgot to fix scipy_distutils/atlas_info.py: library_path = '' to something like library_path = ['/path/to/my/atlas/'] Could you check also that? In my system, directory /path/to/my/atlas/ contains the following files: libatlas.a libcblas.a libf77blas.a liblapack.a libtstatlas.a If that seems to be correct in your system and also the library_path was fixed, then try running cd scipy/linalg python setup_linalg.py build and send me the output. Btw, do you use the latest scipy from CVS? Please, update, if not. Regards, Pearu On Tue, 19 Feb 2002, Karshi wrote: > It seem to me nothing is working : > ---------------------------------------------- > khasanov at compton:~> python > Python 2.1.1 (#1, Jan 13 2002, 14:58:11) > [GCC 2.95.3 20010315 (SuSE)] on linux2 > Type "copyright", "credits" or "license" for more information. > >>> from scipy import * > /usr/local/lib/python2.1/site-packages/scipy/linalg/clapack.so: undefined sy > mbol: clapack_sgetrf > Warning: FFT package not found. Some names will not be available > Traceback (most recent call last): > File "", line 1, in ? > File "/usr/local/lib/python2.1/site-packages/scipy/__init__.py", line 91, > in ? > names2all(__all__, _level1a, globals()) > File "/usr/local/lib/python2.1/site-packages/scipy/__init__.py", line 41, > in names2all > exec("import %s" % name, gldict) > File "", line 1, in ? > File "/usr/local/lib/python2.1/site-packages/scipy/basic1a.py", line 10, i > n ? > from scipy.linalg import eig > ImportError: cannot import name eig > >>> From eric at scipy.org Tue Feb 19 12:56:34 2002 From: eric at scipy.org (eric) Date: Tue, 19 Feb 2002 12:56:34 -0500 Subject: [SciPy-user] installation_problems References: Message-ID: <000901c1b96e$c53328d0$de5bfea9@ericlaptop> Pearu, Is everything working for you? eric ----- Original Message ----- From: "Pearu Peterson" To: Sent: Tuesday, February 19, 2002 1:52 PM Subject: Re: [SciPy-user] installation_problems > > Hi Karshi, > > I have experienced exactly the same failure several times. And each time I > have forgot to fix > > scipy_distutils/atlas_info.py: library_path = '' > > to something like > library_path = ['/path/to/my/atlas/'] > > Could you check also that? In my system, directory /path/to/my/atlas/ > contains the following files: > libatlas.a libcblas.a libf77blas.a liblapack.a libtstatlas.a > > If that seems to be correct in your system and also the library_path was > fixed, then try running > cd scipy/linalg > python setup_linalg.py build > > and send me the output. Btw, do you use the latest scipy from CVS? Please, > update, if not. > > Regards, > Pearu > > On Tue, 19 Feb 2002, Karshi wrote: > > > It seem to me nothing is working : > > ---------------------------------------------- > > khasanov at compton:~> python > > Python 2.1.1 (#1, Jan 13 2002, 14:58:11) > > [GCC 2.95.3 20010315 (SuSE)] on linux2 > > Type "copyright", "credits" or "license" for more information. > > >>> from scipy import * > > /usr/local/lib/python2.1/site-packages/scipy/linalg/clapack.so: undefined sy > > mbol: clapack_sgetrf > > Warning: FFT package not found. Some names will not be available > > Traceback (most recent call last): > > File "", line 1, in ? > > File "/usr/local/lib/python2.1/site-packages/scipy/__init__.py", line 91, > > in ? > > names2all(__all__, _level1a, globals()) > > File "/usr/local/lib/python2.1/site-packages/scipy/__init__.py", line 41, > > in names2all > > exec("import %s" % name, gldict) > > File "", line 1, in ? > > File "/usr/local/lib/python2.1/site-packages/scipy/basic1a.py", line 10, i > > n ? > > from scipy.linalg import eig > > ImportError: cannot import name eig > > >>> > > _______________________________________________ > SciPy-user mailing list > SciPy-user at scipy.net > http://www.scipy.net/mailman/listinfo/scipy-user > From pearu at cens.ioc.ee Tue Feb 19 14:09:42 2002 From: pearu at cens.ioc.ee (Pearu Peterson) Date: Tue, 19 Feb 2002 21:09:42 +0200 (EET) Subject: [SciPy-user] installation_problems In-Reply-To: <000901c1b96e$c53328d0$de5bfea9@ericlaptop> Message-ID: On Tue, 19 Feb 2002, eric wrote: > Is everything working for you? Yes, almost. I just build scipy from latest CVS and it imports fine (recently this has been my only test for success ;). However, now when I run scipy.test(), I get what follows below. Should I be worry about it? ;-) Seems that linear_algebra is still broken :( Pearu creating test suite for: scipy.basic !! FAILURE building test for scipy.basic scipy/tests/test_basic.py:524: NameError: global name 'test_trapz' is not defined (in test_suite) creating test suite for: scipy.stats.new_stats !! FAILURE building test for scipy.stats.new_stats :1: ImportError: No module named test_new_stats (in ?) No test suite found for scipy.stats.rv creating test suite for: scipy.stats.stats !! FAILURE building test for scipy.stats.stats :1: ImportError: No module named test_stats (in ?) FAILURE to import scipy.weave.wx_info :0: AttributeError: 'scipy.weave' module has no attribute 'wx_info' (in ?) FAILURE to import scipy.weave.accelerate_tools :0: AttributeError: 'scipy.weave' module has no attribute 'accelerate_tools' (in ?) FAILURE to import scipy.weave.wx_spec :0: AttributeError: 'scipy.weave' module has no attribute 'wx_spec' (in ?) FAILURE to import scipy.weave.swig_info :0: AttributeError: 'scipy.weave' module has no attribute 'swig_info' (in ?) No test suite found for scipy.integrate ......................E.......E......E....................................................................................dgeev:lwork=0 Edgeev:lwork=0 E....................................warning: specified build_dir '_bad_path_' does not exist or is or is not writable. Trying default locations ....warning: specified build_dir '_bad_path_' does not exist or is or is not writable. Trying default locations .................................................................................................. ====================================================================== ERROR: check_qnan (test_misc.test_isnan) ---------------------------------------------------------------------- Traceback (most recent call last): File "scipy/tests/test_misc.py", line 112, in check_qnan assert(isnan(log(-1.)) == 1) ValueError: math domain error ====================================================================== ERROR: check_basic (test_basic1a.test_roots) ---------------------------------------------------------------------- Traceback (most recent call last): File "scipy/tests/test_basic1a.py", line 19, in check_basic assert_array_almost_equal(roots(a1),[2,2],11) File "scipy/basic1a.py", line 51, in roots roots,dummy = eig(A) File "scipy/linalg/linear_algebra.py", line 440, in eig results = ev(a, jobvl='N', jobvr=vchar, lwork=results[-2][0]) error: ((lwork==-1) || (lwork >= MAX(1,4*n))) failed for 3rd keyword lwork ====================================================================== ERROR: check_inverse (test_basic1a.test_roots) ---------------------------------------------------------------------- Traceback (most recent call last): File "scipy/tests/test_basic1a.py", line 25, in check_inverse assert_array_almost_equal(sort(roots(poly(a))),sort(a),5) File "scipy/basic1a.py", line 51, in roots roots,dummy = eig(A) File "scipy/linalg/linear_algebra.py", line 440, in eig results = ev(a, jobvl='N', jobvr=vchar, lwork=results[-2][0]) error: ((lwork==-1) || (lwork >= MAX(1,4*n))) failed for 3rd keyword lwork ---------------------------------------------------------------------- Ran 262 tests in 5.790s FAILED (errors=5) >>> From eric at scipy.org Tue Feb 19 13:39:02 2002 From: eric at scipy.org (eric) Date: Tue, 19 Feb 2002 13:39:02 -0500 Subject: [SciPy-user] installation_problems References: Message-ID: <00a201c1b974$b3cf3290$de5bfea9@ericlaptop> > > On Tue, 19 Feb 2002, eric wrote: > > > Is everything working for you? > > Yes, almost. I just build scipy from latest CVS and it imports fine > (recently this has been my only test for success ;). Me too. And that didn't exactly make me brim with confidence. > However, now when I run scipy.test(), I get what follows below. > Should I be worry about it? ;-) Well, on the bright side, 257 tests pass. I get some errors also, but they all deal with the isnan IEEE junk. This stuff is a mess and mail fail until we get Tim Peters to tell us what the heck to do with it... :-| > Seems that linear_algebra is still broken :( Hmmm. I get 3 errors on windows, but, as I said, all of them deal with isnan and friends. I don't get any errors in linear_algebra. Did poly() work for you before? I'll hunt down why your getting all the wx_info, wx_spec stuff from weave. I'm not seeing this either, but I think I have some ideas. Also, I think it is OK to have tests that fail at this point. The reason we write them is to tell us what still needs to be worked on. I'd rather have a lot of test that fail instead of no test at all. eric > > Pearu > > > creating test suite for: scipy.basic > !! FAILURE building test for scipy.basic > scipy/tests/test_basic.py:524: NameError: global name 'test_trapz' is > not defined (in test_suite) > > creating test suite for: scipy.stats.new_stats > !! FAILURE building test for scipy.stats.new_stats > :1: ImportError: No module named test_new_stats (in ?) > No test suite found for scipy.stats.rv > creating test suite for: scipy.stats.stats > !! FAILURE building test for scipy.stats.stats > :1: ImportError: No module named test_stats (in ?) > FAILURE to import scipy.weave.wx_info > :0: AttributeError: 'scipy.weave' module has no attribute > 'wx_info' (in ?) > FAILURE to import scipy.weave.accelerate_tools > :0: AttributeError: 'scipy.weave' module has no attribute > 'accelerate_tools' > (in ?) > FAILURE to import scipy.weave.wx_spec > :0: AttributeError: 'scipy.weave' module has no attribute > 'wx_spec' (in ?) > FAILURE to import scipy.weave.swig_info > :0: AttributeError: 'scipy.weave' module has no attribute > 'swig_info' (in ?) > > No test suite found for scipy.integrate > ......................E.......E......E.......................................... ..........................................dgeev:lwork=0 > Edgeev:lwork=0 > E....................................warning: specified build_dir > '_bad_path_' does not exist or is or is not writable. Trying default > locations > ....warning: specified build_dir '_bad_path_' does not exist or is or is > not writable. Trying default locations > ................................................................................ .................. > ====================================================================== > ERROR: check_qnan (test_misc.test_isnan) > ---------------------------------------------------------------------- > Traceback (most recent call last): > File "scipy/tests/test_misc.py", line 112, in check_qnan > assert(isnan(log(-1.)) == 1) > ValueError: math domain error > > ====================================================================== > ERROR: check_basic (test_basic1a.test_roots) > ---------------------------------------------------------------------- > Traceback (most recent call last): > File "scipy/tests/test_basic1a.py", line 19, in check_basic > assert_array_almost_equal(roots(a1),[2,2],11) > File "scipy/basic1a.py", line 51, in roots > roots,dummy = eig(A) > File "scipy/linalg/linear_algebra.py", line 440, in eig > results = ev(a, jobvl='N', jobvr=vchar, lwork=results[-2][0]) > error: ((lwork==-1) || (lwork >= MAX(1,4*n))) failed for 3rd keyword lwork > ====================================================================== > ERROR: check_inverse (test_basic1a.test_roots) > ---------------------------------------------------------------------- > Traceback (most recent call last): > File "scipy/tests/test_basic1a.py", line 25, in check_inverse > assert_array_almost_equal(sort(roots(poly(a))),sort(a),5) > File "scipy/basic1a.py", line 51, in roots > roots,dummy = eig(A) > File "scipy/linalg/linear_algebra.py", line 440, in eig > results = ev(a, jobvl='N', jobvr=vchar, lwork=results[-2][0]) > error: ((lwork==-1) || (lwork >= MAX(1,4*n))) failed for 3rd keyword lwork > ---------------------------------------------------------------------- > Ran 262 tests in 5.790s > > FAILED (errors=5) > > >>> > > > _______________________________________________ > SciPy-user mailing list > SciPy-user at scipy.net > http://www.scipy.net/mailman/listinfo/scipy-user > From nadavh at envision.co.il Wed Feb 20 03:45:42 2002 From: nadavh at envision.co.il (Nadav Horesh) Date: Wed, 20 Feb 2002 10:45:42 +0200 Subject: [SciPy-user] Eror estimation in leastsq Message-ID: <3C736236.4070201@envision.co.il> Hello, Is there a way to estimate the errors (standard deviations) of each optimized parameter, from the output of scipy.opimize.leastsq? Nadav. From jjl at pobox.com Wed Feb 20 08:46:38 2002 From: jjl at pobox.com (John J. Lee) Date: Wed, 20 Feb 2002 13:46:38 +0000 (GMT) Subject: [SciPy-user] Eror estimation in leastsq In-Reply-To: <3C736236.4070201@envision.co.il> Message-ID: On Wed, 20 Feb 2002, Nadav Horesh wrote: > Is there a way to estimate the errors (standard deviations) of each > optimized parameter, from the > output of scipy.opimize.leastsq? Yes, but I don't remember offhand exactly how. Look in the (pure) Python module that calls the C wrapper -- there are comments there explaining what exactly gets returned. John From eric at scipy.org Wed Feb 20 11:01:01 2002 From: eric at scipy.org (eric) Date: Wed, 20 Feb 2002 11:01:01 -0500 Subject: [SciPy-user] Eror estimation in leastsq References: <3C736236.4070201@envision.co.il> Message-ID: <038d01c1ba27$cb648d20$de5bfea9@ericlaptop> Hey Nadav, Hmm. Just noticed there are two least squares algorithms. The one in scipy/optimize/minpack.py (which you are using) can output quite a bit of data, but it isn't obvious that one of them is the residuals to me. Maybe Travis O. has more idea about this. There is also a scipy.linalg.lstsq (note spelling difference). This one uses singular value decomposition while the other one uses minpack routines. It's doc string doesn't say much about it, but looking at the code, its return value is: def lstsq(A, B, rcond=None): ... return x, resids, rank_v, s I'm betting the second one is what you want. Now the next question is why the heck there are two least squares algorithms in SciPy. For that, I have no good answer, other than its still evolving... :-) see ya, eric ----- Original Message ----- From: "Nadav Horesh" To: Sent: Wednesday, February 20, 2002 3:45 AM Subject: [SciPy-user] Eror estimation in leastsq > Hello, > > Is there a way to estimate the errors (standard deviations) of each > optimized parameter, from the > output of scipy.opimize.leastsq? > > Nadav. > > _______________________________________________ > SciPy-user mailing list > SciPy-user at scipy.net > http://www.scipy.net/mailman/listinfo/scipy-user > From oliphant at ee.byu.edu Wed Feb 20 10:11:10 2002 From: oliphant at ee.byu.edu (Travis Oliphant) Date: Wed, 20 Feb 2002 10:11:10 -0500 (EST) Subject: [SciPy-user] Eror estimation in leastsq In-Reply-To: <038d01c1ba27$cb648d20$de5bfea9@ericlaptop> Message-ID: > def lstsq(A, B, rcond=None): > ... > return x, resids, rank_v, s > > I'm betting the second one is what you want. > > Now the next question is why the heck there are two least squares algorithms in > SciPy. For that, I have no good answer, other than its still evolving... :-) > They solve two different problems. The one you mentioned solves the linear least-squares problem, while the one in minpack is much more general. I don't remember exactly how to get residuals from the minpack algorithm. We still need to add a "total least squares" algorithm with its accompanying error estimates. -Travis From fperez at pizero.colorado.edu Wed Feb 20 12:09:40 2002 From: fperez at pizero.colorado.edu (=?ISO-8859-1?Q?Fernando_P=E9rez?=) Date: Wed, 20 Feb 2002 10:09:40 -0700 (MST) Subject: [SciPy-user] Eror estimation in leastsq In-Reply-To: <038d01c1ba27$cb648d20$de5bfea9@ericlaptop> Message-ID: On Wed, 20 Feb 2002, eric wrote: > Now the next question is why the heck there are two least squares algorithms in > SciPy. For that, I have no good answer, other than its still evolving... :-) I haven't looked at the code, but in general using svd for least-squares fitting is a good idea. The one issue that can get tricky is where to cutoff small singular values. But if that's addressed correctly in the linalg version, *and* the other version is using a straight normal equations solution, it would probably be best to keep the linalg one. Solving the normal equations directly is often numerically tricky/unstable. I have some svd code around (in Python), maybe next week I'll take a peek at these two versions, compare them and drop some comments on the issue more informed than what I just blabbered. Need time ... :) Cheers, f. From kern at caltech.edu Wed Feb 20 16:25:49 2002 From: kern at caltech.edu (Robert Kern) Date: Wed, 20 Feb 2002 13:25:49 -0800 Subject: [SciPy-user] Eror estimation in leastsq In-Reply-To: References: <038d01c1ba27$cb648d20$de5bfea9@ericlaptop> Message-ID: <20020220212549.GA15668@taliesen.caltech.edu> On Wed, Feb 20, 2002 at 10:11:10AM -0500, Travis Oliphant wrote: [snip] > They solve two different problems. The one you mentioned solves the > linear least-squares problem, while the one in minpack is much more > general. I don't remember exactly how to get residuals from the minpack > algorithm. I don't think that solves the original problem, though. Nadav requested the estimated standard deviations of the estimated parameters, not the residuals of the data to the fit curve. I don't think you can go from one to the other (IIRC). > We still need to add a "total least squares" algorithm with its > accompanying error estimates. http://www.netlib.org/odrpack is probably the way to go. My wrappers work, for the most part, but I still need to track down a segfault when running multiple fits. It's hand-written C loosely modelled after the minpack wrappers. At the time, I didn't think that f2py had the capabilities I needed. I should rethink that now. Should some free time magically appear, I will hunt down the remaining bugs and submit the module to you all for inclusion in SciPy. If anyone else is interested, mostly up-to-date source is at http://starship.python.net/crew/kernr/source/odr-0.5.1.tar.gz > -Travis -- Robert Kern Ruddock House President kern at caltech.edu "In the fields of hell where the grass grows high Are the graves of dreams allowed to die." -- Richard Harter From oliphant at ee.byu.edu Wed Feb 20 14:39:12 2002 From: oliphant at ee.byu.edu (Travis Oliphant) Date: Wed, 20 Feb 2002 14:39:12 -0500 (EST) Subject: [SciPy-user] Eror estimation in leastsq In-Reply-To: <20020220212549.GA15668@taliesen.caltech.edu> Message-ID: > I don't think that solves the original problem, though. Nadav requested > the estimated standard deviations of the estimated parameters, not the > residuals of the data to the fit curve. I don't think you can go from > one to the other (IIRC). > > > We still need to add a "total least squares" algorithm with its > > accompanying error estimates. > > http://www.netlib.org/odrpack is probably the way to go. My wrappers > work, for the most part, but I still need to track down a segfault when > running multiple fits. > > It's hand-written C loosely modelled after the minpack wrappers. At the > time, I didn't think that f2py had the capabilities I needed. I should > rethink that now. > > Should some free time magically appear, I will hunt down the remaining > bugs and submit the module to you all for inclusion in SciPy. If anyone > else is interested, mostly up-to-date source is at > http://starship.python.net/crew/kernr/source/odr-0.5.1.tar.gz Thanks for the pointers. -Travis From nwagner at mecha.uni-stuttgart.de Thu Feb 21 10:30:32 2002 From: nwagner at mecha.uni-stuttgart.de (Nils Wagner) Date: Thu, 21 Feb 2002 16:30:32 +0100 Subject: [SciPy-user] Problems during build Message-ID: <3C751298.F5B1F87@mecha.uni-stuttgart.de> Hi, I just tried to build scipy from latest CVS ... wagner at lisa:~/mysoftware/scipy> python setup.py build Traceback (most recent call last): File "setup.py", line 127, in ? install_package() File "setup.py", line 93, in install_package config.extend([get_package_config(x,parent_package)for x in standard_packages]) File "setup.py", line 46, in get_package_config config = mod.configuration(parent) File "linalg/setup_linalg.py", line 26, in configuration config['ext_modules'].extend(extensions(dot_join(parent_package,'linalg'))) File "linalg/setup_linalg.py", line 78, in extensions ext = generic_extension(mod,sources,parent_package,use_underscore=1) File "linalg/setup_linalg.py", line 36, in generic_extension raise ValueError, msg ValueError: Atlas libraries not found. Either install them in /usr/lib/atlas or /usr/local/lib/atlas and re try setup.py, or edit setup.py to specify your own blas and lapack directories and libs How can I specify the blas/lapack/atlas directories and libs in setup.py ? These are my local settings: -rw-r--r-- 1 root root 3624658 Sep 24 02:51 /usr/lib/libatlas.a -rw-r--r-- 1 root root 657706 Sep 24 01:00 /usr/lib/libblas.a lrwxrwxrwx 1 root root 12 Okt 22 19:27 /usr/lib/libblas.so -> libblas.so.2 lrwxrwxrwx 1 root root 16 Okt 22 19:27 /usr/lib/libblas.so.2 -> libblas.so.2.2.0 -rwxr-xr-x 1 root root 559600 Sep 24 01:01 /usr/lib/libblas.so.2.2.0 -rw-r--r-- 1 root root 5763150 Sep 24 01:00 /usr/lib/liblapack.a lrwxrwxrwx 1 root root 14 Okt 22 19:27 /usr/lib/liblapack.so -> liblapack.so.3 lrwxrwxrwx 1 root root 18 Okt 22 19:27 /usr/lib/liblapack.so.3 -> liblapack.so.3.0.0 -rwxr-xr-x 1 root root 4826626 Sep 24 01:01 /usr/lib/liblapack.so.3.0.0 -rw-r--r-- 1 root root 157712 Sep 24 02:51 /usr/lib/liblapack_atlas.a Thanks in advance Nils From pearu at cens.ioc.ee Thu Feb 21 09:37:18 2002 From: pearu at cens.ioc.ee (Pearu Peterson) Date: Thu, 21 Feb 2002 16:37:18 +0200 (EET) Subject: [SciPy-user] Problems during build In-Reply-To: <3C751298.F5B1F87@mecha.uni-stuttgart.de> Message-ID: On Thu, 21 Feb 2002, Nils Wagner wrote: > ValueError: Atlas libraries not found. Either install them in > /usr/lib/atlas or /usr/local/lib/atlas and re > try setup.py, or edit setup.py to specify your own blas and lapack > directories and libs > > How can I specify the blas/lapack/atlas directories and libs in setup.py > ? Currently you have to edit /scipy_distutils/atlas_info.py file where in the header write: library_path = ['/usr/lib'] You may also want to check out what follows in that file in order to make other adjustments for your atlas setup. Btw, you'll need also f77blas and cblas libraries. I didn't notice these ones from your library listing. Pearu From nwagner at mecha.uni-stuttgart.de Thu Feb 21 10:55:04 2002 From: nwagner at mecha.uni-stuttgart.de (Nils Wagner) Date: Thu, 21 Feb 2002 16:55:04 +0100 Subject: [SciPy-user] Problems during build References: Message-ID: <3C751858.3411072@mecha.uni-stuttgart.de> Pearu Peterson schrieb: > > On Thu, 21 Feb 2002, Nils Wagner wrote: > > > ValueError: Atlas libraries not found. Either install them in > > /usr/lib/atlas or /usr/local/lib/atlas and re > > try setup.py, or edit setup.py to specify your own blas and lapack > > directories and libs > > > > How can I specify the blas/lapack/atlas directories and libs in setup.py > > ? > > Currently you have to edit /scipy_distutils/atlas_info.py file where in > the header write: > library_path = ['/usr/lib'] > > You may also want to check out what follows in that file in order to make > other adjustments for your atlas setup. > > Btw, you'll need also > f77blas and cblas > libraries. I didn't notice these ones from your library listing. > > Pearu > > _______________________________________________ > SciPy-user mailing list > SciPy-user at scipy.net > http://www.scipy.net/mailman/listinfo/scipy-user Hi Pearu, f77blas and cblas are in /usr/lib as well. wagner at lisa:~/mysoftware/scipy> ls -l /usr/lib/libcb* -rw-r--r-- 1 root root 268504 Sep 24 02:51 /usr/lib/libcblas.a wagner at lisa:~/mysoftware/scipy> ls -l /usr/lib/libf77* -rw-r--r-- 1 root root 343712 Sep 24 02:51 /usr/lib/libf77blas.a wagner at lisa:~/mysoftware/scipy> Is this o.k. ? #import scipy_distutils.atlas_info #scipy_distutils.atlas_info.library_path = ['/usr/lib'] Nils From pearu at cens.ioc.ee Thu Feb 21 10:00:08 2002 From: pearu at cens.ioc.ee (Pearu Peterson) Date: Thu, 21 Feb 2002 17:00:08 +0200 (EET) Subject: [SciPy-user] Problems during build In-Reply-To: <3C751858.3411072@mecha.uni-stuttgart.de> Message-ID: On Thu, 21 Feb 2002, Nils Wagner wrote: > Hi Pearu, > > f77blas and cblas are in /usr/lib as well. > > wagner at lisa:~/mysoftware/scipy> ls -l /usr/lib/libcb* > -rw-r--r-- 1 root root 268504 Sep 24 02:51 > /usr/lib/libcblas.a > wagner at lisa:~/mysoftware/scipy> ls -l /usr/lib/libf77* > -rw-r--r-- 1 root root 343712 Sep 24 02:51 > /usr/lib/libf77blas.a > wagner at lisa:~/mysoftware/scipy> > > Is this o.k. ? I certainly hope so ;-) You can always give a try. > #import scipy_distutils.atlas_info > #scipy_distutils.atlas_info.library_path = ['/usr/lib'] If the above is the first thing in the setup.py file, then it should be ok (never tried that though). However note that the way how scipy_distutils finds system specific libraries now is going to be changed (hopefully to a better one). This work is inprogress and I hope to finish it by the end of this week. Pearu From nwagner at carol.mecha.uni-stuttgart.de Thu Feb 21 11:12:40 2002 From: nwagner at carol.mecha.uni-stuttgart.de (Nils Wagner) Date: Thu, 21 Feb 2002 17:12:40 +0100 Subject: [SciPy-user] Problems during build References: Message-ID: <3C751C78.74EDC421@mecha.uni-stuttgart.de> Pearu Peterson schrieb: > > On Thu, 21 Feb 2002, Nils Wagner wrote: > > > Hi Pearu, > > > > f77blas and cblas are in /usr/lib as well. > > > > wagner at lisa:~/mysoftware/scipy> ls -l /usr/lib/libcb* > > -rw-r--r-- 1 root root 268504 Sep 24 02:51 > > /usr/lib/libcblas.a > > wagner at lisa:~/mysoftware/scipy> ls -l /usr/lib/libf77* > > -rw-r--r-- 1 root root 343712 Sep 24 02:51 > > /usr/lib/libf77blas.a > > wagner at lisa:~/mysoftware/scipy> > > > > Is this o.k. ? > > I certainly hope so ;-) You can always give a try. > > > #import scipy_distutils.atlas_info > > #scipy_distutils.atlas_info.library_path = ['/usr/lib'] > > If the above is the first thing in the setup.py file, then it should be > ok (never tried that though). However note that the way how > scipy_distutils finds system specific libraries now is going to > be changed (hopefully to a better one). This work is inprogress and I hope > to finish it by the end of this week. > > Pearu > > _______________________________________________ > SciPy-user mailing list > SciPy-user at scipy.net > http://www.scipy.net/mailman/listinfo/scipy-user Attached is the current version of my setup.py. Plese let me know if any modification is necessary. Nils -------------- next part -------------- #!/usr/bin/env python # # ??? 1 # ??? open questions -- how to get docs installed? # ??? build every modules ref docs with happydoc and then bundle them all # ??? into one directory? This seems to make sense to me. # ??? # ??? As far as the user-manual docs, I have no clue how we'll do this at the # ??? moment. I think for the beginning I'll have a canned html file built # ??? from a Word document while we wait for the LaTeX framework to be built # ??? # ??? 2 # ??? There are lines like this throughout: # ??? # ??? packages.append('scipy.tests') # ??? # ??? because I don't know how else to force the tests files to be installed. # ??? I'm pretty sure I want them installed in standard distributions. # ??? They offer examples for people to look at and proof that the installation # ??? at least passes the unittest smoke test. import os,sys from scipy_version import scipy_version from scipy_distutils.misc_util import merge_config_dicts from scipy_distutils.core import setup # force g77 for now from scipy_distutils.command import build_flib build_flib.all_compilers = [build_flib.gnu_fortran_compiler] # globals. I know. I know... parent_package = 'scipy' #['Numerical/Include'] #import scipy_distutils.atlas_info #scipy_distutils.atlas_info.library_path = ['/usr/lib'] build_fftw = 1 if sys.platform == 'win32': from scipy_distutils.mingw32_support import * def get_package_config(name,parent=''): sys.path.insert(0,name) try: mod = __import__('setup_'+name) config = mod.configuration(parent) finally: del sys.path[0] return config def install_package(): # 1. Install the scipy_distutils package so that this and all other setup.py # scripts can use it. #import scipy_distutils.setup #scipy_distutils.setup.install_package() from scipy_distutils.misc_util import add_local_to_path from scipy_distutils.misc_util import restore_path, get_path # add the scipy directory to the PYTHONPATH # needed so that f2py2e and other packages are available to sub-packages add_local_to_path(__name__) # make sure we're in the scipy directory when we run this path = get_path(__name__) old_path = os.getcwd() os.chdir(path) # Don't think I need this anymore -- added to separate_packages # install the testing module as a stand alone module. #sys.path.append('scipy_test') #import setup_scipy_test #setup_scipy_test.install_package() #del sys.path[0] from scipy_distutils.misc_util import get_path # make sure we're in the scipy directory when we run this path = get_path(__name__) old_path = os.getcwd() os.chdir(path) try: config = [] import setup_scipy config.append(setup_scipy.configuration()) # standard modules standard_packages = ['io','linalg','special','signal','stats', 'interpolate','integrate','optimize', 'cluster','cow','ga','weave'] if build_fftw: standard_packages.append('fftw') config.extend([get_package_config(x,parent_package)for x in standard_packages]) # graphices modules graphics_packages = ['plt','gplt'] config.extend([get_package_config(x,parent_package)for x in graphics_packages]) # unix modules # only add these if we are not on windows unix_packages = ['xplt'] if sys.platform != 'win32': config.extend([get_package_config(x,parent_package)for x in unix_packages]) # these packages aren't nested under scipy separate_packages = ['gui_thread','scipy_test','scipy_distutils', 'fastumath'] config.extend([get_package_config(x,'')for x in separate_packages]) config_dict = merge_config_dicts(config) print 'SciPy Version',scipy_version setup (name = "SciPy", version = scipy_version, maintainer = "SciPy Developers", maintainer_email = "scipy-dev at scipy.org", description = "Scientific Algorithms Library for Python", url = "http://www.scipy.org", **config_dict ) finally: # restore old path restore_path() os.chdir(old_path) if __name__ == '__main__': install_package() From nadavh at envision.co.il Thu Feb 21 10:16:34 2002 From: nadavh at envision.co.il (Nadav Horesh) Date: Thu, 21 Feb 2002 17:16:34 +0200 Subject: [SciPy-user] Re: Eror estimation in leastsq Message-ID: <3C750F52.6000402@envision.co.il> The problem is indeed nonlinear (iterative) least-squares fit. I looked at two places: 1. Gnuplot: There estimation is given by: err[i] = sqrt(covariance[i][i] * chi_sqr) / (free_degrees_of_freedom). 2. Numerical recipes book (chapter 15.6): Highly recommends against the above method for nonlinear fittings, and suggests more elaborate methods such as bootstrapping. Trying to be not to religious, I read minpack documentation looking a way to implement the Gnuplot's attitude, but I could not understand how to get the covariance matrix. Nadav. From pearu at cens.ioc.ee Thu Feb 21 10:18:23 2002 From: pearu at cens.ioc.ee (Pearu Peterson) Date: Thu, 21 Feb 2002 17:18:23 +0200 (EET) Subject: [SciPy-user] Problems during build In-Reply-To: <3C751C78.74EDC421@mecha.uni-stuttgart.de> Message-ID: On Thu, 21 Feb 2002, Nils Wagner wrote: > Attached is the current version of my setup.py. > Plese let me know if any modification is necessary. First you should uncomment the lines #import scipy_distutils.atlas_info #scipy_distutils.atlas_info.library_path = ['/usr/lib'] and second I have no idea if the build will be succesful. It depends on many factors (eg. if X11 is found, if fftw is installed properly, if it is a good day for building scipy (ref. possibly unstable states of scipy CVS) etc.). Why don't you just try python setup.py build and see what happens? Pearu From nwagner at carol.mecha.uni-stuttgart.de Thu Feb 21 11:32:22 2002 From: nwagner at carol.mecha.uni-stuttgart.de (Nils Wagner) Date: Thu, 21 Feb 2002 17:32:22 +0100 Subject: [SciPy-user] Problems during build References: Message-ID: <3C752116.F423C82D@mecha.uni-stuttgart.de> Pearu Peterson schrieb: > > On Thu, 21 Feb 2002, Nils Wagner wrote: > > > Attached is the current version of my setup.py. > > Plese let me know if any modification is necessary. > > First you should uncomment the lines > > #import scipy_distutils.atlas_info > #scipy_distutils.atlas_info.library_path = ['/usr/lib'] > > and second I have no idea if the build will be succesful. It depends > on many factors (eg. if X11 is found, if fftw is installed properly, if > it is a good day for building scipy (ref. possibly unstable states of > scipy CVS) etc.). > > Why don't you just try > > python setup.py build > > and see what happens? > here it is ... running run_f2py Traceback (most recent call last): File "setup.py", line 127, in ? install_package() File "setup.py", line 117, in install_package url = "http://www.scipy.org", File "scipy_distutils/core.py", line 43, in setup return old_setup(**new_attr) File "/var/tmp/python-root//usr/lib/python2.1/distutils/core.py", line 138, in setup File "/var/tmp/python-root//usr/lib/python2.1/distutils/dist.py", line 899, in run_commands File "/var/tmp/python-root//usr/lib/python2.1/distutils/dist.py", line 919, in run_command File "/var/tmp/python-root//usr/lib/python2.1/distutils/command/build.py", line 106, in run File "/usr/lib/python2.1/cmd.py", line 328, in run_command print "\n" File "/var/tmp/python-root//usr/lib/python2.1/distutils/dist.py", line 919, in run_command File "scipy_distutils/command/run_f2py.py", line 65, in run ext.sources = self.f2py_sources(ext.sources,ext) File "scipy_distutils/command/run_f2py.py", line 105, in f2py_sources for line in f.xreadlines(): ImportError: No module named xreadlines Nils > Pearu > > _______________________________________________ > SciPy-user mailing list > SciPy-user at scipy.net > http://www.scipy.net/mailman/listinfo/scipy-user From pearu at cens.ioc.ee Thu Feb 21 10:38:39 2002 From: pearu at cens.ioc.ee (Pearu Peterson) Date: Thu, 21 Feb 2002 17:38:39 +0200 (EET) Subject: [SciPy-user] Problems during build In-Reply-To: <3C752116.F423C82D@mecha.uni-stuttgart.de> Message-ID: On Thu, 21 Feb 2002, Nils Wagner wrote: > here it is ... > > running run_f2py > Traceback (most recent call last): > File "setup.py", line 127, in ? > install_package() > File "scipy_distutils/command/run_f2py.py", line 105, in f2py_sources > for line in f.xreadlines(): > ImportError: No module named xreadlines This is the strangest failure that I have seen these days. What Python are you using and how it was installed? Is there a reason why xreadlines module is missing in your Python installation? Does anyone know whether this is a scipy_distutils bug (that it uses xreadlines) or the corresponing Python installation is broken? Pearu From nwagner at carol.mecha.uni-stuttgart.de Thu Feb 21 12:02:53 2002 From: nwagner at carol.mecha.uni-stuttgart.de (Nils Wagner) Date: Thu, 21 Feb 2002 18:02:53 +0100 Subject: [SciPy-user] Problems during build References: Message-ID: <3C75283D.B093B3A4@mecha.uni-stuttgart.de> Pearu Peterson schrieb: > > On Thu, 21 Feb 2002, Nils Wagner wrote: > > > here it is ... > > > > running run_f2py > > Traceback (most recent call last): > > File "setup.py", line 127, in ? > > install_package() > > > File "scipy_distutils/command/run_f2py.py", line 105, in f2py_sources > > for line in f.xreadlines(): > > ImportError: No module named xreadlines > > This is the strangest failure that I have seen these days. > What Python are you using and how it was installed? I tried to build it on SuSE 7.3. for further details http://www.suse.com/index_us.html wagner at lisa:~> python Python 2.1.1 (#1, Sep 24 2001, 05:28:47) [GCC 2.95.3 20010315 (SuSE)] on linux2 Type "copyright", "credits" or "license" for more information. >>> Nils > Is there a reason why xreadlines module is missing in your Python > installation? > > Does anyone know whether this is a scipy_distutils bug (that it uses > xreadlines) or the corresponing Python installation is broken? > > Pearu > > _______________________________________________ > SciPy-user mailing list > SciPy-user at scipy.net > http://www.scipy.net/mailman/listinfo/scipy-user From nwagner at carol.mecha.uni-stuttgart.de Thu Feb 21 12:18:21 2002 From: nwagner at carol.mecha.uni-stuttgart.de (Nils Wagner) Date: Thu, 21 Feb 2002 18:18:21 +0100 Subject: [SciPy-user] xreadlines.so Message-ID: <3C752BDD.8B90FEE2@mecha.uni-stuttgart.de> Hi Pearu, xreadlines.so is within in the directory /usr/lib/apache/lib/python2.1/lib-dynload> ls -l xreadlines.so -rwxr-xr-x 1 root root 7509 Sep 24 07:30 xreadlines.so Nils From pearu at cens.ioc.ee Thu Feb 21 11:17:43 2002 From: pearu at cens.ioc.ee (Pearu Peterson) Date: Thu, 21 Feb 2002 18:17:43 +0200 (EET) Subject: [SciPy-user] xreadlines.so In-Reply-To: <3C752BDD.8B90FEE2@mecha.uni-stuttgart.de> Message-ID: On Thu, 21 Feb 2002, Nils Wagner wrote: > xreadlines.so is within in the directory > > /usr/lib/apache/lib/python2.1/lib-dynload> ls -l xreadlines.so > -rwxr-xr-x 1 root root 7509 Sep 24 07:30 xreadlines.so Can you import it? Eg. $ python Python 2.1.2 (#1, Jan 18 2002, 18:05:45) [GCC 2.95.4 (Debian prerelease)] on linux2 Type "copyright", "credits" or "license" for more information. >>> import xreadlines >>> Pearu From pearu at cens.ioc.ee Thu Feb 21 11:17:43 2002 From: pearu at cens.ioc.ee (Pearu Peterson) Date: Thu, 21 Feb 2002 18:17:43 +0200 (EET) Subject: [SciPy-user] xreadlines.so In-Reply-To: <3C752BDD.8B90FEE2@mecha.uni-stuttgart.de> Message-ID: On Thu, 21 Feb 2002, Nils Wagner wrote: > xreadlines.so is within in the directory > > /usr/lib/apache/lib/python2.1/lib-dynload> ls -l xreadlines.so > -rwxr-xr-x 1 root root 7509 Sep 24 07:30 xreadlines.so Can you import it? Eg. $ python Python 2.1.2 (#1, Jan 18 2002, 18:05:45) [GCC 2.95.4 (Debian prerelease)] on linux2 Type "copyright", "credits" or "license" for more information. >>> import xreadlines >>> Pearu From nwagner at carol.mecha.uni-stuttgart.de Thu Feb 21 12:25:52 2002 From: nwagner at carol.mecha.uni-stuttgart.de (Nils Wagner) Date: Thu, 21 Feb 2002 18:25:52 +0100 Subject: [SciPy-user] xreadlines.so References: Message-ID: <3C752DA0.24CADBCD@mecha.uni-stuttgart.de> Pearu Peterson schrieb: > > On Thu, 21 Feb 2002, Nils Wagner wrote: > > > xreadlines.so is within in the directory > > > > /usr/lib/apache/lib/python2.1/lib-dynload> ls -l xreadlines.so > > -rwxr-xr-x 1 root root 7509 Sep 24 07:30 xreadlines.so > > Can you import it? Eg. > $ python > Python 2.1.2 (#1, Jan 18 2002, 18:05:45) > [GCC 2.95.4 (Debian prerelease)] on linux2 > Type "copyright", "credits" or "license" for more information. > >>> import xreadlines > >>> > > Pearu > > _______________________________________________ > SciPy-user mailing list > SciPy-user at scipy.net > http://www.scipy.net/mailman/listinfo/scipy-user I am sorry I can't do it. Python 2.1.1 (#1, Sep 24 2001, 05:28:47) [GCC 2.95.3 20010315 (SuSE)] on linux2 Type "copyright", "credits" or "license" for more information. >>> import xreadlines Traceback (most recent call last): File "", line 1, in ? ImportError: No module named xreadlines >>> Nils From pearu at cens.ioc.ee Thu Feb 21 11:28:53 2002 From: pearu at cens.ioc.ee (Pearu Peterson) Date: Thu, 21 Feb 2002 18:28:53 +0200 (EET) Subject: [SciPy-user] xreadlines.so In-Reply-To: <3C752DA0.24CADBCD@mecha.uni-stuttgart.de> Message-ID: On Thu, 21 Feb 2002, Nils Wagner wrote: > Pearu Peterson schrieb: > > > > On Thu, 21 Feb 2002, Nils Wagner wrote: > > > > > xreadlines.so is within in the directory > > > > > > /usr/lib/apache/lib/python2.1/lib-dynload> ls -l xreadlines.so ^^^^^^ This xreadlines is inside apache but you are using python from /var/tmp/python-root//usr/lib/python2.1/distutils Please fix your python before trying to build scipy. Failures, that you report in building scipy, seem not be releated to scipy. Pearu From oliphant.travis at ieee.org Thu Feb 21 12:50:56 2002 From: oliphant.travis at ieee.org (Travis Oliphant) Date: Thu, 21 Feb 2002 10:50:56 -0700 Subject: [SciPy-user] Re: Eror estimation in leastsq In-Reply-To: <3C750F52.6000402@envision.co.il> References: <3C750F52.6000402@envision.co.il> Message-ID: On Thursday 21 February 2002 08:16 am, you wrote: > The problem is indeed nonlinear (iterative) least-squares fit. I looked > at two places: > > 1. Gnuplot: There estimation is given by: err[i] = > sqrt(covariance[i][i] * chi_sqr) / (free_degrees_of_freedom). > 2. Numerical recipes book (chapter 15.6): Highly recommends against > the above method for nonlinear fittings, and suggests more > elaborate methods such as bootstrapping. > > Trying to be not to religious, I read minpack documentation looking a > way to implement the Gnuplot's attitude, but I could not understand how > to get the covariance matrix. I suspect that it could be obtained from the variables in infodict (use full_output=1) (see help(optimize.minpack)). If you figure out how to get them from the information in there let us know so we can add it. -Travis From andrew.henshaw at mail.com Fri Feb 22 21:26:00 2002 From: andrew.henshaw at mail.com (Andrew M. Henshaw) Date: Fri, 22 Feb 2002 21:26:00 -0500 Subject: [SciPy-user] Flicker image plotting Message-ID: <001b01c1bc11$6fe43fe0$0100a8c0@ty> Is there some mechanism for double-buffering image plots so that they don't flicker when a series of image arrays is generated? From bryan.cole at teraview.co.uk Sat Feb 23 09:54:44 2002 From: bryan.cole at teraview.co.uk (bryan cole) Date: 23 Feb 2002 14:54:44 +0000 Subject: [SciPy-user] Flicker image plotting In-Reply-To: <001b01c1bc11$6fe43fe0$0100a8c0@ty> References: <001b01c1bc11$6fe43fe0$0100a8c0@ty> Message-ID: <1014476090.2809.10.camel@bryan.teraviewhq.local> I'm not sure how much effort would be to fix this in SciPy as I don't use SciPy much, but I have modified the wxPlotCanvas which comes with wxPython, to be flicker-free (plus a few other mods). I use this for real-time data display (line plots and colour-maps). You simply draw everything to a an image held in memory (a MemoryDC) then copy this to the screen (with the Blit function - very fast). Bryan On Sat, 2002-02-23 at 02:26, Andrew M. Henshaw wrote: > Is there some mechanism for double-buffering image plots so that they don't flicker when a series of image arrays is generated? > _______________________________________________ > SciPy-user mailing list > SciPy-user at scipy.net > http://www.scipy.net/mailman/listinfo/scipy-user -- Bryan Cole Teraview Ltd., 302-304 Cambridge Science Park, Milton Road, Cambridge CB4 0WG, United Kingdom. tel: +44 (1223) 435380 / 435386 (direct-dial) fax: +44 (1223) 435382 From eric at scipy.org Sat Feb 23 13:29:28 2002 From: eric at scipy.org (eric) Date: Sat, 23 Feb 2002 13:29:28 -0500 Subject: [SciPy-user] Flicker image plotting References: <001b01c1bc11$6fe43fe0$0100a8c0@ty> <1014476090.2809.10.camel@bryan.teraviewhq.local> Message-ID: <089501c1bc98$07960e50$de5bfea9@ericlaptop> Hey Andrew, I haven't looked at this, but bet you could use the trick Bryan suggests. However, wxplt started with real-time plotting in mind, but that quickly fell by the way side. As such, I doubt the frame rate will be so hot. Still, it might be worth trying because Bryan's suggestion isn't that hard to implement. If someone ends up with a patch, send it my way and I'll take a look at adding it to the distribution. see ya, eric > I'm not sure how much effort would be to fix this in SciPy as I don't > use SciPy much, but I have modified the wxPlotCanvas which comes with > wxPython, to be flicker-free (plus a few other mods). I use this for > real-time data display (line plots and colour-maps). > > You simply draw everything to a an image held in memory (a MemoryDC) > then copy this to the screen (with the Blit function - very fast). > > Bryan > > > > On Sat, 2002-02-23 at 02:26, Andrew M. Henshaw wrote: > > Is there some mechanism for double-buffering image plots so that they don't flicker when a series of image arrays is generated? > > _______________________________________________ > > SciPy-user mailing list > > SciPy-user at scipy.net > > http://www.scipy.net/mailman/listinfo/scipy-user > -- > Bryan Cole > Teraview Ltd., 302-304 Cambridge Science Park, Milton Road, Cambridge > CB4 0WG, United Kingdom. > tel: +44 (1223) 435380 / 435386 (direct-dial) fax: +44 (1223) 435382 > > _______________________________________________ > SciPy-user mailing list > SciPy-user at scipy.net > http://www.scipy.net/mailman/listinfo/scipy-user > From andrew.henshaw at mail.com Sat Feb 23 16:01:05 2002 From: andrew.henshaw at mail.com (Andrew M. Henshaw) Date: Sat, 23 Feb 2002 16:01:05 -0500 Subject: [SciPy-user] Flicker image plotting In-Reply-To: <089501c1bc98$07960e50$de5bfea9@ericlaptop> Message-ID: <001c01c1bcad$36125620$0100a8c0@ty> I'll look at this. I've done this before with a few Win32-based programs, but I didn't want to go off on a wild goose chase, if it was already implemented. Thanks to both of you. Andy -----Original Message----- From: scipy-user-admin at scipy.org [mailto:scipy-user-admin at scipy.org]On Behalf Of eric Sent: Saturday, February 23, 2002 1:29 PM To: scipy-user at scipy.org Subject: Re: [SciPy-user] Flicker image plotting Hey Andrew, I haven't looked at this, but bet you could use the trick Bryan suggests. However, wxplt started with real-time plotting in mind, but that quickly fell by the way side. As such, I doubt the frame rate will be so hot. Still, it might be worth trying because Bryan's suggestion isn't that hard to implement. If someone ends up with a patch, send it my way and I'll take a look at adding it to the distribution. see ya, eric > I'm not sure how much effort would be to fix this in SciPy as I don't > use SciPy much, but I have modified the wxPlotCanvas which comes with > wxPython, to be flicker-free (plus a few other mods). I use this for > real-time data display (line plots and colour-maps). > > You simply draw everything to a an image held in memory (a MemoryDC) > then copy this to the screen (with the Blit function - very fast). > > Bryan > > > > On Sat, 2002-02-23 at 02:26, Andrew M. Henshaw wrote: > > Is there some mechanism for double-buffering image plots so that they don't flicker when a series of image arrays is generated? > > _______________________________________________ > > SciPy-user mailing list > > SciPy-user at scipy.net > > http://www.scipy.net/mailman/listinfo/scipy-user > -- > Bryan Cole > Teraview Ltd., 302-304 Cambridge Science Park, Milton Road, Cambridge > CB4 0WG, United Kingdom. > tel: +44 (1223) 435380 / 435386 (direct-dial) fax: +44 (1223) 435382 > > _______________________________________________ > SciPy-user mailing list > SciPy-user at scipy.net > http://www.scipy.net/mailman/listinfo/scipy-user > _______________________________________________ SciPy-user mailing list SciPy-user at scipy.net http://www.scipy.net/mailman/listinfo/scipy-user From eric at scipy.org Sat Feb 23 15:22:53 2002 From: eric at scipy.org (eric) Date: Sat, 23 Feb 2002 15:22:53 -0500 Subject: [SciPy-user] Flicker image plotting References: <001c01c1bcad$36125620$0100a8c0@ty> Message-ID: <090001c1bca7$dfd71660$de5bfea9@ericlaptop> I'm guessing you can just change the plot_canvas.draw method around line 403 in scipy/plt/wxplt to use a MemoryDC instead of the client DC. Here is the method. def draw(self,dc=None): #if not len(self.line_list) or len(self.image_list): # return # resize if necessary #print 'draw' t1 = time.clock();self.reset_size(dc);t2 = time.clock() #print 'resize:',t2 - t1 if not dc: dc = wx.wxClientDC(self) # draw titles and axes labels t1 = time.clock() for text in self.all_titles: text.draw(dc) for axis in self.axes: axis.draw_labels(dc) t2 = time.clock() #print 'text:',t2 - t1 self.draw_graph_area(dc) Take note that the plotting effort will resume in a different direction very soon. However, I expect wxplt will still be around for a while, so, if it doesn't take much effort, it is a worthwhile addition. eric ----- Original Message ----- From: "Andrew M. Henshaw" To: Sent: Saturday, February 23, 2002 4:01 PM Subject: RE: [SciPy-user] Flicker image plotting > I'll look at this. I've done this before with a few Win32-based programs, but I didn't want to go off on a wild goose chase, if it was already implemented. > > Thanks to both of you. > > Andy > > -----Original Message----- > From: scipy-user-admin at scipy.org [mailto:scipy-user-admin at scipy.org]On > Behalf Of eric > Sent: Saturday, February 23, 2002 1:29 PM > To: scipy-user at scipy.org > Subject: Re: [SciPy-user] Flicker image plotting > > > Hey Andrew, > > I haven't looked at this, but bet you could use the trick Bryan suggests. > However, wxplt started with real-time plotting in mind, but that quickly fell by > the way side. As such, I doubt the frame rate will be so hot. > > Still, it might be worth trying because Bryan's suggestion isn't that hard to > implement. If someone ends up with a patch, send it my way and I'll take a look > at adding it to the distribution. > > see ya, > eric > > > I'm not sure how much effort would be to fix this in SciPy as I don't > > use SciPy much, but I have modified the wxPlotCanvas which comes with > > wxPython, to be flicker-free (plus a few other mods). I use this for > > real-time data display (line plots and colour-maps). > > > > You simply draw everything to a an image held in memory (a MemoryDC) > > then copy this to the screen (with the Blit function - very fast). > > > > Bryan > > > > > > > > On Sat, 2002-02-23 at 02:26, Andrew M. Henshaw wrote: > > > Is there some mechanism for double-buffering image plots so that they don't > flicker when a series of image arrays is generated? > > > _______________________________________________ > > > SciPy-user mailing list > > > SciPy-user at scipy.net > > > http://www.scipy.net/mailman/listinfo/scipy-user > > -- > > Bryan Cole > > Teraview Ltd., 302-304 Cambridge Science Park, Milton Road, Cambridge > > CB4 0WG, United Kingdom. > > tel: +44 (1223) 435380 / 435386 (direct-dial) fax: +44 (1223) 435382 > > > > _______________________________________________ > > SciPy-user mailing list > > SciPy-user at scipy.net > > http://www.scipy.net/mailman/listinfo/scipy-user > > > > > _______________________________________________ > SciPy-user mailing list > SciPy-user at scipy.net > http://www.scipy.net/mailman/listinfo/scipy-user > _______________________________________________ > SciPy-user mailing list > SciPy-user at scipy.net > http://www.scipy.net/mailman/listinfo/scipy-user > From jjl at pobox.com Mon Feb 25 14:12:04 2002 From: jjl at pobox.com (John J. Lee) Date: Mon, 25 Feb 2002 19:12:04 +0000 (GMT) Subject: [SciPy-user] Re: Eror estimation in leastsq In-Reply-To: <3C750F52.6000402@envision.co.il> Message-ID: On Thu, 21 Feb 2002, Nadav Horesh wrote: [...] > Trying to be not to religious, I read minpack documentation looking a > way to implement the Gnuplot's attitude, but I could not understand how > to get the covariance matrix. It's the inverse of what the routine calls the Jacobian and NR calls the Hessian, IIRC. In my limited experience I've found that models that are slightly wrong (probably for uninteresting instrumental reasons) the errors that come out of this are not very useful. YMMV... Bootstrapping is appealingly simple, but as NR says, watch out for the assumptions. I have a routine (not very general IIRC) that does bootstrap using the minpack from scipy, if you can put up with the fat package of mine that it depends on. John From andrew.henshaw at mail.com Tue Feb 26 00:13:11 2002 From: andrew.henshaw at mail.com (Andrew M. Henshaw) Date: Tue, 26 Feb 2002 00:13:11 -0500 Subject: [SciPy-user] Flicker image plotting In-Reply-To: <090001c1bca7$dfd71660$de5bfea9@ericlaptop> Message-ID: <000001c1be84$4a3e4540$0100a8c0@ty> I have modded the wxplt code, as you suggested. There is another source of flicker - the plt.image call eventually invalidates the window with a clear background option set. I've added a "reimage" command that satisfies my requirements; I don't know if this is the best way to handle the issue. However, I have a bit of a side-related problem with the image plotting. I'm testing without my patches, so I don't believe the error is in anything that I've added. I've distilled the problem down to a small bit of code, which follows. ########## import gui_thread from Numeric import * from scipy import plt plt.xaxis('fit') plt.yaxis('fit') numsmp = 32 rgates = 10 A = zeros((numsmp, rgates), 'double') for i in range(0, rgates, 2): A[:, i] = ones(numsmp, 'double') plt.image(A, scale='yes') raw_input() plt.close() Basically, I thought the code would create and display 10 columns of alternating black and white stripes (with length of 32). It seems to display 32 alternating stripes of length 32. If I transpose the array before plotting, then I get the 10 stripes, but they are stair-stepped and don't run strictly east-to-west. Anything wrong with this code? Is this a reproducible problem for anyone else? -----Original Message----- From: scipy-user-admin at scipy.org [mailto:scipy-user-admin at scipy.org]On Behalf Of eric Sent: Saturday, February 23, 2002 3:23 PM To: scipy-user at scipy.org Subject: Re: [SciPy-user] Flicker image plotting I'm guessing you can just change the plot_canvas.draw method around line 403 in scipy/plt/wxplt to use a MemoryDC instead of the client DC. Here is the method. def draw(self,dc=None): #if not len(self.line_list) or len(self.image_list): # return # resize if necessary #print 'draw' t1 = time.clock();self.reset_size(dc);t2 = time.clock() #print 'resize:',t2 - t1 if not dc: dc = wx.wxClientDC(self) # draw titles and axes labels t1 = time.clock() for text in self.all_titles: text.draw(dc) for axis in self.axes: axis.draw_labels(dc) t2 = time.clock() #print 'text:',t2 - t1 self.draw_graph_area(dc) Take note that the plotting effort will resume in a different direction very soon. However, I expect wxplt will still be around for a while, so, if it doesn't take much effort, it is a worthwhile addition. eric ----- Original Message ----- From: "Andrew M. Henshaw" To: Sent: Saturday, February 23, 2002 4:01 PM Subject: RE: [SciPy-user] Flicker image plotting > I'll look at this. I've done this before with a few Win32-based programs, but I didn't want to go off on a wild goose chase, if it was already implemented. > > Thanks to both of you. > > Andy > > -----Original Message----- > From: scipy-user-admin at scipy.org [mailto:scipy-user-admin at scipy.org]On > Behalf Of eric > Sent: Saturday, February 23, 2002 1:29 PM > To: scipy-user at scipy.org > Subject: Re: [SciPy-user] Flicker image plotting > > > Hey Andrew, > > I haven't looked at this, but bet you could use the trick Bryan suggests. > However, wxplt started with real-time plotting in mind, but that quickly fell by > the way side. As such, I doubt the frame rate will be so hot. > > Still, it might be worth trying because Bryan's suggestion isn't that hard to > implement. If someone ends up with a patch, send it my way and I'll take a look > at adding it to the distribution. > > see ya, > eric > > > I'm not sure how much effort would be to fix this in SciPy as I don't > > use SciPy much, but I have modified the wxPlotCanvas which comes with > > wxPython, to be flicker-free (plus a few other mods). I use this for > > real-time data display (line plots and colour-maps). > > > > You simply draw everything to a an image held in memory (a MemoryDC) > > then copy this to the screen (with the Blit function - very fast). > > > > Bryan > > > > > > > > On Sat, 2002-02-23 at 02:26, Andrew M. Henshaw wrote: > > > Is there some mechanism for double-buffering image plots so that they don't > flicker when a series of image arrays is generated? > > > _______________________________________________ > > > SciPy-user mailing list > > > SciPy-user at scipy.net > > > http://www.scipy.net/mailman/listinfo/scipy-user > > -- > > Bryan Cole > > Teraview Ltd., 302-304 Cambridge Science Park, Milton Road, Cambridge > > CB4 0WG, United Kingdom. > > tel: +44 (1223) 435380 / 435386 (direct-dial) fax: +44 (1223) 435382 > > > > _______________________________________________ > > SciPy-user mailing list > > SciPy-user at scipy.net > > http://www.scipy.net/mailman/listinfo/scipy-user > > > > > _______________________________________________ > SciPy-user mailing list > SciPy-user at scipy.net > http://www.scipy.net/mailman/listinfo/scipy-user > _______________________________________________ > SciPy-user mailing list > SciPy-user at scipy.net > http://www.scipy.net/mailman/listinfo/scipy-user > _______________________________________________ SciPy-user mailing list SciPy-user at scipy.net http://www.scipy.net/mailman/listinfo/scipy-user From nwagner at mecha.uni-stuttgart.de Tue Feb 26 08:17:38 2002 From: nwagner at mecha.uni-stuttgart.de (Nils Wagner) Date: Tue, 26 Feb 2002 14:17:38 +0100 Subject: [SciPy-user] Problems during build References: Message-ID: <3C7B8AF2.6E91775@mecha.uni-stuttgart.de> Pearu Peterson schrieb: > > On Thu, 21 Feb 2002, Nils Wagner wrote: > > > here it is ... > > > > running run_f2py > > Traceback (most recent call last): > > File "setup.py", line 127, in ? > > install_package() > > > File "scipy_distutils/command/run_f2py.py", line 105, in f2py_sources > > for line in f.xreadlines(): > > ImportError: No module named xreadlines > > This is the strangest failure that I have seen these days. > What Python are you using and how it was installed? > Is there a reason why xreadlines module is missing in your Python > installation? > > Does anyone know whether this is a scipy_distutils bug (that it uses > xreadlines) or the corresponing Python installation is broken? > > Pearu > > _______________________________________________ > SciPy-user mailing list > SciPy-user at scipy.net > http://www.scipy.net/mailman/listinfo/scipy-user Hi Pearu, It's a bug within SuSE 7.3. Therefore I have used a self-compiled Python, namely Python-2.1.2 ./python Python 2.1.2 (#1, Feb 25 2002, 18:04:21) [GCC 2.95.3 20010315 (SuSE)] on linux2 Type "copyright", "credits" or "license" for more information. >>> import xreadlines >>> Shall I remove the old python, that is Python-2.1.1 (SuSE) before I use make install to install the new Python-2.1.2 ? Shall I use the prefix option ./configure --prefix=/usr to be compatible with SuSE's standard path ? Any help would be appreciated. Nils From pearu at cens.ioc.ee Tue Feb 26 07:31:01 2002 From: pearu at cens.ioc.ee (Pearu Peterson) Date: Tue, 26 Feb 2002 14:31:01 +0200 (EET) Subject: [SciPy-user] Problems during build In-Reply-To: <3C7B8AF2.6E91775@mecha.uni-stuttgart.de> Message-ID: On Tue, 26 Feb 2002, Nils Wagner wrote: > Shall I remove the old python, that is Python-2.1.1 (SuSE) before I use > make install > to install the new Python-2.1.2 ? > Shall I use the prefix option ./configure --prefix=/usr to be compatible > with SuSE's standard path ? > > Any help would be appreciated. You don't need to remove the old python. And I would suggest using ./configure --prefix=/usr/local and later making sure that /usr/local precedes /usr in the PATH environment variable. Pearu From nwagner at mecha.uni-stuttgart.de Tue Feb 26 08:46:56 2002 From: nwagner at mecha.uni-stuttgart.de (Nils Wagner) Date: Tue, 26 Feb 2002 14:46:56 +0100 Subject: [SciPy-user] Problems during build References: Message-ID: <3C7B91D0.2C6C2F0D@mecha.uni-stuttgart.de> Pearu Peterson schrieb: > > On Tue, 26 Feb 2002, Nils Wagner wrote: > > > Shall I remove the old python, that is Python-2.1.1 (SuSE) before I use > > make install > > to install the new Python-2.1.2 ? > > Shall I use the prefix option ./configure --prefix=/usr to be compatible > > with SuSE's standard path ? > > > > Any help would be appreciated. > > You don't need to remove the old python. And I would suggest using > ./configure --prefix=/usr/local > and later making sure that /usr/local precedes /usr in the PATH > environment variable. > > Pearu > > _______________________________________________ > SciPy-user mailing list > SciPy-user at scipy.net > http://www.scipy.net/mailman/listinfo/scipy-user Hi Pearu, I want to make sure if everything is correct. The current output of echo $PATH is /usr/local/bin:/usr/bin:/usr/X11R6/bin:/bin:/usr/lib/java/bin:/usr/games/bin:/usr/games:/opt/gnome/bin:/opt/kde2/bin:.:/opt/gnome/bin Is that o.k. ? If this is not the case, what shall I do ? Nils From pearu at cens.ioc.ee Tue Feb 26 07:44:53 2002 From: pearu at cens.ioc.ee (Pearu Peterson) Date: Tue, 26 Feb 2002 14:44:53 +0200 (EET) Subject: [SciPy-user] Problems during build In-Reply-To: <3C7B91D0.2C6C2F0D@mecha.uni-stuttgart.de> Message-ID: On Tue, 26 Feb 2002, Nils Wagner wrote: > I want to make sure if everything is correct. The current output of echo > $PATH is > /usr/local/bin:/usr/bin:/usr/X11R6/bin:/bin:/usr/lib/java/bin:/usr/games/bin:/usr/games:/opt/gnome/bin:/opt/kde2/bin:.:/opt/gnome/bin > > Is that o.k. ? It is OK. Pearu From nwagner at mecha.uni-stuttgart.de Tue Feb 26 09:43:04 2002 From: nwagner at mecha.uni-stuttgart.de (Nils Wagner) Date: Tue, 26 Feb 2002 15:43:04 +0100 Subject: [SciPy-user] Latest CVS - import scipy fails Message-ID: <3C7B9EF8.CB328547@mecha.uni-stuttgart.de> Hi, Finally I've installed the latest scipy version via cvs. But import scipy fails Python 2.1.2 (#1, Feb 25 2002, 18:04:21) [GCC 2.95.3 20010315 (SuSE)] on linux2 Type "copyright", "credits" or "license" for more information. >>> import scipy /usr/local/lib/python2.1/site-packages/scipy/linalg/clapack.so: undefined symbol: clapack_sgetrf Warning: FFT package not found. Some names will not be available Traceback (most recent call last): File "", line 1, in ? File "/usr/local/lib/python2.1/site-packages/scipy/__init__.py", line 87, in ? names2all(__all__, _level1a, globals()) File "/usr/local/lib/python2.1/site-packages/scipy/__init__.py", line 37, in names2all exec("import %s" % name, gldict) File "", line 1, in ? File "/usr/local/lib/python2.1/site-packages/scipy/basic1a.py", line 10, in ? from scipy.linalg import eig ImportError: cannot import name eig >>> Any idea ? Nils From pearu at cens.ioc.ee Tue Feb 26 10:03:52 2002 From: pearu at cens.ioc.ee (Pearu Peterson) Date: Tue, 26 Feb 2002 17:03:52 +0200 (EET) Subject: [SciPy-user] Latest CVS - import scipy fails In-Reply-To: <3C7B9EF8.CB328547@mecha.uni-stuttgart.de> Message-ID: On Tue, 26 Feb 2002, Nils Wagner wrote: > Hi, > > Finally I've installed the latest scipy version via cvs. > But import scipy fails > > Python 2.1.2 (#1, Feb 25 2002, 18:04:21) > [GCC 2.95.3 20010315 (SuSE)] on linux2 > Type "copyright", "credits" or "license" for more information. > >>> import scipy > /usr/local/lib/python2.1/site-packages/scipy/linalg/clapack.so: > undefined symbol: clapack_sgetrf Could you check your atlas installation? Try to follow the instructions in http://www.scipy.org/Members/fperez/PerezCVSBuild.htm as closely as possible. Pearu From eric at scipy.org Tue Feb 26 14:00:50 2002 From: eric at scipy.org (eric) Date: Tue, 26 Feb 2002 14:00:50 -0500 Subject: [SciPy-user] Flicker image plotting References: <000001c1be84$4a3e4540$0100a8c0@ty> Message-ID: <036b01c1bef7$e9e4a580$6b01a8c0@ericlaptop> Hey Andrew, I think this problem has been addressed. What version of SciPy are you using? I'm not sure it is completely cured, but the example you give does produce the expected 10 vertical alternating black and white stripes in my version of SciPy (from the CVS about 2 days ago). eric ----- Original Message ----- From: "Andrew M. Henshaw" To: Sent: Tuesday, February 26, 2002 12:13 AM Subject: RE: [SciPy-user] Flicker image plotting > I have modded the wxplt code, as you suggested. There is another source of flicker - the plt.image call eventually invalidates the window with a clear background option set. I've added a "reimage" command that satisfies my requirements; I don't know if this is the best way to handle the issue. > > However, I have a bit of a side-related problem with the image plotting. I'm testing without my patches, so I don't believe the error is in anything that I've added. I've distilled the problem down to a small bit of code, which follows. > > ########## > import gui_thread > from Numeric import * > from scipy import plt > > plt.xaxis('fit') > plt.yaxis('fit') > > numsmp = 32 > rgates = 10 > A = zeros((numsmp, rgates), 'double') > > for i in range(0, rgates, 2): > A[:, i] = ones(numsmp, 'double') > plt.image(A, scale='yes') > raw_input() > plt.close() > > Basically, I thought the code would create and display 10 columns of alternating black and white stripes (with length of 32). It seems to display 32 alternating stripes of length 32. If I transpose the array before plotting, then I get the 10 stripes, but they are stair-stepped and don't run strictly east-to-west. Anything wrong with this code? Is this a reproducible problem for anyone else? > > > > -----Original Message----- > From: scipy-user-admin at scipy.org [mailto:scipy-user-admin at scipy.org]On > Behalf Of eric > Sent: Saturday, February 23, 2002 3:23 PM > To: scipy-user at scipy.org > Subject: Re: [SciPy-user] Flicker image plotting > > > I'm guessing you can just change the plot_canvas.draw method around line 403 in > scipy/plt/wxplt to use a MemoryDC instead of the client DC. > > Here is the method. > > def draw(self,dc=None): > #if not len(self.line_list) or len(self.image_list): > # return > # resize if necessary > #print 'draw' > t1 = time.clock();self.reset_size(dc);t2 = time.clock() > #print 'resize:',t2 - t1 > if not dc: dc = wx.wxClientDC(self) > # draw titles and axes labels > t1 = time.clock() > for text in self.all_titles: > text.draw(dc) > for axis in self.axes: > axis.draw_labels(dc) > t2 = time.clock() > #print 'text:',t2 - t1 > self.draw_graph_area(dc) > > Take note that the plotting effort will resume in a different direction very > soon. However, I expect wxplt will still be around for a while, so, if it > doesn't take much effort, it is a worthwhile addition. > > eric > > ----- Original Message ----- > From: "Andrew M. Henshaw" > To: > Sent: Saturday, February 23, 2002 4:01 PM > Subject: RE: [SciPy-user] Flicker image plotting > > > > I'll look at this. I've done this before with a few Win32-based programs, but > I didn't want to go off on a wild goose chase, if it was already implemented. > > > > Thanks to both of you. > > > > Andy > > > > -----Original Message----- > > From: scipy-user-admin at scipy.org [mailto:scipy-user-admin at scipy.org]On > > Behalf Of eric > > Sent: Saturday, February 23, 2002 1:29 PM > > To: scipy-user at scipy.org > > Subject: Re: [SciPy-user] Flicker image plotting > > > > > > Hey Andrew, > > > > I haven't looked at this, but bet you could use the trick Bryan suggests. > > However, wxplt started with real-time plotting in mind, but that quickly fell > by > > the way side. As such, I doubt the frame rate will be so hot. > > > > Still, it might be worth trying because Bryan's suggestion isn't that hard to > > implement. If someone ends up with a patch, send it my way and I'll take a > look > > at adding it to the distribution. > > > > see ya, > > eric > > > > > I'm not sure how much effort would be to fix this in SciPy as I don't > > > use SciPy much, but I have modified the wxPlotCanvas which comes with > > > wxPython, to be flicker-free (plus a few other mods). I use this for > > > real-time data display (line plots and colour-maps). > > > > > > You simply draw everything to a an image held in memory (a MemoryDC) > > > then copy this to the screen (with the Blit function - very fast). > > > > > > Bryan > > > > > > > > > > > > On Sat, 2002-02-23 at 02:26, Andrew M. Henshaw wrote: > > > > Is there some mechanism for double-buffering image plots so that they > don't > > flicker when a series of image arrays is generated? > > > > _______________________________________________ > > > > SciPy-user mailing list > > > > SciPy-user at scipy.net > > > > http://www.scipy.net/mailman/listinfo/scipy-user > > > -- > > > Bryan Cole > > > Teraview Ltd., 302-304 Cambridge Science Park, Milton Road, Cambridge > > > CB4 0WG, United Kingdom. > > > tel: +44 (1223) 435380 / 435386 (direct-dial) fax: +44 (1223) 435382 > > > > > > _______________________________________________ > > > SciPy-user mailing list > > > SciPy-user at scipy.net > > > http://www.scipy.net/mailman/listinfo/scipy-user > > > > > > > > > _______________________________________________ > > SciPy-user mailing list > > SciPy-user at scipy.net > > http://www.scipy.net/mailman/listinfo/scipy-user > > _______________________________________________ > > SciPy-user mailing list > > SciPy-user at scipy.net > > http://www.scipy.net/mailman/listinfo/scipy-user > > > > > _______________________________________________ > SciPy-user mailing list > SciPy-user at scipy.net > http://www.scipy.net/mailman/listinfo/scipy-user > _______________________________________________ > SciPy-user mailing list > SciPy-user at scipy.net > http://www.scipy.net/mailman/listinfo/scipy-user > From andrew.henshaw at mail.com Tue Feb 26 16:35:30 2002 From: andrew.henshaw at mail.com (Andrew Henshaw) Date: Tue, 26 Feb 2002 16:35:30 -0500 Subject: [SciPy-user] Flicker image plotting In-Reply-To: <036b01c1bef7$e9e4a580$6b01a8c0@ericlaptop> Message-ID: I'm using the pre-built Windows version (0.1). I'll get the latest and try that. Thanks again, Andy -----Original Message----- From: scipy-user-admin at scipy.org [mailto:scipy-user-admin at scipy.org]On Behalf Of eric Sent: Tuesday, February 26, 2002 2:01 PM To: scipy-user at scipy.org Subject: Re: [SciPy-user] Flicker image plotting Hey Andrew, I think this problem has been addressed. What version of SciPy are you using? I'm not sure it is completely cured, but the example you give does produce the expected 10 vertical alternating black and white stripes in my version of SciPy (from the CVS about 2 days ago). eric ----- Original Message ----- From: "Andrew M. Henshaw" To: Sent: Tuesday, February 26, 2002 12:13 AM Subject: RE: [SciPy-user] Flicker image plotting > I have modded the wxplt code, as you suggested. There is another source of flicker - the plt.image call eventually invalidates the window with a clear background option set. I've added a "reimage" command that satisfies my requirements; I don't know if this is the best way to handle the issue. > > However, I have a bit of a side-related problem with the image plotting. I'm testing without my patches, so I don't believe the error is in anything that I've added. I've distilled the problem down to a small bit of code, which follows. > > ########## > import gui_thread > from Numeric import * > from scipy import plt > > plt.xaxis('fit') > plt.yaxis('fit') > > numsmp = 32 > rgates = 10 > A = zeros((numsmp, rgates), 'double') > > for i in range(0, rgates, 2): > A[:, i] = ones(numsmp, 'double') > plt.image(A, scale='yes') > raw_input() > plt.close() > > Basically, I thought the code would create and display 10 columns of alternating black and white stripes (with length of 32). It seems to display 32 alternating stripes of length 32. If I transpose the array before plotting, then I get the 10 stripes, but they are stair-stepped and don't run strictly east-to-west. Anything wrong with this code? Is this a reproducible problem for anyone else? > > > > -----Original Message----- > From: scipy-user-admin at scipy.org [mailto:scipy-user-admin at scipy.org]On > Behalf Of eric > Sent: Saturday, February 23, 2002 3:23 PM > To: scipy-user at scipy.org > Subject: Re: [SciPy-user] Flicker image plotting > > > I'm guessing you can just change the plot_canvas.draw method around line 403 in > scipy/plt/wxplt to use a MemoryDC instead of the client DC. > > Here is the method. > > def draw(self,dc=None): > #if not len(self.line_list) or len(self.image_list): > # return > # resize if necessary > #print 'draw' > t1 = time.clock();self.reset_size(dc);t2 = time.clock() > #print 'resize:',t2 - t1 > if not dc: dc = wx.wxClientDC(self) > # draw titles and axes labels > t1 = time.clock() > for text in self.all_titles: > text.draw(dc) > for axis in self.axes: > axis.draw_labels(dc) > t2 = time.clock() > #print 'text:',t2 - t1 > self.draw_graph_area(dc) > > Take note that the plotting effort will resume in a different direction very > soon. However, I expect wxplt will still be around for a while, so, if it > doesn't take much effort, it is a worthwhile addition. > > eric > > ----- Original Message ----- > From: "Andrew M. Henshaw" > To: > Sent: Saturday, February 23, 2002 4:01 PM > Subject: RE: [SciPy-user] Flicker image plotting > > > > I'll look at this. I've done this before with a few Win32-based programs, but > I didn't want to go off on a wild goose chase, if it was already implemented. > > > > Thanks to both of you. > > > > Andy > > > > -----Original Message----- > > From: scipy-user-admin at scipy.org [mailto:scipy-user-admin at scipy.org]On > > Behalf Of eric > > Sent: Saturday, February 23, 2002 1:29 PM > > To: scipy-user at scipy.org > > Subject: Re: [SciPy-user] Flicker image plotting > > > > > > Hey Andrew, > > > > I haven't looked at this, but bet you could use the trick Bryan suggests. > > However, wxplt started with real-time plotting in mind, but that quickly fell > by > > the way side. As such, I doubt the frame rate will be so hot. > > > > Still, it might be worth trying because Bryan's suggestion isn't that hard to > > implement. If someone ends up with a patch, send it my way and I'll take a > look > > at adding it to the distribution. > > > > see ya, > > eric > > > > > I'm not sure how much effort would be to fix this in SciPy as I don't > > > use SciPy much, but I have modified the wxPlotCanvas which comes with > > > wxPython, to be flicker-free (plus a few other mods). I use this for > > > real-time data display (line plots and colour-maps). > > > > > > You simply draw everything to a an image held in memory (a MemoryDC) > > > then copy this to the screen (with the Blit function - very fast). > > > > > > Bryan > > > > > > > > > > > > On Sat, 2002-02-23 at 02:26, Andrew M. Henshaw wrote: > > > > Is there some mechanism for double-buffering image plots so that they > don't > > flicker when a series of image arrays is generated? > > > > _______________________________________________ > > > > SciPy-user mailing list > > > > SciPy-user at scipy.net > > > > http://www.scipy.net/mailman/listinfo/scipy-user > > > -- > > > Bryan Cole > > > Teraview Ltd., 302-304 Cambridge Science Park, Milton Road, Cambridge > > > CB4 0WG, United Kingdom. > > > tel: +44 (1223) 435380 / 435386 (direct-dial) fax: +44 (1223) 435382 > > > > > > _______________________________________________ > > > SciPy-user mailing list > > > SciPy-user at scipy.net > > > http://www.scipy.net/mailman/listinfo/scipy-user > > > > > > > > > _______________________________________________ > > SciPy-user mailing list > > SciPy-user at scipy.net > > http://www.scipy.net/mailman/listinfo/scipy-user > > _______________________________________________ > > SciPy-user mailing list > > SciPy-user at scipy.net > > http://www.scipy.net/mailman/listinfo/scipy-user > > > > > _______________________________________________ > SciPy-user mailing list > SciPy-user at scipy.net > http://www.scipy.net/mailman/listinfo/scipy-user > _______________________________________________ > SciPy-user mailing list > SciPy-user at scipy.net > http://www.scipy.net/mailman/listinfo/scipy-user > _______________________________________________ SciPy-user mailing list SciPy-user at scipy.net http://www.scipy.net/mailman/listinfo/scipy-user From nwagner at mecha.uni-stuttgart.de Wed Feb 27 05:40:17 2002 From: nwagner at mecha.uni-stuttgart.de (Nils Wagner) Date: Wed, 27 Feb 2002 11:40:17 +0100 Subject: [SciPy-user] Latest CVS - import scipy fails --> ATLAS References: Message-ID: <3C7CB791.59D4E43A@mecha.uni-stuttgart.de> Pearu Peterson schrieb: > > On Tue, 26 Feb 2002, Nils Wagner wrote: > > > Hi, > > > > Finally I've installed the latest scipy version via cvs. > > But import scipy fails > > > > Python 2.1.2 (#1, Feb 25 2002, 18:04:21) > > [GCC 2.95.3 20010315 (SuSE)] on linux2 > > Type "copyright", "credits" or "license" for more information. > > >>> import scipy > > /usr/local/lib/python2.1/site-packages/scipy/linalg/clapack.so: > > undefined symbol: clapack_sgetrf > > Could you check your atlas installation? Try to follow the instructions in > > http://www.scipy.org/Members/fperez/PerezCVSBuild.htm > > as closely as possible. > > Pearu > > _______________________________________________ > SciPy-user mailing list > SciPy-user at scipy.net > http://www.scipy.net/mailman/listinfo/scipy-user Hi Pearu, I have already built ATLAS. ~/mysoftware/ATLAS/lib/Linux_ATHLON> ls -l insgesamt 5976 -rw-r--r-- 1 root root 5027632 Feb 27 11:05 libatlas.a -rw-r--r-- 1 root root 272104 Feb 27 11:01 libcblas.a -rw-r--r-- 1 root root 344582 Feb 27 11:05 libf77blas.a -rw-r--r-- 1 root root 157744 Feb 27 11:05 liblapack.a -rw-r--r-- 1 root root 300060 Feb 27 10:09 libtstatlas.a drwxr-xr-x 2 wagner users 3411 Feb 27 11:16 tmp ~/mysoftware/ATLAS/lib/Linux_ATHLON> Now I would like to build one monolithic LAPACK library. For this purpose I followed the first three steps of ../ATLAS/doc/LAPACK.txt. that is in ATLAS/lib/Linux_ATHLON> mkdir tmp cd tmp ar x ../liblapack.a Step 4: mv ../liblapack.a My old LAPACK lib and the old liblapack_atlas.a is located in /usr/lib/ -rw-r--r-- 1 root root 5763150 Sep 24 01:00 liblapack.a lrwxrwxrwx 1 root root 14 Okt 22 19:27 liblapack.so -> liblapack.so.3 lrwxrwxrwx 1 root root 18 Okt 22 19:27 liblapack.so.3 -> liblapack.so.3.0.0 -rwxr-xr-x 1 root root 4826626 Sep 24 01:01 liblapack.so.3.0.0 -rw-r--r-- 1 root root 157712 Sep 24 02:51 liblapack_atlas.a What shall I use for ? What follows is ar r ../liblapack.a *.o cd .. rm -rf tmp Finally the libraries this process created should be copied to /usr/lib. Is that o.k. ? Shall I move the original liblapack.a in /usr/lib to something like liblapack.ori Nils From pearu at cens.ioc.ee Wed Feb 27 04:48:01 2002 From: pearu at cens.ioc.ee (Pearu Peterson) Date: Wed, 27 Feb 2002 11:48:01 +0200 (EET) Subject: [SciPy-user] Latest CVS - import scipy fails --> ATLAS In-Reply-To: <3C7CB791.59D4E43A@mecha.uni-stuttgart.de> Message-ID: On Wed, 27 Feb 2002, Nils Wagner wrote: > What shall I use for ? Use liblapack.a > Finally the libraries this process created should be copied to /usr/lib. > Is that o.k. ? I think if you copy the new liblapack.a to /usr/lib, then building other programs that use lapack, must be linked also against atlas. I am not sure that this will cause trouble in future or not, but you can leave the old liblapack.a in /usr/lib and copy all atlas libraries and the new liblapack.a to /usr/local/atlas. Pearu From nwagner at mecha.uni-stuttgart.de Wed Feb 27 06:19:15 2002 From: nwagner at mecha.uni-stuttgart.de (Nils Wagner) Date: Wed, 27 Feb 2002 12:19:15 +0100 Subject: [SciPy-user] import scipy fails --> ATLAS Message-ID: <3C7CC0B3.3B41A19A@mecha.uni-stuttgart.de> Hi Pearu, I have copied all atlas libraries and the new liblapack.a to /usr/local/atlas ls -l /usr/local/atlas/ insgesamt 11600 -rw-r--r-- 1 root root 5027632 Feb 27 12:12 libatlas.a -rw-r--r-- 1 root root 272104 Feb 27 12:12 libcblas.a -rw-r--r-- 1 root root 344582 Feb 27 12:12 libf77blas.a -rw-r--r-- 1 root root 5920822 Feb 27 12:12 liblapack.a -rw-r--r-- 1 root root 300060 Feb 27 12:12 libtstatlas.a Again import scipy fails Python 2.1.2 (#1, Feb 25 2002, 18:04:21) [GCC 2.95.3 20010315 (SuSE)] on linux2 Type "copyright", "credits" or "license" for more information. >>> import scipy /usr/local/lib/python2.1/site-packages/scipy/linalg/clapack.so: undefined symbol: clapack_sgetrf Warning: FFT package not found. Some names will not be available Traceback (most recent call last): File "", line 1, in ? File "/usr/local/lib/python2.1/site-packages/scipy/__init__.py", line 87, in ? names2all(__all__, _level1a, globals()) File "/usr/local/lib/python2.1/site-packages/scipy/__init__.py", line 37, in names2all exec("import %s" % name, gldict) File "", line 1, in ? File "/usr/local/lib/python2.1/site-packages/scipy/basic1a.py", line 10, in ? from scipy.linalg import eig ImportError: cannot import name eig >>> Any idea ? I guess that the libraries in /usr/lib are preferred. Nils From nwagner at mecha.uni-stuttgart.de Wed Feb 27 06:25:00 2002 From: nwagner at mecha.uni-stuttgart.de (Nils Wagner) Date: Wed, 27 Feb 2002 12:25:00 +0100 Subject: [SciPy-user] ATLAS Message-ID: <3C7CC20C.CCBED806@mecha.uni-stuttgart.de> Hi Pearu, Since I have just installed a new ATLAS library in /usr/local/atlas shall I build scipy again ? Nils From pearu at cens.ioc.ee Wed Feb 27 05:27:42 2002 From: pearu at cens.ioc.ee (Pearu Peterson) Date: Wed, 27 Feb 2002 12:27:42 +0200 (EET) Subject: [SciPy-user] ATLAS In-Reply-To: <3C7CC20C.CCBED806@mecha.uni-stuttgart.de> Message-ID: On Wed, 27 Feb 2002, Nils Wagner wrote: > Since I have just installed a new ATLAS library in /usr/local/atlas > shall I build scipy again ? Yesss. Pearu From nwagner at mecha.uni-stuttgart.de Wed Feb 27 06:48:03 2002 From: nwagner at mecha.uni-stuttgart.de (Nils Wagner) Date: Wed, 27 Feb 2002 12:48:03 +0100 Subject: [SciPy-user] ATLAS References: Message-ID: <3C7CC773.605DCD30@mecha.uni-stuttgart.de> Pearu Peterson schrieb: > > On Wed, 27 Feb 2002, Nils Wagner wrote: > > > Since I have just installed a new ATLAS library in /usr/local/atlas > > shall I build scipy again ? > > Yesss. > > Pearu > > _______________________________________________ > SciPy-user mailing list > SciPy-user at scipy.net > http://www.scipy.net/mailman/listinfo/scipy-user Hi Pearu, I modified my previous setup.py that is import scipy_distutils.atlas_info #scipy_distutils.atlas_info.library_path = ['/usr/lib'] scipy_distutils.atlas_info.library_path = ['/usr/local/atlas'] rm -rf build/ python setup.py build python setup.py install But again import scipy fails Python 2.1.2 (#1, Feb 25 2002, 18:04:21) [GCC 2.95.3 20010315 (SuSE)] on linux2 Type "copyright", "credits" or "license" for more information. >>> import scipy /usr/local/lib/python2.1/site-packages/scipy/linalg/clapack.so: undefined symbol: clapack_sgetrf Warning: FFT package not found. Some names will not be available Traceback (most recent call last): File "", line 1, in ? File "/usr/local/lib/python2.1/site-packages/scipy/__init__.py", line 87, in ? names2all(__all__, _level1a, globals()) File "/usr/local/lib/python2.1/site-packages/scipy/__init__.py", line 37, in names2all exec("import %s" % name, gldict) File "", line 1, in ? File "/usr/local/lib/python2.1/site-packages/scipy/basic1a.py", line 10, in ? from scipy.linalg import eig ImportError: cannot import name eig >>> Any idea ? Nils From pearu at cens.ioc.ee Wed Feb 27 07:18:22 2002 From: pearu at cens.ioc.ee (Pearu Peterson) Date: Wed, 27 Feb 2002 14:18:22 +0200 (EET) Subject: [SciPy-user] ATLAS In-Reply-To: <3C7CC773.605DCD30@mecha.uni-stuttgart.de> Message-ID: On Wed, 27 Feb 2002, Nils Wagner wrote: > I modified my previous setup.py that is > > import scipy_distutils.atlas_info > #scipy_distutils.atlas_info.library_path = ['/usr/lib'] > scipy_distutils.atlas_info.library_path = ['/usr/local/atlas'] > > rm -rf build/ > python setup.py build > python setup.py install > >>> > > Any idea ? Are you using the scipy from CVS? If yes, then modifying scipy_distutils.atlas_info.library_path has no effect (I think). If you run python scipy_distutils/system_info.py then it shows what scipy uses when building the extension modules. > scipy_distutils.atlas_info.library_path = ['/usr/local/atlas'] It was my typo to suggest that atlas libraries should go into /usr/local/atlas The correct path is /usr/local/lib/atlas Could you remove all atlas stuff to this directory and then try again building scipy? If you get still problems, then could you send the full output of 'setup.py build' directly to me (and not to the list)? Pearu From nwagner at mecha.uni-stuttgart.de Wed Feb 27 10:47:27 2002 From: nwagner at mecha.uni-stuttgart.de (Nils Wagner) Date: Wed, 27 Feb 2002 16:47:27 +0100 Subject: [SciPy-user] No module named optimize Message-ID: <3C7CFF8F.FEC7DA0F@mecha.uni-stuttgart.de> Hi all, I can't import optimize that is Python 2.1.2 (#1, Feb 25 2002, 18:04:21) [GCC 2.95.3 20010315 (SuSE)] on linux2 Type "copyright", "credits" or "license" for more information. >>> from scipy import * >>> help(optimize) Optimization Tools A collection of general-purpose optimization routines. fmin -- Nelder-Mead Simplex algorithm (uses only function calls) fmin_bfgs -- Quasi-Newton method (can use function and gradient) fmin_ncg -- Line-search Newton Conjugate Gradient (can use function, gradient and hessian). leastsq -- Minimize the sum of squares of M equations in N unknowns given a starting estimate. fminbound -- Bounded minimization of a scalar function. fsolve -- Non-linear multi-variable equation solver. newton -- Single-variable function root finder. fixed_point -- Single-variable fixed-point solver. >>> from optimize import fsolve Traceback (most recent call last): File "", line 1, in ? ImportError: No module named optimize >>> Any suggestion ? Nils From pearu at cens.ioc.ee Wed Feb 27 09:50:01 2002 From: pearu at cens.ioc.ee (Pearu Peterson) Date: Wed, 27 Feb 2002 16:50:01 +0200 (EET) Subject: [SciPy-user] No module named optimize In-Reply-To: <3C7CFF8F.FEC7DA0F@mecha.uni-stuttgart.de> Message-ID: On Wed, 27 Feb 2002, Nils Wagner wrote: > >>> from optimize import fsolve Either do from scipy.optimize import fsolve or use optimize.fsolve Note that this is basic python issue, not scipy. Pearu From nwagner at mecha.uni-stuttgart.de Wed Feb 27 11:02:27 2002 From: nwagner at mecha.uni-stuttgart.de (Nils Wagner) Date: Wed, 27 Feb 2002 17:02:27 +0100 Subject: [SciPy-user] No module named optimize References: Message-ID: <3C7D0313.3E83E2D4@mecha.uni-stuttgart.de> Pearu Peterson schrieb: > > On Wed, 27 Feb 2002, Nils Wagner wrote: > > > >>> from optimize import fsolve > > Either do > from scipy.optimize import fsolve > or use > optimize.fsolve > > Note that this is basic python issue, not scipy. > > Pearu > > _______________________________________________ > SciPy-user mailing list > SciPy-user at scipy.net > http://www.scipy.net/mailman/listinfo/scipy-user Sorry for that - I have used it directly from tutorial.pdf page 14. BTW, is there any progress in writing an extended tutorial for scipy ? From jjl at pobox.com Wed Feb 27 18:17:41 2002 From: jjl at pobox.com (John J. Lee) Date: Wed, 27 Feb 2002 23:17:41 +0000 (GMT) Subject: [SciPy-user] optimize.minpack.error? Message-ID: The minpack.error Exception seems to be impossible to get at (scipy 0.1, but latest CVS minpackmodule.c seems similar at least). Something to do with this, in minpackmodule.c, I guess: minpack_error = PyErr_NewException ("minpack.error", NULL, NULL); but I couldn't figure out from Python C API manual exactly what this function is supposed to do. John From andrew.henshaw at mail.com Wed Feb 27 23:07:56 2002 From: andrew.henshaw at mail.com (Andrew M. Henshaw) Date: Wed, 27 Feb 2002 23:07:56 -0500 Subject: [SciPy-user] build problem -- flapackmodule.c Message-ID: <000101c1c00d$82026f30$0100a8c0@ty> I'm trying to build from CVS (2/27/02) and the build is bombing in flapackmodule.c with several of these kinds of messages: build\temp.win32-2.1\flapackmodule.c: In function `f2py_rout_flapack_cgesdd': build\temp.win32-2.1\flapackmodule.c:9648: `rwork_capi' undeclared (first use in this function) build\temp.win32-2.1\flapackmodule.c:9658: `iwork_capi' undeclared (first use in this function) I'm using F2PY-2.13.175-1212 with Python 2.1 (ActivePython 2.1 build 211) and mingw32 gcc-2.95.2. Any thoughts? Andy From pearu at cens.ioc.ee Thu Feb 28 02:05:39 2002 From: pearu at cens.ioc.ee (Pearu Peterson) Date: Thu, 28 Feb 2002 09:05:39 +0200 (EET) Subject: [SciPy-user] build problem -- flapackmodule.c In-Reply-To: <000101c1c00d$82026f30$0100a8c0@ty> Message-ID: Hi, On Wed, 27 Feb 2002, Andrew M. Henshaw wrote: > I'm trying to build from CVS (2/27/02) and the build is bombing in flapackmodule.c with several of these kinds of messages: > > build\temp.win32-2.1\flapackmodule.c: In function `f2py_rout_flapack_cgesdd': > build\temp.win32-2.1\flapackmodule.c:9648: `rwork_capi' undeclared (first use in this function) > build\temp.win32-2.1\flapackmodule.c:9658: `iwork_capi' undeclared (first use in this function) > > I'm using F2PY-2.13.175-1212 with Python 2.1 (ActivePython 2.1 build 211) and mingw32 gcc-2.95.2. > > Any thoughts? It could be mingw32 issue with scipy_distutils/system_info.py. What is the output of python scipy_distutils/system_info.py ? You may need to add some prefixes to the code fragment if sys.platform == 'win32': prefixes = ['C:\\'] Tell us how it worked. If the above was not helpful, then send me the full output of cd linalg && setup_linalg.py build Pearu From pearu at cens.ioc.ee Thu Feb 28 02:17:03 2002 From: pearu at cens.ioc.ee (Pearu Peterson) Date: Thu, 28 Feb 2002 09:17:03 +0200 (EET) Subject: [SciPy-user] optimize.minpack.error? In-Reply-To: Message-ID: On Wed, 27 Feb 2002, John J. Lee wrote: > > The minpack.error Exception seems to be impossible to get at (scipy 0.1, > but latest CVS minpackmodule.c seems similar at least). > > Something to do with this, in minpackmodule.c, I guess: > > minpack_error = PyErr_NewException ("minpack.error", NULL, NULL); > > > but I couldn't figure out from Python C API manual exactly what this > function is supposed to do. It is described here http://www.python.org/doc/current/api/exceptionHandling.html#l2h-95 But I don't understand what you are getting at. Do you want minpack.error Exception arised in your app or what? Note that calling minpack functions may fail with exceptions, not just minpack.error. Pearu From nwagner at mecha.uni-stuttgart.de Thu Feb 28 05:20:59 2002 From: nwagner at mecha.uni-stuttgart.de (Nils Wagner) Date: Thu, 28 Feb 2002 11:20:59 +0100 Subject: [SciPy-user] gist.error: Gist GdLines plotter failed Message-ID: <3C7E048B.DD936049@mecha.uni-stuttgart.de> Hi all, I have used xplt to plot some functions. Python 2.1.2 (#1, Feb 25 2002, 18:04:21) [GCC 2.95.3 20010315 (SuSE)] on linux2 Type "copyright", "credits" or "license" for more information. >>> from scipy.xplt import * >>> from scipy import * >>> x = arange(0,10,0.2) >>> print x [ 0. 0.2 0.4 0.6 0.8 1. 1.2 1.4 1.6 1.8 2. 2.2 2.4 2.6 2.8 3. 3.2 3.4 3.6 3.8 4. 4.2 4.4 4.6 4.8 5. 5.2 5.4 5.6 5.8 6. 6.2 6.4 6.6 6.8 7. 7.2 7.4 7.6 7.8 8. 8.2 8.4 8.6 8.8 9. 9.2 9.4 9.6 9.8] >>> plot (x,sin(x)) >>> plot (x,sin(0.1*x)) >>> y = arange(0,10,0.1) >>> plot (y,sin(0.1*y)) >>> plot (y,sin(y)) >>> y = arange(0,10,0.1) >>> plot (y,sin(y)) If I close the window (containing the graph) and run the command >>> plot (y,sin(y)) again, I get Traceback (most recent call last): File "", line 1, in ? File "/usr/local/lib/python2.1/site-packages/scipy/xplt/Mplot.py", line 470, in plot gist.plg(y,x,type=thetype,color=thecolor,marker=themarker,marks=tomark) gist.error: Gist GdLines plotter failed Any idea ? Nils From jh at comunit.de Thu Feb 28 04:42:52 2002 From: jh at comunit.de (Janko) Date: Thu, 28 Feb 2002 10:42:52 +0100 Subject: [SciPy-user] gist.error: Gist GdLines plotter failed In-Reply-To: <3C7E048B.DD936049@mecha.uni-stuttgart.de> References: <3C7E048B.DD936049@mecha.uni-stuttgart.de> Message-ID: <20020228104252.0bf0dc85.jh@comunit.de> On Thu, 28 Feb 2002 11:20:59 +0100 Nils Wagner wrote: You need to reopen a plot window: >>> xplt.window(1) HTH, __Janko > Hi all, > > I have used xplt to plot some functions. > > xplt.window(1) > Python 2.1.2 (#1, Feb 25 2002, 18:04:21) > [GCC 2.95.3 20010315 (SuSE)] on linux2 > Type "copyright", "credits" or "license" for more information. > >>> from scipy.xplt import * > >>> from scipy import * > >>> x = arange(0,10,0.2) > >>> print x > [ 0. 0.2 0.4 0.6 0.8 1. 1.2 1.4 1.6 1.8 2. 2.2 2.4 2.6 > 2.8 > 3. 3.2 3.4 3.6 3.8 4. 4.2 4.4 4.6 4.8 5. 5.2 5.4 > 5.6 > 5.8 6. 6.2 6.4 6.6 6.8 7. 7.2 7.4 7.6 7.8 8. 8.2 > 8.4 > 8.6 8.8 9. 9.2 9.4 9.6 9.8] > >>> plot (x,sin(x)) > >>> plot (x,sin(0.1*x)) > >>> y = arange(0,10,0.1) > >>> plot (y,sin(0.1*y)) > >>> plot (y,sin(y)) > >>> y = arange(0,10,0.1) > >>> plot (y,sin(y)) > > If I close the window (containing the graph) and run the command > > >>> plot (y,sin(y)) > > again, I get > > Traceback (most recent call last): > File "", line 1, in ? > File "/usr/local/lib/python2.1/site-packages/scipy/xplt/Mplot.py", > line 470, in plot > > gist.plg(y,x,type=thetype,color=thecolor,marker=themarker,marks=tomark) > gist.error: Gist GdLines plotter failed > > Any idea ? > > Nils > _______________________________________________ > SciPy-user mailing list > SciPy-user at scipy.net > http://www.scipy.net/mailman/listinfo/scipy-user > From oliphant at ee.byu.edu Thu Feb 28 09:28:43 2002 From: oliphant at ee.byu.edu (Travis Oliphant) Date: Thu, 28 Feb 2002 09:28:43 -0500 (EST) Subject: [SciPy-user] gist.error: Gist GdLines plotter failed In-Reply-To: <3C7E048B.DD936049@mecha.uni-stuttgart.de> Message-ID: > I have used xplt to plot some functions. > > > Python 2.1.2 (#1, Feb 25 2002, 18:04:21) > [GCC 2.95.3 20010315 (SuSE)] on linux2 > Type "copyright", "credits" or "license" for more information. > >>> from scipy.xplt import * > >>> from scipy import * > >>> x = arange(0,10,0.2) > >>> print x > [ 0. 0.2 0.4 0.6 0.8 1. 1.2 1.4 1.6 1.8 2. 2.2 2.4 2.6 > 2.8 > 3. 3.2 3.4 3.6 3.8 4. 4.2 4.4 4.6 4.8 5. 5.2 5.4 > 5.6 > 5.8 6. 6.2 6.4 6.6 6.8 7. 7.2 7.4 7.6 7.8 8. 8.2 > 8.4 > 8.6 8.8 9. 9.2 9.4 9.6 9.8] > >>> plot (x,sin(x)) > >>> plot (x,sin(0.1*x)) > >>> y = arange(0,10,0.1) > >>> plot (y,sin(0.1*y)) > >>> plot (y,sin(y)) > >>> y = arange(0,10,0.1) > >>> plot (y,sin(y)) > > If I close the window (containing the graph) and run the command > > >>> plot (y,sin(y)) > > again, I get > > Traceback (most recent call last): > File "", line 1, in ? > File "/usr/local/lib/python2.1/site-packages/scipy/xplt/Mplot.py", > line 470, in plot > > gist.plg(y,x,type=thetype,color=thecolor,marker=themarker,marks=tomark) > gist.error: Gist GdLines plotter failed Gist won't let you close the window and then plot again without explicitly raising the window. xplt.window(0) followed by the plotting commands will work. This is something that could be fixed in the wrapper. I'll look in to it. -Travis From oliphant at ee.byu.edu Thu Feb 28 09:28:43 2002 From: oliphant at ee.byu.edu (Travis Oliphant) Date: Thu, 28 Feb 2002 09:28:43 -0500 (EST) Subject: [SciPy-user] gist.error: Gist GdLines plotter failed In-Reply-To: <3C7E048B.DD936049@mecha.uni-stuttgart.de> Message-ID: > I have used xplt to plot some functions. > > > Python 2.1.2 (#1, Feb 25 2002, 18:04:21) > [GCC 2.95.3 20010315 (SuSE)] on linux2 > Type "copyright", "credits" or "license" for more information. > >>> from scipy.xplt import * > >>> from scipy import * > >>> x = arange(0,10,0.2) > >>> print x > [ 0. 0.2 0.4 0.6 0.8 1. 1.2 1.4 1.6 1.8 2. 2.2 2.4 2.6 > 2.8 > 3. 3.2 3.4 3.6 3.8 4. 4.2 4.4 4.6 4.8 5. 5.2 5.4 > 5.6 > 5.8 6. 6.2 6.4 6.6 6.8 7. 7.2 7.4 7.6 7.8 8. 8.2 > 8.4 > 8.6 8.8 9. 9.2 9.4 9.6 9.8] > >>> plot (x,sin(x)) > >>> plot (x,sin(0.1*x)) > >>> y = arange(0,10,0.1) > >>> plot (y,sin(0.1*y)) > >>> plot (y,sin(y)) > >>> y = arange(0,10,0.1) > >>> plot (y,sin(y)) > > If I close the window (containing the graph) and run the command > > >>> plot (y,sin(y)) > > again, I get > > Traceback (most recent call last): > File "", line 1, in ? > File "/usr/local/lib/python2.1/site-packages/scipy/xplt/Mplot.py", > line 470, in plot > > gist.plg(y,x,type=thetype,color=thecolor,marker=themarker,marks=tomark) > gist.error: Gist GdLines plotter failed Gist won't let you close the window and then plot again without explicitly raising the window. xplt.window(0) followed by the plotting commands will work. This is something that could be fixed in the wrapper. I'll look in to it. -Travis From a.schmolck at gmx.net Thu Feb 28 14:26:41 2002 From: a.schmolck at gmx.net (A.Schmolck) Date: 28 Feb 2002 19:26:41 +0000 Subject: [SciPy-user] numarray interface and performance issues (for dot product and transpose) Message-ID: Hi, Numeric is an impressively powerful and in many respects easy and comfortable to use package (e.g. it's sophisticated slicing operations, not to mention the power and elegance of the underlying python language) and one would hope that it can one day replace Matlab (which is both expensive and a nightmare as a programming language) as a standard platform for numerical calculations. Two important drawbacks of Numeric (python) compared to Matlab currently seem to be the plotting functionality and the availability of specialist libraries (say for pattern recognition problems etc.). However python seems to be quickly catching up in this area (e.g. scipy and other projects are bustling with activity and, as far as I know, there is currently a promising undergraduate project on the way at my university to provide powerful plotting facilities for python). There is however a problem that, for the use to which I want to put Numeric, runs deeper and provides me with quite a headache: Two essential matrix operations (matrix-multiplication and transposition (which is what I am mainly using) are both considerably a) less efficient and b) less notationally elegant under Numeric than under Matlab. Before I expound on that, I'd better mention two things: Firstly, I, have to confess largely ignorant about the gritty details of implementing efficient numerical computation (and indeed, I can't even claim to possess a good grasp of the underlying Linear Algebra). Secondly, and partly as a consequence, I realize that I'm likely to view this from a rather narrow perspective. Nonetheless, I need to solve these problems at least for myself, even if such a solution might be inappropriate for a more general audience, so I'd be very keen to hear what suggestions or experiences from other Numeric users are. I also want to share what my phd supervisor, Richard Everson, and I have come up with so far in terms of a), as I think it might be quite useful for other people, too (see [2], more about this later). Ok, so now let me proceed to state clearly what I mean by a) and b): The following Matlab fragment M * (C' * C) * V' * u currently has to be written like this in python: dot(dot(dot(M, dot(transpose(C), C)), transpose(v)), u) Or, even worse if one doesn't want to pollute the namespace: Numeric.dot(Numeric.dot(Numeric.dot(Numeric.M, Numeric.dot(Numeric.transpose(C), C)), Numeric.transpose(v)), u) Part a) (efficiency) -------------------- Apart from the syntactic inconveniences, the Matlab above will execute __considerably__ faster if C is a rather larger matrix (say 1000x1000). There are AFAIK two chief reasons for this: 1. Matlab uses a highly optimized ATLAS[3] BLAS routine to compute the dot-product, whereas Numeric uses a straight-forward home-brewn solution. 2. Numeric performs unnecessary transpose operations (prior to 20.3, I think, more about this later). The transpose operation is really damaging with big matrices, because it creates a complete copy, rather than trying to do something lazy (if your memory is already almost half filled up with (matrix) C, then creating a (in principle superfluous) transposed copy is not going to do you any good). The above C' * C actually creates, AFAIK, _3_ versions of C, 2 of them transposed (prior to 20.3; dot(a,b) translates into innerproduct(a, swapaxes(b, -1, -2)) In newer versions of Numeric, this is replaced by multiarray.matrixproduct(a, b) which has the considerable advantage that it doesn't create an unnecessary copy and the considerable disadvantage that it seems to be factor 3 or so slower than the old (already not blazingly fast) version for large Matrix x Matrix multiplication, (see timing results [1])). Now fortunately, we made some significant progress on the performance issues. My supervisor valiantly patched Numeric's multiarrayobject.c to use ATLAS for {scalar,vector,matrix} * {scalar,vector,matrix} multiplications. As a consequence the multiplication of pair of 1000x1000 matrices is computed more than _40_ times faster on my athlon machine (see [1], it seems to work fine but both the timing results as well as the code should be treated with a bit of caution). The BLAS-enabled dot takes almost exactly the same time as Matlab doing the same thing. In my case, this makes the difference between waiting several minutes and a day, and between deciding to use python or something faster but more painful. Although to benefit from this patch, one has to install ATLAS this is straight-forward (see [2] and [3] for pointers). In addition it is easy to install Lapack at the same time, as ATLAS provides significantly optimised versions of some of the Lapack routines too. Using Lapack rather than the lapack_lite that comes with Numeric also has the advantage that it is extensively tested and works very reliably (e.g. the lapack_lite routine that is called by Heigenvalues is pretty broken and searching the net suggests that this has been the case for some time now). This still leaves point 2), transposes, as a potential source for improvement. As far as I understand, the Lapack routines operate on flattened arrays, i.e. the only difference on calling some routine on a matrix and its transpose is that one has to pass a different additional parameters in the second case that specifies the structure of the matrix. Therefore if one were to program C' * C in fortran, only C itself needed to exist in memory. Would it therefore be possible that operations like transpose, slicing etc. be carried out lazily (i.e. so that only _modifying_ the result of the transpose or slice or passing it to some function for some reason needs a real copy resulted in an actual copy operation being performed)? As far as I understand the slicing in numarray is no longer simple aliasing (which is a good thing, since it make arrays more compatible with lists), are there any plans for implementing such an on-demand scheme? Part b) (syntax) ________________ As I said, dot(dot(dot(M, dot(transpose(C), C)), transpose(v)), u) is pretty obscure compared to M * (C' * C) * V' * u) Although linear algebra is not the be all and end all of numerical calculations, many numerical algorithms end up involving a good deal of linear algebra. The awkwardness of python/Numeric's notation for common linear algebra operation severely impairs, in my opinion, the utility of writing, using and debugging algorithms in python/Numeric. This is a particular shame given python's generally elegant and expresssive syntax. So far, I've thought of the following possible solutions: 0. Do nothing: Just live with the awkward syntax. 1. Comment: add a comment like the Matlab line above above complex expressions. 2. Wait for fix. Hope that either: 2.1. numarray will overload '*' as matrix multiplication, rather than elementwise multiplication (highly unlikely, I suspect). 2.2. the core python team accepts one of the proposed operator additions (or even, fond phantasy, allows users to add new operators) 3. Wrap: create a DotMatrix class that overloads '*' to be dot and maybe self.t to return the transpose -- this also means that all the numerical libraries I frequently use need to be wrapped. 4. Preprocess: Write some preprocessor that translates e.g. A ~* B in dot(A,B). 5. Hack: create own customized version of array that overloads e.g. '<<' to mean dot-product. The possible downsides: 0. Do nothing: Seems unacceptable (unless as a temporary solution), because the code becomes much less readable (and consequently harder to understand and more error-prone), thus forgoing one of the main reasons to choose python in the first place. 1. Comment: Unsatisfactory, because there is no way to gurantee that comment and code are in synch, which very likely will lead to difficult to find bugs. 2. Wait for a fix: Either would be nice (I could live with having to write multiply(a,b) -- I suppose however some other people can't, because I can't think of another reason why Numeric didn't overload * for matrixproduct in the first place, moreover it would mean a significant interface change (and code breakage), so I guess it is rather unlikely). From what I gather from searching the net and looking at peps there is also not much of a chance to get a new operator anytime soon. 3. Wrap: Promises to be a maintenance nightmare (BTW: will the new array class be subclassable?), but otherwise looks feasible. Has anyone done this? 4. Preprocess: Would have the advantage that I could always get back to "standard" python/Numeric code, but would involve quite a bit of work and might also break tools that parse python. 5. Hack array: Seems evil, but in principle feasible, because AFAIK '<<' isn't used in Numeric itself and hopefully it wouldn't involve too much work. However, given a free choice '<<' is hardly the operator I would choose to represent the dot product. I am not completely sure what the policy and rationale behind the current division of labor between functions, methods and attributes in numarray is but as far as the lack of a missing transposition postfix operator is concerned, one reasonable approach to make the transpose operation more readable (without the need to change anything in python itself) would seem to me to provide arrays with an attribute .t or .T so that: a.t == transpose(a) While this would admittedly be a bit of a syntactic hack, the operation is so commonplace and the gain in readability (in my eyes) is so significant that it would seem to me to merit serious consideration (maybe a.cc or so might also be an idea for complex conjugate notation, but I'm less convinced about that?). If the addition of a single operator for the exclusive benefit of Numeric users is rejected by the core python team, maybe it's worthwhile lobbying for some mechanism that allows users to define new operators (like e.g. in Haskell)... OK, that's all -- thanks for bearing with me through this long email. Suggestions and comments about the patch [2] and possible solutions to issues raised are greatly appreciated, regards, alexander schmolck Footnotes: [1] timing results for the patched version of Numeric[2] comparing new and old 'dot' performance: http://www.dcs.ex.ac.uk/~aschmolc/Numeric/TimingsForAtlasDot.txt [2] A patch for Numeric 21.1b that speeds up Numeric's dot function by several factors for large matrices can be found here: http://www.dcs.ex.ac.uk/~aschmolc/Numeric/ [3] ATLAS (http://math-atlas.sourceforge.net/) is a project to provide a highly optimized version of BLAS (Basic Linear Algebra Subroutines, a standard and thouroughly tested implementation of basic linear algebra operations) and some LAPACK routines. The charm of ATLAS is that it is platform-independent and yet highly optimized, which it achieves by essentially fine tuning a number of parameters until optimum performance for the _particular_ machine on which it is built is reached. As a consequence complete builds can take some time, but binary versions for ATLAS for common processors are available from http://www.netlib.org/atlas/archives (moreover, even if one decides to build ATLAS oneself, the search space can be considerably cut down if one accepts the suggested "experience" values offered during the make process). -- Alexander Schmolck Postgraduate Research Student Department of Computer Science University of Exeter A.Schmolck at gmx.net http://www.dcs.ex.ac.uk/people/aschmolc/ From cgw at alum.mit.edu Thu Feb 28 15:24:02 2002 From: cgw at alum.mit.edu (Charles G Waldman) Date: Thu, 28 Feb 2002 14:24:02 -0600 Subject: [SciPy-user] [Numpy-discussion] numarray interface and performance issues (for dot product and transpose) In-Reply-To: References: Message-ID: <15486.37346.523821.173804@nyx.dyndns.org> A.Schmolck writes: > > Two essential matrix operations (matrix-multiplication and transposition > (which is what I am mainly using) are both considerably > > a) less efficient and > b) less notationally elegant Your comments about efficiency are well-taken. I have (in a previous life) done work on efficient (in terms of virtual memory access / paging behavior) transposes of large arrays. (Divide and conquer). Anyhow - if there were support for the operation of A*B' (and A'*B) at the C level, you wouldn't need to ever actually have a copy of the transposed array in memory - you would just exchange the roles of "i" and "j" in the computation... > 3. Wrap: create a DotMatrix class that overloads '*' to be dot and maybe > self.t to return the transpose -- this also means that all the numerical > libraries I frequently use need to be wrapped. I guess you haven't yet stumbled across the Matrix.py that comes with Numeric - it overrides "*" to be the dot-product. Unfortunately I don't see a really easy way to simplify the Transpose operator - at the very least you could do T = Numeric.transpose and then you're just writing T(A) instead of the long-winded version. Interestingly, the "~" operator is available, but it calls the function "__invert__". I guess it would be too weird to have ~A denote the transpose? Right now you get an error - one could set things up so that ~A was the matrix inverse of A, but we already have the A**-1 notation (among others) for that... From jjl at pobox.com Thu Feb 28 15:28:13 2002 From: jjl at pobox.com (John J. Lee) Date: Thu, 28 Feb 2002 20:28:13 +0000 (GMT) Subject: [SciPy-user] optimize.minpack.error? In-Reply-To: Message-ID: On Thu, 28 Feb 2002, Pearu Peterson wrote: > On Wed, 27 Feb 2002, John J. Lee wrote: > > The minpack.error Exception seems to be impossible to get at (scipy 0.1, > > but latest CVS minpackmodule.c seems similar at least). [...PyErr_NewException...] > It is described here > http://www.python.org/doc/current/api/exceptionHandling.html#l2h-95 Thanks -- actually I had already found the description of the function, but didn't know quite what the meaning of the module bit of the name arg was: I couldn't tell (without first reading up on the import system) whether this call is sufficient to be able to import the Exception from another module. > But I don't understand what you are getting at. Do you want minpack.error > Exception arised in your app or what? The minpack module throws minpack.error, and I need to catch it. I couldn't import it from minpack or _minpack. How do I get at it? > Note that calling minpack functions may fail with exceptions, not just > minpack.error. Yes, I realise that. John From jjl at pobox.com Thu Feb 28 16:00:00 2002 From: jjl at pobox.com (John J. Lee) Date: Thu, 28 Feb 2002 21:00:00 +0000 (GMT) Subject: [SciPy-user] Re: [Numpy-discussion] numarray interface and performance issues (for dot product and transpose) In-Reply-To: Message-ID: On 28 Feb 2002, A.Schmolck wrote: > Two important drawbacks of Numeric (python) compared to Matlab > currently seem to be the plotting functionality and the availability > of specialist libraries (say for pattern recognition problems etc.). > However python seems to be quickly catching up in this area (e.g. > scipy and other projects are bustling with activity and, as far as I > know, there is currently a promising undergraduate project on the way > at my university to provide powerful plotting facilities for python). http://www.bel-epa.com/pyapache/Python/gdmodule/gdmodule-1.0/doc/ some quick matlab graph-alike thing, don't know if any good, haven't used it I don't much like any of the graphing stuff either, but it's worth looking carefully: there are quite a few Python-accessible packages around that you may not have noticed: plotutils, gnuplot, dislin, Qwt/Qt, BLT/Pmw, Grace, Scigraphica, PGPLOT, PLPLOT, the scipy stuff, probably many others (not to mention 3D library wrappers, and image processing stuff, none of which I use). None of them are wonderful, but they all have their strengths. I use a simple interface to use Qwt, Grace and gnuplot from the same Python code, so I can plot stuff in my GUI, then interactive fiddling about in Grace or run off a postscript plot with gnuplot -- works well for me but certainly not ideal. > There is however a problem that, for the use to which I want to put Numeric, > runs deeper and provides me with quite a headache: > > Two essential matrix operations (matrix-multiplication and transposition > (which is what I am mainly using) are both considerably > > a) less efficient and I think someone has already done some ATLAS work with numpy... > b) less notationally elegant http://matpy.sourceforge.net/ -- not seen any activity for 6 months, but you might find some useful stuff there. John From pearu at cens.ioc.ee Thu Feb 28 16:27:07 2002 From: pearu at cens.ioc.ee (Pearu Peterson) Date: Thu, 28 Feb 2002 23:27:07 +0200 (EET) Subject: [SciPy-user] numarray interface and performance issues (for dot product and transpose) In-Reply-To: Message-ID: Hi, On 28 Feb 2002, A.Schmolck wrote: > So far, I've thought of the following possible solutions: > > 0. Do nothing: > Just live with the awkward syntax. Let me add a subsolution here: 0.1 Wait for scipy to mature (or better yet, help it to do that). Scipy already provides wrappers to both, Fortran and C, LAPACK and BLAS routines, though currently they are under revision. With the new wrappers to these routines you can optimize your code fragments as flexible as if using them from C or Fortran. In principle, one could mix Fortran and C routines (i.e. the corresponding wrappers) so that one avoids all unneccasary transpositions. All matrix operations can be performed in-situ if so desired. Regards, Pearu From R.M.Everson at exeter.ac.uk Thu Feb 28 16:46:43 2002 From: R.M.Everson at exeter.ac.uk (R.M.Everson) Date: 28 Feb 2002 21:46:43 +0000 Subject: [SciPy-user] numarray interface and performance issues (for dot product and transpose) In-Reply-To: References: Message-ID: Alex, Seems like you have generated some discussion at least. Perhaps we should download the new numarray and see what's involved in ATLASifying it. Best, Richard. From eric at scipy.org Thu Feb 28 16:25:08 2002 From: eric at scipy.org (eric) Date: Thu, 28 Feb 2002 16:25:08 -0500 Subject: [SciPy-user] numarray interface and performance issues (for dot product and transpose) References: Message-ID: <091e01c1c09e$65de5580$6b01a8c0@ericlaptop> It should be possibly to just reimplement the NumPy functions like dot and matrixmultiply (name it matmul like F90 or mat_mul) within SciPy to use ATLAS. Then, using the following: from scipy import * ... c = dot(a,b) should go fast. The NumPy guys *might* go for requireing ATLAS, but I find that highly unlikely because it is such a bear to build on some platforms and takes so long on all platforms. One of the beauties of NumPy is that it is self contained. SciPy has already bitten the bullet as far as considering ATLAS as part of the core requirements, so it comes as no extra effort. We should just add the ATLAS code patch to SciPy's linalg module and then import those functions into the main namespace. eric ----- Original Message ----- From: "R.M.Everson" To: Sent: Thursday, February 28, 2002 4:46 PM Subject: Re: [SciPy-user] numarray interface and performance issues (for dot product and transpose) > > Alex, > > Seems like you have generated some discussion at least. Perhaps we > should download the new numarray and see what's involved in ATLASifying > it. > > Best, > Richard. > > _______________________________________________ > SciPy-user mailing list > SciPy-user at scipy.net > http://www.scipy.net/mailman/listinfo/scipy-user > From oliphant at ee.byu.edu Thu Feb 28 15:33:37 2002 From: oliphant at ee.byu.edu (Travis Oliphant) Date: Thu, 28 Feb 2002 15:33:37 -0500 (EST) Subject: [SciPy-user] Re: [Numpy-discussion] numarray interface and performance issues (for dot product and transpose) In-Reply-To: Message-ID: On 28 Feb 2002, A.Schmolck wrote: > Two essential matrix operations (matrix-multiplication and transposition > (which is what I am mainly using) are both considerably > > a) less efficient and > b) less notationally elegant You are not alone in your concerns. The developers of SciPy are quite concerned about speed, hence the required linking to ATLAS. As Pearu mentioned all of the BLAS will be available (much of it is). This will enable very efficient algorithms. The question of notational elegance is stickier because we just can't add new operators. The solution I see is to use other classes. Right now, the Numeric array is an array of numbers (it is not a vector or a matrix) and that is why it has the operations it does. The Matrix class (delivered with Numeric) creates a Matrix object that uses the array of numbers of Numeric arrays. It overloads the * operator and defines .T, and .H for transpose and Hermitian transpose respectively. This requires explictly making your objects matrices (not a bad thing in my book as not all 2-D arrays fit perfectly in a matrix algebra). > The following Matlab fragment > > M * (C' * C) * V' * u > This becomes (using SciPy which defines Mat = Matrix.Matrix and could later redefine it to use the ATLAS libraries for matrix multiplication). C, V, u, M = apply(Mat, (C, V, u, M)) M * (C.H * C) * V.H * M not bad.. and with a Mat class that uses the ATLAS blas (not a very hard thing to do now.), this could be made as fast as MATLAB. Perhaps, as as start we could look at how you make the current Numeric use blas if it is installed to do dot on real and complex arrays (I know you can get rid of lapack_lite and use your own lapack) but, the dot function is defined in multiarray and would have to be modified to use the BLAS instead of its own homegrown algorithm. -Travis From oliphant at ee.byu.edu Thu Feb 28 15:35:31 2002 From: oliphant at ee.byu.edu (Travis Oliphant) Date: Thu, 28 Feb 2002 15:35:31 -0500 (EST) Subject: [SciPy-user] numarray interface and performance issues (for dot product and transpose) In-Reply-To: <091e01c1c09e$65de5580$6b01a8c0@ericlaptop> Message-ID: > should go fast. The NumPy guys *might* go for requireing ATLAS, but I find that > highly unlikely because it is such a bear to build on some platforms and takes > so long on all platforms. One of the beauties of NumPy is that it is self > contained. > > SciPy has already bitten the bullet as far as considering ATLAS as part of the > core requirements, so it comes as no extra effort. We should just add the ATLAS > code patch to SciPy's linalg module and then import those functions into the > main namespace. > I agree. I would much rather do this then change Numeric. One thing MATLAB users find different is that Numeric arrays are arrays not matrices. A separate subclass of arrays can be considered a matrix. -Travis From karshi.hasanov at utoronto.ca Thu Feb 28 23:36:40 2002 From: karshi.hasanov at utoronto.ca (Karshi) Date: Thu, 28 Feb 2002 23:36:40 -0500 Subject: [SciPy-user] image_module Message-ID: <20020301043719Z234720-8939+2@bureau8.utcc.utoronto.ca> Hi Eric, I couldn't get the CVS version of scipy work, and had go back to the version 0.1. The following code works: ----------------------------------------------------------------- from Numeric import * import gui_thread from scipy import plt from scipy import * Img = zeros((256,256), Float32) for i in arange(256): for j in arange(256): r = sqrt(pow(i-128,2)+pow(j-128,2)) if r <= 60: Img[i,j]= 100 #plt.image(Img) x = arange(0,5) y = special.j1(x) plt.plot(x,y) plt.image(Img) plt.grid('off') -------------------------------------------------------------------- But if I take below lines out : x = arange(0,5) y = special.j1(x) plt.plot(x,y) ---------------- It will give me errors like this: >>> Traceback (most recent call last): File "/home/Study/Matlab/Sphere.py", line 15, in ? plt.image(Img) File "/usr/local/lib/python2.1/site-packages/scipy/plt/interface.py", line 427, in image axis('equal') File "/usr/local/lib/python2.1/site-packages/scipy/plt/interface.py", line 174, in axis x_ticks = _active.x_axis.ticks AttributeError: axis_window instance has no attribute 'ticks' ------------------------------------------------------------- I don't understand why ?