From fperez at pizero.colorado.edu Fri Mar 1 02:48:58 2002 From: fperez at pizero.colorado.edu (Fernando Perez) Date: Fri, 1 Mar 2002 00:48:58 -0700 (MST) Subject: [SciPy-user] numarray interface and performance issues (for dot product and transpose) In-Reply-To: Message-ID: A few (spotty) comments on your excellent and detailed report. > scipy and other projects are bustling with activity and, as far as I > know, there is currently a promising undergraduate project on the way > at my university to provide powerful plotting facilities for python). I hope that if this undergraduate project you mention is something serious, they could consider joining forces with the scipy plotting effort. One of the problems with open source projects is that it's all too common to see fifteen half-finished systems with not a single one that really works. In my opinion the scipy leaders have done a great job of not reinventing any wheels unnecessarily, trying to leverage all existing code (when feasible). But now plotting is an area of scipy which isn't recieving excessive attention (because people are super-busy with other things), so I'm sure that outside hands would be appreciated. At least it would be worth discussing it. The hope (at least I have) is that in the future, SciPy will be a one-stop solution for a solid, comprehensive scientific computing environment with Python. But for that to become a reality various projects should if at all possible consider joining forces. > I am not completely sure what the policy and rationale behind the > current division of labor between functions, methods and attributes in > numarray is but as far as the lack of a missing transposition postfix > operator is concerned, one reasonable approach to make the transpose > operation more readable (without the need to change anything in python > itself) would seem to me to provide arrays with an attribute .t or .T > so that: > > a.t == transpose(a) > > While this would admittedly be a bit of a syntactic hack, the > operation is so commonplace and the gain in readability (in my eyes) > is so significant that it would seem to me to merit serious > consideration (maybe a.cc or so might also be an idea for complex > conjugate notation, but I'm less convinced about that?). Well, as others told you using a Matrix instead of an array will get you this notation. Here's an example: In [24]: a=Matrix.Matrix([[0,1+1j],[1,3-1j]]) In [25]: a Out[25]= Matrix([[ 0.+0.j, 1.+1.j], [ 1.+0.j, 3.-1.j]]) In [26]: a.T # transpose Out[26]= Matrix([[ 0.+0.j, 1.+0.j], [ 1.+1.j, 3.-1.j]]) In [27]: a.H # Hermitian conjugate Out[27]= Matrix([[ 0.-0.j, 1.-0.j], [ 1.-1.j, 3.+1.j]]) In [28]: a.H.T # complex conjugate, an a.cc notation would be nice Out[28]= Matrix([[ 0.-0.j, 1.-1.j], [ 1.-0.j, 3.+1.j]]) Now, I don't know internally how these operations are implemented, they probably return copies of their objects (slow, expensive for big matrices). But I was wondering if it would be worth considering making .T and .cc simple flags. Basically, one could define them via getters (properties in python 2.2) so that all the above syntax works, but internally the getter for a.T would return a with a flag for transposition on. Then any matrix operation could check for whether this flag was on or not and arrange its looping order to honor the convention without changing ever the data in a. The same thing could be done for complex conjugation. This may turn out to be a really stupid idea, but at least I'd like to hear other's opinions. This would require all the basic operations on matrices (__getitem__, __add__, __mult__, etc) to check for these flags. But once that job is done (once), operations involving complex conjugation or transposition could be done very fast, only keeping copies of the original data (since both a.T and a.cc can be expressed in terms of a). For the .cc case this would require writing the basic complex algebra in terms of conjugated numbers, but that's a minor annoyance. The tradeoff seems to be one of complexity in the base operations code and a bit of extra checking (an if test at the start). But for large arrays this may give significant benefits. Communication problems: if one needs to pass a.T to an external library which has no idea of how to handle these flags, there's a problem. If this idea doesn't have other fundamental flaws (which it probably does), one way around this crucial problem would be to have _two_ different ways of getting transpose(a) --similar ideas for conjugation: - a.T -> 'flagged' object, only works with libraries which know how to handle it but allows writing clean, fast code with zero-copy. - a.Transpose() -> makes a copy and returns a normal array with the elements transposed. Any routine which can handle a numpy array can handle this. So, I realize this idea may well be fundamentally broken, but it may also have something of interest in it. Feel free to shoot me down. Regards, f. From R.M.Everson at exeter.ac.uk Fri Mar 1 04:41:17 2002 From: R.M.Everson at exeter.ac.uk (R.M.Everson) Date: 01 Mar 2002 09:41:17 +0000 Subject: [SciPy-user] Re: [Numpy-discussion] numarray interface and performance issues (for dot product and transpose) In-Reply-To: References: Message-ID: Hi, On 28 Feb 2002, Travis Oliphant wrote: > On 28 Feb 2002, A.Schmolck wrote: >> Two essential matrix operations (matrix-multiplication and >> transposition (which is what I am mainly using) are both >> considerably >> >> a) less efficient and >> b) less notationally elegant > You are not alone in your concerns. The developers of SciPy are > quite concerned about speed, hence the required linking to ATLAS. > The question of notational elegance is stickier because we just > can't add new operators. > The solution I see is to use other classes. At the moment, I agree this is probably the best solution, although it would be nice if the core python was able to add operators :) >> The following Matlab fragment >> M * (C' * C) * V' * u >> > This becomes (using SciPy which defines Mat = Matrix.Matrix and > could later redefine it to use the ATLAS libraries for matrix > multiplication). > C, V, u, M = apply(Mat, (C, V, u, M)) > M * (C.H * C) * V.H * M Yes, much better. > not bad.. and with a Mat class that uses the ATLAS blas (not a > very hard thing to do now.), this could be made as fast as > MATLAB. > Perhaps, as as start we could look at how you make the current > Numeric use blas if it is installed to do dot on real and complex > arrays (I know you can get rid of lapack_lite and use your own > lapack) but, the dot function is defined in multiarray and would > have to be modified to use the BLAS instead of its own homegrown > algorithm. This is precisely what Alex and I have done. Please see the patch to Numeric and timings on http://www.dcs.ex.ac.uk/~aschmolc/Numeric/ It's not beautiful but about 40 times faster on 1000 by 1000 matrix multiplies. I'll attempt to provide a similar patch for numarray over the next week or so. Many thanks for your comments. Richard. From a.schmolck at gmx.net Fri Mar 1 11:38:26 2002 From: a.schmolck at gmx.net (A.Schmolck) Date: 01 Mar 2002 16:38:26 +0000 Subject: [SciPy-user] numarray interface and performance issues (for dot product and transpose) In-Reply-To: References: Message-ID: Travis Oliphant writes: > [eric writes:] > > should go fast. The NumPy guys *might* go for requireing ATLAS, but I find that > > highly unlikely because it is such a bear to build on some platforms and takes > > so long on all platforms. One of the beauties of NumPy is that it is self > > contained. > > > > SciPy has already bitten the bullet as far as considering ATLAS as part of the > > core requirements, so it comes as no extra effort. We should just add the ATLAS > > code patch to SciPy's linalg module and then import those functions into the > > main namespace. > > > > I agree. I would much rather do this then change Numeric. I perfectly agree that Numeric shouldn't _require_ atlas (especially if it hopes to find its way into python core) and I should have clarified this point in my email. However, I can't see what's wrong with _optionally_ using atlas, if the user so desires. That's what #ifdef's are for :) Indeed, richard's modifications to multiarraymodule.c are neatly wrapped in a couple of #ifdef DOTBLAS statements. Wouldn't such a solution introduce less maintenance work than scipy maintaining its own, atlas/blas-enabled only, version of Numeric? Personally, I'm happy if the patch only makes it into scipy, since I already use scipy anyway, but I guess having and (optional) speedup > factor 40 can't hurt Numeric either (if it doesn't involve too much maintenance work) and will surely make some of its users happy. alex -- Alexander Schmolck Postgraduate Research Student Department of Computer Science University of Exeter A.Schmolck at gmx.net http://www.dcs.ex.ac.uk/people/aschmolc/ From a.schmolck at gmx.net Fri Mar 1 16:07:29 2002 From: a.schmolck at gmx.net (A.Schmolck) Date: 01 Mar 2002 21:07:29 +0000 Subject: [SciPy-user] numarray interface and performance issues (for dot product and transpose) In-Reply-To: References: Message-ID: Hi Fernando, Fernando Perez writes: > I hope that if this undergraduate project you mention is something serious, > they could consider joining forces with the scipy plotting effort. One of the AFAIK it is a serious attempt to provide fully fledged plotting abilities for python (2D, 3D, images, zooming, saving in ps etc.), but unfortunately, I suspect that since the shared nature of open source development fits uneasily with the assessment policy of my university for 3rd year projects, no code will be publicly released before the end of easter (as much as I myself would like to have a look at it now and hack around...). > Well, as others told you using a Matrix instead of an array will get you this > notation. Here's an example: I knew about Matrix, but I was concerned that the required casting to use standard scipy/Numeric functions with "Matrixes" might cause considerable clutter, but maybe that concern was a bit unfounded... [snipped] > Now, I don't know internally how these operations are implemented, they > probably return copies of their objects (slow, expensive for big > matrices). But I was wondering if it would be worth considering making .T and > .cc simple flags. Basically, one could define them via getters (properties in > python 2.2) so that all the above syntax works, but internally the getter for > a.T would return a with a flag for transposition on. Then any matrix operation > could check for whether this flag was on or not and arrange its looping order > to honor the convention without changing ever the data in a. The same thing > could be done for complex conjugation. > > This may turn out to be a really stupid idea, but at least I'd like to hear > other's opinions. This would require all the basic operations on matrices > (__getitem__, __add__, __mult__, etc) to check for these flags. But once that > job is done (once), operations involving complex conjugation or transposition > could be done very fast, only keeping copies of the original data (since both > a.T and a.cc can be expressed in terms of a). For the .cc case this would > require writing the basic complex algebra in terms of conjugated numbers, but > that's a minor annoyance. > > The tradeoff seems to be one of complexity in the base operations code and a > bit of extra checking (an if test at the start). But for large arrays this may > give significant benefits. > > Communication problems: if one needs to pass a.T to an external library which > has no idea of how to handle these flags, there's a problem. If this idea > doesn't have other fundamental flaws (which it probably does), one way around > this crucial problem would be to have _two_ different ways of getting > transpose(a) --similar ideas for conjugation: > > - a.T -> 'flagged' object, only works with libraries which know how to handle > it but allows writing clean, fast code with zero-copy. > > - a.Transpose() -> makes a copy and returns a normal array with the elements > transposed. Any routine which can handle a numpy array can handle this. > Hmm, interestingly you are the only one who picked up on the lazy copying theme so far, apart from another poster who suggested that no unnecessary copies are created anyway (I have no time to check that right now, unfortunately). I'd certainly much prefer if this work would get done behind the scenes (as far as I know it isn't but I might be wrong). I feel Numeric itself should really handle this transparently if possible, but allowing the user to explicitly specify that he wants something like transpose and slice copies done eagerly might be an idea (it shouldn't change the semantics, but might be worthwhile for debugging purposes or for efficiency/desired "realtime" behavior in situations where it is clear that an actual copy operation needs to be performed). Of course talk is cheap and I don't know how much pain such an implementation would involve, but to me it seems very attractive. alex -- Alexander Schmolck Postgraduate Research Student Department of Computer Science University of Exeter A.Schmolck at gmx.net http://www.dcs.ex.ac.uk/people/aschmolc/ From heiko at hhenkelmann.de Fri Mar 1 16:45:09 2002 From: heiko at hhenkelmann.de (Heiko Henkelmann) Date: Fri, 1 Mar 2002 22:45:09 +0100 Subject: [SciPy-user] build problem -- fftw References: Message-ID: <000901c1c16a$5c6c9a80$2361e03e@arrow> I tried to build the latest CVS content: MinGW>python setup.py build -c mingw32 file: build\generated_pyfs\flapack.pyf file: build\generated_pyfs\clapack.pyf file: build\generated_pyfs\fblas.pyf file: build\generated_pyfs\cblas.pyf atlas_info: Looking in E:\usr\lib ... FOUND: libraries = ['lapack', 'f77blas', 'cblas', 'atlas'] library_dirs = ['E:\\usr\\lib'] fftw_info: Looking in C:\fftw ... Looking in C:\ ... Looking in C:\USR\PYTHON21 ... NOT AVAILABLE Traceback (most recent call last): File "setup.py", line 127, in ? install_package() File "setup.py", line 93, in install_package config.extend([get_package_config(x,parent_package)for x in standard_packages]) File "setup.py", line 46, in get_package_config config = mod.configuration(parent) File "fftw\setup_fftw.py", line 12, in configuration raise FFTWNotFoundError,FFTWNotFoundError.__doc__ scipy_distutils.system_info.FFTWNotFoundError: FFTW (http://www.fftw.org/) libraries not found. Either install them in /usr/local/lib or /usr/lib and retry setup.py. One can use also FFTW environment variable to indicate In scipy_distutils I get: MinGW>python system_info.py atlas_info: Looking in E:\usr\lib ... FOUND: libraries = ['lapack', 'f77blas', 'cblas', 'atlas'] library_dirs = ['E:\\usr\\lib'] blas_info: Looking in C:\ ... Looking in C:\USR\PYTHON21 ... NOT AVAILABLE fftw_info: Looking in C:\fftw ... Looking in C:\ ... FOUND: libraries = ['sfftw', 'srfftw', 'sfftw_threads', 'srfftw_threads'] library_dirs = ['C:\\fftw'] lapack_info: Looking in C:\ ... Looking in C:\USR\PYTHON21 ... NOT AVAILABLE x11_info: Looking in C:\ ... Looking in C:\USR\PYTHON21 ... NOT AVAILABLE Any idea what's going on there? From eric at scipy.org Fri Mar 1 15:47:48 2002 From: eric at scipy.org (eric) Date: Fri, 1 Mar 2002 15:47:48 -0500 Subject: [SciPy-user] numarray interface and performance issues (for dot product and transpose) References: Message-ID: <0a6201c1c162$594b2fe0$6b01a8c0@ericlaptop> > > > > - a.Transpose() -> makes a copy and returns a normal array with the elements > > transposed. Any routine which can handle a numpy array can handle this. > > > Hmm, interestingly you are the only one who picked up on the lazy copying > theme so far, apart from another poster who suggested that no unnecessary > copies are created anyway (I have no time to check that right now, > unfortunately). Numeric does not make a copy during transpose. It changes values in the stride and dimensions fields of the C array structure to create a "virtual transpose". Note that this does break the contiguity of your matrix in memory. If you want to force an actual memory transpose (which I know you do not), you can use tranpose(a).copy(). This will return a new, contiguous array with the elements reordered. There isn't a way to do this in place as far as I know. So, as far as transpose goes, Numeric is already doing what you are asking. The speed loss your seeing is due to other issues. eric From laytonjb at bellsouth.net Fri Mar 1 16:59:37 2002 From: laytonjb at bellsouth.net (Jeff Layton) Date: Fri, 01 Mar 2002 16:59:37 -0500 Subject: [SciPy-user] Re: [Numpy-discussion] numarray interface and performance issues(for dot product and transpose) References: Message-ID: <3C7FF9C9.7BD5C285@bellsouth.net> "John J. Lee" wrote: > On 28 Feb 2002, A.Schmolck wrote: > > > Two important drawbacks of Numeric (python) compared to Matlab > > currently seem to be the plotting functionality and the availability > > of specialist libraries (say for pattern recognition problems etc.). > > However python seems to be quickly catching up in this area (e.g. > > scipy and other projects are bustling with activity and, as far as I > > know, there is currently a promising undergraduate project on the way > > at my university to provide powerful plotting facilities for python). > > http://www.bel-epa.com/pyapache/Python/gdmodule/gdmodule-1.0/doc/ > > some quick matlab graph-alike thing, don't know if any good, haven't used > it > > I don't much like any of the graphing stuff either, but it's worth looking > carefully: there are quite a few Python-accessible packages around that > you may not have noticed: plotutils, gnuplot, dislin, Qwt/Qt, BLT/Pmw, > Grace, Scigraphica, PGPLOT, PLPLOT, the scipy stuff, probably many others > (not to mention 3D library wrappers, and image processing stuff, none of > which I use). None of them are wonderful, but they all have their > strengths. I use a simple interface to use Qwt, Grace and gnuplot from > the same Python code, so I can plot stuff in my GUI, then interactive > fiddling about in Grace or run off a postscript plot with gnuplot -- works > well for me but certainly not ideal. I like kmatplot http://kmatplot.sourceforge.net I haven't really begun to use it in Python, but it's setup for it. Jeff > > > > There is however a problem that, for the use to which I want to put Numeric, > > runs deeper and provides me with quite a headache: > > > > Two essential matrix operations (matrix-multiplication and transposition > > (which is what I am mainly using) are both considerably > > > > a) less efficient and > > I think someone has already done some ATLAS work with numpy... > > > b) less notationally elegant > > http://matpy.sourceforge.net/ > > -- not seen any activity for 6 months, but you might find some useful > stuff there. > > John > > _______________________________________________ > SciPy-user mailing list > SciPy-user at scipy.net > http://www.scipy.net/mailman/listinfo/scipy-user From pearu at cens.ioc.ee Fri Mar 1 17:17:54 2002 From: pearu at cens.ioc.ee (Pearu Peterson) Date: Sat, 2 Mar 2002 00:17:54 +0200 (EET) Subject: [SciPy-user] build problem -- fftw In-Reply-To: <000901c1c16a$5c6c9a80$2361e03e@arrow> Message-ID: Hoi, On Fri, 1 Mar 2002, Heiko Henkelmann wrote: > I tried to build the latest CVS content: > > MinGW>python setup.py build -c mingw32 > > fftw_info: > Looking in C:\fftw ... > Looking in C:\ ... > Looking in C:\USR\PYTHON21 ... > NOT AVAILABLE > MinGW>python system_info.py > > fftw_info: > Looking in C:\fftw ... > Looking in C:\ ... > FOUND: > libraries = ['sfftw', 'srfftw', 'sfftw_threads', 'srfftw_threads'] > library_dirs = ['C:\\fftw'] > Any idea what's going on there? No. But could you update system_info.py from CVS and try again. If that does not work, try setting environment variable FFTW to C:\\fftw. I have no idea how to do that under mingw; if it is like bash, then try FFTW=C:\\fftw python setup.py build -c mingw32 Also, you can try FFTW=C:\\Fftw FFTW=C:\\FFTW and other variants that windoza may use. Try also copying FFTW directory to C:\\usr. If all that does not work, then I suggest you try debugging system_info.py. I am pretty perplex (what a nice word I found in dictonary;) about what is going on there under windoza. Regards, Pearu From fperez at pizero.colorado.edu Fri Mar 1 17:36:04 2002 From: fperez at pizero.colorado.edu (Fernando Perez) Date: Fri, 1 Mar 2002 15:36:04 -0700 (MST) Subject: [SciPy-user] numarray interface and performance issues (for dot product and transpose) In-Reply-To: Message-ID: On 1 Mar 2002, A.Schmolck wrote: > Hi Fernando, > > Fernando Perez writes: > > I hope that if this undergraduate project you mention is something serious, > > they could consider joining forces with the scipy plotting effort. One of the > > AFAIK it is a serious attempt to provide fully fledged plotting abilities for > python (2D, 3D, images, zooming, saving in ps etc.), but unfortunately, I > suspect that since the shared nature of open source development fits uneasily > with the assessment policy of my university for 3rd year projects, no code > will be publicly released before the end of easter (as much as I myself would > like to have a look at it now and hack around...). Too bad. Maybe when they're done with the 'class project' they can open it up for public development in the future. There's a lot of plotting projects out there (scigraphica, kmatplot and mathplot are the newest ones that come to mind) and most feel like half-finished toys, or seriously lack scripting abilities. In my mind, even though a gui is very useful in many cases, it's _critical_ to have low-level access to the graphing primitives so the engine can be controlled via code. I have no interest in spending my life clicking on menus to set titles when all that can be automated. The scipy folks seem well directed at a system which does provide that kind of low-level access, but at some point (even if it's not right now) I'm sure that all new hands would be welcome. One good example of what I have in mind is Prabhu's MayaVi: nice gui but fully scriptable (even though the internals are in need of some restructuring to make this task easier). I'm using it to generate sequences of 3-d slices of 4-d datasets, which would be in practice impossible if I couldn't control the thing via code. Your friends might want to take a look at it (if they haven't already). Anyway, I'm glad to see there's interest in this area and people working on it. Eric and the core scipy people are already busy to the top with the basic infrastructure. Cheers, f. From eric at scipy.org Sun Mar 3 07:18:14 2002 From: eric at scipy.org (eric) Date: Sun, 3 Mar 2002 07:18:14 -0500 Subject: [SciPy-user] image_module References: <20020301043719Z234720-8939+2@bureau8.utcc.utoronto.ca> Message-ID: <0ce001c1c2ad$7e985dd0$6b01a8c0@ericlaptop> Hey Karshi, I'm not sure what is causing this problem. One possible solution is to use SciPy 0.1, but install the plt package from the current CVS. You can do this by copying /scipy/plt directory into /usr/local/lib/python2.1/site-packages/scipy. Hopefully that'll solve the problem. see ya, eric ----- Original Message ----- From: "Karshi" To: Cc: Sent: Thursday, February 28, 2002 11:36 PM Subject: [SciPy-user] image_module > > Hi Eric, > > I couldn't get the CVS version of scipy work, and had go back to > the version 0.1. > The following code works: > ----------------------------------------------------------------- > from Numeric import * > import gui_thread > from scipy import plt > from scipy import * > > Img = zeros((256,256), Float32) > > for i in arange(256): > for j in arange(256): > r = sqrt(pow(i-128,2)+pow(j-128,2)) > > if r <= 60: > Img[i,j]= 100 > > #plt.image(Img) > > x = arange(0,5) > y = special.j1(x) > plt.plot(x,y) > > plt.image(Img) > plt.grid('off') > -------------------------------------------------------------------- > But if I take below lines out : > x = arange(0,5) > y = special.j1(x) > plt.plot(x,y) > ---------------- > It will give me errors like this: > >>> > Traceback (most recent call last): > File "/home/Study/Matlab/Sphere.py", line 15, in ? > plt.image(Img) > File "/usr/local/lib/python2.1/site-packages/scipy/plt/interface.py", line > 427, in image > axis('equal') > File "/usr/local/lib/python2.1/site-packages/scipy/plt/interface.py", line > 174, in axis > x_ticks = _active.x_axis.ticks > AttributeError: axis_window instance has no attribute 'ticks' > ------------------------------------------------------------- > I don't understand why ? > _______________________________________________ > SciPy-user mailing list > SciPy-user at scipy.net > http://www.scipy.net/mailman/listinfo/scipy-user > From Claes.Hagstrom at emw.ericsson.se Tue Mar 5 02:23:14 2002 From: Claes.Hagstrom at emw.ericsson.se (Claes Hagstrom) Date: Tue, 5 Mar 2002 08:23:14 +0100 (MET) Subject: [SciPy-user] Linear filter Message-ID: <15492.29282.978648.819967@pistage.mo.emw.ericsson.se> I recently downloaded SciPy-0.1, which I found very interesting. As I work with signal processing simulations, I tried the linear filter. However, there seems to be something wrong either with the filter or my way of using it. I tried the same simple calculations in Matlab as well and got the results I expected. I would appreciate your comments. Claes Hagstr?m ( claes.hagstrom at emw.ericsson.se ) --------------------------- SciPy: Matlab: from scipy import * b = [ 5 -1 ]; a = [ 1 0 ]; b = array([5.,-1.]) a = array([1.,0.]) x = [ 1 3 ; 5 7 ]; x(:,:,2) = [ 2 4 ; 6 8 ]; x = reshape(arange(8,typecode=Float)+1,(2,2,2)) c = filter(b,a,x,[],1); c = signal.lfilter(b,a,x,0) d = filter(b,a,x,[],2); d = signal.lfilter(b,a,x,1) e = filter(b,a,x,[],3); e = signal.lfilter(b,a,x,2) --------------------- >>> print a a = [ 1. 0.] >>> print b 1 0 [ 5. -1.] b = 5 -1 >>> print x[:,:,0] x(:,:,1) = [[ 1. 3.] [ 5. 7.]] 1 3 >>> print x[:,:,1] 5 7 [[ 2. 4.] [ 6. 8.]] x(:,:,2) = 2 4 6 8 >>> print c[:,:,0] c(:,:,1) = [[ 5. 0.] [ 25. 0.]] 5 15 >>> print c[:,:,1] 24 32 [[ 10. 20.] [ 28. 36.]] c(:,:,2) = 10 20 28 36 >>> print d[:,:,0] d(:,:,1) = [[ 5. 14.] [ 0. 35.]] 5 14 >>> print d[:,:,1] 25 30 [[ 10. 18.] [ 30. 34.]] >>> d(:,:,2) = 10 18 30 34 >>> print e[:,:,0] e(:,:,1) = [[ 5. 15.] [ 0. 35.]] 5 15 >>> print e[:,:,1] 25 35 [[ 9. 17.] [ 0. 33.]] e(:,:,2) = 9 17 25 33 From brad.reisfeld at colostate.edu Tue Mar 5 17:30:23 2002 From: brad.reisfeld at colostate.edu (Brad Reisfeld) Date: Tue, 5 Mar 2002 15:30:23 -0700 Subject: [SciPy-user] odeint limitations? Message-ID: Hi, I am a new user of SciPy. It looks like a very convenient method to access powerful numerical codes written in other languages from Python. At this point, I have an application in which I need to solve many (hundreds or thousands) of coupled ode's. Is the 'odeint' functionality in SciPy up to this task? I believe the code uses lsod(e|a|ar|es), which should, in theory, be adequate. Are there any 'issues' associated with the wrapping of these routines in Python? Anyone have experience using odeint for solving large numbers of ode's? Thanks. I appreciate the SciPy team for developing and enhancing this useful software for the Python user community. -Brad From oliphant.travis at ieee.org Wed Mar 6 00:29:06 2002 From: oliphant.travis at ieee.org (Travis Oliphant) Date: Tue, 5 Mar 2002 22:29:06 -0700 Subject: [SciPy-user] Linear filter In-Reply-To: <15492.29282.978648.819967@pistage.mo.emw.ericsson.se> References: <15492.29282.978648.819967@pistage.mo.emw.ericsson.se> Message-ID: On Tuesday 05 March 2002 12:23 am, you wrote: > I recently downloaded SciPy-0.1, which I found very interesting. As I > work with signal processing simulations, I tried the linear filter. > However, there seems to be something wrong either with the filter or > my way of using it. I tried the same simple calculations in Matlab as > well and got the results I expected. I would appreciate your comments. > You have found a bug in the multi-dimensional support in linear_filter. It has not been tested in some time and perhaps some of the changes to Python have created this problem. The filtering seems to work on one-dimensional arrays however. The N-dimensional looping seems to be broken, right now. I will look into it. Thank you for this report. -Travis O. From nwagner at mecha.uni-stuttgart.de Wed Mar 6 11:53:04 2002 From: nwagner at mecha.uni-stuttgart.de (Nils Wagner) Date: Wed, 06 Mar 2002 17:53:04 +0100 Subject: [SciPy-user] expm, sinm, .... Message-ID: <3C864970.23B93EA4@mecha.uni-stuttgart.de> Hi, >>> help(expm) expm(A, q=7) What is the meaning of the second argument q ? How do I compute the matrix exponential of a simple 2 \times 2 matrix a = [0. ,1.; -4.,0.] Nils From nwagner at mecha.uni-stuttgart.de Wed Mar 6 12:48:26 2002 From: nwagner at mecha.uni-stuttgart.de (Nils Wagner) Date: Wed, 06 Mar 2002 18:48:26 +0100 Subject: [SciPy-user] AttributeError: 'scipy.linalg' module has no attribute '_cast' Message-ID: <3C86566A.F6912D2C@mecha.uni-stuttgart.de> Hi, I tried to import linalg from scipy Python 2.1.2 (#1, Feb 25 2002, 18:04:21) [GCC 2.95.3 20010315 (SuSE)] on linux2 Type "copyright", "credits" or "license" for more information. >>> from scipy import * >>> from scipy.linalg import * Traceback (most recent call last): File "", line 1, in ? AttributeError: 'scipy.linalg' module has no attribute '_cast' >>> >>> from scipy.linalg import expm >>> Any idea ? Nils Moreover, the matrix functions do not work properly. >>> a = 1.*identity(2) >>> print sinm(a) array_from_pyobj:intent(inout) array must be contiguous and with a proper type and size. Traceback (most recent call last): File "", line 1, in ? File "/usr/local/lib/python2.1/site-packages/scipy/linalg/linear_algebra.py", line 777, in sinm return toreal(-0.5j*(expm(1j*A) - expm(-1j*A))) File "/usr/local/lib/python2.1/site-packages/scipy/linalg/linear_algebra.py", line 735, in expm F = solve(D,N) File "/usr/local/lib/python2.1/site-packages/scipy/linalg/linear_algebra.py", line 122, in solve results = lapack_routine(a, bt) clapack.error: failed in converting 2nd argument `b' of clapack.zgesv to C/Fortran array >>> From oliphant at ee.byu.edu Wed Mar 6 11:55:26 2002 From: oliphant at ee.byu.edu (Travis Oliphant) Date: Wed, 6 Mar 2002 11:55:26 -0500 (EST) Subject: [SciPy-user] expm, sinm, .... In-Reply-To: <3C864970.23B93EA4@mecha.uni-stuttgart.de> Message-ID: > Hi, > > >>> help(expm) > expm(A, q=7) > > What is the meaning of the second argument q ? It changes the approximation order of the Pade approximation. Don't worry about it if you don't understand. just put in the matrix. expm([[0,1],[-4,0]]) -Travis From oliphant at ee.byu.edu Wed Mar 6 11:55:26 2002 From: oliphant at ee.byu.edu (Travis Oliphant) Date: Wed, 6 Mar 2002 11:55:26 -0500 (EST) Subject: [SciPy-user] expm, sinm, .... In-Reply-To: <3C864970.23B93EA4@mecha.uni-stuttgart.de> Message-ID: > Hi, > > >>> help(expm) > expm(A, q=7) > > What is the meaning of the second argument q ? It changes the approximation order of the Pade approximation. Don't worry about it if you don't understand. just put in the matrix. expm([[0,1],[-4,0]]) -Travis From oliphant at ee.byu.edu Wed Mar 6 11:56:31 2002 From: oliphant at ee.byu.edu (Travis Oliphant) Date: Wed, 6 Mar 2002 11:56:31 -0500 (EST) Subject: [SciPy-user] AttributeError: 'scipy.linalg' module has no attribute '_cast' In-Reply-To: <3C86566A.F6912D2C@mecha.uni-stuttgart.de> Message-ID: > Any idea ? > > Nils > > Moreover, the matrix functions do not work properly. > > >>> a = 1.*identity(2) > >>> print sinm(a) > array_from_pyobj:intent(inout) array must be contiguous and with a > proper type and size. > Traceback (most recent call last): > File "", line 1, in ? > File > "/usr/local/lib/python2.1/site-packages/scipy/linalg/linear_algebra.py", > line 777, in sinm > return toreal(-0.5j*(expm(1j*A) - expm(-1j*A))) > File > "/usr/local/lib/python2.1/site-packages/scipy/linalg/linear_algebra.py", > line 735, in expm > F = solve(D,N) > File > "/usr/local/lib/python2.1/site-packages/scipy/linalg/linear_algebra.py", > line 122, in solve > results = lapack_routine(a, bt) > clapack.error: failed in converting 2nd argument `b' of clapack.zgesv to > C/Fortran array > >>> Unfortunately, I think that the changes to f2py have broken the linalg interfaces. Pearu is working on linalg2 which will replace linalg. -Travis From oliphant at ee.byu.edu Wed Mar 6 11:56:31 2002 From: oliphant at ee.byu.edu (Travis Oliphant) Date: Wed, 6 Mar 2002 11:56:31 -0500 (EST) Subject: [SciPy-user] AttributeError: 'scipy.linalg' module has no attribute '_cast' In-Reply-To: <3C86566A.F6912D2C@mecha.uni-stuttgart.de> Message-ID: > Any idea ? > > Nils > > Moreover, the matrix functions do not work properly. > > >>> a = 1.*identity(2) > >>> print sinm(a) > array_from_pyobj:intent(inout) array must be contiguous and with a > proper type and size. > Traceback (most recent call last): > File "", line 1, in ? > File > "/usr/local/lib/python2.1/site-packages/scipy/linalg/linear_algebra.py", > line 777, in sinm > return toreal(-0.5j*(expm(1j*A) - expm(-1j*A))) > File > "/usr/local/lib/python2.1/site-packages/scipy/linalg/linear_algebra.py", > line 735, in expm > F = solve(D,N) > File > "/usr/local/lib/python2.1/site-packages/scipy/linalg/linear_algebra.py", > line 122, in solve > results = lapack_routine(a, bt) > clapack.error: failed in converting 2nd argument `b' of clapack.zgesv to > C/Fortran array > >>> Unfortunately, I think that the changes to f2py have broken the linalg interfaces. Pearu is working on linalg2 which will replace linalg. -Travis From eric at scipy.org Wed Mar 6 22:31:39 2002 From: eric at scipy.org (eric) Date: Wed, 6 Mar 2002 22:31:39 -0500 Subject: [SciPy-user] odeint limitations? References: Message-ID: <018301c1c588$97decdf0$6b01a8c0@ericlaptop> Hey Brad, I haven't used the ODE stuff much. Hopefully someone else will have an answer for you. eric ----- Original Message ----- From: "Brad Reisfeld" To: Sent: Tuesday, March 05, 2002 5:30 PM Subject: [SciPy-user] odeint limitations? > Hi, > I am a new user of SciPy. It looks like a very convenient method to access > powerful numerical codes written in other languages from Python. > > At this point, I have an application in which I need to solve many (hundreds > or thousands) of coupled ode's. Is the 'odeint' functionality in SciPy up to > this task? I believe the code uses lsod(e|a|ar|es), which should, in theory, > be adequate. Are there any 'issues' associated with the wrapping of these > routines in Python? Anyone have experience using odeint for solving large > numbers of ode's? > > Thanks. > > I appreciate the SciPy team for developing and enhancing this useful > software for the Python user community. > > -Brad > > _______________________________________________ > SciPy-user mailing list > SciPy-user at scipy.net > http://www.scipy.net/mailman/listinfo/scipy-user > From andybee at brain.ncl.ac.uk Thu Mar 7 07:22:37 2002 From: andybee at brain.ncl.ac.uk (andybee) Date: Thu, 07 Mar 2002 12:22:37 +0000 Subject: [SciPy-user] ImportError: No module named scipy Message-ID: <3C875B8D.4030504@brain.ncl.ac.uk> Dear All I have installed Active Python 2.1 on a mandrake 8.1 linuk box, but when I type from scipy import * I get Traceback (most recent call last): File "", line 1, in ? ImportError: No module named scipy With scipy in the default install positon, and the position is in sys.path. Can anyone help me? Andybee From nwagner at mecha.uni-stuttgart.de Thu Mar 7 08:36:30 2002 From: nwagner at mecha.uni-stuttgart.de (Nils Wagner) Date: Thu, 07 Mar 2002 14:36:30 +0100 Subject: [SciPy-user] Re: expm, funm References: Message-ID: <3C876CDE.C0CE47EF@mecha.uni-stuttgart.de> Travis Oliphant schrieb: > > > Any idea ? > > > > Nils > > > > Moreover, the matrix functions do not work properly. > > > > >>> a = 1.*identity(2) > > >>> print sinm(a) > > array_from_pyobj:intent(inout) array must be contiguous and with a > > proper type and size. > > Traceback (most recent call last): > > File "", line 1, in ? > > File > > "/usr/local/lib/python2.1/site-packages/scipy/linalg/linear_algebra.py", > > line 777, in sinm > > return toreal(-0.5j*(expm(1j*A) - expm(-1j*A))) > > File > > "/usr/local/lib/python2.1/site-packages/scipy/linalg/linear_algebra.py", > > line 735, in expm > > F = solve(D,N) > > File > > "/usr/local/lib/python2.1/site-packages/scipy/linalg/linear_algebra.py", > > line 122, in solve > > results = lapack_routine(a, bt) > > clapack.error: failed in converting 2nd argument `b' of clapack.zgesv to > > C/Fortran array > > >>> > > Unfortunately, I think that the changes to f2py have broken the linalg > interfaces. Pearu is working on linalg2 which will replace linalg. > > -Travis > > _______________________________________________ > SciPy-user mailing list > SciPy-user at scipy.net > http://www.scipy.net/mailman/listinfo/scipy-user Travis, Finally, I have used the latest CVS. It turned out all right. BTW, how can I invoke funm for the computation of an arbitrary matrix function ? A small example would be very helpful. >>> help(funm) funm(A, func, disp=1) Matrix function for arbitrary callable object func. Moreover I agree to submit algorithms for both sqrtm and logm. How can I contribute ? Nils From nwagner at mecha.uni-stuttgart.de Thu Mar 7 08:36:30 2002 From: nwagner at mecha.uni-stuttgart.de (Nils Wagner) Date: Thu, 07 Mar 2002 14:36:30 +0100 Subject: [SciPy-user] Re: expm, funm References: Message-ID: <3C876CDE.C0CE47EF@mecha.uni-stuttgart.de> Travis Oliphant schrieb: > > > Any idea ? > > > > Nils > > > > Moreover, the matrix functions do not work properly. > > > > >>> a = 1.*identity(2) > > >>> print sinm(a) > > array_from_pyobj:intent(inout) array must be contiguous and with a > > proper type and size. > > Traceback (most recent call last): > > File "", line 1, in ? > > File > > "/usr/local/lib/python2.1/site-packages/scipy/linalg/linear_algebra.py", > > line 777, in sinm > > return toreal(-0.5j*(expm(1j*A) - expm(-1j*A))) > > File > > "/usr/local/lib/python2.1/site-packages/scipy/linalg/linear_algebra.py", > > line 735, in expm > > F = solve(D,N) > > File > > "/usr/local/lib/python2.1/site-packages/scipy/linalg/linear_algebra.py", > > line 122, in solve > > results = lapack_routine(a, bt) > > clapack.error: failed in converting 2nd argument `b' of clapack.zgesv to > > C/Fortran array > > >>> > > Unfortunately, I think that the changes to f2py have broken the linalg > interfaces. Pearu is working on linalg2 which will replace linalg. > > -Travis > > _______________________________________________ > SciPy-user mailing list > SciPy-user at scipy.net > http://www.scipy.net/mailman/listinfo/scipy-user Travis, Finally, I have used the latest CVS. It turned out all right. BTW, how can I invoke funm for the computation of an arbitrary matrix function ? A small example would be very helpful. >>> help(funm) funm(A, func, disp=1) Matrix function for arbitrary callable object func. Moreover I agree to submit algorithms for both sqrtm and logm. How can I contribute ? Nils From andybee at brain.ncl.ac.uk Thu Mar 7 07:52:57 2002 From: andybee at brain.ncl.ac.uk (andybee) Date: Thu, 07 Mar 2002 12:52:57 +0000 Subject: [SciPy-user] ImportError: No module named fastumath Message-ID: <3C8762A9.8050309@brain.ncl.ac.uk> Dear All After a little bit of trial and error I found that being loged in as root and running python I could get from scipy import * to work However I now get File "", line 1, in ? File "/usr/local/lib/python2.1/site-packages/scipy/__init__.py", line 42, in ? from misc import * File "/usr/local/lib/python2.1/site-packages/scipy/misc.py", line 15, in ? import fastumath ImportError: No module named fastumath ??? Can anyone help? Andybee From pearu at cens.ioc.ee Thu Mar 7 09:24:33 2002 From: pearu at cens.ioc.ee (Pearu Peterson) Date: Thu, 7 Mar 2002 16:24:33 +0200 (EET) Subject: [SciPy-user] ImportError: No module named fastumath In-Reply-To: <3C8762A9.8050309@brain.ncl.ac.uk> Message-ID: On Thu, 7 Mar 2002, andybee wrote: > Can anyone help? How did you install scipy? What version? Are you sure that you have build scipy? Was it succesful if yes? Pearu From oliphant at ee.byu.edu Thu Mar 7 10:02:44 2002 From: oliphant at ee.byu.edu (Travis Oliphant) Date: Thu, 7 Mar 2002 10:02:44 -0500 (EST) Subject: [SciPy-user] odeint limitations? In-Reply-To: <018301c1c588$97decdf0$6b01a8c0@ericlaptop> Message-ID: > > Hi, > > I am a new user of SciPy. It looks like a very convenient method to access > > powerful numerical codes written in other languages from Python. > > > > At this point, I have an application in which I need to solve many (hundreds > > or thousands) of coupled ode's. Is the 'odeint' functionality in SciPy up to > > this task? I believe the code uses lsod(e|a|ar|es), which should, in theory, > > be adequate. Are there any 'issues' associated with the wrapping of these > > routines in Python? Anyone have experience using odeint for solving large > > numbers of ode's? > > I've not tried it for such large scale problems (I've only gone to 4 or 5 coupled ode's). As you say, the underlying code should handle it. The only question is how long will it take. Using Numeric arrays in your function to compute the derivative is definitely recommended, to save space. Why don't you give it a try? -Travis Oliphant From oliphant at ee.byu.edu Thu Mar 7 10:27:16 2002 From: oliphant at ee.byu.edu (Travis Oliphant) Date: Thu, 7 Mar 2002 10:27:16 -0500 (EST) Subject: [SciPy-user] Re: expm, funm In-Reply-To: <3C876CDE.C0CE47EF@mecha.uni-stuttgart.de> Message-ID: > Travis, > > Finally, I have used the latest CVS. It turned out all right. > > BTW, how can I invoke funm for the computation of an arbitrary matrix > function ? > A small example would be very helpful. > > >>> help(funm) > funm(A, func, disp=1) > > Matrix function for arbitrary callable object func. func is just an arbitrary callable object that will be called on the schur values. The only restriction is that it needs to take a vector of arguments and return the function evaluated elementwise over that vector. (See special.genera_function) if you want to "vectorize" a function. So, for example from scipy import * linalg.funm(A, sqrt) should compute the matrix square-root (the algorithm is general purpose and can fail). The disp flag controls whether or not a warning is printed if something doesn't seem right. > > > Moreover I agree to submit algorithms for both sqrtm and logm. > How can I contribute ? > Send them to me. It would be preferrable if there is a test sent along with the algorithm. -Travis From oliphant at ee.byu.edu Thu Mar 7 10:27:16 2002 From: oliphant at ee.byu.edu (Travis Oliphant) Date: Thu, 7 Mar 2002 10:27:16 -0500 (EST) Subject: [SciPy-user] Re: expm, funm In-Reply-To: <3C876CDE.C0CE47EF@mecha.uni-stuttgart.de> Message-ID: > Travis, > > Finally, I have used the latest CVS. It turned out all right. > > BTW, how can I invoke funm for the computation of an arbitrary matrix > function ? > A small example would be very helpful. > > >>> help(funm) > funm(A, func, disp=1) > > Matrix function for arbitrary callable object func. func is just an arbitrary callable object that will be called on the schur values. The only restriction is that it needs to take a vector of arguments and return the function evaluated elementwise over that vector. (See special.genera_function) if you want to "vectorize" a function. So, for example from scipy import * linalg.funm(A, sqrt) should compute the matrix square-root (the algorithm is general purpose and can fail). The disp flag controls whether or not a warning is printed if something doesn't seem right. > > > Moreover I agree to submit algorithms for both sqrtm and logm. > How can I contribute ? > Send them to me. It would be preferrable if there is a test sent along with the algorithm. -Travis From pearu at cens.ioc.ee Thu Mar 7 13:28:55 2002 From: pearu at cens.ioc.ee (Pearu Peterson) Date: Thu, 7 Mar 2002 20:28:55 +0200 (EET) Subject: [SciPy-user] odeint limitations? In-Reply-To: Message-ID: Hi, I have used regularly vode for tasks with 256 upto 12288 coupled nonlinear ordinary diff. equations. Scipy has interface to that routine implemented in integrate.ode. The question of how long it would take does not depend on the interface, it depends 1) on the size of the problem 2) on the properties of the system (stiff, non-stiff) 3) on whether you have choosen the right solver for the problem and tuned its parameters to optimal (again, depends on the system) 4) on how efficiently you can calculate the right-hand-side of the equations 5) on the hardware you have etc. And when the problem is of the size of hundred or thousands equations, it is definitely _not_ recommended to calculate the jacobian in Python or even in C or Fortran, _unless_, the jacobian is banded and really simple. >From my experience it is best to let the integrator to find the approximation to the jacobian matrix and not too wide. There are very good (fast and accurate) integrators in vode that even don't need jacobian, but it again depends on the propertis of the ODE system. My advice would be to try different integrators on a relatively small problem (say few hundreds of equations) and tune the integrator parameters for the best performance (it is usually a good idea to keep accuracy high as for large problems smaller accuracy introduces quickly numerical errors and the integration becomes very slow or does not converge at all). And after that, run the integrator on a large problem. Regards, Pearu On Thu, 7 Mar 2002, Travis Oliphant wrote: > > > Hi, > > > I am a new user of SciPy. It looks like a very convenient method to access > > > powerful numerical codes written in other languages from Python. > > > > > > At this point, I have an application in which I need to solve many (hundreds > > > or thousands) of coupled ode's. Is the 'odeint' functionality in SciPy up to > > > this task? I believe the code uses lsod(e|a|ar|es), which should, in theory, > > > be adequate. Are there any 'issues' associated with the wrapping of these > > > routines in Python? Anyone have experience using odeint for solving large > > > numbers of ode's? > > > > > > I've not tried it for such large scale problems (I've only gone to 4 or 5 > coupled ode's). > > As you say, the underlying code should handle it. > > The only question is how long will it take. Using Numeric arrays in your > function to compute the derivative is definitely recommended, to save > space. > > Why don't you give it a try? > > -Travis Oliphant > > > _______________________________________________ > SciPy-user mailing list > SciPy-user at scipy.net > http://www.scipy.net/mailman/listinfo/scipy-user > From wagner.nils at vdi.de Thu Mar 7 14:56:18 2002 From: wagner.nils at vdi.de (My VDI Freemail) Date: Thu, 07 Mar 2002 20:56:18 +0100 Subject: [SciPy-user] Re: Problems with expm, expm2 In-Reply-To: Message-ID: <200203071950.g27JoG626520@scipy.org> ------------------- > > Dear Travis, > > > > I have tried several standard test problems (see references) . > > Unfortunately, I never get the correct result for expm and expm2. > > This is most likely a problem with the linear algebra interface still. > > Most likely the solve is not working (could you test that on your system)? > > I have tested these routines and am pretty sure that they work.. > > I used the routines from Matrix Computations. > > > -Travis > > Hi Travis, I have used a symmetric test matrix. It works fine. Therefore it seems to be a problem of non-symmetric matrices. What do you think ? Nils -------------- next part -------------- A non-text attachment was scrubbed... Name: testerg.dat Type: application/octet-stream Size: 1840 bytes Desc: not available URL: From oliphant at ee.byu.edu Thu Mar 7 16:29:05 2002 From: oliphant at ee.byu.edu (Travis Oliphant) Date: Thu, 7 Mar 2002 16:29:05 -0500 (EST) Subject: [SciPy-user] Re: Problems with expm, expm2 In-Reply-To: <200203071956.g27JuQx18446@mail1.ee.byu.edu> Message-ID: > Hi Travis, > > I have used a symmetric test matrix. It works fine. > Therefore it seems to be a problem of non-symmetric matrices. > > What do you think ? > I think the problem is with linalg. The new f2py messed up the interface so that the lapack calls no longer work as expected. In order to get them to work, you have to transpose the data at the appropriate places. So, of course, a symmetric matrix would not be affected by this error. I'll try and fix those hot spots with a quick hack and then see if it works. Again, I'm almost 100% positive that your problem is with the linalg routines (solve in particular). -Travis From brad.reisfeld at colostate.edu Sat Mar 9 13:29:53 2002 From: brad.reisfeld at colostate.edu (Brad Reisfeld) Date: Sat, 9 Mar 2002 11:29:53 -0700 Subject: [SciPy-user] odeint limitations? Message-ID: Thank you for the useful feedback. Travis, I will try out the standard odeint with lsode on a fairly large set of ode's I have. I really wanted to know if there were any known issues here, so that I could begin to look for alternatives, instead of finding out later what others had discovered early on. Pearu, I was not able to find the interface to vode anywhere. Please give me some pointers on how I access this function or library. I have the vode source code. What do I do from there? Thanks. -Brad From pearu at cens.ioc.ee Sat Mar 9 15:49:13 2002 From: pearu at cens.ioc.ee (Pearu Peterson) Date: Sat, 9 Mar 2002 22:49:13 +0200 (EET) Subject: [SciPy-user] odeint limitations? In-Reply-To: Message-ID: On Sat, 9 Mar 2002, Brad Reisfeld wrote: > Pearu, I was not able to find the interface to vode anywhere. Please give me > some pointers on how I access this function or library. I have the vode > source code. What do I do from there? I assume that you use Scipy-0.2 from CVS. There is no vode support in Scipy-0.1. So, the first pointer is >>> import scipy >>> print scipy.integrate.ode.__doc__ ode - a generic interface class to numeric integrators. It has the following methods: integrator = ode(f,jac=None) integrator = integrator.set_integrator(name,**params) integrator = integrator.set_initial_value(y0,t0=0.0) integrator = integrator.set_f_params(*args) integrator = integrator.set_jac_params(*args) y1 = integrator.integrate(t1,step=0,relax=0) flag = integrator.successful() Typical usage: r = ode(f,jac).set_integrator('vode').set_initial_value(y0,t0) t1 = dt = while r.successful() and r.t < t1: r.integrate(r.t+dt) print r.t, r.y where f and jac have the following signatures: def f(t,y[,arg1,..]): return def jac(t,y[,arg1,..]): return >>> Second, you can take a look at the source code in scipy/integrate/ode.py There are some general notes in the header. The file contains two simple examples, you should also study these. In fact, you don't need scipy to run vode from python but then you need to have the following files from scipy/integrate/ directory: vode.pyf - this must be compiled with f2py. ode.py - high level interface to vode function vode.f + some blas/lapack functions. Then you can run f2py -c vode.pyf vode.f -L/path/to/atlas -lf77blas -lcblas -latlas -lg2c that will build vode.so extension module that is used by ode.py. Pearu From nwagner at mecha.uni-stuttgart.de Wed Mar 13 12:15:49 2002 From: nwagner at mecha.uni-stuttgart.de (Nils Wagner) Date: Wed, 13 Mar 2002 18:15:49 +0100 Subject: [SciPy-user] funm AttributeError: typecode Message-ID: <3C8F8945.7F952F52@mecha.uni-stuttgart.de> File "/usr/local/lib/python2.1/site-packages/scipy/linalg/linear_algebra.py", line 836, in funm tol = {0:feps, 1:eps}[_array_precision[A.typecode()]] AttributeError: typecode For what reason ? Nils From pearu at cens.ioc.ee Wed Mar 13 13:01:33 2002 From: pearu at cens.ioc.ee (Pearu Peterson) Date: Wed, 13 Mar 2002 20:01:33 +0200 (EET) Subject: [SciPy-user] funm AttributeError: typecode In-Reply-To: <3C8F8945.7F952F52@mecha.uni-stuttgart.de> Message-ID: On Wed, 13 Mar 2002, Nils Wagner wrote: > > File > "/usr/local/lib/python2.1/site-packages/scipy/linalg/linear_algebra.py", > line 836, in funm > tol = {0:feps, 1:eps}[_array_precision[A.typecode()]] > AttributeError: typecode > > > For what reason ? May be because A is not a Numeric array? The current linalg has in many places where asarray is not applied to arguments, arguments are assumed to be Numeric arrays. Pearu From p.Collard at i-net.paiko.gr Wed Mar 13 13:35:36 2002 From: p.Collard at i-net.paiko.gr (p.Collard at i-net.paiko.gr) Date: Wed, 13 Mar 2002 15:35:36 -0300 (BRT) Subject: [SciPy-user] Minimize your phone expenses Message-ID: <1016044590.0169378903@lemnos.geo.auth.gr> An HTML attachment was scrubbed... URL: From nwagner at mecha.uni-stuttgart.de Thu Mar 14 12:22:28 2002 From: nwagner at mecha.uni-stuttgart.de (Nils Wagner) Date: Thu, 14 Mar 2002 18:22:28 +0100 Subject: [SciPy-user] funm AttributeError: typecode References: Message-ID: <3C90DC54.C4A3F00@mecha.uni-stuttgart.de> Pearu Peterson schrieb: > > On Wed, 13 Mar 2002, Nils Wagner wrote: > > > > > File > > "/usr/local/lib/python2.1/site-packages/scipy/linalg/linear_algebra.py", > > line 836, in funm > > tol = {0:feps, 1:eps}[_array_precision[A.typecode()]] > > AttributeError: typecode > > > > > > For what reason ? > > May be because A is not a Numeric array? The current linalg has in many > places where asarray is not applied to arguments, arguments are assumed to > be Numeric arrays. > > Pearu > > _______________________________________________ > SciPy-user mailing list > SciPy-user at scipy.net > http://www.scipy.net/mailman/listinfo/scipy-user Again, I can give no account of it. funm([[1.,0.],[0.,1.]], exp) File "/usr/local/lib/python2.1/site-packages/scipy/linalg/linear_algebra.py", line 836, in funm tol = {0:feps, 1:eps}[_array_precision[A.typecode()]] AttributeError: typecode Nils From fperez at pizero.colorado.edu Thu Mar 14 11:48:15 2002 From: fperez at pizero.colorado.edu (Fernando Perez) Date: Thu, 14 Mar 2002 09:48:15 -0700 (MST) Subject: [SciPy-user] funm AttributeError: typecode In-Reply-To: <3C90DC54.C4A3F00@mecha.uni-stuttgart.de> Message-ID: On Thu, 14 Mar 2002, Nils Wagner wrote: > Again, I can give no account of it. > > funm([[1.,0.],[0.,1.]], exp) > File > "/usr/local/lib/python2.1/site-packages/scipy/linalg/linear_algebra.py", > line 836, in funm > tol = {0:feps, 1:eps}[_array_precision[A.typecode()]] > AttributeError: typecode There's a difference between python lists and Numeric arrays: In [1]: import Numeric In [2]: A_list = [[1.,0.],[0.,1.]] In [3]: A_array = Numeric.array(A_list) In [4]: A_list Out[4]: [[1.0, 0.0], [0.0, 1.0]] In [5]: A_array Out[5]: array([[ 1., 0.], [ 0., 1.]]) In [6]: A_list.typecode() --------------------------------------------------------------------------- AttributeError Traceback (most recent call last) ? AttributeError: 'list' object has no attribute 'typecode' In [7]: A_array.typecode ------> A_array.typecode () Out[7]: 'd' Cheers, f. From nwagner at mecha.uni-stuttgart.de Thu Mar 14 13:02:38 2002 From: nwagner at mecha.uni-stuttgart.de (Nils Wagner) Date: Thu, 14 Mar 2002 19:02:38 +0100 Subject: [SciPy-user] Availability of linalg2 Message-ID: <3C90E5BE.13BB76FD@mecha.uni-stuttgart.de> Hi all, There is still an inconsistency in the computation of matrix functions (expm, expm2, expm3). Is there any bug fix available or shall I wait for linalg 2 ? Nils From pearu at cens.ioc.ee Thu Mar 14 13:14:17 2002 From: pearu at cens.ioc.ee (Pearu Peterson) Date: Thu, 14 Mar 2002 20:14:17 +0200 (EET) Subject: [SciPy-user] Availability of linalg2 In-Reply-To: <3C90E5BE.13BB76FD@mecha.uni-stuttgart.de> Message-ID: On Thu, 14 Mar 2002, Nils Wagner wrote: > Hi all, > > There is still an inconsistency in the computation of matrix functions > (expm, expm2, expm3). Sorry, I have been busy with my real work and have not had time to contribute to linalg2 for awhile. > Is there any bug fix available or shall I wait for linalg 2 ? You could use expm3 that should give correct results, but it might be slow. That is until linalg2 is ready. Pearu From z1z2 at sympatico.ca Sun Mar 17 04:41:03 2002 From: z1z2 at sympatico.ca (MG Publishing) Date: Sun, 17 Mar 2002 04:41:03 -0500 Subject: [SciPy-user] Available; Subsidies, Grants, Loans, Financing. Message-ID: <200203170936.g2H9a6620018@scipy.org> MG PUBLISHING 916 Ste-Adele Blvd. Ste-Adele, Qc Canada J8B 2N2 PRESS RELEASE CANADIAN SUBSIDY DIRECTORY 2002 Legal deposit-National Library of Canada : ISBN 2-922-870-02-02 M.G. Publishing is offering to the public a revised edition of the Canadian Subsidy Directory, a guide containing 2867 direct and indirect financial subsidies, grants and loans offered by government departments and agencies, foundations, associations and organizations. In this new 2002 edition all programs are well described. The Canadian Subsidy Directory is the most comprehensive tool to start up a business, improve existent activities, set up a business plan, or obtain assistance from experts in fields such as: Industry, transport, agriculture, communications, municipal infrastructure, education, import-export, labor, construction and renovation, the service sector, hi-tech industries, research and development, joint ventures, arts, cinema, theatre, music and recording industry, the self employed, contests, and new talents. Assistance from and for foundations and associations, guidance to prepare a business plan, market surveys, computers, and much more! Retail price.....$ 49.95 plus taxes, shipping and handling. To obtain a copy of the Canadian Subsidy Directory please contact one the following resellers. Canadian Business Ressource Center: (250) 381-4822 Fureteur bookstore: (450) 465-5597, fax: (credit cards only) (450) 465-8144 From fperez at pizero.colorado.edu Tue Mar 19 03:23:55 2002 From: fperez at pizero.colorado.edu (Fernando Perez) Date: Tue, 19 Mar 2002 01:23:55 -0700 (MST) Subject: [SciPy-user] ANN: IPython 0.2.8 Message-ID: Hi folks, I've put up a new release of IPython which incorporates many recent fixes and improvements which may be of interest to the scipy crowd. Since there's been interest in using IPython as part of scipy as a user front-end, I'd appreciate comments/suggestions from you. There have already been many useful ones, which I've tried to respond to and incorporate whenever feasible. For those who aren't familiar with what IPython is, some info follows. * WHERE: http://www-hep.colorado.edu/~fperez/ipython * NEW since last public release (ChangeLog included with full details): - Full Emacs integration. Now IPython can be used both in Emacs shell/term buffers and also as your Python interpreter (with C-c !). Instructions on the necessary code for your .emacs file provided. - Interface to Gnuplot. This can be used as a template for interfacing with any other program which one can communicate with (via pipes, sockets, whatever). If anyone cares to do it, it might be nice to implement a MathLink channel for Mathematica, for example. - Many improvements to the embedding facilities. It's now possible to have nested copies of IPython running (such as debugging code with embedded IPython inside IPython itself). Global and local locks for the embedded instances provided to control them better, and the user prompts can be modified for each embedded instance. - Improved macro system for automatically re-executing many lines of previous input. - Better control for running other programs inside IPython via @run. - New -upgrade command line option for old users to update their .ipython/ directory automatically. - Many small bugfixes, documentation very updated. * WHAT is IPython? (this is just for context. See the webpage for full details). IPython tries to: 1. Provide an interactive interpreter superior to Python's default. IPython has many features for object introspection, shell access, and its own special command system for adding functionality when working interactively. 2. Serve as an embeddable, ready to use interpreter for your own programs. IPython can be started with a single call from inside another program, providing access to the current namespace. This can be very useful both for debugging purposes and for situations where a blend of batch-processing and interactive exploration are needed. 3. Offer a flexible framework which can be used as the base environment for other systems with Python as the underlying language. Specifically scientific environments like Mathematica, IDL and Matlab inspired its design, but similar ideas can be useful in many fields. * PORTABILITY: Linux (and other unices, including Mac OSX), Windows XP. Should run fine on all WinNT/2K, and probably also on Win9x (I can't test this). * PYTHON VERSION: requires 2.1 or newer. * LICENSE: LGPL (a few files from third parties carry MIT licenses). Note that for the next major release (which will include an internal rewrite), I've already discussed with Eric a licensing change to bring it in line with SciPy. So please consider this to be a non-issue. Cheers, Fernando Perez. From a.schmolck at gmx.net Fri Mar 22 12:01:48 2002 From: a.schmolck at gmx.net (A.Schmolck) Date: 22 Mar 2002 17:01:48 +0000 Subject: [SciPy-user] Problem with Mplt.py in non-interactive use Message-ID: Hi, I've got a problem with Mplot that I find somewhat baffling: what works fine in interactively on a step-by-step basis doesn't necessarily in non-interactive sessions. For example the following code-fragement will do what one would expect if one executes it line by line (i.e. it draws a plot, then waits the specified time). If I execute it as a whole with "python TestMplot.py", however, it doesn't plot anything, it just fires up an empty window that stays empty (despite the fact that the sleep commands should give any hypothetical concurrent process plenty of time to do whatever it wants). If I use py-shell from emacs or python "-i TestMplot.py", the same happens, except that once the interactive prompt comes back something is actually plotted. I also experienced the more serious problem that something that worked step-by-step in an interactive plot causes: gist.error: Gist GdLines plotter failed after which all line plotting fails... Any idea why that might be the case and what could be done about it? I currently really fail to see what might be going on... ----TestMplot.py---- from scipy.xplt import Mplot as plt import time from Numeric import * print "TESTING" plt.plot([1,2,3], [1,2,3]) time.sleep(2) plt.plot([1,2,3], [1,2,3], "gx") time.sleep(2) plt.plot([1,2,3], [1,2,3], "gx", [1,2,3], [4,6,7], "r-") time.sleep(2) plt.imagesc(reshape(arange(12), (3,4))) time.sleep(5) -------------------- Thanks, alex P.S.: I'm using a recent (last week's, I think) CVS version of scipy -- Alexander Schmolck Postgraduate Research Student Department of Computer Science University of Exeter A.Schmolck at gmx.net http://www.dcs.ex.ac.uk/people/aschmolc/ From nwagner at mecha.uni-stuttgart.de Mon Mar 25 12:39:16 2002 From: nwagner at mecha.uni-stuttgart.de (Nils Wagner) Date: Mon, 25 Mar 2002 18:39:16 +0100 Subject: [SciPy-user] vode.so: undefined symbol: f2py_report_on_exit Message-ID: <3C9F60C4.C2586CFA@mecha.uni-stuttgart.de> After installing the latest SciPy via CVS this is what happens when I start a small test program for matrix functions >python Python 2.1.2 (#1, Feb 25 2002, 18:04:21) [GCC 2.95.3 20010315 (SuSE)] on linux2 Type "copyright", "credits" or "license" for more information. >>> from scipy import * /usr/local/lib/python2.1/site-packages/scipy/integrate/vode.so: undefined symbol: f2py_report_on_exit >>> Any idea ? Nils From pearu at cens.ioc.ee Mon Mar 25 12:07:32 2002 From: pearu at cens.ioc.ee (Pearu Peterson) Date: Mon, 25 Mar 2002 19:07:32 +0200 (EET) Subject: [SciPy-user] vode.so: undefined symbol: f2py_report_on_exit In-Reply-To: <3C9F60C4.C2586CFA@mecha.uni-stuttgart.de> Message-ID: On Mon, 25 Mar 2002, Nils Wagner wrote: > After installing the latest SciPy via CVS this is what happens when > I start a small test program for matrix functions > > >python > Python 2.1.2 (#1, Feb 25 2002, 18:04:21) > [GCC 2.95.3 20010315 (SuSE)] on linux2 > Type "copyright", "credits" or "license" for more information. > >>> from scipy import * > /usr/local/lib/python2.1/site-packages/scipy/integrate/vode.so: > undefined symbol: f2py_report_on_exit > >>> > > Any idea ? This bug occured when one of the SciPy modules disabled f2py_report_at_exit feature but it remained in others. Since all modules use the same fortranobject.o, where symbol f2py_report_on_exit should be defined (but after the disable it is not there), then you'll get this undefined symbol error for modules that have the f2py_report_at_exit enabled. As a fix, get the latest f2py from its CVS (that has f2py_report_at_exit feature disabled by default, I'll make new snapshot available this evening) and rebuild all extension modules (removing the build/ directory is the best thing to do in this situation). Regards, Pearu From oliphant.travis at ieee.org Mon Mar 25 20:46:41 2002 From: oliphant.travis at ieee.org (Travis Oliphant) Date: Mon, 25 Mar 2002 18:46:41 -0700 Subject: [SciPy-user] Linalg should work in latest CVS tree with latest f2py Message-ID: Users, There has been some difficulty with the linalg subpackage and the newest f2py (particularly effecting people who wanted to use matrix exponential algorithm). This has been resolved. You have two options. If your ATLAS is older than 3.3.13 and you don't want to upgrade your ATLAS installation (e.g. because it takes awhile to compile binaries), then you can get the latest CVS, build scipy, and use the linalg algorithms. If you want to go ahead and get ATLAS > 3.3.13 then you can use linalg2 (not imported by default --- you have to specifically import it (e.g. import scipy.linalg2 as linalg2) Not all algorithms are in linalg2 as of yet, but it won't be long. Thanks for your patience. -Travis Oliphant From oliphant.travis at ieee.org Mon Mar 25 20:51:34 2002 From: oliphant.travis at ieee.org (Travis Oliphant) Date: Mon, 25 Mar 2002 18:51:34 -0700 Subject: [SciPy-user] FFTW no longer required Message-ID: In recent versions of the SciPy CVS tree, FFTW is no longer required. It is not built by default and FFTPACK is used instead for fft algorithms. This results in a 5~25% slowdown for fft computations (particularly if your image sizes are products of large prime numbers >5). You can still build the fftw interface by modifying setup.py. -Travis From oliphant.travis at ieee.org Mon Mar 25 20:53:35 2002 From: oliphant.travis at ieee.org (Travis Oliphant) Date: Mon, 25 Mar 2002 18:53:35 -0700 Subject: [SciPy-user] Default functions return complex numbers Message-ID: The functions immediately under the scipy namespace now return complex numbers when necessary. For example: scipy.log(-1) returns pi*1j scipy.sqrt(-1) returns 1j The old behavior for those routines is available as scipy.fastumath.log scipy.fastumath.sqrt etc. -Travis O. From nwagner at mecha.uni-stuttgart.de Tue Mar 26 04:04:42 2002 From: nwagner at mecha.uni-stuttgart.de (Nils Wagner) Date: Tue, 26 Mar 2002 10:04:42 +0100 Subject: [SciPy-user] Linalg should work in latest CVS tree with latest f2py References: Message-ID: <3CA039AA.38E5B16D@mecha.uni-stuttgart.de> Travis Oliphant schrieb: > > Users, > > There has been some difficulty with the linalg subpackage and the newest f2py > (particularly effecting people who wanted to use matrix exponential > algorithm). > > This has been resolved. You have two options. > > If your ATLAS is older than 3.3.13 and you don't want to upgrade your ATLAS > installation (e.g. because it takes awhile to compile binaries), then you can > get the latest CVS, build scipy, and use the linalg algorithms. > > If you want to go ahead and get ATLAS > 3.3.13 then you can use linalg2 (not > imported by default --- you have to specifically import it (e.g. import > scipy.linalg2 as linalg2) > > Not all algorithms are in linalg2 as of yet, but it won't be long. > > Thanks for your patience. > > -Travis Oliphant > _______________________________________________ > SciPy-user mailing list > SciPy-user at scipy.net > http://www.scipy.net/mailman/listinfo/scipy-user After installing the latest SciPy via CVS this is what happens when I start a small test program for matrix functions Matrix function for arbitrary callable object func. array_from_pyobj:FromDims failed: intent(hide) Traceback (most recent call last): File "exptest.py", line 42, in ? print funm(identity(2)*1.0 , exp) File "/usr/local/lib/python2.1/site-packages/scipy/linalg/linear_algebra.py", line 836, in funm Z, T = schur(A) File "/usr/local/lib/python2.1/site-packages/scipy/linalg/linear_algebra.py", line 315, in schur result = lapack_routine(lambda x: None, a, lwork=-1) # query MemoryError: can't allocate memory for array Moreover, there is still a problem concering the computation of the matrix exponential by using a Pade approximation (expm(A)). Any suggestion ? Nils from scipy import * from scipy.linalg import expm, expm2, expm3, funm # # page 820 Cleve Moler and Charles van Loan Nineteen dubious ways ... # eps = 1.e-5 a = array([[1.+eps,1.0],[0.,1.-eps]]) # # Pade approximant # help (expm) print expm(a,5) # # Eigenvalue decomposition # help (expm2) print expm2(a) # # Taylor series # help (expm3) print expm3(a) # # Standard test problem p.184 L. Dieci, A.Papini # Linear Algebra and its applications Vol.308 (2000) 183-202 # omega = 1.0 x = 1.0e6 b = array([[omega,x],[0,omega]]) b = omega*identity(2)+([[0.,x],[0.,0.]]) # help (expm) print expm(b) # help (expm2) print expm2(b) # help (expm3) print expm3(b) help (funm) print funm(identity(2)*1.0 , exp) From nwagner at mecha.uni-stuttgart.de Tue Mar 26 04:31:47 2002 From: nwagner at mecha.uni-stuttgart.de (Nils Wagner) Date: Tue, 26 Mar 2002 10:31:47 +0100 Subject: [SciPy-user] ATLAS > 3.3.13 Message-ID: <3CA04003.BDAC5A15@mecha.uni-stuttgart.de> Hi, The latest stable version of ATLAS is 3.2.1. 3.3.5- 3.3.14 are Developer (unstable) versions. Does it make sense to install an unstable version ? Nils From pearu at scipy.org Tue Mar 26 03:30:36 2002 From: pearu at scipy.org (pearu at scipy.org) Date: Tue, 26 Mar 2002 02:30:36 -0600 (CST) Subject: [SciPy-user] ATLAS > 3.3.13 In-Reply-To: <3CA04003.BDAC5A15@mecha.uni-stuttgart.de> Message-ID: On Tue, 26 Mar 2002, Nils Wagner wrote: > Hi, > > The latest stable version of ATLAS is 3.2.1. > 3.3.5- 3.3.14 are Developer (unstable) versions. > > Does it make sense to install an unstable version ? In case of ATLAS, yes, I think so. The latest stable 3.2.1 is not quite stable as you need to apply some patches. And the latest unstable 3.3.>13 seems to be quite stable to me. There is a rumor going on that ATLAS team will release a new stable release soon. Pearu From oliphant at ee.byu.edu Tue Mar 26 02:00:31 2002 From: oliphant at ee.byu.edu (Travis Oliphant) Date: Tue, 26 Mar 2002 02:00:31 -0500 (EST) Subject: [SciPy-user] Re: [SciPy-dev] ATLAS > 3.3.13 In-Reply-To: <3CA04003.BDAC5A15@mecha.uni-stuttgart.de> Message-ID: > Hi, > > The latest stable version of ATLAS is 3.2.1. > 3.3.5- 3.3.14 are Developer (unstable) versions. > > Does it make sense to install an unstable version ? Well, SciPy in the CVS tree is also "unstable" It depends on how "cutting edge" you want to be. Pearu has noted that there appears to be some gearing up for a release in the ATLAS mailing lists which would indicate that the "unstable" release is about to become "stable." I'm using 3.3.13 and it appears to work fine. -Travis From oliphant at ee.byu.edu Tue Mar 26 02:41:51 2002 From: oliphant at ee.byu.edu (Travis Oliphant) Date: Tue, 26 Mar 2002 02:41:51 -0500 (EST) Subject: [SciPy-user] Test cases for expm Message-ID: The test cases Nils posted recently work fine for me. As expected expm2 does not work well (nor does funm) for the badly conditioned matrix obtained during that computation. expm and expm3 agree and look good. Thanks for the tests. If you are still having trouble, consider removing the build directory and starting again. The error Nils showed was fixed only a few hours ago, and so you should update your CVS copy. -Travis From nwagner at mecha.uni-stuttgart.de Tue Mar 26 06:20:06 2002 From: nwagner at mecha.uni-stuttgart.de (Nils Wagner) Date: Tue, 26 Mar 2002 12:20:06 +0100 Subject: [SciPy-user] Test cases for expm References: Message-ID: <3CA05966.FD2FC933@mecha.uni-stuttgart.de> Travis Oliphant schrieb: > > The test cases Nils posted recently work fine for me. > > As expected expm2 does not work well (nor does funm) for the badly > conditioned matrix obtained during that computation. > > expm and expm3 agree and look good. > > Thanks for the tests. > > If you are still having trouble, consider removing the build directory and > starting again. > > The error Nils showed was fixed only a few hours ago, and so you should > update your CVS copy. > > -Travis > > _______________________________________________ > SciPy-user mailing list > SciPy-user at scipy.net > http://www.scipy.net/mailman/listinfo/scipy-user Travis, I have updated both my CVS copy of scipy and ATLAS atlas3.2.1 --> atlas3.3.13 This is the result of python exptest.py Traceback (most recent call last): File "exptest.py", line 1, in ? from scipy import * File "/usr/local/lib/python2.1/site-packages/scipy/__init__.py", line 72, in ? names2all(__all__, _level0, globals()) File "/usr/local/lib/python2.1/site-packages/scipy/__init__.py", line 37, in names2all exec("import %s" % name, gldict) File "", line 1, in ? File "/usr/local/lib/python2.1/site-packages/scipy/basic.py", line 25, in ? cast = {Numeric.Character: toChar, AttributeError: 'Numeric' module has no attribute 'Character' Any idea ? Nils . From kern at caltech.edu Tue Mar 26 05:17:42 2002 From: kern at caltech.edu (Robert Kern) Date: Tue, 26 Mar 2002 02:17:42 -0800 Subject: [SciPy-dev] Re: [SciPy-user] Test cases for expm In-Reply-To: <3CA05966.FD2FC933@mecha.uni-stuttgart.de> References: <3CA05966.FD2FC933@mecha.uni-stuttgart.de> Message-ID: <20020326101742.GA27996@taliesen.caltech.edu> On Tue, Mar 26, 2002 at 12:20:06PM +0100, Nils Wagner wrote: [snip] > Travis, > > I have updated both my CVS copy of scipy and ATLAS atlas3.2.1 --> > atlas3.3.13 > > This is the result of python exptest.py > > Traceback (most recent call last): > File "exptest.py", line 1, in ? > from scipy import * > File "/usr/local/lib/python2.1/site-packages/scipy/__init__.py", line > 72, in ? > names2all(__all__, _level0, globals()) > File "/usr/local/lib/python2.1/site-packages/scipy/__init__.py", line > 37, in names2all > exec("import %s" % name, gldict) > File "", line 1, in ? > File "/usr/local/lib/python2.1/site-packages/scipy/basic.py", line 25, > in ? > cast = {Numeric.Character: toChar, > AttributeError: 'Numeric' module has no attribute 'Character' > > Any idea ? Just ran into that myself. Upgrade to Numeric 21.0 > Nils -- Robert Kern Ruddock House President kern at caltech.edu "In the fields of hell where the grass grows high Are the graves of dreams allowed to die." -- Richard Harter From nwagner at mecha.uni-stuttgart.de Tue Mar 26 06:35:25 2002 From: nwagner at mecha.uni-stuttgart.de (Nils Wagner) Date: Tue, 26 Mar 2002 12:35:25 +0100 Subject: [SciPy-dev] Re: [SciPy-user] Test cases for expm References: <3CA05966.FD2FC933@mecha.uni-stuttgart.de> <20020326101742.GA27996@taliesen.caltech.edu> Message-ID: <3CA05CFD.114694B@mecha.uni-stuttgart.de> Robert Kern schrieb: > > On Tue, Mar 26, 2002 at 12:20:06PM +0100, Nils Wagner wrote: > > [snip] > > > Travis, > > > > I have updated both my CVS copy of scipy and ATLAS atlas3.2.1 --> > > atlas3.3.13 > > > > This is the result of python exptest.py > > > > Traceback (most recent call last): > > File "exptest.py", line 1, in ? > > from scipy import * > > File "/usr/local/lib/python2.1/site-packages/scipy/__init__.py", line > > 72, in ? > > names2all(__all__, _level0, globals()) > > File "/usr/local/lib/python2.1/site-packages/scipy/__init__.py", line > > 37, in names2all > > exec("import %s" % name, gldict) > > File "", line 1, in ? > > File "/usr/local/lib/python2.1/site-packages/scipy/basic.py", line 25, > > in ? > > cast = {Numeric.Character: toChar, > > AttributeError: 'Numeric' module has no attribute 'Character' > > > > Any idea ? > > Just ran into that myself. Upgrade to Numeric 21.0 > I have upgraded to Numeric 21.0 - it is already done. Any further suggestion ? Nils > > Nils > > -- > Robert Kern > Ruddock House President > kern at caltech.edu > > "In the fields of hell where the grass grows high > Are the graves of dreams allowed to die." > -- Richard Harter > _______________________________________________ > SciPy-user mailing list > SciPy-user at scipy.net > http://www.scipy.net/mailman/listinfo/scipy-user From kern at caltech.edu Tue Mar 26 05:36:04 2002 From: kern at caltech.edu (Robert Kern) Date: Tue, 26 Mar 2002 02:36:04 -0800 Subject: [SciPy-dev] Re: [SciPy-user] Test cases for expm In-Reply-To: <3CA05CFD.114694B@mecha.uni-stuttgart.de> References: <3CA05966.FD2FC933@mecha.uni-stuttgart.de> <20020326101742.GA27996@taliesen.caltech.edu> <3CA05CFD.114694B@mecha.uni-stuttgart.de> Message-ID: <20020326103604.GA25713@taliesen.caltech.edu> On Tue, Mar 26, 2002 at 12:35:25PM +0100, Nils Wagner wrote: > Robert Kern schrieb: [snip] > > Just ran into that myself. Upgrade to Numeric 21.0 > > > I have upgraded to Numeric 21.0 - it is already done. > > Any further suggestion ? I get Python 2.1.2 (#1, Mar 16 2002, 00:56:55) [GCC 2.95.4 20011002 (Debian prerelease)] on linux2 Type "copyright", "credits" or "license" for more information. >>> import Numeric >>> Numeric.Character 'c' >>> Numeric.__version__ '21.0' >>> Character is defined in Numeric's Precision.py . I'd check to be sure you're picking up the right copy of Numeric (and Precision) in case you have old copies around. > Nils -- Robert Kern Ruddock House President kern at caltech.edu "In the fields of hell where the grass grows high Are the graves of dreams allowed to die." -- Richard Harter From nwagner at mecha.uni-stuttgart.de Tue Mar 26 06:48:43 2002 From: nwagner at mecha.uni-stuttgart.de (Nils Wagner) Date: Tue, 26 Mar 2002 12:48:43 +0100 Subject: [SciPy-dev] Re: [SciPy-user] Test cases for expm References: <3CA05966.FD2FC933@mecha.uni-stuttgart.de> <20020326101742.GA27996@taliesen.caltech.edu> <3CA05CFD.114694B@mecha.uni-stuttgart.de> <20020326103604.GA25713@taliesen.caltech.edu> Message-ID: <3CA0601B.FF01874F@mecha.uni-stuttgart.de> Robert Kern schrieb: > > On Tue, Mar 26, 2002 at 12:35:25PM +0100, Nils Wagner wrote: > > Robert Kern schrieb: > > [snip] > > > > Just ran into that myself. Upgrade to Numeric 21.0 > > > > > I have upgraded to Numeric 21.0 - it is already done. > > > > Any further suggestion ? > > I get > > Python 2.1.2 (#1, Mar 16 2002, 00:56:55) > [GCC 2.95.4 20011002 (Debian prerelease)] on linux2 > Type "copyright", "credits" or "license" for more information. > >>> import Numeric > >>> Numeric.Character > 'c' > >>> Numeric.__version__ > '21.0' > >>> > > Character is defined in Numeric's Precision.py . I'd check to be sure > you're picking up the right copy of Numeric (and Precision) in case you > have old copies around. > > > Nils > > -- > Robert Kern > Ruddock House President > kern at caltech.edu > > "In the fields of hell where the grass grows high > Are the graves of dreams allowed to die." > -- Richard Harter > _______________________________________________ > SciPy-user mailing list > SciPy-user at scipy.net > http://www.scipy.net/mailman/listinfo/scipy-user Now I have used python setup.py install --force I get Python 2.1.2 (#1, Feb 25 2002, 18:04:21) [GCC 2.95.3 20010315 (SuSE)] on linux2 Type "copyright", "credits" or "license" for more information. >>> import Numeric >>> Numeric.Character 'c' >>> Numeric.__version__ '21.0' >>> >>> However there is still a problem with scipy Traceback (most recent call last): File "exptest.py", line 1, in ? from scipy import * File "/usr/local/lib/python2.1/site-packages/scipy/__init__.py", line 78, in ? modules2all(__all__, _level1, globals()) File "/usr/local/lib/python2.1/site-packages/scipy/__init__.py", line 48, in modules2all exec("import %s" % name, gldict) File "", line 1, in ? File "/usr/local/lib/python2.1/site-packages/scipy/linalg/__init__.py", line 41, in ? scipy.modules2all(__all__, _modules, globals()) File "/usr/local/lib/python2.1/site-packages/scipy/__init__.py", line 48, in modules2all exec("import %s" % name, gldict) File "", line 1, in ? ImportError: /usr/local/lib/python2.1/site-packages/scipy/linalg/flapack.so: undefined symbol: slaswp_ Nils From pearu at scipy.org Tue Mar 26 06:22:23 2002 From: pearu at scipy.org (pearu at scipy.org) Date: Tue, 26 Mar 2002 05:22:23 -0600 (CST) Subject: [SciPy-dev] Re: [SciPy-user] Test cases for expm In-Reply-To: <3CA0601B.FF01874F@mecha.uni-stuttgart.de> Message-ID: On Tue, 26 Mar 2002, Nils Wagner wrote: > Now I have used python setup.py install --force I am not sure that --force is enough for a clean start. rm -rf build works for sure. > However there is still a problem with scipy > > Traceback (most recent call last): > File "exptest.py", line 1, in ? > from scipy import * > File "/usr/local/lib/python2.1/site-packages/scipy/__init__.py", line > 78, in ? > modules2all(__all__, _level1, globals()) > File "/usr/local/lib/python2.1/site-packages/scipy/__init__.py", line > 48, in modules2all > exec("import %s" % name, gldict) > File "", line 1, in ? > File > "/usr/local/lib/python2.1/site-packages/scipy/linalg/__init__.py", line > 41, in ? > scipy.modules2all(__all__, _modules, globals()) > File "/usr/local/lib/python2.1/site-packages/scipy/__init__.py", line > 48, in modules2all > exec("import %s" % name, gldict) > File "", line 1, in ? > ImportError: > /usr/local/lib/python2.1/site-packages/scipy/linalg/flapack.so: > undefined symbol: slaswp_ Standard first question (and the standard answer is usually no answer;): What is the output of python scipy_distutils/system_info.py ? Pearu From nwagner at mecha.uni-stuttgart.de Tue Mar 26 07:45:58 2002 From: nwagner at mecha.uni-stuttgart.de (Nils Wagner) Date: Tue, 26 Mar 2002 13:45:58 +0100 Subject: [SciPy-dev] Re: [SciPy-user] Test cases for expm References: Message-ID: <3CA06D86.28E5CF9A@mecha.uni-stuttgart.de> pearu at scipy.org schrieb: > > On Tue, 26 Mar 2002, Nils Wagner wrote: > > > Now I have used python setup.py install --force > > I am not sure that --force is enough for a clean start. > rm -rf build > works for sure. > > > However there is still a problem with scipy > > > > Traceback (most recent call last): > > File "exptest.py", line 1, in ? > > from scipy import * > > File "/usr/local/lib/python2.1/site-packages/scipy/__init__.py", line > > 78, in ? > > modules2all(__all__, _level1, globals()) > > File "/usr/local/lib/python2.1/site-packages/scipy/__init__.py", line > > 48, in modules2all > > exec("import %s" % name, gldict) > > File "", line 1, in ? > > File > > "/usr/local/lib/python2.1/site-packages/scipy/linalg/__init__.py", line > > 41, in ? > > scipy.modules2all(__all__, _modules, globals()) > > File "/usr/local/lib/python2.1/site-packages/scipy/__init__.py", line > > 48, in modules2all > > exec("import %s" % name, gldict) > > File "", line 1, in ? > > ImportError: > > /usr/local/lib/python2.1/site-packages/scipy/linalg/flapack.so: > > undefined symbol: slaswp_ > > Standard first question (and the standard answer is usually no answer;): > What is the output of > python scipy_distutils/system_info.py > ? > The ouput is atlas_info: Looking in /usr ... [] FOUND: libraries = ['lapack', 'f77blas', 'cblas', 'atlas'] library_dirs = ['/usr/lib'] blas_info: NOT AVAILABLE fftw_info: Looking in /usr ... FOUND: include_dirs = ['/usr/include'] define_macros = [('SCIPY_DFFTW_H', 1), ('SCIPY_SFFTW_H', 1)] library_dirs = ['/usr/lib', '/usr/lib'] libraries = ['dfftw', 'drfftw', 'dfftw_threads', 'drfftw_threads', 'sfftw', 'srfftw', 'sfftw_threads', 'srfftw_threads'] lapack_info: NOT AVAILABLE x11_info: Looking in /usr ... FOUND: include_dirs = ['/usr/X11/include/X11'] library_dirs = ['/usr/X11/lib'] libraries = ['X11'] Nils > Pearu > > _______________________________________________ > SciPy-user mailing list > SciPy-user at scipy.net > http://www.scipy.net/mailman/listinfo/scipy-user From pearu at scipy.org Tue Mar 26 06:38:33 2002 From: pearu at scipy.org (pearu at scipy.org) Date: Tue, 26 Mar 2002 05:38:33 -0600 (CST) Subject: [SciPy-dev] Re: [SciPy-user] Test cases for expm In-Reply-To: <3CA06D86.28E5CF9A@mecha.uni-stuttgart.de> Message-ID: On Tue, 26 Mar 2002, Nils Wagner wrote: > atlas_info: > Looking in /usr ... > [] > FOUND: > libraries = ['lapack', 'f77blas', 'cblas', 'atlas'] > library_dirs = ['/usr/lib'] ^^^^^^^^^^ To be sure, is this the place where you installed the latest ATLAS? And did you tried `rm -rf' build for a fresh start? Pearu From nwagner at mecha.uni-stuttgart.de Tue Mar 26 07:56:57 2002 From: nwagner at mecha.uni-stuttgart.de (Nils Wagner) Date: Tue, 26 Mar 2002 13:56:57 +0100 Subject: [SciPy-dev] Re: [SciPy-user] Test cases for expm References: Message-ID: <3CA07019.AC38E12F@mecha.uni-stuttgart.de> pearu at scipy.org schrieb: > > On Tue, 26 Mar 2002, Nils Wagner wrote: > > > atlas_info: > > Looking in /usr ... > > [] > > FOUND: > > libraries = ['lapack', 'f77blas', 'cblas', 'atlas'] > > library_dirs = ['/usr/lib'] > ^^^^^^^^^^ > To be sure, is this the place where you installed the latest ATLAS? > And did you tried `rm -rf' build for a fresh start? > I tried rm -rf build/ The latest ATLAS is installed in /usr/local/lib/atlas -rw-r--r-- 1 root root 5301572 M?r 26 12:06 libatlas.a -rw-r--r-- 1 root root 5027632 Feb 27 14:26 libatlas.a.old -rw-r--r-- 1 root root 280064 M?r 26 12:06 libcblas.a -rw-r--r-- 1 root root 272104 Feb 27 14:26 libcblas.a.old -rw-r--r-- 1 root root 352706 M?r 26 12:06 libf77blas.a -rw-r--r-- 1 root root 344582 Feb 27 14:26 libf77blas.a.old -rw-r--r-- 1 root root 310528 M?r 26 12:06 liblapack.a -rw-r--r-- 1 root root 5920822 Feb 27 14:26 liblapack.a.old -rw-r--r-- 1 root root 326216 M?r 26 12:06 libtstatlas.a -rw-r--r-- 1 root root 300060 Feb 27 14:26 libtstatlas.a.old Therefore, I have used the following commands for building and installing scipy ATLAS=/usr/local/lib/atlas /usr/local/bin/python setup.py build and ATLAS=/usr/local/lib/atlas /usr/local/bin/python setup.py install Nils > Pearu > > _______________________________________________ > SciPy-user mailing list > SciPy-user at scipy.net > http://www.scipy.net/mailman/listinfo/scipy-user From nwagner at mecha.uni-stuttgart.de Tue Mar 26 08:16:12 2002 From: nwagner at mecha.uni-stuttgart.de (Nils Wagner) Date: Tue, 26 Mar 2002 14:16:12 +0100 Subject: [SciPy-user] ATLAS Message-ID: <3CA0749C.396F61FA@mecha.uni-stuttgart.de> Pearu, There is a remarkable change in the size of liblapack.a -rw-r--r-- 1 root root 310528 M?r 26 12:06 liblapack.a -rw-r--r-- 1 root root 5920822 Feb 27 14:26 liblapack.a.old liblapack.a.old (ATLAS3.2.1) liblapack.a (ATLAS3.3.13) Nils From kern at caltech.edu Tue Mar 26 07:14:37 2002 From: kern at caltech.edu (Robert Kern) Date: Tue, 26 Mar 2002 04:14:37 -0800 Subject: [SciPy-user] ATLAS In-Reply-To: <3CA0749C.396F61FA@mecha.uni-stuttgart.de> References: <3CA0749C.396F61FA@mecha.uni-stuttgart.de> Message-ID: <20020326121437.GA2020@taliesen.caltech.edu> On Tue, Mar 26, 2002 at 02:16:12PM +0100, Nils Wagner wrote: > Pearu, > > There is a remarkable change in the size of liblapack.a > > -rw-r--r-- 1 root root 310528 M?r 26 12:06 liblapack.a > -rw-r--r-- 1 root root 5920822 Feb 27 14:26 liblapack.a.old > > liblapack.a.old (ATLAS3.2.1) > liblapack.a (ATLAS3.3.13) Aha. There's the problem. ATLAS doesn't provide implementations for every LAPACK function. To get a full LAPACK library, you have to follow the instructions in ATLAS/doc/LibReadme.txt . > Nils -- Robert Kern Ruddock House President kern at caltech.edu "In the fields of hell where the grass grows high Are the graves of dreams allowed to die." -- Richard Harter From nwagner at mecha.uni-stuttgart.de Tue Mar 26 08:45:08 2002 From: nwagner at mecha.uni-stuttgart.de (Nils Wagner) Date: Tue, 26 Mar 2002 14:45:08 +0100 Subject: [SciPy-user] ATLAS <--> Building a complete LAPACK library References: <3CA0749C.396F61FA@mecha.uni-stuttgart.de> <20020326121437.GA2020@taliesen.caltech.edu> Message-ID: <3CA07B64.88316802@mecha.uni-stuttgart.de> Robert Kern schrieb: > > On Tue, Mar 26, 2002 at 02:16:12PM +0100, Nils Wagner wrote: > > Pearu, > > > > There is a remarkable change in the size of liblapack.a > > > > -rw-r--r-- 1 root root 310528 M?r 26 12:06 liblapack.a > > -rw-r--r-- 1 root root 5920822 Feb 27 14:26 liblapack.a.old > > > > liblapack.a.old (ATLAS3.2.1) > > liblapack.a (ATLAS3.3.13) > > Aha. There's the problem. ATLAS doesn't provide implementations for > every LAPACK function. To get a full LAPACK library, you have to follow > the instructions in ATLAS/doc/LibReadme.txt . > > > Nils > > -- > Robert Kern > Ruddock House President > kern at caltech.edu > > "In the fields of hell where the grass grows high > Are the graves of dreams allowed to die." > -- Richard Harter > _______________________________________________ > SciPy-user mailing list > SciPy-user at scipy.net > http://www.scipy.net/mailman/listinfo/scipy-user Robert, Thank you very much. Finally I have solved my problems. Nils Building a complete LAPACK library ATLAS does not provide a full LAPACK library. However, there is a simple way to get ATLAS to provide its faster LAPACK routines to a full LAPACK library. ATLAS's internal routines are distinct from LAPACK's, so it is safe to compile ATLAS's LAPACK routines directly into a netlib-style LAPACK library. First, download and install the standard LAPACK library from the LAPACK homepage. Then, in your ATLAS/lib/ARCH directory (where you should have a liblapack.a), issue the following commands: mkdir tmp cd tmp ar x ../liblapack.a cp ../liblapack.a ar r ../liblapack.a *.o cd .. rm -rf tmp Just linking in ATLAS's liblapack.a first will not get you the best LAPACK performance, mainly because LAPACK's untuned ILAENV will be used instead of ATLAS's tuned one. So, if you use any LAPACK routine that is not provided by ATLAS, it is essential that you create this hybrid LAPACK/ATLAS library in order to get the best performance. From eric at scipy.org Tue Mar 26 10:08:55 2002 From: eric at scipy.org (eric) Date: Tue, 26 Mar 2002 10:08:55 -0500 Subject: [SciPy-user] ATLAS > 3.3.13 References: <3CA04003.BDAC5A15@mecha.uni-stuttgart.de> Message-ID: <031601c1d4d8$263beb10$6b01a8c0@ericlaptop> > The latest stable version of ATLAS is 3.2.1. > 3.3.5- 3.3.14 are Developer (unstable) versions. > > Does it make sense to install an unstable version ? Here is an email that Clint Whaley (main developer of ATLAS) posted on 2/22/02. It looks like there is one more developer release before ATLAS goes stable. The main issues look to be with PPC, Alpha, and dual OS X (Mac) boxes. If your not using one of these platforms, I'd say your reasonably safe. The developer mailing list for ATLAS is here. https://sourceforge.net/mailarchive/forum.php?forum_id=426 That is probably the best place to follow the state of the package. see ya, eric ---------- Clint's email --------- Guys, We will have at least one more developer release, in order to fix the errors Camm has spotted. 3.3.14 has bad arch defaults for PPCG4, and it looks like the GOTO gemm is seg faulting on ev5x (21164). These changes are enough that I'll want a dev release fixing them before going stable. In the meantime, my own tests are showing something rather strange. Does anyone else have access to a dual OS X box with a f77 compiler installed? On the one I have access to, the threaded code is unrepeatably hanging in x?l3blastst_pt, and having this behavior confirmed on another box would be very helpful . . . On the ev5 error, I have restarted the tester without using GOTO's gemm, and it has finished the testers that seg fault, but is still running. If we can't find a fix, and the non-goto ev5 passes all tests, I'll just stop using GOTO's gemm on that platform. If you set some environment variables that GOTO recommends, the seg faults also go away, but we can't have a package that seg faults if the user forgets to set his environment up in the way we like . . . Cheers, Clint